Commit Graph

320 Commits

Author SHA1 Message Date
Haoran Pan 95d81fa71b fix bug in model evaluator final feedback ignoring shape feedback (#501) 2024-12-03 13:55:28 +08:00
Linlang d4e7f9277e chore: fix PR template (#487)
* fix PR template

* subfolders of the log displayed on the web page

* fix code error

* add health check

* reformat with isort

* reformat with black

* update README

* fix bug && experience confusing

* reformat with black & update health check code

* reformat with isort

* reformat with black

* update README.md

* Removed duplicates in documentation and health_check && Upgraded code for filtering folders

* update docs

* reformat with black
2024-11-28 15:35:13 +08:00
you-n-g 1acb1dab8f feat: run benchmark on gpt-4o & llama 3.1 (#497)
* Run benchmark on gpt-4o & llama 3.1

* update link
2024-11-26 12:02:09 +08:00
Xu Yang ac18f6f892 very small fix (#496) 2024-11-25 20:08:05 +08:00
Xu Yang fce241b9f9 feat: a unified CoSTEER to fit more scenarios (#491)
* Use ExtendedBaseSettings to replace BaseSettings

* update a more general way to pass the default setting

* update all code

* fix CI

* fix CI

* fix qlib scenario

* fix CI

* fix CI

* fix CI & add data science interfaces

* remove redundant code

* abandon costeer knowledge base v1

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
2024-11-25 16:27:34 +08:00
you-n-g a4602bdcc2 fix: patching weird azure deployment (#494)
* patching weird azure deployment

* Auto link
2024-11-25 13:47:21 +08:00
you-n-g 7ffe284c12 feat: kaggle refactor (#489)
* init trail

* Add spec info

* auto unzip mlebench prepared data for out scenario

* successfully run example

* successfully run main

* simplify load traing

* extract load_from_raw_data

* split the fies(still buggy)
It should stop on ~20 epoch and reach the end

* some changes

* Fix bug to run example

* (success) until feature

* refine model and ensemble

* add metrics in ens.py

* update README &  spec.md

* ens change

* fix ens bug

* Delete rdagent/scenarios/kaggle/tpl_ex/aerial-cactus-identification/train.py

* add template_path in KG_conf

* fix test kaggle

* CI

* make test_import not check kaggle template codes

---------

Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2024-11-20 17:01:18 +08:00
you-n-g 5d5084b87e feat: new-york-city-taxi-fare-prediction_template (#488)
* copy init version

* feat: new-york-city-taxi-fare-prediction_template

* add move to linear model

* Add more details about docker

* auto lint

* auto lint with new black
2024-11-15 16:47:44 +08:00
Xu Yang 6809154ed7 feat: align mlebench data and evaluation & several fix on kaggle workflow (#477)
* several improvement on kaggle loop

* small refinement on prompt

* fix bugs

* add the score of each model in every experiment

* fix ci error

* fix error in ventilator tpl

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
Co-authored-by: WinstonLiye <1957922024@qq.com>
Co-authored-by: TPLin22 <tplin2@163.com>
2024-11-15 15:40:22 +08:00
XianBW 21ee8b635b feat: template changes for some kaggle competitions (#484)
* change tpl metric codes

* change tpl
2024-11-12 11:24:01 +08:00
XianBW 1178881026 add kaggle tpl (#482) 2024-11-11 22:31:32 +08:00
XianBW 4fb6376a19 feat: (Kaggle) add base template for competition: tabular-playground-series-may-2022 (#481)
* add tpl kaggle

* CI
2024-11-11 21:18:07 +08:00
Xu Yang c095e4992f feat(kaggle): several update in kaggle scenarios (#476)
* udpate plot

* log and reduce token

* trace tag

* add simple_background parameter to get_scenario_all_desc

* update trace

* update first version code

* chat model map

* add annotation for stack index

* add annotation

* reformatted by black

* several update on kaggle scenarios

* update some new change

* fix CI

* fix CI

* fix a bug

* fix bugs in graph RAG

---------

Co-authored-by: Tim <illking@foxmail.com>
2024-11-06 13:14:35 +08:00
Yuante Li 5986f8f4db feat: add a new competition (#474)
* add tabular-playground-series-dec-2021

* finished

* fix a mistake

* fix a bug

* fix a bug
2024-11-05 20:42:20 +08:00
Yuante Li cf93b769ff feat: end-to-end optimization (#473)
* fix

* refine the code

* aligned data with MLE Kaggle competition dataset

* fix ci error

* fix ci
2024-11-05 15:41:50 +08:00
Xu Yang 9bea9a6d1f fix a bug (#470) 2024-11-04 15:05:05 +08:00
Linlang 2e2e75a043 fix: unzip kaggle data (#464)
* unzip kaggle data

* read local_data_path from .env file

* fix build docs error

* recover azure-identity packages

* optimize code logic

* add error when downloading data from kaggle

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-11-04 14:57:10 +08:00
WinstonLiyt 437fbcb577 fix a bug in kaggle runner cache and kaggle costeer execute template (#467) 2024-11-04 11:16:16 +08:00
WinstonLiyt 44271ecfd4 feat: refine logic for qlib_factor_from_report (#463)
* fix a bug in kaggle conf

* fix a ci error

* refine logic for qlib_factor_from_report
2024-10-31 16:04:32 +08:00
XianBW 30a5061012 docs: some changes in kaggle doc (#461)
* add kaggle scenario description for webapp

* catch download button error

* change kaggle auto_submit to False default

* Added guidance for custom templates in kaggle doc

* CI

* CI

* add kaggle data set instruction(NOTE)

* delete repeated description for kaggle scenario

---------

Co-authored-by: TPLin22 <tplin2@163.com>
2024-10-30 17:00:56 +08:00
WinstonLiyt 1d50ffd9ac fix: fix a bug in kaggle conf (#459)
* fix a bug in kaggle conf

* fix a ci error
2024-10-29 15:13:18 +08:00
Haoran Pan 1fe8bdfd6a fix: templates bug (#456)
* Update model_nn.py

* ci issue
2024-10-28 14:22:04 +08:00
Linlang 7672eaeaaa chore: remove package (#449)
* remove package

* reformat with black

* fix pytest error

* fix pytest error

* fix pytest error

* add packages

* change code
2024-10-24 20:38:13 +08:00
Xu Yang c71fd1fba2 feat: independent use_azure_token_provider on chat and embedding (#452)
* independent use_azure_token_provider

* fix CI

* fix doc format
2024-10-23 18:22:58 +08:00
Tim eace48a054 always return str (#453) 2024-10-23 16:47:36 +08:00
XianBW 4f36115d37 fix KGFactorRunner cache system bug (#451) 2024-10-23 15:17:38 +08:00
WinstonLiyt 9d7e1b8b49 docs: add documentation for kaggle scen (#448)
* init for bg & quickstart for kaggle docs

* Add documentation for the environment configuration in the Kaggle scenario.

* add some descriptions in documents

* remove useless docs

* ci issue

---------

Co-authored-by: TPLin22 <tplin2@163.com>
2024-10-23 13:26:57 +08:00
Linlang 33684cafe4 chore: remove torch in main code (#439)
* use np.ndarray in costeer model evaluators

* remove package

* fix test error

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-10-21 15:56:04 +08:00
XianBW 671d4f0600 chore: move use_auto_chat_cache_seed_gen and init_chat_cache_seed to LLMSett… (#444)
* move use_auto_chat_cache_seed_gen and init_chat_cache_seed to LLMSettings

* remove RD_AGENT_SETTINGS in llm_utils.py
2024-10-21 12:03:59 +08:00
XianBW 64df9bec37 feat: use auto gen seed when using LLM cache (#441)
* initial version

* test requirements

* fix bugs

* fix bugs

* add annotation

* fix ruff error

* fix CI

* fix CI

* fix CI

* fix CI

* change random usage

* move cache_seed_gen to core/utils.py

* fix CI

* change cache_seed_gen name

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-10-21 11:32:20 +08:00
XianBW 5ff09b6a2d fix display bug in webapp (#440) 2024-10-18 17:07:25 +08:00
XianBW 75d35bf447 fix problem of reading log file tags on Windows (#437)
Co-authored-by: Bowen Xian (Shanghai Wicresoft Co Ltd) <v-bxian@microsoft.com>
2024-10-17 20:22:39 +08:00
Xu Yang 00c76e0411 add inf evaluator to factor costeer and some minor improvement (#435) 2024-10-16 16:00:55 +08:00
Tim d615c0e31b number of subprocesses <= number of tasks (#434) 2024-10-16 14:13:20 +08:00
Way2Learn 69fc1fd911 feat: supporting various Kaggle competitions & scenarios for RD-Agent (#409)
* Fixes on  kaggle output

* feat: add kaggle s3e14 template (#394)

* add s3e14 template

* fix CI

* Initialisation of a template of competition

* add kaggle s3e16 template (#396)

* get kaggle competition scores (#397)

* Adding a new competition s4e6

* feat: s4e5 (#400)

* init for s4e5

* edit s4e5

* ci issue

* feat: S4e3 (#402)

* Initialisation of a template of competition

* Adding a new competition s4e6

* Competition Initialised

* Fixed to make sure that now it runs

* Fixing for CI

* correct evaluation (#403)

* find rank in leaderboard (#405)

* fix: model templates for KG scenario (#408)

* fix feature selection for some models

* feat select template

* Updating the prompts for a more powerful model tuning

* refine the prompt

* fix: template error in s4e6

* feat: show simple execution time in demo (#410)

* show time in kaggle demo

* change color

* fix a small bug

* edit loop.py and proposal

* delete useless files

* CI issues

* ci issue

---------

Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: Haoran Pan <167847254+TPLin22@users.noreply.github.com>
Co-authored-by: Way2Learn <118058822+Xisen-Wang@users.noreply.github.com>
Co-authored-by: WinstonLiyt <1957922024@qq.com>
Co-authored-by: TPLin22 <tplin2@163.com>
2024-10-15 13:06:36 +08:00
Tim b9a36e1b25 chore: more optional parameters for running benchmark analysis (#431)
* set title and round

* decision from multiple types

* check if decision is true

* reformat

* remove unused file
2024-10-15 12:41:32 +08:00
Xu Yang 5ad3619492 force use_chat_cache False when facing error in evaluators (#432) 2024-10-15 12:21:33 +08:00
Xu Yang 41eb482621 fix: fix cache result logic (#430)
* make cache logic more tidy

* update WeChat QR code
2024-10-14 19:35:21 +08:00
Xu Yang 3a44e715ad fix: fix a small bug in cache using module name and function name as unique folder name (#429) 2024-10-14 17:53:04 +08:00
Tim f3b3e4acb7 convert str to bool (#428) 2024-10-14 17:44:33 +08:00
Xu Yang 768229427d feat: use unified pickle cacher & move llm config into a isolated config (#424)
* simplify RDAgent conf

* add unified cacher(untested)

* fix small bugs

* fix a bug

* fix a small bug in runner

* use hash_key = None to skip cache

* fix CI

* in factor execution, ignore cache when raise exception

* add file locker to avoid mp calling

* fix CI

* use function __module__ name as folder in cache
2024-10-14 17:34:09 +08:00
XianBW e7656883ba auto select content element in kaggle crawler now (#423) 2024-10-11 22:34:12 +08:00
Xu Yang a6ddf08e2a perf: some small upgrade to factor costeer to improve the performance (#420)
* 1. use dataframe.info instead of head
2. in former trace query, add the latest attempt to the last success execution

* fix CI
2024-10-10 20:29:17 +08:00
Linlang 5402c00165 fix: fix json load error (#386)
* fix json load error

* fix os.symlink error

* reformat with black

* fix os.symlink error

* reformat with isort
2024-10-10 11:58:15 +08:00
Xu Yang 9f6b457aa5 improve factor costeer: (#417)
1. when facing odd v2_query_component_limit, make from gt bigger than without gt
2. do code evaluator even value does not pass
3. provide the value header to value failed evaluators
2024-10-09 18:56:37 +08:00
Xu Yang 19985c675e use main thread to get embedding to speed up cache (#415) 2024-10-08 19:34:19 +08:00
you-n-g d9c30f1488 refactor: refine util (#413)
* refine util

* lint
2024-10-08 02:07:20 +08:00
WinstonLiyt 80b392ce2f fix: fix a bug in competition metric evaluation (#407)
* fix a bug in competition metric evaluation

* fix a bug

* fix a bug in rag loading
2024-09-30 14:54:56 +08:00
WinstonLiyt 1cbaa4b15c fix: refine the ucb algorithm (#406)
* refine the ucb algorithm

* fix a ci error
2024-09-30 12:30:42 +08:00
WinstonLiyt 54109f62e4 feat: Add ranking in kaggle scenario (#401)
* fix some bugs in rag

* add ranking in kaggle scenario

* fix two mistouches

* fix two bugs

* fix a bug
2024-09-30 04:15:07 +08:00