Commit Graph

319 Commits

Author SHA1 Message Date
Linlang ea85d5efc8 chore: fix PR template (#487)
* fix PR template

* subfolders of the log displayed on the web page

* fix code error

* add health check

* reformat with isort

* reformat with black

* update README

* fix bug && experience confusing

* reformat with black & update health check code

* reformat with isort

* reformat with black

* update README.md

* Removed duplicates in documentation and health_check && Upgraded code for filtering folders

* update docs

* reformat with black
2024-11-28 15:35:13 +08:00
you-n-g c3f78f8bce feat: run benchmark on gpt-4o & llama 3.1 (#497)
* Run benchmark on gpt-4o & llama 3.1

* update link
2024-11-26 12:02:09 +08:00
Xu Yang d54e14f1d9 very small fix (#496) 2024-11-25 20:08:05 +08:00
Xu Yang b7fcb13c33 feat: a unified CoSTEER to fit more scenarios (#491)
* Use ExtendedBaseSettings to replace BaseSettings

* update a more general way to pass the default setting

* update all code

* fix CI

* fix CI

* fix qlib scenario

* fix CI

* fix CI

* fix CI & add data science interfaces

* remove redundant code

* abandon costeer knowledge base v1

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
2024-11-25 16:27:34 +08:00
you-n-g e1032b6a31 fix: patching weird azure deployment (#494)
* patching weird azure deployment

* Auto link
2024-11-25 13:47:21 +08:00
you-n-g f9bd19179f feat: kaggle refactor (#489)
* init trail

* Add spec info

* auto unzip mlebench prepared data for out scenario

* successfully run example

* successfully run main

* simplify load traing

* extract load_from_raw_data

* split the fies(still buggy)
It should stop on ~20 epoch and reach the end

* some changes

* Fix bug to run example

* (success) until feature

* refine model and ensemble

* add metrics in ens.py

* update README &  spec.md

* ens change

* fix ens bug

* Delete rdagent/scenarios/kaggle/tpl_ex/aerial-cactus-identification/train.py

* add template_path in KG_conf

* fix test kaggle

* CI

* make test_import not check kaggle template codes

---------

Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2024-11-20 17:01:18 +08:00
you-n-g ae8fc773cb feat: new-york-city-taxi-fare-prediction_template (#488)
* copy init version

* feat: new-york-city-taxi-fare-prediction_template

* add move to linear model

* Add more details about docker

* auto lint

* auto lint with new black
2024-11-15 16:47:44 +08:00
Xu Yang 8662211492 feat: align mlebench data and evaluation & several fix on kaggle workflow (#477)
* several improvement on kaggle loop

* small refinement on prompt

* fix bugs

* add the score of each model in every experiment

* fix ci error

* fix error in ventilator tpl

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
Co-authored-by: WinstonLiye <1957922024@qq.com>
Co-authored-by: TPLin22 <tplin2@163.com>
2024-11-15 15:40:22 +08:00
XianBW 892283dc89 feat: template changes for some kaggle competitions (#484)
* change tpl metric codes

* change tpl
2024-11-12 11:24:01 +08:00
XianBW fe7eb4cbe1 add kaggle tpl (#482) 2024-11-11 22:31:32 +08:00
XianBW 8c4c53339c feat: (Kaggle) add base template for competition: tabular-playground-series-may-2022 (#481)
* add tpl kaggle

* CI
2024-11-11 21:18:07 +08:00
Xu Yang 90d9cdd0e9 feat(kaggle): several update in kaggle scenarios (#476)
* udpate plot

* log and reduce token

* trace tag

* add simple_background parameter to get_scenario_all_desc

* update trace

* update first version code

* chat model map

* add annotation for stack index

* add annotation

* reformatted by black

* several update on kaggle scenarios

* update some new change

* fix CI

* fix CI

* fix a bug

* fix bugs in graph RAG

---------

Co-authored-by: Tim <illking@foxmail.com>
2024-11-06 13:14:35 +08:00
Yuante Li bf719e0993 feat: add a new competition (#474)
* add tabular-playground-series-dec-2021

* finished

* fix a mistake

* fix a bug

* fix a bug
2024-11-05 20:42:20 +08:00
Yuante Li 1726caa414 feat: end-to-end optimization (#473)
* fix

* refine the code

* aligned data with MLE Kaggle competition dataset

* fix ci error

* fix ci
2024-11-05 15:41:50 +08:00
Xu Yang 25af7e6cce fix a bug (#470) 2024-11-04 15:05:05 +08:00
Linlang cbbbb4e9a5 fix: unzip kaggle data (#464)
* unzip kaggle data

* read local_data_path from .env file

* fix build docs error

* recover azure-identity packages

* optimize code logic

* add error when downloading data from kaggle

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-11-04 14:57:10 +08:00
WinstonLiyt 450317d8fe fix a bug in kaggle runner cache and kaggle costeer execute template (#467) 2024-11-04 11:16:16 +08:00
WinstonLiyt 09325aed89 feat: refine logic for qlib_factor_from_report (#463)
* fix a bug in kaggle conf

* fix a ci error

* refine logic for qlib_factor_from_report
2024-10-31 16:04:32 +08:00
XianBW 2c206e9ce5 docs: some changes in kaggle doc (#461)
* add kaggle scenario description for webapp

* catch download button error

* change kaggle auto_submit to False default

* Added guidance for custom templates in kaggle doc

* CI

* CI

* add kaggle data set instruction(NOTE)

* delete repeated description for kaggle scenario

---------

Co-authored-by: TPLin22 <tplin2@163.com>
2024-10-30 17:00:56 +08:00
WinstonLiyt 80e4a2aa79 fix: fix a bug in kaggle conf (#459)
* fix a bug in kaggle conf

* fix a ci error
2024-10-29 15:13:18 +08:00
Haoran Pan 62beb3a7ae fix: templates bug (#456)
* Update model_nn.py

* ci issue
2024-10-28 14:22:04 +08:00
Linlang 6fe0d75b38 chore: remove package (#449)
* remove package

* reformat with black

* fix pytest error

* fix pytest error

* fix pytest error

* add packages

* change code
2024-10-24 20:38:13 +08:00
Xu Yang c379b9221a feat: independent use_azure_token_provider on chat and embedding (#452)
* independent use_azure_token_provider

* fix CI

* fix doc format
2024-10-23 18:22:58 +08:00
Tim effce1df15 always return str (#453) 2024-10-23 16:47:36 +08:00
XianBW d57477aefa fix KGFactorRunner cache system bug (#451) 2024-10-23 15:17:38 +08:00
WinstonLiyt 972d8602c2 docs: add documentation for kaggle scen (#448)
* init for bg & quickstart for kaggle docs

* Add documentation for the environment configuration in the Kaggle scenario.

* add some descriptions in documents

* remove useless docs

* ci issue

---------

Co-authored-by: TPLin22 <tplin2@163.com>
2024-10-23 13:26:57 +08:00
Linlang d3c59b2f33 chore: remove torch in main code (#439)
* use np.ndarray in costeer model evaluators

* remove package

* fix test error

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-10-21 15:56:04 +08:00
XianBW 2161956315 chore: move use_auto_chat_cache_seed_gen and init_chat_cache_seed to LLMSett… (#444)
* move use_auto_chat_cache_seed_gen and init_chat_cache_seed to LLMSettings

* remove RD_AGENT_SETTINGS in llm_utils.py
2024-10-21 12:03:59 +08:00
XianBW 46d8247697 feat: use auto gen seed when using LLM cache (#441)
* initial version

* test requirements

* fix bugs

* fix bugs

* add annotation

* fix ruff error

* fix CI

* fix CI

* fix CI

* fix CI

* change random usage

* move cache_seed_gen to core/utils.py

* fix CI

* change cache_seed_gen name

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-10-21 11:32:20 +08:00
XianBW 8d88dbec33 fix display bug in webapp (#440) 2024-10-18 17:07:25 +08:00
XianBW 254c8f9a49 fix problem of reading log file tags on Windows (#437)
Co-authored-by: Bowen Xian (Shanghai Wicresoft Co Ltd) <v-bxian@microsoft.com>
2024-10-17 20:22:39 +08:00
Xu Yang b2de6f3190 add inf evaluator to factor costeer and some minor improvement (#435) 2024-10-16 16:00:55 +08:00
Tim 61189fbe73 number of subprocesses <= number of tasks (#434) 2024-10-16 14:13:20 +08:00
Way2Learn 4fff3370bd feat: supporting various Kaggle competitions & scenarios for RD-Agent (#409)
* Fixes on  kaggle output

* feat: add kaggle s3e14 template (#394)

* add s3e14 template

* fix CI

* Initialisation of a template of competition

* add kaggle s3e16 template (#396)

* get kaggle competition scores (#397)

* Adding a new competition s4e6

* feat: s4e5 (#400)

* init for s4e5

* edit s4e5

* ci issue

* feat: S4e3 (#402)

* Initialisation of a template of competition

* Adding a new competition s4e6

* Competition Initialised

* Fixed to make sure that now it runs

* Fixing for CI

* correct evaluation (#403)

* find rank in leaderboard (#405)

* fix: model templates for KG scenario (#408)

* fix feature selection for some models

* feat select template

* Updating the prompts for a more powerful model tuning

* refine the prompt

* fix: template error in s4e6

* feat: show simple execution time in demo (#410)

* show time in kaggle demo

* change color

* fix a small bug

* edit loop.py and proposal

* delete useless files

* CI issues

* ci issue

---------

Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: Haoran Pan <167847254+TPLin22@users.noreply.github.com>
Co-authored-by: Way2Learn <118058822+Xisen-Wang@users.noreply.github.com>
Co-authored-by: WinstonLiyt <1957922024@qq.com>
Co-authored-by: TPLin22 <tplin2@163.com>
2024-10-15 13:06:36 +08:00
Tim 022a558dd1 chore: more optional parameters for running benchmark analysis (#431)
* set title and round

* decision from multiple types

* check if decision is true

* reformat

* remove unused file
2024-10-15 12:41:32 +08:00
Xu Yang 5988683cd9 force use_chat_cache False when facing error in evaluators (#432) 2024-10-15 12:21:33 +08:00
Xu Yang ef373e1632 fix: fix cache result logic (#430)
* make cache logic more tidy

* update WeChat QR code
2024-10-14 19:35:21 +08:00
Xu Yang a8e4e6b4ae fix: fix a small bug in cache using module name and function name as unique folder name (#429) 2024-10-14 17:53:04 +08:00
Tim 216719670d convert str to bool (#428) 2024-10-14 17:44:33 +08:00
Xu Yang 25472fc1e3 feat: use unified pickle cacher & move llm config into a isolated config (#424)
* simplify RDAgent conf

* add unified cacher(untested)

* fix small bugs

* fix a bug

* fix a small bug in runner

* use hash_key = None to skip cache

* fix CI

* in factor execution, ignore cache when raise exception

* add file locker to avoid mp calling

* fix CI

* use function __module__ name as folder in cache
2024-10-14 17:34:09 +08:00
XianBW f808c35c34 auto select content element in kaggle crawler now (#423) 2024-10-11 22:34:12 +08:00
Xu Yang f1a841ba14 perf: some small upgrade to factor costeer to improve the performance (#420)
* 1. use dataframe.info instead of head
2. in former trace query, add the latest attempt to the last success execution

* fix CI
2024-10-10 20:29:17 +08:00
Linlang a1507aa7ab fix: fix json load error (#386)
* fix json load error

* fix os.symlink error

* reformat with black

* fix os.symlink error

* reformat with isort
2024-10-10 11:58:15 +08:00
Xu Yang f2a5967c12 improve factor costeer: (#417)
1. when facing odd v2_query_component_limit, make from gt bigger than without gt
2. do code evaluator even value does not pass
3. provide the value header to value failed evaluators
2024-10-09 18:56:37 +08:00
Xu Yang b3f1a10000 use main thread to get embedding to speed up cache (#415) 2024-10-08 19:34:19 +08:00
you-n-g a211facf79 refactor: refine util (#413)
* refine util

* lint
2024-10-08 02:07:20 +08:00
WinstonLiyt b8a931d0f5 fix: fix a bug in competition metric evaluation (#407)
* fix a bug in competition metric evaluation

* fix a bug

* fix a bug in rag loading
2024-09-30 14:54:56 +08:00
WinstonLiyt 858857a921 fix: refine the ucb algorithm (#406)
* refine the ucb algorithm

* fix a ci error
2024-09-30 12:30:42 +08:00
WinstonLiyt 8ec00a5120 feat: Add ranking in kaggle scenario (#401)
* fix some bugs in rag

* add ranking in kaggle scenario

* fix two mistouches

* fix two bugs

* fix a bug
2024-09-30 04:15:07 +08:00
WinstonLiyt 59cdf946a7 fix: fix some bugs in rag (#399)
* fix some bugs in rag

* fix a bug in costeer
2024-09-30 02:38:33 +08:00