Commit Graph

156 Commits

Author SHA1 Message Date
Roland Minrui da8fbcf4ee feat: add pipeline coder (#742)
* init commit

* limit problem numbers

* ensemble lower case

* add runtime and spec to coder

* submission check notice

* sub EDA in sample execution

* avoid lightgbm

* add time limit to scenario

* rephrase the submission check

* give positive feedback when facing warning in check

* ENABLE FEEDBACK

* fix feedback bug

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2025-04-03 13:00:11 +08:00
Haoran Pan 7fc5e73f76 fix: replace func "len()" in ensemble test code to support various data type (#739)
* replace len with def get_length

* update get_length implementation
2025-04-02 19:52:54 +08:00
Xu Yang bc8e9dc23a costeer max loop as hyperp & skip workflow in eval (#737) 2025-04-02 14:18:42 +08:00
you-n-g 260b563710 feat: loader prompt & simplify YAML loading and update data loader specifications (#736)
* refactor: Simplify YAML loading and update data loader specifications

* docs: Add comment explaining FunctionLoader usage in tpl.py

* lint

* lint
2025-04-01 22:36:25 +08:00
you-n-g 473b9000b9 feat: update prompts and descriptions for data science components (#731)
* docs: Update prompts and descriptions for data science components

* chore: Remove outdated comments from conf.py

* feat: Add metric_name attribute to DataScienceScen class

* style: Update description in prompts.yaml and reorder metric_name init

* docs: Update prompts.yaml with feature engineering guidelines
2025-04-01 20:54:42 +08:00
XianBW 450571e86e fix: add metric name check for valid scores (#724)
* update metric_name

* fix some bugs

* add an evaluation in workflow

* add an evalution in runner

* fix ci

* test change

* fix CI

---------

Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: yuanteli <1957922024@qq.com>
2025-03-27 19:36:57 +08:00
XianBW 7fc9e18a71 feat: make spec optional (#719)
* feat: Add spec_enabled configuration for data science settings

* make spec alternative

* change spec logic in exp_gen

* remove some general texts

* align

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: yuanteli <1957922024@qq.com>
2025-03-27 17:17:41 +08:00
Linlang 6da6fb5ee5 refactor: remove unused code and update documentation in CoSTEER module (#720)
* refactor: Remove unused code and update documentation in CoSTEER module

* fix mypy error

* fix black error

---------

Co-authored-by: Young <afe.young@gmail.com>
2025-03-26 22:48:01 +08:00
Yuante Li 5f3e382248 fix: fix a bug in ensemble test script (#713)
* fix a bug in ensemble test script

* fix

* add a comment
2025-03-20 17:35:39 +08:00
XianBW 60b66aa6c9 change only model codes in model coder (#707) 2025-03-19 19:37:26 +08:00
Yuante Li 5b14779f69 feat: add cross validation to workflow (#700) 2025-03-19 17:56:42 +08:00
XianBW 31381743f7 feat: out spec change for o1-preview (#666)
* add not_json batcheditout

* ensemble out_spec change

* feature out_spec change

* model out_spec change

* workflow out_spec change

* runner debugger out_spec change

* filter_progress_bar return format fix

* data_loader and spec out_spec change

* show finish_reason in llm log

* json_mode fix

* remove hardcode

* fix CI

* fix grammer

* complete PythonBatchEditOut logic

---------

Co-authored-by: yuanteli <1957922024@qq.com>
2025-03-18 17:43:44 +08:00
Ikko Eltociear Ashimine d1031ed24d refactor: update evolving_strategy.py (#686)
implmentation -> implementation
2025-03-18 16:15:08 +08:00
XianBW d66c045115 fix: add check when retrying gen model codes (#699)
* add check when retrying gen model codes

* fix CI
2025-03-18 15:59:34 +08:00
Yuante Li 7b3473cf30 feat: add model removal and adjust some framework logic (#681)
* prune model task

* add component_description

* add model removal logic to component, hypo, and task gen

* fix ci

* adjust coder to meet the requirement of model removal

* fix and refine the logic of model removal

* add model removal logic in model_eval

* fix ci

* fix ci

* prune some unnecessary codes
2025-03-18 14:42:53 +08:00
XianBW 1d6e97cb99 fix preds_dict's keys (#695) 2025-03-17 23:33:17 +08:00
XianBW 3d7b2b4567 fix: runner COSTEER evaluator (#693)
* fix scores.csv and submission.csv check in runner eval

* add coverage check info in running COSTEER feedback

* feedback value fix
2025-03-17 23:12:59 +08:00
Yuante Li b8a699a0df fix: fix a minor bug (#694) 2025-03-17 21:43:26 +08:00
Yuante Li bb91201dd2 fix: fix a bug related to model_name in ensemble (#692)
* fix a bug related to model_name in ensemble

* refine
2025-03-17 21:23:57 +08:00
XianBW e0ae91b497 validate and update 'final_decision' before init CosTeerFeedback (#691) 2025-03-17 19:53:09 +08:00
Yuante Li b20692cbd4 fix: fix an error in model_coder prompt (#690) 2025-03-17 15:38:39 +08:00
XianBW 9bba28093c check 'ensemble' in scores.csv in workflow eval (#682) 2025-03-14 17:40:06 +08:00
you-n-g 918b0b0d43 feat: condaenv & full docker env (#668)
* use conda to run kaggle and mlebench code

* refactor: Simplify environment configuration and execution logic

* add setting to use local env in ds

* refine dockerfile

* fix: Move MLEBDockerEnv initialization inside conditionals &  fix condaenv

* refactor: reformat code for better readability and consistency

* feat: add conda env to all envs.

* fix: fix bugs when run loop

* refactor: Simplify DockerEnv configuration in mle_summary.py

* fix image bug

* style: reformat code for better readability and consistency

* change commit

* feat: Add entrypoint script for sing_docker scenario in rdagent

* refactor: add Any type hints and comments for clarity in env.py

* feat: Create log directory if it doesn't exist in entrypoint script

* feat: Add debug mode and list root directory in entrypoint script

* fix: Remove specific branch checkout in Dockerfile for RD-Agent

* fix: Add competition argument to loop.py script execution

* fix: Correct directory navigation and dependency installation in entrypoint.sh

* fix: Correct user ownership assignment in entrypoint script

* refactor: Comment out redundant log copying to RD_OUTPUT_DIR

* fix: Unset LOG_TRACE_PATH to prevent log contamination in entrypoint.sh

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2025-03-12 11:36:28 +08:00
you-n-g 2171e99416 fix: fix ExtendedSettingsConfigDict does not work (#660)
* refactor: Replace ExtendedSettingsConfigDict with SettingsConfigDict

* lint

* lint
2025-03-05 18:30:02 +08:00
Haoran Pan b49505be8e feat: variable printing tool of data_science coder testing (#658)
* update debug_info printing

* refine infomation printing

* update ensemble evaluator prompt

* substitute repr with reprlib.repr

* strip comments
2025-03-05 13:08:30 +08:00
Tim 30b52d7d57 scores.csv should not be rewritten (#653) 2025-02-28 19:56:20 +08:00
Xu Yang 74cfc5b906 feat: add type checker to api backend & align litellm and old backend (#647)
* move cache auto continue and retry to all api backend

* add type checker to json mode output

* fix CI

* feat: Add json_mode handling and streaming support in chat completion function

* lint

* fix a bug when returning a dict which value could contain int or bool

* remove litellm

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
2025-02-28 15:13:43 +08:00
XianBW bd258fe970 fix workflow eval logic (#650) 2025-02-27 22:07:20 +08:00
Yuante Li 68bf5665f0 fix: fix some bugs (ensemble output, HPO, model tuning) (#648) 2025-02-27 18:28:13 +08:00
Xu Yang a57e625b17 feat: add eda to data science scenario (#639)
* add eda to data science scenario

* fix CI
2025-02-26 22:35:44 +08:00
Xu Yang d7da15c826 feat: add max time config to costeer in data science (#645)
* add max time config to costeer

* fix a small bug

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-26 18:36:51 +08:00
Yuante Li ac7c2155e4 fix: clarify cross_validation (#644) 2025-02-26 17:52:30 +08:00
XianBW 28f072da31 change ensemble metric prompt's code tips (#640) 2025-02-26 11:40:25 +08:00
XianBW 34e143cd66 fix: avoid try-except in ensemble eval prompts (#637)
* avoid try-except in prompts

* remove try-except limit from coder prompt

* add no try-except criteria in ensemble eval
2025-02-25 23:06:41 +08:00
Tim e73db7e0d7 ignore new numeric type (#638) 2025-02-25 18:20:14 +08:00
Xu Yang f120341c9c fix: qlib and other scenario bugs (#636)
* fix qlib and other scenario bugs

* bug fix
2025-02-25 12:06:07 +08:00
XianBW 54cd73eb85 fix: add ensemble test, change to "use cross-validation if possible" in workflow spec (#634)
* change to "use cross-validation if possible" in workflow spec

* Limit the evaluation indicator to only one

* add metric tips

* string change
2025-02-24 20:28:32 +08:00
Yuante Li b64ab18caa fix: fix a bug in cross validation (#618)
* fix a bug in cross_validation

* fix cross-validation prompt position

* fix

* fix

* append all error messages and full traceback details in execution

---------

Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2025-02-20 16:18:40 +08:00
Yuante Li 433594297a fix: fix some bugs in workflow unit test (#624)
* fix some bugs in workflow unit test

* fix ci
2025-02-20 15:42:17 +08:00
you-n-g dff89d2950 refactor: add run_ret_code method and update run method to use it (#623)
* refactor: Add run_ret_code method and update run method to use it

* feat: Add kwargs support to run methods and test for run_ret_code

* fix: preserve exit code after chmod in DockerEnv entry command

* chore: Change file permissions from 755 to 644 in env_tpl directory

* refactor: Return execution code and update evaluator logic

* lint

* refactor: Use MappingProxyType for running_extra_volume in DockerEnv methods

* lint
2025-02-20 00:42:10 +08:00
Xu Yang 0bd8366254 several bug fix (#621) 2025-02-19 22:03:39 +08:00
XianBW 2222897b3f fix: change ensemble test (#622)
* change ensemble test

* remove redundant info

* fix bug

* add model name duplication check in ensemble test

* change text
2025-02-19 21:26:09 +08:00
Yuante Li 464222094f add a check for the name of ensemble's score in scores.csv (#614) 2025-02-18 22:24:11 +08:00
Xu Yang 75e496d461 fix restart bug (#609) 2025-02-18 15:39:34 +08:00
you-n-g 9fcd5d473e refactor: replace Evaluator with CoSTEEREvaluator in eva_utils.py (#607)
* refactor: Replace Evaluator with CoSTEEREvaluator in eva_utils.py

* fix bug

---------

Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
2025-02-18 13:38:13 +08:00
Yuante Li 21d88f38d0 fix: fix some minor bugs and add AutoML & cross-validation (#604)
* fix some complex sentences

* add AutoML in model and cross-validation in workflow
2025-02-17 19:14:50 +08:00
Xu Yang bddb5eb935 fix: docker error will trigger retry and data science runner loop set to 3 (#602)
* docker error will trigger retry
data science runner loop set to 3

* fix CI
2025-02-17 17:36:05 +08:00
you-n-g ffc85936f1 refactor: refactor core framework to better propogate feedbacks (#599)
* refactor: Update type annotations and remove unused class in evolving modules

* refactor: Simplify evolving agent and feedback handling in CoSTEER module

* lint & CI

* mypy

* ruff for core

* mypy

* refactor: remove unnecessary comments and update feedback handling logic

* refactor: Add prev_task_feedback parameter to evolving strategies

* feat: Clear folder before extracting zip file in DockerEnv

* fix: Correct retrieval of last experiment from history
2025-02-16 01:40:44 +08:00
Yuante Li c321ebfbcb fix: fix some bugs in the ensemble component (#595)
* allow the LLM in the ensemble to focus on the significance of metrics

* fix a bug in feedback

* prun spec

* fix a bug in ensemble

* delete unnecessary prompts

* fix the logic in spec

* generalize dataset split
2025-02-14 15:32:40 +08:00
Xu Yang 1a44a954df feat: integrate azure deepseek r1 (#591)
* fix several task & integrate deepseek R1

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-13 22:20:17 +08:00