Commit Graph

142 Commits

Author SHA1 Message Date
Yuante Li 3e9ba73c2f feat: add model removal and adjust some framework logic (#681)
* prune model task

* add component_description

* add model removal logic to component, hypo, and task gen

* fix ci

* adjust coder to meet the requirement of model removal

* fix and refine the logic of model removal

* add model removal logic in model_eval

* fix ci

* fix ci

* prune some unnecessary codes
2025-03-18 14:42:53 +08:00
XianBW 229866123b fix preds_dict's keys (#695) 2025-03-17 23:33:17 +08:00
XianBW 4c9c29d244 fix: runner COSTEER evaluator (#693)
* fix scores.csv and submission.csv check in runner eval

* add coverage check info in running COSTEER feedback

* feedback value fix
2025-03-17 23:12:59 +08:00
Yuante Li 6a6c9832f4 fix: fix a minor bug (#694) 2025-03-17 21:43:26 +08:00
Yuante Li 119d69185a fix: fix a bug related to model_name in ensemble (#692)
* fix a bug related to model_name in ensemble

* refine
2025-03-17 21:23:57 +08:00
XianBW 699984fe1e validate and update 'final_decision' before init CosTeerFeedback (#691) 2025-03-17 19:53:09 +08:00
Yuante Li 1184be76bc fix: fix an error in model_coder prompt (#690) 2025-03-17 15:38:39 +08:00
XianBW 641beb2203 check 'ensemble' in scores.csv in workflow eval (#682) 2025-03-14 17:40:06 +08:00
you-n-g 20778a9b87 feat: condaenv & full docker env (#668)
* use conda to run kaggle and mlebench code

* refactor: Simplify environment configuration and execution logic

* add setting to use local env in ds

* refine dockerfile

* fix: Move MLEBDockerEnv initialization inside conditionals &  fix condaenv

* refactor: reformat code for better readability and consistency

* feat: add conda env to all envs.

* fix: fix bugs when run loop

* refactor: Simplify DockerEnv configuration in mle_summary.py

* fix image bug

* style: reformat code for better readability and consistency

* change commit

* feat: Add entrypoint script for sing_docker scenario in rdagent

* refactor: add Any type hints and comments for clarity in env.py

* feat: Create log directory if it doesn't exist in entrypoint script

* feat: Add debug mode and list root directory in entrypoint script

* fix: Remove specific branch checkout in Dockerfile for RD-Agent

* fix: Add competition argument to loop.py script execution

* fix: Correct directory navigation and dependency installation in entrypoint.sh

* fix: Correct user ownership assignment in entrypoint script

* refactor: Comment out redundant log copying to RD_OUTPUT_DIR

* fix: Unset LOG_TRACE_PATH to prevent log contamination in entrypoint.sh

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2025-03-12 11:36:28 +08:00
you-n-g 943d2087fc fix: fix ExtendedSettingsConfigDict does not work (#660)
* refactor: Replace ExtendedSettingsConfigDict with SettingsConfigDict

* lint

* lint
2025-03-05 18:30:02 +08:00
Haoran Pan 8ee918a0f3 feat: variable printing tool of data_science coder testing (#658)
* update debug_info printing

* refine infomation printing

* update ensemble evaluator prompt

* substitute repr with reprlib.repr

* strip comments
2025-03-05 13:08:30 +08:00
Tim deac0ac5e7 scores.csv should not be rewritten (#653) 2025-02-28 19:56:20 +08:00
Xu Yang 993ca37125 feat: add type checker to api backend & align litellm and old backend (#647)
* move cache auto continue and retry to all api backend

* add type checker to json mode output

* fix CI

* feat: Add json_mode handling and streaming support in chat completion function

* lint

* fix a bug when returning a dict which value could contain int or bool

* remove litellm

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
2025-02-28 15:13:43 +08:00
XianBW 0cd9d28863 fix workflow eval logic (#650) 2025-02-27 22:07:20 +08:00
Yuante Li 9c8182acfe fix: fix some bugs (ensemble output, HPO, model tuning) (#648) 2025-02-27 18:28:13 +08:00
Xu Yang 2bceddf8a3 feat: add eda to data science scenario (#639)
* add eda to data science scenario

* fix CI
2025-02-26 22:35:44 +08:00
Xu Yang 3392a518fb feat: add max time config to costeer in data science (#645)
* add max time config to costeer

* fix a small bug

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-26 18:36:51 +08:00
Yuante Li 21dfcdabf8 fix: clarify cross_validation (#644) 2025-02-26 17:52:30 +08:00
XianBW 75bdce3631 change ensemble metric prompt's code tips (#640) 2025-02-26 11:40:25 +08:00
XianBW 6db80918fc fix: avoid try-except in ensemble eval prompts (#637)
* avoid try-except in prompts

* remove try-except limit from coder prompt

* add no try-except criteria in ensemble eval
2025-02-25 23:06:41 +08:00
Tim 36dfd6d424 ignore new numeric type (#638) 2025-02-25 18:20:14 +08:00
Xu Yang c109fa8e99 fix: qlib and other scenario bugs (#636)
* fix qlib and other scenario bugs

* bug fix
2025-02-25 12:06:07 +08:00
XianBW 374dbdb553 fix: add ensemble test, change to "use cross-validation if possible" in workflow spec (#634)
* change to "use cross-validation if possible" in workflow spec

* Limit the evaluation indicator to only one

* add metric tips

* string change
2025-02-24 20:28:32 +08:00
Yuante Li 538df31efe fix: fix a bug in cross validation (#618)
* fix a bug in cross_validation

* fix cross-validation prompt position

* fix

* fix

* append all error messages and full traceback details in execution

---------

Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2025-02-20 16:18:40 +08:00
Yuante Li b42c395267 fix: fix some bugs in workflow unit test (#624)
* fix some bugs in workflow unit test

* fix ci
2025-02-20 15:42:17 +08:00
you-n-g 0631de5103 refactor: add run_ret_code method and update run method to use it (#623)
* refactor: Add run_ret_code method and update run method to use it

* feat: Add kwargs support to run methods and test for run_ret_code

* fix: preserve exit code after chmod in DockerEnv entry command

* chore: Change file permissions from 755 to 644 in env_tpl directory

* refactor: Return execution code and update evaluator logic

* lint

* refactor: Use MappingProxyType for running_extra_volume in DockerEnv methods

* lint
2025-02-20 00:42:10 +08:00
Xu Yang 0e302204e8 several bug fix (#621) 2025-02-19 22:03:39 +08:00
XianBW 77ddee64ce fix: change ensemble test (#622)
* change ensemble test

* remove redundant info

* fix bug

* add model name duplication check in ensemble test

* change text
2025-02-19 21:26:09 +08:00
Yuante Li f200320b3f add a check for the name of ensemble's score in scores.csv (#614) 2025-02-18 22:24:11 +08:00
Xu Yang 15584bad43 fix restart bug (#609) 2025-02-18 15:39:34 +08:00
you-n-g 4b949b3c29 refactor: replace Evaluator with CoSTEEREvaluator in eva_utils.py (#607)
* refactor: Replace Evaluator with CoSTEEREvaluator in eva_utils.py

* fix bug

---------

Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
2025-02-18 13:38:13 +08:00
Yuante Li f322766a00 fix: fix some minor bugs and add AutoML & cross-validation (#604)
* fix some complex sentences

* add AutoML in model and cross-validation in workflow
2025-02-17 19:14:50 +08:00
Xu Yang d9576706cd fix: docker error will trigger retry and data science runner loop set to 3 (#602)
* docker error will trigger retry
data science runner loop set to 3

* fix CI
2025-02-17 17:36:05 +08:00
you-n-g 5baed909e7 refactor: refactor core framework to better propogate feedbacks (#599)
* refactor: Update type annotations and remove unused class in evolving modules

* refactor: Simplify evolving agent and feedback handling in CoSTEER module

* lint & CI

* mypy

* ruff for core

* mypy

* refactor: remove unnecessary comments and update feedback handling logic

* refactor: Add prev_task_feedback parameter to evolving strategies

* feat: Clear folder before extracting zip file in DockerEnv

* fix: Correct retrieval of last experiment from history
2025-02-16 01:40:44 +08:00
Yuante Li 3c2f799618 fix: fix some bugs in the ensemble component (#595)
* allow the LLM in the ensemble to focus on the significance of metrics

* fix a bug in feedback

* prun spec

* fix a bug in ensemble

* delete unnecessary prompts

* fix the logic in spec

* generalize dataset split
2025-02-14 15:32:40 +08:00
Xu Yang ed53af4c65 feat: integrate azure deepseek r1 (#591)
* fix several task & integrate deepseek R1

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-13 22:20:17 +08:00
Tim 940a58e142 ignore score value (#586) 2025-02-12 11:39:49 +08:00
XianBW 3d8ec4e272 fix: coder prompt & model test text (#583)
* test text fix

* Require LLM to use print() instead of logging
2025-02-11 21:12:25 +08:00
Xu Yang 203967ae9a add feedback to workspace and ds runner base on costeer 2025-02-11 20:50:19 +08:00
Tim 2bb176427a avoid length exception (#580) 2025-02-11 18:12:45 +08:00
Yuante Li e2b7c9dc78 fix: fix the errors in the coder and evaluator of the five components (#576)
* refine the prompt in feat coder

* refine data loader prompt in coder & eval

* fix

* fix

* refine model prompt in coder & eval

* refine ensemble prompt in coder & eval

* refine workflow prompt in coder & eval

* fix
2025-02-10 20:20:30 +08:00
Yuante Li 63d7388612 add the none check for data loader & feature output (#577) 2025-02-10 20:18:01 +08:00
XianBW b452626cfb text change (#568) 2025-02-08 21:14:13 +08:00
XianBW 1b193fa2e7 fix: add scores.csv checking in ensemble_test (#567)
* add scores.csv checking in ensemble_test

* text modify
2025-02-08 19:03:29 +08:00
Tim a0e274889b support multiple types in feature engineering (#562) 2025-02-08 15:21:11 +08:00
Xu Yang e82d38f88e fix: move mlebench check into runner (#556)
* abandon mlebench check in workflow

* move mlebench check to runner and put it into the exp
2025-02-06 15:37:41 +08:00
you-n-g cade50de89 fix: handle division by zero in percentage calculations (#550)
* fix: Handle division by zero in percentage calculations

* lint
2025-02-05 10:21:53 +08:00
you-n-g 8bc381855e fix: replace API call with build_cls_from_json_with_retry function (#548)
* refactor: Replace API call with build_cls_from_json_with_retry function

* fix lint error

* fix lint errors

* lint

* trigger
2025-01-27 20:19:11 +08:00
Xu Yang 1bd192e061 fix: refine prompt to generate the most simple task in init stage (#546)
* refine prompt to generate the most simple task in init stage

* feature test dtype check improve
2025-01-27 00:51:40 +08:00
XianBW b05cd1a1a7 feat: add mlebench submission validitor (#545)
* add mlebench submission check

* fix CI

* fix bug
2025-01-27 00:41:35 +08:00