Commit Graph

520 Commits

Author SHA1 Message Date
Xu Yang a57e625b17 feat: add eda to data science scenario (#639)
* add eda to data science scenario

* fix CI
2025-02-26 22:35:44 +08:00
Xu Yang d7da15c826 feat: add max time config to costeer in data science (#645)
* add max time config to costeer

* fix a small bug

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-26 18:36:51 +08:00
Yuante Li ac7c2155e4 fix: clarify cross_validation (#644) 2025-02-26 17:52:30 +08:00
you-n-g bccf5aed8a fix: Set default value for 'entry' parameter in Env.run method (#643) 2025-02-26 16:51:38 +08:00
Tim 5bb73ff6cb chore: sample unique labels (#642)
* sample unique labels

* reformat
2025-02-26 15:24:39 +08:00
Roland Minrui b772fe2191 load code from file dict instead of folder (#641)
Co-authored-by: Xu <v-xuminrui@microsoft.com>
2025-02-26 14:21:28 +08:00
XianBW 28f072da31 change ensemble metric prompt's code tips (#640) 2025-02-26 11:40:25 +08:00
XianBW 34e143cd66 fix: avoid try-except in ensemble eval prompts (#637)
* avoid try-except in prompts

* remove try-except limit from coder prompt

* add no try-except criteria in ensemble eval
2025-02-25 23:06:41 +08:00
Tim e73db7e0d7 ignore new numeric type (#638) 2025-02-25 18:20:14 +08:00
Xu Yang f120341c9c fix: qlib and other scenario bugs (#636)
* fix qlib and other scenario bugs

* bug fix
2025-02-25 12:06:07 +08:00
XianBW 54cd73eb85 fix: add ensemble test, change to "use cross-validation if possible" in workflow spec (#634)
* change to "use cross-validation if possible" in workflow spec

* Limit the evaluation indicator to only one

* add metric tips

* string change
2025-02-24 20:28:32 +08:00
Yuante Li 20bd55c3a5 fix: fix a bug in initial tasks (#635)
* fix a bug in initial tasks

* fix ci
2025-02-24 20:23:54 +08:00
you-n-g 403d7fcf80 feat: Make system prompt role customizable in LLM settings (#632) 2025-02-23 00:52:30 +08:00
Yuante Li f5761b1f2d feat: improve the framework's ability to adaptively adjust the model (#629) 2025-02-21 22:46:14 +08:00
Xu Yang d4ade36184 refine component choosing part (#630) 2025-02-21 18:35:36 +08:00
Roland Minrui 982b43e4fd feat: add output_path to load function of LoopBase (#628)
* feat: add output_path to load from checkpoint function

* add default value to output_path

* sort import

* sort imports

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
2025-02-21 16:46:58 +08:00
Yuante Li b64ab18caa fix: fix a bug in cross validation (#618)
* fix a bug in cross_validation

* fix cross-validation prompt position

* fix

* fix

* append all error messages and full traceback details in execution

---------

Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2025-02-20 16:18:40 +08:00
Yuante Li 433594297a fix: fix some bugs in workflow unit test (#624)
* fix some bugs in workflow unit test

* fix ci
2025-02-20 15:42:17 +08:00
you-n-g dff89d2950 refactor: add run_ret_code method and update run method to use it (#623)
* refactor: Add run_ret_code method and update run method to use it

* feat: Add kwargs support to run methods and test for run_ret_code

* fix: preserve exit code after chmod in DockerEnv entry command

* chore: Change file permissions from 755 to 644 in env_tpl directory

* refactor: Return execution code and update evaluator logic

* lint

* refactor: Use MappingProxyType for running_extra_volume in DockerEnv methods

* lint
2025-02-20 00:42:10 +08:00
Xu Yang 0bd8366254 several bug fix (#621) 2025-02-19 22:03:39 +08:00
XianBW 2222897b3f fix: change ensemble test (#622)
* change ensemble test

* remove redundant info

* fix bug

* add model name duplication check in ensemble test

* change text
2025-02-19 21:26:09 +08:00
XianBW fc5f295de7 fix: move docker timeout message to __run() (#620)
* move docker timeout msg to __run()

* fix CI
2025-02-19 19:09:52 +08:00
Haoran Pan 05c7712770 update qrcode (#616) 2025-02-19 13:44:40 +08:00
you-n-g 1737aff7c9 fix: Runnable on first complete & Rename method to next_incomplete_component for clarity (#615) 2025-02-19 01:00:25 +08:00
Yuante Li 464222094f add a check for the name of ensemble's score in scores.csv (#614) 2025-02-18 22:24:11 +08:00
you-n-g 42d15623ca fix: move next_component_required logic to DSTrace class and accurate implement (#612)
* refactor: Move next_component_required logic to DSTrace class

* lint
2025-02-18 21:56:32 +08:00
Haoran Pan 6c20263e2e fix: sota comparison logic (#608)
* concat sota and current result

* fix the bug in current_exp's score vs sota's score

* fix ci

---------

Co-authored-by: yuanteli <1957922024@qq.com>
2025-02-18 20:43:28 +08:00
XianBW 50e637f9cd add PYTHONUNBUFFERED=1 for docker env (#613) 2025-02-18 20:07:49 +08:00
Roland Minrui 1605cddb40 feat: add loop_n parameter to the main loop (#611)
* add loop_n parameter to main loop

* complete the loop_n

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
2025-02-18 19:22:52 +08:00
Xu Yang 75e496d461 fix restart bug (#609) 2025-02-18 15:39:34 +08:00
you-n-g 9fcd5d473e refactor: replace Evaluator with CoSTEEREvaluator in eva_utils.py (#607)
* refactor: Replace Evaluator with CoSTEEREvaluator in eva_utils.py

* fix bug

---------

Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
2025-02-18 13:38:13 +08:00
Xu Yang 5f1c814355 fix: a small bug in exp_gen (#606) 2025-02-17 22:40:05 +08:00
Yuante Li 21d88f38d0 fix: fix some minor bugs and add AutoML & cross-validation (#604)
* fix some complex sentences

* add AutoML in model and cross-validation in workflow
2025-02-17 19:14:50 +08:00
Tim fd27842a98 print head for txt files and shuffle when sample (#603) 2025-02-17 18:44:26 +08:00
Xu Yang bddb5eb935 fix: docker error will trigger retry and data science runner loop set to 3 (#602)
* docker error will trigger retry
data science runner loop set to 3

* fix CI
2025-02-17 17:36:05 +08:00
you-n-g ffc85936f1 refactor: refactor core framework to better propogate feedbacks (#599)
* refactor: Update type annotations and remove unused class in evolving modules

* refactor: Simplify evolving agent and feedback handling in CoSTEER module

* lint & CI

* mypy

* ruff for core

* mypy

* refactor: remove unnecessary comments and update feedback handling logic

* refactor: Add prev_task_feedback parameter to evolving strategies

* feat: Clear folder before extracting zip file in DockerEnv

* fix: Correct retrieval of last experiment from history
2025-02-16 01:40:44 +08:00
Tim 241c05ccac chore: print txt info (#600)
* print head for txt files

* update sample
2025-02-14 19:16:28 +08:00
Xu Yang 50e020b75d fix: auto continue small bug (#598) 2025-02-14 17:03:57 +08:00
Yuante Li d1c005ff6d fix: fix a bug in proposal (add last loop's exception to last task desc) (#596)
* fix a bug in proposal (add last loop's exception to last task desc)

* fix
2025-02-14 16:36:49 +08:00
Xu Yang 96ee62c7a2 small bugs in cache rerun (#597) 2025-02-14 16:24:26 +08:00
Yuante Li c321ebfbcb fix: fix some bugs in the ensemble component (#595)
* allow the LLM in the ensemble to focus on the significance of metrics

* fix a bug in feedback

* prun spec

* fix a bug in ensemble

* delete unnecessary prompts

* fix the logic in spec

* generalize dataset split
2025-02-14 15:32:40 +08:00
Xu Yang 1a44a954df feat: integrate azure deepseek r1 (#591)
* fix several task & integrate deepseek R1

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-13 22:20:17 +08:00
Tim 4d8e334ef8 fix error (#594) 2025-02-13 19:34:49 +08:00
Tim b7be0f35f8 fix: ensure expected type (#593)
* ensure expected type

* reformat
2025-02-13 19:04:26 +08:00
炼金术师华华 97c1f7a021 feat(backend): integrate LiteLLM API Backend (#564)
* File structure for supporting litellm

* more litellm support

* feat: Add CachedAPIBackend class and dynamic API backend retrieval function

* fix: update benchmark folder path and add default values for architecture and hyperparameters

* feat: add LiteLLMAPIBackend and DeprecBackend ; changed structure of the project ; with bus

* fix : deprec_backend

* feat: Add LiteLLMAPIBackend class and related features; update configuration and test cases.

* feat: Enhance LiteLLMAPIBackend with encoder support and dynamic argument handling;Enhance log Colors

* lint

* fix lint...

* fix: Lint

* fix:make auto-lint

* fix:test oai

* fix:redundant _abckend.py

* fix: Optimize LiteLLMAPIBackend on token counting functiona, and clean up unused code;add test on this function

* feat: Add LiteLLMSettings class and update model settings usage

* fix: Update LiteLLMSettings environment variable prefix and model configurations

* fix : gitignore

* test: Consolidate and relocate test files for litellm backend and oai

* fix : lint

* fix: lint

* auto lint

* lint

* LINT

* lint

* chore: remove deprecated backend configuration comments

* refactor: Remove unused functions and imports from deprec.py and llm_utils.py

* refactor: Move md5_hash function from deprec.py to llm_utils.py

* chore: Remove extra newline and add missing import in deprec.py

* lint

* refactor: Move md5_hash function to utils module

* lint

* lint

* lint

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Yihua Chen <v-yihuachen@microsoft.com>
2025-02-13 15:16:18 +08:00
Yuante Li f362a1618d fix: fix a bug in threshold score display (#592)
* fix a bug in threshold score display

* fix
2025-02-13 14:50:56 +08:00
Yuante Li 43703d45d2 feat: add baseline score stat (#590)
* add baseline score stat

* fix ci

* fix

* fix
2025-02-12 18:57:08 +08:00
Tim f06412e628 sample group files by basename (#589) 2025-02-12 16:36:46 +08:00
Xu Yang dc2bf4b442 fix: sort file name for cache reproduction (#588) 2025-02-12 15:29:57 +08:00
you-n-g 8f1f19d4b9 fix: Handle ValueError when resolving relative path for uri (#585) 2025-02-12 13:23:08 +08:00