3e60749f70
fix: fix the bug of Exceed-LLM-Context in online merge of multi-tarce ( #892 )
...
* set constrains on max_sota_retrieved, fix logis on identical problem
* fix: only Auto SOTA selector use max_sota_retrieved_num
* set max_sota_retrieved_num=10 by default
* minor update
* auto lint
2025-05-22 16:10:00 +08:00
cfa5f382aa
docs: update paper report ( #893 )
...
* doc: update paper report
* Update README.md
* Update README.md
* Update README.md
2025-05-22 15:37:34 +08:00
9ea44be5f8
feat: new proposal (structured outputs) prompts ( #887 )
...
* some initial modifications
* first prompt
* refine prompts
* isolate to v3 exp gen
* Revert prompts
* Add hypothesis
* task gen prompt
* minor updates
* minor updates
* update proposal
* Move the pydantic schema upfront to avoid loading error
* Update first prompts
* New prompt
* Update prompt
* Update data folder
* .
* Revert changes
* support v3 in pipeline and feedback
* sort imports
* black format
* Update rdagent/scenarios/data_science/dev/prompts.yaml
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-05-22 15:08:33 +08:00
37e9912ac0
chore: organize the tool functions related to trace and summary ( #889 )
...
* provide get metric direction in kaggle_crawler.py
* move utils function
* move statistics logic
* fix CI
* fix CI
* fix CI
* fix CI
* move some tool functions
* fix CI
* move curves win
* add compare tool
* change function name
* fix CI
2025-05-21 17:20:31 +08:00
bc9cee49ba
feat: multi-trace online merge ( #886 )
...
* prompt: highlight overfitting rist in AutoSOTAexpSelector
* set online merge time in conf
* online multi-trace merge with time-limit policy
* fix typo
* feat: allow soft-knowledge-base + multi_trace
* fix: improve file tree and _walk symlink handling (#877 )
* refactor: improve file tree and _walk symlink handling
* remove unused code
* lint
* prompt: highlight overfitting rist in AutoSOTAexpSelector
* set online merge time in conf
* online multi-trace merge with time-limit policy
* fix typo
* feat: allow soft-knowledge-base + multi_trace
* auto-lint
* put the multi-trace related config together
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-05-19 17:59:42 +08:00
763dff1596
fix mypy check dependence error ( #885 )
2025-05-18 15:15:32 +08:00
fe11ee92e6
chore: ui bug fix ( #884 )
...
* bug fix
* bug fix
* test ci
* bug fix
* ci fix
* ci fix
2025-05-18 11:08:41 +08:00
2c11752a54
chore: log storage refine ( #883 )
...
* log storage change
* fix bug
* fix ci
2025-05-16 18:29:59 +08:00
7aea9a27dc
fix: use fallback messages for missing submission and scores files ( #882 )
2025-05-16 14:39:47 +08:00
0c97fe4ae7
update_QR_code_url ( #881 )
2025-05-16 14:13:40 +08:00
586349af36
fix: update DS env setup with competition volume and timeout ( #878 )
...
* refactor: update DS env setup with competition volume and timeout
* refactor: update volume mapping and timeout based on run type
2025-05-16 02:05:15 +08:00
5b3f5f66cf
fix: improve file tree and _walk symlink handling ( #877 )
...
* refactor: improve file tree and _walk symlink handling
* remove unused code
* lint
2025-05-16 00:11:25 +08:00
69e5c3a7de
doc: small typo in README ( #876 )
2025-05-15 21:40:39 +08:00
b76abe839b
Update README.md ( #875 )
2025-05-15 21:32:20 +08:00
c8c8e26812
add multi trace results ( #873 )
...
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-15 20:29:52 +08:00
42ea6f106b
fix debug_tpl and debug_llm saving method( #874 )
2025-05-15 18:37:51 +08:00
7742a9e759
docs: update README.md ( #871 )
2025-05-14 11:32:23 +08:00
8acd24a016
chore: news MLE-bench release ( #870 )
...
* docs: add MLE-bench details to README
* docs: update README with revised MLE-bench description and leaderboard
* docs: update RD-Agent text and add trial info in README
* Update README.md
* Update README.md
* update by M
* update format
* Add documents
* docs: update RD-Agent references to R&D-Agent
* docs: update README with MLE-Bench complexity level details
2025-05-14 11:21:13 +08:00
54465d6e82
feat: add competition level filter and extract constants to utils ( #869 )
...
* feat: add competition level filter and extract constants to utils
* lint
2025-05-12 21:56:57 +08:00
90f6c94d3e
fix coder bug ( #868 )
...
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-05-12 17:20:26 +08:00
7f4e5096e8
chore: custom data refine ( #864 )
...
* chore: print up to 100 columns in simple mode
* fix: check content for model dump
* chore: add show_nan_columns config
2025-05-10 17:02:12 +08:00
a294bc74e0
feat: truncate by time ( #863 )
...
* refactor: move session file lookup logic to folder utils module
* print more info
* lint
2025-05-10 00:42:59 +08:00
e71d8f6c3c
feat: advanced checkpoint selectors ( #790 )
...
* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* imp SOTA-Jump policy
* fix small bug
* allow to set different selector by .env
* add always-win selector
* add init length for AlwaysWinCKPSelector
* add back_jump selector
* auto lint
* add sota_exp_to_submit attribute; change the name of ckp_selector and sota-selector
* fix bug
* auto lint
* working on auto sota selector
* add subtrace counter
* fix bug, remove unuse selector
* add auto sota selector
* auto lint
* fix bug
* fix small logic bug
* add logging
* add inject_diverse feat
* auto lint
* capable to None-select
* feat: add hypothesis_gen config and ExpGen2TraceAndMerge functionality
* refactor: use dynamic import for experiment generator instantiation
* feat: add BestValidSelector for improved SOTA experiment selection
* runnable twin-trace version
* fix logic error of trace-merge
* auto lint
* use import_class to set selector,
* auto-lint
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-05-09 15:38:25 +08:00
5c7cdf298c
fix: typo in workflow ( #861 )
2025-05-09 13:29:36 +08:00
8a0a0fa3c4
feat: log api status to mlflow ( #860 )
...
* log api status to mlflow to frontend tracking
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-09 12:20:49 +08:00
69c630e983
make timeout fail a hyper parameter ( #859 )
...
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-08 21:30:44 +08:00
ad12fccb61
change method of cut log trace ( #858 )
2025-05-08 21:07:33 +08:00
9986b5f9ce
fix: refine the time/memory constraints prompt in hypothesis proposal ( #856 )
...
* refine prompt
* refine the wording
* add ratelimit retry to align with the suggested wait seconds
* add max retry to 0
* don't delete hist
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-08 21:05:07 +08:00
eb1cefb202
fix: fixed CI execution failures caused by document builds ( #857 )
...
* fixed CI execution failures caused by document builds
* add comments
2025-05-08 20:35:13 +08:00
4f5cce0607
add cache for summary, remove workspace dependency when generate summary ( #854 )
2025-05-08 12:24:48 +08:00
495063c385
fix: adjust ds_trace lookup and add stderr redirect to mlebench command ( #853 )
...
* fix: adjust ds_trace lookup and add stderr redirect to mlebench command
* style: reformat SOTA experiment lookup in ds_trace.py
* feat: add DS_RD_SETTING pipeline to MergeExpGen success message
2025-05-08 02:40:49 +08:00
318acb4922
fix: trace list but ( #852 )
2025-05-07 21:28:24 +08:00
8340ab5d09
add aide.py ( #851 )
2025-05-07 19:59:40 +08:00
22a0c7fe56
feat: revert draft stage into a soft decay in hypothesis selection ( #849 )
...
* revert drafting
* update hypothesis rank logic
* prioritize time constraint in task design
* refine trace_desc and feedback problem prompt
* refine experiment_and_feedback_list_after_init
* fix DSHypothesis default parameter and print logic
* refine the selection weight
* merge simple_trace and trace
* refine weight and prompt
* refine sample logic
* fix CI
* robust code
---------
Co-authored-by: WinstonLiyte <1957922024@qq.com >
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-07 19:05:50 +08:00
02534cfd88
chore: dump test data ( #850 )
...
* debug path for model dump
* update prompt
2025-05-07 17:24:50 +08:00
9a8426f373
fix: non-exist variable test_eval.py ( #847 )
2025-05-07 04:36:00 +08:00
6eef7517ad
fix: wrong variable test_eval.py ( #846 )
2025-05-07 04:17:57 +08:00
29a14e1e22
fix bug ( #845 )
2025-05-06 18:15:20 +08:00
6f9843a7df
feat: custom data ( #810 )
...
* custom data
* fix: simplify competition check and log local description file
* no sample data
* feat: add test evaluation module with error handling support
* fix: update eval path to use eval_sub_dir and add valid_check TODO
* refactor: add MLETestEval check to conditionally run grading steps
* avoid blank stdout
* valid in testeval
* rename test.csv to avoid conflict
* Support Disabling sample submission
* refactoring
* fix: remove DS_KAGGLE_DATA and update prompt instructions
* add try for grade
* ignore submission
* fix: remove tee from eval command and warn about pipeline exit code detection
* optional to use raw description
* support old data
* add execution result to stdout
* add metric to raw description
* custom data explain
* add debug_path
* rst update
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-05-06 16:00:13 +08:00
aa97d5a09d
chore: ui change ( #844 )
...
* ui change
* ui change
2025-05-06 11:34:21 +08:00
6b7988fba1
feat: refine merge ( #842 )
...
* feat: add search_type param and customize success trial desc
* docs: simplify trial descriptions in share.yaml
* lint
2025-05-04 23:17:58 +08:00
93303873d0
fix: adapting UI to mock trace ( #841 )
...
* fix: return first index if 'SOTA Exp Score (valid)' is empty
* feat: add get_state_data_range helper for loops and slider bounds
* fix: exclude batch embedding tag from log filtering
* feat: add feedback support in sota_experiment and adjust merge flow
* fix: use fb function for merging experiments
* style: remove extra whitespace and reformat code
2025-05-04 15:51:32 +08:00
d0e3fc1573
fix new draft bugs ( #840 )
2025-04-30 18:53:27 +08:00
cbedbc7d54
feat: reanalyze competition info & pipeline coding evaluator prompt ( #837 )
...
* update coding evaluator prompt similar to feedback
* reanaylyzing competition description when three sonsecutive coding failures
* update reanalyzing competition implementation
* fix bug
* update prompts and reanalyze
* fix bugs
* ci issue
* improve some code
* fix CI
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-04-30 17:36:35 +08:00
03040de35f
feat: add drafting pipeline ( #832 )
...
* init commit
* add drafting prompt
* complete the drafting
* remove scenario problems from proposal
* rename prompts_drafting.yaml
* fix bug
* fix DSHypothesis print bug
* add failed drafting exp to prompt
* fix small bug
* use get_task_information() for task design
* resolve all comments
* add problem_desc to pesudo hypothesis
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-04-30 16:25:32 +08:00
a73f2dbc12
feat: trace merging ( #836 )
...
* feat: runnalbe -- add exp_gen_cls param, get_leaves and merge exp gen functionalities
* fix: remove unused scenario_desc and update YAML task labels
* feat: override selection and update merge task description
* lint
* lint
* lint
* lint
* lint
* fix: log competition setting to enable mle_summary
* fix name error
2025-04-29 09:30:45 +08:00
45931b1012
when restart with kb and different pkl path, we need to initialize the kb from json file ( #835 )
2025-04-28 15:51:51 +08:00
6c179442bf
chore: log cost object ( #829 )
2025-04-25 19:26:08 +08:00
8affbf89a9
add pipeline to hypothesis spec ( #828 )
2025-04-25 18:29:14 +08:00
8968ab8c9a
feat: propose hypothesis across multiple parts in pipeline ( #827 )
...
* refine hypothesis for pipeline
* refine component selection prompt
* update feedback prompt
* remove feat_eng in pipeline coding
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-04-25 17:17:53 +08:00
xuangu-fang