a12b2fe9ee
chore: organize the tool functions related to trace and summary ( #889 )
...
* provide get metric direction in kaggle_crawler.py
* move utils function
* move statistics logic
* fix CI
* fix CI
* fix CI
* fix CI
* move some tool functions
* fix CI
* move curves win
* add compare tool
* change function name
* fix CI
2025-05-21 17:20:31 +08:00
6f29e4e3bc
feat: multi-trace online merge ( #886 )
...
* prompt: highlight overfitting rist in AutoSOTAexpSelector
* set online merge time in conf
* online multi-trace merge with time-limit policy
* fix typo
* feat: allow soft-knowledge-base + multi_trace
* fix: improve file tree and _walk symlink handling (#877 )
* refactor: improve file tree and _walk symlink handling
* remove unused code
* lint
* prompt: highlight overfitting rist in AutoSOTAexpSelector
* set online merge time in conf
* online multi-trace merge with time-limit policy
* fix typo
* feat: allow soft-knowledge-base + multi_trace
* auto-lint
* put the multi-trace related config together
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-05-19 17:59:42 +08:00
e42e4dae50
fix mypy check dependence error ( #885 )
2025-05-18 15:15:32 +08:00
89deb019e9
chore: ui bug fix ( #884 )
...
* bug fix
* bug fix
* test ci
* bug fix
* ci fix
* ci fix
2025-05-18 11:08:41 +08:00
db1656bb4c
chore: log storage refine ( #883 )
...
* log storage change
* fix bug
* fix ci
2025-05-16 18:29:59 +08:00
db8eec7394
fix: use fallback messages for missing submission and scores files ( #882 )
2025-05-16 14:39:47 +08:00
f23c6878b8
update_QR_code_url ( #881 )
2025-05-16 14:13:40 +08:00
60ec58f5df
fix: update DS env setup with competition volume and timeout ( #878 )
...
* refactor: update DS env setup with competition volume and timeout
* refactor: update volume mapping and timeout based on run type
2025-05-16 02:05:15 +08:00
da6f662dda
fix: improve file tree and _walk symlink handling ( #877 )
...
* refactor: improve file tree and _walk symlink handling
* remove unused code
* lint
2025-05-16 00:11:25 +08:00
7a5453552d
doc: small typo in README ( #876 )
2025-05-15 21:40:39 +08:00
0279037915
Update README.md ( #875 )
2025-05-15 21:32:20 +08:00
bb99485fd3
add multi trace results ( #873 )
...
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-15 20:29:52 +08:00
5e1292974b
fix debug_tpl and debug_llm saving method( #874 )
2025-05-15 18:37:51 +08:00
7943bbcc86
docs: update README.md ( #871 )
2025-05-14 11:32:23 +08:00
558c3b3c45
chore: news MLE-bench release ( #870 )
...
* docs: add MLE-bench details to README
* docs: update README with revised MLE-bench description and leaderboard
* docs: update RD-Agent text and add trial info in README
* Update README.md
* Update README.md
* update by M
* update format
* Add documents
* docs: update RD-Agent references to R&D-Agent
* docs: update README with MLE-Bench complexity level details
2025-05-14 11:21:13 +08:00
d0b9ad8c12
feat: add competition level filter and extract constants to utils ( #869 )
...
* feat: add competition level filter and extract constants to utils
* lint
2025-05-12 21:56:57 +08:00
b8fb9cf3b4
fix coder bug ( #868 )
...
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-05-12 17:20:26 +08:00
eb5f69024a
chore: custom data refine ( #864 )
...
* chore: print up to 100 columns in simple mode
* fix: check content for model dump
* chore: add show_nan_columns config
2025-05-10 17:02:12 +08:00
e2bc29152d
feat: truncate by time ( #863 )
...
* refactor: move session file lookup logic to folder utils module
* print more info
* lint
2025-05-10 00:42:59 +08:00
bb96fc84f6
feat: advanced checkpoint selectors ( #790 )
...
* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* imp SOTA-Jump policy
* fix small bug
* allow to set different selector by .env
* add always-win selector
* add init length for AlwaysWinCKPSelector
* add back_jump selector
* auto lint
* add sota_exp_to_submit attribute; change the name of ckp_selector and sota-selector
* fix bug
* auto lint
* working on auto sota selector
* add subtrace counter
* fix bug, remove unuse selector
* add auto sota selector
* auto lint
* fix bug
* fix small logic bug
* add logging
* add inject_diverse feat
* auto lint
* capable to None-select
* feat: add hypothesis_gen config and ExpGen2TraceAndMerge functionality
* refactor: use dynamic import for experiment generator instantiation
* feat: add BestValidSelector for improved SOTA experiment selection
* runnable twin-trace version
* fix logic error of trace-merge
* auto lint
* use import_class to set selector,
* auto-lint
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-05-09 15:38:25 +08:00
fa53473b6c
fix: typo in workflow ( #861 )
2025-05-09 13:29:36 +08:00
1fd8abce0f
feat: log api status to mlflow ( #860 )
...
* log api status to mlflow to frontend tracking
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-09 12:20:49 +08:00
5fecefb802
make timeout fail a hyper parameter ( #859 )
...
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-08 21:30:44 +08:00
6fbe213f3d
change method of cut log trace ( #858 )
2025-05-08 21:07:33 +08:00
24c4eb7225
fix: refine the time/memory constraints prompt in hypothesis proposal ( #856 )
...
* refine prompt
* refine the wording
* add ratelimit retry to align with the suggested wait seconds
* add max retry to 0
* don't delete hist
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-08 21:05:07 +08:00
eed4331f0f
fix: fixed CI execution failures caused by document builds ( #857 )
...
* fixed CI execution failures caused by document builds
* add comments
2025-05-08 20:35:13 +08:00
9df59eb7f0
add cache for summary, remove workspace dependency when generate summary ( #854 )
2025-05-08 12:24:48 +08:00
e1d08cfe73
fix: adjust ds_trace lookup and add stderr redirect to mlebench command ( #853 )
...
* fix: adjust ds_trace lookup and add stderr redirect to mlebench command
* style: reformat SOTA experiment lookup in ds_trace.py
* feat: add DS_RD_SETTING pipeline to MergeExpGen success message
2025-05-08 02:40:49 +08:00
45b260ba37
fix: trace list but ( #852 )
2025-05-07 21:28:24 +08:00
8ee4c72bab
add aide.py ( #851 )
2025-05-07 19:59:40 +08:00
668ac16150
feat: revert draft stage into a soft decay in hypothesis selection ( #849 )
...
* revert drafting
* update hypothesis rank logic
* prioritize time constraint in task design
* refine trace_desc and feedback problem prompt
* refine experiment_and_feedback_list_after_init
* fix DSHypothesis default parameter and print logic
* refine the selection weight
* merge simple_trace and trace
* refine weight and prompt
* refine sample logic
* fix CI
* robust code
---------
Co-authored-by: WinstonLiyte <1957922024@qq.com >
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-07 19:05:50 +08:00
3b7bafae12
chore: dump test data ( #850 )
...
* debug path for model dump
* update prompt
2025-05-07 17:24:50 +08:00
cda656b4ca
fix: non-exist variable test_eval.py ( #847 )
2025-05-07 04:36:00 +08:00
62e5953a6a
fix: wrong variable test_eval.py ( #846 )
2025-05-07 04:17:57 +08:00
0047114a09
fix bug ( #845 )
2025-05-06 18:15:20 +08:00
46dec7b624
feat: custom data ( #810 )
...
* custom data
* fix: simplify competition check and log local description file
* no sample data
* feat: add test evaluation module with error handling support
* fix: update eval path to use eval_sub_dir and add valid_check TODO
* refactor: add MLETestEval check to conditionally run grading steps
* avoid blank stdout
* valid in testeval
* rename test.csv to avoid conflict
* Support Disabling sample submission
* refactoring
* fix: remove DS_KAGGLE_DATA and update prompt instructions
* add try for grade
* ignore submission
* fix: remove tee from eval command and warn about pipeline exit code detection
* optional to use raw description
* support old data
* add execution result to stdout
* add metric to raw description
* custom data explain
* add debug_path
* rst update
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-05-06 16:00:13 +08:00
eda4a2a807
chore: ui change ( #844 )
...
* ui change
* ui change
2025-05-06 11:34:21 +08:00
e8c39eb63d
feat: refine merge ( #842 )
...
* feat: add search_type param and customize success trial desc
* docs: simplify trial descriptions in share.yaml
* lint
2025-05-04 23:17:58 +08:00
fd98d29982
fix: adapting UI to mock trace ( #841 )
...
* fix: return first index if 'SOTA Exp Score (valid)' is empty
* feat: add get_state_data_range helper for loops and slider bounds
* fix: exclude batch embedding tag from log filtering
* feat: add feedback support in sota_experiment and adjust merge flow
* fix: use fb function for merging experiments
* style: remove extra whitespace and reformat code
2025-05-04 15:51:32 +08:00
5cba25505e
fix new draft bugs ( #840 )
2025-04-30 18:53:27 +08:00
acc15caf7b
feat: reanalyze competition info & pipeline coding evaluator prompt ( #837 )
...
* update coding evaluator prompt similar to feedback
* reanaylyzing competition description when three sonsecutive coding failures
* update reanalyzing competition implementation
* fix bug
* update prompts and reanalyze
* fix bugs
* ci issue
* improve some code
* fix CI
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-04-30 17:36:35 +08:00
5f3da39331
feat: add drafting pipeline ( #832 )
...
* init commit
* add drafting prompt
* complete the drafting
* remove scenario problems from proposal
* rename prompts_drafting.yaml
* fix bug
* fix DSHypothesis print bug
* add failed drafting exp to prompt
* fix small bug
* use get_task_information() for task design
* resolve all comments
* add problem_desc to pesudo hypothesis
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-04-30 16:25:32 +08:00
ec51bb94b6
feat: trace merging ( #836 )
...
* feat: runnalbe -- add exp_gen_cls param, get_leaves and merge exp gen functionalities
* fix: remove unused scenario_desc and update YAML task labels
* feat: override selection and update merge task description
* lint
* lint
* lint
* lint
* lint
* fix: log competition setting to enable mle_summary
* fix name error
2025-04-29 09:30:45 +08:00
7cbf5bf56c
when restart with kb and different pkl path, we need to initialize the kb from json file ( #835 )
2025-04-28 15:51:51 +08:00
6be059dd5d
chore: log cost object ( #829 )
2025-04-25 19:26:08 +08:00
686437d671
add pipeline to hypothesis spec ( #828 )
2025-04-25 18:29:14 +08:00
50d3b7ce61
feat: propose hypothesis across multiple parts in pipeline ( #827 )
...
* refine hypothesis for pipeline
* refine component selection prompt
* update feedback prompt
* remove feat_eng in pipeline coding
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-04-25 17:17:53 +08:00
24c4c6755e
fix: add time to timer when api timeout bug ( #826 )
...
* If not use the session stored timer, need to replace it with the default timer
* when api timeout, add the waiting time to timer
2025-04-25 10:50:27 +08:00
2b821884e7
If not use the session stored timer, need to replace it with the default timer ( #825 )
2025-04-25 00:39:31 +08:00
7f90e44798
fix: fix a bug in docker result extraction ( #824 )
...
* fix a bug in docker result extraction
2025-04-24 19:02:38 +08:00
XianBW