db8eec7394
fix: use fallback messages for missing submission and scores files ( #882 )
2025-05-16 14:39:47 +08:00
60ec58f5df
fix: update DS env setup with competition volume and timeout ( #878 )
...
* refactor: update DS env setup with competition volume and timeout
* refactor: update volume mapping and timeout based on run type
2025-05-16 02:05:15 +08:00
da6f662dda
fix: improve file tree and _walk symlink handling ( #877 )
...
* refactor: improve file tree and _walk symlink handling
* remove unused code
* lint
2025-05-16 00:11:25 +08:00
5e1292974b
fix debug_tpl and debug_llm saving method( #874 )
2025-05-15 18:37:51 +08:00
d0b9ad8c12
feat: add competition level filter and extract constants to utils ( #869 )
...
* feat: add competition level filter and extract constants to utils
* lint
2025-05-12 21:56:57 +08:00
b8fb9cf3b4
fix coder bug ( #868 )
...
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-05-12 17:20:26 +08:00
eb5f69024a
chore: custom data refine ( #864 )
...
* chore: print up to 100 columns in simple mode
* fix: check content for model dump
* chore: add show_nan_columns config
2025-05-10 17:02:12 +08:00
e2bc29152d
feat: truncate by time ( #863 )
...
* refactor: move session file lookup logic to folder utils module
* print more info
* lint
2025-05-10 00:42:59 +08:00
bb96fc84f6
feat: advanced checkpoint selectors ( #790 )
...
* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* imp SOTA-Jump policy
* fix small bug
* allow to set different selector by .env
* add always-win selector
* add init length for AlwaysWinCKPSelector
* add back_jump selector
* auto lint
* add sota_exp_to_submit attribute; change the name of ckp_selector and sota-selector
* fix bug
* auto lint
* working on auto sota selector
* add subtrace counter
* fix bug, remove unuse selector
* add auto sota selector
* auto lint
* fix bug
* fix small logic bug
* add logging
* add inject_diverse feat
* auto lint
* capable to None-select
* feat: add hypothesis_gen config and ExpGen2TraceAndMerge functionality
* refactor: use dynamic import for experiment generator instantiation
* feat: add BestValidSelector for improved SOTA experiment selection
* runnable twin-trace version
* fix logic error of trace-merge
* auto lint
* use import_class to set selector,
* auto-lint
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-05-09 15:38:25 +08:00
fa53473b6c
fix: typo in workflow ( #861 )
2025-05-09 13:29:36 +08:00
1fd8abce0f
feat: log api status to mlflow ( #860 )
...
* log api status to mlflow to frontend tracking
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-09 12:20:49 +08:00
5fecefb802
make timeout fail a hyper parameter ( #859 )
...
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-08 21:30:44 +08:00
6fbe213f3d
change method of cut log trace ( #858 )
2025-05-08 21:07:33 +08:00
24c4eb7225
fix: refine the time/memory constraints prompt in hypothesis proposal ( #856 )
...
* refine prompt
* refine the wording
* add ratelimit retry to align with the suggested wait seconds
* add max retry to 0
* don't delete hist
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-08 21:05:07 +08:00
9df59eb7f0
add cache for summary, remove workspace dependency when generate summary ( #854 )
2025-05-08 12:24:48 +08:00
e1d08cfe73
fix: adjust ds_trace lookup and add stderr redirect to mlebench command ( #853 )
...
* fix: adjust ds_trace lookup and add stderr redirect to mlebench command
* style: reformat SOTA experiment lookup in ds_trace.py
* feat: add DS_RD_SETTING pipeline to MergeExpGen success message
2025-05-08 02:40:49 +08:00
45b260ba37
fix: trace list but ( #852 )
2025-05-07 21:28:24 +08:00
8ee4c72bab
add aide.py ( #851 )
2025-05-07 19:59:40 +08:00
668ac16150
feat: revert draft stage into a soft decay in hypothesis selection ( #849 )
...
* revert drafting
* update hypothesis rank logic
* prioritize time constraint in task design
* refine trace_desc and feedback problem prompt
* refine experiment_and_feedback_list_after_init
* fix DSHypothesis default parameter and print logic
* refine the selection weight
* merge simple_trace and trace
* refine weight and prompt
* refine sample logic
* fix CI
* robust code
---------
Co-authored-by: WinstonLiyte <1957922024@qq.com >
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-05-07 19:05:50 +08:00
3b7bafae12
chore: dump test data ( #850 )
...
* debug path for model dump
* update prompt
2025-05-07 17:24:50 +08:00
cda656b4ca
fix: non-exist variable test_eval.py ( #847 )
2025-05-07 04:36:00 +08:00
62e5953a6a
fix: wrong variable test_eval.py ( #846 )
2025-05-07 04:17:57 +08:00
0047114a09
fix bug ( #845 )
2025-05-06 18:15:20 +08:00
46dec7b624
feat: custom data ( #810 )
...
* custom data
* fix: simplify competition check and log local description file
* no sample data
* feat: add test evaluation module with error handling support
* fix: update eval path to use eval_sub_dir and add valid_check TODO
* refactor: add MLETestEval check to conditionally run grading steps
* avoid blank stdout
* valid in testeval
* rename test.csv to avoid conflict
* Support Disabling sample submission
* refactoring
* fix: remove DS_KAGGLE_DATA and update prompt instructions
* add try for grade
* ignore submission
* fix: remove tee from eval command and warn about pipeline exit code detection
* optional to use raw description
* support old data
* add execution result to stdout
* add metric to raw description
* custom data explain
* add debug_path
* rst update
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-05-06 16:00:13 +08:00
eda4a2a807
chore: ui change ( #844 )
...
* ui change
* ui change
2025-05-06 11:34:21 +08:00
e8c39eb63d
feat: refine merge ( #842 )
...
* feat: add search_type param and customize success trial desc
* docs: simplify trial descriptions in share.yaml
* lint
2025-05-04 23:17:58 +08:00
fd98d29982
fix: adapting UI to mock trace ( #841 )
...
* fix: return first index if 'SOTA Exp Score (valid)' is empty
* feat: add get_state_data_range helper for loops and slider bounds
* fix: exclude batch embedding tag from log filtering
* feat: add feedback support in sota_experiment and adjust merge flow
* fix: use fb function for merging experiments
* style: remove extra whitespace and reformat code
2025-05-04 15:51:32 +08:00
5cba25505e
fix new draft bugs ( #840 )
2025-04-30 18:53:27 +08:00
acc15caf7b
feat: reanalyze competition info & pipeline coding evaluator prompt ( #837 )
...
* update coding evaluator prompt similar to feedback
* reanaylyzing competition description when three sonsecutive coding failures
* update reanalyzing competition implementation
* fix bug
* update prompts and reanalyze
* fix bugs
* ci issue
* improve some code
* fix CI
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-04-30 17:36:35 +08:00
5f3da39331
feat: add drafting pipeline ( #832 )
...
* init commit
* add drafting prompt
* complete the drafting
* remove scenario problems from proposal
* rename prompts_drafting.yaml
* fix bug
* fix DSHypothesis print bug
* add failed drafting exp to prompt
* fix small bug
* use get_task_information() for task design
* resolve all comments
* add problem_desc to pesudo hypothesis
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-04-30 16:25:32 +08:00
ec51bb94b6
feat: trace merging ( #836 )
...
* feat: runnalbe -- add exp_gen_cls param, get_leaves and merge exp gen functionalities
* fix: remove unused scenario_desc and update YAML task labels
* feat: override selection and update merge task description
* lint
* lint
* lint
* lint
* lint
* fix: log competition setting to enable mle_summary
* fix name error
2025-04-29 09:30:45 +08:00
7cbf5bf56c
when restart with kb and different pkl path, we need to initialize the kb from json file ( #835 )
2025-04-28 15:51:51 +08:00
6be059dd5d
chore: log cost object ( #829 )
2025-04-25 19:26:08 +08:00
686437d671
add pipeline to hypothesis spec ( #828 )
2025-04-25 18:29:14 +08:00
50d3b7ce61
feat: propose hypothesis across multiple parts in pipeline ( #827 )
...
* refine hypothesis for pipeline
* refine component selection prompt
* update feedback prompt
* remove feat_eng in pipeline coding
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-04-25 17:17:53 +08:00
24c4c6755e
fix: add time to timer when api timeout bug ( #826 )
...
* If not use the session stored timer, need to replace it with the default timer
* when api timeout, add the waiting time to timer
2025-04-25 10:50:27 +08:00
2b821884e7
If not use the session stored timer, need to replace it with the default timer ( #825 )
2025-04-25 00:39:31 +08:00
7f90e44798
fix: fix a bug in docker result extraction ( #824 )
...
* fix a bug in docker result extraction
2025-04-24 19:02:38 +08:00
f8af5c0682
can set reasoning_effor=None in chat_model_map ( #823 )
2025-04-24 18:47:24 +08:00
f909b1b6bc
feat: using different chat model in different part ( #822 )
...
* using model in the chat_model_map in one tag
* add replace timer to DS loop
* fix CI
* fix CI
* add more custom config in chat_model_map
* fix CI
* fix CI
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-04-24 18:22:51 +08:00
44ccee864d
fix: fix model input shape bug and costeer_model bug ( #821 )
...
* fix model input shape bug and costeer_model bug
* fix a bug
2025-04-24 12:48:25 +08:00
0ce1ca74a0
fix: update runner max loop to 1 in DS scenario ( #820 )
2025-04-23 19:15:29 +08:00
18a115b4e0
fix: fix some minor bugs in qlib scenario ( #817 )
...
* fix some bugs
* fix a bug
* fix a bug in qlib frontend
* fix ci
* fic ci
* fix qlib Dockerfile
2025-04-23 18:48:02 +08:00
39e25d28a1
fix: improve eval alignment check (e.g. small-scale finetuning) ( #802 )
...
* fix
* fix
2025-04-22 18:47:24 +08:00
0f2399667b
feat: add mlflow logger in RD loop to log ( #815 )
...
* add mlflow logger in DS loop
* fix CI
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-04-22 01:23:40 +08:00
29b1cbc316
feat: archive python and csv files in workspace to maintain results ( #814 )
...
* archive workspace also
* remove non python csv and md files in workspace to avoid big workspace dump
* FIX ci
2025-04-21 16:34:29 +08:00
1c9b00474b
fix: align competion_full_desc and scenario_all_desc, remove redundant info in problems proposal ( #808 )
...
* align competition desc & scenario desc string
* remove competition_desc when having used scenario_desc in problem gen
* fix bug
* remove redundant competition desc in naive expgen
* improve proposal prompt
* modify phrase
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-04-18 16:48:26 +08:00
6f3e1a5b8a
add select lite and select best button ( #809 )
2025-04-18 16:28:31 +08:00
2b7122f41f
fix: bug fix in timer start ( #807 )
2025-04-18 16:16:28 +08:00
6646613359
fix: bug in problem identification ( #806 )
2025-04-18 15:46:11 +08:00
you-n-g