07c228ef9f
feat: add user interaction in data science scenario ( #1251 )
...
* feat: add interactor classes and user interaction handling for experiments
* update code
* use fragment retry mechanism instead of rerun()
* fix a bug
* integrate user instructions into proposal and coder
* fix CI
* fix CI
* feat: add approval option for user instructions submission
* feat: enhance user instructions handling in Task and DSExperiment classes
* fix CI
* add user instructions into hypothesis rewrite
* add interface to command line
---------
Co-authored-by: Bowen Xian <xianbowen@outlook.com >
2025-09-18 11:10:12 +08:00
121b2dbe48
chore: valid selector update ( #1248 )
...
* chore: add medal info
* return sota_exp_stat
* update hit check
* update experiment
* udpate experiment
* extract log
* remove old log folder
* update candidates
* keep highest score
* early stop if no medal candidate
2025-09-17 15:48:16 +08:00
e78398996c
fix: add json format response fallback to prompt templates ( #1246 )
...
* fix: add json format response fallback to prompt templates
* more json prompt
2025-09-15 23:27:24 +08:00
4de999c4a3
fix: revert 2 commits ( #1239 )
...
* revert commit:c51a3008f897416a7416f92d1286cbbf6a2291ae. PR 1179
* Revert "fix: change runner prompts (#1223 )"
This reverts commit db2ebc27d70663d6aa85e0b05d9b81f5f6443c17.
* fix ruff check error
* fix: change switch place for fix_seed_and_data_split
2025-09-11 16:14:48 +08:00
262081b2a5
feat: add stdout into workspace for easier debugging ( #1236 )
...
* refactor: use get_truncated_stdout for consistent stdout handling across modules
* lint
* feat: add dump_stdout_type to DSRunnerCoSTEERSettings and use in eval
* fix: avoid circular import by moving DSRunnerEvaluator import inside method
2025-09-10 10:02:49 +08:00
33fde3a0bb
feat: offline selector ( #1231 )
...
* offline selector test
* fix score with tensor
* sort sota list
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-09-09 11:56:42 +08:00
337389bd3f
fix: add a switch for ensemble_time_upper_bound and fix some bug in main ( #1226 )
...
* change runner prompts
* v1
* ensemble_time_upper_bound
* lint
* fix inf bug in function prob_dis_torch() and ensemble prompts
* lint
* lint
---------
Co-authored-by: amstrongzyf <amstrongzyf@126.com >
2025-09-08 16:35:42 +08:00
2e09996154
fix: increase retry count in hypothesis_gen decorator to 10 ( #1230 )
2025-09-08 12:23:58 +08:00
956df9906c
chore: make sure token size below limit ( #1225 )
...
* chore: make sure token size below limit
* refactor: refactor and add import
---------
Co-authored-by: amstrongzyf <amstrongzyf@126.com >
2025-09-04 15:07:42 +08:00
2eaa69536c
fix: change runner prompts ( #1223 )
...
* change runner prompts
* v1
2025-09-04 11:31:42 +08:00
71689b2cac
fix: revert to v10 setting ( #1220 )
...
* feat: add runner_patience
* task gen prompts and torch version
* fix ensemble time bug
* small bug
* lint
* add torch
* fiix: remove useless code and change formula
* fix: rename parameter
---------
Co-authored-by: jingyuanlm <842442862@qq.com >
2025-09-03 16:37:33 +08:00
c8830f3f3d
fix: jinja problem of enumerate ( #1216 )
2025-09-01 14:50:26 +08:00
88a1b08b8c
for ui, a simple fix ( #1215 )
2025-09-01 14:46:51 +08:00
f0b7fa2466
fix: allow prev_out keys to be None in workspace cleanup assertion ( #1214 )
...
* fix: allow prev_out keys to be None in workspace cleanup assertion
* fix: clean workspace only for non-None DSExperiment instances
2025-08-30 07:27:12 +08:00
19eb7ae138
fix: add missing self parameter to instance methods in DSProposalV2ExpGen ( #1213 )
2025-08-30 00:45:55 +08:00
fbc1beb927
fix: update fallback criterion ( #1210 )
...
* fix: update fallback criterion
* fix: ensure evo_fb is initialized and used correctly in fallback logic
* refactor: rename use_new_evo to should_use_new_evo for clarity
2025-08-29 17:07:41 +08:00
1b15b53c1a
feat: add option to enable hyperparameter tuning only in first eval loop ( #1211 )
...
* feat: add option to enable hyperparameter tuning only in first eval loop
* fix: use total_seconds() for accurate time calculations in evolution and tracking
2025-08-29 16:59:22 +08:00
da8f61f874
fix: fix bug for hypo_select_with_llm when not support response_schema ( #1208 )
2025-08-28 14:36:23 +08:00
40f26ac54d
feat: enable LLM‑based hypothesis selection with time‑aware prompt & colored logging ( #1122 )
...
* add hypo select by llm (without time)
* add time_info and log color
* select no smooth
* 2 hypo
* change select
* small change
* fix bug
* fix bug and add hypothesis router and begin flag
* fix bug v1
* fix bug v2
* fix feedback
* add new model
* add filter
* fix bug v3
* fix bug v4
* change prompts v2
* fix bug v5
* fix bug v6
* fix hypo
* fix some bug(sota socre, prompts, ensemble prompts ) and add path legth.
* fix bug v7
* fix bug v8
* fix bug v9
* fix bug v10
* reset to v10 and refine
* fix: translate to english
* fix bug
* fix: use differnet increase_stage for coder & runner
* fix: use different timeout_increase_stage for coder and runner.
* fix: revert logger
* fix: remove duplicate content
* feat: implement LLM-driven extra hypothesis selection and adjust logic
* refactor: relocate Hypothesis models below TraceChallenges
* remove torch fix bug
* fix small bug
* refactor: prefix internal methods with underscore for llm-based hypothesis selection
* add fix_seed_and_data_split and enable_simple_hypothesis
* lint
* refine proposal config order
* code review; comments
* lint
* merge int to float
---------
Co-authored-by: jingyuanlm <842442862@qq.com >
Co-authored-by: Young <afe.young@gmail.com >
2025-08-27 18:36:51 +08:00
b4e11f1895
style: convert CLI flags from snake_case to kebab-case ( #1198 )
...
Co-authored-by: Young <afe.young@gmail.com >
2025-08-25 20:07:01 +08:00
3b57ea2666
fix: increase time default not controlled by LLM ( #1196 )
...
* fix: add longer timeout configuration for data science scenarios
* fix CI
2025-08-21 11:27:17 +08:00
2f54199b44
fix: kaggle competition metric direction ( #1195 )
...
* fix leaderboard and competition direction
* ui fix
2025-08-20 16:40:53 +08:00
ceabfe9686
feat: enable to inject diversity cross async multi-trace ( #1173 )
...
* init multi_trace_async_ diversity inject
* lint
* add task in context for diversity injection, add abstract class for diversity injection
* lint
* feat: update diversity injection strategy and enhance sibling context handling in experiment generation
* add always inject
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-08-18 12:43:21 +08:00
00b588ffd4
fix: skip res_ratio check if timer or res_time is None ( #1189 )
2025-08-17 16:08:55 +08:00
0dae88e2cd
chore: add inference mode for model dump ( #1182 )
...
* chore: add inference mode for model dump
* check runtime environment
* update opened_trace_lines
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-08-15 15:34:49 +08:00
5b785346f7
fix: refine prompts and add additional package info ( #1179 )
...
* refine prompts and add additional package info
* refine prompts to be specific for GBDT models
* minor refine prompts
* use include to replace duplicate info
* refine prompts
* refactor: import DSTrace from base and remove exp_gen __init__
* lint
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-08-13 23:00:23 +08:00
b94414a8ed
feat: refine the logic of enabling hyperparameter tuning and add criteira ( #1175 )
...
* add 4 tuning criteria
* fix equation direction
* fix ci
* add comment
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-08-12 21:12:54 +08:00
3fe2ba44df
feat: streamline hyperparameter tuning checks and update evaluation g… ( #1167 )
...
* feat: streamline hyperparameter tuning checks and update evaluation guidelines
* fix task_gen json check
2025-08-08 13:49:01 +08:00
2764da68a3
feat: improve fallback handling in CoSTEER and add GPU usage guidelin… ( #1165 )
...
* feat: improve fallback handling in CoSTEER and add GPU usage guideline in share.yaml
* update prompt
* update
2025-08-07 15:52:55 +08:00
6dab8fd7dd
feat: add reasoning attribute to DSRunnerFeedback for enhanced evaluation context ( #1162 )
2025-08-07 00:50:08 +08:00
16aadb7a0d
fix: refine DSCoSTEER_eval prompts ( #1157 )
...
* change DSCoSTEER_eval prompts
* fallback to better exp only
* fix fallback
* fix and reformat
* fix bug when base_fb is None
* add reasoning to hyperparameter evaluation
* feat: add acceptable assessment in exp_feedback (#1159 )
* add time
* refine eval prompt and make the logic of tuning check more clear
* some refinement
* fix CI
* fix a small bug, only consider score in runner
* refine comment
* simplify compare function
---------
Co-authored-by: jingyuanlm <842442862@qq.com >
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Jensen Lee <91518020+Jensen246@users.noreply.github.com >
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-08-06 22:55:25 +08:00
acfbd1b097
fix: ignore case when checking metric name ( #1160 )
...
* fix: ignore case when checking metric name
* add case-sensitive to prompts
---------
Co-authored-by: amstrongzyf <amstrongzyf@126.com >
2025-08-06 20:54:08 +08:00
3244ba6833
feat: prob-based trace scheduler ( #1131 )
...
* draft prob-based trace scheduler
* refactor ProbabilisticScheduler
* auto lint
* keep random, Sota-based, length-based trace schedluer
* lint
* example
* example
* refactor
* add inverse option for sota Scheduler
* add trace_Scheduler in conf
* lint
* add scheduler_temperature
---------
Co-authored-by: Young <afe.young@gmail.com >
2025-08-06 16:42:51 +08:00
f5d17a1508
feat: enable finetune llm ( #1055 )
...
* feat: start with previous workspace
* feat: finetune llm
* add PrevModelLoadEvaluator
---------
Co-authored-by: Young <afe.young@gmail.com >
Co-authored-by: v-jianwan <v-jianwan@microsoft.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-08-06 12:24:44 +08:00
0284b73596
feat: refactor CoSTEER classes to use DSCoSTEER and update max seconds handling ( #1156 )
...
* feat: refactor CoSTEER classes to use DSCoSTEER and update max seconds handling
* remove useless line
* enable time_ratio_limit_to_enable_hyperparameter_tuning
2025-08-05 18:16:10 +08:00
50916fa6c9
feat: create Jupyter notebook pipeline file based on main.py file ( #1134 )
...
* First commit
* isort
* black
* tweak prompt
* fix for argparse
* fix typo
* add e2e
* Add test files, clean
* fix black settings
* revert
* fix trailing
* remove extra
* comment
* small fix, updated prompt
* Fix argparse
* small improvements
* fix for merge
* fix for merge
2025-08-05 18:06:57 +08:00
067b5908f4
docs: update configuration docs ( #1155 )
...
* update configuration docs
* update configuration docs
* update configuration docs
2025-08-05 15:48:28 +08:00
84b7c30a90
fix: add metric in scores.csv and avoid reading sample_submission.csv ( #1152 )
...
* add scores.csv metric name in both task_gen and coder
* a little fix to column names
* small fix
* avoid sample submission read in task_gen
* avoid sample_submission reading in coding
* code change summary bug fix
* little update
* little refinement to eval
* refine coder and runner eval prompts
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-08-04 19:53:08 +08:00
5b541841cf
feat: enhance timeout handling in CoSTEER and DataScience scenarios ( #1150 )
...
* add prev loops to runner history
* fix evolving history
* fix bug on initializing feedback without final decision
* reformat
* refine
* add comments
* feat: enhance timeout handling in CoSTEER and DataScience scenarios
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-08-04 17:44:51 +08:00
f3cb3f8870
feat: add previous runner loops to runner history ( #1142 )
...
* add prev loops to runner history
* fix evolving history
* fix bug on initializing feedback without final decision
* reformat
* refine
* add comments
* fix ci
* a little refinement
* fix CI
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com >
Co-authored-by: Xu Yang <peteryang@vip.qq.com >
2025-08-04 17:38:27 +08:00
cf070a004e
save more time info ( #1151 )
2025-08-04 17:28:25 +08:00
88e346b6b5
change something for ui compatiblity ( #1149 )
2025-08-04 12:40:35 +08:00
4b444097b2
fix ( #1148 )
...
Co-authored-by: Xu <v-xuminrui@microsoft.com >
2025-08-02 18:36:31 +08:00
ea7483f5a7
feat: add show_hard_limit option and update time limit handling in DataScience settings ( #1144 )
2025-08-01 18:32:15 +08:00
8c454afd17
fix: fix some bugs in RD-Agent(Q) ( #1143 )
...
* fix some bugs in RD-Agent(Q)
* fix factor from report
* fix ci
2025-08-01 13:20:09 +08:00
684e41f43a
feat: fallback to acceptable results ( #1129 )
...
* refactor: add is_acceptable, fallback logic and generify evolving agent
* refine lint
* small
* lint
* lint
* lint
* feat: add is_acceptable to CoSTEERMultiFeedback
* feat: add in-memory workspace checkpoint and recovery
* feat: preserve symbolic links in workspace checkpoints and recovery
* lint
* lint
* feat: limit workspace checkpoint to files under 100KB
* feat: add workspace checkpoint size limit setting
* prompt
* lint
2025-07-31 17:53:18 +08:00
66659e9de2
fix: remove unused imports in data science scenario module ( #1136 )
2025-07-31 17:05:14 +08:00
e554b3a680
feat: add time ratio limit for hyperparameter tuning in Kaggle settin… ( #1135 )
...
* feat: add time ratio limit for hyperparameter tuning in Kaggle settings and update evaluator logic
* add recommend time limit
* remove 25% hard line
* small fix
2025-07-31 16:47:19 +08:00
1aa5cc2535
feat: enhance timeout management and knowledge base handling in CoSTEER components ( #1130 )
...
* feat: enhance timeout management and knowledge base handling in CoSTEER components
* fix a little bug
* fix small bug
* fix a small bug
* Update rdagent/scenarios/data_science/loop.py
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
* add scale check
* fix a small bug
* fix CI
* use dynamic chat_token_limit & remove repeated lines
* fix CI
* remove useless comment
* fix small bug
* update draft appendix
* fix prompt
* add code correctness as top priority
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2025-07-31 13:06:28 +08:00
dd2e06ade6
feat: add coder check and give more time ( #1127 )
...
* feat: introduce max_seconds_multiplier for timeout management across components
* avoid using assert in test
* hot fix a very small bug
* runner multiply twice
* revert feedback change
* add a switch to longer timeout
2025-07-29 14:59:38 +08:00
Xu Yang