Commit Graph

711 Commits

Author SHA1 Message Date
XianBW f40c1b1df5 use enum value when generate hypothesis (#1091) 2025-07-18 16:22:34 +08:00
XianBW 88eed1474f fix ui bugs (#1090) 2025-07-18 16:01:39 +08:00
Xu Yang f99ceb4158 fix: minor fix to runtime_environment (#1089)
* minor fix to funtime_environment

* add python runtime info header
2025-07-17 19:58:06 +08:00
XianBW f113e88461 chore: add a simple chatbot in LLM_LOG window in DataScience UI (#1088)
* refine time show

* updates

* add chatbot in llm_log_win

* use newest litellm
2025-07-17 19:29:25 +08:00
Xu Yang e8c62ab46b feat: query & cache package_info (#1083)
* feat: add package query in draft.py (not yet enabled)

* feat: integrate package query into task_gen and cache runtime environment

- Remove pkg_query modifications from draft components
- Add package declaration requirement in task_gen prompts
- Add optional packages field to CodingSketch model
- Cache runtime_environment in scenario object for loop-wide reuse
- Parse packages from LLM response and generate runtime environment dynamically

* some refinement

* feat: merge default packages with CLI args in package_info.py

* fix: code style

---------

Co-authored-by: Qizheng Li <jenssenlee@163.com>
2025-07-17 18:35:36 +08:00
Tim 105b5791d9 chore: change default setting (#1087)
live_output = True
2025-07-17 17:19:10 +08:00
you-n-g ec1a5e7a88 fix: ignore class types when filtering workflow steps (#1085)
* fix: ignore class types when filtering workflow steps

* chore: add TODO for record method in RDLoop

* lint
2025-07-17 15:50:41 +08:00
amstrongzyf 32ecc61153 fix: refine the prompt to force complete code & refine the logic of running (#1069)
* change refine prompt for full code

* fix: fix the logic of running

* refine prompt

* fix some bugs

* fix

* add two guidelines

* refactor the code

* make costeer evaluator more logical

* refine eval prompt

* make costeer eval prompt markdown

* update code diff prompt

* correct pipeline

* feat: add apply_patch utility and update ret.py with patch functionality (#1071)

* restore to the right version

* fix the docstring

* fix extract_output fcn

* add inplace parameter to apply patch

* remove enable_runner_iteration and make the eval prompt same as main

* refine runner eval prompt based on main

* Update rdagent/scenarios/data_science/dev/runner/prompts.yaml

* add wait_retry

* refactor: move enable_runner_code_diff to DSRunnerCoSTEERSettings as diff_mode

* reformat and remove enable_runner_code_diff

---------

Co-authored-by: yuanteli <1957922024@qq.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Jensen Lee <91518020+Jensen246@users.noreply.github.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Qizheng Li <jenssenlee@163.com>
2025-07-17 11:59:27 +08:00
Xu Yang fd2c3b0c50 fix: minor conflict in prompts (#1081) 2025-07-16 16:10:56 +08:00
Xu Yang 9897739ad3 fix: align scenario descriptions and include debug timeout (#1079)
* Align scenario descriptions and include debug timeout

- Updated config.py to support debug timeout configuration
- Synchronized prompts in exp_gen and scen modules
- Refactored proposal.py for consistency with new scenario descriptions
- Improved __init__.py for better scenario management

* remove running time in stdout

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-16 14:30:17 +08:00
you-n-g 11c2298958 fix: refine prompt; runner focus on low hanging fruit (#1076)
* fix: refine prompts formatting and logic across data science scenarios

* Make runner evaluator more detailed

* feat: switch to system_debugger prompt and print early stopping stats
2025-07-16 12:29:57 +08:00
you-n-g 0f7f6d176b feat: add enable_cache toggle for UI data caching (#1075) 2025-07-15 22:26:06 +08:00
you-n-g ac55b8bfa1 fix: refine prompt, equal lightgbm, discourage over hypertuning (#1072)
* feat: add mount_path parameter to run command

* feat: add runtime environment info and dynamic timeouts to DS runners
2025-07-15 21:58:19 +08:00
XianBW 07407b770d chore: log LITELLM_SETTINGS when first use (#1074)
* log LITELLM_SETTINGS when first use

* fix ci
2025-07-15 18:41:18 +08:00
XianBW a6acd3d959 fix: filter log folders bug in ui (#1073)
* fix bug

* fix ci
2025-07-15 18:11:48 +08:00
you-n-g 3e6bb27dd1 feat: add ws CLI and support optional timeout/cache (#1066)
* feat: add ws CLI and support optional timeout/cache

* lint

* fix bugs

* convert extra_volumes to dict for multiprocess

* lint
2025-07-12 18:26:45 +08:00
Tim 4c78a601a5 fix: error in prompt template (#1065)
* fix prompt error

* fix prompt error
2025-07-12 09:32:46 +08:00
Yuante Li fa3846df31 refactor: try to convert the DS coder prompt into automaton logic (#1052)
* try to convert the DS coder prompt into automaton logic

* refine

* refine coder

* refine eval

* polish eval

* changes to pipeline_coder_system

* refine eval.py

* merge eval guidelines and steps

* fix a small bug

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: amstrongzyf <amstrongzyf@126.com>
2025-07-11 21:42:52 +08:00
XianBW 2e1d0624d2 refactor: move selector codes to an independent folder (#1059)
* refactor: move exp_gen selection files into select directory

* fix selector value in data_science conf

* fix selector template name

* fix CI

---------

Co-authored-by: Young <afe.young@gmail.com>
2025-07-11 17:42:40 +08:00
xuangu-fang be9490f417 fix: handle the no-update case of root node in uncommited_rec_status (#1062)
* fix: improve scheduler API (suggest_sel) and add timer.remain_time()

* chore: exclude .venv from auto-black and auto-isort tasks

* refactor: wrap RoundRobinScheduler commit and selection in retry loop

* set search_type="ancestors" for experiment_and_feedback_list_after_init

* fix bug:  no update of uncommited_rec_status with root node

---------

Co-authored-by: Young <afe.young@gmail.com>
2025-07-11 17:13:00 +08:00
amstrongzyf e8a45db801 fix: improve the logic of json_schema and refine the reasoning extraction logic for reasoning model (#1044)
* fix: fix a small bug in response_schema

* feat: support response_format parameter in chat completion

* fix: fix between json_mode and response_format

* Update base.py

* Update deprec.py

* add unittest and refine logic

* fix the reasoning extraction logic and refine prompt for deepseek adaptation

* refactor: introduce workflow_check and streamline task parsing

* refine prompt

---------

Co-authored-by: Young <afe.young@gmail.com>
2025-07-11 15:36:03 +08:00
XianBW bd23d1759e add data science ui & grade_summary to rdagent cli (#1057) 2025-07-11 15:30:24 +08:00
Xu Yang f0f5c221bd feat: add sample submission file check (#1053)
* check sample submission & add  package constraint

* add trace.log into clear

* change default

* simplify

* clear CI workspace before running

* move to CI

* use sudo to clean workspace

* move prepare out of global var

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-10 18:10:32 +08:00
Xu Yang 8d5fd9943d fix: path traversal risk (#1050)
* check the target file path is in a designed folder and make sure it's pdf

* Potential fix for code scanning alert no. 55: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 56: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* continue improving

* Potential fix for code scanning alert no. 62: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* apply secure_filename

* Potential fix for code scanning alert no. 64: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix CI

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-10 13:16:14 +08:00
you-n-g 369bd8bc78 feat: add only success filter toggle for traces (#1047) 2025-07-10 12:35:03 +08:00
Roland Minrui a03e0e81c9 fix draft bugs (#1048)
Co-authored-by: Xu <v-xuminrui@microsoft.com>
2025-07-10 11:47:23 +08:00
amstrongzyf 0424cd1baf fix: fix a small bug in response_schema (#1043)
* fix: fix a small bug in response_schema

* black fix
2025-07-09 18:40:48 +08:00
Roland Minrui e5260cb4ac feat: enable drafting with knowledge (#998)
* add pipeline for drafting v2

* fix the pipeline and add general knowledge

* debug

* fix bug

* fix bug

* change draft version1

* add function calling to task gen

* fix circular import bug

* change draft version3

* exp1_test

* feat: add DraftRouterExpGen and make summarizer configurable

* Update rdagent/scenarios/data_science/proposal/exp_gen/proposal.py

* change code structure

* stashed changes

* test

* test1

* revert conf.py

* add runtime enviornment info to general knowledge

* remove redundant code

* clean code

* remove files

* reformat

* fix bug

* fix bug

* simplify code

* fix minor bug

* fix bug and reformat

* revert config

* remove unused prompt

* add general knowledge

* fix ci

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: jingyuanlm <842442862@qq.com>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2025-07-09 18:36:42 +08:00
you-n-g faadc1312e fix: prevent parallelism in feedback and record steps (#1046) 2025-07-09 18:36:05 +08:00
Yuante Li 1459ec3bac fix: fix a bug in return curve display (#1042)
* fix a bug in return curve display

* fix ci
2025-07-09 15:53:03 +08:00
Xu Yang 21809f2a9d fix: split then sample & remove simple model guide in ds proposal (#1034)
* fix code timeout & split_then_sample

* change- code

* change-prompts_v2

* remove more simple guidance in proposal

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: jingyuanlm <842442862@qq.com>
2025-07-09 15:47:37 +08:00
Xu Yang f1d38d5750 fix: stop evolve if global timer is timeout (#1039) 2025-07-09 15:45:25 +08:00
amstrongzyf 7c41fce8b9 fix: fix a small bug in json_mode (#1041) 2025-07-09 14:33:24 +08:00
XianBW 5c7030964b chore: show llm filter calls info, remove redundant remove_timestamp param in Env (#1040)
* show llm calls info

* remove `remove_timestamp` param in Env

* use cache default
2025-07-09 14:20:33 +08:00
you-n-g ed92831712 fix: based on response schema; not function calling (#1038)
* fix: based on response schema; not function calling

* replace all
2025-07-08 21:11:52 +08:00
you-n-g 10c0d0c70a fix: properly assign sota_exp_fb before None comparison (#1037) 2025-07-08 20:31:21 +08:00
you-n-g 061554ef87 refactor: add string validation for feedback attrs and remove unused var (#1035) 2025-07-08 20:24:31 +08:00
XianBW 156549715d ui updates (#1033) 2025-07-08 18:20:27 +08:00
you-n-g 12cf0fb459 fix: scheduler next selection parallel disorder (#1028)
* fix: improve scheduler API (suggest_sel) and add timer.remain_time()

* chore: exclude .venv from auto-black and auto-isort tasks

* refactor: wrap RoundRobinScheduler commit and selection in retry loop

* set search_type="ancestors" for experiment_and_feedback_list_after_init

* refactor: merge sync_dag_parent_and_hist and hist.append into one call

* fix uncommited rec bug

* lint

---------

Co-authored-by: xuangu-fang <xuangufang@gmail.com>
2025-07-08 17:23:05 +08:00
Xu Yang af67910000 fix: remove refine decision & bug fix (#1031)
* remove refine decision & bug fix

* code simpler

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-08 16:34:25 +08:00
Roland Minrui 717f9b9e53 fix: add spec for hyperparameters in task design and coder (#995)
* init commit

* remove the 5-fold spec from prompts

* refine the hyperparameter specification

* do not sample data

* a small spelling issue

* refine prompt to avoid submission cheating

* do not sample data

* simplify code

* refine the coder evaluator prompt

* refine wording

* remove runtime from proposal

* refine wording

* refine prompt

* add gpu info in runtime_info.py

* modify the spec

* add router and add refinement exp gen

* fix prompt bug

* use rule-based logic for router

* complete the prompt

* fix circular import bug

* fix bug

* make refine_decision optional

* update pipeline prompts: (1) add scenary: in an iterative cooding loop and use sample datasets (2)add some generation tops in coding (3)add evaluation guidelines in evaluation (4)polish the json schema and description

* fix a small bug

* fix a small bug

* rdagent/scenarios/data_science/loop.py back to the original version

* refactor: replace _get_exp_gen with default_exp_gen for exp generation

* import

* refactor: make the __init__ back to main

* fix small bugs

* fix bugs for proposal_version

* move refine into runner

* check early stop

* EDA improvement & coder classes number

* fix CI

* slightly refine the prompt

* remove rule_base_eval and remove useless prompt

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: amstrongzyf <amstrongzyf@126.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
2025-07-08 15:22:39 +08:00
Tim ed086dcec4 feat: merge code summary and support more traces (#1025)
* merge with code_change_summary

* fix: prevent max_sota_retrieved_num_per_trace from becoming zero

* select sota from more than 2 traces
2025-07-08 10:00:52 +08:00
Yuante Li b05c2cba09 fix: fix some bugs in quant scen (#1026)
* fix some  bugs

* fix ci

* fix

* refine

* fix ci
2025-07-07 19:21:05 +08:00
you-n-g f71a005d55 refactor: privatize remain_time_duration and use remain_time() (#1027) 2025-07-07 18:52:17 +08:00
Tim a0015c1511 chore: sort trace by loop_id (#977)
* last as sota to submit

* chore: sort trace by loop_id

* Update rdagent/log/ui/utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* reformat

* update _log_path_hash_func

* compare sota_exp_to_submit

* fix ui storage

* move sota_exp_to_submit setting to record(from direct_exp_gen)

* save sota_exp_to_submit

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2025-07-07 17:08:55 +08:00
amstrongzyf ea86b537eb fix: support experimental support for Deepseek models and update docs about configuration (#1024)
* fix qlib running bug for deepseek and add configuration docs about deepseek
2025-07-07 14:18:56 +08:00
Tim 4bf16d03cf chore: restore parameter timeout in loop (#1022)
* restore parameter timeout in loop
2025-07-05 14:55:26 +08:00
you-n-g 6dbef13837 chore: document APIBackend parameters and remove sparse dep (#1019) 2025-07-04 20:31:06 +08:00
XianBW eff5dcc879 add trace DAG in trace, make exp show compatible with old versions (#1018) 2025-07-04 15:47:45 +08:00
Xu Yang 4608a0c63e feat: try coder on whole data (#1017)
* commit all code

* fix feedback bug

* add debug mode in ds

* update prompt

* prioritize performance than time lit

* use run instead

* store running time

* move all data running into running phase

* fix a bug

* use sample data as default

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-04 14:30:40 +08:00