Commit Graph

824 Commits

Author SHA1 Message Date
amstrongzyf dfd73cb619 docs: update installation methods in readme (#1058) 2025-07-11 16:26:16 +08:00
amstrongzyf e4d4ceafa2 fix: improve the logic of json_schema and refine the reasoning extraction logic for reasoning model (#1044)
* fix: fix a small bug in response_schema

* feat: support response_format parameter in chat completion

* fix: fix between json_mode and response_format

* Update base.py

* Update deprec.py

* add unittest and refine logic

* fix the reasoning extraction logic and refine prompt for deepseek adaptation

* refactor: introduce workflow_check and streamline task parsing

* refine prompt

---------

Co-authored-by: Young <afe.young@gmail.com>
2025-07-11 15:36:03 +08:00
XianBW f6a9c68e01 add data science ui & grade_summary to rdagent cli (#1057) 2025-07-11 15:30:24 +08:00
Xu Yang da67df19a4 feat: add sample submission file check (#1053)
* check sample submission & add  package constraint

* add trace.log into clear

* change default

* simplify

* clear CI workspace before running

* move to CI

* use sudo to clean workspace

* move prepare out of global var

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-10 18:10:32 +08:00
you-n-g 083f1bdafe docs: add submission grading steps to devcontainer README (#1054)
* docs: add submission grading steps to devcontainer README

* add change

---------

Co-authored-by: jingyuanlm <842442862@qq.com>
2025-07-10 16:48:25 +08:00
Xu Yang bd0a95efac fix: path traversal risk (#1050)
* check the target file path is in a designed folder and make sure it's pdf

* Potential fix for code scanning alert no. 55: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 56: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* continue improving

* Potential fix for code scanning alert no. 62: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* apply secure_filename

* Potential fix for code scanning alert no. 64: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix CI

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-10 13:16:14 +08:00
you-n-g 6fdf63370f feat: add only success filter toggle for traces (#1047) 2025-07-10 12:35:03 +08:00
Roland Minrui c998ec9ccf fix draft bugs (#1048)
Co-authored-by: Xu <v-xuminrui@microsoft.com>
2025-07-10 11:47:23 +08:00
amstrongzyf 53d7c3b6da fix: fix a small bug in response_schema (#1043)
* fix: fix a small bug in response_schema

* black fix
2025-07-09 18:40:48 +08:00
Roland Minrui 1c35004782 feat: enable drafting with knowledge (#998)
* add pipeline for drafting v2

* fix the pipeline and add general knowledge

* debug

* fix bug

* fix bug

* change draft version1

* add function calling to task gen

* fix circular import bug

* change draft version3

* exp1_test

* feat: add DraftRouterExpGen and make summarizer configurable

* Update rdagent/scenarios/data_science/proposal/exp_gen/proposal.py

* change code structure

* stashed changes

* test

* test1

* revert conf.py

* add runtime enviornment info to general knowledge

* remove redundant code

* clean code

* remove files

* reformat

* fix bug

* fix bug

* simplify code

* fix minor bug

* fix bug and reformat

* revert config

* remove unused prompt

* add general knowledge

* fix ci

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: jingyuanlm <842442862@qq.com>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2025-07-09 18:36:42 +08:00
you-n-g 12d4e8a850 fix: prevent parallelism in feedback and record steps (#1046) 2025-07-09 18:36:05 +08:00
Yuante Li c03a519cbc fix: fix a bug in return curve display (#1042)
* fix a bug in return curve display

* fix ci
2025-07-09 15:53:03 +08:00
Xu Yang ece86d7d51 fix: split then sample & remove simple model guide in ds proposal (#1034)
* fix code timeout & split_then_sample

* change- code

* change-prompts_v2

* remove more simple guidance in proposal

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: jingyuanlm <842442862@qq.com>
2025-07-09 15:47:37 +08:00
Xu Yang 3c2c27c9d3 fix: stop evolve if global timer is timeout (#1039) 2025-07-09 15:45:25 +08:00
amstrongzyf 09dd0ca3db fix: fix a small bug in json_mode (#1041) 2025-07-09 14:33:24 +08:00
XianBW e1d6dc2dec chore: show llm filter calls info, remove redundant remove_timestamp param in Env (#1040)
* show llm calls info

* remove `remove_timestamp` param in Env

* use cache default
2025-07-09 14:20:33 +08:00
Tim 7251976f55 fix: TypeError: cannot unpack non-iterable bool object (#1036) 2025-07-08 21:25:02 +08:00
you-n-g a09b09ff8e fix: based on response schema; not function calling (#1038)
* fix: based on response schema; not function calling

* replace all
2025-07-08 21:11:52 +08:00
you-n-g aa006a50ff fix: properly assign sota_exp_fb before None comparison (#1037) 2025-07-08 20:31:21 +08:00
you-n-g e0ba3b0eee refactor: add string validation for feedback attrs and remove unused var (#1035) 2025-07-08 20:24:31 +08:00
XianBW 3f9c1afea8 ui updates (#1033) 2025-07-08 18:20:27 +08:00
you-n-g f8d08bed50 fix: scheduler next selection parallel disorder (#1028)
* fix: improve scheduler API (suggest_sel) and add timer.remain_time()

* chore: exclude .venv from auto-black and auto-isort tasks

* refactor: wrap RoundRobinScheduler commit and selection in retry loop

* set search_type="ancestors" for experiment_and_feedback_list_after_init

* refactor: merge sync_dag_parent_and_hist and hist.append into one call

* fix uncommited rec bug

* lint

---------

Co-authored-by: xuangu-fang <xuangufang@gmail.com>
2025-07-08 17:23:05 +08:00
Xu Yang 35b4224a03 fix: remove refine decision & bug fix (#1031)
* remove refine decision & bug fix

* code simpler

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-08 16:34:25 +08:00
Roland Minrui 6f3a44c18a fix: add spec for hyperparameters in task design and coder (#995)
* init commit

* remove the 5-fold spec from prompts

* refine the hyperparameter specification

* do not sample data

* a small spelling issue

* refine prompt to avoid submission cheating

* do not sample data

* simplify code

* refine the coder evaluator prompt

* refine wording

* remove runtime from proposal

* refine wording

* refine prompt

* add gpu info in runtime_info.py

* modify the spec

* add router and add refinement exp gen

* fix prompt bug

* use rule-based logic for router

* complete the prompt

* fix circular import bug

* fix bug

* make refine_decision optional

* update pipeline prompts: (1) add scenary: in an iterative cooding loop and use sample datasets (2)add some generation tops in coding (3)add evaluation guidelines in evaluation (4)polish the json schema and description

* fix a small bug

* fix a small bug

* rdagent/scenarios/data_science/loop.py back to the original version

* refactor: replace _get_exp_gen with default_exp_gen for exp generation

* import

* refactor: make the __init__ back to main

* fix small bugs

* fix bugs for proposal_version

* move refine into runner

* check early stop

* EDA improvement & coder classes number

* fix CI

* slightly refine the prompt

* remove rule_base_eval and remove useless prompt

---------

Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: amstrongzyf <amstrongzyf@126.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
2025-07-08 15:22:39 +08:00
you-n-g 7299ab1f6a chore: release 0.7.0
Release-As: 0.7.0
2025-07-08 11:19:08 +08:00
Linlang e4be7f92a3 add dependence (#1029) 2025-07-08 11:12:44 +08:00
Tim ef5848df8b feat: merge code summary and support more traces (#1025)
* merge with code_change_summary

* fix: prevent max_sota_retrieved_num_per_trace from becoming zero

* select sota from more than 2 traces
2025-07-08 10:00:52 +08:00
Yuante Li f2cb3ce909 fix: fix some bugs in quant scen (#1026)
* fix some  bugs

* fix ci

* fix

* refine

* fix ci
2025-07-07 19:21:05 +08:00
you-n-g 508b75f884 refactor: privatize remain_time_duration and use remain_time() (#1027) 2025-07-07 18:52:17 +08:00
Tim f59e7bd486 chore: sort trace by loop_id (#977)
* last as sota to submit

* chore: sort trace by loop_id

* Update rdagent/log/ui/utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* reformat

* update _log_path_hash_func

* compare sota_exp_to_submit

* fix ui storage

* move sota_exp_to_submit setting to record(from direct_exp_gen)

* save sota_exp_to_submit

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2025-07-07 17:08:55 +08:00
amstrongzyf f2272f34c1 fix: support experimental support for Deepseek models and update docs about configuration (#1024)
* fix qlib running bug for deepseek and add configuration docs about deepseek
2025-07-07 14:18:56 +08:00
Tim 3655eb6260 chore: restore parameter timeout in loop (#1022)
* restore parameter timeout in loop
2025-07-05 14:55:26 +08:00
you-n-g e508a71170 chore: document APIBackend parameters and remove sparse dep (#1019) 2025-07-04 20:31:06 +08:00
XianBW c1e99c13a8 add trace DAG in trace, make exp show compatible with old versions (#1018) 2025-07-04 15:47:45 +08:00
Xu Yang d0afa599a7 feat: try coder on whole data (#1017)
* commit all code

* fix feedback bug

* add debug mode in ds

* update prompt

* prioritize performance than time lit

* use run instead

* store running time

* move all data running into running phase

* fix a bug

* use sample data as default

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-07-04 14:30:40 +08:00
XianBW c335ebc25c chore: ui server updates (#951)
* ui server update

* update

* fix bugs

* updates

* use randomname

* fix

* fix bugs

* change time interval in debug server

* some change

* fix CI

* time interval change

* updates

* some changes

* fix

* fix curves

* one IP, one pointers to show msgs return progress

* enable /control

* fix bugs

* fix CI

* fix isort
2025-07-04 12:32:14 +08:00
XianBW ef6b5866c7 feat: show first evo round codes diff (#1009)
* log settings

* add first diff in evolving

* save dict settings

* load settings in data science ui
2025-07-03 19:21:14 +08:00
you-n-g deb01ac5c1 docs: add README for exp_gen folder structure proposal (#1010)
* docs: add README for exp_gen folder structure proposal

* docs: update README with tool strategy and adjust folder structure

* docs: rename 'Tool strategy' section to 'Tools' in exp_gen README
2025-07-03 11:31:51 +08:00
you-n-g 856fbde807 refactor: replace run_ret_code with run and adjust helper methods (#1013) 2025-07-03 11:24:05 +08:00
Yuante Li fb8f2b561e fix: fix a minor bug in DS eval (#1012) 2025-07-02 23:31:29 +08:00
you-n-g d589b859d0 chore: add devcontainer Dockerfile, devcontainer.json, and env (#997)
* chore: add devcontainer Dockerfile, devcontainer.json, and env

* style: remove unnecessary comment on ENABLE_CACHE setting

* docs: add internal devcontainer README with setup instructions

* Update README in .devcontainer based on PR #997

---------

Co-authored-by: jingyuanlm <842442862@qq.com>
2025-07-02 18:54:49 +08:00
Tim 0077b2caa2 chore: sample data map (#1005)
* chore: map data sampler
2025-07-02 15:11:57 +08:00
Yuante Li 667af3e1ea feat: added running time statistics for the DS scenario experiment (#1007)
* added running time statistics for the DS scenario experiment

* update execute_ret_code to return running_time

* fix

* fix

* update describe

* add EnvResult

* update corresponding calls

* add RunningInfo class

* fix

* fix

* fix

* fix ci

* rename function name

* fix ci

* fix

* refine running_time logic

* fix ci
2025-07-02 15:11:18 +08:00
XianBW cbab4830c3 add code diff in evolving window (#1008) 2025-06-30 20:10:34 +08:00
you-n-g cd237292ea feat: add hide_base_name option and update data folder prompts (#1004)
* feat: add hide_base_name option and update data folder prompts

* rename folder
2025-06-29 21:22:16 +08:00
you-n-g 66e0e8ffd8 feat: add code change summary (#1000)
* feat: add code change summary and dict_get_with_warning util

* feat: support code_change_summary in feedback classes

* lint

* feat: validate response_format using BaseModel and warn unknown formats
2025-06-29 13:40:15 +08:00
amstrongzyf 6c007ee7fb refactor: convert data describe functions to class-based implementation and add output example (#999)
* feat: Enhance data folder description for clarity and robustness

* fix bug

* fix present bugs

* delete useless files

* add output example and refactor the hole util.py

* fix bug for file tree

* add corner case example

* delete useless file
2025-06-28 21:23:40 +08:00
you-n-g 5c2f015757 chore: release 0.6.1
Release-As: 0.6.1
2025-06-28 20:06:57 +08:00
Linlang e3278bfe84 fix: fix mount (#1001)
* docs: document extra_volumes dict format in DockerConf

* feat: accept dict values in extra_volumes to specify bind and mode

* fix: skip invalid PDF reports to prevent infinite loop

* from break to raise self.LoopTerminationError

* format with black

---------

Co-authored-by: Young <afe.young@gmail.com>
2025-06-28 20:01:14 +08:00
xuangu-fang cd0aab3c31 fix: handle the bug of wrong dag_parant index (#996)
* fix the dag_parant_index bug caused by the wrong calling order of

* auto-lint
2025-06-27 17:01:29 +08:00