* change refine prompt for full code
* fix: fix the logic of running
* refine prompt
* fix some bugs
* fix
* add two guidelines
* refactor the code
* make costeer evaluator more logical
* refine eval prompt
* make costeer eval prompt markdown
* update code diff prompt
* correct pipeline
* feat: add apply_patch utility and update ret.py with patch functionality (#1071)
* restore to the right version
* fix the docstring
* fix extract_output fcn
* add inplace parameter to apply patch
* remove enable_runner_iteration and make the eval prompt same as main
* refine runner eval prompt based on main
* Update rdagent/scenarios/data_science/dev/runner/prompts.yaml
* add wait_retry
* refactor: move enable_runner_code_diff to DSRunnerCoSTEERSettings as diff_mode
* reformat and remove enable_runner_code_diff
---------
Co-authored-by: yuanteli <1957922024@qq.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Jensen Lee <91518020+Jensen246@users.noreply.github.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Qizheng Li <jenssenlee@163.com>
* Align scenario descriptions and include debug timeout
- Updated config.py to support debug timeout configuration
- Synchronized prompts in exp_gen and scen modules
- Refactored proposal.py for consistency with new scenario descriptions
- Improved __init__.py for better scenario management
* remove running time in stdout
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* check sample submission & add package constraint
* add trace.log into clear
* change default
* simplify
* clear CI workspace before running
* move to CI
* use sudo to clean workspace
* move prepare out of global var
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* init commit
* remove the 5-fold spec from prompts
* refine the hyperparameter specification
* do not sample data
* a small spelling issue
* refine prompt to avoid submission cheating
* do not sample data
* simplify code
* refine the coder evaluator prompt
* refine wording
* remove runtime from proposal
* refine wording
* refine prompt
* add gpu info in runtime_info.py
* modify the spec
* add router and add refinement exp gen
* fix prompt bug
* use rule-based logic for router
* complete the prompt
* fix circular import bug
* fix bug
* make refine_decision optional
* update pipeline prompts: (1) add scenary: in an iterative cooding loop and use sample datasets (2)add some generation tops in coding (3)add evaluation guidelines in evaluation (4)polish the json schema and description
* fix a small bug
* fix a small bug
* rdagent/scenarios/data_science/loop.py back to the original version
* refactor: replace _get_exp_gen with default_exp_gen for exp generation
* import
* refactor: make the __init__ back to main
* fix small bugs
* fix bugs for proposal_version
* move refine into runner
* check early stop
* EDA improvement & coder classes number
* fix CI
* slightly refine the prompt
* remove rule_base_eval and remove useless prompt
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: amstrongzyf <amstrongzyf@126.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
* docs: document extra_volumes dict format in DockerConf
* feat: accept dict values in extra_volumes to specify bind and mode
* fix: skip invalid PDF reports to prevent infinite loop
* from break to raise self.LoopTerminationError
* format with black
---------
Co-authored-by: Young <afe.young@gmail.com>
* start to work on multi-trace + async
* init ver of async-multi-tarce, to test
* add eng-ver log
* complete version of async+ mul-trace
* debug
* fix bug on DS_RD_SETTING.get()
* update
* fix bug + simplif the usage of async in multi-trace
* fix mini bug of arg_name
* Move local_selection into class Experiment & clean the code
* feat: replace hard-coded cache paths with dynamic cache_path config
* style: reorder wait_retry import and format chmod list
* refactor: pass workspace_path to chmod command and use DockerConf check
* refactor: split workflow into pkg, add WorkflowTracker & wait_retry
* feat: add async LoopBase with parallel workers and step semaphores
* fix: replace pickle with dill and run blocking tasks via joblib wrapper
* feat: add log format settings, dynamic parallelism & pickle-based snapshot
* fix: default step semaphore to 1 and avoid subprocess when single worker
* merge bowen's changes
* merge tim's changes
* refactor: extract component task mapping, add conditional logger setup
* lint
* refactor: add type hints and safer remain_time metric logging in workflow
* lint
* fix: allow BadRequestError to be pickled via custom copyreg reducer
* fix: stop loop when LoopTerminationError is raised in LoopBase
* lint
* refactor: make log tag context-local using ContextVar for thread safety
* feat: add subproc_step flag and helper to decide subprocess execution
* fix: use ./cache path and normalize relative volume bind paths
* fix: reset loop_idx to 0 on loop restart/resume to ensure correct flow
* fix: avoid chmod on cache and input dirs in Env timeout wrapper
* fix: skip chmod on 'cache' and 'input' dirs using find -prune
* fix: restrict chmod to immediate mount dirs excluding cache/input
* fix: chmod cache and input dirs alongside their contents after entry run
* fix: guard chmod with directory checks for cache and input
* fix: prefix mount_path in chmod command for cache/input dirs
* fix: drop quotes from find exclude patterns to ensure chmod executes
* fix: skip chmod on cache/input directories to avoid warning spam
* feat: support string volume mappings and poll subprocess stdout/stderr
* support remove symbolic link
* test: use dynamic home path and code volume in LocalEnv local_simple
* fix: skip trace and progress update when loop step is withdrawn
* refactor: add clean_workspace util and non-destructive workspace backup
* fix: preserve symlinks when backing up workspace with copytree
* fix: prevent AttributeError when _pbar not yet initialized in LoopBase
* perf: replace shutil.copytree with rsync for faster workspace backup
* fix: cast log directory Path to str in tar command of data science loop
* fix: use portable 'cp -r -P' instead of rsync for workspace backup
* fix: add retry and logging to workspace backup for robustness
* refactor: extract backup_folder helper and reuse in DataScienceRDLoop
* fix: propagate backup errors & default _pbar getattr to avoid error
* fix the division by zero bug
* refactor: execute RD loops via asyncio.run and add necessary imports
* lint
* lint
* lint
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
* chore: avoid incorporate changes
best as sota
merge hypothesis
fix: max_retrieve_num after decision
chore: select last experiments and feedbacks
* add the set_current_selection before the exp_gen when merging
add trace.NEW_ROOT
fix: no scen_prob_multiplier
fix: use regex with timeout
chore: hypothesis_rank with selected_idx
chore: define is_parent in proposal
chore: rename collect_all_ancestors to get_parent_exps
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* fix model input shape bug and costeer_model bug
* fix a bug
* fix a bug in docker result extraction
* a system-level optimization
* add a filter of stdout
* update
* add stdout to model
* model training_hyperparameters update
* quant scenario
* update some quant settings
* llm choose action
* Thompson Sampling Bandit for action choosing
* refine both scens
* add trace messages for quant scen
* fix some bugs
* fix some bugs
* update
* update
* update
* fix
* fix
* fix
* update for merge
* fix ci
* fix some bugs
* fix ci
* fix ci
* fix ci
* fix ci
* refactor
* default qlib4rdagent local env downloading
* fix ci
* fix ci
* fix a bug
* fix ci
* fix: align all prompts on template (#908)
* use template to render all prompts
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* add fin_quant in cli
* fix a bug
* fix ci
* fix some bugs
* refactor
* remove the columns in hypothesis if no value generated in this column
* fix a bug
* fix ci
* fix conda env
* add qlib gitignore
* remove existed qlib folder & install torch in qlib conda
* fix workspace ui in feedback
* align model config in coder and runner in docker or conda
* fix CI
* fix CI
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* custom data
* fix: simplify competition check and log local description file
* no sample data
* feat: add test evaluation module with error handling support
* fix: update eval path to use eval_sub_dir and add valid_check TODO
* refactor: add MLETestEval check to conditionally run grading steps
* avoid blank stdout
* valid in testeval
* rename test.csv to avoid conflict
* Support Disabling sample submission
* refactoring
* fix: remove DS_KAGGLE_DATA and update prompt instructions
* add try for grade
* ignore submission
* fix: remove tee from eval command and warn about pipeline exit code detection
* optional to use raw description
* support old data
* add execution result to stdout
* add metric to raw description
* custom data explain
* add debug_path
* rst update
---------
Co-authored-by: Young <afe.young@gmail.com>
* using model in the chat_model_map in one tag
* add replace timer to DS loop
* fix CI
* fix CI
* add more custom config in chat_model_map
* fix CI
* fix CI
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* update all code
* update all code
* dump knowledge base
* rename the tag
* add timer to RD-Agent
* fix CI
* fix CI
* use batch embedding
* fix a small bug
* fix prompt bug
* feat: add pause resume to handle K8S cluster pause (#804)
* add resume to cluster running
* fix non-pickle problem
* fix a small bug
* fix a small bug
* avoid shutil move error
* refine the logic
* move knowledge base out of session
* avoid mistake information to pipeline coding
* avoid load and dump in steps
* archive the right folder
* small improvement
* avoid restart when timer is already started
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
* refactor: use dynamic input path and update template loader
* fix: update include syntax for data source in prompts.yaml
* add customization path
* docs: update prompts for ensemble scoring and metric direction
* chore: remove obsolete data_science/share.yaml file
Roland Minrui