Phase 1 — Infrastructure:
- RiskMgmt_RISK_PER_TRADE 0.5% → 1.5% (vbt_backtest.py)
- min_monthly_return_pct=15% acceptance filter (strategy_orchestrator)
- --min-monthly-return 15 CLI option (nexquant.py)
- {{ min_monthly_return }}% in strategy prompts
- MIN_MONTHLY_RETURN_PCT=15.0 in gen_strategies_real_bt + smart_strategy_gen
- realistic_backtest_all.py target_monthly 4→15%
Phase 2 — Factor quality:
- IC thresholds: prompt 0.05→0.08, bandit IC weight 0.10→0.20
- Explicite IC > 0.04 target in RAG prompt
- min_ic filters: data_loader 0.0→0.04, strategy_worker 0.02→0.04, ml_trainer 0.01→0.04
Architecture fix — Daily signal resampling:
- Factors have IC at daily resolution, but z-scores on 1-min collapse IC to ~0
- Resample factors to daily before strategy exec, ffill signal to 1-min for backtest
- Walk-forward IS years 3→1 (only 2 years of data available)
- Removed broken intersection() logic that destroyed 99.99% of 1-min data
- ffill stale propagation limited to 2880 bars (2 trading days)
- Fixed logger crash in _load_strategies
- Preflight: removed constant-signal check (false positive on random sandbox data)
- Tests: test_daily_signal_resampling.py (8 tests)
Non-negotiable rules: R1-R10 in AGENTS.md
- strategy_orchestrator.py: remove unreachable dead 'if not factor_values' after early return
- strategy_orchestrator.py: eliminate duplicate OHLVC load in evaluate_strategy
- env.py: escape single-quotes in Docker entry to prevent shell injection (CWE-78)
- env.py: replace mutable default args with None pattern in DockerEnv subclasses
- factor_runner.py: move pandarallel.initialize() from import-time to lazy init
- Path injection (B614): centralized safe_resolve_path in core/utils.py,
refactored 6 UI modules to use it with safe_root validation
- B701: added explicit autoescape=select_autoescape() to Jinja2
Environment() calls in 3 files
- B101: replaced assert statements with proper if/raise patterns in
12+ files (partial)
- B112: added logger.warning() to bare except:continue blocks in
5 files
- Summarize all but the 2 most recent experiments to compact bullet lines
(factor name, PASS/FAIL, IC value, 120-char observation snippet) instead
of including full verbatim traces; reduces prompt from ~121k to ~40-60k tokens
- Fix _evaluate_factor_directly and _save_factor_values to look for result.h5
and factor.py in sub_workspace_list instead of experiment_workspace
- Fix Series.to_parquet() → Series.to_frame().to_parquet() in _save_factor_values
- Update factor_data_template README: correct bars-per-day (1440, not 96)
- Update prompts to accept 2024-only debug dataset output as valid factor result
- Fix factor_coder prompts: allow 2024 debug data in date-range instruction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Daily factors (e.g. daily_log_return) carried same-day close data at 00:00,
giving the model end-of-day information at bar open — a classic look-ahead bias
that produced spurious IC=0.25 and Sharpe=24 with 98% win rate.
Changes:
- factor_runner.py: add _shift_daily_constant_factor_if_needed() that detects
factors where >90% of days have a single unique intraday value, then shifts
them by 1 trading day before IC computation
- prompts.yaml: add rule #7 instructing LLM to always shift(1) daily aggregates
before forward-filling to minute bars
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Modify factor_runner.py to save factor_code and factor_description
- Add _extract_factor_info() method to extract code from experiment
- Update _save_factor_json() to include code and description
- Now every backtest automatically saves to results/factors/ with:
* Full factor implementation code
* Extracted description (docstring or comments)
* IC, Sharpe, Win Rate, Max Drawdown metrics
This means the normal trading loop (rdagent fin_quant) now automatically
saves complete factor information to results/factors/ - same format as
predix_full_eval.py.
- Fix NameError: name 'os' is not defined
- This caused all 23 parallel runs to fail
- _ensure_results_dirs() uses os functions but import was missing
Tests should still pass
- Add predix_parallel.py: Run multiple factor experiments concurrently
* python predix_parallel.py --runs 5 --api-keys 2 -m openrouter
* Round-robin API key distribution across available keys
* Rich live dashboard with per-run status, elapsed time, exit codes
* Graceful shutdown (Ctrl+C kills all children cleanly)
- Add --run-id parameter to predix.py for isolated single runs
* Separate log files: fin_quant_run{N}.log
* Separate results: results/runs/run{N}/
* Separate workspace: RD-Agent_workspace_run{N}/
* Separate databases per run
- Modify CoSTEER and FactorRunner for PARALLEL_RUN_ID isolation
* _save_intermediate_results uses run-specific directories
* _save_result_to_database and _write_run_log isolated per run
* _ensure_results_dirs creates run-specific paths
- Reduce max_loop from 10 to 3 for faster iterations
- Add docs/parallel_runs.md with full documentation
Tests: 103 passed
- Fix daily/1min contradiction in factor_experiment_loader prompts
- Rename daily_pv.h5 to intraday_pv.h5 (generate.py, utils.py, README)
- Fix FactorDatetimeDailyEvaluator to accept 1min bars as correct
- Add _write_run_log() to log every factor attempt to results/logs/
- Add _ensure_results_dirs() to create all result directories
- Extract all 44 prompt YAML files to prompts/ centralized directory
- Add prompts/INDEX.md for navigation
Tests: 93 passed
- Remove duplicate DB save from quant.py (keep only in factor_runner)
- Add explicit DB path creation with mkdir -p
- Add JSON factor summaries to results/factors/
- Add debug logging for result structure
- Fix logger.debug -> logger.info (RDAgentLog compatibility)
- Update tests to match new architecture (240/240 passing)
- Enhance extract_results.py with progress indicators
* update rdagent cmd
* fix log error message
* use multiProcessing.Process instead of subprocess.Popen
* add traces to gitignore
* add user interactor in RDLoop (finance scenarios)
* add interactor (feedback, hypothesis) for quant scens
* fix the test_end in qlib conf
* add features init config, general instruction to qlib scenarios
* set base features for based exp
* fix bug when combine factors
* move traces folder to git_ignore_folder
* fix bug in features init
* fix quant interact bug
* fix logger warning error
* bug fixes
* modify rdagent logger, now it can set file output
* adjust cli functions and fix logger bug
* fix server port transport problem
* update server_ui in cli
* add web code
* fix CI problem
* black fix
* update web ui README
* update README
* update readme
* fix: prevent calendar index overflow when signal data ends early
* fix: make test_end optional to resolve Qlib backtest calendar misalignment
* fix: enhance GPU information output in get_gpu_info function
* fix: improve GPU information output in get_gpu_info function for better clarity
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
* refactor: unify qlib experiment configs, runners, and templates
* fix: use PropSetting instances instead of class attributes in qlib runners
* docs: add configurable train/valid/test time segments for fintech scenarios
* add custom data setting for the data science scene
* fix ci?
* fix ci
* add custom data as an example
* fix ci
* add package
* fix test_import ci error
* fix model input shape bug and costeer_model bug
* fix a bug
* fix a bug in docker result extraction
* a system-level optimization
* add a filter of stdout
* update
* add stdout to model
* model training_hyperparameters update
* quant scenario
* update some quant settings
* llm choose action
* Thompson Sampling Bandit for action choosing
* refine both scens
* add trace messages for quant scen
* fix some bugs
* fix some bugs
* update
* update
* update
* fix
* fix
* fix
* update for merge
* fix ci
* fix some bugs
* fix ci
* fix ci
* fix ci
* fix ci
* refactor
* default qlib4rdagent local env downloading
* fix ci
* fix ci
* fix a bug
* fix ci
* fix: align all prompts on template (#908)
* use template to render all prompts
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* add fin_quant in cli
* fix a bug
* fix ci
* fix some bugs
* refactor
* remove the columns in hypothesis if no value generated in this column
* fix a bug
* fix ci
* fix conda env
* add qlib gitignore
* remove existed qlib folder & install torch in qlib conda
* fix workspace ui in feedback
* align model config in coder and runner in docker or conda
* fix CI
* fix CI
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* move cache auto continue and retry to all api backend
* add type checker to json mode output
* fix CI
* feat: Add json_mode handling and streaming support in chat completion function
* lint
* fix a bug when returning a dict which value could contain int or bool
* remove litellm
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
* refactor: Update type annotations and remove unused class in evolving modules
* refactor: Simplify evolving agent and feedback handling in CoSTEER module
* lint & CI
* mypy
* ruff for core
* mypy
* refactor: remove unnecessary comments and update feedback handling logic
* refactor: Add prev_task_feedback parameter to evolving strategies
* feat: Clear folder before extracting zip file in DockerEnv
* fix: Correct retrieval of last experiment from history
* refine ds modal for more cases: eval and es
* update model template
* prompts for model and ensemble
* fix a bug
* fix a bug
* init: ds workflow evovingstrategy
* Adding ensemble (#505)
* Initial Draft
* Updating logic for init
* Revising
* Successful Testing
* Updating to use the latest & right class
* bug: bug-fixing for testing
* data science loop changes
* data science loop base
* ds loop feedback
* fix
* remove measure_time because it's duplicated (in LoopBase)
* add the knowledge query for data_loader & feature
* edit ds workflow evaluator
* data_loader bug fix
* stop evolving when all tasks completed
* llm app change
* fix break all complete strategy
* Adding queried knowledge (#508)
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
* fix loop bug
* ds workflow evaluator; test; refine prompts
* workflow spec
* fix ci
* feature task changes
* ds loop change
* fix a bug in feat
* add query knowledge for model and workflow
* llm_debug info(for show) using pickle instead of json
* remove NextLoopException
* loop change
* coder raise CoderError when all sub_tasks failed
* rename code_dict to file_dict in FBWorkspace
* add CoSTEER unittest
* now show self.version in Task.get_task_information(), simplify CoSTEER sub tasks definition
* remove some properties in ModelTask, add model_type in it.
* fix llm app bug
* llm web app bug fix
* ds loop bug fix
* fix: give component code to feature&ens eval
* loop catch error bug
* rename load_from_raw_data to load_data
* feat: Add debug data creation functionality for data science scenarios
* support local folder (#511)
* support local folder
* remove unnecessary random
* KaggleScen Subclass
* small fix
* use template for style description
* update default scen to kaggle
* update sample data script
* make sure frac < 1
* fix a bug
* feature spec changes
* fix
* changeimport order
* clear unnecessary std outputs
* fix a typo
* create sample folder after unzip kaggle data
* feature/model test script update
* Align the data types across modules.
* fix a bug in model eval
* show line number
* move sample entry point to app
* spec & model prompt changes
* Refine the competition specification to address the data type problem and the coherence issue.
* fix some bugs
* add file filter in FBworkspace.code property
* support non-binary prediction
* avoid too much warnings
* fix a bug in ensemble module
* filtered the knowledge query in all modules
* delete RAG in idea proposal
* refine the code in ensemble
* show exp workspace in llm_st
* exp_gen bug fix
* feedback bug fix
* use `feature` instead of `feat01`
* Trace & method of judging if exp is completed change
* fix a bug in package calling and execute ci
* fix code
* bug fix
* bug fix
* fix a bug
* fix some bugs
* fix a bug
* refactor: Enhance error handling and feedback in data science loop
* support different use_azure on chat and embedding models
* multi-model proposal logic
* fix a small syntax error
* loopBase and some changes
* ensemble scores change
* fbworkspace.code -> .all_codes
* use all model codes in workflow coder
* check scores.csv's keys(model_names)
* model name changes
* add a todo in ensemble test
* sota_exp changes
* give model info in exp gen
* add runner time limit
* config using debug data or not in evals
* exp to feedback base
* add feature code when writing model task
* small problem
* copying during sampling
* update
* refactor: Simplify code handling and improve workspace management
* model part output fix
* print model's execution time
* bug fix
* ensemble test fix
* ens small change
* ens_test bug fix
* Refine partial expansion logic to display only a few subfolders when their structure is uniform, improving readability in nested directories.
* several update on prompts
* sample subfolders
* Filter the stdout after code execution to remove irrelevant information e.g. progress bars, whitespace characters, excessive line breaks.
* Add some more prompts and comments
* several update on the first init rounds
* model timeout as error
* fix pattern of getting model codes in workspace
* small bux fix on model prompts
* remove get_code_with_key since we have regex pattern
* fix: Correct tqdm progress bar update logic in LoopBase class
* feat: Add diff generation and enhance feedback mechanism in data science loop
* update some fix to model and workflow prompts
* refine the logic of progress bar filter
* add last_successful_exp in exp_gen
* fix a one line bug
* add a hint in prompt
* fix data sample for bms
* fix data sample for bms
* hypothesis small fix
* crawler readme update
* fix component gen
* fix bug
* annotation change
* load description.md if it exists
* refactor: Simplify SOTA description handling in feedback and prompts
* refactor: Use shared templates for feedback and experiment descriptions
* change webapp for model codes changes
* update proposal
* add timeout message for docker run output
* fix
* refine the code in docker time processing
* use .shape instead of len() when do shape eval
* won't change size during iteration
* support bson sample
* sample support jsonl and bson
* add former_code to coder prompts
* a little speed us in debug data creating
* filter progress bar when eval ens and main
* avoid costeer makes no change to former code
* fix several log error
* add timeout judge threshold
* fix some bugs in the evaluation of component output shapes
* File structure for supporting litellm (#517)
Co-authored-by: Young <afe.young@gmail.com>
* ignore submission and show processing
* ignore submission and show processing
* add efficiency notice
* refactor: Enhance error message with detailed feedback summary
* refactor: Simplify component handling in DSExpGen class
* refactor: Update code structure and add docstring for clarity
* reserve one sample to each label in data sampling
* add Evaluation info
* refine costeer code to avoid giving same code twice
* use raw_description as plain text
* add a prompt hint to avoid same dict key
* model task name bug in first model exp gen
* fix a typo
* add some debug info in costeer tests
* task init change
* enhance data sampling
* refine the code in data_loader
* more reasonable loop
* fix a bug in data folder description
* add error msg & traceback to execution feedback
* fix llm error msg detection
* add task information to costeer eval & add cache to docker run(use zipfile to store the whole workspace)
* fix CI first round
* fix CI second round
* use txt to store test script to avoid pytest
* remove zipfile in requirements
* add azure.identity to requirements
* ignore debug web page
* component test changes
* remove redundent task_desc in model coder
* feat: Add APE module and prompts for automated prompt engineering
* fix: Update .gitignore and improve text formatting in eval.py
* refactor: Update print output and improve code comments and imports
* style: Fix string formatting and import order in ape.py and fmt.py
* exclude ape
* add a data folder notice
* reduce unnecessary output to stdout
* refine the code of describe_data_folder
* fix ci
* style: streamlit style update (#522)
* streamlit style update
* fix import
* fix format
* fix llm_st loop progress bar
* debugapp small change
* fix model str
* refine some prompts
* fix model str
* fix CI
* refine the logic associated with the data_folder
* fix ci
* small change
* set filter_progress_bar as default in execute
* model proposal with workflow
* add submission check in workflow eval
* fix bug
* small change
* fix CI
* fix CI
* refactor: Move generate_diff to utils and update DSExpGen logic
* more reasonable prompt describing metric direction
* fix a minor jinja2 bug
* quick fix exp_gen bugs
* fix the following bug
* fix
* fix some bugs
* remove workflow from model
* add pending_tasks_list in data science to enable coding model and workflow
* refine the code for handling JSON-formatted data descriptions
* assert with information
* ensure correct csv file name
* add logging to help record the output
* log competition
* add log tag for debug llm app
* test: Test ds refactor ll (#523)
* fix bugs to former scenario
* fix a bug because coding in rdloop changed
* fix the bug when feedback gets no hypothesis
* fix trace structure
* change all trace hist when merging hypothesis to experiments
* ignore some error in ruff
* fix kaggle scenario bugs
* refine one line
* another bug
* another small bug
* fix ui bugs
* chage kaggle train.py path
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
* fix CI
* Update rdagent/app/data_science/loop.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add samplecsv into spec prompts
* fix CI
---------
Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: yuanteli <1957922024@qq.com>
Co-authored-by: Xisen Wang <118058822+xisen-w@users.noreply.github.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: Tim <illking@foxmail.com>
Co-authored-by: 炼金术师华华 <37462254+YeewahChan@users.noreply.github.com>
Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Use ExtendedBaseSettings to replace BaseSettings
* update a more general way to pass the default setting
* update all code
* fix CI
* fix CI
* fix qlib scenario
* fix CI
* fix CI
* fix CI & add data science interfaces
* remove redundant code
* abandon costeer knowledge base v1
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
* simplify RDAgent conf
* add unified cacher(untested)
* fix small bugs
* fix a bug
* fix a small bug in runner
* use hash_key = None to skip cache
* fix CI
* in factor execution, ignore cache when raise exception
* add file locker to avoid mp calling
* fix CI
* use function __module__ name as folder in cache
* Init todo
* update all code
* update
* Extract factors from financial reports loop finished
* Fix two small bugs.
* Delete rdagent/app/qlib_rd_loop/run_script.sh
* Minor mod
* Delete rdagent/app/qlib_rd_loop/nohup.out
* Fix a small bug in file reading.
* some updates
* Update the detailed process and prompt of factor loop.
* Evaluation & dataset
* Optimize the prompt for generating hypotheses and feedback in the factor loop.
* Generate new data
* dataset generation
* Performed further optimizations on the factor loop and report extraction loop, added log handling for both processes, and implemented a screenshot feature for report extraction.
* Update rdagent/components/coder/factor_coder/CoSTEER/evaluators.py
* Update package.txt for fitz.
* add the result
* Performed further optimizations on the factor loop and report extraction loop, added log handling for both processes, and implemented a screenshot feature for report extraction. (#100) (#102)
- Performed further optimizations on the factor loop and report extraction loop.
- Added log handling for both processes.
- Implemented a screenshot feature for report extraction.
* Analysis
* Optimized log output.
* Factor update
* A draft of the "Quick Start" section for README
* Add scenario descriptions.
* Updates
* Adjust content
* Enable logging of backtesting in Qlib and store rich-text descriptions in Trace. Support one-step debugging for factor extraction.
* Reformat analysis.py
* CI fix
* Refactor
* remove useless code
* fix bugs (#111)
* Fix two small bugs.
* Fix a merge bug.
* Fix two small bugs.
* fix some bugs.
* Fix some format bugs.
* Restore a file.
* Fix a format bug.
* draft renew of evaluators
* fix a small bug.
* fix a small bug
* Support Factor Report Loop
* Update framework for extracting factors from research reports.
* Refactor report-based factor extraction and fix minor bugs.
* fix a small bug of log.
* change some prompts
* improve factor_runner
* fix a small bug
* change some prompts
* cancel some comments
* cancel some comments and fix some bugs
* fix some bugs in factor from reports loop
---------
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: Suhan Cui <51844791+SH-Src@users.noreply.github.com>
* Init todo
* update all code
* update
* Extract factors from financial reports loop finished
* Fix two small bugs.
* Delete rdagent/app/qlib_rd_loop/run_script.sh
* Minor mod
* Delete rdagent/app/qlib_rd_loop/nohup.out
* Fix a small bug in file reading.
* some updates
* Update the detailed process and prompt of factor loop.
* Evaluation & dataset
* Optimize the prompt for generating hypotheses and feedback in the factor loop.
* Generate new data
* dataset generation
* Performed further optimizations on the factor loop and report extraction loop, added log handling for both processes, and implemented a screenshot feature for report extraction.
* Update rdagent/components/coder/factor_coder/CoSTEER/evaluators.py
* Update package.txt for fitz.
* add the result
* Performed further optimizations on the factor loop and report extraction loop, added log handling for both processes, and implemented a screenshot feature for report extraction. (#100) (#102)
- Performed further optimizations on the factor loop and report extraction loop.
- Added log handling for both processes.
- Implemented a screenshot feature for report extraction.
* Analysis
* Optimized log output.
* Factor update
* A draft of the "Quick Start" section for README
* Add scenario descriptions.
* Updates
* Adjust content
* Enable logging of backtesting in Qlib and store rich-text descriptions in Trace. Support one-step debugging for factor extraction.
* Reformat analysis.py
* CI fix
* Refactor
* remove useless code
* fix bugs (#111)
* Fix two small bugs.
* Fix a merge bug.
* Fix two small bugs.
* fix some bugs.
* Fix some format bugs.
* Restore a file.
* Fix a format bug.
* draft renew of evaluators
* fix a small bug.
* fix a small bug
* Support Factor Report Loop
* Update framework for extracting factors from research reports.
* Refactor report-based factor extraction and fix minor bugs.
* fix a small bug of log.
* change some prompts
* improve factor_runner
* fix a small bug
* change some prompts
* cancel some comments
* cancel some comments and fix some bugs
---------
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: Suhan Cui <51844791+SH-Src@users.noreply.github.com>
* Init todo
* Evaluation & dataset
* Generate new data
* dataset generation
* add the result
* Analysis
* Factor update
* Updates
* Reformat analysis.py
* CI fix
* Further Optimised Model Workflow by Incorporating Feedbacks on Exp Task Card
* Rebasing To build the extraction & implementation demo
* Revised for clean code
* Revised further to show "Knowledge"
* Revised to make model_research_copilot better
* Further Optimised Model Workflow by Incorporating Feedbacks on Exp Task Card
* Rebasing To build the extraction & implementation demo
* Revised for clean code
* Revised further to show "Knowledge"
* Revised to make model_research_copilot better
---------
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com>
* ignore result csv file
* fix app scripts
* rename taskgenerator to developer and generate to develop
* fix a config bug in coder
* fix a small bug in factor coder evaluators
* remove a single logger in factor coder evaluators
* fix a small bug in model coder main.py
* rename Implementation to Workspace
* move the prepare the inject_code into FBWorkspace to align all the behavior
* fix a small bug in model feedback
* remove debug lines for multi processing and simplify evaluators multi proc
* add a copy function to workspace to freeze the workspace && add config prefix to speed up debugging
* make hypothesisgen a abc class
* use Qlib***Experiment
* fix a small bug
* rename Imp to Ws
* rename sub_implementations to sub_workspace_list
* fix a bug in feedback not presented as content in prompts
* move proposal pys to proposal folder
* reformat the folder
* align factor and model qlib workspace and use template to handle the workspace
* add a filter to evoagent to filter out false evo
* align multi_proc_n into RDAGENT seeting
* handle when runner gets empty experiment
* fix logger merge remaining problems
* fix black and isort automatically
TPTBusiness