- MLTrainer class with feature matrix builder from top factors by IC
- LightGBM training with time-series split (80/20) and early stopping
- Feature importance analysis (gain-based) with ranking
- Model persistence: model.txt + metadata.json + feature_importance.json + CSV
- Feedback generation for factor generation loop
- Model loading from disk
- Full pipeline: load factors -> train -> save -> generate feedback
- 46 unit tests covering all features
- lightgbm and scipy added to requirements.txt
Created rdagent/scenarios/qlib/local/data_loader.py:
- OHLCV loading with thread-safe caching
- Factor metadata loading (sorted by IC)
- Factor time-series loading with alignment
- Feature matrix builder
- Randomized factor selection for diverse strategies
- 11 tests passing
Note: data_loader.py is in local/ (closed source)
Test file is public to validate interface.
Problem:
- run_real_backtest generated script with 'FORWARD_BARS = {FORWARD_BARS}'
- FORWARD_BARS was defined in if/else block but not visible in f-string
- Caused 'NameError: name FORWARD_BARS is not defined' in subprocess
Fix:
- FORWARD_BARS now correctly resolved in f-string at script generation time
- Verified with unit test: FORWARD_BARS=96 correctly embedded in generated code
Also added:
- TRADING_STYLE env var support (swing/daytrading)
- Daytrading mode: 12-bar forward returns, RiskMgmt-compliant 10% max DD
- Swing mode: 96-bar forward returns, no DD limit
Complete live trading guide for cTrader + RiskMgmt integration:
- Architecture diagram and how it works (5 steps)
- Setup instructions and API configuration
- Usage examples (paper/live trading)
- Risk management and monitoring
- Troubleshooting guide
- Future enhancements roadmap
Documentation kept in QWEN.md only (internal, not public README).
- Professional PDF reports with all charts embedded
- Cover page with key metrics
- Performance metrics table
- Strategy dashboard PNG
- Individual charts: equity, drawdown, signals, monthly returns
- Factor correlation matrix
- Full factor list and strategy code
- Summary and disclaimer
- Auto-generated after each accepted strategy
9/9 strategies now have PDF reports (589KB each, 7 pages)
Problem:
- Backtest used 1-bar returns (pct_change().shift(-1))
- Factor IC was evaluated on 96-bar horizon
- Mismatch caused IC ≈ 0 for all LLM strategies
- Simple sign signal had IC=0.0005, Sharpe=0.08
Fix:
- Changed to 96-bar forward returns: pct_change(96).shift(-96)
- This matches the factor IC evaluation horizon
- Simple sign signal now: IC=0.1826, Sharpe=11.46, WinRate=56%
Also:
- Removed incorrect signal.shift(1) lag (signal already uses future data from factors)
- Fixed signal alignment to use 96-bar forward return index
Problem:
- Log output contained raw ANSI escape codes like [96m[0m
- This happened because LogColors always output color codes even when not in terminal
- Made logs unreadable when redirected to files or run in background
Fix:
- Added _should_use_colors() check in LogColors class
- Colors now disabled when NO_COLOR=1 env var is set
- Colors now disabled when stdout is not a TTY (background processes)
- All color codes (CYAN, GREEN, BLUE, etc.) become empty strings when disabled
Impact:
- Clean log output when running in background
- Colors still work in interactive terminal sessions
- NO_COLOR=1 can be used to force disable colors
Problem:
- Many parallel runs crashed with: AttributeError: 'QuantTrace' object has no attribute 'controller'
- controller was only initialized in increment_factor_count(), not in __init__
- Code in quant_proposal.py line 66-67 accesses trace.controller before increment_factor_count() is called
Fix:
- Move self.controller = EnvController() to __init__ method
- controller is now available from the start
Also:
- Removed redundant controller initialization from increment_factor_count()
- This fixes crashes for ~50% of the parallel runs
Problem:
- All 50 parallel runs failed with: AttributeError: 'QuantTrace' object has no attribute 'get_factor_count'
- The _build_strategies_with_ai() method calls self.trace.get_factor_count()
- This method didn't exist in the QuantTrace class
Fix:
- Add get_factor_count() method to QuantTrace class
- Add increment_factor_count() method to track factor generation
- Call increment_factor_count() in running() step when factors are generated
- _build_strategies_with_ai() is already wrapped in try/except for safety
Now the parallel runs will work correctly.
Problem:
- Portfolio selection failed with 'Not enough valid overlapping data'
- Factors produce mostly NaN values or fail execution
- No diagnostics shown to user about why factors failed
Solution:
- Add detailed error tracking for each factor (timeout, no result, low values)
- Show summary of skipped factors with reasons
- Increase timeout to 2 minutes per factor
- Add fallback to show top factors by IC if portfolio fails
- Better progress messages showing valid value counts
- Handle symlink failures by copying data file instead
- Minimum 1000 non-NaN values required for factor to be included
Usage:
predix portfolio # Select top 10 from top 50
predix portfolio -n 100 # Select from top 100 candidates
predix portfolio -c 0.5 # Allow higher correlation (0.5)
Problem:
- When Qlib Docker backtest failed (result is None), the experiment was marked as failed
- However, the 'feedback' step still tried to call self.factor_summarizer.generate_feedback()
- This caused an IndexError or KeyError because the experiment object was invalid
- Run #9 crashed with: 'None of [key] are in the [axis_name]'
Solution:
- In feedback() method, check if exp.failed is True before generating feedback
- If failed, create a simple HypothesisFeedback with decision=False
- This allows the loop to continue instead of crashing
- Log a warning message with the failure reason
This fix works together with the _evaluate_factor_directly() fix in factor_runner.py
to ensure that even when Docker fails, the loop continues smoothly.
Problem:
- When factor evaluation timed out (5 min), result was undefined
- save_single_result(result) crashed with NameError
- factor.factor_name[:40] could fail if factor_name wasn't a string
Fix:
- Initialize result = None before try block
- Set result to failed EvalResult on exception
- Only call save_single_result() if result is not None
- Use getattr(factor, 'factor_name', 'unknown') for safe access
- Convert to string before slicing [:40]
Now the evaluator continues even when individual factors timeout.
New CLI commands:
- predix top [-n 20] [-m ic|sharpe]: Show top factors by IC/Sharpe
- predix evaluate [--top N] [--all] [--force]: Evaluate factors
Why 100 new factors mostly failed:
- 56 factors have IC=None (factor values are all NaN or constant)
- LLM generates code that doesn't work with EURUSD 1min data
- Common issues: volume=0 causes division by zero, wrong MultiIndex handling
- Only 346 of 501 factors have valid IC values
Working factors (Top 5):
1. daily_close_open_mom IC=0.255
2. daily_ret_log_1d IC=0.255
3. daily_ret_close_1d IC=0.255
4. daily_close_to_close_ret IC=0.255
5. daily_ret_vol_adj_1d IC=0.235
Usage:
predix top # Show top 20 by IC
predix top -n 50 # Show top 50
predix top -m sharpe # Sort by Sharpe
predix evaluate --all # Evaluate all NEW factors
predix evaluate --force # Re-evaluate ALL
Integrated predix_full_eval.py into the main Predix CLI.
New command:
predix evaluate [--top N] [--all] [--parallel P] [--force]
Features:
- Evaluates factors with full 1min data (2020-2026)
- Computes IC, Sharpe, Max DD, Win Rate
- Automatically skips already evaluated factors
- --force flag to re-evaluate all factors
Problem:
- predix_full_eval.py re-evaluated ALL factors every time
- 382 factors were evaluated even though 164 already had results
- Wasted 8+ minutes of computation time
Solution:
- Add scan_factors(skip_evaluated=True) parameter
- Load existing results from results/factors/*.json
- Skip factors that already have status='success' and valid IC
- Add --force/-f flag to override and re-evaluate ALL factors
Usage:
python predix_full_eval.py --top 100 # Only NEW factors
python predix_full_eval.py --force # Re-evaluate ALL
python predix_full_eval.py --all # All NEW factors
Now the evaluator resumes from where it left off.
- Create prompts/local/factor_discovery_v3.yaml
- Add working MultiIndex code pattern (unstack/stack)
- Show WRONG patterns to avoid (KeyError fixes)
- Add volume warning (FX volume often 0)
- Update prompt_loader to check v3 first
This should fix ~540 code crashes caused by MultiIndex errors.
Also answers: What happens when fin_quant runs now?
1. LLM generates factor code using NEW v3 prompt (with examples)
2. Code is executed and validated
3. Qlib backtest runs in Docker
4. Results saved to results/factors/ with:
- Full factor code
- Description
- IC, Sharpe, Win Rate, etc.
5. Results saved to SQLite database
- Modify factor_runner.py to save factor_code and factor_description
- Add _extract_factor_info() method to extract code from experiment
- Update _save_factor_json() to include code and description
- Now every backtest automatically saves to results/factors/ with:
* Full factor implementation code
* Extracted description (docstring or comments)
* IC, Sharpe, Win Rate, Max Drawdown metrics
This means the normal trading loop (rdagent fin_quant) now automatically
saves complete factor information to results/factors/ - same format as
predix_full_eval.py.
- Add factor_code and factor_description to EvalResult
- Extract docstring or comments from factor code as description
- Enrich 232 existing factor files with code and description
Now each factor JSON in results/factors/ contains:
- factor_name, factor_code, factor_description
- IC, Rank IC, Sharpe, Win Rate, etc.
- Add save_single_result() function
- Call it after each factor evaluation (not just at end)
- Results now appear in results/factors/ in real-time
Usage:
python predix_full_eval.py --all --parallel 4
- Changed save location from results/backtests/ to results/factors/
- ALL successful factor results are now saved individually
- Safe filename handling (special chars removed)
Usage:
python predix_full_eval.py --all --parallel 4
- Fix NameError: name 'os' is not defined
- This caused all 23 parallel runs to fail
- _ensure_results_dirs() uses os functions but import was missing
Tests should still pass
- Fix API key index calculation to respect --api-keys parameter
- Previously always used all available keys regardless of setting
- Now correctly assigns only requested number of API keys
Usage:
python predix_parallel.py --runs 25 --api-keys 1 -m openrouter
-> All 25 runs use API Key 1 only
- Allow up to 50 runs (25 recommended max)
- Add --force flag to bypass warnings
- Show RAM estimate for high run counts
- Update CLI help text with max recommendation
Usage:
python predix_parallel.py --runs 25 -m openrouter
python predix_parallel.py --runs 50 --force -m openrouter
Tests: 103 passed
- Use Rich Group instead of string join for Table + Panel layout
- Fix TypeError: sequence item 0: expected str instance, Table found
- Live dashboard now renders correctly with parallel runs
Tests still passing (103)
- Add predix_parallel.py: Run multiple factor experiments concurrently
* python predix_parallel.py --runs 5 --api-keys 2 -m openrouter
* Round-robin API key distribution across available keys
* Rich live dashboard with per-run status, elapsed time, exit codes
* Graceful shutdown (Ctrl+C kills all children cleanly)
- Add --run-id parameter to predix.py for isolated single runs
* Separate log files: fin_quant_run{N}.log
* Separate results: results/runs/run{N}/
* Separate workspace: RD-Agent_workspace_run{N}/
* Separate databases per run
- Modify CoSTEER and FactorRunner for PARALLEL_RUN_ID isolation
* _save_intermediate_results uses run-specific directories
* _save_result_to_database and _write_run_log isolated per run
* _ensure_results_dirs creates run-specific paths
- Reduce max_loop from 10 to 3 for faster iterations
- Add docs/parallel_runs.md with full documentation
Tests: 103 passed
- TestCLIModelSelection: 8 tests for predix.py CLI
* predix module imports
* fin_quant --model option
* predix quant --model and --log-file options
* OpenRouter API key validation
* TeeWriter existence check
* health and status commands
- TestLoggingTeeWriter: 2 tests for TeeWriter
* Multi-stream writing
* Broken stream handling
All 103 integration tests pass (93 + 10 new).
Also fix predix.py logging:
- Add --log-file flag (default: fin_quant.log)
- TeeWriter writes to both console AND file
- Works for both local and openrouter backends
- Add --model/-m flag to select LLM backend
* local: llama.cpp on localhost (default)
* openrouter: Cloud models via OpenRouter API
- Create predix.py as new CLI entry point with model selection
- Add OPENROUTER_API_KEY and OPENROUTER_MODEL to .env
- Add health and status commands to CLI
- Update rdagent/app/cli.py with model selection logic
Usage:
predix quant # Local (default)
predix quant -m openrouter # OpenRouter cloud
predix quant -m local -d # Local + dashboard
Tests: 93 passed
- Fix daily/1min contradiction in factor_experiment_loader prompts
- Rename daily_pv.h5 to intraday_pv.h5 (generate.py, utils.py, README)
- Fix FactorDatetimeDailyEvaluator to accept 1min bars as correct
- Add _write_run_log() to log every factor attempt to results/logs/
- Add _ensure_results_dirs() to create all result directories
- Extract all 44 prompt YAML files to prompts/ centralized directory
- Add prompts/INDEX.md for navigation
Tests: 93 passed
- Remove duplicate DB save from quant.py (keep only in factor_runner)
- Add explicit DB path creation with mkdir -p
- Add JSON factor summaries to results/factors/
- Add debug logging for result structure
- Fix logger.debug -> logger.info (RDAgentLog compatibility)
- Update tests to match new architecture (240/240 passing)
- Enhance extract_results.py with progress indicators
TPTBusiness