- vbt_backtest: unify risk_* → riskmgmt_* field names in _apply_risk_mask
- results_db: case-insensitive column existence check
- test_ftmo_oos: update test assertions to match renamed fields
Phase 1 — Infrastructure:
- RiskMgmt_RISK_PER_TRADE 0.5% → 1.5% (vbt_backtest.py)
- min_monthly_return_pct=15% acceptance filter (strategy_orchestrator)
- --min-monthly-return 15 CLI option (nexquant.py)
- {{ min_monthly_return }}% in strategy prompts
- MIN_MONTHLY_RETURN_PCT=15.0 in gen_strategies_real_bt + smart_strategy_gen
- realistic_backtest_all.py target_monthly 4→15%
Phase 2 — Factor quality:
- IC thresholds: prompt 0.05→0.08, bandit IC weight 0.10→0.20
- Explicite IC > 0.04 target in RAG prompt
- min_ic filters: data_loader 0.0→0.04, strategy_worker 0.02→0.04, ml_trainer 0.01→0.04
Architecture fix — Daily signal resampling:
- Factors have IC at daily resolution, but z-scores on 1-min collapse IC to ~0
- Resample factors to daily before strategy exec, ffill signal to 1-min for backtest
- Walk-forward IS years 3→1 (only 2 years of data available)
- Removed broken intersection() logic that destroyed 99.99% of 1-min data
- ffill stale propagation limited to 2880 bars (2 trading days)
- Fixed logger crash in _load_strategies
- Preflight: removed constant-signal check (false positive on random sandbox data)
- Tests: test_daily_signal_resampling.py (8 tests)
Non-negotiable rules: R1-R10 in AGENTS.md
- Path injection (B614): centralized safe_resolve_path in core/utils.py,
refactored 6 UI modules to use it with safe_root validation
- B701: added explicit autoescape=select_autoescape() to Jinja2
Environment() calls in 3 files
- B101: replaced assert statements with proper if/raise patterns in
12+ files (partial)
- B112: added logger.warning() to bare except:continue blocks in
5 files
eval() on trainer stdout output replaced with ast.literal_eval() which only
parses Python literals and cannot execute arbitrary code.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- factor.py: check_output([python_bin, path]) instead of shell string
- env.py QlibCondaEnv: all four conda commands use list args
Shell=True with a constructed string allows shell injection if
python_bin or path contain shell metacharacters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The metric parameter was passed directly into an f-string SQL query.
Add explicit validation against _ALLOWED_METRICS before use, raising
ValueError on unknown values. Raises ValueError on injection attempt
instead of silently accepting arbitrary column names.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace f-string SQL queries with whitelist validation:
- Table name must be in _ALLOWED_TABLES
- Column name must be alphanumeric+underscore
- Column type must be in _ALLOWED_COL_TYPES
- Use pragma_table_info() for existence check instead of SELECT f-string
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EUR/USD synthetic data has \$volume=0 for all rows, causing any VWAP or
volume-weighted factor to produce all-NaN output. Insert a guard after
pd.read_hdf() that replaces zero volume with (\$high - \$low) range proxy
so volume-dependent factors produce meaningful signals.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LLM sometimes copies the .reset_index(level=N, drop=True) suffix from
groupby().rolling().method() patterns and adds it after .transform(),
but transform() already preserves the original index. The extra
reset_index() drops an index level and causes ValueError: 'cannot reindex
on an axis with duplicate labels' or shape mismatch on assignment.
Detect: any line containing both .transform( and .reset_index(level=..., drop=True)
Fix: strip the .reset_index() suffix from those lines.
Adds 1 new test (test_transform_reset_index_stripped) — total 30 tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. _fix_instrument_column_access: var['instrument'] = EXPR was incorrectly
converted to var.index.get_level_values(1) = EXPR, producing a SyntaxError
('cannot assign to function call'). Added (?!\s*=) negative lookahead to
skip assignment targets.
2. _fix_groupby_column_on_multiindex: groupby(['instrument','date']) on a
reset_index() variable was converted to groupby([var.index.get_level_values...])
but reset_index() produces a plain RangeIndex, not a MultiIndex, causing
AttributeError: 'RangeIndex' has no attribute 'normalize'. Added reset_vars
guard to skip variables produced by reset_index().
Adds 1 new test (test_assignment_target_not_touched) — total 29 tests, all passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. groupby(level=['instrument','date']) → get_level_values() — string level
names like 'date' don't exist in the (datetime, instrument) MultiIndex;
replaced with get_level_values(0).normalize() + get_level_values(1).
2. groupby(level=['date','instrument']) — symmetric fix for reversed order.
3. groupby(level=['instrument']) → groupby(level=1) — single string level.
4. groupby(level=N)['col'].apply(lambda) → transform(lambda) — apply() on a
grouped Series prepends an extra index level, causing index shape mismatch
when assigned back; transform() preserves the original index.
5. df.loc[instrument] DateParseError fix (instrument_loc_multiindex) — already
committed, adding supporting tests for groupby(level=['instrument','date']).
Adds 5 new tests (TestGroupbyLevelStringNames, TestGroupbyApplyToTransform)
— total 28 tests, all passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When LLM iterates over instruments via get_level_values('instrument').unique()
and then does df.loc[instrument], pandas tries to parse the instrument string
('EURUSD') as a datetime against level-0 of the (datetime, instrument) index,
raising DateParseError.
Fix: detect loop variables bound to get_level_values(1) or get_level_values('instrument')
and replace DF.loc[loop_var] (read) with DF.xs(loop_var, level=1). Assignment
write-backs are left untouched to avoid complex rewrites.
Adds 4 new tests (TestInstrumentLocMultiindex) — total 23 tests, all passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LLM-generated code often accesses df['instrument'] as a column, but
'instrument' is an index level (level 1) in the MultiIndex DataFrame.
Replace with df.index.get_level_values(1) except when the variable
was created via reset_index() (where the column actually exists).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LLM generates invalid Python by putting keyword args inside lists:
df.groupby([level=1, 'date']) ← SyntaxError
Also fixes the regex for the chained groupby Pattern A/B which had
an unescaped ')' causing re.error that silently reverted the fix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The fixer was raising min_periods to match window size, which causes
all-NaN output for intraday factors with 96 bars/day — window=240 means
zero valid bars per day, window=60 means 61% NaN per day. Critics were
consistently flagging this as incorrect for intraday factors. The LLM
now controls its own min_periods.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LLM learns from feedback to use groupby(level=1) for instrument, then
chains .groupby('date') to add the date dimension — but DataFrameGroupBy
has no .groupby() method, causing AttributeError at runtime.
Replace the invalid chain with a correct two-level groupby using
index.get_level_values(), consistent with the existing instrument+date fix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous fixer converted groupby(['instrument','date']) → groupby(level=1),
stripping the date level. This caused intraday calculations (VWAP, rolling-std,
cumsum) to accumulate across trading days instead of resetting daily, producing
all-NaN factor output — causing 100% failure rate on intraday factors.
New behaviour: capture the DataFrame variable name and emit:
var.groupby([var.index.get_level_values(1),
var.index.get_level_values(0).normalize()])
which groups by (instrument, day) as originally intended.
Adds test/qlib/test_auto_fixer.py covering all fixer cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The LLM generates x.rolling(window=N, ddof=1).std() where ddof is passed
to rolling() instead of std() — pandas raises TypeError on any ddof in rolling().
Fix both forms: rolling(..., ddof=N) and rolling(...).std(ddof=N).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- _fix_reset_index_groupby: replace groupby(level=N) on reset_index'd variables
with groupby('instrument') — fixes ValueError: level > 0 only valid with MultiIndex
- _fix_groupby_mixed_levels: strip string level names from groupby(level=[int, 'str'])
to fix AssertionError: Level 'date' not in index
- _fix_groupby_column_on_multiindex: convert groupby(['instrument','date']) on
MultiIndex DataFrames to groupby(level=1) — fixes KeyError on column access
- _fix_rolling_ddof: remove unsupported ddof kwarg from rolling().std()/var()
- fix(proposal): apply history compression to factor_proposal.py (was causing
131k-token prompts from QlibFactorHypothesis2Experiment; pycache had stale .pyc)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Summarize all but the 2 most recent experiments to compact bullet lines
(factor name, PASS/FAIL, IC value, 120-char observation snippet) instead
of including full verbatim traces; reduces prompt from ~121k to ~40-60k tokens
- Fix _evaluate_factor_directly and _save_factor_values to look for result.h5
and factor.py in sub_workspace_list instead of experiment_workspace
- Fix Series.to_parquet() → Series.to_frame().to_parquet() in _save_factor_values
- Update factor_data_template README: correct bars-per-day (1440, not 96)
- Update prompts to accept 2024-only debug dataset output as valid factor result
- Fix factor_coder prompts: allow 2024 debug data in date-range instruction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous monte_carlo_trade_pvalue() used sum(permuted_trades) as test
statistic, which is permutation-invariant (sum is commutative), so beat/n
was always 1.0 and MC_p was always 1.00 for every strategy.
Replace with a one-sided binomial test on trade win rate vs 50% baseline.
Tests whether the observed win rate could occur by chance under H0: p=0.5.
Also add _shift_daily_constant_factor_if_needed() to predix_full_eval.py
so re-evaluations apply the look-ahead bias correction for daily factors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- B301 (pickle): add nosec B301 to pd.read_pickle calls in Kaggle templates
— files are trusted Kaggle-environment inputs, not user-supplied
- B614 (torch.load): add weights_only=True to all torch.load calls in
model benchmark GT code and gt_code.py
- B104 (binding 0.0.0.0): change run_server and CLI default to 127.0.0.1;
add nosec comment where all-interface binding is required for Docker
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- monte_carlo_trade_pvalue(): shuffles trade P&L N times, returns fraction of
permuted sequences that beat real total return (p<0.05 = genuine edge)
- walk_forward_rolling(): multiple IS/OOS windows (IS=3yr, OOS=1yr, step=1yr),
computes wf_oos_sharpe_mean, wf_oos_consistency (% profitable windows)
- backtest_signal_riskmgmt(): new wf_rolling and mc_n_permutations params
- Strategy generator: enables both (200 MC permutations), adds mc_ok and wf_ok
to acceptance filter (mc_p<0.20, wf_consistency>=50%)
- Rebacktest script: enables both, stores all wf_*/mc_* fields in write-back
- 6 new tests covering MC pvalue, disabled-by-default, zero-trades edge case,
rolling WF key presence and consistency range
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split IS (2020-2023) and OOS (2024-2026) periods with independent RiskMgmt
simulations. Strategy acceptance now requires OOS sharpe > 0 and
OOS monthly return > 0 to prevent overfitting. OOS metrics stored in
strategy JSON summary and CSV reports.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace sequential predict() calls with predict_batch() in both
build_kronos_factor and evaluate_kronos_model. Up to batch_size windows
processed simultaneously on GPU, reducing per-window time from ~10s to
~0.13s (10 windows in 1.3s on RTX 5060 Ti, 75x speedup).
Adds --batch-size / -b option (default 32) to both kronos-factor and
kronos-eval CLI commands. Falls back to single inference per window if a
batch fails. Refactors timestamp prep into _build_window_inputs helper.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
KronosPredictor.predict() requires x_timestamp and y_timestamp to be
pandas Series of datetime values for its calc_time_stamps() helper.
Previously we passed integer ranges (after reset_index), which raised
AttributeError on .dt.minute. Fixed by extracting datetime index values
before resetting and using future_idx for y_timestamp.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move top-level `import torch` into _cuda_available() helper so
kronos_adapter.py can be imported in CI environments without torch.
All device defaults resolved at runtime via lazy detection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add vbt_backtest.py as single source of truth for all metric formulas
(Sharpe, drawdown, IC, transaction costs) — backtest_engine.py and
strategy_orchestrator.py now delegate to it
- Add LLMUnavailableError to exception.py; rd_loop.py catches it at the
proposal stage and raises LoopResumeError to avoid corrupting trace
history with None hypotheses
- Guard record() against None exp/hypothesis so loop resets leave
trace.hist in a consistent state
- Refactor strategy_orchestrator and optuna_optimizer to use unified
backtest path; remove duplicate metric calculation code
- Add predix_rebacktest_unified.py script for offline re-evaluation
- Update tests and README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Path injection (#37, #39, #40):
- _safe_resolve() in app.py: return safe_root / candidate.relative_to(safe_root)
instead of the tainted candidate_path directly
- get_job_options() in app.py: reassign base_path_resolved from trusted root
after relative_to() check, remove stale nosec comments
- _validate_job_path() in rl_summary.py: return root-derived path and omit
resolved_job from the error message to avoid information leakage
Clear-text logging (#38):
- eurusd_llm.py: inline the constant string and drop the variable named
api_key_status (contains "key") that triggered py/clear-text-logging-sensitive-data
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TPTBusiness