Commit Graph

36 Commits

Author SHA1 Message Date
TPTBusiness 6f399c1d96 feat: price-action strategy generator — no LLM, no factors, 38 profitable strategies
- Donchian(5,1): Sharpe +5.24, +3.1%/month, 87.6% WR, 354 trades
- MACD(5,20,3): Sharpe +5.57, +3.8%/month, 88.4% WR, 346 trades
- ATR_Breakout(10,1): Sharpe +3.25, +2.1%/month
- 7 strategy templates: Donchian, SMA, RSI, Bollinger, MACD, MA-Envelope, ATR
- Grid search over 90 parameter combinations in 31 seconds
- Uses backtest_signal for consistent evaluation
2026-05-22 15:43:00 +02:00
TPTBusiness 4758de0eee refactor: remove all proprietary terms from codebase and git history
- Rename FTMO_* constants → generic names (RISK_PER_TRADE, MAX_DAILY_LOSS, etc.)
- Rename backtest_signal_ftmo → backtest_signal_risk
- Rename _apply_ftmo_mask → _apply_risk_mask
- Clean all FTMO/riskMgmt mentions from commit messages via filter-branch
- AGENTS.md: add non-negotiable rule — NEVER mention proprietary terms in commits/releases
- Code variables and function names sanitized project-wide
- Force-pushed rewritten history to remote
2026-05-22 15:10:36 +02:00
TPTBusiness d4611b530e feat: model-track bias + daily/portfolio tools
- Bandit: model arm prior bias 2.0, prior_var 5.0 → 77% model preference
- Default first action: model (was factor)
- Daily strategy generator: Kronos + factor grid search on daily resolution
- Grid search tool: fixed template, no LLM, deterministic
- Portfolio optimizer: greedy correlation-aware selection, leverage scaling
2026-05-17 20:09:46 +02:00
TPTBusiness e0000a18d2 feat: 15% monthly return target — infrastructure + daily signal resampling
Phase 1 — Infrastructure:
- RiskMgmt_RISK_PER_TRADE 0.5% → 1.5% (vbt_backtest.py)
- min_monthly_return_pct=15% acceptance filter (strategy_orchestrator)
- --min-monthly-return 15 CLI option (nexquant.py)
- {{ min_monthly_return }}% in strategy prompts
- MIN_MONTHLY_RETURN_PCT=15.0 in gen_strategies_real_bt + smart_strategy_gen
- realistic_backtest_all.py target_monthly 4→15%

Phase 2 — Factor quality:
- IC thresholds: prompt 0.05→0.08, bandit IC weight 0.10→0.20
- Explicite IC > 0.04 target in RAG prompt
- min_ic filters: data_loader 0.0→0.04, strategy_worker 0.02→0.04, ml_trainer 0.01→0.04

Architecture fix — Daily signal resampling:
- Factors have IC at daily resolution, but z-scores on 1-min collapse IC to ~0
- Resample factors to daily before strategy exec, ffill signal to 1-min for backtest
- Walk-forward IS years 3→1 (only 2 years of data available)
- Removed broken intersection() logic that destroyed 99.99% of 1-min data
- ffill stale propagation limited to 2880 bars (2 trading days)
- Fixed logger crash in _load_strategies
- Preflight: removed constant-signal check (false positive on random sandbox data)
- Tests: test_daily_signal_resampling.py (8 tests)

Non-negotiable rules: R1-R10 in AGENTS.md
2026-05-16 19:06:09 +02:00
TPTBusiness 6975f77b77 feat: auto-mode live strategy — factors when fresh, SMA fallback
- LiveSignal auto-detects factor data freshness (<7 days old)
- Falls back to 1h SMA10/30 (+0.40%/month) when factors are stale
- 30min full factor scan script for discovering new signals
- Ready for 30min factor upgrade when data available
2026-05-11 20:37:30 +02:00
TPTBusiness 918639c051 feat: 30min factor combo bests 1h — +3.59%/month (+54% annual) 2026-05-11 18:28:46 +02:00
TPTBusiness e3a65bb140 feat: 1h SMA10/30 signal integrated into RiskMgmt live trader
- _calc_1h_signal(): SMA10/30 crossover from OHLCV, session-filtered 07-17 UTC
- Runs every hour (minute==0), replaces daily signal when active
- Backtest proven: +0.40%/month OOS, -0.86% worst day, RiskMgmt-safe
- Live strategy module (nexquant_live_strategy.py) for API/standalone use
- Multi-timeframe generator (nexquant_strategy_gen.py) auto-selects best freq
- Factor mode (+3.29%/month) ready when fresh factor data available
2026-05-11 18:10:31 +02:00
TPTBusiness c45b911abe feat: live 1h London momentum strategy + multi-timeframe generator
- nexquant_live_strategy.py: real-time signal for RiskMgmt trading
  - 1h London session momentum (2 factors, 07-17 UTC)
  - Returns signal dict with strength, factor agreement
  - Ready for integration with riskmgmt_live_trader
- nexquant_strategy_gen.py: auto-tests 1h/30min/daily
  - Selects best frequency + signal combo
  - Saves config to results/strategies_live/
- live_config.json: proven config +3.29%/month, RiskMgmt-safe
2026-05-11 17:59:40 +02:00
TPTBusiness f10b257152 feat: 1h London session momentum — +3.17%/month unlevered, -1.1% DD, RiskMgmt-safe
BREAKTHROUGH: 1h frequency + session filter unlocks real intraday alpha.
- london_session_momentum: +3.28%/month, 629 trades
- Top-3 combo: +3.17%/month, -1.1% DD, 45.5% annual
- RiskMgmt-safe up to 5x: worst day -1.08% (limit -5%)
- 3x leverage → +9.8%/month, within reach of 10% target
2026-05-11 15:27:29 +02:00
TPTBusiness 15c03df431 feat: inverse factor signal combos — top-3 gives +0.57%/month with -3.7% DD
Key discovery: EUR/USD daily is mean-reverting — inverse momentum factors dominate.
- mom_15min_INV: +0.40%/month (single best)
- Top-3 combo: +0.57%/month, -3.7% DD
- Top-10 combo: +0.46%/month, -4.1% DD
- 196 trades/month — high frequency, low per-trade risk
2026-05-11 14:29:22 +02:00
TPTBusiness 90690c1675 feat: multi-asset data pipeline, daily strategy generator, ML pipeline
B) Extended EUR/USD: 5,821 bars (2003-2026) via yfinance
C) Multi-asset: DXY, GOLD, SPX, GBPUSD, USDJPY, OIL (up to 24,705 bars since 1927)
   - OIL MR50d: +1.65%/month on 25yr data
   - DXY SMA5/25: +0.35%/month since 1971
   - SPX Mom100d: +0.26%/month since 1927
   - EURUSD RSI21: +0.05%/month (2003-2026)
   Validated strategies work across full history, not just 2020-2026 regime
2026-05-10 20:48:07 +02:00
TPTBusiness 5620ea1b0e feat: Optuna-optimized RF ML pipeline for daily strategies (+0.61%/month)
- 54 features from daily OHLCV (MA, RSI, MACD, ATR, ADX, Bollinger, etc.)
- Random Forest with Optuna TPE hyperparameter tuning (50 trials)
- Walk-forward validation with RiskMgmt backtest
- 5d horizon: OOS Sharpe +47.5, +0.61%/month
- Top features: vol50, sma20_100, sma200, calendar month
2026-05-10 18:45:31 +02:00
TPTBusiness 54b8713938 feat: daily strategy generator — grid search SMA/EMA/RSI/MACD/BB (14/55 profitable)
- Systematic grid search on daily EUR/USD data (1,944 bars)
- 14 of 55 strategies OOS-profitable at 2.14 bps
- Best: RSI7(20/80) OOS Sharpe +24.9, SMA10/30 +0.40%/month
- Saves top strategies to results/strategies_daily/
- Proven: daily frequency has real alpha, 1-min is noise
2026-05-10 17:58:25 +02:00
TPTBusiness cbe1c52e00 refactor: rename project from Predix to NexQuant
Rename all source files, scripts, tests, documentation, and configuration
from Predix/predix to NexQuant/nexquant across the entire codebase.
2026-05-09 17:48:22 +02:00
TPTBusiness 33f6daf1d2 feat: continuous strategy generator (WF, MTF, stability, ML models, auto-ensemble) 2026-05-03 22:42:29 +02:00
TPTBusiness 037f7ba7d2 fix: correct Sharpe/MaxDD/WinRate in direct factor eval (was computing on raw factor, now on strategy returns) 2026-05-03 12:17:27 +02:00
TPTBusiness aba88dd090 fix: 15 bug fixes across orchestrator, runner, backtest, and infrastructure
Critical:
- strategy_orchestrator: fix IndentationError that prevented import (line 764)
- factor_runner: fix literal 'sys.executable' string → variable (line 966)

High (path bugs causing wrong directories):
- backtest_engine: fix results_path depth (3→4 .parent hops)
- results_db: fix factors_dir/failed_dir depth (3→4 .parent hops)
- factor_runner: eliminate run_id variable shadowing (parallel_run_id/db_run_id)
- model_runner: fix DB connection leak on add_backtest exception
- optuna_optimizer: fix imported logger shadowed by module-level reassignment

Medium:
- env: handle non-UTF-8 Docker build output with errors='replace'
- env: guard conda env list parsing against empty lines
- factor_runner: add check=False + stderr logging for full-data subprocess
- strategy_orchestrator: log exec() exceptions at ERROR level with traceback
- strategy_orchestrator: warn on unreplaced {{template}} variables in prompts

Low:
- factor_runner: guard IC_max.index access against scalar (AttributeError)
- predix_parallel: close log file handle on Popen failure
- predix_rebacktest_strategies: replace 4 bare except: with except Exception:
2026-05-03 09:37:00 +02:00
TPTBusiness 574e9d6c08 fix: correct project root paths and subprocess handling in parallel runner and CLI
- predix_parallel.py: fix project_root from scripts/ to repo root (parent.parent)
- predix_parallel.py: fix .env loading path and API key distribution logic
- cli.py: fix project_root depth from 4 to 3 .parent hops (7 locations)
- cli.py start_loop: use sys.executable instead of hardcoded python
- cli.py start_loop: replace broad pkill with targeted child process management
- cli.py parallel: remove hardcoded model=local
2026-05-03 08:49:18 +02:00
TPTBusiness ef2a6c5ee0 fix(factors): extend look-ahead rules to session factors and add intraday-factor guidance
- Rule 7 extended: session-based aggregations (London/NY/Asian) must also
  be shifted by 1 trading day before use — same as daily aggregations
- Rule 8 added: prefer pure intraday rolling factors (RSI, Bollinger, VWAP
  deviation, rolling std) that have no look-ahead risk and vary every minute
- predix_full_eval.py: apply _shift_daily_constant_factor_if_needed before IC
- predix_gen_strategies_real_bt.py: improved swing prompt with daily-level
  signal logic guidance for daily-constant factors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 20:19:07 +02:00
TPTBusiness dfbbffd7ea fix(backtest): replace broken MC permutation test with binomial win-rate test
The previous monte_carlo_trade_pvalue() used sum(permuted_trades) as test
statistic, which is permutation-invariant (sum is commutative), so beat/n
was always 1.0 and MC_p was always 1.00 for every strategy.

Replace with a one-sided binomial test on trade win rate vs 50% baseline.
Tests whether the observed win rate could occur by chance under H0: p=0.5.

Also add _shift_daily_constant_factor_if_needed() to predix_full_eval.py
so re-evaluations apply the look-ahead bias correction for daily factors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 09:55:45 +02:00
TPTBusiness 1958544106 fix(strategies): handle None ic/sharpe/dd in rejected strategy log output
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 20:21:21 +02:00
TPTBusiness e886ebab8f fix(strategies): guard against None IC in acceptance check, disable slow wf_rolling
- abs(ic or 0) prevents TypeError crash when backtest returns no IC value
- wf_rolling=False and mc_n_permutations=50 for faster generation runs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 20:51:07 +02:00
TPTBusiness b0490c5f0d feat(backtest): add rolling walk-forward validation and Monte Carlo trade permutation test
- monte_carlo_trade_pvalue(): shuffles trade P&L N times, returns fraction of
  permuted sequences that beat real total return (p<0.05 = genuine edge)
- walk_forward_rolling(): multiple IS/OOS windows (IS=3yr, OOS=1yr, step=1yr),
  computes wf_oos_sharpe_mean, wf_oos_consistency (% profitable windows)
- backtest_signal_riskmgmt(): new wf_rolling and mc_n_permutations params
- Strategy generator: enables both (200 MC permutations), adds mc_ok and wf_ok
  to acceptance filter (mc_p<0.20, wf_consistency>=50%)
- Rebacktest script: enables both, stores all wf_*/mc_* fields in write-back
- 6 new tests covering MC pvalue, disabled-by-default, zero-trades edge case,
  rolling WF key presence and consistency range

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 18:59:00 +02:00
TPTBusiness f726e939ab feat(strategies): make OOS validation mandatory in strategy generator
OOS split is now enforced — no fallback to IS metrics. Strategies are
rejected if OOS data is missing or OOS sharpe/monthly <= 0. Feedback
to LLM now includes OOS metrics so it learns to build generalising strategies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 14:54:56 +02:00
TPTBusiness f166cc3326 feat(backtest): add walk-forward OOS validation to backtest_signal_riskmgmt
Split IS (2020-2023) and OOS (2024-2026) periods with independent RiskMgmt
simulations. Strategy acceptance now requires OOS sharpe > 0 and
OOS monthly return > 0 to prevent overfitting. OOS metrics stored in
strategy JSON summary and CSV reports.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 12:53:00 +02:00
TPTBusiness 9102f3ce96 feat(scripts): add full file logging to strategy generation and rebacktest scripts
Each script now creates a timestamped log file in git_ignore_folder/logs/,
captures all logging calls and Rich console output via _TeeFile, and prints
the log path at startup for easy tail access.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 15:37:03 +02:00
TPTBusiness 6c100170bd feat(backtest): add RiskMgmt-realistic backtest mode with leverage, daily/total loss limits and realistic EUR/USD costs 2026-04-18 15:27:41 +02:00
TPTBusiness 1f6990d04d feat: integrate Kronos-mini OHLCV foundation model (Option A + B)
Add Kronos-mini (4.1M params, AAAI 2026, MIT) as:
- Option A: predicted-return alpha factor via rolling daily inference
  (kronos_factor_gen.py — stride=96 bars/day, ~2k inference calls)
- Option B: standalone model evaluator alongside LightGBM
  (kronos_model_eval.py — IC / hit-rate vs actual realized returns)

KronosAdapter wraps NeoQuasar/Kronos-mini + Kronos-Tokenizer-2k,
auto-detects GPU, gracefully degrades if ~/Kronos repo is missing.
Factor output: MultiIndex (datetime, instrument) with KronosPredReturn.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:49:25 +02:00
TPTBusiness 22e638af86 feat: unified backtest engine, LLM error handling, strategy refactor
- Add vbt_backtest.py as single source of truth for all metric formulas
  (Sharpe, drawdown, IC, transaction costs) — backtest_engine.py and
  strategy_orchestrator.py now delegate to it
- Add LLMUnavailableError to exception.py; rd_loop.py catches it at the
  proposal stage and raises LoopResumeError to avoid corrupting trace
  history with None hypotheses
- Guard record() against None exp/hypothesis so loop resets leave
  trace.hist in a consistent state
- Refactor strategy_orchestrator and optuna_optimizer to use unified
  backtest path; remove duplicate metric calculation code
- Add predix_rebacktest_unified.py script for offline re-evaluation
- Update tests and README

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 22:52:07 +02:00
TPTBusiness 2cec08bc91 feat: add daily log rotation, llama health wait, factor auto-fixer, and README updates
- Add rdagent/log/daily_log.py: daily-rotating structured logs per command
  (fin_quant, strategies, evaluate, parallel) with loguru; all.log combined sink
- predix.py: route TeeWriter output to logs/YYYY-MM-DD/ instead of root dir;
  wrap quant() and evaluate() in daily_log.session() for start/stop/duration tracking
- rdagent/app/cli.py: fin_quant_cli waits for llama.cpp /health endpoint before
  starting pipeline (up to 300 s); daily_log integration for fin_quant,
  generate_strategies, eval_all, parallel commands
- scripts/predix_gen_strategies_real_bt.py: daily_log integration with
  per-strategy ACCEPTED/REJECTED entries and summary on completion
- rdagent/components/coder/factor_coder/auto_fixer.py: new module that patches
  common LLM-generated factor issues (min_periods, inf/NaN, groupby.transform,
  MultiIndex corrections)
- rdagent/components/coder/factor_coder/prompts.yaml: add critical rules for
  EURUSD 1-min intraday factors (min_periods, inf handling, groupby, date range)
- README.md: document --reasoning off and --n-gpu-layers 28 for llama-server;
  explain VRAM constraints when Ollama is running alongside llama.cpp
- .bandit.yml: suppress B615 (HuggingFace unsafe download) for RL benchmark files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 07:20:08 +02:00
TPTBusiness 362cff291c feat: Add start_llama and start_loop CLI commands
Added to rdagent CLI:
- rdagent start_llama: Start llama.cpp server (replaces start_llama.sh)
- rdagent start_loop: Start strategy generator loop (replaces start_strategy_loop.sh)

Features:
- start_llama: --model, --port, --gpu-layers, --ctx-size, --reasoning options
- start_loop: --target, --max-wait options with auto-restart on crash
- Both commands have full help (--help) and examples

Moved .sh scripts to scripts/:
- start_llama.sh → scripts/ (kept as reference)
- start_strategy_loop.sh → scripts/ (kept as reference)

Usage:
  rdagent start_llama                    # Start LLM server
  rdagent start_llama --gpu-layers 40    # Custom GPU layers
  rdagent start_loop                     # Start strategy loop
  rdagent start_loop --target 5          # Generate 5 strategies per run
2026-04-09 14:49:00 +02:00
TPTBusiness a523e62d94 chore: Organize utility scripts into scripts/ directory
Moved 13 scripts from root to scripts/:
- create_strategy.py
- debug_backtest.py
- predix_add_risk_management.py
- predix_batch_backtest.py
- predix_full_eval.py
- predix_gen_strategies_real_bt.py
- predix_parallel.py
- predix_quick_daytrading.py
- predix_rebacktest_strategies.py
- predix_simple_eval.py
- predix_smart_strategy_gen.py
- predix_strategy_report.py
- watchdog_generator.sh

Kept in root (intentional):
- predix.py (main entry point)
- start_llama.sh (convenience startup)
- start_strategy_loop.sh (convenience startup)

Root directory: 44 files → 31 files
2026-04-09 14:45:49 +02:00
TPTBusiness 574a9cb75e fix: Resolve 88% empty backtest results + path fixes
Root Cause: Qlib configs used cn_data (Chinese stocks) instead of eurusd
- provider_uri: cn_data → eurusd_1min_data
- market: csi300 → eurusd
- topk: 50 → 1 (single-asset EURUSD, was opening 0 positions)
- n_drop: 5 → 0, limit_threshold: 0.095 → 0.0

Add failed run tracking and validation:
- factor_runner.py: Validate results before DB save, track failed runs
- model_runner.py: Same validation and tracking
- results_db.py: generate_results_summary() → RESULTS_SUMMARY.md
- extract_results.py: Failed run tracking, progress indicators

Fix project root paths in all modules:
- ResultsDatabase: correct path from rdagent/results/ → results/
- factor_runner: db, factors, failed_runs paths
- model_runner: failed_runs path

All 246 tests passing.
2026-04-03 16:21:59 +02:00
TPTBusiness 612ed8a802 fix: Ensure backtest results save to DB and JSON files
- Remove duplicate DB save from quant.py (keep only in factor_runner)
- Add explicit DB path creation with mkdir -p
- Add JSON factor summaries to results/factors/
- Add debug logging for result structure
- Fix logger.debug -> logger.info (RDAgentLog compatibility)
- Update tests to match new architecture (240/240 passing)
- Enhance extract_results.py with progress indicators
2026-04-03 15:46:52 +02:00
TPTBusiness 633b5639de fix: Add nosec comments for schema migration SQL in results_db.py
Bandit false positive B608: Schema migration uses controlled column names,
not user input. Add nosec comments to suppress warning.
2026-04-03 14:37:22 +02:00
TPTBusiness 2a011d262e chore: Simplify pre-commit to mandatory hooks only
- Remove optional code quality hooks (black, isort, ruff, mypy, toml-sort)
  * These blocked commits when tools not installed
  * Users can run them manually when needed
- Keep only MANDATORY hooks:
  * Integration Tests (60 tests, ~7.5s)
  * Bandit Security Scan
- Both MUST pass before every commit
2026-04-03 12:33:30 +02:00