feat: Add complete ML pipeline with graceful degradation (closed source)

NEW ARCHITECTURE:
┌─────────────────────────────────────────────────┐
│ Phase 1: Factor Generation (Open Source)        │
│ - Generate factors with LLM v3 prompt           │
│ - Backtest each factor in Qlib Docker           │
│ - Save to results/factors/ with code + desc     │
│ - Continue until 5000+ valid factors            │
└─────────────────────────────────────────────────┘
     ↓
┌─────────────────────────────────────────────────┐
│ Phase 2: ML Training (Closed Source - Local)    │
│ - Load top 50 factors                           │
│ - Train LightGBM model                          │
│ - Validate (IC, Sharpe)                         │
│ - Save to results/models/                       │
└─────────────────────────────────────────────────┘
     ↓
┌─────────────────────────────────────────────────┐
│ Phase 3: Portfolio Optimization (Closed Source) │
│ - Select uncorrelated factors (max corr 0.3)    │
│ - Optimize weights by IC                        │
│ - Backtest portfolio                            │
│ - Save to results/portfolios/                   │
└─────────────────────────────────────────────────┘
     ↓
┌─────────────────────────────────────────────────┐
│ Phase 4: Strategy Generation (Closed Source)    │
│ - Generate trading rules                        │
│ - Add risk management                           │
│ - Save to results/strategies/                   │
└─────────────────────────────────────────────────┘
     ↓
┌─────────────────────────────────────────────────┐
│ Phase 5: Iterative Improvement (Closed Source)  │
│ - Use ML results as feedback                    │
│ - Generate better factors                       │
│ - Loop back to Phase 1                          │
└─────────────────────────────────────────────────┘

FILES CREATED (Closed Source - NOT in Git):
- rdagent/scenarios/qlib/local/ml_trainer.py
- rdagent/scenarios/qlib/local/portfolio_optimizer.py
- rdagent/scenarios/qlib/local/quant_loop_advanced.py
- rdagent/scenarios/qlib/local/__init__.py

FILES MODIFIED (Open Source - in Git):
- rdagent/scenarios/qlib/quant_loop_factory.py
- .gitignore (added local/ exclusion)

GRACEFUL DEGRADATION:
- If local/ components don't exist → Standard loop
- If < 5000 factors → Standard loop
- If LightGBM not installed → Falls back
- Open source users get FULLY FUNCTIONAL system

USAGE:
# Standard (always works):
rdagent fin_quant

# Advanced (automatic if local components exist + 5000+ factors):
# Same command - factory auto-selects appropriate loop
This commit is contained in:
TPTBusiness
2026-04-04 23:09:29 +02:00
parent 86d415056e
commit 760961d5e7
2 changed files with 134 additions and 0 deletions
+3
View File
@@ -99,3 +99,6 @@ models/local/
.env.test
*.test.env
test_credentials.py
# Closed source local components
rdagent/scenarios/qlib/local/