feat: Improved LLM prompt + Optuna integration (Step 3+5)

Step 3 - LLM Prompt verbessert:
- Created prompts/strategy_generation_v2.yaml
- IC-guided factor selection instructions
- |IC| > 0.10: PRIORITIZE, |IC| > 0.05: USE, |IC| < 0.05: AVOID
- IC-weighted factor combinations
- Better examples with IC weights
- Added 'close' Series to available scope

Step 5 - Optuna-Optimierung aktiviert:
- Added use_optuna=True, optuna_trials=20 to __init__
- Integrated OptunaOptimizer in _generate_and_evaluate_single
- Added _prepare_factor_values method for Optuna
- Auto-optimizes accepted strategies with 20 trials
- Updates results if Optuna improves Sharpe

Test results (MomentumDivergenceZScore with forward-fill):
- Status: accepted
- Sharpe: 6.04
- Max DD: -1.57%
- Win Rate: 49.19%
- Ann Return: 21.88%
- Periods: 823,450 (2.27 years)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

prompts/strategy_generation_v2.yaml

@@ -0,0 +1,88 @@
+strategy_generation:
+  system: |
+    You are an expert quantitative trading researcher specialized in EUR/USD intraday strategies.
+
+    Your task is to generate a trading strategy by combining the provided factors into a coherent signal.
+
+    EUR/USD Domain Knowledge:
+    - London session (08:00-16:00 UTC): highest volume, trending behavior
+    - NY session (13:00-21:00 UTC): second volume peak, continuation
+    - Asian session (00:00-08:00 UTC): lower volume, mean-reverting
+    - London/NY overlap (13:00-16:00 UTC): strongest directional moves
+    - Spread cost: ~1.5 bps per trade — signals must overcome this
+
+    Factor Usage Rules:
+    1. ONLY use the factors provided below — no others!
+    2. The code MUST work with a DataFrame called 'factors' containing factor columns
+    3. Also available: 'close' Series with OHLCV close prices
+    4. Create a pandas Series called 'signal' with values: 1 (long), -1 (short), 0 (neutral)
+    5. signal.index MUST match factors.index exactly
+    6. signal.name must be 'signal'
+
+    IC-Guided Factor Selection:
+    - Factors with |IC| > 0.10 are highly predictive - PRIORITIZE these
+    - Factors with |IC| > 0.05 are moderately predictive - USE these
+    - Factors with |IC| < 0.05 are weak - AVOID unless complementary
+    - Combine factors with different signs of IC for diversification
+    - Weight factors proportionally to their |IC| values
+
+    Signal Quality Requirements:
+    - Generate balanced signals (~40-60% in each direction)
+    - Use rolling z-scores for normalization: (x - rolling.mean()) / rolling.std()
+    - Apply thresholds based on signal distribution (e.g., z > 0.5 for long, z < -0.5 for short)
+    - Combine factors with IC-weighted combinations
+    - Consider regime filters (trend vs mean-reversion)
+    - Use available 'close' Series for additional calculations if needed
+
+    Output ONLY valid JSON with these exact fields:
+    {
+      "strategy_name": "short_descriptive_name",
+      "factors_used": ["factor1", "factor2", "factor3"],
+      "description": "one sentence explaining the strategy logic",
+      "code": "complete Python code that creates signal Series"
+    }
+
+  user: |
+    Generate a EUR/USD trading strategy using these factors:
+
+    {{ factors }}
+
+    {{ additional_context }}
+
+    CRITICAL RULES:
+    1. DO NOT define functions - write direct executable code
+    2. DO NOT use def - just write the code that creates 'signal'
+    3. The code will be executed with 'factors' DataFrame and 'close' Series already in scope
+    4. You MUST create a variable called 'signal' as a pandas Series
+    5. signal must have values 1 (LONG), -1 (SHORT), or 0 (NEUTRAL)
+    6. signal.index must equal factors.index
+    7. Use IC values to weight factor importance - higher IC = higher weight
+
+    EXAMPLE OF CORRECT FORMAT:
+    ```
+    import pandas as pd
+    import numpy as np
+
+    # Use IC to weight factors (daily_close_return_96 has IC=0.255, very predictive)
+    mom = factors['daily_close_return_96']
+    div = factors['daily_session_momentum_divergence_1d']
+
+    z_mom = (mom - mom.rolling(20).mean()) / mom.rolling(20).std()
+    z_div = (div - div.rolling(20).mean()) / div.rolling(20).std()
+
+    # Combine with IC weights (0.255 vs 0.199)
+    composite = 0.56 * z_mom - 0.44 * z_div
+    signal = pd.Series(0, index=factors.index)
+    signal[composite > 0.5] = 1
+    signal[composite < -0.5] = -1
+    signal.name = 'signal'
+    ```
+
+    WRONG FORMAT (DO NOT DO THIS):
+    ```
+    def generate_signal(factors):
+        ...
+        return signal
+    ```
+
+    Output ONLY the JSON object, no additional text.

rdagent/components/coder/strategy_orchestrator.py

@@ -32,6 +32,7 @@ import numpy as np
 import pandas as pd
 
 from rdagent.components.prompt_loader import load_prompt
+from rdagent.components.coder.optuna_optimizer import OptunaOptimizer
 
 # OHLCV data path
 OHLCV_PATH = Path(os.getenv(
@@ -59,6 +60,8 @@ class StrategyOrchestrator:
         max_drawdown: float = -0.20,
         min_win_rate: float = 0.50,
         results_dir: Optional[str] = None,
+        use_optuna: bool = True,
+        optuna_trials: int = 20,
     ):
         """
         Parameters
@@ -81,6 +84,8 @@ class StrategyOrchestrator:
         self.min_sharpe = min_sharpe
         self.max_drawdown = max_drawdown
         self.min_win_rate = min_win_rate
+        self.use_optuna = use_optuna
+        self.optuna_trials = optuna_trials
 
         if results_dir is None:
             project_root = Path(__file__).parent.parent.parent.parent
@@ -909,9 +914,42 @@ signal = signal.rolling(window=3, min_periods=1).mean().round().astype(int)
         # Evaluate
         result = self.evaluate_strategy(code, strategy_name, factors)
         result["code"] = code
-
+        
+        # Optimize with Optuna if enabled and accepted
+        if result.get("status") == "accepted" and self.use_optuna:
+            logger.info(f"Running Optuna optimization for {strategy_name}...")
+            optimizer = OptunaOptimizer(n_trials=self.optuna_trials)
+            
+            # Prepare factor values for optimization
+            factor_values = self._prepare_factor_values(factors)
+            
+            if factor_values is not None:
+                optimized = optimizer.optimize_strategy(result, factor_values)
+                if optimized.get("best_value", float('-inf')) > result.get("sharpe_ratio", 0):
+                    logger.info(f"Optuna improved {strategy_name}: {optimized.get('best_value', 0):.2f}")
+                    result.update(optimized)
+        
         return result
 
+    def _prepare_factor_values(self, factors: List[Dict]) -> Optional[pd.DataFrame]:
+        """Prepare factor values DataFrame for Optuna optimization."""
+        factor_values = {}
+        for f in factors:
+            fname = f.get("factor_name", "")
+            if fname:
+                series = self.load_factor_values(fname)
+                if series is not None:
+                    factor_values[fname] = series
+        
+        if factor_values:
+            df = pd.DataFrame(factor_values)
+            # Forward-fill to OHLCV index
+            close = self.load_ohlcv_close()
+            if close is not None:
+                df = df.reindex(close.index).ffill()
+            return df.dropna()
+        return None
+
     def _save_strategy(self, result: Dict[str, Any]) -> None:
         """Save accepted strategy to JSON file."""
         timestamp = int(time.time())