optimiz-rs/docs/ENHANCEMENT_STRATEGY.md

OptimizR Enhancement Strategy

Date: January 2, 2025
Context: Post-Polarway Phase 4, exploring integration and improvements
Based On: v0.2.0 codebase review, roadmap analysis, synergy opportunities

Current State Analysis

✅ What's Implemented (v0.2.0)

1. Core Algorithms:

   • Differential Evolution (5 strategies: rand1, best1, currenttobest1, rand2, best2)
   • Hidden Markov Models (Baum-Welch, Viterbi)
   • MCMC Sampling (Metropolis-Hastings, adaptive proposals)
   • Grid Search
   • Information Theory (mutual information, Shannon entropy)

2. Advanced Features (v0.2.0):

   • Sparse Optimization (Sparse PCA, Box-Tao, Elastic Net)
   • Optimal Control (HJB solver, regime switching, jump diffusion)
   • Risk Metrics (Hurst exponent, half-life, bootstrap)
   • Mathematical Toolkit (numerical differentiation, linear algebra, statistics)

3. Architecture:

   • Trait-based design (Optimizer, Sampler, InformationMeasure)
   • Builder pattern for configuration
   • Functional programming utilities (composition, memoization, pipes)
   • Rayon dependency already present
   • Feature flag infrastructure (parallel feature exists)

⚠️ What's Missing/Incomplete

1. Parallelization BLOCKED:

   • Infrastructure exists (Rayon trait, ParallelExecutor trait in core.rs)
   • DE has parallel parameter but disabled due to Python GIL
   • Comment: "Python callbacks cannot be safely parallelized due to GIL"
   • Grid search marked as "future: Expected 50-100x speedup"

2. Advanced DE Variants (Roadmap v0.3.0):

   • JADE (jDE with archive)
   • SHADE (Success-History based Adaptive DE)
   • L-SHADE (with linear population reduction)
   • Current: Only basic jDE adaptive control

3. Multi-Objective Optimization (Roadmap):

   • NSGA-DE (Non-dominated Sorting)
   • MODE (Multi-Objective DE)
   • Pareto front computation

4. GPU Acceleration (Roadmap):

   • CUDA kernels
   • OpenCL support
   • 10-100× additional speedup

5. Additional Algorithms (Roadmap):

   • Particle Swarm Optimization (PSO)
   • CMA-ES (Covariance Matrix Adaptation)
   • Simulated Annealing
   • Ant Colony Optimization

Synergy Opportunities: Polarway + OptimizR

1. Time-Series Feature Engineering for HMM

Description: Use Polarway's time-series operations to create features for regime detection

Implementation:

# Polarway: Fast feature creation
df = client.lag(['price'], periods=1)  # Lagged prices
df = client.pct_change(['price'], periods=1)  # Returns
df = client.diff(['price'], periods=1)  # Price changes

# OptimizR: Regime detection on features
returns = df['price_pct_change'].to_numpy()
hmm = HMM(n_states=3)  # Bull, Bear, Sideways
hmm.fit(returns, n_iterations=100)
states = hmm.predict(returns)

Value:

• Polarway provides fast feature engineering (50-200× faster for large datasets)
• OptimizR provides statistical inference (HMM regime detection)
• Combined: Real-time regime switching for trading strategies

2. Risk Metrics on Time-Series Data

Description: Calculate advanced risk metrics using both systems

Implementation:

# Polarway: Efficient return calculation
df = client.pct_change(['price'], periods=1)
returns = df['price_pct_change'].to_numpy()

# OptimizR: Risk analysis
hurst = compute_hurst_exponent(returns)  # Mean-reversion detection
half_life = estimate_half_life(returns)  # Reversion time
risk_metrics = compute_risk_metrics(returns)  # Comprehensive suite

Value:

• Fast preprocessing (Polarway) + sophisticated analysis (OptimizR)
• Useful for pairs trading, mean-reversion strategies
• Real-time risk monitoring

3. Optimal Control with Market Data

Description: Dynamic portfolio rebalancing with regime-dependent strategies

Implementation:

# Polarway: Multi-asset feature creation
df = client.lag(['spy_price', 'vix'], periods=[1, 5, 20])
df = client.pct_change(['spy_price'], periods=1)

# OptimizR: Solve optimal control problem
# State: [price, volatility regime]
# Control: portfolio weights
value_fn = solve_hjb_regime_switching(...)

Value:

• Combines fast data processing with optimal control theory
• Regime-dependent strategies (bull vs bear market)
• Practical for HFT and algorithmic trading

4. Parameter Optimization for Trading Strategies

Description: Use DE to optimize strategy parameters on time-series data

Implementation:

# Polarway: Backtest execution (fast data ops)
def backtest_strategy(params):
    df = client.lag(['price'], periods=int(params[0]))
    # ... strategy logic ...
    return -sharpe_ratio  # Minimize negative Sharpe

# OptimizR: Find optimal parameters
result = differential_evolution(
    objective_fn=backtest_strategy,
    bounds=[(1, 50), (0.01, 0.5)],  # [lag_period, threshold]
    maxiter=500,
    strategy='rand1'
)

Value:

• Polarway handles heavy data processing
• OptimizR finds optimal parameters
• 74-88× faster than SciPy DE

High-Priority Enhancements

Priority 1: Enable Parallelization for Pure-Rust Objectives

Problem: parallel parameter exists but disabled due to Python GIL issues

Solution: Create Rust-native objective function trait for GIL-free parallelization

Implementation Strategy:

1. Add RustObjectiveFn trait separate from Python callbacks
2. Implement parallel evaluation for Rust-native functions
3. Keep Python callbacks sequential (GIL limitation)
4. Enable parallel grid search (no Python callbacks needed for grid)

Code Outline:

// In src/core.rs or src/differential_evolution.rs

/// Rust-native objective function (no Python, no GIL)
pub trait RustObjective: Send + Sync {
    fn evaluate(&self, x: &[f64]) -> f64;
}

/// Parallel evaluation for Rust objectives
#[cfg(feature = "parallel")]
fn evaluate_population_parallel<F: RustObjective>(
    objective: &F,
    population: &[Vec<f64>]
) -> Vec<f64> {
    use rayon::prelude::*;
    population.par_iter()
        .map(|individual| objective.evaluate(individual))
        .collect()
}

// Python binding for benchmarking
#[pyfunction]
fn differential_evolution_rust(
    objective_name: &str,  // "sphere", "rosenbrock", "rastrigin"
    bounds: Vec<(f64, f64)>,
    parallel: bool,  // Now actually works!
    ...
) -> PyResult<DEResult>

Benefits:

• 10-100× speedup for built-in test functions (sphere, Rosenbrock, Rastrigin)
• Useful for benchmarking and testing
• Grid search can be parallelized (no callbacks)
• Foundation for future Rust-only mode

Effort: Medium (1-2 hours)

Priority 2: Implement SHADE (Success-History Adaptive DE)

Problem: Current adaptive DE uses basic jDE, SHADE is state-of-the-art

Solution: Implement SHADE algorithm from Tanabe & Fukunaga (2013)

Key Features:

• Historical memory of successful parameters (F, CR)
• Weighted random selection from memory
• Better than jDE on CEC benchmarks

Implementation Strategy:

1. Add SHADE variant to DEStrategy enum
2. Create success history buffer (circular buffer of size H=10-100)
3. Update memory after each successful mutation
4. Sample (F, CR) from history using Cauchy/Normal distributions

Code Outline:

pub enum DEAdaptive {
    None,
    JDE,      // Current implementation
    SHADE,    // New: Success-history based
    LSHADE,   // Future: With linear population reduction
}

struct SHADEMemory {
    history_f: Vec<f64>,    // Successful F values
    history_cr: Vec<f64>,   // Successful CR values
    index: usize,           // Circular buffer index
    size: usize,            // Memory size H
}

impl SHADEMemory {
    fn sample_f(&self) -> f64 {
        // Cauchy distribution centered on random history entry
    }
    
    fn sample_cr(&self) -> f64 {
        // Normal distribution centered on random history entry
    }
    
    fn update(&mut self, successful_f: f64, successful_cr: f64) {
        // Add to circular buffer
    }
}

Benefits:

• State-of-the-art adaptive control
• Better than jDE empirically
• Aligns with roadmap (v0.3.0)
• Minimal API changes

Effort: Medium-High (2-4 hours with testing)

Priority 3: Time-Series Integration Helpers

Problem: Using Polarway + OptimizR requires manual glue code

Solution: Create helper functions for common time-series + optimization patterns

Implementation Strategy:

1. Add timeseries_utils module to OptimizR
2. Functions for common workflows
3. Optional Polarway integration (via feature flag)

Code Outline:

// New module: src/timeseries_utils.rs

/// Prepare time-series data for HMM regime detection
pub fn prepare_for_hmm(
    prices: &[f64],
    lag_periods: &[usize],
) -> Vec<Vec<f64>> {
    // Create features: returns, lagged returns, etc.
}

/// Rolling window risk metrics
pub fn rolling_hurst_exponent(
    returns: &[f64],
    window_size: usize,
) -> Vec<f64> {
    // Compute Hurst exponent in rolling windows
}

/// Backtest parameter optimization
pub fn optimize_strategy_params<F>(
    objective_fn: F,
    param_bounds: Vec<(f64, f64)>,
    n_trials: usize,
) -> DEResult
where F: Fn(&[f64]) -> f64
{
    // Wrapper around DE with sensible defaults
}

Python Bindings:

from optimizr import timeseries_utils as tsu

# Prepare features
features = tsu.prepare_for_hmm(prices, lag_periods=[1, 5, 20])

# Rolling risk metrics
rolling_hurst = tsu.rolling_hurst_exponent(returns, window_size=252)

# Strategy optimization
def my_strategy(params):
    # ... backtesting logic ...
    return sharpe_ratio

result = tsu.optimize_strategy_params(
    my_strategy,
    param_bounds=[(1, 50), (0.01, 0.5)],
    n_trials=500
)

Benefits:

• Reduces boilerplate for common use cases
• Makes integration obvious
• Encourages adoption
• Low effort, high value

Effort: Low-Medium (1-2 hours)

Secondary Enhancements (Future Work)

4. Multi-Objective Optimization (NSGA-DE)

• Roadmap: v0.3.0
• Use Case: Portfolio optimization (maximize return, minimize risk)
• Effort: High (5-8 hours)

5. GPU Acceleration

• Roadmap: v0.3.0
• Use Case: Massive population sizes (10K-100K individuals)
• Effort: Very High (multi-day project)

6. Additional Algorithms (PSO, CMA-ES, etc.)

• Roadmap: v0.3.0
• Use Case: Algorithm portfolio for different problem types
• Effort: High per algorithm (3-5 hours each)

Recommended Implementation Order

1. Session 1 (Current): Time-Series Integration Helpers (1-2 hours)

   • Low effort, immediate value
   • Makes Polarway + OptimizR integration obvious
   • Creates examples for documentation

2. Session 2: Enable Rust-Native Parallelization (1-2 hours)

   • Unblocks major performance gain
   • Grid search parallelization
   • Foundation for future work

3. Session 3: Implement SHADE (2-4 hours)

   • State-of-the-art adaptive DE
   • Aligns with roadmap
   • Publishable improvement

4. Future: Multi-objective, GPU, additional algorithms

   • Larger projects
   • Requires more research

Testing Strategy

For each enhancement:

1. Unit tests: Algorithm correctness (sphere function, Rosenbrock)
2. Benchmarks: Performance comparison (before/after)
3. Integration tests: Polarway + OptimizR workflows
4. Documentation: Usage examples, API docs

Git Commit Strategy (per MANDATORY rules)

Each enhancement gets:

1. Feature branch: feature/shade-algorithm or feature/rust-parallelization
2. Implementation commits with tests
3. Benchmark results documented
4. Final commit: feat(de): implement SHADE adaptive DE variant
5. Push to origin
6. Log to historia/

Success Metrics

1. Performance:

   • Rust parallelization: 10-100× speedup on multi-core
   • SHADE: 10-20% better convergence than jDE on benchmarks
   • Time-series helpers: Zero overhead (pure convenience)

2. Usability:

   • Integration examples in documentation
   • Clear API documentation
   • Python usage examples

3. Completeness:

   • All tests passing
   • Benchmarks documented
   • Changes committed to git

---

Next Action: Implement Priority 3 (Time-Series Integration Helpers) as it's lowest effort with immediate value for demonstrating Polarway + OptimizR synergy.