docs(optimizr): add logo and fix rtd deps

2026-02-09 16:15:41 +01:00
parent c1f4c0bde7
commit 43af949904
28 changed files with 805 additions and 220 deletions
@@ -0,0 +1,189 @@
+# Differential Evolution
+
+**Differential Evolution (DE)** is a powerful evolutionary algorithm for global optimization of continuous, non-linear, non-convex functions. It's particularly effective for multimodal optimization landscapes.
+
+## Algorithm Overview
+
+DE works by maintaining a **population** of candidate solutions and iteratively improving them through:
+
+1. **Mutation**: Create mutant vectors by combining existing solutions
+2. **Crossover**: Mix mutant with target vector
+3. **Selection**: Keep better solution (greedy selection)
+
+### Key Parameters
+
+- **Population Size** (`pop_size`): Number of candidate solutions (typically 10× problem dimension)
+- **Mutation Factor** (`F`): Scale factor for difference vectors (0.5-1.0)
+- **Crossover Rate** (`CR`): Probability of using mutant component (0.0-1.0)
+- **Strategy**: Mutation/crossover strategy (see below)
+
+## Strategies
+
+OptimizR implements 5 DE strategies:
+
+### 1. `rand/1/bin`
+```
+mutant = x_r1 + F * (x_r2 - x_r3)
+```
+Most explorative, good for diverse populations.
+
+### 2. `best/1/bin`
+```
+mutant = x_best + F * (x_r1 - x_r2)
+```
+Exploitative, fast convergence but may get stuck.
+
+### 3. `current-to-best/1/bin`
+```
+mutant = x_i + F * (x_best - x_i) + F * (x_r1 - x_r2)
+```
+Balanced exploration/exploitation.
+
+### 4. `rand/2/bin`
+```
+mutant = x_r1 + F * (x_r2 - x_r3) + F * (x_r4 - x_r5)
+```
+More diversity through two difference vectors.
+
+### 5. `best/2/bin`
+```
+mutant = x_best + F * (x_r1 - x_r2) + F * (x_r3 - x_r4)
+```
+Aggressive convergence to best solution.
+
+## Usage Example
+
+```python
+import numpy as np
+from optimizr import differential_evolution
+
+def rastrigin(x):
+    A = 10
+    return A * len(x) + sum(x**2 - A * np.cos(2 * np.pi * x))
+
+best_x, best_fx = differential_evolution(
+    objective_fn=rastrigin,
+    bounds=[(-5.12, 5.12)] * 10,
+    strategy="best1",
+    popsize=20,
+    maxiter=500,
+    adaptive=True,
+)
+
+print(f"Best fitness: {best_fx:.6f}")
+print(f"Best solution: {best_x}")
+```
+
+## Advanced Features
+
+### Adaptive jDE
+
+Enable self-adaptive F and CR parameters:
+
+```python
+de = DifferentialEvolution(
+    bounds=[(-5, 5)] * 20,
+    adaptive=True,  # Enable jDE
+    tau_F=0.1,      # F adaptation rate
+    tau_CR=0.1      # CR adaptation rate
+)
+```
+
+### Constraint Handling
+
+For constrained optimization:
+
+```python
+def constraints(x):
+    """Return array of constraint violations (> 0 means violated)"""
+    return np.array([
+        x[0]**2 + x[1]**2 - 1,  # x0^2 + x1^2 <= 1
+        x[0] + x[1] - 2         # x0 + x1 <= 2
+    ])
+
+de = DifferentialEvolution(
+    bounds=[(-5, 5)] * 2,
+    constraints=constraints,
+    penalty_factor=1000
+)
+```
+
+## Performance Tips
+
+1. **Population Size**: Start with `10 × dim`, increase if stuck
+2. **F parameter**: 
+   - Low (0.4-0.6): Fine-tuning, local search
+   - High (0.8-1.0): Exploration, escape local minima
+3. **CR parameter**:
+   - Low (0.1-0.3): Separable problems
+   - High (0.9-1.0): Non-separable, coupled variables
+4. **Strategy Selection**:
+   - Unknown landscape → `rand/1/bin` or `rand/2/bin`
+   - Smooth, unimodal → `best/1/bin`
+   - Multimodal, deceptive → `current-to-best/1/bin`
+
+## Benchmarks
+
+Performance on standard test functions (10D, 500 iterations):
+
+| Function | Success Rate | Avg Time | Best Fitness |
+|----------|--------------|----------|--------------|
+| Sphere | 100% | 12ms | 1e-12 |
+| Rosenbrock | 98% | 18ms | 3e-6 |
+| Rastrigin | 87% | 22ms | 0.02 |
+| Ackley | 95% | 15ms | 2e-8 |
+
+*Compared to SciPy `differential_evolution`: 50-80× faster*
+
+## Mathematical Details
+
+### Mutation Operator
+
+For strategy `rand/1/bin`:
+
+$$
+\mathbf{v}_{i,g} = \mathbf{x}_{r_1,g} + F \cdot (\mathbf{x}_{r_2,g} - \mathbf{x}_{r_3,g})
+$$
+
+Where:
+- $\mathbf{v}_{i,g}$: Mutant vector for individual $i$ at generation $g$
+- $\mathbf{x}_{r_j,g}$: Randomly selected individuals ($r_1 \neq r_2 \neq r_3 \neq i$)
+- $F \in [0, 2]$: Mutation scaling factor
+
+### Crossover Operator
+
+Binomial crossover:
+
+$$
+u_{i,j,g} = \begin{cases}
+v_{i,j,g} & \text{if } \text{rand}(0,1) < CR \text{ or } j = j_{rand} \\\\
+x_{i,j,g} & \text{otherwise}
+\end{cases}
+$$
+
+Ensures at least one component from mutant.
+
+### Selection Operator
+
+Greedy selection:
+
+$$
+\mathbf{x}_{i,g+1} = \begin{cases}
+\mathbf{u}_{i,g} & \text{if } f(\mathbf{u}_{i,g}) \leq f(\mathbf{x}_{i,g}) \\\\
+\mathbf{x}_{i,g} & \text{otherwise}
+\end{cases}
+$$
+
+## References
+
+1. Storn, R., & Price, K. (1997). *Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces*. Journal of global optimization, 11(4), 341-359.
+
+2. Das, S., & Suganthan, P. N. (2011). *Differential evolution: A survey of the state-of-the-art*. IEEE transactions on evolutionary computation, 15(1), 4-31.
+
+3. Brest, J., et al. (2006). *Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems*. IEEE transactions on evolutionary computation, 10(6), 646-657.
+
+## See Also
+
+- [API Reference](../api/differential_evolution.md)
+- [Jupyter Tutorial](https://github.com/ThotDjehuty/optimiz-r/blob/main/examples/01_differential_evolution_tutorial.ipynb)
+- [Benchmarks](../benchmarks.md)
@@ -0,0 +1,27 @@
+# Grid Search
+
+Deterministic hyper-parameter sweeps with optional Rust acceleration.
+
+## Usage
+
+```python
+from optimizr import grid_search
+
+# Objective returns a scalar score (lower is better)
+def objective(params):
+    lr, dropout = params["lr"], params["dropout"]
+    return (lr - 0.02)**2 + (dropout - 0.1)**2
+
+best_params, best_score = grid_search(
+    objective_fn=objective,
+    param_grid={"lr": [0.005, 0.02, 0.05], "dropout": [0.05, 0.1, 0.2]},
+)
+
+print(best_params)
+print(best_score)
+```
+
+## Notes
+- The objective receives a dict of parameters.
+- Exhaustive search is deterministic; keep grids small for large models.
+- Combine with DE for warm-starting a local region.
@@ -0,0 +1,26 @@
+# Hidden Markov Models
+
+Gaussian HMM for regime detection and sequence modelling.
+
+## Usage
+
+```python
+import numpy as np
+from optimizr import HMM
+
+returns = np.concatenate([
+    np.random.normal(0.01, 0.02, 500),
+    np.random.normal(-0.015, 0.03, 500),
+])
+
+model = HMM(n_states=2)
+model.fit(returns, n_iterations=100)
+
+states = model.predict(returns)
+print(np.unique(states, return_counts=True))
+```
+
+## Notes
+- Uses Rust backend when available; falls back to Python.
+- `fit` runs Baum-Welch; `predict` runs Viterbi.
+- Call `score(X)` to compute log-likelihood.
@@ -0,0 +1,35 @@
+# MCMC Sampling
+
+Metropolis-Hastings sampler for Bayesian inference with Rust acceleration.
+
+## Usage
+
+```python
+import numpy as np
+from optimizr import mcmc_sample
+
+# Log-likelihood of a Gaussian model
+
+def log_likelihood(params, data):
+    mu, sigma = params
+    residuals = (data - mu) / sigma
+    return -0.5 * np.sum(residuals**2) - len(data) * np.log(sigma)
+
+observations = np.random.randn(1000) + 1.2
+samples = mcmc_sample(
+    log_likelihood_fn=log_likelihood,
+    data=observations,
+    initial_params=np.array([0.0, 1.0]),
+    param_bounds=[(-5, 5), (0.1, 5.0)],
+    n_samples=8000,
+    burn_in=500,
+    proposal_std=0.2,
+)
+
+print("Posterior mean:", samples.mean(axis=0))
+```
+
+## Tips
+- Keep `proposal_std` modest to maintain acceptance rate (20–40%).
+- `burn_in` should be at least 5–10% of total samples for stable chains.
+- Provide tight `param_bounds` to avoid exploring invalid regions.
@@ -0,0 +1,34 @@
+# Mean Field Games
+
+Solve 1D Mean Field Games and Mean Field-Type Control problems using the Rust backend.
+
+## Key Concepts
+- Coupled HJB–Fokker-Planck PDE system
+- Density evolution over time/space
+- Congestion and noise parameters control stability
+
+## Usage
+
+```python
+from optimizr import MFGConfig, solve_mfg_1d_rust
+
+config = MFGConfig(
+    nx=64,
+    nt=40,
+    x_min=-3.0,
+    x_max=3.0,
+    T=1.0,
+    epsilon=0.1,
+    kappa=1.0,
+)
+
+solution = solve_mfg_1d_rust(config)
+print("Converged:", solution.converged)
+print("Value function grid:", solution.value_function.shape)
+print("Density grid:", solution.density.shape)
+```
+
+## Tips
+- Increase `nx`/`nt` for smoother solutions; expect higher compute.
+- Reduce `epsilon` for sharper dynamics; increase if unstable.
+- Use `kappa` to control congestion cost.
@@ -0,0 +1,28 @@
+# Optimal Control
+
+High-level bindings for HJB-style optimal control and Kalman filtering utilities.
+
+## Kalman Filter (sensor fusion)
+
+```python
+import numpy as np
+from optimizr import maths_toolkit
+
+# maths_toolkit is provided by the Rust extension (_core)
+F = np.eye(2)  # state transition
+H = np.eye(2)  # observation
+Q = 0.01 * np.eye(2)
+R = 0.1 * np.eye(2)
+
+if maths_toolkit is not None:
+    kf = maths_toolkit.init_kalman_filter(F.tolist(), H.tolist(), Q.tolist(), R.tolist())
+    state = maths_toolkit.kalman_predict(kf, [0.0, 0.0])
+    print(state)
+else:
+    print("Rust backend not available; install with `pip install .`.")
+```
+
+## Notes
+- Rust backend (`optimizr._core`) must be present for control utilities.
+- The API is thin and intentionally low-level; matrices are passed as lists.
+- For 1D Mean Field Games, use the dedicated guide in `mean_field_games.md`.
@@ -0,0 +1,26 @@
+# Risk Metrics
+
+Time-series utilities for risk analysis and mean-reversion signals.
+
+## Quick Start
+
+```python
+import numpy as np
+from optimizr import (
+    hurst_exponent_py,
+    estimate_half_life_py,
+    bootstrap_returns_py,
+)
+
+returns = np.random.randn(2000) * 0.01
+print("Hurst:", hurst_exponent_py(returns))
+print("Half-life:", estimate_half_life_py(returns))
+
+bootstrapped = bootstrap_returns_py(returns, n_samples=1000)
+print("Bootstrap sample shape:", len(bootstrapped))
+```
+
+## Notes
+- Input arrays should be 1D NumPy arrays of returns.
+- Half-life is useful for calibrating mean-reversion strategies.
+- Bootstrap utilities help estimate drawdown and VaR distributions.
@@ -0,0 +1,31 @@
+# Sparse Optimization
+
+Sparse PCA and Elastic Net utilities with Rust speed.
+
+## Sparse PCA
+
+```python
+import numpy as np
+from optimizr import sparse_pca_py
+
+X = np.random.randn(500, 20)
+components = sparse_pca_py(X, n_components=5, l1_ratio=0.15)
+print(components.shape)  # (5, 20)
+```
+
+## Elastic Net
+
+```python
+import numpy as np
+from optimizr import elastic_net_py
+
+X = np.random.randn(200, 8)
+y = np.random.randn(200)
+coeffs = elastic_net_py(X, y, l1_ratio=0.3, alpha=0.01)
+print(coeffs)
+```
+
+## Notes
+- Inputs should be NumPy arrays; data is copied to Rust.
+- `l1_ratio` balances sparsity vs ridge penalty.
+- Standardize features before calling for stable solutions.