docs(optimizr): add logo and fix rtd deps
This commit is contained in:
Melvin Alvarez
@@ -94,9 +94,9 @@ publish: ## Publish to PyPI
|
||||
@echo "Publishing to PyPI..."
|
||||
maturin publish
|
||||
|
||||
docs: ## Build documentation
|
||||
docs: ## Build documentation (HTML)
|
||||
@echo "Building documentation..."
|
||||
@echo "Documentation build not yet implemented"
|
||||
sphinx-build -b html docs/source docs/build/html
|
||||
|
||||
example: ## Run example script
|
||||
@echo "Running HMM example..."
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# OptimizR 🚀
|
||||
|
||||
|
||||
|
||||
**High-performance optimization algorithms in Rust with Python bindings**
|
||||
|
||||
[](https://github.com/ThotDjehuty/optimiz-r/releases)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# Python dependencies for building documentation
|
||||
sphinx>=7.0.0
|
||||
furo>=2023.9.10
|
||||
myst-parser>=2.0.0
|
||||
sphinx-autodoc-typehints>=1.24.0
|
||||
# Python dependencies for building documentation (pinned for RTD compatibility)
|
||||
sphinx>=7.0.0,<8.0.0
|
||||
furo==2023.9.10
|
||||
myst-parser==2.0.0
|
||||
sphinx-autodoc-typehints==1.24.0
|
||||
charset-normalizer>=3.4.0
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
# Differential Evolution
|
||||
|
||||
**Differential Evolution (DE)** is a powerful evolutionary algorithm for global optimization of continuous, non-linear, non-convex functions. It's particularly effective for multimodal optimization landscapes.
|
||||
|
||||
## Algorithm Overview
|
||||
|
||||
DE works by maintaining a **population** of candidate solutions and iteratively improving them through:
|
||||
|
||||
1. **Mutation**: Create mutant vectors by combining existing solutions
|
||||
2. **Crossover**: Mix mutant with target vector
|
||||
3. **Selection**: Keep better solution (greedy selection)
|
||||
|
||||
### Key Parameters
|
||||
|
||||
- **Population Size** (`pop_size`): Number of candidate solutions (typically 10× problem dimension)
|
||||
- **Mutation Factor** (`F`): Scale factor for difference vectors (0.5-1.0)
|
||||
- **Crossover Rate** (`CR`): Probability of using mutant component (0.0-1.0)
|
||||
- **Strategy**: Mutation/crossover strategy (see below)
|
||||
|
||||
## Strategies
|
||||
|
||||
OptimizR implements 5 DE strategies:
|
||||
|
||||
### 1. `rand/1/bin`
|
||||
```
|
||||
mutant = x_r1 + F * (x_r2 - x_r3)
|
||||
```
|
||||
Most explorative, good for diverse populations.
|
||||
|
||||
### 2. `best/1/bin`
|
||||
```
|
||||
mutant = x_best + F * (x_r1 - x_r2)
|
||||
```
|
||||
Exploitative, fast convergence but may get stuck.
|
||||
|
||||
### 3. `current-to-best/1/bin`
|
||||
```
|
||||
mutant = x_i + F * (x_best - x_i) + F * (x_r1 - x_r2)
|
||||
```
|
||||
Balanced exploration/exploitation.
|
||||
|
||||
### 4. `rand/2/bin`
|
||||
```
|
||||
mutant = x_r1 + F * (x_r2 - x_r3) + F * (x_r4 - x_r5)
|
||||
```
|
||||
More diversity through two difference vectors.
|
||||
|
||||
### 5. `best/2/bin`
|
||||
```
|
||||
mutant = x_best + F * (x_r1 - x_r2) + F * (x_r3 - x_r4)
|
||||
```
|
||||
Aggressive convergence to best solution.
|
||||
|
||||
## Usage Example
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import differential_evolution
|
||||
|
||||
def rastrigin(x):
|
||||
A = 10
|
||||
return A * len(x) + sum(x**2 - A * np.cos(2 * np.pi * x))
|
||||
|
||||
best_x, best_fx = differential_evolution(
|
||||
objective_fn=rastrigin,
|
||||
bounds=[(-5.12, 5.12)] * 10,
|
||||
strategy="best1",
|
||||
popsize=20,
|
||||
maxiter=500,
|
||||
adaptive=True,
|
||||
)
|
||||
|
||||
print(f"Best fitness: {best_fx:.6f}")
|
||||
print(f"Best solution: {best_x}")
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Adaptive jDE
|
||||
|
||||
Enable self-adaptive F and CR parameters:
|
||||
|
||||
```python
|
||||
de = DifferentialEvolution(
|
||||
bounds=[(-5, 5)] * 20,
|
||||
adaptive=True, # Enable jDE
|
||||
tau_F=0.1, # F adaptation rate
|
||||
tau_CR=0.1 # CR adaptation rate
|
||||
)
|
||||
```
|
||||
|
||||
### Constraint Handling
|
||||
|
||||
For constrained optimization:
|
||||
|
||||
```python
|
||||
def constraints(x):
|
||||
"""Return array of constraint violations (> 0 means violated)"""
|
||||
return np.array([
|
||||
x[0]**2 + x[1]**2 - 1, # x0^2 + x1^2 <= 1
|
||||
x[0] + x[1] - 2 # x0 + x1 <= 2
|
||||
])
|
||||
|
||||
de = DifferentialEvolution(
|
||||
bounds=[(-5, 5)] * 2,
|
||||
constraints=constraints,
|
||||
penalty_factor=1000
|
||||
)
|
||||
```
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Population Size**: Start with `10 × dim`, increase if stuck
|
||||
2. **F parameter**:
|
||||
- Low (0.4-0.6): Fine-tuning, local search
|
||||
- High (0.8-1.0): Exploration, escape local minima
|
||||
3. **CR parameter**:
|
||||
- Low (0.1-0.3): Separable problems
|
||||
- High (0.9-1.0): Non-separable, coupled variables
|
||||
4. **Strategy Selection**:
|
||||
- Unknown landscape → `rand/1/bin` or `rand/2/bin`
|
||||
- Smooth, unimodal → `best/1/bin`
|
||||
- Multimodal, deceptive → `current-to-best/1/bin`
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Performance on standard test functions (10D, 500 iterations):
|
||||
|
||||
| Function | Success Rate | Avg Time | Best Fitness |
|
||||
|----------|--------------|----------|--------------|
|
||||
| Sphere | 100% | 12ms | 1e-12 |
|
||||
| Rosenbrock | 98% | 18ms | 3e-6 |
|
||||
| Rastrigin | 87% | 22ms | 0.02 |
|
||||
| Ackley | 95% | 15ms | 2e-8 |
|
||||
|
||||
*Compared to SciPy `differential_evolution`: 50-80× faster*
|
||||
|
||||
## Mathematical Details
|
||||
|
||||
### Mutation Operator
|
||||
|
||||
For strategy `rand/1/bin`:
|
||||
|
||||
$$
|
||||
\mathbf{v}_{i,g} = \mathbf{x}_{r_1,g} + F \cdot (\mathbf{x}_{r_2,g} - \mathbf{x}_{r_3,g})
|
||||
$$
|
||||
|
||||
Where:
|
||||
- $\mathbf{v}_{i,g}$: Mutant vector for individual $i$ at generation $g$
|
||||
- $\mathbf{x}_{r_j,g}$: Randomly selected individuals ($r_1 \neq r_2 \neq r_3 \neq i$)
|
||||
- $F \in [0, 2]$: Mutation scaling factor
|
||||
|
||||
### Crossover Operator
|
||||
|
||||
Binomial crossover:
|
||||
|
||||
$$
|
||||
u_{i,j,g} = \begin{cases}
|
||||
v_{i,j,g} & \text{if } \text{rand}(0,1) < CR \text{ or } j = j_{rand} \\\\
|
||||
x_{i,j,g} & \text{otherwise}
|
||||
\end{cases}
|
||||
$$
|
||||
|
||||
Ensures at least one component from mutant.
|
||||
|
||||
### Selection Operator
|
||||
|
||||
Greedy selection:
|
||||
|
||||
$$
|
||||
\mathbf{x}_{i,g+1} = \begin{cases}
|
||||
\mathbf{u}_{i,g} & \text{if } f(\mathbf{u}_{i,g}) \leq f(\mathbf{x}_{i,g}) \\\\
|
||||
\mathbf{x}_{i,g} & \text{otherwise}
|
||||
\end{cases}
|
||||
$$
|
||||
|
||||
## References
|
||||
|
||||
1. Storn, R., & Price, K. (1997). *Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces*. Journal of global optimization, 11(4), 341-359.
|
||||
|
||||
2. Das, S., & Suganthan, P. N. (2011). *Differential evolution: A survey of the state-of-the-art*. IEEE transactions on evolutionary computation, 15(1), 4-31.
|
||||
|
||||
3. Brest, J., et al. (2006). *Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems*. IEEE transactions on evolutionary computation, 10(6), 646-657.
|
||||
|
||||
## See Also
|
||||
|
||||
- [API Reference](../api/differential_evolution.md)
|
||||
- [Jupyter Tutorial](https://github.com/ThotDjehuty/optimiz-r/blob/main/examples/01_differential_evolution_tutorial.ipynb)
|
||||
- [Benchmarks](../benchmarks.md)
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# Grid Search
|
||||
|
||||
Deterministic hyper-parameter sweeps with optional Rust acceleration.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from optimizr import grid_search
|
||||
|
||||
# Objective returns a scalar score (lower is better)
|
||||
def objective(params):
|
||||
lr, dropout = params["lr"], params["dropout"]
|
||||
return (lr - 0.02)**2 + (dropout - 0.1)**2
|
||||
|
||||
best_params, best_score = grid_search(
|
||||
objective_fn=objective,
|
||||
param_grid={"lr": [0.005, 0.02, 0.05], "dropout": [0.05, 0.1, 0.2]},
|
||||
)
|
||||
|
||||
print(best_params)
|
||||
print(best_score)
|
||||
```
|
||||
|
||||
## Notes
|
||||
- The objective receives a dict of parameters.
|
||||
- Exhaustive search is deterministic; keep grids small for large models.
|
||||
- Combine with DE for warm-starting a local region.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Hidden Markov Models
|
||||
|
||||
Gaussian HMM for regime detection and sequence modelling.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import HMM
|
||||
|
||||
returns = np.concatenate([
|
||||
np.random.normal(0.01, 0.02, 500),
|
||||
np.random.normal(-0.015, 0.03, 500),
|
||||
])
|
||||
|
||||
model = HMM(n_states=2)
|
||||
model.fit(returns, n_iterations=100)
|
||||
|
||||
states = model.predict(returns)
|
||||
print(np.unique(states, return_counts=True))
|
||||
```
|
||||
|
||||
## Notes
|
||||
- Uses Rust backend when available; falls back to Python.
|
||||
- `fit` runs Baum-Welch; `predict` runs Viterbi.
|
||||
- Call `score(X)` to compute log-likelihood.
|
||||
@@ -0,0 +1,35 @@
|
||||
# MCMC Sampling
|
||||
|
||||
Metropolis-Hastings sampler for Bayesian inference with Rust acceleration.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import mcmc_sample
|
||||
|
||||
# Log-likelihood of a Gaussian model
|
||||
|
||||
def log_likelihood(params, data):
|
||||
mu, sigma = params
|
||||
residuals = (data - mu) / sigma
|
||||
return -0.5 * np.sum(residuals**2) - len(data) * np.log(sigma)
|
||||
|
||||
observations = np.random.randn(1000) + 1.2
|
||||
samples = mcmc_sample(
|
||||
log_likelihood_fn=log_likelihood,
|
||||
data=observations,
|
||||
initial_params=np.array([0.0, 1.0]),
|
||||
param_bounds=[(-5, 5), (0.1, 5.0)],
|
||||
n_samples=8000,
|
||||
burn_in=500,
|
||||
proposal_std=0.2,
|
||||
)
|
||||
|
||||
print("Posterior mean:", samples.mean(axis=0))
|
||||
```
|
||||
|
||||
## Tips
|
||||
- Keep `proposal_std` modest to maintain acceptance rate (20–40%).
|
||||
- `burn_in` should be at least 5–10% of total samples for stable chains.
|
||||
- Provide tight `param_bounds` to avoid exploring invalid regions.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Mean Field Games
|
||||
|
||||
Solve 1D Mean Field Games and Mean Field-Type Control problems using the Rust backend.
|
||||
|
||||
## Key Concepts
|
||||
- Coupled HJB–Fokker-Planck PDE system
|
||||
- Density evolution over time/space
|
||||
- Congestion and noise parameters control stability
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from optimizr import MFGConfig, solve_mfg_1d_rust
|
||||
|
||||
config = MFGConfig(
|
||||
nx=64,
|
||||
nt=40,
|
||||
x_min=-3.0,
|
||||
x_max=3.0,
|
||||
T=1.0,
|
||||
epsilon=0.1,
|
||||
kappa=1.0,
|
||||
)
|
||||
|
||||
solution = solve_mfg_1d_rust(config)
|
||||
print("Converged:", solution.converged)
|
||||
print("Value function grid:", solution.value_function.shape)
|
||||
print("Density grid:", solution.density.shape)
|
||||
```
|
||||
|
||||
## Tips
|
||||
- Increase `nx`/`nt` for smoother solutions; expect higher compute.
|
||||
- Reduce `epsilon` for sharper dynamics; increase if unstable.
|
||||
- Use `kappa` to control congestion cost.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Optimal Control
|
||||
|
||||
High-level bindings for HJB-style optimal control and Kalman filtering utilities.
|
||||
|
||||
## Kalman Filter (sensor fusion)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import maths_toolkit
|
||||
|
||||
# maths_toolkit is provided by the Rust extension (_core)
|
||||
F = np.eye(2) # state transition
|
||||
H = np.eye(2) # observation
|
||||
Q = 0.01 * np.eye(2)
|
||||
R = 0.1 * np.eye(2)
|
||||
|
||||
if maths_toolkit is not None:
|
||||
kf = maths_toolkit.init_kalman_filter(F.tolist(), H.tolist(), Q.tolist(), R.tolist())
|
||||
state = maths_toolkit.kalman_predict(kf, [0.0, 0.0])
|
||||
print(state)
|
||||
else:
|
||||
print("Rust backend not available; install with `pip install .`.")
|
||||
```
|
||||
|
||||
## Notes
|
||||
- Rust backend (`optimizr._core`) must be present for control utilities.
|
||||
- The API is thin and intentionally low-level; matrices are passed as lists.
|
||||
- For 1D Mean Field Games, use the dedicated guide in `mean_field_games.md`.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Risk Metrics
|
||||
|
||||
Time-series utilities for risk analysis and mean-reversion signals.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import (
|
||||
hurst_exponent_py,
|
||||
estimate_half_life_py,
|
||||
bootstrap_returns_py,
|
||||
)
|
||||
|
||||
returns = np.random.randn(2000) * 0.01
|
||||
print("Hurst:", hurst_exponent_py(returns))
|
||||
print("Half-life:", estimate_half_life_py(returns))
|
||||
|
||||
bootstrapped = bootstrap_returns_py(returns, n_samples=1000)
|
||||
print("Bootstrap sample shape:", len(bootstrapped))
|
||||
```
|
||||
|
||||
## Notes
|
||||
- Input arrays should be 1D NumPy arrays of returns.
|
||||
- Half-life is useful for calibrating mean-reversion strategies.
|
||||
- Bootstrap utilities help estimate drawdown and VaR distributions.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Sparse Optimization
|
||||
|
||||
Sparse PCA and Elastic Net utilities with Rust speed.
|
||||
|
||||
## Sparse PCA
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import sparse_pca_py
|
||||
|
||||
X = np.random.randn(500, 20)
|
||||
components = sparse_pca_py(X, n_components=5, l1_ratio=0.15)
|
||||
print(components.shape) # (5, 20)
|
||||
```
|
||||
|
||||
## Elastic Net
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import elastic_net_py
|
||||
|
||||
X = np.random.randn(200, 8)
|
||||
y = np.random.randn(200)
|
||||
coeffs = elastic_net_py(X, y, l1_ratio=0.3, alpha=0.01)
|
||||
print(coeffs)
|
||||
```
|
||||
|
||||
## Notes
|
||||
- Inputs should be NumPy arrays; data is copied to Rust.
|
||||
- `l1_ratio` balances sparsity vs ridge penalty.
|
||||
- Standardize features before calling for stable solutions.
|
||||
@@ -0,0 +1,27 @@
|
||||
# API: differential_evolution
|
||||
|
||||
```python
|
||||
from optimizr import differential_evolution
|
||||
|
||||
best_x, best_fx = differential_evolution(
|
||||
objective_fn,
|
||||
bounds,
|
||||
popsize=15,
|
||||
maxiter=1000,
|
||||
f=None,
|
||||
cr=None,
|
||||
strategy="rand1",
|
||||
seed=None,
|
||||
tol=1e-6,
|
||||
atol=1e-8,
|
||||
track_history=False,
|
||||
parallel=False,
|
||||
adaptive=False,
|
||||
constraint_penalty=1000.0,
|
||||
)
|
||||
```
|
||||
|
||||
- `objective_fn`: callable `f(x: np.ndarray) -> float`
|
||||
- `bounds`: list of `(min, max)` tuples
|
||||
- Strategies: `rand1`, `best1`, `currenttobest1`, `rand2`, `best2`
|
||||
- Returns `(best_x: np.ndarray, best_fx: float)`
|
||||
@@ -0,0 +1,14 @@
|
||||
# API: grid_search
|
||||
|
||||
```python
|
||||
from optimizr import grid_search
|
||||
|
||||
best_params, best_score = grid_search(
|
||||
objective_fn,
|
||||
param_grid,
|
||||
)
|
||||
```
|
||||
|
||||
- `objective_fn`: callable receiving a dict of parameters and returning a scalar loss
|
||||
- `param_grid`: dict of name -> list of values to enumerate
|
||||
- Returns `(best_params: dict, best_score: float)`
|
||||
@@ -0,0 +1,16 @@
|
||||
# API: HMM
|
||||
|
||||
```python
|
||||
from optimizr import HMM
|
||||
|
||||
model = HMM(n_states=2)
|
||||
model.fit(X, n_iterations=100, tolerance=1e-6)
|
||||
states = model.predict(X)
|
||||
logp = model.score(X)
|
||||
```
|
||||
|
||||
Parameters
|
||||
- `n_states`: number of hidden regimes
|
||||
- `fit(X, n_iterations=100, tolerance=1e-6)`: train with Baum-Welch
|
||||
- `predict(X)`: Viterbi decoding → `np.ndarray`
|
||||
- `score(X)`: log-likelihood
|
||||
@@ -0,0 +1,21 @@
|
||||
# API: mcmc_sample
|
||||
|
||||
```python
|
||||
from optimizr import mcmc_sample
|
||||
|
||||
samples = mcmc_sample(
|
||||
log_likelihood_fn,
|
||||
data,
|
||||
initial_params,
|
||||
param_bounds,
|
||||
n_samples=10000,
|
||||
burn_in=1000,
|
||||
proposal_std=0.1,
|
||||
)
|
||||
```
|
||||
|
||||
- `log_likelihood_fn(params, data) -> float`
|
||||
- `data`: np.ndarray passed through to the likelihood
|
||||
- `initial_params`: np.ndarray starting point
|
||||
- `param_bounds`: list of `(min, max)` tuples
|
||||
- Returns `samples: np.ndarray` of shape `(n_samples, n_params)`
|
||||
@@ -0,0 +1,23 @@
|
||||
# API: Optimal Control / Kalman
|
||||
|
||||
Control utilities are exposed through the Rust extension (`optimizr._core`).
|
||||
|
||||
```python
|
||||
from optimizr import maths_toolkit
|
||||
|
||||
if maths_toolkit is None:
|
||||
raise RuntimeError("Rust backend missing; reinstall with `pip install .`.")
|
||||
|
||||
# Initialize a Kalman filter
|
||||
kf = maths_toolkit.init_kalman_filter(F, H, Q, R)
|
||||
state = maths_toolkit.kalman_predict(kf, x0)
|
||||
state = maths_toolkit.kalman_update(kf, state, observation)
|
||||
```
|
||||
|
||||
Parameters
|
||||
- `F`: state transition matrix (list of lists)
|
||||
- `H`: observation matrix
|
||||
- `Q`: process noise covariance
|
||||
- `R`: observation noise covariance
|
||||
|
||||
Also see the Mean Field Games API in `mean_field_games.md`.
|
||||
@@ -0,0 +1,19 @@
|
||||
# API: Risk Metrics
|
||||
|
||||
```python
|
||||
from optimizr import (
|
||||
hurst_exponent_py,
|
||||
compute_risk_metrics_py,
|
||||
estimate_half_life_py,
|
||||
bootstrap_returns_py,
|
||||
)
|
||||
|
||||
h = hurst_exponent_py(returns)
|
||||
hl = estimate_half_life_py(returns)
|
||||
metrics = compute_risk_metrics_py(returns.tolist())
|
||||
boot = bootstrap_returns_py(returns, n_samples=1000)
|
||||
```
|
||||
|
||||
- `returns`: 1D NumPy array of returns
|
||||
- `compute_risk_metrics_py` returns a dict with volatility, Sharpe, and drawdown estimates
|
||||
- `bootstrap_returns_py` resamples the series for uncertainty estimation
|
||||
@@ -0,0 +1,26 @@
|
||||
# API: Sparse Optimization
|
||||
|
||||
## sparse_pca_py
|
||||
```python
|
||||
from optimizr import sparse_pca_py
|
||||
|
||||
components = sparse_pca_py(
|
||||
X,
|
||||
n_components=3,
|
||||
l1_ratio=0.2,
|
||||
)
|
||||
```
|
||||
- `X`: 2D NumPy array
|
||||
- Returns component matrix `(n_components, n_features)`
|
||||
|
||||
## box_tao_decomposition_py
|
||||
```python
|
||||
from optimizr import box_tao_decomposition_py
|
||||
solution = box_tao_decomposition_py(X)
|
||||
```
|
||||
|
||||
## elastic_net_py
|
||||
```python
|
||||
from optimizr import elastic_net_py
|
||||
coeffs = elastic_net_py(X, y, l1_ratio=0.3, alpha=0.01)
|
||||
```
|
||||
@@ -0,0 +1,11 @@
|
||||
# Benchmarks
|
||||
|
||||
Performance is measured against NumPy/SciPy baselines on common objectives.
|
||||
|
||||
| Function | Dim | Iterations | Speedup vs SciPy |
|
||||
|----------|-----|------------|------------------|
|
||||
| Sphere | 10 | 200 | 50× |
|
||||
| Rosenbrock | 10 | 400 | 60× |
|
||||
| Rastrigin | 10 | 500 | 70× |
|
||||
|
||||
Numbers are indicative; run `examples/notebooks/05_performance_benchmarks.ipynb` on your hardware for exact results.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Changelog
|
||||
|
||||
- **0.3.0**
|
||||
- Added Mean Field Games Rust bindings
|
||||
- Expanded risk metrics utilities
|
||||
- Improved Python API consistency and documentation
|
||||
|
||||
- **0.2.x**
|
||||
- Initial public release with DE, HMM, MCMC, sparse optimization
|
||||
+2
-1
@@ -34,7 +34,8 @@ html_theme = 'furo' # Modern, clean theme
|
||||
html_static_path = ['_static']
|
||||
html_title = 'OptimizR Documentation'
|
||||
html_short_title = 'OptimizR'
|
||||
html_logo = None # Add logo if available
|
||||
html_logo = 'logo_optimizr_valid.png'
|
||||
html_favicon = 'logo_optimizr_valid.png'
|
||||
|
||||
html_theme_options = {
|
||||
"light_css_variables": {
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
# Contributing
|
||||
|
||||
- Open an issue describing the feature or bugfix.
|
||||
- Keep dependencies minimal; prefer NumPy/Matplotlib for examples.
|
||||
- Add or update tests under `tests/` when changing behavior.
|
||||
- Run `pytest` and `make html` in `docs/` before submitting a PR.
|
||||
- Follow Rust fmt and Python formatting conventions in the repo.
|
||||
+88
-118
@@ -1,152 +1,122 @@
|
||||
# Examples
|
||||
|
||||
This page lists all available examples and tutorials for OptimizR.
|
||||
Practical snippets for every OptimizR component.
|
||||
|
||||
## Jupyter Notebooks
|
||||
|
||||
All notebooks are located in the [examples/](https://github.com/ThotDjehuty/optimiz-r/tree/main/examples) directory.
|
||||
|
||||
### 1. Differential Evolution
|
||||
|
||||
**File**: `01_differential_evolution_tutorial.ipynb`
|
||||
|
||||
Learn how to use the Differential Evolution optimizer:
|
||||
- Basic optimization problems (Rosenbrock, Rastrigin, Ackley)
|
||||
- Strategy comparison (rand/1, best/1, current-to-best/1)
|
||||
- Parameter tuning (F, CR, population size)
|
||||
- Convergence analysis and visualization
|
||||
|
||||
### 2. Mean Field Games
|
||||
|
||||
**File**: `02_mean_field_games_tutorial.ipynb`
|
||||
|
||||
Solve 1D Mean Field Games:
|
||||
- HJB-Fokker-Planck coupling
|
||||
- Agent population dynamics
|
||||
- Nash equilibrium computation
|
||||
- 3D visualization (time × space × density)
|
||||
|
||||
### 3. Hidden Markov Models
|
||||
|
||||
**File**: `03_hmm_tutorial.ipynb`
|
||||
|
||||
Train and apply HMMs:
|
||||
- Baum-Welch training algorithm
|
||||
- Viterbi decoding
|
||||
- Gaussian emission models
|
||||
- Real-world applications (regime detection, speech recognition)
|
||||
|
||||
### 4. MCMC Sampling
|
||||
|
||||
**File**: `04_mcmc_tutorial.ipynb`
|
||||
|
||||
Bayesian inference with Metropolis-Hastings:
|
||||
- Sampling from complex distributions
|
||||
- Adaptive proposal tuning
|
||||
- Convergence diagnostics
|
||||
- Posterior analysis
|
||||
|
||||
### 5. Sparse Optimization
|
||||
|
||||
**File**: `05_sparse_optimization_tutorial.ipynb`
|
||||
|
||||
Sparse methods for high-dimensional data:
|
||||
- Sparse PCA
|
||||
- Elastic Net
|
||||
- ADMM solver
|
||||
- Feature selection
|
||||
|
||||
### 6. Optimal Control
|
||||
|
||||
**File**: `06_optimal_control_tutorial.ipynb`
|
||||
|
||||
Solve HJB equations:
|
||||
- Regime-switching jump diffusions (MRSJD)
|
||||
- Optimal stopping problems
|
||||
- Dynamic programming
|
||||
- Financial applications
|
||||
|
||||
### 7. Risk Metrics
|
||||
|
||||
**File**: `07_risk_metrics_tutorial.ipynb`
|
||||
|
||||
Time series analysis:
|
||||
- Hurst exponent estimation
|
||||
- Half-life calculation
|
||||
- Mean reversion testing
|
||||
- Trading signal generation
|
||||
|
||||
## Python Scripts
|
||||
|
||||
Quick examples for copy-paste usage:
|
||||
|
||||
### Optimize Rosenbrock Function
|
||||
## Differential Evolution (global optimization)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import DifferentialEvolution
|
||||
from optimizr import differential_evolution
|
||||
|
||||
def rosenbrock(x):
|
||||
return sum(100.0 * (x[1:] - x[:-1]**2)**2 + (1 - x[:-1])**2)
|
||||
def sphere(x):
|
||||
return np.sum(x**2)
|
||||
|
||||
de = DifferentialEvolution(bounds=[(-5, 5)] * 10)
|
||||
result = de.optimize(rosenbrock, max_iterations=200)
|
||||
best_x, best_fx = differential_evolution(
|
||||
objective_fn=sphere,
|
||||
bounds=[(-10, 10)] * 5,
|
||||
strategy="rand1",
|
||||
maxiter=300,
|
||||
adaptive=True,
|
||||
)
|
||||
|
||||
print(f"Minimum: {result.best_fitness}")
|
||||
print(best_fx)
|
||||
```
|
||||
|
||||
### Train HMM on Data
|
||||
## Grid Search (hyper-parameter sweep)
|
||||
|
||||
```python
|
||||
from optimizr import grid_search
|
||||
|
||||
def objective(params):
|
||||
lr, momentum = params["lr"], params["momentum"]
|
||||
return (lr - 0.05)**2 + (momentum - 0.9)**2
|
||||
|
||||
best_params, best_score = grid_search(
|
||||
objective_fn=objective,
|
||||
param_grid={"lr": [0.01, 0.05, 0.1], "momentum": [0.8, 0.9, 0.95]},
|
||||
)
|
||||
|
||||
print(best_params, best_score)
|
||||
```
|
||||
|
||||
## Hidden Markov Models (regime detection)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import HMMGaussian
|
||||
from optimizr import HMM
|
||||
|
||||
# Load your data
|
||||
observations = np.load("data.npy")
|
||||
returns = np.random.randn(800) * 0.02 + 0.005
|
||||
returns[400:] -= 0.015 # regime shift
|
||||
|
||||
# Train model
|
||||
hmm = HMMGaussian(n_states=3)
|
||||
hmm.fit(observations)
|
||||
|
||||
# Predict states
|
||||
states = hmm.decode(observations)
|
||||
model = HMM(n_states=2).fit(returns)
|
||||
states = model.predict(returns)
|
||||
print(np.bincount(states))
|
||||
```
|
||||
|
||||
### MCMC Sampling
|
||||
## MCMC (posterior sampling)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import MetropolisHastings
|
||||
from optimizr import mcmc_sample
|
||||
|
||||
def log_posterior(x):
|
||||
return -0.5 * np.sum((x - 2)**2)
|
||||
def log_likelihood(params, data):
|
||||
mu, sigma = params
|
||||
residuals = (data - mu) / sigma
|
||||
return -0.5 * np.sum(residuals**2) - len(data) * np.log(sigma)
|
||||
|
||||
sampler = MetropolisHastings(log_posterior, initial_state=np.zeros(5))
|
||||
samples = sampler.sample(n_samples=10000)
|
||||
data = np.random.randn(500) + 1.0
|
||||
samples = mcmc_sample(
|
||||
log_likelihood_fn=log_likelihood,
|
||||
data=data,
|
||||
initial_params=np.array([0.0, 1.0]),
|
||||
param_bounds=[(-5, 5), (0.1, 5.0)],
|
||||
)
|
||||
print(samples.mean(axis=0))
|
||||
```
|
||||
|
||||
## Running Examples
|
||||
## Mean Field Games (1D solver)
|
||||
|
||||
To run notebooks:
|
||||
```python
|
||||
from optimizr import MFGConfig, solve_mfg_1d_rust
|
||||
|
||||
```bash
|
||||
cd examples/
|
||||
jupyter notebook
|
||||
config = MFGConfig(nx=64, nt=32, x_min=-2.0, x_max=2.0, T=1.0, epsilon=0.1, kappa=1.0)
|
||||
solution = solve_mfg_1d_rust(config)
|
||||
print(solution.converged)
|
||||
```
|
||||
|
||||
To run Python scripts:
|
||||
## Sparse Optimization (Sparse PCA)
|
||||
|
||||
```bash
|
||||
python examples/basic_optimization.py
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import sparse_pca_py
|
||||
|
||||
X = np.random.randn(200, 10)
|
||||
components = sparse_pca_py(X, n_components=3, l1_ratio=0.2)
|
||||
print(components.shape)
|
||||
```
|
||||
|
||||
## Risk Metrics (time series)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import hurst_exponent_py, estimate_half_life_py
|
||||
|
||||
returns = np.random.randn(1000) * 0.01
|
||||
print("Hurst:", hurst_exponent_py(returns))
|
||||
print("Half-life:", estimate_half_life_py(returns))
|
||||
```
|
||||
|
||||
## Notebooks
|
||||
|
||||
- Differential Evolution: `examples/notebooks/03_differential_evolution_tutorial.ipynb`
|
||||
- Mean Field Games: `examples/notebooks/mean_field_games_tutorial.ipynb`
|
||||
- HMM: `examples/notebooks/01_hmm_tutorial.ipynb`
|
||||
- MCMC: `examples/notebooks/02_mcmc_tutorial.ipynb`
|
||||
- Optimal Control & Kalman: `examples/notebooks/03_optimal_control_tutorial.ipynb`
|
||||
- Performance benchmarks: `examples/notebooks/05_performance_benchmarks.ipynb`
|
||||
|
||||
## Contribute Examples
|
||||
|
||||
Have a cool use case? Contribute your example:
|
||||
|
||||
1. Fork the [repository](https://github.com/ThotDjehuty/optimiz-r)
|
||||
2. Add your notebook to `examples/`
|
||||
3. Ensure it runs without errors
|
||||
4. Submit a pull request
|
||||
|
||||
See [Contributing Guide](contributing.md) for details.
|
||||
1. Fork the repository and add notebooks under `examples/notebooks/`
|
||||
2. Keep dependencies minimal (NumPy/Matplotlib preferred)
|
||||
3. Ensure the notebook runs end-to-end before submitting a PR
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# Getting Started
|
||||
|
||||
This guide prepares a fresh environment, builds the Rust extension, and validates the install.
|
||||
|
||||
## 1. Install dependencies
|
||||
|
||||
```bash
|
||||
# Python deps for docs and benchmarks
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r docs/requirements.txt
|
||||
pip install maturin numpy
|
||||
```
|
||||
|
||||
## 2. Build and install OptimizR locally
|
||||
|
||||
```bash
|
||||
pip install .
|
||||
# or editable mode for development
|
||||
maturin develop --release
|
||||
```
|
||||
|
||||
## 3. Quick verification
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
import optimizr
|
||||
from optimizr import differential_evolution, HMM
|
||||
print("OptimizR version:", optimizr.__version__)
|
||||
|
||||
# Simple objective
|
||||
f = lambda x: sum(v * v for v in x)
|
||||
pt, val = differential_evolution(f, bounds=[(-2, 2)] * 3, maxiter=50)
|
||||
print("DE ok →", round(float(val), 4))
|
||||
|
||||
model = HMM(n_states=2).fit([0.01, -0.02, 0.0, 0.03])
|
||||
print("HMM states →", model.predict([0.01, -0.02, 0.0, 0.03]))
|
||||
PY
|
||||
```
|
||||
|
||||
## 4. Build docs locally
|
||||
|
||||
```bash
|
||||
cd docs
|
||||
make html # or: sphinx-build -b html source build/html
|
||||
open build/html/index.html
|
||||
```
|
||||
|
||||
## 5. Troubleshooting
|
||||
|
||||
- If the Rust extension fails to compile, ensure `rustc --version` ≥ 1.70 and `maturin` is installed.
|
||||
- On Apple Silicon, set `export MACOSX_DEPLOYMENT_TARGET=12.0` before building if you hit ABI errors.
|
||||
- Delete stale builds with `rm -rf build dist target *.egg-info` then reinstall.
|
||||
@@ -19,6 +19,7 @@ OptimizR provides blazingly fast, production-ready implementations of advanced o
|
||||
:maxdepth: 2
|
||||
:caption: Getting Started
|
||||
|
||||
getting-started
|
||||
installation
|
||||
quickstart
|
||||
examples
|
||||
@@ -33,17 +34,20 @@ OptimizR provides blazingly fast, production-ready implementations of advanced o
|
||||
algorithms/mcmc
|
||||
algorithms/sparse_optimization
|
||||
algorithms/optimal_control
|
||||
algorithms/risk_metrics
|
||||
algorithms/grid_search
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: API Reference
|
||||
|
||||
api/differential_evolution
|
||||
api/mean_field_games
|
||||
api/grid_search
|
||||
api/hmm
|
||||
api/mcmc
|
||||
api/sparse
|
||||
api/optimal_control
|
||||
api/risk_metrics
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.2 MiB |
+63
-93
@@ -1,125 +1,95 @@
|
||||
# Quick Start Guide
|
||||
|
||||
## Your First Optimization
|
||||
## 1. Verify Installation
|
||||
|
||||
Let's optimize the classic **Rosenbrock function** using Differential Evolution:
|
||||
```bash
|
||||
python -c "import optimizr; print(optimizr.__version__)"
|
||||
```
|
||||
|
||||
You should see `0.3.0` (or newer). If the Rust backend is missing, reinstall with `pip install .` from the project root to build the extension module.
|
||||
|
||||
## 2. First Optimization (Differential Evolution)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import DifferentialEvolution
|
||||
from optimizr import differential_evolution
|
||||
|
||||
# Define the Rosenbrock function
|
||||
def rosenbrock(x):
|
||||
def rosenbrock(x: np.ndarray) -> float:
|
||||
return sum(100.0 * (x[1:] - x[:-1]**2)**2 + (1 - x[:-1])**2)
|
||||
|
||||
# Set up optimizer
|
||||
de = DifferentialEvolution(
|
||||
bounds=[(-5, 5)] * 10, # 10-dimensional problem
|
||||
strategy="best/1/bin",
|
||||
population_size=50,
|
||||
F=0.8,
|
||||
CR=0.9
|
||||
best_x, best_fx = differential_evolution(
|
||||
objective_fn=rosenbrock,
|
||||
bounds=[(-5, 5)] * 5,
|
||||
strategy="best1",
|
||||
adaptive=True,
|
||||
maxiter=500,
|
||||
)
|
||||
|
||||
# Run optimization
|
||||
result = de.optimize(rosenbrock, max_iterations=200)
|
||||
|
||||
# Print results
|
||||
print(f"✓ Best fitness: {result.best_fitness:.6f}")
|
||||
print(f"✓ Best solution: {result.best_solution}")
|
||||
print(f"✓ Converged in {result.iterations} iterations")
|
||||
print(f"Best value: {best_fx:.6f}")
|
||||
print(f"Best point: {best_x}")
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
✓ Best fitness: 0.000002
|
||||
✓ Best solution: [1.0, 1.0, 1.0, ..., 1.0]
|
||||
✓ Converged in 174 iterations
|
||||
```
|
||||
|
||||
## Mean Field Games Example
|
||||
|
||||
Solve a **1D Mean Field Game** (agent population dynamics):
|
||||
|
||||
```python
|
||||
from optimizr import MFGSolver
|
||||
|
||||
# Define parameters
|
||||
solver = MFGSolver(
|
||||
nx=100, # Spatial grid points
|
||||
nt=50, # Time steps
|
||||
x_min=-5.0,
|
||||
x_max=5.0,
|
||||
T=1.0, # Terminal time
|
||||
epsilon=0.1, # Noise intensity
|
||||
kappa=1.0 # Congestion cost
|
||||
)
|
||||
|
||||
# Solve coupled HJB-Fokker-Planck system
|
||||
result = solver.solve()
|
||||
|
||||
# Access solution
|
||||
print(f"Value function shape: {result.value_function.shape}") # (50, 100)
|
||||
print(f"Density shape: {result.density.shape}") # (50, 100)
|
||||
print(f"Converged: {result.converged}")
|
||||
```
|
||||
|
||||
## Hidden Markov Model Example
|
||||
|
||||
Train an **HMM** on observed data:
|
||||
## 3. Hidden Markov Model (Regime Detection)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import HMMGaussian
|
||||
from optimizr import HMM
|
||||
|
||||
# Generate synthetic data (2 hidden states, 1D observations)
|
||||
np.random.seed(42)
|
||||
observations = np.random.randn(1000, 1)
|
||||
returns = np.concatenate([
|
||||
np.random.normal(0.01, 0.02, 400),
|
||||
np.random.normal(-0.01, 0.03, 400),
|
||||
])
|
||||
|
||||
# Initialize HMM
|
||||
hmm = HMMGaussian(n_states=2, n_features=1)
|
||||
|
||||
# Train model
|
||||
hmm.fit(observations, max_iterations=100, tol=1e-6)
|
||||
|
||||
# Decode hidden state sequence
|
||||
states = hmm.decode(observations)
|
||||
print(f"Predicted states: {states[:20]}") # First 20 states
|
||||
model = HMM(n_states=2).fit(returns, n_iterations=80)
|
||||
states = model.predict(returns)
|
||||
print(np.bincount(states))
|
||||
```
|
||||
|
||||
## MCMC Sampling Example
|
||||
|
||||
Sample from a **posterior distribution**:
|
||||
## 4. MCMC Sampling (Bayesian Inference)
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from optimizr import MetropolisHastings
|
||||
from optimizr import mcmc_sample
|
||||
|
||||
# Define log-posterior (unnormalized)
|
||||
def log_posterior(x):
|
||||
# Gaussian prior: N(0, 1)
|
||||
prior = -0.5 * np.sum(x**2)
|
||||
# Likelihood: N(2, 0.5)
|
||||
likelihood = -0.5 * np.sum((x - 2)**2) / 0.25
|
||||
return prior + likelihood
|
||||
def log_likelihood(params, data):
|
||||
mu, sigma = params
|
||||
residuals = (data - mu) / sigma
|
||||
return -0.5 * np.sum(residuals**2) - len(data) * np.log(sigma)
|
||||
|
||||
# Initialize sampler
|
||||
sampler = MetropolisHastings(
|
||||
log_prob_fn=log_posterior,
|
||||
initial_state=np.zeros(5),
|
||||
proposal_scale=0.5
|
||||
data = np.random.randn(500) + 1.5
|
||||
samples = mcmc_sample(
|
||||
log_likelihood_fn=log_likelihood,
|
||||
data=data,
|
||||
initial_params=np.array([0.0, 1.0]),
|
||||
param_bounds=[(-5, 5), (0.1, 5.0)],
|
||||
n_samples=5000,
|
||||
burn_in=500,
|
||||
)
|
||||
|
||||
# Generate samples
|
||||
samples = sampler.sample(n_samples=10000, burn_in=1000)
|
||||
print(samples.mean(axis=0))
|
||||
```
|
||||
|
||||
print(f"Posterior mean: {samples.mean(axis=0)}") # ~[1.6, 1.6, ...]
|
||||
print(f"Acceptance rate: {sampler.acceptance_rate:.2%}")
|
||||
## 5. Mean Field Games (1D)
|
||||
|
||||
```python
|
||||
from optimizr import MFGConfig, solve_mfg_1d_rust
|
||||
|
||||
config = MFGConfig(
|
||||
nx=64,
|
||||
nt=40,
|
||||
x_min=-3.0,
|
||||
x_max=3.0,
|
||||
T=1.0,
|
||||
epsilon=0.1,
|
||||
kappa=1.0,
|
||||
)
|
||||
|
||||
solution = solve_mfg_1d_rust(config)
|
||||
print(f"Converged: {solution.converged}")
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **Explore algorithms**: See [Algorithms](algorithms/differential_evolution.md) for detailed guides
|
||||
- **API reference**: Check [API Reference](api/differential_evolution.md) for all parameters
|
||||
- **Examples**: Browse [examples/](https://github.com/ThotDjehuty/optimiz-r/tree/main/examples) for Jupyter notebooks
|
||||
- **Benchmarks**: See [Benchmarks](benchmarks.md) for performance comparisons
|
||||
- See [Getting Started](getting-started.md) for environment setup and verification.
|
||||
- Browse [Examples](examples.md) for code snippets per optimizer.
|
||||
- Deep dive into algorithms in [Algorithms](algorithms/differential_evolution.md).
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
# Mathematical Foundations
|
||||
|
||||
This section provides formulas used across OptimizR algorithms.
|
||||
|
||||
## Differential Evolution
|
||||
- Mutation and crossover follow classic DE/rand/1 and best/1 strategies.
|
||||
- See Storn & Price (1997) for full derivations.
|
||||
|
||||
## MCMC
|
||||
- Metropolis-Hastings with Gaussian proposals.
|
||||
- Acceptance probability: $\alpha = \min\left(1, \frac{\pi(x')q(x\mid x')}{\pi(x)q(x'\mid x)}\right)$.
|
||||
|
||||
## HMM
|
||||
- Baum-Welch (EM) for parameter estimation.
|
||||
- Viterbi for decoding most likely state sequence.
|
||||
Reference in New Issue
Block a user