optimiz-rs/docs/source/algorithms/differential_evolution.md

Differential Evolution

Differential Evolution (DE) is a powerful evolutionary algorithm for global optimization of continuous, non-linear, non-convex functions. It's particularly effective for multimodal optimization landscapes.

Algorithm Overview

DE works by maintaining a population of candidate solutions and iteratively improving them through:

1. Mutation: Create mutant vectors by combining existing solutions
2. Crossover: Mix mutant with target vector
3. Selection: Keep better solution (greedy selection)

Key Parameters

• Population Size (pop_size): Number of candidate solutions (typically 10× problem dimension)
• Mutation Factor (F): Scale factor for difference vectors (0.5-1.0)
• Crossover Rate (CR): Probability of using mutant component (0.0-1.0)
• Strategy: Mutation/crossover strategy (see below)

Strategies

OptimizR implements 5 DE strategies:

1. rand/1/bin

mutant = x_r1 + F * (x_r2 - x_r3)

Most explorative, good for diverse populations.

2. best/1/bin

mutant = x_best + F * (x_r1 - x_r2)

Exploitative, fast convergence but may get stuck.

3. current-to-best/1/bin

mutant = x_i + F * (x_best - x_i) + F * (x_r1 - x_r2)

Balanced exploration/exploitation.

4. rand/2/bin

mutant = x_r1 + F * (x_r2 - x_r3) + F * (x_r4 - x_r5)

More diversity through two difference vectors.

5. best/2/bin

mutant = x_best + F * (x_r1 - x_r2) + F * (x_r3 - x_r4)

Aggressive convergence to best solution.

Usage Example

import numpy as np
from optimizr import differential_evolution

def rastrigin(x):
    A = 10
    return A * len(x) + sum(x**2 - A * np.cos(2 * np.pi * x))

best_x, best_fx = differential_evolution(
    objective_fn=rastrigin,
    bounds=[(-5.12, 5.12)] * 10,
    strategy="best1",
    popsize=20,
    maxiter=500,
    adaptive=True,
)

print(f"Best fitness: {best_fx:.6f}")
print(f"Best solution: {best_x}")

Advanced Features

Adaptive jDE

Enable self-adaptive F and CR parameters:

de = DifferentialEvolution(
    bounds=[(-5, 5)] * 20,
    adaptive=True,  # Enable jDE
    tau_F=0.1,      # F adaptation rate
    tau_CR=0.1      # CR adaptation rate
)

Constraint Handling

For constrained optimization:

def constraints(x):
    """Return array of constraint violations (> 0 means violated)"""
    return np.array([
        x[0]**2 + x[1]**2 - 1,  # x0^2 + x1^2 <= 1
        x[0] + x[1] - 2         # x0 + x1 <= 2
    ])

de = DifferentialEvolution(
    bounds=[(-5, 5)] * 2,
    constraints=constraints,
    penalty_factor=1000
)

Performance Tips

1. Population Size: Start with 10 × dim, increase if stuck
2. F parameter:
   • Low (0.4-0.6): Fine-tuning, local search
   • High (0.8-1.0): Exploration, escape local minima
3. CR parameter:
   • Low (0.1-0.3): Separable problems
   • High (0.9-1.0): Non-separable, coupled variables
4. Strategy Selection:
   • Unknown landscape → rand/1/bin or rand/2/bin
   • Smooth, unimodal → best/1/bin
   • Multimodal, deceptive → current-to-best/1/bin

Benchmarks

Performance on standard test functions (10D, 500 iterations):

Function	Success Rate	Avg Time	Best Fitness
Sphere	100%	12ms	1e-12
Rosenbrock	98%	18ms	3e-6
Rastrigin	87%	22ms	0.02
Ackley	95%	15ms	2e-8

Compared to SciPy differential_evolution: 50-80× faster

Mathematical Details

Mutation Operator

For strategy rand/1/bin:

\mathbf{v}_{i,g} = \mathbf{x}_{r_1,g} + F \cdot (\mathbf{x}_{r_2,g} - \mathbf{x}_{r_3,g})

Where:

• \mathbf{v}_{i,g}: Mutant vector for individual i at generation g
• \mathbf{x}_{r_j,g}: Randomly selected individuals (r_1 \neq r_2 \neq r_3 \neq i)
• F \in [0, 2]: Mutation scaling factor

Crossover Operator

Binomial crossover:

u_{i,j,g} = \begin{cases}
v_{i,j,g} & \text{if } \text{rand}(0,1) < CR \text{ or } j = j_{rand} \\\\
x_{i,j,g} & \text{otherwise}
\end{cases}

Ensures at least one component from mutant.

Selection Operator

Greedy selection:

\mathbf{x}_{i,g+1} = \begin{cases}
\mathbf{u}_{i,g} & \text{if } f(\mathbf{u}_{i,g}) \leq f(\mathbf{x}_{i,g}) \\\\
\mathbf{x}_{i,g} & \text{otherwise}
\end{cases}

References

1. Storn, R., & Price, K. (1997). Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), 341-359.

2. Das, S., & Suganthan, P. N. (2011). Differential evolution: A survey of the state-of-the-art. IEEE transactions on evolutionary computation, 15(1), 4-31.

3. Brest, J., et al. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE transactions on evolutionary computation, 10(6), 646-657.

See Also

• API Reference
• Jupyter Tutorial
• Benchmarks