- Add `CompletedTrial::validate()` checking all f64 fields are finite
- Call validate after deserializing each trial in journal storage
- Make `distribution` and `param` modules public (types already in public fields)
- Add blanket `impl Objective<V> for Fn(&mut Trial) -> Result<V, E>`
so closures work directly with `optimize`
- Rewrite optimize, optimize_async, optimize_parallel to accept
`impl Objective<V>` with before_trial/after_trial hooks
- Remove optimize_with, optimize_with_async, optimize_with_parallel
- Remove max_retries and retry logic from Objective trait
- Add explicit closure type annotations for HRTB inference
- Convert FnMut test closures to Fn via RefCell/Cell
- Sampler, pruner, storage, parameter, and multi_objective types are no
longer re-exported at the crate root; access via module paths instead
(e.g. optimizer::sampler::TpeSampler, optimizer::parameter::FloatParam)
- Prelude continues to re-export everything for convenience
- Add module-level pub use re-exports in sampler/mod.rs and parameter.rs
- Update derive macro to reference optimizer::parameter::Categorical
- Rename sampler::differential_evolution to sampler::de
- Fix all downstream imports in tests, benches, and doctests
- Fix all rustdoc intra-doc links to use explicit module paths
Extract shared evolutionary algorithm infrastructure (genetic operators,
candidate management, Das-Dennis reference points) from NSGA-II into a
new genetic.rs module, then build two new multi-objective samplers on top:
- NSGA-III: reference-point-based niching for well-distributed fronts
on 3+ objective problems (Das-Dennis structured points, normalization,
perpendicular distance association, niching selection)
- MOEA/D: decomposition-based optimization with three scalarization
methods (Tchebycheff, WeightedSum, PBI), weight-vector neighborhoods,
and neighborhood-based mating selection
Both implement MultiObjectiveSampler with builder pattern, seeded RNG,
and SBX crossover / polynomial mutation via the shared genetic module.
Replace the internal Vec<CompletedTrial<V>> with a pluggable Storage<V>
trait. MemoryStorage is the default (no behavior change for existing
users). Behind the `journal` feature flag, JournalStorage persists
trials to a JSONL file with fs2 file locking for multi-process safety.
- Storage<V> trait with push(), trials_arc(), refresh() methods
- MemoryStorage<V> wraps Arc<RwLock<Vec<CompletedTrial<V>>>>
- JournalStorage<V> appends JSON lines with exclusive file locks
- Study::with_sampler_and_storage() general constructor
- Study::with_journal() convenience constructor (journal feature)
- Refresh from storage on create_trial() for multi-process discovery
- MSRV bumped to 1.89
Population-based optimizer with mutation + crossover. Supports three
strategies (Rand1, Best1, CurrentToBest1), configurable F and CR,
and auto-sized populations. No extra dependencies needed.
fastrand is smaller, faster, and has no dependencies. Add rng_util
helper for f64 range generation since fastrand lacks a built-in
equivalent. Migrate all samplers, KDE modules, and fANOVA to use
fastrand's concrete Rng type instead of rand's trait-based generics.
Implement a classical Bayesian optimization sampler using a GP surrogate
with Matérn 5/2 kernel and Expected Improvement acquisition function.
Feature-gated behind `gp = ["dep:nalgebra"]`.
Implement functional ANOVA decomposition behind the `fanova` feature
flag. A self-contained random forest is trained on trial data, then
marginal predictions are used to compute per-parameter main effects
and pairwise interaction effects, normalized to sum to 1.0.
Adds Study::fanova() / fanova_with_config(), FanovaResult, and
FanovaConfig. Includes unit and integration tests covering dominant
parameters, interaction detection, consistency with correlation-based
importance, and error handling.
Add a `visualization` feature flag that generates self-contained HTML
reports with interactive Plotly.js charts for offline visualization of
optimization results. Charts include optimization history, slice plots,
parallel coordinates, parameter importance, trial timeline, and
intermediate values (learning curves).
Extend TPE to handle multi-objective optimization using Pareto-based
splitting. MOTPE uses non-dominated sorting to define "good" (front 0)
vs "bad" (dominated) regions for the KDE models, replacing the
single-objective gamma-based split.
Add MultiObjectiveStudy for optimizing multiple objectives simultaneously,
backed by NSGA-II (Non-dominated Sorting Genetic Algorithm II) with SBX
crossover, polynomial mutation, and constraint-aware dominance.
New public API:
- MultiObjectiveStudy with optimize(), pareto_front(), ask()/tell()
- MultiObjectiveTrial with get(), is_feasible(), user attributes
- MultiObjectiveSampler trait for custom MO samplers
- Nsga2Sampler with builder for population size, crossover/mutation params
- ObjectiveDimensionMismatch error variant
Compute per-parameter importance scores by measuring the absolute
Spearman rank correlation between each parameter's values and the
objective across completed trials. Scores are normalized to sum to 1.0
and returned sorted by descending importance.
BOHB combines TPE's model-guided sampling with Hyperband's budget-aware
evaluation by conditioning the TPE model on trials at specific budget
levels, producing better-calibrated parameter proposals.
Implement CMA-ES (Covariance Matrix Adaptation Evolution Strategy) as a
new sampler for continuous optimization. Uses nalgebra for matrix
operations and implements the full Hansen 2016 algorithm: mean update,
evolution paths, covariance matrix adaptation, and cumulative step-size
adaptation.
Key features:
- Builder API with configurable sigma0, population_size, and seed
- Three-phase state machine: discovery, active sampling, generation update
- Rejection sampling with bounds clipping fallback
- Log-scale and step parameter support via internal-space mapping
- Categorical parameters handled via random sampling fallback
- Numerical robustness: eigenvalue clamping, symmetry enforcement
Add SobolSampler using Owen-scrambled Sobol sequences (sobol_burley
crate) for better uniform coverage of the parameter space compared to
random sampling. Supports all distribution types including log-scale
and stepped parameters.
Add optional structured logging via the tracing crate:
- info-level spans on all optimize* methods (direction, n_trials/duration)
- info events for trial completed, new best value, trial pruned
- debug events for trial failed and parameter sampled
- Zero overhead when the feature is disabled
Add Serialize/Deserialize derives to all public data types (ParamValue,
Distribution, Direction, TrialState, ParamId, AttrValue, CompletedTrial)
gated behind a `serde` feature flag. Introduce StudySnapshot struct and
Study::save()/Study::load() for persisting and restoring study state as
human-readable JSON.
Add AttrValue enum (Float, Int, String, Bool) with From impls and
set_user_attr/user_attr/user_attrs methods on both Trial and
CompletedTrial. Attributes set during optimization are propagated
through complete_trial and prune_trial. AttrValue is re-exported
at crate root and in the prelude.
Wraps any inner Pruner and requires it to recommend pruning for N
consecutive steps before actually pruning. Useful for noisy
intermediate values where a single bad step shouldn't trigger pruning.
Prunes trials whose intermediate values are worse than the median
of completed trials at the same step. Supports configurable warmup
steps and minimum trial count before pruning activates.
- Add `Pruned` variant to `TrialState`
- Add `Error::TrialPruned` variant and standalone `TrialPruned` struct
with `From<TrialPruned> for Error` for ergonomic `?` usage
- Add `state` field to `CompletedTrial` (defaults to `Complete`)
- Add `Study::prune_trial()` and `Study::n_pruned_trials()`
- `optimize()` and `optimize_with_callback()` detect `TrialPruned`
errors via Any downcasting and record pruned trials instead of
failing them
- `best_trial()` / `best_value()` now filter to only `Complete` trials
- Re-export `TrialPruned` from crate root and prelude
Introduce the pruning extension point for the trial pruning system.
The Pruner trait allows deciding whether to stop a trial early based
on intermediate values. NopPruner (never prunes) is the default.
- Add `.name()` builder method on all 5 parameter types for custom labels
- Add `CompletedTrial::get(¶m)` for typed parameter access
- Add `Display` impl on `ParamValue`
- Add prelude module at `optimizer::prelude::*`
- Shadow `_with_sampler` methods on `Study<f64>` so `optimize()` auto-uses
the configured sampler; deprecate `_with_sampler` variants
- Use runtime `Any` downcasting with `trial_factory` to avoid E0592
- Update all examples and tests to use the new API
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Manuel Raimann