Add micro-benchmarks for TPE, Random, and Grid samplers with varying
history sizes, end-to-end optimization benchmarks on Sphere and
Rosenbrock, and a convergence tracking example that outputs CSV data.
Includes 6 standard test functions (Sphere, Rosenbrock, Rastrigin,
Ackley, Branin, Hartmann6) with correctness tests at known optima.
Compute per-parameter importance scores by measuring the absolute
Spearman rank correlation between each parameter's values and the
objective across completed trials. Scores are normalized to sum to 1.0
and returned sorted by descending importance.
Add constraint support so that optimization problems with constraints
(e.g., "model size < 100MB") prefer feasible solutions. Convention:
constraint value <= 0.0 means feasible.
- Add constraint_values field to Trial with set_constraints/getter
- Add constraints field to CompletedTrial with is_feasible() method
- Propagate constraints through complete_trial/prune_trial
- Make best_trial() and top_trials() constraint-aware: feasible trials
rank above infeasible, infeasible ranked by total violation
BOHB combines TPE's model-guided sampling with Hyperband's budget-aware
evaluation by conditioning the TPE model on trials at specific budget
levels, producing better-calibrated parameter proposals.
Implement CMA-ES (Covariance Matrix Adaptation Evolution Strategy) as a
new sampler for continuous optimization. Uses nalgebra for matrix
operations and implements the full Hansen 2016 algorithm: mean update,
evolution paths, covariance matrix adaptation, and cumulative step-size
adaptation.
Key features:
- Builder API with configurable sigma0, population_size, and seed
- Three-phase state machine: discovery, active sampling, generation update
- Rejection sampling with bounds clipping fallback
- Log-scale and step parameter support via internal-space mapping
- Categorical parameters handled via random sampling fallback
- Numerical robustness: eigenvalue clamping, symmetry enforcement
Adds a convenience method that combines optimize_with_callback + save
to automatically checkpoint every N trials. Also makes save() use
atomic writes (write to temp file, then rename) to prevent corrupt
files on crash.
Add Serialize/Deserialize derives to all public data types (ParamValue,
Distribution, Direction, TrialState, ParamId, AttrValue, CompletedTrial)
gated behind a `serde` feature flag. Introduce StudySnapshot struct and
Study::save()/Study::load() for persisting and restoring study state as
human-readable JSON.
Add a human-readable summary method and Display trait for Study<V>
where V: Display. The summary shows optimization direction, trial
counts (with complete/pruned breakdown), best value, and best
parameters with their labels.
Also derive PartialOrd + Ord on ParamId for deterministic parameter
ordering in summary output.
Add Study::enqueue() to push specific parameter configurations onto a
FIFO queue. The next call to ask()/create_trial() or the next iteration
in optimize() pops the front entry and injects it into the trial so
that suggest_param() returns the pre-filled value instead of sampling.
Parameters missing from an enqueued map fall back to normal sampling,
and once the queue is drained regular sampler-driven trials resume.
Add AttrValue enum (Float, Int, String, Bool) with From impls and
set_user_attr/user_attr/user_attrs methods on both Trial and
CompletedTrial. Attributes set during optimization are propagated
through complete_trial and prune_trial. AttrValue is re-exported
at crate root and in the prelude.
Add duration-based variants of all optimization methods that run trials
until a wall-clock deadline rather than for a fixed trial count:
- optimize_until(duration, objective)
- optimize_until_with_callback(duration, objective, callback)
- optimize_until_async(duration, objective)
- optimize_until_parallel(duration, concurrency, objective)
Prunes trials whose intermediate values are worse than the median
of completed trials at the same step. Supports configurable warmup
steps and minimum trial count before pruning activates.
- Add `.name()` builder method on all 5 parameter types for custom labels
- Add `CompletedTrial::get(¶m)` for typed parameter access
- Add `Display` impl on `ParamValue`
- Add prelude module at `optimizer::prelude::*`
- Shadow `_with_sampler` methods on `Study<f64>` so `optimize()` auto-uses
the configured sampler; deprecate `_with_sampler` variants
- Use runtime `Any` downcasting with `trial_factory` to avoid E0592
- Update all examples and tests to use the new API
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Manuel Raimann