When two parameters share the same Distribution (e.g. two
FloatParam::new(-5.0, 5.0) in one study), TpeSampler iterated
trial.distributions (a HashMap) with find_map to extract a value
matching the target distribution. HashMap iteration order varies
across runs, so x's history could be conflated with y's history
non-deterministically — making seeded TPE runs flaky in parallel
test execution.
Pick the smallest-ParamId match instead, so the choice is stable
across runs.
Also:
- src/sampler/genetic.rs: collapse if into match guard (clippy)
- src/lib.rs: drop std_instead_of_core lint (the only remaining
hits are std::io::Error/ErrorKind, which have no stable core
equivalent yet)
- Cargo.toml: relax minimum versions for fastrand/tokio/tracing/
optimizer-derive; bump dev-dep tokio to 1.50 to match in-tree usage
- Clamp KDE candidates to parameter bounds before evaluating l(x)/g(x),
matching the univariate TPE behavior; without this, candidates scored
well at out-of-bounds locations but became suboptimal when clamped
- Sort HashMap iterations by ParamId before consuming the seeded RNG to
eliminate non-deterministic sampling caused by global ParamId counter
- Replace wall-clock timing assertion in async concurrency test with
atomic max-active counter to avoid CI flakiness
- Gate unused HashSet import behind cfg(feature = "async")
* fix: address 18 bugs found during codebase audit
High severity:
- TPE/MOTPE: match parameters by exact distribution equality instead of
flat-mapping over all param values, preventing cross-parameter mixing
- MultivariateTpeSampler: find_matching_param now uses search space
distributions for exact matching instead of type+range heuristic
- JournalStorage: write_to_file no longer advances file_offset (left to
refresh), both operations serialized under single io_lock mutex,
refresh uses fetch_max and deduplicates by trial ID
Medium severity:
- NSGA-III: use actual Pareto front ranks for tournament selection
instead of artificial cyclic indices
- sample_random: apply step quantization after log-scale sampling
- internal_bounds: return None for non-positive log-scale bounds
- SobolSampler: use per-trial dimension HashMap for concurrent safety
- JournalStorage refresh: protect with io_lock mutex, use fetch_max
- n_trials(): filter by TrialState::Complete as documented
- FloatParam: reject NaN/Infinity in validate()
- Pruners: assert n_min_trials >= 1, guard compute_percentile on empty
- Visualization: escape_js for importance chart parameter names
Low severity:
- save(): use peek_next_trial_id() from Storage trait
- csv_escape: handle carriage return per RFC 4180
- from_internal: use saturating arithmetic for stepped Int distributions
- BoolParam: bounds-check categorical index < 2
- min_max: skip NaN values with safe fallback
* ci: trigger CI on pull requests targeting any branch
- Rename stale "GridSearchSampler" panic message to "GridSampler"
- Assert concurrency > 0 in optimize_parallel to prevent deadlock
- Fix inaccurate comment in CSV export (empty for all non-complete trials, not just pruned)
Resolve symlinks and `../` traversals best-effort in `new()` and
`open()`, falling back to the original path if canonicalization fails
(e.g. file doesn't exist yet).
- Timing-based proof that trials run in parallel, not sequentially
- Atomic counter proof that multiple workers overlap simultaneously
- Panic propagation returns TaskError with panic message
- Partial failures with parallel path store only successful trials
- Remove stale #[allow(dead_code)] and #[allow(too_many_lines)] attributes
- Replace #[allow(dead_code)] with #[cfg(test)] for test-only methods
- Delete unused fill_remaining_independent() and ParamMeta.dist field
- Extract sample_float/int/categorical helpers from TpeSampler, MotpeSampler,
and SamplingEngine to shorten match-heavy sample() methods
- Extract try_fit_kdes() helper to consolidate repeated fallback blocks
- Refactor sample_tpe_float/int to take &FloatDistribution/&IntDistribution
instead of destructured args, removing #[allow(too_many_arguments)]
Fix direct path references for in-scope items (GammaStrategy,
CompletedTrial) and convert feature-gated items to plain backtick
formatting so docs build cleanly without --all-features.
- Add sampler/common.rs with distribution helpers (internal_bounds,
from_internal, to_internal, sample_random) used by 8 samplers
- Add sampler/tpe/common.rs with TPE sampling functions
(sample_tpe_float, sample_tpe_int, sample_tpe_categorical)
shared by TpeSampler, MultivariateTpeSampler, and MotpeSampler
- Remove ~940 lines of near-identical code across sampler modules
- Use Vec::with_capacity() in param_importance() and fanova_with_config()
where iteration count is known or bounded
- Fix flaky MultivariateTpeSampler doctest by increasing trials from 30
to 50
- Add Trial::into_completed() and into_multi_objective_trial() to move
fields instead of cloning 5 HashMaps/Vecs per trial completion
- Fire after_trial callback before pushing to storage, eliminating the
clone-from-storage pattern at all 4 call sites
- Optimize top_trials(n) to sort indices and clone only N trials instead
of cloning all completed trials
- Remove unused set_complete/set_pruned methods
- Replace Mutex<fastrand::Rng> with a stored seed + MurmurHash3 mixer
in RandomSampler, TpeSampler, and MotpeSampler so parallel workers
no longer serialize on a shared lock
- Add mix_seed() and distribution_fingerprint() to rng_util for
deterministic per-call RNG derivation from (seed, trial_id, distribution)
- Include an AtomicU64 call counter to disambiguate parameters that
share the same distribution within a trial
Remove the conditional guard on pruned trial assertions since the test
explicitly prunes 2 trials. Assert the expected count first, then check
summary content unconditionally to catch regressions.
- Add `CompletedTrial::validate()` checking all f64 fields are finite
- Call validate after deserializing each trial in journal storage
- Make `distribution` and `param` modules public (types already in public fields)
- Add blanket `impl Objective<V> for Fn(&mut Trial) -> Result<V, E>`
so closures work directly with `optimize`
- Rewrite optimize, optimize_async, optimize_parallel to accept
`impl Objective<V>` with before_trial/after_trial hooks
- Remove optimize_with, optimize_with_async, optimize_with_parallel
- Remove max_retries and retry logic from Objective trait
- Add explicit closure type annotations for HRTB inference
- Convert FnMut test closures to Fn via RefCell/Cell
- Delete 20 duplicate tests already covered by parameter_tests.rs
- Move 11 pure Trial unit tests into src/trial.rs
- Split remaining 84 integration tests into tests/study/ (9 modules)
- Group sampler tests into tests/sampler/ (7 modules)
- Group pruner tests into tests/pruner/ (2 modules)
- Split pruning_and_callbacks into pruning and early_stopping
- Split advanced_features into async_parallel, journal_storage, ask_and_tell, multi_objective
- Each example now requires only its own feature flag
- Trim sampler_comparison winner logic and verbose header
- Update CI workflow and README to match new example names
- Sampler, pruner, storage, parameter, and multi_objective types are no
longer re-exported at the crate root; access via module paths instead
(e.g. optimizer::sampler::TpeSampler, optimizer::parameter::FloatParam)
- Prelude continues to re-export everything for convenience
- Add module-level pub use re-exports in sampler/mod.rs and parameter.rs
- Update derive macro to reference optimizer::parameter::Categorical
- Rename sampler::differential_evolution to sampler::de
- Fix all downstream imports in tests, benches, and doctests
- Fix all rustdoc intra-doc links to use explicit module paths
Extract shared evolutionary algorithm infrastructure (genetic operators,
candidate management, Das-Dennis reference points) from NSGA-II into a
new genetic.rs module, then build two new multi-objective samplers on top:
- NSGA-III: reference-point-based niching for well-distributed fronts
on 3+ objective problems (Das-Dennis structured points, normalization,
perpendicular distance association, niching selection)
- MOEA/D: decomposition-based optimization with three scalarization
methods (Tchebycheff, WeightedSum, PBI), weight-vector neighborhoods,
and neighborhood-based mating selection
Both implement MultiObjectiveSampler with builder pattern, seeded RNG,
and SBX crossover / polynomial mutation via the shared genetic module.
Replace the internal Vec<CompletedTrial<V>> with a pluggable Storage<V>
trait. MemoryStorage is the default (no behavior change for existing
users). Behind the `journal` feature flag, JournalStorage persists
trials to a JSONL file with fs2 file locking for multi-process safety.
- Storage<V> trait with push(), trials_arc(), refresh() methods
- MemoryStorage<V> wraps Arc<RwLock<Vec<CompletedTrial<V>>>>
- JournalStorage<V> appends JSON lines with exclusive file locks
- Study::with_sampler_and_storage() general constructor
- Study::with_journal() convenience constructor (journal feature)
- Refresh from storage on create_trial() for multi-process discovery
- MSRV bumped to 1.89
Population-based optimizer with mutation + crossover. Supports three
strategies (Rand1, Best1, CurrentToBest1), configurable F and CR,
and auto-sized populations. No extra dependencies needed.
fastrand is smaller, faster, and has no dependencies. Add rng_util
helper for f64 range generation since fastrand lacks a built-in
equivalent. Migrate all samplers, KDE modules, and fANOVA to use
fastrand's concrete Rng type instead of rand's trait-based generics.
The no_effect_parameter test (and others) used unseeded random sampling,
causing spurious correlations between the noise parameter and the
objective in unlucky runs. Seed all importance tests with
RandomSampler::with_seed(42) for deterministic results.
Implement a classical Bayesian optimization sampler using a GP surrogate
with Matérn 5/2 kernel and Expected Improvement acquisition function.
Feature-gated behind `gp = ["dep:nalgebra"]`.
Use cargo hack's --partition flag to split the feature powerset across
4 parallel jobs, reducing wall-clock time for the feature-check step.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement functional ANOVA decomposition behind the `fanova` feature
flag. A self-contained random forest is trained on trial data, then
marginal predictions are used to compute per-parameter main effects
and pairwise interaction effects, normalized to sum to 1.0.
Adds Study::fanova() / fanova_with_config(), FanovaResult, and
FanovaConfig. Includes unit and integration tests covering dominant
parameters, interaction detection, consistency with correlation-based
importance, and error handling.
Add a `visualization` feature flag that generates self-contained HTML
reports with interactive Plotly.js charts for offline visualization of
optimization results. Charts include optimization history, slice plots,
parallel coordinates, parameter importance, trial timeline, and
intermediate values (learning curves).
Add to_csv(), export_csv(), and export_json() methods to Study for
exporting trial data to external visualization tools. CSV export works
without extra dependencies; JSON export requires the serde feature.
Add bench-check job (push): smoke-tests benchmarks compile and run.
Add bench-compare job (PRs): runs benchmarks on base and head commits,
compares with critcmp, and posts a comparison comment on the PR.
Add missing examples (benchmark_convergence, parameter_api) to the
examples job.
Extend TPE to handle multi-objective optimization using Pareto-based
splitting. MOTPE uses non-dominated sorting to define "good" (front 0)
vs "bad" (dominated) regions for the KDE models, replacing the
single-objective gamma-based split.
Add MultiObjectiveStudy for optimizing multiple objectives simultaneously,
backed by NSGA-II (Non-dominated Sorting Genetic Algorithm II) with SBX
crossover, polynomial mutation, and constraint-aware dominance.
New public API:
- MultiObjectiveStudy with optimize(), pareto_front(), ask()/tell()
- MultiObjectiveTrial with get(), is_feasible(), user attributes
- MultiObjectiveSampler trait for custom MO samplers
- Nsga2Sampler with builder for population size, crossover/mutation params
- ObjectiveDimensionMismatch error variant
Add micro-benchmarks for TPE, Random, and Grid samplers with varying
history sizes, end-to-end optimization benchmarks on Sphere and
Rosenbrock, and a convergence tracking example that outputs CSV data.
Includes 6 standard test functions (Sphere, Rosenbrock, Rastrigin,
Ackley, Branin, Hartmann6) with correctness tests at known optima.
Compute per-parameter importance scores by measuring the absolute
Spearman rank correlation between each parameter's values and the
objective across completed trials. Scores are normalized to sum to 1.0
and returned sorted by descending importance.
Add constraint support so that optimization problems with constraints
(e.g., "model size < 100MB") prefer feasible solutions. Convention:
constraint value <= 0.0 means feasible.
- Add constraint_values field to Trial with set_constraints/getter
- Add constraints field to CompletedTrial with is_feasible() method
- Propagate constraints through complete_trial/prune_trial
- Make best_trial() and top_trials() constraint-aware: feasible trials
rank above infeasible, infeasible ranked by total violation
BOHB combines TPE's model-guided sampling with Hyperband's budget-aware
evaluation by conditioning the TPE model on trials at specific budget
levels, producing better-calibrated parameter proposals.
Implement CMA-ES (Covariance Matrix Adaptation Evolution Strategy) as a
new sampler for continuous optimization. Uses nalgebra for matrix
operations and implements the full Hansen 2016 algorithm: mean update,
evolution paths, covariance matrix adaptation, and cumulative step-size
adaptation.
Key features:
- Builder API with configurable sigma0, population_size, and seed
- Three-phase state machine: discovery, active sampling, generation update
- Rejection sampling with bounds clipping fallback
- Log-scale and step parameter support via internal-space mapping
- Categorical parameters handled via random sampling fallback
- Numerical robustness: eigenvalue clamping, symmetry enforcement
Add SobolSampler using Owen-scrambled Sobol sequences (sobol_burley
crate) for better uniform coverage of the parameter space compared to
random sampling. Supports all distribution types including log-scale
and stepped parameters.
Add optional structured logging via the tracing crate:
- info-level spans on all optimize* methods (direction, n_trials/duration)
- info events for trial completed, new best value, trial pruned
- debug events for trial failed and parameter sampled
- Zero overhead when the feature is disabled
Manuel