06 — Optimization (Optuna) & Robustness

How to turn the declared search space into a Bayesian search that finds good parameters fast — and how to avoid the trap that ruins most optimization: overfitting to the past. A great backtest is easy; a robust one is the job.

1. Why Optuna (and not a grid)

A parameter grid is exponential: 8 parameters × 10 values each = 100 million combinations. Even at "seconds per backtest" that's infeasible, and most of it is wasteland.

Optuna (https://optuna.org, docs https://optuna.readthedocs.io) uses a TPE sampler (Tree- structured Parzen Estimator) that learns the shape of the objective as it goes and spends trials where results are promising. You get a strong optimum in hundreds to a few thousand trials.

Install: pip install optuna (doc 01). Core concepts you'll use:

Study — one optimization. Maximizes (or minimizes) an objective. Persisted to SQLite so it can be resumed, inspected mid-run, and parallelized.
Trial — one parameter sample + its score. trial.suggest_float/int(name, low, high, step=…) draws each parameter; trial.set_user_attr(...) stashes the metrics for later inspection.
Sampler — TPE by default; fine to start.

import optuna
study = optuna.create_study(
    direction="maximize",
    storage="sqlite:///study.db",     # persist → resume, inspect, parallelize
    study_name="my_iter",
    load_if_exists=True,
)
study.optimize(objective, n_trials=1000, n_jobs=4)   # n_jobs = parallel workers

2. The objective function

The objective receives a trial, samples parameters from the search space, runs the engine, and returns a score. Structure it in four clear steps:

def objective(trial):
    1. sampled  = suggest_params(trial)               # from SEARCH_SPACE (doc 05)
    2. params   = build_params(sampled, frozen_baseline)
    3. result   = engine.run(params, bars, instrument, deposit)
       metrics  = compute_metrics(result)
    4. score, violations = score_metrics(metrics, sampled)
       for k, v in metrics.items(): trial.set_user_attr(k, v)
       trial.set_user_attr("violations", violations)
       return score

Score design

A good single-objective score balances reward against risk, for example:

score = Net − DD_WEIGHT × EquityDrawdown      # reward profit, punish the drawdown that earned it

The weight makes the optimizer prefer a slightly smaller, much calmer equity curve over a fragile spike. (Optuna also supports true multi-objective studies — e.g. maximize Net and minimize DD on a Pareto front — but a weighted single objective is simpler and usually enough to start.)

Constraints as hard rejections, not soft nudges

Some conditions should disqualify a trial outright, not just dock points. Implement them as a large penalty per violation so violators sort to the bottom:

violations = []
if metrics.trades        < MIN_TRADES:      violations.append("too few trades")
if metrics.profit_factor < MIN_PF:          violations.append("PF too low")
if metrics.equity_dd     > HARD_DD_CAP:     violations.append("DD over cap")
score = base_score − PENALTY * len(violations)   # PENALTY huge, e.g. 1e6

Typical constraints worth enforcing:

Minimum trade count (a hard floor, calibrated to the window — e.g. ≥25–30 active trades per year). Few-trade, high-PF results are almost always overfit luck, not edge.
Minimum Profit Factor.
Hard drawdown cap in account currency (e.g. ≤35% of deposit).
A path-independent fragility guard. For a martingale grid, compute the worst-case floating drawdown if a full series accumulates straight down an N% drop, and reject if it's worse than the baseline's. This catches "the backtest never hit the bad path, but the configuration is a time bomb" — the failure mode a pure historical backtest can't see.

3. Don't pick the top-N by score — pick the top-N diverse

Optuna converges: the top 10 trials by score are usually near-clones of one peak. Verifying 3 clones in MT5 tells you nothing about robustness. Instead, pick meaningfully different finalists.

Greedy diverse selection:

1. filter trials by the constraints (drop violators)
2. sort by score
3. start the selected set with the single best trial
4. repeatedly add the remaining trial with the MAXIMUM parameter-distance to the already-selected set,
   lightly weighted by its rank so you still prefer strong scores
5. stop at N (2–3 is plenty)

Parameter distance = average normalized difference across all parameters (normalize each to [0,1]: numeric by min–max, boolean to 0/1, categorical by index). The result is 2–3 finalists that are each good but reach the result by different parameter regions — if they all survive MT5, you have real robustness, not one lucky corner.

4. Robustness layers — proving the result isn't overfit

These are read-only analyses over a finished result (the study, the trade list, the equity curve). They never touch the engine. Run them on finalists before spending MT5 time. Treat them as report-only signals by default; tighten into hard gates as you gain confidence.

Layer	Question it answers	Method
Stability region	Is the winner on a plateau of good configs, or a lone spike?	Cluster the study's top trials by parameter distance; prefer a finalist from the center of a dense good region, not an isolated peak.
Neighborhood / sensitivity	Does a small parameter nudge destroy the edge?	Perturb each lever ±1 step; demand all neighbors stay profitable. A fragile optimum fails this.
Walk-forward (WFE)	Does the edge hold out-of-sample?	Split into in-sample/out-of-sample windows; with the finalist's fixed params (no re-fit), compute `OOS_metric / IS_metric`. Add a purge gap between IS and OOS so leakage can't help.
Monte-Carlo permutation	How lucky was the trade order?	Shuffle trade order, drop a fraction, jitter PnL; build a distribution of drawdown/Net. A finalist whose real drawdown sits in the ugly tail is fragile.
Deflated Sharpe (DSR)	Is the Sharpe real after testing thousands of configs?	Adjust the observed Sharpe for the number of trials (multiple-testing). Many trials inflate the best result even under pure noise; DSR estimates the probability the true Sharpe beats a deflated benchmark.
Era split	Does the edge exist in both halves of history?	Run the finalist on era-1 vs era-2 separately; an edge present in only one era is regime-luck.
Cost stress	Does it survive realistic and adverse costs?	Re-run on `real` / `worst_case` / `best_case` instrument profiles (doc 05). Edge only in `best_case` = no edge.

These are standard quant techniques, not exotic. Walk-forward and Monte-Carlo catch the everyday overfit; the Deflated Sharpe specifically counters the fact that an exhaustive search will surface an impressive-looking config from noise. References worth reading: López de Prado on the Deflated Sharpe Ratio and the dangers of backtest overfitting.

5. What qualifies as a "finalist"

Be strict — MT5 time is the expensive resource, and a weak finalist wastes it:

From a completed search only. An interrupted/partial Optuna study yields preliminary numbers, not verification candidates. The README status must reflect the real process state, never "continues" if the process is dead.
Passes the hard constraints (trades, PF, DD cap, fragility guard).
Not concentrated in one year. If a single year carries >~40% of total Net, it's fragile — one regime is doing all the work.
Honest about flat periods. A gate that sat out a bad year "survived by not trading" — describe it that way, don't dress it up as robust profit.
Survives the robustness layers you've chosen to enforce.
Converge to 2–3 diverse finalists, not 5–6. More finalists is usually indecision, not rigor.

6. Running searches without melting your machine (operational discipline)

Smoke first. A 30–40 trial run on a short period validates the whole script (data load → engine → scoring → storage) in under a minute. Only then launch the full study.
Run heavy studies in the background and poll the trial count in study.db; don't block a shell on a multi-minute run.
One study, many workers (n_jobs=N) beats N competing scripts. Cap heavy concurrency at your performance-core count.
Dedup guard: never relaunch a study you think timed out without checking it isn't still running — duplicates thrash the CPU and waste hours.

Next: 07-mt5-bridge.md — verifying the finalists in the real terminal.

8.7 KiB Raw Permalink Blame History Unescape Escape