mymt5opp/05-config-and-inputs.md

# 05 — Configuration & Input Rules

Every input to a backtest comes from one of **four declared sources**, never scattered through the
code. This is what makes runs reproducible and the engine reusable. The four:

1. **Instrument config** — everything about the symbol/broker.
2. **Search space** — every tunable parameter, its range, its step.
3. **Wizard answers** — the run settings for *this* run (period, profile, trials, deposit, …).
4. **Frozen baseline** — the parameters you are *not* optimizing, pinned to known values.

Keep these separate and you can always answer "what produced this number?" exactly.

---

## 1. Instrument config — the symbol as data

One object describes a symbol under one broker. The engine reads **everything** about the instrument
from here; nothing about ticks, spread, or swap is ever hard-coded elsewhere.

### Fields you need

| Field | Meaning | Where to find it |
|-------|---------|------------------|
| `name`, `symbol` | human label + broker symbol code | your broker |
| `point` | smallest price increment (e.g. `0.01`) | MT5 symbol spec |
| `digits` | price decimal places | MT5 symbol spec |
| `tick_size`, `tick_value` | price step and **money per lot per point** | MT5 symbol spec |
| `contract_size` | units per lot | MT5 symbol spec |
| `volume_min`, `volume_step`, `volume_max` | lot rounding/clamping | MT5 symbol spec |
| `spread_mode` | `bar_column` (real per-bar) \| `fixed_points` \| `annual_avg` | your choice per data quality |
| `spread_fixed_points` | used only with `fixed_points` | broker typical spread |
| `swap_mode` | `fixed_per_lot` \| `annual_pct` | broker / your model |
| `swap_long`, `swap_short` | per-lot-per-day rates (fixed mode) | MT5 symbol spec |
| `swap_annual_pct` | annual rate on notional (percent mode) | your model |
| `triple_swap_weekday` | the day swap is charged ×3 (often Wed) | broker |

> **Get these from MT5's symbol specification** (`Market Watch → right-click symbol → Specification`),
> not from memory. A wrong `tick_value` or `contract_size` scales every PnL by a constant and makes
> the whole backtest meaningless.

### Real / worst / best profiles

Keep **three variants** of each instrument config:

- **`real`** — your broker's actual conditions. The default for ranking.
- **`worst_case`** — wider spread, harsher swap. A finalist that survives this is robust to cost.
- **`best_case`** — tighter spread. Shows the ceiling.

Cost-stress every finalist across all three; if the edge only exists in `best_case`, it isn't real.

### Two instrument traps (both have bitten real labs)

- **Quote currency ≠ account currency.** For a pair quoted in a non-account currency (e.g. a JPY-quoted
  pair on a USD account), an EA's "money mode" lot formula can mis-scale the lot by ~the exchange rate.
  Fix: generate that pair's presets with an **explicit fixed lot** equal to the intended cash amount
  ÷ contract size, and set the money-amount input to zero. Always check the **first trade's lot**
  against an MT5 run before trusting the series.
- **Unreliable historical spread.** For some instruments (notably crypto) the recorded per-bar spread
  is garbage. Use `fixed_points` with a realistic value (and a wider `worst_case`) instead of
  `bar_column`.

---

## 2. Search space — declare every tunable parameter in one place

The parameters Optuna is allowed to vary live in **one declared structure**, with an explicit range
and step for each. Nothing tunable is hidden in the strategy body.

A search space is a mapping of `name → (low, high, step)`, plus a note of which names are integers:

```
SEARCH_SPACE = {
    "open_distance":  (0.1, 1.0, 0.05),    # float
    "grid_step":      (0.8, 3.0, 0.10),    # float
    "grid_mult":      (0.5, 1.3, 0.10),    # float
    "grid_count":     (2,   9,   1),       # int
    "break_even":     (0.0, 0.3, 0.05),    # float
    "trailing_stop":  (0.1, 0.5, 0.05),    # float
    ...
}
INT_PARAMS = {"grid_count"}
```

The optimizer turns each entry into an Optuna `suggest_float` / `suggest_int` (doc 06). Document the
**same** space in a human-readable `parameter-space.md` next to the strategy, with the *reasoning* for
each range — why this floor, why this ceiling, what you expect. That note is half the value: it stops
you (or your assistant) from re-litigating ranges every iteration and makes overfit ranges obvious.

### Rules for the search space

- **Ranges encode priors, not hope.** A range should bracket where the answer plausibly is, informed by
  the instrument's character. Absurdly wide ranges waste trials and invite overfit corners.
- **Step matters.** Too fine a step explodes the space; too coarse misses optima. Match the step to
  what's meaningful (e.g. a 0.05% grid step, a 1-unit count).
- **What is NOT in the space is a decision.** Explicitly list parameters you deliberately exclude
  (because prior tests showed ~0 impact, or because changing them is a *different* hypothesis). An
  empty exclusion list usually means you haven't thought about it.
- **Timeframe is usually fixed per iteration.** Searching across timeframes in one study is
  "timeframe-shopping" and overfits; pick the primary timeframe in the stats phase (doc 08) and lock it.

---

## 3. Frozen baseline — pin what you don't optimize

The parameters outside the search space still need *values*. Pin them to the validated baseline (your
known-good preset) so the optimizer varies only the few things under test, against an otherwise-known
configuration. The run script merges sampled parameters **onto** this frozen baseline before building
the engine params. This keeps each study **small and interpretable** — you're measuring the effect of
the few tuned knobs, not drowning in a 30-dimensional search.

---

## 4. Lot / money mode — the input that silently breaks results

Position sizing deserves its own section because it is the most common source of "my Python and MT5
disagree by a constant factor" bugs. Expose money-based sizing through **two inputs**, plus the
fixed-lot guard:

| Input | Meaning |
|-------|---------|
| `*_LotAmount` | the cash base. Lot = `LotAmount / open_price / contract_size`. |
| `*_LotBalance` | the account balance this base is calibrated for (the scaling window). |
| `*_Lot` | a fixed lot in lots. **Must be `0`** to enable money mode — otherwise the EA uses the fixed lot and **ignores** `LotAmount`. |

Behavior:

| `LotAmount` | `LotBalance` | Mode | Result |
|-------------|--------------|------|--------|
| X | **0** | **static** (default for testing) | lot always sized from cash X, independent of balance |
| X | Y | **dynamic** | lot = base × (current_balance / Y) — grows/shrinks with the account |

### Rules

- **Test with the static lot by default** (`LotBalance = 0`). Compounding (dynamic) is a *production*
  choice; for research you want a clean, balance-independent measurement.
- **Always confirm `*_Lot = 0`** when you intend money mode. A stray non-zero fixed lot is a classic
  silent bug — the run "works" but sizes every trade wrong.
- For a grid, levels multiply from the **base** lot via the grid multiplier (or ATR/other); the base
  comes from the money rule above.
- Mirror whatever mode you choose **identically** in the Python engine and the generated `.set`, or the
  two tiers won't agree.

---

## 5. Wizard — capture run settings once, save them for reproducibility

Before a run, a small interactive **wizard** asks the handful of settings that change run-to-run and
writes them to `wizard-answers.yaml`. That YAML *is* the reproducibility record: anyone (including
future-you) can see exactly what produced an iteration's numbers.

Common wizard questions:

```
period_start / period_end     # the backtest window
instrument_profile            # real / worst_case / best_case
trials                        # Optuna trial budget (e.g. 300 / 1000 / 5000)
max_dd_ccy                    # hard drawdown cap for the constraint
top_n_verify                  # how many finalists to send to MT5
fixed_lot / initial_deposit   # sizing context
```

Design notes:
- Give every question a sensible **default** so a fast run is just pressing Enter.
- Make the wizard **extensible**: a base set of common questions plus a per-strategy hook for extra
  ones (e.g. a grid wizard adds depth/multiplier defaults).
- **Write the answers to disk** next to the iteration. A run with no saved settings is not reproducible
  and shouldn't be promoted.

---

## Putting it together

A single run is fully specified by: **which instrument config** (symbol + profile), **which search
space** (the tunable parameters), **which frozen baseline** (the pinned rest), and **which wizard
answers** (the window, trials, deposit). Four declared sources, each in its own place. That is the
whole "separate inputs" discipline — and it's why any run in this lab can be explained and reproduced
exactly.

Next: [`06-optimization-and-robustness.md`](06-optimization-and-robustness.md) — turning the search
space into an Optuna study and not fooling yourself with the result.