04 — Isolation Rules

The rules that keep a research lab trustworthy as it grows. Every one exists because the alternative quietly corrupts results. Treat them as non-negotiable; they cost a little friction and save you from optimizing on a silently-broken engine or shipping a number you can't reproduce.

Rule 1 — The validated engine is FROZEN

Once an engine reproduces your EA within the expected fidelity gap (doc 03 §8), it becomes the satisfactory baseline and is frozen. Frozen means: you do not edit it to test an idea.

Why so strict: the engine's validation was expensive (trade-by-trade reconciliation against MT5). A one-line "quick tweak" to test a hypothesis can silently shift fills across millions of bars, and now every result the engine produces is suspect — including the ones you already trusted. Freezing protects the validation from regressions you won't notice until much later.

Rule 2 — Engine changes happen in an ISOLATED FORK

Any hypothesis that needs the engine to behave differently is implemented as a separate engine file — a copy of the frozen engine plus the minimal hook for the experimental change. Never by editing the frozen file.

shared/core/grid_engine.py            ← frozen, never touched
shared/core/grid_engine_<idea>.py     ← fork: copy + one experimental hook

The process is fixed:

Fork = exact copy of the frozen engine + the experimental change, defaulting to OFF.
Regression-verify: run the fork with the change disabled and confirm it reproduces the frozen engine 1:1 — identical Net, drawdown, and trade count — on identical data. If it doesn't, your copy isn't clean; fix that before testing anything.
A/B the hypothesis: fork-with-change vs frozen-baseline on the same data.
Decide: significant, robust improvement → consider promotion (Rule 3). Otherwise delete the fork — revert is just removing a file, the core was never touched.

The regression-verify in step 2 is the linchpin. It proves your fork differs from the baseline only in the one thing you're testing, so the A/B measures the hypothesis and nothing else.

Rule 3 — Promoting a fork to the new baseline is a deliberate, rare event

A fork stays a fork until it has passed a full round: A/B + optimization + MT5 verification + an explicit human decision. Only then does it replace the frozen engine as the new baseline — and at that moment every preset/EA built on that engine must be re-validated, because the ground shifted.

This is never automatic. No script and no agent swaps the baseline engine on its own. It is a manual procedure under human supervision, precisely because it invalidates prior numbers and forces re-verification.

Rule 4 — Instruments are isolated as data, not code

Everything symbol- or broker-specific lives in an InstrumentConfig object (doc 05), never in the engine and never hard-coded in a strategy. The engine pulls tick value, spread model, swap, and lot steps from the config.

Consequences:

Testing the same strategy on a different symbol = a different config, zero engine changes.
Cost-stress testing = real / worst_case / best_case variants of the same config.
A broker change = edit one file.

If you ever find yourself writing if symbol == ... in the engine, that logic belongs in the config.

Rule 5 — Strategies are isolated folders; iterations copy, not import

Each strategy is a self-contained folder (doc 02 §4). Within it, each iteration (one research attempt) carries its own snapshot of the run script, search space, and answers.

Iterations copy glue code rather than importing a shared "current" version. That way an iteration from months ago still runs and reproduces exactly, even after you've changed how you do things. Reproducibility beats DRY here — these are lab notebooks, not production code.
Shared infrastructure (engine, indicators, instruments, optimizer) is imported normally; it is import-stable by Rule 1.

Rule 6 — Gates and robustness sit OVER the frozen engine, read-only

Two kinds of add-on never modify the engine:

Gates (shared/gates/) are boolean masks AND-ed into the signal before it reaches the engine: allow = directional_signal & ~block_condition. A regime filter, a time-of-day filter, an exhaustion filter — all are caller-side masks. The engine still just consumes a signal array.
Robustness layers (shared/robustness/, doc 06) are read-only analyses over a finished result (the study, the trade list, the equity curve). They never touch the engine or mutate results.

Keeping both outside the engine means you can add or remove a filter or a check without re-validating the engine.

Rule 7 — The registry is curated, locked, and append-only

registry/ holds only results that passed all three gates: Python search → MT5 verification → human approval. Rules for it:

Append-only in spirit: you don't edit an approved entry; a new finding is a new entry.
Locked: registry entries are not casual scratch space. Treat them like committed releases.
Self-documenting: each entry records the params, the Python metrics, the MT5 report, and the context (period, instrument, why it was approved).

The registry is the trustworthy slice of everything you ever tried. Scratch experiments you don't intend to keep go in a results/ scratchpad, never the registry.

Rule 8 — Heavy compute runs detached; never relaunch a running job

A full-history A/B (millions of bars) or a full Optuna study is minutes of compute and will block a foreground shell or hit a timeout. Discipline:

Always run heavy jobs in the background, then poll (read the study's trial count, tail the output file). Don't sit blocked online.
Smoke-test first: a tiny run (e.g. 30–40 trials, a short period) validates the script end-to-end before you commit to one full run.
Dedup guard: before launching, check whether the same job is already running. Relaunching on a perceived timeout spawns orphan duplicates that thrash the CPU and corrupt nothing but waste everything. One study with N workers beats N competing scripts.

Why these rules pay off

Individually each rule is a small constraint. Together they guarantee three properties that a pile of ad-hoc scripts never has:

Trust — the engine that produced a number is the same validated engine, every time.
Reproducibility — any past result can be re-run from its own snapshot.
Fast, safe revert — a failed idea is a deleted fork, not a half-removed change festering in the core.

Next: 05-config-and-inputs.md — how the test inputs themselves are kept separate and declared.

6.7 KiB Raw Permalink Blame History Unescape Escape