2026-06-26 18:47:35 +08:00
# CLAUDE.md — Adaptive Setup Playbook for an AI Coding Assistant
> This file is written **for an AI coding assistant** (Claude Code, Cursor, Copilot Chat, etc.) so
> it can help its human build this backtesting lab **from scratch on their specific machine**.
> It also works as a generic `AGENTS.md`. If you are a human, you can follow it too — just perform
> the steps yourself instead of asking the assistant.
>
> **Golden rule for the assistant:** this stack is reproduced *from your human's own EAs and
> presets*, not from any bundled code. Never invent a trading strategy or fabricate parameters.
> When trading logic is needed, ask the human for their MQL5 source and presets.
---
## Phase 0 — Profile the device (do this first, always)
Before installing anything, detect the environment. The whole setup branches on **where MT5 runs** .
Run and read the output:
```bash
# OS + architecture
uname -a 2>/dev/null || ver # macOS/Linux print uname; Windows cmd prints `ver`
python3 --version || python --version
```
Then classify the human into **one of three topologies** and confirm it with them before proceeding:
| Topology | Python runs on | MT5 runs on | Bridge needed | Best for |
|----------|----------------|-------------|---------------|----------|
| **A — All-Windows (simplest)** | the Windows machine | the same Windows machine | none — direct, via the `MetaTrader5` pip package or local file paths | a Windows user; recommended starting point |
| **B — Unix + remote Windows** | macOS/Linux | a separate Windows box or VPS | SSH + file copy + a scheduled-task trick | someone on a Mac/Linux who already has a Windows VPS |
| **C — Unix + local Windows VM** | macOS/Linux host | Windows in a VM/Parallels/UTM | shared folder or SSH into the VM | a Mac/Linux user without a separate Windows box |
> **Recommendation to give the human:** if they are on Windows, use **Topology A** — it is by far
> the least moving parts. The official `MetaTrader5` Python package only runs on Windows and lets
> Python drive the terminal directly, so no SSH/scheduled-task plumbing is needed. Topologies B/C
> exist only because some people develop on a Mac/Linux and keep MT5 elsewhere.
Record the chosen topology — every later step references it.
---
## Phase 1 — Install the Python stack
Follow [`01-stack-and-install.md` ](01-stack-and-install.md ). In short:
1. Install **Python 3.12+** (3.13/3.14 fine). On Windows, install from python.org and tick
*"Add Python to PATH"* .
2. Create an **isolated virtual environment** in the project root (`.venv` ). Never install into the
system Python.
3. Install the libraries (`pandas` , `numpy` , `pyarrow` , `optuna` , `numba` , `sqlalchemy` , `pyyaml` ,
`lxml` , `html5lib` , `tqdm` , `requests` ). Exact commands and *why each library exists* are in doc 01.
4. On **Topology A** only, also `pip install MetaTrader5` (Windows-only package).
Verify the venv imports everything before moving on.
---
## Phase 2 — Scaffold the repository skeleton
Create the directory layout from [`02-architecture.md` ](02-architecture.md ). Build **empty modules
with docstrings and type signatures first** — do not write trading logic yet. The skeleton:
```
your-lab/
├── .venv/ # virtual environment (gitignored)
├── data/ # downloaded market data, parquet (gitignored)
├── shared/ # reusable infrastructure
│ ├── core/ # backtest engines (you fill these from your EA logic)
│ ├── indicators/ # indicator functions (RSI, ATR, EMA, your custom ones)
│ ├── instruments/ # per-symbol/broker config objects
│ ├── data/ # data loaders + MT5 report parser
│ ├── optimizer/ # objective + diverse top-N selector
│ ├── robustness/ # anti-overfit layers
│ ├── gates/ # reusable entry-filter masks
│ ├── wizard/ # pre-run interactive Q&A
│ └── mt5_pipeline/ # the MT5 bridge
├── strategies/ # one folder per strategy (your EAs + presets + iterations)
├── registry/ # approved, locked results — the source of truth
└── docs/ # this KB + your own notes
```
Mirror the responsibilities described in doc 02. Leave the engines for Phase 4.
---
## Phase 3 — Bring in the human's trading assets
**Stop and ask the human for:**
1. The **MQL5 EA** they want to test — `.mq5` source *and* compiled `.ex5` , plus any custom
indicators (`.ex5` ) the EA loads.
2. A **representative `.set` preset** (or a screenshot of the EA inputs) so you can see exactly which
inputs exist and their default values.
3. Their **broker + symbol(s)** and a **demo account** for the tester.
From the EA source and preset you (assistant) will:
- Enumerate every EA input and group them into **frozen** (not optimized) vs **tunable** (search space).
- Identify the instrument mechanics (digits, point, contract size, tick value, lot step) to build the
instrument config in Phase 5. These come from the **symbol specification** in MT5
(`Right-click symbol → Specification` ), not from guesses.
---
## Phase 4 — Build the Python mirror of the EA (the engine)
Read [`03-engine-design.md` ](03-engine-design.md ) carefully — it is the hardest and most important
part. The engine must reproduce the EA's **fill and exit logic** bar-by-bar.
- Keep the engine **strategy-agnostic** : it consumes bars + signal arrays + stop/target arrays and
simulates fills. All the *strategy* math (when to enter, where to put stops) lives in the caller.
- Implement the **intra-bar sub-tick model** (doc 03) and the **pessimistic ordering** convention.
2026-06-26 20:50:07 +08:00
This is the default bar-level exit mode — fast and exact for clean-directional setups.
- **If the EA moves its SL during a trade** (break-even, trailing, basket trailing), the bar-level
engine is NOT trustworthy: it produces a − −
§7 measured failure mode). You MUST implement the **M1 tick-level exit simulation** path: load M1
bars covering the same window as the signal bars, pass them to `engine.run(..., m1_bars=m1_bars)` ,
and the engine walks 4 synthetic ticks per M1 bar inside each higher-TF bar (direction-aware
order), separating the BE-update tick from the SL-trigger tick. This brings the gap to ~−
The engine should support BOTH modes and switch on whether `m1_bars` is provided.
2026-06-26 18:47:35 +08:00
- Match the EA's **lot/money mode** , **spread model** , and **swap model** exactly (doc 05).
- The first milestone is **1:1 fidelity on one known preset** : run the EA in MT5 on a short period,
run your Python engine on the same data/preset, and reconcile trade-by-trade until the numbers
2026-06-26 20:50:07 +08:00
line up within the **target gate for the EA's class** (doc 03 §8 table):
- Clean-directional (SL not moved intra-trade), bar-level: ≤ ~2% net gap.
- BE / trailing, **bar-level** : unattainable — do not chase this, switch to M1 tick-level.
- BE / trailing, **M1 tick-level** : ≤ ~10% net gap (residual spread/tick-path differences).
**Do not optimize anything until this passes** — an unvalidated engine optimizes noise. A
trailing/BE EA validated only on the bar-level engine is a silently-broken engine.
2026-06-26 18:47:35 +08:00
> The bundled docs use a **grid martingale** engine as the worked example because it exercises every
> hard case (pending orders, averaging, trailing on the basket, simultaneous closes). Your EA may be
> simpler (single position, signal-to-signal) or different — apply the *principles*, not the grid
> specifics.
---
## Phase 5 — Wire up configuration and inputs
Read [`05-config-and-inputs.md` ](05-config-and-inputs.md ). Build:
- An ** `InstrumentConfig` ** object per symbol/broker (real conditions), plus optional `worst_case`
and `best_case` variants for cost-stress testing.
- A **declared search space** for the strategy (parameter → range → step), kept in code *and*
documented in a `parameter-space.md` next to the strategy.
- A **wizard** that captures run settings (period, profile, trial count, DD cap, deposit) and writes
them to `wizard-answers.yaml` for reproducibility.
---
## Phase 6 — Optimizer + robustness
Read [`06-optimization-and-robustness.md` ](06-optimization-and-robustness.md ). Build the Optuna
objective (score + hard constraints), the diverse top-N selector, and at least the first robustness
checks (neighborhood/plateau, cost stress, era split). Start with a **small smoke run** (30–
trials, short period) to validate the script, then one full run in the background.
---
## Phase 7 — The MT5 bridge
Read [`07-mt5-bridge.md` ](07-mt5-bridge.md ) and implement the variant for the chosen topology:
- **Topology A:** use the `MetaTrader5` package or generate a `tester.ini` + `.set` and launch
`terminal64.exe /config:` locally. Simplest path — no SSH.
- **Topology B/C:** generate `tester.ini` + `.set` , copy to the Windows box, trigger the run via a
one-shot scheduled task (so it survives the SSH disconnect), poll for the report, copy it back.
Then implement the **report parser** (MT5 saves HTML as UTF-16) and the **comparison table** writer.
---
## Phase 8 — Run the full cycle once, end to end
Read [`08-workflow-cycle.md` ](08-workflow-cycle.md ) and run a complete loop on the human's real EA:
hypothesis → scaffold iteration → stats → minimal-scope test → optimize → verify the finalists in
MT5 → write the comparison → let the human decide. When the loop produces a number the human trusts,
the lab is working.
---
## Standing rules for the assistant
- **Never touch a validated engine to test an idea.** Fork it; prove the fork == original with the
change disabled; only then test. (Doc 04.)
2026-06-26 20:50:07 +08:00
- **For trailing/BE EAs, M1 tick-level exit simulation is mandatory.** A trailing/BE EA validated
only on the bar-level engine has a − − *not* a validated engine,
no matter how good the numbers look. Always pass `m1_bars=` to the engine for EAs that move their
SL intra-trade. (Doc 03 §7/§8.)
2026-06-26 18:47:35 +08:00
- **Heavy runs go in the background.** A full-history A/B or a full Optuna study is minutes of
compute — start it detached and poll, never block. Smoke-test first. (Doc 06.)
- **Don't promote partial searches.** A finalist must come from a *completed* search. An interrupted
Optuna study yields preliminary numbers, not verification candidates. (Doc 06, 08.)
- **MT5 is the source of truth for live numbers.** Python is for ranking and A/B. Always verify
finalists in MT5 before the human relies on a result. (Doc 03, 07.)
- **Keep secrets out of code and chat.** Broker login goes in a gitignored env file, read at runtime,
never printed. (Doc 07.)
- **Ask before destructive actions** — deleting data, force-pushing, overwriting a validated preset.
