13 KiB
07 — The MetaTrader 5 Bridge
The bridge connects your fast Python search to the gold-standard MetaTrader 5 Strategy Tester. It does four things: (1) compile your EA, (2) auto-run a backtest from a generated config, (3) parse the HTML report, (4) compare Python vs MT5 side by side.
The bridge has two flavors depending on where MT5 runs (doc CLAUDE.md Phase 0). Read the one
that matches your topology; the parsing and comparison steps are identical for both.
┌──────────────────────────── Topology A (all-Windows, simplest) ───────────────┐
Python ──► generate .set + tester.ini ──► launch terminal64.exe /config: locally ──► report.htm
└────────────────────────────────────────────────────────────────────────────────┘
┌──────────────────── Topology B/C (Unix + remote/VM Windows) ───────────────────┐
Python ──► generate .set + tester.ini ──► copy to Windows ──► scheduled-task run ──► poll ──► pull report.htm
└────────────────────────────────────────────────────────────────────────────────┘
│
parse (UTF-16 HTML → metrics dict)
│
comparison table: Python vs MT5
1. Compiling your EA
The tester runs a compiled .ex5. You compile from .mq5 with MetaEditor (ships with MT5):
- In the GUI: open the
.mq5in MetaEditor →Compile(F7). Fix errors, confirm a clean compile. - From the command line (scriptable, useful for the bridge):
:: Windows
"C:\Program Files\MetaTrader 5\metaeditor64.exe" /compile:"C:\path\to\Expert.mq5" /log
Notes that save hours:
- Custom indicators the EA calls must be compiled too and placed in
MQL5\Indicators\. - If your EA's
#includefiles live in a non-standard folder, compile the terminal in portable mode (/portable) so MetaEditor resolves includes from that terminal'sMQL5\Include\. - After compiling, the
.ex5goes inMQL5\Experts\. The tester references it by name.
2. What MT5 needs to auto-run a test
Two generated files plus the binaries already in place:
2a. The .set file (parameters)
A .set is the EA's inputs serialized as key=value lines. MT5 writes .set files as UTF-16-LE
— generate yours in the same encoding or the tester silently ignores them.
Your bridge's set generator takes the same parameter dict you backtested in Python and writes the
matching .set. The mapping from Python parameter names to EA input names is strategy-specific — keep
a small mapping table documented next to the strategy so the two tiers always agree. Watch the
enum-valued inputs (mode flags, timeframe codes): MT5 inputs are often integers, so map e.g. a
timeframe to its ENUM_TIMEFRAMES integer, a mode to its enum index.
The lot-mode guard from doc 05 applies here too: when you generate the
.set, make sure the fixed-lot input is0if you intend money mode, and handle the quote-currency lot trap. A mismatched.setis the #1 reason a verified MT5 number disagrees with Python.
2b. The tester.ini (run config)
A small INI tells the tester what to run:
[Tester]
Expert=Experts\YourEA.ex5
Symbol=YOURSYMBOL
Period=H1 ; chart timeframe (≤ the EA's signal timeframe)
Model=1 ; tick model — see below
FromDate=2024.01.01
ToDate=2026.01.01
Deposit=10000
Leverage=100
Report=report_myrun ; output HTML name
ShutdownTerminal=1 ; close the terminal when done (so the bridge knows it's finished)
[TesterInputs]
; either point at the .set, or inline the key=value inputs here
The login section ([Common]) carries the demo account; keep credentials out of code (section 5).
Tick models (the speed/accuracy dial)
| Model | Name | Accuracy | Speed (≈ 2-year run) |
|---|---|---|---|
| 0 | Every tick based on real ticks | highest (truest fills) | slowest (~10–30 min) |
| 1 | Every tick (generated from M1) | high | medium |
| 2 | 1-minute OHLC | good for non-tick-sensitive | fast (~2–3 min) |
| 4 | Open prices only | rough | fastest (~30 s) |
Use Model 2 (1-min OHLC) for routine verification; Model 0 (real ticks) for the final check on anything path-sensitive (trailing/grid). For a pure open-bar strategy even Model 4 is exact.
3. Topology A — all on Windows (recommended, simplest)
No SSH, no scheduled tasks. Two ways, easiest first:
3a. The official MetaTrader5 Python package
pip install MetaTrader5 (Windows-only) lets Python drive the terminal directly — initialize, pull
historical bars, and read symbol info. Use it for data download (section 6) and terminal control.
This is the cleanest path on Windows and removes most of the plumbing the remote topologies need.
3b. Launch the tester from a generated config
Write tester.ini, then launch the terminal pointed at it:
start "" "C:\Program Files\MetaTrader 5\terminal64.exe" /config:"C:\path\to\tester.ini"
ShutdownTerminal=1 makes the terminal close itself when the run finishes; your Python wrapper waits
for the process to exit (or for the report file to appear), then parses the report. Because everything
is local, there is no copy/poll step — read the HTML straight from the terminal's report folder.
Topology A is the right starting point for a Windows user. Get this working before considering anything remote.
4. Topology B/C — Python on Unix, MT5 on a remote/VM Windows
You develop on macOS/Linux and MT5 lives elsewhere (a Windows VPS, or a local VM). Two quirks force a specific design:
Quirk 1 — a second, portable MT5 install
If that Windows box already runs a live MT5 for something else, the tester would fight it for chart
windows. Install a separate portable MT5 in its own folder (e.g. C:\MT5_Tester\) with its own
data directory. Copy config\ (the saved login), bases\ (symbol metadata), and profiles\ from the
main terminal so the portable one auto-logs-in and knows your symbols. One-time setup.
Quirk 2 — SSH-launched processes die on disconnect
A process you start over SSH gets killed when the SSH session closes — the tester would die after a few seconds. Solution: a one-shot Windows Task Scheduler task. Your wrapper SSHes in, creates a task scheduled ~20 seconds out, and disconnects. The task fires independently, runs the full backtest, and the terminal shuts itself down.
The end-to-end flow
1. Python: parse the .set → build tester.ini (inject demo login from the env file)
2. copy tester.ini + .set to the Windows box (scp / shared folder)
3. SSH: run a small PowerShell that registers a one-shot scheduled task ~20s out, then disconnect
4. the task fires → portable terminal64.exe /config:tester.ini → backtest → save report → shut down
5. Python: poll the Windows box for the report_*.htm file
6. when present (and the terminal has exited): copy the report back
7. parse → metrics
The two tiny helper scripts that live on the Windows box (a .bat that launches the terminal and a
.ps1 that registers the scheduled task and clears old reports) are the only Windows-side code; keep
mirror copies in your repo so you can re-deploy them.
Manual fallback
If the automated run breaks, you can always: copy the .set over, RDP in, open Strategy Tester
(Ctrl+R), load the preset, Start, then Save as Report, and copy the HTML back. Slow but unblocks you.
5. Credentials — never in code or chat
Broker demo login lives in a gitignored env file (e.g. .env):
MT5_DEMO_LOGIN=...
MT5_DEMO_PASSWORD=...
MT5_DEMO_SERVER=YourBroker-Demo
The bridge reads these at runtime and injects them into tester.ini's [Common] section without
printing them. Alternatively, copying the terminal's accounts.dat into the portable install gives
auto-login with no password in any config at all — the cleanest option.
6. Downloading historical data (feeding the Python tier)
The Python engine needs the same bars MT5 uses. Two ways to get them:
- Via the
MetaTrader5package (Windows):mt5.copy_rates_range(symbol, timeframe, from, to)→ a numpy array → save as Parquet. Pull M1 for the full window; resample to higher timeframes in Python. - Via an export script in MT5: a small MQL5 script that walks
CopyRatesyear by year and writes CSV, which you then convert to Parquet. Useful when Python can't reach the terminal.
For a brand-new symbol you must first let MT5 cache its history (open a chart, set
Tools → Options → Charts → Max bars: Unlimited, scroll back) so the tester and the export have enough
bars. Confirm the exact broker symbol code (it varies: indices and metals especially) before exporting.
6a. Pull M1 even if your signal timeframe is higher — and pass it to the engine
A common mistake: download only the signal timeframe (e.g. M5/M15/H1) and assume that's enough. It is not, if your EA moves its SL during a trade (break-even, trailing, basket trailing). The bar-level engine produces a −40% to −50% net gap on those EAs (doc 03 §7 measured failure mode) because the BE update and the SL trigger fall on the same bar's opposite extreme. The fix is tick-level exit simulation, which needs the M1 bars covering the same window as your signal bars:
# Download BOTH timeframes. Same symbol, same window, exclusive end (match MT5 tester).
m1_bars = load_bars(DATA / f"{SYMBOL}_M1_{start}_{end}.parquet")
m5_bars = load_bars(DATA / f"{SYMBOL}_M5_{start}_{end}.parquet") # the signal timeframe
# Slice both to the exact same window (exclusive end matches MT5 tester's ToDate).
m1_bars = m1_bars[(m1_bars["timestamp"] >= start) & (m1_bars["timestamp"] < end)]
m5_bars = m5_bars[(m5_bars["timestamp"] >= start) & (m5_bars["timestamp"] < end)]
# Pass m1_bars to the engine — it switches to tick-level exit simulation automatically.
result = engine.run(m5_bars, signals_long, signals_short, sl, tp,
instrument, sizing, deposit, m1_bars=m1_bars)
If your EA does not move its SL intra-trade (clean market entries with a fixed SL/TP), m1_bars
can be omitted — the bar-level engine is exact for that class and faster.
7. Parsing the report
MT5 saves the Strategy Tester report as HTML encoded UTF-16-LE — not UTF-8. Read it with the right
encoding, parse with lxml/html5lib, and pull the key fields:
Total Net ProfitProfit FactorTotal TradesBalance Drawdown MaximalandEquity Drawdown Maximal(use Maximal, not Absolute — it's the peak-to-trough that matters)
Emit a metrics dict and (optionally) dump it to JSON next to the run. Validate the report is real
before trusting it (e.g. file size over some floor and a non-zero bar count) — a truncated report means
the run failed.
8. The comparison table
For every finalist, write an auto-verification.md that puts the two tiers side by side:
| Metric | Python | MT5 | Δ |
|---|---|---|---|
| Net | … | … | …% |
| Profit Factor | … | … | … |
| Equity DD max | … | … | …% |
| Total trades | … | … | …% |
Then judge the delta against the expected fidelity gap (doc 03 §7 + §8 target-gate table):
a clean-directional setup (bar-level engine, SL not moved intra-trade) should show ≤ ~2% net gap;
a trailing/BE setup on the bar-level engine is not trustworthy at all (−40% to −50% gap,
doc 03 §7 measured failure mode) — before judging the gap "expected", confirm the Python run
used M1 tick-level exit simulation (m1_bars= passed to the engine), which brings the gap into the
≤ ~10% range. Only a gap inside the §8 target gate counts as "expected"; a wider gap is a
missing-M1 / wrong-engine-mode bug, not fidelity noise. The decision rule: if the MT5 number
still clears your bar after the gap, the finalist is real; if the edge only existed in the
optimistic Python figure, discard it.
A useful warm-up calibration: pick one known preset and run the full Python-vs-MT5 comparison on it first. It tells you your stack's actual gap for your EA, so later finalists are judged against a measured baseline rather than a generic rule of thumb.
Next: 08-workflow-cycle.md — the repeatable loop that ties all of this
together.