mql5-skills/AGENTS.md

AGENTS.md — mql5-skills Project

Overview

MQL5 Agent Skills project. Creates and publishes Agent Skills conforming to the AgentSkills.io Specification.

• Project version: 0.1.0 (pinned during initial phase, not incremented per change)
• SKILL.md version: 0.1 (per-document, pinned during initial phase)
• Language: Python 3.14, uv-managed
• Dependencies: beautifulsoup4, requests

Directory Structure

mql5-skills/
├── AGENTS.md              # This file — project conventions
├── LICENSE                # MIT license
├── README.md              # Public readme
├── pyproject.toml         # uv project config
├── .python-version        # Pins Python 3.14
├── uv.lock                # Locked dependencies
├── sitemaps/              # Source sitemaps from mql5.com
│   ├── sitemap_book_en.xml   # 581 URLs → programming book
│   └── sitemap_docs_en.xml   # 4135 URLs → API reference docs
├── html_cache/            # Downloaded HTML (lossless, gitignored)
│   ├── book/              #   581 .html files
│   └── docs/              #   4135 .html files
├── scripts/               # Extraction scripts (Python)
│   └── extract.py         # Main extraction: XML → HTML → Markdown
├── resources/             # Static resources
│   └── random-user-agents.csv
├── skills/
│   └── mql5/              # The MQL5 development skill
│       ├── SKILL.md       # Skill definition (agentskills.io spec)
│       ├── scripts/
│       │   ├── mql5_helper.py            # Compile/deploy/status via Wine
│       │   ├── parse_tester_report.py    # Backtest report parser + analysis
│       │   ├── parse_optimizer_report.py # Optimization report parser + analysis
│       │   └── verify_sl_tp_formulas.py  # SL/TP risk formula verification
│       └── references/
│           ├── book/      # Programming book markdown (from sitemap_book_en.xml)
│           │   ├── 0000-book.md
│           │   ├── 00-intro/
│           │   │   ├── 0001-intro.md
│           │   │   └── pics/
│           │   └── ...
│           ├── docs/      # API reference markdown (from sitemap_docs_en.xml)
│           │   ├── 0000-docs.md
│           │   ├── 01-basis/
│           │   │   ├── 0001-basis.md
│           │   │   └── pics/
│           │   └── ...
│           └── symbol-spec/  # Broker symbol specifications (CSV)
│               ├── specs-XAUUSD.csv
│               └── specs-USDJPY.csv
├── jobs/                  # Backtest job folders
│   ├── 250013-job.md      # Job specification
│   └── ReportTester-250013/
│       ├── ReportTester-600xxxxx.html   # MT5 Strategy Tester HTML report
│       └── ReportTester-600xxxxx*.png   # Screenshots (equity, holding, MFE/MAE)
└── docs-dev/              # Development documentation
    ├── extraction.md      # Extraction workflow and script design
    ├── naming.md          # Folder/file naming conventions
    ├── skill-design.md    # SKILL.md content plan
    └── symbol-spec.md     # Symbol spec workflow

SKILL.md Convention (skills/mql5/)

Per agentskills.io spec:

• Frontmatter: name (required, max 64 chars, lowercase+hyphens), description (required, max 1024 chars)
• Body: Markdown instructions for the agent
• Optional dirs: scripts/, references/, assets/
• Focus areas: positions, orders, indicators, ticks, bars

Scripts

parse_tester_report.py

Parses MT5 Strategy Tester HTML reports. Supports three output modes:

# Text report (default)
python skills/mql5/scripts/parse_tester_report.py <report.html>

# JSON dump (raw parsed data)
python skills/mql5/scripts/parse_tester_report.py <report.html> --json

# JSON with trade analysis (--analyze includes idle_time, monthly breakdown, etc.)
python skills/mql5/scripts/parse_tester_report.py <report.html> --analyze

Key analysis fields: idle_time (HH:MM:SS flat duration across backtest period), win_loss_ratio, breakeven_win_rate, monthly, reentries, lot_pattern.

windows subcommand

Splits a backtest into N equal time windows and computes the same 7 core metrics (Profit, Expected Payoff, Profit Factor, Recovery Factor, Balance DD Rel%, Trades, Sharpe Ratio) per window. Use to detect over-fitting / regime change.

# N=1: validation — should match the full report within tolerance
python skills/mql5/scripts/parse_tester_report.py <report.html> windows --count 1

# N=4: typical analysis (quarterly for a 1.5y backtest)
python skills/mql5/scripts/parse_tester_report.py <report.html> windows --count 4

# JSON output for further processing
python skills/mql5/scripts/parse_tester_report.py <report.html> windows --count 6 --json

Conventions:

• Time boundaries are equal-length [t_start, t_end) slices, left-closed right-open. Window 0 starts at the backtest start; window N-1 ends at the backtest end. Adjacent windows do not overlap.
• A trade is assigned to the window where it OPENS (entry time). Its P&L lands at exit time, which may fall in a later window — we attribute the P&L to the opening window because that is the "decision moment" the user cares about.
• Balance DD Rel% computes MT5's STAT_BALANCE_DDREL_PERCENT (maximum relative drawdown, i.e. the largest (peak - trough) / peak % across the balance curve). For the full report this matches the HTML's Balance Drawdown Relative field exactly (44.60% vs 44.63% on 246753; the 0.03% gap is from intra-trade floating P&L not in HTML). Per the user's rule "如 Equity DD % 不可用，则以 Balance DD % 代替", the script's bal_dd_rel_pct field is this balance-based relative DD.
• Gross Profit / Gross Loss use MT5's split: each trade's exit-leg P&L goes to GP if positive or GL if non-positive; entry costs always go to GL. Matches the report exactly for window=1.
• Sharpe Ratio uses the standard per-trade HPR formula (mean/std × sqrt(N_per_year)). MT5's reported value uses a different (undocumented) annualization; the value differs from the report for window=1, but the formula is consistent across all sub-windows, so the relative ranking is still meaningful.

Outlier flags per window (z-score vs window mean):

• ▲2σ — at least one metric has |z| ≥ 2 (值得关注 — this window's value is far from the rest of the windows).
• ■EXT — at least one metric has |z| ≥ 5 (极端 — extreme outlier).
• The marker is followed by k=N (count of outlier metrics) and the metric abbreviations with their signed z (e.g. prof(+2.3σ),reco(+2.3σ)).

z = (this window's value − mean across all windows) / std. Direction is sign-bearing (+/-); the threshold is on |z|. Lower-is-better metrics (bal_dd_rel_pct) are NOT inverted — a negative z means "this window's DD is unusually low" (good for safety, neutral for consistency), a positive z means "unusually high DD" (a red flag). For all other metrics, the natural sign applies (high profit, high PF, etc. = good).

A single window with a strong outlier is a regime signal. Multiple windows each with their own outliers point to a high-variance strategy — harder to predict live performance.

Tolerances (N=1 vs report):

• 4 of 7 metrics are exact: Profit, Expected Payoff, Profit Factor, Trades.
• 3 are documented approximations:
  • Balance DD Rel% — MT5's STAT_BALANCE_DDREL_PERCENT (max relative drawdown). Our value matches the HTML's Balance Drawdown Relative field within 0.03%. Per the user's rule "如 Equity DD % 不可用，则以 Balance DD % 代替" — this is what we do.
  • Recovery Factor — downstream of bal_dd_rel_abs.
  • Sharpe Ratio — MT5's reported value is inconsistent with the textbook formula (AHPR - 1) / std_HPR × sqrt(N/year) that the MQL5 community reverse-engineers agree on (forum thread 337071). For 246753 the textbook formula gives 2.49 vs the report's 22.92 — a 9.2× gap. MT5 does not publish the actual computation. The per-trade Sharpe (sharpe_ratio_raw in the JSON output) is still a useful per-window signal.

parse_optimizer_report.py

Parses MT5 Strategy Tester Optimization XML reports (SpreadsheetML format — XML-tagged Excel workbook, also openable in LibreOffice Calc). Companion to parse_tester_report.py; same three output modes:

python skills/mql5/scripts/parse_optimizer_report.py <ReportOptimizer-*.xml>
python skills/mql5/scripts/parse_optimizer_report.py <report.xml> --json
python skills/mql5/scripts/parse_optimizer_report.py <report.xml> --analyze
python skills/mql5/scripts/parse_optimizer_report.py <report.xml> outliers [--sigma K] [--top-outliers N] [--top-normal M] [--sort ABBR_LIST] [--json]

Reads <DocumentProperties> for the strategy environment card (EA / Symbol / Period / Date range from Title, plus Deposit / Leverage / Server / MT5 build / run timestamp) and the single "Tester Optimizator Results" worksheet for one row per parameter pass. --analyze adds:

• Orthogonality: actual pass count vs expected cartesian product
• Parameter effect: which Inp* parameters actually move the result (vs. dead parameters that should be removed from optimization)
• Dead boolean parameters: bit-for-bit identical true/false groups on key metrics — cleanest signal of a parameter not wired into the EA
• Duplicate metric vectors: high count (>30%) usually points to a dead parameter
• Best passes by Profit / Profit Factor / Recovery Factor / Custom
• Trade count distribution with daily rate and correlations vs profit / drawdown (overtrading detection)

Use alongside parse_tester_report.py for the same EA: the latter explains why a specific pass performs, the former explains which pass performs and which parameters are even worth tuning.

outliers subcommand

Per-pass z-score scan over the 8 performance metrics (Result, Profit, Expected Payoff, Profit Factor, Recovery Factor, Sharpe Ratio, Custom, Equity DD %). Two disjoint sets:

• Set A — passes with AT LEAST ONE performance metric crossing ±σ in the favourable direction (higher-is-better metrics: z >= +σ; Equity DD % uses z <= -σ because low DD is good). Sorted by the configured priority chain (default: R↓, EP↓, PF↓, RF↓, SR↓, …), top --top-outliers (default 10) shown.
• Set B — passes with NO performance-metric outlier. Sorted by the same priority chain, top --top-normal (default 5) shown.

Both sets EXCLUDE passes whose Trades count is itself a low-side outlier (z <= -σ) — those have too few trades to trust, and the excluded list is shown separately.

Output: header card (includes Sort priority: … line) + per-metric reference table (mean, std, ±σ threshold) + Set A records (each split into Metrics group and Params group, with the outlier σ values annotated) + Set B records + the Excluded list.

# Default (σ=2, top 10 outliers, top 5 normal)
python skills/mql5/scripts/parse_optimizer_report.py report.xml outliers

# Custom sort priority (abbreviations: R P EP PF RF SR C DD T)
python skills/mql5/scripts/parse_optimizer_report.py report.xml outliers --sort EP,RF,R,P

# Single priority metric; rest in default order
python skills/mql5/scripts/parse_optimizer_report.py report.xml outliers --sort DD

# Tighter threshold + custom counts
python skills/mql5/scripts/parse_optimizer_report.py report.xml outliers --sigma 2.5 --top-outliers 5 --top-normal 3

# JSON output for further processing
python skills/mql5/scripts/parse_optimizer_report.py report.xml outliers --json

Sort abbreviations:

Code	Full metric	Direction
R	Result	↓ (desc)
P	Profit	↓ (desc)
EP	Expected Payoff	↓ (desc)
PF	Profit Factor	↓ (desc)
RF	Recovery Factor	↓ (desc)
SR	Sharpe Ratio	↓ (desc)
C	Custom	↓ (desc)
DD	Equity DD %	↑ (asc)
T	Trades	↓ (desc)

Default order: R↓, EP↓, PF↓, RF↓, SR↓, P↓, DD↑, C↓, T↓. DD↑ sorts ascending (lower drawdown is better); everything else descending (higher values rank first). Supply --sort ABBR_LIST as a comma-separated list to reorder; unmentioned metrics append at the end in their default positional order.

EA-agnostic by design: the script does not hardcode any input parameter name. Type inference reads <Data ss:Type="String"> from the SpreadsheetML header (boolean Inp* rendered as "true"/"false" stays a string; everything else is numeric). Input parameters are detected by column position — every column after Trades is an optimization input, regardless of whether its name starts with Inp. So an EA naming its params StopLoss / TakeProfit / UseNewsFilter parses correctly without script changes.

mql5_helper.py

MT5 development helper for compile/deploy/status via Wine:

python skills/mql5/scripts/mql5_helper.py compile FILE.mq5
python skills/mql5/scripts/mql5_helper.py check FILE.mq5     # syntax only (/s flag)
python skills/mql5/scripts/mql5_helper.py deploy FILE.mq5
python skills/mql5/scripts/mql5_helper.py status
python skills/mql5/scripts/mql5_helper.py list

MT5 paths resolved in order: $MQL5_DIR env → cwd walk-up → Program Files scan → Wine fallback.

Extraction Workflow

Two-phase pipeline (network only needed for Phase 1):

Phase 1: download  — sitemap → fetch HTML → html_cache/
Phase 2: convert   — html_cache/ → parse HTML → download images → .md files

• Script: scripts/extract.py with download, convert, debug subcommands
• HTML cache: html_cache/book/, html_cache/docs/ (lossless, gitignored)
• Output: skills/mql5/references/book/, skills/mql5/references/docs/
• Each sitemap URL → one .md file + corresponding .html in cache
• Images → pics/ subfolder within chapter dir
• Debug target: TesterStatistics page (mixed content: tables, images, code, console output)

Naming Convention

See docs-dev/naming.md for full specification. Key rules:

• 4-digit sequential prefix per file within a chapter folder
• 2-digit prefix on chapter folder names
• Chapter folder names derived from URL path segments
• Max one level of subfolder under book/ or docs/
• Each chapter folder contains a pics/ subfolder

Ad-hoc Verification

No formal test suite. Scripts are verified via temporary scripts under /tmp with hermes-verify- filename prefix. Pattern:

1. Write a focused verification script to /tmp/hermes-verify-<topic>.py
2. Import the changed functions, exercise them with known inputs
3. Run via uv run python /tmp/hermes-verify-<topic>.py (requires project venv for deps)
4. Clean up the temp file after passing

Example (from parse_tester_report.py changes):

# Create /tmp/hermes-verify-parse-tester.py with test cases
# Run:
uv run python /tmp/hermes-verify-parse-tester.py
# Clean up:
rm /tmp/hermes-verify-parse-tester.py

Git Workflow

• Conventional commits
• Do not commit .venv, pycache
• Tag releases per semver