diff --git a/mcp-mt5-quant-macos-arm64.tar.gz b/mcp-mt5-quant-macos-arm64.tar.gz new file mode 100644 index 0000000..96f53a4 Binary files /dev/null and b/mcp-mt5-quant-macos-arm64.tar.gz differ diff --git a/mcp-package/README.md b/mcp-package/README.md new file mode 100644 index 0000000..8c19f1e --- /dev/null +++ b/mcp-package/README.md @@ -0,0 +1,247 @@ +# MT5-Quant + +**MCP server for MT5 strategy development on macOS/Linux.** 85 tools to compile, backtest, analyze, optimize, debug crashes, and manage MQL5 Expert Advisors — no Windows required. + +``` +You: "Backtest MyEA Jan-Mar, what caused the February drawdown?" + +Claude: [compile → clean → backtest → analyze 1,847 deals] + → Feb 14: BUY grid at L6, locking lot 1.75× base + → Cutloss fired 17 points later + → Recommendation: cap locking multiplier to ≤1.2× +``` + +## Why MT5-Quant + +| | MT5-Quant | Other MT5 MCPs | QuantConnect | +|---|---|---|---| +| Backtest pipeline | ✅ Full | ❌ | Cloud only | +| Deal-level analytics | ✅ 15+ dims | ❌ | ❌ | +| MQL5 compilation | ✅ | ❌ | ❌ | +| macOS/Linux native | ✅ | Windows only | Cloud | +| Optimization | ✅ Background | ❌ | ✅ Paid | +| Crash debugging | ✅ Wine/MT5 diagnostics | ❌ | ❌ | + +## Quick Install + +### 1. Download & Setup + +```bash +curl -L -o mt5.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz +tar -xzf mt5.tar.gz +bash scripts/setup.sh +``` + +### 2. Register MCP Server + +| Platform | Command / Config | Docs | +|----------|------------------|------| +| **Claude Code** | `claude mcp add mt5-quant -- $(pwd)/mt5-quant` | [Setup →](docs/QUICKSTART.md) | +| **Windsurf** | Edit `~/.codeium/windsurf/mcp_config.json` | [WINDSURF.md →](docs/WINDSURF.md) | +| **Cursor** | Edit `~/.cursor/mcp.json` or use Settings → MCP | [CURSOR.md →](docs/CURSOR.md) | +| **VS Code** | Edit `.vscode/mcp.json` or run `MCP: Add Server` | [VSCODE.md →](docs/VSCODE.md) | +| **Antigravity** | Agent Panel → ... → MCP Servers → Edit configuration | [ANTIGRAVITY.md →](docs/ANTIGRAVITY.md) | + +> **Note:** Use absolute paths like `/Users/name/mt5-quant/mt5-quant` or `$(pwd)/mt5-quant`, not relative paths like `./mt5-quant`. + +## Quick Start + +``` +Run a backtest on MyEA from 2025.01.01 to 2025.03.31 +``` + +The AI runs the full pipeline: compile → clean cache → backtest → extract → analyze. + +## Documentation + +| Doc | Purpose | +|-----|---------| +| [QUICKSTART.md](docs/QUICKSTART.md) | Complete setup for macOS/Linux | +| [WINDSURF.md](docs/WINDSURF.md) | Windsurf IDE setup | +| [CURSOR.md](docs/CURSOR.md) | Cursor IDE setup | +| [VSCODE.md](docs/VSCODE.md) | VS Code setup | +| [ANTIGRAVITY.md](docs/ANTIGRAVITY.md) | Antigravity IDE setup | +| [CONFIG.md](docs/CONFIG.md) | Configuration reference | +| [TOOLS.md](docs/MCP_TOOLS.md) | All 75 tools documented | +| [ARCHITECTURE.md](docs/ARCHITECTURE.md) | Design and internals | +| [TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) | Common issues | +| [REMOTE_AGENTS.md](docs/REMOTE_AGENTS.md) | Linux optimization agents | + +## MCP Tools (75) + +### Core workflow + +| Tool | Description | +|------|-------------| +| `run_backtest` | Full pipeline: compile → clean → backtest → extract → analyze | +| `run_optimization` | Genetic optimization (background, returns immediately) | +| `get_optimization_results` | Parse optimization results after MT5 finishes | +| `analyze_report` | Read `analysis.json` from any report directory | +| `compare_baseline` | Compare report vs baseline, return winner/loser verdict | +| `compile_ea` | Compile MQL5 EA via MetaEditor | +| `list_experts` | List all EAs in MQL5/Experts directory | +| `list_indicators` | List all indicators in MQL5/Indicators directory | +| `list_scripts` | List all scripts in MQL5/Scripts directory | +| `healthcheck` | Quick server health check | + +### Granular Analytics (individual analysis) + +| Tool | Description | +|------|-------------| +| `analyze_monthly_pnl` | Monthly P/L breakdown only | +| `analyze_drawdown_events` | Drawdown events and causes only | +| `analyze_top_losses` | Worst losing deals only | +| `analyze_loss_sequences` | Consecutive loss patterns only | +| `analyze_position_pairs` | Position hold time and P/L pairs | +| `analyze_direction_bias` | Buy vs Sell performance | +| `analyze_streaks` | Win/loss streak analysis | +| `analyze_concurrent_peak` | Peak simultaneous positions | + +Use these for targeted analysis, or `analyze_report` to run all at once. + +### Deal-Level Analytics (New) + +| Tool | Description | +|------|-------------| +| `list_deals` | List individual deals with filters (type, profit range, volume, dates) | +| `search_deals_by_comment` | Full-text search in deal comments (e.g., "Layer #3") | +| `search_deals_by_magic` | Filter deals by EA magic number | +| `analyze_profit_distribution` | Profit histogram: small/medium/large wins and losses | +| `analyze_time_performance` | Performance by hour of day and day of week | +| `analyze_hold_time_distribution` | Hold time buckets + correlation with profit | +| `analyze_layer_performance` | Grid/martingale layer analysis from comments | +| `analyze_volume_vs_profit` | Volume correlation + performance by lot size | +| `analyze_costs` | Commission and swap impact on profitability | +| `analyze_efficiency` | Profit per hour/day, annualized return, trade frequency | + +### Monitoring + +| Tool | Description | +|------|-------------| +| `verify_setup` | Check Wine/MT5 paths, Wine version, and EA/set file counts | +| `get_backtest_status` | Check live progress of a running backtest pipeline | +| `get_optimization_status` | Check live state of a background optimization job | +| `list_jobs` | All optimization jobs with compact status in one call | + +### Reports & logs + +| Tool | Description | +|------|-------------| +| `list_reports` | Compact table of all runs with key metrics — no full analysis needed | +| `get_latest_report` | Get most recent report with optional equity chart | +| `search_reports` | Find reports by EA, symbol, date range, or profit criteria | +| `get_report_by_id` | Get specific report by ID with equity chart | +| `get_reports_summary` | Aggregate stats: counts, averages, pass rates | +| `get_best_reports` | Top N reports sorted by any metric (profit factor, drawdown, etc.) | +| `search_reports_by_tags` | Find reports by tags | +| `search_reports_by_date_range` | Query by backtest date range | +| `search_reports_by_notes` | Full-text search in report notes | +| `get_reports_by_set_file` | Find all reports using a specific .set file | +| `get_comparable_reports` | Find comparable reports (same EA/symbol/timeframe) | +| `tail_log` | Read last N lines of any log; `filter=errors` to see only failures | +| `prune_reports` | Delete old report directories, keep last N (skips `_opt` dirs) | + +### History & baseline + +| Tool | Description | +|------|-------------| +| `archive_report` | Convert one report dir → compact JSON entry in `backtest_history.json`, optionally delete source | +| `archive_all_reports` | Bulk-archive all report dirs then optionally delete them; keeps N newest safe | +| `get_history` | Query history with filters (EA, symbol, verdict, profit, DD) and sort options | +| `annotate_history` | Attach verdict / notes / tags to any history entry | +| `promote_to_baseline` | Write a history entry or report to `baseline.json` for compare_baseline | + +### Cache management + +| Tool | Description | +|------|-------------| +| `cache_status` | MT5 tester cache size breakdown by symbol — check before cleaning | +| `clean_cache` | Delete tester cache files; supports per-symbol and `dry_run` | + +### Pre-flight & Validation + +| Tool | Description | +|------|-------------| +| `get_active_account` | Get current MT5 account session (login, server, available symbols) | +| `check_symbol_data_status` | Validate symbol has sufficient history data for date range | +| `check_mt5_status` | Check if MT5 terminal is installed and ready | +| `validate_ea_syntax` | Pre-compile syntax check without running full compilation | + +### Debugging & Diagnostics (New) + +| Tool | Description | +|------|-------------| +| `diagnose_wine` | Check Wine installation, version, and prefix health | +| `get_mt5_logs` | Get MT5 terminal, tester, or MetaEditor logs with filtering | +| `search_mt5_errors` | Search logs for error patterns (crash, exception, access violation) | +| `check_mt5_process` | Check if MT5 processes are running, get PID, CPU, memory usage | +| `kill_mt5_process` | Kill stuck MT5 processes (force=true for wineserver) | +| `check_system_resources` | Check disk space, memory, CPU availability | +| `validate_mt5_config` | Validate terminal.ini and tester configuration files | +| `get_wine_prefix_info` | Get Wine prefix details: Windows version, installed programs, registry | +| `get_backtest_crash_info` | Investigate backtest failures: incomplete markers, missing deals.csv, errors | + +### Project Management + +| Tool | Description | +|------|-------------| +| `init_project` | Scaffold new MQL5 project with templates (scalper/swing/grid/basic) | +| `create_set_template` | Generate .set parameter file from EA input variables | +| `export_report` | Export backtest report to CSV, JSON, or Markdown | + +### History & Comparison + +| Tool | Description | +|------|-------------| +| `get_backtest_history` | List all backtests for EA/symbol with summary metrics | +| `compare_backtests` | Compare 2+ backtest results side-by-side with analysis | + +### .set file — read / write + +| Tool | Description | +|------|-------------| +| `list_set_files` | All .set files in tester profiles dir with sweep stats and combination counts | +| `read_set_file` | Parse UTF-16LE `.set` file → structured JSON params | +| `write_set_file` | Write full params dict → UTF-16LE `.set` with `chmod 444` | +| `patch_set_file` | Update specific params in-place, return diff — replaces read→edit→write | +| `clone_set_file` | Copy `.set` to new path with optional overrides in one call | + +### .set file — analysis & generation + +| Tool | Description | +|------|-------------| +| `describe_sweep` | Swept params, value counts, and total optimization combinations | +| `diff_set_files` | Side-by-side diff of two `.set` files — only changed params returned | +| `set_from_optimization` | Generate a clean backtest `.set` from `get_optimization_results` params; optionally narrow sweep | + +### Search & Discovery + +| Tool | Description | +|------|-------------| +| `search_experts` | Search EAs by name pattern across all directories | +| `search_indicators` | Search indicators by name pattern | +| `search_scripts` | Search scripts by name pattern | +| `copy_indicator_to_project` | Copy indicator to project directory | +| `copy_script_to_project` | Copy script to project directory | + +Full schema: [docs/MCP_TOOLS.md](docs/MCP_TOOLS.md) + +## Troubleshooting + +Run `verify_setup` from Claude first — it checks all paths and returns actionable hints. + +For crashes or unexplained failures during backtest/compile/optimization: +- `diagnose_wine` — Check Wine installation and prefix health +- `search_mt5_errors` — Find crash causes in logs +- `check_mt5_process` + `kill_mt5_process` — Detect and kill stuck processes +- `get_backtest_crash_info` — Investigate failed backtest reports + +**[Full Troubleshooting Guide →](docs/TROUBLESHOOTING.md)** + +--- + +## License + +MIT + +--- diff --git a/mcp-package/config/example.set b/mcp-package/config/example.set new file mode 100644 index 0000000..60518ac --- /dev/null +++ b/mcp-package/config/example.set @@ -0,0 +1,27 @@ +; MT5 optimization .set file — example +; Format: param=current_value||start||step||stop||Y (Y=sweep this param) +; param=current_value||N (N=fixed, no sweep) +; +; MT5-Quant handles encoding automatically: +; - Converts to UTF-16LE (MT5 strips ||Y flags from UTF-8 files) +; - Sets read-only flag after writing +; - Resets OptMode in terminal.ini before launch +; +; Copy to your project dir and point --set to it. + +; ── Entry parameters (sweep) ───────────────────────────────────────────────── +Min_Entry_Confidence=0.610||0.580||0.010||0.650||Y +Max_Entry_Spread=30||10||5||50||Y + +; ── Take-profit and stop-loss (sweep) ──────────────────────────────────────── +TP_Pips=400||300||50||600||Y +SL_Pips=800||600||100||1200||Y + +; ── Grid / martingale settings (fixed) ─────────────────────────────────────── +MaxLayers=6||N +LotMultiplier=1.5||N +GridStep_Pips=150||N + +; ── Risk management (fixed) ────────────────────────────────────────────────── +Max_DD_Percent=15.0||N +CutLoss_Percent=25.0||N diff --git a/mcp-package/config/mt5-quant.example.yaml b/mcp-package/config/mt5-quant.example.yaml new file mode 100644 index 0000000..a51a051 --- /dev/null +++ b/mcp-package/config/mt5-quant.example.yaml @@ -0,0 +1,45 @@ +# mt5-quant configuration (flat key format — no YAML nesting) +# Copy this file to config/mt5-quant.yaml and fill in your paths. + +# ── MT5 / Wine paths ────────────────────────────────────────────────────────── + +# macOS (CrossOver / MetaTrader 5.app) +wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64" +terminal_dir: "$HOME/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5" +experts_dir: "$HOME/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5/MQL5/Experts" +tester_profiles_dir: "$HOME/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5/MQL5/Profiles/Tester" +tester_cache_dir: "$HOME/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5/Tester" + +# Linux (Wine) +# wine_executable: "/usr/bin/wine64" +# terminal_dir: "$HOME/.wine/drive_c/Program Files/MetaTrader 5" +# experts_dir: "$HOME/.wine/drive_c/Program Files/MetaTrader 5/MQL5/Experts" +# tester_profiles_dir: "$HOME/.wine/drive_c/Program Files/MetaTrader 5/MQL5/Profiles/Tester" +# tester_cache_dir: "$HOME/.wine/drive_c/Program Files/MetaTrader 5/Tester" + +# ── Display ─────────────────────────────────────────────────────────────────── +# auto: GUI on macOS CrossOver, Xvfb on Linux without $DISPLAY +# gui: always use display (requires active session) +# headless: always use Xvfb (requires xvfb-run) +display_mode: auto + +# ── Project ─────────────────────────────────────────────────────────────────── +# Root directory of your EA project (where .mq5 and .set files live) +project_dir: "/path/to/your/ea/project" + +# ── Backtest defaults ───────────────────────────────────────────────────────── +# These are fallbacks when not passed as MCP tool arguments +backtest_symbol: "EURUSD" +backtest_deposit: 10000 +backtest_currency: USD +backtest_leverage: 100 +backtest_model: 0 +backtest_timeframe: H1 +backtest_timeout: 900 + +# ── Optimization ────────────────────────────────────────────────────────────── +opt_log_dir: /tmp +opt_min_agents: 1 + +# ── Reports output ──────────────────────────────────────────────────────────── +reports_dir: "/path/to/your/ea/project/reports" diff --git a/mcp-package/config/mt5-quant.yaml b/mcp-package/config/mt5-quant.yaml new file mode 100644 index 0000000..6f00be4 --- /dev/null +++ b/mcp-package/config/mt5-quant.yaml @@ -0,0 +1,45 @@ +# mt5-quant configuration — generated by scripts/setup.sh +# Re-run scripts/setup.sh to regenerate, or edit manually. + +mt5: + # Path to Wine / CrossOver wine64 binary + wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64" + + # MT5 installation directory (inside the Wine prefix) + terminal_dir: "/Users/masdevid/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5" + + # Where MT5 looks for Expert Advisor binaries (.ex5) + experts_dir: "/Users/masdevid/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5/MQL5/Experts" + + # Where MT5 reads/writes .set files during backtests + tester_profiles_dir: "/Users/masdevid/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5/MQL5/Profiles/Tester" + + # MT5 tester cache directory (cleared before each run) + tester_cache_dir: "/Users/masdevid/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5/Tester" + +display: + # headless: true → Xvfb (Linux only) + # headless: false → GUI visible + # headless: auto → macOS=GUI, Linux with $DISPLAY=GUI, Linux without=Xvfb + mode: auto + + # Virtual display for Xvfb (Linux headless only) + xvfb_display: ":99" + xvfb_screen: "1024x768x16" + +backtest: + symbol: "XAUUSD" + deposit: 10000 + currency: "USD" + leverage: 500 + model: 0 # 0=every tick (required for martingale/grid EAs) + timeframe: "M5" + timeout: 900 # seconds per backtest run + +optimization: + log_dir: "/tmp" + min_agents: 1 + +reports: + output_dir: "./reports" + keep_last: 20 diff --git a/mcp-package/docs/ANTIGRAVITY.md b/mcp-package/docs/ANTIGRAVITY.md new file mode 100644 index 0000000..5c76bdc --- /dev/null +++ b/mcp-package/docs/ANTIGRAVITY.md @@ -0,0 +1,145 @@ +# Antigravity MCP Integration Setup + +## Quick Setup + +### Option 1: Download Prebuilt Binary (Recommended) + +```bash +# macOS (Apple Silicon) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz +tar -xzf mt5-quant.tar.gz + +# Linux (x64) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz +tar -xzf mt5-quant.tar.gz +``` + +### Option 2: Build from Source + +```bash +cargo build --release +``` + +## Configure Antigravity + +### Step 1: Open MCP Manager + +1. Launch Antigravity +2. Look at the **right-side Agent Panel** +3. Click the **"..."** (More Options) menu at the top +4. Select **MCP Servers** + +### Step 2: Access Configuration + +1. Click **Manage MCP Servers** +2. Click **View raw config** or **Edit configuration** +3. This opens `mcp_config.json` + +### Step 3: Add mt5-quant Configuration + +Add to `mcp_config.json`: + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/absolute/path/to/mt5-quant" + } + } +} +``` + +### Step 4: Reload and Verify + +1. Save the file +2. **Restart Antigravity** (or reload the window) +3. Open the Agent chat and type: `What tools do you have access to?` +4. The agent should list MT5 tools like `verify_setup`, `run_backtest`, etc. + +## Full Example Configuration + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/Users/name/mt5-quant/target/release/mt5-quant" + }, + "github": { + "command": "npx", + "args": ["-y", "@modelcontextprotocol/server-github"], + "env": { + "GITHUB_PERSONAL_ACCESS_TOKEN": "${env:GITHUB_TOKEN}" + } + } + } +} +``` + +## Environment Variables + +Antigravity supports `${VAR_NAME}` syntax for environment variable substitution: + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/Users/name/mt5-quant/target/release/mt5-quant", + "env": { + "CUSTOM_VAR": "${env:MY_VAR}" + } + } + } +} +``` + +This keeps secrets out of the config file. + +## Verify Setup + +In Antigravity chat, type: + +``` +Run verify_setup +``` + +Expected output: +``` +Wine: /Applications/MetaTrader 5.app/.../wine64 +MT5 dir: ~/Library/Application Support/.../MetaTrader 5 +Display: gui +Arch: arch -x86_64 +``` + +## Troubleshooting + +### "Connection Refused" or "Tool not found" + +1. Double-check the server path in `mcp_config.json` +2. Ensure the binary has execute permissions: `chmod +x /path/to/mt5-quant` +3. Try completely restarting Antigravity +4. Check the server is listed in MCP manager + +### "Stdio Error" or JSON Parsing Error + +1. Verify the JSON syntax in `mcp_config.json` +2. Use a JSON validator if needed +3. Ensure no trailing commas + +### Agent Hallucinating Tool Parameters + +If the agent makes up incorrect parameters: +1. Check the tool is actually available: "List your available tools" +2. Restart the agent session +3. Be explicit in your requests + +## Configuration Location + +| Platform | Path | +|----------|------| +| All | Via UI: Agent Panel → ... → MCP Servers → Manage → Edit configuration | + +## Resources + +- [Antigravity Documentation](https://docs.antigravity.dev/) +- [MCP Server Reference](https://github.com/modelcontextprotocol/servers) +- [MT5-Quant Tools Reference](./MCP_TOOLS.md) diff --git a/mcp-package/docs/ARCHITECTURE.md b/mcp-package/docs/ARCHITECTURE.md new file mode 100644 index 0000000..6c08192 --- /dev/null +++ b/mcp-package/docs/ARCHITECTURE.md @@ -0,0 +1,458 @@ +# Architecture Deep Dive + +## Design Philosophy + +**Deal-level over aggregate.** MT5's built-in HTML report gives you: profit, profit factor, max DD%, trade count. That's it. You cannot tell from those numbers whether a drawdown was caused by overleveraged locking, a bad entry during a news spike, or a grid that reached L8 in a trending market. + +MT5-Quant extracts every individual deal — entry price, exit price, P/L, comment string — and reconstructs what was happening when each loss event occurred. The `analysis.json` artifact is the result: AI-readable, stable schema, diffable between runs. + +**Pipeline idempotency.** MT5 caches aggressively. Cached `.ex5` binaries, cached `.set` files, stale `terminal.ini` flags. The pipeline exists specifically to invalidate all of these before every run. A backtest result that came from a cached EA binary is wrong in a way that's impossible to detect without the cache clear step. + +**Background isolation for optimization.** Genetic optimizations run 2-6 hours. Running them inside a parent process that can be killed (Claude task runner, SSH session, terminal) corrupts the MT5 optimization state. The only correct pattern is `nohup + disown` — fully detached from all parent process trees. + +--- + +## Component Map + +``` +MT5-Quant/ +├── src/ +│ ├── main.rs # MCP server entry (stdio transport) +│ ├── mcp_server.rs # MCP protocol handling +│ ├── models/ # Data structures +│ │ ├── config.rs # Configuration +│ │ ├── deals.rs # Deal, PositionPair, DrawdownEvent, etc. +│ │ ├── metrics.rs # Metrics parsing from HTML/XML +│ │ └── report.rs # Report, PipelineMetadata, etc. +│ ├── analytics/ # Report extraction & analysis (migrated from Python) +│ │ ├── extract.rs # HTML/XML report parser → metrics.json + deals.csv +│ │ └── analyze.rs # Deal-level analysis engine → analysis.json +│ ├── compile/ # MQL5 compilation +│ │ └── mql_compiler.rs # MetaEditor wrapper (Wine/CrossOver) +│ ├── pipeline/ # Backtest orchestration +│ │ ├── backtest.rs # 5-stage pipeline (COMPILE→CLEAN→BACKTEST→EXTRACT→ANALYZE) +│ │ └── stages.rs # Pipeline stage definitions +│ └── tools/ # MCP tool definitions +│ ├── definitions/ # Tool schemas (9 domain modules, 43 tools) +│ │ ├── mod.rs +│ │ ├── analytics.rs # 9 analysis tools +│ │ ├── backtest.rs # 4 backtest tools +│ │ ├── baseline.rs # 1 baseline tool +│ │ ├── experts.rs # 4 EA/indicator/script tools +│ │ ├── optimization.rs # 4 optimization tools +│ │ ├── reports.rs # 11 report management tools +│ │ ├── setfiles.rs # 8 .set file tools +│ │ └── system.rs # 3 system tools +│ └── handlers/ # Tool dispatch (9 domain modules) +│ ├── mod.rs +│ ├── analysis.rs +│ ├── backtest.rs +│ ├── experts.rs +│ ├── optimization.rs +│ ├── reports.rs +│ ├── setfiles.rs +│ └── system.rs +│ +├── scripts/ +│ ├── setup.sh # Auto-detect Wine/MT5, write config, register MCP +│ ├── platform_detect.sh # Wine path + headless detection +│ ├── build-rust.sh # Rust build script +│ └── optimize.sh # Genetic optimization launcher (nohup + disown) +│ +├── analytics/ # Legacy Python (reference only) +│ ├── extract.py +│ ├── analyze.py +│ └── optimize_parser.py +│ +├── config/ +│ ├── mt5-quant.example.yaml # Template config +│ └── mt5-quant.yaml # Live config (gitignored) +│ +└── docs/ + ├── ARCHITECTURE.md # This file + ├── MCP_TOOLS.md # Full tool spec + └── REMOTE_AGENTS.md # Linux agent farm setup +``` + +--- + +## Pipeline Stages + +### Stage 1: COMPILE + +```rust +// src/compile/mql_compiler.rs +let compiler = MqlCompiler::new(config); +let result = compiler.compile("src/experts/MyEA.mq5")?; +``` + +Invokes MetaEditor via Wine with the MQL5 source file. Copies resulting `.ex5` to the MT5 Experts directory. Fails the pipeline on compile errors. + +**Why not skip this?** MT5 caches the `.ex5` binary by filename. If you edit your EA and re-run without recompiling, MT5 runs the old binary silently. Always compile. + +--- + +### Stage 2: CLEAN + +```bash +rm -f "${MT5_TESTER_DIR}/cache/*.tst" +rm -f "${MT5_PROFILES_DIR}/Tester/${EXPERT}.set" +``` + +Clears: +- Tester cache (`.tst` files): compiled test results MT5 reuses to skip re-running ticks +- Cached `.set` file: MT5 writes the current parameter values here after each run; if stale, next run picks up wrong params + +**The `.set` encoding trap:** MT5 reads parameter files as UTF-16LE with BOM. It writes them back as UTF-16LE. If you provide a UTF-8 `.set` file for optimization, MT5 reads the parameters correctly (it tries multiple encodings) but when it writes the optimization variants, it uses UTF-16LE and **strips the `||Y` optimization flags**. Every subsequent pass uses the base value. Your 500-combination optimization runs 500 identical backtests. + +Solution: always write `.set` files as UTF-16LE with BOM, mark them read-only before MT5 starts. + +```python +# Python: write UTF-16LE with BOM +content = "\n".join(lines) +with open(set_path, 'w', encoding='utf-16-le') as f: + f.write('\ufeff') # BOM + f.write(content) +os.chmod(set_path, 0o444) # read-only +``` + +--- + +### Stage 3: BACKTEST + +```bash +arch -x86_64 "${WINE}" cmd.exe /c 'C:\_backtest.bat' +``` + +The batch file sets MT5 CLI flags and launches `terminal64.exe`: + +```bat +"C:\Program Files\MetaTrader 5\terminal64.exe" /config:C:\backtest.ini +``` + +`backtest.ini` contains: +```ini +[Tester] +Expert=MyEA +Symbol=XAUUSD +Period=M5 +Deposit=10000 +Currency=USD +Leverage=500 +Model=0 +FromDate=2025.01.01 +ToDate=2025.06.30 +Report=C:\report +Optimization=0 +``` + +MT5 runs in headless mode, writes the report, and exits. + +**Process isolation note:** On macOS with CrossOver, `arch -x86_64` is required — CrossOver ships arm64 Wine wrappers that don't support the x86_64 MT5 binary correctly. Without it, MT5 appears to start but produces no output. + +--- + +### Stage 4: EXTRACT + +Single HTML/XML parse pass that produces three artifacts: + +```rust +// src/analytics/extract.rs +let extractor = ReportExtractor::new(); +let result = extractor.extract(&report_path, &output_dir)?; +// → metrics.json (aggregate summary) +// → deals.csv (all deals, 13 columns) +// → deals.json (same data, JSON) +``` + +**Why single-pass?** MT5 HTML reports are large (1-5MB for 14-month tests). Each regex pass over the file takes ~200ms. The old pipeline ran 5 separate grep/regex passes. The Rust implementation uses a single-pass parser: 5× faster and no partial-read inconsistencies. + +**Format detection:** +```rust +// MT5 Build 48+ saves SpreadsheetML XML, not HTML +let ext = Path::new(&path).extension() + .and_then(|e| e.to_str()) + .unwrap_or(""); + +if ext == "xml" || path.ends_with(".htm.xml") { + // Parse as SpreadsheetML XML + let doc = roxmltree::Document::parse(&text)?; +} else { + // Parse as HTML with regex +} +``` + +**Deal columns (13):** +``` +Time | Type | Direction | Volume | Price | S/L | T/P | Profit | Balance | Comment | Order | Magic | Entry +``` + +The `Comment` column is the key to grid analytics. The EA writes `"Layer #3"`, `"Locking Total"`, `"Zombie Exit"` etc. Pattern matching on comments reconstructs which position was at which layer. + +--- + +### Stage 5: ANALYZE + +```rust +// src/analytics/analyze.rs +let analyzer = DealAnalyzer::new(); +let result = analyzer.analyze(&deals, &metrics, strategy, deep)?; +// → analysis.json +``` + +All functions operate on the parsed deal data — no MT5 or Wine required. + +**Strategy profiles** (defined in `analyze.rs`): +- `grid` — Layer depth tracking, locking/cutloss/zombie keywords +- `scalper` — TP/SL/manual/trailing exit classification +- `trend` — TP/SL/trailing/breakeven/partial exits +- `hedge` — TP/SL/net_close/partial, magic+direction grouping +- `generic` — Simple profit-based TP/SL classification + +#### Strategy profiles + +The analysis engine is driven by a `PROFILES` dict. Each profile controls: + +| Field | Type | Controls | +|-------|------|----------| +| `depth_re` | regex or `None` | Whether/how to extract depth from comments | +| `exit_keywords` | `{reason: [kw]}` | Comment patterns for exit classification | +| `dd_cause_keywords` | `{cause: [kw]}` | Comment patterns for DD cause classification | +| `cycle_group_by` | `'magic'` or `'magic+direction'` | How deals are grouped into cycles | +| `cycle_gap_min` | int | Minutes between opens that mark a new cycle | + +Built-in profiles: + +| Profile | `depth_re` | `cycle_group_by` | `cycle_gap_min` | Exit keywords | +|---------|-----------|-----------------|----------------|---------------| +| `generic` | — | `magic` | 60 | profit-sign only (tp/sl) | +| `grid` | `Layer #N` | `magic+direction` | 60 | locking, cutloss, zombie, timeout | +| `scalper` | — | `magic` | 10 | tp, sl, manual, trailing | +| `trend` | — | `magic` | 240 | breakeven, trailing, partial, tp, sl | +| `hedge` | — | `magic+direction` | 120 | tp, sl, net_close, partial | + +#### Entry points (after `pip install -e .`) + +```bash +mt5-analyze deals.csv # generic +mt5-analyze-grid deals.csv # grid / martingale (default in pipeline) +mt5-analyze-scalper deals.csv # scalper +mt5-analyze-trend deals.csv # trend following +mt5-analyze-hedge deals.csv # hedging +``` + +#### Analytics functions + +**Core (always run, strategy-agnostic):** + +| Function | What it computes | +|----------|-----------------| +| `monthly_pnl` | P/L, trade count, green flag per calendar month | +| `reconstruct_dd_events` | Balance curve → local minima; cause from profile keywords | +| `top_losses` | Worst individual closing deals by P/L | +| `loss_sequences` | Consecutive losing closed deals (runs of length ≥ 2) | +| `position_pairs` | Match in/out by order ticket → hold time, depth at close | +| `direction_bias` | Buy vs sell win rate, total P/L, average trade | +| `streak_analysis` | Max consecutive win/loss streaks; current streak | +| `session_breakdown` | Asian (00–08h) / London (08–13h) / London-NY (13–17h) / New York (17–22h) | +| `weekday_pnl` | Mon–Sun P/L and win rate | +| `concurrent_peak` | Peak simultaneous open positions | + +**Strategy-driven (output varies by profile):** + +| Function | Generic | Grid | Scalper/Trend/Hedge | +|----------|---------|------|---------------------| +| `depth_histogram` | `{}` (empty) | L1–L8+ counts | `{}` (no `depth_re`) | +| `cycle_stats` | magic, 60-min gap | magic+direction, 60-min gap | per-profile config | +| `exit_reason_breakdown` | tp / sl | locking / cutloss / zombie / timeout | profile-specific | + +**Deep analytics (`--deep` flag):** + +| Function | What it computes | +|----------|-----------------| +| `hourly_pnl` | Hour-by-hour (0–23) P/L and win rate | +| `volume_profile` | P/L breakdown by lot size tier | + +**DD event reconstruction:** +1. Walk deals chronologically, track running balance +2. At each local minimum (DD > 1%), record timestamp, depth (%), recovery date +3. Classify `cause` using `profile['dd_cause_keywords']`; returns `"unknown"` for generic/unmatched + +**Cycle statistics:** +Deals are grouped by `cycle_group_by` key. A gap greater than `cycle_gap_min` between consecutive opens marks a new cycle boundary. Win rate is computed per cycle (not per deal), then broken down by max depth reached. + +**Exit reason classification:** +Iterates `exit_keywords` in definition order — more specific patterns must appear before general ones to avoid substring false-positives (e.g. `"stop"` inside `"breakeven stop"`). Falls back to profit-sign if no keyword matches. + +**Loss sequence detection:** +Consecutive closed deals where P/L < 0 (minimum length 2). Captures clusters of losses better than any single worst-trade metric. + +--- + +## Optimization Pipeline + +### Why `nohup + disown` is mandatory + +```bash +nohup ./scripts/optimize.sh ... > /tmp/opt.log 2>&1 & disown +``` + +MT5 optimization uses Unix signals to coordinate between `terminal64.exe` (master) and `metatester64.exe` instances (workers). When the parent process tree is killed: + +1. `SIGHUP` propagates to child processes +2. `metatester64.exe` workers receive the signal and terminate +3. The master `terminal64.exe` detects worker failure and aborts the optimization +4. `terminal.ini` is left with `OptMode=-1`, requiring manual reset before next run + +`nohup` prevents `SIGHUP` propagation. `disown` removes the process from the shell's job table so it's not killed when the shell exits. Both are required. + +--- + +### `OptMode` state machine + +`terminal.ini` contains an `OptMode` key that MT5 uses to track optimization state: + +| `OptMode` value | Meaning | +|----------------|---------| +| `0` | Normal backtest mode (ready) | +| `1` | Optimization in progress | +| `2` | Optimization complete — show results | +| `-1` | Optimization aborted / crashed | + +After any optimization run (complete or aborted), MT5 writes `-1` or `2`. On next launch with `Optimization=2` in `backtest.ini`, MT5 reads `OptMode=-1` and exits immediately without running. + +**Fix:** Before every optimization launch, force `OptMode=0` in `terminal.ini`: + +```bash +sed -i 's/OptMode=.*/OptMode=0/' "${MT5_DIR}/terminal.ini" +# Also remove LastOptimization line if present +sed -i '/LastOptimization=/d' "${MT5_DIR}/terminal.ini" +``` + +--- + +## Remote Agent Architecture + +MT5's distributed testing works via a custom TCP protocol. The master `terminal64.exe` listens on a port. Remote agents (`metatester64.exe`) connect and receive test configurations. + +``` +Mac (master) Linux server (agents) +terminal64.exe metatester64.exe × N + │ │ + └──── TCP:3000 ─────────────────────┘ +``` + +**Linux setup:** +```bash +# On Linux server (Wine required) +wine metatester64.exe /server:MAC_IP:3000 /agents:8 +``` + +MT5 shows remote agents in the agent manager as `Agent-0.0.0.0-PORT` entries when listening, and activates them when the remote `metatester64.exe` connects. + +**Throughput:** Linear scaling with agent count. 10 local + 16 remote = 26 agents. A 17,000-combination optimization that takes 3 hours locally completes in ~70 minutes. + +--- + +## Headless Operation + +MT5-Quant uses MT5's CLI mode (`terminal64.exe /config:backtest.ini`) — no user interaction, no clicking in the Strategy Tester GUI. Whether this is truly "headless" depends on platform: + +| Platform | Status | Notes | +|----------|--------|-------| +| macOS + CrossOver | Near-headless | CrossOver manages the display internally. MT5 window may flash briefly or be suppressed entirely depending on bottle settings. No monitor required in practice. | +| Linux + Wine | Requires Xvfb | Wine needs an X11 display connection. Without one, `wine64 terminal64.exe` fails with `cannot open display`. | +| Linux + Wine + Xvfb | Full headless | Virtual framebuffer satisfies Wine's X11 requirement. Use on servers with no monitor. | + +**Linux headless setup (Xvfb):** + +```bash +# Install Xvfb +sudo apt install xvfb + +# Start virtual display on :99 +Xvfb :99 -screen 0 1024x768x16 & +export DISPLAY=:99 + +# Now Wine can launch MT5 without a physical display +wine64 terminal64.exe /config:backtest.ini +``` + +**Persistent virtual display (systemd):** + +```ini +# /etc/systemd/system/xvfb.service +[Unit] +Description=Virtual Display for MT5 + +[Service] +ExecStart=/usr/bin/Xvfb :99 -screen 0 1024x768x16 +Restart=always + +[Install] +WantedBy=multi-user.target +``` + +```bash +sudo systemctl enable xvfb +sudo systemctl start xvfb +``` + +Then set `DISPLAY=:99` in MT5-Quant's environment config. + +**Note:** `metatester64.exe` (the agent worker process) is fully headless — it runs tick simulation with no display requirement. Only the master `terminal64.exe` needs a display to orchestrate the session. On Mac with CrossOver this is handled transparently. + +--- + +## Known Limitations + +**macOS-specific:** +- Requires Wine. The native MT5.app from the Mac App Store (or MetaQuotes CDN) ships bundled Wine at `MetaTrader 5.app/Contents/SharedSupport/wine/`. CrossOver is an alternative. +- `arch -x86_64` required on Apple Silicon. +- File paths must go through Wine's virtual filesystem (`C:\` = inside the Wine prefix `drive_c/`). + +**Report format dependency:** +- SpreadsheetML XML format (`.htm.xml`) has no documented schema from MetaQuotes. The parser is reverse-engineered from observed output. May break on future MT5 builds. + +**Comment-based analytics:** +- Strategy-specific analytics (depth histogram, exit reason, DD cause) depend on EA comment strings. EAs that don't write structured comments will get `generic` profile results — summary metrics, session breakdown, streaks, and direction bias all still work; only keyword-classified fields fall back to `"unknown"` or profit-sign. +- Custom comment patterns can be supported by adding a new entry to `PROFILES` in `analytics/analyze.py` — no other code changes needed. + +**Single MT5 instance:** +- MT5 is single-instance per Windows drive. Two backtests cannot run simultaneously on the same Wine prefix. Parallelism requires multiple Wine prefixes (separate installations). + +--- + +## Claude Code Integration + +`setup.sh --claude-code` generates two files that give Claude persistent context about the user's trading setup: + +### `config/CLAUDE.template.md` + +A project-level CLAUDE.md template the user copies to their EA project root. Encodes: +- MT5-Quant tool names and when to use them +- Baseline tracking policy (never call something an improvement without comparing to `baseline.json`) +- Symbol name reminder (broker-specific suffix matters — `XAUUSD.cent` ≠ `XAUUSD`) +- Backtest and optimization constraints (model 0, single instance, UTF-16LE .set files) + +### `.claude/hooks/user-prompt-submit.sh` + +A Claude Code hook that runs before every prompt submission. Reads `config/baseline.json` and outputs a JSON context block: + +```json +{"context": "## Production Baseline (config/baseline.json)\n..."} +``` + +Claude Code injects this into the system context for every conversation turn. The result: Claude always knows the current production metrics without the user having to paste them. + +**Hook execution path:** +``` +User types prompt + → user-prompt-submit.sh executes + → reads config/baseline.json + → outputs {"context": "..."} to stdout + → Claude Code prepends to system context + → Claude sees baseline in every prompt +``` + +**Graceful degradation:** If `baseline.json` doesn't exist or is malformed, the hook exits 0 silently — no prompt is blocked. The baseline section simply doesn't appear until the user creates the file. diff --git a/mcp-package/docs/CONFIG.md b/mcp-package/docs/CONFIG.md new file mode 100644 index 0000000..aadb680 --- /dev/null +++ b/mcp-package/docs/CONFIG.md @@ -0,0 +1,107 @@ +# Configuration Reference + +## Config File Location + +``` +config/mt5-quant.yaml # Project directory (development) +~/.config/mt5-quant/config/mt5-quant.yaml # System-wide (production) +``` + +Environment variable to override: +```bash +export MT5_MCP_HOME=/path/to/config +``` + +## Full Config Example + +```yaml +# Required: Wine executable path +wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64" + +# Required: MT5 installation directory +terminal_dir: "~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5" + +# Optional: Default backtest parameters +defaults: + symbol: "XAUUSD.cent" + timeframe: "M5" + deposit: 10000 + currency: "USD" + model: 0 # 0=every tick, 1=1min OHLC, 2=open price + leverage: 500 + +# Optional: Display settings +display: + mode: auto # auto, gui, headless + xvfb_display: ":99" # Linux headless only + xvfb_screen: "1024x768x16" # Linux headless only + +# Optional: Directories (auto-detected from terminal_dir if not set) +experts_dir: "~/.../MetaTrader 5/MQL5/Experts" +indicators_dir: "~/.../MetaTrader 5/MQL5/Indicators" +scripts_dir: "~/.../MetaTrader 5/MQL5/Scripts" + +# Optional: Reports directory +reports_dir: "./reports" + +# Optional: Optimization settings +optimization: + remote_agents: + enabled: false + check_agent_count: true + min_agents: 4 +``` + +## Platform-Specific Examples + +### macOS with MetaTrader 5.app + +```yaml +wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64" +terminal_dir: "~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5" +defaults: + symbol: "XAUUSDc" + timeframe: "M5" + deposit: 10000 +``` + +### macOS with CrossOver + +```yaml +wine_executable: "/Applications/CrossOver.app/Contents/SharedSupport/CrossOver/bin/wine64" +terminal_dir: "~/Library/Application Support/MetaQuotes/Terminal//drive_c/Program Files/MetaTrader 5" +``` + +### Linux + +```yaml +wine_executable: "/usr/bin/wine64" +terminal_dir: "~/.wine/drive_c/Program Files/MetaTrader 5" +display: + mode: headless + xvfb_display: ":99" +``` + +## Headless Mode (Linux VPS) + +```yaml +display: + mode: headless + xvfb_display: ":99" + xvfb_screen: "1024x768x16" +``` + +Requires: +```bash +sudo apt install xvfb +``` + +## Auto-Detection + +`setup.sh` automatically detects: +- Wine executable (MetaTrader 5.app, CrossOver, or system Wine) +- MT5 terminal directory +- Architecture (Apple Silicon adds `arch -x86_64`) +- Display mode (GUI vs headless) + +Run `setup.sh` whenever you move the installation. diff --git a/mcp-package/docs/CURSOR.md b/mcp-package/docs/CURSOR.md new file mode 100644 index 0000000..90fdc12 --- /dev/null +++ b/mcp-package/docs/CURSOR.md @@ -0,0 +1,129 @@ +# Cursor MCP Integration Setup + +## Quick Setup + +### Option 1: Download Prebuilt Binary (Recommended) + +```bash +# macOS (Apple Silicon) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz +tar -xzf mt5-quant.tar.gz + +# Linux (x64) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz +tar -xzf mt5-quant.tar.gz +``` + +### Option 2: Build from Source + +```bash +cargo build --release +``` + +## Configure Cursor + +### Method 1: Settings UI (Recommended) + +1. Open Cursor Settings (`Cmd/Ctrl + ,`) +2. Navigate to **Features** → **MCP** +3. Click **Add Custom MCP** +4. Enter: + - **Name**: `mt5-quant` + - **Command**: Full path to binary (e.g., `/Users/name/mt5-quant/target/release/mt5-quant`) + - **Type**: `stdio` + +### Method 2: Edit mcp.json Directly + +Add to `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project): + +```json +{ + "mcpServers": { + "mt5-quant": { + "type": "stdio", + "command": "/absolute/path/to/mt5-quant" + } + } +} +``` + +Create the file if it doesn't exist: + +```bash +mkdir -p ~/.cursor +cat > ~/.cursor/mcp.json << 'EOF' +{ + "mcpServers": { + "mt5-quant": { + "type": "stdio", + "command": "/path/to/mt5-quant" + } + } +} +EOF +``` + +## Verify Setup + +In Cursor chat, type: + +``` +Run verify_setup +``` + +Expected output: +``` +Wine: /Applications/MetaTrader 5.app/.../wine64 +MT5 dir: ~/Library/Application Support/.../MetaTrader 5 +Display: gui +Arch: arch -x86_64 +``` + +## Configuration Locations + +| Scope | Path | Use Case | +|-------|------|----------| +| Global | `~/.cursor/mcp.json` | Available in all projects | +| Project | `.cursor/mcp.json` | Project-specific tools | + +## Troubleshooting + +### MCP server not appearing + +1. Check MCP panel in Cursor Settings +2. Verify the path is absolute (not relative) +3. Test binary: `/path/to/mt5-quant --help` +4. View MCP logs: Output panel → select "MCP" from dropdown + +### Config interpolation + +Cursor supports variable substitution in `mcp.json`: + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "${userHome}/bin/mt5-quant", + "args": ["--config", "${workspaceFolder}/config.yaml"] + } + } +} +``` + +Available variables: +- `${userHome}` - Home directory +- `${workspaceFolder}` - Project root +- `${workspaceFolderBasename}` - Project folder name +- `${env:VAR_NAME}` - Environment variable + +### Tool not found errors + +If the agent says "Tool not found": +1. Check the server is enabled in MCP settings +2. Try disabling and re-enabling the server +3. Restart Cursor + +## Resources + +- [Cursor MCP Documentation](https://cursor.com/docs/context/mcp) +- [MT5-Quant Tools Reference](./MCP_TOOLS.md) diff --git a/mcp-package/docs/MCP_TOOLS.md b/mcp-package/docs/MCP_TOOLS.md new file mode 100644 index 0000000..824ab66 --- /dev/null +++ b/mcp-package/docs/MCP_TOOLS.md @@ -0,0 +1,1922 @@ +# MCP Tool Specification + +Full input/output schemas for MT5-Quant tools. + +> **Documentation Status:** This file documents 49 of 85 total tools. Missing: +> - `list_experts`, `list_indicators`, `list_scripts` +> - `healthcheck`, `list_symbols` +> - Reports query: `search_reports`, `get_latest_report`, `list_reports`, `prune_reports`, `tail_log`, `get_report_by_id`, `get_reports_summary`, `get_best_reports`, `search_reports_by_tags`, `search_reports_by_date_range`, `search_reports_by_notes`, `get_reports_by_set_file`, `get_comparable_reports` +> - Granular analytics: `analyze_monthly_pnl`, `analyze_drawdown_events`, `analyze_top_losses`, `analyze_loss_sequences`, `analyze_position_pairs`, `analyze_direction_bias`, `analyze_streaks`, `analyze_concurrent_peak` +> - Deal analytics: `list_deals`, `search_deals_by_comment`, `search_deals_by_magic`, `analyze_profit_distribution`, `analyze_time_performance`, `analyze_hold_time_distribution`, `analyze_layer_performance`, `analyze_volume_vs_profit`, `analyze_costs`, `analyze_efficiency` +> - Archive/history tools: `archive_report`, `archive_all_reports`, `get_history`, `annotate_history`, `promote_to_baseline` +> - Experts search: `search_experts`, `search_indicators`, `search_scripts`, `copy_indicator_to_project`, `copy_script_to_project` +> - Debugging/diagnostics: `diagnose_wine`, `get_mt5_logs`, `search_mt5_errors`, `check_mt5_process`, `kill_mt5_process`, `check_system_resources`, `validate_mt5_config`, `get_wine_prefix_info`, `get_backtest_crash_info` + +--- + +## `run_backtest` + +Run a complete backtest pipeline: compile → clean cache → backtest → extract → analyze. + +**When to call:** Any time you need fresh backtest results. Always runs the full pipeline unless `skip_*` flags are set. + +### Input schema + +```typescript +{ + // Required + expert: string; // EA name without path or extension. e.g. "MyEA_v1.2" + + // Date range — use either preset OR from+to + preset?: "last_month" | "last_3months" | "ytd" | "last_year"; + from?: string; // "YYYY-MM-DD" + to?: string; // "YYYY-MM-DD" + + // Optional overrides + symbol?: string; // Default from config. e.g. "XAUUSD" + timeframe?: "M1" | "M5" | "M15" | "M30" | "H1" | "H4" | "D1"; // Default: M5 + deposit?: number; // Default from config. e.g. 10000 + currency?: string; // Default: "USD" + model?: 0 | 1 | 2; // 0=every tick (default), 1=1min OHLC, 2=open price + set_file?: string; // Path to .set file. If omitted, uses EA defaults. + leverage?: number; // Default: 500 + + // Pipeline flags + skip_compile?: boolean; // Skip EA compilation (use existing .ex5) + skip_clean?: boolean; // Skip cache clean (faster but risks stale cache) + skip_analyze?: boolean; // Extract only, skip deal analysis + deep_analyze?: boolean; // Add hourly_pnl and volume_profile to analysis.json + strategy?: "grid" | "scalper" | "trend" | "hedge" | "generic"; + // Analysis strategy profile (default: "grid"). + // Controls depth tracking, exit keywords, and cycle grouping. +} +``` + +### Output schema + +```typescript +{ + success: boolean; + report_dir: string; // "reports/20250619_143022_MyEA_XAUUSD_M5" + duration_seconds: number; + + // Inline summary from metrics.json (always present on success) + metrics: { + net_profit: number; + profit_factor: number; + max_dd_pct: number; + sharpe_ratio: number; + total_trades: number; + recovery_factor: number; + expected_payoff: number; + gross_profit: number; + gross_loss: number; + win_rate_pct: number; + avg_profit: number; + avg_loss: number; + }; + + // Deal analysis summary (present unless skip_analyze=true) + analysis_summary: { + green_months: number; + total_months: number; + worst_month: string; // "2025-10" + worst_month_pnl: number; + worst_dd_event_pct: number; + worst_dd_date: string; + max_grid_depth: number; // highest layer reached in any cycle + l5_plus_count: number; // cycles that reached L5+ + }; + + // File paths for direct reading + files: { + metrics_json: string; + analysis_json: string; + deals_csv: string; + deals_json: string; + }; + + error?: string; // Present on failure +} +``` + +### Example + +```json +// Input +{ + "expert": "MyEA_v1.2", + "from": "2025-01-01", + "to": "2025-06-30", + "deposit": 10000, + "model": 0 +} + +// Output +{ + "success": true, + "report_dir": "reports/20250619_143022_MyEA_XAUUSD_M5", + "duration_seconds": 287, + "metrics": { + "net_profit": 4832.10, + "profit_factor": 1.54, + "max_dd_pct": 12.3, + "sharpe_ratio": 1.18, + "total_trades": 891 + }, + "analysis_summary": { + "green_months": 5, + "total_months": 6, + "worst_month": "2025-03", + "worst_month_pnl": -412.80, + "worst_dd_event_pct": 12.3, + "worst_dd_date": "2025-03-14", + "max_grid_depth": 6, + "l5_plus_count": 8 + }, + "files": { + "metrics_json": "reports/20250619_143022_MyEA_XAUUSD_M5/metrics.json", + "analysis_json": "reports/20250619_143022_MyEA_XAUUSD_M5/analysis.json", + "deals_csv": "reports/20250619_143022_MyEA_XAUUSD_M5/deals.csv", + "deals_json": "reports/20250619_143022_MyEA_XAUUSD_M5/deals.json" + } +} +``` + +--- + +## `run_optimization` + +Launch genetic parameter optimization as a detached background process. + +**Important:** This tool returns immediately. MT5 runs for 2-6 hours. The AI agent must NOT poll for results — the user monitors MT5 and signals when done. Call `get_optimization_results` only after user confirmation. + +**Always uses model 0.** Model 1 (1-min OHLC) overfits grid/martingale EAs because intra-bar price movement is not simulated. Parameters that look optimal on model 1 fail on model 0 verification — this is a known trap. + +### Input schema + +```typescript +{ + expert: string; // EA name + set_file: string; // Path to optimization .set file (with ||Y flags) + from: string; // "YYYY-MM-DD" + to: string; // "YYYY-MM-DD" + symbol?: string; // Default from config + deposit?: number; // Default from config + currency?: string; // Default: "USD" + leverage?: number; // Default: 500 + log_file?: string; // Where to write nohup output (default: /tmp/opt_.log) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + job_id: string; // "opt_20250619_143022" + log_file: string; // "/tmp/opt_20250619_143022.log" + pid: number; // Process ID (for user monitoring if needed) + combinations: number; // Estimated from set_file analysis (product of all ||Y ranges) + message: string; // "Optimization launched. Signal me when MT5 completes." +} +``` + +### Optimization set file format + +```ini +; param=current_value||start||step||stop||Y (Y = include in sweep) +; param=value||N (N = fixed, not swept) + +Min_Entry_Confidence=0.610||0.580||0.010||0.650||Y ; 8 values +TP_Pips_Layer1=400||300||50||500||Y ; 5 values +Max_DD_Percent=15.0||N ; fixed + +; Total combinations: 8 × 5 = 40 +``` + +**MT5-Quant handles automatically:** +- UTF-16LE encoding with BOM +- `chmod 444` (read-only) before launch +- `OptMode=0` reset in `terminal.ini` +- `LastOptimization` line removal from `terminal.ini` +- `ExpertParameters` = filename only (not full path) in launch INI + +--- + +## `get_optimization_results` + +Parse completed optimization results. Handles both HTML (`.htm`) and SpreadsheetML XML (`.htm.xml`) formats transparently. + +### Input schema + +```typescript +{ + job_id?: string; // From run_optimization response. If omitted, uses latest _opt/ dir. + report_dir?: string; // Explicit path to *_opt/ directory + top_n?: number; // How many top results to return (default: 20) + dd_threshold?: number; // Flag results above this DD% as high-risk (default: 20) + sort_by?: "profit" | "profit_factor" | "sharpe"; // Default: "profit" +} +``` + +### Output schema + +```typescript +{ + success: boolean; + total_passes: number; + converged: boolean; // True if passes stopped improving in last 10% + report_format: "html" | "xml"; + + results: Array<{ + rank: number; + net_profit: number; + profit_factor: number; + max_dd_pct: number; + total_trades: number; + sharpe_ratio: number; + high_risk: boolean; // DD > dd_threshold + params: Record; // All swept parameter values + }>; + + convergence_analysis: { + top_10_agreement: Record; // Params same across top 10 = strong signal + high_variance_params: string[]; // Params that vary in top 10 = uncertain + }; + + recommendation: { + best_params: Record; + reasoning: string; + next_step: "verify_model0" | "auto_promote" | "investigate"; + }; +} +``` + +### Convergence analysis + +A parameter that appears with the same value across all top-10 results is a strong optimization signal — the genetic algorithm converged on it. A parameter that varies across top-10 means the optimizer couldn't distinguish between values — either the parameter doesn't matter much, or more passes are needed. + +--- + +## `verify_setup` + +Check all required paths, Wine version, and EA/set file inventory. Run this first if `run_backtest` or `run_optimization` fails with path errors. + +### Input schema + +```typescript +{} // No parameters required +``` + +### Output schema + +```typescript +{ + success: boolean; + wine_path: string; + wine_version: string; + mt5_dir: string; + terminal_exe: string; + experts_dir: string; + display_mode: "gui" | "headless"; + ea_count: number; // .ex5 files found in Experts/ + set_count: number; // .set files found + missing: string[]; // List of paths/tools that couldn't be found + hints: string[]; // Actionable fix hints for each missing item +} +``` + +--- + +## `get_backtest_status` + +Check the current stage and elapsed time of a running backtest pipeline by reading its `progress.log`. + +### Input schema + +```typescript +{ + report_dir: string; // Path to the report directory from run_backtest +} +``` + +### Output schema + +```typescript +{ + success: boolean; + report_dir: string; + stage: "COMPILE" | "CLEAN" | "BACKTEST" | "EXTRACT" | "ANALYZE" | "DONE"; + elapsed_seconds: number; + finished: boolean; + log_lines: string[]; // Last 5 lines of progress.log +} +``` + +--- + +## `get_optimization_status` + +Check the live state of a background optimization job (started by `run_optimization`). + +### Input schema + +```typescript +{ + job_id: string; // From run_optimization response +} +``` + +### Output schema + +```typescript +{ + success: boolean; + job_id: string; + alive: boolean; // True if the optimization process is still running + pid: number; + started_at: string; // ISO timestamp + elapsed_seconds: number; + report_found: boolean; // True if MT5 has written the result file + report_path: string | null; + log_tail: string[]; // Last 10 lines of the nohup log +} +``` + +--- + +## `prune_reports` + +Delete old report directories to reclaim disk space, keeping the most recent N runs. Optimization result directories (`*_opt/`) are always preserved. + +### Input schema + +```typescript +{ + keep_last?: number; // How many recent reports to keep (default from config, usually 10) + dry_run?: boolean; // If true, list what would be deleted without deleting (default: false) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + deleted: string[]; // Paths that were (or would be) deleted + kept: string[]; // Paths that were kept + freed_mb: number; // Approximate disk space freed +} +``` + +--- + +## `analyze_report` + +Read and summarize a completed backtest report without re-running MT5. + +### Input schema + +```typescript +{ + report_dir: string; // Path to report directory + strategy?: "grid" | "scalper" | "trend" | "hedge" | "generic"; + // Strategy profile that was used (default: "grid"). + // Only affects interpretation of analysis.json fields — + // does not re-run analysis. + include_deals?: boolean; // Include top 20 deals in output (default: false) + include_monthly?: boolean; // Include full monthly P/L table (default: true) + include_dd_events?: boolean; // Include DD event reconstruction (default: true) + deep?: boolean; // Include hourly_pnl and volume_profile (default: false) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + report_dir: string; + strategy: string; // Active profile: "grid" | "scalper" | "trend" | "hedge" | "generic" + + metrics: { /* same as run_backtest metrics */ }; + + // ── Always present (strategy-agnostic) ───────────────────────────────────── + + monthly_pnl: Array<{ + month: string; // "2025-01" + pnl: number; + trades: number; + green: boolean; + }>; + + dd_events: Array<{ + peak_dd_pct: number; + start_date: string; + end_date: string; + duration_days: number; + recovery_date: string | null; + recovery_days: number | null; + cause: string; // Profile-driven: e.g. "locking_cascade" (grid) or "whipsaw" (trend) + // Falls back to "unknown" when no keyword matched + }>; + + top_losses: Array<{ + date: string; + loss_usd: number; + grid_depth_at_close: number; // 0 for non-grid strategies + volume: number; + comment: string; + }>; + + loss_sequences: Array<{ + length: number; + total_loss: number; + start: string; + end: string; + }>; + + position_pairs: Array<{ + time: string; + type: "buy" | "sell"; + profit: number; + volume: number; + layer: number; + hold_minutes: number | null; + comment: string; + magic: string; + order: string; + }>; + + // ── Strategy-driven (content varies by profile) ──────────────────────────── + + depth_histogram: Record; + // grid: { L1: n, L2: n, …, "L8+": n } + // others: {} (empty — no depth_re in profile) + + grid_depth_histogram: Record; + // Backward-compat alias for depth_histogram (grid only) + + cycle_stats: { + total_cycles: number; + win_rate: number; // percent + avg_profit: number; + win_rate_by_depth: Record; + // win_rate_by_depth populated for grid; keys = "L?" for non-depth profiles + }; + + exit_reason_breakdown: Record< + string, // Keys depend on strategy profile exit_keywords + // grid: "locking" | "cutloss" | "zombie" | "timeout" | "tp" | "sl" + // scalper: "manual" | "trailing" | "tp" | "sl" + // trend: "breakeven" | "trailing" | "partial" | "tp" | "sl" + // generic: "tp" | "sl" + { count: number; total_pnl: number; avg_pnl: number } + >; + + direction_bias: { + buy?: { trades: number; win_rate: number; total_pnl: number; avg_pnl: number }; + sell?: { trades: number; win_rate: number; total_pnl: number; avg_pnl: number }; + }; + + streak_analysis: { + max_win_streak: number; + max_win_start: string; + max_win_end: string; + max_loss_streak: number; + max_loss_start: string; + max_loss_end: string; + current_streak: number; + current_streak_type: "win" | "loss"; + }; + + session_breakdown: Record< + "asian" | "london" | "london_ny_overlap" | "new_york" | "off_hours", + { trades: number; win_rate: number; total_pnl: number } + >; + + weekday_pnl: Array<{ + day: string; // "Monday" … "Sunday" + pnl: number; + trades: number; + win_rate: number; + }>; + + concurrent_peak: { + peak_open: number; + peak_time: string; + }; + + // ── Deep mode only (deep=true) ────────────────────────────────────────────── + + hourly_pnl?: Array<{ + hour: number; // 0–23 + pnl: number; + trades: number; + win_rate: number; + }>; + + volume_profile?: Array<{ + lot_tier: string; // "0.01" | "0.02-0.04" | "0.05-0.09" | "0.10-0.49" | … + pnl: number; + trades: number; + win_rate: number; + }>; + + // ── Optional raw deals ────────────────────────────────────────────────────── + + deals?: Array<{ /* all 13 deal columns */ }>; // Only if include_deals=true +} +``` + +--- + +## `compare_baseline` + +Compare a report against a baseline and return a structured verdict. + +### Input schema + +```typescript +{ + report_dir: string; // Report to evaluate + baseline: { + net_profit: number; + max_dd_pct: number; + total_trades?: number; + label?: string; // e.g. "v1.2 production" + }; + promote_threshold?: { + profit_gt: number; // Auto-promote if profit > this (default: baseline profit) + dd_lt: number; // AND DD < this (default: 20) + }; +} +``` + +### Output schema + +```typescript +{ + verdict: "winner" | "loser" | "marginal"; + auto_promote: boolean; + + delta: { + profit_usd: number; // positive = improvement + profit_pct: number; // relative to baseline + dd_pp: number; // positive = DD got worse + trades_delta: number; + }; + + summary: string; // Human-readable one-liner + + details: { + candidate: { net_profit: number; max_dd_pct: number; total_trades: number; }; + baseline: { net_profit: number; max_dd_pct: number; label: string; }; + }; +} +``` + +### Example + +```json +// Input +{ + "report_dir": "reports/20250619_143022_MyEA_v1.3_XAUUSD_M5", + "baseline": { + "net_profit": 8660, + "max_dd_pct": 15.66, + "label": "v1.2 production" + } +} + +// Output +{ + "verdict": "winner", + "auto_promote": true, + "delta": { + "profit_usd": 3186.32, + "profit_pct": 36.8, + "dd_pp": -7.27, + "trades_delta": -3 + }, + "summary": "+$3,186 (+37%) profit vs v1.2. DD dropped from 15.66% to 8.39%. Auto-promoting.", + "details": { + "candidate": { "net_profit": 11846.32, "max_dd_pct": 8.39, "total_trades": 1963 }, + "baseline": { "net_profit": 8660.00, "max_dd_pct": 15.66, "label": "v1.2 production" } + } +} +``` + +--- + +## `compile_ea` + +Compile an MQL5 Expert Advisor via MetaEditor (Wine/CrossOver). + +### Input schema + +```typescript +{ + expert_path: string; // e.g. "src/MyEA_v1.2.mq5" + include_dirs?: string[]; // Additional include search paths +} +``` + +### Output schema + +```typescript +{ + success: boolean; + binary_path: string; // Path where .ex5 was written + binary_size_bytes: number; + warnings: number; + errors: number; + error_list: Array<{ + file: string; + line: number; + message: string; + }>; + compile_time_ms: number; +} +``` + +--- + +## Error Handling + +All tools return `success: false` with an `error` field on failure. Pipeline failures are non-fatal by default — the tool returns partial results if any stages completed. + +```typescript +{ + success: false, + error: "COMPILE_FAILED", + error_detail: "2 errors in src/MyEA_v1.2.mq5: line 847: undeclared identifier 'Max_New_Param'", + completed_stages: ["COMPILE"], + failed_stage: "COMPILE" +} +``` + +**Error codes:** + +| Code | Stage | Cause | +|------|-------|-------| +| `COMPILE_FAILED` | COMPILE | MQL5 syntax errors | +| `WINE_NOT_FOUND` | Any | Wine/CrossOver not installed or wrong path | +| `MT5_TIMEOUT` | BACKTEST | MT5 didn't exit within timeout (default: 15min) | +| `REPORT_NOT_FOUND` | EXTRACT | MT5 produced no report (usually parameter error) | +| `EXTRACT_FAILED` | EXTRACT | Report parse error (format change?) | +| `NO_DEALS` | ANALYZE | Report has 0 trades (check date range, symbol) | +| `OPT_NOT_FINISHED` | get_opt_results | Optimization still running | + +--- + +## `list_reports` + +List all backtest report directories with compact key metrics. Use this to survey what runs exist before deciding which to analyze — much cheaper than calling `analyze_report` repeatedly. + +### Input schema + +```typescript +{ + include_opt?: boolean; // Include _opt dirs (default: false) + limit?: number; // Max reports, newest first (default: 30) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + count: number; + reports: Array<{ + name: string; // "20250619_143022_MyEA_XAUUSD_M5" + is_opt: boolean; + net_profit?: number; + max_dd_pct?: number; + total_trades?: number; + symbol?: string; + timeframe?: string; + from_date?: string; + to_date?: string; + metrics?: "missing"; // Present only if metrics.json is absent + }>; +} +``` + +--- + +## `tail_log` + +Read the last N lines of a log file. Supports `filter=errors` to return only lines containing error/fail keywords — avoids streaming full logs into context. + +### Input schema + +```typescript +{ + // Provide one of: report_dir, job_id, or log_file + report_dir?: string; // Reads progress.log from this dir (omit for latest) + job_id?: string; // Reads the nohup log for this optimization job + log_file?: string; // Absolute path to any log file + + n?: number; // Lines to return (default: 50) + filter?: "all" | "errors" | "warnings"; // Default: "all" +} +``` + +### Output schema + +```typescript +{ + success: boolean; + log_file: string; // Resolved path of the file that was read + total_lines: number; // Lines matched after filter applied + lines: string[]; // Last n of the matched lines +} +``` + +--- + +## `cache_status` + +Show the MT5 tester cache directory size broken down by symbol. Use before `clean_cache` to see what's there. + +### Input schema + +```typescript +{} // No parameters +``` + +### Output schema + +```typescript +{ + success: boolean; + cache_dir: string; + total_size_mb: number; + symbols: Array<{ + symbol: string; // Subdirectory name (broker symbol) + size_mb: number; + }>; +} +``` + +--- + +## `clean_cache` + +Delete MT5 tester cache files. Forces MT5 to regenerate tick data on the next backtest (slower first run after clean). Supports dry-run preview and per-symbol targeting. + +### Input schema + +```typescript +{ + symbol?: string; // Delete only this symbol's cache. Omit to delete all. + dry_run?: boolean; // Report what would be deleted without deleting (default: false) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + dry_run: boolean; + deleted_symbols: string[]; + freed_mb: number; + hint: string; // Reminder that next backtest will be slower +} +``` + +--- + +## `read_set_file` + +Parse an MT5 `.set` parameter file (UTF-16LE or UTF-8) into structured JSON. Handles BOM detection automatically. Use this instead of reading raw `.set` files. + +### Input schema + +```typescript +{ + path: string; // Path to .set file +} +``` + +### Output schema + +```typescript +{ + success: boolean; + path: string; + param_count: number; + comments: string[]; // Header comment lines (stripped of semicolons) + params: Record; +} +``` + +### Example + +```json +// Input +{ "path": "config/MyEA_opt.set" } + +// Output +{ + "success": true, + "path": "config/MyEA_opt.set", + "param_count": 5, + "comments": ["MyEA optimization set — XAUUSD M5"], + "params": { + "Min_Entry_Confidence": { "value": "0.610", "from": "0.580", "to": "0.650", "step": "0.010", "optimize": true }, + "TP_Pips": { "value": "400", "from": "300", "to": "500", "step": "50", "optimize": true }, + "Max_DD_Percent": { "value": "15.0" } + } +} +``` + +--- + +## `write_set_file` + +Write an MT5 `.set` parameter file with correct UTF-16LE encoding and `chmod 444`. Overwrites any existing file at the path. + +### Input schema + +```typescript +{ + path: string; // Output path for .set file + + params: Record; +} +``` + +### Output schema + +```typescript +{ + success: boolean; + path: string; + param_count: number; + encoding: "utf-16-le"; + permissions: string; // "444 (read-only, required by MT5)" +} +``` + +### Example + +```json +// Input +{ + "path": "config/MyEA_opt.set", + "params": { + "Min_Entry_Confidence": { "value": 0.61, "from": 0.58, "to": 0.65, "step": 0.01, "optimize": true }, + "TP_Pips": { "value": 400, "from": 300, "to": 500, "step": 50, "optimize": true }, + "Max_DD_Percent": 15.0 + } +} + +// Output +{ + "success": true, + "path": "config/MyEA_opt.set", + "param_count": 3, + "encoding": "utf-16-le", + "permissions": "444 (read-only, required by MT5)" +} +``` + +--- + +## `list_jobs` + +List all optimization jobs tracked in `.mt5mcp_jobs/` with compact status. Cheaper than calling `get_optimization_status` per job. + +### Input schema + +```typescript +{ + include_done?: boolean; // Include completed/failed jobs (default: true) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + count: number; + jobs: Array<{ + job_id: string; // "opt_20250619_143022" + status: "running" | "done" | "failed"; + elapsed_seconds: number | null; + expert: string; + started_at: string; // ISO timestamp + log_file: string; + }>; +} +``` + +--- + +## `patch_set_file` + +Modify specific parameters in an existing `.set` file in-place. Preserves all other params, comments, and sweep config untouched. Returns a diff of what changed. **Use instead of `read_set_file` → edit → `write_set_file`** — saves two round-trips. + +### Input schema + +```typescript +{ + path: string; // .set file to modify (must exist) + patches: Record; +} +``` + +### Output schema + +```typescript +{ + success: boolean; + path: string; + changed_count: number; + param_count: number; + changed: Array<{ name: string; old: string; new: string; }>; +} +``` + +### Example + +```json +// Input — change two params without touching the rest of the file +{ + "path": "config/MyEA_opt.set", + "patches": { + "TP_Pips": 350, + "Min_Entry_Confidence": { "value": 0.62, "from": 0.60, "to": 0.65, "optimize": true } + } +} + +// Output +{ + "success": true, + "path": "config/MyEA_opt.set", + "changed_count": 2, + "param_count": 12, + "changed": [ + { "name": "TP_Pips", "old": "400", "new": "350" }, + { "name": "Min_Entry_Confidence", "old": "0.610", "new": "0.62" } + ] +} +``` + +--- + +## `clone_set_file` + +Copy a `.set` file to a new path, applying optional param overrides. One call instead of read → modify → write. Preserves header comments. + +### Input schema + +```typescript +{ + source: string; // Source .set file + destination: string; // Output path (created if needed) + overrides?: Record; +} +``` + +### Output schema + +```typescript +{ + success: boolean; + source: string; + destination: string; + param_count: number; + overridden_count: number; + overridden: Array<{ name: string; old: string | null; new: string; }>; +} +``` + +--- + +## `set_from_optimization` + +Generate a clean backtest `.set` file directly from an optimization result's params dict. Strips all sweep flags (`||Y`) so the file is ready for `run_backtest`. Optionally fills params not in the optimization result from a template `.set`, and optionally re-adds sweep ranges to selected params for a narrowed follow-on optimization. + +**Typical call**: immediately after `get_optimization_results`, use `results[0].params` as the `params` argument. + +### Input schema + +```typescript +{ + path: string; // Output .set file path + + params: Record; + // Flat param→value dict from optimization result. + // e.g. { "TP_Pips": 400, "Min_Confidence": 0.61 } + + template?: string; // Path to existing .set. Params NOT in 'params' are + // copied from here as fixed values. + + sweep?: Record; + // Re-add sweep ranges to specific params after applying opt values. + // Used to create a narrowed follow-on optimization .set. +} +``` + +### Output schema + +```typescript +{ + success: boolean; + path: string; + param_count: number; + from_template: boolean; + opt_params_applied: number; + swept_params: number; // > 0 if sweep was provided + total_combinations: number; // 0 for pure backtest .set +} +``` + +### Example + +```json +// After get_optimization_results returned: +// results[0].params = { "TP_Pips": 400, "Min_Entry_Confidence": 0.62, "Max_DD_Percent": 15.0 } + +{ + "path": "config/MyEA_v1.3.set", + "params": { "TP_Pips": 400, "Min_Entry_Confidence": 0.62, "Max_DD_Percent": 15.0 }, + "template": "config/MyEA_base.set" +} + +// Output +{ + "success": true, + "path": "config/MyEA_v1.3.set", + "param_count": 12, + "from_template": true, + "opt_params_applied": 3, + "swept_params": 0, + "total_combinations": 0 +} +``` + +--- + +## `diff_set_files` + +Compare two `.set` files and return only the differences. Use instead of reading both files and comparing manually. + +### Input schema + +```typescript +{ + path_a: string; // Baseline / old file + path_b: string; // Candidate / new file +} +``` + +### Output schema + +```typescript +{ + success: boolean; + path_a: string; + path_b: string; + identical: boolean; + added_count: number; // Params in b but not a + removed_count: number; // Params in a but not b + changed_count: number; // Params in both but with different value or sweep flag + + added: Array<{ name: string; value: string; }>; + removed: Array<{ name: string; value: string; }>; + changed: Array<{ + name: string; + a: string; // value in path_a + b: string; // value in path_b + sweep_a?: boolean; // Present only if sweep flag differs + sweep_b?: boolean; + }>; +} +``` + +### Example + +```json +{ + "path_a": "config/MyEA_v1.2.set", + "path_b": "config/MyEA_v1.3.set" +} + +// Output +{ + "success": true, + "identical": false, + "added_count": 1, + "removed_count": 0, + "changed_count": 2, + "added": [{ "name": "Trailing_Activation", "value": "50" }], + "removed": [], + "changed": [ + { "name": "TP_Pips", "a": "400", "b": "350" }, + { "name": "Min_Entry_Confidence", "a": "0.610", "b": "0.620", "sweep_a": true, "sweep_b": false } + ] +} +``` + +--- + +## `describe_sweep` + +Show a `.set` file's sweep configuration: which params are swept, their ranges, per-param value counts, and total combinations. Use before `run_optimization` to verify scope. + +### Input schema + +```typescript +{ + path: string; +} +``` + +### Output schema + +```typescript +{ + success: boolean; + path: string; + total_params: number; + swept_count: number; + fixed_count: number; + total_combinations: number; + swept_params: Array<{ + name: string; + from: string; + to: string; + step: string; + count: number; // Number of distinct values in this param's range + }>; + hint: string; // e.g. "240 combinations. Typical range: 1–8h depending on EA tick speed." +} +``` + +### Example + +```json +// Input +{ "path": "config/MyEA_opt.set" } + +// Output +{ + "success": true, + "total_params": 12, + "swept_count": 3, + "fixed_count": 9, + "total_combinations": 240, + "swept_params": [ + { "name": "TP_Pips", "from": "300", "to": "500", "step": "50", "count": 5 }, + { "name": "Min_Entry_Confidence", "from": "0.58","to": "0.65","step": "0.01", "count": 8 }, + { "name": "Max_DD_Percent", "from": "12", "to": "20", "step": "2", "count": 5 } + ], + "hint": "240 combinations. Typical range: 1–8h depending on EA tick speed." +} +``` + +--- + +## `list_set_files` + +List all `.set` files in the MT5 tester profiles directory with param counts, swept param counts, and total combinations per file. Use to find the right `.set` without reading each one. + +### Input schema + +```typescript +{ + ea?: string; // Filter by EA name substring (case-insensitive) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + profiles_dir: string; + count: number; + files: Array<{ + name: string; // filename only + param_count: number; + swept_count: number; + total_combinations: number; // 0 for backtest-only .set files + modified: string; // "YYYY-MM-DD HH:MM" + error?: string; // Present only if file is unreadable + }>; +} +``` + +--- + +## `get_active_account` + +Get current MT5 account session information: login, server, and available symbols. This is essential for pre-flight checks to ensure symbol availability before backtesting. + +### Input schema + +```typescript +{} // No parameters +``` + +### Output schema + +```typescript +{ + success: boolean; + ready_for_backtest: boolean; // true if account exists and symbols available + account: { + login: string; + server: string; + } | null; + server: string; // Active server name + available_servers: string[]; // All servers with history data + symbols: string[]; // Symbols available for active server + symbol_count: number; + hint: string; // "Ready for backtesting" or instructions +} +``` + +--- + +## `check_symbol_data_status` + +Validate if a symbol has sufficient historical tick data for a specified date range before running backtest. Prevents failed backtests due to missing history data. + +### Input schema + +```typescript +{ + symbol: string; // e.g., "XAUUSDc" + from_date: string; // "YYYY.MM.DD" + to_date: string; // "YYYY.MM.DD" +} +``` + +### Output schema + +```typescript +{ + success: boolean; + symbol: string; + server: string; + has_sufficient_data: boolean; + requested_range: { from: string; to: string }; + data_range: string; // "YYYY.MM.DD - YYYY.MM.DD" or "unknown" + years_available: number; // Count of years with data + hcc_files_count: number; // Number of history cache files + warnings: string[] | null; // Data range issues + suggestion: string; // Action recommendation +} +``` + +--- + +## `check_mt5_status` + +Check if MT5 terminal is properly installed and configured. Returns comprehensive status of all required components. + +### Input schema + +```typescript +{} // No parameters +``` + +### Output schema + +```typescript +{ + success: boolean; + terminal_ready: boolean; // true if all components present + checks: { + mt5_dir_exists: boolean; + terminal64_exe: boolean; + metaeditor64_exe: boolean; + metatester64_exe: boolean; + wine_executable: boolean; + wine_path: string | null; + }; + mt5_version: string | null; + current_account: { + login: string; + server: string; + } | null; + hint: string; +} +``` + +--- + +## `get_backtest_history` + +List all backtests previously run for a specific EA and/or symbol with summary metrics. Use for tracking performance over time. + +### Input schema + +```typescript +{ + expert?: string; // Filter by EA name + symbol?: string; // Filter by symbol + limit?: number; // Max results (default: 10) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + count: number; + total: number; + filters: { + expert: string | null; + symbol: string | null; + }; + history: Array<{ + report_dir: string; + date: string | null; + expert: string | null; + symbol: string | null; + period: string | null; + profit: number | null; + profit_factor: number | null; + expected_payoff: number | null; + drawdown_pct: number | null; + total_trades: number | null; + win_rate: number | null; + }>; + hint: string; +} +``` + +--- + +## `compare_backtests` + +Compare two or more backtest results side-by-side with key metrics analysis. Includes profit/drawdown differences and verdict on which performed better. + +### Input schema + +```typescript +{ + report_dirs: string[]; // List of report directory paths to compare +} +``` + +### Output schema + +```typescript +{ + success: boolean; + count: number; + comparisons: Array<{ + report_dir: string; + expert: string | null; + symbol: string | null; + net_profit: number | null; + profit_factor: number | null; + drawdown_pct: number | null; + total_trades: number | null; + win_rate: number | null; + expected_payoff: number | null; + recovery_factor: number | null; + sharpe_ratio: number | null; + }>; + analysis: Array<{ + compare_to: string | null; + report: string | null; + profit_diff: number; + profit_pct_change: number; + drawdown_diff: number; + profit_factor_diff: number; + verdict: "better" | "worse" | "mixed"; + }> | null; + verdict: string | null; // "Best: " +} +``` + +--- + +## `init_project` + +Create a new MQL5 project with standard directory structure and template files. Supports scalper, swing, grid, and basic templates. + +### Input schema + +```typescript +{ + name: string; // Project name (used for EA filename) + template?: "scalper" | "swing" | "grid" | "basic"; // Default: "basic" +} +``` + +### Output schema + +```typescript +{ + success: boolean; + project_name: string; + template: string; + created_files: string[]; // Paths to created files + hint: string; +} +``` + +--- + +## `validate_ea_syntax` + +Perform pre-compile syntax check on MQL5 source file without running full compilation. Detects common issues before expensive MetaEditor compilation. + +### Input schema + +```typescript +{ + path: string; // Path to .mq5 source file +} +``` + +### Output schema + +```typescript +{ + success: boolean; + valid: boolean; + path: string; + checks: { + has_on_init: boolean; + has_on_tick: boolean; + has_on_deinit: boolean; + lines: number; + }; + errors: Array<{ + line: number; + message: string; + severity: "error"; + }> | null; + warnings: Array<{ + line: number; + message: string; + severity: "warning"; + }> | null; + hint: string; +} +``` + +--- + +## `create_set_template` + +Generate a .set parameter file template based on an EA's input variables. Automatically parses input declarations from source code. + +### Input schema + +```typescript +{ + ea: string; // EA name or path to .mq5/.ex5 file + output_path?: string; // Optional custom output path +} +``` + +### Output schema + +```typescript +{ + success: boolean; + ea: string; + inputs_found: number; + inputs: Array<{ + name: string; + type: string; + default: string; + description: string | null; + }>; + set_file: string; // Path to generated file + hint: string; +} +``` + +--- + +## `export_report` + +Export backtest report to various formats (CSV, JSON, Markdown) for external analysis or sharing. + +### Input schema + +```typescript +{ + report_dir: string; // Path to backtest report directory + format?: "csv" | "json" | "md"; // Default: "csv" + output_path?: string; // Optional custom output file path +} +``` + +### Output schema + +```typescript +{ + success: boolean; + format: string; + output_file: string; + source: string; + hint: string; +} +``` + +--- + +## `archive_report` + +Convert a backtest report directory into a compact JSON entry appended to `config/backtest_history.json`. Idempotent — re-archiving the same report is a no-op. Optionally deletes the source directory to reclaim disk space. + +### Input schema + +```typescript +{ + report_dir?: string; // Directory to archive. Omit for latest. + delete_after?: boolean; // Delete source dir after archiving (default: false) + verdict?: "winner" | "loser" | "marginal" | "reference"; + notes?: string; // Free-text notes for the entry + tags?: string[]; // Tags e.g. ["tight-sl", "new-filter"] +} +``` + +### Output schema + +```typescript +{ + success: boolean; + id: string; // Report dir basename used as history entry id + already_existed: boolean; + deleted_source: boolean; + history_file: string; // Absolute path to backtest_history.json + entry_summary: { + ea: string; + symbol: string; + metrics: { net_profit: number; profit_factor: number; max_dd_pct: number; sharpe_ratio: number; total_trades: number; }; + verdict: string | null; + }; +} +``` + +--- + +## `archive_all_reports` + +Bulk-archive all backtest report directories into `config/backtest_history.json`. Entries already in history are skipped. Use `delete_after=true` to reclaim disk space while preserving all results as JSON. Optimization dirs (`_opt` suffix) are never deleted. + +### Input schema + +```typescript +{ + delete_after?: boolean; // Delete source dirs after archiving (default: false) + keep_last?: number; // Protect newest N dirs from deletion even with delete_after=true (default: 5) + dry_run?: boolean; // Preview without making changes (default: false) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + dry_run: boolean; + archived_count: number; + skipped_count: number; // Already in history + deleted_count: number; + failed_count: number; // Dirs with no parseable metrics + archived: string[]; + skipped: string[]; + deleted: string[]; + failed: string[]; + history_file: string; +} +``` + +--- + +## `get_history` + +Query `config/backtest_history.json` with filters and sorting. Strips `monthly_pnl` arrays by default — set `include_monthly=true` when you need the full breakdown. + +### Input schema + +```typescript +{ + ea?: string; // Substring match on EA name + symbol?: string; // Exact match (uppercase) + verdict?: "winner" | "loser" | "marginal" | "reference"; + tag?: string; // Entry must contain this tag + min_profit?: number; // net_profit >= this + max_dd_pct?: number; // max_dd_pct <= this + sort_by?: "date" | "profit" | "dd" | "sharpe"; // Default: date, newest first + limit?: number; // Default: 20 + include_monthly?: boolean; // Include monthly_pnl arrays (default: false) +} +``` + +### Output schema + +```typescript +{ + success: boolean; + count: number; + entries: Array<{ + id: string; // Report dir basename + archived_at: string; // ISO timestamp + report_dir_deleted: boolean; + ea: string; + symbol: string; + timeframe: string; + from_date: string; + to_date: string; + metrics: { + net_profit: number; + profit_factor: number; + max_dd_pct: number; + sharpe_ratio: number; + total_trades: number; + recovery_factor: number; + win_rate_pct: number; + expected_payoff: number; + }; + summary?: { + green_months: number; + total_months: number; + worst_month: string; + worst_month_pnl: number; + dominant_exit?: string; + max_win_streak?: number; + max_loss_streak?: number; + }; + worst_dd_event?: { + peak_dd_pct: number; + start_date: string; + end_date: string; + duration_days: number; + cause: string; + }; + monthly_pnl?: Array<{ month: string; pnl: number; trades: number; green: boolean; }>; + verdict: string | null; + notes: string; + tags: string[]; + promoted_to_baseline: boolean; + }>; +} +``` + +--- + +## `promote_to_baseline` + +Write a backtest result to `config/baseline.json` — the production reference used by `compare_baseline` and the Claude Code baseline hook. Also marks the source history entry as `promoted_to_baseline: true`. + +### Input schema + +```typescript +{ + // Provide one: history_id, report_dir, or neither (uses latest report) + history_id?: string; // Entry id from get_history + report_dir?: string; // Direct path to report directory + notes?: string; // Written to baseline.json notes field +} +``` + +### Output schema + +```typescript +{ + success: boolean; + baseline_file: string; + baseline: { + ea: string; + symbol: string; + period: string; // "YYYY-MM-DD/YYYY-MM-DD" + net_profit: number; + profit_factor: number; + max_drawdown_pct: number; + sharpe_ratio: number; + total_trades: number; + recovery_factor: number; + promoted_from: string; // History entry id + promoted_at: string; // Date promoted (YYYY-MM-DD) + notes: string; + }; +} +``` + +### Example + +```json +// Input +{ "history_id": "20250619_143022_MyEA_XAUUSD_M5", "notes": "v1.3 after walk-forward validation" } + +// Output +{ + "success": true, + "baseline_file": "/path/to/config/baseline.json", + "baseline": { + "ea": "MyEA", + "symbol": "XAUUSD", + "period": "2025-01-01/2025-06-30", + "net_profit": 4832.10, + "profit_factor": 1.54, + "max_drawdown_pct": 12.3, + "sharpe_ratio": 1.18, + "total_trades": 891, + "recovery_from": "20250619_143022_MyEA_XAUUSD_M5", + "promoted_at": "2025-06-20", + "notes": "v1.3 after walk-forward validation" + } +} +``` + +--- + +## `annotate_history` + +Update the verdict, notes, or tags on an existing history entry. Use this after `compare_baseline` to record the decision, or to tag runs for later retrieval. + +### Input schema + +```typescript +{ + history_id: string; // Required — entry id to update + verdict?: "winner" | "loser" | "marginal" | "reference"; + notes?: string; // Replaces existing notes + tags?: string[]; // Replaces existing tags + add_tags?: string[]; // Appends to existing tags without overwriting +} +``` + +### Output schema + +```typescript +{ + success: boolean; + id: string; + verdict: string | null; + notes: string; + tags: string[]; +} +``` + +--- + +## Token-efficient usage patterns + +### Surveying past runs + +``` +list_reports(limit=10) → see what's there (live dirs) +get_history(ea="MyEA", limit=10) → see what's been archived +analyze_report(report_dir=X) → drill into one specific run +``` + +Never call `analyze_report` on multiple directories to find the best run — use `list_reports` or `get_history` first. + +### Checking logs without noise + +``` +tail_log(job_id=X, filter=errors) → only failures +tail_log(report_dir=X, n=20) → last 20 lines of backtest progress +``` + +### Managing disk space + +``` +archive_all_reports(dry_run=true) → preview what would be archived +archive_all_reports(delete_after=true, keep_last=3) → archive all, delete old, keep 3 newest +get_history(sort_by=profit, limit=5) → find best archived runs +``` + +### Labelling experiments + +``` +annotate_history(history_id=X, verdict="loser", notes="SL too tight, reversed at L3") +annotate_history(history_id=X, add_tags=["walk-forward-fail"]) +get_history(verdict="winner") → all winners across all sessions +``` + +### Promoting a new production config + +``` +run_backtest(...) +compare_baseline(...) → get verdict +archive_report(delete_after=true, verdict="winner") +promote_to_baseline(notes="v1.4 after WF") → update baseline.json +``` + +### Managing cache + +``` +cache_status() → see symbol breakdown and total size +clean_cache(symbol=XAUUSD, dry_run=true) → preview +clean_cache(symbol=XAUUSD) → execute +``` + +### Pre-flight validation + +``` +get_active_account() → current login, server, available symbols +check_symbol_data_status(symbol=XAUUSD, from=2025.01.01, to=2025.03.31) + → verify data availability before backtest +check_mt5_status() → verify MT5 installation and readiness +validate_ea_syntax(path=MyEA.mq5) → pre-compile syntax check +``` + +### Project management + +``` +init_project(name=MyStrategy, template=scalper) → scaffold new EA with template +create_set_template(ea=MyEA) → generate .set from EA inputs +export_report(report_dir=..., format=csv) → export to CSV/JSON/Markdown +``` + +### History and comparison + +``` +get_backtest_history(expert=MyEA, limit=10) → list past backtests with metrics +compare_backtests(report_dirs=["dir1", "dir2"]) → side-by-side comparison +``` + +### Working with set files + +``` +# Inspect +list_set_files(ea="MyEA") → all variants, swept param counts, combinations +describe_sweep(path=MyEA_opt.set) → verify 240 combinations before launching opt +diff_set_files(a=v1.2.set, b=v1.3.set) → only changed params, not full file content + +# Edit (never read+write manually) +patch_set_file(path, {TP_Pips: 350}) → change one param, keep everything else intact +clone_set_file(src, dest, overrides) → create variant from base in one call + +# Generate after optimization +set_from_optimization( → map results[0].params → clean backtest .set + path=MyEA_v1.3.set, + params=results[0].params, + template=MyEA_base.set → fills non-swept params from existing file +) +``` + +--- + +## Autonomous Loop Pattern + +The tools are designed to support a fully autonomous experiment → evaluate → promote → optimize loop: + +``` +1. run_backtest(new_params) +2. compare_baseline(result, current_production) +3a. if winner: + - archive_report(delete_after=true, verdict="winner") + - promote_to_baseline(notes="...") + - write_set_file(new_production.set) + - run_optimization(new_production_set) + - [wait for user signal] + - get_optimization_results() + - set_from_optimization(path=verify.set, params=results[0].params, template=prod.set) + - verify top result: run_backtest(expert, set_file=verify.set, skip_compile=true) + - if still beats baseline → goto step 1 +3b. if loser: + - archive_report(delete_after=true, verdict="loser", notes="root cause") + - analyze_report(result) → find root cause + - read_set_file() → inspect current params + - propose parameter or code change → goto step 1 +``` + +No user confirmation needed between steps 1→2→3. The AI agent drives the full loop; the user monitors and signals when optimization completes (since that runs for hours). Every run is archived before the directory is deleted, so nothing is lost. diff --git a/mcp-package/docs/QUICKSTART.md b/mcp-package/docs/QUICKSTART.md new file mode 100644 index 0000000..bf04b40 --- /dev/null +++ b/mcp-package/docs/QUICKSTART.md @@ -0,0 +1,191 @@ +# Quickstart Guide + +## 1. Download or Build + +### Option A: Prebuilt Binary (Recommended) + +```bash +# macOS (Apple Silicon) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz +tar -xzf mt5-quant.tar.gz + +# Linux (x64) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz +tar -xzf mt5-quant.tar.gz +``` + +### Option B: Build from Source + +```bash +git clone https://github.com/masdevid/mt5-mcp +cd mt5-mcp +bash scripts/build-rust.sh +``` + +## 2. Install MetaTrader 5 + +### macOS - MetaTrader 5.app (Free) + +1. Download from [metatrader5.com](https://www.metatrader5.com/en/download) +2. Install to `/Applications` +3. **Launch once** to initialize Wine prefix (~30s), then quit + +Auto-detected paths: +- Wine: `/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64` +- MT5: `~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5` + +### macOS - CrossOver (Paid, Better Compatibility) + +1. Install [CrossOver](https://www.codeweavers.com/) +2. Create bottle `MetaTrader5` +3. Install MT5 inside bottle + +Auto-detected paths: +- Wine: `/Applications/CrossOver.app/Contents/SharedSupport/CrossOver/bin/wine64` +- MT5: `~/Library/Application Support/MetaQuotes//drive_c/Program Files/MetaTrader 5` + +### Linux + +```bash +# Debian/Ubuntu +sudo apt install wine64 xvfb + +# Fedora/RHEL +sudo dnf install wine xorg-x11-server-Xvfb + +# Install MT5 +wine64 MetaTrader5Setup.exe +``` + +MT5 location: `~/.wine/drive_c/Program Files/MetaTrader 5` + +## 3. Configure + +Run the setup script to auto-detect paths: + +```bash +bash scripts/setup.sh # interactive +bash scripts/setup.sh --yes # non-interactive (CI) +``` + +This creates `config/mt5-quant.yaml` (gitignored). + +Minimum config: +```yaml +wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64" +terminal_dir: "~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5" +``` + +## 4. Register MCP Server + +### Claude Code + +```bash +claude mcp add MT5-Quant -- /path/to/mt5-quant/target/release/mt5-quant +``` + +Verify: +```bash +claude mcp list +``` + +### Windsurf + +Add to `~/.codeium/windsurf/mcp_config.json`: + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/path/to/mt5-quant" + } + } +} +``` + +### Cursor + +Add to `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project): + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/path/to/mt5-quant" + } + } +} +``` + +Or use Settings → MCP → Add Custom MCP. + +### VS Code + +Add to `.vscode/mcp.json` in your workspace: + +```json +{ + "servers": { + "mt5-quant": { + "command": "/path/to/mt5-quant" + } + } +} +``` + +Or run `MCP: Add Server` from Command Palette. + +### Antigravity + +1. Open Agent panel → Click "..." menu → MCP Servers +2. Click "Manage MCP Servers" +3. Click "View raw config" or "Edit configuration" +4. Add to `mcp_config.json`: + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/path/to/mt5-quant" + } + } +} +``` +5. Reload Antigravity to apply changes + +## 5. Verify Setup + +```bash +bash scripts/platform_detect.sh +``` + +Or in Claude/Windsurf: +``` +Run verify_setup +``` + +Expected output: +``` +Wine: /Applications/MetaTrader 5.app/.../wine64 +MT5 dir: ~/Library/Application Support/.../MetaTrader 5 +Display: gui +Arch: arch -x86_64 +``` + +## 6. Run First Backtest + +``` +Run a backtest on MyEA from 2025.01.01 to 2025.03.31 +``` + +The AI will: +1. Verify setup +2. Compile your EA +3. Clean MT5 cache +4. Run backtest +5. Extract and analyze results +6. Report key findings + +--- + +**Next:** See [TOOLS.md](TOOLS.md) for all 43 available tools. diff --git a/mcp-package/docs/REMOTE_AGENTS.md b/mcp-package/docs/REMOTE_AGENTS.md new file mode 100644 index 0000000..7f4371c --- /dev/null +++ b/mcp-package/docs/REMOTE_AGENTS.md @@ -0,0 +1,187 @@ +# Remote Agent Setup (Linux Server) + +MT5's distributed testing lets you farm optimization work across multiple machines. A Linux server running `metatester64.exe` via Wine connects to a Mac master over a local network. + +**Throughput:** Each agent handles one pass at a time. 10 local + 16 remote = 26 agents. A 17,000-combination genetic optimization that takes 3 hours locally finishes in ~70 minutes. + +--- + +## Requirements + +**Mac (master)** +- MetaTrader 5 installed via CrossOver or Wine +- MT5-Quant configured and working for local backtests +- Port 3000 open in macOS firewall (or whichever port MT5 uses — check below) + +**Linux server (agents)** +- Wine 7.0+ (or Wine Staging for better compatibility) +- `metatester64.exe` from an MT5 installation +- Access to the same MT5 data files (tick history) as the master — or let agents download on first run + +--- + +## Step 1: Find the MT5 agent port on Mac + +After running a local optimization, check the MT5 agent directories: + +```bash +ls ~/Library/Application\ Support/MetaQuotes/*/drive_c/Program\ Files/MetaTrader\ 5/ +``` + +You'll see directories like: +``` +Agent-127.0.0.1-3000/ ← local agents (loopback) +Agent-127.0.0.1-3001/ +Agent-0.0.0.0-3000/ ← remote listener (if enabled) +``` + +If you don't see `Agent-0.0.0.0-*` directories, enable remote agents in MT5: +**Tools → Options → Expert Advisors → Allow remote agents** + +Note the port number (default: 3000). + +--- + +## Step 2: Open firewall on Mac + +```bash +# Check current firewall rules +sudo pfctl -s rules + +# Allow incoming on agent port (example: 3000) +# Add to /etc/pf.conf or use macOS Firewall in System Settings +``` + +Or use macOS System Settings → Privacy & Security → Firewall → Firewall Options → Add MetaTrader 5. + +--- + +## Step 3: Install Wine on Linux server + +```bash +# Ubuntu/Debian +sudo dpkg --add-architecture i386 +sudo apt update +sudo apt install wine64 wine32 + +# Verify +wine64 --version +``` + +--- + +## Step 4: Copy `metatester64.exe` to Linux + +From your Mac MT5 installation: +```bash +MAC_MT5="$HOME/Library/Application Support/MetaQuotes/Terminal/XXXXX/drive_c/Program Files/MetaTrader 5" + +scp "${MAC_MT5}/metatester64.exe" user@linux-server:~/mt5agents/ +scp "${MAC_MT5}/metaeditor64.exe" user@linux-server:~/mt5agents/ # not required for agents +``` + +Also copy the required DLLs (they're in the same directory): +```bash +scp "${MAC_MT5}"/*.dll user@linux-server:~/mt5agents/ +``` + +--- + +## Step 5: Launch agents on Linux + +```bash +# On the Linux server +cd ~/mt5agents/ + +# Replace MAC_IP with your Mac's local IP address +# Replace 8 with number of CPU cores - 1 +wine64 metatester64.exe /server:192.168.1.100:3000 /agents:8 +``` + +**What this does:** +- Connects to your Mac's MT5 master at `192.168.1.100:3000` +- Registers 8 worker agents +- MT5 on Mac will show 8 new agents in the agent manager + +**To run as a background service:** +```bash +nohup wine64 metatester64.exe /server:192.168.1.100:3000 /agents:8 \ + > ~/mt5agents/agents.log 2>&1 & +disown +``` + +--- + +## Step 6: Verify agents appear in MT5 + +On your Mac, open MT5: +**View → Strategy Tester → Agents tab** + +You should see entries like: +``` +Agent-192.168.1.200-3000 [Active] +Agent-192.168.1.200-3001 [Active] +... +``` + +If agents appear as `[Inactive]`, check: +1. Mac firewall is allowing incoming connections on the agent port +2. Linux server can reach Mac IP: `ping 192.168.1.100` +3. Port is open: `nc -zv 192.168.1.100 3000` + +--- + +## Step 7: Configure MT5-Quant for remote agents + +In `config/MT5-Quant.yaml`: +```yaml +optimization: + remote_agents: + enabled: true + check_agent_count: true # Verify remote agents are connected before launching + min_agents: 4 # Require at least N agents before optimizing +``` + +MT5-Quant will log the agent count before launching optimization: +``` +[optimize] Local agents: 9, Remote agents: 8, Total: 17 +[optimize] Estimated completion: ~105 minutes (17,640 passes at 17 agents) +``` + +--- + +## Tick Data Sync + +Remote agents need tick history to replay trades. On first run, MT5 automatically downloads ticks from the broker for each symbol+period combination tested. This can take 10-30 minutes per symbol. + +**Speed up first run:** Pre-populate the tick cache on the Linux server by copying from Mac: + +```bash +# On Mac — find tick data location +find ~/Library/Application\ Support/MetaQuotes -name "*.bin" | grep "XAUUSD" + +# Typical path: +# Terminal/XXXXX/drive_c/users/user/AppData/Roaming/MetaQuotes/Terminal/Common/Files/ + +scp -r "${TICK_DIR}" user@linux-server:~/mt5agents/ticks/ +``` + +--- + +## Troubleshooting + +**Agents connect then immediately disconnect** +- MT5 version mismatch between `metatester64.exe` (from Mac) and the master. Use the exact same build number. +- Check `~/mt5agents/agents.log` for Wine errors. + +**Agents show as connected but don't receive work** +- MT5 sometimes requires optimization to be started before assigning work to new agents. +- Start an optimization from the MT5 GUI first to "activate" remote agents, then cancel it and use MT5-Quant. + +**Performance is slower with remote agents** +- Network latency between Mac and Linux server. Results are sent over TCP after each pass. +- Local gigabit network: negligible. WiFi or WAN: significant overhead. Use wired connection. + +**Linux server runs out of memory** +- Each `metatester64.exe` instance uses ~200-400MB. +- 8 agents = ~2-3GB RAM. Size agent count accordingly. diff --git a/mcp-package/docs/TROUBLESHOOTING.md b/mcp-package/docs/TROUBLESHOOTING.md new file mode 100644 index 0000000..2965e35 --- /dev/null +++ b/mcp-package/docs/TROUBLESHOOTING.md @@ -0,0 +1,105 @@ +# Troubleshooting + +Run `verify_setup` first — it checks all paths and returns actionable hints. + +## Wine Not Found + +### macOS + +Confirm `/Applications/MetaTrader 5.app` exists and has been launched at least once. + +Check detection: +```bash +bash scripts/platform_detect.sh +``` + +If using CrossOver, confirm bottle is named `MetaTrader5`. + +### Linux + +```bash +sudo apt install wine64 # Debian/Ubuntu +sudo dnf install wine # Fedora/RHEL +which wine64 # confirm on PATH +``` + +## terminal64.exe Missing + +MT5 unpacks `terminal64.exe` only after first launch. + +1. Open MetaTrader 5.app +2. Wait for initialization (~30s) +3. Quit +4. Re-run setup: +```bash +bash scripts/setup.sh --yes +``` + +## MCP Server Not Appearing + +### Claude Code + +```bash +claude mcp list # should show MT5-Quant +claude mcp remove MT5-Quant # remove stale entry +claude mcp add MT5-Quant -- /absolute/path/to/mt5-quant +``` + +**Must use absolute path** — relative paths break when Claude starts from different directories. + +### Windsurf + +1. Check logs: `~/.windsurf/logs/` +2. Verify executable path is absolute +3. Test manually: `./mt5-quant --help` + +## Config Not Found + +Set `MT5_MCP_HOME` or ensure config exists at: +- macOS: `~/.config/mt5-quant/config/mt5-quant.yaml` +- Project: `config/mt5-quant.yaml` + +## Report Not Found After Backtest + +1. **Wrong symbol name** — brokers use custom names (`XAUUSDm`, `XAUUSD.cent`). Check `verify_setup` or look in `/history/`. + +2. **No history data** — open MT5, open symbol chart, wait for history download. + +3. **EA crash at startup** — check `/MQL5/Logs/` for `OnInit` errors. + +4. **Date range has no trades** — try wider range or confirm symbol was active. + +## MetaEditor Compile Errors + +Check `/MQL5/Logs/`: + +- **Missing `#include`** — copy dependencies into `Experts/` alongside `.mq5` +- **Stale `.ex5`** — delete old binary and recompile + +## No Deals in Backtest Report + +- Use `model=0` (every tick) — models 1/2 skip intra-bar movement, producing zero deals for grid/martingale EAs +- Check `.set` file values appropriate for symbol/broker +- Confirm `OnInit()` returns `INIT_SUCCEEDED` (MT5 Journal tab) + +## Optimization Never Finishes + +```bash +# From Claude: +tail_log(job_id=X, filter=errors) +get_optimization_status(job_id=X) +``` + +If MT5 crashed, edit `/terminal.ini` and remove line containing `OptMode=-1`, then retry. + +## Permission Denied + +```bash +chmod +x /path/to/mt5-quant +``` + +## Still Stuck? + +1. Run `verify_setup` and share output +2. Check `tail_log` for errors +3. Review `/MQL5/Logs/` for EA errors diff --git a/mcp-package/docs/VSCODE.md b/mcp-package/docs/VSCODE.md new file mode 100644 index 0000000..801bc84 --- /dev/null +++ b/mcp-package/docs/VSCODE.md @@ -0,0 +1,147 @@ +# VS Code MCP Integration Setup + +## Quick Setup + +### Option 1: Download Prebuilt Binary (Recommended) + +```bash +# macOS (Apple Silicon) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz +tar -xzf mt5-quant.tar.gz + +# Linux (x64) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz +tar -xzf mt5-quant.tar.gz +``` + +### Option 2: Build from Source + +```bash +cargo build --release +``` + +## Configure VS Code + +### Method 1: Command Palette (Recommended) + +1. Press `Cmd/Ctrl + Shift + P` +2. Run `MCP: Add Server` +3. Choose **Workspace** or **User** scope +4. Enter server name: `mt5-quant` +5. Enter command: Full path to binary (e.g., `/Users/name/mt5-quant/target/release/mt5-quant`) + +### Method 2: Edit mcp.json Directly + +Add to `.vscode/mcp.json` in your workspace: + +```json +{ + "servers": { + "mt5-quant": { + "command": "/absolute/path/to/mt5-quant" + } + } +} +``` + +Create the file: + +```bash +mkdir -p .vscode +cat > .vscode/mcp.json << 'EOF' +{ + "servers": { + "mt5-quant": { + "command": "/path/to/mt5-quant" + } + } +} +EOF +``` + +### Method 3: VS Code CLI + +```bash +code --add-mcp '{"name":"mt5-quant","command":"/path/to/mt5-quant"}' +``` + +## Verify Setup + +In Copilot chat, type: + +``` +Run verify_setup +``` + +Expected output: +``` +Wine: /Applications/MetaTrader 5.app/.../wine64 +MT5 dir: ~/Library/Application Support/.../MetaTrader 5 +Display: gui +Arch: arch -x86_64 +``` + +## Configuration Locations + +| Scope | Path | Use Case | +|-------|------|----------| +| Workspace | `.vscode/mcp.json` | Share with team via source control | +| User | `~/.vscode/mcp.json` | Personal tools across all projects | +| Dev Container | `devcontainer.json` → `customizations.vscode.mcp` | Containerized environments | + +## Troubleshooting + +### MCP server not appearing + +1. Open **Output** panel (`Cmd/Ctrl + Shift + U`) +2. Select **MCP** from dropdown +3. Check for connection errors +4. Verify the path is absolute + +### Config not found + +The binary auto-detects its config, but you can also: +1. Run `setup.sh` to create `config/mt5-quant.yaml` +2. Or let the binary auto-discover on first run + +### Dev Container Setup + +Add to `.devcontainer/devcontainer.json`: + +```json +{ + "customizations": { + "vscode": { + "mcp": { + "servers": { + "mt5-quant": { + "command": "/path/to/mt5-quant" + } + } + } + } + } +} +``` + +## Key Differences from Other IDEs + +VS Code uses `servers` (not `mcpServers`) in the JSON structure: + +```json +{ + "servers": { // ← VS Code uses "servers" + "mt5-quant": { + "command": "..." + } + } +} +``` + +Other platforms use `mcpServers`. + +## Resources + +- [VS Code MCP Documentation](https://code.visualstudio.com/docs/copilot/customization/mcp-servers) +- [MCP Configuration Reference](https://code.visualstudio.com/docs/copilot/reference/mcp-configuration) +- [MT5-Quant Tools Reference](./MCP_TOOLS.md) diff --git a/mcp-package/docs/WINDSURF.md b/mcp-package/docs/WINDSURF.md new file mode 100644 index 0000000..78c88bd --- /dev/null +++ b/mcp-package/docs/WINDSURF.md @@ -0,0 +1,104 @@ +# Windsurf MCP Integration Setup + +## Quick Setup + +### Option 1: Download Prebuilt Binary (Recommended) + +```bash +# macOS (Apple Silicon) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz +tar -xzf mt5-quant.tar.gz + +# Linux (x64) +curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz +tar -xzf mt5-quant.tar.gz +``` + +### Option 2: Build from Source + +```bash +bash scripts/build-rust.sh +``` + +### 2. Configure Windsurf + +Edit `~/.codeium/windsurf/mcp_config.json`: + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/Users/masdevid/jobs/mt5-quant/target/release/mt5-quant" + } + } +} +``` + +Or use the automated setup: + +```bash +bash scripts/setup.sh +``` + +### 3. Restart Windsurf +Close and reopen Windsurf to load the MCP server. + +### 4. Verify +In Windsurf chat, test with: +``` +Run verify_setup +``` + +## Deployment to Multiple Machines + +### Build for Distribution +```bash +# Build release binary +cargo build --release + +# Create tarball +tar -czf mt5-quant-macos-arm64.tar.gz -C target/release mt5-quant + +# Deploy to remote server +scp mt5-quant-macos-arm64.tar.gz user@server:~/ +ssh user@server "tar -xzf mt5-quant-macos-arm64.tar.gz -C /opt/" +ssh user@server "ln -s /opt/mt5-quant /usr/local/bin/" + +# Copy config +scp -r config/mt5-quant.yaml user@server:~/.config/mt5-quant/config/ +``` + +### Target Machine Requirements +- MetaTrader 5 installed (via Wine/CrossOver) +- Config file at `~/.config/mt5-quant/config/mt5-quant.yaml` +- **NO Python required!** + +### Windsurf Config on Target Machine + +```json +{ + "mcpServers": { + "mt5-quant": { + "command": "/usr/local/bin/mt5-quant" + } + } +} +``` + +The binary auto-detects its config location. No environment variables needed. + +## Troubleshooting + +### MCP server not appearing +1. Check Windsurf logs: `~/.windsurf/logs/` +2. Verify executable path is absolute +3. Test executable manually: `./target/release/mt5-quant --help` + +### Config not found +Set `MT5_MCP_HOME` environment variable or ensure config is at default location: +- macOS: `~/.config/mt5-quant/config/mt5-quant.yaml` + +### Permission denied +```bash +chmod +x /path/to/mt5-quant +``` diff --git a/mcp-package/mcp-server/bin/mt5-quant b/mcp-package/mcp-server/bin/mt5-quant new file mode 100755 index 0000000..d3cfb1b Binary files /dev/null and b/mcp-package/mcp-server/bin/mt5-quant differ diff --git a/mcp-package/server.json b/mcp-package/server.json new file mode 100644 index 0000000..39a178c --- /dev/null +++ b/mcp-package/server.json @@ -0,0 +1,30 @@ +{ + "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "name": "io.github.masdevid/mt5-quant", + "title": "MT5-Quant", + "description": "MCP server for MT5 strategy development. 85 tools for backtesting, optimizing, and debugging MQL5 EAs on macOS/Linux.", + "repository": { + "url": "https://github.com/masdevid/mt5-quant", + "source": "github" + }, + "version": "1.30.0", + "packages": [ + { + "registryType": "mcpb", + "version": "1.30.0", + "identifier": "https://github.com/masdevid/mt5-quant/releases/download/v1.30.0/mcp-mt5-quant-macos-arm64.tar.gz", + "fileSha256": "PLACEHOLDER_SHA256", + "transport": { + "type": "stdio" + }, + "environmentVariables": [ + { + "description": "Path to MT5-Quant config directory (default: ~/.config/mt5-quant)", + "isRequired": false, + "format": "string", + "name": "MT5_MCP_HOME" + } + ] + } + ] +} diff --git a/server.json b/server.json index 7d85e97..dabeb61 100644 --- a/server.json +++ b/server.json @@ -2,7 +2,7 @@ "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", "name": "io.github.masdevid/mt5-quant", "title": "MT5-Quant", - "description": "MCP server for MT5 strategy development with 85 tools. Compile, backtest, analyze, optimize, and debug MQL5 EAs on macOS/Linux. Includes Wine/MT5 crash diagnostics.", + "description": "MCP server for MT5 strategy development. 85 tools for backtesting, optimizing, and debugging MQL5 EAs on macOS/Linux.", "repository": { "url": "https://github.com/masdevid/mt5-quant", "source": "github" @@ -13,7 +13,7 @@ "registryType": "mcpb", "version": "1.30.0", "identifier": "https://github.com/masdevid/mt5-quant/releases/download/v1.30.0/mcp-mt5-quant-macos-arm64.tar.gz", - "fileSha256": "PLACEHOLDER_SHA256", + "fileSha256": "6028b87c5be71db157c83e9804936190d9c3c546089e728b87496cbfc4168c0d", "transport": { "type": "stdio" },