2 Commits

Author SHA1 Message Date
Devid HW 740f24784f release: v1.31.2
- Version bump 1.31.1 → 1.31.2
- server.json identifier URL updated (SHA256 set by CI after build)
- CHANGELOG.md updated
2026-04-22 07:01:47 +07:00
Devid HW d03f080b95 refactor: reduce handler boilerplate in analysis and experts modules
analysis.rs: extract required_str, ok_response, err_response, prepare_analysis,
deal_to_json, is_closed_trade helpers — 15 granular handlers reduced from
~14 lines each to 4 lines each (-293 lines total).

experts.rs: extract scan_mql_dir (replaces 6 identical WalkDir loops),
BUILTIN_INDICATORS const (deduplicates list/search handlers), and
copy_mql_to_project (merges copy_indicator/copy_script into one impl,
public handlers delegate in 1 line each) (-256 lines total).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 07:01:41 +07:00
55 changed files with 1969 additions and 5879 deletions
+114
View File
@@ -0,0 +1,114 @@
---
description: Release a new version of mt5-quant — bump, changelog, tag, push
---
# /release — MT5-Quant Release Workflow
Automates the full release cycle: version bump → changelog → git tag → GitHub push → CI triggers build + MCP publish.
## Usage
```
/release [patch|minor|major|X.Y.Z]
```
Default is `patch` if no argument is given.
---
## Step 1 — Check current version and state
```bash
grep '^version' Cargo.toml | head -1
git status --short
git log --oneline -5
```
## Step 2 — Count tools (update docs if needed)
```bash
grep -r "pub fn tool_" src/tools/definitions/ | wc -l
```
If the count changed since the last release, update:
- `README.md` — "MCP Tools (N)" section header and description
- `docs/MCP_TOOLS.md` — "documents X of Y total tools" line
## Step 3 — Run the release script
```bash
bash scripts/release.sh patch
```
Replace `patch` with `minor`, `major`, or an explicit version like `1.33.0`.
The script will:
1. Bump `Cargo.toml` version
2. Update `server.json` version + download URL (SHA256 = TBD, set by CI)
3. Generate a `CHANGELOG.md` entry from `git log` since the last tag
4. Run `cargo check` to verify the build
5. Commit all changes
6. Create an annotated git tag
7. Push branch + tag → triggers GitHub Actions
## Step 4 — Monitor GitHub Actions
```bash
gh run list --limit 5
```
Or open: https://github.com/masdevid/mt5-quant/actions
The CI pipeline will:
- Build macOS arm64 + Linux x64 binaries
- Create the GitHub Release with both artifacts
- Build the MCP installable package
- Compute the real SHA256 and commit it back to `server.json`
- Attempt to publish to the MCP Registry
## Step 5 — Install updated binary locally
```bash
cp target/release/mt5-quant ~/.local/bin/mt5-quant
```
Or rebuild from the release tag:
```bash
cargo build --release && cp target/release/mt5-quant ~/.local/bin/mt5-quant
```
## Step 6 — Verify MCP registration
```bash
claude mcp list
~/.local/bin/mt5-quant --help 2>/dev/null || echo "binary works"
```
---
## Troubleshooting
**MCP publish failed in CI?**
```bash
# Download latest mcp-publisher
curl -L "https://github.com/modelcontextprotocol/registry/releases/latest/download/mcp-publisher_linux_amd64.tar.gz" | tar xz
./mcp-publisher login github
./mcp-publisher publish
```
**Need to re-run CI without a new tag?**
```bash
gh workflow run release.yml -f version=v1.32.0
```
**Rollback a bad release?**
```bash
git tag -d v1.32.0
git push origin :refs/tags/v1.32.0
# Then delete the GitHub Release via web UI or:
gh release delete v1.32.0
```
-4
View File
@@ -1,4 +0,0 @@
# These are supported funding model platforms
github: masdevid
custom: ["https://www.paypal.me/devidhw", "https://saweria.co/depod"]
-11
View File
@@ -1,11 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "monthly"
+1 -27
View File
@@ -88,36 +88,10 @@ jobs:
name: linux-binary
path: dist/mcp-mt5-quant-linux-x64.tar.gz
# ── Cargo Publish ────────────────────────────────────────────────────────────
cargo-publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache Cargo
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: cargo-publish-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-publish-
- name: Publish to crates.io
run: cargo publish --no-verify --allow-dirty
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
# ── GitHub Release ───────────────────────────────────────────────────────────
release:
needs: [build-macos, build-linux, cargo-publish]
needs: [build-macos, build-linux]
runs-on: ubuntu-latest
permissions:
contents: write
+3 -10
View File
@@ -19,21 +19,14 @@ config/mt5-quant.yaml
config/baseline.json
config/backtest_history.json
# Agent configs (personal LLM instructions)
# Claude Code (personal, not for repo)
CLAUDE.md
.claude/
.ocx
.opencode
AGENTS.md
# Windsurf (local workflows, not for repo)
.windsurf/
# VS Code
.vscode/
# Codegraph
.codegraph/
# GitHub Actions (local workflows, not for repo)
.github/workflows/
# macOS
.DS_Store
+8
View File
@@ -0,0 +1,8 @@
{
"json.schemas": [
{
"fileMatch": ["server.json"],
"url": "https://raw.githubusercontent.com/modelcontextprotocol/specification/main/schema/2024-11-05/schema.json"
}
]
}
+113
View File
@@ -0,0 +1,113 @@
---
description: Release workflow — bump version, build, tag, push, publish MCP
tags: [release, publish, workflow]
---
# MT5-Quant Release Workflow
One-command release: `bash scripts/release.sh [patch|minor|major|X.Y.Z]`
---
## 0. Pre-release checklist
- [ ] Features implemented and tested
- [ ] Docs updated (README.md, docs/MCP_TOOLS.md) with correct tool count
- [ ] `cargo check` passes
Check tool count:
// turbo
```bash
grep -r "pub fn tool_" src/tools/definitions/ | wc -l
```
---
## 1. Run the release script
// turbo
```bash
bash scripts/release.sh patch
```
Accepts: `patch` · `minor` · `major` · `1.32.0` · `v1.32.0`
The script handles:
- Version bump in `Cargo.toml`
- `server.json` version + download URL update
- `CHANGELOG.md` entry generation from git log
- `cargo check` verification
- Release commit + annotated git tag
- `git push` → triggers GitHub Actions
---
## 2. Monitor CI
// turbo
```bash
gh run list --limit 3
```
CI pipeline stages:
1. **build-macos** / **build-linux** — parallel Rust builds with cache
2. **release** — creates GitHub Release with both binaries
3. **build-mcp-package** — packages binary + config + docs, computes SHA256
4. **update-server-json** — commits real SHA256 back to `server.json` on master
5. **mcp-publish** — publishes to MCP Registry via GitHub OIDC
6. **release-info** — prints registration commands for all MCP clients
---
## 3. Install locally
// turbo
```bash
cp target/release/mt5-quant ~/.local/bin/mt5-quant
```
---
## 4. Verify
// turbo
```bash
claude mcp list
```
---
## Manual MCP publish (if CI publish fails)
// turbo
```bash
# Get latest mcp-publisher
LATEST=$(curl -sf https://api.github.com/repos/modelcontextprotocol/registry/releases/latest | python3 -c "import sys,json; print(json.load(sys.stdin)['tag_name'])")
VERSION="${LATEST#v}"
curl -fsSL -o pub.tar.gz "https://github.com/modelcontextprotocol/registry/releases/download/${LATEST}/mcp-publisher_${VERSION}_$(uname -s | tr '[:upper:]' '[:lower:]')_amd64.tar.gz"
tar -xzf pub.tar.gz && chmod +x mcp-publisher
./mcp-publisher login github
./mcp-publisher publish
```
---
## Re-trigger CI without a new tag
// turbo
```bash
gh workflow run release.yml -f version=v1.32.0
```
---
## Rollback
```bash
# Remove tag locally and remotely
git tag -d v1.32.0
git push origin :refs/tags/v1.32.0
# Delete GitHub Release
gh release delete v1.32.0 --yes
```
-245
View File
@@ -1,245 +0,0 @@
# AGENTS.md — Developer instructions for MT5-Quant
## Project Overview
Open-source MCP server for MetaTrader 5 backtesting, optimization, and analytics — written entirely in Rust. Exposes 89 stdio MCP tools for compiling MQL5 EAs, running backtests via Wine, extracting deal-level data, organizing reports in SQLite, and performing 19 dimensions of analysis.
Target users: MQL developers on macOS (CrossOver or native MetaTrader5.app) and Linux (Wine/Xvfb).
## Repository Layout
```
MT5-Quant/
├── src/
│ ├── main.rs # CLI entry + stdio MCP transport
│ ├── mcp_server.rs # McpServer struct, tool dispatch, init/notification
│ ├── mt5.rs # Wine/MT5 executable launchers
│ ├── models/
│ │ ├── mod.rs
│ │ ├── config.rs # Config, CurrentAccount, wine detection, .set helpers
│ │ ├── deals.rs # Deal, PositionPair, DrawdownEvent
│ │ ├── metrics.rs # BacktestMetrics parsing (HTML + XML formats)
│ │ └── report.rs # ReportEntry, PipelineMetadata, BacktestJob, status
│ ├── analytics/
│ │ ├── mod.rs
│ │ ├── extract.rs # HTML/XML report → Deal[] + metrics
│ │ └── analyze.rs # DealAnalyzer: 19+ analysis methods
│ ├── compile/
│ │ ├── mod.rs
│ │ └── mql_compiler.rs # MetaEditor compiler via Wine
│ ├── pipeline/
│ │ ├── mod.rs
│ │ ├── backtest.rs # 5-stage pipeline: COMPILE→CLEAN→BACKTEST→EXTRACT→ANALYZE
│ │ └── stages.rs # PipelineStage enum
│ ├── optimization/
│ │ ├── mod.rs
│ │ ├── optimizer.rs # Background optimization launcher
│ │ └── parser.rs # Optimization result XML → best params
│ ├── storage/
│ │ ├── mod.rs
│ │ └── database.rs # ReportDb: reports + deals SQLite tables, queries
│ ├── utils/ # (new) shared utilities — growing directory
│ └── tools/
│ ├── mod.rs # MCP definitions aggregation
│ ├── definitions/ # Tool schemas (JSON Schema) — 11 modules, 90 tools
│ │ ├── mod.rs # exports: all_tools(), system_tools, etc.
│ │ ├── analytics.rs # 19 analysis tools + compare_baseline
│ │ ├── backtest.rs # 7 backtest tools
│ │ ├── baseline.rs # 1 baseline tool
│ │ ├── experts.rs # 9 EA/indicator/script tools
│ │ ├── optimization.rs # 4 optimization tools
│ │ ├── reports.rs # 20 report management tools
│ │ ├── setfiles.rs # 8 .set file tools
│ │ ├── system.rs # 6 system/health tools + version update
│ │ ├── update.rs # update tool definition
│ │ └── utility.rs # 10 pre-flight/compat tools
│ └── handlers/ # Tool dispatch functions — 11 modules
│ ├── mod.rs # ToolHandler struct, dispatch table (89+ cases)
│ ├── analysis.rs # analytics/analyze dispatch
│ ├── backtest.rs # pipeline/backtest dispatch
│ ├── experts.rs # wine + WalkDir helpers: scan_mql_dir, copy_mql_to_project
│ ├── optimization.rs # optimizer dispatch
│ ├── reports.rs # ReportDb + ReportEntry dispatch
│ ├── setfiles.rs # UTF-16LE .set read/write/patch helpers
│ ├── system.rs # healthcheck, version update
│ ├── utility.rs # pre-flight, diagnostics, init_project helpers
│ └── update.rs # update tool handler
├── scripts/
│ ├── setup.sh # Auto-detect Wine/MT5, write config (1271 lines)
│ ├── platform_detect.sh # Wine path + headless detection
│ ├── build-rust.sh # cargo build --release (25 lines)
│ ├── release.sh # Version bump + changelog + tag + push (238 lines)
│ └── optimize.sh # Legacy optimization driver (pending full Rust migration)
├── analytics/ # Legacy Python (reference only — do not modify)
├── config/
│ ├── mt5-quant.example.yaml # Template user copies to mt5-quant.yaml
│ └── mt5-quant.yaml # Generated config (gitignored)
├── tests/
│ ├── test_mcp.py # End-to-end MCP tests (stdio-driven)
│ ├── integration_test.sh # Shell-based integration tests
│ └── fixtures/ # Sample reports + CSVs for testing
├── server.json # MCP registry manifest (tracked)
├── Cargo.toml # version = 1.32.4
└── CHANGELOG.md
```
## Key Design Constraints
- **Headless/GUI**: `platform_detect.sh` auto-selects. macOS CrossOver = GUI (CrossOver abstracts display). Linux without `$DISPLAY` = Xvfb. Controlled by `display.mode` in config (`auto` | `gui` | `headless`).
- **Single MT5 instance**: Never run two backtests in parallel on the same Wine prefix.
- **Model 0 only for optimization**: Model 1 overfits martingale/grid EAs — intra-bar movement not simulated. Use `model=0` (every tick).
- **UTF-16LE .set files**: MT5 strips `||Y` flags from UTF-8. All `.set` writes use UTF-16LE + `chmod 444`. See `setfiles.rs` helpers.
- **OptMode reset**: After any optimization, `terminal.ini` gets `OptMode=-1`. Must reset to `0` before next backtest (handled in `backtest.rs` cleanup stage).
- **Report format detection**: MT5 Build 48+ writes SpreadsheetML XML (`.htm.xml`), not HTML. Both formats handled in `extract.rs` — always check for the XML variant first.
- **EA-agnostic**: Never hardcode magic numbers, strategy names, or assume specific EA behavior. All code must work with any MQL5 EA.
## Architecture
1. **CLI (`main.rs`)** — parses CLi args (`--stdio`, `--port`, `--test-launch`), initializes `McpServer`, runs stdio or TCP transport loop, handles JSON-RPC requests.
2. **McpServer (`mcp_server.rs`)** — wraps `ToolHandler` with `Arc<Mutex>`, handles `initialize`/`notifications`/`tool/error` method routing. Auto-verifies Wine/MT5 on first call.
3. **ToolHandler (`tools/handlers/mod.rs`)** — dispatch table mapping 89 tool names to handler functions. Each handler module is self-contained.
4. **Definitions (`tools/definitions/`)** — JSON Schema objects for tool input/output. Never executes logic.
5. **Analytics (`analytics/`)** — two core structs:
- `ReportExtractor` — reads HTML/XML report + tester log → `Vec<Deal>` + `BacktestMetrics`
- `DealAnalyzer` — takes `Vec<Deal>` → 19 analysis methods (monthly PnL, drawdown events, streaks, etc.)
6. **Pipeline (`pipeline/backtest.rs`)**`BacktestPipeline` struct with 5 stages:
- `COMPILE``MqlCompiler::compile()` via MetaEditor
- `CLEAN` — clean cache, reset OptMode
- `BACKTEST` — launch MT5 tester via Wine, poll status
- `EXTRACT` — parse HTML/XML → deals + metrics
- `ANALYZE` — run all analytics, write to SQLite
7. **Storage (`storage/database.rs`)**`ReportDb` with two tables:
- `reports` — metrics, set file paths, verdict, tags, notes
- `deals` — individual deal rows, keyed by `report_id`
- `CREATE TABLE IF NOT EXISTS` in `init()` — idempotent on startup
## Building
```bash
bash scripts/build-rust.sh
# or
cargo build --release
```
Binary: `target/release/mt5-quant`
## Testing
- **Unit/integration:** `cargo test`
- **End-to-end MCP:** `python3 tests/test_mcp.py` — drives the MCP server over stdio (full backtest + all analytics)
- **Analytics only:** No MT5/Wine needed; loads deals from SQLite DB via `db.get_deals(report_id)`
- **Symbol note:** Testing account uses `XAUUSDc` (broker suffix required), not `XAUUSD`
## Configuration
Users copy `config/mt5-quant.example.yaml``config/mt5-quant.yaml` (gitignored).
Run `bash scripts/setup.sh` to auto-detect Wine, MT5 paths, architecture, and write config.
Minimum config:
```yaml
wine_executable: "/path/to/wine64"
terminal_dir: "/path/to/MetaTrader 5"
```
Optional:
```yaml
defaults:
symbol: "XAUUSD.cent"
timeframe: "M5"
deposit: 10000
display:
mode: auto # auto | gui | headless
reports_dir: "./reports"
experts_dir: "~/.../MQL5/Experts"
```
## Developing New Tools
Each new tool follows a 4-file pattern:
1. **Schema** — Add tool input/output to `tools/definitions/{domain}.rs` via `Tool::new(name) {...}` with `Input::schema` and `Output::schema`.
2. **Export** — Add to `tools/definitions/mod.rs`:
- Include in `all_tools()` array
- Add to domain exports if creating new domain
3. **Handler** — Add function in `tools/handlers/{domain}.rs`:
```rust
pub async fn handle_tool_name(config: &Config, args: &Value) -> Result<Value> {
let required = required_str(args, "param_name")?;
ok_response(json! { "result": "..." })
}
```
4. **Dispatch** — Add case in `handlers/mod.rs` match arm under `ToolHandler::handle()`.
### Handler Helper Functions (use instead of inline logic)
| Helper | Purpose |
|--------|---------|
| `required_str(args, key)` | Extract required `&str` or return error |
| `ok_response(data)` / `err_response(msg)` | Wrap in MCP content envelope |
| `resolve_report(args)` | → `(deals, metrics, report_dir)` — `report_id > report_dir > latest` |
| `prepare_analysis(args)` | → `(deals, metrics, analyzer, report_dir)` — wraps resolve_report |
| `scan_mql_dir(dir, filter, type_label)` | WalkDir with optional filter for list/search |
| `copy_mql_to_project(config, args, fallback)` | Shared copy logic for indicator/script |
Extend helpers when adding new handlers.
## Commit Style
Conventional commits: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `release:`
Keep commits focused — one logical change per commit.
## Release Process
```bash
bash scripts/release.sh <patch|minor|major|X.Y.Z> [--yes]
```
Script handles:
1. Bump `Cargo.toml` + `Cargo.lock` version
2. Update `server.json` (version + SHA256 placeholder)
3. Generate CHANGELOG entry from git log
4. `cargo check --quiet`
5. Commit `release: vX.Y.Z`
6. Tag `vX.Y.Z`
7. Push — CI (`release.yml`) triggered:
- Build macOS arm64 + Linux x64 binaries
- Publish to crates.io
- Create GitHub Release with artifacts
- Build MCP package, compute SHA256
- Update `server.json` SHA256, push
- Publish to MCP Registry
## Deal Storage
- Deals stored in `deals` SQLite table, keyed by `report_id`. **No `deals.csv` written automatically.**
- `db.insert_deals(report_id, &[Deal])` called in pipeline after extraction.
- `db.get_deals(report_id)` loads deals for analytics. `db.get_by_report_dir(path)` resolves by filesystem path.
- `export_deals_csv` tool writes CSV on demand.
## DB Schema Evolution
Add new tables/columns in `ReportDb::init()` using `CREATE TABLE IF NOT EXISTS` / `CREATE INDEX IF NOT EXISTS` — runs on every startup, is idempotent. No migration files needed.
## Wine / MT5 Notes
- Wine executable paths differ across platforms:
- macOS MT5.app: `/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64`
- macOS CrossOver: `/Applications/CrossOver.app/Contents/SharedSupport/CrossOver/bin/wine64`
- Linux: `/usr/bin/wine64` (or first `wine64` on PATH)
- MT5 terminal_dir is auto-resolved from wine_executable path.
- Apple Silicon: append `arch -x86_64` prefix for x86 Wine binaries (auto-detected).
## Common Pitfalls
- **M1/Memory issues**: Wine on Apple Silicon runs x86 via Rosetta 2. Watch memory pressure.
- **Model mismatch**: Grid/martingale EAs need `model=0` (every tick). Model 1/2 will produce zero deals.
- **Set file encoding**: Always use UTF-16LE with `chmod 444`. UTF-8 strips `||Y` flags.
- **OptMode=-1**: Must clear after optimization. Handled in pipeline cleanup stage.
- **Report not found**: Check `<terminal_dir>/history/` for correct symbol name (broker suffix matters: `XAUUSDc` vs `XAUUSD`).
- **Journal-only backtest**: `ShutdownTerminal=1` doesn't exit MT5 on Wine. Inactivity watchdog (in `launch_backtest`) handles this with `inactivity_kill_secs` param.
-86
View File
@@ -1,91 +1,5 @@
# Changelog
## [1.34.0] — 2026-07-02
- docs: update changelog and tool documentation for rolling backtest
- feat: add rolling backtest and optimization improvements
- fix: sync Cargo.lock, remove unused Agents config from optimizer
- fix: sync Cargo.lock version, remove unused agents config from optimizer
## [1.33.0] — 2026-06-25
### Added
- **`src/utils/` module**: shared `read_file_as_utf8` utility for UTF-16LE BOM detection, used across set file and config handlers.
- **OS detection and diagnostics**: `utility.rs` gains `OsType` enum, `get_os_context`, `get_system_memory`, and `crossover_server_running` helper for better CrossOver diagnostics.
- **Wine binary detection**: MT5.app and CrossOver now detect both `wine` and `wine64` binaries; Homebrew and Linux paths add `wine` alongside `wine64`.
### Changed
- **Optimizer launch mechanism**: rewritten from `/mt5mcp_backtest.ini` + `.bat` to `/config:` INI + shell script. Now patches `terminal.ini` directly with full optimization parameters and appends agent configuration — no longer relies on batch file or separate INI.
- **Removed unused local agent config**: stripped the `[Agents]` section (4 entries with UUIDs/IPs/ports) from `terminal.ini` output — no local agent processes are used.
- **OptMode reset**: removed as a separate step; now included inline in `terminal.ini` patching during optimizer startup.
- **Wine prefix resolution**: changed from 2-parent to 3-parent traversal, matching the backtest pipeline's Wine prefix logic.
- **Optimizer `/config:` INI**: now includes `Model=4` and `Period=M1` in the test-configuration passed via `/config:` flag (previously hardcoded in the older INI-based approach).
- **Set file parsing**: parameter delimiter changed from `:` to `=` (matches actual MT5 .set format); `||Y` sweep param parsing rewritten to split on `||` for accurate range extraction.
- **OptimizationParams**: `model` parameter removed (was hardcoded to `0` everywhere anyway; MT5 now defaults correctly).
### Fixed
- **Cargo.lock sync**: lockfile version field is now kept in sync with the release process.
- **Set file cross-platform reading**: `.set` files are now read as UTF-16LE (with BOM) or UTF-8 regardless of platform, fixing potential encoding issues on macOS/Linux.
- **Read-only set file overwrite**: handles pre-existing read-only files during `.set` write by removing them first.
## [1.32.4] — 2026-04-25
### Fixed
- **HTML report extraction**: `ShutdownTerminal=1` does not cause `terminal64.exe` to exit on Wine/macOS.
Inactivity watchdog now waits 30 s for the report file after the tester log goes quiet, then kills
MT5 unconditionally — no longer depends on natural process exit.
- **Duplicate deals**: tester agent log writes each deal twice; deduplicated via `HashSet<deal_number>`
in `parse_journal_deals`.
- **Entry direction inference**: all journal deals were marked `entry="in"` because no direction info
exists in the log. Fixed with a per-symbol position tracker (signed lot accumulation) to infer
"in" / "out" correctly.
- **`list_deals` returning 0 results**: `is_closed_trade()` was gating on `profit != 0.0`; journal
deals legitimately have zero profit. Removed the profit gate.
- **Wrong tester log selected**: `Agent-0.0.0.0` logs only contain startup lines, not deal lines.
`find_active_tester_agent_log` now prefers `Agent-127.0.0.1` and tiebreaks by file size.
### Added
- `launch_backtest` exposes `shutdown` (default `true`) and `inactivity_kill_secs` (default: disabled;
set to e.g. `120` to enable the inactivity watchdog) as explicit tool parameters.
- Report DB stores deals in SQLite; all analytics tools resolve by `report_id` / `report_dir` / latest.
### Verified
- 1-month DPS21/XAUUSD.cent/M5 backtest produces full HTML report: win_rate=70%,
profit_factor=0.77, sharpe=-3.61, max_dd=6.93%. All 17 analytics tools return real data.
## [1.31.5] — 2026-04-22
- feat: add check_update and update tools with background auto-check
- release: v1.31.4
- fix: update all scripts for correctness and consistency
- release: v1.31.3
- docs: clean up public repo — remove IDE files, fix stale refs
- release: v1.31.2
- refactor: reduce handler boilerplate in analysis and experts modules
- fix: registryType mcpbPackageType → mcpb
## [1.31.4] — 2026-04-22
- fix: update all scripts for correctness and consistency
- release: v1.31.3
- docs: clean up public repo — remove IDE files, fix stale refs
- release: v1.31.2
- refactor: reduce handler boilerplate in analysis and experts modules
- fix: registryType mcpbPackageType → mcpb
## [1.31.3] — 2026-04-22
- docs: clean up public repo — remove IDE files, fix stale refs
- release: v1.31.2
- refactor: reduce handler boilerplate in analysis and experts modules
- fix: registryType mcpbPackageType → mcpb
## [1.31.2] — 2026-04-22
- refactor: reduce handler boilerplate in analysis and experts modules
Generated
+1 -1
View File
@@ -481,7 +481,7 @@ dependencies = [
[[package]]
name = "mt5-quant"
version = "1.34.0"
version = "1.31.2"
dependencies = [
"anyhow",
"base64",
+2 -12
View File
@@ -1,19 +1,9 @@
[package]
name = "mt5-quant"
version = "1.34.0"
version = "1.31.2"
edition = "2021"
description = "MCP server for MT5 strategy development on macOS/Linux"
description = "MT5-Quant MCP Server - Exposes MT5 backtest and optimization tools via MCP"
authors = ["masdevid <masdevid@example.com>"]
license = "MIT"
repository = "https://github.com/masdevid/mt5-quant"
readme = "README.md"
keywords = ["mt5", "mql5", "trading", "mcp", "backtest"]
categories = ["finance", "development-tools"]
homepage = "https://github.com/masdevid/mt5-quant"
documentation = "https://github.com/masdevid/mt5-quant"
[package.metadata]
maintenance = { status = "actively-developed" }
[[bin]]
name = "mt5-quant"
+61 -77
View File
@@ -1,6 +1,6 @@
# MT5-Quant
**MCP server for MT5 strategy development on macOS/Linux.** 89 tools to compile, backtest, analyze, optimize, debug crashes, and manage MQL5 Expert Advisors — no Windows required.
**MCP server for MT5 strategy development on macOS/Linux.** 87 tools to compile, backtest, analyze, optimize, debug crashes, and manage MQL5 Expert Advisors — no Windows required.
```
You: "Backtest MyEA Jan-Mar, what caused the February drawdown?"
@@ -13,47 +13,66 @@ Claude: [compile → clean → backtest → analyze 1,847 deals]
## Why MT5-Quant
**Focus:** Backtest organization, reporting, and analytics — capabilities MT5 itself doesn't provide.
| | MT5-Quant | Other MT5 MCPs | QuantConnect |
|---|---|---|---|
| Backtest pipeline | ✅ Full | ❌ | Cloud only |
| Deal-level analytics | ✅ 15+ dims | ❌ | ❌ |
| MQL5 compilation | ✅ | ❌ | ❌ |
| macOS/Linux native | ✅ | Windows only | Cloud |
| Optimization | ✅ Background | ❌ | ✅ Paid |
| Crash debugging | ✅ Wine/MT5 diagnostics | ❌ | ❌ |
| | MT5-Quant | Others |
|---|---|---|
| **Platform** | macOS/Linux native | Windows only |
| **Backtest pipeline** | ✅ Full (compile → run → analyze) | ✅ Via MT5 Python package |
| **Deal-level analytics** | ✅ 19 dimensions, DB-backed | ❌ |
| **Report organization** | ✅ SQLite (reports + deals) + search + history | ❌ |
| **MQL5 compilation** | ✅ Headless (MetaEditor via Wine, no GUI) | ⚠️ Via GUI or terminal |
| **Optimization** | ✅ Background + results parsing + .set generation | ⚠️ Terminal only, no parsing |
| **Crash debugging** | ✅ Wine/MT5 diagnostics | ❌ |
## Quick Install
**Others** typically run on Windows using the [MetaTrader5 Python package](https://pypi.org/project/MetaTrader5/), providing full terminal operations. MT5-Quant fills the gap: **organizing backtest reports, extracting deal-level insights, and managing optimization workflows** — none of which MT5 or its Python API expose natively.
### 1. Download & Setup
```bash
curl -L -o mt5.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz
tar -xzf mt5.tar.gz
bash scripts/setup.sh
```
### 2. Register MCP Server
| Platform | Command / Config | Docs |
|----------|------------------|------|
| **Claude Code** | `claude mcp add mt5-quant -- $(pwd)/mt5-quant` | [Setup →](docs/QUICKSTART.md) |
| **Windsurf** | Edit `~/.codeium/windsurf/mcp_config.json` | [WINDSURF.md →](docs/WINDSURF.md) |
| **Cursor** | Edit `~/.cursor/mcp.json` or use Settings → MCP | [CURSOR.md →](docs/CURSOR.md) |
| **VS Code** | Edit `.vscode/mcp.json` or run `MCP: Add Server` | [VSCODE.md →](docs/VSCODE.md) |
| **Antigravity** | Agent Panel → ... → MCP Servers → Edit configuration | [ANTIGRAVITY.md →](docs/ANTIGRAVITY.md) |
> **Note:** Use absolute paths like `/Users/name/mt5-quant/mt5-quant` or `$(pwd)/mt5-quant`, not relative paths like `./mt5-quant`.
## Quick Start
### Install MT5-Quant
Ask your LLM coding platform to install and configure MT5-Quant:
> "Please install mt5-quant. It's a Rust-based MCP server for MT5 backtesting at https://github.com/masdevid/mt5-quant. Run its `scripts/setup.sh` to auto-detect Wine and MT5 paths, register the MCP server, and configure everything."
The LLM will:
1. Download the pre-built binary (or `cargo build --release`)
2. Run `scripts/setup.sh` to auto-detect Wine/MT5 and write config
3. Register the MCP server with your coding platform
### First Backtest
```
Run a backtest on MyEA from 2025.01.01 to 2025.03.31
```
The AI runs the full pipeline: compile → clean cache → backtest → extract → analyze.
## MCP Tools (89)
## Documentation
| Doc | Purpose |
|-----|---------|
| [QUICKSTART.md](docs/QUICKSTART.md) | Complete setup for macOS/Linux |
| [WINDSURF.md](docs/WINDSURF.md) | Windsurf IDE setup |
| [CURSOR.md](docs/CURSOR.md) | Cursor IDE setup |
| [VSCODE.md](docs/VSCODE.md) | VS Code setup |
| [ANTIGRAVITY.md](docs/ANTIGRAVITY.md) | Antigravity IDE setup |
| [CONFIG.md](docs/CONFIG.md) | Configuration reference |
| [TOOLS.md](docs/MCP_TOOLS.md) | All 87 tools documented |
| [ARCHITECTURE.md](docs/ARCHITECTURE.md) | Design and internals |
| [TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) | Common issues |
| [REMOTE_AGENTS.md](docs/REMOTE_AGENTS.md) | Linux optimization agents |
## MCP Tools (87)
### Core workflow
| Tool | Description |
|------|-----|
|------|-------------|
| `run_backtest` | Full pipeline: compile → clean → backtest → extract → analyze |
| `run_backtest_quick` | Quick backtest using pre-compiled EA (skip compile) |
| `run_backtest_only` | Backtest only - just extract raw trades, no analysis |
@@ -61,7 +80,6 @@ The AI runs the full pipeline: compile → clean cache → backtest → extract
| `get_backtest_status` | Poll running backtest status (MT5 running, report found, elapsed time) |
| `run_optimization` | Genetic optimization (background, returns immediately) |
| `get_optimization_results` | Parse optimization results after MT5 finishes |
| `list_jobs` | List all optimization jobs with status |
| `analyze_report` | Read `analysis.json` from any report directory |
| `compare_baseline` | Compare report vs baseline, return winner/loser verdict |
| `compile_ea` | Compile MQL5 EA via MetaEditor |
@@ -69,12 +87,11 @@ The AI runs the full pipeline: compile → clean cache → backtest → extract
| `list_indicators` | List all indicators in MQL5/Indicators directory |
| `list_scripts` | List all scripts in MQL5/Scripts directory |
| `healthcheck` | Quick server health check |
| `list_symbols` | List all available symbols in MT5 terminal |
### Granular Analytics (individual analysis)
| Tool | Description |
|------|-----|
|------|-------------|
| `analyze_monthly_pnl` | Monthly P/L breakdown only |
| `analyze_drawdown_events` | Drawdown events and causes only |
| `analyze_top_losses` | Worst losing deals only |
@@ -89,7 +106,7 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### Deal-Level Analytics (New)
| Tool | Description |
|------|-----|
|------|-------------|
| `list_deals` | List individual deals with filters (type, profit range, volume, dates) |
| `search_deals_by_comment` | Full-text search in deal comments (e.g., "Layer #3") |
| `search_deals_by_magic` | Filter deals by EA magic number |
@@ -104,7 +121,7 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### Monitoring
| Tool | Description |
|------|-----|
|------|-------------|
| `verify_setup` | Check Wine/MT5 paths, Wine version, and EA/set file counts |
| `get_optimization_status` | Check live state of a background optimization job |
| `list_jobs` | All optimization jobs with compact status in one call |
@@ -112,7 +129,7 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### Reports & logs
| Tool | Description |
|------|-----|
|------|-------------|
| `list_reports` | Compact table of all runs with key metrics — no full analysis needed |
| `get_latest_report` | Get most recent report with optional equity chart |
| `search_reports` | Find reports by EA, symbol, date range, or profit criteria |
@@ -126,28 +143,28 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
| `get_comparable_reports` | Find comparable reports (same EA/symbol/timeframe) |
| `tail_log` | Read last N lines of any log; `filter=errors` to see only failures |
| `prune_reports` | Delete old report directories, keep last N (skips `_opt` dirs) |
| `promote_to_baseline` | Write a history entry or report to `baseline.json` for compare_baseline |
### History & baseline
| Tool | Description |
|------|-----|
|------|-------------|
| `archive_report` | Convert one report dir → compact JSON entry in `backtest_history.json`, optionally delete source |
| `archive_all_reports` | Bulk-archive all report dirs then optionally delete them; keeps N newest safe |
| `get_history` | Query history with filters (EA, symbol, verdict, profit, DD) and sort options |
| `annotate_history` | Attach verdict / notes / tags to any history entry |
| `promote_to_baseline` | Write a history entry or report to `baseline.json` for compare_baseline |
### Cache management
| Tool | Description |
|------|-----|
|------|-------------|
| `cache_status` | MT5 tester cache size breakdown by symbol — check before cleaning |
| `clean_cache` | Delete tester cache files; supports per-symbol and `dry_run` |
### Pre-flight & Validation
| Tool | Description |
|------|-----|
|------|-------------|
| `get_active_account` | Get current MT5 account session (login, server, available symbols) |
| `check_symbol_data_status` | Validate symbol has sufficient history data for date range |
| `check_mt5_status` | Check if MT5 terminal is installed and ready |
@@ -156,7 +173,7 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### Debugging & Diagnostics (New)
| Tool | Description |
|------|-----|
|------|-------------|
| `diagnose_wine` | Check Wine installation, version, and prefix health |
| `get_mt5_logs` | Get MT5 terminal, tester, or MetaEditor logs with filtering |
| `search_mt5_errors` | Search logs for error patterns (crash, exception, access violation) |
@@ -166,13 +183,11 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
| `validate_mt5_config` | Validate terminal.ini and tester configuration files |
| `get_wine_prefix_info` | Get Wine prefix details: Windows version, installed programs, registry |
| `get_backtest_crash_info` | Investigate backtest failures: incomplete markers, missing deals.csv, errors |
| `check_update` | Check if a newer version of MT5-Quant is available |
| `update` | Update MT5-Quant to latest release |
### Project Management
| Tool | Description |
|------|-----|
|------|-------------|
| `init_project` | Scaffold new MQL5 project with templates (scalper/swing/grid/basic) |
| `create_set_template` | Generate .set parameter file from EA input variables |
| `export_report` | Export backtest report to CSV, JSON, or Markdown |
@@ -180,14 +195,14 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### History & Comparison
| Tool | Description |
|------|-----|
|------|-------------|
| `get_backtest_history` | List all backtests for EA/symbol with summary metrics |
| `compare_backtests` | Compare 2+ backtest results side-by-side with analysis |
### .set file — read / write
| Tool | Description |
|------|-----|
|------|-------------|
| `list_set_files` | All .set files in tester profiles dir with sweep stats and combination counts |
| `read_set_file` | Parse UTF-16LE `.set` file → structured JSON params |
| `write_set_file` | Write full params dict → UTF-16LE `.set` with `chmod 444` |
@@ -197,7 +212,7 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### .set file — analysis & generation
| Tool | Description |
|------|-----|
|------|-------------|
| `describe_sweep` | Swept params, value counts, and total optimization combinations |
| `diff_set_files` | Side-by-side diff of two `.set` files — only changed params returned |
| `set_from_optimization` | Generate a clean backtest `.set` from `get_optimization_results` params; optionally narrow sweep |
@@ -205,7 +220,7 @@ Use these for targeted analysis, or `analyze_report` to run all at once.
### Search & Discovery
| Tool | Description |
|------|-----|
|------|-------------|
| `search_experts` | Search EAs by name pattern across all directories |
| `search_indicators` | Search indicators by name pattern |
| `search_scripts` | Search scripts by name pattern |
@@ -216,7 +231,7 @@ Full schema: [docs/MCP_TOOLS.md](docs/MCP_TOOLS.md)
## Troubleshooting
Run `verify_setup` from your LLM first — it checks all paths and returns actionable hints.
Run `verify_setup` from Claude first — it checks all paths and returns actionable hints.
For crashes or unexplained failures during backtest/compile/optimization:
- `diagnose_wine` — Check Wine installation and prefix health
@@ -228,37 +243,6 @@ For crashes or unexplained failures during backtest/compile/optimization:
---
## Acknowledgements
MT5-Quant stands on the shoulders of exceptional open-source projects:
- **[Rust](https://www.rust-lang.org/)** — The language that makes zero-cost abstractions, memory safety, and fearless concurrency practical
- **[Tokio](https://tokio.rs/)** — The async runtime powering all concurrent operations
- **[Wine](https://www.winehq.org/)** — Making MT5 execution possible on macOS and Linux without Windows licensing
- **[MetaTrader 5](https://www.metatrader5.com/)** — MetaQuotes' trading platform (trademark of MetaQuotes Software Corp.)
- **[rusqlite](https://github.com/rusqlite/rusqlite)** — Ergonomic SQLite bindings for Rust
- **[serde](https://serde.rs/)** — The serialization framework making config and report handling painless
- **[scraper](https://github.com/causal-agent/scraper)** — HTML parsing for MT5 report extraction
- **[tempfile](https://github.com/Stebalien/temp-file)** — Secure temporary file handling
Special thanks to the Model Context Protocol (MCP) team at Anthropic for defining the standard that makes AI-powered development workflows possible.
## Disclaimer
**Not Financial Advice.** MT5-Quant is a development and analysis tool for algorithmic trading strategies. It does not provide investment advice, trading recommendations, or guarantee profitability. All backtest results are historical simulations and do not guarantee future performance.
**Use at Your Own Risk.** Trading financial instruments carries substantial risk of loss. The authors and contributors of MT5-Quant accept no liability for:
- Trading losses incurred using strategies developed or tested with this tool
- Data loss or corruption from backtest operations
- Bugs, errors, or incorrect analysis results
- System crashes, Wine compatibility issues, or MT5 failures
**Software Warranty.** This software is provided "as-is" without warranty of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, or non-infringement. See LICENSE for full terms.
**Third-Party Software.** MT5-Quant interacts with MetaTrader 5, Wine, and other third-party software. Users are responsible for complying with all applicable licenses and terms of service for these dependencies. MetaTrader is a trademark of MetaQuotes Software Corp. MT5-Quant is not affiliated with, endorsed by, or sponsored by MetaQuotes.
**Regulatory Compliance.** Users are responsible for ensuring their trading activities comply with applicable financial regulations in their jurisdiction. Automated trading may be restricted or require licensing in some regions.
## License
MIT
+145
View File
@@ -0,0 +1,145 @@
# Antigravity MCP Integration Setup
## Quick Setup
### Option 1: Download Prebuilt Binary (Recommended)
```bash
# macOS (Apple Silicon)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz
tar -xzf mt5-quant.tar.gz
# Linux (x64)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz
tar -xzf mt5-quant.tar.gz
```
### Option 2: Build from Source
```bash
cargo build --release
```
## Configure Antigravity
### Step 1: Open MCP Manager
1. Launch Antigravity
2. Look at the **right-side Agent Panel**
3. Click the **"..."** (More Options) menu at the top
4. Select **MCP Servers**
### Step 2: Access Configuration
1. Click **Manage MCP Servers**
2. Click **View raw config** or **Edit configuration**
3. This opens `mcp_config.json`
### Step 3: Add mt5-quant Configuration
Add to `mcp_config.json`:
```json
{
"mcpServers": {
"mt5-quant": {
"command": "/absolute/path/to/mt5-quant"
}
}
}
```
### Step 4: Reload and Verify
1. Save the file
2. **Restart Antigravity** (or reload the window)
3. Open the Agent chat and type: `What tools do you have access to?`
4. The agent should list MT5 tools like `verify_setup`, `run_backtest`, etc.
## Full Example Configuration
```json
{
"mcpServers": {
"mt5-quant": {
"command": "/Users/name/mt5-quant/target/release/mt5-quant"
},
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "${env:GITHUB_TOKEN}"
}
}
}
}
```
## Environment Variables
Antigravity supports `${VAR_NAME}` syntax for environment variable substitution:
```json
{
"mcpServers": {
"mt5-quant": {
"command": "/Users/name/mt5-quant/target/release/mt5-quant",
"env": {
"CUSTOM_VAR": "${env:MY_VAR}"
}
}
}
}
```
This keeps secrets out of the config file.
## Verify Setup
In Antigravity chat, type:
```
Run verify_setup
```
Expected output:
```
Wine: /Applications/MetaTrader 5.app/.../wine64
MT5 dir: ~/Library/Application Support/.../MetaTrader 5
Display: gui
Arch: arch -x86_64
```
## Troubleshooting
### "Connection Refused" or "Tool not found"
1. Double-check the server path in `mcp_config.json`
2. Ensure the binary has execute permissions: `chmod +x /path/to/mt5-quant`
3. Try completely restarting Antigravity
4. Check the server is listed in MCP manager
### "Stdio Error" or JSON Parsing Error
1. Verify the JSON syntax in `mcp_config.json`
2. Use a JSON validator if needed
3. Ensure no trailing commas
### Agent Hallucinating Tool Parameters
If the agent makes up incorrect parameters:
1. Check the tool is actually available: "List your available tools"
2. Restart the agent session
3. Be explicit in your requests
## Configuration Location
| Platform | Path |
|----------|------|
| All | Via UI: Agent Panel → ... → MCP Servers → Manage → Edit configuration |
## Resources
- [Antigravity Documentation](https://docs.antigravity.dev/)
- [MCP Server Reference](https://github.com/modelcontextprotocol/servers)
- [MT5-Quant Tools Reference](./MCP_TOOLS.md)
+67 -37
View File
@@ -1,12 +1,14 @@
# Architecture
# Architecture Deep Dive
## Design Principles
## Design Philosophy
**Deal-level over aggregate.** MT5's HTML report gives you: profit, profit factor, max DD%, trade count. MT5-Quant extracts every individual deal — entry price, exit price, P/L, comment string — to reconstruct what happened during each loss event. Result: `analysis.json`, AI-readable and diffable between runs.
**Deal-level over aggregate.** MT5's built-in HTML report gives you: profit, profit factor, max DD%, trade count. That's it. You cannot tell from those numbers whether a drawdown was caused by overleveraged locking, a bad entry during a news spike, or a grid that reached L8 in a trending market.
**Pipeline idempotency.** MT5 caches aggressively (`.ex5` binaries, `.set` files, `terminal.ini` flags). The pipeline invalidates all cache before every run to prevent stale results.
MT5-Quant extracts every individual deal — entry price, exit price, P/L, comment string — and reconstructs what was happening when each loss event occurred. The `analysis.json` artifact is the result: AI-readable, stable schema, diffable between runs.
**Background isolation.** Genetic optimizations run 2-6 hours. The `nohup + disown` pattern prevents corruption if the parent process (SSH, Claude runner) is killed.
**Pipeline idempotency.** MT5 caches aggressively. Cached `.ex5` binaries, cached `.set` files, stale `terminal.ini` flags. The pipeline exists specifically to invalidate all of these before every run. A backtest result that came from a cached EA binary is wrong in a way that's impossible to detect without the cache clear step.
**Background isolation for optimization.** Genetic optimizations run 2-6 hours. Running them inside a parent process that can be killed (Claude task runner, SSH session, terminal) corrupts the MT5 optimization state. The only correct pattern is `nohup + disown` — fully detached from all parent process trees.
---
@@ -23,26 +25,24 @@ MT5-Quant/
│ │ ├── metrics.rs # Metrics parsing from HTML/XML
│ │ └── report.rs # Report, PipelineMetadata, etc.
│ ├── analytics/ # Report extraction & analysis (migrated from Python)
│ │ ├── extract.rs # HTML/XML report parser → metrics.json (deals → DB)
│ │ ├── extract.rs # HTML/XML report parser → metrics.json + deals.csv
│ │ └── analyze.rs # Deal-level analysis engine → analysis.json
│ ├── compile/ # MQL5 compilation
│ │ └── mql_compiler.rs # MetaEditor wrapper (Wine/CrossOver)
│ ├── pipeline/ # Backtest orchestration
│ │ ├── backtest.rs # 5-stage pipeline (COMPILE→CLEAN→BACKTEST→EXTRACT→ANALYZE)
│ │ └── stages.rs # Pipeline stage definitions
│ ├── storage/ # SQLite persistence
│ │ └── database.rs # ReportDb: reports table + deals table
│ └── tools/ # MCP tool definitions
│ ├── definitions/ # Tool schemas (9 domain modules, 90 tools)
│ ├── definitions/ # Tool schemas (9 domain modules, 43 tools)
│ │ ├── mod.rs
│ │ ├── analytics.rs # 19 analysis tools (DB-backed)
│ │ ├── backtest.rs # 7 backtest tools
│ │ ├── analytics.rs # 9 analysis tools
│ │ ├── backtest.rs # 4 backtest tools
│ │ ├── baseline.rs # 1 baseline tool
│ │ ├── experts.rs # 9 EA/indicator/script tools
│ │ ├── experts.rs # 4 EA/indicator/script tools
│ │ ├── optimization.rs # 4 optimization tools
│ │ ├── reports.rs # 20 report management tools
│ │ ├── reports.rs # 11 report management tools
│ │ ├── setfiles.rs # 8 .set file tools
│ │ └── system.rs # 6 system tools
│ │ └── system.rs # 3 system tools
│ └── handlers/ # Tool dispatch (9 domain modules)
│ ├── mod.rs
│ ├── analysis.rs
@@ -59,6 +59,11 @@ MT5-Quant/
│ ├── build-rust.sh # Rust build script
│ └── optimize.sh # Genetic optimization launcher (nohup + disown)
├── analytics/ # Legacy Python (reference only)
│ ├── extract.py
│ ├── analyze.py
│ └── optimize_parser.py
├── config/
│ ├── mt5-quant.example.yaml # Template config
│ └── mt5-quant.yaml # Live config (gitignored)
@@ -147,26 +152,17 @@ MT5 runs in headless mode, writes the report, and exits.
---
### Stage 4: EXTRACT + STORE
### Stage 4: EXTRACT
Single HTML/XML parse pass. Deals go directly into the SQLite database; the raw report file is deleted afterwards.
Single HTML/XML parse pass that produces three artifacts:
```rust
// src/analytics/extract.rs
let extractor = ReportExtractor::new();
let result = extractor.extract(&report_path, &output_dir)?;
// → metrics.json (aggregate summary — written to report_dir)
// HTML report deleted after extraction
// src/storage/database.rs
db.insert_deals(&report_id, &result.deals)?;
// → deals table in SQLite (all deals, keyed by report_id)
```
On-demand CSV export is available via the `export_deals_csv` tool:
```
export_deals_csv(report_id: "20260422_051041_DPS21_XAUUSDc_M5_1")
// → report_dir/deals.csv (written only when explicitly requested)
// → metrics.json (aggregate summary)
// → deals.csv (all deals, 13 columns)
// → deals.json (same data, JSON)
```
**Why single-pass?** MT5 HTML reports are large (1-5MB for 14-month tests). Each regex pass over the file takes ~200ms. The old pipeline ran 5 separate grep/regex passes. The Rust implementation uses a single-pass parser: 5× faster and no partial-read inconsistencies.
@@ -186,13 +182,12 @@ if ext == "xml" || path.ends_with(".htm.xml") {
}
```
**Deal columns (stored in DB):**
**Deal columns (13):**
```
time | deal | symbol | deal_type | entry | volume | price | order_id
commission | swap | profit | balance | comment | magic
Time | Type | Direction | Volume | Price | S/L | T/P | Profit | Balance | Comment | Order | Magic | Entry
```
The `comment` column is the key to grid analytics. The EA writes `"Layer #3"`, `"Locking Total"`, `"Zombie Exit"` etc. Pattern matching on comments reconstructs which position was at which layer.
The `Comment` column is the key to grid analytics. The EA writes `"Layer #3"`, `"Locking Total"`, `"Zombie Exit"` etc. Pattern matching on comments reconstructs which position was at which layer.
---
@@ -236,6 +231,15 @@ Built-in profiles:
| `trend` | — | `magic` | 240 | breakeven, trailing, partial, tp, sl |
| `hedge` | — | `magic+direction` | 120 | tp, sl, net_close, partial |
#### Entry points (after `pip install -e .`)
```bash
mt5-analyze deals.csv # generic
mt5-analyze-grid deals.csv # grid / martingale (default in pipeline)
mt5-analyze-scalper deals.csv # scalper
mt5-analyze-trend deals.csv # trend following
mt5-analyze-hedge deals.csv # hedging
```
#### Analytics functions
@@ -412,7 +416,7 @@ Then set `DISPLAY=:99` in MT5-Quant's environment config.
**Comment-based analytics:**
- Strategy-specific analytics (depth histogram, exit reason, DD cause) depend on EA comment strings. EAs that don't write structured comments will get `generic` profile results — summary metrics, session breakdown, streaks, and direction bias all still work; only keyword-classified fields fall back to `"unknown"` or profit-sign.
- Custom comment patterns can be supported by adding a new entry to `PROFILES` in `src/analytics/analyze.rs`.
- Custom comment patterns can be supported by adding a new entry to `PROFILES` in `analytics/analyze.py` — no other code changes needed.
**Single MT5 instance:**
- MT5 is single-instance per Windows drive. Two backtests cannot run simultaneously on the same Wine prefix. Parallelism requires multiple Wine prefixes (separate installations).
@@ -421,8 +425,34 @@ Then set `DISPLAY=:99` in MT5-Quant's environment config.
## Claude Code Integration
`setup.sh` generates `config/CLAUDE.template.md` — a project-level template encoding:
`setup.sh --claude-code` generates two files that give Claude persistent context about the user's trading setup:
### `config/CLAUDE.template.md`
A project-level CLAUDE.md template the user copies to their EA project root. Encodes:
- MT5-Quant tool names and when to use them
- Baseline tracking policy (compare to `baseline.json` before calling improvements)
- Symbol name reminders (`XAUUSD.cent``XAUUSD`)
- Backtest constraints (model 0, UTF-16LE .set files)
- Baseline tracking policy (never call something an improvement without comparing to `baseline.json`)
- Symbol name reminder (broker-specific suffix matters — `XAUUSD.cent``XAUUSD`)
- Backtest and optimization constraints (model 0, single instance, UTF-16LE .set files)
### `.claude/hooks/user-prompt-submit.sh`
A Claude Code hook that runs before every prompt submission. Reads `config/baseline.json` and outputs a JSON context block:
```json
{"context": "## Production Baseline (config/baseline.json)\n..."}
```
Claude Code injects this into the system context for every conversation turn. The result: Claude always knows the current production metrics without the user having to paste them.
**Hook execution path:**
```
User types prompt
→ user-prompt-submit.sh executes
→ reads config/baseline.json
→ outputs {"context": "..."} to stdout
→ Claude Code prepends to system context
→ Claude sees baseline in every prompt
```
**Graceful degradation:** If `baseline.json` doesn't exist or is malformed, the hook exits 0 silently — no prompt is blocked. The baseline section simply doesn't appear until the user creates the file.
+51 -4
View File
@@ -1,15 +1,18 @@
# Configuration Reference
## Config File
## Config File Location
```
config/mt5-quant.yaml # Project directory (development)
~/.config/mt5-quant/config/mt5-quant.yaml # System-wide (production)
```
Override with env: `export MT5_MCP_HOME=/path/to/config`
Environment variable to override:
```bash
export MT5_MCP_HOME=/path/to/config
```
## Full Example
## Full Config Example
```yaml
# Required: Wine executable path
@@ -49,9 +52,53 @@ optimization:
min_agents: 4
```
## Platform-Specific Examples
### macOS with MetaTrader 5.app
```yaml
wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64"
terminal_dir: "~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5"
defaults:
symbol: "XAUUSDc"
timeframe: "M5"
deposit: 10000
```
### macOS with CrossOver
```yaml
wine_executable: "/Applications/CrossOver.app/Contents/SharedSupport/CrossOver/bin/wine64"
terminal_dir: "~/Library/Application Support/MetaQuotes/Terminal/<hash>/drive_c/Program Files/MetaTrader 5"
```
### Linux
```yaml
wine_executable: "/usr/bin/wine64"
terminal_dir: "~/.wine/drive_c/Program Files/MetaTrader 5"
display:
mode: headless
xvfb_display: ":99"
```
## Headless Mode (Linux VPS)
```yaml
display:
mode: headless
xvfb_display: ":99"
xvfb_screen: "1024x768x16"
```
Requires:
```bash
sudo apt install xvfb
```
## Auto-Detection
`setup.sh` detects:
`setup.sh` automatically detects:
- Wine executable (MetaTrader 5.app, CrossOver, or system Wine)
- MT5 terminal directory
- Architecture (Apple Silicon adds `arch -x86_64`)
+129
View File
@@ -0,0 +1,129 @@
# Cursor MCP Integration Setup
## Quick Setup
### Option 1: Download Prebuilt Binary (Recommended)
```bash
# macOS (Apple Silicon)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz
tar -xzf mt5-quant.tar.gz
# Linux (x64)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz
tar -xzf mt5-quant.tar.gz
```
### Option 2: Build from Source
```bash
cargo build --release
```
## Configure Cursor
### Method 1: Settings UI (Recommended)
1. Open Cursor Settings (`Cmd/Ctrl + ,`)
2. Navigate to **Features****MCP**
3. Click **Add Custom MCP**
4. Enter:
- **Name**: `mt5-quant`
- **Command**: `~/.local/bin/mt5-quant`
- **Type**: `stdio`
### Method 2: Edit mcp.json Directly
Add to `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project):
```json
{
"mcpServers": {
"mt5-quant": {
"type": "stdio",
"command": "~/.local/bin/mt5-quant"
}
}
}
```
Create the file if it doesn't exist:
```bash
mkdir -p ~/.cursor
cat > ~/.cursor/mcp.json << 'EOF'
{
"mcpServers": {
"mt5-quant": {
"type": "stdio",
"command": "/path/to/mt5-quant"
}
}
}
EOF
```
## Verify Setup
In Cursor chat, type:
```
Run verify_setup
```
Expected output:
```
Wine: /Applications/MetaTrader 5.app/.../wine64
MT5 dir: ~/Library/Application Support/.../MetaTrader 5
Display: gui
Arch: arch -x86_64
```
## Configuration Locations
| Scope | Path | Use Case |
|-------|------|----------|
| Global | `~/.cursor/mcp.json` | Available in all projects |
| Project | `.cursor/mcp.json` | Project-specific tools |
## Troubleshooting
### MCP server not appearing
1. Check MCP panel in Cursor Settings
2. Verify the path is absolute (not relative)
3. Test binary: `/path/to/mt5-quant --help`
4. View MCP logs: Output panel → select "MCP" from dropdown
### Config interpolation
Cursor supports variable substitution in `mcp.json`:
```json
{
"mcpServers": {
"mt5-quant": {
"command": "${userHome}/bin/mt5-quant",
"args": ["--config", "${workspaceFolder}/config.yaml"]
}
}
}
```
Available variables:
- `${userHome}` - Home directory
- `${workspaceFolder}` - Project root
- `${workspaceFolderBasename}` - Project folder name
- `${env:VAR_NAME}` - Environment variable
### Tool not found errors
If the agent says "Tool not found":
1. Check the server is enabled in MCP settings
2. Try disabling and re-enabling the server
3. Restart Cursor
## Resources
- [Cursor MCP Documentation](https://cursor.com/docs/context/mcp)
- [MT5-Quant Tools Reference](./MCP_TOOLS.md)
+28 -1917
View File
File diff suppressed because it is too large Load Diff
+172 -23
View File
@@ -1,44 +1,193 @@
# Quickstart Guide
For end-users wanting full platform-specific steps, see the [TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md).
## 1. Download or Build
## Install
### Option A: Prebuilt Binary (Recommended)
Ask your LLM platform to install MT5-Quant:
```bash
# macOS (Apple Silicon)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz
tar -xzf mt5-quant.tar.gz
> "Please install mt5-quant from https://github.com/masdevid/mt5-quant. Download the pre-built binary, run `scripts/setup.sh` to auto-detect Wine/MT5 paths, and register the MCP server."
The LLM will handle:
1. Download binary or `cargo build --release`
2. Run setup script to auto-detect Wine and MT5 paths
3. Write `config/mt5-quant.yaml` (gitignored)
4. Register MCP server with your platform
## Minimal Config
If manual config is needed, `config/mt5-quant.yaml` requires only:
```yaml
wine_executable: "/path/to/wine64"
terminal_dir: "/path/to/MetaTrader 5"
# Linux (x64)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz
tar -xzf mt5-quant.tar.gz
```
`setup.sh` auto-detects both.
### Option B: Build from Source
## Verify
```bash
git clone https://github.com/masdevid/mt5-mcp
cd mt5-mcp
bash scripts/build-rust.sh
```
## 2. Install MetaTrader 5
### macOS - MetaTrader 5.app (Free)
1. Download from [metatrader5.com](https://www.metatrader5.com/en/download)
2. Install to `/Applications`
3. **Launch once** to initialize Wine prefix (~30s), then quit
Auto-detected paths:
- Wine: `/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64`
- MT5: `~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5`
### macOS - CrossOver (Paid, Better Compatibility)
1. Install [CrossOver](https://www.codeweavers.com/)
2. Create bottle `MetaTrader5`
3. Install MT5 inside bottle
Auto-detected paths:
- Wine: `/Applications/CrossOver.app/Contents/SharedSupport/CrossOver/bin/wine64`
- MT5: `~/Library/Application Support/MetaQuotes/<hash>/drive_c/Program Files/MetaTrader 5`
### Linux
```bash
# Debian/Ubuntu
sudo apt install wine64 xvfb
# Fedora/RHEL
sudo dnf install wine xorg-x11-server-Xvfb
# Install MT5
wine64 MetaTrader5Setup.exe
```
MT5 location: `~/.wine/drive_c/Program Files/MetaTrader 5`
## 3. Configure
Run the setup script to auto-detect paths:
```bash
bash scripts/setup.sh # interactive
bash scripts/setup.sh --yes # non-interactive (CI)
```
This creates `config/mt5-quant.yaml` (gitignored).
Minimum config:
```yaml
wine_executable: "/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64"
terminal_dir: "~/Library/Application Support/net.metaquotes.wine.metatrader5/drive_c/Program Files/MetaTrader 5"
```
## 4. Register MCP Server
### Claude Code
```bash
claude mcp add io.github.masdevid/mt5-quant -- ~/.local/bin/mt5-quant
```
Verify:
```bash
claude mcp list
```
### Windsurf
Add to `~/.codeium/windsurf/mcp_config.json`:
```json
{
"mcpServers": {
"io.github.masdevid/mt5-quant": {
"command": "~/.local/bin/mt5-quant",
"disabled": false,
"registry": "io.github.masdevid/mt5-quant"
}
}
}
```
### Cursor
Add to `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project):
```json
{
"mcpServers": {
"mt5-quant": {
"command": "~/.local/bin/mt5-quant"
}
}
}
```
Or use Settings → MCP → Add Custom MCP.
### VS Code
Add to `.vscode/mcp.json` in your workspace:
```json
{
"servers": {
"mt5-quant": {
"command": "/path/to/mt5-quant"
}
}
}
```
Or run `MCP: Add Server` from Command Palette.
### Antigravity
1. Open Agent panel → Click "..." menu → MCP Servers
2. Click "Manage MCP Servers"
3. Click "View raw config" or "Edit configuration"
4. Add to `mcp_config.json`:
```json
{
"mcpServers": {
"mt5-quant": {
"command": "/path/to/mt5-quant"
}
}
}
```
5. Reload Antigravity to apply changes
## 5. Verify Setup
```bash
bash scripts/platform_detect.sh
```
Or in Claude/Windsurf:
```
Run verify_setup
```
## Run Backtest
Expected output:
```
Wine: /Applications/MetaTrader 5.app/.../wine64
MT5 dir: ~/Library/Application Support/.../MetaTrader 5
Display: gui
Arch: arch -x86_64
```
## 6. Run First Backtest
```
Run a backtest on MyEA from 2025.01.01 to 2025.03.31
```
The AI runs: compile → clean → backtest → extract → analyze.
The AI will:
1. Verify setup
2. Compile your EA
3. Clean MT5 cache
4. Run backtest
5. Extract and analyze results
6. Report key findings
---
**Next:** See [TOOLS.md](docs/MCP_TOOLS.md) for all 90 tools.
**Next:** See [TOOLS.md](TOOLS.md) for all 43 available tools.
+95 -14
View File
@@ -11,12 +11,14 @@ MT5's distributed testing lets you farm optimization work across multiple machin
**Mac (master)**
- MetaTrader 5 installed via CrossOver or Wine
- MT5-Quant configured and working for local backtests
- Port 3000 open in macOS firewall (or whichever port MT5 uses)
- Port 3000 open in macOS firewall (or whichever port MT5 uses — check below)
**Linux server (agents)**
- Wine 7.0+
- Wine 7.0+ (or Wine Staging for better compatibility)
- `metatester64.exe` from an MT5 installation
- Access to the same MT5 data files (tick history) as the master
- Access to the same MT5 data files (tick history) as the master — or let agents download on first run
---
## Step 1: Find the MT5 agent port on Mac
@@ -29,6 +31,7 @@ ls ~/Library/Application\ Support/MetaQuotes/*/drive_c/Program\ Files/MetaTrader
You'll see directories like:
```
Agent-127.0.0.1-3000/ ← local agents (loopback)
Agent-127.0.0.1-3001/
Agent-0.0.0.0-3000/ ← remote listener (if enabled)
```
@@ -37,23 +40,69 @@ If you don't see `Agent-0.0.0.0-*` directories, enable remote agents in MT5:
Note the port number (default: 3000).
## Step 2: Copy `metatester64.exe` to Linux
---
## Step 2: Open firewall on Mac
```bash
# Check current firewall rules
sudo pfctl -s rules
# Allow incoming on agent port (example: 3000)
# Add to /etc/pf.conf or use macOS Firewall in System Settings
```
Or use macOS System Settings → Privacy & Security → Firewall → Firewall Options → Add MetaTrader 5.
---
## Step 3: Install Wine on Linux server
```bash
# Ubuntu/Debian
sudo dpkg --add-architecture i386
sudo apt update
sudo apt install wine64 wine32
# Verify
wine64 --version
```
---
## Step 4: Copy `metatester64.exe` to Linux
From your Mac MT5 installation:
```bash
MAC_MT5="$HOME/Library/Application Support/MetaQuotes/Terminal/XXX/ drive_c/Program Files/MetaTrader 5"
MAC_MT5="$HOME/Library/Application Support/MetaQuotes/Terminal/XXXXX/drive_c/Program Files/MetaTrader 5"
scp "${MAC_MT5}/metatester64.exe" user@linux-server:~/mt5agents/
scp "${MAC_MT5}/metaeditor64.exe" user@linux-server:~/mt5agents/ # not required for agents
```
Also copy the required DLLs (they're in the same directory):
```bash
scp "${MAC_MT5}"/*.dll user@linux-server:~/mt5agents/
```
## Step 3: Launch agents on Linux
---
## Step 5: Launch agents on Linux
```bash
# On the Linux server
cd ~/mt5agents/
# Replace MAC_IP with your Mac's local IP address
# Replace 8 with number of CPU cores - 1
wine64 metatester64.exe /server:192.168.1.100:3000 /agents:8
```
**What this does:**
- Connects to your Mac's MT5 master at `192.168.1.100:3000`
- Registers 8 worker agents
- MT5 on Mac will show 8 new agents in the agent manager
**To run as a background service:**
```bash
nohup wine64 metatester64.exe /server:192.168.1.100:3000 /agents:8 \
@@ -61,7 +110,9 @@ nohup wine64 metatester64.exe /server:192.168.1.100:3000 /agents:8 \
disown
```
## Step 4: Verify agents appear in MT5
---
## Step 6: Verify agents appear in MT5
On your Mac, open MT5:
**View → Strategy Tester → Agents tab**
@@ -69,38 +120,68 @@ On your Mac, open MT5:
You should see entries like:
```
Agent-192.168.1.200-3000 [Active]
Agent-192.168.1.200-3001 [Active]
...
```
## Step 5: Configure MT5-Quant for remote agents
If agents appear as `[Inactive]`, check:
1. Mac firewall is allowing incoming connections on the agent port
2. Linux server can reach Mac IP: `ping 192.168.1.100`
3. Port is open: `nc -zv 192.168.1.100 3000`
In `config/mt5-quant.yaml`:
---
## Step 7: Configure MT5-Quant for remote agents
In `config/MT5-Quant.yaml`:
```yaml
optimization:
remote_agents:
enabled: true
check_agent_count: true
min_agents: 4
check_agent_count: true # Verify remote agents are connected before launching
min_agents: 4 # Require at least N agents before optimizing
```
MT5-Quant will log the agent count before launching optimization:
```
[optimize] Local agents: 9, Remote agents: 8, Total: 17
[optimize] Estimated completion: ~105 minutes (17,640 passes at 17 agents)
```
---
## Tick Data Sync
On first run, MT5 automatically downloads ticks from the broker. Pre-populate on Linux:
Remote agents need tick history to replay trades. On first run, MT5 automatically downloads ticks from the broker for each symbol+period combination tested. This can take 10-30 minutes per symbol.
**Speed up first run:** Pre-populate the tick cache on the Linux server by copying from Mac:
```bash
# On Mac — find tick data location
find ~/Library/Application\ Support/MetaQuotes -name "*.bin" | grep "XAUUSD"
# Copy to Linux
# Typical path:
# Terminal/XXXXX/drive_c/users/user/AppData/Roaming/MetaQuotes/Terminal/Common/Files/
scp -r "${TICK_DIR}" user@linux-server:~/mt5agents/ticks/
```
---
## Troubleshooting
**Agents connect then immediately disconnect**
- MT5 version mismatch between `metatester64.exe` (from Mac) and the master. Use the exact same build number.
- Check `~/mt5agents/agents.log` for Wine errors.
**Agents show as connected but don't receive work**
- MT5 sometimes requires optimization to be started before assigning work to new agents.
- Start an optimization from the MT5 GUI first to "activate" remote agents, then cancel it and use MT5-Quant.
**Performance is slower with remote agents**
- Use wired connection. WiFi or WAN: significant overhead.
- Network latency between Mac and Linux server. Results are sent over TCP after each pass.
- Local gigabit network: negligible. WiFi or WAN: significant overhead. Use wired connection.
**Linux server runs out of memory**
- Each `metatester64.exe` instance uses ~200-400MB.
- 8 agents = ~2-3GB RAM. Size agent count accordingly.
+3 -61
View File
@@ -17,17 +17,12 @@ If using CrossOver, confirm bottle is named `MetaTrader5`.
### Linux
Install Wine 7.0+ and confirm on PATH:
```bash
# Debian/Ubuntu
sudo apt install wine64
# Fedora/RHEL
sudo dnf install wine
which wine64
sudo apt install wine64 # Debian/Ubuntu
sudo dnf install wine # Fedora/RHEL
which wine64 # confirm on PATH
```
Ask your LLM platform to install Wine if it's missing.
## terminal64.exe Missing
MT5 unpacks `terminal64.exe` only after first launch.
@@ -74,42 +69,6 @@ Set `MT5_MCP_HOME` or ensure config exists at:
4. **Date range has no trades** — try wider range or confirm symbol was active.
## Backtest Completes But Always Shows "journal-only" (No HTML Report)
**Symptom:** Every backtest produces `status: completed_no_html` and all deal profits are `0.0`.
**Cause:** On Wine/macOS, `ShutdownTerminal=1` does **not** cause `terminal64.exe` to exit after the
test completes. The tester agent (MetaTester 5) closes normally, but the terminal process stays alive
indefinitely. Without process exit, MT5 never writes the HTML report.
**How the pipeline handles this:** The inactivity watchdog detects that the tester log has stopped
growing (test done), waits 30 seconds polling for the HTML file, then kills `terminal64.exe`
unconditionally. If the HTML appears during the wait it is extracted; otherwise journal extraction
provides deal counts and final balance (but no per-deal P&L).
**To maximise HTML report chances:** Pass `inactivity_kill_secs=120` explicitly (it is disabled by
default). With `shutdown=true` (default) this gives MT5 120s of quiet time + 30s of HTML-wait before
the kill.
```
launch_backtest(expert: "MyEA", shutdown: true, inactivity_kill_secs: 120)
```
**Fallback data available from journal:**
- Deal count, volume, prices, timestamps ✓
- Final balance (pips) ✓
- Per-deal profit/loss ✗ (always 0.0)
- Win rate, profit factor, Sharpe, drawdown ✗ (require HTML)
## `list_deals` Returns 0 Filtered Results
Analytics tools filter deals with `is_closed_trade()` which checks `entry = "out"`. If all deals
show `entry = "in"`, the position tracker that infers direction from signed lot accumulation may
not have run (old binary in memory).
**Fix:** Restart the MCP server (restart Claude Code / your IDE) after installing a new binary.
The server process caches the binary in memory — `install` doesn't hot-reload it.
## MetaEditor Compile Errors
Check `<terminal_dir>/MQL5/Logs/`:
@@ -123,23 +82,6 @@ Check `<terminal_dir>/MQL5/Logs/`:
- Check `.set` file values appropriate for symbol/broker
- Confirm `OnInit()` returns `INIT_SUCCEEDED` (MT5 Journal tab)
## Analytics Tools: "No reports in DB" or "Report not found"
Analytics tools load deals from the **SQLite database**, not from CSV files on disk. Resolution order:
1. `report_id` (preferred) — ID from `list_reports`
2. `report_dir` (legacy) — filesystem path, looks up matching DB entry
3. No args — uses the latest report automatically
Pre-DB reports (before this version) won't be found by `report_dir`. Re-run the backtest to get a DB-backed report.
**`deals.csv` is no longer written automatically.** Call `export_deals_csv` to generate one on demand:
```
export_deals_csv() # latest report → report_dir/deals.csv
export_deals_csv(report_id: "20260422_…") # specific report
export_deals_csv(output_path: "/tmp/out.csv") # custom path
```
## Optimization Never Finishes
```bash
+147
View File
@@ -0,0 +1,147 @@
# VS Code MCP Integration Setup
## Quick Setup
### Option 1: Download Prebuilt Binary (Recommended)
```bash
# macOS (Apple Silicon)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz
tar -xzf mt5-quant.tar.gz
# Linux (x64)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz
tar -xzf mt5-quant.tar.gz
```
### Option 2: Build from Source
```bash
cargo build --release
```
## Configure VS Code
### Method 1: Command Palette (Recommended)
1. Press `Cmd/Ctrl + Shift + P`
2. Run `MCP: Add Server`
3. Choose **Workspace** or **User** scope
4. Enter server name: `mt5-quant`
5. Enter command: `~/.local/bin/mt5-quant`
### Method 2: Edit mcp.json Directly
Add to `.vscode/mcp.json` in your workspace:
```json
{
"servers": {
"mt5-quant": {
"command": "~/.local/bin/mt5-quant"
}
}
}
```
Create the file:
```bash
mkdir -p .vscode
cat > .vscode/mcp.json << 'EOF'
{
"servers": {
"mt5-quant": {
"command": "~/.local/bin/mt5-quant"
}
}
}
EOF
```
### Method 3: VS Code CLI
```bash
code --add-mcp '{"name":"mt5-quant","command":"~/.local/bin/mt5-quant"}'
```
## Verify Setup
In Copilot chat, type:
```
Run verify_setup
```
Expected output:
```
Wine: /Applications/MetaTrader 5.app/.../wine64
MT5 dir: ~/Library/Application Support/.../MetaTrader 5
Display: gui
Arch: arch -x86_64
```
## Configuration Locations
| Scope | Path | Use Case |
|-------|------|----------|
| Workspace | `.vscode/mcp.json` | Share with team via source control |
| User | `~/.vscode/mcp.json` | Personal tools across all projects |
| Dev Container | `devcontainer.json``customizations.vscode.mcp` | Containerized environments |
## Troubleshooting
### MCP server not appearing
1. Open **Output** panel (`Cmd/Ctrl + Shift + U`)
2. Select **MCP** from dropdown
3. Check for connection errors
4. Verify the path is absolute
### Config not found
The binary auto-detects its config, but you can also:
1. Run `setup.sh` to create `config/mt5-quant.yaml`
2. Or let the binary auto-discover on first run
### Dev Container Setup
Add to `.devcontainer/devcontainer.json`:
```json
{
"customizations": {
"vscode": {
"mcp": {
"servers": {
"mt5-quant": {
"command": "/path/to/mt5-quant"
}
}
}
}
}
}
```
## Key Differences from Other IDEs
VS Code uses `servers` (not `mcpServers`) in the JSON structure:
```json
{
"servers": { // ← VS Code uses "servers"
"mt5-quant": {
"command": "..."
}
}
}
```
Other platforms use `mcpServers`.
## Resources
- [VS Code MCP Documentation](https://code.visualstudio.com/docs/copilot/customization/mcp-servers)
- [MCP Configuration Reference](https://code.visualstudio.com/docs/copilot/reference/mcp-configuration)
- [MT5-Quant Tools Reference](./MCP_TOOLS.md)
+110
View File
@@ -0,0 +1,110 @@
# Windsurf MCP Integration Setup
## Quick Setup
### Option 1: Download Prebuilt Binary (Recommended)
```bash
# macOS (Apple Silicon)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-macos-arm64.tar.gz
tar -xzf mt5-quant.tar.gz
# Linux (x64)
curl -L -o mt5-quant.tar.gz https://github.com/masdevid/mt5-mcp/releases/latest/download/mt5-quant-linux-x64.tar.gz
tar -xzf mt5-quant.tar.gz
```
### Option 2: Build from Source
```bash
bash scripts/build-rust.sh
```
### 2. Configure Windsurf
Edit `~/.codeium/windsurf/mcp_config.json`:
```json
{
"mcpServers": {
"io.github.masdevid/mt5-quant": {
"command": "~/.local/bin/mt5-quant",
"disabled": false,
"registry": "io.github.masdevid/mt5-quant"
}
}
}
```
Or use the automated setup:
```bash
# Install binary to standard location
cp target/release/mt5-quant ~/.local/bin/
# Then configure
bash scripts/setup.sh
```
### 3. Restart Windsurf
Close and reopen Windsurf to load the MCP server.
### 4. Verify
In Windsurf chat, test with:
```
Run verify_setup
```
## Deployment to Multiple Machines
### Build for Distribution
```bash
# Build release binary
cargo build --release
# Create tarball
tar -czf mt5-quant-macos-arm64.tar.gz -C target/release mt5-quant
# Deploy to remote server
scp mt5-quant-macos-arm64.tar.gz user@server:~/
ssh user@server "tar -xzf mt5-quant-macos-arm64.tar.gz -C /opt/"
ssh user@server "ln -s /opt/mt5-quant /usr/local/bin/"
# Copy config
scp -r config/mt5-quant.yaml user@server:~/.config/mt5-quant/config/
```
### Target Machine Requirements
- MetaTrader 5 installed (via Wine/CrossOver)
- Config file at `~/.config/mt5-quant/config/mt5-quant.yaml`
- **NO Python required!**
### Windsurf Config on Target Machine
```json
{
"mcpServers": {
"mt5-quant": {
"command": "/usr/local/bin/mt5-quant"
}
}
}
```
The binary auto-detects its config location. No environment variables needed.
## Troubleshooting
### MCP server not appearing
1. Check Windsurf logs: `~/.windsurf/logs/`
2. Verify executable path is absolute
3. Test executable manually: `./target/release/mt5-quant --help`
### Config not found
Set `MT5_MCP_HOME` environment variable or ensure config is at default location:
- macOS: `~/.config/mt5-quant/config/mt5-quant.yaml`
### Permission denied
```bash
chmod +x /path/to/mt5-quant
```
+23 -30
View File
@@ -1,11 +1,8 @@
#!/usr/bin/env bash
# Build release binary and package it for distribution
# Creates: dist/mcp-mt5-quant-{platform}.tar.gz
#
# Usage:
# bash scripts/build-release.sh
#!/bin/bash
# Build release binaries for distribution
# Creates: dist/mt5-quant-{platform}.tar.gz
set -euo pipefail
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
@@ -13,17 +10,16 @@ PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
cd "$PROJECT_ROOT"
VERSION=$(grep -E '^version = ' Cargo.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
echo "=== Building mt5-quant v${VERSION} ==="
echo "=== Building MCP-MT5-Quant v${VERSION} ==="
echo ""
# Clean previous builds
rm -rf "$PROJECT_ROOT/dist"
mkdir -p "$PROJECT_ROOT/dist"
# Build release binary
echo "Building release binary..."
# Build current platform
echo "Building for current platform..."
RUSTFLAGS="-D warnings" cargo build --release
echo ""
# Detect platform
UNAME=$(uname -s)
@@ -34,46 +30,43 @@ if [[ "$UNAME" == "Darwin" ]]; then
elif [[ "$UNAME" == "Linux" ]]; then
PLATFORM="linux-${ARCH}"
else
PLATFORM="unknown-${ARCH}"
PLATFORM="unknown"
fi
PACKAGE_NAME="mcp-mt5-quant-${PLATFORM}"
PACKAGE_DIR="$PROJECT_ROOT/dist/${PACKAGE_NAME}"
echo "Packaging for ${PLATFORM}..."
mkdir -p "$PACKAGE_DIR/docs"
mkdir -p "$PACKAGE_DIR"
# Binary
# Copy binary
cp "$PROJECT_ROOT/target/release/mt5-quant" "$PACKAGE_DIR/"
chmod +x "$PACKAGE_DIR/mt5-quant"
# Config template
# Copy config template
mkdir -p "$PACKAGE_DIR/config"
cp "$PROJECT_ROOT/config/mt5-quant.example.yaml" "$PACKAGE_DIR/config/"
# Documentation
cp "$PROJECT_ROOT/README.md" "$PACKAGE_DIR/"
cp "$PROJECT_ROOT/docs/QUICKSTART.md" "$PACKAGE_DIR/docs/"
cp "$PROJECT_ROOT/docs/CLAUDE.md" "$PACKAGE_DIR/docs/"
cp "$PROJECT_ROOT/docs/CURSOR.md" "$PACKAGE_DIR/docs/"
cp "$PROJECT_ROOT/docs/VSCODE.md" "$PACKAGE_DIR/docs/"
cp "$PROJECT_ROOT/docs/WINDSURF.md" "$PACKAGE_DIR/docs/"
# Copy docs
cp "$PROJECT_ROOT/README.md" "$PACKAGE_DIR/"
cp "$PROJECT_ROOT/docs/WINDSURF.md" "$PACKAGE_DIR/WINDSURF_SETUP.md"
cp "$PROJECT_ROOT/CLAUDE.md" "$PACKAGE_DIR/"
# Create tarball
cd "$PROJECT_ROOT/dist"
tar -czf "${PACKAGE_NAME}.tar.gz" "$PACKAGE_NAME"
rm -rf "$PACKAGE_NAME"
echo ""
echo "=== Build complete ==="
echo "=== Build Complete ==="
echo ""
echo "Package : dist/${PACKAGE_NAME}.tar.gz"
echo "Size : $(du -h "${PACKAGE_NAME}.tar.gz" | cut -f1)"
echo "Package: dist/${PACKAGE_NAME}.tar.gz"
echo "Binary: mt5-quant"
echo "Size: $(du -h "${PACKAGE_NAME}.tar.gz" | cut -f1)"
echo ""
echo "Contents:"
tar -tzf "${PACKAGE_NAME}.tar.gz"
tar -tzf "${PACKAGE_NAME}.tar.gz" | head -10
echo ""
echo "Install:"
echo "To install:"
echo " tar -xzf ${PACKAGE_NAME}.tar.gz"
echo " sudo cp ${PACKAGE_NAME}/mt5-quant /usr/local/bin/"
echo " cd ${PACKAGE_NAME}"
echo " sudo cp mt5-quant /usr/local/bin/"
echo " mt5-quant --help"
+15 -8
View File
@@ -1,25 +1,32 @@
#!/usr/bin/env bash
# Build MT5-Quant release binary
#!/bin/bash
# Build MT5-Quant Rust MCP Server
# Output: target/release/mt5-quant
set -euo pipefail
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
cd "$PROJECT_ROOT"
echo "=== MT5-Quant build ==="
echo "Root: $PROJECT_ROOT"
echo "=== MT5-Quant Rust Build ==="
echo "Project root: $PROJECT_ROOT"
echo ""
echo "Building release binary..."
cargo build --release
echo ""
echo "=== Done ==="
echo "=== Build Complete ==="
echo ""
echo "Executable location:"
ls -lh "$PROJECT_ROOT/target/release/mt5-quant"
echo ""
echo "Run: ./target/release/mt5-quant --help"
echo "Install: cargo install --path . --force"
echo "To test:"
echo " ./target/release/mt5-quant --help"
echo ""
echo "To install for Windsurf:"
echo " Update ~/.windsurf/config.yaml:"
echo " command: $PROJECT_ROOT/target/release/mt5-quant"
echo ""
+38 -76
View File
@@ -7,7 +7,6 @@
# bash scripts/release.sh major # 1.31.0 → 2.0.0
# bash scripts/release.sh 1.32.0 # explicit version
# bash scripts/release.sh v1.32.0 # with v prefix
# bash scripts/release.sh patch --yes # non-interactive (CI / Claude Code)
set -euo pipefail
cd "$(dirname "$0")/.."
@@ -19,54 +18,44 @@ warn() { echo -e "${YELLOW}⚠ $*${NC}"; }
die() { echo -e "${RED}$*${NC}" >&2; exit 1; }
hr() { echo -e "${BLUE}────────────────────────────────────────${NC}"; }
# ── Argument parsing ───────────────────────────────────────────────────────────
# ── Prerequisites ─────────────────────────────────────────────────────────────
bump="${1:-patch}"
AUTO_YES=false
if [[ "${2:-}" == "--yes" || "${2:-}" == "-y" || "${CI:-}" == "true" ]]; then
AUTO_YES=true
fi
# ── Prerequisites ──────────────────────────────────────────────────────────────
command -v cargo >/dev/null 2>&1 || die "cargo not found"
command -v cargo >/dev/null 2>&1 || die "cargo not found"
command -v python3 >/dev/null 2>&1 || die "python3 not found"
# ── Current version ───────────────────────────────────────────────────────────
# ── Current version ───────────────────────────────────────────────────────────
current=$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)".*/\1/')
[[ -n "$current" ]] || die "Could not parse version from Cargo.toml"
info "Current version: ${BOLD}$current${NC}"
# ── Compute new version ───────────────────────────────────────────────────────
# ── Compute new version ───────────────────────────────────────────────────────
bump="${1:-patch}"
IFS='.' read -r major minor patch_v <<< "$current"
case "$bump" in
major) new="$((major + 1)).0.0" ;;
minor) new="${major}.$((minor + 1)).0" ;;
patch) new="${major}.${minor}.$((patch_v + 1))" ;;
v*.*.*) new="${bump#v}" ;;
*.*.*) new="$bump" ;;
*) die "Usage: $0 [patch|minor|major|X.Y.Z] [--yes]" ;;
major) new="${major+1}.0.0"; new="$((major + 1)).0.0" ;;
minor) new="${major}.$((minor + 1)).0" ;;
patch) new="${major}.${minor}.$((patch_v + 1))" ;;
v*.*.*) new="${bump#v}" ;;
*.*.*) new="$bump" ;;
*) die "Usage: $0 [patch|minor|major|X.Y.Z]" ;;
esac
# Validate semver
[[ "$new" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || die "Invalid version: $new"
hr
info "Releasing: ${BOLD}$current$new${NC}"
hr
# ── Confirm ───────────────────────────────────────────────────────────────────
# ── Confirm ───────────────────────────────────────────────────────────────────
if [[ "$AUTO_YES" == false ]]; then
read -rp "Proceed with release v${new}? [y/N] " confirm
[[ "$confirm" =~ ^[Yy]$ ]] || die "Aborted"
else
info "Auto-confirmed (--yes)"
fi
read -rp "Proceed with release v${new}? [y/N] " confirm
[[ "$confirm" =~ ^[Yy]$ ]] || die "Aborted"
# ── Check git state ───────────────────────────────────────────────────────────
# ── Check git state ───────────────────────────────────────────────────────────
info "Checking git state..."
if ! git diff --quiet 2>/dev/null || ! git diff --cached --quiet 2>/dev/null; then
@@ -75,12 +64,13 @@ fi
current_branch=$(git rev-parse --abbrev-ref HEAD)
info "Branch: $current_branch"
# Check tag doesn't already exist
if git rev-parse "v${new}" >/dev/null 2>&1; then
die "Tag v${new} already exists"
fi
ok "Git state clean"
# ── 1. Bump Cargo.toml ────────────────────────────────────────────────────────
# ── 1. Bump Cargo.toml ────────────────────────────────────────────────────────
info "Bumping Cargo.toml..."
if [[ "$(uname)" == "Darwin" ]]; then
@@ -88,14 +78,16 @@ if [[ "$(uname)" == "Darwin" ]]; then
else
sed -i "s/^version = \"${current}\"/version = \"${new}\"/" Cargo.toml
fi
# Update Cargo.lock
cargo metadata --no-deps --format-version 1 >/dev/null 2>&1 || true
ok "Cargo.toml → $new"
# ── 2. Update server.json ─────────────────────────────────────────────────────
# ── 2. Update server.json ─────────────────────────────────────────────────────
info "Updating server.json..."
NEW_VERSION="$new" python3 - <<'PYEOF'
import json, os, re
import json, os, re, sys
version = os.environ['NEW_VERSION']
@@ -106,6 +98,7 @@ data['version'] = version
for pkg in data.get('packages', []):
pkg['version'] = version
# Update download URL to point at new version tag
if 'identifier' in pkg:
pkg['identifier'] = re.sub(
r'/v[0-9]+\.[0-9]+\.[0-9]+/',
@@ -121,9 +114,9 @@ with open('server.json', 'w') as f:
print(f" server.json version={version}, identifier URL updated")
PYEOF
ok "server.json → $new (SHA256 set by CI)"
ok "server.json → $new (SHA256 updated by CI)"
# ── 3. Generate CHANGELOG entry ───────────────────────────────────────────────
# ── 3. Generate CHANGELOG entry ───────────────────────────────────────────────
info "Generating CHANGELOG entry..."
prev_tag=$(git describe --tags --abbrev=0 2>/dev/null || echo "")
@@ -145,6 +138,7 @@ today=$(date +%Y-%m-%d)
} > /tmp/release_entry.md
if [[ -f CHANGELOG.md ]]; then
# Insert after the first header line
tmp=$(mktemp)
head -1 CHANGELOG.md > "$tmp"
echo "" >> "$tmp"
@@ -152,18 +146,22 @@ if [[ -f CHANGELOG.md ]]; then
tail -n +2 CHANGELOG.md >> "$tmp"
mv "$tmp" CHANGELOG.md
else
{ echo "# Changelog"; echo ""; cat /tmp/release_entry.md; } > CHANGELOG.md
{
echo "# Changelog"
echo ""
cat /tmp/release_entry.md
} > CHANGELOG.md
fi
rm -f /tmp/release_entry.md
ok "CHANGELOG.md updated"
# ── 4. Verify build ────────────────────────────────────────────────────────────
# ── 4. Verify build compiles ──────────────────────────────────────────────────
info "Verifying build (cargo check)..."
cargo check --quiet 2>&1 || die "cargo check failed — fix errors before releasing"
ok "Build check passed"
# ── 5. Commit ─────────────────────────────────────────────────────────────────
# ── 5. Commit ─────────────────────────────────────────────────────────────────
info "Creating release commit..."
git add Cargo.toml Cargo.lock server.json CHANGELOG.md
@@ -174,56 +172,20 @@ git commit -m "release: v${new}
- CHANGELOG.md updated"
ok "Release commit created"
# ── 6. Tag ────────────────────────────────────────────────────────────────────
# ── 6. Tag ────────────────────────────────────────────────────────────────────
info "Creating annotated tag v${new}..."
git tag -a "v${new}" -m "Release v${new}"
ok "Tagged v${new}"
# ── 7. Push (rebase if remote has new commits from CI) ─────────────────────────
# ── 7. Push ───────────────────────────────────────────────────────────────────
info "Pushing to GitHub..."
if ! git push origin "$current_branch" 2>/dev/null; then
warn "Push rejected — rebasing on remote (CI may have committed SHA256)..."
git fetch origin "$current_branch"
# Resolve server.json conflict automatically: keep our version
if ! git rebase "origin/${current_branch}"; then
if git diff --name-only --diff-filter=U | grep -q "server.json"; then
python3 - <<'PYEOF'
import re
with open('server.json') as f:
raw = f.read()
resolved = re.sub(
r'<<<<<<< HEAD.*?=======\n(.*?)>>>>>>> [^\n]+\n',
r'\1',
raw,
flags=re.DOTALL
)
with open('server.json', 'w') as f:
f.write(resolved)
PYEOF
git add server.json
GIT_EDITOR=true git rebase --continue
else
git rebase --abort
die "Rebase failed with unexpected conflicts — push manually"
fi
fi
git push origin "$current_branch"
fi
# Push tag (delete and re-push if rebase moved the commit)
git push origin "v${new}" 2>/dev/null || {
git push origin ":refs/tags/v${new}" 2>/dev/null || true
git tag -f -a "v${new}" -m "Release v${new}"
git push origin "v${new}"
}
git push origin "$current_branch"
git push origin "v${new}"
ok "Pushed — GitHub Actions triggered"
# ── Done ──────────────────────────────────────────────────────────────────────
# ── Done ──────────────────────────────────────────────────────────────────────
hr
echo ""
+2 -2
View File
@@ -14,7 +14,7 @@
# - Windsurf : ~/.codeium/windsurf/mcp_config.json (JSON, mcpServers)
# - Cursor : ~/.cursor/mcp.json (JSON, mcpServers)
# - VS Code : .vscode/mcp.json (JSON, servers - not mcpServers)
# - Claude Desktop: ~/Library/Application Support/Claude/claude_desktop_config.json
# - Antigravity : mcp_config.json via UI (JSON, mcpServers)
#
# Previous installations are auto-detected and uninstalled before re-registering.
@@ -1201,7 +1201,7 @@ _register_all_mcp_platforms() {
echo " - Windsurf: Edit ~/.codeium/windsurf/mcp_config.json"
echo " - Cursor: Edit ~/.cursor/mcp.json"
echo " - VS Code: Edit .vscode/mcp.json (workspace) or use MCP: Add Server command"
echo " - Claude Desktop: Edit ~/Library/Application Support/Claude/claude_desktop_config.json"
echo " - Antigravity:Use Agent panel → MCP Servers → Manage → Edit configuration"
echo ""
return
fi
+4 -4
View File
@@ -7,13 +7,13 @@
"url": "https://github.com/masdevid/mt5-quant",
"source": "github"
},
"version": "1.34.0",
"version": "1.31.2",
"packages": [
{
"registryType": "mcpb",
"version": "1.34.0",
"identifier": "https://github.com/masdevid/mt5-quant/releases/download/v1.34.0/mcp-mt5-quant-macos-arm64.tar.gz",
"fileSha256": "a8228ff8b547cae86b9cd3ce86704cc3f42b8fe25d034e98b5c0f5f599add50a",
"version": "1.31.2",
"identifier": "https://github.com/masdevid/mt5-quant/releases/download/v1.31.2/mcp-mt5-quant-macos-arm64.tar.gz",
"fileSha256": "TBD_CI_WILL_UPDATE",
"transport": {
"type": "stdio"
},
+61 -84
View File
@@ -15,7 +15,7 @@ impl ReportExtractor {
pub fn extract(&self, report_path: &str, output_dir: &str) -> Result<ExtractionResult> {
let format = Self::detect_format(report_path);
let (metrics, deals) = match format {
ReportFormat::Xml => self.parse_xml(report_path)?,
ReportFormat::Html => self.parse_html(report_path)?,
@@ -24,19 +24,22 @@ impl ReportExtractor {
fs::create_dir_all(output_dir)?;
let metrics_path = Path::new(output_dir).join("metrics.json");
let deals_csv_path = Path::new(output_dir).join("deals.csv");
let deals_json_path = Path::new(output_dir).join("deals.json");
self.write_metrics(&metrics, &metrics_path)?;
self.write_deals_json(&deals, &deals_json_path)?;
self.write_deals_csv(&deals, &deals_csv_path)?;
Ok(ExtractionResult {
metrics,
deals,
metrics_path,
deals_csv_path,
deals_json_path,
})
}
pub fn write_deals_to_csv(&self, deals: &[Deal], path: &Path) -> Result<()> {
self.write_deals_csv(deals, path)
}
fn detect_format(path: &str) -> ReportFormat {
if path.ends_with(".xml") || path.ends_with(".htm.xml") {
return ReportFormat::Xml;
@@ -77,96 +80,59 @@ impl ReportExtractor {
fn parse_deals_html(&self, text: &str) -> Result<Vec<Deal>> {
let mut deals = Vec::new();
// (?s) = dotall: makes '.' match '\n' so the regex works on multiline HTML.
// MT5 HTML column order: Time | Deal | Symbol | Type | Direction |
// Volume | Price | Order | Commission | Swap | Profit | Balance | Comment
//
// Strategy: locate the deals table by finding the header <tr> that contains
// the column names, then parse every subsequent <tr> as a data row.
// We try two header patterns to handle different MT5 versions/locales.
let re = regex::Regex::new(r"<tr[^>]*>.*?Deal.*?Time.*?Type.*?Direction.*?</tr>(.*)")
.map_err(|e| anyhow!("Regex error: {}", e))?;
let row_re = regex::Regex::new(r"(?s)<tr[^>]*>(.*?)</tr>")
.map_err(|e| anyhow!("Row regex error: {}", e))?;
let cell_re = regex::Regex::new(r"(?s)<td[^>]*>(.*?)</td>")
.map_err(|e| anyhow!("Cell regex error: {}", e))?;
if let Some(captures) = re.captures(text) {
let section = captures.get(1).map(|m| m.as_str()).unwrap_or("");
// Collect all <tr> blocks once, then find the deals header and parse from there.
let rows: Vec<&str> = row_re.captures_iter(text)
.filter_map(|cap| cap.get(0).map(|m| m.as_str()))
.collect();
let row_re = regex::Regex::new(r"<tr[^>]*>(.*?)</tr>")
.map_err(|e| anyhow!("Regex error: {}", e))?;
// Find the header row index: it must contain both "Deal" and "Symbol" (case-insensitive).
let header_idx = rows.iter().position(|row| {
let lower = row.to_lowercase();
(lower.contains(">deal<") || lower.contains(">deal </")) &&
(lower.contains(">symbol<") || lower.contains(">symbol </"))
});
for row_caps in row_re.captures_iter(section) {
let row = row_caps.get(1).map(|m| m.as_str()).unwrap_or("");
let cell_re = regex::Regex::new(r"<td[^>]*>(.*?)</td>")
.map_err(|e| anyhow!("Regex error: {}", e))?;
let cells: Vec<String> = cell_re.captures_iter(row)
.filter_map(|cap| cap.get(1))
.map(|m| Self::strip_tags(m.as_str()))
.map(|s| s.replace(',', ""))
.collect();
let start_idx = match header_idx {
Some(i) => i + 1,
None => {
// Fallback: look for any row containing Time+Volume+Profit headers
let alt = rows.iter().position(|row| {
let lower = row.to_lowercase();
lower.contains(">time<") && lower.contains(">volume<") && lower.contains(">profit<")
});
match alt {
Some(i) => i + 1,
None => {
tracing::warn!("parse_deals_html: no deals table header found");
return Ok(deals);
}
if cells.len() < 3 || cells[0].is_empty() {
continue;
}
}
};
for row in &rows[start_idx..] {
let cells: Vec<String> = cell_re.captures_iter(row)
.filter_map(|cap| cap.get(1))
.map(|m| Self::strip_tags(m.as_str()))
.map(|s| s.replace(',', ""))
.collect();
if cells.iter().take(5).any(|c| {
let c_lower = c.trim().to_lowercase();
c_lower == "balance" || c_lower == "credit"
}) {
continue;
}
if cells.len() < 3 || cells[0].trim().is_empty() {
continue;
}
let deal = Deal {
time: cells.get(0).cloned().unwrap_or_default(),
deal: cells.get(1).cloned().unwrap_or_default(),
symbol: cells.get(2).cloned().unwrap_or_default(),
deal_type: cells.get(3).cloned().unwrap_or_default(),
entry: cells.get(4).cloned().unwrap_or_default(),
volume: cells.get(5).and_then(|s| s.parse().ok()).unwrap_or(0.0),
price: cells.get(6).and_then(|s| s.parse().ok()).unwrap_or(0.0),
order: cells.get(7).cloned().unwrap_or_default(),
commission: cells.get(8).and_then(|s| s.parse().ok()).unwrap_or(0.0),
swap: cells.get(9).and_then(|s| s.parse().ok()).unwrap_or(0.0),
profit: cells.get(10).and_then(|s| s.parse().ok()).unwrap_or(0.0),
balance: cells.get(11).and_then(|s| s.parse().ok()).unwrap_or(0.0),
comment: cells.get(12).cloned().unwrap_or_default(),
magic: cells.get(13).cloned(),
};
// Skip balance/credit operation rows (Type column is index 3 in MT5 HTML)
let type_cell = cells.get(3).map(|s| s.trim().to_lowercase()).unwrap_or_default();
if type_cell == "balance" || type_cell == "credit" {
continue;
deals.push(deal);
}
// Also skip sub-header rows that repeat column names
if type_cell == "type" || cells.get(1).map(|s| s.trim().to_lowercase()).as_deref() == Some("deal") {
continue;
}
// Skip rows with no deal number (e.g. totals row)
let deal_num = cells.get(1).map(|s| s.trim().to_string()).unwrap_or_default();
if deal_num.is_empty() {
continue;
}
let deal = Deal {
time: cells.get(0).cloned().unwrap_or_default(),
deal: deal_num,
symbol: cells.get(2).cloned().unwrap_or_default(),
deal_type: cells.get(3).cloned().unwrap_or_default(),
entry: cells.get(4).cloned().unwrap_or_default(),
volume: cells.get(5).and_then(|s| s.parse().ok()).unwrap_or(0.0),
price: cells.get(6).and_then(|s| s.parse().ok()).unwrap_or(0.0),
order: cells.get(7).cloned().unwrap_or_default(),
commission: cells.get(8).and_then(|s| s.parse().ok()).unwrap_or(0.0),
swap: cells.get(9).and_then(|s| s.parse().ok()).unwrap_or(0.0),
profit: cells.get(10).and_then(|s| s.parse().ok()).unwrap_or(0.0),
balance: cells.get(11).and_then(|s| s.parse().ok()).unwrap_or(0.0),
comment: cells.get(12).cloned().unwrap_or_default(),
magic: cells.get(13).cloned(),
};
deals.push(deal);
}
tracing::info!("parse_deals_html: extracted {} deals", deals.len());
Ok(deals)
}
@@ -249,6 +215,13 @@ impl ReportExtractor {
Ok(())
}
fn write_deals_json(&self, deals: &[Deal], path: &Path) -> Result<()> {
let json = serde_json::to_string_pretty(deals)?;
let mut file = File::create(path)?;
file.write_all(json.as_bytes())?;
Ok(())
}
fn write_deals_csv(&self, deals: &[Deal], path: &Path) -> Result<()> {
let mut file = File::create(path)?;
writeln!(file, "time,deal,symbol,type,entry,volume,price,order,commission,swap,profit,balance,comment")?;
@@ -306,6 +279,10 @@ pub struct ExtractionResult {
pub deals: Vec<Deal>,
#[allow(dead_code)]
pub metrics_path: PathBuf,
#[allow(dead_code)]
pub deals_csv_path: PathBuf,
#[allow(dead_code)]
pub deals_json_path: PathBuf,
}
#[derive(Debug, Clone, Copy)]
+4 -30
View File
@@ -90,8 +90,8 @@ impl MqlCompiler {
.map(|s| s.to_string())
.collect();
let staged_ex5 = staged_mq5.with_extension("ex5");
if !staged_ex5.exists() {
let ex5_path = staged_mq5.with_extension("ex5");
if !ex5_path.exists() {
let final_errors = if errors.is_empty() {
vec![format!("Compilation failed. Log:\n{}", &log_text[log_text.len().saturating_sub(500)..])]
} else {
@@ -107,37 +107,11 @@ impl MqlCompiler {
});
}
let binary_size = fs::metadata(&staged_ex5)?.len();
// Deploy compiled output to the real MQL5/Experts/{ea_name}/ directory so
// MT5 can actually load it. The temp dir is only needed to avoid Wine path
// issues with spaces; the real experts dir is the authoritative location.
let final_ex5_path = if let Some(experts_dir) = self.config.experts_dir.as_ref() {
let real_experts = PathBuf::from(experts_dir);
let real_ea_dir = real_experts.join(ea_name);
fs::create_dir_all(&real_ea_dir)?;
// Sync source files (.mq5 + .mqh) into the real experts dir so that
// future compiles from the experts dir also work correctly.
if let Err(e) = self.sync_project_to_experts(source_path, &real_experts) {
tracing::warn!("Could not sync source to experts dir: {}", e);
}
// Copy the compiled .ex5 to the real experts dir.
let dest_ex5 = real_ea_dir.join(format!("{}.ex5", ea_name));
fs::copy(&staged_ex5, &dest_ex5)?;
tracing::info!("Deployed {} → {}", staged_ex5.display(), dest_ex5.display());
dest_ex5
} else {
// No experts_dir configured — fall back to the staged path so callers
// still get a valid path even if MT5 won't find it automatically.
tracing::warn!("experts_dir not configured; .ex5 left in staging dir");
staged_ex5
};
let binary_size = fs::metadata(&ex5_path)?.len();
Ok(CompileResult {
success: errors.is_empty(),
ex5_path: Some(final_ex5_path),
ex5_path: Some(ex5_path),
errors,
warnings,
binary_size,
-94
View File
@@ -12,14 +12,10 @@ use anyhow::Result;
use clap::Parser;
use serde_json::{json, Value};
use std::io::{stdout, Write};
use std::time::Instant;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpListener;
use tracing::{info, error};
use crate::models::Config;
use crate::pipeline::backtest::{BacktestPipeline, BacktestParams};
#[derive(Parser)]
#[command(name = "mt5-quant")]
#[command(about = "MT5-Quant MCP Server - Exposes MT5 backtest and optimization tools via MCP")]
@@ -31,18 +27,6 @@ struct Cli {
/// Run on TCP port for debugging
#[arg(short, long)]
port: Option<u16>,
/// Test backtest launch performance (direct Rust call, not MCP)
#[arg(long)]
test_launch: bool,
/// EA name for test launch
#[arg(long)]
ea: Option<String>,
/// Startup delay for test launch (default: 10)
#[arg(long)]
startup_delay: Option<u64>,
}
#[derive(Debug, serde::Deserialize, serde::Serialize)]
@@ -104,11 +88,6 @@ async fn main() -> Result<()> {
let cli = Cli::parse();
if cli.test_launch {
run_test_launch(cli.ea, cli.startup_delay).await?;
return Ok(());
}
if let Some(port) = cli.port {
run_tcp_server(port).await?;
} else {
@@ -118,83 +97,10 @@ async fn main() -> Result<()> {
Ok(())
}
async fn run_test_launch(ea: Option<String>, startup_delay: Option<u64>) -> Result<()> {
let expert = ea.ok_or_else(|| anyhow::anyhow!("--ea is required for test launch"))?;
let delay = startup_delay.unwrap_or(10);
println!("Testing MT5 backtest launch optimizations...");
println!("==============================================");
println!("EA: {}", expert);
println!("Startup delay: {}s", delay);
let config = Config::load()?;
let params = BacktestParams {
expert: expert.clone(),
symbol: "XAUUSD".to_string(),
from_date: "2024.01.01".to_string(),
to_date: "2024.01.31".to_string(),
timeframe: "M5".to_string(),
deposit: 10000,
model: 0,
leverage: 500,
set_file: None,
skip_compile: true,
skip_clean: false,
skip_analyze: true,
deep_analyze: false,
shutdown: false,
kill_existing: false,
timeout: 900,
gui: false,
startup_delay_secs: delay,
inactivity_kill_secs: None,
};
let pipeline = BacktestPipeline::new(config);
println!("\nLaunching backtest...");
let start = Instant::now();
match pipeline.launch_backtest(params).await {
Ok(job) => {
let elapsed = start.elapsed();
println!("✓ Launch completed in {:.2}s", elapsed.as_secs_f64());
println!(" Report ID: {}", job.report_id);
println!(" Report dir: {}", job.report_dir);
println!("\nUse get_backtest_status to monitor progress.");
}
Err(e) => {
let elapsed = start.elapsed();
println!("✗ Launch failed after {:.2}s: {}", elapsed.as_secs_f64(), e);
}
}
println!("\n==============================================");
println!("Test complete.");
Ok(())
}
async fn run_stdio_server() -> Result<()> {
info!("Starting MT5-Quant MCP server on stdio");
let server = std::sync::Arc::new(mcp_server::McpServer::new());
let (notification_tx, mut notification_rx) = tokio::sync::mpsc::unbounded_channel::<mcp_server::Notification>();
server.set_notification_sender(notification_tx).await;
// Spawn notification sender task
tokio::spawn(async move {
while let Some(notification) = notification_rx.recv().await {
let notification_json = json!({
"jsonrpc": "2.0",
"method": notification.method,
"params": notification.params,
});
println!("{}", notification_json);
let _ = stdout().flush();
}
});
let mut reader = BufReader::new(tokio::io::stdin());
let mut line = String::new();
+11 -62
View File
@@ -1,12 +1,9 @@
use serde_json::{json, Value};
use std::sync::Arc;
use tokio::sync::{mpsc, Mutex};
use tokio::sync::Mutex;
use crate::{models::Config as ModelsConfig, tools::ToolHandler, McpError, McpRequest, McpResponse};
#[allow(dead_code)]
type NotificationCallback = Arc<dyn Fn(&str, serde_json::Value) + Send + Sync>;
/// Auto-verify result stored after first initialization
#[derive(Debug, Clone)]
#[allow(dead_code)]
@@ -16,17 +13,11 @@ struct AutoVerifyResult {
config_path: String,
}
#[derive(Debug, Clone)]
pub struct Notification {
pub method: String,
pub params: Value,
}
#[derive(Debug)]
pub struct McpServer {
initialized: Arc<Mutex<bool>>,
tool_handler: Arc<Mutex<Option<ToolHandler>>>,
tool_handler: Arc<ToolHandler>,
auto_verify_result: Arc<Mutex<Option<AutoVerifyResult>>>,
notification_tx: Arc<Mutex<Option<mpsc::UnboundedSender<Notification>>>>,
}
impl McpServer {
@@ -34,38 +25,11 @@ impl McpServer {
let config = ModelsConfig::load().unwrap_or_default();
Self {
initialized: Arc::new(Mutex::new(false)),
tool_handler: Arc::new(Mutex::new(Some(ToolHandler::new(config)))),
tool_handler: Arc::new(ToolHandler::new(config)),
auto_verify_result: Arc::new(Mutex::new(None)),
notification_tx: Arc::new(Mutex::new(None)),
}
}
pub async fn set_notification_sender(&self, tx: mpsc::UnboundedSender<Notification>) {
let mut guard = self.notification_tx.lock().await;
*guard = Some(tx.clone());
// Update tool handler with notification callback
let tx_clone = tx.clone();
let callback = Arc::new(move |method: &str, params: serde_json::Value| {
let _ = tx_clone.send(Notification {
method: method.to_string(),
params,
});
});
let config = ModelsConfig::load().unwrap_or_default();
let new_handler = ToolHandler::with_notification_callback(config, callback);
let mut handler_guard = self.tool_handler.lock().await;
*handler_guard = Some(new_handler);
}
#[allow(dead_code)]
pub async fn get_notification_sender(&self) -> Option<mpsc::UnboundedSender<Notification>> {
let guard = self.notification_tx.lock().await;
guard.clone()
}
/// Run verify_setup in background - non blocking
fn spawn_auto_verify(&self) {
let result_arc = self.auto_verify_result.clone();
@@ -283,27 +247,12 @@ impl McpServer {
}
async fn handle_tool_call(&self, tool_name: &str, arguments: &Value) -> Value {
let handler_guard = self.tool_handler.lock().await;
let handler = handler_guard.as_ref().cloned();
drop(handler_guard);
match handler {
Some(h) => {
h.handle(tool_name, arguments).await.unwrap_or_else(|e| json!({
"content": [{
"type": "text",
"text": format!("Tool execution failed: {}", e)
}],
"isError": true
}))
}
None => json!({
"content": [{
"type": "text",
"text": "Tool handler not initialized"
}],
"isError": true
})
}
self.tool_handler.handle(tool_name, arguments).await.unwrap_or_else(|e| json!({
"content": [{
"type": "text",
"text": format!("Tool execution failed: {}", e)
}],
"isError": true
}))
}
}
+35 -100
View File
@@ -97,7 +97,6 @@ pub struct Config {
pub reports_dir: Option<String>,
pub backtest_login: Option<String>,
pub backtest_server: Option<String>,
pub backtest_password: Option<String>,
pub project_dir: Option<String>,
}
@@ -124,7 +123,6 @@ impl Default for Config {
reports_dir: None,
backtest_login: None,
backtest_server: None,
backtest_password: None,
project_dir: None,
}
}
@@ -247,21 +245,19 @@ impl Config {
fn find_wine(home: &Path) -> Option<String> {
let candidates: &[PathBuf] = &[
// macOS: bundled with the official MT5 app (binary is just named 'wine' on recent builds)
PathBuf::from("/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine"),
// macOS: bundled with the official MT5 app
PathBuf::from("/Applications/MetaTrader 5.app/Contents/SharedSupport/wine/bin/wine64"),
// macOS: CrossOver (new versions may use 'wine', older ones 'wine64')
home.join("Applications/CrossOver.app/Contents/SharedSupport/CrossOver/wine/bin/wine"),
// macOS: CrossOver
home.join("Applications/CrossOver.app/Contents/SharedSupport/CrossOver/wine/bin/wine64"),
// macOS: Homebrew Apple Silicon (prefer 'wine', fall back to 'wine64')
PathBuf::from("/opt/homebrew/bin/wine"),
// macOS: Homebrew (Apple Silicon)
PathBuf::from("/opt/homebrew/bin/wine64"),
// macOS: Homebrew Intel
PathBuf::from("/usr/local/bin/wine"),
PathBuf::from("/opt/homebrew/bin/wine"),
// macOS: Homebrew (Intel)
PathBuf::from("/usr/local/bin/wine64"),
PathBuf::from("/usr/local/bin/wine"),
// Linux
PathBuf::from("/usr/bin/wine"),
PathBuf::from("/usr/bin/wine64"),
PathBuf::from("/usr/bin/wine"),
];
candidates.iter()
.find(|p| p.exists())
@@ -408,7 +404,6 @@ impl Config {
reports_dir: map.get("reports_dir").cloned(),
backtest_login: map.get("backtest_login").cloned(),
backtest_server: map.get("backtest_server").cloned(),
backtest_password: map.get("backtest_password").cloned(),
project_dir: map.get("project_dir").cloned(),
})
}
@@ -435,7 +430,6 @@ impl Config {
"reports_dir" => self.reports_dir.clone().unwrap_or_else(|| "reports".to_string()),
"backtest_login" => self.backtest_login.clone().unwrap_or_default(),
"backtest_server" => self.backtest_server.clone().unwrap_or_default(),
"backtest_password" => self.backtest_password.clone().unwrap_or_default(),
"project_dir" => self.project_dir.clone().unwrap_or_default(),
_ => String::new(),
}
@@ -510,50 +504,29 @@ impl Config {
self.terminal_dir.as_ref().map(|d| Path::new(d).to_path_buf())
}
/// Scan the tester's own history store for symbols with downloaded data.
///
/// MT5 maintains two separate history trees:
/// • `Bases/{server}/history/` — live-trading tick/bar data (NOT usable by tester)
/// • `Tester/bases/{server}/history/` — data the Strategy Tester actually reads
///
/// Scanning `Bases/` (the old approach) returned symbols that exist for live trading
/// but may have no tester data, causing the tester to fail with "symbol does not exist".
/// This function scans `Tester/bases/` instead, which is the authoritative source.
///
/// Falls back to `Bases/` only when `Tester/bases/` is absent (first-run / no backtests yet).
///
/// If `server_filter` is provided only that server's directory is scanned.
/// Scan Bases/*/history/ for symbol directories that contain at least one .hcc file.
/// Returns deduplicated, sorted list of symbol names available for backtesting.
/// If server_filter is provided, only scans that specific server's directory.
pub fn discover_symbols(&self, server_filter: Option<&str>) -> Vec<String> {
let mt5_dir = match self.mt5_dir() {
Some(d) => d,
None => return Vec::new(),
};
// Prefer the tester's own data store; fall back to live-trading Bases/ when absent.
let tester_bases = mt5_dir.join("Tester").join("bases");
let bases_dir = if tester_bases.is_dir() {
tester_bases
} else {
let fallback = mt5_dir.join("Bases");
if !fallback.is_dir() {
return Vec::new();
}
tracing::warn!(
"Tester/bases/ not found — falling back to Bases/ for symbol discovery. \
Run at least one backtest to populate tester data."
);
fallback
};
let bases_dir = mt5_dir.join("Bases");
if !bases_dir.is_dir() {
return Vec::new();
}
let mut symbols = std::collections::HashSet::new();
// {bases_dir}/{server}/history/{symbol}/ — directory presence = data available
// (the tester uses .hst/.hcc files; existence of the directory is sufficient)
// Bases/{server}/history/{symbol}/{year}.hcc
if let Ok(servers) = fs::read_dir(&bases_dir) {
for server in servers.filter_map(|e| e.ok()) {
let server_name_os = server.file_name();
let server_name = server_name_os.to_str().unwrap_or("");
// Skip non-directory entries and filter by server if specified
if server_name.is_empty() {
continue;
}
@@ -562,7 +535,7 @@ impl Config {
continue;
}
}
let history_dir = server.path().join("history");
if !history_dir.is_dir() {
continue;
@@ -573,8 +546,23 @@ impl Config {
if !sym_path.is_dir() {
continue;
}
if let Some(name) = sym_path.file_name().and_then(|n| n.to_str()) {
symbols.insert(name.to_string());
// Only include if at least one .hcc file exists (has downloaded data)
let has_data = fs::read_dir(&sym_path)
.ok()
.map(|entries| {
entries.filter_map(|e| e.ok()).any(|e| {
e.path().extension()
.and_then(|x| x.to_str())
.map(|x| x == "hcc")
.unwrap_or(false)
})
})
.unwrap_or(false);
if has_data {
if let Some(name) = sym_path.file_name().and_then(|n| n.to_str()) {
symbols.insert(name.to_string());
}
}
}
}
@@ -586,59 +574,6 @@ impl Config {
sorted
}
/// Find the closest available tester symbol to the one requested.
///
/// Matching priority (first hit wins):
/// 1. Exact match → `XAUUSD.cent` == `XAUUSD.cent`
/// 2. Case-insensitive exact match → `xauusd.cent` → `XAUUSD.cent`
/// 3. Strip/add common cent suffixes → `XAUUSDc` ↔ `XAUUSD.cent`
/// 4. Prefix match on the base ticker → `XAUUSD` matches `XAUUSD.cent`
pub fn resolve_symbol<'a>(requested: &str, available: &'a [String]) -> Option<&'a str> {
if available.is_empty() {
return None;
}
// 1. Exact
if let Some(s) = available.iter().find(|s| s.as_str() == requested) {
return Some(s.as_str());
}
// 2. Case-insensitive exact
let req_lower = requested.to_lowercase();
if let Some(s) = available.iter().find(|s| s.to_lowercase() == req_lower) {
return Some(s.as_str());
}
// 3. Cent-suffix normalisation: build a normalised "base" for both sides
// Strip known cent suffixes: `.cent`, `c` (trailing, uppercase only), `.c`
fn base_ticker(sym: &str) -> &str {
let s = sym.trim_end_matches(".cent")
.trim_end_matches(".c");
// Strip trailing lowercase 'c' only when the rest is all-uppercase
// (so "XAUUSDc" → "XAUUSD", but "Misc" stays "Misc")
if s.ends_with('c') && s[..s.len()-1].chars().all(|c| c.is_ascii_uppercase() || c.is_ascii_digit()) {
&s[..s.len()-1]
} else {
s
}
}
let req_base = base_ticker(requested).to_lowercase();
if let Some(s) = available.iter().find(|s| base_ticker(s).to_lowercase() == req_base) {
return Some(s.as_str());
}
// 4. Prefix match: available symbol starts with the requested string (or vice-versa)
if let Some(s) = available.iter().find(|s| {
let sl = s.to_lowercase();
sl.starts_with(&req_lower) || req_lower.starts_with(sl.as_str())
}) {
return Some(s.as_str());
}
None
}
/// Get the currently active MT5 account from common.ini
pub fn current_account(&self) -> Option<CurrentAccount> {
self.mt5_dir().and_then(|d| CurrentAccount::from_common_ini(&d))
+4 -4
View File
@@ -14,6 +14,8 @@ pub struct Report {
pub from_date: String,
pub to_date: String,
pub metrics_file: PathBuf,
pub deals_csv: PathBuf,
pub deals_json: PathBuf,
pub analysis_file: Option<PathBuf>,
}
@@ -40,6 +42,8 @@ pub struct PipelineMetadata {
pub struct FilePaths {
pub metrics: String,
pub analysis: String,
pub deals_csv: String,
pub deals_json: String,
}
#[allow(dead_code)]
@@ -79,9 +83,6 @@ pub struct BacktestJob {
pub mt5_pid: Option<u32>,
pub expected_report_path: String,
pub timeout_seconds: u64,
/// Set by background monitor: "running"|"completed"|"completed_no_html"|"failed"|"timeout"
#[serde(default)]
pub status: Option<String>,
}
impl BacktestJob {
@@ -104,7 +105,6 @@ impl BacktestJob {
mt5_pid: None,
expected_report_path,
timeout_seconds,
status: Some("running".to_string()),
}
}
}
+99 -298
View File
@@ -5,28 +5,6 @@ use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use crate::models::Config;
use crate::optimization::OptimizationParser;
/// Read a file that may be UTF-16LE (with BOM) or UTF-8, returning a UTF-8 String.
/// MT5 .set and .ini files are typically UTF-16LE with BOM (0xFF 0xFE).
fn read_file_as_utf8(path: &Path) -> Result<String> {
let bytes = fs::read(path)?;
// Check for UTF-16LE BOM (0xFF 0xFE)
if bytes.len() >= 2 && bytes[0] == 0xFF && bytes[1] == 0xFE {
// UTF-16LE with BOM - skip the 2-byte BOM and decode
let utf16_data: Vec<u16> = bytes[2..]
.chunks_exact(2)
.map(|chunk| u16::from_le_bytes([chunk[0], chunk[1]]))
.collect();
String::from_utf16(&utf16_data)
.map_err(|e| anyhow!("Failed to decode UTF-16LE: {}", e))
} else {
// Try UTF-8
String::from_utf8(bytes)
.map_err(|e| anyhow!("Failed to decode as UTF-8: {}", e))
}
}
pub struct OptimizationParams {
pub expert: String,
@@ -35,9 +13,9 @@ pub struct OptimizationParams {
pub from_date: String,
pub to_date: String,
pub deposit: u32,
pub model: u8,
pub leverage: u32,
pub currency: String,
pub max_passes: Option<u32>,
}
impl Default for OptimizationParams {
@@ -49,9 +27,9 @@ impl Default for OptimizationParams {
from_date: String::new(),
to_date: String::new(),
deposit: 10000,
model: 0,
leverage: 500,
currency: "USD".to_string(),
max_passes: None,
}
}
}
@@ -94,20 +72,13 @@ impl OptimizationRunner {
return Err(anyhow!("Set file not found: {}", params.set_file));
}
// Kill any existing MT5/agent processes to avoid stale zombies
for pat in &["terminal64\\.exe", "metatester64\\.exe"] {
let _ = Command::new("pkill").args(["-KILL", "-f", pat]).output();
}
std::thread::sleep(std::time::Duration::from_secs(3));
// Generate job ID and log file
let timestamp = Utc::now().format("%Y%m%d_%H%M%S").to_string();
let job_id = format!("opt_{}", timestamp);
let log_file = PathBuf::from(format!("/tmp/mt5opt_{}.log", timestamp));
// Count combinations
let combinations = self.count_combinations(&params.set_file)
.map_err(|e| anyhow!("count_combinations failed: {}", e))?;
let combinations = self.count_combinations(&params.set_file)?;
// Get paths
let mt5_dir = self.config.terminal_dir.as_ref()
@@ -118,159 +89,50 @@ impl OptimizationRunner {
// Write .set file as UTF-16LE with BOM directly to MT5 tester directory
let wine_prefix_dir = self.get_wine_prefix_dir(mt5_dir)?;
let tester_dir = wine_prefix_dir.join("drive_c/Program Files/MetaTrader 5/MQL5/Profiles/Tester");
fs::create_dir_all(&tester_dir).map_err(|e| anyhow!("create_dir_all({}) failed: {}", tester_dir.display(), e))?;
fs::create_dir_all(&tester_dir)?;
let dst_set_file = tester_dir.join(format!("{}.set", params.expert));
self.write_utf16le_set(&params.set_file, &dst_set_file)
.map_err(|e| anyhow!("write_utf16le_set({}) failed: {}", dst_set_file.display(), e))?;
self.write_utf16le_set(&params.set_file, &dst_set_file)?;
// Reset OptMode in terminal.ini
// Patch terminal.ini [Tester] section with optimization params (primary mechanism)
let terminal_ini = if Path::new(mt5_dir).join("config").exists() {
Path::new(mt5_dir).join("config").join("terminal.ini")
} else {
Path::new(mt5_dir).join("terminal.ini")
};
let mt5_ini_text = if terminal_ini.exists() {
read_file_as_utf8(&terminal_ini).unwrap_or_default()
} else {
String::new()
};
let expert_path = if let Some(experts_dir) = &self.config.experts_dir {
let nested = Path::new(experts_dir).join(&params.expert).join(format!("{}.mq5", params.expert));
if nested.exists() {
format!("Experts\\{}\\{}.ex5", params.expert, params.expert)
} else {
format!("Experts\\{}.ex5", params.expert)
}
} else {
format!("Experts\\{}.ex5", params.expert)
};
let set_param = {
let base = if !params.set_file.is_empty() && params.set_file != format!("{}.set", params.expert) {
// Extract just the filename from the set file path
std::path::Path::new(&params.set_file).file_name()
.and_then(|n| n.to_str())
.unwrap_or(&format!("{}.set", params.expert))
.to_string()
} else {
format!("{}.set", params.expert)
};
base
};
let mut tester_section = format!(
"[Tester]\n\
Expert={}\n\
ExpertParameters={}\n\
Symbol={}\n\
Period=M1\n\
Model=1\n\
FromDate={}\n\
ToDate={}\n\
ForwardMode=0\n\
Deposit={}\n\
Currency={}\n\
ProfitInPips=0\n\
Leverage={}\n\
Execution=10\n\
Optimization=2\n\
Visual=0\n\
Report=..\\..\\mt5mcp_opt_report.htm\n\
ReplaceReport=1\n\
ShutdownTerminal=1",
expert_path, set_param, params.symbol,
params.from_date, params.to_date, params.deposit, params.currency, params.leverage,
);
if let Some(mp) = params.max_passes {
tester_section.push_str(&format!("\nMaxPass={}", mp));
}
let updated_ini = Self::patch_ini_section(&mt5_ini_text, "Tester", &tester_section);
// Strip any stale [Agents] sections from previous runs (no local agent processes)
let cleaned = Self::strip_ini_section(&updated_ini, "Agents");
let final_ini = cleaned.trim_end().to_string();
let mut utf16_out: Vec<u8> = vec![0xFF, 0xFE];
utf16_out.extend(final_ini.encode_utf16().flat_map(|c| c.to_le_bytes()));
fs::write(&terminal_ini, utf16_out)?;
self.reset_optmode(mt5_dir)?;
// Write /config: INI to trigger tester/optimizer mode
// For /config: format, Expert path is relative to MQL5/Experts/ (no Experts\ prefix)
let opt_config_win = r"C:\mt5opt_config.ini";
let opt_config_host = wine_prefix_dir.join("drive_c").join("mt5opt_config.ini");
let mut opt_ini = String::new();
if let Some(login) = &self.config.backtest_login {
if let Some(server) = &self.config.backtest_server {
opt_ini.push_str("[Common]\n");
opt_ini.push_str(&format!("Login={}\n", login));
opt_ini.push_str(&format!("Server={}\n", server));
if let Some(password) = &self.config.backtest_password {
opt_ini.push_str(&format!("Password={}\n", password));
}
opt_ini.push_str("\n");
}
}
opt_ini.push_str("[Tester]\n");
opt_ini.push_str(&format!("Expert={}.ex5\n", params.expert));
opt_ini.push_str(&format!("ExpertParameters={}\n", set_param));
opt_ini.push_str(&format!("Symbol={}\n", params.symbol));
opt_ini.push_str("Period=M1\n");
opt_ini.push_str("Model=1\n");
opt_ini.push_str("Optimization=2\n");
opt_ini.push_str(&format!("FromDate={}\n", params.from_date));
opt_ini.push_str(&format!("ToDate={}\n", params.to_date));
opt_ini.push_str("ForwardMode=0\n");
opt_ini.push_str(&format!("Deposit={}\n", params.deposit));
opt_ini.push_str(&format!("Currency={}\n", params.currency));
opt_ini.push_str("ProfitInPips=0\n");
opt_ini.push_str(&format!("Leverage={}\n", params.leverage));
opt_ini.push_str("Execution=10\n");
opt_ini.push_str("Visual=0\n");
opt_ini.push_str("Report=..\\..\\mt5mcp_opt_report.htm\n");
opt_ini.push_str("ReplaceReport=1\n");
opt_ini.push_str("ShutdownTerminal=1\n");
if let Some(mp) = params.max_passes {
opt_ini.push_str(&format!("MaxPass={}\n", mp));
}
fs::write(&opt_config_host, opt_ini.as_bytes())?;
// Get Wine prefix directory
let wine_prefix_dir = self.get_wine_prefix_dir(mt5_dir)?;
// Build launch script (macOS-compatible with /config: to trigger tester mode)
let wine_bin = Path::new(wine_exe);
let wine_root = wine_bin
.parent()
.and_then(|p| p.parent())
.ok_or_else(|| anyhow!("Cannot derive Wine root from wine_exe"))?;
let ext_libs = wine_root.join("lib").join("external");
let wine_libs = wine_root.join("lib");
let dyld = format!("{}:{}:/usr/lib:/usr/local/lib",
ext_libs.display(), wine_libs.display());
let terminal_host = wine_prefix_dir.join("drive_c")
.join("Program Files").join("MetaTrader 5").join("terminal64.exe");
// Build optimization INI
let ini_path = wine_prefix_dir.join("drive_c/mt5mcp_backtest.ini");
let ini_content = format!(r#"[Tester]
Expert={}
Symbol={}
Period=M5
Deposit={}
Currency={}
Leverage={}
Model={}
FromDate={}
ToDate={}
Report=C:\mt5mcp_opt_report
Optimization=2
ExpertParameters={}.set
ShutdownTerminal=1
"#, params.expert, params.symbol, params.deposit, params.currency,
params.leverage, params.model, params.from_date, params.to_date, params.expert);
fs::write(&ini_path, ini_content)?;
let script = format!(
"#!/bin/sh\n\
export DYLD_FALLBACK_LIBRARY_PATH='{dyld}'\n\
export WINEPREFIX='{prefix}'\n\
export WINEDEBUG='-all'\n\
nohup '{wine}' '{terminal}' '/config:{config}' >/dev/null 2>&1 &\n",
dyld = dyld,
prefix = wine_prefix_dir.display(),
wine = wine_exe,
terminal = terminal_host.display(),
config = opt_config_win,
);
// Build batch file
let batch_path = wine_prefix_dir.join("drive_c/mt5mcp_run.bat");
let batch_content = format!(r#"@echo off
"C:\Program Files\MetaTrader 5\terminal64.exe" /config:C:\mt5mcp_backtest.ini
"#);
fs::write(&batch_path, batch_content)?;
let script_path = std::env::temp_dir().join("mt5opt_launch.sh");
fs::write(&script_path, &script)?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(&script_path, fs::Permissions::from_mode(0o755))?;
}
let child = Command::new("/bin/sh")
.arg(&script_path)
// Launch detached process
let cmd = format!("cmd.exe /c 'C:\\mt5mcp_run.bat'");
let child = Command::new(wine_exe)
.arg(&cmd)
.stdout(Stdio::null())
.stderr(Stdio::null())
.spawn()
.map_err(|e| anyhow!("spawn /bin/sh {} failed: {}", script_path.display(), e))?;
.spawn()?;
let pid = child.id();
@@ -288,7 +150,7 @@ impl OptimizationRunner {
}
fn count_combinations(&self, set_file: &str) -> Result<u64> {
let content = read_file_as_utf8(Path::new(set_file))?;
let content = fs::read_to_string(set_file)?;
let mut total: u64 = 1;
for line in content.lines() {
@@ -317,23 +179,13 @@ impl OptimizationRunner {
}
fn write_utf16le_set(&self, src: &str, dst: &Path) -> Result<()> {
let content = read_file_as_utf8(Path::new(src))?;
let content = fs::read_to_string(src)?;
// Create parent directory if needed
if let Some(parent) = dst.parent() {
fs::create_dir_all(parent)?;
}
// Remove existing file if read-only from previous run
if dst.exists() {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = fs::set_permissions(dst, fs::Permissions::from_mode(0o644));
}
let _ = fs::remove_file(dst);
}
// Write UTF-16LE with BOM
let mut utf16_content: Vec<u16> = vec![0xFEFF]; // BOM
utf16_content.extend(content.encode_utf16());
@@ -343,18 +195,50 @@ impl OptimizationRunner {
.collect();
fs::write(dst, bytes)?;
// Make read-only
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(dst, fs::Permissions::from_mode(0o444))?;
}
Ok(())
}
fn reset_optmode(&self, mt5_dir: &str) -> Result<()> {
let terminal_ini = Path::new(mt5_dir).join("terminal.ini");
if !terminal_ini.exists() {
return Ok(());
}
let content = fs::read_to_string(&terminal_ini)?;
let updated = content
.lines()
.map(|line| {
if line.starts_with("OptMode=") {
"OptMode=0".to_string()
} else if line.starts_with("LastOptimization=") {
String::new()
} else {
line.to_string()
}
})
.filter(|l| !l.is_empty())
.collect::<Vec<_>>()
.join("\n");
fs::write(&terminal_ini, updated)?;
Ok(())
}
fn get_wine_prefix_dir(&self, mt5_dir: &str) -> Result<PathBuf> {
let path = Path::new(mt5_dir);
// Go up three levels: .../drive_c/Program Files/MetaTrader 5 -> .../net.metaquotes.wine.metatrader5
// (same as backtest pipeline)
// Go up two levels: .../drive_c/Program Files/MetaTrader 5 -> .../drive_c
let prefix_dir = path
.parent()
.and_then(|p| p.parent())
.and_then(|p| p.parent())
.ok_or_else(|| anyhow!("Cannot determine Wine prefix from terminal_dir"))?;
Ok(prefix_dir.to_path_buf())
}
@@ -373,7 +257,6 @@ impl OptimizationRunner {
let meta_path = jobs_dir.join(format!("{}.json", job_id));
let started_at = Utc::now().to_rfc3339();
let report_path = wine_prefix.join("drive_c").join("mt5mcp_opt_report");
let metadata = serde_json::json!({
"job_id": job_id,
@@ -386,7 +269,6 @@ impl OptimizationRunner {
"combinations": combinations,
"log_file": log_file.to_string_lossy(),
"wine_prefix": wine_prefix.to_string_lossy(),
"report_path": report_path.to_string_lossy(),
"started_at": started_at,
});
@@ -394,73 +276,6 @@ impl OptimizationRunner {
Ok(())
}
/// Replace a [section] in an INI string — removes old content and inserts new.
fn patch_ini_section(text: &str, section: &str, new_content: &str) -> String {
let section_header = format!("[{}]", section);
let mut result = String::new();
let mut in_section = false;
let mut section_found = false;
for line in text.lines() {
let trimmed = line.trim();
if trimmed == section_header {
in_section = true;
section_found = true;
continue;
}
if in_section {
if trimmed.starts_with('[') {
in_section = false;
result.push_str(new_content);
if !new_content.ends_with('\n') {
result.push('\n');
}
result.push_str(line);
result.push('\n');
continue;
}
continue;
}
result.push_str(line);
result.push('\n');
}
if !section_found {
if !result.is_empty() && !result.ends_with('\n') {
result.push('\n');
}
result.push_str(new_content);
result.push('\n');
} else if in_section {
result.push_str(new_content);
result.push('\n');
}
result
}
/// Remove all lines belonging to a [section] from the INI text.
fn strip_ini_section(text: &str, section: &str) -> String {
let header = format!("[{}]", section);
let mut result = String::new();
let mut skipping = false;
for line in text.lines() {
let trimmed = line.trim();
if trimmed == header {
skipping = true;
continue;
}
if skipping && trimmed.starts_with('[') {
skipping = false;
}
if !skipping {
result.push_str(line);
result.push('\n');
}
}
result
}
pub fn get_job_status(&self, job_id: &str) -> Result<serde_json::Value> {
let jobs_dir = Path::new(".mt5mcp_jobs");
let meta_path = jobs_dir.join(format!("{}.json", job_id));
@@ -474,51 +289,37 @@ impl OptimizationRunner {
let meta: serde_json::Value = serde_json::from_str(&fs::read_to_string(&meta_path)?)?;
let pid = meta.get("pid").and_then(|v| v.as_u64()).unwrap_or(0) as u32;
// Check if process is still running
let is_running = self.is_process_running(pid);
let mut result = serde_json::json!({
"status": if is_running { "running" } else { "stopped" },
// Check for completion marker in log
let log_file = meta.get("log_file").and_then(|v| v.as_str()).unwrap_or("");
let is_complete = if !log_file.is_empty() && Path::new(log_file).exists() {
fs::read_to_string(log_file)
.map(|content| content.contains("Optimization complete"))
.unwrap_or(false)
} else {
false
};
let status = if is_complete {
"completed"
} else if is_running {
"running"
} else {
"stopped"
};
Ok(serde_json::json!({
"status": status,
"job_id": job_id,
"pid": pid,
"expert": meta.get("expert"),
"symbol": meta.get("symbol"),
"from_date": meta.get("from_date"),
"to_date": meta.get("to_date"),
"started_at": meta.get("started_at"),
});
// If not running, try to parse the optimization report
if !is_running {
let parser = OptimizationParser::new();
match parser.parse_job(job_id) {
Ok(passes) if !passes.is_empty() => {
let mut sorted_by_pf = passes.clone();
sorted_by_pf.sort_by(|a, b| b.profit_factor.partial_cmp(&a.profit_factor).unwrap());
let top10: Vec<_> = sorted_by_pf.into_iter().take(10).collect();
let best_pf = parser.find_best_pass(&passes, "profit_factor");
let best_profit = parser.find_best_pass(&passes, "profit");
let m = result.as_object_mut()
.ok_or_else(|| anyhow!("result is not object"))?;
m.insert("status".into(), serde_json::Value::String("completed".into()));
m.insert("total_passes".into(), serde_json::json!(passes.len()));
m.insert("top_10".into(), serde_json::to_value(&top10).unwrap_or_default());
m.insert("best_pf".into(), serde_json::to_value(best_pf).unwrap_or_default());
m.insert("best_profit".into(), serde_json::to_value(best_profit).unwrap_or_default());
}
_ => {
let m = result.as_object_mut()
.ok_or_else(|| anyhow!("result is not object"))?;
m.insert("status".into(), serde_json::Value::String("stopped".into()));
m.insert("message".into(), serde_json::Value::String(
"Optimization stopped but no report found — may have crashed or was killed early".into()
));
}
}
}
Ok(result)
"log_file": log_file,
}))
}
fn is_process_running(&self, pid: u32) -> bool {
+3 -20
View File
@@ -31,23 +31,6 @@ impl OptimizationParser {
}
let meta: serde_json::Value = serde_json::from_str(&fs::read_to_string(&meta_path)?)?;
// Check if report_path is stored in metadata
if let Some(report_base) = meta.get("report_path").and_then(|v| v.as_str()) {
let base_path = Path::new(report_base);
for ext in &[".htm", ".htm.xml", ".html"] {
let candidate = base_path.with_extension(ext.trim_start_matches('.'));
if candidate.exists() {
return self.parse_file(&candidate);
}
}
return Err(anyhow!(
"Optimization report not found at {}.*. Is MT5 optimization still running?",
base_path.display()
));
}
// Fallback: derive from wine_prefix (legacy)
let wine_prefix = meta.get("wine_prefix")
.and_then(|v| v.as_str())
.ok_or_else(|| anyhow!("wine_prefix not in job metadata"))?;
@@ -186,13 +169,13 @@ impl OptimizationParser {
// Find all rows in Worksheet/Table
for node in doc.descendants() {
if node.has_tag_name(("urn:schemas-microsoft-com:office:spreadsheet", "Row")) ||
if node.has_tag_name(("http://schemas.microsoft.com/office/excel/2003/xml", "Row")) ||
node.has_tag_name("Row") {
let cells: Vec<String> = node.children()
.filter(|n: &roxmltree::Node<'_, '_>| {
n.has_tag_name(("urn:schemas-microsoft-com:office:spreadsheet", "Cell")) ||
n.has_tag_name(("http://schemas.microsoft.com/office/excel/2003/xml", "Cell")) ||
n.has_tag_name("Cell") ||
n.has_tag_name(("urn:schemas-microsoft-com:office:spreadsheet", "Data")) ||
n.has_tag_name(("http://schemas.microsoft.com/office/excel/2003/xml", "Data")) ||
n.has_tag_name("Data")
})
.map(|n| n.text().unwrap_or("").trim().to_string().replace(',', ""))
+83 -801
View File
File diff suppressed because it is too large Load Diff
+22 -168
View File
@@ -3,8 +3,6 @@ use rusqlite::{params, Connection};
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use crate::models::Deal;
#[derive(Debug, Serialize, Deserialize, Clone, Default)]
pub struct ReportEntry {
pub id: String,
@@ -69,25 +67,6 @@ impl ReportDb {
}
let conn = self.connect()?;
conn.execute_batch("
CREATE TABLE IF NOT EXISTS deals (
id INTEGER PRIMARY KEY AUTOINCREMENT,
report_id TEXT NOT NULL REFERENCES reports(id) ON DELETE CASCADE,
time TEXT NOT NULL,
deal TEXT NOT NULL,
symbol TEXT NOT NULL,
deal_type TEXT NOT NULL,
entry TEXT NOT NULL,
volume REAL NOT NULL,
price REAL NOT NULL,
order_id TEXT NOT NULL,
commission REAL NOT NULL,
swap REAL NOT NULL,
profit REAL NOT NULL,
balance REAL NOT NULL,
comment TEXT NOT NULL,
magic TEXT
);
CREATE INDEX IF NOT EXISTS idx_deals_report_id ON deals(report_id);
CREATE TABLE IF NOT EXISTS reports (
id TEXT PRIMARY KEY,
expert TEXT NOT NULL,
@@ -123,69 +102,6 @@ impl ReportDb {
Ok(())
}
pub fn insert_deals(&self, report_id: &str, deals: &[Deal]) -> Result<()> {
if deals.is_empty() {
return Ok(());
}
let conn = self.connect()?;
let mut stmt = conn.prepare(
"INSERT INTO deals (report_id, time, deal, symbol, deal_type, entry, volume, price, \
order_id, commission, swap, profit, balance, comment, magic) \
VALUES (?1,?2,?3,?4,?5,?6,?7,?8,?9,?10,?11,?12,?13,?14,?15)",
)?;
for d in deals {
stmt.execute(params![
report_id,
d.time,
d.deal,
d.symbol,
d.deal_type,
d.entry,
d.volume,
d.price,
d.order,
d.commission,
d.swap,
d.profit,
d.balance,
d.comment,
d.magic,
])?;
}
Ok(())
}
pub fn get_deals(&self, report_id: &str) -> Result<Vec<Deal>> {
let conn = self.connect()?;
let mut stmt = conn.prepare(
"SELECT time, deal, symbol, deal_type, entry, volume, price, order_id, \
commission, swap, profit, balance, comment, magic \
FROM deals WHERE report_id = ? ORDER BY id ASC",
)?;
let deals: Vec<Deal> = stmt
.query_map([report_id], |row| {
Ok(Deal {
time: row.get(0)?,
deal: row.get(1)?,
symbol: row.get(2)?,
deal_type: row.get(3)?,
entry: row.get(4)?,
volume: row.get(5)?,
price: row.get(6)?,
order: row.get(7)?,
commission: row.get(8)?,
swap: row.get(9)?,
profit: row.get(10)?,
balance: row.get(11)?,
comment: row.get(12)?,
magic: row.get(13)?,
})
})?
.filter_map(|r| r.ok())
.collect();
Ok(deals)
}
pub fn insert(&self, entry: &ReportEntry) -> Result<()> {
let conn = self.connect()?;
let tags_json = serde_json::to_string(&entry.tags)?;
@@ -459,57 +375,6 @@ impl ReportDb {
Ok(entry)
}
/// Get a specific report by its report_dir path (exact match)
pub fn get_by_report_dir(&self, report_dir: &str) -> Result<Option<ReportEntry>> {
let conn = self.connect()?;
let mut stmt = conn.prepare(
"SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
created_at, set_file_original, set_snapshot_path, report_dir, charts_dir, \
net_profit, profit_factor, max_dd_pct, sharpe_ratio, total_trades, \
win_rate_pct, recovery_factor, deposit, currency, leverage, \
duration_seconds, tags, notes, verdict \
FROM reports WHERE report_dir = ?"
)?;
let entry = stmt
.query_map([report_dir], |row| {
let tags_json: String = row.get(23)?;
let tags: Vec<String> = serde_json::from_str(&tags_json).unwrap_or_default();
Ok(ReportEntry {
id: row.get(0)?,
expert: row.get(1)?,
symbol: row.get(2)?,
timeframe: row.get(3)?,
model: row.get(4)?,
from_date: row.get(5)?,
to_date: row.get(6)?,
created_at: row.get(7)?,
set_file_original: row.get(8)?,
set_snapshot_path: row.get(9)?,
report_dir: row.get(10)?,
charts_dir: row.get(11)?,
net_profit: row.get(12)?,
profit_factor: row.get(13)?,
max_dd_pct: row.get(14)?,
sharpe_ratio: row.get(15)?,
total_trades: row.get(16)?,
win_rate_pct: row.get(17)?,
recovery_factor: row.get(18)?,
deposit: row.get(19)?,
currency: row.get(20)?,
leverage: row.get(21)?,
duration_seconds: row.get(22)?,
tags,
notes: row.get(24)?,
verdict: row.get(25)?,
})
})?
.filter_map(|r| r.ok())
.next();
Ok(entry)
}
/// Get a specific report by ID
pub fn get_by_id(&self, id: &str) -> Result<Option<ReportEntry>> {
let conn = self.connect()?;
@@ -564,39 +429,26 @@ impl ReportDb {
/// Search reports by tags (at least one tag must match)
pub fn search_by_tags(&self, tags: &[String], limit: usize) -> Result<Vec<ReportEntry>> {
let conn = self.connect()?;
let base_sql = "SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
let mut sql = "SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
created_at, set_file_original, set_snapshot_path, report_dir, charts_dir, \
net_profit, profit_factor, max_dd_pct, sharpe_ratio, total_trades, \
win_rate_pct, recovery_factor, deposit, currency, leverage, \
duration_seconds, tags, notes, verdict \
FROM reports WHERE 1=1";
FROM reports WHERE 1=1"
.to_string();
let (sql, params): (String, Vec<rusqlite::types::Value>) = if tags.is_empty() {
(
format!("{} ORDER BY created_at DESC LIMIT ?1", base_sql),
vec![(limit as i64).into()],
)
} else {
let placeholders = tags.iter().enumerate()
.map(|(i, _)| format!("tags LIKE ?{}", i + 1))
.collect::<Vec<_>>()
.join(" OR ");
let sql = format!(
"{} AND ({}) ORDER BY created_at DESC LIMIT ?{}",
base_sql,
placeholders,
tags.len() + 1
);
let mut p: Vec<rusqlite::types::Value> = tags.iter()
.map(|t| format!("%{}%", t).into())
// Build OR conditions for tags - use JSON1 extension for tag matching
if !tags.is_empty() {
let tag_conditions: Vec<String> = tags.iter()
.map(|tag| format!("tags LIKE '%{}%'", tag.replace("'", "''")))
.collect();
p.push((limit as i64).into());
(sql, p)
};
sql.push_str(&format!(" AND ({})", tag_conditions.join(" OR ")));
}
sql.push_str(&format!(" ORDER BY created_at DESC LIMIT {}", limit));
let mut stmt = conn.prepare(&sql)?;
let entries: Vec<ReportEntry> = stmt
.query_map(rusqlite::params_from_iter(params.iter()), |row| {
.query_map([], |row| {
let tags_str: String = row.get(23).unwrap_or_else(|_| "[]".to_string());
let tags: Vec<String> = serde_json::from_str(&tags_str).unwrap_or_default();
Ok(ReportEntry {
@@ -637,18 +489,19 @@ impl ReportDb {
/// Search reports by notes text (case-insensitive LIKE)
pub fn search_by_notes(&self, query: &str, limit: usize) -> Result<Vec<ReportEntry>> {
let conn = self.connect()?;
let pattern = format!("%{}%", query);
let pattern = format!("%{}%", query.replace("'", "''"));
let mut stmt = conn.prepare(
"SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
&format!("SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
created_at, set_file_original, set_snapshot_path, report_dir, charts_dir, \
net_profit, profit_factor, max_dd_pct, sharpe_ratio, total_trades, \
win_rate_pct, recovery_factor, deposit, currency, leverage, \
duration_seconds, tags, notes, verdict \
FROM reports WHERE notes LIKE ?1 ORDER BY created_at DESC LIMIT ?2"
FROM reports WHERE notes LIKE '{}' ORDER BY created_at DESC LIMIT {}",
pattern, limit)
)?;
let entries: Vec<ReportEntry> = stmt
.query_map(rusqlite::params![pattern, limit as i64], |row| {
.query_map([], |row| {
let tags_str: String = row.get(23).unwrap_or_else(|_| "[]".to_string());
let tags: Vec<String> = serde_json::from_str(&tags_str).unwrap_or_default();
Ok(ReportEntry {
@@ -689,19 +542,20 @@ impl ReportDb {
/// Find reports by set file (original or snapshot)
pub fn search_by_set_file(&self, set_file: &str, limit: usize) -> Result<Vec<ReportEntry>> {
let conn = self.connect()?;
let pattern = format!("%{}%", set_file);
let pattern = format!("%{}%", set_file.replace("'", "''"));
let mut stmt = conn.prepare(
"SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
&format!("SELECT id, expert, symbol, timeframe, model, from_date, to_date, \
created_at, set_file_original, set_snapshot_path, report_dir, charts_dir, \
net_profit, profit_factor, max_dd_pct, sharpe_ratio, total_trades, \
win_rate_pct, recovery_factor, deposit, currency, leverage, \
duration_seconds, tags, notes, verdict \
FROM reports WHERE (set_file_original LIKE ?1 OR set_snapshot_path LIKE ?1) \
ORDER BY created_at DESC LIMIT ?2"
FROM reports WHERE (set_file_original LIKE '{}' OR set_snapshot_path LIKE '{}') \
ORDER BY created_at DESC LIMIT {}",
pattern, pattern, limit)
)?;
let entries: Vec<ReportEntry> = stmt
.query_map(rusqlite::params![pattern, limit as i64], |row| {
.query_map([], |row| {
let tags_str: String = row.get(23).unwrap_or_else(|_| "[]".to_string());
let tags: Vec<String> = serde_json::from_str(&tags_str).unwrap_or_default();
Ok(ReportEntry {
+29 -43
View File
@@ -1,8 +1,5 @@
use serde_json::{json, Value};
// Common description suffix for all analytics tools
const REPORT_HINT: &str = "report_id (preferred), report_dir (legacy path), or omit for latest report.";
pub fn tool_analyze_report() -> Value {
json!({
"name": "analyze_report",
@@ -10,8 +7,7 @@ pub fn tool_analyze_report() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory (use report_id instead)" },
"report_dir": { "type": "string", "description": "Path to report directory containing deals.csv" },
"analytics": {
"type": "array",
"description": "Optional: specific analytics to run. If omitted, runs all.",
@@ -33,8 +29,7 @@ pub fn tool_analyze_monthly_pnl() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string", "description": "Path to report directory containing deals.csv" }
}
}
})
@@ -47,8 +42,7 @@ pub fn tool_analyze_drawdown_events() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -61,8 +55,7 @@ pub fn tool_analyze_top_losses() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" },
"report_dir": { "type": "string" },
"limit": { "type": "integer", "description": "Number of losses to return (default: 10)", "default": 10 }
}
}
@@ -76,8 +69,7 @@ pub fn tool_analyze_loss_sequences() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -90,8 +82,7 @@ pub fn tool_analyze_position_pairs() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -104,8 +95,7 @@ pub fn tool_analyze_direction_bias() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -118,8 +108,7 @@ pub fn tool_analyze_streaks() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -132,8 +121,7 @@ pub fn tool_analyze_concurrent_peak() -> Value {
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -145,9 +133,9 @@ pub fn tool_list_deals() -> Value {
"description": "List individual deals from a backtest report with optional filters",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" },
"report_dir": { "type": "string", "description": "Path to report directory containing deals.csv" },
"deal_type": { "type": "string", "enum": ["buy", "sell"], "description": "Filter by deal type" },
"min_profit": { "type": "number", "description": "Minimum profit (use negative for losses)" },
"max_profit": { "type": "number", "description": "Maximum profit" },
@@ -167,10 +155,9 @@ pub fn tool_search_deals_by_comment() -> Value {
"description": "Search deals by comment text (case-insensitive partial match)",
"inputSchema": {
"type": "object",
"required": ["query"],
"required": ["report_dir", "query"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" },
"report_dir": { "type": "string" },
"query": { "type": "string", "description": "Search text in comments" },
"limit": { "type": "integer", "default": 50 }
}
@@ -184,10 +171,9 @@ pub fn tool_search_deals_by_magic() -> Value {
"description": "Filter deals by magic number (EA identifier)",
"inputSchema": {
"type": "object",
"required": ["magic"],
"required": ["report_dir", "magic"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" },
"report_dir": { "type": "string" },
"magic": { "type": "string", "description": "Magic number to filter by" },
"limit": { "type": "integer", "default": 100 }
}
@@ -201,9 +187,9 @@ pub fn tool_analyze_profit_distribution() -> Value {
"description": "Analyze profit distribution - small/medium/large wins and losses with detailed buckets",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -215,9 +201,9 @@ pub fn tool_analyze_time_performance() -> Value {
"description": "Analyze performance by hour of day and day of week",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -229,9 +215,9 @@ pub fn tool_analyze_hold_time_distribution() -> Value {
"description": "Analyze hold time distribution and correlation with profit",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -243,9 +229,9 @@ pub fn tool_analyze_layer_performance() -> Value {
"description": "Analyze performance by grid layer (extracted from deal comments)",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -257,9 +243,9 @@ pub fn tool_analyze_volume_vs_profit() -> Value {
"description": "Analyze correlation between volume and profit, plus performance by volume bucket",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -271,9 +257,9 @@ pub fn tool_analyze_costs() -> Value {
"description": "Analyze commission and swap costs impact on profitability",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
@@ -285,9 +271,9 @@ pub fn tool_analyze_efficiency() -> Value {
"description": "Calculate efficiency metrics: profit per hour/day, annualized return, trade frequency",
"inputSchema": {
"type": "object",
"required": ["report_dir"],
"properties": {
"report_id": { "type": "string", "description": REPORT_HINT },
"report_dir": { "type": "string", "description": "Legacy: path to report directory" }
"report_dir": { "type": "string" }
}
}
})
+3 -52
View File
@@ -23,8 +23,7 @@ pub fn tool_run_backtest() -> Value {
"shutdown": { "type": "boolean", "description": "Close MT5 after backtest completes" },
"kill_existing": { "type": "boolean", "description": "Kill any running MT5 instance first" },
"timeout": { "type": "integer", "description": "Max wait time in seconds (default: 900)" },
"gui": { "type": "boolean", "description": "Enable MT5 visualization window" },
"startup_delay_secs": { "type": "integer", "description": "Seconds to wait for MT5 initialization (default: 10, set to 0 for default 10s)" }
"gui": { "type": "boolean", "description": "Enable MT5 visualization window" }
}
}
})
@@ -49,8 +48,7 @@ pub fn tool_run_backtest_quick() -> Value {
"deep": { "type": "boolean", "description": "Run deep analysis" },
"shutdown": { "type": "boolean", "description": "Close MT5 after backtest" },
"timeout": { "type": "integer", "description": "Max wait time in seconds (default: 900)" },
"gui": { "type": "boolean", "description": "Enable MT5 visualization" },
"startup_delay_secs": { "type": "integer", "description": "Seconds to wait for MT5 initialization (default: 10)" }
"gui": { "type": "boolean", "description": "Enable MT5 visualization" }
}
}
})
@@ -99,10 +97,7 @@ pub fn tool_launch_backtest() -> Value {
"skip_compile": { "type": "boolean" },
"skip_clean": { "type": "boolean" },
"timeout": { "type": "integer", "description": "Max time in seconds to wait for backtest (default: 900)" },
"shutdown": { "type": "boolean", "description": "Shut down MT5 after test (default: true — required for HTML report to be written)" },
"gui": { "type": "boolean", "description": "Enable visualization during backtest" },
"startup_delay_secs": { "type": "integer", "description": "Seconds to wait for MT5 initialization (default: 10)" },
"inactivity_kill_secs": { "type": "integer", "description": "Kill MT5 if tester log hasn't grown for this many seconds (0 = disabled). Use to abort EAs that stop trading mid-test." }
"gui": { "type": "boolean", "description": "Enable visualization during backtest" }
}
}
})
@@ -121,22 +116,6 @@ pub fn tool_get_backtest_status() -> Value {
})
}
pub fn tool_get_tester_log() -> Value {
json!({
"name": "get_tester_log",
"description": "Read the active MT5 tester agent journal log. Returns parsed deals, final balance, test progress, and raw log tail. Works during a backtest (if the log is being written) or after completion. Use this to inspect what trades occurred, check for EA activity, or debug issues when the HTML report wasn't produced.",
"inputSchema": {
"type": "object",
"properties": {
"tail_lines": {
"type": "integer",
"description": "Number of log tail lines to return (default: 100)"
}
}
}
})
}
pub fn tool_cache_status() -> Value {
json!({
"name": "cache_status",
@@ -147,34 +126,6 @@ pub fn tool_cache_status() -> Value {
})
}
pub fn tool_run_rolling_backtest() -> Value {
json!({
"name": "run_rolling_backtest",
"description": "Rolling backtest: run N consecutive weekly backtests sequentially and return aggregated results. Compiles once, then each week runs with skip_compile. Use this to test EA stability across multiple weeks.",
"inputSchema": {
"type": "object",
"required": ["expert"],
"properties": {
"expert": { "type": "string", "description": "EA name without path or extension" },
"symbol": { "type": "string", "description": "Trading symbol (default: from config or first available)" },
"from_date": { "type": "string", "description": "Start date YYYY.MM.DD (default: auto-calculate N weeks back)" },
"to_date": { "type": "string", "description": "End date YYYY.MM.DD (default: auto-calculate to last Sunday)" },
"timeframe": { "type": "string", "enum": ["M1", "M5", "M15", "M30", "H1", "H4", "D1"], "description": "Chart timeframe (default: M5)" },
"deposit": { "type": "integer", "description": "Initial deposit (default: 10000)" },
"model": { "type": "integer", "enum": [0, 1, 2], "description": "Tick model: 0=Every tick, 1=OHLC, 2=Open prices" },
"set_file": { "type": "string", "description": "Path to .set parameter file for EA inputs" },
"weeks": { "type": "integer", "description": "Number of weekly backtests to run (default: 4)" },
"skip_compile": { "type": "boolean", "description": "Skip initial compilation (default: false — compiles on first week)" },
"shutdown": { "type": "boolean", "description": "Close MT5 after backtest completes (default: true)" },
"kill_existing": { "type": "boolean", "description": "Kill any running MT5 instance first (default: true)" },
"timeout": { "type": "integer", "description": "Max wait time per week in seconds (default: 900)" },
"gui": { "type": "boolean", "description": "Enable MT5 visualization window" },
"startup_delay_secs": { "type": "integer", "description": "Seconds to wait for MT5 initialization (default: 10)" }
}
}
})
}
pub fn tool_clean_cache() -> Value {
json!({
"name": "clean_cache",
-5
View File
@@ -18,8 +18,6 @@ pub fn get_tools_list() -> Value {
backtest::tool_run_backtest_only(), // Minimal: skip compile, do clean + backtest + extract only
backtest::tool_launch_backtest(), // Fire-and-forget: compile + clean + launch MT5
backtest::tool_get_backtest_status(), // Poll for completion
backtest::tool_get_tester_log(), // Live journal reading mid-backtest or after
backtest::tool_run_rolling_backtest(), // Rolling backtest: multiple sequential weeks
backtest::tool_cache_status(),
backtest::tool_clean_cache(),
// Optimization
@@ -65,8 +63,6 @@ pub fn get_tools_list() -> Value {
system::tool_list_symbols(),
system::tool_healthcheck(),
system::tool_get_active_account(),
system::tool_check_update(),
system::tool_update(),
// Utility (8 tools)
utility::tool_check_symbol_data_status(),
utility::tool_get_backtest_history(),
@@ -114,7 +110,6 @@ pub fn get_tools_list() -> Value {
reports::tool_search_reports_by_notes(),
reports::tool_get_reports_by_set_file(),
reports::tool_get_comparable_reports(),
reports::tool_export_deals_csv(),
];
serde_json::json!(tools)
+1 -2
View File
@@ -13,8 +13,7 @@ pub fn tool_run_optimization() -> Value {
"symbol": { "type": "string", "description": "Trading symbol (default: XAUUSD)" },
"from_date": { "type": "string", "description": "Start date YYYY.MM.DD" },
"to_date": { "type": "string", "description": "End date YYYY.MM.DD" },
"deposit": { "type": "integer", "description": "Initial deposit (default: 10000)" },
"max_passes": { "type": "integer", "description": "Cap on genetic optimization passes (MT5 default: ~10496 for 10 params). Use e.g. 5000 to run ~50% fewer passes." }
"deposit": { "type": "integer", "description": "Initial deposit (default: 10000)" }
}
}
})
-14
View File
@@ -278,17 +278,3 @@ pub fn tool_get_comparable_reports() -> Value {
}
})
}
pub fn tool_export_deals_csv() -> Value {
json!({
"name": "export_deals_csv",
"description": "Export deals for a report to a CSV file on demand. Deals are stored in the database — use this to get a CSV file for external tools.",
"inputSchema": {
"type": "object",
"properties": {
"report_id": { "type": "string", "description": "Report ID to export (default: latest report)" },
"output_path": { "type": "string", "description": "File path for the CSV output (default: <report_dir>/deals.csv)" }
}
}
})
}
-22
View File
@@ -45,25 +45,3 @@ pub fn tool_get_active_account() -> Value {
}
})
}
pub fn tool_check_update() -> Value {
json!({
"name": "check_update",
"description": "Check if a newer version of mt5-quant is available on GitHub. A background check runs automatically on the first tool call of each session; this tool returns that cached result instantly or fetches it on demand.",
"inputSchema": {
"type": "object",
"properties": {}
}
})
}
pub fn tool_update() -> Value {
json!({
"name": "update",
"description": "Download and install the latest mt5-quant binary from GitHub Releases, then replace the current executable in place. Restart the MCP connection after updating to load the new version.",
"inputSchema": {
"type": "object",
"properties": {}
}
})
}
-21
View File
@@ -1,21 +0,0 @@
use serde_json::{json, Value};
pub fn tool_check_update() -> Value {
json!({
"name": "check_update",
"description": "Check if a newer version of mt5-quant is available on GitHub. A background check runs automatically on the first tool call of each session; this tool returns that cached result instantly or fetches it on demand.",
"inputSchema": {
"type": "object"
}
})
}
pub fn tool_update() -> Value {
json!({
"name": "update",
"description": "Download and install the latest mt5-quant binary from GitHub Releases, then replace the current executable in place. Restart the MCP connection after updating to load the new version.",
"inputSchema": {
"type": "object"
}
})
}
+1 -1
View File
@@ -289,7 +289,7 @@ pub fn tool_get_wine_prefix_info() -> Value {
pub fn tool_get_backtest_crash_info() -> Value {
json!({
"name": "get_backtest_crash_info",
"description": "Investigate backtest crashes/failures. Checks for incomplete markers, missing metrics.json, DB deal count, error logs. Can scan recent reports.",
"description": "Investigate backtest crashes/failures. Checks for incomplete markers, missing deals.csv, error logs. Can scan recent reports.",
"inputSchema": {
"type": "object",
"properties": {
+91 -53
View File
@@ -7,7 +7,6 @@ use crate::analytics::DealAnalyzer;
use crate::models::deals::Deal;
use crate::models::metrics::Metrics;
use crate::models::Config;
use crate::storage::ReportDb;
// ── Internal helpers ──────────────────────────────────────────────────────────
@@ -31,46 +30,66 @@ fn err_response(msg: impl std::fmt::Display) -> Value {
})
}
/// Resolve a report from args (report_id > report_dir > latest).
/// Returns (deals, metrics, report_dir).
fn resolve_report(args: &Value) -> Result<(Vec<Deal>, Metrics, String)> {
let db = ReportDb::new(&Config::db_path());
db.init()?;
fn load_report_data(report_dir: &str) -> Result<(Vec<Deal>, Metrics)> {
let deals_csv = Path::new(report_dir).join("deals.csv");
let metrics_json = Path::new(report_dir).join("metrics.json");
let entry = if let Some(id) = args.get("report_id").and_then(|v| v.as_str()) {
db.get_by_id(id)?
.ok_or_else(|| anyhow::anyhow!("Report '{}' not found in DB", id))?
} else if let Some(dir) = args.get("report_dir").and_then(|v| v.as_str()) {
db.get_by_report_dir(dir)?
.ok_or_else(|| anyhow::anyhow!(
"No DB entry for report_dir '{}'. This report may predate DB storage.", dir
))?
} else {
db.get_latest()?
.ok_or_else(|| anyhow::anyhow!("No reports in DB. Run a backtest first."))?
};
if !deals_csv.exists() {
return Err(anyhow::anyhow!("deals.csv not found in {}", report_dir));
}
let deals = db.get_deals(&entry.id)?;
let metrics_path = Path::new(&entry.report_dir).join("metrics.json");
let metrics = if metrics_path.exists() {
serde_json::from_str(&fs::read_to_string(&metrics_path)?)?
let deals = read_deals_from_csv(&deals_csv)?;
let metrics = if metrics_json.exists() {
serde_json::from_str(&fs::read_to_string(&metrics_json)?)?
} else {
Metrics::default()
};
Ok((deals, metrics, entry.report_dir))
Ok((deals, metrics))
}
fn prepare_analysis(args: &Value) -> Result<(Vec<Deal>, Metrics, DealAnalyzer, String)> {
let (deals, metrics, report_dir) = resolve_report(args)?;
Ok((deals, metrics, DealAnalyzer::new(), report_dir))
fn prepare_analysis(report_dir: &str) -> Result<(Vec<Deal>, Metrics, DealAnalyzer)> {
let (deals, metrics) = load_report_data(report_dir)?;
Ok((deals, metrics, DealAnalyzer::new()))
}
fn read_deals_from_csv(path: &Path) -> Result<Vec<Deal>> {
let content = fs::read_to_string(path)?;
let mut deals = Vec::new();
let mut lines = content.lines();
let _header = lines.next();
for line in lines {
let parts: Vec<&str> = line.split(',').collect();
if parts.len() >= 12 {
deals.push(Deal {
time: parts[0].to_string(),
deal: parts[1].to_string(),
symbol: parts[2].to_string(),
deal_type: parts[3].to_string(),
entry: parts[4].to_string(),
volume: parts[5].parse().unwrap_or(0.0),
price: parts[6].parse().unwrap_or(0.0),
order: parts[7].to_string(),
commission: parts[8].parse().unwrap_or(0.0),
swap: parts[9].parse().unwrap_or(0.0),
profit: parts[10].parse().unwrap_or(0.0),
balance: parts[11].parse().unwrap_or(0.0),
comment: parts.get(12).unwrap_or(&"").to_string(),
magic: parts.get(13).map(|s| s.to_string()),
});
}
}
Ok(deals)
}
// ── Composite analytics ───────────────────────────────────────────────────────
pub async fn handle_analyze_report(_config: &Config, args: &Value) -> Result<Value> {
let (deals, metrics, analyzer, report_dir) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, metrics, analyzer) = prepare_analysis(report_dir)?;
let requested: Option<HashSet<String>> = args.get("analytics")
.and_then(|v| v.as_array())
@@ -92,7 +111,7 @@ pub async fn handle_analyze_report(_config: &Config, args: &Value) -> Result<Val
if req("streak_analysis") { result["streak_analysis"] = json!(analyzer.streak_analysis(&deals)); }
if req("concurrent_peak") { result["concurrent_peak"] = json!(analyzer.concurrent_peak(&deals)); }
let analysis_path = Path::new(&report_dir).join("analysis.json");
let analysis_path = Path::new(report_dir).join("analysis.json");
fs::write(&analysis_path, serde_json::to_string_pretty(&result)?)?;
Ok(ok_response(json!({
@@ -104,10 +123,10 @@ pub async fn handle_analyze_report(_config: &Config, args: &Value) -> Result<Val
}
pub async fn handle_compare_baseline(_config: &Config, args: &Value) -> Result<Value> {
let (_, _, _, report_dir) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let baseline_path = Path::new("config/baseline.json");
let metrics_path = Path::new(&report_dir).join("metrics.json");
let metrics_path = Path::new(report_dir).join("metrics.json");
if !baseline_path.exists() {
return Ok(ok_response(json!("No baseline.json found in config/")));
@@ -131,78 +150,93 @@ pub async fn handle_compare_baseline(_config: &Config, args: &Value) -> Result<V
// ── Granular analytics handlers ───────────────────────────────────────────────
pub async fn handle_analyze_monthly_pnl(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "monthly_pnl": analyzer.monthly_pnl(&deals) })))
}
pub async fn handle_analyze_drawdown_events(_config: &Config, args: &Value) -> Result<Value> {
let (deals, metrics, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, metrics, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "drawdown_events": analyzer.reconstruct_dd_events(&deals, &metrics) })))
}
pub async fn handle_analyze_top_losses(_config: &Config, args: &Value) -> Result<Value> {
let report_dir = required_str(args, "report_dir")?;
let limit = args.get("limit").and_then(|v| v.as_u64()).unwrap_or(10) as usize;
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "limit": limit, "top_losses": analyzer.top_losses(&deals, limit) })))
}
pub async fn handle_analyze_loss_sequences(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "loss_sequences": analyzer.loss_sequences(&deals) })))
}
pub async fn handle_analyze_position_pairs(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "position_pairs": analyzer.position_pairs(&deals) })))
}
pub async fn handle_analyze_direction_bias(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "direction_bias": analyzer.direction_bias(&deals) })))
}
pub async fn handle_analyze_streaks(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "streak_analysis": analyzer.streak_analysis(&deals) })))
}
pub async fn handle_analyze_concurrent_peak(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "concurrent_peak": analyzer.concurrent_peak(&deals) })))
}
pub async fn handle_analyze_profit_distribution(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "profit_distribution": analyzer.profit_distribution(&deals) })))
}
pub async fn handle_analyze_time_performance(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "time_performance": analyzer.time_performance(&deals) })))
}
pub async fn handle_analyze_hold_time_distribution(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "hold_time_analysis": analyzer.hold_time_analysis(&deals) })))
}
pub async fn handle_analyze_layer_performance(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "layer_performance": analyzer.layer_performance(&deals) })))
}
pub async fn handle_analyze_volume_vs_profit(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "volume_analysis": analyzer.volume_analysis(&deals) })))
}
pub async fn handle_analyze_costs(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "cost_analysis": analyzer.cost_analysis(&deals) })))
}
pub async fn handle_analyze_efficiency(_config: &Config, args: &Value) -> Result<Value> {
let (deals, metrics, analyzer, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, metrics, analyzer) = prepare_analysis(report_dir)?;
Ok(ok_response(json!({ "success": true, "efficiency_analysis": analyzer.efficiency_analysis(&deals, &metrics) })))
}
@@ -225,13 +259,12 @@ fn deal_to_json(d: &Deal) -> Value {
}
fn is_closed_trade(d: &Deal) -> bool {
// Accept any "out" entry. Profit may legitimately be 0.0 for journal-extracted
// deals (where per-deal P&L is unavailable), so we don't gate on profit != 0.0.
d.entry.to_lowercase().contains("out")
d.entry.to_lowercase().contains("out") && d.profit != 0.0
}
pub async fn handle_list_deals(_config: &Config, args: &Value) -> Result<Value> {
let (deals, _, _, _) = prepare_analysis(args)?;
let report_dir = required_str(args, "report_dir")?;
let (deals, _) = load_report_data(report_dir)?;
let deal_type = args.get("deal_type").and_then(|v| v.as_str());
let min_profit = args.get("min_profit").and_then(|v| v.as_f64());
@@ -266,9 +299,11 @@ pub async fn handle_list_deals(_config: &Config, args: &Value) -> Result<Value>
}
pub async fn handle_search_deals_by_comment(_config: &Config, args: &Value) -> Result<Value> {
let report_dir = required_str(args, "report_dir")?;
let query = required_str(args, "query")?;
let limit = args.get("limit").and_then(|v| v.as_u64()).unwrap_or(50) as usize;
let (deals, _, _, _) = prepare_analysis(args)?;
let (deals, _) = load_report_data(report_dir)?;
let query_lower = query.to_lowercase();
let mut filtered: Vec<&Deal> = deals.iter()
@@ -287,9 +322,11 @@ pub async fn handle_search_deals_by_comment(_config: &Config, args: &Value) -> R
}
pub async fn handle_search_deals_by_magic(_config: &Config, args: &Value) -> Result<Value> {
let report_dir = required_str(args, "report_dir")?;
let magic = required_str(args, "magic")?;
let limit = args.get("limit").and_then(|v| v.as_u64()).unwrap_or(100) as usize;
let (deals, _, _, _) = prepare_analysis(args)?;
let (deals, _) = load_report_data(report_dir)?;
let mut filtered: Vec<&Deal> = deals.iter()
.filter(|d| is_closed_trade(d) && d.magic.as_ref().map(|m| m.contains(magic)).unwrap_or(false))
@@ -306,5 +343,6 @@ pub async fn handle_search_deals_by_magic(_config: &Config, args: &Value) -> Res
})))
}
// suppress unused warning — err_response is available for future handlers
#[allow(dead_code)]
fn _err_response_available() { let _ = err_response(""); }
fn _use_err_response() { let _ = err_response(""); }
+86 -596
View File
@@ -1,10 +1,8 @@
use anyhow::Result;
use chrono::Datelike;
use serde_json::{json, Value};
use std::fs;
use std::path::{Path, PathBuf};
use std::path::Path;
use std::process::Command;
use tokio::time::Duration;
use crate::models::Config;
use crate::models::report::BacktestJob;
use crate::pipeline::backtest::{BacktestParams, BacktestPipeline};
@@ -76,87 +74,66 @@ pub async fn handle_run_backtest(config: &Config, args: &Value) -> Result<Value>
.and_then(|v| v.as_str())
.unwrap_or("");
// No tester data at all → hard fail with helpful context
let no_symbols_error = || json!({
"content": [{ "type": "text", "text": json!({
"error": format!("No symbols available for backtesting on server '{}'.", active_server),
"account": { "login": active_login, "server": active_server },
"hint": "Open MT5 → View → Strategy Tester → download history for at least one symbol.",
"pre_check": "no_symbols"
}).to_string() }],
"isError": true
});
let symbol: String = if requested_symbol.is_empty() {
// No symbol requested — use config default or first available tester symbol.
let candidate = config.backtest_symbol.as_deref().unwrap_or("");
if candidate.is_empty() {
preflight.available_symbols.first()
.cloned()
.ok_or(()
).unwrap_or_else(|_| return String::new())
} else {
match Config::resolve_symbol(candidate, &preflight.available_symbols) {
Some(resolved) => {
if resolved != candidate {
tracing::warn!(
"Config symbol '{}' not in tester data for '{}'; using '{}' instead",
candidate, active_server, resolved
);
}
resolved.to_string()
}
None => {
// Config default has no tester data — pick first available
preflight.available_symbols.first()
.cloned()
.unwrap_or_default()
}
}
}
} else {
// Caller specified a symbol — resolve it against actual tester data.
if preflight.available_symbols.is_empty() {
return Ok(no_symbols_error());
}
match Config::resolve_symbol(requested_symbol, &preflight.available_symbols) {
Some(resolved) if resolved == requested_symbol => {
// Exact match — use as-is
resolved.to_string()
}
Some(resolved) => {
// Fuzzy match — proceed with the corrected symbol, surface the substitution
tracing::warn!(
"Symbol '{}' not in tester data for '{}'; substituting '{}'",
requested_symbol, active_server, resolved
);
resolved.to_string()
}
None => {
// No match at all — fail with full context so the caller can act
let symbol = if requested_symbol.is_empty() {
// Use config default or first available symbol
if let Some(default) = config.backtest_symbol.clone() {
if preflight.available_symbols.contains(&default) {
default
} else if let Some(first) = preflight.available_symbols.first() {
tracing::warn!("Default symbol {} not found for server {}; using {}", default, active_server, first);
first.clone()
} else {
return Ok(json!({
"content": [{ "type": "text", "text": json!({
"error": format!(
"Symbol '{}' has no tester data on server '{}' and no close match was found.",
requested_symbol, active_server
),
"account": { "login": active_login, "server": active_server },
"requested_symbol": requested_symbol,
"available_symbols": preflight.available_symbols,
"hint": "The tester data for this symbol hasn't been downloaded yet. \
Open MT5 Strategy Tester select the symbol and click Download.",
"pre_check": "symbol_not_available"
"error": format!("No symbols available for backtesting on server '{}'.", active_server),
"account": {
"login": active_login,
"server": active_server
},
"hint": "Download historical data in MT5 Strategy Tester for this server.",
"pre_check": "no_symbols",
"suggestion": "Use get_active_account to see available symbols."
}).to_string() }],
"isError": true
}));
}
} else if let Some(first) = preflight.available_symbols.first() {
first.clone()
} else {
return Ok(json!({
"content": [{ "type": "text", "text": json!({
"error": format!("No symbols available for backtesting on server '{}'.", active_server),
"account": {
"login": active_login,
"server": active_server
},
"hint": "Download historical data in MT5 Strategy Tester for this server.",
"pre_check": "no_symbols",
"suggestion": "Use get_active_account to see available symbols."
}).to_string() }],
"isError": true
}));
}
} else {
if !preflight.available_symbols.is_empty() && !preflight.available_symbols.contains(&requested_symbol.to_string()) {
return Ok(json!({
"content": [{ "type": "text", "text": json!({
"error": format!("Symbol '{}' is not available for server '{}'.", requested_symbol, active_server),
"account": {
"login": active_login,
"server": active_server
},
"requested_symbol": requested_symbol,
"available_symbols": preflight.available_symbols,
"hint": "The symbol may not have history data for this account's server. Use list_symbols to see available symbols.",
"pre_check": "symbol_not_available",
"suggestion": "Either switch to a different MT5 account with this symbol's data, or download history for this symbol on the current server."
}).to_string() }],
"isError": true
}));
}
requested_symbol.to_string()
};
// Guard against the empty-string edge case (no symbols at all)
if symbol.is_empty() {
return Ok(no_symbols_error());
}
// EA existence check with context
if !preflight.ea_exists {
@@ -170,12 +147,12 @@ pub async fn handle_run_backtest(config: &Config, args: &Value) -> Result<Value>
}));
}
// Date defaulting: current month
// Date defaulting: past complete calendar month
let (from_date, to_date) = {
let f = args.get("from_date").and_then(|v| v.as_str()).unwrap_or("");
let t = args.get("to_date").and_then(|v| v.as_str()).unwrap_or("");
if f.is_empty() || t.is_empty() {
super::current_month()
super::past_complete_month()
} else {
(f.to_string(), t.to_string())
}
@@ -195,12 +172,10 @@ pub async fn handle_run_backtest(config: &Config, args: &Value) -> Result<Value>
skip_clean: args.get("skip_clean").and_then(|v| v.as_bool()).unwrap_or(false),
skip_analyze: args.get("skip_analyze").and_then(|v| v.as_bool()).unwrap_or(false),
deep_analyze: args.get("deep").and_then(|v| v.as_bool()).unwrap_or(false),
shutdown: args.get("shutdown").and_then(|v| v.as_bool()).unwrap_or(true),
shutdown: args.get("shutdown").and_then(|v| v.as_bool()).unwrap_or(false),
kill_existing: args.get("kill_existing").and_then(|v| v.as_bool()).unwrap_or(false),
timeout: args.get("timeout").and_then(|v| v.as_u64()).unwrap_or(900),
gui: args.get("gui").and_then(|v| v.as_bool()).unwrap_or(false),
startup_delay_secs: args.get("startup_delay_secs").and_then(|v| v.as_u64()).unwrap_or(0),
inactivity_kill_secs: args.get("inactivity_kill_secs").and_then(|v| v.as_u64()),
};
let pipeline = BacktestPipeline::new(config.clone());
@@ -217,35 +192,33 @@ pub async fn handle_run_backtest(config: &Config, args: &Value) -> Result<Value>
}))
}
pub async fn handle_run_backtest_quick(handler: &crate::tools::handlers::ToolHandler, args: &Value) -> Result<Value> {
// Quick backtest: skip compile, clean → launch → background monitor → return job.
// Uses the fire-and-forget launch path so the MCP response is returned immediately
// and the result is available via get_backtest_status / get_latest_report once done.
// (The synchronous blocking path exceeds MCP request timeouts for any backtest >2 min.)
pub async fn handle_run_backtest_quick(config: &Config, args: &Value) -> Result<Value> {
// Quick backtest: skip compile, do clean → backtest → extract → analyze
let mut args = args.clone();
if let Some(obj) = args.as_object_mut() {
obj.insert("skip_compile".to_string(), json!(true));
// keep skip_analyze as false (default) to run analysis
}
handle_launch_backtest(handler, &args).await
handle_run_backtest(config, &args).await
}
pub async fn handle_run_backtest_only(handler: &crate::tools::handlers::ToolHandler, args: &Value) -> Result<Value> {
// Backtest only: skip compile and analyze — launch and return job immediately.
pub async fn handle_run_backtest_only(config: &Config, args: &Value) -> Result<Value> {
// Backtest only: skip compile, skip analyze - just backtest and extract
let mut args = args.clone();
if let Some(obj) = args.as_object_mut() {
obj.insert("skip_compile".to_string(), json!(true));
obj.insert("skip_analyze".to_string(), json!(true));
}
handle_launch_backtest(handler, &args).await
handle_run_backtest(config, &args).await
}
pub async fn handle_launch_backtest(handler: &crate::tools::handlers::ToolHandler, args: &Value) -> Result<Value> {
pub async fn handle_launch_backtest(config: &Config, args: &Value) -> Result<Value> {
let expert = args.get("expert")
.and_then(|v| v.as_str())
.ok_or_else(|| anyhow::anyhow!("expert is required"))?;
// Run pre-flight check
let preflight = BacktestPreflight::check(&handler.config, expert);
let preflight = BacktestPreflight::check(config, expert);
// Check account session
if preflight.account.is_none() {
@@ -258,50 +231,17 @@ pub async fn handle_launch_backtest(handler: &crate::tools::handlers::ToolHandle
}));
}
// Get symbol — resolve against actual tester data (same logic as handle_run_backtest)
let active_server = preflight.server.as_deref().unwrap_or("unknown");
// Get symbol
let requested_symbol = args.get("symbol")
.and_then(|v| v.as_str())
.unwrap_or("");
let symbol: String = if requested_symbol.is_empty() {
let candidate = handler.config.backtest_symbol.as_deref().unwrap_or("");
if candidate.is_empty() {
preflight.available_symbols.first().cloned().unwrap_or_default()
} else {
Config::resolve_symbol(candidate, &preflight.available_symbols)
.map(|s| s.to_string())
.or_else(|| preflight.available_symbols.first().cloned())
.unwrap_or_else(|| candidate.to_string())
}
let symbol = if requested_symbol.is_empty() {
config.backtest_symbol.clone()
.or_else(|| preflight.available_symbols.first().cloned())
.unwrap_or_else(|| "EURUSD".to_string())
} else {
match Config::resolve_symbol(requested_symbol, &preflight.available_symbols) {
Some(resolved) => {
if resolved != requested_symbol {
tracing::warn!(
"launch_backtest: symbol '{}' not in tester data for '{}'; using '{}'",
requested_symbol, active_server, resolved
);
}
resolved.to_string()
}
None if preflight.available_symbols.is_empty() => requested_symbol.to_string(),
None => {
return Ok(json!({
"content": [{ "type": "text", "text": json!({
"error": format!(
"Symbol '{}' has no tester data on server '{}' and no close match was found.",
requested_symbol, active_server
),
"requested_symbol": requested_symbol,
"available_symbols": preflight.available_symbols,
"hint": "Open MT5 → Strategy Tester → select the symbol and click Download.",
"pre_check": "symbol_not_available"
}).to_string() }],
"isError": true
}));
}
}
requested_symbol.to_string()
};
// EA existence check
@@ -320,7 +260,7 @@ pub async fn handle_launch_backtest(handler: &crate::tools::handlers::ToolHandle
let f = args.get("from_date").and_then(|v| v.as_str()).unwrap_or("");
let t = args.get("to_date").and_then(|v| v.as_str()).unwrap_or("");
if f.is_empty() || t.is_empty() {
super::current_month()
super::past_complete_month()
} else {
(f.to_string(), t.to_string())
}
@@ -340,24 +280,13 @@ pub async fn handle_launch_backtest(handler: &crate::tools::handlers::ToolHandle
skip_clean: args.get("skip_clean").and_then(|v| v.as_bool()).unwrap_or(false),
skip_analyze: true, // Not needed for launch mode
deep_analyze: false,
// On Wine/macOS, ShutdownTerminal=1 does NOT reliably cause terminal64.exe to exit.
// When inactivity_kill_secs is set, the monitor waits that many seconds after the
// tester log goes quiet, then polls for the HTML report for 30 s, then kills MT5.
// If no inactivity_kill_secs is given (default None → disabled), the monitor relies
// solely on timeout (900 s) or natural MT5 exit for completion detection.
shutdown: args.get("shutdown").and_then(|v| v.as_bool()).unwrap_or(true),
shutdown: false, // Don't shutdown so we can poll
kill_existing: false,
timeout: args.get("timeout").and_then(|v| v.as_u64()).unwrap_or(900),
gui: args.get("gui").and_then(|v| v.as_bool()).unwrap_or(false),
startup_delay_secs: args.get("startup_delay_secs").and_then(|v| v.as_u64()).unwrap_or(0),
inactivity_kill_secs: args.get("inactivity_kill_secs").and_then(|v| v.as_u64()),
};
let pipeline = if let Some(ref callback) = handler.notification_callback {
BacktestPipeline::with_notification_callback(handler.config.clone(), callback.clone())
} else {
BacktestPipeline::new(handler.config.clone())
};
let pipeline = BacktestPipeline::new(config.clone());
let job = pipeline.launch_backtest(params).await?;
Ok(json!({
@@ -377,348 +306,6 @@ pub async fn handle_launch_backtest(handler: &crate::tools::handlers::ToolHandle
}))
}
pub async fn handle_run_rolling_backtest(config: &Config, args: &Value) -> Result<Value> {
let expert = args.get("expert")
.and_then(|v| v.as_str())
.ok_or_else(|| anyhow::anyhow!("expert is required"))?;
let weeks_count = args.get("weeks").and_then(|v| v.as_u64()).unwrap_or(4) as i64;
if weeks_count < 1 || weeks_count > 52 {
return Ok(json!({
"content": [{ "type": "text", "text": "weeks must be between 1 and 52".to_string() }],
"isError": true
}));
}
// Calculate weekly date ranges
let from_arg = args.get("from_date").and_then(|v| v.as_str()).unwrap_or("");
let to_arg = args.get("to_date").and_then(|v| v.as_str()).unwrap_or("");
let weeks: Vec<(String, String, String)> = if !from_arg.is_empty() && !to_arg.is_empty() {
let start = chrono::NaiveDate::parse_from_str(from_arg, "%Y.%m.%d")
.map_err(|e| anyhow::anyhow!("Invalid from_date '{}': {}", from_arg, e))?;
let end = chrono::NaiveDate::parse_from_str(to_arg, "%Y.%m.%d")
.map_err(|e| anyhow::anyhow!("Invalid to_date '{}': {}", to_arg, e))?;
if end <= start {
return Ok(json!({
"content": [{ "type": "text", "text": "to_date must be after from_date".to_string() }],
"isError": true
}));
}
let mut weeks = Vec::new();
let mut current = start;
let mut week_idx = 0;
while current < end && weeks.len() < weeks_count as usize {
week_idx += 1;
let week_end = if current + chrono::Duration::days(7) >= end {
end
} else {
let days_to_sun = 6 - current.weekday().num_days_from_monday();
let week_end_raw = current + chrono::Duration::days(days_to_sun as i64);
std::cmp::min(week_end_raw, end)
};
let label = format!("Week {}", week_idx);
weeks.push((label, current.format("%Y.%m.%d").to_string(), week_end.format("%Y.%m.%d").to_string()));
current = week_end + chrono::Duration::days(1);
}
weeks
} else {
let now = chrono::Utc::now().date_naive();
let days_from_sun = now.weekday().num_days_from_sunday();
let last_sun = now - chrono::Duration::days(days_from_sun as i64);
let mut weeks = Vec::new();
for i in 0..weeks_count {
let i = weeks_count - 1 - i;
let week_end = last_sun - chrono::Duration::days(i * 7);
let week_start = week_end - chrono::Duration::days(6);
let label = format!(
"{} - {}",
week_start.format("%b %d"),
week_end.format("%b %d")
);
weeks.push((label, week_start.format("%Y.%m.%d").to_string(), week_end.format("%Y.%m.%d").to_string()));
}
weeks
};
let symbol = args.get("symbol").and_then(|v| v.as_str()).unwrap_or("").to_string();
let timeframe = args.get("timeframe").and_then(|v| v.as_str()).unwrap_or("M5").to_string();
let deposit = args.get("deposit").and_then(|v| v.as_u64()).unwrap_or(10000) as u32;
let model = args.get("model").and_then(|v| v.as_u64()).unwrap_or(0) as u8;
let leverage: u32 = args.get("leverage").and_then(|v| v.as_u64()).unwrap_or(500) as u32;
let set_file = args.get("set_file").and_then(|v| v.as_str()).map(|s| s.to_string());
let deep_analyze = args.get("deep").and_then(|v| v.as_bool()).unwrap_or(false);
let shutdown = args.get("shutdown").and_then(|v| v.as_bool()).unwrap_or(true);
let kill_existing = args.get("kill_existing").and_then(|v| v.as_bool()).unwrap_or(true);
let timeout = args.get("timeout").and_then(|v| v.as_u64()).unwrap_or(900);
let gui = args.get("gui").and_then(|v| v.as_bool()).unwrap_or(false);
let startup_delay_secs = args.get("startup_delay_secs").and_then(|v| v.as_u64()).unwrap_or(0);
let skip_compile_first = args.get("skip_compile").and_then(|v| v.as_bool()).unwrap_or(false);
// Create a rolling job report dir for status tracking
let report_id = format!(
"ROLLING_{}_{}_{}",
expert,
weeks.first().map(|w| &w.1).unwrap_or(&"unknown".to_string()),
weeks.last().map(|w| &w.2).unwrap_or(&"unknown".to_string()),
);
let report_dir = config.reports_dir().join(&report_id);
fs::create_dir_all(&report_dir)?;
let job_path = report_dir.join("job.json");
let job = BacktestJob {
report_id: report_id.clone(),
report_dir: report_dir.to_string_lossy().to_string(),
expert: expert.to_string(),
symbol: symbol.clone(),
timeframe: timeframe.clone(),
mt5_pid: None,
expected_report_path: String::new(),
timeout_seconds: timeout * weeks.len() as u64,
launched_at: chrono::Utc::now().to_rfc3339(),
status: Some("launched".to_string()),
};
fs::write(&job_path, serde_json::to_string_pretty(&job)?)?;
// Save the weekly schedule for status polling
let weeks_path = report_dir.join("weeks.json");
fs::write(&weeks_path, serde_json::to_string_pretty(&json!({
"weeks": weeks.iter().map(|(l, f, t)| json!({"label": l, "from_date": f, "to_date": t})).collect::<Vec<_>>()
}))?)?;
// Spawn background task to run all weeks sequentially
let config_clone = config.clone();
let report_dir_clone = report_dir.clone();
let expert_clone = expert.to_string();
let symbol_clone = symbol;
let timeframe_clone = timeframe;
let set_file_clone = set_file;
let weeks_clone = weeks.clone();
tokio::spawn(async move {
if let Err(e) = run_rolling_weeks_sequential(
config_clone,
report_dir_clone,
expert_clone,
symbol_clone,
timeframe_clone,
deposit,
model,
leverage,
set_file_clone,
deep_analyze,
shutdown,
kill_existing,
timeout,
gui,
startup_delay_secs,
skip_compile_first,
weeks_clone,
).await {
tracing::error!("Rolling backtest failed: {}", e);
}
});
Ok(json!({
"content": [{ "type": "text", "text": json!({
"success": true,
"message": format!("Rolling backtest launched with {} weeks. Use get_backtest_status to poll for completion.", weeks.len()),
"report_id": report_id,
"report_dir": report_dir.to_string_lossy(),
"expert": expert,
"weeks": weeks.iter().map(|(l, f, t)| json!({"label": l, "from_date": f, "to_date": t})).collect::<Vec<_>>(),
"poll_hint": "Call get_backtest_status with report_dir to check progress"
}).to_string() }],
"isError": false
}))
}
async fn run_rolling_weeks_sequential(
config: Config,
report_dir: PathBuf,
expert: String,
symbol: String,
timeframe: String,
deposit: u32,
model: u8,
leverage: u32,
set_file: Option<String>,
deep_analyze: bool,
_shutdown: bool,
_kill_existing: bool,
timeout: u64,
gui: bool,
startup_delay_secs: u64,
skip_compile_first: bool,
weeks: Vec<(String, String, String)>,
) -> Result<()> {
let pipeline = if cfg!(debug_assertions) {
BacktestPipeline::new(config.clone())
} else {
BacktestPipeline::new(config.clone())
};
let progress_log = report_dir.join("progress.log");
// Clear stale results
let results_path = report_dir.join("rolling_results.json");
let _ = fs::remove_file(&results_path);
let mut week_results = Vec::new();
let mut total_net_profit: f64 = 0.0;
let mut max_drawdown: f64 = 0.0;
let mut total_trades: u64 = 0;
// Kill MT5/wineserver before starting
for pat in &["MetaTrader 5.app", "terminal64.exe", "wineserver"] {
let _ = Command::new("pkill").args(["-KILL", "-f", pat]).output();
}
tokio::time::sleep(Duration::from_secs(5)).await;
for (idx, (label, from_date, to_date)) in weeks.iter().enumerate() {
let skip_compile = if idx == 0 { skip_compile_first } else { true };
fs::write(&progress_log, format!("WEEK {}: {} ({} -> {})\n", idx + 1, label, from_date, to_date))
.unwrap_or(());
tracing::info!("Rolling week {}/{}: {} ({} -> {})", idx + 1, weeks.len(), label, from_date, to_date);
// Clean slate before each week
for pat in &["MetaTrader 5.app", "terminal64.exe", "wineserver"] {
let _ = Command::new("pkill").args(["-KILL", "-f", pat]).output();
}
tokio::time::sleep(Duration::from_secs(3)).await;
// Fire-and-forget launch (handles journal fallback if HTML not produced)
let params = BacktestParams {
expert: expert.clone(),
symbol: symbol.clone(),
from_date: from_date.clone(),
to_date: to_date.clone(),
timeframe: timeframe.clone(),
deposit,
model,
leverage,
set_file: set_file.clone(),
skip_compile,
skip_clean: false,
skip_analyze: false,
deep_analyze,
shutdown: true,
kill_existing: false,
timeout,
gui,
startup_delay_secs,
inactivity_kill_secs: None,
};
let job = match pipeline.launch_backtest(params).await {
Ok(j) => j,
Err(e) => {
tracing::error!("Rolling week {}/{} launch failed: {}", idx + 1, weeks.len(), e);
week_results.push(json!({
"label": label, "from_date": from_date, "to_date": to_date,
"success": false, "error": format!("launch failed: {}", e)
}));
continue;
}
};
// Poll for completion (up to timeout)
let week_report_dir = Path::new(&job.report_dir);
let poll_start = std::time::Instant::now();
let max_wait = Duration::from_secs(timeout);
loop {
tokio::time::sleep(Duration::from_secs(5)).await;
// Check for metrics.json (written after extraction, including journal fallback)
let metrics_path = week_report_dir.join("metrics.json");
if metrics_path.exists() { break; }
let jp = week_report_dir.join("job.json");
if let Ok(content) = fs::read_to_string(&jp) {
if let Ok(j) = serde_json::from_str::<BacktestJob>(&content) {
if let Some(ref s) = j.status {
if s == "completed" || s == "completed_no_html" { break; }
if s == "failed" || s == "timeout" || s == "timeout_inactive" {
tracing::warn!("Rolling week {}/{} background status: {}", idx + 1, weeks.len(), s);
break;
}
}
}
}
if poll_start.elapsed() > max_wait {
tracing::warn!("Rolling week {}/{} timed out after {}s", idx + 1, weeks.len(), timeout);
break;
}
}
// Wait a moment for extraction to finish
tokio::time::sleep(Duration::from_secs(2)).await;
// Read results
let metrics_path = week_report_dir.join("metrics.json");
if let Ok(content) = fs::read_to_string(&metrics_path) {
if let Ok(metrics) = serde_json::from_str::<serde_json::Value>(&content) {
let np = metrics.get("net_profit").and_then(|v| v.as_f64()).unwrap_or(0.0);
let dd = metrics.get("max_dd_pct").and_then(|v| v.as_f64()).unwrap_or(0.0);
let trades = metrics.get("total_trades").and_then(|v| v.as_u64()).unwrap_or(0);
let pf = metrics.get("profit_factor").and_then(|v| v.as_f64()).unwrap_or(0.0);
total_net_profit += np;
if dd > max_drawdown { max_drawdown = dd; }
total_trades += trades;
tracing::info!("Rolling week {}/{} done: profit={:.2}, dd={:.1}%", idx + 1, weeks.len(), np, dd);
week_results.push(json!({
"label": label, "from_date": from_date, "to_date": to_date,
"success": true, "net_profit": np, "max_dd_pct": dd,
"total_trades": trades, "profit_factor": pf,
"report_dir": job.report_dir
}));
continue;
}
}
// No metrics found
week_results.push(json!({
"label": label, "from_date": from_date, "to_date": to_date,
"success": false, "error": "No metrics extracted"
}));
}
// Write final results
let summary = json!({
"success": true,
"weeks_run": week_results.len(),
"summary": {
"total_net_profit": total_net_profit,
"max_drawdown_pct": max_drawdown,
"total_trades": total_trades,
},
"weekly_results": week_results
});
if let Ok(json_str) = serde_json::to_string_pretty(&summary) {
let _ = fs::write(&results_path, &json_str);
}
// Update job status
let job_path = report_dir.join("job.json");
if let Ok(content) = fs::read_to_string(&job_path) {
if let Ok(mut job) = serde_json::from_str::<BacktestJob>(&content) {
job.status = Some("completed".to_string());
if let Ok(json_str) = serde_json::to_string_pretty(&job) {
let _ = fs::write(&job_path, json_str);
}
}
}
fs::write(&progress_log, "DONE\n").unwrap_or(());
tracing::info!("Rolling backtest completed: {} weeks, profit={:.2}, max_dd={:.1}%", week_results.len(), total_net_profit, max_drawdown);
Ok(())
}
pub async fn handle_get_backtest_status(_config: &Config, args: &Value) -> Result<Value> {
let report_dir = args.get("report_dir")
.and_then(|v| v.as_str())
@@ -755,27 +342,16 @@ pub async fn handle_get_backtest_status(_config: &Config, args: &Value) -> Resul
// Check if MT5 is running
let mt5_running = is_mt5_running();
// Check if report file exists (still on disk — it's deleted after extraction)
// Check if report file exists
let report_found = job.as_ref()
.map(|j| Path::new(&j.expected_report_path).exists())
.unwrap_or(false);
// Check for completed artifacts
let metrics_exists = report_path.join("metrics.json").exists();
// Read the authoritative status written by the background monitor into job.json.
// This avoids false "failed" when the HTML was extracted+deleted (report_found=false)
// or when journal extraction ran instead of HTML extraction.
let job_status = job.as_ref()
.and_then(|j| j.status.as_deref())
.unwrap_or("");
let monitor_says_complete = matches!(job_status, "completed" | "completed_no_html");
let monitor_says_failed = matches!(job_status, "failed" | "timeout" | "timeout_inactive");
let is_complete = monitor_says_complete
|| stage == "DONE"
|| (report_found && metrics_exists);
let deals_exists = report_path.join("deals.csv").exists();
let is_complete = stage == "DONE" || (report_found && metrics_exists);
// Calculate elapsed time if job exists
let elapsed_seconds = job.as_ref()
.and_then(|j| {
@@ -784,32 +360,26 @@ pub async fn handle_get_backtest_status(_config: &Config, args: &Value) -> Resul
.map(|t| (chrono::Utc::now() - t.with_timezone(&chrono::Utc)).num_seconds())
})
.unwrap_or(0);
// Determine status message — trust monitor's job.json first
// Determine status message
let status_msg = if is_complete {
"completed"
} else if monitor_says_failed {
if job_status == "timeout" || job_status == "timeout_inactive" { "timeout" } else { "failed" }
} else if stage == "BACKTEST" && mt5_running {
"running"
} else if stage == "BACKTEST" && !mt5_running {
} else if stage == "BACKTEST" && !mt5_running && !report_found {
"failed"
} else if progress_lines > 0 {
"in_progress"
} else {
"not_started"
};
let message = if is_complete {
if job_status == "completed_no_html" {
"Backtest completed (extracted from tester journal — no HTML report)"
} else {
"Backtest completed successfully"
}
"Backtest completed successfully"
} else if stage == "BACKTEST" && mt5_running {
"MT5 is running the backtest"
} else if stage == "BACKTEST" && !mt5_running {
"MT5 process exited report not yet found"
"MT5 process exited but report not found - backtest may have failed"
} else {
&format!("Backtest is at stage: {}", stage)
};
@@ -824,6 +394,7 @@ pub async fn handle_get_backtest_status(_config: &Config, args: &Value) -> Resul
"mt5_running": mt5_running,
"report_found": report_found,
"metrics_extracted": metrics_exists,
"deals_extracted": deals_exists,
"elapsed_seconds": elapsed_seconds,
"message": message,
"job": job.map(|j| {
@@ -858,87 +429,6 @@ fn is_mt5_running() -> bool {
})
}
/// Read the active tester agent log for live/post-test deal inspection.
/// Returns the last N lines and a parsed deal summary.
pub async fn handle_get_tester_log(config: &Config, args: &Value) -> Result<Value> {
use crate::pipeline::backtest::BacktestPipeline;
let tail_lines = args.get("tail_lines").and_then(|v| v.as_u64()).unwrap_or(100) as usize;
let log_path = match BacktestPipeline::find_active_tester_agent_log(config) {
Some(p) => p,
None => {
return Ok(json!({
"content": [{ "type": "text", "text": json!({
"error": "No tester agent log found for today.",
"hint": "Run a backtest first. The log appears after the tester starts."
}).to_string() }],
"isError": true
}));
}
};
let lines = match BacktestPipeline::read_tester_agent_log(&log_path) {
Some(l) => l,
None => {
return Ok(json!({
"content": [{ "type": "text", "text": json!({
"error": format!("Could not read log at {}", log_path.display())
}).to_string() }],
"isError": true
}));
}
};
let (deals, final_balance_pips, progress) = BacktestPipeline::parse_journal_deals(&lines);
// Collect tail lines
let tail: Vec<&str> = lines.iter()
.rev()
.take(tail_lines)
.rev()
.map(|s| s.as_str())
.collect();
// Detect last sim timestamp for progress estimation
let last_sim_time = lines.iter().rev()
.find_map(|l| {
let parts: Vec<&str> = l.split_whitespace().collect();
// Format: XX 0 HH:MM:SS.mmm Core NN YYYY.MM.DD HH:MM:SS ...
if parts.len() >= 6 {
let date = parts[4];
let time = parts[5];
if date.contains('.') && time.contains(':') {
return Some(format!("{} {}", date, time));
}
}
None
})
.unwrap_or_default();
Ok(json!({
"content": [{ "type": "text", "text": json!({
"log_path": log_path.to_string_lossy(),
"total_lines": lines.len(),
"deals_found": deals.len(),
"final_balance_pips": final_balance_pips,
"progress": progress,
"last_sim_time": last_sim_time,
"is_complete": !progress.is_empty(),
"tail_lines": tail,
"deals_summary": deals.iter().map(|d| json!({
"deal": d.deal,
"time": d.time,
"type": d.deal_type,
"volume": d.volume,
"price": d.price,
"symbol": d.symbol,
})).collect::<Vec<_>>()
}).to_string() }],
"isError": false
}))
}
pub async fn handle_cache_status(config: &Config) -> Result<Value> {
let cache_dir = config.tester_cache_dir.as_ref()
.map(|s| Path::new(s))
+8 -59
View File
@@ -2,12 +2,8 @@ use anyhow::Result;
use chrono::Datelike;
use serde_json::{json, Value};
use std::path::Path;
use std::sync::{Arc, OnceLock};
use std::sync::atomic::{AtomicBool, Ordering};
use crate::models::Config;
type NotificationCallback = Arc<dyn Fn(&str, serde_json::Value) + Send + Sync>;
mod system;
mod experts;
mod backtest;
@@ -17,45 +13,23 @@ mod setfiles;
mod reports;
mod utility;
/// Cached result of the background update check.
/// - Not yet initialized: check still in flight (or not spawned yet)
/// - Some(version): a newer version is available
/// - None: already on latest, or GitHub unreachable
pub(crate) static LATEST_VERSION: OnceLock<Option<String>> = OnceLock::new();
static BACKGROUND_CHECK_SPAWNED: AtomicBool = AtomicBool::new(false);
#[derive(Clone)]
#[derive(Debug)]
pub struct ToolHandler {
pub config: Config,
pub notification_callback: Option<NotificationCallback>,
}
impl ToolHandler {
pub fn new(config: Config) -> Self {
Self { config, notification_callback: None }
}
pub fn with_notification_callback(config: Config, callback: NotificationCallback) -> Self {
Self { config, notification_callback: Some(callback) }
Self { config }
}
pub async fn handle(&self, name: &str, args: &Value) -> Result<Value> {
// Spawn a one-shot background update check on the very first tool call of the session.
if !BACKGROUND_CHECK_SPAWNED.swap(true, Ordering::Relaxed) {
tokio::spawn(async {
let result = system::fetch_latest_version().await;
let _ = LATEST_VERSION.set(result);
});
}
match name {
// System handlers
"verify_setup" => system::handle_verify_setup(&self.config).await,
"list_symbols" => system::handle_list_symbols(&self.config).await,
"healthcheck" => system::handle_healthcheck(&self.config, args).await,
"verify_setup" => system::handle_verify_setup(&self.config).await,
"list_symbols" => system::handle_list_symbols(&self.config).await,
"healthcheck" => system::handle_healthcheck(&self.config, args).await,
"get_active_account" => system::handle_get_active_account(&self.config).await,
"check_update" => system::handle_check_update(&self.config).await,
"update" => system::handle_update(&self.config).await,
// Expert/Indicator/Script handlers
"list_experts" => experts::handle_list_experts(&self.config, args).await,
@@ -70,12 +44,10 @@ impl ToolHandler {
// Backtest handlers - Granular pipeline options
"run_backtest" => backtest::handle_run_backtest(&self.config, args).await, // Full: compile + clean + backtest + extract + analyze
"run_backtest_quick" => backtest::handle_run_backtest_quick(self, args).await, // Quick: skip compile, fire-and-forget launch
"run_backtest_only" => backtest::handle_run_backtest_only(self, args).await, // Minimal: skip compile+analyze, fire-and-forget launch
"launch_backtest" => backtest::handle_launch_backtest(self, args).await, // Fire-and-forget mode
"run_backtest_quick" => backtest::handle_run_backtest_quick(&self.config, args).await, // Quick: skip compile, do backtest + extract + analyze
"run_backtest_only" => backtest::handle_run_backtest_only(&self.config, args).await, // Minimal: skip compile, do backtest + extract only
"launch_backtest" => backtest::handle_launch_backtest(&self.config, args).await, // Fire-and-forget mode
"get_backtest_status" => backtest::handle_get_backtest_status(&self.config, args).await,
"get_tester_log" => backtest::handle_get_tester_log(&self.config, args).await,
"run_rolling_backtest" => backtest::handle_run_rolling_backtest(&self.config, args).await,
"cache_status" => backtest::handle_cache_status(&self.config).await,
"clean_cache" => backtest::handle_clean_cache(&self.config, args).await,
@@ -137,7 +109,6 @@ impl ToolHandler {
"search_reports_by_notes" => reports::handle_search_reports_by_notes(args).await,
"get_reports_by_set_file" => reports::handle_get_reports_by_set_file(args).await,
"get_comparable_reports" => reports::handle_get_comparable_reports(args).await,
"export_deals_csv" => reports::handle_export_deals_csv(&self.config, args).await,
// Utility handlers
"check_symbol_data_status" => utility::handle_check_symbol_data_status(&self.config, args).await,
@@ -181,7 +152,6 @@ pub(crate) fn dir_size(path: &Path) -> u64 {
.sum()
}
#[allow(dead_code)]
pub(crate) fn past_complete_month() -> (String, String) {
let now = chrono::Utc::now();
let today = chrono::NaiveDate::from_ymd_opt(now.year(), now.month(), 1)
@@ -195,24 +165,3 @@ pub(crate) fn past_complete_month() -> (String, String) {
last_of_prev.format("%Y.%m.%d").to_string(),
)
}
pub(crate) fn current_month() -> (String, String) {
let now = chrono::Utc::now();
let first_of_month = chrono::NaiveDate::from_ymd_opt(now.year(), now.month(), 1)
.unwrap_or_else(|| chrono::NaiveDate::from_ymd_opt(2024, 1, 1).unwrap());
let last_of_month = if now.month() == 12 {
chrono::NaiveDate::from_ymd_opt(now.year() + 1, 1, 1)
.unwrap_or(first_of_month)
.pred_opt()
.unwrap_or(first_of_month)
} else {
chrono::NaiveDate::from_ymd_opt(now.year(), now.month() + 1, 1)
.unwrap_or(first_of_month)
.pred_opt()
.unwrap_or(first_of_month)
};
(
first_of_month.format("%Y.%m.%d").to_string(),
last_of_month.format("%Y.%m.%d").to_string(),
)
}
+3 -29
View File
@@ -1,7 +1,5 @@
use anyhow::Result;
use serde_json::{json, Value};
use std::fs;
use std::path::Path;
use crate::models::Config;
use crate::optimization::{OptimizationParams, OptimizationParser, OptimizationRunner};
@@ -29,9 +27,9 @@ pub async fn handle_run_optimization(config: &Config, args: &Value) -> Result<Va
from_date: from_date.to_string(),
to_date: to_date.to_string(),
deposit: args.get("deposit").and_then(|v| v.as_u64()).unwrap_or(10000) as u32,
model: 0,
leverage: args.get("leverage").and_then(|v| v.as_u64()).unwrap_or(500) as u32,
currency: args.get("currency").and_then(|v| v.as_str()).unwrap_or("USD").to_string(),
max_passes: args.get("max_passes").and_then(|v| v.as_u64()).map(|v| v as u32),
};
let runner = OptimizationRunner::new(config.clone());
@@ -58,30 +56,6 @@ pub async fn handle_get_optimization_status(config: &Config, args: &Value) -> Re
let runner = OptimizationRunner::new(config.clone());
let status = runner.get_job_status(job_id)?;
// Store completion results back to metadata for persistence
if status.get("status").and_then(|v| v.as_str()) == Some("completed") {
let jobs_dir = Path::new(".mt5mcp_jobs");
let meta_path = jobs_dir.join(format!("{}.json", job_id));
if let Ok(meta_str) = fs::read_to_string(&meta_path) {
if let Ok(mut meta) = serde_json::from_str::<serde_json::Value>(&meta_str) {
if let Some(obj) = meta.as_object_mut() {
obj.insert("status".into(), serde_json::Value::String("completed".into()));
obj.insert("completed_at".into(), serde_json::Value::String(chrono::Utc::now().to_rfc3339()));
if let Some(top) = status.get("top_10") {
obj.insert("top_10".into(), top.clone());
}
if let Some(best) = status.get("best_pf") {
obj.insert("best_pf".into(), best.clone());
}
if let Some(total) = status.get("total_passes") {
obj.insert("total_passes".into(), total.clone());
}
let _ = fs::write(&meta_path, serde_json::to_string_pretty(&meta).unwrap_or_default());
}
}
}
}
Ok(json!({
"content": [{ "type": "text", "text": status.to_string() }],
"isError": false
@@ -92,7 +66,7 @@ pub async fn handle_get_optimization_results(_config: &Config, args: &Value) ->
let job_id = args.get("job_id")
.and_then(|v| v.as_str());
let file = args.get("report_file")
let file = args.get("file")
.and_then(|v| v.as_str());
let parser = OptimizationParser::new();
@@ -106,7 +80,7 @@ pub async fn handle_get_optimization_results(_config: &Config, args: &Value) ->
};
let sort_by = args.get("sort").and_then(|v| v.as_str()).unwrap_or("profit");
let top_n = args.get("top_n").and_then(|v| v.as_u64()).unwrap_or(30) as usize;
let top_n = args.get("top").and_then(|v| v.as_u64()).unwrap_or(30) as usize;
let best = parser.find_best_pass(&passes, sort_by);
-43
View File
@@ -3,7 +3,6 @@ use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64};
use serde_json::{json, Value};
use std::fs;
use std::path::Path;
use crate::analytics::ReportExtractor;
use crate::models::Config;
use crate::storage::{ReportDb, ReportFilters};
@@ -849,45 +848,3 @@ pub async fn handle_get_comparable_reports(args: &Value) -> Result<Value> {
"isError": false
}))
}
pub async fn handle_export_deals_csv(_config: &Config, args: &Value) -> Result<Value> {
let db = ReportDb::new(&Config::db_path());
if let Err(e) = db.init() {
return Ok(json!({ "content": [{ "type": "text", "text": format!("DB error: {}", e) }], "isError": true }));
}
let report_id_opt = args.get("report_id").and_then(|v| v.as_str());
let entry = match report_id_opt {
Some(id) => db.get_by_id(id)?,
None => db.get_latest()?,
};
let entry = match entry {
Some(e) => e,
None => return Ok(json!({ "content": [{ "type": "text", "text": "No report found" }], "isError": true })),
};
let deals = db.get_deals(&entry.id)?;
if deals.is_empty() {
return Ok(json!({ "content": [{ "type": "text", "text": format!("No deals stored for report {}", entry.id) }], "isError": false }));
}
let output_path = match args.get("output_path").and_then(|v| v.as_str()) {
Some(p) => std::path::PathBuf::from(p),
None => Path::new(&entry.report_dir).join("deals.csv"),
};
let extractor = ReportExtractor::new();
extractor.write_deals_to_csv(&deals, &output_path)
.map_err(|e| anyhow::anyhow!("Failed to write CSV: {}", e))?;
Ok(json!({
"content": [{ "type": "text", "text": json!({
"success": true,
"report_id": entry.id,
"deals_count": deals.len(),
"output_path": output_path.to_string_lossy(),
}).to_string() }],
"isError": false
}))
}
+12 -29
View File
@@ -3,32 +3,16 @@ use serde_json::{json, Value};
use std::fs;
use crate::models::Config;
/// Read a file that may be UTF-16LE (with BOM) or UTF-8, returning a UTF-8 String.
fn read_file_as_utf8(path: &str) -> Result<String> {
let bytes = fs::read(path)?;
if bytes.len() >= 2 && bytes[0] == 0xFF && bytes[1] == 0xFE {
let utf16_data: Vec<u16> = bytes[2..]
.chunks_exact(2)
.map(|chunk| u16::from_le_bytes([chunk[0], chunk[1]]))
.collect();
String::from_utf16(&utf16_data)
.map_err(|e| anyhow::anyhow!("Failed to decode UTF-16LE: {}", e))
} else {
String::from_utf8(bytes)
.map_err(|e| anyhow::anyhow!("Failed to decode as UTF-8: {}", e))
}
}
pub async fn handle_read_set_file(args: &Value) -> Result<Value> {
let path = args.get("path")
.and_then(|v| v.as_str())
.ok_or_else(|| anyhow::anyhow!("path is required"))?;
let content = read_file_as_utf8(path)?;
let content = fs::read_to_string(path)?;
let mut params = serde_json::Map::new();
for line in content.lines() {
if let Some((key, value)) = line.split_once('=') {
if let Some((key, value)) = line.split_once(':') {
let key = key.trim();
let value = value.trim();
@@ -102,7 +86,7 @@ pub async fn handle_patch_set_file(args: &Value) -> Result<Value> {
.and_then(|v| v.as_object())
.ok_or_else(|| anyhow::anyhow!("patches object is required"))?;
let content = read_file_as_utf8(path)?;
let content = fs::read_to_string(path)?;
let mut lines: Vec<String> = content.lines().map(|s| s.to_string()).collect();
let mut patched_count = 0;
@@ -180,8 +164,8 @@ pub async fn handle_diff_set_files(args: &Value) -> Result<Value> {
.and_then(|v| v.as_str())
.ok_or_else(|| anyhow::anyhow!("file_b is required"))?;
let content_a = read_file_as_utf8(file_a)?;
let content_b = read_file_as_utf8(file_b)?;
let content_a = fs::read_to_string(file_a)?;
let content_b = fs::read_to_string(file_b)?;
let mut differences = Vec::new();
@@ -239,21 +223,20 @@ pub async fn handle_describe_sweep(args: &Value) -> Result<Value> {
.and_then(|v| v.as_str())
.ok_or_else(|| anyhow::anyhow!("path is required"))?;
let content = read_file_as_utf8(path)?;
let content = fs::read_to_string(path)?;
let mut sweep_params = serde_json::Map::new();
for line in content.lines() {
if let Some((key, value)) = line.split_once('=') {
if let Some((key, value)) = line.split_once(':') {
let key = key.trim();
let value = value.trim();
if value.contains("||Y") {
let parts: Vec<&str> = value.split("||").collect();
if parts.len() >= 5 && parts[4].trim().to_uppercase() == "Y" {
if let Some((from_val, to_val)) = value.split_once("..") {
sweep_params.insert(key.to_string(), json!({
"from": parts[1].trim(),
"to": parts[3].trim(),
"step": parts[2].trim(),
"from": from_val.trim(),
"to": to_val.trim().replace("||Y", ""),
"step": 1.0
}));
}
}
@@ -283,7 +266,7 @@ pub async fn handle_list_set_files(config: &Config) -> Result<Value> {
.unwrap_or("unknown")
.to_string();
let content = read_file_as_utf8(&path.to_string_lossy()).unwrap_or_default();
let content = fs::read_to_string(&path).unwrap_or_default();
let param_count = content.lines().filter(|l| l.contains(':')).count();
let sweep_count = content.lines().filter(|l| l.contains("||Y")).count();
-172
View File
@@ -3,178 +3,6 @@ use serde_json::{json, Value};
use std::path::Path;
use crate::models::Config;
// ── Update helpers ────────────────────────────────────────────────────────────
fn platform_tag() -> &'static str {
#[cfg(all(target_os = "macos", target_arch = "aarch64"))] return "macos-aarch64";
#[cfg(all(target_os = "macos", target_arch = "x86_64"))] return "macos-x86_64";
#[cfg(all(target_os = "linux", target_arch = "x86_64"))] return "linux-x86_64";
#[cfg(not(any(
all(target_os = "macos", target_arch = "aarch64"),
all(target_os = "macos", target_arch = "x86_64"),
all(target_os = "linux", target_arch = "x86_64"),
)))] return "unsupported";
}
fn semver_newer(latest: &str, current: &str) -> bool {
let parse = |s: &str| -> (u32, u32, u32) {
let mut p = s.trim_start_matches('v').splitn(3, '.');
let ma = p.next().and_then(|x| x.parse().ok()).unwrap_or(0);
let mi = p.next().and_then(|x| x.parse().ok()).unwrap_or(0);
let pa = p.next().and_then(|x| x.parse().ok()).unwrap_or(0);
(ma, mi, pa)
};
parse(latest) > parse(current)
}
/// Fetch the latest release tag from GitHub API (5 s timeout via curl).
/// Returns the version string without the leading "v", or None on failure.
pub(super) async fn fetch_latest_version() -> Option<String> {
let output = tokio::process::Command::new("curl")
.args([
"-sf", "--max-time", "5",
"-H", "Accept: application/vnd.github.v3+json",
"-H", "User-Agent: mt5-quant-updater",
"https://api.github.com/repos/masdevid/mt5-quant/releases/latest",
])
.output()
.await
.ok()?;
if !output.status.success() { return None; }
let body: Value = serde_json::from_slice(&output.stdout).ok()?;
body.get("tag_name")
.and_then(|v| v.as_str())
.map(|s| s.trim_start_matches('v').to_string())
}
fn ok_response(data: Value) -> Value {
json!({ "content": [{ "type": "text", "text": data.to_string() }], "isError": false })
}
fn err_response(msg: impl std::fmt::Display) -> Value {
json!({ "content": [{ "type": "text", "text": msg.to_string() }], "isError": true })
}
// ── Update tool handlers ──────────────────────────────────────────────────────
pub async fn handle_check_update(_config: &Config) -> Result<Value> {
let current = env!("CARGO_PKG_VERSION");
// Use cached background-check result if available; otherwise fetch now.
let latest_opt = match super::LATEST_VERSION.get() {
Some(v) => v.clone(),
None => fetch_latest_version().await,
};
let Some(latest) = latest_opt else {
return Ok(ok_response(json!({
"current_version": current,
"update_available": false,
"error": "Could not reach GitHub API — check network connectivity",
})));
};
let update_available = semver_newer(&latest, current);
Ok(ok_response(json!({
"current_version": current,
"latest_version": latest,
"update_available": update_available,
"hint": if update_available {
format!("Run the `update` tool to install v{latest}")
} else {
"You are on the latest version".to_string()
},
})))
}
pub async fn handle_update(_config: &Config) -> Result<Value> {
let current = env!("CARGO_PKG_VERSION");
let latest = match super::LATEST_VERSION.get().and_then(|v| v.as_deref()) {
Some(v) => v.to_string(),
None => match fetch_latest_version().await {
Some(v) => v,
None => return Ok(err_response(
r#"{"success":false,"error":"Could not determine latest version — check network"}"#
)),
},
};
if !semver_newer(&latest, current) {
return Ok(ok_response(json!({
"up_to_date": true,
"version": current,
})));
}
let tag = platform_tag();
if tag == "unsupported" {
return Ok(err_response(
r#"{"success":false,"error":"Auto-update not supported on this platform — build from source"}"#
));
}
let url = format!(
"https://github.com/masdevid/mt5-quant/releases/download/v{latest}/mcp-mt5-quant-{tag}.tar.gz"
);
// Download tarball to a temp file
let tmp_tar = tempfile::NamedTempFile::new()?;
let dl = tokio::process::Command::new("curl")
.args(["-sfL", "--max-time", "120",
"-o", tmp_tar.path().to_str().unwrap_or_default(),
&url])
.status()
.await?;
if !dl.success() {
return Ok(err_response(format!(
r#"{{"success":false,"error":"Download failed","url":"{}"}}"#, url
)));
}
// Extract binary (tarball root dir is mcp-mt5-quant-{platform}/)
let tmp_dir = tempfile::tempdir()?;
let extract = tokio::process::Command::new("tar")
.args(["-xzf", tmp_tar.path().to_str().unwrap_or_default(),
"-C", tmp_dir.path().to_str().unwrap_or_default(),
"--strip-components=1"])
.status()
.await?;
if !extract.success() {
return Ok(err_response(r#"{"success":false,"error":"Failed to extract archive"}"#));
}
let new_bin = tmp_dir.path().join("mt5-quant");
if !new_bin.exists() {
return Ok(err_response(r#"{"success":false,"error":"Binary not found in archive"}"#));
}
// Atomic replace: write to sibling .tmp, then rename (safe on same FS)
let current_exe = std::env::current_exe()?;
let tmp_dest = current_exe.with_extension("update_tmp");
std::fs::copy(&new_bin, &tmp_dest)?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&tmp_dest, std::fs::Permissions::from_mode(0o755))?;
}
std::fs::rename(&tmp_dest, &current_exe)?;
Ok(ok_response(json!({
"success": true,
"previous_version": current,
"updated_to": latest,
"binary": current_exe.to_string_lossy(),
"hint": format!("Updated to v{latest}. Restart the MCP connection to load the new binary."),
})))
}
pub async fn handle_verify_setup(config: &Config) -> Result<Value> {
let mut checks = serde_json::Map::new();
let mut all_ok = true;
+81 -215
View File
@@ -2,60 +2,7 @@ use anyhow::{Context, Result};
use serde_json::{json, Value};
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use crate::models::Config;
use crate::storage::ReportDb;
/// OS detection and information
#[derive(Debug, Clone)]
#[allow(dead_code)]
enum OsType {
Windows,
MacOS,
Linux,
Unknown,
}
impl OsType {
fn detect() -> Self {
#[cfg(target_os = "windows")]
return OsType::Windows;
#[cfg(target_os = "macos")]
return OsType::MacOS;
#[cfg(target_os = "linux")]
return OsType::Linux;
#[cfg(not(any(target_os = "windows", target_os = "macos", target_os = "linux")))]
return OsType::Unknown;
}
fn as_str(&self) -> &'static str {
match self {
OsType::Windows => "windows",
OsType::MacOS => "macos",
OsType::Linux => "linux",
OsType::Unknown => "unknown",
}
}
fn uses_wine(&self) -> bool {
matches!(self, OsType::MacOS | OsType::Linux)
}
}
/// Detect if using CrossOver instead of Wine on macOS
fn detect_crossover(wine_exe: &str) -> bool {
wine_exe.contains("crossover") || wine_exe.contains("CrossOver")
}
/// Get OS context for diagnostic outputs
fn get_os_context() -> serde_json::Value {
let os_type = OsType::detect();
json!({
"os": os_type.as_str(),
"uses_wine": os_type.uses_wine(),
"architecture": std::env::consts::ARCH,
})
}
/// Validate that `user_path` resolves to a location within `allowed_base`.
/// Returns the canonicalized absolute path on success.
@@ -838,6 +785,7 @@ pub async fn handle_export_report(_config: &Config, args: &Value) -> Result<Valu
let path = Path::new(report_dir);
let metrics_path = path.join("metrics.json");
let _deals_path = path.join("deals.csv");
// Read metrics
let metrics: Value = if metrics_path.exists() {
@@ -898,11 +846,8 @@ pub async fn handle_export_report(_config: &Config, args: &Value) -> Result<Valu
/// Diagnose Wine installation and prefix health
pub async fn handle_diagnose_wine(config: &Config, _args: &Value) -> Result<Value> {
let os_type = OsType::detect();
let mut diagnostics = json!({
"os_context": get_os_context(),
"wine_executable": null,
"wine_type": null,
"wine_version": null,
"wine_prefix": null,
"prefix_health": null,
@@ -912,16 +857,12 @@ pub async fn handle_diagnose_wine(config: &Config, _args: &Value) -> Result<Valu
"warnings": Vec::<String>::new(),
});
// Check wine executable (macOS/Linux)
// Check wine executable
if let Some(wine_exe) = config.wine_executable.as_ref() {
diagnostics["wine_executable"] = json!(wine_exe);
// Detect CrossOver vs Wine
let is_crossover = detect_crossover(wine_exe);
diagnostics["wine_type"] = json!(if is_crossover { "crossover" } else { "wine" });
// Get Wine/CrossOver version
let version_output = Command::new(wine_exe)
// Get Wine version
let version_output = std::process::Command::new(wine_exe)
.arg("--version")
.output();
@@ -931,26 +872,14 @@ pub async fn handle_diagnose_wine(config: &Config, _args: &Value) -> Result<Valu
diagnostics["wine_version"] = json!(version);
}
_ => {
let wine_type_str = if is_crossover { "CrossOver" } else { "Wine" };
diagnostics["errors"].as_array_mut().unwrap().push(
json!(format!("Failed to get {} version - may not be properly installed", wine_type_str))
);
}
}
// macOS-specific CrossOver checks
if matches!(os_type, OsType::MacOS) && is_crossover {
// Check if CrossOver app exists
let crossover_app = Path::new("/Applications/CrossOver.app");
if !crossover_app.exists() {
diagnostics["warnings"].as_array_mut().unwrap().push(
json!("CrossOver.app not found in /Applications - may be custom installation")
json!("Failed to get Wine version - Wine may not be properly installed")
);
}
}
} else {
diagnostics["errors"].as_array_mut().unwrap().push(
json!("Wine/CrossOver executable not configured")
json!("Wine executable not configured")
);
}
@@ -1062,7 +991,6 @@ pub async fn handle_get_mt5_logs(config: &Config, args: &Value) -> Result<Value>
};
let mut result = json!({
"os_context": get_os_context(),
"log_type": log_type,
"log_path": log_path.to_string_lossy().to_string(),
"found": false,
@@ -1181,7 +1109,6 @@ pub async fn handle_search_mt5_errors(config: &Config, args: &Value) -> Result<V
}
let result = json!({
"os_context": get_os_context(),
"hours_searched": hours_back,
"errors_found": errors_found.len(),
"max_errors": max_errors,
@@ -1201,17 +1128,16 @@ pub async fn handle_search_mt5_errors(config: &Config, args: &Value) -> Result<V
/// Check MT5 process status
pub async fn handle_check_mt5_process(_config: &Config, _args: &Value) -> Result<Value> {
let _os_type = OsType::detect();
use std::process::Command;
let mut result = json!({
"os_context": get_os_context(),
"is_running": false,
"processes": Vec::<serde_json::Value>::new(),
"wine_server_running": false,
"total_instances": 0,
});
#[cfg(any(target_os = "macos", target_os = "linux"))]
#[cfg(target_os = "macos")]
{
// Check for MT5 processes
let ps_output = Command::new("ps")
@@ -1223,7 +1149,6 @@ pub async fn handle_check_mt5_process(_config: &Config, _args: &Value) -> Result
let mut processes = Vec::new();
let mut mt5_count = 0;
let mut wine_server = false;
let mut crossover_server = false;
for line in content.lines() {
let line_lower = line.to_lowercase();
@@ -1244,11 +1169,45 @@ pub async fn handle_check_mt5_process(_config: &Config, _args: &Value) -> Result
if line_lower.contains("wineserver") {
wine_server = true;
}
}
result["processes"] = json!(processes);
result["is_running"] = json!(mt5_count > 0);
result["total_instances"] = json!(mt5_count);
result["wine_server_running"] = json!(wine_server);
}
}
#[cfg(target_os = "linux")]
{
let ps_output = Command::new("ps")
.args(["aux"])
.output();
if let Ok(output) = ps_output {
let content = String::from_utf8_lossy(&output.stdout);
let mut processes = Vec::new();
let mut mt5_count = 0;
let mut wine_server = false;
for line in content.lines() {
let line_lower = line.to_lowercase();
// Detect CrossOver server on macOS
#[cfg(target_os = "macos")]
if line_lower.contains("cxstart") || line_lower.contains("crossover") {
crossover_server = true;
if line_lower.contains("terminal64") || line_lower.contains("metatrader") {
let parts: Vec<&str> = line.split_whitespace().collect();
if parts.len() >= 11 {
processes.push(json!({
"pid": parts[1],
"cpu": parts[2],
"mem": parts[3],
"command": parts[10..].join(" "),
}));
mt5_count += 1;
}
}
if line_lower.contains("wineserver") {
wine_server = true;
}
}
@@ -1256,11 +1215,6 @@ pub async fn handle_check_mt5_process(_config: &Config, _args: &Value) -> Result
result["is_running"] = json!(mt5_count > 0);
result["total_instances"] = json!(mt5_count);
result["wine_server_running"] = json!(wine_server);
#[cfg(target_os = "macos")]
{
result["crossover_server_running"] = json!(crossover_server);
}
}
}
@@ -1272,7 +1226,7 @@ pub async fn handle_check_mt5_process(_config: &Config, _args: &Value) -> Result
/// Kill stuck MT5 process
pub async fn handle_kill_mt5_process(_config: &Config, args: &Value) -> Result<Value> {
let _os_type = OsType::detect();
use std::process::Command;
let force = args.get("force").and_then(|v| v.as_bool()).unwrap_or(false);
let pid = args.get("pid").and_then(|v| v.as_str());
@@ -1317,12 +1271,6 @@ pub async fn handle_kill_mt5_process(_config: &Config, args: &Value) -> Result<V
// Also kill wineserver if force=true
if force {
let _ = Command::new("killall").arg("wineserver").output();
// On macOS, also kill CrossOver processes
#[cfg(target_os = "macos")]
{
let _ = Command::new("killall").arg("cxstart").output();
}
}
}
@@ -1333,7 +1281,6 @@ pub async fn handle_kill_mt5_process(_config: &Config, args: &Value) -> Result<V
};
let result = json!({
"os_context": get_os_context(),
"killed": killed,
"failed": failed,
"force": force,
@@ -1348,10 +1295,9 @@ pub async fn handle_kill_mt5_process(_config: &Config, args: &Value) -> Result<V
/// Check system resources for MT5
pub async fn handle_check_system_resources(_config: &Config, _args: &Value) -> Result<Value> {
let _os_type = OsType::detect();
use std::process::Command;
let mut result = json!({
"os_context": get_os_context(),
"disk_space": null,
"memory": null,
"cpu_cores": 0,
@@ -1399,66 +1345,34 @@ pub async fn handle_check_system_resources(_config: &Config, _args: &Value) -> R
let vm_output = Command::new("vm_stat").output();
if let Ok(output) = vm_output {
let content = String::from_utf8_lossy(&output.stdout);
// Parse vm_stat output with improved robustness
// Parse vm_stat output
let mut free_pages = 0u64;
let mut active_pages = 0u64;
let mut inactive_pages = 0u64;
let mut wired_pages = 0u64;
let mut page_size = 4096u64;
// Try to get page size from vm_stat output
for line in content.lines() {
if line.contains("Mach Virtual Memory Statistics:") {
// Page size is typically 4096 on macOS
page_size = 4096;
}
}
for line in content.lines() {
// More robust parsing using regex-like pattern matching
if line.contains("Pages free:") {
if let Some(val) = line.split(':').nth(1) {
free_pages = val.trim().trim_end_matches('.').parse().unwrap_or(0);
}
free_pages = line.split_whitespace().nth(2).unwrap_or("0").trim_end_matches('.').parse().unwrap_or(0);
} else if line.contains("Pages active:") {
if let Some(val) = line.split(':').nth(1) {
active_pages = val.trim().trim_end_matches('.').parse().unwrap_or(0);
}
active_pages = line.split_whitespace().nth(2).unwrap_or("0").trim_end_matches('.').parse().unwrap_or(0);
} else if line.contains("Pages inactive:") {
if let Some(val) = line.split(':').nth(1) {
inactive_pages = val.trim().trim_end_matches('.').parse().unwrap_or(0);
}
} else if line.contains("Pages wired:") {
if let Some(val) = line.split(':').nth(1) {
wired_pages = val.trim().trim_end_matches('.').parse().unwrap_or(0);
}
inactive_pages = line.split_whitespace().nth(2).unwrap_or("0").trim_end_matches('.').parse().unwrap_or(0);
}
}
// Calculate memory more accurately
let total_pages = free_pages + active_pages + inactive_pages + wired_pages;
let total_mb = (total_pages * page_size) / 1024 / 1024;
let page_size = 4096u64;
let total_mb = ((free_pages + active_pages + inactive_pages) * page_size) / 1024 / 1024;
let free_mb = (free_pages * page_size) / 1024 / 1024;
let available_mb = ((free_pages + inactive_pages) * page_size) / 1024 / 1024;
result["memory"] = json!({
"total_mb": total_mb,
"free_mb": free_mb,
"available_mb": available_mb,
"active_mb": (active_pages * page_size) / 1024 / 1024,
"wired_mb": (wired_pages * page_size) / 1024 / 1024,
"page_size": page_size,
"unit": "MB",
});
// Adjust memory threshold for macOS (compressed memory makes free appear lower)
if available_mb < 1024 {
if free_mb < 2048 {
result["recommendations"].as_array_mut().unwrap().push(
json!(format!("Low memory available ({} MB free). MT5 may crash during large optimizations. Close other apps.", available_mb))
);
} else if available_mb < 2048 {
result["recommendations"].as_array_mut().unwrap().push(
json!(format!("Memory getting low ({} MB available). Consider closing other applications.", available_mb))
json!("Low memory available. MT5 may crash during large optimizations.")
);
}
}
@@ -1479,13 +1393,6 @@ pub async fn handle_check_system_resources(_config: &Config, _args: &Value) -> R
"free_mb": parts[3].parse::<u64>().unwrap_or(0),
"unit": "MB",
});
let free_mb = parts[3].parse::<u64>().unwrap_or(0);
if free_mb < 1024 {
result["recommendations"].as_array_mut().unwrap().push(
json!("Low memory available. MT5 may crash during large optimizations.")
);
}
}
}
}
@@ -1493,38 +1400,13 @@ pub async fn handle_check_system_resources(_config: &Config, _args: &Value) -> R
}
// Get CPU cores
#[cfg(target_os = "macos")]
{
let nproc_output = Command::new("sysctl")
.args(["-n", "hw.ncpu"])
.output();
if let Ok(output) = nproc_output {
let cores = String::from_utf8_lossy(&output.stdout).trim().parse().unwrap_or(0);
result["cpu_cores"] = json!(cores);
if cores < 4 {
result["recommendations"].as_array_mut().unwrap().push(
json!(format!("Low CPU core count ({} cores). Optimizations may be slow.", cores))
);
}
}
}
let nproc_output = Command::new("sysctl")
.args(["-n", "hw.ncpu"])
.output();
#[cfg(target_os = "linux")]
{
let nproc_output = Command::new("nproc").output();
if let Ok(output) = nproc_output {
let cores = String::from_utf8_lossy(&output.stdout).trim().parse().unwrap_or(0);
result["cpu_cores"] = json!(cores);
if cores < 4 {
result["recommendations"].as_array_mut().unwrap().push(
json!(format!("Low CPU core count ({} cores). Optimizations may be slow.", cores))
);
}
}
if let Ok(output) = nproc_output {
let cores = String::from_utf8_lossy(&output.stdout).trim().parse().unwrap_or(0);
result["cpu_cores"] = json!(cores);
}
}
@@ -1540,7 +1422,6 @@ pub async fn handle_validate_mt5_config(config: &Config, _args: &Value) -> Resul
.ok_or_else(|| anyhow::anyhow!("MT5 directory not configured"))?;
let mut result = json!({
"os_context": get_os_context(),
"terminal_ini": null,
"tester_ini": null,
"config_files_found": Vec::<String>::new(),
@@ -1624,7 +1505,6 @@ pub async fn handle_validate_mt5_config(config: &Config, _args: &Value) -> Resul
/// Get Wine prefix detailed information
pub async fn handle_get_wine_prefix_info(config: &Config, _args: &Value) -> Result<Value> {
let _os_type = OsType::detect();
let mt5_dir = config.mt5_dir()
.ok_or_else(|| anyhow::anyhow!("MT5 directory not configured"))?;
@@ -1634,7 +1514,6 @@ pub async fn handle_get_wine_prefix_info(config: &Config, _args: &Value) -> Resu
.and_then(|p| p.parent());
let mut result = json!({
"os_context": get_os_context(),
"prefix_path": null,
"exists": false,
"windows_version": null,
@@ -1730,7 +1609,6 @@ pub async fn handle_get_backtest_crash_info(config: &Config, args: &Value) -> Re
let hours_back = args.get("hours_back").and_then(|v| v.as_u64()).unwrap_or(6);
let mut result = json!({
"os_context": get_os_context(),
"crashes_found": Vec::<serde_json::Value>::new(),
"recent_failures": 0,
"common_patterns": Vec::<String>::new(),
@@ -1762,33 +1640,21 @@ pub async fn handle_get_backtest_crash_info(config: &Config, args: &Value) -> Re
}
}
// Check deal count in DB
let db = ReportDb::new(&Config::db_path());
if db.init().is_ok() {
match db.get_by_report_dir(dir) {
Ok(Some(entry)) => {
let deal_count = db.get_deals(&entry.id)
.map(|d| d.len())
.unwrap_or(0);
if deal_count == 0 {
result["crashes_found"].as_array_mut().unwrap().push(json!({
"report_dir": dir,
"type": "empty_deals",
"reason": "No deals stored in DB - EA did not trade or extraction failed",
}));
}
}
Ok(None) | Err(_) => {
// Not in DB at all means extraction never completed
let metrics_exists = path.join("metrics.json").exists();
if !metrics_exists {
result["crashes_found"].as_array_mut().unwrap().push(json!({
"report_dir": dir,
"type": "missing_deals",
"reason": "Report not found in DB and no metrics.json - backtest likely crashed before extraction",
}));
}
}
// Check if deals.csv is missing or empty
let deals_csv = path.join("deals.csv");
if !deals_csv.exists() {
result["crashes_found"].as_array_mut().unwrap().push(json!({
"report_dir": dir,
"type": "missing_deals",
"reason": "deals.csv not found - backtest likely failed",
}));
} else if let Ok(meta) = deals_csv.metadata() {
if meta.len() < 100 {
result["crashes_found"].as_array_mut().unwrap().push(json!({
"report_dir": dir,
"type": "empty_deals",
"reason": "deals.csv is nearly empty - no trades were made",
}));
}
}
}
@@ -1810,10 +1676,10 @@ pub async fn handle_get_backtest_crash_info(config: &Config, args: &Value) -> Re
if let Ok(modified) = meta.modified() {
if modified >= cutoff {
// Check for failure indicators
let has_metrics = path.join("metrics.json").exists();
let has_deals = path.join("deals.csv").exists();
let has_incomplete = path.join(".incomplete").exists();
if !has_metrics || has_incomplete {
if !has_deals || has_incomplete {
failures += 1;
}
}
@@ -1835,7 +1701,7 @@ pub async fn handle_get_backtest_crash_info(config: &Config, args: &Value) -> Re
if types.contains(&"missing_deals".to_string()) {
result["common_patterns"].as_array_mut().unwrap().push(
json!("Report not in DB and no metrics.json suggests MT5 crashed before extraction completed")
json!("Missing deals.csv suggests MT5 crashed during backtest")
);
}
if types.contains(&"incomplete".to_string()) {
-20
View File
@@ -1,20 +0,0 @@
/// Read a file that may be UTF-16LE (with BOM) or UTF-8, returning a UTF-8 String.
/// MT5 .set and .ini files are typically UTF-16LE with BOM (0xFF 0xFE).
pub fn read_file_as_utf8(path: &std::path::Path) -> anyhow::Result<String> {
let bytes = std::fs::read(path)?;
// Check for UTF-16LE BOM (0xFF 0xFE)
if bytes.len() >= 2 && bytes[0] == 0xFF && bytes[1] == 0xFE {
// UTF-16LE with BOM - skip the 2-byte BOM and decode
let utf16_data: Vec<u16> = bytes[2..]
.chunks_exact(2)
.map(|chunk| u16::from_le_bytes([chunk[0], chunk[1]]))
.collect();
String::from_utf16(&utf16_data)
.map_err(|e| anyhow::anyhow!("Failed to decode UTF-16LE: {}", e))
} else {
// Try UTF-8
String::from_utf8(bytes)
.map_err(|e| anyhow::anyhow!("Failed to decode as UTF-8: {}", e))
}
}
-193
View File
@@ -1,193 +0,0 @@
#!/usr/bin/env python3
"""MCP test harness: run backtest then exercise all granular analysis tools."""
import json
import subprocess
import sys
import time
import threading
BINARY = "/Users/masdevid/.cargo/bin/mt5-quant"
TIMEOUT = 1200 # 20 min for backtest
def send(proc, msg):
line = json.dumps(msg) + "\n"
proc.stdin.write(line.encode())
proc.stdin.flush()
def recv(proc, timeout=30):
"""Read one JSON-RPC response line."""
proc.stdout.readline() # skip blank / notification lines
deadline = time.time() + timeout
while time.time() < deadline:
line = proc.stdout.readline().decode("utf-8", errors="replace").strip()
if not line:
continue
try:
return json.loads(line)
except json.JSONDecodeError:
pass # skip non-JSON lines (logs etc)
return None
def recv_with_id(proc, expected_id, timeout=1200):
"""Read lines until we get a response with the expected id."""
deadline = time.time() + timeout
while time.time() < deadline:
line = proc.stdout.readline().decode("utf-8", errors="replace").strip()
if not line:
continue
try:
msg = json.loads(line)
if msg.get("id") == expected_id:
return msg
except json.JSONDecodeError:
pass
return None
def tool_call(proc, call_id, name, arguments=None):
send(proc, {
"jsonrpc": "2.0",
"id": call_id,
"method": "tools/call",
"params": {"name": name, "arguments": arguments or {}}
})
def extract_text(resp):
if not resp:
return "NO RESPONSE"
result = resp.get("result", {})
content = result.get("content", [])
if content:
text = content[0].get("text", "")
try:
parsed = json.loads(text)
return json.dumps(parsed, indent=2)[:1500]
except Exception:
return text[:1500]
if "error" in resp:
return f"ERROR: {resp['error']}"
return str(resp)[:500]
def ok(resp):
if not resp:
return False
result = resp.get("result", {})
content = result.get("content", [])
is_error = result.get("isError", True)
if is_error:
return False
if content:
text = content[0].get("text", "")
try:
parsed = json.loads(text)
return parsed.get("success", True) is not False
except Exception:
return True
return not is_error
# ── Start server ──────────────────────────────────────────────────────────────
print("Starting mt5-quant MCP server...")
proc = subprocess.Popen(
[BINARY],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
)
# ── Handshake ─────────────────────────────────────────────────────────────────
send(proc, {
"jsonrpc": "2.0", "id": 0,
"method": "initialize",
"params": {
"protocolVersion": "2024-11-05",
"capabilities": {},
"clientInfo": {"name": "test-harness", "version": "1.0"}
}
})
init_resp = recv_with_id(proc, 0, timeout=10)
if not init_resp:
print("FATAL: no initialize response")
proc.terminate()
sys.exit(1)
print("Initialized OK")
send(proc, {"jsonrpc": "2.0", "method": "notifications/initialized", "params": {}})
# ── Run backtest ──────────────────────────────────────────────────────────────
print("\n=== run_backtest (DPS21, XAUUSD, M5, OHLC, skip_compile) ===")
tool_call(proc, 1, "run_backtest", {
"expert": "DPS21",
"symbol": "XAUUSDc",
"timeframe": "M5",
"model": 1, # OHLC faster
"skip_compile": True,
"timeout": 900,
"startup_delay_secs": 15,
})
resp = recv_with_id(proc, 1, timeout=TIMEOUT)
print(extract_text(resp))
backtest_ok = ok(resp)
print(f"backtest_ok={backtest_ok}")
if not backtest_ok:
print("Backtest failed aborting analysis tests")
proc.terminate()
sys.exit(1)
# ── Full analysis ─────────────────────────────────────────────────────────────
print("\n=== analyze_report (latest, all analytics) ===")
tool_call(proc, 2, "analyze_report", {})
resp = recv_with_id(proc, 2, timeout=60)
print(extract_text(resp))
print(f"analyze_report ok={ok(resp)}")
# ── Granular analytics ────────────────────────────────────────────────────────
granular = [
("analyze_monthly_pnl", {}),
("analyze_drawdown_events", {}),
("analyze_top_losses", {"limit": 5}),
("analyze_loss_sequences", {}),
("analyze_position_pairs", {}),
("analyze_direction_bias", {}),
("analyze_streaks", {}),
("analyze_concurrent_peak", {}),
("analyze_profit_distribution", {}),
("analyze_time_performance", {}),
("analyze_hold_time_distribution",{}),
("analyze_layer_performance", {}),
("analyze_volume_vs_profit", {}),
("analyze_costs", {}),
("analyze_efficiency", {}),
("list_deals", {"limit": 5}),
("search_deals_by_comment", {"query": "tp"}),
("search_deals_by_magic", {"magic": "0"}),
]
results = {}
for i, (tool, args) in enumerate(granular, start=10):
print(f"\n=== {tool} ===")
tool_call(proc, i, tool, args)
resp = recv_with_id(proc, i, timeout=30)
status = "OK" if ok(resp) else "FAIL"
results[tool] = status
print(f" {status}")
if status == "FAIL":
print(" " + extract_text(resp)[:400])
# ── Summary ───────────────────────────────────────────────────────────────────
print("\n\n========== SUMMARY ==========")
passed = [t for t, s in results.items() if s == "OK"]
failed = [t for t, s in results.items() if s != "OK"]
print(f"PASSED ({len(passed)}): {', '.join(passed)}")
if failed:
print(f"FAILED ({len(failed)}): {', '.join(failed)}")
else:
print("All granular analytics PASSED")
proc.terminate()