gavindiaz/mt5-quant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update documentation for new debugging and diagnostic tools

Browse Source
This commit is contained in:
Devid HW
2026-04-22 12:24:12 +07:00
parent 41344b14b8
commit b97011d5b6
3 changed files with 79 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+27 -15
View File
@@ -23,24 +23,26 @@ MT5-Quant/
│ │ ├── metrics.rs # Metrics parsing from HTML/XML
│ │ └── report.rs # Report, PipelineMetadata, etc.
│ ├── analytics/ # Report extraction & analysis (migrated from Python)
│ │ ├── extract.rs # HTML/XML report parser → metrics.json + deals.csv
│ │ ├── extract.rs # HTML/XML report parser → metrics.json (deals → DB)
│ │ └── analyze.rs # Deal-level analysis engine → analysis.json
│ ├── compile/ # MQL5 compilation
│ │ └── mql_compiler.rs # MetaEditor wrapper (Wine/CrossOver)
│ ├── pipeline/ # Backtest orchestration
│ │ ├── backtest.rs # 5-stage pipeline (COMPILE→CLEAN→BACKTEST→EXTRACT→ANALYZE)
│ │ └── stages.rs # Pipeline stage definitions
│ ├── storage/ # SQLite persistence
│ │ └── database.rs # ReportDb: reports table + deals table
│ └── tools/ # MCP tool definitions
│ ├── definitions/ # Tool schemas (9 domain modules, 89 tools)
│ ├── definitions/ # Tool schemas (9 domain modules, 90 tools)
│ │ ├── mod.rs
│ │ ├── analytics.rs # 9 analysis tools
│ │ ├── backtest.rs # 4 backtest tools
│ │ ├── analytics.rs # 19 analysis tools (DB-backed)
│ │ ├── backtest.rs # 7 backtest tools
│ │ ├── baseline.rs # 1 baseline tool
│ │ ├── experts.rs # 4 EA/indicator/script tools
│ │ ├── experts.rs # 9 EA/indicator/script tools
│ │ ├── optimization.rs # 4 optimization tools
│ │ ├── reports.rs # 11 report management tools
│ │ ├── reports.rs # 20 report management tools
│ │ ├── setfiles.rs # 8 .set file tools
│ │ └── system.rs # 3 system tools
│ │ └── system.rs # 6 system tools
│ └── handlers/ # Tool dispatch (9 domain modules)
│ ├── mod.rs
│ ├── analysis.rs
@@ -145,17 +147,26 @@ MT5 runs in headless mode, writes the report, and exits.
---
### Stage 4: EXTRACT
### Stage 4: EXTRACT + STORE
Single HTML/XML parse pass that produces three artifacts:
Single HTML/XML parse pass. Deals go directly into the SQLite database; the raw report file is deleted afterwards.
```rust
// src/analytics/extract.rs
let extractor = ReportExtractor::new();
let result = extractor.extract(&report_path, &output_dir)?;
// → metrics.json (aggregate summary)
// → deals.csv (all deals, 13 columns)
// → deals.json (same data, JSON)
// → metrics.json (aggregate summary — written to report_dir)
// HTML report deleted after extraction
// src/storage/database.rs
db.insert_deals(&report_id, &result.deals)?;
// → deals table in SQLite (all deals, keyed by report_id)
```
On-demand CSV export is available via the `export_deals_csv` tool:
```
export_deals_csv(report_id: "20260422_051041_DPS21_XAUUSDc_M5_1")
// → report_dir/deals.csv (written only when explicitly requested)
```
**Why single-pass?** MT5 HTML reports are large (1-5MB for 14-month tests). Each regex pass over the file takes ~200ms. The old pipeline ran 5 separate grep/regex passes. The Rust implementation uses a single-pass parser: 5× faster and no partial-read inconsistencies.
@@ -175,12 +186,13 @@ if ext == "xml" || path.ends_with(".htm.xml") {
}
```
**Deal columns (13):**
**Deal columns (stored in DB):**
```
Time | Type | Direction | Volume | Price | S/L | T/P | Profit | Balance | Comment | Order | Magic | Entry
time | deal | symbol | deal_type | entry | volume | price | order_id
commission | swap | profit | balance | comment | magic
```
The `Comment` column is the key to grid analytics. The EA writes `"Layer #3"`, `"Locking Total"`, `"Zombie Exit"` etc. Pattern matching on comments reconstructs which position was at which layer.
The `comment` column is the key to grid analytics. The EA writes `"Layer #3"`, `"Locking Total"`, `"Zombie Exit"` etc. Pattern matching on comments reconstructs which position was at which layer.
---
docs/MCP_TOOLS.md
+35 -14
View File
@@ -84,8 +84,8 @@ Run a complete backtest pipeline: compile → clean cache → backtest → extra
files: {
metrics_json: string;
analysis_json: string;
deals_csv: string;
deals_json: string;
// Note: deals are stored in SQLite DB, not on disk.
// Call export_deals_csv(report_id) to generate a CSV file on demand.
};
error?: string; // Present on failure
@@ -128,9 +128,7 @@ Run a complete backtest pipeline: compile → clean cache → backtest → extra
},
"files": {
"metrics_json": "reports/20250619_143022_MyEA_XAUUSD_M5/metrics.json",
"analysis_json": "reports/20250619_143022_MyEA_XAUUSD_M5/analysis.json",
"deals_csv": "reports/20250619_143022_MyEA_XAUUSD_M5/deals.csv",
"deals_json": "reports/20250619_143022_MyEA_XAUUSD_M5/deals.json"
"analysis_json": "reports/20250619_143022_MyEA_XAUUSD_M5/analysis.json"
}
}
```
@@ -157,7 +155,7 @@ Same as `run_backtest`.
Backtest only: clean cache → backtest → extract. No analysis phase.
**When to call:** When you just need raw trade data (deals.csv) and don't need analytics. Fastest option for batch processing.
**When to call:** When you just need raw trade data (stored in DB) and don't need analytics. Fastest option for batch processing. Use `export_deals_csv` afterwards if you need a CSV file.
### Input schema
@@ -484,13 +482,15 @@ Delete old report directories to reclaim disk space, keeping the most recent N r
## `analyze_report`
Read and summarize a completed backtest report without re-running MT5.
Read and summarize a completed backtest report without re-running MT5. Loads deals from the SQLite database.
### Input schema
```typescript
{
report_dir: string; // Path to report directory
report_id?: string; // Preferred: report ID from list_reports
report_dir?: string; // Legacy: path to report directory (looks up DB entry)
// Omit both to use the latest report automatically
strategy?: "grid" | "scalper" | "trend" | "hedge" | "generic";
// Strategy profile that was used (default: "grid").
// Only affects interpretation of analysis.json fields —
@@ -3057,6 +3057,22 @@ Find comparable reports (same EA/symbol/timeframe).
---
## Granular Analytics — Common Input Pattern
All granular analytics tools below (`analyze_monthly_pnl` through `analyze_efficiency`, plus `list_deals` and `search_deals_*`) share the same report resolution logic:
```typescript
{
report_id?: string; // Preferred: ID from list_reports
report_dir?: string; // Legacy: filesystem path (looks up DB entry)
// Omit both → uses the latest report automatically
}
```
Deals are loaded from **SQLite DB**, not from CSV files. The `report_dir` parameter is kept for backward compatibility — if your report is in the DB, passing `report_dir` will resolve to its `report_id` automatically.
---
## `analyze_monthly_pnl`
Monthly P/L breakdown only.
@@ -3067,8 +3083,10 @@ Monthly P/L breakdown only.
```typescript
{
report_dir: string; // Path to report directory
report_id?: string; // Preferred: from list_reports
report_dir?: string; // Legacy: path to report directory
}
// Omit all args to use the latest report
```
### Output schema
@@ -3333,7 +3351,7 @@ Peak simultaneous positions.
## `list_deals`
List individual deals with filters (type, profit range, volume, dates).
List individual deals with filters (type, profit range, volume, dates). Loads from SQLite DB.
**When to call:** To query individual trades with specific criteria.
@@ -3341,7 +3359,9 @@ List individual deals with filters (type, profit range, volume, dates).
```typescript
{
report_dir: string;
report_id?: string; // Preferred: from list_reports
report_dir?: string; // Legacy: path to report directory
// Omit all to use latest report
deal_type?: "buy" | "sell" | "all";
min_profit?: number;
max_profit?: number;
@@ -3377,7 +3397,7 @@ List individual deals with filters (type, profit range, volume, dates).
## `search_deals_by_comment`
Full-text search in deal comments (e.g., "Layer #3").
Full-text search in deal comments (e.g., "Layer #3"). Loads from SQLite DB.
**When to call:** To find deals with specific comment patterns.
@@ -3385,8 +3405,9 @@ Full-text search in deal comments (e.g., "Layer #3").
```typescript
{
report_dir: string;
query: string; // Search pattern in comment field
report_id?: string; // Preferred: from list_reports
report_dir?: string; // Legacy: path to report directory
query: string; // Search pattern in comment field (required)
limit?: number;
}
```
docs/TROUBLESHOOTING.md
+17
View File
@@ -82,6 +82,23 @@ Check `<terminal_dir>/MQL5/Logs/`:
- Check `.set` file values appropriate for symbol/broker
- Confirm `OnInit()` returns `INIT_SUCCEEDED` (MT5 Journal tab)
## Analytics Tools: "No reports in DB" or "Report not found"
Analytics tools load deals from the **SQLite database**, not from CSV files on disk. Resolution order:
1. `report_id` (preferred) — ID from `list_reports`
2. `report_dir` (legacy) — filesystem path, looks up matching DB entry
3. No args — uses the latest report automatically
Pre-DB reports (before this version) won't be found by `report_dir`. Re-run the backtest to get a DB-backed report.
**`deals.csv` is no longer written automatically.** Call `export_deals_csv` to generate one on demand:
```
export_deals_csv() # latest report → report_dir/deals.csv
export_deals_csv(report_id: "20260422_…") # specific report
export_deals_csv(output_path: "/tmp/out.csv") # custom path
```
## Optimization Never Finishes
```bash