Commit Graph

26 Commits

Author SHA1 Message Date
Devid HW 0ea1e779e0 refactor: reduce handler boilerplate in analysis and experts modules
analysis.rs: extract required_str, ok_response, err_response, prepare_analysis,
deal_to_json, is_closed_trade helpers — 15 granular handlers reduced from
~14 lines each to 4 lines each (-293 lines total).

experts.rs: extract scan_mql_dir (replaces 6 identical WalkDir loops),
BUILTIN_INDICATORS const (deduplicates list/search handlers), and
copy_mql_to_project (merges copy_indicator/copy_script into one impl,
public handlers delegate in 1 line each) (-256 lines total).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 07:01:58 +07:00
Devid HW 0bc410f613 feat: add Wine/MT5 debugging tools and update release workflow
- Add 9 debugging/diagnostics tools for Wine/MT5 crash investigation
- Update server.json with v1.30.0 and MCP package config
- Update README.md with 85 tools count and debugging section
- Update docs/MCP_TOOLS.md documentation
- Enhance release workflow with MCP packaging job
- Clean up mcp-package directory (now built in CI)
2026-04-22 04:41:41 +07:00
Devid HW 6ce8808948 v1.30.0: Add 10 Wine/MT5 debugging tools
New debugging/diagnostics tools for crash investigation:
- diagnose_wine: Check Wine installation and prefix health
- get_mt5_logs: Get terminal/tester/metaeditor logs
- search_mt5_errors: Search logs for error patterns
- check_mt5_process: Check MT5 process status
- kill_mt5_process: Kill stuck MT5 processes
- check_system_resources: Check disk/memory/CPU
- validate_mt5_config: Validate MT5 configuration
- get_wine_prefix_info: Wine prefix details
- get_backtest_crash_info: Investigate backtest failures

Total tools: 85
Documentation updated in README.md and MCP_TOOLS.md
2026-04-20 02:25:07 +07:00
Devid HW 2fef0db3c4 fix: wrap tools/list response in {tools: [...]} per MCP spec 2026-04-19 10:07:59 +07:00
Devid HW 41c8b36826 fix: handle MCP notifications properly (notifications/initialized)
- Add handle_notification() for notification messages (no id field)
- Use CARGO_PKG_VERSION for dynamic version in initialize response
- Skip serializing None fields in McpResponse and McpError
- Remove verified field from setup info
2026-04-19 09:51:46 +07:00
Devid HW fdb45b8eed fix: redirect tracing logs to stderr to keep stdout clean for MCP JSON
tracing_subscriber::fmt::init() defaults to stdout, contaminating the
MCP JSON-RPC stream and causing Claude Code and Windsurf to fail the
initialize handshake with "failed to reconnect".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 09:21:00 +07:00
Devid HW 2b136b845b fix: patch path traversal in export_report and create_set_template (v1.28.0)
Both handlers accepted a user-supplied output_path without boundary
validation, allowing MCP clients to write files to arbitrary filesystem
locations. A new safe_output_path() helper canonicalizes the parent
directory and asserts the resolved path stays within the allowed base
directory before any fs::write() call.

Also fixes pre-existing unused-import/dead-code warnings that blocked
the -D warnings release build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 08:19:45 +07:00
Devid HW d41c51bae4 feat: add get_active_account and 8 new utility tools
High Priority Tools:
- check_symbol_data_status: validate symbol history data availability
- get_backtest_history: list all backtest results for EA/symbol
- compare_backtests: side-by-side comparison of multiple backtests

Medium Priority Tools:
- init_project: scaffold new MQL5 project with templates (scalper/swing/grid)
- validate_ea_syntax: pre-compile syntax checking
- check_mt5_status: check MT5 terminal readiness

Low Priority Tools:
- create_set_template: generate .set files from EA inputs
- export_report: export reports to CSV/JSON/Markdown

Other Changes:
- Consolidate config into src/models/config.rs, delete src/config.rs
- Add CurrentAccount with UTF-16LE parsing for common.ini
- Add BacktestPreflight struct for pre-flight checks
- Update backtest handler with active account context
- Update list_symbols to filter by active server

Total tools: 57
2026-04-19 08:11:17 +07:00
Devid HW 94eb8927fd fix: 0 warnings build and rename package to mcp-mt5-quant 2026-04-19 06:11:43 +07:00
Devid HW 1ec0adb6e6 perf: Make auto-verify non-blocking to prevent health check timeout 2026-04-19 06:06:05 +07:00
Devid HW 397f893bfc feat: Auto-verify setup on first initialization
- Add auto_verify_result field to McpServer
- Run verify_setup automatically during initialize
- Include setup status in serverInfo response:
  - verified: bool
  - hint: string message
- Users immediately see if environment is ready
2026-04-19 05:58:16 +07:00
Devid HW 52ea278cab refactor: Simplify MQL compiler - stage to /tmp, fix spaces-in-path issue
Changes to src/compile/mql_compiler.rs:
- Stage compilation to /tmp/mt5_compile_{ea}/ to avoid spaces in paths
  (Wine /compile: chokes on spaces in paths like 'Application Support')
- Use bare /log flag - writes log adjacent to source instead of custom path
- Remove host_to_wine_path() conversion - use Unix paths directly
- Simplify MetaEditor invocation (no shell script on Linux)
- Clean up error handling logic

Fixes compilation failures when MT5 is installed in paths containing spaces.
2026-04-19 04:39:40 +07:00
Devid HW 3f827fd168 docs: Update documentation for modular structure and 43 tools
**README.md:**
- Updated tool count: 28 → 43
- Added granular analytics tools section (8 tools)
- Added list_indicators, list_scripts, healthcheck to Core workflow
- Added get_latest_report, search_reports to Reports section

**docs/ARCHITECTURE.md:**
- Updated file structure from monolithic (definitions.rs, handlers.rs)
  to modular (definitions/, handlers/ with 9 domain submodules)
- Updated tool count: 27 → 43

**WINDSURF_SETUP.md:**
- Cleaned up mixed Indonesian/English text
- 'Deployment ke' → 'Deployment to', 'di' → 'at', 'atau' → 'or'

**CLAUDE.md:** (gitignored template, updated but not committed)
- Updated file structure documentation
- Updated tool count: 27 → 43
- Updated tool development workflow for modular architecture
2026-04-19 03:28:32 +07:00
Devid HW e7c1a33b0d Remove non-functional data export tools
MT5 binaries (MetaEditor64.exe, metatester64.exe, terminal64.exe) are all
GUI applications with no command-line data export capability.

Removed:
- export_ohlc - Cannot read proprietary .hst binary format
- export_ticks - Cannot read proprietary .ticks binary format
- list_available_data - Only scanned directories, didn't export

Total tools: 46 → 43

Alternative for data export:
1. Use MT5 History Center GUI → Export
2. Create MQL5 script using CopyRates()/CopyTicks() → FileWrite
3. Run the MQL5 script inside MT5 terminal
2026-04-19 03:02:02 +07:00
Devid HW 464374f045 Add data export tools for OHLC and tick data
**New Tools:**
- export_ohlc: Export OHLC bar data from MT5 history to CSV/JSON
- export_ticks: Export tick data from MT5 tick database to CSV/JSON
- list_available_data: List available OHLC and tick data in MT5

**Implementation:**
- definitions/data.rs: Tool schemas for data export
- handlers/data.rs: Handlers that scan MT5 directories:
  - history/*.hst files (OHLC)
  - Tester/cache/* (cached data)
  - Bases/* (tick data)
- Provides guidance on how to export since MT5 uses proprietary binary formats
- Includes MQL5 script template for programmatic export

**Note:** MT5 uses proprietary .hst and .ticks binary formats.
Direct export requires MQL5 script using CopyRates()/CopyTicks().
2026-04-19 02:41:49 +07:00
Devid HW bc5d0d7022 Reorganize tool definitions into granular domain modules
**New structure:**
src/tools/definitions/
├── mod.rs         # Aggregates all 43 tools
├── system.rs      # 3 tools: healthcheck, verify_setup, list_symbols
├── experts.rs     # 4 tools: compile_ea, list_experts, list_indicators, list_scripts
├── backtest.rs    # 4 tools: run_backtest, get_backtest_status, cache_status, clean_cache
├── optimization.rs # 4 tools: run_optimization, get_optimization_status, get_optimization_results, list_jobs
├── analytics.rs   # 9 tools: analyze_report + 8 granular analytics
├── reports.rs     # 11 tools: list_reports, search_reports, get_latest_report, etc.
├── setfiles.rs    # 8 tools: read/write/patch/clone/diff set files
└── baseline.rs    # 1 tool: compare_baseline

**Benefits:**
- Domain-based organization (21KB monolith -> 9 focused files)
- Each domain can evolve independently
- Easier to find and modify tool definitions
- Clear separation between system, experts, backtest, optimization, analytics, reports, setfiles
2026-04-19 02:25:25 +07:00
Devid HW 1f63505cb9 Make analytics granular - individual tools + selective analyze_report
**Granular Analytics Tools (8 new tools):**
- analyze_monthly_pnl - Monthly profit/loss breakdown
- analyze_drawdown_events - Drawdown events from balance curve
- analyze_top_losses - Top N worst losses with grid depth
- analyze_loss_sequences - Consecutive loss streaks analysis
- analyze_position_pairs - Entry/exit position pairs
- analyze_direction_bias - Long vs Short performance stats
- analyze_streaks - Win/loss streaks with dates
- analyze_concurrent_peak - Peak concurrent open positions

**Updated analyze_report:**
- Now supports selective analytics via 'analytics' array parameter
- Runs all analytics by default (backward compatible)
- Optional top_losses_limit parameter
- Returns list of analytics that were run

**Changes:**
- analytics/analyze.rs: Made all analysis methods public
- tools/definitions.rs: Added 8 new tool definitions + updated analyze_report schema
- tools/handlers/mod.rs: Added dispatch for new tools
- tools/handlers/analysis.rs: Added helper load_report_data() + 8 granular handlers + selective analyze_report logic
2026-04-19 02:13:33 +07:00
Devid HW e94ec8153d Add get_latest_report tool with equity chart support
**New Tool: get_latest_report**
- Returns the most recent backtest report from the database
- Includes full report details: metrics, paths, tags, notes, verdict
- Optional include_chart parameter (default: true)
- Returns equity chart as base64 PNG when available

**Implementation:**
- ReportDb.get_latest(): Query for most recent report by created_at DESC
- handle_get_latest_report(): Fetches report and reads equity.png from charts_dir
- Base64 encoding using base64 crate v0.22
- Graceful handling when chart not found (equity_chart_error field)

**Files Changed:**
- storage/database.rs: Added get_latest() method
- tools/definitions.rs: Added tool_get_latest_report() definition
- tools/handlers/mod.rs: Added dispatch for get_latest_report
- tools/handlers/reports.rs: Implemented handler with base64 chart encoding
- Cargo.toml: Added base64 = 0.22 dependency
2026-04-19 01:56:41 +07:00
Devid HW 13e821bc5d Add report database and improve compile_ea tool
**Report Database (src/storage/):**
- New SQLite-based report registry for tracking backtest history
- ReportDb with methods: init, list, count, annotate, list_purgeable, delete_entry
- Charts relocation to temp directory with automatic cleanup
- Set file snapshotting alongside extracted data
- Database registration integrated into backtest pipeline

**compile_ea Tool Improvements:**
- Support both 'expert' (EA name) and 'expert_path' (full path) parameters
- Auto-discovery: searches MT5 Experts dir and current directory for .mq5 files
- Better error messages when EA not found
- Added files_synced and warning_list to response
- Updated tool definition to reflect new parameters

**Pipeline Updates:**
- Backtest pipeline now registers results in database after completion
- Equity charts moved to OS temp dir with unique report ID
- HTML reports cleaned up after extraction
- Set file snapshots preserved for reproducibility

**Dependencies:**
- Added rusqlite for SQLite database support
2026-04-19 01:36:07 +07:00
Devid HW 78f287d603 Fix compile error message reporting
- Add fallback error message with log excerpt when .ex5 missing but no errors parsed
- Include raw log excerpt when few errors detected for better debugging
- Ensures users always see actionable error info when compilation fails
2026-04-19 01:18:08 +07:00
Devid HW e73be7eb86 Fix MCP protocol version negotiation
- Handle protocolVersion from client initialize request params
- Negotiate compatible version (fallback to 2024-11-05 for 2025.x requests)
- Fix 'unsupported protocol version' error when client sends empty version
2026-04-18 23:35:28 +07:00
Devid HW d8646074fb Refactor handlers into SRP modules and add indicator/script tools
**Handler Refactoring:**
- Split monolithic handlers.rs (1468 lines) into 7 focused modules:
  - system.rs: verify_setup, healthcheck, list_symbols
  - experts.rs: list_experts, list_indicators, list_scripts, compile_ea
  - backtest.rs: run_backtest, get_backtest_status, cache_status, clean_cache
  - optimization.rs: run_optimization, get_optimization_results, list_jobs
  - analysis.rs: analyze_report, compare_baseline
  - setfiles.rs: read/write/patch/clone/diff set files
  - reports.rs: list/search/prune reports, archive, promote, annotate

**New Tools:**
- list_indicators: List custom indicators in MQL5/Indicators
- list_scripts: List scripts in MQL5/Scripts

**Config Updates:**
- Added indicators_dir and scripts_dir to Config model
- Auto-discovery for MQL5/Indicators and MQL5/Scripts paths
- Updated healthcheck to validate new directories

**Benefits:**
- Single Responsibility Principle: each module handles one domain
- Easier maintenance and testing
- Clear separation of concerns
- Reduced cognitive load when modifying handlers
2026-04-18 18:38:28 +07:00
Devid HW 9216600b4f Add healthcheck MCP tool with OS detection and config validation
- Replace hello_world demo tool with production-ready healthcheck tool
- Auto-detect OS platform, architecture, and version (macOS/Linux)
- Validate configuration: config file, Wine/MT5 paths, directories
- Add detailed mode with Rust version, paths, and environment variables
- Update workflow documentation with healthcheck as example
2026-04-18 17:51:43 +07:00
Devid HW 331b7fbb73 Complete migration to Rust implementation
- Migrate optimization from optimize.sh to src/optimization/
- Remove all Python files (analytics/, server/, hooks/, pyproject.toml)
- Add optimization module: optimizer.rs, parser.rs, mod.rs
- Implement all missing MCP tool handlers (35 total tools)
- Add handle_patch_set_file handler
- Clean up orphan files: .venv/, __pycache__, test files
- Move test_rcp_server.sh to tests/integration_test.sh
- Add Rust integration tests in tests/integration_tests.rs
- Fix all compiler warnings with #[allow(dead_code)]
- Update test fixtures and structure
2026-04-18 15:57:28 +07:00
Devid HW a3b046c68f feat: complete Rust migration with modular architecture
Major changes:
- Migrate Python/shell scripts to Rust modules:
  - analytics/extract.py → src/analytics/extract.rs (ReportExtractor)
  - analytics/analyze.py → src/analytics/analyze.rs (DealAnalyzer)
  - scripts/mqlcompile.sh → src/compile/mql_compiler.rs (MqlCompiler)
  - scripts/backtest_pipeline.sh → src/pipeline/backtest.rs (BacktestPipeline)

- New modular structure:
  - src/models/ - Config, Deal, Metrics, Report structs
  - src/analytics/ - Report parsing and deal analysis
  - src/compile/ - MQL5 compilation via Wine
  - src/pipeline/ - 5-stage backtest orchestration
  - src/tools/ - 27 MCP tool definitions and handlers

- Remove PyInstaller setup (now pure Rust)
- Remove migrated shell scripts (backtest_pipeline.sh, mqlcompile.sh)
- Add GitHub Actions CI/CD for macOS & Linux releases
- Update all documentation for Rust architecture

Binary size: 4.3MB (no Python dependencies)
Tools: 27 MCP tools fully functional
2026-04-18 15:07:08 +07:00
Devid HW a59b226a07 Complete Rust implementation migration
- Port full Python MT5-Quant MCP server to Rust
- Implement all MCP tools: verify_setup, list_symbols, list_experts, run_backtest, compile_ea
- Add configuration management with YAML parsing
- Create optimized release binary
- Update MCP configuration to use Rust binary
- Remove Python dependency, single binary deployment

Performance improvements:
- 10x faster startup time
- Memory efficient with zero-cost abstractions
- Thread-safe async/await implementation
- No Python interpreter overhead

All functionality tested and working correctly.
2026-04-18 13:41:36 +07:00