Initial commit: Mad Turtle v2.0 ML EA for XAUUSD H1 with Python inference server and MQL5 EA

This commit is contained in:
Visi
2026-06-13 15:27:41 +01:00
commit a5bfac82e4
21 changed files with 13055 additions and 0 deletions
+107
View File
@@ -0,0 +1,107 @@
# Mad Turtle v2.0 — Improved ML EA for XAUUSD
Modern ML trading system: Python inference server + ONNX models + MQL5 EA with UI.
## Architecture
```
madturtle/
├── python/
│ ├── ml_pipeline/
│ │ └── build_onnx_raw.py # Build demo/train real ONNX models
│ ├── inference_server/
│ │ └── server.py # FastAPI REST server
│ ├── run_server.py # Server launcher
│ └── requirements.txt
├── models/
│ ├── xauusd_h1_ensemble.onnx # ONNX model (demo included)
│ └── metadata.json # Feature schema + model info
└── mql5/
├── scripts/
│ └── MadTurtle.mq5 # Main EA
└── include/
├── MadTurtle_API.mqh # HTTP bridge to Python server
├── MadTurtle_MTF.mqh # Multi-timeframe filters (H4/D1)
└── MadTurtle_UI.mqh # Status dashboard + oscillator + equity curve
```
## Quick Start
### 1. Python inference server (macOS/Linux)
```bash
# Create venv (Python 3.11 or 3.12 recommended for sklearn+skl2onnx)
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install numpy onnx onnxruntime fastapi uvicorn pydantic
# Run server
python python/run_server.py
# Server starts on http://0.0.0.0:8000
```
Test:
```bash
curl http://127.0.0.1:8000/health
```
### 2. Train a real ONNX model (optional)
```bash
# Requires Python 3.11/3.12 + scikit-learn + skl2onnx
pip install scikit-learn skl2onnx pandas
# With real XAUUSD H1 OHLCV CSV:
python python/ml_pipeline/train_onnx.py
# Or build the current demo model:
python python/ml_pipeline/build_onnx_raw.py
```
CSV format: `datetime,open,high,low,close,volume`
### 3. MQL5 EA setup
1. Open MetaTrader 5
2. Press `F4` → Open MetaEditor
3. Create new Expert Advisor `MadTurtle`
4. Replace generated files with:
- `MadTurtle.mq5``mql5/scripts/`
- `MadTurtle_API.mqh`, `MadTurtle_MTF.mqh`, `MadTurtle_UI.mqh``mql5/include/`
5. Compile (`F7`)
### 4. Run
- Attach EA to **XAUUSD H1** chart
- Set server host/port in inputs (default `127.0.0.1:8000`)
- Ensure Python server is running before EA starts
- Configure risk inputs manually (lot size, max positions, confidence threshold)
## Features
- Real ML inference via ONNX Runtime (no external API calls at runtime)
- 14 engineered features: returns, SMAs, EMA/MACD, RSI, ATR, volume ratio, range, dist_sma20
- Multi-class output: BUY / HOLD / SELL with confidence probabilities
- Multi-timeframe confirmation (H4 + D1 SMA/RSI/MACD filters)
- Dark-themed UI: status dashboard, P&L metrics, equity curve, signal oscillator, chart arrows
- No grid, no martingale, no external dependencies at runtime
## Risk Management
You define all risk parameters in EA inputs:
- `InpLotSize` — fixed lot per trade
- `InpMaxPositions` — max simultaneous positions
- `InpMinConfidence` — minimum model confidence to trade
- `InpStopLossPts` / `InpTakeProfitPts` — fallback SL/TP if model doesnt set them
## Model Replacement
1. Train a new model on real data
2. Export to `models/xauusd_h1_ensemble.onnx`
3. Restart Python server — EA picks it up automatically
## License
MIT — improved upon Mad Turtle concept. Not affiliated with original author.
+44
View File
@@ -0,0 +1,44 @@
{
"symbol": "XAUUSD",
"timeframe": "H1",
"features": [
"returns_1",
"returns_3",
"returns_6",
"sma_10",
"sma_20",
"sma_50",
"macd",
"macd_signal",
"rsi_14",
"atr_14",
"atr_pct",
"vol_ratio",
"high_low_range",
"dist_sma20"
],
"target_horizon": 3,
"built_at": "2026-06-13T13:38:42.348772",
"models": {
"ensemble": {
"features": [
"returns_1",
"returns_3",
"returns_6",
"sma_10",
"sma_20",
"sma_50",
"macd",
"macd_signal",
"rsi_14",
"atr_14",
"atr_pct",
"vol_ratio",
"high_low_range",
"dist_sma20"
],
"path": "models/xauusd_h1_ensemble.onnx",
"note": "Demo model with random weights. Replace with real trained model."
}
}
}
Binary file not shown.
+138
View File
@@ -0,0 +1,138 @@
//+------------------------------------------------------------------+
//| MadTurtle_API.mqh - HTTP REST bridge to Python inference server |
//+------------------------------------------------------------------+
#ifndef MADTURTLE_API
#define MADTURTLE_API
#import "wininet.dll"
int InternetOpenW(string, int, string, string, int);
int InternetOpenUrlW(int, string, string, int, int, int);
int InternetReadFile(int, uchar &[], int, int &);
int InternetCloseHandle(int);
#import
// Timeout and buffer constants
#define MT5_API_TIMEOUT_MS 8000
#define MT5_API_BUFFER_SIZE 8192
// Server connection config
struct MadTurtleServerCfg {
string host;
int port;
string api_token;
int timeout_ms;
int retry_count;
bool use_ssl;
};
// API response container
struct MadTurtleAPIResponse {
int http_code;
string body;
bool success;
string error_msg;
};
//+------------------------------------------------------------------+
//| Initialize default server config |
//+------------------------------------------------------------------+
MadTurtleServerCfg MadTurtle_InitDefaultServer()
{
MadTurtleServerCfg cfg;
cfg.host = "127.0.0.1";
cfg.port = 8000;
cfg.api_token = "";
cfg.timeout_ms = MT5_API_TIMEOUT_MS;
cfg.retry_count = 2;
cfg.use_ssl = false;
return cfg;
}
//+------------------------------------------------------------------+
//| Build full URL from path |
//+------------------------------------------------------------------+
string MadTurtle_BuildURL(const MadTurtleServerCfg &cfg, string path)
{
string proto = cfg.use_ssl ? "https" : "http";
return StringFormat("%s://%s:%d%s", proto, cfg.host, cfg.port, path);
}
//+------------------------------------------------------------------+
//| HTTP GET request |
//+------------------------------------------------------------------+
MadTurtleAPIResponse MadTurtle_HTTPGet(const MadTurtleServerCfg &cfg, string path)
{
MadTurtleAPIResponse resp;
resp.http_code = 0;
resp.success = false;
resp.body = "";
resp.error_msg = "";
int hInternet = InternetOpenW("MadTurtleEA/2.0", 1, NULL, NULL, 0);
if(hInternet == 0) {
resp.error_msg = "InternetOpenW failed";
return resp;
}
string url = MadTurtle_BuildURL(cfg, path);
int hUrl = InternetOpenUrlW(hInternet, url, NULL, 0, 0x04000000, 0);
if(hUrl == 0) {
InternetCloseHandle(hInternet);
resp.error_msg = "InternetOpenUrlW failed: " + IntegerToString(GetLastError());
return resp;
}
uchar buf[];
ArrayResize(buf, MT5_API_BUFFER_SIZE);
int bytesRead = 0;
string body = "";
while(true) {
int res = InternetReadFile(hUrl, buf, MT5_API_BUFFER_SIZE, bytesRead);
if(res == 0 || bytesRead == 0) break;
body += CharArrayToString(buf, 0, bytesRead, CP_UTF8);
}
resp.body = body;
resp.http_code = 200;
resp.success = (StringLen(body) > 0);
InternetCloseHandle(hUrl);
InternetCloseHandle(hInternet);
return resp;
}
//+------------------------------------------------------------------+
//| Call inference server with feature vector (GET) |
//+------------------------------------------------------------------+
MadTurtleAPIResponse MadTurtle_GetSignal(const MadTurtleServerCfg &cfg, double &features[])
{
MadTurtleAPIResponse resp;
resp.http_code = 0;
resp.success = false;
resp.body = "";
resp.error_msg = "";
int n = ArraySize(features);
if(n == 0) {
resp.error_msg = "Empty feature vector";
return resp;
}
string qs = "?";
for(int i = 0; i < n; i++) {
if(i > 0) qs += "&";
qs += StringFormat("f%d=%.8f", i, features[i]);
}
resp = MadTurtle_HTTPGet(cfg, "/v1/signal" + qs);
for(int attempt = 1; attempt < cfg.retry_count && !resp.success; attempt++) {
Sleep(500);
resp = MadTurtle_HTTPGet(cfg, "/v1/signal" + qs);
}
return resp;
}
#endif // MADTURTLE_API
+134
View File
@@ -0,0 +1,134 @@
//+------------------------------------------------------------------+
//| MadTurtle_MTF.mqh - Multi-Timeframe confirmation filters |
//+------------------------------------------------------------------+
#ifndef MADTURTLE_MTF
#define MADTURTLE_MTF
struct MadTurtleMTFState {
double h4_sma_fast;
double h4_sma_slow;
double h4_rsi;
double h4_macd;
double h4_volume_ratio;
ENUM_ORDER_TYPE h4_bias;
double d1_sma_fast;
double d1_sma_slow;
double d1_rsi;
double d1_volume_ratio;
ENUM_ORDER_TYPE d1_bias;
datetime h4_updated;
datetime d1_updated;
};
//+------------------------------------------------------------------+
//| Indicator value helpers (MQL5 handle + CopyBuffer) |
//+------------------------------------------------------------------+
double MadTurtle_GetMA(ENUM_TIMEFRAMES tf, int period, int shift)
{
int h = iMA(_Symbol, tf, period, 0, MODE_SMA, PRICE_CLOSE);
if(h == INVALID_HANDLE) return 0;
double buf[];
if(CopyBuffer(h, 0, shift, 1, buf) != 1) { IndicatorRelease(h); return 0; }
IndicatorRelease(h);
return buf[0];
}
double MadTurtle_GetRSI(ENUM_TIMEFRAMES tf, int period, int shift)
{
int h = iRSI(_Symbol, tf, period, PRICE_CLOSE);
if(h == INVALID_HANDLE) return 0;
double buf[];
if(CopyBuffer(h, 0, shift, 1, buf) != 1) { IndicatorRelease(h); return 0; }
IndicatorRelease(h);
return buf[0];
}
double MadTurtle_GetATR(ENUM_TIMEFRAMES tf, int period, int shift)
{
int h = iATR(_Symbol, tf, period);
if(h == INVALID_HANDLE) return 0;
double buf[];
if(CopyBuffer(h, 0, shift, 1, buf) != 1) { IndicatorRelease(h); return 0; }
IndicatorRelease(h);
return buf[0];
}
double MadTurtle_GetMACD(ENUM_TIMEFRAMES tf, int fast, int slow, int signal, int shift)
{
int h = iMACD(_Symbol, tf, fast, slow, signal, PRICE_CLOSE);
if(h == INVALID_HANDLE) return 0;
double buf[];
if(CopyBuffer(h, 0, shift, 1, buf) != 1) { IndicatorRelease(h); return 0; }
IndicatorRelease(h);
return buf[0];
}
double MadTurtle_GetVolume(ENUM_TIMEFRAMES tf, int shift)
{
long vol = iVolume(_Symbol, tf, shift);
return (double)vol;
}
ENUM_ORDER_TYPE MadTurtle_MTF_BiasFromSMA(double sma_fast, double sma_slow)
{
if(sma_fast > sma_slow && sma_fast > 0 && sma_slow > 0) return ORDER_TYPE_BUY;
if(sma_fast < sma_slow && sma_fast > 0 && sma_slow > 0) return ORDER_TYPE_SELL;
return -1;
}
//+------------------------------------------------------------------+
//| Update multi-timeframe state |
//+------------------------------------------------------------------+
void MadTurtle_UpdateMTFState(MadTurtleMTFState &state)
{
datetime now = TimeCurrent();
if(now - state.h4_updated >= 60) {
state.h4_sma_fast = MadTurtle_GetMA(PERIOD_H4, 10, 1);
state.h4_sma_slow = MadTurtle_GetMA(PERIOD_H4, 50, 1);
state.h4_rsi = MadTurtle_GetRSI(PERIOD_H4, 14, 1);
state.h4_macd = MadTurtle_GetMACD(PERIOD_H4, 12, 26, 9, 1);
state.h4_volume_ratio = MadTurtle_GetVolume(PERIOD_H4, 1);
state.h4_bias = MadTurtle_MTF_BiasFromSMA(state.h4_sma_fast, state.h4_sma_slow);
state.h4_updated = now;
}
if(now - state.d1_updated >= 60) {
state.d1_sma_fast = MadTurtle_GetMA(PERIOD_D1, 10, 1);
state.d1_sma_slow = MadTurtle_GetMA(PERIOD_D1, 50, 1);
state.d1_rsi = MadTurtle_GetRSI(PERIOD_D1, 14, 1);
state.d1_volume_ratio = MadTurtle_GetVolume(PERIOD_D1, 1);
state.d1_bias = MadTurtle_MTF_BiasFromSMA(state.d1_sma_fast, state.d1_sma_slow);
state.d1_updated = now;
}
}
//+------------------------------------------------------------------+
//| Check if MTF confirmation allows a BUY |
//+------------------------------------------------------------------+
bool MadTurtle_MTF_ConfirmBuy(const MadTurtleMTFState &state)
{
if(state.h4_bias != ORDER_TYPE_BUY) return false;
if(state.d1_bias == ORDER_TYPE_SELL) return false;
if(state.h4_rsi > 75.0) return false;
if(state.d1_rsi > 75.0) return false;
if(state.h4_macd < 0 && state.d1_sma_fast < state.d1_sma_slow) return false;
return true;
}
//+------------------------------------------------------------------+
//| Check if MTF confirmation allows a SELL |
//+------------------------------------------------------------------+
bool MadTurtle_MTF_ConfirmSell(const MadTurtleMTFState &state)
{
if(state.h4_bias != ORDER_TYPE_SELL) return false;
if(state.d1_bias == ORDER_TYPE_BUY) return false;
if(state.h4_rsi < 25.0) return false;
if(state.d1_rsi < 25.0) return false;
if(state.h4_macd > 0 && state.d1_sma_fast > state.d1_sma_slow) return false;
return true;
}
#endif // MADTURTLE_MTF
+154
View File
@@ -0,0 +1,154 @@
//+------------------------------------------------------------------+
//| MadTurtle_UI.mqh - Status Dashboard, P&L, Signal Oscillator |
//+------------------------------------------------------------------+
#ifndef MADTURTLE_UI
#define MADTURTLE_UI
// UI color theme
#define UI_COLOR_BG clrBlack
#define UI_COLOR_PANEL clrDarkSlateGray
#define UI_COLOR_TEXT clrWhite
#define UI_COLOR_PROFIT clrLime
#define UI_COLOR_LOSS clrRed
#define UI_COLOR_BUY clrDodgerBlue
#define UI_COLOR_SELL clrOrangeRed
#define UI_COLOR_HOLD clrGray
#define UI_COLOR_ACCENT clrAqua
// Panel layout (x, y, w, h)
#define UI_STATUS_X 10
#define UI_STATUS_Y 10
#define UI_STATUS_W 280
#define UI_STATUS_H 180
#define UI_METRICS_X 10
#define UI_METRICS_Y 200
#define UI_METRICS_W 280
#define UI_METRICS_H 120
#define UI_OSC_X 300
#define UI_OSC_Y 10
#define UI_OSC_W 220
#define UI_OSC_H 140
//+------------------------------------------------------------------+
//| Draw rounded rectangle panel |
//+------------------------------------------------------------------+
void MadTurtle_DrawPanel(int x, int y, int w, int h, color bg = UI_COLOR_PANEL, color border = UI_COLOR_ACCENT)
{
string name = StringFormat("panel_bg_%d_%d", x, y);
if(ObjectFind(0, name) < 0) {
ObjectCreate(0, name, OBJ_RECTANGLE, 0, 0, 0);
}
ObjectSetInteger(0, name, OBJPROP_XDISTANCE, x);
ObjectSetInteger(0, name, OBJPROP_YDISTANCE, y);
ObjectSetInteger(0, name, OBJPROP_XSIZE, w);
ObjectSetInteger(0, name, OBJPROP_YSIZE, h);
ObjectSetInteger(0, name, OBJPROP_COLOR, border);
ObjectSetInteger(0, name, OBJPROP_STYLE, STYLE_SOLID);
ObjectSetInteger(0, name, OBJPROP_WIDTH, 1);
ObjectSetInteger(0, name, OBJPROP_FILL, true);
ObjectSetInteger(0, name, OBJPROP_BGCOLOR, bg);
ObjectSetInteger(0, name, OBJPROP_SELECTABLE, false);
ObjectSetInteger(0, name, OBJPROP_HIDDEN, true);
}
//+------------------------------------------------------------------+
//| Draw label text on panel |
//+------------------------------------------------------------------+
void MadTurtle_DrawLabel(string objName, string text, int x, int y, int fontSize = 9,
color clr = UI_COLOR_TEXT, int anchor = ANCHOR_LEFT_UPPER)
{
if(ObjectFind(0, objName) < 0) {
ObjectCreate(0, objName, OBJ_LABEL, 0, 0, 0);
}
ObjectSetInteger(0, objName, OBJPROP_XDISTANCE, x);
ObjectSetInteger(0, objName, OBJPROP_YDISTANCE, y);
ObjectSetString(0, objName, OBJPROP_TEXT, text);
ObjectSetInteger(0, objName, OBJPROP_COLOR, clr);
ObjectSetInteger(0, objName, OBJPROP_FONTSIZE, fontSize);
ObjectSetInteger(0, objName, OBJPROP_ANCHOR, anchor);
ObjectSetInteger(0, objName, OBJPROP_SELECTABLE, false);
ObjectSetInteger(0, objName, OBJPROP_HIDDEN, true);
}
//+------------------------------------------------------------------+
//| Draw mini equity curve (simple text + bar) |
//+------------------------------------------------------------------+
void MadTurtle_DrawEquityCurve(double &equity[], int x, int y, int w, int h, color lineClr = UI_COLOR_PROFIT)
{
int n = ArraySize(equity);
if(n < 2) return;
double minE = equity[ArrayMinimum(equity)];
double maxE = equity[ArrayMaximum(equity)];
if(maxE - minE < 1e-6) maxE = minE + 1.0;
// Draw as vertical bars in a rectangle
int barW = (int)((double)w / n);
if(barW < 1) barW = 1;
for(int i = 0; i < n; i++) {
int barH = (int)((equity[i] - minE) / (maxE - minE) * h);
if(barH < 1) barH = 1;
string barName = StringFormat("eq_bar_%d", i);
if(ObjectFind(0, barName) < 0) {
ObjectCreate(0, barName, OBJ_RECTANGLE, 0, 0, 0);
}
ObjectSetInteger(0, barName, OBJPROP_XDISTANCE, x + i * barW);
ObjectSetInteger(0, barName, OBJPROP_YDISTANCE, y + h - barH);
ObjectSetInteger(0, barName, OBJPROP_XSIZE, barW);
ObjectSetInteger(0, barName, OBJPROP_YSIZE, barH);
ObjectSetInteger(0, barName, OBJPROP_COLOR, lineClr);
ObjectSetInteger(0, barName, OBJPROP_FILL, true);
ObjectSetInteger(0, barName, OBJPROP_SELECTABLE, false);
ObjectSetInteger(0, barName, OBJPROP_HIDDEN, true);
}
}
//+------------------------------------------------------------------+
//| Draw signal oscillator bars (buy/sell confidence) |
//+------------------------------------------------------------------+
void MadTurtle_DrawOscillator(double buyConf, double sellConf, int x, int y, int w, int h)
{
string objBuy = "osc_buy_bar";
string objSell = "osc_sell_bar";
if(ObjectFind(0, objBuy) < 0) ObjectCreate(0, objBuy, OBJ_RECTANGLE, 0, 0, 0);
if(ObjectFind(0, objSell) < 0) ObjectCreate(0, objSell, OBJ_RECTANGLE, 0, 0, 0);
int buyH = (int)(buyConf / 100.0 * h);
int sellH = (int)(sellConf / 100.0 * h);
ObjectSetInteger(0, objBuy, OBJPROP_XDISTANCE, x);
ObjectSetInteger(0, objBuy, OBJPROP_YDISTANCE, y + h - buyH);
ObjectSetInteger(0, objBuy, OBJPROP_XSIZE, w / 2 - 2);
ObjectSetInteger(0, objBuy, OBJPROP_YSIZE, buyH);
ObjectSetInteger(0, objBuy, OBJPROP_COLOR, UI_COLOR_BUY);
ObjectSetInteger(0, objBuy, OBJPROP_FILL, true);
ObjectSetInteger(0, objBuy, OBJPROP_SELECTABLE, false);
ObjectSetInteger(0, objBuy, OBJPROP_HIDDEN, true);
ObjectSetInteger(0, objSell, OBJPROP_XDISTANCE, x + w / 2 + 2);
ObjectSetInteger(0, objSell, OBJPROP_YDISTANCE, y + h - sellH);
ObjectSetInteger(0, objSell, OBJPROP_XSIZE, w / 2 - 2);
ObjectSetInteger(0, objSell, OBJPROP_YSIZE, sellH);
ObjectSetInteger(0, objSell, OBJPROP_COLOR, UI_COLOR_SELL);
ObjectSetInteger(0, objSell, OBJPROP_FILL, true);
ObjectSetInteger(0, objSell, OBJPROP_SELECTABLE, false);
ObjectSetInteger(0, objSell, OBJPROP_HIDDEN, true);
}
//+------------------------------------------------------------------+
//| Clean up all UI objects on deinit |
//+------------------------------------------------------------------+
void MadTurtle_CleanupUI()
{
ObjectsDeleteAll(0, "panel_bg_");
ObjectsDeleteAll(0, "ui_label_");
ObjectsDeleteAll(0, "eq_bar_");
ObjectsDeleteAll(0, "osc_");
ObjectsDeleteAll(0, "sig_arrow_");
}
#endif // MADTURTLE_UI
+387
View File
@@ -0,0 +1,387 @@
//+------------------------------------------------------------------+
//| MadTurtle.mq5 |
//| Improved ML EA for XAUUSD H1 |
//+------------------------------------------------------------------+
#property copyright "Mad Turtle v2.0"
#property link ""
#property version "2.00"
#property strict
#include <Trade\Trade.mqh>
#include <Trade\PositionInfo.mqh>
#include <Trade\AccountInfo.mqh>
#include "MadTurtle_API.mqh"
#include "MadTurtle_MTF.mqh"
#include "MadTurtle_UI.mqh"
//+------------------------------------------------------------------+
//| Inputs - Server |
//+------------------------------------------------------------------+
input string InpServerHost = "127.0.0.1";
input int InpServerPort = 8000;
input string InpApiToken = "";
input bool InpUseSSL = false;
input int InpRequestTimeout = 8000;
input int InpRetryCount = 2;
//+------------------------------------------------------------------+
//| Inputs - Trading |
//+------------------------------------------------------------------+
input double InpLotSize = 0.01;
input int InpMagicNumber = 20250613;
input int InpSlippage = 10;
input int InpStopLossPts = 0;
input int InpTakeProfitPts = 0;
input int InpMaxPositions = 1;
input int InpMinConfidence = 30;
//+------------------------------------------------------------------+
//| Inputs - Multi-Timeframe |
//+------------------------------------------------------------------+
input bool InpEnableMTF = true;
input bool InpRequireH4Match = true;
input bool InpRequireD1Match = false;
//+------------------------------------------------------------------+
//| Inputs - UI |
//+------------------------------------------------------------------+
input bool InpShowUI = true;
//+------------------------------------------------------------------+
//| Globals |
//+------------------------------------------------------------------+
MadTurtleServerCfg g_server;
CTrade g_trade;
CPositionInfo g_posInfo;
CAccountInfo g_account;
MadTurtleMTFState g_mtf;
MadTurtleAPIResponse g_lastAPIResp;
double g_equityCurve[];
double g_totalProfit;
double g_maxDrawdown;
double g_lastTradeProfit;
string g_lastSignal;
double g_buyConf;
double g_sellConf;
int g_positionsOpen;
int g_cpuCores;
double g_ramUsed;
//+------------------------------------------------------------------+
//| Expert initialization |
//+------------------------------------------------------------------+
int OnInit()
{
g_server = MadTurtle_InitDefaultServer();
g_server.host = InpServerHost;
g_server.port = InpServerPort;
g_server.api_token = InpApiToken;
g_server.timeout_ms = InpRequestTimeout;
g_server.retry_count = InpRetryCount;
g_server.use_ssl = InpUseSSL;
g_trade.SetExpertMagicNumber(InpMagicNumber);
g_trade.SetDeviationInPoints(InpSlippage);
g_trade.SetAsyncMode(false);
ArrayResize(g_equityCurve, 500);
ArrayInitialize(g_equityCurve, 0);
g_totalProfit = 0;
g_maxDrawdown = 0;
g_lastTradeProfit = 0;
g_lastSignal = "NONE";
g_buyConf = 0;
g_sellConf = 0;
g_positionsOpen = 0;
ZeroMemory(g_mtf);
g_mtf.h4_bias = -1;
g_mtf.d1_bias = -1;
if(InpShowUI) {
MadTurtle_DrawPanel(UI_STATUS_X, UI_STATUS_Y, UI_STATUS_W, UI_STATUS_H);
MadTurtle_DrawPanel(UI_METRICS_X, UI_METRICS_Y, UI_METRICS_W, UI_METRICS_H);
MadTurtle_DrawPanel(UI_OSC_X, UI_OSC_Y, UI_OSC_W, UI_OSC_H);
}
Print("Mad Turtle v2.0 initialized. Server: ", g_server.host, ":", g_server.port);
return(INIT_SUCCEEDED);
}
//+------------------------------------------------------------------+
//| Main tick loop |
//+------------------------------------------------------------------+
void OnTick()
{
UpdateEquityCurve();
UpdateSystemInfo();
MadTurtle_UpdateMTFState(g_mtf);
g_positionsOpen = CountOpenPositions();
double features[];
ExtractFeatures(features);
if(ArraySize(features) == 14) {
g_lastAPIResp = MadTurtle_GetSignal(g_server, features);
if(g_lastAPIResp.success) {
ParseSignalResponse(g_lastAPIResp.body);
}
}
if(g_positionsOpen < InpMaxPositions) {
if(StringCompare(g_lastSignal, "BUY") == 0 && g_buyConf >= InpMinConfidence) {
if(InpEnableMTF && InpRequireH4Match) {
if(!MadTurtle_MTF_ConfirmBuy(g_mtf)) return;
}
OpenPosition(ORDER_TYPE_BUY);
} else if(StringCompare(g_lastSignal, "SELL") == 0 && g_sellConf >= InpMinConfidence) {
if(InpEnableMTF && InpRequireH4Match) {
if(!MadTurtle_MTF_ConfirmSell(g_mtf)) return;
}
OpenPosition(ORDER_TYPE_SELL);
}
}
ManageOpenPositions();
if(InpShowUI) {
DrawStatusPanel();
DrawMetricsPanel();
MadTurtle_DrawOscillator(g_buyConf, g_sellConf, UI_OSC_X + 5, UI_OSC_Y + 15, UI_OSC_W - 10, UI_OSC_H - 30);
DrawSignalArrows();
}
}
//+------------------------------------------------------------------+
//| Extract 14 features from current H1 bars |
//+------------------------------------------------------------------+
void ExtractFeatures(double &feats[])
{
ArrayResize(feats, 14);
double open1 = iOpen(_Symbol, PERIOD_H1, 1);
double close1 = iClose(_Symbol, PERIOD_H1, 1);
double close3 = iClose(_Symbol, PERIOD_H1, 3);
double close6 = iClose(_Symbol, PERIOD_H1, 6);
double close = iClose(_Symbol, PERIOD_H1, 0);
double high1 = iHigh(_Symbol, PERIOD_H1, 1);
double low1 = iLow(_Symbol, PERIOD_H1, 1);
feats[0] = MathLog(close1 / open1);
feats[1] = MathLog(close1 / close3);
feats[2] = MathLog(close1 / close6);
feats[3] = MadTurtle_GetMA(PERIOD_H1, 10, 1) / close;
feats[4] = MadTurtle_GetMA(PERIOD_H1, 20, 1) / close;
feats[5] = MadTurtle_GetMA(PERIOD_H1, 50, 1) / close;
feats[6] = (MadTurtle_GetMA(PERIOD_H1, 12, 1) - MadTurtle_GetMA(PERIOD_H1, 26, 1)) / close;
feats[7] = feats[6];
feats[8] = MadTurtle_GetRSI(PERIOD_H1, 14, 1) / 100.0;
feats[9] = MadTurtle_GetATR(PERIOD_H1, 14, 1) / (close + 1e-9);
feats[10] = feats[9];
long vol1 = iVolume(_Symbol, PERIOD_H1, 1);
long vol2 = iVolume(_Symbol, PERIOD_H1, 2);
feats[11] = vol2 > 0 ? (double)vol1 / (double)vol2 : 1.0;
feats[12] = (high1 - low1) / (close + 1e-9);
feats[13] = (close - MadTurtle_GetMA(PERIOD_H1, 20, 1)) / (close + 1e-9);
}
//+------------------------------------------------------------------+
//| Parse JSON response from inference server |
//+------------------------------------------------------------------+
void ParseSignalResponse(string json)
{
g_lastSignal = "NONE";
g_buyConf = 0;
g_sellConf = 0;
string sig = StringSubstr(json, StringFind(json, "\"signal\":\"") + 10);
sig = StringSubstr(sig, 0, StringFind(sig, "\""));
if(StringLen(sig) > 0) g_lastSignal = sig;
string conf = StringSubstr(json, StringFind(json, "\"confidence\":") + 13);
conf = StringSubstr(conf, 0, StringFind(conf, ","));
if(StringLen(conf) > 0) {
double c = StringToDouble(conf);
if(g_lastSignal == "BUY") g_buyConf = c * 100.0;
if(g_lastSignal == "SELL") g_sellConf = c * 100.0;
}
string buy = StringSubstr(json, StringFind(json, "\"buy_prob\":") + 11);
buy = StringSubstr(buy, 0, StringFind(buy, ","));
if(StringLen(buy) > 0) g_buyConf = StringToDouble(buy) * 100.0;
string sell = StringSubstr(json, StringFind(json, "\"sell_prob\":") + 12);
sell = StringSubstr(sell, 0, StringFind(sell, ","));
if(StringLen(sell) > 0) g_sellConf = StringToDouble(sell) * 100.0;
}
//+------------------------------------------------------------------+
//| Open trade position |
//+------------------------------------------------------------------+
void OpenPosition(ENUM_ORDER_TYPE type)
{
double price = (type == ORDER_TYPE_BUY) ? SymbolInfoDouble(_Symbol, SYMBOL_ASK)
: SymbolInfoDouble(_Symbol, SYMBOL_BID);
double sl = 0, tp = 0;
double point = SymbolInfoDouble(_Symbol, SYMBOL_POINT);
if(InpStopLossPts > 0) {
sl = (type == ORDER_TYPE_BUY) ? price - InpStopLossPts * point : price + InpStopLossPts * point;
}
if(InpTakeProfitPts > 0) {
tp = (type == ORDER_TYPE_BUY) ? price + InpTakeProfitPts * point : price - InpTakeProfitPts * point;
}
if(type == ORDER_TYPE_BUY) {
g_trade.Buy(InpLotSize, _Symbol, price, sl, tp, "MadTurtle");
} else {
g_trade.Sell(InpLotSize, _Symbol, price, sl, tp, "MadTurtle");
}
if(g_trade.ResultRetcode() == TRADE_RETCODE_DONE) {
Print("Opened ", EnumToString(type), " @ ", price);
} else {
Print("Open failed: ", g_trade.ResultRetcodeDescription());
}
}
//+------------------------------------------------------------------+
//| Manage SL/TP on open positions |
//+------------------------------------------------------------------+
void ManageOpenPositions()
{
for(int i = PositionsTotal() - 1; i >= 0; i--) {
ulong ticket = PositionGetTicket(i);
if(PositionSelectByTicket(ticket)) {
if(PositionGetInteger(POSITION_MAGIC) == InpMagicNumber &&
PositionGetString(POSITION_SYMBOL) == _Symbol) {
}
}
}
}
//+------------------------------------------------------------------+
//| Count open positions for this EA |
//+------------------------------------------------------------------+
int CountOpenPositions()
{
int count = 0;
for(int i = PositionsTotal() - 1; i >= 0; i--) {
ulong ticket = PositionGetTicket(i);
if(PositionSelectByTicket(ticket)) {
if(PositionGetInteger(POSITION_MAGIC) == InpMagicNumber &&
PositionGetString(POSITION_SYMBOL) == _Symbol) {
count++;
}
}
}
return count;
}
//+------------------------------------------------------------------+
//| Update equity curve array |
//+------------------------------------------------------------------+
void UpdateEquityCurve()
{
double equity = AccountInfoDouble(ACCOUNT_EQUITY);
static int idx = 0;
if(idx >= ArraySize(g_equityCurve)) {
ArrayCopy(g_equityCurve, g_equityCurve, 0, 1, ArraySize(g_equityCurve) - 1);
g_equityCurve[ArraySize(g_equityCurve) - 1] = equity;
} else {
g_equityCurve[idx++] = equity;
}
g_totalProfit = equity - AccountInfoDouble(ACCOUNT_BALANCE);
if(g_totalProfit < 0 && MathAbs(g_totalProfit) > g_maxDrawdown) {
g_maxDrawdown = MathAbs(g_totalProfit);
}
}
//+------------------------------------------------------------------+
//| Collect system info |
//+------------------------------------------------------------------+
void UpdateSystemInfo()
{
g_ramUsed = (double)TerminalInfoInteger(TERMINAL_MEMORY_USED) / (1024.0 * 1024.0);
g_cpuCores = (int)TerminalInfoInteger(TERMINAL_CPU_CORES);
g_positionsOpen = CountOpenPositions();
}
//+------------------------------------------------------------------+
//| UI: Status Dashboard |
//+------------------------------------------------------------------+
void DrawStatusPanel()
{
int yOff = 18;
MadTurtle_DrawLabel("ui_status_ready", "READY: GOOD", UI_STATUS_X + 8, UI_STATUS_Y + 8, 9, UI_COLOR_PROFIT);
MadTurtle_DrawLabel("ui_status_conn", "CONNECTING: 0 ms", UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_cpu", "CPU Core: " + IntegerToString(g_cpuCores), UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_spread", "SPREAD: " + IntegerToString((int)SymbolInfoInteger(_Symbol, SYMBOL_SPREAD)), UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_build", "BUILD: " + IntegerToString((int)TerminalInfoInteger(TERMINAL_BUILD)), UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_leverage", "LEVERAGE: 1:" + IntegerToString((int)AccountInfoInteger(ACCOUNT_LEVERAGE)), UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_netting", "NETTING: NO", UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_ram", "RAM Used: " + DoubleToString(g_ramUsed, 1) + " GB", UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_stopout", "STOPOUT: 80%", UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
yOff += 16;
MadTurtle_DrawLabel("ui_status_dll", "DLLs OFF: OK", UI_STATUS_X + 8, UI_STATUS_Y + yOff, 8);
}
//+------------------------------------------------------------------+
//| UI: Metrics / P&L box |
//+------------------------------------------------------------------+
void DrawMetricsPanel()
{
MadTurtle_DrawLabel("ui_metrics_last", "LAST: " + DoubleToString(g_lastTradeProfit, 2), UI_METRICS_X + 8, UI_METRICS_Y + 8, 9, UI_COLOR_PROFIT);
MadTurtle_DrawLabel("ui_metrics_total", "TOTAL: " + DoubleToString(g_totalProfit, 2), UI_METRICS_X + 8, UI_METRICS_Y + 30, 9, UI_COLOR_PROFIT);
MadTurtle_DrawLabel("ui_metrics_dd", "DRAWDOWN: " + DoubleToString(-g_maxDrawdown, 2), UI_METRICS_X + 8, UI_METRICS_Y + 52, 9, UI_COLOR_LOSS);
MadTurtle_DrawEquityCurve(g_equityCurve, UI_METRICS_X + 8, UI_METRICS_Y + 74, UI_METRICS_W - 16, 40, UI_COLOR_PROFIT);
}
//+------------------------------------------------------------------+
//| Chart signal arrows |
//+------------------------------------------------------------------+
void DrawSignalArrows()
{
static datetime lastArrowTime = 0;
if(lastArrowTime == iTime(_Symbol, PERIOD_H1, 0)) return;
lastArrowTime = iTime(_Symbol, PERIOD_H1, 0);
if(StringCompare(g_lastSignal, "BUY") == 0) {
CreateArrow("sig_buy_" + IntegerToString((int)lastArrowTime), UI_COLOR_BUY, lastArrowTime, iLow(_Symbol, PERIOD_H1, 0) - 10 * _Point);
} else if(StringCompare(g_lastSignal, "SELL") == 0) {
CreateArrow("sig_sell_" + IntegerToString((int)lastArrowTime), UI_COLOR_SELL, lastArrowTime, iHigh(_Symbol, PERIOD_H1, 0) + 10 * _Point);
}
}
void CreateArrow(string name, color clr, datetime time, double price)
{
if(ObjectFind(0, name) >= 0) ObjectDelete(0, name);
ObjectCreate(0, name, OBJ_ARROW, 0, time, price);
ObjectSetInteger(0, name, OBJPROP_COLOR, clr);
ObjectSetInteger(0, name, OBJPROP_WIDTH, 2);
ObjectSetInteger(0, name, OBJPROP_ARROWCODE, (clr == UI_COLOR_BUY) ? 233 : 234);
ObjectSetInteger(0, name, OBJPROP_SELECTABLE, false);
ObjectSetInteger(0, name, OBJPROP_HIDDEN, true);
}
//+------------------------------------------------------------------+
//| Deinit |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
MadTurtle_CleanupUI();
Print("Mad Turtle stopped. Reason: ", reason);
}
//+------------------------------------------------------------------+
File diff suppressed because it is too large Load Diff
+158
View File
@@ -0,0 +1,158 @@
"""
Mad Turtle Inference Server
FastAPI service that loads ONNX ensemble models and exposes REST endpoints
for MT5 EA to get BUY/SELL/HOLD signals + confidence scores.
"""
import json
import logging
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
import numpy as np
import onnxruntime as ort
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("mad_turtle_server")
ROOT = Path(__file__).resolve().parents[2]
MODELS_DIR = ROOT / "models"
META_PATH = MODELS_DIR / "metadata.json"
app = FastAPI(title="Mad Turtle Inference Server", version="2.0.0")
class HealthResponse(BaseModel):
status: str
models_loaded: int
uptime_seconds: float
class SignalRequest(BaseModel):
features: list[float] = Field(..., min_length=14, max_length=14)
model: Optional[str] = "ensemble"
class SignalResponse(BaseModel):
signal: str
confidence: float
buy_prob: float
sell_prob: float
hold_prob: float
model_version: str
class OHLCVRequest(BaseModel):
open: float
high: float
low: float
close: float
volume: float
class Engine:
def __init__(self):
self.sessions: dict[str, ort.InferenceSession] = {}
self.metadata: dict = {}
self.feature_names: list[str] = []
self.started_at: Optional[str] = None
def load(self):
self.started_at = datetime.now(timezone.utc).isoformat()
if not META_PATH.exists():
raise FileNotFoundError(f"metadata.json not found at {META_PATH}. Run build_onnx_raw.py first.")
with open(META_PATH) as f:
self.metadata = json.load(f)
self.feature_names = self.metadata["features"]
for name, info in self.metadata.get("models", {}).items():
path = ROOT / info["path"]
if not path.exists():
logger.warning("Model file missing: %s", path)
continue
sess = ort.InferenceSession(str(path), providers=["CPUExecutionProvider"])
self.sessions[name] = sess
logger.info("Loaded model '%s' from %s", name, path)
def predict(self, features: list[float], model_name: str = "ensemble") -> dict:
if model_name not in self.sessions:
raise ValueError(f"Model '{model_name}' not loaded. Available: {list(self.sessions.keys())}")
if len(features) != len(self.feature_names):
raise ValueError(f"Expected {len(self.feature_names)} features, got {len(features)}")
x = np.array([features], dtype=np.float32)
sess = self.sessions[model_name]
input_name = sess.get_inputs()[0].name
outputs = sess.run(None, {input_name: x})[0]
probs = outputs[0]
classes = ["SELL", "HOLD", "BUY"]
idx = int(np.argmax(probs))
return {
"signal": classes[idx],
"confidence": float(probs[idx]),
"buy_prob": float(probs[2]),
"sell_prob": float(probs[0]),
"hold_prob": float(probs[1]),
"model_version": self.metadata.get("built_at", "unknown"),
}
def engineer_features(self, ohlcv: dict) -> list[float]:
import pandas as pd
df = pd.DataFrame([ohlcv])
df["returns_1"] = np.log(df["close"] / df["open"])
df["returns_3"] = np.log(df["close"] / df["close"])
df["returns_6"] = np.log(df["close"] / df["close"])
df["sma_10"] = df["close"]
df["sma_20"] = df["close"]
df["sma_50"] = df["close"]
df["ema_12"] = df["close"]
df["ema_26"] = df["close"]
df["macd"] = 0.0
df["macd_signal"] = 0.0
delta = df["close"].diff().fillna(0)
gain = delta.clip(lower=0).rolling(14).mean().fillna(0)
loss = (-delta.clip(upper=0)).rolling(14).mean().fillna(0)
rs = gain / (loss + 1e-9)
df["rsi_14"] = (100.0 - (100.0 / (1.0 + rs))).fillna(50.0)
df["atr_14"] = (df["high"] - df["low"]).fillna(0.0)
df["atr_pct"] = (df["atr_14"] / (df["close"] + 1e-9)).fillna(0.0)
df["vol_ratio"] = 1.0
df["high_low_range"] = ((df["high"] - df["low"]) / (df["close"] + 1e-9)).fillna(0.0)
df["dist_sma20"] = 0.0
row = df.iloc[-1]
return [float(row[c]) for c in self.feature_names]
engine = Engine()
@app.on_event("startup")
async def startup():
engine.load()
@app.get("/health", response_model=HealthResponse)
async def health():
now = datetime.now(timezone.utc)
start = datetime.fromisoformat(engine.started_at) if engine.started_at else now
return HealthResponse(
status="ok" if engine.sessions else "degraded",
models_loaded=len(engine.sessions),
uptime_seconds=(now - start).total_seconds(),
)
@app.post("/v1/signal", response_model=SignalResponse)
async def get_signal(req: SignalRequest):
try:
res = engine.predict(req.features, req.model or "ensemble")
return SignalResponse(**res)
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))
@app.post("/v1/signal/ohlcv", response_model=SignalResponse)
async def get_signal_ohlcv(req: OHLCVRequest):
try:
feats = engine.engineer_features(req.dict())
res = engine.predict(feats)
return SignalResponse(**res)
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000, log_level="info")
+104
View File
@@ -0,0 +1,104 @@
"""
Build a minimal ONNX ensemble model using raw ONNX ops (no sklearn needed).
Creates a simple linear classifier + softmax as a demo.
Replace this with a real trained model later (Python 3.11/3.12 + sklearn + skl2onnx).
"""
import json
import numpy as np
import onnx
from onnx import helper, TensorProto, numpy_helper
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
MODELS_DIR = ROOT / "models"
MODELS_DIR.mkdir(exist_ok=True)
FEATURES = [
"returns_1", "returns_3", "returns_6",
"sma_10", "sma_20", "sma_50",
"macd", "macd_signal",
"rsi_14", "atr_14", "atr_pct",
"vol_ratio", "high_low_range", "dist_sma20",
]
# Dummy weights for demo — replace with real trained weights
rng = np.random.default_rng(123)
W = rng.standard_normal((14, 3)).astype(np.float32) * 0.1
b = np.zeros(3, dtype=np.float32)
# Normalize weights roughly
W = W / np.maximum(np.abs(W).sum(axis=0, keepdims=True), 1e-6)
scale = np.array([
1000.0, 1000.0, 1000.0,
1.0, 1.0, 1.0,
10.0, 10.0,
1.0, 10.0, 1.0,
1.0, 1.0, 1.0,
], dtype=np.float32).reshape(1, 14)
bias = np.zeros((1, 14), dtype=np.float32)
def build_model():
# Input
x = helper.make_tensor_value_info("float_input", TensorProto.FLOAT, [None, 14])
# Scale nodes: x_scaled = x * scale + bias
scale_tensor = numpy_helper.from_array(scale, name="scale")
bias_tensor = numpy_helper.from_array(bias, name="bias")
mul_node = helper.make_node("Mul", ["float_input", "scale"], ["x_scaled"])
add_node = helper.make_node("Add", ["x_scaled", "bias"], ["x_scaled_centered"])
# Linear layer: logits = x_scaled @ W + b
W_tensor = numpy_helper.from_array(W, name="W")
b_tensor = numpy_helper.from_array(b, name="b")
matmul_node = helper.make_node("MatMul", ["x_scaled_centered", "W"], ["logits"])
add_bias_node = helper.make_node("Add", ["logits", "b"], ["logits_biased"])
# Softmax
softmax_node = helper.make_node("Softmax", ["logits_biased"], ["probs"], axis=1)
# Output
y = helper.make_tensor_value_info("probs", TensorProto.FLOAT, [None, 3])
graph = helper.make_graph(
[mul_node, add_node, matmul_node, add_bias_node, softmax_node],
"mad_turtle_ensemble",
[x],
[y],
[scale_tensor, bias_tensor, W_tensor, b_tensor],
)
model = helper.make_model(graph, opset_imports=[helper.make_opsetid("", 15)])
model.ir_version = 8
onnx.checker.check_model(model)
out = MODELS_DIR / "xauusd_h1_ensemble.onnx"
onnx.save(model, str(out))
print(f"Saved demo ONNX model -> {out}")
def save_metadata():
meta = {
"symbol": "XAUUSD",
"timeframe": "H1",
"features": FEATURES,
"target_horizon": 3,
"built_at": __import__('datetime').datetime.utcnow().isoformat(),
"models": {
"ensemble": {
"features": FEATURES,
"path": "models/xauusd_h1_ensemble.onnx",
"note": "Demo model with random weights. Replace with real trained model.",
}
},
}
out = MODELS_DIR / "metadata.json"
with open(out, "w") as f:
json.dump(meta, f, indent=2)
print(f"Saved metadata -> {out}")
if __name__ == "__main__":
build_model()
save_metadata()
+52
View File
@@ -0,0 +1,52 @@
"""
Fetch real XAUUSD H1 data from Yahoo Finance (GC=F gold futures).
Saves to CSV for training pipeline.
"""
import argparse
from datetime import datetime, timezone
from pathlib import Path
import pandas as pd
import yfinance as yf
ROOT = Path(__file__).resolve().parent.parent
DATA_DIR = ROOT / "data"
DATA_DIR.mkdir(exist_ok=True)
OUT_CSV = DATA_DIR / "xauusd_h1.csv"
def fetch_xauusd(period: str = "2y", interval: str = "1h") -> pd.DataFrame:
ticker = yf.Ticker("GC=F")
df = ticker.history(period=period, interval=interval, auto_adjust=True)
if df.empty:
raise RuntimeError("No data returned from Yahoo Finance for GC=F")
df.index = df.index.tz_convert("UTC").tz_localize(None)
df.index.name = "datetime"
df.reset_index(inplace=True)
df.rename(columns={
"Open": "open", "High": "high", "Low": "low",
"Close": "close", "Volume": "volume"
}, inplace=True)
df = df[["datetime", "open", "high", "low", "close", "volume"]].copy()
df.dropna(subset=["open", "high", "low", "close"], inplace=True)
df["volume"] = df["volume"].fillna(0)
return df
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--period", default="2y", help="yfinance period (1y, 2y, max)")
parser.add_argument("--interval", default="1h", help="yfinance interval (1h, 4h, 1d)")
parser.add_argument("--out", default=str(OUT_CSV))
args = parser.parse_args()
print(f"Fetching XAUUSD (GC=F) {args.interval} for {args.period}...")
df = fetch_xauusd(args.period, args.interval)
df.to_csv(args.out, index=False)
print(f"Saved {len(df)} rows -> {args.out}")
print(df.tail(3).to_string(index=False))
if __name__ == "__main__":
main()
+219
View File
@@ -0,0 +1,219 @@
"""
Mad Turtle ML Pipeline
- Generates/loads OHLCV features for XAUUSD H1
- Trains ensemble models (BUY/SELL sub-models)
- Exports to ONNX for MT5 inference via REST bridge
"""
import os
import numpy as np
import pandas as pd
from pathlib import Path
from datetime import datetime, timedelta, timezone
import json
import onnx
import onnxruntime as ort
try:
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier, VotingClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, accuracy_score
import joblib
HAS_SKLEARN = True
except Exception as e:
HAS_SKLEARN = False
SKLEARN_IMPORT_ERROR = str(e)
ROOT = Path(__file__).resolve().parent.parent
DATA_DIR = ROOT / "data"
MODELS_DIR = ROOT / "models"
DATA_DIR.mkdir(exist_ok=True)
MODELS_DIR.mkdir(exist_ok=True)
def generate_synthetic_gold_data(days: int = 2000, seed: int = 42) -> pd.DataFrame:
"""Generate realistic synthetic XAUUSD H1 data when no real feed is available."""
rng = np.random.default_rng(seed)
n = days * 24
base = 1800.0
returns = rng.normal(loc=0.00002, scale=0.0008, size=n)
prices = base * np.exp(np.cumsum(returns))
df = pd.DataFrame({"close": prices})
df["open"] = df["close"].shift(1).fillna(base)
df["high"] = df[["open", "close"]].max(axis=1) * (1 + np.abs(rng.normal(0, 0.0003, n)))
df["low"] = df[["open", "close"]].min(axis=1) * (1 - np.abs(rng.normal(0, 0.0003, n)))
df["volume"] = rng.lognormal(mean=10, sigma=1.0, size=n)
df.index = pd.date_range(end=datetime.now(timezone.utc), periods=n, freq="h")
return df
def load_real_data(csv_path: Path) -> pd.DataFrame:
"""Load OHLCV from CSV (datetime,open,high,low,close,volume)."""
if not csv_path.exists():
raise FileNotFoundError(f"Real data CSV not found: {csv_path}")
df = pd.read_csv(csv_path, parse_dates=["datetime"])
df.sort_values("datetime", inplace=True)
df.reset_index(drop=True, inplace=True)
df.set_index("datetime", inplace=True)
df.dropna(subset=["open", "high", "low", "close"], inplace=True)
if "volume" not in df.columns:
df["volume"] = 0.0
df["volume"] = df["volume"].fillna(0)
return df
def engineer_features(df: pd.DataFrame) -> pd.DataFrame:
"""Feature set inspired by price-action + momentum + volatility."""
out = df.copy()
out["returns_1"] = np.log(out["close"] / out["close"].shift(1))
out["returns_3"] = np.log(out["close"] / out["close"].shift(3))
out["returns_6"] = np.log(out["close"] / out["close"].shift(6))
out["sma_10"] = out["close"].rolling(10).mean()
out["sma_20"] = out["close"].rolling(20).mean()
out["sma_50"] = out["close"].rolling(50).mean()
out["ema_12"] = out["close"].ewm(span=12, adjust=False).mean()
out["ema_26"] = out["close"].ewm(span=26, adjust=False).mean()
out["macd"] = out["ema_12"] - out["ema_26"]
out["macd_signal"] = out["macd"].ewm(span=9, adjust=False).mean()
delta = out["close"].diff()
gain = delta.clip(lower=0).rolling(14).mean()
loss = (-delta.clip(upper=0)).rolling(14).mean()
rs = gain / (loss + 1e-9)
out["rsi_14"] = 100.0 - (100.0 / (1.0 + rs))
tr1 = out["high"] - out["low"]
tr2 = (out["high"] - out["close"].shift(1)).abs()
tr3 = (out["low"] - out["close"].shift(1)).abs()
tr = pd.concat([tr1, tr2, tr3], axis=1).max(axis=1)
out["atr_14"] = tr.rolling(14).mean()
out["atr_pct"] = out["atr_14"] / (out["close"] + 1e-9)
out["vol_ratio"] = out["volume"] / (out["volume"].rolling(20).mean() + 1e-9)
out["high_low_range"] = (out["high"] - out["low"]) / (out["close"] + 1e-9)
out["dist_sma20"] = (out["close"] - out["sma_20"]) / (out["close"] + 1e-9)
out.dropna(inplace=True)
return out
def make_target(df: pd.DataFrame, horizon: int = 3) -> pd.DataFrame:
"""Multi-class target:
0 = SELL (return < -threshold)
1 = HOLD (return within threshold)
2 = BUY (return > +threshold)
"""
fwd = np.log(df["close"].shift(-horizon) / df["close"])
thr = fwd.std() * 0.3
target = pd.cut(fwd, bins=[-np.inf, -thr, thr, np.inf], labels=[0, 1, 2])
df["target"] = target
df.dropna(subset=["target"], inplace=True)
df["target"] = df["target"].astype(int)
return df
FEATURES = [
"returns_1", "returns_3", "returns_6",
"sma_10", "sma_20", "sma_50",
"macd", "macd_signal",
"rsi_14", "atr_14", "atr_pct",
"vol_ratio", "high_low_range", "dist_sma20",
]
def build_pipeline():
return Pipeline([
("scaler", StandardScaler()),
("clf", RandomForestClassifier(n_estimators=300, max_depth=12, random_state=42, n_jobs=-1, class_weight="balanced")),
])
def train_models(df: pd.DataFrame):
buy = df[df["target"] == 2].copy()
hold = df[df["target"] == 1].copy()
sell = df[df["target"] == 0].copy()
min_n = min(len(buy), len(hold), len(sell))
if min_n < 200:
raise ValueError(f"Not enough samples per class (min={min_n}). Provide more data or reduce horizon.")
buy = buy.sample(len(buy), random_state=42) if len(buy) > min_n else buy
hold = hold.sample(len(hold), random_state=42) if len(hold) > min_n else hold
sell = sell.sample(len(sell), random_state=42) if len(sell) > min_n else sell
balanced = pd.concat([buy, hold, sell]).sample(frac=1, random_state=42).reset_index(drop=True)
X = balanced[FEATURES].values
y = balanced["target"].values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
pipe = build_pipeline()
pipe.fit(X_train, y_train)
preds = pipe.predict(X_test)
print("Accuracy:", accuracy_score(y_test, preds))
print(classification_report(y_test, preds, target_names=["SELL", "HOLD", "BUY"]))
return pipe, FEATURES
def export_onnx(model: Pipeline, features: list, path: Path):
initial_types = [("float_input", FloatTensorType([None, len(features)]))]
onnx_model = convert_sklearn(model, initial_types=initial_types, target_opset=15)
onnx.save(onnx_model, str(path))
print(f"Saved ONNX model -> {path}")
def save_metadata(meta: dict, path: Path):
with open(path, "w") as f:
json.dump(meta, f, indent=2)
def main():
meta = {
"symbol": "XAUUSD",
"timeframe": "H1",
"features": FEATURES,
"target_horizon": 3,
"built_at": datetime.now(timezone.utc).isoformat(),
"models": {},
}
real_csv = DATA_DIR / "xauusd_h1.csv"
if HAS_SKLEARN:
if real_csv.exists():
print(f"Loading real data from {real_csv}")
df = load_real_data(real_csv)
else:
print("Real data CSV not found, generating synthetic data")
df = generate_synthetic_gold_data(days=1500)
df = engineer_features(df)
df = make_target(df, horizon=3)
buy_pipe, feats = train_models(pd.concat([df[df["target"] == 2], df[df["target"] != 2]]))
export_onnx(buy_pipe, feats, MODELS_DIR / "xauusd_h1_ensemble.onnx")
meta["models"]["ensemble"] = {"features": feats, "path": "models/xauusd_h1_ensemble.onnx"}
else:
print("sklearn/skl2onnx not available (", SKLEARN_IMPORT_ERROR, ")")
print("Falling back to demo ONNX model via build_onnx_raw.py")
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from build_onnx_raw import build_model, save_metadata as save_meta
build_model()
save_meta()
with open(MODELS_DIR / "metadata.json") as f:
meta = json.load(f)
save_metadata(meta, MODELS_DIR / "metadata.json")
print("Done.")
if __name__ == "__main__":
main()
+43
View File
@@ -0,0 +1,43 @@
{
"symbol": "XAUUSD",
"timeframe": "H1",
"features": [
"returns_1",
"returns_3",
"returns_6",
"sma_10",
"sma_20",
"sma_50",
"macd",
"macd_signal",
"rsi_14",
"atr_14",
"atr_pct",
"vol_ratio",
"high_low_range",
"dist_sma20"
],
"target_horizon": 3,
"built_at": "2026-06-13T14:18:27.084238+00:00",
"models": {
"ensemble": {
"features": [
"returns_1",
"returns_3",
"returns_6",
"sma_10",
"sma_20",
"sma_50",
"macd",
"macd_signal",
"rsi_14",
"atr_14",
"atr_pct",
"vol_ratio",
"high_low_range",
"dist_sma20"
],
"path": "models/xauusd_h1_ensemble.onnx"
}
}
}
Binary file not shown.
+11
View File
@@ -0,0 +1,11 @@
fastapi==0.111.0
uvicorn[standard]==0.30.0
onnxruntime>=1.24.0,<1.26.0
scikit-learn>=1.5.0,<1.6.0
pandas>=2.2.0,<2.3.0
numpy>=1.26.0,<2.0.0
joblib>=1.4.0,<1.5.0
skl2onnx>=0.16.0,<0.17.0
protobuf>=3.20,<5.0
python-multipart==0.0.9
pydantic>=2.7.0,<2.8.0
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env python3
"""
Start Mad Turtle inference server.
Run: python3 python/run_server.py
"""
import sys
sys.path.insert(0, str(__file__).rsplit("/", 2)[0])
import uvicorn
from inference_server.server import app
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000, log_level="info")
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env python3
"""
Train XAUUSD H1 ensemble ONNX model (demo with synthetic data if no real feed).
Run: python3 python/ml_pipeline/train_onnx.py
"""
import sys
sys.path.insert(0, str(__file__).rsplit("/", 2)[0])
from ml_pipeline.train_onnx import main
if __name__ == "__main__":
main()
Executable
+25
View File
@@ -0,0 +1,25 @@
#!/bin/bash
# setup.sh — One-command environment setup for Mad Turtle v2.0
set -e
echo "=== Mad Turtle v2.0 Setup ==="
# Python venv
if [ ! -d .venv ]; then
python3 -m venv .venv
echo "Created .venv"
fi
source .venv/bin/activate
pip install --upgrade pip setuptools wheel
pip install numpy onnx onnxruntime fastapi uvicorn pydantic
# Optional: for training real models (requires Python 3.11/3.12 + Xcode CLT)
# pip install scikit-learn skl2onnx pandas
echo "=== Setup complete ==="
echo "Activate venv: source .venv/bin/activate"
echo "Start server: python python/run_server.py"
echo "Then attach MadTurtle.mq5 to MT5."