refactor: Improve documentation and code quality

This commit is contained in:
Ash
2025-10-23 23:21:40 +01:00
parent 5600035f3c
commit efb2fb6567
7 changed files with 216 additions and 165 deletions
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Anaswar-ash
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+27 -59
View File
@@ -1,60 +1,28 @@
# MT5-PY-AI-Tbot
A complete framework for creating an Al-enhanced trading bot in MQL5, using Python for model training and the ONNX format for deployment on the MetaTrader 5 platform.
Author: Ash.
MQL5-Python-Trader
This repository contains a complete, working example of an automated trading bot that uses a Python-trained neural network to make trading decisions directly inside MetaTrader 5. The model is optimized using Time Series Cross Validation to find the best possible parameters and ensure its robustness.
System Architecture 🏗️
The project is built on a modern, decoupled architecture that leverages the best tool for each task.
1. The Python "Lab" 🔬
All data analysis, feature engineering, and model training happens here. We use Python for its powerful data science libraries to create a predictive model. The development is done in a standard .py script, perfect for use in Visual Studio Code.
2. The MQL5 "Trader" ⚙️
This is a lightweight, high-performance Expert Advisor (EA) written in MQL5. Its only jobs are to feed the latest market data into the model and execute trades. It does no complex calculations itself.
3. The ONNX "Bridge" 🌉
The ONNX file format is the magic that connects our Python lab to our MQL5 trader. We save our trained model as an .onnx file, which MQL5 can load and use natively, creating a perfect, efficient link between the two environments.
Project Files
* Export_EURUSD_History.mq5: MQL5 script to pull daily EUR/USD price history into a .csv file.
* Model_Development.py: The Python script where all model training, validation, and hyperparameter tuning occurs.
* benchmark_logistic_model.onnx: A simple baseline model used for performance comparison.
* trading_neural_network_model.onnx: The final, optimized neural network model ready for deployment.
* Benchmark_Model_EA.mq5: The MQL5 Expert Advisor for running the simple benchmark model.
* Neural_Network_Trader_EA.mq5: The main Expert Advisor that runs our advanced neural network model.
The Workflow: From Data to Trade
The process is broken down into three logical steps.
Step 1: Data Export
We run the Export_EURUSD_History.mq5 script in MetaTrader 5. This gives us a raw_price_data.csv file to work with in Python.
Step 2: Model Development & Optimization
This is the core of the project, handled within the Model_Development.py script.
1. Feature Engineering:
* We load the data into a raw_price_df DataFrame.
* [cite_start]Our primary feature, feature_price_change, is the difference between one day's close and the previous day's.
* Our target, y_target_direction, is 1 if the next day's price went up and 0 otherwise.
2. Hyperparameter Tuning with Time Series Cross Validation:
[cite_start]To find the best possible version of our model, we use a search process that respects the chronological order of the data.
* Split Strategy: We first define a TimeSeriesSplit object. [cite_start]This object creates 5 sequential "folds" of data and ensures there is a gap between the training and validation sets equal to our forecast horizon.
* [cite_start]Parameter Search Space: We create a dictionary of potential parameters for the neural network. [cite_start]This includes different activation functions, learning rates, solver algorithms, and hidden layer sizes.
* [cite_start]Randomized Search: We use RandomizedSearchCV to conduct the search. [cite_start]This tool systematically tests 50 random combinations of the parameters across our time series splits. [cite_start]It evaluates each combination and identifies the set of parameters that performs the best on unseen data.
3. Final Model Training:
* [cite_start]After the search identifies the optimal parameters (best_params_), we initialize a new neural_network_model with these settings.
* [cite_start]We then train this final, optimized model on the entire training dataset.
* The fully trained model is exported to trading_neural_network_model.onnx.
Step 3: Trade Execution
The Neural_Network_Trader_EA.mq5 takes over in MetaTrader 5:
* On startup, it loads the .onnx file using its onnxModelHandle.
* On each new daily bar, it calculates the latest feature_price_change.
* It feeds this feature into the model and gets back the buyProbability.
* [cite_start]If buyProbability is greater than the current close price, it executes a BUY order. [cite_start]Otherwise, it executes a SELL order. All trades are protected with stopLossPips and takeProfitPips.
How to Run This Project
* Setup Your Environment:
* Install MetaTrader 5 and open a free demo account.
* Install Visual Studio Code and the Python extension.
* Install the necessary Python packages:
pip install pandas numpy tensorflow scikit-learn onnxruntime skl2onnx tf2onnx MetaTrader5
# AI-Enhanced MQL5 Trading Bot
* Execute the Pipeline:
* Get Data: Place Export_EURUSD_History.mq5 in MQL5/Scripts and run it.
* Train Model: Run the Model_Development.py script. It will automatically perform the cross-validation and save the final .onnx file.
* Deploy EA: Copy Neural_Network_Trader_EA.mq5 to MQL5/Experts and the .onnx file to MQL5/Files.
* Test Your Bot:
* Use the MT5 Strategy Tester for historical backtesting.
* Attach the EA to a EURUSD, D1 chart to run it on your demo account.
Disclaimer: This is an educational project. Trading financial markets involves significant risk. Past performance is not indicative of future results. Never deploy an untested automated strategy on a live account with real money.
An end-to-end automated trading bot linking a Python-trained neural network to a high-performance MQL5 (MetaTrader 5) Expert Advisor via the ONNX model format.
## Features
* **Automated Trading:** The bot automatically executes trades based on the predictions of a neural network model.
* **Machine Learning Model:** The trading decisions are powered by a neural network model trained on historical price data.
* **Robust Model Selection:** The model is trained using a robust model selection process with `RandomizedSearchCV` and `TimeSeriesSplit` to optimize hyperparameters and ensure model validity on chronological financial data.
* **ONNX Integration:** The trained model is exported to the ONNX format, allowing it to be used by the MQL5 Expert Advisor.
* **Benchmark Model:** The project includes a benchmark logistic regression model for comparison.
## Technology Stack
* **Python**: For data analysis, model development, and training.
* **MQL5**: For creating the Expert Advisors (EAs) that run in the MetaTrader 5 terminal.
* **ONNX**: For exporting the trained models to a format that can be used by the MQL5 EAs.
* **Scikit-learn**: For creating and training the machine learning models.
* **TensorFlow**: Used as a backend for the neural network model.
* **Pandas**: For data manipulation and analysis.
## How to Use
Please refer to the [HOWTORUN.md](howtorun.md) file for detailed instructions on how to run the project.
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+23 -8
View File
@@ -7,18 +7,19 @@
#property link "https://github.com/Anaswar-ash"
#property version "1.00"
#include <Trade/Trade.mqh>
#include <Trade/Trade.mqh> // Include the trade library
//--- ONNX model parameters
int onnx_handle;
long onnx_input_shape[] = {1, 1};
long onnx_output_shape[] = {1, 1};
int onnx_handle; // Handle for the ONNX model
long onnx_input_shape[] = {1, 1}; // Shape of the input tensor
long onnx_output_shape[] = {1, 1}; // Shape of the output tensor
//--- Expert Advisor parameters
input double InpLots = 0.1;
input int InpStopLoss = 50;
input int InpTakeProfit = 100;
input double InpLots = 0.1; // Lot size for trades
input int InpStopLoss = 50; // Stop loss in points
input int InpTakeProfit = 100; // Take profit in points
//--- Create a CTrade object for executing trades
CTrade trade;
//+------------------------------------------------------------------+
@@ -26,25 +27,31 @@ CTrade trade;
//+------------------------------------------------------------------+
int OnInit()
{
//--- Create the ONNX model from the file
onnx_handle = OnnxCreateFromFile("benchmark_logistic_model.onnx");
//--- Check if the model was created successfully
if(onnx_handle == INVALID_HANDLE)
{
Print("Failed to create ONNX model from file: ", GetLastError());
return(INIT_FAILED);
}
//--- Set the input shape of the model
if(!OnnxSetInputShape(onnx_handle, 0, onnx_input_shape))
{
Print("OnnxSetInputShape failed with error: ", GetLastError());
return(INIT_FAILED);
}
//--- Set the output shape of the model
if(!OnnxSetOutputShape(onnx_handle, 0, onnx_output_shape))
{
Print("OnnxSetOutputShape failed with error: ", GetLastError());
return(INIT_FAILED);
}
//--- Initialization successful
return(INIT_SUCCEEDED);
}
@@ -53,6 +60,7 @@ int OnInit()
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
//--- Release the ONNX model
if(onnx_handle != INVALID_HANDLE)
OnnxRelease(onnx_handle);
}
@@ -62,24 +70,31 @@ void OnDeinit(const int reason)
//+------------------------------------------------------------------+
void OnTick()
{
//--- Array to store the price data
MqlRates rates[];
//--- Get the last 2 bars
if(CopyRates(_Symbol, PERIOD_D1, 0, 2, rates) < 2)
return;
//--- Create the input data for the model (price change)
float input_data[1];
input_data[0] = (float)(rates[1].close - rates[0].close);
//--- Create the output data array
float output_data[1];
//--- Run the ONNX model
if(!OnnxRun(onnx_handle, input_data, output_data))
{
Print("OnnxRun failed with error: ", GetLastError());
return;
}
//--- Execute a trade based on the model's prediction
if(output_data[0] == 1)
trade.Buy(InpLots, _Symbol, 0, 0, 0, "Buy order");
else
trade.Sell(InpLots, _Symbol, 0, 0, 0, "Sell order");
}
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
+16 -6
View File
@@ -9,28 +9,35 @@
#property script_show_inputs
//--- input parameters
input string InpFileName = "raw_price_data.csv"; // File name
input int InpDays = 1000; // Number of days
input string InpFileName = "raw_price_data.csv"; // File name to save the data to
input int InpDays = 1000; // Number of days of historical data to export
//+------------------------------------------------------------------+
//| Script program start function |
//+------------------------------------------------------------------+
void OnStart()
{
//--- Array to store the price data
MqlRates rates[];
//--- Variable to store the number of bars copied
int copied;
//--- get daily price data
//--- Get daily price data for EURUSD for the specified number of days
copied = CopyRates("EURUSD", PERIOD_D1, 0, InpDays, rates);
//--- Check if the data was copied successfully
if(copied > 0)
{
//--- Open the file to write the data to
int file_handle = FileOpen(InpFileName, FILE_WRITE | FILE_CSV | FILE_ANSI, ',');
//--- Check if the file was opened successfully
if(file_handle != INVALID_HANDLE)
{
//--- write header
//--- Write the header row to the CSV file
FileWrite(file_handle, "Time", "Open", "High", "Low", "Close", "Volume");
//--- write data
//--- Loop through the copied data and write each row to the CSV file
for(int i = 0; i < copied; i++)
{
FileWrite(file_handle,
@@ -42,17 +49,20 @@ void OnStart()
rates[i].tick_volume);
}
//--- Close the file
FileClose(file_handle);
Print("Data successfully exported to ", InpFileName);
}
else
{
//--- Print an error message if the file could not be opened
Print("Error opening file: ", GetLastError());
}
}
else
{
//--- Print an error message if the data could not be copied
Print("Error copying rates: ", GetLastError());
}
}
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
+72 -63
View File
@@ -1,76 +1,85 @@
import pandas as pd
# Load the data
raw_price_df = pd.read_csv("raw_price_data.csv")
# Feature Engineering
raw_price_df["feature_price_change"] = raw_price_df["Close"].diff()
# Target Engineering
raw_price_df["y_target_direction"] = (raw_price_df["Close"].shift(-1) > raw_price_df["Close"]).astype(int)
# Drop rows with NaN values
raw_price_df.dropna(inplace=True)
from sklearn.model_selection import TimeSeriesSplit, RandomizedSearchCV
from sklearn.neural_network import MLPClassifier
# --- Hyperparameter Tuning ---
# Define the parameter search space
param_distributions = {
'hidden_layer_sizes': [(50,), (100,), (50, 50)],
'activation': ['tanh', 'relu'],
'solver': ['adam', 'sgd'],
'learning_rate': ['constant', 'adaptive'],
}
# Create the neural network model
neural_network_model = MLPClassifier(max_iter=1000)
# Create the time series split object
ts_split = TimeSeriesSplit(n_splits=2, gap=1)
# Create the randomized search object
random_search = RandomizedSearchCV(
estimator=neural_network_model,
param_distributions=param_distributions,
n_iter=10, # Reduced for faster execution
cv=ts_split,
scoring='accuracy',
random_state=42,
n_jobs=-1
)
# Separate features and target
X = raw_price_df[['feature_price_change']]
y = raw_price_df['y_target_direction']
# Fit the randomized search to the data
random_search.fit(X, y)
# Print the best parameters
import skl2onnx
import onnxruntime as rt
from skl2onnx.common.data_types import FloatTensorType
# --- Final Model Training ---
# Initialize a new neural network model with the best parameters
final_model = MLPClassifier(**random_search.best_params_, max_iter=1000)
def main():
"""
This script performs the following steps:
1. Loads the historical price data from a CSV file.
2. Performs feature engineering to create a 'feature_price_change' feature.
3. Performs target engineering to create a 'y_target_direction' target variable.
4. Performs hyperparameter tuning using RandomizedSearchCV and TimeSeriesSplit to find the best parameters for an MLPClassifier.
5. Trains a final MLPClassifier model with the best parameters.
6. Exports the trained model to an ONNX file named 'trading_neural_network_model.onnx'.
"""
# Load the data
raw_price_df = pd.read_csv("raw_price_data.csv")
# Train the model on the entire dataset
final_model.fit(X, y)
# Feature Engineering
raw_price_df["feature_price_change"] = raw_price_df["Close"].diff()
# --- Export to ONNX ---
# Define the initial types for the ONNX conversion
initial_type = [('float_input', FloatTensorType([None, 1]))]
# Target Engineering
raw_price_df["y_target_direction"] = (raw_price_df["Close"].shift(-1) > raw_price_df["Close"]).astype(int)
# Convert the model to ONNX format
onnx_model = skl2onnx.convert_sklearn(final_model, initial_types=initial_type)
# Drop rows with NaN values
raw_price_df.dropna(inplace=True)
# Save the ONNX model
with open("trading_neural_network_model.onnx", "wb") as f:
f.write(onnx_model.SerializeToString())
# --- Hyperparameter Tuning ---
# Define the parameter search space
param_distributions = {
'hidden_layer_sizes': [(50,), (100,), (50, 50)],
'activation': ['tanh', 'relu'],
'solver': ['adam', 'sgd'],
'learning_rate': ['constant', 'adaptive'],
}
print("Model successfully exported to trading_neural_network_model.onnx")
# Create the neural network model
neural_network_model = MLPClassifier(max_iter=1000)
# Create the time series split object
ts_split = TimeSeriesSplit(n_splits=2, gap=1)
# Create the randomized search object
random_search = RandomizedSearchCV(
estimator=neural_network_model,
param_distributions=param_distributions,
n_iter=10, # Reduced for faster execution
cv=ts_split,
scoring='accuracy',
random_state=42,
n_jobs=-1
)
# Separate features and target
X = raw_price_df[['feature_price_change']]
y = raw_price_df['y_target_direction']
# Fit the randomized search to the data
random_search.fit(X, y)
print(f"Best parameters found: {random_search.best_params_}")
# --- Final Model Training ---
# Initialize a new neural network model with the best parameters
final_model = MLPClassifier(**random_search.best_params_, max_iter=1000)
# Train the model on the entire dataset
final_model.fit(X, y)
# --- Export to ONNX ---
# Define the initial types for the ONNX conversion
initial_type = [('float_input', FloatTensorType([None, 1]))]
# Convert the model to ONNX format
onnx_model = skl2onnx.convert_sklearn(final_model, initial_types=initial_type)
# Save the ONNX model
with open("trading_neural_network_model.onnx", "wb") as f:
f.write(onnx_model.SerializeToString())
print("Model successfully exported to trading_neural_network_model.onnx")
if __name__ == "__main__":
main()
+23 -8
View File
@@ -7,18 +7,19 @@
#property link "https://github.com/Anaswar-ash"
#property version "1.00"
#include <Trade/Trade.mqh>
#include <Trade/Trade.mqh> // Include the trade library
//--- ONNX model parameters
int onnx_handle;
long onnx_input_shape[] = {1, 1};
long onnx_output_shape[] = {1, 2};
int onnx_handle; // Handle for the ONNX model
long onnx_input_shape[] = {1, 1}; // Shape of the input tensor
long onnx_output_shape[] = {1, 2}; // Shape of the output tensor
//--- Expert Advisor parameters
input double InpLots = 0.1;
input int InpStopLoss = 50;
input int InpTakeProfit = 100;
input double InpLots = 0.1; // Lot size for trades
input int InpStopLoss = 50; // Stop loss in points
input int InpTakeProfit = 100; // Take profit in points
//--- Create a CTrade object for executing trades
CTrade trade;
//+------------------------------------------------------------------+
@@ -26,25 +27,31 @@ CTrade trade;
//+------------------------------------------------------------------+
int OnInit()
{
//--- Create the ONNX model from the file
onnx_handle = OnnxCreateFromFile("trading_neural_network_model.onnx");
//--- Check if the model was created successfully
if(onnx_handle == INVALID_HANDLE)
{
Print("Failed to create ONNX model from file: ", GetLastError());
return(INIT_FAILED);
}
//--- Set the input shape of the model
if(!OnnxSetInputShape(onnx_handle, 0, onnx_input_shape))
{
Print("OnnxSetInputShape failed with error: ", GetLastError());
return(INIT_FAILED);
}
//--- Set the output shape of the model
if(!OnnxSetOutputShape(onnx_handle, 0, onnx_output_shape))
{
Print("OnnxSetOutputShape failed with error: ", GetLastError());
return(INIT_FAILED);
}
//--- Initialization successful
return(INIT_SUCCEEDED);
}
@@ -53,6 +60,7 @@ int OnInit()
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
//--- Release the ONNX model
if(onnx_handle != INVALID_HANDLE)
OnnxRelease(onnx_handle);
}
@@ -62,24 +70,31 @@ void OnDeinit(const int reason)
//+------------------------------------------------------------------+
void OnTick()
{
//--- Array to store the price data
MqlRates rates[];
//--- Get the last 2 bars
if(CopyRates(_Symbol, PERIOD_D1, 0, 2, rates) < 2)
return;
//--- Create the input data for the model (price change)
float input_data[1];
input_data[0] = (float)(rates[1].close - rates[0].close);
//--- Create the output data array
float output_data[2];
//--- Run the ONNX model
if(!OnnxRun(onnx_handle, input_data, output_data))
{
Print("OnnxRun failed with error: ", GetLastError());
return;
}
//--- Execute a trade based on the model's prediction
if(output_data[0] > output_data[1])
trade.Sell(InpLots, _Symbol, 0, 0, 0, "Sell order");
else
trade.Buy(InpLots, _Symbol, 0, 0, 0, "Buy order");
}
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
+34 -21
View File
@@ -1,34 +1,47 @@
import pandas as pd
from sklearn.linear_model import LogisticRegression
import skl2onnx
from skl2onnx.common.data_types import FloatTensorType
# Load the data
raw_price_df = pd.read_csv("raw_price_data.csv")
def main():
"""
This script performs the following steps:
1. Loads the historical price data from a CSV file.
2. Performs feature engineering to create a 'feature_price_change' feature.
3. Performs target engineering to create a 'y_target_direction' target variable.
4. Creates and trains a logistic regression model.
5. Exports the trained model to an ONNX file named 'benchmark_logistic_model.onnx'.
"""
# Load the data
raw_price_df = pd.read_csv("raw_price_data.csv")
# Feature Engineering
raw_price_df["feature_price_change"] = raw_price_df["Close"].diff()
# Feature Engineering
raw_price_df["feature_price_change"] = raw_price_df["Close"].diff()
# Target Engineering
raw_price_df["y_target_direction"] = (raw_price_df["Close"].shift(-1) > raw_price_df["Close"]).astype(int)
# Target Engineering
raw_price_df["y_target_direction"] = (raw_price_df["Close"].shift(-1) > raw_price_df["Close"]).astype(int)
# Drop rows with NaN values
raw_price_df.dropna(inplace=True)
# Drop rows with NaN values
raw_price_df.dropna(inplace=True)
# Separate features and target
X = raw_price_df[['feature_price_change']]
y = raw_price_df['y_target_direction']
# Separate features and target
X = raw_price_df[['feature_price_change']]
y = raw_price_df['y_target_direction']
# Create and train the logistic regression model
log_reg_model = LogisticRegression()
log_reg_model.fit(X, y)
# Create and train the logistic regression model
log_reg_model = LogisticRegression()
log_reg_model.fit(X, y)
# Convert the model to ONNX format
initial_type = [('float_input', FloatTensorType([None, 1]))]
onnx_model = skl2onnx.convert_sklearn(log_reg_model, initial_types=initial_type)
# Convert the model to ONNX format
initial_type = [('float_input', FloatTensorType([None, 1]))]
onnx_model = skl2onnx.convert_sklearn(log_reg_model, initial_types=initial_type)
# Save the ONNX model
with open("benchmark_logistic_model.onnx", "wb") as f:
f.write(onnx_model.SerializeToString())
# Save the ONNX model
with open("benchmark_logistic_model.onnx", "wb") as f:
f.write(onnx_model.SerializeToString())
print("Model successfully exported to benchmark_logistic_model.onnx")
print("Model successfully exported to benchmark_logistic_model.onnx")
if __name__ == "__main__":
main()