1.20

README.md

@@ -7,6 +7,7 @@
 * [Installation](#installation)
 * [Documentation](#documentation)
 * [Usage](#usage)
+* [Live data and streaming events](#live-data-and-streaming-events)
 * [License](#license)
 
 ## About the Project
@@ -17,7 +18,6 @@ This project was developed to work as a server for Backtrader Python trading fra
 Backtrader Python client located here: [Python Backtrader - Metaquotes MQL5 ](https://github.com/khramkov/MQL5-Backtrader-API)
 
 In development:
-* Historical data load speed
 * Add error handling to docs
 * Trades info
 * Experation
@@ -46,7 +46,7 @@ The script uses four ZeroMQ sockets:
 3. `Live socket` - automatically pushes last candle when it closes.
 4. `Streaming socket` - automatically pushes last transaction info every time it happens.
 
-The idea is to send requests via `System socket` and recieve results/errors via `Data socket`. Event handlers should be created for `Live socket` and `Streaming socket` because server sends data to theese sockets automatically. See examples in [Usage](#usage) section.
+The idea is to send requests via `System socket` and recieve results/errors via `Data socket`. Event handlers should be created for `Live socket` and `Streaming socket` because server sends data to theese sockets automatically. See examples in [Live data and streaming events](#live-data-and-streaming-events) section.
 
 `System socket` request uses default JSON dictionary:
 
@@ -93,7 +93,7 @@ TRADE      | POSITION_CLOSE_SYMBOL| Positions close by symbol  |
 TRADE      | ORDER_MODIFY         | Order modify               |
 TRADE      | ORDER_CANCEL         | Order cancel               |
 
-Example Python API class:
+Python 3 API class example:
 
 ``` python
 import zmq
@@ -101,10 +101,10 @@ import zmq
 class MTraderAPI:
     def __init__(self, host=None):
         self.HOST = host or 'localhost'
-        self.SYS_PORT = 15555  # REP/REQ port
+        self.SYS_PORT = 15555       # REP/REQ port
-        self.DATA_PORT = 15556  # PUSH/PULL port
+        self.DATA_PORT = 15556      # PUSH/PULL port
-        self.LIVE_PORT = 15557  # PUSH/PULL port
+        self.LIVE_PORT = 15557      # PUSH/PULL port
-        self.EVENTS_PORT = 15558  # PUSH/PULL port
+        self.EVENTS_PORT = 15558    # PUSH/PULL port
 
         # ZeroMQ timeout in seconds
         sys_timeout = 1
@@ -116,17 +116,19 @@ class MTraderAPI:
         # connect to server sockets
         try:
             self.sys_socket = context.socket(zmq.REQ)
+            # set port timeout
             self.sys_socket.RCVTIMEO = sys_timeout * 1000
             self.sys_socket.connect('tcp://{}:{}'.format(self.HOST, self.SYS_PORT))
 
             self.data_socket = context.socket(zmq.PULL)
+            # set port timeout
             self.data_socket.RCVTIMEO = data_timeout * 1000
             self.data_socket.connect('tcp://{}:{}'.format(self.HOST, self.DATA_PORT))
         except zmq.ZMQError:
             raise zmq.ZMQBindError("Binding ports ERROR")
 
     def _send_request(self, data: dict) -> None:
-        """ Send request to server via ZeroMQ System socket """
+        """Send request to server via ZeroMQ System socket"""
         try:
             self.sys_socket.send_json(data)
             msg = self.sys_socket.recv_string()
@@ -138,7 +140,7 @@ class MTraderAPI:
             raise zmq.NotDone("Sending request ERROR")
 
     def _pull_reply(self):
-        """ Get reply from server via Data socket with timeout """
+        """Get reply from server via Data socket with timeout"""
         try:
             msg = self.data_socket.recv_json()
         except zmq.ZMQError:
@@ -146,25 +148,27 @@ class MTraderAPI:
         return msg
 
     def live_socket(self, context=None):
+        """Connect to socket in a ZMQ context"""
         try:
             context = context or zmq.Context.instance()
             socket = context.socket(zmq.PULL)
             socket.connect('tcp://{}:{}'.format(self.HOST, self.LIVE_PORT))
         except zmq.ZMQError:
-            raise zmq.ZMQBindError("Binding ports ERROR")
+            raise zmq.ZMQBindError("Live port connection ERROR")
         return socket
 
     def streaming_socket(self, context=None):
+        """Connect to socket in a ZMQ context"""
         try:
             context = context or zmq.Context.instance()
             socket = context.socket(zmq.PULL)
             socket.connect('tcp://{}:{}'.format(self.HOST, self.EVENTS_PORT))
         except zmq.ZMQError:
-            raise zmq.ZMQBindError("Binding ports ERROR")
+            raise zmq.ZMQBindError("Data port connection ERROR")
         return socket
 
     def construct_and_send(self, **kwargs) -> dict:
-        """ Construct request dictionary from default """
+        """Construct a request dictionary from default and send it to server"""
 
         # default dictionary
         request = {
@@ -219,15 +223,22 @@ rep = api.construct_and_send(action="ACCOUNT")
 print(rep)
 ```
 
-Get historical data. `fromDate` should be in timestamp format. There are some issues:
+Get historical data. `fromDate` should be in timestamp format. If `toDate` is `None`  There are some issues:
 
 - MetaTrader keeps historical data in cache. But when you make a request for the first time, MetaTrader downloads data from a broker. This operation can exceed `Data socket` timeout. It depends on your broker. Second request will be handeled quickly.
-- Historical data processing code is not optimal. It takes too much time to process more than `50000` candles. Under refactoring now.
+- It takes 6-7 seconds to process `50000` M1 candles. It was tested on Windows 10 in Parallels Desktop container with 4 cores and 4GB RAM. So if you need more data there are three ways to handle it. 1) Increase `Data socket` timeout. 2) You can load data partially using `fromDate` and `toDate`. 3) You can use more powerfull hardware.
 
 ``` python
 rep = api.construct_and_send(action="HISTORY", actionType="DATA", symbol="EURUSD", chartTF="M5", fromDate=1555555555)
 print(rep)
 ```
+
+History data reply example:
+
+```
+{'data': [[1560782340, 1.12271, 1.12288, 1.12269, 1.12277, 46.0],[1560782400, 1.12278, 1.12299, 1.12276, 1.12297, 43.0],[1560782460, 1.12296, 1.12302, 1.12293, 1.123, 23.0]]}
+```
+
 Buy market order.
 
 ``` python
@@ -235,7 +246,7 @@ rep = api.construct_and_send(action="TRADE", actionType="ORDER_TYPE_BUY", symbol
 print(rep)
 ```
 
-Sell limit order. Remember to switch SL/TP depending on BUY/SELL, or you will get 'invalid stops' error.
+Sell limit order. Remember to switch SL/TP depending on BUY/SELL, or you will get `invalid stops` error.
 
 - BUY:  	SL < price < TP
 - SELL: 	SL > price > TP
@@ -244,6 +255,7 @@ Sell limit order. Remember to switch SL/TP depending on BUY/SELL, or you will ge
 rep = api.construct_and_send(action="TRADE", actionType="ORDER_TYPE_SELL_LIMIT", symbol="EURUSD", "volume": 0.1, "price": 1.2, "stoploss": 1.3, "takeprofit": 1.1)
 print(rep)
 ```
+## Live data and streaming events
 
 Event handler example for `Live socket` and `Data socket`.
  
@@ -276,11 +288,11 @@ def _t_streaming_events():
         print(reply)
 
 
-for i in range(3):
+for _ in range(3):
     t = threading.Thread(target=_t_livedata, daemon=True)
     t.start()
 
-for i in range(3):
+for _ in range(3):
     t = threading.Thread(target=_t_streaming_events, daemon=True)
     t.start()
 
@@ -288,7 +300,61 @@ while True:
     pass
 ```
 
+
+There are only two variants of `Live socket` data. When everything is ok, the script sends candle data on close:
+
+```
+{"status":"CONNECTED","data":[1560780120,1.12186,1.12194,1.12186,1.12191,15.00000]}
+```
+
+If the terminal has lost connection to market:
+
+```
+{"status":"DISCONNECTED"}
+```
+
+When the terminal reconnects to market, it sends last closed candle again. So you should update historical data. Make the `action="HISTORY"` request with `fromDate` equal to your last candle timestamp.
+
+`OnTradeTransaction` function is called when the trade transaction event occurs. `Streaming socket` sends `TRADE_TRANSACTION_REQUEST` data every time it happens. You can create and modify orders/positions in terminal manually and check expert logging tub for better understanding. Also see [MQL5 docs](https://www.mql5.com/en/docs/event_handlers/ontradetransaction). 
+
+`TRADE_TRANSACTION_REQUEST` request data:
+
+```
+{
+	'action': 'TRADE_ACTION_DEAL', 
+	'order': 501700843, 
+	'symbol': 'EURUSD', 
+	'volume': 0.1, 
+	'price': 1.12181, 
+	'stoplimit': 0.0, 
+	'sl': 1.1, 
+	'tp': 1.13, 
+	'deviation': 10, 
+	'type': 'ORDER_TYPE_BUY', 
+	'type_filling': 'ORDER_FILLING_FOK', 
+	'type_time': 'ORDER_TIME_GTC', 
+	'expiration': 0, 
+	'comment': None, 
+	'position': 0, 
+	'position_by': 0
+}
+```
+`TRADE_TRANSACTION_REQUEST` result data:
+
+```
+{
+	'retcode': 10009, 
+	'result': 'TRADE_RETCODE_DONE', 
+	'deal': 501700843, 
+	'order': 501700843, 
+	'volume': 0.1, 
+	'price': 1.12181, 
+	'comment': None, 
+	'request_id': 8, 
+	'retcode_external': 0
+}
+```
+
 ## License
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
-
 This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See `LICENSE` for more information.