Compare commits
31 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Jamie Cash
|
7d97b78f8c | ||
|
Jamie Cash
|
06c980a45a | ||
|
Jamie Cash
|
1656c6a4d2 | ||
|
Jamie Cash
|
66cdf47406 | ||
|
Jamie Cash
|
b947b928e7 | ||
|
Jamie Cash
|
f6d05873d2 | ||
|
Jamie Cash
|
029e86d358 | ||
|
Jamie Cash
|
c977e3ac1c | ||
|
Jamie Cash
|
225f869276 | ||
|
Jamie Cash
|
1c59945745 | ||
|
Jamie Cash
|
443ad822ad | ||
|
Jamie Cash
|
d0049546fe | ||
|
Jamie Cash
|
4fdf4bedc7 | ||
|
Jamie Cash
|
21ea7eb809 | ||
|
Jamie Cash
|
ab6c2ae338 | ||
|
Jamie Cash
|
8c2cbf5df8 | ||
|
Jamie Cash
|
0c17c7e5a9 | ||
|
Jamie Cash
|
d08fef0cbb | ||
|
Jamie Cash
|
15f1ca2288 | ||
|
Jamie Cash
|
a77559cc4e | ||
|
Jamie Cash
|
3f4f887894 | ||
|
Jamie Cash
|
19d77470ce | ||
|
Jamie Cash
|
81f5fa6c07 | ||
|
Jamie Cash
|
c85ff8f79e | ||
|
Jamie Cash
|
7ca46b1be2 | ||
|
Jamie Cash
|
e44f9451d0 | ||
|
Jamie Cash
|
6ba28f5a30 | ||
|
Jamie Cash
|
862a55d649 | ||
|
Jamie Cash
|
2349a362e8 | ||
|
Jamie Cash
|
15eec4d08b | ||
|
Jamie Cash
|
ef28f80a85 |
+2
-2
@@ -1,4 +1,4 @@
|
||||
/venv/
|
||||
/log/
|
||||
/out/
|
||||
/.idea/workspace.xml
|
||||
/*.log
|
||||
/*.log.?
|
||||
|
||||
Generated
+3
@@ -10,4 +10,7 @@
|
||||
<orderEntry type="inheritedJdk" />
|
||||
<orderEntry type="sourceFolder" forTests="false" />
|
||||
</component>
|
||||
<component name="PyDocumentationSettings">
|
||||
<option name="renderExternalDocumentation" value="true" />
|
||||
</component>
|
||||
</module>
|
||||
Generated
+7
@@ -0,0 +1,7 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="PySciProjectComponent">
|
||||
<option name="PY_SCI_VIEW" value="true" />
|
||||
<option name="PY_SCI_VIEW_SUGGESTED" value="true" />
|
||||
</component>
|
||||
</project>
|
||||
@@ -1,77 +1,35 @@
|
||||
# mt5-correlation
|
||||
Calculates correlation coefficient between all symbols in MetaTrader5 Market Watch.
|
||||
Calculates correlation coefficient between all symbols in MetaTrader5 Market Watch and monitors previously correlated symbol pairs for divergence
|
||||
|
||||
# Setup
|
||||
1) Set up your MetaTrader 5 environment ensuring that all symbols that you would like to assess for correlation are shown in your Market Watch window;
|
||||
2) Set up your python environment; and
|
||||
3) Install the required libraries.
|
||||
```
|
||||
pip install -r mt5-correlation\requirements.txt
|
||||
|
||||
```shell
|
||||
pip install -r mt5-correlation/requirements.txt
|
||||
```
|
||||
|
||||
# Usage
|
||||
If you set up a virtual environment in the Setup step, ensure this is activated. Then run the script.
|
||||
```
|
||||
python -m mt5_correlations\mt5_correlations.py
|
||||
|
||||
```shell
|
||||
python -m mt5_correlations/mt5_correlations.py
|
||||
```
|
||||
|
||||
A .csv file containing the correlation coefficient for all combinations of sybmols from the MetaTrader market watch will be produced in the current directory.
|
||||
This will open a GUI.
|
||||
|
||||
|Symbol 1 |Symbol 2 |Coefficient|UTC Date From |UTC Date To |Timeframe|
|
||||
|------------|------------|-----------|-------------------|-------------------|---------|
|
||||
|OIL-MAR21 |OILMn-MAR21 |1.0 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|EURUSD |EURHKD |0.99980 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|OILMn-MAR21 |BRENT-APR21 |0.99894 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|OIL-MAR21 |BRENT-APR21 |0.99894 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|GSOIL-FEB21 |BRENT-APR21 |0.99605 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|GSOIL-FEB21 |OILMn-MAR21 |0.99543 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|GSOIL-FEB21 |OIL-MAR21 |0.99543 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
|EU50Cash |FRA40Cash |0.99072 |2021-01-29 11:54:29|2021-02-05 11:54:29|15 |
|
||||
## Calculating Baseline Coefficients
|
||||
On the first time that you run, you will want to calculate the initial set of correlations.
|
||||
1) Open settings and review the settings under the 'Calculate' tab. Hover over the individual settings for help.
|
||||
2) Set the threshold for the correlations to monitor. This can be set under the Settings 'Monitoring' tab and is named 'Monitoring Threshold'. Only settings with a coefficient over this threshold will be displayed and monitored.
|
||||
3) Calculate the coefficients by selecting File/Calculate. All symbol pairs that have a correlation coefficient greater than the monitoring threshold will be displayed. A graph showing the price candle data used to calculate the coefficient for the pair will be displayed when you select the row.
|
||||
4) You may want to save. Select File/Save and choose a file name. This file will contain all the calculated coefficients, and the price data used to calculate them. This file can be loaded to avoid having to recalculate the baseline coefficients every time you use the application.
|
||||
|
||||
# Customising
|
||||
Edit mt5_correlations.py to customise.
|
||||
|
||||
The coefficients are calculated only if:
|
||||
* The smallest set of price data is no less than 90% of the size of the largest set;
|
||||
* The overlapping prices between both sets of price data contains no less than 90% of the prices in the smallest set;
|
||||
* The pearsonr p-value for the calculated coefficient is less than 0.05.
|
||||
|
||||
These settings can all be changed in the call to Correlation.calculate_coefficient by passing values for max_set_size_diff_pct; overlap_pct; or max_p_value.
|
||||
```
|
||||
coefficient = Correlation.calculate_coefficient(symbol1_prices=symbol1_price_data,
|
||||
symbol2_prices=symbol2_price_data, max_set_size_diff_pct=90,
|
||||
overlap_pct=90, max_p_value=0.05)
|
||||
```
|
||||
|
||||
The price data compared is 15 minute price data for the last 7 days. This can be changed by changing the values for the following variables:
|
||||
```
|
||||
utc_to = datetime.now(tz=timezone)
|
||||
utc_from = utc_to - timedelta(days=7)
|
||||
timeframe = mt5.TIMEFRAME_M15
|
||||
```
|
||||
|
||||
The possible values for timeframe are:
|
||||
|
||||
|Timeframe|Description|
|
||||
|--------------|-----------|
|
||||
|TIMEFRAME_M1 |1 minute |
|
||||
|TIMEFRAME_M2 |2 minutes |
|
||||
|TIMEFRAME_M3 |3 minutes |
|
||||
|TIMEFRAME_M4 |4 minutes |
|
||||
|TIMEFRAME_M5 |5 minutes |
|
||||
|TIMEFRAME_M6 |6 minutes |
|
||||
|TIMEFRAME_M10 |10 minutes |
|
||||
|TIMEFRAME_M12 |12 minutes |
|
||||
|TIMEFRAME_M12 |15 minutes |
|
||||
|TIMEFRAME_M20 |20 minutes |
|
||||
|TIMEFRAME_M30 |30 minutes |
|
||||
|TIMEFRAME_H1 |1 hour |
|
||||
|TIMEFRAME_H2 |2 hours |
|
||||
|TIMEFRAME_H3 |3 hours |
|
||||
|TIMEFRAME_H4 |4 hours |
|
||||
|TIMEFRAME_H6 |6 hours |
|
||||
|TIMEFRAME_H8 |8 hours |
|
||||
|TIMEFRAME_H12 |12 hours |
|
||||
|TIMEFRAME_D1 |1 day |
|
||||
|TIMEFRAME_W1 |1 week |
|
||||
|TIMEFRAME_MN1 |1 month |
|
||||
## Monitoring for Divergence
|
||||
Once you have calculated or loaded the baseline coefficients, they can be monitored for divergence.
|
||||
1) Open settings and review the settings under the 'Monitor' tab. Hover over the individual settings for help.
|
||||
2) Switch the monitoring toggle to on. The application will continuously monitor for divergence. The data frame will be updated with the last time that the correlation was checked, and the last coefficient. The chart frame will contain 3 charts which will be updated after every monitoring event:
|
||||
- One showing the price history data used to calculate the baseline coefficient for both symbols in the correlated pair;
|
||||
- One showing the tick data used to calculate the latest coefficient for both symbols in the correlated pair.
|
||||
- One showing every correlation coefficient calculated for the symbol pair.
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
---
|
||||
calculate:
|
||||
from:
|
||||
days: 10
|
||||
timeframe: 15
|
||||
min_prices: 400
|
||||
max_set_size_diff_pct: 90
|
||||
overlap_pct: 90
|
||||
max_p_value: 0.05
|
||||
monitor:
|
||||
interval: 10
|
||||
calculations:
|
||||
long:
|
||||
from: 30
|
||||
min_prices: 300
|
||||
max_set_size_diff_pct: 50
|
||||
overlap_pct: 50
|
||||
max_p_value: 0.05
|
||||
medium:
|
||||
from: 10
|
||||
min_prices: 100
|
||||
max_set_size_diff_pct: 50
|
||||
overlap_pct: 50
|
||||
max_p_value: 0.05
|
||||
short:
|
||||
from: 2
|
||||
min_prices: 30
|
||||
max_set_size_diff_pct: 50
|
||||
overlap_pct: 50
|
||||
max_p_value: 0.05
|
||||
monitoring_threshold: 0.9
|
||||
divergence_threshold: 0.8
|
||||
monitor_inverse: true
|
||||
tick_cache_time: 10
|
||||
autosave: true
|
||||
logging:
|
||||
version: 1
|
||||
disable_existing_loggers: false
|
||||
formatters:
|
||||
brief:
|
||||
format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
datefmt: '%H:%M:%S'
|
||||
precice:
|
||||
format: '%(asctime)s - %(threadName)s - %(name)s - %(levelname)s - %(message)s'
|
||||
datefmt: '%Y-%m-%d %H:%M:%S'
|
||||
handlers:
|
||||
console:
|
||||
level: INFO
|
||||
class: logging.StreamHandler
|
||||
formatter: brief
|
||||
stream: ext://sys.stdout
|
||||
file:
|
||||
level: DEBUG
|
||||
class: logging.handlers.RotatingFileHandler
|
||||
formatter: precice
|
||||
filename: debug.log
|
||||
mode: a
|
||||
maxBytes: 2560000
|
||||
backupCount: 1
|
||||
root:
|
||||
level: DEBUG
|
||||
handlers:
|
||||
- console
|
||||
- file
|
||||
loggers:
|
||||
mt5-correlation:
|
||||
level: DEBUG
|
||||
handlers:
|
||||
- console
|
||||
- file
|
||||
propagate: 0
|
||||
charts:
|
||||
colormap: Dark2
|
||||
developer:
|
||||
inspection: true
|
||||
window:
|
||||
x: -4
|
||||
y: 1
|
||||
width: 1626
|
||||
height: 1043
|
||||
style: 541072960
|
||||
settings_window:
|
||||
x: 354
|
||||
y: 299
|
||||
width: 624
|
||||
height: 328
|
||||
style: 524352
|
||||
...
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
---
|
||||
calculate:
|
||||
from:
|
||||
days:
|
||||
__label: Calculate from (days)
|
||||
__helptext: The number of days of data to be used to calculate the coefficient.
|
||||
timeframe:
|
||||
__label: Timeframe
|
||||
__helptext: The timeframe for price candles to use for the calculation. Possible values are 1=1 Minute candles; 2=2 Minute candles; 3=3 Minute candles; 4=4 Minute candles; 5=5 Minute candles; 6=6 Minute candles; 10=10 Minute candles; 15=15 Minute candles; 20=20 Minute candles; 30=30 Minute candles; 16385=1 Hour candles; 16386=2 Hour candles; 16387=3 Hour candles; 16388=4 Hour candles; 16390=6 Hour candles; 16392=8 Hour candles; 16396=12 Hour candles; 16408=1 Day candles; 32769=1 Week candles; or 49153=1 Month candles.
|
||||
min_prices:
|
||||
__label: Min Prices
|
||||
__helptext: The minimum number of candles required to calculate a coefficient from. If any of the symbols do not have at least this number of candles then the coefficient won't be calculated.
|
||||
max_set_size_diff_pct:
|
||||
__label: Min Set Size Difference %
|
||||
__helptext: For a meaningful coefficient calculation, the two sets of data should be of a similar size. This setting specifies the % difference allowed for a correlation to be calculated. The smallest set of candle data must be at least this values % of the largest set.
|
||||
overlap_pct:
|
||||
__label: Overlap %
|
||||
__helptext: The dates and times in the two sets of data must match. The coefficient will only be calculated against the dates that overlap. Any non overlapping dates will be discarded. This setting specifies the minimum size of the overlapping data when compared to the smallest set as a %. A coefficient will not be calculated if this threshold is not met.
|
||||
max_p_value:
|
||||
__label: Max Pearsonr P Value
|
||||
__helptext: The maximum P value for the coefficient to be considered valid. A full explanation on the correlation coefficient P value is available in the scipy pearsonr documentation.
|
||||
monitor:
|
||||
interval:
|
||||
__label: Monitoring Interval
|
||||
__helptext: The number of seconds between monitoring events.
|
||||
calculations:
|
||||
long:
|
||||
__label: Long
|
||||
__helptext: The settings for the correlation calculation using the longest timeframe.
|
||||
from:
|
||||
__label: From (Minutes)
|
||||
__helptext: The number of minutes of data to be used to calculate the long coefficient.
|
||||
min_prices:
|
||||
__label: Min Prices
|
||||
__helptext: Tick data will be converted to 1 second candles prior to calculation. This will enable data to be matched between symbols. This setting specifies the minimum number of price candles required to calculate a coefficient from. If any of the symbols do not have at least number of candles then the coefficient won't be calculated.
|
||||
max_set_size_diff_pct:
|
||||
__label: Max Set Size Difference %
|
||||
__helptext: For a meaningful coefficient calculation, the two sets of data should be of a similar size. This setting specifies the % difference allowed for a correlation to be calculated. The smallest set of price candles must be at least this values % of the largest set.
|
||||
overlap_pct:
|
||||
__label: Overlap %
|
||||
__helptext: The dates and times in the two sets of data must match. The ticks will be converted to 1 second price candles before calculation. The coefficient will only be calculated against the times from the candles that overlap. Any non overlapping times will be discarded. This setting specifies the minimum size of the overlapping data when compared to the smallest set as a %. A coefficient will not be calculated if this threshold is not met.
|
||||
max_p_value:
|
||||
__label: Max Pearsonr P Value
|
||||
__helptext: The maximum P value for the coefficient to be considered valid. A full explanation on the correlation coefficient P value is available in the scipy pearsonr documentation.
|
||||
medium:
|
||||
__label: Medium
|
||||
__helptext: The settings for the correlation calculation using a medium timeframe.
|
||||
from:
|
||||
__label: From (Minutes)
|
||||
__helptext: The number of minutes of data to be used to calculate the medium coefficient.
|
||||
min_prices:
|
||||
__label: Min Prices
|
||||
__helptext: Tick data will be converted to 1 second candles prior to calculation. This will enable data to be matched between symbols. This setting specifies the minimum number of price candles required to calculate a coefficient from. If any of the symbols do not have at least number of candles then the coefficient won't be calculated.
|
||||
max_set_size_diff_pct:
|
||||
__label: Max Set Size Difference %
|
||||
__helptext: For a meaningful coefficient calculation, the two sets of data should be of a similar size. This setting specifies the % difference allowed for a correlation to be calculated. The smallest set of price candles must be at least this values % of the largest set.
|
||||
overlap_pct:
|
||||
__label: Overlap %
|
||||
__helptext: The dates and times in the two sets of data must match. The ticks will be converted to 1 second price candles before calculation. The coefficient will only be calculated against the times from the candles that overlap. Any non overlapping times will be discarded. This setting specifies the minimum size of the overlapping data when compared to the smallest set as a %. A coefficient will not be calculated if this threshold is not met.
|
||||
max_p_value:
|
||||
__label: Max Pearsonr P Value
|
||||
__helptext: The maximum P value for the coefficient to be considered valid. A full explanation on the correlation coefficient P value is available in the scipy pearsonr documentation.
|
||||
short:
|
||||
__label: Short
|
||||
__helptext: The settings for the correlation calculation using the shortest timeframe.
|
||||
from:
|
||||
__label: From (Minutes)
|
||||
__helptext: The number of minutes of data to be used to calculate the short coefficient.
|
||||
min_prices:
|
||||
__label: Min Prices
|
||||
__helptext: Tick data will be converted to 1 second candles prior to calculation. This will enable data to be matched between symbols. This setting specifies the minimum number of price candles required to calculate a coefficient from. If any of the symbols do not have at least number of candles then the coefficient won't be calculated.
|
||||
max_set_size_diff_pct:
|
||||
__label: Max Set Size Difference %
|
||||
__helptext: For a meaningful coefficient calculation, the two sets of data should be of a similar size. This setting specifies the % difference allowed for a correlation to be calculated. The smallest set of price candles must be at least this values % of the largest set.
|
||||
overlap_pct:
|
||||
__label: Overlap %
|
||||
__helptext: The dates and times in the two sets of data must match. The ticks will be converted to 1 second price candles before calculation. The coefficient will only be calculated against the times from the candles that overlap. Any non overlapping times will be discarded. This setting specifies the minimum size of the overlapping data when compared to the smallest set as a %. A coefficient will not be calculated if this threshold is not met.
|
||||
max_p_value:
|
||||
__label: Max Pearsonr P Value
|
||||
__helptext: The maximum P value for the coefficient to be considered valid. A full explanation on the correlation coefficient P value is available in the scipy pearsonr documentation.
|
||||
monitoring_threshold:
|
||||
__label: Monitoring Threshold
|
||||
__helptext: Only pairs with a coefficient over this threshold will be displayed and monitored.
|
||||
divergence_threshold:
|
||||
__label: Divergence Threshold
|
||||
__helptext: The application will consider a pair to have diverged if the correlation coefficient for all timeframes (long, medium and short) falls below this threshold.
|
||||
monitor_inverse:
|
||||
__label: Monitor Inverse
|
||||
__helptext: Monitor Inverse Correlations (uses negative scale with -1 being fully inversly correlated)
|
||||
tick_cache_time:
|
||||
__label: Tick Cache Time
|
||||
__helptext: Every calculation requires tick data for both symbols. Tick data will be cached for this number of seconds before being retrieved from MetaTrader. Some caching is recommended as a single monitoring run will request the same data for symbols that form multiple correlated pairs.
|
||||
autosave:
|
||||
__label: Auto Save
|
||||
__helptext: Whether to auto save after every monitoring event. If a file was opened or has been saved, then the data will be saved to this file, otherwise the data will be saved to a file named autosave.cpd.
|
||||
charts:
|
||||
colormap:
|
||||
__label: Color Map
|
||||
__helptext: The matplotlib color pallet to use for plotting graphs. A list of pallets is available at https://matplotlib.org/stable/tutorials/colors/colormaps.html
|
||||
developer:
|
||||
inspection:
|
||||
__label: Inspection
|
||||
__helptext: Provide GUI Inspection guidelines for developers modifying the GUI.
|
||||
...
|
||||
+3
-1
@@ -1,3 +1,5 @@
|
||||
import os
|
||||
|
||||
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
HELP_FILE = fr"{ROOT_DIR}\README.md"
|
||||
LOG_FILE = fr"{ROOT_DIR}\debug.log"
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
---
|
||||
version: 1
|
||||
disable_existing_loggers: False
|
||||
|
||||
formatters:
|
||||
brief:
|
||||
format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
datefmt: '%H:%M:%S'
|
||||
|
||||
precice:
|
||||
format: "%(asctime)s - %(threadName)s - %(name)s - %(levelname)s - %(message)s"
|
||||
datefmt: '%Y-%m-%d %H:%M:%S'
|
||||
|
||||
handlers:
|
||||
console:
|
||||
level: WARNING
|
||||
class: logging.StreamHandler
|
||||
formatter: brief
|
||||
stream: ext://sys.stdout
|
||||
|
||||
root:
|
||||
level: DEBUG
|
||||
handlers: [console]
|
||||
|
||||
loggers:
|
||||
mt5-correlation:
|
||||
level: DEBUG
|
||||
handlers: [console]
|
||||
propagate: 0
|
||||
+29
-71
@@ -1,79 +1,37 @@
|
||||
from datetime import datetime, timedelta
|
||||
import logging.config
|
||||
import pandas as pd
|
||||
import pytz
|
||||
import yaml
|
||||
from mt5_correlation.mt5 import MT5
|
||||
from mt5_correlation.correlation import Correlation
|
||||
"""
|
||||
Application to monitor previously correlated symbol pairs for correlation divergence.
|
||||
"""
|
||||
import definitions
|
||||
import logging.config
|
||||
from mt5_correlation.gui import CorrelationMDIFrame
|
||||
import wxconfig as cfg
|
||||
import wx
|
||||
import wx.lib.mixins.inspection as wit
|
||||
|
||||
# Configure logger
|
||||
with open(fr'{definitions.ROOT_DIR}\logging_conf.yaml', 'rt') as file:
|
||||
config = yaml.safe_load(file.read())
|
||||
logging.config.dictConfig(config)
|
||||
log = logging.getLogger()
|
||||
|
||||
# Create mt5 class. This contains required methods for interacting with MT5.
|
||||
mt5 = MT5()
|
||||
class InspectionApp(wx.App, wit.InspectionMixin):
|
||||
# Override app to use inspection.
|
||||
def OnInit(self):
|
||||
self.Init() # initialize the inspection tool
|
||||
return True
|
||||
|
||||
# Gte all visible symbols
|
||||
symbols = mt5.get_symbols()
|
||||
|
||||
# set time zone to UTC to avoid local offset issues, and get from and to dates (a week ago to today)
|
||||
timezone = pytz.timezone("Etc/UTC")
|
||||
utc_to = datetime.now(tz=timezone)
|
||||
utc_from = utc_to - timedelta(days=7)
|
||||
if __name__ == "__main__":
|
||||
# Load the config
|
||||
cfg.Config().load(fr"{definitions.ROOT_DIR}\config.yaml", meta=fr"{definitions.ROOT_DIR}\configmeta.yaml")
|
||||
|
||||
# Set timeframe
|
||||
timeframe = mt5.TIMEFRAME_M15
|
||||
# Get logging config and configure the logger
|
||||
log_config = cfg.Config().get('logging')
|
||||
logging.config.dictConfig(log_config)
|
||||
|
||||
# Get price data for selected symbols. 1 week of 15 min OHLC data for each symbol. Add to dict.
|
||||
price_data = {}
|
||||
for symbol in symbols:
|
||||
price_data[symbol.name] = mt5.get_prices(symbol=symbol, from_date=utc_from, to_date=utc_to,
|
||||
timeframe=timeframe)
|
||||
# Do we have inspection turned on. Create correct version of app
|
||||
inspection = cfg.Config().get('developer.inspection')
|
||||
if inspection:
|
||||
app = InspectionApp()
|
||||
else:
|
||||
app = wx.App(False)
|
||||
|
||||
# Loop through all symbol pair combinations and calculate coefficient. Make sure you don't double count pairs
|
||||
# eg. (USD/GBP AUD/USD vs AUD/USD USD/GBP). Use grid of all symbols with i and j axis. j starts at i + 1 to
|
||||
# avoid duplicating. We will store all coefficients in a dataframe for export as CSV.
|
||||
columns = ['Symbol 1', 'Symbol 2', 'Coefficient', 'UTC Date From', 'UTC Date To', 'Timeframe']
|
||||
coefficients = pd.DataFrame(columns=columns)
|
||||
|
||||
index = 0
|
||||
# There will be (x^2 - x) / 2 pairs where x is number of symbols
|
||||
num_pair_combinations = int((len(symbols) ** 2 - len(symbols)) / 2)
|
||||
|
||||
for i in range(0, len(symbols)):
|
||||
symbol1 = symbols[i]
|
||||
|
||||
for j in range(i + 1, len(symbols)):
|
||||
symbol2 = symbols[j]
|
||||
index += 1
|
||||
|
||||
# Get price data for both symbols
|
||||
symbol1_price_data = price_data[symbol1.name]
|
||||
symbol2_price_data = price_data[symbol2.name]
|
||||
|
||||
# Get coefficient and store if valid
|
||||
coefficient = Correlation.calculate_coefficient(symbol1_prices=symbol1_price_data,
|
||||
symbol2_prices=symbol2_price_data, max_set_size_diff_pct=90,
|
||||
overlap_pct=90, max_p_value=0.05)
|
||||
|
||||
if coefficient is not None:
|
||||
coefficients = coefficients.append({'Symbol 1': symbol1.name, 'Symbol 2': symbol2.name,
|
||||
'Coefficient': coefficient, 'UTC Date From': utc_from,
|
||||
'UTC Date To': utc_to, 'Timeframe': timeframe}, ignore_index=True)
|
||||
|
||||
log.info(f"Pair {index} of {num_pair_combinations}: {symbol1.name}:{symbol2.name} has a coefficient of "
|
||||
f"{coefficient}.")
|
||||
else:
|
||||
log.info(f"Coefficient for pair {index} of {num_pair_combinations}: {symbol1.name}:{symbol2.name} could not"
|
||||
f" be calculated.")
|
||||
|
||||
# Sort, highest correlated first
|
||||
coefficients = coefficients.sort_values('Coefficient', ascending=False)
|
||||
|
||||
# Save as CSV
|
||||
filename = f"Coefficients from {utc_from:%Y%m%d %H%M%S} to {utc_to:%Y%m%d %H%M%S}.csv"
|
||||
log.info(f"Saving coefficients as '{filename}'.")
|
||||
coefficients.to_csv(filename, index=False)
|
||||
# Start the app
|
||||
frame = CorrelationMDIFrame()
|
||||
frame.Show()
|
||||
app.MainLoop()
|
||||
|
||||
+744
-10
@@ -1,26 +1,409 @@
|
||||
import math
|
||||
import logging
|
||||
import pandas as pd
|
||||
from datetime import datetime, timedelta
|
||||
import time
|
||||
import sched
|
||||
import threading
|
||||
import pytz
|
||||
from scipy.stats.stats import pearsonr
|
||||
import pickle
|
||||
import inspect
|
||||
import sys
|
||||
|
||||
from mt5_correlation.mt5 import MT5
|
||||
|
||||
|
||||
class CorrelationStatus:
|
||||
"""
|
||||
The status of the monitoring event for a symbol pair.
|
||||
"""
|
||||
|
||||
val = None
|
||||
text = None
|
||||
long_text = None
|
||||
|
||||
def __init__(self, status_val, status_text, status_long_text=None):
|
||||
"""
|
||||
Creates a status.
|
||||
:param status_val:
|
||||
:param status_text:
|
||||
:param status_long_text
|
||||
:return:
|
||||
"""
|
||||
self.val = status_val
|
||||
self.text = status_text
|
||||
self.long_text = status_long_text
|
||||
|
||||
def __eq__(self, other):
|
||||
"""
|
||||
Compare the status val. We can compare against other CorrelationStatus instances or against int.
|
||||
:param other:
|
||||
:return:
|
||||
"""
|
||||
if isinstance(other, self.__class__):
|
||||
return self.val == other.val
|
||||
elif isinstance(other, int):
|
||||
return self.val == other
|
||||
else:
|
||||
return False
|
||||
|
||||
def __str__(self):
|
||||
"""
|
||||
str is the text for the status.
|
||||
:return:
|
||||
"""
|
||||
return self.text
|
||||
|
||||
|
||||
# All status's for symbol pair from monitoring. Status set from assessing coefficient for all timeframes from last run.
|
||||
STATUS_NOT_CALCULATED = CorrelationStatus(-1, 'NOT CALC', 'Coefficient could not be calculated')
|
||||
STATUS_CORRELATED = CorrelationStatus(1, 'CORRELATED', 'Coefficients for all timeframes are equal to or above the '
|
||||
'divergence threshold')
|
||||
STATUS_DIVERGED = CorrelationStatus(2, 'DIVERGED', 'Coefficients for all timeframes are below the divergence threshold')
|
||||
STATUS_INCONSISTENT = CorrelationStatus(3, 'INCONSISTENT', 'Coefficients not consistently above or below divergence '
|
||||
'threshold and are neither trending towards divergence or '
|
||||
'convergence')
|
||||
STATUS_DIVERGING = CorrelationStatus(4, 'DIVERGING', 'Coefficients, when ordered by timeframe, are trending '
|
||||
'towards convergence. The shortest timeframe is below the '
|
||||
'divergence threshold and the longest timeframe is above the '
|
||||
'divergence threshold')
|
||||
STATUS_CONVERGING = CorrelationStatus(5, 'CONVERGING', 'Coefficients, when ordered by timeframe, are trending '
|
||||
'towards divergence. The shortest timeframe is above the '
|
||||
'divergence threshold and the longest timeframe is below the '
|
||||
'divergence threshold')
|
||||
|
||||
|
||||
class Correlation:
|
||||
"""
|
||||
A class to calculate the correlation coefficient between two sets of price data
|
||||
A class to maintain the state of the calculated correlation coefficients.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def calculate_coefficient(symbol1_prices, symbol2_prices, max_set_size_diff_pct=90, overlap_pct=90,
|
||||
max_p_value=0.05):
|
||||
# Connection to MetaTrader5
|
||||
__mt5 = None
|
||||
|
||||
# Minimum base coefficient for monitoring. Symbol pairs with a lower correlation
|
||||
# coefficient than ths won't be monitored.
|
||||
monitoring_threshold = 0.9
|
||||
|
||||
# Threshold for divergence. Correlation coefficients that were previously above the monitoring_threshold and fall
|
||||
# below this threshold will be considered as having diverged
|
||||
divergence_threshold = 0.8
|
||||
|
||||
# Flag to determine we monitor and report on inverse correlations
|
||||
monitor_inverse = False
|
||||
|
||||
# Toggle on whether we are monitoring or not. Set through start_monitor and stop_monitor
|
||||
__monitoring = False
|
||||
|
||||
# Monitoring calculation params, interval, cache_time, autosave and filename. Passed to start_monitor
|
||||
__monitoring_params = []
|
||||
__interval = None
|
||||
__cache_time = None
|
||||
__autosave = None
|
||||
__filename = None
|
||||
|
||||
# First run of scheduler
|
||||
__first_run = True
|
||||
|
||||
# The price data used to calculate the correlations
|
||||
__price_data = None
|
||||
|
||||
# Coefficient data and history. Will be created in init call to __reset_coefficient_data
|
||||
coefficient_data = None
|
||||
coefficient_history = None
|
||||
|
||||
# Stores tick data used to calculate coefficient during Monitor.
|
||||
# Dict: {Symbol: [retrieved datetime, ticks dataframe]}
|
||||
__monitor_tick_data = {}
|
||||
|
||||
def __init__(self, monitoring_threshold=0.9, divergence_threshold=0.8, monitor_inverse=False):
|
||||
"""
|
||||
Initialises the Correlation class.
|
||||
:param monitoring_threshold: Only correlations that are greater than or equal to this threshold will be
|
||||
monitored.
|
||||
:param divergence_threshold: Correlations that are being monitored and fall below this threshold are considered
|
||||
to have diverged.
|
||||
:param monitor_inverse: Whether we will monitor and report on negative / inverse correlations.
|
||||
"""
|
||||
# Logger
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Connection to MetaTrader5
|
||||
self.__mt5 = MT5()
|
||||
|
||||
# Create dataframe for coefficient data
|
||||
self.__reset_coefficient_data()
|
||||
|
||||
# Create timer for continuous monitoring
|
||||
self.__scheduler = sched.scheduler(time.time, time.sleep)
|
||||
|
||||
# Set thresholds and flags
|
||||
self.monitoring_threshold = monitoring_threshold
|
||||
self.divergence_threshold = divergence_threshold
|
||||
self.monitor_inverse = monitor_inverse
|
||||
|
||||
@property
|
||||
def filtered_coefficient_data(self):
|
||||
"""
|
||||
:return: Coefficient data filtered so that all base coefficients >= monitoring_threshold
|
||||
"""
|
||||
filtered_data = None
|
||||
if self.coefficient_data is not None:
|
||||
if self.monitor_inverse:
|
||||
filtered_data = self.coefficient_data \
|
||||
.loc[(self.coefficient_data['Base Coefficient'] >= self.monitoring_threshold) |
|
||||
(self.coefficient_data['Base Coefficient'] <= self.monitoring_threshold * -1)]
|
||||
|
||||
else:
|
||||
filtered_data = self.coefficient_data.loc[self.coefficient_data['Base Coefficient'] >=
|
||||
self.monitoring_threshold]
|
||||
|
||||
return filtered_data
|
||||
|
||||
@property
|
||||
def diverged_symbols(self):
|
||||
"""
|
||||
:return: dataframe containing all diverged, diverging or converging symbols and count of number of
|
||||
divergences for those symbols.
|
||||
"""
|
||||
filtered_data = None
|
||||
|
||||
if self.coefficient_data is not None:
|
||||
# Only rows where we have a divergence
|
||||
filtered_data = self.coefficient_data \
|
||||
.loc[(self.coefficient_data['Status'] == STATUS_DIVERGED) |
|
||||
(self.coefficient_data['Status'] == STATUS_DIVERGING) |
|
||||
(self.coefficient_data['Status'] == STATUS_CONVERGING)]
|
||||
|
||||
# We only need the symbols
|
||||
all_symbols = pd.DataFrame(columns=['Symbol', 'Count'],
|
||||
data={'Symbol': filtered_data['Symbol 1'].append(filtered_data['Symbol 2']),
|
||||
'Count': 1})
|
||||
|
||||
# Group and count. Reset index so that we have named SYMBOL column.
|
||||
filtered_data = all_symbols.groupby(by='Symbol').count().reset_index()
|
||||
|
||||
# Sort
|
||||
filtered_data = filtered_data.sort_values('Count', ascending=False)
|
||||
|
||||
return filtered_data
|
||||
|
||||
def load(self, filename):
|
||||
"""
|
||||
Loads calculated coefficients, price data used to calculate them and tick data used during monitoring.
|
||||
coefficients
|
||||
:param filename: The filename for the coefficient data to load.
|
||||
:return:
|
||||
"""
|
||||
# Load data
|
||||
with open(filename, 'rb') as file:
|
||||
loaded_dict = pickle.load(file)
|
||||
|
||||
# Get data from loaded dict and save
|
||||
self.coefficient_data = loaded_dict["coefficient_data"]
|
||||
self.__price_data = loaded_dict["price_data"]
|
||||
self.__monitor_tick_data = loaded_dict["monitor_tick_data"]
|
||||
self.coefficient_history = loaded_dict["coefficient_history"]
|
||||
|
||||
def save(self, filename):
|
||||
"""
|
||||
Saves the calculated coefficients, the price data used to calculate and the tick data for monitoring to a file.
|
||||
:param filename: The filename to save the data to.
|
||||
:return:
|
||||
"""
|
||||
# Add data to dict then use pickle to save
|
||||
save_dict = {"coefficient_data": self.coefficient_data, "price_data": self.__price_data,
|
||||
"monitor_tick_data": self.__monitor_tick_data, "coefficient_history": self.coefficient_history}
|
||||
with open(filename, 'wb') as file:
|
||||
pickle.dump(save_dict, file, protocol=pickle.HIGHEST_PROTOCOL)
|
||||
|
||||
def calculate(self, date_from, date_to, timeframe, min_prices=100, max_set_size_diff_pct=90, overlap_pct=90,
|
||||
max_p_value=0.05):
|
||||
"""
|
||||
Calculates correlation coefficient between all symbols in MetaTrader5 Market Watch. Updates coefficient data.
|
||||
|
||||
:param date_from: From date for price data from which to calculate correlation coefficients
|
||||
:param date_to: To date for price data from which to calculate correlation coefficients
|
||||
:param timeframe: Timeframe for price data from which to calculate correlation coefficients
|
||||
:param min_prices: The minimum number of prices that should be used to calculate coefficient. If this threshold
|
||||
is not met then returned coefficient will be None
|
||||
:param max_set_size_diff_pct: Correlations will only be calculated if the sizes of the two price data sets are
|
||||
within this pct of each other
|
||||
:param overlap_pct:The dates and times in the two sets of data must match. The coefficient will only be
|
||||
calculated against the dates that overlap. Any non overlapping dates will be discarded. This setting
|
||||
specifies the minimum size of the overlapping data when compared to the smallest set as a %. A coefficient
|
||||
will not be calculated if this threshold is not met.
|
||||
:param max_p_value: The maximum p value for the correlation to be meaningful
|
||||
|
||||
:return:
|
||||
"""
|
||||
|
||||
coefficient = None
|
||||
|
||||
# If we are monitoring, stop. We will need to restart later
|
||||
was_monitoring = self.__monitoring
|
||||
if self.__monitoring:
|
||||
self.stop_monitor()
|
||||
|
||||
# Clear the existing correlations
|
||||
self.__reset_coefficient_data()
|
||||
|
||||
# Get all visible symbols
|
||||
symbols = self.__mt5.get_symbols()
|
||||
|
||||
# Get price data for selected symbols. 1 week of 15 min OHLC data for each symbol. Add to dict.
|
||||
self.__price_data = {}
|
||||
for symbol in symbols:
|
||||
self.__price_data[symbol] = self.__mt5.get_prices(symbol=symbol, from_date=date_from, to_date=date_to,
|
||||
timeframe=timeframe)
|
||||
|
||||
# Loop through all symbol pair combinations and calculate coefficient. Make sure you don't double count pairs
|
||||
# eg. (USD/GBP AUD/USD vs AUD/USD USD/GBP). Use grid of all symbols with i and j axis. j starts at i + 1 to
|
||||
# avoid duplicating. We will store all coefficients in a dataframe.
|
||||
index = 0
|
||||
# There will be (x^2 - x) / 2 pairs where x is number of symbols
|
||||
num_pair_combinations = int((len(symbols) ** 2 - len(symbols)) / 2)
|
||||
|
||||
for i in range(0, len(symbols)):
|
||||
symbol1 = symbols[i]
|
||||
|
||||
for j in range(i + 1, len(symbols)):
|
||||
symbol2 = symbols[j]
|
||||
index += 1
|
||||
|
||||
# Get price data for both symbols
|
||||
symbol1_price_data = self.__price_data[symbol1]
|
||||
symbol2_price_data = self.__price_data[symbol2]
|
||||
|
||||
# Get coefficient
|
||||
if symbol1_price_data is not None and symbol2_price_data is not None:
|
||||
coefficient = self.calculate_coefficient(symbol1_prices=symbol1_price_data,
|
||||
symbol2_prices=symbol2_price_data,
|
||||
min_prices=min_prices,
|
||||
max_set_size_diff_pct=max_set_size_diff_pct,
|
||||
overlap_pct=overlap_pct, max_p_value=max_p_value)
|
||||
|
||||
# Store if valid
|
||||
if coefficient is not None:
|
||||
|
||||
self.coefficient_data = \
|
||||
self.coefficient_data.append({'Symbol 1': symbol1, 'Symbol 2': symbol2,
|
||||
'Base Coefficient': coefficient, 'UTC Date From': date_from,
|
||||
'UTC Date To': date_to, 'Timeframe': timeframe, 'Status': ''},
|
||||
ignore_index=True)
|
||||
|
||||
self.__log.debug(f"Pair {index} of {num_pair_combinations}: {symbol1}:{symbol2} has a "
|
||||
f"coefficient of {coefficient}.")
|
||||
else:
|
||||
self.__log.debug(f"Coefficient for pair {index} of {num_pair_combinations}: {symbol1}:"
|
||||
f"{symbol2} could no be calculated.")
|
||||
|
||||
# Sort, highest correlated first
|
||||
self.coefficient_data = self.coefficient_data.sort_values('Base Coefficient', ascending=False)
|
||||
|
||||
# If we were monitoring, we stopped, so start again.
|
||||
if was_monitoring:
|
||||
self.start_monitor(interval=self.__interval, calculation_params=self.__monitoring_params,
|
||||
cache_time=self.__cache_time, autosave=self.__autosave, filename=self.__filename)
|
||||
|
||||
def get_price_data(self, symbol):
|
||||
"""
|
||||
Returns the price data used to calculate the base coefficients for the specified symbol
|
||||
:param symbol: Symbol to get price data for.
|
||||
:return: price data
|
||||
"""
|
||||
price_data = None
|
||||
if symbol in self.__price_data:
|
||||
price_data = self.__price_data[symbol]
|
||||
|
||||
return price_data
|
||||
|
||||
def start_monitor(self, interval, calculation_params, cache_time=10, autosave=False, filename='autosave.cpd'):
|
||||
"""
|
||||
Starts monitor to continuously update the coefficient for all symbol pairs in that meet the min_coefficient
|
||||
threshold.
|
||||
|
||||
:param interval: How often to check in seconds
|
||||
:param calculation_params: A single dict or list of dicts containing the parameters for the coefficient
|
||||
calculations. On every iteration, a coefficient will be calculated for every set of params in list. Params
|
||||
contain the following values:
|
||||
from: The number of minutes of tick data to use for calculation. This can be a single value or
|
||||
a list. If a list, then calculations will be performed for every from date in list.
|
||||
min_prices: The minimum number of prices that should be used to calculate coefficient. If this threshold
|
||||
is not met then returned coefficient will be None
|
||||
max_set_size_diff_pct: Correlations will only be calculated if the sizes of the two price data sets are
|
||||
within this pct of each other
|
||||
overlap_pct: The dates and times in the two sets of data must match. The coefficient will only be
|
||||
calculated against the dates that overlap. Any non overlapping dates will be discarded. This
|
||||
setting specifies the minimum size of the overlapping data when compared to the smallest set as a %.
|
||||
A coefficient will not be calculated if this threshold is not met.
|
||||
max_p_value: The maximum p value for the correlation to be meaningful
|
||||
:param cache_time: Tick data is cached so that we can check coefficients for multiple symbol pairs and reuse
|
||||
the tick data. Number of seconds to cache tick data for before it becomes stale.
|
||||
:param autosave: Whether to autosave after every monitor run. If there is no filename specified then will
|
||||
create one named autosave.cpd
|
||||
:param filename: Filename for autosave. Default is autosave.cpd.
|
||||
|
||||
:return: correlation coefficient, or None if coefficient could not be calculated.
|
||||
"""
|
||||
|
||||
if self.__monitoring:
|
||||
self.__log.debug(f"Request to start monitor when monitor is already running. Monitor will be stopped and"
|
||||
f"restarted with new parameters.")
|
||||
self.stop_monitor()
|
||||
|
||||
self.__log.debug(f"Starting monitor.")
|
||||
self.__monitoring = True
|
||||
|
||||
# Store the calculation params. If it isn't a list, convert to list of one to make code simpler later on.
|
||||
self.__monitoring_params = calculation_params if isinstance(calculation_params, list) \
|
||||
else [calculation_params, ]
|
||||
|
||||
# Store the other params. We will need these later if monitor is stopped and needs to be restarted. This
|
||||
# happens in calculate.
|
||||
self.__interval = interval
|
||||
self.__cache_time = cache_time
|
||||
self.__autosave = autosave
|
||||
self.__filename = filename
|
||||
|
||||
# Create thread to run monitoring This will call private __monitor method that will run the calculation and
|
||||
# keep scheduling itself while self.monitoring is True.
|
||||
thread = threading.Thread(target=self.__monitor)
|
||||
thread.start()
|
||||
|
||||
def stop_monitor(self):
|
||||
"""
|
||||
Stops monitoring symbol pairs for correlation.
|
||||
:return:
|
||||
"""
|
||||
if self.__monitoring:
|
||||
self.__log.debug(f"Stopping monitor.")
|
||||
self.__monitoring = False
|
||||
else:
|
||||
self.__log.debug(f"Request to stop monitor when it is not running. No action taken.")
|
||||
|
||||
def calculate_coefficient(self, symbol1_prices, symbol2_prices, min_prices: int = 100,
|
||||
max_set_size_diff_pct: int = 90, overlap_pct: int = 90,
|
||||
max_p_value: float = 0.05):
|
||||
"""
|
||||
Calculates the correlation coefficient between two sets of price data. Uses close price.
|
||||
|
||||
:param symbol1_prices:
|
||||
:param symbol2_prices:
|
||||
:param symbol1_prices: Pandas dataframe containing prices for symbol 1
|
||||
:param symbol2_prices: Pandas dataframe containing prices for symbol 2
|
||||
:param min_prices: The minimum number of prices that should be used to calculate coefficient. If this threshold
|
||||
is not met then returned coefficient will be None
|
||||
:param max_set_size_diff_pct: Correlations will only be calculated if the sizes of the two price data sets are
|
||||
within this pct of each other
|
||||
:param overlap_pct:
|
||||
:param max_p_value: The maximum p value for the correlation to be meaningful
|
||||
|
||||
:return: correlation coefficient, or None if coefficient could not be calculated.
|
||||
:rtype: float or None
|
||||
"""
|
||||
|
||||
assert symbol1_prices is not None and symbol2_prices is not None
|
||||
|
||||
# Calculate size of intersection and determine if prices for symbols have enough overlapping timestamps for
|
||||
# correlation coefficient calculation to be meaningful. Is the smallest set at least max_set_size_diff_pct % of
|
||||
# the size of the largest set and is the overlap set size at least overlap_pct % the size of the smallest set?
|
||||
@@ -31,7 +414,8 @@ class Correlation:
|
||||
len_largest_set = int(max([len(symbol1_prices.index), len(symbol2_prices.index)]))
|
||||
similar_size = len_largest_set * (max_set_size_diff_pct / 100) <= len_smallest_set
|
||||
enough_overlap = len(intersect_dates) >= len_smallest_set * (overlap_pct / 100)
|
||||
suitable = similar_size and enough_overlap
|
||||
enough_prices = len_smallest_set >= min_prices
|
||||
suitable = similar_size and enough_overlap and enough_prices
|
||||
|
||||
if suitable:
|
||||
# Calculate coefficient on close prices
|
||||
@@ -40,16 +424,366 @@ class Correlation:
|
||||
symbol1_prices_filtered = symbol1_prices[symbol1_prices['time'].isin(intersect_dates)]
|
||||
symbol2_prices_filtered = symbol2_prices[symbol2_prices['time'].isin(intersect_dates)]
|
||||
|
||||
# Calculate coefficient. Only use if p value is < 0.01 (highly likely that coefficient is valid and null
|
||||
# hypothesis is false).
|
||||
# Calculate coefficient. Only use if p value is < max_p_value (highly likely that coefficient is valid
|
||||
# and null hypothesis is false).
|
||||
coefficient_with_p_value = pearsonr(symbol1_prices_filtered['close'], symbol2_prices_filtered['close'])
|
||||
coefficient = None if coefficient_with_p_value[1] >= max_p_value else coefficient_with_p_value[0]
|
||||
coefficient = None if coefficient_with_p_value[1] > max_p_value else coefficient_with_p_value[0]
|
||||
|
||||
# If NaN, change to None
|
||||
if coefficient is not None and math.isnan(coefficient):
|
||||
coefficient = None
|
||||
|
||||
self.__log.debug(f"Calculate coefficient returning {coefficient}. "
|
||||
f"Symbol 1 Prices: {len(symbol1_prices)} Symbol 2 Prices: {len(symbol2_prices)} "
|
||||
f"Overlap Prices: {len(intersect_dates)} Similar size: {similar_size} "
|
||||
f"Enough overlap: {enough_overlap} Enough prices: {enough_prices} Suitable: {suitable}.")
|
||||
|
||||
return coefficient
|
||||
|
||||
def get_coefficient_history(self, filters=None):
|
||||
"""
|
||||
Returns the coefficient history that matches the supplied filter.
|
||||
:param filters: Dict of all filters to apply. Possible values in dict are:
|
||||
Symbol 1
|
||||
Symbol 2
|
||||
Coefficient
|
||||
Timeframe
|
||||
Date From
|
||||
Date To
|
||||
|
||||
If filter is not supplied, then all history is returned.
|
||||
|
||||
:return: dataframe containing history of coefficient data.
|
||||
"""
|
||||
history = self.coefficient_history
|
||||
|
||||
# Apply filters
|
||||
if filters is not None:
|
||||
for key in filters:
|
||||
if key in history.columns:
|
||||
history = history[history[key] == filters[key]]
|
||||
else:
|
||||
self.__log.warning(f"Invalid column name provided for filter. Filter column: {key} "
|
||||
f"Valid columns: {history.columns}")
|
||||
|
||||
return history
|
||||
|
||||
def clear_coefficient_history(self):
|
||||
"""
|
||||
Clears the coefficient history for all symbol pairs
|
||||
:return:
|
||||
"""
|
||||
# Create dataframes for coefficient history.
|
||||
coefficient_history_columns = ['Symbol 1', 'Symbol 2', 'Coefficient', 'Timeframe', 'Date To']
|
||||
self.coefficient_history = pd.DataFrame(columns=coefficient_history_columns)
|
||||
|
||||
# Clear tick data
|
||||
self.__monitor_tick_data = {}
|
||||
|
||||
# Clear status from coefficient data
|
||||
self.coefficient_data['Status'] = ''
|
||||
|
||||
def get_ticks(self, symbol, date_from=None, date_to=None, cache_only=False):
|
||||
"""
|
||||
Returns the ticks for the specified symbol. Get's from cache if available and not older than cache_timeframe.
|
||||
|
||||
:param symbol: Name of symbol to get ticks for.
|
||||
:param date_from: Date to get ticks from. Can only be None if getting from cache (cache_only=True)
|
||||
:param date_to:Date to get ticks to. Can only be None if getting from cache (cache_only=True)
|
||||
:param cache_only: Only retrieve from cache. cache_time is ignored. Returns None if symbol is not available in
|
||||
cache.
|
||||
|
||||
:return:
|
||||
"""
|
||||
|
||||
timezone = pytz.timezone("Etc/UTC")
|
||||
utc_now = datetime.now(tz=timezone)
|
||||
|
||||
ticks = None
|
||||
|
||||
# Cache only
|
||||
if cache_only:
|
||||
if symbol in self.__monitor_tick_data:
|
||||
ticks = self.__monitor_tick_data[symbol][1]
|
||||
# Check if we have a cache time defined, if we already have the tick data and it is not stale
|
||||
elif self.__cache_time is not None and symbol in self.__monitor_tick_data and utc_now < \
|
||||
self.__monitor_tick_data[symbol][0] + timedelta(seconds=self.__cache_time):
|
||||
# Cached ticks are not stale. Get them
|
||||
ticks = self.__monitor_tick_data[symbol][1]
|
||||
self.__log.debug(f"Ticks for {symbol} retrieved from cache.")
|
||||
else:
|
||||
# Data does not exist in cache or cached data is stale. Retrieve from source and cache.
|
||||
ticks = self.__mt5.get_ticks(symbol=symbol, from_date=date_from, to_date=date_to)
|
||||
self.__monitor_tick_data[symbol] = [utc_now, ticks]
|
||||
self.__log.debug(f"Ticks for {symbol} retrieved from source and cached.")
|
||||
return ticks
|
||||
|
||||
def get_last_status(self, symbol1, symbol2):
|
||||
"""
|
||||
Get the last status for the specified symbol pair.
|
||||
:param symbol1
|
||||
:param symbol2
|
||||
|
||||
:return: CorrelationStatus instance for symbol pair
|
||||
"""
|
||||
status_col = self.coefficient_data.loc[(self.coefficient_data['Symbol 1'] == symbol1) &
|
||||
(self.coefficient_data['Symbol 2'] == symbol2), 'Status']
|
||||
status = status_col.values[0]
|
||||
return status
|
||||
|
||||
def get_last_calculation(self, symbol1=None, symbol2=None):
|
||||
"""
|
||||
Get the last calculation time the specified symbol pair. If no symbols are specified, then gets the last
|
||||
calculation time across all pairs
|
||||
:param symbol1
|
||||
:param symbol2
|
||||
|
||||
:return: last calculation time
|
||||
"""
|
||||
last_calc = None
|
||||
if self.coefficient_data is not None and len(self.coefficient_data.index) > 0:
|
||||
data = self.coefficient_data.copy()
|
||||
|
||||
# Filter by symbols if specified
|
||||
data = data.loc[data['Symbol 1'] == symbol1] if symbol1 is not None else data
|
||||
data = data.loc[data['Symbol 2'] == symbol2] if symbol2 is not None else data
|
||||
|
||||
# Filter to remove blank dates
|
||||
data = data.dropna(subset=['Last Calculation'])
|
||||
|
||||
# Get the column
|
||||
col = data['Last Calculation']
|
||||
|
||||
# Get max date from column
|
||||
if col is not None and len(col) > 0:
|
||||
last_calc = max(col.values)
|
||||
|
||||
return last_calc
|
||||
|
||||
def get_base_coefficient(self, symbol1, symbol2):
|
||||
"""
|
||||
Returns the base coefficient for the specified symbol pair
|
||||
:param symbol1:
|
||||
:param symbol2:
|
||||
:return:
|
||||
"""
|
||||
base_coefficient = None
|
||||
if self.coefficient_data is not None:
|
||||
row = self.coefficient_data[(self.coefficient_data['Symbol 1'] == symbol1) &
|
||||
(self.coefficient_data['Symbol 2'] == symbol2)]
|
||||
|
||||
if row is not None and len(row) == 1:
|
||||
base_coefficient = row.iloc[0]['Base Coefficient']
|
||||
|
||||
return base_coefficient
|
||||
|
||||
def __monitor(self):
|
||||
"""
|
||||
The actual monitor method. Private. This should not be called outside of this class. Use start_monitoring and
|
||||
stop_monitoring.
|
||||
|
||||
:return: correlation coefficient, or None if coefficient could not be calculated.
|
||||
"""
|
||||
self.__log.debug(f"In monitor event. Monitoring: {self.__monitoring}.")
|
||||
|
||||
# Only run if monitor is not stopped
|
||||
if self.__monitoring:
|
||||
# Update all coefficients
|
||||
self.__update_all_coefficients()
|
||||
|
||||
# Autosave
|
||||
if self.__autosave:
|
||||
self.save(filename=self.__filename)
|
||||
|
||||
# Schedule the timer to run again
|
||||
self.__scheduler.enter(delay=self.__interval, priority=1, action=self.__monitor)
|
||||
|
||||
# Log the stack. Debug stack overflow
|
||||
self.__log.debug(f"Current stack size: {len(inspect.stack())} Recursion limit: {sys.getrecursionlimit()}")
|
||||
|
||||
# Run
|
||||
if self.__first_run:
|
||||
self.__first_run = False
|
||||
self.__scheduler.run()
|
||||
|
||||
def __update_coefficients(self, symbol1, symbol2):
|
||||
"""
|
||||
Updates the coefficients for the specified symbol pair
|
||||
:param symbol1: Name of symbol to calculate coefficient for.
|
||||
:param symbol2: Name of symbol to calculate coefficient for.
|
||||
:return: correlation coefficient, or None if coefficient could not be calculated.
|
||||
"""
|
||||
|
||||
# Get the largest value of from in monitoring_params. This will be used to retrieve the data. We will only
|
||||
# retrieve once and use for every set of params by getting subset of the data.
|
||||
max_from = None
|
||||
for params in self.__monitoring_params:
|
||||
if max_from is None:
|
||||
max_from = params['from']
|
||||
else:
|
||||
max_from = max(max_from, params['from'])
|
||||
|
||||
# Date range for data
|
||||
timezone = pytz.timezone("Etc/UTC")
|
||||
date_to = datetime.now(tz=timezone)
|
||||
date_from = date_to - timedelta(minutes=max_from)
|
||||
|
||||
# Get the tick data for the longest timeframe calculation.
|
||||
symbol1ticks = self.get_ticks(symbol=symbol1, date_from=date_from, date_to=date_to)
|
||||
symbol2ticks = self.get_ticks(symbol=symbol2, date_from=date_from, date_to=date_to)
|
||||
|
||||
# Resample to 1 sec OHLC, this will help with coefficient calculation ensuring that we dont have more than
|
||||
# one tick per second and ensuring that times can match. We will need to set the index to time for the
|
||||
# resample then revert back to a 'time' column. We will then need to remove rows with nan in 'close' price
|
||||
s1_prices = None
|
||||
s2_prices = None
|
||||
if symbol1ticks is not None and symbol2ticks is not None and len(symbol1ticks.index) > 0 and \
|
||||
len(symbol2ticks.index) > 0:
|
||||
|
||||
try:
|
||||
symbol1ticks = symbol1ticks.set_index('time')
|
||||
symbol2ticks = symbol2ticks.set_index('time')
|
||||
s1_prices = symbol1ticks['ask'].resample('1S').ohlc()
|
||||
s2_prices = symbol2ticks['ask'].resample('1S').ohlc()
|
||||
except RecursionError:
|
||||
self.__log.warning(f"Coefficient could not be calculated for {symbol1}:{symbol2}. prices could not "
|
||||
f"be resampled.")
|
||||
else:
|
||||
s1_prices.reset_index(inplace=True)
|
||||
s2_prices.reset_index(inplace=True)
|
||||
s1_prices = s1_prices[s1_prices['close'].notna()]
|
||||
s2_prices = s2_prices[s2_prices['close'].notna()]
|
||||
|
||||
# Calculate for all sets of monitoring_params
|
||||
if s1_prices is not None and s2_prices is not None:
|
||||
coefficients = {}
|
||||
for params in self.__monitoring_params:
|
||||
# Get the from date as a datetime64
|
||||
date_from_subset = pd.Timestamp(date_to - timedelta(minutes=params['from'])).to_datetime64()
|
||||
|
||||
# Get subset of the price data
|
||||
s1_prices_subset = s1_prices[(s1_prices['time'] >= date_from_subset)]
|
||||
s2_prices_subset = s2_prices[(s2_prices['time'] >= date_from_subset)]
|
||||
|
||||
# Calculate the coefficient
|
||||
coefficient = \
|
||||
self.calculate_coefficient(symbol1_prices=s1_prices_subset, symbol2_prices=s2_prices_subset,
|
||||
min_prices=params['min_prices'],
|
||||
max_set_size_diff_pct=params['max_set_size_diff_pct'],
|
||||
overlap_pct=params['overlap_pct'], max_p_value=params['max_p_value'])
|
||||
|
||||
self.__log.debug(f"Symbol pair {symbol1}:{symbol2} has a coefficient of {coefficient} for last "
|
||||
f"{params['from']} minutes.")
|
||||
|
||||
# Add the coefficient to a dict {timeframe: coefficient}. We will update together for all for
|
||||
# symbol pair and time
|
||||
coefficients[params['from']] = coefficient
|
||||
|
||||
# Update coefficient data for all coefficients for all timeframes for this run and symbol pair.
|
||||
self.__update_coefficient_data(symbol1=symbol1, symbol2=symbol2, coefficients=coefficients,
|
||||
date_to=date_to)
|
||||
|
||||
def __update_all_coefficients(self):
|
||||
"""
|
||||
Updates the coefficient for all symbol pairs in that meet the min_coefficient threshold. Symbol pairs that meet
|
||||
the threshold can be accessed through the filtered_coefficient_data property.
|
||||
"""
|
||||
# Update latest coefficient for every pair
|
||||
for index, row in self.filtered_coefficient_data.iterrows():
|
||||
symbol1 = row['Symbol 1']
|
||||
symbol2 = row['Symbol 2']
|
||||
self.__update_coefficients(symbol1=symbol1, symbol2=symbol2)
|
||||
|
||||
def __reset_coefficient_data(self):
|
||||
"""
|
||||
Clears coefficient data and history.
|
||||
:return:
|
||||
"""
|
||||
# Create dataframes for coefficient data.
|
||||
coefficient_data_columns = ['Symbol 1', 'Symbol 2', 'Base Coefficient', 'UTC Date From', 'UTC Date To',
|
||||
'Timeframe', 'Last Calculation', 'Status']
|
||||
self.coefficient_data = pd.DataFrame(columns=coefficient_data_columns)
|
||||
|
||||
# Clear coefficient history
|
||||
self.clear_coefficient_history()
|
||||
|
||||
# Clear price data
|
||||
self.__price_data = None
|
||||
|
||||
def __update_coefficient_data(self, symbol1, symbol2, coefficients, date_to):
|
||||
"""
|
||||
Updates the coefficient data with the latest coefficient and adds to coefficient history.
|
||||
:param symbol1:
|
||||
:param symbol2:
|
||||
:param coefficients: Dict of all coefficients calculated for this run and symbol pair. {timeframe: coefficient}
|
||||
:param date_to: The date from for which the coefficient was calculated
|
||||
:return:
|
||||
"""
|
||||
|
||||
timezone = pytz.timezone("Etc/UTC")
|
||||
now = datetime.now(tz=timezone)
|
||||
|
||||
# Update data if we have a coefficient and add to history
|
||||
if coefficients is not None:
|
||||
# Update the coefficient data table with the Last Calculation time.
|
||||
self.coefficient_data.loc[(self.coefficient_data['Symbol 1'] == symbol1) &
|
||||
(self.coefficient_data['Symbol 2'] == symbol2),
|
||||
'Last Calculation'] = now
|
||||
|
||||
# Are we an inverse correlation
|
||||
inverse = self.get_base_coefficient(symbol1, symbol2) <= self.monitoring_threshold * -1
|
||||
|
||||
# Calculate status and update
|
||||
status = self.__calculate_status(coefficients=coefficients, inverse=inverse)
|
||||
self.coefficient_data.loc[(self.coefficient_data['Symbol 1'] == symbol1) &
|
||||
(self.coefficient_data['Symbol 2'] == symbol2),
|
||||
'Status'] = status
|
||||
|
||||
# Update history data
|
||||
for key in coefficients:
|
||||
row = pd.DataFrame(columns=self.coefficient_history.columns,
|
||||
data=[[symbol1, symbol2, coefficients[key], key, date_to]])
|
||||
self.coefficient_history = self.coefficient_history.append(row)
|
||||
|
||||
def __calculate_status(self, coefficients, inverse):
|
||||
"""
|
||||
Calculates the status from the supplied set of coefficients
|
||||
:param coefficients: Dict of timeframes and coefficients {timeframe: coefficient} to calculate status from
|
||||
:param: Whether we are calculating status based on normal or inverse correlation
|
||||
:return: status
|
||||
"""
|
||||
status = STATUS_NOT_CALCULATED
|
||||
|
||||
# Only continue if we have calculated all coefficients, otherwise we will return STATUS_NOT_CALCULATED
|
||||
if None not in coefficients.values():
|
||||
# Get the values ordered by timeframe descending
|
||||
ordered_values = []
|
||||
for key in sorted(coefficients, reverse=True):
|
||||
ordered_values.append(coefficients[key])
|
||||
|
||||
if self.monitor_inverse and inverse:
|
||||
# Calculation for inverse calculations
|
||||
if all(i <= self.divergence_threshold * -1 for i in ordered_values):
|
||||
status = STATUS_CORRELATED
|
||||
elif all(i > self.divergence_threshold * -1 for i in ordered_values):
|
||||
status = STATUS_DIVERGED
|
||||
elif all(ordered_values[i] <= ordered_values[i+1] for i in range(0, len(ordered_values)-1, 1)):
|
||||
status = STATUS_CONVERGING
|
||||
elif all(ordered_values[i] > ordered_values[i+1] for i in range(0, len(ordered_values)-1, 1)):
|
||||
status = STATUS_DIVERGING
|
||||
else:
|
||||
status = STATUS_INCONSISTENT
|
||||
else:
|
||||
# Calculation for standard correlations
|
||||
if all(i >= self.divergence_threshold for i in ordered_values):
|
||||
status = STATUS_CORRELATED
|
||||
elif all(i < self.divergence_threshold for i in ordered_values):
|
||||
status = STATUS_DIVERGED
|
||||
elif all(ordered_values[i] <= ordered_values[i+1] for i in range(0, len(ordered_values)-1, 1)):
|
||||
status = STATUS_DIVERGING
|
||||
elif all(ordered_values[i] > ordered_values[i+1] for i in range(0, len(ordered_values)-1, 1)):
|
||||
status = STATUS_CONVERGING
|
||||
else:
|
||||
status = STATUS_INCONSISTENT
|
||||
|
||||
return status
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
from mt5_correlation.gui.mdi import CorrelationMDIFrame
|
||||
@@ -0,0 +1,359 @@
|
||||
import abc
|
||||
import importlib
|
||||
import logging
|
||||
|
||||
import pytz
|
||||
import wx
|
||||
import wx.lib.inspection as ins
|
||||
import wxconfig
|
||||
import wxconfig as cfg
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from mt5_correlation import correlation as cor
|
||||
|
||||
|
||||
class CorrelationMDIFrame(wx.MDIParentFrame):
|
||||
"""
|
||||
The MDI Frame window for the correlation monitoring application
|
||||
"""
|
||||
# The correlation instance that calculates coefficients and monitors for divergence. Needs to be accessible to
|
||||
# child frames.
|
||||
cor = None
|
||||
|
||||
__opened_filename = None # So we can save to same file as we opened
|
||||
__log = None # The logger
|
||||
__menu_item_monitor = None # We need to store this menu item so that we can check if it is checked or not.
|
||||
|
||||
def __init__(self):
|
||||
# Super
|
||||
wx.MDIParentFrame.__init__(self, parent=None, id=wx.ID_ANY, title="Divergence Monitor",
|
||||
pos=wx.Point(x=cfg.Config().get('window.x'), y=cfg.Config().get('window.y')),
|
||||
size=wx.Size(width=cfg.Config().get('window.width'),
|
||||
height=cfg.Config().get('window.height')),
|
||||
style=cfg.Config().get('window.style'))
|
||||
|
||||
# Create logger
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Create correlation instance to maintain state of calculated coefficients. Set params from config
|
||||
self.cor = cor.Correlation(monitoring_threshold=cfg.Config().get("monitor.monitoring_threshold"),
|
||||
divergence_threshold=cfg.Config().get("monitor.divergence_threshold"),
|
||||
monitor_inverse=cfg.Config().get("monitor.monitor_inverse"))
|
||||
|
||||
# Status bar. 2 fields, one for monitoring status and one for general status. On open, monitoring status is not
|
||||
# monitoring. SetBackgroundColour will change colour of both. Couldn't find a way to set on single field only.
|
||||
self.__statusbar = self.CreateStatusBar(2)
|
||||
self.__statusbar.SetStatusWidths([100, -1])
|
||||
self.SetStatusText("Not Monitoring", 0)
|
||||
|
||||
# Create menu bar and bind menu items to methods
|
||||
self.menubar = wx.MenuBar()
|
||||
|
||||
# File menu and items
|
||||
menu_file = wx.Menu()
|
||||
self.Bind(wx.EVT_MENU, self.__on_open_file, menu_file.Append(wx.ID_ANY, "&Open", "Open correlations file."))
|
||||
self.Bind(wx.EVT_MENU, self.__on_save_file, menu_file.Append(wx.ID_ANY, "Save", "Save correlations file."))
|
||||
self.Bind(wx.EVT_MENU, self.__on_save_file_as,
|
||||
menu_file.Append(wx.ID_ANY, "Save As", "Save correlations file."))
|
||||
menu_file.AppendSeparator()
|
||||
self.Bind(wx.EVT_MENU, self.__on_open_settings,
|
||||
menu_file.Append(wx.ID_ANY, "Settings", "Change application settings."))
|
||||
menu_file.AppendSeparator()
|
||||
self.Bind(wx.EVT_MENU, self.__on_exit, menu_file.Append(wx.ID_ANY, "Exit", "Close the application"))
|
||||
self.menubar.Append(menu_file, "&File")
|
||||
|
||||
# Coefficient menu and items
|
||||
menu_coef = wx.Menu()
|
||||
self.Bind(wx.EVT_MENU, self.__on_calculate,
|
||||
menu_coef.Append(wx.ID_ANY, "Calculate", "Calculate base coefficients."))
|
||||
self.__menu_item_monitor = menu_coef.Append(wx.ID_ANY, "Monitor",
|
||||
"Monitor correlated pairs for changes to coefficient.",
|
||||
kind=wx.ITEM_CHECK)
|
||||
self.Bind(wx.EVT_MENU, self.__on_monitor, self.__menu_item_monitor)
|
||||
menu_coef.AppendSeparator()
|
||||
self.Bind(wx.EVT_MENU, self.__on_clear,
|
||||
menu_coef.Append(wx.ID_ANY, "Clear", "Clear coefficient and price history."))
|
||||
self.menubar.Append(menu_coef, "Coefficient")
|
||||
|
||||
# View menu and items
|
||||
menu_view = wx.Menu()
|
||||
self.Bind(wx.EVT_MENU, self.__on_view_status, menu_view.Append(wx.ID_ANY, "Status",
|
||||
"View status of correlations."))
|
||||
self.Bind(wx.EVT_MENU, self.__on_view_diverged, menu_view.Append(wx.ID_ANY, "Diverged Symbols",
|
||||
"View diverged symbols."))
|
||||
self.menubar.Append(menu_view, "&View")
|
||||
|
||||
# Help menu and items
|
||||
help_menu = wx.Menu()
|
||||
self.Bind(wx.EVT_MENU, self.__on_view_log, help_menu.Append(wx.ID_ANY, "View Log", "Show application log."))
|
||||
self.Bind(wx.EVT_MENU, self.__on_view_help, help_menu.Append(wx.ID_ANY, "Help",
|
||||
"Show application usage instructions."))
|
||||
|
||||
self.menubar.Append(help_menu, "&Help")
|
||||
|
||||
# Set menu bar
|
||||
self.SetMenuBar(self.menubar)
|
||||
|
||||
# Set up timer to refresh
|
||||
self.timer = wx.Timer(self)
|
||||
self.Bind(wx.EVT_TIMER, self.__on_timer, self.timer)
|
||||
|
||||
# Bind window close event
|
||||
self.Bind(wx.EVT_CLOSE, self.__on_close, self)
|
||||
|
||||
def __on_close(self, event):
|
||||
"""
|
||||
Window closing. Save coefficients and stop monitoring.
|
||||
:param event:
|
||||
:return:
|
||||
"""
|
||||
# Save pos and size
|
||||
x, y = self.GetPosition()
|
||||
width, height = self.GetSize()
|
||||
cfg.Config().set('window.x', x)
|
||||
cfg.Config().set('window.y', y)
|
||||
cfg.Config().set('window.width', width)
|
||||
cfg.Config().set('window.height', height)
|
||||
|
||||
# Style
|
||||
style = self.GetWindowStyle()
|
||||
cfg.Config().set('window.style', style)
|
||||
|
||||
cfg.Config().save()
|
||||
|
||||
# Stop monitoring
|
||||
self.cor.stop_monitor()
|
||||
|
||||
# End
|
||||
event.Skip()
|
||||
|
||||
def __on_open_file(self, evt):
|
||||
with wx.FileDialog(self, "Open Coefficients file", wildcard="cpd (*.cpd)|*.cpd",
|
||||
style=wx.FD_OPEN | wx.FD_FILE_MUST_EXIST) as fileDialog:
|
||||
if fileDialog.ShowModal() == wx.ID_CANCEL:
|
||||
return # the user changed their mind
|
||||
|
||||
# Load the file chosen by the user.
|
||||
self.__opened_filename = fileDialog.GetPath()
|
||||
|
||||
self.SetStatusText(f"Loading file {self.__opened_filename}.", 1)
|
||||
self.cor.load(self.__opened_filename)
|
||||
|
||||
# Show calculated data and refresh all opened frames
|
||||
self.__on_view_status(evt)
|
||||
self.__refresh()
|
||||
|
||||
self.SetStatusText(f"File {self.__opened_filename} loaded.", 1)
|
||||
|
||||
def __on_save_file(self, evt):
|
||||
self.SetStatusText(f"Saving file as {self.__opened_filename}", 1)
|
||||
|
||||
if self.__opened_filename is None:
|
||||
self.__on_save_file_as(evt)
|
||||
else:
|
||||
self.cor.save(self.__opened_filename)
|
||||
|
||||
self.SetStatusText(f"File saved as {self.__opened_filename}", 1)
|
||||
|
||||
def __on_save_file_as(self, evt):
|
||||
with wx.FileDialog(self, "Save Coefficients file", wildcard="cpd (*.cpd)|*.cpd",
|
||||
style=wx.FD_SAVE) as fileDialog:
|
||||
if fileDialog.ShowModal() == wx.ID_CANCEL:
|
||||
return # the user changed their mind
|
||||
|
||||
# Save the file and price data file, changing opened filename so next save writes to new file
|
||||
self.SetStatusText(f"Saving file as {self.__opened_filename}", 1)
|
||||
|
||||
self.__opened_filename = fileDialog.GetPath()
|
||||
self.cor.save(self.__opened_filename)
|
||||
|
||||
self.SetStatusText(f"File saved as {self.__opened_filename}", 1)
|
||||
|
||||
def __on_open_settings(self, evt):
|
||||
settings_dialog = cfg.SettingsDialog(parent=self, exclude=['window'])
|
||||
res = settings_dialog.ShowModal()
|
||||
if res == wx.ID_OK:
|
||||
# Stop the monitor
|
||||
self.cor.stop_monitor()
|
||||
|
||||
# Build calculation params and restart the monitor
|
||||
calculation_params = [cfg.Config().get('monitor.calculations.long'),
|
||||
cfg.Config().get('monitor.calculations.medium'),
|
||||
cfg.Config().get('monitor.calculations.short')]
|
||||
|
||||
self.cor.start_monitor(interval=cfg.Config().get('monitor.interval'),
|
||||
calculation_params=calculation_params,
|
||||
cache_time=cfg.Config().get('monitor.tick_cache_time'),
|
||||
autosave=cfg.Config().get('monitor.autosave'),
|
||||
filename=self.__opened_filename)
|
||||
|
||||
# Refresh all open child frames
|
||||
self.__refresh()
|
||||
|
||||
def __on_exit(self, evt):
|
||||
# Close
|
||||
self.Close()
|
||||
|
||||
def __on_calculate(self, evt):
|
||||
# set time zone to UTC to avoid local offset issues, and get from and to dates (a week ago to today)
|
||||
timezone = pytz.timezone("Etc/UTC")
|
||||
utc_to = datetime.now(tz=timezone)
|
||||
utc_from = utc_to - timedelta(days=cfg.Config().get('calculate.from.days'))
|
||||
|
||||
# Calculate
|
||||
self.SetStatusText("Calculating coefficients.", 1)
|
||||
self.cor.calculate(date_from=utc_from, date_to=utc_to,
|
||||
timeframe=cfg.Config().get('calculate.timeframe'),
|
||||
min_prices=cfg.Config().get('calculate.min_prices'),
|
||||
max_set_size_diff_pct=cfg.Config().get('calculate.max_set_size_diff_pct'),
|
||||
overlap_pct=cfg.Config().get('calculate.overlap_pct'),
|
||||
max_p_value=cfg.Config().get('calculate.max_p_value'))
|
||||
self.SetStatusText("", 1)
|
||||
|
||||
# Show calculated data and refresh frames
|
||||
self.__on_view_status(evt)
|
||||
self.__refresh()
|
||||
|
||||
def __on_monitor(self, evt):
|
||||
# Check state of toggle menu. If on, then start monitoring, else stop
|
||||
if self.__menu_item_monitor.IsChecked():
|
||||
self.__log.info("Starting monitoring for changes to coefficients.")
|
||||
self.SetStatusText("Monitoring", 0)
|
||||
self.__statusbar.SetBackgroundColour('green')
|
||||
self.__statusbar.Refresh()
|
||||
|
||||
self.timer.Start(cfg.Config().get('monitor.interval') * 1000)
|
||||
|
||||
# Autosave filename
|
||||
filename = self.__opened_filename if self.__opened_filename is not None else 'autosave.cpd'
|
||||
|
||||
# Build calculation params and start monitor
|
||||
calculation_params = [cfg.Config().get('monitor.calculations.long'),
|
||||
cfg.Config().get('monitor.calculations.medium'),
|
||||
cfg.Config().get('monitor.calculations.short')]
|
||||
|
||||
self.cor.start_monitor(interval=cfg.Config().get('monitor.interval'),
|
||||
calculation_params=calculation_params,
|
||||
cache_time=cfg.Config().get('monitor.tick_cache_time'),
|
||||
autosave=cfg.Config().get('monitor.autosave'),
|
||||
filename=filename)
|
||||
else:
|
||||
self.__log.info("Stopping monitoring.")
|
||||
self.SetStatusText("Not Monitoring", 0)
|
||||
self.__statusbar.SetBackgroundColour('lightgray')
|
||||
self.__statusbar.Refresh()
|
||||
self.timer.Stop()
|
||||
self.cor.stop_monitor()
|
||||
|
||||
def __on_clear(self, evt):
|
||||
# Clear the history
|
||||
self.cor.clear_coefficient_history()
|
||||
|
||||
# Refresh opened child frames
|
||||
self.__refresh()
|
||||
|
||||
def __on_timer(self, evt):
|
||||
# Refresh opened child frames
|
||||
self.__refresh()
|
||||
|
||||
# Set status message
|
||||
self.SetStatusText(f"Status updated at {self.cor.get_last_calculation():%d-%b %H:%M:%S}.", 1)
|
||||
|
||||
def __on_view_status(self, evt):
|
||||
FrameManager.open_frame(parent=self, frame_module='mt5_correlation.gui.mdi_child_status',
|
||||
frame_class='MDIChildStatus',
|
||||
raise_if_open=True)
|
||||
|
||||
def __on_view_diverged(self, evt):
|
||||
FrameManager.open_frame(parent=self, frame_module='mt5_correlation.gui.mdi_child_diverged_symbols',
|
||||
frame_class='MDIChildDivergedSymbols',
|
||||
raise_if_open=True)
|
||||
|
||||
def __on_view_log(self, evt):
|
||||
FrameManager.open_frame(parent=self, frame_module='mt5_correlation.gui.mdi_child_log',
|
||||
frame_class='MDIChildLog',
|
||||
raise_if_open=True)
|
||||
|
||||
def __on_view_help(self, evt):
|
||||
FrameManager.open_frame(parent=self, frame_module='mt5_correlation.gui.mdi_child_help',
|
||||
frame_class='MDIChildHelp',
|
||||
raise_if_open=True)
|
||||
|
||||
def __refresh(self):
|
||||
"""
|
||||
Refresh all open child frames
|
||||
:return:
|
||||
"""
|
||||
children = self.GetChildren()
|
||||
|
||||
for child in children:
|
||||
if isinstance(child, CorrelationMDIChild):
|
||||
child.refresh()
|
||||
elif isinstance(child, wx.StatusBar) or isinstance(child, ins.InspectionFrame) or \
|
||||
isinstance(child, wxconfig.SettingsDialog):
|
||||
# Ignore
|
||||
pass
|
||||
else:
|
||||
raise Exception(f"MDI Child for application must implement CorrelationMDIChild. MDI Child is "
|
||||
f"{type(child)}.")
|
||||
|
||||
|
||||
class CorrelationMDIChild(wx.MDIChildFrame):
|
||||
"""
|
||||
Interface for all MDI Children supported by the MDIParent
|
||||
"""
|
||||
|
||||
@abc.abstractmethod
|
||||
def refresh(self):
|
||||
"""
|
||||
Must be implemented. Refreshes the content. Called by MDIParents __refresh method
|
||||
:return:
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class FrameManager:
|
||||
"""
|
||||
Manages the opening and raising of MDIChild frames
|
||||
"""
|
||||
@staticmethod
|
||||
def open_frame(parent, frame_module, frame_class, raise_if_open=True, **kwargs):
|
||||
"""
|
||||
Opens the frame specified by the frame class
|
||||
:param parent: The MDIParentFrame to open the child frame into
|
||||
:param frame_module: A string specifying the module containing the frame class to open or raise
|
||||
:param frame_class: A string specifying the frame class to open or raise
|
||||
:param raise_if_open: Whether the frame should raise rather than open if an instance is already open.
|
||||
:param kwargs: A dict of parameters to pass to frame constructor. These will also be checked in raise_if_open
|
||||
to determine uniqueness (i.e. If a frame of the same class is already open but its params are different,
|
||||
then the frame will be opened again with the new params instead of being raised.)
|
||||
:return:
|
||||
"""
|
||||
|
||||
# Load the module and class
|
||||
module = importlib.import_module(frame_module)
|
||||
clazz = getattr(module, frame_class)
|
||||
|
||||
# Do we have an opened instance
|
||||
opened_instance = None
|
||||
for child in parent.GetChildren():
|
||||
if isinstance(child, clazz):
|
||||
# do the args match
|
||||
match = True
|
||||
for key in kwargs:
|
||||
if kwargs[key] != getattr(child, key):
|
||||
match = False
|
||||
|
||||
# Only open existing instance if args matched
|
||||
if match:
|
||||
opened_instance = child
|
||||
|
||||
# If we dont have an opened instance or raise_on_open is False then open new frame, otherwise raise it
|
||||
if opened_instance is None or raise_if_open is False:
|
||||
if len(kwargs) == 0:
|
||||
clazz(parent=parent).Show(True)
|
||||
else:
|
||||
clazz(parent=parent, **kwargs).Show(True)
|
||||
else:
|
||||
opened_instance.Raise()
|
||||
@@ -0,0 +1,215 @@
|
||||
import logging
|
||||
import matplotlib.dates
|
||||
import matplotlib.pyplot as plt
|
||||
import matplotlib.ticker as mticker
|
||||
import wx
|
||||
import wxconfig as cfg
|
||||
import wx.lib.scrolledpanel as scrolled
|
||||
|
||||
from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as FigureCanvas
|
||||
|
||||
import mt5_correlation.gui.mdi as mdi
|
||||
|
||||
|
||||
class MDIChildCorrelationGraph(mdi.CorrelationMDIChild):
|
||||
"""
|
||||
Shows the graphs for the specified correlation
|
||||
"""
|
||||
|
||||
symbols = None # Symbols for correlation. Public as we use to check if window for the symbol pair is already open.
|
||||
|
||||
# Date formats for graphs
|
||||
__tick_fmt_date = matplotlib.dates.DateFormatter('%d-%b')
|
||||
__tick_fmt_time = matplotlib.dates.DateFormatter('%H:%M:%S')
|
||||
|
||||
# Colors for graph lines for symbol1 and symbol2. Will use first 2 colours in colormap
|
||||
__colours = matplotlib.cm.get_cmap(cfg.Config().get("charts.colormap")).colors
|
||||
|
||||
# Fig, axes and canvas
|
||||
__fig = None
|
||||
__axs = None
|
||||
__canvas = None
|
||||
|
||||
def __init__(self, parent, **kwargs):
|
||||
# Super
|
||||
wx.MDIChildFrame.__init__(self, parent=parent, id=wx.ID_ANY,
|
||||
title=f"Correlation Status for {kwargs['symbols'][0]}:{kwargs['symbols'][1]}")
|
||||
|
||||
# Create logger
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Store the symbols
|
||||
self.symbols = kwargs['symbols']
|
||||
|
||||
# We will freeze this frame and thaw once constructed to avoid flicker.
|
||||
self.Freeze()
|
||||
|
||||
# Draw the empty graphs. We will populate with data in refresh. We will have 3 charts:
|
||||
# 1) Data used to calculate base coefficient for both symbols (2 lines on chart);
|
||||
# 2) Data used to calculate latest coefficient for both symbols (2 lines on chart); and
|
||||
# 3) Coefficient history and the divergence threshold lines
|
||||
|
||||
# Create fig and 3 axes.
|
||||
self.__fig, self.__axs = plt.subplots(3)
|
||||
|
||||
# Create additional axis for second line on charts 1 & 2
|
||||
self.__s2axs = [self.__axs[0].twinx(), self.__axs[1].twinx()]
|
||||
|
||||
# Set titles
|
||||
self.__axs[0].set_title(f"Base Coefficient Price Data for {self.symbols[0]}:{self.symbols[1]}")
|
||||
self.__axs[1].set_title(f"Coefficient Tick Data for {self.symbols[0]}:{self.symbols[1]}")
|
||||
self.__axs[2].set_title(f"Coefficient History for {self.symbols[0]}:{self.symbols[1]}")
|
||||
|
||||
# Set Y Labels and tick colours for charts 1 & 2. Left for symbol1, right for symbol2
|
||||
for i in range(0, 2):
|
||||
self.__axs[i].set_ylabel(f"{self.symbols[0]}", color=self.__colours[0], labelpad=10)
|
||||
self.__axs[i].tick_params(axis='y', labelcolor=self.__colours[0])
|
||||
self.__s2axs[i].set_ylabel(f"{self.symbols[1]}", color=self.__colours[1], labelpad=10)
|
||||
self.__s2axs[i].tick_params(axis='y', labelcolor=self.__colours[1])
|
||||
|
||||
# Set Y label and limits for 3rd chart. Limits will be coefficients range from -1 to 1
|
||||
self.__axs[2].set_ylabel('Coefficient', labelpad=10)
|
||||
self.__axs[2].set_ylim([-1, 1])
|
||||
|
||||
# Set X labels to ''. Workaround as matplotlib is not leaving space for ticks
|
||||
for ax in self.__axs:
|
||||
ax.set_xlabel(" ", labelpad=10)
|
||||
|
||||
# Layout with padding between charts
|
||||
self.__fig.tight_layout(pad=0.5)
|
||||
|
||||
# Create panel and sizer. This will provide scrollbar
|
||||
panel = scrolled.ScrolledPanel(self, wx.ID_ANY)
|
||||
sizer = wx.BoxSizer()
|
||||
panel.SetSizer(sizer)
|
||||
|
||||
# Add fig to canvas and canvas to sizer. Thaw window to update
|
||||
self.__canvas = FigureCanvas(panel, wx.ID_ANY, self.__fig)
|
||||
sizer.Add(self.__canvas, 1, wx.ALL | wx.EXPAND)
|
||||
self.Thaw()
|
||||
|
||||
# Setup scrolling
|
||||
panel.SetupScrolling()
|
||||
|
||||
# Refresh to show content
|
||||
self.refresh()
|
||||
|
||||
def refresh(self):
|
||||
"""
|
||||
Refresh the graph
|
||||
:return:
|
||||
"""
|
||||
# Get the price data for the base coefficient calculation, tick data that was used to calculate last
|
||||
# coefficient and and the coefficient history data
|
||||
price_data = [self.GetMDIParent().cor.get_price_data(self.symbols[0]),
|
||||
self.GetMDIParent().cor.get_price_data(self.symbols[1])]
|
||||
|
||||
tick_data = [self.GetMDIParent().cor.get_ticks(self.symbols[0], cache_only=True),
|
||||
self.GetMDIParent().cor.get_ticks(self.symbols[1], cache_only=True)]
|
||||
|
||||
history_data = []
|
||||
for timeframe in cfg.Config().get('monitor.calculations'):
|
||||
frm = cfg.Config().get(f'monitor.calculations.{timeframe}.from')
|
||||
history_data.append(self.GetMDIParent().cor.get_coefficient_history(
|
||||
{'Symbol 1': self.symbols[0], 'Symbol 2': self.symbols[1], 'Timeframe': frm}))
|
||||
|
||||
# Check what data we have available
|
||||
price_data_available = price_data is not None and len(price_data) == 2 and price_data[0] is not None and \
|
||||
price_data[1] is not None and len(price_data[0]) > 0 and len(price_data[1]) > 0
|
||||
|
||||
tick_data_available = tick_data is not None and len(tick_data) == 2 and tick_data[0] is not None and \
|
||||
tick_data[1] is not None and len(tick_data[0]) > 0 and len(tick_data[1]) > 0
|
||||
|
||||
history_data_available = history_data is not None and len(history_data) > 0
|
||||
|
||||
# Get all plots for coefficient history. History can contain multiple plots for different timeframes. They
|
||||
# will all be plotted on the same chart.
|
||||
times = []
|
||||
coefficients = []
|
||||
if history_data_available:
|
||||
for hist in history_data:
|
||||
times.append(hist['Date To'])
|
||||
coefficients.append(hist['Coefficient'])
|
||||
|
||||
# Update graphs where we have data available
|
||||
if price_data_available:
|
||||
# Update range and ticks
|
||||
xrange = [min(min(price_data[0]['time']), min(price_data[1]['time'])),
|
||||
max(max(price_data[0]['time']), max(price_data[1]['time']))]
|
||||
self.__axs[0].set_xlim(xrange)
|
||||
|
||||
# Plot both lines
|
||||
self.__axs[0].plot(price_data[0]['time'], price_data[0]['close'],
|
||||
color=self.__colours[0])
|
||||
self.__s2axs[0].plot(price_data[1]['time'], price_data[1]['close'],
|
||||
color=self.__colours[1])
|
||||
|
||||
# Ticks, labels and formats. Fixing xticks with FixedLocator but also using MaxNLocator to avoid
|
||||
# cramped x-labels
|
||||
if len(price_data[0]['time']) > 0:
|
||||
self.__axs[0].xaxis.set_major_locator(mticker.MaxNLocator(10))
|
||||
ticks_loc = self.__axs[0].get_xticks().tolist()
|
||||
self.__axs[0].xaxis.set_major_locator(mticker.FixedLocator(ticks_loc))
|
||||
self.__axs[0].set_xticklabels(ticks_loc)
|
||||
self.__axs[0].xaxis.set_major_formatter(self.__tick_fmt_date)
|
||||
plt.setp(self.__axs[0].xaxis.get_majorticklabels(), rotation=45)
|
||||
|
||||
if tick_data_available:
|
||||
# Update range and ticks
|
||||
xrange = [min(min(tick_data[0]['time']), min(tick_data[1]['time'])),
|
||||
max(max(tick_data[0]['time']), max(tick_data[1]['time']))]
|
||||
self.__axs[1].set_xlim(xrange)
|
||||
|
||||
# Plot both lines
|
||||
self.__axs[1].plot(tick_data[0]['time'], tick_data[0]['ask'],
|
||||
color=self.__colours[0])
|
||||
self.__s2axs[1].plot(tick_data[1]['time'], tick_data[1]['ask'],
|
||||
color=self.__colours[1])
|
||||
|
||||
if len(tick_data[0]['time']) > 0:
|
||||
self.__axs[1].xaxis.set_major_locator(mticker.MaxNLocator(10))
|
||||
ticks_loc = self.__axs[1].get_xticks().tolist()
|
||||
self.__axs[1].xaxis.set_major_locator(mticker.FixedLocator(ticks_loc))
|
||||
self.__axs[1].set_xticklabels(ticks_loc)
|
||||
self.__axs[1].xaxis.set_major_formatter(self.__tick_fmt_time)
|
||||
plt.setp(self.__axs[1].xaxis.get_majorticklabels(), rotation=45)
|
||||
|
||||
if history_data_available:
|
||||
# Plot. There may be more than one set of data for chart. One for each coefficient date range. Convert
|
||||
# single data to list, then loop to plot
|
||||
xdata = times if isinstance(times, list) else [times, ]
|
||||
ydata = coefficients if isinstance(coefficients, list) else [coefficients, ]
|
||||
|
||||
for i in range(0, len(xdata)):
|
||||
self.__axs[2].scatter(xdata[i], ydata[i], s=1)
|
||||
|
||||
# Ticks, labels and formats. Fixing xticks with FixedLocator but also using MaxNLocator to avoid
|
||||
# cramped x-labels
|
||||
if len(times[0].array) > 0:
|
||||
self.__axs[2].xaxis.set_major_locator(mticker.MaxNLocator(10))
|
||||
ticks_loc = self.__axs[2].get_xticks().tolist()
|
||||
self.__axs[2].xaxis.set_major_locator(mticker.FixedLocator(ticks_loc))
|
||||
self.__axs[2].set_xticklabels(ticks_loc)
|
||||
self.__axs[2].xaxis.set_major_formatter(self.__tick_fmt_time)
|
||||
plt.setp(self.__axs[2].xaxis.get_majorticklabels(), rotation=45)
|
||||
|
||||
# Legend
|
||||
self.__axs[2].legend([f"{cfg.Config().get('monitor.calculations.long.from')} Minutes",
|
||||
f"{cfg.Config().get('monitor.calculations.medium.from')} Minutes",
|
||||
f"{cfg.Config().get('monitor.calculations.short.from')} Minutes"])
|
||||
|
||||
# Lines showing divergence threshold. 2 if we are monitoring inverse correlations.
|
||||
divergence_threshold = self.GetMDIParent().cor.divergence_threshold
|
||||
monitor_inverse = self.GetMDIParent().cor.monitor_inverse
|
||||
|
||||
if divergence_threshold is not None:
|
||||
self.__axs[2].axhline(y=divergence_threshold, color="red", label='_nolegend_', linewidth=1)
|
||||
if monitor_inverse:
|
||||
self.__axs[2].axhline(y=divergence_threshold * -1, color="red", label='_nolegend_', linewidth=1)
|
||||
|
||||
# Redraw canvas
|
||||
self.__canvas.draw()
|
||||
|
||||
def __del__(self):
|
||||
# Close all plots
|
||||
plt.close('all')
|
||||
@@ -0,0 +1,156 @@
|
||||
import logging
|
||||
import pandas as pd
|
||||
import wx
|
||||
import wx.grid
|
||||
|
||||
import mt5_correlation.gui.mdi as mdi
|
||||
|
||||
# Columns for diverged symbols table
|
||||
COLUMN_INDEX = 0
|
||||
COLUMN_SYMBOL = 1
|
||||
COLUMN_NUM_DIVERGENCES = 2
|
||||
|
||||
|
||||
class MDIChildDivergedSymbols(mdi.CorrelationMDIChild):
|
||||
"""
|
||||
Shows the status of all correlations that are within the monitoring threshold
|
||||
"""
|
||||
|
||||
# The table and grid containing the symbols that form part of the diverged correlations. Defined at instance level
|
||||
# to enable refresh.
|
||||
__table = None
|
||||
__grid = None
|
||||
|
||||
# Number of rows. Required for and updated by refresh method
|
||||
__rows = 0
|
||||
|
||||
__log = None # The logger
|
||||
|
||||
def __init__(self, parent):
|
||||
# Super
|
||||
wx.MDIChildFrame.__init__(self, parent=parent, id=wx.ID_ANY, title="Diverged Symbols",
|
||||
size=wx.Size(width=240, height=-1), style=wx.DEFAULT_FRAME_STYLE)
|
||||
|
||||
# Create logger
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Panel and sizer for table
|
||||
panel = wx.Panel(self, wx.ID_ANY)
|
||||
sizer = wx.BoxSizer(wx.VERTICAL)
|
||||
panel.SetSizer(sizer)
|
||||
|
||||
# Create the grid. This is a data table using pandas dataframe for underlying data. Add the
|
||||
# grid to the sizer.
|
||||
self.__table = _DataTable(columns=self.GetMDIParent().cor.diverged_symbols.columns)
|
||||
self.__grid = wx.grid.Grid(panel, wx.ID_ANY)
|
||||
self.__grid.SetTable(self.__table, takeOwnership=True)
|
||||
self.__grid.EnableEditing(False)
|
||||
self.__grid.EnableDragRowSize(False)
|
||||
self.__grid.EnableDragColSize(True)
|
||||
self.__grid.EnableDragGridSize(True)
|
||||
self.__grid.SetSelectionMode(wx.grid.Grid.SelectRows)
|
||||
self.__grid.SetRowLabelSize(0)
|
||||
self.__grid.SetColSize(COLUMN_INDEX, 0) # Index. Hide
|
||||
self.__grid.SetColSize(COLUMN_SYMBOL, 100) # Symbol
|
||||
self.__grid.SetColSize(COLUMN_NUM_DIVERGENCES, 100) # Num divergences
|
||||
self.__grid.SetMinSize((220, 100))
|
||||
sizer.Add(self.__grid, 1, wx.ALL | wx.EXPAND)
|
||||
|
||||
# Bind row doubleclick
|
||||
self.Bind(wx.grid.EVT_GRID_CELL_LEFT_DCLICK, self.__on_doubleckick_row, self.__grid)
|
||||
|
||||
# Refresh to populate
|
||||
self.refresh()
|
||||
|
||||
def refresh(self):
|
||||
"""
|
||||
Refreshes grid. Notifies if rows have been added or deleted.
|
||||
:return:
|
||||
"""
|
||||
self.__log.debug(f"Refreshing grid.")
|
||||
|
||||
# Update data
|
||||
self.__table.data = self.GetMDIParent().cor.diverged_symbols.copy()
|
||||
|
||||
# Start refresh
|
||||
self.__grid.BeginBatch()
|
||||
|
||||
# Check if num rows in dataframe has changed, and send appropriate APPEND or DELETE messages
|
||||
cur_rows = len(self.GetMDIParent().cor.diverged_symbols.index)
|
||||
if cur_rows < self.__rows:
|
||||
# Data has been deleted. Send message
|
||||
msg = wx.grid.GridTableMessage(self.__table, wx.grid.GRIDTABLE_NOTIFY_ROWS_DELETED,
|
||||
self.__rows - cur_rows, self.__rows - cur_rows)
|
||||
self.__grid.ProcessTableMessage(msg)
|
||||
elif cur_rows > self.__rows:
|
||||
# Data has been added. Send message
|
||||
msg = wx.grid.GridTableMessage(self.__table, wx.grid.GRIDTABLE_NOTIFY_ROWS_APPENDED,
|
||||
cur_rows - self.__rows) # how many
|
||||
self.__grid.ProcessTableMessage(msg)
|
||||
|
||||
self.__grid.EndBatch()
|
||||
|
||||
# Send updated message
|
||||
msg = wx.grid.GridTableMessage(self.__table, wx.grid.GRIDTABLE_REQUEST_VIEW_GET_VALUES)
|
||||
self.__grid.ProcessTableMessage(msg)
|
||||
|
||||
# Update row count
|
||||
self.__rows = cur_rows
|
||||
|
||||
def __on_doubleckick_row(self, evt):
|
||||
"""
|
||||
Open the graphs when a row is doubleclicked.
|
||||
:param evt:
|
||||
:return:
|
||||
"""
|
||||
row = evt.GetRow()
|
||||
symbol = self.__grid.GetCellValue(row, COLUMN_SYMBOL)
|
||||
|
||||
mdi.FrameManager.open_frame(parent=self.GetMDIParent(),
|
||||
frame_module='mt5_correlation.gui.mdi_child_divergedgraph',
|
||||
frame_class='MDIChildDivergedGraph',
|
||||
raise_if_open=True,
|
||||
symbol=symbol)
|
||||
|
||||
|
||||
class _DataTable(wx.grid.GridTableBase):
|
||||
"""
|
||||
A data table that holds data in a pandas dataframe.
|
||||
"""
|
||||
data = None # The data for this table. A Pandas DataFrame
|
||||
|
||||
def __init__(self, columns):
|
||||
wx.grid.GridTableBase.__init__(self)
|
||||
self.headerRows = 1
|
||||
self.data = pd.DataFrame(columns=columns)
|
||||
|
||||
def GetNumberRows(self):
|
||||
return len(self.data)
|
||||
|
||||
def GetNumberCols(self):
|
||||
return len(self.data.columns) + 1
|
||||
|
||||
def GetValue(self, row, col):
|
||||
if row < self.RowsCount and col < self.ColsCount:
|
||||
return self.data.index[row] if col == 0 else self.data.iloc[row, col - 1]
|
||||
else:
|
||||
raise Exception(f"Trying to access row {row} and col {col} which does not exist.")
|
||||
|
||||
def SetValue(self, row, col, value):
|
||||
self.data.iloc[row, col - 1] = value
|
||||
|
||||
def GetColLabelValue(self, col):
|
||||
if col == 0:
|
||||
if self.data.index.name is None:
|
||||
return 'Index'
|
||||
else:
|
||||
return self.data.index.name
|
||||
return str(self.data.columns[col - 1])
|
||||
|
||||
def GetTypeName(self, row, col):
|
||||
return wx.grid.GRID_VALUE_STRING
|
||||
|
||||
def GetAttr(self, row, col, prop):
|
||||
attr = wx.grid.GridCellAttr()
|
||||
|
||||
return attr
|
||||
@@ -0,0 +1,194 @@
|
||||
import logging
|
||||
import matplotlib.dates
|
||||
import matplotlib.pyplot as plt
|
||||
import matplotlib.ticker as mticker
|
||||
import wx
|
||||
import wx.lib.scrolledpanel as scrolled
|
||||
import wxconfig as cfg
|
||||
|
||||
from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as FigureCanvas
|
||||
from mpl_toolkits.axes_grid1 import host_subplot
|
||||
from mpl_toolkits import axisartist
|
||||
|
||||
import mt5_correlation.gui.mdi as mdi
|
||||
from mt5_correlation import correlation as cor
|
||||
|
||||
|
||||
class MDIChildDivergedGraph(mdi.CorrelationMDIChild):
|
||||
"""
|
||||
Shows the graphs for the specified correlation
|
||||
"""
|
||||
|
||||
symbol = None # Symbol to chart divergence for. Public as we use to check if window for the symbol is already open.
|
||||
|
||||
# Logger
|
||||
__log = None
|
||||
|
||||
# Date formats for graphs
|
||||
__tick_fmt_date = matplotlib.dates.DateFormatter('%d-%b')
|
||||
__tick_fmt_time = matplotlib.dates.DateFormatter('%H:%M:%S')
|
||||
|
||||
# Graph Canvas
|
||||
__canvas = None
|
||||
|
||||
# Colors for graph lines
|
||||
__colours = matplotlib.cm.get_cmap(cfg.Config().get("charts.colormap")).colors
|
||||
|
||||
# We will store the other symbols last plotted. This will save us rebuilding teh figure if teh symbols haven't
|
||||
# changed.
|
||||
__other_symbols = None
|
||||
|
||||
def __init__(self, parent, **kwargs):
|
||||
# Super
|
||||
wx.MDIChildFrame.__init__(self, parent=parent, id=wx.ID_ANY,
|
||||
title=f"Divergence Graph for {kwargs['symbol']}")
|
||||
|
||||
# Create logger
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Store the symbol
|
||||
self.symbol = kwargs['symbol']
|
||||
|
||||
# Create panel and sizer
|
||||
panel = scrolled.ScrolledPanel(self, wx.ID_ANY)
|
||||
sizer = wx.BoxSizer()
|
||||
panel.SetSizer(sizer)
|
||||
|
||||
# Create figure and canvas. Add canvas to sizer
|
||||
self.__canvas = FigureCanvas(panel, wx.ID_ANY, plt.figure())
|
||||
panel.GetSizer().Add(self.__canvas, 1, wx.ALL | wx.EXPAND)
|
||||
panel.SetupScrolling()
|
||||
|
||||
# Refresh to show content
|
||||
self.refresh()
|
||||
|
||||
def refresh(self):
|
||||
"""
|
||||
Refresh the graph
|
||||
:return:
|
||||
"""
|
||||
|
||||
# Get tick data for base symbol
|
||||
symbol_tick_data = self.GetMDIParent().cor.get_ticks(self.symbol, cache_only=True)
|
||||
|
||||
# Get the other symbols and their tick data
|
||||
other_symbols_data = self.__get_other_symbols_data()
|
||||
|
||||
# Delete all axes from the figure. They will need to be recreated as the symbols may be different
|
||||
for axes in self.__canvas.figure.axes:
|
||||
axes.remove()
|
||||
|
||||
# Plot for all other symbols
|
||||
num_subplots = 1 if len(other_symbols_data.keys()) == 0 else len(other_symbols_data.keys())
|
||||
plotnum = 1
|
||||
axs = [] # Store the axes for sharing x axis
|
||||
for other_symbol in other_symbols_data:
|
||||
axs.append(self.__canvas.figure.add_subplot(num_subplots, 1, plotnum))
|
||||
self.__plot(axes=axs[-1], base_symbol=self.symbol, other_symbol=other_symbol,
|
||||
base_symbol_data=symbol_tick_data, other_symbol_data=other_symbols_data[other_symbol])
|
||||
|
||||
# Next plot
|
||||
plotnum += 1
|
||||
|
||||
# Share x axis of the last axes with all the others
|
||||
self.__share_xaxis(axs)
|
||||
|
||||
# Redraw canvas
|
||||
self.__canvas.figure.tight_layout(pad=0.5)
|
||||
self.__canvas.draw()
|
||||
|
||||
def __get_other_symbols_data(self):
|
||||
"""
|
||||
Gets the data required for the graphs
|
||||
:param self:
|
||||
:return: dict of other symbols their tick data
|
||||
"""
|
||||
# Get the symbols that this one has diverged against.
|
||||
data = self.GetMDIParent().cor.filtered_coefficient_data
|
||||
filtered_data = data.loc[(
|
||||
(data['Status'] == cor.STATUS_DIVERGED) |
|
||||
(data['Status'] == cor.STATUS_DIVERGING) |
|
||||
(data['Status'] == cor.STATUS_CONVERGING)
|
||||
) &
|
||||
(
|
||||
(data['Symbol 1'] == self.symbol) |
|
||||
(data['Symbol 2'] == self.symbol)
|
||||
)]
|
||||
|
||||
# Get all symbols and remove the base one. We will need to ensure that this is first in the list
|
||||
other_symbols = list(filtered_data['Symbol 1'].append(filtered_data['Symbol 2']).drop_duplicates())
|
||||
if self.symbol in other_symbols:
|
||||
other_symbols.remove(self.symbol)
|
||||
|
||||
# Get the tick data other symbols and add to dict
|
||||
other_tick_data = {}
|
||||
for symbol in other_symbols:
|
||||
tick_data = self.GetMDIParent().cor.get_ticks(symbol, cache_only=True)
|
||||
other_tick_data[symbol] = tick_data
|
||||
|
||||
return other_tick_data
|
||||
|
||||
def __plot(self, axes, base_symbol, other_symbol, base_symbol_data, other_symbol_data):
|
||||
"""
|
||||
Plots the data on the axes
|
||||
:param axes: The subplot to plot onto
|
||||
:param base_symbol
|
||||
:param other_symbol:
|
||||
"""
|
||||
# Create the other axes. Will need an axes for the base symbol data and another for the other symbol data
|
||||
other_axes = axes.twinx()
|
||||
|
||||
# Set plot title and axis labels
|
||||
axes.set_title(f"Tick Data for {base_symbol}:{other_symbol}")
|
||||
axes.set_ylabel(base_symbol, color=self.__colours[0], labelpad=10)
|
||||
other_axes.set_ylabel(other_symbol, color=self.__colours[1], labelpad=10)
|
||||
|
||||
# Set the tick and axis colors
|
||||
self.__set_axes_color(axes, self.__colours[0], 'left')
|
||||
self.__set_axes_color(other_axes, self.__colours[1], 'right')
|
||||
|
||||
# Plot both lines
|
||||
axes.plot(base_symbol_data['time'], base_symbol_data['ask'], color=self.__colours[0])
|
||||
other_axes.plot(other_symbol_data['time'], other_symbol_data['ask'], color=self.__colours[1])
|
||||
|
||||
@staticmethod
|
||||
def __set_axes_color(axes, color, axis_loc='right'):
|
||||
"""
|
||||
Set the color for the axes, including axis line, ticks and tick labels
|
||||
:param axes: The axes to set color for.
|
||||
:param color: The color to set to
|
||||
:param axis_loc: The location of the axis, label and ticks. Either left for base symbol or right for others
|
||||
:return:
|
||||
"""
|
||||
# Change the color for the axis line, ticks and tick labels
|
||||
axes.spines[axis_loc].set_color(color)
|
||||
axes.tick_params(axis='y', colors=color)
|
||||
|
||||
def __share_xaxis(self, axs):
|
||||
"""
|
||||
Share the xaxis of the last axes with all other axes. Remove axis tick labels for all but the last. Format axis
|
||||
tick labels for the last.
|
||||
:param axs:
|
||||
:return:
|
||||
"""
|
||||
if len(axs) > 0:
|
||||
last_ax = axs[-1]
|
||||
for ax in axs:
|
||||
# If we are not on the last one, share and hide tick labels. If we are on the last one, format tick
|
||||
# labels.
|
||||
if ax != last_ax:
|
||||
ax.sharex(last_ax)
|
||||
plt.setp(ax.xaxis.get_majorticklabels(), visible=False)
|
||||
else:
|
||||
# Ticks, labels and formats. Fixing xticks with FixedLocator but also using MaxNLocator to avoid
|
||||
# cramped x-labels
|
||||
ax.xaxis.set_major_locator(mticker.MaxNLocator(10))
|
||||
ticks_loc = ax.get_xticks().tolist()
|
||||
ax.xaxis.set_major_locator(mticker.FixedLocator(ticks_loc))
|
||||
ax.set_xticklabels(ticks_loc)
|
||||
ax.xaxis.set_major_formatter(self.__tick_fmt_time)
|
||||
plt.setp(ax.xaxis.get_majorticklabels(), rotation=45)
|
||||
|
||||
def __del__(self):
|
||||
# Close all plots
|
||||
plt.close('all')
|
||||
@@ -0,0 +1,41 @@
|
||||
import definitions
|
||||
import markdown
|
||||
import wx
|
||||
import wx.html
|
||||
|
||||
import mt5_correlation.gui.mdi as mdi
|
||||
|
||||
|
||||
class MDIChildHelp(mdi.CorrelationMDIChild):
|
||||
"""
|
||||
Shows the README.md file
|
||||
"""
|
||||
|
||||
def __init__(self, parent):
|
||||
# Super
|
||||
mdi.CorrelationMDIChild.__init__(self, parent=parent, id=wx.ID_ANY, pos=wx.DefaultPosition, title="Help",
|
||||
size=wx.Size(width=800, height=-1),
|
||||
style=wx.DEFAULT_FRAME_STYLE)
|
||||
|
||||
# Panel and sizer for help file
|
||||
panel = wx.Panel(self, wx.ID_ANY)
|
||||
sizer = wx.BoxSizer()
|
||||
panel.SetSizer(sizer)
|
||||
|
||||
# HtmlWindow
|
||||
html_widget = wx.html.HtmlWindow(parent=panel, id=wx.ID_ANY, style=wx.html.HW_SCROLLBAR_AUTO | wx.html.HW_NO_SELECTION)
|
||||
sizer.Add(html_widget, 1, wx.ALL | wx.EXPAND)
|
||||
|
||||
# Load the help file, convert markdown to HTML and display. The markdown library doesnt understand the shell
|
||||
# format so we will remove.
|
||||
markdown_text = open(definitions.HELP_FILE).read()
|
||||
markdown_text = markdown_text.replace("```shell", "```")
|
||||
html = markdown.markdown(markdown_text)
|
||||
html_widget.SetPage(html)
|
||||
|
||||
def refresh(self):
|
||||
"""
|
||||
Nothing to do on refresh. Help file doesnt change during outside of development.
|
||||
:return:
|
||||
"""
|
||||
pass
|
||||
@@ -0,0 +1,43 @@
|
||||
import definitions
|
||||
import wx
|
||||
import wx.html
|
||||
|
||||
import mt5_correlation.gui.mdi as mdi
|
||||
|
||||
|
||||
class MDIChildLog(mdi.CorrelationMDIChild):
|
||||
"""
|
||||
Shows the debug.log file
|
||||
"""
|
||||
|
||||
__log_window = None # Widget to display log file in
|
||||
|
||||
def __init__(self, parent):
|
||||
# Super
|
||||
mdi.CorrelationMDIChild.__init__(self, parent=parent, id=wx.ID_ANY, pos=wx.DefaultPosition, title="Log",
|
||||
size=wx.Size(width=800, height=200),
|
||||
style=wx.DEFAULT_FRAME_STYLE)
|
||||
|
||||
# Panel and sizer for help file
|
||||
panel = wx.Panel(self, wx.ID_ANY)
|
||||
sizer = wx.BoxSizer()
|
||||
panel.SetSizer(sizer)
|
||||
|
||||
# Log file window
|
||||
self.__log_window = wx.TextCtrl(parent=panel, id=wx.ID_ANY, style=wx.HSCROLL | wx.TE_MULTILINE | wx.TE_READONLY)
|
||||
sizer.Add(self.__log_window, 1, wx.ALL | wx.EXPAND)
|
||||
|
||||
# Refresh to populate
|
||||
self.refresh()
|
||||
|
||||
def refresh(self):
|
||||
"""
|
||||
Refresh the log file
|
||||
:return:
|
||||
"""
|
||||
# Load the help file
|
||||
self.__log_window.LoadFile(definitions.LOG_FILE)
|
||||
|
||||
# Scroll to bottom
|
||||
self.__log_window.SetInsertionPoint(-1)
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
import logging
|
||||
import pandas as pd
|
||||
import wx
|
||||
import wx.grid
|
||||
|
||||
from mt5_correlation import correlation as cor
|
||||
import mt5_correlation.gui.mdi as mdi
|
||||
|
||||
# Columns for coefficient table
|
||||
COLUMN_INDEX = 0
|
||||
COLUMN_SYMBOL1 = 1
|
||||
COLUMN_SYMBOL2 = 2
|
||||
COLUMN_BASE_COEFFICIENT = 3
|
||||
COLUMN_DATE_FROM = 4
|
||||
COLUMN_DATE_TO = 5
|
||||
COLUMN_TIMEFRAME = 6
|
||||
COLUMN_LAST_CALCULATION = 7
|
||||
COLUMN_STATUS = 8
|
||||
|
||||
|
||||
class MDIChildStatus(mdi.CorrelationMDIChild):
|
||||
"""
|
||||
Shows the status of all correlations that are within the monitoring threshold
|
||||
"""
|
||||
|
||||
# The table and grid containing the status of correlations. Defined at instance level to enable refresh.
|
||||
__table = None
|
||||
__grid = None
|
||||
|
||||
# Number of rows. Required for and updated by refresh method
|
||||
__rows = 0
|
||||
|
||||
__log = None # The logger
|
||||
|
||||
def __init__(self, parent):
|
||||
# Super
|
||||
wx.MDIChildFrame.__init__(self, parent=parent, id=wx.ID_ANY, title="Correlation Status",
|
||||
size=wx.Size(width=440, height=-1), style=wx.DEFAULT_FRAME_STYLE)
|
||||
|
||||
# Create logger
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Panel and sizer for table
|
||||
panel = wx.Panel(self, wx.ID_ANY)
|
||||
sizer = wx.BoxSizer(wx.VERTICAL)
|
||||
panel.SetSizer(sizer)
|
||||
|
||||
# Create the correlations grid. This is a data table using pandas dataframe for underlying data. Add the
|
||||
# correlations_grid to the correlations sizer.
|
||||
self.__table = _DataTable(columns=self.GetMDIParent().cor.filtered_coefficient_data.columns)
|
||||
self.__grid = wx.grid.Grid(panel, wx.ID_ANY)
|
||||
self.__grid.SetTable(self.__table, takeOwnership=True)
|
||||
self.__grid.EnableEditing(False)
|
||||
self.__grid.EnableDragRowSize(False)
|
||||
self.__grid.EnableDragColSize(True)
|
||||
self.__grid.EnableDragGridSize(True)
|
||||
self.__grid.SetSelectionMode(wx.grid.Grid.SelectRows)
|
||||
self.__grid.SetRowLabelSize(0)
|
||||
self.__grid.SetColSize(COLUMN_INDEX, 0) # Index. Hide
|
||||
self.__grid.SetColSize(COLUMN_SYMBOL1, 100) # Symbol 1
|
||||
self.__grid.SetColSize(COLUMN_SYMBOL2, 100) # Symbol 2
|
||||
self.__grid.SetColSize(COLUMN_BASE_COEFFICIENT, 100) # Base Coefficient
|
||||
self.__grid.SetColSize(COLUMN_DATE_FROM, 0) # UTC Date From. Hide
|
||||
self.__grid.SetColSize(COLUMN_DATE_TO, 0) # UTC Date To. Hide
|
||||
self.__grid.SetColSize(COLUMN_TIMEFRAME, 0) # Timeframe. Hide.
|
||||
self.__grid.SetColSize(COLUMN_LAST_CALCULATION, 0) # Last Calculation. Hide
|
||||
self.__grid.SetColSize(COLUMN_STATUS, 100) # Status
|
||||
self.__grid.SetMinSize((420, 500))
|
||||
sizer.Add(self.__grid, 1, wx.ALL | wx.EXPAND)
|
||||
|
||||
# Bind row doubleclick
|
||||
self.Bind(wx.grid.EVT_GRID_CELL_LEFT_DCLICK, self.__on_doubleckick_row, self.__grid)
|
||||
|
||||
# Refresh to populate
|
||||
self.refresh()
|
||||
|
||||
def refresh(self):
|
||||
"""
|
||||
Refreshes grid. Notifies if rows have been added or deleted.
|
||||
:return:
|
||||
"""
|
||||
self.__log.debug(f"Refreshing grid.")
|
||||
|
||||
# Update data
|
||||
self.__table.data = self.GetMDIParent().cor.filtered_coefficient_data.copy()
|
||||
|
||||
# Format
|
||||
self.__table.data.loc[:, 'Base Coefficient'] = self.__table.data['Base Coefficient'].map('{:.5f}'.format)
|
||||
self.__table.data.loc[:, 'Last Calculation'] = pd.to_datetime(self.__table.data['Last Calculation'], utc=True)
|
||||
self.__table.data.loc[:, 'Last Calculation'] = \
|
||||
self.__table.data['Last Calculation'].dt.strftime('%d-%m-%y %H:%M:%S')
|
||||
|
||||
# Start refresh
|
||||
self.__grid.BeginBatch()
|
||||
|
||||
# Check if num rows in dataframe has changed, and send appropriate APPEND or DELETE messages
|
||||
cur_rows = len(self.GetMDIParent().cor.filtered_coefficient_data.index)
|
||||
if cur_rows < self.__rows:
|
||||
# Data has been deleted. Send message
|
||||
msg = wx.grid.GridTableMessage(self.__table, wx.grid.GRIDTABLE_NOTIFY_ROWS_DELETED,
|
||||
self.__rows - cur_rows, self.__rows - cur_rows)
|
||||
self.__grid.ProcessTableMessage(msg)
|
||||
elif cur_rows > self.__rows:
|
||||
# Data has been added. Send message
|
||||
msg = wx.grid.GridTableMessage(self.__table, wx.grid.GRIDTABLE_NOTIFY_ROWS_APPENDED,
|
||||
cur_rows - self.__rows) # how many
|
||||
self.__grid.ProcessTableMessage(msg)
|
||||
|
||||
self.__grid.EndBatch()
|
||||
|
||||
# Send updated message
|
||||
msg = wx.grid.GridTableMessage(self.__table, wx.grid.GRIDTABLE_REQUEST_VIEW_GET_VALUES)
|
||||
self.__grid.ProcessTableMessage(msg)
|
||||
|
||||
# Update row count
|
||||
self.__rows = cur_rows
|
||||
|
||||
def __on_doubleckick_row(self, evt):
|
||||
"""
|
||||
Open the graphs when a row is doubleclicked.
|
||||
:param evt:
|
||||
:return:
|
||||
"""
|
||||
row = evt.GetRow()
|
||||
symbol1 = self.__grid.GetCellValue(row, COLUMN_SYMBOL1)
|
||||
symbol2 = self.__grid.GetCellValue(row, COLUMN_SYMBOL2)
|
||||
|
||||
mdi.FrameManager.open_frame(parent=self.GetMDIParent(),
|
||||
frame_module='mt5_correlation.gui.mdi_child_correlationgraph',
|
||||
frame_class='MDIChildCorrelationGraph',
|
||||
raise_if_open=True,
|
||||
symbols=[symbol1, symbol2])
|
||||
|
||||
|
||||
class _DataTable(wx.grid.GridTableBase):
|
||||
"""
|
||||
A data table that holds data in a pandas dataframe. Contains highlighting rules for status.
|
||||
"""
|
||||
data = None # The data for this table. A Pandas DataFrame
|
||||
|
||||
def __init__(self, columns):
|
||||
wx.grid.GridTableBase.__init__(self)
|
||||
self.headerRows = 1
|
||||
self.data = pd.DataFrame(columns=columns)
|
||||
|
||||
def GetNumberRows(self):
|
||||
return len(self.data)
|
||||
|
||||
def GetNumberCols(self):
|
||||
return len(self.data.columns) + 1
|
||||
|
||||
def GetValue(self, row, col):
|
||||
if row < self.RowsCount and col < self.ColsCount:
|
||||
return self.data.index[row] if col == 0 else self.data.iloc[row, col - 1]
|
||||
else:
|
||||
raise Exception(f"Trying to access row {row} and col {col} which does not exist.")
|
||||
|
||||
def SetValue(self, row, col, value):
|
||||
self.data.iloc[row, col - 1] = value
|
||||
|
||||
def GetColLabelValue(self, col):
|
||||
if col == 0:
|
||||
if self.data.index.name is None:
|
||||
return 'Index'
|
||||
else:
|
||||
return self.data.index.name
|
||||
return str(self.data.columns[col - 1])
|
||||
|
||||
def GetTypeName(self, row, col):
|
||||
return wx.grid.GRID_VALUE_STRING
|
||||
|
||||
def GetAttr(self, row, col, prop):
|
||||
attr = wx.grid.GridCellAttr()
|
||||
|
||||
# Check that we are not out of bounds
|
||||
if row < self.RowsCount:
|
||||
# If column is status, check and highlight if diverging or converging.
|
||||
if col in [COLUMN_STATUS]:
|
||||
# Is status one of interest
|
||||
value = self.GetValue(row, col)
|
||||
if value != "":
|
||||
if value in [cor.STATUS_DIVERGING]:
|
||||
attr.SetBackgroundColour(wx.RED)
|
||||
elif value in [cor.STATUS_CONVERGING]:
|
||||
attr.SetBackgroundColour(wx.GREEN)
|
||||
else:
|
||||
attr.SetBackgroundColour(wx.WHITE)
|
||||
|
||||
return attr
|
||||
+74
-43
@@ -1,80 +1,80 @@
|
||||
import pandas as pd
|
||||
import MetaTrader5 as mt5
|
||||
import MetaTrader5
|
||||
import logging
|
||||
|
||||
# Timeframes
|
||||
TIMEFRAME_M1 = MetaTrader5.TIMEFRAME_M1
|
||||
TIMEFRAME_M2 = MetaTrader5.TIMEFRAME_M2
|
||||
TIMEFRAME_M3 = MetaTrader5.TIMEFRAME_M3
|
||||
TIMEFRAME_M4 = MetaTrader5.TIMEFRAME_M4
|
||||
TIMEFRAME_M5 = MetaTrader5.TIMEFRAME_M5
|
||||
TIMEFRAME_M6 = MetaTrader5.TIMEFRAME_M6
|
||||
TIMEFRAME_M10 = MetaTrader5.TIMEFRAME_M10
|
||||
TIMEFRAME_M12 = MetaTrader5.TIMEFRAME_M12
|
||||
TIMEFRAME_M15 = MetaTrader5.TIMEFRAME_M15
|
||||
TIMEFRAME_M20 = MetaTrader5.TIMEFRAME_M20
|
||||
TIMEFRAME_M30 = MetaTrader5.TIMEFRAME_M30
|
||||
TIMEFRAME_H1 = MetaTrader5.TIMEFRAME_H1
|
||||
TIMEFRAME_H2 = MetaTrader5.TIMEFRAME_H2
|
||||
TIMEFRAME_H3 = MetaTrader5.TIMEFRAME_H3
|
||||
TIMEFRAME_H4 = MetaTrader5.TIMEFRAME_H4
|
||||
TIMEFRAME_H6 = MetaTrader5.TIMEFRAME_H6
|
||||
TIMEFRAME_H8 = MetaTrader5.TIMEFRAME_H8
|
||||
TIMEFRAME_H12 = MetaTrader5.TIMEFRAME_H12
|
||||
TIMEFRAME_D1 = MetaTrader5.TIMEFRAME_D1
|
||||
TIMEFRAME_W1 = MetaTrader5.TIMEFRAME_W1
|
||||
TIMEFRAME_MN1 = MetaTrader5.TIMEFRAME_MN1
|
||||
|
||||
|
||||
class MT5:
|
||||
"""
|
||||
A class to connect to and interface with MetaTrader 5
|
||||
"""
|
||||
|
||||
# Timeframes
|
||||
TIMEFRAME_M1 = mt5.TIMEFRAME_M1
|
||||
TIMEFRAME_M2 = mt5.TIMEFRAME_M2
|
||||
TIMEFRAME_M3 = mt5.TIMEFRAME_M3
|
||||
TIMEFRAME_M4 = mt5.TIMEFRAME_M4
|
||||
TIMEFRAME_M5 = mt5.TIMEFRAME_M5
|
||||
TIMEFRAME_M6 = mt5.TIMEFRAME_M6
|
||||
TIMEFRAME_M10 = mt5.TIMEFRAME_M10
|
||||
TIMEFRAME_M12 = mt5.TIMEFRAME_M10
|
||||
TIMEFRAME_M15 = mt5.TIMEFRAME_M15
|
||||
TIMEFRAME_M20 = mt5.TIMEFRAME_M20
|
||||
TIMEFRAME_M30 = mt5.TIMEFRAME_M30
|
||||
TIMEFRAME_H1 = mt5.TIMEFRAME_H1
|
||||
TIMEFRAME_H2 = mt5.TIMEFRAME_H2
|
||||
TIMEFRAME_H3 = mt5.TIMEFRAME_H3
|
||||
TIMEFRAME_H4 = mt5.TIMEFRAME_H4
|
||||
TIMEFRAME_H6 = mt5.TIMEFRAME_H6
|
||||
TIMEFRAME_H8 = mt5.TIMEFRAME_H8
|
||||
TIMEFRAME_H12 = mt5.TIMEFRAME_H12
|
||||
TIMEFRAME_D1 = mt5.TIMEFRAME_D1
|
||||
TIMEFRAME_W1 = mt5.TIMEFRAME_W1
|
||||
TIMEFRAME_MN1 = mt5.TIMEFRAME_MN1
|
||||
|
||||
def __init__(self):
|
||||
# Connect to MetaTrader5. Opens if not already open.
|
||||
|
||||
# Logger
|
||||
self.log = logging.getLogger(__name__)
|
||||
self.__log = logging.getLogger(__name__)
|
||||
|
||||
# Open MT5 and log error if it could not open
|
||||
if not mt5.initialize():
|
||||
self.log.error("initialize() failed")
|
||||
mt5.shutdown()
|
||||
if not MetaTrader5.initialize():
|
||||
self.__log.error("initialize() failed")
|
||||
MetaTrader5.shutdown()
|
||||
|
||||
# Print connection status
|
||||
self.log.debug(mt5.terminal_info())
|
||||
self.__log.debug(MetaTrader5.terminal_info())
|
||||
|
||||
# Print data on MetaTrader 5 version
|
||||
self.log.debug(mt5.version())
|
||||
self.__log.debug(MetaTrader5.version())
|
||||
|
||||
def __del__(self):
|
||||
# shut down connection to the MetaTrader 5 terminal
|
||||
mt5.shutdown()
|
||||
MetaTrader5.shutdown()
|
||||
|
||||
def get_symbols(self):
|
||||
"""
|
||||
Gets list of symbols open in MT5 market watch.
|
||||
:return: list of symbols
|
||||
:return: list of symbol names
|
||||
"""
|
||||
# Iterate symbols and get those in market watch.
|
||||
symbols = mt5.symbols_get()
|
||||
symbols = MetaTrader5.symbols_get()
|
||||
selected_symbols = []
|
||||
for symbol in symbols:
|
||||
if symbol.visible:
|
||||
selected_symbols.append(symbol)
|
||||
selected_symbols.append(symbol.name)
|
||||
|
||||
# Log symbol counts
|
||||
total_symbols = mt5.symbols_total()
|
||||
total_symbols = MetaTrader5.symbols_total()
|
||||
num_selected_symbols = len(selected_symbols)
|
||||
self.log.info(f"{num_selected_symbols} of {total_symbols} available symbols in Market Watch.")
|
||||
self.__log.debug(f"{num_selected_symbols} of {total_symbols} available symbols in Market Watch.")
|
||||
|
||||
return selected_symbols
|
||||
|
||||
def get_prices(self, symbol, from_date, to_date, timeframe):
|
||||
"""
|
||||
Gets OHLC price data for the specified symbol.
|
||||
:param symbol: The MT5 symbol to get the price data for
|
||||
:param symbol: The name of the symbol to get the price data for.
|
||||
:param from_date: Date from when to retrieve data
|
||||
:param to_date: Date where to receive data to
|
||||
:param timeframe: The timeframe for the candes. Possible values are:
|
||||
@@ -101,15 +101,46 @@ class MT5:
|
||||
TIMEFRAME_MN1: 1 month
|
||||
:return: Price data for symbol as dataframe
|
||||
"""
|
||||
# Get prices from MT5
|
||||
prices = mt5.copy_rates_range(symbol.name, timeframe, from_date, to_date)
|
||||
self.log.info(f"{len(prices)} prices retrieved for {symbol.name}.")
|
||||
|
||||
# Create dataframe from data and convert time in seconds to datetime format
|
||||
prices_dataframe = pd.DataFrame(prices)
|
||||
prices_dataframe['time'] = pd.to_datetime(prices_dataframe['time'], unit='s')
|
||||
prices_dataframe = None
|
||||
|
||||
# Get prices from MT5
|
||||
prices = MetaTrader5.copy_rates_range(symbol, timeframe, from_date, to_date)
|
||||
if prices is not None:
|
||||
self.__log.debug(f"{len(prices)} prices retrieved for {symbol}.")
|
||||
|
||||
# Create dataframe from data and convert time in seconds to datetime format
|
||||
prices_dataframe = pd.DataFrame(prices)
|
||||
prices_dataframe['time'] = pd.to_datetime(prices_dataframe['time'], unit='s')
|
||||
|
||||
return prices_dataframe
|
||||
|
||||
def get_ticks(self, symbol, from_date, to_date):
|
||||
"""
|
||||
Gets OHLC price data for the specified symbol.
|
||||
:param symbol: The name of the symbol to get the price data for.
|
||||
:param from_date: Date from when to retrieve data
|
||||
:param to_date: Date where to receive data to
|
||||
:return: Tick data for symbol as dataframe
|
||||
"""
|
||||
|
||||
ticks_dataframe = None
|
||||
|
||||
# Get ticks from MT5
|
||||
ticks = MetaTrader5.copy_ticks_range(symbol, from_date, to_date, MetaTrader5.COPY_TICKS_ALL)
|
||||
|
||||
# If ticks is None, there was an error
|
||||
if ticks is None:
|
||||
error = MetaTrader5.last_error()
|
||||
self.__log.error(f"Error retrieving ticks for {symbol}: {error}")
|
||||
else:
|
||||
self.__log.debug(f"{len(ticks)} ticks retrieved for {symbol}.")
|
||||
|
||||
# Create dataframe from data and convert time in seconds to datetime format
|
||||
try:
|
||||
ticks_dataframe = pd.DataFrame(ticks)
|
||||
ticks_dataframe['time'] = pd.to_datetime(ticks_dataframe['time'], unit='s')
|
||||
except RecursionError:
|
||||
self.__log.warning("Error converting ticks to dataframe.")
|
||||
|
||||
return ticks_dataframe
|
||||
|
||||
+6
-2
@@ -1,7 +1,11 @@
|
||||
pandas==1.2.1
|
||||
matplotlib==3.3.4
|
||||
matplotlib==3.4.2
|
||||
MetaTrader5==5.0.34
|
||||
pytz==2021.1
|
||||
scipy==1.6.0
|
||||
logging==0.4.9.6
|
||||
pyyaml==5.4.1
|
||||
markdown==3.3.4
|
||||
wxpython==4.1.1
|
||||
mock==4.0.3
|
||||
numpy==1.20.0
|
||||
wxconfig==1.2
|
||||
@@ -0,0 +1,362 @@
|
||||
import unittest
|
||||
from unittest.mock import patch, PropertyMock
|
||||
import time
|
||||
import mt5_correlation.correlation as correlation
|
||||
import pandas as pd
|
||||
from datetime import datetime, timedelta
|
||||
from test_mt5 import Symbol
|
||||
import random
|
||||
import os
|
||||
|
||||
|
||||
class TestCorrelation(unittest.TestCase):
|
||||
# Mock symbols. 4 Symbols, 3 visible.
|
||||
mock_symbols = [Symbol(name='SYMBOL1', visible=True),
|
||||
Symbol(name='SYMBOL2', visible=True),
|
||||
Symbol(name='SYMBOL3', visible=False),
|
||||
Symbol(name='SYMBOL4', visible=True),
|
||||
Symbol(name='SYMBOL5', visible=True)]
|
||||
|
||||
# Start and end date for price data and mock prices: base; correlated; and uncorrelated.
|
||||
start_date = None
|
||||
end_date = None
|
||||
price_columns = None
|
||||
mock_base_prices = None
|
||||
mock_correlated_prices = None
|
||||
mock_uncorrelated_prices = None
|
||||
|
||||
def setUp(self):
|
||||
"""
|
||||
Creates some price data fro use in tests
|
||||
:return:
|
||||
"""
|
||||
# Start and end date for price data and mock price dataframes. One for: base; correlated; uncorrelated and
|
||||
# different dates.
|
||||
self.start_date = datetime(2021, 1, 1, 1, 5, 0)
|
||||
self.end_date = datetime(2021, 1, 1, 11, 30, 0)
|
||||
self.price_columns = ['time', 'close']
|
||||
self.mock_base_prices = pd.DataFrame(columns=self.price_columns)
|
||||
self.mock_correlated_prices = pd.DataFrame(columns=self.price_columns)
|
||||
self.mock_uncorrelated_prices = pd.DataFrame(columns=self.price_columns)
|
||||
self.mock_correlated_different_dates = pd.DataFrame(columns=self.price_columns)
|
||||
self.mock_inverse_correlated_prices = pd.DataFrame(columns=self.price_columns)
|
||||
|
||||
# Build the price data for the test. One price every 5 minutes for 500 rows. Base will use min for price,
|
||||
# correlated will use min + 5 and uncorrelated will use random
|
||||
for date in (self.start_date + timedelta(minutes=m) for m in range(0, 500*5, 5)):
|
||||
self.mock_base_prices = self.mock_base_prices.append(pd.DataFrame(columns=self.price_columns,
|
||||
data=[[date, date.minute]]))
|
||||
self.mock_correlated_prices = \
|
||||
self.mock_correlated_prices.append(pd.DataFrame(columns=self.price_columns,
|
||||
data=[[date, date.minute + 5]]))
|
||||
self.mock_uncorrelated_prices = \
|
||||
self.mock_uncorrelated_prices.append(pd.DataFrame(columns=self.price_columns,
|
||||
data=[[date, random.randint(0, 1000000)]]))
|
||||
|
||||
self.mock_correlated_different_dates = \
|
||||
self.mock_correlated_different_dates.append(pd.DataFrame(columns=self.price_columns,
|
||||
data=[[date + timedelta(minutes=100),
|
||||
date.minute + 5]]))
|
||||
self.mock_inverse_correlated_prices = \
|
||||
self.mock_inverse_correlated_prices.append(pd.DataFrame(columns=self.price_columns,
|
||||
data=[[date, (date.minute + 5) * -1]]))
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_calculate(self, mock):
|
||||
"""
|
||||
Test the calculate method. Uses mock for MT5 symbols and prices.
|
||||
:param mock:
|
||||
:return:
|
||||
"""
|
||||
# Mock symbol return values
|
||||
mock.symbols_get.return_value = self.mock_symbols
|
||||
|
||||
# Correlation class
|
||||
cor = correlation.Correlation(monitoring_threshold=1, monitor_inverse=True)
|
||||
|
||||
# Calculate for price data. We should have 100% matching dates in sets. Get prices should be called 4 times.
|
||||
# We don't have a SYMBOL3 as this is set as not visible. Correlations should be as follows:
|
||||
# SYMBOL1:SYMBOL2 should be fully correlated (1)
|
||||
# SYMBOL1:SYMBOL4 should be uncorrelated (0)
|
||||
# SYMBOL1:SYMBOL5 should be negatively correlated
|
||||
# SYMBOL2:SYMBOL5 should be negatively correlated
|
||||
# We will not use p_value as the last set uses random numbers so p value will not be useful.
|
||||
mock.copy_rates_range.side_effect = [self.mock_base_prices, self.mock_correlated_prices,
|
||||
self.mock_uncorrelated_prices, self.mock_inverse_correlated_prices]
|
||||
cor.calculate(date_from=self.start_date, date_to=self.end_date, timeframe=5, min_prices=100,
|
||||
max_set_size_diff_pct=100, overlap_pct=100, max_p_value=1)
|
||||
|
||||
# Test the output. We should have 6 rows. S1:S2 c=1, S1:S4 c<1, S1:S5 c=-1, S2:S5 c=-1. We are not checking
|
||||
# S2:S4 or S4:S5
|
||||
self.assertEqual(len(cor.coefficient_data.index), 6, "There should be six correlations rows calculated.")
|
||||
self.assertEqual(cor.get_base_coefficient('SYMBOL1', 'SYMBOL2'), 1,
|
||||
"The correlation for SYMBOL1:SYMBOL2 should be 1.")
|
||||
self.assertTrue(cor.get_base_coefficient('SYMBOL1', 'SYMBOL4') < 1,
|
||||
"The correlation for SYMBOL1:SYMBOL4 should be <1.")
|
||||
self.assertEqual(cor.get_base_coefficient('SYMBOL1', 'SYMBOL5'), -1,
|
||||
"The correlation for SYMBOL1:SYMBOL5 should be -1.")
|
||||
self.assertEqual(cor.get_base_coefficient('SYMBOL2', 'SYMBOL5'), -1,
|
||||
"The correlation for SYMBOL2:SYMBOL5 should be -1.")
|
||||
|
||||
# Monitoring threshold is 1 and we are monitoring inverse. Get filtered correlations. There should be 3 (S1:S2,
|
||||
# S1:S5 and S2:S5)
|
||||
self.assertEqual(len(cor.filtered_coefficient_data.index), 3,
|
||||
"There should be 3 rows in filtered coefficient data when we are monitoring inverse "
|
||||
"correlations.")
|
||||
|
||||
# Now aren't monitoring inverse correlations. There should only be one correlation when filtered
|
||||
cor.monitor_inverse = False
|
||||
self.assertEqual(len(cor.filtered_coefficient_data.index), 1,
|
||||
"There should be only 1 rows in filtered coefficient data when we are not monitoring inverse "
|
||||
"correlations.")
|
||||
|
||||
# Now were going to recalculate, but this time SYMBOL1:SYMBOL2 will have non overlapping dates and coefficient
|
||||
# should be None. There shouldn't be a row. We should have correlations for S1:S4, S1:S5 and S4:S5
|
||||
mock.copy_rates_range.side_effect = [self.mock_base_prices, self.mock_correlated_different_dates,
|
||||
self.mock_correlated_prices, self.mock_correlated_prices]
|
||||
cor.calculate(date_from=self.start_date, date_to=self.end_date, timeframe=5, min_prices=100,
|
||||
max_set_size_diff_pct=100, overlap_pct=100, max_p_value=1)
|
||||
self.assertEqual(len(cor.coefficient_data.index), 3, "There should be three correlations rows calculated.")
|
||||
self.assertEqual(cor.coefficient_data.iloc[0, 2], 1, "The correlation for SYMBOL1:SYMBOL4 should be 1.")
|
||||
self.assertEqual(cor.coefficient_data.iloc[1, 2], 1, "The correlation for SYMBOL1:SYMBOL5 should be 1.")
|
||||
self.assertEqual(cor.coefficient_data.iloc[2, 2], 1, "The correlation for SYMBOL4:SYMBOL5 should be 1.")
|
||||
|
||||
# Get the price data used to calculate the coefficients for symbol 1. It should match mock_base_prices.
|
||||
price_data = cor.get_price_data('SYMBOL1')
|
||||
self.assertTrue(price_data.equals(self.mock_base_prices), "Price data returned post calculation should match "
|
||||
"mock price data.")
|
||||
|
||||
def test_calculate_coefficient(self):
|
||||
"""
|
||||
Tests the coefficient calculation.
|
||||
:return:
|
||||
"""
|
||||
# Correlation class
|
||||
cor = correlation.Correlation()
|
||||
|
||||
# Test 2 correlated sets
|
||||
coefficient = cor.calculate_coefficient(self.mock_base_prices, self.mock_correlated_prices)
|
||||
self.assertEqual(coefficient, 1, "Coefficient should be 1.")
|
||||
|
||||
# Test 2 uncorrelated sets. Set p value to 1 to force correlation to be returned.
|
||||
coefficient = cor.calculate_coefficient(self.mock_base_prices, self.mock_uncorrelated_prices, max_p_value=1)
|
||||
self.assertTrue(coefficient < 1, "Coefficient should be < 1.")
|
||||
|
||||
# Test 2 sets where prices dont overlap
|
||||
coefficient = cor.calculate_coefficient(self.mock_base_prices, self.mock_correlated_different_dates)
|
||||
self.assertTrue(coefficient < 1, "Coefficient should be None.")
|
||||
|
||||
# Test 2 inversely correlated sets
|
||||
coefficient = cor.calculate_coefficient(self.mock_base_prices, self.mock_inverse_correlated_prices)
|
||||
self.assertEqual(coefficient, -1, "Coefficient should be -1.")
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_get_ticks(self, mock):
|
||||
"""
|
||||
Test that caching works. For the purpose of this test, we can use price data rather than tick data.
|
||||
Mock 2 different sets of prices. Get three times. Base, One within cache threshold and one outside. Set 1
|
||||
should match set 2 but differ from set 3.
|
||||
:param mock:
|
||||
:return:
|
||||
"""
|
||||
|
||||
# Correlation class to test
|
||||
cor = correlation.Correlation()
|
||||
|
||||
# Mock the tick data to contain 2 different sets. Then get twice. They should match as the data was cached.
|
||||
mock.copy_ticks_range.side_effect = [self.mock_base_prices, self.mock_correlated_prices]
|
||||
|
||||
# We need to start and stop the monitor as this will set the cache time
|
||||
cor.start_monitor(interval=10, calculation_params={'from': 10, 'min_prices': 0, 'max_set_size_diff_pct': 0,
|
||||
'overlap_pct': 0, 'max_p_value': 1}, cache_time=3)
|
||||
cor.stop_monitor()
|
||||
|
||||
# Get the ticks within cache time and check that they match
|
||||
base_ticks = cor.get_ticks('SYMBOL1', None, None)
|
||||
cached_ticks = cor.get_ticks('SYMBOL1', None, None)
|
||||
self.assertTrue(base_ticks.equals(cached_ticks),
|
||||
"Both sets of tick data should match as set 2 came from cache.")
|
||||
|
||||
# Wait 3 seconds
|
||||
time.sleep(3)
|
||||
|
||||
# Retrieve again. This one should be different as the cache has expired.
|
||||
non_cached_ticks = cor.get_ticks('SYMBOL1', None, None)
|
||||
self.assertTrue(not base_ticks.equals(non_cached_ticks),
|
||||
"Both sets of tick data should differ as cached data had expired.")
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_start_monitor(self, mock):
|
||||
"""
|
||||
Test that starting the monitor and running for 2 seconds produces two sets of coefficient history when using an
|
||||
interval of 1 second.
|
||||
:param mock:
|
||||
:return:
|
||||
"""
|
||||
# Mock symbol return values
|
||||
mock.symbols_get.return_value = self.mock_symbols
|
||||
|
||||
# Create correlation class. We will set a divergence threshold so that we can test status.
|
||||
cor = correlation.Correlation(divergence_threshold=0.8, monitor_inverse=True)
|
||||
|
||||
# Calculate for price data. We should have 100% matching dates in sets. Get prices should be called 4 times.
|
||||
# We dont have a SYMBOL2 as this is set as not visible. All pairs should be correlated for the purpose of this
|
||||
# test.
|
||||
mock.copy_rates_range.side_effect = [self.mock_base_prices, self.mock_correlated_prices,
|
||||
self.mock_correlated_prices, self.mock_inverse_correlated_prices]
|
||||
|
||||
cor.calculate(date_from=self.start_date, date_to=self.end_date, timeframe=5, min_prices=100,
|
||||
max_set_size_diff_pct=100, overlap_pct=100, max_p_value=1)
|
||||
|
||||
# We will build some tick data for each symbol and patch it in. Tick data will be from 10 seconds ago to now.
|
||||
# We only need to patch in one set of tick data for each symbol as it will be cached.
|
||||
columns = ['time', 'ask']
|
||||
starttime = datetime.now() - timedelta(seconds=10)
|
||||
tick_data_s1 = pd.DataFrame(columns=columns)
|
||||
tick_data_s2 = pd.DataFrame(columns=columns)
|
||||
tick_data_s4 = pd.DataFrame(columns=columns)
|
||||
tick_data_s5 = pd.DataFrame(columns=columns)
|
||||
|
||||
now = datetime.now()
|
||||
price_base = 1
|
||||
while starttime < now:
|
||||
tick_data_s1 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * 0.5]]))
|
||||
tick_data_s2 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * 0.1]]))
|
||||
tick_data_s4 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * 0.25]]))
|
||||
tick_data_s5 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * -0.25]]))
|
||||
starttime = starttime + timedelta(milliseconds=10*random.randint(0, 100))
|
||||
price_base += 1
|
||||
|
||||
# Patch it in
|
||||
mock.copy_ticks_range.side_effect = [tick_data_s1, tick_data_s2, tick_data_s4, tick_data_s5]
|
||||
|
||||
# Start the monitor. Run every second. Use ~10 and ~5 seconds of data. Were not testing the overlap and price
|
||||
# data quality metrics here as that is set elsewhere so these can be set to not take effect. Set cache level
|
||||
# high and don't use autosave. Timer runs in a separate thread so test can continue after it has started.
|
||||
cor.start_monitor(interval=1, calculation_params=[{'from': 0.66, 'min_prices': 0,
|
||||
'max_set_size_diff_pct': 0, 'overlap_pct': 0,
|
||||
'max_p_value': 1},
|
||||
{'from': 0.33, 'min_prices': 0,
|
||||
'max_set_size_diff_pct': 0, 'overlap_pct': 0,
|
||||
'max_p_value': 1}], cache_time=100, autosave=False)
|
||||
|
||||
# Wait 2 seconds so timer runs twice
|
||||
time.sleep(2)
|
||||
|
||||
# Stop the monitor
|
||||
cor.stop_monitor()
|
||||
|
||||
# We should have 2 coefficients calculated for each symbol pair (6), for each date_from value (2),
|
||||
# for each run (2) so 24 in total.
|
||||
self.assertEqual(len(cor.coefficient_history.index), 24)
|
||||
|
||||
# We should have 2 coefficients calculated for a single symbol pair and timeframe
|
||||
self.assertEqual(len(cor.get_coefficient_history({'Symbol 1': 'SYMBOL1', 'Symbol 2': 'SYMBOL2',
|
||||
'Timeframe': 0.66})),
|
||||
2, "We should have 2 history records for SYMBOL1:SYMBOL2 using the 0.66 min timeframe.")
|
||||
|
||||
# The status should be DIVERGED for SYMBOL1:SYMBOL2 and CORRELATED for SYMBOL1:SYMBOL4 and SYMBOL2:SYMBOL4.
|
||||
self.assertTrue(cor.get_last_status('SYMBOL1', 'SYMBOL2') == correlation.STATUS_DIVERGED)
|
||||
self.assertTrue(cor.get_last_status('SYMBOL1', 'SYMBOL4') == correlation.STATUS_CORRELATED)
|
||||
self.assertTrue(cor.get_last_status('SYMBOL2', 'SYMBOL4') == correlation.STATUS_CORRELATED)
|
||||
|
||||
# We are monitoring inverse correlations, status for SYMBOL1:SYMBOL5 should be DIVERGED
|
||||
self.assertTrue(cor.get_last_status('SYMBOL2', 'SYMBOL5') == correlation.STATUS_DIVERGED)
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_load_and_save(self, mock):
|
||||
"""Calculate and run monitor for a few seconds. Store the data. Save it, load it then compare against stored
|
||||
data."""
|
||||
|
||||
# Correlation class
|
||||
cor = correlation.Correlation()
|
||||
|
||||
# Patch symbol and price data, then calculate
|
||||
mock.symbols_get.return_value = self.mock_symbols
|
||||
mock.copy_rates_range.side_effect = [self.mock_base_prices, self.mock_correlated_prices,
|
||||
self.mock_correlated_prices, self.mock_inverse_correlated_prices]
|
||||
cor.calculate(date_from=self.start_date, date_to=self.end_date, timeframe=5, min_prices=100,
|
||||
max_set_size_diff_pct=100, overlap_pct=100, max_p_value=1)
|
||||
|
||||
# Patch the tick data
|
||||
columns = ['time', 'ask']
|
||||
starttime = datetime.now() - timedelta(seconds=10)
|
||||
tick_data_s1 = pd.DataFrame(columns=columns)
|
||||
tick_data_s3 = pd.DataFrame(columns=columns)
|
||||
tick_data_s4 = pd.DataFrame(columns=columns)
|
||||
now = datetime.now()
|
||||
price_base = 1
|
||||
while starttime < now:
|
||||
tick_data_s1 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * 0.5]]))
|
||||
tick_data_s3 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * 0.1]]))
|
||||
tick_data_s4 = tick_data_s1.append(pd.DataFrame(columns=columns, data=[[starttime, price_base * 0.25]]))
|
||||
starttime = starttime + timedelta(milliseconds=10 * random.randint(0, 100))
|
||||
price_base += 1
|
||||
mock.copy_ticks_range.side_effect = [tick_data_s1, tick_data_s3, tick_data_s4]
|
||||
|
||||
# Start monitor and run for a seconds with a 1 second interval to produce some coefficient history. Then stop
|
||||
# the monitor
|
||||
cor.start_monitor(interval=1, calculation_params={'from': 0.66, 'min_prices': 0, 'max_set_size_diff_pct': 0,
|
||||
'overlap_pct': 0, 'max_p_value': 1},
|
||||
cache_time=100, autosave=False)
|
||||
time.sleep(2)
|
||||
cor.stop_monitor()
|
||||
|
||||
# Get copies of data that will be saved.
|
||||
cd_copy = cor.coefficient_data
|
||||
pd_copy = cor.get_price_data('SYMBOL1')
|
||||
mtd_copy = cor.get_ticks('SYMBOL1', cache_only=True)
|
||||
ch_copy = cor.coefficient_history
|
||||
|
||||
# Save, reset data, then reload
|
||||
cor.save("unittest.cpd")
|
||||
cor.load("unittest.cpd")
|
||||
|
||||
# Test that the reloaded data matches the original
|
||||
self.assertTrue(cd_copy.equals(cor.coefficient_data),
|
||||
"Saved and reloaded coefficient data should match original.")
|
||||
self.assertTrue(pd_copy.equals(cor.get_price_data('SYMBOL1')),
|
||||
"Saved and reloaded price data should match original.")
|
||||
self.assertTrue(mtd_copy.equals(cor.get_ticks('SYMBOL1', cache_only=True)),
|
||||
"Saved and reloaded tick data should match original.")
|
||||
self.assertTrue(ch_copy.equals(cor.coefficient_history),
|
||||
"Saved and reloaded coefficient history should match original.")
|
||||
|
||||
# Cleanup. delete the file
|
||||
os.remove("unittest.cpd")
|
||||
|
||||
@patch('mt5_correlation.correlation.Correlation.coefficient_data', new_callable=PropertyMock)
|
||||
def test_diverged_symbols(self, mock):
|
||||
"""
|
||||
Test that diverged_symbols property correctly groups symbols and counts.
|
||||
:param mock:
|
||||
:return:
|
||||
"""
|
||||
# Correlation class
|
||||
cor = correlation.Correlation()
|
||||
|
||||
# Mock the correlation data. Symbol 1 has diverged 3 times; symbols 2 has diverged twice; symbol 3 has
|
||||
# diverged once; symbols 4 has diverged twice and symbol 5 has not diverged at all. Use all diverged status'
|
||||
# (diverged, diverging & converging). Also add a row for a non diverged pair.
|
||||
mock.return_value = pd.DataFrame(columns=['Symbol 1', 'Symbol 2', 'Status'], data=[
|
||||
['SYMBOL1', 'SYMBOL2', correlation.STATUS_DIVERGED],
|
||||
['SYMBOL1', 'SYMBOL3', correlation.STATUS_DIVERGING],
|
||||
['SYMBOL1', 'SYMBOL4', correlation.STATUS_CONVERGING],
|
||||
['SYMBOL2', 'SYMBOL4', correlation.STATUS_DIVERGED],
|
||||
['SYMBOL2', 'SYMBOL3', correlation.STATUS_CORRELATED]])
|
||||
|
||||
# Get the diverged_symbols data and check the counts
|
||||
diverged_symbols = cor.diverged_symbols
|
||||
self.assertEqual(diverged_symbols.loc[(diverged_symbols['Symbol'] == 'SYMBOL1')]['Count'].iloc[0], 3,
|
||||
"Symbol 1 has diverged three times.")
|
||||
self.assertEqual(diverged_symbols.loc[(diverged_symbols['Symbol'] == 'SYMBOL2')]['Count'].iloc[0], 2,
|
||||
"Symbol 2 has diverged twice.")
|
||||
self.assertEqual(diverged_symbols.loc[(diverged_symbols['Symbol'] == 'SYMBOL3')]['Count'].iloc[0], 1,
|
||||
"Symbol 3 has diverged once.")
|
||||
self.assertEqual(diverged_symbols.loc[(diverged_symbols['Symbol'] == 'SYMBOL4')]['Count'].iloc[0], 2,
|
||||
"Symbol 4 has diverged twice.")
|
||||
self.assertFalse('SYMBOL5' in diverged_symbols['Symbol'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,77 @@
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
import mt5_correlation.mt5 as mt5
|
||||
import pandas as pd
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
class Symbol:
|
||||
""" A Mock symbol class"""
|
||||
name = None
|
||||
visible = None
|
||||
|
||||
def __init__(self, name, visible):
|
||||
self.name = name
|
||||
self.visible = visible
|
||||
|
||||
|
||||
class TestMT5(unittest.TestCase):
|
||||
"""
|
||||
Unit test for MT5. Uses mock to mock MetaTrader5 connection.
|
||||
"""
|
||||
|
||||
# Mock symbols. 5 Symbols, 4 visible.
|
||||
mock_symbols = [Symbol(name='SYMBOL1', visible=True),
|
||||
Symbol(name='SYMBOL2', visible=True),
|
||||
Symbol(name='SYMBOL3', visible=False),
|
||||
Symbol(name='SYMBOL4', visible=True),
|
||||
Symbol(name='SYMBOL5', visible=True)]
|
||||
|
||||
# Mock prices for symbol 1
|
||||
mock_prices = pd.DataFrame(columns=['time', 'close'],
|
||||
data=[[datetime(2021, 1, 1, 1, 5, 0), 123.123],
|
||||
[datetime(2021, 1, 1, 1, 10, 0), 123.124],
|
||||
[datetime(2021, 1, 1, 1, 15, 0), 123.125],
|
||||
[datetime(2021, 1, 1, 1, 20, 0), 125.126],
|
||||
[datetime(2021, 1, 1, 1, 25, 0), 123.127],
|
||||
[datetime(2021, 1, 1, 1, 30, 0), 123.128]])
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_get_symbols(self, mock):
|
||||
# Mock return value
|
||||
mock.symbols_get.return_value = self.mock_symbols
|
||||
|
||||
# Call get_symbols
|
||||
symbols = mt5.MT5().get_symbols()
|
||||
|
||||
# There should be four, as one is set as not visible
|
||||
self.assertTrue(len(symbols) == 4, "There should be 5 symbols returned from MT5.")
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_get_prices(self, mock):
|
||||
# Mock return value
|
||||
mock.copy_rates_range.return_value = self.mock_prices
|
||||
|
||||
# Call get prices
|
||||
prices = mt5.MT5().get_prices(symbol='SYMBOL1', from_date='01-JAN-2021 01:00:00',
|
||||
to_date='01-JAN-2021 01:10:25', timeframe=mt5.TIMEFRAME_M5)
|
||||
|
||||
# There should be 6
|
||||
self.assertTrue(len(prices.index) == 6, "There should be 6 prices.")
|
||||
|
||||
@patch('mt5_correlation.mt5.MetaTrader5')
|
||||
def test_get_ticks(self, mock):
|
||||
# Mock return value
|
||||
mock.copy_ticks_range.return_value = self.mock_prices
|
||||
|
||||
# Call get ticks
|
||||
ticks = mt5.MT5().get_ticks(symbol='SYMBOL1', from_date='01-JAN-2021 01:00:00', to_date='01-JAN-2021 01:10:25')
|
||||
|
||||
# There should be 6
|
||||
self.assertTrue(len(ticks.index) == 6, "There should be 6 prices.")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user