Mark Aron Szulyovszky 1cd0119589 feat(DataLoader): caching MVP, added ability to use standard scaling for exogenous data, scaling is now also done before feature selection (#105)
* fix(FeatureExtractor): apply log to transform some series to normality

* feat(DataLoader): add ability of not returning returns when they're not needed (exogenous data), applied log to certain features

* feat(FeatureExtractors): added standard scaling for exogenous data

* feat(FeatureSelection): scale data with the passed in scaler before doing feature-selection

* fix(Config): sweep config

* feat(Models): output probability, store it

* feat(Core): added caching to select_features() and load_data()

* fix(Dependencies): added diskcache

* fix(Training): error when creating results DF

* feat(Models): added xgboost, fixed tests

* refactor(Cache): moved hashing to a separate function, created wrapper functions to separate business logic and caching

* fix(Tests): new syntax

* fix(Model): XGboost can't handle -1 class, so we'll use the deprecated label_encoder fornow

* fix(Model): XGBoost config

* feat(Cache): add run_clear_cache script

* fix(Pipeline) accidentally re-instatiating all_predictions for each asset
2022-01-04 11:44:35 +01:00

Financial time series prediction

And end-to-end pipeline to train predictive Machine Learning models on financial (non-stationary, regime changing) time series.

Why?

Machine learning on financial time series require a fundamentally different approach than used in other ML domains. The data is non-stationary, where the patterns frequently change, and it's extremely important to not to leak out-of-sample data into the training set objective evaluation is important.

There are very few open-source end-to-end machine learning pipelines that can be effectively used to train and evaluate ML models on financial time series. Among them areqlib, AlphaPy.

This repo is different to them in a couple of angles:

  • Feature extraction and selection is an important, pre-built step in the pipeline. Training models on
  • Training and evaluation is done in a walk-forward manner. We argue that that one or two train/test split is not adoquate to evaluate an ML model's performance in a non-stationary, regime changing environment. The walk-forward methodology enables us to evaluate the model's performance on almost the whole time series.
  • The walk-forward training/evaluation methodology enables "online" (ever-changing) models, that adapt to the market environment. You can specify how frequently would you like to re-train the models.

This project is inspired partially by Marcos Lopez de Prado's Advances in Financial Machine Learning and The Alpha Scientist's blogposts.

Installation

Use the conda environment file attached!:)

Pipeline components

  • Feature extraction
  • Dimensionality reduction
  • Feature selection
  • Training Level-1 models
  • Training Level-2 (Ensemble) model
  • Evaluation of models
  • Cross-sectional portfolio construction [IN PROGRESS]
S
Description
No description provided
Readme 11 MiB
Languages
Python 85%
Jupyter Notebook 15%