drift/docs/index.rst

.. drift documentation master file, created by
   Daniel Szemerey and Mark Szulyovszky on Wed Feb 02 14:02:22 2022.



Welcome to drift's documentation!
==================================

TLDR: *Drift helps you train and make predictions on time-series data.*

.. figure:: images/pipeline.png
   :alt: Figure of Data transformation pipeline

   Pipeline of the entire process.

Drift is an **end-to-end***, **composable*** **modelling pipeline for financial time-series prediction**. It was designed for quantitative financial predictions.

Drift was specifically engineered not to fool the user: it uses walk-forward method for analysis and makes sure no future information is introduced.

Drift makes it easy to use state-of-the-art methods financial ML techniques, like the triple-barrier labeling method, ensembling models and adding bet-sizing into the mix (with meta-labeling). 

Drift has two level of usage: You can simply use existing models and transformations and just provide the data and the target asset, or you can customize your own pipeline.





Technical Explanation
==================================

Why Drift
--------------------------------

Machine learning on financial time series requires a fundamentally different approach compared to standard ML domains. 
The (small amount of) data is non-stationary, extremely noisy, where the patterns frequently change, and it's extremely important to not to leak out-of-sample data into the test set.

How Drift is different
--------------------------------

There are very few open-source end-to-end machine learning pipelines that can be effectively used to train and evaluate ML models on financial time series. Among them are: [qlib](https://github.com/microsoft/qlib), [AlphaPy](https://github.com/ScottfreeLLC/AlphaPy).

Drift is different to them in a couple of angles:

- All pre-processing steps are *online (up until a certain point),* ****so they never inject lookahead bias into the mix. (this is a major issue with finML papers)
- Feature extraction and selection are an important, pre-built step in the pipeline. Garbage in, garbage out!
- Evaluation is done in a [walk-forward manner](https://en.wikipedia.org/wiki/Walk_forward_optimization). We argue that that a train/validation/test split is not adequate to evaluate an ML model's performance in a non-stationary, regime changing environment. The walk-forward methodology enables us to evaluate the model's performance on almost the whole time series.
- Training can be done in any way possible, including Combinatorial purged k-fold cross-validation. You can shuffle the past in any way you prefer, but you can never use data from the future to train the model.
- Instead of training one model, you train tons of models **over time**, that are making predictions until they become obsolete. The walk-forward training/evaluation methodology enables "online" (ever-changing) models, that adapt to the market environment. You can specify how frequently would you like to re-train the models.
- Ensemble-by-default: train multiple models, and average their predictions. Improves performance and adds a lot of robustness in a low signal-to-noise environment, like financial time series.
- Bet sizing and Meta-labeling (training a model to evaluate a lower level model's prediction for each timestamp) is a built-in feature.

This project is inspired partially by [Marcos Lopez de Prado's Advances in Financial Machine Learning](https://www.wiley.com/en-us/Advances+in+Financial+Machine+Learning-p-9781119482086) and [The Alpha Scientist's blogposts](https://alphascientist.com/).


External Links
--------------------------------

For more information refer
`here<www.python.org>`


.. py:function:: square(x)

   return the square of a function





Contents
==================================

.. toctree::
   :maxdepth: 2

   setup/index
   basic-usage/index
   advanced-usage/index