gavindiaz/drift
1
0
Fork 0
mirror of https://github.com/webclinic017/drift.git synced 2026-08-19 13:58:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(Transformations): replaced feature selection pre-processing step with online version (with cache) (#170)

Browse Source 
* feat(Transformations): removed feature-selection pre-processing step completely

* fix(Core): removed unnecessary `original_X`

* fix(Transformations): use the X_expanding_window to transform subsequent data

* fix(RFE): should check for model correctly

* fix(Config): only re-train the model every 40 timestamp

* fix(MetaLabeling): pass in the correct X to meta-labeling step

* fix(Transformation): PCA should at least keep as many features as sliding_window_size

* feat(Transformations): cache transformations across the same asset

* fix(Tests): missing preloaded_transformations arg

* chore(Config): got rid of unnecessary 'classification_models' and 'regression_models' dictionary keys
This commit is contained in:
Mark Aron Szulyovszky
2022-01-17 11:43:51 +01:00
committed by GitHub
parent 31dc847be1
commit 6b26643ece
27 changed files with 240 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files
run_pipeline.py
+3 -4
View File
@@ -2,7 +2,7 @@ import pandas as pd
from typing import Callable, Optional
from data_loader.load_data import load_data
from data_loader.process_data import process_data, check_data
from data_loader.process_data import check_data
from reporting.wandb import launch_wandb, register_config_with_wandb
from reporting.reporting import report_results
@@ -56,13 +56,12 @@ def __run_training(model_config:dict, training_config:dict, data_config:dict):
# 1. Load data, check for validity and process data (feature selection, dimensionality reduction, etc.)
X, y, target_returns = load_data(**configs['data_config'])
if check_data(X, y, configs['training_config']) is False: continue
X, original_X = process_data(X, y, configs)
# 2. Train a Primary model with optional metalabeling for each asset
training_step_primary, current_predictions = primary_step(X, y, original_X, asset, target_returns, configs, reporting)
training_step_primary, current_predictions = primary_step(X, y, asset, target_returns, configs, reporting)
# 3. Train an Ensemble model with optional metalabeling for each asset
training_step_secondary = secondary_step(X, y, original_X, current_predictions, asset, target_returns, configs, reporting)
training_step_secondary = secondary_step(X, y, current_predictions, asset, target_returns, configs, reporting)
# 4. Save the models
reporting.all_assets.append(Reporting.Asset(ticker=asset[1], primary=training_step_primary, secondary=training_step_secondary))