gavindiaz/drift
1
0
Fork 0
mirror of https://github.com/webclinic017/drift.git synced 2026-08-18 21:38:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(DataLoader): caching MVP, added ability to use standard scaling for exogenous data, scaling is now also done before feature selection (#105)

Browse Source 
* fix(FeatureExtractor): apply log to transform some series to normality

* feat(DataLoader): add ability of not returning returns when they're not needed (exogenous data), applied log to certain features

* feat(FeatureExtractors): added standard scaling for exogenous data

* feat(FeatureSelection): scale data with the passed in scaler before doing feature-selection

* fix(Config): sweep config

* feat(Models): output probability, store it

* feat(Core): added caching to select_features() and load_data()

* fix(Dependencies): added diskcache

* fix(Training): error when creating results DF

* feat(Models): added xgboost, fixed tests

* refactor(Cache): moved hashing to a separate function, created wrapper functions to separate business logic and caching

* fix(Tests): new syntax

* fix(Model): XGboost can't handle -1 class, so we'll use the deprecated label_encoder fornow

* fix(Model): XGBoost config

* feat(Cache): add run_clear_cache script

* fix(Pipeline) accidentally re-instatiating all_predictions for each asset
This commit is contained in:
Mark Aron Szulyovszky
2022-01-04 11:44:35 +01:00
committed by GitHub
parent 867269df2b
commit 1cd0119589
27 changed files with 324 additions and 206 deletions
Download Patch File Download Diff File Expand all files Collapse all files
config/hashing.py
+24
View File
@@ -0,0 +1,24 @@
from utils.types import DataCollection
def hash_data_config(data_config: dict) -> str:
def hash_data_collection(data_collection: DataCollection) -> str: return ''.join([a[0] + a[1] for a in data_collection])
def hash_feature_extractors(feature_extractos) -> str: return ''.join([f[0] for f in feature_extractos])
def to_str(x): return ''.join([str(i) for i in x])
return '_'.join(to_str([
hash_data_collection(data_config['assets']),
hash_data_collection(data_config['other_assets']),
hash_data_collection(data_config['exogenous_data']),
data_config['target_asset'][0] + data_config['target_asset'][1],
data_config['load_non_target_asset'],
data_config['log_returns'],
data_config['forecasting_horizon'],
hash_feature_extractors(data_config['own_features']),
hash_feature_extractors(data_config['other_features']),
hash_feature_extractors(data_config['exogenous_features']),
data_config['index_column'],
data_config['method'],
data_config['no_of_classes'],
data_config['narrow_format']
]))