* refactor(Project): removed regression method (we can still use regression models, but we'll need map them to classes later)
* fix(Training): removed mistakenly left in `method` parameter
* fix, feat: Fixed inference processing data. Add transformation attribute.
* feat: Added transformations step, refractored the loop to make more sense (divided the train and inference loop).
* feat: Truncated models over time and transformations over time. Fixed some typing aswell.
* fix: Fixed a number of out of array problems.
* feat: Inference now works!
* fix(Steps): runtime error not checking for None
* fix(Steps): preloaded transformers are not optional anymore, sped up training by temporary increasing the retrain_every
* fix(CI): disable ray memory monitoring
* refactor(Inference): removed truncate_models and replaced it with filling X with NaN until inference should start
* feat(Inference): added index_from parameter
* fix(Tests): walk_forward test
* refactor(Pipeline): only predict one asset
* refactor(Inference): removed select_models step, inference code moved to run_inference.py so it matches convention (similar to run_pipeline.py)
* fix(Evaluation): adjust transaction costs
* fix(Config): adjusted retrain_every
Co-authored-by: Daniel Szemerey <szemereydaniel@gmail.com>
Co-authored-by: Mark Aron Szulyovszky <mark.szulyovszky@gmail.com>
* feat(Transformations): removed feature-selection pre-processing step completely
* fix(Core): removed unnecessary `original_X`
* fix(Transformations): use the X_expanding_window to transform subsequent data
* fix(RFE): should check for model correctly
* fix(Config): only re-train the model every 40 timestamp
* fix(MetaLabeling): pass in the correct X to meta-labeling step
* fix(Transformation): PCA should at least keep as many features as sliding_window_size
* feat(Transformations): cache transformations across the same asset
* fix(Tests): missing preloaded_transformations arg
* chore(Config): got rid of unnecessary 'classification_models' and 'regression_models' dictionary keys
* feat: Basic pipeline extended.
* feat: Added conversion of model list to existing structure (model_name, model_in_time). Fixed loading of previous models and dicts.
* fix: Had an unfinished function.
* fix: Inference wasn't getting model_over_time. Now transformations are not getting it either yet.
Co-authored-by: Daniel Szemerey <szemereydaniel@gmail.com>
* feat(Transformations): added Transformations abstraction & handling in walk_forward_train() & inference()
* fix(WalkForward): use Dataframes to call Transformation.fit_transform()
* feat(WalkForward): restored option for models to recieve unscaled data
* fix(Transformations): output DataFrame as expected
* fix(Tests): missing new property
* refr: Took out main primary and secondary loops and data processing.
* feat: Tidied the code up.
* feat: Saving models and results now works in a type safe way.
* fix: There was error in the saving function.
* chore: Took out some remaining comments.
* fix: Fixed the previous data checking process.
* feat: Fixed model selection method. I will continue the inference after we merged.
Co-authored-by: Daniel Szemerey <szemereydaniel@gmail.com>
* refactor(WalkForward): separate train / test functions (draft) to potentially help with inference later
* fix(Training): use the new separate train / test functions
* feat(Training): return and pass in scalers that are necessary for inference
* fix(Project): runtime errors
* fix(WalkForward): use the correct `train_from` value
* fix(Tests): for new walk_forward functions()
* refactor(WalkForward): rename `walk_forward_test()` to `walk_forward_inference()`
* feat: Basic scaffolding up for inference process after training.
* feat: Saving and loading models works. Inference works nearly.
* feat: Added inference pipeline.
* feat: Saving model now accoring to date and time; loading models now selects from latest file. Fixed the creation of dictionary of models.
* feat: Added lightweight asset config, but full pipeline.
* feat: Added new naming for dictionary.
* fix: Fixed dictionary naming convention.
* fix: Fixed naming again, now the model structure is good
* fix: Changed the output path and the return values from run_pipeline.
* feat: Added function to make sure folder exists for output models.
Co-authored-by: Daniel Szemerey <szemereydaniel@gmail.com>
* refactor(Naming): use `primary_models` & `meta_labeling_models`
* refactor(Naming): using primary * meta_labeling across config and in pipeline
* feat(Pipeline): added back Ensemble models
* fix(Pipeline): compiler error
* fix(Config): typo
* chore(Pipeline): removed unused averaging step
* revert the changes in discretizing
* chore(Pipeline): remove sharpe improvement logging
* fix(Pipeline): ensemble predictions should be a pd.Series instead of a DataFrame
* fix(Pipeline): discard unnecessary ensemble_probabilities
* fix(Pipeline): fixes regarding various meta-labeling ensemble bugs
* fix(Reporting): use the new naming convention
* fix(Reporting): use the right variable
* feat(Sweep): new sweep for ensemble models
* fix(Sweep): config reference
* fix(Config): simplified dev config
* fix(Models): use the faster LR model
* fix(Models): use LGBM in the meta-labeling model for speed
* fix(Selection): always use the first model for feature selection, commented out caching from select_features() as it's close to redundant in terms of speed
Mark Aron Szulyovszky