* feat(FeatureExtraction): added fractionally differentiated returns to remove lagged returns
* fix(Sweep): config
* fix(Sweep): name
* fix(Sweep): grid
* feat(Config): separated sliding_window_size_level1 & sliding_window_size_level2
* feat(Dependencies): added ray, now using it to parallel process feature extraction
* fix(Dependencies): added pip explicitly
* fix(Dependencies): removed ray from root
* fix(Models): average model was probably not taking the right timestamp to average
* feat(Config): separated expanding_window_level1 & expanding_window_level2
* fix(Config): set n_features_to_select to the optimal 30
* feat(Selection): added prototype feature selection python script
* feat(Utils): added some helpers for the future from Advances in Financial ML book
* feat(Selection): added RFECV
* feat(Selection): added configurable feature selection step into pipeline
* feat(Config): added level_1 & level_2 default config, PCA before feature selection process starts
* feat(Selection): added backup feature selector models if current one can't output feature importance, removed unnecessary array for level-2 models
* fix(Training): deal with zero first value coming out of static models
* feat(Sweep): added feature selection sweep
* fix(Sweep): config problem
* fix(Sweep): config
* chore(Utils): removed unnecessary purged k-fold crossval class
* feat(Config): added dimensionality_reduction as a separate flag
* fix(Sweep): config updated
* fix(Sweep): sweep name
* chore(Config): updated level_2 config to the best performing configuation
* fix(Reporting): use weighted average (with no_of_samples as weights) and only report level-1 OR level-2 model performance
* chore(Config): updated sweep config
* fix(Reporting): missing import
* fix(Evaluation): get_first_valid_return_index can deal with zero valid indexes
* fix(Training): increase threshold for skipping assets
* fix(DataLoader): target asset should be always the first column
* feat: Added ensemble models to sweep and configured naming convention.
* fix: Default value was misconfigured.
* feat(Sweep): separated level-1 and level-2 sweep configs, skip assets with too few samples to train on, simplified model mapping
* fix(Sweep): syntax error
* chore(Sweep): set sweep names accordingly
* fix(Sweep): set sliding window
* fix(Sweep): adjusted sweep config
* fix(Sweep): removed invalid feature extractor preset
Co-authored-by: Mark Aron Szulyovszky <mark.szulyovszky@gmail.com>
* feat(Data): add option to predict 3 classes
* feat(Evaluation): added ability to evaluate 3 class predictions
* chore(Config): set sensible config for regression models
* feat(Data): added option to use balanced or imbalanced three-class data
* feat(Evaluate): correctly track "no_of_samples" now that we have three classes
* chore(Sweep): remove probably not useful scaler values from sweep
* feat(Config): feature extractors are enabled one-by-one with a bool, added previous model to model.fit()
* fix(Sweep): removed unused `other_features` parameter that fails sweep
* feat(Config): using preset names for defining feature extractors again
* fix(Tests): fixed model stub classes
* feat(Models): added `debug_future_lookahead`, sped up LogisticRegression & DecisionTreeClassifier
* feat(Training): added ability to train on expanding_window
* feat(Models): tuned some hyperparameters, added expanding_window to sweep config, fixed tests
* feat(Models): tune parameters of ensemble models
* fix(Config): use window size that works with ensembling
* refactor(Reporting): only report the last model's results, moved wandb-related functions to `reporting`
* fix(Reporting): use .mean() on axis 1 to retain the metrics, fixed get_model_name()
* fix(Config): sweep file syntax
* fix(Config): changed hyperparameter search method to "bayes"
* chore(Sweep): adjusted sweep config based on the results we saw (removed Momentum as well)
* fix(Sweep): only use classification method for now, we're not yet prepared for regression
* chore(Config): added more options to sweep.yaml
* fix(Config): mistake with declaring `value` <> `values` in sweep.yaml
* fix(Config): set the metric to optimize for
* feat: Parametricized model selection works now.
* feat: Fixed errors. Sweep generates and you can run it, but it gives an error for model.only_columns attribute.
* feat: Factored the wandb management, default config managment and the model_dictionary out of the run_pipeline to a seperate file.
* fix: Took out prints and fixed the mismatch of ensemble models when classifing.
* fix(Models): added StaticMomentum model to the dictionary, hopefully fixed sklearn-ex RandomForestRegressor problem
* fix(Dependencies): pin scikit-learn-ex's version, moved map_model_name_to_function to `models`
* feat(Sweep): added `run_sweep.py` shortcut
* feat(Pipeline): skip training a meta model if array is empty
Co-authored-by: Mark Aron Szulyovszky <mark.szulyovszky@gmail.com>
* feat(Models): added StaticAverageModel for average ensembling
* feat(Models): made sure we only pipe in predictions to StaticAverageModel, added StaticNaiveModel as potential baseline
* chore(Models): removed unnecessary commented out code
* feat: initial wandb configured. Sweep parameters aren't configured yet.
* feat: Wandb logs now results.
* feat: gitignore.
* fix: Took out print()
* feat: Changed default value of wandb to False.
* feat: Added wandb to turn of automatically if there is no environment variable to start it (when we push it). Added environment configuration aswell.
* feat: Each assets model is seperated into a run that tracks the results.
* fix: Nonetype error, truncated assets.
* fix: Fixed the logging to wandb.
* feat: initial wandb configured. Sweep parameters aren't configured yet.
* feat: Wandb logs now results.
* feat: gitignore.
* fix: Took out print()
* feat: Changed default value of wandb to False.
* feat: Added wandb to turn of automatically if there is no environment variable to start it (when we push it). Added environment configuration aswell.
* fix(Dependencies): the package name seems to be python-dotenv
Co-authored-by: Mark Aron Szulyovszky <mark.szulyovszky@gmail.com>
* fix(Core): correct forward returns calculation, classifiers are now working again, only train from when asset returns are available
* feat(Utils): added get_first_valid_return_index()
* feat(Ensemble): return models from `run_whole_pipeline`
* feat(Ensemble): added ensemble step, fixed walk_forward_train_test predictions index confusion,
* chore(Pipeline): remove unnecessary extra ensemble results dataframe
* refactor(Core): removed unnecessary ensemble_train_predict, moved run_single_asset_trainig_pipeline to a separate file
* feat(Training): added scaling on expanding window (the past) to walk_forward_train_test(), now printing out mean sharpe ratio
* feat(CI): added environment.yml file
* chore(Environment): update env.yml
* feat(CI): added testing workflow
* fix(CI): renamed enviroment.yml
* fix(Tests): added missing new parameter to walk_forward_train_test()
* fix(Evaluate): ignore empty data at evaluation time, so we don't inflate the model's performance
* refactor(Pipeline): pass in data_loader arguments to the pipeline
* feat(Evaluation): added sharpe, sortino, etc
* fix: Took out the method to fill NaN numbers with 0s. This way in evaluation we can ignore NaN values.
* fix: Fix of the fix added fillna back. Either we root out NaN lines in the very beginning or we stick with the method you created.
Co-authored-by: Daniel Szemerey <szemy2@gmail.com>
* feat(Tests): added basic unit tests for walk_forward_train_test()
* fix(Tests): inherit from BaseEstimator, fix index problems in walk_forward_train_test
* fix(WalkForward): predictions were mistakenly removed, oops
* fix(WalkForward): mistakenly re-assiging model
* feat(Eval): added format_data_for_backtest()
* feat(Data): added many configurable parameters to load_files to reduce boilerplate and prepare for HPO
* feat(Core): added walk forward method of training/testing
* fix(Model): remove the unnecessary softmax activation from the keras models
* feat(Core): added walk_forward_train_test()
* feat: Refractored and created new model. Pipeline not ready yet.
* feat: Implemented and refactored a data pipeline.
* ref: Refractored to make more sense.
* feat: Training works now with models that you can change.
* feat: Added predict function but without working instructions.
* feat: gitignore.
* feat: Started implementing pytorch-forecasting.
* feat(Forecasting): pytorch-forecasting scaffolding is now working, added narrow data format, fixed missing `time` column index name
Co-authored-by: Daniel Szemerey <szemy2@gmail.com>
Mark Aron Szulyovszky