diff --git a/.github/workflows/repo_status_weekly.yml b/.github/workflows/repo_status.yml similarity index 95% rename from .github/workflows/repo_status_weekly.yml rename to .github/workflows/repo_status.yml index d18c47b..ab10d78 100644 --- a/.github/workflows/repo_status_weekly.yml +++ b/.github/workflows/repo_status.yml @@ -2,7 +2,7 @@ name: Repo-Updater on: schedule: - - cron: '0 0 * * *' # do every hour for now + - cron: '0 0 * * *' # daily jobs: diff --git a/.github/workflows/wiki_gen_daily.yml b/.github/workflows/wiki_gen.yml similarity index 85% rename from .github/workflows/wiki_gen_daily.yml rename to .github/workflows/wiki_gen.yml index 8425de3..cb734e2 100644 --- a/.github/workflows/wiki_gen_daily.yml +++ b/.github/workflows/wiki_gen.yml @@ -25,11 +25,11 @@ jobs: python -m pip install --upgrade pip if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - - name: execute py script # run the run.py to get the latest data - run: | - python git_status.py - env: - GIT_TOKEN: ${{ secrets.GIT_TOKEN }} +# - name: execute py script # run the run.py to get the latest data +# run: | +# python git_status.py +# env: +# GIT_TOKEN: ${{ secrets.GIT_TOKEN }} - name: execute py script # run the wiki_gen run: | diff --git a/README.md b/README.md index 6bb223e..f649d7b 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -[![Repo-Updater](https://github.com/firmai/financial-machine-learning/actions/workflows/repo_status_weekly.yml/badge.svg)](https://github.com/firmai/financial-machine-learning/actions/workflows/repo_status_weekly.yml) -[![Wiki-Generator](https://github.com/firmai/financial-machine-learning/actions/workflows/wiki_gen_daily.yml/badge.svg)](https://github.com/firmai/financial-machine-learning/actions/workflows/wiki_gen_daily.yml) +[![Repo-Updater](https://github.com/firmai/financial-machine-learning/actions/workflows/repo_status.yml/badge.svg)](https://github.com/firmai/financial-machine-learning/actions/workflows/repo_status_weekly.yml) +[![Wiki-Generator](https://github.com/firmai/financial-machine-learning/actions/workflows/wiki_gen.yml/badge.svg)](https://github.com/firmai/financial-machine-learning/actions/workflows/wiki_gen_daily.yml) [![Gitter](https://badges.gitter.im/financial-machine-learning/community.svg)](https://gitter.im/financial-machine-learning/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) # Financial Machine Learning and Data Science @@ -19,7 +19,7 @@ A listed repository should be deprecated if: ___ -- All repos/links status including last commit date is updated weekly +- All repos/links status including last commit date is updated daily - 10 Highest ranked repos/links for each section are displayed on main README.md and full list is available within the wiki page - Both Wikis/README.md is updated in realtime as soon as new information are pushed to the repo ___ @@ -40,15 +40,6 @@ ___ | [Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020](https://github.com/AI4Finance-LLC/Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020) | Part of FinRL and provided code for paper [deep reinformacement learning for automated stock trading](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3690996) focuses on ensemble. | 2020-07-26 13:12:53 | 2021-01-21 18:11:59 | 560.0 | :heavy_check_mark: | :star:x4 | | [awesome-deep-trading](https://github.com/cbailes/awesome-deep-trading) | curated list of papers/repos on topics like CNN/LSTM/GAN/Reinforcement Learning etc. Categorized as deep learning for now but there are other topics here. Manually maintained by cbailes | 2018-11-26 03:23:04 | 2021-01-01 09:41:21 | 551.0 | :heavy_check_mark: | :star:x4 | - - - - - - - - - ## Other Models ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/other_models)) @@ -65,15 +56,6 @@ ___ | [Mixture Models I](https://github.com/BlackArbsCEO/Mixture_Models) | Mixture models to predict market bottoms. | 2017-03-20 18:54:24 | 2017-04-25 23:35:20 | 31.0 | :heavy_multiplication_x: | | | [finance_ml](https://github.com/jjakimoto/finance_ml) | NEW | 2018-06-29 21:21:17 | 2019-02-18 12:34:54 | 282.0 | :heavy_multiplication_x: | | - - - - - - - - - ## Data Processing Techniques and Transformations ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/data_processing_techniques_and_transformations)) @@ -82,163 +64,51 @@ ___ | [Advanced ML II](https://github.com/hudson-and-thames/research) | More implementations of Financial Machine Learning (De Prado). | nan | nan | nan | :heavy_check_mark: | | | [Advanced ML](https://github.com/BlackArbsCEO/Adv_Fin_ML_Exercises) | Exercises too Financial Machine Learning (De Prado). | 2018-04-25 17:22:40 | 2020-01-16 17:25:41 | 973.0 | :heavy_check_mark: | | - - - - - - - - - # Portfolio Management ## Portfolio Selection and Optimisation ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/portfolio_selection_and_optimisation)) -- [Distribution Characteristic Optimisation](https://github.com/VivekPa/OptimalPortfolio) - Extends classical portfolio optimisation to take the skewness and kurtosis of the distribution of market invariants into account. -- [Reinforcement Learning](https://github.com/filangel/qtrader) - Reinforcement Learning for Portfolio Management. -- [Efficient Frontier](https://github.com/tthustla/efficient_frontier/blob/master/Efficient%20_Frontier_implementation.ipynb) - Modern Portfolio Theory. -- [PyPortfolioOpt](https://github.com/robertmartin8/PyPortfolioOpt) - Financial portfolio optimisation, including classical efficient frontier and advanced methods. -- [Policy Gradient Portfolio](https://github.com/ZhengyaoJiang/PGPortfolio) - A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. -- [Deep Portfolio Theory](https://github.com/tcloaa/Deep-Portfolio-Theory) - Autoencoder framework for portfolio selection. -- [401K Portfolio Optimisation](https://github.com/otosman/Python-for-Finance/blob/master/Portfolio%20Optimization%20401k.ipynb) - Portfolio analyses and optimisation for 401K. -- [Online Portfolio Selection](https://nbviewer.jupyter.org/github/paulperry/quant/blob/master/OLPS_Comparison.ipynb) - ****Comparing OLPS algorithms on a diversified set of ETFs. -- [OLMAR Algorithm](https://github.com/charlessutton/OLMAR/blob/master/Part3.ipynb) - Relative importance of each component of the OLMAR algorithm. -- [Modern Portfolio Theory](https://nbviewer.jupyter.org/github/Marigold/universal-portfolios/blob/master/modern-portfolio-theory.ipynb) - Universal portfolios; modern portfolio theory. -- [DeepDow](https://github.com/jankrepl/deepdow) - Portfolio optimization with deep learning. + + ## Factor and Risk Analysis ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/factor_and_risk_analysis)) -- [Various Risk Measures](https://github.com/Jorgencr/Alternative-and-Responsible-Investments/blob/master/Final_masterfile.ipynb) - Risk measures and factors for alternative and responsible investments. -- [Pyfolio](https://github.com/quantopian/pyfolio) - Portfolio and risk analytics in Python. -- [Risk Basic](https://github.com/RJT1990/Active-Portfolio-Management-Notes/blob/master/Chapter%203%2C%20Risk.ipynb) - Active portfolio risk management . -- [CAPM](https://github.com/RJT1990/Active-Portfolio-Management-Notes/blob/master/Chapter%202%2C%20CAPM.ipynb) - Expected returns using CAPM. -- [Factor Analysis](https://github.com/garvit-kudesia91/factor_analysis/blob/master/Factor%20Analysis%20of%20Mutual%20Funds.ipynb) - Factor analysis for mutual funds. -- [VaR GaN](https://github.com/hamaadshah/market_risk_gan_keras) - Estimate Value-at-Risk for market risk management using Keras and TensorFlow. -- [VaR](https://github.com/willb/var-notebook/blob/master/var-notebook/var-pdfs.ipynb) - Value-at-risk calculations. -- [Python for Finance](https://github.com/yhilpisch/py4fi/tree/master/jupyter36) - Various financial notebooks. -- [Performance Analysis](https://github.com/quantopian/alphalens) - Performance analysis of predictive (alpha) stock factors. -- [Quant Finance](https://github.com/mrefermat/quant_finance) - General quant repository. -- [Risk and Return](https://github.com/PyDataBlog/Python-for-Data-Science/tree/master/Tutorials) - Riskiness of portfolios and assets. -- [Convex Optimisation](https://github.com/ssanderson/convex-optimization-for-finance/blob/master/notebooks/Main.ipynb) - Convex Optimization for Finance. -- [Factor Analysis](https://github.com/alpha-miner/alpha-mind/tree/master/notebooks) - Factor strategy notebooks. -- [Statistical Finance](https://github.com/mrefermat/FinancePhD/tree/master/FinancialExperiments) - Various financial experiments. - - + + # Techniques ## Unsupervised ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/unsupervised)) -- [PCA Pairs Trading](https://github.com/joelQF/quant-finance/tree/master/Artificial_IntelIigence_for_Trading) - PCA, Factor Returns, and trading strategies. -- [Fund Clusters](https://github.com/frechfrechfrech/Mutual-Fund-Market-Clusters/blob/master/Initial%20Data%20Exploration.ipynb) - Data exploration of fund clusters. -- [VRA Stock Embedding](https://github.com/ml-hongkong/stock2vec) - Variational Reccurrent Autoencoder for Embedding stocks to vectors based on the price history. -- [Industry Clustering](https://github.com/SeanMcOwen/FinanceAndPython.com-ClusteringIndustries) - Clustering of industries. -- [Pairs Trading](https://github.com/marketneutral/pairs-trading-with-ML/blob/master/Pairs%2BTrading%2Bwith%2BMachine%2BLearning.ipynb) - Finding pairs with cluster analysis. -- [Industry Clustering](https://github.com/SeanMcOwen/FinanceAndPython.com-ClusteringIndustries) - Project to cluster industries according to financial attributes. + + ## Textual ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/textual)) -- [NLP](https://github.com/toamitesh/NLPinFinance) - This project assembles a lot of NLP operations needed for finance domain. -- [Earning call transcripts](https://github.com/lin882/WebAnalyticsProject) - Correlation between mutual fund investment decision and earning call transcripts. -- [Buzzwords](https://github.com/swap9047/Cutting-Edge-Technologies-Effect-on-S-P500-Companies-Performance-and-Mutual-Funds) - Return performance and mutual fund selection. -- [Fund classification](https://github.com/frechfrechfrech/Mutual-Fund-Market-Clusters/blob/master/Initial%20Data%20Exploration.ipynb) - Fund classification using text mining and NLP. -- [NLP Event](https://github.com/yuriak/DLQuant) - Applying Deep Learning and NLP in Quantitative Trading. -- [Financial Sentiment Analysis](https://github.com/EricHe98/Financial-Statements-Text-Analysis) - Sentiment, distance and proportion analysis for trading signals. -- [Financial Statement Sentiment](https://github.com/MAydogdu/TextualAnalysis) - Extracting sentiment from financial statements using neural networks. -- [Extensive NLP](https://github.com/TiesdeKok/Python_NLP_Tutorial/blob/master/NLP_Notebook.ipynb) - Comprehensive NLP techniques for accounting research. -- [Accounting Anomalies](https://github.com/GitiHubi/deepAI/blob/master/GTC_2018_Lab-solutions.ipynb) - Using deep-learning frameworks to identify accounting anomalies. - + + # Other Assets ## Derivatives and Hedging ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/derivatives_and_hedging)) -- [Options](https://github.com/QuantConnect/Tutorials/tree/master/06%20Introduction%20to%20Options%5B%5D) - Introduction to options. -- [Derivative Markets](https://github.com/broughtj/Fin6470/tree/master/Notebooks) - The economics of futures, futures, options, and swaps. -- [Black Scholes](https://github.com/irajwani/numerical_methods_python/blob/master/black_scholes.ipynb) - Options pricing. -- [Computational Derivatives](https://github.com/chenbowen184/Computational_Finance) - Projects focusing on investigating simulations and computational techniques applied in finance. -- [Reinforcement Learning](https://github.com/FinTechies/HedgingRL) - Hedging portfolios with reinforcement learning. -- [Delta Hedging](https://github.com/RobinsonGarcia/delta-hedging) - Advanced derivatives. -- [Options Risk Measures](https://github.com/wanglouis49/risk_estimation) - Efficient financial risk estimation via computer experiment design (regression + variance-reduced sampling). -- [Derivatives Python](https://github.com/yhilpisch/dawp/tree/master/python36) - Derivative analytics with Python. -- [Volatility and Variance Derivatives](https://github.com/yhilpisch/lvvd/tree/master/lvvd) - Volatility derivatives analytics. -- [Options](https://github.com/PHBS/2018.M1.ASP/tree/master/py) - Black Scholes and Copula. -- [Option Strategies](https://github.com/rstreppa/valuation-OptionStrategies) - Valuation of Vanilla and Exotic option strategies (Butterfly, Risk Reversal etc.) with widget animations. -- [Derman](https://github.com/rstreppa/valuation-convertibles-Goldman1994/blob/master/ConvertibleBond_Goldman1994_Derman.ipynb) - Binomial tree for American call. -- [Hull White](https://github.com/rstreppa/valuation-callables-HullWhite/blob/master/CallableBond_HullWhite.ipynb) - Callable Bond, Hull White. + + ## Fixed Income ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/fixed_income)) -- [Vasicek](https://github.com/RobinsonGarcia/fixed-income/blob/master/2.0%20Vasicek%20-%20example.ipynb) - Bootstrapping and interpolation. -- [Binomial Tree](https://github.com/hy-lei/math-finance-exercise) - Utility functions in fixed income securities. -- [Corporate Bonds](https://github.com/ishank011/gs-quantify-bond-prediction) - Predicting the buying and selling volume of the corporate bonds. + + ## Alternative Finance ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/alternative_finance)) -- [Kiva Crowdfunding](https://github.com/CJL89/Kiva-Crowdfunding/blob/master/Kiva%20Crowdfunding.ipynb) - Exploratory data analysis. -- [Venture Capital](https://github.com/julian-chan/etothex) - Insight into a new founder to make data-driven investment decisions. -- [Venture Capital NN](https://github.com/tr7200/National-Culture-and-Venture-Capital-Monitoring) - Cox-PH neural network predictions for VC/innovations finance research. -- [Private Equity](https://github.com/TheVinhLuong102/ChicagoBooth-EntrepreneurialFinancePrivateEquity/blob/master/RightNow%20Technologies/RightNow%20Technologies.ipynb) - Valuation models. -- [VC OLS](https://github.com/fionawhitefield/venture-capital-ols/blob/master/sec_project.ipynb) - VC regression. -- [Watch Valuation](https://github.com/alporter08/Luxury-Watch-Valuation/blob/master/Luxury-Watch-Valuation.ipynb) - Analysis of luxury watch data to classify whether a certain model is likely to be over- or undervalued. -- [Art Valuation](https://github.com/ahmedhosny/theGreenCanvas/blob/gh-pages/ImageProcessing1210.ipynb) - Art evaluation analytics. -- [Blockchain](https://github.com/nud3l/dInvest) - Repository for distributed autonomous investment banking. + + # Extended Research ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/extended_research)) -- [HFT](https://github.com/rorysroes/SGX-Full-OrderBook-Tick-Data-Trading-Strategy) - High frequency trading. -- [Deep Portfolio](https://github.com/DLColumbia/DL_forFinance) - Deep learning for finance Predict volume of bonds. -- [Mathematical Finance](https://github.com/Auquan/Tutorials) - Notebooks for math and financial tutorials. -- [NLP Finance Papers](https://github.com/chenbowen184/Research_Documents_Curation_with_NLP) - Curating quantitative finance papers using machine learning. -- [Simulation](https://github.com/chenbowen184/Computational_Finance) - Investigating simulations as part of computational finance. -- [Market Crash Prediction](https://github.com/sarachmax/MarketCrashes_Prediction/blob/master/LPPL_Comparasion.ipynb) - Predicting market crashes using an LPPL model. -- [Commodity](https://github.com/felipessalvatore/fin2vec/blob/master/src/Commodity2BR.ipynb) - Commodity influence over Brazilian stocks. -- [Finance Graph Theory](https://github.com/AvijitGhosh82/Finance_Graph_Theory) - Modelling Contentedness of Firms in Financial Markets with Heterogeneous Agents. -- [Real Estate Property Fraud](https://github.com/aviroop1/Real_Estate_Property_Fraud) - Unsupervised fraud detection model that can identify likely candidates of fraud. -- [Behavioural Economics](https://github.com/pcmichaud/notebooks) - Behavioural Economics and Finance Python Notebooks. -- [Bayesian Finance](https://github.com/marketneutral/alphatools/blob/master/notebooks/pymc3-minimal.ipynb) - Notebook PyMC3 implementation. -- [Bayesian Finance I](https://github.com/AlexIoannides/pymc-stochastic-process/blob/master/bayes_stoch_proc_calib.ipynb) - Stochastic Process Calibration using Bayesian Inference & Probabilistic Programs. -- [Currency PCA](https://github.com/shanemulqueen/python-finance-pca/blob/master/FX_spots_w_PCA.ipynb) - Forex spots PCA. -- [Backtests](https://github.com/AlgoTraders/stock-analysis-engine) - Trading data and algorithms. -- [High Frequency](https://github.com/cswaney/prickle) - A Python toolkit for high-frequency trade research. -- [Financial Economics](https://github.com/rsvp/fecon235/tree/master/nb) - Financial Economics Models. -- [Critical Transitions](https://github.com/ryanholbrook/critical-transitions) - Detecting critical transitions in financial networks with topological data analysis. -- [Economic Foundations](https://github.com/SeanMcOwen/FinanceAndPython.com-EconomicFoundations) - Basic economic models. -- [Corporate Finance](https://github.com/SeanMcOwen/FinanceAndPython.com-CorporateFinance) - Basic corporate finance. -- [Applied Corporate Finance](https://github.com/chenbowen184/Data_Science_in_Applied_Corporate_Finance) - Studies the empirical behaviours in stock market. -- [M&A](https://github.com/atulram/Finance-and-Stocks) - Mergers and Acquisitions. -- [Life-cycle](https://github.com/atulram/Finance-and-Stocks/blob/master/CompanyLifeCycle.ipynb) - Company life cycle. -- [Computational Finance](https://github.com/lnsongxf/Applied_Computational_Economics_and_Finance) - Applied Computational Economics and Finance. -- [Liquidity and Momentum](https://github.com/mrefermat/quant_finance) - Various factors and portfolio constructions. - + + # Courses ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/courses)) -- [Mathematical Finance](https://github.com/yadongli/nyumath2048) - NYU Math-GA 2048: Scientific Computing in Finance. -- [Algo Trading](https://github.com/JCreeks/Machine-Learning-in-Finance/tree/master/0_Intro_to_Algo_Trading) - Intro to algo trading. -- [Python for Finance](https://github.com/siaen/python_finance_course) - CEU python for finance course material. -- [Handson Python for Finance](https://github.com/PacktPublishing/Hands-on-Python-for-Finance) - Hands-on Python for Finance published by Packt. -- [Machine Learning for Trading](https://github.com/stefan-jansen/machine-learning-for-trading) - Notebooks, resources and references accompanying the book Machine Learning for Algorithmic Trading. -- [ML Specialisation](https://github.com/Ahmed0028/Machine-Learning-and-Reinforcement-Learning-in-Finance-Specialization) - Machine Learning in Finance. -- [Risk Management](https://github.com/andrey-lukyanov/Risk-Management) - Finance risk engagement course resources. -- [Basic Investments](https://github.com/SeanMcOwen/FinanceAndPython.com-Investments) - Basic investment tools in python. -- [Basic Derivatives](https://github.com/SeanMcOwen/FinanceAndPython.com-Derivatives) - Basic forward contracts and hedging. -- [Basic Finance](https://github.com/SeanMcOwen/FinanceAndPython.com-BasicFinance) - Source code notebooks basic finance applications. - + + # Data ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/data)) -- [Capital Markets Data](https://www.capitalmarketsdata.com/) (and [chartpack](https://1drv.ms/b/s!AjyO2n0JmhakhJIy-jZXLsLGxOw38Q?e=uOuPKY)) -- [Employee Count SEC Filings](https://github.com/healthgradient/sec_employee_information_extraction) -- [SEC Parsing](https://github.com/healthgradient/sec-doc-info-extraction/blob/master/classify_sections_containing_relevant_information.ipynb) -- [Open Edgar](https://github.com/LexPredict/openedgar) -- [EDGAR](https://github.com/TiesdeKok/UW_Python_Camp/blob/master/Materials/Session_5/EDGAR_walkthrough.ipynb) -- [IRS](http://social-metrics.org/sox/) -- [Rating Industries](http://www.ratingshistory.info/) -- [Web Scraping (FirmAI)](https://github.com/firmai/business-machine-learning/blob/master/www.firmai.org/data) -- [Financial Corporate](http://raw.rutgers.edu/Corporate%20Financial%20Data.html) -- [Non-financial Corporate](http://raw.rutgers.edu/Non-Financial%20Corporate%20Data.html) -- [http://finance.yahoo.com/](http://finance.yahoo.com/) -- [https://fred.stlouisfed.org/](https://fred.stlouisfed.org/) -- [https://stooq.com](https://stooq.com) -- [https://github.com/timestocome/StockMarketData](https://github.com/timestocome/StockMarketData) - + + # Colleges, Centers and Departments ([Wiki](https://github.com/firmai/financial-machine-learning/wiki/colleges_centers_and_departments)) -- [NYU FRE](https://engineering.nyu.edu/academics/departments/finance-and-risk-engineering) - Finance and Risk Engineering (NYU Tandon) -- [Cornell University](https://www.cornell.edu/) -- [NYU Courant](https://cims.nyu.edu/) - Courant Institute of Mathematical Sciences, New York University -- [Oxford Man](https://www.oxford-man.ox.ac.uk/) - Oxford-Man Institute of Quantitative Finance -- [Stanford Advanced Financial Technologies](https://fintech.stanford.edu/) - Stanford Advanced Financial Technologies Laboratory -- [Berkeley Lab CIFT](https://cs.lbl.gov/news-media/news/news-archive/2010/berkeley-lab-launches-new-center-for-innovative-financial-technology/) - + + diff --git a/wiki_gen.py b/wiki_gen.py index c56768e..08f0f60 100644 --- a/wiki_gen.py +++ b/wiki_gen.py @@ -71,7 +71,7 @@ def generate_wiki_per_category(output_path, update_readme: bool = True): table_str = formatted_df.iloc[:10].to_markdown(index=False) new_str = f" \n" new_str += table_str - new_str += f" \n" + new_str += f"" search_start = re.escape(''.format(clean_category_name)) search_end = re.escape(''.format(clean_category_name))