requirements.txt

# Requirements for runtime.
pydantic-settings

python-Levenshtein
scikit-learn
filelock
loguru
psutil
fire
fuzzywuzzy
openai
litellm>=1.86.2  # to support `from litellm import get_valid_models`
aiohttp>=3.14.0  # CVE-2026-22815, CVE-2026-34515, CVE-2026-34516, CVE-2026-34525; >=3.13.4 due to litellm==1.83.14 exact pin
azure.identity
pyarrow
rich
tqdm
typer

numpy # we use numpy as default data format. So we have to install numpy
pandas # we use pandas as default data format. So we have to install pandas
pandarallel # parallelize pandas
matplotlib
langchain
langchain-community
tiktoken
pymupdf  # Extract shotsreens from pdf

# PDF related
pypdf
azure-ai-formrecognizer

# factor implementations
tables

# CI Fix Tool
tree-sitter-python
tree-sitter

python-dotenv>=1.2.2  # CVE: symlink following allows arbitrary file overwrite

# infrastructure related.
docker

# crawler related
webdriver-manager

# demo related
streamlit>=1.58.0  # to support  input_c.text_area(..., height="content", ...)
plotly
st-theme
randomname
flask
flask-cors
networkx

# kaggle crawler
selenium
kaggle
nbformat # also used for notebook conversion

# tool
setuptools-scm
seaborn
azure.ai.inference

# data folder desc
humanize
genson

# mlflow
mlflow
azureml-mlflow
types-pytz

# Agent
pydantic-ai-slim[mcp,openai,prefect]
nest-asyncio

# visualize SFT train
tensorboard     # tensorboard --logdir git_ignore_folder/RD-Agent_workspace
prefect

# HuggingFace datasets
datasets

# DuckDuckGo search
duckduckgo-search

# Testing
pytest
pytest-cov

# Parameter Optimization
optuna>=3.6.2

# News & Data (Polymarket, ForexFactory, CryptoPanic)
beautifulsoup4>=4.14.3

# ML Training Pipeline
lightgbm>=3.3.5
scipy>=1.15.3

# RL Trading (optional - system works without these)
# Install for full RL training: pip install stable-baselines3[extra] gymnasium
# Without these, RL trading uses simple momentum fallback
# stable-baselines3[extra]>=2.0.0
# gymnasium>=0.29.0
add code security check CI and dependbot (#3 ) 2024-05-09 10:35:54 +08:00			`# Requirements for runtime.`
			`pydantic-settings`

			`python-Levenshtein`
Pdf2 model task (#33 ) 2024-06-30 23:31:00 +08:00			`scikit-learn`
			`filelock`
Move package (#115 ) 2024-07-25 18:36:35 +08:00			`loguru`
perf(kronos): batch GPU inference via predict_batch — 75x faster 2026-04-18 12:04:44 +02:00			`psutil`
Move package (#115 ) 2024-07-25 18:36:35 +08:00			`fire`
			`fuzzywuzzy`
			`openai`
chore(deps): Update litellm requirement from >=1.83.14 to >=1.86.2 (#63 ) 2026-06-04 14:49:40 +02:00			litellm>=1.86.2 # to support `from litellm import get_valid_models`
chore(deps): Update aiohttp requirement from >=3.13.4 to >=3.14.0 (#65 ) 2026-06-04 15:00:15 +02:00			`aiohttp>=3.14.0 # CVE-2026-22815, CVE-2026-34515, CVE-2026-34516, CVE-2026-34525; >=3.13.4 due to litellm==1.83.14 exact pin`
feat: refactor for general data science (#498 ) 2025-01-17 22:53:05 +08:00			`azure.identity`
fix: fix combined_factors_df.pkl not loading in docker (#697 ) 2025-03-18 18:36:21 +08:00			`pyarrow`
fix: fix kaggle templates path error (#747 ) 2025-04-04 11:20:33 +08:00			`rich`
feat: pull image with progress (#777 ) 2025-04-10 12:00:27 +08:00			`tqdm`
chore: add a rdagent server with UI & logger storage refinement(#553 ) 2025-06-06 18:52:19 +08:00			`typer`
Move package (#115 ) 2024-07-25 18:36:35 +08:00
			`numpy # we use numpy as default data format. So we have to install numpy`
			`pandas # we use pandas as default data format. So we have to install pandas`
feat: filter feature which is high correlation to former implemented features (#145 ) 2024-08-02 14:41:17 +08:00			`pandarallel # parallelize pandas`
Move package (#115 ) 2024-07-25 18:36:35 +08:00			`matplotlib`
			`langchain`
fix: package dependency. (#234 ) 2024-08-31 20:35:04 +08:00			`langchain-community`
Move package (#115 ) 2024-07-25 18:36:35 +08:00			`tiktoken`
Update fitz requirements 2024-07-29 13:04:33 +08:00			`pymupdf # Extract shotsreens from pdf`
Move package (#115 ) 2024-07-25 18:36:35 +08:00
			`# PDF related`
			`pypdf`
			`azure-ai-formrecognizer`

Pdf2 model task (#33 ) 2024-06-30 23:31:00 +08:00			`# factor implementations`
			`tables`

CI tool refinement (#13 ) 2024-06-07 00:23:11 +08:00			`# CI Fix Tool`
			`tree-sitter-python`
			`tree-sitter`
New Framework for idea proposal and implementation on RD-Agent (#34 ) 2024-06-28 11:45:23 +08:00
fix(deps): bump python-dotenv to >=1.2.2 (CVE symlink overwrite) 2026-04-21 22:38:18 +02:00			`python-dotenv>=1.2.2 # CVE: symlink following allows arbitrary file overwrite`
Initial framework for docker env (#40 ) 2024-07-03 04:54:24 +08:00
			`# infrastructure related.`
			`docker`
Move package (#115 ) 2024-07-25 18:36:35 +08:00
fix: replace hardcoded ChromeDriver path with webdriver-manager (#1271 ) 2025-10-20 22:15:44 +08:00			`# crawler related`
			`webdriver-manager`

feat: streamlit webapp demo for different scenarios (#135 ) 2024-08-02 18:37:15 +08:00			`# demo related`
chore(deps): Update streamlit requirement from >=1.57.0 to >=1.58.0 (#64 ) 2026-06-04 14:54:55 +02:00			`streamlit>=1.58.0 # to support input_c.text_area(..., height="content", ...)`
feat: streamlit webapp demo for different scenarios (#135 ) 2024-08-02 18:37:15 +08:00			`plotly`
feat: Demo display effect and usage (#162 ) 2024-08-09 20:40:16 +08:00			`st-theme`
chore: add a rdagent server with UI & logger storage refinement(#553 ) 2025-06-06 18:52:19 +08:00			`randomname`
			`flask`
			`flask-cors`
add dependence (#1029 ) 2025-07-08 11:12:44 +08:00			`networkx`
feat: Added loop code for Kaggle scene. (#211 ) 2024-08-19 10:58:28 +08:00
			`# kaggle crawler`
			`selenium`
fix: package dependency. (#234 ) 2024-08-31 20:35:04 +08:00			`kaggle`
feat: create Jupyter notebook pipeline file based on main.py file (#1134 ) 2025-08-05 03:06:57 -07:00			`nbformat # also used for notebook conversion`
test: add test import (#242 ) 2024-09-06 17:18:52 +08:00
feat: Kaggle loop update (Feature & Model) (#241 ) 2024-09-11 15:26:52 +08:00			`# tool`
feat(backend): integrate LiteLLM API Backend (#564 ) 2025-02-13 15:16:18 +08:00			`setuptools-scm`
test: add test import (#242 ) 2024-09-06 17:18:52 +08:00			`seaborn`
feat: integrate azure deepseek r1 (#591 ) 2025-02-13 22:20:17 +08:00			`azure.ai.inference`
feat: add describe_data_folder_v2 (#738 ) 2025-04-02 19:38:43 +08:00
			`# data folder desc`
			`humanize`
			`genson`
feat: add mlflow logger in RD loop to log (#815 ) 2025-04-21 11:23:40 -06:00
			`# mlflow`
			`mlflow`
			`azureml-mlflow`
refactor: add custom data setting for data science scene (#967 ) 2025-06-17 14:55:11 +08:00			`types-pytz`
feat: init pydantic ai agent & context 7 mcp (#1240 ) 2025-09-13 10:25:02 +08:00
			`# Agent`
feat(mcp): cache with one-click toggle (#1269 ) 2025-10-17 16:30:59 +08:00			`pydantic-ai-slim[mcp,openai,prefect]`
feat: init pydantic ai agent & context 7 mcp (#1240 ) 2025-09-13 10:25:02 +08:00			`nest-asyncio`
fix: refine task scheduling logic in MultiProcessEvolvingStrategy for… (#1275 ) 2025-10-23 17:09:18 +08:00
feat: add LLM-finetune scenario (#1314 ) 2026-03-02 19:04:10 +08:00			`# visualize SFT train`
			`tensorboard # tensorboard --logdir git_ignore_folder/RD-Agent_workspace`
			`prefect`

			`# HuggingFace datasets`
feat(rl): add AutoRL-Bench framework and benchmark integrations (#1348 ) 2026-03-17 15:49:30 +08:00			`datasets`

			`# DuckDuckGo search`
test: Add backtesting tests with 98.77% coverage 2026-04-02 18:56:12 +02:00			`duckduckgo-search`

			`# Testing`
			`pytest`
feat: Full system integration - RL + Protections + Backtesting + CLI 2026-04-03 13:53:32 +02:00			`pytest-cov`

feat: Add P5 ML Training Pipeline with LightGBM and 46 tests 2026-04-09 09:46:56 +02:00			`# Parameter Optimization`
chore(deps): Update optuna requirement from >=3.5.0 to >=3.6.2 (#52 ) 2026-05-04 22:31:44 +02:00			`optuna>=3.6.2`
feat: Add P5 ML Training Pipeline with LightGBM and 46 tests 2026-04-09 09:46:56 +02:00
feat: Data Loader module with tests (P0 complete) 2026-04-09 08:15:49 +02:00			`# News & Data (Polymarket, ForexFactory, CryptoPanic)`
chore(deps): Update beautifulsoup4 requirement from >=4.12.0 to >=4.14.3 (#50 ) 2026-05-04 22:31:48 +02:00			`beautifulsoup4>=4.14.3`
feat: Data Loader module with tests (P0 complete) 2026-04-09 08:15:49 +02:00
feat: Add P5 ML Training Pipeline with LightGBM and 46 tests 2026-04-09 09:46:56 +02:00			`# ML Training Pipeline`
chore(deps): Update lightgbm requirement from >=3.3.0 to >=3.3.5 (#34 ) 2026-04-27 16:18:37 +02:00			`lightgbm>=3.3.5`
chore(deps): bump aiohttp >=3.13.5 and scipy >=1.15.3 2026-04-27 16:19:31 +02:00			`scipy>=1.15.3`
feat: Add P5 ML Training Pipeline with LightGBM and 46 tests 2026-04-09 09:46:56 +02:00
feat: Full system integration - RL + Protections + Backtesting + CLI 2026-04-03 13:53:32 +02:00			`# RL Trading (optional - system works without these)`
			`# Install for full RL training: pip install stable-baselines3[extra] gymnasium`
			`# Without these, RL trading uses simple momentum fallback`
			`# stable-baselines3[extra]>=2.0.0`
			`# gymnasium>=0.29.0`