mirror of
https://github.com/NicolasBohn/NexQuant.git
synced 2026-07-27 23:47:46 +00:00
A draft of the "Quick Start" section for your README. Enable logging of backtesting in Qlib and store rich-text descriptions in Trace. Support one-step debugging for factor extraction. (#105)
- A draft of the "Quick Start" section for your README. - Enable logging of backtesting in Qlib and store rich-text descriptions in Trace. - Support one-step debugging for factor extraction.
This commit is contained in:
WinstonLiyt
@@ -1,12 +1,12 @@
|
||||
TODO: Add badges.
|
||||
|
||||
# News
|
||||
# 📰 News
|
||||
| 🗞️News | 📝Description |
|
||||
| -- | ------ |
|
||||
| First release | RDAgent are release on Github |
|
||||
|
||||
|
||||
# Introduction
|
||||
# 🌟 Introduction
|
||||
|
||||
|
||||
|
||||
@@ -31,15 +31,62 @@ We have a quick 🎥demo for one use case of RDAgent.
|
||||
# ⚡Quick start
|
||||
You can try our demo by running the following command:
|
||||
|
||||
```bash
|
||||
# TODO:
|
||||
# prepare environment
|
||||
# installation
|
||||
# App entrance
|
||||
```
|
||||
### 🐍 Create a Conda Environment
|
||||
- Create a new conda environment with Python (3.10 and 3.11 are well tested in our CI):
|
||||
```sh
|
||||
conda create -n rdagent python=3.10
|
||||
```
|
||||
- Activate the environment:
|
||||
```sh
|
||||
conda activate rdagent
|
||||
```
|
||||
|
||||
### 🛠️ Run Make Files
|
||||
TODO: `pip install rdagent` in the future.
|
||||
|
||||
- **Navigate to the directory containing the MakeFile** and set up the development environment:
|
||||
```sh
|
||||
make dev
|
||||
```
|
||||
|
||||
### 📦 Install Pytorch
|
||||
TODO: use docker in quick start intead.
|
||||
|
||||
- Install Pytorch and related libraries:
|
||||
```sh
|
||||
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
pip3 install torch_geometric
|
||||
```
|
||||
|
||||
### ⚙️ Environment Configuration
|
||||
- Place the `.env` file in the same directory as the `.env.example` file.
|
||||
- TOOD: please refer to ... for the detailed explanation of the `.env`
|
||||
- TODO: simplify `.env.example` only keep OpenAI or Azure Azure OpenAI
|
||||
- Export each variable in the `.env` file:
|
||||
```sh
|
||||
export $(grep -v '^#' .env | xargs)
|
||||
```
|
||||
### 🚀 Run the Application
|
||||
TODO: run the front-page demo.
|
||||
|
||||
The [🎥demo]() is implemented by the above commands.
|
||||
|
||||
- Run the factor extraction and implementation application based on financial reports:
|
||||
```sh
|
||||
python rdagent/app/qlib_rd_loop/factor_from_report_sh.py
|
||||
```
|
||||
|
||||
- Run the self-loop factor extraction and implementation application:
|
||||
```sh
|
||||
python rdagent/app/qlib_rd_loop/factor.py
|
||||
```
|
||||
|
||||
- Run the self-loop model extraction and implementation application:
|
||||
```sh
|
||||
python rdagent/app/qlib_rd_loop/model.py
|
||||
```
|
||||
|
||||
|
||||
# Scenarios
|
||||
|
||||
We have applied RD-Agent to multiple valuable data-driven industrial scenarios..
|
||||
@@ -60,9 +107,9 @@ Here is our supported scenarios
|
||||
|
||||
| Scenario/Target | Model Implementation | Data Building |
|
||||
| -- | -- | -- |
|
||||
| 💹Finance | Iteratively Proposing Ideas & Evolving | - Auto reports reading & implementation <br/> - Iteratively Proposing Ideas & Evolving |
|
||||
| 🩺Medical | Iteratively Proposing Ideas & Evolving | - |
|
||||
| General | Auto paper reading & implementation | - |
|
||||
| 💹 Finance | Iteratively Proposing Ideas & Evolving | - Auto reports reading & implementation <br/> - Iteratively Proposing Ideas & Evolving |
|
||||
| 🩺 Medical | Iteratively Proposing Ideas & Evolving | - |
|
||||
| 🏭 General | Auto paper reading & implementation | - |
|
||||
|
||||
Different scenarios vary in entrance and configuration. Please check the detailed setup tutorial in the scenarios documents.
|
||||
|
||||
|
||||
@@ -13,3 +13,27 @@ Configuration
|
||||
=============
|
||||
|
||||
Quick configuration
|
||||
|
||||
|
||||
|
||||
|
||||
Azure OpenAI
|
||||
------------
|
||||
|
||||
|
||||
|
||||
USE_AZURE_TOKEN_PROVIDER
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
### ☁️ Azure Configuration
|
||||
- Install Azure CLI:
|
||||
```sh
|
||||
curl -L https://aka.ms/InstallAzureCli | bash
|
||||
```
|
||||
- Log in to Azure:
|
||||
```sh
|
||||
az login --use-device-code
|
||||
```
|
||||
|
||||
- `exit` and re-login to your environment (this step may not be necessary).
|
||||
|
||||
|
||||
@@ -5,29 +5,107 @@ Scenarios and Quick Start
|
||||
Scenario lists
|
||||
=========================
|
||||
|
||||
TODO: Copy the content in the README.md
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Scenario/Target
|
||||
- Model Implementation
|
||||
- Data Building
|
||||
* - 💹 Finance
|
||||
- Iteratively Proposing Ideas & Evolving
|
||||
- Auto reports reading & implementation
|
||||
Iteratively Proposing Ideas & Evolving
|
||||
* - 🩺 Medical
|
||||
- Iteratively Proposing Ideas & Evolving
|
||||
-
|
||||
* - 🏭 General
|
||||
- Auto paper reading & implementation
|
||||
-
|
||||
|
||||
Scnarios' demo & quick start
|
||||
=========================
|
||||
|
||||
Scen1
|
||||
-----
|
||||
🤖 Knowledge-Based Hypothesis Generation and Iteration
|
||||
|
||||
Scen1 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, our model autonomously generates and tests hypotheses using a knowledge base. The process involves:
|
||||
|
||||
- **🔍 Hypothesis Generation**: The model proposes new hypotheses.
|
||||
- **📝 Factor Creation**: Write and define new factors.
|
||||
- **✅ Factor Validation**: Validate the factors quantitatively.
|
||||
- **📈 Backtesting with Qlib**:
|
||||
|
||||
- **Dataset**: CSI300
|
||||
- **Model**: LGBModel
|
||||
- **Factors**: Alpha158 +
|
||||
- **Data Split**:
|
||||
|
||||
- **Train**: 2008-01-01 to 2014-12-31
|
||||
- **Valid**: 2015-01-01 to 2016-12-31
|
||||
- **Test**: 2017-01-01 to 2020-08-01
|
||||
- **🔄 Feedback Analysis**: Analyze backtest results.
|
||||
- **🔧 Hypothesis Refinement**: Refine hypotheses based on feedback and repeat.
|
||||
|
||||
Scen1 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen1 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the factor extraction process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/factor.py
|
||||
|
||||
|
||||
Usage of modules
|
||||
================
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
|
||||
|
||||
Scen2:
|
||||
-----
|
||||
📄 Research Report-Based Factor Extraction
|
||||
|
||||
Scen2 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, factors and hypotheses are extracted from research reports. The process includes:
|
||||
|
||||
- **🔍 Factor Extraction**: Extract relevant factors from research reports.
|
||||
- **📝 Factor Creation**: Define these extracted factors.
|
||||
- **✅ Factor Validation**: Validate the extracted factors.
|
||||
- **📈 Backtesting with Qlib**:
|
||||
|
||||
- **Dataset**: CSI300
|
||||
- **Model**: LGBModel
|
||||
- **Factors**: Alpha158 +
|
||||
- **Data Split**:
|
||||
|
||||
- **Train**: 2008-01-01 to 2014-12-31
|
||||
- **Valid**: 2015-01-01 to 2016-12-31
|
||||
- **Test**: 2017-01-01 to 2020-08-01
|
||||
- **🔄 Feedback Analysis**: Analyze backtest results.
|
||||
- **🔧 Hypothesis Refinement**: Refine hypotheses based on feedback and continue the cycle.
|
||||
|
||||
Scen2 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen2 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the factor extraction process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/factor_from_report_sh.py
|
||||
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
@@ -1,39 +1,43 @@
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic_settings import BaseSettings
|
||||
from rdagent.components.workflow.conf import BasePropSetting
|
||||
|
||||
|
||||
class PropSetting(BaseSettings):
|
||||
class ModelBasePropSetting(BasePropSetting):
|
||||
class Config:
|
||||
env_prefix = "QLIB_" # Use MODEL_CODER_ as prefix for environment variables
|
||||
env_prefix = "QLIB_MODEL_" # Use MODEL_CODER_ as prefix for environment variables
|
||||
protected_namespaces = () # Add 'model_' to the protected namespaces
|
||||
|
||||
factor_scen: str = "rdagent.scenarios.qlib.experiment.factor_experiment.QlibFactorScenario"
|
||||
factor_hypothesis_gen: str = "rdagent.scenarios.qlib.proposal.factor_proposal.QlibFactorHypothesisGen"
|
||||
factor_hypothesis2experiment: str = (
|
||||
"rdagent.scenarios.qlib.proposal.factor_proposal.QlibFactorHypothesis2Experiment"
|
||||
)
|
||||
factor_coder: str = "rdagent.scenarios.qlib.developer.factor_coder.QlibFactorCoSTEER"
|
||||
factor_runner: str = "rdagent.scenarios.qlib.developer.factor_runner.QlibFactorRunner"
|
||||
factor_summarizer: str = "rdagent.scenarios.qlib.developer.feedback.QlibFactorHypothesisExperiment2Feedback"
|
||||
|
||||
# TODO: model part is not finished yet
|
||||
model_scen: str = "rdagent.scenarios.qlib.experiment.model_experiment.QlibModelScenario"
|
||||
model_hypothesis_gen: str = "rdagent.scenarios.qlib.proposal.model_proposal.QlibModelHypothesisGen"
|
||||
model_hypothesis2experiment: str = "rdagent.scenarios.qlib.proposal.model_proposal.QlibModelHypothesis2Experiment"
|
||||
model_coder: str = "rdagent.scenarios.qlib.developer.model_coder.QlibModelCoSTEER"
|
||||
model_runner: str = "rdagent.scenarios.qlib.developer.model_runner.QlibModelRunner"
|
||||
model_summarizer: str = "rdagent.scenarios.qlib.developer.feedback.QlibModelHypothesisExperiment2Feedback"
|
||||
scen: str = "rdagent.scenarios.qlib.experiment.model_experiment.QlibModelScenario"
|
||||
hypothesis_gen: str = "rdagent.scenarios.qlib.proposal.model_proposal.QlibModelHypothesisGen"
|
||||
hypothesis2experiment: str = "rdagent.scenarios.qlib.proposal.model_proposal.QlibModelHypothesis2Experiment"
|
||||
coder: str = "rdagent.scenarios.qlib.developer.model_coder.QlibModelCoSTEER"
|
||||
runner: str = "rdagent.scenarios.qlib.developer.model_runner.QlibModelRunner"
|
||||
summarizer: str = "rdagent.scenarios.qlib.developer.feedback.QlibModelHypothesisExperiment2Feedback"
|
||||
|
||||
evolving_n: int = 10
|
||||
|
||||
py_bin: str = "/usr/bin/python"
|
||||
local_qlib_folder: Path = Path("/home/rdagent/qlib")
|
||||
|
||||
class FactorBasePropSetting(BasePropSetting):
|
||||
class Config:
|
||||
env_prefix = "QLIB_FACTOR_" # Use MODEL_CODER_ as prefix for environment variables
|
||||
protected_namespaces = () # Add 'model_' to the protected namespaces
|
||||
|
||||
# 1) override base settings
|
||||
# TODO: model part is not finished yet
|
||||
scen: str = "rdagent.scenarios.qlib.experiment.factor_experiment.QlibFactorScenario"
|
||||
hypothesis_gen: str = "rdagent.scenarios.qlib.proposal.factor_proposal.QlibFactorHypothesisGen"
|
||||
hypothesis2experiment: str = "rdagent.scenarios.qlib.proposal.factor_proposal.QlibFactorHypothesis2Experiment"
|
||||
coder: str = "rdagent.scenarios.qlib.developer.factor_coder.QlibFactorCoSTEER"
|
||||
runner: str = "rdagent.scenarios.qlib.developer.factor_runner.QlibFactorRunner"
|
||||
summarizer: str = "rdagent.scenarios.qlib.developer.feedback.QlibFactorHypothesisExperiment2Feedback"
|
||||
|
||||
evolving_n: int = 10
|
||||
|
||||
# 2) sub task specific:
|
||||
origin_report_path: str = "data/report_origin"
|
||||
local_report_path: str = "data/report"
|
||||
report_result_json_file_path: str = "git_ignore_folder/res_dict.json"
|
||||
progress_file_path: str = "git_ignore_folder/progress.pkl"
|
||||
|
||||
|
||||
PROP_SETTING = PropSetting()
|
||||
FACTOR_PROP_SETTING = FactorBasePropSetting()
|
||||
MODEL_PROP_SETTING = ModelBasePropSetting()
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
"""
|
||||
Factor workflow with session control
|
||||
"""
|
||||
|
||||
import fire
|
||||
|
||||
from rdagent.app.qlib_rd_loop.conf import FACTOR_PROP_SETTING
|
||||
from rdagent.components.workflow.rd_loop import RDLoop
|
||||
from rdagent.core.exception import FactorEmptyError
|
||||
|
||||
|
||||
class FactorRDLoop(RDLoop):
|
||||
skip_loop_error = (FactorEmptyError,)
|
||||
|
||||
|
||||
def main(path=None, step_n=None):
|
||||
"""
|
||||
You can continue running session by
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
dotenv run -- python rdagent/app/qlib_rd_loop/factor_w_sc.py $LOG_PATH/__session__/1/0_propose --step_n 1 # `step_n` is a optional paramter
|
||||
|
||||
"""
|
||||
if path is None:
|
||||
model_loop = FactorRDLoop(FACTOR_PROP_SETTING)
|
||||
else:
|
||||
model_loop = FactorRDLoop.load(path)
|
||||
model_loop.run(step_n=step_n)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
fire.Fire(main)
|
||||
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
fire.Fire(main)
|
||||
@@ -1,70 +1,16 @@
|
||||
"""
|
||||
Model workflow with session control
|
||||
It is from `rdagent/app/qlib_rd_loop/model.py` and try to replace `rdagent/app/qlib_rd_loop/RDAgent.py`
|
||||
"""
|
||||
|
||||
import fire
|
||||
from typing import Any
|
||||
from rdagent.app.qlib_rd_loop.conf import PROP_SETTING
|
||||
from rdagent.core.developer import Developer
|
||||
|
||||
from rdagent.app.qlib_rd_loop.conf import MODEL_PROP_SETTING
|
||||
from rdagent.components.workflow.rd_loop import RDLoop
|
||||
from rdagent.core.exception import ModelEmptyError
|
||||
from rdagent.core.proposal import (
|
||||
Hypothesis2Experiment,
|
||||
HypothesisExperiment2Feedback,
|
||||
HypothesisGen,
|
||||
Trace,
|
||||
)
|
||||
from rdagent.core.scenario import Scenario
|
||||
from rdagent.core.utils import import_class
|
||||
from rdagent.log import rdagent_logger as logger
|
||||
|
||||
from rdagent.utils.workflow import LoopMeta, LoopBase
|
||||
|
||||
class ModelLoop(LoopBase, metaclass=LoopMeta):
|
||||
# TODO: supporting customized loop control like catching `ModelEmptyError`
|
||||
|
||||
def __init__(self):
|
||||
scen: Scenario = import_class(PROP_SETTING.model_scen)()
|
||||
|
||||
self.hypothesis_gen: HypothesisGen = import_class(PROP_SETTING.model_hypothesis_gen)(scen)
|
||||
|
||||
self.hypothesis2experiment: Hypothesis2Experiment = import_class(PROP_SETTING.model_hypothesis2experiment)()
|
||||
|
||||
self.qlib_model_coder: Developer = import_class(PROP_SETTING.model_coder)(scen)
|
||||
self.qlib_model_runner: Developer = import_class(PROP_SETTING.model_runner)(scen)
|
||||
|
||||
self.qlib_model_summarizer: HypothesisExperiment2Feedback = import_class(PROP_SETTING.model_summarizer)(scen)
|
||||
self.trace = Trace(scen=scen)
|
||||
super().__init__()
|
||||
|
||||
def propose(self, prev_out: dict[str, Any]):
|
||||
with logger.tag("r"): # research
|
||||
hypothesis = self.hypothesis_gen.gen(self.trace)
|
||||
logger.log_object(hypothesis, tag="hypothesis generation")
|
||||
return hypothesis
|
||||
|
||||
def exp_gen(self, prev_out: dict[str, Any]):
|
||||
with logger.tag("r"): # research
|
||||
exp = self.hypothesis2experiment.convert(prev_out["propose"], self.trace)
|
||||
logger.log_object(exp.sub_tasks, tag="experiment generation")
|
||||
return exp
|
||||
|
||||
def coding(self, prev_out: dict[str, Any]):
|
||||
with logger.tag("d"): # develop
|
||||
exp = self.qlib_model_coder.develop(prev_out["exp_gen"])
|
||||
logger.log_object(exp.sub_workspace_list, tag="model coder result")
|
||||
return exp
|
||||
|
||||
def running(self, prev_out: dict[str, Any]):
|
||||
with logger.tag("ef"): # evaluate and feedback
|
||||
exp = self.qlib_model_runner.develop(prev_out["coding"])
|
||||
logger.log_object(exp, tag="model runner result")
|
||||
return exp
|
||||
|
||||
def feedback(self, prev_out: dict[str, Any]):
|
||||
feedback = self.qlib_model_summarizer.generate_feedback(prev_out["running"], prev_out["propose"], self.trace)
|
||||
logger.log_object(feedback, tag="feedback")
|
||||
self.trace.hist.append((prev_out["propose"],prev_out["running"] , feedback))
|
||||
class ModelRDLoop(RDLoop):
|
||||
skip_loop_error = (ModelEmptyError,)
|
||||
|
||||
|
||||
def main(path=None, step_n=None):
|
||||
@@ -77,9 +23,9 @@ def main(path=None, step_n=None):
|
||||
|
||||
"""
|
||||
if path is None:
|
||||
model_loop = ModelLoop()
|
||||
model_loop = ModelRDLoop(MODEL_PROP_SETTING)
|
||||
else:
|
||||
model_loop = ModelLoop.load(path)
|
||||
model_loop = ModelRDLoop.load(path)
|
||||
model_loop.run(step_n=step_n)
|
||||
|
||||
|
||||
|
||||
@@ -60,5 +60,6 @@ class RDLoop(LoopBase, metaclass=LoopMeta):
|
||||
|
||||
def feedback(self, prev_out: dict[str, Any]):
|
||||
feedback = self.summarizer.generate_feedback(prev_out["running"], prev_out["propose"], self.trace)
|
||||
logger.log_object(feedback, tag="feedback")
|
||||
with logger.tag("ef"): # evaluate and feedback
|
||||
logger.log_object(feedback, tag="feedback")
|
||||
self.trace.hist.append((prev_out["propose"],prev_out["running"] , feedback))
|
||||
|
||||
@@ -39,6 +39,10 @@ class QlibFactorScenario(Scenario):
|
||||
@property
|
||||
def simulator(self) -> str:
|
||||
return prompt_dict["qlib_factor_simulator"]
|
||||
|
||||
@property
|
||||
def rich_style_description(self)->str:
|
||||
return "Below is QlibFactor Evolving Automatic R&D Demo."
|
||||
|
||||
def get_scenario_all_desc(self) -> str:
|
||||
return f"""Background of the scenario:
|
||||
|
||||
@@ -43,3 +43,6 @@ else:
|
||||
metrics.to_csv(output_path)
|
||||
|
||||
print(f"Output has been saved to {output_path}")
|
||||
|
||||
ret_data_frame = latest_recorder.load_object("portfolio_analysis/report_normal_1day.pkl")
|
||||
ret_data_frame.to_pickle("ret.pkl")
|
||||
@@ -31,7 +31,7 @@ class QlibFBWorkspace(FBWorkspace):
|
||||
)
|
||||
|
||||
ret_df = pd.read_pickle(self.workspace_path / "ret.pkl")
|
||||
logger.log_object(ret_df, tag="returns") # TODO: tag
|
||||
logger.log_object(ret_df, tag="Quantitative Backtesting Chart")
|
||||
|
||||
csv_path = self.workspace_path / "qlib_res.csv"
|
||||
|
||||
|
||||
@@ -59,7 +59,7 @@ factor_hypothesis_specification: |-
|
||||
- "Filter stocks by market capitalization before calculating the factors."
|
||||
|
||||
factor_hypothesis_specification: |-
|
||||
Additional Specifications:
|
||||
Specifications:
|
||||
- Hypotheses should grow and evolve based on the previous hypothesis. If there is no previous hypothesis, start with something simple.
|
||||
- Gradually build upon previous hypotheses and feedback.
|
||||
- Ensure that the hypothesis focuses on the creation and selection of factors in quantitative finance.
|
||||
@@ -94,7 +94,7 @@ factor_hypothesis_specification: |-
|
||||
- "Investigate the impact of an earnings surprise factor calculated from recent earnings announcements."
|
||||
- "Develop a composite factor integrating ESG (Environmental, Social, Governance) scores with traditional financial metrics."
|
||||
|
||||
Detailed Workflow:
|
||||
Detailed Workflow: (These are just examples for the format, do not be constrained by these examples)
|
||||
1. Initial Hypothesis:
|
||||
- Begin with a simple factor, such as "Include a momentum factor based on the last 12 months' returns."
|
||||
|
||||
@@ -110,8 +110,11 @@ factor_hypothesis_specification: |-
|
||||
5. Advanced Hypotheses:
|
||||
- Explore sophisticated combinations or new types of factors, e.g., "Develop a composite factor integrating ESG scores with traditional financial metrics."
|
||||
|
||||
Remember: If a hypothesis achieves the desired results, start a new direction while preserving the effective factors from previous hypotheses. New evaluations should combine the newly proposed factors with previously successful factors that surpassed SOTA.
|
||||
|
||||
Important Note:
|
||||
- If a hypothesis achieves the desired results, start a new direction while preserving the effective factors from previous hypotheses.
|
||||
- New evaluations should combine the newly proposed factors with previously successful factors that surpassed SOTA.
|
||||
- If the previous hypothesis factor exceeds SOTA, you can write another factor in the same direction, otherwise, explore new directions.
|
||||
- The final maintained SOTA is the continuous accumulation of factors that surpass each iteration.
|
||||
|
||||
factor_experiment_output_format: |-
|
||||
The output should follow JSON format. The schema is as follows:
|
||||
|
||||
Reference in New Issue
Block a user