From d54dc4cffb0e616268dc227b234386a495b1026d Mon Sep 17 00:00:00 2001 From: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com> Date: Thu, 8 Aug 2024 10:19:46 +0800 Subject: [PATCH] docs: Added documentation for Qlib Factor Loop, Qlib Model Loop, Data-Copilot, and Paper-Copilot scenario. (#178) * Fixed some bugs introduced during refactoring. * Added docs for four scenrios. * Update factor_from_report_w_sc.py --- docs/scens/data_agent_fin.rst | 117 ++++++++++++++++++++---- docs/scens/data_copilot_fin.rst | 123 ++++++++++++++++++++----- docs/scens/model_agent_fin.rst | 131 +++++++++++++++++++++------ docs/scens/model_copilot_general.rst | 122 ++++++++++++++++--------- docs/ui.rst | 1 + 5 files changed, 379 insertions(+), 115 deletions(-) diff --git a/docs/scens/data_agent_fin.rst b/docs/scens/data_agent_fin.rst index 2c416312..8e4a3ee5 100644 --- a/docs/scens/data_agent_fin.rst +++ b/docs/scens/data_agent_fin.rst @@ -5,11 +5,11 @@ Finance Data Agent ===================== -**Automated Quantitative Trading & Iterative Factors Evolution πŸ€–** +**πŸ€– Automated Quantitative Trading & Iterative Factors Evolution** ------------------------------------------------------------------- -Background -~~~~~~~~~~ +πŸ“– Background +~~~~~~~~~~~~~~ In the dynamic world of quantitative trading, **factors** are the secret weapons that traders use to harness market inefficiencies. These powerful toolsβ€”ranging from straightforward metrics like price-to-earnings ratios to intricate discounted cash flow modelsβ€”unlock the potential to predict stock prices with remarkable precision. @@ -18,8 +18,13 @@ By tapping into this rich vein of data, quantitative traders craft sophisticated Embrace the power of factors, and you're not just trading; you're strategically outsmarting the market. -Introduction -~~~~~~~~~~~~ +πŸŽ₯ Demo +~~~~~~~~~~ +TODO: Here should put a video of the demo. + + +🌟 Introduction +~~~~~~~~~~~~~~~~ In this scenario, our agent illustrates the iterative process of hypothesis generation, knowledge construction, and decision-making. It highlights how financial factors evolve through continuous feedback and refinement. @@ -43,7 +48,7 @@ Here's an enhanced outline of the steps: **Step 4 : Backtesting with Qlib πŸ“‰** - Integrate the full dataset into the factor implementation code and prepare the factor library. -- Perform backtesting using the Alpha158+ model in Qlib to assess the factor's effectiveness and performance. +- Conduct backtesting using the Alpha158 plus newly developed factors and LGBModel in Qlib to evaluate the new factors' effectiveness and performance. +----------------+------------+----------------+----------------------------------------------------+ | Dataset | Model | Factors | Data Split | @@ -59,27 +64,101 @@ Here's an enhanced outline of the steps: **Step 5 : Feedback Analysis πŸ”** - - Analyze backtest results. - - Incorporate feedback to refine hypotheses. + +- Analyze backtest results to assess performance. +- Incorporate feedback to refine hypotheses and improve the model. **Step 6 :Hypothesis Refinement ♻️** - - Refine hypotheses based on feedback and repeat the process. -Demo -~~~~~~~~~~ -.. TODO +- Refine hypotheses based on feedback from backtesting. +- Repeat the process to continuously improve the model. -Quick Start +⚑ Quick Start ~~~~~~~~~~~~~~~~~ -To quickly start the Automated Quantitative Trading & Iterative Factors Evolution process, run the following command in your terminal within the `rdagent` virtual environment: +You can try our demo by running the following command: -.. code-block:: sh +- 🐍 Create a Conda Environment + - Create a new conda environment with Python (3.10 and 3.11 are well tested in our CI): + + .. code-block:: sh + + conda create -n rdagent python=3.10 - python rdagent/app/qlib_rd_loop/factor_w_sc.py + - Activate the environment: + + .. code-block:: sh + + conda activate rdagent + +- πŸ› οΈ Run Make Files + - Navigate to the directory containing the MakeFile and set up the development environment: + + .. code-block:: sh + + make dev + +- πŸ“¦ Install Pytorch + - Install Pytorch and related libraries: + + .. code-block:: sh + + pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu + pip3 install torch_geometric + +- βš™οΈ Environment Configuration + - Place the `.env` file in the same directory as the `.env.example` file. + - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) + + - Export each variable in the .env file: + + .. code-block:: sh + + export $(grep -v '^#' .env | xargs) + + - If you want to change the default environment variables, you can refer to `Env Config`_ below + +- πŸš€ Run the Application + .. code-block:: sh + + python rdagent/app/qlib_rd_loop/factor_w_sc.py -Usage of modules -~~~~~~~~~~~~~~~~~ -TODO: Show some examples: +πŸ› οΈ Usage of modules +~~~~~~~~~~~~~~~~~~~~~ + +.. _Env Config: + +- **Env Config** + +The following environment variables can be set in the `.env` file to customize the application's behavior: + - **Path to the folder containing private data (default fundamental data in Qlib):** + + .. code-block:: sh + + FACTOR_CODER_DATA_FOLDER=/path/to/data/factor_implementation_source_data_all + + - **Path to the folder containing partial private data (for debugging):** + + .. code-block:: sh + + FACTOR_CODER_DATA_FOLDER_DEBUG=/path/to/data/factor_implementation_source_data_debug + + - **Maximum time (in seconds) for writing factor code:** + + .. code-block:: sh + + FACTOR_CODER_FILE_BASED_EXECUTION_TIMEOUT=300 + + - **Maximum number of factors to write in one experiment:** + + .. code-block:: sh + + FACTOR_CODER_SELECT_THRESHOLD=5 + + - **Number of developing loops for writing factors:** + + .. code-block:: sh + + FACTOR_CODER_MAX_LOOP=10 diff --git a/docs/scens/data_copilot_fin.rst b/docs/scens/data_copilot_fin.rst index 847790bb..d276ea26 100644 --- a/docs/scens/data_copilot_fin.rst +++ b/docs/scens/data_copilot_fin.rst @@ -5,11 +5,11 @@ Finance Data Copilot ===================== -**Automated Quantitative Trading & Factors Extraction from Financial ReportsπŸ“„** +**πŸ€– Automated Quantitative Trading & Factors Extraction from Financial Reports** --------------------------------------------------------------------------------- -Background -~~~~~~~~~~ +πŸ“– Background +~~~~~~~~~~~~~~ **Research reports** are treasure troves of insights, often unveiling potential **factors** that can drive successful quantitative trading strategies. Yet, with the sheer volume of reports available, extracting the most valuable insights efficiently becomes a daunting task. @@ -17,13 +17,17 @@ Furthermore, rather than hastily replicating factors from a report, it's essenti Does the factor capture the essential market dynamics? How unique is it compared to the factors already in your library? Therefore, there is an urgent need for a systematic approach to design a framework that can effectively manage this process. -This is where our R&D Agent comes into play. +This is where our RDAgent comes into play. +πŸŽ₯ Demo +~~~~~~~~~~ +TODO: Here should put a video of the demo. -Introduction -~~~~~~~~~~~~ -In this scenario, our agent demonstrates the process of extracting factors from financial research reports, implementing these factors, and analyzing their performance through Qlib backtesting. + +🌟 Introduction +~~~~~~~~~~~~~~~~ +In this scenario, RDAgent demonstrates the process of extracting factors from financial research reports, implementing these factors, and analyzing their performance through Qlib backtesting. This process continually expands and refines the factor library. Here's an enhanced outline of the steps: @@ -45,7 +49,7 @@ Here's an enhanced outline of the steps: **Step 4 : Backtesting with Qlib πŸ“‰** - Integrate the full dataset into the factor implementation code and prepare the factor library. -- Perform backtesting using the Alpha158+ model in Qlib to assess the factor's effectiveness and performance. +- Conduct backtesting using the Alpha158 plus newly developed factors and LGBModel in Qlib to evaluate the new factors' effectiveness and performance. +----------------+------------+----------------+----------------------------------------------------+ | Dataset | Model | Factors | Data Split | @@ -59,29 +63,102 @@ Here's an enhanced outline of the steps: | | | | +-----------+--------------------------+ | +----------------+------------+----------------+----------------------------------------------------+ - **Step 5 : Feedback Analysis πŸ”** - - Analyze backtest results. - - Incorporate feedback to refine hypotheses. -**Step 6 : Knowledge Base Refinement ♻️** - - Refine the knowledge base based on feedback and repeat the process. +- Analyze backtest results to assess performance. +- Incorporate feedback to refine hypotheses and improve the model. -Demo -~~~~~~~~~~ -.. TODO +**Step 6 :Hypothesis Refinement ♻️** -Scen2 Quick Start +- Refine hypotheses based on feedback from backtesting. +- Repeat the process to continuously improve the model. + +⚑ Quick Start ~~~~~~~~~~~~~~~~~ -To quickly start the factor extraction process, run the following command in your terminal within the `rdagent` virtual environment: +You can try our demo by running the following command: -.. code-block:: sh +- 🐍 Create a Conda Environment + - Create a new conda environment with Python (3.10 and 3.11 are well tested in our CI): + + .. code-block:: sh + + conda create -n rdagent python=3.10 - python rdagent/app/qlib_rd_loop/factor_from_report_w_sc.py.py + - Activate the environment: + + .. code-block:: sh + + conda activate rdagent + +- πŸ› οΈ Run Make Files + - Navigate to the directory containing the MakeFile and set up the development environment: + + .. code-block:: sh + + make dev + +- πŸ“¦ Install Pytorch + - Install Pytorch and related libraries: + + .. code-block:: sh + + pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu + pip3 install torch_geometric + +- βš™οΈ Environment Configuration + - Place the `.env` file in the same directory as the `.env.example` file. + - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) + + - Export each variable in the .env file: + + .. code-block:: sh + + export $(grep -v '^#' .env | xargs) + + - If you want to change the default environment variables, you can refer to `Env Config`_ below + +- πŸš€ Run the Application + .. code-block:: sh + + python rdagent/app/qlib_rd_loop/factor_from_report_w_sc.py -Usage of modules -~~~~~~~~~~~~~~~~~ -TODO: Show some examples: +πŸ› οΈ Usage of modules +~~~~~~~~~~~~~~~~~~~~~ + +.. _Env Config: + +- **Env Config** + +The following environment variables can be set in the `.env` file to customize the application's behavior: + - **Path to the folder containing research reports:** + + .. code-block:: sh + + QLIB_FACTOR_LOCAL_REPORT_PATH=/path/to/research/reports + + - **Path to the JSON file listing research reports for factor extraction:** + + .. code-block:: sh + + QLIB_FACTOR_REPORT_RESULT_JSON_FILE_PATH=/path/to/reports/list.json + + - **Maximum time (in seconds) for writing factor code:** + + .. code-block:: sh + + FACTOR_CODER_FILE_BASED_EXECUTION_TIMEOUT=300 + + - **Maximum number of factors to write in one experiment:** + + .. code-block:: sh + + FACTOR_CODER_SELECT_THRESHOLD=5 + + - **Number of developing loops for writing factors:** + + .. code-block:: sh + + FACTOR_CODER_MAX_LOOP=10 diff --git a/docs/scens/model_agent_fin.rst b/docs/scens/model_agent_fin.rst index fc5b5d37..30c16622 100644 --- a/docs/scens/model_agent_fin.rst +++ b/docs/scens/model_agent_fin.rst @@ -4,43 +4,116 @@ Finance Model Agent ======================= -πŸ€– Automated Machine Learning Model Construction & Iteration in Quantitative Finance +**πŸ€– Automated Quantitative Trading & Iterative Model Evolution** +------------------------------------------------------------------------------------------ -Intro -~~~~~~~~~~~ +πŸ“– Background +~~~~~~~~~~~~~~ +TODO -In this scenario, our automated system proposes hypothesis, constructs model, implements code, receives back-testing, and uses feedbacks. Hypothesis is iterated in this continuous process. The system aims to automatically optimise performance metrics from Qlib library thereby finding the optimised code through autonomous research and development. - -Automated R&D of model in Quantitative Finance -~~~~~~~~~~~~~ - -**R (Research)** -- Iteration of ideas and hypotheses. -- Continuous learning and knowledge construction. - -**D (Development)** -- Evolving code generation and model refinement. -- Automated implementation and testing of models. - -Objective -~~~~~~~~~ - -The demo showcases the iterative process of hypothesis generation, knowledge construction, and decision-making in model construction in quantitative finance. It highlights how models evolve through continuous feedback and refinement, therefore building the domain knowledge specific to the scenario of quantitative finance. - -Demo +πŸŽ₯ Demo ~~~~~~~~~~ +TODO: Here should put a video of the demo. -.. TODO -Quick Start +🌟 Introduction +~~~~~~~~~~~~~~~~ + +In this scenario, our automated system proposes hypothesis, constructs model, implements code, receives back-testing, and uses feedbacks. +Hypothesis is iterated in this continuous process. +The system aims to automatically optimise performance metrics from Qlib library thereby finding the optimised code through autonomous research and development. + +Here's an enhanced outline of the steps: + +**Step 1 : Hypothesis Generation πŸ”** + +- Generate and propose initial hypotheses based on previous experiment analysis and domain expertise, with thorough reasoning and financial justification. + +**Step 2 : Model Creation ✨** + +- Transform the hypothesis into a task. +- Develop, define, and implement a quantitative model, including its name, description, and formulation. + +**Step 3 : Model Implementation πŸ‘¨β€πŸ’»** + +- Implement the model code based on the detailed description. +- Evolve the model iteratively as a developer would, ensuring accuracy and efficiency. + +**Step 4 : Backtesting with Qlib πŸ“‰** + +- Conduct backtesting using the newly developed model and 20 factors extracted from Alpha158 in Qlib. +- Evaluate the model's effectiveness and performance. + ++----------------+------------+------------------------+----------------------------------------------------+ +| Dataset | Model | Factors | Data Split | ++================+============+========================+====================================================+ +| CSI300 | RDAgent-dev| 20 factors (Alpha158) | +-----------+--------------------------+ | +| | | | | Train | 2008-01-01 to 2014-12-31 | | +| | | | +-----------+--------------------------+ | +| | | | | Valid | 2015-01-01 to 2016-12-31 | | +| | | | +-----------+--------------------------+ | +| | | | | Test | 2017-01-01 to 2020-08-01 | | +| | | | +-----------+--------------------------+ | ++----------------+------------+------------------------+----------------------------------------------------+ + +**Step 5 : Feedback Analysis πŸ”** + +- Analyze backtest results to assess performance. +- Incorporate feedback to refine hypotheses and improve the model. + +**Step 6 :Hypothesis Refinement ♻️** + +- Refine hypotheses based on feedback from backtesting. +- Repeat the process to continuously improve the model. + +⚑ Quick Start ~~~~~~~~~~~~~~~~~ -To quickly start the model proposal process, run the following command in your terminal within the 'rdagent' virtual environment: +You can try our demo by running the following command: -.. code-block:: sh +- 🐍 Create a Conda Environment + - Create a new conda environment with Python (3.10 and 3.11 are well tested in our CI): + + .. code-block:: sh + + conda create -n rdagent python=3.10 - python rdagent/app/qlib_rd_loop/model_w_sc.py + - Activate the environment: -Usage of modules -~~~~~~~~~~~~~~~~~ + .. code-block:: sh + + conda activate rdagent + +- πŸ› οΈ Run Make Files + - Navigate to the directory containing the MakeFile and set up the development environment: + + .. code-block:: sh + + make dev + +- πŸ“¦ Install Pytorch + - Install Pytorch and related libraries: + + .. code-block:: sh + + pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu + pip3 install torch_geometric + +- βš™οΈ Environment Configuration + - Place the `.env` file in the same directory as the `.env.example` file. + - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) + + - Export each variable in the .env file: + + .. code-block:: sh + + export $(grep -v '^#' .env | xargs) + +- πŸš€ Run the Application + .. code-block:: sh + + python rdagent/app/qlib_rd_loop/model_w_sc.py + +πŸ› οΈ Usage of modules +~~~~~~~~~~~~~~~~~~~~~ TODO: Show some examples: diff --git a/docs/scens/model_copilot_general.rst b/docs/scens/model_copilot_general.rst index b27880f3..2e1d2bae 100644 --- a/docs/scens/model_copilot_general.rst +++ b/docs/scens/model_copilot_general.rst @@ -4,14 +4,23 @@ General Model Copilot ====================== -**πŸ€– Automated Model Research & Development Co-Pilot ** +**πŸ€– Automated Model Research & Development Co-Pilot** +-------------------------------------------------------- -Scen5 Intro -~~~~~~~~~~~ +πŸ“– Background +~~~~~~~~~~~~~~ +TODO: + +πŸŽ₯ Demo +~~~~~~~~~~ +TODO: Here should put a video of the demo. + +🌟 Introduction +~~~~~~~~~~~~~~~~ In this scenario, our automated system proposes hypotheses, constructs models, implements code, performs back-testing, and uses feedback to iterate continuously. The system aims to automatically optimize performance metrics from the Qlib library, finding the best code through autonomous research and development. Model R&D CoPilot Scenario -~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~ **Overview** This demo automates the extraction and iterative development of models from academic papers, ensuring functionality and correctness. This scenario automates the development of PyTorch models by reading academic papers or other sources. It supports various data types, including tabular, time-series, and graph data. The primary workflow involves two main components: the Reader and the Coder. @@ -33,57 +42,82 @@ This demo automates the extraction and iterative development of models from acad - **Time-Series Data:** Sequential data points indexed in time order, useful for forecasting and temporal pattern recognition. - **Graph Data:** Data structured as nodes and edges, suitable for network analysis and relational tasks. - -Demo -~~~~~~~~~~ -.. TODO - -Quick Start +⚑ Quick Start ~~~~~~~~~~~~~~~~~ -To quickly start the model co-pilot process, first prepare environment setup: -.. code-block:: sh +You can try our demo by running the following command: - cd rdagent - make dev +- 🐍 Create a Conda Environment + - Create a new conda environment with Python (3.10 and 3.11 are well tested in our CI): + + .. code-block:: sh + + conda create -n rdagent python=3.10 -Then prepare relevant files (in pdf format) by uploading papers to the directory below and copy the path as report_file_path. + - Activate the environment: -.. code-block:: sh + .. code-block:: sh - rdagent/scenarios/general_model + conda activate rdagent -Run the following command in your terminal within the same virtual environment: +- πŸ› οΈ Run Make Files + - Navigate to the directory containing the MakeFile and set up the development environment: -.. code-block:: sh + .. code-block:: sh - python rdagent/app/general_model/general_model.py report_file_path + make dev -Usage of modules -~~~~~~~~~~~~~~~~~ +- πŸ“¦ Install Pytorch + - Install Pytorch and related libraries: + + .. code-block:: sh + + pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu + pip3 install torch_geometric + +- βš™οΈ Environment Configuration + - Place the `.env` file in the same directory as the `.env.example` file. + - The `.env.example` file contains the environment variables required for users using the OpenAI API (Please note that `.env.example` is an example file. `.env` is the one that will be finally used.) + + - Export each variable in the .env file: + + .. code-block:: sh + + export $(grep -v '^#' .env | xargs) + +- πŸš€ Run the Application + - Prepare relevant files (in pdf format) by uploading papers to the directory below and copy the path as report_file_path. + + .. code-block:: sh + + rdagent/scenarios/general_model + + - Run the following command in your terminal within the same virtual environment: + + .. code-block:: sh + + python rdagent/app/general_model/general_model.py report_file_path + +πŸ› οΈ Usage of modules +~~~~~~~~~~~~~~~~~~~~~ There are mainly two modules in this scenario: one that reads the paper and returns a model card & one that reads the model card and returns functional code. The moduldes can also be used separately as components for developers to build up new scenarios. -Configurations -~~~~~~~~~~~~~~~~~ - -The `config.yaml` file located in the `model_template` folder contains the relevant configurations for running the developed model in Qlib. The default settings include key information such as: - -- **market**: Specifies the market, which is set to `csi300`. -- **fields_group**: Defines the fields group, with the value `feature`. -- **col_list**: A list of columns used, including various indicators such as `RESI5`, `WVMA5`, `RSQR5`, and others. -- **start_time**: The start date for the data, set to `2008-01-01`. -- **end_time**: The end date for the data, set to `2020-08-01`. -- **fit_start_time**: The start date for fitting the model, set to `2008-01-01`. -- **fit_end_time**: The end date for fitting the model, set to `2014-12-31`. - -The default hyperparameters used in the configuration are as follows: - -- **n_epochs**: The number of epochs, set to `100`. -- **lr**: The learning rate, set to `1e-3`. -- **early_stop**: The early stopping criterion, set to `10`. -- **batch_size**: The batch size, set to `2000`. -- **metric**: The evaluation metric, set to `loss`. -- **loss**: The loss function, set to `mse`. -- **n_jobs**: The number of parallel jobs, set to `20`. +- Configurations + - The `config.yaml` file located in the `model_template` folder contains the relevant configurations for running the developed model in Qlib. The default settings include key information such as: + - **market**: Specifies the market, which is set to `csi300`. + - **fields_group**: Defines the fields group, with the value `feature`. + - **col_list**: A list of columns used, including various indicators such as `RESI5`, `WVMA5`, `RSQR5`, and others. + - **start_time**: The start date for the data, set to `2008-01-01`. + - **end_time**: The end date for the data, set to `2020-08-01`. + - **fit_start_time**: The start date for fitting the model, set to `2008-01-01`. + - **fit_end_time**: The end date for fitting the model, set to `2014-12-31`. + - The default hyperparameters used in the configuration are as follows: + - **n_epochs**: The number of epochs, set to `100`. + - **lr**: The learning rate, set to `1e-3`. + - **early_stop**: The early stopping criterion, set to `10`. + - **batch_size**: The batch size, set to `2000`. + - **metric**: The evaluation metric, set to `loss`. + - **loss**: The loss function, set to `mse`. + - **n_jobs**: The number of parallel jobs, set to `20`. \ No newline at end of file diff --git a/docs/ui.rst b/docs/ui.rst index 5bea46fe..bc4ebf3e 100644 --- a/docs/ui.rst +++ b/docs/ui.rst @@ -29,6 +29,7 @@ Use Web App 1. Open the sidebar. +.. TODO: update these 2. Select the scenario you want to show. There are some pre-defined scenarios: - Qlib Model - Qlib Factor