mirror of
https://github.com/NicolasBohn/NexQuant.git
synced 2026-07-27 23:47:46 +00:00
feat: Uploaded Documentation, Updated Prompts & Some Code for model demo (#144)
* Added three new keys on hypothesis reasoning * Updated two scenario rich text * Uploaded Documentation & Further Improved Demo of Models * Add docs --------- Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
This commit is contained in:
Xisen Wang
@@ -103,7 +103,10 @@ In this project, we are aiming to build a Agent to automate Data-Driven R\&D tha
|
||||
<!--  -->
|
||||
|
||||
## 📈 Scenarios/Demos
|
||||
Here is our supported scenarios
|
||||
|
||||
In the two key areas of data-driven scenarios, model implementation and data building, our system aims to serve two main roles: 🦾copilot and 🤖agent. The 🦾copilot follows human instructions to automate repetitive tasks. The 🤖agent, being more autonomous, actively proposes ideas for better results in the future.
|
||||
|
||||
The supported scenarios are listed below:
|
||||
|
||||
| Scenario/Target | Model Implementation | Data Building |
|
||||
| -- | -- | -- |
|
||||
|
||||
+1
-4
@@ -12,14 +12,11 @@ Welcome to RDAgent's documentation!
|
||||
|
||||
demo_and_introduction
|
||||
installation_and_configuration
|
||||
scenarios_and_quickstart
|
||||
scens/catalog
|
||||
project_framework_introduction
|
||||
development
|
||||
api_reference
|
||||
policy
|
||||
test_dropdown/test1
|
||||
|
||||
.. test_dropdown/test1
|
||||
|
||||
|
||||
Indices and tables
|
||||
|
||||
@@ -1,111 +0,0 @@
|
||||
=========================
|
||||
Scenarios and Quick Start
|
||||
=========================
|
||||
|
||||
Scenario lists
|
||||
=========================
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Scenario/Target
|
||||
- Model Implementation
|
||||
- Data Building
|
||||
* - 💹 Finance
|
||||
- Iteratively Proposing Ideas & Evolving
|
||||
- Auto reports reading & implementation
|
||||
Iteratively Proposing Ideas & Evolving
|
||||
* - 🩺 Medical
|
||||
- Iteratively Proposing Ideas & Evolving
|
||||
-
|
||||
* - 🏭 General
|
||||
- Auto paper reading & implementation
|
||||
-
|
||||
|
||||
Scnarios' demo & quick start
|
||||
============================
|
||||
|
||||
Scen1
|
||||
-----
|
||||
🤖 Knowledge-Based Hypothesis Generation and Iteration
|
||||
|
||||
Scen1 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, our model autonomously generates and tests hypotheses using a knowledge base. The process involves:
|
||||
|
||||
- **🔍 Hypothesis Generation**: The model proposes new hypotheses.
|
||||
- **📝 Factor Creation**: Write and define new factors.
|
||||
- **✅ Factor Validation**: Validate the factors quantitatively.
|
||||
- **📈 Backtesting with Qlib**:
|
||||
|
||||
- **Dataset**: CSI300
|
||||
- **Model**: LGBModel
|
||||
- **Factors**: Alpha158 +
|
||||
- **Data Split**:
|
||||
|
||||
- **Train**: 2008-01-01 to 2014-12-31
|
||||
- **Valid**: 2015-01-01 to 2016-12-31
|
||||
- **Test**: 2017-01-01 to 2020-08-01
|
||||
- **🔄 Feedback Analysis**: Analyze backtest results.
|
||||
- **🔧 Hypothesis Refinement**: Refine hypotheses based on feedback and repeat.
|
||||
|
||||
Scen1 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen1 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the factor extraction process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/factor.py
|
||||
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
|
||||
|
||||
Scen2:
|
||||
------
|
||||
📄 Research Report-Based Factor Extraction
|
||||
|
||||
Scen2 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, factors and hypotheses are extracted from research reports. The process includes:
|
||||
|
||||
- **🔍 Factor Extraction**: Extract relevant factors from research reports.
|
||||
- **📝 Factor Creation**: Define these extracted factors.
|
||||
- **✅ Factor Validation**: Validate the extracted factors.
|
||||
- **📈 Backtesting with Qlib**:
|
||||
|
||||
- **Dataset**: CSI300
|
||||
- **Model**: LGBModel
|
||||
- **Factors**: Alpha158 +
|
||||
- **Data Split**:
|
||||
|
||||
- **Train**: 2008-01-01 to 2014-12-31
|
||||
- **Valid**: 2015-01-01 to 2016-12-31
|
||||
- **Test**: 2017-01-01 to 2020-08-01
|
||||
- **🔄 Feedback Analysis**: Analyze backtest results.
|
||||
- **🔧 Hypothesis Refinement**: Refine hypotheses based on feedback and continue the cycle.
|
||||
|
||||
Scen2 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen2 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the factor extraction process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/factor_from_report_sh.py
|
||||
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
@@ -0,0 +1,46 @@
|
||||
=========================
|
||||
Scenarios
|
||||
=========================
|
||||
|
||||
Scenario lists
|
||||
=========================
|
||||
|
||||
In the two key areas of data-driven scenarios, model implementation and data building, our system aims to serve two main roles: 🦾copilot and 🤖agent.
|
||||
|
||||
- The 🦾copilot follows human instructions to automate repetitive tasks.
|
||||
- The 🤖agent, being more autonomous, actively proposes ideas for better results in the future.
|
||||
|
||||
The supported scenarios are listed below:
|
||||
|
||||
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Scenario/Target
|
||||
- Model Implementation
|
||||
- Data Building
|
||||
* - 💹 Finance
|
||||
- :ref:`🤖Iteratively Proposing Ideas & Evolving <model_agent_fin>`
|
||||
- :ref:`🦾Auto reports reading & implementation <data_copilot_fin>`
|
||||
|
||||
:ref:`🤖Iteratively Proposing Ideas & Evolving <data_agent_fin>`
|
||||
* - 🩺 Medical
|
||||
- :ref:`🤖Iteratively Proposing Ideas & Evolving <model_agent_med>`
|
||||
-
|
||||
* - 🏭 General
|
||||
- :ref:`🦾Auto paper reading & implementation <model_copilot_general>`
|
||||
-
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Doctree:
|
||||
:hidden:
|
||||
|
||||
data_agent_fin
|
||||
data_copilot_fin
|
||||
model_agent_fin
|
||||
model_agent_med
|
||||
model_copilot_general
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
.. _data_agent_fin:
|
||||
|
||||
=====================
|
||||
Finance Data Agent
|
||||
=====================
|
||||
|
||||
Scen1
|
||||
-----
|
||||
🤖 Knowledge-Based Hypothesis Generation and Iteration
|
||||
|
||||
Scen1 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, our model autonomously generates and tests hypotheses using a knowledge base. The process involves:
|
||||
|
||||
- **🔍 Hypothesis Generation**: The model proposes new hypotheses.
|
||||
- **📝 Factor Creation**: Write and define new factors.
|
||||
- **✅ Factor Validation**: Validate the factors quantitatively.
|
||||
- **📈 Backtesting with Qlib**:
|
||||
|
||||
- **Dataset**: CSI300
|
||||
- **Model**: LGBModel
|
||||
- **Factors**: Alpha158 +
|
||||
- **Data Split**:
|
||||
|
||||
- **Train**: 2008-01-01 to 2014-12-31
|
||||
- **Valid**: 2015-01-01 to 2016-12-31
|
||||
- **Test**: 2017-01-01 to 2020-08-01
|
||||
- **🔄 Feedback Analysis**: Analyze backtest results.
|
||||
- **🔧 Hypothesis Refinement**: Refine hypotheses based on feedback and repeat.
|
||||
|
||||
Scen1 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen1 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the factor extraction process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/factor.py
|
||||
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
.. _data_copilot_fin:
|
||||
|
||||
=====================
|
||||
Finance Data Copilot
|
||||
=====================
|
||||
|
||||
|
||||
Scen2:
|
||||
------
|
||||
📄 Research Report-Based Factor Extraction
|
||||
|
||||
Scen2 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, factors and hypotheses are extracted from research reports. The process includes:
|
||||
|
||||
- **🔍 Factor Extraction**: Extract relevant factors from research reports.
|
||||
- **📝 Factor Creation**: Define these extracted factors.
|
||||
- **✅ Factor Validation**: Validate the extracted factors.
|
||||
- **📈 Backtesting with Qlib**:
|
||||
|
||||
- **Dataset**: CSI300
|
||||
- **Model**: LGBModel
|
||||
- **Factors**: Alpha158 +
|
||||
- **Data Split**:
|
||||
|
||||
- **Train**: 2008-01-01 to 2014-12-31
|
||||
- **Valid**: 2015-01-01 to 2016-12-31
|
||||
- **Test**: 2017-01-01 to 2020-08-01
|
||||
- **🔄 Feedback Analysis**: Analyze backtest results.
|
||||
- **🔧 Hypothesis Refinement**: Refine hypotheses based on feedback and continue the cycle.
|
||||
|
||||
Scen2 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen2 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the factor extraction process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/factor_from_report_sh.py
|
||||
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
.. _model_agent_fin:
|
||||
|
||||
=======================
|
||||
Finance Model Agent
|
||||
=======================
|
||||
|
||||
🤖 Automated Machine Learning Model Construction & Iteration in Quantitative Finance
|
||||
|
||||
Intro
|
||||
~~~~~~~~~~~
|
||||
|
||||
In this scenario, our automated system proposes hypothesis, constructs model, implements code, receives back-testing, and uses feedbacks. Hypothesis is iterated in this continuous process. The system aims to automatically optimise performance metrics from Qlib library thereby finding the optimised code through autonomous research and development.
|
||||
|
||||
Automated R&D of model in Quantitative Finance
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
**R (Research)**
|
||||
- Iteration of ideas and hypotheses.
|
||||
- Continuous learning and knowledge construction.
|
||||
|
||||
**D (Development)**
|
||||
- Evolving code generation and model refinement.
|
||||
- Automated implementation and testing of models.
|
||||
|
||||
Objective
|
||||
~~~~~~~~~
|
||||
|
||||
The demo showcases the iterative process of hypothesis generation, knowledge construction, and decision-making in model construction in quantitative finance. It highlights how models evolve through continuous feedback and refinement, therefore building the domain knowledge specific to the scenario of quantitative finance.
|
||||
|
||||
Demo
|
||||
~~~~~~~~~~
|
||||
|
||||
.. TODO
|
||||
|
||||
Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
To quickly start the model proposal process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/qlib_rd_loop/model_w_sc.py
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples:
|
||||
@@ -0,0 +1,54 @@
|
||||
.. _model_agent_med:
|
||||
|
||||
===================
|
||||
Medical Model Agent
|
||||
===================
|
||||
|
||||
|
||||
Scen4:
|
||||
------
|
||||
🤖 Automated Model Research & Development Co-Pilot
|
||||
|
||||
Scen4 Intro
|
||||
~~~~~~~~~~~
|
||||
In this scenario, our automated system proposes hypotheses, constructs models, implements code, performs back-testing, and uses feedback to iterate continuously. The system aims to automatically optimize performance metrics from the Qlib library, finding the best code through autonomous research and development.
|
||||
|
||||
Model R&D CoPilot Scenario
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
**Overview**
|
||||
|
||||
This demo automates the extraction and iterative development of models from academic papers, ensuring functionality and correctness. This scenario automates the development of PyTorch models by reading academic papers or other sources. It supports various data types, including tabular, time-series, and graph data. The primary workflow involves two main components: the Reader and the Coder.
|
||||
|
||||
**Workflow Components**
|
||||
|
||||
1. **Reader**
|
||||
- Parses and extracts relevant model information from academic papers or sources, including architectures, parameters, and implementation details.
|
||||
- Uses Large Language Models to convert content into a structured format for the Coder.
|
||||
|
||||
2. **Evolving Coder**
|
||||
- Translates structured information from the Reader into executable PyTorch code.
|
||||
- Utilizes an evolving coding mechanism to ensure correct tensor shapes, verified with sample input tensors.
|
||||
- Iteratively refines the code to align with source material specifications.
|
||||
|
||||
#### Supported Data Types
|
||||
|
||||
- **Tabular Data:** Structured data with rows and columns, such as spreadsheets or databases.
|
||||
- **Time-Series Data:** Sequential data points indexed in time order, useful for forecasting and temporal pattern recognition.
|
||||
- **Graph Data:** Data structured as nodes and edges, suitable for network analysis and relational tasks.
|
||||
|
||||
Scen4 Demo
|
||||
~~~~~~~~~~
|
||||
.. TODO
|
||||
|
||||
Scen4 Quick Start
|
||||
~~~~~~~~~~~~~~~~~
|
||||
To quickly start the model proposal process, run the following command in your terminal within the 'rdagent' virtual environment:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python rdagent/app/model_extraction_and_code/model_extraction_and_implementation.py
|
||||
|
||||
Usage of modules
|
||||
~~~~~~~~~~~~~~~~~
|
||||
TODO: Show some examples
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
.. _model_copilot_general:
|
||||
|
||||
======================
|
||||
General Model Copilot
|
||||
======================
|
||||
@@ -1,28 +0,0 @@
|
||||
=========================
|
||||
Test dropdown 1
|
||||
=========================
|
||||
|
||||
title1
|
||||
=========================
|
||||
|
||||
content1
|
||||
|
||||
title2
|
||||
=========================
|
||||
|
||||
content2
|
||||
|
||||
title3
|
||||
=========================
|
||||
|
||||
content3
|
||||
|
||||
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Doctree:
|
||||
:hidden:
|
||||
|
||||
test2
|
||||
@@ -1,18 +0,0 @@
|
||||
=========================
|
||||
Test dropdown 2
|
||||
=========================
|
||||
|
||||
title1
|
||||
=========================
|
||||
|
||||
content1
|
||||
|
||||
title2
|
||||
=========================
|
||||
|
||||
content2
|
||||
|
||||
title3
|
||||
=========================
|
||||
|
||||
content3
|
||||
@@ -31,7 +31,7 @@ class GeneralModelScenario(Scenario):
|
||||
@property
|
||||
def rich_style_description(self) -> str:
|
||||
return """
|
||||
# [General Model Research Co-Pilot] (#_scenario)
|
||||
# [Model Research & Development Co-Pilot] (#_scenario)
|
||||
|
||||
## [Overview](#_summary)
|
||||
|
||||
|
||||
@@ -10,12 +10,12 @@ hypothesis_gen:
|
||||
Here are the specifications: {{ hypothesis_specification }}
|
||||
|
||||
user_prompt: |-
|
||||
The user has made several hypothesis on this scenario and did several evaluation on them.
|
||||
If it is not the first round, then the user has made several hypothesis on this scenario and did several evaluation on them.
|
||||
The former hypothesis and the corresponding feedbacks are as follows (focus on the last one & the new hypothesis that it provides and reasoning to see if you agree):
|
||||
{{ hypothesis_and_feedback }}
|
||||
To help you generate new {{targets}}, we have prepared the following information for you:
|
||||
{{ RAG }}
|
||||
Please generate the new hypothesis based on the information above. Also generate the relevant keys for the reasoning and the distilled knowledge that follows.
|
||||
Please generate the new hypothesis based on the information above. Also generate the relevant keys for the reasoning and the distilled knowledge that follows. For those keys, in particular for knowledge, explain in the context of the specific scenario to build up domain knowledge in the specific field rather than genearl knowledge.
|
||||
|
||||
hypothesis2experiment:
|
||||
system_prompt: |-
|
||||
|
||||
@@ -12,12 +12,12 @@ hypothesis_and_feedback: |-
|
||||
hypothesis_output_format: |-
|
||||
The output should follow JSON format. The schema is as follows:
|
||||
{
|
||||
"hypothesis": "The new hypothesis generated based on the information provided.",
|
||||
"hypothesis": "The new hypothesis generated based on the information provided.", # Note that this should focus on model architecture, not training process or feature engineering or anything else
|
||||
"reason": "The reason why you generate this hypothesis. It should be comprehensive and logical. It should cover the other keys below and extend them.",
|
||||
"concise_reason": Two line summary. First line focuses on the a concise justification for the change. 2nd Line learns from first line and previous experiences (hypothesis & experiments & code & feedbacks) to generalise a knowledge statement (use tend to/because/if/generally/etc. ). This is the summary of the three keys below.
|
||||
"concise_observation": One line summary. It focuses on the observation of the given scenario, data characteristics, or previous experiences (failures & succeses).
|
||||
"concise_justification": One line summary. It focuses on the justification for the change in new hypothesis and the route of exploration supporting the growth of the hypothesis, based on the observation.
|
||||
"concise_knowledge": One line summary. It focuses on a transferable knowledege that comes with the new hypothesis. Use conditional grammar. eg. "If...., ..; When..., .; and etc"
|
||||
"concise_knowledge": One line summary. It focuses on a transferable knowledege that comes with the new hypothesis. Use conditional grammar. eg. "If...., ..; When..., .; and etc" Make sure that you state things clearly without ambiguity. Eg. avoid saying "previous hypothesis", because one wouldn't know what that is.
|
||||
}
|
||||
|
||||
model_hypothesis_specification: |-
|
||||
|
||||
Reference in New Issue
Block a user