gavindiaz/NexQuant
1
0
Fork 0
mirror of https://github.com/NicolasBohn/NexQuant.git synced 2026-08-07 20:17:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: fix the errors in the coder and evaluator of the five components (#576)

Browse Source 
* refine the prompt in feat coder

* refine data loader prompt in coder & eval

* fix

* fix

* refine model prompt in coder & eval

* refine ensemble prompt in coder & eval

* refine workflow prompt in coder & eval

* fix
This commit is contained in:
Yuante Li
2025-02-10 20:20:30 +08:00
committed by GitHub
parent 63d7388612
commit e2b7c9dc78
8 changed files with 376 additions and 328 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rdagent/components/coder/data_science/raw_data_loader/prompts.yaml
+60 -46
View File
@@ -1,6 +1,6 @@
spec:
system: |-
You are a world-class data scientist and machine learning engineer with deep expertise in statistics, mathematics, and computer science.
You are a world-class data scientist and machine learning engineer with deep expertise in statistics, mathematics, and computer science.
Your knowledge spans cutting-edge data analysis techniques, advanced machine learning algorithms, and their practical applications to solve complex real-world problems.
Currently, you are working on a Kaggle competition project.
@@ -76,7 +76,7 @@ spec:
You should follow the provided specifications to improve this task.
{% endif %}
Please respond with a JSON structure as follows:
{
@@ -305,25 +305,19 @@ spec:
data_loader_coder:
system: |-
You are a Python data scientist working on a new project. This project will be used to analyze data and build models to predict future outcomes, and this project codes will be written by GPT.
Your task is described below:
You are a world-class data scientist and machine learning engineer with deep expertise in statistics, mathematics, and computer science.
Your knowledge spans cutting-edge data analysis techniques, advanced machine learning algorithms, and their practical applications to solve complex real-world problems.
## Task Description
{{ task_desc }}
You should follow the provided specifications to complete this task.
You need to write the corresponding data loading code based on the information provided in the user's Data Folder Description, rather than relying on any suggestions that might exist in the spec.
Notice, the data files are stored in the data folder located at `/kaggle/input/`, and the data folder is structured as described in the Data Folder Description. Please don't load the data from the current directory.
Please response the code in the following json format. Here is an example structure for the JSON output:
{
"code": "The Python code as a string."
}
{% if queried_similar_successful_knowledge|length != 0 or queried_former_failed_knowledge|length != 0 %}
-----------Here is the relevant information for this task-----------
## Relevant Information for This Task
{% endif %}
{% if queried_similar_successful_knowledge|length != 0 %}
--------------Successful Implementations for Similar Models:--------------
====={% for similar_successful_knowledge in queried_similar_successful_knowledge %} Model {{loop.index}}:=====
--------- Successful Implementations for Similar Models ---------
====={% for similar_successful_knowledge in queried_similar_successful_knowledge %} Model {{ loop.index }}:=====
{{ similar_successful_knowledge.target_task.get_task_information() }}
=====Code:=====
{{ similar_successful_knowledge.implementation.all_codes }}
@@ -331,7 +325,7 @@ data_loader_coder:
{% endif %}
{% if queried_former_failed_knowledge|length != 0 %}
--------------Previous Failed Attempts:--------------
--------- Previous Failed Attempts ---------
{% for former_failed_knowledge in queried_former_failed_knowledge %} Attempt {{ loop.index }}:
=====Code:=====
{{ former_failed_knowledge.implementation.all_codes }}
@@ -340,66 +334,86 @@ data_loader_coder:
{% endfor %}
{% endif %}
## Guidelines
1. Ensure that the dataset is loaded strictly from `/kaggle/input/`, following the exact folder structure described in the **Data Folder Description**, and do not attempt to load data from the current directory (`./`).
## Output Format
Please response the code in the following json format. Here is an example structure for the JSON output:
{
"code": "The Python code as a string."
}
user: |-
---------Competition Information---------
--------- Competition Information ---------
{{ competition_info }}
---------Data Loader Specification---------
--------- Data Loader Specification ---------
{{ data_loader_spec }}
---------Data Folder Description---------(All path are relative to the data folder)
--------- Data Folder Description (All path are relative to the data folder) ---------
{{ folder_spec }}
{% if latest_code %}
---------Former code---------
{{ latest_code }}
--------- Former code ---------
{{ latest_code }}
{% if latest_code_feedback is not none %}
---------Feedback to former code---------
{{ latest_code_feedback }}
--------- Feedback to former code ---------
{{ latest_code_feedback }}
{% endif %}
The former code has some errors, you should write the correct code based on the former code. Avoid writing the same code to former code.
{% endif %}
The former code contains errors. You should correct the code based on the provided information, ensuring you do not repeat the same mistakes.
{% endif %}
You should strictly follow the function interface specifications provided by the specification to implement the function.
data_loader_eval:
system: |-
You are data scientist writing some data loader code for a Kaggle-style machine learning competition project.
The main code generation task is as follows:
{{task_desc}}
You are a data scientist responsible for evaluating data loader code for a Kaggle-style machine learning competition project.
## Task Description
{{ task_desc }}
The data loader code is in a file named "load_data.py":
## Data Loader Code
The data loader code is located in `load_data.py`:
```python
{{code}}
{{ code }}
```
You are testing the data_loader with the following code
## Testing Process
The data loader is tested using the following script:
```python
{{test_code}}
{{ test_code }}
```
{% if workflow_stdout is not none %}
Your feature engineering code is also part of the whole workflow, the user also tested the whole workflow and provided you the stdout.
The whole workflow code is:
{{workflow_code}}
Please consider both stdout and approve the code when both the feature engineering test and the whole workflow test pass.
### Whole Workflow Consideration
The data loader is part of the whole workflow. The user has executed the entire pipeline and provided additional stdout.
**Workflow Code:**
```python
{{ workflow_code }}
```
You should evaluate both the data loader test results and the overall workflow execution. **Approve the code only if both tests pass.**
{% endif %}
You'll be given the stdout of your testing scripts.
Please respond with your feedback in the following JSON format and order
## Evaluation Criteria
You will be given the standard output (`stdout`) from the data loader test and, if applicable, the workflow test.
Your response must follow this structured JSON format:
```json
{
"execution": "Describe how well the data loader executed, including any errors or issues encountered. Please keep the error message and tracking information",
"return_checking": "Detail the checks performed on the data loaded, including data integrity and correctness.",
"code": "Provide feedback on the code quality, readability, and adherence to specifications. Please also consider the efficiency of the code based on whether it uses multi-threading or GPUs to speed up the process.",
"execution": "Describe how well the data loader executed, including any errors or issues encountered. Retain all error messages and traceback details.",
"return_checking": "Evaluate the correctness and integrity of the loaded data. Check for issues like missing values, incorrect data types, outliers, or formatting inconsistencies.",
"code": "Assess code quality, readability, and adherence to best practices. Consider efficiency, including whether the code utilizes multi-threading or GPU acceleration for faster data loading.",
"final_decision": <true/false>
}
```
user: |-
Data loader test stdout:
{{stdout}}
--------- Data loader test stdout ---------
{{ stdout }}
{% if workflow_stdout is not none %}
Whole workflow test stdout:
{{workflow_stdout}}
--------- Whole workflow test stdout ---------
{{ workflow_stdout }}
{% endif %}