gavindiaz/NexQuant
1
0
Fork 0
mirror of https://github.com/NicolasBohn/NexQuant.git synced 2026-07-27 23:47:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

small fix to runner eval order to prevent both final decision and hyperparameter decision to be true

Browse Source 
Co-authored-by: Xu <v-xuminrui@microsoft.com>
This commit is contained in:
Roland Minrui
2025-07-29 13:31:35 +08:00
committed by GitHub
parent 4cee886c80
commit eb70333b77
2 changed files with 4 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rdagent/scenarios/data_science/dev/prompts.yaml
-2
View File
@@ -81,7 +81,6 @@ exp_feedback:
"Feedback for Hypothesis": Explicitly confirm or refute the hypothesis based on specific data points or performance trends. Limit to two sentences.",
"Evaluation Aligned With Task": "yes or no",
"Replace Best Result": "yes or no",
"Refine Decision": "yes or no",
"Reasoning": "Clearly explain the reason for success or failure of the experiment. Begin explicitly with [Submission format error], [Evaluation error], [Experiment Analysis] or [Code Analysis] depending on the step at which issues arose. Reference specific scores and methodological differences with SOTA. Limit to three sentences.",
"EDA Improvement": "improvement suggestion for EDA code, if needed, otherwise set to 'no'. If there is no EDA code, set to 'no'."
}
@@ -215,7 +214,6 @@ exp_feedback_draft:
"Feedback for Hypothesis": Explicitly confirm or refute the hypothesis based on specific data points or performance trends. Limit to two sentences.",
"Evaluation Aligned With Task": "yes or no",
"Replace Best Result": "yes or no",
"Refine Decision": "yes or no",
"Reasoning": "Clearly explain the reason for success or failure of the experiment. Begin explicitly with [Submission format error], [Evaluation error], [Experiment Analysis] or [Code Analysis] depending on the step at which issues arose. Reference specific scores and methodological differences with SOTA. Limit to three sentences.",
"EDA Improvement": "improvement suggestion for EDA code, if needed, otherwise set to 'no'. If there is no EDA code, set to 'no'."
}
rdagent/scenarios/data_science/dev/runner/prompts.yaml
+4 -4
View File
@@ -33,8 +33,8 @@ DSCoSTEER_eval:
# Evaluation 2: Hyperparameter
## Evaluation Description
The user will provide you the time spent on the whole code execution and the timeout of the code execution. You should decide whether the hyperparameter is reasonable based on the time.
For example, if the code uses only a very small portion of the allowed time, and hyperparameters like `n_estimators` or `epochs` have low values, with early stopping not being triggered and possible signs of underfitting, you should suggest increasing these hyperparameters.
You should also notice other resources utilization hyper-parameters,
If the code uses only a very small portion (below 25%) of the allowed time, and hyperparameters like `n_estimators` or `epochs` have low values, with early stopping not being triggered and possible signs of underfitting, you should suggest increasing these hyperparameters.
You should also notice other resources utilization hyper-parameters.
For example, if you are using a GPU with large memory, and the batch size is set very low, you should suggest increasing the batch size if it is not reasonable.
## Evaluation Guidelines
@@ -56,9 +56,9 @@ DSCoSTEER_eval:
"execution": "Describe whether the whole code base executed successfully and generating the final submission. Include any errors or issues encountered, and retain all error messages and traceback details.",
"return_checking": "Verify the generated files, particularly the submission file. Ensure that its format matches the sample submission",
"code": "Provide feedback on code quality, readability, and adherence to the given specifications.",
"final_decision": <true/false>,
"hyperparameter_tuning_decision": <true/false>,
"hyperparameter_tuning_suggestion": <suggestion in plain text for hyperparameter tuning>,
"final_decision": <true/false>,
}
```
{% else %}
@@ -71,9 +71,9 @@ DSCoSTEER_eval:
"execution": "Describe whether the code executed successfully. Include any errors or issues encountered, and append all error messages and full traceback details without summarizing or omitting any information.",
"return_checking": "Describe the expected file to be generated.",
"code": "Provide feedback on code quality, readability, and adherence to the given specifications.",
"final_decision": <true/false>,
"hyperparameter_tuning_decision": <true/false>,
"hyperparameter_tuning_suggestion": <suggestion in plain text for hyperparameter tuning>,
"final_decision": <true/false>,
}
```
{% endif %}