From 34907192592ba247f4701349bfb2a207910327ac Mon Sep 17 00:00:00 2001
From: Haoran Pan <167847254+TPLin22@users.noreply.github.com>
Date: Fri, 6 Jun 2025 11:49:40 +0800
Subject: [PATCH] feat: update prompt to improve json respond format of some
 LLM models (#928)

* update prompt to improve json respond format of some LLM models

* fix

* fix

---------

Co-authored-by: WinstonLiyte <1957922024@qq.com>
---
 rdagent/components/coder/factor_coder/prompts.yaml | 4 +++-
 rdagent/scenarios/qlib/prompts.yaml                | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/rdagent/components/coder/factor_coder/prompts.yaml b/rdagent/components/coder/factor_coder/prompts.yaml
index 3d7d07f5..8bc24c9a 100644
--- a/rdagent/components/coder/factor_coder/prompts.yaml
+++ b/rdagent/components/coder/factor_coder/prompts.yaml
@@ -49,6 +49,8 @@ evolving_strategy_factor_implementation_v1_system: |-
   
   Your must write your code based on your former latest attempt below which consists of your former code and code feedback, you should read the former attempt carefully and must not modify the right part of your former code.
 
+  Notice that you should not add any other text before or after the json format.
+
   {% if queried_former_failed_knowledge|length != 0 %}
   --------------Your former latest attempt:---------------
   =====Code to the former implementation=====
@@ -188,7 +190,7 @@ evaluator_final_decision_v1_system: |-
   The implementation final decision is considered in the following logic:
   1. If the value and the ground truth value are exactly the same under a small tolerance, the implementation is considered correct.
   2. If the value and the ground truth value have a high correlation on ic or rank ic, the implementation is considered correct.
-  3. If no ground truth value is provided, the implementation is considered correct if the code executes successfully (assuming the data provided is correct). Any exceptions, including those actively raised, are considered faults of the code. Additionally, the code feedback must align with the scenario and factor description.
+  3. If no ground truth value is provided, the implementation is considered correct if the code executes successfully (assuming the data provided is correct). Any exceptions, including those actively raised, are considered faults of the code. Additionally, the code feedback must align with the scenario and factor description. The implementation cannot be considered correct if the code execution failed, no matter what the reason is.
 
   Please response the critic in the json format. Here is an example structure for the JSON output, please strictly follow the format:
   {
diff --git a/rdagent/scenarios/qlib/prompts.yaml b/rdagent/scenarios/qlib/prompts.yaml
index 3838899d..e53aa1af 100644
--- a/rdagent/scenarios/qlib/prompts.yaml
+++ b/rdagent/scenarios/qlib/prompts.yaml
@@ -109,6 +109,7 @@ factor_hypothesis_specification: |-
     - If optimizing a specific type of factor, proceed from simple to complex.
   5. Note
     - Highlight that factors surpassing SOTA are included in the library to avoid re-implementation.
+    - No matter how many factors you plan to generate, only reply with one set of hypothesis and reason. The hypothesis can include the proposal of multiple factors at the same time.
 
 factor_experiment_output_format: |-
   The output should follow JSON format. The schema is as follows: