fix: fix some bugs (ensemble output, HPO, model tuning) (#648)

rdagent/components/coder/data_science/ensemble/eval_tests/ensemble_test.txt

@@ -68,6 +68,8 @@ for key in val_preds_dict.keys():
     else:
         print(f"Model {key} test predictions (test_preds_dict[key]) shape: {test_preds_dict[key].shape}")
 
+print(f"val_y.shape: {val_y.shape}" if not isinstance(val_y, list) else f"val_y(list)'s length: {len(val_y)}")
+
 # Run ensemble
 final_pred = ensemble_workflow(test_preds_dict, val_preds_dict, val_y)
 

rdagent/components/coder/data_science/model/prompts.yaml

@@ -39,8 +39,7 @@ model_coder:
     --------- Feature Engineering Code ---------
     {{ feature_code }}
     2. You should avoid using logging module to output information in your generated code, and instead use the print() function.
-    3. You can decide whether to use AutoML based on the characteristics of the task.
-    4. If the model can both be implemented by PyTorch and Tensorflow, please use pytorch for broader compatibility.
+    3. If the model can both be implemented by PyTorch and Tensorflow, please use pytorch for broader compatibility.
 
     ## Output Format
     {% if out_spec %}

rdagent/scenarios/data_science/proposal/prompts.yaml

@@ -107,9 +107,11 @@ task_gen:
     The user has made several task on this scenario but didn't get the expected result due to wrong implementation or just bad luck. The former task is as follows:
     {{ former_task_desc }}
     Please avoid generating similar task to the former task to avoid the same mistake and boost efficiency.
+    
     {% if targets == "Model" %}
-    If the previous failure was due to exceeding the time limit or memory constraints, you can try reducing the model size or opting for alternative algorithms with lower time or space complexity instead of using a neural network.
+    Based on the feedback from previous experiment failures, if the failure was due to exceeding the time limit or memory constraints, start with the smallest model size or choose alternative algorithms or methods with significantly lower time or space complexity instead of using a neural network. You can then iteratively refine and optimize the model in later stages.
     {% endif %}
+    
     {% endif %}
 
     {% if hypothesis is not none %}
@@ -212,7 +214,7 @@ direct_exp_gen:
     }
 
     {% if extra_requirement %}
-    {{extra_requirement}}
+    {{ extra_requirement }}
     {% endif %}
 
   user: |-
@@ -222,19 +224,19 @@ direct_exp_gen:
     {% if failed_exp_and_feedback_list_desc %}
     # Several trials after the best experiments
     The user has made several hypothesis on this scenario and did several evaluation on them.
-    The former hypothesis and the corresponding feedbacks are as follows (focus on the last one & the new hypothesis that it provides and reasoning to see if you agree):
     {{ failed_exp_and_feedback_list_desc }}
+    
     {% if targets == "Building model" %}
-    If the previous failure was due to exceeding the time limit or memory constraints, you can try reducing the model size or opting for alternative algorithms with lower time or space complexity instead of using a neural network.
+    Based on the feedback from previous experiment failures, if the failure was due to exceeding the time limit or memory constraints, start with the smallest model size or choose alternative algorithms or methods with significantly lower time or space complexity instead of using a neural network. You can then iteratively refine and optimize the model in later stages.
     {% endif %}
-
+    
+    {% endif %}
+    
     {% if last_exp_diff %}
     # Here are the differences between the latest version of implementation and the current best version of implementation
     It is presented in diff format, highlighting changes from the best version to the latest version.
     {{ last_exp_diff }}
     {% endif %}
-    {% endif %}
-
 
 extra_requirement:
   model: |-
@@ -250,6 +252,7 @@ extra_requirement:
     - If you decide to tune an existing model, select the existing model file and generate a new hypothesis.
     - If you decide to add a new model, specify the type of model you would add and generate a new hypothesis related to the new model.
 
+    When building the model, if the runtime permits, consider incorporating hyperparameter search methods to improve performance.
 
 component_gen:
   system: |-