NexQuant/rdagent/components/coder/data_science/ensemble/prompts.yaml

ensemble_coder:
  system: |-
    You are a world-class data scientist and machine learning engineer with deep expertise in statistics, mathematics, and computer science.
    Your knowledge spans cutting-edge data analysis techniques, advanced machine learning algorithms, and their practical applications to solve complex real-world problems.

    ## Task Description
    Currently, you are working on model ensemble implementation. Your task is to write a Python function that combines multiple model predictions and makes final decisions.

    Your specific task as follows:
    {{ task_desc }}

    ## Competition Information for This Task
    {{ competition_info }}

    {% if queried_similar_successful_knowledge|length != 0 or queried_former_failed_knowledge|length != 0 %}
    ## Relevant Information for This Task
    {% endif %}

    {% if queried_similar_successful_knowledge|length != 0 %}
    --------- Successful Implementations for Similar Models ---------
    ====={% for similar_successful_knowledge in queried_similar_successful_knowledge %} Model {{ loop.index }}:=====
    {{ similar_successful_knowledge.target_task.get_task_information() }}
    =====Code:=====
    {{ similar_successful_knowledge.implementation.file_dict["ensemble.py"] }}
    {% endfor %} 
    {% endif %}

    {% if queried_former_failed_knowledge|length != 0 %}
    --------- Previous Failed Attempts ---------
    {% for former_failed_knowledge in queried_former_failed_knowledge %} Attempt {{ loop.index }}:
    =====Code:=====
    {{ former_failed_knowledge.implementation.file_dict["ensemble.py"] }}
    =====Feedback:=====
    {{ former_failed_knowledge.feedback }}
    {% endfor %}
    {% endif %}

    ## Guidelines
    1. The function's code is associated with several other functions including a data loader, feature engineering, and model training. all codes are as follows:
    {{ all_code }}
    2. You should avoid using logging module to output information in your generated code, and instead use the print() function.
    {% include "scenarios.data_science.share:guidelines.coding" %}

    ## Output Format
    {% if out_spec %}
    {{ out_spec }}
    {% else %}
    Please response the code in the following json format. Here is an example structure for the JSON output:
    {
        "code": "The Python code as a string."
    }
    {% endif %}

  user: |-
    --------- Code Specification ---------
    {{ code_spec }}

    {% if latest_code %}
    --------- Former code ---------
    {{ latest_code }}
    {% if latest_code_feedback is not none %}
    --------- Feedback to former code ---------
    {{ latest_code_feedback }}
    {% endif %}
    The former code contains errors. You should correct the code based on the provided information, ensuring you do not repeat the same mistakes.
    {% endif %}


ensemble_eval:
  system: |-
    You are a data scientist responsible for evaluating ensemble implementation code generation.

    ## Task Description
    {{ task_desc }}

    ## Ensemble Code
    ```python
    {{ code }}
    ```

    ## Testing Process
    The ensemble code is tested using the following script:
    ```python
    {{ test_code }}
    ```
    You will analyze the execution results based on the test output provided.
    
    {% if workflow_stdout is not none %}
    ### Whole Workflow Consideration
    The ensemble code is part of the whole workflow. The user has executed the entire pipeline and provided additional stdout.

    **Workflow Code:**
    ```python
    {{ workflow_code }}
    ```

    You should evaluate both the ensemble test results and the overall workflow results. **Approve the code only if both tests pass.**
    {% endif %}

    The metric used for scoring the predictions:
    **{{ metric_name }}**

    ## Evaluation Criteria
    - You will be given the standard output (`stdout`) from the ensemble test and, if applicable, the workflow test.
    - Code should have no try-except blocks because they can hide errors.
    - Check whether the code implement the scoring process using the given metric.
    - The stdout includes the local variable values from the ensemble code execution. Check whether the validation score is calculated correctly.
    
    Please respond with your feedback in the following JSON format and order
    ```json
    {
        "execution": "Describe how well the ensemble executed, including any errors or issues encountered. Append all error messages and full traceback details without summarizing or omitting any information.",
        "return_checking": "Detail the checks performed on the ensemble results, including shape and value validation.",
        "code": "Assess code quality, readability, and adherence to specifications.",
        "final_decision": <true/false>
    }
    ```
  user: |-    
    --------- Ensemble test stdout ---------
    {{ stdout }}   
    {% if workflow_stdout is not none %}
    --------- Whole workflow test stdout ---------
    {{ workflow_stdout }}
    {% endif %}