feat: out spec change for o1-preview (#666)

* add not_json batcheditout

* ensemble out_spec change

* feature out_spec change

* model out_spec change

* workflow out_spec change

* runner debugger out_spec change

* filter_progress_bar return format fix

* data_loader and spec out_spec change

* show finish_reason in llm log

* json_mode fix

* remove hardcode

* fix CI

* fix grammer

* complete PythonBatchEditOut logic

---------

Co-authored-by: yuanteli <1957922024@qq.com>

rdagent/utils/agent/ret.py

@@ -48,3 +48,22 @@ class BatchEditOut(AgentOut):
     @classmethod
     def extract_output(cls, resp: str):
         return json.loads(resp)
+
+
+class PythonBatchEditOut(AgentOut):
+
+    @classmethod
+    def get_spec(cls, with_del=True):
+        return T(".tpl:PythonBatchEditOut").r(with_del=with_del)
+
+    @classmethod
+    def extract_output(cls, resp: str):
+        code_blocks = {}
+        pattern = re.compile(r"```(.*?)\n(.*?)\n```", re.DOTALL)
+        matches = pattern.findall(resp)
+
+        for match in matches:
+            file_name, code = match
+            code_blocks[file_name.strip()] = code.strip()
+
+        return code_blocks

rdagent/utils/agent/tpl.yaml

@@ -5,9 +5,9 @@ PythonAgentOut: |-
   ```
   
 BatchEditOut: |-
-  You should return a edition that applies to multiple files in a workspace in JSON.
+  You should return an edition that applies to multiple files in a workspace in JSON.
   Except for the model file, other files should not be renamed.
-  Files that do not need to be modified do not need to be included in the returned dict.
+  Files that do not need modifications should not be included in the returned text.
 
   For example:
   Inject the code into the folder. Your file name should always contain the suffix. Your file name keys should be unique to avoid delete or replace conflicts.
@@ -19,3 +19,31 @@ BatchEditOut: |-
       <file name2> (optional): "<code>"  // indicate writing <code> into <file name2> (create a new file or update an existing file)
       {% endif %}
   }
+
+PythonBatchEditOut: |-
+  You should return an edition that applies to multiple files in a workspace.
+  Except for the model file, other files should not be renamed.
+  Files that do not need modifications should not be included in the returned text.
+
+  Response format should be like:
+  ```<file name 1>
+  <code>
+  ```
+  ```<file name 2>
+  <code>
+  ```
+  {% if with_del %}
+  ```<file name 3>
+  __DEL__
+  ```
+  {% endif %}
+  ...
+
+  NOTE:
+  - The file name should always contain the suffix.
+  - The file name should be unique to prevent conflicts during removal or replacement.
+  - To indicate writing code into a file, provide the corresponding <code> within the code block (creating a new file or updating an existing one).
+  {% if with_del %}
+  - To explicitly remove a file, provide only `__DEL__` within the code block for that file.
+  - To replace a file with a new one, first provide ` __DEL__` for the original file, then include a separate entry with new file name and the new code.
+  {% endif %}

rdagent/utils/prompts.yaml

@@ -5,7 +5,6 @@ filter_progress_bar:
     Note: About the training log message, if the log message contains useful information like loss or accuracy and it is reported in each epoch, it should not be removed. If the log message is not useful, for example, reporting nan in each iteration or just reporting the iteration number, please remove them.
 
     Respond in the following JSON format and order:
-    ```json
     {
         "needs_sub": <true/false>, 
         "regex_patterns": ["regex pattern 1", "regex pattern 2", ...]