Files
NexQuant/rdagent/app/utils/prompts.yaml
you-n-g f78175b37a feat: refactor for general data science (#498)
* refine ds modal for more cases: eval and es

* update model template

* prompts for model and ensemble

* fix a bug

* fix a bug

* init: ds workflow evovingstrategy

* Adding ensemble (#505)

* Initial Draft

* Updating logic for init

* Revising

* Successful Testing

* Updating to use the latest & right class

* bug: bug-fixing for testing

* data science loop changes

* data science loop base

* ds loop feedback

* fix

* remove measure_time because it's duplicated (in LoopBase)

* add the knowledge query for data_loader & feature

* edit ds workflow evaluator

* data_loader bug fix

* stop evolving when all tasks completed

* llm app change

* fix break all complete strategy

* Adding queried knowledge (#508)

Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>

* fix loop bug

* ds workflow evaluator; test; refine prompts

* workflow spec

* fix ci

* feature task changes

* ds loop change

* fix a bug in feat

* add query knowledge for model and workflow

* llm_debug info(for show) using pickle instead of json

* remove NextLoopException

* loop change

* coder raise CoderError when all sub_tasks failed

* rename code_dict to file_dict in FBWorkspace

* add CoSTEER unittest

* now show self.version in Task.get_task_information(), simplify CoSTEER sub tasks definition

* remove some properties in ModelTask, add model_type in it.

* fix llm app bug

* llm web app bug fix

* ds loop bug fix

* fix: give component code to feature&ens eval

* loop catch error bug

* rename load_from_raw_data to load_data

* feat: Add debug data creation functionality for data science scenarios

* support local folder (#511)

* support local folder

* remove unnecessary random

* KaggleScen Subclass

* small fix

* use template for style description

* update default scen to kaggle

* update sample data script

* make sure frac < 1

* fix a bug

* feature spec changes

* fix

* changeimport order

* clear unnecessary std outputs

* fix a typo

* create sample folder after unzip kaggle data

* feature/model test script update

* Align the data types across modules.

* fix a bug in model eval

* show line number

* move sample entry point to app

* spec & model prompt changes

* Refine the competition specification to address the data type problem and the coherence issue.

* fix some bugs

* add file filter in FBworkspace.code property

* support non-binary prediction

* avoid too much warnings

* fix a bug in ensemble module

* filtered the knowledge query in all modules

* delete RAG in idea proposal

* refine the code in ensemble

* show exp workspace in llm_st

* exp_gen bug fix

* feedback bug fix

* use `feature` instead of `feat01`

* Trace & method of judging if exp is completed change

* fix a bug in package calling and execute ci

* fix code

* bug fix

* bug fix

* fix a bug

* fix some bugs

* fix a bug

* refactor: Enhance error handling and feedback in data science loop

* support different use_azure on chat and embedding models

* multi-model proposal logic

* fix a small syntax error

* loopBase and some changes

* ensemble scores change

* fbworkspace.code -> .all_codes

* use all model codes in workflow coder

* check scores.csv's keys(model_names)

* model name changes

* add a todo in  ensemble test

* sota_exp changes

* give model info in exp gen

* add runner time limit

* config using debug data or not in evals

* exp to feedback base

* add feature code when writing model task

* small problem

* copying during sampling

* update

* refactor: Simplify code handling and improve workspace management

* model part output fix

* print model's execution time

* bug fix

* ensemble test fix

* ens small change

* ens_test bug fix

* Refine partial expansion logic to display only a few subfolders when their structure is uniform, improving readability in nested directories.

* several update on prompts

* sample subfolders

* Filter the stdout after code execution to remove irrelevant information e.g. progress bars, whitespace characters, excessive line breaks.

* Add some more prompts and comments

* several update on the first init rounds

* model timeout as error

* fix pattern of getting model codes in workspace

* small bux fix on model prompts

* remove get_code_with_key since we have regex pattern

* fix: Correct tqdm progress bar update logic in LoopBase class

* feat: Add diff generation and enhance feedback mechanism in data science loop

* update some fix to model and workflow prompts

* refine the logic of progress bar filter

* add last_successful_exp in exp_gen

* fix a one line bug

* add a hint in prompt

* fix data sample for bms

* fix data sample for bms

* hypothesis small fix

* crawler readme update

* fix component gen

* fix bug

* annotation change

* load description.md if it exists

* refactor: Simplify SOTA description handling in feedback and prompts

* refactor: Use shared templates for feedback and experiment descriptions

* change webapp for model codes changes

* update proposal

* add timeout message for docker run output

* fix

* refine the code in docker time processing

* use .shape instead of len() when do shape eval

* won't change size during iteration

* support bson sample

* sample support jsonl and bson

* add former_code to coder prompts

* a little speed us in debug data creating

* filter progress bar when eval ens and main

* avoid costeer makes no change to former code

* fix several log error

* add timeout judge threshold

* fix some bugs in the evaluation of component output shapes

* File structure for supporting litellm (#517)

Co-authored-by: Young <afe.young@gmail.com>

* ignore submission and show processing

* ignore submission and show processing

* add efficiency notice

* refactor: Enhance error message with detailed feedback summary

* refactor: Simplify component handling in DSExpGen class

* refactor: Update code structure and add docstring for clarity

* reserve one sample to each label in data sampling

* add Evaluation info

* refine costeer code to avoid giving same code twice

* use raw_description as plain text

* add a prompt hint to avoid same dict key

* model task name bug in first model exp gen

* fix a typo

* add some debug info in costeer tests

* task init change

* enhance data sampling

* refine the code in data_loader

* more reasonable loop

* fix a bug in data folder description

* add error msg & traceback to execution feedback

* fix llm error msg detection

* add task information to costeer eval & add cache to docker run(use zipfile to store the whole workspace)

* fix CI first round

* fix CI second round

* use txt to store test script to avoid pytest

* remove zipfile in requirements

* add azure.identity to requirements

* ignore debug web page

* component test changes

* remove redundent task_desc in model coder

* feat: Add APE module and prompts for automated prompt engineering

* fix: Update .gitignore and improve text formatting in eval.py

* refactor: Update print output and improve code comments and imports

* style: Fix string formatting and import order in ape.py and fmt.py

* exclude ape

* add a data folder notice

* reduce unnecessary output to stdout

* refine the code of describe_data_folder

* fix ci

* style: streamlit style update (#522)

* streamlit style update

* fix import

* fix format

* fix llm_st loop progress bar

* debugapp small change

* fix model str

* refine some prompts

* fix model str

* fix CI

* refine the logic associated with the data_folder

* fix ci

* small change

* set filter_progress_bar as default in execute

* model proposal with workflow

* add submission check in workflow eval

* fix bug

* small change

* fix CI

* fix CI

* refactor: Move generate_diff to utils and update DSExpGen logic

* more reasonable prompt describing metric direction

* fix a minor jinja2 bug

* quick fix exp_gen bugs

* fix the following bug

* fix

* fix some bugs

* remove workflow from model

* add pending_tasks_list in data science to enable coding model and workflow

* refine the code for handling JSON-formatted data descriptions

* assert with information

* ensure correct csv file name

* add logging to help record the output

* log competition

* add log tag for debug llm app

* test: Test ds refactor ll (#523)

* fix bugs to former scenario

* fix a bug because coding in rdloop changed

* fix the bug when feedback gets no hypothesis

* fix trace structure

* change all trace hist when merging hypothesis to experiments

* ignore some error in ruff

* fix kaggle scenario bugs

* refine one line

* another bug

* another small bug

* fix ui bugs

* chage kaggle  train.py path

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>

* fix CI

* Update rdagent/app/data_science/loop.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add samplecsv into spec prompts

* fix CI

---------

Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: yuanteli <1957922024@qq.com>
Co-authored-by: Xisen Wang <118058822+xisen-w@users.noreply.github.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: Tim <illking@foxmail.com>
Co-authored-by: 炼金术师华华 <37462254+YeewahChan@users.noreply.github.com>
Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-01-17 22:53:05 +08:00

120 lines
5.0 KiB
YAML

ape:
system: |-
We'll provide you with a pair of Chat QA about data science.
We are creating solutions for a Kaggle Competition based on the answers.
Good questions are crucial for getting good answers.
Please suggest how to improve the question.
You can analyze based on these aspects:
- Is the question complete (is all the information needed to answer the question provided?)
The conversation will be provided in the following format:
<question>
<part1>
...text to describe the question...
</part1>
<part2>
...text to describe the question...
</part2>
</question>
<answer>
...text to describe the answer.
</answer>
You response should be very concorete and concise(less than 20 words) and focuse on the mentioned aspects, like
```
Info Missing: the question ask for changing code, but it does not provide the description of current code.
```
Please be very conversatiive when you propose improvements. Only propose improvements when it becomes impossible to give the answer.
Don't propose conerete modifications
user: |-
<question>
<part1>
{{system}}
</part1>
<part2>
{{user}}
</part2>
</question>
<answer>
{{answer}}
</answer>
optional: |-
If you want to suggest modification on the question. Please follow the *SEARCH/REPLACE block* Rules!!!! It is optional.
Please make it concise and less than 20 lines!!!
# *SEARCH/REPLACE block* Rules:
Every *SEARCH/REPLACE block* must use this format:
1. The *FULL* file path alone on a line, verbatim. No bold asterisks, no quotes around it, no escaping of characters, etc.
2. The opening fence and code language, eg: ```python
3. The start of search block: <<<<<<< SEARCH
4. A contiguous chunk of lines to search for in the existing source code
5. The dividing line: =======
6. The lines to replace into the source code
7. The end of the replace block: >>>>>>> REPLACE
8. The closing fence: ```
Use the *FULL* file path, as shown to you by the user.
Every *SEARCH* section must *EXACTLY MATCH* the existing file content, character for character, including all comments, docstrings, etc.
If the file contains code or other data wrapped/escaped in json/xml/quotes or other containers, you need to propose edits to the literal contents of the file, including the container markup.
*SEARCH/REPLACE* blocks will *only* replace the first match occurrence.
Including multiple unique *SEARCH/REPLACE* blocks if needed.
Include enough lines in each SEARCH section to uniquely match each set of lines that need to change.
Keep *SEARCH/REPLACE* blocks concise.
Break large *SEARCH/REPLACE* blocks into a series of smaller blocks that each change a small portion of the file.
Include just the changing lines, and a few surrounding lines if needed for uniqueness.
Do not include long runs of unchanging lines in *SEARCH/REPLACE* blocks.
Only create *SEARCH/REPLACE* blocks for files that the user has added to the chat!
To move code within a file, use 2 *SEARCH/REPLACE* blocks: 1 to delete it from its current location, 1 to insert it in the new location.
Pay attention to which filenames the user wants you to edit, especially if they are asking you to create a new file.
If you want to put code in a new file, use a *SEARCH/REPLACE block* with:
- A new file path, including dir name if needed
- An empty `SEARCH` section
- The new file's contents in the `REPLACE` section
To rename files which have been added to the chat, use shell commands at the end of your response.
If the user just says something like "ok" or "go ahead" or "do that" they probably want you to make SEARCH/REPLACE blocks for the code changes you just proposed.
The user will say when they've applied your edits. If they haven't explicitly confirmed the edits have been applied, they probably want proper SEARCH/REPLACE blocks.
You are diligent and tireless!
You NEVER leave comments describing code without implementing it!
You always COMPLETELY IMPLEMENT the needed code!
ONLY EVER RETURN CODE IN A *SEARCH/REPLACE BLOCK*!
Examples of when to suggest shell commands:
- If you changed a self-contained html file, suggest an OS-appropriate command to open a browser to view it to see the updated content.
- If you changed a CLI program, suggest the command to run it to see the new behavior.
- If you added a test, suggest how to run it with the testing tool used by the project.
- Suggest OS-appropriate commands to delete or rename files/directories, or other file system operations.
- If your code changes add new dependencies, suggest the command to install them.
- Etc.
Here is a example of SEARCH/REPLACE BLOCK to change a function implementation to import.
<<<<<<< SEARCH
def hello():
"print a greeting"
print("hello")
=======
from hello import hello
>>>>>>> REPLACE
# - Is there any ambiguity in the question?