mirror of
https://github.com/NicolasBohn/NexQuant.git
synced 2026-07-27 23:47:46 +00:00
f78175b37a
* refine ds modal for more cases: eval and es * update model template * prompts for model and ensemble * fix a bug * fix a bug * init: ds workflow evovingstrategy * Adding ensemble (#505) * Initial Draft * Updating logic for init * Revising * Successful Testing * Updating to use the latest & right class * bug: bug-fixing for testing * data science loop changes * data science loop base * ds loop feedback * fix * remove measure_time because it's duplicated (in LoopBase) * add the knowledge query for data_loader & feature * edit ds workflow evaluator * data_loader bug fix * stop evolving when all tasks completed * llm app change * fix break all complete strategy * Adding queried knowledge (#508) Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com> * fix loop bug * ds workflow evaluator; test; refine prompts * workflow spec * fix ci * feature task changes * ds loop change * fix a bug in feat * add query knowledge for model and workflow * llm_debug info(for show) using pickle instead of json * remove NextLoopException * loop change * coder raise CoderError when all sub_tasks failed * rename code_dict to file_dict in FBWorkspace * add CoSTEER unittest * now show self.version in Task.get_task_information(), simplify CoSTEER sub tasks definition * remove some properties in ModelTask, add model_type in it. * fix llm app bug * llm web app bug fix * ds loop bug fix * fix: give component code to feature&ens eval * loop catch error bug * rename load_from_raw_data to load_data * feat: Add debug data creation functionality for data science scenarios * support local folder (#511) * support local folder * remove unnecessary random * KaggleScen Subclass * small fix * use template for style description * update default scen to kaggle * update sample data script * make sure frac < 1 * fix a bug * feature spec changes * fix * changeimport order * clear unnecessary std outputs * fix a typo * create sample folder after unzip kaggle data * feature/model test script update * Align the data types across modules. * fix a bug in model eval * show line number * move sample entry point to app * spec & model prompt changes * Refine the competition specification to address the data type problem and the coherence issue. * fix some bugs * add file filter in FBworkspace.code property * support non-binary prediction * avoid too much warnings * fix a bug in ensemble module * filtered the knowledge query in all modules * delete RAG in idea proposal * refine the code in ensemble * show exp workspace in llm_st * exp_gen bug fix * feedback bug fix * use `feature` instead of `feat01` * Trace & method of judging if exp is completed change * fix a bug in package calling and execute ci * fix code * bug fix * bug fix * fix a bug * fix some bugs * fix a bug * refactor: Enhance error handling and feedback in data science loop * support different use_azure on chat and embedding models * multi-model proposal logic * fix a small syntax error * loopBase and some changes * ensemble scores change * fbworkspace.code -> .all_codes * use all model codes in workflow coder * check scores.csv's keys(model_names) * model name changes * add a todo in ensemble test * sota_exp changes * give model info in exp gen * add runner time limit * config using debug data or not in evals * exp to feedback base * add feature code when writing model task * small problem * copying during sampling * update * refactor: Simplify code handling and improve workspace management * model part output fix * print model's execution time * bug fix * ensemble test fix * ens small change * ens_test bug fix * Refine partial expansion logic to display only a few subfolders when their structure is uniform, improving readability in nested directories. * several update on prompts * sample subfolders * Filter the stdout after code execution to remove irrelevant information e.g. progress bars, whitespace characters, excessive line breaks. * Add some more prompts and comments * several update on the first init rounds * model timeout as error * fix pattern of getting model codes in workspace * small bux fix on model prompts * remove get_code_with_key since we have regex pattern * fix: Correct tqdm progress bar update logic in LoopBase class * feat: Add diff generation and enhance feedback mechanism in data science loop * update some fix to model and workflow prompts * refine the logic of progress bar filter * add last_successful_exp in exp_gen * fix a one line bug * add a hint in prompt * fix data sample for bms * fix data sample for bms * hypothesis small fix * crawler readme update * fix component gen * fix bug * annotation change * load description.md if it exists * refactor: Simplify SOTA description handling in feedback and prompts * refactor: Use shared templates for feedback and experiment descriptions * change webapp for model codes changes * update proposal * add timeout message for docker run output * fix * refine the code in docker time processing * use .shape instead of len() when do shape eval * won't change size during iteration * support bson sample * sample support jsonl and bson * add former_code to coder prompts * a little speed us in debug data creating * filter progress bar when eval ens and main * avoid costeer makes no change to former code * fix several log error * add timeout judge threshold * fix some bugs in the evaluation of component output shapes * File structure for supporting litellm (#517) Co-authored-by: Young <afe.young@gmail.com> * ignore submission and show processing * ignore submission and show processing * add efficiency notice * refactor: Enhance error message with detailed feedback summary * refactor: Simplify component handling in DSExpGen class * refactor: Update code structure and add docstring for clarity * reserve one sample to each label in data sampling * add Evaluation info * refine costeer code to avoid giving same code twice * use raw_description as plain text * add a prompt hint to avoid same dict key * model task name bug in first model exp gen * fix a typo * add some debug info in costeer tests * task init change * enhance data sampling * refine the code in data_loader * more reasonable loop * fix a bug in data folder description * add error msg & traceback to execution feedback * fix llm error msg detection * add task information to costeer eval & add cache to docker run(use zipfile to store the whole workspace) * fix CI first round * fix CI second round * use txt to store test script to avoid pytest * remove zipfile in requirements * add azure.identity to requirements * ignore debug web page * component test changes * remove redundent task_desc in model coder * feat: Add APE module and prompts for automated prompt engineering * fix: Update .gitignore and improve text formatting in eval.py * refactor: Update print output and improve code comments and imports * style: Fix string formatting and import order in ape.py and fmt.py * exclude ape * add a data folder notice * reduce unnecessary output to stdout * refine the code of describe_data_folder * fix ci * style: streamlit style update (#522) * streamlit style update * fix import * fix format * fix llm_st loop progress bar * debugapp small change * fix model str * refine some prompts * fix model str * fix CI * refine the logic associated with the data_folder * fix ci * small change * set filter_progress_bar as default in execute * model proposal with workflow * add submission check in workflow eval * fix bug * small change * fix CI * fix CI * refactor: Move generate_diff to utils and update DSExpGen logic * more reasonable prompt describing metric direction * fix a minor jinja2 bug * quick fix exp_gen bugs * fix the following bug * fix * fix some bugs * remove workflow from model * add pending_tasks_list in data science to enable coding model and workflow * refine the code for handling JSON-formatted data descriptions * assert with information * ensure correct csv file name * add logging to help record the output * log competition * add log tag for debug llm app * test: Test ds refactor ll (#523) * fix bugs to former scenario * fix a bug because coding in rdloop changed * fix the bug when feedback gets no hypothesis * fix trace structure * change all trace hist when merging hypothesis to experiments * ignore some error in ruff * fix kaggle scenario bugs * refine one line * another bug * another small bug * fix ui bugs * chage kaggle train.py path --------- Co-authored-by: Xu Yang <peteryang@vip.qq.com> * fix CI * Update rdagent/app/data_science/loop.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add samplecsv into spec prompts * fix CI --------- Co-authored-by: TPLin22 <tplin2@163.com> Co-authored-by: yuanteli <1957922024@qq.com> Co-authored-by: Xisen Wang <118058822+xisen-w@users.noreply.github.com> Co-authored-by: Bowen Xian <xianbowen@outlook.com> Co-authored-by: Xu Yang <peteryang@vip.qq.com> Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com> Co-authored-by: Tim <illking@foxmail.com> Co-authored-by: 炼金术师华华 <37462254+YeewahChan@users.noreply.github.com> Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
120 lines
5.0 KiB
YAML
120 lines
5.0 KiB
YAML
ape:
|
|
system: |-
|
|
We'll provide you with a pair of Chat QA about data science.
|
|
We are creating solutions for a Kaggle Competition based on the answers.
|
|
Good questions are crucial for getting good answers.
|
|
Please suggest how to improve the question.
|
|
You can analyze based on these aspects:
|
|
- Is the question complete (is all the information needed to answer the question provided?)
|
|
|
|
The conversation will be provided in the following format:
|
|
|
|
<question>
|
|
<part1>
|
|
...text to describe the question...
|
|
</part1>
|
|
<part2>
|
|
...text to describe the question...
|
|
</part2>
|
|
</question>
|
|
|
|
<answer>
|
|
...text to describe the answer.
|
|
</answer>
|
|
|
|
You response should be very concorete and concise(less than 20 words) and focuse on the mentioned aspects, like
|
|
```
|
|
Info Missing: the question ask for changing code, but it does not provide the description of current code.
|
|
```
|
|
Please be very conversatiive when you propose improvements. Only propose improvements when it becomes impossible to give the answer.
|
|
|
|
Don't propose conerete modifications
|
|
|
|
user: |-
|
|
<question>
|
|
<part1>
|
|
{{system}}
|
|
</part1>
|
|
<part2>
|
|
{{user}}
|
|
</part2>
|
|
</question>
|
|
|
|
<answer>
|
|
{{answer}}
|
|
</answer>
|
|
|
|
optional: |-
|
|
If you want to suggest modification on the question. Please follow the *SEARCH/REPLACE block* Rules!!!! It is optional.
|
|
Please make it concise and less than 20 lines!!!
|
|
|
|
# *SEARCH/REPLACE block* Rules:
|
|
|
|
Every *SEARCH/REPLACE block* must use this format:
|
|
1. The *FULL* file path alone on a line, verbatim. No bold asterisks, no quotes around it, no escaping of characters, etc.
|
|
2. The opening fence and code language, eg: ```python
|
|
3. The start of search block: <<<<<<< SEARCH
|
|
4. A contiguous chunk of lines to search for in the existing source code
|
|
5. The dividing line: =======
|
|
6. The lines to replace into the source code
|
|
7. The end of the replace block: >>>>>>> REPLACE
|
|
8. The closing fence: ```
|
|
|
|
Use the *FULL* file path, as shown to you by the user.
|
|
|
|
Every *SEARCH* section must *EXACTLY MATCH* the existing file content, character for character, including all comments, docstrings, etc.
|
|
If the file contains code or other data wrapped/escaped in json/xml/quotes or other containers, you need to propose edits to the literal contents of the file, including the container markup.
|
|
|
|
*SEARCH/REPLACE* blocks will *only* replace the first match occurrence.
|
|
Including multiple unique *SEARCH/REPLACE* blocks if needed.
|
|
Include enough lines in each SEARCH section to uniquely match each set of lines that need to change.
|
|
|
|
Keep *SEARCH/REPLACE* blocks concise.
|
|
Break large *SEARCH/REPLACE* blocks into a series of smaller blocks that each change a small portion of the file.
|
|
Include just the changing lines, and a few surrounding lines if needed for uniqueness.
|
|
Do not include long runs of unchanging lines in *SEARCH/REPLACE* blocks.
|
|
|
|
Only create *SEARCH/REPLACE* blocks for files that the user has added to the chat!
|
|
|
|
To move code within a file, use 2 *SEARCH/REPLACE* blocks: 1 to delete it from its current location, 1 to insert it in the new location.
|
|
|
|
Pay attention to which filenames the user wants you to edit, especially if they are asking you to create a new file.
|
|
|
|
If you want to put code in a new file, use a *SEARCH/REPLACE block* with:
|
|
- A new file path, including dir name if needed
|
|
- An empty `SEARCH` section
|
|
- The new file's contents in the `REPLACE` section
|
|
|
|
To rename files which have been added to the chat, use shell commands at the end of your response.
|
|
|
|
If the user just says something like "ok" or "go ahead" or "do that" they probably want you to make SEARCH/REPLACE blocks for the code changes you just proposed.
|
|
The user will say when they've applied your edits. If they haven't explicitly confirmed the edits have been applied, they probably want proper SEARCH/REPLACE blocks.
|
|
|
|
You are diligent and tireless!
|
|
You NEVER leave comments describing code without implementing it!
|
|
You always COMPLETELY IMPLEMENT the needed code!
|
|
|
|
|
|
ONLY EVER RETURN CODE IN A *SEARCH/REPLACE BLOCK*!
|
|
Examples of when to suggest shell commands:
|
|
|
|
- If you changed a self-contained html file, suggest an OS-appropriate command to open a browser to view it to see the updated content.
|
|
- If you changed a CLI program, suggest the command to run it to see the new behavior.
|
|
- If you added a test, suggest how to run it with the testing tool used by the project.
|
|
- Suggest OS-appropriate commands to delete or rename files/directories, or other file system operations.
|
|
- If your code changes add new dependencies, suggest the command to install them.
|
|
- Etc.
|
|
|
|
Here is a example of SEARCH/REPLACE BLOCK to change a function implementation to import.
|
|
|
|
<<<<<<< SEARCH
|
|
def hello():
|
|
"print a greeting"
|
|
|
|
print("hello")
|
|
=======
|
|
from hello import hello
|
|
|
|
>>>>>>> REPLACE
|
|
# - Is there any ambiguity in the question?
|