20 Commits

Author SHA1 Message Date
TPTBusiness cbe1c52e00 refactor: rename project from Predix to NexQuant
Rename all source files, scripts, tests, documentation, and configuration
from Predix/predix to NexQuant/nexquant across the entire codebase.
2026-05-09 17:48:22 +02:00
TPTBusiness 6d514d3ed5 test(backtest): add RiskMgmt and OOS walk-forward validation tests
Covers backtest_signal_riskmgmt leverage caps, zero-signal, IS/OOS split keys,
bar counts, OOS independence from IS losses, and Monte Carlo permutation
tests (marked slow, excluded from default pytest run).

Also excludes slow-marked tests from default addopts in pyproject.toml.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 18:26:32 +02:00
TPTBusiness 2a011d262e chore: Simplify pre-commit to mandatory hooks only
- Remove optional code quality hooks (black, isort, ruff, mypy, toml-sort)
  * These blocked commits when tools not installed
  * Users can run them manually when needed
- Keep only MANDATORY hooks:
  * Integration Tests (60 tests, ~7.5s)
  * Bandit Security Scan
- Both MUST pass before every commit
2026-04-03 12:33:30 +02:00
TPTBusiness 11b347d0e7 chore: prepare repository for Predix public release
- Rebrand from RD-Agent to Predix for EUR/USD quantitative trading
- Update all documentation to English
- Remove Microsoft-specific references
- Clean up temporary files and backups
- Update LICENSE, README, and configuration for PredixAI organization

Breaking changes:
- Project name changed from 'rdagent' to 'predix' in pyproject.toml
- All Microsoft and RD-Agent branding replaced with Predix
- Documentation completely rewritten for EUR/USD focus

Documentation:
- README.md: Professional English documentation with installation, quick start, CLI reference
- CHANGELOG.md: Cleaned up, references upstream RD-Agent for historical changes
- CODE_OF_CONDUCT.md: Switched to Contributor Covenant v2.0
- SECURITY.md: Predix-specific vulnerability reporting process
- SUPPORT.md: Updated support channels (nico@predix.io, GitHub Discussions)
- CONTRIBUTING.md: Adapted for Predix project
- docs/: Sphinx configuration updated for Predix branding

Configuration:
- pyproject.toml: Updated project metadata, keywords, URLs for PredixAI
- .gitignore: Comprehensive Python/gitignore template
- Makefile: Updated CI pages URL
- setup_predix_eurusd.sh: Translated to English

Cleanup:
- Deleted log files, caches, __pycache__ directories
- Removed backup files (*.backup_*)
- Cleaned web/node_modules
2026-03-29 00:10:26 +01:00
BarryC 7cd64a26fd feat(rl): add AutoRL-Bench framework and benchmark integrations (#1348)
* feat: rdkit for chemcotbench

* update qwen2.5&llama3.1 context

* fix: force failure on validation error and remove try/except in validator

* feat: unified error sample extraction (with test scripts)

* feat: set conda cache with .env

* feat: skip data eval if data pass in last evo

* fix: rm redundant param

* fix ui bug

* refactor: centralize assign_code_list_to_evo in MultiProcessEvolvingStrategy

* feat: add test_params.yaml generation and workspace cleanup improvements for finetune

* refactor: replace get_clear_ws_cmd with clear_workspace and update prompts for hard check criteria

* add bioprobench dataset

* fix: handle commas in training config extraction and refactor prompt includes

* bioprobench description

* add bioprobench readme

* feat: merge lora adapter for blackwell gpu

* feat: support for multi benchmarks in one job

* change dfficult aware content for training

* update difficulty-aware and logging principles

* fix: resolve variable name conflict in FTRunnerEvaluator

* set job id accuracy to minute

* feat(ui): display one selected metric per benchmark

* feat: store sota exp, and fix ws_ckp bug

* fix: truncate data.json in feedback

* fix: opencompass data for conda env

* fix: save only the last model

* feat: set log path and ws path

* fix: set overwrite_cache to avoid lock contention(through injecting params)

* feat: redirect stdout to file in localenv

* add pickle cache to dataset desc

* fix CI

* fix: remove redundant wrapper

* feat: set python_unbuffered

* move redirect stdout to env run

* fix a small bug

* move model folder

* feat(ui): display benchmark baseline

* fix: enrich scenario and benchmark description

* fix: rewrite runner eval to accept easier

* feat: compare with baseline when no SOTA

* update tablebench readme

* fix: switch back to single benchmark (for baseline)

* feat(ui): add ws path in ui

* refactor: update SOTA tracking to use DAG traversal and parent selection

* fix: prioritize local_selection in trace and refactor sibling retrieval logic

* refactor: unify error handling in feedback generation and update workspace injection

* feat: add skip_loop_error_stepname to control error skip step in LoopBase

* fix: set local_selection to NEW_ROOT for experiments without parent

* feat: set different ports for jobs

* feat: set different ports for jobs

* feat: add upper data size limit for LLM fine-tuning and update related prompts

* fix: replace get_truncated_stdout() with stdout for consistent output handling

* refactor: remove data.json from cache and workspace logic, focus on script-based reuse

* fix: rm target_scenario

* feat: add selective cache extraction and custom cache key for data processing

* fix(ui): bug when displaying tablebench

* fix: filter config in dataset_info.json

* feat: add test set, set valid set

* feat(ui): update test score, and set color for final decision

* feat: add test score for baseline and update ui

* fix: use [-100:] as test range

* feat: update data_stats in runner

* feat: wait for opencompass init when run multi jobs

* fix: adjust test&valid split

* feat: force to generate COT(with <think> token), and add answer format in scenarios.json

* feat: improve ui

* fix: unify benchmark volume mounts and set extra_volumes for conda env

* fix(ui): number color

* fix: update GPU memory handling to use total memory in GB and streamline code

* fix: set use_cot_postprocessor

* feat: add env_dict to config classes and merge env vars in Env run

* fix: let coder obey proposal

* fix(ui): direction bug and update chemcot core metirc

* fix: set consistent benchmark mount points and env vars for docker and conda

* fix: addintional target for LoRA

* feat: workspace dir log for benchmark running

* fix: tableInstruct path bug and update benchmark description

* feat: timeout for whole job

* fix: align FinanceIQ import to opencompass

* feat: use llm_judge for FinanceIQ

* feat: switch to turn on <think> or not

* feat: using scripts to redirect stdout, and run in different windows

* feat: sync litellm log

* fix: gpu memory format

* fix: escape special characters in benchmark desc

* fix: set data processing timeout to 1h

* feat: set valid_loss and save_best_model

* fix: inject timeout and stage

* fix: loss history extract logic

* feat: inject output dir

* feat: inject eval batch size

* feat: inject save_total_limit

* feat: update data prompt

* fix:  escape shell special characters

* fix: tablebench visualization UI

* fix: move implementation validation to coder, and ignore injected params

* docs: add README for RL-PostTraining evaluation system

* Add AutoRL-Bench evaluation framework for RL post-training

* Add architecture documentation

* docs: update architecture and interface documentation for AutoRL-Bench

* improve doc

* fix

* refactor: YAML配置驱动

* feat: add RL Docker env, workspace test, and update project structure

* feat: 重命名 autorl_bench, 新增 RLWorkspace, 配置 Docker extra_volumes

* Add eval-only AutoRL-Bench pipeline

* sturcture clean

* docs: add autorl_bench README

* feat(rl): Implement RL post-training agent scaffold and example

* refactor: simplify RL scenario classes and update RL CoSTEER integration

* feat(rl): 调通 scaffold,mock 数据跑完 5 步循环

* feat(rl): 接入 LLM 生成代码,支持 model_path 传递

* feat(rl): Docker 执行框架,RLWorkspace.run() + RLPostTrainingRunner

* feat(rl): LLM 生成假设/反馈,完整 loop 跑通

* feat: add RL post-training entry point with configurable options

* refactor: simplify RL proposal and trace classes, update config and docs

* Update rl eval autorl_bench layout

* Update RL workflow and evaluation setup

* Integrate AutoRL-Bench evaluation in RL workflow

* feat(rl): 添加 --base-model/--benchmark CLI 参数,简化 RLTask

* feat(rl): Docker 环境动态选择 + example_agent 完整训练评测流程(无llm)

* fix(rl): 修复 feedback 传递 + 添加 verl 依赖

* refactor: remove unused validate in BenchmarkAdapter and add core utils module

* feat(rl): UI

* Refactor autorl_bench layout and docker entrypoint

* autorl_bench: add aider autoloop tool

* feat(rl): environment docker

* refactor: simplify aider autoloop tooling

* chore: update misc files

* feat(rl): yaml-driven dataset download & auto-download on startup

* feat(rl): yaml-driven dataset download & auto-download on startup

* Refactor RL eval runner and clean up

* Simplify RL eval runner and env

* rl: include litellm in RL docker image

* feat(rl): unified resource path & model repo_id structure

* feat(rl): refactor eval with OpenCompass & add training code template

* feat(rl): refactor eval with OpenCompass & add training code template

* feat(rl): delete test bench

* docs: add benchmark interface notes and TODOs for unified evaluation

* feat(rl): unified benchmark eval interface + shared configs

* feat(rl): 优雅

* feat(rl): prompt prososal+coder improve

* feat(rl): fix eval

* fix(rl): docker

* fix(rl): eval

* v 1.0 tmep

* benchmark v1.0

* benchmark v1.1

* benchmark v1.1: grading日志+代码去重

* benchmark v1.1: grading日志+代码去重

* benchmark v1.1: grading日志+代码去重+task description

* benchmark v1.2: fix

* benchmark v1.3: fix,example-agent ok,rdagent test,openhands develop

* benchmark v1.4: fix,example-agent ok,rdagent ok,openhands develop

* benchmark : add alfworld

* benchmark : update readme

* benchmark : update readme

* benchmark :

* chore: add eval bypass block and mark TODO in grading server

* benchmark

* benchmark

* benchmark

* benchmark

* alfworld

* alfworld

* benchmark

* rdagent

* rdagent

* benchmark

* benchmark:ui

* benchmark:delete docker + log

* 1

* alfworld

* ui

* alfworld

* readme

* alfworld

* parallex

* alfworld

* run

* eval gpu

* alfworld

* alfworld

* fix conda init in start.sh for non-interactive shells

Fallback to common miniconda paths when conda is not in PATH.
Fixes B200 pod startup failure (conda: command not found).

Made-with: Cursor

* simplify start.sh: read TRAINING_PYTHON from .env

No more conda detection logic. Just set TRAINING_PYTHON in .env.
Fallback to conda only if not set.

Made-with: Cursor

* use OPENHANDS_PYTHON from .env to run agent

start.sh now uses OPENHANDS_PYTHON for main.py execution,
since the parent process may be in a different conda env.

Made-with: Cursor

* feat: register OpenCode agent into autorl_bench framework

- Add agents/opencode/ with config.yaml, start.sh, README.md
- Include opencode-rl pipeline code (pipeline/, runner_fsm/, benchmarks/)
- Merge opencode-rl dependencies into autorl_bench requirements.txt
- Remove separate venv requirement, share main environment

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update opencode agent, benchmarks, and eval configs

- Sync opencode-rl runner_fsm with latest simplifications
- Add smith benchmarks integration
- Update opencompass configs and server with GPU support + error handling

* Update OpenCode agent docs for external opencode-rl integration

- Document external repo architecture (opencode-rl as independent plugin)
- Add setup instructions for cloning and configuring opencode-rl
- Add architecture diagram showing RD-Agent ↔ opencode-rl interaction
- Document OPENCODE_RL_ROOT for custom paths

* feat: add smith benchmark discovery and per-sample evaluator

- Add smith/ module for dynamic benchmark discovery from rl-smith
- Add PerSampleEvaluator for per-sample scoring via vLLM
- Update utils.py to support script-based data download for smith benchmarks
- Update opencode agent config

* enforce RL-only in instructions.md; remove embedded opencode-rl

- instructions.md: prohibit SFT, require RL (GRPO/PPO) for all benchmarks
- remove agents/opencode/opencode-rl/ (runtime uses external OPENCODE_RL_ROOT)

Made-with: Cursor

* comment out OpenCode-only deps in requirements.txt

openai, httpx, python-dotenv, tenacity are for OpenCode agent's
separate environment. Keep peft and pydantic as shared deps.

Made-with: Cursor

* refactor: extract _kill_process_group, narrow exception catches

- run.py: replace 2x nested 3-level try/except with shared
  _kill_process_group() using loop + specific exceptions
- server.py: except Exception → except (RuntimeError, ValueError, OSError)
- utils.py: except Exception → except requests.ConnectionError

Made-with: Cursor

* move kill_process_group to core/utils for reuse

Extract from run.py into core/utils.py so other runners
can also use it. Exported via core/__init__.py.

Made-with: Cursor

* add comments to run.py for workspace isolation and signal handling

Made-with: Cursor

* remove OpenCode-only deps from requirements.txt entirely

Made-with: Cursor

* allow SFT in instructions, RL as ultimate goal

Made-with: Cursor

* add workspace isolation rules to instructions.md

Use relative paths, forbid cd outside workspace, ignore symlink targets.

Made-with: Cursor

* update opencode start.sh: use OPENCODE_PYTHON, add PATH for opencode CLI, remove unsupported args

Made-with: Cursor

* opencode start.sh: pass --run-dir to use AutoRL-Bench workspace

Ensures OpenCode-FSM-Runner writes outputs into the workspace prepared
by AutoRL-Bench instead of creating its own runs/ directory.

Made-with: Cursor

* opencode start.sh: prepend training env bin to PATH

Ensures LLM agent bash calls (e.g. python3 -c "from trl import ...")
resolve to the correct training environment, instead of relying on
parent shell conda activation.

Made-with: Cursor

* opencode start.sh: restore --max-retries and --eval-timeout for opencode-rl

Made-with: Cursor

* add humaneval benchmark

* Replace import * cleanup hack with explicit imports in OpenCompass config

- Resolve dataset variable names via importlib before generating config,
  so the template uses `from xxx import datasets` instead of `import *`
- Remove the fragile runtime cleanup hack that set leaked modules to None
- Increase OpenCompass timeout from 3600s to 7200s
- Fix score parsing to average across multiple subdatasets

* refine opencompass config file generating

* add humaneval benchmark dependency instructions

human-eval package requires clone from open-compass/human-eval with
a one-line patch to relax assertion for partial evaluation (test split only).

Made-with: Cursor

* fix: sanitize user-provided paths in RL UI (CodeQL)

* fix: resolve user path relative to safe root (CodeQL)

* fix: use Copilot-suggested path sanitization pattern (CodeQL)

* fix: normalize and reject absolute user paths (CodeQL)

* Fix training params, vLLM OOM cleanup, OpenCompass score parsing, and baseline cache logic

* fix: add setuptools<75 to requirements for opencompass pkg_resources dependency

uv venv does not include setuptools by default, causing OpenCompass baseline
evaluation to fail with "No module named 'pkg_resources'".

Made-with: Cursor

* webshop

* feat(autorl_bench): improve smith benchmark integration and evaluator robustness

- Add smith benchmark docs to README: usage examples, discovery mechanism, SMITH_BENCH_DIR
- Improve PerSampleEvaluator: vLLM GPU cleanup, test_range slicing
- Refactor server.py: extract grading server from utils
- Fix OpenCompass score parsing and baseline cache logic

* fix: add smart fallback for OpenCompass dataset variable resolution

When build_dataset_imports_explicit() fails to import an OpenCompass
dataset module (common in grading server subprocess), it now guesses
the correct variable name from the module path convention instead of
falling back to empty names (which causes import * and breaks BBH
due to leaked file handle objects).

* revert: restore opencompass.py to pre-modification state

Revert vLLM pid cleanup, dash-value checks, and metric-based
score parsing added in 31caff2f and bb32e555.

* keep metric-aware score parsing in opencompass; add baseline column to UI

- opencompass.py: retain metric-type filtering (accuracy/score) instead
  of naive averaging, avoids polluting scores with pass/timeout counters
- ui.py: add Baseline column to Agent Summary table

Made-with: Cursor

* fix: handle non-string answers in extract_answer to prevent TypeError

arc_agi and other benchmarks can have non-string answer fields (e.g. lists),
which caused a crash in re.search(). Adding str() coercion fixes this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update deepsearch qa tasks

* fix: benchmark evaluation reliability (B1-B4)

- B1: auto-detect LoRA adapters and enable vLLM LoRA mode (read base_model from adapter_config.json)
- B2: serialize evaluations with threading.Lock to prevent GPU contention
- B3: cache eval results by model_path to deduplicate concurrent submissions
- B4: propagate error details from OpenCompass to agent (non-numeric scores, load failures)

Made-with: Cursor

* fix(B1): reject LoRA adapter submissions with clear merge instructions

- opencompass.py: detect adapter_config.json and return error with
  merge_and_unload() instructions instead of broken vLLM LoRA mode
- instructions.md: add requirement to submit full merged models
- opencompass_template.yaml: remove unused is_lora/lora_path params

Made-with: Cursor

* update chat completion

* update

* update deepsearch

* md

* codex + benchmark update

* codex + benchmark update

* codex + benchmark update

* codex

* codex

* fix: grading server cache key includes mtime to detect model overwrites

Previously cache used only resolved_path, so overwritten models at the
same path returned stale scores. Now cache key = path@max_mtime so
re-evaluation is triggered when model files change.

Made-with: Cursor

* feat: add gemini/claude agent scaffolds, fix codex binary path

- codex/start.sh: use CODEX_BIN env var instead of bare 'codex'
- Add gemini/ and claude/ agent directories with config.yaml and start.sh

Made-with: Cursor

* chore: remove copied human_readable_trace.py from PostTrainBench

Made-with: Cursor

* update evaluation

* benchmark

* Fix log cleanup and OpenHands env

* fix: webshop env pth problem

* benchmark alpacaeval

* style(rl): apply auto-lint fixes

* fix(rl): address CI and CodeQL issues

* fix(rl): make autorl bench imports CI-safe

---------

Co-authored-by: Qizheng Li <jenssenlee@163.com>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
Co-authored-by: chelsea97 <zhuowbrown@gmail.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: sakura657 <yctangcse@gmail.com>
Co-authored-by: shatianming5 <tianming.sha@stonybrook.edu>
Co-authored-by: Yeyuqing0913 <shatianming4@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 15:49:30 +08:00
amstrongzyf 51f5bc9e11 fix: revert to v10 setting (#1220)
* feat: add runner_patience

* task gen prompts and torch version

* fix ensemble time bug

* small bug

* lint

* add torch

* fiix: remove useless code and change formula

* fix: rename parameter

---------

Co-authored-by: jingyuanlm <842442862@qq.com>
2025-09-03 16:37:33 +08:00
Linlang cd5681ea96 fix: fix kaggle templates path error (#747)
* fix kaggle templates path error

* add env.py dependence

* reformat with lint
2025-04-04 11:20:33 +08:00
Xu Yang ed53af4c65 feat: integrate azure deepseek r1 (#591)
* fix several task & integrate deepseek R1

* fix CI

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
2025-02-13 22:20:17 +08:00
you-n-g f78175b37a feat: refactor for general data science (#498)
* refine ds modal for more cases: eval and es

* update model template

* prompts for model and ensemble

* fix a bug

* fix a bug

* init: ds workflow evovingstrategy

* Adding ensemble (#505)

* Initial Draft

* Updating logic for init

* Revising

* Successful Testing

* Updating to use the latest & right class

* bug: bug-fixing for testing

* data science loop changes

* data science loop base

* ds loop feedback

* fix

* remove measure_time because it's duplicated (in LoopBase)

* add the knowledge query for data_loader & feature

* edit ds workflow evaluator

* data_loader bug fix

* stop evolving when all tasks completed

* llm app change

* fix break all complete strategy

* Adding queried knowledge (#508)

Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>

* fix loop bug

* ds workflow evaluator; test; refine prompts

* workflow spec

* fix ci

* feature task changes

* ds loop change

* fix a bug in feat

* add query knowledge for model and workflow

* llm_debug info(for show) using pickle instead of json

* remove NextLoopException

* loop change

* coder raise CoderError when all sub_tasks failed

* rename code_dict to file_dict in FBWorkspace

* add CoSTEER unittest

* now show self.version in Task.get_task_information(), simplify CoSTEER sub tasks definition

* remove some properties in ModelTask, add model_type in it.

* fix llm app bug

* llm web app bug fix

* ds loop bug fix

* fix: give component code to feature&ens eval

* loop catch error bug

* rename load_from_raw_data to load_data

* feat: Add debug data creation functionality for data science scenarios

* support local folder (#511)

* support local folder

* remove unnecessary random

* KaggleScen Subclass

* small fix

* use template for style description

* update default scen to kaggle

* update sample data script

* make sure frac < 1

* fix a bug

* feature spec changes

* fix

* changeimport order

* clear unnecessary std outputs

* fix a typo

* create sample folder after unzip kaggle data

* feature/model test script update

* Align the data types across modules.

* fix a bug in model eval

* show line number

* move sample entry point to app

* spec & model prompt changes

* Refine the competition specification to address the data type problem and the coherence issue.

* fix some bugs

* add file filter in FBworkspace.code property

* support non-binary prediction

* avoid too much warnings

* fix a bug in ensemble module

* filtered the knowledge query in all modules

* delete RAG in idea proposal

* refine the code in ensemble

* show exp workspace in llm_st

* exp_gen bug fix

* feedback bug fix

* use `feature` instead of `feat01`

* Trace & method of judging if exp is completed change

* fix a bug in package calling and execute ci

* fix code

* bug fix

* bug fix

* fix a bug

* fix some bugs

* fix a bug

* refactor: Enhance error handling and feedback in data science loop

* support different use_azure on chat and embedding models

* multi-model proposal logic

* fix a small syntax error

* loopBase and some changes

* ensemble scores change

* fbworkspace.code -> .all_codes

* use all model codes in workflow coder

* check scores.csv's keys(model_names)

* model name changes

* add a todo in  ensemble test

* sota_exp changes

* give model info in exp gen

* add runner time limit

* config using debug data or not in evals

* exp to feedback base

* add feature code when writing model task

* small problem

* copying during sampling

* update

* refactor: Simplify code handling and improve workspace management

* model part output fix

* print model's execution time

* bug fix

* ensemble test fix

* ens small change

* ens_test bug fix

* Refine partial expansion logic to display only a few subfolders when their structure is uniform, improving readability in nested directories.

* several update on prompts

* sample subfolders

* Filter the stdout after code execution to remove irrelevant information e.g. progress bars, whitespace characters, excessive line breaks.

* Add some more prompts and comments

* several update on the first init rounds

* model timeout as error

* fix pattern of getting model codes in workspace

* small bux fix on model prompts

* remove get_code_with_key since we have regex pattern

* fix: Correct tqdm progress bar update logic in LoopBase class

* feat: Add diff generation and enhance feedback mechanism in data science loop

* update some fix to model and workflow prompts

* refine the logic of progress bar filter

* add last_successful_exp in exp_gen

* fix a one line bug

* add a hint in prompt

* fix data sample for bms

* fix data sample for bms

* hypothesis small fix

* crawler readme update

* fix component gen

* fix bug

* annotation change

* load description.md if it exists

* refactor: Simplify SOTA description handling in feedback and prompts

* refactor: Use shared templates for feedback and experiment descriptions

* change webapp for model codes changes

* update proposal

* add timeout message for docker run output

* fix

* refine the code in docker time processing

* use .shape instead of len() when do shape eval

* won't change size during iteration

* support bson sample

* sample support jsonl and bson

* add former_code to coder prompts

* a little speed us in debug data creating

* filter progress bar when eval ens and main

* avoid costeer makes no change to former code

* fix several log error

* add timeout judge threshold

* fix some bugs in the evaluation of component output shapes

* File structure for supporting litellm (#517)

Co-authored-by: Young <afe.young@gmail.com>

* ignore submission and show processing

* ignore submission and show processing

* add efficiency notice

* refactor: Enhance error message with detailed feedback summary

* refactor: Simplify component handling in DSExpGen class

* refactor: Update code structure and add docstring for clarity

* reserve one sample to each label in data sampling

* add Evaluation info

* refine costeer code to avoid giving same code twice

* use raw_description as plain text

* add a prompt hint to avoid same dict key

* model task name bug in first model exp gen

* fix a typo

* add some debug info in costeer tests

* task init change

* enhance data sampling

* refine the code in data_loader

* more reasonable loop

* fix a bug in data folder description

* add error msg & traceback to execution feedback

* fix llm error msg detection

* add task information to costeer eval & add cache to docker run(use zipfile to store the whole workspace)

* fix CI first round

* fix CI second round

* use txt to store test script to avoid pytest

* remove zipfile in requirements

* add azure.identity to requirements

* ignore debug web page

* component test changes

* remove redundent task_desc in model coder

* feat: Add APE module and prompts for automated prompt engineering

* fix: Update .gitignore and improve text formatting in eval.py

* refactor: Update print output and improve code comments and imports

* style: Fix string formatting and import order in ape.py and fmt.py

* exclude ape

* add a data folder notice

* reduce unnecessary output to stdout

* refine the code of describe_data_folder

* fix ci

* style: streamlit style update (#522)

* streamlit style update

* fix import

* fix format

* fix llm_st loop progress bar

* debugapp small change

* fix model str

* refine some prompts

* fix model str

* fix CI

* refine the logic associated with the data_folder

* fix ci

* small change

* set filter_progress_bar as default in execute

* model proposal with workflow

* add submission check in workflow eval

* fix bug

* small change

* fix CI

* fix CI

* refactor: Move generate_diff to utils and update DSExpGen logic

* more reasonable prompt describing metric direction

* fix a minor jinja2 bug

* quick fix exp_gen bugs

* fix the following bug

* fix

* fix some bugs

* remove workflow from model

* add pending_tasks_list in data science to enable coding model and workflow

* refine the code for handling JSON-formatted data descriptions

* assert with information

* ensure correct csv file name

* add logging to help record the output

* log competition

* add log tag for debug llm app

* test: Test ds refactor ll (#523)

* fix bugs to former scenario

* fix a bug because coding in rdloop changed

* fix the bug when feedback gets no hypothesis

* fix trace structure

* change all trace hist when merging hypothesis to experiments

* ignore some error in ruff

* fix kaggle scenario bugs

* refine one line

* another bug

* another small bug

* fix ui bugs

* chage kaggle  train.py path

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>

* fix CI

* Update rdagent/app/data_science/loop.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add samplecsv into spec prompts

* fix CI

---------

Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: yuanteli <1957922024@qq.com>
Co-authored-by: Xisen Wang <118058822+xisen-w@users.noreply.github.com>
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: Tim <illking@foxmail.com>
Co-authored-by: 炼金术师华华 <37462254+YeewahChan@users.noreply.github.com>
Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-01-17 22:53:05 +08:00
you-n-g 5c061d452f feat: add entry for rdagent. (#187)
* Add entries

* update entry for rdagent

* lint

* fix typo
2024-08-08 17:10:37 +08:00
Linlang b2cf05a98c fix: fix release CI (#165)
* test fix release CI

* test fix release CI

* test fix release CI

* test fix release CI

* test fix release CI

* test fix release CI

* test fix release CI

* test fix release CI

* remove bump2version from release CI

* change code

* split release CI

* format with toml-sort

* change triggers

* Update pyproject.toml

* optimize code

* fix sort-toml error

* merge release & upload

* fix release error

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

* test release ci

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2024-08-08 00:14:15 +08:00
Linlang 06e1c705d2 fix: fix release CI error (#160)
* fix release CI error

* reset version

* fix error code

* change private_key path

* add pr title  rules

* reset version
2024-08-05 17:34:03 +08:00
Linlang ad520a1431 fix: fix_release_ci_error (#150)
* fix_release_ci_error

* refining pr templates
2024-08-02 16:15:12 +08:00
Linlang 7d279721b4 ci: change ci yml (#146)
* change ci yml

* add PULL_REQUEST_TEMPLATE

* change ci yml

* change ci yml

* Update .github/workflows/release.yml

* Update .github/workflows/release.yml

* Update .github/workflows/release.yml

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2024-08-02 12:57:34 +08:00
Linlang faa2fb03ad CI checks that can be automatically repaired (#119)
* fix isort & black & toml-sort & sphinx error

* fix ci error

* fix ci error

* add comments

* Update Makefile

* change sphinx build command

* add auto-lint

* add black args

* format with black

* Auto Linting document

* fix ci error

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Young <afe.young@gmail.com>
2024-07-26 12:12:16 +08:00
XianBW 55959c93ff Ci fix (#22)
* fix replace function of CI tool

* fix ruff errors (ignore some parts)

* add ruff rule ignore comment
2024-06-12 15:12:11 +08:00
XianBW 441ca51af9 CI tool refinement (#13)
* adjust interact and log for CI fix tool

* change method to split large code files

* refine statistics table after fixing one file

* process default --fix option when ruff check

* remove hardcode path

* add tree-sitter requirements

* replace <| |> to < > in comments, for GPT

* update mypy version

* Add Mypy Evaluator
2024-06-07 00:23:11 +08:00
Xu Yang 1e77557293 factor extraction pipeline ready (#16)
* run the code

* update code

* remove some redundant code

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-06-05 15:36:15 +08:00
XianBW 62a2f7a742 add CI fix tool to app (#10)
* add CI fix tool to app

* perform ruff safe fix

* wrap too long lines in prompts.py
2024-05-30 10:33:07 +08:00
Xu Yang 02173ffdcc add code security check CI and dependbot (#3)
* update code

* init structure

* add requirements and fix CI

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-05-09 10:35:54 +08:00