mirror of
https://github.com/NicolasBohn/NexQuant.git
synced 2026-07-27 15:37:44 +00:00
b98c9cd572dd26006df1d5291a29fe45d893f4fb
42 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 TPTBusiness
|
b98c9cd572 |
feat: Add GitHub infrastructure, CI/CD pipelines, and examples
- Add GitHub issue templates (bug, feature, docs) - Add pull request template with closed-source checklist - Add CODEOWNERS for code review assignment - Add CI/CD workflows (ci, lint, security, docs, release) - pytest + coverage with Python 3.10/3.11 matrix - Ruff + MyPy code quality checks - Bandit + safety security scanning - Sphinx docs + GitHub Pages deployment - Automated PyPI releases on tag push - Add 6 comprehensive examples + Jupyter quickstart - 01_factor_discovery.py (LLM factor generation) - 02_factor_evolution.py (factor optimization) - 03_strategy_generation.py (IC-weighted combination) - 04_backtest_simple.py (strategy backtesting) - 05_model_training.py (XGBoost/LSTM training) - 06_rl_trading_agent.py (PPO/DQN/A2C agents) - notebooks/quickstart.ipynb (interactive tutorial) - Restructure .gitignore with explicit closed-source sections - Add CI/coverage/license badges to README - Complete CLI docstrings for all 9 commands - Add data_config.yaml for quant loop configuration |
||
|
TPTBusiness
|
e0a5e6d86c |
chore: Clean up root directory - move generated files to proper locations
Moved from root to results/: - 45+ fin_quant_run*.log files (30+ GB total) → results/logs/ - selector.log → results/logs/ - .coverage → results/ - data_raw/ → results/ - .env.backup, .env.local → results/ - log/ → results/ - pickle_cache/ → results/ - predix.egg-info/ → results/ - prompt_cache.db → results/ - intraday_pv_*.h5 → git_ignore_folder/ Updated .gitignore: - *.log, fin_quant*.log - .coverage, htmlcov/ - ..bfg-report/ - .env.backup, .env.local - data_raw/ - *.h5, intraday_pv*.h5 - pickle_cache/, predix.egg-info/, __pycache__/ - log/, prompt_cache.db, strategies_new/ Root directory: 53 files → 44 files (clean) |
||
|
TPTBusiness
|
087af5b297 |
chore: Remove all JSON strategy files from history and working directory
- Deleted 204+ JSON strategy files from Git history using BFG Repo-Cleaner - Added *.json to .gitignore (excluding package*.json) - Removed all loose JSON files from root directory - Git GC completed: 13,221 objects cleaned These files were accidentally committed strategy outputs that should never have been in the repository. The actual strategy files belong in: - results/strategies_new/ (managed by .gitignore) - strategies/ (managed by .gitignore) |
||
|
TPTBusiness
|
4c45ba33ab |
feat: Realistic backtesting with OHLCV data (P5 continued)
Implemented realistic backtesting: - Load real OHLCV close prices from intraday_pv.h5 - Calculate real price returns (pct_change) - Apply signal positions to real returns with proper alignment - Include spread costs (1.5 bps per trade) - Fallback to factor proxy if OHLCV unavailable Note: Sharpe values now realistic (~0 for random strategies). Strategies need LLM to select predictive factors for positive Sharpe. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> |
||
|
TPTBusiness
|
11d96ec3bd |
feat: Data Loader module with tests (P0 complete)
Created rdagent/scenarios/qlib/local/data_loader.py: - OHLCV loading with thread-safe caching - Factor metadata loading (sorted by IC) - Factor time-series loading with alignment - Feature matrix builder - Randomized factor selection for diverse strategies - 11 tests passing Note: data_loader.py is in local/ (closed source) Test file is public to validate interface. |
||
|
TPTBusiness
|
e31a2e5405 |
fix: Handle timeout exceptions safely in predix_full_eval.py
Problem: - When factor evaluation timed out (5 min), result was undefined - save_single_result(result) crashed with NameError - factor.factor_name[:40] could fail if factor_name wasn't a string Fix: - Initialize result = None before try block - Set result to failed EvalResult on exception - Only call save_single_result() if result is not None - Use getattr(factor, 'factor_name', 'unknown') for safe access - Convert to string before slicing [:40] Now the evaluator continues even when individual factors timeout. |
||
|
TPTBusiness
|
760961d5e7 |
feat: Add complete ML pipeline with graceful degradation (closed source)
NEW ARCHITECTURE:
┌─────────────────────────────────────────────────┐
│ Phase 1: Factor Generation (Open Source) │
│ - Generate factors with LLM v3 prompt │
│ - Backtest each factor in Qlib Docker │
│ - Save to results/factors/ with code + desc │
│ - Continue until 5000+ valid factors │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ Phase 2: ML Training (Closed Source - Local) │
│ - Load top 50 factors │
│ - Train LightGBM model │
│ - Validate (IC, Sharpe) │
│ - Save to results/models/ │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ Phase 3: Portfolio Optimization (Closed Source) │
│ - Select uncorrelated factors (max corr 0.3) │
│ - Optimize weights by IC │
│ - Backtest portfolio │
│ - Save to results/portfolios/ │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ Phase 4: Strategy Generation (Closed Source) │
│ - Generate trading rules │
│ - Add risk management │
│ - Save to results/strategies/ │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ Phase 5: Iterative Improvement (Closed Source) │
│ - Use ML results as feedback │
│ - Generate better factors │
│ - Loop back to Phase 1 │
└─────────────────────────────────────────────────┘
FILES CREATED (Closed Source - NOT in Git):
- rdagent/scenarios/qlib/local/ml_trainer.py
- rdagent/scenarios/qlib/local/portfolio_optimizer.py
- rdagent/scenarios/qlib/local/quant_loop_advanced.py
- rdagent/scenarios/qlib/local/__init__.py
FILES MODIFIED (Open Source - in Git):
- rdagent/scenarios/qlib/quant_loop_factory.py
- .gitignore (added local/ exclusion)
GRACEFUL DEGRADATION:
- If local/ components don't exist → Standard loop
- If < 5000 factors → Standard loop
- If LightGBM not installed → Falls back
- Open source users get FULLY FUNCTIONAL system
USAGE:
# Standard (always works):
rdagent fin_quant
# Advanced (automatic if local components exist + 5000+ factors):
# Same command - factory auto-selects appropriate loop
|
||
|
TPTBusiness
|
68ea969c32 |
feat: Add parallel run system with API key distribution
- Add predix_parallel.py: Run multiple factor experiments concurrently
* python predix_parallel.py --runs 5 --api-keys 2 -m openrouter
* Round-robin API key distribution across available keys
* Rich live dashboard with per-run status, elapsed time, exit codes
* Graceful shutdown (Ctrl+C kills all children cleanly)
- Add --run-id parameter to predix.py for isolated single runs
* Separate log files: fin_quant_run{N}.log
* Separate results: results/runs/run{N}/
* Separate workspace: RD-Agent_workspace_run{N}/
* Separate databases per run
- Modify CoSTEER and FactorRunner for PARALLEL_RUN_ID isolation
* _save_intermediate_results uses run-specific directories
* _save_result_to_database and _write_run_log isolated per run
* _ensure_results_dirs creates run-specific paths
- Reduce max_loop from 10 to 3 for faster iterations
- Add docs/parallel_runs.md with full documentation
Tests: 103 passed
|
||
|
TPTBusiness
|
19855ef7d6 |
feat: Add model loader system (same as prompts)
New structure:
- models/standard/*.py: Default models (XGBoost, LightGBM, RandomForest)
- models/local/*.py: Your improved models (NOT in Git!)
- models/README.md: Documentation
- rdagent/components/model_loader.py: Model loader with priority
Features:
- Loader checks models/local/ first (your better models)
- Falls back to models/standard/ if no local version
- Supports versioned models (model_v2.py, model_v1.py)
- Lists available models
- Test function included
.gitignore updated:
- models/local/ excluded (your proprietary models)
- *.local.py excluded
- *_private.py excluded
Usage:
from rdagent.components.model_loader import load_model
model = load_model('xgboost_factor') # Auto-loads your better version!
Standard models included:
- xgboost_factor.py: XGBoost for tabular data
- lightgbm_factor.py: LightGBM (faster than XGBoost)
|
||
|
TPTBusiness
|
18416da2c9 |
feat: Centralize all prompts in prompts/ directory
New structure:
- prompts/standard_prompts.yaml: Default prompts (committed to Git)
- prompts/local/: Your improved prompts (NOT in Git!)
- prompts/README.md: Documentation
- rdagent/components/loader.py: Prompt loader with priority
Features:
- Loader checks prompts/local/ first (your better prompts)
- Falls back to standard_prompts.yaml if no local version
- Supports sections (system/user)
- Lists available prompts
- Test function included
.gitignore updated:
- prompts/local/ excluded (your proprietary prompts)
- *.local.yaml excluded
- *_private.yaml excluded
Usage:
from rdagent.components.loader import load_prompt
prompt = load_prompt('factor_discovery') # Auto-loads your better version!
|
||
|
TPTBusiness
|
c259a01b91 |
fix: Remove hardcoded credentials from test_benchmark_api.py
- Replace hardcoded API_KEY with os.getenv('TEST_API_KEY')
- Replace hardcoded HF_TOKEN with os.getenv('TEST_HF_TOKEN')
- Replace hardcoded API_BASE with os.getenv('TEST_API_BASE')
- Replace hardcoded MODEL with os.getenv('TEST_MODEL')
- Add test credentials patterns to .gitignore
- Fixes GitHub Security Alert #8 (py/clear-text-storage-sensitive-data)
Sensitive data is now loaded from environment variables instead of clear text.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
||
|
TPTBusiness
|
30ee6c6221 |
chore: Remove unnecessary files for v1.0.0 release
Removed: - Makefile (RD-Agent specific, not fully functional) - predix.py (duplicates rdagent CLI) - QWEN.md (internal dev guide - now in .gitignore) - TODO.md (internal tracking - now in .gitignore) Kept: - pyproject.toml (required for pip install) - start_loop.sh (useful for 24/7 trading) - data_config.yaml (central configuration) - start_loop.sh (24/7 trading) .gitignore updated to exclude internal docs. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> |
||
|
TPTBusiness
|
b25185b7b4 | chore: Add .qwen/ to .gitignore | ||
|
TPTBusiness
|
6b09b96de3 |
chore: Add QWEN.md to .gitignore
Exclude generated documentation files: - QWEN.md (local documentation) - results/ directory already excluded - Improved .gitignore structure |
||
|
TPTBusiness
|
b95bbf5900 |
feat: EURUSD Trading-Verbesserungen implementiert (Phase 1)
Neue Module für quantitatives EURUSD-Trading: 1. Hurst Exponent Regime Detection (eurusd_regime.py) - Erkennt Marktregime: MEAN_REVERSION, NEUTRAL, TRENDING - R/S-Analyse für 1min EURUSD-Daten optimiert - Trading-Empfehlungen pro Regime 2. BM25 Memory-System (eurusd_memory.py) - Speichert vergangene Trades mit Situation/Ergebnis - Findet ähnliche Setups via BM25-Ähnlichkeit - Persistente JSON-Speicherung - Historische Win-Rate Analyse 3. Volatility-Adjusted Position Sizing (eurusd_risk.py) - ATR-basierte Volatilitätsmessung - Positionsgröße nach Volatilitäts-Percentile (0.4x-1.5x) - Regime-Adjustierung (MEAN_REVERSION/TRENDING/NEUTRAL) - Korrelations-Adjustierung für Forex-Paare 4. Multi-Provider LLM Fallback (eurusd_llm.py) - Automatische Fallback-Kette bei API-Ausfällen - Provider: Qwen3.5 → DeepSeek → Gemini → Ollama - Provider-Statistiken für Monitoring - JSON-Modus für strukturierte Outputs Daten-Pipeline verbessert: - 1-Minuten-Daten korrekt in Qlib integriert - Prompts von 15min auf 1min aktualisiert - generate.py für 1min EURUSD-Daten angepasst Alle Module einzeln und im Integrationstest bestanden. |
||
|
TPTBusiness
|
11b347d0e7 |
chore: prepare repository for Predix public release
- Rebrand from RD-Agent to Predix for EUR/USD quantitative trading - Update all documentation to English - Remove Microsoft-specific references - Clean up temporary files and backups - Update LICENSE, README, and configuration for PredixAI organization Breaking changes: - Project name changed from 'rdagent' to 'predix' in pyproject.toml - All Microsoft and RD-Agent branding replaced with Predix - Documentation completely rewritten for EUR/USD focus Documentation: - README.md: Professional English documentation with installation, quick start, CLI reference - CHANGELOG.md: Cleaned up, references upstream RD-Agent for historical changes - CODE_OF_CONDUCT.md: Switched to Contributor Covenant v2.0 - SECURITY.md: Predix-specific vulnerability reporting process - SUPPORT.md: Updated support channels (nico@predix.io, GitHub Discussions) - CONTRIBUTING.md: Adapted for Predix project - docs/: Sphinx configuration updated for Predix branding Configuration: - pyproject.toml: Updated project metadata, keywords, URLs for PredixAI - .gitignore: Comprehensive Python/gitignore template - Makefile: Updated CI pages URL - setup_predix_eurusd.sh: Translated to English Cleanup: - Deleted log files, caches, __pycache__ directories - Removed backup files (*.backup_*) - Cleaned web/node_modules |
||
|
TPTBusiness
|
b39f2b7e46 |
feat: migrate to 1min EURUSD data (2020-2026)
- data_config.yaml: frequency 15min -> 1min, path -> eurusd_1min_data - patches/generate.py: updated qlib.init path and freq - patches/eva_utils.py: updated intraday label to 1min - all prompts/configs: replaced 15min references with 1min - fx_validator config, trader, graph: 1min intraday trading context |
||
|
TPTBusiness
|
30c0a9166e | chore: initial Predix state (RD-Agent fork + EURUSD setup) | ||
|
XianBW
|
14395488b9 |
feat: add a web UI server (#1345)
* update rdagent cmd * fix log error message * use multiProcessing.Process instead of subprocess.Popen * add traces to gitignore * add user interactor in RDLoop (finance scenarios) * add interactor (feedback, hypothesis) for quant scens * fix the test_end in qlib conf * add features init config, general instruction to qlib scenarios * set base features for based exp * fix bug when combine factors * move traces folder to git_ignore_folder * fix bug in features init * fix quant interact bug * fix logger warning error * bug fixes * modify rdagent logger, now it can set file output * adjust cli functions and fix logger bug * fix server port transport problem * update server_ui in cli * add web code * fix CI problem * black fix * update web ui README * update README * update readme |
||
|
BarryC
|
7cd64a26fd |
feat(rl): add AutoRL-Bench framework and benchmark integrations (#1348)
* feat: rdkit for chemcotbench * update qwen2.5&llama3.1 context * fix: force failure on validation error and remove try/except in validator * feat: unified error sample extraction (with test scripts) * feat: set conda cache with .env * feat: skip data eval if data pass in last evo * fix: rm redundant param * fix ui bug * refactor: centralize assign_code_list_to_evo in MultiProcessEvolvingStrategy * feat: add test_params.yaml generation and workspace cleanup improvements for finetune * refactor: replace get_clear_ws_cmd with clear_workspace and update prompts for hard check criteria * add bioprobench dataset * fix: handle commas in training config extraction and refactor prompt includes * bioprobench description * add bioprobench readme * feat: merge lora adapter for blackwell gpu * feat: support for multi benchmarks in one job * change dfficult aware content for training * update difficulty-aware and logging principles * fix: resolve variable name conflict in FTRunnerEvaluator * set job id accuracy to minute * feat(ui): display one selected metric per benchmark * feat: store sota exp, and fix ws_ckp bug * fix: truncate data.json in feedback * fix: opencompass data for conda env * fix: save only the last model * feat: set log path and ws path * fix: set overwrite_cache to avoid lock contention(through injecting params) * feat: redirect stdout to file in localenv * add pickle cache to dataset desc * fix CI * fix: remove redundant wrapper * feat: set python_unbuffered * move redirect stdout to env run * fix a small bug * move model folder * feat(ui): display benchmark baseline * fix: enrich scenario and benchmark description * fix: rewrite runner eval to accept easier * feat: compare with baseline when no SOTA * update tablebench readme * fix: switch back to single benchmark (for baseline) * feat(ui): add ws path in ui * refactor: update SOTA tracking to use DAG traversal and parent selection * fix: prioritize local_selection in trace and refactor sibling retrieval logic * refactor: unify error handling in feedback generation and update workspace injection * feat: add skip_loop_error_stepname to control error skip step in LoopBase * fix: set local_selection to NEW_ROOT for experiments without parent * feat: set different ports for jobs * feat: set different ports for jobs * feat: add upper data size limit for LLM fine-tuning and update related prompts * fix: replace get_truncated_stdout() with stdout for consistent output handling * refactor: remove data.json from cache and workspace logic, focus on script-based reuse * fix: rm target_scenario * feat: add selective cache extraction and custom cache key for data processing * fix(ui): bug when displaying tablebench * fix: filter config in dataset_info.json * feat: add test set, set valid set * feat(ui): update test score, and set color for final decision * feat: add test score for baseline and update ui * fix: use [-100:] as test range * feat: update data_stats in runner * feat: wait for opencompass init when run multi jobs * fix: adjust test&valid split * feat: force to generate COT(with <think> token), and add answer format in scenarios.json * feat: improve ui * fix: unify benchmark volume mounts and set extra_volumes for conda env * fix(ui): number color * fix: update GPU memory handling to use total memory in GB and streamline code * fix: set use_cot_postprocessor * feat: add env_dict to config classes and merge env vars in Env run * fix: let coder obey proposal * fix(ui): direction bug and update chemcot core metirc * fix: set consistent benchmark mount points and env vars for docker and conda * fix: addintional target for LoRA * feat: workspace dir log for benchmark running * fix: tableInstruct path bug and update benchmark description * feat: timeout for whole job * fix: align FinanceIQ import to opencompass * feat: use llm_judge for FinanceIQ * feat: switch to turn on <think> or not * feat: using scripts to redirect stdout, and run in different windows * feat: sync litellm log * fix: gpu memory format * fix: escape special characters in benchmark desc * fix: set data processing timeout to 1h * feat: set valid_loss and save_best_model * fix: inject timeout and stage * fix: loss history extract logic * feat: inject output dir * feat: inject eval batch size * feat: inject save_total_limit * feat: update data prompt * fix: escape shell special characters * fix: tablebench visualization UI * fix: move implementation validation to coder, and ignore injected params * docs: add README for RL-PostTraining evaluation system * Add AutoRL-Bench evaluation framework for RL post-training * Add architecture documentation * docs: update architecture and interface documentation for AutoRL-Bench * improve doc * fix * refactor: YAML配置驱动 * feat: add RL Docker env, workspace test, and update project structure * feat: 重命名 autorl_bench, 新增 RLWorkspace, 配置 Docker extra_volumes * Add eval-only AutoRL-Bench pipeline * sturcture clean * docs: add autorl_bench README * feat(rl): Implement RL post-training agent scaffold and example * refactor: simplify RL scenario classes and update RL CoSTEER integration * feat(rl): 调通 scaffold，mock 数据跑完 5 步循环 * feat(rl): 接入 LLM 生成代码，支持 model_path 传递 * feat(rl): Docker 执行框架，RLWorkspace.run() + RLPostTrainingRunner * feat(rl): LLM 生成假设/反馈，完整 loop 跑通 * feat: add RL post-training entry point with configurable options * refactor: simplify RL proposal and trace classes, update config and docs * Update rl eval autorl_bench layout * Update RL workflow and evaluation setup * Integrate AutoRL-Bench evaluation in RL workflow * feat(rl): 添加 --base-model/--benchmark CLI 参数，简化 RLTask * feat(rl): Docker 环境动态选择 + example_agent 完整训练评测流程（无llm） * fix(rl): 修复 feedback 传递 + 添加 verl 依赖 * refactor: remove unused validate in BenchmarkAdapter and add core utils module * feat(rl): UI * Refactor autorl_bench layout and docker entrypoint * autorl_bench: add aider autoloop tool * feat(rl): environment docker * refactor: simplify aider autoloop tooling * chore: update misc files * feat(rl): yaml-driven dataset download & auto-download on startup * feat(rl): yaml-driven dataset download & auto-download on startup * Refactor RL eval runner and clean up * Simplify RL eval runner and env * rl: include litellm in RL docker image * feat(rl): unified resource path & model repo_id structure * feat(rl): refactor eval with OpenCompass & add training code template * feat(rl): refactor eval with OpenCompass & add training code template * feat(rl): delete test bench * docs: add benchmark interface notes and TODOs for unified evaluation * feat(rl): unified benchmark eval interface + shared configs * feat(rl): 优雅 * feat(rl): prompt prososal+coder improve * feat(rl): fix eval * fix(rl): docker * fix(rl): eval * v 1.0 tmep * benchmark v1.0 * benchmark v1.1 * benchmark v1.1: grading日志+代码去重 * benchmark v1.1: grading日志+代码去重 * benchmark v1.1: grading日志+代码去重+task description * benchmark v1.2: fix * benchmark v1.3: fix，example-agent ok，rdagent test，openhands develop * benchmark v1.4: fix，example-agent ok，rdagent ok，openhands develop * benchmark : add alfworld * benchmark : update readme * benchmark : update readme * benchmark : * chore: add eval bypass block and mark TODO in grading server * benchmark * benchmark * benchmark * benchmark * alfworld * alfworld * benchmark * rdagent * rdagent * benchmark * benchmark：ui * benchmark：delete docker + log * 1 * alfworld * ui * alfworld * readme * alfworld * parallex * alfworld * run * eval gpu * alfworld * alfworld * fix conda init in start.sh for non-interactive shells Fallback to common miniconda paths when conda is not in PATH. Fixes B200 pod startup failure (conda: command not found). Made-with: Cursor * simplify start.sh: read TRAINING_PYTHON from .env No more conda detection logic. Just set TRAINING_PYTHON in .env. Fallback to conda only if not set. Made-with: Cursor * use OPENHANDS_PYTHON from .env to run agent start.sh now uses OPENHANDS_PYTHON for main.py execution, since the parent process may be in a different conda env. Made-with: Cursor * feat: register OpenCode agent into autorl_bench framework - Add agents/opencode/ with config.yaml, start.sh, README.md - Include opencode-rl pipeline code (pipeline/, runner_fsm/, benchmarks/) - Merge opencode-rl dependencies into autorl_bench requirements.txt - Remove separate venv requirement, share main environment Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update opencode agent, benchmarks, and eval configs - Sync opencode-rl runner_fsm with latest simplifications - Add smith benchmarks integration - Update opencompass configs and server with GPU support + error handling * Update OpenCode agent docs for external opencode-rl integration - Document external repo architecture (opencode-rl as independent plugin) - Add setup instructions for cloning and configuring opencode-rl - Add architecture diagram showing RD-Agent ↔ opencode-rl interaction - Document OPENCODE_RL_ROOT for custom paths * feat: add smith benchmark discovery and per-sample evaluator - Add smith/ module for dynamic benchmark discovery from rl-smith - Add PerSampleEvaluator for per-sample scoring via vLLM - Update utils.py to support script-based data download for smith benchmarks - Update opencode agent config * enforce RL-only in instructions.md; remove embedded opencode-rl - instructions.md: prohibit SFT, require RL (GRPO/PPO) for all benchmarks - remove agents/opencode/opencode-rl/ (runtime uses external OPENCODE_RL_ROOT) Made-with: Cursor * comment out OpenCode-only deps in requirements.txt openai, httpx, python-dotenv, tenacity are for OpenCode agent's separate environment. Keep peft and pydantic as shared deps. Made-with: Cursor * refactor: extract _kill_process_group, narrow exception catches - run.py: replace 2x nested 3-level try/except with shared _kill_process_group() using loop + specific exceptions - server.py: except Exception → except (RuntimeError, ValueError, OSError) - utils.py: except Exception → except requests.ConnectionError Made-with: Cursor * move kill_process_group to core/utils for reuse Extract from run.py into core/utils.py so other runners can also use it. Exported via core/__init__.py. Made-with: Cursor * add comments to run.py for workspace isolation and signal handling Made-with: Cursor * remove OpenCode-only deps from requirements.txt entirely Made-with: Cursor * allow SFT in instructions, RL as ultimate goal Made-with: Cursor * add workspace isolation rules to instructions.md Use relative paths, forbid cd outside workspace, ignore symlink targets. Made-with: Cursor * update opencode start.sh: use OPENCODE_PYTHON, add PATH for opencode CLI, remove unsupported args Made-with: Cursor * opencode start.sh: pass --run-dir to use AutoRL-Bench workspace Ensures OpenCode-FSM-Runner writes outputs into the workspace prepared by AutoRL-Bench instead of creating its own runs/ directory. Made-with: Cursor * opencode start.sh: prepend training env bin to PATH Ensures LLM agent bash calls (e.g. python3 -c "from trl import ...") resolve to the correct training environment, instead of relying on parent shell conda activation. Made-with: Cursor * opencode start.sh: restore --max-retries and --eval-timeout for opencode-rl Made-with: Cursor * add humaneval benchmark * Replace import * cleanup hack with explicit imports in OpenCompass config - Resolve dataset variable names via importlib before generating config, so the template uses `from xxx import datasets` instead of `import *` - Remove the fragile runtime cleanup hack that set leaked modules to None - Increase OpenCompass timeout from 3600s to 7200s - Fix score parsing to average across multiple subdatasets * refine opencompass config file generating * add humaneval benchmark dependency instructions human-eval package requires clone from open-compass/human-eval with a one-line patch to relax assertion for partial evaluation (test split only). Made-with: Cursor * fix: sanitize user-provided paths in RL UI (CodeQL) * fix: resolve user path relative to safe root (CodeQL) * fix: use Copilot-suggested path sanitization pattern (CodeQL) * fix: normalize and reject absolute user paths (CodeQL) * Fix training params, vLLM OOM cleanup, OpenCompass score parsing, and baseline cache logic * fix: add setuptools<75 to requirements for opencompass pkg_resources dependency uv venv does not include setuptools by default, causing OpenCompass baseline evaluation to fail with "No module named 'pkg_resources'". Made-with: Cursor * webshop * feat(autorl_bench): improve smith benchmark integration and evaluator robustness - Add smith benchmark docs to README: usage examples, discovery mechanism, SMITH_BENCH_DIR - Improve PerSampleEvaluator: vLLM GPU cleanup, test_range slicing - Refactor server.py: extract grading server from utils - Fix OpenCompass score parsing and baseline cache logic * fix: add smart fallback for OpenCompass dataset variable resolution When build_dataset_imports_explicit() fails to import an OpenCompass dataset module (common in grading server subprocess), it now guesses the correct variable name from the module path convention instead of falling back to empty names (which causes import * and breaks BBH due to leaked file handle objects). * revert: restore opencompass.py to pre-modification state Revert vLLM pid cleanup, dash-value checks, and metric-based score parsing added in 31caff2f and bb32e555. * keep metric-aware score parsing in opencompass; add baseline column to UI - opencompass.py: retain metric-type filtering (accuracy/score) instead of naive averaging, avoids polluting scores with pass/timeout counters - ui.py: add Baseline column to Agent Summary table Made-with: Cursor * fix: handle non-string answers in extract_answer to prevent TypeError arc_agi and other benchmarks can have non-string answer fields (e.g. lists), which caused a crash in re.search(). Adding str() coercion fixes this. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * update deepsearch qa tasks * fix: benchmark evaluation reliability (B1-B4) - B1: auto-detect LoRA adapters and enable vLLM LoRA mode (read base_model from adapter_config.json) - B2: serialize evaluations with threading.Lock to prevent GPU contention - B3: cache eval results by model_path to deduplicate concurrent submissions - B4: propagate error details from OpenCompass to agent (non-numeric scores, load failures) Made-with: Cursor * fix(B1): reject LoRA adapter submissions with clear merge instructions - opencompass.py: detect adapter_config.json and return error with merge_and_unload() instructions instead of broken vLLM LoRA mode - instructions.md: add requirement to submit full merged models - opencompass_template.yaml: remove unused is_lora/lora_path params Made-with: Cursor * update chat completion * update * update deepsearch * md * codex + benchmark update * codex + benchmark update * codex + benchmark update * codex * codex * fix: grading server cache key includes mtime to detect model overwrites Previously cache used only resolved_path, so overwritten models at the same path returned stale scores. Now cache key = path@max_mtime so re-evaluation is triggered when model files change. Made-with: Cursor * feat: add gemini/claude agent scaffolds, fix codex binary path - codex/start.sh: use CODEX_BIN env var instead of bare 'codex' - Add gemini/ and claude/ agent directories with config.yaml and start.sh Made-with: Cursor * chore: remove copied human_readable_trace.py from PostTrainBench Made-with: Cursor * update evaluation * benchmark * Fix log cleanup and OpenHands env * fix: webshop env pth problem * benchmark alpacaeval * style(rl): apply auto-lint fixes * fix(rl): address CI and CodeQL issues * fix(rl): make autorl bench imports CI-safe --------- Co-authored-by: Qizheng Li <jenssenlee@163.com> Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: Bowen Xian <xianbowen@outlook.com> Co-authored-by: chelsea97 <zhuowbrown@gmail.com> Co-authored-by: Xu Yang <peteryang@vip.qq.com> Co-authored-by: sakura657 <yctangcse@gmail.com> Co-authored-by: shatianming5 <tianming.sha@stonybrook.edu> Co-authored-by: Yeyuqing0913 <shatianming4@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
XianBW
|
6e19c9e632 |
feat: add LLM-finetune scenario (#1314)
* refine prompt * small update * fix a small bug * remove debug config after execution * fix: only remove <think> at start * feat: support creating dataset & multi-eval frame (#1302) * feat: add iterative evolve and evaluation support with partial chain stop * feat: add FTDataEvaluator and support multiple implement functions in finetune * feat: data implement for pre-proposal and proposal and add datasets (#1303) * feat:(1) support for multi layer dataset extraction (2) add category.json for dataset in datasets/ * fix: fix bug for generate category.json * feat: add get_dataset_folder_desc * init data proposal and merge qzli/ft * update data proposal prompts and add max_position_embeddings and resolve confilcts * remove sample counts in data proposal * turn data and train to unified hypo_gen * refine prompts * remove category.json and add it to dataset_info * fix jinja problem and proposal done * lint * add ai-generated description and raw readme into dataset_info.json * update prompt for description * add datasets * initial fix for proposal of data * final version for data proposal * lint * feat: add stats in dataset_info, and enable data coder (#1306) * refactor(dataset): add stats into dataset_info.json, and remove dataset from gitignore_folder * feat: enable data coder and run data process * feat: Merge data coder (#1307) * feat: implement finetune data coding, evaluation, and config improvements * fix: deepspeed config path * fix: dataset info columns --------- Co-authored-by: Young <afe.young@gmail.com> * replace str length with token_limit * add readme to dataset_info and remove useless blank lines in scenario description * feat: dataset prepare * fix: extract prams script name * feat: add loss&predictions samples to feedback * remove duplicate envs and and add llm_api_preferences and enhance reasoning token limits * feat: network for ft_env * fix: remove gpt-4o, which has low quota * feat: a simple ui * feat: merge data and train task type (#1309) * feat: filter redundant prams of lf * fix: ui bug caused by removing task_type * fix: force agent to use high concurrency, and remove redundant prompt * feat: extract info from llama factory log, and check data exists before download * fix: add compatibility rules * feat: llm evaluator for data coder * feat: openai package in ft docker, and refine prompt * feat: refine ft ui, add more info * feat: add raw logs * refine data coder prompt(for feedback debug) * feat: select dataset in scen init * fix: ui for docker log seperately * feat: sync log through blob * improve ui, and add llm feedback in Runner&Exp2FB (#1312) * fix: ui bug to visualize docker log, and lint * feat: unified docker log for ft env, and some refactor * fix bugs and improve ui * feat: save log of evaluator(single feedback) * feat: add evaluator, set cleanup docker log * feat: call llm in RunnerEvaluator and Feedback * fix: extract structured error message in RunnerEvaluator * feat: feedback improve, and fix some bugs * feat: feedback improve when runner fails * small update * feat(UI): add running info and benchmark metric in loop expander * feat(UI): add render markdown toggle * feat: refine prompts and add error type in exp2fb * feat: add filterd params reason, set default benchmark timeout to infinite, and refine train loss express * recover dataset deepscaler * feat: set timeout in .env * refactor: unifiied ft_env timeout * feat: debug mode for data coder * feat: deliver data_stats after generate debug_data * feat: use gpt-5.1 as judge model, set judge_retry, and refine debug mode prompt * refine prompt * refactor: llama factory manager logic, and refine data processing prompt * feat(DockerEnv): support GPU selection via CUDA_VISIBLE_DEVICES * feat: set api concurrency via .env * fix: ft env timeout bug * feat: enable CondaEnv run * fix: can't update bin path in first run, and path bug in lf manager * feat(ui): set log path through .env * refactor(ui): wrap_lines, remove css * feat(coder): retry when parse code-block fail * fix: refine single-fb in ui, and fix path bug(not allow proposal to decide path) * fix: opencompass CondaEnv torch compatible with vllm * fix: refine error text in coding * feat: deepspeed config for CondaEnv * feat: memory estimator * fix: deepspeed package for condaenv * fix: use `client.chat.completions.create()` only * feat: flash attention for condaenv * feat: strong and weak models interface * fix: condaenv package dependency * use multi round conversation in llm finetune proposal * refine prompt for data processing * enable evolving in data coder * maximize output token size * fix: refine ui * fix: optional packages for llama factory * fix: torch denpendency for b200 * fix: opencompass dependency * update cot prompts * skip the sub implement * skip conda preparation if env exists * update chemcot datasets * fix: unify docker to use litellm * update readme and instructions * fix: set CUDA_VISIBLE_DEVICES for CondaEnv * feat: add panorama dataset, refactor dataset interface * feat: calculate token using tiktoken, and ndarray bug * fix: download subtasks of chemcotdataset seperately * feat: customized prepare func for datasets * feat: update new benchmarks * add datasets package * docs: readme for llm finetune * feat: download raw data directly, with post-process function * feat: analyze raw dataset * suppress litellm debug info * feat(ui): summary page * feat: run multi-jobs * feat: improve ui * feat: add path and checkout options to LLM finetune loop entrypoint * feat: add FinanceIQ_ppl benchmark with auto-download and dataset desc rendering * refactor: remove unused imports and dead code, fix session folder logging * feat: enable tablebench and tableInstruct dataset * refine dataset readme, and coder prompt * refine proposal and coder prompt * fix: ui path (default log path) * feat: add automatic LoRA model merging for benchmarking with vLLM * refactor: reorganize finetune benchmark and merge modules under benchmark dir * refactor: modularize benchmark config and error extraction for finetune scenario * fix: update benchmark import paths and disable env cache for device info * refactor docke&conda env and fix import bugs * modify init python file * feat: add FinanceIQ dataset split utility and integrate with pipeline * feat: set weak and strong model by env, distribute workload across models * feat: sample dataset and rm params for tensorboard, wandb * update script to run jobs * refine proposal prompt, remove specific dataset name * fix(ui): auto switch log folder * fix: estimate the processed full data after sample * feat: filter raw data more aggressively, and lower data_eval standard * feat: sync workspace to blob * feat: rdkit for chemcotbench * update qwen2.5&llama3.1 context * fix: force failure on validation error and remove try/except in validator * feat: unified error sample extraction (with test scripts) * feat: set conda cache with .env * feat: skip data eval if data pass in last evo * fix: rm redundant param * fix ui bug * refactor: centralize assign_code_list_to_evo in MultiProcessEvolvingStrategy * feat: add test_params.yaml generation and workspace cleanup improvements for finetune * refactor: replace get_clear_ws_cmd with clear_workspace and update prompts for hard check criteria * add bioprobench dataset * fix: handle commas in training config extraction and refactor prompt includes * bioprobench description * add bioprobench readme * feat: merge lora adapter for blackwell gpu * feat: support for multi benchmarks in one job * change dfficult aware content for training * update difficulty-aware and logging principles * fix: resolve variable name conflict in FTRunnerEvaluator * set job id accuracy to minute * feat(ui): display one selected metric per benchmark * feat: store sota exp, and fix ws_ckp bug * fix: truncate data.json in feedback * fix: opencompass data for conda env * fix: save only the last model * feat: set log path and ws path * fix: set overwrite_cache to avoid lock contention(through injecting params) * feat: redirect stdout to file in localenv * add pickle cache to dataset desc * fix CI * fix: remove redundant wrapper * feat: set python_unbuffered * move redirect stdout to env run * fix a small bug * move model folder * feat(ui): display benchmark baseline * fix: enrich scenario and benchmark description * fix: rewrite runner eval to accept easier * feat: compare with baseline when no SOTA * update tablebench readme * fix: switch back to single benchmark (for baseline) * feat(ui): add ws path in ui * refactor: update SOTA tracking to use DAG traversal and parent selection * fix: prioritize local_selection in trace and refactor sibling retrieval logic * refactor: unify error handling in feedback generation and update workspace injection * feat: add skip_loop_error_stepname to control error skip step in LoopBase * fix: set local_selection to NEW_ROOT for experiments without parent * feat: set different ports for jobs * feat: set different ports for jobs * feat: add upper data size limit for LLM fine-tuning and update related prompts * fix: replace get_truncated_stdout() with stdout for consistent output handling * refactor: remove data.json from cache and workspace logic, focus on script-based reuse * fix: rm target_scenario * feat: add selective cache extraction and custom cache key for data processing * fix(ui): bug when displaying tablebench * fix: filter config in dataset_info.json * feat: add test set, set valid set * feat(ui): update test score, and set color for final decision * feat: add test score for baseline and update ui * fix: use [-100:] as test range * feat: update data_stats in runner * feat: wait for opencompass init when run multi jobs * fix: adjust test&valid split * feat: force to generate COT(with <think> token), and add answer format in scenarios.json * feat: improve ui * fix: unify benchmark volume mounts and set extra_volumes for conda env * fix(ui): number color * fix: update GPU memory handling to use total memory in GB and streamline code * fix: set use_cot_postprocessor * feat: add env_dict to config classes and merge env vars in Env run * fix: let coder obey proposal * fix(ui): direction bug and update chemcot core metirc * fix: set consistent benchmark mount points and env vars for docker and conda * fix: addintional target for LoRA * feat: workspace dir log for benchmark running * fix: tableInstruct path bug and update benchmark description * feat: timeout for whole job * fix: align FinanceIQ import to opencompass * feat: use llm_judge for FinanceIQ * feat: switch to turn on <think> or not * feat: using scripts to redirect stdout, and run in different windows * feat: sync litellm log * fix: gpu memory format * fix: escape special characters in benchmark desc * fix: set data processing timeout to 1h * feat: set valid_loss and save_best_model * fix: inject timeout and stage * fix: loss history extract logic * feat: inject output dir * feat: inject eval batch size * feat: inject save_total_limit * feat: update data prompt * fix: escape shell special characters * fix: tablebench visualization UI * fix: move implementation validation to coder, and ignore injected params * feat: README for FinanceIQ dataset * fix: bioprobench desc error * fix: remove task alignment when coder eval * fix: FinanceIQ now extracts last capital as answer * fix: stdout contains binary data * feat: recover estimate full output and set eval setting automatically * fix(ui): precision for summary table * fix(ui): import error * feat: try to use lora * fix(api): fix litellm bug for code block * fix: refine prompts to give agent more decision space * chore(ci): fix mypy typing issues * chore(ci): format code with black * chore(ci): fix ruff lint violations * chore(ci): sort imports with isort * chore(ci): format code with black * test: temporarily skip extract_parameters imports due to numpy pin * fix: compatibility issues for qlib scenarios on finetune branch * fix(fin_factor): skip to fb for coder error * fix(loop): default skip to feedback step on skip_loop_error When skip_loop_error exception happens and skip_loop_error_stepname is not explicitly set, default to jumping to 'feedback' step if it exists, otherwise fall back to the last step (record). This prevents KeyError when record step tries to access feedback data that doesn't exist because we skipped the feedback phase. Also removed redundant skip_loop_error_stepname from finetune loop since it's now the default behavior. * add 'skip to record' to DS scenario like other scenarios * fix 2 scenarios bug about rd_loop class * fix: lint(mypy, ruff, black) error * fix: mypy lint error * fix data science scenario bug --------- Co-authored-by: Xu Yang <peteryang@vip.qq.com> Co-authored-by: Qizheng Li <jenssenlee@163.com> Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> Co-authored-by: amstrongzyf <201840057@smail.nju.edu.cn> Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: amstrongzyf <amstrongzyf@126.com> Co-authored-by: chelsea97 <zhuowbrown@gmail.com> Co-authored-by: SunsetWolf <Lv.Linlang@hotmail.com> |
||
|
Jensen
|
4f493c8d63 |
feat(mcp): cache with one-click toggle (#1269)
* feat: enable cache in mcp * refactor: remove redundant setting * fix: conflicts during installation --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> |
||
|
you-n-g
|
5ba5e8356c |
feat: init pydantic ai agent & context 7 mcp (#1240)
* feat: init pydantic ai agent & context 7 mcp * feat: integrate MCP documentation search into data science pipeline evaluation * fix: disable MCP documentation search and update related docstrings and defaults * lint * fix: correct prompt formatting and conditional blocks in pipeline_eval section * lint * feat: add query method to PAIAgent for synchronous agent execution * fix: apply nest_asyncio for agent and update context7 query method * lint * lint * lint * lint * docs: update MCP folder docstring and rename test class in test_pydantic.py * refactor: centralize completion kwargs logic and update pydantic_ai integration * fixbug * typo * fix: bug triggered by padantic-ai version backtracking. --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> |
||
|
Yuante Li
|
4b838d3f45 |
refactor: refactor RD-Agent(Q) configuration files (#972)
* refactor rdagent(q) conf files * fix * fix ci |
||
|
Linlang
|
c8f1c5364a |
chore: add a rdagent server with UI & logger storage refinement(#553)
* change_log_object * lint code * delete comments * change_log_object * change_log_object * fix import test error * update code * update code * fix bugs * skip mypy error * skip mypy error * skip mypy error * Start the flask server before running the demo. * achieve front and back interaction * fix github-advanced-security comments * fix github-advanced-security comments * tmp ignore * fix CI * move some logic * change format * adjust logic * log2json changes * tmp * fix * fix bug * refine log2json between 5 scenarios * fix * refine codes * fix logic * use localhost * add loop & all_duration param for old scenario startup * merge control logic * add README for server ui api * update README * reuse code in logger * add loop_n and all_duration param * fix upload * ui server now use port in setting * fix port setting * fix port setting * fix mypy check * refine logger and log storage * fix ruff error * fix CI * refine logger, loop, storage * bind one FileStorage with one logger * not truncate log storage * refine LoopBase.load(), use `checkout` instead of `output_path` and `do_truncate` * clear session folder when loading loop to run * move component info init step to ExpGen Class * Update rdagent/utils/workflow.py * move truncate_session function to LoopBase class * add checkout param for other scenarios * fix bug * move WebStorage to UI * change web_storage name * add randomname to requirements * add typer * fix requirements --------- Co-authored-by: WinstonLiyte <1957922024@qq.com> Co-authored-by: Bowen Xian <xianbowen@outlook.com> Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> |
||
|
Yuante Li
|
d1019cb568 |
feat: add RD-Agent-Quant scenario (#838)
* fix model input shape bug and costeer_model bug * fix a bug * fix a bug in docker result extraction * a system-level optimization * add a filter of stdout * update * add stdout to model * model training_hyperparameters update * quant scenario * update some quant settings * llm choose action * Thompson Sampling Bandit for action choosing * refine both scens * add trace messages for quant scen * fix some bugs * fix some bugs * update * update * update * fix * fix * fix * update for merge * fix ci * fix some bugs * fix ci * fix ci * fix ci * fix ci * refactor * default qlib4rdagent local env downloading * fix ci * fix ci * fix a bug * fix ci * fix: align all prompts on template (#908) * use template to render all prompts * fix CI --------- Co-authored-by: Xu Yang <xuyang1@microsoft.com> * add fin_quant in cli * fix a bug * fix ci * fix some bugs * refactor * remove the columns in hypothesis if no value generated in this column * fix a bug * fix ci * fix conda env * add qlib gitignore * remove existed qlib folder & install torch in qlib conda * fix workspace ui in feedback * align model config in coder and runner in docker or conda * fix CI * fix CI --------- Co-authored-by: Xu Yang <peteryang@vip.qq.com> Co-authored-by: Xu Yang <xuyang1@microsoft.com> |
||
|
you-n-g
|
ec51bb94b6 |
feat: trace merging (#836)
* feat: runnalbe -- add exp_gen_cls param, get_leaves and merge exp gen functionalities * fix: remove unused scenario_desc and update YAML task labels * feat: override selection and update merge task description * lint * lint * lint * lint * lint * fix: log competition setting to enable mle_summary * fix name error |
||
|
you-n-g
|
20778a9b87 |
feat: condaenv & full docker env (#668)
* use conda to run kaggle and mlebench code * refactor: Simplify environment configuration and execution logic * add setting to use local env in ds * refine dockerfile * fix: Move MLEBDockerEnv initialization inside conditionals & fix condaenv * refactor: reformat code for better readability and consistency * feat: add conda env to all envs. * fix: fix bugs when run loop * refactor: Simplify DockerEnv configuration in mle_summary.py * fix image bug * style: reformat code for better readability and consistency * change commit * feat: Add entrypoint script for sing_docker scenario in rdagent * refactor: add Any type hints and comments for clarity in env.py * feat: Create log directory if it doesn't exist in entrypoint script * feat: Add debug mode and list root directory in entrypoint script * fix: Remove specific branch checkout in Dockerfile for RD-Agent * fix: Add competition argument to loop.py script execution * fix: Correct directory navigation and dependency installation in entrypoint.sh * fix: Correct user ownership assignment in entrypoint script * refactor: Comment out redundant log copying to RD_OUTPUT_DIR * fix: Unset LOG_TRACE_PATH to prevent log contamination in entrypoint.sh --------- Co-authored-by: Xu Yang <peteryang@vip.qq.com> |
||
|
you-n-g
|
943d2087fc |
fix: fix ExtendedSettingsConfigDict does not work (#660)
* refactor: Replace ExtendedSettingsConfigDict with SettingsConfigDict * lint * lint |
||
|
Roland Minrui
|
7ad0ee2250 |
load code from file dict instead of folder (#641)
Co-authored-by: Xu <v-xuminrui@microsoft.com> |
||
|
炼金术师华华
|
5090c6153f |
feat(backend): integrate LiteLLM API Backend (#564)
* File structure for supporting litellm * more litellm support * feat: Add CachedAPIBackend class and dynamic API backend retrieval function * fix: update benchmark folder path and add default values for architecture and hyperparameters * feat: add LiteLLMAPIBackend and DeprecBackend ; changed structure of the project ; with bus * fix : deprec_backend * feat: Add LiteLLMAPIBackend class and related features; update configuration and test cases. * feat: Enhance LiteLLMAPIBackend with encoder support and dynamic argument handling;Enhance log Colors * lint * fix lint... * fix: Lint * fix:make auto-lint * fix:test oai * fix:redundant _abckend.py * fix: Optimize LiteLLMAPIBackend on token counting functiona, and clean up unused code;add test on this function * feat: Add LiteLLMSettings class and update model settings usage * fix: Update LiteLLMSettings environment variable prefix and model configurations * fix : gitignore * test: Consolidate and relocate test files for litellm backend and oai * fix : lint * fix: lint * auto lint * lint * LINT * lint * chore: remove deprecated backend configuration comments * refactor: Remove unused functions and imports from deprec.py and llm_utils.py * refactor: Move md5_hash function from deprec.py to llm_utils.py * chore: Remove extra newline and add missing import in deprec.py * lint * refactor: Move md5_hash function to utils module * lint * lint * lint --------- Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: Yihua Chen <v-yihuachen@microsoft.com> |
||
|
you-n-g
|
f78175b37a |
feat: refactor for general data science (#498)
* refine ds modal for more cases: eval and es * update model template * prompts for model and ensemble * fix a bug * fix a bug * init: ds workflow evovingstrategy * Adding ensemble (#505) * Initial Draft * Updating logic for init * Revising * Successful Testing * Updating to use the latest & right class * bug: bug-fixing for testing * data science loop changes * data science loop base * ds loop feedback * fix * remove measure_time because it's duplicated (in LoopBase) * add the knowledge query for data_loader & feature * edit ds workflow evaluator * data_loader bug fix * stop evolving when all tasks completed * llm app change * fix break all complete strategy * Adding queried knowledge (#508) Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com> * fix loop bug * ds workflow evaluator; test; refine prompts * workflow spec * fix ci * feature task changes * ds loop change * fix a bug in feat * add query knowledge for model and workflow * llm_debug info(for show) using pickle instead of json * remove NextLoopException * loop change * coder raise CoderError when all sub_tasks failed * rename code_dict to file_dict in FBWorkspace * add CoSTEER unittest * now show self.version in Task.get_task_information(), simplify CoSTEER sub tasks definition * remove some properties in ModelTask, add model_type in it. * fix llm app bug * llm web app bug fix * ds loop bug fix * fix: give component code to feature&ens eval * loop catch error bug * rename load_from_raw_data to load_data * feat: Add debug data creation functionality for data science scenarios * support local folder (#511) * support local folder * remove unnecessary random * KaggleScen Subclass * small fix * use template for style description * update default scen to kaggle * update sample data script * make sure frac < 1 * fix a bug * feature spec changes * fix * changeimport order * clear unnecessary std outputs * fix a typo * create sample folder after unzip kaggle data * feature/model test script update * Align the data types across modules. * fix a bug in model eval * show line number * move sample entry point to app * spec & model prompt changes * Refine the competition specification to address the data type problem and the coherence issue. * fix some bugs * add file filter in FBworkspace.code property * support non-binary prediction * avoid too much warnings * fix a bug in ensemble module * filtered the knowledge query in all modules * delete RAG in idea proposal * refine the code in ensemble * show exp workspace in llm_st * exp_gen bug fix * feedback bug fix * use `feature` instead of `feat01` * Trace & method of judging if exp is completed change * fix a bug in package calling and execute ci * fix code * bug fix * bug fix * fix a bug * fix some bugs * fix a bug * refactor: Enhance error handling and feedback in data science loop * support different use_azure on chat and embedding models * multi-model proposal logic * fix a small syntax error * loopBase and some changes * ensemble scores change * fbworkspace.code -> .all_codes * use all model codes in workflow coder * check scores.csv's keys(model_names) * model name changes * add a todo in ensemble test * sota_exp changes * give model info in exp gen * add runner time limit * config using debug data or not in evals * exp to feedback base * add feature code when writing model task * small problem * copying during sampling * update * refactor: Simplify code handling and improve workspace management * model part output fix * print model's execution time * bug fix * ensemble test fix * ens small change * ens_test bug fix * Refine partial expansion logic to display only a few subfolders when their structure is uniform, improving readability in nested directories. * several update on prompts * sample subfolders * Filter the stdout after code execution to remove irrelevant information e.g. progress bars, whitespace characters, excessive line breaks. * Add some more prompts and comments * several update on the first init rounds * model timeout as error * fix pattern of getting model codes in workspace * small bux fix on model prompts * remove get_code_with_key since we have regex pattern * fix: Correct tqdm progress bar update logic in LoopBase class * feat: Add diff generation and enhance feedback mechanism in data science loop * update some fix to model and workflow prompts * refine the logic of progress bar filter * add last_successful_exp in exp_gen * fix a one line bug * add a hint in prompt * fix data sample for bms * fix data sample for bms * hypothesis small fix * crawler readme update * fix component gen * fix bug * annotation change * load description.md if it exists * refactor: Simplify SOTA description handling in feedback and prompts * refactor: Use shared templates for feedback and experiment descriptions * change webapp for model codes changes * update proposal * add timeout message for docker run output * fix * refine the code in docker time processing * use .shape instead of len() when do shape eval * won't change size during iteration * support bson sample * sample support jsonl and bson * add former_code to coder prompts * a little speed us in debug data creating * filter progress bar when eval ens and main * avoid costeer makes no change to former code * fix several log error * add timeout judge threshold * fix some bugs in the evaluation of component output shapes * File structure for supporting litellm (#517) Co-authored-by: Young <afe.young@gmail.com> * ignore submission and show processing * ignore submission and show processing * add efficiency notice * refactor: Enhance error message with detailed feedback summary * refactor: Simplify component handling in DSExpGen class * refactor: Update code structure and add docstring for clarity * reserve one sample to each label in data sampling * add Evaluation info * refine costeer code to avoid giving same code twice * use raw_description as plain text * add a prompt hint to avoid same dict key * model task name bug in first model exp gen * fix a typo * add some debug info in costeer tests * task init change * enhance data sampling * refine the code in data_loader * more reasonable loop * fix a bug in data folder description * add error msg & traceback to execution feedback * fix llm error msg detection * add task information to costeer eval & add cache to docker run(use zipfile to store the whole workspace) * fix CI first round * fix CI second round * use txt to store test script to avoid pytest * remove zipfile in requirements * add azure.identity to requirements * ignore debug web page * component test changes * remove redundent task_desc in model coder * feat: Add APE module and prompts for automated prompt engineering * fix: Update .gitignore and improve text formatting in eval.py * refactor: Update print output and improve code comments and imports * style: Fix string formatting and import order in ape.py and fmt.py * exclude ape * add a data folder notice * reduce unnecessary output to stdout * refine the code of describe_data_folder * fix ci * style: streamlit style update (#522) * streamlit style update * fix import * fix format * fix llm_st loop progress bar * debugapp small change * fix model str * refine some prompts * fix model str * fix CI * refine the logic associated with the data_folder * fix ci * small change * set filter_progress_bar as default in execute * model proposal with workflow * add submission check in workflow eval * fix bug * small change * fix CI * fix CI * refactor: Move generate_diff to utils and update DSExpGen logic * more reasonable prompt describing metric direction * fix a minor jinja2 bug * quick fix exp_gen bugs * fix the following bug * fix * fix some bugs * remove workflow from model * add pending_tasks_list in data science to enable coding model and workflow * refine the code for handling JSON-formatted data descriptions * assert with information * ensure correct csv file name * add logging to help record the output * log competition * add log tag for debug llm app * test: Test ds refactor ll (#523) * fix bugs to former scenario * fix a bug because coding in rdloop changed * fix the bug when feedback gets no hypothesis * fix trace structure * change all trace hist when merging hypothesis to experiments * ignore some error in ruff * fix kaggle scenario bugs * refine one line * another bug * another small bug * fix ui bugs * chage kaggle train.py path --------- Co-authored-by: Xu Yang <peteryang@vip.qq.com> * fix CI * Update rdagent/app/data_science/loop.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add samplecsv into spec prompts * fix CI --------- Co-authored-by: TPLin22 <tplin2@163.com> Co-authored-by: yuanteli <1957922024@qq.com> Co-authored-by: Xisen Wang <118058822+xisen-w@users.noreply.github.com> Co-authored-by: Bowen Xian <xianbowen@outlook.com> Co-authored-by: Xu Yang <peteryang@vip.qq.com> Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com> Co-authored-by: Tim <illking@foxmail.com> Co-authored-by: 炼金术师华华 <37462254+YeewahChan@users.noreply.github.com> Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Xu Yang
|
0933eb2948 | build factor source data (price and volumns) from qlib if no source data is provided by the user (#168) | ||
|
you-n-g
|
30827f2ae7 |
fix record (#113)
* fix record * fix type * add loguru-mypy |
||
|
you-n-g
|
4f484e009a |
Workflow Support Loading and saving sessions (#98)
* Successfully logging the trace * Start debugging & Add policy file * Support loading sessions * Add docs * Add tqdm |
||
|
WinstonLiyt
|
7220d33668 |
Update the detailed process and prompt of factor loop. (#96)
Update the detailed process and prompt of factor loop. |
||
|
you-n-g
|
9f89511be6 | Model run with logger (#79) | ||
|
Xu Yang
|
e0a24fb46f |
Several update on the repo (see desc) (#76)
* ignore result csv file * fix app scripts * rename taskgenerator to developer and generate to develop * fix a config bug in coder * fix a small bug in factor coder evaluators * remove a single logger in factor coder evaluators * fix a small bug in model coder main.py * rename Implementation to Workspace * move the prepare the inject_code into FBWorkspace to align all the behavior * fix a small bug in model feedback * remove debug lines for multi processing and simplify evaluators multi proc * add a copy function to workspace to freeze the workspace && add config prefix to speed up debugging * make hypothesisgen a abc class * use Qlib***Experiment * fix a small bug * rename Imp to Ws * rename sub_implementations to sub_workspace_list * fix a bug in feedback not presented as content in prompts * move proposal pys to proposal folder * reformat the folder * align factor and model qlib workspace and use template to handle the workspace * add a filter to evoagent to filter out false evo * align multi_proc_n into RDAGENT seeting * handle when runner gets empty experiment * fix logger merge remaining problems * fix black and isort automatically |
||
|
WinstonLiyt
|
57ae5c93ea | Upload the configuration file for running Docker. | ||
|
WinstonLiyt
|
bf1b140f68 | re-commit | ||
|
Linlang
|
833e7ce1b9 |
Initial framework for docker env (#40)
* Initial framework for docker env * Update test name * add features * Download Qlib data with extra_volume * fix pytest error * Fix the parameters --------- Co-authored-by: Young <afe.young@gmail.com> |
||
|
Xu Yang
|
02173ffdcc |
add code security check CI and dependbot (#3)
* update code * init structure * add requirements and fix CI --------- Co-authored-by: xuyang1 <xuyang1@microsoft.com> |