- Path injection (B614): centralized safe_resolve_path in core/utils.py,
refactored 6 UI modules to use it with safe_root validation
- B701: added explicit autoescape=select_autoescape() to Jinja2
Environment() calls in 3 files
- B101: replaced assert statements with proper if/raise patterns in
12+ files (partial)
- B112: added logger.warning() to bare except:continue blocks in
5 files
- Add vbt_backtest.py as single source of truth for all metric formulas
(Sharpe, drawdown, IC, transaction costs) — backtest_engine.py and
strategy_orchestrator.py now delegate to it
- Add LLMUnavailableError to exception.py; rd_loop.py catches it at the
proposal stage and raises LoopResumeError to avoid corrupting trace
history with None hypotheses
- Guard record() against None exp/hypothesis so loop resets leave
trace.hist in a consistent state
- Refactor strategy_orchestrator and optuna_optimizer to use unified
backtest path; remove duplicate metric calculation code
- Add predix_rebacktest_unified.py script for offline re-evaluation
- Update tests and README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* update rdagent cmd
* fix log error message
* use multiProcessing.Process instead of subprocess.Popen
* add traces to gitignore
* add user interactor in RDLoop (finance scenarios)
* add interactor (feedback, hypothesis) for quant scens
* fix the test_end in qlib conf
* add features init config, general instruction to qlib scenarios
* set base features for based exp
* fix bug when combine factors
* move traces folder to git_ignore_folder
* fix bug in features init
* fix quant interact bug
* fix logger warning error
* bug fixes
* modify rdagent logger, now it can set file output
* adjust cli functions and fix logger bug
* fix server port transport problem
* update server_ui in cli
* add web code
* fix CI problem
* black fix
* update web ui README
* update README
* update readme
* refine prompt
* small update
* fix a small bug
* remove debug config after execution
* fix: only remove <think> at start
* feat: support creating dataset & multi-eval frame (#1302)
* feat: add iterative evolve and evaluation support with partial chain stop
* feat: add FTDataEvaluator and support multiple implement functions in finetune
* feat: data implement for pre-proposal and proposal and add datasets (#1303)
* feat:(1) support for multi layer dataset extraction (2) add category.json for dataset in datasets/
* fix: fix bug for generate category.json
* feat: add get_dataset_folder_desc
* init data proposal and merge qzli/ft
* update data proposal prompts and add max_position_embeddings and resolve confilcts
* remove sample counts in data proposal
* turn data and train to unified hypo_gen
* refine prompts
* remove category.json and add it to dataset_info
* fix jinja problem and proposal done
* lint
* add ai-generated description and raw readme into dataset_info.json
* update prompt for description
* add datasets
* initial fix for proposal of data
* final version for data proposal
* lint
* feat: add stats in dataset_info, and enable data coder (#1306)
* refactor(dataset): add stats into dataset_info.json, and remove dataset from gitignore_folder
* feat: enable data coder and run data process
* feat: Merge data coder (#1307)
* feat: implement finetune data coding, evaluation, and config improvements
* fix: deepspeed config path
* fix: dataset info columns
---------
Co-authored-by: Young <afe.young@gmail.com>
* replace str length with token_limit
* add readme to dataset_info and remove useless blank lines in scenario description
* feat: dataset prepare
* fix: extract prams script name
* feat: add loss&predictions samples to feedback
* remove duplicate envs and and add llm_api_preferences and enhance reasoning token limits
* feat: network for ft_env
* fix: remove gpt-4o, which has low quota
* feat: a simple ui
* feat: merge data and train task type (#1309)
* feat: filter redundant prams of lf
* fix: ui bug caused by removing task_type
* fix: force agent to use high concurrency, and remove redundant prompt
* feat: extract info from llama factory log, and check data exists before download
* fix: add compatibility rules
* feat: llm evaluator for data coder
* feat: openai package in ft docker, and refine prompt
* feat: refine ft ui, add more info
* feat: add raw logs
* refine data coder prompt(for feedback debug)
* feat: select dataset in scen init
* fix: ui for docker log seperately
* feat: sync log through blob
* improve ui, and add llm feedback in Runner&Exp2FB (#1312)
* fix: ui bug to visualize docker log, and lint
* feat: unified docker log for ft env, and some refactor
* fix bugs and improve ui
* feat: save log of evaluator(single feedback)
* feat: add evaluator, set cleanup docker log
* feat: call llm in RunnerEvaluator and Feedback
* fix: extract structured error message in RunnerEvaluator
* feat: feedback improve, and fix some bugs
* feat: feedback improve when runner fails
* small update
* feat(UI): add running info and benchmark metric in loop expander
* feat(UI): add render markdown toggle
* feat: refine prompts and add error type in exp2fb
* feat: add filterd params reason, set default benchmark timeout to infinite, and refine train loss express
* recover dataset deepscaler
* feat: set timeout in .env
* refactor: unifiied ft_env timeout
* feat: debug mode for data coder
* feat: deliver data_stats after generate debug_data
* feat: use gpt-5.1 as judge model, set judge_retry, and refine debug mode prompt
* refine prompt
* refactor: llama factory manager logic, and refine data processing prompt
* feat(DockerEnv): support GPU selection via CUDA_VISIBLE_DEVICES
* feat: set api concurrency via .env
* fix: ft env timeout bug
* feat: enable CondaEnv run
* fix: can't update bin path in first run, and path bug in lf manager
* feat(ui): set log path through .env
* refactor(ui): wrap_lines, remove css
* feat(coder): retry when parse code-block fail
* fix: refine single-fb in ui, and fix path bug(not allow proposal to decide path)
* fix: opencompass CondaEnv torch compatible with vllm
* fix: refine error text in coding
* feat: deepspeed config for CondaEnv
* feat: memory estimator
* fix: deepspeed package for condaenv
* fix: use `client.chat.completions.create()` only
* feat: flash attention for condaenv
* feat: strong and weak models interface
* fix: condaenv package dependency
* use multi round conversation in llm finetune proposal
* refine prompt for data processing
* enable evolving in data coder
* maximize output token size
* fix: refine ui
* fix: optional packages for llama factory
* fix: torch denpendency for b200
* fix: opencompass dependency
* update cot prompts
* skip the sub implement
* skip conda preparation if env exists
* update chemcot datasets
* fix: unify docker to use litellm
* update readme and instructions
* fix: set CUDA_VISIBLE_DEVICES for CondaEnv
* feat: add panorama dataset, refactor dataset interface
* feat: calculate token using tiktoken, and ndarray bug
* fix: download subtasks of chemcotdataset seperately
* feat: customized prepare func for datasets
* feat: update new benchmarks
* add datasets package
* docs: readme for llm finetune
* feat: download raw data directly, with post-process function
* feat: analyze raw dataset
* suppress litellm debug info
* feat(ui): summary page
* feat: run multi-jobs
* feat: improve ui
* feat: add path and checkout options to LLM finetune loop entrypoint
* feat: add FinanceIQ_ppl benchmark with auto-download and dataset desc rendering
* refactor: remove unused imports and dead code, fix session folder logging
* feat: enable tablebench and tableInstruct dataset
* refine dataset readme, and coder prompt
* refine proposal and coder prompt
* fix: ui path (default log path)
* feat: add automatic LoRA model merging for benchmarking with vLLM
* refactor: reorganize finetune benchmark and merge modules under benchmark dir
* refactor: modularize benchmark config and error extraction for finetune scenario
* fix: update benchmark import paths and disable env cache for device info
* refactor docke&conda env and fix import bugs
* modify init python file
* feat: add FinanceIQ dataset split utility and integrate with pipeline
* feat: set weak and strong model by env, distribute workload across models
* feat: sample dataset and rm params for tensorboard, wandb
* update script to run jobs
* refine proposal prompt, remove specific dataset name
* fix(ui): auto switch log folder
* fix: estimate the processed full data after sample
* feat: filter raw data more aggressively, and lower data_eval standard
* feat: sync workspace to blob
* feat: rdkit for chemcotbench
* update qwen2.5&llama3.1 context
* fix: force failure on validation error and remove try/except in validator
* feat: unified error sample extraction (with test scripts)
* feat: set conda cache with .env
* feat: skip data eval if data pass in last evo
* fix: rm redundant param
* fix ui bug
* refactor: centralize assign_code_list_to_evo in MultiProcessEvolvingStrategy
* feat: add test_params.yaml generation and workspace cleanup improvements for finetune
* refactor: replace get_clear_ws_cmd with clear_workspace and update prompts for hard check criteria
* add bioprobench dataset
* fix: handle commas in training config extraction and refactor prompt includes
* bioprobench description
* add bioprobench readme
* feat: merge lora adapter for blackwell gpu
* feat: support for multi benchmarks in one job
* change dfficult aware content for training
* update difficulty-aware and logging principles
* fix: resolve variable name conflict in FTRunnerEvaluator
* set job id accuracy to minute
* feat(ui): display one selected metric per benchmark
* feat: store sota exp, and fix ws_ckp bug
* fix: truncate data.json in feedback
* fix: opencompass data for conda env
* fix: save only the last model
* feat: set log path and ws path
* fix: set overwrite_cache to avoid lock contention(through injecting params)
* feat: redirect stdout to file in localenv
* add pickle cache to dataset desc
* fix CI
* fix: remove redundant wrapper
* feat: set python_unbuffered
* move redirect stdout to env run
* fix a small bug
* move model folder
* feat(ui): display benchmark baseline
* fix: enrich scenario and benchmark description
* fix: rewrite runner eval to accept easier
* feat: compare with baseline when no SOTA
* update tablebench readme
* fix: switch back to single benchmark (for baseline)
* feat(ui): add ws path in ui
* refactor: update SOTA tracking to use DAG traversal and parent selection
* fix: prioritize local_selection in trace and refactor sibling retrieval logic
* refactor: unify error handling in feedback generation and update workspace injection
* feat: add skip_loop_error_stepname to control error skip step in LoopBase
* fix: set local_selection to NEW_ROOT for experiments without parent
* feat: set different ports for jobs
* feat: set different ports for jobs
* feat: add upper data size limit for LLM fine-tuning and update related prompts
* fix: replace get_truncated_stdout() with stdout for consistent output handling
* refactor: remove data.json from cache and workspace logic, focus on script-based reuse
* fix: rm target_scenario
* feat: add selective cache extraction and custom cache key for data processing
* fix(ui): bug when displaying tablebench
* fix: filter config in dataset_info.json
* feat: add test set, set valid set
* feat(ui): update test score, and set color for final decision
* feat: add test score for baseline and update ui
* fix: use [-100:] as test range
* feat: update data_stats in runner
* feat: wait for opencompass init when run multi jobs
* fix: adjust test&valid split
* feat: force to generate COT(with <think> token), and add answer format in scenarios.json
* feat: improve ui
* fix: unify benchmark volume mounts and set extra_volumes for conda env
* fix(ui): number color
* fix: update GPU memory handling to use total memory in GB and streamline code
* fix: set use_cot_postprocessor
* feat: add env_dict to config classes and merge env vars in Env run
* fix: let coder obey proposal
* fix(ui): direction bug and update chemcot core metirc
* fix: set consistent benchmark mount points and env vars for docker and conda
* fix: addintional target for LoRA
* feat: workspace dir log for benchmark running
* fix: tableInstruct path bug and update benchmark description
* feat: timeout for whole job
* fix: align FinanceIQ import to opencompass
* feat: use llm_judge for FinanceIQ
* feat: switch to turn on <think> or not
* feat: using scripts to redirect stdout, and run in different windows
* feat: sync litellm log
* fix: gpu memory format
* fix: escape special characters in benchmark desc
* fix: set data processing timeout to 1h
* feat: set valid_loss and save_best_model
* fix: inject timeout and stage
* fix: loss history extract logic
* feat: inject output dir
* feat: inject eval batch size
* feat: inject save_total_limit
* feat: update data prompt
* fix: escape shell special characters
* fix: tablebench visualization UI
* fix: move implementation validation to coder, and ignore injected params
* feat: README for FinanceIQ dataset
* fix: bioprobench desc error
* fix: remove task alignment when coder eval
* fix: FinanceIQ now extracts last capital as answer
* fix: stdout contains binary data
* feat: recover estimate full output and set eval setting automatically
* fix(ui): precision for summary table
* fix(ui): import error
* feat: try to use lora
* fix(api): fix litellm bug for code block
* fix: refine prompts to give agent more decision space
* chore(ci): fix mypy typing issues
* chore(ci): format code with black
* chore(ci): fix ruff lint violations
* chore(ci): sort imports with isort
* chore(ci): format code with black
* test: temporarily skip extract_parameters imports due to numpy pin
* fix: compatibility issues for qlib scenarios on finetune branch
* fix(fin_factor): skip to fb for coder error
* fix(loop): default skip to feedback step on skip_loop_error
When skip_loop_error exception happens and skip_loop_error_stepname is not
explicitly set, default to jumping to 'feedback' step if it exists,
otherwise fall back to the last step (record).
This prevents KeyError when record step tries to access feedback data that
doesn't exist because we skipped the feedback phase.
Also removed redundant skip_loop_error_stepname from finetune loop since
it's now the default behavior.
* add 'skip to record' to DS scenario like other scenarios
* fix 2 scenarios bug about rd_loop class
* fix: lint(mypy, ruff, black) error
* fix: mypy lint error
* fix data science scenario bug
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Qizheng Li <jenssenlee@163.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: amstrongzyf <201840057@smail.nju.edu.cn>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: amstrongzyf <amstrongzyf@126.com>
Co-authored-by: chelsea97 <zhuowbrown@gmail.com>
Co-authored-by: SunsetWolf <Lv.Linlang@hotmail.com>
* feat: add interactor classes and user interaction handling for experiments
* update code
* use fragment retry mechanism instead of rerun()
* fix a bug
* integrate user instructions into proposal and coder
* fix CI
* fix CI
* feat: add approval option for user instructions submission
* feat: enhance user instructions handling in Task and DSExperiment classes
* fix CI
* add user instructions into hypothesis rewrite
* add interface to command line
---------
Co-authored-by: Bowen Xian <xianbowen@outlook.com>
* refactor: use get_truncated_stdout for consistent stdout handling across modules
* lint
* feat: add dump_stdout_type to DSRunnerCoSTEERSettings and use in eval
* fix: avoid circular import by moving DSRunnerEvaluator import inside method
* change DSCoSTEER_eval prompts
* fallback to better exp only
* fix fallback
* fix and reformat
* fix bug when base_fb is None
* add reasoning to hyperparameter evaluation
* feat: add acceptable assessment in exp_feedback (#1159)
* add time
* refine eval prompt and make the logic of tuning check more clear
* some refinement
* fix CI
* fix a small bug, only consider score in runner
* refine comment
* simplify compare function
---------
Co-authored-by: jingyuanlm <842442862@qq.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Jensen Lee <91518020+Jensen246@users.noreply.github.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
* feat: refactor CoSTEER classes to use DSCoSTEER and update max seconds handling
* remove useless line
* enable time_ratio_limit_to_enable_hyperparameter_tuning
* add prev loops to runner history
* fix evolving history
* fix bug on initializing feedback without final decision
* reformat
* refine
* add comments
* fix ci
* a little refinement
* fix CI
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
* change refine prompt for full code
* fix: fix the logic of running
* refine prompt
* fix some bugs
* fix
* add two guidelines
* refactor the code
* make costeer evaluator more logical
* refine eval prompt
* make costeer eval prompt markdown
* update code diff prompt
* correct pipeline
* feat: add apply_patch utility and update ret.py with patch functionality (#1071)
* restore to the right version
* fix the docstring
* fix extract_output fcn
* add inplace parameter to apply patch
* remove enable_runner_iteration and make the eval prompt same as main
* refine runner eval prompt based on main
* Update rdagent/scenarios/data_science/dev/runner/prompts.yaml
* add wait_retry
* refactor: move enable_runner_code_diff to DSRunnerCoSTEERSettings as diff_mode
* reformat and remove enable_runner_code_diff
---------
Co-authored-by: yuanteli <1957922024@qq.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Jensen Lee <91518020+Jensen246@users.noreply.github.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Qizheng Li <jenssenlee@163.com>
* init commit
* remove the 5-fold spec from prompts
* refine the hyperparameter specification
* do not sample data
* a small spelling issue
* refine prompt to avoid submission cheating
* do not sample data
* simplify code
* refine the coder evaluator prompt
* refine wording
* remove runtime from proposal
* refine wording
* refine prompt
* add gpu info in runtime_info.py
* modify the spec
* add router and add refinement exp gen
* fix prompt bug
* use rule-based logic for router
* complete the prompt
* fix circular import bug
* fix bug
* make refine_decision optional
* update pipeline prompts: (1) add scenary: in an iterative cooding loop and use sample datasets (2)add some generation tops in coding (3)add evaluation guidelines in evaluation (4)polish the json schema and description
* fix a small bug
* fix a small bug
* rdagent/scenarios/data_science/loop.py back to the original version
* refactor: replace _get_exp_gen with default_exp_gen for exp generation
* import
* refactor: make the __init__ back to main
* fix small bugs
* fix bugs for proposal_version
* move refine into runner
* check early stop
* EDA improvement & coder classes number
* fix CI
* slightly refine the prompt
* remove rule_base_eval and remove useless prompt
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: TPLin22 <tplin2@163.com>
Co-authored-by: amstrongzyf <amstrongzyf@126.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
* start to work on multi-trace + async
* init ver of async-multi-tarce, to test
* add eng-ver log
* complete version of async+ mul-trace
* debug
* fix bug on DS_RD_SETTING.get()
* update
* fix bug + simplif the usage of async in multi-trace
* fix mini bug of arg_name
* Move local_selection into class Experiment & clean the code
* refactor: split workflow into pkg, add WorkflowTracker & wait_retry
* feat: add async LoopBase with parallel workers and step semaphores
* fix: replace pickle with dill and run blocking tasks via joblib wrapper
* feat: add log format settings, dynamic parallelism & pickle-based snapshot
* fix: default step semaphore to 1 and avoid subprocess when single worker
* merge bowen's changes
* merge tim's changes
* refactor: extract component task mapping, add conditional logger setup
* lint
* refactor: add type hints and safer remain_time metric logging in workflow
* lint
* fix: allow BadRequestError to be pickled via custom copyreg reducer
* fix: stop loop when LoopTerminationError is raised in LoopBase
* lint
* refactor: make log tag context-local using ContextVar for thread safety
* feat: add subproc_step flag and helper to decide subprocess execution
* fix: use ./cache path and normalize relative volume bind paths
* fix: reset loop_idx to 0 on loop restart/resume to ensure correct flow
* fix: avoid chmod on cache and input dirs in Env timeout wrapper
* fix: skip chmod on 'cache' and 'input' dirs using find -prune
* fix: restrict chmod to immediate mount dirs excluding cache/input
* fix: chmod cache and input dirs alongside their contents after entry run
* fix: guard chmod with directory checks for cache and input
* fix: prefix mount_path in chmod command for cache/input dirs
* fix: drop quotes from find exclude patterns to ensure chmod executes
* fix: skip chmod on cache/input directories to avoid warning spam
* feat: support string volume mappings and poll subprocess stdout/stderr
* support remove symbolic link
* test: use dynamic home path and code volume in LocalEnv local_simple
* fix: skip trace and progress update when loop step is withdrawn
* refactor: add clean_workspace util and non-destructive workspace backup
* fix: preserve symlinks when backing up workspace with copytree
* fix: prevent AttributeError when _pbar not yet initialized in LoopBase
* perf: replace shutil.copytree with rsync for faster workspace backup
* fix: cast log directory Path to str in tar command of data science loop
* fix: use portable 'cp -r -P' instead of rsync for workspace backup
* fix: add retry and logging to workspace backup for robustness
* refactor: extract backup_folder helper and reuse in DataScienceRDLoop
* fix: propagate backup errors & default _pbar getattr to avoid error
* fix the division by zero bug
* refactor: execute RD loops via asyncio.run and add necessary imports
* lint
* lint
* lint
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
* chore: avoid incorporate changes
best as sota
merge hypothesis
fix: max_retrieve_num after decision
chore: select last experiments and feedbacks
* add the set_current_selection before the exp_gen when merging
add trace.NEW_ROOT
fix: no scen_prob_multiplier
fix: use regex with timeout
chore: hypothesis_rank with selected_idx
chore: define is_parent in proposal
chore: rename collect_all_ancestors to get_parent_exps
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* fix model input shape bug and costeer_model bug
* fix a bug
* fix a bug in docker result extraction
* a system-level optimization
* add a filter of stdout
* update
* add stdout to model
* model training_hyperparameters update
* quant scenario
* update some quant settings
* llm choose action
* Thompson Sampling Bandit for action choosing
* refine both scens
* add trace messages for quant scen
* fix some bugs
* fix some bugs
* update
* update
* update
* fix
* fix
* fix
* update for merge
* fix ci
* fix some bugs
* fix ci
* fix ci
* fix ci
* fix ci
* refactor
* default qlib4rdagent local env downloading
* fix ci
* fix ci
* fix a bug
* fix ci
* fix: align all prompts on template (#908)
* use template to render all prompts
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* add fin_quant in cli
* fix a bug
* fix ci
* fix some bugs
* refactor
* remove the columns in hypothesis if no value generated in this column
* fix a bug
* fix ci
* fix conda env
* add qlib gitignore
* remove existed qlib folder & install torch in qlib conda
* fix workspace ui in feedback
* align model config in coder and runner in docker or conda
* fix CI
* fix CI
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* fix the problems weights bug
* refactor: remove DSExpGen
* update problems weights calculation
* update problems weights calculation
* remove the selection parameter from exp_gen
* v2 support draft
* v3 also support decomposition
* make the identify_problems an independent function
* fix minor bug
* reformat
* rename exp_num to weighted_exp_num
* add the set_current_selection before the exp_gen when merging
* reformat
* fix wrong selection
* refactor: drop selection arg from ExpGen.gen and DS merge generators
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* imp SOTA-Jump policy
* fix small bug
* allow to set different selector by .env
* add always-win selector
* add init length for AlwaysWinCKPSelector
* add back_jump selector
* auto lint
* add sota_exp_to_submit attribute; change the name of ckp_selector and sota-selector
* fix bug
* auto lint
* working on auto sota selector
* add subtrace counter
* fix bug, remove unuse selector
* add auto sota selector
* auto lint
* fix bug
* fix small logic bug
* add logging
* add inject_diverse feat
* auto lint
* capable to None-select
* feat: add hypothesis_gen config and ExpGen2TraceAndMerge functionality
* refactor: use dynamic import for experiment generator instantiation
* feat: add BestValidSelector for improved SOTA experiment selection
* runnable twin-trace version
* fix logic error of trace-merge
* auto lint
* use import_class to set selector,
* auto-lint
---------
Co-authored-by: Young <afe.young@gmail.com>
* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* move selector to from proposal.py tp seletc.py
* auto lint
* fix line-too-long typos
* aligh the design of "selection", rm extra instance check
* make auto-lint
* add non-trival selector: SOTAjump
* move cache auto continue and retry to all api backend
* add type checker to json mode output
* fix CI
* feat: Add json_mode handling and streaming support in chat completion function
* lint
* fix a bug when returning a dict which value could contain int or bool
* remove litellm
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: Young <afe.young@gmail.com>
TPTBusiness