* refactor: split workflow into pkg, add WorkflowTracker & wait_retry
* feat: add async LoopBase with parallel workers and step semaphores
* fix: replace pickle with dill and run blocking tasks via joblib wrapper
* feat: add log format settings, dynamic parallelism & pickle-based snapshot
* fix: default step semaphore to 1 and avoid subprocess when single worker
* merge bowen's changes
* merge tim's changes
* refactor: extract component task mapping, add conditional logger setup
* lint
* refactor: add type hints and safer remain_time metric logging in workflow
* lint
* fix: allow BadRequestError to be pickled via custom copyreg reducer
* fix: stop loop when LoopTerminationError is raised in LoopBase
* lint
* refactor: make log tag context-local using ContextVar for thread safety
* feat: add subproc_step flag and helper to decide subprocess execution
* fix: use ./cache path and normalize relative volume bind paths
* fix: reset loop_idx to 0 on loop restart/resume to ensure correct flow
* fix: avoid chmod on cache and input dirs in Env timeout wrapper
* fix: skip chmod on 'cache' and 'input' dirs using find -prune
* fix: restrict chmod to immediate mount dirs excluding cache/input
* fix: chmod cache and input dirs alongside their contents after entry run
* fix: guard chmod with directory checks for cache and input
* fix: prefix mount_path in chmod command for cache/input dirs
* fix: drop quotes from find exclude patterns to ensure chmod executes
* fix: skip chmod on cache/input directories to avoid warning spam
* feat: support string volume mappings and poll subprocess stdout/stderr
* support remove symbolic link
* test: use dynamic home path and code volume in LocalEnv local_simple
* fix: skip trace and progress update when loop step is withdrawn
* refactor: add clean_workspace util and non-destructive workspace backup
* fix: preserve symlinks when backing up workspace with copytree
* fix: prevent AttributeError when _pbar not yet initialized in LoopBase
* perf: replace shutil.copytree with rsync for faster workspace backup
* fix: cast log directory Path to str in tar command of data science loop
* fix: use portable 'cp -r -P' instead of rsync for workspace backup
* fix: add retry and logging to workspace backup for robustness
* refactor: extract backup_folder helper and reuse in DataScienceRDLoop
* fix: propagate backup errors & default _pbar getattr to avoid error
* fix the division by zero bug
* refactor: execute RD loops via asyncio.run and add necessary imports
* lint
* lint
* lint
---------
Co-authored-by: Xu <v-xuminrui@microsoft.com>
* ignore result csv file
* fix app scripts
* rename taskgenerator to developer and generate to develop
* fix a config bug in coder
* fix a small bug in factor coder evaluators
* remove a single logger in factor coder evaluators
* fix a small bug in model coder main.py
* rename Implementation to Workspace
* move the prepare the inject_code into FBWorkspace to align all the behavior
* fix a small bug in model feedback
* remove debug lines for multi processing and simplify evaluators multi proc
* add a copy function to workspace to freeze the workspace && add config prefix to speed up debugging
* make hypothesisgen a abc class
* use Qlib***Experiment
* fix a small bug
* rename Imp to Ws
* rename sub_implementations to sub_workspace_list
* fix a bug in feedback not presented as content in prompts
* move proposal pys to proposal folder
* reformat the folder
* align factor and model qlib workspace and use template to handle the workspace
* add a filter to evoagent to filter out false evo
* align multi_proc_n into RDAGENT seeting
* handle when runner gets empty experiment
* fix logger merge remaining problems
* fix black and isort automatically
* Implemented model.py
- Need to run within the RDAgent folder (relevant path)
- Each time copy a template & insert code & run qlib & store result back to experiment
* Create model.py
* Create conf.yaml
This is the sample conf.yaml to be copied each time.
This has gone several times of iteration and is now working for both tabular and Time-Series data.
* Create read_exp.py
This is to read the results within Qlib
* Create ReadMe.md
* Update model.py
* Create test_model.py
A testing file that separates model code generation and running&feedback section.
* move the template folder
* help xisen finish the model runner
* help xisen fix improve model feedback generation
* delete debug file
* rename readme.md
---------
Co-authored-by: Xisen Wang <118058822+Xisen-Wang@users.noreply.github.com>
* use CoSTEER as component name
* rename factorimplementation to avoid confusion
* rename modelimplementation
* align benchmark and evolving evaluators
* add scenario to evaluator init function
* rename all factorimplementationknowledge in CoSTEER
* remove all scenario related information in component
* remove useless code
---------
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
* update all code
* save code
* update first version of factor proposal
* change a comment
* remove a useless comment
---------
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
you-n-g