* remove AttributeError caused by select_threshold
* develop with ground truth
* raise exception for eval_case
* Revert "develop with ground truth"
This reverts commit e68c136588685476f32a3c3696a6a33deb47acc5.
* Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio
* reformatted by black
* Set the threshold for FactorIndexEvaluator to 0.99
* Apply suggestions from code review
* Update rdagent/components/coder/factor_coder/CoSTEER/evaluators.py
* geometric mean for format_succ_rate
* no need to check when similarity is high enough
* black reformat
---------
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
* Init todo
* Evaluation & dataset
* Generate new data
* dataset generation
* add the result
* Analysis
* Factor update
* Updates
* Reformat analysis.py
* CI fix
* Further Optimised Model Workflow by Incorporating Feedbacks on Exp Task Card
* Rebasing To build the extraction & implementation demo
* Revised for clean code
* Revised further to show "Knowledge"
* Revised to make model_research_copilot better
* Further Optimised Model Workflow by Incorporating Feedbacks on Exp Task Card
* Rebasing To build the extraction & implementation demo
* Revised for clean code
* Revised further to show "Knowledge"
* Revised to make model_research_copilot better
---------
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com>
* ignore result csv file
* fix app scripts
* rename taskgenerator to developer and generate to develop
* fix a config bug in coder
* fix a small bug in factor coder evaluators
* remove a single logger in factor coder evaluators
* fix a small bug in model coder main.py
* rename Implementation to Workspace
* move the prepare the inject_code into FBWorkspace to align all the behavior
* fix a small bug in model feedback
* remove debug lines for multi processing and simplify evaluators multi proc
* add a copy function to workspace to freeze the workspace && add config prefix to speed up debugging
* make hypothesisgen a abc class
* use Qlib***Experiment
* fix a small bug
* rename Imp to Ws
* rename sub_implementations to sub_workspace_list
* fix a bug in feedback not presented as content in prompts
* move proposal pys to proposal folder
* reformat the folder
* align factor and model qlib workspace and use template to handle the workspace
* add a filter to evoagent to filter out false evo
* align multi_proc_n into RDAGENT seeting
* handle when runner gets empty experiment
* fix logger merge remaining problems
* fix black and isort automatically
* use CoSTEER as component name
* rename factorimplementation to avoid confusion
* rename modelimplementation
* align benchmark and evolving evaluators
* add scenario to evaluator init function
* rename all factorimplementationknowledge in CoSTEER
* remove all scenario related information in component
* remove useless code
---------
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
* Commit init framework
* Co-authored-by: Yuante Li (FESCO Adecco Human Resources) <v-yuanteli@microsoft.com>
Co-authored-by: XianBW <XianBW@users.noreply.github.com>
* add an import
* refine the whole framework
* benchmark related framework
* fix black and isort errors
* move requirements to folder
* fix black again
---------
Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
you-n-g