feat: advanced checkpoint selectors (#790)

* rebase selection code

* bug-free run: checkpoint selection and dynamic EDA loading

* add prototypes of various selectors, to imp. and test later

* fix EDA write bug

* imp SOTA-Jump policy

* fix small bug

* allow to set different selector by .env

* add always-win selector

* add init length for AlwaysWinCKPSelector

* add back_jump selector

* auto lint

* add sota_exp_to_submit attribute; change the name of ckp_selector and sota-selector

* fix bug

* auto lint

* working on auto sota selector

* add subtrace counter

* fix bug, remove unuse selector

* add auto sota selector

* auto lint

* fix bug

* fix small logic bug

* add logging

* add inject_diverse feat

* auto lint

* capable to None-select

* feat: add hypothesis_gen config and ExpGen2TraceAndMerge functionality

* refactor: use dynamic import for experiment generator instantiation

* feat: add BestValidSelector for improved SOTA experiment selection

* runnable twin-trace version

* fix logic error of trace-merge

* auto lint

* use import_class to set selector,

* auto-lint

---------

Co-authored-by: Young <afe.young@gmail.com>
This commit is contained in:
xuangu-fang
2025-05-09 15:38:25 +08:00
committed by GitHub
parent 5c7cdf298c
commit e71d8f6c3c
13 changed files with 486 additions and 124 deletions
@@ -127,6 +127,13 @@ class PipelineCoSTEEREvaluator(CoSTEEREvaluator):
eda_output = implementation.file_dict.get("EDA.md", None)
eda_output = implementation.file_dict.get("EDA.md", None)
if not isinstance(implementation, FBWorkspace):
eda_output = None
else:
eda_output = implementation.file_dict.get("EDA.md", None)
system_prompt = T(".prompts:pipeline_eval.system").r(
scenario=self.scen.get_scenario_all_desc(eda_output=eda_output),
task_desc=target_task.get_task_information(),