* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* imp SOTA-Jump policy
* fix small bug
* allow to set different selector by .env
* add always-win selector
* add init length for AlwaysWinCKPSelector
* add back_jump selector
* auto lint
* add sota_exp_to_submit attribute; change the name of ckp_selector and sota-selector
* fix bug
* auto lint
* working on auto sota selector
* add subtrace counter
* fix bug, remove unuse selector
* add auto sota selector
* auto lint
* fix bug
* fix small logic bug
* add logging
* add inject_diverse feat
* auto lint
* capable to None-select
* feat: add hypothesis_gen config and ExpGen2TraceAndMerge functionality
* refactor: use dynamic import for experiment generator instantiation
* feat: add BestValidSelector for improved SOTA experiment selection
* runnable twin-trace version
* fix logic error of trace-merge
* auto lint
* use import_class to set selector,
* auto-lint
---------
Co-authored-by: Young <afe.young@gmail.com>
* custom data
* fix: simplify competition check and log local description file
* no sample data
* feat: add test evaluation module with error handling support
* fix: update eval path to use eval_sub_dir and add valid_check TODO
* refactor: add MLETestEval check to conditionally run grading steps
* avoid blank stdout
* valid in testeval
* rename test.csv to avoid conflict
* Support Disabling sample submission
* refactoring
* fix: remove DS_KAGGLE_DATA and update prompt instructions
* add try for grade
* ignore submission
* fix: remove tee from eval command and warn about pipeline exit code detection
* optional to use raw description
* support old data
* add execution result to stdout
* add metric to raw description
* custom data explain
* add debug_path
* rst update
---------
Co-authored-by: Young <afe.young@gmail.com>
* refactor: use dynamic input path and update template loader
* fix: update include syntax for data source in prompts.yaml
* add customization path
* docs: update prompts for ensemble scoring and metric direction
* chore: remove obsolete data_science/share.yaml file
* feat: add model dump flag and multi-evaluator support
* tmp code
* refactor: update evaluator feedback and FBWorkspace types
* feat: add get_clear_ws_cmd and CPU count in Docker environment
* feat: Add model dump check level and enhance evaluator functionality
fix data type bug
* fix: Ensure required files exist before model dump evaluation
* refactor: streamline prompt and file checks in model dump evaluation
* fix: add assertions and reorder file reads in model dump evaluator
* feat: remove EDA part from evaluation output
* docs: update dump_model guidelines and eval prompt to include template
* style: reformat multiline dicts and lists in conf and eval files
* fix: add DOTALL flag to EDA removal regex
* rebase selection code
* bug-free run: checkpoint selection and dynamic EDA loading
* add prototypes of various selectors, to imp. and test later
* fix EDA write bug
* move selector to from proposal.py tp seletc.py
* auto lint
* fix line-too-long typos
* aligh the design of "selection", rm extra instance check
* make auto-lint
* add non-trival selector: SOTAjump
* init commit
* limit problem numbers
* ensemble lower case
* add runtime and spec to coder
* submission check notice
* sub EDA in sample execution
* avoid lightgbm
* add time limit to scenario
* rephrase the submission check
* give positive feedback when facing warning in check
* ENABLE FEEDBACK
* fix feedback bug
---------
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
you-n-g