Files
NexQuant/rdagent/app/benchmark/model
Xu Yang fce241b9f9 feat: a unified CoSTEER to fit more scenarios (#491)
* Use ExtendedBaseSettings to replace BaseSettings

* update a more general way to pass the default setting

* update all code

* fix CI

* fix CI

* fix qlib scenario

* fix CI

* fix CI

* fix CI & add data science interfaces

* remove redundant code

* abandon costeer knowledge base v1

---------

Co-authored-by: Xu Yang <xuyang1@microsoft.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
2024-11-25 16:27:34 +08:00
..
2024-08-09 12:58:34 +08:00

Tasks

Task Extraction

From paper to task.

# python rdagent/app/model_implementation/task_extraction.py
# It may based on rdagent/document_reader/document_reader.py
python rdagent/components/task_implementation/model_implementation/task_extraction.py ./PaperImpBench/raw_paper/

Complete workflow

From paper to implementation

# Similar to
# rdagent/app/factor_extraction_and_implementation/factor_extract_and_implement.py

Paper benchmark

# TODO: it does not work well now.
python rdagent/app/model_implementation/eval.py

TODO:

  • Create reasonable benchmark
    • with uniform input
    • manually create task
  • Create reasonable evaluation metrics

Evolving