Commit Graph

40 Commits

Author SHA1 Message Date
you-n-g 40d108687c Support run one step for debugging (#99) 2024-07-23 19:00:59 +08:00
Xisen Wang 44169507b2 Merge pull request #97 from microsoft/recover-branch-xisen-3
Further Optimisation For Model
2024-07-23 17:19:17 +08:00
you-n-g 4f484e009a Workflow Support Loading and saving sessions (#98)
* Successfully logging the trace

* Start debugging & Add policy file

* Support loading sessions

* Add docs

* Add tqdm
2024-07-23 16:37:41 +08:00
Xisen-Wang 81fe304127 Further Optimised Kwargs & Confs 2024-07-23 06:32:26 +00:00
WinstonLiyt 7220d33668 Update the detailed process and prompt of factor loop. (#96)
Update the detailed process and prompt of factor loop.
2024-07-22 18:18:08 +08:00
WinstonLiyt 761df330de Fix a small bug in file reading. 2024-07-22 14:40:14 +08:00
WinstonLiyt c17244a317 Extract factors from financial reports loop finished. (#90)
- Extract factors from financial reports loop finished.

- Fix some small bugs.
2024-07-20 12:31:40 +08:00
Xisen Wang a59ab982c9 Merge pull request #86 from microsoft/recover-branch-xisen-3
Model Optimisation Update 

After optimization, the rd agent loop for model in the qlib scenario should now work.
2024-07-19 15:05:24 +08:00
Linlang 276b462f40 Fix ruff error1 (#81)
* fix_ruff_error1

* fix_ruff_error

* fix ruff error

* fix ruff error

* pass model.py

* rename exception class

* rename exception class

* rename func name generate_feedback

* remove prepare args

* optimize code

* optimize code

* fix code error
2024-07-18 22:36:04 +08:00
you-n-g c0e9a5637d Update rdagent/app/qlib_rd_loop/RDAgent.py 2024-07-18 18:33:31 +08:00
Xisen-Wang 82c00d8507 Revised all prompts 2024-07-18 09:42:55 +00:00
Xisen-Wang ee486e9d2d Trying 2024-07-18 09:42:55 +00:00
Xu Yang a9cb50f3c4 add_logger_to_factor_and_fix_a_small_bug_in_factor_runner (#85) 2024-07-18 17:00:41 +08:00
you-n-g 9f89511be6 Model run with logger (#79) 2024-07-17 17:29:39 +08:00
Xu Yang e0a24fb46f Several update on the repo (see desc) (#76)
* ignore result csv file

* fix app scripts

* rename taskgenerator to developer and generate to develop

* fix a config bug in coder

* fix a small bug in factor coder evaluators

* remove a single logger in factor coder evaluators

* fix a small bug in model coder main.py

* rename Implementation to Workspace

* move the prepare the inject_code into FBWorkspace to align all the behavior

* fix a small bug in model feedback

* remove debug lines for multi processing and simplify evaluators multi proc

* add a copy function to workspace to freeze the workspace && add config prefix to speed up debugging

* make hypothesisgen a abc class

* use Qlib***Experiment

* fix a small bug

* rename Imp to Ws

* rename sub_implementations to sub_workspace_list

* fix a bug in feedback not presented as content in prompts

* move proposal pys to proposal folder

* reformat the folder

* align factor and model qlib workspace and use template to handle the workspace

* add a filter to evoagent to filter out false evo

* align multi_proc_n into RDAGENT seeting

* handle when runner gets empty experiment

* fix logger merge remaining problems

* fix black and isort automatically
2024-07-17 15:00:13 +08:00
XianBW eee2b3c56a adjust logging system (#51)
* remove ruff comment in log.py

* change log framework and fix llm_utils.py's logs

* Some thoughts for logging

* fix SingletonMeta's definition, maintain an instance dict for each class that inherits it

* adjust log codes directory, add some tag for factor implementation logging

* Update rdagent/core/conf.py

* fix factor task app & log

* fix log import

* Streamlet framework

* fix log tag to path logic

* Add todos

* Add example in docstring

* add log tag for llm_utils.py

* Capture lost content

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2024-07-16 20:35:42 +08:00
Xu Yang be2c19307e first version of model runner and model feedback (#70)
* Implemented model.py

- Need to run within the RDAgent folder (relevant path)
- Each time copy a template & insert code & run qlib & store result back to experiment

* Create model.py

* Create conf.yaml

This is the sample conf.yaml to be copied each time.

This has gone several times of iteration and is now working for both tabular and Time-Series data.

* Create read_exp.py

This is to read the results within Qlib

* Create ReadMe.md

* Update model.py

* Create test_model.py

A testing file that separates model code generation and running&feedback section.

* move the template folder

* help xisen finish the model runner

* help xisen fix improve model feedback generation

* delete debug file

* rename readme.md

---------

Co-authored-by: Xisen Wang <118058822+Xisen-Wang@users.noreply.github.com>
2024-07-16 10:33:53 +08:00
Xu Yang 1ed2f655b9 help yuante on the final version of data code 2024-07-15 08:28:34 +00:00
WinstonLiyt 2bd2ad3896 String together the entire factor process 2024-07-11 11:24:41 +00:00
WinstonLiyt bf1b140f68 re-commit 2024-07-11 08:49:37 +00:00
Xu Yang b1ddba388a model proposal first version (#61)
* init code

* first version of model proposal
2024-07-11 10:50:34 +08:00
Xu Yang 720994c8e0 Implement model (and some factor) coder with evolving (#52)
* store code into FBImplementation

* fix path related bugs

* fix a bug

* fix factor related small bugs

* re-submit all model related code

* new code to model coder

* finish the model evolving code

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-07-10 15:45:43 +08:00
Xisen Wang 7350d9aa1f refine core to store experiment results and hypothesis feedback (#55)
* Update proposal.py

Completed The HypothesisFeedback Class.

* refine the core code

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-07-09 17:31:35 +08:00
Xu Yang 1d9b4cd2ec Align factor coder into new framework (#47)
* use CoSTEER as component name

* rename factorimplementation to avoid confusion

* rename modelimplementation

* align benchmark and evolving evaluators

* add scenario to evaluator init function

* rename all factorimplementationknowledge in CoSTEER

* remove all scenario related information in component

* remove useless code

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-07-05 17:42:00 +08:00
Xu Yang f61453fbb1 First version of factor idea proposal (#46)
* update all code

* save code

* update first version of factor proposal

* change a comment

* remove a useless comment

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-07-04 15:56:14 +08:00
Xu Yang d88ddc0c1c Build model class inheritance (#44)
* update all code

* fix a typo

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-07-03 17:42:07 +08:00
XianBW ab2f61fe9d adjust new core classes for CI (#43)
* add README for CI tool
* adjust new core classes for CI
2024-07-03 16:34:47 +08:00
Xu Yang 1b31bcd2c2 refine class design and inheritance first version code (#41)
* refine class design and inheritance first version code

* fix all typos

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-07-02 17:58:37 +08:00
Xinjie Shen b2fb104108 Pdf2 model task (#33)
* add needed dependency

* add extract_model_and_implement pipeline

* add merge_file_to_model_dict_to_model_dict

* implement `rdagent\app\model_implementation\eval.py`

* Running benchmark

* refine import

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-06-30 23:31:00 +08:00
Xu Yang 6b626eb56d New Framework for idea proposal and implementation on RD-Agent (#34)
* Commit init framework

* Co-authored-by: Yuante Li (FESCO Adecco Human Resources) <v-yuanteli@microsoft.com>
Co-authored-by: XianBW <XianBW@users.noreply.github.com>

* add an import

* refine the whole framework

* benchmark related framework

* fix black and isort errors

* move requirements to folder

* fix black again

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-06-28 11:45:23 +08:00
Haoxue 07a77dd802 Eval process (#31)
* test data load process and fix bug

* fix bug when evaluating

* refine json content

---------

Co-authored-by: USTCKevinF <fengwenjun@mail.ustc.edu.cn>
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-06-27 16:39:17 +08:00
Xinjie Shen f20dc4482b example workflow code for model implementation (#17)
* upload example code for model implementation

* Refactor Model Implement

* add export

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-06-21 16:41:34 +08:00
Xu Yang a126c84c92 Refine all the implementation code to higher quality for release (#29)
* refine CI script

* refine all the code to higher quality

* refine the script to factor extraction and implementation

* add task loader interface

* add a task loader interface && move pdf analysis to pdf task loader

* change the name to global variables

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-06-18 11:50:03 +08:00
USTCKevinF ebb659a018 reporeformat V2 (#23)
* reformat factor implement process

* move some code to more reasonable place

* fix the bug

* add test function in factor_extract_and_implement.py

* change select factor number to ratio , add some factor implement setting and fix some bug while using knowledgebase

* change evoagent

* add abstract class EvoAgent

* add benchmark workflow

* fix some bug in llm_utils

* run wenjun's code

* fix the knowledgebase instance check

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-06-14 12:59:44 +08:00
XianBW 55959c93ff Ci fix (#22)
* fix replace function of CI tool

* fix ruff errors (ignore some parts)

* add ruff rule ignore comment
2024-06-12 15:12:11 +08:00
XianBW 441ca51af9 CI tool refinement (#13)
* adjust interact and log for CI fix tool

* change method to split large code files

* refine statistics table after fixing one file

* process default --fix option when ruff check

* remove hardcode path

* add tree-sitter requirements

* replace <| |> to < > in comments, for GPT

* update mypy version

* Add Mypy Evaluator
2024-06-07 00:23:11 +08:00
Xu Yang 1e77557293 factor extraction pipeline ready (#16)
* run the code

* update code

* remove some redundant code

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-06-05 15:36:15 +08:00
XianBW 62a2f7a742 add CI fix tool to app (#10)
* add CI fix tool to app

* perform ruff safe fix

* wrap too long lines in prompts.py
2024-05-30 10:33:07 +08:00
Xu Yang c6833b0858 update code (#9)
Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-05-21 22:48:41 +08:00
Xu Yang 02173ffdcc add code security check CI and dependbot (#3)
* update code

* init structure

* add requirements and fix CI

---------

Co-authored-by: xuyang1 <xuyang1@microsoft.com>
2024-05-09 10:35:54 +08:00