Commit Graph

186 Commits

Author SHA1 Message Date
WinstonLiyt e152c68c88 fix: refine the prompt (#286) 2024-09-20 18:29:07 +08:00
Way2Learn 7ade6d2db1 feat: Model context for tuning and selection (#284)
* Revised for tuning

* Fixed a bug

* fix a ci bug

---------

Co-authored-by: WinstonLiye <1957922024@qq.com>
2024-09-20 16:40:47 +08:00
Way2Learn 9f8a041b96 feat: Feature selection v3 to support all actions (#280)
* Update feedback.py to support all actions

Feedback.py is updated to support all actions.

* Update prompts.yaml to support all actions

* Revised for CI

* CI

* fix a ci bug

* fix a ci bug

---------

Co-authored-by: WinstonLiye <1957922024@qq.com>
2024-09-20 16:06:49 +08:00
WinstonLiyt a2d86f2efe feat: Add runtime measurement for each step and loop in RDLoop. (#281)
* Add runtime measurement for each step and loop in RDLoop.

* refine some codes

* refine the code (#276)

* show variables only when it exists (#277)

* fix: support seed and fix absolute path (#278)

* fix: support seed and fix absolute path

* Absolute path

* lint

* fix: improve_execution_time_in_kaggle_loop (#279)

* improve_execution_time_in_kaggle_loop

* fix CI

* fix CI

* fix CI

* fix: Update runner.py to fix a small bug (#282)

* fix: Update runner.py to fix a small bug

* fix CI

* refine the code

* Update loop.py

* Update rd_loop.py

* Update model_xgb.py

---------

Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-09-20 16:05:59 +08:00
Haoran Pan 2dd1f342d4 fix ensemble bug in train.py (#283) 2024-09-20 15:33:38 +08:00
Xu Yang ed2130d8aa fix: Update runner.py to fix a small bug (#282)
* fix: Update runner.py to fix a small bug

* fix CI
2024-09-20 14:03:46 +08:00
Xu Yang 0d36157a80 fix: improve_execution_time_in_kaggle_loop (#279)
* improve_execution_time_in_kaggle_loop

* fix CI

* fix CI

* fix CI
2024-09-20 12:07:13 +08:00
you-n-g 431202602e fix: support seed and fix absolute path (#278)
* fix: support seed and fix absolute path

* Absolute path

* lint
2024-09-19 22:03:33 +08:00
XianBW fd65cff556 show variables only when it exists (#277) 2024-09-19 17:37:29 +08:00
WinstonLiyt ce5f26f308 refine the code (#276) 2024-09-19 17:15:01 +08:00
WinstonLiyt e9846f4ae3 fix: fix some bugs in the entire loop (#274)
* fix some bugs in the entire loop

* refine the code
2024-09-19 16:06:57 +08:00
Xu Yang 979bb6707a fix a small bug in kaggle scenario caused by the second RAG update (#273) 2024-09-19 13:31:27 +08:00
Xu Yang e9395ee045 fix: update new feature engineering code format (#272)
* update new feature engineering code format

* fix CI
2024-09-19 12:56:48 +08:00
WinstonLiyt 0166273e15 feat: Added support for loading and storing RAG in Kaggle scenarios. (#269)
* init a scenario for kaggle feature engineering

* Added support for loading and storing RAG in Kaggle scenarios.

* fix a ci bug

* Add RAG after each experiment's feedback.

* add a promt

* fix a bug

* fix a bug

* add a readme

* refine the code in knowledge loading
2024-09-19 11:33:33 +08:00
you-n-g fe2da1a68f feat: add kaggle command (#271)
* feat: add kaggle command

* lint
2024-09-18 16:48:54 +08:00
you-n-g 1cee836206 fix: actively raised errors aer also considered as negative feedback. (#268) 2024-09-18 11:40:54 +08:00
Haoran Pan 2cbd87c635 modify prompts to run kaggle loops (#266) 2024-09-16 15:36:16 +08:00
Way2Learn 1ff4b9f992 feat: Added RepoAnalyzer to empower auto-summary of a workspace (#264)
* Init todo

* Evaluation & dataset

* Generate new data

* dataset generation

* add the result

* Analysis

* Factor update

* Updates

* Reformat analysis.py

* CI fix

* Added a powerful RepoAnalyser

* Improved display of signature

* removed testing

* Delete rdagent/components/benchmark/eval_method.py

* refine the code for ci test

* fix a bug

* fix a ci error

* add a exp

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com>
Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
Co-authored-by: WinstonLiye <1957922024@qq.com>
2024-09-14 13:58:36 +01:00
Tim 9597e0d860 fix: eval_method cannot catch run factor error (#260)
* Update eval_method.py

abort subsequent evaluators for invalid gen_df

* add AttributeError

* add raise_exception for eval_method

* remove custom exception catch
2024-09-14 17:47:27 +08:00
WinstonLiyt 1332003281 feat: Integrate RAG into the Kaggle scenarios. (#262)
* init a scenario for kaggle feature engineering

* add 1st version rag for Kaggle hypo

* refine the code

* fix a bug

* add the process of extracting exp from docs

* Remove the unnecessary file.

* refine the code for ci test

* Delete rdagent/app/kaggle_feature/conf.py

* refine the comments

* Update extract_experience_from_docs.py

* Update extract_experience_from_docs.py
2024-09-13 18:34:57 +08:00
XianBW e75706c8ec fix rdagent factor bug when FactorTask init (#261) 2024-09-12 18:04:21 +08:00
WinstonLiyt 82bef3a375 feat: fix some bugs and add original features' description (#259)
* init a scenario for kaggle feature engineering

* fix some bugs and add original features' description

* refine the process of data downloading

* fix a error

* revert the code

* fix a bug in feedback

* fix a ci bug

* fix a ci bug
2024-09-12 00:30:16 +08:00
Xu Yang dbe2cf12bb feat: Kaggle loop update (Feature & Model) (#241)
* Init todo

* Evaluation & dataset

* Generate new data

* dataset generation

* add the result

* Analysis

* Factor update

* Updates

* Reformat analysis.py

* CI fix

* Revised Preprocessing & Supported Random Forest

* Revised to support three models with feature

* Further revised prompts

* Slight Revision

* docs: update contributors (#230)

* Revised to support three models with feature

* Further revised prompts

* Slight Revision

* feat: kaggle model and feature (#238)

* update first version code

* make hypothesis_gen and experiment_builder fit for both feature and model

* feat: continue kaggle feature and model coder (#239)

* use qlib docker to run qlib models

* feature coder ready

* model coder ready

* fix CI

* finish the first round of runner (#240)

* Optimized the factor scenario and added the front-end.

* fix a small bug

* fix a typo

* update the kaggle scenario

* delete model_template folder

* use experiment to run data preprocess script

* add source data to scenarios

* minor fix

* minor bug fix

* train.py debug

* fixed a bug in train.py and added some TODOs

* For Debugging

* fix two small bugs in based_exp

* fix some bugs

* update preprocess

* fix a bug in preprocess

* fix a bug in train.py

* reformat

* Follow-up

* fix a bug in train.py

* fix a bug in workspace

* fix a bug in feature duplication

* fix a bug in feedback

* fix a bug in preprocessed data

* fix a bug om feature engineering

* fix a ci error

* Debugged & Connected

* Fixed error on feedback & added other fixes

* fix CI errors

* fix a CI bug

* fix: fix_dotenv_error (#257)

* fix_dotenv_error

* format with isort

* Update rdagent/app/cli.py

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>

* chore(main): release 0.2.1 (#249)

Release-As: 0.2.1

* init a scenario for kaggle feature engineering

* delete error codes

* Delete rdagent/app/kaggle_feature/conf.py

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com>
Co-authored-by: Xisen-Wang <xisen_application@163.com>
Co-authored-by: Haotian Chen <113661982+Hytn@users.noreply.github.com>
Co-authored-by: WinstonLiye <1957922024@qq.com>
Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com>
2024-09-11 15:26:52 +08:00
Linlang c8d0d59308 fix: fix_dotenv_error (#257)
* fix_dotenv_error

* format with isort

* Update rdagent/app/cli.py

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2024-09-10 19:38:35 +08:00
you-n-g a7e76a9762 fix: default model value in config (#256)
* fix: git default model value

* Update README.md
2024-09-10 12:49:06 +08:00
Linlang 80d4f32c8f feat: add collect info (#233)
* add collect info

* fix isort error

* optimize code

* fix black error

* Update rdagent/app/cli.py

* Modify the code according to the comments

* fix isort error

* add docker info

* docs: update contributors (#230)

* fix: package dependency. (#234)

* fix package

* fix lint

* docs: Update development.rst (#235)

* feat: add cross validation for kaggle scenario (#236)

* update cross validation for kaggle scenario

* CI Issues

* delete useless file

* CI issues

* docs: Update README.md (#245)

* docs: refine the README (#244)

* init a scenario for kaggle feature engineering

* update the readme

* Delete rdagent/app/kaggle_feature/conf.py

* update some pictures

* Delete rdagent/app/kaggle_feature/model.py

* change a photo

* add pics to docs

* update the readme

* update the README

* for a try

* for another try

* change the style of the pictures in readme

* fix a small bug

* update the readme and the docs

* update the docs

* fix a typo

* change a website url

* change some styles

* fix a typo

* change the size of the logo

* change two urls

* Update README.md

* Update README.md

* Update README.md

* Update README.md

---------

Co-authored-by: Linlang <Lv.Linlang@hotmail.com>

* move the component to other files

* last container

* optimize rdagent info

* format with isort

* format with black

* format_with_black

* fix pip error

* format with isort

* change requirements

* fix pip error

* fix_pip_error

* fix pip error

* format with black

* fix pip error

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Haotian Chen <113661982+Hytn@users.noreply.github.com>
Co-authored-by: Haoran Pan <167847254+TPLin22@users.noreply.github.com>
Co-authored-by: Way2Learn <118058822+Xisen-Wang@users.noreply.github.com>
Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
Co-authored-by: Young <afe.young@gmail.com>
2024-09-07 12:37:08 +08:00
Linlang 9dd35307a1 test: add test import (#242)
* add test import

* format with isort

* format with black

* merge main

* fix pytest error

* fix pytest error

* fix pytest error

* fix pytest error

* format with black

* fix pytest error

* fix pytest error

* fix pytest error

* fix pytest error

* fix pytest error

* format with isort

* Exclude entrance

* Add offline test

* auto-lint

* update coverage rate

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-09-06 17:18:52 +08:00
WinstonLiyt 0dc9247153 docs: refine the README (#244)
* init a scenario for kaggle feature engineering

* update the readme

* Delete rdagent/app/kaggle_feature/conf.py

* update some pictures

* Delete rdagent/app/kaggle_feature/model.py

* change a photo

* add pics to docs

* update the readme

* update the README

* for a try

* for another try

* change the style of the pictures in readme

* fix a small bug

* update the readme and the docs

* update the docs

* fix a typo

* change a website url

* change some styles

* fix a typo

* change the size of the logo

* change two urls

* Update README.md

* Update README.md

* Update README.md

* Update README.md

---------

Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
2024-09-06 15:39:41 +08:00
Haoran Pan 5c5853c7fc feat: add cross validation for kaggle scenario (#236)
* update cross validation for kaggle scenario

* CI Issues

* delete useless file

* CI issues
2024-09-01 17:41:05 +08:00
you-n-g 085c115961 fix: package dependency. (#234)
* fix package

* fix lint
2024-08-31 20:35:04 +08:00
WinstonLiyt 6ce75c6f01 feat: piloting of the framework (#227)
* Initial template

* Updated a bit on the framework

* Revised to enable feat training

* Fixed Accuracy

* fixed mcc

* change some files for feature

* update some logics in feat eng

* update some codes

* update some codes

* Revised further to successfully completed a cycle

* Revised further

* Delete two CSVs

* fix some ci errors

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Xisen-Wang <xisen_application@163.com>
2024-08-27 17:27:11 +08:00
Way2Learn e7b8a43f52 feat: update model_experiment.py to support basic EDA (#220)
* Update model_experiment.py to support basic eda

It looks into the data first before the proposal.

* Update model_experiment.py

Revised linting

* Update model_experiment.py by fixing sorting order

* Update model_experiment.py for black linting

* Update model_experiment.py

* Update model_experiment.py

* Update model_experiment.py

* Update model_experiment.py

---------

Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
2024-08-26 16:38:49 +08:00
Tim 79ee05c1dd fix: remove useless line (#177)
* Update Dockerfile

remove unused line

* specify a fixed version of scipy
2024-08-23 16:08:45 +08:00
TPLin22 9a792cc21e feat: support more models for kaggle scenario (#223)
* init commit for XGBoost

* fix some bugs

* CI issues

* CI issues

* CI issue

* edit prompts for kaggle scenario & fix some bugs

* Revised Prompts To Improve Performance on Model Type & Support of Random Forest

* edit prompts & modify evaluator.py to adapt to Kaggle scenario

* edit prompts

* fix some bugs

* CI issues

---------

Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: Xisen-Wang <xisen_application@163.com>
2024-08-23 15:13:50 +08:00
WinstonLiyt c8e8989edc fix: fix some bugs in llm calling (#217)
* fix some bugs in llm calling

* fix a CI error

* fix a small bug in kmeans group number choosing

* fix a ci bug
2024-08-21 16:48:09 +08:00
Xu Yang e743b2161f use sqlite to store session history and enable reproduction in k-means (#219) 2024-08-20 17:49:45 +08:00
Tim b71adbbc0f check if gen_df is None (#216) 2024-08-20 12:27:13 +08:00
XianBW 16dcc3a772 feat: add progress status for docker env (#215)
* add progress status for docker env
2024-08-19 19:10:30 +08:00
WinstonLiyt 96424bd1ff feat: Added loop code for Kaggle scene. (#211)
* fuse all code into one commit

* remove container auto

* change remove method

* add kaggle env start

* change kaggle api

* change structure

* add crawler

* add requirements

* refeact the code

* delete mistaken codes

* merge docker settings and crawler

* add chrome install README for crawler usage

* Connect scen with Kaggle to download data

* Reformat some files to pass CI.

* fix some ci errors

* fix a ci error

* fix a ci error

---------

Co-authored-by: Bowen Xian <xianbowen@outlook.com>
2024-08-19 10:58:28 +08:00
Xu Yang 2aab3f5cee add relevance check to quant factors (#210) 2024-08-15 18:52:37 +08:00
XianBW 51d28db28b feat: Demo display effect and usage (#162)
Demo changes
2024-08-09 20:40:16 +08:00
XianBW 9da4a76b7e feat: change ui entry (#197)
change ui start cmd
2024-08-09 17:23:45 +08:00
WinstonLiyt f9cd22e05c docs: Updated documentation for all the major scenarios. (#190)
* Fixed some bugs introduced during refactoring.

* update data_agent_fin_doc

* Updated documentation for the four major scenarios

* feat: remove pdfs and enable online pdf readings (#183)

* remove pdfs and enable online pdf readings

* update doc format

* use url as key

* feat: add entry for rdagent. (#187)

* Add entries

* update entry for rdagent

* lint

* fix typo

* docs: Demo links (#188)

add demo links

* fix: Fix a fail href in readme (#189)

* fix a ci bug

* doc

* feat: remove pdfs and enable online pdf readings (#183)

* remove pdfs and enable online pdf readings

* update doc format

* use url as key

* feat: add entry for rdagent. (#187)

* Add entries

* update entry for rdagent

* lint

* fix typo

* doc

* Updated documentation for med_model scenarios.

* fix a ci bug

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: SH-Src <suhan.c@outlook.com>
2024-08-09 13:27:33 +08:00
Xu Yang 007b632497 fix: fix quick start problem (#191)
* all update code

* fix a small bug
2024-08-09 12:58:34 +08:00
you-n-g 5c061d452f feat: add entry for rdagent. (#187)
* Add entries

* update entry for rdagent

* lint

* fix typo
2024-08-08 17:10:37 +08:00
Xu Yang 8c83317528 feat: remove pdfs and enable online pdf readings (#183)
* remove pdfs and enable online pdf readings

* update doc format

* use url as key
2024-08-08 15:20:39 +08:00
WinstonLiyt 60e09c3a53 fix: Add framework handling for task coding failure. (#176)
* Add framework handling for task coding failure.

* fix a ci bug
2024-08-07 19:01:46 +08:00
Xisen Wang 087c1c4b66 doc: Update General Model Scenario Documentation (#173)
* Init todo

* Evaluation & dataset

* Generate new data

* dataset generation

* add the result

* Analysis

* Factor update

* Updates

* Reformat analysis.py

* CI fix

* Revised General Model Doc & Layout

* Update model_copilot_general.rst

* Update eval.py

* Update eval.py

* Update analysis.py

* Update eval_method.py

* Update model_copilot_general.rst to add configurations

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com>
Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
2024-08-06 17:44:10 +08:00
Xu Yang 9c0d3df8f6 put all the property in scenarios into local attribute to enable static scenario content (#175) 2024-08-06 17:07:01 +08:00
WinstonLiyt ba78a87dcb feat: Add description for scenario experiments. (#174)
* Fixed some bugs introduced during refactoring.

* Add description for scenario experiments.

* Update factor_from_report_w_sc.py

* fix some ci bugs.

* add @property to get_experiment_setting

* fix a ci error.
2024-08-06 16:03:40 +08:00