Commit Graph

107 Commits

Author SHA1 Message Date
WinstonLiyt 450317d8fe fix a bug in kaggle runner cache and kaggle costeer execute template (#467) 2024-11-04 11:16:16 +08:00
Linlang 6fe0d75b38 chore: remove package (#449)
* remove package

* reformat with black

* fix pytest error

* fix pytest error

* fix pytest error

* add packages

* change code
2024-10-24 20:38:13 +08:00
Tim effce1df15 always return str (#453) 2024-10-23 16:47:36 +08:00
Linlang d3c59b2f33 chore: remove torch in main code (#439)
* use np.ndarray in costeer model evaluators

* remove package

* fix test error

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-10-21 15:56:04 +08:00
Xu Yang b2de6f3190 add inf evaluator to factor costeer and some minor improvement (#435) 2024-10-16 16:00:55 +08:00
Tim 022a558dd1 chore: more optional parameters for running benchmark analysis (#431)
* set title and round

* decision from multiple types

* check if decision is true

* reformat

* remove unused file
2024-10-15 12:41:32 +08:00
Xu Yang 5988683cd9 force use_chat_cache False when facing error in evaluators (#432) 2024-10-15 12:21:33 +08:00
Tim 216719670d convert str to bool (#428) 2024-10-14 17:44:33 +08:00
Xu Yang 25472fc1e3 feat: use unified pickle cacher & move llm config into a isolated config (#424)
* simplify RDAgent conf

* add unified cacher(untested)

* fix small bugs

* fix a bug

* fix a small bug in runner

* use hash_key = None to skip cache

* fix CI

* in factor execution, ignore cache when raise exception

* add file locker to avoid mp calling

* fix CI

* use function __module__ name as folder in cache
2024-10-14 17:34:09 +08:00
Xu Yang f1a841ba14 perf: some small upgrade to factor costeer to improve the performance (#420)
* 1. use dataframe.info instead of head
2. in former trace query, add the latest attempt to the last success execution

* fix CI
2024-10-10 20:29:17 +08:00
Xu Yang f2a5967c12 improve factor costeer: (#417)
1. when facing odd v2_query_component_limit, make from gt bigger than without gt
2. do code evaluator even value does not pass
3. provide the value header to value failed evaluators
2024-10-09 18:56:37 +08:00
WinstonLiyt 59cdf946a7 fix: fix some bugs in rag (#399)
* fix some bugs in rag

* fix a bug in costeer
2024-09-30 02:38:33 +08:00
you-n-g 7fc53940fd feat: Dynamic scenario based on task (#392)
* OAI api

* more comments

* llm utils

* RD-Framework for dynamic scen

* remove useless code

* auto lint

* fix dynamic scennario

* Add parameter

* auto lint

* remove abstractmethod

* fix lint
2024-09-29 18:43:17 +08:00
cyncyw 5cdffe3510 fix: debug dsagent (#387)
* basic dsagent & debug

debug dsagent

change dsagent conf

* Refine changes

* Update vector_base.py
2024-09-29 01:15:33 -04:00
you-n-g 7b865c3337 fix: cache (#383)
* fix: cache

* update cache

* fix

* Fix lint
2024-09-29 12:07:36 +08:00
you-n-g 346261c1ac fix: partial bug in bench (#368)
* Add more log

* Fix eval

* update

* keep ratio <= 1

* feat: cache exception (#369)

* 0 instead of NaN

* remove unused evaluators

* save gen_factor_l_all_rounds

* black reformat

* cache exception

---------

Co-authored-by: Tim <illking@foxmail.com>
2024-09-29 09:59:54 +08:00
WinstonLiyt a885f9cfdc fix a bug in code costeer (#373) 2024-09-28 03:55:41 +08:00
Xu Yang f7c1c4fd74 feat: implement isolated model feature selection loop (#370)
* rename meta_tpl

* use a isolated coder to deal with model feature selection and refine the structure

* fix CI

* fix: fix some errors in scenario.py, proposal.py and runner.py and several complex competition scenarios(#365)

* fix several bugs in proposal and runner

* fix a bug in feedback-prize-english-language-learning

* fix some bugs and templates

* fix the bug in optiver and nlp problem

* delete unnecessary codes

* remove unnecessary codes

* complete forest and s4e8

* push

* feedback & s4e8 &  forest

* optiver finished

* s3e11 & s3e26

* s4e9 finished

* sf-crime finished

* the last one finished

---------

Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
Co-authored-by: WinstonLiyte <1957922024@qq.com>
2024-09-28 00:40:25 +08:00
Tim 0705e52d7d fix undefined error (#363) 2024-09-27 10:42:12 +08:00
Tim b93f034b19 feat: Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio (#328)
* remove AttributeError caused by select_threshold

* develop with ground truth

* raise exception for eval_case

* Revert "develop with ground truth"

This reverts commit e68c136588685476f32a3c3696a6a33deb47acc5.

* Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio

* reformatted by black

* Set the threshold for FactorIndexEvaluator to 0.99

* Apply suggestions from code review

* Update rdagent/components/coder/factor_coder/CoSTEER/evaluators.py

* geometric mean for format_succ_rate

* no need to check when similarity is high enough

* black reformat

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2024-09-27 09:34:55 +08:00
Xu Yang 28dd0a0471 fix: revert model and make SOTA model available to COSTEER (#351)
* revert model and make SOTA model available to COSTEER

* fix CI

* fix CI

* fix CI
2024-09-26 17:54:45 +08:00
Xu Yang e343a29c75 avoid_generating_more_column_than_data (#332) 2024-09-25 16:41:08 +08:00
Xu Yang 67167041e7 align local data folder and kaggle data folder to store mid result (#314) 2024-09-24 17:01:33 +08:00
Way2Learn 8dc837b731 fix: Use fixed file name in model costeer & fixing cache (#311)
* use fixed name in model costeer to enable cache

* fix ci
2024-09-24 07:50:01 +01:00
Xu Yang 1e905dd426 revert: Revert feat: Factor Implement Search Enhancement (#294) (#305)
* Revert "feat: Factor Implement Search Enhancement (#294)"

This reverts commit 4ecf25f0acf2389a172b14d3dab20895daf2ab89.

* fix CI
2024-09-24 12:27:59 +08:00
Xu Yang 2523234d83 fix: update code to fix a small bug in model cache md5 hash (#303)
* update code to fix a small bug in model cache md5 hash

* fix another bug dumping the wrong name to costeer model

* fix a black CI
2024-09-23 20:27:34 +08:00
you-n-g 1f06e5c3cf fix: refactor Bench (#302)
* refactor for better bench

* autolint

* add cmd

* lint
2024-09-23 19:53:05 +08:00
Xu Yang 003b067499 feat: Initial version if Graph RAG in KAGGLE scenario (#301)
* Initial version if Graph RAG in KAGGLE scenario

* fix CI

* fix a small bug

* fix CI

* fix CI

* fix CI
2024-09-23 19:32:01 +08:00
cyncyw 972bb2d99f feat: Factor Implement Search Enhancement (#294)
* Search enhancement

* refactor: reorganize imports for consistency with isort

* reformatterd by black

---------

Co-authored-by: Tim <illking@foxmail.com>
2024-09-23 15:24:26 +08:00
Way2Learn 954270a5ca feat: Model context for tuning and selection (#284)
* Revised for tuning

* Fixed a bug

* fix a ci bug

---------

Co-authored-by: WinstonLiye <1957922024@qq.com>
2024-09-20 16:40:47 +08:00
WinstonLiyt ca6d3194d4 feat: Add runtime measurement for each step and loop in RDLoop. (#281)
* Add runtime measurement for each step and loop in RDLoop.

* refine some codes

* refine the code (#276)

* show variables only when it exists (#277)

* fix: support seed and fix absolute path (#278)

* fix: support seed and fix absolute path

* Absolute path

* lint

* fix: improve_execution_time_in_kaggle_loop (#279)

* improve_execution_time_in_kaggle_loop

* fix CI

* fix CI

* fix CI

* fix: Update runner.py to fix a small bug (#282)

* fix: Update runner.py to fix a small bug

* fix CI

* refine the code

* Update loop.py

* Update rd_loop.py

* Update model_xgb.py

---------

Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: Xu Yang <peteryang@vip.qq.com>
2024-09-20 16:05:59 +08:00
Xu Yang 597b8a37f9 fix: improve_execution_time_in_kaggle_loop (#279)
* improve_execution_time_in_kaggle_loop

* fix CI

* fix CI

* fix CI
2024-09-20 12:07:13 +08:00
you-n-g ca4e7e928b fix: support seed and fix absolute path (#278)
* fix: support seed and fix absolute path

* Absolute path

* lint
2024-09-19 22:03:33 +08:00
Xu Yang 48c227a2e0 fix: update new feature engineering code format (#272)
* update new feature engineering code format

* fix CI
2024-09-19 12:56:48 +08:00
you-n-g 204580bc52 fix: actively raised errors aer also considered as negative feedback. (#268) 2024-09-18 11:40:54 +08:00
Tim a4d5a2d66a fix: eval_method cannot catch run factor error (#260)
* Update eval_method.py

abort subsequent evaluators for invalid gen_df

* add AttributeError

* add raise_exception for eval_method

* remove custom exception catch
2024-09-14 17:47:27 +08:00
WinstonLiyt 542d41aef4 feat: fix some bugs and add original features' description (#259)
* init a scenario for kaggle feature engineering

* fix some bugs and add original features' description

* refine the process of data downloading

* fix a error

* revert the code

* fix a bug in feedback

* fix a ci bug

* fix a ci bug
2024-09-12 00:30:16 +08:00
Xu Yang 68d47e1f1f feat: Kaggle loop update (Feature & Model) (#241)
* Init todo

* Evaluation & dataset

* Generate new data

* dataset generation

* add the result

* Analysis

* Factor update

* Updates

* Reformat analysis.py

* CI fix

* Revised Preprocessing & Supported Random Forest

* Revised to support three models with feature

* Further revised prompts

* Slight Revision

* docs: update contributors (#230)

* Revised to support three models with feature

* Further revised prompts

* Slight Revision

* feat: kaggle model and feature (#238)

* update first version code

* make hypothesis_gen and experiment_builder fit for both feature and model

* feat: continue kaggle feature and model coder (#239)

* use qlib docker to run qlib models

* feature coder ready

* model coder ready

* fix CI

* finish the first round of runner (#240)

* Optimized the factor scenario and added the front-end.

* fix a small bug

* fix a typo

* update the kaggle scenario

* delete model_template folder

* use experiment to run data preprocess script

* add source data to scenarios

* minor fix

* minor bug fix

* train.py debug

* fixed a bug in train.py and added some TODOs

* For Debugging

* fix two small bugs in based_exp

* fix some bugs

* update preprocess

* fix a bug in preprocess

* fix a bug in train.py

* reformat

* Follow-up

* fix a bug in train.py

* fix a bug in workspace

* fix a bug in feature duplication

* fix a bug in feedback

* fix a bug in preprocessed data

* fix a bug om feature engineering

* fix a ci error

* Debugged & Connected

* Fixed error on feedback & added other fixes

* fix CI errors

* fix a CI bug

* fix: fix_dotenv_error (#257)

* fix_dotenv_error

* format with isort

* Update rdagent/app/cli.py

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>

* chore(main): release 0.2.1 (#249)

Release-As: 0.2.1

* init a scenario for kaggle feature engineering

* delete error codes

* Delete rdagent/app/kaggle_feature/conf.py

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com>
Co-authored-by: Xisen-Wang <xisen_application@163.com>
Co-authored-by: Haotian Chen <113661982+Hytn@users.noreply.github.com>
Co-authored-by: WinstonLiye <1957922024@qq.com>
Co-authored-by: WinstonLiyt <104308117+WinstonLiyt@users.noreply.github.com>
Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com>
2024-09-11 15:26:52 +08:00
Linlang 92c46a66b6 fix: fix_dotenv_error (#257)
* fix_dotenv_error

* format with isort

* Update rdagent/app/cli.py

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2024-09-10 19:38:35 +08:00
Linlang a8f1fc939c test: add test import (#242)
* add test import

* format with isort

* format with black

* merge main

* fix pytest error

* fix pytest error

* fix pytest error

* fix pytest error

* format with black

* fix pytest error

* fix pytest error

* fix pytest error

* fix pytest error

* fix pytest error

* format with isort

* Exclude entrance

* Add offline test

* auto-lint

* update coverage rate

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-09-06 17:18:52 +08:00
you-n-g 44d06c2f2c fix: package dependency. (#234)
* fix package

* fix lint
2024-08-31 20:35:04 +08:00
TPLin22 89d4f92051 feat: support more models for kaggle scenario (#223)
* init commit for XGBoost

* fix some bugs

* CI issues

* CI issues

* CI issue

* edit prompts for kaggle scenario & fix some bugs

* Revised Prompts To Improve Performance on Model Type & Support of Random Forest

* edit prompts & modify evaluator.py to adapt to Kaggle scenario

* edit prompts

* fix some bugs

* CI issues

---------

Co-authored-by: Taozhi Wang <taozhi.mark.wang@gmail.com>
Co-authored-by: Xisen-Wang <xisen_application@163.com>
2024-08-23 15:13:50 +08:00
Tim e9eb22742a check if gen_df is None (#216) 2024-08-20 12:27:13 +08:00
WinstonLiyt a75c19b17c docs: Updated documentation for all the major scenarios. (#190)
* Fixed some bugs introduced during refactoring.

* update data_agent_fin_doc

* Updated documentation for the four major scenarios

* feat: remove pdfs and enable online pdf readings (#183)

* remove pdfs and enable online pdf readings

* update doc format

* use url as key

* feat: add entry for rdagent. (#187)

* Add entries

* update entry for rdagent

* lint

* fix typo

* docs: Demo links (#188)

add demo links

* fix: Fix a fail href in readme (#189)

* fix a ci bug

* doc

* feat: remove pdfs and enable online pdf readings (#183)

* remove pdfs and enable online pdf readings

* update doc format

* use url as key

* feat: add entry for rdagent. (#187)

* Add entries

* update entry for rdagent

* lint

* fix typo

* doc

* Updated documentation for med_model scenarios.

* fix a ci bug

---------

Co-authored-by: Xu Yang <peteryang@vip.qq.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
Co-authored-by: XianBW <36835909+XianBW@users.noreply.github.com>
Co-authored-by: SH-Src <suhan.c@outlook.com>
2024-08-09 13:27:33 +08:00
Xu Yang 05a3b67aa8 fix: fix quick start problem (#191)
* all update code

* fix a small bug
2024-08-09 12:58:34 +08:00
Xu Yang 59c1ace8a2 feat: remove pdfs and enable online pdf readings (#183)
* remove pdfs and enable online pdf readings

* update doc format

* use url as key
2024-08-08 15:20:39 +08:00
WinstonLiyt 1280e681bb fix: Add framework handling for task coding failure. (#176)
* Add framework handling for task coding failure.

* fix a ci bug
2024-08-07 19:01:46 +08:00
Tim 4037dc821a doc: add benchmark.rst (#153)
* add benchmark png

* Create benchmark.rst

* Update benchmark.rst

* Draft

* rename file

* add autopydantic_settings

* remove ignored file

* add example

* sort imports

* sort imports

* format code with black

---------

Co-authored-by: Young <afe.young@gmail.com>
2024-08-06 09:29:22 +08:00
Xu Yang 2f5db22afc build factor source data (price and volumns) from qlib if no source data is provided by the user (#168) 2024-08-05 21:01:59 +08:00
Xu Yang 4618d59ec3 fix: first round app folder cleaning (#166)
* first round app folder cleaning

* fix CI
2024-08-05 18:11:23 +08:00