c321ebfbcb
fix: fix some bugs in the ensemble component ( #595 )
...
* allow the LLM in the ensemble to focus on the significance of metrics
* fix a bug in feedback
* prun spec
* fix a bug in ensemble
* delete unnecessary prompts
* fix the logic in spec
* generalize dataset split
2025-02-14 15:32:40 +08:00
1a44a954df
feat: integrate azure deepseek r1 ( #591 )
...
* fix several task & integrate deepseek R1
* fix CI
---------
Co-authored-by: Xu Yang <xuyang1@microsoft.com >
2025-02-13 22:20:17 +08:00
4d8e334ef8
fix error ( #594 )
2025-02-13 19:34:49 +08:00
b7be0f35f8
fix: ensure expected type ( #593 )
...
* ensure expected type
* reformat
2025-02-13 19:04:26 +08:00
97c1f7a021
feat(backend): integrate LiteLLM API Backend ( #564 )
...
* File structure for supporting litellm
* more litellm support
* feat: Add CachedAPIBackend class and dynamic API backend retrieval function
* fix: update benchmark folder path and add default values for architecture and hyperparameters
* feat: add LiteLLMAPIBackend and DeprecBackend ; changed structure of the project ; with bus
* fix : deprec_backend
* feat: Add LiteLLMAPIBackend class and related features; update configuration and test cases.
* feat: Enhance LiteLLMAPIBackend with encoder support and dynamic argument handling;Enhance log Colors
* lint
* fix lint...
* fix: Lint
* fix:make auto-lint
* fix:test oai
* fix:redundant _abckend.py
* fix: Optimize LiteLLMAPIBackend on token counting functiona, and clean up unused code;add test on this function
* feat: Add LiteLLMSettings class and update model settings usage
* fix: Update LiteLLMSettings environment variable prefix and model configurations
* fix : gitignore
* test: Consolidate and relocate test files for litellm backend and oai
* fix : lint
* fix: lint
* auto lint
* lint
* LINT
* lint
* chore: remove deprecated backend configuration comments
* refactor: Remove unused functions and imports from deprec.py and llm_utils.py
* refactor: Move md5_hash function from deprec.py to llm_utils.py
* chore: Remove extra newline and add missing import in deprec.py
* lint
* refactor: Move md5_hash function to utils module
* lint
* lint
* lint
---------
Co-authored-by: Young <afe.young@gmail.com >
Co-authored-by: Yihua Chen <v-yihuachen@microsoft.com >
2025-02-13 15:16:18 +08:00
f362a1618d
fix: fix a bug in threshold score display ( #592 )
...
* fix a bug in threshold score display
* fix
2025-02-13 14:50:56 +08:00
43703d45d2
feat: add baseline score stat ( #590 )
...
* add baseline score stat
* fix ci
* fix
* fix
2025-02-12 18:57:08 +08:00
f06412e628
sample group files by basename ( #589 )
2025-02-12 16:36:46 +08:00
dc2bf4b442
fix: sort file name for cache reproduction ( #588 )
2025-02-12 15:29:57 +08:00
8f1f19d4b9
fix: Handle ValueError when resolving relative path for uri ( #585 )
2025-02-12 13:23:08 +08:00
064733d82b
fix runner as costeer when using static task ( #587 )
2025-02-12 12:35:17 +08:00
da7169675c
ignore score value ( #586 )
2025-02-12 11:39:49 +08:00
6de33fa13b
fix: coder prompt & model test text ( #583 )
...
* test text fix
* Require LLM to use print() instead of logging
2025-02-11 21:12:25 +08:00
c1085921b5
add feedback to workspace and ds runner base on costeer
2025-02-11 20:50:19 +08:00
18de5eea43
feat: summary and UI update ( #581 )
...
* summary and UI update
* fix CI
* fix
* fix CI
2025-02-11 20:48:30 +08:00
13214ba5d0
avoid length exception ( #580 )
2025-02-11 18:12:45 +08:00
56f3a668f2
fix: handle invalid regex patterns in filter_progress_bar function ( #579 )
...
* fix: handle invalid regex patterns in filter_progress_bar function
* Update rdagent/utils/__init__.py
2025-02-11 18:07:16 +08:00
75f29f0dd9
refine the code in hypo&task generating ( #578 )
2025-02-10 22:09:42 +08:00
345a702501
fix: fix the errors in the coder and evaluator of the five components ( #576 )
...
* refine the prompt in feat coder
* refine data loader prompt in coder & eval
* fix
* fix
* refine model prompt in coder & eval
* refine ensemble prompt in coder & eval
* refine workflow prompt in coder & eval
* fix
2025-02-10 20:20:30 +08:00
d2baa40fb1
add the none check for data loader & feature output ( #577 )
2025-02-10 20:18:01 +08:00
216361c640
score chart error catch ( #575 )
2025-02-10 12:19:29 +08:00
904f4b8523
refactor: Use file dictionaries for diff generation in feedback module ( #574 )
2025-02-10 09:03:13 +08:00
59337a9050
fix: add retry mechanism for GPU device check in DockerEnv ( #573 )
...
* fix: add retry mechanism for GPU device check in DockerEnv
* lint
* my py linting
* refactor: Improve type annotations and retry logic in workflow utils
* fix syntax
2025-02-10 08:59:46 +08:00
0db49991cb
feat: add retry mechanism with wait_retry decorator and refactor diff generation ( #572 )
2025-02-10 01:35:54 +08:00
34bf87c410
refine the logic in feedback ( #571 )
2025-02-08 21:14:30 +08:00
f37ec32981
text change ( #568 )
2025-02-08 21:14:13 +08:00
cefef61d0e
feat: diff mode fix ( #569 )
...
* feat: Add last runnable experiment feedback retrieval and improve diff generation
* lint
2025-02-08 20:13:38 +08:00
3090718ca3
fix: include data information in cache key generation ( #566 )
2025-02-08 20:05:57 +08:00
7eb7f49f52
fix: add scores.csv checking in ensemble_test ( #567 )
...
* add scores.csv checking in ensemble_test
* text modify
2025-02-08 19:03:29 +08:00
c255b79da4
chore: format check result ( #565 )
...
* format check result placed next to the metric result
* reformat
2025-02-08 18:53:16 +08:00
94d0eca0ca
support multiple types in feature engineering ( #562 )
2025-02-08 15:21:11 +08:00
bfdee6b692
add time info and stdout window ( #563 )
2025-02-07 22:00:45 +08:00
732839e4bd
feat: add the shape of the CSV to the dataset description ( #561 )
...
* add csv's shape info to data description
* fix CI
2025-02-07 18:12:18 +08:00
98a0801a32
fix bugs ( #560 )
2025-02-07 11:00:18 +08:00
e1f86d7ca0
fix: add stdout context length setting and improve text shrinking logic ( #559 )
...
* feat: Add stdout context length setting and improve text shrinking logic
* lint
2025-02-06 23:00:15 +08:00
5c7ceec37f
fix: fix task return dict with wrong format ( #558 )
...
* refactor: Simplify exp_gen and enhance workflow with init_kwargs_update_func
* fix: Correct type hint for init_kwargs_udpate_func parameter
* lint
* lint
* lint
* lint
2025-02-06 19:24:14 +08:00
723a3f4225
fix: keep some txt files ( #557 )
...
* keep some txt files
* reformat
2025-02-06 18:01:15 +08:00
b2d56e39e5
fix: move mlebench check into runner ( #556 )
...
* abandon mlebench check in workflow
* move mlebench check to runner and put it into the exp
2025-02-06 15:37:41 +08:00
0e447a3a19
feat: multi log folder, replace "epxx" in workspace path ( #555 )
...
* multi log folder, replace "epxx" in workspace path
* valid trace path
* hypothesis show change
* fix CI
* add total stat in all summary page
2025-02-06 15:14:34 +08:00
e937f8710c
feat: exclude invalid session log folder ( #554 )
...
* refactor: Add is_valid_session check in save_all_grade_info
* refactor: Use is_valid_session to check log trace validity
2025-02-06 09:56:34 +08:00
b1fce69153
fix: change summary info of log folder ( #552 )
...
* update stat data in log summary
* show logic change
* add workspace None judgement
* fix CI
2025-02-05 16:31:30 +08:00
b092c24374
fix: trace summary df showing in dsapp ( #551 )
...
* fix trace summary df show
* fix ci
2025-02-05 10:48:25 +08:00
1678461f5a
fix: handle division by zero in percentage calculations ( #550 )
...
* fix: Handle division by zero in percentage calculations
* lint
2025-02-05 10:21:53 +08:00
35e3156018
fix: replace API call with build_cls_from_json_with_retry function ( #548 )
...
* refactor: Replace API call with build_cls_from_json_with_retry function
* fix lint error
* fix lint errors
* lint
* trigger
2025-01-27 20:19:11 +08:00
611fb28c58
fix: refine prompt to generate the most simple task in init stage ( #546 )
...
* refine prompt to generate the most simple task in init stage
* feature test dtype check improve
2025-01-27 00:51:40 +08:00
10abb20117
feat: add mlebench submission validitor ( #545 )
...
* add mlebench submission check
* fix CI
* fix bug
2025-01-27 00:41:35 +08:00
6e96cff435
fix: fix submission file search and add TODO in env.py ( #544 )
2025-01-26 17:08:59 +08:00
fcf17d3292
fix summary loop_num count ( #543 )
2025-01-26 11:58:12 +08:00
4235440ff4
feat: Dynamically find and use sample submission file in eval tests ( #542 )
2025-01-24 18:16:15 +08:00
111dddda18
chore: rename ens_and_decision ( #541 )
...
* rename ens_and_decision
* clarify workflow prompt
* rename
2025-01-24 17:24:31 +08:00
Yuante Li