8 Commits

Author	SHA1	Message	Date
you-n-g	1acb1dab8f	feat: run benchmark on gpt-4o & llama 3.1 (#497) ... * Run benchmark on gpt-4o & llama 3.1 * update link	2024-11-26 12:02:09 +08:00
Xu Yang	c095e4992f	feat(kaggle): several update in kaggle scenarios (#476) ... * udpate plot * log and reduce token * trace tag * add simple_background parameter to get_scenario_all_desc * update trace * update first version code * chat model map * add annotation for stack index * add annotation * reformatted by black * several update on kaggle scenarios * update some new change * fix CI * fix CI * fix a bug * fix bugs in graph RAG --------- Co-authored-by: Tim <illking@foxmail.com>	2024-11-06 13:14:35 +08:00
Tim	b9a36e1b25	chore: more optional parameters for running benchmark analysis (#431) ... * set title and round * decision from multiple types * check if decision is true * reformat * remove unused file	2024-10-15 12:41:32 +08:00
you-n-g	47940f6589	fix: partial bug in bench (#368) ... * Add more log * Fix eval * update * keep ratio <= 1 * feat: cache exception (#369) * 0 instead of NaN * remove unused evaluators * save gen_factor_l_all_rounds * black reformat * cache exception --------- Co-authored-by: Tim <illking@foxmail.com>	2024-09-29 09:59:54 +08:00
Tim	d848a84156	feat: Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio (#328) ... * remove AttributeError caused by select_threshold * develop with ground truth * raise exception for eval_case * Revert "develop with ground truth" This reverts commit e68c136588685476f32a3c3696a6a33deb47acc5. * Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio * reformatted by black * Set the threshold for FactorIndexEvaluator to 0.99 * Apply suggestions from code review * Update rdagent/components/coder/factor_coder/CoSTEER/evaluators.py * geometric mean for format_succ_rate * no need to check when similarity is high enough * black reformat --------- Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>	2024-09-27 09:34:55 +08:00
you-n-g	444ded9a5d	fix: refactor Bench (#302) ... * refactor for better bench * autolint * add cmd * lint	2024-09-23 19:53:05 +08:00
Linlang	9dd35307a1	test: add test import (#242) ... * add test import * format with isort * format with black * merge main * fix pytest error * fix pytest error * fix pytest error * fix pytest error * format with black * fix pytest error * fix pytest error * fix pytest error * fix pytest error * fix pytest error * format with isort * Exclude entrance * Add offline test * auto-lint * update coverage rate --------- Co-authored-by: Young <afe.young@gmail.com>	2024-09-06 17:18:52 +08:00
Xu Yang	987cfc723e	fix: first round app folder cleaning (#166) ... * first round app folder cleaning * fix CI	2024-08-05 18:11:23 +08:00