feat: Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio (#328)

* remove AttributeError caused by select_threshold * develop with ground truth * raise exception for eval_case * Revert "develop with ground truth" This reverts commit e68c136588685476f32a3c3696a6a33deb47acc5. * Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio * reformatted by black * Set the threshold for FactorIndexEvaluator to 0.99 * Apply suggestions from code review * Update rdagent/components/coder/factor_coder/CoSTEER/evaluators.py * geometric mean for format_succ_rate * no need to check when similarity is high enough * black reformat --------- Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2026-08-06 03:27:44 +00:00 · 2024-09-27 09:34:55 +08:00
parent 0e94e74d20
commit b93f034b19
3 changed files with 34 additions and 27 deletions
@@ -125,10 +125,10 @@ class BaseEval:
        eval_res = []
        for ev in self.evaluator_l:
            try:
+                case_gen.raise_exception = True
                eval_res.append((ev, ev.evaluate(implementation=case_gen, gt_implementation=case_gt)))
                # if the corr ev is successfully evaluated and achieve the best performance, then break
-            except (CoderError, AttributeError) as e:
-                # TODO: remove AttributeError.
+            except CoderError as e:
                return e
            except Exception as e:
                # exception when evaluation