Problem:
- When factor evaluation timed out (5 min), result was undefined
- save_single_result(result) crashed with NameError
- factor.factor_name[:40] could fail if factor_name wasn't a string
Fix:
- Initialize result = None before try block
- Set result to failed EvalResult on exception
- Only call save_single_result() if result is not None
- Use getattr(factor, 'factor_name', 'unknown') for safe access
- Convert to string before slicing [:40]
Now the evaluator continues even when individual factors timeout.
Problem:
- predix_full_eval.py re-evaluated ALL factors every time
- 382 factors were evaluated even though 164 already had results
- Wasted 8+ minutes of computation time
Solution:
- Add scan_factors(skip_evaluated=True) parameter
- Load existing results from results/factors/*.json
- Skip factors that already have status='success' and valid IC
- Add --force/-f flag to override and re-evaluate ALL factors
Usage:
python predix_full_eval.py --top 100 # Only NEW factors
python predix_full_eval.py --force # Re-evaluate ALL
python predix_full_eval.py --all # All NEW factors
Now the evaluator resumes from where it left off.
- Add factor_code and factor_description to EvalResult
- Extract docstring or comments from factor code as description
- Enrich 232 existing factor files with code and description
Now each factor JSON in results/factors/ contains:
- factor_name, factor_code, factor_description
- IC, Rank IC, Sharpe, Win Rate, etc.
- Add save_single_result() function
- Call it after each factor evaluation (not just at end)
- Results now appear in results/factors/ in real-time
Usage:
python predix_full_eval.py --all --parallel 4
- Changed save location from results/backtests/ to results/factors/
- ALL successful factor results are now saved individually
- Safe filename handling (special chars removed)
Usage:
python predix_full_eval.py --all --parallel 4
TPTBusiness