Introduce a configurable sort priority for Set A and Set B in the
`outliers` subcommand, replacing the single-criterion "Result desc".
New CLI flag:
--sort ABBR_LIST comma-separated metric abbreviations
(default: R,EP,PF,RF,SR,P,DD,C,T)
Abbreviations: R=Result, P=Profit, EP=Expected Payoff, PF=Profit Factor,
RF=Recovery Factor, SR=Sharpe Ratio, C=Custom, DD=Equity DD %, T=Trades.
Equity DD % sorts ascending (lower is better); all others descending.
Unmentioned abbreviations are appended at default order.
Two related improvements to the optimization-report parser:
1. Generic column typing — drop the hardcoded 'InpUseNewsFilter'
branch in parse_passes. Now the SpreadsheetML first body row's
<Data ss:Type='String'> marks a column as string (boolean Inp*
rendered as 'true'/'false' stays str); everything else is numeric
(Int64 when whole-numbered, float64 otherwise). No Inp* name is
referenced, so the parser handles any EA's parameter naming.
2. Generic param detection — _param_cols now returns every column
AFTER 'Trades' by position, not by Inp* prefix. Real exports don't
always use the Inp prefix; per user, 'Trades 以后的列数至少有一列,
但数量不定, 它们都是加入优化的输入参数'.
3. New 'outliers' subcommand — per-pass z-score scan over the 8
performance metrics (Result, Profit, Expected Payoff, Profit
Factor, Recovery Factor, Sharpe Ratio, Custom, Equity DD %).
Splits passes into two disjoint sets sorted by Result desc:
- Set A: at least one metric with |z|>=σ in the favourable
direction (higher-is-better metrics: z>=+σ; Equity DD %
uses z<=-σ because low DD is good).
- Set B: no performance-metric outlier.
Both sets EXCLUDE passes whose Trades count is itself a low-side
outlier (z<=-σ) — too few trades to trust. Excluded list shown
separately. Default σ=2, top 10 outliers / top 5 normal;
--sigma / --top-outliers / --top-normal / --json flags.
Output prints a per-metric mean/std/±σ reference table, then
each record split into Metrics group + Params group with the
outlier σ values annotated. Style mirrors the windows subcommand
in parse_tester_report.py.
Verified on jobs/246753 (432 passes, OneShotGold XAUUSD H4): 44 Set A
passes, 382 Set B, 6 excluded low-Trades passes, all 13 ad-hoc
verification assertions pass.
New script skills/mql5/scripts/parse_optimizer_report.py reads the
SpreadsheetML export from MT5 Strategy Tester optimization (one row per
parameter pass, plus a <DocumentProperties> environment card).
Companion to parse_tester_report.py. --analyze adds:
- Strategy environment card (EA / Symbol / Period / date range from
Title; deposit / leverage / server / MT5 build from DocumentProperties)
- Orthogonality check (actual passes vs expected cartesian product)
- Parameter effect ranking (effect_ratio = spread / global std) and
dead-parameter detection (per-group mean range < 1% of max)
- Dead boolean parameter detection (bit-for-bit identical true/false
groups across all key metrics)
- Duplicate metric vector counting (>30% usually means a dead param)
- Top-5 best passes by Profit / Profit Factor / Recovery Factor / Custom
- Trade count distribution with daily rate and correlations vs profit
and drawdown (overtrading / undertrading detection)
SKILL.md adds a new 'Optimization Report Analysis' subsection under
the Backtesting chapter covering environment card, orthogonality,
dead parameter / boolean parameter / duplicate detection, multi-
criteria best-pass selection, parameter effect ranking, trade count
diagnostics, and a reporting template.
Validated against jobs/246753/ReportOptimizer-*.xml (OneShotGold,
XAUUSD H4, 432 passes): correctly identifies InpUseNewsFilter as a
dead boolean parameter (true/false identical on Profit/PF/RF/Trades,
189 of 432 metric-vector duplicate groups), reports the trade/profit
overtrading signal (corr -0.56), and ranks best passes.
Adds pandas dependency (used for groupby/aggregate/corr on the pass
table; the script falls back to dict-list output if needed).
ZhijuCen