bench: add raptorbt as a third engine, and a 10M-bar point (#6)

The harness compared two engines everywhere; it now compares N against a
reference. manifoldbt is the reference: every parity check and every ratio is
a challenger against it, never two challengers against each other.

raptorbt 0.9.0 joins on three of the four workloads. Its sma_cross comes back
bit-identical to the reference's final equity, and its rsi matches to the last
bit; its ema seeds on a different warmup and it has no fixed-quantity sizing,
so the fee workload records it as unsupported with the reason rather than
leaving a blank cell. On the bracket it diverges in its own documented way: it
never re-arms while the entry level holds, so it books exactly the reference's
round-trips minus the ones that re-enter on the exit bar.

Python moves to 3.12, which raptorbt pins rather than we do: it is built
against pyo3 0.20.3, whose maximum supported CPython is 3.12. Timings from runs
before this change are therefore not directly comparable.

The bar matrix gains 10M and the repetition default drops from 7 to 2. Measured,
those two almost cancel: the job stays around 16 minutes. macOS keeps its old
ceiling, since 10M bars adds 1.55 GB on vectorbt's side alone and that runner
has 7 GB.

.github/workflows/bench-vs-vectorbt.yml

@@ -4,9 +4,10 @@
 # published wheel from PyPI, exactly like any user would. That is what makes the
 # result independent rather than self-reported, and it is why anyone can fork
 # this repository and press "Run workflow" to reproduce the numbers on their own
-# runner.
+# runner. Same for the engines it is compared against: they come from PyPI at
+# pinned versions, and the run prints the versions it resolved.
 
-name: Benchmark vs vectorbt
+name: Benchmark vs vectorbt and raptorbt
 
 on:
   workflow_dispatch:
@@ -18,7 +19,14 @@ on:
       reps:
         description: "Interleaved repetitions per point"
         required: false
-        default: "7"
+        # 2, not 7. The budget went into a longer series instead: at 10M bars
+        # one vectorbt call costs 99 s, so seven of them would put the job past
+        # its timeout on Windows. Two repetitions still bracket the number (min,
+        # median and max are all published, and the median of two is their mean)
+        # but the noise flag gets cruder, since an interquartile range wants at
+        # least four samples to mean anything. A point that looks surprising is
+        # worth re-running at a higher `reps` before it is quoted anywhere.
+        default: "2"
   release:
     types: [published]
   schedule:
@@ -42,13 +50,15 @@ jobs:
       matrix:
         include:
           - os: ubuntu-latest
-            bars: "10000 100000 1000000 5000000"
+            bars: "10000 100000 1000000 5000000 10000000"
           - os: windows-latest
-            bars: "10000 100000 1000000 5000000"
+            bars: "10000 100000 1000000 5000000 10000000"
           # macOS runners ship 7 GB of RAM against 16 GB elsewhere, and vectorbt
           # materialises the simulation in memory (roughly 150 MB per million
           # bars, measured). The top size is trimmed so a point is never lost to
-          # swapping, which would time the disk instead of the engine.
+          # swapping, which would time the disk instead of the engine. The same
+          # arithmetic is why 10M bars is added on the other two and not here:
+          # measured, that point adds 1.55 GB on vectorbt's side alone.
           - os: macos-latest
             bars: "10000 100000 1000000"
 
@@ -60,7 +70,15 @@ jobs:
 
       - uses: actions/setup-python@v5
         with:
-          python-version: "3.13"
+          # 3.12, not 3.13, and it is raptorbt that pins it: it is built against
+          # pyo3 0.20.3, whose maximum supported CPython is 3.12. No release up
+          # to 0.9.0 publishes a cp313 wheel and a source build refuses outright
+          # ("the configured Python interpreter version (3.13) is newer than
+          # PyO3's maximum supported version (3.12)"). Comparing engines means
+          # running them in one environment, and the environment has to be one
+          # they all support. Runs before 2026-08-20 used 3.13 with two engines,
+          # so their absolute timings are not directly comparable with these.
+          python-version: "3.12"
 
       - name: Install engines from PyPI
         shell: bash
@@ -81,7 +99,7 @@ jobs:
 
       - name: Record the resolved environment
         shell: bash
-        run: pip freeze | grep -iE '^(manifoldbt|vectorbt|numpy|numba|pandas|psutil)=' || true
+        run: pip freeze | grep -iE '^(manifoldbt|vectorbt|raptorbt|numpy|numba|pandas|psutil)=' || true
 
       - name: Run the benchmark
         shell: bash

benchmarks/vs_vectorbt/README.md

@@ -1,11 +1,11 @@
-# manifoldbt vs vectorbt
+# manifoldbt vs vectorbt vs raptorbt
 
-An engine-to-engine benchmark you can re-run yourself. It installs both engines
-from PyPI, generates its own data, checks that the two engines produced the
-**same result**, and only then reports how long each took.
+An engine-to-engine benchmark you can re-run yourself. It installs every engine
+from PyPI, generates its own data, checks that they produced the **same
+result**, and only then reports how long each took.
 
 ```bash
-pip install manifoldbt vectorbt
+pip install manifoldbt
 pip install -r requirements-lock.txt
 python bench.py --bars 10000 100000 1000000 --reps 7 --cold-start-reps 3 --memory-bars 2000000 --out results.json
 python report.py results.json
@@ -15,22 +15,41 @@ No dataset to download, no API key, no configuration. The same command runs in
 GitHub Actions on a public runner, so every published number has a run URL
 behind it.
 
+**Python 3.12, and not by preference.** raptorbt is built against pyo3 0.20.3,
+whose maximum supported CPython is 3.12: no release up to 0.9.0 publishes a
+cp313 wheel, and a source build refuses outright. Comparing engines means
+running them in one environment, and the environment has to be one all of them
+support. Add or drop a challenger with `--engines`; an engine that is not
+installed is skipped with a printed line rather than crashing the run.
+
+**manifoldbt is the reference.** Every parity check and every ratio is a
+challenger against it, never two challengers against each other: three engines
+make three pairs, and a table of pairs is a matrix, not a benchmark. The
+reference gets no advantage from the position, it is simply the one every
+timing is divided by.
+
 ## The rule this harness is built around
 
-A speed comparison between two backtesters is worthless unless both engines did
-the same work. So parity comes first:
+A speed comparison between backtesters is worthless unless they did the same
+work. So parity comes first:
 
 1. each workload runs once per engine;
-2. total return, round-trip count and fees are compared;
-3. **a workload the engines disagree on gets no published timing.**
+2. total return, round-trip count and fees are compared against the reference;
+3. **a workload an engine disagrees on gets no published timing for it.**
 
 Three verdicts come out of that gate:
 
 | Verdict | Meaning | What gets published |
 |---|---|---|
 | `exact` | agreement down to float-reordering noise (relative tolerance 1e-9) | the timing, in the headline table |
-| `documented` | the engines disagree, the workload declared it in advance, and the cause is written down | the timing, in an annex, with the cause and its measured size |
-| `failed` | the engines disagree and nobody predicted it | nothing. The run exits non-zero |
+| `documented` | that engine disagrees, the workload declared it in advance *for that engine*, and the cause is written down | the timing, in an annex, with the cause and its measured size |
+| `failed` | it disagrees and nobody predicted it | nothing. The run exits non-zero |
+
+A fourth outcome sits beside the gate rather than inside it. `unsupported`
+means an engine cannot express the workload at all: it is not run, and the
+report prints the reason instead of an empty cell. A blank in a speed table
+reads as a defeat, and "this engine has no fixed-quantity sizing" is not a
+defeat, it is a different fact.
 
 The `failed` path is not decoration. It is the reason the other numbers can be
 trusted, and it makes the benchmark fail loudly if a future release of either
@@ -64,12 +83,12 @@ store)`, the documented entry point, not through an internal fast path.
 
 ## What is compared
 
-| Workload | What it exercises | Parity |
-|---|---|---|
-| `sma_cross` | SMA 10/50 crossover, long-only, no cost | exact |
-| `ema_rsi_fees` | EMA 12/26 crossover with an RSI(14) filter and a 5 bps taker fee | exact |
-| `sma_cross_metrics` | the same simulation, plus max drawdown, Sharpe, Sortino and volatility | exact |
-| `bracket_sl_tp` | the same entry with a 15 bps stop and a 30 bps target | documented divergence |
+| Workload | What it exercises | vectorbt | raptorbt |
+|---|---|---|---|
+| `sma_cross` | SMA 10/50 crossover, long-only, no cost | exact | exact |
+| `ema_rsi_fees` | EMA 12/26 crossover with an RSI(14) filter and a 5 bps taker fee | exact | unsupported |
+| `sma_cross_metrics` | the same simulation, plus max drawdown, Sharpe, Sortino and volatility | exact | exact |
+| `bracket_sl_tp` | the same entry with a 15 bps stop and a 30 bps target | documented | documented |
 
 Each of those runs across a range of series lengths. Two further axes, cold
 start and memory, are measured in their own processes because they cannot be
@@ -77,15 +96,25 @@ measured honestly inside the main one.
 
 **Scope, stated twice on purpose.** `sma_cross` and `sma_cross_metrics` run the
 identical simulation; only the second one also produces a performance summary.
-manifoldbt computes that summary inside `run()` whether or not you read it,
-while vectorbt defers the equity curve until a risk metric asks for it and then
-pays to materialise it. Reporting both scopes is the only honest way to present
-the result: a reader who only wants a total return should look at the first
-number, and a reader who wants a Sharpe should look at the second. Parity on the
-summary workload is gated on total return, round-trip count and max drawdown,
-which match exactly; the ratios agree to about 3e-4, because manifoldbt buckets
-its daily returns slightly differently, and that residual is reported rather
-than smoothed over.
+manifoldbt and raptorbt both compute that summary inside the run whether or not
+you read it, while vectorbt defers the equity curve until a risk metric asks for
+it and then pays to materialise it. Reporting both scopes is the only honest way
+to present the result: a reader who only wants a total return should look at the
+first number, and a reader who wants a Sharpe should look at the second. Parity
+on the summary workload is gated on total return, round-trip count and max
+drawdown, which match exactly across all three; the vectorbt ratios agree to
+about 3e-4, because manifoldbt buckets its daily returns slightly differently,
+and that residual is reported rather than smoothed over.
+
+raptorbt annualises its ratios on its own basis and moves that basis between
+releases (the same run reads Sharpe 0.21 on 0.4.1 and 3.43 on 0.9.0, against
+manifoldbt's 8.14), so its Sharpe and Sortino are recorded but not compared:
+subtracting them from the reference would publish a units mismatch as a
+disagreement. It reports no volatility at all, and the harness leaves that cell
+empty rather than recomputing it from the equity curve, which would credit
+raptorbt with the harness's own arithmetic. None of this touches the gate, which
+runs on money, round-trips and drawdown, all three basis-free. Its drawdown
+matches the reference to 4e-15.
 
 The vectorbt side of the summary is written out in pandas rather than through
 `pf.sharpe_ratio()` for two reasons, both in vectorbt's favour or neutral.
@@ -110,6 +139,26 @@ data: building each engine's data representation is a different question and is
 excluded on both sides, in manifoldbt's case a deliberately unflattering choice
 since its one-off ingest is the memory-hungry part.
 
+### Why raptorbt sits out the fee workload
+
+Two blockers, either one sufficient, both measured rather than assumed.
+
+*Sizing.* raptorbt has no fixed-quantity mode. `position_sizes` is a fraction of
+equity (a constant 0.5 buys exactly half the equity of the bar before the entry),
+`lot_size` rounds a computed size down to a multiple of itself, and
+`alloted_capital` fixes the notional rather than the quantity. Reproducing
+`units=5` would mean feeding a fraction derived from an equity curve that does
+not exist until the run is over.
+
+*Indicator.* `raptorbt.ema` seeds on a simple mean of the first `period` bars and
+emits from bar `period-1`; manifoldbt seeds on the first observation and emits
+from bar 0. Same recursion, different warmup, so the signal differs early and the
+round-trip count with it. Its `sma` matches the reference to 3.3e-13 and its
+`rsi` matches to the last bit, which is why the other three workloads run.
+
+Running it anyway with a different size and a different indicator would produce
+a number, and the number would not mean anything.
+
 ### Why the fee workload sizes in units
 
 With `FractionOfEquity` sizing and a non-zero fee the engines size differently:
@@ -121,21 +170,30 @@ isolates the fee arithmetic, which is the thing both engines must agree on.
 ### The documented divergence, in full
 
 When a bracket fires intrabar and the entry condition still holds at that bar's
-close, manifoldbt books two orders on that bar: the stop or target exit, then a
-fresh entry at the close. vectorbt processes one order per bar and re-enters on
-the next bar instead. Neither is wrong. On controlled bars the bracket fills
-themselves match exactly, which the cross-engine parity suite shipped with the
-library pins test by test; the divergence is purely about *when* a re-entry is
-allowed.
+close, the three engines take three different roads:
 
-The harness counts the affected round-trips rather than hand-waving at them, so
-the report states what share of the trades the difference touches.
+- **manifoldbt** books two orders on that bar: the stop or target exit, then a
+  fresh entry at the close.
+- **vectorbt** processes one order per bar and re-enters on the next bar.
+- **raptorbt** does not re-arm at all. The level still being true is not enough;
+  it waits for the level to go false and true again.
+
+None of them is wrong. On controlled bars the bracket fills themselves match
+exactly, which the cross-engine parity suite shipped with the library pins test
+by test; the divergence is purely about *when* a re-entry is allowed.
+
+The harness counts the affected round-trips rather than hand-waving at them, and
+it is one population, not three: at 10,000 bars manifoldbt books 214 round-trips
+of which 82 re-enter on the exit bar, and raptorbt books exactly 214 - 82 = 132.
+The report states the count and the share, so the size of the difference is
+measured instead of asserted.
 
 ## Alignment choices, and why each one exists
 
-Two engines only produce identical numbers if they are told to do the same
-thing. These are the conventions the harness sets, all of them visible in
-[engine_mbt.py](engine_mbt.py) and [engine_vbt.py](engine_vbt.py):
+Engines only produce identical numbers if they are told to do the same thing.
+These are the conventions the harness sets, all of them visible in
+[engine_mbt.py](engine_mbt.py), [engine_vbt.py](engine_vbt.py) and
+[engine_rbt.py](engine_rbt.py):
 
 - `signal_delay=0` with `execution_price="AtClose"`, matching what
   `Portfolio.from_signals` does by default: a signal fills at the close of the
@@ -157,6 +215,21 @@ thing. These are the conventions the harness sets, all of them visible in
   different fill prices while both being correct, so that class of false
   failures is removed from the data rather than argued about in the report.
 
+On the raptorbt side specifically:
+
+- `upon_bar_close=True`, which fills at the close of the signal bar. Turning it
+  off does not add a one-bar delay, it moves the fill to that same bar's *open*,
+  so there is no setting on that side matching a delayed execution.
+- Sizing is left at its default, which takes the whole equity of the bar before
+  the entry. Since the account is flat at that point, that is the same number as
+  the equity at the fill, and the three engines size identically: `sma_cross`
+  comes back with manifoldbt's final equity to the last bit.
+- The bracket is set on the config, in fractions rather than percent: 0.15 means
+  a 15% stop, so the workload's 0.15% is `0.0015`. Passing the percent number is
+  not an error, it is a stop so wide it never triggers.
+- Indicators come from raptorbt's own Rust `sma` and `rsi`, not from numpy: that
+  is what a user would write, and it is what deserves to be timed.
+
 ## Reading the numbers honestly
 
 - On a shared runner with 4 vCPUs, an engine that parallelises is understated.
@@ -164,14 +237,25 @@ thing. These are the conventions the harness sets, all of them visible in
 - The published wheels are CPU-only, and GitHub-hosted runners have no GPU, so
   nothing here says anything about GPU performance.
 - vectorbt is the open-source package (`pip install vectorbt`), not vectorbtpro.
+- raptorbt has no fan-out API for a parameter grid on one instrument, so its
+  sweep column is a Python loop over `run_single_backtest`. That is not a
+  handicap the harness imposed, it is the only spelling available, and the
+  moving averages are hoisted out of the loop so it gets the same courtesy
+  vectorbt gets on its own grid path.
 
 ## Files
 
 - `data.py` - deterministic OHLCV generator and its content digest
 - `probe_child.py` - the cold-start and memory probes, each in a fresh process
-- `workloads.py` - the parameters both engines read, and the declared parity status
-- `engine_mbt.py` / `engine_vbt.py` - one adapter per engine
+- `workloads.py` - the parameters every engine reads, and the per-engine notes
+- `engines.py` - the registry: who is in the comparison, and who is the reference
+- `engine_mbt.py` / `engine_vbt.py` / `engine_rbt.py` - one adapter per engine
 - `parity.py` - the gate
 - `bench.py` - the runner
+- `sweep_child.py` - one parameter-grid point, in its own process
 - `report.py` - JSON to Markdown, and to the GitHub job summary
-- [`.github/workflows/bench-vs-vectorbt.yml`](../../.github/workflows/bench-vs-vectorbt.yml) - the workflow that runs all of the above on a GitHub-hosted runner
+- `ci/bench-vs-vectorbt.yml` - the workflow, deployed to the public repository
+
+The directory is still named `vs_vectorbt` and the workflow file still
+`bench-vs-vectorbt.yml`: renaming either would break the path the public
+repository runs and start a fresh, empty run history.

benchmarks/vs_vectorbt/bench.py

@@ -3,21 +3,26 @@
 Design decisions that are the whole point of this harness
 ---------------------------------------------------------
 *Parity first.* Every workload runs once per engine before any measurement, and
-the results are compared. A workload the engines disagree on gets no published
-timing (see ``parity.py``).
+each challenger's result is compared against the reference. A workload an engine
+disagrees on gets no published timing for that engine (see ``parity.py``).
 
 *Interleaved repetitions.* The engines alternate within each repetition rather
-than running in two blocks. A cloud runner that slows down halfway through
-penalises both engines equally instead of whichever one happened to be second.
+than running in blocks. A cloud runner that slows down halfway through penalises
+all of them equally instead of whichever one happened to be last.
 
 *Ratios are the headline, milliseconds are context.* The per-repetition ratio is
-computed from two measurements taken seconds apart on the same machine, so it
+computed from measurements taken seconds apart on the same machine, so it
 survives the noise that absolute timings on shared hardware do not.
 
 *Dispersion is published.* Every point carries min, median, max and IQR. A point
 whose IQR exceeds 15% of its median is flagged noisy, and a flagged point is not
 headline material no matter how good it looks.
 
+*An engine that cannot run a workload says so.* It is dropped from that workload
+with its reason recorded, never left as a blank cell: a missing number in a
+speed table reads as a loss, and "this engine has no fixed-quantity sizing" is
+not a loss, it is a different fact that deserves its own sentence.
+
 Usage
 -----
     python bench.py --bars 10000 100000 1000000 --reps 7 --out results.json
@@ -39,12 +44,21 @@ from typing import Any, Callable, Dict, List
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 
 import data as data_mod  # noqa: E402
-import engine_mbt  # noqa: E402
-import engine_vbt  # noqa: E402
+import engines as engines_mod  # noqa: E402
 import parity as parity_mod  # noqa: E402
-from workloads import DEFAULT_KEYS, SCOPE_PAIR, WORKLOADS  # noqa: E402
+from engines import CHALLENGERS, ENGINES, REFERENCE  # noqa: E402
+from workloads import (  # noqa: E402
+    DEFAULT_KEYS,
+    SCOPE_PAIR,
+    WORKLOADS,
+    supported,
+    unsupported_by,
+)
 
-SCHEMA_VERSION = 1
+# 2: timings, parity and speedups became per-engine maps when the harness grew
+# past two engines. `report.py` reads version 1 files as well, so the results
+# archived under results/ stay readable.
+SCHEMA_VERSION = 2
 NOISE_THRESHOLD = 0.15
 
 
@@ -88,10 +102,18 @@ def _ram_gb():
 
 
 def _versions() -> Dict[str, str]:
+    """Distribution versions, engines first then the stack underneath them.
+
+    Read from the installed metadata rather than from each package's own
+    ``__version__``, which is a hand-maintained constant and can lag: raptorbt
+    0.4.1 still declares 0.4.0 in its ``__init__``.
+    """
     import importlib.metadata as md
 
+    names = [ENGINES[n].distribution for n in ENGINES]
+    names += ["numpy", "numba", "pandas"]
     out = {}
-    for name in ("manifoldbt", "vectorbt", "numpy", "numba", "pandas"):
+    for name in names:
         try:
             out[name] = md.version(name)
         except Exception:
@@ -201,129 +223,161 @@ def _summarise(samples: List[float]) -> Dict[str, Any]:
     }
 
 
-def measure_pair(keys: List[str], bars: int, reps: int, workdir: str) -> List[Dict[str, Any]]:
-    """Measure two workloads inside ONE interleaved loop.
+def _roll_up(verdicts: Dict[str, Dict[str, Any]]) -> str:
+    """One status for the whole entry: the worst any engine came back with."""
+    statuses = {v["status"] for v in verdicts.values()}
+    for worst in ("failed", "documented"):
+        if worst in statuses:
+            return worst
+    return "exact"
 
-    The report puts these two side by side to show what a performance summary
-    costs each engine. That subtraction is only legitimate if all four timings
-    come from the same stretch of machine time: measured in separate blocks, the
-    drift in absolute timings is larger than the difference being reported, and
-    the table ends up claiming the version doing more work is the faster one.
-    """
-    frame = data_mod.make_ohlcv(bars)
-    runners = {}
-    entries = {}
-    for key in keys:
-        run_mbt = engine_mbt.prepare(key, frame, workdir)
-        run_vbt = engine_vbt.prepare(key, frame, workdir)
-        _, warm_mbt = _time_once(run_mbt)
-        _, warm_vbt = _time_once(run_vbt)
-        verdict = parity_mod.compare(warm_mbt, warm_vbt, key)
-        runners[key] = (run_mbt, run_vbt)
-        entries[key] = {
-            "workload": key,
-            "title": WORKLOADS[key].title,
-            "bars": bars,
-            "data_digest": data_mod.digest(frame),
-            "parity": verdict,
-            "paired_with": [k for k in keys if k != key],
-        }
 
-    samples: Dict[str, Dict[str, List[float]]] = {
-        key: {"manifoldbt": [], "vectorbt": [], "ratio": []} for key in keys
+def _prepare_all(key: str, frame, workdir: str, active: List[str]):
+    """Build one timed closure per engine that can run this workload."""
+    return {
+        name: engines_mod.adapter(name).prepare(key, frame, workdir)
+        for name in active
+        if supported(key, name)
     }
-    threading_use = {key: {"manifoldbt": _Parallelism(), "vectorbt": _Parallelism()}
-                     for key in keys}
-    for _ in range(reps):
-        for key in keys:
-            run_mbt, run_vbt = runners[key]
-            t_mbt, _ = threading_use[key]["manifoldbt"].record(run_mbt)
-            t_vbt, _ = threading_use[key]["vectorbt"].record(run_vbt)
-            samples[key]["manifoldbt"].append(t_mbt)
-            samples[key]["vectorbt"].append(t_vbt)
-            samples[key]["ratio"].append(t_vbt / t_mbt if t_mbt > 0 else float("nan"))
-
-    out = []
-    for key in keys:
-        entry = entries[key]
-        if entry["parity"]["status"] == "failed":
-            entry["timings"] = None
-            entry["note"] = "timing withheld: unexplained disagreement between engines"
-            out.append(entry)
-            continue
-        mbt_stats = _summarise(samples[key]["manifoldbt"])
-        vbt_stats = _summarise(samples[key]["vectorbt"])
-        ratios = samples[key]["ratio"]
-        entry["timings"] = {"manifoldbt": mbt_stats, "vectorbt": vbt_stats}
-        entry["speedup"] = {
-            "median_of_ratios": statistics.median(ratios),
-            "min": min(ratios),
-            "max": max(ratios),
-        }
-        entry["noisy"] = (
-            mbt_stats["iqr_over_median"] > NOISE_THRESHOLD
-            or vbt_stats["iqr_over_median"] > NOISE_THRESHOLD
-        )
-        entry["cpu_over_wall"] = {
-            engine: threading_use[key][engine].ratio for engine in ("manifoldbt", "vectorbt")
-        }
-        out.append(entry)
-    return out
 
 
-def measure(key: str, bars: int, reps: int, workdir: str) -> Dict[str, Any]:
-    frame = data_mod.make_ohlcv(bars)
-    run_mbt = engine_mbt.prepare(key, frame, workdir)
-    run_vbt = engine_vbt.prepare(key, frame, workdir)
+def _warm_and_gate(key: str, runners: Dict[str, Callable]) -> Dict[str, Any]:
+    """One discarded call per engine, then the parity verdicts it feeds.
 
-    # Warmup, discarded: numba compiles on vectorbt's first call, and the engine
-    # warms its own caches. Both are one-off costs, reported separately rather
-    # than smeared across every repetition.
-    _, warm_mbt = _time_once(run_mbt)
-    _, warm_vbt = _time_once(run_vbt)
+    The warmup is not only there to be thrown away: it is the run the gate
+    reads. numba compiles on vectorbt's first call and the engines warm their
+    own caches, so the same call cannot be both the first measurement and a fair
+    one, but it is a perfectly good sample of what each engine *computed*.
+    """
+    warm = {name: _time_once(run)[1] for name, run in runners.items()}
+    reference = warm[REFERENCE]
+    return {
+        name: parity_mod.compare(reference, metrics, key, name)
+        for name, metrics in warm.items()
+        if name != REFERENCE
+    }
 
-    verdict = parity_mod.compare(warm_mbt, warm_vbt, key)
+
+def _entry(key: str, bars: int, frame, verdicts: Dict[str, Any], engines_run: List[str]):
     entry: Dict[str, Any] = {
         "workload": key,
         "title": WORKLOADS[key].title,
         "bars": bars,
         "data_digest": data_mod.digest(frame),
-        "parity": verdict,
+        "engines": engines_run,
+        "parity": verdicts,
+        "status": _roll_up(verdicts),
     }
-    if verdict["status"] == "documented":
-        entry["divergence_scale"] = engine_mbt.diagnose(key, frame, workdir)
+    skipped = unsupported_by(key)
+    if skipped:
+        entry["unsupported"] = skipped
+    return entry
 
-    if verdict["status"] == "failed":
+
+def _collect(entry: Dict[str, Any], runners: Dict[str, Callable],
+             samples: Dict[str, List[float]], threading_use: Dict[str, _Parallelism]) -> None:
+    """Turn raw per-engine samples into the published shape, in place."""
+    published = [
+        name for name in runners
+        if name == REFERENCE or entry["parity"][name]["status"] != "failed"
+    ]
+    # A solo run has nothing to compare and nothing to withhold: the gate exists
+    # to stop a *comparison* being published on mismatched work. When challengers
+    # were present and the gate dropped them all, though, the reference's timing
+    # is the leftover of a comparison, and printing it alone would read as one.
+    solo = len(runners) == 1
+    if REFERENCE not in published or (len(published) == 1 and not solo):
         entry["timings"] = None
-        entry["note"] = "timing withheld: unexplained disagreement between engines"
-        return entry
+        entry["note"] = "timing withheld: unexplained disagreement with the reference"
+        return
 
-    mbt_samples: List[float] = []
-    vbt_samples: List[float] = []
-    ratios: List[float] = []
-    threading_use = {"manifoldbt": _Parallelism(), "vectorbt": _Parallelism()}
+    stats = {name: _summarise(samples[name]) for name in published}
+    reference = samples[REFERENCE]
+    entry["timings"] = stats
+    entry["speedup"] = {}
+    for name in published:
+        if name == REFERENCE:
+            continue
+        ratios = [
+            other / ref if ref > 0 else float("nan")
+            for ref, other in zip(reference, samples[name])
+        ]
+        entry["speedup"][name] = {
+            "median_of_ratios": statistics.median(ratios),
+            "min": min(ratios),
+            "max": max(ratios),
+        }
+    entry["noisy"] = any(s["iqr_over_median"] > NOISE_THRESHOLD for s in stats.values())
+    entry["cpu_over_wall"] = {name: threading_use[name].ratio for name in published}
+
+    withheld = [name for name in runners if name not in published]
+    if withheld:
+        entry["withheld"] = withheld
+
+
+def measure_pair(keys: List[str], bars: int, reps: int, workdir: str,
+                 active: List[str]) -> List[Dict[str, Any]]:
+    """Measure two workloads inside ONE interleaved loop.
+
+    The report puts these two side by side to show what a performance summary
+    costs each engine. That subtraction is only legitimate if every timing comes
+    from the same stretch of machine time: measured in separate blocks, the drift
+    in absolute timings is larger than the difference being reported, and the
+    table ends up claiming the version doing more work is the faster one.
+    """
+    frame = data_mod.make_ohlcv(bars)
+    runners: Dict[str, Dict[str, Callable]] = {}
+    entries: Dict[str, Dict[str, Any]] = {}
+    for key in keys:
+        runners[key] = _prepare_all(key, frame, workdir, active)
+        verdicts = _warm_and_gate(key, runners[key])
+        entries[key] = _entry(key, bars, frame, verdicts, list(runners[key]))
+        entries[key]["paired_with"] = [k for k in keys if k != key]
+
+    samples = {key: {name: [] for name in runners[key]} for key in keys}
+    threading_use = {key: {name: _Parallelism() for name in runners[key]} for key in keys}
     for _ in range(reps):
-        t_mbt, _ = threading_use["manifoldbt"].record(run_mbt)
-        t_vbt, _ = threading_use["vectorbt"].record(run_vbt)
-        mbt_samples.append(t_mbt)
-        vbt_samples.append(t_vbt)
-        ratios.append(t_vbt / t_mbt if t_mbt > 0 else float("nan"))
+        for key in keys:
+            for name, run in runners[key].items():
+                elapsed, _ = threading_use[key][name].record(run)
+                samples[key][name].append(elapsed)
 
-    mbt_stats = _summarise(mbt_samples)
-    vbt_stats = _summarise(vbt_samples)
-    entry["timings"] = {"manifoldbt": mbt_stats, "vectorbt": vbt_stats}
-    entry["speedup"] = {
-        "median_of_ratios": statistics.median(ratios),
-        "min": min(ratios),
-        "max": max(ratios),
-    }
-    entry["noisy"] = (
-        mbt_stats["iqr_over_median"] > NOISE_THRESHOLD
-        or vbt_stats["iqr_over_median"] > NOISE_THRESHOLD
-    )
-    entry["cpu_over_wall"] = {
-        engine: threading_use[engine].ratio for engine in ("manifoldbt", "vectorbt")
-    }
+    for key in keys:
+        _collect(entries[key], runners[key], samples[key], threading_use[key])
+    return [entries[key] for key in keys]
+
+
+def measure(key: str, bars: int, reps: int, workdir: str, active: List[str]) -> Dict[str, Any]:
+    frame = data_mod.make_ohlcv(bars)
+    runners = _prepare_all(key, frame, workdir, active)
+    verdicts = _warm_and_gate(key, runners)
+    entry = _entry(key, bars, frame, verdicts, list(runners))
+
+    scales = {}
+    for name, verdict in verdicts.items():
+        if verdict["status"] != "documented":
+            continue
+        adapter = engines_mod.adapter(name)
+        measured = getattr(adapter, "diagnose", lambda *a, **k: {})(key, frame, workdir)
+        if measured:
+            scales[name] = measured
+    # The reference's own view of the divergence: which of its round-trips are
+    # the ones the others do not take. Recorded under the reference so the two
+    # sides of the subtraction sit next to each other.
+    if scales:
+        reference_view = engines_mod.adapter(REFERENCE).diagnose(key, frame, workdir)
+        if reference_view:
+            scales[REFERENCE] = reference_view
+        entry["divergence_scale"] = scales
+
+    samples = {name: [] for name in runners}
+    threading_use = {name: _Parallelism() for name in runners}
+    for _ in range(reps):
+        for name, run in runners.items():
+            elapsed, _ = threading_use[name].record(run)
+            samples[name].append(elapsed)
+
+    _collect(entry, runners, samples, threading_use)
     return entry
 
 
@@ -344,60 +398,185 @@ def _probe(mode: str, engine: str, workload: str, bars: int) -> Dict[str, Any]:
     raise RuntimeError("probe produced no result: " + out[-500:])
 
 
-def cold_start(workload: str, bars: int, reps: int) -> Dict[str, Any]:
-    """Time to a first backtest in a process that has never seen either engine.
+def cold_start(workload: str, bars: int, reps: int, active: List[str]) -> Dict[str, Any]:
+    """Time to a first backtest in a process that has never seen any engine.
 
     This is the cost the steady-state benchmark throws away as warmup, and the
     one a user actually waits through every time they open a notebook. The
     engines alternate here too, and the interpreter/numpy/pandas baseline is
-    measured alongside so the engine's own share is visible.
+    measured alongside so each engine's own share is visible.
     """
-    samples: Dict[str, List[float]] = {"manifoldbt": [], "vectorbt": [], "baseline": []}
+    order = [name for name in active if supported(workload, name)]
+    samples: Dict[str, List[float]] = {name: [] for name in order}
+    samples["baseline"] = []
     for _ in range(reps):
-        samples["manifoldbt"].append(_probe("coldstart", "mbt", workload, bars)["seconds"])
-        samples["vectorbt"].append(_probe("coldstart", "vbt", workload, bars)["seconds"])
+        for name in order:
+            samples[name].append(
+                _probe("coldstart", ENGINES[name].code, workload, bars)["seconds"])
         samples["baseline"].append(_probe("baseline", "none", workload, bars)["seconds"])
     medians = {k: statistics.median(v) for k, v in samples.items()}
     base = medians["baseline"]
+    reference = medians[REFERENCE]
     return {
         "workload": workload,
         "bars": bars,
+        "engines": order,
         "samples_s": samples,
         "median_s": medians,
-        "engine_share_s": {
-            "manifoldbt": medians["manifoldbt"] - base,
-            "vectorbt": medians["vectorbt"] - base,
+        "engine_share_s": {name: medians[name] - base for name in order},
+        "ratio": {
+            name: (medians[name] / reference if reference else None)
+            for name in order if name != REFERENCE
         },
-        "ratio": medians["vectorbt"] / medians["manifoldbt"] if medians["manifoldbt"] else None,
     }
 
 
-def memory(workload: str, bars: int) -> Dict[str, Any]:
+def memory(workload: str, bars: int, active: List[str]) -> Dict[str, Any]:
     """Resident memory each engine adds while running one backtest.
 
     vectorbt materialises the simulation as arrays, so its footprint grows with
     the series; manifoldbt streams bars out of its store. The number reported is
     what the *run* adds, measured after a warmup, not the process total: the
     one-off cost of building each engine's data representation is a different
-    question and is excluded on both sides.
+    question and is excluded on every side.
     """
-    mbt = _probe("memory", "mbt", workload, bars)
-    vbt = _probe("memory", "vbt", workload, bars)
-    return {
-        "workload": workload,
-        "bars": bars,
-        "manifoldbt": mbt,
-        "vectorbt": vbt,
-    }
+    order = [name for name in active if supported(workload, name)]
+    out: Dict[str, Any] = {"workload": workload, "bars": bars, "engines": order}
+    for name in order:
+        out[name] = _probe("memory", ENGINES[name].code, workload, bars)
+    return out
+
+
+SWEEP_CHILD = os.path.join(os.path.dirname(os.path.abspath(__file__)), "sweep_child.py")
+
+
+def parse_point(spec: str) -> tuple:
+    """`bars:combos[:oos]`, e.g. `20000:5000` or `20000:100000:oos`.
+
+    The `oos` suffix declares the array-materialising engines out of scope for
+    that point: it is for grids whose vectorbt side needs tens of gigabytes,
+    where running it anyway would time the swap file rather than the engine.
+    """
+    parts = spec.split(":")
+    if len(parts) not in (2, 3) or not parts[1]:
+        raise argparse.ArgumentTypeError(
+            f"expected bars:combos or bars:combos:oos, got {spec!r}")
+    mode = "run"
+    if len(parts) == 3:
+        if parts[2] != "oos":
+            raise argparse.ArgumentTypeError(
+                f"third field must be 'oos', got {parts[2]!r}")
+        mode = "oos"
+    return int(parts[0]), int(parts[1]), mode
+
+
+def sweep(points: List[tuple], reps: int, active: List[str]) -> List[Dict[str, Any]]:
+    """Parameter-grid comparison, one point per process.
+
+    Each point is spawned rather than run inline. A large grid is the one thing
+    in this harness that can exhaust the machine, and a point that dies must
+    cost its own result and nothing else: run inline, an out-of-memory kill
+    would take the whole benchmark down and lose every number measured before
+    it. A dead child is recorded as a crashed point and the run continues.
+    """
+    out: List[Dict[str, Any]] = []
+    challengers = [n for n in active if n != REFERENCE]
+    for bars, combos, mode in points:
+        proc = subprocess.run(
+            [sys.executable, SWEEP_CHILD, "--bars", str(bars),
+             "--combos", str(combos), "--reps", str(reps),
+             "--engines", *active, "--vectorbt", mode],
+            capture_output=True, text=True,
+        )
+        payload = None
+        for line in proc.stdout.splitlines():
+            if line.startswith("{"):
+                payload = json.loads(line)
+        if payload is None:
+            # No JSON at all: the child died before it could report. Almost
+            # always the allocator, on the biggest grid of the matrix.
+            payload = {
+                "bars": bars,
+                "combos": combos,
+                "vectorbt_mode": mode,
+                "status": "crashed",
+                "reason": (proc.stderr or proc.stdout or "no output")[-400:].strip(),
+                "exit_code": proc.returncode,
+            }
+        # Memory comes from dedicated single-engine processes, never from the
+        # interleaved timing run above. Timing has to interleave the engines to
+        # be fair, and interleaving is exactly what makes a memory reading
+        # worthless: from the second repetition on, the peak sampled during one
+        # engine's call is the whole process, the other engines' allocations
+        # included. Measured both ways, manifoldbt read 6.7 GB interleaved
+        # against 94 MB alone, at 5000 combinations.
+        payload["memory"] = _sweep_memory(bars, combos, mode, active)
+        out.append(payload)
+    return out
+
+
+def _sweep_memory(bars: int, combos: int, mode: str, active: List[str]) -> Dict[str, Any]:
+    """Peak each engine adds running the grid once, one engine per process."""
+    names = [REFERENCE] if mode == "oos" else list(active)
+    added: Dict[str, Any] = {name: None for name in active}
+    for name in names:
+        proc = subprocess.run(
+            [sys.executable, SWEEP_CHILD, "--bars", str(bars),
+             "--combos", str(combos), "--memory-only", ENGINES[name].code],
+            capture_output=True, text=True,
+        )
+        for line in proc.stdout.splitlines():
+            if line.startswith("{"):
+                added[name] = json.loads(line)["added_mb"]
+    return added
+
+
+def run_sweeps(points: List[tuple], reps: int, active: List[str]) -> tuple:
+    """Drive the sweep matrix and print one line per point. Returns (entries, failures).
+
+    Three outcomes count as failures, and the distinction matters:
+
+    * ``failed``  - the engines disagreed on a grid nobody predicted. The gate.
+    * ``crashed`` - the point could not be measured at all.
+    * ``skipped`` - the point asked for a timing the process was not licensed to
+      produce. Silent here would be the worst outcome of all: the matrix would
+      simply come back short, and a table missing its largest grid reads like a
+      choice rather than a failure.
+    """
+    entries = sweep(points, reps, active)
+    failures = 0
+    for e in entries:
+        head = "  sweep {b:>9,} bars x {c:>6,} combos ... ".format(
+            b=e["bars"], c=e["combos"])
+        status = e.get("status") or e.get("parity", {}).get("status", "?")
+        ratios = (e.get("timings") or {}).get("ratio") or {}
+        if ratios:
+            print(head + "{s:10s} ".format(s=status) + ", ".join(
+                "{n} x{r:.1f}".format(n=name, r=ratio) for name, ratio in ratios.items()))
+        elif e.get("timings"):
+            print(head + "{s:10s} {ref} {t:.2f} s, challengers out of scope".format(
+                s=status, ref=REFERENCE, t=e["timings"]["seconds"][REFERENCE]))
+        else:
+            print(head + "{s:10s} {why}".format(
+                s=status, why=e.get("reason") or e.get("note") or "no timing"))
+            failures += 1
+    return entries, failures
 
 
 # --------------------------------------------------------------------------- #
 # Entry point
 # --------------------------------------------------------------------------- #
 def main() -> int:
-    parser = argparse.ArgumentParser(description="manifoldbt vs vectorbt")
+    parser = argparse.ArgumentParser(description="manifoldbt against other engines")
     parser.add_argument("--bars", type=int, nargs="+", default=[10_000, 100_000, 1_000_000])
     parser.add_argument("--workloads", nargs="+", default=DEFAULT_KEYS, choices=DEFAULT_KEYS)
+    parser.add_argument("--engines", nargs="*", default=CHALLENGERS, choices=CHALLENGERS,
+                        help="challengers to run against " + REFERENCE + ". Pass it "
+                             "with no value for a solo run: no comparison, no "
+                             "ratios, just what the engine costs. That is a "
+                             "regression tracker rather than a benchmark, and it "
+                             "is the only shape whose wall time is the engine's "
+                             "own rather than the slowest challenger's.")
     parser.add_argument("--reps", type=int, default=7)
     parser.add_argument("--out", default="results.json")
     parser.add_argument("--workdir", default=None, help="where the engine store is built")
@@ -405,12 +584,33 @@ def main() -> int:
                         help="0 disables the cold-start probe")
     parser.add_argument("--memory-bars", type=int, default=0,
                         help="bars for the memory probe; 0 disables it")
+    parser.add_argument("--sweep", type=parse_point, nargs="*", default=[],
+                        metavar="BARS:COMBOS",
+                        help="parameter-grid points, e.g. 20000:5000. Needs a "
+                             "licence: an unlicensed fan-out call waits out a "
+                             "fixed interval, so the stopwatch would be timing "
+                             "the wait rather than the engine")
+    parser.add_argument("--sweep-reps", type=int, default=3,
+                        help="repetitions per sweep point; fewer than --reps "
+                             "because a large grid costs seconds, not milliseconds")
     parser.add_argument("--pin-cores", type=int, default=0,
                         help="restrict the process to N logical cores, so a big "
                              "workstation can reproduce what a small cloud runner "
                              "sees; 0 leaves the machine alone")
     args = parser.parse_args()
 
+    if not engines_mod.installed(REFERENCE):
+        print("cannot run: the reference engine ({}) is not installed".format(REFERENCE))
+        return 2
+    active = [REFERENCE] + engines_mod.present(args.engines)
+    for name in args.engines:
+        if name not in active:
+            # Not fatal: somebody benchmarking on their own machine should not
+            # have to install every competitor to read their own numbers. It is
+            # loud, though, because on a runner it means the lock file did not
+            # do its job.
+            print("skipping {}: not installed".format(name))
+
     pinned = None
     if args.pin_cores:
         try:
@@ -422,11 +622,13 @@ def main() -> int:
         except Exception as exc:            # macOS has no affinity API
             print("could not pin to {} cores: {}".format(args.pin_cores, exc))
 
-    workdir = args.workdir or tempfile.mkdtemp(prefix="mbt_vs_vbt_")
+    workdir = args.workdir or tempfile.mkdtemp(prefix="mbt_bench_")
     env = environment()
     env["pinned_cores"] = pinned
+    env["engines"] = active
     versions = env["versions"]
-    print("manifoldbt " + versions["manifoldbt"] + " vs vectorbt " + versions["vectorbt"])
+    print(" vs ".join(
+        "{} {}".format(name, versions[ENGINES[name].distribution]) for name in active))
     print("{cpu} | {cores} logical cores | {ram} GB | {os} {arch}".format(
         cpu=env["cpu"], cores=env["logical_cores"], ram=env["ram_gb"],
         os=env["os"], arch=env["arch"]))
@@ -435,16 +637,19 @@ def main() -> int:
     print(str(args.reps) + " interleaved repetitions per point\n")
 
     def announce(entry: Dict[str, Any]) -> bool:
-        status = entry["parity"]["status"]
         print("  {k:18s} {b:>9,} bars ... ".format(k=entry["workload"], b=entry["bars"]),
               end="", flush=True)
-        if entry.get("timings"):
-            print("{s:10s} manifoldbt x{v:.1f}{m}".format(
-                s=status, v=entry["speedup"]["median_of_ratios"],
-                m=" NOISY" if entry.get("noisy") else ""))
-            return False
-        print("{s:10s} timing withheld".format(s=status))
-        return True
+        if not entry.get("timings"):
+            print("{s:10s} timing withheld".format(s=entry["status"]))
+            return True
+        speeds = ", ".join(
+            "{n} x{v:.1f}".format(n=name, v=s["median_of_ratios"])
+            for name, s in entry["speedup"].items()
+        ) or "no challenger"
+        print("{s:10s} {speeds}{m}".format(
+            s=entry["status"], speeds=speeds,
+            m=" NOISY" if entry.get("noisy") else ""))
+        return entry["status"] == "failed"
 
     # The scope pair is measured together; everything else one workload at a time.
     paired = [k for k in SCOPE_PAIR if k in args.workloads]
@@ -454,38 +659,56 @@ def main() -> int:
     failures = 0
     for bars in args.bars:
         if len(paired) > 1:
-            for entry in measure_pair(paired, bars, args.reps, workdir):
+            for entry in measure_pair(paired, bars, args.reps, workdir, active):
                 results.append(entry)
                 failures += announce(entry)
     for key in singles + (paired if len(paired) == 1 else []):
         for bars in args.bars:
-            entry = measure(key, bars, args.reps, workdir)
+            entry = measure(key, bars, args.reps, workdir, active)
             results.append(entry)
             failures += announce(entry)
 
+    # The side probes run on the first workload every active engine supports, so
+    # a cold-start table cannot come back missing a column because the workload
+    # happened to be one somebody sits out.
+    probe_workload = next(
+        (k for k in args.workloads if all(supported(k, n) for n in active)),
+        args.workloads[0],
+    )
+
     cold = None
     if args.cold_start_reps:
         print("")
         print("  cold start ... ", end="", flush=True)
-        cold = cold_start(args.workloads[0], 20_000, args.cold_start_reps)
-        print("manifoldbt {:.2f} s vs vectorbt {:.2f} s (x{:.1f})".format(
-            cold["median_s"]["manifoldbt"], cold["median_s"]["vectorbt"], cold["ratio"]))
+        cold = cold_start(probe_workload, 20_000, args.cold_start_reps, active)
+        print(", ".join("{n} {t:.2f} s".format(n=name, t=cold["median_s"][name])
+                        for name in cold["engines"]))
 
     mem = None
     if args.memory_bars:
         print("  memory     ... ", end="", flush=True)
-        mem = memory(args.workloads[0], args.memory_bars)
-        print("manifoldbt +{:.0f} MB vs vectorbt +{:.0f} MB at {:,} bars".format(
-            mem["manifoldbt"]["added_mb"], mem["vectorbt"]["added_mb"], args.memory_bars))
+        mem = memory(probe_workload, args.memory_bars, active)
+        print(", ".join("{n} +{v:.0f} MB".format(n=name, v=mem[name]["added_mb"])
+                        for name in mem["engines"]))
+
+    sweeps = None
+    if args.sweep:
+        print("")
+        sweeps, sweep_failures = run_sweeps(args.sweep, args.sweep_reps, active)
+        failures += sweep_failures
 
     payload = {
         "schema_version": SCHEMA_VERSION,
         "generated_at": datetime.now(timezone.utc).isoformat(),
         "environment": env,
+        "reference": REFERENCE,
+        "engines": active,
         "reps": args.reps,
         "results": results,
         "cold_start": cold,
         "memory": mem,
+        "sweeps": sweeps,
+        "sweep_reps": args.sweep_reps if args.sweep else None,
     }
     with open(args.out, "w", encoding="utf-8") as fh:
         json.dump(payload, fh, indent=2)

benchmarks/vs_vectorbt/ci_activate.py

@@ -0,0 +1,57 @@
+"""Activate a licence from the environment, then assert the tier out loud.
+
+Used by the sweep workflow. A sweep benchmark run without a licence does not
+fail, it produces a wrong number: every unlicensed fan-out call waits out a
+fixed interval before doing any work, so the stopwatch measures the wait. On a
+100-cell grid that read 5.00 s against vectorbt's 0.17 s, which would publish
+"vectorbt is 29x faster" from a run where the engine barely ran.
+
+So this exits non-zero rather than let the benchmark continue unlicensed.
+
+The key is written to disk by `activate`, so every child process the harness
+spawns afterwards picks it up without seeing the secret itself.
+
+    MANIFOLDBT_CI_LICENSE=<signed key> python ci_activate.py
+"""
+from __future__ import annotations
+
+import os
+import sys
+
+import manifoldbt as mbt
+
+# The American spelling, because that is how the repository secret is actually
+# named. Both are accepted: the two spellings are one typo apart, and the cost
+# of getting it wrong is a benchmark job that dies at the first step.
+VARS = ("MANIFOLDBT_CI_LICENSE", "MANIFOLDBT_CI_LICENCE")
+
+
+def main() -> int:
+    key = ""
+    for var in VARS:
+        key = (os.environ.get(var) or "").strip()
+        if key:
+            break
+    if not key:
+        print(" and ".join(VARS) + " are empty or unset: "
+              "refusing to benchmark sweeps unlicensed.")
+        return 1
+
+    # Never print the key or the activation message: the message carries the
+    # licensee's address, and a public run log is not the place for it.
+    try:
+        mbt.activate(key)
+    except Exception as exc:                      # noqa: BLE001 - reported, not raised
+        print(f"activation failed: {type(exc).__name__}")
+        return 1
+
+    _used, _limit, is_pro = mbt._native._combo_budget()
+    if not is_pro:
+        print("activation did not yield a licensed tier: refusing to continue.")
+        return 1
+    print("licensed tier active")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/vs_vectorbt/engine_mbt.py

@@ -152,11 +152,12 @@ def diagnose(key: str, df, workdir: str) -> Dict[str, Any]:
     """Untimed measurement of *how much* a documented divergence actually bites.
 
     For the bracket workload this counts the round-trips whose entry lands on the
-    same bar as the previous exit, which is precisely the population where
-    vectorbt takes the next bar instead. Reporting the count turns "the engines
-    differ" into a number a reader can weigh.
+    same bar as the previous exit, which is precisely the population the other
+    engines handle differently: vectorbt takes the next bar, raptorbt does not
+    re-enter at all. Reporting the count turns "the engines differ" into a number
+    a reader can weigh, and it is the same population for both of them.
     """
-    if WORKLOADS[key].parity != "documented":
+    if not any(note.status == "documented" for note in WORKLOADS[key].notes.values()):
         return {}
 
     p = WORKLOADS[key].params

benchmarks/vs_vectorbt/engine_rbt.py

@@ -0,0 +1,192 @@
+"""raptorbt adapter.
+
+Public API only, and the same shape as the other two: everything that can be
+prepared once is prepared before timing, and the timed closure is what a
+raptorbt user writes - indicators, entry and exit arrays, one call to
+``run_single_backtest``, then reading the headline numbers off the result.
+
+Execution conventions, chosen to line up with the reference
+-----------------------------------------------------------
+* ``upon_bar_close=True`` fills at the close of the signal bar, which is
+  manifoldbt's ``signal_delay=0`` plus ``execution_price="AtClose"``. Measured:
+  with it off, the fill lands on that same bar's *open* rather than on the next
+  bar, so it is not a delay switch, and there is no setting on this side that
+  reproduces a one-bar delay.
+* Sizing is left at the default, which takes the whole equity of the bar before
+  the entry. Measured against ``position_sizes``: a constant 0.5 buys exactly
+  half that equity, so the default is the full-equity fraction manifoldbt calls
+  ``FractionOfEquity`` and vectorbt calls ``size_type="percent"``. The account is
+  flat at that point, so the previous bar's equity and the equity at the fill
+  are the same number and the three engines size identically. None of that is
+  taken on trust: ``sma_cross`` comes back bit-identical to manifoldbt's final
+  equity, which is what the gate actually checks.
+* ``fees`` is a fraction of notional charged on each side, so 5 bps taker on
+  both legs is ``fee_bps / 10_000`` (measured against a trade's own ``fees``
+  field, which equals ``size * (entry + exit) * fee``).
+
+Signals are levels, not transitions, exactly as on the vectorbt side: entries is
+"the condition holds", exits is "it no longer holds". raptorbt opens when flat
+and entries is true, which is the target-position semantics of the reference,
+with one documented exception after a bracket exit (see ``workloads.py``).
+
+Indicators come from raptorbt's own Rust implementations rather than from numpy,
+because that is both what a user would write and what deserves to be timed.
+``sma`` and ``rsi`` reproduce the reference to 3.3e-13 and to the last bit
+respectively. ``ema`` does not, which is why the workload that needs one is
+declared unsupported instead of being quietly run on different numbers.
+
+Brackets, and the unit trap in them
+-----------------------------------
+``set_fixed_stop`` and ``set_fixed_target`` exist on ``BacktestConfig`` and on
+``InstrumentConfig`` both. On 0.9.0 the two spellings agree to the last bit
+(verified: same final equity, same round-trips, whether the bracket is set on
+one, the other, or both), so the config-level one is used here as the simpler of
+the two.
+
+Their unit is a fraction, not a percent: 0.15 is a 15% stop, and the workload's
+0.15% is ``0.0015``. Passing the percent number is not an error, it is a stop so
+wide that it never triggers, which is a benchmark that quietly measures a
+different strategy.
+
+Version floor
+-------------
+0.9.0 renamed the config and result classes, dropping the ``Py`` prefix they
+carried through 0.4.x. Rather than support both, the import fails loudly on an
+older install: a silent fallback here would mean publishing a number from an
+engine five minor versions behind the one named in the report.
+"""
+from __future__ import annotations
+
+from typing import Any, Callable, Dict
+
+import numpy as np
+import raptorbt as rbt
+
+from workloads import CAPITAL, WORKLOADS
+
+NAME = "raptorbt"
+
+if not hasattr(rbt, "BacktestConfig"):        # 0.4.x and earlier
+    raise ImportError(
+        "raptorbt is too old for this harness: no BacktestConfig, which means "
+        "a release before 0.9.0. Install the pinned version from "
+        "requirements-lock.txt."
+    )
+
+
+def probe() -> Dict[str, Any]:
+    # From the installed metadata, not from ``rbt.__version__``: that constant
+    # is hand-maintained and has been wrong before (0.4.1 shipped declaring
+    # 0.4.0), and a benchmark that misreports which version it measured is worse
+    # than one that does not say.
+    import importlib.metadata as md
+
+    try:
+        version = md.version("raptorbt")
+    except Exception:
+        version = getattr(rbt, "__version__", "unknown")
+    return {"engine": NAME, "version": version}
+
+
+def _level(key: str, close: np.ndarray) -> np.ndarray:
+    """The entry condition, as a level, from raptorbt's own indicators."""
+    p = WORKLOADS[key].params
+    if key in ("sma_cross", "sma_cross_metrics", "bracket_sl_tp"):
+        fast = np.asarray(rbt.sma(close, p["fast"]), dtype=np.float64)
+        slow = np.asarray(rbt.sma(close, p["slow"]), dtype=np.float64)
+        # A NaN comparison is already False; saying so explicitly keeps the
+        # warmup out of the signal by construction rather than by numpy detail.
+        return (fast > slow) & ~np.isnan(fast) & ~np.isnan(slow)
+    raise KeyError("workload {!r} is not supported by raptorbt".format(key))
+
+
+def _config(key: str):
+    """Everything the engine is told about the run, including the bracket.
+
+    Percentages are handed over as fractions (see the module docstring).
+    """
+    p = WORKLOADS[key].params
+    config = rbt.BacktestConfig(
+        initial_capital=CAPITAL,
+        fees=float(p.get("fee_bps", 0.0)) / 10_000.0,
+        slippage=0.0,
+        upon_bar_close=True,
+    )
+    if "sl_pct" in p:
+        config.set_fixed_stop(p["sl_pct"] / 100.0)
+    if "tp_pct" in p:
+        config.set_fixed_target(p["tp_pct"] / 100.0)
+    return config
+
+
+def prepare(key: str, df, workdir: str | None = None) -> Callable[[], Dict[str, Any]]:
+    """Untimed setup; returns the closure the harness times."""
+    p = WORKLOADS[key].params
+    config = _config(key)
+    wants_metrics = bool(p.get("metrics"))
+
+    # Contiguous float64 columns, built once. The other adapters are handed
+    # their data ready to use too, so nobody pays for marshalling inside the
+    # measurement.
+    timestamps = df["timestamp"].astype("int64").to_numpy()
+    open_ = np.ascontiguousarray(df["open"].to_numpy(dtype=np.float64))
+    high = np.ascontiguousarray(df["high"].to_numpy(dtype=np.float64))
+    low = np.ascontiguousarray(df["low"].to_numpy(dtype=np.float64))
+    close = np.ascontiguousarray(df["close"].to_numpy(dtype=np.float64))
+    volume = np.ascontiguousarray(df["volume"].to_numpy(dtype=np.float64))
+
+    def run() -> Dict[str, Any]:
+        level = _level(key, close)
+        result = rbt.run_single_backtest(
+            timestamps, open_, high, low, close, volume,
+            level, ~level,
+            config=config,
+        )
+        m = result.metrics
+        out = {
+            "total_return": float(m.total_return_pct) / 100.0,
+            "final_equity": float(m.end_value),
+            "round_trips": int(m.total_closed_trades),
+            # raptorbt books round-trips; ``total_trades`` is that same count
+            # rather than a fill count, so there is no honest number to report.
+            "fills": None,
+            "total_fees": float(m.total_fees_paid),
+        }
+        if wants_metrics:
+            # Computed inside the run whether or not anyone reads them, like the
+            # reference and unlike vectorbt. That is the point of the workload
+            # pair, and it means reading them costs nothing measurable here.
+            out.update({
+                # Handed over as a positive percentage; the harness works in
+                # signed fractions of the account, as every other engine does.
+                "max_drawdown": -float(m.max_drawdown_pct) / 100.0,
+                "sharpe": float(m.sharpe_ratio),
+                "sortino": float(m.sortino_ratio),
+                # No volatility in the metrics object. Left absent rather than
+                # recomputed off the equity curve: this column is meant to show
+                # what the engine hands a user, and filling the gap in would
+                # publish the harness's arithmetic as raptorbt's.
+                "volatility": None,
+            })
+        return out
+
+    return run
+
+
+def diagnose(key: str, df, workdir: str | None = None) -> Dict[str, Any]:
+    """Untimed measurement of how far the bracket divergence goes.
+
+    The reference counts the round-trips it opens on an exit bar. The mirror
+    image on this side is how many round-trips are missing: raptorbt never
+    re-arms while the level holds, so its count is the reference's minus that
+    same population. Reporting both makes the claim checkable instead of asking
+    a reader to take the subtraction on faith.
+    """
+    note = WORKLOADS[key].notes.get(NAME)
+    if note is None or note.status != "documented":
+        return {}
+    metrics = prepare(key, df, workdir)()
+    return {
+        "round_trips": metrics["round_trips"],
+        "final_equity": metrics["final_equity"],
+    }

benchmarks/vs_vectorbt/engines.py

@@ -0,0 +1,81 @@
+"""The engine registry: who is in the comparison, and what each one can do.
+
+One backtester is the *reference* and the others are *challengers*. Every parity
+check joins a challenger to the reference, never two challengers to each other:
+with three engines there are three pairs, and reporting all of them turns a
+speed benchmark into a matrix nobody reads. manifoldbt is the reference because
+this harness ships with it, which is a statement about the plumbing and not a
+claim about the engine: the reference gets no advantage from the position, it is
+simply the one every timing is divided by.
+
+A challenger that is not installed is skipped with a printed line rather than
+crashing the run. The CI lock file pins all of them, so a skip on a runner is a
+finding; a skip on a laptop is somebody who did not want to install a competitor
+to read their own numbers.
+
+Ratio basis
+-----------
+``ratio_basis`` records how an engine annualises Sharpe and Sortino. manifoldbt
+and the vectorbt adapter both compute them on daily returns annualised by
+sqrt(365), so their ratios are directly comparable and small differences are
+worth reporting. raptorbt returns its own, on its own basis, and comparing those
+numbers to manifoldbt's would report a units mismatch as a disagreement. The
+parity gate never depends on this either way: it gates on money, round-trips and
+drawdown, all three basis-free.
+"""
+from __future__ import annotations
+
+import importlib
+from dataclasses import dataclass
+from typing import Dict, List
+
+
+@dataclass(frozen=True)
+class Engine:
+    name: str
+    # Short code used by the out-of-process probes, which take argv strings.
+    code: str
+    module: str
+    # "daily_365" ratios are comparable with the reference's; "native" are not.
+    ratio_basis: str
+    # PyPI distribution name, for the version stamp in the result envelope.
+    distribution: str
+
+
+REFERENCE = "manifoldbt"
+
+ENGINES: Dict[str, Engine] = {
+    e.name: e
+    for e in (
+        Engine("manifoldbt", "mbt", "engine_mbt", "daily_365", "manifoldbt"),
+        Engine("vectorbt", "vbt", "engine_vbt", "daily_365", "vectorbt"),
+        Engine("raptorbt", "rbt", "engine_rbt", "native", "raptorbt"),
+    )
+}
+
+CHALLENGERS: List[str] = [n for n in ENGINES if n != REFERENCE]
+
+BY_CODE: Dict[str, Engine] = {e.code: e for e in ENGINES.values()}
+
+
+def adapter(name: str):
+    """Import an adapter module on demand.
+
+    Lazy on purpose: the cold-start probe measures a process that has imported
+    exactly one engine, and a registry that imported all three at module scope
+    would make that measurement impossible to take.
+    """
+    return importlib.import_module(ENGINES[name].module)
+
+
+def installed(name: str) -> bool:
+    try:
+        adapter(name)
+        return True
+    except ImportError:
+        return False
+
+
+def present(names: List[str]) -> List[str]:
+    """Filter to the engines actually importable here, preserving order."""
+    return [n for n in names if installed(n)]

benchmarks/vs_vectorbt/parity.py

@@ -1,26 +1,34 @@
 """The gate: no speed number is published for a workload the engines disagree on.
 
-A benchmark between two backtesters is only a benchmark if both engines did the
-same work. This module compares what each engine produced and classifies the
-result, and ``bench.py`` refuses to report a timing for anything it classifies
-as a failure.
+A benchmark between backtesters is only a benchmark if they did the same work.
+This module compares what a challenger produced against the reference and
+classifies the result, and ``bench.py`` refuses to report a timing for anything
+it classifies as a failure.
+
+Every comparison is one challenger against the reference. Challengers are never
+joined to each other: three engines make three pairs, and a table of pairs is a
+matrix, not a benchmark. It also would not add anything, since agreement with
+the reference is transitive enough for the only question being asked here, which
+is whether a published timing describes the same simulation.
 
 Three verdicts:
 
 ``exact``
     Agreement down to float-reordering noise. The timing is publishable.
 ``documented``
-    The engines disagree, the workload declared it in advance, and the reason is
-    written down. The timing goes to the annex with the reason attached.
+    The two disagree, the workload declared it in advance for this specific
+    engine, and the reason is written down. The timing goes to the annex with
+    the reason attached.
 ``failed``
-    The engines disagree and nobody predicted it. That is a finding about the
-    engines, not about their speed: the timing is withheld.
+    They disagree and nobody predicted it. That is a finding about the engines,
+    not about their speed: the timing is withheld.
 """
 from __future__ import annotations
 
 from typing import Any, Dict
 
-from workloads import CAPITAL, WORKLOADS
+from engines import ENGINES, REFERENCE
+from workloads import CAPITAL, expectation, why
 
 # Float reordering across two implementations of the same arithmetic lands
 # around 1e-13 of the account on a million bars. Anything above this is a
@@ -43,13 +51,13 @@ def _vs_capital(a: float, b: float) -> float:
     return abs(a - b) / CAPITAL
 
 
-def compare(mbt: Dict[str, Any], vbt: Dict[str, Any], key: str) -> Dict[str, Any]:
-    expected = WORKLOADS[key].parity
+def compare(ref: Dict[str, Any], other: Dict[str, Any], key: str, engine: str) -> Dict[str, Any]:
+    expected = expectation(key, engine)
 
     diffs = {
-        "final_equity_vs_capital": _vs_capital(mbt["final_equity"], vbt["final_equity"]),
-        "round_trips_delta": mbt["round_trips"] - vbt["round_trips"],
-        "total_fees_vs_capital": _vs_capital(mbt["total_fees"], vbt["total_fees"]),
+        "final_equity_vs_capital": _vs_capital(ref["final_equity"], other["final_equity"]),
+        "round_trips_delta": ref["round_trips"] - other["round_trips"],
+        "total_fees_vs_capital": _vs_capital(ref["total_fees"], other["total_fees"]),
     }
     agrees = (
         diffs["final_equity_vs_capital"] <= REL_TOL
@@ -58,18 +66,26 @@ def compare(mbt: Dict[str, Any], vbt: Dict[str, Any], key: str) -> Dict[str, Any
     )
 
     # Workloads that also produce a performance summary are gated on the
-    # drawdown, which both engines compute at full bar resolution and which must
-    # match. The ratios are reported but not gated: manifoldbt buckets its daily
-    # returns slightly differently, a difference worth stating rather than
-    # hiding, and worth nothing at all as an argument about speed.
-    if "max_drawdown" in mbt and "max_drawdown" in vbt:
-        diffs["max_drawdown_rel"] = _rel(mbt["max_drawdown"], vbt["max_drawdown"])
+    # drawdown, which every engine here computes at full bar resolution and
+    # which must match. The ratios are reported but not gated: the reference
+    # buckets its daily returns slightly differently from vectorbt, a difference
+    # worth stating rather than hiding, and worth nothing at all as an argument
+    # about speed.
+    if other.get("max_drawdown") is not None and ref.get("max_drawdown") is not None:
+        diffs["max_drawdown_rel"] = _rel(ref["max_drawdown"], other["max_drawdown"])
         agrees = agrees and diffs["max_drawdown_rel"] <= REL_TOL
-        diffs["advisory_ratio_rel"] = {
-            name: _rel(mbt[name], vbt[name])
-            for name in ("sharpe", "sortino", "volatility")
-            if name in mbt and name in vbt
-        }
+        # Only between engines that annualise the same way. raptorbt returns its
+        # ratios on its own basis, and subtracting those from the reference's
+        # would publish a units mismatch as a disagreement (measured: Sharpe
+        # 0.21 against 8.14 on a run whose equity curve is bit-identical).
+        if ENGINES[engine].ratio_basis == ENGINES[REFERENCE].ratio_basis:
+            diffs["advisory_ratio_rel"] = {
+                name: _rel(ref[name], other[name])
+                for name in ("sharpe", "sortino", "volatility")
+                if ref.get(name) is not None and other.get(name) is not None
+            }
+        else:
+            diffs["ratio_basis"] = ENGINES[engine].ratio_basis
 
     if agrees:
         status = "exact"
@@ -83,6 +99,6 @@ def compare(mbt: Dict[str, Any], vbt: Dict[str, Any], key: str) -> Dict[str, Any
         "expected": expected,
         "publishable": status == "exact",
         "diffs": diffs,
-        "metrics": {"manifoldbt": mbt, "vectorbt": vbt},
-        "note": WORKLOADS[key].divergence if status == "documented" else "",
+        "metrics": {REFERENCE: ref, engine: other},
+        "note": why(key, engine) if status == "documented" else "",
     }

benchmarks/vs_vectorbt/probe_child.py

@@ -6,11 +6,11 @@ Two things cannot be measured honestly inside the main harness process:
 compiles its numba kernels on the first call, which is a real cost a user pays
 in every new notebook or script, and which the steady-state benchmark
 deliberately discards. Measuring it requires a process that has never imported
-either engine.
+any of the engines.
 
 *Memory* - peak resident memory attributable to the run. Once one engine has
-run in a process, the allocator has already grown and the other engine's
-measurement is meaningless.
+run in a process, the allocator has already grown and any other engine's
+measurement in it is meaningless.
 
 The ``baseline`` mode measures the same process doing everything except calling
 an engine (interpreter start, numpy and pandas import, data generation) so the
@@ -18,6 +18,7 @@ engine's own share can be read off rather than argued about.
 
     python probe_child.py coldstart mbt sma_cross 20000
     python probe_child.py memory    vbt sma_cross 5000000
+    python probe_child.py coldstart rbt sma_cross 20000
     python probe_child.py baseline  none sma_cross 20000
 """
 from __future__ import annotations
@@ -42,18 +43,24 @@ def _rss_mb() -> float:
 
 
 def _build(engine: str, workload: str, bars: int):
+    """Import exactly one adapter and hand back its timed closure.
+
+    The import happens here, inside the measurement, because on the cold-start
+    path it *is* part of what is being measured. Which is also why the engine is
+    named by its short code rather than passed as a module: this process must
+    have imported one engine and no others by the time it runs.
+    """
     import data as data_mod
 
     frame = data_mod.make_ohlcv(bars)
-    if engine == "mbt":
-        import engine_mbt
+    if engine == "none":
+        return lambda: {}
 
-        return engine_mbt.prepare(workload, frame, tempfile.mkdtemp(prefix="mbt_probe_"))
-    if engine == "vbt":
-        import engine_vbt
+    import engines as engines_mod
 
-        return engine_vbt.prepare(workload, frame, None)
-    return lambda: {}
+    name = engines_mod.BY_CODE[engine].name
+    workdir = tempfile.mkdtemp(prefix=engine + "_probe_")
+    return engines_mod.adapter(name).prepare(workload, frame, workdir)
 
 
 def cold_start(engine: str, workload: str, bars: int) -> dict:

benchmarks/vs_vectorbt/report.py

@@ -8,6 +8,11 @@ to the job summary so the numbers are visible without downloading an artifact.
 This is a run output, not an article: tables, and only the glue needed to read
 them. Every "why" belongs in README.md, which is written once instead of being
 reprinted underneath every single run.
+
+Two result schemas are accepted. Version 2 keys everything by engine name;
+version 1, written while the harness compared exactly two engines, is normalised
+into that shape on load so the results archived under ``results/`` keep
+rendering after the third engine arrived.
 """
 from __future__ import annotations
 
@@ -19,7 +24,74 @@ from typing import Any, Dict, List
 
 METHOD_LINK = "benchmarks/vs_vectorbt/README.md"
 
+# What version 1 files were, before the shape became a map.
+V1_REFERENCE = "manifoldbt"
+V1_CHALLENGER = "vectorbt"
 
+
+# --------------------------------------------------------------------------- #
+# Reading
+# --------------------------------------------------------------------------- #
+def normalise(payload: Dict[str, Any]) -> Dict[str, Any]:
+    """Bring a version 1 payload up to the per-engine shape used below."""
+    if payload.get("schema_version", 1) >= 2:
+        return payload
+
+    payload["reference"] = V1_REFERENCE
+    payload["engines"] = [V1_REFERENCE, V1_CHALLENGER]
+    for row in payload.get("results", []):
+        verdict = row.get("parity") or {}
+        row["status"] = verdict.get("status", "exact")
+        row["parity"] = {V1_CHALLENGER: verdict}
+        row["engines"] = [V1_REFERENCE, V1_CHALLENGER]
+        if row.get("speedup"):
+            row["speedup"] = {V1_CHALLENGER: row["speedup"]}
+        if row.get("divergence_scale"):
+            row["divergence_scale"] = {V1_REFERENCE: row["divergence_scale"]}
+
+    cold = payload.get("cold_start")
+    if cold:
+        cold["engines"] = [V1_REFERENCE, V1_CHALLENGER]
+        if not isinstance(cold.get("ratio"), dict):
+            cold["ratio"] = {V1_CHALLENGER: cold.get("ratio")}
+    mem = payload.get("memory")
+    if mem:
+        mem["engines"] = [V1_REFERENCE, V1_CHALLENGER]
+    for point in payload.get("sweeps") or []:
+        timings = point.get("timings")
+        if timings and "seconds" not in timings:
+            point["timings"] = {
+                "seconds": {
+                    V1_REFERENCE: timings.get("manifoldbt_s"),
+                    V1_CHALLENGER: timings.get("vectorbt_s"),
+                },
+                "ratio": ({V1_CHALLENGER: timings["ratio"]}
+                          if timings.get("ratio") is not None else {}),
+            }
+        memory = point.get("memory")
+        if memory and "manifoldbt_added_mb" in memory:
+            point["memory"] = {
+                V1_REFERENCE: memory.get("manifoldbt_added_mb"),
+                V1_CHALLENGER: memory.get("vectorbt_added_mb"),
+            }
+        if point.get(V1_CHALLENGER) and "status" in point[V1_CHALLENGER]:
+            point["out_of_scope"] = {V1_CHALLENGER: point[V1_CHALLENGER]}
+        if point.get("parity") and "status" in point["parity"]:
+            point["parity"] = {V1_CHALLENGER: point["parity"]}
+    return payload
+
+
+def _engines(payload: Dict[str, Any]) -> List[str]:
+    return payload.get("engines") or [V1_REFERENCE, V1_CHALLENGER]
+
+
+def _reference(payload: Dict[str, Any]) -> str:
+    return payload.get("reference", V1_REFERENCE)
+
+
+# --------------------------------------------------------------------------- #
+# Rendering
+# --------------------------------------------------------------------------- #
 def _ms(seconds: float) -> str:
     if seconds < 1.0:
         return "{:.1f} ms".format(seconds * 1e3)
@@ -33,7 +105,8 @@ def _header(payload: Dict[str, Any]) -> List[str]:
     if env.get("pinned_cores"):
         cores += " (pinned to {})".format(env["pinned_cores"])
     lines = [
-        "# manifoldbt {} vs vectorbt {}".format(versions["manifoldbt"], versions["vectorbt"]),
+        "# " + " vs ".join(
+            "{} {}".format(name, versions.get(name, "?")) for name in _engines(payload)),
         "",
         "`{os} {arch}` | {cpu} | {cores} cores | {ram} GB | python {py} | "
         "numpy {np} / numba {nb} / pandas {pd} | {reps} interleaved reps | {when}".format(
@@ -48,31 +121,53 @@ def _header(payload: Dict[str, Any]) -> List[str]:
     return lines
 
 
-def _speed_table(rows: List[Dict[str, Any]], title: str) -> List[str]:
+def _speed_table(rows: List[Dict[str, Any]], title: str, payload: Dict[str, Any]) -> List[str]:
+    """One row per workload and size, one column per engine, then the ratios.
+
+    Columns are taken from the run rather than hardcoded, and a cell is only
+    empty when that engine was withheld or sat the workload out. Those cases get
+    a marker and a sentence of their own further down, because a blank in a
+    speed table reads as a defeat.
+    """
     if not rows:
         return []
+    reference = _reference(payload)
+    engines = _engines(payload)
+    challengers = [e for e in engines if e != reference]
+
+    head = "| Workload | Bars | " + " | ".join(engines) + " | "
+    head += " | ".join("vs " + c for c in challengers) + " |"
     lines = [
         "## " + title,
         "",
-        "| Workload | Bars | manifoldbt | vectorbt | Ratio |",
-        "|---|---:|---:|---:|---:|",
+        head,
+        "|---|---:|" + "---:|" * (len(engines) + len(challengers)),
     ]
     for row in rows:
         timings = row["timings"]
-        lines.append(
-            "| {w} | {b:,} | {m} | {v} | **x{s:.1f}**{f} |".format(
-                w=row["workload"], b=row["bars"],
-                m=_ms(timings["manifoldbt"]["median_s"]),
-                v=_ms(timings["vectorbt"]["median_s"]),
-                s=row["speedup"]["median_of_ratios"],
-                f=" ~" if row.get("noisy") else "",
-            )
-        )
+        cells = [
+            _ms(timings[e]["median_s"]) if e in timings else "-" for e in engines
+        ]
+        speedup = row.get("speedup") or {}
+        # The noise flag rides on the last ratio, where a reader's eye already
+        # is when deciding whether to believe the number.
+        mark = " ~" if row.get("noisy") else ""
+        ratios = [
+            "**x{:.1f}**".format(speedup[c]["median_of_ratios"]) if c in speedup else "-"
+            for c in challengers
+        ]
+        # On the last ratio that actually has a number: hung on a "-" it would
+        # look like a comment on the engine that did not run.
+        present = [i for i, cell in enumerate(ratios) if cell != "-"]
+        if mark and present:
+            ratios[present[-1]] += mark
+        lines.append("| {w} | {b:,} | {cells} |".format(
+            w=row["workload"], b=row["bars"], cells=" | ".join(cells + ratios)))
     lines.append("")
     return lines
 
 
-def _summary_cost(exact: List[Dict[str, Any]]) -> List[str]:
+def _summary_cost(exact: List[Dict[str, Any]], payload: Dict[str, Any]) -> List[str]:
     plain = {r["bars"]: r for r in exact if r["workload"] == "sma_cross"}
     summarised = {r["bars"]: r for r in exact if r["workload"] == "sma_cross_metrics"}
     # Only subtract timings that were measured in the same interleaved loop.
@@ -92,21 +187,45 @@ def _summary_cost(exact: List[Dict[str, Any]]) -> List[str]:
         "|---:|---|---:|---:|---:|",
     ]
     for bars in shared:
-        for engine in ("manifoldbt", "vectorbt"):
-            without = plain[bars]["timings"][engine]["median_s"]
-            with_ = summarised[bars]["timings"][engine]["median_s"]
+        for engine in _engines(payload):
+            without = (plain[bars]["timings"] or {}).get(engine)
+            with_ = (summarised[bars]["timings"] or {}).get(engine)
+            if not (without and with_):
+                continue
+            a, c = without["median_s"], with_["median_s"]
             lines.append(
                 "| {b:,} | {e} | {a} | {c} | {d} |".format(
-                    b=bars, e=engine, a=_ms(without), c=_ms(with_),
-                    d=("+" + _ms(with_ - without)) if with_ > without else "none measurable",
+                    b=bars, e=engine, a=_ms(a), c=_ms(c),
+                    d=("+" + _ms(c - a)) if c > a else "none measurable",
                 )
             )
     lines.append("")
-    advisory = summarised[shared[-1]]["parity"]["diffs"].get("advisory_ratio_rel")
+
+    top = summarised[shared[-1]]
+    advisory = {}
+    basis_differs = {}
+    for engine, verdict in top["parity"].items():
+        diffs = verdict.get("diffs", {})
+        if diffs.get("advisory_ratio_rel"):
+            advisory[engine] = max(diffs["advisory_ratio_rel"].values())
+        elif diffs.get("ratio_basis"):
+            basis_differs[engine] = diffs.get("max_drawdown_rel")
     if advisory:
         lines += [
-            "Gated on total return, round-trips and max drawdown (exact). Sharpe, Sortino "
-            "and volatility agree to {:.1e}, from daily bucketing.".format(max(advisory.values())),
+            "Gated on total return, round-trips and max drawdown (exact). Sharpe, "
+            "Sortino and volatility agree to {:.1e}.".format(max(advisory.values())),
+            "",
+        ]
+    for engine, drawdown_rel in basis_differs.items():
+        if drawdown_rel is None:
+            agreement = "was not compared"
+        elif drawdown_rel == 0.0:
+            agreement = "is identical"
+        else:
+            agreement = "agrees to {:.1e}".format(drawdown_rel)
+        lines += [
+            "{e} annualises its ratios on its own basis, so only the drawdown is "
+            "compared there: it {a}.".format(e=engine, a=agreement),
             "",
         ]
     return lines
@@ -115,82 +234,216 @@ def _summary_cost(exact: List[Dict[str, Any]]) -> List[str]:
 def _side_measures(payload: Dict[str, Any], results: List[Dict[str, Any]]) -> List[str]:
     cold = payload.get("cold_start")
     mem = payload.get("memory")
+    engines = _engines(payload)
     threading_rows = [
         r for r in results
         if r.get("cpu_over_wall")
-        and all(r["cpu_over_wall"].get(e) is not None for e in ("manifoldbt", "vectorbt"))
+        and all(r["cpu_over_wall"].get(e) is not None for e in engines)
     ]
     if not (cold or mem or threading_rows):
         return []
 
-    lines = ["## Cold start, memory, threads", "", "| | manifoldbt | vectorbt |", "|---|---:|---:|"]
+    lines = [
+        "## Cold start, memory, threads",
+        "",
+        "| | " + " | ".join(engines) + " |",
+        "|---|" + "---:|" * len(engines),
+    ]
     if cold:
         medians, share = cold["median_s"], cold["engine_share_s"]
-        lines.append("| Fresh process to first backtest | {:.2f} s | {:.2f} s |".format(
-            medians["manifoldbt"], medians["vectorbt"]))
-        lines.append("| ... minus the {:.2f} s python baseline | {:.2f} s | {:.2f} s |".format(
-            medians["baseline"], share["manifoldbt"], share["vectorbt"]))
+        lines.append("| Fresh process to first backtest | " + " | ".join(
+            "{:.2f} s".format(medians[e]) if e in medians else "-" for e in engines) + " |")
+        lines.append("| ... minus the {:.2f} s python baseline | ".format(medians["baseline"])
+                     + " | ".join("{:.2f} s".format(share[e]) if e in share else "-"
+                                  for e in engines) + " |")
     if mem:
-        lines.append("| RAM added by the run, per 1M bars | {:.0f} MB | {:.0f} MB |".format(
-            mem["manifoldbt"]["added_mb_per_million_bars"],
-            mem["vectorbt"]["added_mb_per_million_bars"]))
+        lines.append("| RAM added by the run, per 1M bars | " + " | ".join(
+            "{:.0f} MB".format(mem[e]["added_mb_per_million_bars"]) if e in mem else "-"
+            for e in engines) + " |")
     if threading_rows:
         biggest = max(threading_rows, key=lambda r: r["bars"])
         ratio = biggest["cpu_over_wall"]
-        lines.append("| CPU over wall time at {:,} bars | {:.2f} | {:.2f} |".format(
-            biggest["bars"], ratio["manifoldbt"], ratio["vectorbt"]))
+        lines.append("| CPU over wall time at {:,} bars | ".format(biggest["bars"])
+                     + " | ".join("{:.2f}".format(ratio[e]) for e in engines) + " |")
     lines.append("")
     return lines
 
 
+def _not_run(payload: Dict[str, Any], results: List[Dict[str, Any]]) -> List[str]:
+    """Workloads an engine sits out, and why.
+
+    Kept as prose rather than a table because the reason is the content. An
+    engine that cannot express a workload has told you something about itself,
+    and compressing that into an empty cell would throw away the only part worth
+    reading.
+    """
+    seen: Dict[str, Dict[str, str]] = {}
+    for row in results:
+        for engine, why in (row.get("unsupported") or {}).items():
+            if engine in _engines(payload):
+                seen.setdefault(engine, {})[row["workload"]] = why
+    if not seen:
+        return []
+
+    lines = ["## Not run, and why", ""]
+    for engine, entries in seen.items():
+        for workload, why in entries.items():
+            lines += ["- **{e} on `{w}`.** {why}".format(e=engine, w=workload, why=why), ""]
+    return lines
+
+
+def _gb(mb: float) -> str:
+    return "{:.1f} GB".format(mb / 1024) if mb >= 1024 else "{:.0f} MB".format(mb)
+
+
+def _sweep_section(payload: Dict[str, Any]) -> List[str]:
+    """The parameter-grid table, plus the memory that decides what is runnable.
+
+    Memory is reported next to the timings rather than in its own annex because
+    on a grid it is not a footnote: it is the first thing to run out. A machine
+    that cannot hold the grid does not produce a slow number, it produces no
+    number, and a reader sizing a job needs both columns side by side.
+    """
+    sweeps = payload.get("sweeps")
+    if not sweeps:
+        return []
+
+    reference = _reference(payload)
+    engines = _engines(payload)
+    challengers = [e for e in engines if e != reference]
+    timed = [s for s in sweeps if s.get("timings")]
+    untimed = [s for s in sweeps if not s.get("timings")]
+    oos = [s for s in timed if s.get("out_of_scope")]
+    lines = ["## Parameter sweeps", ""]
+
+    if timed:
+        header = "| Bars | Combinations | " + " | ".join(engines) + " | "
+        header += " | ".join("vs " + c for c in challengers) + " | "
+        header += " | ".join("RAM " + e for e in engines) + " |"
+        lines += [
+            header,
+            # bars, combinations, one column per engine, one ratio per
+            # challenger, then one RAM column per engine.
+            "|---:|---:|" + "---:|" * (2 * len(engines) + len(challengers)),
+        ]
+        for s in timed:
+            mem = s.get("memory") or {}
+            seconds = s["timings"]["seconds"]
+            ratios = s["timings"].get("ratio") or {}
+            cells = [_ms(seconds[e]) if seconds.get(e) is not None else "not run"
+                     for e in engines]
+            cells += ["**x{:.1f}**".format(ratios[c]) if ratios.get(c) is not None else "-"
+                      for c in challengers]
+            cells += [_gb(mem[e]) if mem.get(e) is not None else "-" for e in engines]
+            lines.append("| {b:,} | {c:,} | {cells} |".format(
+                b=s["bars"], c=s["combos"], cells=" | ".join(cells)))
+        lines += [
+            "",
+            "Each grid is checked cell by cell before any of it is timed: the "
+            "engines are joined on the parameter pair they actually ran, not on "
+            "position, and the worst disagreement in the grid is what the gate "
+            "sees. A single wrong cell among thousands is exactly the failure a "
+            "sweep can have and a single backtest cannot.",
+            "",
+        ]
+
+    if oos:
+        lines += ["### Where a challenger was not run", ""]
+        for s in oos:
+            for engine, detail in s["out_of_scope"].items():
+                verdict = (s.get("parity") or {}).get(engine) or {}
+                lines.append(
+                    "- **{e}, {c:,} combinations at {b:,} bars.** {why} The "
+                    "cross-engine check for this point therefore covers the same "
+                    "code path at {a:,} combinations, not this grid: agreement "
+                    "was `{st}`.".format(
+                        e=engine, c=s["combos"], b=s["bars"], why=detail["reason"],
+                        a=verdict.get("checked_at_combos", 0),
+                        st=verdict.get("status", "?"),
+                    )
+                )
+        lines.append("")
+
+    if untimed:
+        lines += ["No timing for these points:", ""]
+        for s in untimed:
+            lines.append("- {b:,} bars x {c:,} combinations: {why}".format(
+                b=s["bars"], c=s["combos"],
+                why=s.get("reason") or s.get("note") or "no result",
+            ))
+        lines.append("")
+    return lines
+
+
+def _divergence_note(row: Dict[str, Any], payload: Dict[str, Any]) -> List[str]:
+    """The measured size of a documented divergence, engine by engine."""
+    scales = row.get("divergence_scale") or {}
+    reference = _reference(payload)
+    view = scales.get(reference)
+    if not view:
+        return []
+    lines = [
+        "`{w}` at {b:,} bars: {n} of {t} {ref} round-trips re-enter on the exit "
+        "bar ({p:.0%}), from {sl} stop and {tp} target exits.".format(
+            w=row["workload"], b=row["bars"], n=view["reentries_on_exit_bar"],
+            t=view["round_trips"], p=view["share_of_round_trips"],
+            sl=view["sl_exits"], tp=view["tp_exits"], ref=reference,
+        ),
+    ]
+    for engine, scale in scales.items():
+        if engine == reference or "round_trips" not in scale:
+            continue
+        lines[-1] += " {e} books {n}.".format(e=engine, n=scale["round_trips"])
+    lines += ["", "Cause in {}.".format(METHOD_LINK), ""]
+    return lines
+
+
 def render(payload: Dict[str, Any]) -> str:
+    payload = normalise(payload)
     results = payload["results"]
-    exact = [r for r in results if r["parity"]["status"] == "exact" and r.get("timings")]
-    documented = [r for r in results if r["parity"]["status"] == "documented"]
-    failed = [r for r in results if r["parity"]["status"] == "failed"]
+    exact = [r for r in results if r.get("status") == "exact" and r.get("timings")]
+    documented = [r for r in results if r.get("status") == "documented"]
+    failed = [r for r in results if r.get("status") == "failed"]
 
     lines = _header(payload)
-    lines += _speed_table(exact, "Same results, both engines")
-    lines += _summary_cost(exact)
+    lines += _speed_table(exact, "Same results, every engine", payload)
+    lines += _summary_cost(exact, payload)
     lines += _side_measures(payload, results)
+    lines += _not_run(payload, results)
+    lines += _sweep_section(payload)
 
     timed = [r for r in documented if r.get("timings")]
     if timed:
-        lines += _speed_table(timed, "Results differ, kept out of the headline")
-        scales = [r for r in documented if r.get("divergence_scale")]
-        if scales:
-            row = scales[0]
-            scale = row["divergence_scale"]
-            lines += [
-                "`{w}`: {n} of {t} round-trips re-enter on the exit bar ({p:.0%}) at {b:,} "
-                "bars, from {sl} stop and {tp} target exits. Cause in {link}.".format(
-                    w=row["workload"], n=scale["reentries_on_exit_bar"],
-                    t=scale["round_trips"], p=scale["share_of_round_trips"],
-                    b=row["bars"], sl=scale["sl_exits"], tp=scale["tp_exits"],
-                    link=METHOD_LINK,
-                ),
-                "",
-            ]
+        lines += _speed_table(timed, "Results differ, kept out of the headline", payload)
+        for row in documented:
+            if row.get("divergence_scale"):
+                lines += _divergence_note(row, payload)
+                break
 
     if failed:
         lines += ["## Timing withheld", ""]
         for row in failed:
-            diffs = row["parity"]["diffs"]
-            lines.append(
-                "- `{w}` at {b:,} bars: final equity differs by {d:.2e} of capital, "
-                "round-trips by {t}.".format(
-                    w=row["workload"], b=row["bars"],
-                    d=diffs["final_equity_vs_capital"], t=diffs["round_trips_delta"],
+            for engine, verdict in row["parity"].items():
+                if verdict["status"] != "failed":
+                    continue
+                diffs = verdict["diffs"]
+                lines.append(
+                    "- `{w}` at {b:,} bars, {e}: final equity differs by {d:.2e} of "
+                    "capital, round-trips by {t}.".format(
+                        w=row["workload"], b=row["bars"], e=engine,
+                        d=diffs["final_equity_vs_capital"],
+                        t=diffs["round_trips_delta"],
+                    )
                 )
-            )
         lines.append("")
 
     lines += [
         "---",
         "",
         "`~` = IQR above 15% of the median, indicative only. Ratios are medians of "
-        "per-repetition ratios with the engines interleaved; data loading excluded, warmup "
-        "discarded. Method and caveats: {}".format(METHOD_LINK),
+        "per-repetition ratios against {ref}, with the engines interleaved; data loading "
+        "excluded, warmup discarded. Method and caveats: {link}".format(
+            ref=_reference(payload), link=METHOD_LINK),
         "",
     ]
     return "\n".join(lines)

benchmarks/vs_vectorbt/requirements-lock.txt

@@ -5,7 +5,17 @@
 #
 # manifoldbt itself is NOT pinned here: the workflow installs the version under
 # test (`pip install manifoldbt==X`), so the same lock file serves every release.
+#
+# raptorbt has no dependency of its own, so pinning it moves nothing else in the
+# set. It does pin the *interpreter*, though: it is built against pyo3 0.20.3,
+# whose maximum supported CPython is 3.12, so there is no cp313 wheel in any
+# release up to 0.9.0 and a source build refuses outright ("the configured
+# Python interpreter version (3.13) is newer than PyO3's maximum supported
+# version (3.12)"). That is why the workflow runs 3.12: comparing engines means
+# running them in one environment, and the environment has to be one they all
+# support.
 vectorbt==0.28.4
+raptorbt==0.9.0
 numpy==2.4.3
 numba==0.64.0
 pandas==2.3.3

benchmarks/vs_vectorbt/sweep_child.py

@@ -0,0 +1,670 @@
+"""One sweep point, one fresh process, one JSON line on stdout.
+
+A parameter sweep is where vectorbt is supposed to be at its strongest: it
+broadcasts the whole grid into one vectorised simulation. So it is the workload
+where a speed claim needs the tightest gate, and where engines are the easiest
+to compare *wrongly*.
+
+Its own process, for three reasons:
+
+* memory. The peak of a large grid is the number that decides what a machine can
+  run at all, and it cannot be read once another engine has already grown the
+  allocator in the same process.
+* tier. The engine's fan-out allowance is counted per process, so two points
+  sharing one would interfere.
+* isolation. A 100k-combination grid that runs out of memory takes its process
+  down with it; one point dying should not cost the whole benchmark.
+
+    python sweep_child.py --bars 100000 --combos 250
+
+The grid alignment trap
+-----------------------
+Every engine is asked for the same set of parameter pairs, but nothing forces
+them to return the results in the same ORDER. manifoldbt enumerates its grid in
+alphabetical key order (``fast`` outer, ``slow`` inner); vectorbt returns one
+column per combination in the order its parameter product was built. Zip two of
+them together wrongly and every number still looks plausible: the arrays have the
+same length, the same distribution, even the same best value. Only the mapping
+is scrambled, and the comparison silently becomes meaningless.
+
+So the pairing is not assumed. Each engine returns its results keyed by the
+``(fast, slow)`` pair it actually ran, and the comparison joins on that key.
+
+Not every engine has a grid to broadcast
+----------------------------------------
+raptorbt has no fan-out API for a parameter grid on one instrument, so its
+column here is a Python loop over ``run_single_backtest``. That is not a handicap
+imposed by the harness, it is the only spelling available, and it is the one a
+raptorbt user would write. It gets the same courtesy vectorbt gets on its own
+path: each distinct moving average is computed once and reused across every
+combination it appears in, rather than recomputed per cell.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+import tempfile
+import threading
+import time
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from engines import BY_CODE, ENGINES, REFERENCE  # noqa: E402
+
+CAPITAL_TOL = 1e-9  # same yardstick as parity.py: difference over capital
+
+
+def _mem_mb() -> float:
+    import psutil
+
+    info = psutil.Process().memory_info()
+    # RSS under-reports under memory pressure, which is exactly the regime a
+    # large grid puts the machine in. Prefer the private working set where the
+    # platform exposes it.
+    return (getattr(info, "private", None) or info.rss) / (1024 * 1024)
+
+
+class _Peak(threading.Thread):
+    # `_halt`, not `_stop`: Thread already has a private `_stop()` method, and
+    # shadowing it with an Event makes `join()` raise "'Event' object is not
+    # callable" on CPython 3.12, where `_wait_for_tstate_lock` calls it. The
+    # sampler is the last thing anyone suspects when a sweep dies in join().
+    def __init__(self, interval: float = 0.002):
+        super().__init__(daemon=True)
+        self.interval = interval
+        self.peak = 0.0
+        self._halt = threading.Event()
+
+    def run(self):
+        while not self._halt.is_set():
+            self.peak = max(self.peak, _mem_mb())
+            time.sleep(self.interval)
+
+    def stop(self) -> float:
+        self._halt.set()
+        self.join(timeout=2.0)
+        self.peak = max(self.peak, _mem_mb())
+        return self.peak
+
+
+def grid(combos: int) -> tuple[list[int], list[int]]:
+    """Split `combos` into a near-square fast x slow grid.
+
+    Near-square rather than a long strip: a 1 x N grid would sweep a single fast
+    period and measure indicator reuse rather than fan-out. The fast and slow
+    ranges are kept disjoint so no cell has fast >= slow, which would never trade
+    and would pad the grid with empty simulations.
+    """
+    n_fast = int(combos ** 0.5)
+    while combos % n_fast:
+        n_fast -= 1
+    n_slow = combos // n_fast
+    fast_vals = [5 + 2 * i for i in range(n_fast)]
+    slow_vals = [max(fast_vals) + 10 + 5 * i for i in range(n_slow)]
+    return fast_vals, slow_vals
+
+
+def tier() -> dict:
+    """What the engine thinks it is allowed to do, right now.
+
+    Read before and after the sweep. The downgrade that follows a refused
+    licence ping lands from a background thread, so a run can legitimately start
+    Pro and finish Community: a number measured across that boundary is not a
+    measurement of anything.
+    """
+    import manifoldbt as mbt
+
+    used, limit, is_pro = mbt._native._combo_budget()
+    return {"pro": bool(is_pro), "budget_used": int(used), "budget_limit": int(limit)}
+
+
+def run_mbt(df, fast_vals, slow_vals, workdir, metrics=False):
+    import manifoldbt as mbt
+    from manifoldbt.expr import col, lit, when
+    from manifoldbt.helpers import Interval, Slippage
+    from manifoldbt.indicators import close as close_px, sma
+
+    from workloads import CAPITAL
+
+    store = mbt.import_dataframe(
+        df,
+        symbol="BENCH",
+        symbol_id=1,
+        interval="1m",
+        data_root=os.path.join(workdir, "data"),
+        metadata_db=os.path.join(workdir, "metadata.sqlite"),
+    )
+    last_ns = int(df["timestamp"].iloc[-1].value)
+    config = mbt.BacktestConfig(
+        universe=[1],
+        time_range_start=0,
+        time_range_end=last_ns + 86_400_000_000_000,
+        bar_interval=Interval.minutes(1),
+        initial_capital=CAPITAL,
+        execution=mbt.ExecutionConfig(
+            signal_delay=0,
+            execution_price="AtClose",
+            max_position_pct=1.0,
+            allow_short=False,
+            position_sizing_mode="FractionOfEquity",
+        ),
+        fees=mbt.FeeConfig.zero(),
+        slippage=Slippage.none(),
+        warmup_bars=0,
+    )
+    strategy = (
+        mbt.Strategy.create("sma_sweep")
+        .signal("fast", sma(close_px, mbt.param("fast")))
+        .signal("slow", sma(close_px, mbt.param("slow")))
+        .size(when(col("fast") > col("slow"), lit(1.0), lit(0.0)))
+    )
+
+    def call():
+        batch = mbt.run_sweep_lite(
+            strategy, {"fast": fast_vals, "slow": slow_vals}, config, store
+        )
+        # Alphabetical key order: "fast" is the outer loop, "slow" the inner one.
+        # Keyed by the pair rather than by position, so a change to that order
+        # breaks the join loudly instead of scrambling the comparison.
+        out = {}
+        i = 0
+        for f in fast_vals:
+            for s in slow_vals:
+                m = batch[i].metrics
+                out[(f, s)] = (
+                    (float(m["total_return"]), float(m["max_drawdown"]))
+                    if metrics else float(m["total_return"])
+                )
+                i += 1
+        if i != len(batch):
+            raise RuntimeError(f"grid decode consumed {i} of {len(batch)} results")
+        return out
+
+    return call
+
+
+def run_vbt(df, fast_vals, slow_vals, metrics=False):
+    """vectorbt's grid path, taking the faster of its two ways of building it.
+
+    The obvious spelling is to hand `MA.run` one window per combination and let
+    `ma_above` line the two up. It is also the wrong one to benchmark: it
+    recomputes every moving average once per combination it appears in, so a
+    10 x 25 grid computes 500 averages instead of 35. Measured on 20k bars and
+    250 combinations, that spelling takes 8.6x longer to build the same signal
+    matrix, bit for bit.
+
+    So the averages are computed once each and the product is formed by indexing
+    columns. vectorbt is credited with its quicker path, exactly as the fee
+    workload credits it with the quicker of its two metric paths.
+    """
+    import itertools
+
+    import numpy as np
+    import pandas as pd
+    import vectorbt as vbt
+
+    from workloads import CAPITAL, FREQ
+
+    index = pd.DatetimeIndex(df["timestamp"])
+    close = pd.Series(df["close"].to_numpy(dtype=np.float64), index=index)
+    pairs = list(itertools.product(fast_vals, slow_vals))
+    columns = pd.MultiIndex.from_tuples(pairs, names=["fast_window", "slow_window"])
+    fast_idx = np.repeat(np.arange(len(fast_vals)), len(slow_vals))
+    slow_idx = np.tile(np.arange(len(slow_vals)), len(fast_vals))
+
+    def call():
+        fast_ma = vbt.MA.run(close, window=fast_vals, short_name="fast").ma.to_numpy()
+        slow_ma = vbt.MA.run(close, window=slow_vals, short_name="slow").ma.to_numpy()
+        entries = pd.DataFrame(
+            fast_ma[:, fast_idx] > slow_ma[:, slow_idx], index=index, columns=columns
+        )
+        portfolio = vbt.Portfolio.from_signals(
+            close,
+            entries=entries,
+            exits=~entries,
+            init_cash=CAPITAL,
+            size=1.0,
+            size_type="percent",
+            fees=0.0,
+            slippage=0.0,
+            direction="longonly",
+            accumulate=False,
+            freq=FREQ,
+        )
+        returns = portfolio.total_return()
+        # Columns carry a MultiIndex of the two window levels. Read the pair off
+        # the index rather than trusting positional order.
+        names = list(returns.index.names)
+        fi = names.index("fast_window")
+        si = names.index("slow_window")
+        if not metrics:
+            return {
+                (int(kv[fi]), int(kv[si])): float(v)
+                for kv, v in zip(returns.index, returns.to_numpy())
+            }
+        # The drawdown is what makes this workload different: it cannot be read
+        # off the trade records, it needs the equity curve of every column, and
+        # `from_signals` defers building that until something asks. Written out
+        # rather than through `portfolio.max_drawdown()` because the accessor is
+        # measurably slower on the same data, and vectorbt is credited with the
+        # quicker of its two paths here exactly as it is everywhere else.
+        equity = portfolio.value().to_numpy()
+        drawdown = (equity / np.maximum.accumulate(equity, axis=0) - 1.0).min(axis=0)
+        return {
+            (int(kv[fi]), int(kv[si])): (float(r), float(d))
+            for kv, r, d in zip(returns.index, returns.to_numpy(), drawdown)
+        }
+
+    return call
+
+
+def run_rbt(df, fast_vals, slow_vals, metrics=False):
+    """raptorbt's grid path: one backtest per cell, in Python.
+
+    There is no fan-out entry point to call. ``run_multi_backtest`` broadcasts
+    over *instruments*, not over parameters, so a parameter grid on one symbol is
+    a loop, and the loop is what a user would write.
+
+    The moving averages are hoisted out of it. Computing them per cell would
+    recompute the same average once per combination it appears in, which is the
+    spelling explicitly rejected on the vectorbt side; rejecting it there and
+    accepting it here would be scoring the two engines with different rulers.
+    """
+    import numpy as np
+    import raptorbt as rbt
+
+    from workloads import CAPITAL
+
+    timestamps = df["timestamp"].astype("int64").to_numpy()
+    open_ = np.ascontiguousarray(df["open"].to_numpy(dtype=np.float64))
+    high = np.ascontiguousarray(df["high"].to_numpy(dtype=np.float64))
+    low = np.ascontiguousarray(df["low"].to_numpy(dtype=np.float64))
+    close = np.ascontiguousarray(df["close"].to_numpy(dtype=np.float64))
+    volume = np.ascontiguousarray(df["volume"].to_numpy(dtype=np.float64))
+    config = rbt.BacktestConfig(
+        initial_capital=CAPITAL, fees=0.0, slippage=0.0, upon_bar_close=True
+    )
+
+    def call():
+        periods = sorted(set(fast_vals) | set(slow_vals))
+        averages = {p: np.asarray(rbt.sma(close, p), dtype=np.float64) for p in periods}
+        finite = {p: ~np.isnan(a) for p, a in averages.items()}
+        out = {}
+        for f in fast_vals:
+            fast, fast_ok = averages[f], finite[f]
+            for s in slow_vals:
+                level = (fast > averages[s]) & fast_ok & finite[s]
+                result = rbt.run_single_backtest(
+                    timestamps, open_, high, low, close, volume,
+                    level, ~level, config=config,
+                )
+                m = result.metrics
+                ret = float(m.total_return_pct) / 100.0
+                # Already computed inside the run, like the reference: asking
+                # for it costs raptorbt nothing, and that is a result, not a
+                # concession.
+                out[(f, s)] = ((ret, -float(m.max_drawdown_pct) / 100.0)
+                               if metrics else ret)
+        return out
+
+    return call
+
+
+def builder(name: str, df, fast_vals, slow_vals, workdir: str, metrics: bool = False):
+    """The grid closure for one engine, ready to be timed."""
+    if name == "manifoldbt":
+        return run_mbt(df, fast_vals, slow_vals, workdir, metrics)
+    if name == "vectorbt":
+        return run_vbt(df, fast_vals, slow_vals, metrics)
+    if name == "raptorbt":
+        return run_rbt(df, fast_vals, slow_vals, metrics)
+    raise KeyError("no sweep path for engine {!r}".format(name))
+
+
+def compare(reference_out: dict, other_out: dict) -> dict:
+    """Join on the parameter pair, then gate on the worst cell, not the average.
+
+    An average would hide a single badly wrong combination in a grid of
+    thousands, which is precisely the failure a sweep can have and a single
+    backtest cannot.
+    """
+    from workloads import CAPITAL
+
+    missing = sorted(set(reference_out) ^ set(other_out))
+    if missing:
+        return {
+            "status": "failed",
+            "reason": f"{len(missing)} parameter pairs present in one engine only",
+            "examples": [list(p) for p in missing[:5]],
+        }
+    # Both sides report a total return, a fraction of the account. Comparing
+    # those directly is the same yardstick parity.py uses for a single backtest
+    # (a difference in final equity over capital), without the round trip
+    # through money.
+    _ = CAPITAL
+    # A cell is either a total return, or a (return, drawdown) pair when the
+    # sweep was asked for a performance summary. Both components are gated: a
+    # drawdown that disagrees is the same class of finding as a return that
+    # does, and letting it through would publish a timing for work that was not
+    # the same work.
+    worst_pair, worst, worst_component = None, 0.0, None
+    for pair, mv in reference_out.items():
+        ov = other_out[pair]
+        components = zip(mv, ov) if isinstance(mv, tuple) else ((mv, ov),)
+        for index, (a, b) in enumerate(components):
+            d = abs(a - b)
+            if d > worst:
+                worst, worst_pair = d, pair
+                worst_component = ("return", "max_drawdown")[index]
+    return {
+        "status": "exact" if worst <= CAPITAL_TOL else "failed",
+        "combos_compared": len(reference_out),
+        "worst_abs_return_delta": worst,
+        "worst_component": worst_component,
+        "worst_pair": list(worst_pair) if worst_pair else None,
+    }
+
+
+# vectorbt is the only engine here whose grid footprint grows with the number of
+# combinations, so it is the only one a point can declare out of scope. The
+# others are looped or streamed and cost the same memory at any grid size.
+OOS_ENGINE = "vectorbt"
+OOS_REASON = (
+    "vectorbt materialises the simulation per combination (measured: 1.57 MB "
+    "per combination at 20k bars), so this grid would need tens of gigabytes. "
+    "Running it would measure the swap file."
+)
+
+
+def main() -> int:
+    """Entry point. The scratch store is removed on the way out, always.
+
+    One point leaves one mkdtemp behind, and a matrix run spawns a child per
+    point per engine per memory probe. A day of measurements left 87 of them and
+    14.7 GB on the volume TEMP points at, which on a full system drive is not
+    housekeeping, it is the benchmark failing with "Espace insuffisant sur le
+    disque" in the middle of a run.
+    """
+    try:
+        return _main(ap_parse())
+    finally:
+        _cleanup()
+
+
+_SCRATCH: list = []
+
+
+def _cleanup() -> None:
+    import shutil
+
+    for path in _SCRATCH:
+        shutil.rmtree(path, ignore_errors=True)
+
+
+def _scratch(prefix: str) -> str:
+    """A temporary directory that will be removed when this process exits."""
+    path = tempfile.mkdtemp(prefix=prefix)
+    _SCRATCH.append(path)
+    return path
+
+
+def ap_parse():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--bars", type=int, required=True)
+    ap.add_argument("--combos", type=int, required=True)
+    ap.add_argument("--reps", type=int, default=1)
+    ap.add_argument(
+        "--metrics",
+        action="store_true",
+        help="ask every cell for a performance summary, not just a total "
+             "return. This is what a user sweeping thousands of combinations "
+             "actually reads, and it is the one axis where the engines differ "
+             "in kind rather than in speed: manifoldbt and raptorbt compute the "
+             "drawdown inside the run whether it is read or not, while vectorbt "
+             "defers the equity curve until something asks for it and then has "
+             "to build one per column.",
+    )
+    ap.add_argument("--engines", nargs="+", default=list(ENGINES),
+                    choices=list(ENGINES),
+                    help="engines to run at this point; the reference is always "
+                         "included whether it is named or not")
+    ap.add_argument(
+        "--parity-only",
+        action="store_true",
+        help="check that the engines agree, and skip timing. The only mode that "
+             "says anything useful without a licence.",
+    )
+    ap.add_argument(
+        "--vectorbt",
+        choices=("run", "oos"),
+        default="run",
+        help="'oos' declares vectorbt out of scope for this point and runs the "
+             "others. For grids whose vectorbt side does not fit in memory on "
+             "any ordinary machine: not running it is the honest reading, since "
+             "the alternative is a number produced by swapping.",
+    )
+    ap.add_argument(
+        "--parity-anchor",
+        type=int,
+        default=250,
+        help="with --vectorbt oos, the grid size vectorbt DOES run, so the point "
+             "still carries a cross-engine check of the same code path.",
+    )
+    ap.add_argument(
+        "--memory-only",
+        choices=sorted(BY_CODE),
+        help="run ONE engine and report the memory its grid added. Memory has to "
+             "be measured this way: once any engine has run, the allocator has "
+             "grown and whatever runs second is measured against a heap it can "
+             "reuse. Measured in the same process, the second engine reads about "
+             "half its true peak.",
+    )
+    return ap.parse_args()
+
+
+def _main(args) -> int:
+    import data as data_mod
+
+    fast_vals, slow_vals = grid(args.combos)
+    actual = len(fast_vals) * len(slow_vals)
+    df = data_mod.make_ohlcv(args.bars)
+    workdir = _scratch("mbt-sweep-")
+
+    tier_before = tier()
+
+    if args.memory_only:
+        # What is wanted here is the PEAK of running this grid once, because that
+        # is what a machine has to survive: a box that cannot hold the grid does
+        # not return a slow number, it returns no number. Running the grid twice
+        # and measuring the second call would answer a different question (what a
+        # repeat costs in a warm process: 4.9 MB against 1.4 GB, measured), and
+        # that number tells a reader nothing about whether the job fits.
+        #
+        # But a cold first call would also charge vectorbt its numba compilation,
+        # a one-off that has nothing to do with grid size. So the engine is warmed
+        # on a 2x2 grid -- enough to compile, too small to pre-allocate the real
+        # one -- and the baseline is taken after that.
+        name = BY_CODE[args.memory_only].name
+        warm = builder(name, df, fast_vals[:2], slow_vals[:2],
+                       os.path.join(workdir, "warm"), args.metrics)
+        call = builder(name, df, fast_vals, slow_vals, workdir, args.metrics)
+        warm()
+        base = _mem_mb()
+        watch = _Peak()
+        watch.start()
+        call()
+        peak = watch.stop()
+        print(json.dumps({
+            "bars": args.bars,
+            "combos": actual,
+            "engine": name,
+            "baseline_mb": round(base, 1),
+            "added_mb": round(peak - base, 1),
+            "tier_before": tier_before,
+            "tier_after": tier(),
+        }))
+        return 0
+
+    # An unlicensed sweep CANNOT be timed, and this is not a matter of degree.
+    # Every accepted fan-out call waits out a fixed interval before any work
+    # starts, so the stopwatch measures the pause, not the engine: a 100-cell
+    # grid on 20k bars measured 5.00 s against vectorbt's 0.17 s here, which
+    # would publish "vectorbt is 29x faster" from a run where the engine did
+    # almost nothing. Timing is therefore refused outright without a licence,
+    # rather than gated on grid size.
+    #
+    # Checking tier_before against tier_after is not enough on its own: a run
+    # that starts AND finishes unlicensed shows no change at all, and would sail
+    # through that comparison carrying a meaningless ratio.
+    if not tier_before["pro"] and not args.parity_only:
+        print(json.dumps({
+            "bars": args.bars,
+            "combos": actual,
+            "status": "skipped",
+            "reason": (
+                "sweep timing requires a licence: unlicensed fan-out calls wait "
+                "out a fixed interval, so the measurement would be of that wait. "
+                "Re-run with --parity-only to check agreement without timing."
+            ),
+            "tier_before": tier_before,
+        }))
+        return 2
+
+    # Parity-only still has to fit the unlicensed allowance, which is spent
+    # across the whole process: one warmup call per engine and nothing more.
+    if args.parity_only and not tier_before["pro"] and actual > tier_before["budget_limit"]:
+        print(json.dumps({
+            "bars": args.bars,
+            "combos": actual,
+            "status": "skipped",
+            "reason": (
+                f"{actual} combinations exceeds the unlicensed allowance of "
+                f"{tier_before['budget_limit']} for a single call"
+            ),
+            "tier_before": tier_before,
+        }))
+        return 2
+
+    running = [REFERENCE] + [n for n in args.engines if n != REFERENCE]
+    out_of_scope = {}
+    if args.vectorbt == "oos" and OOS_ENGINE in running:
+        running.remove(OOS_ENGINE)
+        out_of_scope[OOS_ENGINE] = {"status": "out of scope", "reason": OOS_REASON}
+
+    calls = {
+        name: builder(name, df, fast_vals, slow_vals,
+                      os.path.join(workdir, name), args.metrics)
+        for name in running
+    }
+
+    result = {
+        "bars": args.bars,
+        "combos": actual,
+        "requested_combos": args.combos,
+        "grid": {"fast": len(fast_vals), "slow": len(slow_vals)},
+        "data_digest": data_mod.digest(df),
+        "engines": running,
+        "metrics": args.metrics,
+        "tier_before": tier_before,
+    }
+    if out_of_scope:
+        result["out_of_scope"] = out_of_scope
+
+    # Warmup, discarded: it is where vectorbt compiles its numba kernels, and
+    # charging that to every repetition would inflate the result. The warm call
+    # is what the gate reads, so nothing is computed twice for it.
+    warm = {name: call() for name, call in calls.items()}
+    verdicts = {
+        name: compare(warm[REFERENCE], out)
+        for name, out in warm.items() if name != REFERENCE
+    }
+
+    for name, detail in out_of_scope.items():
+        # The engine is not run at this size, so the point cannot carry a
+        # cross-engine check against it on its own grid. It carries one of the
+        # same code path at a size that engine can hold: same data, same
+        # strategy, same adapters, fewer cells. That is weaker than checking the
+        # grid itself, and the report says so rather than implying otherwise.
+        a_fast, a_slow = grid(args.parity_anchor)
+        anchor = compare(
+            builder(REFERENCE, df, a_fast, a_slow,
+                    os.path.join(workdir, "anchor"), args.metrics)(),
+            builder(name, df, a_fast, a_slow, workdir, args.metrics)(),
+        )
+        verdicts[name] = {
+            **anchor,
+            "checked_at_combos": len(a_fast) * len(a_slow),
+            "scope": "anchor grid, not this grid",
+        }
+        detail["parity_scope"] = "anchor grid"
+
+    result["parity"] = verdicts
+    result["status"] = "failed" if any(
+        v["status"] == "failed" for v in verdicts.values()) else "exact"
+
+    if result["status"] == "failed":
+        # No timing for a grid the engines disagree on. That is the whole point
+        # of the gate, and a sweep is where it earns its keep.
+        result["timings"] = None
+        result["note"] = "timing withheld: unexplained disagreement between engines"
+        result["tier_after"] = tier()
+        print(json.dumps(result))
+        return 1
+
+    samples = {name: [] for name in running}
+    peaks = {name: 0.0 for name in running}
+    base_mb = _mem_mb()
+    for _ in range(args.reps):
+        # Interleaved, so a runner that slows down mid-point penalises everyone.
+        # Sampled per call rather than across the whole loop: a single peak over
+        # every engine is the max of them, attributed to none, and memory is the
+        # number that decides which grid sizes a machine can run at all.
+        for name, call in calls.items():
+            watch = _Peak()
+            watch.start()
+            t0 = time.perf_counter()
+            call()
+            elapsed = time.perf_counter() - t0
+            peaks[name] = max(peaks[name], watch.stop())
+            samples[name].append(elapsed)
+
+    tier_after = tier()
+
+    def med(xs):
+        return sorted(xs)[len(xs) // 2]
+
+    seconds = {name: med(samples[name]) for name in running}
+    reference_s = max(1e-12, seconds[REFERENCE])
+    result["timings"] = {
+        "seconds": {**seconds, **{name: None for name in out_of_scope}},
+        "ratio": {name: seconds[name] / reference_s
+                  for name in running if name != REFERENCE},
+        "reps": args.reps,
+        "per_combo_us": seconds[REFERENCE] / actual * 1e6,
+    }
+    # Added by the call, on top of an already-built store / already-prepared
+    # Series: what running the grid costs, not what holding the data costs.
+    result["memory"] = {name: round(peaks[name] - base_mb, 1) for name in running}
+    result["memory_baseline_mb"] = round(base_mb, 1)
+    result["tier_after"] = tier_after
+
+    # A sweep measured across a tier change is not a measurement. The downgrade
+    # arrives from a background thread after a refused licence ping, so this can
+    # only be checked after the fact.
+    if tier_before["pro"] != tier_after["pro"]:
+        result["timings"] = None
+        result["note"] = (
+            f"timing withheld: tier changed mid-point "
+            f"(pro={tier_before['pro']} -> {tier_after['pro']})"
+        )
+        print(json.dumps(result))
+        return 1
+
+    print(json.dumps(result))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/vs_vectorbt/workloads.py

@@ -1,18 +1,25 @@
-"""Workload definitions: the numbers both engines read, in one place.
+"""Workload definitions: the numbers every engine reads, in one place.
 
-Nothing here is engine-specific. Each adapter (``engine_mbt``, ``engine_vbt``)
-reads the same constants, so a workload cannot drift between the two sides by
-someone editing one file and forgetting the other.
+Nothing here is engine-specific. Each adapter (``engine_mbt``, ``engine_vbt``,
+``engine_rbt``) reads the same constants, so a workload cannot drift between two
+sides by someone editing one file and forgetting another.
+
+What a workload *does* carry per engine is a note: the places where an engine is
+known in advance to disagree, or is unable to run the workload at all. Those are
+written down here, next to the parameters they apply to, rather than in the
+adapters, so a reader can see the whole map of "who runs what, and where they
+part ways" without opening three files.
 
 Sizing policy, and why it changes when fees are on
 --------------------------------------------------
-With ``FractionOfEquity`` sizing and a non-zero fee, the two engines size a
-position differently: manifoldbt charges the fee on top of a full-equity
-notional, vectorbt reserves it out of cash first. Both are defensible product
-decisions, and comparing them would compare *policy*, not speed or correctness.
-The fee workload therefore sizes in fixed units, which isolates the fee
-arithmetic itself. This mirrors the choice already made in the cross-engine
-parity suite shipped with the library.
+With ``FractionOfEquity`` sizing and a non-zero fee, engines size a position
+differently: manifoldbt charges the fee on top of a full-equity notional,
+vectorbt reserves it out of cash first. Both are defensible product decisions,
+and comparing them would compare *policy*, not speed or correctness. The fee
+workload therefore sizes in fixed units, which isolates the fee arithmetic
+itself. This mirrors the choice already made in the cross-engine parity suite
+shipped with the library. It is also the reason raptorbt sits that workload out:
+it has no fixed-quantity sizing to offer (see its note below).
 """
 from __future__ import annotations
 
@@ -27,18 +34,35 @@ CAPITAL = 100_000.0
 FREQ = "1min"
 
 
+@dataclass(frozen=True)
+class Note:
+    """What is known in advance about one engine on one workload.
+
+    ``documented``
+        The engine is known to disagree with the reference, and ``why`` says on
+        what. It still gets timed, but in the annex, never in the headline
+        table, and always with the reason printed next to the number.
+    ``unsupported``
+        The engine cannot express this workload at all. It is not run, and the
+        report says so rather than leaving a blank a reader would read as a
+        loss. An unsupported entry is a finding about the engine's API, so
+        ``why`` has to be specific enough to be checked.
+    """
+
+    status: str
+    why: str
+
+
 @dataclass(frozen=True)
 class Workload:
     key: str
     title: str
     why: str
     params: Dict[str, Any] = field(default_factory=dict)
-    # "exact": the engines must agree to float-reordering noise, or the timing is
-    # not published. "documented": they are known to disagree for a reason
-    # written down in `divergence`; the timing goes to the annex, never to the
-    # headline table.
-    parity: str = "exact"
-    divergence: str = ""
+    # Engine name -> Note. An engine with no entry here is expected to agree
+    # with the reference down to float-reordering noise, and a disagreement is
+    # a failure that withholds the timing.
+    notes: Dict[str, Note] = field(default_factory=dict)
 
 
 WORKLOADS: Dict[str, Workload] = {
@@ -63,6 +87,26 @@ WORKLOADS: Dict[str, Workload] = {
                 "engines on a wiped-out account compares rounding noise.",
             params=dict(fast=12, slow=26, rsi_period=14, rsi_lo=30.0, rsi_hi=70.0,
                         units=5.0, fee_bps=5.0),
+            notes={
+                "raptorbt": Note(
+                    "unsupported",
+                    "Two blockers, either one sufficient. Sizing: raptorbt has no "
+                    "fixed-quantity mode. `position_sizes` is a fraction of equity "
+                    "(measured: 0.5 buys exactly half the equity of the bar before "
+                    "the entry), `lot_size` rounds a computed size down to a "
+                    "multiple, and `alloted_capital` fixes the notional, not the "
+                    "quantity. Reproducing `units=5` would mean feeding a fraction "
+                    "derived from an equity curve that does not exist until the run "
+                    "is over. Indicator: `raptorbt.ema` seeds on a simple mean of "
+                    "the first `period` bars and emits from bar `period-1`, while "
+                    "manifoldbt seeds on the first observation and emits from bar 0 "
+                    "(measured: 11 leading NaN for span 12, and ema[11] equal to "
+                    "sma(12)[11] to the last bit). The two are the same recursion "
+                    "with a different warmup, so the signal differs early and the "
+                    "round-trip count with it. Its `sma` and `rsi` do match, which "
+                    "is why the other three workloads run.",
+                ),
+            },
         ),
         Workload(
             key="sma_cross_metrics",
@@ -85,22 +129,60 @@ WORKLOADS: Dict[str, Workload] = {
                 "level, the fill on the triggering bar, and whether a re-entry "
                 "is allowed on the bar after an exit.",
             params=dict(fast=10, slow=50, alloc=1.0, sl_pct=0.15, tp_pct=0.30),
-            parity="documented",
-            divergence=(
-                "Re-entry on the exit bar. When a bracket fires intrabar and the "
-                "entry condition still holds at that bar's close, manifoldbt books "
-                "two orders on that bar (the stop or target exit, then a fresh "
-                "entry at the close); vectorbt processes one order per bar and "
-                "re-enters on the next bar instead. Neither is wrong, and on "
-                "controlled bars the bracket fills themselves match exactly (see "
-                "the cross-engine parity suite shipped with the library). The "
-                "harness counts the affected round-trips so the size of the "
-                "divergence is measured, not asserted."
-            ),
+            notes={
+                "vectorbt": Note(
+                    "documented",
+                    "Re-entry on the exit bar. When a bracket fires intrabar and "
+                    "the entry condition still holds at that bar's close, "
+                    "manifoldbt books two orders on that bar (the stop or target "
+                    "exit, then a fresh entry at the close); vectorbt processes one "
+                    "order per bar and re-enters on the next bar instead. Neither "
+                    "is wrong, and on controlled bars the bracket fills themselves "
+                    "match exactly (see the cross-engine parity suite shipped with "
+                    "the library). The harness counts the affected round-trips so "
+                    "the size of the divergence is measured, not asserted.",
+                ),
+                "raptorbt": Note(
+                    "documented",
+                    "Same fork in the road, taken further: raptorbt does not "
+                    "re-arm at all. Once a bracket closes a position, the entry "
+                    "level being still true is not enough to open another one; it "
+                    "waits for the level to go false and true again. So on the same "
+                    "bars the three engines book a different number of round-trips "
+                    "from the same signal, manifoldbt re-entering on the exit bar, "
+                    "vectorbt on the next bar, raptorbt not until the next crossing. "
+                    "The count of affected round-trips is measured rather than "
+                    "asserted, and it is the same population for both challengers.",
+                ),
+            },
         ),
     )
 }
 
+def expectation(key: str, engine: str) -> str:
+    """What this engine is expected to do on this workload, before it runs."""
+    note = WORKLOADS[key].notes.get(engine)
+    return note.status if note else "exact"
+
+
+def why(key: str, engine: str) -> str:
+    note = WORKLOADS[key].notes.get(engine)
+    return note.why if note else ""
+
+
+def supported(key: str, engine: str) -> bool:
+    return expectation(key, engine) != "unsupported"
+
+
+def unsupported_by(key: str) -> Dict[str, str]:
+    """Engines that sit this workload out, and the reason each one gives."""
+    return {
+        name: note.why
+        for name, note in WORKLOADS[key].notes.items()
+        if note.status == "unsupported"
+    }
+
+
 DEFAULT_KEYS = list(WORKLOADS)
 
 # Two workloads that the report compares directly against each other, so they