mirror of
https://github.com/manifoldbt/manifoldbt.git
synced 2026-08-24 22:48:05 +00:00
bench: add raptorbt as a third engine, and a 10M-bar point (#6)
The harness compared two engines everywhere; it now compares N against a reference. manifoldbt is the reference: every parity check and every ratio is a challenger against it, never two challengers against each other. raptorbt 0.9.0 joins on three of the four workloads. Its sma_cross comes back bit-identical to the reference's final equity, and its rsi matches to the last bit; its ema seeds on a different warmup and it has no fixed-quantity sizing, so the fee workload records it as unsupported with the reason rather than leaving a blank cell. On the bracket it diverges in its own documented way: it never re-arms while the entry level holds, so it books exactly the reference's round-trips minus the ones that re-enter on the exit bar. Python moves to 3.12, which raptorbt pins rather than we do: it is built against pyo3 0.20.3, whose maximum supported CPython is 3.12. Timings from runs before this change are therefore not directly comparable. The bar matrix gains 10M and the repetition default drops from 7 to 2. Measured, those two almost cancel: the job stays around 16 minutes. macOS keeps its old ceiling, since 10M bars adds 1.55 GB on vectorbt's side alone and that runner has 7 GB.
This commit is contained in:
@@ -0,0 +1,57 @@
|
||||
"""Activate a licence from the environment, then assert the tier out loud.
|
||||
|
||||
Used by the sweep workflow. A sweep benchmark run without a licence does not
|
||||
fail, it produces a wrong number: every unlicensed fan-out call waits out a
|
||||
fixed interval before doing any work, so the stopwatch measures the wait. On a
|
||||
100-cell grid that read 5.00 s against vectorbt's 0.17 s, which would publish
|
||||
"vectorbt is 29x faster" from a run where the engine barely ran.
|
||||
|
||||
So this exits non-zero rather than let the benchmark continue unlicensed.
|
||||
|
||||
The key is written to disk by `activate`, so every child process the harness
|
||||
spawns afterwards picks it up without seeing the secret itself.
|
||||
|
||||
MANIFOLDBT_CI_LICENSE=<signed key> python ci_activate.py
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import manifoldbt as mbt
|
||||
|
||||
# The American spelling, because that is how the repository secret is actually
|
||||
# named. Both are accepted: the two spellings are one typo apart, and the cost
|
||||
# of getting it wrong is a benchmark job that dies at the first step.
|
||||
VARS = ("MANIFOLDBT_CI_LICENSE", "MANIFOLDBT_CI_LICENCE")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
key = ""
|
||||
for var in VARS:
|
||||
key = (os.environ.get(var) or "").strip()
|
||||
if key:
|
||||
break
|
||||
if not key:
|
||||
print(" and ".join(VARS) + " are empty or unset: "
|
||||
"refusing to benchmark sweeps unlicensed.")
|
||||
return 1
|
||||
|
||||
# Never print the key or the activation message: the message carries the
|
||||
# licensee's address, and a public run log is not the place for it.
|
||||
try:
|
||||
mbt.activate(key)
|
||||
except Exception as exc: # noqa: BLE001 - reported, not raised
|
||||
print(f"activation failed: {type(exc).__name__}")
|
||||
return 1
|
||||
|
||||
_used, _limit, is_pro = mbt._native._combo_budget()
|
||||
if not is_pro:
|
||||
print("activation did not yield a licensed tier: refusing to continue.")
|
||||
return 1
|
||||
print("licensed tier active")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user