mirror of
https://github.com/manifoldbt/manifoldbt.git
synced 2026-08-24 22:48:05 +00:00
Exocet92andGitHub
52cbe1ba54
The harness compared two engines everywhere; it now compares N against a reference. manifoldbt is the reference: every parity check and every ratio is a challenger against it, never two challengers against each other. raptorbt 0.9.0 joins on three of the four workloads. Its sma_cross comes back bit-identical to the reference's final equity, and its rsi matches to the last bit; its ema seeds on a different warmup and it has no fixed-quantity sizing, so the fee workload records it as unsupported with the reason rather than leaving a blank cell. On the bracket it diverges in its own documented way: it never re-arms while the entry level holds, so it books exactly the reference's round-trips minus the ones that re-enter on the exit bar. Python moves to 3.12, which raptorbt pins rather than we do: it is built against pyo3 0.20.3, whose maximum supported CPython is 3.12. Timings from runs before this change are therefore not directly comparable. The bar matrix gains 10M and the repetition default drops from 7 to 2. Measured, those two almost cancel: the job stays around 16 minutes. macOS keeps its old ceiling, since 10M bars adds 1.55 GB on vectorbt's side alone and that runner has 7 GB.
82 lines
2.9 KiB
Python
82 lines
2.9 KiB
Python
"""The engine registry: who is in the comparison, and what each one can do.
|
|
|
|
One backtester is the *reference* and the others are *challengers*. Every parity
|
|
check joins a challenger to the reference, never two challengers to each other:
|
|
with three engines there are three pairs, and reporting all of them turns a
|
|
speed benchmark into a matrix nobody reads. manifoldbt is the reference because
|
|
this harness ships with it, which is a statement about the plumbing and not a
|
|
claim about the engine: the reference gets no advantage from the position, it is
|
|
simply the one every timing is divided by.
|
|
|
|
A challenger that is not installed is skipped with a printed line rather than
|
|
crashing the run. The CI lock file pins all of them, so a skip on a runner is a
|
|
finding; a skip on a laptop is somebody who did not want to install a competitor
|
|
to read their own numbers.
|
|
|
|
Ratio basis
|
|
-----------
|
|
``ratio_basis`` records how an engine annualises Sharpe and Sortino. manifoldbt
|
|
and the vectorbt adapter both compute them on daily returns annualised by
|
|
sqrt(365), so their ratios are directly comparable and small differences are
|
|
worth reporting. raptorbt returns its own, on its own basis, and comparing those
|
|
numbers to manifoldbt's would report a units mismatch as a disagreement. The
|
|
parity gate never depends on this either way: it gates on money, round-trips and
|
|
drawdown, all three basis-free.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import importlib
|
|
from dataclasses import dataclass
|
|
from typing import Dict, List
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class Engine:
|
|
name: str
|
|
# Short code used by the out-of-process probes, which take argv strings.
|
|
code: str
|
|
module: str
|
|
# "daily_365" ratios are comparable with the reference's; "native" are not.
|
|
ratio_basis: str
|
|
# PyPI distribution name, for the version stamp in the result envelope.
|
|
distribution: str
|
|
|
|
|
|
REFERENCE = "manifoldbt"
|
|
|
|
ENGINES: Dict[str, Engine] = {
|
|
e.name: e
|
|
for e in (
|
|
Engine("manifoldbt", "mbt", "engine_mbt", "daily_365", "manifoldbt"),
|
|
Engine("vectorbt", "vbt", "engine_vbt", "daily_365", "vectorbt"),
|
|
Engine("raptorbt", "rbt", "engine_rbt", "native", "raptorbt"),
|
|
)
|
|
}
|
|
|
|
CHALLENGERS: List[str] = [n for n in ENGINES if n != REFERENCE]
|
|
|
|
BY_CODE: Dict[str, Engine] = {e.code: e for e in ENGINES.values()}
|
|
|
|
|
|
def adapter(name: str):
|
|
"""Import an adapter module on demand.
|
|
|
|
Lazy on purpose: the cold-start probe measures a process that has imported
|
|
exactly one engine, and a registry that imported all three at module scope
|
|
would make that measurement impossible to take.
|
|
"""
|
|
return importlib.import_module(ENGINES[name].module)
|
|
|
|
|
|
def installed(name: str) -> bool:
|
|
try:
|
|
adapter(name)
|
|
return True
|
|
except ImportError:
|
|
return False
|
|
|
|
|
|
def present(names: List[str]) -> List[str]:
|
|
"""Filter to the engines actually importable here, preserving order."""
|
|
return [n for n in names if installed(n)]
|