gavindiaz/manifoldbt
1
0
Fork 0
mirror of https://github.com/manifoldbt/manifoldbt.git synced 2026-11-02 07:56:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

bench: do not claim a parity check that gates the report

Browse Source 
The introduction and the docstring of bench.py said that the engines are checked for doing the same work and only then timed. The harness reports how far they did the same work next to every timing and does not hold a timing back, as the previous commit says in the README. The wording now matches.
This commit is contained in:
Jimmy7892
2026-10-23 23:06:58 +02:00
parent 10ae85205a
commit ae8c7ce291
2 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/vs_hft/README.md
+2 -2
View File
@@ -2,8 +2,8 @@
An engine-to-engine benchmark you can re-run yourself. It installs every engine
from PyPI at a pinned version, downloads one pinned day of public market data
(checked against a SHA-256), runs the same trading rules in the three engines,
checks that they did **the same work**, and only then reports how long each took.
(checked against a SHA-256), runs the same trading rules in the three engines, and
reports, beside how long each took, how far they did **the same work**.
```bash
pip install manifoldbt
benchmarks/vs_hft/bench.py
+2 -2
View File
@@ -3,8 +3,8 @@
python bench.py --day 2025-08-23 --reps 3 --out results.json
Prepares one pinned day of public data, runs every workload in every engine (one
fresh process per run, engines interleaved inside each repetition), checks that
the engines did the same work, and only then reports how long each took.
fresh process per run, engines interleaved inside each repetition), and reports how
long each took together with how far the engines did the same work.
"""
import argparse
import hashlib