mirror of
https://github.com/manifoldbt/manifoldbt.git
synced 2026-11-02 07:56:11 +00:00
Jimmy7892
7e81ffe0d6
New directory benchmarks/vs_hft. The rules are written once (spec.py) and each engine implements them in its own idiom: a two-sided maker on 25 book levels and on best bid/ask, a flow taker on the trade clock, a maker that joins the best on an order-by-order feed, and an open-loop probe whose quotes depend on the market only, so that each one gets a verdict that does not depend on what the engine did before. The data is public and pinned. One Bybit spot BTCUSDT day (book and trades) and the Nasdaq TotalView-ITCH half session of 28 November 2025 (one symbol), each archive checked against a SHA-256 before anything runs. Parity comes before timing, and it is not read from fill counts: an engine that serves an order in pieces shows more fills for the same quantity. It is read as actions (which quotes were sent), verdicts (served or not, for how much, how late) and totals, with manifoldbt as the reference and the same exact / documented / failed statuses as the vectorbt benchmark. On the order-by-order day every engine is also judged against an exact replay of the raw messages (fifo_truth.py), so none of them has to be trusted as the reference. The README has a section, "Why the engines differ", that names each convention behind a difference in the results and gives its figure for each engine. Parameter sweeps are included: manifoldbt through run_sweep and run_sweep_lite, hftbacktest and NautilusTrader as the loop and the pool of processes a user would write. The result of a pool is checked against the loop, and a sample of sweep combinations against the same combination run alone. journal_of keeps the instant of a first fill as an integer. A nanosecond epoch in a float64 is only exact to 256 ns, and the exact-replay judgment compares those instants with the order of the raw messages, so the column never goes through a float. test_journal.py checks it to the nanosecond. The lock file pins the engines and their dependencies; every pinned version has a manylinux wheel for CPython 3.12, and manifoldbt resolves together with it.
28 lines
1.8 KiB
Python
28 lines
1.8 KiB
Python
"""Workloads: which strategy, on which data tier, under which venue settings.
|
|
|
|
A workload is engine-neutral. `latency` is one of:
|
|
nominal order 5 ms, cancel 5 ms, response 5 ms, feed 2 ms (the taker: order 20 ms only)
|
|
zero every delay 0 (the strictest parity setting)
|
|
"""
|
|
import spec
|
|
|
|
MM_LADDER = dict(clip=spec.MMLadder.CLIP, max_clips=spec.MMLadder.MAX_CLIPS, off_bid=0, off_ask=0, ladder=True)
|
|
MM_TOUCH = dict(clip=spec.MMTouch.CLIP, max_clips=spec.MMTouch.MAX_CLIPS, off_bid=0, off_ask=0, ladder=False)
|
|
PROBE = dict(clip=spec.MMLadder.CLIP, max_clips=spec.MMLadder.MAX_CLIPS, off_bid=0, off_ask=0, ladder=False,
|
|
probe=True)
|
|
TAKER = dict(window_ns=spec.FlowTaker.WINDOW_NS, threshold=spec.FlowTaker.THRESHOLD, qty=spec.FlowTaker.QTY)
|
|
|
|
WORKLOADS = {
|
|
"mm_ladder_l2": dict(kind="mm", tier="l2", params=MM_LADDER, latency="nominal",
|
|
doc="two post-only quotes, one level back per clip held (example 31), 25 levels"),
|
|
"mm_touch_bbo": dict(kind="mm", tier="bbo", params=MM_TOUCH, latency="nominal",
|
|
doc="the same maker on a best-bid/ask feed: it can only join the touch"),
|
|
"flow_taker_bbo": dict(kind="taker", tier="bbo", params=TAKER, latency="nominal",
|
|
doc="aggressor-imbalance taker on the trade clock (example 33), market orders"),
|
|
# open-loop probe: parity of the queue / fill model, short window
|
|
"probe_l2_zero": dict(kind="mm", tier="l2", params=dict(PROBE, ladder=False), latency="zero",
|
|
doc="one quote a side at the touch re-posted at every wake-up, no inventory", short=True),
|
|
"probe_l2_nominal": dict(kind="mm", tier="l2", params=dict(PROBE, ladder=False), latency="nominal",
|
|
doc="the same probe under the 5/5/5/2 ms venue", short=True),
|
|
}
|