gavindiaz/manifoldbt
1
0
Fork 0
mirror of https://github.com/manifoldbt/manifoldbt.git synced 2026-11-02 07:56:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

release: v0.27.2

Browse Source
This commit is contained in:
github-actions[bot]
2026-10-16 20:49:21 +00:00
parent 9e7fad2168
commit 270da1d8ef
14 changed files with 1313 additions and 394 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+390 -319
View File
@@ -2,329 +2,269 @@
<img src="https://raw.githubusercontent.com/manifoldbt/manifoldbt/master/assets/logo.png" width="110" alt="Manifold-BT logo">
</p>
<p align="center">
<strong>Manifold-BT</strong><br>
Rust-powered backtesting engine for quantitative research
</p>
<h1 align="center">Manifold-BT</h1>
<p align="center">
<a href="https://discord.gg/bvU6Wjc72d"><img src="https://img.shields.io/badge/Discord-Join%20the%20community-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join the Manifold-BT Discord" height="34"></a>
<strong>Python backtesting with a Rust engine. Fast enough to sweep, strict about fills.</strong>
</p>
<p align="center">
<a href="https://pypi.org/project/manifoldbt/"><img src="https://img.shields.io/pypi/v/manifoldbt?logo=pypi&logoColor=white&color=2f6fed" alt="PyPI"></a>
<img src="https://img.shields.io/badge/python-3.9%2B-3776AB?logo=python&logoColor=white" alt="Python 3.9+">
<img src="https://img.shields.io/badge/core-Rust-dea584?logo=rust&logoColor=white" alt="Rust core">
<img src="https://img.shields.io/badge/python-3.9%E2%80%933.13-3776AB?logo=python&logoColor=white" alt="Python 3.9 to 3.13">
<a href="https://github.com/manifoldbt/manifoldbt/actions/workflows/ci.yml"><img src="https://github.com/manifoldbt/manifoldbt/actions/workflows/ci.yml/badge.svg?branch=master" alt="Tests against the PyPI wheel"></a>
<a href="https://github.com/manifoldbt/manifoldbt/actions/workflows/bench-vs-vectorbt.yml"><img src="https://img.shields.io/badge/benchmarks-public%20CI-2ea44f?logo=githubactions&logoColor=white" alt="Benchmarks in public CI"></a>
<a href="https://discord.gg/bvU6Wjc72d"><img src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white" alt="Discord"></a>
</p>
<p align="center">
<a href="https://www.manifoldbt.com">Website</a> &middot;
<a href="https://www.manifoldbt.com/docs/documentation.html">Documentation</a> &middot;
<a href="https://github.com/manifoldbt/manifoldbt/tree/master/examples">Examples</a>
<a href="https://www.manifoldbt.com/documentation">Documentation</a> &middot;
<a href="#quick-start">Quick start</a> &middot;
<a href="#performance">Benchmarks</a> &middot;
<a href="#correctness">Correctness</a> &middot;
<a href="https://github.com/manifoldbt/manifoldbt/tree/master/examples">Examples</a> &middot;
<a href="https://www.manifoldbt.com">Website</a>
</p>
---
Manifold-BT is a Python backtesting library with a Rust core. You declare a strategy in a
fluent Python DSL and hand it to the engine once: the whole backtest then runs in Rust,
signals, fills, fees, slippage, funding and equity included, with look-ahead protection.
Python stays out of the hot loop.
**Vectorized speed with event-driven execution realism.**
Manifold-BT is a backtesting library for quantitative research. You describe a
strategy once, in a fluent Python DSL, and hand it to the engine: the whole
backtest then runs in Rust, signals, fills, fees, slippage, funding and equity
included. Python stays out of the hot loop.
## Why Manifold-BT
- **Fast**: 10M bars in 329 ms. 79x faster than vectorbt, and 311x once you also want drawdown and Sharpe. [Measured in public CI](#performance), every run linked.
- **Expressive**: fluent DSL with 104 indicators and 38 candlestick patterns, conditional logic, cross-asset references
- **Rigorous**: Monte Carlo, walk-forward, parameter sweeps, lookahead detection, exposure diagnostics
- **Portable**: `pip install`, no Rust toolchain needed. Works on Python 3.9+.
## Installation
- **Fast.** 10 million bars with a full performance summary in **337 ms**,
where vectorbt takes 85.6 s. 10,000 parameter sets over 200,000 bars in
**9.3 s inside 104 MB**. Measured on a standard 4-vCPU GitHub runner,
[every run linked](#performance).
- **Checked.** Entry price, exit price and exit reason are asserted against
vectorbt in CI, on the wheel published to PyPI. A benchmark workload the
engines disagree on gets no timing at all.
- **Explicit.** Every fill rule is [written down](#execution-semantics),
pessimistic wherever a choice existed, and so is
[what the engine does not model](#what-is-not-covered).
```bash
pip install manifoldbt # engine only: backtests, sweeps, metrics
pip install manifoldbt[plot] # + interactive charts and native windows (show=True)
pip install manifoldbt[all] # everything: plots, windows, PNG export, pandas/polars
pip install manifoldbt[gpu] # + NVIDIA runtime compiler, for device="cuda" (Pro)
pip install manifoldbt
```
The base install stays light (no browser, no GUI) for scripts, servers and CI.
`[plot]` adds plotly and a native window backend; `[all]` also pulls kaleido for
static PNG/SVG export (which bundles a headless Chromium).
Wheels for Linux, Windows and macOS. No Rust toolchain, no compiler, no account.
The Linux and Windows x86_64 wheels already carry the CUDA kernels, so `[gpu]`
only adds the NVIDIA runtime compiler (~180 MB) that compiles them on your
machine. Skip it if you already have a CUDA toolkit installed. An NVIDIA driver
is required, and GPU acceleration is a Pro feature; everything else runs at full
speed on the CPU.
## Quick start
### Staying up to date
manifoldbt asks PyPI once a day, in the background, whether a newer release
exists, and prints a one-line notice under the banner when one does. It never
delays an import (the notice is the previous run's answer, read from a local
cache) and it sends nothing: the request is a plain GET of a public JSON
document. Set `MANIFOLDBT_NO_UPDATE_CHECK=1` to turn it off, and
`mbt.check_for_update()` to ask on demand -- it returns the newer version, or
`None` when you are current.
## Quick Start
Two years of hourly Bitcoin, downloaded from Binance's public archive, and an
EMA crossover with perpetual-futures fees and slippage. It runs as-is on a
fresh install, in a couple of seconds including the download.
```python
import manifoldbt as mbt
from manifoldbt.indicators import close, ema
from manifoldbt.helpers import time_range, Interval, Slippage
fast = ema(close, 12)
slow = ema(close, 26)
# Two years of hourly BTCUSDT from Binance's public archive. No API key.
store = mbt.ingest(provider="binance", symbol="BTCUSDT", symbol_id=1, interval="1h",
start="2023-01-01T00:00:00Z", end="2025-01-01T00:00:00Z")
fast, slow = ema(close, 12), ema(close, 26)
strategy = (
mbt.Strategy.create("ema_crossover")
mbt.Strategy.create("ema_cross")
.signal("fast", fast)
.signal("slow", slow)
.signal("signal", mbt.when(fast > slow, mbt.lit(1.0), mbt.lit(-1.0)))
.size(mbt.col("signal") * mbt.lit(0.25))
.size(mbt.when(fast > slow, 0.5, -0.5)) # half the equity, long or short
)
start, end = time_range("2022-01-01", "2025-01-01")
start, end = time_range("2023-01-01", "2025-01-01")
config = mbt.BacktestConfig(
universe=[1],
time_range_start=start,
time_range_end=end,
bar_interval=Interval.hours(12),
initial_capital=10_000,
execution=mbt.ExecutionConfig(allow_short=True, max_position_pct=0.5),
universe=[1], time_range_start=start, time_range_end=end,
bar_interval=Interval.hours(1), initial_capital=10_000, warmup_bars=26,
execution=mbt.ExecutionConfig(allow_short=True),
fees=mbt.FeeConfig.binance_perps(),
slippage=Slippage.fixed_bps(2),
warmup_bars=30,
)
store = mbt.ingest(provider="binance", symbol="BTCUSDT", symbol_id=1,
start="2022-01-01T00:00:00Z", end="2025-01-01T00:00:00Z", interval="1h")
result = mbt.run(strategy, config, store)
print(result.summary())
```
## Loading data
Output, trimmed:
Bring your own data, or pull it from a built-in connector. Both return a
`DataStore` ready for `mbt.run(...)`.
**CSV**, free on all tiers, auto-detects standard / MetaTrader 4 / MetaTrader 5:
```python
store = mbt.import_csv("EURUSD_1m.csv", symbol="EURUSD", symbol_id=1,
interval="1m", asset_class="forex")
```text
Strategy: ema_cross
----------------------------------------
Total Return -0.63%
Sharpe 0.11
Max Drawdown -36.38%
...
Trades
--------------------------------------
Total 605
Total Fees 2807.07
```
**Market data connectors**: Binance, Bybit, Hyperliquid, dYdX, Bitstamp, Yahoo Finance, Dukascopy (free);
Databento, Massive (Pro):
A losing strategy, and that is the lesson. Remove the `fees=` and `slippage=`
lines and the same crossover returns **+51.8%**: 605 trades paid $2,807 on a
$10,000 account. Costs belong in the first run, not the last.
### Sweep a grid, read the surface
Replace a number with `mbt.param(...)` and the strategy becomes a grid. One
call runs every combination in parallel:
```python
store = mbt.ingest(provider="binance", symbol="BTCUSDT", symbol_id=1,
start="2024-01-01T00:00:00Z", end="2025-01-01T00:00:00Z")
```
Yahoo Finance covers stocks, ETFs, indices (`^GSPC`), FX (`EURUSD=X`), futures
(`ES=F`) and crypto (`BTC-USD`) without an API key. Prices are dividend-adjusted,
like `yfinance`'s `auto_adjust=True`; pass `dataset="raw"` for unadjusted quotes.
Yahoo's own history limits apply: 1m over the last 30 days, 1h over ~2 years,
daily back to the listing date.
```python
store = mbt.ingest(provider="yahoo", symbol="AAPL", symbol_id=1, interval="1d",
asset_class="equity",
start="2015-01-01T00:00:00Z", end="2026-01-01T00:00:00Z")
```
Or from the CLI:
```bash
manifoldbt import-csv data.csv --symbol EURUSD --symbol-id 1 --interval 1m
manifoldbt ingest --provider binance --symbol BTCUSDT --symbol-id 1 --start ... --end ...
```
## Higher timeframes
Declare the timeframes you want alongside the simulation one, then read them
with `mbt.tf(...)`. Columns are forward-filled onto the simulation grid, and a
bar's value only becomes readable once that bar has closed, so there is no
look-ahead.
```python
config = mbt.BacktestConfig(
...,
bar_interval=Interval.minutes(1), # simulate on 1m
extra_timeframes={"1h": Interval.hours(1)}, # also resample to 1h
# to_df() needs pandas: pip install "manifoldbt[pandas]"
fast = ema(close, mbt.param("fast"))
slow = ema(close, mbt.param("slow"))
strategy = (
mbt.Strategy.create("ema_grid")
.signal("fast", fast)
.signal("slow", slow)
.size(mbt.when(fast > slow, 0.5, -0.5))
)
h1 = mbt.tf("1h")
h1.close # the last closed hourly close, held across the minute bars
sweep = mbt.run_sweep(strategy, {"fast": range(5, 55, 5), "slow": range(60, 260, 20)},
config, store)
surface = sweep.to_df(backend="pandas").pivot(
index="param_fast", columns="param_slow", values="sharpe")
```
For an **indicator** on a higher timeframe, use `.apply(...)`. It evaluates the
expression on that timeframe's own grid, so the period counts in *its* bars:
A hundred backtests come back as one table, a row per combination with its
parameters and its metrics. Read the shape rather than the maximum: a lone
peak in a flat field is usually a fitted accident, a broad plateau is a
candidate. Then take the candidate to [walk-forward](https://github.com/manifoldbt/manifoldbt/blob/master/examples/07_walk_forward.py)
and [Monte Carlo](https://github.com/manifoldbt/manifoldbt/blob/master/examples/10_monte_carlo.py).
With the `[plot]` extra, `sweep.plot_metric("sharpe")` draws the heatmap.
```python
from manifoldbt.indicators import close, sma
## Performance
band = mbt.tf("1h").apply(sma(close, 20)) # mean of 20 HOURLY closes
Every number in this section comes from a benchmark that runs in public CI on a
standard GitHub runner. It installs each engine from PyPI the way a user would,
generates its own data, checks that the engines produced the **same result**,
and only then reports how long each took.
```text
SMA crossover with drawdown, Sharpe, Sortino and volatility, 10M bars
Manifold-BT ▎ 0.34 s
raptorbt ▍ 0.74 s
vectorbt ████████████████████████████████████████████████ 85.6 s
Parameter sweep, 10,000 combinations over 200,000 bars
Manifold-BT ███ 9.3 s
raptorbt ████████████████████████████████████████████████ 146.4 s
vectorbt not run: it materialises every combination, this grid needs tens of GB
```
> Careful: `sma(mbt.tf("1h").close, 20)` is **not** the same thing. That reads
> the step-held hourly series on the simulation grid, so the period counts in
> simulation bars: on a 1m simulation it is a 20-*minute* smoothing of an hourly
> staircase. Use `.apply(...)` whenever you want an indicator *of* the higher
> timeframe.
**[Run 35590549822](https://github.com/manifoldbt/manifoldbt/actions/runs/35590549822)**:
Linux x86_64, 4 vCPU, Python 3.12, manifoldbt 0.26.0, vectorbt 0.28.4,
raptorbt 0.9.0. Seven interleaved repetitions (two for sweeps), medians
reported; the ratio is the median of the per-repetition ratios. All five rows
below agree with vectorbt exactly (relative tolerance 1e-9).
## Sweeping a choice, not just a number
| Workload | Bars | Manifold-BT | vectorbt | raptorbt |
|---|---:|---:|---:|---:|
| SMA 30/150 crossover | 10M | **341 ms** | 22.0 s (×64.5) | 759 ms (×2.2) |
| ...plus drawdown, Sharpe, Sortino, volatility | 10M | **337 ms** | 85.6 s (**×250**) | 743 ms (×2.2) |
| ...with a 5 bps fee and 2 bps slippage | 10M | **335 ms** | 21.4 s (×63.7) | not supported |
| EMA 12/26 + RSI(14) filter, 5 bps fee | 1M | **52 ms** | 1.64 s (×31.5) | not supported |
| Five assets in one book, 5 bps fee | 1M | **161 ms** | 2.06 s (×12.4) | not supported |
`mbt.param(...)` sweeps numbers. `mbt.choice(...)` sweeps *expressions*: the
selector becomes a grid axis, and each combination resolves to its branch before
the simulation runs, so the branches it did not pick cost nothing.
The second row is the one worth reading twice. A performance summary costs
Manifold-BT nothing measurable, because it computes one during the run whether
you read it or not. vectorbt defers the equity curve until a risk metric asks
for it, then has to build one, and that costs it over a minute on this data.
"Not supported" means raptorbt has no way to express the workload (fixed
quantity sizing, a multi-asset book); the benchmark prints the reason instead
of an empty cell.
```python
band = mbt.choice("band", {
"30m": mbt.tf("30m").apply(sma(close, mbt.param("len"))),
"1h": mbt.tf("1h").apply(sma(close, mbt.param("len"))),
"2h": mbt.tf("2h").apply(sma(close, mbt.param("len"))),
})
strategy = (mbt.Strategy.create("band_cross")
.signal("band", band)
.size(mbt.when(close > mbt.col("band"), 1.0, 0.0)))
config.extra_timeframes = {"30m": Interval.minutes(30), # every tf(...) a branch uses
"1h": Interval.hours(1), "2h": Interval.hours(2)}
### Parameter sweeps
sweep = mbt.run_sweep(strategy, {"band": ["30m", "1h", "2h"],
"len": range(10, 210, 10)}, config, store)
```
| Bars | Combinations | Manifold-BT | vectorbt | raptorbt |
|---:|---:|---:|---:|---:|
| 20,000 | 5,000 | **0.47 s**, 23 MB | 5.23 s, 2.5 GB | 6.65 s, 36 MB |
| 20,000 | 20,000 | **1.84 s**, 65 MB | not run | 26.8 s |
| 200,000 | 10,000 | **9.31 s**, 104 MB | not run | 146.4 s |
The branches can hold any expression, so the same mechanism sweeps which
exogenous column to use, which asset to reference, or which indicator to apply.
Past a certain grid the question stops being speed. vectorbt materialises the
simulation per combination, 1.57 MB of it at 20,000 bars, so the larger grids
would ask the runner for tens of gigabytes. Each sweep is checked for parity
before it is timed: every combination against raptorbt, and against vectorbt
on the full grid where it fits, or on a 250-combination anchor grid where it
does not. The sweep job runs with a Pro licence, since Community sweeps stop
at 256 combinations.
## Examples
### Where it does not win
Clone the repository and run one:
- **Against raptorbt the gap is about 2×, not orders of magnitude.** Both are
compiled engines; the large ratios in the tables are against vectorbt.
- **Multi-asset is the smallest lead over vectorbt (×12.4).** Broadcasting a
column per asset is close to free for vectorbt, while walking five books is
not free for anything.
- **On a stop-loss/take-profit bracket, raptorbt is level** (×1.0 at 1M bars,
×1.15 at 10M). Deciding which side triggers first inside a bar is a
sequential walk in both engines. That workload is also not in the table
above: the engines disagree on whether to re-enter on the bar a bracket
closes, so its timing sits in the benchmark's annex with the cause written
down.
```bash
python examples/13_stochastic_simulation.py
```
Reproduce any of it: fork the repository and press **Run workflow** on
[the benchmark](https://github.com/manifoldbt/manifoldbt/actions/workflows/bench-vs-vectorbt.yml),
or run [`benchmarks/vs_vectorbt/`](https://github.com/manifoldbt/manifoldbt/tree/master/benchmarks/vs_vectorbt)
locally. The method, the parity gate and the known divergences are in
[its README](https://github.com/manifoldbt/manifoldbt/blob/master/benchmarks/vs_vectorbt/README.md).
Each release re-runs it, and the latest results are on
[manifoldbt.com/benchmarks](https://www.manifoldbt.com/benchmarks).
Most of them backtest real market data, which is not in the repository.
`python examples/setup_data.py` downloads it once, from free connectors that
need no API key; [examples/README.md](https://github.com/manifoldbt/manifoldbt/blob/master/examples/README.md)
says which files need it and which run on nothing at all. Charts come from the
optional `[plot]` extra: without it an example still runs to the end and skips
its chart.
| # | Example | What it shows |
|---|---------|---------------|
| 00 | [Template](https://github.com/manifoldbt/manifoldbt/blob/master/examples/00_template.py) | Minimal starting point |
| 01 | [Trend Following](https://github.com/manifoldbt/manifoldbt/blob/master/examples/01_trend_following.py) | EMA crossover, volume filter, stop-loss |
| 02 | [Mean Reversion](https://github.com/manifoldbt/manifoldbt/blob/master/examples/02_mean_reversion.py) | EMA crossover with parameter sweep |
| 03 | [Multi-Asset Momentum](https://github.com/manifoldbt/manifoldbt/blob/master/examples/03_multi_asset_momentum.py) | Cross-asset signals |
| 04 | [Linear Regression](https://github.com/manifoldbt/manifoldbt/blob/master/examples/04_linear_regression.py) | Regression-based signal |
| 05 | [Statistical Arbitrage](https://github.com/manifoldbt/manifoldbt/blob/master/examples/05_stat_arb.py) | Pairs trading, spread z-score |
| 06 | [Full Visualization](https://github.com/manifoldbt/manifoldbt/blob/master/examples/06_full_visualization.py) | Tearsheet and charts |
| 07 | [Walk-Forward](https://github.com/manifoldbt/manifoldbt/blob/master/examples/07_walk_forward.py) | Out-of-sample validation |
| 08 | [2D Sweep](https://github.com/manifoldbt/manifoldbt/blob/master/examples/08_sweep_2d_heatmap.py) | Parameter grid heatmap |
| 09 | [3D Surface](https://github.com/manifoldbt/manifoldbt/blob/master/examples/09_surface_3d.py) | Parameter surface plot |
| 10 | [Monte Carlo](https://github.com/manifoldbt/manifoldbt/blob/master/examples/10_monte_carlo.py) | Permutation-based robustness |
| 11 | [Portfolio](https://github.com/manifoldbt/manifoldbt/blob/master/examples/11_portfolio.py) | Multi-strategy portfolio |
| 12 | [Diagnostics](https://github.com/manifoldbt/manifoldbt/blob/master/examples/12_diagnostics.py) | Lookahead & exposure safety checks |
| 13 | [Stochastic Simulation](https://github.com/manifoldbt/manifoldbt/blob/master/examples/13_stochastic_simulation.py) | SDE path simulation (GBM, Heston, …) |
| 14 | [Multi-Timeframe](https://github.com/manifoldbt/manifoldbt/blob/master/examples/14_multi_timeframe.py) | Combining signals across timeframes |
| 15 | [Cross-Exchange](https://github.com/manifoldbt/manifoldbt/blob/master/examples/15_cross_exchange.py) | Signal on one venue, execute on another |
| 16 | [Exogenous Data](https://github.com/manifoldbt/manifoldbt/blob/master/examples/16_hashrate_exogene.py) | External series (e.g. hashrate) as a signal |
| 17 | [Per-Venue Fees](https://github.com/manifoldbt/manifoldbt/blob/master/examples/17_per_venue_fees.py) | Per-venue funding & borrow costs |
| 18 | [CSV Import](https://github.com/manifoldbt/manifoldbt/blob/master/examples/18_csv_import.py) | Load OHLCV from CSV (standard / MT4 / MT5) |
| 19 | [Custom Indicators](https://github.com/manifoldbt/manifoldbt/blob/master/examples/19_custom_indicators.py) | Write the ones the library does not ship |
| 20 | [Entry Orders](https://github.com/manifoldbt/manifoldbt/blob/master/examples/20_entry_orders.py) | Rest an entry at a price instead of taking the close |
| 21 | [Computed Fill Level](https://github.com/manifoldbt/manifoldbt/blob/master/examples/21_fill_at_computed_level.py) | Fill at a level the strategy computes |
| 22 | [Yahoo Equities](https://github.com/manifoldbt/manifoldbt/blob/master/examples/22_yahoo_equities.py) | Stocks, ETFs, indices, FX and futures |
| 23 | [Crypto Options](https://github.com/manifoldbt/manifoldbt/blob/master/examples/23_deribit_options.py) | Deribit contracts that actually expire |
| 24 | [Option Spread](https://github.com/manifoldbt/manifoldbt/blob/master/examples/24_option_spread.py) | A bull call spread, held to expiration |
| 25 | [Look-Ahead Trap](https://github.com/manifoldbt/manifoldbt/blob/master/examples/25_lookahead_trap.py) | Which audit answers which question |
## Look-ahead
`mbt.detect_lookahead` re-runs a strategy over different windows and compares
the trades they have in common. That is what isolates bias coming from the
engine or from a strategy's own use of time.
One shape of leak is invisible to that comparison by construction: a signal
that reads a fixed number of bars ahead (`close.lead(1)`, or a column
precomputed with future bars and imported as data). Bar T sees the same T+1
in every window, so the shorter run trades exactly like the full one and the
re-runs report PASS, next to a Sharpe in the hundreds. No choice of split
changes that. The detector therefore also walks the strategy's expressions
and fails any explicit `lead()`, naming the signal. For an implicit one, the
check belongs to the pipeline that built the column: perturb the raw data
after some bar K, rebuild the column from it, and require the equity up to K
to be bit-identical. Perturbing the bars alone leaves a precomputed column
untouched and sees nothing, as `examples/25_lookahead_trap.py` shows.
A parameter derived from the data *before* the backtest is a different
question: a threshold computed over the whole history in a notebook and then
passed in as a number is the same number in every run, so no re-run-based
method can weigh it. Treat any parameter that came from data as part of the
pipeline, re-derive it on the window under test, and compare the results.
[`examples/25_lookahead_trap.py`](https://github.com/manifoldbt/manifoldbt/blob/master/examples/25_lookahead_trap.py)
runs both on the same strategy, including perturbing every future bar, and
prints what each method concludes, so the difference is visible rather than
asserted.
Against an event-driven engine the comparison is only an order of magnitude:
backtrader runs an EMA(12/26) + RSI(14) strategy on 500K 1-minute bars in
46.9 s against 13 ms here
([`benchmarks/bench_vs_competitors.py`](https://github.com/manifoldbt/manifoldbt/blob/master/benchmarks/bench_vs_competitors.py),
median of 3, on a developer machine). It stays out of the CI suite because its
fills differ, and the parity gate publishes no timing for engines that did not
do the same work.
## Correctness
Speed is worth nothing if the fills are wrong. Everything in this section is a
test in this repository, and it runs in CI against the wheel **published on
PyPI**, not against the source, on five Python versions. No licence is
configured in that job on purpose, so what it exercises is the experience of
someone who has just run `pip install manifoldbt`, and the run prints which
tests skipped rather than showing a green tick that hides them.
PyPI**, not against the source, on Python 3.9 to 3.13. No licence is configured
in that job on purpose, so what it exercises is the experience of someone who
has just run `pip install manifoldbt`, and the run prints which tests skipped
rather than showing a green tick that hides them.
**Fill-level parity with vectorbt.**
[`test_parity_vectorbt.py`](https://github.com/manifoldbt/manifoldbt/blob/master/python/tests/test_parity_vectorbt.py)
checks **where a trade actually filled and why it exited**, not a summary
statistic. Each scenario is a short series built to produce one clean round
trip, and the entry price, the exit price, the exit reason and the final return
are all asserted against vectorbt: market take-profit, market stop-loss,
trip, and the entry price, exit price, exit reason and final return are all
asserted against vectorbt: market take-profit, market stop-loss,
stop-loss/take-profit brackets, shorts and trailing stops. A further scenario
pins the fee arithmetic across two round trips. The equity curve is deliberately
left out, because a Community build caps it to daily resolution and it would not
be an apples-to-apples comparison.
pins the fee arithmetic across two round trips.
**Resting limit entries are checked without vectorbt**, which has no resting
order to compare against. They are pinned instead by a NumPy model that computes
the fill from raw OHLC with no call into the engine. Read it for what it is: it
catches an implementation drifting away from the documented rule, and it does
not independently prove the rule is the right one, because the rule was measured
from the engine before being re-implemented.
**Resting limit entries** have no vectorbt equivalent to compare against. They
are pinned instead by a NumPy model that computes the fill from raw OHLC with no
call into the engine. That catches an implementation drifting away from the
documented rule; it does not independently prove the rule is right, because the
rule was measured from the engine before being re-implemented.
**Look-ahead.** Two regression tests in
[`python/tests/`](https://github.com/manifoldbt/manifoldbt/tree/master/python/tests), the stricter of which corrupts every bar after
a cut point, re-runs, and requires the equity before that point to come back
**bit-identical**: any decision that read a future bar moves the prefix. They sit
alongside the worked example above, which exists to demonstrate a leak the
detector **cannot** see and to say so plainly.
**Look-ahead.** Regression tests in
[`python/tests/`](https://github.com/manifoldbt/manifoldbt/tree/master/python/tests),
the strictest of which corrupts every bar after a cut point, re-runs, and
requires the equity before that point to come back **bit-identical**: any
decision that read a future bar moves the prefix.
### Execution semantics
The rules below are pinned by tests. They are written out here so you can check
them against the wheel you installed rather than take them on faith. Where a
choice was available, the pessimistic one was taken.
These rules are pinned by tests. They are written out so you can check them
against the wheel you installed rather than take them on faith. Where a choice
was available, the pessimistic one was taken.
| Situation | What the engine does |
|---|---|
| Stop-loss and take-profit are both touched in the same bar | **The stop wins.** The path within a bar is unknown, so the unfavourable outcome is assumed rather than the profitable one. |
| Stop-loss and take-profit both touched in the same bar | **The stop wins.** The path within a bar is unknown, so the unfavourable outcome is assumed. |
| Price gaps through a stop | Fills at the bar's **open**, not at the stop price. A stop at 100 on a bar opening at 95 fills at 95. |
| A limit entry is never reached | It expires at the end of its good-till window without filling. |
| A limit entry is immediate-or-cancel | It is cancelled, and will not fill on a later bar that would have triggered it. |
| A stop-limit's stop level is touched | The order arms and then rests at its limit, which may never fill. |
| An order is too large for the bar | It fills over several bars at the configured participation rate, and the partially filled position is bracketed **while** it fills. |
| Trailing stops | Never trigger on the same bar they ratchet on, which would require reading the intra-bar path. |
| An order is too large for the bar | It fills over several bars at the configured participation rate, and the partial position is bracketed **while** it fills. |
| Trailing stops | Never trigger on the bar they ratchet on, which would require reading the intra-bar path. |
| An execution-price expression evaluates to NaN | Falls back to the close and warns, rather than dropping the trade silently. |
| A short option's maintenance margin exceeds equity | The position is force-closed on that bar. |
| Fees, funding and borrow rates | Resolved per symbol, so venues inside one portfolio keep their own rates. |
@@ -346,116 +286,247 @@ The edges, stated rather than left to be discovered:
- The largest universe under test is a handful of instruments. Cross-sectional
research across thousands of assets is exercised by the sweep benchmarks, not
by the correctness suite.
- Look-ahead coverage is two regression tests and one worked example, not a
systematic battery of leak archetypes.
- Look-ahead coverage is a set of regression tests and one worked example, not
a systematic battery of leak archetypes.
The engine's own Rust suite is not published, which is why the rules above are
written out rather than linked. Independent verification is welcome, and a
reproduction showing a fill this engine gets wrong is the most useful report
this project can receive.
## Performance
## Look-ahead
Every number below comes from a benchmark that runs in public CI on a standard
GitHub runner, and links back to the run that produced it. It installs each
engine from PyPI the way a user would, generates its own data, checks that the
engines produced the **same result**, and only then reports how long each took:
a workload they disagree on gets no published timing at all.
`mbt.detect_lookahead` re-runs a strategy over different windows and compares
the trades they have in common. That isolates bias coming from the engine or
from a strategy's own use of time.
**Latest run: [#13](https://github.com/manifoldbt/manifoldbt/actions/runs/32469701489)**
ran on Linux x86_64, 4 vCPU (AMD EPYC 7763), Python 3.12, manifoldbt 0.18.0 /
vectorbt 0.28.4 / raptorbt 0.9.0, 3 interleaved repetitions, medians reported.
One shape of leak is invisible to that comparison by construction: a signal
that reads a fixed number of bars ahead (`close.lead(1)`, or a column
precomputed with future bars and imported as data). Bar T sees the same T+1 in
every window, so the re-runs report PASS next to a Sharpe in the hundreds. The
detector therefore also walks the strategy's expressions and fails any explicit
`lead()`, naming the signal. For an implicit one, the check belongs to the
pipeline that built the column: perturb the raw data after some bar K, rebuild
the column from it, and require the equity up to K to be bit-identical.
Perturbing the bars alone leaves a precomputed column untouched and sees
nothing.
| Workload | Bars | Manifold-BT | vectorbt | raptorbt |
|---|---:|---:|---:|---:|
| SMA crossover | 10M | **327 ms** | 26.12 s (x79) | 913 ms (x2.8) |
| ...with drawdown, Sharpe, Sortino, volatility | 10M | **329 ms** | 102.38 s (**x311**) | 909 ms (x2.8) |
| ...with a 5 bps fee and 2 bps slippage | 10M | **337 ms** | 26.08 s (x79) | not supported |
| EMA + RSI filter, 5 bps fee | 1M | **57 ms** | 2.35 s (x40) | not supported |
| Five assets in one book | 1M | **148 ms** | 2.54 s (x17) | not supported |
| Stop-loss and take-profit bracket | 10M | **934 ms** | 26.24 s (x28) | 916 ms (**x1.0**) |
A parameter derived from the data *before* the backtest is a different
question: a threshold computed over the whole history in a notebook is the same
number in every run, so no re-run can weigh it. Re-derive it on the window under
test.
[`examples/25_lookahead_trap.py`](https://github.com/manifoldbt/manifoldbt/blob/master/examples/25_lookahead_trap.py)
runs every one of these checks on the same strategy and prints what each
concludes, so the difference is visible rather than asserted.
The second row is the one worth reading twice. Asking for a performance summary
costs Manifold-BT nothing measurable, because it computes one during the run
whether you read it or not, and costs vectorbt 102 seconds, because it defers
the equity curve until a risk metric needs it and then has to build one.
## What you can build
The last two rows are the ones where Manifold-BT does worst, and they are
published for that reason. Broadcasting a column per asset is close to free for
vectorbt, while walking five books is not free for anything. And on a
stop-loss/take-profit bracket, raptorbt is level with us: the intra-bar check
that decides which of the two triggers first is a sequential walk in both
engines, so there is no vectorization left to win with.
**Data.** Built-in connectors, all returning a store ready for `mbt.run`:
Binance, Bybit, Hyperliquid, dYdX, Bitstamp, Deribit, Yahoo Finance and
Dukascopy on every tier, Databento and Massive with Pro. Or bring your own CSV
(standard, MetaTrader 4 and MetaTrader 5 layouts are detected):
### Parameter sweeps
```python
store = mbt.import_csv("EURUSD_1m.csv", symbol="EURUSD", symbol_id=1,
interval="1m", asset_class="forex")
| Bars | Combinations | Manifold-BT | vectorbt | raptorbt |
|---:|---:|---:|---:|---:|
| 20,000 | 5,000 | **446 ms**, 40 MB | 5.84 s, 2.5 GB | 7.08 s |
| 200,000 | 10,000 | **9.96 s**, 79 MB | out of memory | 164.50 s |
# Stocks, ETFs, indices, FX and futures from Yahoo Finance, no API key
store = mbt.ingest(provider="yahoo", symbol="AAPL", symbol_id=1, interval="1d",
asset_class="equity",
start="2015-01-01T00:00:00Z", end="2026-01-01T00:00:00Z")
```
Past a certain grid the question stops being speed. vectorbt materialises the
simulation per combination, 1.57 MB of it at 20,000 bars, so the second row
would ask a machine for tens of gigabytes. Manifold-BT runs it in ten seconds
inside 79 MB.
Yahoo prices are dividend-adjusted, like `yfinance`'s `auto_adjust=True`; pass
`dataset="raw"` for unadjusted quotes. Deribit option contracts carry their
strike, expiry and settlement into the store and settle at intrinsic value on
the expiration bar.
Reproduce any of it yourself: fork the repository and press **Run workflow** on
[the benchmark](https://github.com/manifoldbt/manifoldbt/actions/workflows/bench-vs-vectorbt.yml),
or run it locally from
[`benchmarks/vs_vectorbt/`](https://github.com/manifoldbt/manifoldbt/tree/master/benchmarks/vs_vectorbt).
The method, the parity gate and the known divergences are written up in
[its README](https://github.com/manifoldbt/manifoldbt/blob/master/benchmarks/vs_vectorbt/README.md).
**Signals.** Over 100 indicator functions and 38 candlestick patterns, from
moving averages and oscillators to rolling regression, Kalman filters, GARCH,
and cross-sectional z-scores and neutralization, all composable with
conditions, lags and references to other assets. The ones the library does not
ship
[you can write yourself](https://github.com/manifoldbt/manifoldbt/blob/master/examples/19_custom_indicators.py).
### Against an event-driven engine
**Higher timeframes, without look-ahead.** Declare them next to the simulation
timeframe and read them with `mbt.tf(...)`. A higher-timeframe bar only becomes
readable once it has closed. `.apply(...)` computes an indicator on that
timeframe's own grid, so the period counts in *its* bars:
backtrader runs the same EMA(12/26) + RSI(14) strategy on 500K 1-minute bars in
**46,944 ms**, against **13 ms** for Manifold-BT: a factor of **3,556**. Measured
with `benchmarks/bench_vs_competitors.py`, median of 3 runs, on a developer
machine and not the CI runner, so it is not comparable line-for-line with the
table above.
```python
from manifoldbt.indicators import close, sma
It sits outside the CI suite because its event-driven fills produce a different
PnL, and the parity gate publishes no timing for engines that did not do the
same work. Treat it as an order of magnitude, not a benchmark: the two engines
are not doing the same thing.
config = mbt.BacktestConfig(
...,
bar_interval=Interval.minutes(1), # simulate on 1m
extra_timeframes={"1h": Interval.hours(1)}, # also resample to 1h
)
band = mbt.tf("1h").apply(sma(close, 20)) # mean of 20 HOURLY closes
```
### How it compares
`sma(mbt.tf("1h").close, 20)` is not the same thing: it smooths the step-held
hourly series over 20 *simulation* bars.
| | Manifold-BT | vectorbt | backtrader | Nautilus |
|---|---|---|---|---|
| Engine | Rust (vectorized + sequential fills) | Numba/NumPy (vectorized) | Python (event-driven) | Rust/Python (event-driven) |
| Execution realism¹ | High | Basic | High | High |
| Focus | Backtesting + research | Backtesting at scale | Backtest + live | Backtest + live (production) |
**Sweep a choice, not just a number.** `mbt.choice(...)` makes an expression an
axis of the grid. Each combination resolves to its branch before simulation, so
the branches it did not pick cost nothing:
¹ fees, slippage, funding, partial fills, look-ahead detection.
```python
band = mbt.choice("band", {
"30m": mbt.tf("30m").apply(sma(close, mbt.param("len"))),
"1h": mbt.tf("1h").apply(sma(close, mbt.param("len"))),
})
strategy = (mbt.Strategy.create("band_cross")
.signal("band", band)
.size(mbt.when(close > mbt.col("band"), 1.0, 0.0)))
config.extra_timeframes = {"30m": Interval.minutes(30), "1h": Interval.hours(1)}
> On GPU (Pro), the Monte Carlo engine runs **~36x faster** than the all-core CPU path (SDE path simulation, RTX 3090, f32).
sweep = mbt.run_sweep(strategy, {"band": ["30m", "1h"], "len": range(10, 210, 10)},
config, store)
```
## Documentation
The same mechanism sweeps which exogenous column to use, which asset to
reference, or which indicator to apply.
Full API reference, indicator list, configuration guide, and best practices:
**Execution.** Maker and taker fees per venue, perpetual funding, borrow costs,
slippage, partial fills at a participation rate, limit, stop and stop-limit
entries, brackets and trailing stops, and fills at a price the strategy
computes. Simulation on bars from one minute up, or from one second with Pro.
**[www.manifoldbt.com/docs/documentation.html](https://www.manifoldbt.com/docs/documentation.html)**
**Research.** Walk-forward optimization, Monte Carlo resampling, stochastic path
simulation (GBM, Heston, Merton jump-diffusion, GARCH with jumps), multi-asset
and multi-strategy portfolios, cross-exchange universes, exposure and risk
checks, and HTML tearsheets.
## Community vs Pro
**Trades and the order book** (Researcher). Resolve stops and targets against
the individual trades inside each bar instead of assuming the order of high and
low, simulate a market maker on the trade clock with a queue and a latency, and
reconcile the backtest against the fills a broker actually granted.
| | Community | Pro |
|---|---|---|
| Single backtests (`mbt.run`) | Unlimited, full speed | Unlimited, full speed |
| Parameter sweeps & batches | Up to 256 backtests per sweep | Unlimited |
| Simulation resolution (`bar_interval`) | 1 minute and coarser | Down to 1 second |
| Output resolution | Daily | Down to 1 second |
| Monte Carlo | 1K sims | Unlimited |
| Walk-Forward | - | Anchored + Rolling |
| Parameter Stability | - | Yes |
| Free connectors (Binance, Bybit, Hyperliquid, dYdX, Bitstamp, Yahoo, Dukascopy) | Yes | Yes |
| Databento & Massive connectors | - | Yes |
| GPU acceleration (`device="cuda"`) | - | Yes |
| Safety checks (lookahead, exposure) | - | Yes |
| Tearsheets & export | Yes | Yes |
## Examples
Clone the repository and run one:
```bash
python examples/13_stochastic_simulation.py
```
Most examples backtest real market data, which is not in the repository.
`python examples/setup_data.py` downloads it once, from free connectors that
need no API key, and
[examples/README.md](https://github.com/manifoldbt/manifoldbt/blob/master/examples/README.md)
says which files need it and which run on nothing at all. Charts come from the
optional `[plot]` extra: without it an example still runs to the end and skips
its chart.
Good places to start:
[01 Trend following](https://github.com/manifoldbt/manifoldbt/blob/master/examples/01_trend_following.py) &middot;
[05 Statistical arbitrage](https://github.com/manifoldbt/manifoldbt/blob/master/examples/05_stat_arb.py) &middot;
[07 Walk-forward](https://github.com/manifoldbt/manifoldbt/blob/master/examples/07_walk_forward.py) &middot;
[25 Look-ahead trap](https://github.com/manifoldbt/manifoldbt/blob/master/examples/25_lookahead_trap.py) &middot;
[26 Fill costs](https://github.com/manifoldbt/manifoldbt/blob/master/examples/26_fill_costs.py)
<details>
<summary><strong>All 30 examples</strong></summary>
| # | Example | What it shows |
|---|---------|---------------|
| 00 | [Template](https://github.com/manifoldbt/manifoldbt/blob/master/examples/00_template.py) | Minimal starting point |
| 01 | [Trend Following](https://github.com/manifoldbt/manifoldbt/blob/master/examples/01_trend_following.py) | EMA crossover, volume filter, stop-loss |
| 02 | [Mean Reversion](https://github.com/manifoldbt/manifoldbt/blob/master/examples/02_mean_reversion.py) | EMA crossover with parameter sweep |
| 03 | [Multi-Asset Momentum](https://github.com/manifoldbt/manifoldbt/blob/master/examples/03_multi_asset_momentum.py) | Cross-asset signals |
| 04 | [Linear Regression](https://github.com/manifoldbt/manifoldbt/blob/master/examples/04_linear_regression.py) | Regression-based signal |
| 05 | [Statistical Arbitrage](https://github.com/manifoldbt/manifoldbt/blob/master/examples/05_stat_arb.py) | Pairs trading, spread z-score |
| 06 | [Full Visualization](https://github.com/manifoldbt/manifoldbt/blob/master/examples/06_full_visualization.py) | Tearsheet and charts |
| 07 | [Walk-Forward](https://github.com/manifoldbt/manifoldbt/blob/master/examples/07_walk_forward.py) | Out-of-sample validation |
| 08 | [2D Sweep](https://github.com/manifoldbt/manifoldbt/blob/master/examples/08_sweep_2d_heatmap.py) | Parameter grid heatmap |
| 09 | [3D Surface](https://github.com/manifoldbt/manifoldbt/blob/master/examples/09_surface_3d.py) | Parameter surface plot |
| 10 | [Monte Carlo](https://github.com/manifoldbt/manifoldbt/blob/master/examples/10_monte_carlo.py) | Permutation-based robustness |
| 11 | [Portfolio](https://github.com/manifoldbt/manifoldbt/blob/master/examples/11_portfolio.py) | Multi-strategy portfolio |
| 12 | [Diagnostics](https://github.com/manifoldbt/manifoldbt/blob/master/examples/12_diagnostics.py) | Look-ahead and exposure checks |
| 13 | [Stochastic Simulation](https://github.com/manifoldbt/manifoldbt/blob/master/examples/13_stochastic_simulation.py) | SDE path simulation (GBM, Heston, ...) |
| 14 | [Multi-Timeframe](https://github.com/manifoldbt/manifoldbt/blob/master/examples/14_multi_timeframe.py) | Combining signals across timeframes |
| 15 | [Cross-Exchange](https://github.com/manifoldbt/manifoldbt/blob/master/examples/15_cross_exchange.py) | Signal on one venue, execute on another |
| 16 | [Exogenous Data](https://github.com/manifoldbt/manifoldbt/blob/master/examples/16_hashrate_exogene.py) | External series (e.g. hashrate) as a signal |
| 17 | [Per-Venue Fees](https://github.com/manifoldbt/manifoldbt/blob/master/examples/17_per_venue_fees.py) | Per-venue funding and borrow costs |
| 18 | [CSV Import](https://github.com/manifoldbt/manifoldbt/blob/master/examples/18_csv_import.py) | Load OHLCV from CSV (standard / MT4 / MT5) |
| 19 | [Custom Indicators](https://github.com/manifoldbt/manifoldbt/blob/master/examples/19_custom_indicators.py) | Write the ones the library does not ship |
| 20 | [Entry Orders](https://github.com/manifoldbt/manifoldbt/blob/master/examples/20_entry_orders.py) | Rest an entry at a price instead of taking the close |
| 21 | [Computed Fill Level](https://github.com/manifoldbt/manifoldbt/blob/master/examples/21_fill_at_computed_level.py) | Fill at a level the strategy computes |
| 22 | [Yahoo Equities](https://github.com/manifoldbt/manifoldbt/blob/master/examples/22_yahoo_equities.py) | Stocks, ETFs, indices, FX and futures |
| 23 | [Crypto Options](https://github.com/manifoldbt/manifoldbt/blob/master/examples/23_deribit_options.py) | Deribit contracts that actually expire |
| 24 | [Option Spread](https://github.com/manifoldbt/manifoldbt/blob/master/examples/24_option_spread.py) | A bull call spread, held to expiration |
| 25 | [Look-Ahead Trap](https://github.com/manifoldbt/manifoldbt/blob/master/examples/25_lookahead_trap.py) | Which audit answers which question |
| 26 | [Fill Costs](https://github.com/manifoldbt/manifoldbt/blob/master/examples/26_fill_costs.py) | The same signal filled four ways, and what each costs |
| 27 | [Bars vs Tape](https://github.com/manifoldbt/manifoldbt/blob/master/examples/27_bars_vs_tape.py) | A bracket resolved on the candle and on the trades inside it (Researcher) |
| 28 | [Trade-Clock Market Maker](https://github.com/manifoldbt/manifoldbt/blob/master/examples/28_trade_clock_market_maker.py) | Queue, latency, and what each fill cost (Researcher) |
| 29 | [Reconcile Live Fills](https://github.com/manifoldbt/manifoldbt/blob/master/examples/29_reconcile_live_fills.py) | A broker's own fills against the backtest of the same days (Researcher) |
</details>
## Installation options
```bash
pip install manifoldbt # engine only: backtests, sweeps, metrics
pip install manifoldbt[pandas] # + DataFrame output
pip install manifoldbt[plot] # + interactive charts and native windows (show=True)
pip install manifoldbt[all] # everything: plots, windows, PNG export, pandas/polars
pip install manifoldbt[gpu] # + NVIDIA runtime compiler, for device="cuda"
```
The base install stays light (no browser, no GUI) for scripts, servers and CI.
`[plot]` adds plotly and a native window backend; `[all]` also pulls kaleido
for static PNG/SVG export, which bundles a headless Chromium.
The Linux and Windows x86_64 wheels already carry the CUDA kernels, so `[gpu]`
only adds the NVIDIA runtime compiler (~180 MB) that compiles them on your
machine. Skip it if you already have a CUDA toolkit installed. An NVIDIA driver
is required, and GPU acceleration is a Researcher feature; everything else runs
at full speed on the CPU.
**Staying up to date.** manifoldbt asks PyPI once a day, in the background,
whether a newer release exists, and prints a one-line notice under the banner
when one does. It never delays an import (the notice is the previous run's
answer, read from a local cache) and the request is a plain GET of a public
JSON document. Set `MANIFOLDBT_NO_UPDATE_CHECK=1` to turn it off, or call
`mbt.check_for_update()` to ask on demand.
## Community, Pro and Researcher
Single backtests are free and run at full speed, with no run limit and no
account. The
paid tiers add scale and depth. See [pricing](https://www.manifoldbt.com/#pricing).
| | Community | Pro | Researcher |
|---|:---:|:---:|:---:|
| Single backtests (`mbt.run`) | Unlimited, full speed | Unlimited | Unlimited |
| Parameter sweeps and batches | Up to 256 backtests per sweep | Unlimited | Unlimited |
| Simulation resolution (`bar_interval`) | 1 minute and coarser | Down to 1 second | Down to 1 second |
| Output resolution | Daily | Down to 1 second | Down to 1 second |
| Monte Carlo | 1,000 simulations | Unlimited | Unlimited |
| Walk-forward optimization | - | Yes | Yes |
| Look-ahead detection (`detect_lookahead`) | - | Yes | Yes |
| Exposure and risk checks (`risk_check`) | Yes | Yes | Yes |
| Cross-exchange universes | - | Yes | Yes |
| Free connectors and CSV import | Yes | Yes | Yes |
| Databento and Massive connectors | - | Yes | Yes |
| Tearsheets and export | Yes | Yes | Yes |
| GPU acceleration (`device="cuda"`) | - | - | Yes |
| Trade tape, order book, queue and latency | - | - | Yes |
## Documentation and community
- API reference, indicator list, configuration guide:
**[manifoldbt.com/documentation](https://www.manifoldbt.com/documentation)**
- Writing strategies, from the first signal to the tape:
[docs/strategy-authoring.md](https://github.com/manifoldbt/manifoldbt/blob/master/docs/strategy-authoring.md)
- Questions, ideas, or a backtest to share: [Discord](https://discord.gg/bvU6Wjc72d)
- Found a fill the engine gets wrong? [Open an issue](https://github.com/manifoldbt/manifoldbt/issues)
with a reproduction. It is the most useful report this project can receive.
## License
Apache 2.0 with Commons Clause. The source is available, free to use,
modify and self-host. Reselling the software or offering it as a paid
hosted service is not permitted. See [LICENSE](https://github.com/manifoldbt/manifoldbt/blob/master/LICENSE) for the full text.
Apache 2.0 with Commons Clause. The source is available, free to use, modify and
self-host. Reselling the software or offering it as a paid hosted service is not
permitted. See [LICENSE](https://github.com/manifoldbt/manifoldbt/blob/master/LICENSE)
for the full text.
docs/strategy-authoring.md
+234 -44
View File
@@ -1089,7 +1089,9 @@ config.execution.fill_model = {
update, on a side standing empty, or deeper than the deepest level stored or
read (`book_levels` when it is set: see
[How deep a run reads](#how-deep-a-run-reads)) — the queue is **unknown**: `assumed_queue` takes over as the declared fallback
when one was given, and the run is refused by name when none was. It is never
when one was given, and the run is refused by name when none was (a strategy
that quotes, like `mbt.sim`, refuses that one order instead and goes on: see
[Quoting in the DSL](#quoting-in-the-dsl)). It is never
read as zero, which would put the order at the front of a queue nobody saw.
**A level off the tick grid finds no depth in the book**, so it lands in this
case every single time: set `execution.tick_size` (see
@@ -1123,6 +1125,14 @@ absolute quantity has no scale from one symbol to the next: the same ten days
measured a median queue of 62 times the quoted size on BTCUSDT and 7 times on
XRPUSDT.
An assumed queue is held to its size: under `depth_source="assumed"` the order
waits behind `assumed_queue` times its own size, and no state of the book moves
that, whatever the level shows. The bound that brings a queue down to the
depth of its level (nobody can be ahead of us for more than the level holds)
belongs to `depth_source="book"`; applied to an assumed queue it would make
every `assumed_queue` above the displayed size the same queue, and the axis you
are asked to sweep would stop moving the result.
### One decision per event
The queue is walked print by print, and the walk of an event stops on the first
@@ -1158,31 +1168,63 @@ result.fill_fragility
# 'would_fill_touch': 7432, 'would_fill_traverse': 1329}
```
- `queue_decided_fills` — the volume ahead was consumed and the trade reached the
order. This is what modelling the queue bought.
- `traverse_fills` — a print through the level, or the book crossing it, served
the order. This is what the `traverse` convention already gave for free.
- `partial_fills` — orders whose life ended having filled part of their size.
- `book_unknown_at_post` — orders posted behind an `assumed_queue` because the
- `queue_decided_fills` counts **fills**: the volume ahead was consumed and the trade
reached the order. This is what modelling the queue bought.
- `traverse_fills` counts **fills**: a print through the level, or the book crossing
it, served the order. This is what the `traverse` convention already gave for
free. It counts the `traverse` and the `book_cross` rows of a fill table
alike.
- `partial_fills` counts **orders** whose life ended having filled part of their size.
- `book_unknown_at_post` counts **orders** posted behind an `assumed_queue` because the
stored book could not answer at their level and instant.
- `fills_from_book_cross` — of `traverse_fills`, the ones decided by a **book
cross** rather than by a print through the level. Kept apart because an
archive can carry a transient crossing the live sequence never had: subtract
it if you do not believe yours.
- `fills_from_book_cross` counts **fills**, a subset of `traverse_fills`: the ones
decided by a **book cross** rather than by a print through the level. Kept
apart because an archive can carry a transient crossing the live sequence never
had: subtract it if you do not believe yours.
`would_fill_touch` and `would_fill_traverse` sit beside it: how many of the same
order lives the `touch` and `traverse` conventions would have filled. They bound
the queue result without a second run:
`queue_decided_fills` and `traverse_fills` add up to `maker_fills`. An order
served in three pieces is three fills, one order and one life.
`would_fill_touch` and `would_fill_traverse` sit beside them in a third unit:
**lives**. Each time an order stops resting at a level (filled, cancelled,
expired, or requoted to another level), the run adds one to `would_fill_touch` if
a print at or through that level was seen during the stay, and one to
`would_fill_traverse` if a print strictly through it was. It is how many of the
same order lives the `touch` and `traverse` conventions would have filled,
whether or not the queue served them.
Because the units differ, fills cannot be set against lives: one order served in
two pieces is two fills and one life. Set the lives against the **orders
served**, the orders with at least one maker fill, and they bound the queue
result without a second run:
```
would_fill_traverse <= queue_decided_fills + traverse_fills
would_fill_traverse <= orders_served
<= would_fill_touch + fills_from_book_cross
```
with `orders_served <= queue_decided_fills + traverse_fills`, an equality when no
order is served in pieces. On a `bt.sim` run:
```python
fills = res.fills_df()
orders_served = fills.loc[fills["maker"], "order_id"].nunique()
```
Both halves are the service rule read backwards. A print through the level
serves the queue whatever is ahead, so every traversal is a fill; and a queue
fill needs a print AT the level, which `touch` would have taken — except the
ones no print witnessed, which the book crossing decided.
serves the rest of the order whatever is ahead, so every order a traversal
reached is served; and an order served by the queue needs a print AT the level,
which `touch` would have taken, except the ones no print witnessed, which the
book crossing decided. On two hours of a real BTCUSDT day (a maker a tick behind
the touch, `risk_adverse`), the envelope reads 862 <= 1,285 <= 1,308: 1,285
orders were served, in 1,325 fills, and the fills cross the right-hand side where
the orders do not.
The envelope is for a market stored by price. A market stored order by order
serves its orders from the order events, while these two counters are still
counted off the prints, so the two can disagree, in either direction. A rule
that declines fills (see [Execution Models](#execution-models)) moves the left
side the same way.
They are counted on the lives *this* run produced, so they are an envelope of
this run and not of two other ones: `touch` fills sooner, so a touch run would
@@ -1465,9 +1507,9 @@ config.execution.fill_model = {
`risk_adverse` refuses to move the volume ahead of an order when the level
shrinks with no print at it: a level-2 archive never says whose order left, and
assuming the cancellations were in front of us is assuming our own luck. Its one
bound is the level itself: nobody can be ahead of us for more than the level
holds, so a level shown below the volume ahead brings the queue down to what it
shows. These two answer with a stated probability instead. When the depth at our own level
bound, under `depth_source="book"`, is the level itself: nobody can be ahead of
us for more than the level holds, so a level shown below the volume ahead brings
the queue down to what it shows. These two answer with a stated probability instead. When the depth at our own level
falls by `chg` units and no print explains it,
```
@@ -1480,10 +1522,19 @@ with `front` the volume ahead, `back = prev_depth - front` the volume behind,
and `f(x) = x^n` (`power`) or `f(x) = log(1 + x)` (`log`). The depth is read
just before each print: the state stamped at a print's own instant may already
show it, and that print is counted once, as a trade, not a second time as a
decrease with nothing to explain it. The formula is
decrease with nothing to explain it: the prints at our level add up, however
many there are and whatever else prints in between, and start again only when
the depth read has moved and the formula has been given the change. The formula is
`hftbacktest`'s `ProbQueueModel`, reproduced rather than approximated so the two
engines can be compared fill by fill.
The formula reads the market's volume. On an order-by-order run (`mbt.sim`) the
strategy's own earlier orders at the level are in the volume ahead of an order
and not in the depth the book shows, so they are set aside for the formula and
put back after it: two orders of ours at one level are served in arrival order
under these queues as under `risk_adverse`, and the second is never served by
the volume that serves the first.
`n` steers how sharply the model believes the queue: near zero it shares every
cancellation half and half whatever the position; large, it gives almost all of
it to whichever side holds more volume. `log` is the same idea with a fixed,
@@ -1491,7 +1542,8 @@ gentler curvature and no parameter.
They need `depth_source="book"`. An assumed queue carries no depth at the level
to watch, so a probabilistic queue on one would silently be the historical rule
under another name; it is refused.
under another name; it is refused, by `mbt.run` and by `mbt.sim` alike, before the
first event.
**The duel.** Three BTCUSDT days, the same 15 415, 16 745 and 17 232 quotes, the
same one-bar life, the same size, fed to `hftbacktest` through its own converter
@@ -1502,12 +1554,12 @@ quote, at zero latency:
|---|---|---|---|---|---|
| 2025-08-13 | `risk_adverse` | 15 415 | 1 653 | 1 654 | 99.994 % |
| 2025-08-13 | `power`, n=3 | 15 415 | 1 654 | 1 656 | 99.974 % |
| 2025-08-13 | `log` | 15 415 | 1 657 | 1 658 | 99.955 % |
| 2025-08-13 | `log` | 15 415 | 1 654 | 1 658 | 99.974 % |
| 2025-08-23 | `risk_adverse` | 16 745 | 733 | 732 | 99.994 % |
| 2025-08-23 | `power`, n=3 | 16 745 | 736 | 735 | 99.970 % |
| 2025-08-23 | `log` | 16 745 | 738 | 736 | 99.964 % |
| 2025-08-23 | `power`, n=3 | 16 745 | 734 | 735 | 99.982 % |
| 2025-08-23 | `log` | 16 745 | 735 | 736 | 99.982 % |
| 2025-08-30 | `risk_adverse` | 17 232 | 740 | 740 | 100 % |
| 2025-08-30 | `power`, n=3 | 17 232 | 740 | 741 | 99.983 % |
| 2025-08-30 | `power`, n=3 | 17 232 | 739 | 741 | 99.988 % |
| 2025-08-30 | `log` | 17 232 | 740 | 741 | 99.994 % |
Every `risk_adverse` disagreement was traced quote by quote on the raw event
@@ -1517,8 +1569,8 @@ the order's price by a buying aggressor serves it here and not in `hftbacktest`,
which reads only selling prints against a bid: an offer stood under the bid while
the order rested. A print stamped at the very instant the order leaves (the
next bar's label) is counted by the harness that drives `hftbacktest`; here the
order no longer rests at the next bar's open. The probabilistic rules keep three
to six a day, not traced yet.
order no longer rests at the next bar's open. The probabilistic rules keep one
to four a day, not traced yet.
### Conditional adverse selection
@@ -2328,7 +2380,11 @@ levels per side of the stored book the venue and the strategy read: **every
stored level by default**, at most `levels` when it is given (the stored depth
when the book is shallower). Past it the book says nothing, as past the stored depth: `levels()`
stops there, and an order resting further out joins a queue the book cannot
measure (`assumed_queue`, or a refusal by name).
measure (`assumed_queue`, or a refusal by name). That refusal is the order's
alone, not the run's: a new order ends `rejected` (a `rejected` event in the
journal), a replacement is answered `replace_rejected` and the order stays as
it was, and the run goes on. The first order refused on each market is said in
a Python `UserWarning` that names the price, the side and the levels read.
### The strategy's side
@@ -2405,13 +2461,24 @@ and the fills show it. Without it, a sell a hair under `116994.6` would read a
print AT `116994.6` as one THROUGH its price and be served whole, ahead of the
queue the same order posted at `116994.6` waits in. A symbol without a tick
(`tick_size=0`) has its prices put on the stored book's own price grid, eight
decimals, and nothing else moves.
decimals, and nothing else moves. That grid, one hundred-millionth, is also the
finest the venue compares prices on, so a market whose tick is finer than it
(`tick_size=1e-9`) or falls between two of its points (`tick_size=1.5e-8`) is
refused by name when it is built (`Market.from_ladders`, `Market.from_store`,
and the quotes of `mbt.run`): two of its levels would be one point of the grid.
A tick of `0`, or any multiple of `1e-8`, is accepted.
**An order that would not execute on arrival rests.** Its queue is the size the
book shows at its price, plus the strategy's own earlier orders at that price
(first come, first served among them). Prints at its price burn that queue, and
what they leave past it serves the order, in part or in full. The queue can
never exceed the size the level shows. A print strictly through the order's
(first come, first served among them). When one of those earlier orders leaves
its place (it is cancelled, it expires, a replacement sends it to another price
or, being larger, to the back) or weighs less (a smaller replacement keeps its
place), the orders of ours behind it have exactly that much less ahead of them,
whatever the book has shown in the meantime. Prints at its price burn that
queue, and
what they leave past it serves the order, in part or in full. A queue read off
the book can never exceed the size the level shows; an assumed queue
(`depth_source="assumed"`) is `assumed_queue` times the order and stays so. A print strictly through the order's
price, whichever side crossed, or an opposite best price at or through it,
serves the rest: the level was emptied.
@@ -2420,19 +2487,90 @@ level by level, within its limit, and the rest of it expires. The simulated
fills take nothing out of the stored book, so a rest left there would be
crossed at once by the very liquidity it just took. A `GTX` that would execute
is rejected; a `FOK` that cannot fill whole takes nothing; a market order takes
what the ladder offers.
what the ladder offers. An order the size of the whole displayed ladder (the
float sum of its levels, whichever side of it the last digit falls on) is
served whole and ends `filled`: what it leaves is a rest only above a
trillionth of its own size.
**A replacement** at the same price with a smaller size keeps the order's place
in the queue; any other change sends it to the back of the queue at its new
price. A new price that would execute is refused, and the order stays as it
was.
was; so is a new price past the levels read with no `assumed_queue` (see
`levels=` above and "What refuses, and why").
**Fees** are the maker fee on a fill of a resting order and the taker fee on a
fill at arrival (`maker_fee_bps`, `taker_fee_bps`).
fill at arrival (`maker_fee_bps`, `taker_fee_bps`). A negative rate is a
rebate: the fill's fee is negative and the cash is credited with it
(`maker_fee_bps=-1` on a fill of 100.0 pays 0.01).
The simulated orders do not move the market: the stored book and tape are
replayed as they were.
### At the edges
What the venue does where the rules above meet each other, on a market stored by
price.
**A print at an order's price burns its queue only from the side that consumes
it.** A selling aggressor for a bid, a buying one for an ask: rule 6 of
[Queue Model](#queue-model). "Prints at its price burn that queue", above, means
the prints of that side. A print at a resting bid's price whose aggressor is a
buyer is the other side's trade: it neither burns the bid's queue nor serves it.
A print strictly THROUGH the price serves the rest of the order whichever side
crossed.
**A cancellation can overtake its order.** An order sent at `t` reaches the venue
at `t + entry`, a cancellation sent at `t'` at `t' + cancel`. When the
cancellation gets there strictly first (a `cancel` latency shorter than `entry`),
the venue has nothing to cancel and refuses it (`cancel_rejected` in the
journal). The order then arrives, rests and can be served, though the strategy
asked for the opposite. Cancelling an order still in flight works only if the
cancellation does not reach the venue before the order does; at the same instant
the order, sent first, goes first.
**A replacement** at the same price keeps the order's place when its total size
is smaller **or unchanged**; a larger size, or another price, sends it to the
back. A new total size that does not exceed what the order has already filled
cancels the rest: the order ends `cancelled` with what it filled, and its `qty`
stays the size it was sent with, not the new one. That holds when the new size
equals the filled part exactly: the order is not `filled`.
**A GTD order** (`expire_at`, a venue instant in nanoseconds):
- The market comes first at `expire_at` as at any instant: a print stamped
`expire_at` still serves the order, and the order expires after it.
- An `expire_at` at or before the instant the order reaches the venue expires it
on arrival. It never rests, and it does not execute even when its price would;
one nanosecond later it does.
- A cancellation and the expiry that land on the same instant: whichever the
venue booked first decides. The expiry is booked when the venue accepts the
order, a cancellation when the strategy sends it. So a cancellation sent before
the order reached the venue (the `cancel` shorter than `entry` case above) wins
and the order ends `cancelled`; one sent when or after the order reached it
comes second, the order ends `expired`, and the cancellation is refused.
**Statuses of what did not execute.** A `FOK` that cannot fill whole ends
`expired` with nothing filled; `rejected` is the post-only order that would
execute. An `IOC`, a market order or an executable `GTC` whose size found only
part of its liquidity ends `expired` too, and keeps the part that filled.
**Case.** `side`, `tif` and the symbol are read without regard to case: `"BUY"`,
`"gtc"` and a lower-case ticker are accepted, and the tables show `buy`, `GTC`
and the symbol as the market names it.
**The clock.** `sim.now()` starts at the first event, a book state or a print, of
ANY market of the run, not only of the ones the strategy trades. A symbol that is
in the run and idle but starts earlier moves the whole grid of `elapse(dt)`
wake-ups of the others, and with it the orders and the fills.
**The end of the data.** A `post` after `elapse` has returned `False` is accepted.
The order reaches the venue only if the strategy moves the clock again; a
function that returns right after it leaves the order `in_flight`.
**The seed.** `Config.seed` changes nothing today: no queue model draws
(`risk_adverse`, `power`, `log` and the exact queue are deterministic), so two
seeds give the same run to the bit.
### What comes out
| | |
@@ -2441,9 +2579,9 @@ replayed as they were.
| `res.fills_df()` | Every fill at the venue's instant, with its `channel`: `queue`, `traverse` (a print through the price), `book_cross`, `taker` |
| `res.events_df()` | The order journal: what the strategy sent (local time) and what the venue did (venue time) |
| `res.equity_df()` | Cash plus each position at its venue mid, at each wake-up |
| `res.metrics` | Orders, fills, volume, fees, P&L, maximum drawdown, final position |
| `res.metrics` | Orders, fills, volume, fees, P&L, maximum drawdown, final position. `orders_filled`, and the "N filled" of `res.summary()`, count the orders served at least in part, so an order filled in part and then cancelled or expired is counted: it can exceed the orders whose status is `filled` |
| `res.fill_marks` | The markouts of [Marking a Fill](#marking-a-fill), with the horizons counted from the fill itself (`anchor == "fill"`) |
| `res.fill_fragility` | What the queue decided for the orders that rested, in the shape of [Queue Model](#queue-model): `maker_fills`, and under `queue` the `queue_decided_fills`, `traverse_fills`, `fills_from_book_cross`, `partial_fills` and `book_unknown_at_post` (orders posted behind `assumed_queue`); `would_fill_touch` / `would_fill_traverse` count the lives of an order at one level that a print reached, or went through. `touch_only_fills` is a bar convention and stays `0` |
| `res.fill_fragility` | What the queue decided for the orders that rested, in the shape of [Queue Model](#queue-model): `maker_fills`, and under `queue` the `queue_decided_fills`, `traverse_fills`, `fills_from_book_cross`, `partial_fills` and `book_unknown_at_post` (orders posted behind `assumed_queue`); `would_fill_touch` / `would_fill_traverse` count the lives of an order at one level that a print reached, or went through. The counters under `queue` count fills, except `partial_fills` and `book_unknown_at_post`, which count orders; `traverse_fills` counts the `traverse` and the `book_cross` fills. Read the units in [Queue Model](#queue-model), "What comes out", before setting one against another. `touch_only_fills` is a bar convention and stays `0` |
| `res.levels_snapped` | Orders whose price was between two ticks and moved to the passive side |
### A market without a store
@@ -2461,6 +2599,18 @@ market = mbt.sim.Market.from_ladders(
res = mbt.sim.run(strategy, markets=[market])
```
It refuses what it cannot read, by name: states or prints out of time order,
levels out of price order, a price or a size that is not finite and positive. It
does not check that the market makes sense, and accepts:
- a **crossed or locked book**, a bid at or above the ask: an order that crosses
it executes at once against the displayed opposite side;
- a market with **prints and no book state**: every book read is empty
(`best_bid` is `None`) and a market order expires;
- a level **off the tick grid**: `book.levels()` shows it as written, while an
order priced there is put on the grid, on its passive side, so it does not
queue behind that level.
### Against another engine
On three BTCUSDT days, the quotes of the queue duel replayed through `mbt.sim`
@@ -2486,8 +2636,19 @@ differences come from two named rules:
- Cancelling or replacing an order the strategy knows to be finished, or a
market order's price: refused. Cancelling an order whose cancellation is
already on its way does nothing.
- A latency key other than the four, a negative latency, an unknown queue key:
refused before the run starts.
- A latency key other than the four, a negative latency, an unknown queue key,
a latency, a fee, `initial_cash` or a queue number that is NaN or infinite, a
negative `seed`: refused before the run starts, naming the key and the value.
- An order id that no `post` returned (`sim.cancel(-1)`, `sim.replace(-1, ...)`):
refused as an order never sent (`sim.order(-1)` is `None`). A `levels` below 1
when a market is read from a store, or `book.levels(n)` with a negative `n`:
refused, naming the parameter.
- An order posted, or replaced to a price, past the levels read, with no
`assumed_queue` to stand in for the queue there: refused on its own, by
name, and the run goes on (`rejected`, or `replace_rejected` with the order
left as it was). The words are those of a `UserWarning`, given once per run
and market, with the price, the side, the number of levels read and the way
out: `levels=`, or `assumed_queue`.
- A symbol with no stored book over the range: refused by name ("ingest it
first"). A range with no prints is accepted: the book alone can cross an
order.
@@ -2717,6 +2878,20 @@ counts no further, `bid_price_at(k)` past it is NaN as past the stored depth, a
book column deeper than it is refused before the run, and an order resting
further out joins a queue the venue cannot measure (`assumed_queue`).
**A quote past the levels read is refused on its own.** Without
`fill_model.queue.assumed_queue`, a quote at a price the stored book cannot
measure (further out than the levels read, or at an instant the book does not
cover) is refused on arrival and the run goes on: its order ends `rejected`
(the `rejected` event of `res.sim`), frees its quote, and the next wake-up asks
again. Each is counted in `res.order_activity["refused_beyond_levels"]`,
apart from `post_only_rejected`, which counts the GTX quotes that would have
crossed; the first one is said in a `UserWarning` (the price, the side, the
levels read, the way out: `book_levels`, or `assumed_queue`), once per run, and
in `res.warnings`. A sweep refuses it in the combinations that met it, each
counted in its own `order_activity`, and says the first in a single
`UserWarning` for the whole call. With an `assumed_queue` the quote rests behind
it and nothing is refused (`book_unknown_at_post` counts them).
At the wake-up these combine with arithmetic, comparisons, `&`, `|`, `~`,
`mbt.when`, the element-wise functions of `manifoldbt.indicators` (`abs_val`,
`min_val`, `max_val`, `sqrt`, `log`, `exp`, ...), `.round_to(step)` /
@@ -2855,7 +3030,7 @@ way, and the venue's own record beside it.
| `res.metrics` | The metrics of every run, as returns on `initial_capital`, and what the run ended with in money: `final_equity` and `pnl`. On a run of one day, as above, the ones taken from daily returns (Sharpe, Sortino, volatility, CAGR, Calmar...) are NaN and `res.warnings` says why: see [Metrics Reference](#metrics-reference) |
| `res.trades_df()` | One row per fill |
| `res.equity_df()`, `res.positions_df()` | At `output_resolution` |
| `res.order_activity` | `orders_posted`, `requotes` (a post that replaced a live order of the same quote), `expired_unfilled` (orders cancelled with nothing filled), with a latency `stale_fills` and `overlapping_live_orders`, and when they are not zero `post_only_rejected` (GTX quotes refused on arrival because they would have crossed) and `levels_snapped` (quotes between two ticks, posted on the passive side; `res.warnings` says so too) |
| `res.order_activity` | `orders_posted`, `requotes` (a post that replaced a live order of the same quote), `expired_unfilled` (orders cancelled with nothing filled), with a latency `stale_fills` and `overlapping_live_orders`, and when they are not zero `post_only_rejected` (GTX quotes refused on arrival because they would have crossed), `refused_beyond_levels` (quotes refused for a price past the levels read, with no `assumed_queue`; the run goes on) and `levels_snapped` (quotes between two ticks, posted on the passive side; `res.warnings` says so too) |
| `res.fill_fragility` | What the queue decided, as for a bar run under [Queue Model](#queue-model): `maker_fills`, `queue` (`queue_decided_fills`, `traverse_fills`, `fills_from_book_cross`, `partial_fills`, `book_unknown_at_post`: read it to know how many quotes stood behind `assumed_queue`), `would_fill_touch`, `would_fill_traverse` (per life of an order at one level). `touch_only_fills` is a bar convention and stays `0`. A sweep's `service_rate` and `queue_decided_fills` columns read it |
| `res.fill_marks` | The markouts of [Marking a Fill](#marking-a-fill), counted from the fill itself (`anchor == "fill"`) |
| `res.orders_df()`, `res.fills_df()`, `res.events_df()` | The tables of `mbt.sim`: every order, every fill with its `channel`, the order journal. They speak the venue's words, those of the order API: `qty`, `fee`, `price`, `side` as `"buy"` or `"sell"`; `res.trades_df()` keeps the trade log's own, the same for every run: `quantity`, `fees`, `fill_price`, `side` as 1 or 2 |
@@ -3042,7 +3217,9 @@ them).
A busy stock's book goes thousands of levels deep, down to orders parked at a
hundredth of a cent and at the price cap: it is read **1000 levels deep** unless
`levels=` (or `BacktestConfig.book_levels`) says otherwise, as a book stored a
thousand levels deep.
thousand levels deep. A side that shows fewer levels than are read shows every
level it has: an order resting past its last level, still within the levels
read, joins an empty queue, whatever the market does after it.
### The exact queue
@@ -3073,7 +3250,10 @@ replacement going to the back.
`queue_ahead` (`sim.order(id).queue_ahead`, `bt.queue_ahead(side)` in a
quote) is the volume ahead of the order exactly, ours included, as the feed
shows it at the strategy's clock: `feed` behind the venue, from the moment the
acknowledgement has told the strategy which order is its own.
acknowledgement has told the strategy which order is its own. Each answer
about the order (a fill, a replacement that keeps its place) brings its own
value as it arrives, and the feed never takes the reading back to an older one:
it moves on again once it has caught up with that answer.
### Naming the queue
@@ -3166,6 +3346,11 @@ on AAPL, 63 ms later on QQQ), except on QQQ with 100 shares: a partial fill.
- An order resting past the levels read, on a side that shows as many as it
can: the volume there is not read. Read more levels, or quote within them.
The order alone is refused (`rejected`, or `replace_rejected` for a
replacement, which leaves its order as it was), with a `UserWarning` once per
run and market; the run goes on. A side that shows fewer levels than are read
shows every level it has: an order past its last one is not refused, it
joins an empty queue.
- `"fifo"` named on a book stored by price, or in a run on bars or on the
trade clock.
- A range with no event of the symbol in it: ingest the day first.
@@ -3198,6 +3383,11 @@ mbt.FeeConfig(
)
```
A negative `maker_fee_bps` or `taker_fee_bps` is a rebate (an exchange that pays
its makers): the fee of a fill is negative and the cash is credited with it.
`min_fee` is the floor of a commission, a fee of zero or more; it never lifts a
rebate. A run whose fees are all zero or more is unchanged.
### Slippage
```python
pyproject.toml
+1 -1
View File
@@ -5,7 +5,7 @@
[project]
name = "manifoldbt"
version = "0.27.1"
version = "0.27.2"
description = "Rust-powered backtesting engine for quantitative research"
readme = "README.md"
requires-python = ">=3.9"
python/manifoldbt/_native.pyi
+35 -4
View File
@@ -204,7 +204,22 @@ class Market:
tick_size: float,
states: List[Tuple[int, List[Tuple[float, float]], List[Tuple[float, float]]]],
trades: Optional[List[Tuple[int, float, float, str]]] = None,
) -> Market: ...
) -> Market:
"""A market written out by hand: ``states`` as ``(ts_ns, bids, asks)``
with each side a list of ``(price, size)`` from the best, ``trades`` as
``(ts_ns, price, qty, "buy" | "sell")``.
Refused by name: states or prints out of time order, levels out of
price order, a price or a size that is not finite and positive, and a
``tick_size`` that is neither ``0`` (no tick) nor a multiple of
``1e-8``, the grid the venue compares prices on. Not
checked, and accepted: a crossed or locked book (an order that crosses
it executes at once against the displayed opposite side); prints and no
book state (every book read is empty and a market order expires); a
level off the tick grid (``Book.levels()`` shows it as written, while an
order priced there is put on the grid, on its passive side, and does not
queue behind that level)."""
...
@property
def name(self) -> str: ...
@property
@@ -281,9 +296,25 @@ class Sim:
def post(
self, symbol: Union[str, int], side: str, price: Optional[float], qty: float,
tif: str = "GTC", expire_at: Optional[int] = None,
) -> int: ...
def cancel(self, id: int) -> None: ...
def replace(self, id: int, price: Optional[float] = None, qty: Optional[float] = None) -> None: ...
) -> int:
"""Send an order and get its id. ``side``, ``tif`` and the symbol are
read without regard to case. ``price=None`` is a market order; a price
is put on the tick grid first. A GTD order whose ``expire_at`` is at or
before its arrival expires without executing. A post after the data is
over is accepted: the order reaches the venue only if the clock moves
again, and stays ``in_flight`` otherwise."""
...
def cancel(self, id: int) -> None:
"""Send a cancellation. A cancellation that reaches the venue before its
order (a cancel latency shorter than the entry latency) is refused, and
the order then rests and can be served."""
...
def replace(self, id: int, price: Optional[float] = None, qty: Optional[float] = None) -> None:
"""Change the price, the total size, or both. The same price with a size
smaller or unchanged keeps the order's place; anything else sends it to
the back. A total size that does not exceed what filled cancels the rest:
the order ends ``cancelled``, ``qty`` unchanged."""
...
class SimResult:
symbols: List[str]
python/manifoldbt/config.py
+7 -2
View File
@@ -784,6 +784,10 @@ class FeeConfig:
(named fee schedules) plus ``symbol_venue`` (which symbol trades where).
Single-venue configs are unchanged — leaving ``per_venue``/``symbol_venue``
empty serializes to the exact same JSON as before.
A negative ``maker_fee_bps`` / ``taker_fee_bps`` is a rebate: the fee of a
fill is negative and credits the cash. ``min_fee`` floors a commission (a
fee of zero or more) and never lifts a rebate.
"""
maker_fee_bps: float = 0.0
@@ -1190,8 +1194,9 @@ class BacktestConfig:
quotes, the fill marks, and the book a quoting strategy reads at each
wake-up (``bt.book.bid_price_at(k)``, ``bt.book.bid_levels()``). Past it
the run knows nothing, as past the depth stored: an order resting further
out joins a queue the book cannot measure (``assumed_queue`` then, or a
refusal by name), and a book column naming a deeper level is refused
out joins a queue the book cannot measure (``assumed_queue`` then, or, for a
quote, that quote refused alone and the run goes on, counted in
``order_activity["refused_beyond_levels"]``), and a book column naming a deeper level is refused
before anything is read. A bound is for speed: on a BTCUSDT day stored
200 levels deep as whole ladders, a maker quoting three levels a side
every 200 ms runs in 0.75 to 1.3 s reading every level and about 0.55 s
python/manifoldbt/result.py
+4 -2
View File
@@ -447,8 +447,10 @@ class QuoteResult(Result):
``trades`` per fill, ``metrics`` (returns on the capital, and in money
``final_equity`` and ``pnl``), ``order_activity`` (orders posted,
requotes, orders cancelled unfilled; under a latency ``stale_fills`` and
``overlapping_live_orders``, counted per quote; ``post_only_rejected`` and
``levels_snapped`` when not zero), ``fill_fragility`` (what the queue
``overlapping_live_orders``, counted per quote; ``post_only_rejected``,
``refused_beyond_levels`` (quotes refused alone for a price past the
levels read, with no ``assumed_queue``) and ``levels_snapped`` when not
zero), ``fill_fragility`` (what the queue
decided: ``maker_fills`` and the ``queue`` counters, among them
``book_unknown_at_post``, the quotes posted behind ``assumed_queue``) and
``fill_marks`` (the markouts of every fill, counted from the fill itself).
python/manifoldbt/sim.py
+90 -17
View File
@@ -31,9 +31,22 @@ What the venue does:
crossed at once by the very liquidity it just took;
* an order that would not execute rests at the back of its level's queue,
behind the size the book shows there and behind the strategy's own earlier
orders at that price;
orders at that price (one of them that leaves or shrinks moves the orders
behind it up by what it held);
* a print at the price of a resting order burns its queue only from the side
that consumes it (a selling aggressor for a bid, a buying one for an ask);
a print strictly through the price serves the rest, whichever side crossed;
* at one venue instant the market comes first: the prints, then the book state
that shows them, then the actions that arrive at that instant.
that shows them, then the actions that arrive at that instant. A print
stamped at a GTD order's ``expire_at`` still serves it;
* a cancellation that reaches the venue before its order (a cancel latency
shorter than the entry latency) is refused, and the order then rests and
can be served;
* a replacement at the same price keeps the order's place when its size is
smaller or unchanged; a total size that does not exceed what filled cancels
the rest (status ``cancelled``, ``qty`` unchanged);
* a GTD order whose ``expire_at`` is at or before its arrival expires without
executing; a FOK that cannot fill whole ends ``expired``.
A market stored order by order (``bt.ingest_mbo``) serves a resting order in
its exact place in the queue instead: behind the orders that reached its level
@@ -44,6 +57,7 @@ Its book by price is read as any stored book.
from __future__ import annotations
import json
import math
import sys
from dataclasses import dataclass
from datetime import datetime, timezone
@@ -72,14 +86,17 @@ def _duration_ns(what: str, raw: Any) -> int:
f"{what}: Interval.trades() is a simulation clock, not a duration; "
"pass Interval.millis(n), Interval.seconds(n) or nanoseconds"
)
nanos = _interval_to_nanos(raw)
try:
nanos = _interval_to_nanos(raw)
except (ValueError, OverflowError) as err:
raise ValueError(f"{what} must be a finite duration, got {raw!r} ({err})") from None
if nanos is None and hasattr(raw, "total_seconds") and hasattr(raw, "microseconds"):
nanos = (raw.days * 86_400 + raw.seconds) * 1_000_000_000 + raw.microseconds * 1_000
if nanos is None:
if isinstance(raw, bool) or not isinstance(raw, int):
raise TypeError(
f"{what} takes a duration: Interval.millis(n), Interval.seconds(n), "
"a datetime.timedelta, or a number of nanoseconds"
f"{what} takes a duration, got {raw!r}: Interval.millis(n), "
"Interval.seconds(n), a datetime.timedelta, or a number of nanoseconds"
)
nanos = raw
if nanos < 0:
@@ -87,6 +104,17 @@ def _duration_ns(what: str, raw: Any) -> int:
return int(nanos)
def _finite(what: str, raw: Any) -> float:
"""``raw`` as a float, refused by name when it is NaN or infinite."""
try:
value = float(raw)
except (TypeError, ValueError):
raise TypeError(f"{what} takes a number, got {raw!r}") from None
if not math.isfinite(value):
raise ValueError(f"{what} must be a finite number, got {raw!r}")
return value
def _instant_ns(what: str, raw: Any) -> int:
"""Nanoseconds since the epoch, UTC, behind a date, an instant or an int."""
from manifoldbt.helpers import date_to_ns
@@ -128,11 +156,20 @@ class Config:
stored by price). Left out: the exact queue on a market stored
order by order, ``risk_adverse`` reading the stored book on the
others. Named, it is the model of every market of the run.
``depth_source="assumed"`` makes the queue ``assumed_queue``
times the order, whatever the book shows: it is not brought down
to the displayed size, which is the rule of ``"book"``. ``"power"``
and ``"log"`` read the depth at the order's level, so they need
``depth_source="book"``: with ``"assumed"`` the run is refused by
name before it starts.
maker_fee_bps: Fee on a fill of a resting order, in basis points of the
notional.
taker_fee_bps: Fee on a fill at arrival.
notional. Negative is a rebate: the fee is negative and credits the
cash.
taker_fee_bps: Fee on a fill at arrival; negative is a rebate too.
initial_cash: Cash at the start.
seed: Seed of the draws a probabilistic queue model makes.
seed: Accepted and passed to the venue, but no queue model draws from
it: ``risk_adverse``, ``power``, ``log`` and the exact queue are
deterministic, so two seeds give the same run to the bit.
"""
latency: Optional[Dict[str, Any]] = None
@@ -150,17 +187,23 @@ class Config:
f"unknown latency key(s) {sorted(unknown)}: bt.sim.Config.latency takes "
'"entry", "cancel", "response" and "feed"'
)
if self.seed < 0:
raise ValueError(f"bt.sim.Config.seed must be zero or more, got {self.seed!r}")
body = {
"latency": {
k: _duration_ns(f"latency.{k}", latency[k]) for k in _LATENCY_KEYS if k in latency
},
"maker_fee_bps": float(self.maker_fee_bps),
"taker_fee_bps": float(self.taker_fee_bps),
"initial_cash": float(self.initial_cash),
"maker_fee_bps": _finite("bt.sim.Config.maker_fee_bps", self.maker_fee_bps),
"taker_fee_bps": _finite("bt.sim.Config.taker_fee_bps", self.taker_fee_bps),
"initial_cash": _finite("bt.sim.Config.initial_cash", self.initial_cash),
"seed": int(self.seed),
}
if self.queue is not None:
body["queue"] = dict(self.queue)
queue = dict(self.queue)
for key, value in queue.items():
if isinstance(value, float):
_finite(f"bt.sim.Config.queue[{key!r}]", value)
body["queue"] = queue
return json.dumps(body)
@@ -193,10 +236,18 @@ def run(
already built, from :meth:`Market.from_store` or written out by
hand with :meth:`Market.from_ladders`.
levels: Levels per side of the stored book to load: every stored
level when left out; at most that many otherwise, all of them when
the book is stored shallower. A level past it reads as absent,
as past the stored depth, and an order resting there joins a queue
the venue cannot measure (``assumed_queue``).
level when left out (1000 for a market stored order by order); at
most that many otherwise, all of them when the book is stored
shallower. A level past it reads as absent, as past the stored
depth, and an order resting there joins a queue the venue cannot
measure (``assumed_queue``). Without ``assumed_queue``, that order
is refused on its own and the run goes on: a new order ends
``rejected``, a replacement is answered ``replace_rejected`` and
the order stays as it was. The first order refused on each market
is said in a ``UserWarning`` (the price, the side, the levels
read). On a market stored order by order, a side that shows fewer
levels than are read shows every level it has: an order past its
last level joins an empty queue.
by_day: Hold each market a UTC day at a time (``True``), whole
(``False``), or day by day when ``[start, end)`` spans more than
one day (``"auto"``, the default). Held day by day, the engine
@@ -355,7 +406,23 @@ class SimResult:
the orders posted behind ``assumed_queue``), ``would_fill_touch`` and
``would_fill_traverse`` (per life of an order at one level, whether a
print reached it or went through it). ``touch_only_fills`` is a bar
convention and stays 0."""
convention and stays 0.
Units differ. ``queue_decided_fills``, ``traverse_fills`` (the
``traverse`` and the ``book_cross`` fills) and ``fills_from_book_cross``
count fills, and the first two add up to ``maker_fills``;
``partial_fills`` and ``book_unknown_at_post`` count orders;
``would_fill_touch`` and ``would_fill_traverse`` count lives. An order
served in several pieces is several fills but one life, so fills cannot
be set against lives. On a market stored by price, with
``orders_served`` the orders that have a maker fill::
would_fill_traverse <= orders_served
<= would_fill_touch + fills_from_book_cross
and ``orders_served <= queue_decided_fills + traverse_fills``. A market
stored order by order is served from its order events, not from the
prints these two counters read: no such envelope holds there."""
return self._raw.fill_fragility
@property
@@ -421,6 +488,10 @@ class SimResult:
volume and notional traded, fees, final equity, P&L against the
initial cash, and the deepest drawdown of the equity samples.
``orders_filled`` counts the orders served at least in part, not the
orders whose status is ``filled``: one filled in part and then
cancelled or expired is counted.
Counted in the engine, once: a new dict at each read, from totals kept
after the first."""
totals = self._totals
@@ -447,6 +518,8 @@ class SimResult:
}
def summary(self) -> str:
"""One line: orders, the orders served at least in part ('N filled'),
fills, volume, P&L and fees."""
m = self.metrics
return (
f"{m['orders']} orders, {m['orders_filled']} filled; {m['fills']} fills "
python/manifoldbt/strategy.py
+6 -1
View File
@@ -281,7 +281,12 @@ class Strategy:
counted in ``order_activity["levels_snapped"]``.
An order that has filled, been cancelled or been refused frees its
quote without a cancellation. The expressions may read the batch
quote without a cancellation. A quote at a price past the levels the
book was read to, with no ``fill_model.queue.assumed_queue``, is
refused on its own and the run goes on: it ends ``rejected``, is
counted in ``order_activity["refused_beyond_levels"]`` and the first
one is said in a ``UserWarning`` (once per run, and once per sweep
call). The expressions may read the batch
(the book columns ``bt.book.*``, the strategy's own signals, params)
and the state at the wake-up: ``position()``, ``cash()``,
``live_qty(side)``, ``order_age(side)``, ``order_price(side)``,
python/tests/test_frais_negatifs.py
+123
View File
@@ -0,0 +1,123 @@
"""A negative fee rate is a rebate, on both doors.
`maker_fee_bps` / `taker_fee_bps` below zero used to be accepted and the fee
brought back to zero by the `min_fee` floor. A rebate now gives a negative fee
(the cash is credited), and `min_fee` floors a commission only. A run with
fees of zero or more is unchanged.
"""
import os
import pytest
pd = pytest.importorskip("pandas")
np = pytest.importorskip("numpy")
import manifoldbt as bt # noqa: E402
from manifoldbt.expr import col, lit, when # noqa: E402
from manifoldbt.helpers import Interval, Slippage # noqa: E402
from manifoldbt.indicators import close as close_px, sma # noqa: E402
MS = 1_000_000
POST, END = 2 * MS, 10 * MS
CAPITAL = 100_000.0
START = pd.Timestamp("2021-03-01", tz="UTC")
def _sim_layer_open():
try:
bt.sim.Market.from_ladders("X", 0.1, [(0, [(1.0, 1.0)], [(2.0, 1.0)])])
except PermissionError:
return False
return True
def _fills(maker_bps, taker_bps):
market = bt.sim.Market.from_ladders(
"BTCUSDT",
0.1,
[(MS, [(100.0, 5.0), (99.9, 9.0)], [(100.1, 5.0), (100.5, 6.0)])],
[(3 * MS, 99.9, 10.0, "sell")],
)
def strategy(sim):
sim.elapse(POST - sim.now())
sim.post("BTCUSDT", "buy", 100.0, 1.0) # rests, served by the print
sim.elapse(2 * MS)
sim.post("BTCUSDT", "sell", None, 2.0, tif="IOC") # takes at arrival
sim.elapse(END - sim.now())
cfg = bt.sim.Config(maker_fee_bps=maker_bps, taker_fee_bps=taker_bps, initial_cash=1_000.0)
return bt.sim.run(strategy, cfg, markets=[market])
@pytest.mark.skipif(not _sim_layer_open(), reason="order-level simulation locked")
def test_a_rebate_credits_the_cash_in_bt_sim():
paid = _fills(1.0, 5.0)
rebated = _fills(-1.0, -5.0)
fees = [round(f, 6) for f in rebated.fills_df(backend="pandas")["fee"]]
assert fees == [-0.01, -0.1]
assert round(rebated.fees, 6) == -0.11
# The same fills, the fees turned around: the cash moves by twice them.
assert round(rebated.cash - paid.cash, 6) == round(2 * (0.01 + 0.1), 6)
# A commission is what it was.
assert [round(f, 6) for f in paid.fills_df(backend="pandas")["fee"]] == [0.01, 0.1]
def _store(tmp_path, rows=3000):
rng = np.random.default_rng(5)
close = 100.0 * np.exp(np.cumsum(rng.normal(0.0, 6e-4, rows)))
open_ = np.empty(rows)
open_[0] = 100.0
open_[1:] = close[:-1]
wick = rng.uniform(0.2, 1.8, rows) * 3e-4 * close
frame = pd.DataFrame({
"timestamp": pd.date_range(START, periods=rows, freq="1min"),
"open": open_,
"high": np.maximum(open_, close) + wick,
"low": np.minimum(open_, close) - wick,
"close": close,
"volume": np.full(rows, 1_000.0),
})
root = tmp_path / "store"
os.makedirs(root, exist_ok=True)
return bt.import_dataframe(
frame, symbol="REB", symbol_id=1, interval="1m",
data_root=os.path.join(root, "data"),
metadata_db=os.path.join(root, "meta.sqlite"),
), rows
def _run(store, rows, fees):
config = bt.BacktestConfig(
universe=[1],
time_range_start=int(START.value),
time_range_end=int(START.value) + rows * 60_000_000_000,
bar_interval=Interval.minutes(1),
initial_capital=CAPITAL,
execution=bt.ExecutionConfig(
signal_delay=0, execution_price="AtClose", max_position_pct=1.0,
position_sizing_mode="FractionOfEquity",
),
fees=fees,
slippage=Slippage.none(),
warmup_bars=0,
)
sized = when(col("fast") > col("slow"), lit(1.0), lit(0.0))
strategy = (bt.Strategy.create("reb")
.signal("fast", sma(close_px, 5))
.signal("slow", sma(close_px, 20))
.size(sized))
return bt.run(strategy, config, store).metrics["total_return"]
def test_a_rebate_pays_in_bt_run_and_the_minimum_floors_a_commission_only(tmp_path):
store, rows = _store(tmp_path)
free = _run(store, rows, bt.FeeConfig.zero())
paid = _run(store, rows, bt.FeeConfig(taker_fee_bps=5.0, maker_fee_bps=5.0))
rebate = _run(store, rows, bt.FeeConfig(taker_fee_bps=-5.0, maker_fee_bps=-5.0))
assert paid < free < rebate
# The minimum is a floor under a commission: it never lifts a rebate.
floored = _run(store, rows, bt.FeeConfig(taker_fee_bps=-5.0, maker_fee_bps=-5.0, min_fee=500.0))
assert floored == rebate
# And a commission below the minimum (50 on a trade of 100 000) is raised to it.
assert _run(store, rows, bt.FeeConfig(taker_fee_bps=5.0, maker_fee_bps=5.0, min_fee=500.0)) < paid
python/tests/test_quote_dsl.py
+20 -4
View File
@@ -470,14 +470,30 @@ def test_a_quoting_run_reports_its_queue_and_its_refusals():
@needs_layer
def test_a_quote_deeper_than_the_levels_read_is_refused_as_such():
def test_a_quote_deeper_than_the_levels_read_is_refused_alone_and_the_run_goes_on():
deep = bt.Strategy.create("profond").quote("buy", book.bid_price_at(1) - 3.0, 0.01)
config = quote_config(execution={"fill_model": {"queue": {"depth_source": "book"}}})
with pytest.raises(Exception, match="levels read from the store") as err:
run_on(walk_market(), deep, config)
assert "ingest the book" not in str(err.value)
with pytest.warns(UserWarning, match="the stored book says nothing at") as seen:
res = run_on(walk_market(), deep, config)
# Told once for the run, with the words bt.sim uses: the price, the side,
# what to do about it.
told = [w for w in seen if "the stored book says nothing at" in str(w.message)]
assert len(told) == 1
assert "the run goes on" in str(told[0].message)
assert "fill_model.queue.assumed_queue" in str(told[0].message)
# The count is the result's, apart from the post-only refusals; each quote
# refused is an order that ended rejected.
activity = res.order_activity
statuses = res.orders_df(backend="dict")["status"]
assert activity["refused_beyond_levels"] == statuses.count("rejected") > 0
assert "post_only_rejected" not in activity
assert activity["orders_posted"] >= activity["refused_beyond_levels"]
assert any("refused_beyond_levels" in w for w in res.warnings)
assert res.fill_fragility["maker_fills"] == 0
# With the declared fallback it rests behind the assumed queue, no warning.
res = run_on(walk_market(), deep, quote_config())
assert res.fill_fragility["queue"]["book_unknown_at_post"] > 0
assert "refused_beyond_levels" not in res.order_activity
# ---------------------------------------------------------------------------
python/tests/test_quote_sweep.py
+44
View File
@@ -13,6 +13,7 @@ file checks the Python door: the routing of ``run_sweep``, ``run_sweep_lite``,
"""
import json
import os
import warnings
import pytest
@@ -179,6 +180,49 @@ def test_a_batch_of_strategies_that_quote_is_each_run_alone():
assert json.dumps(l.metrics, sort_keys=True) == json.dumps(raw0.metrics, sort_keys=True)
def deep_maker():
"""One quote ``off`` below the best bid: readable at 0, past the levels
read once ``off`` is large enough."""
off = bt.param("off", default=0.0)
return bt.Strategy.create("deep").quote("buy", book.bid_price_at(1) - off, 0.01)
@needs_layer
@pytest.mark.parametrize("lite", [False, True])
def test_a_sweep_refuses_a_deep_quote_alone_in_its_combination_and_says_it_once(lite):
market = walk_market(seed=11, n=1_500)
no_fallback = config(execution=bt.ExecutionConfig(
fill_model={"queue": {"depth_source": "book"}}))
grid = {"off": [{"Float64": v} for v in (0.0, 3.0, 5.0, 8.0)]}
fn = _native.run_quote_sweep_lite if lite else _native.run_quote_sweep
with pytest.warns(UserWarning, match="the stored book says nothing at") as seen:
out = fn(deep_maker().to_json(), json.dumps(grid), no_fallback.to_json(),
None, [market], 2, "{}")
# One for the call, not one per combination.
told = [w for w in seen if "the stored book says nothing at" in str(w.message)]
assert len(told) == 1
assert len(out) == 4
if lite:
# The metrics are all a lite result keeps, and the warnings.
counts = [any("refused_beyond_levels" in w for w in r.warnings) for r in out]
else:
counts = [r[0].order_activity.get("refused_beyond_levels", 0) > 0 for r in out]
assert counts == [False, True, True, True]
if not lite:
# Each combination is the run alone, to the bit, refusals included.
for i, off in enumerate((0.0, 3.0, 5.0, 8.0)):
with warnings.catch_warnings():
warnings.simplefilter("ignore")
raw, sim = _native.run_quotes(
deep_maker_at(off).to_json(), no_fallback.to_json(), None, [market])
assert raw.order_activity == out[i][0].order_activity
assert journal(sim) == journal(out[i][1])
def deep_maker_at(off):
return bt.Strategy.create("deep").quote("buy", book.bid_price_at(1) - off, 0.01)
@needs_layer
def test_the_sweeps_refuse_by_name(tmp_path):
store = bt.DataStore(data_root=str(tmp_path / "data"),
python/tests/test_sim.py
+137
View File
@@ -329,6 +329,67 @@ def test_a_malformed_run_is_refused_before_it_starts():
bt.sim.Market.from_ladders("X", 0.1, [_ladder(2 * MS, 1.0), _ladder(MS, 1.0)])
def test_a_refusal_names_the_parameter_and_the_value(tmp_path):
market = _market([_ladder(MS, 5.0)])
nan, inf = float("nan"), float("inf")
for field, value in (
("maker_fee_bps", nan),
("taker_fee_bps", inf),
("initial_cash", nan),
):
config = bt.sim.Config(**{field: value})
with pytest.raises(ValueError, match=rf"Config\.{field} must be a finite number, got {value}"):
bt.sim.run(lambda s: None, config, markets=[market])
with pytest.raises(ValueError, match="assumed_queue.*finite number, got nan"):
bt.sim.run(
lambda s: None,
bt.sim.Config(queue={"model": "risk_adverse", "assumed_queue": nan}),
markets=[market],
)
with pytest.raises(TypeError, match="latency.feed takes a duration, got inf"):
bt.sim.run(lambda s: None, bt.sim.Config(latency={"feed": inf}), markets=[market])
with pytest.raises(ValueError, match="latency.entry must be a finite duration"):
bt.sim.run(
lambda s: None, bt.sim.Config(latency={"entry": bt.Interval.millis(nan)}), markets=[market]
)
with pytest.raises(ValueError, match=r"Config\.seed must be zero or more, got -1"):
bt.sim.run(lambda s: None, bt.sim.Config(seed=-1), markets=[market])
# An id no `post` returned is an order never sent, not an OverflowError.
def strategy(sim):
for call in (lambda: sim.cancel(-1), lambda: sim.replace(-1, price=100.0)):
with pytest.raises(ValueError, match="no order -1 was sent"):
call()
with pytest.raises(ValueError, match="no order 36893488147419103232 was sent"):
sim.cancel(2**65)
assert sim.order(-1) is None
with pytest.raises(ValueError, match="levels.*0 or more, got -1"):
sim.book("BTCUSDT").levels(-1)
assert sim.book("BTCUSDT").levels(0) == ([], [])
bt.sim.run(strategy, markets=[market])
# `levels` of a stored market is a number of levels, 1 or more.
store = bt.DataStore(
data_root=str(tmp_path / "data"), metadata_db=str(tmp_path / "meta.sqlite")
)
for levels in (-1, 0):
with pytest.raises(ValueError, match=rf"levels must be at least 1.*got {levels}"):
bt.sim.Market.from_store(store, "BTCUSDT", 0, MS, levels)
def test_a_tick_finer_than_the_price_grid_is_refused_by_name():
# The venue compares prices on a grid of 1e-8: a tick of 1e-9 would make
# an order take a level beyond its limit.
state = (MS, [(9.0e-8, 5.0)], [(1.0e-7, 5.0), (1.1e-7, 5.0)])
for tick in (1e-9, 1.5e-8):
with pytest.raises(ValueError, match=r"tick size .*1e-8"):
bt.sim.Market.from_ladders("T", tick, [state])
# No tick, and ticks on the grid, stand.
for tick in (0.0, 1e-8, 0.07, 0.1, 1.0):
bt.sim.Market.from_ladders("T", tick, [_ladder(MS, 5.0)])
# ---------------------------------------------------------------------------
# A stored market
# ---------------------------------------------------------------------------
@@ -356,3 +417,79 @@ def test_a_stored_day_runs_end_to_end():
res = bt.sim.run(strategy, store=store, symbols=[int(sid)], start=day, end=nxt)
assert posted, "the stored day never showed a book"
assert res.metrics["orders"] == len(posted)
# ---------------------------------------------------------------------------
# An order past the levels read refuses itself, not the run
# ---------------------------------------------------------------------------
def test_an_order_past_the_levels_read_is_refused_alone_with_a_warning():
out = {}
def strategy(sim):
sim.elapse(POST - sim.now())
out["far"] = sim.post("BTCUSDT", "buy", 90.0, 1.0) # 3 levels read: 99.0 is the last
out["near"] = sim.post("BTCUSDT", "buy", 100.0, 1.0)
out["again"] = sim.post("BTCUSDT", "buy", 80.0, 1.0)
sim.elapse(END - sim.now())
out["status"] = {k: sim.order(out[k]).status for k in ("far", "near", "again")}
with pytest.warns(UserWarning, match=r"90.*Bid.*beyond the 3 level.*assumed_queue") as seen:
res = bt.sim.run(strategy, markets=[_market([_ladder(MS, 5.0)])])
# Said once per run and market, for the first order refused.
assert len([w for w in seen if "beyond the 3 level" in str(w.message)]) == 1
assert out["status"] == {"far": "rejected", "near": "resting", "again": "rejected"}
events = res.events_df(backend="pandas")
assert list(events[events["order_id"] == out["far"]]["event"]) == ["sent", "rejected"]
def test_a_replacement_past_the_levels_read_is_refused_and_the_order_stays():
out = {}
def strategy(sim):
sim.elapse(POST - sim.now())
out["id"] = sim.post("BTCUSDT", "buy", 100.0, 1.0)
sim.elapse(MS)
sim.replace(out["id"], price=90.0)
sim.elapse(END - sim.now())
out["order"] = sim.order(out["id"])
with pytest.warns(UserWarning, match="replacement of order"):
res = bt.sim.run(strategy, markets=[_market([_ladder(MS, 5.0)], [(6 * MS, 100.0, 20.0, "sell")])])
assert (out["order"].status, out["order"].price, round(out["order"].filled, 6)) == ("filled", 100.0, 1.0)
events = res.events_df(backend="pandas")
kinds = list(events[events["order_id"] == out["id"]]["event"])
assert "replace_rejected" in kinds and "replaced" not in kinds
def test_an_order_past_the_levels_read_joins_the_assumed_queue_without_a_warning(recwarn):
def strategy(sim):
sim.elapse(POST - sim.now())
sim.post("BTCUSDT", "buy", 90.0, 1.0)
sim.elapse(END - sim.now())
cfg = bt.sim.Config(queue={"depth_source": "book", "assumed_queue": 2.0})
bt.sim.run(strategy, cfg, markets=[_market([_ladder(MS, 5.0)])])
assert not [w for w in recwarn if "beyond" in str(w.message)]
def test_queue_ahead_is_that_of_the_last_answer_after_a_fill_and_after_a_replacement():
out = {}
def strategy(sim):
sim.elapse(POST - sim.now())
oid = sim.post("BTCUSDT", "buy", 100.0, 2.0)
sim.elapse(MS)
out["posted"] = sim.order(oid).queue_ahead
sim.elapse(3 * MS) # a print of 2 at 100.0 burns the queue to 3
sim.replace(oid, qty=1.5) # smaller, same price: keeps its place
sim.elapse(MS)
out["replaced"] = sim.order(oid).queue_ahead
sim.elapse(END - sim.now()) # a print of 6 serves 1.0 of it
o = sim.order(oid)
out["filled"] = (round(o.filled, 6), o.queue_ahead)
trades = [(5 * MS, 100.0, 2.0, "sell"), (8 * MS, 100.0, 6.0, "sell")]
bt.sim.run(strategy, markets=[_market([_ladder(MS, 5.0)], trades)])
assert out == {"posted": 5.0, "replaced": 3.0, "filled": (1.5, 0.0)}
python/tests/test_sim_file.py
+180
View File
@@ -0,0 +1,180 @@
"""``bt.sim``: the queue of a resting order on a book stored by price.
The rules are pinned by the engine's own tests (``bt-sim/tests/lifecycle.rs``,
``bt-core``); this file checks that Python sees the same verdicts: our own
orders at one price, the probabilistic queues with two of them, the prints that
explain a fall of the level, the assumed queue as a swept axis, and what is
refused by name.
Every market is written out by hand (``bt.sim.Market.from_ladders``), on a 0.1
tick, bids 100.0 (the size under test), 99.9 x 9 and 99.0 x 9, asks 100.1 x 5
and 100.5 x 6.
"""
import pytest
import manifoldbt as bt
MS = 1_000_000
def _layer_open():
try:
bt.sim.Market.from_ladders("X", 0.1, [(0, [(1.0, 1.0)], [(2.0, 1.0)])])
except PermissionError:
return False
return True
pytestmark = pytest.mark.skipif(
not _layer_open(),
reason="order-level simulation locked (runs in the engine's own debug builds)",
)
def _ladder(ts, size_at_100):
return (ts, [(100.0, size_at_100), (99.9, 9.0), (99.0, 9.0)], [(100.1, 5.0), (100.5, 6.0)])
def _market(states, trades=()):
return bt.sim.Market.from_ladders("BTCUSDT", 0.1, list(states), list(trades))
# ---------------------------------------------------------------------------
# Our own orders at one price
# ---------------------------------------------------------------------------
def _b_behind_a(act, states, a_qty=1.0, sold=2.5):
"""2 stand at 100.0; A then B (1) rest behind them; at 3 ms ``act`` is done
to A, at 4 ms ``sold`` are sold into the bid at 100.0. What B got."""
out = {}
def strategy(sim):
sim.elapse(2 * MS - sim.now())
a = sim.post("BTCUSDT", "buy", 100.0, a_qty)
out["b"] = sim.post("BTCUSDT", "buy", 100.0, 1.0)
sim.elapse(MS)
act(sim, a)
sim.elapse(10 * MS - sim.now())
out["filled"] = sim.order(out["b"]).filled
bt.sim.run(strategy, markets=[_market(states, [(4 * MS, 100.0, sold, "sell")])])
return round(out["filled"], 6)
@pytest.mark.parametrize("grows", [False, True], ids=["quiet", "level grows behind"])
@pytest.mark.parametrize(
"act, a_qty, expected",
[
(lambda s, a: s.cancel(a), 1.0, 0.5),
(lambda s, a: s.replace(a, price=99.9), 1.0, 0.5),
(lambda s, a: s.replace(a, qty=2.0), 1.0, 0.5),
(lambda s, a: s.replace(a, qty=0.4), 1.0, 0.1),
(lambda s, a: None, 1.0, 0.0),
],
ids=["cancelled", "other price", "larger, to the back", "smaller, keeps its place", "stays"],
)
def test_an_order_of_ours_that_leaves_shortens_the_queue_behind_it(act, a_qty, expected, grows):
states = [_ladder(MS, 2.0)] + ([_ladder(5 * MS // 2, 8.0)] if grows else [])
assert _b_behind_a(act, states, a_qty) == expected
# ---------------------------------------------------------------------------
# The probabilistic queues
# ---------------------------------------------------------------------------
PROBABILISTIC = [
{"model": "log"},
{"model": "power", "n": 2.0},
{"model": "power", "n": 3.0},
]
def _two_orders(queue, states, trades):
"""A then B (1 each) post at 100.0 at 2 ms; what each got by 10 ms."""
out = {}
def strategy(sim):
sim.elapse(2 * MS - sim.now())
a = sim.post("BTCUSDT", "buy", 100.0, 1.0)
b = sim.post("BTCUSDT", "buy", 100.0, 1.0)
sim.elapse(10 * MS - sim.now())
out["a"], out["b"] = sim.order(a).filled, sim.order(b).filled
bt.sim.run(strategy, bt.sim.Config(queue=queue), markets=[_market(states, trades)])
return round(out["a"], 6), round(out["b"], 6)
@pytest.mark.parametrize("queue", [{"model": "risk_adverse"}] + PROBABILISTIC, ids=str)
def test_two_of_our_orders_are_not_served_by_the_same_volume(queue):
# 4 shown, the level falls to 2 with no print, 2.5 are sold: 2 stand ahead
# of A at most, and those 2 and A ahead of B.
got = _two_orders(queue, [_ladder(MS, 4.0), _ladder(3 * MS, 2.0)],
[(7 * MS // 2, 100.0, 2.5, "sell")])
assert got == (0.5, 0.0)
@pytest.mark.parametrize("queue", PROBABILISTIC, ids=str)
@pytest.mark.parametrize(
"prints",
[
[(3 * MS, 100.0, 4.0, "sell")],
[(3 * MS, 100.0, 2.0, "sell"), (4 * MS, 100.0, 2.0, "sell")],
[(3 * MS, 100.0, 4.0, "sell"), (4 * MS, 100.1, 0.1, "buy")],
[(5 * MS, 100.0, 1.0, "sell")] * 4,
],
ids=["one print", "two prints", "a print of the other side", "four prints at the state's instant"],
)
def test_a_print_that_explains_a_fall_is_counted_once(queue, prints):
# 10 shown, 4 print at the level, the book then shows 6: the prints explain
# the whole fall, so 6.5 sold afterwards reach the order for 0.5.
out = {}
def strategy(sim):
sim.elapse(2 * MS - sim.now())
out["id"] = sim.post("BTCUSDT", "buy", 100.0, 1.0)
sim.elapse(10 * MS - sim.now())
out["filled"] = sim.order(out["id"]).filled
market = _market([_ladder(MS, 10.0), _ladder(5 * MS, 6.0)], prints + [(6 * MS, 100.0, 6.5, "sell")])
bt.sim.run(strategy, bt.sim.Config(queue=queue), markets=[market])
assert round(out["filled"], 6) == 0.5
# ---------------------------------------------------------------------------
# The assumed queue
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"assumed, expected", [(0.5, 1.0), (3.0, 0.5), (100.0, 0.0), (1000.0, 0.0)]
)
def test_an_assumed_queue_is_not_brought_down_to_the_displayed_size(assumed, expected):
# 1 shown, a later state that leaves the level alone, 3.5 sold at 100.0.
# Whatever the book shows, the order waits behind `assumed` times its size:
# the axis the guide asks to sweep has to move the result.
out = {}
def strategy(sim):
sim.elapse(2 * MS - sim.now())
out["id"] = sim.post("BTCUSDT", "buy", 100.0, 1.0)
sim.elapse(10 * MS - sim.now())
out["filled"] = sim.order(out["id"]).filled
later = (5 * MS // 2, [(100.0, 1.0), (99.9, 8.0), (99.0, 9.0)], [(100.1, 5.0), (100.5, 6.0)])
market = _market([_ladder(MS, 1.0), later], [(4 * MS, 100.0, 3.5, "sell")])
config = bt.sim.Config(queue={"depth_source": "assumed", "assumed_queue": assumed})
bt.sim.run(strategy, config, markets=[market])
assert round(out["filled"], 6) == expected
@pytest.mark.parametrize("model", [{"model": "log"}, {"model": "power", "n": 2.0}], ids=str)
def test_a_probabilistic_queue_on_an_assumed_depth_is_refused_by_name(model):
ran = []
config = bt.sim.Config(queue=dict(model, depth_source="assumed", assumed_queue=3.0))
with pytest.raises(Exception, match=r"queue\.model=.*assumed"):
bt.sim.run(lambda sim: ran.append(1), config, markets=[_market([_ladder(MS, 5.0)])])
assert ran == [], "refused before the strategy is called"
# The historical rule on an assumed queue stays what it was.
ok = bt.sim.Config(queue={"model": "risk_adverse", "depth_source": "assumed", "assumed_queue": 3.0})
bt.sim.run(lambda sim: None, ok, markets=[_market([_ladder(MS, 5.0)])])
python/tests/test_sim_mbo.py
+42
View File
@@ -283,3 +283,45 @@ def test_a_real_day_quoted_in_its_exact_queue(tmp_path):
fills = res.sim.fills_df(backend="pandas")
assert len(fills) > 0
assert res.sim.fill_fragility["queue"]["queue_decided_fills"] > 0
def _three_bid_levels():
return [
add(0, 1, True, 100, 10.00),
add(0, 2, True, 100, 9.99),
add(0, 3, True, 100, 9.98),
add(0, 4, False, 100, 10.01),
add(20, 5, True, 100, 9.97), # a fourth level, after the order
add(40, 6, False, 1, 10.02),
]
def _post_at(price, store, **kw):
out = {}
def strategy(sim):
sim.elapse(OPEN_UTC + 2 * MS - sim.now())
out["id"] = sim.post("XYZ", "buy", price, 5.0)
sim.elapse(OPEN_UTC + 10 * MS - sim.now())
out["order"] = sim.order(out["id"])
res = bt.sim.run(strategy, None, store, symbols=["XYZ"], start=OPEN_UTC,
end=OPEN_UTC + 60 * 1_000 * MS, **kw)
return res, out
def test_a_side_that_shows_fewer_levels_than_are_read_has_nothing_past_its_last_level(tmp_path):
store = _store(tmp_path, _three_bid_levels())
# 1000 levels are read and the bid side shows three: the fourth price is
# empty, whatever the market shows later.
_, out = _post_at(9.97, store)
assert (out["order"].status, out["order"].queue_ahead) == ("resting", 0.0)
def test_an_order_past_the_levels_read_of_a_market_by_order_is_refused_alone(tmp_path):
store = _store(tmp_path, _three_bid_levels())
with pytest.warns(UserWarning, match=r"read 3 levels deep.*9.97"):
res, out = _post_at(9.97, store, levels=3)
assert out["order"].status == "rejected"
events = res.events_df(backend="pandas")
assert list(events[events["order_id"] == out["id"]]["event"]) == ["sent", "rejected"]