diff --git a/README.md b/README.md
index f35169c..0f0450b 100644
--- a/README.md
+++ b/README.md
@@ -8,14 +8,7 @@
- 
-
-
-
- 
- 
- 
- 
+ 
@@ -33,7 +26,7 @@ sequential fill simulation with realistic fees, slippage, funding and look-ahead
## Why ManifoldBT
-- **Fast**: 10M bars in 329 ms. 79x faster than vectorbt, and 311x once you also want drawdown and Sharpe. [Measured in public CI](#performance), every run linked.
+- **Fast**: 10M bars in 317 ms. 78x faster than vectorbt, 308x once you also want drawdown and Sharpe, ~3,500x faster than backtrader. [Measured in public CI](#performance), every run linked.
- **Expressive**: fluent DSL with 30+ indicators, conditional logic, cross-asset references
- **Rigorous**: Monte Carlo, walk-forward, parameter sweeps, lookahead detection, exposure diagnostics
- **Portable**: `pip install`, no Rust toolchain needed. Works on Python 3.9+.
@@ -107,13 +100,26 @@ store = mbt.import_csv("EURUSD_1m.csv", symbol="EURUSD", symbol_id=1,
interval="1m", asset_class="forex")
```
-**Exchange connectors**: Binance, Bybit, Hyperliquid, dYdX, Bitstamp (free); Databento, Massive (Pro):
+**Market data connectors**: Binance, Bybit, Hyperliquid, dYdX, Bitstamp, Yahoo Finance (free);
+Databento, Massive (Pro):
```python
store = mbt.ingest(provider="binance", symbol="BTCUSDT", symbol_id=1,
start="2024-01-01T00:00:00Z", end="2025-01-01T00:00:00Z")
```
+Yahoo Finance covers stocks, ETFs, indices (`^GSPC`), FX (`EURUSD=X`), futures
+(`ES=F`) and crypto (`BTC-USD`) without an API key. Prices are dividend-adjusted,
+like `yfinance`'s `auto_adjust=True`; pass `dataset="raw"` for unadjusted quotes.
+Yahoo's own history limits apply: 1m over the last 30 days, 1h over ~2 years,
+daily back to the listing date.
+
+```python
+store = mbt.ingest(provider="yahoo", symbol="AAPL", symbol_id=1, interval="1d",
+ asset_class="equity",
+ start="2015-01-01T00:00:00Z", end="2026-01-01T00:00:00Z")
+```
+
Or from the CLI:
```bash
@@ -121,59 +127,6 @@ manifoldbt import-csv data.csv --symbol EURUSD --symbol-id 1 --interval 1m
manifoldbt ingest --provider binance --symbol BTCUSDT --symbol-id 1 --start ... --end ...
```
-## Higher timeframes
-
-Declare the timeframes you want alongside the simulation one, then read them
-with `mbt.tf(...)`. Columns are forward-filled onto the simulation grid, and a
-bar's value only becomes readable once that bar has closed, so there is no
-look-ahead.
-
-```python
-config = mbt.BacktestConfig(
- ...,
- bar_interval=Interval.minutes(1), # simulate on 1m
- extra_timeframes={"1h": Interval.hours(1)}, # also resample to 1h
-)
-
-h1 = mbt.tf("1h")
-h1.close # the last closed hourly close, held across the minute bars
-```
-
-For an **indicator** on a higher timeframe, use `.apply(...)`. It evaluates the
-expression on that timeframe's own grid, so the period counts in *its* bars:
-
-```python
-from manifoldbt.indicators import close, sma
-
-band = mbt.tf("1h").apply(sma(close, 20)) # mean of 20 HOURLY closes
-```
-
-> Careful: `sma(mbt.tf("1h").close, 20)` is **not** the same thing. That reads
-> the step-held hourly series on the simulation grid, so the period counts in
-> simulation bars: on a 1m simulation it is a 20-*minute* smoothing of an hourly
-> staircase. Use `.apply(...)` whenever you want an indicator *of* the higher
-> timeframe.
-
-## Sweeping a choice, not just a number
-
-`mbt.param(...)` sweeps numbers. `mbt.choice(...)` sweeps *expressions*: the
-selector becomes a grid axis, and each combination resolves to its branch before
-the simulation runs, so the branches it did not pick cost nothing.
-
-```python
-band = mbt.choice("band", {
- "30m": mbt.tf("30m").apply(sma(close, mbt.param("len"))),
- "1h": mbt.tf("1h").apply(sma(close, mbt.param("len"))),
- "2h": mbt.tf("2h").apply(sma(close, mbt.param("len"))),
-})
-
-sweep = mbt.run_sweep(strategy, {"band": ["30m", "1h", "2h"],
- "len": range(10, 210, 10)}, config, store)
-```
-
-The branches can hold any expression, so the same mechanism sweeps which
-exogenous column to use, which asset to reference, or which indicator to apply.
-
## Examples
| # | Example | What it shows |
@@ -206,30 +159,26 @@ engine from PyPI the way a user would, generates its own data, checks that the
engines produced the **same result**, and only then reports how long each took:
a workload they disagree on gets no published timing at all.
-**Latest run: [#13](https://github.com/manifoldbt/manifoldbt/actions/runs/32469701489)**
-ran on Linux x86_64, 4 vCPU (AMD EPYC 7763), Python 3.12, manifoldbt 0.18.0 /
-vectorbt 0.28.4 / raptorbt 0.9.0, 3 interleaved repetitions, medians reported.
+**Latest run: [#11](https://github.com/manifoldbt/manifoldbt/actions/runs/32396472073)**
+ran on Linux x86_64, 4 vCPU, Python 3.12, manifoldbt 0.17.3 / vectorbt 0.28.4 /
+raptorbt 0.9.0, 3 interleaved repetitions.
| Workload | Bars | ManifoldBT | vectorbt | raptorbt |
|---|---:|---:|---:|---:|
-| SMA crossover | 10M | **327 ms** | 26.12 s (x79) | 913 ms (x2.8) |
-| ...with drawdown, Sharpe, Sortino, volatility | 10M | **329 ms** | 102.38 s (**x311**) | 909 ms (x2.8) |
-| ...with a 5 bps fee and 2 bps slippage | 10M | **337 ms** | 26.08 s (x79) | not supported |
-| EMA + RSI filter, 5 bps fee | 1M | **57 ms** | 2.35 s (x40) | not supported |
-| Five assets in one book | 1M | **148 ms** | 2.54 s (x17) | not supported |
-| Stop-loss and take-profit bracket | 10M | **934 ms** | 26.24 s (x28) | 916 ms (**x1.0**) |
+| SMA crossover | 10M | **317 ms** | 24.75 s (x78) | 878 ms (x2.8) |
+| ...with drawdown, Sharpe, Sortino, volatility | 10M | **317 ms** | 97.46 s (**x308**) | 894 ms (x2.8) |
+| ...with a 5 bps fee and 2 bps slippage | 10M | **316 ms** | 24.53 s (x78) | not supported |
+| EMA + RSI filter, 5 bps fee | 1M | **52 ms** | 2.21 s (x41) | not supported |
+| Five assets in one book | 1M | **140 ms** | 2.34 s (x17) | not supported |
The second row is the one worth reading twice. Asking for a performance summary
costs ManifoldBT nothing measurable, because it computes one during the run
-whether you read it or not, and costs vectorbt 102 seconds, because it defers
-the equity curve until a risk metric needs it and then has to build one.
+whether you read it or not, and costs vectorbt 73 seconds, because it defers the
+equity curve until a risk metric needs it and then has to build one.
-The last two rows are the ones where ManifoldBT does worst, and they are
-published for that reason. Broadcasting a column per asset is close to free for
-vectorbt, while walking five books is not free for anything. And on a
-stop-loss/take-profit bracket, raptorbt is level with us: the intra-bar check
-that decides which of the two triggers first is a sequential walk in both
-engines, so there is no vectorization left to win with.
+The fifth row is the one where ManifoldBT does worst, and it is published for
+that reason: broadcasting a column per asset is close to free for vectorbt,
+while walking five books is not free for anything.
### Parameter sweeps
@@ -254,14 +203,9 @@ The method, the parity gate and the known divergences are written up in
backtrader runs the same EMA(12/26) + RSI(14) strategy on 500K 1-minute bars in
**46,944 ms**, against **13 ms** for ManifoldBT: a factor of **3,556**. Measured
-with `benchmarks/bench_vs_competitors.py`, median of 3 runs, on a developer
-machine and not the CI runner, so it is not comparable line-for-line with the
-table above.
-
-It sits outside the CI suite because its event-driven fills produce a different
-PnL, and the parity gate publishes no timing for engines that did not do the
-same work. Treat it as an order of magnitude, not a benchmark: the two engines
-are not doing the same thing.
+with `benchmarks/bench_vs_competitors.py`, median of 3 runs. It sits outside the
+CI suite because its event-driven fills produce a different PnL, and the parity
+gate publishes no timing for engines that did not do the same work.
### How it compares
@@ -289,7 +233,7 @@ Full API reference, indicator list, configuration guide, and best practices:
| Monte Carlo | 1K sims | Unlimited |
| Walk-Forward | - | Anchored + Rolling |
| Parameter Stability | - | Yes |
-| Crypto connectors (Binance, Bybit, Hyperliquid) | Yes | Yes |
+| Free connectors (Binance, Bybit, Hyperliquid, dYdX, Bitstamp, Yahoo) | Yes | Yes |
| Databento & Massive connectors | - | Yes |
| Safety checks (lookahead, exposure) | - | Yes |
| Tearsheets & export | - | Yes |
diff --git a/examples/00_template.py b/examples/00_template.py
index cc48a23..c5391ca 100644
--- a/examples/00_template.py
+++ b/examples/00_template.py
@@ -1,5 +1,10 @@
"""Strategy template — copy this file and modify.
+Demonstrates:
+ - the minimal shape of a backtest: strategy, config, store, run
+
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/00_template.py
"""
diff --git a/examples/01_trend_following.py b/examples/01_trend_following.py
index 5a87d9c..37be671 100644
--- a/examples/01_trend_following.py
+++ b/examples/01_trend_following.py
@@ -8,6 +8,8 @@ Demonstrates:
- Diagnostics (lookahead, exposure stability, risk)
- result.summary() rich output
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/01_trend_following.py
"""
diff --git a/examples/02_mean_reversion.py b/examples/02_mean_reversion.py
index 602ed29..a2fa579 100644
--- a/examples/02_mean_reversion.py
+++ b/examples/02_mean_reversion.py
@@ -5,6 +5,8 @@ Demonstrates:
- Long and short positions
- Continuous sizing (signal * 0.25)
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/02_mean_reversion.py
"""
diff --git a/examples/03_multi_asset_momentum.py b/examples/03_multi_asset_momentum.py
index ba49b1f..beaf975 100644
--- a/examples/03_multi_asset_momentum.py
+++ b/examples/03_multi_asset_momentum.py
@@ -5,6 +5,8 @@ Demonstrates:
- Momentum via smoothed ROC on 12h bars
- Volatility-adjusted sizing
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/03_multi_asset_momentum.py
"""
diff --git a/examples/04_linear_regression.py b/examples/04_linear_regression.py
index 34a4292..0c62342 100644
--- a/examples/04_linear_regression.py
+++ b/examples/04_linear_regression.py
@@ -11,6 +11,8 @@ The idea: fit a rolling OLS regression on price. When the slope is steep
and the R² is high (price moves in a straight line), we have a strong trend.
Size proportionally to slope strength * R² confidence.
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/04_linear_regression.py
"""
diff --git a/examples/05_stat_arb.py b/examples/05_stat_arb.py
index 2f3f381..8067950 100644
--- a/examples/05_stat_arb.py
+++ b/examples/05_stat_arb.py
@@ -5,6 +5,8 @@ Demonstrates:
- Kalman filter for spread equilibrium
- Z-score mean-reversion sizing
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/05_stat_arb.py
"""
diff --git a/examples/06_full_visualization.py b/examples/06_full_visualization.py
index c533739..f04054f 100644
--- a/examples/06_full_visualization.py
+++ b/examples/06_full_visualization.py
@@ -7,6 +7,13 @@ Strategy:
Demonstrates every plotting function available in manifoldbt.
+Demonstrates:
+ - every plotting function in manifoldbt, on one result
+ - tearsheet, equity, drawdown, monthly and annual returns
+ - rolling Sharpe and volatility, returns histogram
+
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/06_full_visualization.py
"""
diff --git a/examples/07_walk_forward.py b/examples/07_walk_forward.py
index 5b788d1..a133b78 100644
--- a/examples/07_walk_forward.py
+++ b/examples/07_walk_forward.py
@@ -5,6 +5,8 @@ Demonstrates:
- param() for sweep-able parameters
- Walk-forward fold results inspection
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/07_walk_forward.py
"""
diff --git a/examples/08_sweep_2d_heatmap.py b/examples/08_sweep_2d_heatmap.py
index ed41a8e..e57015d 100644
--- a/examples/08_sweep_2d_heatmap.py
+++ b/examples/08_sweep_2d_heatmap.py
@@ -5,6 +5,8 @@ Demonstrates:
- run_sweep() for Cartesian grid search
- Heatmap visualization with mbt.plot.heatmap_2d()
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/08_sweep_2d_heatmap.py
"""
diff --git a/examples/09_surface_3d.py b/examples/09_surface_3d.py
index 4a73c29..0e9337f 100644
--- a/examples/09_surface_3d.py
+++ b/examples/09_surface_3d.py
@@ -5,6 +5,8 @@ Demonstrates:
- run_sweep_lite() for fast parameter grid search
- 3D surface visualization with mbt.plot.surface_3d()
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/09_surface_3d.py
"""
diff --git a/examples/10_monte_carlo.py b/examples/10_monte_carlo.py
index f6566c4..4c4e6b4 100644
--- a/examples/10_monte_carlo.py
+++ b/examples/10_monte_carlo.py
@@ -5,6 +5,8 @@ Demonstrates:
- Monte Carlo fan chart visualization
- Risk metrics from simulated distributions
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/10_monte_carlo.py
"""
diff --git a/examples/11_portfolio.py b/examples/11_portfolio.py
index 26f5103..6dc9e9e 100644
--- a/examples/11_portfolio.py
+++ b/examples/11_portfolio.py
@@ -7,6 +7,8 @@ Demonstrates:
- Periodic rebalancing
- Per-strategy breakdown
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/11_portfolio.py
"""
diff --git a/examples/12_diagnostics.py b/examples/12_diagnostics.py
index 3ee96ec..8a8ad44 100644
--- a/examples/12_diagnostics.py
+++ b/examples/12_diagnostics.py
@@ -5,6 +5,8 @@ Demonstrates:
- check_exposure_stability(): verify positions are consistent across time windows
- risk_check(): post-run risk metrics validation
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/12_diagnostics.py
"""
diff --git a/examples/13_stochastic_simulation.py b/examples/13_stochastic_simulation.py
index 68ca03c..52c7053 100644
--- a/examples/13_stochastic_simulation.py
+++ b/examples/13_stochastic_simulation.py
@@ -7,6 +7,9 @@ Demonstrates:
- CUDA GPU acceleration (device="cuda")
- All expressions compile to native Rust — full Rayon / CUDA parallelism
+Data: none — this file simulates price paths, it does not backtest on any
+series. The paths are its output, not its input.
+
Usage:
python examples/13_stochastic_simulation.py
"""
diff --git a/examples/14_multi_timeframe.py b/examples/14_multi_timeframe.py
index 3036143..47648fc 100644
--- a/examples/14_multi_timeframe.py
+++ b/examples/14_multi_timeframe.py
@@ -10,6 +10,8 @@ Logic:
- 1h entry: RSI(14) < 35 during bullish regime → buy the dip
- Size: 50% of initial capital when conditions met, else flat
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/14_multi_timeframe.py
"""
@@ -23,9 +25,12 @@ from manifoldbt.helpers import time_range, Slippage, Interval
h12 = mbt.tf("12h") # references columns like "12h.close"
# -- Indicators ---------------------------------------------------------------
-# Trend filter on 12-hour bars (forward-filled onto 1h grid)
-trend_fast = ema(h12.close, 20)
-trend_slow = ema(h12.close, 50)
+# Trend filter on 12-hour bars. `apply()` evaluates the EMA on the 12h grid, so
+# 20 and 50 count 12-HOUR candles. Written `ema(h12.close, 20)` they would count
+# 20 rows of the 1h simulation grid over a step-held column -- under two 12h
+# candles, not twenty. See `bt.tf`.
+trend_fast = h12.apply(ema(close, 20))
+trend_slow = h12.apply(ema(close, 50))
bullish = trend_fast > trend_slow
# Entry signal on 1-hour bars (native resolution)
@@ -59,7 +64,7 @@ config = mbt.BacktestConfig(
),
fees=mbt.FeeConfig.binance_perps(),
slippage=Slippage.fixed_bps(2),
- warmup_bars=50,
+ warmup_bars=50 * 12, # 50 twelve-hour candles, counted in 1h simulation bars
extra_timeframes={
"12h": Interval.hours(12),
},
diff --git a/examples/15_cross_exchange.py b/examples/15_cross_exchange.py
index 58310fb..4f8ed1c 100644
--- a/examples/15_cross_exchange.py
+++ b/examples/15_cross_exchange.py
@@ -1,4 +1,4 @@
-"""Example 15: Cross-Exchange — Signal Binance, Execution dYdX.
+"""Cross-exchange — signals on Binance, execution on dYdX.
Simple RSI mean-reversion:
- RSI computed on Binance BTC perp data
@@ -7,8 +7,18 @@ Simple RSI mean-reversion:
- Per-venue fees: each symbol is charged its own exchange's fee schedule
(see FeeConfig.multi_venue below)
+Demonstrates:
+ - signals computed on one venue, orders filled on another
+ - a dict universe spanning two providers, with no special config
+ - per-venue fees: each symbol charged its own exchange's schedule
+
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Prerequisite:
Binance perp data (bars_1m/201.arrow) + dYdX data (dydx/1h/BTC-USD.arrow)
+
+Usage:
+ python examples/15_cross_exchange.py
"""
import time
diff --git a/examples/16_hashrate_exogene.py b/examples/16_hashrate_exogene.py
index 231c614..56b4cad 100644
--- a/examples/16_hashrate_exogene.py
+++ b/examples/16_hashrate_exogene.py
@@ -1,4 +1,4 @@
-"""Example 16: BTC-Hashrate Spread — Exogenous Data Strategy.
+"""Exogenous data — a BTC/hashrate spread as a signal.
Thesis: Bitcoin hashrate is a proxy for miner commitment and network
security. When BTC price drops but hashrate holds (or rises), miners
@@ -10,6 +10,15 @@ The strategy normalizes both BTC price and hashrate via EMA ratios
A rolling z-score of the spread generates the signal: negative z means
price is cheap relative to hashrate (long), positive means expensive.
+Demonstrates:
+ - an exogenous series ASOF-joined onto the bar grid
+ - a spread between two EMA-normalised series as a signal
+ - a rolling z-score turning that spread into a position
+
+Data: shared store — real BTC bars from `data/`, plus a hashrate series
+(fetched, or generated by the sample generator below when absent).
+See examples/README.md.
+
Exogenous data flow:
1. Fetch hashrate CSV (or use sample generator below)
2. Register via mbt.register_exo("hashrate", df)
@@ -18,6 +27,9 @@ Exogenous data flow:
Prerequisite:
Binance BTC perp data + hashrate exo registered in data/mega/exo/
+
+Usage:
+ python examples/16_hashrate_exogene.py
"""
import time
diff --git a/examples/17_per_venue_fees.py b/examples/17_per_venue_fees.py
index 7e8a037..09eb484 100644
--- a/examples/17_per_venue_fees.py
+++ b/examples/17_per_venue_fees.py
@@ -1,4 +1,4 @@
-"""Example 17: Per-Venue Fees — charge each symbol its own fee schedule.
+"""Per-venue fees — charge each symbol its own fee schedule.
Real desks route different assets to different exchanges (or liquidity tiers),
each with its own maker/taker fees, funding column and borrow rate. ``FeeConfig``
@@ -9,6 +9,13 @@ Here a 4-asset momentum portfolio executes the majors (BTC, ETH) on a cheap
venue and the alts (XRP, DOT) on a more expensive one. Single-provider universe,
so it runs without Pro.
+Demonstrates:
+ - FeeConfig.per_venue: named fee schedules in one book
+ - symbol_venue: which symbol trades where
+ - the cost gap between routing majors and alts differently
+
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/17_per_venue_fees.py
"""
diff --git a/examples/18_csv_import.py b/examples/18_csv_import.py
index 2f363c0..c50f837 100644
--- a/examples/18_csv_import.py
+++ b/examples/18_csv_import.py
@@ -8,6 +8,8 @@ Demonstrates:
The standard format is a header row + `timestamp,open,high,low,close,volume`
where timestamp is Unix milliseconds. MT4/MT5 exports are auto-detected.
+Data: synthetic — a sample generated by this file, reproducible
+
Usage:
python examples/18_csv_import.py
"""
diff --git a/examples/19_custom_indicators.py b/examples/19_custom_indicators.py
index 1d92342..403dd5a 100644
--- a/examples/19_custom_indicators.py
+++ b/examples/19_custom_indicators.py
@@ -1,109 +1,115 @@
-"""Créer ses propres indicateurs (indicateurs absents de la base).
+"""Writing your own indicators — the ones the library does not ship.
+Demonstrates:
+ - an indicator as a plain function returning an `Expr`
+ - `scan` for stateful indicators no rolling window can express
+ - `param(...)` to make a custom indicator sweepable
+
+Data: shared store — real market data from `data/` (see examples/README.md)
+
+Usage:
python examples/19_custom_indicators.py
────────────────────────────────────────────────────────────────────────────
-LE MODÈLE MENTAL
+THE MENTAL MODEL
────────────────────────────────────────────────────────────────────────────
-Un indicateur, ici, n'est RIEN d'autre qu'une fonction Python qui renvoie un
-`Expr`. Un `Expr` est un *nœud dans un graphe de calcul* : quand vous écrivez
-`(high + low) / 2`, aucune donnée n'est touchée — vous décrivez une opération.
-Le graphe complet est ensuite compilé et évalué **en Rust**, en une passe,
-vectorisé. C'est pour ça que vos indicateurs maison tournent à la vitesse des
-indicateurs natifs : ils finissent dans le même moteur.
+An indicator here is NOTHING but a Python function returning an `Expr`. An
+`Expr` is a *node in a computation graph*: writing `(high + low) / 2` touches
+no data — it describes an operation. The whole graph is then compiled and
+evaluated **in Rust**, in one vectorised pass. That is why your own indicators
+run at the speed of the built-in ones: they end up in the same engine.
-Toute la lib `manifoldbt.indicators` est écrite comme ça (`sma` ==
-`source.rolling_mean(period)`). Donc « ajouter un indicateur » = « écrire une
-fonction qui compose des `Expr` ». Trois niveaux, du plus simple au plus rare.
+The whole `manifoldbt.indicators` library is written this way (`sma` ==
+`source.rolling_mean(period)`). So "adding an indicator" means "writing a
+function that composes `Expr`s". Three levels, from the common to the rare.
"""
import os
from time import perf_counter
import manifoldbt as mbt
-# Colonnes de base (ce sont déjà des Expr) + quelques helpers.
+# Base columns (already Exprs) plus a few helpers.
from manifoldbt.indicators import open, high, low, close, volume, sma, rsi, ema
-# Briques bas niveau : lit (constante), col (colonne par nom), when (if/else),
-# scan/s (état récursif), param (paramètre balayable).
+# Low-level bricks: lit (constant), col (column by name), when (if/else),
+# scan/s (recursive state), param (sweepable parameter).
from manifoldbt.expr import lit, col, when, scan, s, param
from manifoldbt.helpers import time_range, Slippage, Interval
# ═══════════════════════════════════════════════════════════════════════════
-# NIVEAU 1 — COMPOSER LES PRIMITIVES (99 % des cas)
+# LEVEL 1 — COMPOSING PRIMITIVES (99% of cases)
# ═══════════════════════════════════════════════════════════════════════════
-# On combine colonnes + opérateurs (+ - * /, > < >= & | ~) + méthodes d'Expr
+# Combine columns + operators (+ - * /, > < >= & | ~) + Expr methods
# (rolling_mean/std/min/max/median, ewm_mean, zscore, pct_change, diff, lag,
-# rsi, linreg_*, cross_above/below, cumsum, rank, ...). Chaque appel renvoie
-# un Expr, donc tout se chaîne.
+# rsi, linreg_*, cross_above/below, cumsum, rank, ...). Every call returns
+# an Expr, so everything chains.
def awesome_oscillator(fast=5, slow=34):
- """Awesome Oscillator (Bill Williams) — ABSENT de la base.
+ """Awesome Oscillator (Bill Williams) — NOT in the library.
- AO = SMA(prix médian, 5) − SMA(prix médian, 34), prix médian = (H+L)/2
+ AO = SMA(median price, 5) − SMA(median price, 34), median = (H+L)/2
- Momentum : positif = pression acheteuse, négatif = vendeuse.
+ Momentum: positive means buying pressure, negative means selling.
"""
- median_price = (high + low) / 2 # Expr : opération sur 2 colonnes
- return sma(median_price, fast) - sma(median_price, slow) # Expr résultat
+ median_price = (high + low) / 2 # Expr: an operation on 2 columns
+ return sma(median_price, fast) - sma(median_price, slow) # the result Expr
def dist_to_ma_pct(period=20):
- """Écart en % du prix à sa moyenne mobile — ABSENT de la base.
+ """Distance from price to its moving average, in % — NOT in the library.
- Négatif = le prix est SOUS sa moyenne (survendu) → brique idéale pour du
- retour à la moyenne. Une seule ligne de composition.
+ Negative means the price sits BELOW its average (oversold), which makes it
+ a natural building block for mean reversion. One line of composition.
"""
ma = sma(close, period)
return (close - ma) / ma * 100.0
def intraday_range_pct():
- """Amplitude de la bougie en % du close — ABSENT de la base.
+ """Bar range as a % of the close — NOT in the library.
- Un proxy de volatilité instantané. Montre qu'on mélange librement les
- colonnes OHLC.
+ An instant volatility proxy. Shows that OHLC columns mix freely.
"""
return (high - low) / close * 100.0
def rsi_zscore(period=14, lookback=365):
- """RSI standardisé : à quel point le RSI est extrême vs SA PROPRE histoire.
+ """Standardised RSI: how extreme the RSI is against ITS OWN history.
- Compose un indicateur natif (rsi) avec des stats roulantes. C'est
- exactement le motif utilisé dans strategies/rsi_dynamic_alloc.py.
+ Composes a built-in indicator (rsi) with rolling statistics — the same
+ pattern used in strategies/rsi_dynamic_alloc.py.
"""
r = rsi(close, period)
return (r - r.rolling_mean(lookback)) / r.rolling_std(lookback)
# ═══════════════════════════════════════════════════════════════════════════
-# NIVEAU 2 — `scan` : INDICATEURS À ÉTAT / RÉCURSIFS
+# LEVEL 2 — `scan`: STATEFUL / RECURSIVE INDICATORS
# ═══════════════════════════════════════════════════════════════════════════
-# Quand la valeur d'aujourd'hui dépend de celle d'HIER (récursion) et qu'aucun
-# rolling ne suffit, on utilise `scan`. Il tourne comme une petite VM scalaire,
-# entièrement en Rust (pas de callback Python par barre).
+# When today's value depends on YESTERDAY's (recursion) and no rolling window
+# suffices, reach for `scan`. It runs as a small scalar VM, entirely in Rust
+# (no Python callback per bar).
#
# scan(state=..., update=..., output=...)
-# • state : variables d'état + leur valeur initiale (1re ligne)
-# • update : expressions évaluées à chaque barre, DANS L'ORDRE
-# - s.prev("x") = valeur de "x" à la barre précédente
-# - s.var("k") = valeur calculée plus tôt DANS LE MÊME pas
-# - si un nom d'update == un nom d'état, on réécrit cet état
-# • output : quelle variable émettre comme résultat
+# • state : state variables and their initial value (first row)
+# • update : expressions evaluated on every bar, IN ORDER
+# - s.prev("x") = value of "x" on the previous bar
+# - s.var("k") = value computed earlier WITHIN THE SAME step
+# - an update name matching a state name rewrites that state
+# • output : which variable to emit as the result
#
-# Preuve que c'est puissant : le Kalman et le GARCH livrés sont écrits
-# UNIQUEMENT avec scan (voir manifoldbt/indicators.py).
+# Proof that it is enough: the shipped Kalman and GARCH are written with scan
+# ALONE (see manifoldbt/indicators.py).
def up_streak():
- """Nombre de bougies HAUSSIÈRES consécutives — ABSENT de la base, et
- impossible avec un simple rolling (il faut un compteur qui se réinitialise).
+ """Count of consecutive UP bars — NOT in the library, and impossible with
+ a plain rolling window (it needs a counter that resets).
- streak = streak_précédent + 1 si close > close(-1), sinon 0
+ streak = previous streak + 1 if close > close(-1), else 0
"""
- is_up = close > close.lag(1) # Expr booléen (1.0 / 0.0) par barre
+ is_up = close > close.lag(1) # boolean Expr (1.0 / 0.0) per bar
return scan(
- state={"n": lit(0.0)}, # compteur initialisé à 0
+ state={"n": lit(0.0)}, # counter seeded at 0
update={
# if is_up: prev(n) + 1 else: 0
"n": when(is_up, s.prev("n") + lit(1.0), lit(0.0)),
@@ -113,66 +119,66 @@ def up_streak():
def ema_from_scratch(alpha=0.1):
- """EMA « à la main » via scan — juste pour illustrer le mécanisme.
- (L'EMA existe en natif : `ema(close, span)`. Ici c'est pédagogique.)
+ """A hand-rolled EMA via scan — purely to show the mechanism.
+ (EMA is built in: `ema(close, span)`. This one is pedagogical.)
- ema = alpha * close + (1 - alpha) * ema_précédent
+ ema = alpha * close + (1 - alpha) * previous ema
"""
return scan(
- state={"ema": close}, # graine = 1er close
+ state={"ema": close}, # seeded with the first close
update={"ema": lit(alpha) * close + lit(1.0 - alpha) * s.prev("ema")},
output="ema",
)
# ═══════════════════════════════════════════════════════════════════════════
-# NIVEAU 3 — LES LIMITES (À CONNAÎTRE)
+# LEVEL 3 — THE LIMITS (WORTH KNOWING)
# ═══════════════════════════════════════════════════════════════════════════
-# • PAS de callback Python par barre : `scan` s'exécute en Rust, on ne peut pas
-# y injecter une fonction Python appelée sur chaque bougie (ce serait lent).
-# Tant que la logique s'exprime avec Expr + when + scan, ça passe.
-# • Un indicateur VRAIMENT nouveau, non exprimable ainsi, demande d'ajouter un
-# variant `Expr` + son kernel côté Rust — chemin contributeur, pas utilisateur.
-# • Données externes (hashrate, funding, sentiment…) : `mbt.register_exo(...)`
-# puis `exo("nom")` renvoie un Expr utilisable comme n'importe quelle colonne.
+# • NO Python callback per bar: `scan` runs in Rust, and you cannot inject a
+# Python function called on every candle (it would be slow). As long as the
+# logic expresses in Expr + when + scan, it works.
+# • A GENUINELY new indicator, not expressible that way, needs a new `Expr`
+# variant and its Rust kernel — the contributor path, not the user path.
+# • External data (hashrate, funding, sentiment…): `mbt.register_exo(...)`,
+# then `exo("name")` returns an Expr usable like any other column.
# ═══════════════════════════════════════════════════════════════════════════
-# BONUS — RENDRE SON INDICATEUR BALAYABLE (sweep)
+# BONUS — MAKING YOUR INDICATOR SWEEPABLE
# ═══════════════════════════════════════════════════════════════════════════
-# Les périodes acceptent `param(...)` à la place d'un entier. Le moteur
-# recompile alors une fois par combinaison et balaie la grille en parallèle,
-# sans changer une ligne de l'indicateur :
+# Periods accept `param(...)` in place of an integer. The engine then
+# recompiles once per combination and sweeps the grid in parallel, without
+# changing a line of the indicator:
#
# ao = awesome_oscillator(fast=param("fast"), slow=param("slow"))
-# # puis, avec la grille passée séparément (l'indicateur ne change pas) :
+# # then, with the grid passed separately (the indicator is unchanged):
# # batch = mbt.run_sweep_lite(
# # strategy,
# # {"fast": [3, 5, 8], "slow": [21, 34, 55]},
# # config, store,
# # )
#
-# (voir examples/08_sweep_2d_heatmap.py pour le sweep complet.)
+# (see examples/08_sweep_2d_heatmap.py for the full sweep.)
# ═══════════════════════════════════════════════════════════════════════════
-# METTRE UN INDICATEUR MAISON DANS UNE STRATÉGIE + BACKTEST
+# PUTTING A CUSTOM INDICATOR IN A STRATEGY AND BACKTESTING IT
# ═══════════════════════════════════════════════════════════════════════════
-# On utilise `dist_to_ma_pct` (retour à la moyenne) : long quand le prix est
-# nettement sous sa moyenne, on sort quand il l'a rejointe.
+# Using `dist_to_ma_pct` (mean reversion): long when the price sits well below
+# its average, out when it has caught up.
-dist = dist_to_ma_pct(period=48) # notre indicateur maison
-streak = up_streak() # et un second, pour l'exposer aussi
+dist = dist_to_ma_pct(period=48) # our custom indicator
+streak = up_streak() # a second one, exposed too
-signal = when(dist < -5.0, 1.0, # >5 % sous la MM → achat du creux
- when(dist > 0.0, 0.0)) # revenu à la MM → sortie, sinon hold
+signal = when(dist < -5.0, 1.0, # >5% below the MA -> buy the dip
+ when(dist > 0.0, 0.0)) # back at the MA -> exit, else hold
strategy = (
mbt.Strategy.create("custom_indicator_demo")
- .signal("dist_to_ma_%", dist) # .signal() = exposer pour le rapport
+ .signal("dist_to_ma_%", dist) # .signal() exposes it in the report
.signal("up_streak", streak)
.size(signal)
- .describe("Retour à la moyenne piloté par un indicateur maison (écart à la MM)")
+ .describe("Mean reversion driven by a custom indicator (distance to the MA)")
)
# -- Config -------------------------------------------------------------------
@@ -185,9 +191,9 @@ config = mbt.BacktestConfig(
bar_interval=Interval.hours(1),
initial_capital=10_000,
execution=mbt.ExecutionConfig(allow_short=False, max_position_pct=1.0),
- fees=mbt.FeeConfig.zero(), # sans frais, pour l'exemple
+ fees=mbt.FeeConfig.zero(), # fee-free, for the example
slippage=Slippage.fixed_bps(2),
- warmup_bars=60, # >= la plus longue fenêtre utilisée
+ warmup_bars=60, # >= the longest window used
)
# -- Run ----------------------------------------------------------------------
diff --git a/examples/20_entry_orders.py b/examples/20_entry_orders.py
index 0a966fa..2f4e921 100644
--- a/examples/20_entry_orders.py
+++ b/examples/20_entry_orders.py
@@ -8,6 +8,13 @@ the same signal four ways so the difference is visible in one place:
stop wait for a breakout, fill through the level (taker + gap)
limit on a signal rest on a level the DSL computes (here: 1 ATR below close)
+Demonstrates:
+ - market, limit, stop and signal-priced entries, side by side
+ - passive fills (maker, no slippage) against aggressive ones
+ - an entry resting on a level the DSL computes
+
+Data: shared store — real market data from `data/` (see examples/README.md)
+
Usage:
python examples/20_entry_orders.py
"""
diff --git a/examples/21_fill_at_computed_level.py b/examples/21_fill_at_computed_level.py
index 4dfdbc2..6a60a3a 100644
--- a/examples/21_fill_at_computed_level.py
+++ b/examples/21_fill_at_computed_level.py
@@ -1,17 +1,36 @@
"""Filling at a computed level — ExecutionPrice.custom().
A mean-reversion band strategy on native 1-minute bars: short at the touch of
-an upper band around an hourly SMA, cover at the lower band. The engine always
-knew how to COMPUTE the band; this example shows the fill landing ON it.
+an upper band, cover at the lower one. The engine always knew how to COMPUTE
+the band; this example shows the fill landing ON it.
-The touch bar itself proves the level traded: it opens below the band and its
-high crosses it, so the band price sits inside [open, high]. Yet with
-``AtClose`` the only reachable fill is the bar's close — on a mean-reverting
-touch, systematically on the wrong side of the level. The same run is done
-both ways so the difference is visible in one place.
+**This is a feature demo, not a claim about markets.** The data is synthetic and
+detrended by construction, so mean reversion cannot lose on it whatever the
+parameters — the returns printed below describe the fixture, nothing else. The
+same run is done both ways only so you can see that the setting takes effect.
+
+Which of the two fills is the better one depends entirely on what the market
+does after the touch, and this fixture reverts by construction. Do not read a
+rule into the sign of the gap.
+
+The touch bar proves the level traded: it opens below the band and its high
+crosses it, so the band price sits inside [open, high]. On this data all 248
+intended levels land inside their bar. A level that did not would be clamped
+back into [low, high] and reported in ``result.warnings`` — the band comes from
+the previous closed hour, so it can sit outside a bar that never reached it.
+
+Look-ahead was tested for and ruled out: the fill price does not move when the
+triggering bar's high is inflated by 3%.
Self-contained: generates its own synthetic data in a temp store.
+Demonstrates:
+ - ExecutionPrice.custom(): the fill price as a strategy signal
+ - tf("1h").apply(...): an indicator whose period counts hourly candles
+ - a level computed from a higher timeframe, known before the bar opens
+
+Data: synthetic (seed 7) — generated by this file, reproducible
+
Usage:
python examples/21_fill_at_computed_level.py
"""
@@ -26,11 +45,21 @@ import manifoldbt as mbt
from manifoldbt.indicators import close, high, low, open as open_px, sma
from manifoldbt.helpers import ExecutionPrice, Interval, Slippage
-# -- Synthetic 1m data: a mean-reverting walk ---------------------------------
+# -- Synthetic 1m data: a walk with its drift REMOVED -------------------------
+# `- np.linspace(0, level[-1], N)` subtracts the entire trend, forcing the
+# series to end exactly where it started: the raw walk loses 25.6%, this one
+# returns -0.0%. Mean reversion is therefore GUARANTEED here, by construction.
+#
+# That is deliberate, and it is why the absolute returns printed below mean
+# nothing. Measured over nine parameter sets (bands 0.002 to 0.008, periods 4h
+# to 20h), the custom fill returns +2.2% to +9.1% and AtClose -25.5% to +4.2%:
+# neither range is a property of the strategy, and AtClose even turns positive
+# on the widest band. What holds across all nine is only the ORDER — the custom
+# fill is ahead every time — and that gap is the whole subject.
N = 30 * 1440 # 30 days of 1-minute bars
rng = np.random.default_rng(7)
steps = rng.normal(0.0, 0.0010, N)
-level = np.cumsum(steps) * 0.85 # pull the walk back toward its mean
+level = np.cumsum(steps) * 0.85
px = 100.0 * np.exp(level - np.linspace(0, level[-1], N))
o, c = px, np.roll(px, -1)
c[-1] = px[-1]
@@ -42,12 +71,16 @@ frame = pd.DataFrame(
"close": c, "volume": rng.uniform(1_000, 5_000, N)}
)
-# -- Bands around an hourly SMA, evaluated on 1m native bars ------------------
+# -- Bands around an 8-hour mean, on 1m native bars ---------------------------
+# `apply()` evaluates the SMA on the hourly grid, so its period counts hourly
+# candles. Written `sma(h1.close, 8)` the period would count 8 SIMULATION bars
+# over a step-held column, which is 8 minutes, not 8 hours. See `bt.tf`.
DEV_UP, DEV_DN = 0.004, 0.003
-h1 = mbt.tf("1h") # hourly columns, as of the last closed hour
-band_up = sma(h1.close, 8) * (1 + DEV_UP)
-band_dn = sma(h1.close, 8) * (1 - DEV_DN)
+h1 = mbt.tf("1h")
+hourly_mean = h1.apply(sma(close, 8)) # mean of the last 8 hourly closes
+band_up = hourly_mean * (1 + DEV_UP)
+band_dn = hourly_mean * (1 - DEV_DN)
touch_up = high >= band_up # entry: short at the touch of the upper band
touch_dn = low <= band_dn # exit: cover at the touch of the lower band
@@ -98,17 +131,34 @@ if __name__ == "__main__":
)
return mbt.run(strategy, config, store)
- print(f"{'execution price':<22} {'trades':>7} {'return':>9} first entry fills")
+ print("Synthetic detrended data: the LEVELS below are an artifact of the")
+ print("fixture, not a result. Only the gap between the two rows is real.\n")
+ print(f"{'execution price':<22} {'trades':>7} {'return*':>9} first entry fills")
print("-" * 78)
+
+ returns = {}
+ n_trades = 0
for label, price in (("AtClose", "AtClose"),
("custom('exec_level')", ExecutionPrice.custom("exec_level"))):
result = run(price)
tr = result.trades_df()
entries = tr[tr["fill_price"] > 0].head(3)["fill_price"].round(4).tolist()
- print(f"{label:<22} {len(tr):>7} {result.metrics['total_return']:>8.2%} {entries}")
+ returns[label] = result.metrics["total_return"]
+ n_trades = len(tr)
+ print(f"{label:<22} {len(tr):>7} {returns[label]:>8.2%} {entries}")
+ gap = returns["custom('exec_level')"] - returns["AtClose"]
+ print(f"\n GAP: {gap:.1%} on identical signals and identical bars.")
print(
- "\nSame signals, same bars: only WHERE the order fills changed. The"
- "\ncustom fills land on the band level (inside the touch bar's range),"
- "\nnot on its close. A fill outside [low, high] would be warned about."
+ "\n* Both figures describe the fixture, not a market: the series is"
+ "\n detrended, so reversion wins on it by construction. What the two"
+ "\n rows show is that the setting takes effect -- the custom fills land"
+ "\n on the band, computed from the previous closed hour, where AtClose"
+ "\n can only reach the bar's close. Which of the two is the better"
+ "\n price depends on what the market does next, and this series was"
+ "\n built to revert."
+ "\n"
+ f"\n All {n_trades} intended levels land inside their own bar here. One"
+ f"\n that did not would be clamped into [low, high] and reported in"
+ f"\n result.warnings."
)
diff --git a/examples/22_yahoo_equities.py b/examples/22_yahoo_equities.py
new file mode 100644
index 0000000..aa7de98
--- /dev/null
+++ b/examples/22_yahoo_equities.py
@@ -0,0 +1,63 @@
+"""Yahoo Finance -- stocks, ETFs, indices, FX and futures, free on all tiers.
+
+Demonstrates:
+ - mbt.ingest(provider="yahoo") -- no API key, no license required
+ - Backtesting daily equity bars, exactly like a crypto connector
+ - Dividend-adjusted prices (same convention as yfinance's auto_adjust=True)
+
+Yahoo imposes its own history limits: 1m bars go back 30 days, 1h about two
+years, daily bars back to the listing date. Tickers follow Yahoo's own
+notation: AAPL, SPY, ^GSPC (index), EURUSD=X (FX), ES=F (future),
+BTC-USD (crypto), AIR.PA (Euronext).
+
+Pass `dataset="raw"` to keep unadjusted quotes.
+
+Data: self-contained (network) — ingested on each run from a free connector
+
+Usage:
+ python examples/22_yahoo_equities.py
+"""
+import os
+import tempfile
+
+import manifoldbt as mbt
+from manifoldbt.indicators import close, ema
+from manifoldbt.helpers import time_range, Interval
+
+# -- 1. Pull daily bars from Yahoo (free, all tiers) --------------------------
+tmp = tempfile.mkdtemp()
+store = mbt.ingest(
+ provider="yahoo",
+ symbol="AAPL",
+ symbol_id=1,
+ start="2020-01-01T00:00:00Z",
+ end="2024-01-01T00:00:00Z",
+ interval="1d",
+ asset_class="equity",
+ data_root=os.path.join(tmp, "data"),
+ metadata_db=os.path.join(tmp, "meta.sqlite"),
+)
+print("Ingested:", store.list_symbols())
+
+# -- 2. Backtest on it like any other data ------------------------------------
+strategy = (
+ mbt.Strategy.create("ema_cross")
+ .signal("fast", ema(close, 20))
+ .signal("slow", ema(close, 50))
+ .size(mbt.when(ema(close, 20) > ema(close, 50), 1.0, 0.0))
+ .describe("EMA(20/50) crossover on daily AAPL bars from Yahoo Finance")
+)
+
+start, end = time_range("2020-01-01", "2024-01-01")
+config = mbt.BacktestConfig(
+ universe=[1],
+ time_range_start=start,
+ time_range_end=end,
+ bar_interval=Interval.days(1),
+ initial_capital=10_000,
+ warmup_bars=60,
+)
+
+if __name__ == "__main__":
+ result = mbt.run(strategy, config, store)
+ print(result.summary())
diff --git a/examples/23_deribit_options.py b/examples/23_deribit_options.py
new file mode 100644
index 0000000..e653a91
--- /dev/null
+++ b/examples/23_deribit_options.py
@@ -0,0 +1,119 @@
+"""Crypto options -- Deribit contracts that actually expire.
+
+Demonstrates:
+ - mbt.ingest(provider="deribit", ...) -- no API key, expired contracts included
+ - Contract terms (strike, expiry, side, settlement) recorded alongside the bars
+ - Cash settlement at expiration, at intrinsic value
+ - config.option_underlyings -- which price series settles the contract
+ - Per-leg sizing with col("symbol_id"), for multi-leg structures
+
+Why Deribit and not Binance: Deribit serves the history of contracts that have
+already expired, which is the only data an option backtest can run on. Binance's
+options API answers HTTP 400 for anything past its expiration date.
+
+Three things worth knowing before reading the numbers:
+
+ - **Everything is in BTC.** A Deribit BTC option is quoted, margined and
+ settled in BTC, so `initial_capital` below is 10 BTC, not 10 dollars. The
+ payoff of a call is `max(0, S - K) / S` BTC per contract.
+
+ - **You choose the settlement reference.** Deribit settles against its own
+ `BTC_USD` index, whose ticker matches no series you can ingest.
+ `BTC-PERPETUAL` stands in for it here; the basis between the two is small
+ but real, and it is not modelled.
+
+ - **Positions are counted in units of the underlying.** On Deribit a contract
+ is one unit, so the two are the same thing. On a 100-multiplier listed
+ option, holding one contract means a position of 100.
+
+Data: self-contained (network) — ingested on each run from a free connector
+
+Usage:
+ python examples/23_deribit_options.py
+"""
+import os
+import tempfile
+
+import manifoldbt as mbt
+from manifoldbt.indicators import col
+from manifoldbt.helpers import time_range, Interval
+
+# Two contracts that expired on 2025-06-27, and the series that settles them.
+UNDERLYING, UNDERLYING_ID = "BTC-PERPETUAL", 1
+CALL_100K, CALL_ID = "BTC-27JUN25-100000-C", 2
+PUT_90K, PUT_ID = "BTC-27JUN25-90000-P", 3
+START, END = "2025-05-01T00:00:00Z", "2025-07-01T00:00:00Z"
+
+tmp = tempfile.mkdtemp()
+common = dict(
+ start=START,
+ end=END,
+ interval="1d",
+ data_root=os.path.join(tmp, "data"),
+ metadata_db=os.path.join(tmp, "meta.sqlite"),
+)
+
+# -- 1. The settlement reference, then the contracts ---------------------------
+store = mbt.ingest(
+ provider="deribit", symbol=UNDERLYING, symbol_id=UNDERLYING_ID,
+ asset_class="crypto_perp", **common
+)
+store = mbt.ingest(
+ provider="deribit", symbol=CALL_100K, symbol_id=CALL_ID,
+ asset_class="option", **common
+)
+store = mbt.ingest(
+ provider="deribit", symbol=PUT_90K, symbol_id=PUT_ID,
+ asset_class="option", **common
+)
+
+# The connector asked Deribit for the terms and the store kept them; nothing
+# below is inferred from the instrument name.
+for symbol_id, terms in sorted(store.option_contracts().items()):
+ print(f"{symbol_id}: {terms['option_type']} {terms['strike']:.0f}, "
+ f"settles {terms['settlement']}")
+
+# -- 2. A risk reversal: long the 100k call, short the 90k put -----------------
+# Legs are told apart by `col("symbol_id")`. Do NOT discriminate on price level
+# (e.g. "close < 100"): a premium that crosses the threshold silently flips the
+# leg to zero and the strategy closes its own position.
+size = (
+ mbt.when(col("symbol_id") == float(CALL_ID), 1.0, 0.0)
+ + mbt.when(col("symbol_id") == float(PUT_ID), -1.0, 0.0)
+)
+strategy = (
+ mbt.Strategy.create("risk_reversal")
+ .signal("leg", col("symbol_id"))
+ .size(size)
+ .describe("Long the 100k call, short the 90k put, both held to expiration")
+)
+
+start, end = time_range("2025-05-01", "2025-07-01")
+config = mbt.BacktestConfig(
+ universe=[UNDERLYING_ID, CALL_ID, PUT_ID],
+ time_range_start=start,
+ time_range_end=end,
+ bar_interval=Interval.days(1),
+ initial_capital=10.0, # 10 BTC
+ currency="BTC",
+ option_underlyings={CALL_ID: UNDERLYING_ID, PUT_ID: UNDERLYING_ID},
+ option_margin_model="deribit", # the short put pays margin
+ execution=mbt.ExecutionConfig(position_sizing_mode="Units", allow_short=True),
+)
+
+if __name__ == "__main__":
+ result = mbt.run(strategy, config, store)
+ trades = result.trades.to_pandas()
+
+ print("\nTrades on the option legs:")
+ print(
+ trades[trades.symbol_id != UNDERLYING_ID][
+ ["symbol_id", "side", "quantity", "fill_price", "exit_reason"]
+ ].to_string(index=False)
+ )
+ # exit_reason 5 is OptionExpiry: the venue settled the contract, the
+ # strategy did not sell it. The put settles at 0 because BTC finished far
+ # above its 90k strike.
+ print("\nFinal equity: %.6f BTC" % float(result.equity_curve[-1]))
+ if result.warnings:
+ print("Warnings:", result.warnings)
diff --git a/examples/24_option_spread.py b/examples/24_option_spread.py
new file mode 100644
index 0000000..89c2449
--- /dev/null
+++ b/examples/24_option_spread.py
@@ -0,0 +1,101 @@
+"""Option strategy -- a bull call spread, held to expiration.
+
+Demonstrates:
+ - A two-leg option structure: long a low strike, short a higher one
+ - Per-leg sizing with col("symbol_id")
+ - A SHORT option paying margin under the venue's own formula
+ - Both legs cash-settled at expiry, which is what caps the payoff
+
+The structure: buy the 100k call, sell the 110k call, same expiration. The
+short leg pays for part of the long one, and in exchange it caps the gain at
+the distance between the strikes. Classic, and the cheapest way to see the
+engine settle two contracts on the same day with different outcomes.
+
+**A currency trap worth knowing.** Every leg of a strategy has to be quoted in
+the same currency, because the engine carries one cash balance. On Deribit an
+option is quoted in BTC, but `BTC-PERPETUAL` is quoted in USD. So a covered
+call (long the perpetual, short a call) would add dollars to bitcoin in a single
+number and produce a meaningless equity curve. A spread has both legs in BTC,
+which is why this example is a spread. The perpetual appears below only as the
+settlement reference, never as a position.
+
+Data: self-contained (network) — ingested on each run from a free connector
+
+Usage:
+ python examples/24_option_spread.py
+"""
+import os
+import tempfile
+
+import manifoldbt as mbt
+from manifoldbt.indicators import col
+from manifoldbt.helpers import time_range, Interval
+
+# Both legs expired on 2025-06-27, so the whole life of the trade is history.
+UNDERLYING, UNDERLYING_ID = "BTC-PERPETUAL", 1
+LONG_LEG, LONG_ID = "BTC-27JUN25-100000-C", 2
+SHORT_LEG, SHORT_ID = "BTC-27JUN25-110000-C", 3
+START, END = "2025-05-01T00:00:00Z", "2025-07-01T00:00:00Z"
+
+tmp = tempfile.mkdtemp()
+common = dict(
+ start=START,
+ end=END,
+ interval="1d",
+ data_root=os.path.join(tmp, "data"),
+ metadata_db=os.path.join(tmp, "meta.sqlite"),
+)
+
+store = mbt.ingest(
+ provider="deribit", symbol=UNDERLYING, symbol_id=UNDERLYING_ID,
+ asset_class="crypto_perp", **common
+)
+for symbol, symbol_id in ((LONG_LEG, LONG_ID), (SHORT_LEG, SHORT_ID)):
+ store = mbt.ingest(
+ provider="deribit", symbol=symbol, symbol_id=symbol_id,
+ asset_class="option", **common
+ )
+
+# -- The strategy --------------------------------------------------------------
+# Legs are told apart by symbol id. Never discriminate on price level: a premium
+# crossing the threshold would flip its own leg to zero and close the position.
+size = (
+ mbt.when(col("symbol_id") == float(LONG_ID), 1.0, 0.0) # buy the 100k call
+ + mbt.when(col("symbol_id") == float(SHORT_ID), -1.0, 0.0) # sell the 110k call
+)
+strategy = (
+ mbt.Strategy.create("bull_call_spread")
+ .signal("leg", col("symbol_id"))
+ .size(size)
+ .describe("Long the 100k call, short the 110k call, held to expiration")
+)
+
+start, end = time_range("2025-05-01", "2025-07-01")
+config = mbt.BacktestConfig(
+ universe=[UNDERLYING_ID, LONG_ID, SHORT_ID],
+ time_range_start=start,
+ time_range_end=end,
+ bar_interval=Interval.days(1),
+ initial_capital=10.0, # 10 BTC: everything here is in BTC
+ currency="BTC",
+ option_underlyings={LONG_ID: UNDERLYING_ID, SHORT_ID: UNDERLYING_ID},
+ option_margin_model="deribit", # the short leg posts margin
+ execution=mbt.ExecutionConfig(position_sizing_mode="Units", allow_short=True),
+)
+
+if __name__ == "__main__":
+ result = mbt.run(strategy, config, store)
+ trades = result.trades.to_pandas()
+
+ names = {LONG_ID: "long 100k", SHORT_ID: "short 110k"}
+ print("\nTrades, by leg:")
+ for _, t in trades[trades.symbol_id != UNDERLYING_ID].iterrows():
+ what = "settled" if t.exit_reason == 5 else "traded"
+ print(f" {names[t.symbol_id]:<12} {what:<8} {t.quantity:>4.1f} @ {t.fill_price:.6f} BTC")
+
+ equity = float(result.equity_curve[-1])
+ print(f"\nFinal equity: {equity:.6f} BTC ({equity - 10.0:+.6f})")
+ # The short leg expiring worthless is what the spread pays for: it financed
+ # part of the long call, and capped the gain at the strike distance.
+ if result.warnings:
+ print("Warnings:", result.warnings)
diff --git a/examples/25_lookahead_trap.py b/examples/25_lookahead_trap.py
new file mode 100644
index 0000000..81c24d3
--- /dev/null
+++ b/examples/25_lookahead_trap.py
@@ -0,0 +1,190 @@
+"""Look-ahead — the leak the detector cannot see.
+
+The classic beginner's leak, and the one that survives review: computing a
+statistic over the *whole* dataset in the notebook, then using it as a strategy
+parameter. Here it is the mean price of the entire asset, used from bar 0 —
+where 749 of the 750 days it summarises are still in the future.
+
+**There IS a leak in this strategy. It is deliberate.** The question the
+example answers is not whether the leak exists, but which audit methods can
+see it — and two of the three cannot. Read every "sees nothing" below as a
+statement about the *method*, never as a clean bill of health for the
+strategy.
+
+What this example shows, in order:
+
+ 1. the leak triples the return and doubles the Sharpe;
+ 2. `detect_lookahead` sees nothing;
+ 3. perturbing every future bar sees nothing either;
+ 4. the one method that catches it: re-derive the parameter on truncated data.
+
+**Why the first two are blind.** Both re-run the *same strategy* on shorter or
+altered data. The mean is not recomputed by the engine: it is a number baked
+into the strategy at research time. Re-running cannot see it, because the leak
+already happened, in the notebook, before the backtest existed.
+
+That is not a bug to fix in the detector, it is the shape of the problem. No
+re-run method can audit a constant. The only defence is to treat every
+parameter derived from data as part of the pipeline, and re-derive it on
+whatever window you are testing.
+
+Self-contained: generates its own synthetic data in a temp store.
+
+Demonstrates:
+ - a parameter computed over the whole dataset, used from bar 0
+ - two audit methods that do not see it, and why
+ - the one that does: re-deriving the parameter per window
+
+Data: synthetic (seed 11) — generated by this file, reproducible.
+The fixture DETERMINES the outcome: see examples/README.md.
+
+Usage:
+ python examples/25_lookahead_trap.py
+"""
+import os
+import tempfile
+
+import numpy as np
+import pandas as pd
+
+import manifoldbt as mbt
+from manifoldbt.indicators import close
+from manifoldbt.helpers import Interval, Slippage
+
+# -- A mean-reverting series: exactly where knowing the mean is gold ----------
+N_DAYS = 750
+rng = np.random.default_rng(11)
+level = np.cumsum(rng.normal(0.0, 0.018, N_DAYS))
+px = 100.0 * np.exp(level - np.linspace(0, level[-1], N_DAYS))
+o = px
+c = np.roll(px, -1)
+c[-1] = px[-1]
+amp = np.abs(rng.normal(0.0, 0.004, N_DAYS))
+frame = pd.DataFrame({
+ "timestamp": pd.date_range("2022-01-01", periods=N_DAYS, freq="1D", tz="UTC"),
+ "open": o,
+ "high": np.maximum(o, c) * (1 + amp),
+ "low": np.minimum(o, c) * (1 - amp),
+ "close": c,
+ "volume": rng.uniform(1_000, 5_000, N_DAYS),
+})
+
+
+def store_of(df, tag):
+ root = tempfile.mkdtemp(prefix=f"mbt_ex25_{tag}_")
+ return mbt.import_dataframe(
+ df, symbol="SYNTH", symbol_id=1, interval="1d",
+ data_root=os.path.join(root, "data"),
+ metadata_db=os.path.join(root, "meta.sqlite"),
+ )
+
+
+def config_of(df):
+ ts = pd.DatetimeIndex(df["timestamp"])
+ return mbt.BacktestConfig(
+ universe=[1],
+ time_range_start=int(ts[0].value),
+ time_range_end=int(ts[-1].value) + 86_400_000_000_000,
+ bar_interval=Interval.days(1),
+ initial_capital=10_000,
+ execution=mbt.ExecutionConfig(
+ signal_delay=1, max_position_pct=1.0,
+ allow_short=True, position_sizing_mode="FractionOfEquity",
+ ),
+ slippage=Slippage.fixed_bps(0),
+ warmup_bars=0,
+ )
+
+
+def leaky(mean_price):
+ """THE LEAK: `mean_price` comes from `df.close.mean()` over everything."""
+ return (
+ mbt.Strategy.create("global_mean_leak")
+ .signal("edge", close)
+ .size(mbt.when(close < mean_price, 1.0, -1.0))
+ .describe("Long below the global mean, short above")
+ )
+
+
+def causal(window):
+ """The honest twin: a rolling mean knows only the past."""
+ return (
+ mbt.Strategy.create("rolling_mean")
+ .signal("edge", close)
+ .size(mbt.when(close < close.rolling_mean(window), 1.0, -1.0))
+ .describe("Long below the rolling mean")
+ )
+
+
+def equity(result):
+ return np.array([float(x) for x in result.equity_curve])
+
+
+# The two words this example turns on. "SEES NOTHING" describes the METHOD,
+# never the strategy: the leak below is there whatever any audit reports.
+BLIND, CAUGHT = "SEES NOTHING", "CATCHES THE LEAK"
+
+
+if __name__ == "__main__":
+ global_mean = float(frame["close"].mean())
+ store = store_of(frame, "full")
+
+ # -- 1. The seduction -----------------------------------------------------
+ leaked = mbt.run(leaky(global_mean), config_of(frame), store)
+ honest = mbt.run(causal(60), config_of(frame), store)
+
+ print(f"Global mean over all {N_DAYS} days: {global_mean:.4f}")
+ print("(at bar 0, 749 of those days have not happened yet)")
+ print("\nThis strategy IS leaking. Below: which audits notice.\n")
+ for label, r in (("with the leak ", leaked), ("rolling mean ", honest)):
+ m = r.metrics
+ print(f" {label} return {100 * m['total_return']:+8.2f}% "
+ f"sharpe {m['sharpe']:5.2f} trades {r.trade_count}")
+
+ # -- 2. The detector -------------------------------------------------------
+ from manifoldbt.diagnostics import detect_lookahead
+
+ report = detect_lookahead(leaky(global_mean), config_of(frame), store, mode="all")
+ compared = sum(r.total_trades_overlap for r in report.reports)
+ verdict = BLIND if report.passed else CAUGHT
+ print(f"\n [1] detect_lookahead .......... {verdict}")
+ print(f" ({compared} trades compared, so the verdict is not empty)")
+
+ # -- 3. Perturbing the future ---------------------------------------------
+ split = 500
+ reference = equity(mbt.run(leaky(global_mean), config_of(frame), store))
+ corrupted = frame.copy()
+ tail = slice(split + 1, None)
+ factor = 1.0 + np.random.default_rng(99).uniform(-0.05, 0.05, N_DAYS - split - 1)
+ for col in ("open", "high", "low", "close"):
+ corrupted.loc[corrupted.index[tail], col] = corrupted[col].to_numpy()[tail] * factor
+ perturbed = equity(mbt.run(leaky(global_mean), config_of(frame),
+ store_of(corrupted, "pert")))
+ n = min(len(reference), len(perturbed), split + 1)
+ drift = float(np.abs(reference[:n] - perturbed[:n]).max())
+ verdict = CAUGHT if drift > 0 else BLIND
+ print(f" [2] future perturbation ....... {verdict}")
+ print(f" (past moved by {drift:.3e} while the future moved by "
+ f"{abs(reference[-1] - perturbed[-1]):.0f})")
+
+ # -- 4. The method that works ---------------------------------------------
+ # Treat the mean as what it is: a pipeline step, not a constant. A
+ # researcher standing at bar 500 only has the first 500 bars.
+ truncated = frame.iloc[:split + 1]
+ honest_mean = float(truncated["close"].mean())
+ with_future = equity(mbt.run(leaky(global_mean), config_of(truncated),
+ store_of(truncated, "t1")))
+ with_past = equity(mbt.run(leaky(honest_mean), config_of(truncated),
+ store_of(truncated, "t2")))
+ m = min(len(with_future), len(with_past))
+ gap = float(np.abs(with_future[:m] - with_past[:m]).max())
+ verdict = CAUGHT if gap > 0 else BLIND
+ print(f" [3] re-deriving the parameter . {verdict}")
+ print(f" mean from the whole set : {global_mean:.4f} -> "
+ f"final equity {with_future[-1]:,.0f}")
+ print(f" mean from the past only : {honest_mean:.4f} -> "
+ f"final equity {with_past[-1]:,.0f}")
+ print(f" same window, same bars, equity differs by {gap:.0f}")
+ print("\n Verdict: the leak is real, and only [3] found it. A")
+ print(" 'SEES NOTHING' is a statement about the method, not about")
+ print(" the strategy.")
diff --git a/examples/README.md b/examples/README.md
new file mode 100644
index 0000000..68917c7
--- /dev/null
+++ b/examples/README.md
@@ -0,0 +1,112 @@
+# Examples
+
+Each file demonstrates **one mechanism** of the engine and stops there. They are
+documentation that happens to execute, not strategies: several would lose money
+traded as written, and that is not a defect.
+
+## The contract
+
+Every example opens with the same four blocks, in this order:
+
+```
+"""One-line title — the mechanism, named.
+
+Demonstrates:
+ - the two or three API surfaces the file exists to show
+
+Data:
+
+Usage:
+ python examples/NN_name.py
+"""
+```
+
+`Data:` is the block that matters most, and it is not optional. An example is
+read by someone deciding whether to trust a number, so where the number came
+from is part of the example.
+
+## The three data provenances
+
+**`Data: shared store`** — real market data from `data/`, ingested once and
+reused. These examples need that store to exist (see below). Their results are
+real in the sense that the prices are real; they are still not investment
+advice, and none of the strategies is tuned.
+
+**`Data: self-contained (network)`** — the file ingests what it needs on each
+run, from a free connector. Runs on a fresh clone, needs a network.
+
+**`Data: synthetic (seed N)`** — the file generates its own series. Always
+reproducible, never a market claim.
+
+## The rule on synthetic data
+
+Synthetic data is legitimate and often the right choice: it isolates a
+mechanism from market noise, and it lets an example run anywhere with no
+dependency. It is used deliberately here, not as a fallback.
+
+The one thing it must never do is let a reader mistake a fixture for a result.
+So when the construction of the series **determines the outcome**, the file says
+so before printing any figure, and marks the affected numbers. Two examples are
+in that case:
+
+| | What the fixture determines |
+|---|---|
+| `21_fill_at_computed_level.py` | the series is detrended, so mean reversion cannot lose on it — only the *gap* between two execution prices is meaningful |
+| `25_lookahead_trap.py` | the leak is deliberate; the file exists to show which audits fail to see it |
+
+A synthetic example whose fixture does **not** predetermine the outcome carries
+no such warning, because there is nothing to warn about.
+
+## The rule on performance
+
+No example is selected, tuned, or presented for its returns. Where a figure
+appears it illustrates the mechanism under discussion. If an example ever
+produced a reproducible edge on real data, it would belong in a private
+repository rather than in documentation.
+
+## The shared store
+
+Examples marked `Data: shared store` read `data/` and `metadata/`, which are not
+in the repository. Populate them once:
+
+```python
+import manifoldbt as mbt
+mbt.ingest(provider="binance", symbol="BTCUSDT", symbol_id=1,
+ start="2021-01-01T00:00:00Z", end="2025-01-01T00:00:00Z")
+```
+
+Add the symbols an example asks for; each names them in its `universe`. The
+self-contained and synthetic examples need none of this.
+
+## Index
+
+| | Example | Mechanism | Data |
+|---|---|---|---|
+| 00 | `00_template.py` | the minimal shape of a backtest | shared store |
+| 01 | `01_trend_following.py` | fluent builder, EMA, stop-loss, diagnostics | shared store |
+| 02 | `02_mean_reversion.py` | bands, z-score, conditional sizing | shared store |
+| 03 | `03_multi_asset_momentum.py` | cross-sectional ranking over a universe | shared store |
+| 04 | `04_linear_regression.py` | rolling regression as a signal | shared store |
+| 05 | `05_stat_arb.py` | pair spread, cross-asset references | shared store |
+| 06 | `06_full_visualization.py` | the whole plotting surface | shared store |
+| 07 | `07_walk_forward.py` | fold geometries, out-of-sample selection | shared store |
+| 08 | `08_sweep_2d_heatmap.py` | a two-parameter sweep, read as a surface | shared store |
+| 09 | `09_surface_3d.py` | the same surface in three dimensions | shared store |
+| 10 | `10_monte_carlo.py` | bootstrap resampling, rare-event metrics | shared store |
+| 11 | `11_portfolio.py` | several strategies in one book, rebalancing | shared store |
+| 12 | `12_diagnostics.py` | look-ahead, exposure, risk checks | shared store |
+| 13 | `13_stochastic_simulation.py` | SDE paths from the expression DSL | synthetic |
+| 14 | `14_multi_timeframe.py` | higher timeframes, and how periods count | shared store |
+| 15 | `15_cross_exchange.py` | signal on one venue, execution on another | shared store |
+| 16 | `16_hashrate_exogene.py` | an exogenous series joined onto the bars | shared store |
+| 17 | `17_per_venue_fees.py` | per-venue fee schedules in one book | shared store |
+| 18 | `18_csv_import.py` | CSV / MT4 / MT5 import | synthetic |
+| 19 | `19_custom_indicators.py` | composing an indicator from primitives | shared store |
+| 20 | `20_entry_orders.py` | limit, stop and market-if-touched entries | shared store |
+| 21 | `21_fill_at_computed_level.py` | filling at a level computed in advance | synthetic ⚠ |
+| 22 | `22_yahoo_equities.py` | stocks, ETFs, indices, FX via Yahoo | network |
+| 23 | `23_deribit_options.py` | option contracts that expire and settle | network |
+| 24 | `24_option_spread.py` | a two-leg option structure | network |
+| 25 | `25_lookahead_trap.py` | the look-ahead no re-run can detect | synthetic ⚠ |
+
+⚠ marks the two files whose fixture determines the outcome, as described above.
diff --git a/pyproject.toml b/pyproject.toml
index 25e2146..b6e752c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "manifoldbt"
-version = "0.18.0"
+version = "0.19.0"
description = "Rust-powered backtesting engine for quantitative research"
requires-python = ">=3.9"
license = { file = "LICENSE" }
diff --git a/python/manifoldbt/__init__.py b/python/manifoldbt/__init__.py
index 480ac07..d246665 100644
--- a/python/manifoldbt/__init__.py
+++ b/python/manifoldbt/__init__.py
@@ -54,7 +54,7 @@ from manifoldbt.exceptions import (
LicenseError,
StrategyError,
)
-from manifoldbt.expr import AssetRef, Expr, TimeframeRef, asset, col, exo, hold, lit, param, s, scan, symbol_ref, tf, when
+from manifoldbt.expr import AssetRef, Expr, TimeframeRef, asset, choice, col, exo, hold, lit, param, s, scan, symbol_ref, tf, when
from manifoldbt.helpers import (
ExecutionPrice,
FillModel,
@@ -452,9 +452,78 @@ def _prepare_config(config: BacktestConfig, strategy, store: DataStore) -> Backt
# so the engine applies them per-strategy. This lets one batch/sweep call run
# strategies carrying different brackets over a single data load. A bracket
# set directly on config.execution.orders still applies as the fallback.
+
+ _attach_option_contracts(cfg, store)
return cfg
+def _attach_option_contracts(cfg: BacktestConfig, store: DataStore) -> None:
+ """Fill ``cfg.option_contracts`` from what the store recorded at ingest.
+
+ The terms come from the venue, so nothing here is guessed. The one thing the
+ caller must supply is ``option_underlyings``: Deribit settles against its own
+ index, whose ticker matches no series anyone can ingest, so which price
+ stands in for it is a decision, not a lookup. Getting it wrong silently would
+ settle every contract against the wrong number, so a missing entry raises.
+ """
+ if cfg.option_contracts:
+ return # explicitly overridden by the caller
+ try:
+ available = store.option_contracts()
+ except AttributeError:
+ return # store predates option support (mock stores in tests)
+
+ universe = cfg.universe if isinstance(cfg.universe, list) else []
+ in_universe = {int(sid) for sid in universe if isinstance(sid, int)}
+ underlyings = {int(k): int(v) for k, v in (cfg.option_underlyings or {}).items()}
+
+ # An option whose terms were never recorded is the dangerous case: it
+ # prices, it trades, it never expires, and nothing looks wrong. Catch it
+ # before the engine sees a plain price series.
+ try:
+ classes = store.asset_classes()
+ except AttributeError:
+ classes = {}
+ untermed = [
+ int(sid)
+ for sid, klass in classes.items()
+ if klass == "EquityOption"
+ and int(sid) in in_universe
+ and int(sid) not in {int(k) for k in available}
+ ]
+ if untermed:
+ names = {int(i): t for i, t in store.list_symbols()}
+ listed = ", ".join(f"{sid} ({names.get(sid, '?')})" for sid in sorted(untermed))
+ raise ValueError(
+ f"symbol(s) {listed} are recorded as options but carry no contract terms. "
+ "The connector that ingested them does not report a strike and an expiration, "
+ "so the engine would hold them forever at their last quoted premium instead of "
+ "settling them. Re-ingest from a connector that reports contract terms "
+ "(deribit, databento), or set config.option_contracts by hand."
+ )
+
+ missing = []
+ contracts = {}
+ for sid, terms in available.items():
+ sid = int(sid)
+ if sid not in in_universe:
+ continue
+ if sid not in underlyings:
+ missing.append(sid)
+ continue
+ contracts[sid] = dict(terms, underlying_id=underlyings[sid])
+
+ if missing:
+ names = {int(i): t for i, t in store.list_symbols()}
+ listed = ", ".join(f"{sid} ({names.get(sid, '?')})" for sid in sorted(missing))
+ raise ValueError(
+ f"option symbol(s) {listed} have contract terms but no settlement "
+ "underlying. Set config.option_underlyings = {option_id: underlying_id}; "
+ "an option cannot be settled against its own last traded premium."
+ )
+ cfg.option_contracts = contracts
+
+
def _is_sub_daily(res: Any) -> bool:
"""Return True if an Interval dict represents sub-daily resolution."""
if not isinstance(res, dict):
@@ -612,7 +681,8 @@ def ingest(
"""Ingest bars from a data provider into the Arrow IPC store.
Providers (free): ``"binance"``, ``"bybit"``, ``"hyperliquid"``, ``"dydx"``,
- ``"bitstamp"``. Pro: ``"databento"``, ``"massive"``.
+ ``"bitstamp"``, ``"deribit"``, ``"yahoo"`` (alias ``"yfinance"``).
+ Pro: ``"databento"``, ``"massive"``.
Returns a :class:`DataStore` ready for :func:`run`.
@@ -634,6 +704,50 @@ def ingest(
start="2020-06-01T00:00:00Z",
end="2026-03-01T00:00:00Z",
)
+
+ Example (stocks, ETFs, indices, FX and futures via Yahoo Finance)::
+
+ store = bt.ingest(
+ provider="yahoo",
+ symbol="AAPL",
+ symbol_id=1,
+ start="2015-01-01T00:00:00Z",
+ end="2026-01-01T00:00:00Z",
+ interval="1d",
+ asset_class="equity",
+ )
+
+ Yahoo caps its own history: 1m goes back 30 days, 1h about 2 years, daily
+ to the listing date. Prices are dividend-adjusted like ``yfinance``'s
+ ``auto_adjust=True``; pass ``dataset="raw"`` for unadjusted quotes.
+
+ Example (a Deribit option, including one that has already expired)::
+
+ store = bt.ingest(
+ provider="deribit",
+ symbol="BTC-27JUN25-100000-C",
+ symbol_id=2,
+ start="2025-05-01T00:00:00Z",
+ end="2025-07-01T00:00:00Z",
+ interval="1d",
+ asset_class="option",
+ )
+
+ Deribit is the only free connector here that serves expired contracts, which
+ is what an option backtest needs. The strike, expiration, side and settlement
+ style are read from the venue and stored beside the bars, so the engine can
+ settle the contract instead of holding it forever. Prices are quoted in the
+ base currency, so such a backtest is denominated in BTC, ``initial_capital``
+ included. Set ``config.option_underlyings`` to say which series settles it.
+
+ ``databento`` and ``massive`` (both Pro) report the same terms for US listed
+ options: Databento from the ``definition`` schema of a dataset such as
+ ``OPRA.PILLAR``, Massive from ``/v3/reference/options/contracts`` on an OSI
+ ticker like ``"O:SPY251219C00650000"``. Two things differ from Deribit.
+ Positions are counted in units of the underlying, so one 100-multiplier
+ contract is a position of 100. And US listed equity options are physically
+ settled, which the engine models as cash at intrinsic: exact for an index
+ option, an approximation for a single-stock one.
"""
_PRO_PROVIDERS = {"databento", "massive"}
if provider in _PRO_PROVIDERS:
@@ -1642,6 +1756,7 @@ __all__ = [
"s",
"scan",
"symbol_ref",
+ "choice",
"tf",
"when",
# Strategy & config
diff --git a/python/manifoldbt/cli.py b/python/manifoldbt/cli.py
index cf387e9..a052068 100644
--- a/python/manifoldbt/cli.py
+++ b/python/manifoldbt/cli.py
@@ -18,7 +18,14 @@ def main() -> None:
# ── ingest ────────────────────────────────────────────────────────────
ing = sub.add_parser("ingest", help="Ingest bars from a data provider")
- ing.add_argument("--provider", required=True, help="binance | bybit | hyperliquid | databento")
+ ing.add_argument(
+ "--provider",
+ required=True,
+ help=(
+ "binance | bybit | hyperliquid | dydx | bitstamp | deribit | yahoo "
+ "| databento | massive"
+ ),
+ )
ing.add_argument("--symbol", required=True, help="e.g. BTCUSDT, ESH5")
ing.add_argument("--symbol-id", required=True, type=int, help="Unique integer ID for this symbol")
ing.add_argument("--start", required=True, help="RFC3339 start (e.g. 2025-01-01T00:00:00Z)")
diff --git a/python/manifoldbt/config.py b/python/manifoldbt/config.py
index 2bb0c65..20539a2 100644
--- a/python/manifoldbt/config.py
+++ b/python/manifoldbt/config.py
@@ -375,6 +375,32 @@ class BacktestConfig:
"""Explicit mapping from signal symbol to execution symbol.
Required when signal and execution have different tickers.
Example: ``{"BTC-USDT:perp": "BTC-USD:perp"}``"""
+ option_underlyings: Dict[int, int] = field(default_factory=dict)
+ """Maps an option symbol id to the symbol whose price settles it.
+
+ Required for every option in the universe. Deribit settles against its own
+ index, whose ticker matches no series you can ingest, so the substitute is
+ yours to name (``BTC-PERPETUAL`` in practice). The engine refuses to run an
+ option without one rather than settle it against its own last traded
+ premium, which on an illiquid strike is days stale.
+ Example: ``{2: 1}`` to settle option id 2 against symbol id 1."""
+ option_margin_model: str = "none"
+ """Margin formula short option positions pay: ``"none"`` or ``"deribit"``.
+
+ ``"none"`` charges nothing, which is only honest when the strategy never
+ sells an option. ``"deribit"`` applies the venue's published per-contract
+ formula, refuses a short that does not fit initial margin, and force-closes
+ the book when maintenance margin passes equity."""
+ option_contracts: Dict = field(default_factory=dict)
+ """Contract terms per option symbol id. Filled automatically from the data
+ store at run time; set it by hand only to override what was ingested.
+
+ Positions on an option are counted in **units of the underlying**, not in
+ exchange contracts. On Deribit the two are the same thing (contract size 1).
+ On a listed equity option, one contract is 100 units: to hold one SPY
+ contract quoted at 4.70, target 100, which costs the 470 a contract costs
+ and settles for what a contract settles for. ``contract_size`` in the terms
+ is what converts a position back into contracts."""
# Deprecated — kept for backward compat
provider: Optional[str] = None
exo_sources: Dict = field(default_factory=dict)
@@ -417,6 +443,12 @@ class BacktestConfig:
d["signal_source"] = self.signal_source
if self.execution_source:
d["execution_source"] = self.execution_source
+ if self.option_contracts:
+ d["option_contracts"] = {
+ str(sid): spec for sid, spec in self.option_contracts.items()
+ }
+ if self.option_margin_model and self.option_margin_model != "none":
+ d["option_margin_model"] = self.option_margin_model
# Deprecated fields (backward compat)
if self.provider:
d["provider"] = self.provider
diff --git a/python/manifoldbt/diagnostics.py b/python/manifoldbt/diagnostics.py
index 3955d6d..a734dea 100644
--- a/python/manifoldbt/diagnostics.py
+++ b/python/manifoldbt/diagnostics.py
@@ -111,11 +111,24 @@ def detect_lookahead(
Data is loaded once and sliced for each sub-test (no redundant I/O).
Two sub-tests:
- * **extension** — split at 2/3 of the period. Catches *global*
- look-ahead (e.g. ``np.mean(all_prices)`` instead of rolling).
+ * **extension** — split at 2/3 of the period. Catches look-ahead that
+ depends on how much data the run was given.
* **truncation** — split at 1/3 of the period. Catches *rolling*
look-ahead (e.g. signal at bar T using bar T+1).
+ .. warning::
+ **What this cannot see.** Both sub-tests re-run the *same strategy* on
+ a shorter window. A parameter computed from the data *before* the
+ backtest — ``threshold = df.close.mean()`` in a notebook, then passed in
+ as a number — is unchanged by re-running, so the trades match and the
+ verdict is PASS. The leak already happened, outside the engine.
+
+ This is a property of every re-run-based method, not a gap to be closed
+ here: no such test can audit a constant. The defence is to treat any
+ parameter derived from data as part of the pipeline and re-derive it on
+ the window under test. ``examples/25_lookahead_trap.py`` demonstrates
+ the blind spot and the technique that does catch it.
+
Args:
strategy: Strategy definition.
config: BacktestConfig.
diff --git a/python/manifoldbt/expr.py b/python/manifoldbt/expr.py
index db0203b..d1a5746 100644
--- a/python/manifoldbt/expr.py
+++ b/python/manifoldbt/expr.py
@@ -182,6 +182,17 @@ class Expr:
if v == "IfElse":
return {v: [args[0].to_json(), args[1].to_json(), args[2].to_json()]}
+ if v == "Choice":
+ # Choice(String, Vec<(String, Expr)>) -- serde attend une liste de
+ # paires, pas un dict : l'ORDRE des branches est porteur (la
+ # premiere sert de defaut a la compilation initiale).
+ return {"Choice": [args[0], [[k, e.to_json()] for k, e in args[1]]]}
+
+ if v == "OnTimeframe":
+ # OnTimeframe(String, Box) -- l'expression est evaluee sur la
+ # grille de la timeframe nommee puis etalee en escalier.
+ return {"OnTimeframe": [args[0], args[1].to_json()]}
+
if v == "Column":
return {"Column": args[0]}
if v == "Literal":
@@ -518,6 +529,62 @@ def when(condition: Expr, true_value: Any = 1.0, false_value: Any = float("nan")
return Expr("IfElse", condition, _coerce(true_value), _coerce(false_value))
+def choice(name: str, branches: "dict[str, Expr]", *, description: str = "") -> Expr:
+ """Balayer un CHOIX d'expression, pas seulement un nombre.
+
+ Un ``param()`` ordinaire porte une valeur numerique. ``choice()`` porte un
+ NOM, et chaque nom designe une sous-expression differente. Le moteur
+ remplace le noeud entier par la branche choisie AVANT de simuler, donc une
+ combinaison n'evalue que sa propre branche : les autres n'existent plus.
+
+ C'est ce qui le distingue d'un ``when()`` imbrique, qui construit toutes
+ les variantes et tranche barre par barre.
+
+ Usage (balayer la timeframe d'une bande, sim en 1m)::
+
+ bande = mbt.choice("band", {
+ "30m": mbt.tf("30m").apply(sma(close, mbt.param("len"))),
+ "1h": mbt.tf("1h").apply(sma(close, mbt.param("len"))),
+ "2h": mbt.tf("2h").apply(sma(close, mbt.param("len"))),
+ })
+ # grille : {"len": [10, 20, 30], "band": ["30m", "1h", "2h"]}
+
+ Noter le ``apply()``. Ecrit ``sma(mbt.tf("30m").close, param("len"))``, le
+ balayage n'aurait pas le sens attendu : la periode compterait des barres de
+ SIMULATION sur une colonne etalee en escalier, donc les trois branches
+ lisseraient le meme nombre de MINUTES au lieu de 10, 20 ou 30 bougies de
+ leur timeframe. Voir :func:`tf`.
+
+ Les branches acceptent n'importe quelle expression, donc le meme mecanisme
+ balaie une colonne exogene, un actif ou un type d'indicateur.
+
+ Args:
+ name: nom du parametre selecteur, a mettre dans la grille.
+ branches: nom de branche -> expression. L'ordre compte : la premiere
+ sert de defaut quand le parametre est absent.
+ description: metadonnee libre.
+
+ Raises:
+ ValueError: si ``branches`` est vide.
+ """
+ if not branches:
+ raise ValueError(
+ f"choice({name!r}) needs at least one branch; an empty choice has "
+ f"nothing to resolve to."
+ )
+ items = [(str(k), _coerce(v)) for k, v in branches.items()]
+ expr = Expr("Choice", name, items)
+ # Declare le selecteur comme un parametre a part entiere, sans quoi le
+ # balayer serait refuse par la validation ("parameter not declared").
+ expr._param_meta = {
+ "name": name,
+ "default": items[0][0],
+ "range": None,
+ "description": description,
+ }
+ return expr
+
+
def exo(name: str, column: Optional[str] = None) -> Expr:
"""Reference an exogenous data column.
@@ -626,6 +693,28 @@ class TimeframeRef:
"""Reference any column from this timeframe."""
return col(f"{self._tf}.{name}")
+ def apply(self, expr: "Expr") -> Expr:
+ """Evaluate *expr* ON this timeframe's own grid, then step-hold the
+ result back onto the simulation grid (forward-filled, no lookahead:
+ a completed bar's value becomes readable from the next bar on).
+
+ This is what makes higher-timeframe INDICATORS correct. Periods
+ inside *expr* count in THIS timeframe's bars::
+
+ h1 = bt.tf("1h")
+ band = h1.apply(sma(close, mbt.param("len"))) # len = HOURS
+
+ is a true SMA of ``len`` hourly closes, sweepable like any param.
+ By contrast ``sma(h1.close, 20)`` counts 20 SIMULATION bars over a
+ step-held hourly series -- on a 1m simulation that is a 20-MINUTE
+ smoothing of a staircase, not a 20-hour average.
+
+ Inside *expr*, ``close``/``open``/... refer to this timeframe's own
+ resampled columns. Requires ``extra_timeframes`` to declare the
+ timeframe. Nesting ``apply`` inside another ``apply`` is rejected.
+ """
+ return Expr("OnTimeframe", self._tf, _coerce(expr))
+
def __repr__(self) -> str:
return f"TimeframeRef({self._tf!r})"
@@ -633,12 +722,48 @@ class TimeframeRef:
def tf(timeframe: str) -> TimeframeRef:
"""Reference a higher timeframe for multi-TF strategies.
- Usage::
+ Two different things, and the distinction matters::
h1 = bt.tf("1h")
- trend = ema(h1.close, 20) > ema(h1.close, 50)
+
+ h1.close # a COLUMN: the last closed hourly
+ # close, held across the minute bars
+ h1.apply(ema(close, 20)) # an INDICATOR on the hourly grid:
+ # 20 counts hourly candles
Requires ``extra_timeframes={"1h": Interval.hours(1)}`` in config.
+
+ .. warning::
+ **An indicator applied to** ``h1.close`` **counts SIMULATION bars, not
+ candles of the higher timeframe.** The column is forward-filled onto the
+ simulation grid, so an indicator over it counts rows of that grid.
+
+ On 1-minute bars, ``sma(h1.close, 8)`` averages the last 8 *minutes* of
+ a step function — which is the last closed hourly close, not an 8-hour
+ average. Measured on a ramp of +10/hour, it lags 1.63 h where a true
+ 8-hour mean lags 5.45 h.
+
+ Multiplying by the ratio of the two intervals does not fix it either.
+ ``sma(h1.close, 8 * 60)`` averages 480 rows of the step function: at
+ every move it ramps in over 60 minutes instead of stepping, and its
+ window spans 9 hourly values with unequal weights rather than 8 with
+ equal ones. Measured against a true 8-hour mean on an impulse (one hour
+ at 200, base 100, so the true signal spans 12.5): the error reaches
+ 12.29, or 98 % of that span. A ramp cannot reveal this — a box filter
+ leaves a straight line straight — which is why a lag measurement alone
+ reads correct.
+
+ Use :meth:`TimeframeRef.apply`, which evaluates on the hourly grid and
+ then step-holds the result. On that same impulse it matches the true
+ 8-hour mean exactly, on every bar::
+
+ sma(h1.close, 8) # 8 minutes of a step (gap 12.29)
+ sma(h1.close, 8 * 60) # 480 minutes of a step (gap 12.29)
+ h1.apply(sma(close, 8)) # the 8-hour mean (gap 0.00)
+
+ The ~1 h of lag common to all three is the timeframe itself: an hourly
+ bar is only readable once closed, which is what makes it free of
+ look-ahead.
"""
return TimeframeRef(timeframe)
diff --git a/python/manifoldbt/plot/_convert.py b/python/manifoldbt/plot/_convert.py
index 24b8063..8b6e501 100644
--- a/python/manifoldbt/plot/_convert.py
+++ b/python/manifoldbt/plot/_convert.py
@@ -101,3 +101,77 @@ def trades_arrays(result) -> dict:
else:
out[name] = arrow_to_numpy(col)
return out
+
+
+def run_currency(result) -> str:
+ """Currency the run is denominated in, from its manifest.
+
+ The manifest embeds the full BacktestConfig, so nothing is guessed. "USD"
+ is only the last resort for a result that has no manifest at all (mock
+ objects in tests).
+ """
+ try:
+ code = result.manifest["config"]["currency"]
+ return str(code) if code else "USD"
+ except Exception:
+ return "USD"
+
+
+_CURRENCY_PREFIX = {"USD": "$", "EUR": "€", "GBP": "£"}
+
+
+def money_hovertemplate(values: np.ndarray, currency: str) -> str:
+ """Hover template for a money series, currency- and magnitude-aware.
+
+ The old template was a hardcoded "$%{y:,.0f}": a 10-BTC equity hovered as
+ "$10" - wrong currency, and a precision that erased every variation the
+ chart existed to show. Decimals follow the magnitude of the series, and
+ the currency is written as a symbol when it has one, as a suffix code
+ (10.0443 BTC) when it does not.
+ """
+ peak = float(np.nanmax(np.abs(values))) if len(values) else 0.0
+ decimals = 0 if peak >= 10_000 else 2 if peak >= 100 else 4
+ code = (currency or "USD").upper()
+ amount = "%{y:,." + str(decimals) + "f}"
+ prefix = _CURRENCY_PREFIX.get(code)
+ amount = prefix + amount if prefix else amount + " " + code
+ return "%{x|%d %b %Y} " + amount + ""
+
+
+def date_tickformat(dates: np.ndarray) -> str:
+ """Date-axis tick format adapted to the span of the series.
+
+ Hardcoding "%b %Y" labelled every tick of a two-month backtest "May 2025"
+ (and every tick of a 30-day synthetic run "Jan 2024"): the format must
+ follow the span, not assume it. Thresholds are where the coarser format
+ stops producing distinct labels for ~6 ticks.
+ """
+ if len(dates) < 2:
+ return "%b %Y"
+ span_days = float(
+ (np.datetime64(dates[-1], "ns") - np.datetime64(dates[0], "ns"))
+ / np.timedelta64(1, "D")
+ )
+ if span_days <= 3:
+ return "%d %b %H:%M"
+ if span_days <= 180:
+ return "%d %b"
+ # Beyond ~6 months the historical "%b %Y" is already distinct per tick,
+ # whatever the span: plotly widens the tick spacing with the range. Only
+ # the short end was ever broken.
+ return "%b %Y"
+
+
+def percent_tickformat(magnitude: float) -> str:
+ """Percent-axis tick format adapted to the magnitude of the series.
+
+ The date-axis disease, on the value axis: ".0%" labelled every tick of a
+ -0.9% max-drawdown chart "0%". Decimals follow the extreme value, so the
+ ticks always spell out distinct numbers.
+ """
+ m = abs(float(magnitude))
+ if m >= 0.05:
+ return ".0%"
+ if m >= 0.005:
+ return ".1%"
+ return ".2%"
diff --git a/python/manifoldbt/plot/backtest.py b/python/manifoldbt/plot/backtest.py
index 4a06a6c..4e0d4fd 100644
--- a/python/manifoldbt/plot/backtest.py
+++ b/python/manifoldbt/plot/backtest.py
@@ -25,6 +25,10 @@ from manifoldbt.plot._convert import (
positions_arrays,
trades_arrays,
_ts_to_int64,
+ date_tickformat,
+ percent_tickformat,
+ money_hovertemplate,
+ run_currency,
)
from manifoldbt.plot._decimate import maybe_decimate
from manifoldbt.plot._utils import finalize, format_pct, new_figure
@@ -277,10 +281,10 @@ def equity(
dates, values = maybe_decimate(dates, values)
fig.add_traces(_area_traces(
dates, values, float(values.min()), color, width=1.5,
- hovertemplate="%{x|%d %b %Y} $%{y:,.0f}",
+ hovertemplate=money_hovertemplate(values, run_currency(result)),
))
fig.update_yaxes(title_text="Equity")
- fig.update_xaxes(tickformat="%b %Y")
+ fig.update_xaxes(tickformat=date_tickformat(dates))
return finalize(fig, show=show, save=save)
@@ -324,7 +328,7 @@ def benchmark_equity(
line=dict(color=benchmark_color, width=1.0),
))
fig.update_yaxes(title_text="Normalized" if normalize else "Equity")
- fig.update_xaxes(tickformat="%b %Y")
+ fig.update_xaxes(tickformat=date_tickformat(d1))
fig.update_layout(legend=dict(x=0.01, y=0.99))
return finalize(fig, show=show, save=save)
@@ -357,9 +361,10 @@ def drawdown(
hovertemplate="%{x|%d %b %Y} %{y:.1%}",
))
dd_min = float(dd.min()) if len(dd) else -0.01
- fig.update_yaxes(title_text="Drawdown", tickformat=".0%",
+ fig.update_yaxes(title_text="Drawdown",
+ tickformat=percent_tickformat(dd_min),
range=[dd_min * 1.08, 0])
- fig.update_xaxes(tickformat="%b %Y")
+ fig.update_xaxes(tickformat=date_tickformat(dates))
return finalize(fig, show=show, save=save)
diff --git a/python/tests/test_choice.py b/python/tests/test_choice.py
new file mode 100644
index 0000000..d765b70
--- /dev/null
+++ b/python/tests/test_choice.py
@@ -0,0 +1,140 @@
+"""Tests for bt.choice() — sweeping a CHOICE of expression, not just a number.
+
+The contract under test: `choice("sel", {...})` resolves to exactly one branch
+per combo BEFORE simulation, so a sweep over the selector must be
+bit-identical to running each branch inlined by hand. The selector must count
+as a declared parameter (otherwise `_validate_swept_params` would reject the
+sweep), and an unknown branch must fail with a message naming the known ones.
+"""
+import json
+import os
+
+import pytest
+
+import manifoldbt as bt
+from manifoldbt.indicators import close, sma
+
+pd = pytest.importorskip("pandas")
+np = pytest.importorskip("numpy")
+
+N_BARS = 3_000
+
+
+def _store(tmp_path):
+ ts = pd.date_range("2022-01-01", periods=N_BARS, freq="1min", tz="UTC")
+ rng = np.random.default_rng(11)
+ px = 100.0 + np.cumsum(np.sin(np.arange(N_BARS) / 90.0) * 0.3 + rng.normal(0, 0.2, N_BARS)) * 0.05
+ px = np.maximum(px, 1.0)
+ df = pd.DataFrame(
+ {
+ "timestamp": ts,
+ "open": px,
+ "high": px * 1.0005,
+ "low": px * 0.9995,
+ "close": px,
+ "volume": [1000.0] * N_BARS,
+ }
+ )
+ root = tmp_path / "choice_store"
+ return bt.import_dataframe(
+ df,
+ symbol="ZC",
+ symbol_id=1,
+ interval="1m",
+ asset_class="equity",
+ exchange="TEST",
+ data_root=str(root / "data"),
+ metadata_db=str(root / "metadata.sqlite"),
+ )
+
+
+def _config():
+ start, end = bt.time_range("2022-01-01", "2022-01-03")
+ cfg = bt.BacktestConfig(
+ universe=[1],
+ time_range_start=start,
+ time_range_end=end,
+ initial_capital=10_000.0,
+ provider="TEST",
+ bar_interval=bt.Interval.minutes(1),
+ symbol_names={"ZC": 1},
+ )
+ cfg.warmup_bars = 0
+ return cfg
+
+
+def _strategy_with_choice():
+ band = bt.choice(
+ "pick",
+ {
+ "fast": sma(close, 5),
+ "slow": sma(close, 20),
+ },
+ )
+ return (
+ bt.Strategy.create("choice_e2e")
+ .signal("band", band)
+ .size(bt.when(close > bt.col("band"), 1.0, 0.0))
+ )
+
+
+def _strategy_inlined(period):
+ return (
+ bt.Strategy.create(f"inline_{period}")
+ .signal("band", sma(close, period))
+ .size(bt.when(close > bt.col("band"), 1.0, 0.0))
+ )
+
+
+def test_serializes_as_ordered_pairs():
+ """serde expects Choice(String, Vec<(String, Expr)>): a list of pairs,
+ order preserved — the first branch is the compile-time default."""
+ e = bt.choice("pick", {"a": close, "b": sma(close, 3)})
+ payload = json.loads(json.dumps(e.to_json()))
+ assert list(payload) == ["Choice"]
+ name, branches = payload["Choice"]
+ assert name == "pick"
+ assert [k for k, _ in branches] == ["a", "b"]
+
+
+def test_empty_branches_rejected():
+ with pytest.raises(ValueError, match="at least one branch"):
+ bt.choice("pick", {})
+
+
+def test_selector_counts_as_declared_parameter():
+ """Sweeping the selector must pass strategy-side validation: choice()
+ declares it via _param_meta exactly like param() does."""
+ strat = _strategy_with_choice()
+ assert "pick" in (strat.to_json_dict().get("parameters") or {})
+
+
+def test_sweep_over_choice_matches_inlined_branches(tmp_path):
+ """The money test: each combo of the selector sweep is bit-identical to
+ the strategy with that branch written directly."""
+ store = _store(tmp_path)
+ cfg = _config()
+
+ sweep = bt.run_sweep_lite(
+ _strategy_with_choice(), {"pick": ["fast", "slow"]}, cfg, store, device="cpu"
+ )
+ assert len(sweep) == 2
+
+ by_branch = dict(zip(["fast", "slow"], sweep))
+ for name, period in (("fast", 5), ("slow", 20)):
+ ref = bt.run_sweep_lite(
+ _strategy_inlined(period), {}, cfg, store, device="cpu"
+ )[0]
+ got, want = by_branch[name].metrics, ref.metrics
+ for key in ("total_return", "sharpe", "max_drawdown"):
+ assert got.get(key) == want.get(key), (
+ f"branch {name!r}: {key} diverged ({got.get(key)} vs {want.get(key)})"
+ )
+
+
+def test_unknown_branch_names_the_known_ones(tmp_path):
+ store = _store(tmp_path)
+ with pytest.raises(Exception, match="fast"):
+ bt.run_sweep_lite(
+ _strategy_with_choice(), {"pick": ["nope"]}, _config(), store, device="cpu"
+ )
diff --git a/python/tests/test_lookahead_blind_spot.py b/python/tests/test_lookahead_blind_spot.py
new file mode 100644
index 0000000..8bed227
--- /dev/null
+++ b/python/tests/test_lookahead_blind_spot.py
@@ -0,0 +1,161 @@
+"""The look-ahead the detector cannot see, pinned as a characterization test.
+
+`detect_lookahead` used to document itself as catching global look-ahead,
+"e.g. np.mean(all_prices) instead of rolling". It does not, and it cannot: both
+its sub-tests re-run the *same strategy* on a shorter window, so a threshold
+computed in a notebook and passed in as a number is identical in every run.
+
+This file asserts the blind spot on purpose. A test that pins a limitation is
+worth more than a docstring promising the opposite, because the docstring was
+wrong for as long as nobody tried it.
+
+It also pins the method that DOES catch it, so the boundary is not just
+described but demonstrated: re-derive the parameter on the truncated window and
+compare the same prefix.
+"""
+import os
+
+import pytest
+
+import manifoldbt as bt
+
+np = pytest.importorskip("numpy")
+pd = pytest.importorskip("pandas")
+
+from manifoldbt.helpers import Interval, Slippage # noqa: E402
+
+N_DAYS = 400
+SPLIT = 260
+
+
+def _mean_reverting_daily():
+ """A series that pulls back to its mean, where knowing that mean pays."""
+ rng = np.random.default_rng(11)
+ level = np.cumsum(rng.normal(0.0, 0.018, N_DAYS))
+ px = 100.0 * np.exp(level - np.linspace(0, level[-1], N_DAYS))
+ o = px
+ c = np.roll(px, -1)
+ c[-1] = px[-1]
+ amp = np.abs(rng.normal(0.0, 0.004, N_DAYS))
+ return pd.DataFrame({
+ "timestamp": pd.date_range("2022-01-01", periods=N_DAYS, freq="1D", tz="UTC"),
+ "open": o,
+ "high": np.maximum(o, c) * (1 + amp),
+ "low": np.minimum(o, c) * (1 - amp),
+ "close": c,
+ "volume": rng.uniform(1_000, 5_000, N_DAYS),
+ })
+
+
+def _store(frame, tmp_path, tag):
+ root = os.path.join(str(tmp_path), tag)
+ return bt.import_dataframe(
+ frame, symbol="SYNTH", symbol_id=1, interval="1d",
+ data_root=os.path.join(root, "data"),
+ metadata_db=os.path.join(root, "meta.sqlite"),
+ )
+
+
+def _config(frame):
+ ts = pd.DatetimeIndex(frame["timestamp"])
+ return bt.BacktestConfig(
+ universe=[1],
+ time_range_start=int(ts[0].value),
+ time_range_end=int(ts[-1].value) + 86_400_000_000_000,
+ bar_interval=Interval.days(1),
+ initial_capital=10_000,
+ execution=bt.ExecutionConfig(
+ signal_delay=1, max_position_pct=1.0,
+ allow_short=True, position_sizing_mode="FractionOfEquity",
+ ),
+ slippage=Slippage.fixed_bps(0),
+ warmup_bars=0,
+ )
+
+
+def _leaky(mean_price):
+ """The threshold is a number the researcher computed over everything."""
+ from manifoldbt.indicators import close
+
+ return (
+ bt.Strategy.create("global_mean_leak")
+ .signal("edge", close)
+ .size(bt.when(close < mean_price, 1.0, -1.0))
+ )
+
+
+def _equity(result):
+ return np.array([float(x) for x in result.equity_curve])
+
+
+def test_a_parameter_baked_at_research_time_flatters_the_result(tmp_path):
+ """First establish there IS a leak, otherwise the blind spot is moot."""
+ from manifoldbt.indicators import close
+
+ frame = _mean_reverting_daily()
+ store = _store(frame, tmp_path, "seduction")
+ global_mean = float(frame["close"].mean())
+
+ leaked = bt.run(_leaky(global_mean), _config(frame), store)
+ honest = bt.run(
+ bt.Strategy.create("rolling")
+ .signal("edge", close)
+ .size(bt.when(close < close.rolling_mean(60), 1.0, -1.0)),
+ _config(frame), store,
+ )
+
+ assert leaked.metrics["total_return"] > honest.metrics["total_return"], (
+ "the global mean did not flatter the result, so this fixture no longer "
+ "demonstrates a leak worth detecting"
+ )
+
+
+def test_the_detector_is_blind_to_it(tmp_path):
+ """Pinned limitation: PASS here is the documented, expected answer.
+
+ If this ever starts failing, the detector gained the ability to audit a
+ baked parameter. That would be good news, and the warning in
+ `detect_lookahead`'s docstring should be revisited rather than this test
+ silenced.
+ """
+ from manifoldbt.diagnostics import detect_lookahead
+
+ frame = _mean_reverting_daily()
+ result = detect_lookahead(
+ _leaky(float(frame["close"].mean())),
+ _config(frame), _store(frame, tmp_path, "blind"), mode="all",
+ )
+
+ compared = sum(r.total_trades_overlap for r in result.reports)
+ assert compared > 0, "empty verdict, the blind spot is not what is being shown"
+ assert result.passed, (
+ "the detector now catches a research-time constant; update the docstring "
+ "warning instead of deleting this test"
+ )
+
+
+def test_re_deriving_the_parameter_catches_it(tmp_path):
+ """The technique that works, and the reason the blind spot is acceptable.
+
+ Same window, same strategy shape: only the threshold differs, one computed
+ with the future and one without. The equity must diverge.
+ """
+ frame = _mean_reverting_daily()
+ truncated = frame.iloc[:SPLIT + 1]
+
+ with_future = _equity(bt.run(
+ _leaky(float(frame["close"].mean())), # knows all 400 days
+ _config(truncated), _store(truncated, tmp_path, "future"),
+ ))
+ with_past = _equity(bt.run(
+ _leaky(float(truncated["close"].mean())), # knows only the first 261
+ _config(truncated), _store(truncated, tmp_path, "past"),
+ ))
+
+ n = min(len(with_future), len(with_past))
+ assert n > 100, f"only {n} bars compared, too few to conclude"
+ gap = float(np.abs(with_future[:n] - with_past[:n]).max())
+ assert gap > 0.0, (
+ "re-deriving the threshold changed nothing, so this method would not "
+ "catch the leak either"
+ )
diff --git a/python/tests/test_lookahead_custom_exec.py b/python/tests/test_lookahead_custom_exec.py
new file mode 100644
index 0000000..f5397aa
--- /dev/null
+++ b/python/tests/test_lookahead_custom_exec.py
@@ -0,0 +1,285 @@
+"""Anti-look-ahead tests for `ExecutionPrice.custom(...)` with `signal_delay=0`.
+
+This is the configuration of `examples/21_fill_at_computed_level.py`, and the
+one with the most room for leakage in the whole engine: the fill price is read
+from a strategy signal, the order acts on the same bar it was computed on, and
+the level itself comes from a higher timeframe. Three chances for a bar to be
+priced with information it could not have had.
+
+Two independent methods, because a single one can pass for the wrong reason:
+
+ * **future perturbation** — corrupt every bar after K, re-run, and require
+ the equity of bars 0..K to be *bit-identical*. Any decision that read a
+ future bar moves the prefix.
+ * **the engine's own detector** — `detect_lookahead`, which compares the
+ trades of truncated runs against the full run.
+
+Both carry an anti-vacuity guard. A look-ahead test that compares nothing
+passes just as loudly as one that compares everything, which is exactly how
+the built-in detector reports PASS when its splits fall outside the data.
+"""
+import os
+import tempfile
+
+import pytest
+
+import manifoldbt as bt
+
+np = pytest.importorskip("numpy")
+pd = pytest.importorskip("pandas")
+
+from manifoldbt.helpers import ExecutionPrice, Interval, Slippage # noqa: E402
+
+# Three days of 1-minute bars: enough for the hourly SMA to have a history,
+# small enough not to weigh on the suite.
+N_BARS = 3 * 1440
+SPLIT = 2 * 1440 # perturb everything after this bar
+
+
+def _mean_reverting_bars(seed=7):
+ """The construction of example 21, at a size a test can afford."""
+ rng = np.random.default_rng(seed)
+ steps = rng.normal(0.0, 0.0010, N_BARS)
+ level = np.cumsum(steps) * 0.85
+ px = 100.0 * np.exp(level - np.linspace(0, level[-1], N_BARS))
+ o = px
+ c = np.roll(px, -1)
+ c[-1] = px[-1]
+ amp = np.abs(rng.normal(0.0, 0.0012, N_BARS))
+ return pd.DataFrame({
+ "timestamp": pd.date_range("2024-01-01", periods=N_BARS, freq="1min", tz="UTC"),
+ "open": o,
+ "high": np.maximum(o, c) * (1 + amp),
+ "low": np.minimum(o, c) * (1 - amp),
+ "close": c,
+ "volume": rng.uniform(1_000, 5_000, N_BARS),
+ })
+
+
+def _band_strategy():
+ """Short the upper band, cover the lower one, filling ON the band."""
+ from manifoldbt.indicators import close, high, low, open as open_px, sma
+
+ h1 = bt.tf("1h")
+ band_up = sma(h1.close, 8) * 1.004
+ band_dn = sma(h1.close, 8) * 0.997
+ touch_up = high >= band_up
+ touch_dn = low <= band_dn
+ target = bt.when(touch_dn, 0.0, bt.when(touch_up, -1.0))
+ exec_level = bt.when(
+ touch_dn, bt.when(open_px <= band_dn, open_px, band_dn),
+ bt.when(touch_up, bt.when(open_px >= band_up, open_px, band_up), close),
+ )
+ return (
+ bt.Strategy.create("band_touch_short")
+ .signal("position", target)
+ .signal("exec_level", exec_level)
+ .size(target)
+ .stop_loss(pct=25.0)
+ )
+
+
+def _store(frame, tmp_path, tag):
+ root = os.path.join(str(tmp_path), tag)
+ return bt.import_dataframe(
+ frame, symbol="SYNTH", symbol_id=1, interval="1m",
+ data_root=os.path.join(root, "data"),
+ metadata_db=os.path.join(root, "meta.sqlite"),
+ )
+
+
+def _config(frame):
+ """A time range bounded by the DATA, not by the epoch.
+
+ `time_range_start=0` would stretch the range over 54 years, which is what
+ makes the built-in detector split outside the data and pass vacuously.
+ """
+ ts = pd.DatetimeIndex(frame["timestamp"])
+ return bt.BacktestConfig(
+ universe=[1],
+ time_range_start=int(ts[0].value),
+ time_range_end=int(ts[-1].value) + 60_000_000_000,
+ bar_interval=Interval.minutes(1),
+ initial_capital=10_000,
+ execution=bt.ExecutionConfig(
+ signal_delay=0,
+ execution_price=ExecutionPrice.custom("exec_level"),
+ max_position_pct=0.4,
+ allow_short=True,
+ position_sizing_mode="FractionOfEquity",
+ ),
+ fees=bt.FeeConfig.binance_perps(),
+ slippage=Slippage.fixed_bps(2),
+ warmup_bars=60 * 4,
+ extra_timeframes={"1h": Interval.hours(1)},
+ )
+
+
+def _equity(result):
+ return np.array([float(x) for x in result.equity_curve])
+
+
+def test_future_bars_cannot_move_the_past(tmp_path):
+ """The decisive test: corrupt the future, the past must not budge."""
+ frame = _mean_reverting_bars()
+ strategy = _band_strategy()
+ config = _config(frame)
+
+ reference = _equity(bt.run(strategy, _config(frame), _store(frame, tmp_path, "ref")))
+
+ # Same multiplicative factor on all four price columns, so the bars stay
+ # valid (high >= max(open, close), low <= min(open, close)). Small enough
+ # that the short strategy survives it: a 3x future turns the equity
+ # negative and the run refuses, which would prove nothing.
+ rng = np.random.default_rng(1234)
+ corrupted = frame.copy()
+ tail = slice(SPLIT + 1, None)
+ factor = 1.0 + rng.uniform(-0.005, 0.005, N_BARS - SPLIT - 1)
+ for col in ("open", "high", "low", "close"):
+ corrupted.loc[corrupted.index[tail], col] = corrupted[col].to_numpy()[tail] * factor
+
+ perturbed = _equity(bt.run(strategy, config, _store(corrupted, tmp_path, "pert")))
+
+ # Anti-vacuity: if the corruption changed nothing at all, an identical
+ # prefix would be meaningless.
+ assert abs(reference[-1] - perturbed[-1]) > 1e-6, (
+ "the perturbation left the future untouched; the test would be vacuous"
+ )
+
+ n = min(len(reference), len(perturbed), SPLIT + 1)
+ assert n > 1000, f"only {n} bars compared, too few to conclude"
+ delta = np.abs(reference[:n] - perturbed[:n])
+ first = int(np.argmax(delta > 0)) if delta.max() > 0 else -1
+ assert delta.max() == 0.0, (
+ f"future data leaked into the past: bar {first} differs by {delta.max():.3e}"
+ )
+
+
+def test_builtin_detector_agrees_and_is_not_vacuous(tmp_path):
+ """The engine's own detector, plus a check that it compared something.
+
+ `trades=0, mismatched=0` is reported as PASS. Asserting only on `.passed`
+ would accept that empty verdict.
+ """
+ from manifoldbt.diagnostics import detect_lookahead
+
+ frame = _mean_reverting_bars()
+ result = detect_lookahead(
+ _band_strategy(), _config(frame), _store(frame, tmp_path, "det"), mode="all"
+ )
+
+ assert result.passed, f"look-ahead reported: {result}"
+ compared = sum(r.total_trades_overlap for r in result.reports)
+ assert compared > 0, (
+ f"the detector compared no trade at all, its PASS is empty: {result.reports}"
+ )
+
+
+def test_every_fill_lands_on_a_level_known_before_the_bar(tmp_path):
+ """No fill may be priced at its own bar's close.
+
+ A fill at the close is only knowable once the bar is over. It is also what
+ the example would produce if `custom(...)` silently fell back to AtClose,
+ which would make the whole feature a no-op.
+ """
+ frame = _mean_reverting_bars()
+ result = bt.run(_band_strategy(), _config(frame), _store(frame, tmp_path, "fills"))
+
+ trades = result.trades_df()
+ assert len(trades) > 20, f"only {len(trades)} trades, too few to conclude"
+
+ bars = frame.set_index(pd.DatetimeIndex(frame["timestamp"]))
+ at = bars.reindex(pd.DatetimeIndex(pd.to_datetime(trades["execution_timestamp"], utc=True)))
+ intended = trades["intended_price"].to_numpy()
+
+ on_close = np.isclose(intended, at["close"].to_numpy(), rtol=0, atol=1e-12)
+ assert not on_close.any(), (
+ f"{int(on_close.sum())} fill(s) landed on their bar's close, "
+ "which is AtClose behaviour, not a computed level"
+ )
+
+
+def _leaking_strategy():
+ """The clean strategy, with one deliberate leak: the entry reads ahead."""
+ from manifoldbt.indicators import close, low, open as open_px, sma
+
+ h1 = bt.tf("1h")
+ base = sma(h1.close, 8)
+ band_up = base * 1.004
+ band_dn = base * 0.997
+ touch_up = close.lead(5) >= band_up # <- the leak
+ touch_dn = low <= band_dn
+ target = bt.when(touch_dn, 0.0, bt.when(touch_up, -1.0))
+ exec_level = bt.when(
+ touch_dn, bt.when(open_px <= band_dn, open_px, band_dn),
+ bt.when(touch_up, bt.when(open_px >= band_up, open_px, band_up), close),
+ )
+ return (
+ bt.Strategy.create("leaky")
+ .signal("position", target)
+ .signal("exec_level", exec_level)
+ .size(target)
+ .stop_loss(pct=25.0)
+ )
+
+
+def test_the_perturbation_method_catches_a_real_leak(tmp_path):
+ """A look-ahead test that cannot fail proves nothing.
+
+ Same data, same perturbation, same comparison as
+ :func:`test_future_bars_cannot_move_the_past` -- only the strategy reads
+ five bars ahead. The prefix MUST diverge, or the method above is blind and
+ its PASS is worthless.
+ """
+ frame = _mean_reverting_bars()
+ strategy = _leaking_strategy()
+ config = _config(frame)
+
+ reference = _equity(bt.run(strategy, config, _store(frame, tmp_path, "leak_ref")))
+
+ rng = np.random.default_rng(1234)
+ corrupted = frame.copy()
+ tail = slice(SPLIT + 1, None)
+ factor = 1.0 + rng.uniform(-0.005, 0.005, N_BARS - SPLIT - 1)
+ for col in ("open", "high", "low", "close"):
+ corrupted.loc[corrupted.index[tail], col] = corrupted[col].to_numpy()[tail] * factor
+
+ perturbed = _equity(bt.run(strategy, config, _store(corrupted, tmp_path, "leak_pert")))
+
+ n = min(len(reference), len(perturbed), SPLIT + 1)
+ delta = np.abs(reference[:n] - perturbed[:n])
+ assert delta.max() > 0.0, (
+ "a strategy reading 5 bars ahead went undetected: the perturbation "
+ "method is blind and every PASS in this file is meaningless"
+ )
+ # The divergence must sit just before the split, where the lead reaches
+ # into the corrupted tail -- not somewhere unrelated.
+ first = int(np.argmax(delta > 0))
+ assert SPLIT - 60 <= first <= SPLIT, (
+ f"divergence at bar {first}, expected it near the split at {SPLIT}"
+ )
+
+
+def test_detector_splits_on_the_data_not_on_the_configured_range(tmp_path):
+ """A config starting at the epoch must not empty the detector.
+
+ `time_range_start=0` is what `examples/21_fill_at_computed_level.py`
+ writes, and it stretches the period over five decades: both split points
+ used to land before the first bar, so both truncated runs saw no data and
+ the detector announced PASS having compared nothing.
+ """
+ from manifoldbt.diagnostics import detect_lookahead
+
+ frame = _mean_reverting_bars()
+ config = _config(frame)
+ config.time_range_start = 0 # the epoch, as the example does
+
+ result = detect_lookahead(
+ _band_strategy(), config, _store(frame, tmp_path, "epoch"), mode="all"
+ )
+
+ compared = sum(r.total_trades_overlap for r in result.reports)
+ assert compared > 0, (
+ "the detector compared no trade: its splits fell outside the data again"
+ )
+ assert result.passed, f"look-ahead reported: {result}"
diff --git a/python/tests/test_mtf_indicator_period.py b/python/tests/test_mtf_indicator_period.py
new file mode 100644
index 0000000..9c04004
--- /dev/null
+++ b/python/tests/test_mtf_indicator_period.py
@@ -0,0 +1,214 @@
+"""Indicator periods over a higher timeframe count SIMULATION bars.
+
+`bt.tf("1h").close` is the last closed hourly close, forward-filled onto the
+simulation grid. An indicator over it counts rows of that grid, so on 1-minute
+bars `sma(h1.close, 8)` averages 8 *minutes* of a step function — it tracks the
+last closed hourly close instead of averaging 8 hours.
+
+That reading is surprising enough that `tf()`'s own usage example used to show
+`ema(h1.close, 20)` as if 20 meant hours. These tests pin the real semantics
+and the conversion, measured rather than argued: on a ramp of +10 per hour, the
+lag of an average over K hours is (K+1)/2 hours, plus the ~1 h the timeframe
+itself costs (an hourly bar is only readable once closed).
+"""
+import os
+
+import pytest
+
+import manifoldbt as bt
+
+np = pytest.importorskip("numpy")
+pd = pytest.importorskip("pandas")
+
+from manifoldbt.helpers import ExecutionPrice, Interval, Slippage # noqa: E402
+
+HOURS, PER_HOUR = 200, 60
+N_BARS = HOURS * PER_HOUR
+SLOPE = 10.0 # the price gains exactly this much per hour
+
+
+def _ramp_bars():
+ """Hourly closes of 100, 110, 120 … so a lag reads directly as hours."""
+ ts = pd.date_range("2024-01-01", periods=N_BARS, freq="1min", tz="UTC")
+ px = 100.0 + SLOPE * (np.arange(N_BARS) // PER_HOUR)
+ return pd.DataFrame({
+ "timestamp": ts, "open": px, "high": px * 1.5, "low": px * 0.5,
+ "close": px, "volume": np.full(N_BARS, 1000.0),
+ })
+
+
+def _observed_series(period_or_expr, frame, tmp_path, tag):
+ """The values a higher-timeframe band actually took, by timestamp.
+
+ Takes either a period — read as `sma(tf("1h").close, period)`, the
+ staircase form — or a ready-made expression, so the same probe serves both
+ candidates.
+
+ The indicator is read back through `ExecutionPrice.custom`, which fills at
+ the value of a named signal: the trade log then carries the series itself.
+ """
+ from manifoldbt.indicators import close, sma
+ from manifoldbt.expr import Expr
+
+ root = os.path.join(str(tmp_path), tag)
+ store = bt.import_dataframe(
+ frame, symbol="S", symbol_id=1, interval="1m",
+ data_root=os.path.join(root, "data"),
+ metadata_db=os.path.join(root, "meta.sqlite"),
+ )
+ band = (period_or_expr if isinstance(period_or_expr, Expr)
+ else sma(bt.tf("1h").close, period_or_expr))
+ strategy = (
+ bt.Strategy.create("probe")
+ .signal("band", band)
+ .size(bt.when(close > sma(close, 3), 1.0, -1.0))
+ )
+ ts = pd.DatetimeIndex(frame["timestamp"])
+ config = bt.BacktestConfig(
+ universe=[1],
+ time_range_start=int(ts[0].value),
+ time_range_end=int(ts[-1].value) + 60_000_000_000,
+ bar_interval=Interval.minutes(1),
+ initial_capital=1_000_000,
+ execution=bt.ExecutionConfig(
+ signal_delay=0,
+ execution_price=ExecutionPrice.custom("band"),
+ max_position_pct=0.1, allow_short=True,
+ position_sizing_mode="FractionOfEquity",
+ ),
+ slippage=Slippage.fixed_bps(0),
+ warmup_bars=700, # clears the widest window under test
+ extra_timeframes={"1h": Interval.hours(1)},
+ )
+ result = bt.run(strategy, config, store)
+ trades = result.trades_df()
+ assert len(trades) > 50, f"only {len(trades)} trades, too few to measure a lag"
+
+ at = pd.DatetimeIndex(pd.to_datetime(trades["execution_timestamp"], utc=True))
+ return pd.Series(trades["intended_price"].to_numpy(), index=at)
+
+
+def _observed_lag_hours(period, frame, tmp_path, tag):
+ """Average lag of `sma(tf("1h").close, period)`, in hours."""
+ observed = _observed_series(period, frame, tmp_path, tag)
+ ts = pd.DatetimeIndex(frame["timestamp"])
+ price_now = pd.Series(frame["close"].to_numpy(), index=ts).reindex(observed.index)
+ return float(np.median((price_now.to_numpy() - observed.to_numpy()) / SLOPE))
+
+
+def test_a_bare_period_does_not_average_the_higher_timeframe(tmp_path):
+ """`sma(h1.close, 8)` is NOT an 8-hour mean.
+
+ An 8-hour mean would lag about (8+1)/2 + 1 = 5.5 hours. This lags under 2,
+ which is the timeframe's own delay: it is tracking the last closed hourly
+ close, not averaging eight of them.
+ """
+ lag = _observed_lag_hours(8, _ramp_bars(), tmp_path, "bare")
+ assert lag < 2.5, f"lag {lag:.2f} h — this would be a real multi-hour average"
+ assert lag > 0.5, f"lag {lag:.2f} h — the timeframe delay itself is missing"
+
+
+def test_multiplying_the_period_by_the_interval_ratio_matches_the_lag(tmp_path):
+ """`sma(h1.close, K * 60)` carries the LAG of a K-hour mean.
+
+ Expected lag: (K+1)/2 from the average, plus ~1 h for the timeframe. This
+ is the whole of what a ramp can establish, and it is not enough to call the
+ result a K-hour mean: see `test_the_interval_ratio_is_not_the_hourly_mean`,
+ which uses an impulse to show the two series apart.
+ """
+ frame = _ramp_bars()
+ for hours, expected in ((4, 1 + (4 + 1) / 2), (8, 1 + (8 + 1) / 2)):
+ lag = _observed_lag_hours(hours * PER_HOUR, frame, tmp_path, f"k{hours}")
+ assert abs(lag - expected) < 0.5, (
+ f"{hours}-hour mean lags {lag:.2f} h, expected about {expected:.2f} h"
+ )
+
+
+def _impulse_bars():
+ """Hourly closes flat at 100 but for ONE hour at 200.
+
+ A ramp cannot separate the two candidates: a box filter leaves a straight
+ line straight, so a biased weighting still reports the right lag. An
+ impulse makes each hour's weight readable in the value itself.
+
+ The last minute of every hour keeps the base value, so the hourly closes --
+ the only rows `tf("1h")` reads -- stay exactly 100 or 200. The intra-hour
+ zigzag exists only to make the probe strategy trade on every bar.
+ """
+ n = 60 * PER_HOUR
+ ts = pd.date_range("2024-01-01", periods=n, freq="1min", tz="UTC")
+ base = np.where(np.arange(n) // PER_HOUR == 30, 200.0, 100.0)
+ minute = np.arange(n) % PER_HOUR
+ px = base + np.where(minute == PER_HOUR - 1, 0.0, np.where(minute % 2 == 0, 0.5, -0.5))
+ return pd.DataFrame({
+ "timestamp": ts, "open": px, "high": px * 1.5, "low": px * 0.5,
+ "close": px, "volume": np.full(n, 1000.0),
+ })
+
+
+def test_the_interval_ratio_is_not_the_hourly_mean(tmp_path):
+ """`sma(h1.close, 8 * 60)` is not an equal-weight mean of 8 hourly closes.
+
+ It averages 480 rows of a step function, so a move ramps in over 60 minutes
+ instead of stepping, and the window spans 9 hourly values with unequal
+ weights rather than 8 with equal ones. On the impulse the true signal spans
+ 12.5 (100 to 112.5) and the gap reaches nearly all of it.
+ """
+ frame = _impulse_bars()
+ observed = _observed_series(8 * PER_HOUR, frame, tmp_path, "impulse")
+
+ ts = pd.DatetimeIndex(frame["timestamp"])
+ hourly = pd.Series(frame["close"].to_numpy(), index=ts).resample("1h").last()
+ assert set(np.round(hourly.dropna().unique(), 6)) <= {100.0, 200.0}
+ truth = hourly.rolling(8).mean().shift(1).reindex(ts, method="ffill")
+
+ gap = (observed - truth.reindex(observed.index)).dropna().abs()
+ assert gap.max() > 10.0, (
+ f"largest gap to a true 8-hour mean is {gap.max():.2f} on a signal "
+ "spanning 12.5; the two would then be the same series"
+ )
+
+
+def test_apply_is_the_hourly_mean_and_reads_only_closed_bars(tmp_path):
+ """`h1.apply(sma(close, 8))` IS the mean of 8 hourly closes — exactly.
+
+ Same impulse that separates the two staircase forms. Two alignments are
+ checked against, and they disagree on 120 bars, so matching one excludes
+ the other: the shifted reference reads only CLOSED hours, the unshifted one
+ would need the hour in progress. Landing on the shifted one is what rules
+ out look-ahead.
+ """
+ from manifoldbt.indicators import close, sma
+
+ frame = _impulse_bars()
+ band = bt.tf("1h").apply(sma(close, 8))
+ observed = _observed_series(band, frame, tmp_path, "apply")
+
+ ts = pd.DatetimeIndex(frame["timestamp"])
+ hourly = pd.Series(frame["close"].to_numpy(), index=ts).resample("1h").last()
+ rolled = hourly.rolling(8).mean()
+ safe = rolled.shift(1).reindex(ts, method="ffill") # closed hours only
+ leaking = rolled.reindex(ts, method="ffill") # the hour in progress
+
+ disagree = (safe - leaking).dropna().abs()
+ assert (disagree > 1e-9).sum() > 50, "the two alignments must differ to discriminate"
+
+ gap_safe = (observed - safe.reindex(observed.index)).dropna().abs()
+ gap_leak = (observed - leaking.reindex(observed.index)).dropna().abs()
+ assert gap_safe.max() < 1e-9, (
+ f"apply() is off the true 8-hour mean by {gap_safe.max():.4f}"
+ )
+ assert gap_leak.max() > 1.0, (
+ "apply() matches the alignment that reads the hour in progress"
+ )
+
+
+def test_a_longer_period_lags_more(tmp_path):
+ """The ordering alone would catch a period silently ignored."""
+ frame = _ramp_bars()
+ short = _observed_lag_hours(4 * PER_HOUR, frame, tmp_path, "ord4")
+ long = _observed_lag_hours(8 * PER_HOUR, frame, tmp_path, "ord8")
+ assert long > short + 1.0, (
+ f"8-hour mean lags {long:.2f} h vs {short:.2f} h for 4 hours; "
+ "the period is not doing what it should"
+ )
diff --git a/python/tests/test_on_timeframe.py b/python/tests/test_on_timeframe.py
new file mode 100644
index 0000000..924d756
--- /dev/null
+++ b/python/tests/test_on_timeframe.py
@@ -0,0 +1,205 @@
+"""Tests for tf(..).apply(..) — indicators evaluated ON the higher timeframe.
+
+The defect this feature fixes, pinned by `test_apply_differs_from_staircase`:
+an indicator over a step-held `tf()` column counts its period in SIMULATION
+bars, so `sma(tf("1h").close, 20)` on a 1m simulation is a 20-MINUTE smoothing
+of an hourly staircase — mid-hour it equals the previous hourly close exactly.
+`tf("1h").apply(sma(close, 20))` is the true 20-HOUR average.
+
+The reference implementation (`_hand_band`) is the exo-column recipe users had
+to build by hand before this feature: resample to 1h in pandas, indicator on
+the 1h grid, shift(1) (a closed bar is readable from the next bar on), ffill
+onto the 1m grid. `test_apply_matches_hand_rolled_exo` demands bit-identical
+metrics against it.
+"""
+import os
+
+import pytest
+
+import manifoldbt as bt
+from manifoldbt.indicators import close, sma
+
+pd = pytest.importorskip("pandas")
+np = pytest.importorskip("numpy")
+
+N_BARS = 20_000 # ~13.9 days of 1m bars -> ~333 hourly bars
+PERIOD = 20
+
+
+def _bars_df():
+ ts = pd.date_range("2022-01-01", periods=N_BARS, freq="1min", tz="UTC")
+ rng = np.random.default_rng(3)
+ px = 100.0 + np.cumsum(np.sin(np.arange(N_BARS) / 700.0) * 0.5 + rng.normal(0, 0.4, N_BARS)) * 0.01
+ px = np.maximum(px, 1.0)
+ return pd.DataFrame(
+ {
+ "timestamp": ts,
+ "open": px,
+ "high": px * 1.0005,
+ "low": px * 0.9995,
+ "close": px,
+ "volume": [1000.0] * N_BARS,
+ }
+ )
+
+
+def _store(tmp_path, df):
+ root = tmp_path / "otf_store"
+ return bt.import_dataframe(
+ df,
+ symbol="ZT",
+ symbol_id=1,
+ interval="1m",
+ asset_class="equity",
+ exchange="TEST",
+ data_root=str(root / "data"),
+ metadata_db=str(root / "metadata.sqlite"),
+ ), str(root / "data")
+
+
+def _hand_band(df, period):
+ """The pre-feature recipe: hourly SMA built by hand, no lookahead."""
+ h1 = df.set_index("timestamp")["close"].resample("1h").last()
+ band = h1.rolling(period).mean().shift(1)
+ return band.reindex(df["timestamp"], method="ffill")
+
+
+def _config(tmp_path=None, exo=False):
+ start, end = bt.time_range("2022-01-01", "2022-01-14")
+ cfg = bt.BacktestConfig(
+ universe=[1],
+ time_range_start=start,
+ time_range_end=end,
+ initial_capital=10_000.0,
+ provider="TEST",
+ bar_interval=bt.Interval.minutes(1),
+ symbol_names={"ZT": 1},
+ extra_timeframes={} if exo else {"1h": bt.Interval.hours(1)},
+ exo_data=["hand_band"] if exo else [],
+ )
+ cfg.warmup_bars = 0
+ return cfg
+
+
+def _strategy(band_expr, name):
+ return (
+ bt.Strategy.create(name)
+ .signal("band", band_expr)
+ .size(bt.when(close > bt.col("band"), 1.0, 0.0))
+ )
+
+
+def test_serializes_as_on_timeframe():
+ e = bt.tf("1h").apply(sma(close, PERIOD))
+ payload = e.to_json()
+ assert list(payload) == ["OnTimeframe"]
+ label, inner = payload["OnTimeframe"]
+ assert label == "1h"
+ assert list(inner) == ["RollingMean"]
+
+
+def test_apply_matches_hand_rolled_exo(tmp_path):
+ """The money test: tf("1h").apply(sma(close, 20)) must be bit-identical to
+ the hand-precomputed hourly-SMA exo column it replaces."""
+ df = _bars_df()
+ store, data_root = _store(tmp_path, df)
+
+ band = _hand_band(df, PERIOD)
+ bt.register_exo(
+ "hand_band",
+ pd.DataFrame({"timestamp": df["timestamp"], "value": band.values}),
+ data_root=data_root,
+ )
+
+ native = bt.run(
+ _strategy(bt.tf("1h").apply(sma(close, PERIOD)), "native"),
+ _config(),
+ store,
+ )
+ hand = bt.run(
+ _strategy(bt.exo("hand_band", "value"), "hand"),
+ _config(exo=True),
+ store,
+ )
+
+ for key in ("total_return", "sharpe", "max_drawdown", "volatility"):
+ assert native.metrics.get(key) == hand.metrics.get(key), (
+ f"{key}: native {native.metrics.get(key)} != hand {hand.metrics.get(key)}"
+ )
+ assert len(native.trades_df()) == len(hand.trades_df())
+
+
+def test_apply_differs_from_staircase(tmp_path):
+ """Guard against regressing to the old semantics: the staircase version
+ (indicator over the step-held tf() column) must NOT equal apply()."""
+ df = _bars_df()
+ store, _ = _store(tmp_path, df)
+
+ applied = bt.run(
+ _strategy(bt.tf("1h").apply(sma(close, PERIOD)), "applied"),
+ _config(),
+ store,
+ )
+ staircase = bt.run(
+ _strategy(sma(bt.tf("1h").close, PERIOD), "staircase"),
+ _config(),
+ store,
+ )
+ assert applied.metrics.get("total_return") != staircase.metrics.get("total_return"), (
+ "apply() and the staircase smoothing agreed; the coarse-grid evaluation "
+ "is not actually happening"
+ )
+
+
+def test_swept_period_matches_fixed_runs(tmp_path):
+ """param() INSIDE apply(): each combo must equal the fixed-period run."""
+ df = _bars_df()
+ store, _ = _store(tmp_path, df)
+ periods = [10, 20, 40]
+
+ sweep = bt.run_sweep_lite(
+ _strategy(bt.tf("1h").apply(sma(close, bt.param("len"))), "swept"),
+ {"len": periods},
+ _config(),
+ store,
+ device="cpu",
+ )
+ assert len(sweep) == len(periods)
+
+ for got, period in zip(sweep, periods):
+ ref = bt.run_sweep_lite(
+ _strategy(bt.tf("1h").apply(sma(close, period)), f"fixed_{period}"),
+ {},
+ _config(),
+ store,
+ device="cpu",
+ )[0]
+ for key in ("total_return", "sharpe", "max_drawdown"):
+ assert got.metrics.get(key) == ref.metrics.get(key), (
+ f"len={period}: {key} diverged"
+ )
+
+
+def test_lite_matches_run(tmp_path):
+ df = _bars_df()
+ store, _ = _store(tmp_path, df)
+ strat = _strategy(bt.tf("1h").apply(sma(close, PERIOD)), "parity")
+
+ full = bt.run(strat, _config(), store)
+ lite = bt.run_sweep_lite(strat, {}, _config(), store, device="cpu")[0]
+ for key in ("total_return", "sharpe"):
+ assert full.metrics.get(key) == lite.metrics.get(key), f"{key} diverged"
+ # max_drawdown carries a pre-existing ~1e-16 run-vs-lite float-noise gap
+ # (measured on a plain sma(close, 20) strategy with no OnTimeframe on this
+ # same data), so exact equality would pin the wrong thing here.
+ a, b = full.metrics["max_drawdown"], lite.metrics["max_drawdown"]
+ assert a == pytest.approx(b, rel=1e-12), f"max_drawdown diverged: {a} vs {b}"
+
+
+def test_missing_extra_timeframe_is_a_clear_error(tmp_path):
+ df = _bars_df()
+ store, _ = _store(tmp_path, df)
+ cfg = _config()
+ cfg.extra_timeframes = {}
+ with pytest.raises(Exception, match="extra_timeframes"):
+ bt.run(_strategy(bt.tf("1h").apply(sma(close, PERIOD)), "no_tf"), cfg, store)
diff --git a/python/tests/test_options.py b/python/tests/test_options.py
new file mode 100644
index 0000000..7713bea
--- /dev/null
+++ b/python/tests/test_options.py
@@ -0,0 +1,365 @@
+"""Tests for the option path from Python: contract terms in, settlement out.
+
+The Rust side already proves the payoff arithmetic and the simulation loop.
+What is under test here is the bridge: terms recorded at ingest must reach the
+engine, and the one thing the user has to decide (which price series settles the
+contract) must fail loudly when it is missing rather than be guessed.
+"""
+import os
+
+import pytest
+
+import manifoldbt as bt
+
+pd = pytest.importorskip("pandas")
+
+OPTION_ID = 2
+UNDERLYING_ID = 1
+STRIKE = 50_000.0
+N_BARS = 40
+# Expiry lands on bar 30 of a 40-bar daily series starting 2020-01-01.
+EXPIRY_MS = 1_577_836_800_000 + 30 * 86_400_000
+
+
+def _daily(prices):
+ ts = pd.date_range("2020-01-01", periods=len(prices), freq="1D", tz="UTC")
+ return pd.DataFrame(
+ {
+ "timestamp": ts,
+ "open": prices,
+ "high": prices,
+ "low": prices,
+ "close": prices,
+ "volume": [100.0] * len(prices),
+ }
+ )
+
+
+def _store(tmp_path, underlying_price, premium, option_class="option"):
+ """A two-symbol store: a perpetual and a call written against it.
+
+ ``option_class`` exists so a test can write the same series as a plain
+ linear instrument, which is a different thing from an option missing its
+ terms.
+ """
+ root = os.path.join(str(tmp_path), "data")
+ meta = os.path.join(str(tmp_path), "m.sqlite")
+ store = bt.import_dataframe(
+ _daily([underlying_price] * N_BARS),
+ symbol="BTC-PERPETUAL",
+ symbol_id=UNDERLYING_ID,
+ interval="1d",
+ data_root=root,
+ metadata_db=meta,
+ asset_class="crypto_perp",
+ )
+ store = bt.import_dataframe(
+ _daily([premium] * N_BARS),
+ symbol="BTC-CALL",
+ symbol_id=OPTION_ID,
+ interval="1d",
+ data_root=root,
+ metadata_db=meta,
+ asset_class=option_class,
+ )
+ return store, root, meta
+
+
+def _write_terms(meta_db, settlement="cash_inverse"):
+ """Record contract terms the way an option connector would."""
+ import sqlite3
+
+ conn = sqlite3.connect(meta_db)
+ conn.execute(
+ "UPDATE symbols SET option_underlying = ?, option_type = ?, option_strike = ?,"
+ " option_expiry = ?, option_contract_size = ?, option_settlement = ?"
+ " WHERE id = ?",
+ (
+ "BTC_USD index",
+ "call",
+ STRIKE,
+ pd.Timestamp(EXPIRY_MS, unit="ms", tz="UTC").isoformat().replace("+00:00", "Z"),
+ 1.0,
+ settlement,
+ OPTION_ID,
+ ),
+ )
+ conn.commit()
+ conn.close()
+
+
+def _config(**kwargs):
+ from manifoldbt.helpers import time_range, Interval
+
+ start, end = time_range("2020-01-01", "2020-03-01")
+ base = dict(
+ universe=[UNDERLYING_ID, OPTION_ID],
+ time_range_start=start,
+ time_range_end=end,
+ bar_interval=Interval.days(1),
+ initial_capital=10.0,
+ currency="BTC",
+ execution=bt.ExecutionConfig(position_sizing_mode="Units"),
+ )
+ base.update(kwargs)
+ return bt.BacktestConfig(**base)
+
+
+def _hold(**per_symbol):
+ """Hold a fixed number of units of each named symbol id, every bar.
+
+ Legs are told apart by `col("symbol_id")` rather than by price level. A
+ price threshold is a trap: a premium crossing it flips the leg to zero and
+ the strategy closes its own position, which is exactly how an earlier
+ version of this file broke.
+ """
+ from manifoldbt.indicators import col
+
+ size = bt.when(col("symbol_id") < 0.0, 0.0, 0.0) # a typed zero to fold onto
+ for symbol_id, units in per_symbol.items():
+ size = size + bt.when(col("symbol_id") == float(symbol_id), float(units), 0.0)
+ return (
+ bt.Strategy.create("hold")
+ .signal("leg", col("symbol_id"))
+ .size(size)
+ .describe("Fixed units per leg, held into expiry")
+ )
+
+
+def _long_one_option():
+ return _hold(**{str(OPTION_ID): 1.0})
+
+
+def test_contract_terms_round_trip_to_python(tmp_path):
+ _, _, meta = _store(tmp_path, 60_000.0, 0.05)
+ _write_terms(meta)
+ store = bt.DataStore(os.path.join(str(tmp_path), "data"), meta)
+
+ terms = store.option_contracts()
+ assert OPTION_ID in terms
+ assert terms[OPTION_ID]["option_type"] == "call"
+ assert terms[OPTION_ID]["strike"] == STRIKE
+ assert terms[OPTION_ID]["settlement"] == "cash_inverse"
+ assert UNDERLYING_ID not in terms, "a perpetual has no contract terms"
+
+
+def test_an_option_without_a_declared_underlying_is_refused(tmp_path):
+ store, _, meta = _store(tmp_path, 60_000.0, 0.05)
+ _write_terms(meta)
+
+ # The public API re-classifies the failure, so catch what a user catches.
+ from manifoldbt.exceptions import DataError
+
+ with pytest.raises(DataError) as excinfo:
+ bt.run(_long_one_option(), _config(), store)
+
+ message = str(excinfo.value)
+ assert "option_underlyings" in message
+ assert "own last traded premium" in message
+
+
+def test_a_call_expiring_in_the_money_settles_at_intrinsic(tmp_path):
+ # S = 60k against a 50k strike, inverse settlement: 10000/60000 BTC.
+ store, _, meta = _store(tmp_path, 60_000.0, 0.05)
+ _write_terms(meta)
+
+ result = bt.run(
+ _long_one_option(),
+ _config(option_underlyings={OPTION_ID: UNDERLYING_ID}),
+ store,
+ )
+
+ trades = result.trades.to_pandas()
+ settlements = trades[(trades.symbol_id == OPTION_ID) & (trades.exit_reason == 5)]
+ assert len(settlements) == 1, f"expected one settlement, got:\n{trades}"
+ assert settlements.iloc[0].fill_price == pytest.approx(10_000.0 / 60_000.0, abs=1e-12)
+
+
+def test_a_call_expiring_out_of_the_money_settles_at_zero(tmp_path):
+ store, _, meta = _store(tmp_path, 40_000.0, 0.05)
+ _write_terms(meta)
+
+ result = bt.run(
+ _long_one_option(),
+ _config(option_underlyings={OPTION_ID: UNDERLYING_ID}),
+ store,
+ )
+
+ trades = result.trades.to_pandas()
+ settlements = trades[(trades.symbol_id == OPTION_ID) & (trades.exit_reason == 5)]
+ assert len(settlements) == 1
+ assert settlements.iloc[0].fill_price == 0.0
+ # The premium paid is the whole loss, and it is a loss.
+ assert float(result.equity_curve[-1]) < float(result.equity_curve[0])
+
+
+def test_a_linear_universe_is_untouched_by_the_option_path(tmp_path):
+ # Two ordinary linear instruments: the option path must not touch them.
+ store, _, _ = _store(tmp_path, 40_000.0, 0.05, option_class="crypto_spot")
+
+ result = bt.run(_long_one_option(), _config(), store)
+
+ trades = result.trades.to_pandas()
+ assert (trades.exit_reason != 5).all(), "nothing may settle without contract terms"
+
+
+def test_an_option_symbol_without_contract_terms_is_refused(tmp_path):
+ """The Databento case before this branch: an option that never expires.
+
+ A symbol recorded as an option but carrying no strike or expiration would
+ otherwise price, trade and be held forever at its last quoted premium, with
+ nothing in the output looking wrong.
+ """
+ from manifoldbt.exceptions import DataError
+
+ store, _, _ = _store(tmp_path, 60_000.0, 0.05) # asset_class="option", no terms
+
+ with pytest.raises(DataError) as excinfo:
+ bt.run(_long_one_option(), _config(), store)
+
+ message = str(excinfo.value)
+ assert "no contract terms" in message
+ assert "deribit, databento" in message
+
+
+def test_a_multiplier_option_costs_and_settles_like_one_contract(tmp_path):
+ """A listed-style option: 100 units of premium IS one exchange contract."""
+ # Premium 4.70, underlying 490, strike 470 -> one contract pays (490-470)*100.
+ store, _, meta = _store(tmp_path, 490.0, 4.70)
+ import sqlite3
+
+ conn = sqlite3.connect(meta)
+ conn.execute(
+ "UPDATE symbols SET option_underlying = ?, option_type = ?, option_strike = ?,"
+ " option_expiry = ?, option_contract_size = ?, option_settlement = ? WHERE id = ?",
+ (
+ "SPY",
+ "call",
+ 470.0,
+ pd.Timestamp(EXPIRY_MS, unit="ms", tz="UTC").isoformat().replace("+00:00", "Z"),
+ 100.0,
+ "cash_linear",
+ OPTION_ID,
+ ),
+ )
+ conn.commit()
+ conn.close()
+
+ # 100 units of the option leg, nothing on the underlying.
+ hold_one_contract = _hold(**{str(OPTION_ID): 100.0})
+ config = _config(
+ initial_capital=100_000.0,
+ option_underlyings={OPTION_ID: UNDERLYING_ID},
+ )
+
+ result = bt.run(hold_one_contract, config, store)
+ trades = result.trades.to_pandas()
+ legs = trades[trades.symbol_id == OPTION_ID]
+
+ entry = legs[legs.exit_reason == 0].iloc[0]
+ assert entry.quantity * entry.fill_price == pytest.approx(470.0), "what one contract costs"
+
+ settlement = legs[legs.exit_reason == 5].iloc[0]
+ assert settlement.fill_price == pytest.approx(20.0), "intrinsic per share, not per contract"
+ assert settlement.quantity * settlement.fill_price == pytest.approx(
+ 2_000.0
+ ), "what one contract pays"
+
+
+PUT_ID = 3
+
+
+def _store_two_legs(tmp_path, underlying_price, call_premium, put_premium):
+ """Underlying + a call + a put, all daily, all the same length."""
+ root = os.path.join(str(tmp_path), "data")
+ meta = os.path.join(str(tmp_path), "m.sqlite")
+ for symbol, symbol_id, price, klass in (
+ ("BTC-PERPETUAL", UNDERLYING_ID, underlying_price, "crypto_perp"),
+ ("BTC-CALL", OPTION_ID, call_premium, "option"),
+ ("BTC-PUT", PUT_ID, put_premium, "option"),
+ ):
+ store = bt.import_dataframe(
+ _daily([price] * N_BARS),
+ symbol=symbol,
+ symbol_id=symbol_id,
+ interval="1d",
+ data_root=root,
+ metadata_db=meta,
+ asset_class=klass,
+ )
+ return store, meta
+
+
+def _write_leg_terms(meta_db, symbol_id, option_type, strike):
+ import sqlite3
+
+ conn = sqlite3.connect(meta_db)
+ conn.execute(
+ "UPDATE symbols SET option_underlying = ?, option_type = ?, option_strike = ?,"
+ " option_expiry = ?, option_contract_size = ?, option_settlement = ? WHERE id = ?",
+ (
+ "BTC_USD index",
+ option_type,
+ strike,
+ pd.Timestamp(EXPIRY_MS, unit="ms", tz="UTC").isoformat().replace("+00:00", "Z"),
+ 1.0,
+ "cash_inverse",
+ symbol_id,
+ ),
+ )
+ conn.commit()
+ conn.close()
+
+
+def test_a_two_leg_structure_settles_each_leg_on_its_own_terms(tmp_path):
+ """A risk reversal: long a call, short a put, both expiring together.
+
+ Each leg settles against the same underlying but on its own strike and
+ side, so one finishes in the money and the other worthless.
+ """
+ # S = 60k at expiry: the 50k call is ITM, the 40k put is worthless.
+ store, meta = _store_two_legs(tmp_path, 60_000.0, 0.05, 0.03)
+ _write_leg_terms(meta, OPTION_ID, "call", 50_000.0)
+ _write_leg_terms(meta, PUT_ID, "put", 40_000.0)
+
+ config = _config(
+ universe=[UNDERLYING_ID, OPTION_ID, PUT_ID],
+ option_underlyings={OPTION_ID: UNDERLYING_ID, PUT_ID: UNDERLYING_ID},
+ option_margin_model="deribit",
+ execution=bt.ExecutionConfig(position_sizing_mode="Units", allow_short=True),
+ )
+ result = bt.run(
+ _hold(**{str(OPTION_ID): 1.0, str(PUT_ID): -1.0}),
+ config,
+ store,
+ )
+
+ trades = result.trades.to_pandas()
+ settlements = trades[trades.exit_reason == 5]
+ assert set(settlements.symbol_id) == {OPTION_ID, PUT_ID}, "both legs must settle"
+
+ call = settlements[settlements.symbol_id == OPTION_ID].iloc[0]
+ put = settlements[settlements.symbol_id == PUT_ID].iloc[0]
+ assert call.fill_price == pytest.approx(10_000.0 / 60_000.0, abs=1e-12)
+ assert put.fill_price == 0.0, "a 40k put is worthless with the underlying at 60k"
+
+ # Long the call, short the put: the short is bought back to close.
+ assert call.side == 2 and put.side == 1
+
+
+def test_per_leg_sizing_leaves_the_other_leg_flat(tmp_path):
+ """`col("symbol_id")` must target one leg without disturbing the others."""
+ store, meta = _store_two_legs(tmp_path, 60_000.0, 0.05, 0.03)
+ _write_leg_terms(meta, OPTION_ID, "call", 50_000.0)
+ _write_leg_terms(meta, PUT_ID, "put", 40_000.0)
+
+ config = _config(
+ universe=[UNDERLYING_ID, OPTION_ID, PUT_ID],
+ option_underlyings={OPTION_ID: UNDERLYING_ID, PUT_ID: UNDERLYING_ID},
+ )
+ result = bt.run(_hold(**{str(OPTION_ID): 1.0}), config, store)
+
+ trades = result.trades.to_pandas()
+ assert (trades.symbol_id == OPTION_ID).all(), (
+ f"only the call leg may trade, got:\n{trades}"
+ )
diff --git a/python/tests/test_plot_charts.py b/python/tests/test_plot_charts.py
index 92d84fb..e4beece 100644
--- a/python/tests/test_plot_charts.py
+++ b/python/tests/test_plot_charts.py
@@ -94,3 +94,58 @@ def test_returns_histogram_has_no_unnamed_legend_entry(monkeypatch):
assert not any(
(name or "").startswith("trace ") for name in legend_names
), f"auto-generated trace label in the legend: {legend_names}"
+
+
+# ── Adaptive axis / hover formats ────────────────────────────────────────────
+# These formats used to be hardcoded, and the failures were invisible in any
+# assertion on figure structure: a two-month backtest labelled every date tick
+# "May 2025", a -0.9% drawdown labelled every value tick "0%", and a 10-BTC
+# equity hovered as "$10". Pure functions now decide them, so the contract is
+# testable without rendering.
+
+
+def _dates(days):
+ np = pytest.importorskip("numpy")
+ return np.arange("2025-01-01", np.timedelta64(days, "D") + np.datetime64("2025-01-01"),
+ dtype="datetime64[D]").astype("datetime64[ns]")
+
+
+def test_date_tickformat_follows_the_span():
+ from manifoldbt.plot._convert import date_tickformat
+
+ assert date_tickformat(_dates(2)) == "%d %b %H:%M"
+ assert date_tickformat(_dates(61)) == "%d %b", "a two-month window must show days"
+ assert date_tickformat(_dates(365 * 4)) == "%b %Y", "long spans keep the historical format"
+
+
+def test_money_hovertemplate_is_currency_and_magnitude_aware():
+ np = pytest.importorskip("numpy")
+ from manifoldbt.plot._convert import money_hovertemplate
+
+ btc = money_hovertemplate(np.array([10.0, 10.04]), "BTC")
+ assert "BTC" in btc and "%{y:,.4f}" in btc, btc
+ assert "$" not in btc, "a BTC equity must not hover in dollars"
+
+ usd = money_hovertemplate(np.array([10_000.0, 21_313.0]), "USD")
+ assert "$%{y:,.0f}" in usd, usd
+
+ eur = money_hovertemplate(np.array([500.0]), "EUR")
+ assert "\u20ac" in eur and "%{y:,.2f}" in eur, eur
+
+
+def test_percent_tickformat_keeps_small_drawdowns_legible():
+ from manifoldbt.plot._convert import percent_tickformat
+
+ assert percent_tickformat(-0.35) == ".0%"
+ assert percent_tickformat(-0.02) == ".1%"
+ assert percent_tickformat(-0.0037) == ".2%", "a -0.37% max dd must not read as 0%"
+
+
+def test_run_currency_reads_the_manifest_and_survives_its_absence():
+ from manifoldbt.plot._convert import run_currency
+
+ class WithManifest:
+ manifest = {"config": {"currency": "BTC"}}
+
+ assert run_currency(WithManifest()) == "BTC"
+ assert run_currency(object()) == "USD", "no manifest must fall back, not raise"