tickreplay 0.1.0: tick-accurate MT5 backtesting engine + research toolkit
Memory-mapped tickstores, a nogil numba replay kernel validated trade-for-trade against an independent reference implementation, purged walk-forward CV, an append-only FDR registry, tick-exact labels, and online changepoint detection. 129 tests, 4 runnable examples on synthetic data, CI for linux/macos.
This commit is contained in:
danielmjrr
@@ -0,0 +1,21 @@
|
||||
name: ci
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest]
|
||||
python-version: ["3.11", "3.13"]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: pip
|
||||
cache-dependency-path: pyproject.toml
|
||||
- run: pip install -e ".[dev]"
|
||||
- run: pytest -q
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
.eggs/
|
||||
build/
|
||||
dist/
|
||||
.venv/
|
||||
venv/
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
/data/
|
||||
/*.npy
|
||||
.DS_Store
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 danielmjrr
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,140 @@
|
||||
# tickreplay
|
||||
|
||||
[](https://github.com/danielmjrr/tickreplay/actions/workflows/ci.yml)
|
||||
|
||||
Tick-accurate backtesting for MetaTrader 5 data, built around two ideas:
|
||||
|
||||
1. **Bar backtests lie.** Inside one OHLC bar you cannot know whether the high or the
|
||||
low printed first, so any bar-level simulation of a stop loss and take profit is
|
||||
guessing about path. tickreplay replays every tick, so barrier ordering, spread
|
||||
costs, and fill prices are exact.
|
||||
2. **Most backtest profits are statistical artifacts.** The library ships the
|
||||
machinery to catch that: purged walk-forward splits, overlap-corrected
|
||||
significance tests, and an append-only evaluation registry with cumulative
|
||||
false-discovery control.
|
||||
|
||||
It is fast enough that neither idea costs you iteration speed: the replay kernel
|
||||
streams a year of ticks in well under a second on a laptop.
|
||||
|
||||
## Install
|
||||
|
||||
```
|
||||
pip install git+https://github.com/danielmjrr/tickreplay
|
||||
```
|
||||
|
||||
Python ≥ 3.11. Dependencies: numpy, pandas, numba, pyarrow, scipy.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
import tickreplay as tr
|
||||
|
||||
# synthetic data so this runs without any broker export
|
||||
time_ns, bid, ask = tr.synth.generate_ticks(days=30, seed=7)
|
||||
store = tr.TickStore(time_ns, bid, ask)
|
||||
bars = store.bars("M5")
|
||||
|
||||
# a signal is one int per bar: +1 buy, -1 sell, 0 nothing
|
||||
fast = bars["close"].rolling(10).mean()
|
||||
slow = bars["close"].rolling(50).mean()
|
||||
above = (fast > slow).astype("int8")
|
||||
signal = np.where(above.diff() > 0, 1, np.where(above.diff() < 0, -1, 0))
|
||||
|
||||
result = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=3.0, tp=6.0, max_hold="4h", cost=0.05))
|
||||
print(result.summary())
|
||||
print(result.trades.head())
|
||||
```
|
||||
|
||||
Entries happen at the first tick after the signal bar closes (buys at ask, sells at
|
||||
bid); exits are checked tick-by-tick in the order stop → target → time → opposite
|
||||
signal. One position at a time — the result is the *tradable* expectancy of the
|
||||
strategy, not an overlapped ideal. Full semantics are documented in
|
||||
`tickreplay/engine.py`.
|
||||
|
||||
## Using your own data
|
||||
|
||||
Export ticks from MT5 (View → Symbols → select symbol → Ticks tab → request a range →
|
||||
Export). That produces a tab-separated CSV. Then:
|
||||
|
||||
```python
|
||||
import tickreplay as tr
|
||||
|
||||
tr.ingest.ticks_to_monthly_parquet("XAUUSD_ticks.csv", "data/xauusd/parquet")
|
||||
store = tr.TickStore.build("data/xauusd/parquet", "data/xauusd/store")
|
||||
# later sessions:
|
||||
store = tr.TickStore.open("data/xauusd/store")
|
||||
```
|
||||
|
||||
The store is a set of memory-mapped flat arrays: opening it is instant, the OS page
|
||||
cache shares it across processes, and a years-long tick history costs no RAM until
|
||||
touched. `tr.ingest.validate_against_broker_bars` cross-checks the tick export
|
||||
against the broker's own M1 bars — recommended before trusting any new export
|
||||
(mismatches usually mean the tick and bar files came from different servers).
|
||||
|
||||
## Why it's fast
|
||||
|
||||
- **Storage**: ticks live in memory-mapped `int64/float64` arrays; bars carry their
|
||||
tick index spans, so mapping a bar to its ticks is O(1) and derived data never
|
||||
duplicates the tick stream.
|
||||
- **Kernel**: the replay loop is a single numba-JIT function, `nogil` and
|
||||
branch-light, that streams the arrays once. It is memory-bandwidth-bound — on an
|
||||
Apple M4 it processes roughly a billion ticks per second.
|
||||
- **Sweeps**: because the kernel releases the GIL and the data is shared, parameter
|
||||
grids fan out over threads with zero copies (`tr.sweep_exits`). A 100-config grid
|
||||
over years of ticks takes seconds.
|
||||
|
||||
Run `python examples/benchmark.py` for numbers on your machine.
|
||||
|
||||
## Why you can trust it
|
||||
|
||||
- **Two independent engines.** The numba kernel is tested trade-for-trade,
|
||||
byte-for-byte against a deliberately naive pure-Python reference implementation
|
||||
(`tests/reference.py`) across randomized fixtures and config combinations. Any
|
||||
semantic drift between the two fails CI.
|
||||
- **Look-ahead is tested, not assumed.** Truncation tests verify that removing
|
||||
future data leaves all earlier trades byte-identical; bar spans make the
|
||||
no-look-ahead boundary explicit (a bar exists only after its last tick).
|
||||
- **The research module assumes you will fool yourself** and gets in the way:
|
||||
- `research.validation.purged_expanding_folds` — walk-forward splits that purge
|
||||
training samples whose label window leaks into the test period (plus embargo);
|
||||
- `research.stats.thin_by_gap` — overlapping trade windows inflate t-statistics;
|
||||
test a non-overlapping subset instead;
|
||||
- `research.registry` — an append-only log of *every* evaluation you ever ran,
|
||||
with Benjamini-Hochberg q-values recomputed over the full history on every
|
||||
read. Run a thousand configs and the bar for "significant" rises accordingly;
|
||||
- `research.labels` — tick-exact excursion and first-passage labels for
|
||||
measurement (where bar data cannot even order target-vs-stop correctly);
|
||||
- `research.bocpd` — online Bayesian changepoint detection (Adams & MacKay),
|
||||
strictly filtered, with causality enforced by test.
|
||||
|
||||
`examples/sweep_and_validate.py` shows the intended workflow end to end — including
|
||||
what an honest negative result looks like.
|
||||
|
||||
## Examples
|
||||
|
||||
All examples run on synthetic data in about a second, no broker export needed:
|
||||
|
||||
- [`examples/ma_cross.py`](examples/ma_cross.py) — moving-average cross, fixed stops
|
||||
- [`examples/breakout.py`](examples/breakout.py) — N-bar breakout with
|
||||
opposite-signal exits and an exit-reason breakdown
|
||||
- [`examples/sweep_and_validate.py`](examples/sweep_and_validate.py) — the intended
|
||||
research loop: sweep a grid, then test the winner with thinning and a
|
||||
selection-adjusted threshold
|
||||
- [`examples/benchmark.py`](examples/benchmark.py) — throughput measurement on your
|
||||
hardware (on an M4 MacBook Air: ~1,100M ticks/s scanned inside positions, 20M-tick
|
||||
store replayed in under 4 ms)
|
||||
|
||||
## Scope and limitations
|
||||
|
||||
- Single position, netting-style: no pyramiding, no hedged legs, no partial fills.
|
||||
- Quote data only (bid/ask). Spot FX/metals CFDs have no real volume feed; if your
|
||||
strategy needs volume, you need futures data instead.
|
||||
- Costs = spread (paid via bid/ask fills) + a fixed round-trip add. No market-impact
|
||||
model — realistic for retail sizes, not for size.
|
||||
- Timestamps are broker time; the library never converts time zones.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Replay throughput on synthetic ticks.
|
||||
|
||||
Run: python examples/benchmark.py [--ticks 20000000]
|
||||
"""
|
||||
import argparse
|
||||
import statistics
|
||||
import time
|
||||
|
||||
import numpy as np
|
||||
|
||||
import tickreplay as tr
|
||||
|
||||
TICKS_PER_DAY = 200_000
|
||||
WARMUP_TICKS = 100_000
|
||||
|
||||
|
||||
def build_store(n_ticks, seed):
|
||||
# generate_ticks skips weekends, so ask for enough calendar days to cover it.
|
||||
days = int(np.ceil(n_ticks / TICKS_PER_DAY * 7 / 5)) + 3
|
||||
time_ns, bid, ask = tr.synth.generate_ticks(
|
||||
days=days, ticks_per_day=TICKS_PER_DAY, seed=seed)
|
||||
return tr.TickStore(time_ns[:n_ticks], bid[:n_ticks], ask[:n_ticks])
|
||||
|
||||
|
||||
def ma_cross_signal(bars):
|
||||
fast = bars["close"].rolling(10).mean()
|
||||
slow = bars["close"].rolling(50).mean()
|
||||
regime = np.sign(fast - slow).fillna(0.0).to_numpy()
|
||||
return np.where(regime != np.r_[0.0, regime[:-1]], regime, 0).astype("int8")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Replay throughput on synthetic ticks.")
|
||||
parser.add_argument("--ticks", type=int, default=20_000_000,
|
||||
help="store size; 20M ticks needs about 1 GB of RAM")
|
||||
parser.add_argument("--timeframe", default="M5", choices=sorted(tr.TIMEFRAMES),
|
||||
help="bar timeframe the signal is built on")
|
||||
parser.add_argument("--seed", type=int, default=1, help="synthetic data seed")
|
||||
args = parser.parse_args()
|
||||
|
||||
t0 = time.perf_counter()
|
||||
store = build_store(args.ticks, args.seed)
|
||||
gen_ms = (time.perf_counter() - t0) * 1e3
|
||||
|
||||
config = tr.Config(sl=3.0, tp=6.0, max_hold="4h", cost=0.05)
|
||||
|
||||
# Compile the kernel on a small slice so JIT time stays out of the measurement.
|
||||
warm = tr.TickStore(store.time_ns[:WARMUP_TICKS], store.bid[:WARMUP_TICKS],
|
||||
store.ask[:WARMUP_TICKS])
|
||||
warm_bars = warm.bars(args.timeframe)
|
||||
tr.replay(warm, warm_bars, ma_cross_signal(warm_bars), config)
|
||||
|
||||
t0 = time.perf_counter()
|
||||
bars = store.bars(args.timeframe)
|
||||
bars_ms = (time.perf_counter() - t0) * 1e3
|
||||
signal = ma_cross_signal(bars)
|
||||
|
||||
timings = []
|
||||
for _ in range(3):
|
||||
t0 = time.perf_counter()
|
||||
result = tr.replay(store, bars, signal, config)
|
||||
timings.append((time.perf_counter() - t0) * 1e3)
|
||||
median_ms = statistics.median(timings)
|
||||
|
||||
# The kernel only walks ticks while a position is open, so quote the covered
|
||||
# fraction alongside the headline rate.
|
||||
visited = int((result.arrays["exit_idx"] - result.arrays["entry_idx"]).sum())
|
||||
|
||||
print(f"ticks {len(store):>14,}")
|
||||
print(f"{args.timeframe} bars{'':<10}{len(bars):>14,}")
|
||||
print(f"trades {len(result):>14,}")
|
||||
print(f"ticks in position{visited:>14,} ({visited / max(len(store), 1):.1%})")
|
||||
print(f"generate {gen_ms:>14,.1f} ms")
|
||||
print(f"derive bars {bars_ms:>14,.1f} ms")
|
||||
print(f"replay (median) {median_ms:>14,.1f} ms of {[round(x, 1) for x in timings]}")
|
||||
print(f"kernel rate {visited / median_ms / 1e3:>14,.1f} M ticks/s scanned")
|
||||
print(f"end-to-end rate {len(store) / median_ms / 1e3:>14,.1f} M store ticks/s")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,46 @@
|
||||
"""N-bar breakout on M5 bars, exiting on the opposite breakout.
|
||||
|
||||
Run: python examples/breakout.py
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
import tickreplay as tr
|
||||
|
||||
LOOKBACK = 20
|
||||
|
||||
|
||||
def main():
|
||||
time_ns, bid, ask = tr.synth.generate_ticks(days=30, seed=11)
|
||||
store = tr.TickStore(time_ns, bid, ask)
|
||||
bars = store.bars("M5")
|
||||
|
||||
# shift(1) keeps the current bar out of its own reference range.
|
||||
prior_high = bars["high"].shift(1).rolling(LOOKBACK).max()
|
||||
prior_low = bars["low"].shift(1).rolling(LOOKBACK).min()
|
||||
state = np.where(bars["close"] > prior_high, 1,
|
||||
np.where(bars["close"] < prior_low, -1, 0)).astype("int8")
|
||||
|
||||
# Fire on the bar that breaks out, not on every bar that stays beyond the range.
|
||||
signal = np.where(state != np.r_[np.int8(0), state[:-1]], state, 0).astype("int8")
|
||||
|
||||
config = tr.Config(sl=4.0, tp=8.0, max_hold="8h", exit_on_opposite=True, cost=0.05)
|
||||
result = tr.replay(store, bars, signal, config)
|
||||
|
||||
print(f"ticks {len(store):>12,}")
|
||||
print(f"M5 bars {len(bars):>12,}")
|
||||
print(f"signals {int((signal != 0).sum()):>12,} ({LOOKBACK}-bar breakout)")
|
||||
print("config sl=4.0 tp=8.0 max_hold=8h exit_on_opposite=True cost=0.05")
|
||||
print()
|
||||
for key, value in result.summary().items():
|
||||
print(f"{key:<14}{value:>12.4f}" if isinstance(value, float)
|
||||
else f"{key:<14}{value:>12,}")
|
||||
|
||||
print()
|
||||
print("exit reasons")
|
||||
counts = result.trades["exit_reason"].value_counts()
|
||||
for reason, count in counts.items():
|
||||
print(f" {reason:<10}{count:>6,}{count / counts.sum():>8.1%}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Moving-average cross on M5 bars, replayed tick by tick.
|
||||
|
||||
Run: python examples/ma_cross.py
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
import tickreplay as tr
|
||||
|
||||
|
||||
def main():
|
||||
time_ns, bid, ask = tr.synth.generate_ticks(days=30, seed=7)
|
||||
store = tr.TickStore(time_ns, bid, ask)
|
||||
bars = store.bars("M5")
|
||||
|
||||
fast = bars["close"].rolling(10).mean()
|
||||
slow = bars["close"].rolling(50).mean()
|
||||
|
||||
# Regime is +1 while fast is above slow. The signal fires only on the bar where
|
||||
# the regime flips, so one cross is one entry instead of one entry per bar.
|
||||
regime = np.sign(fast - slow).fillna(0.0).to_numpy()
|
||||
previous = np.r_[0.0, regime[:-1]]
|
||||
signal = np.where(regime != previous, regime, 0).astype("int8")
|
||||
|
||||
# signal[b] is built from closes up to and including bar b, so it is knowable
|
||||
# only once bar b has closed. No extra shift is needed: the engine enters at the
|
||||
# first tick after the signal bar closes.
|
||||
config = tr.Config(sl=3.0, tp=6.0, max_hold="4h", cost=0.05)
|
||||
result = tr.replay(store, bars, signal, config)
|
||||
|
||||
print(f"ticks {len(store):>12,}")
|
||||
print(f"M5 bars {len(bars):>12,}")
|
||||
print(f"signals {int((signal != 0).sum()):>12,}")
|
||||
print("config sl=3.0 tp=6.0 max_hold=4h cost=0.05 (price units)")
|
||||
print()
|
||||
for key, value in result.summary().items():
|
||||
print(f"{key:<14}{value:>12.4f}" if isinstance(value, float)
|
||||
else f"{key:<14}{value:>12,}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Sweep exit rules, then test the winner honestly.
|
||||
|
||||
Picking the best cell of a grid and reporting its p-value as-is is the standard way
|
||||
to manufacture an edge. This example sweeps, then thins overlapping trades before
|
||||
testing, and reads the result against a selection-adjusted threshold.
|
||||
|
||||
Run: python examples/sweep_and_validate.py
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
import tickreplay as tr
|
||||
|
||||
HOUR_NS = 3_600 * 10**9
|
||||
|
||||
|
||||
def main():
|
||||
time_ns, bid, ask = tr.synth.generate_ticks(days=60, seed=3)
|
||||
store = tr.TickStore(time_ns, bid, ask)
|
||||
bars = store.bars("M5")
|
||||
|
||||
fast = bars["close"].rolling(10).mean()
|
||||
slow = bars["close"].rolling(50).mean()
|
||||
regime = np.sign(fast - slow).fillna(0.0).to_numpy()
|
||||
signal = np.where(regime != np.r_[0.0, regime[:-1]], regime, 0).astype("int8")
|
||||
|
||||
base = {"max_hold": "4h", "cost": 0.05}
|
||||
grid = [{"sl": sl, "tp": tp} for sl in (2.0, 3.0, 4.0) for tp in (4.0, 6.0, 8.0)]
|
||||
table = tr.sweep_exits(store, bars, signal, grid, base=base)
|
||||
table = table.sort_values("total_net", ascending=False).reset_index(drop=True)
|
||||
|
||||
print(f"{len(grid)} exit configs over {len(store):,} ticks\n")
|
||||
print(table[["sl", "tp", "n_trades", "total_net", "hit_rate",
|
||||
"max_drawdown"]].to_string(index=False, float_format="%.3f"))
|
||||
|
||||
best = table.iloc[0]
|
||||
result = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=best["sl"], tp=best["tp"], **base))
|
||||
pnl = result.arrays["pnl_net"]
|
||||
|
||||
# Trades whose holding windows overlap are correlated; a t-stat over all of them
|
||||
# is inflated. Test a non-overlapping subset spaced by the max hold time.
|
||||
keep = tr.research.stats.thin_by_gap(result.arrays["entry_time_ns"], 4 * HOUR_NS)
|
||||
t_stat, p_value = tr.research.stats.one_sided_positive(pnl[keep])
|
||||
threshold = 0.05 / len(grid)
|
||||
|
||||
print(f"\nbest by total_net: sl={best['sl']} tp={best['tp']}")
|
||||
print(f" trades {len(pnl):>8,} -> {int(keep.sum()):,} after thinning")
|
||||
print(f" one-sided t {t_stat:>8.3f}")
|
||||
print(f" p (mean > 0) {p_value:>8.4f}")
|
||||
print(f" threshold (0.05/{len(grid)}) {threshold:>8.4f}")
|
||||
print(f"\n verdict: {'NOT significant' if p_value > threshold else 'passes'}"
|
||||
" once selection over the grid is accounted for.")
|
||||
print(" Expected: this is a synthetic random walk with costs, so there is no edge"
|
||||
"\n to find. A pass here would mean the procedure is broken, not that the"
|
||||
"\n strategy works.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,38 @@
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "tickreplay"
|
||||
version = "0.1.0"
|
||||
description = "Tick-accurate backtesting for MT5 data: memory-mapped tickstores, a JIT replay kernel, and research tools that guard against overfitting"
|
||||
readme = "README.md"
|
||||
license = { file = "LICENSE" }
|
||||
authors = [{ name = "danielmjrr" }]
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"numpy>=1.26",
|
||||
"pandas>=2.2",
|
||||
"numba>=0.59",
|
||||
"pyarrow>=15.0",
|
||||
"scipy>=1.11",
|
||||
]
|
||||
classifiers = [
|
||||
"Programming Language :: Python :: 3",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Operating System :: OS Independent",
|
||||
"Intended Audience :: Financial and Insurance Industry",
|
||||
"Topic :: Office/Business :: Financial :: Investment",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/danielmjrr/tickreplay"
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = ["pytest>=8.0"]
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/tickreplay"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
@@ -0,0 +1,27 @@
|
||||
"""tickreplay: tick-accurate backtesting for MT5 data.
|
||||
|
||||
Quickstart::
|
||||
|
||||
import numpy as np
|
||||
import tickreplay as tr
|
||||
|
||||
store = tr.TickStore.open("data/my_symbol") # built once from an MT5 export
|
||||
bars = store.bars("M5")
|
||||
|
||||
fast = bars.close.rolling(10).mean()
|
||||
slow = bars.close.rolling(50).mean()
|
||||
signal = np.sign(fast - slow).fillna(0).astype("int8")
|
||||
signal[signal == np.roll(signal, 1)] = 0 # act on crosses only
|
||||
|
||||
result = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=5.0, tp=10.0, max_hold="4h", cost=0.09))
|
||||
print(result.summary())
|
||||
"""
|
||||
from . import ingest, research, synth
|
||||
from .engine import Config, Result, replay
|
||||
from .store import TIMEFRAMES, TickStore, derive_bars
|
||||
from .sweep import run_sweep, sweep_exits
|
||||
|
||||
__version__ = "0.1.0"
|
||||
__all__ = ["Config", "Result", "TickStore", "TIMEFRAMES", "derive_bars", "ingest",
|
||||
"replay", "research", "run_sweep", "sweep_exits", "synth"]
|
||||
@@ -0,0 +1,181 @@
|
||||
"""Single-position tick replay.
|
||||
|
||||
Semantics, chosen to match how a live EA on a netting account behaves:
|
||||
|
||||
- A nonzero signal on bar ``b`` enters at the first tick after the bar closes
|
||||
(``tick_last + 1``) — buys at ask, sells at bid. No same-bar fills, no look-ahead.
|
||||
- One position at a time. Signals arriving while a position is open are skipped, so
|
||||
results measure the *tradable* expectancy of the strategy, not an overlapped ideal.
|
||||
- Exit checks per tick, in priority order: stop loss, take profit, max hold time,
|
||||
opposite signal. Anything still open at the last tick closes there (``eod``).
|
||||
- Positions are marked to the liquidation side: longs to bid, shorts to ask. Stops
|
||||
and targets are distances from the entry price in price units and may vary per bar
|
||||
(e.g. ATR-scaled).
|
||||
- A signal on the exit tick itself is ignored; the next entry needs a later tick.
|
||||
"""
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from numba import njit
|
||||
|
||||
EXIT_REASONS = {1: "sl", 2: "tp", 3: "time", 4: "opposite", 5: "eod"}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
"""Exit rules and costs for one replay.
|
||||
|
||||
``sl``/``tp``: distance from entry in price units — a scalar for all bars or an
|
||||
array per bar; 0 disables. ``max_hold``: pandas offset string ("4h") or seconds;
|
||||
0 disables. ``cost``: round-trip cost added on top of the spread, in price units
|
||||
(spread itself is already paid via bid/ask fills).
|
||||
"""
|
||||
sl: float | np.ndarray = 0.0
|
||||
tp: float | np.ndarray = 0.0
|
||||
max_hold: str | float = 0
|
||||
exit_on_opposite: bool = False
|
||||
allow_buy: bool = True
|
||||
allow_sell: bool = True
|
||||
cost: float = 0.0
|
||||
|
||||
def max_hold_ns(self):
|
||||
if isinstance(self.max_hold, str):
|
||||
return pd.Timedelta(self.max_hold).value
|
||||
return int(self.max_hold) * 10**9 + int(self.max_hold % 1 * 1e9)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Result:
|
||||
"""Trade ledger from one replay. ``arrays`` holds the raw numpy columns."""
|
||||
arrays: dict = field(repr=False)
|
||||
config: Config = field(repr=False)
|
||||
|
||||
def __len__(self):
|
||||
return len(self.arrays["dir"])
|
||||
|
||||
@property
|
||||
def trades(self):
|
||||
a = self.arrays
|
||||
df = pd.DataFrame({k: v for k, v in a.items() if k not in
|
||||
("entry_time_ns", "exit_time_ns", "exit_reason")})
|
||||
df["entry_time"] = pd.to_datetime(a["entry_time_ns"], unit="ns")
|
||||
df["exit_time"] = pd.to_datetime(a["exit_time_ns"], unit="ns")
|
||||
df["exit_reason"] = pd.Series(a["exit_reason"]).map(EXIT_REASONS)
|
||||
return df
|
||||
|
||||
def summary(self):
|
||||
pnl = self.arrays["pnl_net"]
|
||||
if len(pnl) == 0:
|
||||
return {"n_trades": 0, "total_net": 0.0, "mean_net": 0.0, "hit_rate": 0.0,
|
||||
"max_drawdown": 0.0}
|
||||
cum = np.cumsum(pnl)
|
||||
dd = np.maximum.accumulate(cum) - cum
|
||||
return {
|
||||
"n_trades": len(pnl),
|
||||
"total_net": float(cum[-1]),
|
||||
"mean_net": float(pnl.mean()),
|
||||
"hit_rate": float((pnl > 0).mean()),
|
||||
"max_drawdown": float(dd.max()),
|
||||
}
|
||||
|
||||
|
||||
@njit(cache=True, nogil=True)
|
||||
def _replay(time_ns, bid, ask, bar_tick_last, bar_signal, sl_dist, tp_dist,
|
||||
allow_buy, allow_sell, max_hold_ns, exit_on_opposite,
|
||||
o_dir, o_entry_idx, o_exit_idx, o_entry_price, o_exit_price,
|
||||
o_pnl_gross, o_mfe, o_mae, o_reason):
|
||||
n_ticks = time_ns.shape[0]
|
||||
n_bars = bar_signal.shape[0]
|
||||
t = 0
|
||||
min_trigger = 0 # post-exit: exit tick + 1 (signals at the exit tick are ignored)
|
||||
for b in range(n_bars):
|
||||
s = bar_signal[b]
|
||||
if s == 0 or (s == 1 and not allow_buy) or (s == -1 and not allow_sell):
|
||||
continue
|
||||
e = bar_tick_last[b] + 1
|
||||
if e < min_trigger:
|
||||
continue
|
||||
if e >= n_ticks - 1: # no tick after entry to evaluate -> no trade
|
||||
continue
|
||||
entry_price = ask[e] if s == 1 else bid[e]
|
||||
deadline = time_ns[e] + max_hold_ns
|
||||
sl = sl_dist[b]
|
||||
tp = tp_dist[b]
|
||||
opp_trigger = n_ticks # sentinel: never reached
|
||||
if exit_on_opposite:
|
||||
for ob in range(b + 1, n_bars):
|
||||
if bar_signal[ob] == -s:
|
||||
opp_trigger = bar_tick_last[ob] + 1
|
||||
break
|
||||
mfe = 0.0
|
||||
mae = 0.0
|
||||
exit_idx = n_ticks - 1
|
||||
reason = 5 # eod
|
||||
for i in range(e + 1, n_ticks):
|
||||
pnl_raw = bid[i] - entry_price if s == 1 else entry_price - ask[i]
|
||||
if pnl_raw > mfe:
|
||||
mfe = pnl_raw
|
||||
if -pnl_raw > mae:
|
||||
mae = -pnl_raw
|
||||
if sl > 0 and pnl_raw <= -sl:
|
||||
exit_idx = i; reason = 1; break
|
||||
if tp > 0 and pnl_raw >= tp:
|
||||
exit_idx = i; reason = 2; break
|
||||
if max_hold_ns > 0 and time_ns[i] >= deadline:
|
||||
exit_idx = i; reason = 3; break
|
||||
if i == opp_trigger:
|
||||
exit_idx = i; reason = 4; break
|
||||
exit_price = bid[exit_idx] if s == 1 else ask[exit_idx]
|
||||
o_dir[t] = s
|
||||
o_entry_idx[t] = e
|
||||
o_exit_idx[t] = exit_idx
|
||||
o_entry_price[t] = entry_price
|
||||
o_exit_price[t] = exit_price
|
||||
o_pnl_gross[t] = exit_price - entry_price if s == 1 else entry_price - exit_price
|
||||
o_mfe[t] = mfe
|
||||
o_mae[t] = mae
|
||||
o_reason[t] = reason
|
||||
t += 1
|
||||
min_trigger = exit_idx + 1
|
||||
return t
|
||||
|
||||
|
||||
def replay(store, bars, signal, config=None):
|
||||
"""Replay ``signal`` (+1 buy / -1 sell / 0 flat, one value per bar) through the
|
||||
tick stream. Returns a :class:`Result`."""
|
||||
config = config or Config()
|
||||
signal = np.ascontiguousarray(signal, dtype=np.int8)
|
||||
if len(signal) != len(bars):
|
||||
raise ValueError(f"signal has {len(signal)} values for {len(bars)} bars")
|
||||
tick_last = np.ascontiguousarray(bars["tick_last"].to_numpy(), np.int64)
|
||||
if len(tick_last) > 1 and (np.diff(tick_last) <= 0).any():
|
||||
raise ValueError("bars must have strictly increasing tick_last")
|
||||
n_bars = len(bars)
|
||||
sl = np.ascontiguousarray(np.broadcast_to(np.nan_to_num(config.sl), n_bars),
|
||||
dtype=np.float64)
|
||||
tp = np.ascontiguousarray(np.broadcast_to(np.nan_to_num(config.tp), n_bars),
|
||||
dtype=np.float64)
|
||||
cap = int((signal != 0).sum()) + 1
|
||||
o = {
|
||||
"dir": np.zeros(cap, np.int8),
|
||||
"entry_idx": np.zeros(cap, np.int64), "exit_idx": np.zeros(cap, np.int64),
|
||||
"entry_price": np.zeros(cap, np.float64), "exit_price": np.zeros(cap, np.float64),
|
||||
"pnl_gross": np.zeros(cap, np.float64),
|
||||
"mfe": np.zeros(cap, np.float64), "mae": np.zeros(cap, np.float64),
|
||||
"exit_reason": np.zeros(cap, np.int8),
|
||||
}
|
||||
time_ns = np.ascontiguousarray(store.time_ns, np.int64)
|
||||
n = _replay(time_ns,
|
||||
np.ascontiguousarray(store.bid, np.float64),
|
||||
np.ascontiguousarray(store.ask, np.float64),
|
||||
tick_last,
|
||||
signal, sl, tp, config.allow_buy, config.allow_sell,
|
||||
np.int64(config.max_hold_ns()), config.exit_on_opposite,
|
||||
o["dir"], o["entry_idx"], o["exit_idx"], o["entry_price"],
|
||||
o["exit_price"], o["pnl_gross"], o["mfe"], o["mae"], o["exit_reason"])
|
||||
arrays = {k: v[:n].copy() for k, v in o.items()}
|
||||
arrays["entry_time_ns"] = time_ns[arrays["entry_idx"]]
|
||||
arrays["exit_time_ns"] = time_ns[arrays["exit_idx"]]
|
||||
arrays["pnl_net"] = arrays["pnl_gross"] - config.cost
|
||||
return Result(arrays, config)
|
||||
@@ -0,0 +1,123 @@
|
||||
"""MT5 export ingestion: tick CSV -> monthly parquet, plus reconstruction checks.
|
||||
|
||||
MT5 tick exports are tab-separated: <DATE> <TIME> <BID> <ASK> <LAST> <VOLUME> <FLAGS>.
|
||||
Rows may carry only one quote side; the missing side is forward-filled (quote state),
|
||||
matching how the terminal itself maintains the current bid/ask. MT5 bars are bid-based.
|
||||
"""
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def _parse_time(date_col, time_col):
|
||||
s = date_col + " " + time_col
|
||||
try:
|
||||
return pd.to_datetime(s, format="%Y.%m.%d %H:%M:%S.%f")
|
||||
except ValueError:
|
||||
return pd.to_datetime(s, format="mixed", dayfirst=False)
|
||||
|
||||
|
||||
def read_mt5_ticks(src, chunksize=5_000_000):
|
||||
"""Yield DataFrames [time, bid, ask] with quote state forward-filled across chunks.
|
||||
Rows before both sides are first known are dropped."""
|
||||
carry_bid = float("nan")
|
||||
carry_ask = float("nan")
|
||||
for chunk in pd.read_csv(src, sep="\t", chunksize=chunksize):
|
||||
chunk.columns = [c.strip("<>").lower() for c in chunk.columns]
|
||||
bid = pd.concat([pd.Series([carry_bid]), chunk["bid"].astype("float64")],
|
||||
ignore_index=True).ffill().iloc[1:].reset_index(drop=True)
|
||||
ask = pd.concat([pd.Series([carry_ask]), chunk["ask"].astype("float64")],
|
||||
ignore_index=True).ffill().iloc[1:].reset_index(drop=True)
|
||||
carry_bid = bid.iloc[-1]
|
||||
carry_ask = ask.iloc[-1]
|
||||
df = pd.DataFrame({
|
||||
"time": _parse_time(chunk["date"], chunk["time"]).reset_index(drop=True),
|
||||
"bid": bid,
|
||||
"ask": ask,
|
||||
}).dropna(subset=["bid", "ask"]).reset_index(drop=True)
|
||||
if len(df):
|
||||
yield df
|
||||
|
||||
|
||||
def ticks_to_monthly_parquet(src, out_dir, chunksize=5_000_000):
|
||||
"""Convert an MT5 tick CSV to per-month parquet files; returns a manifest.
|
||||
|
||||
Assumes chronological input (MT5 exports are). A month is flushed once a later
|
||||
month appears, keeping peak memory at ~one month + one chunk.
|
||||
"""
|
||||
out_dir = Path(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
buffers = {}
|
||||
months_written = {}
|
||||
t_min = t_max = None
|
||||
|
||||
def flush(month):
|
||||
df = pd.concat(buffers.pop(month), ignore_index=True)
|
||||
path = out_dir / f"ticks_{month}.parquet"
|
||||
if path.exists():
|
||||
raise RuntimeError(f"non-chronological input: {month} seen again after flush")
|
||||
df.to_parquet(path, index=False)
|
||||
months_written[str(month)] = len(df)
|
||||
|
||||
for df in read_mt5_ticks(src, chunksize=chunksize):
|
||||
t_min = df["time"].iloc[0] if t_min is None else t_min
|
||||
t_max = df["time"].iloc[-1]
|
||||
for month, part in df.groupby(df["time"].dt.to_period("M"), sort=True):
|
||||
buffers.setdefault(str(month), []).append(part.reset_index(drop=True))
|
||||
latest = max(buffers)
|
||||
for month in [m for m in sorted(buffers) if m != latest]:
|
||||
flush(month)
|
||||
for month in sorted(buffers):
|
||||
flush(month)
|
||||
|
||||
manifest = {
|
||||
"source": str(src),
|
||||
"total_ticks": int(sum(months_written.values())),
|
||||
"months": months_written,
|
||||
"range": [str(t_min), str(t_max)],
|
||||
}
|
||||
(out_dir / "manifest.json").write_text(json.dumps(manifest, indent=2))
|
||||
return manifest
|
||||
|
||||
|
||||
def read_mt5_bars(src):
|
||||
"""Parse an MT5 bar CSV (any timeframe) to a time-indexed OHLC frame."""
|
||||
df = pd.read_csv(src, sep="\t")
|
||||
df.columns = [c.strip("<>").lower() for c in df.columns]
|
||||
if "time" in df.columns:
|
||||
df["time"] = pd.to_datetime(df["date"] + " " + df["time"],
|
||||
format="%Y.%m.%d %H:%M:%S")
|
||||
else: # daily and above: no <TIME> column
|
||||
df["time"] = pd.to_datetime(df["date"], format="%Y.%m.%d")
|
||||
return df.set_index("time")[["open", "high", "low", "close", "tickvol", "spread"]]
|
||||
|
||||
|
||||
def reconstruct_m1(ticks):
|
||||
"""Bid-based M1 OHLC from a tick frame; minutes without ticks produce no bar."""
|
||||
g = ticks.set_index("time")["bid"].resample("1min")
|
||||
bars = g.agg(["first", "max", "min", "last"]).dropna()
|
||||
bars.columns = ["open", "high", "low", "close"]
|
||||
return bars
|
||||
|
||||
|
||||
def validate_against_broker_bars(ticks, broker_bars):
|
||||
"""Compare tick-reconstructed M1 OHLC against broker-exported bars on the overlap.
|
||||
|
||||
Zero mismatches is the expected outcome for a clean export; anything else usually
|
||||
means the tick and bar files come from different terminals or accounts.
|
||||
"""
|
||||
recon = reconstruct_m1(ticks)
|
||||
j = recon.join(broker_bars[["open", "high", "low", "close"]],
|
||||
how="inner", lsuffix="_r", rsuffix="_b")
|
||||
mism = j[(j.open_r != j.open_b) | (j.high_r != j.high_b)
|
||||
| (j.low_r != j.low_b) | (j.close_r != j.close_b)]
|
||||
in_range = broker_bars.index[(broker_bars.index >= recon.index.min())
|
||||
& (broker_bars.index <= recon.index.max())]
|
||||
return {
|
||||
"compared": int(len(j)),
|
||||
"ohlc_mismatches": int(len(mism)),
|
||||
"mismatch_sample": [str(t) for t in mism.index[:20]],
|
||||
"broker_only": int(len(in_range.difference(recon.index))),
|
||||
"recon_only": int(len(recon.index.difference(broker_bars.index))),
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
"""Research toolkit: measurement and overfitting protection.
|
||||
|
||||
The engine answers "what would this strategy have made"; this package answers the
|
||||
harder question — "is that number real". Excursion and barrier labels for
|
||||
measurement, purged walk-forward splits, overlap-thinned significance tests, an
|
||||
append-only registry with cumulative false-discovery control, and an online
|
||||
changepoint detector.
|
||||
"""
|
||||
from . import bocpd, labels, registry, stats, validation
|
||||
|
||||
__all__ = ["bocpd", "labels", "registry", "stats", "validation"]
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Bayesian online changepoint detection (Adams & MacKay 2007), Student-t predictive.
|
||||
|
||||
Strictly filtered: the run-length posterior at index i uses observations <= i only.
|
||||
Normal-Inverse-Gamma conjugate observation model; constant hazard; run-length vector
|
||||
truncated for speed (mass < trunc dropped, hard cap on vector size).
|
||||
"""
|
||||
import numpy as np
|
||||
from scipy.special import gammaln
|
||||
|
||||
|
||||
def _t_logpdf(x, df, loc, scale):
|
||||
z = (x - loc) / scale
|
||||
return (gammaln((df + 1) / 2) - gammaln(df / 2)
|
||||
- 0.5 * np.log(df * np.pi) - np.log(scale)
|
||||
- (df + 1) / 2 * np.log1p(z * z / df))
|
||||
|
||||
|
||||
def bocpd_features(x, hazard=1 / 250, mu0=0.0, kappa0=1.0, alpha0=1.0, beta0=1.0,
|
||||
cap=3000, trunc=1e-6):
|
||||
"""Returns per-observation filtered {"p_cp_10": P(run<=10), "erl": E[run length]}."""
|
||||
x = np.asarray(x, dtype="float64")
|
||||
n = len(x)
|
||||
p_cp = np.zeros(n)
|
||||
erl = np.zeros(n)
|
||||
rl = np.array([0]) # run lengths carried by each hypothesis
|
||||
R = np.array([1.0])
|
||||
mu, kappa = np.array([mu0]), np.array([kappa0])
|
||||
alpha, beta = np.array([alpha0]), np.array([beta0])
|
||||
for i in range(n):
|
||||
pred = np.exp(_t_logpdf(x[i], 2 * alpha, mu,
|
||||
np.sqrt(beta * (kappa + 1) / (alpha * kappa))))
|
||||
growth = R * pred * (1 - hazard)
|
||||
cp = float((R * pred).sum() * hazard)
|
||||
R = np.r_[cp, growth]
|
||||
R /= R.sum()
|
||||
rl = np.r_[0, rl + 1]
|
||||
beta = np.r_[beta0, beta + kappa * (x[i] - mu) ** 2 / (2 * (kappa + 1))]
|
||||
mu = np.r_[mu0, (kappa * mu + x[i]) / (kappa + 1)]
|
||||
alpha = np.r_[alpha0, alpha + 0.5]
|
||||
kappa = np.r_[kappa0, kappa + 1]
|
||||
p_cp[i] = R[rl <= 10].sum()
|
||||
erl[i] = (rl * R).sum()
|
||||
keep = R >= trunc
|
||||
keep[0] = True
|
||||
if keep.sum() > cap:
|
||||
keep &= R >= np.partition(R, -cap)[-cap]
|
||||
rl, R = rl[keep], R[keep]
|
||||
R /= R.sum()
|
||||
mu, kappa, alpha, beta = mu[keep], kappa[keep], alpha[keep], beta[keep]
|
||||
return {"p_cp_10": p_cp, "erl": erl}
|
||||
@@ -0,0 +1,102 @@
|
||||
"""Per-signal tick-level labels for research (not trading).
|
||||
|
||||
Unlike the replay engine, every signal gets labeled — overlap allowed — because
|
||||
these are measurement tools: excursion profiles and barrier outcomes feed statistics,
|
||||
not a trade ledger. The marking convention matches the engine: longs to bid, shorts
|
||||
to ask, entry at the first tick after the signal bar.
|
||||
"""
|
||||
import numpy as np
|
||||
from numba import njit, prange
|
||||
|
||||
|
||||
@njit(cache=True, parallel=True)
|
||||
def _excursions(time_ns, bid, ask, entry_ticks, dirs, checkpoints_ns,
|
||||
mfe, mae, ret, entry_price):
|
||||
n_ticks = time_ns.shape[0]
|
||||
n_cp = checkpoints_ns.shape[0]
|
||||
for k in prange(entry_ticks.shape[0]):
|
||||
e = entry_ticks[k]
|
||||
d = dirs[k]
|
||||
ep = ask[e] if d == 1 else bid[e]
|
||||
entry_price[k] = ep
|
||||
t0 = time_ns[e]
|
||||
f = 0.0
|
||||
a = 0.0
|
||||
pnl = 0.0
|
||||
j = e + 1
|
||||
for ci in range(n_cp):
|
||||
limit = t0 + checkpoints_ns[ci]
|
||||
while j < n_ticks and time_ns[j] <= limit:
|
||||
pnl = bid[j] - ep if d == 1 else ep - ask[j]
|
||||
if pnl > f:
|
||||
f = pnl
|
||||
if -pnl > a:
|
||||
a = -pnl
|
||||
j += 1
|
||||
mfe[k, ci] = f
|
||||
mae[k, ci] = a
|
||||
ret[k, ci] = pnl # signed value at the last tick <= horizon
|
||||
|
||||
|
||||
def excursions(time_ns, bid, ask, entry_ticks, dirs, checkpoints_ns):
|
||||
"""Running MFE/MAE and close-out return at each horizon checkpoint (ascending).
|
||||
Returns dict of (n_signals, n_checkpoints) arrays plus entry prices."""
|
||||
n, m = len(entry_ticks), len(checkpoints_ns)
|
||||
mfe = np.zeros((n, m))
|
||||
mae = np.zeros((n, m))
|
||||
ret = np.zeros((n, m))
|
||||
entry_price = np.zeros(n)
|
||||
_excursions(np.ascontiguousarray(time_ns, np.int64),
|
||||
np.ascontiguousarray(bid, np.float64),
|
||||
np.ascontiguousarray(ask, np.float64),
|
||||
np.ascontiguousarray(entry_ticks, np.int64),
|
||||
np.ascontiguousarray(dirs, np.int8),
|
||||
np.ascontiguousarray(checkpoints_ns, np.int64),
|
||||
mfe, mae, ret, entry_price)
|
||||
return {"mfe": mfe, "mae": mae, "ret": ret, "entry_price": entry_price}
|
||||
|
||||
|
||||
@njit(cache=True, parallel=True)
|
||||
def _first_passage(time_ns, bid, ask, entry_ticks, dirs, targets, stops, horizon_ns,
|
||||
outcome, pnl_out, exit_time):
|
||||
n_ticks = time_ns.shape[0]
|
||||
for k in prange(entry_ticks.shape[0]):
|
||||
e = entry_ticks[k]
|
||||
d = dirs[k]
|
||||
ep = ask[e] if d == 1 else bid[e]
|
||||
limit = time_ns[e] + horizon_ns
|
||||
pnl = 0.0
|
||||
oc = 0
|
||||
et = min(limit, time_ns[n_ticks - 1])
|
||||
j = e + 1
|
||||
while j < n_ticks and time_ns[j] <= limit:
|
||||
pnl = bid[j] - ep if d == 1 else ep - ask[j]
|
||||
if pnl >= targets[k]:
|
||||
oc, et = 1, time_ns[j]
|
||||
break
|
||||
if -pnl >= stops[k]:
|
||||
oc, et = -1, time_ns[j]
|
||||
break
|
||||
j += 1
|
||||
outcome[k] = oc
|
||||
pnl_out[k] = pnl
|
||||
exit_time[k] = et
|
||||
|
||||
|
||||
def first_passage(time_ns, bid, ask, entry_ticks, dirs, targets, stops, horizon_ns):
|
||||
"""Which barrier is touched first within the horizon: +1 target, -1 stop,
|
||||
0 timeout. On ticks the ordering is exact (no OHLC path ambiguity), and a gap
|
||||
through a barrier pays the real crossing price, not the barrier level."""
|
||||
n = len(entry_ticks)
|
||||
outcome = np.zeros(n, np.int8)
|
||||
pnl = np.zeros(n)
|
||||
exit_time = np.zeros(n, np.int64)
|
||||
_first_passage(np.ascontiguousarray(time_ns, np.int64),
|
||||
np.ascontiguousarray(bid, np.float64),
|
||||
np.ascontiguousarray(ask, np.float64),
|
||||
np.ascontiguousarray(entry_ticks, np.int64),
|
||||
np.ascontiguousarray(dirs, np.int8),
|
||||
np.ascontiguousarray(targets, np.float64),
|
||||
np.ascontiguousarray(stops, np.float64),
|
||||
np.int64(horizon_ns), outcome, pnl, exit_time)
|
||||
return {"outcome": outcome, "pnl": pnl, "exit_time_ns": exit_time}
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Append-only evaluation registry: every evaluation, no exceptions.
|
||||
|
||||
The point is multiple-testing honesty. If you run a thousand backtests and report
|
||||
the best one, its p-value is meaningless; the registry records every attempt, and
|
||||
q-values are recomputed over the FULL history on read — never stored — so each new
|
||||
evaluation makes every past q slightly stricter. Keep the CSV in version control.
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from .stats import bh_qvalues
|
||||
|
||||
REQUIRED = ["campaign", "symbol", "config", "n_trades", "mean_net", "t_stat", "p_value"]
|
||||
|
||||
|
||||
def append(path, rows):
|
||||
for r in rows:
|
||||
missing = [c for c in REQUIRED if c not in r]
|
||||
if missing:
|
||||
raise ValueError(f"registry row missing {missing}")
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
ts = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
df = pd.DataFrame([{**r, "ts": ts} for r in rows])
|
||||
df.to_csv(path, mode="a", header=not path.exists(), index=False)
|
||||
|
||||
|
||||
def load(path):
|
||||
"""Full registry with q-values computed over every row ever appended."""
|
||||
df = pd.read_csv(path)
|
||||
df["q_value"] = bh_qvalues(df["p_value"].to_numpy())
|
||||
return df
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Honest statistics for strategy evaluation.
|
||||
|
||||
Three tools that together keep a research program from fooling itself:
|
||||
|
||||
- a one-sided positive-expectancy test (the promotion hypothesis is "makes money";
|
||||
losing variants are avoid-knowledge, and letting their tiny two-sided p-values
|
||||
into a multiple-testing correction quietly loosens it for the winners),
|
||||
- overlap thinning (labels from overlapping windows are correlated; a t-stat over
|
||||
them is inflated — test a greedy non-overlapping subset instead),
|
||||
- Benjamini-Hochberg q-values over *every evaluation ever run* (see registry).
|
||||
"""
|
||||
import numpy as np
|
||||
from scipy import stats as sps
|
||||
|
||||
|
||||
def one_sided_positive(pnl):
|
||||
"""(t_stat, p_value) for H1: mean > 0. Degenerate samples -> p = 1."""
|
||||
x = np.asarray(pnl, dtype="float64")
|
||||
if x.size < 2 or np.std(x, ddof=1) == 0:
|
||||
return 0.0, 1.0
|
||||
res = sps.ttest_1samp(x, 0.0, alternative="greater")
|
||||
return float(res.statistic), float(res.pvalue)
|
||||
|
||||
|
||||
def thin_by_gap(entry_time_ns, min_gap_ns):
|
||||
"""Boolean mask keeping a greedy subset of entries at least ``min_gap_ns`` apart."""
|
||||
t = np.asarray(entry_time_ns, dtype="int64")
|
||||
keep = np.zeros(t.size, dtype=bool)
|
||||
last = -np.int64(2**62)
|
||||
for i in range(t.size):
|
||||
if t[i] - last >= min_gap_ns:
|
||||
keep[i] = True
|
||||
last = t[i]
|
||||
return keep
|
||||
|
||||
|
||||
def bh_qvalues(pvals):
|
||||
"""Benjamini-Hochberg adjusted p-values (q-values)."""
|
||||
p = np.asarray(pvals, dtype="float64")
|
||||
n = p.size
|
||||
order = np.argsort(p)
|
||||
q_sorted = p[order] * n / np.arange(1, n + 1)
|
||||
q_sorted = np.minimum.accumulate(q_sorted[::-1])[::-1]
|
||||
q = np.empty(n)
|
||||
q[order] = np.minimum(q_sorted, 1.0)
|
||||
return q
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Leak-proof evaluation splits.
|
||||
|
||||
``purged_expanding_folds`` implements purged walk-forward cross-validation
|
||||
(Lopez de Prado): expanding train windows, chronological test fifths, and a purge
|
||||
that drops any training sample whose *label window* — not just its timestamp —
|
||||
reaches into the test period, plus an embargo margin on top.
|
||||
|
||||
``holdout_cut`` supports the discipline of never touching a final period until one
|
||||
pre-registered confirmation shot.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
|
||||
def holdout_cut(time_ns, holdout_start_ns):
|
||||
"""Index of the first tick inside the holdout; data at [cut:] stays untouched."""
|
||||
return int(np.searchsorted(np.asarray(time_ns), np.int64(holdout_start_ns)))
|
||||
|
||||
|
||||
def purged_expanding_folds(entry_time_ns, horizon_ns, embargo_ns, n_folds=5):
|
||||
"""Yield (train_idx, test_idx) pairs. Fold k tests the k-th time slice and trains
|
||||
on everything before it minus samples whose window (t + horizon + embargo)
|
||||
reaches the test start. The first slice has no prior data and is never a test."""
|
||||
t = np.asarray(entry_time_ns, dtype="int64")
|
||||
lo, hi = t.min(), t.max() + 1
|
||||
edges = lo + (hi - lo) * np.arange(n_folds + 1) // n_folds
|
||||
folds = []
|
||||
for k in range(1, n_folds):
|
||||
test = np.flatnonzero((t >= edges[k]) & (t < edges[k + 1]))
|
||||
train = np.flatnonzero(t + horizon_ns + embargo_ns <= edges[k])
|
||||
if len(test) and len(train):
|
||||
folds.append((train, test))
|
||||
return folds
|
||||
@@ -0,0 +1,134 @@
|
||||
"""Tick storage and bar derivation.
|
||||
|
||||
A tickstore is a directory of flat little-endian arrays (``time_ns.npy``, ``bid.npy``,
|
||||
``ask.npy``) plus a manifest. Loading is zero-copy via ``np.memmap``, and the OS page
|
||||
cache shares physical pages across processes — parallel workers cost no extra RAM
|
||||
for data.
|
||||
|
||||
Bars are derived from ticks on demand. Each bar carries its tick index span
|
||||
(``tick_first``/``tick_last``), which is both an O(1) bar-to-ticks mapping and the
|
||||
no-look-ahead boundary: a bar's values exist only after its last tick has printed.
|
||||
"""
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pyarrow.parquet as pq
|
||||
|
||||
TIMEFRAMES = {"M1": 60, "M5": 300, "M15": 900, "M30": 1800, "H1": 3600,
|
||||
"H4": 14400, "D1": 86400}
|
||||
|
||||
_ARRAYS = {"time_ns": np.int64, "bid": np.float64, "ask": np.float64}
|
||||
|
||||
|
||||
def derive_bars(time_ns, bid, ask, timeframe):
|
||||
"""Bid-based OHLC bars with tick index spans, epoch-aligned like MT5
|
||||
(H4 bars at 00/04/08..., D1 at midnight). Empty buckets produce no bar, so bar
|
||||
times are not calendar-uniform: the row index is positional, and rolling windows
|
||||
silently span session and weekend gaps — same as bars in the MT5 terminal."""
|
||||
tf_ns = TIMEFRAMES[timeframe] * 10**9
|
||||
bucket = time_ns // tf_ns
|
||||
first = np.flatnonzero(np.r_[True, bucket[1:] != bucket[:-1]])
|
||||
last = np.r_[first[1:] - 1, len(bucket) - 1]
|
||||
count = last - first + 1
|
||||
return pd.DataFrame({
|
||||
"time": pd.to_datetime(bucket[first] * tf_ns, unit="ns"),
|
||||
"open": bid[first],
|
||||
"high": np.maximum.reduceat(bid, first),
|
||||
"low": np.minimum.reduceat(bid, first),
|
||||
"close": bid[last],
|
||||
"spread_mean": np.add.reduceat(ask - bid, first) / count,
|
||||
"tick_first": first.astype("int64"),
|
||||
"tick_last": last.astype("int64"),
|
||||
"tick_count": count.astype("int64"),
|
||||
})
|
||||
|
||||
|
||||
class TickStore:
|
||||
"""Memory-mapped tick data for one symbol.
|
||||
|
||||
Build once from parquet files or in-memory arrays, then ``open`` cheaply from
|
||||
any process. ``bars`` results are cached per timeframe.
|
||||
"""
|
||||
|
||||
def __init__(self, time_ns, bid, ask, manifest=None):
|
||||
self.time_ns = time_ns
|
||||
self.bid = bid
|
||||
self.ask = ask
|
||||
self.manifest = manifest or {}
|
||||
self._bars = {}
|
||||
|
||||
def __len__(self):
|
||||
return len(self.time_ns)
|
||||
|
||||
@classmethod
|
||||
def open(cls, store_dir):
|
||||
store_dir = Path(store_dir)
|
||||
arrays = {name: np.load(store_dir / f"{name}.npy", mmap_mode="r")
|
||||
for name in _ARRAYS}
|
||||
manifest = json.loads((store_dir / "manifest.json").read_text())
|
||||
return cls(arrays["time_ns"], arrays["bid"], arrays["ask"], manifest)
|
||||
|
||||
@classmethod
|
||||
def build(cls, parquet_dir, out_dir):
|
||||
"""Concatenate ``ticks_*.parquet`` files (chronological order by filename)
|
||||
into a tickstore. Rejects non-monotonic timestamps."""
|
||||
files = sorted(Path(parquet_dir).glob("ticks_*.parquet"))
|
||||
if not files:
|
||||
raise ValueError(f"no ticks_*.parquet in {parquet_dir}")
|
||||
total = sum(pq.ParquetFile(f).metadata.num_rows for f in files)
|
||||
out_dir = Path(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
maps = {name: np.lib.format.open_memmap(out_dir / f"{name}.npy", mode="w+",
|
||||
dtype=dt, shape=(total,))
|
||||
for name, dt in _ARRAYS.items()}
|
||||
pos = 0
|
||||
last_t = None
|
||||
for f in files:
|
||||
tbl = pq.read_table(f, columns=["time", "bid", "ask"])
|
||||
t = tbl["time"].to_numpy().astype("datetime64[ns]").astype("int64")
|
||||
if (last_t is not None and len(t) and t[0] < last_t) or (np.diff(t) < 0).any():
|
||||
raise ValueError(f"non-monotonic time in {f.name}")
|
||||
maps["time_ns"][pos:pos + len(t)] = t
|
||||
maps["bid"][pos:pos + len(t)] = tbl["bid"].to_numpy()
|
||||
maps["ask"][pos:pos + len(t)] = tbl["ask"].to_numpy()
|
||||
last_t = t[-1] if len(t) else last_t
|
||||
pos += len(t)
|
||||
for m in maps.values():
|
||||
m.flush()
|
||||
manifest = {
|
||||
"n_ticks": int(total),
|
||||
"time_range": [str(np.datetime64(int(maps["time_ns"][0]), "ns")),
|
||||
str(np.datetime64(int(maps["time_ns"][-1]), "ns"))],
|
||||
"sources": [f.name for f in files],
|
||||
}
|
||||
(out_dir / "manifest.json").write_text(json.dumps(manifest, indent=2))
|
||||
return cls.open(out_dir)
|
||||
|
||||
@classmethod
|
||||
def write(cls, out_dir, time_ns, bid, ask, extra_manifest=None):
|
||||
"""Write in-memory arrays as a tickstore (used for synthetic data and tests)."""
|
||||
time_ns = np.ascontiguousarray(time_ns, np.int64)
|
||||
if (np.diff(time_ns) < 0).any():
|
||||
raise ValueError("non-monotonic time")
|
||||
out_dir = Path(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
np.save(out_dir / "time_ns.npy", time_ns)
|
||||
np.save(out_dir / "bid.npy", np.ascontiguousarray(bid, np.float64))
|
||||
np.save(out_dir / "ask.npy", np.ascontiguousarray(ask, np.float64))
|
||||
manifest = {
|
||||
"n_ticks": int(len(time_ns)),
|
||||
"time_range": [str(np.datetime64(int(time_ns[0]), "ns")),
|
||||
str(np.datetime64(int(time_ns[-1]), "ns"))],
|
||||
**(extra_manifest or {}),
|
||||
}
|
||||
(out_dir / "manifest.json").write_text(json.dumps(manifest, indent=2))
|
||||
return cls.open(out_dir)
|
||||
|
||||
def bars(self, timeframe):
|
||||
if timeframe not in self._bars:
|
||||
self._bars[timeframe] = derive_bars(
|
||||
np.asarray(self.time_ns), np.asarray(self.bid),
|
||||
np.asarray(self.ask), timeframe)
|
||||
return self._bars[timeframe]
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Parallel parameter sweeps.
|
||||
|
||||
Threads, not processes: the replay kernel releases the GIL and the tickstore is
|
||||
memory-mapped, so workers share one copy of the data and scale near-linearly with
|
||||
zero IPC. A full parameter grid over years of ticks runs in seconds.
|
||||
"""
|
||||
import os
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from .engine import Config, replay
|
||||
|
||||
|
||||
def run_sweep(evaluate, configs, workers=None):
|
||||
"""Map ``evaluate`` over configs concurrently; returns a DataFrame with one row
|
||||
per config (config keys + whatever dict ``evaluate`` returns)."""
|
||||
workers = workers or max(1, (os.cpu_count() or 2) - 1)
|
||||
with ThreadPoolExecutor(max_workers=workers) as ex:
|
||||
results = list(ex.map(evaluate, configs))
|
||||
return pd.DataFrame([{**c, **r} for c, r in zip(configs, results)])
|
||||
|
||||
|
||||
def sweep_exits(store, bars, signal, grid, base=None, workers=None):
|
||||
"""Sweep exit-rule variants for one signal: ``grid`` is a list of dicts of
|
||||
:class:`Config` field overrides applied on top of ``base``."""
|
||||
base = base or {}
|
||||
|
||||
def evaluate(overrides):
|
||||
return replay(store, bars, signal, Config(**{**base, **overrides})).summary()
|
||||
|
||||
return run_sweep(evaluate, grid, workers=workers)
|
||||
@@ -0,0 +1,44 @@
|
||||
"""Synthetic tick data — lets examples, tests, and benchmarks run without broker data.
|
||||
|
||||
Not a market model: a regime-switching random walk with irregular tick arrivals,
|
||||
session-shaped spread, and weekend gaps. Realistic enough to exercise every engine
|
||||
code path (stops, targets, time exits, quiet periods, spread costs).
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
|
||||
def generate_ticks(days=5, ticks_per_day=50_000, start="2024-01-01", price=2000.0,
|
||||
tick_vol=0.02, spread=0.10, seed=0):
|
||||
"""Returns (time_ns, bid, ask). Weekdays only; ~22h of trading per day."""
|
||||
rng = np.random.default_rng(seed)
|
||||
day0 = np.datetime64(start).astype("datetime64[D]")
|
||||
times, mids, spreads = [], [], []
|
||||
mid = float(price)
|
||||
for d in range(days):
|
||||
day = day0 + d
|
||||
if np.datetime64(day).astype("datetime64[D]").item().weekday() >= 5:
|
||||
continue
|
||||
# irregular arrivals over 22 hours
|
||||
gaps = rng.exponential(22 * 3600 / ticks_per_day, ticks_per_day)
|
||||
t = np.cumsum(gaps)
|
||||
t = t[t < 22 * 3600]
|
||||
# two-state volatility regime (calm/wild), persistent at tick scale
|
||||
switch = rng.random(len(t)) < 0.0005
|
||||
regime = np.bitwise_xor.accumulate(switch)
|
||||
vol = np.where(regime, 4.0 * tick_vol, tick_vol)
|
||||
steps = rng.normal(0.0, 1.0, len(t)) * vol
|
||||
path = mid + np.cumsum(steps)
|
||||
mid = float(path[-1])
|
||||
# spread widens outside core hours and in the wild regime
|
||||
hour = t / 3600
|
||||
session = 1.0 + 0.8 * ((hour < 1) | (hour > 21)) + 0.5 * regime
|
||||
spreads.append(spread * session * rng.uniform(0.8, 1.2, len(t)))
|
||||
times.append(day.astype("datetime64[ns]").astype("int64")
|
||||
+ (t * 1e9).astype("int64"))
|
||||
mids.append(path)
|
||||
if not times:
|
||||
raise ValueError(f"no weekdays in the requested window ({start} + {days} days)")
|
||||
time_ns = np.concatenate(times)
|
||||
mid_all = np.concatenate(mids)
|
||||
half = np.concatenate(spreads) / 2
|
||||
return time_ns, mid_all - half, mid_all + half
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Golden-reference replay engine: a deliberately plain tick-by-tick Python loop.
|
||||
|
||||
This is the correctness anchor the numba kernel is judged against in
|
||||
test_engine.py. Slow on purpose — clarity over speed. Both implementations were
|
||||
written independently against the same semantics; the tests require them to agree
|
||||
trade-for-trade, byte-for-byte.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
_COLUMNS = ("dir", "entry_idx", "exit_idx", "entry_time_ns", "exit_time_ns",
|
||||
"entry_price", "exit_price", "pnl_gross", "pnl_net", "mfe", "mae",
|
||||
"exit_reason")
|
||||
_DTYPES = {"dir": "int8", "entry_idx": "int64", "exit_idx": "int64",
|
||||
"entry_time_ns": "int64", "exit_time_ns": "int64",
|
||||
"entry_price": "float64", "exit_price": "float64",
|
||||
"pnl_gross": "float64", "pnl_net": "float64",
|
||||
"mfe": "float64", "mae": "float64", "exit_reason": "int8"}
|
||||
|
||||
|
||||
def replay(time_ns, bid, ask, bar_tick_last, bar_signal, sl_dist, tp_dist,
|
||||
allow_buy=True, allow_sell=True, max_hold_ns=0, exit_on_opposite=False,
|
||||
cost=0.0):
|
||||
n_ticks = len(time_ns)
|
||||
|
||||
# Scatter each bar's signal onto its entry tick for O(1) lookup in the loop.
|
||||
# A bar whose entry tick falls beyond the data is dropped (no-trade rule).
|
||||
entry_signal = np.zeros(n_ticks, dtype="int8")
|
||||
entry_sl = np.zeros(n_ticks, dtype="float64")
|
||||
entry_tp = np.zeros(n_ticks, dtype="float64")
|
||||
for b in range(len(bar_tick_last)):
|
||||
if bar_signal[b] == 0:
|
||||
continue
|
||||
i = bar_tick_last[b] + 1
|
||||
if i >= n_ticks:
|
||||
continue
|
||||
entry_signal[i] = bar_signal[b]
|
||||
entry_sl[i] = sl_dist[b]
|
||||
entry_tp[i] = tp_dist[b]
|
||||
|
||||
rows = {col: [] for col in _COLUMNS}
|
||||
pos_dir = 0 # 0 = flat, +1 = long, -1 = short
|
||||
entry_idx = entry_time = entry_price = None
|
||||
locked_sl = locked_tp = 0.0
|
||||
mfe = mae = 0.0
|
||||
|
||||
def open_trade(i, direction, sl, tp):
|
||||
nonlocal pos_dir, entry_idx, entry_time, entry_price, locked_sl, locked_tp, mfe, mae
|
||||
pos_dir = direction
|
||||
entry_idx = i
|
||||
entry_time = time_ns[i]
|
||||
entry_price = ask[i] if direction == 1 else bid[i]
|
||||
locked_sl, locked_tp = sl, tp
|
||||
mfe, mae = 0.0, 0.0
|
||||
|
||||
def close_trade(i, reason):
|
||||
nonlocal pos_dir
|
||||
exit_price = bid[i] if pos_dir == 1 else ask[i]
|
||||
pnl = (exit_price - entry_price) if pos_dir == 1 else (entry_price - exit_price)
|
||||
rows["dir"].append(pos_dir)
|
||||
rows["entry_idx"].append(entry_idx)
|
||||
rows["exit_idx"].append(i)
|
||||
rows["entry_time_ns"].append(entry_time)
|
||||
rows["exit_time_ns"].append(time_ns[i])
|
||||
rows["entry_price"].append(entry_price)
|
||||
rows["exit_price"].append(exit_price)
|
||||
rows["pnl_gross"].append(pnl)
|
||||
rows["pnl_net"].append(pnl - cost)
|
||||
rows["mfe"].append(mfe)
|
||||
rows["mae"].append(mae)
|
||||
rows["exit_reason"].append(reason)
|
||||
pos_dir = 0
|
||||
|
||||
for i in range(n_ticks):
|
||||
if pos_dir != 0:
|
||||
pnl_raw = (bid[i] - entry_price) if pos_dir == 1 else (entry_price - ask[i])
|
||||
mfe = max(mfe, pnl_raw)
|
||||
mae = max(mae, -pnl_raw)
|
||||
|
||||
exited = True
|
||||
if locked_sl > 0 and pnl_raw <= -locked_sl:
|
||||
close_trade(i, 1)
|
||||
elif locked_tp > 0 and pnl_raw >= locked_tp:
|
||||
close_trade(i, 2)
|
||||
elif max_hold_ns > 0 and time_ns[i] >= entry_time + max_hold_ns:
|
||||
close_trade(i, 3)
|
||||
elif exit_on_opposite and entry_signal[i] != 0 and entry_signal[i] == -pos_dir:
|
||||
close_trade(i, 4)
|
||||
else:
|
||||
exited = False
|
||||
|
||||
if exited:
|
||||
continue # the exit tick never also opens a trade -- signals don't queue
|
||||
|
||||
# an entry on the final tick (nothing after it to evaluate) is no trade;
|
||||
# the scattered signal still counts for opposite-exit detection above
|
||||
if pos_dir == 0 and i < n_ticks - 1:
|
||||
sig = entry_signal[i]
|
||||
if sig == 1 and allow_buy:
|
||||
open_trade(i, 1, entry_sl[i], entry_tp[i])
|
||||
elif sig == -1 and allow_sell:
|
||||
open_trade(i, -1, entry_sl[i], entry_tp[i])
|
||||
|
||||
if pos_dir != 0:
|
||||
close_trade(n_ticks - 1, 5)
|
||||
|
||||
return {col: np.array(rows[col], dtype=_DTYPES[col]) for col in _COLUMNS}
|
||||
@@ -0,0 +1,67 @@
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from tickreplay.research.bocpd import bocpd_features
|
||||
|
||||
HAZARD = 1 / 250
|
||||
|
||||
|
||||
def synth(seed=3):
|
||||
"""Three regimes: a mean shift at 300 and a volatility shift at 600."""
|
||||
rng = np.random.default_rng(seed)
|
||||
return np.r_[rng.normal(0, 1, 300), rng.normal(4, 1, 300), rng.normal(4, 5, 300)]
|
||||
|
||||
|
||||
def test_detects_planted_mean_and_volatility_shifts():
|
||||
p = bocpd_features(synth(), hazard=HAZARD)["p_cp_10"]
|
||||
|
||||
assert p[302:310].max() > 0.5
|
||||
assert p[602:612].max() > 0.5
|
||||
assert p[200:290].mean() < 0.1
|
||||
assert p[500:590].mean() < 0.1
|
||||
|
||||
|
||||
def test_expected_run_length_collapses_at_a_break():
|
||||
erl = bocpd_features(synth(), hazard=HAZARD)["erl"]
|
||||
|
||||
assert erl[290] > 100
|
||||
assert erl[310] < erl[290] / 2
|
||||
assert erl[590] > 100
|
||||
assert erl[610] < erl[590] / 2
|
||||
|
||||
|
||||
def test_run_length_grows_inside_a_stable_segment():
|
||||
erl = bocpd_features(synth(), hazard=HAZARD)["erl"]
|
||||
|
||||
assert erl[100] < erl[200] < erl[290]
|
||||
|
||||
|
||||
def test_is_strictly_causal():
|
||||
x = synth()
|
||||
baseline = bocpd_features(x, hazard=HAZARD)
|
||||
mutated = x.copy()
|
||||
mutated[400:] = 99.0
|
||||
after = bocpd_features(mutated, hazard=HAZARD)
|
||||
|
||||
np.testing.assert_array_equal(baseline["p_cp_10"][:400], after["p_cp_10"][:400])
|
||||
np.testing.assert_array_equal(baseline["erl"][:400], after["erl"][:400])
|
||||
|
||||
|
||||
def test_outputs_are_valid_probabilities_and_bounded_run_lengths():
|
||||
out = bocpd_features(synth(), hazard=HAZARD)
|
||||
p, erl = out["p_cp_10"], out["erl"]
|
||||
|
||||
assert p.shape == erl.shape == (900,)
|
||||
assert ((p >= 0) & (p <= 1)).all()
|
||||
assert (erl >= 0).all()
|
||||
assert (erl <= np.arange(1, len(erl) + 1)).all()
|
||||
assert np.isfinite(p).all()
|
||||
assert np.isfinite(erl).all()
|
||||
|
||||
|
||||
def test_first_observation_is_always_a_fresh_run():
|
||||
out = bocpd_features(np.array([0.0, 0.1, -0.2, 0.05]), hazard=HAZARD)
|
||||
|
||||
assert out["p_cp_10"][0] == 1.0
|
||||
# a single observation can only be run length 0 or 1, weighted by the hazard
|
||||
assert out["erl"][0] == pytest.approx(1 - HAZARD)
|
||||
@@ -0,0 +1,261 @@
|
||||
"""The replay kernel must agree with the golden reference, trade for trade."""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
import reference
|
||||
import tickreplay as tr
|
||||
|
||||
LEDGER = ("dir", "entry_idx", "exit_idx", "entry_time_ns", "exit_time_ns",
|
||||
"entry_price", "exit_price", "pnl_gross", "pnl_net", "mfe", "mae",
|
||||
"exit_reason")
|
||||
|
||||
CONFIGS = {
|
||||
"default": dict(),
|
||||
"max_hold": dict(max_hold=120),
|
||||
"exit_on_opposite": dict(exit_on_opposite=True),
|
||||
"combined_with_cost": dict(max_hold=300, exit_on_opposite=True, cost=0.09),
|
||||
"buy_only": dict(allow_sell=False),
|
||||
"sell_only": dict(allow_buy=False, max_hold=60),
|
||||
}
|
||||
|
||||
|
||||
def synthetic(seed, n_ticks=30_000, n_bars=500):
|
||||
"""Irregular ticks, sparse signals, per-bar stops with some disabled (0)."""
|
||||
rng = np.random.default_rng(seed)
|
||||
time_ns = np.cumsum(rng.integers(10**8, 3 * 10**9, n_ticks)).astype("int64")
|
||||
bid = 4000 + np.cumsum(rng.normal(0, 0.05, n_ticks))
|
||||
ask = bid + rng.uniform(0.05, 0.2, n_ticks)
|
||||
tick_last = np.sort(rng.choice(np.arange(1, n_ticks - 1), n_bars, replace=False))
|
||||
signal = np.zeros(n_bars, dtype="int8")
|
||||
hot = rng.choice(n_bars, n_bars // 10, replace=False)
|
||||
signal[hot] = rng.choice(np.array([-1, 1], dtype="int8"), hot.size)
|
||||
sl = rng.uniform(0.2, 0.8, n_bars)
|
||||
tp = rng.uniform(0.2, 1.0, n_bars)
|
||||
sl[rng.random(n_bars) < 0.3] = 0.0
|
||||
tp[rng.random(n_bars) < 0.3] = 0.0
|
||||
# a traded bar with no stop can hold to the end of data under the stopless default
|
||||
# config and starve every later signal, so keep one side live on those bars
|
||||
sl[(signal != 0) & (sl == 0)] = 0.5
|
||||
store = tr.TickStore(time_ns, bid, ask)
|
||||
bars = pd.DataFrame({"tick_last": tick_last.astype("int64")})
|
||||
return store, bars, signal, sl, tp
|
||||
|
||||
|
||||
def five_exit_reasons():
|
||||
"""Hand-built ticks that fire each exit reason exactly once, in order.
|
||||
|
||||
Ticks are 1s apart until index 104, then 0.1s apart so the final trade runs out
|
||||
of data before it runs out of max_hold.
|
||||
"""
|
||||
n = 120
|
||||
time_ns = np.arange(n, dtype="int64") * 10**9
|
||||
time_ns[105:] = time_ns[104] + np.arange(1, n - 104, dtype="int64") * 10**8
|
||||
bid = np.full(n, 100.0)
|
||||
bid[7] = 99.4 # stop touch for the trade entered at tick 5
|
||||
bid[27] = 100.8 # target touch for the trade entered at tick 25
|
||||
store = tr.TickStore(time_ns, bid, bid + 0.1)
|
||||
bars = pd.DataFrame({"tick_last": np.array([4, 24, 44, 64, 68, 104], dtype="int64")})
|
||||
signal = np.array([1, 1, 1, 1, -1, 1], dtype="int8")
|
||||
sl = np.array([0.5, 0.5, 0.0, 0.0, 0.0, 0.0])
|
||||
tp = np.array([0.5, 0.5, 0.0, 0.0, 0.0, 0.0])
|
||||
return store, bars, signal, sl, tp
|
||||
|
||||
|
||||
def run_reference(store, bars, signal, config):
|
||||
return reference.replay(
|
||||
np.asarray(store.time_ns), np.asarray(store.bid), np.asarray(store.ask),
|
||||
bars["tick_last"].to_numpy(), signal,
|
||||
np.broadcast_to(config.sl, len(bars)), np.broadcast_to(config.tp, len(bars)),
|
||||
allow_buy=config.allow_buy, allow_sell=config.allow_sell,
|
||||
max_hold_ns=config.max_hold_ns(), exit_on_opposite=config.exit_on_opposite,
|
||||
cost=config.cost)
|
||||
|
||||
|
||||
def assert_ledgers_equal(a, b):
|
||||
assert len(a["dir"]) == len(b["dir"]), "trade count"
|
||||
for col in LEDGER:
|
||||
np.testing.assert_array_equal(a[col], b[col], err_msg=col)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("seed", range(5))
|
||||
@pytest.mark.parametrize("overrides", CONFIGS.values(), ids=CONFIGS)
|
||||
def test_fast_engine_matches_golden_reference(seed, overrides):
|
||||
store, bars, signal, sl, tp = synthetic(seed)
|
||||
config = tr.Config(sl=sl, tp=tp, **overrides)
|
||||
fast = tr.replay(store, bars, signal, config).arrays
|
||||
assert len(fast["dir"]) > 0, "fixture produced no trades to compare"
|
||||
assert_ledgers_equal(fast, run_reference(store, bars, signal, config))
|
||||
|
||||
|
||||
def test_golden_matrix_is_not_vacuous():
|
||||
"""Guards the test above: the fixtures must trade a lot and hit every exit path."""
|
||||
reasons, trades = set(), 0
|
||||
for overrides in CONFIGS.values():
|
||||
for seed in range(5):
|
||||
store, bars, signal, sl, tp = synthetic(seed)
|
||||
arrays = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=sl, tp=tp, **overrides)).arrays
|
||||
reasons |= set(arrays["exit_reason"].tolist())
|
||||
trades += len(arrays["dir"])
|
||||
assert trades > 500
|
||||
assert reasons == {1, 2, 3, 4, 5}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("side, overrides", [(1, CONFIGS["buy_only"]),
|
||||
(-1, CONFIGS["sell_only"])])
|
||||
def test_direction_filters_take_only_their_own_side(side, overrides):
|
||||
store, bars, signal, sl, tp = synthetic(0)
|
||||
arrays = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=sl, tp=tp, **overrides)).arrays
|
||||
assert len(arrays["dir"]) > 3
|
||||
assert set(arrays["dir"].tolist()) == {side}
|
||||
|
||||
|
||||
def test_fast_engine_matches_golden_reference_on_every_exit_reason():
|
||||
store, bars, signal, sl, tp = five_exit_reasons()
|
||||
config = tr.Config(sl=sl, tp=tp, max_hold=12, exit_on_opposite=True)
|
||||
fast = tr.replay(store, bars, signal, config).arrays
|
||||
assert list(fast["exit_reason"]) == [1, 2, 3, 4, 5]
|
||||
assert_ledgers_equal(fast, run_reference(store, bars, signal, config))
|
||||
|
||||
|
||||
def test_replay_is_deterministic():
|
||||
store, bars, signal, sl, tp = synthetic(9)
|
||||
config = tr.Config(sl=sl, tp=tp, max_hold=300, exit_on_opposite=True, cost=0.09)
|
||||
first = tr.replay(store, bars, signal, config).arrays
|
||||
second = tr.replay(store, bars, signal, config).arrays
|
||||
assert len(first["dir"]) > 3
|
||||
assert_ledgers_equal(first, second)
|
||||
|
||||
|
||||
def test_trades_closed_before_a_cut_do_not_depend_on_later_data():
|
||||
store, bars, signal, sl, tp = synthetic(3)
|
||||
overrides = dict(max_hold=100, exit_on_opposite=True, cost=0.09)
|
||||
full = tr.replay(store, bars, signal, tr.Config(sl=sl, tp=tp, **overrides)).arrays
|
||||
|
||||
cut = len(store) // 2
|
||||
n_bars = int(np.searchsorted(bars["tick_last"].to_numpy(), cut - 1))
|
||||
truncated = tr.replay(
|
||||
tr.TickStore(store.time_ns[:cut], store.bid[:cut], store.ask[:cut]),
|
||||
bars.iloc[:n_bars], signal[:n_bars],
|
||||
tr.Config(sl=sl[:n_bars], tp=tp[:n_bars], **overrides)).arrays
|
||||
|
||||
closed = full["exit_idx"] < cut - 1
|
||||
assert closed.sum() > 5
|
||||
for col in LEDGER:
|
||||
np.testing.assert_array_equal(truncated[col][:closed.sum()], full[col][closed],
|
||||
err_msg=col)
|
||||
|
||||
|
||||
def test_scalar_stops_broadcast_to_every_bar():
|
||||
store, bars, signal, _, _ = synthetic(7)
|
||||
n = len(bars)
|
||||
scalar = tr.replay(store, bars, signal, tr.Config(sl=1.5, tp=2.5)).arrays
|
||||
per_bar = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=np.full(n, 1.5), tp=np.full(n, 2.5))).arrays
|
||||
assert len(scalar["dir"]) > 3
|
||||
assert_ledgers_equal(scalar, per_bar)
|
||||
|
||||
|
||||
def test_signal_length_mismatch_raises():
|
||||
store, bars, signal, _, _ = synthetic(1)
|
||||
with pytest.raises(ValueError, match=r"499 values for 500 bars"):
|
||||
tr.replay(store, bars, signal[:-1])
|
||||
|
||||
|
||||
def test_non_increasing_tick_last_raises():
|
||||
# duplicate tick_last would let two bars share an entry tick, where the fast and
|
||||
# reference engines resolve opposite-signal exits differently -- reject it outright
|
||||
store, bars, signal, _, _ = synthetic(1)
|
||||
bad = bars.copy()
|
||||
bad.loc[bad.index[10], "tick_last"] = bad["tick_last"].iloc[9]
|
||||
with pytest.raises(ValueError, match="strictly increasing"):
|
||||
tr.replay(store, bad, signal)
|
||||
|
||||
|
||||
def test_trades_frame_exposes_datetimes_and_named_exit_reasons():
|
||||
store, bars, signal, sl, tp = five_exit_reasons()
|
||||
result = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=sl, tp=tp, max_hold=12, exit_on_opposite=True))
|
||||
df = result.trades
|
||||
|
||||
assert len(df) == len(result) == 5
|
||||
assert df["entry_time"].dtype == np.dtype("datetime64[ns]")
|
||||
assert df["exit_time"].dtype == np.dtype("datetime64[ns]")
|
||||
assert list(df["exit_reason"]) == ["sl", "tp", "time", "opposite", "eod"]
|
||||
assert not {"entry_time_ns", "exit_time_ns"} & set(df.columns)
|
||||
np.testing.assert_array_equal(df["entry_time"].astype("int64").to_numpy(),
|
||||
result.arrays["entry_time_ns"])
|
||||
np.testing.assert_array_equal(df["exit_time"].astype("int64").to_numpy(),
|
||||
result.arrays["exit_time_ns"])
|
||||
assert df["pnl_gross"].to_numpy() == pytest.approx([-0.7, 0.7, -0.1, -0.1, -0.1])
|
||||
|
||||
|
||||
def test_summary_agrees_with_the_ledger():
|
||||
store, bars, signal, sl, tp = synthetic(2)
|
||||
result = tr.replay(store, bars, signal,
|
||||
tr.Config(sl=sl, tp=tp, max_hold=300, cost=0.09))
|
||||
pnl = result.arrays["pnl_net"]
|
||||
summary = result.summary()
|
||||
|
||||
assert set(summary) == {"n_trades", "total_net", "mean_net", "hit_rate",
|
||||
"max_drawdown"}
|
||||
assert summary["n_trades"] == len(pnl) == len(result)
|
||||
assert summary["total_net"] == pytest.approx(pnl.sum())
|
||||
assert summary["mean_net"] == pytest.approx(pnl.mean())
|
||||
assert summary["hit_rate"] == pytest.approx((pnl > 0).sum() / len(pnl))
|
||||
assert 0 < summary["hit_rate"] < 1
|
||||
|
||||
equity = peak = drawdown = 0.0
|
||||
for i, x in enumerate(pnl):
|
||||
equity += x
|
||||
peak = equity if i == 0 else max(peak, equity)
|
||||
drawdown = max(drawdown, peak - equity)
|
||||
assert summary["max_drawdown"] == pytest.approx(drawdown)
|
||||
|
||||
|
||||
def test_max_hold_accepts_sub_second_and_offset_string_forms():
|
||||
n = 20
|
||||
price = np.full(n, 100.0)
|
||||
store = tr.TickStore(np.arange(n, dtype="int64") * 10**8, price, price + 0.1)
|
||||
bars = pd.DataFrame({"tick_last": np.array([4], dtype="int64")})
|
||||
signal = np.array([1], dtype="int8") # enters at tick 5, i.e. t = 0.5s
|
||||
|
||||
for max_hold in (0.25, "250ms"):
|
||||
result = tr.replay(store, bars, signal, tr.Config(max_hold=max_hold))
|
||||
assert result.arrays["exit_idx"].tolist() == [8], max_hold
|
||||
assert list(result.trades["exit_reason"]) == ["time"], max_hold
|
||||
|
||||
|
||||
def test_signal_entering_on_the_last_tick_produces_no_trade():
|
||||
n = 20
|
||||
price = np.full(n, 100.0)
|
||||
store = tr.TickStore(np.arange(n, dtype="int64") * 10**9, price, price + 0.1)
|
||||
signal = np.array([1], dtype="int8")
|
||||
|
||||
entering_on_last = pd.DataFrame({"tick_last": np.array([n - 2], dtype="int64")})
|
||||
entering_one_earlier = pd.DataFrame({"tick_last": np.array([n - 3], dtype="int64")})
|
||||
|
||||
assert len(tr.replay(store, entering_on_last, signal)) == 0
|
||||
assert len(tr.replay(store, entering_one_earlier, signal)) == 1
|
||||
|
||||
|
||||
def test_breakeven_trade_is_not_counted_as_a_win():
|
||||
n = 30
|
||||
price = np.full(n, 100.0)
|
||||
store = tr.TickStore(np.arange(n, dtype="int64") * 10**9, price, price)
|
||||
bars = pd.DataFrame({"tick_last": np.array([4], dtype="int64")})
|
||||
|
||||
result = tr.replay(store, bars, np.array([1], dtype="int8"), tr.Config(max_hold=10))
|
||||
|
||||
assert result.arrays["pnl_net"].tolist() == [0.0]
|
||||
assert result.summary()["hit_rate"] == 0.0
|
||||
|
||||
|
||||
def test_summary_of_an_empty_ledger():
|
||||
store, bars, _, _, _ = synthetic(4)
|
||||
result = tr.replay(store, bars, np.zeros(len(bars), dtype="int8"))
|
||||
assert len(result) == 0
|
||||
assert result.summary() == {"n_trades": 0, "total_net": 0.0, "mean_net": 0.0,
|
||||
"hit_rate": 0.0, "max_drawdown": 0.0}
|
||||
@@ -0,0 +1,154 @@
|
||||
"""MT5 export ingestion: quote forward-fill, monthly parquet split, M1 reconstruction."""
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
from tickreplay.ingest import (read_mt5_ticks, reconstruct_m1, ticks_to_monthly_parquet,
|
||||
validate_against_broker_bars)
|
||||
|
||||
HEADER = "<DATE>\t<TIME>\t<BID>\t<ASK>\t<LAST>\t<VOLUME>\t<FLAGS>"
|
||||
|
||||
# (date, time, bid, ask) — "" means the row carries only the other quote side.
|
||||
ONE_SIDED_ROWS = [
|
||||
("2024.01.15", "10:30:00.000", "", ""),
|
||||
("2024.01.15", "10:30:00.100", "2000.10", ""),
|
||||
("2024.01.15", "10:30:00.200", "", "2000.30"),
|
||||
("2024.01.15", "10:30:00.300", "", ""),
|
||||
("2024.01.15", "10:30:00.400", "2000.50", ""),
|
||||
("2024.01.15", "10:30:00.500", "", "2000.70"),
|
||||
]
|
||||
|
||||
|
||||
def _write_tick_csv(path, rows):
|
||||
lines = [HEADER] + [f"{d}\t{t}\t{bid}\t{ask}\t0\t0\t2" for d, t, bid, ask in rows]
|
||||
path.write_text("\n".join(lines) + "\n")
|
||||
return path
|
||||
|
||||
|
||||
def _day_rows(date, n, base):
|
||||
return [(date, f"10:{m:02d}:00.000", f"{base + m * 0.1:.2f}",
|
||||
f"{base + m * 0.1 + 0.03:.2f}") for m in range(n)]
|
||||
|
||||
|
||||
def _ticks():
|
||||
return pd.DataFrame({
|
||||
"time": pd.to_datetime([
|
||||
"2024-01-15 10:30:00.100", "2024-01-15 10:30:20.000",
|
||||
"2024-01-15 10:30:40.000", "2024-01-15 10:30:59.900",
|
||||
"2024-01-15 10:31:05.000", "2024-01-15 10:31:50.000",
|
||||
"2024-01-15 10:33:10.000", "2024-01-15 10:34:00.000",
|
||||
]),
|
||||
"bid": [2000.0, 2001.5, 1999.5, 2000.5, 2000.6, 2000.9, 2002.0, 2003.25],
|
||||
"ask": [2000.03, 2001.53, 1999.53, 2000.53, 2000.63, 2000.93, 2002.03, 2003.28],
|
||||
})
|
||||
|
||||
|
||||
def _broker_bars():
|
||||
"""Hand-computed M1 bars for _ticks(), plus a 10:32 bar the ticks do not cover."""
|
||||
return pd.DataFrame(
|
||||
{"open": [2000.00, 2000.6, 2001.0, 2002.0],
|
||||
"high": [2001.50, 2000.9, 2001.2, 2002.0],
|
||||
"low": [1999.50, 2000.6, 2000.8, 2002.0],
|
||||
"close": [2000.50, 2000.9, 2001.1, 2002.0],
|
||||
"tickvol": [4, 2, 3, 1],
|
||||
"spread": [3, 3, 3, 3]},
|
||||
index=pd.DatetimeIndex(["2024-01-15 10:30", "2024-01-15 10:31",
|
||||
"2024-01-15 10:32", "2024-01-15 10:33"], name="time"))
|
||||
|
||||
|
||||
def test_read_mt5_ticks_forward_fills_missing_quote_side(tmp_path):
|
||||
src = _write_tick_csv(tmp_path / "ticks.csv", ONE_SIDED_ROWS)
|
||||
df = pd.concat(read_mt5_ticks(src), ignore_index=True)
|
||||
|
||||
assert list(df.columns) == ["time", "bid", "ask"]
|
||||
expected_times = pd.to_datetime([
|
||||
"2024-01-15 10:30:00.200", "2024-01-15 10:30:00.300",
|
||||
"2024-01-15 10:30:00.400", "2024-01-15 10:30:00.500"])
|
||||
assert np.array_equal(df["time"].to_numpy(), expected_times.to_numpy())
|
||||
assert df["bid"].tolist() == [2000.10, 2000.10, 2000.50, 2000.50]
|
||||
assert df["ask"].tolist() == [2000.30, 2000.30, 2000.30, 2000.70]
|
||||
|
||||
|
||||
def test_read_mt5_ticks_carries_quote_state_across_chunk_boundaries(tmp_path):
|
||||
src = _write_tick_csv(tmp_path / "ticks.csv", ONE_SIDED_ROWS)
|
||||
whole = pd.concat(read_mt5_ticks(src), ignore_index=True)
|
||||
chunks = list(read_mt5_ticks(src, chunksize=3))
|
||||
|
||||
assert len(chunks) == 2
|
||||
chunked = pd.concat(chunks, ignore_index=True)
|
||||
# The first row of chunk 2 is only keepable because both sides carried over.
|
||||
assert len(chunked) == 4
|
||||
assert chunked["bid"].iloc[1] == 2000.10
|
||||
assert chunked["ask"].iloc[1] == 2000.30
|
||||
pd.testing.assert_frame_equal(chunked, whole)
|
||||
|
||||
|
||||
def test_ticks_to_monthly_parquet_splits_rows_by_month(tmp_path):
|
||||
rows = (_day_rows("2024.01.15", 5, 2000.0) + _day_rows("2024.01.20", 4, 2010.0)
|
||||
+ _day_rows("2024.02.03", 6, 2020.0) + _day_rows("2024.03.02", 3, 2030.0))
|
||||
src = _write_tick_csv(tmp_path / "ticks.csv", rows)
|
||||
out_dir = tmp_path / "parquet"
|
||||
|
||||
manifest = ticks_to_monthly_parquet(src, out_dir, chunksize=4)
|
||||
|
||||
assert sorted(p.name for p in out_dir.glob("*.parquet")) == [
|
||||
"ticks_2024-01.parquet", "ticks_2024-02.parquet", "ticks_2024-03.parquet"]
|
||||
assert manifest["months"] == {"2024-01": 9, "2024-02": 6, "2024-03": 3}
|
||||
assert manifest["total_ticks"] == len(rows) == 18
|
||||
assert manifest["range"] == ["2024-01-15 10:00:00", "2024-03-02 10:02:00"]
|
||||
assert json.loads((out_dir / "manifest.json").read_text()) == manifest
|
||||
|
||||
for month in manifest["months"]:
|
||||
part = pd.read_parquet(out_dir / f"ticks_{month}.parquet")
|
||||
assert (part["time"].dt.to_period("M").astype(str) == month).all()
|
||||
|
||||
combined = pd.concat([pd.read_parquet(out_dir / f"ticks_{m}.parquet")
|
||||
for m in sorted(manifest["months"])], ignore_index=True)
|
||||
pd.testing.assert_frame_equal(combined, pd.concat(read_mt5_ticks(src), ignore_index=True))
|
||||
|
||||
|
||||
def test_ticks_to_monthly_parquet_rejects_non_chronological_input(tmp_path):
|
||||
rows = (_day_rows("2024.01.15", 4, 2000.0) + _day_rows("2024.02.03", 4, 2020.0)
|
||||
+ _day_rows("2024.01.20", 4, 2010.0))
|
||||
src = _write_tick_csv(tmp_path / "ticks.csv", rows)
|
||||
|
||||
with pytest.raises(RuntimeError, match="non-chronological"):
|
||||
ticks_to_monthly_parquet(src, tmp_path / "parquet", chunksize=4)
|
||||
|
||||
|
||||
def test_reconstruct_m1_matches_hand_computed_bars():
|
||||
bars = reconstruct_m1(_ticks())
|
||||
|
||||
assert list(bars.columns) == ["open", "high", "low", "close"]
|
||||
expected_index = pd.to_datetime(["2024-01-15 10:30", "2024-01-15 10:31",
|
||||
"2024-01-15 10:33", "2024-01-15 10:34"])
|
||||
assert np.array_equal(bars.index.to_numpy(), expected_index.to_numpy())
|
||||
assert np.array_equal(bars.to_numpy(), np.array([
|
||||
[2000.00, 2001.50, 1999.50, 2000.50],
|
||||
[2000.60, 2000.90, 2000.60, 2000.90],
|
||||
[2002.00, 2002.00, 2002.00, 2002.00],
|
||||
[2003.25, 2003.25, 2003.25, 2003.25]]))
|
||||
|
||||
|
||||
def test_validate_against_broker_bars_reports_no_mismatch_for_clean_export():
|
||||
result = validate_against_broker_bars(_ticks(), _broker_bars())
|
||||
|
||||
assert result["compared"] == 3
|
||||
assert result["ohlc_mismatches"] == 0
|
||||
assert result["mismatch_sample"] == []
|
||||
assert result["broker_only"] == 1
|
||||
assert result["recon_only"] == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("column", ["open", "high", "low", "close"])
|
||||
def test_validate_against_broker_bars_flags_a_corrupted_bar(column):
|
||||
broker = _broker_bars()
|
||||
broker.loc[pd.Timestamp("2024-01-15 10:31"), column] = 9999.0
|
||||
|
||||
result = validate_against_broker_bars(_ticks(), broker)
|
||||
|
||||
assert result["compared"] == 3
|
||||
assert result["ohlc_mismatches"] == 1
|
||||
assert result["mismatch_sample"] == ["2024-01-15 10:31:00"]
|
||||
@@ -0,0 +1,178 @@
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from tickreplay.research.labels import excursions, first_passage
|
||||
|
||||
|
||||
def random_ticks(seed, n=5000):
|
||||
rng = np.random.default_rng(seed)
|
||||
time_ns = np.cumsum(rng.integers(10**8, 2 * 10**9, n)).astype("int64")
|
||||
bid = 100 + np.cumsum(rng.normal(0, 0.05, n))
|
||||
ask = bid + rng.uniform(0.01, 0.05, n)
|
||||
return time_ns, bid, ask
|
||||
|
||||
|
||||
def oracle_excursions(time_ns, bid, ask, entry_tick, d, checkpoints_ns):
|
||||
"""Running MFE/MAE and signed close-out return at each checkpoint after entry."""
|
||||
entry_price = ask[entry_tick] if d == 1 else bid[entry_tick]
|
||||
t0 = time_ns[entry_tick]
|
||||
out = []
|
||||
mfe = mae = last_pnl = 0.0
|
||||
j = entry_tick + 1
|
||||
for cp in checkpoints_ns:
|
||||
while j < len(time_ns) and time_ns[j] <= t0 + cp:
|
||||
last_pnl = bid[j] - entry_price if d == 1 else entry_price - ask[j]
|
||||
mfe = max(mfe, last_pnl)
|
||||
mae = max(mae, -last_pnl)
|
||||
j += 1
|
||||
out.append((mfe, mae, last_pnl))
|
||||
return entry_price, out
|
||||
|
||||
|
||||
def oracle_first_passage(time_ns, bid, ask, entry_tick, d, target, stop, horizon_ns):
|
||||
entry_price = ask[entry_tick] if d == 1 else bid[entry_tick]
|
||||
t0 = time_ns[entry_tick]
|
||||
pnl = 0.0
|
||||
j = entry_tick + 1
|
||||
while j < len(time_ns) and time_ns[j] <= t0 + horizon_ns:
|
||||
pnl = bid[j] - entry_price if d == 1 else entry_price - ask[j]
|
||||
if pnl >= target:
|
||||
return 1, pnl, time_ns[j]
|
||||
if -pnl >= stop:
|
||||
return -1, pnl, time_ns[j]
|
||||
j += 1
|
||||
return 0, pnl, min(t0 + horizon_ns, time_ns[-1])
|
||||
|
||||
|
||||
def test_excursions_match_bruteforce_oracle():
|
||||
time_ns, bid, ask = random_ticks(9)
|
||||
rng = np.random.default_rng(9)
|
||||
entries = np.sort(rng.choice(np.arange(1, len(time_ns) - 500), 40, replace=False))
|
||||
entries = entries.astype("int64")
|
||||
dirs = rng.choice(np.array([-1, 1], dtype="int8"), 40)
|
||||
cps = np.array([60, 300, 900], dtype="int64") * 10**9
|
||||
|
||||
out = excursions(time_ns, bid, ask, entries, dirs, cps)
|
||||
|
||||
assert out["mfe"].shape == (40, 3)
|
||||
assert out["mae"].shape == (40, 3)
|
||||
assert out["ret"].shape == (40, 3)
|
||||
for k in range(40):
|
||||
ep, expected = oracle_excursions(time_ns, bid, ask, entries[k], dirs[k], cps)
|
||||
assert out["entry_price"][k] == pytest.approx(ep)
|
||||
for ci, (mfe, mae, ret) in enumerate(expected):
|
||||
assert out["mfe"][k, ci] == pytest.approx(mfe, abs=1e-12), (k, ci)
|
||||
assert out["mae"][k, ci] == pytest.approx(mae, abs=1e-12), (k, ci)
|
||||
assert out["ret"][k, ci] == pytest.approx(ret, abs=1e-12), (k, ci)
|
||||
|
||||
|
||||
def test_excursions_enter_at_ask_for_long_and_bid_for_short():
|
||||
time_ns = np.arange(3, dtype="int64") * 10**9
|
||||
bid = np.array([100.0, 100.0, 100.0])
|
||||
ask = bid + 0.4
|
||||
entries = np.array([0, 0], dtype="int64")
|
||||
dirs = np.array([1, -1], dtype="int8")
|
||||
cps = np.array([10 * 10**9], dtype="int64")
|
||||
|
||||
out = excursions(time_ns, bid, ask, entries, dirs, cps)
|
||||
|
||||
assert out["entry_price"][0] == pytest.approx(100.4)
|
||||
assert out["entry_price"][1] == pytest.approx(100.0)
|
||||
# marking to the opposite side: a flat market still costs the spread both ways
|
||||
assert out["ret"][0, 0] == pytest.approx(-0.4)
|
||||
assert out["ret"][1, 0] == pytest.approx(-0.4)
|
||||
assert out["mae"][0, 0] == pytest.approx(0.4)
|
||||
assert out["mae"][1, 0] == pytest.approx(0.4)
|
||||
assert out["mfe"][0, 0] == 0.0
|
||||
assert out["mfe"][1, 0] == 0.0
|
||||
|
||||
|
||||
def test_excursions_are_monotone_across_checkpoints():
|
||||
time_ns, bid, ask = random_ticks(21, n=2000)
|
||||
entries = np.array([10, 500, 900], dtype="int64")
|
||||
dirs = np.array([1, -1, 1], dtype="int8")
|
||||
cps = np.array([60, 300, 900], dtype="int64") * 10**9
|
||||
|
||||
out = excursions(time_ns, bid, ask, entries, dirs, cps)
|
||||
|
||||
assert (np.diff(out["mfe"], axis=1) >= 0).all()
|
||||
assert (np.diff(out["mae"], axis=1) >= 0).all()
|
||||
|
||||
|
||||
def test_entry_at_last_tick_gives_zero_excursions():
|
||||
time_ns = np.arange(5, dtype="int64") * 10**9
|
||||
bid = np.full(5, 10.0)
|
||||
ask = bid + 0.1
|
||||
|
||||
out = excursions(time_ns, bid, ask, np.array([4], dtype="int64"),
|
||||
np.array([1], dtype="int8"), np.array([10**9], dtype="int64"))
|
||||
|
||||
assert out["mfe"][0, 0] == 0.0
|
||||
assert out["mae"][0, 0] == 0.0
|
||||
assert out["ret"][0, 0] == 0.0
|
||||
assert out["entry_price"][0] == pytest.approx(10.1)
|
||||
|
||||
|
||||
def test_first_passage_matches_bruteforce_oracle():
|
||||
time_ns, bid, ask = random_ticks(11)
|
||||
rng = np.random.default_rng(11)
|
||||
entries = np.sort(rng.choice(np.arange(1, len(time_ns) - 500), 40, replace=False))
|
||||
entries = entries.astype("int64")
|
||||
dirs = rng.choice(np.array([-1, 1], dtype="int8"), 40)
|
||||
targets = rng.uniform(0.2, 1.5, 40)
|
||||
stops = rng.uniform(0.2, 1.5, 40)
|
||||
horizon = np.int64(900 * 10**9)
|
||||
|
||||
out = first_passage(time_ns, bid, ask, entries, dirs, targets, stops, horizon)
|
||||
|
||||
outcomes = set()
|
||||
for k in range(40):
|
||||
oc, pnl, et = oracle_first_passage(time_ns, bid, ask, entries[k], dirs[k],
|
||||
targets[k], stops[k], horizon)
|
||||
outcomes.add(oc)
|
||||
assert out["outcome"][k] == oc, k
|
||||
assert out["pnl"][k] == pytest.approx(pnl, abs=1e-12), k
|
||||
assert out["exit_time_ns"][k] == et, k
|
||||
assert len(outcomes) > 1
|
||||
|
||||
|
||||
def test_gap_through_target_pays_actual_crossing_price():
|
||||
time_ns = np.arange(4, dtype="int64") * 10**9
|
||||
bid = np.array([100.0, 100.0, 103.0, 103.0])
|
||||
ask = bid + 0.1
|
||||
|
||||
out = first_passage(time_ns, bid, ask, np.array([0], dtype="int64"),
|
||||
np.array([1], dtype="int8"), np.array([1.0]),
|
||||
np.array([1.0]), np.int64(10 * 10**9))
|
||||
|
||||
assert out["outcome"][0] == 1
|
||||
assert out["pnl"][0] == pytest.approx(103.0 - 100.1)
|
||||
assert out["exit_time_ns"][0] == 2 * 10**9
|
||||
|
||||
|
||||
def test_timeout_reports_closeout_value():
|
||||
time_ns = np.arange(4, dtype="int64") * 10**9
|
||||
bid = np.array([100.0, 100.2, 100.1, 105.0])
|
||||
ask = bid + 0.1
|
||||
|
||||
out = first_passage(time_ns, bid, ask, np.array([0], dtype="int64"),
|
||||
np.array([1], dtype="int8"), np.array([2.0]),
|
||||
np.array([2.0]), np.int64(2 * 10**9))
|
||||
|
||||
assert out["outcome"][0] == 0
|
||||
assert out["pnl"][0] == pytest.approx(100.1 - 100.1)
|
||||
assert out["exit_time_ns"][0] == 2 * 10**9
|
||||
|
||||
|
||||
def test_stop_first_for_sell():
|
||||
time_ns = np.arange(3, dtype="int64") * 10**9
|
||||
bid = np.array([100.0, 101.0, 101.0])
|
||||
ask = bid + 0.1
|
||||
|
||||
out = first_passage(time_ns, bid, ask, np.array([0], dtype="int64"),
|
||||
np.array([-1], dtype="int8"), np.array([0.5]),
|
||||
np.array([0.5]), np.int64(10 * 10**9))
|
||||
|
||||
assert out["outcome"][0] == -1
|
||||
assert out["pnl"][0] == pytest.approx(100.0 - 101.1)
|
||||
assert out["exit_time_ns"][0] == 10**9
|
||||
@@ -0,0 +1,90 @@
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from tickreplay.research.registry import REQUIRED, append, load
|
||||
from tickreplay.research.stats import bh_qvalues
|
||||
|
||||
|
||||
def row(config, p_value, **over):
|
||||
base = {"campaign": "c1", "symbol": "SYNTH", "config": config, "n_trades": 120,
|
||||
"mean_net": 0.02, "t_stat": 2.0, "p_value": p_value}
|
||||
base.update(over)
|
||||
return base
|
||||
|
||||
|
||||
def test_append_rejects_rows_missing_required_columns(tmp_path):
|
||||
path = tmp_path / "registry.csv"
|
||||
incomplete = row("a", 0.01)
|
||||
del incomplete["p_value"]
|
||||
|
||||
with pytest.raises(ValueError, match="p_value"):
|
||||
append(path, [incomplete])
|
||||
|
||||
assert not path.exists()
|
||||
|
||||
|
||||
def test_append_validates_every_row_before_writing_any(tmp_path):
|
||||
path = tmp_path / "registry.csv"
|
||||
bad = row("b", 0.01)
|
||||
del bad["n_trades"]
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
append(path, [row("a", 0.01), bad])
|
||||
|
||||
assert not path.exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("column", REQUIRED)
|
||||
def test_every_required_column_is_enforced(tmp_path, column):
|
||||
path = tmp_path / f"registry_{column}.csv"
|
||||
incomplete = row("a", 0.01)
|
||||
del incomplete[column]
|
||||
|
||||
with pytest.raises(ValueError, match=column):
|
||||
append(path, [incomplete])
|
||||
|
||||
|
||||
def test_append_twice_then_load_returns_all_rows_with_qvalues(tmp_path):
|
||||
path = tmp_path / "nested" / "registry.csv"
|
||||
|
||||
append(path, [row("a", 0.01), row("b", 0.50)])
|
||||
append(path, [row("c", 0.60), row("d", 0.70)])
|
||||
|
||||
df = load(path)
|
||||
|
||||
assert len(df) == 4
|
||||
assert list(df["config"]) == ["a", "b", "c", "d"]
|
||||
assert "q_value" in df.columns
|
||||
assert "ts" in df.columns
|
||||
assert set(REQUIRED) <= set(df.columns)
|
||||
np.testing.assert_allclose(df["q_value"].to_numpy(),
|
||||
bh_qvalues(df["p_value"].to_numpy()))
|
||||
|
||||
|
||||
def test_qvalues_are_recomputed_cumulatively_not_stored(tmp_path):
|
||||
path = tmp_path / "registry.csv"
|
||||
|
||||
append(path, [row("a", 0.01), row("b", 0.50)])
|
||||
first = load(path)
|
||||
assert first.loc[0, "q_value"] == pytest.approx(0.02)
|
||||
assert first.loc[1, "q_value"] == pytest.approx(0.50)
|
||||
|
||||
append(path, [row("c", 0.60), row("d", 0.70)])
|
||||
second = load(path)
|
||||
|
||||
assert second.loc[0, "q_value"] == pytest.approx(0.04)
|
||||
assert second.loc[1, "q_value"] == pytest.approx(0.70)
|
||||
assert second.loc[0, "q_value"] > first.loc[0, "q_value"]
|
||||
assert "q_value" not in path.read_text().splitlines()[0]
|
||||
|
||||
|
||||
def test_registry_is_append_only(tmp_path):
|
||||
path = tmp_path / "registry.csv"
|
||||
|
||||
append(path, [row("a", 0.01)])
|
||||
append(path, [row("a", 0.01)])
|
||||
|
||||
df = load(path)
|
||||
|
||||
assert len(df) == 2
|
||||
assert path.read_text().count("campaign,symbol") == 1
|
||||
@@ -0,0 +1,85 @@
|
||||
import numpy as np
|
||||
import pytest
|
||||
from scipy import stats as sps
|
||||
|
||||
from tickreplay.research.stats import bh_qvalues, one_sided_positive, thin_by_gap
|
||||
|
||||
|
||||
def test_one_sided_positive_matches_scipy():
|
||||
sample = np.array([0.4, -0.1, 0.9, 0.2, -0.3, 1.1, 0.05, 0.6, -0.2, 0.35])
|
||||
ref = sps.ttest_1samp(sample, 0.0, alternative="greater")
|
||||
|
||||
t_stat, p_value = one_sided_positive(sample)
|
||||
|
||||
assert t_stat == pytest.approx(float(ref.statistic))
|
||||
assert p_value == pytest.approx(float(ref.pvalue))
|
||||
|
||||
|
||||
def test_one_sided_positive_is_one_sided():
|
||||
rng = np.random.default_rng(0)
|
||||
winners = rng.normal(0.4, 1.0, 400)
|
||||
losers = -winners
|
||||
|
||||
assert one_sided_positive(winners)[1] < 0.01
|
||||
assert one_sided_positive(losers)[1] > 0.99
|
||||
|
||||
|
||||
def test_one_sided_positive_rejects_degenerate_samples():
|
||||
assert one_sided_positive([]) == (0.0, 1.0)
|
||||
assert one_sided_positive([1.0]) == (0.0, 1.0)
|
||||
assert one_sided_positive([3.0, 3.0, 3.0, 3.0]) == (0.0, 1.0)
|
||||
assert one_sided_positive([0.0, 0.0]) == (0.0, 1.0)
|
||||
|
||||
|
||||
def test_thin_by_gap_keeps_greedy_non_overlapping_subset():
|
||||
times = np.array([0, 5, 9, 10, 25, 26, 40], dtype="int64")
|
||||
|
||||
keep = thin_by_gap(times, 10)
|
||||
|
||||
np.testing.assert_array_equal(keep, [True, False, False, True, True, False, True])
|
||||
assert (np.diff(times[keep]) >= 10).all()
|
||||
|
||||
|
||||
def test_thin_by_gap_edges():
|
||||
times = np.array([0, 1, 2, 3], dtype="int64")
|
||||
|
||||
assert thin_by_gap(times, 0).all()
|
||||
np.testing.assert_array_equal(thin_by_gap(times, 10**9),
|
||||
[True, False, False, False])
|
||||
assert thin_by_gap(np.array([], dtype="int64"), 10).size == 0
|
||||
|
||||
|
||||
def test_bh_qvalues_matches_hand_computed_example():
|
||||
# raw = p * n / rank -> [.006, .024, .078, .0615, .0504, .06]; cumulative min
|
||||
# from the largest p downwards enforces monotonicity.
|
||||
pvals = np.array([0.001, 0.008, 0.039, 0.041, 0.042, 0.060])
|
||||
|
||||
q = bh_qvalues(pvals)
|
||||
|
||||
np.testing.assert_allclose(q, [0.006, 0.024, 0.0504, 0.0504, 0.0504, 0.060])
|
||||
|
||||
|
||||
def test_bh_qvalues_independent_of_input_order():
|
||||
pvals = np.array([0.001, 0.008, 0.039, 0.041, 0.042, 0.060])
|
||||
perm = np.array([3, 0, 5, 2, 4, 1])
|
||||
|
||||
q = bh_qvalues(pvals)
|
||||
q_perm = bh_qvalues(pvals[perm])
|
||||
|
||||
np.testing.assert_allclose(q_perm, q[perm])
|
||||
|
||||
|
||||
def test_bh_qvalues_are_monotone_and_bounded():
|
||||
rng = np.random.default_rng(7)
|
||||
pvals = np.r_[rng.uniform(0, 1, 200), rng.uniform(0, 1e-4, 20), 1.0, 0.0]
|
||||
|
||||
q = bh_qvalues(pvals)
|
||||
|
||||
assert ((q >= 0) & (q <= 1)).all()
|
||||
order = np.argsort(pvals)
|
||||
assert (np.diff(q[order]) >= -1e-12).all()
|
||||
assert (q >= pvals - 1e-12).all()
|
||||
|
||||
|
||||
def test_bh_qvalues_single_value_is_unchanged():
|
||||
np.testing.assert_allclose(bh_qvalues([0.03]), [0.03])
|
||||
@@ -0,0 +1,194 @@
|
||||
"""TickStore round-trips and bar derivation against a brute-force pandas oracle."""
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
from tickreplay.store import TIMEFRAMES, TickStore, derive_bars
|
||||
|
||||
|
||||
def _random_ticks(n=4000, seed=7, start="2024-01-02T00:00:00", max_gap_s=600):
|
||||
"""Irregularly spaced ticks: gaps up to max_gap_s leave many buckets empty."""
|
||||
rng = np.random.default_rng(seed)
|
||||
t0 = np.datetime64(start, "ns").astype("int64")
|
||||
time_ns = (t0 + np.cumsum(rng.integers(1, max_gap_s * 10**9, size=n))).astype("int64")
|
||||
bid = 2000.0 + np.cumsum(rng.normal(0.0, 0.1, n))
|
||||
ask = bid + rng.uniform(0.01, 0.05, n)
|
||||
return time_ns, bid, ask
|
||||
|
||||
|
||||
def _oracle_bars(time_ns, bid, ask, timeframe):
|
||||
tf_ns = TIMEFRAMES[timeframe] * 10**9
|
||||
df = pd.DataFrame({"bucket": time_ns // tf_ns, "bid": bid, "spread": ask - bid})
|
||||
out = df.groupby("bucket", sort=True).agg(
|
||||
open=("bid", "first"), high=("bid", "max"), low=("bid", "min"),
|
||||
close=("bid", "last"), spread_mean=("spread", "mean"), tick_count=("bid", "size"))
|
||||
out.insert(0, "time", pd.to_datetime(out.index.to_numpy() * tf_ns, unit="ns"))
|
||||
return out.reset_index(drop=True)
|
||||
|
||||
|
||||
def _month_frame(start, n, seed, max_gap_s=3600):
|
||||
rng = np.random.default_rng(seed)
|
||||
t0 = np.datetime64(start, "ns").astype("int64")
|
||||
times = t0 + np.cumsum(rng.integers(1, max_gap_s * 10**9, size=n))
|
||||
bid = 2000.0 + rng.normal(0.0, 1.0, n)
|
||||
return pd.DataFrame({"time": pd.to_datetime(times, unit="ns"),
|
||||
"bid": bid, "ask": bid + 0.03})
|
||||
|
||||
|
||||
@pytest.mark.parametrize("timeframe", ["M1", "M5", "M15", "M30", "H1", "H4", "D1"])
|
||||
def test_derive_bars_matches_pandas_oracle(timeframe):
|
||||
time_ns, bid, ask = _random_ticks()
|
||||
bars = derive_bars(time_ns, bid, ask, timeframe)
|
||||
oracle = _oracle_bars(time_ns, bid, ask, timeframe)
|
||||
|
||||
assert len(oracle) > 1
|
||||
assert len(bars) == len(oracle)
|
||||
assert np.array_equal(bars["time"].to_numpy(), oracle["time"].to_numpy())
|
||||
for col in ("open", "high", "low", "close"):
|
||||
assert np.array_equal(bars[col].to_numpy(), oracle[col].to_numpy())
|
||||
np.testing.assert_allclose(bars["spread_mean"].to_numpy(),
|
||||
oracle["spread_mean"].to_numpy(), rtol=1e-12)
|
||||
assert np.array_equal(bars["tick_count"].to_numpy(), oracle["tick_count"].to_numpy())
|
||||
|
||||
|
||||
def test_derive_bars_emits_no_bar_for_empty_buckets():
|
||||
time_ns, bid, ask = _random_ticks()
|
||||
tf_ns = TIMEFRAMES["M1"] * 10**9
|
||||
bars = derive_bars(time_ns, bid, ask, "M1")
|
||||
|
||||
spanned = int(time_ns[-1] // tf_ns - time_ns[0] // tf_ns) + 1
|
||||
assert len(bars) < spanned
|
||||
occupied = np.unique(time_ns // tf_ns)
|
||||
assert np.array_equal(bars["time"].to_numpy().astype("int64") // tf_ns, occupied)
|
||||
|
||||
|
||||
def test_bar_times_are_epoch_aligned():
|
||||
time_ns, bid, ask = _random_ticks()
|
||||
for timeframe, seconds in TIMEFRAMES.items():
|
||||
times = derive_bars(time_ns, bid, ask, timeframe)["time"]
|
||||
assert (times.to_numpy().astype("int64") % (seconds * 10**9) == 0).all()
|
||||
|
||||
h4 = derive_bars(time_ns, bid, ask, "H4")["time"].dt
|
||||
assert set(h4.hour.unique()) == {0, 4, 8, 12, 16, 20}
|
||||
assert (h4.minute == 0).all()
|
||||
assert (h4.second == 0).all()
|
||||
|
||||
d1 = derive_bars(time_ns, bid, ask, "D1")["time"]
|
||||
assert len(d1) > 1
|
||||
assert (d1 == d1.dt.normalize()).all()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("timeframe", ["M1", "M15", "H4"])
|
||||
def test_tick_spans_partition_every_tick(timeframe):
|
||||
time_ns, bid, ask = _random_ticks()
|
||||
bars = derive_bars(time_ns, bid, ask, timeframe)
|
||||
first = bars["tick_first"].to_numpy()
|
||||
last = bars["tick_last"].to_numpy()
|
||||
count = bars["tick_count"].to_numpy()
|
||||
|
||||
assert first[0] == 0
|
||||
assert last[-1] == len(time_ns) - 1
|
||||
assert (last >= first).all()
|
||||
assert np.array_equal(first[1:], last[:-1] + 1)
|
||||
assert np.array_equal(count, last - first + 1)
|
||||
assert count.sum() == len(time_ns)
|
||||
|
||||
bucket = time_ns // (TIMEFRAMES[timeframe] * 10**9)
|
||||
assert np.array_equal(bucket[first], bucket[last])
|
||||
assert np.array_equal(bars["time"].to_numpy().astype("int64"),
|
||||
bucket[first] * TIMEFRAMES[timeframe] * 10**9)
|
||||
|
||||
|
||||
def test_write_open_roundtrip(tmp_path):
|
||||
time_ns, bid, ask = _random_ticks(n=500, seed=1)
|
||||
store_dir = tmp_path / "store"
|
||||
written = TickStore.write(store_dir, time_ns, bid, ask, {"symbol": "XAUUSD"})
|
||||
expected_range = [str(np.datetime64(int(time_ns[0]), "ns")),
|
||||
str(np.datetime64(int(time_ns[-1]), "ns"))]
|
||||
|
||||
for store in (written, TickStore.open(store_dir)):
|
||||
assert len(store) == len(time_ns)
|
||||
assert np.array_equal(np.asarray(store.time_ns), time_ns)
|
||||
assert np.array_equal(np.asarray(store.bid), bid)
|
||||
assert np.array_equal(np.asarray(store.ask), ask)
|
||||
assert store.manifest["n_ticks"] == len(time_ns)
|
||||
assert store.manifest["time_range"] == expected_range
|
||||
assert store.manifest["symbol"] == "XAUUSD"
|
||||
|
||||
assert json.loads((store_dir / "manifest.json").read_text()) == written.manifest
|
||||
|
||||
|
||||
def test_write_rejects_non_monotonic_time(tmp_path):
|
||||
backwards = np.array([10, 30, 20], dtype="int64") * 10**9
|
||||
with pytest.raises(ValueError, match="non-monotonic"):
|
||||
TickStore.write(tmp_path / "bad", backwards, [1.0, 2.0, 3.0], [1.1, 2.1, 3.1])
|
||||
|
||||
repeated = np.array([10, 20, 20, 30], dtype="int64") * 10**9
|
||||
store = TickStore.write(tmp_path / "ok", repeated, [1.0] * 4, [1.1] * 4)
|
||||
assert len(store) == 4
|
||||
|
||||
|
||||
def test_build_concatenates_monthly_parquet_files(tmp_path):
|
||||
months = {"2024-01": _month_frame("2024-01-02T00:00:00", 50, 0),
|
||||
"2024-02": _month_frame("2024-02-01T00:00:00", 40, 1),
|
||||
"2024-03": _month_frame("2024-03-01T00:00:00", 30, 2)}
|
||||
parquet_dir = tmp_path / "parquet"
|
||||
parquet_dir.mkdir()
|
||||
for month, frame in months.items():
|
||||
frame.to_parquet(parquet_dir / f"ticks_{month}.parquet", index=False)
|
||||
|
||||
store = TickStore.build(parquet_dir, tmp_path / "store")
|
||||
expected = pd.concat(months.values(), ignore_index=True)
|
||||
|
||||
assert len(store) == len(expected) == 120
|
||||
assert np.array_equal(np.asarray(store.time_ns),
|
||||
expected["time"].to_numpy().astype("datetime64[ns]").astype("int64"))
|
||||
assert np.array_equal(np.asarray(store.bid), expected["bid"].to_numpy())
|
||||
assert np.array_equal(np.asarray(store.ask), expected["ask"].to_numpy())
|
||||
assert store.manifest["n_ticks"] == 120
|
||||
assert store.manifest["sources"] == ["ticks_2024-01.parquet", "ticks_2024-02.parquet",
|
||||
"ticks_2024-03.parquet"]
|
||||
assert store.manifest["time_range"][0].startswith("2024-01")
|
||||
assert store.manifest["time_range"][1].startswith("2024-03")
|
||||
assert TickStore.open(tmp_path / "store").manifest == store.manifest
|
||||
|
||||
|
||||
def test_build_rejects_non_monotonic_time_within_a_file(tmp_path):
|
||||
frame = _month_frame("2024-01-02T00:00:00", 20, 0)
|
||||
frame.loc[10, "time"] = frame.loc[2, "time"]
|
||||
parquet_dir = tmp_path / "parquet"
|
||||
parquet_dir.mkdir()
|
||||
frame.to_parquet(parquet_dir / "ticks_2024-01.parquet", index=False)
|
||||
|
||||
with pytest.raises(ValueError, match="non-monotonic"):
|
||||
TickStore.build(parquet_dir, tmp_path / "store")
|
||||
|
||||
|
||||
def test_build_rejects_time_going_backwards_across_files(tmp_path):
|
||||
parquet_dir = tmp_path / "parquet"
|
||||
parquet_dir.mkdir()
|
||||
_month_frame("2024-02-01T00:00:00", 20, 0).to_parquet(
|
||||
parquet_dir / "ticks_2024-01.parquet", index=False)
|
||||
_month_frame("2024-01-02T00:00:00", 20, 1).to_parquet(
|
||||
parquet_dir / "ticks_2024-02.parquet", index=False)
|
||||
|
||||
with pytest.raises(ValueError, match="non-monotonic"):
|
||||
TickStore.build(parquet_dir, tmp_path / "store")
|
||||
|
||||
|
||||
def test_build_requires_at_least_one_parquet_file(tmp_path):
|
||||
(tmp_path / "empty").mkdir()
|
||||
with pytest.raises(ValueError, match="no ticks_"):
|
||||
TickStore.build(tmp_path / "empty", tmp_path / "store")
|
||||
|
||||
|
||||
def test_bars_are_cached_per_timeframe(tmp_path):
|
||||
time_ns, bid, ask = _random_ticks(n=800, seed=3)
|
||||
store = TickStore.write(tmp_path / "store", time_ns, bid, ask)
|
||||
|
||||
m5 = store.bars("M5")
|
||||
assert store.bars("M5") is m5
|
||||
assert store.bars("M15") is not m5
|
||||
pd.testing.assert_frame_equal(m5, derive_bars(time_ns, bid, ask, "M5"))
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Sweeps must fan out without scrambling which result belongs to which config."""
|
||||
import time
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
import tickreplay as tr
|
||||
|
||||
|
||||
def crossover_case():
|
||||
time_ns, bid, ask = tr.synth.generate_ticks(days=3, ticks_per_day=8000, seed=5)
|
||||
store = tr.TickStore(time_ns, bid, ask)
|
||||
bars = store.bars("M5")
|
||||
trend = np.sign(bars["close"].rolling(5).mean() - bars["close"].rolling(20).mean())
|
||||
signal = trend.fillna(0).to_numpy().astype("int8")
|
||||
signal[signal == np.roll(signal, 1)] = 0 # act on crosses only
|
||||
return store, bars, signal
|
||||
|
||||
|
||||
def test_sweep_exits_returns_one_row_per_config_matching_a_direct_replay():
|
||||
store, bars, signal = crossover_case()
|
||||
base = {"max_hold": "1h", "cost": 0.09}
|
||||
grid = [{"sl": sl, "tp": tp} for sl in (1.0, 3.0) for tp in (2.0, 6.0)]
|
||||
|
||||
df = tr.sweep_exits(store, bars, signal, grid, base=base)
|
||||
|
||||
assert len(df) == len(grid)
|
||||
assert df["n_trades"].min() > 0
|
||||
assert df["total_net"].nunique() == len(grid), "grid must actually discriminate"
|
||||
for row, overrides in zip(df.to_dict("records"), grid):
|
||||
assert (row["sl"], row["tp"]) == (overrides["sl"], overrides["tp"])
|
||||
expected = tr.replay(store, bars, signal,
|
||||
tr.Config(**base, **overrides)).summary()
|
||||
assert set(expected) <= set(row)
|
||||
for key, value in expected.items():
|
||||
assert row[key] == pytest.approx(value), f"{overrides}: {key}"
|
||||
|
||||
|
||||
def test_sweep_exits_applies_base_under_the_grid():
|
||||
store, bars, signal = crossover_case()
|
||||
grid = [{"sl": 2.0}]
|
||||
|
||||
with_hold = tr.sweep_exits(store, bars, signal, grid, base={"max_hold": "15m"})
|
||||
without_hold = tr.sweep_exits(store, bars, signal, grid)
|
||||
|
||||
assert with_hold.loc[0, "n_trades"] != without_hold.loc[0, "n_trades"]
|
||||
|
||||
|
||||
def test_run_sweep_preserves_config_to_result_pairing():
|
||||
# descending delays make workers finish in roughly reverse submission order
|
||||
configs = [{"idx": i, "delay": (7 - i) * 0.02} for i in range(8)]
|
||||
|
||||
def evaluate(config):
|
||||
time.sleep(config["delay"])
|
||||
return {"echo": config["idx"] * 10}
|
||||
|
||||
df = tr.run_sweep(evaluate, configs, workers=4)
|
||||
|
||||
assert list(df["idx"]) == [c["idx"] for c in configs]
|
||||
assert list(df["echo"]) == [c["idx"] * 10 for c in configs]
|
||||
|
||||
|
||||
def test_run_sweep_runs_configs_concurrently():
|
||||
def evaluate(config):
|
||||
time.sleep(0.1)
|
||||
return {"idx_seen": config["idx"]}
|
||||
|
||||
configs = [{"idx": i} for i in range(8)]
|
||||
started = time.perf_counter()
|
||||
df = tr.run_sweep(evaluate, configs, workers=8)
|
||||
elapsed = time.perf_counter() - started
|
||||
|
||||
assert list(df["idx_seen"]) == list(range(8))
|
||||
assert elapsed < 0.4, f"8 x 100ms across 8 workers took {elapsed:.2f}s -- serialized?"
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Synthetic ticks stand in for broker data, so their invariants have to hold."""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
import tickreplay as tr
|
||||
|
||||
# 2024-01-01 is a Monday, so a 9-day window spans exactly one weekend
|
||||
NINE_DAYS = dict(days=9, ticks_per_day=2000, start="2024-01-01", seed=2)
|
||||
|
||||
|
||||
def test_time_never_goes_backwards():
|
||||
time_ns, _, _ = tr.synth.generate_ticks(days=5, ticks_per_day=3000, seed=1)
|
||||
assert len(time_ns) > 1000
|
||||
assert (np.diff(time_ns) >= 0).all()
|
||||
|
||||
|
||||
def test_ask_is_always_above_bid():
|
||||
_, bid, ask = tr.synth.generate_ticks(days=5, ticks_per_day=3000, seed=1)
|
||||
assert len(bid) > 1000
|
||||
assert (ask > bid).all()
|
||||
|
||||
|
||||
def test_no_ticks_land_on_a_weekend():
|
||||
time_ns, _, _ = tr.synth.generate_ticks(**NINE_DAYS)
|
||||
weekday = pd.DatetimeIndex(time_ns.astype("datetime64[ns]")).dayofweek
|
||||
assert set(weekday) == {0, 1, 2, 3, 4}
|
||||
|
||||
|
||||
def test_day_count_covers_the_requested_window_minus_weekends():
|
||||
time_ns, _, _ = tr.synth.generate_ticks(**NINE_DAYS)
|
||||
days = np.unique(time_ns.astype("datetime64[ns]").astype("datetime64[D]"))
|
||||
expected = np.array(["2024-01-01", "2024-01-02", "2024-01-03", "2024-01-04",
|
||||
"2024-01-05", "2024-01-08", "2024-01-09"], "datetime64[D]")
|
||||
np.testing.assert_array_equal(days, expected)
|
||||
|
||||
|
||||
def test_ticks_stay_inside_the_22h_session():
|
||||
time_ns, _, _ = tr.synth.generate_ticks(**NINE_DAYS)
|
||||
seconds_into_day = (time_ns % (86_400 * 10**9)) / 10**9
|
||||
assert seconds_into_day.max() < 22 * 3600
|
||||
assert seconds_into_day.max() > 20 * 3600, "session barely used -- fixture too thin"
|
||||
|
||||
|
||||
def test_same_seed_reproduces_the_same_ticks():
|
||||
kwargs = dict(days=3, ticks_per_day=2000, seed=11)
|
||||
for first, second in zip(tr.synth.generate_ticks(**kwargs),
|
||||
tr.synth.generate_ticks(**kwargs)):
|
||||
np.testing.assert_array_equal(first, second)
|
||||
|
||||
|
||||
def test_different_seeds_produce_different_ticks():
|
||||
_, bid_a, _ = tr.synth.generate_ticks(days=3, ticks_per_day=2000, seed=11)
|
||||
_, bid_b, _ = tr.synth.generate_ticks(days=3, ticks_per_day=2000, seed=12)
|
||||
assert len(bid_a) != len(bid_b) or not np.array_equal(bid_a, bid_b)
|
||||
|
||||
|
||||
def test_weekend_only_window_raises_a_clear_error():
|
||||
with pytest.raises(ValueError, match="no weekdays"):
|
||||
tr.synth.generate_ticks(days=2, start="2024-01-06") # Sat + Sun
|
||||
@@ -0,0 +1,104 @@
|
||||
import numpy as np
|
||||
|
||||
from tickreplay.research.validation import holdout_cut, purged_expanding_folds
|
||||
|
||||
HOUR_NS = 3600 * 10**9
|
||||
DAY_NS = 24 * HOUR_NS
|
||||
|
||||
|
||||
def sample_times(n=6000, step_ns=37 * 60 * 10**9):
|
||||
return 1_700_000_000 * 10**9 + np.arange(n, dtype="int64") * step_ns
|
||||
|
||||
|
||||
def slice_edges(t, n_folds):
|
||||
lo, hi = t.min(), t.max() + 1
|
||||
return lo + (hi - lo) * np.arange(n_folds + 1) // n_folds
|
||||
|
||||
|
||||
def test_purged_folds_never_let_a_train_window_reach_the_test_start():
|
||||
t = sample_times()
|
||||
horizon, embargo = np.int64(HOUR_NS), np.int64(DAY_NS)
|
||||
edges = slice_edges(t, 5)
|
||||
|
||||
folds = purged_expanding_folds(t, horizon, embargo, n_folds=5)
|
||||
|
||||
assert len(folds) == 4
|
||||
for k, (train, test) in enumerate(folds, start=1):
|
||||
assert len(train) > 0
|
||||
assert len(test) > 0
|
||||
assert (t[train] + horizon + embargo <= edges[k]).all()
|
||||
assert (t[train] + horizon + embargo <= t[test].min()).all()
|
||||
|
||||
|
||||
def test_purged_folds_train_strictly_before_test():
|
||||
t = sample_times()
|
||||
|
||||
folds = purged_expanding_folds(t, np.int64(HOUR_NS), np.int64(DAY_NS), n_folds=5)
|
||||
|
||||
for train, test in folds:
|
||||
assert t[train].max() < t[test].min()
|
||||
assert not set(train.tolist()) & set(test.tolist())
|
||||
|
||||
|
||||
def test_purging_drops_samples_a_plain_split_would_have_kept():
|
||||
t = sample_times()
|
||||
edges = slice_edges(t, 5)
|
||||
|
||||
unpurged = purged_expanding_folds(t, np.int64(0), np.int64(0), n_folds=5)
|
||||
purged = purged_expanding_folds(t, np.int64(HOUR_NS), np.int64(DAY_NS), n_folds=5)
|
||||
|
||||
for (train_u, _), (train_p, _) in zip(unpurged, purged):
|
||||
assert len(train_p) < len(train_u)
|
||||
assert set(train_p.tolist()) < set(train_u.tolist())
|
||||
assert (t[unpurged[0][0]].max() < edges[1])
|
||||
|
||||
|
||||
def test_first_slice_is_never_a_test_fold():
|
||||
t = sample_times()
|
||||
edges = slice_edges(t, 5)
|
||||
|
||||
folds = purged_expanding_folds(t, np.int64(0), np.int64(0), n_folds=5)
|
||||
|
||||
for _, test in folds:
|
||||
assert t[test].min() >= edges[1]
|
||||
|
||||
|
||||
def test_test_folds_partition_the_remaining_time_into_equal_slices():
|
||||
t = sample_times(n=8760, step_ns=HOUR_NS)
|
||||
|
||||
folds = purged_expanding_folds(t, np.int64(0), np.int64(0), n_folds=5)
|
||||
|
||||
sizes = [len(test) for _, test in folds]
|
||||
assert max(sizes) - min(sizes) <= 0.2 * max(sizes)
|
||||
covered = np.concatenate([test for _, test in folds])
|
||||
assert len(covered) == len(set(covered.tolist()))
|
||||
edges = slice_edges(t, 5)
|
||||
assert len(covered) == int((t >= edges[1]).sum())
|
||||
|
||||
|
||||
def test_purged_folds_honour_n_folds():
|
||||
t = sample_times()
|
||||
|
||||
assert len(purged_expanding_folds(t, np.int64(0), np.int64(0), n_folds=3)) == 2
|
||||
assert len(purged_expanding_folds(t, np.int64(0), np.int64(0), n_folds=10)) == 9
|
||||
|
||||
|
||||
def test_holdout_cut_returns_first_index_at_or_after_the_boundary():
|
||||
t = np.arange(10, dtype="int64") * 1000
|
||||
|
||||
assert holdout_cut(t, 5000) == 5
|
||||
assert holdout_cut(t, 4999) == 5
|
||||
assert holdout_cut(t, 5001) == 6
|
||||
assert holdout_cut(t, 0) == 0
|
||||
assert holdout_cut(t, -1) == 0
|
||||
assert holdout_cut(t, 10**9) == len(t)
|
||||
|
||||
|
||||
def test_holdout_cut_splits_cleanly_around_the_boundary():
|
||||
t = sample_times()
|
||||
boundary = int(t[0] + (t[-1] - t[0]) * 0.8)
|
||||
|
||||
cut = holdout_cut(t, boundary)
|
||||
|
||||
assert (t[:cut] < boundary).all()
|
||||
assert (t[cut:] >= boundary).all()
|
||||
Reference in New Issue
Block a user