3ebcb3f758
Adds a `throughput` benchmark to every target and closes two small test-coverage documentation/QA gaps. One PR, no merge of binding code beyond the additive benchmarks and one C test. ## 1. Per-binding throughput benchmarks (all 9 targets) Each benchmark feeds a deterministic synthetic OHLCV series through three indicators chosen by **FFI call-signature archetype** (not algorithm — the same Rust core runs underneath all bindings): - `SMA(20)` — 1-in → 1-out (baseline boundary cost) - `ATR(14)` — multi-in → 1-out (input marshalling) - `MACD(12,26,9)` — 1-in → multi-out (output marshalling) Streaming is timed for all three; batch for the single-output SMA and ATR (median of 3 runs, after a warmup pass). New: Python (PyO3), WASM, C (CMake), C# (Stopwatch), Go, Java (FFM), R, and the Rust core baseline (`examples/rust/.../throughput.rs`, **no FFI** — the ceiling the bindings are measured against and the value their batch paths converge towards). Node already had `throughput.js`. **Not a speed claim:** there is no comparable streaming TA library for C, C#, Go, Java, R or WASM to compare against, so these are raw per-binding throughput numbers documenting each language's FFI overhead — see BENCHMARKS.md §3. The "Wickra is fast" claim still lives in §1/§2 (Rust core + the Python/Rust cross-library runs). ## 2. README `## Testing`: C# and C bullets The section listed every layer except C# and C, even though both have suites. Adds the two missing bullets. ## 3. C archetype ctest `examples/c/archetypes.c` drives one indicator per FFI archetype through the real C boundary (scalar + batch==streaming, multi-output, bars, profile, array input) plus reset, invalid-parameter and NULL-safety — the C counterpart of the Go/R/Java archetype suites. Runs on three OSes via the existing CMake/ctest. ## Notes - Benchmarks are not CI-gated (manual-run scripts, like the existing `throughput.js`); no `ci.yml`/`release.yml` changes. - Docs: BENCHMARKS.md §3, a `## Benchmark` section in every binding README, a CHANGELOG entry. - Verified locally by running: Rust, Python, C, C#, Go, Java (real numbers); the C archetype ctest with `-Wall -Wextra -Wpedantic -Werror`. WASM and R are API-correct and syntax-checked but need their own toolchains to run.
117 lines
4.0 KiB
Python
117 lines
4.0 KiB
Python
"""Throughput benchmark for the Wickra Python binding.
|
|
|
|
Measures how many indicator updates per second the binding sustains, both
|
|
per-tick (streaming ``update``) and bulk (``batch``), over a synthetic OHLCV
|
|
series. It is the Python counterpart of the Node ``throughput.js`` and the Rust
|
|
criterion benches: it benchmarks Wickra's own O(1) streaming engine across the
|
|
Python<->Rust boundary, so the headline number is raw per-binding throughput /
|
|
FFI overhead, not a cross-library ratio.
|
|
|
|
For the cross-library comparison against TA-Lib, pandas-ta, tulipy and finta,
|
|
see ``benchmarks/compare_libraries.py`` instead.
|
|
|
|
Three indicators are timed, chosen by call-signature archetype rather than
|
|
algorithm: SMA (1-in -> 1-out), ATR (multi-in -> 1-out) and MACD (1-in ->
|
|
multi-out). Streaming is timed for all three; batch only for the single-output
|
|
SMA and ATR (multi-output batch returns a 2-D array and is not compared here).
|
|
|
|
Install the binding first (``maturin develop --release`` in bindings/python),
|
|
then run from bindings/python::
|
|
|
|
python -m benchmarks.throughput # 200k bars (default)
|
|
python -m benchmarks.throughput --bars 1000000
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import time
|
|
|
|
import numpy as np
|
|
|
|
import wickra as ta
|
|
|
|
|
|
def _time_ns(fn, reps: int = 3) -> float:
|
|
"""Median elapsed-ns over a few repetitions, after one warmup pass."""
|
|
fn() # warmup
|
|
samples = []
|
|
for _ in range(reps):
|
|
t0 = time.perf_counter_ns()
|
|
fn()
|
|
samples.append(time.perf_counter_ns() - t0)
|
|
samples.sort()
|
|
return samples[len(samples) // 2]
|
|
|
|
|
|
def main() -> None:
|
|
parser = argparse.ArgumentParser(description="Wickra Python throughput benchmark")
|
|
parser.add_argument("--bars", type=int, default=200_000, help="number of synthetic bars")
|
|
args = parser.parse_args()
|
|
bars = args.bars if args.bars >= 1000 else 200_000
|
|
|
|
# Deterministic synthetic OHLCV (no RNG, so runs are comparable).
|
|
idx = np.arange(bars, dtype=np.float64)
|
|
mid = 100 + np.sin(idx * 0.001) * 20 + idx * 1e-4
|
|
close = mid + np.sin(idx * 0.05) * 2
|
|
high = np.maximum(close, mid) + 1.5
|
|
low = np.minimum(close, mid) - 1.5
|
|
open_ = mid
|
|
volume = 1000 + (idx % 97) * 13
|
|
|
|
close_list = close.tolist()
|
|
# ATR streams a 6-tuple (open, high, low, close, volume, timestamp) per tick.
|
|
candles = list(
|
|
zip(open_.tolist(), high.tolist(), low.tolist(), close_list, volume.tolist(), range(bars))
|
|
)
|
|
|
|
def mups(ns: float) -> float:
|
|
return bars / (ns / 1e9) / 1e6
|
|
|
|
def sma_stream() -> None:
|
|
ind = ta.SMA(20)
|
|
for value in close_list:
|
|
ind.update(value)
|
|
|
|
def sma_batch() -> None:
|
|
ta.SMA(20).batch(close)
|
|
|
|
def atr_stream() -> None:
|
|
ind = ta.ATR(14)
|
|
for candle in candles:
|
|
ind.update(candle)
|
|
|
|
def atr_batch() -> None:
|
|
ta.ATR(14).batch(high, low, close)
|
|
|
|
def macd_stream() -> None:
|
|
ind = ta.MACD(12, 26, 9)
|
|
for value in close_list:
|
|
ind.update(value)
|
|
|
|
# SMA (scalar 1-in/1-out), ATR (multi-in/1-out), MACD (1-in/multi-out).
|
|
indicators = [
|
|
("SMA(20)", sma_stream, sma_batch),
|
|
("ATR(14)", atr_stream, atr_batch),
|
|
("MACD(12,26,9)", macd_stream, None), # multi-output: streaming only
|
|
]
|
|
|
|
print(f"Wickra Python throughput - {bars:,} bars (median of 3 runs)\n")
|
|
print(f"{'Indicator':<22}{'streaming (Mupd/s)':>20}{'batch (Mupd/s)':>18}")
|
|
print("-" * 60)
|
|
for name, stream, batch in indicators:
|
|
stream_mups = f"{mups(_time_ns(stream)):.1f}"
|
|
batch_mups = "-" if batch is None else f"{mups(_time_ns(batch)):.1f}"
|
|
print(f"{name:<22}{stream_mups:>20}{batch_mups:>18}")
|
|
|
|
print(
|
|
"\nMupd/s = million indicator updates per second. Streaming is the per-tick\n"
|
|
"`update` path crossing the Python<->Rust boundary once per value; batch is\n"
|
|
"the bulk numpy path (one boundary crossing). Higher is better. Numbers are\n"
|
|
"machine-dependent - use them for relative comparison, not as a speed claim."
|
|
)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|