Per-binding throughput benchmarks + test-coverage gaps (#246)
Adds a `throughput` benchmark to every target and closes two small test-coverage documentation/QA gaps. One PR, no merge of binding code beyond the additive benchmarks and one C test. ## 1. Per-binding throughput benchmarks (all 9 targets) Each benchmark feeds a deterministic synthetic OHLCV series through three indicators chosen by **FFI call-signature archetype** (not algorithm — the same Rust core runs underneath all bindings): - `SMA(20)` — 1-in → 1-out (baseline boundary cost) - `ATR(14)` — multi-in → 1-out (input marshalling) - `MACD(12,26,9)` — 1-in → multi-out (output marshalling) Streaming is timed for all three; batch for the single-output SMA and ATR (median of 3 runs, after a warmup pass). New: Python (PyO3), WASM, C (CMake), C# (Stopwatch), Go, Java (FFM), R, and the Rust core baseline (`examples/rust/.../throughput.rs`, **no FFI** — the ceiling the bindings are measured against and the value their batch paths converge towards). Node already had `throughput.js`. **Not a speed claim:** there is no comparable streaming TA library for C, C#, Go, Java, R or WASM to compare against, so these are raw per-binding throughput numbers documenting each language's FFI overhead — see BENCHMARKS.md §3. The "Wickra is fast" claim still lives in §1/§2 (Rust core + the Python/Rust cross-library runs). ## 2. README `## Testing`: C# and C bullets The section listed every layer except C# and C, even though both have suites. Adds the two missing bullets. ## 3. C archetype ctest `examples/c/archetypes.c` drives one indicator per FFI archetype through the real C boundary (scalar + batch==streaming, multi-output, bars, profile, array input) plus reset, invalid-parameter and NULL-safety — the C counterpart of the Go/R/Java archetype suites. Runs on three OSes via the existing CMake/ctest. ## Notes - Benchmarks are not CI-gated (manual-run scripts, like the existing `throughput.js`); no `ci.yml`/`release.yml` changes. - Docs: BENCHMARKS.md §3, a `## Benchmark` section in every binding README, a CHANGELOG entry. - Verified locally by running: Rust, Python, C, C#, Go, Java (real numbers); the C archetype ctest with `-Wall -Wextra -Wpedantic -Werror`. WASM and R are API-correct and syntax-checked but need their own toolchains to run.
This commit is contained in:
@@ -59,6 +59,17 @@ indicators take the OHLCV fields plus a timestamp, e.g.
|
||||
`update(atr, open, high, low, close, volume, timestamp)`. The native handle is
|
||||
freed automatically when the object is garbage-collected.
|
||||
|
||||
## Benchmark
|
||||
|
||||
`benchmarks/throughput.R` reports streaming and batch updates-per-second for
|
||||
`SMA`, `ATR` and `MACD`. It measures this binding's FFI overhead, not a
|
||||
cross-library ratio (the same Rust core runs under every binding) — see the
|
||||
repository [BENCHMARKS.md](https://github.com/wickra-lib/wickra/blob/main/BENCHMARKS.md) §3.
|
||||
|
||||
```bash
|
||||
Rscript benchmarks/throughput.R
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
The full indicator catalogue, guides, quickstarts, and API reference live in the
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env Rscript
|
||||
#
|
||||
# Throughput benchmark for the Wickra R bindings.
|
||||
#
|
||||
# Measures how many indicator updates per second the R binding sustains, both
|
||||
# per-tick (streaming `update`) and bulk (`batch`), over a synthetic OHLCV
|
||||
# series. It is the R counterpart of the Node `throughput.js` and the Rust
|
||||
# criterion benches: it benchmarks Wickra's own O(1) streaming engine across
|
||||
# the R<->C-ABI boundary (there is no comparable streaming TA library on CRAN
|
||||
# to compare against), so the headline number is raw per-binding throughput /
|
||||
# FFI overhead, not a cross-library ratio.
|
||||
#
|
||||
# Three indicators are timed, chosen by FFI call-signature archetype rather
|
||||
# than algorithm: SMA (1-in -> 1-out), ATR (multi-in -> 1-out), and MACD
|
||||
# (1-in -> multi-out). Streaming is timed for all three; batch only for the
|
||||
# single-output SMA and ATR (multi-output batch is not exposed uniformly).
|
||||
#
|
||||
# Install the package first (it links the C ABI; see bindings/r/README.md),
|
||||
# then run:
|
||||
#
|
||||
# Rscript bindings/r/benchmarks/throughput.R # 200k bars (default)
|
||||
# Rscript bindings/r/benchmarks/throughput.R --bars 1000000
|
||||
|
||||
suppressMessages(library(wickra))
|
||||
|
||||
parse_bars <- function() {
|
||||
args <- commandArgs(trailingOnly = TRUE)
|
||||
i <- match("--bars", args)
|
||||
if (!is.na(i) && length(args) >= i + 1L) {
|
||||
n <- suppressWarnings(as.integer(args[i + 1L]))
|
||||
if (!is.na(n) && n >= 1000L) {
|
||||
return(n)
|
||||
}
|
||||
stop("--bars must be an integer >= 1000")
|
||||
}
|
||||
200000L
|
||||
}
|
||||
|
||||
bars <- parse_bars()
|
||||
|
||||
# Deterministic synthetic OHLCV (no RNG, so runs are comparable).
|
||||
idx <- seq.int(0L, bars - 1L)
|
||||
mid <- 100 + sin(idx * 0.001) * 20 + idx * 1e-4
|
||||
close <- mid + sin(idx * 0.05) * 2
|
||||
high <- pmax(close, mid) + 1.5
|
||||
low <- pmin(close, mid) - 1.5
|
||||
open <- mid
|
||||
volume <- 1000 + (idx %% 97L) * 13
|
||||
# `numeric` (double), not integer: the candle batch path coerces the timestamp
|
||||
# column with REAL(), which rejects an integer vector.
|
||||
timestamp <- as.numeric(idx)
|
||||
|
||||
# Median elapsed-ns over a few repetitions, after one warmup pass.
|
||||
time_ns <- function(fn, reps = 3L) {
|
||||
fn() # warmup
|
||||
samples <- numeric(reps)
|
||||
for (r in seq_len(reps)) {
|
||||
t0 <- Sys.time()
|
||||
fn()
|
||||
samples[r] <- as.numeric(Sys.time() - t0, units = "secs") * 1e9
|
||||
}
|
||||
median(samples)
|
||||
}
|
||||
|
||||
mups_from_ns <- function(ns) bars / (ns / 1e9) / 1e6
|
||||
|
||||
# SMA (scalar 1-in/1-out), ATR (multi-in/1-out), MACD (1-in/multi-out).
|
||||
indicators <- list(
|
||||
list(
|
||||
name = "SMA(20)",
|
||||
stream = function() {
|
||||
ind <- Sma(20)
|
||||
for (i in seq_len(bars)) update(ind, close[i])
|
||||
},
|
||||
batch = function() {
|
||||
batch(Sma(20), close)
|
||||
}
|
||||
),
|
||||
list(
|
||||
name = "ATR(14)",
|
||||
stream = function() {
|
||||
ind <- Atr(14)
|
||||
for (i in seq_len(bars)) {
|
||||
update(ind, open[i], high[i], low[i], close[i], volume[i], timestamp[i])
|
||||
}
|
||||
},
|
||||
batch = function() {
|
||||
batch(Atr(14), open, high, low, close, volume, timestamp)
|
||||
}
|
||||
),
|
||||
list(
|
||||
name = "MACD(12,26,9)",
|
||||
stream = function() {
|
||||
ind <- MacdIndicator(12, 26, 9)
|
||||
for (i in seq_len(bars)) update(ind, close[i])
|
||||
},
|
||||
batch = NULL # multi-output: streaming only
|
||||
)
|
||||
)
|
||||
|
||||
cat(sprintf(
|
||||
"Wickra R throughput - %s bars (median of 3 runs)\n\n",
|
||||
format(bars, big.mark = ",")
|
||||
))
|
||||
cat(sprintf("%-22s%20s%18s\n", "Indicator", "streaming (Mupd/s)", "batch (Mupd/s)"))
|
||||
cat(strrep("-", 60), "\n", sep = "")
|
||||
|
||||
for (ind in indicators) {
|
||||
stream_mups <- sprintf("%.1f", mups_from_ns(time_ns(ind$stream)))
|
||||
batch_mups <- if (is.null(ind$batch)) "-" else sprintf("%.1f", mups_from_ns(time_ns(ind$batch)))
|
||||
cat(sprintf("%-22s%20s%18s\n", ind$name, stream_mups, batch_mups))
|
||||
}
|
||||
|
||||
cat(paste0(
|
||||
"\nMupd/s = million indicator updates per second. Streaming is the per-tick\n",
|
||||
"`update` path crossing the R<->C-ABI boundary once per value; batch is the\n",
|
||||
"bulk vector path (one boundary crossing). Higher is better. Numbers are\n",
|
||||
"machine-dependent - use them for relative comparison, not as a speed claim.\n"
|
||||
))
|
||||
Reference in New Issue
Block a user