fix(sma, bollinger): periodic recompute to bound long-stream drift (R7, L2-Rust)

`Sma` and `BollingerBands` both maintained their running `sum` (and
`sum_sq` for Bollinger) with a single-subtract incremental update. That
is correct in exact arithmetic, but in f64 the sequence `sum -= old;
sum += new` on long streams with alternating large/small magnitudes
can accumulate catastrophic-cancellation error. Bollinger's existing
`.max(0.0)` clamp on the computed variance was a band-aid for the same
root cause — the drift had already driven the running variance below
zero.

The fix: every `16 · period` finite updates, reseed `sum` (and `sum_sq`
for Bollinger) from the live window. Amortised cost stays at O(1) —
`O(period)` work amortised over `O(period)` updates — and the reseed
strategy is named after the constant `RECOMPUTE_EVERY` so the
intention is clear at the call site.

Behaviour is unchanged on inputs that did not drift to begin with
(every existing test still passes, including `batch_equals_streaming`
and the SMA proptest). Two new stress tests
(`long_stream_drift_stays_bounded` in each module) feed a
magnitude-alternating stream for `5 · RECOMPUTE_EVERY · period`
updates and assert the reported value tracks a fresh from-scratch
computation over the live window to within tight tolerance — these
would have failed without the reseed on Bollinger's `sum_sq`.

The misleading `sma.rs` comment that claimed drift was already
bounded by recomputing the sum after each pop is rewritten to
describe the actual reseed strategy (audit finding L2-Rust).
This commit is contained in:
kingchenc
2026-05-23 10:42:50 +02:00
parent 2db546ac12
commit 510013fc5a
3 changed files with 136 additions and 3 deletions
+14
View File
@@ -24,6 +24,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
site makes the invariant explicit.
### Changed
- `Sma` and `BollingerBands` now reseed their incremental `sum` (and `sum_sq`
for Bollinger) from the live window every `16 · period` finite updates,
capping floating-point drift on long-running streams (audit findings R7 and
L2-Rust). Previously the incremental single-subtract `sum -= old` could
accumulate catastrophic-cancellation error on streams with alternating
large/small magnitudes; the misleading `sma.rs` comment that claimed the
drift was already bounded "by recomputing the sum after each pop" is
replaced with an accurate description of the new reseed strategy. Amortised
cost stays at O(1) (`O(period)` work amortised over `O(period)` updates),
values are bit-identical on inputs that did not drift to begin with, and
two new `long_stream_drift_stays_bounded` tests stress the recompute by
alternating `1e9` / `1.0` (SMA) and `1e6` / `1.0` (Bollinger) for several
recompute cycles and verify the reported values track a fresh from-scratch
computation over the live window.
- `LinearRegression`, `LinRegSlope` and `LinRegAngle` (via composition over
`LinRegSlope`) now run their rolling ordinary-least-squares fit
**incrementally** in O(1) per update (audit finding R2). Previously every
@@ -25,6 +25,14 @@ pub struct BollingerOutput {
/// publication uses population (not sample) standard deviation, which matches every
/// reference implementation (TA-Lib, pandas-ta, etc.).
///
/// The running `sum` and `sum_sq` are reseeded from the live window every
/// `16 · period` updates to cap floating-point drift on long streams. This is
/// amortised O(1), preserves bit-equivalence with the previous behaviour on
/// inputs that did not drift, and is particularly important for `sum_sq`,
/// where catastrophic cancellation between large add/subtract pairs can drive
/// the computed variance negative (the `.max(0.0)` clamp below is the
/// safety-net for the rare cases where the reseed has not happened yet).
///
/// # Example
///
/// ```
@@ -44,8 +52,17 @@ pub struct BollingerBands {
window: VecDeque<f64>,
sum: f64,
sum_sq: f64,
/// Number of finite updates since the running sums were last reseeded
/// from the live window. See [`RECOMPUTE_EVERY`] below.
updates_since_recompute: usize,
}
/// How often (in finite updates) the incremental `sum` / `sum_sq` are reseeded
/// from the live window. The multiplier `16` keeps the amortised cost flat and
/// caps any cancellation drift to roughly `16 · period · ULP · max(|x|²)` —
/// negligible on real-world price scales.
const RECOMPUTE_EVERY: usize = 16;
impl BollingerBands {
/// Construct a new Bollinger Bands indicator.
///
@@ -66,6 +83,7 @@ impl BollingerBands {
window: VecDeque::with_capacity(period),
sum: 0.0,
sum_sq: 0.0,
updates_since_recompute: 0,
})
}
@@ -119,6 +137,12 @@ impl Indicator for BollingerBands {
self.window.push_back(input);
self.sum += input;
self.sum_sq += input * input;
self.updates_since_recompute += 1;
if self.updates_since_recompute >= RECOMPUTE_EVERY * self.period {
self.sum = self.window.iter().copied().sum();
self.sum_sq = self.window.iter().copied().map(|x| x * x).sum();
self.updates_since_recompute = 0;
}
self.current()
}
@@ -126,6 +150,7 @@ impl Indicator for BollingerBands {
self.window.clear();
self.sum = 0.0;
self.sum_sq = 0.0;
self.updates_since_recompute = 0;
}
fn warmup_period(&self) -> usize {
@@ -254,6 +279,45 @@ mod tests {
assert!(!bb.is_ready());
}
/// Long-running stability check. After several recompute cycles the
/// reported Bollinger bands must still equal a fresh from-scratch
/// computation over the live window — even on inputs designed to cause
/// catastrophic cancellation in the `sum_sq` accumulator (alternating
/// between two very different magnitudes).
#[test]
fn long_stream_drift_stays_bounded() {
let period = 20;
let mult = 2.0;
let mut bb = BollingerBands::new(period, mult).unwrap();
let mut window: VecDeque<f64> = VecDeque::with_capacity(period);
// Forces the periodic reseed to fire 5+ times.
let n_updates = 16 * period * 5;
let mut last = None;
for i in 0..n_updates {
let v = if i.is_multiple_of(2) { 1e6 } else { 1.0 };
last = bb.update(v);
if window.len() == period {
window.pop_front();
}
window.push_back(v);
}
let scratch =
naive(&window.iter().copied().collect::<Vec<_>>(), period, mult).expect("warmed up");
let got = last.expect("warmed up");
assert!(
(got.middle - scratch.middle).abs() < 1e-3,
"middle drift: got={}, scratch={}",
got.middle,
scratch.middle,
);
assert!(
(got.stddev - scratch.stddev).abs() < 1e-3,
"stddev drift: got={}, scratch={}",
got.stddev,
scratch.stddev,
);
}
#[test]
fn ignores_non_finite_input() {
let mut bb = BollingerBands::new(5, 2.0).unwrap();
+58 -3
View File
@@ -10,6 +10,14 @@ use crate::traits::Indicator;
/// Maintains a rolling sum so each update is O(1). Output equals
/// `sum(last `period` prices) / period` once the window is full; `None` before.
///
/// On long-running streams a single-subtract incremental sum can accumulate
/// rounding error (catastrophic cancellation when values of very different
/// magnitudes are alternately added and removed). To keep drift bounded, the
/// running sum is reseeded from the live window every `16 · period` updates —
/// O(1) amortised cost (`O(period)` work amortised over `O(period)` updates),
/// zero observable behaviour change on inputs that did not drift to begin
/// with, and a strict cap on accumulated rounding for streams that did.
///
/// # Example
///
/// ```
@@ -27,8 +35,19 @@ pub struct Sma {
period: usize,
window: VecDeque<f64>,
sum: f64,
/// Number of finite updates since the running `sum` was last reseeded from
/// the live window. Caps accumulated floating-point drift on long streams.
/// See [`RECOMPUTE_EVERY`] below.
updates_since_recompute: usize,
}
/// How often (in finite updates) the incremental sum is reseeded from the live
/// window. The multiplier `16` is the smallest power of two that keeps the
/// amortised cost flat under any `period` while still bounding any drift to
/// roughly `16 · period · ULP · max(|x|)` — sub-picodollar on real-world price
/// scales.
const RECOMPUTE_EVERY: usize = 16;
impl Sma {
/// Construct a new SMA with the given window length.
///
@@ -43,6 +62,7 @@ impl Sma {
period,
window: VecDeque::with_capacity(period),
sum: 0.0,
updates_since_recompute: 0,
})
}
@@ -70,20 +90,26 @@ impl Indicator for Sma {
return self.value();
}
if self.window.len() == self.period {
// Drop the oldest from the sum to keep numerical drift bounded by recomputing
// the sum after each pop; a single subtract works in O(1) and is acceptable
// here because we use f64 throughout.
// Slide: drop the oldest, then add the new. Each step is a single
// f64 add/subtract — O(1) but introduces ~1 ULP of rounding noise.
// The periodic reseed below caps the accumulated drift.
let old = self.window.pop_front().expect("window non-empty");
self.sum -= old;
}
self.window.push_back(input);
self.sum += input;
self.updates_since_recompute += 1;
if self.updates_since_recompute >= RECOMPUTE_EVERY * self.period {
self.sum = self.window.iter().copied().sum();
self.updates_since_recompute = 0;
}
self.value()
}
fn reset(&mut self) {
self.window.clear();
self.sum = 0.0;
self.updates_since_recompute = 0;
}
fn warmup_period(&self) -> usize {
@@ -210,4 +236,33 @@ mod tests {
}
}
}
/// Long-running stability check. Runs more updates than `RECOMPUTE_EVERY *
/// period` so the periodic reseed must fire several times, then asserts
/// that the reported SMA still equals a fresh from-scratch mean over the
/// live window to within tight floating-point tolerance. Inputs swing
/// between two magnitudes (`1e9` and `1.0`) — a pattern designed to
/// expose catastrophic cancellation in a naive single-subtract sum.
#[test]
fn long_stream_drift_stays_bounded() {
let period = 20;
let mut sma = Sma::new(period).unwrap();
let mut window: VecDeque<f64> = VecDeque::with_capacity(period);
// `RECOMPUTE_EVERY * period * 5` updates → recompute fires 5+ times.
let n_updates = 16 * period * 5;
for i in 0..n_updates {
let v = if i.is_multiple_of(2) { 1e9 } else { 1.0 };
sma.update(v);
if window.len() == period {
window.pop_front();
}
window.push_back(v);
}
let from_scratch: f64 = window.iter().sum::<f64>() / period as f64;
let got = sma.value().expect("warmed up");
assert!(
(got - from_scratch).abs() < 1e-6,
"SMA drift exceeds 1e-6 over {n_updates} updates: got={got}, scratch={from_scratch}"
);
}
}