QuanTAlib/lib/statistics/zscore/Zscore.md

ZSCORE: Z-Score (Population Standard Score, also known as STANDARDIZE)

Property	Value
Category	Statistic
Inputs	Source (close)
Parameters	period (default 14)
Outputs	Single series (Zscore)
Output range	Unbounded
Warmup	period bars

TL;DR

• The Z-Score measures how many population standard deviations a value lies from the rolling mean over a lookback window.
• Parameterized by period (default 14).
• Output range: Unbounded.
• Requires period bars of warmup before first valid output (IsHot = true).
• Validated against manual computation, PineScript parity, and statistical invariants.

"How far from normal is this?" — Every risk manager, every day.

Introduction

The Z-Score measures how many population standard deviations a value lies from the rolling mean over a lookback window. ZSCORE is the canonical implementation for z-score standardization in QuanTAlib (the former Standardize indicator, which used sample standard deviation with N-1, has been consolidated into this indicator). ZSCORE uses population standard deviation, matching the PineScript ta.zscore convention. Output is unbounded, typically ranging from -3 to +3 for normally distributed data. A z-score of 0 means the value equals the window mean; ±2 flags statistical outliers at the 95% level.

Historical Context

The z-score originates from Karl Pearson's work in the 1890s on the theory of statistics. It transforms any distribution into units of standard deviation, making cross-series comparison possible. In trading, z-scores power mean-reversion strategies (enter when |z| > 2, exit when |z| < 0.5), pairs trading (z-score of spread), and anomaly detection. The population variant (N denominator) is standard in PineScript and most trading platforms because the rolling window IS the population of interest — not a sample from a larger population.

Architecture and Physics

1. Core Formula

z = \frac{x - \mu}{\sigma}

where:

• \mu = \frac{1}{N} \sum_{i=1}^{N} x_i (population mean over window)
• \sigma = \sqrt{\frac{1}{N} \sum_{i=1}^{N} (x_i - \mu)^2} (population standard deviation)

2. Computational Form

Using the identity \text{Var}(X) = E[X^2] - (E[X])^2:

\sigma = \sqrt{\frac{\sum x_i^2}{N} - \left(\frac{\sum x_i}{N}\right)^2}

This avoids a two-pass algorithm. One pass computes both \sum x_i and \sum x_i^2.

3. Edge Cases

Condition	Result
N < 2	0.0
\sigma < 10^{-10}	0.0 (constant data)
Input is NaN/Infinity	Substitute last valid value
Negative variance (floating-point)	Clamp to 0.0

4. Population vs Sample

Variant	Denominator	Use Case
ZSCORE (this)	N	Rolling window IS the population
Standardize (removed, consolidated into ZSCORE)	N - 1	Window is sample from larger population

Relationship: z_{\text{pop}} = z_{\text{sample}} \cdot \sqrt{\frac{N}{N-1}}

5. State Management

Uses RingBuffer for the sliding window. State rollback via record struct State with _s/_ps pattern for bar correction support.

Mathematical Foundation

Z-Score Derivation

Given a window of N values \{x_1, x_2, \ldots, x_N\}:

\mu = \frac{1}{N} \sum_{i=1}^{N} x_i \sigma^2 = \frac{1}{N} \sum_{i=1}^{N} (x_i - \mu)^2 = \frac{1}{N} \sum_{i=1}^{N} x_i^2 - \mu^2 z = \frac{x_N - \mu}{\sigma}

Scale Invariance

For any linear transform y = ax + b where a > 0:

z(y) = \frac{(ax + b) - (a\mu + b)}{a\sigma} = \frac{x - \mu}{\sigma} = z(x)

Z-scores are invariant under positive linear transformations. This property makes them ideal for comparing series measured in different units.

Performance Profile

Operation Count (per Update)

Operation	Count
Additions	N (sum scan)
Multiplications	N (sumSq scan)
Division	3
Square root	1
Comparison	2

Complexity

Method	Time	Space
Update	O(N)	O(1) auxiliary
Batch(Span)	O(N \cdot P)	stackalloc or ArrayPool

Quality Metrics

Metric	Score
Accuracy	9/10
Numerical stability	8/10
Memory efficiency	9/10
SIMD potential	Limited (sequential dependency on current value)

Validation

Library	Status	Notes
Manual	Verified	Known-value tests match hand computation
Standardize (consolidated)	Cross-validated	z_{\text{pop}} = z_{\text{sample}} \cdot \sqrt{N/(N-1)} holds
PineScript	Formula match	Population stddev, same edge-case handling

Common Pitfalls

1. Population vs sample confusion. ZSCORE uses N denominator. The former Standardize indicator (now consolidated into ZSCORE) used N-1. The difference matters for small windows: at period=5, the ratio is \sqrt{5/4} = 1.118, an 11.8% discrepancy.

2. Assuming normality. Z-scores measure distance in sigma units but don't guarantee the underlying distribution is normal. Fat-tailed financial returns make |z| > 3 more common than the 0.3% a normal distribution predicts.

3. Constant data edge case. When all values in the window are identical, \sigma = 0 and division is undefined. Implementation returns 0.0.

4. Floating-point variance. The formula E[X^2] - (E[X])^2 can produce tiny negative values due to floating-point arithmetic. Clamped to zero before taking square root.

5. Warmup period. Requires at least 2 data points for meaningful output. During warmup (N < 2), returns 0.0.

6. NaN propagation. Non-finite inputs are substituted with the last valid value to prevent NaN from contaminating the rolling statistics.

References

• Pearson, K. (1894). "Contributions to the Mathematical Theory of Evolution." Philosophical Transactions of the Royal Society.
• TradingView PineScript Reference: ta.zscore
• Bollinger, J. (2001). Bollinger on Bollinger Bands. McGraw-Hill. (Z-score normalization of Bollinger %B)