QuanTAlib/lib/trends_IIR/yzvama/Yzvama.md

# YZVAMA: Yang-Zhang Volatility Adjusted Moving Average

> "ATR tells you how much the market moved. Yang-Zhang tells you how much it *should* have moved given the gaps and intrabar action. YZVAMA uses that distinction to know when the market is lying about its volatility."

## The Core Insight

Most adaptive moving averages measure volatility using close-to-close changes (standard deviation) or high-low ranges (ATR). Both approaches miss a critical market dynamic: overnight gaps. A stock that gaps up 5% at the open but closes unchanged shows zero close-to-close volatility, yet anyone trading that day felt every point of that 5% move.

YZVAMA solves this by using Yang-Zhang volatility, a gap-aware OHLC-based estimator that properly accounts for overnight and intrabar components. But here's the twist: instead of using the raw volatility level to adjust smoothing (which breaks when volatility regimes shift), YZVAMA uses the *percentile rank* of current volatility within its recent history.

The result: adaptation that works regardless of whether you're trading a 10% daily volatility crypto or a 0.5% daily volatility bond ETF. The scale is always "where does current volatility sit within recent experience" rather than "how many ATR units are we moving."

## Historical Context

The Yang-Zhang estimator was introduced by Dennis Yang and Qiang Zhang in their 2000 paper "Drift-Independent Volatility Estimation Based on High, Low, Open, and Close Prices." Their key insight was decomposing total volatility into three components:

1. **Overnight variance** (close-to-open)
2. **Open-to-close variance** (intraday drift)
3. **Rogers-Satchell variance** (intrabar range without drift assumption)

Previous estimators either ignored gaps (Parkinson, Garman-Klass) or required drift estimation (classical). Yang-Zhang achieves minimum variance among all estimators using only OHLC data without assuming zero drift.

YZVAMA extends this by recognizing that volatility levels mean nothing in isolation. A 2% daily move might be panic in treasuries but a quiet Tuesday in biotech. By percentile-ranking volatility within its own history, YZVAMA creates a universal adaptation signal.

## Architecture

YZVAMA consists of four interconnected subsystems:

### 1. Yang-Zhang Variance Engine

Each bar produces a daily variance proxy using log returns:

```text
r_overnight = ln(Open / Close_prev)      # Gap component
r_close     = ln(Close / Open)           # Intraday drift
r_high      = ln(High / Open)            # Upper excursion
r_low       = ln(Low / Open)             # Lower excursion
```

The Rogers-Satchell term captures intrabar range without drift assumption:

$$\sigma_{RS}^2 = r_h(r_h - r_c) + r_l(r_l - r_c)$$

The combined estimator:

$$\sigma^2_{daily} = \sigma^2_{overnight} + k \cdot \sigma^2_{close} + (1-k) \cdot \sigma^2_{RS}$$

where $k$ is the Yang-Zhang weighting constant optimized for minimum variance:

$$k = \frac{0.34}{1.34 + \frac{n+1}{n-1}}$$

### 2. Bias-Compensated RMA Smoothing

The daily variance proxy is smoothed using RMA (Wilder's exponential average) with bias compensation:

$$\alpha = \frac{1}{\text{period}}$$

$$RMA_t = \alpha \cdot \sigma^2_t + (1 - \alpha) \cdot RMA_{t-1}$$

$$e_t = (1 - \alpha)^t$$

$$RMA_{compensated} = \frac{RMA_{raw}}{1 - e_t}$$

The bias compensation prevents the typical EMA startup distortion where early values are systematically biased toward zero.

Short-term YZV ($\sqrt{RMA_{short}}$) captures current volatility state. Long-term YZV ($\sqrt{RMA_{long}}$) provides historical reference (maintained for PineScript parity though not used in percentile calculation).

### 3. Percentile Rank Calculator

The percentile rank places current short-term YZV within its recent distribution:

```text
percentile = (count of historical YZV values < current YZV) / (total count - 1) × 100
```

A circular buffer stores the last `percentileLookback` YZV readings. On each bar, the buffer is sorted and binary search locates the current value's rank. This produces a 0-100 score indicating where current volatility sits relative to recent history.

### 4. Dynamic SMA Calculator

The percentile maps linearly to an adjusted SMA length:

$$\text{adjustedLength} = \text{maxLength} - \frac{\text{percentile}}{100} \times (\text{maxLength} - \text{minLength})$$

| Percentile | Interpretation | Adjusted Length |
|------------|----------------|-----------------|
| 0 (lowest volatility) | Quiet market | maxLength (smoothest) |
| 50 (median volatility) | Normal conditions | (maxLength + minLength) / 2 |
| 100 (highest volatility) | Extreme activity | minLength (fastest) |

A circular buffer holds recent source values, and SMA is computed over the dynamically chosen window by iterating backwards from the most recent entry.

## Mathematical Foundation

### Yang-Zhang Variance Components

Given OHLC data and previous close $C_{t-1}$:

**Overnight component:**
$$\sigma^2_o = \left(\ln\frac{O_t}{C_{t-1}}\right)^2$$

**Close-to-close component:**
$$\sigma^2_c = \left(\ln\frac{C_t}{O_t}\right)^2$$

**Rogers-Satchell component:**
$$\sigma^2_{RS} = \ln\frac{H_t}{O_t}\left(\ln\frac{H_t}{O_t} - \ln\frac{C_t}{O_t}\right) + \ln\frac{L_t}{O_t}\left(\ln\frac{L_t}{O_t} - \ln\frac{C_t}{O_t}\right)$$

**Combined daily variance:**
$$\sigma^2_t = \sigma^2_o + k \cdot \sigma^2_c + (1-k) \cdot \sigma^2_{RS}$$

### Optimal k Derivation

Yang and Zhang derived the optimal weighting constant $k$ that minimizes the estimator's variance:

$$k = \frac{0.34}{1.34 + \frac{n+1}{n-1}}$$

For typical period values:

| Period | k Value |
|--------|---------|
| 3 | 0.113 |
| 10 | 0.133 |
| 50 | 0.160 |
| 100 | 0.165 |

The constant 0.34/1.34 comes from the theoretical ratio of overnight to intraday variance assuming continuous trading.

### Percentile Rank Properties

The percentile transformation provides several desirable properties:

1. **Scale invariance**: Works identically whether volatility is 0.1% or 10%
2. **Regime adaptation**: Automatically recalibrates as volatility regimes shift
3. **Bounded output**: Always produces 0-100 regardless of input distribution
4. **Non-parametric**: Makes no assumptions about volatility distribution shape

## Parameters

| Parameter | Default | Valid Range | Purpose |
|-----------|---------|-------------|---------|
| `yzvShortPeriod` | 3 | > 0 | RMA period for short-term YZV (current volatility) |
| `yzvLongPeriod` | 50 | > 0 | RMA period for long-term YZV (PineScript parity) |
| `percentileLookback` | 100 | > 0 | Window for percentile rank calculation |
| `minLength` | 5 | > 0, ≤ maxLength | Minimum SMA length (high volatility) |
| `maxLength` | 100 | ≥ minLength | Maximum SMA length (low volatility) |

### Parameter Selection Guidelines

**yzvShortPeriod (default 3):**
Short periods (2-5) make YZVAMA highly reactive to volatility spikes. Longer periods (10-20) smooth out single-bar volatility anomalies. The short period should be significantly less than the percentile lookback.

**percentileLookback (default 100):**
Determines the "memory" for what constitutes normal volatility. 100 bars provides roughly 4 months of daily data context. Shorter lookbacks (50) adapt faster to new regimes; longer lookbacks (200) provide more stable percentile rankings.

**minLength / maxLength (default 5/100):**
The ratio determines adaptation intensity. A 5/100 ratio (20:1) creates dramatic smoothing differences between quiet and volatile markets. A 10/50 ratio (5:1) produces more moderate adaptation.

## Implementation Notes

### Complexity Analysis

| Operation | Complexity | Notes |
|-----------|------------|-------|
| YZ variance | O(1) | Log returns and arithmetic |
| RMA updates | O(1) | Recursive smoothing |
| Buffer insertion | O(1) | Circular buffer |
| Percentile sort | O(n log n) | Where n = percentileLookback |
| SMA calculation | O(adjustedLength) | Sum over dynamic window |

The percentile calculation dominates at O(n log n) per bar. For `percentileLookback = 100`, this adds approximately 600-700 comparisons. Still fast enough for real-time use, but noticeably slower than pure O(1) indicators.

### Memory Layout

- `YzvamaState` struct: RMA states, buffer heads, running sums (~64 bytes)
- Source circular buffer: `double[maxLength]` (800 bytes at default)
- YZV circular buffer: `double[percentileLookback]` (800 bytes at default)
- Work array for sorting: `double[percentileLookback]` (800 bytes)
- State copies for bar correction: Duplicate of above

Total footprint approximately 5KB at default parameters.

### Bar Correction (isNew=false)

YZVAMA supports bar correction by maintaining previous state (`_p_state`, `_p_sourceBuffer`, `_p_yzvBuffer`). When `isNew=false`, all state rolls back before recalculation. This handles real-time bar updates where the current bar's OHLC changes before bar close.

### Single-Value Input Limitation

YZVAMA requires OHLC data for proper Yang-Zhang volatility calculation. When fed single values (TValue), a synthetic bar is created with O=H=L=C. This produces:

- Zero overnight variance (no gap)
- Zero Rogers-Satchell variance (no range)
- Zero close variance (O=C)

Result: YZV = 0 for all bars, percentile undefined, and adjusted length defaults toward center of range. **For meaningful volatility adaptation, use TBar input.**

## Performance Profile

### Operation Count (Streaming Mode)

YZVAMA has four computational phases: YZ variance, RMA smoothing, percentile ranking, and dynamic SMA.

**Phase 1: Yang-Zhang Variance (per bar)**

| Operation | Count | Cost (cycles) | Subtotal |
| :--- | :---: | :---: | :---: |
| LOG (4 log returns) | 4 | 40 | 160 |
| MUL (squares, products) | 6 | 3 | 18 |
| SUB (differences) | 4 | 1 | 4 |
| ADD (combination) | 3 | 1 | 3 |
| **Phase 1 subtotal** | **17** | — | **~185 cycles** |

**Phase 2: Dual RMA Smoothing**

| Operation | Count | Cost (cycles) | Subtotal |
| :--- | :---: | :---: | :---: |
| FMA (short RMA) | 1 | 4 | 4 |
| FMA (long RMA) | 1 | 4 | 4 |
| MUL (compensator ×2) | 2 | 3 | 6 |
| DIV (bias correction ×2) | 2 | 15 | 30 |
| SQRT (YZV from variance) | 1 | 15 | 15 |
| **Phase 2 subtotal** | **7** | — | **~59 cycles** |

**Phase 3: Percentile Ranking (O(n log n) where n = percentileLookback)**

| Operation | Count | Cost (cycles) | Subtotal |
| :--- | :---: | :---: | :---: |
| Array copy | n | 1 | n |
| SORT (comparison-based) | n log n | ~1 | n log n |
| Binary search | log n | ~3 | 3 log n |
| DIV (rank / count) | 1 | 15 | 15 |
| **Phase 3 subtotal** | — | — | **~n log n + n + 15** |

For n = 100: ~100 × 6.6 + 100 + 15 ≈ **~775 cycles**.

**Phase 4: Dynamic SMA (O(L) where L = adjustedLength)**

| Operation | Count | Cost (cycles) | Subtotal |
| :--- | :---: | :---: | :---: |
| MUL/SUB (percentile → length) | 3 | 3 | 9 |
| ADD (sum L values) | L | 1 | L |
| DIV (sum / L) | 1 | 15 | 15 |
| **Phase 4 subtotal** | **4 + L** | — | **~24 + L cycles** |

**Total per bar:** ~185 + 59 + 775 + 24 + L ≈ **~1043 + L cycles** (n=100, typical L=20).

| Component | Cycles | % of Total |
| :--- | :---: | :---: |
| YZ variance (4 LOGs) | ~185 | 17% |
| RMA + SQRT | ~59 | 6% |
| Percentile sort (n=100) | ~775 | 73% |
| Dynamic SMA (L=20) | ~44 | 4% |
| **Total** | **~1063** | 100% |

**Dominant cost:** Percentile sort at O(n log n) accounts for ~73% of computation.

Post-warmup (no bias correction): subtract ~30 cycles → **~1033 cycles/bar**.

### Batch Mode (SIMD Analysis)

YZVAMA has limited SIMD potential due to recursive components and sort:

| Component | SIMD Potential | Notes |
| :--- | :--- | :--- |
| YZ variance | Partial | 4 LOGs could use SVML |
| RMA smoothing | None | Recursive IIR filter |
| Percentile sort | None | Comparison-based, not vectorizable |
| SMA summation | **Yes** | Horizontal sum of buffer |

| Optimization | Cycles Saved |
| :--- | :---: |
| SIMD LOG (4 values) | ~120 cycles (160 → 40) |
| SIMD SMA sum (L=32) | ~24 cycles |
| **Total potential** | ~144 cycles (~14% improvement) |

### Benchmark Results

| Metric | Value | Notes |
| :--- | :--- | :--- |
| **Throughput** | ~2M bars/sec | TBar input, includes sort overhead |
| **Allocations** | 0 bytes | Hot path allocation-free (reuses work array) |
| **Complexity** | O(n log n + L) | n = percentileLookback, L = adjustedLength |
| **Warmup** | max(yzvLongPeriod, maxLength, percentileLookback) | All components must fill |
| **State Size** | ~5 KB | Buffers + work array at default params |

### Quality Metrics

| Metric | Score | Notes |
| :--- | :---: | :--- |
| **Accuracy** | 8/10 | Faithful to price within adaptive window |
| **Timeliness** | 9/10 | Accelerates dramatically during volatility spikes |
| **Overshoot** | 7/10 | SMA-based, no overshoot by construction |
| **Smoothness** | 8/10 | Smooth in low-volatility regimes |

## Validation

| Library | Status | Notes |
|:---|:---|:---|
| **PineScript** | ✅ | Reference implementation matches |
| **TA-Lib** | N/A | Not implemented |
| **Skender** | N/A | Not implemented |
| **Tulip** | N/A | Not implemented |
| **Ooples** | N/A | Not implemented |

YZVAMA is a novel indicator without widespread implementation. Validation is performed against the PineScript reference implementation in `yzvama.pine`.

## Usage Patterns

### Basic Usage (Recommended)

```csharp
var yzvama = new Yzvama(yzvShortPeriod: 3, percentileLookback: 100, minLength: 5, maxLength: 100);

foreach (var bar in bars)
{
    var result = yzvama.Update(bar, isNew: true);
    // result.Value contains the volatility-adjusted average
}
```

### Batch Processing

```csharp
// Process entire bar series
var results = Yzvama.Batch(barSeries, yzvShortPeriod: 3, percentileLookback: 100);
```

### Event-Driven Chaining

```csharp
var source = new TBarSeries();
var yzvama = new Yzvama(source, yzvShortPeriod: 3);

// YZVAMA subscribes to source.Pub events
source.Add(new TBar(...));  // Triggers YZVAMA update
```

### Custom Source Value

```csharp
// Use High instead of Close as the smoothed value
foreach (var bar in bars)
{
    var result = yzvama.Update(bar, sourceValue: bar.High, isNew: true);
}
```

## Common Pitfalls

1. **Using TValue input**: YZVAMA needs OHLC for Yang-Zhang volatility. Single values produce zero YZV and disable meaningful adaptation. The indicator will still work, but the adaptive mechanism is defeated.

2. **Short percentileLookback**: With lookback < 50, percentile rankings become unstable. Single outlier days can dominate the distribution. Use at least 100 for daily data.

3. **Ignoring warmup**: YZVAMA has significant warmup requirements (max of all period parameters). Early values before `IsHot` are approximations based on incomplete history.

4. **Expecting trend following**: YZVAMA adapts to volatility, not trend direction. High volatility could mean a strong trend *or* chaotic whipsaws. It provides faster response in active markets, not directional guidance.

5. **Over-optimization**: The default parameters work across diverse instruments because percentile ranking is inherently adaptive. Excessive parameter tuning often indicates overfitting to historical data.

## Comparison with Alternatives

| Indicator | Volatility Measure | Adaptation Mechanism | Gap-Aware |
|-----------|-------------------|---------------------|-----------|
| **YZVAMA** | Yang-Zhang (OHLC) | Percentile rank → SMA length | Yes |
| **VAMA** | ATR (True Range) | Volatility ratio → SMA length | Partial |
| **KAMA** | Efficiency Ratio | Directional efficiency → EMA alpha | No |
| **VIDYA** | CMO | Momentum strength → EMA alpha | No |
| **JMA** | Proprietary | Multi-stage adaptive filter | No |

YZVAMA's unique contribution is the combination of:

1. **Gap-aware volatility** via Yang-Zhang (vs ATR's partial gap handling)
2. **Percentile normalization** (vs raw volatility ratios that break across regimes)
3. **Dynamic SMA length** (vs dynamic EMA alpha approaches)

The percentile approach means YZVAMA works identically whether applied to a 0.3% daily volatility instrument or a 5% daily volatility one. No parameter adjustment required when switching asset classes.

## Theoretical Foundations

### Why Yang-Zhang Over Alternatives?

| Estimator | Gap Handling | Drift Assumption | Efficiency |
|-----------|--------------|------------------|------------|
| Close-to-close | None | None | 1.0 (baseline) |
| Parkinson (H-L) | None | Zero drift | 5.2× |
| Garman-Klass | None | Zero drift | 7.4× |
| Rogers-Satchell | None | Any drift | 6.2× |
| Yang-Zhang | Full | Any drift | 8.1× |

Yang-Zhang achieves the highest efficiency (minimum variance for given sample size) among all OHLC-based estimators while properly handling both gaps and non-zero drift. The 8.1× efficiency means YZV extracts as much information from 1 bar as close-to-close volatility extracts from 8 bars.

### Why Percentile Over Ratio?

Ratio-based approaches (e.g., short_vol / long_vol) have two problems:

1. **Scale sensitivity**: A ratio of 2.0 means different things at different volatility levels
2. **Regime breaks**: During regime changes, ratios can produce extreme values

Percentile ranking solves both:

1. **Scale invariant**: 75th percentile means the same thing at any volatility level
2. **Bounded**: Output always in [0, 100] regardless of input extremes

## References

- Yang, D., & Zhang, Q. (2000). "Drift-Independent Volatility Estimation Based on High, Low, Open, and Close Prices." *Journal of Business*, 73(3), 477-491.
- Rogers, L.C.G., & Satchell, S.E. (1991). "Estimating Variance from High, Low and Closing Prices." *Annals of Applied Probability*, 1(4), 504-512.
- PineScript reference implementation: `yzvama.pine`