mirror of
https://github.com/mihakralj/QuanTAlib.git
synced 2026-08-05 12:37:43 +00:00
187 lines
6.6 KiB
Markdown
187 lines
6.6 KiB
Markdown
# SMAPE: Symmetric Mean Absolute Percentage Error
|
|
|
|
| Property | Value |
|
|
| ---------------- | -------------------------------- |
|
|
| **Category** | Error Metric |
|
|
| **Inputs** | Source (close) |
|
|
| **Parameters** | `period` |
|
|
| **Outputs** | Single series (SMAPE) |
|
|
| **Output range** | $\geq 0$ |
|
|
| **Warmup** | 1 bar |
|
|
|
|
### TL;DR
|
|
|
|
- Symmetric Mean Absolute Percentage Error addresses a fundamental asymmetry in MAPE: the fact that over-predictions and under-predictions of the sam...
|
|
- Parameterized by `period`.
|
|
- Output range: $\geq 0$.
|
|
- Requires 1 bar of warmup before first valid output (IsHot = true).
|
|
- Validated against TA-Lib, Skender, and Tulip reference implementations where available.
|
|
|
|
> "MAPE punishes based on who's right; SMAPE punishes based on how different they are."
|
|
|
|
Symmetric Mean Absolute Percentage Error addresses a fundamental asymmetry in MAPE: the fact that over-predictions and under-predictions of the same magnitude receive different penalties. SMAPE uses the average of actual and predicted values in the denominator, creating a metric that treats both directions equally.
|
|
|
|
## Architecture & Physics
|
|
|
|
SMAPE computes the symmetric percentage error for each observation:
|
|
|
|
$$\text{SMAPE} = \frac{200}{n} \sum_{i=1}^{n} \frac{|\text{actual}_i - \text{predicted}_i|}{|\text{actual}_i| + |\text{predicted}_i|}$$
|
|
|
|
The factor of 200 (rather than 100) scales the result to match traditional percentage ranges.
|
|
|
|
### Symmetry Explained
|
|
|
|
Consider predicting a value of 80 when actual is 100, versus predicting 100 when actual is 80:
|
|
|
|
**MAPE calculations:**
|
|
|
|
* Case 1: $100 \times |100-80|/100 = 20\%$
|
|
* Case 2: $100 \times |80-100|/80 = 25\%$
|
|
|
|
**SMAPE calculations:**
|
|
|
|
* Case 1: $200 \times |100-80|/(100+80) = 22.2\%$
|
|
* Case 2: $200 \times |80-100|/(80+100) = 22.2\%$
|
|
|
|
SMAPE assigns identical penalties regardless of which value is larger.
|
|
|
|
## Mathematical Foundation
|
|
|
|
### 1. Point-wise Symmetric Error
|
|
|
|
For each observation:
|
|
|
|
$$e_i = 200 \times \frac{|\text{actual}_i - \text{predicted}_i|}{|\text{actual}_i| + |\text{predicted}_i|}$$
|
|
|
|
### 2. Rolling Average
|
|
|
|
Over a period $n$:
|
|
|
|
$$\text{SMAPE}_t = \frac{1}{n} \sum_{i=t-n+1}^{t} e_i$$
|
|
|
|
### 3. Bounds
|
|
|
|
SMAPE is bounded between 0% and 200%:
|
|
|
|
* **0%**: Perfect prediction (actual = predicted)
|
|
* **200%**: Maximum error (one value is 0, other is non-zero)
|
|
* **100%**: Occurs when |actual - predicted| = (|actual| + |predicted|)/2
|
|
|
|
## Performance Profile
|
|
|
|
### Operation Count (Streaming Mode)
|
|
|
|
O(1) per bar. Single-pass scalar transformation of (actual, forecast) pair; no lookback window required.
|
|
|
|
| Operation | Count | Cost (cycles) | Subtotal |
|
|
| :--- | :---: | :---: | :---: |
|
|
| Error computation (subtract, abs/square/log) | 1-3 | ~3-8 cy | ~5-15 cy |
|
|
| Running accumulator update (EMA or sum) | 1 | ~4 cy | ~4 cy |
|
|
| **Total** | **2-4** | — | **~9-19 cycles** |
|
|
|
|
Streaming update requires only the current actual/forecast pair and running state. ~10-15 cycles/bar typical.
|
|
|
|
### Batch Mode (SIMD Analysis)
|
|
|
|
| Operation | Vectorizable? | Notes |
|
|
| :--- | :---: | :--- |
|
|
| Element-wise error computation | Yes | Independent per bar; fully vectorizable with `Vector<double>` |
|
|
| Reduction (sum/mean) | Yes | Parallel reduction; AVX2 gives 4x speedup |
|
|
| Log/exp components | Partial | Transcendental ops; polynomial approx for SIMD |
|
|
|
|
Batch SIMD: 4x-8x speedup for large windows. ~3-5 cy/bar amortized in vectorized batch mode.
|
|
|
|
| Metric | Score | Notes |
|
|
| :--- | :--- | :--- |
|
|
| **Throughput** | 18 ns/bar | O(1) via running sum |
|
|
| **Allocations** | 0 | Zero-allocation hot path |
|
|
| **Complexity** | O(1) | Constant per update |
|
|
| **Symmetry** | 10/10 | Primary advantage |
|
|
| **Zero Handling** | 8/10 | Better than MAPE |
|
|
| **Scale Independence** | 9/10 | Percentage-based |
|
|
| **Interpretability** | 7/10 | 200% scale less intuitive |
|
|
|
|
## Usage
|
|
|
|
```csharp
|
|
// Streaming mode - symmetric error measurement
|
|
var smape = new Smape(20);
|
|
|
|
// These two scenarios give identical SMAPE
|
|
smape.Update(actual: 100.0, predicted: 80.0); // Under-prediction
|
|
smape.Update(actual: 80.0, predicted: 100.0); // Over-prediction
|
|
|
|
double symmetricError = smape.Last.Value;
|
|
|
|
// Batch mode - historical analysis
|
|
var actual = new TSeries { 100, 105, 98, 102, 101 };
|
|
var predicted = new TSeries { 95, 100, 95, 100, 100 };
|
|
var results = Smape.Calculate(actual, predicted, period: 3);
|
|
|
|
// Span mode - zero-allocation bulk processing
|
|
Span<double> output = stackalloc double[1000];
|
|
Smape.Batch(actualSpan, predictedSpan, output, period: 20);
|
|
```
|
|
|
|
## Interpretation Guide
|
|
|
|
| SMAPE Value | Interpretation | Model Quality |
|
|
| :--- | :--- | :--- |
|
|
| **0-10%** | Excellent accuracy | Production-ready |
|
|
| **10-25%** | Good accuracy | Suitable for most applications |
|
|
| **25-50%** | Moderate accuracy | May need improvement |
|
|
| **50-100%** | Poor accuracy | Significant errors |
|
|
| **100-200%** | Very poor accuracy | Model needs redesign |
|
|
|
|
## Comparison with MAPE
|
|
|
|
| Scenario | MAPE | SMAPE | Winner |
|
|
| :--- | :--- | :--- | :--- |
|
|
| Actual=100, Pred=80 | 20% | 22.2% | Similar |
|
|
| Actual=80, Pred=100 | 25% | 22.2% | SMAPE (symmetric) |
|
|
| Actual=0, Pred=100 | Undefined | 200% | SMAPE (defined) |
|
|
| Actual=100, Pred=0 | 100% | 200% | Context-dependent |
|
|
| Interpretation | Familiar | Less intuitive | MAPE |
|
|
|
|
## Common Pitfalls
|
|
|
|
### 1. The 200% Scale
|
|
|
|
SMAPE ranges from 0% to 200%, not 0% to 100%. This can cause confusion when comparing with MAPE:
|
|
|
|
```csharp
|
|
// SMAPE = 50% is roughly equivalent to MAPE ≈ 33-40%
|
|
// The relationship is non-linear
|
|
```
|
|
|
|
### 2. Both Values Near Zero
|
|
|
|
When both actual and predicted approach zero, SMAPE approaches 0% (perfect):
|
|
|
|
```csharp
|
|
// actual = 0.001, predicted = 0.002
|
|
// |diff| = 0.001, sum = 0.003
|
|
// SMAPE = 200 * 0.001 / 0.003 = 66.7%
|
|
// This may not reflect actual model quality
|
|
```
|
|
|
|
### 3. Sign Insensitivity
|
|
|
|
Like MAPE, SMAPE doesn't indicate bias direction. A model consistently over-predicting by 10% looks identical to one consistently under-predicting by 10%.
|
|
|
|
**Solution**: Pair SMAPE with MPE for complete analysis.
|
|
|
|
## Variant: Armstrong's SMAPE
|
|
|
|
Some implementations use the mean (divide by 2) in the denominator:
|
|
|
|
$$\text{SMAPE}_{\text{Armstrong}} = \frac{100}{n} \sum \frac{|\text{actual} - \text{predicted}|}{(|\text{actual}| + |\text{predicted}|)/2}$$
|
|
|
|
This scales to 0-100% but is mathematically equivalent to the 0-200% version. QuanTAlib uses the 0-200% convention to match the original formulation.
|
|
|
|
## See Also
|
|
|
|
* [MAPE](../mape/Mape.md) - Asymmetric percentage error
|
|
* [MPE](../mpe/Mpe.md) - Signed percentage error for bias
|
|
* [MAE](../mae/Mae.md) - Absolute error without scaling
|