mirror of
https://github.com/mihakralj/QuanTAlib.git
synced 2026-08-03 03:47:42 +00:00
Miha Kralj
86fe32a682
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat> Co-authored-by: Warp <agent@warp.dev>
148 lines
4.8 KiB
Markdown
148 lines
4.8 KiB
Markdown
# SMAPE: Symmetric Mean Absolute Percentage Error
|
|
|
|
> "MAPE punishes based on who's right; SMAPE punishes based on how different they are."
|
|
|
|
Symmetric Mean Absolute Percentage Error addresses a fundamental asymmetry in MAPE: the fact that over-predictions and under-predictions of the same magnitude receive different penalties. SMAPE uses the average of actual and predicted values in the denominator, creating a metric that treats both directions equally.
|
|
|
|
## Architecture & Physics
|
|
|
|
SMAPE computes the symmetric percentage error for each observation:
|
|
|
|
$$\text{SMAPE} = \frac{200}{n} \sum_{i=1}^{n} \frac{|\text{actual}_i - \text{predicted}_i|}{|\text{actual}_i| + |\text{predicted}_i|}$$
|
|
|
|
The factor of 200 (rather than 100) scales the result to match traditional percentage ranges.
|
|
|
|
### Symmetry Explained
|
|
|
|
Consider predicting a value of 80 when actual is 100, versus predicting 100 when actual is 80:
|
|
|
|
**MAPE calculations:**
|
|
|
|
* Case 1: $100 \times |100-80|/100 = 20\%$
|
|
* Case 2: $100 \times |80-100|/80 = 25\%$
|
|
|
|
**SMAPE calculations:**
|
|
|
|
* Case 1: $200 \times |100-80|/(100+80) = 22.2\%$
|
|
* Case 2: $200 \times |80-100|/(80+100) = 22.2\%$
|
|
|
|
SMAPE assigns identical penalties regardless of which value is larger.
|
|
|
|
## Mathematical Foundation
|
|
|
|
### 1. Point-wise Symmetric Error
|
|
|
|
For each observation:
|
|
|
|
$$e_i = 200 \times \frac{|\text{actual}_i - \text{predicted}_i|}{|\text{actual}_i| + |\text{predicted}_i|}$$
|
|
|
|
### 2. Rolling Average
|
|
|
|
Over a period $n$:
|
|
|
|
$$\text{SMAPE}_t = \frac{1}{n} \sum_{i=t-n+1}^{t} e_i$$
|
|
|
|
### 3. Bounds
|
|
|
|
SMAPE is bounded between 0% and 200%:
|
|
|
|
* **0%**: Perfect prediction (actual = predicted)
|
|
* **200%**: Maximum error (one value is 0, other is non-zero)
|
|
* **100%**: Occurs when |actual - predicted| = (|actual| + |predicted|)/2
|
|
|
|
## Performance Profile
|
|
|
|
| Metric | Score | Notes |
|
|
| :--- | :--- | :--- |
|
|
| **Throughput** | 18 ns/bar | O(1) via running sum |
|
|
| **Allocations** | 0 | Zero-allocation hot path |
|
|
| **Complexity** | O(1) | Constant per update |
|
|
| **Symmetry** | 10/10 | Primary advantage |
|
|
| **Zero Handling** | 8/10 | Better than MAPE |
|
|
| **Scale Independence** | 9/10 | Percentage-based |
|
|
| **Interpretability** | 7/10 | 200% scale less intuitive |
|
|
|
|
## Usage
|
|
|
|
```csharp
|
|
// Streaming mode - symmetric error measurement
|
|
var smape = new Smape(20);
|
|
|
|
// These two scenarios give identical SMAPE
|
|
smape.Update(actual: 100.0, predicted: 80.0); // Under-prediction
|
|
smape.Update(actual: 80.0, predicted: 100.0); // Over-prediction
|
|
|
|
double symmetricError = smape.Last.Value;
|
|
|
|
// Batch mode - historical analysis
|
|
var actual = new TSeries { 100, 105, 98, 102, 101 };
|
|
var predicted = new TSeries { 95, 100, 95, 100, 100 };
|
|
var results = Smape.Calculate(actual, predicted, period: 3);
|
|
|
|
// Span mode - zero-allocation bulk processing
|
|
Span<double> output = stackalloc double[1000];
|
|
Smape.Batch(actualSpan, predictedSpan, output, period: 20);
|
|
```
|
|
|
|
## Interpretation Guide
|
|
|
|
| SMAPE Value | Interpretation | Model Quality |
|
|
| :--- | :--- | :--- |
|
|
| **0-10%** | Excellent accuracy | Production-ready |
|
|
| **10-25%** | Good accuracy | Suitable for most applications |
|
|
| **25-50%** | Moderate accuracy | May need improvement |
|
|
| **50-100%** | Poor accuracy | Significant errors |
|
|
| **100-200%** | Very poor accuracy | Model needs redesign |
|
|
|
|
## Comparison with MAPE
|
|
|
|
| Scenario | MAPE | SMAPE | Winner |
|
|
| :--- | :--- | :--- | :--- |
|
|
| Actual=100, Pred=80 | 20% | 22.2% | Similar |
|
|
| Actual=80, Pred=100 | 25% | 22.2% | SMAPE (symmetric) |
|
|
| Actual=0, Pred=100 | Undefined | 200% | SMAPE (defined) |
|
|
| Actual=100, Pred=0 | 100% | 200% | Context-dependent |
|
|
| Interpretation | Familiar | Less intuitive | MAPE |
|
|
|
|
## Common Pitfalls
|
|
|
|
### 1. The 200% Scale
|
|
|
|
SMAPE ranges from 0% to 200%, not 0% to 100%. This can cause confusion when comparing with MAPE:
|
|
|
|
```csharp
|
|
// SMAPE = 50% is roughly equivalent to MAPE ≈ 33-40%
|
|
// The relationship is non-linear
|
|
```
|
|
|
|
### 2. Both Values Near Zero
|
|
|
|
When both actual and predicted approach zero, SMAPE approaches 0% (perfect):
|
|
|
|
```csharp
|
|
// actual = 0.001, predicted = 0.002
|
|
// |diff| = 0.001, sum = 0.003
|
|
// SMAPE = 200 * 0.001 / 0.003 = 66.7%
|
|
// This may not reflect actual model quality
|
|
```
|
|
|
|
### 3. Sign Insensitivity
|
|
|
|
Like MAPE, SMAPE doesn't indicate bias direction. A model consistently over-predicting by 10% looks identical to one consistently under-predicting by 10%.
|
|
|
|
**Solution**: Pair SMAPE with MPE for complete analysis.
|
|
|
|
## Variant: Armstrong's SMAPE
|
|
|
|
Some implementations use the mean (divide by 2) in the denominator:
|
|
|
|
$$\text{SMAPE}_{\text{Armstrong}} = \frac{100}{n} \sum \frac{|\text{actual} - \text{predicted}|}{(|\text{actual}| + |\text{predicted}|)/2}$$
|
|
|
|
This scales to 0-100% but is mathematically equivalent to the 0-200% version. QuanTAlib uses the 0-200% convention to match the original formulation.
|
|
|
|
## See Also
|
|
|
|
* [MAPE](../mape/Mape.md) - Asymmetric percentage error
|
|
* [MPE](../mpe/Mpe.md) - Signed percentage error for bias
|
|
* [MAE](../mae/Mae.md) - Absolute error without scaling
|