mirror of
https://github.com/mihakralj/QuanTAlib.git
synced 2026-08-02 11:37:42 +00:00
Miha Kralj
86fe32a682
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat> Co-authored-by: Warp <agent@warp.dev>
142 lines
4.9 KiB
Markdown
142 lines
4.9 KiB
Markdown
# MPE: Mean Percentage Error
|
||
|
||
> "MAPE tells you how wrong you are; MPE tells you which direction you're wrong in."
|
||
|
||
Mean Percentage Error measures the average percentage difference between actual and predicted values while preserving the sign. Unlike MAPE, which takes absolute values, MPE reveals systematic bias in predictions—whether a model consistently over-predicts or under-predicts.
|
||
|
||
## Architecture & Physics
|
||
|
||
MPE computes the signed percentage error for each data point and averages over a rolling window:
|
||
|
||
$$\text{MPE} = \frac{100}{n} \sum_{i=1}^{n} \frac{(\text{actual}_i - \text{predicted}_i)}{\text{actual}_i}$$
|
||
|
||
The sign preservation makes MPE invaluable for bias detection:
|
||
|
||
* **Positive MPE**: Model systematically under-predicts (actual > predicted)
|
||
* **Negative MPE**: Model systematically over-predicts (actual < predicted)
|
||
* **MPE near zero**: No systematic bias (though individual errors may be large)
|
||
|
||
### Bias Detection
|
||
|
||
Consider a weather forecasting model:
|
||
|
||
* If MPE = +15%, the model consistently predicts temperatures 15% lower than actual
|
||
* If MPE = -10%, the model consistently predicts temperatures 10% higher than actual
|
||
* If MPE ≈ 0% but MAPE = 20%, errors cancel out (no bias) but magnitude is still significant
|
||
|
||
## Mathematical Foundation
|
||
|
||
### 1. Point-wise Percentage Error
|
||
|
||
For each observation:
|
||
|
||
$$e_i = 100 \times \frac{\text{actual}_i - \text{predicted}_i}{\text{actual}_i}$$
|
||
|
||
### 2. Rolling Average
|
||
|
||
Over a period $n$:
|
||
|
||
$$\text{MPE}_t = \frac{1}{n} \sum_{i=t-n+1}^{t} e_i$$
|
||
|
||
### 3. Relationship to MAPE
|
||
|
||
$$\text{MAPE} = \frac{100}{n} \sum |e_i / 100|$$
|
||
$$\text{MPE} = \frac{100}{n} \sum (e_i / 100)$$
|
||
|
||
When errors are consistently in one direction: $|\text{MPE}| \approx \text{MAPE}$
|
||
When errors alternate: $|\text{MPE}| < \text{MAPE}$
|
||
|
||
## Performance Profile
|
||
|
||
| Metric | Score | Notes |
|
||
| :--- | :--- | :--- |
|
||
| **Throughput** | 15 ns/bar | O(1) via running sum |
|
||
| **Allocations** | 0 | Zero-allocation hot path |
|
||
| **Complexity** | O(1) | Constant per update |
|
||
| **Bias Detection** | 10/10 | Primary strength |
|
||
| **Magnitude Info** | 3/10 | Errors can cancel |
|
||
| **Scale Independence** | 9/10 | Percentage-based |
|
||
| **Outlier Sensitivity** | 5/10 | Linear in error magnitude |
|
||
|
||
## Usage
|
||
|
||
```csharp
|
||
// Streaming mode - bias detection in real-time
|
||
var mpe = new Mpe(20);
|
||
|
||
// Actual values consistently higher than predictions
|
||
mpe.Update(actual: 105.0, predicted: 100.0); // +5%
|
||
mpe.Update(actual: 110.0, predicted: 100.0); // +10%
|
||
// MPE will be positive, indicating under-prediction bias
|
||
|
||
double currentBias = mpe.Last.Value;
|
||
if (currentBias > 5.0)
|
||
Console.WriteLine("Model is under-predicting by {0:F1}%", currentBias);
|
||
else if (currentBias < -5.0)
|
||
Console.WriteLine("Model is over-predicting by {0:F1}%", Math.Abs(currentBias));
|
||
else
|
||
Console.WriteLine("Model shows no significant bias");
|
||
|
||
// Batch mode - analyze historical predictions
|
||
var actual = new TSeries { 100, 105, 98, 102, 101 };
|
||
var predicted = new TSeries { 95, 100, 95, 100, 100 };
|
||
var results = Mpe.Calculate(actual, predicted, period: 3);
|
||
|
||
// Span mode - zero-allocation bulk processing
|
||
Span<double> output = stackalloc double[1000];
|
||
Mpe.Batch(actualSpan, predictedSpan, output, period: 20);
|
||
```
|
||
|
||
## Interpretation Guide
|
||
|
||
| MPE Value | Interpretation | Action |
|
||
| :--- | :--- | :--- |
|
||
| **> +10%** | Severe under-prediction | Add positive bias correction |
|
||
| **+5% to +10%** | Moderate under-prediction | Consider model recalibration |
|
||
| **-5% to +5%** | Acceptable bias range | Monitor for drift |
|
||
| **-10% to -5%** | Moderate over-prediction | Consider model recalibration |
|
||
| **< -10%** | Severe over-prediction | Add negative bias correction |
|
||
|
||
## Comparison with Related Metrics
|
||
|
||
| Metric | Formula | Preserves Sign | Use Case |
|
||
| :--- | :--- | :--- | :--- |
|
||
| **MPE** | 100 × (A-P)/A | ✓ | Bias detection |
|
||
| **MAPE** | 100 × \|A-P\|/A | ✗ | Magnitude only |
|
||
| **ME** | A - P | ✓ | Absolute bias |
|
||
| **MAE** | \|A - P\| | ✗ | Absolute magnitude |
|
||
|
||
## Common Pitfalls
|
||
|
||
### 1. Zero Actuals
|
||
|
||
MPE is undefined when actual = 0. The implementation uses epsilon fallback:
|
||
|
||
```csharp
|
||
double divisor = Math.Abs(actual) < 1e-10 ? 1e-10 : actual;
|
||
```
|
||
|
||
### 2. Cancellation Effect
|
||
|
||
Errors of opposite signs cancel out. A model alternating between +50% and -50% errors would show MPE ≈ 0%, masking severe inaccuracy.
|
||
|
||
**Solution**: Use MPE alongside MAPE:
|
||
|
||
* Low MAPE + Low |MPE|: Good model
|
||
* Low MAPE + High |MPE|: Unlikely (mathematically constrained)
|
||
* High MAPE + Low |MPE|: High variance, no bias
|
||
* High MAPE + High |MPE|: High variance with bias
|
||
|
||
### 3. Asymmetric Bounds
|
||
|
||
Unlike MAPE (bounded at 0% to ∞), MPE can range from -∞ to +100%:
|
||
|
||
* Maximum positive: actual = 100, predicted = 0 → MPE = +100%
|
||
* No upper bound on negative: actual = 100, predicted = 1000 → MPE = -900%
|
||
|
||
## See Also
|
||
|
||
* [MAPE](../mape/Mape.md) - Unsigned percentage error for magnitude
|
||
* [ME](../me/Me.md) - Signed absolute error for absolute bias
|
||
* [MAE](../mae/Mae.md) - Unsigned absolute error for magnitude
|