mirror of
https://github.com/mihakralj/QuanTAlib.git
synced 2026-08-06 04:57:44 +00:00
142 lines
4.9 KiB
Markdown
142 lines
4.9 KiB
Markdown
|
|
# MPE: Mean Percentage Error
|
|||
|
|
|
|||
|
|
> "MAPE tells you how wrong you are; MPE tells you which direction you're wrong in."
|
|||
|
|
|
|||
|
|
Mean Percentage Error measures the average percentage difference between actual and predicted values while preserving the sign. Unlike MAPE, which takes absolute values, MPE reveals systematic bias in predictions—whether a model consistently over-predicts or under-predicts.
|
|||
|
|
|
|||
|
|
## Architecture & Physics
|
|||
|
|
|
|||
|
|
MPE computes the signed percentage error for each data point and averages over a rolling window:
|
|||
|
|
|
|||
|
|
$$\text{MPE} = \frac{100}{n} \sum_{i=1}^{n} \frac{(\text{actual}_i - \text{predicted}_i)}{\text{actual}_i}$$
|
|||
|
|
|
|||
|
|
The sign preservation makes MPE invaluable for bias detection:
|
|||
|
|
|
|||
|
|
* **Positive MPE**: Model systematically under-predicts (actual > predicted)
|
|||
|
|
* **Negative MPE**: Model systematically over-predicts (actual < predicted)
|
|||
|
|
* **MPE near zero**: No systematic bias (though individual errors may be large)
|
|||
|
|
|
|||
|
|
### Bias Detection
|
|||
|
|
|
|||
|
|
Consider a weather forecasting model:
|
|||
|
|
|
|||
|
|
* If MPE = +15%, the model consistently predicts temperatures 15% lower than actual
|
|||
|
|
* If MPE = -10%, the model consistently predicts temperatures 10% higher than actual
|
|||
|
|
* If MPE ≈ 0% but MAPE = 20%, errors cancel out (no bias) but magnitude is still significant
|
|||
|
|
|
|||
|
|
## Mathematical Foundation
|
|||
|
|
|
|||
|
|
### 1. Point-wise Percentage Error
|
|||
|
|
|
|||
|
|
For each observation:
|
|||
|
|
|
|||
|
|
$$e_i = 100 \times \frac{\text{actual}_i - \text{predicted}_i}{\text{actual}_i}$$
|
|||
|
|
|
|||
|
|
### 2. Rolling Average
|
|||
|
|
|
|||
|
|
Over a period $n$:
|
|||
|
|
|
|||
|
|
$$\text{MPE}_t = \frac{1}{n} \sum_{i=t-n+1}^{t} e_i$$
|
|||
|
|
|
|||
|
|
### 3. Relationship to MAPE
|
|||
|
|
|
|||
|
|
$$\text{MAPE} = \frac{100}{n} \sum |e_i / 100|$$
|
|||
|
|
$$\text{MPE} = \frac{100}{n} \sum (e_i / 100)$$
|
|||
|
|
|
|||
|
|
When errors are consistently in one direction: $|\text{MPE}| \approx \text{MAPE}$
|
|||
|
|
When errors alternate: $|\text{MPE}| < \text{MAPE}$
|
|||
|
|
|
|||
|
|
## Performance Profile
|
|||
|
|
|
|||
|
|
| Metric | Score | Notes |
|
|||
|
|
| :--- | :--- | :--- |
|
|||
|
|
| **Throughput** | 15 ns/bar | O(1) via running sum |
|
|||
|
|
| **Allocations** | 0 | Zero-allocation hot path |
|
|||
|
|
| **Complexity** | O(1) | Constant per update |
|
|||
|
|
| **Bias Detection** | 10/10 | Primary strength |
|
|||
|
|
| **Magnitude Info** | 3/10 | Errors can cancel |
|
|||
|
|
| **Scale Independence** | 9/10 | Percentage-based |
|
|||
|
|
| **Outlier Sensitivity** | 5/10 | Linear in error magnitude |
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
|
|||
|
|
```csharp
|
|||
|
|
// Streaming mode - bias detection in real-time
|
|||
|
|
var mpe = new Mpe(20);
|
|||
|
|
|
|||
|
|
// Actual values consistently higher than predictions
|
|||
|
|
mpe.Update(actual: 105.0, predicted: 100.0); // +5%
|
|||
|
|
mpe.Update(actual: 110.0, predicted: 100.0); // +10%
|
|||
|
|
// MPE will be positive, indicating under-prediction bias
|
|||
|
|
|
|||
|
|
double currentBias = mpe.Last.Value;
|
|||
|
|
if (currentBias > 5.0)
|
|||
|
|
Console.WriteLine("Model is under-predicting by {0:F1}%", currentBias);
|
|||
|
|
else if (currentBias < -5.0)
|
|||
|
|
Console.WriteLine("Model is over-predicting by {0:F1}%", Math.Abs(currentBias));
|
|||
|
|
else
|
|||
|
|
Console.WriteLine("Model shows no significant bias");
|
|||
|
|
|
|||
|
|
// Batch mode - analyze historical predictions
|
|||
|
|
var actual = new TSeries { 100, 105, 98, 102, 101 };
|
|||
|
|
var predicted = new TSeries { 95, 100, 95, 100, 100 };
|
|||
|
|
var results = Mpe.Calculate(actual, predicted, period: 3);
|
|||
|
|
|
|||
|
|
// Span mode - zero-allocation bulk processing
|
|||
|
|
Span<double> output = stackalloc double[1000];
|
|||
|
|
Mpe.Batch(actualSpan, predictedSpan, output, period: 20);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Interpretation Guide
|
|||
|
|
|
|||
|
|
| MPE Value | Interpretation | Action |
|
|||
|
|
| :--- | :--- | :--- |
|
|||
|
|
| **> +10%** | Severe under-prediction | Add positive bias correction |
|
|||
|
|
| **+5% to +10%** | Moderate under-prediction | Consider model recalibration |
|
|||
|
|
| **-5% to +5%** | Acceptable bias range | Monitor for drift |
|
|||
|
|
| **-10% to -5%** | Moderate over-prediction | Consider model recalibration |
|
|||
|
|
| **< -10%** | Severe over-prediction | Add negative bias correction |
|
|||
|
|
|
|||
|
|
## Comparison with Related Metrics
|
|||
|
|
|
|||
|
|
| Metric | Formula | Preserves Sign | Use Case |
|
|||
|
|
| :--- | :--- | :--- | :--- |
|
|||
|
|
| **MPE** | 100 × (A-P)/A | ✓ | Bias detection |
|
|||
|
|
| **MAPE** | 100 × \|A-P\|/A | ✗ | Magnitude only |
|
|||
|
|
| **ME** | A - P | ✓ | Absolute bias |
|
|||
|
|
| **MAE** | \|A - P\| | ✗ | Absolute magnitude |
|
|||
|
|
|
|||
|
|
## Common Pitfalls
|
|||
|
|
|
|||
|
|
### 1. Zero Actuals
|
|||
|
|
|
|||
|
|
MPE is undefined when actual = 0. The implementation uses epsilon fallback:
|
|||
|
|
|
|||
|
|
```csharp
|
|||
|
|
double divisor = Math.Abs(actual) < 1e-10 ? 1e-10 : actual;
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 2. Cancellation Effect
|
|||
|
|
|
|||
|
|
Errors of opposite signs cancel out. A model alternating between +50% and -50% errors would show MPE ≈ 0%, masking severe inaccuracy.
|
|||
|
|
|
|||
|
|
**Solution**: Use MPE alongside MAPE:
|
|||
|
|
|
|||
|
|
* Low MAPE + Low |MPE|: Good model
|
|||
|
|
* Low MAPE + High |MPE|: Unlikely (mathematically constrained)
|
|||
|
|
* High MAPE + Low |MPE|: High variance, no bias
|
|||
|
|
* High MAPE + High |MPE|: High variance with bias
|
|||
|
|
|
|||
|
|
### 3. Asymmetric Bounds
|
|||
|
|
|
|||
|
|
Unlike MAPE (bounded at 0% to ∞), MPE can range from -∞ to +100%:
|
|||
|
|
|
|||
|
|
* Maximum positive: actual = 100, predicted = 0 → MPE = +100%
|
|||
|
|
* No upper bound on negative: actual = 100, predicted = 1000 → MPE = -900%
|
|||
|
|
|
|||
|
|
## See Also
|
|||
|
|
|
|||
|
|
* [MAPE](../mape/Mape.md) - Unsigned percentage error for magnitude
|
|||
|
|
* [ME](../me/Me.md) - Signed absolute error for absolute bias
|
|||
|
|
* [MAE](../mae/Mae.md) - Unsigned absolute error for magnitude
|