mirror of
https://github.com/mihakralj/QuanTAlib.git
synced 2026-07-28 01:37:43 +00:00
Miha Kralj
172 lines
6.5 KiB
Markdown
172 lines
6.5 KiB
Markdown
# Theil's U: Theil's U Statistic
|
|
|
|
> *The forecast that matters is the one that beats a naive guess.*
|
|
|
|
| Property | Value |
|
|
| ---------------- | -------------------------------- |
|
|
| **Category** | Error Metric |
|
|
| **Inputs** | Actual vs Predicted (dual input) |
|
|
| **Parameters** | `period` |
|
|
| **Outputs** | Single series (TheilU) |
|
|
| **Output range** | $\geq 0$ |
|
|
| **Warmup** | `period` bars |
|
|
| **PineScript** | [theilu.pine](theilu.pine) |
|
|
|
|
- Theil's U Statistic measures forecast accuracy relative to a naive no-change forecast.
|
|
- **Similar:** [MASE](../mase/Mase.md), [Rsquared](../rsquared/Rsquared.md) | **Trading note:** Theil's U statistic; <1 = forecast beats naïve, >1 = worse than naïve random walk.
|
|
- Validated against TA-Lib, Skender, and Tulip reference implementations where available.
|
|
|
|
Theil's U Statistic measures forecast accuracy relative to a naive no-change forecast. A value below 1 indicates the model outperforms simply predicting that tomorrow equals today; above 1 means you'd be better off not forecasting at all.
|
|
|
|
## Historical Context
|
|
|
|
Developed by Dutch econometrician Henri Theil in the 1960s, Theil's U was designed to evaluate economic forecasts against the simplest possible benchmark: the assumption of no change. This was revolutionary because many sophisticated models fail to beat this naive approach, especially in financial markets.
|
|
|
|
## Architecture & Physics
|
|
|
|
Theil's U computes two parallel error metrics: one for the forecast and one for a naive prediction. The ratio reveals whether the forecasting effort adds value. A forecast might have low absolute error but still be worse than doing nothing.
|
|
|
|
### Properties
|
|
|
|
* **Relative benchmark**: Compares against naive no-change forecast
|
|
* **Scale-independent**: Ratio is unitless
|
|
* **Interpretable threshold**: U = 1 is the break-even point
|
|
* **Range**: 0 to ∞, with 0 being perfect and > 1 being worse than naive
|
|
|
|
## Mathematical Foundation
|
|
|
|
### 1. Forecast Error
|
|
|
|
Calculate squared errors for the actual forecast:
|
|
|
|
$$FPE = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$$
|
|
|
|
Where:
|
|
* $y_i$ = actual value at time i
|
|
* $\hat{y}_i$ = predicted value at time i
|
|
|
|
### 2. Naive Error
|
|
|
|
Calculate squared errors for naive prediction (previous actual):
|
|
|
|
$$NPE = \sum_{i=1}^{n} (y_i - y_{i-1})^2$$
|
|
|
|
### 3. Theil's U Calculation
|
|
|
|
Take the ratio of forecast to naive:
|
|
|
|
$$U = \sqrt{\frac{FPE}{NPE}} = \sqrt{\frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - y_{i-1})^2}}$$
|
|
|
|
### 4. Running Update (O(1))
|
|
|
|
QuanTAlib maintains running sums of both squared error terms:
|
|
|
|
$$S_{f,new} = S_{f,old} - e_{f,oldest}^2 + e_{f,newest}^2$$
|
|
|
|
$$S_{n,new} = S_{n,old} - e_{n,oldest}^2 + e_{n,newest}^2$$
|
|
|
|
$$U = \sqrt{\frac{S_{f,new}}{S_{n,new}}}$$
|
|
|
|
## Implementation Details
|
|
|
|
### Usage Patterns
|
|
|
|
```csharp
|
|
// Streaming mode - update with each new observation
|
|
var theilU = new TheilU(period: 20);
|
|
var result = theilU.Update(actualValue, predictedValue);
|
|
|
|
// Batch mode - calculate for entire series
|
|
var results = TheilU.Calculate(actualSeries, predictedSeries, period: 20);
|
|
|
|
// Span mode - zero-allocation for high performance
|
|
TheilU.Batch(actualSpan, predictedSpan, outputSpan, period: 20);
|
|
```
|
|
|
|
### Parameters
|
|
|
|
| Parameter | Type | Description |
|
|
| :--- | :--- | :--- |
|
|
| **period** | int | Lookback window for calculation (must be > 0) |
|
|
|
|
### Properties
|
|
|
|
| Property | Type | Description |
|
|
| :--- | :--- | :--- |
|
|
| **Last** | TValue | Most recent Theil's U value |
|
|
| **IsHot** | bool | True when buffer is full |
|
|
| **Name** | string | Indicator name (e.g., "TheilU(20)") |
|
|
| **WarmupPeriod** | int | Number of periods before valid output |
|
|
|
|
## Performance Profile
|
|
|
|
### Operation Count (Streaming Mode)
|
|
|
|
Theil's U statistic: U = sqrt(MSE_forecast) / sqrt(MSE_naive). Requires two running mean-squared-error accumulators.
|
|
|
|
| Operation | Count | Cost (cycles) | Subtotal |
|
|
| :--- | :---: | :---: | :---: |
|
|
| Forecast MSE update (e^2 + EMA) | 2 | ~5 cy | ~10 cy |
|
|
| Naive MSE update (naive_e^2 + EMA) | 2 | ~5 cy | ~10 cy |
|
|
| U = sqrt(MSE_f) / sqrt(MSE_n) | 2 | ~15 cy | ~30 cy |
|
|
| **Total** | **~6** | — | **~50 cycles** |
|
|
|
|
O(1) per bar. Two parallel EMA accumulators + ratio with sqrt. ~50 cycles/bar.
|
|
|
|
### Batch Mode (SIMD Analysis)
|
|
|
|
| Operation | Vectorizable? | Notes |
|
|
| :--- | :---: | :--- |
|
|
| Squared error accumulation | Yes | Element-wise squares + reduction |
|
|
| sqrt ratio | No | Single scalar at end |
|
|
|
|
Batch MSE accumulation vectorizable; final ratio is scalar. ~8 cy/bar for squared-error accumulation.
|
|
|
|
| Metric | Score | Notes |
|
|
| :--- | :--- | :--- |
|
|
| **Throughput** | ~15 ns/bar | O(1) update complexity |
|
|
| **Allocations** | 0 | Uses pre-allocated ring buffers |
|
|
| **Complexity** | O(1) | Constant time per update |
|
|
| **Accuracy** | 10/10 | Exact calculation |
|
|
| **Timeliness** | 9/10 | No lag beyond the period |
|
|
| **Interpretability** | 10/10 | Clear benchmark comparison |
|
|
|
|
## Interpretation
|
|
|
|
| Theil's U | Interpretation |
|
|
| :--- | :--- |
|
|
| **0** | Perfect prediction |
|
|
| **< 0.5** | Excellent (error < 50% of naive) |
|
|
| **0.5 - 0.8** | Good forecasting skill |
|
|
| **0.8 - 1.0** | Marginal improvement over naive |
|
|
| **= 1.0** | Equal to naive forecast |
|
|
| **> 1.0** | Worse than naive (model adds noise) |
|
|
|
|
## Why Use Theil's U?
|
|
|
|
| Scenario | Low MAE but High U | High MAE but Low U |
|
|
| :--- | :--- | :--- |
|
|
| **Meaning** | Series is easy to predict | Model adds value despite errors |
|
|
| **Example** | Stable prices, any model works | Volatile prices, model captures moves |
|
|
| **Recommendation** | Use simpler model | Keep using the model |
|
|
|
|
## Common Use Cases
|
|
|
|
1. **Economic Forecasting**: Evaluate macro predictions against random walk
|
|
2. **Financial Markets**: Test trading signals against buy-and-hold
|
|
3. **Model Selection**: Choose models that beat naive benchmarks
|
|
4. **Forecast Validation**: Ensure forecasting effort is worthwhile
|
|
|
|
## Edge Cases
|
|
|
|
* **Zero Naive Error**: Returns infinity when series is perfectly flat (naive is perfect)
|
|
* **NaN Handling**: Uses last valid value substitution
|
|
* **Single Input**: Not supported (requires two series)
|
|
* **Period = 1**: Returns 0 (insufficient data for naive comparison)
|
|
* **First Value**: Needs at least 2 values for naive benchmark
|
|
|
|
## Related Indicators
|
|
|
|
* [RMSE](../rmse/Rmse.md) - Root Mean Squared Error (absolute, not relative)
|
|
* [MASE](../mase/Mase.md) - Mean Absolute Scaled Error (similar concept)
|
|
* [R-Squared](../rsquared/RSquared.md) - Coefficient of Determination |