mirror of
https://github.com/mihakralj/QuanTAlib.git
synced 2026-08-07 05:27:43 +00:00
Miha Kralj
86fe32a682
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat> Co-authored-by: Warp <agent@warp.dev>
137 lines
4.8 KiB
Markdown
137 lines
4.8 KiB
Markdown
# Theil's U: Theil's U Statistic
|
|
|
|
> "The forecast that matters is the one that beats a naive guess."
|
|
|
|
Theil's U Statistic measures forecast accuracy relative to a naive no-change forecast. A value below 1 indicates the model outperforms simply predicting that tomorrow equals today; above 1 means you'd be better off not forecasting at all.
|
|
|
|
## Historical Context
|
|
|
|
Developed by Dutch econometrician Henri Theil in the 1960s, Theil's U was designed to evaluate economic forecasts against the simplest possible benchmark: the assumption of no change. This was revolutionary because many sophisticated models fail to beat this naive approach, especially in financial markets.
|
|
|
|
## Architecture & Physics
|
|
|
|
Theil's U computes two parallel error metrics: one for the forecast and one for a naive prediction. The ratio reveals whether the forecasting effort adds value. A forecast might have low absolute error but still be worse than doing nothing.
|
|
|
|
### Properties
|
|
|
|
* **Relative benchmark**: Compares against naive no-change forecast
|
|
* **Scale-independent**: Ratio is unitless
|
|
* **Interpretable threshold**: U = 1 is the break-even point
|
|
* **Range**: 0 to ∞, with 0 being perfect and > 1 being worse than naive
|
|
|
|
## Mathematical Foundation
|
|
|
|
### 1. Forecast Error
|
|
|
|
Calculate squared errors for the actual forecast:
|
|
|
|
$$FPE = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$$
|
|
|
|
Where:
|
|
* $y_i$ = actual value at time i
|
|
* $\hat{y}_i$ = predicted value at time i
|
|
|
|
### 2. Naive Error
|
|
|
|
Calculate squared errors for naive prediction (previous actual):
|
|
|
|
$$NPE = \sum_{i=1}^{n} (y_i - y_{i-1})^2$$
|
|
|
|
### 3. Theil's U Calculation
|
|
|
|
Take the ratio of forecast to naive:
|
|
|
|
$$U = \sqrt{\frac{FPE}{NPE}} = \sqrt{\frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - y_{i-1})^2}}$$
|
|
|
|
### 4. Running Update (O(1))
|
|
|
|
QuanTAlib maintains running sums of both squared error terms:
|
|
|
|
$$S_{f,new} = S_{f,old} - e_{f,oldest}^2 + e_{f,newest}^2$$
|
|
|
|
$$S_{n,new} = S_{n,old} - e_{n,oldest}^2 + e_{n,newest}^2$$
|
|
|
|
$$U = \sqrt{\frac{S_{f,new}}{S_{n,new}}}$$
|
|
|
|
## Implementation Details
|
|
|
|
### Usage Patterns
|
|
|
|
```csharp
|
|
// Streaming mode - update with each new observation
|
|
var theilU = new TheilU(period: 20);
|
|
var result = theilU.Update(actualValue, predictedValue);
|
|
|
|
// Batch mode - calculate for entire series
|
|
var results = TheilU.Calculate(actualSeries, predictedSeries, period: 20);
|
|
|
|
// Span mode - zero-allocation for high performance
|
|
TheilU.Batch(actualSpan, predictedSpan, outputSpan, period: 20);
|
|
```
|
|
|
|
### Parameters
|
|
|
|
| Parameter | Type | Description |
|
|
| :--- | :--- | :--- |
|
|
| **period** | int | Lookback window for calculation (must be > 0) |
|
|
|
|
### Properties
|
|
|
|
| Property | Type | Description |
|
|
| :--- | :--- | :--- |
|
|
| **Last** | TValue | Most recent Theil's U value |
|
|
| **IsHot** | bool | True when buffer is full |
|
|
| **Name** | string | Indicator name (e.g., "TheilU(20)") |
|
|
| **WarmupPeriod** | int | Number of periods before valid output |
|
|
|
|
## Performance Profile
|
|
|
|
| Metric | Score | Notes |
|
|
| :--- | :--- | :--- |
|
|
| **Throughput** | ~15 ns/bar | O(1) update complexity |
|
|
| **Allocations** | 0 | Uses pre-allocated ring buffers |
|
|
| **Complexity** | O(1) | Constant time per update |
|
|
| **Accuracy** | 10/10 | Exact calculation |
|
|
| **Timeliness** | 9/10 | No lag beyond the period |
|
|
| **Interpretability** | 10/10 | Clear benchmark comparison |
|
|
|
|
## Interpretation
|
|
|
|
| Theil's U | Interpretation |
|
|
| :--- | :--- |
|
|
| **0** | Perfect prediction |
|
|
| **< 0.5** | Excellent (error < 50% of naive) |
|
|
| **0.5 - 0.8** | Good forecasting skill |
|
|
| **0.8 - 1.0** | Marginal improvement over naive |
|
|
| **= 1.0** | Equal to naive forecast |
|
|
| **> 1.0** | Worse than naive (model adds noise) |
|
|
|
|
## Why Use Theil's U?
|
|
|
|
| Scenario | Low MAE but High U | High MAE but Low U |
|
|
| :--- | :--- | :--- |
|
|
| **Meaning** | Series is easy to predict | Model adds value despite errors |
|
|
| **Example** | Stable prices, any model works | Volatile prices, model captures moves |
|
|
| **Recommendation** | Use simpler model | Keep using the model |
|
|
|
|
## Common Use Cases
|
|
|
|
1. **Economic Forecasting**: Evaluate macro predictions against random walk
|
|
2. **Financial Markets**: Test trading signals against buy-and-hold
|
|
3. **Model Selection**: Choose models that beat naive benchmarks
|
|
4. **Forecast Validation**: Ensure forecasting effort is worthwhile
|
|
|
|
## Edge Cases
|
|
|
|
* **Zero Naive Error**: Returns infinity when series is perfectly flat (naive is perfect)
|
|
* **NaN Handling**: Uses last valid value substitution
|
|
* **Single Input**: Not supported (requires two series)
|
|
* **Period = 1**: Returns 0 (insufficient data for naive comparison)
|
|
* **First Value**: Needs at least 2 values for naive benchmark
|
|
|
|
## Related Indicators
|
|
|
|
* [RMSE](../rmse/Rmse.md) - Root Mean Squared Error (absolute, not relative)
|
|
* [MASE](../mase/Mase.md) - Mean Absolute Scaled Error (similar concept)
|
|
* [R-Squared](../rsquared/RSquared.md) - Coefficient of Determination
|