QuanTAlib/lib/errors/logcosh/LogCosh.md

Log-Cosh: Logarithm of Hyperbolic Cosine Loss

"The smooth operator that acts like L2 for small errors and L1 for large ones."

Log-Cosh Loss combines the best properties of L1 (absolute) and L2 (squared) error metrics through the logarithm of the hyperbolic cosine function. It provides smooth gradients everywhere while remaining robust to outliers.

Historical Context

Log-Cosh emerged from the machine learning community as a loss function for neural network training. Its smooth, differentiable nature makes it ideal for gradient-based optimization, while its asymptotic L1 behavior provides robustness similar to absolute error. It has since been adopted as a general-purpose error metric.

Architecture & Physics

The function log(cosh(x)) has remarkable properties: for small x, it approximates x²/2 (L2 behavior), while for large x, it approximates |x| - log(2) (L1 behavior). This creates a smooth transition between squared and absolute error regimes.

Properties

• Smooth everywhere: Infinitely differentiable
• Non-negative: Always ≥ 0, with 0 for perfect prediction
• Robust: Large errors grow linearly, not quadratically
• Convex: Guarantees a unique minimum for optimization

Mathematical Foundation

1. Log-Cosh Error

For each observation, compute:

e_i = \log(\cosh(y_i - \hat{y}_i))

Where:

• y_i = actual value
• \hat{y}_i = predicted value
• \cosh(x) = \frac{e^x + e^{-x}}{2}

2. Approximations

For small errors:

\log(\cosh(x)) \approx \frac{x^2}{2}

For large errors:

\log(\cosh(x)) \approx |x| - \log(2)

3. Mean Calculation

Average the log-cosh errors:

LogCosh = \frac{1}{n} \sum_{i=1}^{n} \log(\cosh(y_i - \hat{y}_i))

4. Running Update (O(1))

QuanTAlib uses a ring buffer with running sum for O(1) updates:

S_{new} = S_{old} - e_{oldest} + e_{newest} LogCosh = \frac{S_{new}}{n}

Implementation Details

Usage Patterns

// Streaming mode - update with each new observation
var logCosh = new LogCosh(period: 20);
var result = logCosh.Update(actualValue, predictedValue);

// Batch mode - calculate for entire series
var results = LogCosh.Calculate(actualSeries, predictedSeries, period: 20);

// Span mode - zero-allocation for high performance
LogCosh.Batch(actualSpan, predictedSpan, outputSpan, period: 20);

Parameters

Parameter	Type	Description
period	int	Lookback window for averaging (must be > 0)

Properties

Property	Type	Description
Last	TValue	Most recent Log-Cosh value
IsHot	bool	True when buffer is full
Name	string	Indicator name (e.g., "LogCosh(20)")
WarmupPeriod	int	Number of periods before valid output

Performance Profile

Metric	Score	Notes
Throughput	~18 ns/bar	O(1) update, log/cosh computation
Allocations	0	Uses pre-allocated ring buffer
Complexity	O(1)	Constant time per update
Accuracy	10/10	Exact calculation
Timeliness	9/10	No lag beyond the period
Smoothness	10/10	Infinitely differentiable

Interpretation

Log-Cosh Range	Interpretation	Approximate Error
0	Perfect prediction	0
< 0.1	Very small error	< 0.45
0.1 - 0.5	Small error	0.45 - 1.0
0.5 - 2.0	Moderate error	1.0 - 2.0
> 2.0	Large error	> 2.0 (linear growth)

Comparison with L1/L2

Error Magnitude	L2 (MSE)	L1 (MAE)	Log-Cosh
0.1	0.01	0.1	0.005
1.0	1.0	1.0	0.433
5.0	25.0	5.0	4.31
10.0	100.0	10.0	9.31
100.0	10000.0	100.0	99.3

Key Insight

For large errors, Log-Cosh grows approximately linearly (like L1), avoiding the explosion of L2 with outliers. For small errors, it provides the smooth quadratic behavior of L2.

Common Use Cases

1. Machine Learning: Differentiable loss function for training
2. Robust Regression: When outliers exist but smooth gradients needed
3. Financial Modeling: Price prediction with occasional spikes
4. Hybrid Metrics: Combining L1 and L2 benefits

Edge Cases

• Perfect Predictions: Returns exactly 0 (log(cosh(0)) = log(1) = 0)
• NaN Handling: Uses last valid value substitution
• Single Input: Not supported (requires two series)
• Period = 1: Returns current log-cosh error
• Large Errors: Numerically stable via cosh implementation

Related Indicators

• MAE - Mean Absolute Error (pure L1)
• MSE - Mean Squared Error (pure L2)
• Huber - Huber Loss (piecewise L1/L2)
• PseudoHuber - Smooth Huber approximation