SIMD Refactor: Merge simd-dev into dev (#55)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
Co-authored-by: Warp <agent@warp.dev>

lib/numerics/relu/Relu.md

@@ -0,0 +1,170 @@
+# RELU: Rectified Linear Unit
+
+> "The simplest non-linearity that works—ReLU's computational efficiency and gradient-friendly properties made deep learning practical."
+
+The Rectified Linear Unit (ReLU) activation function applies `max(0, x)` to each value, passing positive inputs unchanged while zeroing negative ones. Its simplicity belies its importance: ReLU enabled the training of deep neural networks by mitigating vanishing gradients, and its computational efficiency makes it the default activation for most architectures.
+
+## Mathematical Foundation
+
+### Core Formula
+
+$$
+\text{ReLU}(x) = \max(0, x) = \begin{cases} x & \text{if } x > 0 \\ 0 & \text{if } x \leq 0 \end{cases}
+$$
+
+### Key Properties
+
+| Property | Formula | Description |
+|:---------|:--------|:------------|
+| **Non-negativity** | $\text{ReLU}(x) \geq 0$ | Output always ≥ 0 |
+| **Identity for Positives** | $\text{ReLU}(x) = x$ for $x > 0$ | Passthrough for positive values |
+| **Sparsity Inducing** | $\text{ReLU}(x) = 0$ for $x \leq 0$ | Creates sparse activations |
+| **Derivative** | $\frac{d}{dx}\text{ReLU}(x) = \mathbf{1}_{x>0}$ | 1 for positive, 0 for negative |
+| **Scale Equivariance** | $\text{ReLU}(\alpha x) = \alpha \cdot \text{ReLU}(x)$ for $\alpha > 0$ | Positive scaling preserved |
+
+### Domain and Range
+
+| | Value |
+|:--|:--|
+| **Domain** | $(-\infty, +\infty)$ |
+| **Range** | $[0, +\infty)$ |
+
+## Financial Applications
+
+### Threshold-Based Signals
+
+Zero out values below a threshold (e.g., only consider positive returns):
+
+$$
+\text{PositiveReturns}_t = \text{ReLU}(r_t)
+$$
+
+### Asymmetric Risk Metrics
+
+Compute downside deviation using ReLU on negated returns:
+
+$$
+\text{Downside}_t = \text{ReLU}(-r_t)
+$$
+
+### Clamping Negative Values
+
+Ensure non-negative inputs to subsequent calculations:
+
+$$
+\text{Volume}_{\text{clamped}} = \text{ReLU}(\text{Volume} - \text{Threshold})
+$$
+
+### Neural Network Features
+
+Pre-processing layer for ML-based trading models where ReLU activation is standard.
+
+## Implementation Details
+
+### SIMD Optimization
+
+The implementation uses AVX2 vectorization when available:
+- Processes 4 doubles per instruction using `Avx.Max`
+- Falls back to scalar `Math.Max` for remaining elements
+- Achieves ~4× throughput improvement on compatible hardware
+
+### NaN Handling
+
+Non-finite inputs (NaN, ±Infinity) are replaced with the last valid output value, maintaining series continuity.
+
+### Streaming Characteristics
+
+| Metric | Value |
+|:-------|:------|
+| **Warmup Period** | 0 |
+| **Memory** | O(1) |
+| **Complexity** | O(1) per update |
+
+## Performance Profile
+
+### Operation Count (Scalar)
+
+| Operation | Count | Notes |
+|:----------|:-----:|:------|
+| CMP | 1 | Comparison with zero |
+| MOV | 1 | Conditional move |
+| **Total** | ~2-3 cycles | Branch-free with CMOV |
+
+### SIMD Performance (AVX2)
+
+| Mode | Throughput | Notes |
+|:-----|:-----------|:------|
+| Scalar | 1 value/cycle | Single comparison |
+| AVX2 | 4 values/cycle | `vpmaxpd` instruction |
+| **Speedup** | ~4× | For aligned batch operations |
+
+### Quality Metrics
+
+| Metric | Score | Notes |
+|:-------|:-----:|:------|
+| **Accuracy** | 10/10 | Exact computation |
+| **Timeliness** | 10/10 | Zero lag |
+| **Smoothness** | 7/10 | Discontinuous derivative at origin |
+
+## Usage Examples
+
+### Basic Usage
+
+```csharp
+var relu = new Relu();
+var input = new TValue(DateTime.UtcNow, -5.0);
+var result = relu.Update(input);  // Returns 0.0
+
+input = new TValue(DateTime.UtcNow, 3.5);
+result = relu.Update(input);  // Returns 3.5
+```
+
+### Filtering Negative Returns
+
+```csharp
+var returns = new TSeries();
+// ... populate with return values
+
+var relu = new Relu();
+var positiveReturns = relu.Update(returns);
+// All negative returns become 0
+```
+
+### Batch Processing with SIMD
+
+```csharp
+double[] source = { -2.0, -1.0, 0.0, 1.0, 2.0, 3.0 };
+double[] output = new double[source.Length];
+
+Relu.Calculate(source.AsSpan(), output.AsSpan());
+// output: { 0, 0, 0, 1, 2, 3 }
+```
+
+## Common Pitfalls
+
+1. **Dead Neurons**: In neural network contexts, neurons with ReLU can "die" if they receive consistently negative inputs during training—they output zero and have zero gradient.
+
+2. **Unbounded Output**: Unlike sigmoid, ReLU has no upper bound. Large positive inputs pass through unchanged, potentially causing numerical issues downstream.
+
+3. **Non-differentiable at Origin**: The derivative is technically undefined at x=0. In practice, implementations choose either 0 or 1; this rarely matters for gradient descent.
+
+4. **Loss of Negative Information**: ReLU discards all information from negative values. If negative values carry meaningful signals, consider alternatives like LeakyReLU or using the raw values.
+
+5. **Not Zero-Centered**: ReLU outputs are always non-negative, which can slow convergence in some optimization scenarios.
+
+## Validation
+
+| Test | Status |
+|:-----|:------:|
+| **Math.Max(0, x) Parity** | ✅ |
+| **Zero Passthrough** | ✅ |
+| **Negative → Zero** | ✅ |
+| **Positive Passthrough** | ✅ |
+| **SIMD/Scalar Consistency** | ✅ |
+| **NaN Handling** | ✅ |
+
+## References
+
+- Nair, V. & Hinton, G. (2010). "Rectified Linear Units Improve Restricted Boltzmann Machines." *ICML*.
+- Glorot, X., Bordes, A., & Bengio, Y. (2011). "Deep Sparse Rectifier Neural Networks." *AISTATS*.
+- Goodfellow, I., Bengio, Y., & Courville, A. (2016). *Deep Learning*. MIT Press.