gavindiaz/optimiz-rs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(diagrams): add 4 remaining SVG figures for §8 HMM + §10 geometry

Browse Source 
- fig_hmm_regime: 3-state Bull/Neutral/Bear HMM state machine + emission table
- fig_viterbi_trellis: Viterbi trellis K=3 T=4 with MAP path highlighted
- fig_std_vs_nat_gradient: standard vs natural gradient property comparison panels
- fig_lie_group_hierarchy: GL/SL/O/SO/Sp/H Lie group tree with finance annotations

All 27 SVGs regenerated; mathematical_foundations.md now 0 remaining ASCII blocks.
_fix_diagrams3.py added for reproducibility.
This commit is contained in:
ThotDjehuty
2026-03-07 13:39:43 +01:00
parent 5d06f0ab57
commit 49be6c7bf7
30 changed files with 8808 additions and 1086 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/theory/mathematical_foundations.md
+23 -64
View File
@@ -1132,26 +1132,12 @@ $Y_t \mid Z_t=k \sim B_k(y)$.
**State machine diagram ($K=3$ regimes):**
```
HMM regime state machine (K = 3)
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
```{figure} ../_static/diagrams/fig_hmm_regime.svg
:align: center
:width: 90%
A₁₂ → A₂₃ →
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ State 1 │──────▶│ State 2 │──────▶│ State 3 │
│ Bull │◀──────│ Neutral │◀──────│ Bear │
└─────────────┘ └─────────────┘ └─────────────┘
← A₂₁ ← A₃₂
Emission B_k(y) = 𝒩(μ_k, σ_k²):
┌────────┬────────┬────────┬──────────────────┐
│ State │ μ │ σ │ Character │
├────────┼────────┼────────┼──────────────────┤
│ Bull │ +0.05 │ 0.12 │ high return, low vol │
│ Neutral│ 0.00 │ 0.18 │ flat, medium vol │
│ Bear │ -0.08 │ 0.35 │ crash, high vol │
└────────┴────────┴────────┴──────────────────┘
(self-transition: A₁₁=0.97, A₂₂=0.97, A₃₃=0.90)
HMM $K=3$ state machine with Bull / Neutral / Bear regimes and Gaussian emission
parameters. Self-transitions $A_{11}=A_{22}=0.97$, $A_{33}=0.90$.
```
### 8.2 Baum-Welch (EM)
@@ -1174,22 +1160,13 @@ iteration monotonically increases $\mathcal{L}(\theta)$ by Jensen's inequality.
**Viterbi trellis diagram ($K=3$, $T=4$):**
```
Viterbi trellis (K=3, T=4)
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
```{figure} ../_static/diagrams/fig_viterbi_trellis.svg
:align: center
:width: 82%
State t=1 t=2 t=3 t=4
1 ○─────────▶○─────────▶○─────────▶○
╲ ╳
2 ○─────────▶●─────────▶●─────────▶○ ● = MAP path
╲ ╲ ╲
3 ○─────────▶○─────────▶○─────────▶○
δ_t(k) = max_j [δ_{t−1}(j) · A_jk · B_k(y_t)]
ψ_t(k) = argmax_j ← backtrack pointer
Traceback: z_4★ ← z_3★ ← z_2★ ← z_1★ via ψ
Viterbi trellis ($K=3$, $T=4$). Filled nodes mark the MAP (most probable) state
sequence; arrows show transition candidates. Backtracking via $\psi_t(k)$ recovers
$z_1^\star o z_4^\star$.
```
**Viterbi (MAP path):** $\delta_t(k) = \max_j \delta_{t-1}(j)A_{jk} \cdot B_k(y_t)$, $O(TK^2)$.
@@ -1288,21 +1265,13 @@ The statistical manifold $\mathcal{M} = \{p(\cdot;\theta)\}$ carries the
**Standard vs natural gradient:**
```
Standard vs natural gradient
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
```{figure} ../_static/diagrams/fig_std_vs_nat_gradient.svg
:align: center
:width: 88%
Standard: θ_{k+1} = θ_k − η·∇ℒ Natural: θ_{k+1} = θ_k − η·ℐ(θ)^{−1}∇ℒ
────────────────────────────────────────────
┌────────────────────┐ ┌────────────────────┐
│ Flat ℝᵈ geometry │ │ Riemannian metric ℐ(θ) │
│ Ignores curvature │ │ Adapts to geometry │
│ Slow on ill-cond ℐ │ │ Reparam invariant │
│ O(κ(ℐ)) iters │ │ O(1) on exp families │
└────────────────────┘ └────────────────────┘
On Gaussian / exponential family: ℐ⁻¹∇ℒ = MLE step → 1 iteration!
Standard versus natural gradient: geometric properties. On exponential families
the natural gradient equals the MLE Newton step, achieving convergence in one
iteration.
```
**Natural gradient (Amari 1998):**
@@ -1344,23 +1313,13 @@ linearises the group at the identity.
**Matrix Lie group hierarchy:**
```
Matrix Lie group hierarchy
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
```{figure} ../_static/diagrams/fig_lie_group_hierarchy.svg
:align: center
:width: 90%
GL(n,ℝ) ─ all invertible n×n real matrices
│
├──▶ SL(n,ℝ) det = 1
│
├──▶ O(n) RᵀR = I (orthogonal)
│ └─▶ SO(n) det = +1 (pure rotations)
│ ↳ portfolio factor rotation, PCA constraints
│
└──▶ Sp(2n,ℝ) preserves symplectic form ω
↳ Hamiltonian mechanics, PMP §4.2 / §10.4
H(n) Heisenberg ─ upper triangular, 1s on diagonal
↳ path-signature feature maps
Matrix Lie group hierarchy: subgroup inclusions and their quantitative-finance
applications. $SO(n)$ underpins PCA factor rotation; $\mathrm{Sp}(2n,\mathbb{R})$
governs Hamiltonian mechanics (PMP §10.4); $H(n)$ drives path-signature features.
```
**Left-invariant control system on $G$:**