This page develops the core mathematics underlying Optimiz-rs's Rust kernels --- from first principles through advanced theory. Each section opens with a \textbf{definition block}, builds intuition through \textbf{examples and diagrams}, and closes with a \textbf{notebook micro-check}. For complete walkthroughs see \texttt{examples/notebooks/}.
DE is a gradient-free population-based optimizer for \(f: \mathbb{R}^d \to\mathbb{R}\), not required to be smooth or convex. At generation \(g\) we maintain \(N\) candidate solutions \(\{\mathbf{x}_{i,g}\}\subset\mathbb{R}^d\).
\textbf{Key insight:} The difference vector \(\mathbf{x}_{r_2}-\mathbf{x}_{r_3}\) is an unbiased directional finite-difference of \(f\), so DE implicitly estimates curvature without Jacobians.
\subsubsection{\texorpdfstring{1.1 Geometric Intuition --- Mutation in \(\mathbb{R}^2\)}{1.1 Geometric Intuition --- Mutation in \textbackslash mathbb\{R\}\^{}2}}\label{geometric-intuition-mutation-in-mathbbr2}
\(\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3\) are three \textbf{distinct} randomly selected parents.
\item
The mutant \(\mathbf{v}_i\) lands on the other side relative to \(\mathbf{x}_{r_1}\).
\item
\textbf{Crossover} then mixes \(\mathbf{v}_i\) and \(\mathbf{x}_i\) dimension-by-dimension with probability \(CR\), producing trial vector \(\mathbf{u}_i\).
\item
\textbf{Selection} keeps \(\mathbf{u}_i\) only if it improves over \(\mathbf{x}_i\) --- pure greedy.
\textbf{Convergence (informal):} Under bounded population diversity and Lipschitz \(f\), the best-so-far value converges a.s. to a stationary point as \(N,g\to\infty\) (Price et al.~2005).
\subsubsection{1.3 Self-Adaptive jDE (Optimiz-rs default)}\label{self-adaptive-jde-optimiz-rs-default}
Parameters \(F,CR\) are per-individual and reset stochastically each generation:
\(\tau_1=\tau_2=0.1\) by default. On rugged landscapes this produces bimodal \(F\) histograms concentrated near 0.8 --- a sign the landscape is highly multimodal.
\subsubsection{1.4 Example --- Minimising the Rastrigin Function}\label{example-minimising-the-rastrigin-function}
The Rastrigin function \(f(\mathbf{x})=10d +\sum_i[x_i^2-10\cos(2\pi x_i)]\) has \(\approx10^d\) local minima (global minimum \(f^*=0\) at \(\mathbf{x}^*=\mathbf{0}\)).
\textbf{Why gradient methods fail:} The gradient \(\partial_{x_i}f =2x_i +20\pi\sin(2\pi x_i)\) oscillates rapidly --- any gradient step hops between basins.
\textbf{Why DE succeeds:} The difference vector \(F(\mathbf{x}_{r_2}-\mathbf{x}_{r_3})\) spans the characteristic basin width (\textasciitilde1.0), enabling inter-basin jumps.
\begin{verbatim}
Rastrigin 1D sketch (d=1):
f(x)
| * * * <- local minima (many)
| * * * * * *
|* *
| * *
+------|------+------|-- x
-1 0 1
^
global min f=0
\end{verbatim}
\textbf{Typical jDE convergence} (\(d=10\), \(N=100\), \(\tau_1=\tau_2=0.1\)):
\begin{verbatim}
Gen Best f Mean F Mean CR
---- ------- ------- -------
1 48.3 0.50 0.50
50 12.1 0.78 0.31
200 3.4 0.82 0.24 <- F clusters near 0.8 (bimodal)
500 0.0 0.83 0.22 <- converged
\end{verbatim}
\begin{tcolorbox}[colback=thmbg!60,colframe=optgold,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Theorem / Key Idea}} — Tip — Diagnosing Stagnation},breakable]
If best-$f$ does not decrease for 100+ generations:
1. **Check $F$ histogram.** Bimodal near 0.8 -> landscape is multimodal (increase $N$).
Collapsed near 0 -> diversity loss; restart with random perturbation.
2. **Check $CR$ distribution.** Uniform -> dimensions not interacting.
Collapsed near 0 -> DE treating dimensions independently (separable function).
3. **Increase $N$** to $\approx10d$ for $d > 20$.
\end{tcolorbox}
\textbf{Notebook check} (\texttt{05\_performance\_benchmarks.ipynb}): Plot \(F_i, CR_i\) histograms every 50 generations; expect values clustering in \([0.5,0.9]\) on hard problems.
These form the probabilistic backbone of all continuous-time models in Optimiz-rs. We build the theory from scratch: random walk → Brownian motion → Itō calculus → SDEs → jump-diffusions.
\paragraph{2.1.0 Intuitive Construction --- From Random Walk to BM}\label{intuitive-construction-from-random-walk-to-bm}
\textbf{Step 1 --- Discrete random walk.} Flip a fair coin \(n\) times per unit time. Define \(\xi_k =+1\) (heads) or \(-1\) (tails) i.i.d. After \(n\) steps of size \(1/\sqrt{n}\):
\[S^{(n)}_t =\frac{1}{\sqrt{n}}\sum_{k=1}^{\lfloor nt \rfloor} \xi_k.\]
By the \textbf{Central Limit Theorem}, as \(n\to\infty\): \(S^{(n)}_t \xrightarrow{d} W_t \sim\mathcal{N}(0,t)\).
\begin{verbatim}
Coin-flip random walk (n=20 steps per unit time):
W_t
+2 | * *
| * * *
0 | * * * * *
|* * *
-2 | *
+----------------------------> t
0 0.5 1.0
n → ∞ ("zoom out"): jagged → smooth BM fan
\end{verbatim}
\textbf{Step 2 --- Scaling limit.} The normalization \(1/\sqrt{n}\) is crucial: - Without it, variance grows as \(n\) (diverges). - With \(n^{-1/2}\): variance = \(n \cdot(1/\sqrt{n})^2\cdot t = t\) --- exactly right.
This is why \(W_t \sim\mathcal{N}(0,t)\): \textbf{variance accumulates linearly in time}.
\begin{tcolorbox}[colback=defbg!60,colframe=optblue,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Definition}} — Definition — Wiener Process},breakable]
A stochastic process $W =(W_t)_{t\ge0}$ on $(\Omega,\mathcal{F},\mathbb{P})$
is a *standard Brownian motion* if:
1. $W_0=0$ a.s.
2. Increments are **independent**: $W_t - W_s \perp\mathcal{F}_s$ for $t>s$.
3. $W_t - W_s \sim\mathcal{N}(0, t-s)$ for all $0\le s<t$.
4. Paths $t\mapsto W_t(\omega)$ are **continuous** a.s.
\textbf{Quadratic variation derivation (step by step):}
Partition \([0,T]\) into \(n\) pieces of width \(\Delta= T/n\). Sum of squared increments:
\[\sum_{k=0}^{n-1}(W_{t_{k+1}}-W_{t_k})^2\overset{?}{=} T \quad\text{as } n\to\infty.\]
\textbf{Step 1} --- Each increment: \((W_{t_{k+1}}-W_{t_k})^2\sim\Delta\cdot\chi_1^2\), so \(\mathbb{E}[(W_{t_{k+1}}-W_{t_k})^2]=\Delta\).
\textbf{Step 2} --- Sum of means: \(\sum_{k=0}^{n-1} \Delta= n\Delta= T\).
\textbf{Step 3} --- Variance of the sum: \(\operatorname{Var}\!\left(\sum(W_{t_{k+1}}-W_{t_k})^2\right)= n \cdot2\Delta^2=2T^2/n \xrightarrow{n\to\infty} 0\).
\textbf{Conclusion:}\(\sum(W_{t_{k+1}}-W_{t_k})^2\xrightarrow{L^2} T\). We write \(dW_t^2= dt\).\\
This \textbf{single identity} is the engine of all Itō calculus.
\paragraph{2.2.0 Why You Cannot Use Ordinary Integration}\label{why-you-cannot-use-ordinary-integration}
Attempt to define \(\int_0^T W_t\,dW_t\) using a Riemann sum: pick \(W_{t_k}\) at the \textbf{left endpoint} → get one answer; pick \((W_{t_k}+W_{t_{k+1}})/2\) (midpoint) → get a \emph{different} answer.
This ambiguity occurs because \(W\) is not of bounded variation. \textbf{Itō's convention} (left endpoint) is the only one that produces a \textbf{martingale} --- ensuring no look-ahead.
For \(j \neq k\) (say \(j < k\)): \(f_{t_j}\Delta W_j\) and \(f_{t_k}\) are both \(\mathcal{F}_{t_k}\)-measurable, while \(\Delta W_k\) is \textbf{independent} of \(\mathcal{F}_{t_k}\) with mean 0 → cross term \(=0\).
For \(j = k\): \(\mathbb{E}[f_{t_j}^2(\Delta W_j)^2]=\mathbb{E}[f_{t_j}^2]\Delta t_j\) (independence of \(f_{t_j}\) and \(\Delta W_j\)).
\textbf{Rule of thumb:} Use Itō in finance (causality, no-arbitrage); use Stratonovich in physics/differential geometry (coordinate-invariant chain rule).
\subsubsection{2.3 General Itō SDEs}\label{general-itux14d-sdes}
Let \(\varepsilon_n(t)=\mathbb{E}\!\left[\sup_{s\le t}|X_s^{(n+1)}-X_s^{(n)}|^2\right]\).
By Doob's \(L^2\)-inequality and Lipschitz:
\[\varepsilon_{n+1}(t)\le2(L^2 T + L^2)\int_0^t \varepsilon_n(s)\,ds.\]
By induction: \(\varepsilon_n(t)\le C \cdot\frac{(2L^2(T+1)t)^n}{n!} \to0\). Geometric series → \(X^{(n)}\) is Cauchy in \(L^2\) → converges to the unique solution.
using Itō's lemma on \(\phi(X_t)\). Integration by parts in the \(x\)-integral transfers derivatives from \(\phi\) to \(p\), giving the Fokker-Planck equation.
\textbf{Visual --- density flows rightward (positive drift) and spreads (positive diffusion):}
(Here we used Itō's product rule: \(d(e^{\kappa t}X_t)= e^{\kappa t}dX_t + X_t\cdot\kappa e^{\kappa t}dt\) --- no quadratic variation cross term since \(e^{\kappa t}\) is deterministic.)
\textbf{Step 2 --- Integrate both sides from \(0\) to \(t\):}
\(\theta\)& Long-run equilibrium (the ``anchor'') \\
\((X_0-\theta)e^{-\kappa t}\)& Deterministic decay: initial displacement shrinks at rate \(\kappa\)\\
\(\sigma\int_0^t e^{-\kappa(t-s)}dW_s\)& Stochastic part: weighted sum of all past noise shocks, with \textbf{exponential forgetting}\\
\end{longtable}
}
The stochastic integral \(I_t =\sigma\int_0^t e^{-\kappa(t-s)}dW_s\) is a \textbf{Gaussian} random variable (linear functional of Brownian motion) with:
As \(t\to\infty\): \(X_t \to\mathcal{N}(\theta, \sigma^2/2\kappa)\) --- the stationary distribution.
\paragraph{\texorpdfstring{2.4.2 Transition Density (Conditional on \(X_s\))}{2.4.2 Transition Density (Conditional on X\_s)}}\label{transition-density-conditional-on-x_s}
\[X_t \mid X_s \sim\mathcal{N}\!\left(\theta+(X_s-\theta)e^{-\kappa(t-s)},\;\frac{\sigma^2}{2\kappa}(1-e^{-2\kappa(t-s)})\right), \quad t > s.\]
This is exact (no approximation) because the OU process is \textbf{linear}. Key formulas:
These are nonlinear in \(\kappa\); Optimiz-rs solves them with DE (\texttt{ou\_estimator::fit\_mle()}).
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Calibrating OU to an Equity-Pair Spread},breakable]
**Data:** Daily log-spread $X_t =\log(P_A / P_B)$ for a co-integrated pair,
$n=250$ observations, $\Delta t=1/252$ years.
**Step 1 — MLE:** Maximize $\ell(\kappa, \theta, \sigma)$ using `ou_estimator::fit_mle()`.
**Step 2 — Intermediate verification:** The OU log-likelihood surface:
```
ell(kappa, theta | sigma_hat)
kappa
^
| +++ <- log-lik thickening near true kappa
| +++++
| +++++++
| +++++
| +++
|
+-------------------------> theta
(theta_hat is the sample mean of X_t, very well identified)
(kappa is harder: need long series to identify mean-reversion speed)
```
**Typical results:**
| Parameter | Estimate | Interpretation |
|-----------|----------|----------------|
| $\hat\kappa$ | 55/yr | half-life approx 4.6 days |
A counting process $N =(N_t)_{t\ge0}$ is a *Poisson process with
intensity* $\lambda > 0$ if:
1. $N_0=0$.
2. Independent, stationary increments.
3. $\mathbb{P}(N_{t+h}-N_t=1)=\lambda h + o(h)$ and $\mathbb{P}(\Delta N > 1)= o(h)$.
\end{tcolorbox}
Equivalently, \(N_t \sim\text{Poisson}(\lambda t)\) and inter-arrival times are \(\text{Exp}(\lambda)\). The \emph{compensated} process \(\tilde N_t = N_t -\lambda t\) is a martingale.
\textbf{Sample path --- step function with random jumps (\(\lambda=2\) per unit time):}
where \(\lambda' =\lambda e^{\mu_J+\frac12\sigma_J^2}\), \(r_n = r -\lambda(e^{\mu_J+\frac12\sigma_J^2}-1)+ n(\mu_J+\tfrac12\sigma_J^2)/T\), and \(\sigma_n^2=\sigma^2+ n\sigma_J^2/T\).
\textbf{Intuition:} Condition on exactly \(n\) jumps occurring (probability \(e^{-\lambda' T}(\lambda' T)^n/n!\)). In that scenario the world is a BS world with adjusted drift \(r_n\) and total variance \(\sigma^2 T + n\sigma_J^2\). Average over the Poisson distribution of \(n\).
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Fitting Merton to a Crash Event},breakable]
**Observed:** S&P 500, March 2020. Implied vol surface shows a vol smile —
OTM puts are expensive (fat left tail), which pure BS cannot explain.
\subsection{4 · Optimal Control (HJB, PMP, Jumps)}\label{optimal-control-hjb-pmp-jumps}
\textbf{Big picture.} Optimal control asks: \emph{given a stochastic system we can steer with a control \(u_t\), what policy minimises expected cost?} Three complementary tools answer this:
\textbf{Intuition --- three terms inside the infimum:} - \(\ell(x,u)\) --- instantaneous running cost (pay now). - \(\nabla_x V^\top b\) --- drift of the value function (first-order Taylor in state change). - \(\tfrac12\operatorname{Tr}(\sigma\sigma^\top\nabla^2 V)\) --- curvature correction due to noise (stochastic analogue of the second-order Taylor term).
Under smooth \(V\), the \textbf{feedback law} is \(u^\star(t,x)=\arg\min_u[\ell(x,u)+\nabla_x V^\top b(x,u)].\)
\textbf{LQR special case} (\(\ell= x^\top Q x + u^\top R u\), \(b=Ax+Bu\), \(\sigma\) constant): \(V(t,x)=x^\top P(t)x + v(t)\) with \(P\) solving the \emph{matrix Riccati ODE}:
\[-\dot P = A^\top P + PA - PBR^{-1}B^\top P + Q,\quad P(T)=Q_T.\]
The optimal control is \textbf{linear feedback}: \(u^\star_t =-R^{-1}B^\top P(t)X_t\).
The PMP avoids the curse of dimensionality --- it converts HJB into a \textbf{two-point boundary-value ODE} in \((X_t, p_t)\), feasible when a PDE grid is intractable.
This is exactly the adjoint / backpropagation equation of deep learning --- PMP is the continuous-time version of gradient backpropagation through a dynamical system.
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — PMP for the Merton Problem},breakable]
With $\ell=0$, $g(x)=-\ln x$, $b =(r+u(\mu-r))x$,
the Hamiltonian is $\mathcal{H}(x,u,p)= p(r+u(\mu-r))x$.
\subsubsection{4.3 HJB with Jumps (HJBI)}\label{hjb-with-jumps-hjbi}
When the state can jump (§3.4), the HJB equation gains a \textbf{non-local integral operator}:
\[-\partial_t V =\inf_{u}\Bigl[\ell+\nabla V^\top b +\tfrac12\operatorname{Tr}(\sigma\sigma^\top\nabla^2 V)
+\underbrace{\int\bigl[V(x+c(x,u,z))-V(x)-\nabla V^\top c(x,u,z)\bigr]\nu(dz)}_{\text{expected value change from jumps}}\Bigr].\]
\textbf{Intuition for the integral term.} A jump of size \(c\) moves the state from \(x\) to \(x+c\), changing the value function by \(V(x+c)-V(x)\). The compensator \(\nabla V^\top c\) subtracts the linear part already counted in the drift.
The \texttt{optimal\_control} module discretises the integral on truncated support using Gaussian quadrature.
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Optimal Execution with Jump Risk},breakable]
When \(V\) fails to be \(C^{1,2}\) --- degenerate diffusion, constraints, or non-smooth terminal conditions --- classical solutions may not exist. \textbf{Viscosity solutions} (Crandall-Lions 1983) provide a rigorous weak notion that restores existence and uniqueness.
\textbf{Practical interpretation:} Classical: ``\(V\) satisfies the PDE pointwise.'' Viscosity: ``\(V\) satisfies the PDE in an averaged sense --- even at kinks.''
Optimiz-rs's backward DP converges to the viscosity solution under CFL: \(\Delta t \le C\,(\Delta x)^2\).
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — American Option as a Viscosity Problem},breakable]
American put payoff $g(x)=(K-x)^+$ gives the **variational inequality**:
\textbf{Convergence:} For monotone coupling (Lasry-Lions 2007), the system has a unique solution and the fixed-point iteration contracts.
\textbf{Practical tip:} Monitor both \(\|m^{k+1}-m^k\|_1\) and \(\|u^{k+1}-u^k\|_\infty\); divergence of either signals non-monotone coupling or too large a time step.
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Optimal Liquidation with Many Agents},breakable]
**Setup:** $N \gg1$ traders each hold $x_t$ shares and must liquidate by $T$.
Aggregate selling rate $\bar u_t =\int u\,m(t,dx)$ depresses the price.
The Kalman filter computes the exact conditional mean \(\hat{\mathbf{x}}_t =\mathbb{E}[\mathbf{x}_t \mid\mathbf{y}_{1:t}]\) in Gaussian models and minimises \(D_{\mathrm{KL}}(p(\mathbf{x}_t|\mathbf{y}_{1:t})\,\|\,\mathcal{N}(\hat{\mathbf{x}}_t, P_t))\) over all Gaussian approximations.
For \(d\mathbf{X}_t = A\mathbf{X}_t\,dt + B\,d\mathbf{W}_t\), \(d\mathbf{Y}_t = C\mathbf{X}_t\,dt + d\mathbf{V}_t\), the error covariance satisfies the \emph{Riccati ODE}:
\[\dot P = AP + PA^\top+ BQB^\top- PC^\top R^{-1}CP,\qquad P(0)=P_0.\]
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Tracking a Noisy AR(1) Signal},breakable]
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Calibrating OU Parameters via MCMC},breakable]
**Goal:** Full Bayesian inference on $(\kappa, \theta, \sigma)$ of an OU process.
\textbf{Information-theoretic view:} Baum-Welch is EM on the complete-data log-likelihood; each iteration monotonically increases \(\mathcal{L}(\theta)\) by Jensen's inequality.
D_KL(p||q): fitting q to match p (mean-seeking, mode-averaging)
D_KL(q||p): q must cover p (mode-seeking, mode-fitting)
\end{verbatim}
\textbf{Connection to model selection:} AIC \(=2k -2\ln\hat{\mathcal{L}}\) and BIC \(= k\ln n -2\ln\hat{\mathcal{L}}\) bound \(D_{\mathrm{KL}}(p_{\text{true}}\,\|\,p_\theta)\).
\textbf{Example:} For \(B_k =\mathcal{N}(\mu_k,\sigma_k^2)\): \(\mathcal{I}(\mu_k)=\sigma_k^{-2}\), \(\mathcal{I}(\sigma_k^2)=(2\sigma_k^4)^{-1}\). Higher emission variance -\textgreater{} smaller Fisher info -\textgreater{} less certain parameter estimates.
\textbf{Interpretation:}\(I(X;Y)\) = how much knowing \(Y\) reduces uncertainty about \(X\). \(X \perp Y \Rightarrow I=0\). \(Y\) determines \(X\) fully \(\Rightarrow I = H(X)\).
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Entropy of HMM Regime Probabilities},breakable]
Classical gradient descent ignores parameter-space geometry. The \emph{natural gradient} replaces \(\nabla_\theta\mathcal{L}\) with \(\mathcal{I}(\theta)^{-1}\nabla_\theta\mathcal{L}\), giving a reparametrisation-invariant update --- see §10.2 for the full geometric development.
where \(\Gamma^k_{ij} =\tfrac12 g^{kl}(\partial_i g_{jl}+\partial_j g_{il}-\partial_l g_{ij})\) are the \emph{Christoffel symbols} encoding intrinsic curvature.
\subsubsection{10.2 Information Geometry and Fisher-Rao Metric}\label{information-geometry-and-fisher-rao-metric}
The statistical manifold \(\mathcal{M} =\{p(\cdot;\theta)\}\) carries the \textbf{Fisher-Rao metric}\(g_{ij}(\theta)=\mathcal{I}(\theta)_{ij}\).
\textbf{Standard vs natural gradient:}
\begin{verbatim}
Standard gradient descent: Natural gradient descent:
theta_{k+1} = theta_k - eta * grad L theta_{k+1} = theta_k - eta * I^{-1} grad L
Parameter space = flat R^d. Parameter space = Riemannian (metric I(theta)).
Ignores curvature. Adapts step to local geometry.
Slow on ill-conditioned I. Invariant to reparametrisation.
O(kappa(I)) iterations. O(1) iterations on exponential families.
\textbf{KL geometry:}\(D_{\mathrm{KL}}(p_\theta\,\|\,p_{\theta+d\theta})=\tfrac12\,d\theta^\top\mathcal{I}(\theta)\,d\theta+ O(\|d\theta\|^3)\), confirming Fisher-Rao as the intrinsic KL metric.
\begin{tcolorbox}[colback=notebg!60,colframe=optgreen,fonttitle=\bfseries\sffamily,title={\textsf{\textbf{Example}} — Example — Natural Gradient on a Gaussian Model},breakable]
For $p(x;\theta)=\mathcal{N}(\mu, \sigma^2)$, $\theta=(\mu,\sigma^2)$:
PMP on Lie groups yields the \emph{Lie-Poisson (Euler-Poincare) equations} (Holm-Marsden-Ratiu), providing structure-preserving optimal trajectories.
\subsubsection{10.4 Symplectic Geometry and Hamiltonian Structure}\label{symplectic-geometry-and-hamiltonian-structure}
The phase space \((T^\star M, \omega)\) carries the symplectic 2-form \(\omega=\sum_i dp_i \wedge dq_i\). Hamilton's equations preserve \(\omega\) (\emph{Liouville's theorem} --- phase-space volume conserved).
\textbf{Connection to PMP:} The costate pair \((X_t^\star, p_t)\) solves Hamilton's equations, i.e., the PMP is a symplectic flow on \(T^\star\mathbb{R}^d\).
\textbf{Symplectic integrators} (Stormer-Verlet, Ruth-Forest) preserve \(\omega\) discretely, keeping the Hamiltonian nearly constant over long horizons --- critical for multi-year allocation back-tests in Optimiz-rs.
\subsubsection{10.5 Sectional Curvature and Landscape Geometry}\label{sectional-curvature-and-landscape-geometry}
The sectional curvature \(K(\sigma)\) governs how quickly nearby geodesics diverge:
\begin{verbatim}
K > 0 (sphere): geodesics converge -> compact optimiser trajectories
K = 0 (flat ): Euclidean behaviour -> Newton / natural gradient exact
K < 0 (hyper.): exponential spread -> efficient landscape exploration
\end{verbatim}
For exponential families in natural/mean parameters \(K=0\) --- explaining exact Newton convergence without curvature correction.
