docs(v2.0.0-alpha.5): rich math+physics background per chapter, fix notebook download links
Each of the eight v2.0 algorithm pages (bsde, pde, stochastic_control, quadratic_impact_control, mckean_vlasov, agent_based, robust_drift, generative_calibration_hooks) gains: - A dedicated 'Mathematical background' section with the central theorem (Pardoux-Peng, Sznitman propagation of chaos, Pontryagin-Bismut, Huber-IRLS, Gretton MMD, Kolmogorov forward, etc.), key derivations and the analytic closed-form solution that the unit tests target. - An 'Applications' / 'Why it matters' paragraph listing concrete research and engineering use-cases so newcomers grasp the value of each primitive. - A repaired companion-notebook block: the broken relative path '../../examples/notebooks/...ipynb' (which 404s on RTD) is replaced by an explicit GitHub blob (view) + raw (download) URL pair. Sphinx now builds the full doc set with zero new warnings.
This commit is contained in:
@@ -1,9 +1,85 @@
|
||||
Generative calibration — Gaussian MMD loss
|
||||
Generative calibration — Gaussian-MMD loss
|
||||
==========================================
|
||||
|
||||
Maximum-Mean-Discrepancy distance with Gaussian kernel (`mmd_gaussian`). Self-distance is exactly zero; the metric grows monotonically with sample shift.
|
||||
Kernel-based **Maximum Mean Discrepancy** distance (Gretton et al. 2012) — a closed-form,
|
||||
differentiable, distribution-free metric between two empirical samples. Used as the loss
|
||||
function of every generative-calibration loop in `optimiz-rs`.
|
||||
|
||||
.. note:: Companion executed notebook: `17_generative_calibration.ipynb <../../examples/notebooks/17_generative_calibration.ipynb>`_
|
||||
Mathematical background
|
||||
-----------------------
|
||||
|
||||
**Definition.** For a positive-definite kernel $k : \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$
|
||||
with reproducing-kernel Hilbert space (RKHS) $\mathcal{H}_k$, the *kernel mean embedding* of a
|
||||
probability measure $P$ is $\mu_P := \mathbb{E}_{X \sim P}[k(X, \cdot)] \in \mathcal{H}_k$.
|
||||
The **squared MMD** is the RKHS distance between embeddings:
|
||||
|
||||
.. math::
|
||||
|
||||
\mathrm{MMD}^2(P, Q) \;:=\; \| \mu_P - \mu_Q \|_{\mathcal{H}_k}^2
|
||||
\;=\; \mathbb{E}\,[k(X, X')] \;-\; 2\, \mathbb{E}\,[k(X, Y)] \;+\; \mathbb{E}\,[k(Y, Y')] ,
|
||||
|
||||
where $X, X' \sim P$ and $Y, Y' \sim Q$ are independent. When $k$ is *characteristic*
|
||||
(e.g. Gaussian RBF), $\mathrm{MMD}(P, Q) = 0 \iff P = Q$.
|
||||
|
||||
**U-statistic estimator.** Given i.i.d. samples $\{x_i\}_{i=1}^n$ and $\{y_j\}_{j=1}^m$, the
|
||||
unbiased estimator is
|
||||
|
||||
.. math::
|
||||
|
||||
\widehat{\mathrm{MMD}}^2 \;=\;
|
||||
\frac{1}{n(n-1)}\!\sum_{i \ne i'} k(x_i, x_{i'})
|
||||
\;-\; \frac{2}{n m}\!\sum_{i, j} k(x_i, y_j)
|
||||
\;+\; \frac{1}{m(m-1)}\!\sum_{j \ne j'} k(y_j, y_{j'}) .
|
||||
|
||||
It is unbiased, computable in $O((n + m)^2)$ for $d = 1$ (the case implemented), and asymptotically
|
||||
normal under the alternative. Self-distance is **exactly zero**.
|
||||
|
||||
**Kernel.** The shipped routine uses the Gaussian RBF
|
||||
$k_\sigma(x, y) = \exp\!\bigl(-(x - y)^2 / (2\sigma^2)\bigr)$ with bandwidth $\sigma$. Standard
|
||||
reproducing-kernel theory shows that this kernel is *characteristic*, hence MMD metrises weak
|
||||
convergence on bounded subsets.
|
||||
|
||||
**Closed forms for two notable cases.**
|
||||
|
||||
* **Pure translation, equal samples.** If $Q$ is the law of $X + \Delta$ with $X \sim P$ on
|
||||
$\mathbb{R}$ and $P = \delta$ atomic, the squared MMD is $2 - 2 e^{-\Delta^2 / (2\sigma^2)}$ —
|
||||
smooth, monotone in $|\Delta|$, asymptote $2$ as $\Delta \to \infty$. This is the analytic
|
||||
ground-truth verified by the *bandwidth dependence* cell of the companion notebook.
|
||||
* **Two Gaussians.** For $P = \mathcal{N}(\mu_1, \sigma_1^2)$ and $Q = \mathcal{N}(\mu_2, \sigma_2^2)$,
|
||||
|
||||
.. math::
|
||||
|
||||
\mathrm{MMD}^2_\sigma(P, Q) \;=\;
|
||||
\frac{\sigma}{\sqrt{\sigma^2 + 2\sigma_1^2}}
|
||||
\;-\; \frac{2\sigma}{\sqrt{\sigma^2 + \sigma_1^2 + \sigma_2^2}}\, e^{-\frac{(\mu_1 - \mu_2)^2}{2(\sigma^2 + \sigma_1^2 + \sigma_2^2)}}
|
||||
\;+\; \frac{\sigma}{\sqrt{\sigma^2 + 2\sigma_2^2}} ,
|
||||
|
||||
giving an exact reference for unit tests.
|
||||
|
||||
**Statistical guarantee.** Gretton et al. (2012, Thm. 12) give the deviation bound
|
||||
$\Pr\!\bigl(\widehat{\mathrm{MMD}}^2 - \mathrm{MMD}^2 > \varepsilon\bigr) \le \exp\bigl(-\varepsilon^2 nm / (8 K^2 (n + m))\bigr)$
|
||||
for $|k| \le K$. Hence MMD detects fixed alternatives at the optimal $n^{-1/2}$ rate.
|
||||
|
||||
**Connection with Wasserstein.** Both metrise weak convergence, but MMD is *quadratic in the
|
||||
sample size* (no transport plan to solve) and admits unbiased low-variance gradient estimators —
|
||||
the reason it is the loss of choice in implicit-generative-model training
|
||||
(generator-loss / score-matching alternatives).
|
||||
|
||||
Why it matters
|
||||
--------------
|
||||
|
||||
* **Generative calibration.** Train an implicit sampler (neural SDE, copula generator, GAN-like
|
||||
architecture) by minimising $\widehat{\mathrm{MMD}}^2$ between the simulator output and the
|
||||
target distribution. The trait `GenerativeSampler` plus `calibration_step` is the abstract
|
||||
glue.
|
||||
* **Two-sample testing.** Distribution drift detection in streaming data, A/B-test signal
|
||||
extraction, anomaly detection.
|
||||
* **Model selection.** Replace likelihood ratios when likelihoods are intractable
|
||||
(simulator-based inference, ABC).
|
||||
|
||||
.. note::
|
||||
📓 **Companion notebook** — `view on GitHub <https://github.com/ThotDjehuty/optimiz-rs/blob/main/examples/notebooks/17_generative_calibration.ipynb>`_
|
||||
· `download .ipynb <https://raw.githubusercontent.com/ThotDjehuty/optimiz-rs/main/examples/notebooks/17_generative_calibration.ipynb>`_
|
||||
|
||||
17 — MMD calibration loss
|
||||
=========================
|
||||
|
||||
Reference in New Issue
Block a user