{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0a22bd06",
   "metadata": {},
   "source": [
    "# Graph Laplacians and Spectral Clustering\n",
    "\n",
    "Companion notebook for the `graph` module of `optimiz-rs`.\n",
    "\n",
    "Reference documentation: [graph_spectral.html](https://optimiz-r.readthedocs.io/en/latest/algorithms/graph_spectral.html)\n",
    "\n",
    "We demonstrate the four Python-exposed primitives:\n",
    "\n",
    "1. `combinatorial_laplacian_py`\n",
    "2. `normalised_laplacian_py`\n",
    "3. `random_walk_laplacian_py`\n",
    "4. `spectral_cluster_py`\n",
    "\n",
    "Each is verified against an analytic ground truth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3e190cea",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-12T09:44:15.206941Z",
     "iopub.status.busy": "2026-05-12T09:44:15.206599Z",
     "iopub.status.idle": "2026-05-12T09:44:16.002530Z",
     "shell.execute_reply": "2026-05-12T09:44:16.000885Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from optimizr import _core as opt\n",
    "\n",
    "rng = np.random.default_rng(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af83bd65",
   "metadata": {},
   "source": [
    "## 1. Combinatorial Laplacian\n",
    "\n",
    "For a non-negative symmetric weight matrix $W \\in \\mathbb{R}^{n\\times n}$ with degree matrix $D = \\mathrm{diag}(W \\mathbf{1})$:\n",
    "\n",
    "$$\n",
    "L \\;=\\; D - W .\n",
    "$$\n",
    "\n",
    "It is symmetric positive semidefinite and $L \\mathbf{1} = 0$, so the constant vector lies in its kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c0ae98c1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-12T09:44:16.006248Z",
     "iopub.status.busy": "2026-05-12T09:44:16.005847Z",
     "iopub.status.idle": "2026-05-12T09:44:16.362683Z",
     "shell.execute_reply": "2026-05-12T09:44:16.361515Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "L =\n",
      " [[ 2. -1. -1.]\n",
      " [-1.  2. -1.]\n",
      " [-1. -1.  2.]]\n",
      "|| L @ 1 ||_inf = 0.0\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 20856 characters]",
      "text/plain": [
       "<Figure size 400x320 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Triangle graph (3-clique).\n",
    "W3 = [\n",
    "    [0.0, 1.0, 1.0],\n",
    "    [1.0, 0.0, 1.0],\n",
    "    [1.0, 1.0, 0.0],\n",
    "]\n",
    "L = np.array(opt.combinatorial_laplacian_py(W3))\n",
    "print('L =\\n', L)\n",
    "\n",
    "# Analytic check: L @ 1 = 0.\n",
    "ones = np.ones(3)\n",
    "err_kernel = float(np.max(np.abs(L @ ones)))\n",
    "print('|| L @ 1 ||_inf =', err_kernel)\n",
    "assert err_kernel < 1e-12\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(4, 3.2))\n",
    "im = ax.imshow(L, cmap='RdBu_r', vmin=-2, vmax=2)\n",
    "ax.set_title('Combinatorial Laplacian (3-clique)')\n",
    "plt.colorbar(im, ax=ax)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13cb95ba",
   "metadata": {},
   "source": [
    "## 2. Symmetric normalised Laplacian\n",
    "\n",
    "$$\n",
    "L_{\\mathrm{sym}} \\;=\\; I - D^{-1/2} W D^{-1/2}.\n",
    "$$\n",
    "\n",
    "Eigenvalues lie in $[0, 2]$ and the multiplicity of the zero eigenvalue equals the number of connected components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cc02bdc7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-12T09:44:16.365813Z",
     "iopub.status.busy": "2026-05-12T09:44:16.365536Z",
     "iopub.status.idle": "2026-05-12T09:44:17.003051Z",
     "shell.execute_reply": "2026-05-12T09:44:17.001539Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eigenvalues = [4.4408921e-16 4.4408921e-16 1.5000000e+00 1.5000000e+00 1.5000000e+00\n",
      " 1.5000000e+00]\n",
      "number of zero eigenvalues = 2\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 32480 characters]",
      "text/plain": [
       "<Figure size 800x320 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Two disconnected triangles -> two zero eigenvalues.\n",
    "W6 = np.zeros((6, 6))\n",
    "for (i, j) in [(0, 1), (1, 2), (0, 2), (3, 4), (4, 5), (3, 5)]:\n",
    "    W6[i, j] = 1.0\n",
    "    W6[j, i] = 1.0\n",
    "Lsym = np.array(opt.normalised_laplacian_py(W6.tolist()))\n",
    "eigvals = np.sort(np.linalg.eigvalsh(Lsym))\n",
    "print('eigenvalues =', eigvals)\n",
    "\n",
    "# Two connected components -> exactly two near-zero eigenvalues.\n",
    "n_zero = int(np.sum(np.abs(eigvals) < 1e-10))\n",
    "print('number of zero eigenvalues =', n_zero)\n",
    "assert n_zero == 2\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(8, 3.2))\n",
    "im = axes[0].imshow(Lsym, cmap='RdBu_r', vmin=-1, vmax=1)\n",
    "axes[0].set_title(r'$L_{\\mathrm{sym}}$ (2 disconnected triangles)')\n",
    "plt.colorbar(im, ax=axes[0])\n",
    "axes[1].plot(eigvals, 'o-')\n",
    "axes[1].axhline(0.0, color='k', linewidth=0.5)\n",
    "axes[1].set_xlabel('index')\n",
    "axes[1].set_ylabel('eigenvalue')\n",
    "axes[1].set_title('Spectrum of $L_{\\\\mathrm{sym}}$')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee010f7",
   "metadata": {},
   "source": [
    "## 3. Random-walk Laplacian\n",
    "\n",
    "$$\n",
    "L_{\\mathrm{rw}} \\;=\\; I - D^{-1} W.\n",
    "$$\n",
    "\n",
    "The matrix $P = D^{-1} W$ is the row-stochastic transition matrix of the simple random walk on the graph. Hence $L_{\\mathrm{rw}} \\mathbf{1} = 0$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "249bb72a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-12T09:44:17.006260Z",
     "iopub.status.busy": "2026-05-12T09:44:17.005978Z",
     "iopub.status.idle": "2026-05-12T09:44:17.298401Z",
     "shell.execute_reply": "2026-05-12T09:44:17.297301Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "row sums of P = [1. 1. 1. 1. 1. 1.]\n",
      "|| P @ 1 - 1 ||_inf = 0.0\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 20112 characters]",
      "text/plain": [
       "<Figure size 400x320 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "Lrw = np.array(opt.random_walk_laplacian_py(W6.tolist()))\n",
    "P = np.eye(6) - Lrw\n",
    "row_sums = P.sum(axis=1)\n",
    "print('row sums of P =', row_sums)\n",
    "err_stochastic = float(np.max(np.abs(row_sums - 1.0)))\n",
    "print('|| P @ 1 - 1 ||_inf =', err_stochastic)\n",
    "assert err_stochastic < 1e-12\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(4, 3.2))\n",
    "im = ax.imshow(Lrw, cmap='RdBu_r', vmin=-1, vmax=1)\n",
    "ax.set_title(r'$L_{\\mathrm{rw}}$ (2 disconnected triangles)')\n",
    "plt.colorbar(im, ax=ax)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baaf3fe7",
   "metadata": {},
   "source": [
    "## 4. Spectral clustering (Ng--Jordan--Weiss)\n",
    "\n",
    "Given a similarity matrix $W$ and a target number of clusters $k$:\n",
    "\n",
    "1. Build $L_{\\mathrm{sym}} = I - D^{-1/2} W D^{-1/2}$.\n",
    "2. Stack the $k$ eigenvectors of $L_{\\mathrm{sym}}$ associated with the smallest eigenvalues as columns of $U \\in \\mathbb{R}^{n \\times k}$.\n",
    "3. Row-normalise $U$ and apply Lloyd's $k$-means with k-means++ seeding to its rows.\n",
    "\n",
    "We test on two well-separated 2D Gaussian blobs. Ground truth: cluster purity equals $1.0$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ddbc3936",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-12T09:44:17.302450Z",
     "iopub.status.busy": "2026-05-12T09:44:17.302058Z",
     "iopub.status.idle": "2026-05-12T09:44:17.805918Z",
     "shell.execute_reply": "2026-05-12T09:44:17.803760Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first 6 eigenvalues = [-6.47704929e-16  6.07023906e-07  9.56672916e-01  9.57721212e-01\n",
      "  9.82109668e-01  9.84579761e-01]\n",
      "fiedler value      = 6.070239058352574e-07\n",
      "cluster purity = 1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 57896 characters]",
      "text/plain": [
       "<Figure size 900x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_per = 30\n",
    "X1 = rng.normal(loc=[-3.0, 0.0], scale=0.35, size=(n_per, 2))\n",
    "X2 = rng.normal(loc=[+3.0, 0.0], scale=0.35, size=(n_per, 2))\n",
    "X = np.vstack([X1, X2])\n",
    "y_true = np.array([0] * n_per + [1] * n_per)\n",
    "n = X.shape[0]\n",
    "\n",
    "# Gaussian similarity, zero diagonal.\n",
    "sigma = 1.0\n",
    "D2 = np.sum((X[:, None, :] - X[None, :, :]) ** 2, axis=-1)\n",
    "W = np.exp(-D2 / (2.0 * sigma ** 2))\n",
    "np.fill_diagonal(W, 0.0)\n",
    "\n",
    "result = opt.spectral_cluster_py(W.tolist(), k=2, n_kmeans_iter=200, seed=7)\n",
    "labels = np.array(result['labels'])\n",
    "eigvals = np.array(result['eigenvalues'])\n",
    "fiedler = result['fiedler_value']\n",
    "print('first 6 eigenvalues =', eigvals[:6])\n",
    "print('fiedler value      =', fiedler)\n",
    "\n",
    "# Cluster purity (label-permutation invariant).\n",
    "def purity(y_true, y_pred):\n",
    "    classes = np.unique(y_pred)\n",
    "    correct = 0\n",
    "    for c in classes:\n",
    "        mask = y_pred == c\n",
    "        if mask.any():\n",
    "            correct += int(np.bincount(y_true[mask]).max())\n",
    "    return correct / len(y_true)\n",
    "\n",
    "p = purity(y_true, labels)\n",
    "print('cluster purity =', p)\n",
    "assert p == 1.0\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(9, 3.6))\n",
    "axes[0].scatter(X[:, 0], X[:, 1], c=labels, cmap='coolwarm', edgecolor='k')\n",
    "axes[0].set_title('Spectral cluster labels')\n",
    "axes[0].set_xlabel('x')\n",
    "axes[0].set_ylabel('y')\n",
    "axes[1].plot(eigvals[:10], 'o-')\n",
    "axes[1].axhline(0.0, color='k', linewidth=0.5)\n",
    "axes[1].set_xlabel('index')\n",
    "axes[1].set_ylabel('eigenvalue')\n",
    "axes[1].set_title(r'Smallest 10 eigenvalues of $L_{\\mathrm{sym}}$')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17d3fce2",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "Verified against analytic ground truth:\n",
    "\n",
    "- $L \\mathbf{1} = 0$ for the combinatorial Laplacian (3-clique) — error $< 10^{-12}$.\n",
    "- For two disconnected triangles, $L_{\\mathrm{sym}}$ has exactly $2$ zero eigenvalues — error $< 10^{-10}$.\n",
    "- $P = I - L_{\\mathrm{rw}}$ is row-stochastic, $P \\mathbf{1} = \\mathbf{1}$ — error $< 10^{-12}$.\n",
    "- Spectral clustering recovers two well-separated Gaussian blobs with purity $= 1.0$ — error $= 0$."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
