874 lines
33 KiB
Plaintext
874 lines
33 KiB
Plaintext
|
{
|
|||
|
|
"cells": [
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "118782fe",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Import required libraries\n",
|
|||
|
|
"import numpy as np\n",
|
|||
|
|
"import matplotlib.pyplot as plt\n",
|
|||
|
|
"from matplotlib import cm\n",
|
|||
|
|
"from mpl_toolkits.mplot3d import Axes3D\n",
|
|||
|
|
"import seaborn as sns\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Set style\n",
|
|||
|
|
"sns.set_style('whitegrid')\n",
|
|||
|
|
"plt.rcParams['figure.figsize'] = (14, 6)\n",
|
|||
|
|
"plt.rcParams['font.size'] = 11\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"✅ Libraries loaded successfully\")\n",
|
|||
|
|
"print(\"\\n📚 This tutorial covers:\")\n",
|
|||
|
|
"print(\" 1. Regime Switching Systems\")\n",
|
|||
|
|
"print(\" 2. Jump Diffusion Processes\")\n",
|
|||
|
|
"print(\" 3. Combined MRSJD Models\")\n",
|
|||
|
|
"print(\" 4. Numerical Methods (Finite Differences, Upwind Schemes)\")\n",
|
|||
|
|
"print(\" 5. Practical Parameter Selection\")"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "dcfec9d8",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## 2. Mathematical Background <a id=\"math\"></a>\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Stochastic Differential Equations (SDEs)\n",
|
|||
|
|
"\n",
|
|||
|
|
"A general SDE has the form:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"dX_t = \\mu(X_t)dt + \\sigma(X_t)dW_t\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"where:\n",
|
|||
|
|
"- $\\mu(X_t)$ = **drift** (deterministic trend)\n",
|
|||
|
|
"- $\\sigma(X_t)$ = **diffusion** (volatility)\n",
|
|||
|
|
"- $dW_t$ = **Wiener process** increment: $dW_t \\sim \\mathcal{N}(0, dt)$\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Key Properties\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Itô's Lemma** (chain rule for SDEs):\n",
|
|||
|
|
"\n",
|
|||
|
|
"For $Y_t = f(X_t)$:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"dY_t = f'(X_t)dX_t + \\frac{1}{2}f''(X_t)\\sigma^2(X_t)dt\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Feynman-Kac Formula** (connects PDEs to expectations):\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"V(x,t) = \\mathbb{E}_x\\left[ \\int_t^T e^{-\\rho(s-t)} L(X_s)ds + e^{-\\rho(T-t)}\\Phi(X_T) \\right]\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"satisfies the PDE:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\\frac{\\partial V}{\\partial t} + \\mu(x)\\frac{\\partial V}{\\partial x} + \\frac{1}{2}\\sigma^2(x)\\frac{\\partial^2 V}{\\partial x^2} - \\rho V + L(x) = 0\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Example: Ornstein-Uhlenbeck Process\n",
|
|||
|
|
"\n",
|
|||
|
|
"Mean-reverting process:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"dX_t = \\theta(\\mu - X_t)dt + \\sigma dW_t\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"- $\\theta$ = speed of mean reversion\n",
|
|||
|
|
"- $\\mu$ = long-term mean\n",
|
|||
|
|
"- $\\sigma$ = volatility\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Half-life**: $t_{1/2} = \\frac{\\ln 2}{\\theta}$"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "b25e1746",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Simulate Ornstein-Uhlenbeck process\n",
|
|||
|
|
"def simulate_ou(theta, mu, sigma, x0, T, dt):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Simulate Ornstein-Uhlenbeck process using Euler-Maruyama method\n",
|
|||
|
|
" \n",
|
|||
|
|
" dX_t = θ(μ - X_t)dt + σ dW_t\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" n_steps = int(T / dt)\n",
|
|||
|
|
" t = np.linspace(0, T, n_steps)\n",
|
|||
|
|
" X = np.zeros(n_steps)\n",
|
|||
|
|
" X[0] = x0\n",
|
|||
|
|
" \n",
|
|||
|
|
" for i in range(1, n_steps):\n",
|
|||
|
|
" dW = np.random.normal(0, np.sqrt(dt))\n",
|
|||
|
|
" X[i] = X[i-1] + theta * (mu - X[i-1]) * dt + sigma * dW\n",
|
|||
|
|
" \n",
|
|||
|
|
" return t, X\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Example: Temperature control\n",
|
|||
|
|
"theta = 0.5 # Mean reversion speed\n",
|
|||
|
|
"mu = 20.0 # Target temperature (°C)\n",
|
|||
|
|
"sigma = 2.0 # Noise level\n",
|
|||
|
|
"x0 = 10.0 # Initial temperature\n",
|
|||
|
|
"T = 10.0 # Time horizon (seconds)\n",
|
|||
|
|
"dt = 0.01 # Time step\n",
|
|||
|
|
"\n",
|
|||
|
|
"t, X = simulate_ou(theta, mu, sigma, x0, T, dt)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Plot\n",
|
|||
|
|
"fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Trajectory\n",
|
|||
|
|
"ax1.plot(t, X, linewidth=1.5, color='steelblue', label='Temperature')\n",
|
|||
|
|
"ax1.axhline(y=mu, color='red', linestyle='--', label=f'Target μ={mu}')\n",
|
|||
|
|
"ax1.fill_between(t, mu-sigma, mu+sigma, alpha=0.2, color='red', label='±σ band')\n",
|
|||
|
|
"ax1.set_xlabel('Time (s)')\n",
|
|||
|
|
"ax1.set_ylabel('Temperature (°C)')\n",
|
|||
|
|
"ax1.set_title('Ornstein-Uhlenbeck Process (Mean-Reverting System)')\n",
|
|||
|
|
"ax1.legend()\n",
|
|||
|
|
"ax1.grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Distribution at equilibrium\n",
|
|||
|
|
"equilibrium_samples = X[len(X)//2:] # Second half (near equilibrium)\n",
|
|||
|
|
"ax2.hist(equilibrium_samples, bins=30, density=True, alpha=0.7, color='steelblue', edgecolor='black')\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Theoretical distribution: N(μ, σ²/(2θ))\n",
|
|||
|
|
"x_range = np.linspace(X.min(), X.max(), 100)\n",
|
|||
|
|
"theoretical_std = sigma / np.sqrt(2 * theta)\n",
|
|||
|
|
"from scipy.stats import norm\n",
|
|||
|
|
"ax2.plot(x_range, norm.pdf(x_range, mu, theoretical_std), \n",
|
|||
|
|
" 'r-', linewidth=2, label=f'Theory: N({mu:.1f}, {theoretical_std:.2f}²)')\n",
|
|||
|
|
"\n",
|
|||
|
|
"ax2.set_xlabel('Temperature (°C)')\n",
|
|||
|
|
"ax2.set_ylabel('Probability Density')\n",
|
|||
|
|
"ax2.set_title('Equilibrium Distribution')\n",
|
|||
|
|
"ax2.legend()\n",
|
|||
|
|
"ax2.grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"plt.tight_layout()\n",
|
|||
|
|
"plt.show()\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(f\"\\n📊 OU Process Analysis:\")\n",
|
|||
|
|
"print(f\" Half-life: {np.log(2)/theta:.2f} seconds\")\n",
|
|||
|
|
"print(f\" Theoretical equilibrium std: {theoretical_std:.2f}\")\n",
|
|||
|
|
"print(f\" Observed equilibrium std: {equilibrium_samples.std():.2f}\")"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "78636a0b",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## 3. Regime Switching Systems <a id=\"regime\"></a>\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Motivation\n",
|
|||
|
|
"\n",
|
|||
|
|
"Many real systems exhibit **multiple operating modes** or **regimes**:\n",
|
|||
|
|
"- Weather: sunny ↔ rainy ↔ stormy\n",
|
|||
|
|
"- Manufacturing: normal ↔ maintenance ↔ failure\n",
|
|||
|
|
"- Traffic: free-flow ↔ congested ↔ gridlock\n",
|
|||
|
|
"- Economic activity: expansion ↔ recession\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Continuous-Time Markov Chain\n",
|
|||
|
|
"\n",
|
|||
|
|
"The regime $i_t \\in \\{1, 2, ..., N\\}$ follows a Markov chain with **transition rate matrix** $Q$:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\\mathbb{P}(i_{t+dt} = j | i_t = i) = \n",
|
|||
|
|
"\\begin{cases}\n",
|
|||
|
|
"q_{ij} dt & \\text{if } i \\neq j \\\\\n",
|
|||
|
|
"1 + q_{ii} dt & \\text{if } i = j\n",
|
|||
|
|
"\\end{cases}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"where $q_{ii} = -\\sum_{j \\neq i} q_{ij}$ (rows sum to zero).\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Coupled HJB System\n",
|
|||
|
|
"\n",
|
|||
|
|
"The value function $V^i(x)$ in regime $i$ satisfies:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\\rho V^i(x) = \\sup_u \\left[ \\mu^i(x,u) (V^i)'(x) + \\frac{1}{2}(\\sigma^i)^2(x,u) (V^i)''(x) + L^i(x,u) + \\sum_{j \\neq i} q_{ij}[V^j(x) - V^i(x)] \\right]\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"Key insight: The term $\\sum_{j \\neq i} q_{ij}[V^j(x) - V^i(x)]$ represents the **expected change in value due to regime switching**.\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Stationary Distribution\n",
|
|||
|
|
"\n",
|
|||
|
|
"The long-run probability of being in each regime solves:\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"Q^T \\pi = 0, \\quad \\sum_i \\pi_i = 1\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Parameter Selection Tips\n",
|
|||
|
|
"\n",
|
|||
|
|
"| Parameter | Typical Range | Effect | How to Choose |\n",
|
|||
|
|
"|-----------|--------------|--------|---------------|\n",
|
|||
|
|
"| $q_{ij}$ | 0.1 - 10.0 | Regime persistence | Higher = faster switching. Set $q_{ij} = 1/\\text{expected duration}$ |\n",
|
|||
|
|
"| $\\mu^i$ | Problem-specific | Drift in regime $i$ | Estimate from data or physics |\n",
|
|||
|
|
"| $\\sigma^i$ | $> 0$ | Volatility in regime $i$ | Measure from observations or experiments |\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Example**: If regime 1 typically lasts 5 time units, set $q_{12} \\approx 0.2$."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "79238099",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Simulate regime-switching process\n",
|
|||
|
|
"def simulate_regime_switching(Q, regime_params, x0, T, dt):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Simulate regime-switching stochastic process\n",
|
|||
|
|
" \n",
|
|||
|
|
" Args:\n",
|
|||
|
|
" Q: Transition rate matrix (N x N)\n",
|
|||
|
|
" regime_params: List of (mu, sigma) for each regime\n",
|
|||
|
|
" x0: Initial state\n",
|
|||
|
|
" T: Time horizon\n",
|
|||
|
|
" dt: Time step\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" n_steps = int(T / dt)\n",
|
|||
|
|
" n_regimes = Q.shape[0]\n",
|
|||
|
|
" \n",
|
|||
|
|
" t = np.linspace(0, T, n_steps)\n",
|
|||
|
|
" X = np.zeros(n_steps)\n",
|
|||
|
|
" regimes = np.zeros(n_steps, dtype=int)\n",
|
|||
|
|
" \n",
|
|||
|
|
" X[0] = x0\n",
|
|||
|
|
" regimes[0] = 0 # Start in regime 0\n",
|
|||
|
|
" \n",
|
|||
|
|
" for i in range(1, n_steps):\n",
|
|||
|
|
" current_regime = regimes[i-1]\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Check for regime transition\n",
|
|||
|
|
" for j in range(n_regimes):\n",
|
|||
|
|
" if j != current_regime:\n",
|
|||
|
|
" if np.random.rand() < Q[current_regime, j] * dt:\n",
|
|||
|
|
" current_regime = j\n",
|
|||
|
|
" break\n",
|
|||
|
|
" \n",
|
|||
|
|
" regimes[i] = current_regime\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Evolve state according to current regime\n",
|
|||
|
|
" mu, sigma = regime_params[current_regime]\n",
|
|||
|
|
" dW = np.random.normal(0, np.sqrt(dt))\n",
|
|||
|
|
" X[i] = X[i-1] + mu * dt + sigma * dW\n",
|
|||
|
|
" \n",
|
|||
|
|
" return t, X, regimes\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Example: 3-regime system (Slow/Normal/Fast)\n",
|
|||
|
|
"Q = np.array([\n",
|
|||
|
|
" [-0.5, 0.3, 0.2], # Slow regime\n",
|
|||
|
|
" [ 0.4, -0.7, 0.3], # Normal regime\n",
|
|||
|
|
" [ 0.3, 0.4, -0.7] # Fast regime\n",
|
|||
|
|
"])\n",
|
|||
|
|
"\n",
|
|||
|
|
"regime_params = [\n",
|
|||
|
|
" (0.1, 0.2), # Slow: low drift, low vol\n",
|
|||
|
|
" (0.3, 0.4), # Normal: medium drift, medium vol\n",
|
|||
|
|
" (0.5, 0.8) # Fast: high drift, high vol\n",
|
|||
|
|
"]\n",
|
|||
|
|
"\n",
|
|||
|
|
"t, X, regimes = simulate_regime_switching(Q, regime_params, x0=0.0, T=50.0, dt=0.01)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Plot\n",
|
|||
|
|
"fig, (ax1, ax2, ax3) = plt.subplots(3, 1, figsize=(14, 10), sharex=True)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# State trajectory\n",
|
|||
|
|
"colors = ['blue', 'green', 'red']\n",
|
|||
|
|
"for i in range(len(t)-1):\n",
|
|||
|
|
" ax1.plot(t[i:i+2], X[i:i+2], color=colors[regimes[i]], alpha=0.8, linewidth=0.8)\n",
|
|||
|
|
"\n",
|
|||
|
|
"ax1.set_ylabel('State X')\n",
|
|||
|
|
"ax1.set_title('Regime-Switching Process')\n",
|
|||
|
|
"ax1.grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Regime evolution\n",
|
|||
|
|
"ax2.step(t, regimes, where='post', linewidth=1.5, color='black')\n",
|
|||
|
|
"ax2.set_ylabel('Regime')\n",
|
|||
|
|
"ax2.set_yticks([0, 1, 2])\n",
|
|||
|
|
"ax2.set_yticklabels(['Slow', 'Normal', 'Fast'])\n",
|
|||
|
|
"ax2.set_title('Regime Evolution')\n",
|
|||
|
|
"ax2.grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Regime distribution\n",
|
|||
|
|
"regime_counts = np.bincount(regimes, minlength=3) / len(regimes)\n",
|
|||
|
|
"ax3.bar([0, 1, 2], regime_counts, color=colors, alpha=0.7, edgecolor='black')\n",
|
|||
|
|
"ax3.set_xlabel('Regime')\n",
|
|||
|
|
"ax3.set_ylabel('Frequency')\n",
|
|||
|
|
"ax3.set_xticks([0, 1, 2])\n",
|
|||
|
|
"ax3.set_xticklabels(['Slow', 'Normal', 'Fast'])\n",
|
|||
|
|
"ax3.set_title('Regime Distribution')\n",
|
|||
|
|
"ax3.grid(alpha=0.3, axis='y')\n",
|
|||
|
|
"\n",
|
|||
|
|
"plt.tight_layout()\n",
|
|||
|
|
"plt.show()\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Compute stationary distribution\n",
|
|||
|
|
"from scipy.linalg import null_space\n",
|
|||
|
|
"pi_stationary = null_space(Q.T)\n",
|
|||
|
|
"pi_stationary = pi_stationary / pi_stationary.sum()\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"\\n📊 Regime Switching Analysis:\")\n",
|
|||
|
|
"print(f\" Observed frequencies: {regime_counts}\")\n",
|
|||
|
|
"print(f\" Theoretical stationary: {pi_stationary.flatten()}\")"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "b3751412",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## 4. Jump Diffusion Processes <a id=\"jumps\"></a>\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Motivation\n",
|
|||
|
|
"\n",
|
|||
|
|
"Continuous diffusion models fail to capture **sudden, discrete events**:\n",
|
|||
|
|
"- Market crashes/rallies\n",
|
|||
|
|
"- Equipment failures\n",
|
|||
|
|
"- Policy changes\n",
|
|||
|
|
"- Natural disasters\n",
|
|||
|
|
"- Phase transitions\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Lévy Processes and Compound Poisson\n",
|
|||
|
|
"\n",
|
|||
|
|
"A jump diffusion process combines:\n",
|
|||
|
|
"1. **Continuous diffusion**: $\\sigma dW_t$\n",
|
|||
|
|
"2. **Discrete jumps**: $dJ_t = \\sum_{i=1}^{N_t} Y_i$\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"dX_t = \\mu dt + \\sigma dW_t + dJ_t\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"where:\n",
|
|||
|
|
"- $N_t \\sim \\text{Poisson}(\\lambda t)$ = number of jumps by time $t$\n",
|
|||
|
|
"- $Y_i \\sim F$ = jump size distribution\n",
|
|||
|
|
"- $\\lambda$ = **jump intensity** (expected jumps per unit time)\n",
|
|||
|
|
"\n",
|
|||
|
|
"### HJB with Jump Integral\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\\rho V(x) = \\sup_u \\left[ \\mu(x,u) V'(x) + \\frac{1}{2}\\sigma^2(x,u) V''(x) + L(x,u) + \\lambda \\int [V(x+y) - V(x)] F(dy) \\right]\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"The integral term $\\lambda \\mathbb{E}[V(x+Y) - V(x)]$ represents the **expected value change from jumps**.\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Jump Size Distributions\n",
|
|||
|
|
"\n",
|
|||
|
|
"| Distribution | Density | Use Case |\n",
|
|||
|
|
"|--------------|---------|----------|\n",
|
|||
|
|
"| Normal | $\\mathcal{N}(\\mu_j, \\sigma_j^2)$ | Symmetric jumps (up/down equally likely) |\n",
|
|||
|
|
"| Exponential | $\\lambda e^{-\\lambda y}$ | One-sided jumps (failures, crashes) |\n",
|
|||
|
|
"| Laplace | $\\frac{1}{2b}e^{-|y-\\mu|/b}$ | Heavy-tailed jumps |\n",
|
|||
|
|
"| Uniform | $U(a, b)$ | Bounded jumps |\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Parameter Selection\n",
|
|||
|
|
"\n",
|
|||
|
|
"| Parameter | Typical Range | Effect | How to Choose |\n",
|
|||
|
|
"|-----------|--------------|--------|---------------|\n",
|
|||
|
|
"| $\\lambda$ | 0.01 - 5.0 | Jump frequency | Count events per unit time from data |\n",
|
|||
|
|
"| $\\mu_j$ | Problem-specific | Average jump size | Measure typical event magnitude |\n",
|
|||
|
|
"| $\\sigma_j$ | $> 0$ | Jump size variability | Standard deviation of observed jumps |\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Rule of thumb**: If you expect ~1 jump per 10 time units, set $\\lambda = 0.1$."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "733539e5",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Simulate jump diffusion process\n",
|
|||
|
|
"def simulate_jump_diffusion(mu, sigma, lambda_jump, jump_mean, jump_std, x0, T, dt):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Simulate Merton jump diffusion model\n",
|
|||
|
|
" \n",
|
|||
|
|
" dX_t = μ dt + σ dW_t + dJ_t\n",
|
|||
|
|
" \n",
|
|||
|
|
" where J_t is compound Poisson with Normal jumps\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" n_steps = int(T / dt)\n",
|
|||
|
|
" t = np.linspace(0, T, n_steps)\n",
|
|||
|
|
" X = np.zeros(n_steps)\n",
|
|||
|
|
" jumps = np.zeros(n_steps)\n",
|
|||
|
|
" \n",
|
|||
|
|
" X[0] = x0\n",
|
|||
|
|
" \n",
|
|||
|
|
" for i in range(1, n_steps):\n",
|
|||
|
|
" # Diffusion component\n",
|
|||
|
|
" dW = np.random.normal(0, np.sqrt(dt))\n",
|
|||
|
|
" dX = mu * dt + sigma * dW\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Jump component\n",
|
|||
|
|
" n_jumps = np.random.poisson(lambda_jump * dt)\n",
|
|||
|
|
" if n_jumps > 0:\n",
|
|||
|
|
" jump_sizes = np.random.normal(jump_mean, jump_std, n_jumps)\n",
|
|||
|
|
" total_jump = jump_sizes.sum()\n",
|
|||
|
|
" dX += total_jump\n",
|
|||
|
|
" jumps[i] = total_jump\n",
|
|||
|
|
" \n",
|
|||
|
|
" X[i] = X[i-1] + dX\n",
|
|||
|
|
" \n",
|
|||
|
|
" return t, X, jumps\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Example: System with occasional failures/shocks\n",
|
|||
|
|
"mu = 0.5 # Baseline drift\n",
|
|||
|
|
"sigma = 0.3 # Continuous volatility\n",
|
|||
|
|
"lambda_jump = 2.0 # 2 jumps per time unit (on average)\n",
|
|||
|
|
"jump_mean = -0.5 # Negative jumps (failures)\n",
|
|||
|
|
"jump_std = 0.2 # Jump size variability\n",
|
|||
|
|
"x0 = 10.0 # Initial state\n",
|
|||
|
|
"T = 20.0 # Time horizon\n",
|
|||
|
|
"dt = 0.01\n",
|
|||
|
|
"\n",
|
|||
|
|
"t, X, jumps = simulate_jump_diffusion(mu, sigma, lambda_jump, jump_mean, jump_std, x0, T, dt)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Also simulate without jumps for comparison\n",
|
|||
|
|
"t_nodiff, X_nodiff, _ = simulate_jump_diffusion(mu, sigma, 0.0, 0.0, 0.0, x0, T, dt)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Plot\n",
|
|||
|
|
"fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(14, 10), sharex=True)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Trajectories comparison\n",
|
|||
|
|
"ax1.plot(t, X, linewidth=1.5, color='red', label='With Jumps', alpha=0.8)\n",
|
|||
|
|
"ax1.plot(t_nodiff, X_nodiff, linewidth=1.5, color='blue', label='Pure Diffusion', alpha=0.6)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Mark jump times\n",
|
|||
|
|
"jump_times = t[jumps != 0]\n",
|
|||
|
|
"jump_values = X[jumps != 0]\n",
|
|||
|
|
"ax1.scatter(jump_times, jump_values, color='black', s=50, zorder=5, label='Jump Events', alpha=0.7)\n",
|
|||
|
|
"\n",
|
|||
|
|
"ax1.set_ylabel('State X')\n",
|
|||
|
|
"ax1.set_title('Jump Diffusion Process vs Pure Diffusion')\n",
|
|||
|
|
"ax1.legend(fontsize=11)\n",
|
|||
|
|
"ax1.grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Jump sizes over time\n",
|
|||
|
|
"ax2.stem(t, jumps, linefmt='red', markerfmt='ro', basefmt=' ', label='Jump Sizes')\n",
|
|||
|
|
"ax2.axhline(y=0, color='black', linewidth=0.8)\n",
|
|||
|
|
"ax2.set_xlabel('Time')\n",
|
|||
|
|
"ax2.set_ylabel('Jump Size')\n",
|
|||
|
|
"ax2.set_title('Jump Events')\n",
|
|||
|
|
"ax2.grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"plt.tight_layout()\n",
|
|||
|
|
"plt.show()\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Statistics\n",
|
|||
|
|
"n_observed_jumps = np.sum(jumps != 0)\n",
|
|||
|
|
"expected_jumps = lambda_jump * T\n",
|
|||
|
|
"avg_jump_size = jumps[jumps != 0].mean() if n_observed_jumps > 0 else 0\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"\\n📊 Jump Diffusion Analysis:\")\n",
|
|||
|
|
"print(f\" Expected jumps: {expected_jumps:.1f}\")\n",
|
|||
|
|
"print(f\" Observed jumps: {n_observed_jumps}\")\n",
|
|||
|
|
"print(f\" Average jump size: {avg_jump_size:.3f} (theoretical: {jump_mean})\")\n",
|
|||
|
|
"print(f\" Std of jumps: {jumps[jumps != 0].std() if n_observed_jumps > 0 else 0:.3f} (theoretical: {jump_std})\")\n",
|
|||
|
|
"print(f\"\\n Impact: Final value with jumps = {X[-1]:.2f} vs {X_nodiff[-1]:.2f} without jumps\")"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "fca7aba3",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## 5. Combined MRSJD Models <a id=\"mrsjd\"></a>\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Why Combine Regime Switching and Jumps?\n",
|
|||
|
|
"\n",
|
|||
|
|
"Real systems often exhibit **both**:\n",
|
|||
|
|
"1. **State-dependent behavior** (regimes)\n",
|
|||
|
|
"2. **Sudden shocks** (jumps)\n",
|
|||
|
|
"\n",
|
|||
|
|
"Examples:\n",
|
|||
|
|
"- **Manufacturing**: Normal/maintenance regimes + equipment failures (jumps)\n",
|
|||
|
|
"- **Power grid**: Low/high demand regimes + blackout events (jumps)\n",
|
|||
|
|
"- **Epidemic**: Endemic/outbreak regimes + super-spreader events (jumps)\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Full MRSJD Dynamics\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"dX_t = \\mu^{i_t}(X_t)dt + \\sigma^{i_t}(X_t)dW_t + dJ_t^{i_t}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"where:\n",
|
|||
|
|
"- Drift $\\mu^i$ and volatility $\\sigma^i$ depend on current regime $i_t$\n",
|
|||
|
|
"- Jump intensity $\\lambda^i$ and distribution $F^i$ also regime-dependent\n",
|
|||
|
|
"- Regime switches according to $Q$\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Coupled HJB with Both Effects\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\\boxed{\n",
|
|||
|
|
"\\rho V^i(x) = \\sup_u \\left[ \\mu^i V^i_x + \\frac{(\\sigma^i)^2}{2} V^i_{xx} + L^i(x,u) + \\lambda^i \\int [V^i(x+y) - V^i(x)] F^i(dy) + \\sum_{j \\neq i} q_{ij}[V^j(x) - V^i(x)] \\right]\n",
|
|||
|
|
"}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"This is the **most general** formulation combining:\n",
|
|||
|
|
"1. ✅ Diffusion: $(\\sigma^i)^2 V^i_{xx}$\n",
|
|||
|
|
"2. ✅ Jumps: $\\lambda^i \\int [V^i(x+y) - V^i(x)] F^i(dy)$\n",
|
|||
|
|
"3. ✅ Regime Switching: $\\sum_{j \\neq i} q_{ij}[V^j(x) - V^i(x)]$\n",
|
|||
|
|
"4. ✅ Optimal Control: $\\sup_u$\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Numerical Solution: Finite Differences with Upwind Schemes\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Grid discretization**: $x_k = x_{\\min} + k \\Delta x$, $k = 0, ..., N$\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Value function approximation**: $V^i(x_k) \\approx V^i_k$\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Derivatives**:\n",
|
|||
|
|
"- Forward: $V_x \\approx (V_{k+1} - V_k) / \\Delta x$\n",
|
|||
|
|
"- Backward: $V_x \\approx (V_k - V_{k-1}) / \\Delta x$\n",
|
|||
|
|
"- Central: $V_{xx} \\approx (V_{k+1} - 2V_k + V_{k-1}) / (\\Delta x)^2$\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Upwind scheme** (for stability when $\\mu \\neq 0$):\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"V_x \\approx \n",
|
|||
|
|
"\\begin{cases}\n",
|
|||
|
|
"(V_{k+1} - V_k) / \\Delta x & \\text{if } \\mu > 0 \\text{ (forward)} \\\\\n",
|
|||
|
|
"(V_k - V_{k-1}) / \\Delta x & \\text{if } \\mu < 0 \\text{ (backward)}\n",
|
|||
|
|
"\\end{cases}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Why upwind?** Prevents numerical oscillations when advection dominates diffusion.\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Algorithm: Value Iteration\n",
|
|||
|
|
"\n",
|
|||
|
|
"```\n",
|
|||
|
|
"1. Initialize V^i_k = 0 for all regimes i and grid points k\n",
|
|||
|
|
"2. Repeat until convergence:\n",
|
|||
|
|
" For each regime i:\n",
|
|||
|
|
" For each grid point k:\n",
|
|||
|
|
" a. Compute derivatives V_x, V_xx\n",
|
|||
|
|
" b. Compute jump integral ∫[V(x+y) - V(x)]F(dy)\n",
|
|||
|
|
" c. Compute regime switching term Σ q_ij[V^j - V^i]\n",
|
|||
|
|
" d. Optimize over control: u* = argmax_u RHS(u)\n",
|
|||
|
|
" e. Update: V^i_k ← RHS(u*) / ρ\n",
|
|||
|
|
"3. Convergence check: ||V_new - V_old|| < tol\n",
|
|||
|
|
"```\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Computational Complexity\n",
|
|||
|
|
"\n",
|
|||
|
|
"- **Per iteration**: $O(N \\cdot M \\cdot K)$\n",
|
|||
|
|
" - $N$ = number of regimes\n",
|
|||
|
|
" - $M$ = grid points\n",
|
|||
|
|
" - $K$ = control discretization\n",
|
|||
|
|
"- **Iterations**: Typically 100-1000\n",
|
|||
|
|
"- **Total**: $O(10^5 - 10^7)$ operations\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Speedup techniques**:\n",
|
|||
|
|
"- Parallel computation across regimes (Rayon)\n",
|
|||
|
|
"- Adaptive grid refinement\n",
|
|||
|
|
"- Policy iteration instead of value iteration\n",
|
|||
|
|
"- Sparse matrix operations"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "f104f849",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Simplified MRSJD simulation (for illustration)\n",
|
|||
|
|
"def simulate_mrsjd(Q, regime_params_list, x0, T, dt):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Simulate Markov Regime Switching Jump Diffusion\n",
|
|||
|
|
" \n",
|
|||
|
|
" Each regime has: (mu, sigma, lambda_jump, jump_mean, jump_std)\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" n_steps = int(T / dt)\n",
|
|||
|
|
" n_regimes = Q.shape[0]\n",
|
|||
|
|
" \n",
|
|||
|
|
" t = np.linspace(0, T, n_steps)\n",
|
|||
|
|
" X = np.zeros(n_steps)\n",
|
|||
|
|
" regimes = np.zeros(n_steps, dtype=int)\n",
|
|||
|
|
" jump_events = []\n",
|
|||
|
|
" \n",
|
|||
|
|
" X[0] = x0\n",
|
|||
|
|
" regimes[0] = 0\n",
|
|||
|
|
" \n",
|
|||
|
|
" for i in range(1, n_steps):\n",
|
|||
|
|
" current_regime = regimes[i-1]\n",
|
|||
|
|
" mu, sigma, lam, jmu, jsig = regime_params_list[current_regime]\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Check regime transition\n",
|
|||
|
|
" for j in range(n_regimes):\n",
|
|||
|
|
" if j != current_regime and np.random.rand() < Q[current_regime, j] * dt:\n",
|
|||
|
|
" current_regime = j\n",
|
|||
|
|
" break\n",
|
|||
|
|
" \n",
|
|||
|
|
" regimes[i] = current_regime\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Diffusion\n",
|
|||
|
|
" dW = np.random.normal(0, np.sqrt(dt))\n",
|
|||
|
|
" dX = mu * dt + sigma * dW\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Jumps (regime-dependent)\n",
|
|||
|
|
" n_jumps = np.random.poisson(lam * dt)\n",
|
|||
|
|
" if n_jumps > 0:\n",
|
|||
|
|
" jump_size = np.sum(np.random.normal(jmu, jsig, n_jumps))\n",
|
|||
|
|
" dX += jump_size\n",
|
|||
|
|
" jump_events.append((t[i], jump_size, current_regime))\n",
|
|||
|
|
" \n",
|
|||
|
|
" X[i] = X[i-1] + dX\n",
|
|||
|
|
" \n",
|
|||
|
|
" return t, X, regimes, jump_events\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Example: 2-regime system with regime-dependent jumps\n",
|
|||
|
|
"Q = np.array([\n",
|
|||
|
|
" [-0.3, 0.3],\n",
|
|||
|
|
" [0.5, -0.5]\n",
|
|||
|
|
"])\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Regime 0: Stable (low vol, rare small jumps)\n",
|
|||
|
|
"# Regime 1: Volatile (high vol, frequent large jumps)\n",
|
|||
|
|
"regime_params_list = [\n",
|
|||
|
|
" (0.2, 0.3, 0.5, -0.1, 0.05), # Stable: mu, sigma, lambda, jump_mu, jump_sigma\n",
|
|||
|
|
" (0.1, 0.8, 2.0, -0.3, 0.15) # Volatile\n",
|
|||
|
|
"]\n",
|
|||
|
|
"\n",
|
|||
|
|
"t, X, regimes, jump_events = simulate_mrsjd(Q, regime_params_list, x0=5.0, T=30.0, dt=0.01)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Plot\n",
|
|||
|
|
"fig, axes = plt.subplots(3, 1, figsize=(14, 12), sharex=True)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# State trajectory\n",
|
|||
|
|
"regime_colors = ['blue', 'red']\n",
|
|||
|
|
"for i in range(len(t)-1):\n",
|
|||
|
|
" axes[0].plot(t[i:i+2], X[i:i+2], color=regime_colors[regimes[i]], alpha=0.8, linewidth=1.0)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Mark jumps\n",
|
|||
|
|
"if jump_events:\n",
|
|||
|
|
" jump_t = [j[0] for j in jump_events]\n",
|
|||
|
|
" jump_idx = [np.argmin(np.abs(t - jt)) for jt in jump_t]\n",
|
|||
|
|
" axes[0].scatter([t[i] for i in jump_idx], [X[i] for i in jump_idx], \n",
|
|||
|
|
" color='black', s=60, zorder=5, marker='x', label='Jumps')\n",
|
|||
|
|
"\n",
|
|||
|
|
"axes[0].set_ylabel('State X')\n",
|
|||
|
|
"axes[0].set_title('MRSJD: Combined Regime Switching + Jump Diffusion')\n",
|
|||
|
|
"axes[0].legend()\n",
|
|||
|
|
"axes[0].grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Regime evolution\n",
|
|||
|
|
"axes[1].step(t, regimes, where='post', linewidth=1.5, color='black')\n",
|
|||
|
|
"axes[1].fill_between(t, regimes, alpha=0.3, step='post', \n",
|
|||
|
|
" color=['blue' if r==0 else 'red' for r in regimes])\n",
|
|||
|
|
"axes[1].set_ylabel('Regime')\n",
|
|||
|
|
"axes[1].set_yticks([0, 1])\n",
|
|||
|
|
"axes[1].set_yticklabels(['Stable', 'Volatile'])\n",
|
|||
|
|
"axes[1].set_title('Regime Transitions')\n",
|
|||
|
|
"axes[1].grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Jump events by regime\n",
|
|||
|
|
"if jump_events:\n",
|
|||
|
|
" regime_0_jumps = [j for j in jump_events if j[2] == 0]\n",
|
|||
|
|
" regime_1_jumps = [j for j in jump_events if j[2] == 1]\n",
|
|||
|
|
" \n",
|
|||
|
|
" if regime_0_jumps:\n",
|
|||
|
|
" axes[2].scatter([j[0] for j in regime_0_jumps], [j[1] for j in regime_0_jumps],\n",
|
|||
|
|
" color='blue', s=50, alpha=0.7, label='Stable Regime Jumps')\n",
|
|||
|
|
" if regime_1_jumps:\n",
|
|||
|
|
" axes[2].scatter([j[0] for j in regime_1_jumps], [j[1] for j in regime_1_jumps],\n",
|
|||
|
|
" color='red', s=50, alpha=0.7, label='Volatile Regime Jumps')\n",
|
|||
|
|
"\n",
|
|||
|
|
"axes[2].axhline(y=0, color='black', linewidth=0.8)\n",
|
|||
|
|
"axes[2].set_xlabel('Time')\n",
|
|||
|
|
"axes[2].set_ylabel('Jump Size')\n",
|
|||
|
|
"axes[2].set_title('Jump Events by Regime')\n",
|
|||
|
|
"axes[2].legend()\n",
|
|||
|
|
"axes[2].grid(alpha=0.3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"plt.tight_layout()\n",
|
|||
|
|
"plt.show()\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Statistics\n",
|
|||
|
|
"print(\"\\n📊 MRSJD Analysis:\")\n",
|
|||
|
|
"print(f\" Total jumps: {len(jump_events)}\")\n",
|
|||
|
|
"regime_times = [np.sum(regimes == i) * dt for i in range(2)]\n",
|
|||
|
|
"print(f\" Time in Stable regime: {regime_times[0]:.1f} ({regime_times[0]/T*100:.1f}%)\")\n",
|
|||
|
|
"print(f\" Time in Volatile regime: {regime_times[1]:.1f} ({regime_times[1]/T*100:.1f}%)\")\n",
|
|||
|
|
"if jump_events:\n",
|
|||
|
|
" avg_jump_0 = np.mean([j[1] for j in jump_events if j[2] == 0]) if len([j for j in jump_events if j[2] == 0]) > 0 else 0\n",
|
|||
|
|
" avg_jump_1 = np.mean([j[1] for j in jump_events if j[2] == 1]) if len([j for j in jump_events if j[2] == 1]) > 0 else 0\n",
|
|||
|
|
" print(f\" Average jump size (Stable): {avg_jump_0:.3f}\")\n",
|
|||
|
|
" print(f\" Average jump size (Volatile): {avg_jump_1:.3f}\")"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "822b77f8",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## 6. Practical Parameter Selection Guide <a id=\"params\"></a>\n",
|
|||
|
|
"\n",
|
|||
|
|
"### How to Choose Parameters for Your Problem\n",
|
|||
|
|
"\n",
|
|||
|
|
"#### Step 1: Identify Regimes\n",
|
|||
|
|
"\n",
|
|||
|
|
"Ask: Does the system have distinct \"modes\" or \"states\"?\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Examples**:\n",
|
|||
|
|
"- Manufacturing: Normal / Degraded / Failed\n",
|
|||
|
|
"- Weather: Clear / Cloudy / Storm\n",
|
|||
|
|
"- Network: Low / Medium / High traffic\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Tip**: Start with 2-3 regimes. More regimes = more parameters to estimate.\n",
|
|||
|
|
"\n",
|
|||
|
|
"#### Step 2: Estimate Regime Persistence\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Question**: How long does each regime typically last?\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Formula**: $q_{ij} = \\frac{1}{\\text{expected duration in regime } i}$\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Example**: If \"Normal\" regime lasts ~10 time units:\n",
|
|||
|
|
"- Total exit rate from Normal: $q_{01} + q_{02} = 0.1$\n",
|
|||
|
|
"- Split based on transition probabilities\n",
|
|||
|
|
"\n",
|
|||
|
|
"#### Step 3: Characterize Within-Regime Dynamics\n",
|
|||
|
|
"\n",
|
|||
|
|
"For each regime $i$:\n",
|
|||
|
|
"\n",
|
|||
|
|
"| Parameter | Method | Example |\n",
|
|||
|
|
"|-----------|--------|----------|\n",
|
|||
|
|
"| $\\mu^i$ | Sample mean of increments | $\\bar{\\Delta X} / \\Delta t$ |\n",
|
|||
|
|
"| $\\sigma^i$ | Sample std of increments | $\\text{std}(\\Delta X) / \\sqrt{\\Delta t}$ |\n",
|
|||
|
|
"| $\\lambda^i$ | Count events per time | $N_{\\text{jumps}} / T$ |\n",
|
|||
|
|
"| Jump mean | Average jump size | $\\bar{Y}$ |\n",
|
|||
|
|
"| Jump std | Std of jump sizes | $\\text{std}(Y)$ |\n",
|
|||
|
|
"\n",
|
|||
|
|
"#### Step 4: Validate with Simulations\n",
|
|||
|
|
"\n",
|
|||
|
|
"Before solving the HJB:\n",
|
|||
|
|
"1. Simulate the process with chosen parameters\n",
|
|||
|
|
"2. Check if trajectories \"look right\"\n",
|
|||
|
|
"3. Compare summary statistics to data\n",
|
|||
|
|
"4. Adjust and iterate\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Common Pitfalls and Solutions\n",
|
|||
|
|
"\n",
|
|||
|
|
"| Problem | Symptom | Solution |\n",
|
|||
|
|
"|---------|---------|----------|\n",
|
|||
|
|
"| Too many regimes | Overfitting, unstable estimates | Use 2-3 regimes; combine similar ones |\n",
|
|||
|
|
"| Wrong time scale | Unrealistic dynamics | Match $q_{ij}$ to actual durations |\n",
|
|||
|
|
"| Numerical instability | Oscillations, divergence | Reduce grid spacing, use upwind scheme |\n",
|
|||
|
|
"| Slow convergence | Many iterations needed | Better initial guess, increase tolerance |\n",
|
|||
|
|
"| High dimensionality | Curse of dimensionality | Reduce state space, use approximations |\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Sensitivity Analysis\n",
|
|||
|
|
"\n",
|
|||
|
|
"Always check how results change with parameters:\n",
|
|||
|
|
"1. Vary each parameter by ±20%\n",
|
|||
|
|
"2. Observe impact on optimal policy and value function\n",
|
|||
|
|
"3. Identify which parameters matter most\n",
|
|||
|
|
"4. Focus calibration efforts on sensitive parameters\n",
|
|||
|
|
"\n",
|
|||
|
|
"### When to Use vs. Not Use\n",
|
|||
|
|
"\n",
|
|||
|
|
"✅ **Good fit for HJB optimal control**:\n",
|
|||
|
|
"- Continuous state space (position, temperature, concentration)\n",
|
|||
|
|
"- Known or learnable dynamics\n",
|
|||
|
|
"- Quantifiable objectives\n",
|
|||
|
|
"- Medium-dimensional problems (1-3 state variables)\n",
|
|||
|
|
"- Offline planning acceptable\n",
|
|||
|
|
"\n",
|
|||
|
|
"❌ **Not recommended**:\n",
|
|||
|
|
"- Purely discrete decisions (use dynamic programming)\n",
|
|||
|
|
"- Unknown dynamics (use reinforcement learning)\n",
|
|||
|
|
"- High-dimensional state (>5 variables)\n",
|
|||
|
|
"- Real-time requirements (<1ms response)\n",
|
|||
|
|
"- Purely deterministic problems (use calculus of variations)\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Alternative Approaches\n",
|
|||
|
|
"\n",
|
|||
|
|
"| Method | When to Use | Pros | Cons |\n",
|
|||
|
|
"|--------|-------------|------|------|\n",
|
|||
|
|
"| **LQR/LQG** | Linear dynamics, quadratic cost | Fast, analytical solution | Limited to LQ problems |\n",
|
|||
|
|
"| **MPC** | Need real-time receding horizon | Handles constraints well | Computational cost |\n",
|
|||
|
|
"| **RL (DQN, PPO)** | Unknown dynamics | Model-free, flexible | Sample inefficient |\n",
|
|||
|
|
"| **PID Control** | Simple SISO systems | Easy to tune | No optimality guarantee |\n",
|
|||
|
|
"| **Bang-Bang** | Hard constraints | Simple implementation | Non-smooth control |\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Further Reading\n",
|
|||
|
|
"\n",
|
|||
|
|
"1. **Books**:\n",
|
|||
|
|
" - Fleming & Rishel: \"Deterministic and Stochastic Optimal Control\"\n",
|
|||
|
|
" - Øksendal & Sulem: \"Applied Stochastic Control of Jump Diffusions\"\n",
|
|||
|
|
" - Bertsekas: \"Dynamic Programming and Optimal Control\"\n",
|
|||
|
|
"\n",
|
|||
|
|
"2. **Papers**:\n",
|
|||
|
|
" - Guo & Hernandez-Lerma: \"Continuous-Time Markov Decision Processes\"\n",
|
|||
|
|
" - Pham: \"Continuous-time Stochastic Control and Optimization with Financial Applications\"\n",
|
|||
|
|
"\n",
|
|||
|
|
"3. **Software**:\n",
|
|||
|
|
" - This library (optimizr): Generic optimal control solvers\n",
|
|||
|
|
" - PROPT: MATLAB optimal control toolbox\n",
|
|||
|
|
" - CasADi: Nonlinear optimization and optimal control"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "432ef4af",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## Summary and Next Steps\n",
|
|||
|
|
"\n",
|
|||
|
|
"### What We Covered\n",
|
|||
|
|
"\n",
|
|||
|
|
"1. ✅ **Optimal Control Theory**: HJB equations, value functions\n",
|
|||
|
|
"2. ✅ **Regime Switching**: Markov chains, coupled HJB systems\n",
|
|||
|
|
"3. ✅ **Jump Diffusion**: Lévy processes, compound Poisson\n",
|
|||
|
|
"4. ✅ **MRSJD Models**: Combined framework for complex systems\n",
|
|||
|
|
"5. ✅ **Numerical Methods**: Finite differences, upwind schemes, value iteration\n",
|
|||
|
|
"6. ✅ **Parameter Selection**: Practical guidance and sensitivity analysis\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Key Takeaways\n",
|
|||
|
|
"\n",
|
|||
|
|
"- Optimal control finds the **best** policy, not just a good one\n",
|
|||
|
|
"- HJB equations require **solving PDEs** (computational cost)\n",
|
|||
|
|
"- Regime switching captures **state-dependent behavior**\n",
|
|||
|
|
"- Jumps model **sudden events** and tail risk\n",
|
|||
|
|
"- Start simple (2 regimes, pure diffusion) then add complexity\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Exercises for Practice\n",
|
|||
|
|
"\n",
|
|||
|
|
"1. **Temperature Control**: Design an optimal heating/cooling policy to maintain room temperature near 20°C while minimizing energy cost\n",
|
|||
|
|
"\n",
|
|||
|
|
"2. **Inventory Management**: Optimize reorder policy for warehouse with regime-switching demand (normal/holiday)\n",
|
|||
|
|
"\n",
|
|||
|
|
"3. **Robot Navigation**: Find optimal path for robot avoiding obstacles with uncertain dynamics and occasional sensor failures (jumps)\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Next Tutorial\n",
|
|||
|
|
"\n",
|
|||
|
|
"- **Hidden Markov Models (HMM)**: When regime is not directly observable\n",
|
|||
|
|
"- **MCMC Sampling**: Bayesian inference for parameter estimation\n",
|
|||
|
|
"- **Sparse Optimization**: High-dimensional problems with sparsity\n",
|
|||
|
|
"\n",
|
|||
|
|
"---\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Questions?** Open an issue on the repository or consult the API documentation.\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Happy Optimizing! 🚀**"
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"metadata": {
|
|||
|
|
"language_info": {
|
|||
|
|
"name": "python"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"nbformat": 4,
|
|||
|
|
"nbformat_minor": 5
|
|||
|
|
}
|