Initial commit: Quantum Terminal — Free & Open Source Trading Platform

.gitignore

@@ -0,0 +1,51 @@
+# === Python ===
+__pycache__/
+*.py[cod]
+*$py.class
+*.pyo
+*.egg-info/
+*.egg
+dist/
+*.whl
+
+# === Virtual Environments ===
+.env
+.venv
+env/
+venv/
+ENV/
+
+# === IDE / Editors ===
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# === Node.js ===
+node_modules/
+
+# === Build Artifacts ===
+build/
+*.spec
+
+# === OS Files ===
+.DS_Store
+Thumbs.db
+desktop.ini
+
+# === Logs & Databases ===
+*.log
+*.sqlite3
+*.db
+logs/
+
+# === Local Config (generated at runtime) ===
+consumer_config.ini
+cache/
+
+# === Data Library (local market data — large files) ===
+data_library/
+
+# === Test Caches ===
+.pytest_cache/

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Quantum Terminal Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,108 @@
+# 🌌 Quantum Terminal
+
+<div align="center">
+  <p><strong>A Next-Generation Open Source Trading & Quantitative Analysis Platform</strong></p>
+  <img src="https://img.shields.io/badge/Python-3.10+-blue.svg" alt="Python Version">
+  <img src="https://img.shields.io/badge/FastAPI-Modern_API-009688.svg" alt="FastAPI">
+  <img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License">
+  <img src="https://img.shields.io/badge/Architecture-Electron%20%7C%20React%20%7C%20Python-orange.svg" alt="Architecture">
+</div>
+
+---
+
+## 📖 About The Project (نظرة عامة على المشروع)
+
+**Quantum Terminal** is an advanced, fully free, and open-source trading terminal designed to bridge the gap between institutional-grade quantitative analysis and retail day trading. 
+
+Unlike traditional platforms that rely on lagging indicators (like RSI or MACD), Quantum Terminal processes **real-time tick data** from professional brokerages and overlays complex mathematical probability models directly onto your charts. It acts as a localized display hub, separating heavy mathematical computations from real-time charting to ensure zero-latency trade monitoring.
+
+---
+
+## ✨ Core Features (المميزات الأساسية)
+
+### 1. 🌐 Multi-Provider Live Feeds (ربط مباشر مع الأسواق)
+Quantum Terminal is built to handle ultra-fast data streams from the most reliable data providers in the industry:
+- **MetaTrader 5 (MT5):** Direct integration via Python API for Forex, Crypto, and CFD markets.
+- **Rithmic:** High-frequency, sub-millisecond tick processing for CME Futures (NQ, ES).
+- **Tradovate:** Seamless integration for retail futures traders.
+*(The platform features a dedicated `focus_tick_loop` ensuring the chart you are actively watching refreshes at maximum FPS without bottlenecking the system).*
+
+### 2. 📐 Advanced Quantitative Models (النماذج الكمية المتقدمة)
+Move beyond simple support and resistance. The platform integrates institutional volatility models to forecast price action probabilities:
+- **Volatility Cones:** Visualizing expected price ranges using Geometric Brownian Motion (**GBM**), Merton Jump Diffusion (**MJD**), and **Bates** models.
+- **Scalp Bands & Micro Volatility:** Real-time bands designed specifically for high-frequency scalpers.
+- **Probability Fields:** Heatmaps mapping the statistical likelihood of price reverting or trending.
+
+### 3. 🌊 Options Flow & Market Maker Positioning (تحليل سيولة الخيارات)
+Understanding where Market Makers are hedging is critical. Quantum Terminal visualizes:
+- **Gamma Exposure (GEX):** Identifies sticky price levels and volatility triggers.
+- **Max Pain:** Highlights the strike price with the most open options contracts.
+- **Option Walls:** Visualizes heavy Call/Put resistance and support barriers.
+
+### 4. 🌍 Macro Regime & Sector Rotation (تحليل الماكرو والسيولة)
+Using data from `yfinance`, the terminal analyzes global market regimes:
+- Tracks ETF Sector Rotation to see where smart money is flowing.
+- Identifies the current market regime (Risk-On, Risk-Off, Inflationary, etc.) to align your trades with the broader macroeconomic trend.
+
+---
+
+## 🏗️ Architecture & Repository Structure (هيكلية المشروع)
+
+The project follows a decoupled architecture, ensuring the UI remains buttery smooth while the backend handles heavy lifting.
+
+```text
+QuantumTerminal/
+├── backend/                # The Core Engine (Python/FastAPI)
+│   ├── providers/          # MT5, Rithmic, and Tradovate API wrappers
+│   ├── data_server.py      # Main FastAPI server and WebSocket broadcaster
+│   └── launcher.py         # Entry point and config manager
+│
+├── electron_shell/         # Desktop Application Wrapper
+│   ├── main.js             # Electron main process
+│   └── preload.js          # Security and IPC bridging
+│
+└── frontend_build/         # Compiled React UI
+    ├── index.html          # Main entry point
+    └── static/             # Minified JS/CSS for the charting interface
+```
+
+> **Note on Open Source:** The Python backend and Electron shell are fully open source. The React frontend is provided as a compiled production build (`frontend_build/`), meaning it is fully functional and ready to run, though the raw `.jsx` source files are not included in this repository.
+
+---
+
+## 🚀 Getting Started (طريقة التشغيل)
+
+### Prerequisites
+- **Python 3.10** or higher.
+- **MetaTrader 5** (If you intend to use the MT5 data feed).
+- `Node.js` (Optional, if you wish to re-package the Electron shell).
+
+### Installation
+1. Clone the repository:
+   ```bash
+   git clone https://github.com/YourUsername/QuantumTerminal.git
+   cd QuantumTerminal/backend
+   ```
+2. Install Python dependencies:
+   ```bash
+   pip install fastapi uvicorn websockets watchfiles yfinance MetaTrader5
+   ```
+3. Run the Backend Server:
+   ```bash
+   python launcher.py
+   ```
+4. The terminal will automatically serve the UI on `http://127.0.0.1:8502` and launch your default browser. 
+
+*(To run as a standalone desktop app, you can package the `electron_shell` directory using Electron Forge or Electron Builder).*
+
+---
+
+## 🛠️ Contributing (المساهمة)
+This is a community-driven project! If you are a Python developer, Quant, or React engineer, your pull requests are welcome. 
+- **Areas needing help:** Enhancing thread safety for MT5 in `data_server.py`, adding new broker providers (like Interactive Brokers), and rebuilding an open-source React UI.
+
+## ⚖️ Disclaimer
+*Quantum Terminal is for educational and analytical purposes only. Trading financial markets carries a high level of risk and may not be suitable for all investors. The mathematical models provided by this software are probabilistic, not predictive.*
+
+## 📄 License
+This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for more details.

backend/account_manager.py

@@ -0,0 +1,805 @@
+"""
+================================================================================
+ACCOUNT MANAGER — Real Account State & Persistent P&L Tracking
+================================================================================
+Quantum Terminal | Layer 3 — Account & Risk Management
+
+Bridges the signal engine to real account data. Reads MT5 account balance/equity,
+tracks daily and weekly P&L with persistence across restarts, and provides
+configurable trading rules for prop firm compliance.
+
+Architecture:
+    MT5 Account ──► AccountManager ──► PortfolioState (signal engine)
+                         │
+                         ▼
+                  account_state.json (persistent)
+                         │
+                         ▼
+                  RiskGovernor (pre-trade checks)
+
+Settings are stored in account_settings.json (user-editable from terminal UI).
+Account state (daily P&L, peak equity, etc.) is stored in account_state.json.
+
+Usage:
+    from account_manager import AccountManager
+    mgr = AccountManager()
+    mgr.sync_from_mt5()
+    portfolio = mgr.get_portfolio_state()  # feeds into signal engine
+    mgr.record_trade_result(pnl_usd=150.0)
+
+Dependencies: MetaTrader5 (optional), json, pathlib
+================================================================================
+"""
+
+import json
+import logging
+import os
+from dataclasses import dataclass, field, asdict
+from typing import Dict, List, Optional
+from datetime import datetime, date, timedelta
+from pathlib import Path
+
+# version: v4 (broker-TZ-aware period-pnl bucketing)
+# v4 — get_period_pnl now corrects for the broker TZ before bucketing
+#      deals into today/yesterday/this_week/etc. The MetaTrader5 library
+#      returns deal.time as broker server time treated as a Unix epoch
+#      (i.e. broker_local seconds, not real UTC). Our boundaries
+#      (datetime.now() + datetime(year, month, day)) are in real UTC
+#      via .timestamp(). Without correction, the comparison was off by
+#      the broker's TZ offset — for an operator + broker both at GMT+3,
+#      ~3h of deals spilled across day boundaries, e.g. yesterday-late
+#      trades counted as today and vice versa. v4 detects the broker
+#      offset from a recent tick (round to whole hours) and converts
+#      d.time → real-UTC before comparison.
+log = logging.getLogger("mk.account_manager")
+
+PROJECT_ROOT = Path(__file__).resolve().parent
+
+# v3: AppData dir name from env var so v1 / v2 builds get separate dirs.
+APP_DIR = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal")
+
+
+def _get_user_data_dir() -> Path:
+    """Per-user writable dir for account_settings.json + account_state.json.
+    Fixes Errno 13 Permission denied under Program Files on installed consumer.
+    v3: AppData dir name comes from MK_APP_DIR_NAME (default "QuantumTerminal")."""
+    import platform as _platform
+    if _platform.system() == "Windows":
+        appdata = os.environ.get("APPDATA", "")
+        base = Path(appdata) / APP_DIR if appdata else \
+               Path.home() / "AppData" / "Roaming" / APP_DIR
+    else:
+        base = Path.home() / f".{APP_DIR.lower()}"
+    base.mkdir(parents=True, exist_ok=True)
+    return base
+
+
+def _resolve_default_settings_path() -> Path:
+    """v2: Prefer the AppData path. If a legacy file exists next to the module
+    (dev-mode / old install), migrate it on first use."""
+    user_dir = _get_user_data_dir()
+    user_path = user_dir / "account_settings.json"
+    legacy = PROJECT_ROOT / "account_settings.json"
+    if legacy.exists() and not user_path.exists():
+        try:
+            user_path.write_bytes(legacy.read_bytes())
+            log.info(f"Migrated {legacy} → {user_path}")
+        except Exception as e:
+            log.warning(f"Settings migration failed: {e}")
+    return user_path
+
+
+def _resolve_default_state_path() -> Path:
+    user_dir = _get_user_data_dir()
+    user_path = user_dir / "account_state.json"
+    legacy = PROJECT_ROOT / "account_state.json"
+    if legacy.exists() and not user_path.exists():
+        try:
+            user_path.write_bytes(legacy.read_bytes())
+            log.info(f"Migrated {legacy} → {user_path}")
+        except Exception as e:
+            log.warning(f"State migration failed: {e}")
+    return user_path
+
+
+# ============================================================
+# 1. TRADING SETTINGS (user-configurable)
+# ============================================================
+
+@dataclass
+class TradingSettings:
+    """
+    User-configurable trading rules. Saved to account_settings.json.
+    Editable from the terminal settings panel.
+    """
+    # -- Account --
+    account_size: float = 100_000.0        # Initial account balance (or prop firm start)
+    currency: str = "USD"
+
+    # -- Prop Firm Rules --
+    max_daily_loss_pct: float = 2.0        # Max daily loss as % of initial balance
+    max_daily_profit_pct: float = 0.0      # Max daily profit lock (0 = disabled)
+    max_total_drawdown_pct: float = 10.0   # Max drawdown from initial balance
+    max_trailing_drawdown_pct: float = 0.0 # Max drawdown from peak equity (0 = disabled)
+
+    # -- Position Limits --
+    max_open_positions: int = 4
+    max_correlated_positions: int = 2      # Max same-direction in correlated group
+    risk_per_trade_pct: float = 0.5        # Fixed fractional risk per trade
+
+    # -- Kelly --
+    use_kelly: bool = True
+    kelly_fraction: float = 0.25           # Quarter-Kelly
+
+    # -- Calibration Gating --
+    min_calibration_grade: str = "C"       # Filter signals below this grade
+    require_calibration: bool = False      # If True, only trade calibrated setups
+
+    # -- Session Rules --
+    trading_enabled: bool = True           # Master switch
+    auto_sync_mt5: bool = True             # Auto-sync equity from MT5 on startup
+
+    # -- Auto Trading (Phase 3G) --
+    auto_trading_enabled: bool = False     # Auto-execute signals (separate from live_trading)
+    auto_min_confidence: float = 0.60      # Min signal confidence to auto-trade
+    auto_min_grade: str = "C"              # Min calibration grade to auto-trade
+    auto_max_daily_trades: int = 5         # Max auto-trades per day
+    auto_allowed_types: list = field(default_factory=lambda: [
+        "cone_boundary_fade", "drift_momentum", "institutional_anchor",
+        "cone_convergence", "regime_transition", "cone_breakout",
+    ])
+    auto_allowed_tickers: list = field(default_factory=list)  # Empty = all universe
+    auto_scan_interval: float = 30.0       # Seconds between scans
+    auto_log_only: bool = True             # True = dry run (log but don't execute)
+
+    # -- Scheduler (Phase 3G) --
+    scheduler_enabled: bool = True
+    scheduler_weekly_enabled: bool = True
+    scheduler_weekly_day: int = 6              # 0=Mon, 6=Sun
+    scheduler_weekly_time: str = "21:30"       # HH:MM UTC
+    scheduler_daily_enabled: bool = True
+    scheduler_daily_time: str = "21:30"        # HH:MM UTC
+    scheduler_daily_skip_weekends: bool = True
+    scheduler_calc_before_weekly: bool = True   # Run cones before weekly orch
+    # Calibration
+    calibration_workers: int = 1               # CPU threads for ATR/management sweep
+    scheduler_calc_modules: str = "anchors,cones"
+
+    def to_dict(self) -> dict:
+        return asdict(self)
+
+    @classmethod
+    def from_dict(cls, d: dict) -> 'TradingSettings':
+        # Only take keys that exist in the dataclass
+        valid_keys = {f.name for f in cls.__dataclass_fields__.values()}
+        filtered = {k: v for k, v in d.items() if k in valid_keys}
+        return cls(**filtered)
+
+
+# ============================================================
+# 2. ACCOUNT STATE (persistent)
+# ============================================================
+
+@dataclass
+class AccountState:
+    """
+    Persistent account state — survives server restarts.
+    Saved to account_state.json.
+    """
+    # -- Equity tracking --
+    initial_balance: float = 100_000.0     # Set once at start (prop firm funded amount)
+    current_equity: float = 100_000.0      # Last known equity
+    peak_equity: float = 100_000.0         # Highest equity ever (for trailing DD)
+
+    # -- Daily tracking --
+    daily_start_equity: float = 100_000.0  # Equity at start of current day
+    daily_pnl: float = 0.0                 # P&L since daily reset
+    daily_trades: int = 0                  # Trade count today
+    current_date: str = ""                 # ISO date of current tracking day
+
+    # -- Weekly tracking --
+    weekly_start_equity: float = 100_000.0
+    weekly_pnl: float = 0.0
+    weekly_trades: int = 0
+    current_week: str = ""                 # ISO week string (e.g., "2026-W12")
+
+    # -- Circuit breaker states --
+    daily_loss_halt: bool = False           # True = daily loss limit hit
+    daily_profit_lock: bool = False         # True = daily profit target hit
+    total_drawdown_halt: bool = False       # True = max drawdown breached
+    trailing_drawdown_halt: bool = False    # True = trailing DD breached
+
+    # -- Trade log (lightweight) --
+    recent_trades: List[Dict] = field(default_factory=list)  # Last 50 trades
+
+    # -- Timestamps --
+    last_mt5_sync: str = ""
+    last_updated: str = ""
+
+    def to_dict(self) -> dict:
+        return asdict(self)
+
+    @classmethod
+    def from_dict(cls, d: dict) -> 'AccountState':
+        valid_keys = {f.name for f in cls.__dataclass_fields__.values()}
+        filtered = {k: v for k, v in d.items() if k in valid_keys}
+        # Handle list fields that may come as None
+        if 'recent_trades' not in filtered or filtered['recent_trades'] is None:
+            filtered['recent_trades'] = []
+        return cls(**filtered)
+
+
+# ============================================================
+# 3. ACCOUNT MANAGER
+# ============================================================
+
+class AccountManager:
+    """
+    Manages account state, MT5 sync, and trading settings.
+    """
+
+    def __init__(self, settings_path: Optional[Path] = None,
+                 state_path: Optional[Path] = None):
+        # v2: default to %APPDATA%\QuantumTerminal\ (writable) instead of the module
+        # directory, which lives under Program Files and is read-only for
+        # non-admin users. Legacy files are migrated on first use.
+        self._settings_path = settings_path or _resolve_default_settings_path()
+        self._state_path = state_path or _resolve_default_state_path()
+
+        self.settings = self._load_settings()
+        self.state = self._load_state()
+
+        # Check for day/week rollover
+        self._check_daily_reset()
+        self._check_weekly_reset()
+
+    # ────────────────────────────────────────────────────────
+    # PERSISTENCE
+    # ────────────────────────────────────────────────────────
+
+    def _load_settings(self) -> TradingSettings:
+        """Load or create settings file."""
+        if self._settings_path.exists():
+            try:
+                with open(self._settings_path) as f:
+                    data = json.load(f)
+                log.info(f"Loaded trading settings from {self._settings_path}")
+                return TradingSettings.from_dict(data)
+            except Exception as e:
+                log.warning(f"Failed to load settings: {e} — using defaults")
+
+        settings = TradingSettings()
+        self._save_settings(settings)
+        return settings
+
+    def _save_settings(self, settings: TradingSettings = None):
+        """Save settings to JSON."""
+        if settings is None:
+            settings = self.settings
+        try:
+            with open(self._settings_path, "w") as f:
+                json.dump(settings.to_dict(), f, indent=2)
+        except Exception as e:
+            log.warning(f"Failed to save settings: {e}")
+
+    def _load_state(self) -> AccountState:
+        """Load or create state file."""
+        if self._state_path.exists():
+            try:
+                with open(self._state_path) as f:
+                    data = json.load(f)
+                log.info(f"Loaded account state from {self._state_path}")
+                return AccountState.from_dict(data)
+            except Exception as e:
+                log.warning(f"Failed to load state: {e} — using defaults")
+
+        state = AccountState(
+            initial_balance=self.settings.account_size,
+            current_equity=self.settings.account_size,
+            peak_equity=self.settings.account_size,
+            daily_start_equity=self.settings.account_size,
+            weekly_start_equity=self.settings.account_size,
+        )
+        self._save_state(state)
+        return state
+
+    def _save_state(self, state: AccountState = None):
+        """Save state to JSON."""
+        if state is None:
+            state = self.state
+        state.last_updated = datetime.now().isoformat()
+        try:
+            with open(self._state_path, "w") as f:
+                json.dump(state.to_dict(), f, indent=2, default=str)
+        except Exception as e:
+            log.warning(f"Failed to save state: {e}")
+
+    def update_settings(self, new_settings: dict):
+        """Update settings from terminal UI (partial update)."""
+        old_account_size = self.settings.account_size
+        current = self.settings.to_dict()
+        current.update(new_settings)
+        self.settings = TradingSettings.from_dict(current)
+        self._save_settings()
+        log.info(f"Trading settings updated: {list(new_settings.keys())}")
+
+        # Auto-reset baseline when account_size changes
+        if "account_size" in new_settings and new_settings["account_size"] != old_account_size:
+            self.reset_baseline(self.settings.account_size)
+
+    def reset_baseline(self, new_balance: float = None):
+        """
+        Reset account state baseline. Call when:
+        - Setting up a new funded account
+        - Account size changes in settings
+        - Starting fresh after a reset
+
+        Sets initial_balance, peak_equity, daily/weekly start to the new value.
+        Clears all circuit breaker halts. Preserves current_equity if MT5-synced.
+        """
+        bal = new_balance or self.settings.account_size
+        equity = self.state.current_equity if self.state.current_equity > 0 else bal
+
+        self.state.initial_balance = bal
+        self.state.peak_equity = max(bal, equity)
+        self.state.daily_start_equity = equity
+        self.state.weekly_start_equity = equity
+        self.state.daily_pnl = 0.0
+        self.state.weekly_pnl = 0.0
+        self.state.daily_trades = 0
+        self.state.weekly_trades = 0
+
+        # Clear all circuit breaker halts
+        self.state.daily_loss_halt = False
+        self.state.daily_profit_lock = False
+        self.state.total_drawdown_halt = False
+        self.state.trailing_drawdown_halt = False
+
+        self._save_state()
+        log.info(f"Account baseline reset: initial_balance=${bal:,.2f}, "
+                 f"equity=${equity:,.2f}, all circuit breakers cleared")
+
+    # ────────────────────────────────────────────────────────
+    # MT5 SYNC
+    # ────────────────────────────────────────────────────────
+
+    def sync_from_mt5(self) -> bool:
+        """
+        Read real account equity from MT5.
+        Returns True if sync successful.
+        """
+        try:
+            import MetaTrader5 as mt5
+
+            if not mt5.terminal_info():
+                log.warning("MT5 not connected — cannot sync account")
+                return False
+
+            account = mt5.account_info()
+            if account is None:
+                log.warning("MT5 account_info() returned None")
+                return False
+
+            equity = account.equity
+            balance = account.balance
+
+            self.state.current_equity = equity
+            self.state.peak_equity = max(self.state.peak_equity, equity)
+            self.state.last_mt5_sync = datetime.now().isoformat()
+
+            # Update daily P&L
+            self.state.daily_pnl = equity - self.state.daily_start_equity
+            self.state.weekly_pnl = equity - self.state.weekly_start_equity
+
+            # Check circuit breakers
+            self._check_circuit_breakers()
+
+            self._save_state()
+            log.info(f"MT5 sync: equity=${equity:,.2f}, balance=${balance:,.2f}, "
+                     f"daily P&L=${self.state.daily_pnl:+,.2f}")
+            return True
+
+        except ImportError:
+            log.info("MetaTrader5 package not available — manual equity mode")
+            return False
+        except Exception as e:
+            log.warning(f"MT5 sync failed: {e}")
+            return False
+
+    def sync_positions_from_mt5(self) -> List[Dict]:
+        """Read open positions from MT5."""
+        try:
+            import MetaTrader5 as mt5
+
+            if not mt5.terminal_info():
+                return []
+
+            positions = mt5.positions_get()
+            if positions is None:
+                return []
+
+            result = []
+            for pos in positions:
+                result.append({
+                    'ticket': pos.ticket,
+                    'ticker': pos.symbol,
+                    'direction': 'long' if pos.type == 0 else 'short',
+                    'size': pos.volume,
+                    'entry': pos.price_open,
+                    'current_price': pos.price_current,
+                    'current_pnl': pos.profit,
+                    'sl': pos.sl,
+                    'tp': pos.tp,
+                    'magic': pos.magic,
+                    'comment': pos.comment,
+                })
+
+            return result
+
+        except (ImportError, Exception):
+            return []
+
+    # ────────────────────────────────────────────────────────
+    # DAILY/WEEKLY RESETS
+    # ────────────────────────────────────────────────────────
+
+    def _check_daily_reset(self):
+        """Reset daily tracking if it's a new day."""
+        today = date.today().isoformat()
+        if self.state.current_date != today:
+            if self.state.current_date:
+                log.info(f"Daily reset: {self.state.current_date} → {today} "
+                         f"(yesterday P&L: ${self.state.daily_pnl:+,.2f})")
+            self.state.daily_start_equity = self.state.current_equity
+            self.state.daily_pnl = 0.0
+            self.state.daily_trades = 0
+            self.state.daily_loss_halt = False
+            self.state.daily_profit_lock = False
+            self.state.current_date = today
+            self._save_state()
+
+    def _check_weekly_reset(self):
+        """Reset weekly tracking if it's a new week."""
+        current_week = date.today().isocalendar()
+        week_str = f"{current_week[0]}-W{current_week[1]:02d}"
+        if self.state.current_week != week_str:
+            if self.state.current_week:
+                log.info(f"Weekly reset: {self.state.current_week} → {week_str} "
+                         f"(last week P&L: ${self.state.weekly_pnl:+,.2f})")
+            self.state.weekly_start_equity = self.state.current_equity
+            self.state.weekly_pnl = 0.0
+            self.state.weekly_trades = 0
+            self.state.current_week = week_str
+            self._save_state()
+
+    # ────────────────────────────────────────────────────────
+    # CIRCUIT BREAKERS
+    # ────────────────────────────────────────────────────────
+
+    def _check_circuit_breakers(self):
+        """Check all prop firm circuit breakers."""
+        s = self.settings
+        st = self.state
+
+        # Daily loss limit
+        if s.max_daily_loss_pct > 0:
+            daily_loss_limit = st.initial_balance * (s.max_daily_loss_pct / 100)
+            if st.daily_pnl <= -daily_loss_limit:
+                if not st.daily_loss_halt:
+                    log.warning(f"🛑 DAILY LOSS LIMIT HIT: ${st.daily_pnl:+,.2f} "
+                                f"exceeds -{s.max_daily_loss_pct}% of ${st.initial_balance:,.0f}")
+                st.daily_loss_halt = True
+
+        # Daily profit lock
+        if s.max_daily_profit_pct > 0:
+            daily_profit_limit = st.initial_balance * (s.max_daily_profit_pct / 100)
+            if st.daily_pnl >= daily_profit_limit:
+                if not st.daily_profit_lock:
+                    log.info(f"🔒 DAILY PROFIT LOCK: ${st.daily_pnl:+,.2f} "
+                             f"exceeds +{s.max_daily_profit_pct}% target")
+                st.daily_profit_lock = True
+
+        # Total drawdown from initial
+        if s.max_total_drawdown_pct > 0:
+            total_dd = (st.initial_balance - st.current_equity) / st.initial_balance * 100
+            if total_dd >= s.max_total_drawdown_pct:
+                if not st.total_drawdown_halt:
+                    log.warning(f"🛑 MAX DRAWDOWN BREACHED: {total_dd:.1f}% from initial "
+                                f"(limit: {s.max_total_drawdown_pct}%)")
+                st.total_drawdown_halt = True
+
+        # Trailing drawdown from peak
+        if s.max_trailing_drawdown_pct > 0 and st.peak_equity > 0:
+            trailing_dd = (st.peak_equity - st.current_equity) / st.peak_equity * 100
+            if trailing_dd >= s.max_trailing_drawdown_pct:
+                if not st.trailing_drawdown_halt:
+                    log.warning(f"🛑 TRAILING DRAWDOWN BREACHED: {trailing_dd:.1f}% from peak "
+                                f"${st.peak_equity:,.2f} (limit: {s.max_trailing_drawdown_pct}%)")
+                st.trailing_drawdown_halt = True
+
+    @property
+    def is_trading_allowed(self) -> bool:
+        """Check if trading is allowed based on all circuit breakers."""
+        if not self.settings.trading_enabled:
+            return False
+        st = self.state
+        return not (st.daily_loss_halt or st.daily_profit_lock or
+                    st.total_drawdown_halt or st.trailing_drawdown_halt)
+
+    @property
+    def halt_reason(self) -> str:
+        """Get the reason trading is halted, or empty string."""
+        if not self.settings.trading_enabled:
+            return "Trading disabled in settings"
+        st = self.state
+        reasons = []
+        if st.daily_loss_halt:
+            reasons.append(f"Daily loss limit ({self.settings.max_daily_loss_pct}%)")
+        if st.daily_profit_lock:
+            reasons.append(f"Daily profit locked ({self.settings.max_daily_profit_pct}%)")
+        if st.total_drawdown_halt:
+            reasons.append(f"Max drawdown ({self.settings.max_total_drawdown_pct}%)")
+        if st.trailing_drawdown_halt:
+            reasons.append(f"Trailing drawdown ({self.settings.max_trailing_drawdown_pct}%)")
+        return " | ".join(reasons)
+
+    # ────────────────────────────────────────────────────────
+    # TRADE RECORDING
+    # ────────────────────────────────────────────────────────
+
+    def record_trade_result(self, pnl_usd: float, ticker: str = "",
+                            direction: str = "", signal_type: str = "",
+                            lots: float = 0.0, magic: int = 0):
+        """
+        Record a trade result and update P&L tracking.
+        Called when a trade closes (from lifecycle manager or MT5 bridge).
+        """
+        self.state.daily_pnl += pnl_usd
+        self.state.weekly_pnl += pnl_usd
+        self.state.daily_trades += 1
+        self.state.weekly_trades += 1
+        self.state.current_equity += pnl_usd
+        self.state.peak_equity = max(self.state.peak_equity, self.state.current_equity)
+
+        # Append to recent trades (keep last 50)
+        self.state.recent_trades.append({
+            'timestamp': datetime.now().isoformat(),
+            'ticker': ticker,
+            'direction': direction,
+            'signal_type': signal_type,
+            'lots': lots,
+            'pnl_usd': round(pnl_usd, 2),
+            'magic': magic,
+            'equity_after': round(self.state.current_equity, 2),
+        })
+        if len(self.state.recent_trades) > 50:
+            self.state.recent_trades = self.state.recent_trades[-50:]
+
+        self._check_circuit_breakers()
+        self._save_state()
+
+        log.info(f"Trade recorded: {ticker} {direction} {signal_type} → "
+                 f"PnL=${pnl_usd:+,.2f} | Daily=${self.state.daily_pnl:+,.2f} | "
+                 f"Equity=${self.state.current_equity:,.2f}")
+
+    # ────────────────────────────────────────────────────────
+    # BRIDGE TO SIGNAL ENGINE
+    # ────────────────────────────────────────────────────────
+
+    def get_portfolio_state(self):
+        """
+        Build a PortfolioState for the signal engine from current account data.
+        """
+        # Import here to avoid circular dependency
+        from signal_engine_v2 import PortfolioState
+
+        positions = self.sync_positions_from_mt5()
+
+        return PortfolioState(
+            equity=self.state.current_equity,
+            open_positions=[
+                {
+                    'ticker': p['ticker'],
+                    'direction': p['direction'],
+                    'size': p['size'],
+                    'entry': p['entry'],
+                    'current_pnl': p['current_pnl'],
+                    'risk_usd': abs(p['entry'] - p.get('sl', p['entry'])) * p['size'],
+                }
+                for p in positions
+            ],
+            daily_pnl=self.state.daily_pnl,
+            weekly_pnl=self.state.weekly_pnl,
+            peak_equity=self.state.peak_equity,
+        )
+
+    # ────────────────────────────────────────────────────────
+    # REST API HELPERS (for data_server.py endpoints)
+    # ────────────────────────────────────────────────────────
+
+    def get_status_dict(self) -> dict:
+        """Get full account status for the terminal dashboard."""
+        s = self.settings
+        st = self.state
+
+        daily_loss_limit = st.initial_balance * (s.max_daily_loss_pct / 100) if s.max_daily_loss_pct > 0 else 0
+        daily_profit_limit = st.initial_balance * (s.max_daily_profit_pct / 100) if s.max_daily_profit_pct > 0 else 0
+        total_dd = (st.initial_balance - st.current_equity) / st.initial_balance * 100 if st.initial_balance > 0 else 0
+        trailing_dd = (st.peak_equity - st.current_equity) / st.peak_equity * 100 if st.peak_equity > 0 else 0
+
+        return {
+            "trading_allowed": self.is_trading_allowed,
+            "halt_reason": self.halt_reason,
+            "equity": round(st.current_equity, 2),
+            "initial_balance": round(st.initial_balance, 2),
+            "peak_equity": round(st.peak_equity, 2),
+            "daily_pnl": round(st.daily_pnl, 2),
+            "daily_pnl_pct": round(st.daily_pnl / st.initial_balance * 100, 2) if st.initial_balance > 0 else 0,
+            "daily_loss_limit": round(daily_loss_limit, 2),
+            "daily_profit_limit": round(daily_profit_limit, 2),
+            "daily_loss_halt": st.daily_loss_halt,
+            "daily_profit_lock": st.daily_profit_lock,
+            "weekly_pnl": round(st.weekly_pnl, 2),
+            "total_drawdown_pct": round(total_dd, 2),
+            "trailing_drawdown_pct": round(trailing_dd, 2),
+            "total_drawdown_halt": st.total_drawdown_halt,
+            "trailing_drawdown_halt": st.trailing_drawdown_halt,
+            "daily_trades": st.daily_trades,
+            "weekly_trades": st.weekly_trades,
+            "open_positions": len(self.sync_positions_from_mt5()),
+            "max_positions": s.max_open_positions,
+            "last_mt5_sync": st.last_mt5_sync,
+            "last_updated": st.last_updated,
+        }
+
+    def get_settings_dict(self) -> dict:
+        """Get current settings for the terminal UI."""
+        return self.settings.to_dict()
+
+    # ────────────────────────────────────────────────────────
+    # MT5 DEAL HISTORY — REAL PERIOD P&L
+    # ────────────────────────────────────────────────────────
+
+    def get_period_pnl(self) -> dict:
+        """
+        Query MT5 closed deal history for accurate period P&L.
+
+        Returns real closed-trade P&L (profit + swap + commission) for:
+          - today, yesterday
+          - this week, last week
+          - this month, last month
+
+        This is the source of truth — NOT the equity-delta tracking
+        which drifts when weekly_start_equity is stale.
+        """
+        try:
+            import MetaTrader5 as mt5
+            import time as _time
+
+            if not mt5.terminal_info():
+                return self._empty_period_pnl("MT5 not connected")
+
+            # v4: detect broker TZ offset (broker_time treated as UTC -
+            # real_UTC). Round to whole hours since broker TZs are integer
+            # offsets. Falls back to 0 (treat as UTC broker) on failure.
+            broker_offset_sec = 0
+            try:
+                for sym in ("EURUSD", "XAUUSD", "USDJPY", "BTCUSD"):
+                    tick = mt5.symbol_info_tick(sym)
+                    if tick and getattr(tick, "time", 0):
+                        raw = int(tick.time) - int(_time.time())
+                        broker_offset_sec = round(raw / 3600) * 3600
+                        # Sanity: clamp to ±14h (no real broker outside that)
+                        if abs(broker_offset_sec) > 14 * 3600:
+                            broker_offset_sec = 0
+                        break
+            except Exception as e:
+                log.debug(f"broker offset detection failed: {e}")
+
+            now = datetime.now()
+            today_start = datetime(now.year, now.month, now.day)
+            yesterday_start = today_start - timedelta(days=1)
+
+            # Week boundaries (Monday 00:00)
+            weekday = now.weekday()  # 0=Mon
+            this_week_start = today_start - timedelta(days=weekday)
+            last_week_start = this_week_start - timedelta(days=7)
+            last_week_end = this_week_start
+
+            # Month boundaries
+            this_month_start = datetime(now.year, now.month, 1)
+            if now.month == 1:
+                last_month_start = datetime(now.year - 1, 12, 1)
+            else:
+                last_month_start = datetime(now.year, now.month - 1, 1)
+            last_month_end = this_month_start
+
+            # v4: fetch a wider window than strictly needed so deals near the
+            # boundaries don't get clipped by MT5's own pseudo-UTC interpretation
+            # of the input range. We filter precisely in Python below.
+            query_from = last_month_start - timedelta(days=2)
+            query_to   = now + timedelta(days=2)
+            all_deals = mt5.history_deals_get(query_from, query_to)
+            if all_deals is None:
+                all_deals = ()
+
+            # Filter to closing deals only (entry: 1=out, 2=inout, 3=out_by)
+            # Deal type 6 = balance operations — skip those
+            close_deals = [
+                d for d in all_deals
+                if d.entry in (1, 2, 3) and d.type not in (6,)
+            ]
+
+            def _sum_pnl(deals, dt_from, dt_to):
+                """Sum profit + swap + commission for deals in [dt_from, dt_to).
+                v4: convert d.time (broker pseudo-UTC) to real UTC before
+                comparing against the boundaries (which are real UTC via
+                .timestamp())."""
+                ts_from = dt_from.timestamp()
+                ts_to = dt_to.timestamp()
+                total = 0.0
+                count = 0
+                for d in deals:
+                    d_real_utc = d.time - broker_offset_sec
+                    if ts_from <= d_real_utc < ts_to:
+                        total += d.profit + d.swap + d.commission
+                        count += 1
+                return round(total, 2), count
+
+            today_pnl, today_trades = _sum_pnl(close_deals, today_start, now + timedelta(hours=1))
+            yesterday_pnl, yesterday_trades = _sum_pnl(close_deals, yesterday_start, today_start)
+            this_week_pnl, this_week_trades = _sum_pnl(close_deals, this_week_start, now + timedelta(hours=1))
+            last_week_pnl, last_week_trades = _sum_pnl(close_deals, last_week_start, last_week_end)
+            this_month_pnl, this_month_trades = _sum_pnl(close_deals, this_month_start, now + timedelta(hours=1))
+            last_month_pnl, last_month_trades = _sum_pnl(close_deals, last_month_start, last_month_end)
+
+            equity = self.state.current_equity or self.state.initial_balance
+
+            return {
+                "today": {"pnl": today_pnl, "trades": today_trades,
+                          "pct": round(today_pnl / equity * 100, 2) if equity > 0 else 0},
+                "yesterday": {"pnl": yesterday_pnl, "trades": yesterday_trades,
+                              "pct": round(yesterday_pnl / equity * 100, 2) if equity > 0 else 0},
+                "this_week": {"pnl": this_week_pnl, "trades": this_week_trades,
+                              "pct": round(this_week_pnl / equity * 100, 2) if equity > 0 else 0},
+                "last_week": {"pnl": last_week_pnl, "trades": last_week_trades,
+                              "pct": round(last_week_pnl / equity * 100, 2) if equity > 0 else 0},
+                "this_month": {"pnl": this_month_pnl, "trades": this_month_trades,
+                               "pct": round(this_month_pnl / equity * 100, 2) if equity > 0 else 0},
+                "last_month": {"pnl": last_month_pnl, "trades": last_month_trades,
+                               "pct": round(last_month_pnl / equity * 100, 2) if equity > 0 else 0},
+                "source": "mt5_deals",
+                "deals_scanned": len(close_deals),
+            }
+
+        except ImportError:
+            return self._empty_period_pnl("MetaTrader5 not available")
+        except Exception as e:
+            log.warning(f"Period P&L fetch failed: {e}")
+            return self._empty_period_pnl(str(e))
+
+    @staticmethod
+    def _empty_period_pnl(reason: str = "") -> dict:
+        """Return empty period P&L structure."""
+        empty = {"pnl": 0, "trades": 0, "pct": 0}
+        return {
+            "today": empty.copy(), "yesterday": empty.copy(),
+            "this_week": empty.copy(), "last_week": empty.copy(),
+            "this_month": empty.copy(), "last_month": empty.copy(),
+            "source": "unavailable", "reason": reason,
+            "deals_scanned": 0,
+        }
+
+
+# ============================================================
+# 4. SINGLETON
+# ============================================================
+
+_manager_instance = None
+
+def get_account_manager() -> AccountManager:
+    """Get the singleton AccountManager instance."""
+    global _manager_instance
+    if _manager_instance is None:
+        _manager_instance = AccountManager()
+    return _manager_instance

backend/cache_watcher.py

@@ -0,0 +1,323 @@
+# version: v3
+"""
+================================================================================
+Quantum Terminal Consumer — Cache Directory Watcher
+================================================================================
+Async task that watches %APPDATA%\\QuantumTerminal\\cache\\ for JSON file changes and:
+  1. Loads the changed file into the sync client's in-memory store.
+  2. Broadcasts a 'data_sync_complete' WS event so the frontend re-fetches.
+
+Purpose
+-------
+During the Data Center transition (and for local debugging) the user may drop
+files directly into the cache dir. Without this watcher the running server
+never re-reads disk after startup, so those drops are invisible until a full
+restart. This watcher closes the gap.
+
+It also gives live updates when sync_all() writes new cache files — the usual
+periodic_sync broadcast happens too, but this provides a second, finer-grained
+trigger (per-file, immediate) and covers the case where sync_all is not the
+writer (manual drop, future DC auto-uploader, etc.).
+
+Mechanism
+---------
+Polls os.stat() mtime on *.json in the cache dir (no new dependency). A file
+must have a *stable* mtime for two consecutive ticks before we act on it —
+this debounces partial-write races.
+
+Poll interval defaults to 1 second (configurable). Zero compute, Rule C1 safe.
+
+Config (consumer_config.ini)
+----------------------------
+    [sync]
+    cache_watcher_enabled = true        ; set false to disable entirely
+    cache_watcher_interval_seconds = 1  ; floor enforced at 0.25s
+
+Integration
+-----------
+Wired from data_server.py lifespan, same pattern as periodic_sync. Takes the
+WS broadcast_event callable. Cancellation honored via asyncio.CancelledError.
+
+v2 — Deletions detected and propagated. When a file vanishes from the cache
+     directory, the corresponding entry in the sync client's in-memory store is
+     purged and the broadcast payload reports it via a "removed_files" list.
+     Frontend treats this like any other data_sync_complete event → refetches
+     and gets a 404/null → clears the overlay.
+
+v3 — Filename parsing now delegates to data_sync_client.resolve_cache_filename
+     so bare globals ("money_flow.json"), prefixed globals with multi-word
+     categories ("GLOBAL_probability_state.json"), and the legacy
+     GLOBAL_<lowercase_ticker>_<category>.json misnaming all route to the
+     correct in-memory slot. Same resolver used by _load_all_from_cache at
+     startup, ensuring watcher and loader stay consistent.
+
+Rule C1 compliance: this module performs ZERO calculations — it only reads
+files already on disk and pushes WS events.
+================================================================================
+"""
+
+import asyncio
+import configparser
+import json
+import logging
+import os
+
+# v2: AppData dir name (env-var driven; default "QuantumTerminal" preserves v1 path).
+APP_DIR = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal")
+import re
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Awaitable, Callable, Optional
+
+log = logging.getLogger("mk.cache_watcher")
+
+DEFAULT_INTERVAL_SECONDS = 1.0
+MIN_INTERVAL_SECONDS = 0.25
+
+# Same filename pattern as data_sync_client._load_all_from_cache().
+# Greedy first group + underscore-delimited category tail.
+_FILE_PATTERN = re.compile(r'^(.+)_([a-z_]+)\.json$', re.IGNORECASE)
+
+
+def _read_config() -> tuple[bool, float]:
+    """Read cache_watcher_enabled + interval from consumer_config.ini.
+    Returns (enabled, interval_seconds). Defaults: enabled=True, interval=1.0."""
+    project_root = Path(__file__).resolve().parent
+
+    candidates = [
+        project_root / "consumer_config.ini",
+        project_root.parent / "consumer_config.ini",
+    ]
+
+    import sys as _sys
+    if getattr(_sys, "frozen", False):
+        candidates.append(Path(_sys.executable).parent / "consumer_config.ini")
+
+    appdata = os.environ.get("APPDATA", "")
+    if appdata:
+        candidates.append(Path(appdata) / APP_DIR / "consumer_config.ini")
+
+    enabled = True
+    interval = DEFAULT_INTERVAL_SECONDS
+
+    for p in candidates:
+        if not p.exists():
+            continue
+        try:
+            cp = configparser.ConfigParser()
+            cp.read(str(p), encoding="utf-8-sig")
+            if cp.has_option("sync", "cache_watcher_enabled"):
+                enabled = cp.getboolean("sync", "cache_watcher_enabled")
+            if cp.has_option("sync", "cache_watcher_interval_seconds"):
+                try:
+                    interval = max(MIN_INTERVAL_SECONDS,
+                                   cp.getfloat("sync", "cache_watcher_interval_seconds"))
+                except (ValueError, TypeError):
+                    pass
+            break
+        except Exception as e:
+            log.warning(f"Could not parse cache_watcher config from {p.name}: {e}")
+            break
+
+    return enabled, interval
+
+
+def _get_cache_dir() -> Optional[Path]:
+    """Resolve %APPDATA%\\QuantumTerminal\\cache via the same helper data_sync_client uses."""
+    try:
+        from data_sync_client import _get_cache_dir as _dsc_cache_dir
+        return _dsc_cache_dir()
+    except Exception:
+        appdata = os.environ.get("APPDATA", "")
+        if not appdata:
+            return None
+        return Path(appdata) / APP_DIR / "cache"
+
+
+def create_cache_watcher_task(
+    broadcast_event: Callable[[dict], Awaitable[None]],
+) -> Optional[Callable]:
+    """
+    Returns an async coroutine factory that watches the cache dir.
+    Returns None if disabled via config, so the caller can skip task creation.
+
+    Usage in data_server.py lifespan:
+        from cache_watcher import create_cache_watcher_task
+        watcher = create_cache_watcher_task(manager.broadcast_event)
+        if watcher:
+            tasks.append(asyncio.create_task(watcher()))
+    """
+    enabled, interval = _read_config()
+    if not enabled:
+        log.info("Cache watcher disabled via config")
+        return None
+
+    cache_dir = _get_cache_dir()
+    if cache_dir is None:
+        log.warning("Cache watcher: cannot resolve cache dir — disabled")
+        return None
+
+    log.info(f"Cache watcher configured: dir={cache_dir}, interval={interval}s")
+
+    async def _task():
+        # Track stable mtimes (the "known good" state we've already processed).
+        stable_mtimes: dict[str, int] = {}
+        # Track the most-recent-seen mtime so we can require two consecutive
+        # equal readings before acting (debounce against partial writes).
+        last_seen: dict[str, int] = {}
+
+        # Seed stable_mtimes with everything currently on disk so we don't
+        # fire a flood of "data_sync_complete" events on startup.
+        try:
+            if cache_dir.exists():
+                for f in cache_dir.glob("*.json"):
+                    if f.name.startswith("_"):
+                        continue
+                    try:
+                        stable_mtimes[f.name] = f.stat().st_mtime_ns
+                    except OSError:
+                        continue
+        except Exception as e:
+            log.warning(f"Cache watcher seed failed: {e}")
+
+        while True:
+            try:
+                if not cache_dir.exists():
+                    await asyncio.sleep(interval)
+                    continue
+
+                changed: list[Path] = []
+                current_names: set[str] = set()
+
+                for f in cache_dir.glob("*.json"):
+                    if f.name.startswith("_"):
+                        continue
+                    current_names.add(f.name)
+                    try:
+                        mt = f.stat().st_mtime_ns
+                    except OSError:
+                        continue
+
+                    prev_seen = last_seen.get(f.name)
+                    prev_stable = stable_mtimes.get(f.name)
+
+                    if mt == prev_stable:
+                        # Already processed at this mtime — nothing to do.
+                        last_seen[f.name] = mt
+                        continue
+
+                    if prev_seen == mt:
+                        # Stable for 2 ticks at a new mtime → accept.
+                        changed.append(f)
+                        stable_mtimes[f.name] = mt
+                    last_seen[f.name] = mt
+
+                # Detect deletions: files that were known-stable last tick but
+                # are no longer in the directory. Drop bookkeeping AND purge
+                # the sync client's in-memory store so /api endpoints stop
+                # serving the stale data.
+                deleted: list[str] = []
+                for gone in list(stable_mtimes.keys()):
+                    if gone not in current_names:
+                        deleted.append(gone)
+                        stable_mtimes.pop(gone, None)
+                        last_seen.pop(gone, None)
+
+                if changed or deleted:
+                    await _handle_changes(changed, deleted, broadcast_event)
+
+            except asyncio.CancelledError:
+                log.info("Cache watcher task cancelled")
+                break
+            except Exception as e:
+                log.error(f"Cache watcher tick error: {e}", exc_info=True)
+
+            try:
+                await asyncio.sleep(interval)
+            except asyncio.CancelledError:
+                break
+
+    return _task
+
+
+async def _handle_changes(
+    changed: list[Path],
+    deleted: list[str],
+    broadcast_event: Callable[[dict], Awaitable[None]],
+) -> None:
+    """Reconcile the sync client's in-memory store with disk, then broadcast."""
+    try:
+        from data_sync_client import get_sync_client
+        client = get_sync_client()
+    except Exception as e:
+        log.warning(f"Cache watcher: sync client unavailable: {e}")
+        return
+
+    # v3: use data_sync_client's shared resolver for consistent naming rules.
+    try:
+        from data_sync_client import resolve_cache_filename
+    except Exception:
+        resolve_cache_filename = None
+
+    def _resolve(fname: str):
+        if not fname.lower().endswith(".json"):
+            return None
+        stem = fname[:-5]
+        if resolve_cache_filename:
+            return resolve_cache_filename(stem)
+        # Fallback — legacy regex if sync client unavailable for any reason.
+        m = _FILE_PATTERN.match(fname)
+        return (m.group(1).upper(), m.group(2).lower()) if m else None
+
+    loaded: list[dict] = []
+    for path in changed:
+        resolved = _resolve(path.name)
+        if not resolved:
+            continue
+        ticker, category = resolved
+
+        try:
+            with path.open("r", encoding="utf-8") as fh:
+                data = json.load(fh)
+        except Exception as e:
+            log.warning(f"Cache watcher: failed to read {path.name}: {e}")
+            continue
+
+        try:
+            client._get_store(category)[ticker] = data
+            client._known_categories.add(category)
+            loaded.append({"file": path.name, "ticker": ticker, "category": category})
+            log.info(f"Cache watcher: reloaded {path.name} → ({ticker}, {category})")
+        except Exception as e:
+            log.warning(f"Cache watcher: store update failed for {path.name}: {e}")
+
+    removed: list[dict] = []
+    for fname in deleted:
+        resolved = _resolve(fname)
+        if not resolved:
+            continue
+        ticker, category = resolved
+        try:
+            store = client._get_store(category)
+            if ticker in store:
+                store.pop(ticker, None)
+                removed.append({"file": fname, "ticker": ticker, "category": category})
+                log.info(f"Cache watcher: purged {fname} (deleted from disk)")
+        except Exception as e:
+            log.warning(f"Cache watcher: store purge failed for {fname}: {e}")
+
+    if not loaded and not removed:
+        return
+
+    try:
+        await broadcast_event({
+            "type": "data_sync_complete",
+            "synced": len(loaded),
+            "deleted": len(removed),
+            "source": "cache_watcher",
+            "files": [x["file"] for x in loaded],
+            "removed_files": [x["file"] for x in removed],
+            "timestamp": datetime.now(timezone.utc).isoformat(),
+        })
+    except Exception as e:
+        log.warning(f"Cache watcher: broadcast failed: {e}")

backend/chart_presets_routes.py

@@ -0,0 +1,82 @@
+# version: v1
+"""Chart-preset REST endpoints.
+
+Wire into data_server.py:
+    from chart_presets_routes import create_chart_presets_router
+    app.include_router(create_chart_presets_router())
+
+Endpoints:
+    GET    /api/chart-presets         — returns dict of all named presets
+    PUT    /api/chart-presets/{name}  — body is the style dict; creates or replaces
+    DELETE /api/chart-presets/{name}  — removes the named preset (idempotent)
+"""
+
+import logging
+from typing import Any
+from urllib.parse import unquote
+
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+import config_manager
+
+
+log = logging.getLogger("mk.chart_presets_routes")
+
+# Pydantic body model — accepts the full chart-style shape but is permissive
+# on field presence so future shape changes don't break the API.
+class ChartStyleIn(BaseModel):
+    chartType: str | None = None
+    background: str | None = None
+    bullishColor: str | None = None
+    bearishColor: str | None = None
+    gridLines: bool | None = None
+    marketCloseSeparator: dict | None = None
+    rthSeparator: dict | None = None
+
+    class Config:
+        extra = "allow"   # forward-compat: ignore unknown keys gracefully
+
+
+_RESERVED_NAMES = {"defaults", "default"}
+_MAX_NAME_LEN = 60
+
+
+def _validate_name(name: str) -> None:
+    if not name or not name.strip():
+        raise HTTPException(400, detail="Preset name cannot be empty")
+    if len(name) > _MAX_NAME_LEN:
+        raise HTTPException(400, detail=f"Preset name exceeds {_MAX_NAME_LEN} characters")
+    if name.strip().lower() in _RESERVED_NAMES:
+        raise HTTPException(400, detail=f"'{name}' is a reserved name")
+
+
+def create_chart_presets_router() -> APIRouter:
+    router = APIRouter(prefix="/api/chart-presets", tags=["chart-presets"])
+
+    @router.get("")
+    async def list_presets() -> dict[str, Any]:
+        return config_manager.get_chart_presets()
+
+    @router.put("/{name}")
+    async def upsert_preset(name: str, body: ChartStyleIn) -> dict[str, Any]:
+        decoded = unquote(name)
+        _validate_name(decoded)
+        try:
+            config_manager.save_chart_preset(decoded, body.model_dump(exclude_none=False))
+        except Exception as e:
+            log.exception("save_chart_preset failed")
+            raise HTTPException(500, detail=f"Save failed: {e}") from e
+        return config_manager.get_chart_presets()
+
+    @router.delete("/{name}")
+    async def delete_preset(name: str) -> dict[str, Any]:
+        decoded = unquote(name)
+        try:
+            config_manager.delete_chart_preset(decoded)
+        except Exception as e:
+            log.exception("delete_chart_preset failed")
+            raise HTTPException(500, detail=f"Delete failed: {e}") from e
+        return config_manager.get_chart_presets()
+
+    return router

backend/chart_templates_routes.py

@@ -0,0 +1,84 @@
+# version: v1
+"""Chart-template REST endpoints.
+
+A "chart template" is a named full-chart-config snapshot the operator can
+save and re-apply to any chart ("XAUUSD swing", "XAUUSD scalp", ...).
+Distinct from /api/chart-presets, which holds chart-STYLE-only fragments.
+
+Wire into data_server.py:
+    from chart_templates_routes import create_chart_templates_router
+    app.include_router(create_chart_templates_router())
+
+Endpoints:
+    GET    /api/chart-templates         — returns dict of all named templates {name: {sourceTicker, config, savedAt}}
+    PUT    /api/chart-templates/{name}  — body is {sourceTicker, config, savedAt}; creates or replaces
+    DELETE /api/chart-templates/{name}  — removes the named template (idempotent)
+"""
+
+import logging
+from typing import Any
+from urllib.parse import unquote
+
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+import config_manager
+
+
+log = logging.getLogger("mk.chart_templates_routes")
+
+
+# Permissive body model: a template carries a sourceTicker, a config dict
+# (the full chart-config bundle), and a savedAt timestamp. `extra = "allow"`
+# keeps the API forward-compatible if the shape grows.
+class ChartTemplateIn(BaseModel):
+    sourceTicker: str | None = None
+    config: dict | None = None
+    savedAt: str | None = None
+
+    class Config:
+        extra = "allow"
+
+
+_RESERVED_NAMES = {"defaults", "default"}
+_MAX_NAME_LEN = 60
+
+
+def _validate_name(name: str) -> None:
+    if not name or not name.strip():
+        raise HTTPException(400, detail="Template name cannot be empty")
+    if len(name) > _MAX_NAME_LEN:
+        raise HTTPException(400, detail=f"Template name exceeds {_MAX_NAME_LEN} characters")
+    if name.strip().lower() in _RESERVED_NAMES:
+        raise HTTPException(400, detail=f"'{name}' is a reserved name")
+
+
+def create_chart_templates_router() -> APIRouter:
+    router = APIRouter(prefix="/api/chart-templates", tags=["chart-templates"])
+
+    @router.get("")
+    async def list_templates() -> dict[str, Any]:
+        return config_manager.get_chart_templates()
+
+    @router.put("/{name}")
+    async def upsert_template(name: str, body: ChartTemplateIn) -> dict[str, Any]:
+        decoded = unquote(name)
+        _validate_name(decoded)
+        try:
+            config_manager.save_chart_template(decoded, body.model_dump(exclude_none=False))
+        except Exception as e:
+            log.exception("save_chart_template failed")
+            raise HTTPException(500, detail=f"Save failed: {e}") from e
+        return config_manager.get_chart_templates()
+
+    @router.delete("/{name}")
+    async def delete_template(name: str) -> dict[str, Any]:
+        decoded = unquote(name)
+        try:
+            config_manager.delete_chart_template(decoded)
+        except Exception as e:
+            log.exception("delete_chart_template failed")
+            raise HTTPException(500, detail=f"Delete failed: {e}") from e
+        return config_manager.get_chart_templates()
+
+    return router

backend/config_routes.py

@@ -0,0 +1,203 @@
+# version: v2
+"""
+config_routes.py — Terminal Configuration API Routes
+=====================================================
+
+v2 — PATCH handler now detects broker_symbol changes in
+     universe.custom_tickers[*].broker_symbol and, for each affected ticker,
+     calls provider.reset_symbol() on every live provider so MT5 picks up
+     the new mapping at the next tick/bar fetch (no restart required).
+     Also triggers a data_sync_complete broadcast so the frontend refetches
+     bars for the affected tickers.
+Serves GET /api/config and all mutation endpoints used by useConfig.js.
+
+Endpoints:
+    GET    /api/config                       → full merged config
+    PATCH  /api/config                       → partial update
+    POST   /api/config/tickers               → add ticker
+    DELETE /api/config/tickers/{ticker}       → remove ticker
+    POST   /api/config/tickers/{ticker}/toggle → hide/show
+    PUT    /api/config/tickers/order          → reorder
+    POST   /api/config/profiles              → save profile
+    POST   /api/config/profiles/{name}/load  → load profile
+    DELETE /api/config/profiles/{name}        → delete profile
+
+This module does NOT introduce any calculation triggers.
+"""
+
+import logging
+from typing import Callable, Optional, Set
+from fastapi import APIRouter, Request, HTTPException
+
+log = logging.getLogger("config_routes")
+
+
+# v2 — helpers for broker_symbol hot-rewiring.
+def _collect_broker_symbol_changes(patch: dict) -> Set[str]:
+    """Walk a PATCH body and return the set of tickers whose broker_symbol
+    is being set (value may be null to clear). Safe for any patch shape."""
+    out = set()
+    try:
+        custom = (patch.get("universe", {}) or {}).get("custom_tickers", {}) or {}
+        for ticker, fields in custom.items():
+            if isinstance(fields, dict) and "broker_symbol" in fields:
+                out.add(str(ticker).upper())
+    except Exception as e:
+        log.warning(f"Failed to scan patch for broker_symbol changes: {e}")
+    return out
+
+
+def _apply_broker_symbol_resets(cfg_manager, tickers: Set[str]) -> None:
+    """For each live provider that supports it, evict and re-resolve each
+    ticker so the new broker_symbol override takes effect at the next fetch."""
+    try:
+        providers = cfg_manager.get_all_providers()
+    except Exception as e:
+        log.warning(f"get_all_providers failed: {e}")
+        return
+    for pid, prov in (providers or {}).items():
+        if not hasattr(prov, "reset_symbol"):
+            continue
+        for t in tickers:
+            try:
+                resolved = prov.reset_symbol(t)
+                log.info(f"[{pid}] reset_symbol({t}) → {resolved}")
+            except Exception as e:
+                log.warning(f"[{pid}] reset_symbol({t}) failed: {e}")
+
+
+def create_config_router(cfg_manager, broadcast_event: Optional[Callable] = None) -> APIRouter:
+    """
+    Create config API router wired to the given ConfigManager instance.
+    broadcast_event: async callable to push WS events on config change.
+    """
+    router = APIRouter(prefix="/api/config", tags=["config"])
+
+    async def _notify():
+        """Push config_update event to all WS clients."""
+        if broadcast_event:
+            try:
+                await broadcast_event({"type": "config_update"})
+            except Exception:
+                pass  # Non-critical
+
+    # ── GET /api/config — full merged config ──
+    @router.get("")
+    async def get_config():
+        return cfg_manager.get_config()
+
+    # ── PATCH /api/config — partial update ──
+    @router.patch("")
+    async def patch_config(request: Request):
+        try:
+            patch = await request.json()
+        except Exception:
+            raise HTTPException(400, "Invalid JSON body")
+
+        # v2: detect broker_symbol changes in custom_tickers so we can re-resolve
+        # the affected tickers on the live MT5 provider without a restart.
+        affected_tickers = _collect_broker_symbol_changes(patch)
+
+        cfg_manager.update(patch)
+
+        if affected_tickers:
+            _apply_broker_symbol_resets(cfg_manager, affected_tickers)
+            try:
+                if broadcast_event:
+                    await broadcast_event({
+                        "type": "data_sync_complete",
+                        "source": "broker_symbol_change",
+                        "tickers": sorted(affected_tickers),
+                    })
+            except Exception:
+                pass  # non-critical
+
+        await _notify()
+        return cfg_manager.get_config()
+
+    # ── POST /api/config/tickers — add ticker ──
+    @router.post("/tickers")
+    async def add_ticker(request: Request):
+        try:
+            body = await request.json()
+        except Exception:
+            raise HTTPException(400, "Invalid JSON body")
+
+        ticker = body.get("ticker", "").strip().upper()
+        if not ticker:
+            raise HTTPException(400, "ticker is required")
+
+        metadata = body.get("metadata")
+        success = cfg_manager.add_ticker(ticker, metadata)
+        if success:
+            await _notify()
+        return {"success": success, "ticker": ticker}
+
+    # ── DELETE /api/config/tickers/{ticker} — remove ticker ──
+    @router.delete("/tickers/{ticker}")
+    async def remove_ticker(ticker: str):
+        canonical = ticker.upper()
+        success = cfg_manager.remove_ticker(canonical)
+        if success:
+            await _notify()
+        return {"success": success, "ticker": canonical}
+
+    # ── POST /api/config/tickers/{ticker}/toggle — hide/show ──
+    @router.post("/tickers/{ticker}/toggle")
+    async def toggle_ticker(ticker: str):
+        canonical = ticker.upper()
+        visible = cfg_manager.toggle_ticker_visibility(canonical)
+        await _notify()
+        return {"ticker": canonical, "visible": visible}
+
+    # ── PUT /api/config/tickers/order — reorder display ──
+    @router.put("/tickers/order")
+    async def reorder_tickers(request: Request):
+        try:
+            body = await request.json()
+        except Exception:
+            raise HTTPException(400, "Invalid JSON body")
+
+        order = body.get("order", [])
+        if not isinstance(order, list):
+            raise HTTPException(400, "order must be a list of ticker strings")
+
+        cfg_manager.reorder_tickers(order)
+        await _notify()
+        return {"success": True, "order": order}
+
+    # ── POST /api/config/profiles — save profile ──
+    @router.post("/profiles")
+    async def save_profile(request: Request):
+        try:
+            body = await request.json()
+        except Exception:
+            raise HTTPException(400, "Invalid JSON body")
+
+        name = body.get("name", "").strip()
+        if not name:
+            raise HTTPException(400, "name is required")
+
+        cfg_manager.save_profile(name)
+        await _notify()
+        return {"success": True, "name": name}
+
+    # ── POST /api/config/profiles/{name}/load — load profile ──
+    @router.post("/profiles/{name}/load")
+    async def load_profile(name: str):
+        success = cfg_manager.load_profile(name)
+        if not success:
+            raise HTTPException(404, f"Profile '{name}' not found")
+        await _notify()
+        return {"success": True, "name": name}
+
+    # ── DELETE /api/config/profiles/{name} — delete profile ──
+    @router.delete("/profiles/{name}")
+    async def delete_profile(name: str):
+        success = cfg_manager.delete_profile(name)
+        if not success:
+            raise HTTPException(404, f"Profile '{name}' not found")
+        await _notify()
+        return {"success": True, "name": name}
+
+    return router

backend/consumer_startup.py

@@ -0,0 +1,251 @@
+# version: v3
+"""
+================================================================================
+Quantum Terminal Consumer — Startup Gate & Route Wiring
+================================================================================
+v3 — consumer_gate() now accepts `skip_sync=False`. data_server's lifespan
+     calls it with `skip_sync=True` so the auth phase runs synchronously
+     (needed for login routing) while the slow sync_all() is kicked off in
+     a background thread after lifespan returns. HTTP routes become
+     available immediately; frontend polls /api/sync/status for the SYNCING
+     pill. Eliminates the "15-minute loading screen" symptom caused by
+     persistent 5xx responses on specific data endpoints.
+
+v2 — Wire /api/version (version_info.py) so the app can report its running
+     version and check for updates via /api/version endpoint.
+Bridges the consumer-specific modules (data_sync_client)
+into the existing data_server.py with MINIMAL changes to the PRO codebase.
+
+Usage in data_server.py lifespan:
+    from consumer_startup import consumer_gate, wire_consumer_routes
+
+    # In lifespan, BEFORE provider connect:
+    gate_result = consumer_gate()
+    if not gate_result["ok"]:
+        # Terminal won't render — gate_result has details
+        pass
+
+    # After app creation:
+    wire_consumer_routes(app)
+
+This module does NOT introduce any calculation triggers.
+================================================================================
+"""
+
+import logging
+from typing import Optional
+
+log = logging.getLogger("mk.consumer_startup")
+
+
+# ============================================================
+# 1. CONSUMER STARTUP GATE (Rule C2)
+# ============================================================
+
+def consumer_gate(skip_sync: bool = False) -> dict:
+    """
+    Run the consumer startup gate (Rule C2).
+
+    Three conditions must be met before terminal renders:
+        1. Auth always passes (Quantum Terminal is free)
+        2. data_sync_client completes download OR loads from cache
+        3. MT5 connection established (handled by existing data_server code)
+
+    This function handles conditions 1 and 2.
+
+    v3: when `skip_sync=True`, only the auth phase runs (fast). Caller is
+        responsible for running sync in a background thread so the FastAPI
+        lifespan isn't blocked by a slow/unhealthy server. `sync_summary`
+        will be returned as {"in_progress": True} in that case — populate it
+        from the thread when sync completes.
+    Returns dict with gate status:
+        {
+            "ok": bool,
+            "auth_result": AuthResult dict,
+            "sync_summary": dict,
+            "needs_login": bool,
+            "needs_update": bool,
+            "offline_mode": bool,
+        }
+    """
+    result = {
+        "ok": False,
+        "auth_result": None,
+        "sync_summary": None,
+        "needs_login": False,
+        "needs_update": False,
+        "offline_mode": False,
+    }
+
+    # ── Gate 1: Auth validation ──
+    # Removed in Quantum Terminal: Always authenticated
+    result["auth_result"] = {"status": "ok", "message": "Quantum Terminal - Free Edition"}
+    result["needs_login"] = False
+
+    # ── Gate 2: Data sync ──
+    # v3: skip_sync=True → lifespan will run sync_all() in a background
+    #   thread instead, so the HTTP server is responsive immediately.
+    if skip_sync:
+        result["sync_summary"] = {"in_progress": True}
+    else:
+        try:
+            from data_sync_client import get_sync_client
+            sync_client = get_sync_client()
+            sync_summary = sync_client.sync_all()
+            result["sync_summary"] = sync_summary
+
+            # Rule C6: format version mismatch blocks terminal
+            if not sync_summary.get("format_version_ok", True):
+                log.error("Startup gate: FORMAT VERSION MISMATCH — update required")
+                result["needs_update"] = True
+                result["ok"] = False
+                return result
+
+            if sync_summary.get("offline_mode", False):
+                result["offline_mode"] = True
+
+        except ImportError:
+            log.warning("data_sync_client not available — skipping data sync (dev mode)")
+            result["sync_summary"] = {"note": "dev mode — no sync"}
+        except Exception as e:
+            log.error(f"Data sync error: {e}")
+            result["sync_summary"] = {"error": str(e)}
+
+    result["ok"] = True
+    log.info("Consumer startup gate PASSED")
+    return result
+
+
+# ============================================================
+# 2. ROUTE WIRING
+# ============================================================
+
+def wire_consumer_routes(app):
+    """
+    Wire consumer-specific FastAPI routes into the existing app.
+    Call this AFTER app = FastAPI() in data_server.py.
+
+    Adds:
+        /api/auth/*       — login, register, logout, status, refresh
+        /api/sync/*       — sync status, cone/options data, staleness, refresh
+        /api/consumer/*   — consumer-specific status endpoints
+    """
+    routes_added = []
+
+    # Auth routes
+    # Removed in Quantum Terminal
+
+    # Data sync routes
+    try:
+        from data_sync_client import create_data_sync_routes
+        app.include_router(create_data_sync_routes())
+        routes_added.append("data_sync")
+        log.info("Consumer data sync routes wired: /api/sync/*")
+    except ImportError:
+        log.warning("data_sync_client not available — sync routes not wired")
+    except Exception as e:
+        log.error(f"Failed to wire data sync routes: {e}")
+
+    # v2: version/update-check route
+    try:
+        from version_info import create_version_router, CURRENT_VERSION
+        app.include_router(create_version_router())
+        routes_added.append("version")
+        log.info(f"Consumer version route wired: /api/version (running v{CURRENT_VERSION})")
+    except ImportError:
+        log.warning("version_info not available — /api/version not wired")
+    except Exception as e:
+        log.error(f"Failed to wire version route: {e}")
+
+    # Consumer status endpoint
+    from fastapi import APIRouter
+    consumer_router = APIRouter(tags=["consumer"])
+
+    @consumer_router.get("/api/consumer/status")
+    async def consumer_status():
+        """Consumer terminal status — used by frontend to decide what to show."""
+        status = {
+            "is_consumer": True,
+            "routes_available": routes_added,
+        }
+
+        # Auth status
+        status["authenticated"] = True
+        status["user_email"] = "free@quantumterminal"
+        status["subscription_status"] = "active"
+
+        # Sync status
+        try:
+            from data_sync_client import get_sync_client
+            sc = get_sync_client()
+            status["synced_tickers"] = sc.synced_tickers
+            status["offline_mode"] = sc.is_offline
+            status["format_version_ok"] = sc.format_version_ok
+            status["sync_in_progress"] = sc.is_syncing   # drives the top-bar pill
+            status["last_synced_at"] = sc.last_synced_at
+
+            staleness = sc.get_overall_staleness()
+            status["analysis_date"] = staleness.display_date
+            status["staleness_level"] = staleness.level.value
+        except Exception:
+            status["synced_tickers"] = []
+            status["offline_mode"] = True
+            status["sync_in_progress"] = False
+
+        return status
+
+    @consumer_router.post("/api/consumer/restart")
+    async def consumer_restart():
+        """
+        Restart the terminal backend process. Spawns a fresh instance and
+        exits the current one after a brief delay so the HTTP response can
+        flush before the process exits.
+
+        Rule C1 compliance: this restarts the display server only — no
+        calculation modules are involved.
+        """
+        import sys, os, subprocess, threading
+        import time as _time
+
+        def _do_restart():
+            _time.sleep(1.0)
+            try:
+                creationflags = 0
+                if os.name == "nt":
+                    creationflags = getattr(subprocess, "DETACHED_PROCESS", 0) | \
+                                    getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
+                subprocess.Popen(
+                    [sys.executable] + sys.argv,
+                    close_fds=True,
+                    creationflags=creationflags,
+                )
+            except Exception as e:
+                log.error(f"Restart spawn failed: {e}")
+            os._exit(0)
+
+        threading.Thread(target=_do_restart, daemon=True).start()
+        log.warning("Consumer restart requested via /api/consumer/restart")
+        return {"status": "restarting", "delay_ms": 1000}
+
+    app.include_router(consumer_router)
+    log.info("Consumer status route wired: /api/consumer/status")
+
+    return routes_added
+
+
+# ============================================================
+# 3. BLOCKED ENDPOINTS (Rule C1)
+# ============================================================
+
+# These endpoint prefixes must NEVER exist in the consumer terminal.
+# consumer_verify.py test script checks for them.
+BLOCKED_ENDPOINT_PREFIXES = [
+    "/api/calculate",
+    "/api/calibrate",
+    "/api/calibration/run",
+    "/api/run-backtest",
+    "/api/wf/run",
+    "/api/flow/calibrate",
+    "/api/anchor/calibrate",
+    "/api/dev/clear-cache",   # calc target clears calculation data
+]

backend/data_manager/__init__.py

@@ -0,0 +1,50 @@
+"""
+Data Management Module (DMM)
+============================
+
+Unified data access layer for the Quantum Terminal quant platform.
+Parquet-cached, gap-filling, source-agnostic data delivery.
+
+Quick start (singleton — shares one cache across all scripts):
+    from data_manager import dm
+    df = dm.get_bars("EURUSD", "D1", n_bars=1500)
+
+Or instantiate with custom settings:
+    from data_manager import DataManager
+    dm = DataManager(library_root="D:/MyData", auto_init_mt5=True)
+    df = dm.get_bars("XAUUSD", "M15", date_from=..., date_to=...)
+"""
+
+from .data_manager import DataManager
+from .data_store import DataStore
+from .data_catalog import DataCatalog
+
+import logging
+
+# Configure DMM logging — INFO by default, callers can override
+logging.getLogger("dmm").setLevel(logging.INFO)
+
+# ── Lazy singleton ───────────────────────────────────────────────────────────
+# Created on first access.  Uses default library_root (../data_library/).
+# The forecaster's MT5 init is expected to happen BEFORE first dm.get_bars().
+
+_dm_instance = None
+
+
+def get_dm(**kwargs) -> DataManager:
+    """Get or create the module-level DataManager singleton."""
+    global _dm_instance
+    if _dm_instance is None:
+        _dm_instance = DataManager(**kwargs)
+    return _dm_instance
+
+
+# Convenience alias — `from data_manager import dm` gives you the singleton
+class _LazyDM:
+    """Proxy that creates the real DataManager on first attribute access."""
+    def __getattr__(self, name):
+        return getattr(get_dm(), name)
+
+dm = _LazyDM()
+
+__all__ = ["DataManager", "DataStore", "DataCatalog", "dm", "get_dm"]

backend/data_manager/cli.py

@@ -0,0 +1,134 @@
+"""
+Data Manager CLI — command-line interface for data library management.
+
+Usage:
+    python -m data_manager.cli --status
+    python -m data_manager.cli --prefetch --symbols EURUSD XAUUSD --tf D1 M15
+    python -m data_manager.cli --history --symbol EURUSD --limit 20
+    python -m data_manager.cli --purge --symbol EURUSD --tf D1
+    python -m data_manager.cli --validate
+"""
+
+import argparse
+import sys
+import logging
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+# Ensure parent dir is on path for relative imports
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from data_manager import DataManager
+
+
+def cmd_status(dm: DataManager):
+    """Print catalog summary."""
+    summary = dm.get_catalog_summary()
+    if not summary:
+        print("\n  [DATA LIBRARY] Empty — no datasets cached yet.\n")
+        return
+    print(f"\n  [DATA LIBRARY] {dm.library_root}")
+    print(f"  {'Symbol':<10} {'TF':<6} {'Type':<12} {'From':<12} {'To':<12} {'Rows':>8} {'Source':<6}")
+    print("  " + "─" * 72)
+    for r in summary:
+        d_from = r.get("date_from", "")[:10] if r.get("date_from") else "—"
+        d_to   = r.get("date_to", "")[:10] if r.get("date_to") else "—"
+        print(f"  {r['symbol']:<10} {r['timeframe']:<6} {r['data_type']:<12} "
+              f"{d_from:<12} {d_to:<12} {r.get('row_count',0):>8} {r.get('source','?'):<6}")
+    print()
+
+
+def cmd_prefetch(dm: DataManager, symbols: list, timeframes: list, days: int):
+    """Batch warm-up cache for given symbols/timeframes."""
+    date_from = datetime.now(timezone.utc) - timedelta(days=days)
+    date_to = datetime.now(timezone.utc)
+    print(f"\n  Prefetching {len(symbols)} symbols × {len(timeframes)} timeframes "
+          f"({days} days back)...\n")
+    dm.prefetch(symbols, timeframes, date_from, date_to)
+    print("\n  Done.\n")
+
+
+def cmd_history(dm: DataManager, symbol: str, limit: int):
+    """Show fetch history."""
+    history = dm.get_fetch_history(symbol, limit)
+    if not history:
+        print(f"\n  No fetch history{' for '+symbol if symbol else ''}.\n")
+        return
+    print(f"\n  {'Time':<22} {'Symbol':<10} {'TF':<6} {'Rows':>8} {'ms':>8} {'Status':<10}")
+    print("  " + "─" * 68)
+    for r in history:
+        ts = r.get("timestamp", "")[:19]
+        print(f"  {ts:<22} {r['symbol']:<10} {r['timeframe']:<6} "
+              f"{r.get('rows_returned',0):>8} {r.get('duration_ms',0):>8} {r.get('status','?'):<10}")
+    print()
+
+
+def cmd_validate(dm: DataManager):
+    """Validate all cached Parquet files."""
+    summary = dm.get_catalog_summary()
+    if not summary:
+        print("\n  Nothing to validate — library is empty.\n")
+        return
+    print(f"\n  Validating {len(summary)} datasets...")
+    all_ok = True
+    for r in summary:
+        result = dm.store.validate(r["symbol"], r["timeframe"], r.get("data_type", "ohlcv"))
+        tag = "✓" if result.valid else "✗"
+        print(f"  {tag} {r['symbol']:<10} {r['timeframe']:<6} rows={result.row_count}", end="")
+        if not result.valid:
+            all_ok = False
+            print(f"  ISSUES: {'; '.join(result.issues)}")
+        else:
+            print()
+    print(f"\n  {'All datasets valid.' if all_ok else 'Some datasets have issues — see above.'}\n")
+
+
+def cmd_purge(dm: DataManager, symbol: str, timeframe: str, data_type: str):
+    """Remove a specific dataset from cache."""
+    dm.purge(symbol, timeframe, data_type)
+    print(f"\n  Purged {symbol} {timeframe} {data_type}\n")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Data Manager CLI")
+    parser.add_argument("--status", action="store_true", help="Show catalog summary")
+    parser.add_argument("--prefetch", action="store_true", help="Batch prefetch data")
+    parser.add_argument("--history", action="store_true", help="Show fetch history")
+    parser.add_argument("--validate", action="store_true", help="Validate all cached files")
+    parser.add_argument("--purge", action="store_true", help="Purge a dataset")
+
+    parser.add_argument("--symbols", nargs="+", default=[])
+    parser.add_argument("--symbol", type=str, default=None)
+    parser.add_argument("--tf", nargs="+", default=["D1"])
+    parser.add_argument("--days", type=int, default=1500, help="Days back for prefetch")
+    parser.add_argument("--limit", type=int, default=50)
+    parser.add_argument("--data-type", type=str, default="ohlcv")
+    parser.add_argument("--mt5-path", type=str, default=None)
+
+    args = parser.parse_args()
+
+    logging.basicConfig(level=logging.INFO, format="%(message)s")
+    dm = DataManager(mt5_terminal_path=args.mt5_path, auto_init_mt5=True)
+
+    if args.status:
+        cmd_status(dm)
+    elif args.prefetch:
+        if not args.symbols:
+            print("  --prefetch requires --symbols")
+            sys.exit(1)
+        cmd_prefetch(dm, args.symbols, args.tf, args.days)
+    elif args.history:
+        cmd_history(dm, args.symbol, args.limit)
+    elif args.validate:
+        cmd_validate(dm)
+    elif args.purge:
+        if not args.symbol or not args.tf:
+            print("  --purge requires --symbol and --tf")
+            sys.exit(1)
+        cmd_purge(dm, args.symbol, args.tf[0], args.data_type)
+    else:
+        parser.print_help()
+
+
+if __name__ == "__main__":
+    main()

backend/data_manager/data_catalog.py

@@ -0,0 +1,253 @@
+"""
+DataCatalog — SQLite-backed metadata catalog for the data library.
+
+Tracks what data exists, date coverage, row counts, gaps, and fetch history.
+All paths stored as relative strings for portability.
+"""
+
+import sqlite3
+import json
+from pathlib import Path
+from datetime import datetime, timezone
+from dataclasses import dataclass, field
+from typing import Optional, List
+
+import logging
+log = logging.getLogger("dmm.catalog")
+
+
+@dataclass
+class CoverageResult:
+    """What the catalog knows about a (symbol, timeframe, data_type) tuple."""
+    symbol: str
+    timeframe: str
+    data_type: str
+    covered: bool = False
+    date_from: Optional[datetime] = None
+    date_to: Optional[datetime] = None
+    row_count: int = 0
+    file_path: Optional[str] = None
+    # Gaps that need filling relative to a request
+    gaps: List[dict] = field(default_factory=list)
+
+    @property
+    def needs_fetch(self) -> bool:
+        return not self.covered or len(self.gaps) > 0
+
+
+class DataCatalog:
+    """
+    SQLite metadata catalog.
+
+    Parameters
+    ----------
+    db_path : Path
+        Absolute path to catalog.db (created if missing).
+    """
+
+    def __init__(self, db_path: Path):
+        self.db_path = Path(db_path)
+        self.db_path.parent.mkdir(parents=True, exist_ok=True)
+        self._conn = sqlite3.connect(str(self.db_path), check_same_thread=False)
+        self._conn.row_factory = sqlite3.Row
+        self._create_tables()
+
+    # ── schema ───────────────────────────────────────────────────────────────
+
+    def _create_tables(self):
+        c = self._conn.cursor()
+        c.execute("""
+            CREATE TABLE IF NOT EXISTS data_registry (
+                id          INTEGER PRIMARY KEY AUTOINCREMENT,
+                symbol      TEXT NOT NULL,
+                timeframe   TEXT NOT NULL,
+                data_type   TEXT NOT NULL DEFAULT 'ohlcv',
+                source      TEXT NOT NULL DEFAULT 'mt5',
+                date_from   TEXT,
+                date_to     TEXT,
+                row_count   INTEGER DEFAULT 0,
+                file_path   TEXT,
+                last_updated TEXT,
+                checksum    TEXT,
+                gaps        TEXT DEFAULT '[]',
+                UNIQUE(symbol, timeframe, data_type)
+            )
+        """)
+        c.execute("""
+            CREATE TABLE IF NOT EXISTS fetch_log (
+                id            INTEGER PRIMARY KEY AUTOINCREMENT,
+                symbol        TEXT NOT NULL,
+                timeframe     TEXT NOT NULL,
+                source        TEXT NOT NULL,
+                fetch_from    TEXT,
+                fetch_to      TEXT,
+                rows_returned INTEGER DEFAULT 0,
+                duration_ms   INTEGER DEFAULT 0,
+                status        TEXT DEFAULT 'success',
+                error_msg     TEXT,
+                timestamp     TEXT
+            )
+        """)
+        self._conn.commit()
+
+    # ── coverage API ─────────────────────────────────────────────────────────
+
+    def check_coverage(
+        self,
+        symbol: str,
+        timeframe: str,
+        date_from: datetime,
+        date_to: datetime,
+        data_type: str = "ohlcv",
+    ) -> CoverageResult:
+        """
+        Check what coverage exists for a request and identify gaps to fill.
+        """
+        result = CoverageResult(
+            symbol=symbol.upper(),
+            timeframe=timeframe.upper(),
+            data_type=data_type,
+        )
+
+        row = self._conn.execute(
+            "SELECT * FROM data_registry WHERE symbol=? AND timeframe=? AND data_type=?",
+            (symbol.upper(), timeframe.upper(), data_type),
+        ).fetchone()
+
+        if row is None:
+            # Nothing cached — entire range is a gap
+            result.gaps.append({
+                "from": date_from.isoformat(),
+                "to": date_to.isoformat(),
+            })
+            return result
+
+        result.covered = True
+        result.date_from = datetime.fromisoformat(row["date_from"]) if row["date_from"] else None
+        result.date_to = datetime.fromisoformat(row["date_to"]) if row["date_to"] else None
+        result.row_count = row["row_count"] or 0
+        result.file_path = row["file_path"]
+
+        # Identify leading gap (request starts before cached range)
+        if result.date_from and date_from < result.date_from:
+            result.gaps.append({
+                "from": date_from.isoformat(),
+                "to": result.date_from.isoformat(),
+            })
+
+        # Identify trailing gap (request ends after cached range)
+        if result.date_to and date_to > result.date_to:
+            result.gaps.append({
+                "from": result.date_to.isoformat(),
+                "to": date_to.isoformat(),
+            })
+
+        return result
+
+    def check_latest(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> CoverageResult:
+        """Quick check: what's the latest bar we have?"""
+        result = CoverageResult(symbol=symbol.upper(), timeframe=timeframe.upper(), data_type=data_type)
+        row = self._conn.execute(
+            "SELECT * FROM data_registry WHERE symbol=? AND timeframe=? AND data_type=?",
+            (symbol.upper(), timeframe.upper(), data_type),
+        ).fetchone()
+
+        if row:
+            result.covered = True
+            result.date_from = datetime.fromisoformat(row["date_from"]) if row["date_from"] else None
+            result.date_to = datetime.fromisoformat(row["date_to"]) if row["date_to"] else None
+            result.row_count = row["row_count"] or 0
+            result.file_path = row["file_path"]
+
+        return result
+
+    # ── registration & updates ───────────────────────────────────────────────
+
+    def register_dataset(
+        self,
+        symbol: str,
+        timeframe: str,
+        data_type: str,
+        source: str,
+        date_from: datetime,
+        date_to: datetime,
+        row_count: int,
+        file_path: str,
+        checksum: Optional[str] = None,
+    ):
+        """Insert or update a dataset record in the registry."""
+        now = datetime.now(timezone.utc).isoformat()
+        self._conn.execute("""
+            INSERT INTO data_registry (symbol, timeframe, data_type, source,
+                                        date_from, date_to, row_count, file_path,
+                                        last_updated, checksum)
+            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+            ON CONFLICT(symbol, timeframe, data_type) DO UPDATE SET
+                source=excluded.source,
+                date_from=excluded.date_from,
+                date_to=excluded.date_to,
+                row_count=excluded.row_count,
+                file_path=excluded.file_path,
+                last_updated=excluded.last_updated,
+                checksum=excluded.checksum
+        """, (
+            symbol.upper(), timeframe.upper(), data_type, source,
+            date_from.isoformat(), date_to.isoformat(), row_count,
+            file_path, now, checksum,
+        ))
+        self._conn.commit()
+
+    def log_fetch(
+        self,
+        symbol: str,
+        timeframe: str,
+        source: str,
+        fetch_from: datetime,
+        fetch_to: datetime,
+        rows_returned: int,
+        duration_ms: int,
+        status: str = "success",
+        error_msg: Optional[str] = None,
+    ):
+        """Record a fetch attempt in the audit log."""
+        self._conn.execute("""
+            INSERT INTO fetch_log (symbol, timeframe, source, fetch_from, fetch_to,
+                                    rows_returned, duration_ms, status, error_msg, timestamp)
+            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+        """, (
+            symbol.upper(), timeframe.upper(), source,
+            fetch_from.isoformat(), fetch_to.isoformat(),
+            rows_returned, duration_ms, status, error_msg,
+            datetime.now(timezone.utc).isoformat(),
+        ))
+        self._conn.commit()
+
+    # ── summary / status ─────────────────────────────────────────────────────
+
+    def get_summary(self) -> list:
+        """Return all registry entries as a list of dicts."""
+        rows = self._conn.execute("SELECT * FROM data_registry ORDER BY symbol, timeframe").fetchall()
+        return [dict(r) for r in rows]
+
+    def get_fetch_history(self, symbol: Optional[str] = None, limit: int = 50) -> list:
+        if symbol:
+            rows = self._conn.execute(
+                "SELECT * FROM fetch_log WHERE symbol=? ORDER BY timestamp DESC LIMIT ?",
+                (symbol.upper(), limit),
+            ).fetchall()
+        else:
+            rows = self._conn.execute(
+                "SELECT * FROM fetch_log ORDER BY timestamp DESC LIMIT ?", (limit,)
+            ).fetchall()
+        return [dict(r) for r in rows]
+
+    def purge(self, symbol: str, timeframe: str, data_type: str = "ohlcv"):
+        """Remove a dataset from the registry (caller must also delete the Parquet file)."""
+        self._conn.execute(
+            "DELETE FROM data_registry WHERE symbol=? AND timeframe=? AND data_type=?",
+            (symbol.upper(), timeframe.upper(), data_type),
+        )
+        self._conn.commit()
+
+    def close(self):
+        self._conn.close()

backend/data_manager/data_manager.py

@@ -0,0 +1,332 @@
+"""
+DataManager — Main entry point for the Data Management Module.
+
+Callers use this class exclusively.  It coordinates:
+  - DataCatalog  (knows what's cached)
+  - DataStore    (reads/writes Parquet)
+  - Fetchers     (fill gaps from live sources)
+
+All paths are derived relative to a configurable library root.
+Default root is  <project_dir>/data_library/  for full portability.
+
+Usage
+-----
+    from data_manager import dm              # pre-wired singleton
+    df = dm.get_bars("EURUSD", "D1", 1500)   # last 1500 daily bars
+    df = dm.get_bars("XAUUSD", "M15",        # date range fetch
+                     date_from=datetime(2024,1,1),
+                     date_to=datetime(2025,1,1))
+"""
+
+import time
+import pandas as pd
+from pathlib import Path
+from datetime import datetime, timedelta, timezone
+from typing import Optional, List, Union
+
+from .data_catalog import DataCatalog, CoverageResult
+from .data_store import DataStore
+from .fetchers.base_fetcher import BaseFetcher
+from .fetchers.mt5_fetcher import MT5Fetcher, DEFAULT_MT5_PATH
+
+import logging
+log = logging.getLogger("dmm.manager")
+
+
+class DataManager:
+    """
+    Unified data access layer.
+
+    Parameters
+    ----------
+    library_root : Path or str or None
+        Where Parquet files and catalog.db live.
+        Default: <directory of this file>/../data_library/
+    mt5_terminal_path : str or None
+        Path to MT5 terminal64.exe. If None, uses DEFAULT_MT5_PATH
+        from mt5_fetcher.py (single source of truth for the terminal path).
+    auto_init_mt5 : bool
+        If True, DataManager will call mt5.initialize() automatically.
+        If False (default), the caller is responsible for MT5 init —
+        this is the typical pattern since the forecaster already inits MT5.
+    cache_enabled : bool
+        If False, always fetch fresh from source (passthrough mode for testing).
+    """
+
+    def __init__(
+        self,
+        library_root: Union[str, Path, None] = None,
+        mt5_terminal_path: Optional[str] = None,
+        auto_init_mt5: bool = False,
+        cache_enabled: bool = True,
+    ):
+        # Resolve library root relative to this file (portable)
+        if library_root is None:
+            library_root = Path(__file__).resolve().parent.parent / "data_library"
+        self.library_root = Path(library_root).resolve()
+
+        self.cache_enabled = cache_enabled
+        self.store = DataStore(self.library_root)
+        self.catalog = DataCatalog(self.library_root / "catalog.db")
+
+        # Primary fetcher: MT5
+        self._fetcher = MT5Fetcher(
+            terminal_path=mt5_terminal_path,
+            auto_init=auto_init_mt5,
+        )
+
+        # Clean up any interrupted writes from previous runs
+        self.store.cleanup_temp()
+
+        log.info(f"DataManager ready | library: {self.library_root} | cache: {self.cache_enabled}")
+
+    # ═══════════════════════════════════════════════════════════════════════════
+    #  PUBLIC API — what callers use
+    # ═══════════════════════════════════════════════════════════════════════════
+
+    def get_bars(
+        self,
+        symbol: str,
+        timeframe: str,
+        n_bars: Optional[int] = None,
+        date_from: Optional[datetime] = None,
+        date_to: Optional[datetime] = None,
+        data_type: str = "ohlcv",
+    ) -> Optional[pd.DataFrame]:
+        """
+        Get OHLCV bars — the main workhorse method.
+
+        Two calling modes:
+          1. n_bars mode:     dm.get_bars("EURUSD", "D1", n_bars=1500)
+          2. date range mode: dm.get_bars("EURUSD", "M15", date_from=..., date_to=...)
+
+        Returns a DataFrame with DatetimeIndex('time') and lowercase columns,
+        identical to what MT5 returns (drop-in compatible).
+        """
+        symbol = symbol.upper()
+        timeframe = timeframe.upper()
+
+        # ── Mode 1: last N bars ──────────────────────────────────────────────
+        if n_bars is not None:
+            return self._get_bars_n(symbol, timeframe, n_bars, data_type)
+
+        # ── Mode 2: date range ───────────────────────────────────────────────
+        if date_from is None:
+            raise ValueError("Must provide either n_bars or date_from")
+
+        if date_to is None:
+            date_to = datetime.now(timezone.utc)
+
+        # Ensure UTC
+        if date_from.tzinfo is None:
+            date_from = date_from.replace(tzinfo=timezone.utc)
+        if date_to.tzinfo is None:
+            date_to = date_to.replace(tzinfo=timezone.utc)
+
+        # Clamp future dates to now
+        now = datetime.now(timezone.utc)
+        if date_to > now:
+            date_to = now
+
+        return self._get_bars_range(symbol, timeframe, date_from, date_to, data_type)
+
+    def get_bars_as_forecaster(
+        self,
+        symbol: str,
+        timeframe: str,
+        n_bars: Optional[int] = None,
+        date_from: Optional[datetime] = None,
+        date_to: Optional[datetime] = None,
+    ) -> Optional[pd.DataFrame]:
+        """
+        Same as get_bars() but returns with capitalized column names
+        (Open, High, Low, Close, Volume) and 'time' as the index —
+        matching what the forecaster currently expects from fetch_daily_data()
+        and fetch_intraday_data().
+        """
+        df = self.get_bars(symbol, timeframe, n_bars=n_bars,
+                           date_from=date_from, date_to=date_to)
+        if df is None:
+            return None
+
+        # Rename to match forecaster convention
+        rename_map = {
+            "open": "Open", "high": "High", "low": "Low",
+            "close": "Close", "tick_volume": "Volume",
+        }
+        df.rename(columns=rename_map, inplace=True)
+
+        # Strip timezone for compat with existing code that uses tz-naive datetimes
+        if df.index.tz is not None:
+            df.index = df.index.tz_localize(None)
+
+        return df
+
+    def prefetch(
+        self,
+        symbols: List[str],
+        timeframes: List[str],
+        date_from: datetime,
+        date_to: datetime,
+        data_type: str = "ohlcv",
+    ):
+        """Batch warm-up: ensure all (symbol, tf) combos are cached."""
+        total = len(symbols) * len(timeframes)
+        done = 0
+        for sym in symbols:
+            for tf in timeframes:
+                done += 1
+                log.info(f"[{done}/{total}] Prefetching {sym} {tf}...")
+                self.get_bars(sym, tf, date_from=date_from, date_to=date_to, data_type=data_type)
+
+    def get_catalog_summary(self) -> list:
+        """Return a summary of all cached datasets."""
+        return self.catalog.get_summary()
+
+    def get_fetch_history(self, symbol: Optional[str] = None, limit: int = 50) -> list:
+        return self.catalog.get_fetch_history(symbol, limit)
+
+    def purge(self, symbol: str, timeframe: str, data_type: str = "ohlcv"):
+        """Remove cached data for a specific (symbol, tf, data_type) tuple."""
+        fpath = self.store._parquet_path(symbol, timeframe, data_type)
+        if fpath.exists():
+            fpath.unlink()
+        meta = self.store._meta_path(symbol, timeframe, data_type)
+        if meta.exists():
+            meta.unlink()
+        self.catalog.purge(symbol, timeframe, data_type)
+        log.info(f"Purged {symbol} {timeframe} {data_type}")
+
+    # ═══════════════════════════════════════════════════════════════════════════
+    #  INTERNAL — fetch orchestration
+    # ═══════════════════════════════════════════════════════════════════════════
+
+    def _get_bars_n(self, symbol: str, timeframe: str, n_bars: int,
+                    data_type: str) -> Optional[pd.DataFrame]:
+        """
+        Get last N bars.  Strategy:
+        1. If cache has enough rows and is recent enough, return from cache.
+        2. Otherwise, fetch from source, cache, return.
+        """
+        if self.cache_enabled:
+            coverage = self.catalog.check_latest(symbol, timeframe, data_type)
+            if coverage.covered and coverage.row_count >= n_bars:
+                # Check if cache is reasonably fresh (within the timeframe interval)
+                # For D1, "fresh" means updated today; for M15, within the last hour
+                df = self.store.load(symbol, timeframe, data_type)
+                if df is not None and len(df) >= n_bars:
+                    # Still fetch a small update to catch new bars
+                    self._update_trailing(symbol, timeframe, data_type, coverage)
+                    df = self.store.load(symbol, timeframe, data_type)
+                    if df is not None:
+                        return df.tail(n_bars)
+
+        # Cache miss or disabled — fetch from source
+        t0 = time.time()
+        df = self._fetcher.fetch_bars_n(symbol, timeframe, n_bars)
+        elapsed_ms = int((time.time() - t0) * 1000)
+
+        if df is None or df.empty:
+            self.catalog.log_fetch(symbol, timeframe, self._fetcher.name,
+                                   datetime.now(timezone.utc), datetime.now(timezone.utc),
+                                   0, elapsed_ms, "failed", "No data returned")
+            return None
+
+        # Cache the result
+        if self.cache_enabled:
+            self.store.append(symbol, timeframe, df, data_type)
+            checksum = self.store.compute_checksum(symbol, timeframe, data_type)
+            file_path = str(self.store._parquet_path(symbol, timeframe, data_type).relative_to(self.library_root))
+            self.catalog.register_dataset(
+                symbol, timeframe, data_type, self._fetcher.name,
+                df.index.min(), df.index.max(), len(df),
+                file_path, checksum,
+            )
+
+        self.catalog.log_fetch(symbol, timeframe, self._fetcher.name,
+                               df.index.min(), df.index.max(),
+                               len(df), elapsed_ms, "success")
+
+        return df.tail(n_bars)
+
+    def _get_bars_range(self, symbol: str, timeframe: str,
+                        date_from: datetime, date_to: datetime,
+                        data_type: str) -> Optional[pd.DataFrame]:
+        """
+        Get bars for a date range.  Strategy:
+        1. Check catalog for coverage.
+        2. Fetch only gaps (smart gap filling).
+        3. Return full range from cache.
+        """
+        if not self.cache_enabled:
+            return self._fetch_and_log(symbol, timeframe, date_from, date_to, data_type)
+
+        coverage = self.catalog.check_coverage(symbol, timeframe, date_from, date_to, data_type)
+
+        if coverage.needs_fetch:
+            for gap in coverage.gaps:
+                gap_from = datetime.fromisoformat(gap["from"])
+                gap_to = datetime.fromisoformat(gap["to"])
+                log.info(f"Filling gap: {symbol} {timeframe} {gap_from.date()}→{gap_to.date()}")
+                fetched = self._fetch_and_log(symbol, timeframe, gap_from, gap_to, data_type)
+                if fetched is not None and not fetched.empty:
+                    self.store.append(symbol, timeframe, fetched, data_type)
+
+            # Update catalog with new combined coverage
+            full_df = self.store.load(symbol, timeframe, data_type)
+            if full_df is not None and not full_df.empty:
+                checksum = self.store.compute_checksum(symbol, timeframe, data_type)
+                file_path = str(self.store._parquet_path(symbol, timeframe, data_type).relative_to(self.library_root))
+                self.catalog.register_dataset(
+                    symbol, timeframe, data_type, self._fetcher.name,
+                    full_df.index.min(), full_df.index.max(), len(full_df),
+                    file_path, checksum,
+                )
+
+        # Return requested slice from cache
+        df = self.store.load(symbol, timeframe, data_type, date_from, date_to)
+        return df
+
+    def _update_trailing(self, symbol: str, timeframe: str, data_type: str,
+                         coverage: CoverageResult):
+        """Fetch only bars newer than what's already cached."""
+        if coverage.date_to is None:
+            return
+        # Start from last cached bar (will be deduped)
+        fetch_from = coverage.date_to - timedelta(hours=1)  # small overlap for safety
+        fetch_to = datetime.now(timezone.utc)
+        if fetch_to <= fetch_from:
+            return
+
+        fetched = self._fetch_and_log(symbol, timeframe, fetch_from, fetch_to, data_type)
+        if fetched is not None and not fetched.empty:
+            self.store.append(symbol, timeframe, fetched, data_type)
+            # Re-read and update catalog
+            full_df = self.store.load(symbol, timeframe, data_type)
+            if full_df is not None and not full_df.empty:
+                checksum = self.store.compute_checksum(symbol, timeframe, data_type)
+                file_path = str(self.store._parquet_path(symbol, timeframe, data_type).relative_to(self.library_root))
+                self.catalog.register_dataset(
+                    symbol, timeframe, data_type, self._fetcher.name,
+                    full_df.index.min(), full_df.index.max(), len(full_df),
+                    file_path, checksum,
+                )
+
+    def _fetch_and_log(self, symbol: str, timeframe: str,
+                       date_from: datetime, date_to: datetime,
+                       data_type: str) -> Optional[pd.DataFrame]:
+        """Fetch from source with timing and audit logging."""
+        t0 = time.time()
+        df = self._fetcher.fetch_bars(symbol, timeframe, date_from, date_to)
+        elapsed_ms = int((time.time() - t0) * 1000)
+
+        if df is None or df.empty:
+            self.catalog.log_fetch(symbol, timeframe, self._fetcher.name,
+                                   date_from, date_to, 0, elapsed_ms,
+                                   "failed", "No data returned")
+            return None
+
+        self.catalog.log_fetch(symbol, timeframe, self._fetcher.name,
+                               date_from, date_to, len(df), elapsed_ms, "success")
+        log.info(f"Fetched {len(df)} rows {symbol} {timeframe} ({elapsed_ms}ms)")
+        return df

backend/data_manager/data_store.py

@@ -0,0 +1,278 @@
+"""
+DataStore — Format-agnostic data storage engine with append, dedup, and validation.
+
+Storage format priority:
+  1. Parquet (pyarrow) — production default, industry-standard columnar format
+  2. Pickle — fallback when pyarrow is unavailable (testing / lightweight setups)
+
+All file operations use relative paths anchored to the data_library root.
+Designed for portability: no hardcoded absolute paths anywhere.
+"""
+
+import pandas as pd
+import numpy as np
+import json
+import hashlib
+from pathlib import Path
+from datetime import datetime, timezone
+from typing import Optional, List
+
+import logging
+log = logging.getLogger("dmm.store")
+
+# ─── Detect best available engine ────────────────────────────────────────────
+_ENGINE = "pickle"  # safe default
+try:
+    import pyarrow
+    _ENGINE = "parquet"
+except ImportError:
+    try:
+        import fastparquet
+        _ENGINE = "parquet"
+    except ImportError:
+        pass  # stay on pickle
+
+_EXT = ".parquet" if _ENGINE == "parquet" else ".pkl"
+
+# ─── Expected OHLCV schema from MT5 ─────────────────────────────────────────
+OHLCV_COLUMNS = ["time", "open", "high", "low", "close", "tick_volume", "spread", "real_volume"]
+OHLCV_DTYPES  = {
+    "open": "float64", "high": "float64", "low": "float64", "close": "float64",
+    "tick_volume": "int64", "spread": "int64", "real_volume": "int64",
+}
+
+
+class ValidationResult:
+    """Result of a data file validation check."""
+    def __init__(self, valid: bool, issues: List[str] = None, row_count: int = 0):
+        self.valid = valid
+        self.issues = issues or []
+        self.row_count = row_count
+
+    def __repr__(self):
+        tag = "OK" if self.valid else f"FAIL({len(self.issues)})"
+        return f"<ValidationResult {tag} rows={self.row_count}>"
+
+
+class DataStore:
+    """
+    Data store with auto-detected format (Parquet or Pickle).
+
+    Parameters
+    ----------
+    library_root : Path
+        Absolute path to the data_library/ directory (created if missing).
+    """
+
+    def __init__(self, library_root: Path):
+        self.root = Path(library_root)
+        self.root.mkdir(parents=True, exist_ok=True)
+        self._temp = self.root / "_temp"
+        self._temp.mkdir(exist_ok=True)
+        self.engine = _ENGINE
+        self.ext = _EXT
+        log.info(f"DataStore engine: {self.engine} (ext: {self.ext})")
+
+    # ── path helpers ─────────────────────────────────────────────────────────
+
+    def _data_path(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> Path:
+        return self.root / symbol.upper() / timeframe.upper() / f"{data_type}{self.ext}"
+
+    def _meta_path(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> Path:
+        return self.root / symbol.upper() / timeframe.upper() / f"{data_type}.meta.json"
+
+    def _temp_path(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> Path:
+        return self._temp / f"{symbol}_{timeframe}_{data_type}{self.ext}"
+
+    # Backward-compat alias used by DataManager
+    def _parquet_path(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> Path:
+        return self._data_path(symbol, timeframe, data_type)
+
+    # ── low-level IO ─────────────────────────────────────────────────────────
+
+    def _write_df(self, df: pd.DataFrame, path: Path):
+        if self.engine == "parquet":
+            df.to_parquet(path, engine="pyarrow", compression="snappy", index=True)
+        else:
+            df.to_pickle(path)
+
+    def _read_df(self, path: Path, columns: Optional[List[str]] = None) -> pd.DataFrame:
+        if self.engine == "parquet":
+            return pd.read_parquet(path, columns=columns)
+        else:
+            df = pd.read_pickle(path)
+            if columns is not None:
+                avail = [c for c in columns if c in df.columns]
+                df = df[avail]
+            return df
+
+    # ── core API ─────────────────────────────────────────────────────────────
+
+    def exists(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> bool:
+        return self._data_path(symbol, timeframe, data_type).exists()
+
+    def load(
+        self,
+        symbol: str,
+        timeframe: str,
+        data_type: str = "ohlcv",
+        date_from: Optional[datetime] = None,
+        date_to: Optional[datetime] = None,
+        columns: Optional[List[str]] = None,
+    ) -> Optional[pd.DataFrame]:
+        """Load data with optional date and column filtering. Returns None if missing."""
+        fpath = self._data_path(symbol, timeframe, data_type)
+        if not fpath.exists():
+            return None
+
+        try:
+            df = self._read_df(fpath, columns)
+        except Exception as e:
+            log.error(f"Failed to read {fpath}: {e}")
+            return None
+
+        # Ensure DatetimeIndex
+        if "time" in df.columns and not isinstance(df.index, pd.DatetimeIndex):
+            df["time"] = pd.to_datetime(df["time"], utc=True)
+            df.set_index("time", inplace=True)
+        elif isinstance(df.index, pd.DatetimeIndex):
+            if df.index.tz is None:
+                df.index = df.index.tz_localize("UTC")
+
+        if date_from is not None:
+            if hasattr(date_from, 'tzinfo') and date_from.tzinfo is None:
+                date_from = date_from.replace(tzinfo=timezone.utc)
+            df = df[df.index >= date_from]
+        if date_to is not None:
+            if hasattr(date_to, 'tzinfo') and date_to.tzinfo is None:
+                date_to = date_to.replace(tzinfo=timezone.utc)
+            df = df[df.index <= date_to]
+
+        return df
+
+    def save(self, symbol: str, timeframe: str, df: pd.DataFrame,
+             data_type: str = "ohlcv") -> Path:
+        """Full write. Writes to temp first, then atomic move to final path."""
+        df = self._normalize(df)
+        fpath = self._data_path(symbol, timeframe, data_type)
+        fpath.parent.mkdir(parents=True, exist_ok=True)
+
+        tmp = self._temp_path(symbol, timeframe, data_type)
+        self._write_df(df, tmp)
+        tmp.replace(fpath)
+
+        self._write_meta(symbol, timeframe, data_type, df)
+        log.info(f"Saved {len(df)} rows -> {fpath.relative_to(self.root)} [{self.engine}]")
+        return fpath
+
+    def append(self, symbol: str, timeframe: str, new_data: pd.DataFrame,
+               data_type: str = "ohlcv") -> Path:
+        """Append new rows with dedup at the seam. Creates file if missing."""
+        new_data = self._normalize(new_data)
+        fpath = self._data_path(symbol, timeframe, data_type)
+
+        if not fpath.exists():
+            return self.save(symbol, timeframe, new_data, data_type)
+
+        existing = self._read_df(fpath)
+        if "time" in existing.columns:
+            existing["time"] = pd.to_datetime(existing["time"], utc=True)
+            existing.set_index("time", inplace=True)
+        if isinstance(existing.index, pd.DatetimeIndex) and existing.index.tz is None:
+            existing.index = existing.index.tz_localize("UTC")
+
+        combined = pd.concat([existing, new_data])
+        combined = combined[~combined.index.duplicated(keep="last")]
+        combined.sort_index(inplace=True)
+
+        return self.save(symbol, timeframe, combined, data_type)
+
+    def validate(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> ValidationResult:
+        """Zero-trust validation: readability, schema, NaN, monotonic timestamps."""
+        issues = []
+        fpath = self._data_path(symbol, timeframe, data_type)
+
+        if not fpath.exists():
+            return ValidationResult(False, ["File does not exist"], 0)
+
+        try:
+            df = self._read_df(fpath)
+        except Exception as e:
+            return ValidationResult(False, [f"Cannot read file: {e}"], 0)
+
+        row_count = len(df)
+        if row_count == 0:
+            issues.append("File is empty")
+
+        if "time" in df.columns:
+            df["time"] = pd.to_datetime(df["time"])
+            if not df["time"].is_monotonic_increasing:
+                issues.append("Timestamps are not monotonic")
+        elif isinstance(df.index, pd.DatetimeIndex):
+            if not df.index.is_monotonic_increasing:
+                issues.append("Timestamps are not monotonic")
+        else:
+            issues.append("No 'time' column or DatetimeIndex found")
+
+        if data_type == "ohlcv":
+            for col in ["open", "high", "low", "close"]:
+                if col in df.columns and df[col].isna().any():
+                    n = df[col].isna().sum()
+                    issues.append(f"NaN in '{col}': {n} rows")
+
+        return ValidationResult(valid=len(issues) == 0, issues=issues, row_count=row_count)
+
+    def compute_checksum(self, symbol: str, timeframe: str, data_type: str = "ohlcv") -> Optional[str]:
+        fpath = self._data_path(symbol, timeframe, data_type)
+        if not fpath.exists():
+            return None
+        h = hashlib.md5()
+        with open(fpath, "rb") as f:
+            for chunk in iter(lambda: f.read(8192), b""):
+                h.update(chunk)
+        return h.hexdigest()
+
+    # ── internals ────────────────────────────────────────────────────────────
+
+    def _normalize(self, df: pd.DataFrame) -> pd.DataFrame:
+        """Ensure consistent format: DatetimeIndex named 'time', UTC, sorted."""
+        df = df.copy()
+
+        if "time" in df.columns and not isinstance(df.index, pd.DatetimeIndex):
+            df["time"] = pd.to_datetime(df["time"], utc=True)
+            df.set_index("time", inplace=True)
+
+        if isinstance(df.index, pd.DatetimeIndex):
+            if df.index.tz is None:
+                df.index = df.index.tz_localize("UTC")
+            elif str(df.index.tz) != "UTC":
+                df.index = df.index.tz_convert("UTC")
+
+        df.index.name = "time"
+        df.sort_index(inplace=True)
+        df.columns = [c.lower() for c in df.columns]
+
+        return df
+
+    def _write_meta(self, symbol: str, timeframe: str, data_type: str, df: pd.DataFrame):
+        meta_path = self._meta_path(symbol, timeframe, data_type)
+        meta = {
+            "symbol": symbol.upper(),
+            "timeframe": timeframe.upper(),
+            "data_type": data_type,
+            "storage_engine": self.engine,
+            "row_count": len(df),
+            "date_from": str(df.index.min()),
+            "date_to": str(df.index.max()),
+            "last_updated": datetime.now(timezone.utc).isoformat(),
+            "schema_version": 1,
+        }
+        with open(meta_path, "w") as f:
+            json.dump(meta, f, indent=2)
+
+    def cleanup_temp(self):
+        """Remove any leftover temp files from interrupted writes."""
+        for f in self._temp.iterdir():
+            if f.is_file():
+                f.unlink()
+                log.info(f"Cleaned temp file: {f.name}")

backend/data_manager/fetchers/base_fetcher.py

@@ -0,0 +1,54 @@
+"""
+BaseFetcher — abstract interface for data source adapters.
+
+Every fetcher (MT5, DXFeed, futures bridge, CSV import) implements this.
+"""
+
+from abc import ABC, abstractmethod
+from datetime import datetime
+from typing import Optional
+import pandas as pd
+
+
+class BaseFetcher(ABC):
+    """
+    Abstract data fetcher interface.
+
+    Implementations must return DataFrames with:
+      - DatetimeIndex named 'time' (UTC)
+      - Lowercase column names: open, high, low, close, tick_volume, spread, real_volume
+    """
+
+    name: str = "base"  # Override in subclass
+
+    @abstractmethod
+    def fetch_bars(
+        self,
+        symbol: str,
+        timeframe: str,
+        date_from: datetime,
+        date_to: datetime,
+    ) -> Optional[pd.DataFrame]:
+        """
+        Fetch OHLCV bars for the given symbol/timeframe/date range.
+        Returns None on failure.
+        """
+        ...
+
+    @abstractmethod
+    def fetch_bars_n(
+        self,
+        symbol: str,
+        timeframe: str,
+        n_bars: int,
+    ) -> Optional[pd.DataFrame]:
+        """
+        Fetch the most recent N bars.
+        Returns None on failure.
+        """
+        ...
+
+    @abstractmethod
+    def is_available(self) -> bool:
+        """Check if this data source is currently reachable."""
+        ...

backend/data_manager/fetchers/mt5_fetcher.py

@@ -0,0 +1,272 @@
+"""
+MT5Fetcher — MetaTrader 5 data source adapter.
+
+Wraps mt5.copy_rates_range() and mt5.copy_rates_from_pos() with:
+  - Automatic timezone conversion to UTC
+  - Consistent lowercase column names
+  - Graceful failure handling
+  - Symbol alias resolution (GER40 → DE40 etc.) via server_config.SYMBOL_ALIASES
+"""
+
+import pandas as pd
+import numpy as np
+from datetime import datetime, timezone
+from typing import Optional, Dict, List
+from pathlib import Path
+
+from .base_fetcher import BaseFetcher
+
+import logging
+log = logging.getLogger("dmm.mt5_fetcher")
+
+# ── Load MT5 terminal path from local_config.ini (Rule 3 — no hardcoded paths) ──
+def _load_mt5_path_from_ini() -> Optional[str]:
+    """Read MT5 terminal path from local_config.ini if it exists."""
+    import configparser
+    ini = Path(__file__).resolve().parent.parent.parent / "local_config.ini"
+    if not ini.exists():
+        return None
+    cp = configparser.ConfigParser()
+    cp.read(str(ini))
+    return cp.get("mt5", "terminal_path", fallback=None)
+
+DEFAULT_MT5_PATH = _load_mt5_path_from_ini()
+
+# ── Symbol alias map — imported from server_config if available ──
+_SYMBOL_ALIASES: Dict[str, List[str]] = {}
+
+def _load_aliases():
+    """Load SYMBOL_ALIASES from server_config.py (same project root)."""
+    global _SYMBOL_ALIASES
+    if _SYMBOL_ALIASES:
+        return
+    try:
+        import sys
+        project_root = str(Path(__file__).resolve().parent.parent.parent)
+        if project_root not in sys.path:
+            sys.path.insert(0, project_root)
+        from server_config import SYMBOL_ALIASES
+        _SYMBOL_ALIASES = SYMBOL_ALIASES
+    except ImportError:
+        log.info("server_config not found — symbol aliases unavailable in MT5Fetcher")
+
+# ── MT5 timeframe mapping ────────────────────────────────────────────────────
+_TF_MAP = {}  # Populated lazily after mt5 import
+
+
+def _ensure_tf_map():
+    """Build timeframe string → MT5 constant map. Safe to call repeatedly."""
+    global _TF_MAP
+    if _TF_MAP:
+        return
+    try:
+        import MetaTrader5 as mt5
+        _TF_MAP.update({
+            "M1":  mt5.TIMEFRAME_M1,
+            "M2":  mt5.TIMEFRAME_M2,
+            "M3":  mt5.TIMEFRAME_M3,
+            "M4":  mt5.TIMEFRAME_M4,
+            "M5":  mt5.TIMEFRAME_M5,
+            "M6":  mt5.TIMEFRAME_M6,
+            "M10": mt5.TIMEFRAME_M10,
+            "M12": mt5.TIMEFRAME_M12,
+            "M15": mt5.TIMEFRAME_M15,
+            "M20": mt5.TIMEFRAME_M20,
+            "M30": mt5.TIMEFRAME_M30,
+            "H1":  mt5.TIMEFRAME_H1,
+            "H2":  mt5.TIMEFRAME_H2,
+            "H3":  mt5.TIMEFRAME_H3,
+            "H4":  mt5.TIMEFRAME_H4,
+            "H6":  mt5.TIMEFRAME_H6,
+            "H8":  mt5.TIMEFRAME_H8,
+            "H12": mt5.TIMEFRAME_H12,
+            "D1":  mt5.TIMEFRAME_D1,
+            "W1":  mt5.TIMEFRAME_W1,
+            "MN":  mt5.TIMEFRAME_MN1,
+        })
+    except (ModuleNotFoundError, AttributeError):
+        log.warning("MetaTrader5 module not available — MT5Fetcher will be non-functional")
+
+
+class MT5Fetcher(BaseFetcher):
+    """
+    MetaTrader 5 data fetcher.
+
+    Parameters
+    ----------
+    terminal_path : str or None
+        Path to terminal64.exe. If None, reads from local_config.ini.
+    auto_init : bool
+        If True, call mt5.initialize() on first fetch. If False, caller
+        is responsible for ensuring MT5 is already initialized.
+    """
+
+    name = "mt5"
+
+    def __init__(self, terminal_path: Optional[str] = None, auto_init: bool = False):
+        self.terminal_path = terminal_path or DEFAULT_MT5_PATH
+        self.auto_init = auto_init
+        self._initialized = False
+        self._symbol_cache: Dict[str, str] = {}  # canonical → broker symbol
+
+    def _resolve_symbol(self, canonical: str) -> str:
+        """
+        Resolve canonical ticker to broker symbol via alias lookup.
+        Caches successful resolutions. Returns canonical if no alias found.
+        """
+        if canonical in self._symbol_cache:
+            return self._symbol_cache[canonical]
+
+        _load_aliases()
+
+        try:
+            import MetaTrader5 as mt5
+
+            # Try canonical name first
+            info = mt5.symbol_info(canonical)
+            if info is not None:
+                if not info.visible:
+                    mt5.symbol_select(canonical, True)
+                self._symbol_cache[canonical] = canonical
+                return canonical
+
+            # Try aliases
+            aliases = _SYMBOL_ALIASES.get(canonical, [])
+            for alias in aliases:
+                if alias == canonical:
+                    continue
+                info = mt5.symbol_info(alias)
+                if info is not None:
+                    if not info.visible:
+                        mt5.symbol_select(alias, True)
+                    log.info(f"Symbol alias resolved: {canonical} → {alias}")
+                    self._symbol_cache[canonical] = alias
+                    return alias
+
+            # No alias found — return canonical (will fail downstream)
+            log.warning(f"No broker symbol found for {canonical}")
+            self._symbol_cache[canonical] = canonical
+            return canonical
+
+        except Exception:
+            return canonical
+
+    def _ensure_init(self) -> bool:
+        """Initialize MT5 if auto_init is on and we haven't yet."""
+        if self._initialized:
+            return True
+        if not self.auto_init:
+            # Assume caller already initialized MT5
+            self._initialized = True
+            return True
+        try:
+            import MetaTrader5 as mt5
+            kwargs = {}
+            if self.terminal_path:
+                kwargs["path"] = self.terminal_path
+            if mt5.initialize(**kwargs):
+                self._initialized = True
+                return True
+            else:
+                log.error(f"MT5 init failed: {mt5.last_error()}")
+                return False
+        except ModuleNotFoundError:
+            log.error("MetaTrader5 package not installed")
+            return False
+
+    def is_available(self) -> bool:
+        return self._ensure_init()
+
+    def fetch_bars(
+        self,
+        symbol: str,
+        timeframe: str,
+        date_from: datetime,
+        date_to: datetime,
+    ) -> Optional[pd.DataFrame]:
+        """
+        Fetch OHLCV bars via mt5.copy_rates_range().
+        """
+        _ensure_tf_map()
+        if not self._ensure_init():
+            return None
+
+        tf_const = _TF_MAP.get(timeframe.upper())
+        if tf_const is None:
+            log.error(f"Unknown timeframe: {timeframe}")
+            return None
+
+        try:
+            import MetaTrader5 as mt5
+
+            broker_sym = self._resolve_symbol(symbol)
+
+            if date_from.tzinfo is None:
+                date_from = date_from.replace(tzinfo=timezone.utc)
+            if date_to.tzinfo is None:
+                date_to = date_to.replace(tzinfo=timezone.utc)
+
+            rates = mt5.copy_rates_range(broker_sym, tf_const, date_from, date_to)
+            if rates is None or len(rates) == 0:
+                log.warning(f"No data returned for {symbol} ({broker_sym}) {timeframe} "
+                          f"{date_from.date()}→{date_to.date()}")
+                return None
+
+            return self._to_dataframe(rates)
+
+        except Exception as e:
+            log.error(f"MT5 fetch_bars error for {symbol}: {e}")
+            return None
+
+    def fetch_bars_n(
+        self,
+        symbol: str,
+        timeframe: str,
+        n_bars: int,
+    ) -> Optional[pd.DataFrame]:
+        """
+        Fetch the most recent N bars via mt5.copy_rates_from_pos().
+        """
+        _ensure_tf_map()
+        if not self._ensure_init():
+            return None
+
+        tf_const = _TF_MAP.get(timeframe.upper())
+        if tf_const is None:
+            log.error(f"Unknown timeframe: {timeframe}")
+            return None
+
+        try:
+            import MetaTrader5 as mt5
+
+            broker_sym = self._resolve_symbol(symbol)
+
+            rates = mt5.copy_rates_from_pos(broker_sym, tf_const, 0, n_bars)
+            if rates is None or len(rates) == 0:
+                log.warning(f"No data returned for {symbol} ({broker_sym}) {timeframe} (last {n_bars} bars)")
+                return None
+
+            return self._to_dataframe(rates)
+
+        except Exception as e:
+            log.error(f"MT5 fetch_bars_n error for {symbol}: {e}")
+            return None
+
+    # ── internal ─────────────────────────────────────────────────────────────
+
+    @staticmethod
+    def _to_dataframe(rates) -> pd.DataFrame:
+        """Convert MT5 rates array to standardized DataFrame."""
+        df = pd.DataFrame(rates)
+        df["time"] = pd.to_datetime(df["time"], unit="s", utc=True)
+        df.set_index("time", inplace=True)
+        df.index.name = "time"
+
+        # Ensure lowercase column names
+        df.columns = [c.lower() for c in df.columns]
+
+        # Keep only standard columns if present
+        keep = ["open", "high", "low", "close", "tick_volume", "spread", "real_volume"]
+        df = df[[c for c in keep if c in df.columns]]
+
+        return df

backend/data_manager/forcaster_dmm_patch.py

@@ -0,0 +1,207 @@
+"""
+forcaster_dmm_patch.py — Data Management Module integration for forcasterv21.
+
+This module provides drop-in replacement functions for the forecaster's
+fetch_daily_data() and fetch_intraday_data() that route through the DMM
+instead of calling MT5 directly.
+
+INTEGRATION (3 steps in forcasterv21_institutional_anchors.py):
+─────────────────────────────────────────────────────────────────
+
+STEP 1 — Add import near line 28 (after other module imports):
+
+    from data_manager import DataManager
+
+STEP 2 — Replace the fetch functions (lines 66-93) with:
+
+    # ==========================================
+    # 📊 DATA INGESTION (via Data Management Module)
+    # ==========================================
+    _dm = None  # Initialized in __main__ after MT5 init
+
+    def fetch_daily_data(ticker, days_back=1500):
+        if _dm is None:
+            return _fetch_daily_data_legacy(ticker, days_back)
+        df = _dm.get_bars_as_forecaster(ticker, "D1", n_bars=days_back)
+        return df
+
+    def fetch_intraday_data(ticker, start_time, end_time=None):
+        if _dm is None:
+            return _fetch_intraday_data_legacy(ticker, start_time, end_time)
+        df = _dm.get_bars_as_forecaster(ticker, "M15", n_bars=20000)
+        if df is None: return None
+        df = df[df.index >= start_time]
+        if end_time is not None:
+            df = df[df.index <= end_time]
+        return df
+
+    # Legacy fallback (direct MT5) — used if DMM init fails
+    def _fetch_daily_data_legacy(ticker, days_back=1500):
+        if not mt5.initialize(path=MT5_TERMINAL_PATH):
+            print(f"[!] MT5 Init failed for {ticker}: {mt5.last_error()}")
+            return None
+        rates = mt5.copy_rates_from_pos(ticker, mt5.TIMEFRAME_D1, 0, days_back)
+        if rates is None or len(rates) == 0: return None
+        df = pd.DataFrame(rates)
+        df['time'] = pd.to_datetime(df['time'], unit='s')
+        df.rename(columns={'open': 'Open', 'high': 'High', 'low': 'Low',
+                           'close': 'Close', 'tick_volume': 'Volume'}, inplace=True)
+        df.set_index('time', inplace=True)
+        return df
+
+    def _fetch_intraday_data_legacy(ticker, start_time, end_time=None):
+        rates = mt5.copy_rates_from_pos(ticker, mt5.TIMEFRAME_M15, 0, 20000)
+        if rates is None or len(rates) == 0: return None
+        df = pd.DataFrame(rates)
+        df['time'] = pd.to_datetime(df['time'], unit='s')
+        df.rename(columns={'open': 'Open', 'high': 'High', 'low': 'Low',
+                           'close': 'Close'}, inplace=True)
+        df.set_index('time', inplace=True)
+        df = df[df.index >= start_time]
+        if end_time is not None:
+            df = df[df.index <= end_time]
+        return df
+
+
+STEP 3 — In __main__ block (after MT5 init, ~line 1855), add DMM initialization:
+
+    if __name__ == "__main__":
+        if not mt5.initialize(path=MT5_TERMINAL_PATH):
+            print("[!] MT5 Init failed.")
+            quit()
+
+        # >>> NEW: Initialize Data Management Module <<<
+        try:
+            _dm = DataManager(mt5_terminal_path=MT5_TERMINAL_PATH)
+            print(f"[DMM] Data library: {_dm.library_root} | Engine: {_dm.store.engine}")
+        except Exception as e:
+            print(f"[DMM] Init failed ({e}) — falling back to direct MT5")
+            _dm = None
+
+        # ... rest of __main__ unchanged ...
+
+That's it. Three changes. Everything else stays the same.
+"""
+
+# ─── Standalone helper for quick integration testing ─────────────────────────
+# You can run this file directly to test DMM + forecaster compatibility:
+#   python forcaster_dmm_patch.py
+
+import sys
+from pathlib import Path
+
+def test_integration():
+    """Test that DMM produces forecaster-compatible DataFrames."""
+    sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+    from data_manager import DataManager
+    from datetime import datetime, timezone
+    import pandas as pd
+    import numpy as np
+
+    print("=" * 60)
+    print("  FORECASTER ↔ DMM INTEGRATION TEST")
+    print("=" * 60)
+
+    dm = DataManager(
+        library_root=Path(__file__).resolve().parent / "_test_fcst_lib",
+        cache_enabled=True,
+    )
+
+    # Inject mock to simulate MT5
+    class MockMT5Fetcher:
+        name = "mock_mt5"
+        def fetch_bars(self, symbol, timeframe, date_from, date_to):
+            n = max(1, (date_to - date_from).days + 1)
+            return _make_synthetic(date_from, n)
+        def fetch_bars_n(self, symbol, timeframe, n_bars):
+            return _make_synthetic(datetime(2020, 1, 1, tzinfo=timezone.utc), n_bars)
+        def is_available(self):
+            return True
+
+    def _make_synthetic(start, n):
+        if hasattr(start, 'tzinfo') and start.tzinfo is None:
+            start = start.replace(tzinfo=timezone.utc)
+        from datetime import timedelta
+        dates = pd.DatetimeIndex(
+            [start + timedelta(days=i) for i in range(n)],
+            name="time"
+        )
+        rng = np.random.default_rng(42)
+        close = 1800.0 + np.cumsum(rng.normal(0, 10, n))
+        return pd.DataFrame({
+            "open": close + rng.normal(0, 2, n),
+            "high": close + np.abs(rng.normal(0, 5, n)),
+            "low":  close - np.abs(rng.normal(0, 5, n)),
+            "close": close,
+            "tick_volume": rng.integers(100, 5000, n),
+            "spread": rng.integers(1, 30, n),
+            "real_volume": np.zeros(n, dtype=int),
+        }, index=dates)
+
+    dm._fetcher = MockMT5Fetcher()
+    passed = 0
+    failed = 0
+
+    # Test 1: get_bars_as_forecaster matches expected schema
+    df = dm.get_bars_as_forecaster("XAUUSD", "D1", n_bars=1500)
+    if df is not None and all(c in df.columns for c in ["Open", "High", "Low", "Close", "Volume"]):
+        print("  ✓ Column names: Open, High, Low, Close, Volume")
+        passed += 1
+    else:
+        print(f"  ✗ Column names wrong: {df.columns.tolist() if df is not None else 'None'}")
+        failed += 1
+
+    if df is not None and df.index.tz is None:
+        print("  ✓ Timezone-naive index (forecaster compat)")
+        passed += 1
+    else:
+        print("  ✗ Index has timezone (forecaster expects tz-naive)")
+        failed += 1
+
+    if df is not None and len(df) == 1500:
+        print("  ✓ Correct row count (1500)")
+        passed += 1
+    else:
+        print(f"  ✗ Row count: {len(df) if df is not None else 'None'}")
+        failed += 1
+
+    if df is not None and isinstance(df.index, pd.DatetimeIndex):
+        print("  ✓ DatetimeIndex preserved")
+        passed += 1
+    else:
+        print("  ✗ Index is not DatetimeIndex")
+        failed += 1
+
+    # Test 2: Intraday M15 with date filtering
+    from datetime import timedelta
+    df15 = dm.get_bars_as_forecaster("XAUUSD", "M15", n_bars=500)
+    if df15 is not None and len(df15) == 500:
+        print("  ✓ M15 intraday fetch (500 bars)")
+        passed += 1
+    else:
+        print(f"  ✗ M15 fetch: {len(df15) if df15 is not None else 'None'}")
+        failed += 1
+
+    # Test 3: Second fetch is faster (from cache)
+    import time
+    t0 = time.time()
+    dm.get_bars_as_forecaster("XAUUSD", "D1", n_bars=1500)
+    elapsed = time.time() - t0
+    print(f"  ✓ Cached D1 fetch: {elapsed*1000:.0f}ms" if elapsed < 1 else f"  ⚠ Cached fetch slow: {elapsed:.1f}s")
+    passed += 1
+
+    # Cleanup
+    import shutil
+    test_lib = Path(__file__).resolve().parent / "_test_fcst_lib"
+    if test_lib.exists():
+        shutil.rmtree(test_lib)
+
+    print(f"\n  RESULTS: {passed} passed, {failed} failed")
+    print("=" * 60)
+    return failed == 0
+
+
+if __name__ == "__main__":
+    success = test_integration()
+    sys.exit(0 if success else 1)

backend/data_manager/test_data_manager.py

@@ -0,0 +1,318 @@
+"""
+Data Management Module — Integration Test Suite
+
+Validates the full stack (DataStore → DataCatalog → DataManager) using
+synthetic data. No MT5 connection required.
+
+Run from project root:
+    python test_data_manager.py
+"""
+
+import sys
+import shutil
+import traceback
+from pathlib import Path
+from datetime import datetime, timedelta, timezone
+
+import pandas as pd
+import numpy as np
+
+# Ensure data_manager is importable
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from data_manager.data_store import DataStore, ValidationResult
+from data_manager.data_catalog import DataCatalog, CoverageResult
+from data_manager.data_manager import DataManager
+
+# ── Test infrastructure ──────────────────────────────────────────────────────
+TEST_DIR = Path(__file__).resolve().parent / "_test_library"
+PASS = 0
+FAIL = 0
+
+
+def setup():
+    """Clean test directory."""
+    if TEST_DIR.exists():
+        shutil.rmtree(TEST_DIR)
+    TEST_DIR.mkdir(parents=True)
+
+
+def teardown():
+    """Remove test directory."""
+    if TEST_DIR.exists():
+        shutil.rmtree(TEST_DIR)
+
+
+def check(name: str, condition: bool, detail: str = ""):
+    global PASS, FAIL
+    if condition:
+        PASS += 1
+        print(f"  ✓ {name}")
+    else:
+        FAIL += 1
+        print(f"  ✗ {name} — {detail}")
+
+
+def make_ohlcv(start: datetime, n_bars: int, freq_hours: float = 24.0) -> pd.DataFrame:
+    """Generate synthetic OHLCV data."""
+    dates = [start + timedelta(hours=freq_hours * i) for i in range(n_bars)]
+    dates = [d.replace(tzinfo=timezone.utc) for d in dates]
+    rng = np.random.default_rng(42)
+    close = 100.0 + np.cumsum(rng.normal(0, 1, n_bars))
+    df = pd.DataFrame({
+        "open":        close + rng.normal(0, 0.5, n_bars),
+        "high":        close + np.abs(rng.normal(0, 1, n_bars)),
+        "low":         close - np.abs(rng.normal(0, 1, n_bars)),
+        "close":       close,
+        "tick_volume":  rng.integers(100, 10000, n_bars),
+        "spread":       rng.integers(1, 20, n_bars),
+        "real_volume":  np.zeros(n_bars, dtype=int),
+    }, index=pd.DatetimeIndex(dates, name="time"))
+    return df
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+#  TEST 1 — DataStore basics
+# ═══════════════════════════════════════════════════════════════════════════════
+def test_datastore():
+    print("\n─── TEST 1: DataStore (Parquet read/write) ───")
+    store = DataStore(TEST_DIR / "store_test")
+
+    # Save
+    df = make_ohlcv(datetime(2024, 1, 1), 500)
+    store.save("EURUSD", "D1", df)
+    check("Save creates file", store.exists("EURUSD", "D1"))
+
+    # Load full
+    loaded = store.load("EURUSD", "D1")
+    check("Load returns correct rows", loaded is not None and len(loaded) == 500,
+          f"got {len(loaded) if loaded is not None else 'None'}")
+
+    # Load with date filter
+    mid = datetime(2024, 6, 1, tzinfo=timezone.utc)
+    filtered = store.load("EURUSD", "D1", date_from=mid)
+    check("Date filter works", filtered is not None and filtered.index.min() >= mid,
+          f"min={filtered.index.min() if filtered is not None else 'None'}")
+
+    # Append + dedup
+    overlap_start = df.index[-50]
+    new_data = make_ohlcv(overlap_start.replace(tzinfo=None), 100, freq_hours=24.0)
+    store.append("EURUSD", "D1", new_data)
+    appended = store.load("EURUSD", "D1")
+    check("Append deduplicates overlap", appended is not None and len(appended) < 600,
+          f"expected <600, got {len(appended) if appended is not None else 'None'}")
+    check("Append is sorted", appended.index.is_monotonic_increasing if appended is not None else False)
+
+    # Validate
+    result = store.validate("EURUSD", "D1")
+    check("Validation passes", result.valid, f"issues: {result.issues}")
+
+    # Checksum
+    cs = store.compute_checksum("EURUSD", "D1")
+    check("Checksum computed", cs is not None and len(cs) == 32)
+
+    # Non-existent
+    check("Non-existent returns None", store.load("FAKE", "D1") is None)
+
+    # Column filter
+    cols_only = store.load("EURUSD", "D1", columns=["close"])
+    check("Column filter works", cols_only is not None and list(cols_only.columns) == ["close"])
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+#  TEST 2 — DataCatalog
+# ═══════════════════════════════════════════════════════════════════════════════
+def test_datacatalog():
+    print("\n─── TEST 2: DataCatalog (SQLite metadata) ───")
+    db_path = TEST_DIR / "catalog_test" / "catalog.db"
+    cat = DataCatalog(db_path)
+
+    # Register
+    d_from = datetime(2024, 1, 1, tzinfo=timezone.utc)
+    d_to   = datetime(2024, 6, 1, tzinfo=timezone.utc)
+    cat.register_dataset("EURUSD", "D1", "ohlcv", "mt5", d_from, d_to, 120,
+                         "EURUSD/D1/ohlcv.parquet")
+    summary = cat.get_summary()
+    check("Register creates record", len(summary) == 1)
+    check("Symbol stored correctly", summary[0]["symbol"] == "EURUSD")
+
+    # Coverage — full hit
+    cov = cat.check_coverage("EURUSD", "D1", d_from, d_to)
+    check("Full coverage detected", cov.covered and len(cov.gaps) == 0,
+          f"gaps={cov.gaps}")
+
+    # Coverage — trailing gap
+    future = datetime(2024, 9, 1, tzinfo=timezone.utc)
+    cov2 = cat.check_coverage("EURUSD", "D1", d_from, future)
+    check("Trailing gap detected", cov2.needs_fetch and len(cov2.gaps) == 1,
+          f"gaps={cov2.gaps}")
+
+    # Coverage — leading gap
+    earlier = datetime(2023, 6, 1, tzinfo=timezone.utc)
+    cov3 = cat.check_coverage("EURUSD", "D1", earlier, d_to)
+    check("Leading gap detected", cov3.needs_fetch and len(cov3.gaps) == 1)
+
+    # Coverage — both gaps
+    cov4 = cat.check_coverage("EURUSD", "D1", earlier, future)
+    check("Both gaps detected", len(cov4.gaps) == 2, f"gaps={len(cov4.gaps)}")
+
+    # Coverage — unknown symbol
+    cov5 = cat.check_coverage("FAKE", "D1", d_from, d_to)
+    check("Unknown symbol = needs fetch", cov5.needs_fetch)
+
+    # Fetch log
+    cat.log_fetch("EURUSD", "D1", "mt5", d_from, d_to, 120, 350, "success")
+    history = cat.get_fetch_history("EURUSD")
+    check("Fetch log recorded", len(history) == 1)
+
+    # Upsert (update existing record)
+    d_to_new = datetime(2024, 9, 1, tzinfo=timezone.utc)
+    cat.register_dataset("EURUSD", "D1", "ohlcv", "mt5", d_from, d_to_new, 200,
+                         "EURUSD/D1/ohlcv.parquet")
+    summary2 = cat.get_summary()
+    check("Upsert updates (not duplicates)", len(summary2) == 1 and summary2[0]["row_count"] == 200)
+
+    cat.close()
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+#  TEST 3 — DataManager (full orchestration, mock fetcher)
+# ═══════════════════════════════════════════════════════════════════════════════
+def test_data_manager():
+    print("\n─── TEST 3: DataManager (orchestration with mock fetcher) ───")
+
+    dm = DataManager(library_root=TEST_DIR / "dm_test", cache_enabled=True)
+
+    # Inject a mock fetcher that returns synthetic data
+    class MockFetcher:
+        name = "mock"
+        call_count = 0
+        def fetch_bars(self, symbol, timeframe, date_from, date_to):
+            self.call_count += 1
+            # +1 to ensure we cover date_to inclusively
+            n_bars = max(1, (date_to - date_from).days + 1)
+            return make_ohlcv(date_from.replace(tzinfo=None), n_bars)
+        def fetch_bars_n(self, symbol, timeframe, n_bars):
+            self.call_count += 1
+            return make_ohlcv(datetime(2024, 1, 1), n_bars)
+        def is_available(self):
+            return True
+
+    mock = MockFetcher()
+    dm._fetcher = mock
+
+    # First call — should hit fetcher
+    d_from = datetime(2024, 1, 1, tzinfo=timezone.utc)
+    d_to   = datetime(2024, 6, 1, tzinfo=timezone.utc)
+    df1 = dm.get_bars("EURUSD", "D1", date_from=d_from, date_to=d_to)
+    check("First call returns data", df1 is not None and len(df1) > 0)
+    first_calls = mock.call_count
+
+    # Second identical call — cache covers bulk of range; may do small trailing fill
+    mock.call_count = 0
+    df2 = dm.get_bars("EURUSD", "D1", date_from=d_from, date_to=d_to)
+    check("Second call: at most 1 trailing gap fill", mock.call_count <= 1,
+          f"fetcher called {mock.call_count} times")
+    check("Cached data matches or grows", df2 is not None and len(df2) >= len(df1),
+          f"df2={len(df2) if df2 is not None else 0}, df1={len(df1)}")
+
+    # Third call — now cache fully covers range, zero fetches expected
+    mock.call_count = 0
+    df2b = dm.get_bars("EURUSD", "D1", date_from=d_from, date_to=d_to)
+    check("Third call uses cache (zero fetches)", mock.call_count == 0,
+          f"fetcher called {mock.call_count} times")
+
+    # Extend range — should only fetch the gap
+    mock.call_count = 0
+    d_to_ext = datetime(2024, 9, 1, tzinfo=timezone.utc)
+    df3 = dm.get_bars("EURUSD", "D1", date_from=d_from, date_to=d_to_ext)
+    check("Extended range triggers gap fetch", mock.call_count == 1,
+          f"expected 1 fetch, got {mock.call_count}")
+    check("Extended result is longer", df3 is not None and len(df3) > len(df1),
+          f"df3={len(df3) if df3 is not None else 0}, df1={len(df1)}")
+
+    # n_bars mode
+    mock.call_count = 0
+    df4 = dm.get_bars("XAUUSD", "M15", n_bars=500)
+    check("n_bars mode works", df4 is not None and len(df4) == 500)
+
+    # Catalog summary
+    summary = dm.get_catalog_summary()
+    check("Catalog has 2 datasets", len(summary) == 2,
+          f"got {len(summary)}")
+
+    # Forecaster-compatible output
+    df_fc = dm.get_bars_as_forecaster("EURUSD", "D1", date_from=d_from, date_to=d_to)
+    check("Forecaster compat: has 'Close'", df_fc is not None and "Close" in df_fc.columns)
+    check("Forecaster compat: tz-naive index",
+          df_fc is not None and df_fc.index.tz is None)
+
+    # Purge
+    dm.purge("XAUUSD", "M15")
+    check("Purge removes dataset", len(dm.get_catalog_summary()) == 1)
+
+    # Fetch history
+    history = dm.get_fetch_history()
+    check("Fetch history recorded", len(history) > 0)
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+#  TEST 4 — Path portability
+# ═══════════════════════════════════════════════════════════════════════════════
+def test_portability():
+    print("\n─── TEST 4: Path portability ───")
+    lib_root = TEST_DIR / "portable_test"
+    dm = DataManager(library_root=lib_root)
+
+    # All paths should be under lib_root
+    check("Library root exists", lib_root.exists())
+    check("Catalog in library root", (lib_root / "catalog.db").exists())
+    check("Temp dir in library root", (lib_root / "_temp").exists())
+
+    # No absolute paths in catalog file_path entries
+    class MockFetcher:
+        name = "mock"
+        def fetch_bars(self, s, tf, df, dt):
+            return make_ohlcv(df.replace(tzinfo=None), 10)
+        def fetch_bars_n(self, s, tf, n):
+            return make_ohlcv(datetime(2024, 1, 1), n)
+        def is_available(self):
+            return True
+
+    dm._fetcher = MockFetcher()
+    dm.get_bars("TEST", "D1", n_bars=10)
+
+    summary = dm.get_catalog_summary()
+    if summary:
+        fp = summary[0].get("file_path", "")
+        check("Catalog stores relative path", not fp.startswith("/") and not fp.startswith("C:"),
+              f"got: {fp}")
+    else:
+        check("Catalog stores relative path", False, "no summary entries")
+
+
+# ═══════════════════════════════════════════════════════════════════════════════
+#  RUN ALL
+# ═══════════════════════════════════════════════════════════════════════════════
+if __name__ == "__main__":
+    print("=" * 60)
+    print("  DATA MANAGEMENT MODULE — INTEGRATION TESTS")
+    print("=" * 60)
+
+    setup()
+    try:
+        test_datastore()
+        test_datacatalog()
+        test_data_manager()
+        test_portability()
+    except Exception as e:
+        print(f"\n  ✗ UNEXPECTED ERROR: {e}")
+        traceback.print_exc()
+        FAIL += 1
+    finally:
+        teardown()
+
+    print("\n" + "=" * 60)
+    print(f"  RESULTS: {PASS} passed, {FAIL} failed")
+    print("=" * 60)
+    sys.exit(1 if FAIL > 0 else 0)

backend/execution_routes.py

@@ -0,0 +1,312 @@
+# version: v3
+"""
+execution_routes.py — MT5 trade execution HTTP layer (v3).
+
+v1 was a stub (PRO-only). v2 exposes the provider's execution methods as
+HTTP endpoints so the consumer OrderTicket / ExposureStrip / quick-trade
+buttons can act on the user's live MT5 account.
+v3 adds the pending-order lifecycle: list / modify-price-or-SLTP / cancel.
+
+Endpoints:
+    GET    /api/positions                  — list open positions
+    POST   /api/orders                     — place a market/limit/stop order
+    POST   /api/positions/{ticket}/close   — close (full or partial) a position
+    PATCH  /api/positions/{ticket}         — modify SL / TP on an open position
+    GET    /api/orders/pending             — list resting pending orders           (v3)
+    PATCH  /api/orders/{ticket}            — modify pending-order price / SL / TP  (v3)
+    DELETE /api/orders/{ticket}            — cancel a pending order                (v3)
+
+All endpoints are thin wrappers over `provider.place_order` / `get_positions`
+/ `close_position` / `modify_position`. Results flow back as JSON.
+
+Consumer-safety notes:
+    - Every request goes through the cached MT5 provider — no fresh connect.
+    - provider None or disconnected → returns a structured error, not 500.
+    - Magic number defaults to 777 (Quantum Terminal tag) unless overridden.
+"""
+
+import logging
+from typing import Optional
+
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+from models import OrderRequest
+
+log = logging.getLogger("execution_routes")
+
+MK_MAGIC = 777  # Quantum Terminal tag on broker side for attribution / filtering
+
+
+class OrderBody(BaseModel):
+    ticker: str
+    direction: str                         # "BUY" | "SELL"
+    order_type: str = "MARKET"             # "MARKET" | "LIMIT" | "STOP"
+    lots: float
+    price: Optional[float] = None          # required for LIMIT/STOP
+    stop_loss: Optional[float] = None
+    take_profit: Optional[float] = None
+    comment: Optional[str] = "Quantum Terminal"
+    # Signal attribution (optional — logged only)
+    signal_id: Optional[str] = None
+    signal_type: Optional[str] = None
+    signal_confidence: Optional[float] = None
+
+
+class ClosePositionBody(BaseModel):
+    lots: Optional[float] = None  # None = full close
+
+
+class ModifyPositionBody(BaseModel):
+    stop_loss: Optional[float] = None
+    take_profit: Optional[float] = None
+
+
+class ModifyOrderBody(BaseModel):
+    """v3: Modify a pending order — any field left None is preserved server-side."""
+    price: Optional[float] = None
+    stop_loss: Optional[float] = None
+    take_profit: Optional[float] = None
+
+
+def _pending_to_dict(o) -> dict:
+    """Normalize a PendingOrder object to the JSON shape the frontend expects."""
+    def g(obj, *names, default=None):
+        for n in names:
+            v = getattr(obj, n, None)
+            if v is not None and v != 0:
+                return v
+        return default
+    return {
+        "ticket":      str(g(o, "ticket") or ""),
+        "ticker":      g(o, "ticker", "symbol", default=""),
+        "direction":   g(o, "direction", default=""),
+        "order_type":  g(o, "order_type", default=""),
+        "lots":        float(g(o, "lots", "volume", default=0) or 0),
+        "price":       float(g(o, "price", "price_open", default=0) or 0),
+        "stop_loss":   float(g(o, "stop_loss", "sl", default=0) or 0),
+        "take_profit": float(g(o, "take_profit", "tp", default=0) or 0),
+        "comment":     str(getattr(o, "comment", "") or ""),
+        "time_setup":  str(g(o, "time_setup", "time_open") or ""),
+    }
+
+
+def _position_to_dict(p) -> dict:
+    """Normalize a Position object to the JSON shape the frontend expects."""
+    def g(obj, *names, default=None):
+        for n in names:
+            v = getattr(obj, n, None)
+            if v is not None and v != 0:
+                return v
+        return default
+    return {
+        "ticket":       str(g(p, "ticket") or ""),
+        "ticker":       g(p, "ticker", "symbol", default=""),
+        "direction":    g(p, "direction", default=""),
+        "lots":         float(g(p, "lots", "volume", default=0) or 0),
+        "open_price":   float(g(p, "open_price", "price_open", default=0) or 0),
+        "current_price":float(g(p, "current_price", "price_current", default=0) or 0),
+        "stop_loss":    float(g(p, "stop_loss", "sl", default=0) or 0),
+        "take_profit":  float(g(p, "take_profit", "tp", default=0) or 0),
+        "profit":       float(getattr(p, "profit", 0) or 0),
+        "swap":         float(getattr(p, "swap", 0) or 0),
+        "commission":   float(getattr(p, "commission", 0) or 0),
+        "open_time":    str(g(p, "open_time") or getattr(p, "time_open", "") or ""),
+        "comment":      str(getattr(p, "comment", "") or ""),
+    }
+
+
+def create_execution_router(cfg_manager=None, app=None, broadcast_event=None) -> APIRouter:
+    router = APIRouter(tags=["execution"])
+
+    def _provider():
+        if cfg_manager is None:
+            return None
+        try:
+            return cfg_manager.get_provider("mt5_default")
+        except Exception:
+            return None
+
+    # ── GET /api/positions ─────────────────────────────────────────────
+    @router.get("/api/positions")
+    async def get_positions():
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            return {"positions": [], "mt5_connected": False}
+        try:
+            positions = await asyncio.to_thread(prov.get_positions)
+        except Exception as e:
+            log.warning(f"get_positions failed: {e}")
+            return {"positions": [], "mt5_connected": True, "error": str(e)}
+        return {
+            "positions": [_position_to_dict(p) for p in (positions or [])],
+            "mt5_connected": True,
+        }
+
+    # ── POST /api/orders ───────────────────────────────────────────────
+    @router.post("/api/orders")
+    async def place_order(body: OrderBody):
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            raise HTTPException(503, {"error": "mt5_disconnected",
+                                      "message": "MT5 is not connected. Connect in Settings → Providers."})
+
+        # Build the provider's OrderRequest. FlexModel accepts extra fields.
+        req = OrderRequest(
+            ticker=body.ticker.upper(),
+            direction=body.direction.upper(),
+            order_type=body.order_type.upper(),
+            lots=float(body.lots),
+            volume=float(body.lots),   # dual-name for safety
+            price=body.price or 0,
+            stop_loss=body.stop_loss,
+            take_profit=body.take_profit,
+            sl=float(body.stop_loss) if body.stop_loss else 0,
+            tp=float(body.take_profit) if body.take_profit else 0,
+            comment=body.comment or "Quantum Terminal",
+            magic=MK_MAGIC,
+        )
+
+        log.info(
+            f"ORDER {req.direction} {req.lots} {req.ticker} @ "
+            f"{body.price or 'market'} SL={body.stop_loss} TP={body.take_profit}"
+            + (f" signal={body.signal_id}" if body.signal_id else "")
+        )
+
+        try:
+            result = await asyncio.to_thread(prov.place_order, req)
+        except Exception as e:
+            log.error(f"place_order exception: {e}")
+            raise HTTPException(500, {"error": "place_order_failed", "message": str(e)})
+
+        # Normalize response — mt5_provider returns OrderResult (FlexModel).
+        ok = bool(getattr(result, "success", False))
+        payload = {
+            "success":  ok,
+            "order_id": str(getattr(result, "order_id", "") or ""),
+            "price":    float(getattr(result, "price", 0) or 0),
+            "ticker":   getattr(result, "ticker", req.ticker) or req.ticker,
+            "direction": getattr(result, "direction", req.direction) or req.direction,
+            "lots":     float(getattr(result, "lots", req.lots) or req.lots),
+        }
+        if not ok:
+            payload["error"] = str(
+                getattr(result, "error", None)
+                or getattr(result, "message", None)
+                or "Order rejected"
+            )
+        return payload
+
+    # ── POST /api/positions/{ticket}/close ─────────────────────────────
+    @router.post("/api/positions/{ticket}/close")
+    async def close_position(ticket: str, body: Optional[ClosePositionBody] = None):
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            raise HTTPException(503, {"error": "mt5_disconnected"})
+
+        lots = body.lots if body else None
+        log.info(f"CLOSE position {ticket} lots={lots or 'full'}")
+        try:
+            result = await asyncio.to_thread(prov.close_position, ticket, lots)
+        except Exception as e:
+            log.error(f"close_position exception: {e}")
+            raise HTTPException(500, {"error": "close_failed", "message": str(e)})
+
+        ok = bool(getattr(result, "success", False))
+        return {
+            "success": ok,
+            "ticket":  ticket,
+            "price":   float(getattr(result, "price", 0) or 0),
+            **({"error": str(getattr(result, "error", None) or "")} if not ok else {}),
+        }
+
+    # ── PATCH /api/positions/{ticket} ──────────────────────────────────
+    @router.patch("/api/positions/{ticket}")
+    async def modify_position(ticket: str, body: ModifyPositionBody):
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            raise HTTPException(503, {"error": "mt5_disconnected"})
+
+        if not hasattr(prov, "modify_position"):
+            raise HTTPException(501, {"error": "modify_unsupported"})
+
+        log.info(f"MODIFY position {ticket} SL={body.stop_loss} TP={body.take_profit}")
+        try:
+            result = await asyncio.to_thread(
+                prov.modify_position, ticket, body.stop_loss, body.take_profit
+            )
+        except Exception as e:
+            log.error(f"modify_position exception: {e}")
+            raise HTTPException(500, {"error": "modify_failed", "message": str(e)})
+
+        # modify_position returns a dict per mt5_provider
+        if isinstance(result, dict):
+            return {"success": bool(result.get("success", False)), **result}
+        return {"success": True}
+
+    # ── GET /api/orders/pending ────────────────────────────────── v3 ──
+    @router.get("/api/orders/pending")
+    async def list_pending_orders():
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            return {"orders": [], "mt5_connected": False}
+        try:
+            orders = await asyncio.to_thread(prov.get_pending_orders)
+        except Exception as e:
+            log.warning(f"get_pending_orders failed: {e}")
+            return {"orders": [], "mt5_connected": True, "error": str(e)}
+        return {
+            "orders": [_pending_to_dict(o) for o in (orders or [])],
+            "mt5_connected": True,
+        }
+
+    # ── PATCH /api/orders/{ticket} ─────────────────────────────── v3 ──
+    @router.patch("/api/orders/{ticket}")
+    async def modify_pending_order(ticket: str, body: ModifyOrderBody):
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            raise HTTPException(503, {"error": "mt5_disconnected"})
+        if not hasattr(prov, "modify_order"):
+            raise HTTPException(501, {"error": "modify_order_unsupported"})
+        log.info(f"MODIFY order {ticket} price={body.price} SL={body.stop_loss} TP={body.take_profit}")
+        try:
+            result = await asyncio.to_thread(
+                prov.modify_order, ticket, body.price, body.stop_loss, body.take_profit
+            )
+        except Exception as e:
+            log.error(f"modify_order exception: {e}")
+            raise HTTPException(500, {"error": "modify_order_failed", "message": str(e)})
+        ok = bool(getattr(result, "success", False))
+        payload = {"success": ok, "ticket": ticket}
+        if not ok:
+            payload["error"] = str(getattr(result, "error", None) or "Modify rejected")
+        return payload
+
+    # ── DELETE /api/orders/{ticket} ────────────────────────────── v3 ──
+    @router.delete("/api/orders/{ticket}")
+    async def cancel_pending_order(ticket: str):
+        import asyncio
+        prov = _provider()
+        if prov is None or not getattr(prov, "connected", False):
+            raise HTTPException(503, {"error": "mt5_disconnected"})
+        if not hasattr(prov, "cancel_order"):
+            raise HTTPException(501, {"error": "cancel_order_unsupported"})
+        log.info(f"CANCEL order {ticket}")
+        try:
+            result = await asyncio.to_thread(prov.cancel_order, ticket)
+        except Exception as e:
+            log.error(f"cancel_order exception: {e}")
+            raise HTTPException(500, {"error": "cancel_order_failed", "message": str(e)})
+        ok = bool(getattr(result, "success", False))
+        payload = {"success": ok, "ticket": ticket}
+        if not ok:
+            payload["error"] = str(getattr(result, "error", None) or "Cancel rejected")
+        return payload
+
+    return router

backend/freshness_routes.py

@@ -0,0 +1,10 @@
+"""
+freshness_routes.py — Stub for consumer terminal.
+PRO-only routes. Consumer does not use these endpoints.
+"""
+from fastapi import APIRouter
+
+
+def create_freshness_router(*args, **kwargs) -> APIRouter:
+    """Return empty router — PRO endpoints not available in consumer."""
+    return APIRouter(tags=["freshness"])

backend/launcher.py

@@ -0,0 +1,323 @@
+"""
+================================================================================
+Quantum Terminal Consumer — Launcher
+================================================================================
+Single entry point for the packaged .exe.
+Starts data server + serves React frontend + opens browser.
+Auto-creates consumer_config.ini on first run.
+
+Usage:
+    python launcher.py              # Dev mode
+    QuantumTerminal.exe                    # Packaged mode
+
+This module does NOT introduce any calculation triggers.
+================================================================================
+"""
+
+import sys
+import os
+import threading
+import time
+import webbrowser
+import shutil
+import logging
+from pathlib import Path
+
+log = logging.getLogger("mk.launcher")
+
+# — Determine base paths (works for both dev and PyInstaller) —
+if getattr(sys, 'frozen', False):
+    # Running as packaged .exe — everything is flat in _MEIPASS
+    BASE_DIR = Path(sys._MEIPASS)
+    EXE_DIR = Path(sys.executable).parent
+    BACKEND_DIR = BASE_DIR          # PyInstaller bundles flat, no backend/ subfolder
+    FRONTEND_DIR = BASE_DIR / "frontend"
+else:
+    # Running as script (dev mode)
+    BASE_DIR = Path(__file__).resolve().parent
+    EXE_DIR = BASE_DIR
+    BACKEND_DIR = BASE_DIR
+    FRONTEND_DIR = BASE_DIR.parent / "terminal_app" / "build"
+
+PORT = int(os.environ.get("MK_PORT", "8502"))   # v2: Electron passes MK_PORT
+
+# v2: AppData dir name (env-var driven; default "QuantumTerminal" preserves v1 path).
+APP_DIR = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal")
+
+# — Default config content (f-string so APP_DIR is substituted at module load) —
+DEFAULT_CONFIG = f"""\
+[server]
+# No external server needed for Quantum Terminal (open source)
+base_url = 
+
+[data]
+cache_dir = %APPDATA%\\{APP_DIR}\\cache
+stale_warning_hours = 48
+stale_error_hours = 72
+"""
+
+
+def ensure_config():
+    """Create consumer_config.ini on first run. User never touches this."""
+    # Check next to .exe first (packaged mode)
+    config_path = EXE_DIR / "consumer_config.ini"
+
+    if config_path.exists():
+        return config_path
+
+    # Also check AppData (fallback location)
+    appdata_config = _get_appdata_base() / "consumer_config.ini"
+    if appdata_config.exists():
+        return appdata_config
+
+    # Try to write next to .exe
+    try:
+        template = BASE_DIR / "consumer_config.ini.template"
+        if template.exists():
+            shutil.copy2(template, config_path)
+        else:
+            config_path.write_text(DEFAULT_CONFIG, encoding="utf-8")
+        print(f"  Config created: {config_path}")
+        return config_path
+    except PermissionError:
+        # Program Files is write-protected — use AppData instead
+        appdata_dir = _get_appdata_base()
+        appdata_dir.mkdir(parents=True, exist_ok=True)
+        appdata_config = appdata_dir / "consumer_config.ini"
+        try:
+            template = BASE_DIR / "consumer_config.ini.template"
+            if template.exists():
+                shutil.copy2(template, appdata_config)
+            else:
+                appdata_config.write_text(DEFAULT_CONFIG, encoding="utf-8")
+            print(f"  Config created (AppData): {appdata_config}")
+            return appdata_config
+        except Exception as e:
+            print(f"  [WARN] Could not create config: {e}")
+            return appdata_config
+
+
+def _get_appdata_base() -> Path:
+    """Get AppData\\Roaming\\<APP_DIR> path. APP_DIR comes from MK_APP_DIR_NAME env var."""
+    import platform
+    if platform.system() == "Windows":
+        appdata = os.environ.get("APPDATA", "")
+        if appdata:
+            return Path(appdata) / APP_DIR
+    return Path.home() / f".{APP_DIR.lower()}"
+
+
+def ensure_appdata():
+    """Create AppData directories for auth cache and data cache."""
+    appdata = os.environ.get("APPDATA", "")
+    if appdata:
+        base = Path(appdata) / APP_DIR
+    else:
+        base = Path.home() / "AppData" / "Roaming" / APP_DIR
+
+    for subdir in ["", "cache"]:
+        d = base / subdir if subdir else base
+        d.mkdir(parents=True, exist_ok=True)
+
+    return base
+
+
+def open_browser():
+    """Wait for server to start, then open browser."""
+    for _ in range(30):  # Wait up to 30 seconds
+        time.sleep(1)
+        try:
+            import urllib.request
+            urllib.request.urlopen(f"http://127.0.0.1:{PORT}/api/health", timeout=2)
+            break
+        except Exception:
+            continue
+
+    webbrowser.open(f"http://127.0.0.1:{PORT}")
+
+
+def main():
+    # v2 Phase D: persistent rotating log file under %APPDATA%\<APP_DIR>\logs\.
+    #   Console behavior unchanged; file logging is additive.
+    try:
+        from logging_setup import setup_logging
+        _log_file = setup_logging()
+    except Exception as _e:
+        _log_file = None
+        print(f"[launcher] logging setup skipped: {_e}")
+
+    print()
+    print("=" * 50)
+    print("  Quantum Terminal Consumer Terminal")
+    print("=" * 50)
+    if _log_file:
+        print(f"  Logs   : {_log_file}")
+
+    # Step 1: Ensure config exists
+    config_path = ensure_config()
+    print(f"  Config : {config_path}")
+
+    # Step 2: Ensure AppData dirs exist
+    appdata = ensure_appdata()
+    print(f"  AppData: {appdata}")
+
+    # Step 3: Setup Python path
+    sys.path.insert(0, str(BACKEND_DIR))
+    os.chdir(str(BACKEND_DIR))
+
+    # Also add EXE_DIR so consumer_config.ini is found
+    if str(EXE_DIR) not in sys.path:
+        sys.path.insert(0, str(EXE_DIR))
+
+    # Add config directory to path so modules find it
+    config_dir = str(config_path.parent)
+    if config_dir not in sys.path:
+        sys.path.insert(0, config_dir)
+
+    # Copy config next to backend modules if packaged
+    if getattr(sys, 'frozen', False):
+        backend_config = BACKEND_DIR / "consumer_config.ini"
+        if not backend_config.exists() and config_path.exists():
+            try:
+                shutil.copy2(config_path, backend_config)
+            except PermissionError:
+                pass  # Read-only _internal — config found via sys.path instead
+
+    print(f"  Server : http://127.0.0.1:{PORT}")
+    print(f"  Press Ctrl+C to stop")
+    print("=" * 50)
+    print()
+
+    # Step 4: Open browser in background — skipped when running under Electron.
+    if not os.environ.get("MK_NO_BROWSER"):
+        threading.Thread(target=open_browser, daemon=True).start()
+
+    # Step 5: Import and start server
+    from data_server import app
+
+    # Serve React build as static files (production mode)
+    # IMPORTANT: Use StaticFiles mount (not catch-all routes) so API routes
+    # always take priority. The catch-all @app.get("/{path}") pattern causes
+    # 405 errors on PATCH/POST to /api/* endpoints.
+    if FRONTEND_DIR.exists() and (FRONTEND_DIR / "index.html").exists():
+        from fastapi.staticfiles import StaticFiles
+        from fastapi.responses import FileResponse
+        from starlette.middleware.base import BaseHTTPMiddleware
+        from starlette.responses import Response as StarletteResponse
+
+        # Helper: serve index.html with no-cache headers so browser
+        # always fetches the latest after an update. JS/CSS files have
+        # content hashes in their filenames so they cache safely.
+        def _serve_index():
+            return FileResponse(
+                str(FRONTEND_DIR / "index.html"),
+                headers={
+                    "Cache-Control": "no-cache, no-store, must-revalidate",
+                    "Pragma": "no-cache",
+                    "Expires": "0",
+                },
+            )
+
+        # Serve /static/* assets (JS, CSS, images)
+        app.mount("/static", StaticFiles(directory=str(FRONTEND_DIR / "static")), name="static")
+
+        # SPA middleware: for non-API, non-static GET requests that don't match
+        # a file, serve index.html. This replaces the old @app.get("/{full_path:path}")
+        # catch-all which caused 405 errors on API PATCH/POST requests.
+        class SPAMiddleware(BaseHTTPMiddleware):
+            async def dispatch(self, request, call_next):
+                response = await call_next(request)
+
+                # If the API returned 404 and this is a GET request
+                # that doesn't target /api/ or /ws/, serve index.html
+                # (React client-side routing)
+                if (
+                    response.status_code == 404
+                    and request.method == "GET"
+                    and not request.url.path.startswith("/api/")
+                    and not request.url.path.startswith("/ws/")
+                    and not request.url.path.startswith("/static/")
+                ):
+                    # Check if it's a real file in the build directory
+                    file_path = FRONTEND_DIR / request.url.path.lstrip("/")
+                    if file_path.exists() and file_path.is_file():
+                        return FileResponse(str(file_path))
+                    # Otherwise serve index.html for React Router
+                    return _serve_index()
+
+                return response
+
+        app.add_middleware(SPAMiddleware)
+
+        # Root route — serve index.html
+        @app.get("/", include_in_schema=False)
+        async def serve_root():
+            return _serve_index()
+
+        # Favicon
+        favicon_path = FRONTEND_DIR / "favicon.ico"
+        if favicon_path.exists():
+            @app.get("/favicon.ico", include_in_schema=False)
+            async def serve_favicon():
+                return FileResponse(str(favicon_path))
+
+        print("  Frontend: serving React build")
+    else:
+        print("  Frontend: not found — use npm start for dev mode")
+
+    # v2 Phase E: graceful shutdown.
+    #   Electron sends SIGTERM (Windows: terminates the process group) when
+    #   the user closes the window. uvicorn handles SIGTERM by setting its
+    #   `should_exit` flag and draining in-flight requests. We additionally
+    #   register a Python-level signal handler that flushes logs and lets
+    #   any pending sync_timestamps write finish before exit. SIGINT (Ctrl+C
+    #   in dev) goes through the same path.
+    import uvicorn
+    import signal
+
+    config = uvicorn.Config(
+        app=app,
+        host="127.0.0.1",
+        port=PORT,
+        log_level="info",
+        # Give in-flight HTTP requests up to 5s to finish before forcing exit.
+        timeout_graceful_shutdown=5,
+    )
+    server = uvicorn.Server(config)
+
+    def _on_signal(signum, frame):
+        log.info(f"Received signal {signum} — initiating graceful shutdown")
+        server.should_exit = True
+
+    for sig in (signal.SIGINT, signal.SIGTERM):
+        try:
+            signal.signal(sig, _on_signal)
+        except Exception:
+            # Some platforms don't allow custom handlers in non-main threads.
+            pass
+
+    try:
+        server.run()
+    finally:
+        # Final flush — make sure log records on disk are complete and
+        # any in-memory state (sync timestamps) has been persisted.
+        try:
+            from data_sync_client import get_sync_client as _gsc
+            _sc = _gsc()
+            try:
+                # Best-effort: persist sync timestamps if anything was changed
+                # mid-request when the signal arrived.
+                from data_sync_client import _save_sync_timestamps
+                _save_sync_timestamps(_sc._sync_timestamps)
+            except Exception:
+                pass
+        except Exception:
+            pass
+        try:
+            logging.shutdown()
+        except Exception:
+            pass
+
+
+if __name__ == "__main__":
+    main()

backend/logging_setup.py

@@ -0,0 +1,91 @@
+"""
+================================================================================
+Quantum Terminal Consumer v2 — Logging setup
+================================================================================
+Centralized logging configuration. Adds a daily-rotating file handler so logs
+survive across runs and can be attached to support tickets, while keeping the
+console output identical to v1.
+
+Log location: %APPDATA%\\<APP_DIR>\\logs\\backend-YYYY-MM-DD.log
+Rotation:     daily at midnight, 14 days retained.
+Format:       same as console — `YYYY-MM-DD HH:MM:SS [logger.name] LEVEL: msg`
+
+Call setup_logging() ONCE at process startup — launcher.py does this.
+================================================================================
+"""
+from __future__ import annotations
+
+import logging
+import logging.handlers
+import os
+import sys
+from pathlib import Path
+
+# Match the APP_DIR convention used elsewhere (license_client / data_sync_client)
+APP_DIR = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal")
+
+_LOG_FORMAT = "%(asctime)s [%(name)s] %(levelname)s: %(message)s"
+_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
+_RETENTION_DAYS = 14
+_FILE_MAX_BYTES = 50 * 1024 * 1024  # 50 MB hard cap per file as a backstop
+
+
+def _appdata_logs_dir() -> Path:
+    """Resolve %APPDATA%\\<APP_DIR>\\logs\\ (or ~/.<app_dir>/logs/ off-Windows)."""
+    appdata = os.environ.get("APPDATA", "")
+    if appdata:
+        return Path(appdata) / APP_DIR / "logs"
+    return Path.home() / f".{APP_DIR.lower()}" / "logs"
+
+
+def setup_logging(level: int = logging.INFO) -> Path:
+    """Configure root logging once. Returns the log file path so callers can
+    print it on startup (`launcher.py` does)."""
+    log_dir = _appdata_logs_dir()
+    log_dir.mkdir(parents=True, exist_ok=True)
+    log_file = log_dir / "backend.log"
+
+    root = logging.getLogger()
+    root.setLevel(level)
+
+    # If already configured (e.g., re-import in tests), don't double-attach.
+    if any(getattr(h, "_mk_managed", False) for h in root.handlers):
+        return log_file
+
+    fmt = logging.Formatter(_LOG_FORMAT, datefmt=_DATE_FORMAT)
+
+    # Console handler — stderr keeps the existing capture path used by Electron.
+    ch = logging.StreamHandler(stream=sys.stderr)
+    ch.setLevel(level)
+    ch.setFormatter(fmt)
+    ch._mk_managed = True
+    root.addHandler(ch)
+
+    # Daily rotating file handler. Use TimedRotatingFileHandler (rolls at
+    # midnight) with 14 backups. Hard size cap as a backstop in case of
+    # log floods.
+    try:
+        fh = logging.handlers.TimedRotatingFileHandler(
+            filename=str(log_file),
+            when="midnight",
+            interval=1,
+            backupCount=_RETENTION_DAYS,
+            encoding="utf-8",
+            delay=True,  # don't open until first emit (avoids a stale empty file)
+        )
+        fh.setLevel(level)
+        fh.setFormatter(fmt)
+        fh._mk_managed = True
+        root.addHandler(fh)
+    except Exception as e:
+        # File logging is a nice-to-have — don't kill startup if disk is read-only.
+        print(f"[logging] file handler unavailable: {e}", file=sys.stderr)
+
+    # uvicorn / fastapi loggers tend to set up their own handlers; let them
+    # bubble up to root by stripping their handlers and unsetting `propagate=False`.
+    for noisy in ("uvicorn", "uvicorn.error", "uvicorn.access", "fastapi"):
+        lg = logging.getLogger(noisy)
+        lg.handlers.clear()
+        lg.propagate = True
+
+    return log_file

backend/macro_data.py

@@ -0,0 +1,304 @@
+"""
+macro_data.py — Macro Rotation Data Provider
+Quantum Terminal Analytical Terminal
+
+Fetches sector ETF and country ETF performance data from yfinance.
+Computes 1-week and 4-week percentage changes.
+Results cached with configurable TTL to avoid API hammering.
+
+Endpoints served:
+    GET /api/macro/sectors   → sector ETF performance
+    GET /api/macro/countries → country ETF performance
+"""
+
+import logging
+import time
+from typing import Dict, List, Optional
+
+log = logging.getLogger("mk.macro_data")
+
+# ════════════════════════════════════════════════════════
+# ETF UNIVERSE — no hardcoded assets, but ETFs are
+# benchmark instruments (not part of trading universe)
+# ════════════════════════════════════════════════════════
+
+SECTOR_ETFS = {
+    "XLE":  {"name": "Energy",                  "color": "#e67e22"},
+    "XLF":  {"name": "Financials",              "color": "#3498db"},
+    "XLK":  {"name": "Technology",              "color": "#9b59b6"},
+    "XLRE": {"name": "Real Estate",             "color": "#1abc9c"},
+    "XLU":  {"name": "Utilities",               "color": "#f39c12"},
+    "XLV":  {"name": "Healthcare",              "color": "#2ecc71"},
+    "XLB":  {"name": "Materials",               "color": "#e74c3c"},
+    "XLI":  {"name": "Industrials",             "color": "#34495e"},
+    "XLC":  {"name": "Communication",           "color": "#e84393"},
+    "XLY":  {"name": "Consumer Discretionary",  "color": "#00cec9"},
+    "XLP":  {"name": "Consumer Staples",        "color": "#fdcb6e"},
+}
+
+COUNTRY_ETFS = {
+    "SPY":  {"name": "United States",   "code": "US"},
+    "EWJ":  {"name": "Japan",           "code": "JP"},
+    "FXI":  {"name": "China",           "code": "CN"},
+    "EWG":  {"name": "Germany",         "code": "DE"},
+    "EWU":  {"name": "United Kingdom",  "code": "GB"},
+    "EWZ":  {"name": "Brazil",          "code": "BR"},
+    "EWA":  {"name": "Australia",       "code": "AU"},
+    "EWC":  {"name": "Canada",          "code": "CA"},
+    "EWY":  {"name": "South Korea",     "code": "KR"},
+    "EWT":  {"name": "Taiwan",          "code": "TW"},
+    "INDA": {"name": "India",           "code": "IN"},
+    "EWQ":  {"name": "France",          "code": "FR"},
+    "EWI":  {"name": "Italy",           "code": "IT"},
+    "EWP":  {"name": "Spain",           "code": "ES"},
+    "EWW":  {"name": "Mexico",          "code": "MX"},
+    "EZA":  {"name": "South Africa",    "code": "ZA"},
+    "TUR":  {"name": "Turkey",          "code": "TR"},
+    "KSA":  {"name": "Saudi Arabia",    "code": "SA"},
+    "EWS":  {"name": "Singapore",       "code": "SG"},
+    "EWH":  {"name": "Hong Kong",       "code": "HK"},
+    "EWM":  {"name": "Malaysia",        "code": "MY"},
+    "EWN":  {"name": "Netherlands",     "code": "NL"},
+    "EWD":  {"name": "Sweden",          "code": "SE"},
+    "EWL":  {"name": "Switzerland",     "code": "CH"},
+    "THD":  {"name": "Thailand",        "code": "TH"},
+}
+
+
+# ════════════════════════════════════════════════════════
+# CACHE
+# ════════════════════════════════════════════════════════
+
+_cache: Dict[str, dict] = {}
+CACHE_TTL_SECONDS = 3600  # 1 hour
+
+
+def _is_cache_valid(key: str) -> bool:
+    if key not in _cache:
+        return False
+    return (time.time() - _cache[key]["ts"]) < CACHE_TTL_SECONDS
+
+
+# ════════════════════════════════════════════════════════
+# DATA FETCH
+# ════════════════════════════════════════════════════════
+
+def _fetch_etf_changes(tickers: List[str], period: str = "2mo") -> Dict[str, dict]:
+    """
+    Fetch OHLCV for a list of ETF tickers and compute
+    1-week and 4-week percentage changes from latest close.
+
+    Returns dict keyed by ticker:
+        {ticker: {"close": float, "chg_1w": float, "chg_4w": float}}
+    """
+    try:
+        import yfinance as yf
+    except ImportError:
+        log.warning("yfinance not installed — macro data unavailable")
+        return {}
+
+    results = {}
+    try:
+        # Batch download — single API call for all tickers
+        data = yf.download(tickers, period=period, interval="1d",
+                           auto_adjust=True, progress=False, threads=True)
+
+        if data.empty:
+            log.warning("yfinance returned empty data for macro ETFs")
+            return {}
+
+        close = data["Close"] if "Close" in data.columns else data.get("close")
+        if close is None or close.empty:
+            log.warning("No Close column in yfinance macro data")
+            return {}
+
+        # Handle single ticker case (returns Series, not DataFrame)
+        if isinstance(close, type(data)):
+            pass  # already DataFrame
+        else:
+            # Single ticker returns Series — unlikely but handle
+            close = close.to_frame(name=tickers[0])
+
+        for ticker in tickers:
+            if ticker not in close.columns:
+                continue
+
+            series = close[ticker].dropna()
+            if len(series) < 2:
+                continue
+
+            latest = float(series.iloc[-1])
+
+            # 1-week change (5 trading days back)
+            idx_1w = min(5, len(series) - 1)
+            close_1w = float(series.iloc[-1 - idx_1w])
+            chg_1w = ((latest - close_1w) / close_1w) * 100 if close_1w != 0 else 0.0
+
+            # 4-week change (20 trading days back)
+            idx_4w = min(20, len(series) - 1)
+            close_4w = float(series.iloc[-1 - idx_4w])
+            chg_4w = ((latest - close_4w) / close_4w) * 100 if close_4w != 0 else 0.0
+
+            results[ticker] = {
+                "close": round(latest, 2),
+                "chg_1w": round(chg_1w, 2),
+                "chg_4w": round(chg_4w, 2),
+            }
+
+    except Exception as e:
+        log.warning(f"yfinance macro fetch failed: {e}")
+
+    return results
+
+
+# ════════════════════════════════════════════════════════
+# PUBLIC API
+# ════════════════════════════════════════════════════════
+
+def get_sector_data(force_refresh: bool = False) -> dict:
+    """
+    Returns sector performance data.
+
+    Response shape:
+    {
+        "sectors": [
+            {"ticker": "XLE", "name": "Energy", "color": "#e67e22",
+             "close": 88.5, "chg_1w": 2.3, "chg_4w": -1.1},
+            ...
+        ],
+        "top5_1w": [...],
+        "bottom5_1w": [...],
+        "top5_4w": [...],
+        "bottom5_4w": [...],
+        "updated_at": 1700000000.0
+    }
+    """
+    cache_key = "sectors"
+
+    if not force_refresh and _is_cache_valid(cache_key):
+        return _cache[cache_key]["data"]
+
+    tickers = list(SECTOR_ETFS.keys())
+    raw = _fetch_etf_changes(tickers)
+
+    sectors = []
+    for ticker, meta in SECTOR_ETFS.items():
+        perf = raw.get(ticker, {})
+        sectors.append({
+            "ticker": ticker,
+            "name": meta["name"],
+            "color": meta["color"],
+            "close": perf.get("close", 0),
+            "chg_1w": perf.get("chg_1w", 0),
+            "chg_4w": perf.get("chg_4w", 0),
+        })
+
+    # Sort helpers
+    sorted_1w = sorted(sectors, key=lambda s: s["chg_1w"], reverse=True)
+    sorted_4w = sorted(sectors, key=lambda s: s["chg_4w"], reverse=True)
+
+    result = {
+        "sectors": sectors,
+        "top5_1w": sorted_1w[:5],
+        "bottom5_1w": sorted_1w[-5:],
+        "top5_4w": sorted_4w[:5],
+        "bottom5_4w": sorted_4w[-5:],
+        "updated_at": time.time(),
+    }
+
+    _cache[cache_key] = {"data": result, "ts": time.time()}
+    log.info(f"Sector data refreshed — {len(sectors)} sectors loaded")
+    return result
+
+
+def get_country_data(force_refresh: bool = False) -> dict:
+    """
+    Returns country ETF performance data.
+
+    Response shape:
+    {
+        "countries": [
+            {"ticker": "SPY", "name": "United States", "code": "US",
+             "close": 440.5, "chg_1w": 1.2, "chg_4w": 3.5},
+            ...
+        ],
+        "top5_1w": [...],
+        "bottom5_1w": [...],
+        "top5_4w": [...],
+        "bottom5_4w": [...],
+        "updated_at": 1700000000.0
+    }
+    """
+    cache_key = "countries"
+
+    if not force_refresh and _is_cache_valid(cache_key):
+        return _cache[cache_key]["data"]
+
+    tickers = list(COUNTRY_ETFS.keys())
+    raw = _fetch_etf_changes(tickers)
+
+    countries = []
+    for ticker, meta in COUNTRY_ETFS.items():
+        perf = raw.get(ticker, {})
+        countries.append({
+            "ticker": ticker,
+            "name": meta["name"],
+            "code": meta["code"],
+            "close": perf.get("close", 0),
+            "chg_1w": perf.get("chg_1w", 0),
+            "chg_4w": perf.get("chg_4w", 0),
+        })
+
+    sorted_1w = sorted(countries, key=lambda c: c["chg_1w"], reverse=True)
+    sorted_4w = sorted(countries, key=lambda c: c["chg_4w"], reverse=True)
+
+    result = {
+        "countries": countries,
+        "top5_1w": sorted_1w[:5],
+        "bottom5_1w": sorted_1w[-5:],
+        "top5_4w": sorted_4w[:5],
+        "bottom5_4w": sorted_4w[-5:],
+        "updated_at": time.time(),
+    }
+
+    _cache[cache_key] = {"data": result, "ts": time.time()}
+    log.info(f"Country data refreshed — {len(countries)} countries loaded")
+    return result
+
+
+# ════════════════════════════════════════════════════════
+# CLI DIAGNOSTIC
+# ════════════════════════════════════════════════════════
+
+if __name__ == "__main__":
+    logging.basicConfig(level=logging.INFO,
+                        format="%(asctime)s [%(name)s] %(message)s")
+
+    print("=" * 60)
+    print("MACRO DATA — SECTOR ETFS")
+    print("=" * 60)
+    sd = get_sector_data(force_refresh=True)
+    for s in sd["sectors"]:
+        arrow_1w = "▲" if s["chg_1w"] >= 0 else "▼"
+        arrow_4w = "▲" if s["chg_4w"] >= 0 else "▼"
+        print(f"  {s['ticker']:5s}  {s['name']:28s}  "
+              f"1W: {arrow_1w} {s['chg_1w']:+6.2f}%   "
+              f"4W: {arrow_4w} {s['chg_4w']:+6.2f}%")
+
+    print(f"\n  TOP 5 (1W):    {', '.join(s['name'] for s in sd['top5_1w'])}")
+    print(f"  BOTTOM 5 (1W): {', '.join(s['name'] for s in sd['bottom5_1w'])}")
+
+    print("\n" + "=" * 60)
+    print("MACRO DATA — COUNTRY ETFS")
+    print("=" * 60)
+    cd = get_country_data(force_refresh=True)
+    for c in cd["countries"]:
+        arrow_1w = "▲" if c["chg_1w"] >= 0 else "▼"
+        arrow_4w = "▲" if c["chg_4w"] >= 0 else "▼"
+        print(f"  {c['ticker']:5s}  {c['code']:3s}  {c['name']:20s}  "
+              f"1W: {arrow_1w} {c['chg_1w']:+6.2f}%   "
+              f"4W: {arrow_4w} {c['chg_4w']:+6.2f}%")
+
+    print(f"\n  TOP 5 (1W):    {', '.join(c['name'] for c in cd['top5_1w'])}")
+    print(f"  BOTTOM 5 (1W): {', '.join(c['name'] for c in cd['bottom5_1w'])}")
+    print("\nDONE")

backend/models.py

@@ -0,0 +1,178 @@
+"""
+models.py — Data models for consumer terminal.
+Provides TickData, BarData, etc. that providers and data_server import.
+
+All classes accept and silently ignore unknown keyword arguments
+so PRO provider code that passes extra fields won't crash.
+"""
+from typing import Optional, Dict, Any
+from datetime import datetime
+
+
+def _make_flex(cls):
+    """Decorator: wraps __init__ to silently drop unknown kwargs."""
+    orig = cls.__init__
+    import inspect
+    params = set(inspect.signature(orig).parameters.keys()) - {'self'}
+
+    def flex_init(self, *args, **kwargs):
+        valid = {k: v for k, v in kwargs.items() if k in params}
+        orig(self, *args, **valid)
+        # Store extras as attributes
+        for k, v in kwargs.items():
+            if k not in params:
+                setattr(self, k, v)
+
+    cls.__init__ = flex_init
+
+    # Add to_dict if not present
+    if not hasattr(cls, 'to_dict'):
+        def to_dict(self):
+            d = {}
+            for k in params:
+                d[k] = getattr(self, k, None)
+            for k, v in self.__dict__.items():
+                if k not in d:
+                    d[k] = v
+            return d
+        cls.to_dict = to_dict
+
+    return cls
+
+
+from dataclasses import dataclass
+
+
+@_make_flex
+@dataclass
+class TickData:
+    ticker: str = ""
+    symbol: str = ""
+    bid: float = 0.0
+    ask: float = 0.0
+    last: float = 0.0
+    volume: float = 0.0
+    time: Optional[datetime] = None
+    spread: float = 0.0
+    time_msc: int = 0
+    flags: int = 0
+    volume_real: float = 0.0
+
+
+@_make_flex
+@dataclass
+class BarData:
+    ticker: str = ""
+    symbol: str = ""
+    timeframe: str = ""
+    time: Optional[datetime] = None
+    open: float = 0.0
+    high: float = 0.0
+    low: float = 0.0
+    close: float = 0.0
+    volume: float = 0.0
+    tick_volume: float = 0.0
+    spread: int = 0
+    real_volume: float = 0.0
+
+
+@_make_flex
+@dataclass
+class AccountInfo:
+    balance: float = 0.0
+    equity: float = 0.0
+    margin: float = 0.0
+    free_margin: float = 0.0
+    currency: str = "USD"
+    leverage: int = 100
+    server: str = ""
+    login: int = 0
+    name: str = ""
+    company: str = ""
+    profit: float = 0.0
+    margin_level: float = 0.0
+
+
+@_make_flex
+@dataclass
+class SymbolInfo:
+    ticker: str = ""
+    symbol: str = ""
+    description: str = ""
+    point: float = 0.0
+    digits: int = 5
+    tick_size: float = 0.0
+    tick_value: float = 0.0
+    volume_min: float = 0.01
+    volume_max: float = 100.0
+    volume_step: float = 0.01
+    trade_contract_size: float = 100000.0
+    currency_base: str = ""
+    currency_profit: str = ""
+    currency_margin: str = ""
+    spread: int = 0
+    trade_mode: int = 0
+    trade_stops_level: int = 0
+    swap_long: float = 0.0
+    swap_short: float = 0.0
+
+
+@_make_flex
+@dataclass
+class OrderRequest:
+    symbol: str = ""
+    ticker: str = ""
+    direction: str = ""
+    volume: float = 0.01
+    price: float = 0.0
+    sl: float = 0.0
+    tp: float = 0.0
+    comment: str = ""
+    order_type: str = "market"
+
+
+@_make_flex
+@dataclass
+class OrderResult:
+    success: bool = False
+    order_id: int = 0
+    message: str = ""
+    price: float = 0.0
+    volume: float = 0.0
+    retcode: int = 0
+
+
+@_make_flex
+@dataclass
+class Position:
+    ticket: int = 0
+    symbol: str = ""
+    ticker: str = ""
+    direction: str = ""
+    volume: float = 0.0
+    price_open: float = 0.0
+    price_current: float = 0.0
+    sl: float = 0.0
+    tp: float = 0.0
+    profit: float = 0.0
+    comment: str = ""
+    time_open: Optional[datetime] = None
+    swap: float = 0.0
+    commission: float = 0.0
+
+
+@_make_flex
+@dataclass
+class PendingOrder:
+    """A resting (unfilled) order — LIMIT or STOP — on the broker."""
+    ticket: int = 0
+    symbol: str = ""
+    ticker: str = ""
+    direction: str = ""          # "BUY" | "SELL"
+    order_type: str = ""         # "LIMIT" | "STOP"
+    volume: float = 0.0
+    price_open: float = 0.0
+    sl: float = 0.0
+    tp: float = 0.0
+    comment: str = ""
+    time_setup: Optional[datetime] = None

backend/mt5_routes.py

@@ -0,0 +1,346 @@
+# version: v6
+"""
+mt5_routes.py — MT5 Connection Management API
+================================================
+Endpoints for the consumer Settings → Providers panel.
+
+    GET    /api/mt5/config      → read MT5 connection settings
+    PATCH  /api/mt5/config      → save MT5 connection settings
+    POST   /api/mt5/connect     → connect to MT5 with current config
+    POST   /api/mt5/disconnect  → disconnect from MT5
+
+Settings are stored in user_config.json under providers.accounts.mt5_default.
+This module does NOT introduce any calculation triggers.
+
+v2 — PATCH /config now disconnects and reinitializes the MT5 provider after
+     saving, so the next /connect picks up the new terminal_path / server /
+     login immediately. Previously the provider cached the old path in memory
+     and stale connects kept firing until the backend was restarted — e.g.
+     user couldn't revert an invalid GoDo path back to auto-detect.
+
+v3 — Added `use_tick_data` to the GET/PATCH config payload. When True, the
+     backend tick loop polls MT5 every 0.2s instead of 1.0s, giving the chart
+     a live MT5-style "tick feel" on the last candle. Default OFF. Toggle
+     surfaces in Settings → PROVIDERS.
+
+v4 — PATCH /config no longer disconnects + reinits the MT5 provider on every
+     change. Reinit now triggers ONLY when a connection-relevant field
+     (connection_method / terminal_path / server / login / password) is
+     touched. Flipping `use_tick_data` (and any other future cosmetic field)
+     leaves the live connection alone. Without this, toggling USE TICK DATA
+     in the UI caused a ~5s outage with a flood of 503 responses while the
+     reconnect loop healed the connection.
+
+v5 — POST /browse-terminal opens a native Windows file-picker so users with
+     multiple MT5 installations can select the correct terminal64.exe without
+     manually typing (or pasting) a long path. Runs tkinter.filedialog on a
+     worker thread, returns the chosen absolute path.
+"""
+
+import logging
+from fastapi import APIRouter, Request, HTTPException
+
+log = logging.getLogger("mt5_routes")
+
+
+def create_mt5_router(cfg_manager, app) -> APIRouter:
+    router = APIRouter(prefix="/api/mt5", tags=["mt5"])
+
+    def _get_mt5_config() -> dict:
+        """Read MT5 config from user_config providers section."""
+        config = cfg_manager.get_config()
+        providers = config.get("providers", {})
+        accounts = providers.get("accounts", {})
+        mt5_acct = accounts.get("mt5_default", {})
+        return {
+            "connection_method": mt5_acct.get("connection_method", "path"),
+            "terminal_path": mt5_acct.get("terminal_path") or "",
+            "server": mt5_acct.get("server", ""),
+            "login": mt5_acct.get("login", ""),
+            "password": "",  # Never send password back
+            "enabled": mt5_acct.get("enabled", True),
+            # v3: USE TICK DATA — when ON, tick loop polls MT5 5×/sec instead
+            # of 1×/sec. Default OFF; user opts in via Settings → PROVIDERS.
+            "use_tick_data": bool(mt5_acct.get("use_tick_data", False)),
+        }
+
+    # ── GET /api/mt5/config ──
+    @router.get("/config")
+    async def get_mt5_config():
+        return _get_mt5_config()
+
+    # ── PATCH /api/mt5/config ──
+    @router.patch("/config")
+    async def patch_mt5_config(request: Request):
+        import asyncio
+
+        try:
+            patch = await request.json()
+        except Exception:
+            raise HTTPException(400, "Invalid JSON")
+
+        # Build the nested update for providers.accounts.mt5_default
+        mt5_update = {}
+        for key in ["connection_method", "terminal_path", "server", "login", "password",
+                    "use_tick_data"]:  # v3
+            if key in patch:
+                mt5_update[key] = patch[key]
+
+        if mt5_update:
+            cfg_manager.update({
+                "providers": {
+                    "accounts": {
+                        "mt5_default": mt5_update
+                    }
+                }
+            })
+            log.info(f"MT5 config updated: {list(mt5_update.keys())}")
+
+            # v4: reinit only when a connection-relevant field changed.
+            # use_tick_data is read live by focus_tick_loop on every iteration
+            # — flipping it must NOT bounce the MT5 connection (5s 503 outage).
+            CONNECTION_FIELDS = {"connection_method", "terminal_path",
+                                 "server", "login", "password"}
+            if CONNECTION_FIELDS.intersection(mt5_update.keys()):
+                try:
+                    prov = cfg_manager.get_provider("mt5_default")
+                    if prov is not None:
+                        try:
+                            if getattr(prov, "connected", False):
+                                await asyncio.to_thread(prov.disconnect)
+                        except Exception as e:
+                            log.warning(f"MT5 disconnect during reinit failed: {e}")
+                    await asyncio.to_thread(cfg_manager.init_providers)
+                    log.info("MT5 provider reinitialized with new config")
+                except Exception as e:
+                    log.warning(f"MT5 provider reinit failed: {e}")
+
+        return {"config": _get_mt5_config()}
+
+    # ── POST /api/mt5/browse-terminal ──
+    @router.post("/browse-terminal")
+    async def browse_terminal():
+        """v5: Open a native OS file picker for the user to select terminal64.exe.
+        Runs tkinter.filedialog on a worker thread (FastAPI's async loop can't
+        block on a dialog). Returns {"path": "..."} on pick, {"path": null,
+        "cancelled": true} on cancel. Windows-only for now — defaults to the
+        most likely MT5 install dir if nothing is configured yet."""
+        import asyncio
+        import os
+        import platform
+
+        if platform.system() != "Windows":
+            raise HTTPException(501, "Browse dialog is Windows-only for now.")
+
+        # Pick a sensible initial directory: current configured path → its dir,
+        # or %ProgramFiles% as a fallback.
+        current = _get_mt5_config().get("terminal_path") or ""
+        if current and os.path.isfile(current):
+            initial_dir = os.path.dirname(current)
+        elif current and os.path.isdir(current):
+            initial_dir = current
+        else:
+            pf = os.environ.get("ProgramFiles", r"C:\Program Files")
+            initial_dir = pf
+
+        def _pick() -> dict:
+            # v6: switched tkinter → native comdlg32 via ctypes.
+            # tkinter is not bundled in the PyInstaller build (strips by
+            # default unless explicitly hidden-imported), so the installed
+            # .exe would fail with "No module named 'tkinter'". comdlg32.dll
+            # is core Windows — always available, no dependencies, no build
+            # change needed. Still runs on a worker thread because the native
+            # call blocks until the user picks or cancels.
+            try:
+                import ctypes
+                from ctypes import wintypes, Structure, byref, sizeof, c_void_p
+
+                class OPENFILENAMEW(Structure):
+                    _fields_ = [
+                        ("lStructSize",      wintypes.DWORD),
+                        ("hwndOwner",        wintypes.HWND),
+                        ("hInstance",        wintypes.HINSTANCE),
+                        ("lpstrFilter",      wintypes.LPCWSTR),
+                        ("lpstrCustomFilter",wintypes.LPWSTR),
+                        ("nMaxCustFilter",   wintypes.DWORD),
+                        ("nFilterIndex",     wintypes.DWORD),
+                        ("lpstrFile",        wintypes.LPWSTR),
+                        ("nMaxFile",         wintypes.DWORD),
+                        ("lpstrFileTitle",   wintypes.LPWSTR),
+                        ("nMaxFileTitle",    wintypes.DWORD),
+                        ("lpstrInitialDir",  wintypes.LPCWSTR),
+                        ("lpstrTitle",       wintypes.LPCWSTR),
+                        ("Flags",            wintypes.DWORD),
+                        ("nFileOffset",      wintypes.WORD),
+                        ("nFileExtension",   wintypes.WORD),
+                        ("lpstrDefExt",      wintypes.LPCWSTR),
+                        ("lCustData",        wintypes.LPARAM),
+                        ("lpfnHook",         c_void_p),
+                        ("lpTemplateName",   wintypes.LPCWSTR),
+                        ("pvReserved",       c_void_p),
+                        ("dwReserved",       wintypes.DWORD),
+                        ("FlagsEx",          wintypes.DWORD),
+                    ]
+
+                OFN_FILEMUSTEXIST = 0x00001000
+                OFN_PATHMUSTEXIST = 0x00000800
+                OFN_EXPLORER      = 0x00080000
+                OFN_NOCHANGEDIR   = 0x00000008
+
+                buf = ctypes.create_unicode_buffer(4096)
+                ofn = OPENFILENAMEW()
+                ctypes.memset(byref(ofn), 0, sizeof(ofn))
+                ofn.lStructSize   = sizeof(OPENFILENAMEW)
+                ofn.lpstrFile     = ctypes.cast(buf, wintypes.LPWSTR)
+                ofn.nMaxFile      = 4096
+                # Filter string: pairs of "label\0pattern\0" terminated with an extra \0.
+                ofn.lpstrFilter   = (
+                    "MT5 terminal\0terminal64.exe;terminal.exe\0"
+                    "Executables\0*.exe\0"
+                    "All files\0*.*\0\0"
+                )
+                ofn.lpstrInitialDir = initial_dir
+                ofn.lpstrTitle      = "Select MT5 terminal (terminal64.exe)"
+                ofn.Flags           = (OFN_FILEMUSTEXIST | OFN_PATHMUSTEXIST
+                                       | OFN_EXPLORER | OFN_NOCHANGEDIR)
+
+                ok = ctypes.windll.comdlg32.GetOpenFileNameW(byref(ofn))
+                if not ok:
+                    # User cancelled OR dialog failed. CommDlgExtendedError
+                    # returns 0 for cancel, non-zero for actual errors.
+                    err = ctypes.windll.comdlg32.CommDlgExtendedError()
+                    if err != 0:
+                        return {"path": None, "error": f"comdlg32 error {err:#x}"}
+                    return {"path": None, "cancelled": True}
+
+                picked = buf.value
+                if not picked:
+                    return {"path": None, "cancelled": True}
+                return {"path": os.path.normpath(picked), "cancelled": False}
+            except Exception as e:
+                log.error(f"browse-terminal dialog failed: {e}")
+                return {"path": None, "error": str(e)}
+
+        try:
+            result = await asyncio.to_thread(_pick)
+        except Exception as e:
+            raise HTTPException(500, f"Dialog launch failed: {e}")
+        if result.get("error"):
+            raise HTTPException(500, result["error"])
+        return result
+
+
+    # ── POST /api/mt5/connect ──
+    @router.post("/connect")
+    async def connect_mt5(request: Request):
+        import asyncio
+
+        provider = cfg_manager.get_provider("mt5_default")
+        if provider is None:
+            # Try to init providers first
+            try:
+                await asyncio.to_thread(cfg_manager.init_providers)
+                provider = cfg_manager.get_provider("mt5_default")
+            except Exception as e:
+                raise HTTPException(500, f"Failed to initialize MT5 provider: {e}")
+
+        if provider is None:
+            raise HTTPException(500, "MT5 provider not available")
+
+        # If already connected, return current state
+        if provider.connected:
+            account = None
+            try:
+                import MetaTrader5 as mt5
+                info = mt5.account_info()
+                if info:
+                    account = {
+                        "login": info.login,
+                        "server": info.server,
+                        "balance": info.balance,
+                        "equity": info.equity,
+                        "currency": info.currency,
+                    }
+            except Exception:
+                pass
+            return {"connected": True, "account": account}
+
+        # Try to connect
+        try:
+            connected = await asyncio.wait_for(
+                asyncio.to_thread(provider.connect),
+                timeout=15.0,
+            )
+        except asyncio.TimeoutError:
+            return {"connected": False, "error": "Connection timed out (15s)"}
+        except Exception as e:
+            return {"connected": False, "error": str(e)}
+
+        if not connected:
+            return {"connected": False, "error": "MT5 connection failed — check terminal path or credentials"}
+
+        # Update app state
+        app.state.provider = provider
+
+        # Resolve symbols
+        try:
+            universe = cfg_manager.get_active_universe()
+            await asyncio.to_thread(provider.resolve_universe, universe)
+        except Exception:
+            pass
+
+        # Get account info
+        account = None
+        try:
+            import MetaTrader5 as mt5
+            info = mt5.account_info()
+            if info:
+                account = {
+                    "login": info.login,
+                    "server": info.server,
+                    "balance": info.balance,
+                    "equity": info.equity,
+                    "currency": info.currency,
+                }
+        except Exception:
+            pass
+
+        log.info("MT5 connected via settings panel")
+        return {"connected": True, "account": account}
+
+    # ── POST /api/mt5/disconnect ──
+    @router.post("/disconnect")
+    async def disconnect_mt5():
+        import asyncio
+        provider = cfg_manager.get_provider("mt5_default")
+        if provider and provider.connected:
+            try:
+                await asyncio.to_thread(provider.disconnect)
+                log.info("MT5 disconnected via settings panel")
+            except Exception as e:
+                log.warning(f"MT5 disconnect error: {e}")
+        return {"connected": False}
+
+    # ── GET /api/mt5/status ──
+    @router.get("/status")
+    async def get_mt5_status():
+        """Quick status check — used by settings panel."""
+        provider = cfg_manager.get_provider("mt5_default")
+        connected = provider.connected if provider else False
+        account = None
+        if connected:
+            try:
+                import MetaTrader5 as mt5
+                info = mt5.account_info()
+                if info:
+                    account = {
+                        "login": info.login,
+                        "server": info.server,
+                        "balance": info.balance,
+                        "equity": info.equity,
+                    }
+            except Exception:
+                pass
+        return {"connected": connected, "account": account}
+
+    return router

backend/options_routes.py

@@ -0,0 +1,142 @@
+"""
+options_routes.py — Options Quant REST Endpoints
+=================================================
+Quantum Terminal | Phase 4F — Options Quant Engine
+
+Wire into data_server.py:
+    from options_routes import create_options_router
+    app.include_router(create_options_router())
+
+Endpoints:
+    GET /api/options/symbols          — supported canonical assets + Yahoo mapping
+    GET /api/options/{ticker}/snapshot — full metrics (GEX + levels + IV divergence)
+    GET /api/options/{ticker}/chain   — raw calls/puts chain data
+    GET /api/options/{ticker}/expiries — available expiry dates from Yahoo
+"""
+
+import logging
+from pathlib import Path
+from fastapi import APIRouter, HTTPException, Query
+from typing import Optional
+
+log = logging.getLogger("mk.options_routes")
+
+PROJECT_ROOT = Path(__file__).resolve().parent
+
+
+def create_options_router() -> APIRouter:
+    router = APIRouter(tags=["options"])
+
+    from options_data_manager import get_options_dm, OPTIONS_SYMBOL_MAP
+    from options_engine import compute_all, compute_all_multi
+
+    @router.get("/api/options/symbols")
+    async def get_options_symbols():
+        """
+        Return list of assets with options support and their Yahoo mappings.
+        """
+        return {
+            "supported": [
+                {"canonical": k, "cboe_ticker": v}
+                for k, v in OPTIONS_SYMBOL_MAP.items()
+            ]
+        }
+
+    @router.get("/api/options/{ticker}/snapshot")
+    async def get_options_snapshot(
+        ticker: str,
+        expiry: Optional[str] = Query(None, description="ISO date e.g. 2025-03-21"),
+    ):
+        """
+        Full options analytics snapshot for one asset.
+        Includes GEX, flip level, max pain, walls, P/C ratio, IV divergence.
+        Response is cached 15 minutes — stale data served outside market hours.
+        """
+        canonical = ticker.upper()
+        if canonical not in OPTIONS_SYMBOL_MAP:
+            raise HTTPException(
+                404,
+                f"{canonical} not supported. Supported: {list(OPTIONS_SYMBOL_MAP.keys())}"
+            )
+
+        odm = get_options_dm()
+        snap = odm.get_snapshot(canonical, expiry=expiry)
+
+        if snap is None:
+            raise HTTPException(503, f"Could not fetch options data for {canonical}. "
+                                     f"Check yfinance install and network.")
+
+        try:
+            metrics = compute_all(snap)
+        except Exception as e:
+            log.error(f"Options engine error for {canonical}: {e}")
+            raise HTTPException(500, f"Options calculation failed: {e}")
+
+        return metrics
+
+    @router.get("/api/options/{ticker}/chain")
+    async def get_options_chain(
+        ticker: str,
+        expiry: Optional[str] = Query(None),
+    ):
+        """
+        Raw options chain (calls + puts) for one asset and expiry.
+        Useful for frontend to render custom strike tables.
+        """
+        canonical = ticker.upper()
+        if canonical not in OPTIONS_SYMBOL_MAP:
+            raise HTTPException(404, f"{canonical} not supported")
+
+        odm = get_options_dm()
+        snap = odm.get_snapshot(canonical, expiry=expiry)
+
+        if snap is None:
+            raise HTTPException(503, f"Could not fetch chain for {canonical}")
+
+        return {
+            "canonical":   snap["canonical"],
+            "cboe_ticker": snap.get("cboe_ticker"),
+            "expiry":      snap.get("expiry"),
+            "spot":        snap.get("spot"),
+            "calls":       snap.get("calls", []),
+            "puts":        snap.get("puts",  []),
+            "cached_at":   snap.get("cached_at"),
+            "is_stale":    snap.get("is_stale", False),
+        }
+
+    @router.get("/api/options/{ticker}/expiries")
+    async def get_options_expiries(ticker: str):
+        """Available expiry dates. Returns list of ISO date strings."""
+        canonical = ticker.upper()
+        if canonical not in OPTIONS_SYMBOL_MAP:
+            raise HTTPException(404, f"{canonical} not supported")
+        odm = get_options_dm()
+        return {"canonical": canonical, "expiries": odm.get_available_expiries(canonical)}
+
+    @router.get("/api/options/{ticker}/full")
+    async def get_options_full(
+        ticker: str,
+        n: int = Query(8, description="Number of expiries for GEX stack (1-12)"),
+    ):
+        """
+        Full multi-expiry options analytics.
+        Includes GEX stack (GEX1–GEXn), 0DTE GEX, HVL, expected move.
+        This is the primary endpoint for the enhanced OPTIONS QUANT UI.
+        """
+        canonical = ticker.upper()
+        if canonical not in OPTIONS_SYMBOL_MAP:
+            raise HTTPException(404, f"{canonical} not supported")
+
+        odm = get_options_dm()
+        n = max(1, min(n, 12))
+        snap = odm.get_full_snapshot(canonical, n_expiries=n)
+        if snap is None:
+            raise HTTPException(503, f"Could not fetch options data for {canonical}")
+
+        try:
+            return compute_all_multi(snap)
+        except Exception as e:
+            log.error(f"Options multi-expiry calc error {canonical}: {e}")
+            raise HTTPException(500, f"Calculation failed: {e}")
+
+    return router

backend/periodic_sync.py

@@ -0,0 +1,172 @@
+# version: v2
+"""
+================================================================================
+Quantum Terminal Consumer — Periodic Data Sync Poller
+================================================================================
+Background asyncio task that polls the VPS at a configurable interval and
+broadcasts a 'data_sync_complete' WebSocket event whenever fresh data arrives.
+This solves the problem where a long-running terminal would never re-check
+the server after the initial startup gate.
+
+Configurable via consumer_config.ini:
+    [sync]
+    poll_interval_seconds = 300
+
+Default interval: 300s (5 minutes). Floor: 60s (no hammering the server).
+
+Rule C1 compliance: this module performs ZERO calculations. It only re-runs
+the same display-data sync as the startup gate (consumer_gate -> sync_all).
+
+This module does NOT introduce any calculation triggers.
+================================================================================
+"""
+
+import asyncio
+import logging
+import os
+
+# v2: AppData dir name (env-var driven; default "QuantumTerminal" preserves v1 path).
+APP_DIR = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal")
+import configparser
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Awaitable, Callable
+
+from pulse_room.sync_hook import on_sync_complete as _pulse_on_sync_complete
+
+log = logging.getLogger("mk.periodic_sync")
+
+DEFAULT_INTERVAL_SECONDS = 300  # 5 minutes
+MIN_INTERVAL_SECONDS = 60       # safety floor
+
+
+def _read_interval() -> int:
+    """
+    Read sync.poll_interval_seconds from consumer_config.ini.
+    Searches the same locations as data_sync_client._read_config().
+    Returns DEFAULT_INTERVAL_SECONDS if not configured.
+    Floor is enforced at MIN_INTERVAL_SECONDS.
+    """
+    project_root = Path(__file__).resolve().parent
+
+    candidates = [
+        project_root / "consumer_config.ini",
+        project_root.parent / "consumer_config.ini",
+    ]
+
+    import sys as _sys
+    if getattr(_sys, "frozen", False):
+        candidates.append(Path(_sys.executable).parent / "consumer_config.ini")
+
+    appdata = os.environ.get("APPDATA", "")
+    if appdata:
+        candidates.append(Path(appdata) / APP_DIR / "consumer_config.ini")
+
+    for p in candidates:
+        if not p.exists():
+            continue
+        try:
+            cp = configparser.ConfigParser()
+            cp.read(str(p), encoding="utf-8-sig")
+            if cp.has_option("sync", "poll_interval_seconds"):
+                val = cp.getint("sync", "poll_interval_seconds")
+                return max(MIN_INTERVAL_SECONDS, val)
+            break
+        except Exception as e:
+            log.warning(f"Could not parse sync interval from {p.name}: {e}")
+            break
+
+    return DEFAULT_INTERVAL_SECONDS
+
+
+def create_periodic_sync_task(
+    broadcast_event: Callable[[dict], Awaitable[None]],
+) -> Callable:
+    """
+    Returns an async coroutine factory that polls sync_all() at a configurable
+    interval. Pass the result to asyncio.create_task() inside the FastAPI
+    lifespan.
+
+    Usage in data_server.py lifespan:
+        from periodic_sync import create_periodic_sync_task
+        sync_task = create_periodic_sync_task(manager.broadcast_event)
+        tasks.append(asyncio.create_task(sync_task()))
+
+    Each tick the task:
+        1. Calls data_sync_client.sync_all() in a worker thread
+        2. Logs the result (synced / skipped / offline)
+        3. If new files arrived (synced > 0), broadcasts WS event:
+              {"type": "data_sync_complete", "synced": N, ...}
+           Frontend (useTerminalData.js) listens and refreshes panels.
+
+    Errors do not kill the task — they back off and retry on the next interval.
+    """
+    interval = _read_interval()
+    log.info(f"Periodic sync task configured: interval={interval}s")
+
+    async def _task():
+        # Wait one full interval before first poll —
+        # the startup gate has already done the initial sync.
+        try:
+            await asyncio.sleep(interval)
+        except asyncio.CancelledError:
+            return
+
+        while True:
+            try:
+                from data_sync_client import get_sync_client
+                client = get_sync_client()
+
+                # sync_all() does blocking HTTP — run in a thread
+                summary = await asyncio.to_thread(client.sync_all)
+
+                synced = summary.get("synced", 0)
+                skipped = summary.get("skipped", 0)
+                offline = summary.get("offline_mode", False)
+
+                log.info(
+                    f"Periodic sync tick: synced={synced}, "
+                    f"skipped={skipped}, offline={offline}"
+                )
+
+                # Only broadcast when fresh data actually arrived.
+                # Skipping silent ticks keeps frontend quiet during quiet hours.
+                if synced > 0:
+                    try:
+                        await broadcast_event({
+                            "type": "data_sync_complete",
+                            "synced": synced,
+                            "skipped": skipped,
+                            "offline_mode": offline,
+                            "timestamp": datetime.now(timezone.utc).isoformat(),
+                        })
+                    except Exception as e:
+                        log.warning(f"Failed to broadcast data_sync_complete: {e}")
+
+                    # Pulse Room auto-rebuild — only when fresh source data
+                    # arrived. Fires only if scanner ON + watchlist non-empty
+                    # (eligibility check is inside on_sync_complete itself);
+                    # daemon thread, fire-and-forget; never breaks sync pipeline.
+                    try:
+                        _pulse_on_sync_complete()
+                    except Exception:
+                        log.exception("pulse_room.sync_hook threw")
+
+            except asyncio.CancelledError:
+                log.info("Periodic sync task cancelled")
+                break
+            except Exception as e:
+                log.error(f"Periodic sync task error: {e}", exc_info=True)
+                # Back off on error but never longer than the configured interval
+                try:
+                    await asyncio.sleep(min(interval, 120))
+                except asyncio.CancelledError:
+                    break
+                continue
+
+            try:
+                await asyncio.sleep(interval)
+            except asyncio.CancelledError:
+                break
+
+    return _task

backend/providers/__init__.py

@@ -0,0 +1,68 @@
+"""
+================================================================================
+Quantum Terminal — Provider Registry
+================================================================================
+Maps provider type names to their implementation classes.
+
+To add a new provider:
+    1. Create providers/my_provider.py implementing BaseProvider
+    2. Add "my_provider": MyProvider to PROVIDER_REGISTRY below
+    3. Add account config in user_config.json under providers.accounts
+
+That's it — the config_manager will instantiate and manage it.
+================================================================================
+"""
+
+from providers.base_provider import BaseProvider
+from providers.mt5_provider import MT5Provider
+
+# ── Rithmic provider (graceful if async_rithmic not installed) ──
+try:
+    from providers.rithmic_provider import RithmicProvider
+    RITHMIC_AVAILABLE = True
+except ImportError:
+    RithmicProvider = None
+    RITHMIC_AVAILABLE = False
+
+# ── Provider type → class mapping ──
+# Key = the "type" field in account config
+# Value = class that implements BaseProvider
+PROVIDER_REGISTRY = {
+    "mt5": MT5Provider,
+}
+
+# Register Rithmic only if async_rithmic is installed
+if RITHMIC_AVAILABLE:
+    PROVIDER_REGISTRY["rithmic"] = RithmicProvider
+
+
+def create_provider(account_config: dict) -> BaseProvider:
+    """
+    Factory: create a provider instance from account config dict.
+    
+    Expected config shape:
+        {
+            "id": "mt5_primary",
+            "type": "mt5",
+            "label": "MT5 — CFI (Live)",
+            "terminal_path": null,
+            "aliases": {}
+        }
+    
+    Raises KeyError if provider type is not registered.
+    """
+    provider_type = account_config.get("type", "")
+    if provider_type not in PROVIDER_REGISTRY:
+        registered = ", ".join(PROVIDER_REGISTRY.keys())
+        raise KeyError(
+            f"Unknown provider type: '{provider_type}'. "
+            f"Registered types: {registered}"
+        )
+    
+    cls = PROVIDER_REGISTRY[provider_type]
+    return cls(account_config)
+
+
+def list_provider_types() -> list:
+    """Return list of registered provider type names."""
+    return list(PROVIDER_REGISTRY.keys())

backend/providers/base_provider.py

@@ -0,0 +1,276 @@
+"""
+================================================================================
+Quantum Terminal — Base Provider Interface
+================================================================================
+Abstract contract that every data/execution provider must implement.
+
+The data_server and config_manager talk to providers ONLY through this
+interface. MT5, Binance, Polygon, or any future source plugs in by
+subclassing BaseProvider and implementing the required methods.
+
+Design principles:
+    - All methods are synchronous (callers use asyncio.to_thread)
+    - Providers manage their own connection lifecycle
+    - Canonical ticker names everywhere — providers map internally
+    - Providers declare their capabilities (data-only vs data+execution)
+
+Usage:
+    from providers.base_provider import BaseProvider
+    from providers.mt5_provider import MT5Provider
+
+    provider = MT5Provider(account_config)
+    provider.connect()
+    ticks = provider.get_latest_ticks(["XAUUSD", "EURUSD"])
+================================================================================
+"""
+
+from abc import ABC, abstractmethod
+from typing import Dict, List, Optional
+from models import (
+    TickData, BarData, AccountInfo, SymbolInfo,
+    OrderRequest, OrderResult, Position, PendingOrder,
+)
+
+
+class BaseProvider(ABC):
+    """
+    Abstract provider interface.
+    
+    Every provider has:
+        - A type name (e.g., "mt5", "binance")
+        - A unique instance ID (e.g., "mt5_primary", "binance_spot")
+        - Connection lifecycle (connect/disconnect/reconnect)
+        - Market data methods (ticks, bars, symbol info)
+        - Optional execution methods (orders, positions)
+    
+    Providers are synchronous. The data_server wraps calls in
+    asyncio.to_thread() to avoid blocking the event loop.
+    """
+
+    # ── Identity ──
+
+    @property
+    @abstractmethod
+    def provider_type(self) -> str:
+        """Provider type identifier. E.g., 'mt5', 'binance', 'polygon'."""
+        ...
+
+    @property
+    @abstractmethod
+    def provider_id(self) -> str:
+        """Unique instance ID. E.g., 'mt5_primary'. Set from account config."""
+        ...
+
+    @property
+    @abstractmethod
+    def label(self) -> str:
+        """Human-readable label. E.g., 'MT5 — CFI (Live)'."""
+        ...
+
+    # ── Capabilities ──
+
+    @property
+    def can_stream_ticks(self) -> bool:
+        """Whether this provider supports live tick polling."""
+        return True
+
+    @property
+    def can_execute(self) -> bool:
+        """Whether this provider supports order execution."""
+        return False
+
+    @property
+    def supported_timeframes(self) -> List[str]:
+        """List of timeframe strings this provider supports."""
+        return ["M1", "M5", "M15", "M30", "H1", "H4", "D1", "W1"]
+
+    # ── Connection Lifecycle ──
+
+    @property
+    @abstractmethod
+    def connected(self) -> bool:
+        """Whether the provider is currently connected."""
+        ...
+
+    @abstractmethod
+    def connect(self) -> bool:
+        """
+        Establish connection to the data/execution source.
+        Returns True on success, False on failure.
+        Must be idempotent — calling connect() when already connected is safe.
+        """
+        ...
+
+    @abstractmethod
+    def disconnect(self) -> None:
+        """Cleanly shut down the connection."""
+        ...
+
+    def reconnect(self) -> bool:
+        """Disconnect and reconnect. Override for custom reconnect logic."""
+        self.disconnect()
+        return self.connect()
+
+    def heartbeat(self) -> bool:
+        """
+        Lightweight connection health check.
+        Returns True if the connection is alive, False otherwise.
+        If False, sets internal connected state to False so reconnect_loop picks it up.
+        Override in subclasses for provider-specific health checks.
+        """
+        return self.connected
+
+    # ── Market Data ──
+
+    @abstractmethod
+    def get_latest_ticks(self, symbols: List[str]) -> Dict[str, TickData]:
+        """
+        Fetch latest tick for each symbol.
+        
+        Args:
+            symbols: List of canonical ticker names (e.g., ["XAUUSD", "EURUSD"])
+        
+        Returns:
+            Dict mapping canonical ticker → TickData.
+            Missing/failed symbols are simply omitted.
+        """
+        ...
+
+    @abstractmethod
+    def get_bars(
+        self, ticker: str, timeframe: str = "M15", count: int = 200
+    ) -> List[BarData]:
+        """
+        Fetch recent OHLCV bars for a canonical ticker.
+        
+        Args:
+            ticker: Canonical symbol name
+            timeframe: Timeframe string (M1, M5, M15, H1, H4, D1, etc.)
+            count: Number of bars to fetch
+        
+        Returns:
+            List of BarData, oldest first. Empty list on failure.
+        """
+        ...
+
+    @abstractmethod
+    def check_new_bars(
+        self, symbols: List[str], timeframe: str = "M15"
+    ) -> List[dict]:
+        """
+        Detect newly closed bars since last check.
+        
+        Returns list of dicts:
+            {"type": "bar", "ticker": str, "timeframe": str, "bar": BarData.to_dict()}
+        
+        Implementation must track last-seen bar timestamps internally.
+        """
+        ...
+
+    @abstractmethod
+    def get_symbol_info(self, ticker: str) -> Optional[SymbolInfo]:
+        """
+        Get metadata for a symbol (decimals, lot sizing, contract size, etc.)
+        Returns None if symbol not found.
+        """
+        ...
+
+    def get_all_symbol_info(self, symbols: List[str]) -> Dict[str, SymbolInfo]:
+        """
+        Batch symbol info for multiple tickers.
+        Default implementation calls get_symbol_info() in a loop.
+        Override for providers that support batch queries.
+        """
+        result = {}
+        for s in symbols:
+            info = self.get_symbol_info(s)
+            if info is not None:
+                result[s] = info
+        return result
+
+    # ── Account Info ──
+
+    @abstractmethod
+    def get_account_info(self) -> Optional[AccountInfo]:
+        """
+        Get current account snapshot (balance, equity, margin, etc.)
+        Returns None if not connected or not applicable.
+        """
+        ...
+
+    # ── Execution (optional — override if can_execute is True) ──
+
+    def place_order(self, order: OrderRequest) -> OrderResult:
+        """Place an order. Override in execution-capable providers."""
+        return OrderResult(
+            success=False,
+            error=f"Provider '{self.provider_type}' does not support execution",
+        )
+
+    def get_positions(self) -> List[Position]:
+        """Get all open positions. Override in execution-capable providers."""
+        return []
+
+    def close_position(self, ticket: str, lots: Optional[float] = None) -> OrderResult:
+        """
+        Close a position (fully or partially).
+        Override in execution-capable providers.
+        """
+        return OrderResult(
+            success=False,
+            error=f"Provider '{self.provider_type}' does not support execution",
+        )
+
+    def get_bars_range(self, ticker, timeframe, from_dt, to_dt):
+        """Fetch OHLCV bars between two datetimes.
+        Override in providers that support historical range queries."""
+        return []
+
+    def get_pending_orders(self) -> List[PendingOrder]:
+        """
+        List all resting (non-filled) pending orders (LIMIT / STOP).
+        Override in execution-capable providers.
+        """
+        return []
+
+    def cancel_order(self, ticket: str) -> OrderResult:
+        """Cancel a resting pending order. Override in execution-capable providers."""
+        return OrderResult(
+            success=False,
+            error=f"Provider '{self.provider_type}' does not support execution",
+        )
+
+    def modify_order(
+        self, ticket: str,
+        price: Optional[float] = None,
+        stop_loss: Optional[float] = None,
+        take_profit: Optional[float] = None,
+    ) -> OrderResult:
+        """Modify price / SL / TP on a pending order. Override in execution-capable providers."""
+        return OrderResult(
+            success=False,
+            error=f"Provider '{self.provider_type}' does not support execution",
+        )
+
+    # ── Symbol Resolution ──
+
+    @abstractmethod
+    def resolve_symbol(self, canonical: str) -> Optional[str]:
+        """
+        Map a canonical ticker name to the provider's native symbol.
+        Returns None if the symbol is not available in this provider.
+        """
+        ...
+
+    def get_available_symbols(self) -> List[str]:
+        """
+        Return list of all canonical symbols this provider can serve.
+        Default: empty (override to support symbol discovery).
+        """
+        return []
+
+    # ── String Representation ──
+
+    def __repr__(self) -> str:
+        status = "connected" if self.connected else "disconnected"
+        return f"<{self.__class__.__name__} id='{self.provider_id}' [{status}]>"

backend/providers/rithmic_provider.py

@@ -0,0 +1,758 @@
+"""
+================================================================================
+Quantum Terminal — Rithmic Provider
+================================================================================
+Implements BaseProvider for Rithmic futures data feed.
+
+Uses the async_rithmic library (Protocol Buffer API over WebSocket) to:
+    - Stream live BBO ticks for futures instruments
+    - Stream live time bars (M1/M5/M15/M30/H1)
+    - Fetch historical bars on demand
+    - Auto-resolve front month contracts (ES → ESM6, etc.)
+
+Architecture:
+    async_rithmic requires running inside a proper asyncio event loop.
+    This provider exposes async_connect()/async_disconnect() methods that
+    run directly in the caller's event loop (FastAPI's uvicorn loop).
+
+    data_server.py lifespan calls:
+        await provider.async_connect()      # in the main event loop
+    
+    Sync methods (get_latest_ticks, get_bars, etc.) read from caches
+    populated by streaming callbacks running in the same event loop.
+
+    For non-async contexts (test scripts), connect() falls back to
+    asyncio.run() which works but blocks the calling thread.
+
+Config dict keys:
+    id:          str  — unique provider ID (e.g., "rithmic_default")
+    type:        str  — "rithmic"
+    label:       str  — display name (e.g., "Rithmic — Paper Trading")
+    enabled:     bool — True
+    user:        str  — Rithmic username (from local_config.ini)
+    password:    str  — Rithmic password (from local_config.ini)
+    system_name: str  — "Rithmic Test" / "Rithmic 01" etc.
+    url:         str  — server URL (e.g., "rituz00100.rithmic.com:443")
+    app_name:    str  — "Quantum Terminal" (default)
+    app_version: str  — "1.0" (default)
+    symbol_map:  dict — canonical → {base, exchange} overrides (optional)
+
+Dependencies:
+    pip install async_rithmic
+================================================================================
+"""
+
+import logging
+import threading
+import asyncio
+from collections import deque
+from datetime import datetime, timezone, timedelta
+from typing import Dict, List, Optional, Any
+
+from models import (
+    TickData, BarData, AccountInfo, SymbolInfo,
+    OrderRequest, OrderResult, Position,
+)
+from providers.base_provider import BaseProvider
+
+log = logging.getLogger("provider.rithmic")
+
+# ── async_rithmic imported lazily ──
+try:
+    from async_rithmic import (
+        RithmicClient,
+        TimeBarType,
+        DataType,
+        LastTradePresenceBits,
+        BestBidOfferPresenceBits,
+    )
+    RITHMIC_AVAILABLE = True
+except ImportError:
+    RITHMIC_AVAILABLE = False
+    RithmicClient = None
+    TimeBarType = None
+    DataType = None
+
+
+# ============================================================
+# 1. SYMBOL MAPPING — Canonical → Rithmic
+# ============================================================
+
+# Maps Quantum Terminal canonical tickers to Rithmic base symbols + exchange.
+# get_front_month_contract() auto-resolves the actual contract code
+# (e.g., "ES" → "ESM6" for June 2026).
+#
+# IMPORTANT: Futures canonical tickers are SEPARATE from CFD tickers.
+#   ES   = CME E-mini S&P 500 futures   (Rithmic)
+#   US500 = S&P 500 CFD                  (MT5/CFI)
+# They coexist in the universe as independent instruments.
+RITHMIC_SYMBOL_MAP = {
+    # Equity index futures
+    "ES":   {"base": "ES",   "exchange": "CME"},
+    "NQ":   {"base": "NQ",   "exchange": "CME"},
+    "YM":   {"base": "YM",   "exchange": "CBOT"},
+    # Metal futures
+    "GC":   {"base": "GC",   "exchange": "COMEX"},
+    "SI":   {"base": "SI",   "exchange": "COMEX"},
+    # Energy futures
+    "CL":   {"base": "CL",   "exchange": "NYMEX"},
+    "BZ":   {"base": "BZ",   "exchange": "NYMEX"},
+    # European index futures (Eurex — may not be available on all accounts)
+    "FDAX": {"base": "FDAX", "exchange": "EUREX"},
+    "Z":    {"base": "Z",    "exchange": "LIFFE"},
+}
+
+# Maps our timeframe strings to (TimeBarType enum name, period) pairs.
+RITHMIC_TF_MAP = {
+    "M1":  ("MINUTE_BAR", 1),
+    "M5":  ("MINUTE_BAR", 5),
+    "M15": ("MINUTE_BAR", 15),
+    "M30": ("MINUTE_BAR", 30),
+    "H1":  ("MINUTE_BAR", 60),
+    "H4":  ("MINUTE_BAR", 240),
+    "D1":  ("DAILY_BAR",  1),
+}
+
+# Futures contract specs (static — used for SymbolInfo).
+FUTURES_SPECS = {
+    "ES":   {"decimals": 2, "tick_size": 0.25, "tick_value": 12.50,
+             "contract_size": 50.0, "description": "E-mini S&P 500",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "NQ":   {"decimals": 2, "tick_size": 0.25, "tick_value": 5.00,
+             "contract_size": 20.0, "description": "E-mini NASDAQ-100",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "YM":   {"decimals": 0, "tick_size": 1.0,  "tick_value": 5.00,
+             "contract_size": 5.0,  "description": "E-mini Dow",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "GC":   {"decimals": 2, "tick_size": 0.10, "tick_value": 10.00,
+             "contract_size": 100.0, "description": "Gold Futures",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "SI":   {"decimals": 3, "tick_size": 0.005, "tick_value": 25.00,
+             "contract_size": 5000.0, "description": "Silver Futures",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "CL":   {"decimals": 2, "tick_size": 0.01, "tick_value": 10.00,
+             "contract_size": 1000.0, "description": "Crude Oil WTI",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "BZ":   {"decimals": 2, "tick_size": 0.01, "tick_value": 10.00,
+             "contract_size": 1000.0, "description": "Brent Crude Oil",
+             "currency": "USD", "min_lot": 1, "lot_step": 1, "max_lot": 100},
+    "FDAX": {"decimals": 1, "tick_size": 0.5,  "tick_value": 12.50,
+             "contract_size": 25.0, "description": "DAX Futures",
+             "currency": "EUR", "min_lot": 1, "lot_step": 1, "max_lot": 50},
+    "Z":    {"decimals": 1, "tick_size": 0.5,  "tick_value": 5.00,
+             "contract_size": 10.0, "description": "FTSE 100 Futures",
+             "currency": "GBP", "min_lot": 1, "lot_step": 1, "max_lot": 50},
+}
+
+# Minutes per timeframe — used to compute bar fetch windows
+TF_MINUTES = {
+    "M1": 1, "M5": 5, "M15": 15, "M30": 30,
+    "H1": 60, "H4": 240, "D1": 1440, "W1": 10080,
+}
+
+
+# ============================================================
+# 2. RITHMIC PROVIDER
+# ============================================================
+
+class RithmicProvider(BaseProvider):
+    """
+    Rithmic futures data provider.
+
+    Data-only provider (can_execute=False for now). Streams live
+    ticks and bars via async_rithmic, serves data through the
+    synchronous BaseProvider interface.
+
+    IMPORTANT: async_rithmic must run inside a proper asyncio event
+    loop. Use async_connect() from FastAPI's lifespan, or connect()
+    which falls back to asyncio.run() for standalone scripts.
+    """
+
+    def __init__(self, account_config: Dict[str, Any]):
+        self._id = account_config.get("id", "rithmic_default")
+        self._label = account_config.get("label", "Rithmic")
+        self._user = account_config.get("user", "")
+        self._password = account_config.get("password", "")
+        self._system_name = account_config.get("system_name", "Rithmic Test")
+        self._url = account_config.get("url", "")
+        self._app_name = account_config.get("app_name", "Quantum Terminal")
+        self._app_version = account_config.get("app_version", "1.0")
+
+        # Symbol map: merge defaults with user overrides
+        self._symbol_map = dict(RITHMIC_SYMBOL_MAP)
+        user_map = account_config.get("symbol_map", {})
+        self._symbol_map.update(user_map)
+
+        # Connection state
+        self._connected = False
+        self._client: Optional[Any] = None  # RithmicClient instance
+
+        # Reference to the event loop we're running in (set during connect)
+        self._loop: Optional[asyncio.AbstractEventLoop] = None
+
+        # Resolved front month contracts: canonical → "ESM6" etc.
+        self._front_months: Dict[str, str] = {}
+        # Reverse map: "ESM6" → "ES"
+        self._reverse_map: Dict[str, str] = {}
+
+        # Tick cache: canonical → TickData (written from async callbacks,
+        # read from sync methods — both in the same thread in production)
+        self._tick_cache: Dict[str, TickData] = {}
+
+        # Bar buffer: "TICKER_TF" → deque of BarData (ring buffer)
+        self._bar_buffers: Dict[str, deque] = {}
+        self._max_bar_buffer = 500
+
+        # Bar tracking for check_new_bars
+        self._last_bar_times: Dict[str, str] = {}
+
+        # Subscribed symbols (canonical names that resolved successfully)
+        self._subscribed: List[str] = []
+
+    # ── Identity ──
+
+    @property
+    def provider_type(self) -> str:
+        return "rithmic"
+
+    @property
+    def provider_id(self) -> str:
+        return self._id
+
+    @property
+    def label(self) -> str:
+        return self._label
+
+    # ── Capabilities ──
+
+    @property
+    def can_execute(self) -> bool:
+        return False  # Data-only for now
+
+    @property
+    def can_stream_ticks(self) -> bool:
+        return True
+
+    @property
+    def supported_timeframes(self) -> List[str]:
+        return list(RITHMIC_TF_MAP.keys())
+
+    # ── Connection Lifecycle ──
+
+    @property
+    def connected(self) -> bool:
+        return self._connected
+
+    def connect(self) -> bool:
+        """
+        Synchronous connect — for use in non-async contexts (test scripts).
+
+        In production (data_server.py), use async_connect() instead.
+        This method uses asyncio.run() which blocks the calling thread.
+        """
+        if not RITHMIC_AVAILABLE:
+            log.warning("async_rithmic package not installed — "
+                        "run: pip install async_rithmic")
+            return False
+
+        if self._connected:
+            return True
+
+        if not self._user or not self._password or not self._url:
+            log.error("Rithmic credentials missing — "
+                      "check [rithmic] in local_config.ini")
+            return False
+
+        try:
+            return asyncio.run(self.async_connect())
+        except Exception as e:
+            log.error(f"Rithmic connect error: {e}")
+            return False
+
+    async def async_connect(self) -> bool:
+        """
+        Async connect — runs in the caller's event loop.
+
+        Called from data_server.py lifespan:
+            connected = await provider.async_connect()
+
+        1. Create RithmicClient
+        2. Connect to server
+        3. Resolve front month contracts
+        4. Subscribe to BBO ticks + M1 time bars
+        """
+        if not RITHMIC_AVAILABLE:
+            log.warning("async_rithmic not installed")
+            return False
+
+        if self._connected:
+            return True
+
+        if not self._user or not self._password or not self._url:
+            log.error("Rithmic credentials missing")
+            return False
+
+        try:
+            self._loop = asyncio.get_running_loop()
+
+            self._client = RithmicClient(
+                user=self._user,
+                password=self._password,
+                system_name=self._system_name,
+                app_name=self._app_name,
+                app_version=self._app_version,
+                url=self._url,
+            )
+
+            await self._client.connect()
+            log.info("Rithmic client connected to server")
+
+            # Register event callbacks
+            self._client.on_tick += self._on_tick
+            self._client.on_time_bar += self._on_time_bar
+
+            # Resolve front month contracts
+            for canonical, mapping in self._symbol_map.items():
+                base = mapping["base"]
+                exchange = mapping["exchange"]
+                try:
+                    front = await self._client.get_front_month_contract(
+                        base, exchange
+                    )
+                    self._front_months[canonical] = front
+                    self._reverse_map[front] = canonical
+                    log.info(f"  {canonical} -> {front} ({exchange})")
+                except Exception as e:
+                    log.warning(
+                        f"  {canonical} -> FAILED to resolve "
+                        f"{base}@{exchange}: {e}"
+                    )
+
+            # Subscribe to live data for resolved symbols
+            for canonical, contract in self._front_months.items():
+                exchange = self._symbol_map[canonical]["exchange"]
+                try:
+                    # Subscribe to BBO ticks
+                    data_type = DataType.LAST_TRADE | DataType.BBO
+                    await self._client.subscribe_to_market_data(
+                        contract, exchange, data_type
+                    )
+
+                    # Subscribe to M1 time bars
+                    await self._client.subscribe_to_time_bar_data(
+                        contract, exchange, TimeBarType.MINUTE_BAR, 1
+                    )
+
+                    self._subscribed.append(canonical)
+
+                    # Initialize bar buffer
+                    self._bar_buffers[f"{canonical}_M1"] = deque(
+                        maxlen=self._max_bar_buffer
+                    )
+
+                    log.info(
+                        f"  Subscribed: {canonical} ({contract}@{exchange})"
+                    )
+                except Exception as e:
+                    log.warning(
+                        f"  Subscribe failed for {canonical} "
+                        f"({contract}@{exchange}): {e}"
+                    )
+
+            if self._subscribed:
+                self._connected = True
+                log.info(
+                    f"Rithmic connected: {self._system_name} | "
+                    f"Subscribed: {len(self._subscribed)} symbols | "
+                    f"URL: {self._url}"
+                )
+                return True
+            else:
+                log.error("No symbols subscribed — connection not useful")
+                await self._async_disconnect()
+                return False
+
+        except Exception as e:
+            log.error(f"Rithmic async connect failed: {e}")
+            return False
+
+    def disconnect(self) -> None:
+        """
+        Synchronous disconnect — for use in non-async contexts.
+
+        In production, use async_disconnect() instead, or this method
+        will schedule the disconnect in the running event loop.
+        """
+        if not self._connected:
+            return
+
+        # If we have a reference to the event loop and it's running,
+        # schedule the async disconnect
+        if self._loop and self._loop.is_running():
+            future = asyncio.run_coroutine_threadsafe(
+                self._async_disconnect(), self._loop
+            )
+            try:
+                future.result(timeout=10)
+            except Exception as e:
+                log.warning(f"Rithmic disconnect error: {e}")
+        else:
+            # No running loop — try asyncio.run as last resort
+            try:
+                asyncio.run(self._async_disconnect())
+            except Exception:
+                pass
+
+        self._connected = False
+        self._subscribed.clear()
+        self._front_months.clear()
+        self._reverse_map.clear()
+        self._tick_cache.clear()
+        self._bar_buffers.clear()
+        self._client = None
+        log.info("Rithmic disconnected")
+
+    async def async_disconnect(self) -> None:
+        """Async disconnect — called from lifespan shutdown."""
+        await self._async_disconnect()
+        self._connected = False
+        self._subscribed.clear()
+        self._front_months.clear()
+        self._reverse_map.clear()
+        self._tick_cache.clear()
+        self._bar_buffers.clear()
+        self._client = None
+        log.info("Rithmic disconnected")
+
+    def heartbeat(self) -> bool:
+        """Check if the Rithmic connection is alive."""
+        # async_rithmic handles heartbeats internally
+        return self._connected
+
+    # ── Market Data ──
+
+    def get_latest_ticks(self, symbols: List[str]) -> Dict[str, TickData]:
+        """Return latest cached ticks for requested symbols."""
+        result = {}
+        for s in symbols:
+            if s in self._tick_cache:
+                result[s] = self._tick_cache[s]
+        return result
+
+    def get_bars(
+        self, ticker: str, timeframe: str = "M15", count: int = 200
+    ) -> List[BarData]:
+        """
+        Fetch bars for a canonical ticker.
+
+        Checks local buffer first. If not enough bars, fetches from
+        Rithmic history plant via the event loop.
+        """
+        if ticker not in self._front_months:
+            return []
+
+        # Check buffer first
+        buf_key = f"{ticker}_{timeframe}"
+        if buf_key in self._bar_buffers:
+            bars = list(self._bar_buffers[buf_key])
+            if len(bars) >= count:
+                return bars[-count:]
+
+        # Fetch from history — need the event loop
+        if not self._loop or not self._loop.is_running():
+            return []
+
+        try:
+            future = asyncio.run_coroutine_threadsafe(
+                self._async_get_bars(ticker, timeframe, count),
+                self._loop,
+            )
+            return future.result(timeout=30)
+        except Exception as e:
+            log.warning(f"[{ticker}] Historical bar fetch failed: {e}")
+            return []
+
+    def check_new_bars(
+        self, symbols: List[str], timeframe: str = "M15"
+    ) -> List[dict]:
+        """Return bars received since last check."""
+        new_bars = []
+        for s in symbols:
+            buf_key = f"{s}_{timeframe}"
+            if buf_key not in self._bar_buffers:
+                continue
+
+            track_key = f"{s}_{timeframe}"
+            last_time = self._last_bar_times.get(track_key)
+            buf = self._bar_buffers[buf_key]
+
+            for bar in buf:
+                if last_time is None or bar.time > last_time:
+                    new_bars.append({
+                        "type": "bar",
+                        "ticker": s,
+                        "timeframe": timeframe,
+                        "bar": bar.to_dict(),
+                    })
+                    self._last_bar_times[track_key] = bar.time
+
+        return new_bars
+
+    def get_symbol_info(self, ticker: str) -> Optional[SymbolInfo]:
+        """Return static futures contract metadata."""
+        if ticker not in self._symbol_map:
+            return None
+
+        mapping = self._symbol_map[ticker]
+        base = mapping["base"]
+        spec = FUTURES_SPECS.get(base)
+        if not spec:
+            return None
+
+        broker_symbol = self._front_months.get(ticker, base)
+
+        return SymbolInfo(
+            ticker=ticker,
+            broker_symbol=broker_symbol,
+            asset_class="FUTURES",
+            decimals=spec["decimals"],
+            description=spec["description"],
+            trade_allowed=False,
+            min_lot=spec["min_lot"],
+            max_lot=spec["max_lot"],
+            lot_step=spec["lot_step"],
+            contract_size=spec["contract_size"],
+            currency_profit=spec["currency"],
+            currency_margin=spec["currency"],
+            tick_size=spec["tick_size"],
+            tick_value=spec["tick_value"],
+        )
+
+    def get_account_info(self) -> Optional[AccountInfo]:
+        """Not applicable for data-only provider."""
+        return None
+
+    # ── Symbol Resolution ──
+
+    def resolve_symbol(self, canonical: str) -> Optional[str]:
+        """Map canonical ticker to resolved Rithmic contract code."""
+        return self._front_months.get(canonical)
+
+    def resolve_universe(self, universe: List[str]) -> List[str]:
+        """Return which universe tickers this provider can serve."""
+        return [t for t in universe if t in self._front_months]
+
+    def get_available_symbols(self) -> List[str]:
+        """Return canonical symbols that were successfully subscribed."""
+        return list(self._subscribed)
+
+    # ============================================================
+    # INTERNAL — Async Operations
+    # ============================================================
+
+    async def _async_disconnect(self) -> None:
+        """Async cleanup: unsubscribe and disconnect."""
+        if not self._client:
+            return
+
+        try:
+            for canonical in self._subscribed:
+                if canonical not in self._front_months:
+                    continue
+                contract = self._front_months[canonical]
+                exchange = self._symbol_map[canonical]["exchange"]
+                try:
+                    await self._client.unsubscribe_from_market_data(
+                        contract, exchange,
+                        DataType.LAST_TRADE | DataType.BBO,
+                    )
+                    await self._client.unsubscribe_from_time_bar_data(
+                        contract, exchange, TimeBarType.MINUTE_BAR, 1,
+                    )
+                except Exception:
+                    pass
+
+            await self._client.disconnect()
+        except Exception as e:
+            log.warning(f"Rithmic async disconnect error: {e}")
+
+    async def _async_get_bars(
+        self, ticker: str, timeframe: str, count: int
+    ) -> List[BarData]:
+        """Fetch historical time bars from Rithmic history plant."""
+        if ticker not in self._front_months or not self._client:
+            return []
+
+        contract = self._front_months[ticker]
+        exchange = self._symbol_map[ticker]["exchange"]
+
+        tf_info = RITHMIC_TF_MAP.get(timeframe)
+        if not tf_info:
+            log.warning(f"[{ticker}] Unsupported timeframe: {timeframe}")
+            return []
+
+        bar_type_name, period = tf_info
+
+        if bar_type_name == "MINUTE_BAR":
+            bar_type = TimeBarType.MINUTE_BAR
+        elif bar_type_name == "DAILY_BAR":
+            bar_type = TimeBarType.DAILY_BAR
+        else:
+            bar_type = TimeBarType.MINUTE_BAR
+
+        # Calculate time window
+        minutes_per_bar = TF_MINUTES.get(timeframe, 15)
+        total_minutes = int(count * minutes_per_bar * 1.2)  # 20% buffer
+        end_time = datetime.now(timezone.utc)
+        start_time = end_time - timedelta(minutes=total_minutes)
+
+        try:
+            raw_bars = await self._client.get_time_bars(
+                contract, exchange, bar_type, period,
+                start_time, end_time,
+            )
+
+            bars = []
+            for rb in raw_bars:
+                bar = self._parse_bar(rb)
+                if bar:
+                    bars.append(bar)
+
+            bars.sort(key=lambda b: b.time)
+            bars = bars[-count:]
+
+            # Cache in buffer
+            buf_key = f"{ticker}_{timeframe}"
+            self._bar_buffers[buf_key] = deque(bars, maxlen=self._max_bar_buffer)
+
+            log.info(
+                f"[{ticker}] Fetched {len(bars)} {timeframe} bars "
+                f"from Rithmic history"
+            )
+            return bars
+
+        except AttributeError:
+            log.warning(
+                f"[{ticker}] get_time_bars() not available. "
+                f"Bars will accumulate from live stream."
+            )
+            return []
+        except Exception as e:
+            log.warning(f"[{ticker}] Historical bar fetch error: {e}")
+            return []
+
+    # ============================================================
+    # INTERNAL — Streaming Callbacks
+    # ============================================================
+
+    async def _on_tick(self, data: dict) -> None:
+        """Callback for live tick data. Updates tick cache."""
+        try:
+            security_code = data.get("symbol", "")
+            canonical = self._reverse_map.get(security_code)
+            if not canonical:
+                return
+
+            data_type = data.get("data_type")
+            presence = data.get("presence_bits", 0)
+
+            # Preserve existing values
+            existing = self._tick_cache.get(canonical)
+            bid = existing.bid if existing else 0.0
+            ask = existing.ask if existing else 0.0
+            last = existing.last if existing else 0.0
+
+            if data_type == DataType.BBO:
+                if presence & BestBidOfferPresenceBits.BID:
+                    bid = float(data.get("bid_price", bid))
+                if presence & BestBidOfferPresenceBits.ASK:
+                    ask = float(data.get("ask_price", ask))
+            elif data_type == DataType.LAST_TRADE:
+                if presence & LastTradePresenceBits.LAST_TRADE:
+                    last = float(data.get("trade_price", last))
+
+            if bid > 0 or ask > 0 or last > 0:
+                now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
+                spread = round(ask - bid, 6) if (bid > 0 and ask > 0) else 0.0
+                if last == 0 and bid > 0:
+                    last = (bid + ask) / 2
+
+                self._tick_cache[canonical] = TickData(
+                    ticker=canonical,
+                    bid=bid,
+                    ask=ask,
+                    last=last,
+                    time=now,
+                    spread=spread,
+                )
+
+        except Exception as e:
+            log.debug(f"Tick callback error: {e}")
+
+    async def _on_time_bar(self, data: dict) -> None:
+        """Callback for live time bar data. Adds to ring buffer."""
+        try:
+            security_code = data.get("symbol", "")
+            canonical = self._reverse_map.get(security_code)
+            if not canonical:
+                return
+
+            bar = self._parse_bar(data)
+            if not bar:
+                return
+
+            period = data.get("period", 1)
+            tf_key = self._period_to_tf(period)
+            buf_key = f"{canonical}_{tf_key}"
+
+            if buf_key not in self._bar_buffers:
+                self._bar_buffers[buf_key] = deque(
+                    maxlen=self._max_bar_buffer
+                )
+            self._bar_buffers[buf_key].append(bar)
+
+        except Exception as e:
+            log.debug(f"Time bar callback error: {e}")
+
+    # ============================================================
+    # INTERNAL — Helpers
+    # ============================================================
+
+    def _parse_bar(self, data: dict) -> Optional[BarData]:
+        """Parse a raw Rithmic bar dict into a BarData object."""
+        try:
+            open_p = float(data.get("open_price", data.get("open", 0)))
+            high_p = float(data.get("high_price", data.get("high", 0)))
+            low_p = float(data.get("low_price", data.get("low", 0)))
+            close_p = float(data.get("close_price", data.get("close", 0)))
+            volume = int(data.get("volume", 0))
+
+            bar_time = data.get("bar_end_time", data.get("time", ""))
+            if isinstance(bar_time, datetime):
+                time_str = bar_time.strftime("%Y-%m-%dT%H:%M:%S")
+            elif isinstance(bar_time, (int, float)):
+                time_str = datetime.fromtimestamp(
+                    bar_time, tz=timezone.utc
+                ).strftime("%Y-%m-%dT%H:%M:%S")
+            else:
+                time_str = str(bar_time)
+
+            if open_p == 0 and close_p == 0:
+                return None
+
+            return BarData(
+                time=time_str,
+                open=open_p,
+                high=high_p,
+                low=low_p,
+                close=close_p,
+                volume=volume,
+            )
+        except Exception:
+            return None
+
+    @staticmethod
+    def _period_to_tf(period: int) -> str:
+        """Convert minute period to our timeframe string."""
+        tf_map = {1: "M1", 5: "M5", 15: "M15", 30: "M30", 60: "H1", 240: "H4"}
+        return tf_map.get(period, "M1")

backend/providers/tradovate_provider.py

@@ -0,0 +1,454 @@
+# version: v1
+"""
+================================================================================
+Quantum Terminal — Tradovate Provider (POC)
+
+Proof-of-work integration with Tradovate for users who don't run MT5.
+
+What this does:
+  · Authenticates against /auth/accesstokenrequest (REST)
+  · Resolves CFD-style tickers (XAUUSD, US500, ...) → Tradovate futures roots
+    (GC, ES, ...) → front-month contracts (GCM6, ESM6, ...)
+  · Fetches historical bars over the market-data WebSocket (md/getChart)
+
+What this does NOT do (deferred to later phases):
+  · Live tick streaming (comes next)
+  · Order placement (futures orders are quite different from CFDs)
+  · Price conversion between CFD and futures price space
+  · Rollover-adjusted continuous contracts
+  · Settlement/margin accounting
+
+Reversibility: this file is standalone. Nothing in MT5 or the base terminal
+touches it. Delete the file + one route-include line and Tradovate goes
+away entirely.
+================================================================================
+"""
+
+import asyncio
+import json
+import logging
+import time
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta, timezone
+from typing import Any, Dict, List, Optional
+
+log = logging.getLogger("tradovate_provider")
+
+# ─── Dependencies (both already in consumer_venv) ────────────────────────
+try:
+    import httpx
+except ImportError:
+    httpx = None
+    log.warning("httpx not installed — Tradovate provider will be inert")
+
+try:
+    import websockets
+except ImportError:
+    websockets = None
+    log.warning("websockets not installed — Tradovate provider will be inert")
+
+
+# ─── Endpoint config ─────────────────────────────────────────────────────
+ENDPOINTS = {
+    "demo": {
+        "rest":  "https://demo.tradovateapi.com/v1",
+        "md_ws": "wss://md-demo.tradovateapi.com/v1/websocket",
+    },
+    "live": {
+        "rest":  "https://live.tradovateapi.com/v1",
+        "md_ws": "wss://md.tradovateapi.com/v1/websocket",
+    },
+}
+
+
+# ─── CFD → Futures root map (POC v1) ────────────────────────────────────
+# Add more entries as we validate them. Unmapped tickers raise a clean
+# "symbol not supported by Tradovate" error rather than crashing.
+CFD_TO_FUTURES_ROOT = {
+    "XAUUSD": "GC",   # Gold futures (100 oz, COMEX)
+    "XAGUSD": "SI",   # Silver futures (5,000 oz, COMEX)
+    "XTIUSD": "CL",   # Crude oil futures (1,000 bbl, NYMEX)
+    "US500":  "ES",   # E-mini S&P 500 (CME)
+    "USTEC":  "NQ",   # E-mini Nasdaq 100 (CME)
+    "GER40":  "FDAX", # DAX futures (Eurex) — requires exchange subscription
+    "BTCUSD": "BTC",  # Micro Bitcoin or BTC futures (CME)
+    # FX / UK100 / SOLUSD intentionally unmapped for POC.
+}
+
+# ─── Futures month codes ────────────────────────────────────────────────
+MONTH_CODES = {1:"F", 2:"G", 3:"H", 4:"J", 5:"K", 6:"M",
+               7:"N", 8:"Q", 9:"U", 10:"V", 11:"X", 12:"Z"}
+
+# Which months does each product expire in?
+# Quarterlies = Mar/Jun/Sep/Dec, monthly = every month, bi-monthly = even months.
+PRODUCT_EXPIRY_MONTHS = {
+    "ES":  [3, 6, 9, 12],   # E-mini S&P — quarterly
+    "NQ":  [3, 6, 9, 12],
+    "GC":  [2, 4, 6, 8, 10, 12],  # Gold — bi-monthly
+    "SI":  [1, 3, 5, 7, 9, 12],   # Silver — not every month
+    "CL":  list(range(1, 13)),     # Crude — monthly
+    "BTC": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],  # Bitcoin — monthly
+    "FDAX":[3, 6, 9, 12],
+}
+
+
+# ─── Timeframe → Tradovate chart description ────────────────────────────
+def _tf_to_chart_desc(timeframe: str) -> Dict[str, Any]:
+    tf_map = {
+        "M1":  ("MinuteBar",   1),
+        "M5":  ("MinuteBar",   5),
+        "M15": ("MinuteBar",  15),
+        "M30": ("MinuteBar",  30),
+        "H1":  ("MinuteBar",  60),
+        "H4":  ("MinuteBar", 240),
+        "D1":  ("DailyBar",    1),
+    }
+    underlying, size = tf_map.get(timeframe.upper(), ("MinuteBar", 15))
+    return {
+        "underlyingType": underlying,
+        "elementSize": size,
+        "elementSizeUnit": "UnderlyingUnits",
+        "withHistogram": False,
+    }
+
+
+@dataclass
+class TradovateAuth:
+    access_token: str = ""
+    md_access_token: str = ""
+    user_id: int = 0
+    name: str = ""
+    has_live: bool = False
+    user_status: str = ""
+    expires_at: float = 0.0   # epoch seconds
+    md_expires_at: float = 0.0
+
+    def is_valid(self) -> bool:
+        return bool(self.access_token) and time.time() < self.expires_at - 30
+
+
+class TradovateProvider:
+    """POC Tradovate provider. Not a full BaseProvider subclass yet —
+    we'll upgrade to that once the smoke test passes. For now it exposes
+    the methods tradovate_routes.py needs to service the POC endpoint."""
+
+    def __init__(self, config: Dict[str, Any]):
+        self._config = config
+        self._auth = TradovateAuth()
+        self._last_error: str = ""
+
+    # ── Config updates (called from PATCH /api/tradovate/config) ──
+    def update_config(self, new_config: Dict[str, Any]) -> None:
+        self._config.update(new_config or {})
+        # Invalidate cached auth so the next connect uses new creds.
+        self._auth = TradovateAuth()
+
+    @property
+    def connected(self) -> bool:
+        return self._auth.is_valid()
+
+    @property
+    def is_delayed(self) -> bool:
+        # Free Tradovate demo accounts are delayed 10 minutes for most CME
+        # products. hasLive=True means the user has paid live market data;
+        # False means delayed. Real instrument-by-instrument delay info comes
+        # from md/getContract-like queries — POC just surfaces the boolean.
+        return not self._auth.has_live
+
+    @property
+    def status_dict(self) -> Dict[str, Any]:
+        return {
+            "connected":   self.connected,
+            "user_id":     self._auth.user_id,
+            "name":        self._auth.name,
+            "user_status": self._auth.user_status,
+            "has_live":    self._auth.has_live,
+            "delayed":     self.is_delayed,
+            "env":         self._config.get("env", "demo"),
+            "expires_at":  self._auth.expires_at,
+            "error":       self._last_error,
+        }
+
+    # ── Authentication ────────────────────────────────────────────
+    async def authenticate(self) -> Dict[str, Any]:
+        self._last_error = ""
+        if httpx is None:
+            self._last_error = "httpx not installed in venv"
+            return {"success": False, "error": self._last_error}
+        cfg = self._config
+        env = (cfg.get("env") or "demo").lower()
+        if env not in ENDPOINTS:
+            self._last_error = f"unknown env '{env}' (expected demo or live)"
+            return {"success": False, "error": self._last_error}
+        required = ["app_id", "cid", "sec", "username", "password"]
+        missing = [k for k in required if not cfg.get(k)]
+        if missing:
+            self._last_error = f"missing creds: {', '.join(missing)}"
+            return {"success": False, "error": self._last_error}
+
+        url = ENDPOINTS[env]["rest"] + "/auth/accesstokenrequest"
+        body = {
+            "name": cfg["username"],
+            "password": cfg["password"],
+            "appId": cfg["app_id"],
+            "appVersion": cfg.get("app_version", "1.0"),
+            "cid": int(cfg["cid"]),
+            "sec": cfg["sec"],
+            "deviceId": cfg.get("device_id", "QuantumTerminal-consumer-poc"),
+        }
+        try:
+            async with httpx.AsyncClient(timeout=15.0) as client:
+                r = await client.post(url, json=body)
+            if r.status_code != 200:
+                self._last_error = f"HTTP {r.status_code}: {r.text[:200]}"
+                return {"success": False, "error": self._last_error}
+            data = r.json()
+        except Exception as e:
+            self._last_error = f"auth request failed: {e}"
+            log.error(self._last_error)
+            return {"success": False, "error": self._last_error}
+
+        # Tradovate returns error info inside a 200 response for bad creds
+        err_code = data.get("errorText") or data.get("p-ticket")
+        if err_code and not data.get("accessToken"):
+            self._last_error = f"tradovate rejected auth: {err_code}"
+            return {"success": False, "error": self._last_error}
+
+        # Parse token expiration — Tradovate returns ISO8601 in expirationTime
+        def _parse_iso(s):
+            if not s:
+                return time.time() + 4000  # ~66 min fallback
+            try:
+                s = s.replace("Z", "+00:00")
+                return datetime.fromisoformat(s).timestamp()
+            except Exception:
+                return time.time() + 4000
+
+        self._auth = TradovateAuth(
+            access_token   = data.get("accessToken", ""),
+            md_access_token= data.get("mdAccessToken", ""),
+            user_id        = data.get("userId", 0) or 0,
+            name           = data.get("name", "") or cfg["username"],
+            has_live       = bool(data.get("hasLive", False)),
+            user_status    = data.get("userStatus", "") or "",
+            expires_at     = _parse_iso(data.get("expirationTime")),
+            md_expires_at  = _parse_iso(data.get("expirationTime")),
+        )
+        if not self._auth.access_token:
+            self._last_error = "no accessToken in response"
+            return {"success": False, "error": self._last_error}
+        return {"success": True, **self.status_dict}
+
+    def disconnect(self) -> None:
+        self._auth = TradovateAuth()
+        self._last_error = ""
+
+    # ── Symbol resolution ─────────────────────────────────────────
+    def _resolve_root(self, ticker: str) -> Optional[str]:
+        t = (ticker or "").upper()
+        if t in CFD_TO_FUTURES_ROOT:
+            return CFD_TO_FUTURES_ROOT[t]
+        # Pass-through: if the user passes a root directly (e.g. "GC"), accept
+        if t in PRODUCT_EXPIRY_MONTHS:
+            return t
+        return None
+
+    def _front_month_contract(self, root: str, today: Optional[datetime] = None) -> str:
+        """Cheap calendar-based front-month resolver. Good enough for POC.
+        For production we should hit /contract/suggest for accurate roll
+        timing (contracts roll a few days before the last notice date).
+        """
+        months = PRODUCT_EXPIRY_MONTHS.get(root)
+        if not months:
+            # Default to quarterly if unknown
+            months = [3, 6, 9, 12]
+        now = today or datetime.now(timezone.utc)
+        y, m = now.year, now.month
+        # Pick the next expiry month that's >= current month. Add 5-day lead
+        # to avoid the last-trading-day rush — we want the LIQUID contract.
+        lead_day = now.day >= 10
+        candidate_month = None
+        for mm in months:
+            if mm > m or (mm == m and not lead_day):
+                candidate_month = mm
+                break
+        if candidate_month is None:
+            # Wrap to next year's first expiry month
+            candidate_month = months[0]
+            y += 1
+        return f"{root}{MONTH_CODES[candidate_month]}{y % 10}"
+
+    async def resolve_contract_via_api(self, root: str) -> Optional[str]:
+        """Ask Tradovate to suggest the current tradeable contract. Used as
+        a fallback/verification — if this disagrees with the calendar
+        heuristic we prefer this answer."""
+        if httpx is None or not self._auth.access_token:
+            return None
+        url = ENDPOINTS[self._config.get("env", "demo")]["rest"] + "/contract/suggest"
+        try:
+            async with httpx.AsyncClient(timeout=10.0) as client:
+                r = await client.get(
+                    url, params={"t": root, "l": 5},
+                    headers={"Authorization": f"Bearer {self._auth.access_token}"},
+                )
+            if r.status_code != 200:
+                return None
+            contracts = r.json() or []
+            # Return the shortest-name contract (usually the front month).
+            if contracts:
+                contracts.sort(key=lambda c: len(c.get("name", "")))
+                return contracts[0].get("name")
+        except Exception as e:
+            log.warning(f"contract/suggest failed for {root}: {e}")
+        return None
+
+    # ── Market-data WebSocket helpers ─────────────────────────────
+    async def _ws_fetch_bars(self, contract: str, timeframe: str, count: int) -> List[Dict[str, Any]]:
+        """Open a one-shot WebSocket, auth, request bars, return them."""
+        if websockets is None or not self._auth.md_access_token:
+            raise RuntimeError("not authenticated or websockets missing")
+        env = self._config.get("env", "demo")
+        url = ENDPOINTS[env]["md_ws"]
+
+        # Tradovate WebSocket protocol: each message is a plaintext frame
+        #   <endpoint>\n<id>\n\n<body>
+        # Responses come as JSON arrays prefixed by "a" (for "array frames")
+        # and single-letter frames "o" (open), "h" (heartbeat), "c" (close).
+        def _encode(endpoint: str, msg_id: int, body: Any = "") -> str:
+            body_str = json.dumps(body) if not isinstance(body, str) else body
+            return f"{endpoint}\n{msg_id}\n\n{body_str}"
+
+        bars: List[Dict[str, Any]] = []
+        base_price = 0.0
+        tick_size = 0.01
+        tick_mult = 1.0
+
+        try:
+            async with websockets.connect(url, ping_interval=20, ping_timeout=20) as ws:
+                # 1st server frame should be "o"
+                open_frame = await asyncio.wait_for(ws.recv(), timeout=10.0)
+                if not (isinstance(open_frame, str) and open_frame.startswith("o")):
+                    raise RuntimeError(f"unexpected open frame: {open_frame!r}")
+
+                # Authorize with MD token
+                await ws.send(_encode("authorize", 1, self._auth.md_access_token))
+
+                # Request chart
+                chart_req = {
+                    "symbol": contract,
+                    "chartDescription": _tf_to_chart_desc(timeframe),
+                    "timeRange": {"asMuchAsElements": max(1, min(int(count), 2000))},
+                }
+                await ws.send(_encode("md/getChart", 2, chart_req))
+
+                # Read frames until we receive the charts packet for our id
+                deadline = time.time() + 20.0
+                subscription_id: Optional[int] = None
+                while time.time() < deadline and len(bars) < count:
+                    try:
+                        frame = await asyncio.wait_for(ws.recv(), timeout=5.0)
+                    except asyncio.TimeoutError:
+                        break
+                    if not isinstance(frame, str):
+                        continue
+                    # "h" = heartbeat — send one back to stay alive
+                    if frame == "h":
+                        await ws.send("[]")
+                        continue
+                    if frame.startswith("a"):
+                        try:
+                            arr = json.loads(frame[1:])
+                        except Exception:
+                            continue
+                        for item in arr or []:
+                            # Response to md/getChart (id=2) returns subscription id
+                            if item.get("i") == 2 and item.get("s") == 200:
+                                body = item.get("d") or {}
+                                subscription_id = body.get("subscriptionId") or body.get("id")
+                            # Streaming chart data comes with "e": "chart"
+                            if item.get("e") == "chart":
+                                cd = item.get("d") or {}
+                                charts = cd.get("charts") or []
+                                for ch in charts:
+                                    base_price = ch.get("bp", base_price)
+                                    tick_size  = ch.get("ts", tick_size) or tick_size
+                                    tick_mult  = ch.get("tm", tick_mult) or tick_mult
+                                    raw_bars = ch.get("bars") or []
+                                    for b in raw_bars:
+                                        bars.append(_decode_bar(b, base_price, tick_size, tick_mult))
+
+                # Clean up — unsubscribe if we got a subscription id
+                if subscription_id is not None:
+                    try:
+                        await ws.send(_encode("md/cancelChart", 3, {"subscriptionId": subscription_id}))
+                    except Exception:
+                        pass
+        except Exception as e:
+            self._last_error = f"ws fetch failed: {e}"
+            log.error(self._last_error)
+            raise
+
+        return bars[-count:] if len(bars) > count else bars
+
+    async def get_bars(self, ticker: str, timeframe: str = "M15", count: int = 200) -> List[Dict[str, Any]]:
+        """Main entry point. Maps ticker → contract, fetches bars via WS."""
+        if not self._auth.is_valid():
+            raise RuntimeError("not authenticated")
+        root = self._resolve_root(ticker)
+        if not root:
+            raise ValueError(f"{ticker} not mapped to a Tradovate futures root")
+        # Try API-based resolution first, fall back to calendar heuristic.
+        contract = await self.resolve_contract_via_api(root)
+        if not contract:
+            contract = self._front_month_contract(root)
+        bars = await self._ws_fetch_bars(contract, timeframe, count)
+        return bars
+
+
+# ─── Bar decoding helper ──────────────────────────────────────────
+def _decode_bar(b: Dict[str, Any], base_price: float, tick_size: float, tick_mult: float) -> Dict[str, Any]:
+    """Tradovate returns bars with fields encoded as offsets from a base
+    price in tick units. Decode to absolute OHLC floats."""
+    # Some Tradovate responses send bars already in absolute prices; others
+    # send deltas. Handle both by checking if base_price + tick_size make
+    # the output make sense.
+    def _px(val):
+        if val is None:
+            return None
+        # Heuristic: if val is very small (|val| < 1e6) and base_price > 0
+        # and tick_size > 0, treat as offset. Otherwise treat as absolute.
+        if base_price > 0 and tick_size > 0 and abs(val) < 1_000_000:
+            return base_price + (val * tick_size * (tick_mult or 1))
+        return float(val)
+
+    ts = b.get("timestamp") or b.get("t")
+    # timestamp can be ISO string or epoch ms
+    if isinstance(ts, (int, float)):
+        iso = datetime.fromtimestamp(ts / (1000 if ts > 1e11 else 1), tz=timezone.utc) \
+                      .strftime("%Y-%m-%dT%H:%M:%S")
+    elif isinstance(ts, str):
+        iso = ts.replace("Z", "").split(".")[0]
+    else:
+        iso = ""
+
+    return {
+        "time":   iso,
+        "open":   _px(b.get("open"))  or 0.0,
+        "high":   _px(b.get("high"))  or 0.0,
+        "low":    _px(b.get("low"))   or 0.0,
+        "close":  _px(b.get("close")) or 0.0,
+        "volume": int(b.get("upVolume", 0) or 0) + int(b.get("downVolume", 0) or 0)
+                  or int(b.get("volume", 0) or 0),
+    }
+
+
+# ─── Singleton accessor (one provider instance per process) ─────
+_instance: Optional[TradovateProvider] = None
+
+def get_tradovate_provider(config: Optional[Dict[str, Any]] = None) -> TradovateProvider:
+    global _instance
+    if _instance is None:
+        _instance = TradovateProvider(config or {})
+    elif config:
+        _instance.update_config(config)
+    return _instance

backend/pulse_room/__init__.py

@@ -0,0 +1,10 @@
+# version: v1
+"""Pulse Room — Phase 1 foundation.
+
+Reads cached cones/bands JSONs, builds a pulse snapshot,
+serves REST endpoints for the React Pulse Room UI.
+
+See docs/specs/2026-05-04-pulse-room-design.md for full design.
+"""
+
+# Re-exports added as submodules land. Until then this package is empty.

backend/pulse_room/builder.py

@@ -0,0 +1,463 @@
+# version: v4
+"""Pulse snapshot builder.
+
+Reads cached cones/bands JSONs, extracts SD edges as forward curves,
+and writes pulse_snapshot.json to AppData. Atomic write — partial
+runs never corrupt the live snapshot.
+"""
+
+import json
+import logging
+import os
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Optional
+
+from . import state
+
+
+log = logging.getLogger("mk.pulse_room.builder")
+
+
+# ---------- path resolvers ----------
+
+def _default_cache_dir() -> Path:
+    appdata = os.environ.get("APPDATA")
+    if not appdata:
+        appdata = str(Path.home() / ".config")
+    dir_name = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal-v2")
+    return Path(appdata) / dir_name / "cache"
+
+
+def cones_path(ticker: str, cache_dir: Optional[Path] = None) -> Path:
+    cd = cache_dir or _default_cache_dir()
+    return cd / f"{ticker.upper()}_cones.json"
+
+
+def bands_path(ticker: str, cache_dir: Optional[Path] = None) -> Path:
+    cd = cache_dir or _default_cache_dir()
+    return cd / f"GLOBAL_{ticker.lower()}_bands.json"
+
+
+# ---------- safe JSON loader ----------
+
+def load_json(path: Path) -> Optional[dict]:
+    """Load JSON; return None on missing or malformed."""
+    if not path.exists():
+        return None
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            return json.load(f)
+    except (OSError, json.JSONDecodeError) as e:
+        log.warning("load_json failed for %s: %s", path, e)
+        return None
+
+
+# ---------- cone extraction ----------
+
+def _parse_iso(s: str) -> datetime:
+    """Parse a cone-format date like '2026-04-27 00:00' as UTC."""
+    # Cones use 'YYYY-MM-DD HH:MM' (no seconds, no TZ)
+    if "T" in s:
+        s_normalized = s.replace("Z", "+00:00")
+        try:
+            dt = datetime.fromisoformat(s_normalized)
+            if dt.tzinfo is None:
+                dt = dt.replace(tzinfo=timezone.utc)
+            return dt
+        except ValueError:
+            pass
+    return datetime.strptime(s, "%Y-%m-%d %H:%M").replace(tzinfo=timezone.utc)
+
+
+def _date_offsets_minutes(dates: list, base: Optional[datetime] = None) -> list:
+    """Convert a list of date strings to minutes-from-base.
+
+    If `base` is None, uses dates[0] (legacy behavior, kept so existing tests
+    that don't pass a `now` continue to work). When the production code path
+    in `build_snapshot` calls this, it always passes `base=now` so that t_min
+    is minutes-from-computed_at (per spec §3.1) — past timestamps get t_min<0.
+    """
+    parsed = [_parse_iso(d) for d in dates]
+    base_dt = base if base is not None else parsed[0]
+    return [int((d - base_dt).total_seconds() // 60) for d in parsed]
+
+
+def _index_at_offset_minutes(dates: list, target_minutes: int, base: Optional[datetime] = None) -> int:
+    """Return the smallest index whose offset (relative to `base`, default dates[0])
+    is at or after `target_minutes`."""
+    offsets = _date_offsets_minutes(dates, base=base)
+    for i, m in enumerate(offsets):
+        if m >= target_minutes:
+            return i
+    return len(offsets) - 1
+
+
+# ---------- cone origins ----------
+# 8 origin variants — each is a separate cone scope inside the same
+# <TICKER>_cones.json file. All use the gbm model.
+# Format: (json_key, origin_id, display_label)
+# Aligned with the producer's canonical schema:
+#   gbm_now           — live, anchored at last D1 close / current px (uncached)
+#   gbm_curr          — Monday open of current trading week (cached)
+#   gbm_prev          — Monday open of last week (cached)
+#   gbm_week_prev_2   — 2 weeks back (conditionally cached)
+#   gbm_month_curr    — first trading day of current month (cached)
+#   gbm_month_prev    — first trading day of last month (cached)
+#   gbm_month_prev_2  — first trading day of month -2 (cached)
+#   gbm_extreme_high  — bar of highest high in last 45 days (cached)
+#   gbm_extreme_low   — bar of lowest low in last 45 days (cached)
+CONE_ORIGINS = [
+    ("gbm_now",            "now",            "now"),
+    ("gbm_curr",           "current_week",   "current week"),
+    ("gbm_prev",           "prev_week",      "previous week"),
+    ("gbm_week_prev_2",    "week_prev_2",    "week -2"),
+    ("gbm_month_curr",     "current_month",  "current month"),
+    ("gbm_month_prev",     "prev_month",     "previous month"),
+    ("gbm_month_prev_2",   "prev_month_2",   "month -2"),
+    ("gbm_extreme_high",   "extreme_high",   "extreme high"),
+    ("gbm_extreme_low",    "extreme_low",    "extreme low"),
+]
+
+# Priority order for sourcing the canonical 1-day volatility yardstick when
+# multiple origins are available. gbm_curr is the historical canonical key;
+# fall back to gbm_now (this-week scope), then gbm_month_curr.
+_CONE_1D_SD_PRIORITY = ("gbm_curr", "gbm_now", "gbm_month_curr")
+
+
+def _compute_cone_1d_sd(
+    cones_json: dict,
+    now: Optional[datetime] = None,
+) -> Optional[float]:
+    """Compute cone_1d_sd from the highest-priority origin variant present.
+
+    Returns None if no priority variant is available or extraction fails.
+    """
+    for key in _CONE_1D_SD_PRIORITY:
+        v = cones_json.get(key)
+        if not v:
+            continue
+        try:
+            dates = v["dates"]
+            idx_1d = _index_at_offset_minutes(dates, 1440, base=now)
+            up_width = v["sd1_high"][idx_1d] - v["median"][idx_1d]
+            dn_width = v["median"][idx_1d] - v["sd1_low"][idx_1d]
+            return float((up_width + dn_width) / 2.0)
+        except (KeyError, IndexError, TypeError, ValueError):
+            continue
+    return None
+
+
+def extract_cone_elements(
+    cones_json: dict,
+    now: Optional[datetime] = None,
+) -> tuple[list, float]:
+    """Return (elements_list, cone_1d_sd).
+
+    Iterates CONE_ORIGINS and emits 7 series per present origin variant:
+    median, sd1_upper/lower, sd2_upper/lower, sd3_upper/lower. Each
+    element carries an `origin` field (e.g. "now", "current_week").
+    Missing variant keys are silently skipped — some assets may have
+    fewer variants in their cones JSON.
+
+    cone_1d_sd is sourced from the highest-priority variant present
+    (gbm_curr → gbm_now → gbm_month_curr).
+
+    If `now` is provided, t_min values are re-based so t_min=0 corresponds
+    to `now`; past timestamps get t_min<0, future get t_min>0. If `now` is
+    None, falls back to legacy behavior (t_min=0 at dates[0]) so existing
+    test fixtures keep working.
+    """
+    elements = []
+    for json_key, origin_id, display_label in CONE_ORIGINS:
+        v = cones_json.get(json_key)
+        if not v:
+            continue
+        try:
+            dates = v["dates"]
+        except (KeyError, TypeError):
+            continue
+        offsets = _date_offsets_minutes(dates, base=now)
+
+        # Median series → "<label> mean"
+        try:
+            median_series = v["median"]
+        except (KeyError, TypeError):
+            median_series = None
+        if median_series is not None:
+            elements.append({
+                "id": f"co_{origin_id}_median",
+                "family": "CO",
+                "origin": origin_id,
+                "label": f"{display_label} mean",
+                "model": "gbm",
+                "curve": [
+                    {"t_min": int(t), "price": float(p)}
+                    for t, p in zip(offsets, median_series)
+                ],
+            })
+
+        # SD edges
+        for sd in (1, 2, 3):
+            for side, key_suffix in (("upper", "high"), ("lower", "low")):
+                series_key = f"sd{sd}_{key_suffix}"
+                if series_key not in v:
+                    continue
+                series = v[series_key]
+                elements.append({
+                    "id": f"co_{origin_id}_sd{sd}_{side}",
+                    "family": "CO",
+                    "origin": origin_id,
+                    "label": f"{display_label} SD{sd} {side}",
+                    "model": "gbm",
+                    "curve": [
+                        {"t_min": int(t), "price": float(p)}
+                        for t, p in zip(offsets, series)
+                    ],
+                })
+
+    cone_1d_sd = _compute_cone_1d_sd(cones_json, now=now)
+    if cone_1d_sd is None:
+        # Preserve legacy behavior: raise so callers can decide whether
+        # to skip the asset entirely.
+        raise ValueError(
+            "cones JSON has no usable origin variant for cone_1d_sd "
+            "(checked gbm_curr, gbm_now, gbm_month_curr)"
+        )
+
+    return elements, cone_1d_sd
+
+
+# ---------- band extraction ----------
+
+def extract_band_elements(
+    bands_json: dict,
+    now: Optional[datetime] = None,
+) -> list:
+    """Seven band elements: gbm sd1/2/3 × upper/lower + median.
+
+    Every band element carries `origin: "bands"` so the schema is uniform
+    with cone elements. (The radar UI colors by `origin`, not by family.)
+
+    If `now` is provided, t_min values are re-based so t_min=0 corresponds
+    to `now` (per spec §3.1: t_min is minutes from computed_at). Past
+    timestamps get t_min<0, future get t_min>0. If `now` is None, falls
+    back to legacy timestamps[0]-based offsets so existing fixtures work.
+    """
+    timestamps = bands_json["timestamps"]
+    offsets = _date_offsets_minutes(timestamps, base=now)
+    elements = []
+
+    # Median (mean) series
+    if "mean" in bands_json:
+        mean_series = bands_json["mean"]
+        elements.append({
+            "id": "qb_median",
+            "family": "QB",
+            "origin": "bands",
+            "label": "Bands mean",
+            "model": "gbm",
+            "curve": [
+                {"t_min": int(t), "price": float(p)}
+                for t, p in zip(offsets, mean_series)
+            ],
+        })
+
+    # SD edges
+    for sd in (1, 2, 3):
+        for side, key_suffix in (("upper", "high"), ("lower", "low")):
+            series = bands_json[f"gbm_{sd}sd_{key_suffix}"]
+            elements.append({
+                "id": f"qb_sd{sd}_{side}",
+                "family": "QB",
+                "origin": "bands",
+                "label": f"Bands SD{sd} {side}",
+                "model": "gbm",
+                "curve": [
+                    {"t_min": int(t), "price": float(p)}
+                    for t, p in zip(offsets, series)
+                ],
+            })
+    return elements
+
+
+# ---------- pip size lookup ----------
+
+# Phase 1 minimal pip table. Replace with a v2 metadata lookup once the
+# implementer locates the existing per-symbol pip source. Defaulting to
+# 0.01 covers most CFDs reasonably; FX overrides handle the major pairs.
+_PIP_DEFAULTS = {
+    "EURUSD": 0.0001, "GBPUSD": 0.0001, "AUDUSD": 0.0001, "NZDUSD": 0.0001,
+    "USDCAD": 0.0001, "USDCHF": 0.0001, "USDJPY": 0.01,
+    "EURJPY": 0.01, "GBPJPY": 0.01,
+    "XAUUSD": 0.01, "XAGUSD": 0.001,
+    "XTIUSD": 0.01, "BRENT": 0.01,
+    "US500": 0.01, "USTEC": 0.01, "US30": 1.0,
+    "UK100": 0.1, "GER40": 0.1,
+    "BTCUSD": 1.0, "ETHUSD": 0.1, "SOLUSD": 0.01, "XRPUSD": 0.0001,
+    "DXY": 0.001,
+}
+
+
+def _pip_size(symbol: str) -> float:
+    return _PIP_DEFAULTS.get(symbol.upper(), 0.01)
+
+
+# ---------- top-level snapshot build ----------
+
+def build_snapshot(
+    watchlist: dict,
+    cache_dir: Optional[Path] = None,
+    write: bool = True,
+) -> dict:
+    """Build the snapshot dict; optionally persist to AppData.
+
+    Returns the snapshot dict regardless. If `write=True`, writes to
+    %APPDATA%/QuantumTerminal-v2/pulse_snapshot.json atomically.
+
+    Missing or malformed source files for an asset cause that asset
+    to be skipped (with a logged warning). Other assets still build.
+    """
+    cd = cache_dir or _default_cache_dir()
+    # Capture the wall-clock "now" once. Used both as snapshot["computed_at"]
+    # and as the time-origin for re-basing curve t_min values (spec §3.1:
+    # t_min is minutes from computed_at).
+    now = datetime.now(timezone.utc)
+    snapshot = {
+        "version": "1.0",
+        "computed_at": now.isoformat(),
+        "threshold_sd": 0.5,
+        "assets": {},
+    }
+
+    earliest_source_ts = None
+
+    for asset_cfg in watchlist.get("assets", []):
+        ticker = asset_cfg["symbol"]
+        families = asset_cfg.get("families", [])
+        if not families:
+            continue
+
+        elements = []
+        cone_1d_sd = None
+
+        if "CO" in families:
+            cones_file = cones_path(ticker, cache_dir=cd)
+            cones = load_json(cones_file)
+            if cones is None:
+                log.warning("skipping %s: cones JSON missing/malformed", ticker)
+                continue
+            try:
+                cone_elements, cone_1d_sd = extract_cone_elements(cones, now=now)
+                elements.extend(cone_elements)
+            except (KeyError, ValueError) as e:
+                log.warning("skipping %s: cones extraction error: %s", ticker, e)
+                continue
+            # Use the cache file's mtime as the "freshness" timestamp for this
+            # source. The cones JSON itself has no `computed_at` field, so the
+            # previous fallback to `cones["ticker"]` produced a ticker name
+            # (e.g. "XAUUSD") which broke staleness detection downstream.
+            try:
+                ts = datetime.fromtimestamp(
+                    cones_file.stat().st_mtime, tz=timezone.utc
+                ).isoformat()
+                earliest_source_ts = _min_iso(earliest_source_ts, ts)
+            except OSError:
+                pass
+
+        if "QB" in families:
+            bands_file = bands_path(ticker, cache_dir=cd)
+            bands = load_json(bands_file)
+            if bands is None:
+                log.warning("skipping %s bands: bands JSON missing/malformed", ticker)
+                # If cones succeeded, we still include the asset with cones-only.
+                # If cones were never requested, this asset is empty — skip it.
+                if not elements:
+                    continue
+            else:
+                try:
+                    band_elements = extract_band_elements(bands, now=now)
+                    elements.extend(band_elements)
+                except (KeyError, ValueError) as e:
+                    log.warning("skipping %s bands: extraction error: %s", ticker, e)
+                # Track bands file mtime for staleness too.
+                try:
+                    ts = datetime.fromtimestamp(
+                        bands_file.stat().st_mtime, tz=timezone.utc
+                    ).isoformat()
+                    earliest_source_ts = _min_iso(earliest_source_ts, ts)
+                except OSError:
+                    pass
+
+        if not elements:
+            continue
+
+        # If only QB was selected, derive cone_1d_sd from the cones JSON anyway
+        # (we always need a volatility yardstick). This is a conservative
+        # fallback — load cones silently to compute the SD even when not in families.
+        if cone_1d_sd is None:
+            cones_for_sd = load_json(cones_path(ticker, cache_dir=cd))
+            if cones_for_sd is not None:
+                try:
+                    _, cone_1d_sd = extract_cone_elements(cones_for_sd, now=now)
+                except (KeyError, ValueError):
+                    pass
+
+        if cone_1d_sd is None or cone_1d_sd <= 0:
+            log.warning("skipping %s: cone_1d_sd unavailable or invalid", ticker)
+            continue
+
+        snapshot["assets"][ticker] = {
+            "pip_size": _pip_size(ticker),
+            "cone_1d_sd": cone_1d_sd,
+            "selected_families": list(families),
+            "elements": elements,
+        }
+
+    snapshot["source_sync_at"] = earliest_source_ts or snapshot["computed_at"]
+    snapshot["watchlist_hash"] = state.watchlist_hash(watchlist)
+
+    if write:
+        snap_path = state._pulse_dir() / "pulse_snapshot.json"
+        state._atomic_write(snap_path, snapshot)
+        log.info("pulse snapshot written: %d assets", len(snapshot["assets"]))
+
+    return snapshot
+
+
+def _parse_iso_loose(s: str) -> datetime:
+    """Parse various ISO-ish strings to a tz-aware UTC datetime.
+
+    Handles trailing 'Z', timezone-aware ISO, and the cone "YYYY-MM-DD HH:MM"
+    format. Naive inputs are treated as UTC.
+    """
+    s_clean = s.replace("Z", "+00:00")
+    try:
+        dt = datetime.fromisoformat(s_clean)
+    except ValueError:
+        # Fallback for the cone "YYYY-MM-DD HH:MM" format
+        dt = datetime.strptime(s, "%Y-%m-%d %H:%M")
+    if dt.tzinfo is None:
+        dt = dt.replace(tzinfo=timezone.utc)
+    return dt
+
+
+def _min_iso(a: Optional[str], b: Optional[str]) -> Optional[str]:
+    """Return whichever ISO timestamp string is earlier in real time.
+
+    Parses to datetime before comparing (lexicographic comparison was wrong
+    when the two strings used different format conventions, e.g.
+    'YYYY-MM-DD HH:MM' vs 'YYYY-MM-DDTHH:MM:SS+00:00').
+    """
+    if a is None:
+        return b
+    if b is None:
+        return a
+    try:
+        da = _parse_iso_loose(a)
+        db = _parse_iso_loose(b)
+    except (ValueError, TypeError):
+        # If we can't parse either, fall back to lexicographic compare so we
+        # at least return something deterministic rather than crashing.
+        return a if a < b else b
+    return a if da <= db else b

backend/pulse_room/routes.py

@@ -0,0 +1,123 @@
+# version: v2
+"""Pulse Room REST endpoints.
+
+Wire into data_server.py:
+    from pulse_room.routes import create_pulse_router
+    app.include_router(create_pulse_router())
+
+Endpoints:
+    GET  /api/pulse/state       — full state (scanner, watchlist, snapshot, meta)
+    POST /api/pulse/calculate   — rebuild snapshot (sync; 409 if in-flight)
+    PUT  /api/pulse/watchlist   — replace watchlist
+    PUT  /api/pulse/scanner     — set scanner enabled flag
+"""
+
+import asyncio
+import json
+import logging
+from pathlib import Path
+from typing import Any
+
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+from . import builder, state
+# Concurrency: only one Calculate at a time. Lock lives in state.py so
+# sync_hook.py shares the same instance and the two paths can't race.
+from .state import calc_lock as _calc_lock
+
+
+log = logging.getLogger("mk.pulse_room.routes")
+
+
+# ---------- request models ----------
+
+class WatchlistAssetIn(BaseModel):
+    symbol: str
+    families: list[str]
+
+
+class WatchlistIn(BaseModel):
+    assets: list[WatchlistAssetIn]
+
+
+class ScannerIn(BaseModel):
+    enabled: bool
+
+
+# ---------- helpers ----------
+
+def _read_snapshot() -> tuple[dict | None, dict]:
+    """Return (snapshot, meta). Snapshot may be None if not yet built."""
+    path = state._pulse_dir() / "pulse_snapshot.json"
+    if not path.exists():
+        return None, {"exists": False}
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            snap = json.load(f)
+    except (OSError, json.JSONDecodeError):
+        return None, {"exists": False}
+    meta = {
+        "exists": True,
+        "computed_at": snap.get("computed_at"),
+        "source_sync_at": snap.get("source_sync_at"),
+        "watchlist_hash": snap.get("watchlist_hash"),
+        "asset_count": len(snap.get("assets", {})),
+        "size_bytes": path.stat().st_size,
+    }
+    return snap, meta
+
+
+def _build_state_response() -> dict:
+    snap, meta = _read_snapshot()
+    return {
+        "scanner_enabled": state.load_scanner().get("enabled", False),
+        "watchlist": state.load_watchlist(),
+        "snapshot_meta": meta,
+        "snapshot": snap,
+    }
+
+
+# ---------- router ----------
+
+def create_pulse_router() -> APIRouter:
+    router = APIRouter(prefix="/api/pulse", tags=["pulse"])
+
+    @router.get("/state")
+    async def get_state():
+        return _build_state_response()
+
+    @router.put("/scanner")
+    async def put_scanner(body: ScannerIn):
+        state.save_scanner({"enabled": body.enabled})
+        return _build_state_response()
+
+    @router.put("/watchlist")
+    async def put_watchlist(body: WatchlistIn):
+        wl = {"assets": [a.model_dump() for a in body.assets]}
+        state.save_watchlist(wl)
+        return _build_state_response()
+
+    @router.post("/calculate")
+    async def post_calculate():
+        wl = state.load_watchlist()
+        if not wl.get("assets"):
+            raise HTTPException(400, detail="Watchlist is empty")
+
+        if not _calc_lock.acquire(blocking=False):
+            raise HTTPException(409, detail="A calculate is already in progress")
+        try:
+            try:
+                # build_snapshot does sync file I/O; offload to a worker
+                # thread so we don't block the FastAPI event loop (which
+                # would freeze /api/ticks, /ws/prices, etc. for 0.5–3 s).
+                await asyncio.to_thread(builder.build_snapshot, wl, write=True)
+            except Exception as e:
+                log.exception("Calculate failed")
+                raise HTTPException(500, detail=f"Calculate failed: {e}") from e
+        finally:
+            _calc_lock.release()
+
+        return _build_state_response()
+
+    return router

backend/pulse_room/state.py

@@ -0,0 +1,118 @@
+# version: v2
+"""Pulse Room state persistence.
+
+Owns the two AppData JSON files:
+  - pulse_watchlist.json   : per-asset family selection
+  - pulse_state.json       : scanner enabled flag
+
+All writes go through atomic_write() so partial writes can never
+corrupt the live file.
+"""
+
+import hashlib
+import json
+import os
+import tempfile
+import threading
+from pathlib import Path
+from typing import Any
+
+
+# Process-wide lock guarding pulse snapshot rebuilds. Both the user-initiated
+# Calculate (routes.py) and the auto-rebuild after sync (sync_hook.py) acquire
+# this so they cannot race. Defined here (rather than in routes.py) so both
+# importers see the same lock instance.
+calc_lock = threading.Lock()
+
+
+def _appdata_root() -> Path:
+    """Return %APPDATA%/QuantumTerminal-v2 (or whatever MK_APP_DIR_NAME points to).
+
+    This mirrors v2's existing AppData isolation pattern. We import
+    config_manager lazily to avoid a circular import at module load time.
+    """
+    appdata = os.environ.get("APPDATA")
+    if not appdata:
+        # Fallback for non-Windows dev setups.
+        appdata = str(Path.home() / ".config")
+    dir_name = os.environ.get("MK_APP_DIR_NAME", "QuantumTerminal-v2")
+    return Path(appdata) / dir_name
+
+
+def _pulse_dir() -> Path:
+    p = _appdata_root()
+    p.mkdir(parents=True, exist_ok=True)
+    return p
+
+
+def _atomic_write(path: Path, payload: Any) -> None:
+    """Write JSON atomically: tempfile next to target, then rename."""
+    path.parent.mkdir(parents=True, exist_ok=True)
+    fd, tmp_path = tempfile.mkstemp(
+        prefix=path.stem + ".",
+        suffix=".tmp",
+        dir=str(path.parent),
+    )
+    try:
+        with os.fdopen(fd, "w", encoding="utf-8") as f:
+            json.dump(payload, f, indent=2, sort_keys=True)
+        os.replace(tmp_path, path)
+    except Exception:
+        if os.path.exists(tmp_path):
+            os.unlink(tmp_path)
+        raise
+
+
+# ---------- watchlist ----------
+
+_WATCHLIST_DEFAULT = {"assets": []}
+
+
+def load_watchlist() -> dict:
+    path = _pulse_dir() / "pulse_watchlist.json"
+    if not path.exists():
+        return dict(_WATCHLIST_DEFAULT)
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            return json.load(f)
+    except (OSError, json.JSONDecodeError):
+        return dict(_WATCHLIST_DEFAULT)
+
+
+def save_watchlist(wl: dict) -> None:
+    _atomic_write(_pulse_dir() / "pulse_watchlist.json", wl)
+
+
+def watchlist_hash(wl: dict) -> str:
+    """Stable hash regardless of asset order or family-list order."""
+    canonical = {
+        "assets": sorted(
+            (
+                {"symbol": a["symbol"], "families": sorted(a.get("families", []))}
+                for a in wl.get("assets", [])
+            ),
+            key=lambda a: a["symbol"],
+        )
+    }
+    encoded = json.dumps(canonical, sort_keys=True).encode("utf-8")
+    return "sha256:" + hashlib.sha256(encoded).hexdigest()
+
+
+# ---------- scanner ----------
+
+_SCANNER_DEFAULT = {"enabled": False}
+
+
+def load_scanner() -> dict:
+    path = _pulse_dir() / "pulse_state.json"
+    if not path.exists():
+        return dict(_SCANNER_DEFAULT)
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            return json.load(f)
+    except (OSError, json.JSONDecodeError):
+        return dict(_SCANNER_DEFAULT)
+
+
+def save_scanner(s: dict) -> None:
+    _atomic_write(_pulse_dir() / "pulse_state.json", s)

backend/pulse_room/sync_hook.py

@@ -0,0 +1,43 @@
+# version: v2
+"""Sync-completion hook for Pulse Room.
+
+Called by periodic_sync.py after a successful background sync.
+If scanner is ON and watchlist non-empty, rebuild the snapshot
+in a daemon thread (non-blocking to the sync caller).
+"""
+
+import logging
+import threading
+
+from . import builder, state
+
+
+log = logging.getLogger("mk.pulse_room.sync_hook")
+
+
+def on_sync_complete() -> None:
+    """Fire-and-forget: spawn a thread that rebuilds if eligible."""
+    threading.Thread(target=_rebuild_if_eligible, daemon=True).start()
+
+
+def _rebuild_if_eligible() -> None:
+    try:
+        scanner = state.load_scanner()
+        if not scanner.get("enabled"):
+            return
+        wl = state.load_watchlist()
+        if not wl.get("assets"):
+            return
+        # Don't fight a user-initiated Calculate. If the lock is held, skip
+        # this rebuild silently — the user's Calculate will produce a fresh
+        # snapshot momentarily.
+        if not state.calc_lock.acquire(blocking=False):
+            log.info("calc lock held — skipping auto-rebuild")
+            return
+        try:
+            log.info("auto-rebuilding pulse snapshot after sync")
+            builder.build_snapshot(wl, write=True)
+        finally:
+            state.calc_lock.release()
+    except Exception:
+        log.exception("auto-rebuild failed; existing snapshot retained")

backend/regime_routes.py

@@ -0,0 +1,108 @@
+# version: v2
+"""
+regime_routes.py — Consumer terminal HMM regime routes.
+
+v2 — Lit up from stub. Serves the DC HMM-regime cache files produced by the
+     Data Center and synced into %APPDATA%\\QuantumTerminal\\cache\\:
+
+     /api/regime/macro           → GLOBAL_macro_regime.json
+                                   (4-state macro HMM, daily, no ticker dim)
+     /api/regime/transitions     → GLOBAL_regime_transitions.json
+                                   (macro transition analytics — aux, optional)
+     /api/regime/{ticker}        → GLOBAL_<lower>_regime.json
+                                   (per-ticker slow D1 + fast M15; 404 for
+                                    tickers where DC's hmm_regime flag is off)
+
+     Pattern mirrors /api/historical_cones in data_server.py (v17): try the
+     sync client's in-memory store first so periodic-sync populated stores
+     are reused, then fall back to direct disk reads for both `{TICKER}_*`
+     and `GLOBAL_<lower>_*` naming.
+
+     Rule C1: display-only. No computation here — just reads cache JSON and
+     returns it unmodified.
+"""
+import json as _json
+from fastapi import APIRouter, HTTPException
+from pathlib import Path as _Path
+
+
+def create_regime_router(cfg_manager=None, app=None) -> APIRouter:
+    router = APIRouter(tags=["regime"])
+
+    def _read_disk(*candidates):
+        """Return first existing JSON file from candidates, else None."""
+        for p in candidates:
+            if p and p.is_file():
+                try:
+                    return _json.loads(p.read_text(encoding="utf-8"))
+                except Exception as e:
+                    # Corrupt file — surface as 500 to the caller.
+                    raise HTTPException(500, f"corrupt regime file: {p.name} ({e})")
+        return None
+
+    def _cache_dir() -> _Path:
+        try:
+            from data_sync_client import _get_cache_dir
+            return _get_cache_dir()
+        except Exception:
+            return None
+
+    def _try_memory_store(canonical: str, data_type: str):
+        """Ask the sync client's in-memory store. Returns None if absent."""
+        try:
+            from data_sync_client import get_sync_client
+            client = get_sync_client()
+            return client._get_data(canonical, data_type)
+        except Exception:
+            return None
+
+    @router.get("/api/regime/macro")
+    async def get_macro_regime():
+        """Macro HMM regime — 4-state daily (GOLDILOCKS / REFLATION / STAGFLATION / DEFLATION)."""
+        cache = _cache_dir()
+        data = _read_disk(
+            cache / "GLOBAL_macro_regime.json" if cache else None,
+            cache / "macro_regime.json"        if cache else None,
+        )
+        if data is None:
+            raise HTTPException(404, "macro regime not available")
+        return data
+
+    @router.get("/api/regime/transitions")
+    async def get_regime_transitions():
+        """Macro regime transition analytics (7d/30d/90d probs, duration stats)."""
+        cache = _cache_dir()
+        data = _read_disk(
+            cache / "GLOBAL_regime_transitions.json" if cache else None,
+            cache / "regime_transitions.json"        if cache else None,
+        )
+        if data is None:
+            raise HTTPException(404, "regime transitions not available")
+        return data
+
+    @router.get("/api/regime/{ticker}")
+    async def get_ticker_regime(ticker: str):
+        """Per-ticker HMM regime — slow D1 (palette-adaptive) + fast M15 hazard.
+
+        404 is the normal response when the ticker's `hmm_regime` flag is off
+        in the Data Center's universe config — the consumer should treat 404
+        as "not enabled for this ticker" rather than an error.
+        """
+        canonical = ticker.upper()
+        # (1) sync-client in-memory store (populated by sync_all / _load_all_from_cache)
+        mem = _try_memory_store(canonical, "regime")
+        if mem:
+            return mem
+        # (2) direct-disk fallback — handles both naming conventions.
+        cache = _cache_dir()
+        if cache is None:
+            raise HTTPException(503, "cache directory unavailable")
+        data = _read_disk(
+            cache / f"{canonical}_regime.json",
+            cache / f"GLOBAL_{canonical.lower()}_regime.json",
+        )
+        if data is None:
+            raise HTTPException(404, f"no regime data for {canonical}")
+        return data
+
+    return router

backend/requirements.txt

@@ -0,0 +1,9 @@
+# Quantum Terminal — Python Dependencies
+fastapi>=0.104.0
+uvicorn[standard]>=0.24.0
+httpx>=0.25.0
+websockets>=12.0
+watchfiles>=0.21.0
+yfinance>=0.2.30
+MetaTrader5>=5.0.45
+pydantic>=2.5.0

backend/server_config.py

@@ -0,0 +1,173 @@
+"""
+================================================================================
+Quantum Terminal — Data Server Configuration
+================================================================================
+Central config for the FastAPI data server (Phase 3C).
+
+All paths are relative to PROJECT_ROOT for portability (Rule 3).
+MT5 terminal path is read from local_config.ini (gitignored, Rule 3).
+Universe must match the orchestrator's ASSET_UNIVERSE.
+
+Usage:
+    from server_config import ServerConfig, PROJECT_ROOT
+    config = ServerConfig()
+================================================================================
+"""
+
+import configparser
+from pathlib import Path
+from dataclasses import dataclass, field
+from typing import List, Optional, Dict
+
+# ── Portable project root (Rule 3) ──
+PROJECT_ROOT = Path(__file__).resolve().parent
+
+
+# ── Load local_config.ini for machine-specific values (Rule 3) ──
+def _load_mt5_path() -> Optional[str]:
+    """
+    Read MT5 terminal path from local_config.ini.
+    Falls back to None (MT5 auto-detect) if file or key is missing.
+    
+    Expected format in local_config.ini:
+        [mt5]
+        terminal_path = C:/Program Files/MetaTrader 5/terminal64.exe
+    """
+    ini_path = PROJECT_ROOT / "local_config.ini"
+    if not ini_path.exists():
+        return None
+    cp = configparser.ConfigParser()
+    cp.read(str(ini_path))
+    return cp.get("mt5", "terminal_path", fallback=None)
+
+
+MT5_TERMINAL_PATH = _load_mt5_path()
+
+
+# ── Asset universe — must stay in sync with orchestrator.py ──
+ASSET_UNIVERSE = [
+    # Forex
+    "EURUSD", "GBPUSD", "USDJPY", "AUDUSD", "USDCHF", "USDCAD",
+    # Indices
+    "US500", "USTEC", "GER40", "UK100",
+    # Commodities
+    "XAUUSD", "XAGUSD", "XTIUSD",
+]
+
+
+# ── Symbol alias map ──
+# Canonical name → list of possible broker names to try (in order).
+# The adapter tries each until one resolves in the connected MT5 terminal.
+# Only needed for symbols whose canonical name doesn't match the broker.
+# Forex pairs (EURUSD etc.) are universal and don't need aliases.
+SYMBOL_ALIASES: Dict[str, List[str]] = {
+    "US500":  ["US500", "US500.cash", "SPX500", "SP500", "US500Cash", "USA500"],
+    "USTEC":  ["USTEC", "USTEC.cash", "NAS100", "NSDQ100", "USTECCash", "USTECH"],
+    "GER40":  ["GER40", "GER40.cash", "DAX40", "DE40", "GER40Cash", "GDAXI"],
+    "UK100":  ["UK100", "UK100.cash", "FTSE100", "UK100Cash"],
+    "XTIUSD": ["XTIUSD", "USOIL", "WTI", "XTIUSD.cash", "USOIL.cash", "CL"],
+    "XBRUSD": ["XBRUSD", "UKOIL", "BRENT", "XBRUSD.cash", "UKOIL.cash"],
+    "XAUUSD": ["XAUUSD", "GOLD", "XAUUSD.cash"],
+    "XAGUSD": ["XAGUSD", "SILVER", "XAGUSD.cash"],
+}
+
+
+# ── MT5 timeframe mapping (string → MT5 constant) ──
+# These are the actual MetaTrader5 Python API constants.
+# M1-M30 happen to equal their minute value, but H1+ use encoded values.
+MT5_TIMEFRAMES = {
+    "M1":  1,      # mt5.TIMEFRAME_M1
+    "M5":  5,      # mt5.TIMEFRAME_M5
+    "M15": 15,     # mt5.TIMEFRAME_M15
+    "M30": 30,     # mt5.TIMEFRAME_M30
+    "H1":  16385,  # mt5.TIMEFRAME_H1
+    "H4":  16388,  # mt5.TIMEFRAME_H4
+    "D1":  16408,  # mt5.TIMEFRAME_D1
+    "W1":  32769,  # mt5.TIMEFRAME_W1
+}
+
+
+# ── Decimal precision per asset class ──
+ASSET_DECIMALS = {
+    # Forex — 5 decimal places (pipettes)
+    "EURUSD": 5, "GBPUSD": 5, "AUDUSD": 5, "USDCHF": 5, "USDCAD": 5,
+    "USDJPY": 3,
+    # Indices — 2 decimal places
+    "US500": 2, "USTEC": 2, "GER40": 2, "UK100": 2,
+    # Commodities
+    "XAUUSD": 2, "XAGUSD": 3, "XTIUSD": 2,
+}
+
+
+# ── Asset class classification ──
+ASSET_CLASSES = {
+    "EURUSD": "FX", "GBPUSD": "FX", "USDJPY": "FX",
+    "AUDUSD": "FX", "USDCHF": "FX", "USDCAD": "FX",
+    "US500": "INDEX", "USTEC": "INDEX", "GER40": "INDEX", "UK100": "INDEX",
+    "XAUUSD": "COMMODITY", "XAGUSD": "COMMODITY",
+    "XTIUSD": "COMMODITY",
+    # Futures (Rithmic)
+    "ES": "FUTURES", "NQ": "FUTURES", "YM": "FUTURES",
+    "GC": "FUTURES", "SI": "FUTURES", "CL": "FUTURES", "BZ": "FUTURES",
+}
+
+
+@dataclass
+class ServerConfig:
+    """
+    Master configuration for the data server.
+    All file paths are relative to PROJECT_ROOT.
+    MT5 terminal path comes from local_config.ini (Rule 3).
+    """
+
+    # ── Network ──
+    host: str = "127.0.0.1"
+    port: int = 8501
+
+    # ── MT5 ──
+    mt5_terminal_path: Optional[str] = field(default_factory=lambda: MT5_TERMINAL_PATH)
+
+    # ── Universe ──
+    universe: List[str] = field(default_factory=lambda: list(ASSET_UNIVERSE))
+
+    # ── MT5 polling intervals (seconds) ──
+    tick_interval: float = 1.0        # How often to poll latest ticks
+    bar_check_interval: float = 5.0   # How often to check for new closed bars
+
+    # ── State file paths (relative to PROJECT_ROOT) ──
+    terminal_payload: str = "terminal_payload.json"
+    daily_signals: str = "daily_signals.json"
+    signal_lifecycle: str = "signal_lifecycle.json"
+    weekly_state: str = "weekly_state.json"
+    bands_data_dir: str = "bands_data"
+
+    # ── Display defaults ──
+    default_timeframe: str = "M15"
+    default_lookback_bars: int = 200
+
+    # ── Frontend CORS ──
+    cors_origins: List[str] = field(default_factory=lambda: [
+        "http://localhost:3000",
+        "http://127.0.0.1:3000",
+    ])
+
+    # ── Reconnection ──
+    mt5_reconnect_delay: float = 5.0      # Seconds between reconnect attempts
+    mt5_max_reconnect_attempts: int = 10   # Then log error, keep trying
+
+    # ── File watcher ──
+    file_watch_debounce: float = 1.0  # Seconds to debounce rapid file changes
+
+    def resolve_path(self, relative: str) -> Path:
+        """Resolve a config path relative to PROJECT_ROOT."""
+        return PROJECT_ROOT / relative
+
+    @property
+    def watched_files(self) -> List[Path]:
+        """All state files that the file watcher should monitor."""
+        return [
+            self.resolve_path(self.terminal_payload),
+            self.resolve_path(self.daily_signals),
+            self.resolve_path(self.signal_lifecycle),
+            self.resolve_path(self.weekly_state),
+        ]

backend/system_routes.py

@@ -0,0 +1,10 @@
+"""
+system_routes.py — Stub for consumer terminal.
+PRO-only routes. Consumer does not use these endpoints.
+"""
+from fastapi import APIRouter
+
+
+def create_system_router(*args, **kwargs) -> APIRouter:
+    """Return empty router — PRO endpoints not available in consumer."""
+    return APIRouter(tags=["system"])

backend/trade_mapper_routes.py

@@ -0,0 +1,10 @@
+"""
+trade_mapper_routes.py — Stub for consumer terminal.
+PRO-only routes. Consumer does not use these endpoints.
+"""
+from fastapi import APIRouter
+
+
+def create_trade_mapper_router(*args, **kwargs) -> APIRouter:
+    """Return empty router — PRO endpoints not available in consumer."""
+    return APIRouter(tags=["trade_mapper"])

backend/tradovate_routes.py

@@ -0,0 +1,144 @@
+# version: v1
+"""
+tradovate_routes.py — HTTP API shell for the Tradovate POC.
+
+Routes mirror the MT5 management routes so the SettingsPanel can drive
+both providers with the same UX pattern. Tradovate state is held in the
+singleton TradovateProvider from providers/tradovate_provider.py.
+
+    GET    /api/tradovate/config       → read saved creds + status
+    PATCH  /api/tradovate/config       → save creds (env, app_id, cid, sec,
+                                          username, password)
+    POST   /api/tradovate/connect      → authenticate with current creds
+    POST   /api/tradovate/disconnect   → clear tokens
+    GET    /api/tradovate/bars/{ticker}?timeframe=&count=   → fetch bars
+
+Reversibility: this module is standalone. Nothing in MT5 or the base
+terminal references it.  Dropping the include_router() line from
+consumer_startup.py removes every route above in one edit.
+"""
+
+import logging
+from typing import Optional
+
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+from providers.tradovate_provider import get_tradovate_provider
+
+log = logging.getLogger("tradovate_routes")
+
+
+class TradovateConfigBody(BaseModel):
+    env:        Optional[str] = None   # "demo" | "live"
+    app_id:     Optional[str] = None
+    app_version:Optional[str] = None
+    cid:        Optional[str] = None
+    sec:        Optional[str] = None
+    username:   Optional[str] = None
+    password:   Optional[str] = None
+    device_id:  Optional[str] = None
+
+
+def create_tradovate_router(cfg_manager=None) -> APIRouter:
+    router = APIRouter(tags=["tradovate"])
+
+    def _cfg_store() -> dict:
+        """Slice of user_config where Tradovate creds live. Uses
+        cfg_manager's internal _user_config dict (same pattern mt5_routes
+        uses) so edits persist to disk via _save_user_config()."""
+        if cfg_manager is None or not hasattr(cfg_manager, "_user_config"):
+            # Fallback: in-memory only (creds lost on restart)
+            if not hasattr(_cfg_store, "_mem"):
+                _cfg_store._mem = {}
+            store = _cfg_store._mem
+        else:
+            store = cfg_manager._user_config
+        providers = store.setdefault("providers", {}) if isinstance(store, dict) else {}
+        accounts  = providers.setdefault("accounts", {}) if isinstance(providers, dict) else {}
+        tv        = accounts.setdefault("tradovate_default", {
+            "id": "tradovate_default",
+            "type": "tradovate",
+            "label": "Tradovate",
+            "enabled": False,
+            "env": "demo",
+            "app_id": "",
+            "app_version": "1.0",
+            "cid": "",
+            "sec": "",
+            "username": "",
+            "password": "",
+            "device_id": "QuantumTerminal-consumer-poc",
+        })
+        return tv
+
+    def _masked(cfg: dict) -> dict:
+        """Never return secrets to the UI — mask them so we don't leak."""
+        out = dict(cfg or {})
+        for k in ("sec", "password"):
+            if out.get(k):
+                out[k] = "••••••"
+        return out
+
+    # ── GET /api/tradovate/config ──────────────────────────────
+    @router.get("/api/tradovate/config")
+    async def get_config():
+        cfg = _cfg_store()
+        prov = get_tradovate_provider(cfg)
+        return {"config": _masked(cfg), "status": prov.status_dict}
+
+    # ── PATCH /api/tradovate/config ────────────────────────────
+    @router.patch("/api/tradovate/config")
+    async def patch_config(body: TradovateConfigBody):
+        cfg = _cfg_store()
+        updates = {k: v for k, v in body.model_dump().items() if v is not None and v != ""}
+        cfg.update(updates)
+        # Persist via the same internal save hook config_manager uses.
+        try:
+            if cfg_manager and hasattr(cfg_manager, "_save_user_config"):
+                cfg_manager._save_user_config()
+        except Exception as e:
+            log.warning(f"_save_user_config failed: {e}")
+        # Push new values into the live provider instance (clears token cache)
+        get_tradovate_provider().update_config(cfg)
+        return {"config": _masked(cfg), "status": get_tradovate_provider().status_dict}
+
+    # ── POST /api/tradovate/connect ────────────────────────────
+    @router.post("/api/tradovate/connect")
+    async def connect():
+        cfg = _cfg_store()
+        prov = get_tradovate_provider(cfg)
+        result = await prov.authenticate()
+        return result
+
+    # ── POST /api/tradovate/disconnect ─────────────────────────
+    @router.post("/api/tradovate/disconnect")
+    async def disconnect():
+        get_tradovate_provider().disconnect()
+        return {"success": True, "status": get_tradovate_provider().status_dict}
+
+    # ── GET /api/tradovate/bars/{ticker} ───────────────────────
+    @router.get("/api/tradovate/bars/{ticker}")
+    async def get_bars(ticker: str, timeframe: str = "M15", count: int = 200):
+        prov = get_tradovate_provider()
+        if not prov.connected:
+            raise HTTPException(503, {"error": "tradovate_not_connected",
+                                       "message": "Connect to Tradovate in Settings → Providers first."})
+        try:
+            bars = await prov.get_bars(ticker, timeframe, count)
+        except ValueError as e:
+            # Unmapped ticker — clean 404
+            raise HTTPException(404, {"error": "symbol_not_supported",
+                                       "message": str(e)})
+        except Exception as e:
+            raise HTTPException(500, {"error": "tradovate_bars_failed",
+                                       "message": str(e)})
+        return {
+            "ticker":    ticker.upper(),
+            "timeframe": timeframe,
+            "bars":      bars,
+            "delayed":   prov.is_delayed,
+            "count":     len(bars),
+        }
+
+    return router

backend/version_info.py

@@ -0,0 +1,81 @@
+# version: v2 (Quantum Terminal)
+"""
+version_info.py — Quantum Terminal version endpoint.
+
+Exposes:
+    GET /api/version
+        → {
+            current:          "1.0.0",
+            latest:           "1.0.0",
+            update_available: false,
+            download_url:     "https://github.com/YourUsername/QuantumTerminal/releases",
+            released_at:      "",
+            banner_message:   "",
+            telegram_url:     "",
+            checked_at:       "2026-05-21T00:00:00Z",
+          }
+
+Behavior:
+    - current reads VERSION from bundled data (PyInstaller) or repo root (dev).
+    - No remote update checking in open-source version.
+"""
+
+from __future__ import annotations
+
+import logging
+import sys
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+from fastapi import APIRouter
+
+log = logging.getLogger("qt.version")
+
+DOWNLOAD_BASE = "https://github.com/YourUsername/QuantumTerminal/releases"
+CACHE_TTL_SEC = 600  # 10 min
+
+
+def _read_bundled_version() -> str:
+    """Load VERSION from PyInstaller bundle or dev repo root. Falls back to '1.0.0'."""
+    candidates = []
+    if getattr(sys, "frozen", False):
+        candidates.append(Path(getattr(sys, "_MEIPASS", ".")) / "VERSION")
+    candidates.append(Path(__file__).resolve().parents[2] / "VERSION")  # repo root
+    candidates.append(Path(__file__).resolve().parents[1] / "VERSION")
+    candidates.append(Path.cwd() / "VERSION")
+
+    for p in candidates:
+        try:
+            if p.is_file():
+                v = p.read_text(encoding="utf-8").strip()
+                if v:
+                    return v
+        except Exception:
+            continue
+    log.warning("VERSION file not found — using 1.0.0")
+    return "1.0.0"
+
+
+CURRENT_VERSION = _read_bundled_version()
+log.info(f"Quantum Terminal version: {CURRENT_VERSION}")
+
+
+def create_version_router() -> APIRouter:
+    router = APIRouter(tags=["version"])
+
+    @router.get("/api/version")
+    async def api_version(refresh: bool = False):
+        return {
+            "current": CURRENT_VERSION,
+            "latest": CURRENT_VERSION,
+            "update_available": False,
+            "download_url": DOWNLOAD_BASE,
+            "released_at": "",
+            "banner_message": "",
+            "telegram_url": "",
+            "checked_at": datetime.now(timezone.utc).isoformat(),
+        }
+
+    return router

electron_shell/main.js

@@ -0,0 +1,698 @@
+// version: v9
+// v9 — F-24 (Security wave 2): three new IPC handlers for encrypted
+//      credential storage via Electron safeStorage. Replaces the
+//      consumer-side plaintext password in localStorage (the audit's
+//      F-24 vulnerability). safeStorage uses the OS keychain — DPAPI on
+//      Windows, Keychain on macOS, libsecret on Linux. Encrypted blob
+//      written to `app.getPath('userData')/mk_credentials.json` (same
+//      Electron userData dir as session storage). LoginScreen.jsx v50
+//      consumes these via window.mkElectron.secureSavePassword/
+//      secureLoadPassword/secureClearPassword and runs a one-time
+//      migration that reads any legacy localStorage.mk_saved_pass,
+//      encrypts it, then deletes the plaintext. Bus thread:
+//      security-audit-2026-05-09 + terminal_server_vps msg #166.
+// v8 — Ctrl+= / Ctrl+- / Ctrl+0 zoom shortcuts + external-URL routing
+//      (2026-05-08). Two operator-requested changes:
+//       (1) Restored browser-style font zoom that v1 had natively (browser
+//           mode). Wired via webContents.setZoomLevel; Ctrl+0 resets to
+//           100%. Range clamped to [-8, +9] (V8's safe zone). Applied to
+//           BOTH the main window and detached windows. Existing F12 dev-
+//           tools toggle stays dev-only; zoom is always-on.
+//       (2) Added setWindowOpenHandler to BrowserWindows so window.open()
+//           calls and <a target="_blank"> clicks route to the OS default
+//           browser via shell.openExternal instead of spawning an embedded
+//           Electron child window. Specific motivator: the "OPEN FREE
+//           DEMO ACCOUNT" button in SettingsPanel was opening Tickmill
+//           inside a stripped-down Electron sub-window.
+// v7 — Drop the /S NSIS silent flag from the installer spawn. Electron-
+//      builder's oneClick:true installer is already near-silent (small
+//      progress window only) AND its .onInstSuccess auto-launches the
+//      new exe. The /S flag put NSIS into truly-silent mode which
+//      ALSO suppressed the auto-launch hook — so the update finished
+//      with no terminal running, requiring the user to click the
+//      shortcut. With /S dropped, oneClick handles both: silent UX
+//      (small NSIS progress only) + auto-relaunch.
+// v6 — Silent NSIS install. The installer is now spawned with /S and
+//      windowsHide: true so no NSIS GUI flashes during update. Combined
+//      with oneClick:true in electron-builder NSIS config, the installer
+//      runs silently and auto-launches the new exe on completion.
+// v5 — IPC: mk:download-and-install. Replaces the old "open download URL
+//      in a new window" flow which spawned a blank Electron child window
+//      and then chained Chromium's download dialog. Now the URL is fetched
+//      directly via https/http (with redirects), streamed to %TEMP%, the
+//      installer is spawned detached, and the running terminal quits so
+//      NSIS can replace the install dir without "in use" conflicts.
+// Quantum Terminal Consumer — Electron main process
+// =============================================================================
+// Approach: minimal-risk, maximum-reuse.
+//
+//   1. Spawn the existing Python backend (PyInstaller-built data_server.exe in
+//      production, or `python data_server.py` from venv_consumer in dev) as a
+//      hidden child process — `windowsHide: true` so the user sees no console.
+//
+//   2. Wait for /api/health to come up.
+//
+//   3. Open a single BrowserWindow pointing at http://127.0.0.1:8502/ — same
+//      URL the browser opens today. All React fetch('/api/...') calls resolve
+//      relative to that host, no frontend code changes required.
+//
+//   4. On window close: kill the backend child cleanly so no zombie process.
+//
+// This is intentionally a thin shell. The whole React/LWC/data pipeline is
+// untouched — same code as production. If this proves stable, follow-ups can
+// switch the renderer to load from a local file:// build for full offline /
+// no-localhost-port operation.
+
+const { app, BrowserWindow, dialog, Menu, ipcMain, shell, safeStorage } = require("electron"); // v9: safeStorage for F-24 credential encryption
+const { spawn } = require("child_process");
+const path = require("path");
+const fs   = require("fs");
+const http = require("http");
+const https = require("https");
+const os    = require("os");
+const { URL } = require("url");
+
+// v2 uses port 8503 (v1 uses 8502) so both can run simultaneously without
+// conflict. Explicitly passed to the backend via --port so server_config
+// defaults can't override.
+const BACKEND_PORT = 8503;
+const BACKEND_URL  = `http://127.0.0.1:${BACKEND_PORT}`;
+const HEALTH_URL   = `${BACKEND_URL}/api/health`;
+const READY_TIMEOUT_MS = 60_000;   // give the backend up to 60s to come up
+const HEALTH_POLL_MS   = 500;
+
+let backendProc = null;
+let mainWindow  = null;
+let isQuitting  = false;
+
+// Registry of detached child windows (right-click "Open in New Window" → page).
+// `parent: mainWindow` causes Electron to auto-close these when main closes,
+// so we just need to track membership for cleanup + future broadcast features.
+const detachedWindows = new Set();
+
+// Allowlist must match the renderer's VALID_PAGES in useDetachedMode.js.
+const VALID_DETACHED_PAGES = new Set(["quant", "fundamental", "flow", "options", "stress", "learn"]);
+
+// ── 1. Locate the backend ─────────────────────────────────────────────────
+//   Production (after `npm run dist`):
+//     extraResources copies `source code/backend/` to `<resourcesDir>/backend/`
+//     PyInstaller exe is expected at `<resourcesDir>/backend/dist/data_server/data_server.exe`
+//     Falling back to `<resourcesDir>/backend/data_server.exe` if the user
+//     copied the PyInstaller output one level up.
+//   Development (`npm start` or `npm run start:dev`):
+//     Look for venv_consumer + data_server.py beside the project tree.
+function findBackendCommand() {
+  const isDev = !app.isPackaged;
+  const projectRoot = isDev
+    ? path.resolve(__dirname, "..")             // v2/ folder
+    : process.resourcesPath;                    // electron resources/ in installed app
+
+  const candidates = [];
+
+  // Spawn launcher.py — it imports data_server.app, mounts the React build
+  // at /, and runs uvicorn. Spawning data_server.py directly only serves
+  // /api/* and 404s on the root URL.
+  // Port + no-browser are env vars (MK_PORT, MK_NO_BROWSER), set in startBackend().
+
+  // Production: prebuilt PyInstaller exe inside extraResources
+  if (!isDev) {
+    candidates.push({
+      cmd:  path.join(projectRoot, "backend", "dist", "data_server", "data_server.exe"),
+      args: [],
+      cwd:  path.join(projectRoot, "backend", "dist", "data_server"),
+      label: "packaged: extraResources/backend/dist/data_server/",
+    });
+    candidates.push({
+      cmd:  path.join(projectRoot, "backend", "data_server.exe"),
+      args: [],
+      cwd:  path.join(projectRoot, "backend"),
+      label: "packaged: extraResources/backend/",
+    });
+  }
+
+  // Dev: run launcher.py via v2's own venv. Independent of v1 — no fallbacks.
+  if (isDev) {
+    const v2Venv     = path.join(projectRoot, "source code", "venv_consumer", "Scripts", "python.exe");
+    const launcherPy = path.join(projectRoot, "source code", "backend", "launcher.py");
+    if (fs.existsSync(v2Venv) && fs.existsSync(launcherPy)) {
+      candidates.push({
+        cmd: v2Venv, args: [launcherPy],
+        cwd: path.dirname(launcherPy),
+        label: "dev: v2 venv + launcher.py",
+      });
+    }
+    // System python only if v2 venv isn't built yet — useful for the
+    // "I cloned the repo, haven't run the venv setup" first-run case.
+    if (fs.existsSync(launcherPy)) {
+      candidates.push({
+        cmd: "python", args: [launcherPy],
+        cwd: path.dirname(launcherPy),
+        label: "dev: system python (no venv detected, may fail without deps installed)",
+      });
+    }
+  }
+
+  for (const c of candidates) {
+    if (c.cmd === "python" || (c.cmd && fs.existsSync(c.cmd))) {
+      console.log(`[electron] backend: ${c.label}`);
+      return c;
+    }
+  }
+  return null;
+}
+
+// ── 2. Spawn backend ──────────────────────────────────────────────────────
+function startBackend() {
+  const target = findBackendCommand();
+  if (!target) {
+    dialog.showErrorBox(
+      "Backend not found",
+      "Could not locate the Python backend. Expected a PyInstaller-built data_server.exe inside extraResources/backend/, or a dev venv_consumer with data_server.py."
+    );
+    app.exit(1);
+    return;
+  }
+
+  console.log(`[electron] spawning: ${target.cmd} ${target.args.join(" ")}`);
+  backendProc = spawn(target.cmd, target.args, {
+    cwd: target.cwd,
+    windowsHide: true,
+    detached: false,
+    stdio: ["ignore", "pipe", "pipe"],
+    env: {
+      ...process.env,
+      PYTHONUNBUFFERED: "1",
+      MK_PORT: String(BACKEND_PORT),       // launcher.py reads this
+      MK_NO_BROWSER: "1",                  // skip auto-launch of system browser
+      MK_APP_DIR_NAME: "QuantumTerminal-v2",      // isolate AppData from v1 (auth, cache, account state)
+    },
+  });
+
+  backendProc.stdout.on("data", (d) => console.log(`[backend] ${d.toString().trimEnd()}`));
+  backendProc.stderr.on("data", (d) => console.error(`[backend ERR] ${d.toString().trimEnd()}`));
+  backendProc.on("exit", (code, signal) => {
+    console.log(`[electron] backend exited code=${code} signal=${signal}`);
+    if (!isQuitting) {
+      // Backend died unexpectedly — close the window to make it visible.
+      if (mainWindow) mainWindow.close();
+    }
+  });
+  backendProc.on("error", (err) => {
+    console.error("[electron] backend spawn error:", err);
+    dialog.showErrorBox("Backend spawn failed", String(err));
+  });
+}
+
+// ── 3. Poll /api/health ───────────────────────────────────────────────────
+function waitForBackend() {
+  return new Promise((resolve, reject) => {
+    const startTs = Date.now();
+    const tick = () => {
+      const req = http.get(HEALTH_URL, (res) => {
+        if (res.statusCode === 200) return resolve();
+        retry();
+      });
+      req.on("error", retry);
+      req.setTimeout(2000, () => { req.destroy(); retry(); });
+    };
+    const retry = () => {
+      if (Date.now() - startTs > READY_TIMEOUT_MS) return reject(new Error("backend health check timed out"));
+      setTimeout(tick, HEALTH_POLL_MS);
+    };
+    tick();
+  });
+}
+
+// ── 4. Create the window ──────────────────────────────────────────────────
+function createWindow() {
+  mainWindow = new BrowserWindow({
+    width:  1600,
+    height: 1000,
+    minWidth:  1024,
+    minHeight: 720,
+    icon: path.join(app.isPackaged ? process.resourcesPath : __dirname, "..", "icon.ico"),
+    backgroundColor: "#000000",
+    show: false,
+    autoHideMenuBar: true,
+    webPreferences: {
+      contextIsolation: true,
+      nodeIntegration:  false,
+      preload: path.join(__dirname, "preload.js"),
+      // Allow the renderer to talk to http://127.0.0.1:8502 — no mixed-content
+      // headache because Electron's default security model already permits
+      // localhost http when the page itself is loaded over http.
+    },
+  });
+
+  // Strip the default File/Edit/View menu — keeps the chrome looking like an app.
+  // Comment this out if you want devtools/menu access during testing.
+  Menu.setApplicationMenu(null);
+
+  mainWindow.loadURL(BACKEND_URL);
+  mainWindow.once("ready-to-show", () => mainWindow.show());
+  mainWindow.on("closed", () => { mainWindow = null; });
+
+  // v8: route window.open() and target=_blank links to OS default browser
+  //     instead of spawning an embedded Electron sub-window.
+  mainWindow.webContents.setWindowOpenHandler(({ url }) => {
+    if (/^https?:\/\//.test(url)) {
+      shell.openExternal(url).catch(err =>
+        console.error("[electron] shell.openExternal failed:", err));
+    }
+    return { action: "deny" };
+  });
+
+  // v8: keyboard shortcuts. Ctrl+= / Ctrl+- / Ctrl+0 zoom (always on);
+  //     F12 devtools (dev mode only). Single listener so we don't
+  //     register two before-input-event handlers on the same wc.
+  mainWindow.webContents.on("before-input-event", (event, input) => {
+    if (input.type !== "keyDown") return;
+    // F12 dev tools — gated to dev / unpackaged builds.
+    if (input.key === "F12" &&
+        (process.env.ELECTRON_DEV === "1" || !app.isPackaged)) {
+      mainWindow.webContents.toggleDevTools();
+      return;
+    }
+    // Ctrl+= / Ctrl+- / Ctrl+0 zoom — always on, production included.
+    if (input.control || input.meta) {
+      const wc = mainWindow.webContents;
+      if (input.key === "=" || input.key === "+") {
+        wc.setZoomLevel(Math.min(wc.getZoomLevel() + 0.5, 9));
+        event.preventDefault();
+      } else if (input.key === "-") {
+        wc.setZoomLevel(Math.max(wc.getZoomLevel() - 0.5, -8));
+        event.preventDefault();
+      } else if (input.key === "0") {
+        wc.setZoomLevel(0);
+        event.preventDefault();
+      }
+    }
+  });
+}
+
+// ── 5. Lifecycle ──────────────────────────────────────────────────────────
+app.whenReady().then(async () => {
+  startBackend();
+  try {
+    await waitForBackend();
+    console.log("[electron] backend healthy — opening window");
+    createWindow();
+  } catch (e) {
+    console.error("[electron] giving up:", e);
+    dialog.showErrorBox("Backend failed to start", String(e));
+    app.exit(1);
+  }
+});
+
+app.on("window-all-closed", () => {
+  isQuitting = true;
+  app.quit();
+});
+
+app.on("before-quit", () => {
+  isQuitting = true;
+  if (backendProc && !backendProc.killed) {
+    try { backendProc.kill("SIGTERM"); } catch {}
+    setTimeout(() => {
+      if (backendProc && !backendProc.killed) { try { backendProc.kill("SIGKILL"); } catch {} }
+    }, 3000);
+  }
+});
+
+// ── IPC: secure credential storage (F-24, v9) ──────────────────────────
+// safeStorage encrypts via OS keychain (DPAPI on Windows, Keychain on
+// macOS, libsecret on Linux). Encrypted blob stored at:
+//   path.join(app.getPath('userData'), 'mk_credentials.json')
+// File shape: { email: <plain>, passwordEncryptedB64: <base64 of encrypted Buffer> }
+// Email kept plain (not sensitive — visible on screen during entry).
+// Password encrypted at rest. LoginScreen.jsx v50 calls these via
+// window.mkElectron.secureSavePassword/secureLoadPassword/secureClearPassword.
+
+function _credsFilePath() {
+  return path.join(app.getPath("userData"), "mk_credentials.json");
+}
+
+ipcMain.handle("mk:secure-save-password", async (_event, { email, password }) => {
+  try {
+    if (!safeStorage.isEncryptionAvailable()) {
+      return { ok: false, error: "encryption_unavailable" };
+    }
+    if (typeof password !== "string" || password.length === 0) {
+      return { ok: false, error: "empty_password" };
+    }
+    const encrypted = safeStorage.encryptString(password);
+    const payload = {
+      email: typeof email === "string" ? email : "",
+      passwordEncryptedB64: encrypted.toString("base64"),
+      savedAt: new Date().toISOString(),
+    };
+    await fs.promises.writeFile(_credsFilePath(), JSON.stringify(payload), "utf8");
+    return { ok: true };
+  } catch (e) {
+    console.error("[electron] secure-save-password failed:", e);
+    return { ok: false, error: String(e?.message || e) };
+  }
+});
+
+ipcMain.handle("mk:secure-load-password", async () => {
+  try {
+    const file = _credsFilePath();
+    if (!fs.existsSync(file)) return { ok: true, email: "", password: "" };
+    if (!safeStorage.isEncryptionAvailable()) {
+      return { ok: false, error: "encryption_unavailable" };
+    }
+    const raw = await fs.promises.readFile(file, "utf8");
+    const data = JSON.parse(raw);
+    if (!data?.passwordEncryptedB64) {
+      return { ok: true, email: data?.email || "", password: "" };
+    }
+    const encrypted = Buffer.from(data.passwordEncryptedB64, "base64");
+    const password = safeStorage.decryptString(encrypted);
+    return { ok: true, email: data.email || "", password };
+  } catch (e) {
+    console.error("[electron] secure-load-password failed:", e);
+    return { ok: false, error: String(e?.message || e) };
+  }
+});
+
+ipcMain.handle("mk:secure-clear-password", async () => {
+  try {
+    const file = _credsFilePath();
+    if (fs.existsSync(file)) await fs.promises.unlink(file);
+    return { ok: true };
+  } catch (e) {
+    console.error("[electron] secure-clear-password failed:", e);
+    return { ok: false, error: String(e?.message || e) };
+  }
+});
+
+// ── IPC: save screenshot via native dialog ─────────────────────────────
+// Renderer sends raw PNG bytes; main pops a native dialog + writes the file.
+// Bypasses Chromium's download manager → instant dialog, no base64 hop.
+ipcMain.handle("mk:save-screenshot", async (_event, { suggestedName, bytes }) => {
+  try {
+    const win = BrowserWindow.fromWebContents(_event.sender) || mainWindow;
+    const { canceled, filePath } = await dialog.showSaveDialog(win, {
+      title: "Save chart screenshot",
+      defaultPath: suggestedName || "chart.png",
+      filters: [{ name: "PNG image", extensions: ["png"] }],
+    });
+    if (canceled || !filePath) return { saved: false };
+    // bytes arrives as Uint8Array (preload wraps ArrayBuffer). Buffer.from
+    // accepts Uint8Array and writes raw bytes — no base64 conversion.
+    await fs.promises.writeFile(filePath, Buffer.from(bytes));
+    return { saved: true, path: filePath };
+  } catch (e) {
+    console.error("[electron] save-screenshot failed:", e);
+    return { saved: false, error: String(e) };
+  }
+});
+
+// ── IPC: fetch Trump-related news from Google News RSS ───────────────
+// Renderer's `fetch` can't reach news.google.com (no CORS). Main process
+// fetches the RSS, parses items with regex (no XML lib dep), caches for
+// 5 minutes so repeated toast-opens don't spam the upstream.
+let trumpNewsCache = { items: null, fetchedAt: 0 };
+const TRUMP_CACHE_MS = 5 * 60 * 1000;
+
+function decodeHtml(s) {
+  return String(s || "")
+    .replace(/&amp;/g,  "&")
+    .replace(/&lt;/g,   "<")
+    .replace(/&gt;/g,   ">")
+    .replace(/&quot;/g, '"')
+    .replace(/&#39;/g,  "'")
+    .replace(/&apos;/g, "'");
+}
+function parseRssItems(xml) {
+  const items = [];
+  const itemRe = /<item>([\s\S]*?)<\/item>/g;
+  let m;
+  while ((m = itemRe.exec(xml)) !== null) {
+    const block = m[1];
+    const titleMatch   = block.match(/<title>(?:<!\[CDATA\[)?([\s\S]*?)(?:\]\]>)?<\/title>/);
+    const linkMatch    = block.match(/<link>([\s\S]*?)<\/link>/);
+    const dateMatch    = block.match(/<pubDate>([\s\S]*?)<\/pubDate>/);
+    const sourceMatch  = block.match(/<source[^>]*>([\s\S]*?)<\/source>/);
+    items.push({
+      title:   decodeHtml((titleMatch  ? titleMatch[1]  : "").trim()),
+      link:    (linkMatch  ? linkMatch[1]  : "").trim(),
+      pubDate: (dateMatch  ? dateMatch[1]  : "").trim(),
+      source:  decodeHtml((sourceMatch ? sourceMatch[1] : "").trim()),
+    });
+  }
+  return items;
+}
+
+ipcMain.handle("mk:fetch-trump-news", async (_event, opts) => {
+  const force = opts && opts.force;
+  const now = Date.now();
+  if (!force && trumpNewsCache.items && now - trumpNewsCache.fetchedAt < TRUMP_CACHE_MS) {
+    return {
+      ok: true, items: trumpNewsCache.items,
+      fetchedAt: trumpNewsCache.fetchedAt, cached: true,
+    };
+  }
+  try {
+    const url = "https://news.google.com/rss/search?q=Trump&hl=en-US&gl=US&ceid=US:en";
+    const res = await fetch(url, {
+      headers: { "User-Agent": "Mozilla/5.0 Quantum Terminal Consumer Terminal" },
+    });
+    if (!res.ok) {
+      return { ok: false, error: `HTTP ${res.status} from Google News` };
+    }
+    const xml = await res.text();
+    // v4: sort by pubDate descending (latest first) BEFORE slicing so we
+    //     never drop fresher items just because Google's RSS shipped them
+    //     out of order. Items without a parseable date sink to the bottom.
+    const all = parseRssItems(xml);
+    all.sort((a, b) => {
+      const ta = new Date(a.pubDate).getTime();
+      const tb = new Date(b.pubDate).getTime();
+      const sa = isNaN(ta) ? -Infinity : ta;
+      const sb = isNaN(tb) ? -Infinity : tb;
+      return sb - sa;
+    });
+    const items = all.slice(0, 30);
+    trumpNewsCache = { items, fetchedAt: now };
+    return { ok: true, items, fetchedAt: now, cached: false };
+  } catch (e) {
+    console.error("[electron] fetch-trump-news failed:", e);
+    return { ok: false, error: String(e) };
+  }
+});
+
+// ── IPC: open URL in user's default browser ──────────────────────────
+ipcMain.handle("mk:open-external", async (_event, url) => {
+  try {
+    if (typeof url !== "string" || !/^https?:\/\//.test(url)) {
+      return { ok: false, error: "invalid url" };
+    }
+    await shell.openExternal(url);
+    return { ok: true };
+  } catch (e) {
+    return { ok: false, error: String(e) };
+  }
+});
+
+// ── v5: IPC — download installer + auto-launch + quit terminal ────────
+// Renderer (UpdateAvailableModal) calls this with the installer URL.
+// We follow redirects (max 5 hops), stream to %TEMP%\mk-trades-update.exe
+// emitting `mk:installer-progress` events to the renderer for the progress
+// UI, then spawn the .exe detached and quit the terminal so NSIS can
+// replace the install directory without file-in-use errors.
+function _httpGet(url, redirectsLeft = 5) {
+  return new Promise((resolve, reject) => {
+    let parsed;
+    try { parsed = new URL(url); } catch (e) { return reject(e); }
+    const lib = parsed.protocol === "https:" ? https : http;
+    const req = lib.get(url, { headers: { "User-Agent": "MK-TRADES-Updater" } }, (res) => {
+      const code = res.statusCode || 0;
+      if ([301, 302, 303, 307, 308].includes(code)) {
+        if (redirectsLeft <= 0) return reject(new Error("too many redirects"));
+        const loc = res.headers.location;
+        res.resume();
+        if (!loc) return reject(new Error("redirect without Location"));
+        const next = loc.startsWith("http") ? loc : new URL(loc, parsed).toString();
+        return _httpGet(next, redirectsLeft - 1).then(resolve, reject);
+      }
+      if (code < 200 || code >= 300) {
+        res.resume();
+        return reject(new Error(`HTTP ${code}`));
+      }
+      resolve(res);
+    });
+    req.on("error", reject);
+    req.setTimeout(120_000, () => { try { req.destroy(new Error("timeout")); } catch {} });
+  });
+}
+
+ipcMain.handle("mk:download-and-install", async (event, url) => {
+  if (typeof url !== "string" || !/^https?:\/\//.test(url)) {
+    return { ok: false, error: "invalid url" };
+  }
+  // Path: %TEMP%\mk-trades-update-<timestamp>.exe (timestamp avoids
+  // file-locked rename-on-rerun scenarios).
+  const fname = `mk-trades-update-${Date.now()}.exe`;
+  const dest  = path.join(os.tmpdir(), fname);
+  try {
+    const res = await _httpGet(url);
+    const total = parseInt(res.headers["content-length"] || "0", 10) || 0;
+    let received = 0;
+    let lastEmit = 0;
+    const out = fs.createWriteStream(dest);
+    await new Promise((resolve, reject) => {
+      res.on("data", (chunk) => {
+        received += chunk.length;
+        const now = Date.now();
+        // Throttle progress events to ~6/s so renderer doesn't get hammered.
+        if (now - lastEmit > 160) {
+          lastEmit = now;
+          try {
+            event.sender.send("mk:installer-progress", {
+              received, total,
+              pct: total > 0 ? received / total : null,
+            });
+          } catch {}
+        }
+      });
+      res.on("error", reject);
+      out.on("error", reject);
+      out.on("finish", resolve);
+      res.pipe(out);
+    });
+    // Final progress emit so the UI can flip to "launching".
+    try {
+      event.sender.send("mk:installer-progress", {
+        received, total: received, pct: 1, done: true,
+      });
+    } catch {}
+
+    // Spawn the installer detached so it survives the terminal quitting.
+    // NSIS handles its own "stop running app" but we quit explicitly so
+    // the install dir is unlocked the moment NSIS reaches the file copy.
+    //
+    // v7: NO /S flag. With electron-builder oneClick:true the installer
+    //   is already near-silent (just a small "Installing…" progress
+    //   window) AND it auto-launches the new exe via .onInstSuccess.
+    //   /S would suppress BOTH the progress window AND the auto-launch,
+    //   leaving the user with no running app after update.
+    //   windowsHide:true is kept to avoid any console flash, but the
+    //   NSIS progress window (a GUI window, not a console) still shows.
+    try {
+      const child = spawn(dest, [], {
+        detached: true,
+        stdio: "ignore",
+        windowsHide: true,
+      });
+      child.unref();
+    } catch (e) {
+      return { ok: false, error: `spawn failed: ${String(e)}` };
+    }
+
+    // Tiny delay so the installer process is alive before we quit — avoids
+    // the Windows "starting up" race where a freshly-spawned detached child
+    // can be killed if its parent exits within ~50ms on some configs.
+    setTimeout(() => {
+      try { app.quit(); } catch {}
+    }, 800);
+    return { ok: true, file: dest };
+  } catch (e) {
+    try { fs.unlinkSync(dest); } catch {}
+    return { ok: false, error: String(e?.message || e) };
+  }
+});
+
+// ── IPC: open a detached child window for a given page ────────────────
+// Renderer (TabContextMenu → preload.openInNewWindow) calls this with a
+// pageId. We validate against the allowlist, spawn a BrowserWindow with
+// parent: mainWindow (so it closes when main closes), and load the same
+// app URL with ?detached=<pageId> so the renderer boots into DetachedShell.
+ipcMain.handle("mk:open-in-new-window", async (_event, payload) => {
+  try {
+    const pageId = payload && payload.pageId;
+    if (!VALID_DETACHED_PAGES.has(pageId)) {
+      return { ok: false, error: `invalid pageId: ${String(pageId)}` };
+    }
+    if (!mainWindow) {
+      return { ok: false, error: "main window not available" };
+    }
+
+    const childIcon = path.join(app.isPackaged ? process.resourcesPath : __dirname, "..", "icon.ico");
+
+    const win = new BrowserWindow({
+      width:  1200,
+      height: 800,
+      minWidth:  640,
+      minHeight: 480,
+      icon: childIcon,
+      backgroundColor: "#000000",
+      parent: mainWindow,
+      autoHideMenuBar: true,
+      show: false,
+      webPreferences: {
+        contextIsolation: true,
+        nodeIntegration:  false,
+        preload: path.join(__dirname, "preload.js"),
+      },
+    });
+
+    const url = `${BACKEND_URL}/?detached=${encodeURIComponent(pageId)}`;
+    win.loadURL(url);
+    win.once("ready-to-show", () => win.show());
+    win.on("closed", () => detachedWindows.delete(win));
+
+    // v8: external-URL routing (matches main window).
+    win.webContents.setWindowOpenHandler(({ url: u }) => {
+      if (/^https?:\/\//.test(u)) {
+        shell.openExternal(u).catch(err =>
+          console.error("[electron] shell.openExternal failed:", err));
+      }
+      return { action: "deny" };
+    });
+
+    // v8: keyboard shortcuts. Ctrl+= / Ctrl+- / Ctrl+0 zoom (always on);
+    //     F12 devtools (dev mode only). Mirrors main window.
+    win.webContents.on("before-input-event", (event, input) => {
+      if (input.type !== "keyDown") return;
+      if (input.key === "F12" &&
+          (process.env.ELECTRON_DEV === "1" || !app.isPackaged)) {
+        win.webContents.toggleDevTools();
+        return;
+      }
+      if (input.control || input.meta) {
+        const wc = win.webContents;
+        if (input.key === "=" || input.key === "+") {
+          wc.setZoomLevel(Math.min(wc.getZoomLevel() + 0.5, 9));
+          event.preventDefault();
+        } else if (input.key === "-") {
+          wc.setZoomLevel(Math.max(wc.getZoomLevel() - 0.5, -8));
+          event.preventDefault();
+        } else if (input.key === "0") {
+          wc.setZoomLevel(0);
+          event.preventDefault();
+        }
+      }
+    });
+
+    detachedWindows.add(win);
+    console.log(`[electron] spawned detached window: pageId=${pageId} url=${url}`);
+    return { ok: true };
+  } catch (e) {
+    console.error("[electron] open-in-new-window failed:", e);
+    return { ok: false, error: String(e) };
+  }
+});
+
+// Single-instance lock — clicking the icon while running just focuses the window
+const gotLock = app.requestSingleInstanceLock();
+if (!gotLock) {
+  app.quit();
+} else {
+  app.on("second-instance", () => {
+    if (mainWindow) {
+      if (mainWindow.isMinimized()) mainWindow.restore();
+      mainWindow.focus();
+    }
+  });
+}

electron_shell/preload.js

@@ -0,0 +1,101 @@
+// version: v5
+// Electron preload — context bridge for privileged native APIs.
+//
+// Anything renderer-side that needs to talk to the OS (file save dialog,
+// open new window, etc.) goes through here.
+//
+// v5 — F-24 (Security wave 2): exposes secureSavePassword /
+//      secureLoadPassword / secureClearPassword that route to main's
+//      safeStorage via IPC. Replaces consumer-side plaintext password
+//      in localStorage (audit F-24 vulnerability). LoginScreen.jsx v50
+//      consumes these; one-time migration in v50 reads any legacy
+//      localStorage.mk_saved_pass, encrypts it, then deletes the
+//      plaintext. See main.js v9 for IPC handler details + storage path.
+// v4 — exposes downloadAndInstall(url, onProgress) for the in-app updater.
+//      The main process streams the installer to %TEMP%, fires
+//      "mk:installer-progress" events for the UI, then spawns the .exe
+//      detached and quits the terminal. Renderer-side callers wire the
+//      onProgress callback to a progress bar and await the return value.
+
+const { contextBridge, ipcRenderer } = require("electron");
+
+contextBridge.exposeInMainWorld("mkElectron", {
+  isElectron: true,
+
+  // saveScreenshot(suggestedName, bytes)
+  //   suggestedName: string filename ("US500_M15_2026-04-27.png")
+  //   bytes:         Uint8Array | ArrayBuffer of raw PNG bytes
+  //   returns:       Promise<{ saved: boolean, path?: string }>
+  //
+  // Bypasses Chromium's download manager — instead the main process pops
+  // a native dialog.showSaveDialog and fs.writeFile-s the buffer directly.
+  // Much faster than `link.click()` on a data URL.
+  saveScreenshot: async (suggestedName, bytes) => {
+    // Bytes can be Uint8Array (good) or ArrayBuffer (we wrap). IPC handles
+    // typed arrays as Buffer on the main side.
+    const buf = bytes instanceof ArrayBuffer ? new Uint8Array(bytes) : bytes;
+    return ipcRenderer.invoke("mk:save-screenshot", { suggestedName, bytes: buf });
+  },
+
+  // openInNewWindow(pageId)
+  //   pageId: one of "quant" | "fundamental" | "flow" | "options" | "stress" | "learn"
+  //   returns: Promise<{ ok: boolean, error?: string }>
+  //
+  // Asks the main process to spawn a child BrowserWindow that loads the
+  // same React app with `?detached=<pageId>` so the renderer boots into
+  // DetachedShell mode. Each child window has parent: mainWindow so it
+  // closes automatically when the user closes the main window.
+  openInNewWindow: (pageId) => ipcRenderer.invoke("mk:open-in-new-window", { pageId }),
+
+  // fetchTrumpNews({ force? })
+  //   returns Promise<{ ok, items?: [{title, link, pubDate, source}], fetchedAt?, cached?, error? }>
+  //
+  // Main process pulls the Google News RSS feed (search=Trump), parses items,
+  // caches in-memory for 5 min. Pass { force: true } to bypass cache.
+  fetchTrumpNews: (opts) => ipcRenderer.invoke("mk:fetch-trump-news", opts || {}),
+
+  // openExternal(url)
+  //   Opens the given URL in the user's default browser via shell.openExternal.
+  //   Only http(s) URLs are allowed.
+  openExternal: (url) => ipcRenderer.invoke("mk:open-external", url),
+
+  // v5: secureSavePassword(email, password)
+  //   F-24 — Replaces localStorage.mk_saved_pass with OS-keychain
+  //   encrypted storage via main process safeStorage. Email kept plain
+  //   (not sensitive); password encrypted at rest. Returns
+  //   Promise<{ ok, error? }>. error="encryption_unavailable" on
+  //   platforms where safeStorage can't access keychain (rare Linux).
+  secureSavePassword: (email, password) =>
+    ipcRenderer.invoke("mk:secure-save-password", { email, password }),
+
+  // v5: secureLoadPassword()
+  //   Returns Promise<{ ok, email?, password?, error? }>. When no creds
+  //   saved, returns { ok: true, email: "", password: "" }. Decryption
+  //   failures (encrypted-but-keychain-rotated) return { ok: false,
+  //   error: ... }. Callers should treat any error path as "no saved
+  //   password" and prompt the user.
+  secureLoadPassword: () => ipcRenderer.invoke("mk:secure-load-password"),
+
+  // v5: secureClearPassword()
+  //   Deletes the encrypted creds file. Returns Promise<{ ok, error? }>.
+  //   Idempotent — succeeds even if no file exists.
+  secureClearPassword: () => ipcRenderer.invoke("mk:secure-clear-password"),
+
+  // v4: downloadAndInstall(url, onProgress?)
+  //   Streams installer to %TEMP%, spawns it detached, then quits the
+  //   running terminal. onProgress({ received, total, pct, done? }) is
+  //   called periodically while downloading.
+  //   Returns Promise<{ ok, file?, error? }> — terminal will quit ~800ms
+  //   after { ok: true } so the renderer typically only sees the resolve
+  //   moment briefly before the window closes.
+  downloadAndInstall: (url, onProgress) => {
+    const handler = (_e, data) => {
+      try { if (typeof onProgress === "function") onProgress(data || {}); } catch {}
+    };
+    ipcRenderer.on("mk:installer-progress", handler);
+    const cleanup = () => { try { ipcRenderer.removeListener("mk:installer-progress", handler); } catch {} };
+    return ipcRenderer.invoke("mk:download-and-install", url)
+      .then((res) => { cleanup(); return res; })
+      .catch((e)  => { cleanup(); throw e; });
+  },
+});

frontend_build/asset-manifest.json

@@ -0,0 +1,13 @@
+{
+  "files": {
+    "main.css": "/static/css/main.64fab706.css",
+    "main.js": "/static/js/main.be9bfbf6.js",
+    "index.html": "/index.html",
+    "main.64fab706.css.map": "/static/css/main.64fab706.css.map",
+    "main.be9bfbf6.js.map": "/static/js/main.be9bfbf6.js.map"
+  },
+  "entrypoints": [
+    "static/css/main.64fab706.css",
+    "static/js/main.be9bfbf6.js"
+  ]
+}

frontend_build/index.html

@@ -0,0 +1 @@
+<!doctype html><html lang="en"><head><meta charset="utf-8"/><meta name="viewport" content="width=device-width,initial-scale=1"/><title>MK.TRADES Terminal</title><style>*{margin:0;padding:0;box-sizing:border-box}body{background:#000;overflow:hidden}</style><script defer="defer" src="/static/js/main.be9bfbf6.js"></script><link href="/static/css/main.64fab706.css" rel="stylesheet"></head><body><div id="root"></div></body></html>

frontend_build/public/index.html

@@ -0,0 +1,10 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="utf-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1" />
+  <title>MK.TRADES Analytical Terminal</title>
+  <style>body{margin:0;padding:0;background:#000;overflow:hidden;}</style>
+</head>
+<body><div id="root"></div></body>
+</html>

frontend_build/static/css/main.64fab706.css

@@ -0,0 +1,2 @@
+.styles_page__b0DNw{background:#06060e;min-height:100%}.styles_cardBg__Dprkb{background:#09091a;border-radius:8px;padding:12px}.styles_borderDefault__63iOJ{border:1px solid #e8b84b14}.styles_borderHigh__0Thra{border:1px solid #dc3c3c4d}.styles_borderMed__7cBgD{border:1px solid #e8b84b38}.styles_borderLow__aw5uJ{border:1px solid #466e462e}.styles_brandSmall__8m128{color:#e8b84b;font:10px monospace;letter-spacing:5px;text-transform:uppercase}.styles_title__v9zrC{color:#ffd875;font-family:monospace;font-size:13px;font-weight:700;letter-spacing:5px;text-transform:uppercase}.styles_muted__uUsPK{color:#4a3c28}.styles_dim__UW4rx{color:#9a7830}.styles_white__PIvc3{color:#fff}.styles_topBar__lY3HL{align-items:center;background:#06060e;border-bottom:1px solid #e8b84b1f;display:flex;height:44px;justify-content:space-between;padding:0 16px}.styles_liveDot__gb6L2{animation:styles_pulse__SsdTc 1.4s ease-in-out infinite;background:#50c878;border-radius:50%;height:8px;width:8px}@keyframes styles_pulse__SsdTc{0%{opacity:0}50%{opacity:1}to{opacity:0}}.styles_liveText__QU6Tj{color:#50c878;margin-left:6px}.styles_liveText__QU6Tj,.styles_scannerOff__xct80{font:9px monospace;letter-spacing:1.5px;text-transform:uppercase}.styles_scannerOff__xct80{color:#4a3c28}.styles_scoreHigh__ayJlz{background:#dc3c3c1a;color:#e04040}.styles_scoreMed__2CyFx{background:#e8b84b14;color:#e8b84b}.styles_scoreLow__a\+3FK{background:#50785012;color:#507050}.styles_scoreBadge__1jsGg{border-radius:3px;font:11px monospace;font-weight:700;letter-spacing:1px;padding:2px 8px}.styles_tagQB__Olltu{background:#e8b84b1a;border:.5px solid #e8b84b40;color:#e8b84b}.styles_tagCO__uOmNR{background:#64a0dc1a;border:.5px solid #64a0dc40;color:#80b8e0}.styles_tag__EDr2T{border-radius:3px;font:8px monospace;letter-spacing:.5px;padding:2px 5px;text-transform:uppercase}.styles_grid__v5R95{grid-gap:8px;display:grid;gap:8px;grid-template-columns:repeat(3,1fr);padding:12px}.styles_alertStrip__Skiwe{padding:10px 14px}.styles_alertHeader__qyKjd{color:#9a7830;font:9px monospace;letter-spacing:4px;margin-bottom:8px;text-transform:uppercase}.styles_alertCrit__ACC1X{background:#dc3c3c0d;border-left:2px solid #e04040;min-height:36px;padding:8px 10px}.styles_alertWarn__7EynK{background:#e8b84b0a;border-left:2px solid #e8b84b;min-height:36px;padding:8px 10px}.styles_alertAsset__OUKUM{color:#fff;font:10px monospace;font-weight:700;letter-spacing:2px;min-width:56px}.styles_alertBody__pt4jk{color:#60503a;font:9px monospace}.styles_statsBar__SEvBO{align-items:center;border-top:1px solid #e8b84b12;display:flex;gap:18px;height:36px;padding:8px 16px}.styles_statLabel__ZHEhS{color:#3a3020;font:8px monospace;letter-spacing:2px;text-transform:uppercase}.styles_statValue__i5m8r{color:#e8b84b;font:10px monospace;font-weight:700;margin-left:6px}.styles_statValueImpact__AEpRP{color:#e04040}.styles_sep__MXc4n{background:#e8b84b12;height:14px;width:1px}.styles_staleBanner__HJc\+d{align-items:center;background:#e8b84b0f;border-bottom:1px solid #e8b84b2e;color:#e8b84b;display:flex;font:10px monospace;justify-content:space-between;letter-spacing:1px;padding:8px 16px}.styles_calcOverlay__edps3{align-items:center;background:#06060ec7;color:#e8b84b;display:flex;font:11px monospace;inset:0;justify-content:center;letter-spacing:3px;position:absolute;text-transform:uppercase;z-index:50}.styles_originNow__6HDzO{color:#fff}.styles_originCurrentWeek__J3qxS{color:#80b8e0}.styles_originPrevWeek__snrh3{color:#5c8cb0}.styles_originWeekPrev2__IHP51{color:#3a6080}.styles_originCurrentMonth__jPJOb{color:#80e0a0}.styles_originPrevMonth__bM7yU{color:#5ca070}.styles_originPrevMonth2__jcnEL{color:#4a7e58}.styles_originExtremeHigh__jODcD{color:#ff80b0}.styles_originExtremeLow__hRBvN{color:#ffa060}.styles_originBands__g7kwV{color:#e8b84b}.styles_swatch__aWIbc{border-radius:2px;display:inline-block;height:8px;margin-right:6px;vertical-align:middle;width:8px}.styles_elementList__lsO6Z{display:flex;flex:1 1;flex-direction:column;font-family:monospace;font-size:9px;gap:2px;max-height:154px;min-width:0;overflow-y:auto}.styles_elementRow__9g31h{align-items:center;display:flex;gap:6px;overflow:hidden;white-space:nowrap}.styles_elementLabel__qQVri{color:#fff;flex:1 1;overflow:hidden;text-overflow:ellipsis}.styles_elementDist__ZUtS-{color:#9a7830;flex-shrink:0;font-size:8px}.styles_elementApex__bPLem{color:#ffd875;font-size:8px;font-weight:700;letter-spacing:1px}.styles_cardBody__ZPNha{align-items:flex-start;display:flex;gap:8px;margin:8px 0}
+/*# sourceMappingURL=main.64fab706.css.map*/

frontend_build/static/js/main.be9bfbf6.js.LICENSE.txt

@@ -0,0 +1,87 @@
+/*!
+ * @license
+ * TradingView Lightweight Charts™ v5.1.0
+ * Copyright (c) 2025 TradingView, Inc.
+ * Licensed under Apache License 2.0 https://www.apache.org/licenses/LICENSE-2.0
+ */
+
+/*! decimal.js-light v2.5.1 https://github.com/MikeMcl/decimal.js-light/LICENCE */
+
+/**
+ * @license React
+ * react-dom.production.min.js
+ *
+ * Copyright (c) Facebook, Inc. and its affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * @license React
+ * react-jsx-runtime.production.min.js
+ *
+ * Copyright (c) Facebook, Inc. and its affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * @license React
+ * react.production.min.js
+ *
+ * Copyright (c) Facebook, Inc. and its affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * @license React
+ * scheduler.production.min.js
+ *
+ * Copyright (c) Facebook, Inc. and its affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * @license React
+ * use-sync-external-store-shim.production.js
+ *
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * @license React
+ * use-sync-external-store-shim/with-selector.production.js
+ *
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * @license React
+ * use-sync-external-store-with-selector.production.js
+ *
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/** @license React v17.0.2
+ * react-is.production.min.js
+ *
+ * Copyright (c) Facebook, Inc. and its affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */

frontend_build/static/mk_debug_telemetry.js

@@ -0,0 +1,244 @@
+// version: v1
+// mk_debug_telemetry.js — disposable diagnostic harness.
+//
+// Loaded as a raw <script> tag from public/index.html (gated by
+// REACT_APP_MK_DEBUG=1 at build time). Runs BEFORE the React bundle so
+// the monkey-patches on global timers see all subsequent allocations.
+//
+// Public API:
+//   window.__mkDebug
+//     .longTasks  []  { ts, duration, name }
+//     .memorySnaps[]  { ts, usedJS, totalJS, jsLimit }
+//     .counters   []  { ts, intervals, timeouts, rafs, observers,
+//                       assetsMapSize, barsArrLen, signalsLen }
+//     .syncEvents []  { ts, type, detail? }
+//     .notes      []  { ts, text }
+//     .config     {}  { sampleIntervalMs, maxLongTasks, maxCounters }
+//     .note(text)     push a free-form timestamped note
+//     .dump()         copy capture to clipboard as JSON
+//     .reset()        clear all arrays
+//
+// After the diagnostic session, run __mkDebug.dump() in DevTools and
+// paste into a file `frontend_capture.json`.
+//
+// Disposable — see spec §8 (Removal procedure):
+//   docs/specs/2026-05-05-debug-telemetry-harness-design.md
+
+(function () {
+  "use strict";
+  if (typeof window === "undefined") return;
+
+  function nowIso() {
+    return new Date().toISOString();
+  }
+
+  // ----- Config + buffers -----
+  const CFG = { sampleIntervalMs: 30000, maxLongTasks: 10000, maxCounters: 500 };
+
+  const dbg = {
+    startedAt: nowIso(),
+    longTasks: [],
+    memorySnaps: [],
+    counters: [],
+    syncEvents: [],
+    notes: [],
+    config: CFG,
+
+    note(text) {
+      try {
+        dbg.notes.push({ ts: nowIso(), text: String(text == null ? "" : text) });
+      } catch (e) {
+        console.error("[mkDebug] note() failed:", e);
+      }
+    },
+
+    dump() {
+      const payload = JSON.stringify({
+        startedAt: dbg.startedAt,
+        capturedAt: nowIso(),
+        longTasks: dbg.longTasks,
+        memorySnaps: dbg.memorySnaps,
+        counters: dbg.counters,
+        syncEvents: dbg.syncEvents,
+        notes: dbg.notes,
+        config: dbg.config,
+        userAgent: (typeof navigator !== "undefined") ? navigator.userAgent : "",
+      }, null, 2);
+      dbg.lastDump = payload;
+
+      try {
+        navigator.clipboard.writeText(payload).then(
+          () => console.info(
+            "[mkDebug] capture (%d KB) copied to clipboard",
+            Math.round(payload.length / 1024)
+          ),
+          (err) => console.error(
+            "[mkDebug] clipboard failed:", err,
+            "— payload also in __mkDebug.lastDump"
+          )
+        );
+      } catch (e) {
+        console.error(
+          "[mkDebug] clipboard not available; payload in __mkDebug.lastDump"
+        );
+      }
+      return payload.length;
+    },
+
+    reset() {
+      dbg.longTasks = [];
+      dbg.memorySnaps = [];
+      dbg.counters = [];
+      dbg.syncEvents = [];
+      dbg.notes = [];
+    },
+  };
+
+  window.__mkDebug = dbg;
+
+  // ----- Long-task observer -----
+  try {
+    const po = new PerformanceObserver((list) => {
+      const entries = list.getEntries();
+      for (let i = 0; i < entries.length; i++) {
+        const e = entries[i];
+        if (dbg.longTasks.length >= CFG.maxLongTasks) break;
+        dbg.longTasks.push({
+          ts: nowIso(),
+          duration: Math.round(e.duration),
+          name: e.name || "self",
+        });
+      }
+    });
+    po.observe({ entryTypes: ["longtask"] });
+  } catch (e) {
+    console.warn("[mkDebug] longtask observer not supported:", e);
+  }
+
+  // ----- Memory snapshots -----
+  function snapMemory() {
+    if (!performance || !performance.memory) return;
+    if (dbg.memorySnaps.length >= 200) return; // cap
+    dbg.memorySnaps.push({
+      ts: nowIso(),
+      usedJS: performance.memory.usedJSHeapSize,
+      totalJS: performance.memory.totalJSHeapSize,
+      jsLimit: performance.memory.jsHeapSizeLimit,
+    });
+  }
+  setInterval(snapMemory, CFG.sampleIntervalMs);
+  snapMemory(); // initial baseline
+
+  // ----- Active counter monkey-patches -----
+  // setInterval / clearInterval
+  const _origSetInterval = window.setInterval.bind(window);
+  const _origClearInterval = window.clearInterval.bind(window);
+  const _activeIntervals = new Set();
+  window.setInterval = function (handler, timeout) {
+    const args = Array.prototype.slice.call(arguments);
+    const id = _origSetInterval.apply(window, args);
+    _activeIntervals.add(id);
+    return id;
+  };
+  window.clearInterval = function (id) {
+    _activeIntervals.delete(id);
+    return _origClearInterval(id);
+  };
+
+  // setTimeout / clearTimeout
+  const _origSetTimeout = window.setTimeout.bind(window);
+  const _origClearTimeout = window.clearTimeout.bind(window);
+  const _activeTimeouts = new Set();
+  window.setTimeout = function (handler, timeout) {
+    const args = Array.prototype.slice.call(arguments);
+    const id = _origSetTimeout.apply(window, args);
+    _activeTimeouts.add(id);
+    // self-cleanup: when the timer fires it's no longer active
+    return id;
+  };
+  window.clearTimeout = function (id) {
+    _activeTimeouts.delete(id);
+    return _origClearTimeout(id);
+  };
+
+  // requestAnimationFrame: track rate of scheduling over the last 30s.
+  // RAFs self-fire and self-clear, so "active count" doesn't fit; instead
+  // record a sliding window of timestamps.
+  const _origRAF = window.requestAnimationFrame.bind(window);
+  const _rafTimes = [];
+  window.requestAnimationFrame = function (cb) {
+    _rafTimes.push(Date.now());
+    return _origRAF(cb);
+  };
+  function rafsInLastWindow() {
+    const cutoff = Date.now() - CFG.sampleIntervalMs;
+    while (_rafTimes.length && _rafTimes[0] < cutoff) _rafTimes.shift();
+    return _rafTimes.length;
+  }
+
+  // ResizeObserver count
+  let _activeObservers = 0;
+  if (typeof window.ResizeObserver === "function") {
+    const _OrigRO = window.ResizeObserver;
+    window.ResizeObserver = function PatchedResizeObserver(cb) {
+      const ro = new _OrigRO(cb);
+      _activeObservers++;
+      const origDisconnect = ro.disconnect.bind(ro);
+      ro.disconnect = function () {
+        _activeObservers = Math.max(0, _activeObservers - 1);
+        return origDisconnect();
+      };
+      return ro;
+    };
+    // Preserve prototype for `instanceof` checks
+    window.ResizeObserver.prototype = _OrigRO.prototype;
+  }
+
+  // ----- Counter snapshot loop -----
+  function snapCounters() {
+    if (dbg.counters.length >= CFG.maxCounters) return;
+    let assetsMapSize = null;
+    let barsArrLen = null;
+    let signalsLen = null;
+    try {
+      const h = window.__mkDebugHandle;
+      if (h) {
+        if (h.assets && typeof h.assets.size === "number") assetsMapSize = h.assets.size;
+        else if (h.assets && Array.isArray(h.assets)) assetsMapSize = h.assets.length;
+        if (typeof h.barsLen === "number") barsArrLen = h.barsLen;
+        if (typeof h.signalsLen === "number") signalsLen = h.signalsLen;
+      }
+    } catch (e) { /* swallow */ }
+    dbg.counters.push({
+      ts: nowIso(),
+      intervals: _activeIntervals.size,
+      timeouts: _activeTimeouts.size,
+      rafs: rafsInLastWindow(),
+      observers: _activeObservers,
+      assetsMapSize: assetsMapSize,
+      barsArrLen: barsArrLen,
+      signalsLen: signalsLen,
+    });
+  }
+  setInterval(snapCounters, CFG.sampleIntervalMs);
+  snapCounters(); // initial baseline
+
+  // ----- Sync-event tap -----
+  try {
+    window.addEventListener("mk:data-sync-complete", function (e) {
+      if (dbg.syncEvents.length >= 5000) return;
+      dbg.syncEvents.push({
+        ts: nowIso(),
+        type: "mk:data-sync-complete",
+        detail: (e && e.detail) ? e.detail : null,
+      });
+    });
+  } catch (e) {
+    console.warn("[mkDebug] sync-event tap install failed:", e);
+  }
+
+  console.info(
+    "[mkDebug] harness installed at %s — call __mkDebug.dump() to capture",
+    dbg.startedAt
+  );
+})();