# version: v27
# v27 — Registered the /api/chart-templates router (chart_templates_routes.py).
#       Companion to /api/chart-presets — named full-chart-config snapshots
#       the operator saves/loads via the TPL ▾ button. (2026-05-11)
# v26 — /api/bands attaches `_last_modified` (unix epoch float) sourced from
#       the resolved JSON file's mtime (PRO local OR sync cache). Frontend
#       uses it to detect when bands are older than the most recent market
#       close and flag the BANDS toolbar button red. Best-effort: if the
#       cache file path can't be resolved (legacy fallback or unknown sync
#       layout), the field is omitted and the frontend treats the bands as
#       fresh (no red flag, no false alarms).
"""
================================================================================
Quantum Terminal — Data Server (Consumer Build)
================================================================================
FastAPI backend for the consumer terminal. Display-only — no computation.

v25 — Added env-gated install of debug_telemetry harness during lifespan
      startup. No effect when MK_DEBUG_TELEMETRY != "1". Disposable —
      see docs/specs/2026-05-05-debug-telemetry-harness-design.md.

v19 — Non-blocking initial sync. Previous lifespan ran consumer_gate() end-
      to-end before the FastAPI HTTP server became responsive, so every
      retry/slow endpoint in sync_all() added to the user-visible loading
      screen (up to 15 min in the field). Now:
        1. consumer_gate(skip_sync=True) runs synchronously — auth only,
           fast. Lifespan returns, HTTP routes go live immediately.
        2. A daemon thread runs sync_all() in the background; when it
           finishes, `app.state.consumer_gate["sync_summary"]` is
           populated. Frontend polls /api/sync/status for the SYNCING
           pill (already wired) and shows the terminal UI the whole time.
      Net: users always see the terminal within seconds, regardless of
      server-side data-endpoint health. Per CLAUDE.md Rule C2 relaxed.

v18 — Force OPTIONS_AVAILABLE = False unconditionally. The PRO
      `options_routes.create_options_router` serves `/api/options/{ticker}/full`
      directly from `compute_all_multi(snap)` and never applies the ETF→CFD
      conversion — so if its `options_data_manager` + `options_engine`
      dependencies happen to be bundled into the consumer's PyInstaller
      payload (older builds shipped them), the router takes over the route
      and every GEX level, wall, max pain etc. renders in raw ETF price
      space. Bug symptom reported in the field: XAUUSD candles stream fine
      (MT5 connected) but options levels are in ETF numbers (GLD ~200 vs
      XAUUSD ~4700). Consumer is display-only (Rule C1) — the PRO options
      engine should never run here, period. We keep the import attempt (so
      upstream module resolution doesn't change) but set OPTIONS_AVAILABLE
      to False after, guaranteeing the sync-cache fallback route (which
      applies the CFD conversion) always handles /api/options/*.

v2 — Data Center transition: /api/bands/{ticker} now reads standalone
     {TICKER}_bands.json from the sync cache (Data Center schema), with a
     legacy fallback to the old nested bands key inside {TICKER}_cones.json.
     /api/cones was already shape-flexible (wrapped or flat) — no change.

v3 — Cache directory watcher: new background task started in consumer lifespan
     that polls %APPDATA%\\QuantumTerminal\\cache\\ and broadcasts data_sync_complete
     when any {TICKER}_{category}.json changes. Enables live updates without
     browser refresh for manual drops and future auto-uploaders. Feature-flagged
     via consumer_config.ini [sync] cache_watcher_enabled (default: on).

v4 — New endpoint /api/historical_cones/{ticker} for the REPLAY sub-tab of
     quant analysis. Serves per-anchor historical cone payloads from the
     sync cache (weekly + monthly anchors, each with gbm/mjd/bates models).

v9 — Focused-symbol fast tick loop. Per-symbol MT5 polling is the bottleneck
     (≈5ms × universe_size per iteration) so dropping the global tick interval
     hits a wall ≈ 5-7 fps at universe=20. New focus_tick_loop polls ONLY the
     symbol the user is watching every FOCUS_TICK_INTERVAL (0.05s = 20 fps),
     while the existing tick_loop keeps the rest of the universe fresh at the
     normal 1s rate. Frontend sets focus via POST /api/focus/{ticker}.

v8 — Tick loop reads providers.accounts.mt5_default.use_tick_data on every
     iteration. When True, polls MT5 every 0.2s instead of 1.0s, giving the
     chart MT5-style live tick movement. Toggle surfaces in Settings →
     PROVIDERS. Default OFF; switching it via PATCH /api/mt5/config takes
     effect on the next loop tick — no restart required.

v7 — /api/quarterly_cones/{ticker} now also resolves the Data Center's
     GLOBAL_<lower>_quarterly_cones.json naming via the sync client + a
     direct second-path fallback. Schema upgraded to quarterly_cones_v2
     (4 anchors × 2 models). Frontend maps the 8 keys into the QUARTERLY
     row of the CONES dropdown with per-anchor sub-toggles.

v6 — New endpoint /api/quarterly_cones/{ticker} — reads
     {TICKER}_quarterly_cones.json directly from the consumer cache. Temporary
     wiring: the frontend maps gbm_quarter_curr / mjd_quarter_curr /
     bates_quarter_curr into the existing "MANUAL" cone slots so the user can
     see the quarterly forecast by toggling MANUAL in the CONES dropdown.

v5 — New endpoint /api/scalp_bands/{ticker} for the MICRO toggle in the
     bands dropdown. Serves the scalp bands JSON (per-horizon point values
     anchored at a recent timestamp) from the sync cache.

Responsibilities:
    1. MT5 Price Feed — poll ticks and bars, detect new closed bars
    2. WebSocket Broadcasting — push prices and state-change events to clients
    3. REST API — serve pre-computed data (cones, bands, anchors, probability field)
    4. File Watching — monitor JSON state files for changes, notify clients
    5. Consumer Auth — JWT login/register via license_client proxy
    6. Data Sync — download pre-computed data from server via data_sync_client

REMOVED from PRO (Rule C1 — no calculation triggers):
    - Signal engine evaluation (signal_engine_v2, signal_live_manager)
    - OU analysis (ou_mean_reversion)
    - Flow confirmation (flow_confirmation)
    - Auto executor (auto_executor)
    - Orchestrator endpoints (/api/orchestrator/*)
    - Calculate endpoints (/api/calculate/*)
    - Signal evaluate endpoints (/api/signals/evaluate/*)
    - Anchor calibration endpoints (/api/anchor/calibrate)
    - Manual cone computation (/api/cones/manual/*)
    - Dev endpoints (/api/dev/*)
    - Trade reconciliation loop
    - Options archive scheduler
    - Broker DPP cache population

Usage:
    python data_server.py --port 8502

This file does NOT introduce any calculation triggers.
================================================================================
"""

import json
import asyncio
import logging
import time
import argparse
import os
from pathlib import Path
from datetime import datetime, timezone
from typing import Dict, Set, Optional, Any
from contextlib import asynccontextmanager

# ── Project root (portable — Rule 3) ──
PROJECT_ROOT = Path(__file__).resolve().parent

import sys
if str(PROJECT_ROOT) not in sys.path:
    sys.path.insert(0, str(PROJECT_ROOT))

from server_config import ServerConfig, PROJECT_ROOT
from config_manager import ConfigManager
from config_routes import create_config_router
from execution_routes import create_execution_router
from regime_routes import create_regime_router
from system_routes import create_system_router
from account_routes import create_account_router
# v18: keep the import attempt (some downstream module trees expect the
#   symbol to resolve) but force OPTIONS_AVAILABLE = False so the PRO
#   router can never register over the sync-cache fallback. The fallback
#   at `/api/options/{ticker}/full` is the ONLY path that applies the
#   ETF→CFD conversion for strikes/walls/levels/GEX.
try:
    from options_routes import create_options_router  # noqa: F401
except ImportError:
    create_options_router = None  # noqa: F811
OPTIONS_AVAILABLE = False
from freshness_routes import create_freshness_router
from trade_mapper_routes import create_trade_mapper_router
from pulse_room.routes import create_pulse_router
from chart_presets_routes import create_chart_presets_router
from chart_templates_routes import create_chart_templates_router

# ── Macro rotation data (yfinance sector + country ETFs) ──
try:
    from macro_data import get_sector_data, get_country_data
    MACRO_DATA_AVAILABLE = True
except ImportError:
    MACRO_DATA_AVAILABLE = False

# ── Consumer terminal wiring (Project B) ──
try:
    from consumer_startup import consumer_gate, wire_consumer_routes
    CONSUMER_MODE = True
except ImportError:
    CONSUMER_MODE = False


# ── Third-party imports ──
from fastapi import FastAPI, WebSocket, WebSocketDisconnect, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
import uvicorn

# ── MT5 availability flag (for provider type detection, no direct calls) ──
try:
    import MetaTrader5 as _mt5_check
    MT5_AVAILABLE = True
    del _mt5_check
except ImportError:
    MT5_AVAILABLE = False

# ── File watcher ──
try:
    from watchfiles import awatch, Change
    WATCHFILES_AVAILABLE = True
except ImportError:
    WATCHFILES_AVAILABLE = False

# ── Signal live manager — REMOVED in consumer (Rule C1) ──
SIGNAL_MANAGER_AVAILABLE = False
lifecycle_mgr = None
signal_evaluator = None

# ── Auto executor — REMOVED in consumer (Rule C1) ──
AUTO_EXECUTOR_AVAILABLE = False


log = logging.getLogger("data_server")

# ── Session file logging ──
try:
    from session_logger import setup_session_logging, create_logs_router
    _session_log_path = setup_session_logging()
    log.info(f"Session log: {_session_log_path}")
except ImportError:
    log.info("session_logger not available — console only")
    create_logs_router = None
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(name)s] %(levelname)s: %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)




# ============================================================
# 3. WEBSOCKET CONNECTION MANAGER
# ============================================================

class ConnectionManager:
    """Manages WebSocket connections across channels."""

    def __init__(self):
        self._price_clients: Set[WebSocket] = set()
        self._event_clients: Set[WebSocket] = set()

    async def connect_prices(self, ws: WebSocket):
        await ws.accept()
        self._price_clients.add(ws)
        log.info(f"Price client connected ({len(self._price_clients)} total)")

    async def connect_events(self, ws: WebSocket):
        await ws.accept()
        self._event_clients.add(ws)
        log.info(f"Event client connected ({len(self._event_clients)} total)")

    def disconnect_prices(self, ws: WebSocket):
        self._price_clients.discard(ws)
        log.info(f"Price client disconnected ({len(self._price_clients)} remaining)")

    def disconnect_events(self, ws: WebSocket):
        self._event_clients.discard(ws)
        log.info(f"Event client disconnected ({len(self._event_clients)} remaining)")

    async def broadcast_prices(self, message: dict):
        """Send to all price channel subscribers."""
        if not self._price_clients:
            return
        data = json.dumps(message)
        dead = []
        for ws in self._price_clients:
            try:
                await ws.send_text(data)
            except Exception:
                dead.append(ws)
        for ws in dead:
            self._price_clients.discard(ws)

    async def broadcast_event(self, message: dict):
        """Send to all event channel subscribers."""
        if not self._event_clients:
            return
        data = json.dumps(message)
        dead = []
        for ws in self._event_clients:
            try:
                await ws.send_text(data)
            except Exception:
                dead.append(ws)
        for ws in dead:
            self._event_clients.discard(ws)

    @property
    def price_client_count(self) -> int:
        return len(self._price_clients)

    @property
    def event_client_count(self) -> int:
        return len(self._event_clients)


# ============================================================
# 4. FILE STATE READER
# ============================================================

def read_json_safe(path: Path) -> Optional[dict]:
    """Read a JSON file, returning None on any error."""
    try:
        if not path.exists():
            return None
        with open(path, "r") as f:
            return json.load(f)
    except (json.JSONDecodeError, IOError, OSError) as e:
        log.warning(f"Failed to read {path.name}: {e}")
        return None


# ============================================================
# 5. BACKGROUND TASKS
# ============================================================

# ── Tick price cache (populated by tick_loop, read by signal lifecycle) ──
_latest_ticks: Dict[str, float] = {}


FAST_TICK_INTERVAL  = 1.0   # v9: full-universe loop stays at config rate when
                             # USE TICK DATA is on (fast loop now handles the
                             # focused chart at FOCUS_TICK_INTERVAL).
FOCUS_TICK_INTERVAL = 0.1    # v9/v10: focused-symbol fast loop — 10 fps.
                             # Was 0.05 (20 fps) but that saturated React's
                             # main thread with re-renders, causing chart drag
                             # to freeze. 10 fps still feels live and leaves
                             # event-loop budget for mouse input. True MT5-style
                             # fluidity needs Option C (bypass React state on
                             # the price hot path) — defer until requested.

# v9: focused symbol the user is actively watching. None = no focus → fast
# loop idles. Set via POST /api/focus/{ticker}.
_focused_symbol: Optional[str] = None


def set_focused_symbol(ticker: Optional[str]):
    global _focused_symbol
    _focused_symbol = ticker.upper() if ticker else None
    log.info(f"Focused symbol = {_focused_symbol or '(none)'}")


def get_focused_symbol() -> Optional[str]:
    return _focused_symbol


def _resolve_tick_interval(cfg_manager: ConfigManager, base: float) -> float:
    """v9: Main full-universe tick loop always uses base config.tick_interval.
    The fast path is now focus_tick_loop, which polls only the focused symbol
    at FOCUS_TICK_INTERVAL. (Kept for backward-compat / future hooks.)"""
    return base


async def tick_loop(app_state, manager: ConnectionManager, config: ServerConfig, cfg_manager: ConfigManager):
    """
    Polls MT5 for latest ticks. Default interval = config.tick_interval (1.0s).
    v8: When user enables 'USE TICK DATA' in Settings → PROVIDERS, switches
    to FAST_TICK_INTERVAL (0.2s) on the next iteration — no restart needed.
    Broadcasts to all /ws/prices subscribers.
    """
    log.info(f"Tick loop started (base interval={config.tick_interval}s, "
             f"fast interval={FAST_TICK_INTERVAL}s when use_tick_data=True)")
    last_mode = None
    while True:
        try:
            provider = app_state.provider
            if provider and provider.connected and manager.price_client_count > 0:
                universe = cfg_manager.get_active_universe()
                ticks = await asyncio.to_thread(provider.get_latest_ticks, universe)
                if ticks:
                    # Update tick cache for signal lifecycle price checks
                    for t, tick in ticks.items():
                        _latest_ticks[t] = tick.last
                    # Send all ticks as one batch — avoids React render-frame drops
                    await manager.broadcast_prices({
                        "type": "ticks_batch",
                        "ticks": {t: tick.to_dict() for t, tick in ticks.items()},
                    })
            elif provider and provider.connected:
                # Still update cache even if no WS clients (signals need prices)
                universe = cfg_manager.get_active_universe()
                ticks = await asyncio.to_thread(provider.get_latest_ticks, universe)
                if ticks:
                    for t, tick in ticks.items():
                        _latest_ticks[t] = tick.last
        except Exception as e:
            log.error(f"Tick loop error: {e}")
        # v8: dynamic sleep — re-read each iteration so the user toggle is live.
        interval = _resolve_tick_interval(cfg_manager, config.tick_interval)
        if interval != last_mode:
            log.info(f"Tick interval = {interval}s "
                     f"({'FAST / use_tick_data ON' if interval == FAST_TICK_INTERVAL else 'NORMAL'})")
            last_mode = interval
        await asyncio.sleep(interval)


async def focus_tick_loop(app_state, manager: ConnectionManager, cfg_manager: ConfigManager):
    """v9: Fast loop dedicated to the symbol the user is actively watching.
    Polls ONLY the focused symbol every FOCUS_TICK_INTERVAL when USE TICK DATA
    is on. This avoids the universe-size ceiling that caps the main tick loop.

    Sleeps 1s when:
      - USE TICK DATA is off
      - No focused symbol set
      - No WS price clients connected
      - MT5 disconnected
    """
    log.info(f"Focus tick loop started (interval={FOCUS_TICK_INTERVAL}s when active)")
    last_active = None
    while True:
        try:
            mt5_cfg = (cfg_manager.get_config().get("providers", {})
                       .get("accounts", {}).get("mt5_default", {}) or {})
            use_tick = bool(mt5_cfg.get("use_tick_data", False))
            sym = get_focused_symbol()
            provider = app_state.provider

            active = (
                use_tick
                and sym is not None
                and provider is not None
                and provider.connected
                and manager.price_client_count > 0
            )
            if active != last_active:
                log.info(
                    f"Focus tick loop: {'ACTIVE' if active else 'IDLE'} "
                    f"(use_tick={use_tick}, sym={sym}, "
                    f"connected={provider.connected if provider else False}, "
                    f"clients={manager.price_client_count})"
                )
                last_active = active

            if active:
                ticks = await asyncio.to_thread(provider.get_latest_ticks, [sym])
                if ticks:
                    for t, tick in ticks.items():
                        _latest_ticks[t] = tick.last
                    await manager.broadcast_prices({
                        "type": "ticks_batch",
                        "ticks": {t: tick.to_dict() for t, tick in ticks.items()},
                    })
                await asyncio.sleep(FOCUS_TICK_INTERVAL)
            else:
                await asyncio.sleep(1.0)
        except Exception as e:
            log.error(f"Focus tick loop error: {e}")
            await asyncio.sleep(1.0)


async def bar_check_loop(
    app_state, manager: ConnectionManager, config: ServerConfig, cfg_manager: ConfigManager
):
    """
    Checks for newly closed M15 bars at config.bar_check_interval.
    Broadcasts new bars to /ws/prices subscribers.
    """
    log.info(f"Bar check loop started (interval={config.bar_check_interval}s)")
    while True:
        try:
            provider = app_state.provider
            if provider and provider.connected:
                universe = cfg_manager.get_active_universe()
                new_bars = await asyncio.to_thread(
                    provider.check_new_bars, universe, config.default_timeframe
                )
                for bar_msg in new_bars:
                    await manager.broadcast_prices(bar_msg)
                    log.debug(
                        f"New {bar_msg['timeframe']} bar: "
                        f"{bar_msg['ticker']} @ {bar_msg['bar']['time']}"
                    )
        except Exception as e:
            log.error(f"Bar check error: {e}")
        await asyncio.sleep(config.bar_check_interval)


async def mt5_reconnect_loop(app_state, config: ServerConfig):
    """
    Monitors MT5 connection. Reconnects with backoff if dropped.
    """
    while True:
        await asyncio.sleep(10)
        provider = app_state.provider
        if provider and not provider.connected:
            log.info("MT5 disconnected — attempting reconnect...")
            for attempt in range(config.mt5_max_reconnect_attempts):
                success = await asyncio.to_thread(provider.reconnect)
                if success:
                    log.info("MT5 reconnected successfully")
                    break
                delay = min(config.mt5_reconnect_delay * (2 ** attempt), 60)
                log.warning(
                    f"MT5 reconnect attempt {attempt + 1} failed, "
                    f"retrying in {delay:.0f}s"
                )
                await asyncio.sleep(delay)
        elif provider:
            # Heartbeat check — ask the provider to verify its connection
            try:
                alive = await asyncio.to_thread(provider.heartbeat)
                if not alive:
                    log.warning("Provider heartbeat failed — will reconnect next cycle")
            except Exception:
                log.warning("Provider heartbeat exception — will reconnect next cycle")


async def file_watch_loop(manager: ConnectionManager, config: ServerConfig):
    """
    Watches state JSON files for changes and pushes events to /ws/events.
    Falls back to polling if watchfiles is unavailable.
    """
    watched_paths = config.watched_files
    path_to_source = {
        config.resolve_path(config.terminal_payload): "terminal_payload",
        config.resolve_path(config.daily_signals): "daily_signals",
        config.resolve_path(config.signal_lifecycle): "signal_lifecycle",
        config.resolve_path(config.weekly_state): "weekly_state",
    }

    if WATCHFILES_AVAILABLE:
        log.info(f"File watcher started (watchfiles, {len(watched_paths)} paths)")
        # Watch the project root directory, filter for our files
        watch_dir = PROJECT_ROOT

        async for changes in awatch(watch_dir, debounce=1000):
            for change_type, changed_path in changes:
                changed = Path(changed_path)
                source = path_to_source.get(changed)
                if source is None:
                    continue

                log.info(f"File change detected: {changed.name} ({change_type.name})")
                now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")

                if source == "terminal_payload":
                    # Read which tickers were updated
                    data = read_json_safe(changed)
                    tickers = list(data.get("assets", {}).keys()) if data else []
                    await manager.broadcast_event({
                        "type": "payload_update",
                        "source": source,
                        "timestamp": now,
                        "tickers_updated": tickers,
                    })
                elif source in ("daily_signals", "signal_lifecycle"):
                    data = read_json_safe(changed)
                    summary = {}
                    if data and isinstance(data, dict):
                        # Count signals per state if available
                        all_signals = []
                        for ticker_signals in data.values():
                            if isinstance(ticker_signals, list):
                                all_signals.extend(ticker_signals)
                        summary = {
                            "total_signals": len(all_signals),
                            "tickers_affected": list(data.keys()),
                        }
                    await manager.broadcast_event({
                        "type": "state_update",
                        "source": source,
                        "timestamp": now,
                        "summary": summary,
                    })
                else:
                    await manager.broadcast_event({
                        "type": "state_update",
                        "source": source,
                        "timestamp": now,
                        "summary": {},
                    })
    else:
        # Fallback: polling-based file watcher
        log.info("File watcher started (polling mode — install watchfiles for efficiency)")
        mtimes: Dict[str, float] = {}

        while True:
            for path in watched_paths:
                if not path.exists():
                    continue
                mtime = path.stat().st_mtime
                key = str(path)
                if key in mtimes and mtime > mtimes[key]:
                    source = path_to_source.get(path, path.stem)
                    log.info(f"File change detected (poll): {path.name}")
                    now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
                    await manager.broadcast_event({
                        "type": "state_update",
                        "source": source,
                        "timestamp": now,
                        "summary": {},
                    })
                mtimes[key] = mtime

            await asyncio.sleep(config.file_watch_debounce)


# ============================================================
# 6. FASTAPI APPLICATION
# ============================================================

server_config = ServerConfig()
cfg_manager = ConfigManager()
manager = ConnectionManager()
start_time = time.time()

def _get_current_prices() -> dict:
    """Return latest known prices for all tickers from tick cache."""
    return dict(_latest_ticks)



@asynccontextmanager
async def lifespan(app: FastAPI):
    """Startup and shutdown logic."""
    # ── Startup ──
    log.info("=" * 60)
    log.info("  Quantum Terminal Data Server — Starting")


    # ── Consumer startup gate (Rule C2, relaxed — v19) ──
    # Run auth synchronously (need the result for login routing); spawn the
    # slow sync_all() in a daemon thread so this lifespan returns in <1s and
    # the HTTP server starts serving API routes immediately.
    if CONSUMER_MODE:
        gate_result = consumer_gate(skip_sync=True)
        app.state.consumer_gate = gate_result
        log.info(
            f"Consumer gate (auth phase): ok={gate_result['ok']}, "
            f"login={gate_result.get('needs_login')}, "
            f"offline={gate_result.get('offline_mode')}"
        )

        # Spawn background sync only when auth succeeded (no login needed).
        # If login is required, sync waits — user logs in, periodic_sync.py
        # or a post-login hook picks it up.
        if gate_result.get("ok") and not gate_result.get("needs_login"):
            import threading
            def _run_initial_sync():
                try:
                    from data_sync_client import get_sync_client
                    sc = get_sync_client()
                    log.info("[initial-sync] thread started")
                    summary = sc.sync_all()
                    # Persist summary onto app.state so /api/sync/status can surface it.
                    try:
                        app.state.consumer_gate["sync_summary"] = summary
                        if summary.get("offline_mode"):
                            app.state.consumer_gate["offline_mode"] = True
                        # Rule C6 — format version mismatch surfaces after sync too.
                        if not summary.get("format_version_ok", True):
                            app.state.consumer_gate["needs_update"] = True
                    except Exception:
                        pass
                    log.info(
                        f"[initial-sync] complete: "
                        f"synced={summary.get('synced', 0)} "
                        f"skipped={summary.get('skipped', 0)} "
                        f"offline={summary.get('offline_mode', False)}"
                    )
                except Exception as e:
                    log.error(f"[initial-sync] failed: {e}")
                    try:
                        app.state.consumer_gate["sync_summary"] = {"error": str(e)}
                    except Exception:
                        pass
            threading.Thread(
                target=_run_initial_sync,
                daemon=True,
                name="consumer-initial-sync",
            ).start()
            log.info("Consumer gate: initial sync spawned in background")
    log.info("=" * 60)

    # Initialize providers from config (graceful — server starts even if MT5 fails)
    provider = None
    connected = False
    try:
        providers = await asyncio.to_thread(cfg_manager.init_providers)
        provider = cfg_manager.get_provider()  # Active data provider

        if provider:
            try:
                connected = await asyncio.wait_for(
                    asyncio.to_thread(provider.connect),
                    timeout=15.0,  # Don't hang forever if MT5 is stuck
                )
            except asyncio.TimeoutError:
                log.warning("MT5 connection timed out (15s) — continuing without live prices")
                connected = False
            except Exception as e:
                log.warning(f"MT5 connection failed: {e} — continuing without live prices")
                connected = False

            if connected:
                # Resolve universe symbols through the provider
                universe = cfg_manager.get_active_universe()
                resolved = await asyncio.to_thread(provider.resolve_universe, universe)
                log.info(f"Provider ready: {len(resolved)} symbols resolved")
            else:
                log.warning("Running in DEMO MODE — provider failed to connect")
        else:
            log.warning("Running in DEMO MODE — no provider configured")
    except Exception as e:
        log.warning(f"Provider initialization failed: {e} — server will start without MT5")
        provider = None
        connected = False

    # Store provider reference for background tasks
    app.state.provider = provider
    app.state.cfg_manager = cfg_manager

    # Launch background tasks (consumer: tick loop + bar check + file watch only)
    tasks = [
        asyncio.create_task(tick_loop(app.state, manager, server_config, cfg_manager)),
        asyncio.create_task(focus_tick_loop(app.state, manager, cfg_manager)),  # v9
        asyncio.create_task(bar_check_loop(app.state, manager, server_config, cfg_manager)),
        asyncio.create_task(file_watch_loop(manager, server_config)),
    ]
    if provider and provider.connected:
        tasks.append(asyncio.create_task(mt5_reconnect_loop(app.state, server_config)))

    # ── Consumer periodic sync (Rule C1: display-only refresh, 5 min default) ──
    # Polls the VPS at a configurable interval and broadcasts data_sync_complete
    # WS event when fresh files arrive. Frontend listens and refreshes panels.
    if CONSUMER_MODE:
        try:
            from periodic_sync import create_periodic_sync_task
            sync_task_factory = create_periodic_sync_task(manager.broadcast_event)
            tasks.append(asyncio.create_task(sync_task_factory()))
            log.info("Consumer periodic sync task started")
        except Exception as e:
            log.warning(f"Failed to start periodic sync task: {e}")

        # ── Cache directory watcher (v3) ───────────────────────────────────
        # Live updates whenever a file in the cache dir changes on disk.
        # Covers manual drops and future automated DC uploaders. Disable via
        # consumer_config.ini → [sync] cache_watcher_enabled = false.
        try:
            from cache_watcher import create_cache_watcher_task
            watcher_factory = create_cache_watcher_task(manager.broadcast_event)
            if watcher_factory is not None:
                tasks.append(asyncio.create_task(watcher_factory()))
                log.info("Cache watcher task started")
        except Exception as e:
            log.warning(f"Failed to start cache watcher task: {e}")

    # NOTE: Signal lifecycle, auto executor, options archive, trade reconciliation
    # are all PRO-only features — removed in consumer (Rule C1).

    log.info(
        f"Data server ready — "
        f"{len(cfg_manager.get_active_universe())} assets, "
        f"Provider {'connected' if (provider and provider.connected) else 'disconnected (demo mode)'}"
    )
    log.info(f"Listening on http://{server_config.host}:{server_config.port}")

    # --- Disposable debug telemetry harness (env-gated) ---
    if os.environ.get("MK_DEBUG_TELEMETRY") == "1":
        import debug_telemetry
        debug_telemetry.install(app)

    yield

    # ── Shutdown ──
    log.info("Shutting down...")
    for t in tasks:
        t.cancel()
    # Disconnect all providers
    for pid, prov in cfg_manager.get_all_providers().items():
        prov.disconnect()
    log.info("Data server stopped")


app = FastAPI(
    title="Quantum Terminal Data Server",
    version="1.0.0",
    lifespan=lifespan,
)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=False,
    allow_methods=["*"],
    allow_headers=["*"],
)

# ── Wire consumer routes (auth + sync + status) ──
if CONSUMER_MODE:
    wire_consumer_routes(app)
    log.info("Consumer routes wired")

# ── Wire API routers (must be at module level, not inside lifespan) ──
app.include_router(create_config_router(cfg_manager, manager.broadcast_event))
app.include_router(create_execution_router(cfg_manager, app, manager.broadcast_event))
app.include_router(create_regime_router(cfg_manager, app))
app.include_router(create_system_router())
app.include_router(create_account_router(manager.broadcast_event))
app.include_router(create_freshness_router(cfg_manager))
app.include_router(create_trade_mapper_router(app))
app.include_router(create_pulse_router())
app.include_router(create_chart_presets_router())
app.include_router(create_chart_templates_router())

# ── MT5 connection management routes (consumer settings panel) ──
try:
    from mt5_routes import create_mt5_router
    app.include_router(create_mt5_router(cfg_manager, app))
    log.info("MT5 management routes registered (/api/mt5/*)")
except ImportError:
    log.warning("mt5_routes not available — MT5 settings panel won't work")

# ── Tradovate POC routes (consumer settings panel) ──
# v13: standalone proof-of-work integration. Removing this block + the file
#       backend/tradovate_routes.py + providers/tradovate_provider.py fully
#       disables Tradovate without touching anything else.
try:
    from tradovate_routes import create_tradovate_router
    app.include_router(create_tradovate_router(cfg_manager))
    log.info("Tradovate POC routes registered (/api/tradovate/*)")
except ImportError as _e:
    log.warning(f"tradovate_routes not available: {_e}")

if create_logs_router is not None:
    app.include_router(create_logs_router())

if OPTIONS_AVAILABLE:
    try:
        app.include_router(create_options_router())
        log.info("Options quant routes registered (/api/options/*)")
    except Exception as e:
        log.warning(f"Options routes failed to load: {e}")
        OPTIONS_AVAILABLE = False

# ── Fallback options from sync cache (consumer mode) ──
# v12: convert ETF strikes → live CFD prices server-side so frontend never
#      does ratio math. Ratio = mt5_live_price / options.spot (ETF). Frozen
#      per request so both MainChart's sauce lines and OptionsPanel eat the
#      exact same numbers. When MT5 is offline we pass the raw payload
#      through with _conversion_applied=false and the frontend can flag it.

def _mul_options_prices_inplace(data: dict, ratio: float) -> None:
    """Multiply known strike/level/price fields by ratio in-place."""
    def mul(v):
        return v * ratio if isinstance(v, (int, float)) and v is not None else v

    for k in ("hvl", "max_pain", "gex_flip"):
        if k in data and data[k] is not None:
            data[k] = mul(data[k])

    w = data.get("walls") or {}
    for k in ("primary_call_wall", "primary_put_wall"):
        if k in w and w[k] is not None:
            w[k] = mul(w[k])
    for arr_key in ("call_walls", "put_walls"):
        for item in (w.get(arr_key) or []):
            if isinstance(item, dict) and item.get("strike") is not None:
                item["strike"] = mul(item["strike"])

    # v15: convert both swing `expected_move` and the new `expected_move_0dte`
    #   block (shipped by DC 2026-04-21). Same shape — only `pct` is
    #   dimensionless and stays untouched.
    for em_key in ("expected_move", "expected_move_0dte"):
        em = data.get(em_key) or {}
        for k in ("atm_strike", "move_up", "move_down", "straddle"):
            if k in em and em[k] is not None:
                em[k] = mul(em[k])

    for iv_key in ("iv_divergence", "iv_skew"):
        iv = data.get(iv_key) or {}
        for k in ("atm_strike", "call_25d_strike", "put_25d_strike"):
            if k in iv and iv[k] is not None:
                iv[k] = mul(iv[k])

    dh = data.get("dealer_heatmap") or {}
    for k in ("max_buy_strike", "max_sell_strike"):
        if k in dh and dh[k] is not None:
            dh[k] = mul(dh[k])
    if isinstance(dh.get("strikes"), list):
        dh["strikes"] = [mul(s) for s in dh["strikes"]]

    for gex_key in ("gex", "gex_0dte", "gex_stack"):
        g = data.get(gex_key) or {}
        if isinstance(g.get("strikes"), list):
            g["strikes"] = [mul(s) for s in g["strikes"]]

    # v16: convert both `sauce` (swing) and the new `sauce_0dte` block
    #   (shipped by DC 2026-04-21). Same per-level strike conversion — no
    #   mixing. sauce_0dte is null when today ∉ chain (expected).
    for sauce_key in ("sauce", "sauce_0dte"):
        s_block = data.get(sauce_key) or {}
        for s_key in ("volatility_gravity", "structural_integrity"):
            s = s_block.get(s_key) or {}
            for item in (s.get("levels") or []):
                if isinstance(item, dict) and item.get("strike") is not None:
                    item["strike"] = mul(item["strike"])


def _mt5_mid_for(ticker: str) -> Optional[float]:
    """Fetch current MT5 mid price ((bid+ask)/2) for a canonical ticker.
    Returns None if provider offline or tick unavailable."""
    prov = app.state.provider
    if not prov or not getattr(prov, "connected", False):
        return None
    try:
        ticks = prov.get_latest_ticks([ticker])
        t = ticks.get(ticker) if ticks else None
        if t is None:
            return None
        bid = getattr(t, "bid", None)
        ask = getattr(t, "ask", None)
        if bid and ask:
            return (bid + ask) / 2.0
        return getattr(t, "last", None)
    except Exception:
        return None


if not OPTIONS_AVAILABLE:
    @app.get("/api/options/{ticker}/full")
    async def get_options_from_sync(ticker: str, n: int = 8):
        """Serve options data from consumer sync cache with ETF→CFD conversion
        baked in (v12). All strikes / levels / walls in the returned JSON are
        in MT5 CFD price space — frontend consumes as-is."""
        canonical = ticker.upper()
        try:
            import copy
            from data_sync_client import get_sync_client
            raw = get_sync_client().get_options(canonical)
            if not raw:
                raise HTTPException(404, f"No options data for {canonical}")
            # v12 fix: sync_client returns the live cached reference — mutating
            # it double-multiplies strikes on every subsequent request. Deep
            # copy so conversion only applies to the response, not the cache.
            data = copy.deepcopy(raw)

            # Normalize keys: sync cache may use spot_etf; ensure we have an ETF spot.
            etf_spot = data.get("spot_etf") or data.get("spot")
            if etf_spot is None:
                raise HTTPException(502, f"Options data for {canonical} missing spot price")

            if "canonical" not in data:
                data["canonical"] = canonical
            if "cboe_ticker" not in data:
                cboe_map = {"US500": "SPY", "USTEC": "QQQ", "XAUUSD": "GLD"}
                data["cboe_ticker"] = cboe_map.get(canonical)

            mt5_price = await asyncio.to_thread(_mt5_mid_for, canonical)
            levels_space = data.get("levels_space")

            if levels_space == "broker":
                # Producer (v2.0+) pre-multiplied levels to broker space; consumer no-op.
                if mt5_price:
                    data["spot"] = mt5_price
                data["_conversion_applied"] = True
                data["_conversion_source"] = "broker_native"
            elif mt5_price and etf_spot > 0:
                ratio = mt5_price / etf_spot
                _mul_options_prices_inplace(data, ratio)
                # Always preserve the raw ETF spot; replace top-level spot with the
                # live MT5 price so frontend sees CFD space consistently.
                data["spot_etf"] = etf_spot
                data["spot"] = mt5_price
                data["_conversion_applied"] = True
                data["_conversion_ratio"] = ratio
                data["_conversion_source"] = "mt5_live"
            else:
                # No MT5 price available — fall through with raw ETF values so
                # the panel still renders, but flagged so the UI can warn.
                if "spot" not in data:
                    data["spot"] = etf_spot
                data["_conversion_applied"] = False
                data["_conversion_source"] = "mt5_unavailable"
            return data
        except HTTPException:
            raise
        except Exception as e:
            log.warning(f"options fallback failed for {canonical}: {e}")
            raise HTTPException(404, f"No options data for {canonical}")

    log.info("Options fallback route registered (sync cache, v12 CFD-converted)")


# ── Macro Rotation Endpoints (Money Flow Tabs 2 & 3) ──
if MACRO_DATA_AVAILABLE:
    @app.get("/api/macro/sectors")
    async def api_macro_sectors(refresh: bool = False):
        try:
            from data_sync_client import get_sync_client
            data = get_sync_client().get_macro_sectors()
            if data:
                return data
        except Exception as e:
            log.warning(f"[MACRO] macro_sectors failed: {e}")
        raise HTTPException(404, "No sector data available")

    @app.get("/api/macro/countries")
    async def api_macro_countries(refresh: bool = False):
        try:
            from data_sync_client import get_sync_client
            data = get_sync_client().get_macro_countries()
            if data:
                return data
        except Exception as e:
            log.warning(f"[MACRO] macro_countries failed: {e}")
        raise HTTPException(404, "No country data available")

    @app.get("/api/macro/flows")
    async def api_macro_flows(period: str = "1W"):
        try:
            from data_sync_client import get_sync_client
            data = get_sync_client().get_macro_flows()
            if data:
                key = "4W" if period.upper() == "4W" else "1W"
                flows_obj = data.get(key)
                if flows_obj and "flows" in flows_obj:
                    return flows_obj
        except Exception as e:
            log.warning(f"[MACRO] macro_flows failed: {e}")
        raise HTTPException(404, "No country flow data available")

    log.info("Macro rotation routes registered (/api/macro/sectors, /api/macro/countries, /api/macro/flows)")
else:
    log.info("macro_data not available -- yfinance may not be installed")

# -- Money Flow from sync cache (consumer mode) --
@app.get("/api/fundamental/money-flow")
async def get_money_flow(period: str = "macro"):
    """Serve capital flow data from consumer sync cache."""
    try:
        from data_sync_client import get_sync_client
        data = get_sync_client().get_global("macro_sector_flows")
        if data:
            # Data is keyed by period (1W, 4W). Extract requested period.
            period_map = {"macro": "macro", "weekly": "weekly", "1w": "macro", "4w": "weekly"}
            key = period_map.get(period.lower(), period)
            if key in data:
                return data[key]
            # If only one period exists, return it
            for k, v in data.items():
                if isinstance(v, dict) and "flows" in v:
                    return v
            return data
    except Exception:
        pass
    raise HTTPException(404, "No money flow data available")


# -- Fundamental state (macro regime, liquidity, yields, COT, scores) --
@app.get("/api/fundamental_state")
async def get_fundamental_state():
    """Serve fundamental_state.json from consumer sync cache (Rule C1: read-only)."""
    try:
        from data_sync_client import get_sync_client
        data = get_sync_client().get_fundamentals()
        if data:
            return data
    except Exception as e:
        log.warning(f"[FUNDAMENTAL] fundamental_state failed: {e}")
    raise HTTPException(404, "No fundamental state data available")




# ── REST ENDPOINTS ──────────────────────────────────────────

@app.get("/api/health")
async def health():
    """Server health check — backwards-compatible top-level fields plus
    a v11+ `components` block where each subsystem reports its own
    status. Frontends that want detailed diagnostics read components;
    legacy clients that just check `status == "ok"` still work."""
    uptime = time.time() - start_time
    provider = app.state.provider

    # ── Component health ──
    components: Dict[str, Any] = {}

    # MT5 / data provider
    mt5_connected = bool(provider and provider.connected)
    components["mt5"] = {
        "status": "ok" if mt5_connected else "disconnected",
        "type":   provider.provider_type if provider else None,
        "label":  provider.label if provider else None,
        "connected": mt5_connected,
        "active_symbols": len(cfg_manager.get_active_universe()),
    }

    # License / auth — Quantum Terminal: always free
    components[\"license\"] = {
        \"status\": \"ok\",
        \"user_email\": \"free@quantumterminal\",
        \"subscription_status\": \"active\",
        \"subscription_type\": \"free\",
        \"offline\": False,
    }

    # Sync status
    try:
        from data_sync_client import get_sync_client as _gsc
        _sc = _gsc()
        sync_in_progress = bool(getattr(_sc, "_sync_in_progress", False))
        sync_total = int(getattr(_sc, "_sync_total", 0))
        sync_done  = int(getattr(_sc, "_sync_done",  0))
        sync_errors = list(getattr(_sc, "_sync_errors", []) or [])
        last_synced = getattr(_sc, "last_synced_at", None)
        # Status: error if errors and not in progress, syncing if in progress, ok otherwise
        if sync_in_progress:
            sync_status = "syncing"
        elif sync_errors:
            sync_status = "degraded"
        else:
            sync_status = "ok"
        components["sync"] = {
            "status": sync_status,
            "in_progress": sync_in_progress,
            "progress": (sync_done, sync_total),
            "last_synced_at": last_synced,
            "format_version_ok": bool(getattr(_sc, "format_version_ok", True)),
            "errors_count": len(sync_errors),
            "offline": bool(getattr(_sc, "is_offline", False)),
        }
    except Exception as e:
        components["sync"] = {"status": "error", "error": str(e)}

    # WebSocket clients
    components["websockets"] = {
        "status": "ok",
        "price_clients": manager.price_client_count,
        "event_clients": manager.event_client_count,
    }

    # Build version
    version = None
    try:
        from pathlib import Path as _Path
        for vp in [_Path(__file__).parent / "VERSION",
                   _Path(__file__).parent.parent.parent / "VERSION",
                   _Path(__file__).parent.parent / "VERSION"]:
            if vp.exists():
                version = vp.read_text(encoding="utf-8").strip()
                break
    except Exception:
        pass

    # Roll up overall status — error if any component errored, degraded
    # if any is in a non-ok-but-recoverable state, otherwise ok.
    overall = "ok"
    for c in components.values():
        s = c.get("status")
        if s == "error":
            overall = "error"
            break
        if s in ("disconnected", "auth_required", "degraded", "syncing"):
            overall = "degraded"

    return {
        # Back-compat top-level fields (v1 and earlier readers)
        "status": "ok" if overall != "error" else "error",
        "mt5_connected": mt5_connected,
        "uptime_seconds": round(uptime, 1),
        "active_symbols": len(cfg_manager.get_active_universe()),
        "ws_price_clients": manager.price_client_count,
        "ws_event_clients": manager.event_client_count,
        "universe": cfg_manager.get_display_order(),
        "signal_engine_available": False,
        "active_signals": 0,
        "provider": {
            "type": provider.provider_type if provider else None,
            "label": provider.label if provider else None,
            "connected": mt5_connected,
        },
        # v11+ structured component view
        "overall": overall,
        "version": version,
        "components": components,
    }


@app.get("/api/ticks")
async def get_all_ticks():
    """
    Latest tick for every symbol in the universe.
    Used by the frontend on initial load to seed prices
    (before the WebSocket starts streaming).
    """
    provider = app.state.provider
    if not provider or not provider.connected:
        return {"ticks": {}, "note": "MT5 not connected"}
    universe = cfg_manager.get_active_universe()
    ticks = await asyncio.to_thread(provider.get_latest_ticks, universe)
    return {
        "ticks": {t: tick.to_dict() for t, tick in ticks.items()},
    }


@app.post("/api/focus/{ticker}")
async def post_focus(ticker: str):
    """v9: Tell the backend which symbol the user is actively watching.
    The focus_tick_loop polls only this symbol at FOCUS_TICK_INTERVAL when
    USE TICK DATA is on, giving the chart MT5-style live tick movement
    without paying universe-size MT5 polling cost."""
    set_focused_symbol(ticker)
    return {"focused": get_focused_symbol(), "interval": FOCUS_TICK_INTERVAL}


@app.delete("/api/focus")
async def delete_focus():
    """v9: Clear the focused symbol (focus_tick_loop goes idle)."""
    set_focused_symbol(None)
    return {"focused": None}


@app.get("/api/symbol-info/{ticker}")
async def get_symbol_info(ticker: str):
    """
    Real broker symbol metadata — tick_value, contract_size, etc.
    Used by OrderTicket for accurate P&L preview.
    """
    provider = app.state.provider
    if not provider or not provider.connected:
        raise HTTPException(503, "Provider not connected")
    canonical = ticker.upper()
    info = await asyncio.to_thread(provider.get_symbol_info, canonical)
    if info is None:
        raise HTTPException(404, f"Symbol info not found for {canonical}")
    return info.to_dict()


@app.get("/api/symbol-info")
async def get_all_symbol_info():
    """Batch symbol info for the entire universe."""
    provider = app.state.provider
    if not provider or not provider.connected:
        return {"symbols": {}, "note": "Provider not connected"}
    universe = cfg_manager.get_active_universe()
    infos = await asyncio.to_thread(provider.get_all_symbol_info, universe)
    return {
        "symbols": {t: info.to_dict() for t, info in infos.items()},
    }


@app.get("/api/payload")
async def get_payload():
    """Full terminal_payload.json contents. Returns empty in consumer mode."""
    path = server_config.resolve_path(server_config.terminal_payload)
    data = read_json_safe(path)
    if data is None:
        # Consumer mode: no local payload file — data comes via sync cache
        return {"assets": {}, "note": "Consumer mode — data served via /api/cones and /api/sync"}
    return data


@app.get("/api/payload/{ticker}")
async def get_payload_ticker(ticker: str):
    """Single ticker's payload entry."""
    path = server_config.resolve_path(server_config.terminal_payload)
    data = read_json_safe(path)
    if data is None:
        return {"note": "Consumer mode — use /api/sync/cones/{ticker}"}
    assets = data.get("assets", {})
    canonical = ticker.upper()
    if canonical not in assets:
        return {"note": f"No payload entry for {canonical}"}
    return assets[canonical]


@app.get("/api/signals")
async def get_signals():
    """All signals from daily_signals.json."""
    path = server_config.resolve_path(server_config.daily_signals)
    data = read_json_safe(path)
    if data is None:
        return {"signals": {}, "note": "No signals file found"}
    return data


@app.get("/api/signals/{ticker}")
async def get_signals_ticker(ticker: str):
    """Signals for one ticker."""
    path = server_config.resolve_path(server_config.daily_signals)
    data = read_json_safe(path)
    if data is None:
        raise HTTPException(404, "daily_signals.json not found")
    canonical = ticker.upper()
    if canonical not in data:
        return {"ticker": canonical, "signals": []}
    return {"ticker": canonical, "signals": data[canonical]}


# ── Stress Lab (v10) ─────────────────────────────────────────────────────
# Phase 1: consume DC's stress_lab JSON files from the standard AppData
# cache. Files land there either via data_sync_client (future wiring) or
# manual copy for testing. No computation, no synthesis — display-only (C1).
#
# Expected files in %APPDATA%\QuantumTerminal\cache\ :
#   _stress_lab_index.json         — watchlist
#   {TICKER}_stress_lab.json       — per-asset thesis


def _stress_lab_cache_dir() -> Path:
    """Same cache dir the sync_client writes to — flat layout."""
    from data_sync_client import _get_cache_dir
    return _get_cache_dir()


@app.get("/api/stress_lab/index")
async def get_stress_lab_index():
    """Watchlist — list of assets with conviction + spot + one-liner."""
    path = _stress_lab_cache_dir() / "_stress_lab_index.json"
    data = read_json_safe(path)
    if data is None:
        raise HTTPException(404, "Stress Lab index not available yet — waiting on first sync")
    return data


@app.get("/api/stress_lab/{ticker}")
async def get_stress_lab_thesis(ticker: str):
    """Full per-asset thesis JSON produced by DC stress_lab_engine."""
    canonical = ticker.upper()
    path = _stress_lab_cache_dir() / f"{canonical}_stress_lab.json"
    data = read_json_safe(path)
    if data is None:
        raise HTTPException(404, f"No Stress Lab thesis for {canonical}")
    return data


# v11: Historical backtest positions for the "last filled signal" replay panel.
@app.get("/api/stress_lab/backtest/positions")
async def get_stress_lab_backtest_positions():
    path = _stress_lab_cache_dir() / "_stress_lab_backtest_positions.json"
    data = read_json_safe(path)
    if data is None:
        raise HTTPException(404, "Backtest positions not available yet")
    return data


# v11: Historical bar range — replay chart on Stress Lab last-signal panel.
#      `from` is a Python keyword, so we accept it via Query(alias="from").
from fastapi import Query as _Query
from datetime import datetime as _dt, timezone as _tz

@app.get("/api/bars_range/{ticker}")
async def get_bars_range(
    ticker: str,
    timeframe: str = "M15",
    from_: str = _Query(..., alias="from"),
    to: str = _Query(...),
):
    """OHLCV bars between two ISO-8601 UTC datetimes.
    Used by the Stress Lab last-signal replay chart."""
    canonical = ticker.upper()
    provider = app.state.provider
    if not provider or not provider.connected:
        raise HTTPException(503, "No data provider connected")
    try:
        f_dt = _dt.fromisoformat(from_.replace("Z", "+00:00"))
        t_dt = _dt.fromisoformat(to.replace("Z", "+00:00"))
    except ValueError as e:
        raise HTTPException(400, f"bad ISO datetime: {e}")
    if f_dt.tzinfo is None: f_dt = f_dt.replace(tzinfo=_tz.utc)
    if t_dt.tzinfo is None: t_dt = t_dt.replace(tzinfo=_tz.utc)
    if not hasattr(provider, "get_bars_range"):
        raise HTTPException(501, "provider does not support range fetch")
    bars = await asyncio.to_thread(
        provider.get_bars_range, canonical, timeframe, f_dt, t_dt
    )
    if not bars:
        return {"bars": [], "ticker": canonical, "timeframe": timeframe,
                "from": from_, "to": to}
    # Normalize to the shape frontend expects (LWC candles).
    return {
        "ticker": canonical,
        "timeframe": timeframe,
        "from": from_,
        "to": to,
        "bars": [{
            "time": b.time, "open": b.open, "high": b.high,
            "low": b.low, "close": b.close, "volume": b.volume,
        } for b in bars],
    }


@app.get("/api/bands/{ticker}")
async def get_bands(ticker: str):
    """Probability bands data for a ticker. Falls back to sync cache."""
    canonical = ticker.upper()
    # Try PRO local file first
    bands_dir = server_config.resolve_path(server_config.bands_data_dir)
    bands_file = bands_dir / f"{canonical}_bands.json"
    data = read_json_safe(bands_file)
    if data:
        # v26: stamp file mtime for staleness detection on the frontend.
        try:
            data["_last_modified"] = bands_file.stat().st_mtime
        except Exception:
            pass
        return data

    # Fallback 1 (v2 — Data Center): standalone {TICKER}_bands.json in sync cache.
    try:
        from data_sync_client import get_sync_client
        client = get_sync_client()
        bands_data = client.get_bands(canonical)
        if bands_data:
            # v26: best-effort mtime from the sync cache file. Naming
            # patterns observed: <TICKER>_bands.json or GLOBAL_<lower>_bands.json.
            try:
                cache_dir = getattr(client, "cache_dir", None)
                if cache_dir is not None:
                    from pathlib import Path as _P
                    cache_dir_p = _P(cache_dir) if not isinstance(cache_dir, _P) else cache_dir
                    for name in (f"{canonical}_bands.json",
                                 f"GLOBAL_{canonical.lower()}_bands.json"):
                        candidate = cache_dir_p / name
                        if candidate.exists():
                            bands_data["_last_modified"] = candidate.stat().st_mtime
                            break
            except Exception:
                pass
            return bands_data
    except Exception:
        pass

    # Fallback 2 (legacy): bands nested inside the cones sync blob.
    # Retained so any cache still holding pre-transition files keeps working.
    try:
        sync_data = client.get_cones(canonical)
        if sync_data and "bands" in sync_data:
            return sync_data["bands"]
    except Exception:
        pass

    raise HTTPException(404, f"No bands data for {canonical}")


def _sanitize_json(data):
    """Replace NaN/Infinity with None — these are not valid JSON."""
    raw = json.dumps(data, allow_nan=True, default=str)
    raw = raw.replace(": NaN", ": null").replace(":NaN", ":null")
    raw = raw.replace(": Infinity", ": null").replace(":Infinity", ":null")
    raw = raw.replace(": -Infinity", ": null").replace(":-Infinity", ":null")
    return json.loads(raw)


@app.get("/api/state")
async def get_state():
    """Weekly state (regime, anchors, WF params)."""
    path = server_config.resolve_path(server_config.weekly_state)
    data = read_json_safe(path)
    if data is None:
        return {"assets": {}, "note": "No weekly state found — run orchestrator weekly tier"}
    return _sanitize_json(data)


@app.get("/api/bars/{ticker}")
async def get_bars(ticker: str, timeframe: str = "M15", count: int = 200):
    """
    Fetch historical bars directly from MT5.
    Used for initial chart load and timeframe switches.
    """
    canonical = ticker.upper()
    if canonical not in cfg_manager.get_active_universe():
        raise HTTPException(404, f"Ticker {canonical} not in universe")

    provider = app.state.provider
    if not provider or not provider.connected:
        raise HTTPException(503, "No data provider connected")

    bars = await asyncio.to_thread(provider.get_bars, canonical, timeframe, count)
    if not bars:
        raise HTTPException(404, f"No bar data for {canonical} {timeframe}")

    return {
        "ticker": canonical,
        "timeframe": timeframe,
        "count": len(bars),
        "bars": [b.to_dict() for b in bars],
    }


@app.get("/api/lifecycle")
async def get_lifecycle():
    """Signal lifecycle state."""
    path = server_config.resolve_path(server_config.signal_lifecycle)
    data = read_json_safe(path)
    if data is None:
        return {"signals": {}, "note": "No lifecycle file found"}
    return data



# REMOVED: /api/signals/evaluate, /api/orchestrator, /api/calculate (Rule C1)


@app.get("/api/anchors/{ticker}")
async def get_anchors(ticker: str):
    """Institutional anchors + extremes. Falls back to sync cache."""
    canonical = ticker.upper()
    # Try PRO local file first
    anchors_path = PROJECT_ROOT / "terminal_anchors.json"
    data = read_json_safe(anchors_path)
    if data and canonical in data:
        return data[canonical]

    # Fallback: consumer sync cache — anchors are inside the cones response
    try:
        from data_sync_client import get_sync_client
        sync_data = get_sync_client().get_cones(canonical)
        if sync_data:
            result = {}
            if "anchors" in sync_data:
                result["anchors"] = sync_data["anchors"]
            if "extremes" in sync_data:
                result["extremes"] = sync_data["extremes"]
            if result:
                return result
    except Exception:
        pass

    raise HTTPException(404, f"No anchor data for {canonical}")


@app.get("/api/anchors")
async def get_all_anchors():
    """All tickers' anchor data. Falls back to sync cache."""
    anchors_path = PROJECT_ROOT / "terminal_anchors.json"
    data = read_json_safe(anchors_path)
    if data:
        return data
    # Fallback: consumer sync cache
    try:
        from data_sync_client import get_sync_client
        client = get_sync_client()
        result = {}
        for ticker in client.synced_tickers:
            sync_data = client.get_cones(ticker)
            if sync_data:
                entry = {}
                if "anchors" in sync_data:
                    entry["anchors"] = sync_data["anchors"]
                if "extremes" in sync_data:
                    entry["extremes"] = sync_data["extremes"]
                if entry:
                    result[ticker] = entry
        if result:
            return result
    except Exception:
        pass
    return {"anchors": {}, "note": "No anchor data found"}


@app.get("/api/regime_live/{ticker}")
async def get_regime_live_stub(ticker: str, timeframe: str = "M15", bars: int = 500):
    """Stub — regime_live is a PRO-only endpoint (Rule C1).
    Returns empty payload so the frontend stops 404-retrying."""
    return {
        "ticker": ticker.upper(),
        "timeframe": timeframe,
        "data": [],
        "note": "regime_live unavailable in consumer build"
    }


@app.get("/api/cones/{ticker}")
async def get_cones(ticker: str):
    """Probability cone data for a ticker. Falls back to sync cache in consumer mode.
    Also merges GEX cones from the options sync cache (gex_cones lives in
    options JSON, not cones JSON, on the producer side)."""
    canonical = ticker.upper()
    # Try PRO local file first
    cones_file = server_config.resolve_path("terminal_cones.json")
    data = read_json_safe(cones_file)
    if data and canonical in data:
        return data[canonical]

    # Fallback: consumer sync cache (Rule C5)
    result = None
    try:
        from data_sync_client import get_sync_client
        client = get_sync_client()
        sync_data = client.get_cones(canonical)
        if sync_data and "cones" in sync_data:
            result = dict(sync_data.get("cones", {}))
        elif sync_data:
            result = dict(sync_data)

        # Merge gex_cones from the options sync cache. The producer writes
        # gex_now / gex_weekly_1 / gex_weekly_2 into the options JSON, not
        # the cones JSON. Their shape (median, sd1_high, sd1_low, sd2_high,
        # sd2_low, sd3_high, sd3_low, dates) matches the standard cone shape
        # the frontend renderer expects, so a flat merge is enough.
        opt_data = client.get_options(canonical)
        if opt_data and isinstance(opt_data.get("gex_cones"), dict):
            if result is None:
                result = {}
            for k, v in opt_data["gex_cones"].items():
                # Don't clobber any pre-existing key (cones JSON wins if both exist)
                if k not in result:
                    result[k] = v
    except Exception:
        pass

    if result:
        return result
    raise HTTPException(404, f"No cone data for {canonical}")


@app.get("/api/quarterly_cones/{ticker}")
async def get_quarterly_cones(ticker: str):
    """Quarterly cone forecast (183-day+ horizon, 4 anchors × 2 models).
    v6 — Reads {TICKER}_quarterly_cones.json from the consumer cache.
    v7 — Also handles the Data Center's GLOBAL_<lower>_quarterly_cones.json
         naming convention (same legacy pattern as historical_cones / scalp_bands).
         Routes through the sync client first (in-memory store populated by
         resolve_cache_filename), then falls back to direct file reads.
    Shape: median + sd1/2/3 high/low + dates per model, keys prefixed
           gbm_quarter_*, mjd_quarter_*."""
    import json as _json
    canonical = ticker.upper()
    try:
        from data_sync_client import get_sync_client, _get_cache_dir
        client = get_sync_client()
        # (1) in-memory store (populated by sync_all + _load_all_from_cache)
        data = client._get_data(canonical, "quarterly_cones")
        if data:
            return data
        # (2) direct disk fallback — try both naming conventions explicitly,
        #     in case the cache scan hasn't run yet for this session.
        cache_dir = _get_cache_dir()
        for path in (
            cache_dir / f"{canonical}_quarterly_cones.json",
            cache_dir / f"GLOBAL_{canonical.lower()}_quarterly_cones.json",
        ):
            if path.is_file():
                return _json.loads(path.read_text(encoding="utf-8"))
    except Exception as e:
        log.warning(f"quarterly_cones load failed for {canonical}: {e}")
    raise HTTPException(404, f"No quarterly cone data for {canonical}")


# v14: TEMPORARY dev-only route. Reads the producer's bands-replay JSON
#      directly from D:\MK_DATA_CENTER\temp_out so the replay view can
#      render the new per-day anchor schema before the real sync pipeline
#      exists. Delete this function (and its import side-effects) to revert.
@app.get("/api/bands_replay/{ticker}")
async def get_bands_replay(ticker: str):
    """Bands-replay prototype — reads {TICKER}_bands_replay.json from
    D:\\MK_DATA_CENTER\\temp_out (dev path only). Schema matches
    /api/historical_cones but with a 'daily' bucket, a 'head' warm-start
    block, and fusion_instructions metadata. Returns 404 if the file
    isn't present (non-XAUUSD tickers, production boxes, etc)."""
    import json as _json
    from pathlib import Path as _Path
    canonical = ticker.upper()
    test_dir = _Path(r"D:\MK_DATA_CENTER\temp_out")
    path = test_dir / f"{canonical}_bands_replay.json"
    if not path.is_file():
        raise HTTPException(404, f"No bands_replay data for {canonical}")
    try:
        return _json.loads(path.read_text(encoding="utf-8"))
    except Exception as e:
        log.warning(f"bands_replay load failed for {canonical}: {e}")
        raise HTTPException(500, f"bands_replay parse failed for {canonical}")


@app.get("/api/scalp_bands/{ticker}")
async def get_scalp_bands(ticker: str):
    """Scalp bands (5-min anchor + per-horizon projections) for the MICRO
    toggle in the BANDS dropdown. Served from the sync cache.
    Source file: GLOBAL_<lower_ticker>_scalp_bands.json (resolver-routed)."""
    canonical = ticker.upper()
    try:
        from data_sync_client import get_sync_client
        client = get_sync_client()
        data = client._get_data(canonical, "scalp_bands")
        if data:
            return data
    except Exception:
        pass
    raise HTTPException(404, f"No scalp bands data for {canonical}")


@app.get("/api/historical_cones/{ticker}")
async def get_historical_cones(ticker: str):
    """Historical cone anchors (weekly + monthly) for the REPLAY sub-tab.
    Each anchor date maps to a dict with gbm / mjd / bates model payloads
    (median + sd1/2/3 high/low + dates). Served from the sync cache.
    v17: mirrors the quarterly_cones endpoint — falls back to direct file
    reads (both `{TICKER}_*.json` and `GLOBAL_<lower>_*.json` shapes)
    when the in-memory store hasn't been populated yet. Without this, a
    cold-boot or stalled-sync session would 404 on tickers that have a
    file-on-disk but no store entry yet (e.g. EURUSD).
    """
    import json as _json
    canonical = ticker.upper()
    try:
        from data_sync_client import get_sync_client, _get_cache_dir
        client = get_sync_client()
        # (1) in-memory store (populated by sync_all + _load_all_from_cache)
        data = client._get_data(canonical, "historical_cones")
        if data:
            return data
        # (2) direct disk fallback — handles legacy GLOBAL_<lower> naming.
        cache_dir = _get_cache_dir()
        for path in (
            cache_dir / f"{canonical}_historical_cones.json",
            cache_dir / f"GLOBAL_{canonical.lower()}_historical_cones.json",
        ):
            if path.is_file():
                return _json.loads(path.read_text(encoding="utf-8"))
    except Exception as e:
        log.warning(f"historical_cones load failed for {canonical}: {e}")
    raise HTTPException(404, f"No historical cone data for {canonical}")


@app.get("/api/cones")
async def get_all_cones():
    """All computed cone data. Falls back to sync cache."""
    cones_file = server_config.resolve_path("terminal_cones.json")
    data = read_json_safe(cones_file)
    if data:
        return data
    # Fallback: consumer sync cache
    try:
        from data_sync_client import get_sync_client
        client = get_sync_client()
        result = {}
        for ticker in client.synced_tickers:
            sync_data = client.get_cones(ticker)
            if sync_data and "cones" in sync_data:
                result[ticker] = sync_data["cones"]
            elif sync_data:
                result[ticker] = sync_data
        if result:
            return result
    except Exception:
        pass
    return {"note": "No cones data found"}


# ── PROBABILITY FIELD ENDPOINT ────────────────────────────

@app.get("/api/probability-field/{ticker}")
async def get_probability_field(ticker: str):
    """
    Probability field (v2.0) — consumer version.

    Serves the full pre-computed payload from sync cache.
    Both modes (month_curr, month_prev) are bundled in one response —
    the frontend picks which mode to render from the cached payload.

    Rule C1: no live computation. Rule C5: cache-only serving.
    """
    canonical = ticker.upper()

    try:
        from data_sync_client import get_sync_client
        probfield = get_sync_client().get_probfield(canonical)
    except Exception as e:
        log.warning(f"[probfield] get_probfield failed for {canonical}: {e}")
        probfield = None

    if not probfield:
        raise HTTPException(
            404,
            f"No probability field data for {canonical}. "
            f"Data may not have synced yet."
        )

    # Return the full v2.0 payload unchanged.
    # Frontend reads probfield.modes.month_curr / month_prev directly.
    return probfield

# ── PATH OUTCOME FOREST (PRO) ─────────────────────────────
# v20: serve the per-ticker path_forest forecast from sync cache. Producer
# writes GLOBAL_<lower>_path_forest.json once live; meanwhile DC has shipped
# 5 scenario fixtures (trend / range / stress / mixed / miscal) so the
# frontend can develop against varied conditions. The `scenario` query param
# picks among the fixtures; live filename takes priority when present.

@app.get("/api/forest")
async def list_forest_assets():
    """
    v21: List every ticker that has a path_forest forecast in cache.
    Used by the Stress Lab → Forest Path sub-tab so its asset selector
    auto-populates with whatever DC has shipped instead of relying on a
    hardcoded probe list. Returns ISO sorted upper-case tickers.
    """
    import re
    from data_sync_client import _get_cache_dir
    cache_dir = _get_cache_dir()
    if not cache_dir or not cache_dir.exists():
        return {"tickers": []}
    pattern = re.compile(r"^GLOBAL_([a-z0-9_]+)_path_forest(?:_[a-z]+)?\.json$")
    found = set()
    try:
        for p in cache_dir.iterdir():
            if not p.is_file():
                continue
            m = pattern.match(p.name)
            if m:
                found.add(m.group(1).upper())
    except Exception as e:
        log.warning(f"[forest] index scan failed: {e}")
    return {"tickers": sorted(found)}


@app.get("/api/forest/{ticker}")
async def get_forest(ticker: str, scenario: str = "mixed"):
    """
    Path Outcome Forest forecast (PRO feature).
    Reads the cached path_forest JSON for the given ticker.
    Rule C1: no live computation. Rule C5: cache-only.
    """
    from data_sync_client import _get_cache_dir
    canonical = (ticker or "").lower()
    if not canonical:
        raise HTTPException(400, "ticker required")

    valid = {"trend", "range", "stress", "mixed", "miscal"}
    if scenario not in valid:
        scenario = "mixed"

    cache_dir = _get_cache_dir()
    candidates = [
        cache_dir / f"GLOBAL_{canonical}_path_forest.json",
        cache_dir / f"GLOBAL_{canonical}_path_forest_{scenario}.json",
    ]
    for p in candidates:
        try:
            if p.exists():
                with open(p, "r", encoding="utf-8") as f:
                    return json.load(f)
        except Exception as e:
            log.warning(f"[forest] read failed for {p}: {e}")
            continue

    raise HTTPException(
        404,
        f"No path_forest data for {ticker} (tried scenario '{scenario}'). "
        f"Producer may not have fired yet."
    )

# ── MANUAL CONE ENDPOINTS ──────────────────────────────────


# REMOVED: /api/cones/manual (Rule C1 — computation endpoint)


@app.get("/api/performance/summary")
async def get_performance_summary(days: int = 90):
    """
    Comprehensive performance summary from auto_executor_log.json.
    Includes win/loss metrics in $ and pips, gate funnel, daily timeline.
    Used by the PerformanceDashboard component.
    Note: win/loss are EXPECTED (based on TP/SL), not realised (no outcome tracking yet).
    """
    log_path = PROJECT_ROOT / "auto_executor_log.json"
    if not log_path.exists():
        return {"total": 0, "days": days}

    try:
        with open(log_path) as f:
            all_records = json.load(f)
    except Exception:
        return {"total": 0, "error": "Failed to read log"}

    from datetime import timedelta
    cutoff = (datetime.now(timezone.utc) - timedelta(days=days)).isoformat()
    records = [r for r in all_records if r.get("timestamp", "") >= cutoff]

    if not records:
        return {"total": 0, "days": days}

    # ── Gate funnel (all records) ──
    funnel: dict = {}
    for r in records:
        action = r.get("action", "unknown")
        gate   = r.get("gate", action)
        key = gate if action in ("skipped", "blocked") else action
        funnel[key] = funnel.get(key, 0) + 1

    # ── Executed trades ONLY for P&L metrics ──
    executed = [r for r in records if r.get("action") == "executed"]
    # Dry-run trades for activity display only
    dry_run  = [r for r in records if r.get("action") == "dry_run"]
    trades   = executed + dry_run  # combined for non-P&L distributions

    ne = len(executed)  # count of real executions

    # ── Win/loss helpers (expected, based on TP/SL distances) ──
    def _win_usd(r):
        return round(r.get("risk_usd", 0) * r.get("risk_reward", 0), 2)

    def _loss_usd(r):
        return round(r.get("risk_usd", 0), 2)

    def _win_pips(r):
        entry  = r.get("fill_price") or r.get("entry_price", 0)
        target = r.get("take_profit", 0)
        if not entry or not target:
            return 0.0
        return _to_pips(r.get("ticker", ""), abs(target - entry))

    def _loss_pips(r):
        entry = r.get("fill_price") or r.get("entry_price", 0)
        sl    = r.get("stop_loss", 0)
        if not entry or not sl:
            return 0.0
        return _to_pips(r.get("ticker", ""), abs(entry - sl))

    avg_win_usd  = round(sum(_win_usd(r)  for r in executed) / ne, 2) if ne else 0
    avg_loss_usd = round(sum(_loss_usd(r) for r in executed) / ne, 2) if ne else 0
    avg_win_pips  = round(sum(_win_pips(r)  for r in executed) / ne, 1) if ne else 0
    avg_loss_pips = round(sum(_loss_pips(r) for r in executed) / ne, 1) if ne else 0
    total_risk_usd   = round(sum(_loss_usd(r) for r in executed), 2)
    avg_risk_pct     = round(sum(r.get("risk_pct", 0) for r in executed) / ne, 4) if ne else 0
    avg_conf         = round(sum(r.get("confidence", 0) for r in executed) / ne, 4) if ne else 0
    avg_rr           = round(sum(r.get("risk_reward", 0) for r in executed) / ne, 2) if ne else 0

    # ── By ticker (executed only) ──
    by_ticker: dict = {}
    for r in executed:
        t = r.get("ticker", "?")
        by_ticker.setdefault(t, {
            "count": 0, "risk_usd": 0.0, "win_usd": 0.0,
            "win_pips": 0.0, "loss_pips": 0.0, "confidence": [],
        })
        by_ticker[t]["count"] += 1
        by_ticker[t]["risk_usd"]  = round(by_ticker[t]["risk_usd"]  + _loss_usd(r), 2)
        by_ticker[t]["win_usd"]   = round(by_ticker[t]["win_usd"]   + _win_usd(r),  2)
        by_ticker[t]["win_pips"]  = round(by_ticker[t]["win_pips"]  + _win_pips(r), 1)
        by_ticker[t]["loss_pips"] = round(by_ticker[t]["loss_pips"] + _loss_pips(r), 1)
        by_ticker[t]["confidence"].append(r.get("confidence", 0))
    for t, d in by_ticker.items():
        c = d.pop("confidence")
        cnt = d["count"]
        d["avg_confidence"] = round(sum(c) / len(c), 4) if c else 0
        d["avg_win_usd"]    = round(d["win_usd"]   / cnt, 2) if cnt else 0
        d["avg_loss_usd"]   = round(d["risk_usd"]  / cnt, 2) if cnt else 0
        d["avg_win_pips"]   = round(d["win_pips"]  / cnt, 1) if cnt else 0
        d["avg_loss_pips"]  = round(d["loss_pips"] / cnt, 1) if cnt else 0

    # ── By signal type (executed only) ──
    by_type: dict = {}
    for r in executed:
        st = r.get("signal_type", "?")
        by_type[st] = by_type.get(st, 0) + 1

    # ── By calibration grade (executed only) ──
    by_grade: dict = {}
    for r in executed:
        g = r.get("calibration_grade", "?") or "?"
        by_grade[g] = by_grade.get(g, 0) + 1

    # ── By regime (executed only) ──
    by_regime: dict = {}
    for r in executed:
        reg = r.get("regime", "?") or "?"
        by_regime[reg] = by_regime.get(reg, 0) + 1

    # ── By direction (executed only) ──
    by_direction: dict = {"long": 0, "short": 0}
    for r in executed:
        d = r.get("direction", "")
        if d in by_direction:
            by_direction[d] += 1

    # ── Daily timeline (all records for activity view) ──
    daily: dict = {}
    for r in records:
        ts = r.get("timestamp", "")[:10]
        if not ts:
            continue
        if ts not in daily:
            daily[ts] = {"executed": 0, "dry_run": 0, "skipped": 0, "blocked": 0, "awaiting": 0}
        action = r.get("action", "")
        if action == "executed":           daily[ts]["executed"] += 1
        elif action == "dry_run":          daily[ts]["dry_run"] += 1
        elif action == "skipped":          daily[ts]["skipped"] += 1
        elif action == "blocked":          daily[ts]["blocked"] += 1
        elif action == "awaiting_confirmation": daily[ts]["awaiting"] += 1
    timeline = [{"date": d, **counts} for d, counts in sorted(daily.items())]

    # ── Skipped by gate ──
    by_gate: dict = {}
    for r in records:
        if r.get("action") == "skipped":
            g = r.get("gate", "unknown")
            by_gate[g] = by_gate.get(g, 0) + 1

    return {
        "days": days,
        "total_records": len(records),
        "total_executed": ne,
        "total_dry_run": len(dry_run),
        "avg_confidence": avg_conf,
        "avg_risk_reward": avg_rr,
        "avg_win_usd": avg_win_usd,
        "avg_loss_usd": avg_loss_usd,
        "avg_win_pips": avg_win_pips,
        "avg_loss_pips": avg_loss_pips,
        "total_risk_usd": total_risk_usd,
        "avg_risk_pct": avg_risk_pct,
        "funnel": funnel,
        "by_gate": by_gate,
        "timeline": timeline,
        "by_ticker": by_ticker,
        "by_type": by_type,
        "by_grade": by_grade,
        "by_regime": by_regime,
        "by_direction": by_direction,
    }


# ── WEBSOCKET ENDPOINTS ─────────────────────────────────────

@app.websocket("/ws/prices")
async def ws_prices(ws: WebSocket):
    """
    Live price stream.
    Client can optionally send subscribe messages to filter tickers.
    By default, all universe tickers are streamed.
    """
    await manager.connect_prices(ws)
    try:
        while True:
            # Keep connection alive; process client messages
            try:
                raw = await asyncio.wait_for(ws.receive_text(), timeout=30.0)
                # Parse client commands (subscribe/unsubscribe)
                try:
                    msg = json.loads(raw)
                    if msg.get("action") == "subscribe":
                        # Future: per-client filtering
                        log.debug(f"Client subscribe: {msg.get('tickers', [])}")
                except json.JSONDecodeError:
                    pass
            except asyncio.TimeoutError:
                # Send keepalive ping
                try:
                    await ws.send_text(json.dumps({"type": "ping"}))
                except Exception:
                    break
    except WebSocketDisconnect:
        pass
    finally:
        manager.disconnect_prices(ws)


@app.websocket("/ws/events")
async def ws_events(ws: WebSocket):
    """
    State change event stream.
    Pushes notifications when state files change.
    """
    await manager.connect_events(ws)
    try:
        while True:
            try:
                # Just keep alive — events are pushed server-side
                await asyncio.wait_for(ws.receive_text(), timeout=30.0)
            except asyncio.TimeoutError:
                try:
                    await ws.send_text(json.dumps({"type": "ping"}))
                except Exception:
                    break
    except WebSocketDisconnect:
        pass
    finally:
        manager.disconnect_events(ws)


# ============================================================
# 7. ENTRY POINT
# ============================================================

def main():
    parser = argparse.ArgumentParser(description="Quantum Terminal Data Server")
    parser.add_argument("--host", default=server_config.host, help="Bind address")
    parser.add_argument("--port", type=int, default=server_config.port, help="Port number")
    parser.add_argument("--reload", action="store_true", help="Auto-reload on code changes")
    args = parser.parse_args()

    server_config.host = args.host
    server_config.port = args.port

    uvicorn.run(
        "data_server:app",
        host=args.host,
        port=args.port,
        reload=args.reload,
        log_level="info",
    )


if __name__ == "__main__":
    main()