From d184bc323c533760bf49c1968198f3a42243a290 Mon Sep 17 00:00:00 2001 From: "2569718930@qq.com" <2569718930@qq.com> Date: Mon, 9 Mar 2026 04:41:24 +0800 Subject: [PATCH] feat: introduce PolyWeather web map API and frontend with integrated weather data collection and analysis. --- README.md | 32 +- README_ZH.md | 32 +- docs/API_ZH.md | 246 +++-- docs/COMMERCIALIZATION.md | 17 +- docs/TECH_DEBT.md | 6 +- frontend/README.md | 8 +- frontend/app/page.tsx | 5 +- frontend/public/legacy/index.html | 160 +-- frontend/public/static/app.js | 1381 ++++++++++++++++-------- frontend/public/static/style.css | 170 +++ src/data_collection/weather_sources.py | 89 +- tmp_fix_legacy.js | 130 +++ web/app.py | 59 + 13 files changed, 1701 insertions(+), 634 deletions(-) create mode 100644 tmp_fix_legacy.js diff --git a/README.md b/README.md index 10ca2e73..384f506d 100644 --- a/README.md +++ b/README.md @@ -41,11 +41,11 @@ This project utilizes a production-grade decoupled architecture for high availab ## 🚀 Core Features - **📡 Full-Spectrum Collection** - - **Major Models**: Real-time sync for ECMWF, GFS, ICON, GEM, and JMA high temperatures. - - **Observed Data**: Global airport METAR reports + official Turkish MGM station-level data. - - **Centralized Correction**: Integrated `17130` (Center) official data specifically for Ankara. + - **Major Models**: Real-time sync for ECMWF, GFS, ICON, GEM, JMA, and Open-Meteo daily/hourly guidance. + - **Observed Data**: Aviation Weather / METAR as the primary observation source, plus official Turkish MGM station-level data for Ankara. + - **City Specialization**: Integrated `17130` (`Ankara (Bölge/Center)`) as the key Ankara lead station, without replacing LTAC settlement observation. - **⚖️ DEB Smart Blending** - - Dynamic weighting of forecasts based on recent 7-day historical performance. + - Dynamic weighting of forecasts based on recent historical performance and city-specific reliability. - **🔔 Alert Engine** - **Momentum Spike**: Captures rapid temperature changes within 30 minutes. - **Forecast Breakthrough**: Fires when observations exceed all model predictions plus a safety margin. @@ -67,6 +67,23 @@ This project utilizes a production-grade decoupled architecture for high availab --- +## 🧭 Current Data Logic + +- **Primary observation source**: Aviation Weather / METAR +- **Ankara enhancement**: + - Settlement observation: `LTAC / Esenboğa` + - Official lead station: `Ankara (Bölge/Center)` / `17130` + - Nearby station layer: Turkish MGM station network +- **Other cities nearby layer**: + - Current production logic uses Aviation Weather METAR clusters + - U.S. cities may later receive Mesonet augmentation, but METAR remains the base layer +- **Historical benchmarking rules**: + - Web dashboard: rolling last 15 settled days + - Telegram `/deb`: rolling last 7 settled days + - The current in-progress day is excluded from win rate and MAE + +--- + ## 🏗️ System Architecture ```mermaid @@ -127,7 +144,7 @@ Associate the `frontend` directory as the project root on Vercel for automatic C | :-------- | :-------------------------------------- | :------------- | | `/city` | Query real-time analysis for a city | `/city ankara` | | `/deb` | View historical accuracy of DEB model | `/deb london` | -| `/points` | View your activity points & leaderboard | `/points` | +| `/top` | View activity leaderboard | `/top` | | `/help` | Get detailed instructions | `/help` | --- @@ -136,11 +153,14 @@ Associate the `frontend` directory as the project root on Vercel for automatic C > **Commercialization**: This project currently offers **Web Dashboard ($5/mo)** and **Telegram Signal Channel ($1/mo)** subscriptions. > Point-earning via group participation is active and points can be redeemed for access. +> [!NOTE] +> **Frontend Model**: The production website still uses the legacy high-density dashboard shell under `frontend/public/legacy/index.html`, embedded through the Next.js app shell. + --- --- -**📅 Last Updated**: 2026-03-08 +**📅 Last Updated**: 2026-03-09 **🚀 Status**: v1.0 Stable - Professional Quant UI Locked > [!TIP] diff --git a/README_ZH.md b/README_ZH.md index 8e23e242..df8e3264 100644 --- a/README_ZH.md +++ b/README_ZH.md @@ -41,11 +41,11 @@ PolyWeather 是一套专为 **Polymarket** 深度博弈者设计的实时情报 ## 🚀 核心功能 - **📡 多源全量采集** - - **主流模型**：ECMWF, GFS, ICON, GEM, JMA 实时最高温同步。 - - **实测数据**：全球机场 METAR 定时报文 + 土耳其 MGM 局点官方实测。 - - **中心化纠偏**：针对安卡拉特别接入 `17130` (Center) 官方指挥中心数据。 + - **主流模型**：ECMWF、GFS、ICON、GEM、JMA、Open-Meteo 的日/小时温度指导。 + - **实测数据**：Aviation Weather / METAR 作为主观测源，安卡拉叠加土耳其 MGM 官方站点实测。 + - **城市特化**：安卡拉特别接入 `17130`（`Ankara (Bölge/Center)`）作为领先站，不替代 LTAC 结算主站。 - **⚖️ DEB 智能融合** - - 基于近期 7 天历史表现，动态调整各模型权重的博弈预测。 + - 基于近期历史表现和城市级可靠性，动态调整各模型权重。 - **🔔 异动预警系统 (Alert Engine)** - **动量突变**：捕捉 30 分钟内的急剧温变。 - **预测突破**：当实测击穿所有预报上限时触发告警。 @@ -67,6 +67,23 @@ PolyWeather 是一套专为 **Polymarket** 深度博弈者设计的实时情报 --- +## 🧭 当前数据逻辑 + +- **主观测源**：Aviation Weather / METAR +- **安卡拉增强逻辑**： + - 结算主观测：`LTAC / Esenboğa` + - 官方领先站：`Ankara (Bölge/Center)` / `17130` + - 周边站层：土耳其 MGM 周边站网络 +- **其他城市周边站层**： + - 生产环境当前使用 Aviation Weather METAR cluster + - 美国城市后续可追加 Mesonet，但 METAR 仍是基础层 +- **历史对账口径**： + - 网页端：滚动近 15 个已结算自然日 + - Telegram `/deb`：滚动近 7 个已结算自然日 + - 当天未结算样本不参与胜率与 MAE 统计 + +--- + ## 🏗️ 架构解析 ```mermaid @@ -127,7 +144,7 @@ docker-compose up -d --build | :-------- | :------------------------ | :------------- | | `/city` | 查询指定城市实时分析 | `/city ankara` | | `/deb` | 查看 DEB 模型的历史准确率 | `/deb london` | -| `/points` | 查看您的活跃积分与排行榜 | `/points` | +| `/top` | 查看活跃积分排行榜 | `/top` | | `/help` | 获取详细功能说明 | `/help` | --- @@ -136,11 +153,14 @@ docker-compose up -d --build > **商业化提示**：本项目目前提供 **Web 仪表盘 ($5/月)** 与 **Telegram 信号频道 ($1/月)** 订阅服务。 > 发言获取积分逻辑已上线，活跃用户可兑换相应权限。 +> [!NOTE] +> **前端现状**：生产环境网页仍使用 `frontend/public/legacy/index.html` 的高密度 legacy dashboard，由 Next.js 外壳嵌入并在 Vercel 上部署。 + --- --- -**📅 最后更新**：2026-03-08 +**📅 最后更新**：2026-03-09 **🚀 状态**：v1.0 稳定版 - 专业量化 UI 已锁定 > [!TIP] diff --git a/docs/API_ZH.md b/docs/API_ZH.md index 8167aed3..32caec53 100644 --- a/docs/API_ZH.md +++ b/docs/API_ZH.md @@ -1,26 +1,32 @@ -# PolyWeather API 接口文档 (v1.0) +# PolyWeather API 接口文档 (v1.1) -本文档详细介绍了 PolyWeather 后端提供的 RESTful API 接口。该后端基于 FastAPI 构建，主要为前端地图和终端面板提供实时天气分析、模型预测及历史对账数据。 +本文档说明当前 PolyWeather 后端实际提供的 HTTP API。后端由 `web/app.py` 提供，前端网页通过 Next.js BFF 代理访问这些接口。 --- ## 1. 基础信息 -- **Base URL**: `http://127.0.0.1:8000` (本地) 或 `https://your-vps-ip:8000` (服务器) -- **数据格式**: JSON -- **速率限制**: 默认无限制，建议前端缓存时间 5 分钟。 +- **本地 Base URL**: `http://127.0.0.1:8000` +- **生产 Base URL**: `http://:8000` 或绑定后的 HTTPS API 域名 +- **响应格式**: JSON +- **缓存策略**: + - `/api/cities`: 5 分钟 + - `/api/city/{name}`: 30 秒 + - `/api/history/{name}`: 15 分钟 + - `/api/city/{name}/summary`: 30 秒 + - `/api/city/{name}/detail`: 30 秒 --- ## 2. 接口列表 -### 2.1 获取城市列表 - -返回所有受监控的城市及其基础坐标和风险等级。 +### 2.1 获取监控城市列表 - **URL**: `/api/cities` - **Method**: `GET` -- **返回示例**: +- **用途**: 返回首页左侧监控城市与世界地图 marker 的基础元数据。 + +**响应示例** ```json { @@ -28,11 +34,11 @@ { "name": "ankara", "display_name": "Ankara", - "lat": 39.93, - "lon": 32.85, + "lat": 40.1281, + "lon": 32.9951, "risk_level": "medium", "risk_emoji": "🟡", - "airport": "Esenboga", + "airport": "Esenboğa", "icao": "LTAC", "temp_unit": "celsius", "is_major": true @@ -41,115 +47,187 @@ } ``` -### 2.2 获取城市详情分析 - -获取指定城市的实时实测数据、多模型预测对比、DEB 算法预测以及 AI 决策建议。 +### 2.2 获取城市实时分析 - **URL**: `/api/city/{name}` - **Method**: `GET` - **参数**: - - `name` (string): 城市名称或别名（不区分大小写，如 `ankara`, `ldn`）。 - - `force_refresh` (bool, optional): 是否强制跳过缓存刷新数据。默认 `false`。 -- **返回示例**: + - `name`: 城市名或别名，如 `ankara`、`new-york` + - `force_refresh` (可选): `true` 时跳过缓存 +- **用途**: 首页右侧详情面板与地图数据的主接口。 -```json -{ - "name": "ankara", - "display_name": "Ankara", - "lat": 39.93, - "lon": 32.85, - "temp_symbol": "°C", - "local_time": "2024-03-08 10:30:00", - "risk": { - "level": "medium", - "warning": "模型分歧较大", - "icao": "LTAC" - }, - "current": { - "temp": 12.5, - "max_so_far": 14.2, - "obs_time": "10:20", - "wx_desc": "Cloudy" - }, - "multi_model": { - "ECMWF": 15.1, - "GFS": 14.8, - "MGM": 15.5 - }, - "deb": { - "prediction": 15.2 - }, - "ai_analysis": "当前模型一致性较好，建议参考 DEB 预测值..." -} -``` +**当前核心字段** + +- `display_name` +- `local_time` +- `local_date` +- `temp_symbol` +- `risk` +- `current` +- `mgm` +- `mgm_nearby` +- `forecast` +- `multi_model` +- `deb` +- `ensemble` +- `probabilities` +- `trend` +- `metar_today_obs` +- `metar_recent_obs` +- `hourly` +- `hourly_next_48h` +- `updated_at` + +**说明** + +- `current.raw_metar` 现在直接透出 Aviation Weather API 返回的原始 METAR 报文。 +- `mgm` 只对 Ankara 这类确实有 Turkish MGM 覆盖的城市有值。 +- `mgm_nearby` 当前是一个复用字段： + - Ankara: Turkish MGM 周边站 + - 多数其他城市: AviationWeather METAR cluster ### 2.3 获取历史对账数据 -获取指定城市的过去几天内的预测值与真实最高温的对比记录，用于评估模型准确率。 - - **URL**: `/api/history/{name}` - **Method**: `GET` -- **参数**: - - `name` (string): 城市名称。 -- **返回示例**: +- **用途**: 历史准确率对账弹窗与机器人 `/deb` 命令的数据基础。 + +**响应示例** ```json { "history": [ { - "date": "2024-03-07", - "actual": 14.0, - "deb": 14.2, - "mu": 14.1, - "mgm": 14.5 + "date": "2026-03-07", + "actual": 7.0, + "deb": 6.5, + "mu": 7.2, + "mgm": 8.0 } ] } ``` +**说明** + +- 网页端历史对账只统计 **近 15 天已结算样本** +- 当天未结算样本可以画在图里，但不计入胜率与 MAE + +### 2.4 获取城市摘要 + +- **URL**: `/api/city/{name}/summary` +- **Method**: `GET` +- **用途**: 轻量摘要接口，适合未来做 hover 预取或低开销列表更新。 + +**字段** + +- `name` +- `display_name` +- `icao` +- `local_time` +- `temp_symbol` +- `current.temp` +- `deb.prediction` +- `risk.level` +- `risk.warning` +- `updated_at` + +### 2.5 获取城市聚合详情 + +- **URL**: `/api/city/{name}/detail` +- **Method**: `GET` +- **用途**: 面向未来的单请求聚合详情接口，便于把 `city + summary + history + future analysis` 整合到一个载荷中。 + +**当前结构** + +- `overview` +- `official` +- `timeseries` +- `models` +- `probabilities` +- `future` + +**说明** + +- 当前首页 legacy 布局还主要使用 `/api/city/{name}` 和 `/api/history/{name}` +- `/api/city/{name}/detail` 已用于后续更完整详情态的聚合设计 + --- ## 3. 核心对象定义 -### 3.1 风险等级 (Risk Level) +### 3.1 风险等级 -- `low` (🟢): 预测一致，波动小。 -- `medium` (🟡): 存在轻微模型分歧或数据延迟。 -- `high` (🔴): 模型严重打架，市场风险极高。 +- `low`: 低风险，模型与实测较一致 +- `medium`: 中风险，存在一定分歧或本地站点偏置 +- `high`: 高风险，模型冲突大或盘面博弈价值高 -### 3.2 预测模型 (Models) +### 3.2 DEB -- `ECMWF`: 欧洲中期天气预报中心（全球最准）。 -- `GFS`: 美国全球预测系统。 -- `MGM`: 土耳其国家气象局（针对特定城市非常精准）。 -- `DEB`: PolyWeather 独家加权融合算法成果。 +`DEB` 是 PolyWeather 的动态融合预测层，不是简单平均值。它会结合： + +- 多模型预测值 +- 近期表现 +- 实况修正 +- 城市级偏置 + +### 3.3 μ + +`μ` 代表当前结算分布中心，是一个**动态期望值**，会随着模型、实况、趋势变化而变化。 +它不应直接与固定 forecast 用同一口径做静态历史对账。 --- -## 4. 数据来源与第三方 API +## 4. 数据源与第三方 API -系统通过多源聚合链路确保数据的真实性与前瞻性，以下是目前集成的核心数据源： +### 4.1 主观测源 -### 4.1 核心实测数据 (Settlement Data) +- **Aviation Weather / METAR** + - 当前全球机场主观测源 + - 同时提供结构化字段与 `rawOb` -- **NOAA Aviation Weather (METAR)**: 全球机场官方航空气象报文，也是 Polymarket 官方结算参考源。 -- **Iowa Mesonet (IEM)**: 提供 1-5 分钟级别的超高频 ASOS/METAR 实时观测，是高频交易对账的核心源。 -- **MGM (土耳其国家气象局)**: 提供土耳其境内（如安卡拉）最精细的本地观测站数据。 +### 4.2 Ankara 专属源 -### 4.2 模型预测数据 (Forecast Data) +- **Turkish MGM** + - Ankara 主官方增强层 + - 包括 `Ankara (Bölge/Center)` 与周边站点 -- **Open-Meteo**: 聚合 ECMWF (欧洲中期)、GFS (美国)、ICON (德国) 等全球顶级大型数值预报模型。 -- **Meteoblue**: 针对地形复杂的城市提供高精度修正预测（需 API Key）。 -- **NWS (美国气象局)**: 美国本土城市的官方权威预报。 +### 4.3 预测源 -### 4.3 智能化分析 - -- **Groq API**: 驱动 AI 态势感知引擎，基于多源数据生成实时中文交易分析报告（需 API Key）。 +- **Open-Meteo** +- **weather.gov**（美国城市） +- **Meteoblue**（部分城市） +- **多模型集合**: ECMWF / GFS / ICON / GEM / JMA --- -## 5. 常见问题 +## 5. 当前口径说明 -- **404 Error**: 检查城市名是否拼写正确，或是否在支持列表中。 -- **500 Error**: 后端与气象数据源（如 METAR）同步失败，通常 1 分钟后会自动恢复。 -- **数据延迟**: METAR 实测数据通常每 30-60 分钟更新一次。 +- 首页地图主 marker 显示 **当前温度** +- 右侧详情面板展示当前实测、DEB、结算概率、多模型、多日预报 +- 未来日期分析模态框： + - 显示温度走势、结算概率分布、多模型预报、未来 6-48 小时趋势、未来 0-2 小时临近判断 + - 已移除独立“冷锋 / 暖锋判断”模块 + +--- + +## 6. 常见问题 + +- **接口 500** + - 先检查 `polyweather_web` 是否启动成功 + - 再看 `docker-compose logs -f polyweather_web` + +- **METAR 看起来慢几分钟** + - 通常是官方链路入库/发布延迟，不一定是本地轮询慢 + - 请同时看： + - `current.obs_time` + - `current.report_time` + - `current.receipt_time` + +- **网页显示旧内容** + - 先确认 Vercel 已部署新版本 + - 再强刷浏览器缓存 + +--- + +**最后更新**: 2026-03-09 diff --git a/docs/COMMERCIALIZATION.md b/docs/COMMERCIALIZATION.md index 30ced037..9303c3d5 100644 --- a/docs/COMMERCIALIZATION.md +++ b/docs/COMMERCIALIZATION.md @@ -50,6 +50,7 @@ graph LR - Manual subscription activation via Telegram DM. - Small, focused paid Telegram channel for signal tests. - Invitation-only Web Access (Vercel). + - Validate city-specific edge before scaling, with Ankara as the flagship strategy city. ### 🛠️ Phase 2: Automation (USDC) @@ -66,6 +67,20 @@ graph LR - **Accuracy Leaderboard**: Monthly reports of DEB vs Market outcomes. - **Self-Serve Portal**: User dashboard for billing and alert settings. +### 📡 API Expansion Priority + +- **P0-1 Market Layer** + - Polymarket Gamma discovery + `py-clob-client` pricing / order book +- **P0-2 Official Observation Layer** + - Aviation Weather / METAR + - weather.gov official forecast / observation / alert context +- **P1 Lead Layer** + - Ankara keeps Turkish MGM nearby network + - U.S. cities may later receive Mesonet enhancement without replacing METAR +- **P2 Product Layer** + - Stripe / Polygon-USDC automation + - Realtime sync and user entitlement system + --- ## 🚧 Critical Constraints @@ -76,4 +91,4 @@ graph LR --- -**📅 Last Updated**: 2026-03-06 +**📅 Last Updated**: 2026-03-09 diff --git a/docs/TECH_DEBT.md b/docs/TECH_DEBT.md index e9ebd2f1..f8bf4b83 100644 --- a/docs/TECH_DEBT.md +++ b/docs/TECH_DEBT.md @@ -22,6 +22,7 @@ The core engine is stable, but several infrastructure "shortcut" decisions remai - [x] DEB Blending Algorithm - [x] Proactive Telegram Alert Engine - [x] Vercel Dashboard Infrastructure +- [x] Legacy dashboard running behind Next.js/Vercel --- @@ -32,6 +33,7 @@ The core engine is stable, but several infrastructure "shortcut" decisions remai | **Monolithic Bot** | `bot_listener.py` is hard to test and evolve. | Isolate UI interaction from business logic into `src/analysis`. | | **Subscription Store** | No persistent record of who has paid. | Migrate from in-memory user checks to **Supabase/PostgreSQL**. | | **Alert Transparency** | Operators cannot easily audit "why" an alert fired. | Add an `Evidence` metadata block to all internal alert payloads. | +| **Encoding Drift** | Legacy frontend files have suffered mixed encodings.| Normalize all legacy static files to UTF-8 and stop editing them with incompatible encodings. | --- @@ -42,6 +44,7 @@ The core engine is stable, but several infrastructure "shortcut" decisions remai | **Hard-coded Thresholds** | Modification requires code changes (e.g., 5s CD). | Extract all business constants into a structured `config.yaml`. | | **Simulation Harness** | No way to "replay" a rainy day to test alert logic. | Build a `ReplayEngine` using `data/daily_records.json`. | | **Backend Naming** | Artifacts of "market price" logic remain in naming. | Systematic refactor of variable names to reflect weather-intelligence focus. | +| **Legacy Frontend Debt** | Large `public/static/app.js` mixes data, UI, charts, and modal logic. | Gradually extract panel, chart, and modal logic into typed modules without changing the current layout contract. | --- @@ -59,7 +62,8 @@ The core engine is stable, but several infrastructure "shortcut" decisions remai 1. **DB Integration**: Connect Supabase to `src/database/db_manager.py`. 2. **Alert Transparency**: Append logic metrics (slope, lead delta) to push messages. 3. **Authentication**: Secure `/api/city` on Vercel with subscription keys. +4. **UTF-8 Cleanup**: Remove remaining mojibake from legacy static files and comments. --- -**📅 Last Updated**: 2026-03-06 +**📅 Last Updated**: 2026-03-09 diff --git a/frontend/README.md b/frontend/README.md index 065f8d68..3c6e9028 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -18,6 +18,7 @@ Production URL: - Vercel serves the web UI - FastAPI on VPS serves API only - The old FastAPI static website has been removed +- The production page shell is still the legacy dashboard embedded by `app/page.tsx` Current request flow: - Browser -> Vercel frontend @@ -52,6 +53,11 @@ Thin BFF routes currently exposed by Next: - `GET /api/city/[name]` - `GET /api/history/[name]` +Current frontend behavior: +- `/` keeps the world overview layout +- City clicks stay inside the same layout and load the right-side panel +- Future forecast dates open a modal instead of mutating the base panel + ## Vercel Deployment 1. Import the repo into Vercel @@ -65,4 +71,4 @@ Thin BFF routes currently exposed by Next: - The page shell currently embeds the legacy dashboard HTML from `public/legacy/index.html` - If you change files under `public/static`, deploy to Vercel to make them live -Last updated: 2026-03-06 +Last updated: 2026-03-09 diff --git a/frontend/app/page.tsx b/frontend/app/page.tsx index 486a0fe3..368a68db 100644 --- a/frontend/app/page.tsx +++ b/frontend/app/page.tsx @@ -1,11 +1,12 @@ -export default function HomePage() { +export default function HomePage() { return (