docs: full README rewrite reflecting current architecture and capabilities

This commit is contained in:
2569718930@qq.com
2026-03-04 02:31:26 +08:00
parent 946430e05a
commit 63ea4d0a80
2 changed files with 88 additions and 78 deletions
+41 -36
View File
@@ -1,6 +1,5 @@
# 🌡️ PolyWeather: Intelligent Weather Quant Analysis Bot
[![Python application CI](https://github.com/yangyuan-zhen/PolyWeather/actions/workflows/python-app.yml/badge.svg)](https://github.com/yangyuan-zhen/PolyWeather/actions/workflows/python-app.yml)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/yangyuan-zhen/PolyWeather)
@@ -25,10 +24,11 @@ PolyWeather is a weather analysis tool built for prediction markets like **Polym
### 1. 🌐 Interactive Web Map Dashboard
- **Global Overview**: Real-time Leaflet-based map pinpointed exactly to the official Polymarket settlement airport coordinates.
- **Progressive Background Loading**: Intelligently fetches multi-source data across all cities seamlessly without hitting rate limits.
- **Rich Visualization**: Includes Chart.js-powered temperature trends, multi-model comparison bars, Gaussian probability distribution, and dynamic risk badges.
- **Dual-Engine Co-existence**: Runs concurrently with the Telegram bot using a FastAPI backend, sharing the exact same data and caching layers.
- **Global Overview**: Real-time Leaflet-based dark-themed map pinpointed to official Polymarket settlement airport coordinates.
- **Progressive Background Loading**: Intelligently fetches multi-source data across all cities without hitting API rate limits.
- **Rich Visualization**: Chart.js-powered temperature trends with METAR scatter overlay, multi-model comparison bars, Gaussian probability distribution, and dynamic risk badges.
- **Cinematic Interaction**: City selection triggers a smooth fly-to zoom animation on the map.
- **Dual-Engine Architecture**: Runs concurrently with the Telegram bot via a FastAPI backend, sharing the same data collection, analysis logic (`analyze_weather_trend`), and AI prompt pipeline.
### 2. 🧬 Dynamic Ensemble Blending (DEB Algorithm)
@@ -40,17 +40,18 @@ The system automatically tracks the historical performance of weather models (EC
- **Accuracy Tracking**: Use the `/deb` command to view DEB's historical WU settlement hit rate and MAE, compared against individual models.
- **Auto-Cleanup**: Only retains the last 14 days of records to prevent unbounded data growth.
### 2. 🎲 Math Probability Engine (Settlement Probability)
### 3. 🎲 Math Probability Engine (Settlement Probability)
Automatically computes the probability for each possible WU settlement integer using a Gaussian distribution:
Automatically computes the probability for each possible settlement integer using a Gaussian distribution:
- **Distribution Center μ**: Weighted average of DEB/multi-model median (70%) and ensemble median (30%). Auto-corrects upward when METAR max exceeds μ.
- **Reality-Anchored μ**: When actual max temperature is significantly below forecasts during/after the peak window (forecast bust), μ anchors on the observed max instead of failed predictions. Otherwise, uses a weighted average of DEB/multi-model median (70%) and ensemble median (30%).
- **Standard Deviation σ — Three-Layer Pipeline**:
1. **Ensemble Base**: σ = (P90-P10) / 2.56
2. **MAE Floor**: Uses DEBs historical MAE as σ minimum—prevents ensembles from underestimating true uncertainty
2. **MAE Floor**: Uses DEB's historical MAE as σ minimum—prevents ensembles from underestimating true uncertainty
3. **Shock Score Amplifier**: σ × (1 + 0.5 × shock_score) when weather is changing rapidly
- **Time Decay**: Before peak σ×1.0 → During peak σ×0.7 → After peak σ×0.3
- **Observed Floor**: Temperatures below the current METAR max WU value are excluded
- **Dead Market Override**: When a dead market is confirmed, probability collapses to 100% at the settled value
#### 💥 Shock Score: Weather Disruption Soft Scorer (0~1)
@@ -62,25 +63,27 @@ Evaluates environmental stability from the last 4 METAR observations. Higher = m
| Cloud Cover Jump | 0~0.35 | Cloud code escalation (FEW→BKN, etc.) |
| Pressure Change | 0~0.25 | >2hPa change within 2 hours |
### 3. 🤖 AI Deep Analysis (Groq LLaMA 3.3 70B)
### 4. 🤖 AI Deep Analysis (Groq LLaMA 3.3 70B)
Feeds all weather data into LLaMA 70B, analyzed via a **P1→P4 Priority Chain**:
Feeds all weather data into LLaMA 70B, analyzed via a **P0→P4 Priority Chain**:
- **P1 Real-Time Rhythm** (highest priority): 2 consecutive METAR highs → still warming; 2 non-highs past peak → dead market. Warming under low radiation → advection-driven, forecasts often underestimate.
- **P2 Inhibitors**: Humidity >80% **and** BKN/OVC sustained 2 reports → effective suppression. "Partly cloudy" alone is insufficient.
- **P3 Math Probability**: References settlement probability but cannot override P1 observations.
- **P4 Forecast Background**: DEB/forecasts used for ceiling estimation; downweighted when actuals exceed them.
- **Dead Market Trigger**: Past peak window + 2 consecutive non-highs + cloud buildup or precipitation → dead market declared.
- **High Availability**: Auto-retry + fallback model degradation (70B → 8B) to withstand Groq API outages.
- **P0 Forecast Bust Detection** (highest priority): Graded severity (light/medium/heavy) when actual temps diverge from forecasts. Requires slope + wind/cloud verification before declaring settlement locked. "Bust ≠ locked" — still checks for second-wave warming.
- **P1 Real-Time Rhythm**: 2 consecutive METAR highs → still warming; 2 non-highs with slope ≤ 0 → dead market. Low-radiation warming → multi-factor (advection/mixing layer/heat island), no single-factor attribution.
- **P2 Inhibitors** (city-aware): Precipitation → strong suppression. High humidity + thick clouds sustained 2+ reports → possible suppression, but thresholds vary by city type (maritime vs. continental). Single factor insufficient.
- **P3 Probability Cross-Check**: References settlement probability for consistency check with P1. Contradictions explained with deviation rationale.
- **P4 Forecast Background**: DEB/forecasts for ceiling estimation; silenced when actuals significantly deviate.
- **Single Source of Truth**: Both web and Telegram bot share the same `analyze_weather_trend` function and `get_ai_analysis` prompt — identical context, identical decisions.
- **High Availability**: Auto-retry + fallback model degradation (70B → 8B). Proxy support for restricted networks.
### 4. ⏱️ Real-time Airport Observations (Zero-Cache METAR)
### 5. ⏱️ Real-time Airport Observations (Zero-Cache METAR)
- **Precise Timing**: Extracts actual observation time from raw METAR text (`rawOb`), not the API's rounded `reportTime`. Accurate to the minute.
- **Live Passthrough**: Bypasses CDN caching via dynamic headers to obtain first-hand METAR reports.
- **Settlement Warning**: Automatically calculates the Wunderground settlement boundary (X.5 rounding line).
- **Anomaly Filtering**: Automatically filters out -9999 sentinel values from sources like MGM to prevent garbage data in output.
- **Settlement Warning**: Automatically calculates the settlement boundary (X.5 rounding line).
- **MGM Fallback**: For Turkish cities (Ankara), falls back to MGM data when METAR is unavailable.
- **Anomaly Filtering**: Automatically filters out -9999 sentinel values to prevent garbage data in output.
### 5. 📈 Historical Data Collection
### 6. 📈 Historical Data Collection
- Includes `fetch_history.py` to retrieve up to 3 years of hourly historical weather data (temperature, humidity, radiation, pressure, 10+ dimensions), providing data foundation for future ML models (XGBoost/MOS).
@@ -148,48 +151,50 @@ _(Note: The `update.sh` script automatically fetches the latest code, kills old
```mermaid
graph TD
User[User] -->|Query Command| Bot[bot_listener.py Core Scheduler]
User[User] -->|Query| Bot["bot_listener.py (Core Scheduler)"]
User -->|Browser| Web["web/app.py (FastAPI)"]
subgraph Data Acquisition
Bot --> Collector[WeatherDataCollector]
Web --> Collector
Collector --> OM[Open-Meteo Forecast/Ensemble]
Collector --> MM[Multi-Model ECMWF/GFS/ICON/GEM/JMA]
Collector --> METAR["Live Airport METAR (rawOb precise time)"]
Collector --> METAR["Live Airport METAR (rawOb)"]
Collector --> MGM["MGM Fallback (Turkey)"]
end
subgraph Algorithm Layer
Collector --> Peak[Peak Hour Prediction]
Collector --> DEB[DEB Dynamic Weighting]
DEB --> DB[(daily_records Database)]
Peak --> Prob[Gaussian Probability Engine]
Peak --> Prob["Probability Engine (Reality-Anchored μ)"]
Collector --> Prob
METAR --> Shock[Shock Score]
Shock --> Prob
Collector --> Logic[Settlement Boundary / Trend Analysis]
Collector --> Logic["Settlement Boundary / Dead Market"]
end
subgraph AI Decision Layer
DEB --> AI[Groq LLaMA 70B]
Prob --> AI
Logic --> AI
METAR --> AI
subgraph Shared Analysis
Bot --> ATF["analyze_weather_trend()"]
Web --> ATF
ATF --> AI["Groq LLaMA 70B (P0→P4)"]
end
AI -->|Market Call + Logic + Confidence| Bot
Bot -->|DEB Blend + Probability + AI Analysis| User
AI -->|Market Call + Logic + Confidence| Web
```
---
## 💡 Trading Tips
1. **Real-time Rhythm First**: AI analysis follows P1→P4 priority. If live METAR trends (P1) conflict with math probabilities (P3)—e.g., probability favors 7°C but its still surging toward 8°C—always prioritize the live trend.
1. **Real-time Rhythm First**: AI analysis follows P0→P4 priority. If live METAR trends (P1) conflict with math probabilities (P3), always prioritize the live trend.
2. **Watch Settlement Probabilities**: Based on Gaussian models, direction is most certain when a temperature has > 70% probability while P1 rhythm is flat.
3. **Reference DEB Bias**: Use `/deb` to check for systematic bias. If a city is consistently "underestimated," habitually bid one WU notch higher.
4. **Identify Dead Market Signals**: When AI declares a "Dead Market," it usually means warming power is exhausted (post-peak window + no new highs + cloud buildup). This is an opportunity to harvest remaining value.
5. **Mind the Boundaries**: When the observed high is near X.5 (e.g., 7.50°C), be wary of Wunderground rounding up to 8 due to tiny fluctuations.
6. **Center Point μ**: The μ value represents the expected actual high. When market prices deviate significantly from μ, an arbitrage opportunity may exist.
4. **Identify Dead Market Signals**: When the system declares a "Dead Market," probability collapses to 100% at the settled value. Warming power is exhausted.
5. **Mind the Boundaries**: When the observed high is near X.5 (e.g., 7.50°C), be wary of rounding up due to tiny fluctuations.
6. **Forecast Bust Awareness**: When the AI reports a forecast bust (especially medium/heavy grade), all model predictions have lost reference value. Focus exclusively on METAR actuals.
---
_Updated 2026-03-03_
_Updated 2026-03-04_
+47 -42
View File
@@ -1,6 +1,5 @@
# 🌡️ PolyWeather: 智能天气量化分析机器人
[![Python application CI](https://github.com/yangyuan-zhen/PolyWeather/actions/workflows/python-app.yml/badge.svg)](https://github.com/yangyuan-zhen/PolyWeather/actions/workflows/python-app.yml)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/yangyuan-zhen/PolyWeather)
@@ -16,7 +15,7 @@ PolyWeather 是一款专为 **Polymarket** 等预测市场打造的天气分析
<p align="center">
<img src="./docs/images/demo_map.png" alt="PolyWeather Web Map" width="800">
<br>
<em>🗺️ 交互式网页地图:全球城市实时监控与丰富的数据可视化可视化</em>
<em>🗺️ 交互式网页地图:全球城市实时监控与丰富的数据可视化</em>
</p>
---
@@ -25,10 +24,11 @@ PolyWeather 是一款专为 **Polymarket** 等预测市场打造的天气分析
### 1. 🌐 交互式网页地图面板
- **全球纵览**:基于 Leaflet 的暗黑全屏实时地图,直接采用官方结算机场经纬度进行最精确的追踪与显示。
- **渐进式数据流加载**:进图后智能在后台进行无感知拉取更新,确保不触发高频 API 频率限制的前提下填满所有城市数据
- **丰富的数据可视化**引入了基于 Chart.js 的高分辨率全天走势预测平滑曲线,多模型直达对比横空出世,以及高颜值结算概率条,各城市实时风险等级色彩系统。
- **双引擎共生**:后端采用了基于原代码重构利用的 FastAPI,地图网络服务可以在任意 VPS 上零门槛与原本的 Telegram Bot(机器人)共享同一个内存核心并存互不打架
- **全球纵览**:基于 Leaflet 的暗黑全屏实时地图,直接采用官方结算机场经纬度进行追踪与显示。
- **渐进式数据流加载**:进图后智能在后台无感知拉取不触发 API 频率限制。
- **丰富的数据可视化**Chart.js 温度走势图叠加 METAR 实测散点,多模型对比条形图,高斯概率分布条,实时风险等级色彩系统。
- **镜头联动**:点击城市触发平滑飞入缩放动画
- **双引擎共生**FastAPI 后端与 Telegram Bot 共享同一份分析逻辑(`analyze_weather_trend`)和 AI Prompt 管线,确保两端输出完全一致。
### 2. 🧬 动态权重集合预报 (DEB 算法)
@@ -40,17 +40,18 @@ PolyWeather 是一款专为 **Polymarket** 等预测市场打造的天气分析
- **准确率追踪**:通过 `/deb` 命令查看 DEB 融合预测的历史 WU 结算命中率和 MAE,并与各个单一模型对比。
- **自动清理**:只保留最近 14 天的记录,防止数据无限增长。
### 2. 🎲 数学概率引擎 (Settlement Probability)
### 3. 🎲 数学概率引擎 (Settlement Probability)
基于集合预报的正态分布拟合,自动计算每个 WU 结算整数温度的概率:
基于集合预报的正态分布拟合,自动计算每个结算整数温度的概率:
- **分布中心 μ** DEB/多模型中位数(70%)和集合中位数(30%)的加权均值为中心。实测超过 μ 时自动上修
- **实况锚定 μ**当实测最高温在峰值窗口期间/之后显著低于预报中位数(预报崩盘),μ 直接锚定在实测值上,而非失败的预报。正常情况下使用 DEB/多模型中位数(70%)和集合中位数(30%)的加权均。
- **标准差 σ 三层修正管线**
1. **集合基础**:σ = (P90-P10) / 2.56
2. **MAE **:用 DEB 历史 MAE 作为 σ 下限——防止集合预报低估真实不确定性
2. **MAE **:用 DEB 历史 MAE 作为 σ 下限——防止集合预报低估真实不确定性
3. **Shock Score 放宽**:σ × (1 + 0.5 × shock_score),气象突变时自动加宽
- **时间衰减**:峰值前 σ×1.0 → 峰值窗口 σ×0.7 → 峰值后 σ×0.3
- **实测过滤**:已实测 WU 值以下的候选自动排除
- **死盘覆盖**:确认死盘后,概率直接坍缩为结算值 100%
#### 💥 Shock Score:气象突变软评分 (0~1)
@@ -62,25 +63,27 @@ PolyWeather 是一款专为 **Polymarket** 等预测市场打造的天气分析
| 云量阶跃 | 0~0.35 | FEW→BKN 等云码跳变 |
| 气压变化 | 0~0.25 | 2h 内气压差 > 2hPa |
### 3. 🤖 AI 深度分析 (Groq LLaMA 3.3 70B)
### 4. 🤖 AI 深度分析 (Groq LLaMA 3.3 70B)
将全部气象数据投喂给 LLaMA 70B,按 **P1→P4 优先级链** 分析
将全部气象数据投喂给 LLaMA 70B,按 **P0→P4 分析框架** 决策
- **P1 实况节奏**(最高优先级):连续 2 报创新高→升温未止;连续 2 报未创新高且过峰→偏死盘。低辐射升温→暖平流驱动
- **P2 阻碍因子**:湿度>80% **且** BKN/OVC 持续 2 报→压温有效。单"多云"不足以判断受限
- **P3 数学概率**:参考结算概率分布,但不可压过 P1 实况
- **P4 预报背景**:DEB/预报用于判断上沿,实测超预报时降权
- **死盘判定**:峰值窗口已过 + 连续 2 报未创新高 + 云量回补或降水→判定死盘
- **高可用保障**:自动重试 + 备用模型降级(70B → 8B),抵御 Groq API 故障
- **P0 预报失准检测**(最高优先级):分级失准(轻/中/重),根据偏差幅度自动标记。"失准 ≠ 已定局"——还需检查斜率 + 风云条件。支持二次抬升判断
- **P1 实况节奏**:连续 2 报创新高 → 升温未止;连续 2 报未创新高且斜率 ≤ 0 → 偏死盘。低辐射升温 → 可能多因子叠加(平流/混合层/热岛),不做单因子归因
- **P2 阻碍因子**(需结合城市特性判断):降水 → 强压温。高湿度 + 厚云层持续 2 报以上 → 可能压温,但阈值因城市类型(海洋 vs 大陆)而异。单因子不足以断定
- **P3 概率与一致性校验**:参考结算概率分布,与 P1 实况做交叉检查。矛盾时以实况为准并说明偏离原因
- **P4 预报背景**(最低优先级):可参考 DEB/预报做上沿评估。实测显著偏离时禁止引用
- **统一分析源**Web 和 Telegram Bot 共用同一个 `analyze_weather_trend` 函数和 `get_ai_analysis` 提示词——完全相同的上下文,完全相同的决策
- **高可用保障**:自动重试 + 备用模型降级(70B → 8B)。支持代理配置。
### 4. ⏱️ 实时机场观测 (Zero-Cache METAR)
### 5. ⏱️ 实时机场观测 (Zero-Cache METAR)
- **精确时间**:从 METAR 原始报文 (`rawOb`) 中提取真实观测时间,而非 API 取整后的 `reportTime`精确到分钟。
- **精确时间**:从 METAR 原始报文 (`rawOb`) 中提取真实观测时间,精确到分钟。
- **实时穿透**:通过动态请求头绕过 CDN 缓存,获取机场第一手 METAR 报文。
- **结算预警**:自动计算 Wunderground 结算边界(X.5 进位线),提醒潜在波动。
- **异常过滤**:自动过滤 MGM 数据源的 -9999 哨兵值,避免垃圾数据污染输出
- **结算预警**:自动计算结算边界(X.5 进位线),提醒潜在波动。
- **MGM 回退**:土耳其城市(安卡拉)METAR 不可用时回退至 MGM 数据。
- **异常过滤**:自动过滤 -9999 等哨兵值,避免垃圾数据污染输出。
### 5. 📈 历史数据采集
### 6. 📈 历史数据采集
- 提供 `fetch_history.py` 脚本,可一键获取各城市过去 3 年的小时级历史气象数据(温度、湿度、辐射、气压等 10+ 维度),为后续机器学习模型(XGBoost/MOS)提供数据基础。
@@ -93,7 +96,7 @@ PolyWeather 是一款专为 **Polymarket** 等预测市场打造的天气分析
- **Python 3.11+** 或 **Docker & Docker Compose**
- **环境变量**: 在 `.env` 中设置关键参数(参考 `.env.example`)。
### 🐳 Docker 推荐部署 (推荐)
### 🐳 Docker 部署 (推荐)
最简单、稳定的部署方式,避免系统依赖冲突。
@@ -118,14 +121,14 @@ PolyWeather 是一款专为 **Polymarket** 等预测市场打造的天气分析
1. 安装依赖: `pip install -r requirements.txt`
2. 配置 `.env` 文件。
3. 利用项目中已包含的 `update.sh` 实现机器人和网站的双轨后台一键重启(适用于 `nohup`
3. 利用项目中已包含的 `update.sh` 实现机器人和网站的双轨后台一键重启:
```bash
# 每次代码变更后,只需在 VPS 执行此命令
./update.sh
```
_(该脚本将自动执行 git 抓取、杀僵尸进程、解绑释放死锁端口、并分别利用 nohup 重新唤醒 bot_listener.py 和 FastAPI app.py服务。)_
_(该脚本将自动执行 git 抓取、杀僵尸进程、解绑端口、并分别利用 nohup 重新唤醒 bot_listener.py 和 FastAPI app.py 服务。)_
---
@@ -148,48 +151,50 @@ _(该脚本将自动执行 git 抓取、杀僵尸进程、解绑释放死锁端
```mermaid
graph TD
User[用户] -->|查询指令| Bot[bot_listener.py 核心调度器]
User[用户] -->|查询指令| Bot["bot_listener.py (核心调度器)"]
User -->|浏览器| Web["web/app.py (FastAPI)"]
subgraph 数据获取层
Bot --> Collector[WeatherDataCollector]
Web --> Collector
Collector --> OM[Open-Meteo 预报/集合]
Collector --> MM[多模型 ECMWF/GFS/ICON/GEM/JMA]
Collector --> METAR["机场实时 METAR (rawOb精确时间)"]
Collector --> METAR["机场实时 METAR (rawOb)"]
Collector --> MGM["MGM 回退 (土耳其)"]
end
subgraph 算法层
Collector --> Peak[峰值时段预测]
Collector --> DEB[DEB 动态权重融合]
DEB --> DB[(每日记录数据库)]
Peak --> Prob[正态分布概率引擎]
Peak --> Prob["概率引擎 (实况锚定μ)"]
Collector --> Prob
METAR --> Shock[Shock Score 突变评分]
Shock --> Prob
Collector --> Logic[结算边界 / 趋势分析]
Collector --> Logic["结算边界 / 死盘检测"]
end
subgraph AI 决策
DEB --> AI[Groq LLaMA 70B]
Prob --> AI
Logic --> AI
METAR --> AI
subgraph 共享分析
Bot --> ATF["analyze_weather_trend()"]
Web --> ATF
ATF --> AI["Groq LLaMA 70B (P0→P4)"]
end
AI -->|盘口 + 逻辑 + 置信度| Bot
Bot -->|DEB融合 + 概率 + AI分析| User
AI -->|盘口 + 逻辑 + 置信度| Web
```
---
## 💡 交易提示
1. **实况节奏优先**:AI 分析遵循 P1→P4 优先级。如果实况趋势(P1)与数学概率(P3)冲突(例如概率看好 7°C 但实况仍猛涨冲向 8°C),请务必以实况走势为准。
1. **实况节奏优先**:AI 分析遵循 P0→P4 优先级。如果实况趋势(P1)与数学概率(P3)冲突以实况走势为准。
2. **紧盯结算概率**:概率引擎基于数学模型,当某个温度概率 > 70% 且 P1 节奏持平时,方向最为明确。
3. **参考 DEB 偏差**:通过 `/deb` 查看城市的系统性偏差。如果某个城市经常低估,交易时应习惯性看高 1 个 WU 档位。
4. **识别死盘信号**当 AI 判定死盘时,通常意味着升温动力彻底枯竭(峰值窗后+不创新高+云量回补),此时是反向对收割残余价值的机会
5. **注意结算边界**:实测最高温接近 X.5(如 7.50°C)时,Wunderground 可能会因极微小波动从 7 进位到 8,需防范偷鸡
6. **分布中心 μ**:μ 值代表算法预期的实际最高温中心点,直接对标盘口价格。当价格严重偏离 μ,通常存在套利空间
3. **参考 DEB 偏差**:通过 `/deb` 查看城市的系统性偏差。如果某个城市经常"低估",交易时应看高 1 个 WU 档位。
4. **识别死盘信号**系统判定"死盘"时,概率会直接坍缩为结算值 100%。升温动力彻底枯竭
5. **注意结算边界**:实测最高温接近 X.5 时,微小波动可能导致进位,需防范"偷鸡"
6. **预报崩盘意识**:当 AI 标记预报失准(尤其中/重级),所有模型预测已失去参考价值,需专注 METAR 实测
---
_更新于 2026-03-03_
_更新于 2026-03-04_