Document model stack and DEB deduplication

2026-04-17 00:40:46 +08:00
parent 9ec86e0504
commit 19169e4bdb
3 changed files with 302 additions and 4 deletions
@@ -0,0 +1,214 @@
+# 模型栈与 DEB 去重规则
+
+本文档记录 PolyWeather 当前开放模型接入、区域覆盖差异，以及 DEB 在新增模型后的计权规则。
+
+## 1. 接入方式
+
+当前多模型层通过 Open-Meteo model API 接入开放 NWP / AI 预报，不直接下载原始 GRIB。
+
+入口：
+
+- `src/data_collection/nws_open_meteo_sources.py`
+- `WeatherDataCollector.fetch_multi_model(...)`
+
+返回结构继续保持向后兼容：
+
+- `multi_model.forecasts`
+- `multi_model.daily_forecasts`
+- `multi_model.dates`
+
+新增元数据：
+
+- `multi_model.provider`
+- `multi_model.model_metadata`
+- `multi_model.model_keys`
+- `multi_model.attribution`
+
+Web API 会把这部分元数据挂到：
+
+- `source_forecasts.open_meteo_multi_model`
+
+## 2. 当前模型清单
+
+| 显示名 | Open-Meteo key | 来源 | 层级 | 说明 |
+| --- | --- | --- | --- | --- |
+| ECMWF | `ecmwf_ifs025` | ECMWF | global | IFS 全球传统数值模式 |
+| ECMWF AIFS | `ecmwf_aifs025_single` | ECMWF | global_ai | ECMWF AI forecast |
+| GFS | `gfs_seamless` | NOAA | global | NOAA 全球参考 |
+| ICON | `icon_seamless` | DWD | global | DWD ICON 全球基准 |
+| ICON-EU | `icon_eu` | DWD | regional_europe | 欧洲区域高分辨率 |
+| ICON-D2 | `icon_d2` | DWD | short_range_europe | 欧洲短时高分辨率 |
+| GEM | `gem_seamless` | ECCC | global | 加拿大 GEM seamless |
+| GDPS | `gem_global` | ECCC | global | 加拿大全球模式 |
+| RDPS | `gem_regional` | ECCC | regional_north_america | 北美区域模式 |
+| HRDPS | `gem_hrdps_continental` | ECCC | short_range_north_america | 北美短时高分辨率 |
+| JMA | `jma_seamless` | JMA | global | 日本气象厅全球参考 |
+
+## 3. 区域覆盖差异
+
+同一个多模型请求会带上完整模型清单，但 Open-Meteo 只会返回覆盖当前坐标的模型字段。区域模型不覆盖时不会进入下游。
+
+### 欧洲城市
+
+常见模型：
+
+- ECMWF
+- ECMWF AIFS
+- GFS
+- ICON
+- ICON-EU
+- ICON-D2
+- GEM / GDPS
+- JMA
+
+欧洲高分辨率重点来自 DWD ICON-EU / ICON-D2。
+
+### 北美城市
+
+常见模型：
+
+- ECMWF
+- ECMWF AIFS
+- GFS
+- ICON
+- GEM / GDPS
+- RDPS
+- HRDPS
+- JMA
+- NWS
+
+北美高分辨率重点来自 ECCC RDPS / HRDPS，NWS 继续作为美国城市官方预报参考。
+
+### 亚洲城市
+
+常见模型：
+
+- ECMWF
+- ECMWF AIFS
+- GFS
+- ICON
+- GEM / GDPS
+- JMA
+
+通常不会出现：
+
+- ICON-EU
+- ICON-D2
+- RDPS
+- HRDPS
+
+亚洲城市更依赖本地观测增强层，例如 JMA、KMA、NMC、HKO、CWA、METAR、TAF。
+
+## 4. DEB 家族去重
+
+DEB 不直接把所有模型按“每个模型一票”计入。新增区域模型后，如果不去重，会造成同一模型机构重复放大。
+
+处理入口：
+
+- `src/analysis/deb_algorithm.py`
+- `_collapse_forecasts_for_deb(...)`
+- `calculate_dynamic_weights(...)`
+
+### DWD ICON 家族
+
+归并成员：
+
+- ICON
+- ICON-EU
+- ICON-D2
+
+优先级：
+
+```text
+ICON-D2 > ICON-EU > ICON
+```
+
+### ECCC GEM 家族
+
+归并成员：
+
+- GEM
+- GDPS
+- RDPS
+- HRDPS
+
+优先级：
+
+```text
+HRDPS > RDPS > GDPS > GEM
+```
+
+### 独立保留
+
+以下模型路径不合并：
+
+- ECMWF IFS
+- ECMWF AIFS
+- GFS
+- JMA
+- MGM
+- NWS
+- HKO
+- LGBM
+- Open-Meteo
+
+ECMWF IFS 与 ECMWF AIFS 分开保留，因为前者是传统 NWP，后者是 AI forecast。
+
+## 5. DEB 权重流程
+
+当前流程：
+
+```text
+raw current_forecasts
+  -> 过滤不可用值与排除模型
+  -> 按模型家族去重
+  -> 历史 MAE 统计
+  -> MAE 倒数权重
+  -> 输出 blended_high + weights_info
+```
+
+当 `weights_info` 出现 `家族去重`，表示当前输入模型数量多于 DEB 实际入模数量，系统已先折叠同家族模型。
+
+## 6. 前端展示
+
+网页的模型展示读取：
+
+- `multi_model`
+- `multi_model_daily`
+- `source_forecasts.open_meteo_multi_model.model_metadata`
+
+显示分组：
+
+- 全球基准
+- AI 预报
+- 欧洲高分辨率
+- 北美高分辨率
+
+展示字段：
+
+- 可用模型数量
+- 模型分歧 spread
+- 来源
+- provider
+- model
+- resolution
+- horizon
+
+区域模型不覆盖时不显示空模型。
+
+## 7. 测试覆盖
+
+相关测试：
+
+- `tests/test_multi_model_sources.py`
+- `tests/test_deb_model_family.py`
+- `tests/test_lgbm_features.py`
+
+重点覆盖：
+
+- Open-Meteo 多模型解析
+- 新模型元数据输出
+- 区域模型缺失时降级
+- DEB 家族去重
+- 历史不足时的去重等权
+- 有历史 MAE 时的去重动态权重
@@ -823,10 +823,11 @@ export function ModelForecast({
                    <div
                      className="model-bar-fill"
                      style={{ width: `${width}%` }}
-                    >
+                    />
+                    <span className="model-bar-value">
                      {value}
                      {detail.temp_symbol}
-                    </div>
+                    </span>
                    {debLine != null && (
                      <div
                        className="model-deb-line"
@@ -860,10 +861,11 @@ export function ModelForecast({
                style={{
                  width: `${((Number(view.deb) - minValue) / range) * 100}%`,
                }}
-              >
+              />
+              <span className="model-bar-value deb">
                {Number(view.deb)}
                {detail.temp_symbol}
-              </div>
+              </span>
            </div>
          </div>
        )}
@@ -161,6 +161,88 @@ export const DOCS_PAGES: DocsPage[] = [
      },
    },
  },
+  {
+    slug: "model-stack-deb",
+    group: "analysis",
+    content: {
+      "zh-CN": {
+        title: "模型栈与 DEB",
+        description: "这页说明 PolyWeather 当前接入哪些开放模型，不同地区为什么看到的模型不一样，以及 DEB 如何避免重复计权。",
+        sections: [
+          {
+            id: "model-sources",
+            title: "当前接入的开放模型",
+            blocks: [
+              { type: "paragraph", text: "PolyWeather 的多模型层通过 Open-Meteo 模型接口接入开放 NWP / AI 预报。它不是直接下载原始 GRIB，而是把可用模型统一归一到最高温模型栈里，供模型分歧、概率层和 DEB 使用。" },
+              { type: "bullets", items: ["ECMWF IFS：全球传统数值模式。", "ECMWF AIFS：ECMWF AI forecast，作为独立 AI 路径保留。", "DWD ICON：全球 ICON 基准层。", "DWD ICON-EU：欧洲区域高分辨率层。", "DWD ICON-D2：欧洲短时高分辨率层。", "ECCC GEM / GDPS：加拿大系全球模式。", "ECCC RDPS / HRDPS：北美区域与短时高分辨率层。", "GFS / JMA：继续作为全球参考模型保留。"] },
+            ],
+          },
+          {
+            id: "regional-coverage",
+            title: "为什么欧洲、亚洲、美国城市看到的模型不一样",
+            blocks: [
+              { type: "paragraph", text: "模型覆盖域不同，所以同一套请求在不同坐标上返回的模型也不同。区域模型不覆盖某个城市时，不会返回空值，也不会在前端显示。" },
+              { type: "bullets", items: ["欧洲城市：通常会看到 ECMWF / AIFS / GFS / ICON / ICON-EU / ICON-D2 / GEM 或 GDPS / JMA。欧洲高分辨率重点来自 DWD ICON-EU 和 ICON-D2。", "北美城市：通常会看到 ECMWF / AIFS / GFS / ICON / GEM / GDPS / RDPS / HRDPS / JMA，并继续叠加 NWS。北美高分辨率重点来自 ECCC RDPS 和 HRDPS。", "亚洲城市：通常以 ECMWF / AIFS / GFS / ICON / GEM 或 GDPS / JMA 为主，通常不会有 ICON-EU、ICON-D2、RDPS、HRDPS；亚洲城市更依赖本地官方站、METAR、TAF、JMA、KMA、NMC、HKO、CWA 等观测增强层。"] },
+            ],
+          },
+          {
+            id: "deb-dedup",
+            title: "DEB 如何处理新增模型",
+            blocks: [
+              { type: "paragraph", text: "DEB 不会把所有新模型按“每个模型一票”直接等权加入。否则 ICON / ICON-EU / ICON-D2 会让 DWD 模型家族被重复放大，GEM / GDPS / RDPS / HRDPS 也会让加拿大模型家族被重复放大。" },
+              { type: "bullets", items: ["ICON / ICON-EU / ICON-D2 归为 DWD ICON 家族，优先级为 ICON-D2 > ICON-EU > ICON。", "GEM / GDPS / RDPS / HRDPS 归为 ECCC GEM 家族，优先级为 HRDPS > RDPS > GDPS > GEM。", "ECMWF IFS 与 ECMWF AIFS 分开保留，因为一个是传统数值模式，一个是 AI forecast。", "GFS、JMA、MGM、NWS、LGBM、Open-Meteo 等保持独立路径。", "DEB 权重信息中出现“家族去重”时，表示系统已经先折叠同家族模型，再进行历史 MAE 倒数加权。"] },
+              { type: "callout", tone: "info", title: "产品含义", text: "新增模型提升的是区域代表性和解释力，不是让某个地区因为模型数量更多就天然拥有更高权重。" },
+            ],
+          },
+          {
+            id: "display",
+            title: "网页上如何展示",
+            blocks: [
+              { type: "paragraph", text: "网页的“模型区间与分歧”会按全球基准、AI 预报、欧洲高分辨率、北美高分辨率分组显示，并展示可用模型数量、模型分歧、来源、模型名称、分辨率和预报时效。没有覆盖的区域模型不会显示。" },
+            ],
+          },
+        ],
+      },
+      "en-US": {
+        title: "Model Stack & DEB",
+        description: "This page explains which open models PolyWeather uses, why model coverage differs by region, and how DEB avoids duplicate family weighting.",
+        sections: [
+          {
+            id: "model-sources",
+            title: "Open models currently integrated",
+            blocks: [
+              { type: "paragraph", text: "PolyWeather's multi-model layer uses the Open-Meteo model API to integrate open NWP and AI forecasts. It does not download raw GRIB directly; instead, available models are normalized into the daily-high model stack used by spread, probabilities, and DEB." },
+              { type: "bullets", items: ["ECMWF IFS: global traditional NWP.", "ECMWF AIFS: ECMWF AI forecast, kept as a separate AI path.", "DWD ICON: global ICON baseline.", "DWD ICON-EU: European regional high-resolution layer.", "DWD ICON-D2: European short-range high-resolution layer.", "ECCC GEM / GDPS: Canadian global model family.", "ECCC RDPS / HRDPS: North American regional and short-range high-resolution layers.", "GFS / JMA: retained as global reference models."] },
+            ],
+          },
+          {
+            id: "regional-coverage",
+            title: "Why Europe, Asia, and US cities show different models",
+            blocks: [
+              { type: "paragraph", text: "Model domains differ, so the same request can return different model fields depending on the city coordinates. If a regional model does not cover a city, it is omitted rather than shown as an empty value." },
+              { type: "bullets", items: ["European cities usually show ECMWF / AIFS / GFS / ICON / ICON-EU / ICON-D2 / GEM or GDPS / JMA. The high-resolution European layer comes mainly from DWD ICON-EU and ICON-D2.", "North American cities usually show ECMWF / AIFS / GFS / ICON / GEM / GDPS / RDPS / HRDPS / JMA, plus existing NWS context. The high-resolution North American layer comes mainly from ECCC RDPS and HRDPS.", "Asian cities usually rely on ECMWF / AIFS / GFS / ICON / GEM or GDPS / JMA. ICON-EU, ICON-D2, RDPS, and HRDPS usually do not cover Asia, so Asian reads lean more on local official stations, METAR, TAF, JMA, KMA, NMC, HKO, CWA, and other observation enhancement layers."] },
+            ],
+          },
+          {
+            id: "deb-dedup",
+            title: "How DEB handles the new models",
+            blocks: [
+              { type: "paragraph", text: "DEB does not treat every new model as one independent vote. Otherwise ICON / ICON-EU / ICON-D2 would over-amplify the DWD family, and GEM / GDPS / RDPS / HRDPS would over-amplify the Canadian family." },
+              { type: "bullets", items: ["ICON / ICON-EU / ICON-D2 are collapsed into one DWD ICON family, with priority ICON-D2 > ICON-EU > ICON.", "GEM / GDPS / RDPS / HRDPS are collapsed into one ECCC GEM family, with priority HRDPS > RDPS > GDPS > GEM.", "ECMWF IFS and ECMWF AIFS are kept separate because one is traditional NWP and the other is an AI forecast.", "GFS, JMA, MGM, NWS, LGBM, and Open-Meteo remain independent paths.", "When the DEB weight string includes “family deduplication” or “家族去重”, the system has collapsed same-family models before applying historical inverse-MAE weighting."] },
+              { type: "callout", tone: "info", title: "Product meaning", text: "The new models improve regional representativeness and explanation quality. They do not let a region gain extra weight simply because more related model variants exist there." },
+            ],
+          },
+          {
+            id: "display",
+            title: "How the site displays this",
+            blocks: [
+              { type: "paragraph", text: "The Model Range & Spread panel groups the model stack into Global Baseline, AI Forecast, Europe High-resolution, and North America High-resolution. It shows available model count, spread, source, model name, resolution, and forecast horizon. Regional models outside their domain are simply not shown." },
+            ],
+          },
+        ],
+      },
+    },
+  },
  {
    slug: "taf-signal",
    group: "analysis",