- Exponential backoff: retry waits are now 1s, 2s, 4s (was linear 1s, 2s, 3s)
- Cache poisoning prevention: validate Content-Type is not HTML before JSON parse,
reject non-dict/list responses before caching
- Add 18 live smoke tests against real public APIs (USGS, CoinGecko, Yahoo Finance,
SEC EDGAR, FAA, NOAA SWPC, Hacker News, NASA EONET, etc.)
- Smoke tests excluded from default pytest run; opt-in with: pytest -m smoke
- All 220 unit tests + 18 smoke tests pass
Phase 16: Vector intelligence (+3 tools = 104 total)
New tools:
- intel_semantic_search: Natural language search across all stored intelligence
- intel_similar_events: Find historically similar events for pattern matching
- intel_timeline: Chronological timeline of stored data with domain filtering
Architecture:
- vector_store.py: Qdrant client with FastEmbed (ONNX, bge-small-en-v1.5)
- Background worker queue (asyncio) for non-blocking storage
- Auto-populates from every Fetcher.get_json() call
- 40+ domain categories mapped for filtered search
- Graceful degradation: vector store is optional, all 101 existing tools work without it
Integration:
- Fetcher: new vector_store parameter, stores data after each successful fetch
- server.py: vector store init + worker lifecycle + 3 dispatch cases
- dashboard/app.py: vector store worker starts on dashboard boot
- intel_status: now includes vector store statistics
- pyproject.toml: [vector] optional dependency (qdrant-client, fastembed)
186/186 tests passing.
- infrastructure.py: Add IODA (Georgia Tech) as fallback when Cloudflare
Radar returns 403 (no API token). Internet outage data now available
without CLOUDFLARE_API_TOKEN.
- conflict.py: Increase UCDP GED API timeout from 15s to 30s to handle
slow responses from ucdpapi.pcr.uu.se.
- dashboard/app.py: Add /api/report/pdf endpoint for PDF daily brief
export via weasyprint (optional dependency).
- pyproject.toml: Add dashboard + pdf optional deps, intel-dashboard
script entry point, update description for 55 tools.
- tests: Add 11 new tests covering health, sanctions, elections, shipping,
social, nuclear, infrastructure (IODA fallback + cable health), and
UCDP conflict. Fix cross-loop asyncio lock issue in conftest.
- Total: 50 tests passing, 55 tools verified.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Marc Shade