gavindiaz/my-polymarket-whale-watcher
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
gavindiaz
2026-07-12 09:18:28 +08:00
commit cc79650769
63 changed files with 10726 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+13
View File
@@ -0,0 +1,13 @@
venv/
.env
.git/
__pycache__/
*.pyc
data/
reports/
daily_briefings/
leading_signals/
price_volatility/
anomaly_signals/
*.db
*.json.bak
.env.example
+68
View File
@@ -0,0 +1,68 @@
# ============================================================
# Polymarket Whale Watcher - 配置文件
# ============================================================
# 复制到 .env 并填入你的 API 密钥：
# cp .env.example .env
#
# 只需 LLM_API_KEY 是必填项，其余均为可选。
# ============================================================
# --- 必填项 ---
# 任意 OpenAI 兼容的 API
LLM_API_KEY=sk-your-key-here
LLM_BASE_URL=https://api.openai.com/v1/
LLM_MODEL=gpt-4o
# --- 交易数据源 ---
# "official" = Polymarket 公共 API（无需认证，默认）
# "internal" = 私有 API（需配置 INTERNAL_API_URL + INTERNAL_API_KEY）
TRADE_API_MODE=official
# 内部 API（仅当 TRADE_API_MODE=internal 时需要）
# INTERNAL_API_URL=http://103.197.25.170:18088
# INTERNAL_API_KEY=your_internal_api_key_here
# --- LLM 设置 ---
LLM_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
LLM_MODEL=gemini-3-flash-preview
LLM_TEMPERATURE=0
# --- HTTP 代理（用于无法直连 Polymarket 的网络环境，如中国大陆） ---
HTTP_PROXY=http://127.0.0.1:7890
# --- 可选：搜索与数据 API（增强 LLM 分析能力） ---
# TAVILY_API_KEY= # 网页搜索 (https://tavily.com)
# TWITTER_API_KEY= # Twitter 社交情绪
# SERPER_API_KEY= # 网页搜索备用 (https://serper.dev)
# POLYGON_API_KEY= # 股票、外汇数据 (https://polygon.io)
# FRED_API_KEY= # 美国经济指标 (https://fred.stlouisfed.org)
# ETHERSCAN_API_KEY= # 链上数据 (https://etherscan.io)
# CONGRESS_API_KEY= # 美国立法动态 (https://api.congress.gov)
# --- 可选：Telegram 频道监控 ---
# TELEGRAM_API_ID=
# TELEGRAM_API_HASH=
# TELEGRAM_SESSION_STRING=
# TELEGRAM_CHANNELS=
# --- 鲸鱼检测参数调优 ---
MIN_TRADE_SIZE_USD=1000
MIN_PRICE=0
MAX_PRICE=0.7
FETCH_INTERVAL_SECONDS=15
TRENDING_MARKETS_LIMIT=50
# --- 邮件提醒（可选） ---
EMAIL_ENABLED=false
# EMAIL_SMTP_SERVER=smtp.qq.com
# EMAIL_SMTP_PORT=465
# EMAIL_SENDER=
# EMAIL_PASSWORD=
# EMAIL_RECIPIENT=
# --- 交易执行（默认关闭，谨慎使用） ---
ENABLE_TRADE_EXECUTION=false
# POLYGON_WALLET_PRIVATE_KEY=
# --- 日志 ---
LOG_LEVEL=INFO
.gitignore
+47
View File
@@ -0,0 +1,47 @@
# Environment variables
.env
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# Virtual environments
venv/
ENV/
env/
.venv/
# IDE
.idea/
.vscode/
*.swp
*.swo
# OS
.DS_Store
Thumbs.db
# Reports (optional - uncomment if you don't want to track reports)
# reports/
# Data files
data/*.json
data/*.csv
Dockerfile
+21
View File
@@ -0,0 +1,21 @@
FROM python:3.12-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
&& rm -rf /var/lib/apt/lists/*
# Install Python dependencies
COPY pyproject.toml .
RUN pip install --no-cache-dir .
# Copy application code
COPY src/ src/
# Create data directories
RUN mkdir -p data reports daily_briefings
# Default command
CMD ["python", "-m", "src.main", "run"]
Makefile
+43
View File
@@ -0,0 +1,43 @@
.PHONY: setup run dashboard briefing markets test lint clean docker-build docker-run help
PYTHON = python3
VENV = venv
PIP = $(VENV)/bin/pip
PY = $(VENV)/bin/python
help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'
setup: ## One-click setup (venv + dependencies + .env)
@chmod +x setup.sh && ./setup.sh
run: ## Start the whale watcher
$(PY) -m src.main run
run-debug: ## Start with debug logging
$(PY) -m src.main run --debug
dashboard: ## Start the web dashboard
$(PY) -m src.main dashboard
briefing: ## Generate today's briefing
$(PY) -m src.main briefing --today
markets: ## Show trending markets
$(PY) -m src.main check-markets --limit 20
test: ## Run tests
$(PY) -m pytest tests/ -v
lint: ## Run linter
$(VENV)/bin/ruff check src/
clean: ## Remove generated files (keep data)
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
find . -type f -name '*.pyc' -delete 2>/dev/null || true
docker-build: ## Build Docker image
docker build -t polymarket-whale-watcher .
docker-run: ## Run in Docker
docker run --env-file .env -v $(PWD)/data:/app/data -v $(PWD)/reports:/app/reports polymarket-whale-watcher
README.md
+525
View File
@@ -0,0 +1,525 @@
<div align="center">
# 🐋 Polymarket Whale Watcher
**AI-Powered Whale Trade Intelligence for Polymarket Prediction Markets**
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-3776AB.svg?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-22c55e.svg?style=for-the-badge)](LICENSE)
[![Polymarket](https://img.shields.io/badge/Polymarket-API-8B5CF6.svg?style=for-the-badge)](https://docs.polymarket.com/)
[![LLM](https://img.shields.io/badge/LLM-OpenAI_Compatible-412991.svg?style=for-the-badge&logo=openai&logoColor=white)](https://platform.openai.com/)
<br/>
**Real-time monitoring** of 700+ markets &nbsp;·&nbsp; **14 autonomous research tools** &nbsp;·&nbsp; **Multi-step deep analysis** &nbsp;·&nbsp; **Signal accuracy tracking**
<br/>
[Quick Start](#-quick-start) &nbsp;&nbsp;|&nbsp;&nbsp; [How It Works](#-how-it-works) &nbsp;&nbsp;|&nbsp;&nbsp; [Sample Report](#-sample-report) &nbsp;&nbsp;|&nbsp;&nbsp; [Configuration](#%EF%B8%8F-configuration) &nbsp;&nbsp;|&nbsp;&nbsp; [Dashboard](#-dashboard)
---
<img width="720" alt="pipeline" src="https://img.shields.io/badge/Tiered_Markets_(700+)→Whale_Detection→Anomaly_Scoring→LLM_Investigation→Signal_Tracking-000?style=flat-square&labelColor=000"/>
</div>
<br/>
## What It Does
Whale Watcher continuously monitors **700+ active Polymarket markets** across three volume tiers, detects large trades with anomalous patterns, and deploys an LLM agent with **14 autonomous research tools** to conduct multi-step deep investigations. Each whale trade undergoes a structured 7-step analysis pipeline — from trader profiling and cross-market position mapping to information gap assessment — producing an **Information Asymmetry Score** that quantifies the likelihood of non-public information advantage.
<br/>
## Live Demo
<details open>
<summary><strong>Terminal Output</strong> — Real-time whale detection and analysis</summary>
<br/>
```
$ python -m src.main run
============================================================
WHALE WATCHER STARTED
============================================================
Monitoring: 765 markets
Interval: 10 seconds
Min Trade Size: $10,000 USD
Price Range: 0.1 - 0.9
============================================================
Tiered monitoring: Tier1=8 (>500K), Tier2=198 (>10K), Tier3=559 (>1K)
[23:41:12] WHALE TRADE DETECTED!
Amount: $9,600.00 USDC
Side: BUY Yes
Price: 0.7142
Market: US x Iran diplomatic meeting by June 30, 2026?
Generating analysis report...
Round 1: LLM requested 3 tool call(s)
→ search_web("US Iran diplomatic meeting June 2026")
→ search_twitter("US Iran meeting diplomacy")
→ get_wallet_transfers("0xceza...rn132")
Round 2: LLM requested 2 tool call(s)
→ search_web("Islamabad Iran talks Witkoff April 2026")
→ search_twitter("POLYMARKET Iran meeting odds fading")
Round 3: LLM requested 1 tool call(s)
→ search_web("Iran FM Araghchi 3 phase deal proposal")
Analysis complete after 3 round(s)
Information Asymmetry Score: 0.32 (LOW)
Trader Credibility: MEDIUM (#1733, PnL: $84K)
Verdict: Thesis continuation / loss recovery — HOLD/PASS
Report saved: reports/20260504/...
```
</details>
<br/>
<details>
<summary><strong>Sample Report</strong> — 7-Step Deep Analysis (click to expand)</summary>
<br/>
Each whale trade generates a comprehensive markdown report with structured multi-step analysis:
> **Full example**: [docs/examples/sample_report.md](docs/examples/sample_report.md)
#### Report Structure
```
======================================================================
# Whale Trade Analysis Report
======================================================================
┌─ Trade Summary ─────────────────────────────────────────────────────┐
│ Market, trade size, direction, price, odds, time, trader rank │
└─────────────────────────────────────────────────────────────────────┘
┌─ Step 2: Trade Signal Analysis ─────────────────────────────────────┐
│ Trader profile: rank, PnL, avg size, large trade ratio │
│ Domain expertise detection, trade timing analysis │
└─────────────────────────────────────────────────────────────────────┘
┌─ Step 3: Event-Related Position Analysis ───────────────────────────┐
│ Cross-market positions, roll-forward detection │
│ Loss recovery patterns, hedge identification │
└─────────────────────────────────────────────────────────────────────┘
┌─ Step 4: Market Long/Short Analysis ────────────────────────────────┐
│ Top 5 bulls & bears with rankings and PnL │
│ Smart money consensus assessment │
└─────────────────────────────────────────────────────────────────────┘
┌─ Step 5: Information Gap Analysis ──────────────────────────────────┐
│ Public information audit (web, Twitter, Telegram) │
│ Market pricing efficiency check │
│ Non-public information evidence search │
└─────────────────────────────────────────────────────────────────────┘
┌─ Step 6: Historical Pattern ────────────────────────────────────────┐
│ Trader's past bets on related events │
│ Strategy pattern recognition (laddering, hedging, etc.) │
└─────────────────────────────────────────────────────────────────────┘
┌─ Step 7: Information Asymmetry Assessment ──────────────────────────┐
│ Score (0–1), trader credibility, evidence, reasoning │
│ Multi-factor summary table with signal strength │
│ Recommended action: BUY / HOLD / PASS │
└─────────────────────────────────────────────────────────────────────┘
```
#### Example Summary Table
| Factor | Assessment | Signal |
|--------|-----------|--------|
| Trader Rank/PnL | Rank #1733, $84K PnL | Moderate |
| Trade Size vs. Normal | ~$9.6K vs. avg $10K | Routine — neutral |
| Related Position | Heavy loser on May 15 market (-$6.5K) | Suppresses signal |
| Domain Expertise | Iran geopolitics specialist | Supportive |
| Public Info Coverage | Extensive public news | Reduces asymmetry |
| Smart Money Bulls | Rank #278 also long | Modest support |
| **Overall** | **Thesis continuation, not insider signal** | **Low-Medium** |
</details>
<br/>
<details>
<summary><strong>Daily Briefing</strong> — Automated intelligence summary</summary>
<br/>
Daily briefings are generated at **10:00 AM local time** and emailed automatically.
> **Full example**: [docs/examples/sample_briefing.md](docs/examples/sample_briefing.md)
**Includes:**
- High-confidence signals (IAS >= 60%) with full analysis summaries
- Fallback: top 5 signals by score if none reach the threshold
- Abnormal price volatility alerts
- Historical signal performance — win rate and ROI by confidence tier
</details>
<br/>
---
## Quick Start
### One-Click Setup
```bash
git clone https://github.com/chaoleiyv/polymarket-whale-watcher.git
cd polymarket-whale-watcher
chmod +x setup.sh && ./setup.sh
```
The setup script will:
1. Check Python 3.10+ is installed
2. Create a virtual environment
3. Install all dependencies
4. Create `.env` from template
Then add your API key and start:
```bash
# Add your LLM API key (the only required key)
echo "LLM_API_KEY=your_key_here" >> .env
# Activate the environment and run
source .venv/bin/activate
python -m src.main run
```
> **Get an API key**: [OpenAI](https://platform.openai.com/api-keys) · [Google AI Studio](https://aistudio.google.com/apikey) (free tier) · [DeepSeek](https://platform.deepseek.com/)
### Docker
```bash
docker build -t whale-watcher .
docker run --env-file .env -v ./data:/app/data -v ./reports:/app/reports whale-watcher
```
---
## How It Works
```mermaid
flowchart LR
A[Polymarket API] --> B[Market Fetcher]
B --> C[700+ Tiered Markets]
C --> D[Trade Monitor]
D --> E{Whale\nTrade?}
E -->|No| D
E -->|Yes| F[Anomaly Detector]
F --> G{Score >= 0.65?}
G -->|No| D
G -->|Yes| H[LLM Analyzer]
H --> I[14 Research Tools]
I --> J[Signal + Report]
J --> K[Resolution Tracker]
K --> L[Dashboard + Email]
```
### Pipeline
| Stage | What Happens |
|-------|-------------|
| **1. Market Selection** | Fetches all active markets from Polymarket Gamma API, classifies into 3 tiers by 24h volume, adds token launch markets. Refreshes every 15 minutes. |
| **2. Trade Monitoring** | Parallel async tasks per market (700+), polls official Polymarket data-api for new taker BUY trades, deduplicates by transaction hash. Connection pool: 50 connections, 120s timeout. |
| **3. Whale Pre-filter** | Price range 0.10–0.90, $5K hard floor, dynamic threshold scaled by volume ($5K–$100K), conviction check (must pay above mid), resolution window 6h–90d. |
| **4. Anomaly Scoring** | 5-factor model (max 1.0): base confidence (0.50) + premium ratio (0.20) + signal cleanliness (0.10) + depth ratio (0.10) + cluster tier (0.10). Threshold: >= 0.65. |
| **5. LLM Investigation** | Builds rich context (trade + trader profile + event positions + market top holders + historical signals). LLM autonomously selects tools for up to 3 rounds. Produces structured 7-step analysis with information asymmetry score (0–1). |
| **6. Signal Tracking** | Resolution tracker checks every 30 min, validates signal correctness, computes theoretical ROI. Daily briefings at 10:00 AM, emailed to recipients. |
---
## Features
| Feature | Description |
|---------|-------------|
| **Tiered Market Monitoring** | 700+ markets across 3 tiers: Tier1 (>$500K, 15s), Tier2 (>$10K, 60s), Tier3 (>$1K, 300s) |
| **5-Factor Anomaly Detection** | Premium ratio, signal cleanliness, depth ratio, cluster signals, base confidence |
| **7-Step Deep Analysis** | Trade signal → Event positions → Long/short mapping → Info gap → Historical pattern → Asymmetry score |
| **14 Autonomous Research Tools** | Web, Twitter, Telegram, crypto, DeFi, stocks, on-chain, legislation |
| **Cross-Market Position Analysis** | Detects roll-forwards, hedges, and loss recovery patterns across related markets |
| **Signal Accuracy Tracking** | Auto resolution checking every 30 min, win rate stats by confidence tier |
| **Daily Briefings** | 10:00 AM automated summary with high-confidence signals, emailed to recipients |
| **Real-time Email Alerts** | Instant notifications for high information-asymmetry signals (>= 60%) |
| **Web Dashboard** | FastAPI-based signal performance dashboard with ROI breakdowns |
### 14 LLM Research Tools
The LLM agent autonomously selects and chains these tools during its multi-round investigation:
| Category | Tools | Use Case |
|----------|-------|----------|
| **Social & Sentiment** | `search_twitter` · `search_telegram` · `search_web` | Public sentiment, insider chatter, news coverage |
| **Crypto & DeFi** | `get_crypto_price` · `get_crypto_market_overview` · `get_protocol_tvl` · `get_token_unlocks` · `get_protocol_revenue` | Token prices, TVL, unlocks, protocol health |
| **Financial Data** | `get_stock_price` · `get_stock_news` · `get_economic_data` | Equities, ETFs, macro indicators |
| **On-Chain** | `get_wallet_transfers` · `get_contract_info` | Wallet activity, contract deployments |
| **Legislation** | `get_bill_status` · `get_recent_legislation` | US bills, regulatory actions |
---
## Sample Report
Every whale trade produces a structured multi-step report. Here's a condensed view:
```
======================================================================
# Whale Trade Analysis Report
======================================================================
Trade Summary
Market: US x Iran diplomatic meeting by June 30, 2026?
Size: $9,600 USDC | Direction: BUY Yes (71.4%) | Trader: #1733
Step 2: Trade Signal Analysis
→ Mid-tier trader, $84K PnL, Iran geopolitics specialist
→ Trade size ($9.6K) matches avg ($10K) — routine, not exceptional
Step 3: Event-Related Position Analysis ← KEY FINDING
→ Losing -$6,508 on earlier "May 15 meeting" market (15.5% odds)
→ This trade is a thesis roll-forward, not a fresh insider bet
Step 4: Market Long/Short Analysis
→ Biggest Yes holder is a chronic loser (PnL: -$5.5M) — red flag
→ One elite trader (Rank #278) also long — modest support
Step 5: Information Gap Analysis
→ All supporting info widely reported in mainstream media
→ Market at 69.5% — already fairly priced
Step 6: Historical Pattern
→ "Timeline ladder" strategy across multiple Iran-related deadlines
Step 7: Information Asymmetry Assessment
→ Score: 0.32 (LOW) | Credibility: MEDIUM
→ Verdict: Thesis continuation / loss recovery — HOLD/PASS
======================================================================
```
> **Full report**: [docs/examples/sample_report.md](docs/examples/sample_report.md)
---
## Configuration
Copy `.env.example` to `.env` and configure:
### Required
| Variable | Description | Get It |
|----------|-------------|--------|
| `LLM_API_KEY` | LLM API key for analysis (OpenAI-compatible) | [OpenAI](https://platform.openai.com/api-keys) / [Google AI Studio](https://aistudio.google.com/apikey) |
### Optional (enhances analysis quality)
<details open>
<summary><strong>Data Source API Keys</strong></summary>
| Variable | Description | Get It |
|----------|-------------|--------|
| `TAVILY_API_KEY` | Web search (primary) | [tavily.com](https://tavily.com) |
| `SERPER_API_KEY` | Web search (fallback) | [serper.dev](https://serper.dev) |
| `TWITTER_API_KEY` | Twitter sentiment search | [twitterapi.io](https://twitterapi.io) |
| `POLYGON_API_KEY` | Stock/ETF/forex data | [polygon.io](https://polygon.io) |
| `FRED_API_KEY` | Economic indicators | [FRED](https://fred.stlouisfed.org/docs/api/api_key.html) |
| `ETHERSCAN_API_KEY` | On-chain wallet analysis (Polygon V2) | [etherscan.io](https://etherscan.io/apis) |
| `CONGRESS_API_KEY` | US legislation data | [congress.gov](https://api.congress.gov/) |
| `TELEGRAM_API_ID` / `TELEGRAM_API_HASH` | Telegram channel monitoring | [my.telegram.org](https://my.telegram.org) |
</details>
<details>
<summary><strong>LLM Settings</strong></summary>
| Variable | Default | Description |
|----------|---------|-------------|
| `LLM_MODEL` | `gemini-3-flash-preview` | Model name (any OpenAI-compatible) |
| `LLM_BASE_URL` | Google AI endpoint | OpenAI-compatible API base URL |
| `LLM_TEMPERATURE` | `0` | LLM temperature |
</details>
<details>
<summary><strong>Whale Detection Tuning</strong></summary>
| Variable | Default | Description |
|----------|---------|-------------|
| `MIN_TRADE_SIZE_USD` | `10000` | Minimum trade size to consider |
| `MIN_PRICE` / `MAX_PRICE` | `0.10` / `0.90` | Price range filter |
| `FETCH_INTERVAL_SECONDS` | `10` | Default polling interval |
</details>
<details>
<summary><strong>Tiered Market Monitoring</strong></summary>
| Variable | Default | Description |
|----------|---------|-------------|
| `FULL_MARKET_SCAN` | `true` | Enable tiered monitoring (all active markets) |
| `TIER1_VOLUME_MIN` | `500000` | Tier 1 volume threshold |
| `TIER2_VOLUME_MIN` | `10000` | Tier 2 volume threshold |
| `TIER3_VOLUME_MIN` | `1000` | Tier 3 volume threshold |
| `TIER1_POLL_INTERVAL` | `15` | Tier 1 polling interval (seconds) |
| `TIER2_POLL_INTERVAL` | `60` | Tier 2 polling interval (seconds) |
| `TIER3_POLL_INTERVAL` | `300` | Tier 3 polling interval (seconds) |
</details>
<details>
<summary><strong>Email Alerts</strong></summary>
| Variable | Default | Description |
|----------|---------|-------------|
| `EMAIL_ENABLED` | `false` | Enable email notifications |
| `EMAIL_SENDER` | — | Sender email address |
| `EMAIL_PASSWORD` | — | Sender email password (app password) |
| `EMAIL_RECIPIENT` | — | Comma-separated recipient emails |
</details>
---
## Commands
```bash
# Core
python -m src.main run [--debug] # Start monitoring
python -m src.main check-markets --limit 20 # View trending markets
# Analysis
python -m src.main test-analyze <market_id> # Test LLM on a specific market
# Reports
python -m src.main briefing --today # Generate today's briefing
python -m src.main briefing --date 2026-04-17 # Briefing for a specific date
# Dashboard
python -m src.main dashboard --port 8000 # Start web dashboard
# Maintenance
python -m src.main migrate # Migrate legacy JSON to SQLite
```
---
## Dashboard
```bash
python -m src.main dashboard
# Open http://localhost:8000
```
The dashboard shows:
- Overall signal statistics (total signals, win rate, avg ROI)
- Performance breakdown by confidence tier
- Top best/worst signals by theoretical ROI
- Paginated signal history
---
## Architecture
```
┌──────────────────────────────┐
│ Polymarket Gamma API │
│ (all active markets) │
└──────────────┬───────────────┘
│
┌──────────────▼───────────────┐
│ Market Fetcher │
│ Tier1: >$500K (15s poll) │
│ Tier2: >$10K (60s poll) │
│ Tier3: >$1K (300s poll) │
└──────────────┬───────────────┘
│
┌────────────────────▼────────────────────┐
│ Trade Monitor (async, 700+) │
│ Official Polymarket data-api │
│ Per-market parallel tasks │
│ Pool: 50 connections, 120s timeout │
└────────────────────┬────────────────────┘
│
┌────────────────────▼────────────────────┐
│ Pre-filter + Scoring │
│ $5K+ size, 0.10-0.90 price, conviction │
│ 5-factor anomaly score >= 0.65 │
└────────────────────┬────────────────────┘
│
┌────────────────────▼────────────────────┐
│ LLM Analyzer — 7-Step Pipeline │
│ 14 tools · up to 3 rounds │
├──────────┬────────┬────────┬────────────┤
│ Twitter │ Web │ DeFi │ On-Chain │
│ Telegram │ Search │ Crypto │ Legislation│
└──────────┴───┬────┴────────┴────────────┘
│
┌──────────────▼─────────────────────────┐
│ Signal Storage (SQLite) │
│ → Resolution Tracker (every 30min) │
│ → Daily Briefing (10:00 AM + email) │
│ → Email Alerts (IAS >= 60%) │
│ → Dashboard (FastAPI) │
└────────────────────────────────────────┘
```
---
## Project Structure
```
src/
├── config/settings.py # Environment configuration (Pydantic)
├── models/ # Data models
│ ├── market.py # Market, TrendingMarket
│ ├── trade.py # TradeActivity, WhaleTrade, TraderRanking
│ ├── decision.py # TradeRecommendation, LLMDecision
│ └── anomaly_signal.py # AnomalySignal (stored signal)
├── services/ # Business logic
│ ├── market_fetcher.py # Polymarket Gamma API (tiered market selection)
│ ├── trade_monitor.py # Per-market parallel monitoring (official API)
│ ├── anomaly_detector.py # 5-factor anomaly scoring
│ ├── llm_analyzer.py # LLM with tool-use (14 tools, 3 rounds)
│ ├── tools.py # Tool registry
│ ├── resolution_tracker.py # Market resolution checking
│ ├── stats_engine.py # Performance statistics
│ ├── daily_briefing.py # Daily summary generation + email
│ ├── twitter_search.py # Twitter API search
│ ├── telegram_search.py # Telegram channel monitoring
│ ├── web_search.py # Tavily/Serper/DuckDuckGo search
│ ├── coingecko.py # Crypto prices and market data
│ ├── defillama.py # DeFi TVL, revenue, token unlocks
│ ├── fred.py # FRED macroeconomic data
│ ├── polygon.py # Stock/ETF prices and news
│ ├── etherscan.py # On-chain data (Polygon, Etherscan V2)
│ └── congress.py # US legislation data
├── prompts/ # LLM system prompts
│ ├── whale_analyzer.py # Whale trade analysis prompt
│ └── volatility_analyzer.py # Price volatility analysis prompt
├── db/database.py # SQLite signal storage
└── main.py # CLI entry point (Typer)
```
---
## Disclaimer
This system is for **research and educational purposes only**. Prediction market trading involves significant risk. The information asymmetry scores and analyses are AI-generated estimates — not financial advice. Always conduct your own research and verify independently before making any trading decisions.
---
<div align="center">
**MIT License** &nbsp;·&nbsp; Built with Polymarket API + LLM
</div>
data/signals.db
BIN
View File
Binary file not shown.
data/signals.db-shm
BIN
View File
Binary file not shown.
data/signals.db-wal
BIN
View File
Binary file not shown.
deploy.sh
+54
View File
@@ -0,0 +1,54 @@
#!/bin/bash
# ============================================================
# Polymarket Whale Watcher - 云端部署脚本
# 支持 Ubuntu 20.04+ / Debian 11+
# ============================================================
set -e
echo "=== Polymarket Whale Watcher 部署 ==="
# 1. 安装系统依赖
echo "[1/5] 安装系统依赖..."
sudo apt-get update -qq
sudo apt-get install -y -qq python3 python3-pip python3-venv screen
# 2. 创建虚拟环境
echo "[2/5] 创建 Python 虚拟环境..."
python3 -m venv venv
source venv/bin/activate
# 3. 安装项目依赖
echo "[3/5] 安装 Python 依赖..."
pip install --upgrade pip
pip install -e .
# 4. 创建数据目录
echo "[4/5] 创建数据目录..."
mkdir -p data reports daily_briefings
# 5. 配置 .env（如果不存在）
if [ ! -f .env ]; then
echo "[5/5] 创建 .env 配置文件..."
cp .env.example .env
echo "请编辑 .env 填入你的 LLM_API_KEY 和 HTTP_PROXY"
else
echo "[5/5] .env 已存在，跳过"
fi
echo ""
echo "=== 部署完成 ==="
echo ""
echo "启动监控:"
echo " screen -S watcher"
echo " source venv/bin/activate"
echo " python -m src.main run"
echo " 按 Ctrl+A D 分离"
echo ""
echo "启动面板:"
echo " screen -S dashboard"
echo " source venv/bin/activate"
echo " python -m src.main dashboard --host 0.0.0.0"
echo " 按 Ctrl+A D 分离"
echo ""
echo "Docker 方式（推荐）:"
echo " docker compose up -d"
docker-compose.yml
+31
View File
@@ -0,0 +1,31 @@
services:
watcher:
build: .
restart: unless-stopped
env_file:
- .env
volumes:
- ./data:/app/data
- ./reports:/app/reports
- ./daily_briefings:/app/daily_briefings
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
dashboard:
build: .
restart: unless-stopped
command: ["python", "-m", "src.main", "dashboard", "--host", "0.0.0.0", "--port", "8517"]
env_file:
- .env
ports:
- "8517:8517"
volumes:
- ./data:/app/data
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
docs/examples/sample_briefing.md
+84
View File
@@ -0,0 +1,84 @@
# Daily Signal Briefing - 2026-04-15
Generated at: 2026-04-16 10:00:03
## Today's Overview
- High-confidence information asymmetry signals: **3** (confidence >= 60%)
- Abnormal price volatility events: **2**
---
## High-Confidence Information Asymmetry Signals
### 1. Will MegaETH launch a token by June 30, 2026?
| Metric | Value |
|--------|-------|
| Info Asymmetry | **72%** |
| Direction | BUY Yes Token (Bullish) |
| Entry Price | 0.4200 (Odds 2.4x) |
| Trade Size | **$92,336** USDC |
| Detected At | 2026-04-15 14:23:07 |
**Analysis**: High-ranked trader (#47, PnL $284K) making concentrated bets. Unannounced smart contract deployed by MegaETH team 6 hours prior. KOL insider chatter on Twitter preceded the trade.
**Insider Evidence**: New ERC-20 contract deployed by MegaETH deployer wallet, not yet publicly announced.
### 2. US forces enter Iran by April 30
| Metric | Value |
|--------|-------|
| Info Asymmetry | **65%** |
| Direction | BUY Yes Token (Bullish) |
| Entry Price | 0.3100 (Odds 3.2x) |
| Trade Size | **$46,500** USDC |
| Detected At | 2026-04-15 09:21:53 |
**Analysis**: Top-50 trader with $500K+ PnL placing aggressive bets. Multiple high-ranked traders converging on the same direction. Trade occurred hours before Pentagon press briefing.
### 3. Will the next Prime Minister of Hungary be Viktor Orban?
| Metric | Value |
|--------|-------|
| Info Asymmetry | **61%** |
| Direction | BUY No Token (Bearish) |
| Entry Price | 0.2800 (Odds 3.6x) |
| Trade Size | **$65,699** USDC |
| Detected At | 2026-04-15 16:31:22 |
**Analysis**: Verified trader with strong European politics track record betting against Orban. Telegram channels discussing potential coalition shift not yet covered by mainstream media.
---
## Abnormal Price Volatility
| Market | Direction | Change | Start Price | End Price | Detected At |
|--------|-----------|--------|-------------|-----------|-------------|
| EdgeX FDV above 500M one day... | Up | 35.2% | 22.00% | 57.20% | 2026-04-15 03:41 |
| Iran leadership change by Dec... | Up | 22.8% | 31.00% | 53.80% | 2026-04-15 11:15 |
---
## Signal History
| Metric | Value |
|--------|-------|
| Total Signals | 847 |
| Resolved | 312 |
| Correct | 198 |
| Win Rate | **63.5%** |
| Avg ROI | **+28.3%** |
| Theoretical Total PnL | **+42.71x** |
### Performance by Confidence Tier
| Confidence Range | Signals | Resolved | Win Rate | Avg ROI |
|-----------------|---------|----------|----------|---------|
| 0.8 - 1.0 | 23 | 15 | 80.0% | +52.1% |
| 0.6 - 0.8 | 89 | 47 | 68.1% | +35.7% |
| 0.4 - 0.6 | 284 | 132 | 59.1% | +18.4% |
---
*This briefing was automatically generated by Polymarket Whale Watcher*
docs/examples/sample_report.md
+139
View File
@@ -0,0 +1,139 @@
======================================================================
# Whale Trade Analysis Report
======================================================================
**Generated at**: 2026-05-04 16:00:00 UTC
## Trade Summary
| Field | Details |
|-------|---------|
| **Market** | US x Iran diplomatic meeting by June 30, 2026? |
| **Trade Size** | $9,600.00 USDC |
| **Direction** | BUY Yes Token (Bullish) |
| **Trade Price** | 0.7142 (71.4%) |
| **Current Odds** | Yes: 69.5% | No: 30.5% |
| **Trade Time** | 2026-05-04 23:41:12 |
| **Trader Rank** | #1733 (PnL: $84,695.33) |
======================================================================
## Full Analysis Report
### Step 2: Trade Signal Analysis
**Trader Profile Assessment:**
- **Rank #1733** — mid-tier experienced trader, not top-tier elite
- **PnL: $84,695** — solid positive track record across ~$4M lifetime volume
- **Avg trade size: $10,018** — this $9,600 trade is perfectly in-line with his normal size, not an unusual outlier
- **Large trade ratio: 66%** — consistent whale behavior, large trades are routine for this trader
- **Domain expertise: Strong Iran/geopolitics focus** — recent trades include Iranian regime fall (April/June), US-Iran meetings (April/June), Russia-Ukraine ceasefire, Ukraine Donbas. This is clearly a geopolitics specialist with deep focus on Iran specifically.
**Trade Timing:** The trade was made at 23:41 UTC on May 4, 2026. Given the active diplomatic situation, this is a routine positioning trade, not an oddly-timed pre-announcement bet.
---
### Step 3: Event-Related Position Analysis
The whale has a **losing position** in the earlier "US x Iran diplomatic meeting by May 15, 2026?" market:
- Holding Yes tokens @ avg 65.29%, now priced at **15.50%**
- Cost basis: $8,534 | Current value: $2,026 | **PnL: -$6,508 (LOSS)**
This is critical context:
1. The trader **bet heavily on a May 15 meeting** and that bet is **nearly dead** (15.5% odds)
2. The June 30 buy ($9,600) appears to be a **portfolio roll-forward** — doubling down on the same thesis with a later deadline to recoup losses
3. This is NOT a fresh, confident insider bet — it's a **loss recovery / thesis extension trade**
4. The pattern suggests the trader is sticking to a bullish Iran-diplomacy thesis despite being wrong on the nearer-term deadline
---
### Step 4: Market Long/Short Analysis
**Bulls (Yes side):**
- **anoin123** (Rank #2,682,942, PnL **-$5.54M**) — massive loser, leads the Yes side with $39K. This is a **red flag** — the biggest Yes holder is a chronic loser
- **ArmageddonRewardsBilly** (Rank #278, PnL $525K) — elite trader, $11K on Yes. This is the strongest bull signal
- **cezarn132** (the whale) — Rank #1733, $9.3K Yes
**Bears (No side):**
- **aaron107** (Rank #48,390) — low-ranked, leads No
- **yungstalin** (Rank #1,969, PnL $73,759) — solid mid-tier trader on No side
**Smart Money Consensus:** Mixed. One elite trader (Rank #278) is bullish, but the biggest Yes position belongs to a massive loser. The No side lacks elite concentration too. No clear smart money consensus signal in either direction.
---
### Step 5: Information Gap Analysis
**Public Information Summary:**
- There was already an indirect round of talks in Islamabad, Pakistan in late April 2026 (confirmed by Wikipedia)
- US envoy Steve Witkoff stated Washington is "in conversation" with Iran
- Iran's FM Araghchi has made public proposals for a 3-phase deal
- Market odds for June 30 meeting are ~69.5% — already well-priced for the optimistic scenario
- However, the **May 15 market is at just 15.5%**, showing markets believe a near-term meeting is unlikely
- Twitter data shows @faststocknewss noting "POLYMARKET ODDS OF A US-IRAN DIPLOMATIC MEETING ARE FADING" — the June 30 market was previously at 80%+ and has dropped to 66-69%
**Key Information Assessment:**
- All available information about Iran-US diplomacy is extensively covered in public media
- The market at 69.5% appears to **reasonably price** the known information
- No evidence of non-public information advantage — this appears to be public-information-based analysis
- The trade is consistent with this trader's established Iran geopolitics thesis
---
### Step 6: Historical Pattern
This trader has a **clear and consistent pattern**:
- Bought "Iranian regime fall by April 30" at 0.86 → **SOLD at 0.98** (big win)
- Bought "Iranian regime fall by June 30" at 0.76 and 0.74 → Still holding
- Bet on "US x Iran diplomatic meeting by April 25/May 15" → **LOSING** at 15.5%
The pattern suggests this trader uses a "timeline ladder" approach — betting on Iran-related events across multiple deadlines. When early deadlines miss, they roll to later ones. This reduces the information asymmetry signal significantly.
---
## Step 7: Information Asymmetry Assessment
```json
{
"information_asymmetry_score": 0.32,
"trader_credibility": "MEDIUM",
"reasoning": "Trader cezarn132 (Rank #1733, PnL $84K) is a confirmed geopolitics specialist with deep Iran-focused trading history. However, several factors suppress the information asymmetry score: (1) The $9,600 trade is exactly in-line with this trader's average trade size ($10K), making it routine rather than an exceptional conviction bet. (2) The trader currently holds a heavily losing position in the nearer 'May 15' meeting market (down $6,508), and this June 30 buy appears to be a thesis roll-forward/loss recovery strategy rather than a fresh insider signal. (3) All supporting information (Islamabad talks, Witkoff statements, Iranian proposals) is extensively public and already priced into the ~69.5% market odds.",
"insider_evidence": "No insider evidence found. All information supporting the bullish Iran diplomacy view is widely reported in mainstream media. The trade is most consistent with a thesis-continuation/loss-recovery strategy following a losing May 15 position."
}
```
---
### Summary Table
| Factor | Assessment | Signal Strength |
|--------|-----------|-----------------|
| Trader Rank/PnL | Rank #1733, $84K PnL | Moderate |
| Trade Size vs. Normal | ~$9.6K vs. avg $10K | Routine — neutral |
| Related Position | Heavy loser on May 15 meeting (-$6.5K) | Suppresses signal |
| Domain Expertise | Iran geopolitics specialist | Supportive |
| Public Info Coverage | Extensive public news on US-Iran talks | Reduces asymmetry |
| Market Pricing | Already at 69.5% — well-priced | Reduces asymmetry |
| Smart Money Bulls | Rank #278 also long | Modest support |
| Biggest Yes Holder | Rank #2.6M, -$5.5M PnL | Negative signal |
| **Overall** | **Likely thesis continuation, not insider signal** | **Low-Medium** |
**Recommended Action: HOLD/PASS** — This trade reflects a geopolitics analyst rolling forward a losing thesis rather than a trader acting on non-public information.
======================================================================
## Information Asymmetry Assessment
======================================================================
| Field | Assessment |
|-------|------------|
| **Information Asymmetry** | Low Information Asymmetry (32%) |
| **Trader Credibility** | Medium Credibility (#1733) |
**Key Evidence**: No insider evidence found. All information supporting the bullish Iran diplomacy view is widely reported in mainstream media. The trade is most consistent with a thesis-continuation/loss-recovery strategy following a losing May 15 position.
**Reasoning**: Trader cezarn132 (Rank #1733, PnL $84K) is a confirmed geopolitics specialist with deep Iran-focused trading history. However, several factors suppress the information asymmetry score: (1) routine trade size, (2) losing position on nearer deadline, (3) extensive public information coverage, (4) declining market odds, (5) mixed smart money picture, (6) large geopolitical market with many participants.
======================================================================
Disclaimer: This report is AI-generated for informational purposes only and does not constitute investment advice.
======================================================================
docs/examples/sample_terminal.txt
+54
View File
@@ -0,0 +1,54 @@
$ python -m src.main run
╭──────────────────────────────────────────────────────────╮
│ 🐋 Polymarket Whale Watcher │
│ │
│ Markets Monitored: 50 │
│ Polling Interval: 15s │
│ Min Trade Size: $1,000 │
│ Price Range: 0 - 0.7 │
╰──────────────────────────────────────────────────────────╯
[14:22:41] 🔄 Refreshing trending markets... Found 50 active markets
[14:22:43] 👀 Monitoring: Will MegaETH launch a token by June 30, 2026?
[14:22:43] 👀 Monitoring: US forces enter Iran by April 30
[14:22:43] 👀 Monitoring: Will the next Prime Minister of Hungary be Viktor Orban?
... (47 more markets)
[14:22:51] 🐋 WHALE DETECTED on "Will MegaETH launch a token by June 30, 2026?"
BUY Yes @ 0.4200 | $92,336 USDC | Wallet: 0x7a3b...f91e
Anomaly Score: 0.78/1.00
[14:22:52] 🔍 Analyzing whale trade...
→ Fetching trader ranking... Rank #47 (PnL: $284,521)
→ Fetching trader history... 156 recent trades
→ Fetching event positions... 3 related markets
→ Fetching market top traders... 5 bulls, 5 bears
[14:22:53] 🤖 LLM Analysis started (model: gemini-3-flash-preview)
→ Tool call: search_web("MegaETH token launch date 2026")
→ Tool call: search_twitter("MegaETH $METH token TGE")
→ Tool call: get_protocol_tvl("megaeth")
→ Tool call: get_contract_info("0x4f9b...2a1c")
→ Tool call: search_telegram("MegaETH launch")
[14:23:07] ✅ Analysis complete
Information Asymmetry Score: 0.72 (HIGH)
Recommendation: BUY Yes | Confidence: 0.75
Report saved: reports/20260415/20260415_142307_BUY_92336USD_MegaETH_token.md
[14:23:07] 📧 Email alert sent (IAS >= 60%)
[14:23:15] 🐋 WHALE DETECTED on "US forces enter Iran by April 30"
BUY Yes @ 0.3100 | $46,500 USDC | Wallet: 0x2c8e...a4d2
Anomaly Score: 0.71/1.00
[14:23:16] 🔍 Analyzing whale trade...
...
[14:37:41] 🔄 Refreshing trending markets... Found 48 active markets (2 resolved)
[15:00:00] 📊 Resolution check: 3 markets resolved since last check
→ "EdgeX FDV above 400M" resolved YES — Signal was CORRECT (ROI: +142%)
→ "Will Trump talk to Rutte" resolved NO — Signal was INCORRECT
→ "Over 9M committed to P2P" resolved YES — Signal was CORRECT (ROI: +67%)
pyproject.toml
+49
View File
@@ -0,0 +1,49 @@
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "polymarket-whale-watcher"
version = "0.1.0"
description = "AI-powered whale trade detection and analysis bot for Polymarket"
readme = "README.md"
requires-python = ">=3.10"
license = {text = "MIT"}
dependencies = [
"httpx>=0.27.0",
"websockets>=13.0",
"requests>=2.28.0",
"pydantic>=2.0.0",
"pydantic-settings>=2.0.0",
"python-dotenv>=1.0.0",
"openai>=1.0.0",
"rich>=13.0.0",
"typer>=0.9.0",
"duckduckgo-search>=7.0.0",
"telethon>=1.36.0",
"fastapi>=0.100.0",
"uvicorn>=0.20.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0.0",
"pytest-asyncio>=0.23.0",
"ruff>=0.1.0",
"mypy>=1.0.0",
]
[project.scripts]
whale-watcher = "src.main:main"
[tool.ruff]
line-length = 100
target-version = "py310"
[tool.ruff.lint]
select = ["E", "F", "I", "W"]
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
reports/20260505/20260505_000000_BUY_9599USD_US_x_Iran_diplomatic_meeting_by_June_30_2026.md
+143
View File
@@ -0,0 +1,143 @@
======================================================================
# Whale Trade Analysis Report
======================================================================
**Generated at**: 2026-05-04 16:00:00 UTC
## Trade Summary
| Field | Details |
|-------|---------|
| **Market** | US x Iran diplomatic meeting by June 30, 2026? |
| **Trade Size** | $9,600.00 USDC |
| **Direction** | BUY Yes Token (Bullish) |
| **Trade Price** | 0.7142 (71.4%) |
| **Current Odds** | Yes: 69.5% | No: 30.5% |
| **Trade Time** | 2026-05-04 23:41:12 |
| **Trader Rank** | #1733 (PnL: $84,695.33) |
======================================================================
I have comprehensive data. Let me now conduct a thorough analysis.
---
## Full Analysis Report
### Step 2: Trade Signal Analysis
**Trader Profile Assessment:**
- **Rank #1733** — mid-tier experienced trader, not top-tier elite
- **PnL: $84,695** — solid positive track record across ~$4M lifetime volume
- **Avg trade size: $10,018** — this $9,600 trade is perfectly in-line with his normal size, not an unusual outlier
- **Large trade ratio: 66%** — consistent whale behavior, large trades are routine for this trader
- **Domain expertise: Strong Iran/geopolitics focus** — recent trades include Iranian regime fall (April/June), US-Iran meetings (April/June), Russia-Ukraine ceasefire, Ukraine Donbas. This is clearly a geopolitics specialist with deep focus on Iran specifically.
**Trade Timing:** The trade was made at 23:41 UTC on May 4, 2026. Given the active diplomatic situation, this is a routine positioning trade, not an oddly-timed pre-announcement bet.
---
### Step 3: Event-Related Position Analysis — 🚨 KEY FINDING
The whale has a **losing position** in the earlier "US x Iran diplomatic meeting by May 15, 2026?" market:
- Holding Yes tokens @ avg 65.29%, now priced at **15.50%**
- Cost basis: $8,534 | Current value: $2,026 | **PnL: -$6,508 (LOSS)**
This is critical context:
1. The trader **bet heavily on a May 15 meeting** and that bet is **nearly dead** (15.5% odds)
2. The June 30 buy ($9,600) appears to be a **portfolio roll-forward** — doubling down on the same thesis with a later deadline to recoup losses
3. This is NOT a fresh, confident insider bet — it's a **loss recovery / thesis extension trade**
4. The pattern suggests the trader is sticking to a bullish Iran-diplomacy thesis despite being wrong on the nearer-term deadline
---
### Step 4: Market Long/Short Analysis
**Bulls (Yes side):**
- **anoin123** (Rank #2,682,942, PnL **-$5.54M**) — massive loser, leads the Yes side with $39K. This is a **red flag** — the biggest Yes holder is a chronic loser
- **ArmageddonRewardsBilly** (Rank #278, PnL $525K) — elite trader, $11K on Yes. This is the strongest bull signal
- **cezarn132** (the whale) — Rank #1733, $9.3K Yes
**Bears (No side):**
- **aaron107** (Rank #48,390) — low-ranked, leads No
- **yungstalin** (Rank #1,969, PnL $73,759) — solid mid-tier trader on No side
**Smart Money Consensus:** Mixed. One elite trader (Rank #278) is bullish, but the biggest Yes position belongs to a massive loser. The No side lacks elite concentration too. No clear smart money consensus signal in either direction.
---
### Step 5: Information Gap Analysis
**Public Information Summary:**
- There was already an indirect round of talks in Islamabad, Pakistan in late April 2026 (confirmed by Wikipedia)
- US envoy Steve Witkoff stated Washington is "in conversation" with Iran
- Iran's FM Araghchi has made public proposals for a 3-phase deal
- Market odds for June 30 meeting are ~69.5% — already well-priced for the optimistic scenario
- However, the **May 15 market is at just 15.5%**, showing markets believe a near-term meeting is unlikely
- Twitter data shows @faststocknewss noting "POLYMARKET ODDS OF A US-IRAN DIPLOMATIC MEETING ARE FADING" — the June 30 market was previously at 80%+ and has dropped to 66-69%
**Key Information Assessment:**
- All available information about Iran-US diplomacy is extensively covered in public media
- The market at 69.5% appears to **reasonably price** the known information
- No evidence of non-public information advantage — this appears to be public-information-based analysis
- The trade is consistent with this trader's established Iran geopolitics thesis
---
### Step 6: Historical Pattern
This trader has a **clear and consistent pattern**:
- Bought "Iranian regime fall by April 30" at 0.86 → **SOLD at 0.98** (big win ✅)
- Bought "Iranian regime fall by June 30" at 0.76 and 0.74 → Still holding
- Bet on "US x Iran diplomatic meeting by April 25/May 15" → **LOSING** at 15.5%
The pattern suggests this trader uses a "timeline ladder" approach — betting on Iran-related events across multiple deadlines. When early deadlines miss, they roll to later ones. This reduces the information asymmetry signal significantly.
---
## Step 7: Information Asymmetry Assessment
```json
{
"information_asymmetry_score": 0.32,
"trader_credibility": "MEDIUM",
"reasoning": "Trader cezarn132 (Rank #1733, PnL $84K) is a confirmed geopolitics specialist with deep Iran-focused trading history. However, several factors suppress the information asymmetry score: (1) The $9,600 trade is exactly in-line with this trader's average trade size ($10K), making it routine rather than an exceptional conviction bet. (2) The trader currently holds a heavily losing position in the nearer 'May 15' meeting market (down $6,508), and this June 30 buy appears to be a thesis roll-forward/loss recovery strategy rather than a fresh insider signal. (3) All supporting information (Islamabad talks, Witkoff statements, Iranian proposals) is extensively public and already priced into the ~69.5% market odds. (4) The market was previously at 80%+ and has been declining — this is a contrarian hold, not a momentum play. (5) The smart money picture is mixed — the largest Yes holder is a chronic losing trader. (6) This is a large geopolitical market with many participants, where public analysis rather than non-public information drives most trades. The trader demonstrates domain expertise and pattern consistency, but no evidence of non-public information access.",
"insider_evidence": "No insider evidence found. All information supporting the bullish Iran diplomacy view (Islamabad round 1 talks, Witkoff statements, Iranian FM proposals, Oman mediation) is widely reported in mainstream media. The trade is most consistent with a thesis-continuation/loss-recovery strategy following a losing May 15 position, rather than a fresh signal based on non-public information."
}
```
---
### Summary Table
| Factor | Assessment | Signal Strength |
|--------|-----------|-----------------|
| Trader Rank/PnL | Rank #1733, $84K PnL | Moderate ✅ |
| Trade Size vs. Normal | ~$9.6K vs. avg $10K | Routine — neutral ➖ |
| Related Position | Heavy loser on May 15 meeting (-$6.5K) | Suppresses signal ❌ |
| Domain Expertise | Iran geopolitics specialist | Supportive ✅ |
| Public Info Coverage | Extensive public news on US-Iran talks | Reduces asymmetry ❌ |
| Market Pricing | Already at 69.5% — well-priced | Reduces asymmetry ❌ |
| Smart Money Bulls | Rank #278 also long | Modest support ✅ |
| Biggest Yes Holder | Rank #2.6M, -$5.5M PnL | Negative signal ❌ |
| **Overall** | **Likely thesis continuation, not insider signal** | **Low-Medium** |
**Recommended Action: HOLD/PASS** — This trade reflects a geopolitics analyst rolling forward a losing thesis rather than a trader acting on non-public information. The market is already fairly priced at ~69.5% based on extensive public information about ongoing US-Iran diplomatic channels.
======================================================================
## Information Asymmetry Assessment
======================================================================
| Field | Assessment |
|-------|------------|
| **Information Asymmetry** | Low Information Asymmetry (32%) |
| **Trader Credibility** | Medium Credibility (#1733) |
**Key Evidence**: No insider evidence found. All information supporting the bullish Iran diplomacy view (Islamabad round 1 talks, Witkoff statements, Iranian FM proposals, Oman mediation) is widely reported in mainstream media. The trade is most consistent with a thesis-continuation/loss-recovery strategy following a losing May 15 position, rather than a fresh signal based on non-public information.
**Reasoning**: Trader cezarn132 (Rank #1733, PnL $84K) is a confirmed geopolitics specialist with deep Iran-focused trading history. However, several factors suppress the information asymmetry score: (1) The $9,600 trade is exactly in-line with this trader's average trade size ($10K), making it routine rather than an exceptional conviction bet. (2) The trader currently holds a heavily losing position in the nearer 'May 15' meeting market (down $6,508), and this June 30 buy appears to be a thesis roll-forward/loss recovery strategy rather than a fresh insider signal. (3) All supporting information (Islamabad talks, Witkoff statements, Iranian proposals) is extensively public and already priced into the ~69.5% market odds. (4) The market was previously at 80%+ and has been declining — this is a contrarian hold, not a momentum play. (5) The smart money picture is mixed — the largest Yes holder is a chronic losing trader. (6) This is a large geopolitical market with many participants, where public analysis rather than non-public information drives most trades. The trader demonstrates domain expertise and pattern consistency, but no evidence of non-public information access.
======================================================================
Disclaimer: This report is AI-generated for informational purposes only and does not constitute investment advice.
======================================================================
scripts/telegram_auth.py
+42
View File
@@ -0,0 +1,42 @@
"""
One-time script to generate a Telegram session string.
Usage:
python scripts/telegram_auth.py
You'll need:
1. Go to https://my.telegram.org and create an application
2. Get your api_id and api_hash
3. Run this script and enter your phone number
4. Enter the verification code sent to your Telegram
5. Copy the session string into your .env file as TELEGRAM_SESSION_STRING
"""
import asyncio
from telethon import TelegramClient
from telethon.sessions import StringSession
async def main():
print("=== Telegram Session Generator ===\n")
print("Get api_id and api_hash from https://my.telegram.org\n")
api_id = input("Enter api_id: ").strip()
api_hash = input("Enter api_hash: ").strip()
client = TelegramClient(StringSession(), int(api_id), api_hash)
await client.start()
session_string = client.session.save()
print(f"\n{'='*60}")
print("Your session string (add to .env):\n")
print(f"TELEGRAM_SESSION_STRING={session_string}")
print(f"\n{'='*60}")
print("Keep this string secret! It provides access to your account.")
await client.disconnect()
if __name__ == "__main__":
asyncio.run(main())
setup.sh
+220
View File
@@ -0,0 +1,220 @@
#!/usr/bin/env bash
# Polymarket Whale Watcher - One-Click Setup
# Usage: chmod +x setup.sh && ./setup.sh
set -e
CYAN='\033[0;36m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
DIM='\033[2m'
BOLD='\033[1m'
NC='\033[0m'
echo -e "${CYAN}"
echo "╭──────────────────────────────────────────────────────────╮"
echo "│ 🐋 Polymarket Whale Watcher Setup │"
echo "╰──────────────────────────────────────────────────────────╯"
echo -e "${NC}"
# ============================================================
# Step 1: Check Python
# ============================================================
echo -e "${CYAN}[1/4]${NC} Checking Python version..."
if ! command -v python3 &> /dev/null; then
echo -e "${RED}Error: Python 3 is required but not installed.${NC}"
echo " Install from https://www.python.org/downloads/"
exit 1
fi
PYTHON_VERSION=$(python3 -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')
PYTHON_MAJOR=$(echo "$PYTHON_VERSION" | cut -d. -f1)
PYTHON_MINOR=$(echo "$PYTHON_VERSION" | cut -d. -f2)
if [ "$PYTHON_MAJOR" -lt 3 ] || ([ "$PYTHON_MAJOR" -eq 3 ] && [ "$PYTHON_MINOR" -lt 10 ]); then
echo -e "${RED}Error: Python 3.10+ required, found $PYTHON_VERSION${NC}"
exit 1
fi
echo -e " ${GREEN}✓${NC} Python $PYTHON_VERSION"
# ============================================================
# Step 2: Virtual environment
# ============================================================
echo -e "${CYAN}[2/4]${NC} Setting up virtual environment..."
if [ ! -d "venv" ]; then
python3 -m venv venv
echo -e " ${GREEN}✓${NC} Virtual environment created"
else
echo -e " ${GREEN}✓${NC} Virtual environment already exists"
fi
# ============================================================
# Step 3: Install dependencies
# ============================================================
echo -e "${CYAN}[3/4]${NC} Installing dependencies..."
source venv/bin/activate
pip install --upgrade pip -q
pip install -r requirements.txt -q
echo -e " ${GREEN}✓${NC} Dependencies installed"
# Create data directories
mkdir -p data reports daily_briefings
# ============================================================
# Step 4: API Key Configuration
# ============================================================
echo -e "${CYAN}[4/4]${NC} Configuring API keys..."
echo ""
if [ -f ".env" ]; then
echo -e " ${YELLOW}Found existing .env file.${NC}"
read -p " Overwrite and reconfigure? [y/N] " overwrite
if [[ ! "$overwrite" =~ ^[Yy]$ ]]; then
echo -e " ${GREEN}✓${NC} Keeping existing .env"
echo ""
echo -e "${GREEN}Setup complete! Run:${NC}"
echo " source venv/bin/activate"
echo " python -m src.main run"
exit 0
fi
fi
cp .env.example .env
# Helper function: prompt for a key and write to .env
set_key() {
local var_name="$1"
local prompt_text="$2"
local url="$3"
local required="$4"
echo ""
if [ "$required" = "required" ]; then
echo -e " ${BOLD}${var_name}${NC} ${RED}(required)${NC}"
else
echo -e " ${BOLD}${var_name}${NC} ${DIM}(optional, press Enter to skip)${NC}"
fi
echo -e " ${DIM}$prompt_text${NC}"
echo -e " ${DIM}→ $url${NC}"
while true; do
read -p " Key: " key_value
if [ -z "$key_value" ]; then
if [ "$required" = "required" ]; then
echo -e " ${RED}This key is required. Please enter a value.${NC}"
continue
else
echo -e " ${DIM}Skipped${NC}"
return
fi
fi
break
done
# Replace the placeholder in .env
if grep -q "^${var_name}=" .env; then
sed -i.bak "s|^${var_name}=.*|${var_name}=${key_value}|" .env
elif grep -q "^# ${var_name}=" .env; then
sed -i.bak "s|^# ${var_name}=.*|${var_name}=${key_value}|" .env
else
echo "${var_name}=${key_value}" >> .env
fi
rm -f .env.bak
echo -e " ${GREEN}✓ Saved${NC}"
}
echo -e "${CYAN}╭──────────────────────────────────────────────────────────╮${NC}"
echo -e "${CYAN}│ API Key Configuration │${NC}"
echo -e "${CYAN}│ │${NC}"
echo -e "${CYAN}│ The more keys you add, the better the analysis. │${NC}"
echo -e "${CYAN}│ Only LLM_API_KEY is required. Others are optional but │${NC}"
echo -e "${CYAN}│ strongly recommended for full coverage. │${NC}"
echo -e "${CYAN}╰──────────────────────────────────────────────────────────╯${NC}"
# --- Required ---
echo ""
echo -e "${YELLOW}━━━ Required (LLM Engine) ━━━${NC}"
set_key "LLM_API_KEY" \
"Powers the LLM analysis (OpenAI-compatible)" \
"https://platform.openai.com/api-keys" \
"required"
# --- Core Search ---
echo ""
echo -e "${YELLOW}━━━ Web Search (strongly recommended) ━━━${NC}"
echo -e "${DIM} Without these, LLM falls back to DuckDuckGo (lower quality)${NC}"
set_key "TAVILY_API_KEY" \
"Best web search quality, 1000 free searches/month" \
"https://app.tavily.com/home"
set_key "SERPER_API_KEY" \
"Google search fallback, 2500 free searches" \
"https://serper.dev"
# --- Social Sentiment ---
echo ""
echo -e "${YELLOW}━━━ Social Sentiment ━━━${NC}"
set_key "TWITTER_API_KEY" \
"Twitter/X sentiment & breaking news search" \
"https://developer.x.com/en/portal/dashboard"
# --- Financial Data ---
echo ""
echo -e "${YELLOW}━━━ Financial & On-Chain Data ━━━${NC}"
set_key "POLYGON_API_KEY" \
"Stock, ETF, forex, commodities data (free tier)" \
"https://polygon.io/dashboard/signup"
set_key "FRED_API_KEY" \
"US economic indicators (free, instant approval)" \
"https://fred.stlouisfed.org/docs/api/api_key.html"
set_key "ETHERSCAN_API_KEY" \
"On-chain wallet & contract analysis (free tier)" \
"https://etherscan.io/myapikey"
set_key "CONGRESS_API_KEY" \
"US legislation & bills tracking (free)" \
"https://api.congress.gov/sign-up/"
# --- Summary ---
echo ""
echo ""
# Count configured keys
configured=0
total=8
for var in LLM_API_KEY TAVILY_API_KEY SERPER_API_KEY TWITTER_API_KEY POLYGON_API_KEY FRED_API_KEY ETHERSCAN_API_KEY CONGRESS_API_KEY; do
val=$(grep "^${var}=" .env 2>/dev/null | cut -d= -f2-)
if [ -n "$val" ] && [ "$val" != "your_api_key_here" ]; then
configured=$((configured + 1))
fi
done
echo -e "${GREEN}╭──────────────────────────────────────────────────────────╮${NC}"
echo -e "${GREEN}│ ✅ Setup Complete! │${NC}"
echo -e "${GREEN}│ │${NC}"
echo -e "${GREEN}│ API Keys configured: ${configured}/${total} │${NC}"
echo -e "${GREEN}╰──────────────────────────────────────────────────────────╯${NC}"
if [ "$configured" -lt 4 ]; then
echo ""
echo -e " ${YELLOW}Tip: More API keys = better analysis coverage.${NC}"
echo -e " ${YELLOW}You can edit .env anytime to add more keys later.${NC}"
fi
echo ""
echo " Start monitoring:"
echo -e " ${BOLD}source venv/bin/activate${NC}"
echo -e " ${BOLD}python -m src.main run${NC}"
echo ""
echo " Other commands:"
echo " python -m src.main check-markets # View trending markets"
echo " python -m src.main dashboard # Web dashboard"
echo " python -m src.main briefing --today # Daily briefing"
echo ""
src/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Polymarket Whale Watcher - AI-powered whale trade detection and analysis."""
src/config/__init__.py
+4
View File
@@ -0,0 +1,4 @@
"""Configuration module."""
from .settings import Settings, get_settings
__all__ = ["Settings", "get_settings"]
src/config/settings.py
+95
View File
@@ -0,0 +1,95 @@
"""Application settings and configuration."""
import os
from functools import lru_cache
from typing import Optional
from dotenv import load_dotenv
from pydantic import Field
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
"""Application settings loaded from environment variables."""
# LLM API (OpenAI-compatible proxy)
llm_api_key: str = Field(default="", alias="LLM_API_KEY")
llm_base_url: str = Field(default="https://generativelanguage.googleapis.com/v1beta/openai/", alias="LLM_BASE_URL")
# Twitter API (for social sentiment search)
twitter_api_key: str = Field(default="", alias="TWITTER_API_KEY")
# Tavily API (for web search, replaces Google Search)
tavily_api_key: str = Field(default="", alias="TAVILY_API_KEY")
# Serper API (web search fallback)
serper_api_key: str = Field(default="", alias="SERPER_API_KEY")
# FRED API (macroeconomic data)
fred_api_key: str = Field(default="", alias="FRED_API_KEY")
# Polygon.io API (stocks, forex, commodities)
polygon_api_key: str = Field(default="", alias="POLYGON_API_KEY")
# Congress.gov API (U.S. legislation)
congress_api_key: str = Field(default="", alias="CONGRESS_API_KEY")
# Etherscan API (on-chain data)
etherscan_api_key: str = Field(default="", alias="ETHERSCAN_API_KEY")
# Telegram API (crypto channel monitoring)
telegram_api_id: str = Field(default="", alias="TELEGRAM_API_ID")
telegram_api_hash: str = Field(default="", alias="TELEGRAM_API_HASH")
telegram_session_string: str = Field(default="", alias="TELEGRAM_SESSION_STRING")
telegram_channels: str = Field(default="", alias="TELEGRAM_CHANNELS")
# MongoDB
mongodb_uri: str = Field(default="mongodb://localhost:27017/whale_watcher", alias="MONGODB_URI")
# SQLite database
db_path: str = Field(default="data/signals.db", alias="DB_PATH")
# HTTP Proxy (for users behind firewalls)
http_proxy: str = Field(default="", alias="HTTP_PROXY")
# Whale Detection Settings
min_trade_size_usd: float = Field(default=1000.0, alias="MIN_TRADE_SIZE_USD")
min_price: float = Field(default=0.10, alias="MIN_PRICE")
max_price: float = Field(default=0.90, alias="MAX_PRICE")
# Monitoring Settings
trending_markets_limit: int = Field(default=50, alias="TRENDING_MARKETS_LIMIT")
# Market coverage (volume thresholds for monitoring list)
full_market_scan: bool = Field(default=True, alias="FULL_MARKET_SCAN")
tier1_volume_min: float = Field(default=500_000, alias="TIER1_VOLUME_MIN")
tier2_volume_min: float = Field(default=10_000, alias="TIER2_VOLUME_MIN")
tier3_volume_min: float = Field(default=1_000, alias="TIER3_VOLUME_MIN")
# LLM Settings
llm_model: str = Field(default="gemini-3.1-pro-preview", alias="LLM_MODEL")
llm_temperature: float = Field(default=0.0, alias="LLM_TEMPERATURE")
# Email notification
email_smtp_server: str = Field(default="smtp.qq.com", alias="EMAIL_SMTP_SERVER")
email_smtp_port: int = Field(default=465, alias="EMAIL_SMTP_PORT")
email_sender: str = Field(default="", alias="EMAIL_SENDER")
email_password: str = Field(default="", alias="EMAIL_PASSWORD")
email_recipient: str = Field(default="1253608463@qq.com,lyk@sii.edu.cn,1286874010@qq.com,tianhao.alex.huang@gmail.com", alias="EMAIL_RECIPIENT")
email_enabled: bool = Field(default=False, alias="EMAIL_ENABLED")
# Logging
log_level: str = Field(default="INFO", alias="LOG_LEVEL")
class Config:
env_file = ".env"
env_file_encoding = "utf-8"
extra = "ignore"
@lru_cache
def get_settings() -> Settings:
"""Get cached settings instance."""
load_dotenv()
return Settings()
src/dashboard.py
+224
View File
@@ -0,0 +1,224 @@
"""FastAPI dashboard for signal performance tracking."""
import logging
from typing import Optional
from fastapi import FastAPI, Query
from fastapi.responses import HTMLResponse
from src.config import get_settings
from src.db.database import SignalDatabase
from src.services.stats_engine import StatsEngine
logger = logging.getLogger(__name__)
app = FastAPI(title="Polymarket Whale Watcher - Signal Dashboard")
def _get_db() -> SignalDatabase:
settings = get_settings()
return SignalDatabase(settings.db_path)
@app.get("/api/stats")
def api_stats():
"""Overall signal performance statistics."""
db = _get_db()
return db.get_stats()
@app.get("/api/stats/tiers")
def api_stats_tiers():
"""Signal stats by information_asymmetry_score tier."""
db = _get_db()
return db.get_stats_by_tier()
@app.get("/api/signals")
def api_signals(
limit: int = Query(50, ge=1, le=500),
offset: int = Query(0, ge=0),
):
"""Paginated signal list (newest first)."""
db = _get_db()
signals = db.get_all_signals(limit=limit, offset=offset)
return [s.model_dump(mode="json") for s in signals]
@app.get("/api/signals/best-worst")
def api_best_worst(n: int = Query(5, ge=1, le=20)):
"""Best and worst signals by theoretical ROI."""
db = _get_db()
result = db.get_best_worst(n=n)
return {
"best": [s.model_dump(mode="json") for s in result["best"]],
"worst": [s.model_dump(mode="json") for s in result["worst"]],
}
@app.get("/", response_class=HTMLResponse)
def dashboard_page():
"""HTML dashboard page."""
return HTML_TEMPLATE
HTML_TEMPLATE = """<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Polymarket Whale Watcher - Signal Dashboard</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #0f1117; color: #e0e0e0; padding: 20px; }
h1 { color: #fff; margin-bottom: 8px; font-size: 1.8em; }
h2 { color: #a0a8c0; margin: 24px 0 12px; font-size: 1.2em; }
.subtitle { color: #666; margin-bottom: 24px; }
.stats-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(160px, 1fr)); gap: 12px; margin-bottom: 24px; }
.stat-card { background: #1a1d28; border-radius: 8px; padding: 16px; text-align: center; }
.stat-value { font-size: 1.8em; font-weight: bold; color: #4fc3f7; }
.stat-value.green { color: #66bb6a; }
.stat-value.red { color: #ef5350; }
.stat-label { color: #888; font-size: 0.85em; margin-top: 4px; }
table { width: 100%; border-collapse: collapse; margin-bottom: 24px; }
th { background: #1a1d28; color: #a0a8c0; text-align: left; padding: 10px 12px; font-weight: 600; font-size: 0.85em; }
td { padding: 10px 12px; border-bottom: 1px solid #222; font-size: 0.9em; }
tr:hover { background: #1a1d28; }
.correct { color: #66bb6a; }
.incorrect { color: #ef5350; }
.pending { color: #888; }
.badge { display: inline-block; padding: 2px 8px; border-radius: 4px; font-size: 0.8em; font-weight: bold; }
.badge-high { background: #ef535033; color: #ef5350; }
.badge-med { background: #ffb74d33; color: #ffb74d; }
.badge-low { background: #66bb6a33; color: #66bb6a; }
.tier-table th, .tier-table td { text-align: center; }
#loading { color: #666; text-align: center; padding: 40px; }
</style>
</head>
<body>
<h1>Polymarket Whale Watcher</h1>
<p class="subtitle">Signal Performance Dashboard</p>
<p id="update-info" style="color:#666;font-size:0.85em;margin-bottom:12px"></p>
<div id="loading">Loading...</div>
<div id="content" style="display:none">
<div class="stats-grid" id="stats-grid"></div>
<h2>Stats by Likelihood Tier</h2>
<table class="tier-table" id="tier-table">
<thead><tr><th>Tier</th><th>Total</th><th>Resolved</th><th>Correct</th><th>Win Rate</th><th>Avg ROI</th></tr></thead>
<tbody></tbody>
</table>
<h2>Best Signals</h2>
<table id="best-table">
<thead><tr><th>Market</th><th>Side</th><th>Price</th><th>Size</th><th>Likelihood</th><th>Outcome</th><th>ROI</th></tr></thead>
<tbody></tbody>
</table>
<h2>Worst Signals</h2>
<table id="worst-table">
<thead><tr><th>Market</th><th>Side</th><th>Price</th><th>Size</th><th>Likelihood</th><th>Outcome</th><th>ROI</th></tr></thead>
<tbody></tbody>
</table>
<h2>Recent Signals</h2>
<table id="signals-table">
<thead><tr><th>Detected</th><th>Market</th><th>Side</th><th>Price</th><th>Size</th><th>Likelihood</th><th>Result</th><th>ROI</th></tr></thead>
<tbody></tbody>
</table>
</div>
<script>
const fmt = (v, d=1) => v !== null && v !== undefined ? (v*100).toFixed(d)+'%' : 'N/A';
const fmtRoi = v => v !== null && v !== undefined ? (v >= 0 ? '+' : '') + (v*100).toFixed(1)+'%' : 'Pending';
const fmtUsd = v => '$' + Number(v).toLocaleString('en-US', {maximumFractionDigits: 0});
const likeBadge = v => {
if (v >= 0.8) return `<span class="badge badge-high">${fmt(v,0)}</span>`;
if (v >= 0.6) return `<span class="badge badge-med">${fmt(v,0)}</span>`;
return `<span class="badge badge-low">${fmt(v,0)}</span>`;
};
const resultClass = s => {
if (s.signal_correct === true) return 'correct';
if (s.signal_correct === false) return 'incorrect';
return 'pending';
};
const resultText = s => {
if (!s.market_resolved) return 'Pending';
return s.signal_correct ? 'Correct' : 'Incorrect';
};
function signalRow(s, showDate=true) {
const cols = [];
if (showDate) cols.push(`<td>${(s.detected_at||'').slice(0,16)}</td>`);
cols.push(`<td>${(s.market_question||'').slice(0,60)}</td>`);
cols.push(`<td>${s.trade_side} ${s.trade_outcome}</td>`);
cols.push(`<td>${Number(s.trade_price).toFixed(4)}</td>`);
cols.push(`<td>${fmtUsd(s.trade_size_usd)}</td>`);
cols.push(`<td>${likeBadge(s.information_asymmetry_score)}</td>`);
if (showDate) cols.push(`<td class="${resultClass(s)}">${resultText(s)}</td>`);
else cols.push(`<td>${s.resolved_outcome||'Pending'}</td>`);
cols.push(`<td class="${resultClass(s)}">${fmtRoi(s.theoretical_roi)}</td>`);
return '<tr>' + cols.join('') + '</tr>';
}
async function load() {
try {
const [statsRes, tiersRes, bwRes, sigRes] = await Promise.all([
fetch('/api/stats'), fetch('/api/stats/tiers'),
fetch('/api/signals/best-worst?n=5'), fetch('/api/signals?limit=100')
]);
const stats = await statsRes.json();
const tiers = await tiersRes.json();
const bw = await bwRes.json();
const signals = await sigRes.json();
const info = document.getElementById('update-info');
const now = new Date().toLocaleTimeString();
if (stats.last_updated) {
info.textContent = `数据最后更新: ${stats.last_updated.replace('T', ' ').slice(0, 19)} | 页面刷新: ${now} | 每30秒自动刷新`;
} else {
info.textContent = `暂无信号数据，请运行 python -m src.main run 开始采集 | 页面刷新: ${now}`;
info.style.color = '#ffb74d';
}
// Stats cards
const grid = document.getElementById('stats-grid');
const cards = [
['Total Signals', stats.total_signals, ''],
['Resolved', stats.resolved, ''],
['Win Rate', fmt(stats.win_rate), stats.win_rate >= 0.5 ? 'green' : 'red'],
['Avg ROI', fmtRoi(stats.avg_roi), stats.avg_roi >= 0 ? 'green' : 'red'],
['Correct', stats.correct, 'green'],
['Total PnL', (stats.total_theoretical_pnl >= 0 ? '+' : '') + Number(stats.total_theoretical_pnl).toFixed(2) + 'x', stats.total_theoretical_pnl >= 0 ? 'green' : 'red'],
];
grid.innerHTML = cards.map(([label, value, cls]) =>
`<div class="stat-card"><div class="stat-value ${cls}">${value}</div><div class="stat-label">${label}</div></div>`
).join('');
// Tier table
const tierBody = document.querySelector('#tier-table tbody');
tierBody.innerHTML = tiers.map(t =>
`<tr><td>${t.tier}</td><td>${t.total}</td><td>${t.resolved}</td><td>${t.correct}</td><td>${t.resolved > 0 ? fmt(t.win_rate) : 'N/A'}</td><td>${t.resolved > 0 ? fmtRoi(t.avg_roi) : 'N/A'}</td></tr>`
).join('');
// Best/worst
document.querySelector('#best-table tbody').innerHTML = bw.best.map(s => signalRow(s, false)).join('');
document.querySelector('#worst-table tbody').innerHTML = bw.worst.map(s => signalRow(s, false)).join('');
// All signals
document.querySelector('#signals-table tbody').innerHTML = signals.map(s => signalRow(s)).join('');
document.getElementById('loading').style.display = 'none';
document.getElementById('content').style.display = 'block';
} catch(e) {
document.getElementById('loading').textContent = 'Error loading data: ' + e.message;
}
}
load();
setInterval(load, 30000);
</script>
</body>
</html>"""
src/db/__init__.py
+4
View File
@@ -0,0 +1,4 @@
"""Database module for signal storage and tracking."""
from src.db.database import SignalDatabase
__all__ = ["SignalDatabase"]
src/db/database.py
+404
View File
@@ -0,0 +1,404 @@
"""SQLite database for anomaly signal storage and resolution tracking."""
import json
import logging
import sqlite3
from datetime import datetime
from pathlib import Path
from typing import List, Optional
from src.models.anomaly_signal import AnomalySignal
from src.models.trade import TraderRanking, TraderHistory
logger = logging.getLogger(__name__)
class SignalDatabase:
"""SQLite-backed storage for anomaly signals with resolution tracking."""
def __init__(self, db_path: str = "data/signals.db"):
self.db_path = Path(db_path)
self.db_path.parent.mkdir(parents=True, exist_ok=True)
self._init_db()
def _get_conn(self) -> sqlite3.Connection:
conn = sqlite3.connect(str(self.db_path))
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA journal_mode=WAL")
return conn
def _init_db(self):
with self._get_conn() as conn:
# Migrate: rename old column if it exists
try:
conn.execute(
"ALTER TABLE signals RENAME COLUMN insider_trading_likelihood TO information_asymmetry_score"
)
logger.info("Migrated column: insider_trading_likelihood -> information_asymmetry_score")
except Exception:
pass # Column already renamed or table doesn't exist yet
conn.execute("""
CREATE TABLE IF NOT EXISTS signals (
id TEXT,
market_id TEXT NOT NULL,
market_question TEXT NOT NULL,
market_slug TEXT,
condition_id TEXT,
transaction_hash TEXT UNIQUE NOT NULL,
trade_timestamp INTEGER NOT NULL,
trade_side TEXT NOT NULL,
trade_price REAL NOT NULL,
trade_size_usd REAL NOT NULL,
trade_outcome TEXT NOT NULL,
trader_wallet TEXT,
trader_ranking_json TEXT,
trader_history_json TEXT,
information_asymmetry_score REAL NOT NULL DEFAULT 0.0,
reasoning TEXT DEFAULT '',
insider_evidence TEXT DEFAULT '',
detected_at TEXT NOT NULL,
market_resolved INTEGER DEFAULT 0,
market_resolved_at TEXT,
resolved_outcome TEXT,
signal_correct INTEGER,
theoretical_roi REAL
)
""")
conn.execute("""
CREATE INDEX IF NOT EXISTS idx_signals_market_id
ON signals(market_id)
""")
conn.execute("""
CREATE INDEX IF NOT EXISTS idx_signals_market_resolved
ON signals(market_resolved)
""")
conn.execute("""
CREATE INDEX IF NOT EXISTS idx_signals_likelihood
ON signals(information_asymmetry_score DESC)
""")
def insert_signal(self, signal: AnomalySignal) -> bool:
"""Insert a signal, deduplicating by transaction_hash. Returns True if inserted."""
try:
with self._get_conn() as conn:
conn.execute("""
INSERT OR IGNORE INTO signals (
id, market_id, market_question, market_slug, condition_id,
transaction_hash, trade_timestamp, trade_side, trade_price,
trade_size_usd, trade_outcome, trader_wallet,
trader_ranking_json, trader_history_json,
information_asymmetry_score, reasoning, insider_evidence,
detected_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""", (
signal.id,
signal.market_id,
signal.market_question,
signal.market_slug,
signal.condition_id,
signal.transaction_hash,
signal.trade_timestamp,
signal.trade_side,
signal.trade_price,
signal.trade_size_usd,
signal.trade_outcome,
signal.trader_wallet,
signal.trader_ranking.model_dump_json() if signal.trader_ranking else None,
signal.trader_history.model_dump_json() if signal.trader_history else None,
signal.information_asymmetry_score,
signal.reasoning,
signal.insider_evidence,
signal.detected_at.isoformat(),
))
return conn.total_changes > 0
except sqlite3.IntegrityError:
logger.debug(f"Signal already exists: {signal.transaction_hash}")
return False
except Exception as e:
logger.error(f"Failed to insert signal: {e}")
return False
def _row_to_signal(self, row: sqlite3.Row) -> AnomalySignal:
"""Convert a database row to an AnomalySignal."""
trader_ranking = None
if row["trader_ranking_json"]:
try:
trader_ranking = TraderRanking.model_validate_json(row["trader_ranking_json"])
except Exception:
pass
trader_history = None
if row["trader_history_json"]:
try:
trader_history = TraderHistory.model_validate_json(row["trader_history_json"])
except Exception:
pass
return AnomalySignal(
id=row["id"] or "",
market_id=row["market_id"],
market_question=row["market_question"],
market_slug=row["market_slug"],
condition_id=row["condition_id"],
transaction_hash=row["transaction_hash"],
trade_timestamp=row["trade_timestamp"],
trade_side=row["trade_side"],
trade_price=row["trade_price"],
trade_size_usd=row["trade_size_usd"],
trade_outcome=row["trade_outcome"],
trader_wallet=row["trader_wallet"],
trader_ranking=trader_ranking,
trader_history=trader_history,
information_asymmetry_score=row["information_asymmetry_score"],
reasoning=row["reasoning"] or "",
insider_evidence=row["insider_evidence"] or "",
detected_at=datetime.fromisoformat(row["detected_at"]),
market_resolved=bool(row["market_resolved"]),
market_resolved_at=(
datetime.fromisoformat(row["market_resolved_at"])
if row["market_resolved_at"] else None
),
resolved_outcome=row["resolved_outcome"],
signal_correct=bool(row["signal_correct"]) if row["signal_correct"] is not None else None,
theoretical_roi=row["theoretical_roi"],
)
def get_signals_for_market(
self,
market_id: str,
top_recent: int = 5,
top_likelihood: int = 5,
min_likelihood: float = 0.4,
) -> List[AnomalySignal]:
"""Get top recent + top likelihood signals for a market, deduplicated.
Only returns signals with likelihood >= min_likelihood (for LLM context)."""
with self._get_conn() as conn:
# Top recent (above threshold only)
recent_rows = conn.execute(
"SELECT * FROM signals WHERE market_id = ? AND information_asymmetry_score >= ? ORDER BY trade_timestamp DESC LIMIT ?",
(market_id, min_likelihood, top_recent),
).fetchall()
# Top likelihood (above threshold only)
likelihood_rows = conn.execute(
"SELECT * FROM signals WHERE market_id = ? AND information_asymmetry_score >= ? ORDER BY information_asymmetry_score DESC LIMIT ?",
(market_id, min_likelihood, top_likelihood),
).fetchall()
seen = set()
combined = []
for row in list(recent_rows) + list(likelihood_rows):
tx_hash = row["transaction_hash"]
if tx_hash not in seen:
seen.add(tx_hash)
combined.append(self._row_to_signal(row))
combined.sort(key=lambda s: s.trade_timestamp, reverse=True)
return combined
def get_unresolved_market_ids(self) -> List[str]:
"""Return distinct market_ids that have unresolved signals."""
with self._get_conn() as conn:
rows = conn.execute(
"SELECT DISTINCT market_id FROM signals WHERE market_resolved = 0"
).fetchall()
return [row["market_id"] for row in rows]
def mark_market_resolved(
self,
market_id: str,
resolved_outcome: str,
resolved_at: datetime,
) -> int:
"""
Mark all signals for a market as resolved and compute correctness/ROI.
Returns number of updated rows.
"""
with self._get_conn() as conn:
rows = conn.execute(
"SELECT transaction_hash, trade_outcome, trade_price FROM signals WHERE market_id = ? AND market_resolved = 0",
(market_id,),
).fetchall()
updated = 0
for row in rows:
correct = row["trade_outcome"] == resolved_outcome
if correct:
roi = (1.0 - row["trade_price"]) / row["trade_price"] if row["trade_price"] > 0 else 0.0
else:
roi = -1.0
conn.execute("""
UPDATE signals SET
market_resolved = 1,
market_resolved_at = ?,
resolved_outcome = ?,
signal_correct = ?,
theoretical_roi = ?
WHERE transaction_hash = ?
""", (
resolved_at.isoformat(),
resolved_outcome,
int(correct),
roi,
row["transaction_hash"],
))
updated += 1
return updated
def get_stats(self) -> dict:
"""Aggregate statistics: total, resolved, correct, win_rate, avg_roi."""
with self._get_conn() as conn:
row = conn.execute("""
SELECT
COUNT(*) as total,
SUM(CASE WHEN market_resolved = 1 THEN 1 ELSE 0 END) as resolved,
SUM(CASE WHEN signal_correct = 1 THEN 1 ELSE 0 END) as correct,
AVG(CASE WHEN market_resolved = 1 THEN theoretical_roi END) as avg_roi,
SUM(CASE WHEN market_resolved = 1 THEN theoretical_roi ELSE 0 END) as total_pnl
FROM signals
""").fetchone()
total = row["total"]
resolved = row["resolved"] or 0
correct = row["correct"] or 0
win_rate = correct / resolved if resolved > 0 else 0.0
last_updated = None
if total > 0:
last_row = conn.execute(
"SELECT detected_at FROM signals ORDER BY detected_at DESC LIMIT 1"
).fetchone()
last_updated = last_row["detected_at"] if last_row else None
return {
"total_signals": total,
"resolved": resolved,
"correct": correct,
"win_rate": win_rate,
"avg_roi": row["avg_roi"] or 0.0,
"total_theoretical_pnl": row["total_pnl"] or 0.0,
"last_updated": last_updated,
}
def get_stats_by_tier(self) -> List[dict]:
"""Stats grouped by information_asymmetry_score tiers."""
tiers = [
("0.4-0.6", 0.4, 0.6),
("0.6-0.8", 0.6, 0.8),
("0.8-1.0", 0.8, 1.01),
]
results = []
with self._get_conn() as conn:
for label, low, high in tiers:
row = conn.execute("""
SELECT
COUNT(*) as total,
SUM(CASE WHEN market_resolved = 1 THEN 1 ELSE 0 END) as resolved,
SUM(CASE WHEN signal_correct = 1 THEN 1 ELSE 0 END) as correct,
AVG(CASE WHEN market_resolved = 1 THEN theoretical_roi END) as avg_roi
FROM signals
WHERE information_asymmetry_score >= ? AND information_asymmetry_score < ?
""", (low, high)).fetchone()
resolved = row["resolved"] or 0
correct = row["correct"] or 0
results.append({
"tier": label,
"total": row["total"],
"resolved": resolved,
"correct": correct,
"win_rate": correct / resolved if resolved > 0 else 0.0,
"avg_roi": row["avg_roi"] or 0.0,
})
return results
def get_all_signals(self, limit: int = 50, offset: int = 0) -> List[AnomalySignal]:
"""Paginated query of all signals, newest first."""
with self._get_conn() as conn:
rows = conn.execute(
"SELECT * FROM signals ORDER BY detected_at DESC LIMIT ? OFFSET ?",
(limit, offset),
).fetchall()
return [self._row_to_signal(row) for row in rows]
def get_all_market_ids(self) -> List[str]:
"""Get all distinct market IDs."""
with self._get_conn() as conn:
rows = conn.execute("SELECT DISTINCT market_id FROM signals").fetchall()
return [row["market_id"] for row in rows]
def get_signal_count(self, market_id: Optional[str] = None) -> int:
"""Count signals, optionally filtered by market_id."""
with self._get_conn() as conn:
if market_id:
row = conn.execute(
"SELECT COUNT(*) as cnt FROM signals WHERE market_id = ?", (market_id,)
).fetchone()
else:
row = conn.execute("SELECT COUNT(*) as cnt FROM signals").fetchone()
return row["cnt"]
def cleanup_old_signals(self, max_age_days: int = 30) -> int:
"""Remove signals older than max_age_days. Returns count removed."""
from datetime import timedelta
cutoff = (datetime.utcnow() - timedelta(days=max_age_days)).isoformat()
with self._get_conn() as conn:
cursor = conn.execute(
"DELETE FROM signals WHERE detected_at < ?", (cutoff,)
)
return cursor.rowcount
def get_recent_resolved(self, limit: int = 20) -> List[AnomalySignal]:
"""Get recently resolved signals."""
with self._get_conn() as conn:
rows = conn.execute(
"SELECT * FROM signals WHERE market_resolved = 1 ORDER BY market_resolved_at DESC LIMIT ?",
(limit,),
).fetchall()
return [self._row_to_signal(row) for row in rows]
def get_best_worst(self, n: int = 5) -> dict:
"""Get best and worst signals by ROI."""
with self._get_conn() as conn:
best_rows = conn.execute(
"SELECT * FROM signals WHERE market_resolved = 1 ORDER BY theoretical_roi DESC LIMIT ?",
(n,),
).fetchall()
worst_rows = conn.execute(
"SELECT * FROM signals WHERE market_resolved = 1 ORDER BY theoretical_roi ASC LIMIT ?",
(n,),
).fetchall()
return {
"best": [self._row_to_signal(r) for r in best_rows],
"worst": [self._row_to_signal(r) for r in worst_rows],
}
def migrate_from_json(self, json_dir: Path) -> int:
"""One-time migration from JSON files to SQLite. Returns count migrated."""
if not json_dir.exists():
logger.warning(f"JSON directory not found: {json_dir}")
return 0
count = 0
for json_file in json_dir.glob("*.json"):
try:
with open(json_file, 'r', encoding='utf-8') as f:
data = json.load(f)
signals = data if isinstance(data, list) else data.get("signals", [])
for item in signals:
try:
signal = AnomalySignal.model_validate(item)
if self.insert_signal(signal):
count += 1
except Exception as e:
logger.warning(f"Failed to parse signal from {json_file.name}: {e}")
except Exception as e:
logger.error(f"Failed to load {json_file}: {e}")
logger.info(f"Migrated {count} signals from JSON to SQLite")
return count
src/main.py
+604
View File
@@ -0,0 +1,604 @@
"""
Polymarket Whale Watcher - Main Entry Point
This bot monitors Polymarket markets for large (whale) trades via
real-time WebSocket (RTDS) and generates AI-powered analysis reports.
Flow:
1. Fetch market list (for enrichment metadata)
2. RTDS WebSocket receives ALL trades in real-time (zero missed trades)
3. Filter for whale trades (size, price range, conviction)
4. Generate analysis reports using LLM
5. Output reports for user review (no automatic trading)
"""
import asyncio
import os
import re
import signal
import smtplib
import sys
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from datetime import datetime, timedelta
from pathlib import Path
from typing import Optional
import typer
from src.config import get_settings
from src.db.database import SignalDatabase
from src.services.market_fetcher import MarketFetcher
from src.services.trade_monitor import TradeMonitor
from src.services.price_monitor import PriceMonitor, VolatilityAlert
from src.services.llm_analyzer import LLMAnalyzer
from src.services.volatility_analyzer import VolatilityAnalyzer
from src.services.daily_briefing import DailyBriefingGenerator
from src.services.resolution_tracker import ResolutionTracker
from src.models.trade import WhaleTrade
from src.utils.logger import setup_logging, WhaleWatcherLogger
app = typer.Typer(help="Polymarket Whale Watcher - AI-powered whale trade analysis")
logger = WhaleWatcherLogger()
class WhaleWatcher:
"""Main whale watcher application."""
# Reports directory
REPORTS_DIR = Path(__file__).parent.parent / "reports"
def __init__(self):
self.settings = get_settings()
self.market_fetcher = MarketFetcher()
self.trade_monitor = TradeMonitor(on_whale_detected=self.on_whale_detected)
self.llm_analyzer = LLMAnalyzer()
# Volatility analyzer for detecting "price leads news" signals
self.volatility_analyzer = VolatilityAnalyzer()
# Price monitor for ALL active markets (independent from trade monitor)
self.price_monitor = PriceMonitor(
window_seconds=3600, # 1 hour
threshold=0.20, # 20%
poll_interval=60, # Poll every 60 seconds
on_volatility_detected=self.on_volatility_detected,
)
# Database and resolution tracker
self.db = SignalDatabase(self.settings.db_path)
self.resolution_tracker = ResolutionTracker(self.db)
# Daily briefing generator
self.briefing_generator = DailyBriefingGenerator(self.settings.db_path)
self._running = False
self._refresh_interval = 900 # Refresh markets every 15 minutes
self._resolution_check_interval = 1800 # Check resolutions every 30 minutes
self._last_briefing_date = None # Track last briefing date
# Ensure reports directory exists
self.REPORTS_DIR.mkdir(parents=True, exist_ok=True)
def _sanitize_filename(self, text: str, max_length: int = 50) -> str:
"""Sanitize text for use in filename."""
# Remove special characters, keep alphanumeric and spaces
sanitized = re.sub(r'[^\w\s-]', '', text)
# Replace spaces with underscores
sanitized = re.sub(r'\s+', '_', sanitized)
# Truncate if too long
return sanitized[:max_length]
def _save_report(self, whale_trade: WhaleTrade, full_report: str) -> str:
"""
Save report to a markdown file.
Args:
whale_trade: The whale trade
full_report: The formatted report
Returns:
Path to the saved file
"""
trade = whale_trade.trade
date_str = datetime.now().strftime("%Y%m%d")
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
market_name = self._sanitize_filename(whale_trade.market_question)
day_dir = self.REPORTS_DIR / date_str
day_dir.mkdir(parents=True, exist_ok=True)
filename = f"{timestamp}_{trade.side}_{int(trade.usdc_size)}USD_{market_name}.md"
filepath = day_dir / filename
with open(filepath, "w", encoding="utf-8") as f:
f.write(full_report)
return str(filepath)
async def on_whale_detected(self, whale_trade: WhaleTrade) -> None:
"""
Callback when a whale trade is detected.
Args:
whale_trade: The detected whale trade
"""
trade = whale_trade.trade
# Log detection
logger.whale_detected(
amount=trade.usdc_size,
side=f"BUY {trade.outcome}",
price=trade.price,
market=whale_trade.market_question,
)
# Analyze with LLM
logger.info("Generating analysis report...")
decision = await self.llm_analyzer.analyze_whale_trade(whale_trade)
# Print the full report (includes analysis + decision summary)
full_report = self.llm_analyzer.format_full_report(
whale_trade,
decision,
historical_signal_count=self.llm_analyzer.last_historical_signal_count,
)
print(full_report)
# Save report to file
filepath = self._save_report(whale_trade, full_report)
logger.info(f"Report saved to: {filepath}")
# Real-time email alert for high information asymmetry (>= 60%)
ias = decision.recommendation.information_asymmetry_score
if ias >= 0.6:
self._send_alert_email(whale_trade, full_report, ias)
logger.separator()
def _send_alert_email(self, whale_trade: WhaleTrade, report: str, likelihood: float):
"""Send real-time email alert for high information asymmetry signals."""
settings = get_settings()
if not settings.email_enabled or not settings.email_sender or not settings.email_password:
return
alert_recipient = "1253608463@qq.com"
trade = whale_trade.trade
try:
subject = (
f"Anomalous Trade Alert ({likelihood:.0%}) — "
f"BUY {trade.outcome} @ {trade.price:.4f} "
f"${trade.usdc_size:,.0f} — {whale_trade.market_question[:50]}"
)
msg = MIMEMultipart("alternative")
msg["Subject"] = subject
msg["From"] = settings.email_sender
msg["To"] = alert_recipient
msg.attach(MIMEText(report, "plain", "utf-8"))
with smtplib.SMTP_SSL(settings.email_smtp_server, settings.email_smtp_port) as server:
server.login(settings.email_sender, settings.email_password)
server.sendmail(settings.email_sender, alert_recipient, msg.as_string())
logger.info(f"Insider alert email sent to {alert_recipient} (likelihood: {likelihood:.0%})")
except Exception as e:
logger.error(f"Failed to send alert email: {e}")
async def on_volatility_detected(self, alert: VolatilityAlert) -> None:
"""
Callback when price volatility is detected.
Analyzes the volatility to determine if it's a "price leads news" signal.
Args:
alert: The volatility alert
"""
logger.info(
f"Volatility detected: {alert.market_question[:50]}... "
f"{alert.direction} {abs(alert.price_change_percent):.1%}"
)
# Analyze with LLM to check if price leads news
logger.info("Analyzing volatility for leading signal detection...")
signal = await self.volatility_analyzer.analyze_volatility(alert)
if signal:
# Print the analysis report
report = self.volatility_analyzer.format_signal_report(signal)
print(report)
if signal.is_leading_signal:
logger.info(
f"LEADING SIGNAL recorded: {alert.market_question[:50]}... "
f"Time advantage: {signal.time_advantage_minutes} minutes"
)
else:
logger.info(
f"Signal analyzed: {signal.signal_type.value} "
f"(confidence: {signal.confidence:.1%})"
)
# Print stats
stats = self.volatility_analyzer.get_leading_signals_stats()
logger.info(
f"Dataset stats: {stats['total_signals']} total, "
f"{stats['leading_signals']} leading signals"
)
logger.separator()
async def refresh_markets(self) -> None:
"""Fetch and update the list of monitored markets."""
if self.settings.full_market_scan:
await self._refresh_markets_tiered()
else:
await self._refresh_markets_legacy()
async def _refresh_markets_tiered(self) -> None:
"""Full-coverage tiered monitoring (mirrors options flow passive approach)."""
logger.info("Fetching ALL active markets for tiered monitoring...")
tiers = self.market_fetcher.get_tiered_markets()
# Also merge token launch markets into appropriate tiers
existing_ids = set()
for tier_markets in tiers.values():
for tm in tier_markets:
existing_ids.add(tm.market.id)
token_markets = self.market_fetcher.get_token_launch_markets()
token_added = 0
for tm in token_markets:
if tm.market.id not in existing_ids:
# Assign to tier based on volume
vol = tm.volume_24hr
if vol >= self.settings.tier1_volume_min:
tiers["tier1"].append(tm)
elif vol >= self.settings.tier2_volume_min:
tiers["tier2"].append(tm)
else:
tiers["tier3"].append(tm)
existing_ids.add(tm.market.id)
token_added += 1
if token_added:
logger.info(f"Added {token_added} token launch markets to tiers")
total = sum(len(v) for v in tiers.values())
if total > 0:
self.trade_monitor.set_tiered_markets(tiers)
logger.info(f"Tiered monitoring active: {total} markets total")
else:
logger.error("Failed to fetch any markets")
async def _refresh_markets_legacy(self) -> None:
"""Original Top-N trending markets mode."""
logger.info("Fetching trending markets...")
trending_markets = self.market_fetcher.get_trending_markets(
limit=self.settings.trending_markets_limit
)
existing_ids = {tm.market.id for tm in trending_markets}
token_markets = self.market_fetcher.get_token_launch_markets()
token_added = 0
for tm in token_markets:
if tm.market.id not in existing_ids:
trending_markets.append(tm)
existing_ids.add(tm.market.id)
token_added += 1
if token_added:
logger.info(
f"Added {token_added} token launch markets "
f"(total: {len(trending_markets)})"
)
if trending_markets:
self.trade_monitor.set_monitored_markets(trending_markets)
logger.info(f"Now monitoring {len(trending_markets)} markets")
else:
logger.error("Failed to fetch trending markets")
async def run(self) -> None:
"""Run the main whale watcher loop."""
self._running = True
# Initial market fetch
await self.refresh_markets()
# Log startup
logger.monitoring_started(
market_count=len(self.trade_monitor._monitored_markets),
min_trade_size=self.settings.min_trade_size_usd,
min_price=self.settings.min_price,
max_price=self.settings.max_price,
)
# Start monitoring tasks:
# 1. Trade monitor - RTDS WebSocket real-time trade stream
# 2. Market refresh - refreshes market metadata periodically
# 3. Daily briefing - generates daily summary at midnight
# 4. Resolution check - checks if markets with signals have resolved
# NOTE: Price volatility monitor is temporarily disabled
monitor_task = asyncio.create_task(self.trade_monitor.run())
# price_monitor_task = asyncio.create_task(self.price_monitor.run())
refresh_task = asyncio.create_task(self._refresh_loop())
briefing_task = asyncio.create_task(self._briefing_loop())
resolution_task = asyncio.create_task(self._resolution_check_loop())
try:
await asyncio.gather(monitor_task, refresh_task, briefing_task, resolution_task)
except asyncio.CancelledError:
logger.info("Shutting down...")
finally:
self.trade_monitor.stop()
self.price_monitor.stop()
await self.trade_monitor.close()
async def _refresh_loop(self) -> None:
"""Periodically refresh the market list."""
while self._running:
await asyncio.sleep(self._refresh_interval)
if self._running:
await self.refresh_markets()
def _briefing_already_sent(self, date: datetime) -> bool:
"""Check if briefing for a date was already generated (file exists)."""
from src.services.daily_briefing import BRIEFINGS_DIR
date_str = date.strftime("%Y-%m-%d")
return (BRIEFINGS_DIR / f"briefing_{date_str}.md").exists()
async def _briefing_loop(self) -> None:
"""Generate daily briefing for previous day at 10:00 local time."""
while self._running:
now = datetime.now()
today = now.date()
yesterday = now - timedelta(days=1)
# Generate at 10:00 local time, skip if already sent (survives restart)
if now.hour == 10 and now.minute >= 0:
if self._last_briefing_date != today and not self._briefing_already_sent(yesterday):
try:
filepath = self.briefing_generator.generate_briefing()
if filepath:
logger.info(f"Daily briefing generated: {filepath}")
self._last_briefing_date = today
except Exception as e:
logger.error(f"Error generating daily briefing: {e}")
else:
self._last_briefing_date = today
# Check every minute
await asyncio.sleep(60)
async def _resolution_check_loop(self) -> None:
"""Periodically check if markets with signals have resolved."""
# Initial delay to let the system start up
await asyncio.sleep(60)
while self._running:
try:
result = await self.resolution_tracker.check_all()
if result["resolved"] > 0:
logger.info(
f"Resolution check: {result['resolved']} markets resolved, "
f"{result['signals_updated']} signals updated"
)
except Exception as e:
logger.error(f"Error in resolution check: {e}")
await asyncio.sleep(self._resolution_check_interval)
def stop(self) -> None:
"""Stop the whale watcher."""
self._running = False
self.trade_monitor.stop()
self.price_monitor.stop()
# Global instance for signal handling
_watcher: Optional[WhaleWatcher] = None
def signal_handler(signum, frame):
"""Handle shutdown signals."""
logger.info("Received shutdown signal...")
if _watcher:
_watcher.stop()
sys.exit(0)
@app.command()
def run(
debug: bool = typer.Option(False, "--debug", "-d", help="Enable debug logging"),
):
"""Start the whale watcher bot."""
global _watcher
# Setup logging
setup_logging("DEBUG" if debug else "INFO")
# Setup signal handlers
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
# Create and run watcher
_watcher = WhaleWatcher()
try:
asyncio.run(_watcher.run())
except KeyboardInterrupt:
logger.info("Interrupted by user")
finally:
logger.info("Whale watcher stopped")
@app.command()
def check_markets(
limit: int = typer.Option(10, "--limit", "-l", help="Number of markets to show"),
):
"""Check current trending markets."""
setup_logging("INFO")
fetcher = MarketFetcher()
markets = fetcher.get_trending_markets(limit=limit)
print(f"\n{'='*80}")
print(f"Top {len(markets)} Trending Markets by 24hr Volume")
print(f"{'='*80}\n")
for tm in markets:
m = tm.market
prices = ", ".join(
[f"{o}: {p:.2%}" for o, p in zip(m.outcomes, m.outcome_prices)]
)
print(f"#{tm.rank} | Vol24h: ${tm.volume_24hr:,.0f}")
print(f" Question: {m.question}")
print(f" Prices: {prices}")
print(f" ID: {m.id}")
print()
@app.command()
def test_analyze(
market_id: str = typer.Argument(..., help="Market ID to test analysis on"),
):
"""Test LLM analysis on a specific market (simulates a whale trade)."""
setup_logging("INFO")
fetcher = MarketFetcher()
market = fetcher.get_market_by_id(market_id)
if not market:
print(f"Market {market_id} not found")
raise typer.Exit(1)
# Create a simulated whale trade
from src.models.trade import TradeActivity, WhaleTrade
import time
fake_activity = TradeActivity(
transaction_hash="test_" + str(int(time.time())),
timestamp=int(time.time()),
condition_id=market.condition_id or "",
asset=market.clob_token_ids[0] if market.clob_token_ids else "",
side="BUY",
size=50000.0,
usdc_size=25000.0, # Simulated $25k trade
price=0.45, # Simulated price
outcome=market.outcomes[0] if market.outcomes else "",
outcome_index=0,
title=market.question,
)
whale_trade = WhaleTrade(
id=f"test_{market_id}",
trade=fake_activity,
market_id=market.id,
market_question=market.question,
market_description=market.description,
market_outcomes=market.outcomes,
market_outcome_prices=market.outcome_prices,
)
print(f"\nSimulating whale trade analysis for:")
print(f" Market: {market.question}")
print(f" Trade: $25,000 BUY @ 0.45")
print(f"\nAnalyzing with LLM...\n")
analyzer = LLMAnalyzer()
decision = asyncio.run(analyzer.analyze_whale_trade(whale_trade))
print(analyzer.format_decision_report(decision))
print("\nFull Analysis:")
print("-" * 60)
print(decision.analysis)
@app.command()
def briefing(
date: str = typer.Option(None, "--date", "-d", help="Date in YYYY-MM-DD format (defaults to yesterday)"),
today: bool = typer.Option(False, "--today", "-t", help="Generate briefing for today instead of yesterday"),
):
"""Generate daily briefing manually."""
setup_logging("INFO")
settings = get_settings()
generator = DailyBriefingGenerator(settings.db_path)
if today:
filepath = generator.generate_today_briefing()
elif date:
try:
target_date = datetime.strptime(date, "%Y-%m-%d")
filepath = generator.generate_briefing(target_date)
except ValueError:
print(f"Invalid date format: {date}. Use YYYY-MM-DD")
raise typer.Exit(1)
else:
filepath = generator.generate_briefing() # Yesterday by default
if filepath:
print(f"\nBriefing generated: {filepath}")
# Print the content
with open(filepath, 'r', encoding='utf-8') as f:
print("\n" + "=" * 80)
print(f.read())
else:
print("\nNo signals found for the specified date. No briefing generated.")
@app.command()
def migrate():
"""Migrate anomaly signals from JSON files to SQLite database."""
setup_logging("INFO")
settings = get_settings()
db = SignalDatabase(settings.db_path)
json_dir = Path(__file__).parent.parent / "anomaly_signals"
print(f"Migrating signals from {json_dir} to {settings.db_path}")
count = db.migrate_from_json(json_dir)
print(f"Migration complete: {count} signals migrated")
# Show stats
stats = db.get_stats()
print(f"\nDatabase stats:")
print(f" Total signals: {stats['total_signals']}")
print(f" Resolved: {stats['resolved']}")
@app.command()
def dashboard(
port: int = typer.Option(8517, "--port", "-p", help="Port to run the dashboard on"),
host: str = typer.Option("127.0.0.1", "--host", "-h", help="Host to bind to"),
):
"""Start the signal performance dashboard web server."""
setup_logging("INFO")
import asyncio
import uvicorn
from src.dashboard import app as dashboard_app
print(f"Starting dashboard at http://{host}:{port}")
config = uvicorn.Config(dashboard_app, host=host, port=port)
server = uvicorn.Server(config)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(server.serve())
def main():
"""Entry point."""
app()
if __name__ == "__main__":
main()
src/models/__init__.py
+13
View File
@@ -0,0 +1,13 @@
"""Data models module."""
from .market import Market, TrendingMarket
from .trade import WhaleTrade, TradeActivity
from .decision import LLMDecision, TradeRecommendation
__all__ = [
"Market",
"TrendingMarket",
"WhaleTrade",
"TradeActivity",
"LLMDecision",
"TradeRecommendation",
]
src/models/anomaly_signal.py
+94
View File
@@ -0,0 +1,94 @@
"""Anomaly signal models for storing historical anomalous trades."""
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field
from src.models.trade import TraderRanking, TraderHistory
class AnomalySignal(BaseModel):
"""
Represents a stored anomaly signal for a market.
This captures the raw trade and trader information for trades with medium
or higher information asymmetry score. The information_asymmetry_score is stored
for sorting/filtering purposes, but NOT shown to LLM - the model will
re-analyze all signals (historical + current) together without bias.
"""
# Unique identifier
id: str = Field(default_factory=lambda: "")
# Market identification
market_id: str
market_question: str
market_slug: Optional[str] = None
condition_id: Optional[str] = None
# Trade information
transaction_hash: str
trade_timestamp: int # Unix timestamp of the trade
trade_side: str # BUY or SELL
trade_price: float
trade_size_usd: float
trade_outcome: str
# Trader information
trader_wallet: Optional[str] = None
trader_ranking: Optional[TraderRanking] = None
trader_history: Optional[TraderHistory] = None
# Information asymmetry score (for sorting/filtering only, NOT shown to LLM)
information_asymmetry_score: float = Field(default=0.0, ge=0.0, le=1.0)
# LLM analysis results
reasoning: str = ""
insider_evidence: str = ""
# Metadata
detected_at: datetime = Field(default_factory=datetime.utcnow)
# Resolution tracking
market_resolved: bool = False
market_resolved_at: Optional[datetime] = None
resolved_outcome: Optional[str] = None
signal_correct: Optional[bool] = None
theoretical_roi: Optional[float] = None
def to_context_string(self) -> str:
"""
Format this anomaly signal as a context string for LLM.
Returns:
Formatted string describing this historical anomaly signal.
"""
trade_time = datetime.fromtimestamp(self.trade_timestamp).strftime('%Y-%m-%d %H:%M:%S')
# Trader ranking info
trader_rank_str = "Unranked"
trader_pnl_str = "N/A"
trader_vol_str = "N/A"
if self.trader_ranking:
if self.trader_ranking.rank:
trader_rank_str = f"#{self.trader_ranking.rank}"
if self.trader_ranking.pnl is not None:
trader_pnl_str = f"${self.trader_ranking.pnl:,.2f}"
if self.trader_ranking.volume is not None:
trader_vol_str = f"${self.trader_ranking.volume:,.2f}"
# Trader history info
trader_history_str = ""
if self.trader_history:
trader_history_str = f"""
- Recent Trades: {self.trader_history.total_trades}
- Total Volume: ${self.trader_history.total_volume:,.2f}
- Large Trades: {self.trader_history.large_trades_count}"""
return f"""**Trade Time**: {trade_time}
**Direction**: {self.trade_side}
**Trade Size**: ${self.trade_size_usd:,.2f} USDC
**Trade Price**: {self.trade_price:.4f}
**Outcome**: {self.trade_outcome}
**Trader Wallet**: {self.trader_wallet or 'Unknown'}
**Trader Rank**: {trader_rank_str} (PnL: {trader_pnl_str}, Volume: {trader_vol_str}){trader_history_str}"""
src/models/decision.py
+59
View File
@@ -0,0 +1,59 @@
"""LLM decision models."""
from datetime import datetime
from typing import Optional
from enum import Enum
from pydantic import BaseModel, Field
class TradeAction(str, Enum):
"""Recommended trade action."""
BUY = "BUY"
SELL = "SELL"
HOLD = "HOLD" # Do not trade
class TraderCredibility(str, Enum):
"""Trader credibility level based on leaderboard ranking."""
HIGH = "HIGH" # Top 100
MEDIUM = "MEDIUM" # 100-500
LOW = "LOW" # 500+
UNKNOWN = "UNKNOWN" # Not on leaderboard
class TradeRecommendation(BaseModel):
"""Trade recommendation from LLM."""
action: TradeAction
outcome: str # Which outcome to trade
confidence: float = Field(ge=0.0, le=1.0) # 0-1 confidence score
suggested_price: Optional[float] = None
suggested_size_percent: float = Field(default=0.1, ge=0.0, le=1.0) # % of balance
reasoning: str = ""
# Information asymmetry assessment fields
information_asymmetry_score: float = Field(default=0.0, ge=0.0, le=1.0) # 0-1 score
trader_credibility: TraderCredibility = TraderCredibility.UNKNOWN
insider_evidence: str = "" # Evidence supporting information asymmetry assessment
class LLMDecision(BaseModel):
"""Complete LLM decision for a whale trade."""
whale_trade_id: str
market_id: str
analysis: str # Full LLM analysis text
recommendation: TradeRecommendation
created_at: datetime = Field(default_factory=datetime.utcnow)
executed: bool = False
execution_result: Optional[str] = None
@property
def should_trade(self) -> bool:
"""Check if we should execute this trade."""
return (
self.recommendation.action != TradeAction.HOLD
and self.recommendation.confidence >= 0.6
)
src/models/leading_signal.py
+124
View File
@@ -0,0 +1,124 @@
"""Data models for leading signal detection - price moves before news."""
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from typing import List, Optional
class SignalType(str, Enum):
"""Type of price volatility signal."""
LEADING_SIGNAL = "LEADING_SIGNAL" # Price moved before news
NEWS_DRIVEN = "NEWS_DRIVEN" # Price reacted to news
SOCIAL_DRIVEN = "SOCIAL_DRIVEN" # Price driven by social media
SPECULATION = "SPECULATION" # No clear information source
@dataclass
class LeadingSignal:
"""
A case where price movement preceded public news.
This is used to build a dataset of "price leads news" events
for research purposes.
"""
# Basic info
id: str
market_id: str
market_question: str
# Price movement details
price_change_percent: float # e.g., 0.25 for 25%
direction: str # "UP" or "DOWN"
start_price: float
end_price: float
window_seconds: int
# Timing
detected_at: str # ISO format timestamp
volatility_detected_at: str # When price volatility was detected
# LLM analysis results
signal_type: SignalType
confidence: float # 0-1
is_leading_signal: bool
# News analysis
news_found: bool
earliest_news_time: Optional[str] = None # ISO format
key_news_headlines: List[str] = field(default_factory=list)
# Social media analysis
earliest_social_time: Optional[str] = None # ISO format
key_social_posts: List[str] = field(default_factory=list)
# Time advantage
time_advantage_minutes: int = 0 # How many minutes price led news
# Analysis
reasoning: str = ""
potential_information_source: str = ""
# Full LLM analysis text
full_analysis: str = ""
def to_dict(self) -> dict:
"""Convert to dictionary for JSON serialization."""
return {
"id": self.id,
"market_id": self.market_id,
"market_question": self.market_question,
"price_change_percent": self.price_change_percent,
"direction": self.direction,
"start_price": self.start_price,
"end_price": self.end_price,
"window_seconds": self.window_seconds,
"detected_at": self.detected_at,
"volatility_detected_at": self.volatility_detected_at,
"signal_type": self.signal_type.value if isinstance(self.signal_type, SignalType) else self.signal_type,
"confidence": self.confidence,
"is_leading_signal": self.is_leading_signal,
"news_found": self.news_found,
"earliest_news_time": self.earliest_news_time,
"key_news_headlines": self.key_news_headlines,
"earliest_social_time": self.earliest_social_time,
"key_social_posts": self.key_social_posts,
"time_advantage_minutes": self.time_advantage_minutes,
"reasoning": self.reasoning,
"potential_information_source": self.potential_information_source,
"full_analysis": self.full_analysis,
}
@classmethod
def from_dict(cls, data: dict) -> "LeadingSignal":
"""Create from dictionary."""
signal_type = data.get("signal_type", "SPECULATION")
if isinstance(signal_type, str):
try:
signal_type = SignalType(signal_type)
except ValueError:
signal_type = SignalType.SPECULATION
return cls(
id=data["id"],
market_id=data["market_id"],
market_question=data["market_question"],
price_change_percent=data["price_change_percent"],
direction=data["direction"],
start_price=data["start_price"],
end_price=data["end_price"],
window_seconds=data["window_seconds"],
detected_at=data["detected_at"],
volatility_detected_at=data["volatility_detected_at"],
signal_type=signal_type,
confidence=data.get("confidence", 0.0),
is_leading_signal=data.get("is_leading_signal", False),
news_found=data.get("news_found", False),
earliest_news_time=data.get("earliest_news_time"),
key_news_headlines=data.get("key_news_headlines", []),
earliest_social_time=data.get("earliest_social_time"),
key_social_posts=data.get("key_social_posts", []),
time_advantage_minutes=data.get("time_advantage_minutes", 0),
reasoning=data.get("reasoning", ""),
potential_information_source=data.get("potential_information_source", ""),
full_analysis=data.get("full_analysis", ""),
)
src/models/market.py
+44
View File
@@ -0,0 +1,44 @@
"""Market data models."""
from datetime import datetime
from typing import Optional, List
from pydantic import BaseModel, Field
class Market(BaseModel):
"""Polymarket market data model."""
id: str
question: str
condition_id: Optional[str] = None
slug: Optional[str] = None
description: Optional[str] = None
end_date: Optional[str] = None
outcomes: List[str] = Field(default_factory=list)
outcome_prices: List[float] = Field(default_factory=list)
clob_token_ids: List[str] = Field(default_factory=list)
volume: float = 0.0
volume_24hr: float = 0.0
liquidity: float = 0.0
active: bool = True
closed: bool = False
neg_risk: bool = False
class TrendingMarket(BaseModel):
"""Trending market with additional metrics."""
market: Market
volume_24hr: float = 0.0
liquidity: float = 0.0
rank: int = 0
fetched_at: datetime = Field(default_factory=datetime.utcnow)
@property
def is_valid_for_monitoring(self) -> bool:
"""Check if market is valid for whale monitoring."""
return (
self.market.active
and not self.market.closed
and len(self.market.clob_token_ids) > 0
)
src/models/trade.py
+240
View File
@@ -0,0 +1,240 @@
"""Trade data models."""
from datetime import datetime
from typing import Optional
from enum import Enum
from pydantic import BaseModel, Field
class TradeSide(str, Enum):
"""Trade side enum."""
BUY = "BUY"
SELL = "SELL"
class TradeActivity(BaseModel):
"""Raw trade activity from Polymarket API."""
transaction_hash: str
timestamp: int
condition_id: str
asset: str
side: str
size: float # Token size
usdc_size: float # USD value
price: float
outcome: str
outcome_index: int
title: str
slug: Optional[str] = None
event_slug: Optional[str] = None
proxy_wallet: Optional[str] = None
name: Optional[str] = None
class TraderRanking(BaseModel):
"""Trader ranking information from leaderboard."""
rank: Optional[int] = None # Position on leaderboard (None if not ranked)
pnl: Optional[float] = None # Profit/Loss
volume: Optional[float] = None # Trading volume
user_name: Optional[str] = None # Display name
profile_image: Optional[str] = None # Avatar URL
verified: bool = False # Verified badge
time_period: str = "ALL" # Time period for ranking
class TraderHistory(BaseModel):
"""Trader's recent trading history summary."""
total_trades: int = 0 # Total number of recent trades
total_volume: float = 0.0 # Total trading volume in USDC
avg_trade_size: float = 0.0 # Average trade size
win_rate: Optional[float] = None # Win rate if calculable
recent_markets: list[str] = Field(default_factory=list) # Recent markets traded
large_trades_count: int = 0 # Number of trades >= $5000
recent_trades: list[dict] = Field(default_factory=list) # Recent trade details
class EventPosition(BaseModel):
"""Whale's position in a related market under the same event."""
market_question: str
condition_id: str = ""
outcome: str = "" # "Yes" or "No"
size: float = 0.0 # token size held
avg_price: float = 0.0 # average entry price
current_price: float = 0.0 # current market price
current_value: float = 0.0 # current position value in USD
initial_value: float = 0.0 # cost basis
pnl: float = 0.0 # realized + unrealized PnL
side_summary: str = "" # human readable summary
class MarketTopTrader(BaseModel):
"""Top trader on a market (by net volume)."""
wallet: str
name: Optional[str] = None
rank: Optional[int] = None
pnl: Optional[float] = None
net_volume_usd: float = 0.0 # positive = net buyer of Yes, negative = net seller
trade_count: int = 0
class WhaleTrade(BaseModel):
"""Whale trade that meets detection criteria."""
id: str = Field(default_factory=lambda: "")
trade: TradeActivity
market_id: str
market_question: str
market_description: Optional[str] = None
market_outcomes: list[str] = Field(default_factory=list)
market_outcome_prices: list[float] = Field(default_factory=list)
detected_at: datetime = Field(default_factory=datetime.utcnow)
processed: bool = False
llm_analyzed: bool = False
# Trader ranking info
trader_ranking: Optional[TraderRanking] = None
# Trader history info
trader_history: Optional[TraderHistory] = None
# Whale's positions across the same event
whale_event_positions: list[EventPosition] = Field(default_factory=list)
# Top traders on this market (bulls and bears)
market_top_buyers: list[MarketTopTrader] = Field(default_factory=list)
market_top_sellers: list[MarketTopTrader] = Field(default_factory=list)
@property
def is_whale_trade(self) -> bool:
"""Check if this qualifies as a whale trade."""
return self.trade.usdc_size >= 10000
@property
def is_valid_price_range(self) -> bool:
"""Check if trade price is in valid range (0.2-0.8)."""
return 0.2 <= self.trade.price <= 0.8
def format_event_positions(self) -> str:
"""Format whale's event positions for LLM context."""
if self.whale_event_positions:
info = "### Whale's Positions in Other Markets Under the Same Event\n"
info += "(Used to identify hedging or correlated bets)\n\n"
for pos in self.whale_event_positions:
pnl_str = f"PnL ${pos.pnl:+,.0f}" if pos.pnl else ""
info += (
f"- **{pos.market_question[:60]}{'...' if len(pos.market_question) > 60 else ''}**\n"
f" {pos.side_summary} | "
f"Current Value ${pos.current_value:,.0f} | Cost Basis ${pos.initial_value:,.0f} | "
f"{pnl_str}\n"
)
return info
return "### Whale's Positions in Other Markets Under the Same Event\n- No other related positions (single-market event or no cross-market trades)\n"
def format_top_traders(self) -> str:
"""Format market top holders for LLM context."""
info = "### Top 5 Bulls and Bears on This Market\n"
info += "(Reflects the stance and credentials of major participants)\n"
if self.market_top_buyers:
info += "\n**Bulls (Holding Yes Token)**:\n"
for i, t in enumerate(self.market_top_buyers, 1):
rank_str = f"Rank #{t.rank}" if t.rank else "Unranked"
pnl_str = f"PnL ${t.pnl:,.0f}" if t.pnl is not None else ""
name_str = t.name or t.wallet[:10] + "..."
info += (
f" {i}. **{name_str}** ({rank_str}{', ' + pnl_str if pnl_str else ''}) "
f"— Position Value ${t.net_volume_usd:,.0f}\n"
)
else:
info += "\n**Bulls**: No significant positions\n"
if self.market_top_sellers:
info += "\n**Bears (Holding No Token)**:\n"
for i, t in enumerate(self.market_top_sellers, 1):
rank_str = f"Rank #{t.rank}" if t.rank else "Unranked"
pnl_str = f"PnL ${t.pnl:,.0f}" if t.pnl is not None else ""
name_str = t.name or t.wallet[:10] + "..."
info += (
f" {i}. **{name_str}** ({rank_str}{', ' + pnl_str if pnl_str else ''}) "
f"— Position Value ${t.net_volume_usd:,.0f}\n"
)
else:
info += "\n**Bears**: No significant positions\n"
return info
def to_llm_context(self) -> str:
"""Generate context string for LLM analysis."""
# Format trader ranking info
trader_info = ""
if self.trader_ranking:
rank_str = f"#{self.trader_ranking.rank}" if self.trader_ranking.rank else "Unranked"
pnl_str = f"${self.trader_ranking.pnl:,.2f}" if self.trader_ranking.pnl else "N/A"
vol_str = f"${self.trader_ranking.volume:,.2f}" if self.trader_ranking.volume else "N/A"
verified_str = "Verified" if self.trader_ranking.verified else "Unverified"
trader_info = f"""
### Trader Ranking (PnL Leaderboard)
- **Rank**: {rank_str} (Period: {self.trader_ranking.time_period})
- **Cumulative PnL**: {pnl_str}
- **Volume**: {vol_str}
- **Username**: {self.trader_ranking.user_name or 'Anonymous'}
- **Verification**: {verified_str}
"""
else:
trader_info = """
### Trader Ranking
- This trader is not on the PnL leaderboard (possibly a new user or small trader)
"""
# Format trader history info
history_info = ""
if self.trader_history:
history_info = f"""
### Trader History
- **Recent Trades**: {self.trader_history.total_trades}
- **Recent Volume**: ${self.trader_history.total_volume:,.2f} USDC
- **Avg Trade Size**: ${self.trader_history.avg_trade_size:,.2f} USDC
- **Large Trades** (>=$5000): {self.trader_history.large_trades_count}
- **Active Markets**: {', '.join(self.trader_history.recent_markets[:5]) if self.trader_history.recent_markets else 'N/A'}
"""
# Add recent large trades details
if self.trader_history.recent_trades:
history_info += "\n**Recent Large Trade Details**:\n"
for i, t in enumerate(self.trader_history.recent_trades[:5], 1):
history_info += f" {i}. {t.get('side', 'N/A')} ${t.get('usdc_size', 0):,.2f} @ {t.get('price', 0):.4f} - {t.get('title', 'N/A')[:40]}...\n"
else:
history_info = """
### Trader History
- Unable to retrieve this trader's trading history
"""
return f"""
## Anomalous Trade Detection
### Trade Information
- Direction: BUY {self.trade.outcome} Token ({'Bullish — expects the event to occur' if self.trade.outcome == 'Yes' else 'Bearish — expects the event will not occur'})
- Trade Size: ${self.trade.usdc_size:,.2f} USDC
- Entry Price: {self.trade.price:.4f} (Odds ~{1/self.trade.price:.1f}x)
- Trade Time: {datetime.fromtimestamp(self.trade.timestamp).strftime('%Y-%m-%d %H:%M:%S')}
- Trader Wallet: {self.trade.proxy_wallet or 'Unknown'}
{trader_info}{history_info}
{self.format_event_positions()}
{self.format_top_traders()}
### Market Information
- Market Question: {self.market_question}
- Market Description: {self.market_description or 'N/A'}
- Possible Outcomes: {', '.join(self.market_outcomes)}
- Current Prices: {', '.join([f'{o}: {p:.4f}' for o, p in zip(self.market_outcomes, self.market_outcome_prices)])}
### Key Analysis Points
1. This large trade (${self.trade.usdc_size:,.2f}) is **BUY {self.trade.outcome} Token**, {'indicating the trader is Bullish and expects the event to occur' if self.trade.outcome == 'Yes' else 'indicating the trader is Bearish and expects the event will not occur'}
2. Entry price {self.trade.price:.4f}, odds ~{1/self.trade.price:.1f}x
3. **Trader ranking and history are key references for assessing information asymmetry credibility**
4. **Review the whale's other positions under the same event** — opposite positions may indicate a hedging strategy
5. **Check the top bulls and bears on this market** — which side has the elite traders
"""
src/prompts/__init__.py
+4
View File
@@ -0,0 +1,4 @@
"""Prompts module."""
from .whale_analyzer import WhaleAnalyzerPrompts
__all__ = ["WhaleAnalyzerPrompts"]
src/prompts/volatility_analyzer.py
+222
View File
@@ -0,0 +1,222 @@
"""Prompts for price volatility analysis - detecting leading signals."""
from datetime import datetime, timezone
class VolatilityAnalyzerPrompts:
"""Prompts for LLM price volatility analysis."""
@staticmethod
def system_prompt() -> str:
"""Get the system prompt for volatility analysis."""
current_utc = datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')
return f"""You are a professional prediction market analyst specializing in the "price leads news" phenomenon.
**Current real time**: {current_utc}
**Your core task**: Determine whether a detected price anomaly "leads public news" — i.e., the price movement occurred before related news was publicly reported.
## Background
In prediction markets, the following pattern sometimes occurs:
1. Market price suddenly moves sharply
2. But mainstream news media has not yet reported the related event
3. Subsequently (hours or days later), the related news becomes public
This "price leads news" phenomenon may indicate:
- Informed participants traded on information before it became public
- Market participants obtained information through social media, unofficial channels, etc.
- Pure market speculation or technical volatility
## Your Workflow
### Step 1: Analyze provided Web search results
- Analyze the latest news related to the market topic
- Pay special attention to news publication timestamps
- Determine if any major news can explain this price movement
### Step 2: Analyze Twitter social media data
- Analyze provided Twitter search results
- Check for early social media discussions
- Note timing of KOL and insider posts
### Step 3: Classify the price movement
Based on search results, classify the volatility as one of:
1. **LEADING_SIGNAL**: Price movement clearly preceded public news
- No news found that explains the movement
- Or news publication time is significantly later than the price movement
- This is the type we care about most!
2. **NEWS_DRIVEN**: Price movement is a reaction to published news
- Clear related news found
- News publication time is before or close to the price movement time
3. **SOCIAL_DRIVEN**: Price movement driven by social media discussion
- Significant Twitter discussion, but mainstream media has not yet reported
- Between leading signal and news-driven
4. **SPECULATION**: No clear information source for the movement
- No related news or discussions found
- Likely pure market speculation
**Key principles**:
- Carefully analyze the Web search results provided
- Carefully analyze the Twitter search results
- Pay special attention to timestamps of news and discussions
- If it's a LEADING_SIGNAL, document evidence in detail"""
@staticmethod
def analyze_volatility(
market_question: str,
price_change_percent: float,
direction: str,
start_price: float,
end_price: float,
window_seconds: int,
detected_at: str,
twitter_context: str = "",
web_search_context: str = "",
) -> str:
"""
Get the prompt for analyzing a price volatility event.
Args:
market_question: The market question
price_change_percent: Price change as decimal (e.g., 0.25 for 25%)
direction: "UP" or "DOWN"
start_price: Starting price
end_price: Ending price
window_seconds: Time window in seconds
detected_at: Detection timestamp
twitter_context: Twitter search results
web_search_context: Web search results from Tavily
Returns:
Complete prompt for LLM
"""
direction_label = "UP" if direction == "UP" else "DOWN"
window_minutes = window_seconds // 60
web_search_section = ""
if web_search_context:
web_search_section = f"""
---
## Web Search Results (News & Analysis)
The following are recent web search results related to this market. Please carefully analyze publication timestamps and content:
{web_search_context}
---
"""
twitter_section = ""
if twitter_context:
twitter_section = f"""
---
## Twitter Social Media Search Results
The following are real-time Twitter discussions related to this market. Please carefully analyze timestamps and content:
{twitter_context}
---
"""
return f"""## Anomalous Price Movement Detection Report
### Movement Details
- **Market question**: {market_question}
- **Price change**: {direction_label} {abs(price_change_percent):.1%}
- **Start price**: {start_price:.2%}
- **End price**: {end_price:.2%}
- **Time window**: Within {window_minutes} minutes
- **Detection time**: {detected_at}
{web_search_section}{twitter_section}
---
# Price Movement Verification Task
A significant anomalous price movement has been detected. Please determine whether this is a "price leads news" signal.
---
## Step 1: Web Search Results Analysis (mandatory!)
**Please carefully analyze the Web search results provided above, focusing on:**
1. Latest news related to "{market_question}" (focus on the past 24 hours)
2. Events or announcements that may have triggered this price movement
3. Publication timestamp of each news item
**Web search results summary**:
(Please list key news from search results here, MUST include publication times)
---
## Step 2: Twitter Social Media Analysis
**Analyze the Twitter search results provided above:**
1. When was the earliest related discussion?
2. What were the main topics discussed?
3. Were there any KOL or insider posts?
4. Did social media discussion precede mainstream news coverage?
**Twitter analysis summary**:
(Please summarize key information and timeline from Twitter here)
---
## Step 3: Timeline Comparison Analysis
**Key question**: Did the price movement occur before or after news became public?
- Price movement detection time: {detected_at}
- Earliest related news publication time found: [please fill in]
- Earliest social media discussion time found: [please fill in]
**Timeline conclusion**:
(Did the price movement lead or lag the news?)
---
## Step 4: Final Determination
Based on the above analysis, provide your judgment in the following JSON format:
```json
{{
"signal_type": "LEADING_SIGNAL/NEWS_DRIVEN/SOCIAL_DRIVEN/SPECULATION",
"confidence": 0.0-1.0,
"is_leading_signal": true/false,
"news_found": true/false,
"earliest_news_time": "earliest related news publication time found, format YYYY-MM-DD HH:MM UTC, or null if none",
"earliest_social_time": "earliest social media discussion time found, format YYYY-MM-DD HH:MM UTC, or null if none",
"time_advantage_minutes": minutes price led news (if leading signal), otherwise 0,
"key_news_headlines": ["related news headline 1", "related news headline 2"],
"key_social_posts": ["key social media post summary 1", "key social media post summary 2"],
"reasoning": "brief explanation of your judgment basis",
"potential_information_source": "hypothesized information source (e.g., insider, social media leak, advance official notice, etc.)"
}}
```
**Judgment criteria**:
- **LEADING_SIGNAL**: At the time of price movement, web search finds no related news, or news publication time is significantly later than price movement (>=30 minutes)
- **NEWS_DRIVEN**: Clear related news found, with publication time before or close to the price movement time
- **SOCIAL_DRIVEN**: Early Twitter discussion found, but mainstream media has not yet reported
- **SPECULATION**: Neither news nor social discussion found, likely pure speculation
**Special notes**:
- When is_leading_signal is true, detailed evidence must be provided
- time_advantage_minutes represents the time advantage of price over news
- This data will be used to build a "price leads news" research dataset
---
Disclaimer: This analysis is for research purposes only and does not constitute investment advice."""
src/prompts/whale_analyzer.py
+315
View File
@@ -0,0 +1,315 @@
"""Prompts for whale trade analysis."""
from datetime import datetime, timezone
from typing import List
class WhaleAnalyzerPrompts:
"""Prompts for LLM whale trade analysis."""
@staticmethod
def system_prompt() -> str:
"""System prompt for whale trade analysis with tool-use."""
current_utc = datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')
return f"""You are a professional prediction market analyst and information asymmetry detection expert, specializing in analyzing large anomalous trades on Polymarket.
**Current real time**: {current_utc}
## Your Core Task
Verify whether a flagged anomalous trade exhibits information asymmetry — i.e., whether the trader may possess information not yet reflected in market prices.
## Data You Will Receive
For each analysis task, you will receive the following structured data (in the user message):
1. **Trade Details** — The whale trade that triggered the alert: amount, direction (BUY Yes or BUY No), purchase price, timestamp, trader wallet address, anomaly score
2. **Trade Interpretation** — Direction meaning (bullish/bearish), implied probability
3. **Trader Profile (JSON)** — Raw data about the trader:
- `ranking`: rank, PnL, total volume, verification status, username
- `behavior`: total trades, total volume, average trade size, large trade count and ratio, active markets
- `recent_trades`: recent trading records
6. **Whale's positions in other markets under the same event** — For detecting hedging, correlated bets, or arbitrage (real-time data from Polymarket positions API)
7. **Market Top 5 buyers and sellers** — Top 5 traders on each side with ranking, PnL, net volume (reflects smart money consensus direction)
8. **Market Information** — Market question, description, possible outcomes, current odds
9. **Historical anomaly signals** (if any) — Past anomalous trade signals detected on this market, for trend comparison
**You must synthesize ALL of the above data in your analysis — do not neglect any dimension.**
## Available Tools
You can call the following tools to obtain real-time information (all results are real live data):
- **search_web**: Search web news and analysis articles. Use for: event verification, official announcements, regulatory news, earnings, court rulings, legislative progress, etc.
- **search_twitter**: Search Twitter/X social media. Use for: real-time sentiment, KOL opinions, crypto community reactions, breaking news, etc.
- **search_telegram**: Search Telegram channels (WuBlockchain, Whale Alert, Polymarket official & news channels, etc.). Use for: crypto intelligence, token launch announcements, whale on-chain transfer alerts, and Polymarket community discussions on geopolitics, economics, politics, etc.
- **get_crypto_price**: Get real-time crypto prices (price, 24h/7d/30d change, market cap, volume, ATH). Use for: markets involving crypto price targets (e.g., "Will BTC reach $100k").
- **get_crypto_market_overview**: Get global crypto market overview (total market cap, BTC/ETH dominance, 24h change). Use for: gauging overall crypto sentiment.
- **get_economic_data**: Get FRED macroeconomic data. Supports: fed_rate, cpi, unemployment, gdp, oil_price, wti, brent, gold, vix, sp500, yield_curve, jobless_claims, etc. Use for: Fed policy, inflation, employment, commodities, recession indicators.
- **get_stock_price**: Get stock/ETF real-time snapshot (price, change, volume). Supports: AAPL, TSLA, GS, SPY, QQQ, GLD, USO, etc. Use for: markets involving specific companies or sectors.
- **get_stock_news**: Get latest stock/company news. Use for: company events (IPO, earnings, lawsuits, M&A), CEO statements, regulatory actions.
- **get_bill_status**: Get US Congress bill status (requires congress number, bill type, and number). Use for: markets involving specific legislation (e.g., TikTok ban, crypto regulation, immigration bills).
- **get_recent_legislation**: Get recently updated US Congress bills. Use for: current legislative dynamics, political markets.
- **get_protocol_tvl**: Get DeFi protocol TVL, TVL changes (1h/24h/7d), chain distribution. Use for: token FDV markets, DeFi fundamentals, project health assessment.
- **get_token_unlocks**: Get token unlock/vesting schedules. Use for: token supply dynamics, FDV markets, predicting unlock sell pressure.
- **get_protocol_revenue**: Get DeFi protocol fees and revenue (24h/7d/30d/all-time). Use for: protocol fundamentals, comparing revenue to FDV.
- **get_wallet_transfers**: Get recent ERC-20 token transfers from an Ethereum wallet (USDC/USDT/WETH/DAI). Use for: checking if whale just received large USDC inflow (funding preparation), tracking wallet fund flows.
- **get_contract_info**: Query whether an Ethereum address is a smart contract, contract name, verification status. Use for: verifying project contract deployment, judging token launch market project progress.
**Tool usage principles**:
- Based on market type and trade characteristics, decide which tools to call
- You may call one, multiple, or zero tools
- **You have a maximum of 3 tool-call rounds. Budget wisely: use Round 1 for broad search (web + twitter + telegram in parallel), Round 2 for targeted follow-up if needed, then produce your final analysis. Do NOT use all rounds just searching — reserve capacity for your final answer.**
- Do NOT call the same tool (e.g. search_web) more than 3 times total across all rounds
- If the first search already covers the topic well, stop searching and analyze
**Tool collaboration and cross-verification (important)**:
- Information from different tools should be **cross-verified** — but 2-3 sources are sufficient, do not over-search
- When multiple tools return **contradictory results**, explicitly note the discrepancy and lower confidence
- Prefer breadth (web + twitter + domain-specific tool) over depth (web × 8 with slightly different keywords)
## Polymarket Trading Mechanics
Trade data represents taker's actual buy actions (SELL/close trades are filtered out), **no normalization applied**:
- **BUY Yes** = Buy Yes Token = **Bullish** (believes event will occur)
- **BUY No** = Buy No Token = **Bearish** (believes event will not occur)
- **Price** is taker's actual purchase price (0.0~1.0) — lower price means higher odds and more uncertainty
- Example: BUY Yes @ 0.06 = pay $0.06 per share, receive $1 if event occurs (~17x odds)
- Example: BUY No @ 0.30 = pay $0.30 per share, receive $1 if event doesn't occur (~3.3x odds)
- **Trade amount** (usdc_size) is taker's actual USDC spend
- We only monitor trades with buy price <= 0.7 (high-price buys on near-certain outcomes have no signal value)
## Analysis Framework
### Trader Credibility
Trader credibility (HIGH/MEDIUM/LOW/UNKNOWN) should be assessed comprehensively using all available raw data — do not rely on a single metric. Consider:
- **Ranking**: Lower rank number = more experienced participant. null means unranked
- **PnL**: Cumulative profit/loss directly reflects historical performance — high PnL is stronger evidence than high rank
- **Trading behavior**: Total trades, average trade size, large trade ratio reflect style and experience
- **Active markets**: Recent market types reflect the trader's domain expertise — is it relevant to the current market?
- **Recent trades**: Specific buy/sell directions, amounts, and prices help identify the trader's strategy pattern
### Information Asymmetry Scoring Criteria (must be strictly followed)
**"Information asymmetry" has a very strict definition**: The trader must possess information not yet reflected in the market — non-public, specific information (e.g., unannounced policy decisions, unreleased data, private negotiation outcomes). Simply being "a smart analyst", "experienced", or "highly ranked" does **NOT** constitute information asymmetry.
**Score calibration benchmark (most trades should fall between 0.2-0.5)**:
- **0.8-1.0 (Very High)**: Only when **clear evidence of non-public information** is found. Example: trade timing precisely hours before a major announcement that was completely unpredictable; or trader has known information channels (e.g., identified as a political insider). **Very few trades should reach this level.**
- **0.6-0.8 (High)**: High-ranked trader + trade timing highly aligned with an upcoming unpriced event + search reveals specific information not yet reflected in market. Multiple strong pieces of evidence must be present simultaneously.
- **0.4-0.6 (Medium)**: High-ranked trader's large trade + some information support but uncertainty about whether it's non-public. This is where **most moderately suspicious trades** should fall.
- **0.2-0.4 (Low)**: Some anomalous features but lacking information support, or trader ranking is average. **Most ordinary whale trades** should be in this range.
- **0.0-0.2 (Very Low)**: Unranked trader's routine trade, no anomalous signals.
**Common overestimation mistakes (must avoid)**:
- Do NOT give 0.7+ just because the trader ranks high (high-ranked traders make many trades daily, the vast majority show no information asymmetry)
- Do NOT give 0.6+ just because the trade amount is large (large trades are routine for whales)
- Do NOT give high scores to short-term price prediction markets (e.g., "Bitcoin Up or Down 5 minutes") — these markets almost never involve non-public information
- Do NOT give high scores to near-expiry markets or trades with prices near 0 or 1 — this usually reflects normal market consensus
- Do NOT give high scores when all found information is public news (public information ≠ non-public information)
- Do NOT easily give high scores to large geopolitical/macro markets (e.g., Iran situation, presidential impeachment) — these markets have many participants and complex information sources; whale trades mostly reflect public analysis rather than non-public information
**High-value scenarios to focus on**:
- **Niche markets** (daily volume < $500k) with large trades — fewer participants, larger information gap, whale signals more meaningful
- **New projects/token launches** (FDV, TGE, public sale) — project teams and early investors may have non-public information
- **Specific verifiable events** (will someone do something, will a company announce a decision) — small circle of insiders, clear information
- **Quiet markets suddenly attracting high-ranked traders with large trades** — anomalous behavior is the strongest signal
## Event-Related Position Analysis
Trade data includes the whale's positions in other markets under the same Event. You must analyze:
- **Hedge detection**: If the whale holds opposing positions in different markets under the same event, it may be a hedging strategy rather than a directional bet — lower information asymmetry score
- **Correlated bets**: If the whale holds same-direction positions across multiple markets under the same event (e.g., bullish on multiple related markets), this strengthens the signal
- **Arbitrage**: Price inconsistencies across markets under the same event may indicate arbitrage — this is not an information asymmetry signal
## Market Long/Short Analysis
Trade data includes the market's Top 5 buyers and sellers with rankings and positions. Analyze:
- **Smart money consensus**: If multiple high-ranked, high-PnL traders are on the same side, the signal is stronger
- **Counterparty analysis**: If the whale's counterparties are all low-ranked traders, the signal is more reliable; if counterparties include high-ranked traders, more caution is needed
- **Market concentration**: If one side's positions are heavily concentrated in a few large holders, the market may be more prone to sharp volatility
## Key Principles
- Proactively use tools to gather latest information for trade verification
- When searches yield no supporting information, information asymmetry likelihood should decrease
- When confidence is low, recommend HOLD
- Whales can also be wrong or have other motivations (hedging, probing, etc.)
- Synthesize event-related positions and market long/short dynamics for comprehensive judgment
- **Time judgment**: Do NOT guess unknown event times (e.g., match start times). If you need to determine whether a trade occurred before or after an event, you MUST use tools to confirm the event time — never speculate"""
@staticmethod
def analyze_whale_trade(trade_context: str, historical_context: str = "") -> str:
"""
Build the user prompt for analyzing a whale trade.
Args:
trade_context: Formatted trade context from AnomalyDetector
historical_context: Historical anomaly signals (optional)
Returns:
Complete user prompt
"""
history_section = ""
if historical_context:
history_section = f"""
---
{historical_context}
---
"""
return f"""{trade_context}
{history_section}
---
# Whale Trade Verification Task
## Step 0: Pre-screening (must complete first)
Before any search or analysis, determine whether this signal warrants a full report.
**Screening criteria:**
- **Prioritize (low threshold)**: Niche markets, crypto/token launch related (FDV, TGE, public sale, protocol governance), specific verifiable events, quiet markets with sudden large trades
- **Higher threshold (need especially strong signals)**: Large geopolitical markets (war, sanctions, diplomacy), macro/Fed rate/election markets with many participants
- **Skip directly**: Sports/game results, markets with price near 0 or 1 (>=0.95 or <=0.05)
Assess holistically based on trade amount, trader rank and profile, anomaly score, and market type.
**If deemed not worth analyzing, output the following JSON and stop — do not proceed to subsequent steps:**
```json
{{{{"action": "SKIP", "reason": "one-line reason"}}}}
```
**If deemed worth analyzing, continue with the following steps.**
---
## Complete the following steps:
### 1. Information Gathering
Based on market topic, use available tools to search for relevant information:
- Latest news and developments on the market topic
- Social media discussions and sentiment
- Any events that may have triggered this trade
### 2. Trade Signal Analysis
- Trader ranking and historical P&L performance
- Structured profile (ranking, PnL, trading behavior data, recent trades)
- Whether the trade timing is anomalous
### 3. Event-Related Position Analysis
- Does the whale have positions in other markets under the same event?
- If opposing positions exist (e.g., holding both Yes and No, or hedging in related markets), it may be a hedge/arbitrage strategy — lower information asymmetry score
- If same-direction bets across multiple related markets, the signal is strengthened
### 4. Market Long/Short Analysis
- Who are the Top 5 on each side? What are their rankings?
- Which side has the concentration of high-ranked, high-PnL traders? This represents smart money consensus
- What is the quality of the whale's counterparties? If counterparties also include high-ranked traders, be more cautious
### 5. Information Gap Analysis
- Does the information found support the trade's direction?
- Has this information been fully priced by the market?
- If an information gap exists, how large is it?
### 6. Historical Signal Comparison (if available)
- Are historical signals directionally consistent with the current signal?
- Were high-ranked traders involved?
- What are the trends in trade amounts and prices?
### 7. Information Asymmetry Assessment
Assess the trader's information advantage relative to public information. Core logic:
- I_public = the set of public information you can obtain through all tools
- I_trader = the set of information the trader used to make this trade decision
- Information asymmetry = I_trader - I_public
- If public information can fully explain the trade behavior → low score
- If public information cannot explain the trade behavior (trader may have additional sources, domain expertise, data speed advantage) → high score
Output JSON assessment:
```json
{{
"information_asymmetry_score": 0.0-1.0,
"trader_credibility": "HIGH/MEDIUM/LOW/UNKNOWN",
"reasoning": "brief reasoning process",
"insider_evidence": "key evidence"
}}
```
Notes:
- information_asymmetry_score must be strictly calibrated: most trades should be 0.2-0.5, only give 0.7+ when clear evidence of information advantage is found
- Information advantage includes but is not limited to: domain expertise, data source speed difference, non-public channels, precise timing
- Trader ranking or trade size alone should NOT push information_asymmetry_score above 0.5
- Ensure valid JSON output"""
@staticmethod
def superforecaster_prompt(question: str, description: str, outcomes: List[str]) -> str:
"""Superforecaster-style analysis prompt."""
outcomes_str = ", ".join(outcomes)
return f"""As a superforecaster, analyze the following prediction market:
**Question**: {question}
**Description**: {description}
**Possible Outcomes**: {outcomes_str}
Please use the following systematic approach:
### 1. Problem Decomposition
- Break the question into smaller, more manageable parts
- Identify key components needed to answer the question
### 2. Information Gathering
- Consider relevant quantitative data and qualitative insights
- Think about the latest relevant news and expert analysis
### 3. Base Rate
- Use statistical baselines or historical averages as starting points
- Compare the current situation with similar historical events
### 4. Factor Assessment
- List factors that may influence the outcome
- Assess each factor's impact, considering both positive and negative factors
- Weigh these factors using evidence
### 5. Probabilistic Thinking
- Express predictions as probabilities, not certainties
- Assign likelihoods to different outcomes
- Acknowledge uncertainty
Please provide probability estimates for each outcome, ensuring all probabilities sum to 100%.
Output format:
```json
{{
"analysis": "your detailed analysis",
"probabilities": {{
"outcome1": 0.XX,
"outcome2": 0.XX
}},
"confidence_level": "low/medium/high",
"key_factors": ["factor1", "factor2", "factor3"]
}}
```"""
@staticmethod
def quick_decision_prompt(trade_summary: str) -> str:
"""Quick decision prompt for time-sensitive situations."""
return f"""Quickly analyze the following whale trade and provide a recommendation:
{trade_summary}
Output your decision in JSON format:
```json
{{
"action": "BUY/SELL/HOLD",
"outcome": "the outcome to trade on",
"confidence": 0.0-1.0,
"reasoning": "one-line reason"
}}
```"""
src/services/__init__.py
+14
View File
@@ -0,0 +1,14 @@
"""Services module."""
from .market_fetcher import MarketFetcher
from .trade_monitor import TradeMonitor
from .anomaly_detector import AnomalyDetector
from .llm_analyzer import LLMAnalyzer
from .twitter_search import TwitterSearchService
__all__ = [
"MarketFetcher",
"TradeMonitor",
"AnomalyDetector",
"LLMAnalyzer",
"TwitterSearchService",
]
src/services/anomaly_detector.py
+391
View File
@@ -0,0 +1,391 @@
"""
Anomaly detection service — confidence scoring for whale trades.
Mirrors the options flow confidence scoring from llm-trading-agent:
Base confidence: 0.50
+ Premium-to-threshold ratio: +0.20 (sqrt-scaled by market liquidity)
+ Signal cleanliness (ask ratio): +0.10
+ Volume/OI equivalent (depth ratio): +0.10
+ Alert rule / cluster tier: +0.10
Total max = 1.0. Pre-filter threshold = 0.60 (matches options flow pipeline).
"""
import logging
import math
import time
from collections import defaultdict, deque
from datetime import datetime
from typing import Dict, List, Optional, Tuple
from src.config import get_settings
from src.models.trade import WhaleTrade, TradeActivity, TraderHistory
from src.models.market import Market
from src.services.trader_profiler import TraderProfiler
logger = logging.getLogger(__name__)
class AnomalyDetector:
"""
Confidence scoring for whale trades, mirroring options flow pipeline.
5-factor scoring (same structure as OptionsFlowSignalProvider._calculate_confidence):
1. Base confidence: 0.50
2. Premium-to-threshold ratio: +0.20 (trade_size vs dynamic threshold)
3. Signal cleanliness: +0.10 (conviction / price displacement)
4. Depth ratio: +0.10 (trade_size vs market liquidity, like Volume/OI)
5. Cluster tier: +0.10 (repeated same-direction trades, like alert_rule)
"""
# Cluster detection
_CLUSTER_WINDOW_SECONDS = 300 # 5 minutes
_CLUSTER_MIN_COUNT = 3
def __init__(self):
self.settings = get_settings()
self.trader_profiler = TraderProfiler()
self._recent_trades: Dict[str, deque] = defaultdict(
lambda: deque(maxlen=50)
)
# ================================================================
# Core scoring — mirrors OptionsFlowSignalProvider._calculate_confidence
# ================================================================
def get_anomaly_score(
self,
activity: TradeActivity,
market: Optional[Market] = None,
trader_history: Optional[TraderHistory] = None,
market_id: str = "",
) -> Tuple[float, dict]:
"""
Calculate confidence score using options-flow-style 5-factor model.
Returns:
(total_score, breakdown_dict) — total in [0.0, 1.0].
"""
breakdown = {}
# --- 1. Base confidence: 0.50 ---
breakdown["base"] = 0.50
# --- 2. Premium-to-threshold ratio: +0.20 max ---
# Mirrors _premium_ratio_bonus: min(0.20, (sqrt(ratio) - 1) * 0.20)
# where ratio = trade_size / dynamic_threshold
# dynamic_threshold = base_size * sqrt(market_volume / baseline_volume)
breakdown["premium_ratio"] = self._premium_ratio_bonus(activity, market)
# --- 3. Signal cleanliness (conviction): +0.10 max ---
# Mirrors _ask_ratio_bonus: measures taker aggressiveness.
# In options: ASK-side ratio > 0.8 = full bonus.
# In Polymarket: price displacement from market mid = conviction.
breakdown["signal_clean"] = self._signal_cleanliness_bonus(activity, market)
# --- 4. Depth ratio (like Volume/OI): +0.10 max ---
# Mirrors _volume_oi_bonus: trade_size / market_liquidity.
# High ratio = new significant positioning.
breakdown["depth_ratio"] = self._depth_ratio_bonus(activity, market)
# --- 5. Cluster tier (like alert_rule): +0.10 max ---
# Mirrors _alert_rule_bonus: repeated same-direction activity.
# In options: RepeatedHitsAscendingFill = 0.10, RepeatedHits = 0.07.
# In Polymarket: multiple same-direction trades in 5-min window.
breakdown["cluster_tier"] = self._cluster_tier_bonus(activity, market_id)
# --- Total ---
total = sum(breakdown.values())
total = min(1.0, max(0.0, total))
return total, breakdown
@staticmethod
def _premium_ratio_bonus(activity: TradeActivity, market: Optional[Market]) -> float:
"""
Trade size vs dynamic threshold, sqrt-scaled.
Mirrors OptionsFlowSignalProvider._premium_ratio_bonus:
threshold = 100K * sqrt(market_cap / 40B)
bonus = min(0.20, (sqrt(premium / threshold) - 1) * 0.20)
Polymarket mapping:
threshold = base_size * sqrt(market_volume / baseline_volume)
base_size = $5,000, baseline_volume = $1,000,000
"""
max_bonus = 0.20
trade_size = activity.usdc_size
if trade_size <= 0:
return 0.0
# Dynamic threshold based on market volume (like market-cap scaling)
base_size = 10_000.0
baseline_volume = 1_000_000.0
if market and market.volume > 0:
threshold = base_size * math.sqrt(market.volume / baseline_volume)
threshold = max(5_000.0, threshold) # floor $5K
else:
threshold = base_size
ratio = trade_size / threshold
if ratio <= 1.0:
return 0.0
bonus = (math.sqrt(ratio) - 1) * max_bonus
return min(max_bonus, max(0.0, bonus))
@staticmethod
def _signal_cleanliness_bonus(activity: TradeActivity, market: Optional[Market]) -> float:
"""
Taker conviction / price displacement from market mid.
Mirrors _ask_ratio_bonus logic:
ASK ratio > 0.8 → 0.10 (full), > 0.6 → 0.05 (half)
Polymarket mapping:
A buyer paying significantly above market mid = aggressive taker (like ASK-side).
Displacement > 5% → 0.10, > 2% → 0.05.
"""
if not market or not market.outcome_prices:
return 0.0
# Get market mid price for the outcome the trader bought
if activity.outcome == "Yes":
market_mid = market.outcome_prices[0]
elif len(market.outcome_prices) > 1:
market_mid = market.outcome_prices[1]
else:
market_mid = 1.0 - market.outcome_prices[0]
displacement = activity.price - market_mid
if displacement > 0.05:
return 0.10 # strong conviction (like ASK ratio > 0.8)
if displacement > 0.02:
return 0.05 # moderate conviction (like ASK ratio > 0.6)
return 0.0
@staticmethod
def _depth_ratio_bonus(activity: TradeActivity, market: Optional[Market]) -> float:
"""
Trade size vs market liquidity (like Volume/OI).
Mirrors _volume_oi_bonus:
V/OI > 3.0 → 0.10, > 1.5 → 0.07, > 1.0 → 0.03
Polymarket mapping:
depth_ratio = trade_size / market_liquidity
> 0.10 → 0.10, > 0.05 → 0.07, > 0.02 → 0.03
"""
if not market or not market.liquidity or market.liquidity <= 0:
return 0.0
ratio = activity.usdc_size / market.liquidity
if ratio > 0.10:
return 0.10
if ratio > 0.05:
return 0.07
if ratio > 0.02:
return 0.03
return 0.0
def _cluster_tier_bonus(self, activity: TradeActivity, market_id: str = "") -> float:
"""
Cluster of same-direction trades in short window (like alert_rule tiers).
Mirrors _alert_rule_bonus tier structure:
RepeatedHitsAscendingFill → 0.10
RepeatedHits → 0.07
SweepsFollowedByFloor → 0.05
Single sweep → 0.03
Polymarket mapping:
5+ same-direction trades in 5min → 0.10
3-4 trades → 0.07
2 trades with large volume → 0.03
"""
key = market_id or activity.condition_id
recent = self._recent_trades.get(key)
if not recent:
return 0.0
now = activity.timestamp
cutoff = now - self._CLUSTER_WINDOW_SECONDS
same_dir_count = 0
same_dir_volume = 0.0
for ts, side, size in recent:
if ts >= cutoff and side == activity.side:
same_dir_count += 1
same_dir_volume += size
if same_dir_count >= 5:
return 0.10
if same_dir_count >= self._CLUSTER_MIN_COUNT:
return 0.07
if same_dir_count >= 2 and same_dir_volume > 20_000:
return 0.03
return 0.0
def record_trade(self, activity: TradeActivity, market_id: str):
"""Record a trade for cluster detection. Call for every trade, not just whales."""
self._recent_trades[market_id].append((
activity.timestamp,
activity.side,
activity.usdc_size,
))
# ================================================================
# Pre-filter (before LLM)
# ================================================================
def should_analyze(
self,
activity: TradeActivity,
market: Optional[Market] = None,
trader_history: Optional[TraderHistory] = None,
market_id: str = "",
min_score: float = 0.55,
) -> Tuple[bool, float, dict]:
"""
Decide whether a whale trade warrants LLM analysis.
Threshold 0.65: requires at least base (0.50) + one strong factor
to trigger LLM analysis.
"""
score, breakdown = self.get_anomaly_score(
activity, market, trader_history, market_id=market_id,
)
return score >= min_score, score, breakdown
# ================================================================
# Legacy compatibility
# ================================================================
def is_anomalous_trade(self, activity: TradeActivity) -> bool:
"""Check if a trade is anomalous based on size and price."""
if activity.usdc_size < self.settings.min_trade_size_usd:
return False
if not (self.settings.min_price <= activity.price <= self.settings.max_price):
return False
return True
def filter_whale_trades(
self,
trades: List[WhaleTrade],
min_score: float = 0.55,
) -> List[WhaleTrade]:
"""Filter whale trades by confidence score."""
filtered = []
for trade in trades:
score, _ = self.get_anomaly_score(trade.trade)
if score >= min_score:
filtered.append(trade)
return filtered
# ================================================================
# LLM context formatting
# ================================================================
def analyze_trade_context(self, whale_trade: WhaleTrade) -> dict:
"""Analyze the context of a whale trade for LLM input."""
trade = whale_trade.trade
if trade.outcome == "Yes":
direction_meaning = f"Trader bought Yes Token @ {trade.price:.4f} — Bullish (believes event will occur)"
else:
direction_meaning = f"Trader bought No Token @ {trade.price:.4f} — Bearish (believes event will NOT occur)"
implied_prob = trade.price
market_state = "uncertain"
if whale_trade.market_outcome_prices:
max_price = max(whale_trade.market_outcome_prices)
if max_price > 0.7:
market_state = "leaning towards one outcome"
elif max_price < 0.6:
market_state = "highly uncertain"
score, breakdown = self.get_anomaly_score(trade)
return {
"trade_size_usd": trade.usdc_size,
"trade_side": trade.side,
"trade_price": trade.price,
"trade_outcome": trade.outcome,
"direction_meaning": direction_meaning,
"implied_probability": implied_prob,
"market_state": market_state,
"anomaly_score": score,
"anomaly_breakdown": breakdown,
"market_question": whale_trade.market_question,
"market_outcomes": whale_trade.market_outcomes,
"current_prices": whale_trade.market_outcome_prices,
}
def format_for_llm(self, whale_trade: WhaleTrade) -> str:
"""Format whale trade data for LLM analysis."""
context = self.analyze_trade_context(whale_trade)
trade = whale_trade.trade
prices_str = ""
for outcome, price in zip(context["market_outcomes"], context["current_prices"]):
prices_str += f" - {outcome}: {price:.2%}\n"
trader_profile = self.trader_profiler.generate_profile(
wallet_address=trade.proxy_wallet or "Unknown",
ranking=whale_trade.trader_ranking,
history=whale_trade.trader_history,
)
trader_profile_str = self.trader_profiler.format_profile_for_llm(trader_profile)
bd = context["anomaly_breakdown"]
breakdown_str = (
f" Base: {bd.get('base', 0):.2f} | "
f"Premium ratio: {bd.get('premium_ratio', 0):.2f} | "
f"Signal clean: {bd.get('signal_clean', 0):.2f} | "
f"Depth ratio: {bd.get('depth_ratio', 0):.2f} | "
f"Cluster tier: {bd.get('cluster_tier', 0):.2f}"
)
return f"""
## Whale Trade Anomaly Detection Report
### Trade Details
- **Trade amount**: ${context['trade_size_usd']:,.2f} USDC
- **Trade direction**: BUY {context['trade_outcome']} Token ({'Bullish' if context['trade_outcome'] == 'Yes' else 'Bearish'})
- **Buy price**: {context['trade_price']:.4f} (~{1/context['trade_price']:.1f}x odds)
- **Trade time**: {datetime.fromtimestamp(trade.timestamp).strftime('%Y-%m-%d %H:%M:%S UTC')}
- **Trader wallet**: {trade.proxy_wallet or 'Unknown'}
### Confidence Score
- **Overall score**: {context['anomaly_score']:.2f}/1.00
- **Score breakdown**:
{breakdown_str}
### Trade Interpretation
- **Direction**: {context['direction_meaning']}
{trader_profile_str}
### Market Information
- **Market question**: {context['market_question']}
- **Market description**: {whale_trade.market_description or 'N/A'}
- **Market state**: {context['market_state']}
- **Current odds**:
{prices_str}
{whale_trade.format_event_positions()}
{whale_trade.format_top_traders()}
### Analysis Points
1. This is a ${context['trade_size_usd']:,.2f} large trade, direction: **BUY {context['trade_outcome']} Token**
2. {context['direction_meaning']}
3. **Focus on the Trader Profile JSON above — assess trader credibility from ranking, PnL, trading behavior, and recent trades**
4. **Analyze the whale's positions in other markets under the same event** — opposing positions may indicate hedging
5. **Reference the market's Top long/short holders** — which side has the concentration of high-ranked traders
Please analyze the information asymmetry likelihood of this trade.
"""
src/services/anomaly_history.py
+147
View File
@@ -0,0 +1,147 @@
"""Anomaly history service - stores and retrieves historical anomaly signals via SQLite."""
import logging
from typing import List, Optional
from src.db.database import SignalDatabase
from src.models.anomaly_signal import AnomalySignal
logger = logging.getLogger(__name__)
class AnomalyHistoryService:
"""
Service for storing and retrieving historical anomaly signals.
Backend: SQLite via SignalDatabase.
All analyzed signals are stored for tracking accuracy.
Only signals with likelihood >= 0.4 are used as historical context for LLM.
"""
# Minimum information asymmetry score to use as historical context for LLM
MIN_CONTEXT_LIKELIHOOD = 0.4
def __init__(self, db_path: str = "data/signals.db"):
"""
Initialize the anomaly history service.
Args:
db_path: Path to the SQLite database file.
"""
self.db = SignalDatabase(db_path)
def should_store_signal(self, insider_likelihood: float) -> bool:
"""All analyzed signals should be stored for accuracy tracking."""
return True
def store_signal(self, signal: AnomalySignal) -> bool:
"""
Store an anomaly signal for accuracy tracking.
All analyzed signals are stored regardless of likelihood.
"""
stored = self.db.insert_signal(signal)
if stored:
logger.info(
f"Stored signal for market {signal.market_id}: "
f"${signal.trade_size_usd:,.2f} {signal.trade_side} "
f"(IAS: {signal.information_asymmetry_score:.0%})"
)
return stored
def get_signals_for_market(
self,
market_id: str,
top_recent: int = 5,
top_likelihood: int = 5,
) -> List[AnomalySignal]:
"""
Get historical anomaly signals for a market.
Selects the most recent signals and highest insider likelihood signals,
then deduplicates and returns the combined list.
Args:
market_id: The market ID
top_recent: Number of most recent signals to include
top_likelihood: Number of highest insider likelihood signals to include
Returns:
List of AnomalySignal objects (deduplicated, sorted by trade timestamp newest first)
"""
return self.db.get_signals_for_market(market_id, top_recent, top_likelihood)
def format_historical_signals_context(
self,
signals: List[AnomalySignal],
) -> str:
"""
Format historical anomaly signals into a context string for LLM.
Args:
signals: List of historical anomaly signals
Returns:
Formatted string for LLM context
"""
if not signals:
return ""
signal_count = len(signals)
context = f"""
### Historical Anomaly Signals ({signal_count} total)
**Important**: {signal_count} anomalous trades have been previously detected on this market. Please analyze these historical signals together with the current signal to provide a comprehensive information asymmetry assessment.
"""
for i, signal in enumerate(signals, 1):
context += f"""
---
#### Historical Signal {i}
{signal.to_context_string()}
---
"""
context += """
**Comprehensive Analysis Points**:
1. Compare trade directions across all signals (historical + current) — is there a consistent trend?
2. Compare different traders' rankings and histories — where is the "smart money" flowing?
3. If multiple high-ranked traders point in the same direction, information asymmetry likelihood increases significantly
4. If signal directions conflict, analyze reasons (time changes, new information, differing judgments)
5. Consider time factor: more recent signals are more relevant
6. Observe trade amount trends: are amounts increasing?
"""
return context
def get_all_market_ids(self) -> List[str]:
"""
Get all market IDs that have stored anomaly signals.
Returns:
List of market IDs
"""
return self.db.get_all_market_ids()
def get_signal_count(self, market_id: Optional[str] = None) -> int:
"""
Get the number of stored signals.
Args:
market_id: Optional market ID to filter by
Returns:
Number of stored signals
"""
return self.db.get_signal_count(market_id)
def cleanup_old_signals(self, max_age_days: int = 30) -> int:
"""
Remove signals older than the specified number of days.
Args:
max_age_days: Maximum age of signals to keep
Returns:
Number of signals removed
"""
return self.db.cleanup_old_signals(max_age_days)
src/services/coingecko.py
+169
View File
@@ -0,0 +1,169 @@
"""CoinGecko API service for cryptocurrency market data."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
COINGECKO_API = "https://api.coingecko.com/api/v3"
# Common coin ID mapping (Polymarket markets often use ticker symbols)
TICKER_TO_ID = {
"BTC": "bitcoin",
"ETH": "ethereum",
"SOL": "solana",
"XRP": "ripple",
"DOGE": "dogecoin",
"ADA": "cardano",
"AVAX": "avalanche-2",
"DOT": "polkadot",
"MATIC": "matic-network",
"LINK": "chainlink",
"UNI": "uniswap",
"SHIB": "shiba-inu",
"LTC": "litecoin",
"BNB": "binancecoin",
"NEAR": "near",
"ARB": "arbitrum",
"OP": "optimism",
"APT": "aptos",
"SUI": "sui",
"PEPE": "pepe",
}
def _resolve_coin_id(query: str) -> str:
"""Resolve a ticker or name to a CoinGecko coin ID."""
q = query.strip().upper()
if q in TICKER_TO_ID:
return TICKER_TO_ID[q]
# Try lowercase as-is (CoinGecko IDs are lowercase)
return query.strip().lower()
class CoinGeckoService:
"""
CoinGecko API client for crypto market data.
Free tier: 30 calls/min, no API key required.
"""
def __init__(self):
self._client = get_client(timeout=15.0)
def get_price(self, coin: str) -> str:
"""
Get current price, 24h change, market cap, and volume for a cryptocurrency.
Args:
coin: Ticker symbol (BTC, ETH, SOL) or CoinGecko ID (bitcoin, ethereum)
Returns:
Formatted price report string.
"""
coin_id = _resolve_coin_id(coin)
try:
resp = self._client.get(
f"{COINGECKO_API}/coins/{coin_id}",
params={
"localization": "false",
"tickers": "false",
"community_data": "false",
"developer_data": "false",
"sparkline": "false",
},
)
if resp.status_code == 404:
return f"Coin '{coin}' (id: {coin_id}) not found on CoinGecko."
resp.raise_for_status()
data = resp.json()
market = data.get("market_data", {})
name = data.get("name", coin_id)
symbol = data.get("symbol", "").upper()
price = market.get("current_price", {}).get("usd")
change_24h = market.get("price_change_percentage_24h")
change_7d = market.get("price_change_percentage_7d")
change_30d = market.get("price_change_percentage_30d")
high_24h = market.get("high_24h", {}).get("usd")
low_24h = market.get("low_24h", {}).get("usd")
market_cap = market.get("market_cap", {}).get("usd")
volume_24h = market.get("total_volume", {}).get("usd")
ath = market.get("ath", {}).get("usd")
ath_change = market.get("ath_change_percentage", {}).get("usd")
lines = [
f"--- {name} ({symbol}) Market Data ---",
f"Price: ${price:,.2f}" if price else "Price: N/A",
]
if high_24h and low_24h:
lines.append(f"24h Range: ${low_24h:,.2f} - ${high_24h:,.2f}")
if change_24h is not None:
lines.append(f"24h Change: {change_24h:+.2f}%")
if change_7d is not None:
lines.append(f"7d Change: {change_7d:+.2f}%")
if change_30d is not None:
lines.append(f"30d Change: {change_30d:+.2f}%")
if market_cap:
lines.append(f"Market Cap: ${market_cap:,.0f}")
if volume_24h:
lines.append(f"24h Volume: ${volume_24h:,.0f}")
if ath and ath_change is not None:
lines.append(f"ATH: ${ath:,.2f} ({ath_change:+.1f}% from ATH)")
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"CoinGecko API error for '{coin}': {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"CoinGecko query failed for '{coin}': {e}"
logger.error(msg)
return msg
def get_market_overview(self) -> str:
"""
Get global crypto market overview: total market cap, BTC dominance, etc.
Returns:
Formatted global market overview string.
"""
try:
resp = self._client.get(f"{COINGECKO_API}/global")
resp.raise_for_status()
data = resp.json().get("data", {})
total_cap = data.get("total_market_cap", {}).get("usd", 0)
total_vol = data.get("total_volume", {}).get("usd", 0)
btc_dom = data.get("market_cap_percentage", {}).get("btc", 0)
eth_dom = data.get("market_cap_percentage", {}).get("eth", 0)
change_24h = data.get("market_cap_change_percentage_24h_usd", 0)
active_coins = data.get("active_cryptocurrencies", 0)
lines = [
"--- Global Crypto Market Overview ---",
f"Total Market Cap: ${total_cap:,.0f}",
f"24h Change: {change_24h:+.2f}%",
f"24h Volume: ${total_vol:,.0f}",
f"BTC Dominance: {btc_dom:.1f}%",
f"ETH Dominance: {eth_dom:.1f}%",
f"Active Coins: {active_coins:,}",
"---",
]
return "\n".join(lines)
except Exception as e:
msg = f"CoinGecko global query failed: {e}"
logger.error(msg)
return msg
src/services/congress.py
+205
View File
@@ -0,0 +1,205 @@
"""Congress.gov API service for U.S. legislative data."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
CONGRESS_API = "https://api.congress.gov/v3"
class CongressService:
"""
Congress.gov API client for U.S. legislative data.
Covers: bills, votes, members, committees, nominations.
"""
def __init__(self, api_key: str):
self.api_key = api_key
self._client = get_client(timeout=15.0)
def is_available(self) -> bool:
return bool(self.api_key and self.api_key.strip())
def _get(self, path: str, params: Optional[dict] = None) -> dict:
"""Make authenticated GET request."""
params = params or {}
params["api_key"] = self.api_key
params["format"] = "json"
resp = self._client.get(f"{CONGRESS_API}{path}", params=params)
resp.raise_for_status()
return resp.json()
def search_bills(self, query: str, limit: int = 5) -> str:
"""
Search for bills by keyword.
Args:
query: Search keywords (e.g. 'TikTok ban', 'crypto regulation', 'immigration')
limit: Number of results (1-10)
Returns:
Formatted report of matching bills with status.
"""
limit = max(1, min(limit, 10))
try:
data = self._get("/bill", params={
"limit": limit,
"sort": "updateDate+desc",
})
bills = data.get("bills", [])
if not bills:
return f"No bills found on Congress.gov."
# Filter by query keyword in title (API doesn't support text search directly)
# So we fetch recent bills and note to user
lines = [f"--- Congress.gov: Recent Bills ---"]
lines.append(f"(Showing {len(bills)} most recently updated bills)")
lines.append("")
for i, bill in enumerate(bills, 1):
bill_type = bill.get("type", "")
number = bill.get("number", "")
title = bill.get("title", "No title")
congress = bill.get("congress", "")
update_date = bill.get("updateDate", "")[:10]
latest_action = bill.get("latestAction", {})
action_text = latest_action.get("text", "")
action_date = latest_action.get("actionDate", "")
bill_id = f"{bill_type} {number}" if bill_type and number else "N/A"
lines.append(f"{i}. **{bill_id}** (Congress {congress})")
lines.append(f" Title: {title[:150]}")
lines.append(f" Updated: {update_date}")
if action_text:
lines.append(f" Latest Action ({action_date}): {action_text[:150]}")
lines.append("")
lines.append("---")
return "\n".join(lines)
except Exception as e:
msg = f"Congress.gov search failed: {e}"
logger.error(msg)
return msg
def get_bill_status(self, congress: int, bill_type: str, bill_number: int) -> str:
"""
Get detailed status of a specific bill.
Args:
congress: Congress number (e.g. 119 for current)
bill_type: Bill type (hr, s, hjres, sjres)
bill_number: Bill number
Returns:
Formatted bill status report.
"""
bt = bill_type.strip().lower()
try:
data = self._get(f"/bill/{congress}/{bt}/{bill_number}")
bill = data.get("bill", {})
if not bill:
return f"Bill {bt.upper()} {bill_number} (Congress {congress}) not found."
title = bill.get("title", "No title")
introduced = bill.get("introducedDate", "N/A")
sponsors = bill.get("sponsors", [])
sponsor_str = ", ".join(
f"{s.get('firstName', '')} {s.get('lastName', '')} ({s.get('party', '')}-{s.get('state', '')})"
for s in sponsors[:3]
) if sponsors else "N/A"
latest_action = bill.get("latestAction", {})
action_text = latest_action.get("text", "N/A")
action_date = latest_action.get("actionDate", "")
policy_area = bill.get("policyArea", {}).get("name", "N/A")
committees_count = bill.get("committees", {}).get("count", 0)
cosponsors_count = bill.get("cosponsors", {}).get("count", 0)
actions_count = bill.get("actions", {}).get("count", 0)
# Determine bill progress
constitutional = bill.get("constitutionalAuthorityStatementText", "")
lines = [
f"--- Bill Status: {bt.upper()} {bill_number} (Congress {congress}) ---",
f"Title: {title}",
f"Introduced: {introduced}",
f"Sponsor: {sponsor_str}",
f"Cosponsors: {cosponsors_count}",
f"Policy Area: {policy_area}",
f"Committees Referred: {committees_count}",
f"Total Actions: {actions_count}",
f"",
f"Latest Action ({action_date}): {action_text}",
f"---",
]
return "\n".join(lines)
except httpx.HTTPStatusError as e:
if e.response.status_code == 404:
return f"Bill {bt.upper()} {bill_number} (Congress {congress}) not found."
return f"Congress.gov API error: HTTP {e.response.status_code}"
except Exception as e:
msg = f"Congress.gov bill query failed: {e}"
logger.error(msg)
return msg
def get_recent_votes(self, chamber: str = "senate", limit: int = 5) -> str:
"""
Get recent roll call votes.
Args:
chamber: 'senate' or 'house'
limit: Number of votes (1-10)
Returns:
Formatted report of recent votes.
"""
chamber = chamber.strip().lower()
if chamber not in ("senate", "house"):
chamber = "senate"
limit = max(1, min(limit, 10))
try:
# Get current congress number (119th as of 2025-2026)
congress = 119
data = self._get(f"/bill", params={
"limit": limit,
"sort": "updateDate+desc",
})
# Use the nominations endpoint for Senate votes
# or fall back to recent bill actions
lines = [f"--- Recent Congressional Activity ({chamber.title()}) ---"]
bills = data.get("bills", [])
for i, bill in enumerate(bills[:limit], 1):
bill_type = bill.get("type", "")
number = bill.get("number", "")
title = bill.get("title", "")[:100]
latest = bill.get("latestAction", {})
action = latest.get("text", "")[:120]
date = latest.get("actionDate", "")
lines.append(f"{i}. {bill_type} {number}: {title}")
lines.append(f" {date}: {action}")
lines.append("")
lines.append("---")
return "\n".join(lines)
except Exception as e:
msg = f"Congress.gov votes query failed: {e}"
logger.error(msg)
return msg
src/services/daily_briefing.py
+390
View File
@@ -0,0 +1,390 @@
"""Daily briefing service - generates daily summary of high-value signals."""
import logging
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from datetime import datetime, timedelta
from pathlib import Path
from typing import List, Dict, Optional
from src.config import get_settings
from src.db.database import SignalDatabase
from src.services.stats_engine import StatsEngine
logger = logging.getLogger(__name__)
# Directories
VOLATILITY_DIR = Path(__file__).parent.parent.parent / "price_volatility"
BRIEFINGS_DIR = Path(__file__).parent.parent.parent / "daily_briefings"
class DailyBriefingGenerator:
"""
Generates daily briefings summarizing high-value signals.
Includes:
- Smart money signals with information asymmetry score >= 60%
- Price volatility alerts
- Historical signal performance stats
"""
# Minimum information asymmetry score to include in briefing
MIN_IAS = 0.6 # 60%
# Maximum signals to include when falling back to top-N
FALLBACK_TOP_N = 5
def __init__(self, db_path: str = "data/signals.db"):
"""Initialize the briefing generator."""
BRIEFINGS_DIR.mkdir(parents=True, exist_ok=True)
self.db = SignalDatabase(db_path)
self.stats_engine = StatsEngine(self.db)
def _get_date_range(self, date: datetime) -> tuple:
"""
Get start and end timestamps for a given date.
Args:
date: The date to get range for
Returns:
Tuple of (start_timestamp, end_timestamp)
"""
start = datetime(date.year, date.month, date.day, 0, 0, 0)
end = start + timedelta(days=1)
return int(start.timestamp()), int(end.timestamp())
def _load_insider_signals(self, date: datetime) -> tuple:
"""
Load smart money signals for a specific date from the database.
First tries to find signals with likelihood >= 60%.
If none found, falls back to the top 5 by likelihood.
Args:
date: The date to load signals for
Returns:
Tuple of (signals list as dicts, is_fallback bool)
"""
date_str = date.strftime("%Y-%m-%d")
# Query all signals detected on this date
all_signals = self.db.get_all_signals(limit=500, offset=0)
day_signals = []
for signal in all_signals:
if signal.detected_at.strftime("%Y-%m-%d") == date_str:
day_signals.append(signal)
if not day_signals:
return [], False
# Sort by likelihood descending
day_signals.sort(key=lambda s: s.information_asymmetry_score, reverse=True)
# Convert to dicts for backward compat with _format_briefing
def signal_to_dict(s):
return {
"market_id": s.market_id,
"market_question": s.market_question,
"transaction_hash": s.transaction_hash,
"trade_size_usd": s.trade_size_usd,
"trade_price": s.trade_price,
"trade_outcome": s.trade_outcome,
"information_asymmetry_score": s.information_asymmetry_score,
"reasoning": s.reasoning,
"insider_evidence": s.insider_evidence,
"detected_at": s.detected_at.isoformat(),
}
# Filter high-likelihood signals
high_likelihood = [
signal_to_dict(s) for s in day_signals
if s.information_asymmetry_score >= self.MIN_IAS
]
if high_likelihood:
return high_likelihood, False
# Fallback: top N signals by likelihood
return [signal_to_dict(s) for s in day_signals[:self.FALLBACK_TOP_N]], True
def _load_volatility_alerts(self, date: datetime) -> List[Dict]:
"""
Load price volatility alerts for a specific date.
Args:
date: The date to load alerts for
Returns:
List of volatility alerts
"""
import json
alerts_file = VOLATILITY_DIR / "volatility_alerts.json"
date_str = date.strftime("%Y-%m-%d")
if not alerts_file.exists():
return []
try:
with open(alerts_file, 'r', encoding='utf-8') as f:
all_alerts = json.load(f)
# Filter alerts for the target date
day_alerts = [
alert for alert in all_alerts
if alert.get("detected_at", "").startswith(date_str)
]
# Sort by price change magnitude descending
day_alerts.sort(
key=lambda x: abs(x.get("price_change_percent", 0)),
reverse=True
)
return day_alerts
except Exception as e:
logger.error(f"Error loading volatility alerts: {e}")
return []
def _format_briefing(
self,
date: datetime,
insider_signals: List[Dict],
volatility_alerts: List[Dict],
is_fallback: bool = False,
) -> str:
"""
Format the daily briefing as markdown.
Args:
date: The date of the briefing
insider_signals: List of insider signals
volatility_alerts: List of price volatility alerts
is_fallback: True if signals are fallback (none >= 60%)
Returns:
Formatted markdown briefing
"""
date_str = date.strftime("%Y-%m-%d")
lines = [
f"# Daily Signal Briefing - {date_str}",
"",
f"Generated at: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
"",
]
# Summary stats
if is_fallback:
summary_line = f"- No signals with confidence >= 60% today; showing the top **{len(insider_signals)}** by confidence"
else:
summary_line = f"- High-confidence information asymmetry signals: **{len(insider_signals)}** (confidence >= 60%)"
lines.extend([
"## Today's Overview",
"",
summary_line,
f"- Abnormal price volatility events: **{len(volatility_alerts)}**",
"",
])
# Insider trading signals section
if is_fallback:
section_title = "## Today's Top Anomalous Trades by Confidence"
else:
section_title = "## High-Confidence Information Asymmetry Signals"
lines.extend([
"---",
"",
section_title,
"",
])
if insider_signals:
for i, signal in enumerate(insider_signals, 1):
likelihood = signal.get("information_asymmetry_score", 0)
market_question = signal.get("market_question", "Unknown")
trade_size = signal.get("trade_size_usd", 0)
trade_price = signal.get("trade_price", 0)
trade_outcome = signal.get("trade_outcome", "Yes")
reasoning = signal.get("reasoning", "")
insider_evidence = signal.get("insider_evidence", "")
detected_at = signal.get("detected_at", "")
# Odds calculation
odds_str = f"{1/trade_price:.1f}x" if trade_price > 0 else "N/A"
lines.extend([
f"### {i}. {market_question[:80]}{'...' if len(market_question) > 80 else ''}",
"",
f"| Metric | Value |",
f"|--------|-------|",
f"| Info Asymmetry | **{likelihood:.0%}** |",
f"| Direction | BUY {trade_outcome} Token ({'Bullish' if trade_outcome == 'Yes' else 'Bearish'}) |",
f"| Entry Price | {trade_price:.4f} (Odds {odds_str}) |",
f"| Trade Size | **${trade_size:,.0f}** USDC |",
f"| Detected At | {detected_at} |",
"",
])
if reasoning:
lines.extend([
f"**Analysis**: {reasoning}",
"",
])
if insider_evidence:
lines.extend([
f"**Insider Evidence**: {insider_evidence}",
"",
])
lines.append("")
else:
lines.extend([
"*No anomalous trade signals today*",
"",
])
# Volatility alerts section
lines.extend([
"---",
"",
"## Abnormal Price Volatility",
"",
])
if volatility_alerts:
lines.extend([
"| Market | Direction | Change | Start Price | End Price | Detected At |",
"|--------|-----------|--------|-------------|-----------|-------------|",
])
for alert in volatility_alerts:
market_question = alert.get("market_question", "Unknown")
# Truncate long market questions
if len(market_question) > 40:
market_question = market_question[:37] + "..."
direction = "Down" if alert.get("direction") == "DOWN" else "Up"
price_change = abs(alert.get("price_change_percent", 0))
start_price = alert.get("start_price", 0)
end_price = alert.get("end_price", 0)
detected_at = alert.get("detected_at", "")[:16] # Trim to minute
lines.append(
f"| {market_question} | {direction} | {price_change:.1%} | "
f"{start_price:.2%} | {end_price:.2%} | {detected_at} |"
)
lines.append("")
else:
lines.extend([
"*No abnormal price volatility today*",
"",
])
# Signal performance stats section
stats_summary = self.stats_engine.format_stats_summary()
if stats_summary:
lines.extend([
"---",
"",
stats_summary,
])
# Footer
lines.extend([
"---",
"",
"*This briefing was automatically generated by Polymarket Whale Watcher*",
])
return "\n".join(lines)
def generate_briefing(self, date: Optional[datetime] = None) -> Optional[str]:
"""
Generate daily briefing for a specific date.
Args:
date: The date to generate briefing for (defaults to yesterday)
Returns:
Path to the saved briefing file, or None if no signals
"""
if date is None:
# Default to yesterday
date = datetime.now() - timedelta(days=1)
date_str = date.strftime("%Y-%m-%d")
logger.info(f"Generating daily briefing for {date_str}")
# Load signals
insider_signals, is_fallback = self._load_insider_signals(date)
volatility_alerts = self._load_volatility_alerts(date)
# Check if there's anything to report
if not insider_signals and not volatility_alerts:
logger.info(f"No signals for {date_str}, skipping briefing")
return None
# Generate briefing
briefing_content = self._format_briefing(date, insider_signals, volatility_alerts, is_fallback)
# Save to file
filename = f"briefing_{date_str}.md"
filepath = BRIEFINGS_DIR / filename
with open(filepath, 'w', encoding='utf-8') as f:
f.write(briefing_content)
logger.info(
f"Daily briefing saved to {filepath} "
f"({len(insider_signals)} insider signals, {len(volatility_alerts)} volatility alerts)"
)
# Send email notification
self._send_email(date_str, briefing_content)
return str(filepath)
def _send_email(self, date_str: str, content: str) -> None:
"""Send briefing via email if configured."""
settings = get_settings()
if not settings.email_enabled:
return
if not settings.email_sender or not settings.email_password:
logger.warning("Email enabled but sender/password not configured, skipping")
return
try:
recipients = [r.strip() for r in settings.email_recipient.split(",") if r.strip()]
msg = MIMEMultipart("alternative")
msg["Subject"] = f"Polymarket Whale Daily Briefing - {date_str}"
msg["From"] = settings.email_sender
msg["To"] = ", ".join(recipients)
# Markdown content as plain text
text_part = MIMEText(content, "plain", "utf-8")
msg.attach(text_part)
with smtplib.SMTP_SSL(settings.email_smtp_server, settings.email_smtp_port) as server:
server.login(settings.email_sender, settings.email_password)
server.sendmail(settings.email_sender, recipients, msg.as_string())
logger.info(f"Daily briefing emailed to {', '.join(recipients)}")
except Exception as e:
logger.error(f"Failed to send briefing email: {e}")
def generate_today_briefing(self) -> Optional[str]:
"""
Generate briefing for today (useful for testing or end-of-day summary).
Returns:
Path to the saved briefing file, or None if no signals
"""
return self.generate_briefing(datetime.now())
src/services/ddg_search.py
+62
View File
@@ -0,0 +1,62 @@
"""DuckDuckGo web search service — free, no API key required."""
import logging
logger = logging.getLogger(__name__)
class DDGSearchService:
"""Web search service using duckduckgo-search (no API key needed)."""
def __init__(self):
self._available: bool | None = None
def is_available(self) -> bool:
if self._available is None:
try:
from duckduckgo_search import DDGS # noqa: F401
self._available = True
except ImportError:
logger.warning(
"duckduckgo-search not installed. "
"Install with: pip install duckduckgo-search"
)
self._available = False
return self._available
def search(self, query: str, max_results: int = 5) -> str:
if not self.is_available():
return "Web search unavailable: duckduckgo-search package not installed."
try:
from duckduckgo_search import DDGS
with DDGS() as ddgs:
results = list(ddgs.text(query, max_results=max_results))
if not results:
return f"No web search results found for '{query}'."
report = [f"--- Web Search Results for '{query}' ---"]
for idx, item in enumerate(results, 1):
title = item.get("title", "No title")
url = item.get("href", "")
body = item.get("body", "")[:300]
if len(item.get("body", "")) > 300:
body += "..."
report.append(f"{idx}. **{title}**")
report.append(f" Source: {url}")
report.append(f" {body}")
report.append("")
report.append("-------------------------------------------")
return "\n".join(report)
except Exception as e:
logger.error(f"DuckDuckGo search failed: {e}")
return f"Web search failed: {str(e)}"
def search_for_market(self, market_question: str, max_results: int = 5) -> str:
query = market_question[:200]
result = self.search(query, max_results=max_results)
if "No web search results" not in result and "Error" not in result and "unavailable" not in result:
return "## 🔍 Web Search Results (News & Analysis)\n" + result
return f"No relevant web results found for: {market_question[:50]}..."
src/services/defillama.py
+340
View File
@@ -0,0 +1,340 @@
"""DeFiLlama API service for DeFi protocol data (TVL, revenue, token unlocks)."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
DEFILLAMA_API = "https://api.llama.fi"
def _fmt_usd(value) -> str:
"""Format a dollar value with appropriate suffix."""
if value is None:
return "N/A"
if isinstance(value, list):
return "N/A"
try:
value = float(value)
except (TypeError, ValueError):
return "N/A"
abs_val = abs(value)
if abs_val >= 1_000_000_000:
return f"${value / 1_000_000_000:,.2f}B"
if abs_val >= 1_000_000:
return f"${value / 1_000_000:,.2f}M"
if abs_val >= 1_000:
return f"${value / 1_000:,.2f}K"
return f"${value:,.2f}"
class DefiLlamaService:
"""
DeFiLlama API client for DeFi protocol analytics.
Free API, no key required. Rate limits are generous.
"""
def __init__(self):
self._client = get_client(timeout=20.0)
self._protocols_cache: Optional[list] = None
@staticmethod
def is_available() -> bool:
"""Always available - no API key needed."""
return True
# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
def _fetch_protocols_list(self) -> list:
"""Fetch and cache the full protocols list for slug lookups."""
if self._protocols_cache is not None:
return self._protocols_cache
try:
resp = self._client.get(f"{DEFILLAMA_API}/protocols")
resp.raise_for_status()
self._protocols_cache = resp.json()
return self._protocols_cache
except Exception as e:
logger.error(f"Failed to fetch DeFiLlama protocols list: {e}")
return []
def _resolve_slug(self, query: str) -> Optional[str]:
"""
Fuzzy-match a user query to a DeFiLlama protocol slug.
Tries exact slug match, then name match, then substring match.
"""
q = query.strip().lower()
protocols = self._fetch_protocols_list()
# 1) Exact slug match
for p in protocols:
if p.get("slug", "").lower() == q:
return p["slug"]
# 2) Exact name match (case-insensitive)
for p in protocols:
if p.get("name", "").lower() == q:
return p["slug"]
# 3) Substring match on slug or name – prefer shortest match (most specific)
candidates = []
for p in protocols:
slug = p.get("slug", "").lower()
name = p.get("name", "").lower()
if q in slug or q in name:
candidates.append(p)
if candidates:
# Sort by TVL descending so the most prominent protocol wins ties
candidates.sort(key=lambda p: p.get("tvl") or 0, reverse=True)
return candidates[0]["slug"]
return None
# ------------------------------------------------------------------
# Public methods – all return formatted strings
# ------------------------------------------------------------------
def get_protocol_tvl(self, protocol: str) -> str:
"""
Get protocol TVL, TVL changes, and chain breakdown.
Args:
protocol: Protocol name or slug (e.g., "aave", "Lido", "uniswap").
Returns:
Formatted TVL report string.
"""
slug = self._resolve_slug(protocol)
if slug is None:
return f"Protocol '{protocol}' not found on DeFiLlama."
try:
resp = self._client.get(f"{DEFILLAMA_API}/protocol/{slug}")
if resp.status_code == 404:
return f"Protocol '{protocol}' (slug: {slug}) not found on DeFiLlama."
resp.raise_for_status()
data = resp.json()
name = data.get("name", slug)
symbol = data.get("symbol", "")
category = data.get("category", "N/A")
# tvl field is a historical list; get current TVL from last entry or currentChainTvls
tvl_data = data.get("tvl")
if isinstance(tvl_data, list) and tvl_data:
tvl = tvl_data[-1].get("totalLiquidityUSD", 0)
elif isinstance(tvl_data, (int, float)):
tvl = tvl_data
else:
tvl = None
chain_tvls = data.get("chainTvls", {})
# TVL changes
change_1h = data.get("change_1h")
change_1d = data.get("change_1d")
change_7d = data.get("change_7d")
lines = [
f"--- {name} ({symbol}) TVL Report ---",
f"Category: {category}",
f"Total TVL: {_fmt_usd(tvl)}",
]
if change_1h is not None:
lines.append(f"1h Change: {change_1h:+.2f}%")
if change_1d is not None:
lines.append(f"24h Change: {change_1d:+.2f}%")
if change_7d is not None:
lines.append(f"7d Change: {change_7d:+.2f}%")
# Chain breakdown – show top chains by TVL
if chain_tvls:
# chainTvls has sub-objects; the latest TVL per chain is the last entry
chain_summary = {}
for chain_name, chain_data in chain_tvls.items():
# Skip aggregated keys like "staking", "borrowed", "pool2"
if "-" in chain_name or chain_name in ("staking", "borrowed", "pool2", "vesting"):
continue
if isinstance(chain_data, dict):
tvl_history = chain_data.get("tvl", [])
if tvl_history:
chain_summary[chain_name] = tvl_history[-1].get("totalLiquidityUSD", 0)
elif isinstance(chain_data, (int, float)):
chain_summary[chain_name] = chain_data
if chain_summary:
sorted_chains = sorted(chain_summary.items(), key=lambda x: x[1], reverse=True)
lines.append("Chain Breakdown:")
for chain_name, chain_tvl in sorted_chains[:10]:
lines.append(f" {chain_name}: {_fmt_usd(chain_tvl)}")
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"DeFiLlama API error for '{protocol}': {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"DeFiLlama TVL query failed for '{protocol}': {e}"
logger.error(msg)
return msg
def get_token_unlocks(self, protocol: str) -> str:
"""
Get token unlock/vesting schedule for a protocol.
Args:
protocol: Protocol name or slug.
Returns:
Formatted token unlock schedule string.
"""
slug = self._resolve_slug(protocol)
if slug is None:
return f"Protocol '{protocol}' not found on DeFiLlama."
try:
resp = self._client.get(f"{DEFILLAMA_API}/api/emission/{slug}")
if resp.status_code == 404:
return f"No token unlock data for '{protocol}' on DeFiLlama."
resp.raise_for_status()
data = resp.json()
name = data.get("name", slug)
token_price = data.get("tokenPrice", {})
categories = data.get("categories", {})
events = data.get("events", [])
lines = [f"--- {name} Token Unlock Schedule ---"]
# Token price info
if isinstance(token_price, dict):
price = token_price.get("price")
symbol = token_price.get("symbol", "")
if price:
lines.append(f"Token: {symbol.upper()} @ ${price:,.4f}")
# Emission categories
if categories:
lines.append("Allocation Categories:")
for cat_name, cat_data in categories.items():
if isinstance(cat_data, dict):
pct = cat_data.get("percentage")
if pct is not None:
lines.append(f" {cat_name}: {pct:.1f}%")
else:
lines.append(f" {cat_name}")
# Upcoming events
if events:
lines.append("Upcoming Unlock Events:")
shown = 0
for event in events[:10]:
desc = event.get("description", "Unlock")
date = event.get("date", "TBD")
amount = event.get("noOfTokens")
if amount:
lines.append(f" {date}: {desc} ({amount:,.0f} tokens)")
else:
lines.append(f" {date}: {desc}")
shown += 1
if len(events) > 10:
lines.append(f" ... and {len(events) - 10} more events")
if len(lines) == 1:
lines.append("No detailed unlock data available.")
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"DeFiLlama API error for '{protocol}' unlocks: {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"DeFiLlama unlock query failed for '{protocol}': {e}"
logger.error(msg)
return msg
def get_protocol_revenue(self, protocol: str) -> str:
"""
Get protocol fees and revenue data.
Args:
protocol: Protocol name or slug.
Returns:
Formatted fees/revenue report string.
"""
slug = self._resolve_slug(protocol)
if slug is None:
return f"Protocol '{protocol}' not found on DeFiLlama."
try:
resp = self._client.get(f"{DEFILLAMA_API}/summary/fees/{slug}")
if resp.status_code == 404:
return f"No fee/revenue data for '{protocol}' on DeFiLlama."
resp.raise_for_status()
data = resp.json()
name = data.get("name", slug)
category = data.get("category", "N/A")
total_24h = data.get("total24h")
total_7d = data.get("total7d")
total_30d = data.get("total30d")
total_all_time = data.get("totalAllTime")
revenue_24h = data.get("revenue24h")
revenue_7d = data.get("revenue7d")
revenue_30d = data.get("revenue30d")
lines = [
f"--- {name} Fees & Revenue ---",
f"Category: {category}",
]
# Fees
lines.append("Fees:")
if total_24h is not None:
lines.append(f" 24h Fees: {_fmt_usd(total_24h)}")
if total_7d is not None:
lines.append(f" 7d Fees: {_fmt_usd(total_7d)}")
if total_30d is not None:
lines.append(f" 30d Fees: {_fmt_usd(total_30d)}")
if total_all_time is not None:
lines.append(f" All-Time Fees: {_fmt_usd(total_all_time)}")
# Revenue (protocol revenue, subset of fees)
has_revenue = any(v is not None for v in [revenue_24h, revenue_7d, revenue_30d])
if has_revenue:
lines.append("Revenue (protocol share):")
if revenue_24h is not None:
lines.append(f" 24h Revenue: {_fmt_usd(revenue_24h)}")
if revenue_7d is not None:
lines.append(f" 7d Revenue: {_fmt_usd(revenue_7d)}")
if revenue_30d is not None:
lines.append(f" 30d Revenue: {_fmt_usd(revenue_30d)}")
# Chain breakdown if available
chain_data = data.get("totalDataChartBreakdown")
if not chain_data and data.get("chains"):
lines.append(f"Available on chains: {', '.join(data['chains'][:15])}")
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"DeFiLlama API error for '{protocol}' revenue: {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"DeFiLlama revenue query failed for '{protocol}': {e}"
logger.error(msg)
return msg
src/services/etherscan.py
+319
View File
@@ -0,0 +1,319 @@
"""Etherscan API service for Ethereum on-chain data (wallet balances, token transfers, contracts)."""
import logging
from datetime import datetime, timezone
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
ETHERSCAN_API_V2 = "https://api.etherscan.io/v2/api"
# Default chain: Polygon (137) where Polymarket operates.
# Ethereum mainnet = 1, can be overridden per-call.
DEFAULT_CHAIN_ID = 137
# Well-known ERC-20 token contracts on Polygon
TOKEN_CONTRACTS = {
"USDC": "0x3c499c542cef5e3811e1192ce70d8cc03d5c3359", # USDC native on Polygon
"USDC.e": "0x2791bca1f2de4661ed88a30c99a7a9449aa84174", # USDC.e (bridged) on Polygon
"USDT": "0xc2132d05d31c914a87c6611c10748aeb04b58e8f",
"WETH": "0x7ceb23fd6bc0add59e62ac25578270cff1b9f619",
"DAI": "0x8f3cf7ad23cd3cadbd9735aff958023239c6a063",
}
# Decimals per token (used for converting raw amounts)
TOKEN_DECIMALS = {
"USDC": 6,
"USDC.e": 6,
"USDT": 6,
"WETH": 18,
"DAI": 18,
}
def _format_amount(raw_value: str, decimals: int) -> float:
"""Convert a raw token amount string to a human-readable float."""
try:
return int(raw_value) / (10 ** decimals)
except (ValueError, TypeError):
return 0.0
def _short_address(address: str) -> str:
"""Shorten an Ethereum address for display."""
if len(address) >= 10:
return f"{address[:6]}...{address[-4:]}"
return address
def _ts_to_str(timestamp: str) -> str:
"""Convert a unix timestamp string to a readable UTC datetime."""
try:
dt = datetime.fromtimestamp(int(timestamp), tz=timezone.utc)
return dt.strftime("%Y-%m-%d %H:%M UTC")
except (ValueError, TypeError):
return timestamp
class EtherscanService:
"""
Etherscan API client for Ethereum on-chain data.
Note: Free tier is limited to 5 calls/sec. Add delays between rapid
successive calls if needed.
"""
def __init__(self, api_key: str = ""):
self.api_key = api_key
self._client = get_client(timeout=20.0)
def is_available(self) -> bool:
return bool(self.api_key and self.api_key.strip())
def _get(self, params: dict, chain_id: int = DEFAULT_CHAIN_ID) -> dict:
"""Make authenticated GET request to Etherscan V2 API."""
params["apikey"] = self.api_key
params["chainid"] = chain_id
resp = self._client.get(ETHERSCAN_API_V2, params=params)
resp.raise_for_status()
return resp.json()
# ------------------------------------------------------------------
# 1. Token transfers
# ------------------------------------------------------------------
def get_wallet_token_transfers(self, address: str, token: str = "USDC") -> str:
"""
Get recent ERC-20 token transfers for a wallet.
Args:
address: Ethereum wallet address.
token: Token symbol to filter on (USDC, USDT, etc.).
Pass "ALL" to show all ERC-20 transfers.
Returns:
Formatted transfer report string for LLM consumption.
"""
addr = address.strip().lower()
token_upper = token.strip().upper()
try:
params = {
"module": "account",
"action": "tokentx",
"address": addr,
"sort": "desc",
"page": "1",
"offset": "20",
}
data = self._get(params)
if data.get("status") != "1" or not data.get("result"):
message = data.get("message", "No transfers found")
return f"No ERC-20 token transfers found for {_short_address(addr)}: {message}"
transfers = data["result"]
# Filter by token if not "ALL"
if token_upper != "ALL":
# For USDC, match both native and bridged (USDC.e) contracts
if token_upper == "USDC":
allowed = {
TOKEN_CONTRACTS.get("USDC", "").lower(),
TOKEN_CONTRACTS.get("USDC.e", "").lower(),
}
allowed.discard("")
transfers = [
tx for tx in transfers
if tx.get("contractAddress", "").lower() in allowed
]
else:
contract = TOKEN_CONTRACTS.get(token_upper, "").lower()
if contract:
transfers = [
tx for tx in transfers
if tx.get("contractAddress", "").lower() == contract
]
else:
# Try matching by symbol in the response
transfers = [
tx for tx in transfers
if tx.get("tokenSymbol", "").upper() == token_upper
]
if not transfers:
return f"No {token_upper} transfers found for {_short_address(addr)} in the last 20 token transactions."
lines = [f"--- Token Transfers for {_short_address(addr)} ({token_upper}) ---"]
for tx in transfers:
tx_from = tx.get("from", "").lower()
tx_to = tx.get("to", "").lower()
symbol = tx.get("tokenSymbol", "???")
decimals = int(tx.get("tokenDecimal", TOKEN_DECIMALS.get(symbol.upper(), 18)))
raw_value = tx.get("value", "0")
amount = _format_amount(raw_value, decimals)
ts = _ts_to_str(tx.get("timeStamp", ""))
tx_hash = tx.get("hash", "")
# Determine direction
if tx_from == addr:
direction = "OUT"
counterparty = _short_address(tx_to)
elif tx_to == addr:
direction = "IN"
counterparty = _short_address(tx_from)
else:
direction = "???"
counterparty = f"{_short_address(tx_from)} -> {_short_address(tx_to)}"
# Flag large transfers
large_flag = ""
if symbol.upper() in ("USDC", "USDT", "DAI") and amount > 10_000:
large_flag = " [LARGE]"
elif symbol.upper() == "WETH" and amount > 5:
large_flag = " [LARGE]"
lines.append(
f" {direction} {amount:,.2f} {symbol}{large_flag} | "
f"{'to' if direction == 'OUT' else 'from'}: {counterparty} | {ts}"
)
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"Etherscan API error fetching token transfers for '{address}': {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"Etherscan token transfer query failed for '{address}': {e}"
logger.error(msg)
return msg
# ------------------------------------------------------------------
# 2. Contract info
# ------------------------------------------------------------------
def get_contract_info(self, address: str) -> str:
"""
Check if an address is a smart contract and retrieve basic contract metadata.
Args:
address: Ethereum address to inspect.
Returns:
Formatted contract info string for LLM consumption.
"""
addr = address.strip()
try:
# First check if ABI is available (verified contract)
abi_data = self._get({
"module": "contract",
"action": "getabi",
"address": addr,
})
is_verified = abi_data.get("status") == "1"
# Get source code info (includes contract name, compiler, etc.)
source_data = self._get({
"module": "contract",
"action": "getsourcecode",
"address": addr,
})
results = source_data.get("result", [])
lines = [f"--- Contract Info for {_short_address(addr)} ---"]
if not results or (isinstance(results, list) and len(results) == 0):
lines.append("No contract data returned. Address may be an EOA (externally owned account).")
lines.append("---")
return "\n".join(lines)
info = results[0] if isinstance(results, list) else results
contract_name = info.get("ContractName", "")
compiler = info.get("CompilerVersion", "")
optimization = info.get("OptimizationUsed", "")
proxy = info.get("Proxy", "0")
implementation = info.get("Implementation", "")
if not contract_name:
lines.append("This address does not appear to be a verified contract.")
lines.append("It may be an EOA (regular wallet) or an unverified contract.")
else:
lines.append(f"Contract Name: {contract_name}")
lines.append(f"Verified: {'Yes' if is_verified else 'No'}")
if compiler:
lines.append(f"Compiler: {compiler}")
if optimization:
lines.append(f"Optimization: {'Yes' if optimization == '1' else 'No'}")
if proxy == "1":
lines.append(f"Proxy Contract: Yes")
if implementation:
lines.append(f"Implementation: {implementation}")
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"Etherscan API error fetching contract info for '{address}': {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"Etherscan contract query failed for '{address}': {e}"
logger.error(msg)
return msg
# ------------------------------------------------------------------
# 3. ETH balance
# ------------------------------------------------------------------
def get_wallet_eth_balance(self, address: str) -> str:
"""
Get ETH balance for a wallet address.
Args:
address: Ethereum wallet address.
Returns:
Formatted ETH balance string for LLM consumption.
"""
addr = address.strip()
try:
data = self._get({
"module": "account",
"action": "balance",
"address": addr,
"tag": "latest",
})
if data.get("status") != "1":
message = data.get("message", "Unknown error")
return f"Could not fetch ETH balance for {_short_address(addr)}: {message}"
raw_balance = data.get("result", "0")
eth_balance = _format_amount(raw_balance, 18)
lines = [
f"--- ETH Balance for {_short_address(addr)} ---",
f"Balance: {eth_balance:,.6f} ETH",
"---",
]
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"Etherscan API error fetching ETH balance for '{address}': {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"Etherscan balance query failed for '{address}': {e}"
logger.error(msg)
return msg
src/services/fred.py
+180
View File
@@ -0,0 +1,180 @@
"""FRED (Federal Reserve Economic Data) API service for macroeconomic indicators."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
FRED_API = "https://api.stlouisfed.org/fred"
# Common series IDs for prediction market analysis
SERIES_MAP = {
# Interest rates
"fed_funds_rate": "FEDFUNDS",
"fed_rate": "FEDFUNDS",
"interest_rate": "FEDFUNDS",
"10y_treasury": "DGS10",
"2y_treasury": "DGS2",
"30y_mortgage": "MORTGAGE30US",
# Inflation
"cpi": "CPIAUCSL",
"core_cpi": "CPILFESL",
"pce": "PCEPI",
"core_pce": "PCEPILFE",
"inflation": "CPIAUCSL",
# Employment
"unemployment": "UNRATE",
"unemployment_rate": "UNRATE",
"nonfarm_payrolls": "PAYEMS",
"jobs": "PAYEMS",
"initial_claims": "ICSA",
"jobless_claims": "ICSA",
# GDP
"gdp": "GDP",
"real_gdp": "GDPC1",
"gdp_growth": "A191RL1Q225SBEA",
# Markets / Financial conditions
"sp500": "SP500",
"vix": "VIXCLS",
"yield_curve": "T10Y2Y",
"financial_stress": "STLFSI2",
# Dollar
"dollar_index": "DTWEXBGS",
"usd": "DTWEXBGS",
# Oil / Commodities
"oil_price": "DCOILWTICO",
"wti": "DCOILWTICO",
"crude_oil": "DCOILWTICO",
"brent": "DCOILBRENTEU",
"gas_price": "GASREGW",
"gold": "GOLDAMGBD228NLBM",
}
def _resolve_series_id(query: str) -> str:
"""Resolve a common name to a FRED series ID."""
q = query.strip().lower().replace(" ", "_")
if q in SERIES_MAP:
return SERIES_MAP[q]
# If already looks like a FRED series ID (uppercase), use as-is
return query.strip().upper()
class FREDService:
"""
FRED API client for macroeconomic data.
Free, unlimited usage with API key.
"""
def __init__(self, api_key: str):
self.api_key = api_key
self._client = get_client(timeout=15.0)
def is_available(self) -> bool:
return bool(self.api_key and self.api_key.strip())
def get_series(self, query: str, num_observations: int = 10) -> str:
"""
Get recent observations for an economic data series.
Args:
query: Common name (e.g. 'fed_rate', 'cpi', 'unemployment', 'oil_price')
or a FRED series ID (e.g. 'FEDFUNDS', 'UNRATE')
num_observations: Number of recent data points to return
Returns:
Formatted report with series info and recent values.
"""
series_id = _resolve_series_id(query)
try:
# Get series metadata
meta_resp = self._client.get(
f"{FRED_API}/series",
params={
"series_id": series_id,
"api_key": self.api_key,
"file_type": "json",
},
)
if meta_resp.status_code == 400:
return (
f"Series '{query}' (id: {series_id}) not found on FRED. "
f"Common names: fed_rate, cpi, unemployment, gdp, oil_price, "
f"vix, yield_curve, gold, sp500, jobless_claims"
)
meta_resp.raise_for_status()
meta = meta_resp.json().get("seriess", [{}])[0]
title = meta.get("title", series_id)
frequency = meta.get("frequency", "")
units = meta.get("units", "")
last_updated = meta.get("last_updated", "")
# Get recent observations
obs_resp = self._client.get(
f"{FRED_API}/series/observations",
params={
"series_id": series_id,
"api_key": self.api_key,
"file_type": "json",
"sort_order": "desc",
"limit": num_observations,
},
)
obs_resp.raise_for_status()
observations = obs_resp.json().get("observations", [])
lines = [
f"--- FRED: {title} ({series_id}) ---",
f"Units: {units}",
f"Frequency: {frequency}",
f"Last Updated: {last_updated}",
"",
"Recent Data:",
]
for obs in reversed(observations):
date = obs.get("date", "")
value = obs.get("value", ".")
if value == ".":
lines.append(f" {date}: N/A")
else:
try:
v = float(value)
lines.append(f" {date}: {v:,.2f}")
except ValueError:
lines.append(f" {date}: {value}")
# Add trend info if enough data
valid_vals = []
for obs in observations:
v = obs.get("value", ".")
if v != ".":
try:
valid_vals.append(float(v))
except ValueError:
pass
if len(valid_vals) >= 2:
latest = valid_vals[0]
prev = valid_vals[1]
change = latest - prev
pct = (change / abs(prev) * 100) if prev != 0 else 0
lines.append(f"\nLatest vs Previous: {change:+.2f} ({pct:+.2f}%)")
lines.append("---")
return "\n".join(lines)
except httpx.HTTPError as e:
msg = f"FRED API error for '{query}' ({series_id}): {e}"
logger.error(msg)
return msg
except Exception as e:
msg = f"FRED query failed for '{query}': {e}"
logger.error(msg)
return msg
src/services/llm_analyzer.py
+466
View File
@@ -0,0 +1,466 @@
"""
LLM analyzer service - analyzes whale trades using AI with tool-use.
Architecture:
1. Build context (trade info + historical signals)
2. Send to LLM with tool schemas (search_twitter, search_web, etc.)
3. LLM decides which tools to call (if any)
4. Execute tool calls, return results to LLM
5. LLM produces final analysis + JSON decision
The LLM controls which information sources to query based on the market type.
"""
import json
import logging
import re
from datetime import datetime
from typing import Optional
from openai import OpenAI
from src.config import get_settings
from src.models.trade import WhaleTrade
from src.models.decision import LLMDecision, TradeRecommendation, TradeAction, TraderCredibility
from src.models.anomaly_signal import AnomalySignal
from src.services.anomaly_detector import AnomalyDetector
from src.services.anomaly_history import AnomalyHistoryService
from src.services.tools import ToolRegistry
from src.prompts.whale_analyzer import WhaleAnalyzerPrompts
logger = logging.getLogger(__name__)
# Maximum tool-use rounds to prevent infinite loops
# 14 tools available; LLM can call multiple per round but may need
# several rounds for chain-of-investigation (search → discover → verify)
MAX_TOOL_ROUNDS = 3
class LLMAnalyzer:
"""Analyzes whale trades using LLM with function-calling tools."""
def __init__(self):
self.settings = get_settings()
self.client = OpenAI(
base_url=self.settings.llm_base_url,
api_key=self.settings.llm_api_key,
)
self.anomaly_detector = AnomalyDetector()
self.prompts = WhaleAnalyzerPrompts()
self.anomaly_history = AnomalyHistoryService(self.settings.db_path)
# Tool registry — LLM decides which tools to call
self.tool_registry = ToolRegistry(
twitter_api_key=self.settings.twitter_api_key,
tavily_api_key=self.settings.tavily_api_key,
fred_api_key=self.settings.fred_api_key,
polygon_api_key=self.settings.polygon_api_key,
congress_api_key=self.settings.congress_api_key,
etherscan_api_key=self.settings.etherscan_api_key,
serper_api_key=self.settings.serper_api_key,
telegram_api_id=self.settings.telegram_api_id,
telegram_api_hash=self.settings.telegram_api_hash,
telegram_session_string=self.settings.telegram_session_string,
telegram_channels=self.settings.telegram_channels,
)
self._last_historical_signal_count = 0
@property
def last_historical_signal_count(self) -> int:
return self._last_historical_signal_count
# ================================================================
# Response parsing
# ================================================================
# Fields that identify the final recommendation JSON (vs intermediate tool-call JSONs)
_RECOMMENDATION_FIELDS = {"information_asymmetry_score", "confidence", "trader_credibility"}
def _extract_json_from_response(self, response: str) -> Optional[dict]:
"""Extract the final recommendation JSON from LLM response text.
When the response contains multiple JSON code blocks (e.g. an
intermediate ANALYZE decision followed by the real assessment),
prefer the block that contains recommendation-specific fields.
Falls back to the last parseable block.
"""
candidates: list[dict] = []
for match in re.findall(r"```(?:json)?\s*([\s\S]*?)```", response):
try:
candidates.append(json.loads(match.strip()))
except json.JSONDecodeError:
continue
if candidates:
# Prefer the block that looks like a final recommendation
for c in reversed(candidates):
if c.keys() & self._RECOMMENDATION_FIELDS:
return c
# No block has recommendation fields — return the last one
return candidates[-1]
# Try raw JSON
try:
start = response.find("{")
end = response.rfind("}") + 1
if start >= 0 and end > start:
return json.loads(response[start:end])
except json.JSONDecodeError:
pass
return None
def _parse_recommendation(self, json_data: dict) -> TradeRecommendation:
"""Parse JSON into TradeRecommendation."""
action_str = json_data.get("action", "HOLD").upper()
try:
action = TradeAction(action_str)
except ValueError:
action = TradeAction.HOLD
confidence = max(0.0, min(1.0, float(json_data.get("confidence", 0.0))))
suggested_price = json_data.get("suggested_price")
if suggested_price is not None:
suggested_price = float(suggested_price)
suggested_size = max(0.0, min(1.0, float(json_data.get("suggested_size_percent", 0.1))))
insider_likelihood = max(0.0, min(1.0, float(json_data.get("information_asymmetry_score", 0.0))))
credibility_str = json_data.get("trader_credibility", "UNKNOWN").upper()
try:
trader_credibility = TraderCredibility(credibility_str)
except ValueError:
trader_credibility = TraderCredibility.UNKNOWN
return TradeRecommendation(
action=action,
outcome=str(json_data.get("outcome", "")),
confidence=confidence,
suggested_price=suggested_price,
suggested_size_percent=suggested_size,
reasoning=str(json_data.get("reasoning", "")),
information_asymmetry_score=insider_likelihood,
trader_credibility=trader_credibility,
insider_evidence=str(json_data.get("insider_evidence", "")),
)
# ================================================================
# Anomaly signal storage
# ================================================================
def _store_anomaly_signal_if_qualified(
self,
whale_trade: WhaleTrade,
decision: LLMDecision,
) -> None:
"""Store anomaly signal if information asymmetry score qualifies."""
rec = decision.recommendation
if not self.anomaly_history.should_store_signal(rec.information_asymmetry_score):
logger.debug(
f"Signal not stored: IAS {rec.information_asymmetry_score:.2f} "
f"below threshold"
)
return
signal = AnomalySignal(
id=whale_trade.id,
market_id=whale_trade.market_id,
market_question=whale_trade.market_question,
market_slug=whale_trade.trade.slug,
condition_id=whale_trade.trade.condition_id,
transaction_hash=whale_trade.trade.transaction_hash,
trade_timestamp=whale_trade.trade.timestamp,
trade_side=whale_trade.trade.side,
trade_price=whale_trade.trade.price,
trade_size_usd=whale_trade.trade.usdc_size,
trade_outcome=whale_trade.trade.outcome,
trader_wallet=whale_trade.trade.proxy_wallet,
trader_ranking=whale_trade.trader_ranking,
trader_history=whale_trade.trader_history,
information_asymmetry_score=rec.information_asymmetry_score,
reasoning=rec.reasoning,
insider_evidence=rec.insider_evidence,
detected_at=whale_trade.detected_at,
)
stored = self.anomaly_history.store_signal(signal)
if stored:
logger.info(
f"Stored anomaly signal: {whale_trade.market_question[:50]}... "
f"IAS={rec.information_asymmetry_score:.0%}"
)
# ================================================================
# Tool-use loop
# ================================================================
def _execute_tool_calls(self, tool_calls) -> list[dict]:
"""Execute tool calls from the LLM and return message dicts."""
results = []
for tc in tool_calls:
fn_name = tc.function.name
try:
fn_args = json.loads(tc.function.arguments)
except json.JSONDecodeError:
fn_args = {}
logger.info(f"LLM requested tool: {fn_name}({fn_args})")
output = self.tool_registry.call(fn_name, **fn_args)
results.append({
"role": "tool",
"tool_call_id": tc.id,
"content": output,
})
return results
async def analyze_whale_trade(self, whale_trade: WhaleTrade) -> LLMDecision:
"""
Analyze a whale trade using LLM with tool-use.
Flow:
0. Pre-screening: lightweight check if signal is worth full analysis
1. Build initial context (trade + historical signals)
2. Send to LLM with available tool schemas
3. If LLM requests tools → execute → return results → repeat (up to MAX_TOOL_ROUNDS)
4. Parse final text response for JSON decision
"""
# Build context
trade_context = self.anomaly_detector.format_for_llm(whale_trade)
historical_context = ""
historical_signals = self.anomaly_history.get_signals_for_market(
whale_trade.market_id, top_recent=5, top_likelihood=5,
)
self._last_historical_signal_count = len(historical_signals)
if historical_signals:
historical_context = self.anomaly_history.format_historical_signals_context(historical_signals)
logger.info(f"Found {len(historical_signals)} historical anomaly signals for market")
# Build initial messages
system_prompt = self.prompts.system_prompt()
user_prompt = self.prompts.analyze_whale_trade(trade_context, historical_context)
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt},
]
# Tool schemas (empty list if no tools available)
tool_schemas = self.tool_registry.openai_tool_schemas()
try:
analysis_text = ""
# Tool-use loop
for round_idx in range(MAX_TOOL_ROUNDS + 1):
# Last round: no tools, force final answer
is_last_round = (round_idx == MAX_TOOL_ROUNDS)
if is_last_round:
messages.append({
"role": "user",
"content": (
"You have used all available tool rounds. "
"Based on all information gathered, provide your final "
"analysis and output the JSON assessment now."
),
})
# Call LLM
call_kwargs = {
"model": self.settings.llm_model,
"messages": messages,
"max_tokens": 8192,
}
if tool_schemas and not is_last_round:
call_kwargs["tools"] = tool_schemas
call_kwargs["tool_choice"] = "auto"
response = self.client.chat.completions.create(**call_kwargs)
msg = response.choices[0].message
# If LLM wants to call tools
if msg.tool_calls:
logger.info(
f"Round {round_idx + 1}: LLM requested "
f"{len(msg.tool_calls)} tool call(s)"
)
# Append assistant message with tool calls
messages.append(msg.model_dump())
# Execute tools and append results
tool_results = self._execute_tool_calls(msg.tool_calls)
messages.extend(tool_results)
continue # Next round — LLM processes tool results
# No tool calls — final response
analysis_text = msg.content or ""
finish_reason = response.choices[0].finish_reason
logger.info(
f"Analysis complete after {round_idx + 1} round(s) "
f"({len(analysis_text)} chars, finish={finish_reason})"
)
if len(analysis_text) < 200:
logger.warning(f"Suspiciously short response: {analysis_text[:200]}")
break
# Parse JSON decision from final response
json_data = self._extract_json_from_response(analysis_text)
if json_data:
# Check if LLM decided to skip (pre-screening in prompt)
if json_data.get("action") == "SKIP":
reason = json_data.get("reason", "not in scope")
logger.info(
f"⏭️ Pre-screening SKIP: {reason} "
f"(market: {whale_trade.market_question[:40]}...)"
)
return LLMDecision(
whale_trade_id=whale_trade.id,
market_id=whale_trade.market_id,
analysis=f"Pre-screening: {reason}",
recommendation=TradeRecommendation(
action=TradeAction.HOLD,
outcome="",
confidence=0.0,
reasoning=f"Signal filtered: {reason}",
),
)
recommendation = self._parse_recommendation(json_data)
else:
logger.warning("Could not parse LLM response as JSON, defaulting to HOLD")
recommendation = TradeRecommendation(
action=TradeAction.HOLD,
outcome="",
confidence=0.0,
reasoning="Failed to parse LLM response",
)
decision = LLMDecision(
whale_trade_id=whale_trade.id,
market_id=whale_trade.market_id,
analysis=analysis_text,
recommendation=recommendation,
)
self._store_anomaly_signal_if_qualified(whale_trade, decision)
return decision
except Exception as e:
logger.error(f"Error in LLM analysis: {e}")
return LLMDecision(
whale_trade_id=whale_trade.id,
market_id=whale_trade.market_id,
analysis=f"Error during analysis: {str(e)}",
recommendation=TradeRecommendation(
action=TradeAction.HOLD,
outcome="",
confidence=0.0,
reasoning=f"Analysis failed: {str(e)}",
),
)
# ================================================================
# Report formatting (unchanged)
# ================================================================
def format_full_report(
self,
whale_trade: WhaleTrade,
decision: LLMDecision,
historical_signal_count: int = 0,
) -> str:
"""Format a complete analysis report."""
trade = whale_trade.trade
rec = decision.recommendation
prices_str = ""
if whale_trade.market_outcomes and whale_trade.market_outcome_prices:
prices_str = " | ".join([
f"{o}: {p:.1%}"
for o, p in zip(whale_trade.market_outcomes, whale_trade.market_outcome_prices)
])
action_indicator = {
TradeAction.BUY: "🟢 BUY",
TradeAction.SELL: "🔴 SELL",
TradeAction.HOLD: "⚪ HOLD",
}
ias = rec.information_asymmetry_score
if ias >= 0.7:
insider_indicator = f"High Information Asymmetry ({ias:.0%})"
elif ias >= 0.4:
insider_indicator = f"Medium Information Asymmetry ({ias:.0%})"
else:
insider_indicator = f"Low Information Asymmetry ({ias:.0%})"
rank_num = whale_trade.trader_ranking.rank if whale_trade.trader_ranking and whale_trade.trader_ranking.rank else None
credibility_indicators = {
TraderCredibility.HIGH: f"High Credibility (#{rank_num})" if rank_num else "High Credibility",
TraderCredibility.MEDIUM: f"Medium Credibility (#{rank_num})" if rank_num else "Medium Credibility",
TraderCredibility.LOW: f"Low Credibility (#{rank_num})" if rank_num else "Low Credibility",
TraderCredibility.UNKNOWN: "Unknown (Unranked)",
}
credibility_str = credibility_indicators.get(rec.trader_credibility, "Unknown")
trader_ranking_str = ""
if whale_trade.trader_ranking:
tr = whale_trade.trader_ranking
rank_str = f"#{tr.rank}" if tr.rank else "Unranked"
pnl_str = f"${tr.pnl:,.2f}" if tr.pnl else "N/A"
trader_ranking_str = f"| **Trader Rank** | {rank_str} (PnL: {pnl_str}) |"
historical_info = ""
if historical_signal_count > 0:
historical_info = f"\n**Historical Anomaly Signals Referenced**: {historical_signal_count} (analyzed together)"
report = f"""
{'='*70}
# Whale Trade Analysis Report
{'='*70}
**Generated at**: {datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')} UTC{historical_info}
## Trade Summary
| Field | Details |
|-------|---------|
| **Market** | {whale_trade.market_question} |
| **Trade Size** | ${trade.usdc_size:,.2f} USDC |
| **Direction** | BUY {trade.outcome} Token ({'Bullish' if trade.outcome == 'Yes' else 'Bearish'}) |
| **Trade Price** | {trade.price:.4f} ({trade.price:.1%}) |
| **Current Odds** | {prices_str} |
| **Trade Time** | {datetime.fromtimestamp(trade.timestamp).strftime('%Y-%m-%d %H:%M:%S') if trade.timestamp else 'N/A'} |
{trader_ranking_str}
{'='*70}
{decision.analysis}
{'='*70}
## Information Asymmetry Assessment
{'='*70}
| Field | Assessment |
|-------|------------|
| **Information Asymmetry** | {insider_indicator} |
| **Trader Credibility** | {credibility_str} |
**Key Evidence**: {rec.insider_evidence or 'No clear evidence'}
**Reasoning**: {rec.reasoning}
{'='*70}
Disclaimer: This report is AI-generated for informational purposes only and does not constitute investment advice.
{'='*70}
"""
return report
src/services/market_fetcher.py
+531
View File
@@ -0,0 +1,531 @@
"""Market fetching service - fetches trending markets from Polymarket."""
import json
import logging
from typing import List, Optional
import httpx
from src.config import get_settings
from src.models.market import Market, TrendingMarket
from src.utils.http import get_client
logger = logging.getLogger(__name__)
class MarketFetcher:
"""Fetches and manages trending markets from Polymarket Gamma API."""
# Short-term price prediction markets to filter out (no signal value)
# Matches patterns like "Bitcoin Up or Down - March 27, 2:00AM-2:15AM ET"
SHORT_TERM_PRICE_KEYWORDS = [
"up or down", # "Bitcoin Up or Down - March 27, 2:00AM"
"higher or lower", # price higher or lower
"above or below", # close above or below
"opens up or down", # "S&P 500 Opens Up or Down"
"green or red", # daily candle color
]
# Temperature/weather markets (no signal value)
WEATHER_KEYWORDS = [
"highest temperature",
"lowest temperature",
"temperature in",
"weather",
"rainfall",
"°f on",
"°c on",
]
# Sports-related keywords to filter out
SPORTS_KEYWORDS = [
"nba", "nfl", "mlb", "nhl", "mls", "ufc", "wwe", "pga", "atp", "wta",
"fifa", "uefa", "epl", "premier league", "la liga", "serie a", "bundesliga",
"champions league", "world cup", "olympics", "olympic",
"basketball", "football", "soccer", "baseball", "hockey", "tennis",
"golf", "boxing", "mma", "wrestling", "cricket", "rugby", "f1", "formula 1",
"nascar", "racing", "motorsport",
"game", "match", "vs", "versus", "playoff", "playoffs", "finals",
"championship", "tournament", "season", "super bowl", "world series",
"score", "points", "goals", "touchdowns", "wins", "win against",
"beat", "defeat",
"mvp", "rookie", "all-star", "draft", "trade",
"lakers", "celtics", "warriors", "bulls", "heat", "knicks",
"yankees", "dodgers", "red sox", "cubs", "mets",
"cowboys", "patriots", "chiefs", "eagles", "49ers",
"manchester", "barcelona", "real madrid", "liverpool", "chelsea",
]
def __init__(self):
self.settings = get_settings()
self.gamma_url = "https://gamma-api.polymarket.com"
self.markets_endpoint = f"{self.gamma_url}/markets"
self.events_endpoint = f"{self.gamma_url}/events"
self._client = get_client(timeout=30.0)
def __del__(self):
"""Cleanup HTTP client."""
if hasattr(self, "_client"):
self._client.close()
def _should_filter_market(self, market_data: dict) -> str:
"""
Check if a market should be filtered out.
Returns:
Filter reason string if should be filtered, empty string if OK.
"""
question = (market_data.get("question") or "").lower()
description = (market_data.get("description") or "").lower()
slug = (market_data.get("slug") or "").lower()
text = f"{question} {description} {slug}"
for keyword in self.SPORTS_KEYWORDS:
if keyword in text:
return "sports"
for keyword in self.SHORT_TERM_PRICE_KEYWORDS:
if keyword in text:
return "short_term_price"
for keyword in self.WEATHER_KEYWORDS:
if keyword in text:
return "weather"
return ""
def _is_sports_market(self, market_data: dict) -> bool:
"""Legacy compatibility."""
return bool(self._should_filter_market(market_data))
def _parse_market(self, data: dict) -> Optional[Market]:
"""Parse raw market data into Market model."""
try:
# Parse outcome prices (comes as stringified list)
outcome_prices = data.get("outcomePrices", [])
if isinstance(outcome_prices, str):
outcome_prices = json.loads(outcome_prices)
outcome_prices = [float(p) for p in outcome_prices]
# Parse clob token IDs
clob_token_ids = data.get("clobTokenIds", [])
if isinstance(clob_token_ids, str):
clob_token_ids = json.loads(clob_token_ids)
# Parse outcomes
outcomes = data.get("outcomes", [])
if isinstance(outcomes, str):
outcomes = json.loads(outcomes)
return Market(
id=str(data.get("id", "")),
question=data.get("question", ""),
condition_id=data.get("conditionId"),
slug=data.get("slug"),
description=data.get("description"),
end_date=data.get("endDate"),
outcomes=outcomes,
outcome_prices=outcome_prices,
clob_token_ids=clob_token_ids,
volume=float(data.get("volume", 0) or 0),
volume_24hr=float(data.get("volume24hr", 0) or 0),
liquidity=float(data.get("liquidity", 0) or 0),
active=data.get("active", False),
closed=data.get("closed", False),
neg_risk=data.get("negRisk", False),
)
except Exception as e:
logger.warning(f"Failed to parse market {data.get('id')}: {e}")
return None
def get_trending_markets(self, limit: Optional[int] = None) -> List[TrendingMarket]:
"""
Fetch trending markets sorted by 24-hour volume.
Filters out sports-related markets since they lack fundamental analysis value.
Args:
limit: Maximum number of non-sports markets to return (defaults to settings)
Returns:
List of TrendingMarket objects (excluding sports markets)
"""
limit = limit or self.settings.trending_markets_limit
trending_markets = []
offset = 0
batch_size = 100 # Fetch more to account for sports filtering
max_iterations = 10 # Safety limit to prevent infinite loops
try:
iteration = 0
while len(trending_markets) < limit and iteration < max_iterations:
iteration += 1
# Fetch active markets sorted by volume
params = {
"active": True,
"closed": False,
"archived": False,
"limit": batch_size,
"offset": offset,
"order": "volume24hr",
"ascending": False,
"enableOrderBook": True, # Only markets with CLOB enabled
}
response = self._client.get(self.markets_endpoint, params=params)
response.raise_for_status()
data = response.json()
if not data:
break # No more markets
filtered_counts = {"sports": 0, "short_term_price": 0, "weather": 0}
for market_data in data:
reason = self._should_filter_market(market_data)
if reason:
filtered_counts[reason] = filtered_counts.get(reason, 0) + 1
continue
market = self._parse_market(market_data)
if market:
trending_market = TrendingMarket(
market=market,
volume_24hr=market.volume_24hr,
liquidity=market.liquidity,
rank=len(trending_markets) + 1,
)
if trending_market.is_valid_for_monitoring:
trending_markets.append(trending_market)
if len(trending_markets) >= limit:
break
total_filtered = sum(filtered_counts.values())
if total_filtered:
parts = [f"{k}={v}" for k, v in filtered_counts.items() if v > 0]
logger.debug(
f"Batch {iteration}: fetched {len(data)}, "
f"filtered {total_filtered} ({', '.join(parts)}), "
f"kept: {len(trending_markets)}"
)
else:
logger.debug(
f"Batch {iteration}: fetched {len(data)}, "
f"kept: {len(trending_markets)}"
)
if len(data) < batch_size:
break # No more markets available
offset += batch_size
logger.info(
f"Fetched {len(trending_markets)} trending markets "
f"(filtered: sports, short-term price, weather)"
)
return trending_markets
except httpx.HTTPError as e:
logger.error(f"HTTP error fetching trending markets: {e}")
return trending_markets # Return what we have so far
except Exception as e:
logger.error(f"Error fetching trending markets: {e}")
return trending_markets
# Keywords that identify token launch / crypto project markets
TOKEN_LAUNCH_KEYWORDS = [
"fdv", "market cap (fdv)", "launch a token", "tge",
"listing", "airdrop", "public sale",
]
def _is_token_launch_market(self, market_data: dict) -> bool:
"""Check if a market is related to token launches / crypto projects."""
question = (market_data.get("question") or "").lower()
return any(kw in question for kw in self.TOKEN_LAUNCH_KEYWORDS)
def get_token_launch_markets(self, max_scan: int = 2000) -> List[TrendingMarket]:
"""
Scan active markets for token launch / crypto project markets
that may not be in the top trending list.
Returns:
List of TrendingMarket objects for token launch markets.
"""
token_markets = []
offset = 0
batch_size = 100
seen_ids = set()
try:
while offset < max_scan:
params = {
"active": True, "closed": False, "archived": False,
"limit": batch_size, "offset": offset,
"order": "volume24hr", "ascending": False,
"enableOrderBook": True,
}
response = self._client.get(self.markets_endpoint, params=params)
response.raise_for_status()
data = response.json()
if not data:
break
for market_data in data:
if not self._is_token_launch_market(market_data):
continue
market = self._parse_market(market_data)
if market and market.id not in seen_ids:
seen_ids.add(market.id)
tm = TrendingMarket(
market=market,
volume_24hr=market.volume_24hr,
liquidity=market.liquidity,
)
if tm.is_valid_for_monitoring:
token_markets.append(tm)
if len(data) < batch_size:
break
offset += batch_size
logger.info(f"Found {len(token_markets)} token launch markets")
return token_markets
except Exception as e:
logger.error(f"Error fetching token launch markets: {e}")
return token_markets
def get_niche_markets(
self,
limit: int = 50,
min_volume_24hr: float = 5_000,
max_volume_24hr: float = 500_000,
offset_start: int = 200,
max_scan: int = 1500,
) -> List[TrendingMarket]:
"""
Fetch niche markets (lower volume) that may have higher information
asymmetry value. Scans markets ranked beyond the top trending list.
Args:
limit: Max number of niche markets to return
min_volume_24hr: Minimum 24h volume (filter out dead markets)
max_volume_24hr: Maximum 24h volume (filter out large/macro markets)
offset_start: Start scanning from this rank
max_scan: Stop scanning after this offset
Returns:
List of TrendingMarket objects for niche markets.
"""
niche_markets = []
offset = offset_start
batch_size = 100
try:
while offset < max_scan and len(niche_markets) < limit:
params = {
"active": True, "closed": False, "archived": False,
"limit": batch_size, "offset": offset,
"order": "volume24hr", "ascending": False,
"enableOrderBook": True,
}
response = self._client.get(self.markets_endpoint, params=params)
response.raise_for_status()
data = response.json()
if not data:
break
for market_data in data:
vol = float(market_data.get("volume24hr", 0) or 0)
# Volume filter: not too small (dead), not too large (macro)
if vol < min_volume_24hr or vol > max_volume_24hr:
continue
# Apply standard filters (sports, weather, short-term price)
if self._should_filter_market(market_data):
continue
market = self._parse_market(market_data)
if market:
tm = TrendingMarket(
market=market,
volume_24hr=market.volume_24hr,
liquidity=market.liquidity,
)
if tm.is_valid_for_monitoring:
niche_markets.append(tm)
if len(niche_markets) >= limit:
break
if len(data) < batch_size:
break
offset += batch_size
logger.info(
f"Found {len(niche_markets)} niche markets "
f"(volume ${min_volume_24hr:,.0f}-${max_volume_24hr:,.0f})"
)
return niche_markets
except Exception as e:
logger.error(f"Error fetching niche markets: {e}")
return niche_markets
def get_tiered_markets(self) -> dict[str, list[TrendingMarket]]:
"""
Fetch ALL active markets and classify into 3 tiers by 24h volume.
Returns dict with keys "tier1", "tier2", "tier3", each a list of TrendingMarket.
Mirrors options flow's "passive receive all signals" approach:
scan everything, filter later.
"""
settings = self.settings
all_markets = self.get_all_current_markets()
tiers: dict[str, list[TrendingMarket]] = {"tier1": [], "tier2": [], "tier3": []}
for market in all_markets:
# Apply standard filters
raw = {
"question": market.question,
"description": market.description,
"slug": market.slug,
}
if self._should_filter_market(raw):
continue
if not market.clob_token_ids or not market.active or market.closed:
continue
vol = market.volume_24hr
tm = TrendingMarket(
market=market,
volume_24hr=vol,
liquidity=market.liquidity,
)
if vol >= settings.tier1_volume_min:
tiers["tier1"].append(tm)
elif vol >= settings.tier2_volume_min:
tiers["tier2"].append(tm)
elif vol >= settings.tier3_volume_min:
tiers["tier3"].append(tm)
# vol < tier3_volume_min → dead market, skip
# Sort each tier by volume descending
for tier in tiers.values():
tier.sort(key=lambda t: t.volume_24hr, reverse=True)
logger.info(
f"Tiered markets: Tier1={len(tiers['tier1'])} (>{settings.tier1_volume_min/1000:.0f}K), "
f"Tier2={len(tiers['tier2'])} (>{settings.tier2_volume_min/1000:.0f}K), "
f"Tier3={len(tiers['tier3'])} (>{settings.tier3_volume_min/1000:.0f}K)"
)
return tiers
def get_market_by_id(self, market_id: str) -> Optional[Market]:
"""
Fetch a single market by ID.
Args:
market_id: The market ID
Returns:
Market object or None
"""
try:
url = f"{self.markets_endpoint}/{market_id}"
response = self._client.get(url)
response.raise_for_status()
data = response.json()
return self._parse_market(data)
except httpx.HTTPError as e:
logger.error(f"HTTP error fetching market {market_id}: {e}")
return None
except Exception as e:
logger.error(f"Error fetching market {market_id}: {e}")
return None
def get_market_by_condition_id(self, condition_id: str) -> Optional[Market]:
"""
Fetch a market by condition ID.
Args:
condition_id: The condition ID
Returns:
Market object or None
"""
try:
params = {"conditionId": condition_id}
response = self._client.get(self.markets_endpoint, params=params)
response.raise_for_status()
data = response.json()
if data and len(data) > 0:
return self._parse_market(data[0])
return None
except httpx.HTTPError as e:
logger.error(f"HTTP error fetching market by condition {condition_id}: {e}")
return None
except Exception as e:
logger.error(f"Error fetching market by condition {condition_id}: {e}")
return None
def get_all_current_markets(self, batch_size: int = 100) -> List[Market]:
"""
Fetch all current active markets (paginated).
Args:
batch_size: Number of markets per request
Returns:
List of all active Market objects
"""
all_markets = []
offset = 0
while True:
try:
params = {
"active": True,
"closed": False,
"archived": False,
"limit": batch_size,
"offset": offset,
"enableOrderBook": True,
}
response = self._client.get(self.markets_endpoint, params=params)
response.raise_for_status()
data = response.json()
if not data:
break
for market_data in data:
market = self._parse_market(market_data)
if market:
all_markets.append(market)
if len(data) < batch_size:
break
offset += batch_size
except Exception as e:
error_str = str(e)
if "422" in error_str:
logger.debug(f"Reached end of pagination at offset {offset}")
else:
logger.error(f"Error fetching markets at offset {offset}: {e}")
break
logger.info(f"Fetched {len(all_markets)} total active markets")
return all_markets
src/services/polygon.py
+157
View File
@@ -0,0 +1,157 @@
"""Polygon.io API service for stocks, forex, and commodities market data."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
POLYGON_API = "https://api.polygon.io"
class PolygonService:
"""
Polygon.io API client for financial market data.
Covers: stocks, options, forex, crypto, indices, commodities futures.
"""
def __init__(self, api_key: str):
self.api_key = api_key
self._client = get_client(timeout=15.0)
def is_available(self) -> bool:
return bool(self.api_key and self.api_key.strip())
def _get(self, path: str, params: Optional[dict] = None) -> dict:
"""Make authenticated GET request."""
params = params or {}
params["apiKey"] = self.api_key
resp = self._client.get(f"{POLYGON_API}{path}", params=params)
resp.raise_for_status()
return resp.json()
def get_ticker_snapshot(self, ticker: str) -> str:
"""
Get previous day close + recent daily bars for a ticker.
Args:
ticker: Ticker symbol (AAPL, TSLA, GS, SPY, QQQ, GLD, USO)
Returns:
Formatted price and market data report.
"""
t = ticker.strip().upper()
try:
# Previous close (free tier)
prev_data = self._get(f"/v2/aggs/ticker/{t}/prev")
results = prev_data.get("results", [])
if not results:
return f"No data found for '{ticker}' on Polygon.io."
bar = results[0]
close = bar.get("c", 0)
open_p = bar.get("o", 0)
high = bar.get("h", 0)
low = bar.get("l", 0)
volume = bar.get("v", 0)
vwap = bar.get("vw", 0)
change = close - open_p if open_p else 0
change_pct = (change / open_p * 100) if open_p else 0
lines = [
f"--- {t} Last Trading Day (Polygon.io) ---",
f"Close: ${close:,.2f}",
f"Open: ${open_p:,.2f}",
f"High: ${high:,.2f}",
f"Low: ${low:,.2f}",
f"Change: {change:+.2f} ({change_pct:+.2f}%)",
]
if vwap:
lines.append(f"VWAP: ${vwap:,.2f}")
if volume:
lines.append(f"Volume: {volume:,.0f}")
# Also try to get 5-day bars for trend
try:
from datetime import date, timedelta
end = date.today()
start = end - timedelta(days=10)
range_data = self._get(
f"/v2/aggs/ticker/{t}/range/1/day/{start.isoformat()}/{end.isoformat()}",
params={"adjusted": "true", "sort": "asc", "limit": 10},
)
bars = range_data.get("results", [])
if len(bars) >= 2:
first_close = bars[0].get("c", 0)
last_close = bars[-1].get("c", 0)
if first_close:
week_change = ((last_close - first_close) / first_close) * 100
lines.append(f"~{len(bars)}-day Change: {week_change:+.2f}%")
except Exception:
pass # trend data is optional
lines.append("---")
return "\n".join(lines)
except httpx.HTTPStatusError as e:
if e.response.status_code == 404:
return f"Ticker '{ticker}' not found on Polygon.io."
return f"Polygon API error for '{ticker}': HTTP {e.response.status_code}"
except Exception as e:
msg = f"Polygon query failed for '{ticker}': {e}"
logger.error(msg)
return msg
def get_market_news(self, ticker: str, limit: int = 5) -> str:
"""
Get recent news articles for a ticker.
Args:
ticker: Stock/crypto ticker (e.g. AAPL, TSLA, GS)
limit: Number of articles (1-10)
Returns:
Formatted news report.
"""
t = ticker.strip().upper()
limit = max(1, min(limit, 10))
try:
data = self._get("/v2/reference/news", params={
"ticker": t,
"limit": limit,
"order": "desc",
"sort": "published_utc",
})
results = data.get("results", [])
if not results:
return f"No recent news found for '{ticker}'."
lines = [f"--- {t} Recent News (Polygon.io) ---"]
for i, article in enumerate(results, 1):
title = article.get("title", "No title")
published = article.get("published_utc", "")[:19]
source = article.get("publisher", {}).get("name", "Unknown")
desc = article.get("description", "")[:200]
if len(article.get("description", "")) > 200:
desc += "..."
lines.append(f"{i}. **{title}**")
lines.append(f" Source: {source} | {published}")
if desc:
lines.append(f" {desc}")
lines.append("")
lines.append("---")
return "\n".join(lines)
except Exception as e:
msg = f"Polygon news query failed for '{ticker}': {e}"
logger.error(msg)
return msg
src/services/price_monitor.py
+428
View File
@@ -0,0 +1,428 @@
"""Price monitoring service - monitors ALL active market prices for volatility."""
import asyncio
import json
import logging
from collections import deque
from dataclasses import dataclass, field
from datetime import datetime
from pathlib import Path
from typing import Awaitable, Callable, Dict, List, Optional
import httpx
from src.utils.http import get_async_client
logger = logging.getLogger(__name__)
# Gamma API for fetching market prices
GAMMA_API_URL = "https://gamma-api.polymarket.com/markets"
# Storage directory for price volatility alerts
VOLATILITY_DIR = Path(__file__).parent.parent.parent / "price_volatility"
@dataclass
class PricePoint:
"""A single price observation."""
timestamp: int
yes_price: float
@dataclass
class VolatilityAlert:
"""A price volatility alert."""
market_id: str
market_question: str
start_timestamp: int
end_timestamp: int
start_price: float
end_price: float
price_change: float
price_change_percent: float
direction: str # "UP" or "DOWN"
window_seconds: int
detected_at: str = field(default_factory=lambda: datetime.utcnow().isoformat())
def to_dict(self) -> dict:
return {
"market_id": self.market_id,
"market_question": self.market_question,
"start_timestamp": self.start_timestamp,
"end_timestamp": self.end_timestamp,
"start_price": self.start_price,
"end_price": self.end_price,
"price_change": self.price_change,
"price_change_percent": self.price_change_percent,
"direction": self.direction,
"window_seconds": self.window_seconds,
"detected_at": self.detected_at,
}
class PriceMonitor:
"""
Monitors ALL active market prices for short-term volatility.
Tracks Yes prices for all active markets and alerts when
price changes exceed threshold within the time window.
"""
# Default configuration
DEFAULT_WINDOW_SECONDS = 300 # 5 minutes
DEFAULT_THRESHOLD = 0.10 # 10%
DEFAULT_MAX_HISTORY_SECONDS = 3600 # Keep 1 hour of history
DEFAULT_POLL_INTERVAL = 30 # Poll all markets every 30 seconds
def __init__(
self,
window_seconds: int = DEFAULT_WINDOW_SECONDS,
threshold: float = DEFAULT_THRESHOLD,
max_history_seconds: int = DEFAULT_MAX_HISTORY_SECONDS,
poll_interval: int = DEFAULT_POLL_INTERVAL,
on_volatility_detected: Optional[Callable[["VolatilityAlert"], Awaitable[None]]] = None,
):
"""
Initialize the price monitor.
Args:
window_seconds: Time window for volatility detection (default 5 minutes)
threshold: Price change threshold to trigger alert (default 10%)
max_history_seconds: How long to keep price history (default 1 hour)
poll_interval: Interval for polling all markets (default 30 seconds)
on_volatility_detected: Async callback when volatility is detected
"""
self.window_seconds = window_seconds
self.threshold = threshold
self.max_history_seconds = max_history_seconds
self.poll_interval = poll_interval
# Callback for volatility detection
self._on_volatility_detected = on_volatility_detected
# Price history per market: market_id -> deque of PricePoints
self._price_history: Dict[str, deque] = {}
# Market info cache: market_id -> question
self._market_info: Dict[str, str] = {}
# Track recent alerts to avoid duplicates (market_id -> last_alert_timestamp)
self._recent_alerts: Dict[str, int] = {}
# Minimum interval between alerts for same market (seconds)
self._alert_cooldown = 3600 # 1 hour (match window_seconds)
# HTTP client for API calls
self._client: Optional[httpx.AsyncClient] = None
# Control flag
self._running = False
# Ensure storage directory exists
VOLATILITY_DIR.mkdir(parents=True, exist_ok=True)
def record_price(self, market_id: str, market_question: str, yes_price: float) -> Optional[VolatilityAlert]:
"""
Record a price observation and check for volatility.
Args:
market_id: The market ID
market_question: The market question text
yes_price: Current Yes price (0-1)
Returns:
VolatilityAlert if threshold exceeded, None otherwise
"""
now = int(datetime.utcnow().timestamp())
# Initialize history for new markets
if market_id not in self._price_history:
self._price_history[market_id] = deque()
history = self._price_history[market_id]
# Add new price point
history.append(PricePoint(timestamp=now, yes_price=yes_price))
# Clean up old entries
cutoff = now - self.max_history_seconds
while history and history[0].timestamp < cutoff:
history.popleft()
# Check for volatility
alert = self._check_volatility(market_id, market_question, now)
if alert:
# Check cooldown
last_alert = self._recent_alerts.get(market_id, 0)
if now - last_alert < self._alert_cooldown:
logger.debug(f"Alert suppressed for {market_id} (cooldown)")
return None
# Record alert
self._recent_alerts[market_id] = now
self._store_alert(alert)
# Log warning
logger.warning(
f"🚨 PRICE VOLATILITY: {market_question[:50]}... "
f"{alert.direction} {abs(alert.price_change_percent):.1%} "
f"({alert.start_price:.2%} → {alert.end_price:.2%}) "
f"in {alert.window_seconds // 60}min"
)
return alert
return None
def _check_volatility(
self, market_id: str, market_question: str, current_time: int
) -> Optional[VolatilityAlert]:
"""
Check if price volatility exceeds threshold within the time window.
Args:
market_id: The market ID
market_question: The market question text
current_time: Current timestamp
Returns:
VolatilityAlert if threshold exceeded, None otherwise
"""
history = self._price_history.get(market_id)
if not history or len(history) < 2:
return None
current_price = history[-1].yes_price
window_start = current_time - self.window_seconds
# Find the oldest price within the window
oldest_in_window = None
for point in history:
if point.timestamp >= window_start:
oldest_in_window = point
break
if oldest_in_window is None:
return None
# Calculate price change
price_change = current_price - oldest_in_window.yes_price
price_change_abs = abs(price_change)
if price_change_abs < self.threshold:
return None
# Create alert
return VolatilityAlert(
market_id=market_id,
market_question=market_question,
start_timestamp=oldest_in_window.timestamp,
end_timestamp=current_time,
start_price=oldest_in_window.yes_price,
end_price=current_price,
price_change=price_change,
price_change_percent=price_change,
direction="UP" if price_change > 0 else "DOWN",
window_seconds=current_time - oldest_in_window.timestamp,
)
def _store_alert(self, alert: VolatilityAlert) -> None:
"""
Store a volatility alert to file.
Args:
alert: The alert to store
"""
alerts_file = VOLATILITY_DIR / "volatility_alerts.json"
# Load existing alerts
existing_alerts = []
if alerts_file.exists():
try:
with open(alerts_file, 'r', encoding='utf-8') as f:
existing_alerts = json.load(f)
except Exception as e:
logger.error(f"Failed to load existing alerts: {e}")
# Add new alert
existing_alerts.append(alert.to_dict())
# Save back
try:
with open(alerts_file, 'w', encoding='utf-8') as f:
json.dump(existing_alerts, f, ensure_ascii=False, indent=2)
except Exception as e:
logger.error(f"Failed to store volatility alert: {e}")
def get_price_history(self, market_id: str) -> List[dict]:
"""
Get price history for a market.
Args:
market_id: The market ID
Returns:
List of price points as dicts
"""
history = self._price_history.get(market_id, deque())
return [{"timestamp": p.timestamp, "yes_price": p.yes_price} for p in history]
def get_all_alerts(self) -> List[dict]:
"""
Get all stored volatility alerts.
Returns:
List of alerts as dicts
"""
alerts_file = VOLATILITY_DIR / "volatility_alerts.json"
if not alerts_file.exists():
return []
try:
with open(alerts_file, 'r', encoding='utf-8') as f:
return json.load(f)
except Exception as e:
logger.error(f"Failed to load alerts: {e}")
return []
def clear_history(self, market_id: Optional[str] = None) -> None:
"""
Clear price history.
Args:
market_id: If provided, clear only this market's history. Otherwise clear all.
"""
if market_id:
if market_id in self._price_history:
del self._price_history[market_id]
logger.info(f"Cleared price history for {market_id}")
else:
self._price_history.clear()
logger.info("Cleared all price history")
async def _fetch_all_active_markets(self) -> List[dict]:
"""
Fetch all active markets from Gamma API.
Returns:
List of market data dicts with id, question, and outcomePrices
"""
all_markets = []
offset = 0
batch_size = 100
try:
while True:
params = {
"active": True,
"closed": False,
"archived": False,
"limit": batch_size,
"offset": offset,
"enableOrderBook": True,
}
response = await self._client.get(GAMMA_API_URL, params=params)
response.raise_for_status()
data = response.json()
if not data:
break
for market in data:
market_id = str(market.get("id", ""))
question = market.get("question", "")
outcome_prices = market.get("outcomePrices", [])
if isinstance(outcome_prices, str):
outcome_prices = json.loads(outcome_prices)
if market_id and outcome_prices:
all_markets.append({
"id": market_id,
"question": question,
"yes_price": float(outcome_prices[0]) if outcome_prices else None,
})
# Cache market info
self._market_info[market_id] = question
if len(data) < batch_size:
break
offset += batch_size
except Exception as e:
logger.error(f"Error fetching active markets: {e}")
return all_markets
async def _poll_all_prices(self) -> List[VolatilityAlert]:
"""
Poll prices for all active markets and check for volatility.
Returns:
List of volatility alerts triggered
"""
alerts = []
markets = await self._fetch_all_active_markets()
logger.debug(f"Polling prices for {len(markets)} active markets")
for market in markets:
market_id = market["id"]
question = market["question"]
yes_price = market.get("yes_price")
if yes_price is not None:
alert = self.record_price(market_id, question, yes_price)
if alert:
alerts.append(alert)
return alerts
async def run(self) -> None:
"""
Start the price monitoring loop.
Continuously polls all active markets at the configured interval.
"""
self._running = True
self._client = get_async_client(timeout=60.0)
logger.info(
f"Starting price monitor (interval: {self.poll_interval}s, "
f"window: {self.window_seconds}s, threshold: {self.threshold:.0%})"
)
try:
while self._running:
try:
alerts = await self._poll_all_prices()
if alerts:
logger.info(f"Detected {len(alerts)} volatility alerts")
# Call callback for each alert
if self._on_volatility_detected:
for alert in alerts:
try:
await self._on_volatility_detected(alert)
except Exception as e:
logger.error(f"Error in volatility callback: {e}")
except Exception as e:
logger.error(f"Error in price monitoring loop: {e}")
await asyncio.sleep(self.poll_interval)
finally:
await self._client.aclose()
self._client = None
def stop(self) -> None:
"""Stop the price monitoring loop."""
self._running = False
logger.info("Price monitor stopping...")
def get_monitored_market_count(self) -> int:
"""Get the number of markets currently being monitored."""
return len(self._price_history)
src/services/resolution_tracker.py
+115
View File
@@ -0,0 +1,115 @@
"""Resolution tracker - checks if markets with signals have resolved and updates correctness."""
import asyncio
import logging
from datetime import datetime
from typing import Optional
from src.db.database import SignalDatabase
from src.services.market_fetcher import MarketFetcher
logger = logging.getLogger(__name__)
class ResolutionTracker:
"""Tracks market resolutions and computes signal correctness."""
def __init__(self, db: SignalDatabase):
self.db = db
self.market_fetcher = MarketFetcher()
def _determine_resolved_outcome(self, market) -> Optional[str]:
"""
Determine the resolved outcome from a market.
A market is considered resolved if closed==True and one outcome price >= 0.99.
Returns:
The winning outcome string (e.g. "Yes" or "No"), or None if not resolved.
"""
if not market.closed:
return None
if not market.outcomes or not market.outcome_prices:
return None
for outcome, price in zip(market.outcomes, market.outcome_prices):
if price >= 0.99:
return outcome
return None
def _is_past_end_date(self, market) -> bool:
"""Check if a market's end_date has passed."""
if not market.end_date:
return True # No end date, always check
try:
end_dt = datetime.fromisoformat(market.end_date.replace("Z", "+00:00"))
return datetime.utcnow().replace(tzinfo=end_dt.tzinfo) >= end_dt
except (ValueError, TypeError):
return True
async def check_all(self) -> dict:
"""
Check all unresolved markets for resolution.
Returns:
Summary dict with counts.
"""
unresolved_ids = self.db.get_unresolved_market_ids()
if not unresolved_ids:
logger.debug("No unresolved markets to check")
return {"checked": 0, "resolved": 0, "signals_updated": 0}
logger.info(f"Checking {len(unresolved_ids)} unresolved markets for resolution")
checked = 0
resolved = 0
signals_updated = 0
for market_id in unresolved_ids:
try:
market = self.market_fetcher.get_market_by_id(market_id)
if not market:
logger.debug(f"Market {market_id} not found on API")
checked += 1
await asyncio.sleep(0.5)
continue
# Optimization: skip markets whose end_date hasn't passed yet
if not self._is_past_end_date(market):
checked += 1
await asyncio.sleep(0.5)
continue
outcome = self._determine_resolved_outcome(market)
if outcome:
updated = self.db.mark_market_resolved(
market_id=market_id,
resolved_outcome=outcome,
resolved_at=datetime.utcnow(),
)
resolved += 1
signals_updated += updated
logger.info(
f"Market resolved: {market.question[:50]}... "
f"outcome={outcome}, {updated} signals updated"
)
checked += 1
await asyncio.sleep(0.5) # Rate limiting
except Exception as e:
logger.error(f"Error checking market {market_id}: {e}")
checked += 1
await asyncio.sleep(0.5)
result = {
"checked": checked,
"resolved": resolved,
"signals_updated": signals_updated,
}
logger.info(
f"Resolution check complete: {checked} checked, "
f"{resolved} resolved, {signals_updated} signals updated"
)
return result
src/services/rtds_client.py
+237
View File
@@ -0,0 +1,237 @@
"""
RTDS (Real-Time Data Socket) client for Polymarket.
Connects to wss://ws-live-data.polymarket.com and subscribes to
activity/trades for real-time trade data across ALL markets.
Replaces the per-market HTTP polling approach with a single persistent
WebSocket connection — zero missed trades, sub-second latency.
"""
import asyncio
import json
import logging
import time
from typing import Awaitable, Callable, Optional
import websockets
from websockets.asyncio.client import ClientConnection
from src.config.settings import get_settings
from src.models.trade import TradeActivity
logger = logging.getLogger(__name__)
RTDS_URI = "wss://ws-live-data.polymarket.com"
HEARTBEAT_INTERVAL = 5 # seconds
RECONNECT_DELAYS = [1, 2, 5, 10, 30, 60] # backoff schedule
class RTDSClient:
"""
Persistent WebSocket client for Polymarket RTDS trade stream.
Features:
- Auto-reconnect with exponential backoff
- Heartbeat (PING every 5s)
- Parses raw messages into TradeActivity objects
- Fires an async callback for each trade
"""
def __init__(
self,
on_trade: Optional[Callable[[TradeActivity], Awaitable[None]]] = None,
proxy: Optional[str] = None,
):
self._on_trade = on_trade
self._proxy = proxy
self._running = False
self._ws: Optional[ClientConnection] = None
self._trade_count = 0
self._connect_count = 0
# ================================================================
# Message parsing
# ================================================================
@staticmethod
def _parse_trade(payload: dict) -> Optional[TradeActivity]:
"""Convert an RTDS trade payload into a TradeActivity."""
try:
side = (payload.get("side") or "").upper()
size = float(payload.get("size", 0) or 0)
price = float(payload.get("price", 0) or 0)
usdc_size = size * price
outcome = payload.get("outcome", "Yes")
outcome_index = int(payload.get("outcomeIndex", 0 if outcome == "Yes" else 1))
ts = int(payload.get("timestamp", 0) or 0)
if ts == 0:
ts = int(time.time())
return TradeActivity(
transaction_hash=payload.get("transactionHash", ""),
timestamp=ts,
condition_id=payload.get("conditionId", ""),
asset=payload.get("asset", ""),
side=side,
size=size,
usdc_size=usdc_size,
price=price,
outcome=outcome,
outcome_index=outcome_index,
title=payload.get("title", ""),
slug=payload.get("slug"),
event_slug=payload.get("eventSlug"),
proxy_wallet=payload.get("proxyWallet"),
name=payload.get("name") or payload.get("pseudonym"),
)
except Exception as e:
logger.debug(f"Failed to parse RTDS trade: {e}")
return None
# ================================================================
# Connection lifecycle
# ================================================================
async def _heartbeat(self, ws: ClientConnection) -> None:
"""Send PING every HEARTBEAT_INTERVAL seconds."""
try:
while True:
await asyncio.sleep(HEARTBEAT_INTERVAL)
await ws.send("PING")
except (asyncio.CancelledError, websockets.ConnectionClosed):
pass
async def _subscribe(self, ws: ClientConnection) -> None:
"""Subscribe to the activity/trades stream."""
msg = {
"action": "subscribe",
"subscriptions": [
{"topic": "activity", "type": "trades", "filters": ""}
],
}
await ws.send(json.dumps(msg))
logger.info("Subscribed to RTDS activity/trades")
async def _consume(self, ws: ClientConnection) -> None:
"""Read messages from the WebSocket and dispatch trades."""
async for raw in ws:
if not self._running:
break
if raw == "PONG" or not raw.strip():
continue
try:
msg = json.loads(raw)
except json.JSONDecodeError:
continue
if msg.get("topic") != "activity" or msg.get("type") != "trades":
continue
payload = msg.get("payload")
if not payload:
continue
activity = self._parse_trade(payload)
if not activity:
continue
self._trade_count += 1
if self._on_trade:
try:
await self._on_trade(activity)
except Exception as e:
logger.error(f"Error in trade callback: {e}")
async def _connect_and_run(self) -> None:
"""Single connection attempt: connect → subscribe → consume."""
self._connect_count += 1
logger.info(
f"Connecting to RTDS ({self._connect_count})... "
f"(total trades so far: {self._trade_count})"
)
connect_kwargs = {"ping_interval": None}
if self._proxy:
connect_kwargs["proxy"] = self._proxy
logger.info(f"RTDS using proxy: {self._proxy}")
async with websockets.connect(RTDS_URI, **connect_kwargs) as ws:
self._ws = ws
logger.info("RTDS connected")
await self._subscribe(ws)
hb_task = asyncio.create_task(self._heartbeat(ws))
try:
await self._consume(ws)
finally:
hb_task.cancel()
self._ws = None
# ================================================================
# Public API
# ================================================================
async def run(self) -> None:
"""
Start the RTDS client with auto-reconnect.
Runs forever until stop() is called.
"""
self._running = True
consecutive_failures = 0
while self._running:
try:
await self._connect_and_run()
# Clean disconnect (stop() called) — exit
if not self._running:
break
# Unexpected clean close — reconnect immediately
consecutive_failures = 0
except (
websockets.ConnectionClosed,
websockets.InvalidURI,
websockets.InvalidHandshake,
OSError,
ConnectionError,
) as e:
if not self._running:
break
delay_idx = min(consecutive_failures, len(RECONNECT_DELAYS) - 1)
delay = RECONNECT_DELAYS[delay_idx]
consecutive_failures += 1
logger.warning(
f"RTDS disconnected: {type(e).__name__}: {e}. "
f"Reconnecting in {delay}s (attempt {consecutive_failures})"
)
await asyncio.sleep(delay)
except Exception as e:
if not self._running:
break
logger.error(f"Unexpected RTDS error: {e}. Reconnecting in 10s")
await asyncio.sleep(10)
logger.info(f"RTDS client stopped (total trades received: {self._trade_count})")
def stop(self) -> None:
"""Stop the RTDS client."""
self._running = False
if self._ws:
asyncio.ensure_future(self._ws.close())
logger.info("RTDS client stopping...")
@property
def trade_count(self) -> int:
"""Total number of trades received since start."""
return self._trade_count
@property
def is_connected(self) -> bool:
"""Whether the WebSocket is currently connected."""
return self._ws is not None and self._ws.state.name == "OPEN"
src/services/serper_search.py
+96
View File
@@ -0,0 +1,96 @@
"""Serper.dev web search service."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
SERPER_SEARCH_URL = "https://google.serper.dev/search"
class SerperSearchService:
"""Web search service using Serper.dev API (Google Search results)."""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or ""
if not self.api_key:
logger.debug("SERPER_API_KEY not set. Serper search will be disabled.")
def is_available(self) -> bool:
return bool(self.api_key and self.api_key.strip())
def search(self, query: str, max_results: int = 5) -> str:
if not self.is_available():
return "Web search unavailable: SERPER_API_KEY not configured."
headers = {
"X-API-KEY": self.api_key,
"Content-Type": "application/json",
}
payload = {"q": query, "num": max_results}
try:
with get_client(timeout=20) as client:
response = client.post(SERPER_SEARCH_URL, json=payload, headers=headers)
if response.status_code != 200:
logger.error(f"Serper API error: {response.status_code} - {response.text[:200]}")
return f"Web search API Error: {response.status_code}"
data = response.json()
# Format results
report = [f"--- Web Search Results for '{query}' ---"]
# Knowledge graph answer
kg = data.get("knowledgeGraph")
if kg:
title = kg.get("title", "")
desc = kg.get("description", "")
if title and desc:
report.append(f"**Summary**: {title} — {desc}\n")
# Answer box
answer_box = data.get("answerBox")
if answer_box:
answer = answer_box.get("answer") or answer_box.get("snippet", "")
if answer:
report.append(f"**Summary**: {answer}\n")
# Organic results
organic = data.get("organic", [])
if not organic:
return f"No web search results found for '{query}'."
for idx, item in enumerate(organic[:max_results], 1):
title = item.get("title", "No title")
url = item.get("link", "")
snippet = item.get("snippet", "")
report.append(f"{idx}. **{title}**")
report.append(f" Source: {url}")
report.append(f" {snippet}")
report.append("")
report.append("-------------------------------------------")
return "\n".join(report)
except Exception as e:
logger.error(f"Serper search failed: {e}")
return f"Web search failed: {str(e)}"
def search_for_market(self, market_question: str, max_results: int = 5) -> str:
if not self.is_available():
return "Web search unavailable: SERPER_API_KEY not configured."
query = market_question[:200]
result = self.search(query, max_results=max_results)
# Propagate errors so the unified WebSearchService can fall back
if "API Error" in result or "search failed" in result:
return result
if "No web search results" not in result and "Error" not in result:
return "## 🔍 Web Search Results (News & Analysis)\n" + result
return f"No relevant web results found for: {market_question[:50]}..."
src/services/stats_engine.py
+90
View File
@@ -0,0 +1,90 @@
"""Stats engine - computes signal performance statistics from the database."""
import logging
from typing import List
from src.db.database import SignalDatabase
from src.models.anomaly_signal import AnomalySignal
logger = logging.getLogger(__name__)
class StatsEngine:
"""Computes signal performance statistics."""
def __init__(self, db: SignalDatabase):
self.db = db
def get_overview(self) -> dict:
"""
Get overall signal performance stats.
Returns:
Dict with total_signals, resolved, correct, win_rate, avg_roi, total_theoretical_pnl.
"""
return self.db.get_stats()
def get_stats_by_likelihood_tier(self) -> List[dict]:
"""
Get stats broken down by information_asymmetry_score tiers.
Tiers: 0.4-0.6, 0.6-0.8, 0.8-1.0
Returns:
List of tier stat dicts.
"""
return self.db.get_stats_by_tier()
def get_recent_resolved(self, limit: int = 20) -> List[AnomalySignal]:
"""Get recently resolved signals."""
return self.db.get_recent_resolved(limit)
def get_best_worst(self, n: int = 5) -> dict:
"""Get best and worst signals by ROI."""
return self.db.get_best_worst(n)
def format_stats_summary(self) -> str:
"""
Format a human-readable stats summary for briefings.
Returns:
Markdown-formatted stats string.
"""
stats = self.get_overview()
tier_stats = self.get_stats_by_likelihood_tier()
if stats["resolved"] == 0:
return ""
lines = [
"## Signal Performance History",
"",
f"| Metric | Value |",
f"|--------|-------|",
f"| Total Signals | {stats['total_signals']} |",
f"| Resolved | {stats['resolved']} |",
f"| Correct | {stats['correct']} |",
f"| Win Rate | **{stats['win_rate']:.1%}** |",
f"| Avg ROI | **{stats['avg_roi']:+.1%}** |",
f"| Total Theoretical PnL | **{stats['total_theoretical_pnl']:+.2f}x** |",
"",
]
# Tier breakdown
has_resolved_tiers = any(t["resolved"] > 0 for t in tier_stats)
if has_resolved_tiers:
lines.extend([
"### By Signal Confidence Tier",
"",
"| Confidence Range | Signals | Resolved | Win Rate | Avg ROI |",
"|-----------------|---------|----------|----------|---------|",
])
for t in tier_stats:
if t["total"] > 0:
wr = f"{t['win_rate']:.0%}" if t["resolved"] > 0 else "N/A"
roi = f"{t['avg_roi']:+.1%}" if t["resolved"] > 0 else "N/A"
lines.append(
f"| {t['tier']} | {t['total']} | {t['resolved']} | {wr} | {roi} |"
)
lines.append("")
return "\n".join(lines)
src/services/tavily_search.py
+141
View File
@@ -0,0 +1,141 @@
"""Tavily web search service - replaces Google Search for whale trade verification."""
import logging
from typing import Optional
import httpx
from src.utils.http import get_client
logger = logging.getLogger(__name__)
TAVILY_SEARCH_URL = "https://api.tavily.com/search"
def _format_search_results(query: str, results: list[dict]) -> str:
"""Format Tavily search results as a readable report."""
report = [f"--- Web Search Results for '{query}' ---"]
for idx, item in enumerate(results, 1):
title = item.get("title", "No title")
url = item.get("url", "")
content = item.get("content", "")[:300]
if len(item.get("content", "")) > 300:
content += "..."
report.append(f"{idx}. **{title}**")
report.append(f" Source: {url}")
report.append(f" {content}")
report.append("")
report.append("-------------------------------------------")
return "\n".join(report)
class TavilySearchService:
"""
Web search service using Tavily API for whale trade verification.
Replaces Google Search grounding with explicit Tavily web search,
passing results as context to the LLM.
"""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or ""
if not self.api_key:
logger.warning("TAVILY_API_KEY not set. Web search will be disabled.")
def is_available(self) -> bool:
"""Check if Tavily search is available (API key is set)."""
return bool(self.api_key and self.api_key.strip())
def search(
self,
query: str,
max_results: int = 5,
search_depth: str = "basic",
) -> str:
"""
Search the web using Tavily API.
Args:
query: Search query
max_results: Number of results to return (1-10)
search_depth: "basic" for fast search, "advanced" for deeper search
Returns:
Formatted search results report.
"""
if not self.is_available():
return "Web search unavailable: TAVILY_API_KEY not configured."
max_results = max(1, min(max_results, 10))
payload = {
"api_key": self.api_key,
"query": query,
"search_depth": search_depth,
"max_results": max_results,
"include_answer": True,
}
try:
with get_client(timeout=20) as client:
response = client.post(TAVILY_SEARCH_URL, json=payload)
if response.status_code != 200:
logger.error(f"Tavily API error: {response.status_code} - {response.text[:200]}")
return f"Web search API Error: {response.status_code}"
data = response.json()
results = data.get("results", [])
if not results:
return f"No web search results found for '{query}'."
report_parts = []
# Include AI-generated answer summary if available
answer = data.get("answer")
if answer:
report_parts.append(f"**Summary**: {answer}\n")
report_parts.append(_format_search_results(query, results))
return "\n".join(report_parts)
except Exception as e:
logger.error(f"Tavily search failed: {e}")
return f"Web search failed: {str(e)}"
def search_for_market(
self,
market_question: str,
max_results: int = 5,
) -> str:
"""
Search the web for information relevant to a prediction market.
Performs a deeper search for market-relevant news.
Args:
market_question: The market question to search for
max_results: Number of results per query
Returns:
Combined search results.
"""
if not self.is_available():
return "Web search unavailable: TAVILY_API_KEY not configured."
results = []
# Search with the market question directly
query = market_question[:200]
main_result = self.search(query, max_results=max_results, search_depth="advanced")
# Propagate errors so the unified WebSearchService can fall back
if "API Error" in main_result or "search failed" in main_result:
return main_result
if "No web search results" not in main_result and "Error" not in main_result:
results.append("## 🔍 Web Search Results (News & Analysis)\n" + main_result)
if not results:
return f"No relevant web results found for: {market_question[:50]}..."
return "\n\n".join(results)
src/services/telegram_search.py
+191
View File
@@ -0,0 +1,191 @@
"""Telegram search service for crypto and geopolitical channel monitoring."""
import asyncio
import logging
from datetime import datetime, timedelta
from typing import Optional, List
try:
import nest_asyncio
nest_asyncio.apply()
except ImportError:
pass
logger = logging.getLogger(__name__)
# Default public channels: crypto + geopolitics/politics
# Verified 2026-03-31: all channels return text messages with meaningful content
DEFAULT_CHANNELS = [
# Crypto / macro
"CryptoVIPSignalTA", # crypto signals + macro news
"whale_alert_io", # on-chain whale transfers
"WatcherGuru", # crypto/macro breaking news
# Geopolitics & politics
"DDGeopolitics", # geopolitical analysis (views ~20k)
"disclosetv", # US politics, breaking news (views ~50-70k)
"realDonaldTrump", # Trump's own posts
"intelslava", # Russia/Ukraine, geopolitics (views ~50k)
"TheScrollOfBenjamin", # Middle East geopolitics
"WarMonitors", # conflict breaking news (views ~14k, active)
]
class TelegramSearchService:
"""
Searches crypto-relevant public Telegram channels for messages.
Uses Telethon (MTProto API) to search public channels by keyword.
Requires a one-time auth to generate a session string.
"""
def __init__(
self,
api_id: str = "",
api_hash: str = "",
session_string: str = "",
channels: Optional[List[str]] = None,
):
self.api_id = api_id
self.api_hash = api_hash
self.session_string = session_string
self.channels = channels or DEFAULT_CHANNELS
self._client = None
def is_available(self) -> bool:
"""Check if Telegram search is available."""
if not (self.api_id and self.api_hash and self.session_string):
return False
try:
import telethon # noqa: F401
return True
except ImportError:
logger.warning("telethon not installed. Telegram search disabled.")
return False
def _get_client(self):
"""Get or create the Telethon client."""
if self._client is None:
from telethon import TelegramClient
from telethon.sessions import StringSession
self._client = TelegramClient(
StringSession(self.session_string),
int(self.api_id),
self.api_hash,
)
return self._client
async def _search_channel(
self,
channel: str,
query: str,
limit: int = 5,
) -> List[dict]:
"""Search a single channel for messages matching a query."""
from telethon.errors import (
FloodWaitError,
ChannelPrivateError,
UsernameNotOccupiedError,
UsernameInvalidError,
)
results = []
try:
client = self._get_client()
async for message in client.iter_messages(
channel,
search=query,
limit=limit,
):
if not message.text:
continue
results.append({
"channel": channel,
"text": message.text,
"date": message.date.strftime("%Y-%m-%d %H:%M UTC") if message.date else "",
"views": message.views or 0,
"forwards": message.forwards or 0,
})
except FloodWaitError as e:
logger.warning(f"Telegram flood wait: {e.seconds}s for channel {channel}")
except (ChannelPrivateError, UsernameNotOccupiedError, UsernameInvalidError):
logger.debug(f"Channel {channel} not accessible, skipping")
except Exception as e:
logger.error(f"Error searching Telegram channel {channel}: {e}")
return results
async def _search_all_channels(self, query: str, limit_per_channel: int = 5) -> List[dict]:
"""Search all configured channels."""
client = self._get_client()
async with client:
all_results = []
for channel in self.channels:
results = await self._search_channel(channel, query, limit_per_channel)
all_results.extend(results)
# Sort by views descending
all_results.sort(key=lambda x: x["views"], reverse=True)
return all_results
def _format_report(self, query: str, messages: List[dict]) -> str:
"""Format search results as a report string."""
if not messages:
return f"No Telegram messages found for '{query}'."
total_views = sum(m["views"] for m in messages)
lines = [
f"--- Telegram Search Results for '{query}' ---",
f"Total Views in Sample: {total_views:,}",
f"Channels Searched: {', '.join(self.channels)}",
"Messages:",
]
for idx, msg in enumerate(messages):
text_preview = msg["text"][:200]
if len(msg["text"]) > 200:
text_preview += "..."
lines.append(
f'{idx + 1}. [{msg["channel"]}] ({msg["date"]}, '
f'👁 {msg["views"]:,}): "{text_preview}"'
)
lines.append("-------------------------------------------")
return "\n".join(lines)
def search_for_market(self, query: str, limit: int = 10) -> str:
"""
Search Telegram channels for messages relevant to a market.
Args:
query: Search query
limit: Max total messages to return
Returns:
Formatted report string
"""
if not self.is_available():
return "Telegram search unavailable: credentials not configured or telethon not installed."
try:
# Calculate per-channel limit
limit_per_channel = max(3, limit // len(self.channels))
# Run async search — handle both sync and async calling contexts
coro = self._search_all_channels(query, limit_per_channel)
try:
loop = asyncio.get_running_loop()
# Already inside an async event loop — use a new thread
import concurrent.futures
with concurrent.futures.ThreadPoolExecutor() as pool:
messages = pool.submit(
asyncio.run, coro
).result(timeout=30)
except RuntimeError:
# No running loop — safe to use run_until_complete
loop = asyncio.get_event_loop()
messages = loop.run_until_complete(coro)
# Trim to total limit
messages = messages[:limit]
return self._format_report(query, messages)
except Exception as e:
logger.error(f"Telegram search failed: {e}")
return f"Telegram search failed: {str(e)}"
src/services/tools.py
+467
View File
@@ -0,0 +1,467 @@
"""
Tool registry for LLM function calling.
Each tool is a callable that the LLM can invoke on demand.
Tools are registered with their OpenAI-compatible function schema
and an executor function that performs the actual work.
"""
import json
import logging
from dataclasses import dataclass
from typing import Callable, Dict, List
from src.services.twitter_search import TwitterSearchService
from src.services.web_search import WebSearchService
from src.services.coingecko import CoinGeckoService
from src.services.fred import FREDService
from src.services.polygon import PolygonService
from src.services.congress import CongressService
from src.services.defillama import DefiLlamaService
from src.services.etherscan import EtherscanService
from src.services.telegram_search import TelegramSearchService
logger = logging.getLogger(__name__)
@dataclass
class Tool:
"""A tool available to the LLM."""
name: str
description: str
parameters: dict # JSON Schema for parameters
execute: Callable[..., str] # (kwargs) -> result string
class ToolRegistry:
"""
Registry of tools available for LLM function calling.
Usage:
registry = ToolRegistry(twitter_api_key="...", tavily_api_key="...")
schemas = registry.openai_tool_schemas() # pass to LLM
result = registry.call("search_twitter", query="Bitcoin") # execute
"""
def __init__(
self,
twitter_api_key: str,
tavily_api_key: str,
fred_api_key: str = "",
polygon_api_key: str = "",
congress_api_key: str = "",
etherscan_api_key: str = "",
serper_api_key: str = "",
telegram_api_id: str = "",
telegram_api_hash: str = "",
telegram_session_string: str = "",
telegram_channels: str = "",
):
self._tools: Dict[str, Tool] = {}
# -- Twitter search --
twitter = TwitterSearchService(api_key=twitter_api_key)
if twitter.is_available():
self._register(Tool(
name="search_twitter",
description=(
"Search Twitter/X for real-time social sentiment, KOL opinions, "
"and breaking news about a topic. Returns top and latest tweets "
"with engagement metrics. Best for: real-time sentiment, crypto "
"community reactions, political commentary, breaking news that "
"hasn't hit mainstream media yet."
),
parameters={
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query (e.g. 'Bitcoin ETF', 'Trump indictment')",
},
},
"required": ["query"],
},
execute=lambda query: twitter.search_for_market(query, limit=10),
))
# -- Telegram search (crypto channels) --
telegram = TelegramSearchService(
api_id=telegram_api_id,
api_hash=telegram_api_hash,
session_string=telegram_session_string,
channels=telegram_channels.split(",") if telegram_channels.strip() else None,
)
if telegram.is_available():
self._register(Tool(
name="search_telegram",
description=(
"Search Telegram channels for recent messages about a topic. "
"Covers crypto (Whale Alert, WatcherGuru, CryptoVIPSignalTA) and "
"politics/geopolitics (Disclose.tv, DDGeopolitics, Intel Slava, "
"PoliticsForAll, Trump, TheScrollOfBenjamin). "
"Returns messages with view counts. "
"Best for: US politics, Trump news, approval ratings, elections, "
"geopolitics, military conflicts, Russia/Ukraine, Middle East, "
"macro economics, crypto news, whale transfers, and breaking news."
),
parameters={
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query (e.g. 'MegaETH launch', 'Solana outage')",
},
},
"required": ["query"],
},
execute=lambda query: telegram.search_for_market(query, limit=10),
))
# -- Web search (Tavily -> Serper -> DuckDuckGo fallback) --
web_search = WebSearchService(
tavily_api_key=tavily_api_key,
serper_api_key=serper_api_key,
)
if web_search.is_available():
self._register(Tool(
name="search_web",
description=(
"Search the web for recent news articles, analysis, and factual "
"information about a topic. Returns article summaries with sources. "
"Best for: verifying events, finding official announcements, "
"regulatory news, earnings reports, court rulings, legislation status."
),
parameters={
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query for news and analysis",
},
},
"required": ["query"],
},
execute=lambda query: web_search.search_for_market(query, max_results=5),
))
# -- CoinGecko crypto data --
coingecko = CoinGeckoService()
self._register(Tool(
name="get_crypto_price",
description=(
"Get real-time cryptocurrency price, 24h/7d/30d change, market cap, "
"volume, and ATH data. Accepts ticker symbols (BTC, ETH, SOL) or "
"CoinGecko IDs. Best for: any market involving crypto price targets "
"(e.g. 'Will BTC hit $100k'), crypto market conditions."
),
parameters={
"type": "object",
"properties": {
"coin": {
"type": "string",
"description": "Ticker symbol (BTC, ETH, SOL) or CoinGecko coin ID",
},
},
"required": ["coin"],
},
execute=lambda coin: coingecko.get_price(coin),
))
self._register(Tool(
name="get_crypto_market_overview",
description=(
"Get global crypto market overview: total market cap, 24h change, "
"BTC/ETH dominance, trading volume. Best for: understanding overall "
"crypto market sentiment and conditions."
),
parameters={
"type": "object",
"properties": {},
},
execute=lambda: coingecko.get_market_overview(),
))
# -- FRED macroeconomic data --
fred = FREDService(api_key=fred_api_key)
if fred.is_available():
self._register(Tool(
name="get_economic_data",
description=(
"Get macroeconomic data from FRED (Federal Reserve). Supports "
"common names: fed_rate, cpi, inflation, unemployment, gdp, "
"oil_price, wti, brent, gold, vix, sp500, yield_curve, "
"jobless_claims, 10y_treasury, 2y_treasury, dollar_index. "
"Also accepts any FRED series ID (e.g. FEDFUNDS, UNRATE). "
"Returns recent data points with trend. Best for: Fed policy "
"markets, inflation bets, employment data, oil/commodity prices, "
"recession indicators."
),
parameters={
"type": "object",
"properties": {
"query": {
"type": "string",
"description": (
"Common name (fed_rate, cpi, unemployment, oil_price, "
"vix, gold, sp500) or FRED series ID"
),
},
},
"required": ["query"],
},
execute=lambda query: fred.get_series(query),
))
# -- Polygon.io financial data --
polygon = PolygonService(api_key=polygon_api_key)
if polygon.is_available():
self._register(Tool(
name="get_stock_price",
description=(
"Get real-time stock/ETF price snapshot from Polygon.io. "
"Includes price, daily change, volume, day range. "
"Examples: AAPL, TSLA, GS, META, SPY, QQQ, GLD, USO. "
"Best for: markets involving specific company events "
"(IPOs, earnings, lawsuits), sector ETFs, gold/oil ETFs."
),
parameters={
"type": "object",
"properties": {
"ticker": {
"type": "string",
"description": "Ticker symbol (e.g. AAPL, TSLA, GS, SPY, GLD)",
},
},
"required": ["ticker"],
},
execute=lambda ticker: polygon.get_ticker_snapshot(ticker),
))
self._register(Tool(
name="get_stock_news",
description=(
"Get recent news articles for a stock/company from Polygon.io. "
"Returns headlines, sources, and summaries. "
"Best for: company-specific events, earnings surprises, "
"M&A rumors, regulatory actions, CEO statements."
),
parameters={
"type": "object",
"properties": {
"ticker": {
"type": "string",
"description": "Stock ticker (e.g. AAPL, TSLA, GS)",
},
},
"required": ["ticker"],
},
execute=lambda ticker: polygon.get_market_news(ticker),
))
# -- Congress.gov legislative data --
congress_svc = CongressService(api_key=congress_api_key)
if congress_svc.is_available():
self._register(Tool(
name="get_bill_status",
description=(
"Get status of a specific U.S. Congressional bill. "
"Requires congress number (e.g. 119), bill type (hr, s, hjres, sjres), "
"and bill number. Returns sponsor, cosponsors, latest action, "
"committee referrals. Best for: markets about specific legislation "
"(TikTok ban, crypto regulation, immigration reform, tax bills)."
),
parameters={
"type": "object",
"properties": {
"congress": {
"type": "integer",
"description": "Congress number (119 for 2025-2026)",
},
"bill_type": {
"type": "string",
"description": "Bill type: hr (House), s (Senate), hjres, sjres",
},
"bill_number": {
"type": "integer",
"description": "Bill number",
},
},
"required": ["congress", "bill_type", "bill_number"],
},
execute=lambda congress, bill_type, bill_number: congress_svc.get_bill_status(
congress, bill_type, bill_number
),
))
self._register(Tool(
name="get_recent_legislation",
description=(
"Get recently updated U.S. Congressional bills. "
"Returns latest bills with their current status and actions. "
"Best for: understanding current legislative activity, "
"political markets about government actions, policy changes."
),
parameters={
"type": "object",
"properties": {},
},
execute=lambda: congress_svc.search_bills("", limit=5),
))
# -- DeFiLlama (DeFi protocol data, no API key needed) --
defillama = DefiLlamaService()
self._register(Tool(
name="get_protocol_tvl",
description=(
"Get DeFi protocol TVL (Total Value Locked), TVL changes (1h/24h/7d), "
"and chain breakdown from DeFiLlama. Accepts protocol name or slug "
"(e.g. 'aave', 'uniswap', 'lido', 'eigenlayer'). "
"Best for: token launch FDV markets, DeFi protocol health, "
"evaluating project fundamentals before/after token launch."
),
parameters={
"type": "object",
"properties": {
"protocol": {
"type": "string",
"description": "Protocol name or slug (e.g. 'aave', 'uniswap', 'megaeth')",
},
},
"required": ["protocol"],
},
execute=lambda protocol: defillama.get_protocol_tvl(protocol),
))
self._register(Tool(
name="get_token_unlocks",
description=(
"Get token unlock/vesting schedule for a DeFi protocol from DeFiLlama. "
"Shows allocation categories and upcoming unlock events. "
"Best for: understanding token supply dynamics, evaluating FDV markets, "
"predicting sell pressure from upcoming unlocks."
),
parameters={
"type": "object",
"properties": {
"protocol": {
"type": "string",
"description": "Protocol name or slug (e.g. 'arbitrum', 'optimism', 'eigenlayer')",
},
},
"required": ["protocol"],
},
execute=lambda protocol: defillama.get_token_unlocks(protocol),
))
self._register(Tool(
name="get_protocol_revenue",
description=(
"Get DeFi protocol fees and revenue (24h/7d/30d/all-time) from DeFiLlama. "
"Best for: evaluating protocol fundamentals, comparing revenue vs FDV, "
"assessing if a token launch valuation is justified."
),
parameters={
"type": "object",
"properties": {
"protocol": {
"type": "string",
"description": "Protocol name or slug (e.g. 'aave', 'uniswap', 'gmx')",
},
},
"required": ["protocol"],
},
execute=lambda protocol: defillama.get_protocol_revenue(protocol),
))
# -- Etherscan (on-chain data) --
etherscan = EtherscanService(api_key=etherscan_api_key)
if etherscan.is_available():
self._register(Tool(
name="get_wallet_transfers",
description=(
"Get recent ERC-20 token transfers (USDC, USDT, WETH, DAI) for an "
"Ethereum wallet address from Etherscan. Shows direction (IN/OUT), "
"amount, counterparty, and flags large transfers (>$10k). "
"Best for: checking if a Polymarket whale recently received large "
"USDC deposits (funding for trades), tracking wallet activity."
),
parameters={
"type": "object",
"properties": {
"address": {
"type": "string",
"description": "Ethereum wallet address (0x...)",
},
"token": {
"type": "string",
"description": "Token to track: USDC, USDT, WETH, or DAI (default: USDC)",
},
},
"required": ["address"],
},
execute=lambda address, token="USDC": etherscan.get_wallet_token_transfers(address, token),
))
self._register(Tool(
name="get_contract_info",
description=(
"Check if an Ethereum address is a smart contract, when it was created, "
"its name and verification status from Etherscan. "
"Best for: verifying if a crypto project has deployed contracts, "
"checking contract activity for token launch markets."
),
parameters={
"type": "object",
"properties": {
"address": {
"type": "string",
"description": "Ethereum contract address (0x...)",
},
},
"required": ["address"],
},
execute=lambda address: etherscan.get_contract_info(address),
))
def _register(self, tool: Tool):
self._tools[tool.name] = tool
logger.info(f"Registered tool: {tool.name}")
@property
def available_tools(self) -> List[str]:
return list(self._tools.keys())
def openai_tool_schemas(self) -> List[dict]:
"""Return tool schemas in OpenAI function-calling format."""
return [
{
"type": "function",
"function": {
"name": tool.name,
"description": tool.description,
"parameters": tool.parameters,
},
}
for tool in self._tools.values()
]
def call(self, name: str, **kwargs) -> str:
"""
Execute a tool by name.
Returns the tool's string result, or an error message if the tool
is not found or execution fails.
"""
tool = self._tools.get(name)
if not tool:
msg = f"Tool '{name}' not found. Available: {self.available_tools}"
logger.error(msg)
return msg
try:
result = tool.execute(**kwargs)
logger.info(f"Tool {name} executed successfully ({len(result)} chars)")
return result
except Exception as e:
msg = f"Tool '{name}' failed: {e}"
logger.error(msg)
return msg
src/services/trade_monitor.py
+698
View File
@@ -0,0 +1,698 @@
"""
Trade monitoring service — RTDS WebSocket architecture.
Single WebSocket connection receives ALL trades in real-time from
Polymarket RTDS (wss://ws-live-data.polymarket.com).
For each incoming trade:
1. Record for cluster detection (anomaly detector)
2. Dedup by transaction hash
3. Filter: whale pre-filter (price range, size, conviction)
4. Enrich: trader ranking + history → anomaly score
5. If score passes threshold → full enrichment + LLM callback
Replaces the previous per-market HTTP polling architecture.
"""
import asyncio
import json
import logging
import math
import time as _time
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Optional, Set, Callable, Awaitable
import httpx
from src.config import get_settings
from src.models.market import Market, TrendingMarket
from src.models.trade import (
TradeActivity, WhaleTrade, TraderRanking, TraderHistory,
EventPosition, MarketTopTrader,
)
from src.services.anomaly_detector import AnomalyDetector
from src.services.rtds_client import RTDSClient
from src.utils.http import get_async_client
logger = logging.getLogger(__name__)
# Gamma API for fetching latest market prices
GAMMA_API_URL = "https://gamma-api.polymarket.com/markets"
# File to persist processed transaction hashes
PROCESSED_TXNS_FILE = Path(__file__).parent.parent.parent / "data" / "processed_transactions.json"
class TradeMonitor:
"""
Monitors Polymarket markets for large trades via RTDS WebSocket.
Architecture: single WebSocket connection → filter → enrich → callback.
"""
def __init__(
self,
on_whale_detected: Optional[Callable[[WhaleTrade], Awaitable[None]]] = None,
):
self.settings = get_settings()
# RTDS WebSocket client (created in run())
self._rtds: Optional[RTDSClient] = None
# HTTP client for enrichment API calls (trader ranking, history, etc.)
self.data_api_url = "https://data-api.polymarket.com"
self.leaderboard_endpoint = f"{self.data_api_url}/v1/leaderboard"
self.trades_endpoint = f"{self.data_api_url}/trades"
self._client = get_async_client(
timeout=httpx.Timeout(30.0, pool=120.0),
limits=httpx.Limits(
max_connections=50,
max_keepalive_connections=20,
keepalive_expiry=30,
),
)
# Cache for trader rankings to avoid repeated API calls
self._trader_ranking_cache: Dict[str, TraderRanking] = {}
# Markets being monitored: condition_id -> Market
# Used for enrichment (market question, description, etc.)
self._monitored_markets: Dict[str, Market] = {}
# condition_id -> market_id mapping
self._condition_to_market_id: Dict[str, str] = {}
# Track processed transactions to avoid duplicates
self._processed_txns: Set[str] = set()
self._load_processed_txns()
# Anomaly detector for multi-dimensional scoring
self._anomaly_detector = AnomalyDetector()
# Callback for whale detection
self._on_whale_detected = on_whale_detected
# Control flag
self._running = False
# Flag to suppress alerts during initial warmup
self._warmup_complete = False
self._warmup_seconds = 10 # seconds to collect baseline before alerting
# ================================================================
# Persistence
# ================================================================
def _load_processed_txns(self):
"""Load processed transaction hashes from JSON file."""
try:
if PROCESSED_TXNS_FILE.exists():
with open(PROCESSED_TXNS_FILE, "r") as f:
data = json.load(f)
self._processed_txns = set(data.get("transactions", []))
logger.info(f"Loaded {len(self._processed_txns)} processed transactions from file")
except json.JSONDecodeError as e:
logger.warning(f"Corrupted JSON file, backing up and starting fresh: {e}")
if PROCESSED_TXNS_FILE.exists():
backup_file = PROCESSED_TXNS_FILE.with_suffix('.json.bak')
PROCESSED_TXNS_FILE.rename(backup_file)
logger.info(f"Backed up corrupted file to {backup_file}")
self._processed_txns = set()
except Exception as e:
logger.warning(f"Failed to load processed transactions: {e}")
self._processed_txns = set()
def _save_processed_txns(self):
"""Save processed transaction hashes to JSON file."""
try:
PROCESSED_TXNS_FILE.parent.mkdir(parents=True, exist_ok=True)
with open(PROCESSED_TXNS_FILE, "w") as f:
json.dump({
"transactions": list(self._processed_txns),
"count": len(self._processed_txns),
"last_updated": datetime.now().isoformat()
}, f, indent=2)
logger.debug(f"Saved {len(self._processed_txns)} processed transactions to file")
except Exception as e:
logger.warning(f"Failed to save processed transactions: {e}")
async def close(self):
"""Cleanup resources."""
self._save_processed_txns()
await self._client.aclose()
# ================================================================
# Market list management
# ================================================================
def set_monitored_markets(self, markets: List[TrendingMarket]):
"""Update the list of markets to monitor."""
self._monitored_markets = {}
self._condition_to_market_id = {}
for tm in markets:
m = tm.market
if m.id and m.condition_id:
self._monitored_markets[m.condition_id] = m
self._condition_to_market_id[m.condition_id] = m.id
logger.info(f"Now monitoring {len(self._monitored_markets)} markets")
def set_tiered_markets(self, tiers: dict[str, list]) -> None:
"""Set markets from tiered scan (same interface as before)."""
self._monitored_markets = {}
self._condition_to_market_id = {}
for tier_name, markets in tiers.items():
for tm in markets:
m = tm.market
if m.id and m.condition_id:
self._monitored_markets[m.condition_id] = m
self._condition_to_market_id[m.condition_id] = m.id
total = len(self._monitored_markets)
tier_counts = {k: len(v) for k, v in tiers.items()}
logger.info(f"Tiered monitoring: {tier_counts}, total={total}")
# ================================================================
# RTDS trade handler (core of the new architecture)
# ================================================================
async def _on_rtds_trade(self, activity: TradeActivity) -> None:
"""
Called for every trade received from RTDS WebSocket.
This replaces the per-market polling loop.
"""
condition_id = activity.condition_id
# Look up market info (enrichment data)
market = self._monitored_markets.get(condition_id)
market_id = self._condition_to_market_id.get(condition_id, "")
# Record every trade for cluster detection (even unmonitored markets)
if market_id:
self._anomaly_detector.record_trade(activity, market_id)
# Dedup by transaction hash + outcome (same tx can have multiple fills)
dedup_key = f"{activity.transaction_hash}_{activity.outcome}_{activity.size}"
if dedup_key in self._processed_txns:
return
self._processed_txns.add(dedup_key)
# Skip unmonitored markets
if not market:
return
# Skip during warmup period (avoid alerting on historical trades)
if not self._warmup_complete:
return
# Only track BUY trades (new positions)
if activity.side != "BUY":
return
# Whale pre-filter
if not self._is_whale_trade(activity, market=market):
return
# Handle whale (enrich + score + callback)
asyncio.create_task(self._handle_whale(activity, market_id, market))
# ================================================================
# Whale detection (unchanged from original)
# ================================================================
def _is_whale_trade(self, activity: TradeActivity, market: Optional[Market] = None) -> bool:
"""
Multi-layer pre-filter mirroring options flow SignalFilter._check_signal.
Filter chain (early rejection):
1. Price range — like moneyness filter (OTM/ITM range)
2. Direction — BUY only (like enabled direction_filters)
3. Resolution window — like DTE filter (3-60 days sweet spot)
4. Size — like premium filter ($250K+ minimum)
5. Dynamic size — like dynamic_premium (base × √(vol / baseline))
5.5 Normalized size — like normalized_premium (usdc / √(vol))
6. Signal strength — like ask_ratio filter (conviction check)
"""
# --- 1. Price range ---
if not (self.settings.min_price <= activity.price <= self.settings.max_price):
return False
# --- 2. Direction: BUY only ---
# Already enforced upstream
# --- 3. Resolution window ---
if market and market.end_date:
try:
end_dt = datetime.fromisoformat(market.end_date.replace("Z", "+00:00"))
now_dt = datetime.utcnow().replace(tzinfo=end_dt.tzinfo) if end_dt.tzinfo else datetime.utcnow()
hours_to_resolution = max(0, (end_dt - now_dt).total_seconds() / 3600)
if hours_to_resolution < 3:
return False
if hours_to_resolution > 180 * 24:
return False
except (ValueError, TypeError):
pass
# --- 4. Size ---
if activity.usdc_size < 3_000:
return False
# --- 5. Dynamic size ---
base_size = 5_000.0
baseline_volume = 1_000_000.0
if market and market.volume > 0:
threshold = base_size * math.sqrt(market.volume / baseline_volume)
threshold = max(3_000.0, min(threshold, 50_000.0))
else:
threshold = base_size
if activity.usdc_size < threshold:
return False
# --- 5.5 Normalized size (like normalized_premium) ---
# usdc_size / √(volume) makes signals comparable across market sizes.
# A $5K trade in a $50K market is far more significant than $20K in a $10M market.
if market and market.volume > 0:
normalized = activity.usdc_size / math.sqrt(market.volume)
# Minimum normalized threshold: filters out trades that are trivial
# relative to market size (calibrated: $5K in a $1M market → 5.0)
if normalized < 1.5:
return False
# --- 6. Signal strength ---
if market and market.outcome_prices:
if activity.outcome == "Yes":
market_mid = market.outcome_prices[0]
elif len(market.outcome_prices) > 1:
market_mid = market.outcome_prices[1]
else:
market_mid = 1.0 - market.outcome_prices[0]
if activity.price < market_mid + 0.01:
return False
return True
async def _handle_whale(self, activity: TradeActivity, market_id: str, market: Market):
"""
Handle a single whale trade:
1. Fetch trader info (ranking + history) for anomaly scoring
2. Compute multi-dimensional anomaly score as pre-filter
3. If score passes threshold, fetch full enrichment data and fire LLM callback
"""
try:
# Phase 1: Quick fetch — ranking + history for anomaly scoring
trader_ranking, trader_history = await asyncio.gather(
self.fetch_trader_ranking(activity.proxy_wallet),
self.fetch_trader_history(activity.proxy_wallet),
)
# Phase 2: Anomaly scoring
should_analyze, score, breakdown = self._anomaly_detector.should_analyze(
activity, market=market, trader_history=trader_history,
market_id=market_id,
)
rank_str = f"(Rank #{trader_ranking.rank})" if trader_ranking and trader_ranking.rank else "(Unranked)"
breakdown_short = " | ".join(f"{k}={v:.2f}" for k, v in breakdown.items())
if not should_analyze:
logger.info(
f"⚪ Whale below threshold: ${activity.usdc_size:,.2f} "
f"BUY {activity.outcome} @ {activity.price:.4f} {rank_str} "
f"score={score:.2f} [{breakdown_short}] — skipped LLM"
)
return
logger.info(
f"🐋 Whale trade detected! ${activity.usdc_size:,.2f} "
f"BUY {activity.outcome} @ {activity.price:.4f} {rank_str} "
f"score={score:.2f} [{breakdown_short}] on '{market.question[:50]}...'"
)
# Phase 3: Full enrichment
event_positions, (top_buyers, top_sellers) = await asyncio.gather(
self.fetch_whale_event_positions(
activity.proxy_wallet,
activity.event_slug,
market.condition_id or "",
),
self.fetch_market_top_traders(
market_id, condition_id=market.condition_id or "",
outcome_prices=market.outcome_prices,
),
)
whale_trade = WhaleTrade(
id=f"{market_id}_{activity.transaction_hash}",
trade=activity,
market_id=market_id,
market_question=market.question,
market_description=market.description,
market_outcomes=market.outcomes,
market_outcome_prices=market.outcome_prices,
trader_ranking=trader_ranking,
trader_history=trader_history,
whale_event_positions=event_positions,
market_top_buyers=top_buyers,
market_top_sellers=top_sellers,
)
# Fire callback (LLM report generation)
if self._on_whale_detected:
await self._on_whale_detected(whale_trade)
except Exception as e:
logger.error(f"Error handling whale trade in {market_id}: {e}")
# ================================================================
# Enrichment API calls (unchanged — still uses HTTP)
# ================================================================
async def fetch_trader_ranking(self, wallet_address: str) -> Optional[TraderRanking]:
"""Fetch trader ranking from the leaderboard API."""
if not wallet_address:
return None
if wallet_address in self._trader_ranking_cache:
return self._trader_ranking_cache[wallet_address]
try:
params = {
"user": wallet_address,
"timePeriod": "ALL",
"orderBy": "PNL",
}
response = await self._client.get(self.leaderboard_endpoint, params=params)
response.raise_for_status()
data = response.json()
if data and len(data) > 0:
user_data = data[0]
ranking = TraderRanking(
rank=user_data.get("rank"),
pnl=float(user_data.get("pnl", 0) or 0),
volume=float(user_data.get("vol", 0) or 0),
user_name=user_data.get("userName"),
profile_image=user_data.get("profileImage"),
verified=bool(user_data.get("verifiedBadge")),
time_period="ALL",
)
self._trader_ranking_cache[wallet_address] = ranking
logger.debug(f"Fetched ranking for {wallet_address}: #{ranking.rank}")
return ranking
return None
except httpx.HTTPError as e:
logger.debug(f"HTTP error fetching ranking for {wallet_address}: {e}")
return None
except Exception as e:
logger.debug(f"Error fetching ranking for {wallet_address}: {e}")
return None
async def fetch_trader_history(self, wallet_address: str) -> Optional[TraderHistory]:
"""Fetch trader's recent trading history."""
if not wallet_address:
return None
try:
params = {
"user": wallet_address,
"limit": 100,
}
response = await self._client.get(self.trades_endpoint, params=params)
response.raise_for_status()
data = response.json()
if not data:
return None
total_trades = len(data)
total_volume = 0.0
large_trades_count = 0
recent_markets: Set[str] = set()
recent_trades = []
for trade in data:
usdc_size = float(trade.get("usdcSize", 0) or 0)
if usdc_size == 0:
size = float(trade.get("size", 0) or 0)
price = float(trade.get("price", 0) or 0)
usdc_size = size * price
total_volume += usdc_size
if usdc_size >= 5000:
large_trades_count += 1
recent_trades.append({
"side": trade.get("side", ""),
"usdc_size": usdc_size,
"price": float(trade.get("price", 0) or 0),
"title": trade.get("title", trade.get("marketTitle", "")),
"timestamp": trade.get("timestamp", 0),
})
title = trade.get("title", trade.get("marketTitle", ""))
if title:
recent_markets.add(title[:50])
avg_trade_size = total_volume / total_trades if total_trades > 0 else 0
recent_trades.sort(key=lambda x: x["usdc_size"], reverse=True)
return TraderHistory(
total_trades=total_trades,
total_volume=total_volume,
avg_trade_size=avg_trade_size,
large_trades_count=large_trades_count,
recent_markets=list(recent_markets)[:10],
recent_trades=recent_trades[:10],
)
except httpx.HTTPError as e:
logger.debug(f"HTTP error fetching history for {wallet_address}: {e}")
return None
except Exception as e:
logger.debug(f"Error fetching history for {wallet_address}: {e}")
return None
async def fetch_whale_event_positions(
self,
wallet_address: str,
event_slug: str,
current_condition_id: str,
) -> List[EventPosition]:
"""Fetch the whale's positions across all markets in the same event."""
if not wallet_address or not event_slug:
return []
try:
response = await self._client.get(
f"{self.data_api_url}/positions",
params={"user": wallet_address},
)
response.raise_for_status()
all_positions = response.json()
if not all_positions:
return []
result = []
for pos in all_positions:
pos_event_slug = pos.get("eventSlug", "")
pos_condition_id = pos.get("conditionId", "")
if pos_event_slug != event_slug:
continue
if pos_condition_id == current_condition_id:
continue
size = float(pos.get("size", 0) or 0)
if size == 0:
continue
outcome = pos.get("outcome", "Yes")
avg_price = float(pos.get("avgPrice", 0) or 0)
cur_price = float(pos.get("curPrice", 0) or 0)
current_value = float(pos.get("currentValue", 0) or 0)
initial_value = float(pos.get("initialValue", 0) or 0)
cash_pnl = float(pos.get("cashPnl", 0) or 0)
title = pos.get("title", "")
if outcome == "Yes":
side_summary = f"Holding Yes {size:,.0f} tokens @ avg {avg_price:.2%}, current {cur_price:.2%}"
else:
side_summary = f"Holding No {size:,.0f} tokens @ avg {avg_price:.2%}, current {cur_price:.2%}"
result.append(EventPosition(
market_question=title,
condition_id=pos_condition_id,
outcome=outcome,
size=size,
avg_price=avg_price,
current_price=cur_price,
current_value=current_value,
initial_value=initial_value,
pnl=cash_pnl,
side_summary=side_summary,
))
result.sort(key=lambda x: x.current_value, reverse=True)
logger.debug(
f"Found {len(result)} event positions for {wallet_address} "
f"in event '{event_slug}'"
)
return result
except Exception as e:
logger.warning(f"Error fetching whale event positions: {e}")
return []
async def fetch_market_top_traders(
self, market_id: str, condition_id: str = "",
outcome_prices: Optional[List[float]] = None, top_n: int = 5,
) -> tuple[List[MarketTopTrader], List[MarketTopTrader]]:
"""Fetch top holders (bulls and bears) for a market."""
if not condition_id:
return [], []
yes_price = outcome_prices[0] if outcome_prices and len(outcome_prices) > 0 else 0.5
no_price = outcome_prices[1] if outcome_prices and len(outcome_prices) > 1 else 0.5
try:
response = await self._client.get(
f"{self.data_api_url}/holders",
params={"market": condition_id, "limit": top_n},
)
response.raise_for_status()
data = response.json()
if not data:
return [], []
top_buyers = []
top_sellers = []
for token_group in data:
holders = token_group.get("holders", [])
if not holders:
continue
outcome_index = holders[0].get("outcomeIndex", 0)
token_price = yes_price if outcome_index == 0 else no_price
for h in holders[:top_n]:
wallet = h.get("proxyWallet", "")
name = h.get("name") or h.get("pseudonym") or None
amount = float(h.get("amount", 0) or 0)
usd_value = amount * token_price
trader = MarketTopTrader(
wallet=wallet,
name=name,
net_volume_usd=usd_value,
trade_count=0,
)
if outcome_index == 0:
top_buyers.append(trader)
else:
top_sellers.append(trader)
# Fetch rankings in parallel
ranking_tasks = []
trader_refs = []
for t in top_buyers + top_sellers:
ranking_tasks.append(self.fetch_trader_ranking(t.wallet))
trader_refs.append(t)
if ranking_tasks:
rankings = await asyncio.gather(*ranking_tasks, return_exceptions=True)
for trader, ranking in zip(trader_refs, rankings):
if isinstance(ranking, TraderRanking) and ranking:
trader.rank = ranking.rank
trader.pnl = ranking.pnl
if ranking.user_name:
trader.name = ranking.user_name
logger.debug(
f"Market {market_id}: {len(top_buyers)} top Yes holders, "
f"{len(top_sellers)} top No holders"
)
return top_buyers, top_sellers
except Exception as e:
logger.warning(f"Error fetching top holders for {market_id}: {e}")
return [], []
# ================================================================
# Main run loop
# ================================================================
async def run(self):
"""
Start the RTDS-based monitoring loop.
Architecture:
- Single RTDS WebSocket receives ALL trades in real-time
- _on_rtds_trade filters and handles each trade
- Periodic persistence of processed transactions
"""
self._running = True
# Create RTDS client with our trade handler
proxy = self.settings.http_proxy.strip() or None
self._rtds = RTDSClient(on_trade=self._on_rtds_trade, proxy=proxy)
logger.info(
f"Starting RTDS trade monitor "
f"({len(self._monitored_markets)} monitored markets)"
)
# Start warmup timer — suppress alerts for first N seconds
# to avoid firing on trades already in the RTDS pipeline
async def warmup_timer():
await asyncio.sleep(self._warmup_seconds)
self._warmup_complete = True
logger.info(
f"Warmup complete ({self._warmup_seconds}s). "
f"Now alerting on new whale trades."
)
warmup_task = asyncio.create_task(warmup_timer())
# Periodic persistence task
async def persistence_loop():
while self._running:
await asyncio.sleep(60)
self._save_processed_txns()
# Trim processed txns set to prevent unbounded growth
if len(self._processed_txns) > 100_000:
# Keep only the most recent 50K (approximate — set is unordered,
# but old hashes won't repeat so trimming is safe)
excess = len(self._processed_txns) - 50_000
for _ in range(excess):
self._processed_txns.pop()
logger.info(f"Trimmed processed txns to {len(self._processed_txns)}")
persistence_task = asyncio.create_task(persistence_loop())
try:
# Run RTDS client (blocks until stop)
await self._rtds.run()
finally:
warmup_task.cancel()
persistence_task.cancel()
self._save_processed_txns()
def stop(self):
"""Stop the monitoring loop."""
self._running = False
if self._rtds:
self._rtds.stop()
logger.info("Trade monitor stopping...")
def clear_processed_transactions(self):
"""Clear the processed transactions cache."""
count = len(self._processed_txns)
self._processed_txns.clear()
logger.info(f"Cleared {count} processed transactions from cache")
src/services/trader_profiler.py
+81
View File
@@ -0,0 +1,81 @@
"""Trader profiler service - generates structured trader profiles for LLM consumption."""
import json
import logging
from typing import Optional
from src.models.trade import TraderRanking, TraderHistory
logger = logging.getLogger(__name__)
class TraderProfiler:
"""
Generates structured trader profiles for LLM consumption.
Only organizes raw data into a clean JSON structure.
All interpretation and judgment is left to the LLM.
"""
def generate_profile(
self,
wallet_address: str,
ranking: Optional[TraderRanking],
history: Optional[TraderHistory],
) -> dict:
"""
Generate a structured trader profile from raw data.
Returns:
dict with raw trader data for LLM consumption
"""
# Ranking - raw numbers only
ranking_data = {
"rank": ranking.rank if ranking else None,
"pnl": ranking.pnl if ranking else None,
"total_volume": ranking.volume if ranking else None,
"verified": ranking.verified if ranking else False,
"username": ranking.user_name if ranking else None,
}
# Trading behavior - raw numbers only
large_trade_ratio = 0.0
if history and history.total_trades > 0:
large_trade_ratio = history.large_trades_count / history.total_trades
behavior_data = {
"total_trades": history.total_trades if history else 0,
"total_volume": history.total_volume if history else 0.0,
"avg_trade_size": history.avg_trade_size if history else 0.0,
"large_trades_count": history.large_trades_count if history else 0,
"large_trade_ratio": round(large_trade_ratio, 3),
"active_markets": history.recent_markets[:5] if history and history.recent_markets else [],
}
# Recent trades - raw data
recent_trades = []
if history and history.recent_trades:
for t in history.recent_trades[:10]:
recent_trades.append({
"side": t.get("side", ""),
"size_usd": t.get("usdc_size", 0),
"price": t.get("price", 0),
"market": t.get("title", "")[:50],
})
return {
"ranking": ranking_data,
"behavior": behavior_data,
"recent_trades": recent_trades,
}
def format_profile_for_llm(self, profile: dict) -> str:
"""Format the profile dict as JSON for LLM input."""
profile_json = json.dumps(profile, ensure_ascii=False, indent=2)
return f"""
### Trader Profile
```json
{profile_json}
```
"""
src/services/twitter_search.py
+220
View File
@@ -0,0 +1,220 @@
"""Twitter search service for whale trade verification."""
import os
import logging
from typing import Optional, Literal
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
logger = logging.getLogger(__name__)
# ---------- Retry-enabled HTTP GET ----------
_shared_session: Optional[requests.Session] = None
def robust_get(url: str, **kwargs) -> requests.Response:
"""GET request with automatic retry (3 retries, exponential backoff)."""
global _shared_session
if _shared_session is None:
_shared_session = requests.Session()
retry = Retry(
total=3,
backoff_factor=0.5,
status_forcelist=(429, 500, 502, 503, 504),
allowed_methods=["GET"],
raise_on_status=False,
)
adapter = HTTPAdapter(max_retries=retry)
_shared_session.mount("http://", adapter)
_shared_session.mount("https://", adapter)
kwargs.setdefault("timeout", 15)
return _shared_session.get(url, **kwargs)
# Twitter module constants and helpers (extracted to avoid langchain @tool decorator issues)
TWITTER_API_KEY = os.getenv("TWITTER_API_KEY", "NONE")
BASE_URL = "https://api.twitterapi.io/twitter"
SEARCH_ENDPOINT = f"{BASE_URL}/tweet/advanced_search"
USER_TWEETS_ENDPOINT = f"{BASE_URL}/user/last_tweets"
def _parse_tweet_text(tweet_data: dict) -> Optional[dict]:
"""Parse and format a single tweet."""
try:
# API returns author (not user), with userName (not username)
author = tweet_data.get("author") or tweet_data.get("user") or {}
user = author.get("userName") or author.get("username") or "unknown"
text = tweet_data.get("text", "")
likes = tweet_data.get("likeCount") or tweet_data.get("favorite_count") or 0
retweets = tweet_data.get("retweetCount") or tweet_data.get("retweet_count") or 0
created_at = tweet_data.get("createdAt") or tweet_data.get("created_at") or ""
engagement = int(likes) + int(retweets)
return {
"user": user,
"text": text,
"engagement": engagement,
"time": created_at,
}
except Exception:
return None
def _format_tweets_report(title: str, parsed_tweets: list[dict], total_engagement: int) -> str:
"""Format tweets list as a report."""
report = [f"--- {title} ---"]
report.append(f"Total Engagement in Sample: {total_engagement} (Likes+RTs)")
report.append("Top Discussions:")
for idx, item in enumerate(parsed_tweets):
text_preview = item["text"][:200]
if len(item["text"]) > 200:
text_preview += "..."
report.append(f'{idx + 1}. @{item["user"]} (🔥{item["engagement"]}): "{text_preview}"')
report.append("-------------------------------------------")
return "\n".join(report)
class TwitterSearchService:
"""
Twitter search service for whale trade verification.
Uses Twitter API for social sentiment search.
"""
def __init__(self, api_key: Optional[str] = None):
"""
Initialize Twitter search service.
Args:
api_key: Twitter API key. If not provided, reads from TWITTER_API_KEY env var.
"""
self.api_key = api_key or os.getenv("TWITTER_API_KEY", "")
if not self.api_key:
logger.warning("TWITTER_API_KEY not set. Twitter search will be disabled.")
def _get_headers(self) -> dict[str, str]:
"""Get request headers with API key."""
return {"X-API-Key": self.api_key}
def is_available(self) -> bool:
"""Check if Twitter search is available (API key is set)."""
return bool(self.api_key and self.api_key.strip().upper() != "NONE")
def search_tweets(
self,
query: str,
search_mode: Literal["top", "latest"] = "top",
limit: int = 10,
) -> str:
"""
Search Twitter for tweets matching a query.
Args:
query: Search query (e.g., "Trump", "Bitcoin", "Fed rate")
search_mode: "top" for most relevant, "latest" for most recent
limit: Number of tweets to return (1-20)
Returns:
Formatted report of tweets with engagement metrics.
"""
if not self.is_available():
return "Twitter search unavailable: TWITTER_API_KEY not configured."
limit = max(1, min(limit, 20))
query_type = "Top" if search_mode == "top" else "Latest"
params = {
"query": query,
"queryType": query_type,
"limit": limit,
}
try:
response = robust_get(
SEARCH_ENDPOINT,
params=params,
headers=self._get_headers(),
timeout=15,
)
if response.status_code != 200:
logger.error(f"Twitter API error: {response.status_code} - {response.text[:200]}")
return f"Twitter API Error: {response.status_code}"
data = response.json()
tweets_raw = data.get("tweets", [])
if not tweets_raw:
return f"No recent tweets found for '{query}'."
# Parse tweets
parsed_tweets = []
total_engagement = 0
for t in tweets_raw:
p = _parse_tweet_text(t)
if p:
parsed_tweets.append(p)
total_engagement += p["engagement"]
mode_label = "Hot" if search_mode == "top" else "Latest"
return _format_tweets_report(
f"Twitter Search Results for '{query}' [{mode_label}]",
parsed_tweets,
total_engagement,
)
except Exception as e:
logger.error(f"Twitter search failed: {e}")
return f"Twitter search failed: {str(e)}"
def search_for_market(
self,
market_question: str,
limit: int = 10,
) -> str:
"""
Search Twitter for information relevant to a prediction market.
Combines both TOP (importance/engagement) and LATEST (timeliness) results
to balance relevance and recency.
Args:
market_question: The market question to search for
limit: Number of tweets per search mode (will search both top and latest)
Returns:
Combined search results from both search modes.
"""
if not self.is_available():
return "Twitter search unavailable: TWITTER_API_KEY not configured."
results = []
query = market_question[:100] # Limit query length for API
# 1. Search TOP tweets - high engagement, represents importance
top_result = self.search_tweets(query, search_mode="top", limit=limit)
if "No recent tweets" not in top_result and "Error" not in top_result:
results.append("## Hot Tweets (High Engagement / Importance)\n" + top_result)
# 2. Search LATEST tweets - real-time info, represents timeliness
latest_result = self.search_tweets(query, search_mode="latest", limit=limit)
if "No recent tweets" not in latest_result and "Error" not in latest_result:
results.append("## Latest Tweets (Real-Time / Timeliness)\n" + latest_result)
if not results:
return f"No relevant tweets found for: {market_question[:50]}..."
return "\n\n".join(results)
# Singleton instance
_twitter_service: Optional[TwitterSearchService] = None
def get_twitter_service() -> TwitterSearchService:
"""Get the singleton Twitter search service instance."""
global _twitter_service
if _twitter_service is None:
_twitter_service = TwitterSearchService()
return _twitter_service
src/services/volatility_analyzer.py
+372
View File
@@ -0,0 +1,372 @@
"""Volatility analyzer service - analyzes price volatility using AI to detect leading signals."""
import json
import logging
import re
from datetime import datetime
from pathlib import Path
from typing import Optional
from openai import OpenAI
from src.config import get_settings
from src.models.leading_signal import LeadingSignal, SignalType
from src.services.price_monitor import VolatilityAlert
from src.services.twitter_search import TwitterSearchService
from src.prompts.volatility_analyzer import VolatilityAnalyzerPrompts
logger = logging.getLogger(__name__)
# Directory for storing leading signals dataset
LEADING_SIGNALS_DIR = Path(__file__).parent.parent.parent / "leading_signals"
class VolatilityAnalyzer:
"""
Analyzes price volatility events using LLM to detect "price leads news" signals.
Uses Tavily web search and Twitter to verify whether a price movement
preceded public news, building a dataset of leading signals.
"""
def __init__(self):
self.settings = get_settings()
# Configure OpenAI-compatible API client
self.client = OpenAI(
base_url=self.settings.llm_base_url,
api_key=self.settings.llm_api_key,
)
self.prompts = VolatilityAnalyzerPrompts()
self.twitter_search = TwitterSearchService(api_key=self.settings.twitter_api_key)
from src.services.web_search import WebSearchService
self.web_search = WebSearchService(
tavily_api_key=self.settings.tavily_api_key,
serper_api_key=self.settings.serper_api_key,
)
# Ensure storage directory exists
LEADING_SIGNALS_DIR.mkdir(parents=True, exist_ok=True)
def _extract_json_from_response(self, response: str) -> Optional[dict]:
"""
Extract JSON from LLM response.
Args:
response: The LLM response text
Returns:
Parsed JSON dict or None
"""
# Try to find JSON in code blocks
json_pattern = r"```(?:json)?\s*([\s\S]*?)```"
matches = re.findall(json_pattern, response)
for match in matches:
try:
return json.loads(match.strip())
except json.JSONDecodeError:
continue
# Try to find raw JSON
try:
start = response.find("{")
end = response.rfind("}") + 1
if start >= 0 and end > start:
return json.loads(response[start:end])
except json.JSONDecodeError:
pass
return None
def _parse_signal_type(self, type_str: str) -> SignalType:
"""Parse signal type string to enum."""
try:
return SignalType(type_str.upper())
except ValueError:
return SignalType.SPECULATION
def _store_leading_signal(self, signal: LeadingSignal) -> str:
"""
Store a leading signal to the dataset.
Args:
signal: The leading signal to store
Returns:
Path to the stored file
"""
# Create filename with timestamp and market info
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
market_slug = re.sub(r'[^\w\s-]', '', signal.market_question)[:40]
market_slug = re.sub(r'\s+', '_', market_slug)
filename = f"{timestamp}_{signal.signal_type.value}_{market_slug}.json"
filepath = LEADING_SIGNALS_DIR / filename
with open(filepath, 'w', encoding='utf-8') as f:
json.dump(signal.to_dict(), f, ensure_ascii=False, indent=2)
return str(filepath)
def _store_all_signals_index(self, signal: LeadingSignal) -> None:
"""
Append signal to the master index file for easy querying.
Args:
signal: The signal to append
"""
index_file = LEADING_SIGNALS_DIR / "signals_index.jsonl"
with open(index_file, 'a', encoding='utf-8') as f:
f.write(json.dumps(signal.to_dict(), ensure_ascii=False) + "\n")
async def analyze_volatility(self, alert: VolatilityAlert) -> Optional[LeadingSignal]:
"""
Analyze a price volatility event to determine if it's a leading signal.
Args:
alert: The volatility alert to analyze
Returns:
LeadingSignal if analysis successful, None otherwise
"""
logger.info(
f"Analyzing volatility: {alert.market_question[:50]}... "
f"{alert.direction} {abs(alert.price_change_percent):.1%}"
)
# Search web (Tavily) for news verification
web_search_context = ""
if self.web_search.is_available():
logger.info(f"Searching web for: {alert.market_question[:50]}...")
web_result = self.web_search.search_for_market(
market_question=alert.market_question,
max_results=5,
)
if web_result and "unavailable" not in web_result.lower():
web_search_context = web_result
logger.info("Web search (Tavily) completed")
# Search Twitter for social sentiment
twitter_context = ""
if self.twitter_search.is_available():
logger.info(f"Searching Twitter for: {alert.market_question[:50]}...")
twitter_result = self.twitter_search.search_for_market(
market_question=alert.market_question,
limit=10,
)
if twitter_result and "unavailable" not in twitter_result.lower():
twitter_context = twitter_result
logger.info("Twitter search completed")
# Build prompts
system_prompt = self.prompts.system_prompt()
user_prompt = self.prompts.analyze_volatility(
market_question=alert.market_question,
price_change_percent=alert.price_change_percent,
direction=alert.direction,
start_price=alert.start_price,
end_price=alert.end_price,
window_seconds=alert.window_seconds,
detected_at=alert.detected_at,
twitter_context=twitter_context,
web_search_context=web_search_context,
)
try:
# Call LLM API
response = self.client.chat.completions.create(
model=self.settings.llm_model,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt},
],
)
analysis_text = response.choices[0].message.content
logger.debug(f"LLM response: {analysis_text[:500]}...")
# Extract JSON from response
json_data = self._extract_json_from_response(analysis_text)
if not json_data:
logger.warning("Could not parse LLM response as JSON")
return None
# Create LeadingSignal from response
signal_id = f"vol_{alert.market_id}_{int(datetime.now().timestamp())}"
signal = LeadingSignal(
id=signal_id,
market_id=alert.market_id,
market_question=alert.market_question,
price_change_percent=alert.price_change_percent,
direction=alert.direction,
start_price=alert.start_price,
end_price=alert.end_price,
window_seconds=alert.window_seconds,
detected_at=datetime.utcnow().isoformat(),
volatility_detected_at=alert.detected_at,
signal_type=self._parse_signal_type(json_data.get("signal_type", "SPECULATION")),
confidence=float(json_data.get("confidence", 0.0)),
is_leading_signal=bool(json_data.get("is_leading_signal", False)),
news_found=bool(json_data.get("news_found", False)),
earliest_news_time=json_data.get("earliest_news_time"),
key_news_headlines=json_data.get("key_news_headlines", []),
earliest_social_time=json_data.get("earliest_social_time"),
key_social_posts=json_data.get("key_social_posts", []),
time_advantage_minutes=int(json_data.get("time_advantage_minutes", 0)),
reasoning=str(json_data.get("reasoning", "")),
potential_information_source=str(json_data.get("potential_information_source", "")),
full_analysis=analysis_text,
)
# Store the signal
filepath = self._store_leading_signal(signal)
self._store_all_signals_index(signal)
# Log result
if signal.is_leading_signal:
logger.warning(
f"🚨 LEADING SIGNAL DETECTED: {alert.market_question[:50]}... "
f"Time advantage: {signal.time_advantage_minutes} minutes"
)
else:
logger.info(
f"Volatility analyzed: {signal.signal_type.value} "
f"(confidence: {signal.confidence:.1%})"
)
logger.info(f"Signal stored: {filepath}")
return signal
except Exception as e:
logger.error(f"Error analyzing volatility: {e}")
return None
def format_signal_report(self, signal: LeadingSignal) -> str:
"""
Format a leading signal as a readable report.
Args:
signal: The signal to format
Returns:
Formatted report string
"""
direction_label = "Up" if signal.direction == "UP" else "Down"
signal_type_label = {
SignalType.LEADING_SIGNAL: "Leading Signal (Price Preceded News)",
SignalType.NEWS_DRIVEN: "News-Driven",
SignalType.SOCIAL_DRIVEN: "Social-Driven",
SignalType.SPECULATION: "Speculative Volatility",
}
news_headlines = "\n".join([f" - {h}" for h in signal.key_news_headlines]) or " None"
social_posts = "\n".join([f" - {p}" for p in signal.key_social_posts]) or " None"
report = f"""
{'='*70}
# Price Volatility Analysis Report
{'='*70}
**Analysis Time**: {signal.detected_at}
## Volatility Details
| Field | Details |
|-------|---------|
| **Market** | {signal.market_question} |
| **Price Change** | {direction_label} {abs(signal.price_change_percent):.1%} |
| **Start Price** | {signal.start_price:.2%} |
| **End Price** | {signal.end_price:.2%} |
| **Time Window** | {signal.window_seconds // 60} min |
{'='*70}
## Analysis Results
{'='*70}
| Field | Result |
|-------|--------|
| **Signal Type** | {signal_type_label.get(signal.signal_type, 'Unknown')} |
| **Confidence** | {signal.confidence:.1%} |
| **Is Leading Signal** | {'Yes' if signal.is_leading_signal else 'No'} |
| **Time Advantage** | {signal.time_advantage_minutes} min |
**Earliest News Time**: {signal.earliest_news_time or 'N/A'}
**Earliest Social Time**: {signal.earliest_social_time or 'N/A'}
## Key News
{news_headlines}
## Key Social Posts
{social_posts}
## Reasoning
{signal.reasoning}
## Suspected Information Source
{signal.potential_information_source or 'Unknown'}
{'='*70}
{signal.full_analysis}
{'='*70}
"""
return report
def get_leading_signals_stats(self) -> dict:
"""
Get statistics about collected leading signals.
Returns:
Dictionary with stats
"""
index_file = LEADING_SIGNALS_DIR / "signals_index.jsonl"
if not index_file.exists():
return {
"total_signals": 0,
"leading_signals": 0,
"news_driven": 0,
"social_driven": 0,
"speculation": 0,
}
stats = {
"total_signals": 0,
"leading_signals": 0,
"news_driven": 0,
"social_driven": 0,
"speculation": 0,
"avg_time_advantage_minutes": 0,
}
time_advantages = []
try:
with open(index_file, 'r', encoding='utf-8') as f:
for line in f:
if line.strip():
data = json.loads(line)
stats["total_signals"] += 1
signal_type = data.get("signal_type", "SPECULATION")
if signal_type == "LEADING_SIGNAL":
stats["leading_signals"] += 1
time_advantages.append(data.get("time_advantage_minutes", 0))
elif signal_type == "NEWS_DRIVEN":
stats["news_driven"] += 1
elif signal_type == "SOCIAL_DRIVEN":
stats["social_driven"] += 1
else:
stats["speculation"] += 1
if time_advantages:
stats["avg_time_advantage_minutes"] = sum(time_advantages) / len(time_advantages)
except Exception as e:
logger.error(f"Error reading signals index: {e}")
return stats
src/services/web_search.py
+79
View File
@@ -0,0 +1,79 @@
"""Unified web search with fallback: Tavily -> Serper -> DuckDuckGo."""
import logging
from typing import Optional
from src.services.tavily_search import TavilySearchService
from src.services.serper_search import SerperSearchService
from src.services.ddg_search import DDGSearchService
logger = logging.getLogger(__name__)
# Responses that indicate a search engine failed (not just "no results")
_FAILURE_KEYWORDS = ("API Error", "search failed", "unavailable", "exceeds your plan")
def _is_failure(result: str) -> bool:
return any(kw in result for kw in _FAILURE_KEYWORDS)
class WebSearchService:
"""
Unified web search with automatic fallback.
Priority: Tavily (best quality) -> Serper -> DuckDuckGo (free).
Falls back to the next engine when the current one errors.
"""
def __init__(
self,
tavily_api_key: str = "",
serper_api_key: str = "",
):
self._engines = []
tavily = TavilySearchService(api_key=tavily_api_key)
if tavily.is_available():
self._engines.append(("Tavily", tavily))
serper = SerperSearchService(api_key=serper_api_key)
if serper.is_available():
self._engines.append(("Serper", serper))
ddg = DDGSearchService()
if ddg.is_available():
self._engines.append(("DuckDuckGo", ddg))
if self._engines:
names = [name for name, _ in self._engines]
logger.info(f"Web search engines: {' -> '.join(names)}")
else:
logger.warning("No web search engine available.")
def is_available(self) -> bool:
return len(self._engines) > 0
def search(self, query: str, max_results: int = 5) -> str:
if not self._engines:
return "Web search unavailable: no search engine configured."
for name, engine in self._engines:
result = engine.search(query, max_results=max_results)
if _is_failure(result):
logger.warning(f"{name} search failed, trying next engine...")
continue
return result
return f"All web search engines failed for: '{query}'"
def search_for_market(self, market_question: str, max_results: int = 5) -> str:
if not self._engines:
return "Web search unavailable: no search engine configured."
for name, engine in self._engines:
result = engine.search_for_market(market_question, max_results=max_results)
if _is_failure(result):
logger.warning(f"{name} market search failed, trying next engine...")
continue
return result
return f"No relevant web results found for: {market_question[:50]}..."
src/utils/__init__.py
+4
View File
@@ -0,0 +1,4 @@
"""Utilities module."""
from .logger import setup_logging, get_logger
__all__ = ["setup_logging", "get_logger"]
src/utils/http.py
+32
View File
@@ -0,0 +1,32 @@
"""Shared HTTP client factory with proxy support.
All services that need httpx clients should import from here instead of
creating httpx.Client/AsyncClient directly. This ensures proxy settings
are applied consistently across the entire application.
"""
import httpx
from src.config.settings import get_settings
def get_proxy_config() -> str | None:
"""Return proxy URL from settings, or None if not configured."""
settings = get_settings()
proxy = settings.http_proxy.strip()
return proxy if proxy else None
def get_client(**kwargs) -> httpx.Client:
"""Create a synchronous httpx.Client with proxy support."""
proxy = get_proxy_config()
if proxy:
kwargs.setdefault("proxy", proxy)
kwargs.setdefault("timeout", 30.0)
return httpx.Client(**kwargs)
def get_async_client(**kwargs) -> httpx.AsyncClient:
"""Create an asynchronous httpx.AsyncClient with proxy support."""
proxy = get_proxy_config()
if proxy:
kwargs.setdefault("proxy", proxy)
return httpx.AsyncClient(**kwargs)
src/utils/logger.py
+120
View File
@@ -0,0 +1,120 @@
"""Logging utilities."""
import logging
import sys
from datetime import datetime
from typing import Optional
from rich.console import Console
from rich.logging import RichHandler
from src.config import get_settings
def setup_logging(level: Optional[str] = None) -> None:
"""
Set up application logging with rich formatting.
Args:
level: Log level (DEBUG, INFO, WARNING, ERROR). Defaults to settings.
"""
settings = get_settings()
log_level = level or settings.log_level
# Create rich console
console = Console()
# Configure root logger
logging.basicConfig(
level=log_level,
format="%(message)s",
datefmt="[%X]",
handlers=[
RichHandler(
console=console,
rich_tracebacks=True,
show_path=False,
)
],
)
# Reduce noise from third-party libraries
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("web3").setLevel(logging.WARNING)
def get_logger(name: str) -> logging.Logger:
"""
Get a logger instance.
Args:
name: Logger name (usually __name__)
Returns:
Logger instance
"""
return logging.getLogger(name)
class WhaleWatcherLogger:
"""Custom logger for whale watcher with formatted output."""
def __init__(self):
self.console = Console()
self.logger = logging.getLogger("whale_watcher")
def whale_detected(
self,
amount: float,
side: str,
price: float,
market: str,
) -> None:
"""Log a whale trade detection."""
self.console.print(
f"\n[bold cyan]{'='*60}[/bold cyan]\n"
f"[bold yellow]🐋 WHALE TRADE DETECTED![/bold yellow]\n"
f"[bold cyan]{'='*60}[/bold cyan]\n"
f"[green]Amount:[/green] ${amount:,.2f} USDC\n"
f"[green]Side:[/green] {side}\n"
f"[green]Price:[/green] {price:.4f}\n"
f"[green]Market:[/green] {market}\n"
f"[green]Time:[/green] {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n"
f"[bold cyan]{'='*60}[/bold cyan]\n"
)
def report_generated(self, market: str) -> None:
"""Log that a report was generated."""
self.console.print(
f"\n[bold magenta]{'='*60}[/bold magenta]\n"
f"[bold magenta]📊 ANALYSIS REPORT GENERATED[/bold magenta]\n"
f"[bold magenta]{'='*60}[/bold magenta]\n"
f"[green]Market:[/green] {market[:50]}...\n"
f"[bold magenta]{'='*60}[/bold magenta]\n"
)
def monitoring_started(self, market_count: int, interval: int = 0, min_trade_size: float = 1000, min_price: float = 0.2, max_price: float = 0.8) -> None:
"""Log monitoring start."""
self.console.print(
f"\n[bold green]{'='*60}[/bold green]\n"
f"[bold green]🚀 WHALE WATCHER STARTED (RTDS WebSocket)[/bold green]\n"
f"[bold green]{'='*60}[/bold green]\n"
f"[green]Monitored Markets:[/green] {market_count}\n"
f"[green]Mode:[/green] Real-time WebSocket (zero missed trades)\n"
f"[green]Min Trade Size:[/green] ${min_trade_size:,.0f} USD\n"
f"[green]Price Range:[/green] {min_price} - {max_price}\n"
f"[bold green]{'='*60}[/bold green]\n"
)
def error(self, message: str) -> None:
"""Log an error."""
self.console.print(f"[bold red]❌ ERROR:[/bold red] {message}")
def info(self, message: str) -> None:
"""Log an info message."""
self.console.print(f"[blue]ℹ️[/blue] {message}")
def separator(self) -> None:
"""Print a separator line."""
self.console.print(f"[dim]{'─'*60}[/dim]")