diff --git a/README.md b/README.md index 78bed6d..a716abb 100644 --- a/README.md +++ b/README.md @@ -2,80 +2,103 @@ **Detect informed money before the market moves.** -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![CI](https://github.com/pselamy/polymarket-insider-tracker/actions/workflows/ci.yml/badge.svg)](https://github.com/pselamy/polymarket-insider-tracker/actions/workflows/ci.yml) [![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) + +Real-time detection of suspicious trading patterns on Polymarket: fresh wallets, unusual sizing, niche-market activity, and funding chain analysis. Streams trades via WebSocket, profiles wallets on-chain (Polygon), scores risk with ML + heuristics, and dispatches alerts to Discord/Telegram. --- -## The Opportunity +## Quick Start (< 2 minutes) -On January 3, 2026, a trader spotted a significant political event on Polymarket **before it happened**. How? Not by predicting the future, but by tracking suspicious trading behavior. +### 1. Install -> "You don't need to predict the future, you need to track suspicious behavior." -> — [@DidiTrading](https://x.com/DidiTrading) - -An insider wallet turned **$35,000 into $442,000** (12.6x return) by entering a position hours before a major market move. The tool that detected this activity flagged five separate alerts before the event occurred. - -**This repository builds that tool.** - ---- - -## What This Does - -The Polymarket Insider Tracker monitors prediction market trading activity in real-time and identifies patterns that suggest informed trading: - -| Signal | What It Detects | Why It Matters | -|--------|-----------------|----------------| -| **Fresh Wallets** | Brand new wallets making large trades | Insiders create new wallets to hide their identity | -| **Unusual Sizing** | Trades that are disproportionately large for the market | Informed traders bet bigger when they have edge | -| **Niche Markets** | Activity in low-volume, specific-outcome markets | Easier to have inside information on obscure events | -| **Funding Chains** | Where wallet funds originated from | Links seemingly separate wallets to the same entity | - -When suspicious activity is detected, you receive an instant alert with actionable intelligence. - ---- - -## How It Works - -``` -┌─────────────────┐ ┌──────────────────┐ ┌────────────────────┐ -│ Polymarket API │────>│ Wallet Profiler │────>│ Anomaly Detector │ -│ (Real-time) │ │ (Blockchain) │ │ (ML + Heuristics) │ -└─────────────────┘ └──────────────────┘ └────────────────────┘ - │ - ┌────────────────────────────┘ - v - ┌─────────────────────┐ - │ Alert Dispatcher │───> Discord / Telegram / Email - │ "Fresh wallet │ - │ buying YES @7.5¢ │ - │ on niche market" │ - └─────────────────────┘ +```bash +# Requires: Python 3.11+, Docker +git clone https://github.com/pselamy/polymarket-insider-tracker.git +cd polymarket-insider-tracker +uv sync --all-extras # or: pip install -e ".[dev]" ``` -### Detection Algorithms +### 2. Start infrastructure -1. **Fresh Wallet Detection** - - Checks wallet transaction history on Polygon - - Flags wallets with fewer than 5 lifetime transactions making trades over $1,000 - - Traces funding source to identify if connected to known entities +```bash +docker compose up -d # PostgreSQL 15 + Redis 7 +docker compose ps # wait for healthy +``` -2. **Liquidity Impact Analysis** - - Calculates trade size relative to market depth - - Flags trades consuming more than 2% of visible order book - - Weights by market category (niche markets score higher) +### 3. Configure -3. **Sniper Cluster Detection** - - Uses DBSCAN clustering to find wallets that consistently enter markets within minutes of creation - - Identifies coordinated behavior patterns +```bash +cp .env.example .env +# Edit .env — only DATABASE_URL and REDIS_URL are required for local dev +# (defaults in .env.example work with docker compose) +``` -4. **Event Correlation** - - Cross-references trading activity with news feeds - - Detects positions opened 1-4 hours before related news breaks +### 4. Run migrations + start + +```bash +uv run alembic upgrade head +uv run python -m polymarket_insider_tracker +``` + +You should see live trades within seconds: + +``` +INFO Connection state: disconnected -> connecting +INFO Connected to wss://ws-live-data.polymarket.com and subscribed to trades +DEBUG Trade: BUY 450 @ 1.00 on fifwc-ger-kor-2026-06-14-ger +DEBUG Trade: SELL 5 @ 0.86 on chi1-cd1-cdl-2026-06-14-draw +``` + +### CLI Options + +```bash +python -m polymarket_insider_tracker --help + --version Show version + --config-check Validate configuration and exit + --log-level DEBUG Override log level + --dry-run Run pipeline without sending alerts + --health-port 8080 Override health check port +``` --- -## Sample Alert +## Environment Variables + +| Variable | Required | Default | Description | +|----------|----------|---------|-------------| +| `DATABASE_URL` | Yes | — | PostgreSQL connection string | +| `REDIS_URL` | No | `redis://localhost:6379` | Redis connection string | +| `POLYGON_RPC_URL` | No | `https://polygon-rpc.com` | Polygon RPC (public default works) | +| `POLYGON_FALLBACK_RPC_URL` | No | — | Fallback RPC endpoint | +| `POLYMARKET_WS_URL` | No | `wss://ws-live-data.polymarket.com` | WebSocket endpoint | +| `POLYMARKET_API_KEY` | No | — | Optional API key for higher rate limits | +| `DISCORD_WEBHOOK_URL` | No | — | Discord alerts | +| `TELEGRAM_BOT_TOKEN` | No | — | Telegram alerts (needs `TELEGRAM_CHAT_ID` too) | +| `TELEGRAM_CHAT_ID` | No | — | Telegram chat for alerts | +| `LOG_LEVEL` | No | `INFO` | Logging level | +| `DRY_RUN` | No | `false` | Skip sending alerts | +| `HEALTH_PORT` | No | `8080` | Health check HTTP port | + +No API keys are needed for basic operation — the Polymarket WebSocket and CLOB REST APIs are public. + +--- + +## What It Detects + +| Signal | Detection Method | Threshold | +|--------|-----------------|-----------| +| **Fresh Wallets** | Wallet age < 48h, nonce <= 5, making trades > $1k | Confidence 0.5-0.9 | +| **Size Anomalies** | Trade size > 2% of 24h volume or > 5% of order book | Weighted by niche factor | +| **Niche Markets** | Low-volume markets (< $50k daily) with specific outcomes | 1.5x risk multiplier | +| **Funding Chains** | Trace wallet funding to known entities (exchanges, etc.) | On-chain lineage | +| **Sniper Clusters** | DBSCAN clustering of wallets entering within minutes | Coordinated behavior | + +Risk scoring combines signals with configurable weights (default threshold: 0.6). Multi-signal bonuses: 2 signals +20%, 3+ signals +30%. + +### Sample Alert ``` SUSPICIOUS ACTIVITY DETECTED @@ -99,199 +122,62 @@ Confidence: HIGH (3/4 signals triggered) --- -## Quick Start +## Architecture -### Prerequisites +``` +Polymarket WebSocket ──> Ingestor ──> Profiler ──> Detector ──> Alerter +(wss://ws-live-data) (trades) (on-chain) (scoring) (Discord/TG) + | + Polygon RPC +``` -- Python 3.11+ -- Docker and Docker Compose -- Polygon RPC endpoint (Alchemy, QuickNode, or self-hosted) -- Polymarket API key (free at [docs.polymarket.com](https://docs.polymarket.com)) +### Components -### Installation +| Module | Purpose | +|--------|---------| +| `ingestor/` | WebSocket trade stream + CLOB REST client with rate limiting | +| `profiler/` | Polygon wallet analysis, entity identification, funding chain tracing | +| `detector/` | Fresh wallet, size anomaly, sniper cluster detection, composite risk scorer | +| `alerter/` | Multi-channel dispatch (Discord webhooks, Telegram bot) with dedup | +| `storage/` | SQLAlchemy ORM + Alembic migrations (PostgreSQL) | +| `pipeline.py` | Orchestrator wiring all components together | +| `shutdown.py` | Graceful SIGTERM/SIGINT handling with cleanup callbacks | + +--- + +## Development ```bash -# Clone the repository -git clone https://github.com/pselamy/polymarket-insider-tracker.git -cd polymarket-insider-tracker - -# Copy environment template -cp .env.example .env -# Edit .env with your API keys - -# Start infrastructure (PostgreSQL, Redis) -docker compose up -d - -# Wait for services to be healthy -docker compose ps - -# Install Python dependencies -uv sync --all-extras - -# Run database migrations -uv run alembic upgrade head - -# Run the tracker -uv run python -m polymarket_insider_tracker +uv run pytest # run tests +uv run ruff check src/ tests/ # lint +uv run ruff format src/ tests/ # format +uv run mypy src/ # type check (strict mode) ``` ### Docker Services -The development stack includes: - | Service | Port | Description | |---------|------|-------------| | PostgreSQL 15 | 5432 | Primary database | | Redis 7 | 6379 | Caching and pub/sub | -| Adminer | 8080 | Database admin UI (optional) | -| RedisInsight | 5540 | Redis admin UI (optional) | - -```bash -# Start core services only -docker compose up -d - -# Start with development tools (Adminer, RedisInsight) -docker compose --profile tools up -d - -# View logs -docker compose logs -f - -# Stop all services -docker compose down - -# Stop and remove volumes (reset data) -docker compose down -v -``` - -### Configuration - -```bash -# .env file -POLYGON_RPC_URL=https://polygon-mainnet.g.alchemy.com/v2/YOUR_KEY -POLYMARKET_API_KEY=your_polymarket_api_key - -# Alert destinations (optional) -DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/... -TELEGRAM_BOT_TOKEN=your_bot_token -TELEGRAM_CHAT_ID=your_chat_id - -# Detection thresholds -MIN_TRADE_SIZE_USDC=1000 -FRESH_WALLET_MAX_NONCE=5 -LIQUIDITY_IMPACT_THRESHOLD=0.02 -``` +| Adminer | 8080 | Database admin UI (optional, `--profile tools`) | +| RedisInsight | 5540 | Redis admin UI (optional, `--profile tools`) | --- -## Project Structure +## Troubleshooting -``` -polymarket-insider-tracker/ -├── src/ -│ └── polymarket_insider_tracker/ -│ ├── __main__.py # CLI entry point -│ ├── pipeline.py # Core detection pipeline -│ ├── ingestor/ # Real-time market data ingestion -│ │ ├── clob_client.py # Polymarket CLOB API wrapper -│ │ └── websocket.py # WebSocket event handler -│ ├── profiler/ # Wallet analysis -│ ├── detector/ # Anomaly detection engines -│ ├── alerter/ # Notification dispatch -│ └── storage/ # Persistence layer -├── tests/ # Test suite -├── scripts/ -│ └── backtest.py # Historical analysis -├── docker-compose.yml -├── pyproject.toml -└── README.md -``` +**No trades received / silent connection** +The WebSocket subscription requires `action: "subscribe"` in the envelope. If you're on an older version, update — this was fixed in the WebSocket protocol alignment (see #89). ---- +**Connection timeout / DNS errors** +Verify `wss://ws-live-data.polymarket.com` is reachable from your network. Some corporate firewalls block WebSocket connections. -## Roadmap +**Database migration errors** +Ensure PostgreSQL is running (`docker compose ps`) and `DATABASE_URL` matches your docker-compose config. Run `uv run alembic upgrade head` after any schema changes. -### Phase 1: Core Detection (Current) -- [x] Project structure and documentation -- [ ] Polymarket CLOB API integration -- [ ] Fresh wallet detection -- [ ] Size anomaly detection -- [ ] Basic alerting (Discord/Telegram) - -### Phase 2: Advanced Intelligence -- [ ] Funding chain analysis -- [ ] Sniper cluster detection (DBSCAN) -- [ ] Market categorization (niche vs mainstream) -- [ ] Historical backtesting framework - -### Phase 3: Production Hardening -- [ ] High-availability deployment -- [ ] Rate limit management -- [ ] False positive feedback loop -- [ ] Web dashboard - ---- - -## Why This Matters - -Prediction markets are becoming a critical source of real-time probability estimates for world events. As they grow, so does the incentive for informed actors to exploit information asymmetry. - -This tool democratizes access to the same detection capabilities that sophisticated traders use. Whether you are: - -- **A trader** looking for alpha signals -- **A researcher** studying market microstructure -- **A platform operator** monitoring for manipulation - -...this tracker provides visibility into the hidden flows that move markets. - ---- - -## Technical Background - -### Polymarket Architecture - -Polymarket is a prediction market platform built on Polygon (Ethereum L2). Key characteristics: - -- **CLOB (Central Limit Order Book)**: Centralized matching engine for speed -- **On-chain Settlement**: Final trades settle on Polygon blockchain -- **USDC Collateral**: All positions denominated in USDC stablecoin -- **Binary Outcomes**: Shares priced between $0.00 and $1.00 - -### Data Sources - -| Source | Purpose | Latency | -|--------|---------|---------| -| Polymarket CLOB API | Real-time trades, orderbook | Milliseconds | -| Polygon RPC | Wallet history, nonce, funding | 1-2 seconds | -| Market Metadata API | Market categorization | On-demand | - -### Detection Challenges - -1. **Sybil Resistance**: Insiders use fresh wallets per trade -2. **Rate Limits**: Polygon RPC calls require caching strategy -3. **Market Classification**: NLP needed to categorize market niches -4. **Timing**: CLOB data leads on-chain by seconds - ---- - -## Contributing - -Contributions are welcome! Please read our Contributing Guide before submitting PRs. - -### Development Setup - -```bash -# Install dev dependencies -uv sync --all-extras - -# Run tests -uv run pytest - -# Run linting -uv run ruff check src/ - -# Run type checking -uv run mypy src/ -``` +**Rate limiting on Polygon RPC** +The default public RPC (`https://polygon-rpc.com`) has low rate limits. For production use, set `POLYGON_RPC_URL` to a dedicated provider (Alchemy, QuickNode, etc.). --- @@ -304,20 +190,6 @@ This software is provided for **educational and research purposes only**. - Insider trading is illegal in regulated markets; this tool is for transparency and research - Users are responsible for compliance with applicable laws ---- - ## License MIT License - see [LICENSE](LICENSE) for details. - ---- - -## Acknowledgments - -- Inspired by [@DidiTrading](https://x.com/DidiTrading) and [@spacexbt](https://x.com/spacexbt) -- Built on the open Polymarket API ecosystem -- Community contributions welcome - ---- - -**Questions?** Open an issue or start a discussion. diff --git a/docs/skill-tracking-prediction-market-flow.md b/docs/skill-tracking-prediction-market-flow.md new file mode 100644 index 0000000..efda1a4 --- /dev/null +++ b/docs/skill-tracking-prediction-market-flow.md @@ -0,0 +1,113 @@ +# Skill: tracking-prediction-market-flow + +Use when analyzing prediction market activity for informed-flow signals, insider +trading patterns, or suspicious wallet behavior on Polymarket. + +## What This Tool Does + +polymarket-insider-tracker streams real-time trades from Polymarket's WebSocket +feed, profiles trader wallets on the Polygon blockchain, and scores each trade +for informed-flow risk using multiple detection signals: + +- **Fresh wallet detection**: New wallets (age < 48h, nonce <= 5) making large + trades (> $1k). Insiders create disposable wallets per trade. +- **Size anomaly detection**: Trades consuming > 2% of 24h volume or > 5% of + visible order book depth. Informed traders bet bigger when they have edge. +- **Niche market scoring**: Low-volume markets (< $50k daily) get a 1.5x risk + multiplier. Easier to have inside information on obscure events. +- **Funding chain analysis**: Traces wallet funding sources on-chain to link + seemingly separate wallets to the same entity or exchange. +- **Sniper cluster detection**: DBSCAN clustering identifies wallets that + consistently enter markets within minutes of creation. + +Composite risk scoring combines signals with configurable weights (default +alert threshold: 0.6). Multi-signal bonuses: 2 signals +20%, 3+ signals +30%. + +## Installation + +```bash +git clone https://github.com/pselamy/polymarket-insider-tracker.git +cd polymarket-insider-tracker +uv sync --all-extras +docker compose up -d # PostgreSQL + Redis +cp .env.example .env # defaults work for local dev +uv run alembic upgrade head +``` + +No API keys required for basic operation (Polymarket APIs are public). + +## Usage + +```bash +# Start the tracker (streams trades, profiles wallets, scores risk, alerts) +uv run python -m polymarket_insider_tracker + +# Dry run (no alerts sent) +uv run python -m polymarket_insider_tracker --dry-run + +# Debug mode (see every trade) +uv run python -m polymarket_insider_tracker --log-level DEBUG + +# Validate config without starting +uv run python -m polymarket_insider_tracker --config-check +``` + +## Interpreting Signals + +### Risk Assessment Output + +Each flagged trade produces a risk assessment with: + +- **Confidence score** (0.0-1.0): Composite of weighted signals +- **Signal breakdown**: Which detectors fired and their individual confidence +- **Wallet profile**: Age, nonce, transaction count, funding source +- **Market context**: Volume, category, order book depth + +### Signal Interpretation Guide + +| Score Range | Interpretation | Action | +|-------------|---------------|--------| +| 0.6-0.7 | Moderate: single strong signal or two weak ones | Monitor, note the market | +| 0.7-0.85 | High: multiple signals converging | Investigate the market and wallet | +| 0.85-1.0 | Critical: fresh wallet + large size + niche market | High-confidence informed flow | + +### What This Is NOT + +- Not a trading signal generator. Informed flow != actionable alpha without + further analysis (hypothesis -> leakage-aware backtest -> capital). +- Not real-time enough for front-running. The tool detects patterns for + research and monitoring, not millisecond-level execution. +- Detection of informed flow does not prove insider trading. Many legitimate + reasons exist for the patterns this tool flags. + +## Rate Limits and Etiquette + +- **Polymarket WebSocket**: No explicit rate limit; one persistent connection. + Do not open multiple connections unnecessarily. +- **Polymarket CLOB REST**: Built-in rate limiter at 10 req/s with retry + backoff on 429/5xx. Respect this for metadata/orderbook queries. +- **Polygon RPC**: Public endpoints (polygon-rpc.com) have low limits. For + sustained use, configure a dedicated RPC provider via `POLYGON_RPC_URL`. + Built-in token-bucket rate limiter at 25 req/s with Redis caching (5min TTL). + +## Known Pitfalls + +1. **WebSocket subscription format**: Must include `action: "subscribe"` in the + envelope. Without it, the server accepts the connection but delivers zero + trade events (silent failure). Fixed in the current version. + +2. **Message routing**: Live-data WebSocket pushes `{connection_id, payload: + {...trade fields}}`, not `{topic, type, payload}`. Route by checking for + `transactionHash` + `proxyWallet` keys in `payload`. + +3. **Public RPC rate limits**: Default Polygon RPC will throttle under load. + Use a dedicated provider for production. + +4. **Database required**: PostgreSQL + Redis must be running. Use + `docker compose up -d` for local dev. + +## Cross-References + +- **Repository**: https://github.com/pselamy/polymarket-insider-tracker +- **Issues**: https://github.com/pselamy/polymarket-insider-tracker/issues +- **Agent skill landing** (follow-on): selamy-labs/agent-skills diff --git a/src/polymarket_insider_tracker/ingestor/websocket.py b/src/polymarket_insider_tracker/ingestor/websocket.py index 133e915..19829dd 100644 --- a/src/polymarket_insider_tracker/ingestor/websocket.py +++ b/src/polymarket_insider_tracker/ingestor/websocket.py @@ -150,7 +150,7 @@ class TradeStreamHandler: elif self._market_filter: subscription["filters"] = json.dumps({"market_slug": self._market_filter}) - return {"subscriptions": [subscription]} + return {"action": "subscribe", "subscriptions": [subscription]} async def _connect(self) -> ClientConnection: """Establish WebSocket connection.""" @@ -183,12 +183,9 @@ class TradeStreamHandler: try: data = json.loads(message) - # Check if this is a trade message - topic = data.get("topic") - msg_type = data.get("type") - - if topic == "activity" and msg_type == "trades": - payload = data.get("payload", {}) + # ws-live-data pushes {connection_id, payload:{...trade fields}} + payload = data.get("payload") + if isinstance(payload, dict) and "transactionHash" in payload and "proxyWallet" in payload: trade = TradeEvent.from_websocket_message(payload) self._stats.trades_received += 1 @@ -208,8 +205,7 @@ class TradeStreamHandler: logger.error("Error in trade callback: %s", e) else: - # Log other message types for debugging - logger.debug("Received message: topic=%s type=%s", topic, msg_type) + logger.debug("Received non-trade message: %s", str(data)[:120]) except json.JSONDecodeError as e: logger.warning("Invalid JSON message: %s", e) diff --git a/tests/ingestor/test_websocket.py b/tests/ingestor/test_websocket.py index 37c671a..218c99b 100644 --- a/tests/ingestor/test_websocket.py +++ b/tests/ingestor/test_websocket.py @@ -84,7 +84,10 @@ class TestTradeStreamHandler: """Test building subscription message without filters.""" msg = handler._build_subscription_message() - assert msg == {"subscriptions": [{"topic": "activity", "type": "trades"}]} + assert msg == { + "action": "subscribe", + "subscriptions": [{"topic": "activity", "type": "trades"}], + } def test_build_subscription_message_with_event_filter(self, on_trade_mock: AsyncMock) -> None: """Test building subscription message with event filter.""" @@ -110,11 +113,10 @@ class TestTradeStreamHandler: async def test_handle_message_trade( self, handler: TradeStreamHandler, on_trade_mock: AsyncMock ) -> None: - """Test handling a valid trade message.""" + """Test handling a valid trade message (payload-based routing).""" message = json.dumps( { - "topic": "activity", - "type": "trades", + "connection_id": "abc123", "payload": { "conditionId": "0xmarket", "transactionHash": "0xtx", @@ -142,11 +144,10 @@ class TestTradeStreamHandler: async def test_handle_message_non_trade( self, handler: TradeStreamHandler, on_trade_mock: AsyncMock ) -> None: - """Test handling a non-trade message.""" + """Test handling a non-trade message (no transactionHash/proxyWallet).""" message = json.dumps( { - "topic": "comments", - "type": "comment_created", + "connection_id": "abc123", "payload": {"body": "Hello"}, } ) @@ -156,6 +157,35 @@ class TestTradeStreamHandler: on_trade_mock.assert_not_called() assert handler.stats.trades_received == 0 + @pytest.mark.asyncio + async def test_handle_message_payload_missing_proxy_wallet( + self, handler: TradeStreamHandler, on_trade_mock: AsyncMock + ) -> None: + """Ratchet: payload with transactionHash but no proxyWallet is not a trade.""" + message = json.dumps( + { + "connection_id": "abc", + "payload": {"transactionHash": "0xtx", "other": "field"}, + } + ) + + await handler._handle_message(message) + + on_trade_mock.assert_not_called() + assert handler.stats.trades_received == 0 + + @pytest.mark.asyncio + async def test_handle_message_no_payload_key( + self, handler: TradeStreamHandler, on_trade_mock: AsyncMock + ) -> None: + """Ratchet: message without payload key is ignored.""" + message = json.dumps({"connection_id": "abc", "status": "ok"}) + + await handler._handle_message(message) + + on_trade_mock.assert_not_called() + assert handler.stats.trades_received == 0 + @pytest.mark.asyncio async def test_handle_message_invalid_json( self, handler: TradeStreamHandler, on_trade_mock: AsyncMock @@ -175,8 +205,7 @@ class TestTradeStreamHandler: message = json.dumps( { - "topic": "activity", - "type": "trades", + "connection_id": "abc", "payload": { "conditionId": "0x", "transactionHash": "0x", @@ -240,8 +269,9 @@ class TestTradeStreamHandler: assert ws is mock_ws mock_ws.send.assert_called_once() - # Verify subscription message + # Verify subscription message includes action: subscribe sent_msg = json.loads(mock_ws.send.call_args[0][0]) + assert sent_msg["action"] == "subscribe" assert "subscriptions" in sent_msg assert sent_msg["subscriptions"][0]["topic"] == "activity" assert sent_msg["subscriptions"][0]["type"] == "trades" @@ -292,8 +322,7 @@ class TestTradeStreamHandlerIntegration: trade_message = json.dumps( { - "topic": "activity", - "type": "trades", + "connection_id": "test-conn", "payload": { "conditionId": "0xtest", "transactionHash": "0xtx",