Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 pselamyandClaude Opus 4.6
|
b1b88a3dd0 | ||
|
schrodinger01andpselamy
|
8ebcd6b4c6 | ||
|
schrodinger01andpselamy
|
ff1e23a0d3 | ||
|
schrodinger01andpselamy
|
82f3e8eb6a | ||
|
|
b962bdaee2 |
@@ -2,80 +2,103 @@
|
||||
|
||||
**Detect informed money before the market moves.**
|
||||
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](https://github.com/pselamy/polymarket-insider-tracker/actions/workflows/ci.yml)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
|
||||
Real-time detection of suspicious trading patterns on Polymarket: fresh wallets, unusual sizing, niche-market activity, and funding chain analysis. Streams trades via WebSocket, profiles wallets on-chain (Polygon), scores risk with ML + heuristics, and dispatches alerts to Discord/Telegram.
|
||||
|
||||
---
|
||||
|
||||
## The Opportunity
|
||||
## Quick Start (< 2 minutes)
|
||||
|
||||
On January 3, 2026, a trader spotted a significant political event on Polymarket **before it happened**. How? Not by predicting the future, but by tracking suspicious trading behavior.
|
||||
### 1. Install
|
||||
|
||||
> "You don't need to predict the future, you need to track suspicious behavior."
|
||||
> — [@DidiTrading](https://x.com/DidiTrading)
|
||||
|
||||
An insider wallet turned **$35,000 into $442,000** (12.6x return) by entering a position hours before a major market move. The tool that detected this activity flagged five separate alerts before the event occurred.
|
||||
|
||||
**This repository builds that tool.**
|
||||
|
||||
---
|
||||
|
||||
## What This Does
|
||||
|
||||
The Polymarket Insider Tracker monitors prediction market trading activity in real-time and identifies patterns that suggest informed trading:
|
||||
|
||||
| Signal | What It Detects | Why It Matters |
|
||||
|--------|-----------------|----------------|
|
||||
| **Fresh Wallets** | Brand new wallets making large trades | Insiders create new wallets to hide their identity |
|
||||
| **Unusual Sizing** | Trades that are disproportionately large for the market | Informed traders bet bigger when they have edge |
|
||||
| **Niche Markets** | Activity in low-volume, specific-outcome markets | Easier to have inside information on obscure events |
|
||||
| **Funding Chains** | Where wallet funds originated from | Links seemingly separate wallets to the same entity |
|
||||
|
||||
When suspicious activity is detected, you receive an instant alert with actionable intelligence.
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌──────────────────┐ ┌────────────────────┐
|
||||
│ Polymarket API │────>│ Wallet Profiler │────>│ Anomaly Detector │
|
||||
│ (Real-time) │ │ (Blockchain) │ │ (ML + Heuristics) │
|
||||
└─────────────────┘ └──────────────────┘ └────────────────────┘
|
||||
│
|
||||
┌────────────────────────────┘
|
||||
v
|
||||
┌─────────────────────┐
|
||||
│ Alert Dispatcher │───> Discord / Telegram / Email
|
||||
│ "Fresh wallet │
|
||||
│ buying YES @7.5¢ │
|
||||
│ on niche market" │
|
||||
└─────────────────────┘
|
||||
```bash
|
||||
# Requires: Python 3.11+, Docker
|
||||
git clone https://github.com/pselamy/polymarket-insider-tracker.git
|
||||
cd polymarket-insider-tracker
|
||||
uv sync --all-extras # or: pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
### Detection Algorithms
|
||||
### 2. Start infrastructure
|
||||
|
||||
1. **Fresh Wallet Detection**
|
||||
- Checks wallet transaction history on Polygon
|
||||
- Flags wallets with fewer than 5 lifetime transactions making trades over $1,000
|
||||
- Traces funding source to identify if connected to known entities
|
||||
```bash
|
||||
docker compose up -d # PostgreSQL 15 + Redis 7
|
||||
docker compose ps # wait for healthy
|
||||
```
|
||||
|
||||
2. **Liquidity Impact Analysis**
|
||||
- Calculates trade size relative to market depth
|
||||
- Flags trades consuming more than 2% of visible order book
|
||||
- Weights by market category (niche markets score higher)
|
||||
### 3. Configure
|
||||
|
||||
3. **Sniper Cluster Detection**
|
||||
- Uses DBSCAN clustering to find wallets that consistently enter markets within minutes of creation
|
||||
- Identifies coordinated behavior patterns
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# Edit .env — only DATABASE_URL and REDIS_URL are required for local dev
|
||||
# (defaults in .env.example work with docker compose)
|
||||
```
|
||||
|
||||
4. **Event Correlation**
|
||||
- Cross-references trading activity with news feeds
|
||||
- Detects positions opened 1-4 hours before related news breaks
|
||||
### 4. Run migrations + start
|
||||
|
||||
```bash
|
||||
uv run alembic upgrade head
|
||||
uv run python -m polymarket_insider_tracker
|
||||
```
|
||||
|
||||
You should see live trades within seconds:
|
||||
|
||||
```
|
||||
INFO Connection state: disconnected -> connecting
|
||||
INFO Connected to wss://ws-live-data.polymarket.com and subscribed to trades
|
||||
DEBUG Trade: BUY 450 @ 1.00 on fifwc-ger-kor-2026-06-14-ger
|
||||
DEBUG Trade: SELL 5 @ 0.86 on chi1-cd1-cdl-2026-06-14-draw
|
||||
```
|
||||
|
||||
### CLI Options
|
||||
|
||||
```bash
|
||||
python -m polymarket_insider_tracker --help
|
||||
--version Show version
|
||||
--config-check Validate configuration and exit
|
||||
--log-level DEBUG Override log level
|
||||
--dry-run Run pipeline without sending alerts
|
||||
--health-port 8080 Override health check port
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sample Alert
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Required | Default | Description |
|
||||
|----------|----------|---------|-------------|
|
||||
| `DATABASE_URL` | Yes | — | PostgreSQL connection string |
|
||||
| `REDIS_URL` | No | `redis://localhost:6379` | Redis connection string |
|
||||
| `POLYGON_RPC_URL` | No | `https://polygon-rpc.com` | Polygon RPC (public default works) |
|
||||
| `POLYGON_FALLBACK_RPC_URL` | No | — | Fallback RPC endpoint |
|
||||
| `POLYMARKET_WS_URL` | No | `wss://ws-live-data.polymarket.com` | WebSocket endpoint |
|
||||
| `POLYMARKET_API_KEY` | No | — | Optional API key for higher rate limits |
|
||||
| `DISCORD_WEBHOOK_URL` | No | — | Discord alerts |
|
||||
| `TELEGRAM_BOT_TOKEN` | No | — | Telegram alerts (needs `TELEGRAM_CHAT_ID` too) |
|
||||
| `TELEGRAM_CHAT_ID` | No | — | Telegram chat for alerts |
|
||||
| `LOG_LEVEL` | No | `INFO` | Logging level |
|
||||
| `DRY_RUN` | No | `false` | Skip sending alerts |
|
||||
| `HEALTH_PORT` | No | `8080` | Health check HTTP port |
|
||||
|
||||
No API keys are needed for basic operation — the Polymarket WebSocket and CLOB REST APIs are public.
|
||||
|
||||
---
|
||||
|
||||
## What It Detects
|
||||
|
||||
| Signal | Detection Method | Threshold |
|
||||
|--------|-----------------|-----------|
|
||||
| **Fresh Wallets** | Wallet age < 48h, nonce <= 5, making trades > $1k | Confidence 0.5-0.9 |
|
||||
| **Size Anomalies** | Trade size > 2% of 24h volume or > 5% of order book | Weighted by niche factor |
|
||||
| **Niche Markets** | Low-volume markets (< $50k daily) with specific outcomes | 1.5x risk multiplier |
|
||||
| **Funding Chains** | Trace wallet funding to known entities (exchanges, etc.) | On-chain lineage |
|
||||
| **Sniper Clusters** | DBSCAN clustering of wallets entering within minutes | Coordinated behavior |
|
||||
|
||||
Risk scoring combines signals with configurable weights (default threshold: 0.6). Multi-signal bonuses: 2 signals +20%, 3+ signals +30%.
|
||||
|
||||
### Sample Alert
|
||||
|
||||
```
|
||||
SUSPICIOUS ACTIVITY DETECTED
|
||||
@@ -99,199 +122,62 @@ Confidence: HIGH (3/4 signals triggered)
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
## Architecture
|
||||
|
||||
### Prerequisites
|
||||
```
|
||||
Polymarket WebSocket ──> Ingestor ──> Profiler ──> Detector ──> Alerter
|
||||
(wss://ws-live-data) (trades) (on-chain) (scoring) (Discord/TG)
|
||||
|
|
||||
Polygon RPC
|
||||
```
|
||||
|
||||
- Python 3.11+
|
||||
- Docker and Docker Compose
|
||||
- Polygon RPC endpoint (Alchemy, QuickNode, or self-hosted)
|
||||
- Polymarket API key (free at [docs.polymarket.com](https://docs.polymarket.com))
|
||||
### Components
|
||||
|
||||
### Installation
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `ingestor/` | WebSocket trade stream + CLOB REST client with rate limiting |
|
||||
| `profiler/` | Polygon wallet analysis, entity identification, funding chain tracing |
|
||||
| `detector/` | Fresh wallet, size anomaly, sniper cluster detection, composite risk scorer |
|
||||
| `alerter/` | Multi-channel dispatch (Discord webhooks, Telegram bot) with dedup |
|
||||
| `storage/` | SQLAlchemy ORM + Alembic migrations (PostgreSQL) |
|
||||
| `pipeline.py` | Orchestrator wiring all components together |
|
||||
| `shutdown.py` | Graceful SIGTERM/SIGINT handling with cleanup callbacks |
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/pselamy/polymarket-insider-tracker.git
|
||||
cd polymarket-insider-tracker
|
||||
|
||||
# Copy environment template
|
||||
cp .env.example .env
|
||||
# Edit .env with your API keys
|
||||
|
||||
# Start infrastructure (PostgreSQL, Redis)
|
||||
docker compose up -d
|
||||
|
||||
# Wait for services to be healthy
|
||||
docker compose ps
|
||||
|
||||
# Install Python dependencies
|
||||
uv sync --all-extras
|
||||
|
||||
# Run database migrations
|
||||
uv run alembic upgrade head
|
||||
|
||||
# Run the tracker
|
||||
uv run python -m polymarket_insider_tracker
|
||||
uv run pytest # run tests
|
||||
uv run ruff check src/ tests/ # lint
|
||||
uv run ruff format src/ tests/ # format
|
||||
uv run mypy src/ # type check (strict mode)
|
||||
```
|
||||
|
||||
### Docker Services
|
||||
|
||||
The development stack includes:
|
||||
|
||||
| Service | Port | Description |
|
||||
|---------|------|-------------|
|
||||
| PostgreSQL 15 | 5432 | Primary database |
|
||||
| Redis 7 | 6379 | Caching and pub/sub |
|
||||
| Adminer | 8080 | Database admin UI (optional) |
|
||||
| RedisInsight | 5540 | Redis admin UI (optional) |
|
||||
|
||||
```bash
|
||||
# Start core services only
|
||||
docker compose up -d
|
||||
|
||||
# Start with development tools (Adminer, RedisInsight)
|
||||
docker compose --profile tools up -d
|
||||
|
||||
# View logs
|
||||
docker compose logs -f
|
||||
|
||||
# Stop all services
|
||||
docker compose down
|
||||
|
||||
# Stop and remove volumes (reset data)
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
```bash
|
||||
# .env file
|
||||
POLYGON_RPC_URL=https://polygon-mainnet.g.alchemy.com/v2/YOUR_KEY
|
||||
POLYMARKET_API_KEY=your_polymarket_api_key
|
||||
|
||||
# Alert destinations (optional)
|
||||
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
|
||||
TELEGRAM_BOT_TOKEN=your_bot_token
|
||||
TELEGRAM_CHAT_ID=your_chat_id
|
||||
|
||||
# Detection thresholds
|
||||
MIN_TRADE_SIZE_USDC=1000
|
||||
FRESH_WALLET_MAX_NONCE=5
|
||||
LIQUIDITY_IMPACT_THRESHOLD=0.02
|
||||
```
|
||||
| Adminer | 8080 | Database admin UI (optional, `--profile tools`) |
|
||||
| RedisInsight | 5540 | Redis admin UI (optional, `--profile tools`) |
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
## Troubleshooting
|
||||
|
||||
```
|
||||
polymarket-insider-tracker/
|
||||
├── src/
|
||||
│ └── polymarket_insider_tracker/
|
||||
│ ├── __main__.py # CLI entry point
|
||||
│ ├── pipeline.py # Core detection pipeline
|
||||
│ ├── ingestor/ # Real-time market data ingestion
|
||||
│ │ ├── clob_client.py # Polymarket CLOB API wrapper
|
||||
│ │ └── websocket.py # WebSocket event handler
|
||||
│ ├── profiler/ # Wallet analysis
|
||||
│ ├── detector/ # Anomaly detection engines
|
||||
│ ├── alerter/ # Notification dispatch
|
||||
│ └── storage/ # Persistence layer
|
||||
├── tests/ # Test suite
|
||||
├── scripts/
|
||||
│ └── backtest.py # Historical analysis
|
||||
├── docker-compose.yml
|
||||
├── pyproject.toml
|
||||
└── README.md
|
||||
```
|
||||
**No trades received / silent connection**
|
||||
The WebSocket subscription requires `action: "subscribe"` in the envelope. If you're on an older version, update — this was fixed in the WebSocket protocol alignment (see #89).
|
||||
|
||||
---
|
||||
**Connection timeout / DNS errors**
|
||||
Verify `wss://ws-live-data.polymarket.com` is reachable from your network. Some corporate firewalls block WebSocket connections.
|
||||
|
||||
## Roadmap
|
||||
**Database migration errors**
|
||||
Ensure PostgreSQL is running (`docker compose ps`) and `DATABASE_URL` matches your docker-compose config. Run `uv run alembic upgrade head` after any schema changes.
|
||||
|
||||
### Phase 1: Core Detection (Current)
|
||||
- [x] Project structure and documentation
|
||||
- [ ] Polymarket CLOB API integration
|
||||
- [ ] Fresh wallet detection
|
||||
- [ ] Size anomaly detection
|
||||
- [ ] Basic alerting (Discord/Telegram)
|
||||
|
||||
### Phase 2: Advanced Intelligence
|
||||
- [ ] Funding chain analysis
|
||||
- [ ] Sniper cluster detection (DBSCAN)
|
||||
- [ ] Market categorization (niche vs mainstream)
|
||||
- [ ] Historical backtesting framework
|
||||
|
||||
### Phase 3: Production Hardening
|
||||
- [ ] High-availability deployment
|
||||
- [ ] Rate limit management
|
||||
- [ ] False positive feedback loop
|
||||
- [ ] Web dashboard
|
||||
|
||||
---
|
||||
|
||||
## Why This Matters
|
||||
|
||||
Prediction markets are becoming a critical source of real-time probability estimates for world events. As they grow, so does the incentive for informed actors to exploit information asymmetry.
|
||||
|
||||
This tool democratizes access to the same detection capabilities that sophisticated traders use. Whether you are:
|
||||
|
||||
- **A trader** looking for alpha signals
|
||||
- **A researcher** studying market microstructure
|
||||
- **A platform operator** monitoring for manipulation
|
||||
|
||||
...this tracker provides visibility into the hidden flows that move markets.
|
||||
|
||||
---
|
||||
|
||||
## Technical Background
|
||||
|
||||
### Polymarket Architecture
|
||||
|
||||
Polymarket is a prediction market platform built on Polygon (Ethereum L2). Key characteristics:
|
||||
|
||||
- **CLOB (Central Limit Order Book)**: Centralized matching engine for speed
|
||||
- **On-chain Settlement**: Final trades settle on Polygon blockchain
|
||||
- **USDC Collateral**: All positions denominated in USDC stablecoin
|
||||
- **Binary Outcomes**: Shares priced between $0.00 and $1.00
|
||||
|
||||
### Data Sources
|
||||
|
||||
| Source | Purpose | Latency |
|
||||
|--------|---------|---------|
|
||||
| Polymarket CLOB API | Real-time trades, orderbook | Milliseconds |
|
||||
| Polygon RPC | Wallet history, nonce, funding | 1-2 seconds |
|
||||
| Market Metadata API | Market categorization | On-demand |
|
||||
|
||||
### Detection Challenges
|
||||
|
||||
1. **Sybil Resistance**: Insiders use fresh wallets per trade
|
||||
2. **Rate Limits**: Polygon RPC calls require caching strategy
|
||||
3. **Market Classification**: NLP needed to categorize market niches
|
||||
4. **Timing**: CLOB data leads on-chain by seconds
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! Please read our Contributing Guide before submitting PRs.
|
||||
|
||||
### Development Setup
|
||||
|
||||
```bash
|
||||
# Install dev dependencies
|
||||
uv sync --all-extras
|
||||
|
||||
# Run tests
|
||||
uv run pytest
|
||||
|
||||
# Run linting
|
||||
uv run ruff check src/
|
||||
|
||||
# Run type checking
|
||||
uv run mypy src/
|
||||
```
|
||||
**Rate limiting on Polygon RPC**
|
||||
The default public RPC (`https://polygon-rpc.com`) has low rate limits. For production use, set `POLYGON_RPC_URL` to a dedicated provider (Alchemy, QuickNode, etc.).
|
||||
|
||||
---
|
||||
|
||||
@@ -304,20 +190,6 @@ This software is provided for **educational and research purposes only**.
|
||||
- Insider trading is illegal in regulated markets; this tool is for transparency and research
|
||||
- Users are responsible for compliance with applicable laws
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see [LICENSE](LICENSE) for details.
|
||||
|
||||
---
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
- Inspired by [@DidiTrading](https://x.com/DidiTrading) and [@spacexbt](https://x.com/spacexbt)
|
||||
- Built on the open Polymarket API ecosystem
|
||||
- Community contributions welcome
|
||||
|
||||
---
|
||||
|
||||
**Questions?** Open an issue or start a discussion.
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
# Skill: tracking-prediction-market-flow
|
||||
|
||||
Use when analyzing prediction market activity for informed-flow signals, insider
|
||||
trading patterns, or suspicious wallet behavior on Polymarket.
|
||||
|
||||
## What This Tool Does
|
||||
|
||||
polymarket-insider-tracker streams real-time trades from Polymarket's WebSocket
|
||||
feed, profiles trader wallets on the Polygon blockchain, and scores each trade
|
||||
for informed-flow risk using multiple detection signals:
|
||||
|
||||
- **Fresh wallet detection**: New wallets (age < 48h, nonce <= 5) making large
|
||||
trades (> $1k). Insiders create disposable wallets per trade.
|
||||
- **Size anomaly detection**: Trades consuming > 2% of 24h volume or > 5% of
|
||||
visible order book depth. Informed traders bet bigger when they have edge.
|
||||
- **Niche market scoring**: Low-volume markets (< $50k daily) get a 1.5x risk
|
||||
multiplier. Easier to have inside information on obscure events.
|
||||
- **Funding chain analysis**: Traces wallet funding sources on-chain to link
|
||||
seemingly separate wallets to the same entity or exchange.
|
||||
- **Sniper cluster detection**: DBSCAN clustering identifies wallets that
|
||||
consistently enter markets within minutes of creation.
|
||||
|
||||
Composite risk scoring combines signals with configurable weights (default
|
||||
alert threshold: 0.6). Multi-signal bonuses: 2 signals +20%, 3+ signals +30%.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
git clone https://github.com/pselamy/polymarket-insider-tracker.git
|
||||
cd polymarket-insider-tracker
|
||||
uv sync --all-extras
|
||||
docker compose up -d # PostgreSQL + Redis
|
||||
cp .env.example .env # defaults work for local dev
|
||||
uv run alembic upgrade head
|
||||
```
|
||||
|
||||
No API keys required for basic operation (Polymarket APIs are public).
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# Start the tracker (streams trades, profiles wallets, scores risk, alerts)
|
||||
uv run python -m polymarket_insider_tracker
|
||||
|
||||
# Dry run (no alerts sent)
|
||||
uv run python -m polymarket_insider_tracker --dry-run
|
||||
|
||||
# Debug mode (see every trade)
|
||||
uv run python -m polymarket_insider_tracker --log-level DEBUG
|
||||
|
||||
# Validate config without starting
|
||||
uv run python -m polymarket_insider_tracker --config-check
|
||||
```
|
||||
|
||||
## Interpreting Signals
|
||||
|
||||
### Risk Assessment Output
|
||||
|
||||
Each flagged trade produces a risk assessment with:
|
||||
|
||||
- **Confidence score** (0.0-1.0): Composite of weighted signals
|
||||
- **Signal breakdown**: Which detectors fired and their individual confidence
|
||||
- **Wallet profile**: Age, nonce, transaction count, funding source
|
||||
- **Market context**: Volume, category, order book depth
|
||||
|
||||
### Signal Interpretation Guide
|
||||
|
||||
| Score Range | Interpretation | Action |
|
||||
|-------------|---------------|--------|
|
||||
| 0.6-0.7 | Moderate: single strong signal or two weak ones | Monitor, note the market |
|
||||
| 0.7-0.85 | High: multiple signals converging | Investigate the market and wallet |
|
||||
| 0.85-1.0 | Critical: fresh wallet + large size + niche market | High-confidence informed flow |
|
||||
|
||||
### What This Is NOT
|
||||
|
||||
- Not a trading signal generator. Informed flow != actionable alpha without
|
||||
further analysis (hypothesis -> leakage-aware backtest -> capital).
|
||||
- Not real-time enough for front-running. The tool detects patterns for
|
||||
research and monitoring, not millisecond-level execution.
|
||||
- Detection of informed flow does not prove insider trading. Many legitimate
|
||||
reasons exist for the patterns this tool flags.
|
||||
|
||||
## Rate Limits and Etiquette
|
||||
|
||||
- **Polymarket WebSocket**: No explicit rate limit; one persistent connection.
|
||||
Do not open multiple connections unnecessarily.
|
||||
- **Polymarket CLOB REST**: Built-in rate limiter at 10 req/s with retry
|
||||
backoff on 429/5xx. Respect this for metadata/orderbook queries.
|
||||
- **Polygon RPC**: Public endpoints (polygon-rpc.com) have low limits. For
|
||||
sustained use, configure a dedicated RPC provider via `POLYGON_RPC_URL`.
|
||||
Built-in token-bucket rate limiter at 25 req/s with Redis caching (5min TTL).
|
||||
|
||||
## Known Pitfalls
|
||||
|
||||
1. **WebSocket subscription format**: Must include `action: "subscribe"` in the
|
||||
envelope. Without it, the server accepts the connection but delivers zero
|
||||
trade events (silent failure). Fixed in the current version.
|
||||
|
||||
2. **Message routing**: Live-data WebSocket pushes `{connection_id, payload:
|
||||
{...trade fields}}`, not `{topic, type, payload}`. Route by checking for
|
||||
`transactionHash` + `proxyWallet` keys in `payload`.
|
||||
|
||||
3. **Public RPC rate limits**: Default Polygon RPC will throttle under load.
|
||||
Use a dedicated provider for production.
|
||||
|
||||
4. **Database required**: PostgreSQL + Redis must be running. Use
|
||||
`docker compose up -d` for local dev.
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Repository**: https://github.com/pselamy/polymarket-insider-tracker
|
||||
- **Issues**: https://github.com/pselamy/polymarket-insider-tracker/issues
|
||||
- **Agent skill landing** (follow-on): selamy-labs/agent-skills
|
||||
@@ -18,7 +18,9 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
class DatabaseSettings(BaseSettings):
|
||||
"""Database connection settings."""
|
||||
|
||||
model_config = SettingsConfigDict(env_prefix="", env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="", env_file=".env", env_file_encoding="utf-8", extra="ignore"
|
||||
)
|
||||
|
||||
url: str = Field(
|
||||
alias="DATABASE_URL",
|
||||
@@ -37,7 +39,9 @@ class DatabaseSettings(BaseSettings):
|
||||
class RedisSettings(BaseSettings):
|
||||
"""Redis connection settings."""
|
||||
|
||||
model_config = SettingsConfigDict(env_prefix="", env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="", env_file=".env", env_file_encoding="utf-8", extra="ignore"
|
||||
)
|
||||
|
||||
url: str = Field(
|
||||
default="redis://localhost:6379",
|
||||
@@ -57,7 +61,9 @@ class RedisSettings(BaseSettings):
|
||||
class PolygonSettings(BaseSettings):
|
||||
"""Polygon blockchain RPC settings."""
|
||||
|
||||
model_config = SettingsConfigDict(env_prefix="POLYGON_", env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="POLYGON_", env_file=".env", env_file_encoding="utf-8", extra="ignore"
|
||||
)
|
||||
|
||||
rpc_url: str = Field(
|
||||
default="https://polygon-rpc.com",
|
||||
@@ -84,7 +90,9 @@ class PolygonSettings(BaseSettings):
|
||||
class PolymarketSettings(BaseSettings):
|
||||
"""Polymarket API settings."""
|
||||
|
||||
model_config = SettingsConfigDict(env_prefix="POLYMARKET_", env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="POLYMARKET_", env_file=".env", env_file_encoding="utf-8", extra="ignore"
|
||||
)
|
||||
|
||||
ws_url: str = Field(
|
||||
default="wss://ws-subscriptions-clob.polymarket.com/ws/market",
|
||||
@@ -109,7 +117,9 @@ class PolymarketSettings(BaseSettings):
|
||||
class DiscordSettings(BaseSettings):
|
||||
"""Discord notification settings."""
|
||||
|
||||
model_config = SettingsConfigDict(env_prefix="DISCORD_", env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="DISCORD_", env_file=".env", env_file_encoding="utf-8", extra="ignore"
|
||||
)
|
||||
|
||||
webhook_url: SecretStr | None = Field(
|
||||
default=None,
|
||||
@@ -126,7 +136,9 @@ class DiscordSettings(BaseSettings):
|
||||
class TelegramSettings(BaseSettings):
|
||||
"""Telegram notification settings."""
|
||||
|
||||
model_config = SettingsConfigDict(env_prefix="TELEGRAM_", env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="TELEGRAM_", env_file=".env", env_file_encoding="utf-8", extra="ignore"
|
||||
)
|
||||
|
||||
bot_token: SecretStr | None = Field(
|
||||
default=None,
|
||||
@@ -142,7 +154,12 @@ class TelegramSettings(BaseSettings):
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
"""Check if Telegram notifications are enabled."""
|
||||
return self.bot_token is not None and bool(self.bot_token.get_secret_value().strip()) and self.chat_id is not None and bool(self.chat_id.strip())
|
||||
return (
|
||||
self.bot_token is not None
|
||||
and bool(self.bot_token.get_secret_value().strip())
|
||||
and self.chat_id is not None
|
||||
and bool(self.chat_id.strip())
|
||||
)
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
|
||||
@@ -150,7 +150,7 @@ class TradeStreamHandler:
|
||||
elif self._market_filter:
|
||||
subscription["filters"] = json.dumps({"market_slug": self._market_filter})
|
||||
|
||||
return {"subscriptions": [subscription]}
|
||||
return {"action": "subscribe", "subscriptions": [subscription]}
|
||||
|
||||
async def _connect(self) -> ClientConnection:
|
||||
"""Establish WebSocket connection."""
|
||||
@@ -183,12 +183,13 @@ class TradeStreamHandler:
|
||||
try:
|
||||
data = json.loads(message)
|
||||
|
||||
# Check if this is a trade message
|
||||
topic = data.get("topic")
|
||||
msg_type = data.get("type")
|
||||
|
||||
if topic == "activity" and msg_type == "trades":
|
||||
payload = data.get("payload", {})
|
||||
# ws-live-data pushes {connection_id, payload:{...trade fields}}
|
||||
payload = data.get("payload")
|
||||
if (
|
||||
isinstance(payload, dict)
|
||||
and "transactionHash" in payload
|
||||
and "proxyWallet" in payload
|
||||
):
|
||||
trade = TradeEvent.from_websocket_message(payload)
|
||||
|
||||
self._stats.trades_received += 1
|
||||
@@ -208,8 +209,7 @@ class TradeStreamHandler:
|
||||
logger.error("Error in trade callback: %s", e)
|
||||
|
||||
else:
|
||||
# Log other message types for debugging
|
||||
logger.debug("Received message: topic=%s type=%s", topic, msg_type)
|
||||
logger.debug("Received non-trade message: %s", str(data)[:120])
|
||||
|
||||
except json.JSONDecodeError as e:
|
||||
logger.warning("Invalid JSON message: %s", e)
|
||||
|
||||
@@ -26,8 +26,46 @@ logger = logging.getLogger(__name__)
|
||||
USDC_BRIDGED = "0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174"
|
||||
USDC_NATIVE = "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359"
|
||||
|
||||
# ERC20 Transfer event signature
|
||||
# ERC20 Transfer event signature. ``HexBytes.hex()`` returns a *bare* hex
|
||||
# string without the ``0x`` prefix; publicnode tolerates that, but stricter
|
||||
# providers (e.g. drpc — which we use as the fallback) reject it with
|
||||
# ``invalid argument 0: hex string without 0x prefix``. Always pass the
|
||||
# 0x-prefixed form to ``eth_getLogs``.
|
||||
TRANSFER_EVENT_SIGNATURE = AsyncWeb3.keccak(text="Transfer(address,address,uint256)")
|
||||
TRANSFER_EVENT_TOPIC = "0x" + TRANSFER_EVENT_SIGNATURE.hex().removeprefix("0x")
|
||||
|
||||
# eth_getLogs block-range chunking. Most public Polygon RPCs (publicnode, ankr,
|
||||
# llamarpc) cap the range at 10_000 blocks per call; pick a window slightly
|
||||
# under the cap so off-by-one differences between providers don't trip us up.
|
||||
DEFAULT_CHUNK_SIZE_BLOCKS = 9_000
|
||||
# Polygon block time is ~2.0s. publicnode (the most common free RPC) prunes
|
||||
# log history aggressively — empirically only ~100k blocks (~55 hours) are
|
||||
# served before requests start returning "History has been pruned". We default
|
||||
# to 80k blocks (~44 hours), which is more than enough for fresh-wallet
|
||||
# funding traces (those wallets are by definition new) and fits comfortably
|
||||
# inside what most public providers retain.
|
||||
DEFAULT_MAX_LOOKBACK_BLOCKS = 80_000
|
||||
|
||||
# Substrings that, when present in an RPC error, indicate the chunk we just
|
||||
# asked for is outside the provider's archive horizon. Walking further back
|
||||
# is futile, so we stop the trace early instead of hammering every chunk.
|
||||
_PRUNED_HISTORY_MARKERS: tuple[str, ...] = (
|
||||
"history has been pruned",
|
||||
"missing trie node",
|
||||
"older than",
|
||||
)
|
||||
|
||||
|
||||
def _is_pruned_history_error(err: BaseException) -> bool:
|
||||
"""Return True if the RPC error indicates pruned history.
|
||||
|
||||
Public Polygon nodes only retain a recent slice of log history. When we
|
||||
walk back through that slice in chunks and hit the cutoff, every further
|
||||
chunk will fail with the same message — so we stop early instead of
|
||||
burning quota on guaranteed failures.
|
||||
"""
|
||||
text = str(err).lower()
|
||||
return any(marker in text for marker in _PRUNED_HISTORY_MARKERS)
|
||||
|
||||
|
||||
class FundingTracer:
|
||||
@@ -50,6 +88,8 @@ class FundingTracer:
|
||||
*,
|
||||
max_hops: int = 3,
|
||||
usdc_addresses: list[str] | None = None,
|
||||
chunk_size_blocks: int = DEFAULT_CHUNK_SIZE_BLOCKS,
|
||||
max_lookback_blocks: int = DEFAULT_MAX_LOOKBACK_BLOCKS,
|
||||
) -> None:
|
||||
"""Initialize the funding tracer.
|
||||
|
||||
@@ -58,6 +98,11 @@ class FundingTracer:
|
||||
entity_registry: Registry for entity classification. Creates default if None.
|
||||
max_hops: Maximum hops to trace back (default 3).
|
||||
usdc_addresses: USDC contract addresses to track. Uses defaults if None.
|
||||
chunk_size_blocks: Block window size per eth_getLogs call. Public
|
||||
Polygon RPCs cap at 10_000 blocks; default leaves a safety margin.
|
||||
max_lookback_blocks: How far back to scan when caller passes
|
||||
``from_block=0``. Default ~44 hours at 2s block time, which
|
||||
fits inside the pruned-history horizon of most public RPCs.
|
||||
"""
|
||||
self.polygon_client = polygon_client
|
||||
self.entity_registry = entity_registry or EntityRegistry()
|
||||
@@ -65,6 +110,8 @@ class FundingTracer:
|
||||
self._usdc_addresses = [
|
||||
addr.lower() for addr in (usdc_addresses or [USDC_BRIDGED, USDC_NATIVE])
|
||||
]
|
||||
self._chunk_size_blocks = chunk_size_blocks
|
||||
self._max_lookback_blocks = max_lookback_blocks
|
||||
|
||||
async def trace(
|
||||
self,
|
||||
@@ -209,46 +256,144 @@ class FundingTracer:
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Get ERC20 Transfer event logs.
|
||||
|
||||
Public Polygon RPCs (publicnode, ankr, llamarpc) cap ``eth_getLogs`` at
|
||||
10_000 blocks per call. To work around this we resolve the requested
|
||||
range into a concrete block window (defaulting to the last
|
||||
``max_lookback_blocks`` when caller passes ``from_block=0``) and walk
|
||||
the window in chunks of ``chunk_size_blocks``, oldest-first, stopping
|
||||
once ``limit`` matches are collected. Walking oldest-first preserves
|
||||
the "first transfer" semantics expected by the funding chain tracer.
|
||||
|
||||
If a chunk comes back with a "history has been pruned" style error
|
||||
the rest of the walk is short-circuited — every subsequent chunk
|
||||
would hit the same archive cutoff and there's no point burning quota
|
||||
on guaranteed failures.
|
||||
|
||||
Args:
|
||||
to_address: Filter by recipient address.
|
||||
token_address: ERC20 token contract address.
|
||||
limit: Maximum logs to return.
|
||||
from_block: Starting block number.
|
||||
to_block: Ending block number.
|
||||
from_block: Starting block number (0 means
|
||||
``latest - max_lookback_blocks``).
|
||||
to_block: Ending block number ("latest" resolves to current head).
|
||||
|
||||
Returns:
|
||||
List of log dictionaries.
|
||||
List of log dictionaries, oldest first, capped at ``limit``.
|
||||
"""
|
||||
# Pad address to 32 bytes for topic filter
|
||||
padded_to = "0x" + to_address.lower().replace("0x", "").zfill(64)
|
||||
topics = [
|
||||
TRANSFER_EVENT_TOPIC, # Transfer event (must be 0x-prefixed for drpc)
|
||||
None, # from (any)
|
||||
padded_to, # to (target address)
|
||||
]
|
||||
contract_address = AsyncWeb3.to_checksum_address(token_address)
|
||||
|
||||
start_block, end_block = await self._resolve_block_range(from_block, to_block)
|
||||
if start_block > end_block:
|
||||
return []
|
||||
|
||||
results: list[dict[str, Any]] = []
|
||||
chunk_size = max(1, self._chunk_size_blocks)
|
||||
chunk_start = start_block
|
||||
|
||||
while chunk_start <= end_block:
|
||||
chunk_end = min(chunk_start + chunk_size - 1, end_block)
|
||||
try:
|
||||
chunk_logs = await self._fetch_logs_chunk(
|
||||
contract_address=contract_address,
|
||||
topics=topics,
|
||||
from_block=chunk_start,
|
||||
to_block=chunk_end,
|
||||
)
|
||||
except Exception as e:
|
||||
if _is_pruned_history_error(e):
|
||||
# The provider has dropped this slice of history. Walking
|
||||
# further back will hit the same wall on every chunk;
|
||||
# stop now and return what we already have.
|
||||
logger.info(
|
||||
"eth_getLogs chunk %d-%d outside archive horizon for %s; stopping trace",
|
||||
chunk_start,
|
||||
chunk_end,
|
||||
to_address,
|
||||
)
|
||||
break
|
||||
logger.warning(
|
||||
"eth_getLogs chunk %d-%d failed for %s: %s",
|
||||
chunk_start,
|
||||
chunk_end,
|
||||
to_address,
|
||||
e,
|
||||
)
|
||||
# Skip this window and keep walking — partial data is better
|
||||
# than aborting the whole trace on a single flaky chunk.
|
||||
chunk_start = chunk_end + 1
|
||||
continue
|
||||
|
||||
for log in chunk_logs:
|
||||
results.append(dict(log))
|
||||
if len(results) >= limit:
|
||||
return results
|
||||
|
||||
chunk_start = chunk_end + 1
|
||||
|
||||
return results
|
||||
|
||||
async def _resolve_block_range(
|
||||
self,
|
||||
from_block: int | str,
|
||||
to_block: int | str,
|
||||
) -> tuple[int, int]:
|
||||
"""Resolve symbolic block params to concrete numeric bounds.
|
||||
|
||||
``from_block=0`` (the historical default) is rewritten to
|
||||
``latest - max_lookback_blocks`` so we don't try to scan all of Polygon.
|
||||
"""
|
||||
w3 = self._select_w3()
|
||||
|
||||
if isinstance(to_block, str):
|
||||
await self.polygon_client._rate_limiter.acquire()
|
||||
head = int(await w3.eth.block_number)
|
||||
end = head
|
||||
else:
|
||||
end = int(to_block)
|
||||
|
||||
if isinstance(from_block, str):
|
||||
# Treat any symbolic from-block (e.g. "earliest") as "go back
|
||||
# max_lookback_blocks from end"; that's what callers actually want.
|
||||
start = max(0, end - self._max_lookback_blocks)
|
||||
elif from_block == 0:
|
||||
start = max(0, end - self._max_lookback_blocks)
|
||||
else:
|
||||
start = int(from_block)
|
||||
|
||||
return start, end
|
||||
|
||||
async def _fetch_logs_chunk(
|
||||
self,
|
||||
contract_address: str,
|
||||
topics: list[Any],
|
||||
from_block: int,
|
||||
to_block: int,
|
||||
) -> list[Any]:
|
||||
"""Issue a single bounded ``eth_getLogs`` call."""
|
||||
await self.polygon_client._rate_limiter.acquire()
|
||||
|
||||
# Use the web3 instance from polygon client
|
||||
w3 = (
|
||||
self.polygon_client._w3
|
||||
if self.polygon_client._primary_healthy
|
||||
else (self.polygon_client._w3_fallback or self.polygon_client._w3)
|
||||
)
|
||||
|
||||
# Get logs with Transfer event filtering by recipient
|
||||
w3 = self._select_w3()
|
||||
# Note: web3 typing is overly restrictive for block params
|
||||
logs = await w3.eth.get_logs(
|
||||
return await w3.eth.get_logs(
|
||||
{
|
||||
"address": AsyncWeb3.to_checksum_address(token_address),
|
||||
"topics": [
|
||||
TRANSFER_EVENT_SIGNATURE.hex(), # Transfer event
|
||||
None, # from (any)
|
||||
padded_to, # to (target address)
|
||||
],
|
||||
"address": contract_address,
|
||||
"topics": topics,
|
||||
"fromBlock": from_block, # type: ignore[typeddict-item]
|
||||
"toBlock": to_block, # type: ignore[typeddict-item]
|
||||
}
|
||||
)
|
||||
|
||||
# Convert to list of dicts and limit
|
||||
result = [dict(log) for log in logs[:limit]]
|
||||
return result
|
||||
def _select_w3(self) -> AsyncWeb3:
|
||||
"""Pick primary or fallback web3 instance based on health."""
|
||||
if self.polygon_client._primary_healthy:
|
||||
return self.polygon_client._w3
|
||||
return self.polygon_client._w3_fallback or self.polygon_client._w3
|
||||
|
||||
async def _log_to_funding_transfer(
|
||||
self,
|
||||
|
||||
@@ -84,7 +84,10 @@ class TestTradeStreamHandler:
|
||||
"""Test building subscription message without filters."""
|
||||
msg = handler._build_subscription_message()
|
||||
|
||||
assert msg == {"subscriptions": [{"topic": "activity", "type": "trades"}]}
|
||||
assert msg == {
|
||||
"action": "subscribe",
|
||||
"subscriptions": [{"topic": "activity", "type": "trades"}],
|
||||
}
|
||||
|
||||
def test_build_subscription_message_with_event_filter(self, on_trade_mock: AsyncMock) -> None:
|
||||
"""Test building subscription message with event filter."""
|
||||
@@ -110,11 +113,10 @@ class TestTradeStreamHandler:
|
||||
async def test_handle_message_trade(
|
||||
self, handler: TradeStreamHandler, on_trade_mock: AsyncMock
|
||||
) -> None:
|
||||
"""Test handling a valid trade message."""
|
||||
"""Test handling a valid trade message (payload-based routing)."""
|
||||
message = json.dumps(
|
||||
{
|
||||
"topic": "activity",
|
||||
"type": "trades",
|
||||
"connection_id": "abc123",
|
||||
"payload": {
|
||||
"conditionId": "0xmarket",
|
||||
"transactionHash": "0xtx",
|
||||
@@ -142,11 +144,10 @@ class TestTradeStreamHandler:
|
||||
async def test_handle_message_non_trade(
|
||||
self, handler: TradeStreamHandler, on_trade_mock: AsyncMock
|
||||
) -> None:
|
||||
"""Test handling a non-trade message."""
|
||||
"""Test handling a non-trade message (no transactionHash/proxyWallet)."""
|
||||
message = json.dumps(
|
||||
{
|
||||
"topic": "comments",
|
||||
"type": "comment_created",
|
||||
"connection_id": "abc123",
|
||||
"payload": {"body": "Hello"},
|
||||
}
|
||||
)
|
||||
@@ -156,6 +157,35 @@ class TestTradeStreamHandler:
|
||||
on_trade_mock.assert_not_called()
|
||||
assert handler.stats.trades_received == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_handle_message_payload_missing_proxy_wallet(
|
||||
self, handler: TradeStreamHandler, on_trade_mock: AsyncMock
|
||||
) -> None:
|
||||
"""Ratchet: payload with transactionHash but no proxyWallet is not a trade."""
|
||||
message = json.dumps(
|
||||
{
|
||||
"connection_id": "abc",
|
||||
"payload": {"transactionHash": "0xtx", "other": "field"},
|
||||
}
|
||||
)
|
||||
|
||||
await handler._handle_message(message)
|
||||
|
||||
on_trade_mock.assert_not_called()
|
||||
assert handler.stats.trades_received == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_handle_message_no_payload_key(
|
||||
self, handler: TradeStreamHandler, on_trade_mock: AsyncMock
|
||||
) -> None:
|
||||
"""Ratchet: message without payload key is ignored."""
|
||||
message = json.dumps({"connection_id": "abc", "status": "ok"})
|
||||
|
||||
await handler._handle_message(message)
|
||||
|
||||
on_trade_mock.assert_not_called()
|
||||
assert handler.stats.trades_received == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_handle_message_invalid_json(
|
||||
self, handler: TradeStreamHandler, on_trade_mock: AsyncMock
|
||||
@@ -175,8 +205,7 @@ class TestTradeStreamHandler:
|
||||
|
||||
message = json.dumps(
|
||||
{
|
||||
"topic": "activity",
|
||||
"type": "trades",
|
||||
"connection_id": "abc",
|
||||
"payload": {
|
||||
"conditionId": "0x",
|
||||
"transactionHash": "0x",
|
||||
@@ -240,8 +269,9 @@ class TestTradeStreamHandler:
|
||||
assert ws is mock_ws
|
||||
mock_ws.send.assert_called_once()
|
||||
|
||||
# Verify subscription message
|
||||
# Verify subscription message includes action: subscribe
|
||||
sent_msg = json.loads(mock_ws.send.call_args[0][0])
|
||||
assert sent_msg["action"] == "subscribe"
|
||||
assert "subscriptions" in sent_msg
|
||||
assert sent_msg["subscriptions"][0]["topic"] == "activity"
|
||||
assert sent_msg["subscriptions"][0]["type"] == "trades"
|
||||
@@ -292,8 +322,7 @@ class TestTradeStreamHandlerIntegration:
|
||||
|
||||
trade_message = json.dumps(
|
||||
{
|
||||
"topic": "activity",
|
||||
"type": "trades",
|
||||
"connection_id": "test-conn",
|
||||
"payload": {
|
||||
"conditionId": "0xtest",
|
||||
"transactionHash": "0xtx",
|
||||
|
||||
@@ -327,6 +327,10 @@ class TestGetTransferLogs:
|
||||
await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
# Explicit numeric range so we stay inside one chunk and skip
|
||||
# the "latest" → block_number resolution path.
|
||||
from_block=1,
|
||||
to_block=8_000,
|
||||
)
|
||||
|
||||
mock_w3.eth.get_logs.assert_called_once()
|
||||
@@ -334,10 +338,15 @@ class TestGetTransferLogs:
|
||||
|
||||
# Verify topics structure
|
||||
assert len(call_args["topics"]) == 3
|
||||
assert call_args["topics"][0] == TRANSFER_EVENT_SIGNATURE.hex()
|
||||
# The Transfer event topic must be 0x-prefixed; drpc rejects bare hex.
|
||||
assert call_args["topics"][0] == "0x" + TRANSFER_EVENT_SIGNATURE.hex().removeprefix("0x")
|
||||
assert call_args["topics"][0].startswith("0x")
|
||||
assert call_args["topics"][1] is None # from (any)
|
||||
# to address should be padded to 32 bytes
|
||||
assert call_args["topics"][2].endswith(TEST_WALLET.lower().replace("0x", ""))
|
||||
# And the chunk bounds match what we asked for.
|
||||
assert call_args["fromBlock"] == 1
|
||||
assert call_args["toBlock"] == 8_000
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_respects_limit(
|
||||
@@ -355,6 +364,8 @@ class TestGetTransferLogs:
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
limit=3,
|
||||
from_block=1,
|
||||
to_block=8_000,
|
||||
)
|
||||
|
||||
assert len(result) == 3
|
||||
@@ -374,10 +385,242 @@ class TestGetTransferLogs:
|
||||
await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
from_block=1,
|
||||
to_block=8_000,
|
||||
)
|
||||
|
||||
mock_fallback.eth.get_logs.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_chunks_large_ranges(
|
||||
self,
|
||||
funding_tracer: FundingTracer,
|
||||
mock_polygon_client: MagicMock,
|
||||
) -> None:
|
||||
"""Ranges wider than chunk_size are split into multiple eth_getLogs calls.
|
||||
|
||||
This is the regression guard for the publicnode 10_000-block cap that
|
||||
was rejecting every funding trace before chunking landed.
|
||||
"""
|
||||
mock_w3 = MagicMock()
|
||||
mock_w3.eth.get_logs = AsyncMock(return_value=[])
|
||||
mock_polygon_client._w3 = mock_w3
|
||||
|
||||
# 25_000 blocks at 9_000-per-chunk → 3 calls (9000 + 9000 + 7001).
|
||||
await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
from_block=1_000_000,
|
||||
to_block=1_025_000,
|
||||
)
|
||||
|
||||
assert mock_w3.eth.get_logs.call_count == 3
|
||||
windows = [call[0][0] for call in mock_w3.eth.get_logs.call_args_list]
|
||||
assert windows[0]["fromBlock"] == 1_000_000
|
||||
assert windows[0]["toBlock"] == 1_008_999
|
||||
assert windows[1]["fromBlock"] == 1_009_000
|
||||
assert windows[1]["toBlock"] == 1_017_999
|
||||
assert windows[2]["fromBlock"] == 1_018_000
|
||||
assert windows[2]["toBlock"] == 1_025_000
|
||||
# No window exceeds the chunk size — that's what RPC providers reject.
|
||||
for win in windows:
|
||||
assert win["toBlock"] - win["fromBlock"] + 1 <= 9_000
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_stops_when_limit_hit_mid_walk(
|
||||
self,
|
||||
funding_tracer: FundingTracer,
|
||||
mock_polygon_client: MagicMock,
|
||||
) -> None:
|
||||
"""Walking should stop as soon as ``limit`` matches are gathered."""
|
||||
mock_w3 = MagicMock()
|
||||
# First chunk yields 5 logs, more than the limit, so subsequent chunks
|
||||
# must not be queried.
|
||||
mock_w3.eth.get_logs = AsyncMock(return_value=[MagicMock() for _ in range(5)])
|
||||
mock_polygon_client._w3 = mock_w3
|
||||
|
||||
result = await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
limit=2,
|
||||
from_block=1_000_000,
|
||||
to_block=1_025_000,
|
||||
)
|
||||
|
||||
assert len(result) == 2
|
||||
mock_w3.eth.get_logs.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_skips_failing_chunk(
|
||||
self,
|
||||
funding_tracer: FundingTracer,
|
||||
mock_polygon_client: MagicMock,
|
||||
) -> None:
|
||||
"""A flaky chunk must not abort the whole trace — we move on."""
|
||||
mock_w3 = MagicMock()
|
||||
good_log = MagicMock()
|
||||
responses: list[Any] = [
|
||||
RuntimeError("RPC hiccup"),
|
||||
[good_log],
|
||||
]
|
||||
|
||||
async def fake_get_logs(_params: dict[str, Any]) -> list[Any]:
|
||||
outcome = responses.pop(0)
|
||||
if isinstance(outcome, BaseException):
|
||||
raise outcome
|
||||
return outcome
|
||||
|
||||
mock_w3.eth.get_logs = AsyncMock(side_effect=fake_get_logs)
|
||||
mock_polygon_client._w3 = mock_w3
|
||||
|
||||
result = await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
from_block=1_000_000,
|
||||
to_block=1_018_000, # forces 3 chunks; we exercise chunks 1+2
|
||||
)
|
||||
|
||||
# The error chunk is skipped; the second chunk contributes one log.
|
||||
assert result == [dict(good_log)]
|
||||
assert mock_w3.eth.get_logs.call_count >= 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_resolves_latest_via_block_number(
|
||||
self,
|
||||
funding_tracer: FundingTracer,
|
||||
mock_polygon_client: MagicMock,
|
||||
) -> None:
|
||||
"""``to_block='latest'`` should resolve via ``eth.block_number``.
|
||||
|
||||
And ``from_block=0`` should not become a full-history scan — it must
|
||||
be clamped to ``latest - max_lookback_blocks``.
|
||||
"""
|
||||
|
||||
async def _block_number_coro() -> int:
|
||||
return 5_000
|
||||
|
||||
mock_eth = MagicMock()
|
||||
mock_eth.get_logs = AsyncMock(return_value=[])
|
||||
# Property-style awaitable: web3.py exposes block_number as a property
|
||||
# returning a coroutine, so each access must yield a fresh awaitable.
|
||||
type(mock_eth).block_number = property( # type: ignore[misc]
|
||||
lambda _self: _block_number_coro()
|
||||
)
|
||||
mock_w3 = MagicMock()
|
||||
mock_w3.eth = mock_eth
|
||||
mock_polygon_client._w3 = mock_w3
|
||||
|
||||
await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
)
|
||||
|
||||
# block_number=5000 < chunk_size, so it's one chunk that bottoms at 0.
|
||||
mock_eth.get_logs.assert_called_once()
|
||||
call_args = mock_eth.get_logs.call_args[0][0]
|
||||
assert call_args["fromBlock"] == 0
|
||||
assert call_args["toBlock"] == 5_000
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_breaks_on_pruned_history(
|
||||
self,
|
||||
funding_tracer: FundingTracer,
|
||||
mock_polygon_client: MagicMock,
|
||||
) -> None:
|
||||
"""A pruned-history error must short-circuit the whole walk.
|
||||
|
||||
Public Polygon RPCs prune log history. Once we walk past the cutoff,
|
||||
every subsequent chunk will raise the same error — keep walking and
|
||||
we just burn quota on guaranteed failures. The first such error must
|
||||
end the walk and return whatever we already collected.
|
||||
"""
|
||||
mock_w3 = MagicMock()
|
||||
good_log = MagicMock()
|
||||
|
||||
responses: list[Any] = [
|
||||
[good_log],
|
||||
RuntimeError(
|
||||
"{'code': -32701, 'message': 'History has been pruned for "
|
||||
"this block. To remove restrictions, order a dedicated full "
|
||||
"node here: https://www.allnodes.com/pol/host'}"
|
||||
),
|
||||
# If the early-break logic is missing, this third chunk would
|
||||
# also be requested. The test asserts it isn't.
|
||||
[MagicMock()],
|
||||
]
|
||||
|
||||
async def fake_get_logs(_params: dict[str, Any]) -> list[Any]:
|
||||
outcome = responses.pop(0)
|
||||
if isinstance(outcome, BaseException):
|
||||
raise outcome
|
||||
return outcome
|
||||
|
||||
mock_w3.eth.get_logs = AsyncMock(side_effect=fake_get_logs)
|
||||
mock_polygon_client._w3 = mock_w3
|
||||
|
||||
# 3 chunks total. The pruned error fires on chunk #2; chunk #3 must
|
||||
# never be issued.
|
||||
result = await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
from_block=1_000_000,
|
||||
to_block=1_027_000,
|
||||
)
|
||||
|
||||
assert result == [dict(good_log)]
|
||||
assert mock_w3.eth.get_logs.call_count == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_default_lookback_fits_pruned_horizon(
|
||||
self,
|
||||
) -> None:
|
||||
"""Default ``max_lookback_blocks`` must stay inside what public RPCs serve.
|
||||
|
||||
publicnode prunes after ~100k blocks. If we default to 1.3M, every
|
||||
funding trace blows through the archive horizon and produces nothing
|
||||
but pruned-history warnings. Pin the default at <= 100k as a
|
||||
regression guard.
|
||||
"""
|
||||
from polymarket_insider_tracker.profiler.funding import (
|
||||
DEFAULT_MAX_LOOKBACK_BLOCKS,
|
||||
)
|
||||
|
||||
assert DEFAULT_MAX_LOOKBACK_BLOCKS <= 100_000
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_transfer_logs_topic_is_0x_prefixed(
|
||||
self,
|
||||
funding_tracer: FundingTracer,
|
||||
mock_polygon_client: MagicMock,
|
||||
) -> None:
|
||||
"""The Transfer event topic passed to ``eth_getLogs`` must begin with ``0x``.
|
||||
|
||||
``HexBytes.hex()`` returns a bare hex string. publicnode tolerates
|
||||
that, but stricter providers like drpc (our fallback) reject it with
|
||||
``invalid argument 0: hex string without 0x prefix`` and every chunk
|
||||
in the trace fails. This guards against regressing back to the
|
||||
bare-hex form.
|
||||
"""
|
||||
mock_w3 = MagicMock()
|
||||
mock_w3.eth.get_logs = AsyncMock(return_value=[])
|
||||
mock_polygon_client._w3 = mock_w3
|
||||
|
||||
await funding_tracer._get_transfer_logs(
|
||||
to_address=TEST_WALLET,
|
||||
token_address=USDC_BRIDGED,
|
||||
from_block=1,
|
||||
to_block=8_000,
|
||||
)
|
||||
|
||||
topics = mock_w3.eth.get_logs.call_args[0][0]["topics"]
|
||||
assert topics[0].startswith("0x")
|
||||
# And the topic also has to be 32 bytes (64 hex chars) as required by
|
||||
# the JSON-RPC spec.
|
||||
assert len(topics[0]) == 2 + 64
|
||||
# The padded `to` topic was already 0x-prefixed; double-check that
|
||||
# didn't regress either.
|
||||
assert topics[2].startswith("0x")
|
||||
|
||||
|
||||
class TestLogToFundingTransfer:
|
||||
"""Tests for _log_to_funding_transfer method."""
|
||||
|
||||
@@ -6,14 +6,13 @@ wallet_profiles and funding_transfers tables when fresh wallets are detected.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from datetime import UTC, datetime
|
||||
from decimal import Decimal
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from polymarket_insider_tracker.config import Settings
|
||||
from polymarket_insider_tracker.detector.models import FreshWalletSignal
|
||||
@@ -83,9 +82,7 @@ async def db_manager(async_engine):
|
||||
manager._sync_engine = None
|
||||
manager._async_engine = async_engine
|
||||
manager._sync_session_factory = None
|
||||
manager._async_session_factory = async_sessionmaker(
|
||||
bind=async_engine, expire_on_commit=False
|
||||
)
|
||||
manager._async_session_factory = async_sessionmaker(bind=async_engine, expire_on_commit=False)
|
||||
return manager
|
||||
|
||||
|
||||
@@ -266,9 +263,7 @@ class TestPipelinePersistence:
|
||||
|
||||
# Use a broken db_manager that raises on get_async_session
|
||||
broken_db = MagicMock()
|
||||
broken_db.get_async_session = MagicMock(
|
||||
side_effect=Exception("DB connection failed")
|
||||
)
|
||||
broken_db.get_async_session = MagicMock(side_effect=Exception("DB connection failed"))
|
||||
pipeline._db_manager = broken_db
|
||||
|
||||
fresh_signal = FreshWalletSignal(
|
||||
|
||||
Reference in New Issue
Block a user