72 lines
3.3 KiB
Markdown
72 lines
3.3 KiB
Markdown
|
# wide/ — bulk subgraph edge scanner
|
|||
|
|
|
|||
|
|
Find wallets with a real edge across **all** of Polymarket (~1.76M traders,
|
|||
|
|
268k conditions) by bulk-ingesting the on-chain subgraph into a local DuckDB
|
|||
|
|
and ranking with SQL — instead of per-wallet API calls (which cap out at a few
|
|||
|
|
hundred wallets and rate-limit).
|
|||
|
|
|
|||
|
|
## Why a local DB, not the API
|
|||
|
|
|
|||
|
|
The Goldsky orderbook subgraph **times out on any `orderBy`** over a
|
|||
|
|
non-indexed field (`numTrades`, `scaledProfit`, …). You cannot ask it for "top
|
|||
|
|
wallets." The only scalable pattern is: cursor-paginate every row by `id` (the
|
|||
|
|
indexed key), land it locally, and rank in DuckDB. That constraint *is* the
|
|||
|
|
architecture.
|
|||
|
|
|
|||
|
|
## Why the win rate here is honest
|
|||
|
|
|
|||
|
|
The data-api hides losers: Polymarket only redeems winning shares, so losing
|
|||
|
|
positions sit unredeemed and never enter `/closed-positions`. Measuring win
|
|||
|
|
rate there reads ~90% when the truth is ~50% (see ../FINDINGS.md). The subgraph
|
|||
|
|
records a `marketPosition` for **every** buy regardless of redemption, so the
|
|||
|
|
survivorship bias structurally does not exist in this data.
|
|||
|
|
|
|||
|
|
## What "edge" means
|
|||
|
|
|
|||
|
|
A high win rate is not edge — you can hit 90% by only buying 95¢ favorites.
|
|||
|
|
Edge is **beating the prices you paid**:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
p = valueBought / quantityBought (entry price, 0..1)
|
|||
|
|
won = the outcome you held paid out
|
|||
|
|
z = (wins − Σp) / sqrt(Σ p(1−p)) standard deviations above odds-implied
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`z` high over enough bets, on a wallet that isn't a market-maker, is the real
|
|||
|
|
signal (the same one ../insider.py computes, here over the entire market).
|
|||
|
|
|
|||
|
|
## Pipeline
|
|||
|
|
|
|||
|
|
| step | script | source | notes |
|
|||
|
|
|------|--------|--------|-------|
|
|||
|
|
| 1 | `ingest.py conditions` | subgraph | resolution + payoutNumerators → winning outcome |
|
|||
|
|
| 2 | `gamma_tokens.py` | Gamma | token_id → (condition, outcome_index); subgraph's `outcomeIndex` is null |
|
|||
|
|
| 3 | `ingest.py accounts` | subgraph | `numTrades` (market-maker filter), `creationTimestamp` (freshness) |
|
|||
|
|
| 4 | `ingest.py market_positions` | subgraph | the heavy table: entry price + win/loss per bet |
|
|||
|
|
| 5 | `score.py` | DuckDB | z, true win rate, profit + FDR + out-of-sample |
|
|||
|
|
|
|||
|
|
All ingests are **resumable** (per-table `id` cursor in `_cursor`), so a long
|
|||
|
|
run can be stopped and restarted. Tables join in `edge.sql`.
|
|||
|
|
|
|||
|
|
## Guardrails (so a 1.76M-wallet scan doesn't just surface luck)
|
|||
|
|
|
|||
|
|
- **min-n + market-maker cap.** A 300k-trade grinder posts huge z with no
|
|||
|
|
information. Require ≥30 resolved bets and cap `numTrades`.
|
|||
|
|
- **Benjamini–Hochberg FDR.** Scan 100k wallets and thousands clear z>3 by
|
|||
|
|
chance (look-elsewhere effect). `score.py` reports how many survive 5% FDR.
|
|||
|
|
- **Out-of-sample.** `score.py --cutoff YYYY-MM-DD` selects wallets on bets
|
|||
|
|
resolved before the date, then measures the *same* wallets forward. Edge that
|
|||
|
|
is real persists; edge that is curve-fit reverts to z≈0 — which is what every
|
|||
|
|
strategy in ../FINDINGS.md did. **Do not size up on in-sample z.**
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pip install duckdb
|
|||
|
|
python3 ingest.py conditions accounts # small tables
|
|||
|
|
python3 gamma_tokens.py # token→outcome map
|
|||
|
|
python3 ingest.py -p market_positions # heavy (~53M rows); parallel + resumable
|
|||
|
|
python3 score.py --min-n 15 --max-trades 5000 --top 40
|
|||
|
|
python3 score.py --cutoff 2026-04-30 --min-n 15 # in-sample vs forward
|
|||
|
|
```
|