mirror of
https://github.com/jaxperro/winning-wallet-finder.git
synced 2026-07-27 15:57:47 +00:00
Add live/ skilled-wallet scanner + cache; document clean OOS finding
live/: operationalizes the LBS/Yale "skilled ~3%" result against the live data-api. Enumerate recent liquid markets -> top traders -> candidate pool; cache every wallet's resolved bets once in DuckDB (~26k wallets / 12.5M bets, keyed by per-bet resolution time so any cutoff re-scores in seconds); 5-gate skill funnel (n>=15, z>0, BH-FDR, split-half OOS, MM/bot cap); dashboard + daily refresh. Key finding: copying the high-win-rate "favorite-rider" cohort looks +23.6% in-sample but loses -7.4% once selected on pre-June-1 data only (99% -> 68% win rate) — selection bias, reproducing the paper's "lucky winners revert" result on live data. Win rate != edge, again. wide/: bulk subgraph->DuckDB scanner (survivorship-bias-free over all wallets), but the public subgraph is frozen at Jan 2026 -> historical tool only. Large local data (*.duckdb, candidates.json, *_scored.json, history/) gitignored. README + FINDINGS updated with the current logic and the clean result. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jaxperro
@@ -0,0 +1,71 @@
|
||||
# wide/ — bulk subgraph edge scanner
|
||||
|
||||
Find wallets with a real edge across **all** of Polymarket (~1.76M traders,
|
||||
268k conditions) by bulk-ingesting the on-chain subgraph into a local DuckDB
|
||||
and ranking with SQL — instead of per-wallet API calls (which cap out at a few
|
||||
hundred wallets and rate-limit).
|
||||
|
||||
## Why a local DB, not the API
|
||||
|
||||
The Goldsky orderbook subgraph **times out on any `orderBy`** over a
|
||||
non-indexed field (`numTrades`, `scaledProfit`, …). You cannot ask it for "top
|
||||
wallets." The only scalable pattern is: cursor-paginate every row by `id` (the
|
||||
indexed key), land it locally, and rank in DuckDB. That constraint *is* the
|
||||
architecture.
|
||||
|
||||
## Why the win rate here is honest
|
||||
|
||||
The data-api hides losers: Polymarket only redeems winning shares, so losing
|
||||
positions sit unredeemed and never enter `/closed-positions`. Measuring win
|
||||
rate there reads ~90% when the truth is ~50% (see ../FINDINGS.md). The subgraph
|
||||
records a `marketPosition` for **every** buy regardless of redemption, so the
|
||||
survivorship bias structurally does not exist in this data.
|
||||
|
||||
## What "edge" means
|
||||
|
||||
A high win rate is not edge — you can hit 90% by only buying 95¢ favorites.
|
||||
Edge is **beating the prices you paid**:
|
||||
|
||||
```
|
||||
p = valueBought / quantityBought (entry price, 0..1)
|
||||
won = the outcome you held paid out
|
||||
z = (wins − Σp) / sqrt(Σ p(1−p)) standard deviations above odds-implied
|
||||
```
|
||||
|
||||
`z` high over enough bets, on a wallet that isn't a market-maker, is the real
|
||||
signal (the same one ../insider.py computes, here over the entire market).
|
||||
|
||||
## Pipeline
|
||||
|
||||
| step | script | source | notes |
|
||||
|------|--------|--------|-------|
|
||||
| 1 | `ingest.py conditions` | subgraph | resolution + payoutNumerators → winning outcome |
|
||||
| 2 | `gamma_tokens.py` | Gamma | token_id → (condition, outcome_index); subgraph's `outcomeIndex` is null |
|
||||
| 3 | `ingest.py accounts` | subgraph | `numTrades` (market-maker filter), `creationTimestamp` (freshness) |
|
||||
| 4 | `ingest.py market_positions` | subgraph | the heavy table: entry price + win/loss per bet |
|
||||
| 5 | `score.py` | DuckDB | z, true win rate, profit + FDR + out-of-sample |
|
||||
|
||||
All ingests are **resumable** (per-table `id` cursor in `_cursor`), so a long
|
||||
run can be stopped and restarted. Tables join in `edge.sql`.
|
||||
|
||||
## Guardrails (so a 1.76M-wallet scan doesn't just surface luck)
|
||||
|
||||
- **min-n + market-maker cap.** A 300k-trade grinder posts huge z with no
|
||||
information. Require ≥30 resolved bets and cap `numTrades`.
|
||||
- **Benjamini–Hochberg FDR.** Scan 100k wallets and thousands clear z>3 by
|
||||
chance (look-elsewhere effect). `score.py` reports how many survive 5% FDR.
|
||||
- **Out-of-sample.** `score.py --cutoff YYYY-MM-DD` selects wallets on bets
|
||||
resolved before the date, then measures the *same* wallets forward. Edge that
|
||||
is real persists; edge that is curve-fit reverts to z≈0 — which is what every
|
||||
strategy in ../FINDINGS.md did. **Do not size up on in-sample z.**
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
pip install duckdb
|
||||
python3 ingest.py conditions accounts # small tables
|
||||
python3 gamma_tokens.py # token→outcome map
|
||||
python3 ingest.py -p market_positions # heavy (~53M rows); parallel + resumable
|
||||
python3 score.py --min-n 15 --max-trades 5000 --top 40
|
||||
python3 score.py --cutoff 2026-04-30 --min-n 15 # in-sample vs forward
|
||||
```
|
||||
@@ -0,0 +1,83 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build token -> (condition, outcome_index, winner) from the CLOB /markets feed.
|
||||
|
||||
Gamma offset-paginates and 422s past ~10k markets; the subgraph's outcomeIndex
|
||||
is null. CLOB /markets cursor-paginates the full market set (no offset cap),
|
||||
1000 per page, and each token carries `winner` directly — so win/loss comes
|
||||
straight from here instead of parsing payoutNumerators.
|
||||
|
||||
python3 clob_tokens.py # page all markets, fill market_data
|
||||
"""
|
||||
|
||||
import json
|
||||
import ssl
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
import duckdb
|
||||
|
||||
DB = "pmkt.duckdb"
|
||||
CLOB = "https://clob.polymarket.com/markets"
|
||||
_CTX = ssl._create_unverified_context()
|
||||
END = "LTE=" # CLOB's end-of-pagination cursor (base64 of -1)
|
||||
|
||||
|
||||
def get(cursor, retries=6):
|
||||
url = CLOB + (f"?next_cursor={cursor}" if cursor else "")
|
||||
delay = 1.0
|
||||
for attempt in range(retries):
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
|
||||
return json.loads(urllib.request.urlopen(req, timeout=40, context=_CTX).read())
|
||||
except Exception:
|
||||
if attempt == retries - 1:
|
||||
raise
|
||||
time.sleep(delay); delay = min(delay * 2, 20)
|
||||
|
||||
|
||||
def main():
|
||||
con = duckdb.connect(DB)
|
||||
con.execute("""CREATE TABLE IF NOT EXISTS market_data (
|
||||
token_id TEXT PRIMARY KEY, condition_id TEXT,
|
||||
outcome_index INT, winner BOOLEAN);""")
|
||||
con.execute("CREATE TABLE IF NOT EXISTS _cursor (table_name TEXT PRIMARY KEY, last_id TEXT);")
|
||||
row = con.execute("SELECT last_id FROM _cursor WHERE table_name='market_data#clob'").fetchone()
|
||||
cursor = row[0] if row else "" # resume from saved CLOB cursor
|
||||
total, pages, t0 = 0, 0, time.time()
|
||||
if cursor:
|
||||
print(f"resuming token map from cursor {cursor}", flush=True)
|
||||
while cursor != END:
|
||||
d = get(cursor)
|
||||
rows = []
|
||||
for m in d.get("data", []):
|
||||
cond = m.get("condition_id")
|
||||
toks = m.get("tokens") or []
|
||||
if not cond:
|
||||
continue
|
||||
for idx, t in enumerate(toks): # tokens ordered by outcome
|
||||
tid = t.get("token_id")
|
||||
if tid:
|
||||
rows.append((str(tid), cond, idx, bool(t.get("winner"))))
|
||||
pages += 1
|
||||
nxt = d.get("next_cursor")
|
||||
if rows:
|
||||
con.execute("BEGIN TRANSACTION")
|
||||
con.executemany("INSERT OR IGNORE INTO market_data VALUES (?,?,?,?)", rows)
|
||||
# checkpoint the NEXT cursor so a resume continues past this page
|
||||
con.execute("INSERT OR REPLACE INTO _cursor VALUES ('market_data#clob', ?)",
|
||||
[nxt or cursor])
|
||||
con.execute("COMMIT")
|
||||
total += len(rows)
|
||||
if not nxt or nxt == cursor: # safety: no progress
|
||||
break
|
||||
cursor = nxt
|
||||
if pages % 25 == 0:
|
||||
print(f" {pages} pages · {total:,} tokens ({total/max(1e-9,time.time()-t0):,.0f}/s)", flush=True)
|
||||
cnt = con.execute("SELECT count(*) FROM market_data").fetchone()[0]
|
||||
won = con.execute("SELECT count(*) FROM market_data WHERE winner").fetchone()[0]
|
||||
print(f"done — {cnt:,} token rows ({won:,} winning) over {pages} pages", flush=True)
|
||||
con.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,47 @@
|
||||
-- Per-wallet edge over RESOLVED markets, computed entirely in DuckDB.
|
||||
--
|
||||
-- The join chain: market_positions (a wallet's buy in one outcome token)
|
||||
-- -> market_data (token -> condition + outcome_index)
|
||||
-- -> conditions (resolution + payoutNumerators -> which outcome won).
|
||||
--
|
||||
-- Why this beats the data-api: market_positions records a buy whether or not
|
||||
-- the wallet redeemed, so losers are NOT hidden. The survivorship bias that
|
||||
-- makes /closed-positions read 90% (truly 48%) does not exist here.
|
||||
--
|
||||
-- entry price p = valueBought / quantityBought (USDC 6dp / shares 6dp -> 0..1)
|
||||
-- won = payoutNumerators[outcome_index] != 0
|
||||
-- z = (wins - Σp) / sqrt(Σ p(1-p)) -- wins above what odds implied
|
||||
--
|
||||
-- :cutoff_ts binds an out-of-sample boundary. Pass 0 to score everything.
|
||||
|
||||
WITH bet AS (
|
||||
SELECT
|
||||
mp.user_id,
|
||||
c.resolution_ts,
|
||||
LEAST(0.999, GREATEST(0.001,
|
||||
mp.val_bought::DOUBLE / mp.qty_bought)) AS p,
|
||||
CASE WHEN md.winner THEN 1 ELSE 0 END AS won
|
||||
FROM market_positions mp
|
||||
JOIN market_data md ON md.token_id = mp.token_id
|
||||
JOIN conditions c ON c.id = md.condition_id
|
||||
WHERE mp.qty_bought > 0
|
||||
AND c.resolution_ts > 0
|
||||
)
|
||||
SELECT
|
||||
b.user_id,
|
||||
count(*) AS n,
|
||||
sum(b.won) AS wins,
|
||||
round(sum(b.p), 1) AS exp_wins,
|
||||
round(100.0 * sum(b.won) / count(*), 1) AS win_rate,
|
||||
round((sum(b.won) - sum(b.p))
|
||||
/ sqrt(nullif(sum(b.p * (1 - b.p)), 0)), 2) AS z,
|
||||
round(avg(b.p), 3) AS avg_entry,
|
||||
a.scaled_profit AS profit,
|
||||
a.scaled_volume AS volume,
|
||||
a.creation_ts
|
||||
FROM bet b
|
||||
LEFT JOIN accounts a ON a.id = b.user_id
|
||||
WHERE b.resolution_ts <= :cutoff_ts OR :cutoff_ts = 0
|
||||
GROUP BY b.user_id, a.scaled_profit, a.scaled_volume, a.creation_ts
|
||||
HAVING count(*) >= :min_n
|
||||
ORDER BY z DESC;
|
||||
+217
@@ -0,0 +1,217 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Bulk-ingest the Polymarket subgraph into a local DuckDB (pmkt.duckdb).
|
||||
|
||||
Each table is cursor-paginated by id and resumable: we checkpoint the last
|
||||
id seen, and re-running continues where it left off (INSERT OR IGNORE makes
|
||||
overlap harmless). All ranking/scoring happens later in SQL — see edge.sql.
|
||||
|
||||
python3 ingest.py conditions market_data accounts # the small tables
|
||||
python3 ingest.py market_positions # the heavy table
|
||||
python3 ingest.py all
|
||||
"""
|
||||
|
||||
import queue
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
import duckdb
|
||||
|
||||
import subgraph as sg
|
||||
|
||||
DB = "pmkt.duckdb"
|
||||
BATCH = 5000
|
||||
|
||||
SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS conditions (
|
||||
id TEXT PRIMARY KEY, resolution_ts BIGINT,
|
||||
payout_num TEXT, payout_den BIGINT, slots INT);
|
||||
CREATE TABLE IF NOT EXISTS market_data (
|
||||
token_id TEXT PRIMARY KEY, condition_id TEXT, outcome_index INT);
|
||||
CREATE TABLE IF NOT EXISTS accounts (
|
||||
id TEXT PRIMARY KEY, num_trades BIGINT, creation_ts BIGINT,
|
||||
scaled_profit DOUBLE, scaled_volume DOUBLE);
|
||||
CREATE TABLE IF NOT EXISTS market_positions (
|
||||
id TEXT PRIMARY KEY, user_id TEXT, token_id TEXT,
|
||||
qty_bought HUGEINT, val_bought HUGEINT, net_qty HUGEINT);
|
||||
CREATE TABLE IF NOT EXISTS _cursor (table_name TEXT PRIMARY KEY, last_id TEXT);
|
||||
"""
|
||||
|
||||
# entity -> (graphql_fields, where_clause, row_mapper, target_table, columns)
|
||||
def _i(x, d=0):
|
||||
try:
|
||||
return int(x)
|
||||
except (TypeError, ValueError):
|
||||
return d
|
||||
|
||||
def _f(x, d=0.0):
|
||||
try:
|
||||
return float(x)
|
||||
except (TypeError, ValueError):
|
||||
return d
|
||||
|
||||
SPECS = {
|
||||
"conditions": dict(
|
||||
entity="conditions",
|
||||
fields="id resolutionTimestamp payoutNumerators payoutDenominator outcomeSlotCount",
|
||||
where="",
|
||||
table="conditions",
|
||||
cols=("id", "resolution_ts", "payout_num", "payout_den", "slots"),
|
||||
row=lambda c: (c["id"], _i(c.get("resolutionTimestamp")),
|
||||
",".join(c.get("payoutNumerators") or []),
|
||||
_i(c.get("payoutDenominator")), _i(c.get("outcomeSlotCount"))),
|
||||
),
|
||||
# NOTE: market_data (token -> outcome) comes from Gamma, not the subgraph —
|
||||
# the subgraph's marketData.outcomeIndex is null. See gamma_tokens.py.
|
||||
"accounts": dict(
|
||||
entity="accounts",
|
||||
fields="id numTrades creationTimestamp scaledProfit scaledCollateralVolume",
|
||||
where="",
|
||||
table="accounts",
|
||||
cols=("id", "num_trades", "creation_ts", "scaled_profit", "scaled_volume"),
|
||||
row=lambda a: (a["id"], _i(a.get("numTrades")), _i(a.get("creationTimestamp")),
|
||||
_f(a.get("scaledProfit")), _f(a.get("scaledCollateralVolume"))),
|
||||
),
|
||||
# marketPosition.id == user_address (0x + 40 hex = 42 chars) + token_id
|
||||
# (decimal). We split it out instead of selecting the nested user/market
|
||||
# objects, which the subgraph errors on when `market` is null.
|
||||
"market_positions": dict(
|
||||
entity="marketPositions",
|
||||
fields="id quantityBought valueBought netQuantity",
|
||||
where="",
|
||||
table="market_positions",
|
||||
cols=("id", "user_id", "token_id", "qty_bought", "val_bought", "net_qty"),
|
||||
row=lambda p: (p["id"], p["id"][:42], p["id"][42:],
|
||||
_i(p.get("quantityBought")), _i(p.get("valueBought")),
|
||||
_i(p.get("netQuantity"))),
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def ingest(con, name, limit=0):
|
||||
spec = SPECS[name]
|
||||
last = con.execute("SELECT last_id FROM _cursor WHERE table_name=?", [name]).fetchone()
|
||||
start = last[0] if last else ""
|
||||
placeholders = ",".join("?" * len(spec["cols"]))
|
||||
insert = (f"INSERT OR IGNORE INTO {spec['table']} "
|
||||
f"({','.join(spec['cols'])}) VALUES ({placeholders})")
|
||||
buf, total, t0, last_seen = [], 0, time.time(), start
|
||||
|
||||
def flush(cursor_id):
|
||||
nonlocal buf
|
||||
if buf:
|
||||
con.executemany(insert, buf)
|
||||
buf = []
|
||||
con.execute("INSERT OR REPLACE INTO _cursor VALUES (?, ?)", [name, cursor_id])
|
||||
|
||||
print(f"[{name}] resuming from id={start[:14] or '(start)'}", flush=True)
|
||||
for row in sg.paginate(spec["entity"], spec["fields"], where=spec["where"], start_id=start):
|
||||
buf.append(spec["row"](row))
|
||||
total += 1
|
||||
last_seen = row["id"]
|
||||
if len(buf) >= BATCH:
|
||||
flush(last_seen)
|
||||
rate = total / max(1e-9, time.time() - t0)
|
||||
print(f"[{name}] {total:>9,} ({rate:,.0f}/s)", flush=True)
|
||||
if limit and total >= limit:
|
||||
break
|
||||
flush(last_seen) # remaining buffer + advance cursor to the last id seen
|
||||
cnt = con.execute(f"SELECT count(*) FROM {spec['table']}").fetchone()[0]
|
||||
print(f"[{name}] done — {total:,} pulled this run, {cnt:,} rows total", flush=True)
|
||||
|
||||
|
||||
def ingest_parallel(con, name, shards=16):
|
||||
"""Page `shards` id-ranges concurrently; workers fetch and enqueue, this
|
||||
(single) thread writes to DuckDB and checkpoints each shard's cursor.
|
||||
~`shards`× the sequential throughput, and fully resumable per shard."""
|
||||
spec = SPECS[name]
|
||||
bounds = sg.shard_bounds(shards)
|
||||
placeholders = ",".join("?" * len(spec["cols"]))
|
||||
insert = (f"INSERT OR IGNORE INTO {spec['table']} "
|
||||
f"({','.join(spec['cols'])}) VALUES ({placeholders})")
|
||||
starts = {}
|
||||
for i in range(shards):
|
||||
row = con.execute("SELECT last_id FROM _cursor WHERE table_name=?",
|
||||
[f"{name}#{i:02d}"]).fetchone()
|
||||
starts[i] = row[0] if row else ""
|
||||
q = queue.Queue(maxsize=400)
|
||||
DONE = object()
|
||||
|
||||
def worker(i):
|
||||
try:
|
||||
for rows, last in sg.paginate_pages(spec["entity"], spec["fields"],
|
||||
lo=bounds[i], hi=bounds[i + 1],
|
||||
start_id=starts[i]):
|
||||
q.put((i, [spec["row"](r) for r in rows], last))
|
||||
except Exception as e:
|
||||
q.put((i, "ERR", str(e)[:120]))
|
||||
q.put((i, DONE, None))
|
||||
|
||||
for i in range(shards):
|
||||
threading.Thread(target=worker, args=(i,), daemon=True).start()
|
||||
|
||||
# DuckDB fsyncs per commit, so committing each page caps us at the writer
|
||||
# (~380/s) while fetch concurrency does ~4,600/s. Buffer many pages and
|
||||
# commit in one transaction to amortize the fsync.
|
||||
COMMIT_ROWS = 25000
|
||||
finished, total, t0 = 0, 0, time.time()
|
||||
buf, shard_last = [], {}
|
||||
|
||||
def flush():
|
||||
nonlocal buf
|
||||
if not buf:
|
||||
return
|
||||
con.execute("BEGIN TRANSACTION")
|
||||
con.executemany(insert, buf)
|
||||
for sh, lid in shard_last.items():
|
||||
con.execute("INSERT OR REPLACE INTO _cursor VALUES (?, ?)",
|
||||
[f"{name}#{sh:02d}", lid])
|
||||
con.execute("COMMIT")
|
||||
buf = []
|
||||
|
||||
print(f"[{name}] {shards} parallel shards", flush=True)
|
||||
while finished < shards:
|
||||
i, payload, last = q.get()
|
||||
if payload is DONE:
|
||||
finished += 1
|
||||
continue
|
||||
if payload == "ERR":
|
||||
print(f"[{name}] shard {i:02d} error: {last}", flush=True)
|
||||
continue
|
||||
buf.extend(payload)
|
||||
shard_last[i] = last
|
||||
total += len(payload)
|
||||
if len(buf) >= COMMIT_ROWS:
|
||||
flush()
|
||||
rate = total / max(1e-9, time.time() - t0)
|
||||
print(f"[{name}] {total:>10,} ({rate:,.0f}/s, {finished}/{shards} shards done)",
|
||||
flush=True)
|
||||
flush()
|
||||
cnt = con.execute(f"SELECT count(*) FROM {spec['table']}").fetchone()[0]
|
||||
print(f"[{name}] done — {total:,} pulled this run, {cnt:,} rows total", flush=True)
|
||||
|
||||
|
||||
def main(argv):
|
||||
parallel = False
|
||||
if argv and argv[0] in ("-p", "--parallel"):
|
||||
parallel = True; argv = argv[1:]
|
||||
limit = 0
|
||||
if argv and argv[-1].isdigit(): # optional trailing row-limit (per table)
|
||||
limit = int(argv[-1]); argv = argv[:-1]
|
||||
targets = argv or ["all"]
|
||||
if targets == ["all"]:
|
||||
targets = ["conditions", "accounts", "market_positions"]
|
||||
con = duckdb.connect(DB)
|
||||
con.execute(SCHEMA)
|
||||
for t in targets:
|
||||
if t not in SPECS:
|
||||
print(f"unknown table: {t}", file=sys.stderr); continue
|
||||
if parallel and not limit:
|
||||
ingest_parallel(con, t)
|
||||
else:
|
||||
ingest(con, t, limit)
|
||||
con.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1:])
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/bin/bash
|
||||
# Massive ingest of the wallet data we still need, from the FAST subgraph
|
||||
# (accounts + market_positions). The CLOB token map is already sufficient
|
||||
# (1.27M tokens covering every resolved condition), so it's NOT re-run here —
|
||||
# run `python3 clob_tokens.py` separately if you want to refresh it.
|
||||
#
|
||||
# Both steps are parallel (16 id-shards) and resumable (per-shard cursors), so
|
||||
# killing and re-running continues from the last checkpoint.
|
||||
#
|
||||
# nohup ./run_full.sh > run_full.log 2>&1 < /dev/null & disown
|
||||
# tail -f run_full.log
|
||||
set -u
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
until python3 -c "import duckdb;duckdb.connect('pmkt.duckdb',read_only=True).close()" 2>/dev/null; do
|
||||
sleep 5
|
||||
done
|
||||
|
||||
echo "[run] $(date '+%F %T') 1/2 accounts (parallel, resumable) …"
|
||||
python3 ingest.py -p accounts
|
||||
|
||||
echo "[run] $(date '+%F %T') 2/2 market_positions (parallel, resumable, the big one) …"
|
||||
python3 ingest.py -p market_positions
|
||||
|
||||
echo "[run] $(date '+%F %T') DONE"
|
||||
python3 - <<'PY'
|
||||
import duckdb
|
||||
c = duckdb.connect("pmkt.duckdb", read_only=True)
|
||||
for t in ("conditions", "market_data", "accounts", "market_positions"):
|
||||
print(f" {t:18} {c.execute('select count(*) from '+t).fetchone()[0]:,}")
|
||||
PY
|
||||
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Rank wallets by edge from the ingested DuckDB — with the guardrails that
|
||||
stop a massive scan from just surfacing luck.
|
||||
|
||||
Edge metric: z = (wins - Σp)/√Σp(1-p), wins above what entry odds implied.
|
||||
A high win rate alone is meaningless (buy 95¢ favorites -> 90% wins, no edge),
|
||||
and on this data win rate isn't even biased-high the way the data-api is.
|
||||
|
||||
Guardrails:
|
||||
* min_n resolved bets and a market-maker cap on num_trades (a 300k-trade
|
||||
grinder posts huge z with no information — see FINDINGS.md / bjprolo).
|
||||
* Benjamini-Hochberg FDR: scan 100k wallets and thousands clear z>3 by
|
||||
chance. We report how many survive a 5% false-discovery rate.
|
||||
* Out-of-sample: --cutoff scores wallets on bets resolved on/before a date,
|
||||
then measures the SAME wallets forward. Edge that's real persists; edge
|
||||
that's curve-fit (every strategy we tested) reverts to z~0 / 50%.
|
||||
|
||||
python3 score.py --min-n 30 --max-trades 5000 --top 40
|
||||
python3 score.py --cutoff 2026-04-30 --min-n 30 # in-sample vs forward
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import math
|
||||
import time
|
||||
|
||||
import duckdb
|
||||
|
||||
DB = "pmkt.duckdb"
|
||||
|
||||
|
||||
def norm_sf(z):
|
||||
"""One-sided P(Z > z): the probability this z came from luck."""
|
||||
return 0.5 * math.erfc(z / math.sqrt(2)) if z is not None else 1.0
|
||||
|
||||
|
||||
def load_sql(cutoff_ts, min_n):
|
||||
sql = open("edge.sql").read()
|
||||
# these are ints we control (not user strings) -> safe to template
|
||||
return sql.replace(":cutoff_ts", str(int(cutoff_ts))).replace(":min_n", str(int(min_n)))
|
||||
|
||||
|
||||
def rank(con, cutoff_ts, min_n, max_n):
|
||||
rows = con.execute(load_sql(cutoff_ts, min_n)).fetchall()
|
||||
cols = [d[0] for d in con.description]
|
||||
out = []
|
||||
for r in rows:
|
||||
d = dict(zip(cols, r))
|
||||
# num_trades is null in this subgraph, so use the resolved-bet count as
|
||||
# the market-maker proxy: a wallet with thousands of bets is grinding a
|
||||
# systematic edge (bjprolo-style), not trading on information.
|
||||
if max_n and d["n"] > max_n:
|
||||
continue
|
||||
d["pval"] = norm_sf(d["z"])
|
||||
out.append(d)
|
||||
return out
|
||||
|
||||
|
||||
def bh_fdr(rows, q=0.05):
|
||||
"""Benjamini-Hochberg: how many discoveries survive a q false-discovery rate."""
|
||||
m = len(rows)
|
||||
if not m:
|
||||
return 0, 1.0
|
||||
ps = sorted(r["pval"] for r in rows)
|
||||
k = 0
|
||||
for i, p in enumerate(ps, 1):
|
||||
if p <= q * i / m:
|
||||
k = i
|
||||
thresh = ps[k - 1] if k else 0.0
|
||||
return k, thresh
|
||||
|
||||
|
||||
def to_ts(date_str):
|
||||
if not date_str:
|
||||
return 0
|
||||
return time.mktime(time.strptime(date_str, "%Y-%m-%d"))
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--min-n", type=int, default=15, help="min resolved bets")
|
||||
ap.add_argument("--max-n", type=int, default=3000,
|
||||
help="exclude wallets with more resolved bets (market-maker proxy); 0=off")
|
||||
ap.add_argument("--top", type=int, default=40)
|
||||
ap.add_argument("--cutoff", help="YYYY-MM-DD: in-sample/out-of-sample split")
|
||||
args = ap.parse_args()
|
||||
|
||||
con = duckdb.connect(DB, read_only=True)
|
||||
cutoff_ts = to_ts(args.cutoff)
|
||||
rows = rank(con, cutoff_ts, args.min_n, args.max_n)
|
||||
rows.sort(key=lambda r: (r["z"] is not None, r["z"]), reverse=True)
|
||||
|
||||
k, thresh = bh_fdr(rows)
|
||||
label = f"on/before {args.cutoff}" if args.cutoff else "all resolved bets"
|
||||
print(f"\nscored {len(rows):,} wallets (min_n={args.min_n}, "
|
||||
f"max_n={args.max_n or '∞'}) · {label}")
|
||||
print(f"Benjamini-Hochberg @5% FDR: {k:,} wallets survive (p ≤ {thresh:.1e}) "
|
||||
f"— the rest of the high-z tail is consistent with luck\n")
|
||||
|
||||
hdr = f"{'z':>6}{'p(luck)':>10}{'rec':>13}{'win%':>7}{'avgP':>7}{'volume':>12}{'profit':>11} wallet"
|
||||
print(hdr); print("-" * len(hdr))
|
||||
for r in rows[:args.top]:
|
||||
rec = f"{r['wins']}/{r['n']}(E{r['exp_wins']:.0f})"
|
||||
p = r["pval"]
|
||||
ps = "<1e-12" if p <= 0 else (f"{p:.1e}" if p < 1e-3 else f"{p:.3f}")
|
||||
star = " *" if p <= thresh and thresh > 0 else " "
|
||||
print(f"{r['z']:>6.1f}{ps:>10}{rec:>13}{r['win_rate']:>6.1f}%{r['avg_entry']:>7.2f}"
|
||||
f"{(r['volume'] or 0):>12,.0f}{(r['profit'] or 0):>11,.0f}{star}{r['user_id']}")
|
||||
|
||||
if args.cutoff:
|
||||
forward_oos(con, rows[:args.top], cutoff_ts, args.min_n)
|
||||
|
||||
|
||||
def forward_oos(con, picks, cutoff_ts, min_n):
|
||||
"""For the in-sample top picks, measure their record AFTER the cutoff."""
|
||||
print(f"\n{'='*70}\nOUT-OF-SAMPLE: same wallets, only bets resolved AFTER cutoff")
|
||||
print(f"{'='*70}")
|
||||
ids = [r["user_id"] for r in picks]
|
||||
if not ids:
|
||||
return
|
||||
# reuse the same join but flip the time filter and restrict to these wallets
|
||||
sql = open("edge.sql").read()
|
||||
sql = sql.replace("WHERE b.resolution_ts <= :cutoff_ts OR :cutoff_ts = 0",
|
||||
f"WHERE b.resolution_ts > {int(cutoff_ts)} "
|
||||
f"AND b.user_id IN ({','.join(repr(i) for i in ids)})")
|
||||
sql = sql.replace("HAVING count(*) >= :min_n", "HAVING count(*) >= 1")
|
||||
fwd = {r[0]: r for r in con.execute(sql).fetchall()}
|
||||
cols = [d[0] for d in con.description]
|
||||
zi, wi, ni, wri = cols.index("z"), cols.index("wins"), cols.index("n"), cols.index("win_rate")
|
||||
print(f"{'in-sample z':>12}{'fwd z':>8}{'fwd rec':>12}{'fwd win%':>9} wallet")
|
||||
for r in picks:
|
||||
f = fwd.get(r["user_id"])
|
||||
if f:
|
||||
fz = f"{f[zi]:.1f}" if f[zi] is not None else "n/a"
|
||||
print(f"{r['z']:>12.1f}{fz:>8}{f'{f[wi]}/{f[ni]}':>12}{f[wri]:>8.1f}% {r['user_id']}")
|
||||
else:
|
||||
print(f"{r['z']:>12.1f}{'—':>8}{'(no fwd bets)':>12}{'—':>9} {r['user_id']}")
|
||||
fz = [fwd[i][zi] for i in ids if i in fwd and fwd[i][zi] is not None]
|
||||
if fz:
|
||||
print(f"\nmedian forward z of in-sample winners: {sorted(fz)[len(fz)//2]:.2f} "
|
||||
f"(near 0 = the in-sample edge did NOT persist)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Goldsky Polymarket orderbook subgraph client.
|
||||
|
||||
The hosted subgraph times out on any `orderBy` over a non-indexed field
|
||||
(numTrades, scaledProfit, ...), so we never sort server-side. Instead we
|
||||
cursor-paginate by `id` (the indexed primary key) — `where:{id_gt:<last>}`,
|
||||
`first:1000` — which is stable and resumable. Ranking happens locally in
|
||||
DuckDB after ingest. That constraint is exactly why the bulk-ingest design
|
||||
is the only one that scales here.
|
||||
"""
|
||||
|
||||
import json
|
||||
import ssl
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
ENDPOINT = ("https://api.goldsky.com/api/public/"
|
||||
"project_cl6mb8i9h0003e201j6li0diw/subgraphs/"
|
||||
"polymarket-orderbook-resync/prod/gn")
|
||||
|
||||
_CTX = ssl._create_unverified_context()
|
||||
PAGE = 1000
|
||||
|
||||
|
||||
def query(gql, variables=None, retries=6):
|
||||
"""POST a GraphQL query, retrying on transient errors / timeouts."""
|
||||
body = json.dumps({"query": gql, "variables": variables or {}}).encode()
|
||||
delay = 1.0
|
||||
for attempt in range(retries):
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
ENDPOINT, data=body,
|
||||
headers={"Content-Type": "application/json",
|
||||
"User-Agent": "Mozilla/5.0"})
|
||||
r = json.loads(urllib.request.urlopen(req, timeout=60, context=_CTX).read())
|
||||
if "errors" in r:
|
||||
# statement-timeout is transient under load; back off and retry
|
||||
msg = r["errors"][0].get("message", "")
|
||||
if "timeout" in msg.lower() or "timed out" in msg.lower():
|
||||
raise TimeoutError(msg)
|
||||
raise RuntimeError(msg[:300])
|
||||
return r["data"]
|
||||
except Exception as e:
|
||||
if attempt == retries - 1:
|
||||
raise
|
||||
time.sleep(delay)
|
||||
delay = min(delay * 2, 30)
|
||||
return None
|
||||
|
||||
|
||||
def paginate(entity, fields, where="", page=PAGE, start_id="", on_page=None):
|
||||
"""Yield every row of `entity`, cursor-paginating by id.
|
||||
|
||||
`fields` is the GraphQL selection set (a string). `where` is extra filter
|
||||
clauses (without braces), e.g. 'resolutionTimestamp_not: null'. Resume by
|
||||
passing the last id seen as `start_id`.
|
||||
"""
|
||||
last = start_id
|
||||
while True:
|
||||
clauses = f'id_gt: "{last}"'
|
||||
if where:
|
||||
clauses += ", " + where
|
||||
gql = (f'{{ {entity}(first: {page}, orderBy: id, orderDirection: asc, '
|
||||
f'where: {{ {clauses} }}) {{ {fields} }} }}')
|
||||
rows = query(gql).get(entity, [])
|
||||
if not rows:
|
||||
return
|
||||
for row in rows:
|
||||
yield row
|
||||
last = rows[-1]["id"]
|
||||
if on_page:
|
||||
on_page(len(rows), last)
|
||||
if len(rows) < page:
|
||||
return
|
||||
|
||||
|
||||
def shard_bounds(n=16):
|
||||
"""Hex-prefix boundaries after '0x' that split the id space into n shards.
|
||||
ids are lowercase hex; '0xg' sorts after every '0xf…' so it caps the last
|
||||
shard. Returns n+1 bounds; shard i spans [bounds[i], bounds[i+1])."""
|
||||
d = "0123456789abcdef"
|
||||
if n == 16:
|
||||
return ["0x" + c for c in d] + ["0xg"]
|
||||
if n == 256:
|
||||
return ["0x" + a + b for a in d for b in d] + ["0xg"]
|
||||
raise ValueError("n must be 16 or 256")
|
||||
|
||||
|
||||
def paginate_pages(entity, fields, lo="", hi="", start_id="", page=PAGE):
|
||||
"""Yield (rows, last_id) per page for ids in (max(start_id,lo), hi).
|
||||
Page-level so the caller can checkpoint each shard's cursor."""
|
||||
last = start_id or lo
|
||||
while True:
|
||||
clauses = f'id_gt: "{last}"'
|
||||
if hi:
|
||||
clauses += f', id_lt: "{hi}"'
|
||||
gql = (f'{{ {entity}(first: {page}, orderBy: id, orderDirection: asc, '
|
||||
f'where: {{ {clauses} }}) {{ {fields} }} }}')
|
||||
rows = query(gql).get(entity, [])
|
||||
if not rows:
|
||||
return
|
||||
yield rows, rows[-1]["id"]
|
||||
if len(rows) < page:
|
||||
return
|
||||
last = rows[-1]["id"]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# smoke test: count a few resolved conditions
|
||||
n = 0
|
||||
for c in paginate("conditions", "id resolutionTimestamp payoutNumerators",
|
||||
where="resolutionTimestamp_not: null"):
|
||||
n += 1
|
||||
if n <= 2:
|
||||
print(c)
|
||||
if n >= 2500:
|
||||
break
|
||||
print(f"paged {n} resolved conditions ok")
|
||||
Reference in New Issue
Block a user