Files
winning-wallet-finder/FINDINGS.md
T
jaxperro 3d0bc7f001 Add live/ skilled-wallet scanner + cache; document clean OOS finding
live/: operationalizes the LBS/Yale "skilled ~3%" result against the live
data-api. Enumerate recent liquid markets -> top traders -> candidate pool;
cache every wallet's resolved bets once in DuckDB (~26k wallets / 12.5M bets,
keyed by per-bet resolution time so any cutoff re-scores in seconds); 5-gate
skill funnel (n>=15, z>0, BH-FDR, split-half OOS, MM/bot cap); dashboard +
daily refresh.

Key finding: copying the high-win-rate "favorite-rider" cohort looks +23.6%
in-sample but loses -7.4% once selected on pre-June-1 data only (99% -> 68%
win rate) — selection bias, reproducing the paper's "lucky winners revert"
result on live data. Win rate != edge, again.

wide/: bulk subgraph->DuckDB scanner (survivorship-bias-free over all wallets),
but the public subgraph is frozen at Jan 2026 -> historical tool only.

Large local data (*.duckdb, candidates.json, *_scored.json, history/) gitignored.
README + FINDINGS updated with the current logic and the clean result.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 11:16:20 -06:00

159 lines
9.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Findings — what works and what doesn't on Polymarket
A research log of an honest attempt to find a systematic, automatable edge on
Polymarket using public data. The short version: **almost nothing works** — the
market is efficient — and the one thing that does isn't a money-printer, it's a
detection signal.
## The goal
Find a repeatable way to make money on Polymarket: identify "smart money"
wallets, copy them, farm rewards, or arbitrage — anything systematic and
automatable from public data.
## Scorecard
| Approach | Verdict | Why |
|----------|---------|-----|
| **Copy high-win-rate wallets** | ❌ dead | Win rate was an illusion (see below). True rates ~50%. Flat-size copying backtested **48%** over a week. |
| **Rank by leaderboard / PnL** | ❌ dead | Raw PnL is variance; top wallets win ~50% and profit via sizing/timing you can't copy. |
| **LP reward farming** | ❌ dead | The fat "thin-book" APRs are illusory — Polymarket **refunds unearned pool to the sponsor** when liquidity is low. Real yield is modest and adverse-selection-dominated. |
| **Binary YES+NO arbitrage** | ❌ dead | Efficient — min observed sum was 1.001 (the spread). Closed instantly by the engine. |
| **Multi-outcome logical arb** | ❌ dead | True partitions priced efficiently (min sum 0.999). Apparent "arbs" were non-exclusive market groupings. |
| **Cross-venue arb (Polymarket↔Kalshi)** | ❌ dead | Venues agree to ~1¢; locking both legs costs **>$1 after fees**. Real gaps last ~seconds and are taken by bots. |
| **Insider / sharp detection** | ✅ **works** | Statistical improbability (z-score of wins vs. odds) is a real, hard-to-fake edge signal. See `insider.py`. |
## The big technical findings
**1. Win rate on Polymarket is survivorship-biased — badly.**
The platform only redeems *winning* shares; losing shares are worth $0 and sit
unredeemed in `/positions` at `curPrice 0` forever, never entering
`/closed-positions`. Measuring win rate over `/closed-positions` alone counts
almost only winners. We saw a wallet read **90.6%** that was truly **48.3%**.
A correct win rate must union both endpoints. *Lesson: a high reported win rate
is a red flag for a measurement bug, not a sharp.*
**2. Win rate ≠ profit, and PnL ≠ reliability.**
A wallet winning 54% made millions; the all-time #1 wallet (43% win) was $3.8M
over 90 days. Profit comes from sizing and entry prices, not hit rate.
**3. The market is efficient.** Six systematic public-data edges, all closed or
illusory. There is no turnkey retail edge sitting in public data in 2026 —
durable edge requires *speed/infrastructure* (arb bots), *private information*,
or *getting paid to provide liquidity* (modest, adverse-selection-dominated).
**4. The one real signal: statistical improbability (z-score).**
Each bet entered at price `p` has an odds-implied win probability `p`. A wallet
winning far more than `Σp` is beating the market's own pricing — measured as a
z-score and one-sided p-value. This is the rigorous version of the edge metric
the whole project was chasing. It distinguishes:
- **Sharps** — high z, normal entry timing (skill over many bets).
- **Insiders** — high z **+** late (pre-resolution) entry **+** fresh wallet.
Plus **funding-cluster linking** (à la Bubblemaps / the 2026 *60 Minutes*
investigation): trace each wallet's USDC funders on Polygon and link wallets
that share a *personal* funding hub — judged by the funder's own outbound degree
so shared exchanges don't false-link everyone. (See `insider.py`.)
## Practical conclusion
- **Don't** fund copy-trading, LP farming, or arb based on this work — we tested
them and they don't clear.
- **Do** use `insider.py`'s z-score as a rigorous "who actually has edge" filter,
far better than leaderboard or win rate.
- A genuine money-making edge has to come from *you* — a niche you understand
better than the market — with tooling built around it, not from a public-data
scanner.
- **Legal note:** *detecting* suspected insider trading is fine; *trading on*
material nonpublic information is illegal, and blindly following a suspected
insider is not a safe strategy.
## Insider detection — what the z-score signal actually found
Building `insider.py` and sweeping markets (`hunt.py`, `huntwide.py`) surfaced
genuinely improbable wallets. Out of ~289 scored:
- **DREAMBIG.** (z=8.9, p≈2e-19) and **qcp14** (z=5.3) on the Iran ceasefire
market — 4577% of wins entered <24h before resolution. Textbook insider
fingerprint, on exactly the theme the *60 Minutes* story covered.
- **Famecesgoal** (z=9.6) won only 14.5% of bets — but bet longshots and hit
+98 above what the odds implied. The clearest "beats the prices it pays" case.
Two refinements proved essential:
- **Trade count separates insiders from bots.** `bjprolo` scored z=37 — but on
**306,873** lifetime trades. That's a market-maker grinding a tiny systematic
edge, not information. Real edge wallets show concentrated z over 13k trades.
- **Funding-cluster linking** (Alchemy, the Bubblemaps "who-funded-whom" step)
works *only* with a personal-hub filter: a shared exchange (everyone uses
Coinbase) is not a shared operator. Judge a funder by its own outbound degree.
## The copy-trade verdict — in-sample vs out-of-sample
The decisive test: does copying z-selected wallets make money?
- **In-sample** (`copyback.py`): copy the edge wallets from May 30, z-weighted,
reinvest 100%. Result: **+545%** in 15 days. Looks incredible — and it's
circular (the wallets were *selected* for winning over that very window).
86% of it came from one wallet; the highest-z pick contributed $23.
- **Out-of-sample** (`oos.py`): select wallets using **only data through
Apr 30**, then copy forward May 30→now. Result: **+168%** — but **entirely
from one longshot lottery wallet** (1.5% pre-period win rate hitting again).
The two strongest pre-period signals made **$0** forward. Forward hit rate was
27%. That's variance, not edge that persists.
**Conclusion:** even the one real signal (z-score), when tested for whether you
can *profit by copying it*, fails out-of-sample — joining every other strategy.
The detector is valuable for *finding* anomalous wallets; copying them is not a
proven, fundable edge. The live watcher (`webhook_receiver.py`) exists to gather
real forward (out-of-sample) data on these wallets — observe before you size up.
## Practical conclusion 2
- **Don't** fund a copy strategy — both the +545% and +168% are
variance/concentration, not repeatable edge.
- **Do** use the detector to find statistically anomalous wallets and watch them
live; judge persistence forward with your own eyes.
- A durable trading edge has to come from *you* (a niche you know), with this
tooling built around your judgment.
## The skilled-3% scan, and a clean out-of-sample loss (June 2026)
External validation arrived: an LBS/Yale study (Gomez-Cram, Guo, Kung, Jensen,
Apr 2026; SSRN 5910522) over 1.72M accounts found only **~3.14%** of traders are
genuinely skilled — measured by randomizing each trader's bet *directions* 10k×
(a Monte-Carlo z-score) and requiring out-of-sample persistence. That is exactly
this project's z-score + `oos.py` method, independently confirmed.
Built `live/` to operationalize it at scale: enumerate recent liquid markets →
cache every candidate's resolved bets locally (~26k wallets / 12.5M bets, so
re-scoring at any cutoff is seconds) → a 5-gate funnel (n≥15, z>0, BH-FDR,
split-half OOS, MM/bot cap). It surfaced 107 "validated" wallets.
**The decisive test.** Copying the high-win-rate "favorite-rider" cohort, $1000,
no execution lag, June 1→now:
- selected *through* the test window (look-ahead): 99% win rate, **+23.6%**.
- selected on **pre-June-1 data only** (honest): 68% win rate, **7.4%**
(19% on the settled portion).
The +23.6% was selection bias. Done cleanly, the favorites **lose** — a textbook
reproduction of the paper's "~60% of lucky winners become losers out-of-sample,"
now on our own live data. *Lesson reinforced: high win rate is the most
misleading signal on the platform; favorite-riders are uncopyable.* The
underdog/`value` archetype (beats longshot prices) is the only one left worth
testing.
## Repo layout
- `insider.py` — the detector: z-score/p-value, timing/freshness/sizing signals,
Alchemy funding-cluster ring detection.
- `hunt.py` / `huntwide.py` — market sweeps that surface edge wallets.
- `copyback.py` / `oos.py` — in-sample and out-of-sample copy-trade backtests.
- `webhook_receiver.py` — push-based live trade watcher (Alchemy → Discord).
- `smart_money.py` — data foundation + dashboard (true-win-rate scanner).
- `live/` — current scanner: cache-backed skilled-3% finder + watchlist + daily
refresh + dashboard. See `live/README.md`.
- `wide/` — bulk subgraph→DuckDB scanner (survivorship-bias-free, all wallets);
public subgraph frozen at Jan 2026, so historical-only. See `wide/README.md`.
- `archive/` — the strategies that didn't work, kept for reference. See
`archive/README.md`.