Files
winning-wallet-finder_github/live/collect.py
T
jaxperro 3d0bc7f001 Add live/ skilled-wallet scanner + cache; document clean OOS finding
live/: operationalizes the LBS/Yale "skilled ~3%" result against the live
data-api. Enumerate recent liquid markets -> top traders -> candidate pool;
cache every wallet's resolved bets once in DuckDB (~26k wallets / 12.5M bets,
keyed by per-bet resolution time so any cutoff re-scores in seconds); 5-gate
skill funnel (n>=15, z>0, BH-FDR, split-half OOS, MM/bot cap); dashboard +
daily refresh.

Key finding: copying the high-win-rate "favorite-rider" cohort looks +23.6%
in-sample but loses -7.4% once selected on pre-June-1 data only (99% -> 68%
win rate) — selection bias, reproducing the paper's "lucky winners revert"
result on live data. Win rate != edge, again.

wide/: bulk subgraph->DuckDB scanner (survivorship-bias-free over all wallets),
but the public subgraph is frozen at Jan 2026 -> historical tool only.

Large local data (*.duckdb, candidates.json, *_scored.json, history/) gitignored.
README + FINDINGS updated with the current logic and the clean result.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 11:16:20 -06:00

46 lines
1.6 KiB
Python

#!/usr/bin/env python3
"""Collect EVERY candidate wallet's resolved bets into the cache, up to present.
One-time (per refresh window) comprehensive pull so the whole candidate pool is
local. Resumable: cache.get_bets skips wallets pulled within MAX_AGE_DAYS, so
killing and re-running continues where it left off. Most-active wallets first,
so a partial cache already covers the wallets most likely to be skilled.
python3 collect.py
"""
import json
import os
import sys
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
import cache
HERE = os.path.dirname(__file__)
WORKERS = 16
def main():
cands = json.load(open(os.path.join(HERE, "candidates.json")))
cands.sort(key=lambda c: c.get("markets_seen", 0), reverse=True)
wallets = [c["wallet"] for c in cands]
print(f"collecting {len(wallets):,} wallets up to present · {WORKERS} workers", flush=True)
done, t0 = 0, time.time()
with ThreadPoolExecutor(max_workers=WORKERS) as ex:
futs = [ex.submit(cache.get_bets, w) for w in wallets]
for _ in as_completed(futs):
done += 1
if done % 200 == 0:
w, b = cache.stats()
rate = done / max(1e-9, time.time() - t0)
eta = (len(wallets) - done) / max(1e-9, rate) / 3600
print(f" {done:,}/{len(wallets):,} · cache {w:,}w/{b:,}bets · "
f"{rate:.1f}/s · ETA {eta:.1f}h", flush=True)
w, b = cache.stats()
print(f"DONE {time.strftime('%F %T')} — cache: {w:,} wallets, {b:,} bets", flush=True)
if __name__ == "__main__":
main()