jaxperro
3d0bc7f001
live/: operationalizes the LBS/Yale "skilled ~3%" result against the live data-api. Enumerate recent liquid markets -> top traders -> candidate pool; cache every wallet's resolved bets once in DuckDB (~26k wallets / 12.5M bets, keyed by per-bet resolution time so any cutoff re-scores in seconds); 5-gate skill funnel (n>=15, z>0, BH-FDR, split-half OOS, MM/bot cap); dashboard + daily refresh. Key finding: copying the high-win-rate "favorite-rider" cohort looks +23.6% in-sample but loses -7.4% once selected on pre-June-1 data only (99% -> 68% win rate) — selection bias, reproducing the paper's "lucky winners revert" result on live data. Win rate != edge, again. wide/: bulk subgraph->DuckDB scanner (survivorship-bias-free over all wallets), but the public subgraph is frozen at Jan 2026 -> historical tool only. Large local data (*.duckdb, candidates.json, *_scored.json, history/) gitignored. README + FINDINGS updated with the current logic and the clean result. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
46 lines
1.6 KiB
Python
46 lines
1.6 KiB
Python
#!/usr/bin/env python3
|
|
"""Collect EVERY candidate wallet's resolved bets into the cache, up to present.
|
|
|
|
One-time (per refresh window) comprehensive pull so the whole candidate pool is
|
|
local. Resumable: cache.get_bets skips wallets pulled within MAX_AGE_DAYS, so
|
|
killing and re-running continues where it left off. Most-active wallets first,
|
|
so a partial cache already covers the wallets most likely to be skilled.
|
|
|
|
python3 collect.py
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import sys
|
|
import time
|
|
from concurrent.futures import ThreadPoolExecutor, as_completed
|
|
|
|
import cache
|
|
|
|
HERE = os.path.dirname(__file__)
|
|
WORKERS = 16
|
|
|
|
|
|
def main():
|
|
cands = json.load(open(os.path.join(HERE, "candidates.json")))
|
|
cands.sort(key=lambda c: c.get("markets_seen", 0), reverse=True)
|
|
wallets = [c["wallet"] for c in cands]
|
|
print(f"collecting {len(wallets):,} wallets up to present · {WORKERS} workers", flush=True)
|
|
done, t0 = 0, time.time()
|
|
with ThreadPoolExecutor(max_workers=WORKERS) as ex:
|
|
futs = [ex.submit(cache.get_bets, w) for w in wallets]
|
|
for _ in as_completed(futs):
|
|
done += 1
|
|
if done % 200 == 0:
|
|
w, b = cache.stats()
|
|
rate = done / max(1e-9, time.time() - t0)
|
|
eta = (len(wallets) - done) / max(1e-9, rate) / 3600
|
|
print(f" {done:,}/{len(wallets):,} · cache {w:,}w/{b:,}bets · "
|
|
f"{rate:.1f}/s · ETA {eta:.1f}h", flush=True)
|
|
w, b = cache.stats()
|
|
print(f"DONE {time.strftime('%F %T')} — cache: {w:,} wallets, {b:,} bets", flush=True)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|