The sharps table's P&L was reconstructed from won×entry×size and diverged
from PM /profit by up to 10x, with sign flips — four root causes found by
decomposing outliers against PM:
1. bets pull capped at 2000 rows/180d -> high-volume wallets truncated
(ewww1: 740 of 4088 positions cached -> k shown vs k real)
2. both-sides (hedged) markets: one-per-market dedup dropped the paired
side (suraxy: -k of dropped losing legs -> overcount)
3. initialValue=0 on big longshot winners -> mis-sized reconstruction
4. corrupt near-epoch res_t rows polluting sums
Fix: All-Time/Conv/30d P&L are now the sum of Polymarket's OWN realizedPnl
per closed position over FULL history (cache.closed_exits, extended to store
realized_pnl; smart_money.closed_exits captures it + relaxes the avg/tb
guard). This is the wallet's realized track record — what a copier mirroring
buy/sell/hold banks — and sums to PM. Immune to all four bugs (each asset is
its own realized row; no size/entry/res_t needed). Win% = share of closed
positions that made money. Verified vs PM: ewww1 1.00x, KBO30 0.0x->0.97x,
suraxy 2.7x->1.03x, sign flip fixed; oliman2 1.63x is correct-by-design
(realized track record vs PM's open-book unrealized marks). README gotcha 11
rewritten; dashboard tooltips reframed off the copy-ceiling language.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per the data-completeness call: closed_exits pulls each wallet's ENTIRE
closed-position history, not a 180d window. smart_money.closed_exits returns
(exits, reached_end) — reached_end True only when the pull hits the true
start of history (empty/short page), so the cache's flag tells a
finished backfill from an interrupted one and a killed deep pull re-completes
instead of falsely reporting done. One-time deep cost, amortized by the
incremental cache (later runs page only new closes). Verified: imwalkinghere
full 65d lifetime (879 exits, its real age not a cap), complete=True, 2nd
call 0.00s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The row cap was the wrong instrument — it could truncate a hyperactive
wallet's window (compromising data) while the real fix is bounding by DATE.
cache.closed_exits now pulls exactly the 180d window the bets cache scores,
so the exit overlay COMPLETELY covers every bet the sharps/backtest stats
touch. Tracks oldest_ts so an interrupted backfill re-completes instead of
falsely reporting done; once complete, refreshes only page new closes.
since_ts (180d/30d per caller) is the real bound — this fixes completeness
AND the runtime stall (the old since_ts=0 pulled all-history). Verified:
imwalkinghere full window in 9s, 2nd call 0.00s cached.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/closed-positions serves 50-row pages regardless of limit; stepping by the
requested size truncated every wallet's exit history to its most recent ~50
closes (1% coverage for hyperactive wallets) — sold-mirroring was silently
falling back to the hold-to-res ceiling almost everywhere. Now: page by
returned size, cache exits incrementally in duckdb (immutable events —
deep backfill once, a page or two per refresh; title/outcome columns for
display). Deep-exit 30d replay: $18,270 (+1727%), 259W/79L/29R/284S,
misses 253 -> 1 (mirrored exits recycle capital as fast as the signal does).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two data bugs were hiding the best copy targets (FINDINGS 'The holder
blind spot'):
* live/trust.py (new): only score rows whose res_t matches the market's
consensus resolution time across >=2 wallets, pulled after resolution,
resolved != False. Kills the res_t=ts fallback poison that let scalpers
masquerade as 99%-win holders (ArbTraderRookie's rows were 100% this).
* conviction_scan.py: trusted rows only; p80 conviction cutoff from the
train window only (look-ahead leak); new gates z_all>2 (whole-book z,
~doubles pooled forward copy-ROI) and median conviction stake >=$50
(dust filter). 55 wallets selected, forward 30/38 profitable, +21.4%
pooled (was 284 selected, +16.0%).
* validate_timing.py: the held-edge gates now read the trailing-90d
trusted conviction record via trust.conviction_record instead of the
replay's held leg, which is structurally ~all-unresolved for week-lead
holders (whale 0x73afc816 showed 'held 0-0, 21 unresolved' and was
rejected at 100% fwd win). 25 copy-positive holders now, including
Stavenson (51-0), the whale (112-4) and iohihoo (98-10).
* cache.py: query() helper so trust.py shares the single in-process
connection instead of fighting the single-writer lock.
* README: gotcha 8 (fake res_t/won) + candidate next data sources
(Goldsky pipelines, PolymarketData order books, Pinnacle CLV,
Polysights); FINDINGS: dated section correcting the iohihoo/ArbTrader
scalper-trap verdicts as winner=False-bug artifacts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- bets table gains asset (token id = position identity), src/ts (endpoint
provenance + close time), resolved (False = early-sold in an unended market;
won is a curPrice mark). Auto-migrates v1 in place (~8s, 3,697 exact dupes
merged); legacy rows carry NULLs until their wallet refreshes.
- refresh is now an upsert by token instead of a wallet wipe: rows sliding out
of the rolling pull window survive, so per-wallet history accumulates into a
permanent archive. Same-asset rows from both endpoints (partially-closed
positions) dedupe to the larger-stake row - kills the two-endpoint
double-count class (~35k suspect pairs found in the audit).
- p stored raw (0 = avgPrice missing), clamped on read by get_bets, so missing
prices stay distinguishable from real 0.1c longshots; insider CLI + oos clamp
their own direct use.
- resolved_bets(strict=True): a failed page raises instead of returning a
silently truncated history; get_bets no longer caches or marks failed pulls
(pre-v2 an API error cached the wallet as empty-and-fresh for 14 days -
Kruto2027 was a live victim of this last night).
Verified: migration 18,289,320 -> 18,285,623 rows; forced refreshes of two
sharps show 0 same-asset dups, 0 legacy/new mixing, clamped reads, conviction
stats intact (Kruto conv win 73%).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- copybot: model Polymarket taker fees (V2, since 2026-03-30: shares*rate*p*(1-p),
sports 0.03) on every paper/live fill; track fees_paid in state + feed; settle
P&L nets the entry fee. publish_feed now commits state + fills ledger with the
feed (autostash rebase) so the local poller is the sole state writer.
- validate_timing: copy_pnl/held_pnl are fee-aware -> sharp selection now requires
clearing real copy costs; conv stats + lead profile use resolved bets only.
- conviction_scan/skill: exclude res_t>now rows (early-sold positions in
unresolved markets scored at curPrice - a mark, not an outcome; was ~5% of the
June test window at a 72% pseudo-win rate).
- portfolio: skip unresolved rows (stake used to vanish from equity); missed bets
of kind=open no longer KeyError - mark-to-market hypothetical P&L.
- collect: cap stale refreshes at STALE_CAP=2500/run (bulk-aged pool turned the
daily refresh into a ~40h pull holding the DuckDB lock).
- Actions cron disabled: GitHub ran */5 every ~1.5-2.5h and the 10-min stale
window skipped the rest -> 1 of ~104 qualifying buys copied. launchd --poll 60
is now the runner; workflow stays as manual backstop.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap the flat $200 conviction cutoff for a per-wallet percentile (top
20% of each wallet's own stake sizes) everywhere it was used:
- cache.py: canonical CONV_PCTILE=0.80 + conv_cutoff() helper (matches
the dashboard's pctl: filter >0, sort, linear interp)
- conviction_scan.py: per-wallet quantile_cont(size,0.8) in SQL, was
`size >= 200`
- validate_timing.py, pnl_focused.py: use cache.conv_cutoff
Rationale + validation: p80 reproduces flat-$200's win-rate lift on the
sharps while adapting to scale (a whale's $200 isn't conviction, a
minnow's is). Re-running the pipeline under p80: scan finds 218 profile
wallets (was 69), forward 62/83 profitable (p~0), +16% pooled ROI — edge
persists out-of-sample. Regenerated conviction_wallets.json /
watch_sharps.json; docs updated. skill.py/strategy.py/insider.py
untouched (score over all bets / size as copyability heuristic only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
live/: operationalizes the LBS/Yale "skilled ~3%" result against the live
data-api. Enumerate recent liquid markets -> top traders -> candidate pool;
cache every wallet's resolved bets once in DuckDB (~26k wallets / 12.5M bets,
keyed by per-bet resolution time so any cutoff re-scores in seconds); 5-gate
skill funnel (n>=15, z>0, BH-FDR, split-half OOS, MM/bot cap); dashboard +
daily refresh.
Key finding: copying the high-win-rate "favorite-rider" cohort looks +23.6%
in-sample but loses -7.4% once selected on pre-June-1 data only (99% -> 68%
win rate) — selection bias, reproducing the paper's "lucky winners revert"
result on live data. Win rate != edge, again.
wide/: bulk subgraph->DuckDB scanner (survivorship-bias-free over all wallets),
but the public subgraph is frozen at Jan 2026 -> historical tool only.
Large local data (*.duckdb, candidates.json, *_scored.json, history/) gitignored.
README + FINDINGS updated with the current logic and the clean result.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jaxperro