From 28fdf7f42b71ace9aa869ad73044c4d2f87654ea Mon Sep 17 00:00:00 2001 From: jaxperro Date: Tue, 23 Jun 2026 14:51:04 -0600 Subject: [PATCH] docs: document copy-positive selection, Copy P&L, cache-based portfolio Update the dev-facing docs so others can follow the current system: - live/README: copy-positive-holder selection (replaces lead-time gate), Copy P&L as the copyability metric, new Paper portfolio (portfolio.py) + Dashboard feeds (watch_sharps.json / portfolio.json) sections, the full 8-step daily flow, the cache rolling-180d/replace retention gotcha, and a Copy execution (copybot/sync_floors, separate WIP) note. - README: top portfolio is now precomputed off the cache (correct recycling), judge by Copy P&L not win%, copy execution is separate. - FINDINGS: capital-recycling / $1k-book section + repo-layout refresh. Co-Authored-By: Claude Opus 4.8 --- FINDINGS.md | 27 ++++++++++++- README.md | 32 +++++++++------- live/README.md | 101 +++++++++++++++++++++++++++++++++++++++---------- 3 files changed, 126 insertions(+), 34 deletions(-) diff --git a/FINDINGS.md b/FINDINGS.md index 14c8ea27..fedb9cb8 100644 --- a/FINDINGS.md +++ b/FINDINGS.md @@ -222,6 +222,25 @@ by position win%. Conviction must be measured at the position level (a wallet's stake in a market), not per individual buy — a scalper splits one position across many small buys, so a per-trade threshold copies far more (and worse) bets than intended.* +## Capital recycling & the $1k book (2026-06-23) + +The $1,000 paper book (`live/portfolio.py` → `portfolio.json`, rendered at +jaxperro.com/trading) surfaced two things: + +- **"Saturation" was mostly a measurement artifact.** The old browser replay froze + capital in positions whose resolution date the data-api didn't return, so it + skipped bets it could afford (340 phantom misses on a 4-wallet book). Computing + the book **off the cache** — which stores each bet's resolution time (`res_t`) — + frees cash at the true resolution moment: misses dropped to ~0 and the book + recycled ~23× over the window. With *real* money this isn't even a problem (cash + returns on redemption); it was purely the paper sim mis-measuring. +- **More wallets help only up to the bankroll's slot count.** A combo backtest over + the copy-positive holders showed returns rise with basket size *until* peak + concurrent demand exceeds ~$1k ÷ $50 = 20 slots, after which a high-volume wallet + just crowds out the others. So: pick wallets that **fit** the bankroll, favor + **fast-resolving** markets (capital velocity > bet size on $1k), and don't + diversify past what you can fund. Two well-chosen holders beat four that overflow. + ## Repo layout - `insider.py` — the detector: z-score/p-value, timing/freshness/sizing signals, @@ -230,8 +249,12 @@ small buys, so a per-trade threshold copies far more (and worse) bets than inten - `copyback.py` / `oos.py` — in-sample and out-of-sample copy-trade backtests. - `webhook_receiver.py` — push-based live trade watcher (Alchemy → Discord). - `smart_money.py` — data foundation + dashboard (true-win-rate scanner). -- `live/` — current scanner: cache-backed skilled-3% finder + watchlist + daily - refresh + dashboard. See `live/README.md`. +- `live/` — current system: cache-backed finder + **copy-positive-holder sharps + selection** (`conviction_scan.py` + `validate_timing.py` → `watch_sharps.json`, + ranked by Copy P&L) + **$1k paper book** (`portfolio.py` → `portfolio.json`) + + daily refresh. The dashboard (jaxperro repo) renders those two JSON feeds. See + `live/README.md`. *Copy execution (`copybot.py`, `sync_floors.py`) is a separate, + in-progress system — this finder is selection + tracking only.* - `wide/` — bulk subgraph→DuckDB scanner (survivorship-bias-free, all wallets); public subgraph frozen at Jan 2026, so historical-only. See `wide/README.md`. - `archive/` — the strategies that didn't work, kept for reference. See diff --git a/README.md b/README.md index d8c8a019..e5688af6 100644 --- a/README.md +++ b/README.md @@ -144,20 +144,26 @@ Keep the two wallet lists in sync: Alchemy's address list (what *triggers*) and ### 2. Live paper portfolio — `trading/` → [jaxperro.com/trading](https://jaxperro.com/trading) -A **$1,000 paper account that behaves like real money**: it replays every -watched-wallet trade since inception, **enters when they enter** (if there's -cash), holds each bet **to resolution**, then settles (win → payout, loss → $0) -and frees the cash. Shows **Liquid** (cash), **Invested** (open bets marked to -market), **Realized** (settled P&L), a **Current Bets** table with per-bet entry -/ mark / P&L / *settle date*, and — crucially — **Missed P&L**: the profit left -on the table from trades skipped because the bankroll was fully deployed (the -real cost of a small account). It runs **100% client-side** off Polymarket's -public API (CORS-open) — zero backend, zero added cost. +A **$1,000 paper account that behaves like real money**: it mirrors each followed +wallet's **conviction** bets (top 20% of their own stake sizes) at $50/trade, +**enters when they enter** (if there's cash), holds to resolution, then settles +and frees the cash. Shows **Liquid**, **Invested**, **Realized**, a **Current +Bets** table, and **Missed P&L** (profit skipped when the bankroll was fully +deployed — the cost of a small account). -> **What the tracker taught us:** $1,000 across many hyperactive wallets gets -> fully deployed almost instantly — you can follow only a few percent of their -> trades. Concentrating on a handful of high-conviction wallets with bigger -> stakes is the only way a small bankroll meaningfully mirrors them. +It's now **precomputed server-side off the cache** (`live/portfolio.py` → +`portfolio.json`) rather than replayed in the browser: the cache stores each bet's +resolution time, so capital **recycles at the true resolution moment** (the +client replay used to phantom-lock capital when the data-api lacked resolution +dates). The page is a static renderer of `portfolio.json` + `watch_sharps.json`, +with the old client-side replay kept as a fallback. + +> **What the tracker taught us:** $1,000 across many hyperactive wallets saturates +> instantly — concentrate on a few wallets that fit the bankroll. And **win% lies +> about copyability**: judge candidates by **Copy P&L** (the sharps table's headline +> column — actual flat-$50 copy result, scalpers exposed), not win rate. *Actually +> placing* the trades is a separate in-progress system (`copybot.py`); this repo is +> selection + paper tracking. --- diff --git a/live/README.md b/live/README.md index a6ce8fac..f9972dbf 100644 --- a/live/README.md +++ b/live/README.md @@ -37,17 +37,28 @@ underdog/longshot prices — the copyable alpha), `balanced`, or `favorite` | enumerate | `enumerate.py [days]` | recent liquid markets (Gamma `end_date_min`) → top traders → candidate pool (`candidates.json`), accumulates across runs | | cache | `cache.py` / `collect.py` | pull each wallet's resolved bets **once** into `cache.duckdb` (24s → 0.003s on re-read). Stores `res_t` per bet, so any date cutoff reads the same cache | | score | `skill.py [N]` | the 5-gate funnel over cached candidates → `watch_skilled.json` (webhook-compatible) | +| sharps | `conviction_scan.py` + `validate_timing.py` | conviction-profile scan → copy-positive-holder selection → `watch_sharps.json` (see "The repeatable find") | +| portfolio | `portfolio.py` | $1k paper book off the cache → `portfolio.json` (see "Paper portfolio") | | dashboard | `dashboard.py` | self-contained `dashboard.html` — sortable, archetype-tagged, live recent-trade lookup | | backtest | `backtest_june.py [arch]` | copy an archetype's June-1+ entries, $1000, no lag → P&L | | clean test | `clean_test.sh` | **the honest test**: re-select on pre-June-1 data only, then backtest June-1+ forward | ## The cache is the point -`cache.duckdb` holds ~26k wallets / 12.5M+ bets, pulled once. Every score — +`cache.duckdb` holds ~30k wallets / 14M+ bets, pulled once. Every score — any archetype, any cutoff, the clean OOS test — now runs in **seconds** instead of hours of API pulls. `MAX_AGE_DAYS=14`: the broad pool refreshes biweekly; the watchlist is force-refreshed daily (`cache.invalidate`) for forward tracking. +**Retention gotcha — the cache is NOT append-only.** Each wallet's refresh does +`DELETE FROM bets WHERE wallet=?` then re-inserts a fresh pull, and that pull is a +**rolling 180-day window** (`WINDOW_DAYS`) capped at ~2k bets/endpoint +(`max_pages`). So wallet *coverage* grows (new wallets are kept), but any single +wallet's history is a capped, rolling, overwrite-on-refresh snapshot — bets older +than ~180d are dropped on the next re-pull. For a permanent long-horizon archive, +use the append-style `../wide/pmkt.duckdb` subgraph dataset instead, or change the +pull to upsert + drop the cutoff. + ## The clean test (why the favorites are a mirage) `clean_test.sh` selects favorites using **only bets resolved before June 1**, @@ -84,27 +95,79 @@ wallet's scale (a whale's $200 bet isn't conviction; a minnow's is). - `conviction_scan.py` — train pre-June / validate June on conviction bets → 218 matches, **62/83 profitable forward (p≈0)**. → `conviction_wallets.json`. -- `validate_timing.py` — the copyability gate: entry→resolution **lead time** on - winning conviction bets separates copyable sharps (multi-day lead) from - "last-minute" wallets (median lead <24h) we can't mirror in time — could be - genuine insiders or just fast-resolving-market specialists, can't tell which. - Also drops wallets inactive >30d. → **~31 validated copyable sharps** in - `watch_sharps.json` (shown live on jaxperro.com/trading). It additionally - precomputes every stat the dashboard renders — incl. **`copy_pnl`**, the - authoritative flat-$50 copy P&L (replay entries, mirror exits, settle held bets - at clob resolution by token_id). **Copy P&L is the real copyability signal** — - position win% over-counts scalpers (see FINDINGS "the scalper trap"); the table - defaults to sorting by it. +- `validate_timing.py` — the **copyability selection** (rewritten 2026-06-23). + For every conviction wallet it runs a flat-$50 copy replay and keeps only the + ones genuinely profitable to copy: `copy_pnl > 0` **and** a real hold-to- + resolution edge (`held_pnl > 0`, held win-rate ≥55% over ≥8 resolved held bets), + active in 30d, median lead ≥1h (drops sub-hour snipers). It also precomputes + every stat the dashboard renders — including **`copy_pnl`**, the authoritative + flat-$50 copy P&L (replay their entries, mirror their exits, settle held bets at + CLOB resolution by `token_id`). → **~14 copy-positive holders** in + `watch_sharps.json` (read live by jaxperro.com/trading; the table defaults to + sorting by Copy P&L). -Identifiers for a follow-worthy wallet: on its **≥$200 bets** — win ≥65%, avg -entry 0.35–0.70 (edge, not favorites), +copy-ROI, FDR-significant, **median lead -≥24h** (copyable), and it **holds out-of-sample**. +**Copy P&L is the one number that matters for picking copy targets.** It replaced +an earlier lead-time gate. The lesson (see FINDINGS "the scalper trap"): position +**win% over-counts scalpers** — `cache.won` (curPrice ≥ 0.5) scores a sell-at- +profit as a "win", so a wallet can show ~100% conviction win% yet *lose money when +copied* (ArbTrader: 99.5% win, **−$790** copy P&L). Conviction itself = the top 20% +of a wallet's **own** stake sizes, measured at the **position** level (a wallet's +total stake in a market, not per-trade — a scalper splits one position over many +small buys). Judge by Copy P&L, never win%. -## Daily (`daily.sh`) +## Paper portfolio (`portfolio.py` → `portfolio.json`) -1. discover (enumerate last 14d) → 2. freshen cache (force-refresh watchlist + -top up new wallets) → 3. re-score (instant from cache) → 4. regenerate dashboard -+ snapshot to `history/`. Schedule via launchd/cron (Mac must be awake). +A $1,000 paper book that mirrors a chosen set of wallets' conviction bets at a +flat $50/trade, **computed off the cache** (not client-side). The cache stores +each bet's resolution time (`res_t`), so capital **recycles at the true resolution +moment** — fixing the phantom capital-lock the old browser replay suffered (it +missed resolution dates for high-volume wallets and skipped bets it could afford). +Edit the `WALLETS` list at the top of `portfolio.py` to change who's followed. +Output `portfolio.json` (equity, splits, current/resolved/missed tables, per- +wallet) is read by the dashboard in one request. Hold-to-resolution model (the +cache has no sell events — the right model for the holders we select); a small +live `/positions` pull supplies the "current open bets" panel. + +> **Caveat carried through the whole stack:** every P&L here is *idealized* — no +> slippage, fees, or copy lag, and the wallets were selected in-sample. Treat the +> headline % as a ceiling, not a forecast (the project's own history: an in-sample +> copy backtest hit +168% then collapsed out-of-sample). + +## Dashboard feeds (jaxperro.com/trading) + +The dashboard (in the `jaxperro` repo, `trading/index.html`) is static and reads +two precomputed JSONs from this repo via raw.githubusercontent: + +- **`watch_sharps.json`** — the sharps table (copy-positive holders + per-wallet + Copy P&L, win%/record, avg bet, leads). +- **`portfolio.json`** — the $1k paper book up top. + +Both are committed + pushed by `daily.sh`, so the page is just a renderer (no +per-wallet API calls). It keeps a client-side replay as a fallback if the feed is +unreachable. + +## Copy execution (separate, in progress) + +Actually *placing* the copied trades lives outside this finder — `copybot.py` + +`sync_floors.py` (a teammate's work). `sync_floors.py` recomputes each followed +wallet's p80 conviction floor from the fresh cache into `../config.json` so the +bot's entry gate stays in parity with the dashboard's top-20%-by-stake definition. +This finder's job is selection + tracking; execution is downstream. + +## Daily (`daily.sh`, launchd 10:00) + +1. **discover** (`enumerate.py 14`) — recent liquid markets → candidate pool. +2. **freshen** — force-refresh the watchlists, then `collect.py` tops up new/stale. +3. **re-score** (`skill.py`) → `watch_skilled.json`. +4. **sharps** (`conviction_scan.py` + `validate_timing.py`) → `conviction_wallets.json` + + `watch_sharps.json` (the copy-positive holders). +5. **floors** (`sync_floors.py`) — copy-bot conviction floors → `../config.json` (local). +6. **portfolio** (`portfolio.py`) → `portfolio.json` (the $1k paper book). +7. **dashboard** (`dashboard.py`) + snapshot to `history/`. +8. **publish** — commit + push the JSON feeds, then ping Discord (`daily_webhook` + in the gitignored `../config.json`). + +Schedule via launchd/cron (Mac must be awake). ## Usage