diff --git a/FINDINGS.md b/FINDINGS.md index c3bf06b0..ee833f66 100644 --- a/FINDINGS.md +++ b/FINDINGS.md @@ -353,6 +353,115 @@ The backtest is in-sample (a ceiling), but every Set D wallet also clears the out-of-window Copy P&L signal — that's what separates it from curve-fitting. The live July book remains the only out-of-sample truth. +*Superseded the same day by Set E, after the alignment audit below found the +replay itself was still dropping and mislabeling bets.* + +## Aligning the three books — backtest ↔ bot ↔ Polymarket (2026-07-08) + +The live bot showed Kruto2027 mirror-sells the backtest didn't have. Pulling +that thread found the replay was **silently losing real bets** three ways: + +1. **Stale entry maps** — the daily freshen reset the bets and exits cursors + but never `pulled_entries` (14-day TTL), so any market a wallet first + entered since the last entries pull had no `first_buy` and the replay + dropped its bets entirely (`if not et: continue`) — not won, not lost, not + open. *Gone.* The recovered bets included hidden LOSSES — the stale + backtest was flattering. +2. **`res_t` can't detect in-play sells** — it's endDate metadata (game-day + midnight, sometimes future), so the `exit < res_t − 300` test never fired + on in-play markets and every pre-resolution sell booked as + held-to-resolution. Fixed with the price test: a redeem prints exactly the + payout; a mid print is a sell (booked at the wallet's actual exit price — + which also self-corrects poisoned `won` flags). +3. **Resolved round trips vanished** — the round-trip path skipped + resolved-on-chain markets assuming "the cache row will cover it", but rows + with bogus forward `res_t` never qualify. Now redeem-closes book at chain + truth and sell-closes mirror the sell. + +Plus one parity fix: Set wallets now replay on the bot's **pinned floors** +(copybot.paper.json), not a recomputed p80 that drifts a few percent and takes +different bets. + +**Proof of alignment:** matching every settled bet in the bot's real book +against the backtest row-by-row — **7/9 agree exactly, 0 absent** (was 0/9 +agree before the fixes). The 2 disagreements are genuine execution divergence +(the sharp sold on a fast-resolving market, the bot couldn't catch the exit +and rode to resolution) — each book correctly records what happened *to it*, +and that divergence class is permanent. + +**The honest price:** the 30d Set D replay fell **$29.1k → $17.4k** as the +flattering artifacts (phantom held-to-$1.00 winners, missing salvage exits, +hidden losses) came out. + +**Set E (deployed 2026-07-08):** with the replay finally honest, all 35 sharps +were re-ranked by individual 30d copy replay, and combined sets tested in a +shared book (one position per market, shared cash): + +| set | equity (30d, $1k) | note | +|-----|-------------------|------| +| top-4 only | $12,777 | pruning alone loses carry | +| Set D (control) | $17,362 | | +| **Set E (7)** | **$24,378 (+2338%)** | every member positive | +| Set E + lma0o0o0o | $24,437 | carries a −$1,775 wallet — rejected | + +**Set E = LSB1, imwalkinghere, Kruto2027, 0xbadaf319 + gkmgkldfmg, AIcAIc, +1kto1m.** Dropped: 42021 (+16% on 22 bets), BikesAreTheBikes (+12%). Rejected +on the audit evidence: oliman2 (true lifetime ~$19k, not PM's $112k; +21% to +copy with 22 stuck-open) and leegunner (elite lifetime +$274k but **negative** +to copy — 7.6-day holds kill compounding). Newcomer caveat: gkmgkldfmg +(z=2.05) and 1kto1m (z=2.4) sit near the selection gate floor — their seats +are earned on a strong month, not deep statistical edge; AIcAIc's held-win is +only 42% (his profit is sell-timing, the most lag-fragile edge class). They +are the demotion watch-list, in that order. + +## The refund harvesters — a new sharp archetype (2026-07-08) + +Splitting SOLD out of the record columns (same W/L/R/S taxonomy as the bot and +backtest; win% is now held-outcomes only) exposed something the sign-based +tally had been calling "wins": two of the highest-z wallets in the list are +**refund-harvesting machines**. The signature is exits at *exactly* $0.50 to +float precision — only refund redemptions print there — at enormous scale: + +- **0xb0E43B…** (z=20.2, "94.4% all-time" under the old tally): 797 exact-0.5 + redeems = **$148k of his $218k lifetime P&L**. True held record: 76W–31L. +- **ArbTraderRookie** (z=29, "99%"): 1,150 of the same. + +The strategy: buy ITF tennis totals just under 50¢, harvest the chronic ITF +cancellation/retirement rate (50/50 refunds pay $0.50/share). The edge is +real, clever, and **structurally uncopyable** — it clears 1–2¢/share and a +taker copy pays ~0.75¢ fee each way plus slippage into it (honest replay: +6% +and −5% respectively). This closes the loop on the project's oldest lesson: +*win rate lies, z finds real skill, and only the fee-and-lag replay decides +whether the skill transfers to a follower.* + +Also fixed in the same pass: `_open_split` now classifies decided-unredeemed +positions by the data-api's **`redeemable` flag** (exact, set at on-chain +resolution for winners and losers alike) instead of price-pinning — verified +by reproducing PM's per-position books to the dollar on the two biggest +All-Time-vs-PM divergences. + +## The calibration experiment (started 2026-07-08) + +Everything above makes the *accounting* honest. It does not make the +2338% +**forecast** honest: Set E is an in-sample maximum (ranked and assembled on +the same 30-day window it's scored on — winner's curse applies), and the +replay's fill model (their price +0.5%, always filled) ignores adverse +selection — the market moves fastest on exactly the bets where the sharp knew +something, and thin ITF books won't always fill a FAK copy at size. The one +piece of measured ground truth — the old paper book's +$229.79 (~23%) over two +buggy weeks vs. four-figure replay percentages for the same era — says the +live-to-model discount is large. + +So the paper book was **reset to a fresh $1,000 on 2026-07-08** (old book +archived in git history + `archive/copybot_fills.pre-reset-2026-07-08.jsonl`) +running Set E with every fix live from day one. **The measured ratio between +this book and the published backtest over the coming weeks is the number that +sizes real money** — not the replay percentage. Bank-size note for that +decision: the replay compounds *faster* on smaller banks (`--bank 500` → ++2728% vs $1k's +2103%) because 4%-of-equity stakes hit the never-bigger-than- +their-bet ceilings later — percentage returns from small books are the most +optimistic view, discount accordingly. + ## Repo layout - `insider.py` — the detector: z-score/p-value, timing/freshness/sizing signals, diff --git a/HANDOFF.md b/HANDOFF.md index 71b1eaf9..a6e6d10c 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,98 +1,83 @@ -# Session handoff — 2026-07-08 (rev 2: Phase 1 COMPLETE) +# Session handoff — 2026-07-08 (rev 3: books aligned, Set E live, fresh $1k) -Self-contained pickup for a fresh session. Read [README.md](README.md) gotchas -1–11, [FINDINGS.md](FINDINGS.md), and [LIVE_ROLLOUT.md](LIVE_ROLLOUT.md) first. -Project memory (`polymarket-resolution-truth`, `polymarket-us-expansion`) has -deeper history; the repo is authoritative. +Self-contained pickup for a fresh session (human or AI). Read +[README.md](README.md) gotchas 1–15, [FINDINGS.md](FINDINGS.md) (especially +the three 2026-07-08 sections at the end), and [LIVE_ROLLOUT.md](LIVE_ROLLOUT.md). +The repo is authoritative; project memory has deeper history. ## Where things stand (all green) -- **Live paper bot**: Fly.io app `wwf-copybot`, region `arn` (Stockholm), - push mode. Following **Set E** (2026-07-08 PM): LSB1, imwalkinghere, - Kruto2027, 0xbadaf319, gkmgkldfmg, AIcAIc, 1kto1m — 7 volume wallets chosen - by the ALIGNED honest replay (Set E $24.4k/+2338% vs Set D $17.4k/+1636% - 30d; every member positive in the shared book; 42021/BikesAreTheBikes - dropped as mid-pack, oliman2/leegunner excluded by the anti-survivorship - audit — oliman2's true lifetime is ~$19k, not PM's $112k; leegunner is - elite lifetime but replays NEGATIVE for a copier, 7.6-day holds). Caveat: - newcomers gkmgkldfmg z=2.05 / 1kto1m z=2.4 sit near the selection gate - floor — watch their forward month. `flyctl logs --app wwf-copybot`. -- **P&L is honest end-to-end** (sharps All-Time = realized track record + - abandoned-loser fold-in; Open P&L = in-flight exposure — see rev 1 in git - history for the full derivation). -- **Ledger reconciles** (`ledger_drift` 0.00, self-checked every heartbeat AND - after every trade — see below). -- **Daily pipeline** (launchd 08:00) runs the corrected code; exits cache - incremental (86/96 wallets `complete`). +- **Live paper bot**: Fly.io `wwf-copybot`, region `arn` (Stockholm), push + mode, ONE machine. Following **Set E** (7 volume wallets): LSB1, + imwalkinghere, Kruto2027, 0xbadaf319, gkmgkldfmg, AIcAIc, 1kto1m. + `flyctl logs --app wwf-copybot`. +- **The book is a fresh $1,000, reset 2026-07-08 ~20:45 UTC** — this is the + CALIBRATION EXPERIMENT: measure the live-to-backtest ratio for Set E with + every fix live from day one; that ratio (not the replay %) sizes real money. + Old book (Jun 25 → Jul 8, +$229.79) is in git history + + `archive/copybot_fills.pre-reset-2026-07-08.jsonl`. +- **The three books agree by construction** (2026-07-08 alignment work): + same W/L/R/S taxonomy (win% = held-to-resolution only), same + sold-vs-redeem price discriminator (redeem prints exactly the payout), same + pinned floors, same follow set. Row-level cross-check bot↔backtest: 7/9 + exact, 0 absent, 2 legit execution diffs. Sharps table reproduces PM's + per-position books to the dollar; PM's LEADERBOARD is the biased number. +- **Dashboards**: [jaxperro.com/trading](https://jaxperro.com/trading) = + paper book + backtest + sharps; [jaxperro.com/live](https://jaxperro.com/live) + = REAL MONEY page (NOT STARTED until Phase 2). Shared renderer: + `jaxperro/trading/copybot-section.js`. `?botFeed=`/`?rmFeed=` query params + override feeds for local testing. +- **Daily pipeline** (Mac launchd 08:00, fires on wake — typically ~10:00) + re-derives everything under the new rules: entries cursors now invalidate + daily, `_open_split` folds by the `redeemable` flag, `rtally` splits SOLD. +- **Backtest**: `portfolio.py` takes `--bank` (backtest.json `"bank"`). + Published feed stays at $1,000 to match the paper book. $500 run: +2728% + (small banks compound faster — their-bet ceilings bind later). -## DONE 2026-07-08 (this session) — the proper fix + Phase 1 closeout +## Watch list (the standing question: does Set E hold up?) -**Orphan/cash-debit seam CLOSED (Option A, commit 167a0b9).** The mechanism -that created the $+36.35 drift orphan is fixed, not just guarded: - -- `on_wallet_activity` now records EVERY drained buy fill: the tok-match keeps - `_record_lag` (real lag/slippage); leftovers get `_record_untracked_buy` - (bet record synthesized from the fill, conds from the position, audit line - with `"untracked":true`). `reconcile_exits`' drain does the same instead of - discarding returned buys. -- `check_book()` asserts the invariant **after every trade and at boot**: - every `my_pos` token ⇒ bet record + conds entry + cash debited. Records and - conds self-correct in place. The cash leg heals ONLY at boot - (`check_book(heal_cash=True)`, called after `bot.seed()`) and ONLY when - `ledger_drift` matches ONE un-vouched position's cost+fee within - max($0.10, 1%) — vouched = fills-ledger BUY line for the token, or a real - `_record_lag` record (`their_price` set). A drift that matches nothing stays - a loud `⚠ LEDGER DRIFT`, never papered over. -- Supporting plumbing: `my_pos` entries carry `cond` (engine stamps it on - OPEN/ADD); fills-ledger BUY lines carry `token`; `_synth_bet` (shared with - `write_feed`) estimates the fee with the same `taker_fee` formula - `_drain_fills` charges, so a synthesized record over correctly-debited cash - closes to drift 0 — and a never-debited orphan shows drift = cost+fee - exactly, which is what the heal keys on. -- Tested: 6-case harness (untracked booking, synth+conds, matching heal, - non-matching refusal, vouched exclusion, multi-orphan refusal) all pass; - dry-run against the real state was a clean no-op (drift $0.0055, nothing - touched). Deployed; heartbeats clean. - -**Phase 1 is 8/8 DONE** (LIVE_ROLLOUT.md): 1.1 feed separation, 1.2 env -secrets, 1.3 live deps in the worker image (py-clob-client==0.34.6 + -web3==7.16.0 in fly.Dockerfile, import-verified via `flyctl ssh console`), -1.4 on-chain cash anchor, 1.5 fatal geocheck, 1.6 exit retries, 1.7 config -sync, 1.8 dashboard REAL MONEY section (jaxperro `trading/index.html` commit -871e56e — fourth board chip + section reading `live/copybot_live_real.json`, -shared `copybotSection()` renderer, ledger drift/adjustments surfaced on both -books, verified against a mode:"live" feed copy and the 404 NOT STARTED -empty-state; `?rmFeed=`/`?botFeed=` query params override feeds for testing). +- **AIcAIc** — held-win only 42%; his edge is sell-timing (most lag-fragile + class). First demotion candidate. +- **1kto1m** (z=2.4) and **gkmgkldfmg** (z=2.05) — near the gate floor; + strong month, shallow statistical edge. +- Re-ranking is cheap: the per-wallet replay harness pattern is in FINDINGS + ("Aligning the three books"); `python3 live/portfolio.py --wallets 0x… + --days 30 --out /tmp/w.json` per candidate, then combined-set runs. +- Rejected-with-evidence (don't re-add without new facts): oliman2 (true + lifetime ~$19k), leegunner (negative to copy), 0xb0E43B/ArbTraderRookie + (refund harvesters — real edge, uncopyable margins), lma0o0o0o (negative in + the shared book), Winnertraders (6.6-day capital locks). ## Next: Phase 2 — funding (USER ONLY) -Everything before real money is done. Per LIVE_ROLLOUT.md: fund the live -wallet (test size, rule 0.6 caps stay at $5/trade · $25/day · $30 exposure), -then Phase 3 preflight (`preflight_live.py`), Phase 4 supervised first fill, -Phase 5 edge-case matrix. The typed-phrase interlock (rule 0.7) is the human -checkpoint — never automated. +Phase 1 is 8/8 complete (LIVE_ROLLOUT.md). Before funding, the user should +decide the stranding tolerance explicitly: real money turns the primary risk +from P&L into the ACCOUNT (US person routing orders via Stockholm through the +geoblock — a frozen wallet is a different loss category than a losing bet). +Caps stay $5/trade · $25/day · $30 exposure until Phase 6. Then Phase 3 +preflight (`preflight_live.py`) → Phase 4 supervised first fill → Phase 5 +edge-case matrix. The typed-phrase interlock is the human checkpoint — never +automated. ## Standing to-dos (user-only or external) -- **ROTATE THE LEAKED ALCHEMY KEY.** A real key briefly hit the public repo - (`config.live.example.json`, removed from HEAD but in git history). Generate a - new key at dashboard.alchemy.com, then update `alchemy_key` in the local - gitignored `config.json` AND the `ALCHEMY_RPC_URL` Fly secret - (`flyctl secrets set ALCHEMY_RPC_URL=...`). Blocks nothing but is a live key. -- Delete the stopped Railway project `magnificent-kindness` when comfortable - (saves the idle spend; the Fly worker fully replaced it). +- **ROTATE THE LEAKED ALCHEMY KEY** (config.live.example.json git history). + New key at dashboard.alchemy.com → update gitignored `config.json` AND + `flyctl secrets set ALCHEMY_RPC_URL=...`. +- Delete the stopped Railway project `magnificent-kindness` when comfortable. ## Operational quick-reference -- Change the follow set: edit `live/copybot.paper.json` `wallets`, then +- **Change the follow set**: edit `live/copybot.paper.json` wallets → `./live/deploy_bot.sh` (validates → pins floors → syncs Alchemy webhook → - restarts Fly → confirms banner). `backtest.json` should mirror it. -- Backtest any set: `python3 live/portfolio.py --wallets 0xa,0xb --days 30 - --out /tmp/t.json`. -- Never keep 2 Fly machines (2 book-writers): `flyctl scale count 1`. -- Cache is single-writer; don't run cache-touching scripts while the daily - pipeline or a regen is running. -- Bot commits its own state/feed every few minutes — always - `git pull --rebase --autostash` before pushing. -- Image changes (fly.Dockerfile) need `flyctl deploy --remote-only`; code-only - changes just need a push + `flyctl apps restart` (start.sh clones main). + restarts Fly → confirms banner). Keep `backtest.json` mirrored. +- **State surgery (reset/repair the book)**: `flyctl machine stop` FIRST, + push the new state, `flyctl apps restart`, then VERIFY the first heartbeat + shows the book you wrote (gotcha 15: the bot's memory wins conflicts by + design, and a boot clone can read a stale GitHub replica for a few seconds). +- Never 2 Fly machines (`flyctl scale count 1`) — one book, one writer. +- Cache is single-writer; don't run cache-touching scripts during the daily + pipeline or a regen. Three cursors per wallet (gotcha 12). +- Bot commits its own state/feed — `git pull --rebase --autostash` before + pushing. Image changes (fly.Dockerfile, host/) need + `flyctl deploy --remote-only`; code/config changes just push + restart. diff --git a/LIVE_ROLLOUT.md b/LIVE_ROLLOUT.md index 1c49ad4c..124b61fa 100644 --- a/LIVE_ROLLOUT.md +++ b/LIVE_ROLLOUT.md @@ -2,7 +2,7 @@ Written 2026-07-07 after the pre-money audit. This document is **self-contained**: a session with no prior context (Sonnet/Opus) should be able to execute every -phase from here + the repo. Read [README.md](README.md) gotchas 1–11 and +phase from here + the repo. Read [README.md](README.md) gotchas 1–15 and [LIVE_TEST.md](LIVE_TEST.md) first. Project memory (`polymarket-resolution-truth`, `polymarket-us-expansion`) has deeper background; the repo is authoritative. diff --git a/README.md b/README.md index 5f15cc7c..944680ca 100644 --- a/README.md +++ b/README.md @@ -26,22 +26,24 @@ Three deployed pieces + one static dashboard: | **daily pipeline** (`live/daily.sh`) | this Mac, launchd **08:00** (runs on wake if the Mac was asleep) | refresh the bet cache → 5-gate skill scan → fee-aware sharp selection → conviction floors → backtest book → publish JSON feeds to GitHub | | **copybot worker** (`copybot.py` via `host/start.sh`) | **Fly.io app `wwf-copybot`, region `arn` (Stockholm), 24/7** — migrated off Railway 2026-07-06: Railway ran it in a US region, which Polymarket's IP geoblock would 403 the moment orders got real; Stockholm is unrestricted AND ~25ms from the CLOB's eu-west-2 primaries. Every boot self-checks the geo-gate (`host/geocheck.py`) | **push mode**: an Alchemy address-activity webhook (`copybot follow set`, Polygon) pings `POST /alchemy` the moment a followed wallet trades (~2–5s detection), signature-verified; a 60s heartbeat settles/publishes and a 5-min backstop poll catches dropped pushes. Paper-copies with real fees/lag/slippage, settles at CLOB resolution, commits its book back to the repo | | **Discord digest** (`live/discord_daily.py`) | end of the daily pipeline | one message/day: the sharp list with profile links + 30-day conviction stats (per-trade pings retired 2026-07-04; the old Alchemy watcher lives in `archive/webhook_receiver.py`) | -| **dashboard** | [jaxperro.com/trading](https://jaxperro.com/trading) (static, in the `jaxperro` repo) | renders the three JSON feeds: live bot book, backtest book, sharp table | +| **dashboard** | [jaxperro.com/trading](https://jaxperro.com/trading) + [jaxperro.com/live](https://jaxperro.com/live) (static, in the `jaxperro` repo) | `/trading` renders the paper book, backtest book and sharp table; `/live` is the REAL MONEY page (reads `live/copybot_live_real.json`, NOT STARTED until Phase 2). Both pages share `trading/copybot-section.js` — one renderer, no drift | -**The July 2026 live test:** a fresh $1,000 paper book (started 2026-07-02 on -Railway; moved to Fly.io Stockholm 2026-07-06 with the book intact) following -the wallet set in **`live/copybot.paper.json`** — the single source of truth. -The follow set is **Set D** (2026-07-08): six moderate-bet, copyable, copy-P&L- -positive wallets — **LSB1, imwalkinghere, Kruto2027, 42021, 0xbadaf319, -BikesAreTheBikes** — chosen by simulation over the corrected data (see FINDINGS -"Choosing the month's follow set"). All are **volume** class: copied on their -**conviction bets only** (top-20%-by-stake, floor pinned daily from the trusted -cache p80 via `sync_floors.py`), 4% of equity/bet, **capped at the signal's own -bet size**. The whale class (12%/bet, follow-all) is retired — those wallets -fell off the sharp list once refunds and abandoned losers were scored honestly. +**The calibration experiment (running now):** a fresh $1,000 paper book, +**reset 2026-07-08** so it measures exactly one thing — the follow set in +**`live/copybot.paper.json`** (the single source of truth) from a clean start, +with every bookkeeping fix live from day one. The follow set is **Set E** +(2026-07-08): seven moderate-bet volume wallets — **LSB1, imwalkinghere, +Kruto2027, 0xbadaf319, gkmgkldfmg, AIcAIc, 1kto1m** — chosen by the *aligned* +honest replay (see FINDINGS "Aligning the three books"). All are copied on +their **conviction bets only** (top-20%-by-stake, floor pinned daily from the +trusted cache p80 via `sync_floors.py`), 4% of equity/bet, **capped at the +signal's own bet size**. The whale class (12%/bet, follow-all) is retired. Every fill records detection lag, price slippage, and the taker fee; missed -bets are recorded and settled hypothetically. If this month's *measured* -numbers hold up, real money follows (see [`LIVE_ROLLOUT.md`](LIVE_ROLLOUT.md)). +bets are recorded and settled hypothetically. **The point of the fresh book: +the backtest of the same set says +2103%/30d — an in-sample ceiling. The +measured ratio between this live book and that model over the next weeks is +the number that decides real-money sizing** (see [`LIVE_ROLLOUT.md`](LIVE_ROLLOUT.md) +for the phased path; Phase 1 code prep is complete, Phase 2 is funding). ``` data layer selection execution display @@ -63,10 +65,11 @@ numbers hold up, real money follows (see [`LIVE_ROLLOUT.md`](LIVE_ROLLOUT.md)). | task | how | |------|-----| | **add / remove / reclass a LIVE wallet** | edit the `wallets` list in `live/copybot.paper.json` (`{"wallet","name","class":"volume"\|"whale","floor":123?}` — floor optional, auto p80 at boot; whales ignore floors) then run **`./live/deploy_bot.sh`** — it validates, previews, **syncs the Alchemy webhook's address list** (`live/sync_webhook.py`, Notify API), commits, pushes, restarts the Fly machine, and confirms the boot banner. Fully self-contained — nothing to click in any dashboard | -| **backtest any wallet set** | edit `live/backtest.json` (same entry shape) → `python3 live/portfolio.py`; ad-hoc without touching the dashboard: `python3 portfolio.py --wallets 0xabc,0xdef:whale --days 14 --out /tmp/t.json` | +| **backtest any wallet set** | edit `live/backtest.json` (same entry shape) → `python3 live/portfolio.py`; ad-hoc without touching the dashboard: `python3 portfolio.py --wallets 0xabc,0xdef:whale --days 14 --bank 500 --out /tmp/t.json` (`--bank`/backtest.json `"bank"` sets the starting bankroll; smaller books compound at a *higher* rate because 4%-of-equity stakes hit the their-bet ceilings later) | | **promote a wallet to live** | prove it in `backtest.json` first, copy the same entry into `copybot.paper.json`, run `deploy_bot.sh` | | **watch the live bot** | `flyctl logs --app wwf-copybot` (one summary line per 60s heartbeat); the book is also committed as `live/copybot_live.json` and rendered on the dashboard | | **restart / redeploy the bot** | `flyctl apps restart wwf-copybot` (config/code changes: the worker clones the repo fresh at boot, so a restart IS the deploy). **Changes to `host/` or `fly.toml` need an image rebuild: `flyctl deploy --remote-only`.** Secrets: `flyctl secrets set K=V` (applies with an automatic restart). Keep it ONE machine — Fly loves creating a second for "high availability", and two bots = two writers on one book (`flyctl scale count 1`) | +| **reset the paper book** | **stop the machine FIRST** (`flyctl machine stop --app wwf-copybot`), then write a fresh `new_state()` into `copybot_state.json`, archive `copybot_fills.jsonl`, commit+push, then `flyctl apps restart`. Order matters: the running bot's memory is the book's single writer and its publish flow will re-commit its own book over yours (by design — see gotcha 15). Verify the first heartbeat reads `free $1,000/$1,000 · realized $+0` | | **run the daily pipeline manually** | `cd live && bash daily.sh` (launchd runs it 08:00; ~40 min, mostly collect). Never run two at once — the cache is single-writer | | **refresh just the sharp list** | `cd live && python3 conviction_scan.py && python3 validate_timing.py` | | **daily Discord digest** | sent by `live/discord_daily.py` at the end of `daily.sh`; webhook = `daily_webhook` in gitignored `config.json` | @@ -311,7 +314,51 @@ runner is retired (GitHub throttled `*/5` to ~2h in practice — it copied 1 of reconstruction (`won × entry × size`), which diverged by up to 10× and flipped signs — four bugs killed with it: the 2,000-row pull cap, both-sides double-drop, `initialValue = 0` mis-sizing, and corrupt near-epoch `res_t`. - Win % is the share of decided positions that *made money*. + **Win % is held-outcomes only** (see gotcha 13); sold P&L still counts in + the P&L columns. And when our number disagrees with PM's **leaderboard** + (`lb-api /profit`), check PM against itself first: for mega-abandoners the + leaderboard doesn't foot against PM's own per-position books (oliman2: + per-position sums say ~$19k true; leaderboard says $112k). The per-position + data is the truth source, never the leaderboard. + +12. **The cache keeps THREE per-wallet cursors** — `pulled` (bets), + `pulled_exits` (incremental), `pulled_entries` (14-day-TTL full snapshot) — + and **anything not re-pulled goes silently stale, not visibly wrong.** + `cache.invalidate()` must clear ALL of them (it does, since 2026-07-08): + when it only cleared `pulled`, watchlist wallets' entry maps went up to 14 + days stale and `portfolio.py` silently dropped every bet on a market first + entered since the last pull (`if not et: continue`) — Kruto's Brewers sells + were visible in the live bot and absent from the backtest for exactly this. + +13. **`res_t` is endDate METADATA, not resolution time** — for in-play markets + it's game-day midnight (it can pre-date the wallet's own entry) or even a + date in the future (a Jul-5 tennis match carried res_t Jul-14). Never use + it to detect a pre-resolution sell. The reliable discriminator is PRICE: a + redeem prints **exactly** the payout (1 / 0 / 0.5 to float precision — the + exit_p reconstruction is exact for redeems), a real sell prints a mid + price. `portfolio._sold_pre_resolution` and validate_timing's `rtally` + both use it; that's what makes W/L (held to resolution) vs S (sold) vs R + (refund) mean the same thing in the sharps table, the backtest, and the + bot. In `/positions`, the **`redeemable` flag is the resolution truth** + (set on-chain-resolution for winners AND losers) — `_open_split` folds on + it, not on price-pinning. + +14. **A high z-score can be a refund-harvesting machine, not a picker.** The + exact-0.5 exit signature at scale = buying ITF totals just under 50¢ and + banking the chronic ITF cancellation rate (0xb0E43B: $148k of a $218k + lifetime P&L is 797 refund redeems; ArbTraderRookie is the same species). + The edge is real but pays 1–2¢/share — taker fees + slippage consume it, so + it never survives the copy replay. General rule: **z finds skill; only the + honest replay decides whether the skill is harvestable by a follower.** + +15. **The running bot is the book's single writer — remote state surgery must + stop the machine first.** The publish flow recovers from conflicted + rebases by resyncing to origin and re-committing ITS OWN in-memory book on + top (so a wedge can't silently kill publishing — the 2026-07-08 BOOK RESET + race left the repo in UU and every publish dead until reboot). Two + corollaries: a state push while the bot runs WILL be overwritten, and a + boot clone seconds after a push can read a stale GitHub replica — after + any surgery, verify the first heartbeat shows the book you wrote. ---