mirror of
https://github.com/jaxperro/winning-wallet-finder.git
synced 2026-07-27 15:57:47 +00:00
jaxperro
5c8ee5961b
The nightly Mac-coupled bulk ingest was the fragile link (today: Mac slept through 08:00, the pipeline started 09:31, and the sftp bulk pull wedged in timeout cascades holding the db lock — digest blocked behind it). Stage 0 moves the fold onto the box and reduces the Mac to an incremental mirror: - recorder/fold.py (sidecar, capture stays PID 1): closed gz segments -> zstd parquet /data/parquet/<fam>/date=*/segment.parquet, row-parity verified, manifest-logged, THEN gz deleted. Deletion invariant STRONGER: raw needs a verified parquet; parquet needs a Mac ACK before the disk guard may prune it. duckdb capped 384MB; VM 256MB -> 1GB. - recorder/sync_tape.py (com.jaxperro.tape-sync every 15 min + daily.sh): manifest-driven incremental sftp pull, per-file row verify, append into rtds.duckdb NATIVE tables (views-over-parquet rejected: sim's per-asset point queries would crawl), segment-keyed idempotence shared with the legacy ingest.py (kept as fallback), ack back to the box. - recorder/bootstrap_parquet.py: pre-fold history exported to the mirror, parity OK (13.84M trades + 3.07M aux). live/parquet/ is now the complete durable layer Stage 1 (MotherDuck/ClickHouse) would consume. - research/tape.py connect(): brief retry — the 15-min sync holds the write lock for seconds. First run: backlog folded in <60s on the box, 36/36 files mirrored+ verified, rtds.duckdb 13.8M -> 18.2M trades, tape age 24h+ -> ~15-45 min. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
95 lines
6.0 KiB
Markdown
95 lines
6.0 KiB
Markdown
# HANDOFF — the living state of the world
|
||
|
||
**The work queue lives in GitHub Issues** (migrated 2026-07-19):
|
||
`gh issue list` · labels = silo (`copybot`/`recorder`/`pipeline`/`dashboard`)
|
||
× type (`hardening`/`bug`/`research`/`strategy`/`ops`/`user-todo`/
|
||
`pre-registered`/`post-mortem`) · milestones = the standing rhythms (audit
|
||
backlog, Friday bench reviews, verdict gates). Commits that resolve one say
|
||
`closes #N`. The July-2026 narrative/evidence log is FROZEN in
|
||
[HANDOFF_ARCHIVE.md](HANDOFF_ARCHIVE.md) — incident post-mortems, decision
|
||
rationale, the whole story. This file stays ~a page: boundary, snapshot, ops.
|
||
|
||
## Issue burndown 2026-07-19 (audit → shipped)
|
||
Migrated the queue to GitHub Issues (labels/milestones; `closes #N` in
|
||
commits). Closed same day: #3 webhook rotated · #5 listeners start after
|
||
seed · #6 dead-code sweep (retired copytrade CLI + LiveExecutor) · #7
|
||
chain-balance gate in reconcile_exits + retry (kills the BetBoom zombie
|
||
class) · #8 rpc-down heartbeat flag · #9 memory caps (skipped recency-
|
||
bounded, settled bets >30d spool to an archive with drift-invariant
|
||
compensation — TESTED identical drift) · #10 publish-failure alarm · #11
|
||
tape ingest sftp-first (base64 fallback) · #12 reconcile_entries 5m→30m.
|
||
REOPENED #4: the redeemer's phantom-cash BOOBY TRAP is fixed + proven
|
||
(on-chain sim SUCCESS; live fails honestly), but actually redeeming needs a
|
||
Builder API Key (SDK gasless relay) the runtime bot deliberately lacks —
|
||
LOW urgency, the platform auto-redeems winners itself. auto_redeem OFF.
|
||
Open now: #1 Signal A, #2 tape research, #4 redeem builder-key, #13 Fri
|
||
bench, #14 edge verdict, #16 surge momentum (forward window OPEN), #17
|
||
oracle fair value (accumulating, nothing frozen). #18 (empty-cond copies
|
||
unsettleable) closed same day: RTDS seed enrichment + falsy-cond repair
|
||
pass + 1h alarm.
|
||
|
||
## Operating boundary (user, 2026-07-13 — standing)
|
||
**Full autonomy on the bots**; the real-money bot **stays ARMED**. Never
|
||
touch the private key, Discord alerts webhook ROTATED 2026-07-19 (#3 closed),
|
||
sizing/caps/deposits are the USER's call. USER DIRECTIVE 2026-07-16: the
|
||
live bot mirrors the paper test EXACTLY (one deliberate exception:
|
||
min_order_usd $1 = venue reality). If something looks genuinely dangerous,
|
||
DISARM (`flyctl secrets unset LIVE_CONFIRM -a wwf-copybot-live`) rather than
|
||
push through.
|
||
|
||
## Snapshot (2026-07-19)
|
||
- **wwf-copybot-live** (REAL, ARMED): ~$62 equity ($66.42 contributed,
|
||
realized −$12.17 lifetime — day-one incident + honest recognitions),
|
||
6-wallet Set E rev 4, 4% of working equity/bet, alarm-free after the
|
||
07-20 reconcile (CASH≠CHAIN −$1.75: empty-cond Odyssey bet the venue
|
||
auto-redeemed → hand-settled won +$0.68, $0.05 rounding folded; root
|
||
cause FIXED same day, #18: RTDS metadata-less rows now enrich at seed,
|
||
repair_market_meta un-sticks cond-less books each settle pass). **wwf-copybot** (paper $1k): same set, FAK-parity fills.
|
||
Both on the audit-hardened build (locks, chain-gated sweep, boot-id
|
||
single-writer guard, TLS'd user-ws — HANDOFF_ARCHIVE rev 16). 2026-07-20:
|
||
FAK no-match OPENs get one re-quote retry on both bots, PER-NICHE waits
|
||
from measured crater-refill times (crypto 4s / esports 10s / sports+slow
|
||
25s — research/params/requote_timing.json; `fak_retry_niche_s` override,
|
||
`fak_retry_s` fallback+kill-switch; second rejection tags "twice").
|
||
- **wwf-recorder**: the FULL firehose (trades + order matches + comments
|
||
+ crypto ticks, ~8M events/day, dual-socket ~99.9% capture, 25GB volume).
|
||
**Stage-0 warehouse 2026-07-21**: the box folds its own segments →
|
||
zstd Parquet partitions on the volume (fold.py sidecar, row-parity
|
||
verified, manifest + Mac-ack deletion protocol — invariant STRONGER than
|
||
before); the Mac mirrors + appends into `live/rtds.duckdb` every 15 min
|
||
(`com.jaxperro.tape-sync` → recorder/sync_tape.py). Tape freshness:
|
||
nightly → ~15 min; the box no longer needs the Mac to stay healthy.
|
||
`live/parquet/` = complete durable layer (Stage-1 MotherDuck feedstock).
|
||
- **VALUE experiment: CLOSED 2026-07-19** — sub-2¢ hypothesis refuted
|
||
(1W/993L, 0.075x); post-mortem in value/PLAN.md; app destroyed.
|
||
- **research/ (NEW 2026-07-20, SILO — never touches the bots)**: tape-era
|
||
edge factory. Sharp screen (`live/tape_sharps.py`, resolution proxy
|
||
chain-validated 742/742, 25 copyable candidates); execution sim fitted on
|
||
our own 29 labeled attempts (−2¢/fill optimism bias, thresholds sit 2×
|
||
above it); Study A surge momentum #16 (identity NULL result — controls
|
||
match the informed set) and Study B oracle fair value #17 (86% craters,
|
||
winner's-curse inversion). Verdicts ONLY from research/forward_ledger.jsonl.
|
||
- **Verdicts pending**: edge/size-up (#14, ~end of July, pre-registered) ·
|
||
bench review Friday (#13: five new sharps + AIcAIc's prove-it week —
|
||
cross-read the tape screen's candidates in live/tape_sharps.json) ·
|
||
#16/#17 forward thresholds (in their issues).
|
||
- Dashboards: jaxperro.com/{trading,live,value} · daily pipeline on the Mac
|
||
at 08:00 (launchd, lockfile) — floors, bench forward table, edge row,
|
||
tape ingest, Discord digest · research nightly at 09:15
|
||
(com.jaxperro.research-nightly → research/nightly.sh, self-commits the
|
||
forward ledger; remove with `launchctl unload`).
|
||
|
||
## Ops quick-reference
|
||
- Follow-set change: edit live/copybot.paper.json → `./live/deploy_bot.sh`;
|
||
mirror config.live.example.json (nothing auto-writes it) + backtest.json.
|
||
- State surgery: stop the machine, **watch heartbeats actually CEASE**
|
||
(gotcha 15c — and the boot-id guard now makes a zombie writer yield),
|
||
pull → edit → push → start → verify the first heartbeat.
|
||
- Code deploy: push + `flyctl apps restart <app>` (boots clone main,
|
||
clone-guard verifies). Image changes: `flyctl deploy --remote-only -c
|
||
<toml> --ha=false`. Never two machines per app.
|
||
- The bots commit their own state — always `git pull --rebase --autostash`
|
||
before pushing from a session.
|
||
- Read next: README.md (architecture + gotchas 1-18) · FINDINGS.md (research
|
||
story) · value/PLAN.md (the refutation) · HANDOFF_ARCHIVE.md (history).
|