cb438fdf4e
copybot: live paper feed [skip ci]
2026-07-22 05:00:53 +00:00
83389e71bd
copybot: live paper feed [skip ci]
2026-07-22 01:16:36 +00:00
9f6b30dcd5
copybot: live paper feed [skip ci]
2026-07-22 01:15:43 +00:00
0af9c83f78
copybot: live paper feed [skip ci]
2026-07-22 01:07:07 +00:00
61ed166d6b
copybot: live paper feed [skip ci]
2026-07-22 01:05:51 +00:00
77fdb6b422
copybot: live paper feed [skip ci]
2026-07-22 00:31:23 +00:00
44d85b73ad
copybot: live paper feed [skip ci]
2026-07-22 00:16:12 +00:00
ca859acf57
copybot: live paper feed [skip ci]
2026-07-22 00:14:41 +00:00
84450f1ea9
copybot: live paper feed [skip ci]
2026-07-21 23:11:23 +00:00
3a9fcba4c1
copybot: live paper feed [skip ci]
2026-07-21 23:11:17 +00:00
a57d8aaec1
copybot: live paper feed [skip ci]
2026-07-21 22:25:38 +00:00
9e6fd969e8
copybot: live paper feed [skip ci]
2026-07-21 21:50:30 +00:00
77bdddc563
copybot: live paper feed [skip ci]
2026-07-21 21:49:17 +00:00
b889830c17
copybot: live paper feed [skip ci]
2026-07-21 20:44:44 +00:00
354b2a938b
copybot: live paper feed [skip ci]
2026-07-21 20:44:09 +00:00
aad2577074
copybot: live paper feed [skip ci]
2026-07-21 20:37:24 +00:00
63e6308c50
copybot: live paper feed [skip ci]
2026-07-21 19:50:02 +00:00
bda2e48e72
copybot: live paper feed [skip ci]
2026-07-21 19:49:33 +00:00
e8eb0a7dbd
copybot: live paper feed [skip ci]
2026-07-21 19:08:11 +00:00
c0caff58f0
copybot: live paper feed [skip ci]
2026-07-21 19:08:08 +00:00
e5e212a0bb
copybot: live paper feed [skip ci]
2026-07-21 19:05:57 +00:00
8cf5c65594
copybot: live paper feed [skip ci]
2026-07-21 18:55:02 +00:00
1877478be7
copybot: live paper feed [skip ci]
2026-07-21 18:33:35 +00:00
ff3062ca92
copybot: live paper feed [skip ci]
2026-07-21 18:33:07 +00:00
d0e4fc4c0b
copybot: live paper feed [skip ci]
2026-07-21 18:22:02 +00:00
df364a1ed9
copybot: live paper feed [skip ci]
2026-07-21 18:20:17 +00:00
b3bbb05ca9
copybot: live paper feed [skip ci]
2026-07-21 18:19:50 +00:00
8add61be3b
copybot: live paper feed [skip ci]
2026-07-21 17:36:27 +00:00
935db9a906
copybot: live paper feed [skip ci]
2026-07-21 17:35:45 +00:00
8edccda58a
docs: stage-0 coherence pass across README/HANDOFF/recorder/research
...
- README: recorder row + architecture diagram reflect fold->parquet->
15-min mirror (and research/ joins the diagram); cheat-sheet gains a
'sync the tape now' row
- HANDOFF: snapshot dated 2026-07-21, launchd roster (tape-sync +
research-nightly wait behavior), #15 back in the open queue, tape ops
line in quick-reference
- recorder/README: ops block (sync_tape + fold health, deploy gap note),
storage economics (/mo, mirror as Stage-1 feedstock)
- research/README: nightly.sh wait-for-fresh-tape semantics
- live/tape_sharps.json: refreshed 4-day chain-validated screen (the
smoke run had briefly left a chain-less version)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-21 13:07:31 -04:00
b5166630fd
copybot: live paper feed [skip ci]
2026-07-21 17:06:05 +00:00
9b17928b27
copybot: live paper feed [skip ci]
2026-07-21 17:04:37 +00:00
56cefc80f0
copybot: live paper feed [skip ci]
2026-07-21 17:04:37 +00:00
5c8ee5961b
stage-0 warehouse: recorder folds its own tape to parquet; Mac mirrors every 15 min
...
The nightly Mac-coupled bulk ingest was the fragile link (today: Mac slept
through 08:00, the pipeline started 09:31, and the sftp bulk pull wedged in
timeout cascades holding the db lock — digest blocked behind it). Stage 0
moves the fold onto the box and reduces the Mac to an incremental mirror:
- recorder/fold.py (sidecar, capture stays PID 1): closed gz segments ->
zstd parquet /data/parquet/<fam>/date=*/segment.parquet, row-parity
verified, manifest-logged, THEN gz deleted. Deletion invariant STRONGER:
raw needs a verified parquet; parquet needs a Mac ACK before the disk
guard may prune it. duckdb capped 384MB; VM 256MB -> 1GB.
- recorder/sync_tape.py (com.jaxperro.tape-sync every 15 min + daily.sh):
manifest-driven incremental sftp pull, per-file row verify, append into
rtds.duckdb NATIVE tables (views-over-parquet rejected: sim's per-asset
point queries would crawl), segment-keyed idempotence shared with the
legacy ingest.py (kept as fallback), ack back to the box.
- recorder/bootstrap_parquet.py: pre-fold history exported to the mirror,
parity OK (13.84M trades + 3.07M aux). live/parquet/ is now the complete
durable layer Stage 1 (MotherDuck/ClickHouse) would consume.
- research/tape.py connect(): brief retry — the 15-min sync holds the
write lock for seconds.
First run: backlog folded in <60s on the box, 36/36 files mirrored+
verified, rtds.duckdb 13.8M -> 18.2M trades, tape age 24h+ -> ~15-45 min.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-21 12:46:54 -04:00
803df06944
copybot: live paper feed [skip ci]
2026-07-21 16:22:46 +00:00
6892aa5b1e
live: daily refresh — skilled + sharp wallets [skip ci]
2026-07-21 12:19:03 -04:00
6c43d4410e
copybot: live paper feed [skip ci]
2026-07-21 15:52:46 +00:00
76c7ec96ae
copybot: live paper feed [skip ci]
2026-07-21 15:52:22 +00:00
cf78e95db7
copybot: live paper feed [skip ci]
2026-07-21 15:50:37 +00:00
81f46edeae
copybot: live paper feed [skip ci]
2026-07-21 15:50:10 +00:00
935200c901
copybot: live paper feed [skip ci]
2026-07-21 15:47:57 +00:00
dba34bf026
copybot: live paper feed [skip ci]
2026-07-21 15:34:47 +00:00
7497f5e867
copybot: live paper feed [skip ci]
2026-07-21 15:34:20 +00:00
85cc84de0d
copybot: live paper feed [skip ci]
2026-07-21 15:32:10 +00:00
05b17c52ad
copybot: live paper feed [skip ci]
2026-07-21 14:31:00 +00:00
26e2d59b8e
copybot: live paper feed [skip ci]
2026-07-21 14:20:32 +00:00
0b15441ea0
copybot: live paper feed [skip ci]
2026-07-21 14:18:21 +00:00
f98861b5cd
copybot: live paper feed [skip ci]
2026-07-21 14:17:54 +00:00
81abef5636
copybot: live paper feed [skip ci]
2026-07-21 10:40:28 +00:00
23e00ec485
copybot: live paper feed [skip ci]
2026-07-21 10:38:58 +00:00
copybot[bot]