mirror of https://github.com/jaxperro/winning-wallet-finder.git synced 2026-07-27 15:57:47 +00:00

Files

T

jaxperro 5c8ee5961b stage-0 warehouse: recorder folds its own tape to parquet; Mac mirrors every 15 min

The nightly Mac-coupled bulk ingest was the fragile link (today: Mac slept
through 08:00, the pipeline started 09:31, and the sftp bulk pull wedged in
timeout cascades holding the db lock — digest blocked behind it). Stage 0
moves the fold onto the box and reduces the Mac to an incremental mirror:

- recorder/fold.py (sidecar, capture stays PID 1): closed gz segments ->
  zstd parquet /data/parquet/<fam>/date=*/segment.parquet, row-parity
  verified, manifest-logged, THEN gz deleted. Deletion invariant STRONGER:
  raw needs a verified parquet; parquet needs a Mac ACK before the disk
  guard may prune it. duckdb capped 384MB; VM 256MB -> 1GB.
- recorder/sync_tape.py (com.jaxperro.tape-sync every 15 min + daily.sh):
  manifest-driven incremental sftp pull, per-file row verify, append into
  rtds.duckdb NATIVE tables (views-over-parquet rejected: sim's per-asset
  point queries would crawl), segment-keyed idempotence shared with the
  legacy ingest.py (kept as fallback), ack back to the box.
- recorder/bootstrap_parquet.py: pre-fold history exported to the mirror,
  parity OK (13.84M trades + 3.07M aux). live/parquet/ is now the complete
  durable layer Stage 1 (MotherDuck/ClickHouse) would consume.
- research/tape.py connect(): brief retry — the 15-min sync holds the
  write lock for seconds.

First run: backlog folded in <60s on the box, 36/36 files mirrored+
verified, rtds.duckdb 13.8M -> 18.2M trades, tape age 24h+ -> ~15-45 min.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-21 12:46:54 -04:00

params

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

forward_ledger.jsonl

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

forward.py

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

nightly.sh

research: nightly waits for fresh tape + absolute python

2026-07-21 11:26:26 -04:00

README.md

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

requote.py

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

robustness_flow.py

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

sim.py

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

study_flow.py

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

study_oracle.py

research silo: edge factory — calibrated sim, Study A/B, forward ledger (#2 , #16 , #17 )

2026-07-20 17:34:22 -04:00

tape.py

stage-0 warehouse: recorder folds its own tape to parquet; Mac mirrors every 15 min

2026-07-21 12:46:54 -04:00

README.md

research/ — the edge factory (SILO)

Standing rules (user directive 2026-07-20: "built in a silo to not affect anything on the live bot"):

Nothing here is imported by, or imports, the bot (copybot.py, copytrade.py, their configs). The Fly workers run pinned entrypoints; this directory is inert to them.
Tape access is read-only (duckdb.connect(..., read_only=True)).
The ONE shared write: live/cache.duckdb::resolutions via live/payouts.py — append-only immutable chain facts, the same store the daily pipeline already feeds. Nothing else in live/ is touched.
The existing paper bot is the live test's CONTROL — graduated edges get their own paper harness here, never that one.
Studies are pre-registered (GitHub issue per study: hypothesis, params, verdict + kill criteria) BEFORE their forward window opens. Exploration happens on already-collected tape; verdicts only come from forward_ledger.jsonl rows dated after the params freeze commit.

Layout: tape.py read-only loaders · tape proxy-resolution (terminal-VWAP + sibling veto, the 742/742 chain-validated method) · title parsers (niche, crypto strike/expiry/sprint) sim.py execution replayer calibrated on OUR live fills ledger (lag, FAK no-match, protected band, 3% taker fee) study_flow.py Study A — informed-flow state signal study_oracle.py Study B — crypto oracle fair value vs the book requote.py crater→requote timing measurement (retry tuning) forward.py scores frozen studies on new tape days → forward_ledger.jsonl params/ frozen study parameters (committed = frozen) nightly.sh manual/launchd runner (separate from daily.sh)