exits: FULL history, no window, no cap (holistic data > speed)

Per the data-completeness call: closed_exits pulls each wallet's ENTIRE
closed-position history, not a 180d window. smart_money.closed_exits returns
(exits, reached_end) — reached_end True only when the pull hits the true
start of history (empty/short page), so the cache's  flag tells a
finished backfill from an interrupted one and a killed deep pull re-completes
instead of falsely reporting done. One-time deep cost, amortized by the
incremental cache (later runs page only new closes). Verified: imwalkinghere
full 65d lifetime (879 exits, its real age not a cap), complete=True, 2nd
call 0.00s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jaxperro
2026-07-07 11:49:17 -04:00
parent 6d59b354ee
commit 2b33ec5683
2 changed files with 54 additions and 50 deletions
+23 -27
View File
@@ -192,43 +192,39 @@ for _col in ("title", "outcome"):
except Exception:
pass # already there
_con.execute("CREATE TABLE IF NOT EXISTS pulled_exits(wallet TEXT PRIMARY KEY, newest_ts BIGINT, pulled_at BIGINT)")
try:
_con.execute("ALTER TABLE pulled_exits ADD COLUMN oldest_ts BIGINT") # backfill-completeness marker
except Exception:
pass
for _col, _type in (("oldest_ts", "BIGINT"), ("complete", "BOOLEAN")):
try:
_con.execute(f"ALTER TABLE pulled_exits ADD COLUMN {_col} {_type}")
except Exception:
pass # already there
def closed_exits(wallet, max_age_s=6 * 3600, window_days=WINDOW_DAYS):
"""{asset: {ts, exit_p, p, iv, cond}} of the wallet's fully-closed
positions over the last `window_days` (default 180 — the same window the
bets cache scores, so the exit overlay COMPLETELY covers every bet the
sharps/backtest stats touch; no arbitrary row cap that could truncate a
hyperactive wallet's window). Shared exit model for the backtest
def closed_exits(wallet, max_age_s=6 * 3600):
"""{asset: {ts, exit_p, p, iv, cond}} of the wallet's FULLY-CLOSED
positions the COMPLETE history, no window and no cap (the exit overlay
covers every bet any stat could touch). Shared exit model for the backtest
(portfolio.py) and the sharps stats (validate_timing.py).
Correct + incremental: `oldest_ts` records how far back the cache is known
complete. If the window isn't fully covered yet (first run, or a backfill
that got interrupted) it re-pulls the whole window down to the target; once
complete, later refreshes only page the NEW closes since `newest_ts` (a
page or two). Close events are immutable, so nothing already cached is
re-fetched. NB the endpoint serves 50-row pages regardless of `limit` —
see smart_money.closed_exits for the paging gotcha this caused."""
Correct + incremental: `complete` marks that the backfill reached the true
start of the wallet's history. Until then (first run, or an interrupted
deep pull) it re-pulls the full history from newest; once complete, later
refreshes only page the NEW closes since `newest_ts` (a page or two). Close
events are immutable, so nothing already cached is re-fetched. The deep
first pull is a one-time cost. NB the endpoint serves 50-row pages
regardless of `limit` — see smart_money.closed_exits for the paging gotcha."""
import smart_money as _sm # local import: smart_money has no local deps
now = int(time.time())
target_oldest = now - window_days * 86400
with _lock:
r = _con.execute("SELECT newest_ts, pulled_at, oldest_ts FROM pulled_exits "
r = _con.execute("SELECT newest_ts, pulled_at, complete FROM pulled_exits "
"WHERE wallet=?", [wallet]).fetchone()
newest, oldest = (r[0], r[2]) if r else (0, None)
newest, complete = (r[0], bool(r[2])) if r else (0, False)
fresh = r and (now - r[1] < max_age_s)
complete = oldest is not None and oldest <= target_oldest
if not fresh or not complete:
if complete:
new = _sm.closed_exits(wallet, newest_bound=newest) # only new closes
new_oldest = oldest
new, _ = _sm.closed_exits(wallet, newest_bound=newest) # only new closes
reached = True # already complete
else:
new = _sm.closed_exits(wallet, since_ts=target_oldest) # full window (re)pull
new_oldest = target_oldest
new, reached = _sm.closed_exits(wallet) # FULL history (re)pull
with _lock:
if new:
_con.executemany(
@@ -237,8 +233,8 @@ def closed_exits(wallet, max_age_s=6 * 3600, window_days=WINDOW_DAYS):
c.get("title") or "", c.get("outcome") or "")
for a, c in new.items()])
newest = max(newest, max(c["ts"] for c in new.values()))
_con.execute("INSERT OR REPLACE INTO pulled_exits(wallet,newest_ts,pulled_at,oldest_ts) "
"VALUES (?,?,?,?)", [wallet, newest, now, new_oldest])
_con.execute("INSERT OR REPLACE INTO pulled_exits(wallet,newest_ts,pulled_at,complete) "
"VALUES (?,?,?,?)", [wallet, newest, now, reached])
with _lock:
rows = _con.execute(
"SELECT asset, ts, exit_p, p, iv, cond, title, outcome FROM exits WHERE wallet=?",