cache: wait out the single-writer lock at import instead of raising

duckdb is single-writer and cache.py connects at IMPORT time, so any
daily/nightly overlap raised out of 'import cache' — which killed
forward.py and, via nightly.sh's set -e, every grader behind it. Twice
now (07-27, 07-28) the night's grades silently never ran. Bounded wait
(24x15s) covers a collect pass; all consumers inherit the fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jaxperro
2026-07-28 12:46:03 -04:00
parent 79d6fc2b61
commit 7ab613bd26
+22 -1
View File
@@ -64,7 +64,28 @@ def conv_cutoff(sizes, q=CONV_PCTILE):
return s[f] if f + 1 >= len(s) else s[f] + (s[f + 1] - s[f]) * (k - f)
_lock = threading.Lock()
_con = duckdb.connect(DB)
def _connect(db=None, tries=24, wait=15):
"""duckdb is SINGLE-WRITER and this module connects at IMPORT time, so a
daily/nightly overlap used to raise straight out of `import cache` —
killing forward.py and, through nightly.sh's `set -e`, every grader
behind it (2026-07-27 and again 07-28: the graders silently never ran).
Wait the other writer out instead; 6 min covers a collect pass."""
db = db or DB
for i in range(tries):
try:
return duckdb.connect(db)
except Exception as e:
if "lock" not in str(e).lower() or i == tries - 1:
raise
if i == 0:
print(f"[cache] {os.path.basename(db)} locked by another "
f"writer — waiting…", flush=True)
time.sleep(wait)
_con = _connect()
_con.execute("""CREATE TABLE IF NOT EXISTS bets(
wallet TEXT, cond TEXT, asset TEXT, won BOOLEAN, p DOUBLE, res_t BIGINT,
size DOUBLE, src TEXT, ts BIGINT, resolved BOOLEAN)""")