nightly lock: a pid-less lock ages out (6h) instead of being assumed stale

The first guard treated 'no pid file' as stale, so the launchd run stole a
live old-code lock and TWO nightlies scored concurrently (2026-07-27).
Ledger tolerated it (latest-per-day wins, no dupes) but the races are
avoidable. Conservative now: no pid -> skip unless the lock dir is >6h old.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jaxperro
2026-07-27 12:10:59 -04:00
parent ad03bcc73b
commit adec29b9f3
+16 -4
View File
@@ -27,11 +27,23 @@ if ! mkdir "$LOCK" 2>/dev/null; then
# later night then skips SILENTLY. Trust the recorded PID: if it is gone,
# the lock is debris, so take it over.
OWNER=$(cat "$LOCK/pid" 2>/dev/null)
if [ -n "$OWNER" ] && kill -0 "$OWNER" 2>/dev/null; then
echo "$(date -u +%FT%TZ) already running (pid $OWNER) — skip" >> forward.log
exit 0
if [ -n "$OWNER" ]; then
if kill -0 "$OWNER" 2>/dev/null; then
echo "$(date -u +%FT%TZ) already running (pid $OWNER) — skip" >> forward.log
exit 0
fi
echo "$(date -u +%FT%TZ) STALE lock (pid $OWNER gone) — taking over" >> forward.log
else
# No pid file = a lock from before this guard existed (or a crash between
# mkdir and the write). Do NOT assume stale — that ran two nightlies at
# once on 2026-07-27. Age it out instead: 6h is longer than any real run.
AGE=$(( $(date +%s) - $(stat -f %m "$LOCK" 2>/dev/null || echo 0) ))
if [ "$AGE" -lt 21600 ]; then
echo "$(date -u +%FT%TZ) lock held (no pid, age ${AGE}s < 6h) — skip" >> forward.log
exit 0
fi
echo "$(date -u +%FT%TZ) STALE lock (no pid, age ${AGE}s) — taking over" >> forward.log
fi
echo "$(date -u +%FT%TZ) STALE lock (pid ${OWNER:-none} gone) — taking over" >> forward.log
fi
echo $$ > "$LOCK/pid"
trap 'rmdir "$LOCK"' EXIT