Claude Code session watchdog

Resumes Claude Code background sessions that died on a usage limit, an API outage, or a network failure.

Why it lives outside Claude Code

Claude Code has no auto-resume: on a usage limit it blocks further requests until the reset time and the process exits. Nothing scheduled inside a session (a 15-minute self-check, a /loop, a scheduled task) can rescue it, because there is no process left to fire the schedule. Recovery has to come from outside, so this is a cron job.

It also costs nothing while everything is healthy — it reads state files on disk rather than waking sessions up. An in-session poll would re-send the session's full context on every tick, which draws down the very limit it is trying to survive.

How it decides

Every 15 minutes it runs claude agents --json and, for each background session, resumes it only when all of these hold:

The process is gone — no pid, or the pid no longer exists.
~/.claude/jobs/<id>/state.json says "state": "blocked".
detail names a recoverable cause — session/weekly/Opus limit, usage or rate limit, API error, overload, or a network/connection failure.
detail does not name a human — "input needed", "waiting for", permission, approval, or awaiting your direction/decision. A session parked on your decision must stay parked, and this veto wins even when the same detail also names a limit.
Any stated reset time has passed. detail carries it verbatim, e.g. You've hit your session limit · resets 7pm (Europe/Stockholm), so the watchdog waits for that moment instead of guessing. Day-less times are anchored to the state.json mtime; weekday forms like resets Mon 12:00am are handled too. The result is clamped to the limit window — 5h for a session or Opus limit, 7 days for a weekly one — because state.json is sometimes rewritten after the reset already passed, which would otherwise push the parsed time-of-day a full day into the future and strand the session. Blocks with no stated reset (API outage, network) are retried on the next tick, with no wait.
Attempt caps are not exhausted — see the circuit breakers below.

It resumes with claude --bg --resume <sessionId> --name "<original name>" from the session's own recorded working directory, so the conversation, its context and its name all carry over. Oldest block is resumed first.

The resume registers a new job id, leaving the original as a dead husk that is permanently blocked yet still listed as active. The watchdog therefore runs claude stop <old id> after a successful resume. Without that the husk would be picked up again on a later tick and resumed into a second session working the same conversation — two agents racing on the same git worktree.

Circuit breakers

Three limits stop a broken session, or a still-ongoing outage, from spawning an endless chain of blocked jobs:

Limit	Default	Meaning
`--max-per-run`	0	Resumes per tick; 0 means every eligible session
`--max-per-day`	50	Resumes across all sessions in a rolling 24h
`--max-attempts`	1	Resumes of any one session, ever

--max-attempts 1 is not a one-shot recovery: if the resumed session stalls again it is a new session id and qualifies on its own, so an outage spanning several reset windows still gets retried. What the cap prevents is one conversation being resumed into several concurrent copies. A resume that fails to launch at all records no attempt and is retried on the next tick.

--max-per-day is the one guard that cannot be derived from anything else. A usage-limit block is self-pacing, because the stated reset time is respected — but a network or API block states no reset, so it is retried every tick, and each retry re-sends the session's whole context. During a multi-hour outage that is the one path that can burn a lot of tokens for nothing. Set --max-per-run to a small number instead if you would rather stagger.

What it deliberately does not do

Interactive sessions — you are sitting there; you can press enter.
Sessions waiting on a human (condition 4 above).
Sessions you stopped ("state": "stopped") or that finished ("done").
Anything whose working directory has disappeared (deleted worktree) — logged and skipped.

Install

Needs Claude Code (for claude agents --json and claude --bg --resume) and Python 3.9+ — standard library only.

Clone anywhere — ~/.claude/session-watchdog just keeps it travelling with the rest of ~/.claude. Then add one cron entry with crontab -e:

git clone https://gitea.larvit.se/larvit/claude-session-watchdog.git ~/.claude/session-watchdog

7,22,37,52 * * * * $HOME/.claude/session-watchdog/resume-stalled-sessions.py >> $HOME/.claude/session-watchdog/watchdog.log 2>&1

:07 :22 :37 :52 keeps it off the :00 crowd. Nothing wraps it — no PATH prefix, no flock, no chmod, because the script handles both of the things a scheduler would otherwise have to:

Finding claude — PATH first, then ~/.local/bin, /usr/local/bin, /opt/homebrew/bin. If it finds nothing it says so and exits non-zero, rather than logging nothing to resume and looking healthy while resuming nothing, forever. --claude-bin still wins if you want a specific install.
Not overlapping itself — it takes an exclusive lock on ~/.claude/session-watchdog/.lock and exits if another run holds it. Two concurrent runs would read ledger.json before either writes it and resume the same session twice, into two agents racing on one git worktree.

So any scheduler that fires every 15 minutes will do: cron, launchd with StartInterval 900, a systemd timer, Task Scheduler.

Use absolute paths rather than $HOME if your cron does not export HOME (busybox crond does). On Alpine, user crontabs live in /etc/crontabs/<user> and crontab -e writes there via /var/spool/cron/crontabs, so nothing needs root.

Verify and observe

# What would it do right now? Resumes nothing.
~/.claude/session-watchdog/resume-stalled-sessions.py --dry-run

# The same, with cron's bare environment instead of your shell's.
env -i HOME=$HOME PATH=/usr/bin:/bin ~/.claude/session-watchdog/resume-stalled-sessions.py --dry-run

crontab -l                                       # is it scheduled?
tail -f ~/.claude/session-watchdog/watchdog.log  # what it has done

One nothing to resume line per tick, so the log grows by roughly 8 KB a day. Truncate it whenever you like — nothing reads it back.

A stalled session that has been resumed shows up as a new job in claude agents; claude logs <id> shows it picking the work back up.

Configuration

Every option is a CLI flag with an environment-variable fallback — nothing is hardcoded. To make one permanent, add it to the crontab line.

Flag	Env var	Default
`--claude-bin`	`CLAUDE_BIN`	`claude`
`--claude-home`	`CLAUDE_HOME`	`~/.claude`
`--max-attempts`	`WATCHDOG_MAX_ATTEMPTS`	`1`
`--max-per-day`	`WATCHDOG_MAX_PER_DAY`	`50`
`--max-per-run`	`WATCHDOG_MAX_PER_RUN`	`0` (all)
`--prompt`	`WATCHDOG_PROMPT`	see `--help`
`--dry-run`	—	off

Files

Path	Role
`resume-stalled-sessions.py`	The watchdog — the only file it needs
`~/.claude/session-watchdog/ledger.json`	Attempt counts and resume timestamps
`~/.claude/session-watchdog/.lock`	Held for the duration of a run

Both live under --claude-home, wherever the clone itself is. ledger.json is generated: delete it to forget all history, or edit the attempts map to give a specific session another chance.

Uninstall

Drop the line from crontab -e and delete the clone.

8.4 KiB Raw Permalink Blame History