# Claude Code session watchdog Resumes Claude Code **background** sessions that died on a usage limit, an API outage, or a network failure. ## Why it lives outside Claude Code Claude Code has no auto-resume: on a usage limit it blocks further requests until the reset time and the process exits. Nothing scheduled *inside* a session (a 15-minute self-check, a `/loop`, a scheduled task) can rescue it, because there is no process left to fire the schedule. Recovery has to come from outside, so this is a cron job. It also costs nothing while everything is healthy — it reads state files on disk rather than waking sessions up. An in-session poll would re-send the session's full context on every tick, which draws down the very limit it is trying to survive. ## How it decides Every 15 minutes it runs `claude agents --json` and, for each **background** session, resumes it only when all of these hold: 1. **The process is gone** — no `pid`, or the pid no longer exists. 2. **`~/.claude/jobs//state.json` says `"state": "blocked"`.** 3. **`detail` names a recoverable cause** — session/weekly/Opus limit, usage or rate limit, API error, overload, or a network/connection failure. 4. **`detail` does not name a human** — "input needed", "waiting for", permission, approval, or awaiting your direction/decision. A session parked on your decision must stay parked, and this veto wins even when the same detail also names a limit. 5. **Any stated reset time has passed.** `detail` carries it verbatim, e.g. `You've hit your session limit · resets 7pm (Europe/Stockholm)`, so the watchdog waits for that moment instead of guessing. Day-less times are anchored to the `state.json` mtime; weekday forms like `resets Mon 12:00am` are handled too. The result is clamped to the limit window — 5h for a session or Opus limit, 7 days for a weekly one — because `state.json` is sometimes rewritten *after* the reset already passed, which would otherwise push the parsed time-of-day a full day into the future and strand the session. Blocks with no stated reset (API outage, network) are retried on the next tick, with no wait. 6. **Attempt caps are not exhausted** — see the circuit breakers below. It resumes with `claude --bg --resume --name ""` from the session's own recorded working directory, so the conversation, its context and its name all carry over. Oldest block is resumed first. The resume registers a **new** job id, leaving the original as a dead husk that is permanently `blocked` yet still listed as active. The watchdog therefore runs `claude stop ` after a successful resume. Without that the husk would be picked up again on a later tick and resumed into a *second* session working the same conversation — two agents racing on the same git worktree. ### Circuit breakers Three limits stop a broken session, or a still-ongoing outage, from spawning an endless chain of blocked jobs: | Limit | Default | Meaning | | ----------------- | ------- | ------------------------------------------------ | | `--max-per-run` | 0 | Resumes per tick; 0 means every eligible session | | `--max-per-day` | 50 | Resumes across all sessions in a rolling 24h | | `--max-attempts` | 1 | Resumes of any one session, ever | `--max-attempts 1` is not a one-shot recovery: if the resumed session stalls again it is a *new* session id and qualifies on its own, so an outage spanning several reset windows still gets retried. What the cap prevents is one conversation being resumed into several concurrent copies. A resume that fails to launch at all records no attempt and is retried on the next tick. `--max-per-day` is the one guard that cannot be derived from anything else. A usage-limit block is self-pacing, because the stated reset time is respected — but a *network or API* block states no reset, so it is retried every tick, and each retry re-sends the session's whole context. During a multi-hour outage that is the one path that can burn a lot of tokens for nothing. Set `--max-per-run` to a small number instead if you would rather stagger. ## What it deliberately does not do - **Interactive sessions** — you are sitting there; you can press enter. - **Sessions waiting on a human** (condition 4 above). - **Sessions you stopped** (`"state": "stopped"`) or that finished (`"done"`). - **Anything whose working directory has disappeared** (deleted worktree) — logged and skipped. ## Install Needs Claude Code (for `claude agents --json` and `claude --bg --resume`) and Python 3.9+ — standard library only. Clone anywhere — `~/.claude/session-watchdog` just keeps it travelling with the rest of `~/.claude`. Then add one cron entry with `crontab -e`: ```bash git clone https://gitea.larvit.se/larvit/claude-session-watchdog.git ~/.claude/session-watchdog ``` ``` 7,22,37,52 * * * * $HOME/.claude/session-watchdog/resume-stalled-sessions.py >> $HOME/.claude/session-watchdog/watchdog.log 2>&1 ``` `:07 :22 :37 :52` keeps it off the `:00` crowd. Nothing wraps it — no `PATH` prefix, no `flock`, no `chmod`, because the script handles both of the things a scheduler would otherwise have to: - **Finding `claude`** — `PATH` first, then `~/.local/bin`, `/usr/local/bin`, `/opt/homebrew/bin`. If it finds nothing it says so and exits non-zero, rather than logging `nothing to resume` and looking healthy while resuming nothing, forever. `--claude-bin` still wins if you want a specific install. - **Not overlapping itself** — it takes an exclusive lock on `~/.claude/session-watchdog/.lock` and exits if another run holds it. Two concurrent runs would read `ledger.json` before either writes it and resume the same session twice, into two agents racing on one git worktree. So any scheduler that fires every 15 minutes will do: cron, launchd with `StartInterval 900`, a systemd timer, Task Scheduler. Use absolute paths rather than `$HOME` if your cron does not export `HOME` (busybox crond does). On Alpine, user crontabs live in `/etc/crontabs/` and `crontab -e` writes there via `/var/spool/cron/crontabs`, so nothing needs root. ## Verify and observe ```bash # What would it do right now? Resumes nothing. ~/.claude/session-watchdog/resume-stalled-sessions.py --dry-run # The same, with cron's bare environment instead of your shell's. env -i HOME=$HOME PATH=/usr/bin:/bin ~/.claude/session-watchdog/resume-stalled-sessions.py --dry-run crontab -l # is it scheduled? tail -f ~/.claude/session-watchdog/watchdog.log # what it has done ``` One `nothing to resume` line per tick, so the log grows by roughly 8 KB a day. Truncate it whenever you like — nothing reads it back. A stalled session that has been resumed shows up as a new job in `claude agents`; `claude logs ` shows it picking the work back up. ## Configuration Every option is a CLI flag with an environment-variable fallback — nothing is hardcoded. To make one permanent, add it to the crontab line. | Flag | Env var | Default | | ----------------- | ------------------------- | -------------------- | | `--claude-bin` | `CLAUDE_BIN` | `claude` | | `--claude-home` | `CLAUDE_HOME` | `~/.claude` | | `--max-attempts` | `WATCHDOG_MAX_ATTEMPTS` | `1` | | `--max-per-day` | `WATCHDOG_MAX_PER_DAY` | `50` | | `--max-per-run` | `WATCHDOG_MAX_PER_RUN` | `0` (all) | | `--prompt` | `WATCHDOG_PROMPT` | see `--help` | | `--dry-run` | — | off | ## Files | Path | Role | | ------------------------------------------ | ------------------------------------------ | | `resume-stalled-sessions.py` | The watchdog — the only file it needs | | `~/.claude/session-watchdog/ledger.json` | Attempt counts and resume timestamps | | `~/.claude/session-watchdog/.lock` | Held for the duration of a run | Both live under `--claude-home`, wherever the clone itself is. `ledger.json` is generated: delete it to forget all history, or edit the `attempts` map to give a specific session another chance. ## Uninstall Drop the line from `crontab -e` and delete the clone.