Resume spend-limit and usage-credit blocks, retried hourly
This commit is contained in:
@@ -23,12 +23,18 @@ session, resumes it only when all of these hold:
|
||||
|
||||
1. **The process is gone** — no `pid`, or the pid no longer exists.
|
||||
2. **`~/.claude/jobs/<id>/state.json` says `"state": "blocked"`.**
|
||||
3. **`detail` names a recoverable cause** — session/weekly/Opus limit, usage or
|
||||
rate limit, API error, overload, or a network/connection failure.
|
||||
3. **`detail` names a recoverable cause** — any quota Claude Code reports as
|
||||
`You've hit your <limit>`: session, weekly, Opus, Sonnet, Fable 5, fast,
|
||||
usage, rate, and the ones that cost money rather than time (monthly or
|
||||
individual spend limit, usage credit limit, out of usage credits or extra
|
||||
usage). Plus API errors, overload, and network/connection failures.
|
||||
4. **`detail` does not name a human** — "input needed", "waiting for",
|
||||
permission, approval, or awaiting your direction/decision. A session parked
|
||||
on your decision must stay parked, and this veto wins even when the same
|
||||
detail also names a limit.
|
||||
detail also names a limit. It also covers entitlements only an admin can
|
||||
change, which no amount of waiting clears: a seat type that excludes usage
|
||||
credits, an allocation disabled by your admin, a group limit set to $0, a
|
||||
service disabled for your org.
|
||||
5. **Any stated reset time has passed.** `detail` carries it verbatim, e.g.
|
||||
`You've hit your session limit · resets 7pm (Europe/Stockholm)`, so the
|
||||
watchdog waits for that moment instead of guessing. Day-less times are
|
||||
@@ -39,6 +45,11 @@ session, resumes it only when all of these hold:
|
||||
push the parsed time-of-day a full day into the future and strand the
|
||||
session. Blocks with no stated reset (API outage, network) are retried on
|
||||
the next tick, with no wait.
|
||||
|
||||
A **spend** block states no reset because none exists: it clears when you
|
||||
raise the cap, when the month rolls over, or when the subscription window
|
||||
puts the session back on included quota. So it is neither retried every tick
|
||||
nor parked for 5h — it is retried hourly until one of those happens.
|
||||
6. **Attempt caps are not exhausted** — see the circuit breakers below.
|
||||
|
||||
It resumes with `claude --bg --resume <sessionId> --name "<original name>"`
|
||||
@@ -75,6 +86,12 @@ each retry re-sends the session's whole context. During a multi-hour outage
|
||||
that is the one path that can burn a lot of tokens for nothing. Set
|
||||
`--max-per-run` to a small number instead if you would rather stagger.
|
||||
|
||||
A *spend* block is the other open-ended one: if you never raise the cap it
|
||||
retries hourly until the month turns over, one job per session per hour. Those
|
||||
retries are rejected before the model sees them, so they cost no tokens — but
|
||||
they do spend the daily budget, so raise `--max-per-day` if you routinely have
|
||||
several sessions blocked at once.
|
||||
|
||||
## What it deliberately does not do
|
||||
|
||||
- **Interactive sessions** — you are sitting there; you can press enter.
|
||||
@@ -86,7 +103,7 @@ that is the one path that can burn a lot of tokens for nothing. Set
|
||||
## Install
|
||||
|
||||
Needs Claude Code (for `claude agents --json` and `claude --bg --resume`) and
|
||||
Python 3.9+ — standard library only.
|
||||
Python 3.9+ — standard library only, 3.14+ to run the tests.
|
||||
|
||||
Clone anywhere — `~/.claude/session-watchdog` just keeps it travelling with the
|
||||
rest of `~/.claude`. Then add one cron entry with `crontab -e`:
|
||||
@@ -123,6 +140,9 @@ root.
|
||||
## Verify and observe
|
||||
|
||||
```bash
|
||||
# Does it still classify every block correctly?
|
||||
python3 ~/.claude/session-watchdog/test-resume-stalled-sessions.py
|
||||
|
||||
# What would it do right now? Resumes nothing.
|
||||
~/.claude/session-watchdog/resume-stalled-sessions.py --dry-run
|
||||
|
||||
@@ -159,6 +179,7 @@ hardcoded. To make one permanent, add it to the crontab line.
|
||||
| Path | Role |
|
||||
| ------------------------------------------ | ------------------------------------------ |
|
||||
| `resume-stalled-sessions.py` | The watchdog — the only file it needs |
|
||||
| `test-resume-stalled-sessions.py` | Its tests, on real Claude Code block texts |
|
||||
| `~/.claude/session-watchdog/ledger.json` | Attempt counts and resume timestamps |
|
||||
| `~/.claude/session-watchdog/.lock` | Held for the duration of a run |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user