292 lines
19 KiB
Python
292 lines
19 KiB
Python
"""Run a task as headless claude cycles; a judge interrupts only when a decision is the human's.
|
|
|
|
usage: loop-claude <task>
|
|
|
|
Run it anywhere inside a project checkout whose README.md has ## Goals and ## Audience, the
|
|
premises; the task runs at the checkout root. Each cycle a worker runs the task. A chunk's first
|
|
worker prompt also quotes the premises, which choose between ways to deliver the chunk's todo.md
|
|
items, or else the task, and never widen them. As a chunk starts, before it is routed, loop-claude
|
|
runs bin/chunk-preflight (bin/chunk_preflight/), which finds the chunk's facts and makes its
|
|
worktree; chunk-preflight --help lists them, and prompts.preflight_facts in
|
|
bin/loop_claude/cycle/prompts.py quotes them in the first prompt. A preflight that fails, or is
|
|
still running after 600 s and is stopped, leaves the worker to find the facts itself. A judge then
|
|
reads the transcript's end, the premises, and the records: the project's AGENTS.md and
|
|
docs/decisions.md, and the human's ~/.claude/CLAUDE.md, ~/.claude/rules/*.md and
|
|
~/.claude/skills/larv-review/standards/*.md, where they exist. It rules whether the chunk, the work
|
|
of one PR, is finished. A chunk stays unfinished until each answer the human typed in it that a
|
|
later chunk could need is recorded: the premises or the records hold it, or the transcript shows the worker
|
|
writing it into the premises, AGENTS.md, docs/decisions.md, todo.md or a code comment. A choice the
|
|
worker made that the premises or the records settle is not asked about. When the worker asks a
|
|
question they answer, the judge quotes the answer, loop-claude prints it, and the worker resumes
|
|
with it. A quote they do not hold word for word is dropped, and the question goes to the human. So
|
|
does a question asked again after its answer. The fourth question they answer in one cycle goes to
|
|
the human too, with the quote.
|
|
A question only the human can answer, or a release ready to cut, is an interrupt: `claude --resume`
|
|
on the terminal, every worktree first marked trusted in ~/.claude.json. An unfinished chunk resumes
|
|
its session next cycle, and the worker is shown the passage the judge quoted as the work remaining.
|
|
The next chunk starts a fresh session.
|
|
Cycles continue until an exit below or Ctrl-C. Words: ~/.claude/CLAUDE.md → "Vocabulary".
|
|
|
|
Each project has one session, named "A <key> loop-claude", where <key> is the checkout's path
|
|
relative to ~/code, or to / outside it. The session is deleted when its chunk finishes. At every
|
|
interrupt, and as every cycle ends, Ctrl-C and a failure included, the session's transcript is
|
|
copied to $LOOP_CLAUDE_TRANSCRIPTS/<key>/<chunk>/<sha256>.jsonl (default
|
|
~/.claude/loop-claude-transcripts), named by its hash. <chunk> is the hash of the transcript's first
|
|
line with a timestamp, the same for every cycle of a chunk. Beside the copy, <sha256>.outcomes.md
|
|
holds the verdict, with the transcript passage the judge rested it on, any passage it quoted to
|
|
show an escalation trigger fired, and the judge's rung; each answer the judge quoted, with the
|
|
question it answers; each judge reply the loop rejected, with the reason and the reply as json;
|
|
and why the loop stopped. Each of these opens with its UTC time. Claude Code keeps a folder per
|
|
session beside its transcript, holding the subagents' transcripts among other files; that folder,
|
|
and every other file named after the session except the transcript, are copied into
|
|
<chunk>/session/. A file there is replaced only by a version that grew from it. A file Claude Code
|
|
rewrote lands beside its copy as <name>.<hash of its content>.
|
|
|
|
Every claude loop-claude starts runs the project's model, saved per <key> in
|
|
~/.claude/loop-claude-models.json and read each cycle, unless routing, below, puts the worker,
|
|
the interrupt and the judge on another rung; a project without an entry, as on its first
|
|
run, is asked for one. LOOP_CLAUDE_JUDGE_MODEL overrides it for the judge, LOOP_CLAUDE_WORKER_MODEL
|
|
for the worker and the interrupt; with both set, nothing is asked.
|
|
|
|
Where that model is on the routing ladder (bin/model_routing/ladder.py), the worker and the
|
|
interrupt run on one of the ladder's rungs: a model at an effort. As a chunk starts, todo.md on the
|
|
default branch, as chunk-preflight has just fetched it, routes it: its earliest release's top item
|
|
that does not wait, the item the preflight states where that release is the next. An item waits
|
|
while it is marked question, or while an item that blocks it is still in the table. The item's L
|
|
picks the worker's rung, stepping from the model's cheapest rung by WORKER_STEPS in
|
|
bin/model_routing/ladder.py, below it for a low L; without such an item, or with its L, E or I
|
|
unscored, the chunk runs on the model's cheapest rung. The
|
|
item's E + I picks the reviewers' rung, stepping up from the cheapest rung by REVIEW_STEPS there,
|
|
never below the worker's rung. The worker's first prompt names the rung and the item, and asks the
|
|
worker to name each item the chunk takes, with its L and E + I, and to count the item's size again
|
|
once it has read the code (prompts.recount). The prompt names the rung of each review that runs on
|
|
a rung other than the worker's: the reviewers' rung for the reviewers, and the cheapest rung for
|
|
each comprehension seat and the security pass. It also names the reviewers' rung
|
|
each higher E + I picks, so the worker reviews an item it takes in on that rung in the same run.
|
|
When the worker runs below the cheapest rung, the prompt has it launch each subagent that files or
|
|
scores todo.md items on the rung subagent-rung --plan prints. The prompt after
|
|
each unfinished verdict says this again, since a step-up may have moved the worker. A reviewer or
|
|
seat that rules the work below standard may ask to rerun one rung up; each prompt names that rung.
|
|
loop-claude hands the worker's claude its own rung, the project's and the reviewers', in
|
|
LOOP_CLAUDE_WORKER_RUNG, LOOP_CLAUDE_PROJECT_RUNG and LOOP_CLAUDE_REVIEWER_RUNG, and rule-hook
|
|
refuses a review launched below the rung named for it. The judge reports the highest L and the
|
|
highest E + I the worker states for an item it takes in. The next cycle steps the worker up to the
|
|
rung that L picks, never down, and the reviewers to the rung picked by the higher of two: that
|
|
E + I, or the routed item's (Route.reviewed_on in bin/model_routing/route.py). A level off its
|
|
scale counts as none. Every second unfinished verdict on a chunk steps it up one rung
|
|
(routing.stepped). So does an unfinished verdict on a worker that ended its run asking for a
|
|
higher rung, once the judge quotes the transcript showing that the escalation trigger it named
|
|
fired (~/.claude/CLAUDE.md → "Working with the codebase" lists them). That step stops two rungs
|
|
above the rung the chunk's item picks, and the worker's prompts offer it only while the worker is
|
|
below that limit. Each finished or unfinished verdict, stop and step-up in the archive names the
|
|
worker's rung, with the item, L and E + I that routed the chunk, and any taken-in E + I that picks a
|
|
higher review rung, which is archived as a step-up too. A rerun resumes an open chunk on the latest
|
|
route named there, and counts its unfinished verdicts afresh. A rung no longer on the ladder, or
|
|
below the one its item's L picks, is dropped, and the chunk is routed afresh from todo.md. So is a
|
|
chunk whose archived line names an R + C, as loop-claude wrote before it routed on L. A todo.md or
|
|
an archive that cannot be read or parsed is said on stderr, and the chunk is routed without it.
|
|
|
|
The judge of a model listed in JUDGE_REPLAYED (bin/loop_claude/routing.py) runs on the rung that
|
|
the reviewers' E + I (Route.reviewed_on) picks for a review from that model. It never runs below the
|
|
model's cheapest rung, and runs on it when no item routed the chunk or was taken in. Its rung stays
|
|
put when the worker steps up. A model is listed once judge-replay found its cheapest rung returning
|
|
every verdict the reference judge reproduced. Any other model's judge runs at the effort its
|
|
settings give.
|
|
|
|
Counting the cycles a verdict closed across all projects in the archive, the first loop to close
|
|
one 20 or more cycles after the last measurement measures every finished chunk there, with no model
|
|
run: tokens by project, subagent type, rung and the tool whose results they took in, and, for each
|
|
chunk a todo.md item routed, grouped by the rung it started on: its review rounds past each pass's
|
|
first, its unfinished verdicts and its subagents' rungs. It writes them to routing-figures.md at the archive's root. They call for a
|
|
routing change when chunks routed onto a rung took more review rounds or unfinished verdicts than
|
|
chunks routed onto the next rung up, each rung with 5 chunks or more, or when the reports of 5 or
|
|
more subagents of one type were longer than the tool results they read. Each call not raised before
|
|
is an interrupt: claude, on the project's model, on the terminal in ~/.claude, asked to put the
|
|
calls to the human. Once that claude ends, the calls are listed in routing-calls-raised.md beside the
|
|
figures, and not raised again. Off a terminal, the calls are said on stderr, the loop goes on, and
|
|
they are raised again at the next measurement. A line `- <UTC time> Reset` in
|
|
routing-calls-raised.md restarts both: the figures count only chunks that finished after it, and a
|
|
call raised before it may be raised again. A chunk that cannot be read is skipped and named in the
|
|
figures; figures that cannot be measured or written are said on stderr, and the loop goes on.
|
|
|
|
A change to bin/loop-claude or any bin/**/*.py takes effect at the next cycle boundary, once
|
|
loop-claude --help succeeds: the loop reloads into the edited script.
|
|
|
|
Exit 1, saying why, when:
|
|
- no task is given
|
|
- git cannot start or finds no checkout
|
|
- README.md, or AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/*.md or
|
|
~/.claude/skills/larv-review/standards/*.md where they exist, cannot be read or is not UTF-8
|
|
- README.md holds no premises
|
|
- the models file cannot be read or written
|
|
- claude leaves no transcript
|
|
- a transcript cannot be read or archived
|
|
- a finished chunk's session cannot be deleted
|
|
- claude cannot start for an interrupt
|
|
- python cannot start for a reload
|
|
- a reload cannot read the state the running script handed it, likely because the edit changed that
|
|
handover
|
|
- the worker or the judge fails three times running, retried a minute apart within the cycle. A
|
|
worker fails when claude cannot start, exits non-zero, is killed, or ends without a result event;
|
|
a judge when claude cannot start, exits non-zero or prints no json reply, or its reply misses or
|
|
mistypes a field, interrupts or quotes an answer without a question, or quotes a passage the
|
|
transcript does not hold; it holds an elided passage when each fragment appears in order.
|
|
A usage limit is not a failure, and restarts the count of three: the worker or the judge is
|
|
retried 2 minutes after the limit resets, or every 10 minutes when the reset time is unknown or
|
|
more than 2 minutes past. The judge's reply gives its reset time only as text, so the judge is
|
|
always retried every 10 minutes. stderr says when the next try is due.
|
|
|
|
Exit 2, when the human is needed and cannot answer:
|
|
- stdin or stdout is not a terminal, when a model must be picked or an interrupt answered
|
|
- the input ends while a model is picked
|
|
- an interrupt ends with nothing typed
|
|
Rerunning resumes any open session.
|
|
|
|
"""
|
|
import dataclasses
|
|
import pathlib
|
|
import sys
|
|
import typing
|
|
|
|
from transcript_archive.write import archive
|
|
from transcript_archive.location import archive_root, project_archive
|
|
from loop_claude.loop.routing_check import check_routing
|
|
from loop_claude.project.checkout import checkout_root
|
|
from loop_claude.cycle.run import Cycle, run_cycle
|
|
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
|
|
from git_repo import default_branch_todo
|
|
from loop_claude.effects import Effects
|
|
from failure import Failure
|
|
from loop_claude.cycle.judge import Finished, Unfinished
|
|
from loop_claude.routing import first_route, kept, routed_item, stepped
|
|
from loop_claude.project.model_choice import get_or_ask_models
|
|
from todo_table import Item
|
|
from transcript_archive.read import latest_worker
|
|
from transcript_archive import outcomes
|
|
from loop_claude.paths import Paths
|
|
from project_key import project_key
|
|
from model_routing.ladder import project_rung, Rung
|
|
from loop_claude.cycle import prompts
|
|
from loop_claude.project.premises import read_premises
|
|
from loop_claude.project.records import read_records
|
|
from model_routing.route import Route, route_from
|
|
from claude_code.session import Session, session_named, session_open, transcript_file
|
|
from loop_claude.stop import Stop
|
|
|
|
# Seconds before routing gives up asking origin for its default branch, which it asks only where the checkout records none, and routes on the project's rung.
|
|
ORIGIN_TIMEOUT = 60
|
|
|
|
|
|
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
|
|
try:
|
|
stop = run_loop(argv, env, paths, launcher, cwd, effects)
|
|
except KeyboardInterrupt:
|
|
stop = Stop(130, "")
|
|
if stop.reason:
|
|
print(stop.reason, file=sys.stderr if stop.code else sys.stdout)
|
|
return stop.code
|
|
|
|
|
|
def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> Stop:
|
|
loop_run = loop_run_from(argv, cwd, effects)
|
|
if isinstance(loop_run, Stop):
|
|
return loop_run
|
|
baseline = fingerprint(launcher)
|
|
key = project_key(loop_run.root, paths.home)
|
|
archive_folder = project_archive(loop_run.root, paths.home, env)
|
|
# "A " is the human's cue in the session picker that the session is the loop's.
|
|
session = session_named(f"A {key} loop-claude", paths.claude_code_transcripts)
|
|
while True:
|
|
if stop := exec_if_edited(launcher, baseline, loop_run, effects):
|
|
return stop
|
|
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
|
|
premises = read_premises(loop_run.root)
|
|
if isinstance(premises, Stop):
|
|
return premises
|
|
records = read_records(loop_run.root, paths.claude_dir)
|
|
if isinstance(records, Stop):
|
|
return records
|
|
models = get_or_ask_models(key, paths, env, effects)
|
|
if isinstance(models, Stop):
|
|
return models
|
|
# Before routing, which reads the plan as the preflight's fetch left it; after the reads above, so a loop that stops on one makes no worktree.
|
|
facts = "" if session_open(session) else chunk_facts(launcher.with_name("chunk-preflight"), loop_run.root, effects)
|
|
anchor = project_rung(models.worker)
|
|
root = loop_run.root
|
|
loop_run = routed(loop_run, anchor, lambda: archived_route(archive_folder, session), lambda: plan_item(default_branch_todo(root, ORIGIN_TIMEOUT)))
|
|
route = loop_run.route or Route(Rung(models.worker))
|
|
print(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", flush=True)
|
|
verdict = run_cycle(Cycle(anchor=anchor, effects=effects, facts=facts, judge_model=models.judge, paths=paths, premises=premises, project_archive=archive_folder, records=records, root=loop_run.root, route=route, session=session, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
|
|
if isinstance(verdict, Stop):
|
|
return verdict
|
|
before, loop_run = loop_run.route, after_verdict(loop_run, anchor, verdict)
|
|
if stop := archive_step(archive_folder, session, loop_run, before, effects):
|
|
return stop
|
|
if stop := check_routing(archive_root(paths.home, env), anchor or Rung(models.worker), paths, effects):
|
|
return stop
|
|
|
|
|
|
def routed(loop_run: LoopRun, anchor: Rung | None, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Item | None]) -> LoopRun:
|
|
"""The loop run with its chunk's route, or None when the project's model is off the ladder. A chunk keeps its route, or on a rerun the one its archive last names,
|
|
while that is at or above the rung its item's L picks from anchor, the project's rung; else the plan item routes it afresh."""
|
|
if anchor is None:
|
|
return dataclasses.replace(loop_run, route=None)
|
|
route = loop_run.route or archived()
|
|
return dataclasses.replace(loop_run, route=route if route and kept(route, anchor) else first_route(anchor, item()))
|
|
|
|
|
|
def after_verdict(loop_run: LoopRun, anchor: Rung | None, verdict: Finished | Unfinished) -> LoopRun:
|
|
if isinstance(verdict, Finished):
|
|
return dataclasses.replace(loop_run, route=None, unfinished_evidence="")
|
|
route = stepped(anchor, loop_run.route, verdict.bundled, bool(verdict.escalation)).taking_in(verdict.bundled_escape_and_impact) if anchor and loop_run.route else loop_run.route
|
|
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence)
|
|
|
|
|
|
def archive_step(archive_folder: pathlib.Path, session: Session, loop_run: LoopRun, before: Route | None, effects: Effects) -> Stop | None:
|
|
"""Archives the rung an unfinished verdict stepped the chunk up to, or the higher E + I it took in; the verdict itself names the route its cycle ran on."""
|
|
if not (before and loop_run.route) or str(loop_run.route) == str(before):
|
|
return None
|
|
path = transcript_file(session)
|
|
failure = archive(archive_folder, path, loop_run.task, outcomes.stepped_up(str(loop_run.route)), effects.now()) if path else None
|
|
return Stop(1, failure.reason) if failure else None
|
|
|
|
|
|
def archived_route(archive_folder: pathlib.Path, session: Session) -> Route | None:
|
|
"""The route the latest verdict archived for the session's open chunk names; None for a chunk not yet judged."""
|
|
transcript = transcript_file(session)
|
|
worker = latest_worker(archive_folder, transcript) if transcript else None
|
|
if isinstance(worker, Failure):
|
|
print(f"routing the chunk afresh: {worker.reason}", file=sys.stderr)
|
|
return None
|
|
return route_from(worker) if worker else None
|
|
|
|
|
|
def chunk_facts(preflight: pathlib.Path, root: str, effects: Effects) -> str:
|
|
"""What chunk-preflight printed at the checkout root, for a fresh worker's prompt, or why it could not run."""
|
|
done = effects.python([str(preflight)], root)
|
|
if done.returncode:
|
|
return prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}")
|
|
return prompts.preflight_facts(done.stdout.strip())
|
|
|
|
|
|
def plan_item(todo: str | Failure) -> Item | None:
|
|
"""The item that routes a chunk, from todo.md as git_repo.default_branch_todo reads it, as chunk-preflight does, so both take todo_table.next_item's pick. None without one, so the chunk runs on the project's rung."""
|
|
item = todo if isinstance(todo, Failure) else routed_item(todo)
|
|
if isinstance(item, Failure):
|
|
print(f"routing on the project's rung: {item.reason}", file=sys.stderr)
|
|
return None
|
|
return item
|
|
|
|
|
|
def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop:
|
|
if argv[:1] != [RELOAD_FLAG]:
|
|
task = " ".join(argv)
|
|
if task in ("", "-h", "--help"):
|
|
return Stop(0 if task else 1, __doc__.strip())
|
|
root = checkout_root(cwd, effects)
|
|
return root if isinstance(root, Stop) else LoopRun(0, root, task, "")
|
|
loop_run = decode_handed(argv[1], LoopRun) if len(argv) == 2 else None
|
|
if not (loop_run and loop_run.root and loop_run.task):
|
|
return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
|
|
return loop_run
|