Merge pull request 'Route the judge and the reviewers on the highest E + I in their chunk, an item the worker takes in included' (#301) from item-153 into main

This commit was merged in pull request #301.
This commit is contained in:
2026-10-17 00:15:07 +02:00
16 changed files with 293 additions and 101 deletions
+2 -2
View File
@@ -39,13 +39,13 @@ Decisions live in `docs/decisions.md`:
- review: stability and product-owner report only a defect, a broken requirement of the plan or the goals, a breach of a technical principle, or for stability a hard place the change touches; each architect finding names a later change the PR makes harder; prose runs only on a chunk that changes prose
- review: the stability pass mutation-tests the lines a chunk changes, in place of measuring test coverage
- routing: one ladder of (model, effort) rungs, ordered by the Artificial Analysis Intelligence Index with Fable 5.1 at max on top; a task's L picks its worker's rung and its E + I its reviewers', by fixed steps counted from the cheapest rung of the project's model
- routing: a chunk's worker runs on the rung its item's L picks, below the project's rung too; its reviewers run on the highest of the worker's rung, the project's and the one its item's E + I picks, the seats and the security pass always on the project's
- routing: a chunk's worker runs on the rung its item's L picks, below the project's rung too; its reviewers run on the highest of the worker's rung, the project's and the one picked by the highest E + I among the chunk's items, the seats and the security pass always on the project's
- routing: the worker changes rung only as a chunk starts, after every second unfinished verdict, or when the judge reports a taken-in item that picks a higher rung or an escalation trigger the worker cites
- routing: loop-claude measures review rounds and unfinished verdicts per rung every 20 cycles, and calls for a routing change when, of two rungs with 5 or more chunks each, the lower took more of either than the one above, or when 5 or more subagents of one type had reports longer than the tool results they read
- routing: a worker delegates a task only where it needs just the task's conclusion, and each subagent runs on the rung its task's L picks from the project's rung, never below the project's rung when it files or scores todo.md items
- routing: a worker hands a task one rung up on each escalation trigger, up to two rungs above the rung its item picks unless the human agrees, and opens the subagent's prompt with the trigger so the routing figures count it
- routing: a worker whose run ends asking for a higher rung, citing an escalation trigger the judge finds in the transcript, resumes one rung up next cycle, up to two rungs above the rung its item picks
- routing: on a model judge-replay covered, the judge runs on the rung its chunk's item's E + I picks for a review, never below the project's rung
- routing: on a model judge-replay covered, the judge runs on the rung a review would get from the highest E + I among its chunk's items, never below the project's rung
- routing: the stability reviewer runs no lower than the project's rung while the worker runs below it, until `stability-replay` shows a lower rung finding every 🔴 and 🟡 the project's rung finds
## Working rules
+1 -1
View File
@@ -7,7 +7,7 @@
- Reshape every project's `todo.md` to the new shape below: until it is reshaped, that is the project's only work, as it is while README.md lacks Goals and Audience, and `rank_todo.py` says so. First move every machine to this release: an older release rejects the new formula, so it fails on every reshaped `todo.md`. Then mark the README's goals `**Maintain:**` or `**Achieve:**`, score each item's L, E and I, and set a bar on the new scale below the score of every item to keep, since the old bar would drop most items. Until then, an old-shape `todo.md` still ranks by its own formula and weights.
- `todo.md` ranks items by value per cost: this setup's own plan uses `Score = 100 * V / (2**S + 8)`, where 8 is a chunk's fixed cost, about one size-3 change, and `rank_todo.py --help` shows it. A Moves column replaces A, G and Goals, naming each achieve goal an item moves and how far, such as `1.1 by 8, 1.2 by 5`. V sums each named goal's weight times its move. `rank_todo.py` weighs each achieve goal by its position in README.md, the first heaviest, so the goal weights table goes. An item names the most specific goal it moves, never a goal beside its own sub-goal. `rules/todo-items.md` and `rank_todo.py --help` show the new shape.
- L, E and I replace C, and set R. L is the chance the change holds a defect, from 1 to 4; E the chance a defect escapes tests and review, from 1 to 3; I the damage it would do, from 1 to 5, counting every project and session that runs the change. `rules/todo-items.md` defines each level in words, and `rank_todo.py` writes R as L + E + I − 2, at least 7 when I is 5 and at least 5 when I is 4. The table's score columns are `| L | E | I | R | S |`. `score-outcomes` takes `L E I S`, and reports how well L, E and I each order finished chunks by review rounds and 🔴 findings.
- `loop-claude` routes a chunk's worker on its item's L, from two rungs below the project's rung at L 1 to one above it at L 4. It routes the reviewers on the item's E + I, one rung above the project's for each point past 4, never below the worker's rung. It runs the judge of an Opus 5.5 project on the rung E + I picks, never below the project's, and keeps it there when the worker steps up. The comprehension seats and the security pass stay on the project's rung. rule-hook holds each reviewer launch to the reviewers' rung, which `loop-claude` hands the worker in `LOOP_CLAUDE_REVIEWER_RUNG`. A delegated task's subagent takes the rung its L picks: `subagent-rung` takes L in place of R + C. Until a project reshapes its `todo.md`, every chunk there runs on the project's rung. Expect reviews of items whose defects would escape far or break much to cost more, and escalation from an item of L 4 to reach Fable 5.1 only with your yes.
- `loop-claude` routes a chunk's worker on its item's L, from two rungs below the project's rung at L 1 to one above it at L 4. It routes the reviewers on the highest E + I among the chunk's items, counting each item the worker takes in: one rung above the project's for each point past 4, never below the worker's rung. It runs the judge of an Opus 5.5 project on the rung that highest E + I picks, never below the project's, and keeps it there when the worker steps up. The comprehension seats and the security pass stay on the project's rung. rule-hook holds each reviewer launch to the reviewers' rung, which `loop-claude` hands the worker in `LOOP_CLAUDE_REVIEWER_RUNG`. A delegated task's subagent takes the rung its L picks: `subagent-rung` takes L in place of R + C. Until a project reshapes its `todo.md`, every chunk there runs on the project's rung. Expect reviews of items whose defects would escape far or break much to cost more, and escalation from an item of L 4 to reach Fable 5.1 only with your yes.
- An item that restores a maintain goal the project breaks today is exempt as `defect`, and its Moves may name that maintain goal, weighed as the heaviest achieve goal. A defect ranks by its score like any item. Only two kinds rank ahead of the rest of their release: questions first, since an answer may change other items, then security items.
- The stability reviewer applies `principles/technical-principles.md` as written, in place of its own copies of it. Its test-data check now asks for UUID v7, where it asked for randomised UUIDs.
- larv-review's principles audit is a pass of its own, and runs on every chunk that changes code, config or tests, not only on one with an architectural decision. A chunk of prose or planning alone skips it.
+1 -1
View File
@@ -122,7 +122,7 @@ class Main(unittest.TestCase):
@staticmethod
def verdict(interrupt, evidence):
output = {"answer": "", "bundled": 0, "escalation": "", "evidence": evidence, "finished": not interrupt, "interrupt": interrupt, "question": "Which?" if interrupt else ""}
output = {"answer": "", "bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "evidence": evidence, "finished": not interrupt, "interrupt": interrupt, "question": "Which?" if interrupt else ""}
return {"type": "result", "structured_output": output}
def test_replays_the_models_projects_on_the_reference_then_from_the_projects_rung_down_and_prints_the_lowest_that_returns_all(self):
+8 -2
View File
@@ -10,6 +10,7 @@ from failure import Failure
from loop_claude.project.records import HUMAN_RULE_NAMES
from todo_table import LEVELS
from markdown_text import listed
from model_routing.route import on_its_scale
from loop_claude.cycle.retry import retried
from model_routing.ladder import Rung
from loop_claude.stop import Stop
@@ -56,6 +57,8 @@ class Reply:
answer: str = dataclasses.field(metadata={"description": "With interrupt false, the passage of README.md or the records that answers the agent's question, quoted verbatim, without elisions. Empty otherwise."})
# Routes the next cycle's worker. Counting the item the chunk was routed on is harmless: loop-claude never steps the worker down.
bundled: int = dataclasses.field(metadata={"description": "The highest L the agent states for a todo.md item it took into this chunk; 0 when it states none."})
# Routes the next cycle's judge and reviewers, as bundled routes the worker.
bundled_escape_and_impact: int = dataclasses.field(metadata={"description": "The highest E + I the agent states for a todo.md item it took into this chunk; 0 when it states none."})
# Routes the next cycle's worker one rung up: docs/decisions.md → `routing: a worker whose run ends asking for a higher rung`.
escalation: str = dataclasses.field(metadata={"description": "Empty unless the chunk is unfinished and the agent's run ends asking loop-claude for a higher rung, citing an escalation trigger. Then: a verbatim quote, without elisions, of an earlier transcript passage showing the trigger fired after any earlier such request, such as a review round's commit title or the agent's report of the finding or failing test. Never the request itself."})
evidence: str = dataclasses.field(metadata={"description": "A verbatim quote of the one transcript passage the verdict rests on, without elisions: what triggers the interrupt; for a quoted answer, the agent's question; else what shows the chunk finished or the work remaining."})
@@ -106,6 +109,8 @@ class Unfinished:
bundled: int = 0
# The judge's quote showing that the worker's cited escalation trigger fired; empty for none.
escalation: str = ""
# The bundled E + I the judge saw, 0 for none.
bundled_escape_and_impact: int = 0
Verdict = Interrupt | QuotedAnswer | Finished | Unfinished
@@ -128,9 +133,10 @@ def verdict_from(output: object) -> Verdict | Failure:
return QuotedAnswer(output["evidence"], output["question"], output["answer"])
if output["finished"]:
return Finished(output["evidence"])
# A bundled L off its scale counts as none: holding the rung is safe, and three failures would stop an unattended loop.
# A bundled level off its scale counts as none: holding the rung is safe, and three failures would stop an unattended loop.
bundled = output["bundled"] if 0 <= output["bundled"] <= HIGHEST_L else 0
return Unfinished(output["evidence"], bundled, output["escalation"])
bundled_escape_and_impact = output["bundled_escape_and_impact"] if on_its_scale(output["bundled_escape_and_impact"]) else 0
return Unfinished(output["evidence"], bundled, output["escalation"], bundled_escape_and_impact)
def judge(root: str, task: str, premises: str, records: str, rendered_transcript: str, rung: Rung, effects: Effects, keep: typing.Callable[[Rejected], Stop | None]) -> Verdict | Stop:
+28 -7
View File
@@ -1,5 +1,5 @@
from model_routing.ladder import climbed, LADDER, position, review_rungs, ReviewKind, Rung
from model_routing.route import ESCALATION_MARKER, Route
from model_routing.route import ESCALATION_MARKER, HIGHEST_ESCAPE_AND_IMPACT, Route
from loop_claude.routing import escalation_rungs
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
from markdown_text import listed, quoted
@@ -24,13 +24,13 @@ def routed_by(route: Route, anchor: Rung | None) -> str:
def review_launches(route: Route, anchor: Rung | None) -> str:
"""Which reviews to launch on a rung other than the worker's, and on which; below the project's rung, anchor, also where to launch a subagent that files or scores todo.md items. Empty off the ladder, and where every review runs on the worker's rung.
"""Which reviews to launch on a rung other than the worker's, and on which; below the project's rung, anchor, also where to launch a subagent that files or scores todo.md items; then taken_in_launches, the reviewers' rung each higher E + I picks. Empty off the ladder.
For a worker on Sonnet 5.5 at high, on an Opus 5.5 project, routed by item 3 of E + I 5, it opens: ` Launch each reviewer with the Agent tool set to model opus, effort high, the rung item 3's E + I of 5 picks. Launch each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.`
"""
if anchor is None:
return ""
rungs = review_rungs(anchor, route.rung, route.escape_and_impact)
rungs = review_rungs(anchor, route.rung, route.reviewed_on())
launched: dict[Rung, list[str]] = {}
for reviews, kind in REVIEWS:
if rungs[kind] != route.rung:
@@ -38,12 +38,33 @@ def review_launches(route: Route, anchor: Rung | None) -> str:
launches = "".join(f" Launch {listed(reviews, 'and')} with the Agent tool set to {rung.agent_args()}, {rung_named(rung, route, anchor)}." for rung, reviews in launched.items())
scan = " Run a full scan only from such a subagent, since the scan's agents run on the model that starts them." if rungs["security"] != route.rung else ""
planning = f" Your rung is below the project's: launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan {anchor.model} <L>` prints." if position(route.rung) < position(anchor) else ""
return f"{launches}{scan}{planning}"
return f"{launches}{scan}{planning}{taken_in_launches(route, anchor)}"
def taken_in_launches(route: Route, anchor: Rung) -> str:
"""The rung each higher E + I picks for the reviewers, so the worker reviews an item it takes in on that rung in the same run; the judge reports it as the cycle ends, so loop-claude routes on it only from the next cycle. Empty where no higher E + I picks a higher rung.
For a worker on Opus 5.5 at medium, routed by an item of E + I 6, it reads: ` Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort max` from 7 and `model fable, effort max` from 8.`
"""
reviewers = review_rungs(anchor, route.rung, route.reviewed_on())["reviewer"]
higher: dict[Rung, int] = {}
for escape_and_impact in range(route.reviewed_on() + 1, HIGHEST_ESCAPE_AND_IMPACT + 1):
rung = review_rungs(anchor, route.rung, escape_and_impact)["reviewer"]
if position(rung) > position(reviewers):
higher.setdefault(rung, escape_and_impact)
if not higher:
return ""
picks = [f"`{rung.agent_args()}` from {escape_and_impact}" for rung, escape_and_impact in higher.items()]
return f" Where an item you take into the chunk has an E + I of {min(higher.values())} or more, launch each reviewer with the Agent tool set to {listed(picks, 'and')}."
def rung_named(rung: Rung, route: Route, anchor: Rung) -> str:
"""Why a review runs on rung: it is the project's, anchor, or the one the route's item's E + I picks."""
return "the project's rung" if rung == anchor else f"the rung item {route.item}'s E + I of {route.escape_and_impact} picks"
"""Why a review runs on rung: it is the project's, anchor, or the one the route's reviewed_on E + I picks, the item's or one taken in."""
if rung == anchor:
return "the project's rung"
if route.escape_and_impact_taken_in:
return f"the rung picked by an item of E + I {route.escape_and_impact_taken_in} taken into the chunk"
return f"the rung item {route.item}'s E + I of {route.escape_and_impact} picks"
def escalation(route: Route, anchor: Rung | None) -> str:
@@ -64,7 +85,7 @@ def reruns(route: Route, anchor: Rung | None) -> str:
"""The prompt's sentence naming the rung one above a reviewer's and a seat's, on which each reruns when it asks; empty off the ladder, and leaving out one already on the top rung."""
if anchor is None or route.rung not in LADDER:
return ""
rungs = review_rungs(anchor, route.rung, route.escape_and_impact)
rungs = review_rungs(anchor, route.rung, route.reviewed_on())
launched = (("reviewer", rungs["reviewer"]), ("comprehension seat", rungs["seat"]))
rerun = {name: climbed(rung, 1) for name, rung in launched if rung != LADDER[-1]}
if not rerun:
+32 -29
View File
@@ -51,37 +51,40 @@ stepping from the model's cheapest rung by WORKER_STEPS in bin/model_routing/lad
for a low L; without such an item, or with its L, E or I unscored, the chunk runs on the model's
cheapest rung. The
item's E + I picks the reviewers' rung, stepping up from the cheapest rung by REVIEW_STEPS there,
never below the worker's rung. The worker's first prompt names the rung and the item, and asks the worker to
name each item the chunk takes, with its L and E + I. The prompt names the rung of each review that
runs on a rung other than the worker's: the reviewers' rung for the reviewers, and the cheapest rung
for each comprehension seat and the security pass. When the worker runs below the cheapest rung,
the prompt has it launch each subagent that files or scores todo.md items on the rung
subagent-rung --plan prints. The prompt after
never below the worker's rung. The worker's first prompt names the rung and the item, and asks the
worker to name each item the chunk takes, with its L and E + I. The prompt names the rung of each
review that runs on a rung other than the worker's: the reviewers' rung for the reviewers, and the
cheapest rung for each comprehension seat and the security pass. It also names the reviewers' rung
each higher E + I picks, so the worker reviews an item it takes in on that rung in the same run.
When the worker runs below the cheapest rung, the prompt has it launch each subagent that files or
scores todo.md items on the rung subagent-rung --plan prints. The prompt after
each unfinished verdict says this again, since a step-up may have moved the worker. A reviewer or
seat that rules the work below standard may ask to rerun one rung up; each prompt names that rung.
loop-claude hands the worker's claude its own rung, the project's and the reviewers', in
LOOP_CLAUDE_WORKER_RUNG, LOOP_CLAUDE_PROJECT_RUNG and LOOP_CLAUDE_REVIEWER_RUNG, and rule-hook
refuses a review launched below the rung named for it. The judge reports the highest L the worker
states, and the next cycle steps up to the rung that L picks, never down; an L off its scale counts
as none. Every second unfinished
verdict on a chunk steps it up one rung. So does an unfinished verdict on a worker that ended its
run asking for a higher rung, once the judge quotes the transcript showing that the escalation
trigger it named fired (~/.claude/CLAUDE.md → "Working with the codebase" lists them). That step
stops two rungs above the rung the chunk's item picks, and the worker's prompts offer it only while
the worker is below that limit. Each finished or unfinished verdict, stop and step-up in the
archive names the worker's rung, with the item, L and E + I that routed the chunk; a rerun resumes
an open chunk on the latest rung named there, and counts its unfinished verdicts afresh. A rung no
longer on the ladder, or below the one its item's L picks, is dropped, and the chunk is routed
afresh from todo.md. So is a chunk whose archived line names an R + C, as loop-claude wrote before
it routed on L. A todo.md or an archive that cannot be read or parsed is said on stderr, and the chunk is routed
without it.
refuses a review launched below the rung named for it. The judge reports the highest L and the
highest E + I the worker states for an item it takes in. The next cycle steps the worker up to the
rung that L picks, never down, and the reviewers to the rung picked by the higher of two: that
E + I, or the routed item's (Route.reviewed_on in bin/model_routing/route.py). A level off its
scale counts as none. Every second unfinished verdict on a chunk steps it up one rung
(routing.stepped). So does an unfinished verdict on a worker that ended its run asking for a
higher rung, once the judge quotes the transcript showing that the escalation trigger it named
fired (~/.claude/CLAUDE.md → "Working with the codebase" lists them). That step stops two rungs
above the rung the chunk's item picks, and the worker's prompts offer it only while the worker is
below that limit. Each finished or unfinished verdict, stop and step-up in the archive names the
worker's rung, with the item, L and E + I that routed the chunk, and any taken-in E + I that picks a
higher review rung, which is archived as a step-up too. A rerun resumes an open chunk on the latest
route named there, and counts its unfinished verdicts afresh. A rung no longer on the ladder, or
below the one its item's L picks, is dropped, and the chunk is routed afresh from todo.md. So is a
chunk whose archived line names an R + C, as loop-claude wrote before it routed on L. A todo.md or
an archive that cannot be read or parsed is said on stderr, and the chunk is routed without it.
The judge of a model listed in JUDGE_REPLAYED (bin/loop_claude/routing.py) runs on the rung the
routed item's E + I picks for a review from that model, never below the model's cheapest rung, and
on the cheapest rung when no item routed the chunk. Its rung stays put when the worker steps up or
takes in another item. A model is listed once judge-replay found its cheapest rung returning every
verdict the reference judge reproduced. Any other model's judge runs at the effort its settings
give.
The judge of a model listed in JUDGE_REPLAYED (bin/loop_claude/routing.py) runs on the rung that
the reviewers' E + I (Route.reviewed_on) picks for a review from that model. It never runs below the
model's cheapest rung, and runs on it when no item routed the chunk or was taken in. Its rung stays
put when the worker steps up. A model is listed once judge-replay found its cheapest rung returning
every verdict the reference judge reproduced. Any other model's judge runs at the effort its
settings give.
Counting the cycles a verdict closed across all projects in the archive, the first loop to close
one 20 or more cycles after the last measurement measures every finished chunk there, with no model
@@ -221,13 +224,13 @@ def routed(loop_run: LoopRun, anchor: Rung | None, archived: typing.Callable[[],
def after_verdict(loop_run: LoopRun, anchor: Rung | None, verdict: Finished | Unfinished) -> LoopRun:
if isinstance(verdict, Finished):
return dataclasses.replace(loop_run, route=None, unfinished_evidence="")
route = stepped(anchor, loop_run.route, verdict.bundled, bool(verdict.escalation)) if anchor and loop_run.route else loop_run.route
route = stepped(anchor, loop_run.route, verdict.bundled, bool(verdict.escalation)).taking_in(verdict.bundled_escape_and_impact) if anchor and loop_run.route else loop_run.route
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence)
def archive_step(archive_folder: pathlib.Path, session: Session, loop_run: LoopRun, before: Route | None, effects: Effects) -> Stop | None:
"""Archives the rung an unfinished verdict stepped the chunk up to; the verdict itself names the rung its cycle ran on."""
if not (before and loop_run.route) or loop_run.route.rung == before.rung:
"""Archives the rung an unfinished verdict stepped the chunk up to, or the higher E + I it took in; the verdict itself names the route its cycle ran on."""
if not (before and loop_run.route) or str(loop_run.route) == str(before):
return None
path = transcript_file(session)
failure = archive(archive_folder, path, loop_run.task, outcomes.stepped_up(str(loop_run.route)), effects.now()) if path else None
+2 -4
View File
@@ -82,13 +82,11 @@ def taken_in(anchor: Rung, route: Route, bundled: int) -> Rung:
def judge_rung(judge_model: str, route: Route) -> Rung:
"""The rung the route's item's E + I picks for a review, counted from the judge model's cheapest rung and never below it, whatever rung the worker stepped to.
An item the worker takes in beyond the routed one lifts the worker's rung, through taken_in, but not the judge's, until todo.md item 153 lifts the judge too.
"""The rung the route's reviewed_on E + I picks for a review, counted from the judge model's cheapest rung and never below it, whatever rung the worker stepped to.
A model off the ladder, or not in JUDGE_REPLAYED, gets no effort flag, so claude runs it at the effort its settings give.
"""
anchor = project_rung(judge_model)
if anchor is None or judge_model not in JUDGE_REPLAYED:
return Rung(judge_model)
return review_floor(anchor, route.escape_and_impact)
return review_floor(anchor, route.reviewed_on())
+20 -2
View File
@@ -125,8 +125,8 @@ def worker(**step):
return {"role": "worker", **step}
def judge(interrupt=False, finished=False, evidence="working", question="", bundled=0, escalation="", **step):
verdict = {"answer": "", "bundled": bundled, "escalation": escalation, "evidence": evidence, "finished": finished, "interrupt": interrupt, "question": question}
def judge(interrupt=False, finished=False, evidence="working", question="", bundled=0, bundled_escape_and_impact=0, escalation="", **step):
verdict = {"answer": "", "bundled": bundled, "bundled_escape_and_impact": bundled_escape_and_impact, "escalation": escalation, "evidence": evidence, "finished": finished, "interrupt": interrupt, "question": question}
return {"role": "judge", "verdict": verdict, **step}
@@ -432,6 +432,24 @@ class Cycles(unittest.TestCase):
self.assertIn("Judge: claude-opus-5-5 at medium effort\nWorker: claude-sonnet-5-5 at high effort", "".join(checkout.archived().values()))
self.assertIn("Launch each reviewer, each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.", argument(first, "-p"))
def test_routes_the_judge_and_the_reviewers_on_the_highest_e_plus_i_the_worker_takes_in(self):
checkout = Checkout(self)
checkout.env["LOOP_CLAUDE_WORKER_MODEL"] = checkout.env["LOOP_CLAUDE_JUDGE_MODEL"] = "claude-opus-5-5"
(checkout.root / "todo.md").write_text(CHEAP)
run = checkout.run([
worker(say="one"), judge(evidence="one", bundled_escape_and_impact=6),
worker(say="two"), judge(finished=True, evidence="two"),
worker(say="next chunk"),
])
self.assertEqual(run.code, 130)
self.assertEqual([argument(call, "--effort") for call in run.calls if call["role"] == "judge"], ["medium", "xhigh", "medium"])
workers = [call for call in run.calls if call["role"] == "worker"]
self.assertEqual({argument(call, "--effort") for call in workers}, {"high"})
self.assertIn("Launch each reviewer with the Agent tool set to model opus, effort xhigh, the rung picked by an item of E + I 6 taken into the chunk.", argument(workers[1], "-p"))
outcomes = "".join(checkout.archived().values())
self.assertIn("<UTC> Stepped up\n\nWorker: claude-sonnet-5-5 at high effort, routed by item 4's L of 2 and E + I of 3, taking in an E + I of 6\n", outcomes)
self.assertIn("Judge: claude-opus-5-5 at xhigh effort\nWorker: claude-sonnet-5-5 at high effort, routed by item 4's L of 2 and E + I of 3, taking in an E + I of 6\n", outcomes)
def test_a_rerun_resumes_an_open_chunk_on_the_rung_it_last_ran(self):
checkout = Checkout(self)
checkout.env["LOOP_CLAUDE_WORKER_MODEL"] = "claude-opus-5-5"
+97 -17
View File
@@ -104,7 +104,7 @@ def ran(stdout="", code=0, stderr=""):
def judged(verdict, evidence="working", answer=""):
reply = {"answer": answer, "bundled": 0, "escalation": "", "evidence": evidence, "finished": verdict == "finished", "interrupt": verdict == "interrupt", "question": "Which one?" if verdict in ("interrupt", "quoted") else ""}
reply = {"answer": answer, "bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "evidence": evidence, "finished": verdict == "finished", "interrupt": verdict == "interrupt", "question": "Which one?" if verdict in ("interrupt", "quoted") else ""}
return ran(json.dumps({"structured_output": reply}))
@@ -624,6 +624,18 @@ class Routing(unittest.TestCase):
self.assertEqual(routing.stepped(self.OPUS, Route(self.XHIGH), 0, True), Route(self.XHIGH, unfinished=1))
self.assertEqual(routing.stepped(self.OPUS, Route(self.SONNET, 3, 2, 4), 4, True), Route(self.HIGH, 3, 2, 4, unfinished=1))
def test_keeps_an_e_plus_i_taken_in_only_where_it_picks_a_higher_review_rung(self):
taken = Route(self.OPUS, 3, 3, 4).taking_in(6)
self.assertEqual(taken, Route(self.OPUS, 3, 3, 4, escape_and_impact_taken_in=6))
self.assertEqual(taken.taking_in(7), Route(self.OPUS, 3, 3, 4, escape_and_impact_taken_in=7))
self.assertEqual(taken.taking_in(5), taken)
self.assertEqual(taken.taking_in(6), taken)
self.assertEqual(Route(self.OPUS, 3, 3, 4).taking_in(4), Route(self.OPUS, 3, 3, 4))
self.assertEqual(Route(self.OPUS, 3, 2, 2).taking_in(4), Route(self.OPUS, 3, 2, 2))
self.assertEqual(Route(self.OPUS).taking_in(5), Route(self.OPUS, escape_and_impact_taken_in=5))
self.assertEqual(Route(self.OPUS).taking_in(3), Route(self.OPUS))
self.assertEqual(Route(self.OPUS).taking_in(0), Route(self.OPUS))
def test_a_rung_an_edit_took_off_the_ladder_counts_as_the_projects(self):
self.assertEqual(routing.stepped(self.OPUS, Route(Rung("claude-opus-5-5", "low")), 0, False).rung, self.OPUS)
self.assertEqual(routing.stepped(self.OPUS, Route(Rung("claude-opus-5-5", "low")), 2, False).rung, self.OPUS)
@@ -637,12 +649,22 @@ class Routing(unittest.TestCase):
self.assertEqual(routing.judge_rung("judge-model", Route(self.HIGH, 3, 4, 5)), Rung("judge-model"))
self.assertEqual(routing.judge_rung("claude-sonnet-5-5", Route(self.HIGH, 3, 4, 5)), Rung("claude-sonnet-5-5"))
def test_runs_the_judge_on_the_rung_the_highest_e_plus_i_taken_in_picks(self):
self.assertEqual(routing.judge_rung("claude-opus-5-5", Route(self.OPUS, 3, 3, 4, escape_and_impact_taken_in=6)), self.XHIGH)
self.assertEqual(routing.judge_rung("claude-opus-5-5", Route(self.OPUS, escape_and_impact_taken_in=5)), self.HIGH)
def test_refuses_a_route_whose_item_has_levels_off_their_scales_or_levels_without_an_item(self):
for item, likelihood, escape_and_impact in ((3, 0, 5), (3, 5, 5), (3, 4, 1), (3, 4, 9), (0, 4, 5), (0, 4, 0), (0, 0, 5)):
with self.subTest(item=item, likelihood=likelihood, escape_and_impact=escape_and_impact):
with self.assertRaises(ValueError):
Route(self.HIGH, item, likelihood, escape_and_impact)
def test_refuses_a_route_taking_in_an_e_plus_i_off_its_scale_or_not_above_its_items(self):
for taken_in in (1, 9, -1, 5, 4):
with self.subTest(taken_in=taken_in):
with self.assertRaises(ValueError):
Route(self.HIGH, 3, 4, 5, escape_and_impact_taken_in=taken_in)
def test_takes_a_route_whose_item_holds_each_levels_lowest_value(self):
self.assertEqual(Route(self.HIGH, 3, 1, 2).escape_and_impact, 2)
@@ -657,6 +679,14 @@ class Routing(unittest.TestCase):
self.assertEqual(route_from(str(Route(self.HIGH))), Route(self.HIGH))
self.assertEqual(str(Route(Rung("worker-model"))), "worker-model")
self.assertIsNone(route_from("worker-model"))
taken = Route(self.HIGH, 3, 4, 5, escape_and_impact_taken_in=7)
self.assertEqual(str(taken), "claude-opus-5-5 at high effort, routed by item 3's L of 4 and E + I of 5, taking in an E + I of 7")
self.assertEqual(route_from(str(taken)), taken)
unrouted = Route(self.HIGH, escape_and_impact_taken_in=6)
self.assertEqual(str(unrouted), "claude-opus-5-5 at high effort, taking in an E + I of 6")
self.assertEqual(route_from(str(unrouted)), unrouted)
self.assertIsNone(route_from("claude-opus-5-5 at high effort, taking in an E + I of 9"))
self.assertIsNone(route_from("claude-opus-5-5 at high effort, routed by item 3's L of 4 and E + I of 5, taking in an E + I of 5"))
def test_reads_no_route_from_a_line_whose_levels_are_off_their_scales_or_an_r_plus_c_route_wrote(self):
for line in ("claude-opus-5-5 at high effort, routed by item 3's L of 5 and E + I of 5", "claude-opus-5-5 at high effort, routed by item 3's L of 0 and E + I of 5", "claude-opus-5-5 at high effort, routed by item 3's L of 4 and E + I of 9", "claude-opus-5-5 at high effort, routed by item 3's L of 4 and E + I of 1", "claude-opus-5-5 at high effort, routed by item 3's R + C of 11"):
@@ -740,11 +770,28 @@ class Steps(unittest.TestCase):
self.assertEqual(main.archived_route(archive, opened), Route(Rung("claude-opus-5-5", "high"), 3, 3, 4))
self.assertIsNone(main.archive_step(archive, opened, after, after.route, FakeEffects().effects()))
self.assertIsNone(main.archive_step(archive, opened, after, None, FakeEffects().effects()))
finished = main.after_verdict(after, self.OPUS, Finished("done"))
self.assertIsNone(main.archive_step(archive, opened, finished, after.route, FakeEffects().effects()))
self.assertEqual(main.archived_route(archive, opened), Route(Rung("claude-opus-5-5", "high"), 3, 3, 4))
blocked = pathlib.Path(home, "blocked")
blocked.write_text("")
stop = main.archive_step(blocked, opened, after, before, FakeEffects().effects())
self.assertEqual((stop.code, stop.reason.split(":")[0]), (1, "could not archive the transcript"))
def test_archives_a_higher_e_plus_i_taken_in_so_a_rerun_keeps_it(self):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects"))
archive = pathlib.Path(home, "archive")
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
before = Route(self.OPUS, 3, 3, 4)
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(str(before), "working", "j", ""), NOW))
after = main.after_verdict(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence="", route=before), self.OPUS, Unfinished("working", 0, bundled_escape_and_impact=6))
self.assertIsNone(main.archive_step(archive, opened, after, before, FakeEffects().effects()))
self.assertEqual(main.archived_route(archive, opened), Route(self.OPUS, 3, 3, 4, escape_and_impact_taken_in=6))
class WorkerArgs(unittest.TestCase):
SCAN = " Run a full scan only from such a subagent, since the scan's agents run on the model that starts them."
@@ -760,30 +807,44 @@ class WorkerArgs(unittest.TestCase):
def test_keeps_every_review_on_the_projects_rung_while_the_worker_runs_below_it(self):
opus = Rung("claude-opus-5-5", "medium")
below = prompts.fresh("Do it.", "## Goals", Route(Rung("claude-sonnet-5-5", "high"), 3, 2, 4), opus)
self.assertIn(f"whether or not it is item 3. Launch each reviewer, each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.{self.SCAN} Your rung is below the project's: launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan claude-opus-5-5 <L>` prints. When an escalation trigger", below)
self.assertIn(f"whether or not it is item 3. Launch each reviewer, each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.{self.SCAN} Your rung is below the project's: launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan claude-opus-5-5 <L>` prints. Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model opus, effort high` from 5, `model opus, effort xhigh` from 6, `model opus, effort max` from 7 and `model fable, effort max` from 8. When an escalation trigger", below)
self.assertNotIn("Launch each", prompts.fresh("Do it.", "## Goals", Route(opus, 3, 3, 4), opus))
def test_launches_reviewers_on_the_rung_the_items_e_plus_i_picks_above_the_workers(self):
opus = Rung("claude-opus-5-5", "medium")
level = prompts.fresh("Do it.", "## Goals", Route(opus, 3, 3, 6), opus)
self.assertIn("whether or not it is item 3. Launch each reviewer with the Agent tool set to model opus, effort xhigh, the rung item 3's E + I of 6 picks. When an escalation trigger", level)
self.assertIn("whether or not it is item 3. Launch each reviewer with the Agent tool set to model opus, effort xhigh, the rung item 3's E + I of 6 picks. Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort max` from 7 and `model fable, effort max` from 8. When an escalation trigger", level)
below = prompts.fresh("Do it.", "## Goals", Route(Rung("claude-sonnet-5-5", "high"), 3, 2, 5), opus)
self.assertIn(f"whether or not it is item 3. Launch each reviewer with the Agent tool set to model opus, effort high, the rung item 3's E + I of 5 picks.{self.SEATS}{self.SCAN} Your rung is below the project's:", below)
self.assertNotIn("Launch each reviewer", prompts.resumed("", Route(Rung("claude-opus-5-5", "xhigh"), 3, 4, 6), opus))
def test_names_the_reviewers_rung_each_higher_e_plus_i_an_item_taken_in_picks(self):
opus = Rung("claude-opus-5-5", "medium")
taking_in = " Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model opus, effort high` from 5, `model opus, effort xhigh` from 6, `model opus, effort max` from 7 and `model fable, effort max` from 8."
self.assertIn(f"whether or not it is item 3.{taking_in} When an escalation trigger", prompts.fresh("Do it.", "## Goals", Route(opus, 3, 3, 4), opus))
self.assertIn(" Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort max` from 7 and `model fable, effort max` from 8.", prompts.resumed("", Route(opus, 3, 3, 6), opus))
self.assertIn(" Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort max` from 7 and `model fable, effort max` from 8.", prompts.resumed("", Route(Rung("claude-opus-5-5", "xhigh"), 3, 4, 4), opus))
self.assertNotIn("Where an item you take into", prompts.resumed("", Route(opus, 3, 3, 8), opus))
self.assertNotIn("Where an item you take into", prompts.resumed("", Route(Rung("m")), None))
def test_launches_reviewers_on_the_rung_the_highest_e_plus_i_taken_in_picks(self):
opus = Rung("claude-opus-5-5", "medium")
self.assertIn(f"loop-claude runs you on claude-opus-5-5 at medium effort. Launch each reviewer with the Agent tool set to model opus, effort max, the rung picked by an item of E + I 7 taken into the chunk. Where an item you take into the chunk has an E + I of 8 or more, launch each reviewer with the Agent tool set to `model fable, effort max` from 8. When an escalation trigger", prompts.resumed("", Route(opus, 3, 3, 5, escape_and_impact_taken_in=7), opus))
self.assertIn(" Launch each reviewer with the Agent tool set to model opus, effort high, the rung picked by an item of E + I 5 taken into the chunk.", prompts.resumed("", Route(opus, escape_and_impact_taken_in=5), opus))
def test_keeps_seats_and_the_security_pass_on_the_projects_rung_while_the_worker_runs_above_it(self):
opus = Rung("claude-opus-5-5", "medium")
above = prompts.fresh("Do it.", "## Goals", Route(Rung("claude-opus-5-5", "high"), 3, 4, 4), opus)
stepped_up = prompts.resumed("", Route(Rung("claude-opus-5-5", "high"), 3, 3, 4), opus)
self.assertIn(f"{prompts.ANSWERS_RECORDED}. loop-claude runs you on claude-opus-5-5 at high effort.{self.SEATS}{self.SCAN} When an escalation trigger", stepped_up)
self.assertIn(f"{prompts.ANSWERS_RECORDED}. loop-claude runs you on claude-opus-5-5 at high effort.{self.SEATS}{self.SCAN} Where an item you take into the chunk has an E + I of 6 or more, launch each reviewer with the Agent tool set to `model opus, effort xhigh` from 6, `model opus, effort max` from 7 and `model fable, effort max` from 8. When an escalation trigger", stepped_up)
self.assertNotIn("the project's", prompts.resumed("", Route(opus, 3, 3, 4), opus))
self.assertNotIn("runs you on", prompts.resumed("", Route(Rung("m")), None))
self.assertIn(f"whether or not it is item 3.{self.SEATS}{self.SCAN} When an escalation trigger", above)
self.assertIn(f"whether or not it is item 3.{self.SEATS}{self.SCAN} Where an item you take into the chunk has an E + I of 6 or more, launch each reviewer with the Agent tool set to `model opus, effort xhigh` from 6, `model opus, effort max` from 7 and `model fable, effort max` from 8. When an escalation trigger", above)
def test_names_the_rungs_an_escalation_trigger_lets_the_worker_hand_a_task_to_at_most_two_above_its_items(self):
opus = Rung("claude-opus-5-5", "medium")
fresh = prompts.fresh("Do it.", "## Goals", Route(opus, 3, 3, 4), opus)
self.assertIn("whether or not it is item 3. When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model opus, effort high`, one rung above yours, and on each further trigger one rung higher, up to `model opus, effort xhigh`; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number.", fresh)
self.assertIn("whether or not it is item 3. Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model opus, effort high` from 5, `model opus, effort xhigh` from 6, `model opus, effort max` from 7 and `model fable, effort max` from 8. When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model opus, effort high`, one rung above yours, and on each further trigger one rung higher, up to `model opus, effort xhigh`; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number.", fresh)
self.assertIn(" When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model opus, effort xhigh`, one rung above yours; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number.", prompts.resumed("", Route(Rung("claude-opus-5-5", "high"), 3, 3, 4), opus))
self.assertIn(" An escalation trigger in `~/.claude/CLAUDE.md` lets you hand the hard part of a task to a subagent with the Agent tool set to `model opus, effort max`, one rung above yours, only once the human agrees; loop-claude will not resume you higher for a trigger either, so ask the human for that yes. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number.", prompts.resumed("", Route(Rung("claude-opus-5-5", "xhigh"), 3, 3, 4), opus))
self.assertNotIn("escalation", prompts.resumed("", Route(Rung("claude-fable-5-1", "max"), 3, 4, 8), opus))
@@ -899,19 +960,19 @@ class Grounded(unittest.TestCase):
class Verdict(unittest.TestCase):
def test_needs_every_field_the_schema_requires(self):
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Finished("e"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": "ignored"}), Unfinished("e"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "", "interrupt": True, "evidence": "e", "finished": True, "question": "Which one?"}), Interrupt("e", "Which one?"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?"}), QuotedAnswer("e", "Which one?", "A."))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "A.", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?"}), Interrupt("e", "Which one?"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Failure("the judge quoted an answer without a question"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "", "interrupt": "no", "evidence": "", "finished": True, "question": ""}), Failure("the judge's reply has no interrupt of type boolean"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "", "interrupt": False, "evidence": None, "finished": True, "question": ""}), Failure("the judge's reply has no evidence of type string"))
self.assertEqual(verdict_from({"bundled": 0, "escalation": "", "answer": "", "interrupt": True, "evidence": "x", "finished": False, "question": " "}), Failure("the judge interrupted without a question"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Finished("e"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": "ignored"}), Unfinished("e"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "", "interrupt": True, "evidence": "e", "finished": True, "question": "Which one?"}), Interrupt("e", "Which one?"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?"}), QuotedAnswer("e", "Which one?", "A."))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "A.", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?"}), Interrupt("e", "Which one?"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Failure("the judge quoted an answer without a question"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "", "interrupt": "no", "evidence": "", "finished": True, "question": ""}), Failure("the judge's reply has no interrupt of type boolean"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "", "interrupt": False, "evidence": None, "finished": True, "question": ""}), Failure("the judge's reply has no evidence of type string"))
self.assertEqual(verdict_from({"bundled": 0, "bundled_escape_and_impact": 0, "escalation": "", "answer": "", "interrupt": True, "evidence": "x", "finished": False, "question": " "}), Failure("the judge interrupted without a question"))
self.assertEqual(verdict_from(None), Failure("the judge returned no reply"))
def test_takes_a_bundled_l_on_its_scale_and_counts_any_other_as_none(self):
reply = {"answer": "", "escalation": "", "interrupt": False, "evidence": "e", "finished": False, "question": ""}
reply = {"answer": "", "bundled_escape_and_impact": 0, "escalation": "", "interrupt": False, "evidence": "e", "finished": False, "question": ""}
self.assertEqual(verdict_from({**reply, "bundled": 4}), Unfinished("e", 4))
self.assertEqual(verdict_from({**reply, "bundled": True}), Failure("the judge's reply has no bundled of type integer"))
for bundled in (5, 13, -1):
@@ -919,8 +980,17 @@ class Verdict(unittest.TestCase):
self.assertEqual(verdict_from({**reply, "bundled": bundled}), Unfinished("e", 0))
def test_takes_a_bundled_e_plus_i_on_its_scale_and_counts_any_other_as_none(self):
reply = {"answer": "", "bundled": 0, "escalation": "", "interrupt": False, "evidence": "e", "finished": False, "question": ""}
self.assertEqual(verdict_from({**reply, "bundled_escape_and_impact": 6}), Unfinished("e", 0, bundled_escape_and_impact=6))
self.assertEqual(verdict_from({**reply, "bundled_escape_and_impact": 2}), Unfinished("e", 0, bundled_escape_and_impact=2))
self.assertEqual(verdict_from({**reply, "bundled_escape_and_impact": False}), Failure("the judge's reply has no bundled_escape_and_impact of type integer"))
for bundled in (1, 9, -1):
with self.subTest(bundled=bundled):
self.assertEqual(verdict_from({**reply, "bundled_escape_and_impact": bundled}), Unfinished("e", 0))
def test_takes_an_escalation_quote_on_an_unfinished_verdict(self):
reply = {"answer": "", "bundled": 0, "escalation": "round 2: 🔴", "interrupt": False, "evidence": "e", "finished": False, "question": ""}
reply = {"answer": "", "bundled": 0, "bundled_escape_and_impact": 0, "escalation": "round 2: 🔴", "interrupt": False, "evidence": "e", "finished": False, "question": ""}
self.assertEqual(verdict_from(reply), Unfinished("e", 0, "round 2: 🔴"))
self.assertEqual(verdict_from({**reply, "finished": True}), Finished("e"))
self.assertEqual(verdict_from({**reply, "escalation": None}), Failure("the judge's reply has no escalation of type string"))
@@ -953,6 +1023,16 @@ class LoopRuns(unittest.TestCase):
off_scale = dataclasses.asdict(state)
off_scale["route"]["likelihood"] = 9
self.assertIsNone(reload.decode_handed(json.dumps(off_scale), main.LoopRun))
older = dataclasses.asdict(state)
del older["route"]["escape_and_impact_taken_in"]
self.assertEqual(reload.decode_handed(json.dumps(older), main.LoopRun), state)
taken = dataclasses.replace(state, route=dataclasses.replace(state.route, escape_and_impact_taken_in=7))
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(taken)), main.LoopRun), taken)
for taken_in in (9, 5):
with self.subTest(taken_in=taken_in):
handed = dataclasses.asdict(state)
handed["route"]["escape_and_impact_taken_in"] = taken_in
self.assertIsNone(reload.decode_handed(json.dumps(handed), main.LoopRun))
def test_drops_a_route_an_older_loop_claude_handed_under_r_plus_c_so_the_chunk_routes_afresh(self):
older = {"cycle_number": 3, "root": "/r", "task": "do it", "unfinished_evidence": "", "route": {"rung": {"model": "claude-opus-5-5", "effort": "high"}, "item": 3, "risk_and_complexity": 11, "unfinished": 1}}
+43 -22
View File
@@ -4,35 +4,45 @@ import re
import typing
from todo_table import LEVELS
from model_routing.ladder import Rung
from model_routing.ladder import REVIEW_STEPS, Rung
# How a worker opens the prompt of a subagent it escalates a task to, `Escalation trigger 2: …`: docs/decisions.md → `routing: a worker hands a task one rung up`.
ESCALATION_MARKER = "Escalation trigger"
# An archived worker line: the rung, then the item and the levels that routed it, if any. route_from reads only LEVELS_ROUTED's levels, so a line an older loop-claude wrote, naming an R + C, routes its chunk afresh; archived_worker still reads its rung for the routing figures.
ROUTED = re.compile(r"(?P<rung>.+?)(?:, routed by item (?P<item>[0-9]+)'s (?P<levels>.+))?")
LEVELS_ROUTED = re.compile(r"L of (?P<likelihood>[0-9]+) and E \+ I of (?P<escape_and_impact>[0-9]+)")
# An archived worker line: the rung, then the item and the levels that routed it, if any, then the E + I taken in, if any. A line an older loop-claude wrote names an R + C in place of L and E + I: archived_worker reads only its rung, for the routing figures; with no levels, route_from returns None and the chunk routes afresh.
ROUTED = re.compile(r"(?P<rung>.+?)(?:, routed by item (?P<item>[0-9]+)'s (?:L of (?P<likelihood>[0-9]+) and E \+ I of (?P<escape_and_impact>[0-9]+)|.+?))?(?:, taking in an E \+ I of (?P<escape_and_impact_taken_in>[0-9]+))?")
class ArchivedWorker(typing.NamedTuple):
"""What an archived worker line names: the worker's rung, the item that routed it, 0 for none, and the text naming that item's levels, which an older loop-claude wrote as an R + C."""
"""What an archived worker line names: the worker's rung, the item that routed it, its L and E + I, and the E + I taken in; 0 for each it does not name."""
rung: Rung
item: int
levels: str
likelihood: int
escape_and_impact: int
escape_and_impact_taken_in: int
HIGHEST_ESCAPE_AND_IMPACT = LEVELS["E"] + LEVELS["I"]
def on_its_scale(escape_and_impact: int) -> bool:
"""Whether E + I is a level on the scales todo_table.LEVELS gives; E and I each start at 1."""
return 2 <= escape_and_impact <= HIGHEST_ESCAPE_AND_IMPACT
def on_their_scales(likelihood: int, escape_and_impact: int) -> bool:
"""Whether L and E + I are levels on the scales todo_table.LEVELS gives; E and I each start at 1."""
return 1 <= likelihood <= LEVELS["L"] and 2 <= escape_and_impact <= LEVELS["E"] + LEVELS["I"]
"""Whether L and E + I are levels on the scales todo_table.LEVELS gives."""
return 1 <= likelihood <= LEVELS["L"] and on_its_scale(escape_and_impact)
def levels_fit(item: int, likelihood: int, escape_and_impact: int) -> bool:
"""Whether a route may hold these: an item with levels on their scales, or no item and no levels."""
return on_their_scales(likelihood, escape_and_impact) if item else not (likelihood or escape_and_impact)
def levels_fit(item: int, likelihood: int, escape_and_impact: int, escape_and_impact_taken_in: int) -> bool:
"""Whether a route may hold these. It has an item with levels on their scales, or no item and no levels. Its E + I taken in is 0, or on its scale and above the item's; Route.taking_in keeps only one that steps the reviewers further."""
routed = on_their_scales(likelihood, escape_and_impact) if item else not (likelihood or escape_and_impact)
return routed and (escape_and_impact_taken_in == 0 or (on_its_scale(escape_and_impact_taken_in) and escape_and_impact_taken_in > escape_and_impact))
@dataclasses.dataclass(frozen=True)
class Route:
"""A chunk's worker rung and the plan item that picked it, 0 for none. The item's L picked the worker's rung; its E + I picks the reviewers'. A route holds only levels that levels_fit, so no rung is picked from a level off its scale.
"""A chunk's worker rung and the plan item that picked it, 0 for none. The item's L picked the worker's rung; reviewed_on, the higher of its E + I and the highest E + I the worker took in, picks the reviewers' and the judge's. A route holds only levels that levels_fit, so no rung is picked from a level off its scale.
unfinished counts the unfinished verdicts towards the next one-rung step; a reload hands it over with the rung.
"""
@@ -43,35 +53,46 @@ class Route:
likelihood: int = 0
escape_and_impact: int = 0
unfinished: int = 0
# The highest E + I the judge saw the worker take in, kept only where it picks a higher review step than the item's; 0 otherwise.
escape_and_impact_taken_in: int = 0
def __post_init__(self) -> None:
# Raises on a programming error only, so nothing catches it: route_from and the reload decoder check levels_fit before they build a route.
if not levels_fit(self.item, self.likelihood, self.escape_and_impact):
raise ValueError(f"a route by item {self.item} cannot hold an L of {self.likelihood} and an E + I of {self.escape_and_impact}")
if not levels_fit(self.item, self.likelihood, self.escape_and_impact, self.escape_and_impact_taken_in):
raise ValueError(f"a route by item {self.item} cannot hold an L of {self.likelihood}, an E + I of {self.escape_and_impact} and an E + I taken in of {self.escape_and_impact_taken_in}")
@staticmethod
def fits(fields: typing.Mapping[str, typing.Any]) -> bool:
"""Whether a route may be built from these fields, as the reload decoder asks of a handed one."""
return levels_fit(fields.get("item", 0), fields.get("likelihood", 0), fields.get("escape_and_impact", 0))
return levels_fit(fields.get("item", 0), fields.get("likelihood", 0), fields.get("escape_and_impact", 0), fields.get("escape_and_impact_taken_in", 0))
def taking_in(self, escape_and_impact: int) -> "Route":
"""The route after the judge saw the worker take in an item of this E + I, 0 for none: it keeps the E + I only where REVIEW_STEPS steps it further than reviewed_on, so the archive records a step-up only where REVIEW_STEPS steps the review floor further."""
return dataclasses.replace(self, escape_and_impact_taken_in=escape_and_impact) if REVIEW_STEPS.get(escape_and_impact, 0) > REVIEW_STEPS.get(self.reviewed_on(), 0) else self
def reviewed_on(self) -> int:
"""The E + I that picks the reviewers' and the judge's rung: the routed item's or the highest taken in, whichever is higher; 0 for neither."""
return max(self.escape_and_impact, self.escape_and_impact_taken_in)
def __str__(self) -> str:
"""What the archive records as each verdict's worker; ROUTED reads it back."""
return f"{self.rung}, routed by item {self.item}'s L of {self.likelihood} and E + I of {self.escape_and_impact}" if self.item else str(self.rung)
routed = f", routed by item {self.item}'s L of {self.likelihood} and E + I of {self.escape_and_impact}" if self.item else ""
taken_in = f", taking in an E + I of {self.escape_and_impact_taken_in}" if self.escape_and_impact_taken_in else ""
return f"{self.rung}{routed}{taken_in}"
def archived_worker(worker: str) -> ArchivedWorker | None:
"""What an archived worker line names; None for a model run off the ladder."""
found = ROUTED.fullmatch(worker)
rung = Rung.parse(found["rung"]) if found else None
return ArchivedWorker(rung, int(found["item"] or 0), found["levels"] or "") if found and rung else None
if not (found and rung):
return None
return ArchivedWorker(rung, *(int(found[name] or 0) for name in ("item", "likelihood", "escape_and_impact", "escape_and_impact_taken_in")))
def route_from(worker: str) -> Route | None:
"""The route an archived worker line names; None for a model run off the ladder, or a route by anything but levels on their scales."""
archived = archived_worker(worker)
if archived is None or not archived.item:
return Route(archived.rung) if archived else None
levels = LEVELS_ROUTED.fullmatch(archived.levels)
if not levels or not levels_fit(archived.item, int(levels["likelihood"]), int(levels["escape_and_impact"])):
if archived is None or not levels_fit(archived.item, archived.likelihood, archived.escape_and_impact, archived.escape_and_impact_taken_in):
return None
return Route(archived.rung, archived.item, int(levels["likelihood"]), int(levels["escape_and_impact"]))
return Route(archived.rung, archived.item, archived.likelihood, archived.escape_and_impact, escape_and_impact_taken_in=archived.escape_and_impact_taken_in)
+1
View File
@@ -13,6 +13,7 @@ class Environment(unittest.TestCase):
self.assertEqual(environment(self.PROJECT, Route(self.WORKER, 3, 1, 5)), {"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort", "LOOP_CLAUDE_REVIEWER_RUNG": "claude-opus-5-5 at high effort"})
self.assertEqual(environment(self.PROJECT, Route(self.WORKER))["LOOP_CLAUDE_REVIEWER_RUNG"], "claude-opus-5-5 at medium effort")
self.assertEqual(environment(self.PROJECT, Route(Rung("claude-opus-5-5", "xhigh"), 3, 4, 2))["LOOP_CLAUDE_REVIEWER_RUNG"], "claude-opus-5-5 at xhigh effort")
self.assertEqual(environment(self.PROJECT, Route(self.WORKER, 3, 1, 5, escape_and_impact_taken_in=6))["LOOP_CLAUDE_REVIEWER_RUNG"], "claude-opus-5-5 at xhigh effort")
def test_a_project_off_the_ladder_is_handed_empty_rungs_so_none_is_inherited(self):
self.assertEqual(environment(None, Route(self.WORKER)), {"LOOP_CLAUDE_PROJECT_RUNG": "", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort", "LOOP_CLAUDE_REVIEWER_RUNG": ""})
+1 -1
View File
@@ -13,7 +13,7 @@ REVIEWER_RUNG = "LOOP_CLAUDE_REVIEWER_RUNG"
def environment(project: Rung | None, route: Route) -> dict[str, str]:
"""The variables a worker's claude on route starts with. A project off the ladder gets empty project and reviewer rungs, so none is inherited from the environment."""
reviewers = review_rungs(project, route.rung, route.escape_and_impact)["reviewer"] if project else None
reviewers = review_rungs(project, route.rung, route.reviewed_on())["reviewer"] if project else None
return {PROJECT_RUNG: str(project) if project else "", WORKER_RUNG: str(route.rung), REVIEWER_RUNG: str(reviewers) if reviewers else ""}
+27
View File
@@ -2033,3 +2033,30 @@ Mid A, decided by:
Junior A, decided by:
- Locality, Overall: `bin/score_outcomes/main.py:86` `report_text`: the report is one pure function that states its section order and no longer writes R into the caller's dict, so one section changes without tracing the print order through main
## 2026-10-16T22:13:23Z, PR #301 at 2057d8f, against 11b9ec3
Ruling: same
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | same | same | worse | better | same |
| Inherited architect | better | worse | worse | better | same |
Mid A, decided by:
- Shape: `bin/model_routing/route.py:69` `Route.taking_in`: sits beside `routing.taken_in`, a near-identical name on another axis, a Rung picked by L against a Route holding an E + I, and `after_verdict` chains both
- Shape: `bin/loop_claude/cycle/judge.py:59` `Reply`: `bundled_escape_and_impact` beside `bundled`, which means the bundled L, so one name of each pair spells its axis and the other hides it
- Shape: `bin/model_routing/route.py:12` `ROUTED`: two small regexes became one dense one with optional levels, a retired R + C branch and a taken-in suffix, though `route_from` reads simpler
- Self-sufficiency: `bin/loop_claude/routing.py:84` `judge_rung`: the rule now lives in `Route.reviewed_on`, which every consumer calls, in place of a pointer to a plan item
- Self-sufficiency: `workflow-bench/review_git.py:1` module docstring: states in place what it allows and why, where the bench skipped every permission check with no explanation
Inherited architect, decided by:
- Locality, Shape: `bin/loop_claude/loop/main.py:227` `after_verdict`: one taken-in item updates the route in two places, `stepped` through `taken_in` and the chained `taking_in`, with "bundled" a third word for the same idea
- Locality: `bin/model_routing/route.py:73` `Route.reviewed_on`: `escape_and_impact` no longer picks the reviewers, so a caller reading the field in place of `reviewed_on()` gets a quietly low rung
- Shape, Locality: `bin/model_routing/route.py:12` `ROUTED`: one dense regex with nested optional groups, and `levels_fit` accepts more taken-in E + I than `taking_in` keeps
- Shape: `bin/loop_claude/loop/main.py:231` `archive_step`: a "Stepped up" outcome is written when only the reviewers' E + I rose, so the outcome's name no longer tells what happened
- Navigation, Self-sufficiency: `bin/loop_claude/cycle/run.py:122` `keep_rejection`: a loop that exits 1 after a rejected judge reply now leads to the archived reply, named like `keep_answer`
- Self-sufficiency, Navigation: `workflow-bench/review_git.py:1` module docstring: explains the rule syntax and what the wrapper checks inline
- Self-sufficiency: `bin/loop_claude/routing.py:84` `judge_rung`: points to `reviewed_on` in place of a plan item
+7 -7
View File
@@ -236,15 +236,15 @@ A project's rung is its model's cheapest rung, which for Opus 5.5 is its default
Valid while the index ranks these models, and while no newer model joins the catalog.
## routing: a chunk's worker runs on the rung its item's L picks, below the project's rung too; its reviewers run on the highest of the worker's rung, the project's and the one its item's E + I picks, the seats and the security pass always on the project's
## routing: a chunk's worker runs on the rung its item's L picks, below the project's rung too; its reviewers run on the highest of the worker's rung, the project's and the one picked by the highest E + I among the chunk's items, the seats and the security pass always on the project's
2026-10-14, lilleman. Serves goals 1.1 and 1.4.
Only chunks run below the project's rung can prove that a cheaper worker holds quality (goal 1.1). So on 2026-10-14 lilleman chose a trial on real chunks: a replay of past chunks would spend on work that never ships (goal 1.4). Review holds quality meanwhile. While the worker runs below the project's rung, loop-claude's prompt has it launch each comprehension seat and the security pass's subagent on the project's rung, and each reviewer there too unless its item's E + I picks a higher rung, through the Agent tool's model and effort. The worker runs a full scan only from such a subagent, because the `claude-security` plugin's researchers and verifiers inherit the model that starts them. A subagent that cannot run the scan leaves it unavailable, so larv-review runs `/security-review` in its place. lilleman chose the project's rung, medium, knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes.
Only chunks run below the project's rung can prove that a cheaper worker holds quality (goal 1.1). So on 2026-10-14 lilleman chose a trial on real chunks: a replay of past chunks would spend on work that never ships (goal 1.4). Review holds quality meanwhile. While the worker runs below the project's rung, loop-claude's prompt has it launch each comprehension seat and the security pass's subagent on the project's rung, and each reviewer there too unless the highest E + I among its items picks a higher rung, through the Agent tool's model and effort. The worker runs a full scan only from such a subagent, because the `claude-security` plugin's researchers and verifiers inherit the model that starts them. A subagent that cannot run the scan leaves it unavailable, so larv-review runs `/security-review` in its place. lilleman chose the project's rung, medium, knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes.
While the worker runs above the project's rung, loop-claude's prompt has it launch the comprehension seats and the security pass's subagent on the project's rung too. So the seats score every chunk on one rung, and the security pass spends no more than the project chose. The prompt after each unfinished verdict repeats this, since a step-up may have taken the worker above the project's rung. The other reviewers never run below the worker's rung: a worker climbs because its chunk proved harder than its score said, which is when a cheaper reviewer is the bigger risk. A reviewer or seat that rules the work below standard may rerun one rung up: `review: a reviewer or comprehension seat that rules the work below standard`.
On 2026-10-16 lilleman routed the reviewers on their item's E + I as well, on either side of the project's rung, which may lift them above both the worker's rung and the project's: how far a defect would escape, and what it would break, set how hard review must look (goal 1.1). loop-claude names that rung in the worker's prompt, and hands it to the worker's claude beside the project's and the worker's, so rule-hook holds each reviewer launch to it. A low E + I never steps a reviewer below the project's rung: lilleman chose to lower a pass only where a replay shows a lower rung finding every 🔴 and 🟡 the project's rung finds, which `todo.md` item 159 measures for each pass.
On 2026-10-16 lilleman routed the reviewers on their item's E + I as well, on either side of the project's rung, which may lift them above both the worker's rung and the project's: how far a defect would escape, and what it would break, set how hard review must look (goal 1.1). loop-claude names that rung in the worker's prompt, and hands it to the worker's claude beside the project's and the worker's, so rule-hook holds each reviewer launch to it. A low E + I never steps a reviewer below the project's rung: lilleman chose to lower a pass only where a replay shows a lower rung finding every 🔴 and 🟡 the project's rung finds, which `todo.md` item 159 measures for each pass. An item the worker takes in beyond the routed one counts too, since its E + I scores its defects as the routed item's scores theirs. The worker's prompt names the reviewers' rung each higher E + I picks, so the worker reviews a taken-in item on it in the same run. The judge reports the highest E + I the worker states for a taken-in item. From the next cycle loop-claude routes on that E + I, and archives it with the route so a rerun keeps it.
Valid until the trial on real chunks shows whether a worker below the project's rung holds quality.
@@ -262,7 +262,7 @@ Valid while a model switch inside a session discards its prompt cache.
The cost of a cheaper worker shows as extra review rounds and unfinished verdicts. Every 20 cycles across the archive, loop-claude measures them in code, with no model run (goal 1.4), and interrupts lilleman with the calls not raised before, when chunks routed onto a rung took more of either than chunks routed onto the next rung up, each rung with 5 chunks or more. Five keeps one odd chunk from calling for a change (goal 1.2). It also calls when the reports of 5 or more subagents of one type were longer than the tool results they read, so a delegated search cost the worker more context than it saved. `bin/routing_figures/` measures and words the calls. A loop on another project cannot commit here, so it writes the figures and the calls raised at the archive's root, and this repo's next chunk files the plan item, as `AGENTS.md` says. Off a terminal, a call is said on stderr and the loop goes on: it is about this setup, not the running project's chunk. A reset line in the calls file restarts the figures after a routing change, so they show whether the change worked.
Since 2026-10-16 the reviewers may run above the worker's rung, on the one their item's E + I picks, so chunks whose worker ran on a lower rung may take more rounds from stronger reviewers, and a call could blame the worker's rung for them; `todo.md` item 171 groups the rounds by the reviewers' rung too. Valid while loop-claude's archive holds every finished chunk's review rounds and unfinished verdicts.
Since 2026-10-16 the reviewers may run above the worker's rung, on the one the highest E + I among their chunk's items picks, so chunks whose worker ran on a lower rung may take more rounds from stronger reviewers, and a call could blame the worker's rung for them; `todo.md` item 171 groups the rounds by the reviewers' rung too. Valid while loop-claude's archive holds every finished chunk's review rounds and unfinished verdicts.
## routing: a worker delegates a task only where it needs just the task's conclusion, and each subagent runs on the rung its task's L picks from the project's rung, never below the project's rung when it files or scores todo.md items
@@ -294,15 +294,15 @@ The archive quotes the judge's passage under `Escalation:` in the unfinished ver
Valid while the worker reports in its transcript the triggers that fired.
## routing: on a model judge-replay covered, the judge runs on the rung its chunk's item's E + I picks for a review, never below the project's rung
## routing: on a model judge-replay covered, the judge runs on the rung a review would get from the highest E + I among its chunk's items, never below the project's rung
2026-10-15, lilleman's plan, settled by the replay below. Serves goals 1.2 and 1.4, under the ladder and steps of `routing: one ladder`.
On a model listed in `JUDGE_REPLAYED` in `bin/loop_claude/routing.py`, the judge runs on the rung its chunk's item's E + I picks for a review, never below the project's rung, and keeps that rung when the worker steps up. Without an item, it runs on the project's rung. Each archived verdict names the judge's rung on a `Judge:` line.
On a model listed in `JUDGE_REPLAYED` in `bin/loop_claude/routing.py`, the judge runs on the rung a review would get from the highest E + I among its chunk's items: the routed item and those the worker takes in. It never runs below the project's rung, and keeps its rung when the worker steps up. Without an item, it runs on the project's rung. Each archived verdict names the judge's rung on a `Judge:` line.
On 2026-10-15, `bin/judge-replay claude-opus-5-5` replayed the latest 15 archived verdicts of each kind from the Opus 5.5 projects, 39 in all. The reference judge, Opus 5.5 at the xhigh effort `settings.json` gave every judge before, reproduced 31 of them. The other 8 may rest on records or judge rules that have changed since; either way no rung is held to them. On the 31, Haiku 5.5 missed 7 at xhigh and 8 at max. That run tried each judge once. 4 of Haiku's misses, 1 at xhigh and 3 at max, misquoted the transcript, which the loop and `judge-replay` now retry; without them it missed 6 and 5. Sonnet 5.5 at high returned another kind on 3 interrupts, and Opus 5.5 at medium missed none. A missed interrupt lets the worker settle a decision that is lilleman's (goal 1.2), so the judge never runs below the project's rung.
Only Opus 5.5 was replayed, so the judge of a project on another model keeps the effort its settings give until `judge-replay` covers that model (goal 1.1). Every routed chunk on record had an item of R + C 9, so the replay could not show that the project's rung suffices for harder items, and the judge climbs with the routed item. Since 2026-10-16, lilleman's call, it climbs with the item's E + I, as a review does: the judge checks the worker's decisions as the reviewers check its code, and a decision the judge lets through reaches lilleman, as a defect that review lets through reaches the users (goal 1.2). It runs above the project's rung only on an E + I of 5 or more (goal 1.4). An item the worker takes in beyond the routed one lifts the worker's rung but not the judge's, until `todo.md` item 153 lifts the judge too.
Only Opus 5.5 was replayed, so the judge of a project on another model keeps the effort its settings give until `judge-replay` covers that model (goal 1.1). Every routed chunk on record had an item of R + C 9, so the replay could not show that the project's rung suffices for harder items, and the judge climbs with the routed item. Since 2026-10-16, lilleman's call, it climbs with the item's E + I, as a review does: the judge checks the worker's decisions as the reviewers check its code, and a decision the judge lets through reaches lilleman, as a defect that review lets through reaches the users (goal 1.2). It runs above the project's rung only on an E + I of 5 or more (goal 1.4). It counts a taken-in item from the next cycle on.
Valid while the archived verdicts show what a judge should return, and until a new model joins the ladder: replay then.
+17
View File
@@ -569,3 +569,20 @@ Read most (opens and searches):
- `settings.json` 1
- `workflow-bench/bench.py` 1
- `workflow-bench/run-prompt.md` 1
## Reading cost, chunk 6ea020c688fa, finished 2026-10-16T21:28:06Z
107738 new input tokens before the first commit.
Read most (opens and searches):
- `todo.md` 6
- `bin/loop_claude/test_loop_claude.py` 5
- `bin/loop_claude/loop/main.py` 4
- `README.md` 2
- `bin/loop_claude/cycle/judge.py` 2
- `bin/loop_claude/cycle/retry.py` 2
- `bin/loop_claude/cycle/run.py` 2
- `docs/decisions.md` 2
- `AGENTS.md` 1
- `CHANGELOG.md` 1
+6 -6
View File
@@ -6,13 +6,12 @@
`Bar = 0.1`
`Next ID = 174`
`Next ID = 175`
## Items
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 153 | 0.2.0 | | decision | **Route the judge and the reviewers on the highest E + I in their chunk, an item the worker takes in included.** | 3 | 1 | 3 | 5 | 4 | 1.2 by 5 | 3.0 |
| 161 | 0.2.0 | | | **Have loop-claude run a preflight script before each fresh worker and put its findings in the worker's prompt, so the worker no longer spends tool calls finding the chunk's mechanical facts.** | 4 | 2 | 3 | 7 | 6 | 1.1 by 4, 1.4 by 6, 1.5 by 6 | 2.9 |
| 84 | 0.2.0 | | | **Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.** | 2 | 2 | 2 | 4 | 4 | 1.4 by 5 | 2.3 |
| 168 | 0.2.0 | | | **Have the human answer an interrupt on loop-claude's terminal and resume the worker headless with the answer, opening the interactive session only when they ask for it.** | 4 | 2 | 3 | 7 | 5 | 1.5 by 8 | 1.8 |
@@ -43,6 +42,7 @@
| 171 | 0.3.0 | | decision | **Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 3, 1.4 by 3 | 3.2 |
| 165 | 0.3.0 | | | **Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, 1.4 by 3 | 3.0 |
| 169 | 0.3.0 | | | **Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.** | 3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 2.7 |
| 174 | 0.3.0 | | | **In a chunk no item routed, ask the worker to name each todo.md item it takes, with its L and E + I, so loop-claude can route on an item it takes in.** | 1 | 2 | 3 | 4 | 2 | 1.1 by 1 | 1.8 |
| 122 | 0.4.0 | | | **Assemble the judge's prompt in `bin/loop_claude/cycle/judge.py`, importing only wording the worker also sees, and call the worker "the worker" in it.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 |
| 158 | 0.4.0 | | decision | **Measure the fixed cost of a chunk from loop-claude's archive, in tokens and hours against its items' S, and set F in `Score = 100 * V / (2**S + F)` to it.** | 2 | 2 | 1 | 3 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 |
| 170 | 0.4.0 | | decision | **Re-tune `WORKER_STEPS` and `REVIEW_STEPS` once 10 chunks have shipped under them, from how well L, E and I order their review rounds and 🔴 findings.** | 2 | 2 | 3 | 5 | 2 | 1.1 by 3, 1.4 by 5 | 10.0 |
@@ -82,10 +82,6 @@
## Details
### 153. Route the judge and the reviewers on the highest E + I in their chunk, an item the worker takes in included.
The judge reports the L of an item the worker takes in, and `routing.stepped` lifts the worker to the rung it picks, but the judge and the reviewers route on the routed item's E + I alone, which the archived route line holds. So when the worker takes in an item whose defects would escape further or break more, the judge and the reviewers stay on a lower rung than that item needs: a missed interrupt lets the worker settle a decision that is lilleman's (goal 1.2), and a weaker review lets a defect through (goal 1.1). The judge must report the highest E + I taken in too, and the route must carry it, archived so a rerun keeps it; loop-claude then hands the reviewers' rung it picks to the worker's prompt and environment.
### 161. Have loop-claude run a preflight script before each fresh worker and put its findings in the worker's prompt, so the worker no longer spends tool calls finding the chunk's mechanical facts.
In item 140's chunk, the worker spent 5 min 46 s of 30 min, 16 of its 68 messages and 226k of its 491k fresh tokens before its first edit, through 12 Bash and 5 Read calls. About half of those found facts a program computes the same way in every chunk:
@@ -235,6 +231,10 @@ A test audit across lilleman's projects on 2026-10-16 found 10 of 33 sampled tes
The hook judges a call before it runs, and `changed` in `bin/rule_hook/todo_check.py` compares `todo.md` with the index and HEAD as they stand then. A call that edits `todo.md` and commits it, such as a heredoc script followed by `git commit`, passes unchecked: PR #296's commit 149108e did, and the override marker it carried was never logged. larv-review's gate still runs `rank_todo.py --check`, so an unranked plan stops at merge rather than at commit. A git pre-commit hook would see the staged file.
### 174. In a chunk no item routed, ask the worker to name each todo.md item it takes, with its L and E + I, so loop-claude can route on an item it takes in.
The principles pass of PR #301 found that only `routed_by` in `bin/loop_claude/cycle/prompts.py` asks the worker to name each item it takes with its L and E + I, and `fresh` uses it only where an item routed the chunk. In an unrouted chunk the judge has no levels to report, so neither the taken-in L nor the taken-in E + I lifts the worker, the reviewers or the judge, and the first prompt names no reviewers' rung for a higher E + I.
### 122. Assemble the judge's prompt in `bin/loop_claude/cycle/judge.py`, importing only wording the worker also sees, and call the worker "the worker" in it.
All four seats of PR #257's scoring run named `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`. The prompt is spliced from constants in `cycle/prompts.py` (`ANSWER_DESTINATIONS`, `ANSWERS_RECORDED`, `either`) and `project/records.py` (`HUMAN_RULE_NAMES`), so a reader rebuilds it from three files. It says "the agent" where the code and `CLAUDE.md` → "Vocabulary" say "worker". Wording the worker also sees, such as `ANSWER_DESTINATIONS` in `prompts.resumed`, may stay in `cycle/prompts.py`. Make `JUDGE_RULES` easier to read.