Measure routing figures every 20th cycle and raise each call for a routing change once

This commit is contained in:
2026-10-15 00:37:01 +02:00
parent 8bc0d88b60
commit b1168ca8c5
14 changed files with 606 additions and 12 deletions
+2
View File
@@ -46,6 +46,8 @@
!bin/repo_checks/*.py
!bin/round_titles.py
!bin/rule-hook
!bin/routing_figures/
!bin/routing_figures/*.py
!bin/rule_hook/
!bin/rule_hook/*.py
!bin/score-outcomes
+1
View File
@@ -37,6 +37,7 @@ Decisions live in `docs/decisions.md`:
## Working rules
- When `routing-calls-raised.md`, at the root of loop-claude's archive (`$LOOP_CLAUDE_TRANSCRIPTS`, default `~/.claude/loop-claude-transcripts`), lists a call stamped after the latest stamp `todo.md` or `docs/decisions.md` → `routing:` cites from it, file one `todo.md` item to settle the calls stamped since, or refresh the one already filed. Its detail quotes them, with the latest stamp and the figures in `routing-figures.md` beside it. A change to the ladder, its bands or the delegation rule is the human's.
- When `~/.claude/settings.json` has uncommitted changes, commit them on a branch of their own and merge them through a PR: Claude Code writes that file from any session. Read their diff first. When it holds a credential, such as a token in `env` or `mcpServers`, ask the human to move it out of the file, and commit only once it is gone. A token in `env` moves to the shell profile; a server in `mcpServers` moves to `~/.claude.json` with `claude mcp add --scope user`.
## Scoring run
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- Every 20th cycle across all projects, `loop-claude` measures every finished chunk in its archive, with no model run, and writes the figures to `routing-figures.md` at the archive's root: tokens by project, subagent type, rung and tool, and for each chunk an item routed, by the rung it started on, its review rounds past each pass's first, its unfinished verdicts and its subagents' rungs. It interrupts you, with `claude` on the terminal in `~/.claude`, when chunks routed onto a rung took more review rounds or unfinished verdicts than chunks routed onto the next rung up, each rung with 5 chunks or more, or when 5 or more subagents of one type wrote reports longer than the tool results they read. Each call is raised once, then listed in `routing-calls-raised.md` beside the figures; the next chunk of this repo files a `todo.md` item for it. Expect a pause of about ten seconds at each 20th cycle, and an interrupt off a terminal to exit 2 as other interrupts do.
- A session delegates a task to a subagent only when it needs just the task's conclusion, such as a search across many files, and keeps edits in its own thread. It scores the task's R and C as for a `todo.md` item, and launches the subagent on the model and effort that the new `bin/subagent-rung <model> <R+C>` picks from the project's model, such as `model sonnet, effort high`. `CLAUDE.md` states the rule. Expect searches to run on cheaper models, and less of each session's context to fill with search output.
- `loop-claude` routes each chunk's worker on a ladder of model and effort rungs, from Haiku 5.5 at low effort to Fable 5.1 at max, when the project's model is on it. As a chunk starts, the R + C of `todo.md`'s top item in the next release picks the rung: 11–12 picks one rung above the project's rung, so Opus 5.5 at high for an Opus 5.5 project, and so on up to four rungs. An item of R + C 10 or less runs the worker at or below the project's model's cheapest rung, which for Opus 5.5 is medium effort, below the xhigh `settings.json` sets. While the worker runs below the project's model's cheapest rung, it launches every reviewer, comprehension seat and security pass, the full scan included, in a subagent on that rung. While it runs above that rung, it launches each comprehension seat and the security pass on that rung. It steps up one rung after every second unfinished verdict on a chunk, and to a higher rung when it takes in an item of higher R + C. Each cycle's header, and each finished or unfinished verdict, stop and step-up in the archive, name the worker's rung; a rerun resumes an open chunk on the latest rung archived for it. `docs/decisions.md` → `routing:` gives the ladder, its source and the bands. Expect a chunk on an easy item to cost much less than under xhigh, and perhaps to take more review rounds, and one on an item of R + C 15 or more to cost more.
- A session reads `todo.md` up to `## Details`, and the details of only the items its chunk takes. It reads a `docs/decisions.md` entry only where the entry's title leaves open what it needs. Both rules are in `CLAUDE.md`, so they hold from a session's start. The decisions rule replaces one in `rules/decisions.md`, which loads only once a session has opened `docs/decisions.md` or `AGENTS.md`. A chunk's opening line now counts the unreleased changes instead of listing them. Expect fewer tokens read before a chunk's first commit.
+1 -1
View File
@@ -92,7 +92,7 @@ pytest_add_cli_args=-k
Then, in the copy: `uvx --with pytest 'mutmut>=3,<4' run && uvx --with pytest 'mutmut>=3,<4' results`, and `uvx --with pytest 'mutmut>=3,<4' show <mutant>` to see a mutant's change. The config keys are mutmut 3's.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them, `rule_hook` imports `rank_todo.py`'s check, and `score_outcomes` imports `reading_cost`'s record path. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool; `model_routing/` holds the routing ladder, which loop-claude and `subagent-rung` read. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them, `rule_hook` imports `rank_todo.py`'s check, and `score_outcomes` imports `reading_cost`'s record path. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool; `model_routing/` holds the routing ladder, which loop-claude and `subagent-rung` read. `routing_figures/` measures loop-claude's archive for loop-claude's routing calls, reading tokens as `reading_cost` does and routes as `loop_claude/routing.py` does. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
## Tracking a new file
+16 -1
View File
@@ -60,6 +60,18 @@ and counts its unfinished verdicts afresh. A rung no longer on the ladder, or be
item's R + C picks, is dropped, and the chunk is routed afresh from todo.md. A todo.md or an
archive that cannot be read or parsed is said on stderr, and the chunk is routed without it.
Every 20th cycle across all projects in the archive, counting the cycles a verdict closed, it
measures every finished chunk there, with no model run: tokens by project, subagent type, rung and
the tool whose results they took in, and for each chunk an item routed, by the rung it started on,
its review rounds past each pass's first, its unfinished verdicts and its subagents' rungs. It
writes them to routing-figures.md at the archive's root. They call for a routing change when chunks
routed onto a rung took more review rounds or unfinished verdicts than chunks routed onto the next
rung up, each rung with 5 chunks or more, or when the reports of 5 or more subagents of one type
were longer than the tool results they read. Each call not raised before is an interrupt: claude
on the terminal in ~/.claude, asked the calls. Once that claude ends, the calls are listed in
routing-calls-raised.md beside the figures, and never raised again. A figure that cannot be
measured or written is said on stderr, and the loop goes on.
A change to bin/loop-claude or any bin/**/*.py takes effect at the next cycle boundary, once
loop-claude --help succeeds: the loop reloads into the edited script.
@@ -101,6 +113,8 @@ import sys
import typing
from transcript_archive.write import archive
from transcript_archive.location import archive_root, project_archive
from loop_claude.loop.routing_check import check_routing
from loop_claude.project.checkout import checkout_root
from loop_claude.cycle.run import Cycle, run_cycle
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
@@ -113,7 +127,6 @@ from todo_table import Item
from transcript_archive.read import latest_worker
from transcript_archive import outcomes
from loop_claude.paths import Paths
from transcript_archive.location import project_archive
from project_key import project_key
from model_routing.ladder import project_rung, Rung
from loop_claude.project.premises import read_premises
@@ -165,6 +178,8 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
before, loop_run = loop_run.route, after_verdict(loop_run, anchor, verdict)
if stop := archive_step(archive_folder, session, loop_run, before, effects):
return stop
if stop := check_routing(archive_root(paths.home, env), paths, effects):
return stop
def routed(loop_run: LoopRun, anchor: Rung | None, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Item | None]) -> LoopRun:
+55
View File
@@ -0,0 +1,55 @@
"""Every CYCLES_PER_CHECK-th cycle across all projects, measure the archive's routing figures, keep them at its root, and interrupt the human with each call for a routing change not raised before: docs/decisions.md → `routing:`."""
import pathlib
import sys
from routing_figures import archive
from routing_figures.measure import calls, render, totals
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.stop import needs_terminal, Stop
from loop_claude.paths import Paths
from claude_code.trust import trust
from utc_stamp import utc
from loop_claude.project.checkout import worktrees
CYCLES_PER_CHECK = 20
ANSWER_CALLS = "answer the routing calls above"
def check_routing(root: pathlib.Path, paths: Paths, effects: Effects) -> Stop | None:
"""Stops only for an interrupt it cannot hold; a figure it cannot measure or keep is said on stderr, and the loop goes on."""
new = new_calls(root, paths, effects)
if isinstance(new, Failure):
print(f"routing figures: {new.reason}", file=sys.stderr)
return None
return raise_calls(root, new, paths, effects) if new else None
def new_calls(root: pathlib.Path, paths: Paths, effects: Effects) -> list[str] | Failure:
"""The calls not raised before, measured and kept at every CYCLES_PER_CHECK-th cycle; none at any other."""
cycle = archive.cycles(root)
if isinstance(cycle, Failure) or not cycle or cycle % CYCLES_PER_CHECK:
return cycle if isinstance(cycle, Failure) else []
print(f"\n=== cycle {cycle} across loop-claude's archive: measuring the routing figures ===", flush=True)
measured = archive.measure_archive(root, str(paths.home))
if isinstance(measured, Failure):
return measured
figures = totals(measured)
found = calls(figures)
return archive.write_figures(root, render(figures, found, utc(effects.now()), cycle)) or archive.unraised(root, found)
def raise_calls(root: pathlib.Path, new: list[str], paths: Paths, effects: Effects) -> Stop | None:
"""Hands the terminal to a claude in this setup's checkout, where the routing decision lives; a call counts as raised once that claude ends."""
question = "loop-claude's routing figures call for a routing change, which is yours to decide:\n" + "".join(f"- {call}\n" for call in new) + f"The figures are in {root / archive.FIGURES}. The ladder's bands and the delegation rule are docs/decisions.md → `routing:`."
print(f"\n{question}", file=sys.stderr)
if stop := needs_terminal(ANSWER_CALLS, effects.on_terminal()):
return stop
trust(paths.claude_json, worktrees(str(paths.claude_dir), effects))
try:
effects.claude_on_terminal(["--permission-mode", "bypassPermissions", question], str(paths.claude_dir))
except OSError as failure:
return Stop(1, f"could not start claude for the routing calls: {failure.strerror}")
if unmarked := archive.mark_raised(root, new, utc(effects.now())):
print(f"routing figures: {unmarked.reason}", file=sys.stderr)
return None
+61 -1
View File
@@ -5,6 +5,7 @@ import io
import json
import os
import pathlib
import shutil
import stat
import subprocess
import sys
@@ -23,7 +24,7 @@ from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinishe
from loop_claude.cycle import interrupt, judge, prompts, retry, run, states, worker
from model_routing.ladder import LADDER, Rung
from loop_claude.cycle.retry import Limited
from loop_claude.loop import main, reload
from loop_claude.loop import main, reload, routing_check
from transcript_archive import outcomes, write
from loop_claude import paths, routing
from transcript_archive.location import project_archive
@@ -365,6 +366,64 @@ class Interrupts(unittest.TestCase):
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
class RoutingCheck(unittest.TestCase):
def setUp(self):
self.home = pathlib.Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.home)
self.root = self.home / "archive"
self.paths = paths.paths_from(self.home)
def chunks(self, count, verdicts):
"""count finished chunks, each closed by verdicts verdicts, each with one Explore subagent whose report outgrew its searches."""
for number in range(count):
folder = self.root / "log" / f"c{number}"
subagents = folder / "session" / "s" / "subagents"
subagents.mkdir(parents=True)
kept = "".join(f"2026-10-14T10:0{verdict}:00Z {outcomes.unfinished('w', 'e')}\n" for verdict in range(verdicts - 1))
(folder / "copy.outcomes.md").write_text(f"Task: t\n\n{kept}2026-10-14T11:00:00Z {outcomes.finished('w', 'm')}\n")
launch = {"type": "assistant", "timestamp": "2026-10-14T09:00:00Z", "message": {"id": "m", "content": [{"type": "tool_use", "id": "a", "name": "Agent", "input": {"subagent_type": "Explore"}}]}}
report = {"type": "user", "message": {"content": [{"type": "tool_result", "tool_use_id": "a", "content": "long report"}]}}
(folder / "copy.jsonl").write_text(json.dumps(launch) + "\n" + json.dumps(report) + "\n")
(subagents / "agent-a.jsonl").write_text("")
(subagents / "agent-a.meta.json").write_text(json.dumps({"agentType": "Explore", "toolUseId": "a"}))
def check(self, fake):
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()) as said:
stop = routing_check.check_routing(self.root, self.paths, fake.effects())
return stop, said.getvalue()
def test_measures_only_at_every_20th_cycle(self):
self.chunks(5, 3)
self.assertEqual(self.check(FakeEffects()), (None, ""))
self.assertFalse((self.root / "routing-figures.md").exists())
def test_hands_the_terminal_over_once_for_each_new_call(self):
self.chunks(5, 4)
fake = FakeEffects(on_terminal_claude=[0], git=[ran()])
stop, said = self.check(fake)
self.assertIsNone(stop)
call = "Explore subagents' reports took more of the worker's context than their searches would have."
self.assertIn(f"- {call}", said)
self.assertIn(f"- {call}", (self.root / "routing-figures.md").read_text())
((args),) = fake.made("claude_on_terminal")
self.assertEqual(args[:2], ["--permission-mode", "bypassPermissions"])
self.assertIn(call, args[2])
self.assertEqual(self.check(FakeEffects()), (None, ""))
def test_stops_off_a_terminal_and_raises_the_call_again_next_time(self):
self.chunks(5, 4)
stop, _ = self.check(FakeEffects(on_terminal=False))
self.assertEqual(stop, Stop(2, "the human is needed to answer the routing calls above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"))
self.assertEqual(self.check(FakeEffects(on_terminal=False))[0].code, 2)
def test_says_a_figure_it_cannot_write_and_goes_on(self):
self.chunks(5, 4)
(self.root / "routing-figures.md").mkdir()
stop, said = self.check(FakeEffects())
self.assertIsNone(stop)
self.assertIn("routing figures: could not write", said)
class ModelPrompt(unittest.TestCase):
OFFERED = [("claude-opus-5-5", "Opus 5.5"), ("claude-haiku-4-5", "Haiku 4.5")]
@@ -725,6 +784,7 @@ class ExitContract(unittest.TestCase):
("loop/main.py", "loop_run_from", "0 if task else 1", "__doc__.strip()", "no task is given"),
("loop/main.py", "loop_run_from", "1", 'f"the edited loop-claude cannot read the state', "a reload cannot read the state the running script handed it"),
("loop/reload.py", "exec_if_edited", "1", "f'could not reload loop-claude:", "python cannot start for a reload"),
("loop/routing_check.py", "raise_calls", "1", "f'could not start claude for the routing calls:", "claude cannot start for an interrupt"),
("project/checkout.py", "checkout_root", "1", "f'could not start git:", git),
("project/checkout.py", "checkout_root", "1", "found.stderr.strip()", git),
("project/model_choice.py", "get_or_ask_model", "1", "f'could not save the model in", models_),
View File
+92
View File
@@ -0,0 +1,92 @@
"""Reads the chunks of every project in loop-claude's archive, and keeps the routing figures and the calls already raised at the archive's root."""
import json
import pathlib
import re
from routing_figures.measure import Chunk, ChunkFigures, measured, Subagent
from failure import Failure
from transcript_archive.read import finished_chunks
from claude_code.entries import parse_entry
from file_writes import replace_file
from utc_stamp import STAMPED
FIGURES = "routing-figures.md"
RAISED = "routing-calls-raised.md"
RAISED_HEADING = "# Routing calls raised\n"
RAISED_CALL = re.compile(rf"^- {STAMPED} (.+)$", re.MULTILINE)
VERDICT = re.compile(rf"^{STAMPED} Verdict: (?:finished|unfinished)$", re.MULTILINE)
def cycles(root: pathlib.Path) -> int | Failure:
"""The cycles every project's loop has closed with a verdict."""
try:
return sum(len(VERDICT.findall(path.read_text(encoding="utf-8"))) for path in root.rglob("*.outcomes.md"))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not count the cycles in {root}: {failure}")
def measure_archive(root: pathlib.Path, home: str) -> list[ChunkFigures] | Failure:
"""Each finished chunk's figures, measured one chunk at a time, since the archive outgrows memory."""
try:
projects = sorted({path.parent.parent for path in root.rglob("*.outcomes.md")})
except OSError as failure:
return Failure(f"could not read the archive {root}: {failure}")
found = []
for project in projects:
chunks = finished_chunks(project)
if isinstance(chunks, Failure):
return chunks
for chunk in chunks:
folder = project / chunk.folder
try:
kept = "\n".join(path.read_text(encoding="utf-8") for path in sorted(folder.glob("*.outcomes.md")))
subagents = [Subagent(meta(path), entries(path)) for path in chunk.subagents]
found.append(measured(Chunk(str(project.relative_to(root)), kept, entries(chunk.transcript), subagents), home))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read the chunk {folder}: {failure}")
return found
def entries(path: pathlib.Path) -> list[dict]:
with open(path, "rb") as handle:
return [entry for line in handle if (entry := parse_entry(line))]
def meta(transcript: pathlib.Path) -> dict:
"""The .meta.json Claude Code writes beside a subagent's transcript; empty where it wrote none or one this cannot read."""
path = transcript.with_name(transcript.name.removesuffix(".jsonl") + ".meta.json")
try:
found = json.loads(path.read_text(encoding="utf-8"))
except (FileNotFoundError, ValueError):
return {}
return found if isinstance(found, dict) else {}
def write_figures(root: pathlib.Path, text: str) -> Failure | None:
unwritten = replace_file(root / FIGURES, text.encode())
return Failure(f"could not write {root / FIGURES}: {unwritten.reason}") if unwritten else None
def unraised(root: pathlib.Path, found: list[str]) -> list[str] | Failure:
"""The calls among found that were never raised."""
raised = read_raised(root)
return raised if isinstance(raised, Failure) else [call for call in found if call not in set(RAISED_CALL.findall(raised))]
def mark_raised(root: pathlib.Path, raised: list[str], at: str) -> Failure | None:
kept = read_raised(root)
if isinstance(kept, Failure):
return kept
# The first call opens a list below the heading; each later one joins it.
gap = "\n" if RAISED_CALL.search(kept) else "\n\n"
unwritten = replace_file(root / RAISED, (kept.rstrip("\n") + gap + "".join(f"- {at} {call}\n" for call in raised)).encode())
return Failure(f"could not write {root / RAISED}: {unwritten.reason}") if unwritten else None
def read_raised(root: pathlib.Path) -> str | Failure:
try:
return (root / RAISED).read_text(encoding="utf-8")
except FileNotFoundError:
return RAISED_HEADING
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {root / RAISED}: {failure}")
+211
View File
@@ -0,0 +1,211 @@
"""Token spend and routing outcomes of the chunks loop-claude finished, and the routing changes they call for: docs/decisions.md → `routing:`."""
import collections
import dataclasses
import re
import statistics
import typing
from claude_code.entries import assistant_parts, parts, tool_call
from git_command import commits
from reading_cost.measure import count
from model_routing.ladder import position, Rung
from round_titles import round_counts
from loop_claude.routing import route_from
from shell_command import simple_commands
from utc_stamp import STAMPED
from transcript_archive.outcomes import workers
# A rung or a subagent type is compared only once this many of its chunks or subagents are measured, so a single odd chunk calls for no change.
MEASURED_AT_LEAST = 5
DIMENSIONS = ("project", "subagent type", "rung", "tool")
# The tool label of a message that follows no tool call, such as the first message after a prompt.
PROMPT = "prompt"
UNFINISHED_AT = re.compile(rf"^{STAMPED} Verdict: unfinished$", re.MULTILINE)
@dataclasses.dataclass(frozen=True)
class Subagent:
"""meta is the subagent's .meta.json: its agentType, and the toolUseId of the worker's Agent call that launched it."""
meta: dict
entries: list[dict]
@dataclasses.dataclass(frozen=True)
class Chunk:
"""outcomes joins every outcomes file archived for the chunk; worker is its finished transcript's entries."""
project: str
outcomes: str
worker: list[dict]
subagents: list[Subagent]
@dataclasses.dataclass(frozen=True)
class Reports:
"""How many subagents of a type reported, and the characters of their reports and of the tool results they read."""
subagents: int
report: int
searches: int
def __add__(self, other: "Reports") -> "Reports":
return Reports(self.subagents + other.subagents, self.report + other.report, self.searches + other.searches)
@dataclasses.dataclass(frozen=True)
class ChunkFigures:
"""tokens maps each of DIMENSIONS to its labels' tokens; start is the rung a routed chunk started on, None where no item routed it."""
tokens: dict[str, collections.Counter[str]]
start: Rung | None
rounds_past_first: int
unfinished: int
delegated: collections.Counter[str]
reports: dict[str, Reports]
@dataclasses.dataclass(frozen=True)
class Figures:
chunks: int
tokens: dict[str, collections.Counter[str]]
routed: dict[Rung, list[ChunkFigures]]
reports: dict[str, Reports]
def measured(chunk: Chunk, home: str) -> ChunkFigures:
cycles = sorted(workers(chunk.outcomes), key=lambda cycle: cycle[0])
tokens: dict[str, collections.Counter[str]] = {dimension: collections.Counter() for dimension in DIMENSIONS}
for at, tools, spent in spend(chunk.worker):
add(tokens, chunk.project, "worker", str(rung_at(cycles, at)), tools, spent)
launches = {str(part.get("id")): (str(entry.get("timestamp", "")), arguments) for entry in chunk.worker for part in assistant_parts(entry) if (found := tool_call(part)) and found[0] == "Agent" and isinstance(arguments := found[1], dict)}
reported = results(chunk.worker)
delegated: collections.Counter[str] = collections.Counter()
reports: dict[str, Reports] = {}
for subagent in chunk.subagents:
launch = str(subagent.meta.get("toolUseId"))
launched_at, arguments = launches.get(launch, ("", {}))
kind = str(subagent.meta.get("agentType") or arguments.get("subagent_type") or "unknown")
rung = str(subagent_rung(subagent.entries, arguments, rung_at(cycles, launched_at)))
delegated[rung] += 1
reports[kind] = reports.get(kind, Reports(0, 0, 0)) + Reports(1, len(reported.get(launch, "")), sum(map(len, results(subagent.entries).values())))
for _, tools, spent in spend(subagent.entries):
add(tokens, chunk.project, kind, rung, tools, spent)
start = route_from(cycles[0][1]) if cycles else None
return ChunkFigures(tokens, start.rung if start and start.item else None, rounds_past_first(chunk.worker, home), len(UNFINISHED_AT.findall(chunk.outcomes)), delegated, reports)
def add(tokens: dict[str, collections.Counter[str]], project: str, kind: str, rung: str, tools: list[str], spent: int) -> None:
tokens["project"][project] += spent
tokens["subagent type"][kind] += spent
tokens["rung"][rung] += spent
# Split whole, the first tools taking the remainder, so the tool table adds up to the others.
for place, tool in enumerate(tools):
tokens["tool"][tool] += spent // len(tools) + (place < spent % len(tools))
def rung_at(cycles: list[tuple[str, str]], timestamp: str) -> Rung | str:
"""The rung the worker line stamped first at or after timestamp names: the rung of the cycle that ran then. A line off the ladder names itself."""
second = timestamp[:19] + "Z"
found = next((rung for stamp, rung in cycles if stamp >= second), cycles[-1][1] if cycles else "unrouted")
route = route_from(found)
return route.rung if route else found
def subagent_rung(entries: list[dict], arguments: dict, worker: Rung | str) -> Rung:
"""The model the subagent ran, at the effort its Agent call set, or else at the worker's."""
model = next((message["model"] for entry in entries if entry.get("type") == "assistant" and isinstance(message := entry.get("message"), dict) and isinstance(message.get("model"), str)), str(arguments.get("model") or "unknown"))
effort = arguments.get("effort") or (worker.effort if isinstance(worker, Rung) else "")
return Rung(model, str(effort))
def spend(entries: list[dict]) -> list[tuple[str, list[str], int]]:
"""Each message's timestamp, the tools whose results it took in, and its input, cache-written and output tokens.
Claude Code writes one entry per content block of a message, each with the message's usage, so a message counts once.
"""
messages: dict[str, tuple[str, object, list[str]]] = {}
for entry in entries:
if entry.get("type") != "assistant" or not isinstance(message := entry.get("message"), dict):
continue
key = str(message.get("id") or entry.get("uuid"))
kept = messages.setdefault(key, (str(entry.get("timestamp", "")), message.get("usage"), []))
kept[2].extend(name for part in assistant_parts(entry) if (found := tool_call(part)) and isinstance(name := found[0], str))
spent = []
before = [PROMPT]
for at, usage, called in messages.values():
tokens = sum(count(usage, field) for field in ("input_tokens", "cache_creation_input_tokens", "output_tokens")) if isinstance(usage, dict) else 0
spent.append((at, before, tokens))
before = called or [PROMPT]
return spent
def results(entries: list[dict]) -> dict[str, str]:
"""Each tool result's text by the id of the call it answers."""
found = {}
for entry in entries:
for part in parts(entry):
if isinstance(part, dict) and part.get("type") == "tool_result":
body = part.get("content")
found[str(part.get("tool_use_id"))] = body if isinstance(body, str) else "".join(block["text"] for block in body if isinstance(block, dict) and isinstance(block.get("text"), str)) if isinstance(body, list) else ""
return found
def rounds_past_first(worker: list[dict], home: str) -> int:
# A commit retried after a failed hook repeats its title, so each title counts once.
titles = dict.fromkeys(
commit.messages[0].split("\n", 1)[0]
for entry in worker for part in assistant_parts(entry)
if (found := tool_call(part)) and found[0] == "Bash" and isinstance(found[1], dict)
for commit in commits(simple_commands(str(found[1].get("command", "")), str(entry.get("cwd") or "/"), home), home) if commit.messages
)
return sum(max(0, rounds - 1) for rounds in round_counts(list(titles)).values())
def totals(chunks: list[ChunkFigures]) -> Figures:
tokens = {dimension: sum((chunk.tokens.get(dimension, collections.Counter()) for chunk in chunks), collections.Counter()) for dimension in DIMENSIONS}
routed: dict[Rung, list[ChunkFigures]] = {}
reports: dict[str, Reports] = {}
for chunk in chunks:
if chunk.start:
routed.setdefault(chunk.start, []).append(chunk)
for kind, found in chunk.reports.items():
reports[kind] = reports.get(kind, Reports(0, 0, 0)) + found
return Figures(len(chunks), tokens, routed, reports)
def calls(figures: Figures) -> list[str]:
"""Each routing change the figures call for, worded alike whatever the numbers, so a call already raised is known again."""
compared = sorted((rung for rung, chunks in figures.routed.items() if len(chunks) >= MEASURED_AT_LEAST and position(rung) >= 0), key=position)
found = []
for lower, upper in zip(compared, compared[1:]):
for what, of in (("review rounds past each pass's first", lambda chunk: chunk.rounds_past_first), ("unfinished verdicts", lambda chunk: chunk.unfinished)):
if mean(figures.routed[lower], of) > mean(figures.routed[upper], of):
found.append(f"Chunks routed onto {lower} took more {what} than chunks routed onto {upper}.")
# Characters stand in for tokens: both sides are text the worker would read.
found += [f"{kind} subagents' reports took more of the worker's context than their searches would have." for kind, reports in sorted(figures.reports.items()) if reports.subagents >= MEASURED_AT_LEAST and reports.report > reports.searches]
return found
def mean(chunks: list[ChunkFigures], of: typing.Callable[[ChunkFigures], int]) -> float:
return statistics.mean(map(of, chunks))
def render(figures: Figures, found: list[str], at: str, cycle: int) -> str:
sections = [
"# Routing figures",
f"Measured {at}, at cycle {cycle} across loop-claude's archive, from {figures.chunks} finished chunk{'' if figures.chunks == 1 else 's'}. Tokens are input, cache-written and output tokens; cache reads are left out. A message's tokens count towards the tools whose results it took in, split evenly.",
"## Calls for a routing change",
"".join(f"- {call}\n" for call in found).rstrip("\n") or "None.",
*[part for dimension in DIMENSIONS for part in (f"## Tokens by {dimension}", table((dimension.capitalize(), "Tokens"), [(label, str(tokens)) for label, tokens in figures.tokens[dimension].most_common()]))],
"## Routed chunks by the rung they started on",
table(("Rung", "Chunks", "Review rounds past each pass's first, mean", "Unfinished verdicts, mean", "Subagent rungs"), [
(str(rung), str(len(chunks)), f"{mean(chunks, lambda chunk: chunk.rounds_past_first):.1f}", f"{mean(chunks, lambda chunk: chunk.unfinished):.1f}", ", ".join(f"{label} {times}" for label, times in sum((chunk.delegated for chunk in chunks), collections.Counter()).most_common()))
for rung, chunks in sorted(figures.routed.items(), key=lambda item: position(item[0]))
]),
"## Subagent reports against the searches they ran, in characters",
table(("Subagent type", "Subagents", "Reports", "Searches"), [(kind, str(found.subagents), str(found.report), str(found.searches)) for kind, found in sorted(figures.reports.items())]),
]
return "\n\n".join(sections) + "\n"
def table(header: tuple[str, ...], rows: list[tuple[str, ...]]) -> str:
if not rows:
return "None."
return "\n".join(f"| {' | '.join(row)} |" for row in [header, tuple("---" for _ in header), *rows])
+158
View File
@@ -0,0 +1,158 @@
import collections
import json
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from failure import Failure
from model_routing.ladder import Rung
from routing_figures import archive
from routing_figures.measure import Chunk, ChunkFigures, calls, measured, render, Reports, Subagent, totals
from transcript_archive import outcomes
LOW = Rung("claude-opus-5-5", "medium")
HIGH = Rung("claude-opus-5-5", "high")
LOW_ROUTE = f"{LOW}, routed by item 7's R + C of 9"
def message(id_, at, *parts, model="claude-opus-5-5", tokens=(1, 10, 100)):
return {"type": "assistant", "timestamp": at, "cwd": "/c", "message": {"id": id_, "model": model, "content": list(parts), "usage": {"input_tokens": tokens[0], "cache_creation_input_tokens": tokens[1], "output_tokens": tokens[2], "cache_read_input_tokens": 5000}}}
def call(name, id_="t1", **arguments):
return {"type": "tool_use", "id": id_, "name": name, "input": arguments}
def result(id_, text):
return {"type": "user", "message": {"content": [{"type": "tool_result", "tool_use_id": id_, "content": [{"type": "text", "text": text}]}]}}
def stamped(at, outcome):
return f"{at} {outcome}\n"
def figures(start=LOW, rounds=0, unfinished=0, reports=None):
return ChunkFigures({}, start, rounds, unfinished, collections.Counter(), reports or {})
class Measured(unittest.TestCase):
def test_counts_each_messages_tokens_once_by_project_type_rung_and_the_tool_it_followed(self):
worker = [
message("m1", "2026-10-14T10:00:00.100Z", {"type": "text", "text": "reading"}),
message("m1", "2026-10-14T10:00:00.200Z", call("Read", file_path="/c/a")),
message("m2", "2026-10-14T10:05:00.000Z", {"type": "text", "text": "done"}),
message("m3", "2026-10-14T11:00:00.000Z", {"type": "text", "text": "on"}),
]
kept = stamped("2026-10-14T10:30:00Z", outcomes.unfinished(LOW_ROUTE, "e")) + stamped("2026-10-14T10:30:00Z", outcomes.stepped_up(str(HIGH))) + stamped("2026-10-14T12:00:00Z", outcomes.finished(str(HIGH), "merged"))
found = measured(Chunk("log", kept, worker, []), "/h")
self.assertEqual(found.tokens["project"], {"log": 333})
self.assertEqual(found.tokens["subagent type"], {"worker": 333})
self.assertEqual(found.tokens["rung"], {str(LOW): 222, str(HIGH): 111})
self.assertEqual(found.tokens["tool"], {"prompt": 222, "Read": 111})
self.assertEqual((found.start, found.unfinished), (LOW, 1))
def test_reads_a_subagents_rung_from_its_model_and_the_effort_its_call_set_or_else_the_workers(self):
worker = [
message("m1", "2026-10-14T10:00:00Z", call("Agent", "a1", subagent_type="Explore", model="sonnet", effort="high", prompt="find"), call("Agent", "a2", subagent_type="Explore", prompt="find")),
result("a1", "x" * 30),
result("a2", "y" * 5),
]
subagents = [
Subagent({"agentType": "Explore", "toolUseId": "a1"}, [message("s1", "2026-10-14T10:00:01Z", model="claude-sonnet-5-5"), result("r", "z" * 20)]),
Subagent({"agentType": "Explore", "toolUseId": "a2"}, [message("s2", "2026-10-14T10:00:01Z")]),
]
found = measured(Chunk("log", stamped("2026-10-14T11:00:00Z", outcomes.finished(LOW_ROUTE, "m")), worker, subagents), "/h")
self.assertEqual(found.delegated, {"claude-sonnet-5-5 at high effort": 1, str(LOW): 1})
self.assertEqual(found.tokens["subagent type"], {"worker": 111, "Explore": 222})
self.assertEqual(found.reports, {"Explore": Reports(2, 35, 20)})
def test_counts_review_rounds_past_each_passs_first_once_per_title(self):
commit = lambda id_, title: message(id_, "2026-10-14T10:00:00Z", call("Bash", command=f"git commit -m '{title}'"))
worker = [commit("m1", "Answer stability round 1: 1 🔴, 0 🟡"), commit("m2", "Answer stability round 2: 0 🔴, 0 🟡"), commit("m3", "Answer stability round 2: 0 🔴, 0 🟡"), commit("m4", "Answer prose round 1: 0 🔴, 1 🟡")]
self.assertEqual(measured(Chunk("log", "", worker, []), "/h").rounds_past_first, 1)
def test_starts_no_route_for_a_chunk_no_item_routed(self):
kept = stamped("2026-10-14T10:00:00Z", outcomes.finished(str(LOW), "m"))
found = measured(Chunk("log", kept, [message("m1", "2026-10-14T09:00:00Z")], []), "/h")
self.assertIsNone(found.start)
self.assertEqual(found.tokens["rung"], {str(LOW): 111})
class Calls(unittest.TestCase):
def test_calls_when_chunks_on_a_rung_took_more_rounds_or_unfinished_verdicts_than_the_rung_above(self):
chunks = [figures(LOW, rounds=2)] * 5 + [figures(HIGH, unfinished=1)] * 5
self.assertEqual(calls(totals(chunks)), [
f"Chunks routed onto {LOW} took more review rounds past each pass's first than chunks routed onto {HIGH}.",
])
self.assertEqual(calls(totals([figures(LOW, unfinished=1)] * 5 + [figures(HIGH)] * 5)), [
f"Chunks routed onto {LOW} took more unfinished verdicts than chunks routed onto {HIGH}.",
])
def test_compares_only_rungs_with_five_chunks_each(self):
self.assertEqual(calls(totals([figures(LOW, rounds=2)] * 5 + [figures(HIGH)] * 4)), [])
def test_calls_when_a_subagent_types_reports_outgrow_the_searches_they_ran(self):
self.assertEqual(calls(totals([figures(reports={"Explore": Reports(1, 30, 20)})] * 5)), [
"Explore subagents' reports took more of the worker's context than their searches would have.",
])
self.assertEqual(calls(totals([figures(reports={"Explore": Reports(1, 30, 20)})] * 4)), [])
self.assertEqual(calls(totals([figures(reports={"Explore": Reports(1, 20, 30)})] * 5)), [])
def test_renders_each_dimension_largest_first_and_the_calls(self):
chunk = ChunkFigures({"project": collections.Counter({"a": 2, "b": 20})}, LOW, 1, 0, collections.Counter({str(HIGH): 2}), {"Explore": Reports(2, 3, 4)})
text = render(totals([chunk]), ["Call."], "2026-10-15T00:00:00Z", 40)
self.assertIn("Measured 2026-10-15T00:00:00Z, at cycle 40 across loop-claude's archive, from 1 finished chunk.", text)
self.assertIn("## Calls for a routing change\n\n- Call.\n", text)
self.assertIn("| b | 20 |\n| a | 2 |", text)
self.assertIn(f"| {LOW} | 1 | 1.0 | 0.0 | {HIGH} 2 |", text)
self.assertIn("| Explore | 2 | 3 | 4 |", text)
self.assertIn("## Calls for a routing change\n\nNone.\n", render(totals([]), [], "2026-10-15T00:00:00Z", 20))
class Archive(unittest.TestCase):
def setUp(self):
self.root = pathlib.Path(tempfile.mkdtemp())
self.addCleanup(lambda: __import__("shutil").rmtree(self.root))
def chunk(self, key, name, kept, worker, subagent=None):
folder = self.root / key / name
(folder / "session" / "s" / "subagents").mkdir(parents=True)
(folder / "copy.outcomes.md").write_text(f"Task: t\n\n{kept}")
(folder / "copy.jsonl").write_text("".join(json.dumps(entry) + "\n" for entry in worker))
if subagent:
meta, entries = subagent
(folder / "session" / "s" / "subagents" / "agent-a.jsonl").write_text("".join(json.dumps(entry) + "\n" for entry in entries))
(folder / "session" / "s" / "subagents" / "agent-a.meta.json").write_text(json.dumps(meta))
def test_counts_the_verdicts_across_every_projects_chunks(self):
self.chunk("home/x", "c1", stamped("2026-10-14T10:00:00Z", outcomes.unfinished("w", "e")) + stamped("2026-10-14T10:01:00Z", outcomes.stopped("w", "Ctrl-C")) + stamped("2026-10-14T10:02:00Z", outcomes.finished("w", "e")), [])
self.chunk("log", "c2", stamped("2026-10-14T10:00:00Z", outcomes.interrupted("q", "e")), [])
self.assertEqual(archive.cycles(self.root), 2)
self.assertEqual(archive.cycles(self.root / "missing"), 0)
def test_measures_each_finished_chunk_keyed_by_its_project(self):
self.chunk("home/x", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m")), [message("m1", "2026-10-14T09:00:00Z", call("Agent", "a1", subagent_type="Explore"))], ({"agentType": "Explore", "toolUseId": "a1"}, [message("s1", "2026-10-14T09:00:01Z")]))
self.chunk("log", "c2", stamped("2026-10-14T10:00:00Z", outcomes.unfinished(LOW_ROUTE, "e")), [message("m1", "2026-10-14T09:00:00Z")])
found = archive.measure_archive(self.root, "/h")
self.assertEqual([chunk.tokens["project"] for chunk in found], [{"home/x": 222}])
self.assertEqual(found[0].tokens["subagent type"], {"worker": 111, "Explore": 111})
def test_keeps_the_figures_and_says_which_calls_it_raises_for_the_first_time(self):
self.assertIsNone(archive.write_figures(self.root, "# Routing figures\n"))
self.assertEqual((self.root / archive.FIGURES).read_text(), "# Routing figures\n")
self.assertEqual(archive.unraised(self.root, ["A.", "B."]), ["A.", "B."])
self.assertIsNone(archive.mark_raised(self.root, ["A."], "2026-10-15T00:00:00Z"))
self.assertEqual(archive.unraised(self.root, ["A.", "B."]), ["B."])
self.assertIsNone(archive.mark_raised(self.root, ["B."], "2026-10-15T01:00:00Z"))
self.assertEqual((self.root / archive.RAISED).read_text(), "# Routing calls raised\n\n- 2026-10-15T00:00:00Z A.\n- 2026-10-15T01:00:00Z B.\n")
def test_fails_on_an_archive_it_cannot_read(self):
(self.root / archive.RAISED).mkdir()
self.assertIsInstance(archive.unraised(self.root, ["A."]), Failure)
if __name__ == "__main__":
unittest.main()
+6 -2
View File
@@ -6,9 +6,13 @@ from claude_code.transcript import first_timestamped_line
from project_key import project_key
def archive_root(home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
"""Where loop-claude archives every project's chunks."""
return pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else home / ".claude" / "loop-claude-transcripts"
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
archive = pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else home / ".claude" / "loop-claude-transcripts"
return archive / project_key(checkout_root, home)
return archive_root(home, env) / project_key(checkout_root, home)
def chunk_folder(project_archive: pathlib.Path, transcript: bytes) -> pathlib.Path:
+1 -1
View File
@@ -176,7 +176,7 @@ Valid while a mutation run on a chunk's changed lines stays cheap beside the rev
The rungs, least competent first: Haiku 5.5 at low, medium, high, xhigh and max; Sonnet 5.5 at high; Opus 5.5 at medium, high, xhigh and max; Fable 5.1 at max. The order and the cut come from the Artificial Analysis Intelligence Index (https://artificialanalysis.ai/leaderboards/models, read 2026-10-14), which scores each model at each effort and gives a cost per task. A rung stays only where no cheaper rung scores as high, so a step up always buys more score (goal 1.4). That cut drops Sonnet 5.5 at low, medium, xhigh and max, Opus 5.5 at low, and every Fable 5.1 rung, and every tie in the index falls among the dropped rungs. lilleman put Fable 5.1 at max back on top: the index scores it under Opus 5.5 at high, but Anthropic's model catalog names Fable the most capable model, and for the hardest tasks lilleman wants the highest quality at any price.
A chunk's worker runs on the rung its item picks, below the project's rung too. Only chunks run below the project's rung can prove that a cheaper worker holds quality (goal 1.1). So on 2026-10-14 lilleman chose a trial on real chunks: a replay of past chunks spends on work nothing ships (goal 1.4). Review holds quality meanwhile. While the worker runs below the project's rung, loop-claude's prompt has it launch each reviewer, each comprehension seat and the security pass's subagent on the project's rung, through the Agent tool's model and effort. It runs a full scan only from such a subagent, because the `claude-security` plugin's researchers and verifiers inherit the model that starts them. A subagent that cannot run the scan leaves it unavailable, so larv-review runs `/security-review` in its place. The cost of a cheaper worker shows as extra review rounds and unfinished verdicts, which `todo.md` item 94 is to measure, interrupting lilleman when a rung takes more of either than the rung above. A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. lilleman chose medium knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes.
A chunk's worker runs on the rung its item picks, below the project's rung too. Only chunks run below the project's rung can prove that a cheaper worker holds quality (goal 1.1). So on 2026-10-14 lilleman chose a trial on real chunks: a replay of past chunks spends on work nothing ships (goal 1.4). Review holds quality meanwhile. While the worker runs below the project's rung, loop-claude's prompt has it launch each reviewer, each comprehension seat and the security pass's subagent on the project's rung, through the Agent tool's model and effort. It runs a full scan only from such a subagent, because the `claude-security` plugin's researchers and verifiers inherit the model that starts them. A subagent that cannot run the scan leaves it unavailable, so larv-review runs `/security-review` in its place. The cost of a cheaper worker shows as extra review rounds and unfinished verdicts. Every 20th cycle across the archive, loop-claude measures them in code, with no model run (goal 1.4), and interrupts lilleman once per call when chunks routed onto a rung took more of either than chunks routed onto the next rung up, each rung with 5 chunks or more. Five keeps one odd chunk from calling for a change (goal 1.2). It also calls when the reports of 5 or more subagents of one type were longer than the tool results they read, so a delegated search cost the worker more context than it saved. `bin/routing_figures/` measures and words the calls. A loop on another project cannot commit here, so it writes the figures and the calls raised at the archive's root, and this repo's next chunk files the plan item, as `AGENTS.md` says. A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. lilleman chose medium knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes.
A model off the ladder routes nothing: claude runs it at the effort its settings give. R + C steps a task from the project's rung: 2–4 three rungs down, 5–6 two down, 7–8 one down, 9–10 none, 11–12 one up, 13–14 two up, 15–16 three up, 17 and above four up. A step past either end stops there. The bands spread the R + C that `todo.md`'s items held on 2026-10-14, 2 to 12, across the rungs, so most tasks step down. S never moves a rung, so a huge task with low R and C runs cheap. While the worker runs above the project's rung, loop-claude's prompt has it launch the comprehension seats and the security pass's subagent on the project's rung too. So the seats score every chunk on one rung, and the security pass spends no more than the project chose. The prompt after each unfinished verdict repeats this, since a step-up may have taken the worker above the project's rung. The other reviewers inherit the worker's higher rung until item 90 routes them. `score-outcomes` had too few finished chunks to ground the bands, and a change to the ladder or the bands is lilleman's.
+1 -6
View File
@@ -26,7 +26,6 @@ Release 0.2.0 takes items 125, 114 and 124 first, whatever their scores: Shape i
| ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score |
|---|---|---|---|---|---|---|---|---|---|---|
| 94 | 0.1.0 | decision | **Every 20th cycle, have loop-claude measure token spend and routing outcomes, file them here, and interrupt the human only when they call for a routing change.** | 3 | 5 | 6 | 6 | 7 | 1.1, 1.4 | 21.6 |
| 90 | 0.1.0 | | **Route each reviewer by the chunk's R + C, once a replay of past chunks shows the routed reviewer finds the 🔴 and 🟡 findings on record.** | 4 | 5 | 8 | 6 | 7 | 1.4 | 20.2 |
| 89 | 0.1.0 | | **Run the judge on the rung its chunk's R + C picks, once a replay of archived transcripts shows it returns the verdicts on record.** | 3 | 4 | 7 | 5 | 6 | 1.4 | 17.5 |
| 136 | 0.1.0 | decision | **Route a chunk on the item its worker says it takes, in place of the next release's first row.** | 2 | 3 | 4 | 4 | 5 | 1.4 | 14.8 |
@@ -85,10 +84,6 @@ Release 0.2.0 takes items 125, 114 and 124 first, whatever their scores: Shape i
## Details
### 94. Every 20th cycle, have loop-claude measure token spend and routing outcomes, file them here, and interrupt the human only when they call for a routing change.
Count the cycles across all projects in loop-claude's archive, which holds every project's transcripts on the machine. Measure in code, as `bin/reading-cost` does, so the check itself costs no model run (goal 1.4): tokens by project, subagent type, rung and tool, and per routed chunk its review rounds and unfinished verdicts, and the rungs its worker delegated subagents on, so the figures show a cheap subagent whose conclusions cost review rounds. A loop on another project cannot commit here, so it writes the figures to the archive, and this repo's next chunk files or refreshes one plan item from them. The ladder's thresholds and the delegation rule are decisions, so a change to them is the human's. Interrupt when chunks on a rung took more review rounds or unfinished verdicts than chunks on the rung above, or when subagent reports fill more of the worker's peak context than the searches they replaced would have; never twice for the same figures (goal 1.2).
### 90. Route each reviewer by the chunk's R + C, once a replay of past chunks shows the routed reviewer finds the 🔴 and 🟡 findings on record.
larv-review passes the rung through the Agent tool's `model` and `effort`. A reviewer never steps down when the worker has stepped down on the same chunk, so a cheap worker always meets a reviewer on the project model or above.
@@ -189,7 +184,7 @@ PR #257's scoring run put Shape at 6.25, the lowest dimension. `bin/` holds abou
### 118. Have `score-outcomes` count review rounds only from chunks merged after 2026-10-13, when review passes were narrowed, so C's medians compare chunks reviewed under one rule.
Since 2026-10-13 (`docs/decisions.md` → "review: stability and product-owner report only…"), reviewers report fewer findings and later rounds review only the answers to earlier ones, to cut rounds. A chunk merged before it took more rounds for the same C, so mixing both eras skews C's medians and item 94's comparison of rungs.
Since 2026-10-13 (`docs/decisions.md` → "review: stability and product-owner report only…"), reviewers report fewer findings and later rounds review only the answers to earlier ones, to cut rounds. A chunk merged before it took more rounds for the same C, so mixing both eras skews C's medians.
### 130. Move the vote counting out of `bin/larv_review/ruling.py`'s `votes`, so reading a votes file decides nothing.