Keep the routing ladder on the call of 2026-11-18
CI / tests (pull_request) Has been cancelled

This commit is contained in:
2026-11-18 16:15:18 +01:00
parent e898fc0f03
commit 2ed5c042d3
+2
View File
@@ -686,6 +686,8 @@ A project's rung is its model's cheapest rung, which for Opus 5.5 is its default
2026-11-15, lilleman asked for the ladder as a file that a project can replace, on a day when an outage left only Fable 5.2 answering: the rungs were a constant in `bin/model_routing/ladder.py`, so no project could leave Sonnet 5.5 and Opus 5.5 without a release. The shipped ladder is `bin/model_routing/routing-ladder.json`, a JSON list of rungs, least competent first. A project that keeps a `routing-ladder.json` in its config's folder, `~/.claude-projects/<key>/`, runs on that ladder whole, and the steps count on it as on the shipped one. Every project still runs the same routing rules and steps, so the workflow of goal 1.3 holds; the ladder joins what a project's config folder may supply, which that goal's list does not name yet. lilleman chose a file per project over one per machine, so a single project can differ. A ladder holds one model of each family, since the Agent tool launches a subagent or a review by its family's name. The project's file is lilleman's to write, as any change to the ladder is, and the loop never commits it. loop-claude reads the project's ladder as each cycle starts; `subagent-rung` and rule-hook's `reviewer-rung` read it each time they run, from the checkout they run in. A ladder file that holds a fault stops each of them, since work routed on the shipped ladder in its place could run on a model the project cannot reach. The routing figures, `judge-replay` and `review-replay` span every project, so they read the shipped ladder: a rung only a project's own ladder holds is left out of the figures' calls, and no replay qualifies it. lilleman shipped the change as 0.10.1, a patch cut from the tag v0.10.0, on the day of the outage: it adds a file format and an exit 1 to `subagent-rung`, which the technical principles' semantic versioning gives a minor, and lilleman asked for the patch "even if that breaks semver", so that the release in use could run again without taking the unreleased work on `main`.
2026-11-18, lilleman kept the ladder and the steps as they are, settling the call raised on the routing figures measured at 2026-11-18T15:04:48Z: chunks routed onto Sonnet 5.5 at high took more review rounds past each pass's first than chunks routed onto Opus 5.5 at medium. When lilleman ruled, `routing-calls-raised.md` held no line for that call, so this note cites the time the figures were measured. The figures came from 19 chunks finished since the 2026-11-17 reset. The 13 chunks started on Sonnet 5.5 at high took a mean 0.5 rounds, 0.0 unfinished verdicts and 0.0 later fixes. The 6 started on Opus 5.5 at medium took 0.5, 0.2 and 0.0. Both means round to 0.5, so the gap in rounds is under 0.1. Sonnet 5.5 did better on unfinished verdicts, and it is the cheaper rung (goal 1.4).
Valid while the index ranks these models, and while no newer model joins the catalog.
## routing: the offline classifier may decide finished, since lilleman ruled a miss rate of about 1 in 100 decided close enough to none; the judge's model keeps every other verdict kind