Answer stability round 1: 1 🔴, 3 🟡 (0 misread); keep the global rules generic, and say in one always-loaded rule of this repo where AGENTS.md's parts went

This commit is contained in:
2026-11-11 16:46:27 +01:00
parent 9b770e4893
commit dcb9abf644
10 changed files with 34 additions and 29 deletions
+3 -8
View File
@@ -1,12 +1,7 @@
---
paths:
- "todo.md"
---
# Routing calls: file them in the plan
`routing-calls-raised.md` sits at the root of loop-claude's archive (`$LOOP_CLAUDE_TRANSCRIPTS`, default `~/.claude/loop-claude-transcripts`). Each of its lines is a call or a `Reset`, with a stamp. A call is new when it is stamped after the latest `Reset` and after the latest stamp that `todo.md` or `docs/decisions.md` → `routing:` cites.
A routing call is a change to the routing ladder, its steps or the delegation rule that loop-claude's figures call for. loop-claude puts each call to the human once, and appends it to `routing-calls-raised.md`, at the root of its archive (`$LOOP_CLAUDE_TRANSCRIPTS`, default `~/.claude/loop-claude-transcripts`). Each line of that file is a call or a `Reset`, with a stamp. A call is new when it is stamped after the latest `Reset` and after the latest stamp that `todo.md` or `docs/decisions.md` → `routing:` cites.
When there are new calls, file one `todo.md` item to settle them, or refresh the item already filed. Its detail quotes those calls with their stamps, and the routed-chunks table of `routing-figures.md` beside it, never the project names its other tables hold.
When there are new calls, file one `todo.md` item to settle them, or refresh the item already filed. Its detail quotes the new calls with their stamps. It also quotes the routed-chunks table of `routing-figures.md`, which sits in the same folder, and none of that file's other tables, since they name other projects.
A change to the ladder, its steps or the delegation rule is the human's. The chunk that settles the calls cites the latest stamp it settled in `docs/decisions.md` → `routing:`, whatever the human decided, so they are not filed again. Once a chunk that changes the ladder, its steps or the delegation rule has merged, append `- <UTC time> Reset` to `routing-calls-raised.md`: the figures then count only chunks finished after it, and its calls may be raised again.
Settling a call is the human's. The chunk that settles the calls cites, in `docs/decisions.md` → `routing:`, the latest stamp it settled, whatever the human decided, so the calls are not filed again. Once a chunk that changes the ladder, its steps or the delegation rule has merged, append `- <UTC time> Reset` to `routing-calls-raised.md`. The figures then count only chunks finished after the reset, and a call settled before it may be raised again.
@@ -0,0 +1,7 @@
# This repo keeps no `AGENTS.md`
`~/.claude` is a checkout of this repo, so an `AGENTS.md` here would load into every other project's sessions. Where a rule names a project's `AGENTS.md`, read it here as:
- the project principles and the title of each decision no project principle covers: the head of `docs/decisions.md`, before `# Decisions`;
- a working rule: a file in `.claude/rules/`, which loads only in this repo;
- the scoring-run values: `skills/comprehension-panel/seat-prompts.md` → "claude-config's values".
+2 -2
View File
@@ -2,8 +2,8 @@
## Unreleased
- `~/.claude` holds no `AGENTS.md` any more, so every session in another project starts about 3,600 tokens smaller. In `~/code/claude-config`, only the clone's `CLAUDE.md` loads, not the release's as well. This repo's working rules load from its `.claude/rules/`, and its scoring-run values sit in `skills/comprehension-panel/seat-prompts.md`.
- The head of `docs/decisions.md` groups most decision titles into seven project principles, and each new decision is checked against them: a principle covers it, it breaks one, or it suggests a new one for you to approve.
- `~/.claude` holds no `AGENTS.md` any more, so each session in your projects starts about 3,600 tokens smaller, and a session in `~/code/claude-config` loads its `CLAUDE.md` once instead of twice.
- A project's `AGENTS.md` may group decision titles into project principles, and each new decision is checked against them: a principle covers it, it breaks one, or it suggests a new one for you to approve.
- The prose reviewer now holds prose to the negation rule: write `doesn't cost anything`, never `costs nothing`, after any verb or its preposition.
- `rule-hook` now checks files at `git commit`: each file that differs from HEAD or is untracked and not ignored, even one the commit leaves out. It holds each to `action-pin`, `image-pin`, `ignorefile-sorted` and `no-ai-attribution`, so a file a Bash command wrote is checked as an Edit or Write is. A `# rule-override` line in the file or the command overrides a breach.
- `chunk-pr merge` now refuses while a scoring run the plan calls is unrecorded on the branch, and `larv-review needs-scoring-run` calls one after a release is cut with none.
+2 -2
View File
@@ -10,9 +10,9 @@ How work proceeds in any repo, unless the repo's `AGENTS.md` says otherwise. The
- **When the next release has no items left and work has landed towards it, its release review is the chunk** (`larv-review` §3 → "Release review"). **Once that has merged and the release holds no item, the release is finished: stop.** Report what it holds and that it is ready to cut. Under `/loop`, this ends the loop. Until the human cuts it, only a security fix, or an item its release review filed, lands, into that release; never start a later release's items or plan a new one.
- **Open every chunk with one line:** the last version cut, how many changes are unreleased (the entries under `CHANGELOG.md` → `## Unreleased`, or else the commits since the last tag), and the release this chunk ships into. A repo that doesn't publish anything says so once.
- **Read the README's `## Goals` and `## Audience` before starting a chunk, unless your prompt quotes them.** The chunk's items set its scope. The goals choose between ways to deliver those items, and never add to them.
- **Read the plan and the decisions only as far as the chunk needs.** Where your prompt quotes `chunk-preflight`'s facts, they hold the next item's L, E + I, tagline and detail, and each item that may join it with its L, E + I, S, tagline and detail; open `todo.md` only for what they leave out. Otherwise run `grep -n '^## Details' todo.md`, then read `todo.md` with the Read tool up to the line it prints, or whole when it prints none. Then read the details of only the items the chunk takes. `AGENTS.md`, or the head of `docs/decisions.md` in a project that keeps no `AGENTS.md`, lists the project principles and the title of each decision no project principle covers. Read an entry in `docs/decisions.md` only where the project principle or the title leaves open what the chunk or the review needs.
- **Read the plan and the decisions only as far as the chunk needs.** Where your prompt quotes `chunk-preflight`'s facts, they hold the next item's L, E + I, tagline and detail, and each item that may join it with its L, E + I, S, tagline and detail; open `todo.md` only for what they leave out. Otherwise run `grep -n '^## Details' todo.md`, then read `todo.md` with the Read tool up to the line it prints, or whole when it prints none. Then read the details of only the items the chunk takes. `AGENTS.md` lists the project principles and the title of each decision no project principle covers. Read an entry in `docs/decisions.md` only where the project principle or the title leaves open what the chunk or the review needs.
- **Work one chunk at a time, and put every item that can share its review into it.** Every review has a fixed cost, so a chunk takes the next plan item plus every other item for which three things hold: they name one release; they fire the same review passes (`larv-review` §3); and no item outside the chunk blocks one of them. The items need not share a purpose. An item whose risk R, in `todo.md`, is above 5 ships alone: it joins no chunk, and none joins it. The next plan item is the first that does not wait: an item waits while it is marked `question`, until the human answers, or while an item that blocks it is still in the plan. Tiny items, a few lines of change each, need not fire the same passes: bundle up to ten that name one release, and that no item outside the chunk blocks, into one chunk. Split a milestone into items in `todo.md` before starting it. Fix a finding in the chunk's own change within the chunk. File a finding about anything else in `todo.md`.
- **Ask, don't guess.** Three choices are always the human's: a goal; a release, meaning the items planned into it and when it is cut; and a change to loop-claude's routing ladder, its rungs included. Any other rule that asks the human's approval reserves that choice too. Ask about a reserved choice. For any other choice, first check whether the goals already answer it. If they nearly do, propose a change to the goals and show how it answers this case: an answer settles one question, a goal settles every question like it. Otherwise take the option you would recommend, and record it with its reason: in the item's detail, or, where it governs later work, as the next sentence says. A recommendation you can give a reason for is not a guess. Ask only where you have no recommendation. Write every answer once, where it belongs, never only in chat: a goal in `README.md`, a working rule in `AGENTS.md`, or in the `.claude/rules/` file its paths scope where the project keeps no `AGENTS.md`, a standing choice in a comment at the one place it governs, or in `docs/decisions.md` once made in more than one place. When an answer holds a goal and a choice, write the goal in the README, and record as a choice only what the goal leaves open.
- **Ask, don't guess.** Three choices are always the human's: a goal; a release, meaning the items planned into it and when it is cut; and a change to loop-claude's routing ladder, its rungs included. Any other rule that asks the human's approval reserves that choice too. Ask about a reserved choice. For any other choice, first check whether the goals already answer it. If they nearly do, propose a change to the goals and show how it answers this case: an answer settles one question, a goal settles every question like it. Otherwise take the option you would recommend, and record it with its reason: in the item's detail, or, where it governs later work, as the next sentence says. A recommendation you can give a reason for is not a guess. Ask only where you have no recommendation. Write every answer once, where it belongs, never only in chat: a goal in `README.md`, a working rule in `AGENTS.md`, a standing choice in a comment at the one place it governs, or in `docs/decisions.md` once made in more than one place. When an answer holds a goal and a choice, write the goal in the README, and record as a choice only what the goal leaves open.
- **Bench a syntax with the people who write it.** Before a new or changed spelling of something users read or write ships — markdown, HTML, a config key, a CLI flag — a panel of fresh-context readers, seated as those users from the README's `## Audience`, picks it.
- **Implement the simplest change that solves the item, and ship it with `/larv-review`.** Whenever a test can check the change, write that test first and see it fail. Take its expected values from the item, never from the code. larv-review's reviewers hold the change to the technical principles, and a finding against one names it. Read a file under `~/.claude/principles/` only when the item changes it, a finding or a record you write names it, or you decline a finding.
- **Remove a finished item's row and detail from `todo.md`, and revise any decision that cites it.** If the repo publishes something others consume — a package, an image, an API — reword the item for them and move it to `CHANGELOG.md` under `## Unreleased`. A changelog is for the public, never for the next agent. Otherwise delete it; git history is the record. `~/.claude/bin/todo-item finish <ID>` removes the row and detail, and adds the bullet you pass with `--changelog <bullet>`; it refuses while a decision cites the item.
+2 -4
View File
@@ -77,12 +77,10 @@ Move `~/.claude` to the newest release:
~/code/claude-config/bin/deploy-release
```
It refuses while a file `~/.claude` tracks has an uncommitted change, such as a hook Claude Code wrote into `settings.json`. Merge that into `~/code/claude-config` on a branch of its own, through a PR, then cut a release. A model picked with `/model` is no such change.
For `~/.claude/settings.json`, which Claude Code writes from any session, read the diff first: `git -C ~/.claude diff HEAD -- settings.json`. An empty diff means only the model or the key order changed, which `deploy-release` takes as no change. When the diff holds a credential, such as a token in `env` or `mcpServers`, move it out of the file, and commit only once it is gone: a token in `env` moves to the shell profile, and a server in `mcpServers` moves to `~/.claude.json` with `claude mcp add --scope user`.
It prints `CHANGELOG.md`'s section for each version it moved past: do what each asks, such as a new step under Install. `deploy-release --help` says what else it checks.
It refuses while a file `~/.claude` tracks has an uncommitted change, such as a hook Claude Code wrote into `settings.json`; a model picked with `/model` is no such change. Read the change first, as in `git -C ~/.claude diff HEAD -- settings.json`. When it holds a credential, such as a token in `env` or `mcpServers`, move that out of the file: a token in `env` to the shell profile, a server in `mcpServers` to `~/.claude.json` with `claude mcp add --scope user`. Then carry the change into `~/code/claude-config` on a branch of its own, merge it through a PR, and cut a release.
## Release
Once the next release holds no item in `todo.md` and its release review has landed, cut it from `~/code/claude-config`:
+10 -5
View File
@@ -1,7 +1,5 @@
# Project principles
This repo keeps no `AGENTS.md`: `~/.claude` is a checkout of it, and Claude Code would load an `AGENTS.md` there into every other project's sessions. So its project principles, and the title of each decision no project principle covers, head its decision log.
Each project principle states one rule that the decisions below apply; their entries hold the specifics and the evidence. The numbers name the principles, and don't rank them.
1. **Cut cost only where a measurement of archived chunks shows quality holds.** Chunk bundling, the round cap, which fixes rerun a pass, a reviewer of its own for each pass, the review a branch changing only Markdown no agent follows gets, the passes kept for the release review, review opt-outs, the security pass, compaction and the routing ladder all apply it.
@@ -10,12 +8,13 @@ Each project principle states one rule that the decisions below apply; their ent
4. **A rule a program can check is enforced by a program, and reviewers rule on the rest.** rule-hook and its overrides, `docker-run-memory`, the refused writes of reviewers under `agents/`, and CI with `main`'s branch protection apply it.
5. **A chunk can't vouch for itself.** What a chunk writes is approved by a review pass, by the human, or by a precedent the repo held before the chunk began. So an unanswered ask leaves the PR unmerged, the human approves the premises chunk and runs `deploy-release --cut`, and the judge holds a chunk open while an answer the human typed isn't recorded anywhere.
6. **An unattended run never waits on a prompt, and a lasting failure stops it.** Every claude loop-claude starts bypasses permissions inside its sandbox, each claude it or loop-inbox opens on a terminal has `CLAUDE_CODE_SANDBOXED` set, and interactive sessions run in auto mode with the dangerous-mode prompt skipped. A failed archive stops the loop, while a `claude` that cannot start, as during an update, is retried.
7. **One word names one thing, and one rule has one home.** In `bin/`, an outcome is an entry of a `.outcomes.md` file in loop-claude's archive, a record is a document tools or people keep entries in, and a place is what a comprehension seat names. `CLAUDE.md` holds only the rules no path can scope, and each `rules/` file loads by its `paths:`. This repo's own working rules sit in `.claude/rules/`, which loads only in this repo.
7. **One word names one thing, and one rule has one home.** In `bin/`, an outcome is an entry of a `.outcomes.md` file in loop-claude's archive, a record is a document tools or people keep entries in, and a place is what a comprehension seat names. `CLAUDE.md` holds only the rules no path can scope, and each `rules/` file loads by its `paths:`.
# Decisions no project principle covers
- bin/: the tools run on the host's Python, with no container, and so do their tests outside CI
- bin/: the tools support only the latest stable Python, and the tests, written in plain unittest, run on it in four pytest-xdist workers
- deploy: this repo keeps no `AGENTS.md`, since `~/.claude` is its checkout; its principles and decision titles head `docs/decisions.md`, its working rules sit in `.claude/rules/`, and its `.claude/settings.json` excludes every `.claude/CLAUDE.md`, so a session in the clone loads only the clone's `CLAUDE.md`
- loop-claude: one session per project, its id derived from the checkout's path relative to `~/code`, or to `/` outside it
- loop-inbox: a loop-claude lists itself while it runs, and its waiting question, as files per session under `~/.local/state/loop-claude/`, and takes the first answer, from its terminal or from loop-inbox, that names that question
- machine load: a heavy command waits for one of the machine's heavy slots, one for every 4 CPUs, which every session shares, and loop-claude starts a worker only while at least a quarter of the machine's memory is available
@@ -70,6 +69,12 @@ The cap is 5, `bin/release_plan/todo_table.py`'s `BUNDLE_MAX_R`, from the same c
2026-10-29, lilleman. Serves goal 1.3 and goal 2. `~/.claude` was this repo's development checkout, so every merge reached every session on the machine at once, its unfinished work included, while other machines took only tags. Now each machine takes only releases: what a release holds is what every session runs, and cutting one is the human's step that ships it. `deploy-release` automates README → Update. It refuses an uncommitted change to a tracked file that the new tag doesn't hold, such as a hook Claude Code wrote into `settings.json`, and a move that would drop commits no release holds; a model pick, which the clean filter drops, is no change. Valid while Claude Code reads its configuration from one directory per machine.
## deploy: this repo keeps no `AGENTS.md`, since `~/.claude` is its checkout; its principles and decision titles head `docs/decisions.md`, its working rules sit in `.claude/rules/`, and its `.claude/settings.json` excludes every `.claude/CLAUDE.md`, so a session in the clone loads only the clone's `CLAUDE.md`
2026-11-11, lilleman through item 314. Serves goal 1.4. Claude Code loaded `~/.claude/AGENTS.md` into every other project's sessions, and each 1,000 tokens every thread starts with cost $68.60 of the $9,154 spent from 2026-09-30 to 2026-11-11: the 14.4 KB file cost about 2.1% of spend, and put this repo's rules in front of other projects. Other projects keep the name `AGENTS.md`, so the setup stays portable to other harnesses. `.claude/rules/` is a project folder, which loads only in a session working in this repo. In the clone, the release's `~/.claude/CLAUDE.md` and the clone's both loaded, about 0.9% of spend; the exclude drops the release's, so a chunk runs under the rules it is changing. A session working in `~/.claude` itself loads no `CLAUDE.md`, since there both are the same file; none works there, as the entry above says.
Valid while `~/.claude` is a checkout of this repo.
## deploy: `deploy-release --cut` cuts a release through a pull request merged fast-forward only, then tags the merged commit and moves `~/.claude` to it; the human runs it
2026-11-08, lilleman, todo item 271. Serves goal 1.5 and goal 2. Nobody can push to `main` on the remote, and it merges only by fast-forward, so a cut that commits on `main` and pushes no longer works. The tool cuts only the release `chunk-preflight` would call ready to cut, through `bin/release_plan/release_standing.py`, so the two never disagree on which release is next or whether it is ready; the version is that release's, never the tool's choice. It builds the cut commit in a worktree of its own, so the clone's checkout stays as it is. It finds the merged commit on `main` by its title, `Cut release <version>`, since GitHub's nearest merge, a rebase merge, may give the commit a new hash. It takes only a pull request from origin's own branch whose head is the commit that branch points at, and has the forge merge it only while its head is still that commit, so a fork's pull request from a branch of the same name, or a push after the checks passed, is never merged and deployed. Each step first checks whether an earlier run did it, so a rerun continues. Releases stay the human's, so the human runs it, and the forge credentials stay theirs. Valid while `main` takes changes only through a pull request.
@@ -141,7 +146,7 @@ The trials: four stability rounds and five principles rounds archived on 2026-10
- A cap on tool results. Claude Code caps a Bash result at about 30k characters already. A PostToolUse hook capping it at 12k, which `updatedToolOutput` allows, left the stability reviewers 3 of 6 findings, two lost in one round. Read results, 59.9M characters in subagents' threads with 58% of their characters in results over 12k, are out of such a hook's reach. Not worth an item.
- A scout: Haiku 5.5 reading raw content, whole files or long output, and returning facts as path:line and the ranges worth reading, so the agent's own context holds the summary and the chosen lines. In session, as an agent under `agents/`, its first request writes 37k tokens, $0.005, and a read of a 38KB file costs $0.008 in 8 seconds. As a slim headless `claude -p` with its own system prompt, no setting sources and no dynamic sections, which keeps the subscription login, it starts at 6k tokens and $0.0003 in 3 to 5 seconds, its prefix cached across calls, and the same read costs $0.004 in 7 seconds; a direct API call and `claude --bare` need an API key, so they are out. Stability reviewers with the agent scout found 5 of 6 findings and read 29% fewer characters at 6% less cost and 14% more time; with the CLI scout 4 of 6, 38% fewer, 17% less and 39% more; the auditors with the agent scout 3 of 6, the 🔴 among the lost, at the same cost and twice the time. Workers offered the scout never called it.
- What a reviewer reads: the auditors read the project whole by design. Of their 🔴 and 🟡 findings since the reset, 47% named a file the chunk edited and 51% another file, so reading only the change would drop half of them (`review: the principles audit runs only in the release review, and reads the project whole with one auditor checking the goals and every principles file that applies`).
- The base context: every thread starts with `~/.claude/CLAUDE.md` (19k characters), `~/.claude/AGENTS.md` (11k, this repo's decisions, which Claude Code loaded as user instructions into every project's sessions until 2026-11-11, when they moved into this log), the project's `AGENTS.md`, and, for the worker and general-purpose subagents, the skill listing (18k) and the agent listing (12k). Subagents' first messages wrote $97 of it. Trimming the listings would save about $20, 2%, and is not worth an item: `disableBundledSkills` would also drop `/security-review`, the security pass of record.
- The base context: every thread starts with `~/.claude/CLAUDE.md` (19k characters), `~/.claude/AGENTS.md` (11k, this repo's decisions, which Claude Code loaded as user instructions into every project's sessions until 2026-11-11, when they moved out of `~/.claude`), the project's `AGENTS.md`, and, for the worker and general-purpose subagents, the skill listing (18k) and the agent listing (12k). Subagents' first messages wrote $97 of it. Trimming the listings would save about $20, 2%, and is not worth an item: `disableBundledSkills` would also drop `/security-review`, the security pass of record.
Quality. Under the 200k window, every figure's mean per chunk fell. Measured on 2026-11-04, the first 20 chunks whose worker started under it, from 2026-10-28, took a mean 4.0 review rounds past each pass's first against 5.8 for the 20 before them, from 2026-10-23 to 2026-10-26; 0.20 unfinished verdicts against 0.30; 1.15 🔴 against 1.90; 11.9 🟡 against 19.4; 0.11 misread findings against 0.25; $22 against $34; and 9.6 hours against 42. Only the median dollars went up, $15.17 against $11.72: the 20 before held four chunks of $94 to $167, and the 20 after none above $76. This decision weighs all spend, so the mean is the figure that counts. The two groups also differ in projects and routing, so how much of the fall the window caused is unknown. Before the window, 19 chunks compacted between 2026-10-10 and 2026-10-22, at the compaction points Claude Code set then, 214k to 972k tokens. Per 100 worker messages they took a median 3.6 review rounds past each pass's first against 4.0 for the 68 uncompacted chunks of 100 messages or more, and 13.9 🔴 and 🟡 findings against 16.6. 4 of 19 had an unfinished verdict against 3 of 68, each 8 to 10 hours after the compaction and followed by a finished verdict within minutes; in those four verdicts the judge quoted the worker's own passages and the human's typed answers, never a summary. One cause was possible and is closed here: Claude Code writes the summary as a user entry, which `bin/claude_code/entries.py` read as the human's words, so the judge would see a `[human]` line of thousands of words and `/talk` would count it as typed; it now reads as Claude Code's. Two summaries read whole kept the chunk's intent, files, errors and next step in 12k to 20k characters, and the worker resumed without re-reading.
@@ -540,7 +545,7 @@ Three places carry the rule. `rules/todo-items.md` states it and loads with `tod
2026-10-13, lilleman through todo item 104, implemented by claude. Serves goals 1.4 and 1.5. Like red, green, refactor: the worker makes the item work, and the review passes raise it to the bar. A loop-claude worker loaded `CLAUDE.md` and every file under `rules/`, 4,247 words, and `CLAUDE.md` told it to read `technical-principles.md` too, 2,426 words more. The reviewers read the principles anyway, and a finding against one names it.
The worker keeps what it needs to write its own work. Each project's decision titles stay in its `AGENTS.md`, since a worker that misses a decision builds the wrong behaviour. This repo's head its decision log instead, since `~/.claude` is its checkout and an `AGENTS.md` there would load into every other project's sessions. 2026-11-11, lilleman: a project principle in `AGENTS.md` now stands in for the titles of the decisions it covers. A core of "Prose" stays in `CLAUDE.md`, since no reviewer reads the worker's chat. lilleman chose both on 2026-10-13. The rules for writing Goals and Audience, `todo.md` items and decisions stay in `rules/`, since writing those is the worker's job. `bin/rule-hook` stays: it blocks what a program can check, at no context cost. "Quality", the rules for comments and READMEs, and the examples behind "Prose" moved to `skills/larv-review/standards/`, which no path loads. loop-claude's judge and the prose reviewer read them, and the stability reviewer reads "Quality". They sit beside larv-review, whose passes apply them; `principles/` stays where it is, since the principles auditor reads it per file.
The worker keeps what it needs to write its own work. Each project's decision titles stay in its `AGENTS.md`, since a worker that misses a decision builds the wrong behaviour. 2026-11-11, lilleman: a project principle in `AGENTS.md` now stands in for the titles of the decisions it covers. A core of "Prose" stays in `CLAUDE.md`, since no reviewer reads the worker's chat. lilleman chose both on 2026-10-13. The rules for writing Goals and Audience, `todo.md` items and decisions stay in `rules/`, since writing those is the worker's job. `bin/rule-hook` stays: it blocks what a program can check, at no context cost. "Quality", the rules for comments and READMEs, and the examples behind "Prose" moved to `skills/larv-review/standards/`, which no path loads. loop-claude's judge and the prose reviewer read them, and the stability reviewer reads "Quality". They sit beside larv-review, whose passes apply them; `principles/` stays where it is, since the principles auditor reads it per file.
A finding against a principle names it, so the worker reads only that one when it fixes or declines the finding. The worker still writes a failing test first wherever a test can check the change, since no reviewer can make a test come first. On 2026-10-13 lilleman widened it from code changes to anything there is to test. A chunk that changes no code, config or test, such as one that only plans, meets no principles review; the chunk that implements its items does. The prose reviewer checks that a doc sorts a list whose order carries no meaning.
+2 -2
View File
@@ -6,8 +6,8 @@ paths:
# Decisions: one home, each linked to a goal or a technical principle
**The decision log has one home: `docs/decisions.md`**. A title states the whole decision in one line: what holds at the commit it ships in, in words a reader finds in the code or the plan. `AGENTS.md`, or the head of `docs/decisions.md` in a project that keeps no `AGENTS.md`, lists the project principles and the title of each decision no project principle covers, so a session works by the decisions without loading their background. The entry holds the background: its date, who made it, the reasoning, and the premise it rests on ("valid while X") so it can expire honestly when that premise dies.
**The decision log has one home: `docs/decisions.md`**. A title states the whole decision in one line: what holds at the commit it ships in, in words a reader finds in the code or the plan. `AGENTS.md` lists the project principles and the title of each decision no project principle covers, so a session works by the decisions without loading their background. The entry holds the background: its date, who made it, the reasoning, and the premise it rests on ("valid while X") so it can expire honestly when that premise dies.
**Every decision links to a goal or a technical principle.** An entry names the goal it serves or the technical principle under `~/.claude/principles/` it applies; one that can do neither is a missing goal, and an entry already in the repo with neither behind it is a finding. Propose the goal, never a reason of your own — `~/.claude/CLAUDE.md` → "Working loop", "Ask, don't guess" says how. An entry working against a goal or a technical principle stands only while it names the `todo.md` item that ends it.
**Check each new decision against the project's principles.** A project principle, listed under `Project principles` in the project's `AGENTS.md`, or else at the head of `docs/decisions.md`, states one rule that several decisions apply. Each time a decision is recorded, ask three things. Does a project principle already cover it? Then it adds no title to the list of decisions no project principle covers, and needs no ask. Does it break one? Then either the principle or the decision is wrong. Does it apply the same rule as decisions already listed? Then that rule is a new project principle, which covers those decisions too. A break, a new project principle or a change to one is the human's: ask, with your recommendation, and leave the project principles as they are until the human answers. Once the human approves a new one, the titles it covers leave that list.
**Check each new decision against the project's principles.** A project principle, listed in the project's `AGENTS.md` under `## Project principles`, states one rule that several decisions apply. Each time a decision is recorded, ask three things. Does a project principle already cover it? Then it adds no title to `AGENTS.md`, and needs no ask. Does it break one? Then either the principle or the decision is wrong. Does it apply the same rule as decisions already listed? Then that rule is a new project principle, which covers those decisions too. A break, a new project principle or a change to one is the human's: ask, with your recommendation, and leave the project principles as they are until the human answers. Once the human approves a new one, the titles it covers leave `AGENTS.md`.
+1 -1
View File
@@ -96,7 +96,7 @@ the end. Tell everyone not to manufacture confusion.
A scoring run needs four seats — Junior A, Mid A, the maintainability senior and the inherited
architect — plus the calibration seat ("The score") when more than one project is in play. Build
the four seats' prompts from `seat-prompts.md`, filled in from the project's `AGENTS.md` → "Scoring
run", or the values `seat-prompts.md` names for a project without one. Change neither between runs, or the scores are not comparable. A project without that
run". Change neither between runs, or the scores are not comparable. A project without that
section gets it at its first scoring run or paired ruling.
## The paired ruling
+1 -1
View File
@@ -9,7 +9,7 @@ A scoring-run seat's prompt is the opening, the seat's background, the shared br
- `{files}`: every tracked file in `git -C {checkout} ls-files`, except test files and those `{forbidden}` names. Give one recursive glob per top-level source directory, say which test files it leaves out, and list each file no glob matches. Example: `bin/**/*.py` except `test_*.py`, plus launchers such as `bin/loop-claude`.
- `{preloaded}`: the files the harness pre-loads into a seat, found per `SKILL.md` → "What makes the result trustworthy".
- `{anchors}` and `{dimensions}`: the six anchor bullets and the four dimension bullets in `SKILL.md` → "The score", verbatim, without bold and each on one line.
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run", or for claude-config, which keeps no `AGENTS.md`, "claude-config's values" below.
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run".
## Opening
+4 -4
View File
@@ -99,7 +99,7 @@ Fable 5.2 (`claude-fable-5-2`) has been released. The ladder in `bin/model_routi
Read the index again (https://artificialanalysis.ai/leaderboards/models), at each effort it scores, with its cost per task. Rebuild the ladder by the decision's cut: a rung stays only where no cheaper rung scores as high. Fable 5.2 may earn more than one rung, and a cheaper effort of it may push an Opus 5.5 rung off. Where the order changes which rung a project's model starts on, or how far the L and E + I steps reach, say so in the decision. The rungs are the human's choice (`CLAUDE.md` → "Working loop", "Ask, don't guess"), so propose the new ladder with the index's figures and get the human's approval before changing code.
Then replace `claude-fable-5-1` where it is named: `ladder.py`, `bin/routing_figures/prices.py` (Fable 5.2's prices), `settings.json` → `modelSettings`, the tests in `bin/model_routing/`, `bin/loop_claude/`, `bin/judge_replay/` and `bin/rule_hook/`, `AGENTS.md`, `docs/decisions.md` and `presentation/index.html`. Keep Fable 5.1 in `prices.py` while archived chunks that ran on it are still priced from there.
Then replace `claude-fable-5-1` where it is named: `ladder.py`, `bin/routing_figures/prices.py` (Fable 5.2's prices), `settings.json` → `modelSettings`, the tests in `bin/model_routing/`, `bin/loop_claude/`, `bin/judge_replay/` and `bin/rule_hook/`, `docs/decisions.md` and `presentation/index.html`. Keep Fable 5.1 in `prices.py` while archived chunks that ran on it are still priced from there.
### 320. Bench a worker below the project's rung with the advisor tool against the same worker alone and the project's rung, and file the switch if it wins.
@@ -131,7 +131,7 @@ Build on item 303's archive layout for the judge's part.
Today a rung qualifies only if it re-finds every finding the project's rung found. The project's rung itself re-finds only 69 to 73% of its own findings on a replay (29 of 40, 27 of 39, 20 of 29), so even a reviewer exactly as good would fail, and "no pass has such a rung" says less than it seems. Qualify a rung when its share of the archived findings is no lower than the project's rung's own replay share, within a margin, over enough rounds, and it misses no 🔴 that both project-rung runs found. Then replay Opus 5.5 at low effort on the prose pass first, the pass whose misses cost least; the `effort` frontmatter key sets it per reviewer.
lilleman approved on 2026-11-11 as somewhat risky, with a close evaluation in the next release: item 321. Revise the decision `routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds…`, and `AGENTS.md`'s title for it.
lilleman approved on 2026-11-11 as somewhat risky, with a close evaluation in the next release: item 321. Revise the decision `routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds…`, and its title at the head of `docs/decisions.md`.
### 308. Skip a stability round 2 whose every must-fix finding named a surviving mutant or a failing test that the pre-review gate now kills.
@@ -229,7 +229,7 @@ Today the helpers live in two tools. `Forge` is defined in `bin/chunk_preflight/
Both comprehension seats of PR #453's paired ruling (2026-11-09) named this web as a place that cost them time, and neither saw a reason for it: a second forge would need edits in `chunk_preflight`, `deploy_release`, `chunk_pr` and `larv_review`.
Put the seven helpers in a folder named by what it holds, such as `bin/forge/`, and import them from there. Keep each helper's behaviour and its tests. `chunk_preflight/choices.py` keeps `pulls_command`, which is preflight's own. Check `AGENTS.md` and `docs/decisions.md` for a mention of where a helper lives, and update it.
Put the seven helpers in a folder named by what it holds, such as `bin/forge/`, and import them from there. Keep each helper's behaviour and its tests. `chunk_preflight/choices.py` keeps `pulls_command`, which is preflight's own. Check `docs/decisions.md` for a mention of where a helper lives, and update it.
### 288. Point the two reading-cost decisions at code health and goal 1.4's counter-signal, in place of the goal 1.1 they cite.
@@ -392,7 +392,7 @@ Today the folder is the SHA-256 of the transcript's first timestamped line, whic
### 322. Have loop-claude's judge read a project's `.claude/rules/*.md` as records, and count a write there as a recorded answer.
`CLAUDE.md` → "Working loop", "Ask, don't guess" records a working rule in the `.claude/rules/` file its paths scope where a project keeps no `AGENTS.md`, as this repo now does. The judge reads only `AGENTS.md` and `docs/decisions.md` of the project (`bin/loop_claude/project/records.py` → `RECORD_FILES`), and `ANSWER_DESTINATIONS` in `bin/loop_claude/cycle/prompts.py` lists the same. So an answer recorded as such a rule leaves the judge holding the chunk open, or asking again. Add the project's `.claude/rules/*.md` to both, and to the records `bin/loop_claude/loop/usage.py` names.
`.claude/rules/this-repo-keeps-no-agents-md.md` has this repo's sessions record a working rule in a file in `.claude/rules/`, where `CLAUDE.md` → "Working loop", "Ask, don't guess" names `AGENTS.md`. The judge reads only `AGENTS.md` and `docs/decisions.md` of the project (`bin/loop_claude/project/records.py` → `RECORD_FILES`), and `ANSWER_DESTINATIONS` in `bin/loop_claude/cycle/prompts.py` lists the same. So an answer recorded as such a rule leaves the judge holding the chunk open, or asking again. Add the project's `.claude/rules/*.md` to both, to the records `bin/loop_claude/loop/usage.py` names, and to "judge's records" in `CLAUDE.md` → "Vocabulary".
### 187. Count the tokens a worker spends reading reviewers' reports, so goal 1.4's signal shows whether report files lowered what a review costs the worker's context.