Answer architecture round 1: stamp runs in UTC, define a moved run's shape, correct the changelog, file three findings outside the chunk

This commit is contained in:
2026-10-08 14:43:18 +02:00
parent c14707da54
commit a58a0a8262
6 changed files with 25 additions and 23 deletions
+2 -2
View File
@@ -4,7 +4,7 @@
- larv-review appends every scoring run to the project's `docs/comprehension-history.md`: the commit the seats read, each seat's five scores and their mean, and the places each seat named. The ratchet compares a run with the history's last run on the default branch. The `comprehension floor:` entry in `docs/decisions.md` keeps only the decisions to ship under the floor; a project's next scoring run moves the last scores the entry still holds into the history. The seats do not read the history.
- The README's goals put quality with the fewest interrupts first, as goal 1; sharing the setup, backups and leaking nothing follow as goals 2–4.
- larv-review reruns the comprehension scoring run before merge when any commit since the last run changes more than `docs/decisions.md`, `todo.md` or `CHANGELOG.md`, so the recorded scores are the merged commit's. A score under the floor, or below the default branch's recorded scores, blocks the merge and is never filed in `todo.md`. A fix lifts either; your decision to ship under the floor lifts only a score under the floor. A rerun counts toward the three-round cap only when it reports a blocking score that no decision of yours covers.
- larv-review reruns the comprehension scoring run before merge when any commit since the last run changes more than `docs/decisions.md`, `docs/comprehension-history.md`, `todo.md` or `CHANGELOG.md`, so the recorded scores are the merged commit's. A score under the floor, or below the default branch's recorded scores, blocks the merge and is never filed in `todo.md`. A fix lifts either; your decision to ship under the floor lifts only a score under the floor. A rerun counts toward the three-round cap only when it reports a blocking score that no decision of yours covers.
- larv-review skips the comprehension scoring run for a chunk whose plan items a project decision names. The decision also names the later item whose scoring run ends the suspension. Neither the comprehension floor nor the ratchet can block a skipped chunk, its summary cites the decision, and a chunk shipping its release's last item is always measured.
- README → Install and Update give the commands that check the latest tag out into `~/.claude`: on a new machine, whether or not Claude Code has already created the directory, and on one already on an older tag.
- A session commits changes Claude Code made to `settings.json` only once their diff holds no credential, and asks you to move any credential out of the file first.
@@ -12,7 +12,7 @@
- larv-review's principles audit reads only the files under `~/.claude/principles/` that apply to the project, launching a file's auditor when unsure. A file's `Applies to` line says what it covers; one without the line covers every project. The summary names each file left out. After its first round, the architecture pass reruns the scoring run and only the reviewers whose 🔴 or 🟡 the fixes answer, unless the fixes carry an architectural decision.
- When `loop-claude`'s judge rules a chunk unfinished, the resumed worker is shown the passage the judge quoted as the work remaining. A loop started before this change exits 1 at its next reload; rerun it.
- larv-review's architecture pass and full security scan get short names, `Architecture` and `Full scan`. Each round's commit is titled with its pass's name in lower case: `Answer full scan round 1: …`.
- larv-review's comprehension floor now says, without another file open, what the scoring run measures. A project's `docs/decisions.md` keeps the last scoring run's scores, and a later run scoring below them blocks the merge until fixed. After three rounds larv-review asks you; an unattended run leaves the PR unmerged with the question. A PR under the floor that scores no lower than the last run merges once you decide to ship. A decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours. A project without those scores gets them at its next scoring run.
- larv-review's comprehension floor now says, without another file open, what the scoring run measures. A run scoring below the last one blocks the merge until fixed. After three rounds larv-review asks you; an unattended run leaves the PR unmerged with the question. A PR under the floor that scores no lower than the last run merges once you decide to ship. A decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours. A project without those scores gets them at its next scoring run.
- `loop-claude`'s judge reads your `~/.claude/CLAUDE.md` beside the project's `AGENTS.md` and `docs/decisions.md`. It no longer interrupts for a choice that file settles, and it answers the worker's questions from it.
- `loop-claude`'s judge holds a chunk open until each answer you typed in it that a later chunk could need is recorded. An answer counts as recorded once the README's goals or audience, `AGENTS.md`, `docs/decisions.md` or `~/.claude/CLAUDE.md` hold it. It also counts once the transcript shows the worker writing it into the README's goals or audience, `AGENTS.md`, `docs/decisions.md`, `todo.md` or a code comment. The worker writing to `~/.claude/CLAUDE.md` does not count, since new rules there are yours. The resumed worker is told this rule.
- `loop-claude` archives each answer the judge quotes, with the question it answers and its UTC time, in `<sha256>.outcomes.md`.
+1 -1
View File
@@ -1,6 +1,6 @@
# Claude Code configuration
My personal `~/.claude`, version controlled: `CLAUDE.md` and `AGENTS.md` for instructions, the words under `CLAUDE.md` → "Vocabulary"; `agents/` and `skills/`; `principles/` for what the reviewer agents hold projects to; `docs/` for decisions; `settings.json` for the harness and `.claude/settings.json` for the harness inside this repo; `CHANGELOG.md` and `todo.md`; the tools in `bin/`; and `workflow-bench/`, the scripts and prompts of the 2026-09-30 bench of larv-review.
My personal `~/.claude`, version controlled: `CLAUDE.md` and `AGENTS.md` for instructions, the words under `CLAUDE.md` → "Vocabulary"; `agents/` and `skills/`; `principles/` for what the reviewer agents hold projects to; `docs/` for decisions and the comprehension history; `settings.json` for the harness and `.claude/settings.json` for the harness inside this repo; `CHANGELOG.md` and `todo.md`; the tools in `bin/`; and `workflow-bench/`, the scripts and prompts of the 2026-09-30 bench of larv-review.
## Goals
+1 -14
View File
@@ -1,20 +1,7 @@
# Comprehension history
## 2026-10-04, PR #215 at 5baedb5
Moved from `docs/decisions.md`, which kept only the mean and not the commit the seats read; 5baedb5 merged PR #215.
## 2026-10-04T18:28:07Z, PR #215 merged at 5baedb5
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mean | 7.38 | 6.63 | 7.0 | 6.88 | 6.88 |
Places named, all seats together:
- `bin/loop_claude/cycle/judge_reply.py`, `cycle/worker.py`, `cycle/run.py`: the judge's contract, spread over three files.
- `bin/loop_claude/loop/main.py`, `loop/reload.py` `LoopRun`, `cycle/run.py`, `worker_args`: `remaining`, which is `Unfinished.evidence` under another name.
- `bin/loop_claude/cycle/states.py` `transition` and `cycle/run.py` `question_for`: the `quoted_answers` cap, held by case order and restated.
- `bin/loop_claude/cycle/run.py`: the four outcome texts built inline for `archive`.
- `skills/larv-review/SKILL.md`: needs `CLAUDE.md` → "Vocabulary" open beside it.
- `bin/loop_claude/loop/main.py` docstring: the archive layout, defined there and not at `cycle/archive.py`.
- `bin/loop_claude/cycle/judge_reply.py` `JUDGE_RULES`: its precedence, restated by `verdict_from`'s branch order and the `Reply` field descriptions.
- `bin/loop-claude`: the `sys.path` insert that the `from failure import` lines rely on, unstated at the import sites.
+1 -1
View File
@@ -4,7 +4,7 @@ Each seat's prompt is the four parts below, in order: the opening, the seat's ba
- `{checkout}` and `{branch}`: the worktree the seats read, and its branch.
- `{main}`: the project's main checkout.
- `{files}`: every tracked non-test file but `todo.md`, `docs/decisions.md` and `docs/comprehension-history.md`, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
- `{files}`: every tracked non-test file the shared brief does not forbid, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
- `{preloaded}`: the files the harness pre-loads into a seat, found per `SKILL.md` → "What makes the result trustworthy".
- `{anchors}` and `{dimensions}`: the six anchor bullets and the four dimension bullets in `SKILL.md` → "The score", verbatim, without bold and each on one line.
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run".
+3 -3
View File
@@ -87,18 +87,18 @@ Then the comprehension floor, which the scoring run measures. Four simulated rea
The project's `docs/decisions.md` keeps one entry whose title opens `comprehension floor:`. It holds each decision to ship under the floor, with its scores.
The project's `docs/comprehension-history.md` keeps every scoring run, appended and never edited. A run is a `## <date>, <PR or branch> at <commit>` heading naming the commit the seats read, a table of each seat's five scores and their mean, and under each seat the places it ranked hardest, or, for the architect, would restructure first, each as `path:line` and the unit's name.
The project's `docs/comprehension-history.md` keeps every scoring run, appended and never edited. A run is a `## <UTC time, ISO 8601>, <PR or branch> at <commit>` heading naming when the seats finished and the commit they read, a table of each seat's five scores and their mean, and under each seat the places it ranked hardest, or, for the architect, would restructure first, each as `path:line` and the unit's name.
The chunk does not merge while either holds:
- **Floor.** Any dimension or the overall is below 7.0.
- **Ratchet.** Any score is below the mean of the history's last run as it stands on the default branch, even within the panel's noise. Until the history holds a run, a last scoring run that the `comprehension floor:` entry still holds stands in. An equal score passes, and so does every run while the default branch has neither.
- **Ratchet.** Any score is below the mean of the history's last run as it stands on the default branch, even within the panel's noise. Until the history holds a run, the entry's line opening `The last scoring run` stands in. An equal score passes, and so does every run while the default branch has neither.
Each score under the floor or the ratchet is a 🔴 of the scoring run. It is never filed, and stands until a later scoring run clears it or, for a score under the floor, a decision to ship answers it.
Fix the places the seats name under the lowest dimension and under each dimension that fell. When only the floor holds, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A fall never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Put each run's five scores in the summary, and append the run to the history on this branch. A last scoring run that the `comprehension floor:` entry still holds moves into the history as its first run, under that run's own date and PR. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
Put each run's five scores in the summary, and append the run to the history on this branch. That line, where the entry still holds it, moves into the history ahead of every run there: `## <merge time>, <PR> merged at <merge commit>`, since the entry never named the commit the seats read, and a table with the mean alone. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
**Fresh.** Where the chunk ran a scoring run, the architecture pass is done only once its last scoring run measured the commit to merge. That run may predate commits that change only `docs/decisions.md`, `docs/comprehension-history.md`, `todo.md` or `CHANGELOG.md`. Any other commit after it, a prose fix included, because the seats read the docs, calls for a rescore: the scoring run again, once every other pass is done, the auto-reviewer included. A rescore reporting a 🔴 that no decision to ship answers is an architecture round. Any other rescore is not a round: commit only its run, appended to the history, titled `Rescore: …`. A `Rescore:` commit re-enters no pass: push it and wait for the forge's checks (§4 step 3), not for an auto-review.
+17 -2
View File
@@ -6,7 +6,7 @@
`Bar = 9`
`Next ID = 80`
`Next ID = 83`
| Goal | W |
|---|---|
@@ -37,6 +37,7 @@
| 49 | 0.1.0 | principle | **Check that `AGENTS.md` lists exactly the titles of `docs/decisions.md`.** | 1 | 2 | 5 | 6 | 1.3 | 20.1 |
| 59 | 0.1.0 | | **Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.** | 3 | 3 | 6 | 6 | 1.4 | 19.8 |
| 12 | 0.1.0 | | **Run the test suite in CI on every PR, or record in `docs/decisions.md` why the local run is enough.** | 2 | 4 | 4 | 6 | 1.1 | 16.9 |
| 80 | 0.1.0 | | **State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.** | 1 | 1 | 5 | 5 | 4 | 16.8 |
| 5 | 0.1.0 | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 5 | 5 | 5 | 6 | 1.1 | 15.6 |
| 67 | 0.1.0 | principle | **Point stability-reviewer at the technical principles in place of its own restatement of them.** | 2 | 3 | 4 | 5 | 1.3 | 14.9 |
| 66 | 0.1.0 | principle | **Hold systems-architect's method thresholds only while the comprehension panel finds the flagged code hard.** | 3 | 3 | 4 | 5 | 1.1 | 14.2 |
@@ -44,9 +45,11 @@
| 29 | 0.1.0 | | **Make the contributor a standing member of every project's audience, unless its `## Audience` says maintainability does not matter.** | 2 | 2 | 3 | 5 | 1.1, 1.4 | 13.4 |
| 68 | 0.1.0 | principle | **Test `workflow-bench/compare.py`'s blinding and `bench.py`'s run plan.** | 2 | 4 | 3 | 5 | 1.1 | 12.9 |
| 65 | 0.1.0 | | **Archive each judge reply the loop rejects.** | 3 | 4 | 4 | 5 | 3 | 12.5 |
| 82 | 0.1.0 | principle | **Sort `AGENTS.md`'s decision titles and `docs/decisions.md`'s entries by title.** | 1 | 2 | 3 | 4 | 1.3 | 12.3 |
| 6 | 0.1.0 | decision | **Move what `loop-claude` writes out of `~/.claude.json`, and ask the human about each write that must stay.** | 5 | 5 | 4 | 5 | 1.1 | 11.7 |
| 51 | 0.1.0 | principle | **Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.** | 3 | 5 | 3 | 5 | 1.1 | 11.7 |
| 50 | 0.1.0 | principle | **Grant the bench's runs only the tools a review needs, in place of skipping every permission.** | 2 | 2 | 3 | 4 | 4 | 9.9 |
| 81 | 0.1.0 | principle | **Have `workflow-bench/bench.py` return each run's failure as a value, so one failed clone does not hide the later runs' results.** | 2 | 2 | 3 | 4 | 4 | 9.9 |
| 4 | 0.1.0 | | **Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.** | 3 | 2 | 4 | 3 | 3 | 9.6 |
| 52 | 0.1.0 | principle | **Record why this checkout lives in Claude Code's own directory, or move it out and link it in.** | 6 | 6 | 3 | 5 | 2 | 7.5 |
@@ -54,7 +57,7 @@
### 79. Measure each chunk's reading cost from its transcript as loop-claude archives it, append it to `docs/comprehension-history.md`, and file a plan item naming the most-read places when the trend rises.
The human wants comprehension measured from real work. The archived transcript already records what the worker opened and searched before its first edit, and how much it read per line it changed; a script derives both at archive time with no model call. One chunk's figure is noise, since tasks differ; the trend over many chunks is the signal. The files read most often before an edit name where to act.
The human wants comprehension measured from real work. The archived transcript already records what the worker opened and searched before its first edit, and how much it read per line it changed; a script derives both at archive time with no model call. One chunk's figure is noise, since tasks differ; the trend over many chunks is the signal. The files read most often before an edit name where to act. Decide who commits the figure, and in which PR: loop-claude archives from the main checkout, outside any PR, and a tracked file it writes there dirties that checkout. Give its entries a heading the ratchet's "last run" skips.
### 72. Rerun after a fix only the pass whose finding it answers, over the fix's diff alone; close a pass on a round of only 🟢, and give prose one round.
@@ -110,6 +113,10 @@ Raised by the product-owner review of PR #211. The resumed worker gets the judge
The repo has no CI, so larv-review's gate is the local run, and larv-review's Gate skips even that when the last stability round left the suite green: an unattended chunk can merge on the worker's own word.
### 80. State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.
Raised by the principles audit of PR #224. The `settings.json` decision is valid while every machine taking this checkout is a sandbox, but Install asks for a sandbox only for `loop-claude`, so a new machine is never told.
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
The key names the project's session, archive folder and models entry. `project_key` drops the leading `/`, so `~/code/a` and `/a` share all three, and one loop can resume or delete the other's session.
@@ -138,6 +145,10 @@ State that in `CLAUDE.md`'s goals-and-audience rule, and have larv-review's comp
Goal 3, raised by the principles audit of PR #215: the judge runs with `--no-session-persistence`, and the first two rejected replies reach only stderr, so the archive cannot answer the 3am question of why the loop rejected a verdict.
### 82. Sort `AGENTS.md`'s decision titles and `docs/decisions.md`'s entries by title.
"Deterministic order", in `principles/technical-principles.md`, raised by the principles audit of PR #224: their order carries no meaning, so two chunks adding a decision conflict at the end, and a reader cannot find a title by scanning.
### 6. Move what `loop-claude` writes out of `~/.claude.json`, and ask the human about each write that must stay.
Claude Code owns that file and writes it. `trust` reads it, changes it and writes it back, so a write that lands in between is lost.
@@ -150,6 +161,10 @@ Claude Code owns that file and writes it. `trust` reads it, changes it and write
"Always grant explicit access to data", in `principles/technical-principles.md`: `workflow-bench/bench.py` starts claude with `--dangerously-skip-permissions`, and no rule confines it to a sandbox, so a run on the host reaches the host's credentials and other checkouts.
### 81. Have `workflow-bench/bench.py` return each run's failure as a value, so one failed clone does not hide the later runs' results.
"Errors as values", in `principles/technical-principles.md`, raised by the principles audit of PR #224: `bench.py` raises `KeyError` and `CalledProcessError`, and `pool.map` re-raises the first while the other threads run on unreported.
### 4. Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.
Today the folder is the SHA-256 of the transcript's first timestamped line, which tells the human nothing; that line carries the start time. Drop the fallback to the first line, which every chunk of a project shares, and give the tests' fake transcripts ISO timestamps.