Answer architecture round 4: anchor the record's headings to a UTC stamp, import the archive's names from their owner, keep docs/reading-cost.md out of the panel's reading, and order the decision's reasons

This commit is contained in:
2026-10-08 18:02:03 +02:00
parent 946daf5e3f
commit 8c9bf090f1
9 changed files with 38 additions and 25 deletions
+1 -1
View File
@@ -5,7 +5,7 @@
- larv-review runs the new `bin/reading-cost`, which measures each chunk `loop-claude` finished from its archive: the new input tokens the worker and its subagents took in before the worker's first commit, and the files git tracks that they read most. It appends each figure the project's `docs/reading-cost.md` lacks. Once 10 chunks are measured, and the last 5 chunks' median is above the 5 before them, larv-review files one plan item naming the files read most, or refreshes it. The chunk that ships it runs `reading-cost --reset`, so the trend counts only chunks that finished after it.
- larv-review appends every scoring run to the project's `docs/comprehension-history.md`: the commit the seats read, each seat's five scores and the seats' mean, and the places each seat named. The ratchet compares a run with the history's last run on the default branch. The `comprehension floor:` entry in `docs/decisions.md` keeps only the decisions to ship under the floor; a project's next scoring run moves the last scores the entry still holds into the history. The seats do not read the history.
- The README's goals put quality with the fewest interrupts first, as goal 1; sharing the setup, backups and leaking nothing follow as goals 2–4.
- larv-review reruns the comprehension scoring run before merge when any commit since the last run changes more than `docs/decisions.md`, `docs/comprehension-history.md`, `todo.md` or `CHANGELOG.md`, so the recorded scores are the merged commit's. A score under the floor, or below the default branch's recorded scores, blocks the merge and is never filed in `todo.md`. A fix lifts either; your decision to ship under the floor lifts only a score under the floor. A rerun counts toward the three-round cap only when it reports a blocking score that no decision of yours covers.
- larv-review reruns the comprehension scoring run before merge when any commit since the last run changes more than `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`, so the recorded scores are the merged commit's. A score under the floor, or below the default branch's recorded scores, blocks the merge and is never filed in `todo.md`. A fix lifts either; your decision to ship under the floor lifts only a score under the floor. A rerun counts toward the three-round cap only when it reports a blocking score that no decision of yours covers.
- larv-review skips the comprehension scoring run for a chunk whose plan items a project decision names. The decision also names the later item whose scoring run ends the suspension. Neither the comprehension floor nor the ratchet can block a skipped chunk, its summary cites the decision, and a chunk shipping its release's last item is always measured.
- README → Install and Update give the commands that check the latest tag out into `~/.claude`: on a new machine, whether or not Claude Code has already created the directory, and on one already on an older tag.
- A session commits changes Claude Code made to `settings.json` only once their diff holds no credential, and asks you to move any credential out of the file first.
+2 -1
View File
@@ -15,7 +15,8 @@ from loop_claude.claude_code.session import session_files
FINISHED = "Verdict: finished"
UNFINISHED = "Verdict: unfinished"
STAMP = "%Y-%m-%dT%H:%M:%SZ"
FINISHED_AT = re.compile(rf"^(\d{{4}}-\d\d-\d\dT\d\d:\d\d:\d\dZ) {re.escape(FINISHED)}$", re.MULTILINE)
STAMPED = r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ"
FINISHED_AT = re.compile(rf"^({STAMPED}) {re.escape(FINISHED)}$", re.MULTILINE)
@dataclasses.dataclass(frozen=True)
+1 -1
View File
@@ -2,7 +2,7 @@ import pathlib
import typing
from failure import Failure
from loop_claude.transcript_archive import finished_chunks, FinishedChunk, utc
from loop_claude.transcript_archive import FinishedChunk
from loop_claude.paths import paths_from
from loop_claude.project.checkout import project_key
+12 -9
View File
@@ -3,20 +3,21 @@ import re
import statistics
from failure import Failure
from reading_cost.archive import FinishedChunk
from loop_claude.transcript_archive import FinishedChunk, STAMPED
from reading_cost.measure import Reading
from markdown_sections import sections
HEADING = "Reading cost, chunk "
FIGURE = re.compile(rf"^Reading cost, chunk (\w+), finished ({STAMPED})$")
PLACE = re.compile(r"^- `(.+)` (\d+)$")
PLACES_KEPT = 10
RESET = "Reading cost reset, "
RESET_AT = re.compile(rf"^{RESET}({STAMPED})$")
TOKENS = re.compile(r"^(\d+) new input tokens before the first commit\.$")
WINDOW = 5
def unheld(chunks: list[FinishedChunk], record: str) -> list[FinishedChunk]:
held = {heading.removeprefix(HEADING).split(",")[0] for heading in sections(record) if heading.startswith(HEADING)}
held = {match.group(1) for heading in sections(record) if (match := FIGURE.match(heading))}
return [chunk for chunk in chunks if chunk_id(chunk) not in held]
@@ -26,7 +27,7 @@ def chunk_id(chunk: FinishedChunk) -> str:
def figure(chunk: FinishedChunk, reading: Reading) -> str:
places = "".join(f"- `{path}` {times}\n" for path, times in ranked(reading.places))
return f"## {HEADING}{chunk_id(chunk)}, finished {chunk.finished}\n\n{reading.tokens} new input tokens before the first commit.\n" + (f"\nRead most (opens and searches):\n\n{places}" if places else "")
return f"## Reading cost, chunk {chunk_id(chunk)}, finished {chunk.finished}\n\n{reading.tokens} new input tokens before the first commit.\n" + (f"\nRead most (opens and searches):\n\n{places}" if places else "")
# The figures after the last reset, in the order their chunks finished: each machine appends its own archive's chunks.
@@ -34,15 +35,17 @@ def readings_of(record: str) -> list[Reading] | Failure:
dated = []
resets = [""]
for heading, bodies in sections(record).items():
if heading.startswith(RESET):
resets.append(heading.removeprefix(RESET))
elif heading.startswith(HEADING):
if reset := RESET_AT.match(heading):
resets.append(reset.group(1))
elif named := FIGURE.match(heading):
lines = bodies[0].splitlines() if bodies else []
tokens = [int(match.group(1)) for line in lines if (match := TOKENS.match(line))]
if len(tokens) != 1:
return Failure(f"the figure under \"## {heading}\" holds no single tokens line; restore it from git history")
dated.append((heading.rsplit(", finished ", 1)[-1], Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := PLACE.match(line))}))))
return [reading for finished, reading in sorted(dated, key=lambda item: item[0]) if finished > max(resets)]
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := PLACE.match(line))}))))
elif heading.startswith("Reading cost"):
return Failure(f"\"## {heading}\" names no chunk and UTC time, or no reset time; restore it from git history")
return [reading for finished, _, reading in sorted(dated, key=lambda item: item[:2]) if finished > max(resets)]
def ranked(places: collections.Counter[str]) -> list[tuple[str, int]]:
+13 -7
View File
@@ -22,6 +22,7 @@ lists the files the last 5 chunks read most. With fewer than 10 figures, it says
Exit 1, saying why, on unknown arguments, or when git cannot start or finds no checkout, the
archive or the record cannot be read or written, or a figure in the record cannot be read back.
"""
import dataclasses
import os
import pathlib
import subprocess
@@ -31,9 +32,16 @@ import time
from reading_cost.measure import Checkout, measure, Reading
from failure import Failure
from reading_cost.figures import figure, merged, readings_of, RESET, trend, unheld
from reading_cost.archive import finished_chunks, project_archive, read_threads, utc
from loop_claude.transcript_archive import finished_chunks, utc
from reading_cost.archive import project_archive, read_threads
from file_writes import replace_file
@dataclasses.dataclass(frozen=True)
class Updated:
appended: int
readings: list[Reading]
def main(argv: list[str]) -> int:
reset = argv[:1] == ["--reset"]
given = argv[1:] if reset else argv
@@ -52,14 +60,12 @@ def main(argv: list[str]) -> int:
updated = update(record, archive, checkout, str(home), utc(time.time()) if reset else None)
if isinstance(updated, Failure):
return failed(updated)
appended, readings = updated
print(f"Appended {appended} reading-cost figure{'' if appended == 1 else 's'} from {archive}{' and a reset' if reset else ''} to {record}.")
print(trend(readings))
print(f"Appended {updated.appended} reading-cost figure{'' if updated.appended == 1 else 's'} from {archive}{' and a reset' if reset else ''} to {record}.")
print(trend(updated.readings))
return 0
def update(record: pathlib.Path, archive: pathlib.Path, checkout: Checkout, home: str, reset_at: str | None) -> tuple[int, list[Reading]] | Failure:
"""How many figures it appended, and the readings the trend counts."""
def update(record: pathlib.Path, archive: pathlib.Path, checkout: Checkout, home: str, reset_at: str | None) -> Updated | Failure:
text = read_record(record)
if isinstance(text, Failure):
return text
@@ -73,7 +79,7 @@ def update(record: pathlib.Path, archive: pathlib.Path, checkout: Checkout, home
return readings
if marks and (unwritten := write_record(record, text)):
return unwritten
return len(figures), readings
return Updated(len(figures), readings)
def new_figures(record: str, archive: pathlib.Path, checkout: Checkout, home: str) -> list[str] | Failure:
+6 -3
View File
@@ -79,10 +79,13 @@ class Figures(unittest.TestCase):
def test_a_figure_reads_back_as_the_reading_it_was_written_from_and_a_damaged_one_fails(self):
reading = Reading(7, collections.Counter({"b.py": 1, "a.py": 3}))
written = figures.figure(FinishedChunk("abcdef0123456789", "t", pathlib.Path(), []), reading)
self.assertEqual(written, "## Reading cost, chunk abcdef012345, finished t\n\n7 new input tokens before the first commit.\n\nRead most (opens and searches):\n\n- `a.py` 3\n- `b.py` 1\n")
written = figures.figure(FinishedChunk("abcdef0123456789", "2026-10-08T00:00:00Z", pathlib.Path(), []), reading)
self.assertEqual(written, "## Reading cost, chunk abcdef012345, finished 2026-10-08T00:00:00Z\n\n7 new input tokens before the first commit.\n\nRead most (opens and searches):\n\n- `a.py` 3\n- `b.py` 1\n")
self.assertEqual(figures.readings_of(written), [reading])
self.assertEqual(figures.readings_of(written.replace("7 new", "seven new")), Failure('the figure under "## Reading cost, chunk abcdef012345, finished t" holds no single tokens line; restore it from git history'))
self.assertEqual(figures.readings_of(written.replace("7 new", "seven new")), Failure('the figure under "## Reading cost, chunk abcdef012345, finished 2026-10-08T00:00:00Z" holds no single tokens line; restore it from git history'))
for damaged in (written.replace(", finished", ""), "## Reading cost reset, x\n"):
with self.subTest(damaged=damaged):
self.assertIsInstance(figures.readings_of(damaged), Failure)
def test_counts_only_figures_finished_after_the_last_reset_in_the_order_they_finished(self):
+1 -1
View File
@@ -112,7 +112,7 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
## reading cost: larv-review measures each finished chunk from loop-claude's archive and appends the figure to `docs/reading-cost.md` in the next chunk's PR
2026-10-08, lilleman. Serves goal 1.1: comprehension measured from real work, at no model cost (goal 1.4). loop-claude archives from the main checkout, outside any PR, so a tracked file it wrote there would dirty that checkout. So `bin/reading-cost` measures each finished chunk `docs/reading-cost.md` lacks from the archive, in the next chunk's worktree. The figures keep a file of their own, so `docs/comprehension-history.md` keeps one writer, the scoring run. A PR that never merges then loses no figure, and loop-claude needs no change. Serves goal 4 by naming only files git tracks, and token counts. Rejected: loop-claude opening a PR of its own, one more PR per chunk and a tie to the forge; the worker measuring its own transcript before merging, which misses the chunk's tail. The figure counts new input tokens, uncached and cache-written, because workers read mostly through Bash and subagents, where parsed paths alone miss what they read. Valid while loop-claude archives every finished chunk with its subagents' transcripts.
2026-10-08, lilleman. Serves goal 1.1: comprehension measured from real work, at no model cost (goal 1.4). loop-claude archives from the main checkout, outside any PR, so a tracked file it wrote there would dirty that checkout. So `bin/reading-cost` measures each finished chunk `docs/reading-cost.md` lacks from the archive, in the next chunk's worktree. A PR that never merges then loses no figure, and loop-claude needs no change. The figures keep a file of their own, so `docs/comprehension-history.md` keeps one writer, the scoring run. Serves goal 4 by naming only files git tracks, and token counts. Rejected: loop-claude opening a PR of its own, one more PR per chunk and a tie to the forge; the worker measuring its own transcript before merging, which misses the chunk's tail. The figure counts new input tokens, uncached and cache-written, because workers read mostly through Bash and subagents, where parsed paths alone miss what they read. Valid while loop-claude archives every finished chunk with its subagents' transcripts.
## reading cost: a rise is the last 5 chunks' median above the 5 before them, and files or refreshes one plan item
+1 -1
View File
@@ -27,7 +27,7 @@ You are one seat of a comprehension panel. Tools: Read, Grep, Glob, and Bash onl
```
Target: the project checked out at {checkout} (branch `{branch}`). Read ONLY there — never {main}, which is a different version.
Depth 1, the whole project: read README.md, AGENTS.md and CLAUDE.md (the charter), then every non-test tracked file: {files}. Do NOT open todo.md, docs/decisions.md or docs/comprehension-history.md (the plan, the decision log and past panels' scores), nor any review output. You may open a test file when the code does not answer a question; say when you did.
Depth 1, the whole project: read README.md, AGENTS.md and CLAUDE.md (the charter), then every non-test tracked file: {files}. Do NOT open todo.md, docs/decisions.md, docs/comprehension-history.md or docs/reading-cost.md (the plan, the decision log, past panels' scores and past chunks' reading cost), nor any review output. You may open a test file when the code does not answer a question; say when you did.
Your harness pre-loads these files into your context before your first tool call, from a DIFFERENT version (main): {preloaded}. Treat the worktree's own copies as the ones under review. A pre-loaded AGENTS.md may list decision titles; do not use them as evidence.
+1 -1
View File
@@ -115,7 +115,7 @@ Raised by the principles audit of PR #224. The `settings.json` decision is valid
### 83. Group `bin/` by tool, so each tool's modules and tests sit together.
Raised by the architect on PR #225 and by the inherited architect of PR #224's scoring run: `bin/` is one flat folder of tools, the modules they share and the repo's checks, with `loop_claude`'s tests beside them instead of beside the package.
Raised by the architect on PR #225 and by the inherited architect of PR #224's scoring run: `bin/` is one flat folder of tools, the modules they share and the repo's checks, with `loop_claude`'s tests beside them instead of beside the package. `loop_claude/claude_code/` parses Claude Code's own formats and `bin/reading_cost/measure.py` imports it too, so it is a package of its own.
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.