Answer prose round 1: 0 🔴, 6 🟡 (0 misread); rewrite each <verb> nothing to the negation rule, tighten the new docstrings and comments
This commit is contained in:
@@ -110,7 +110,7 @@ Install puts only `loop-claude` and `loop-inbox` on `PATH`; call the others by t
|
||||
- `mutate-changes [--base <ref>] [--prefix <shell text>] [--jobs <n>]` mutation-tests the lines a branch changes and prints each mutant its tests don't kill; `mutate-changes --help` says which tool it runs for each language.
|
||||
- `rank_todo.py [--check] [todo.md]` scores and ranks a project's `todo.md`; `rank_todo.py --help` states the format.
|
||||
- `reading-cost [--reset] [record]` appends a figure for each chunk `loop-claude` finished to a project's `docs/reading-cost.md`, and says whether the figures rise; `reading-cost --help` states the measure.
|
||||
- `review-replay <pass> <model>` replays a review pass's archived rounds from the projects run on the model, on the code each reviewed, and prints the lowest rung below the project's that, like each rung above it, qualifies: it finds no less of the 🔴 and 🟡 findings than two replays on the project's rung find, within a margin, and misses no 🔴 that both found. The rungs tried start with the project's model at a lower effort. `review-replay --help` states the replay. With `--omit-claude-md` it tests instead whether the pass's reviewer may start without `CLAUDE.md`: it replays each round three times as archived and once without the instruction files, and prints whether the reviewer passes: whether the last replay missed no more findings than the most an unchanged replay missed, with each count and each finding it missed. Each replayed reviewer skips every permission prompt, stability-reviewer has Bash, and an archived prompt may name paths on the host.
|
||||
- `review-replay <pass> <model>` replays a review pass's archived rounds from the projects run on the model, on the code each reviewed, and prints the lowest rung below the project's that, like each rung above it, qualifies: it finds no less of the 🔴 and 🟡 findings than two replays on the project's rung find, within a margin, and misses no 🔴 that both found. `review-replay --help` states the replay. With `--omit-claude-md` it tests instead whether the pass's reviewer may start without `CLAUDE.md`: it replays each round three times as archived and once without the instruction files, and prints whether the reviewer passes: whether the last replay missed no more findings than the most an unchanged replay missed, with each count and each finding it missed. Each replayed reviewer skips every permission prompt, stability-reviewer has Bash, and an archived prompt may name paths on the host.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Agent, Bash, Edit, MultiEdit, Read, Skill and Write call, and `rule-hook --help` states the rules and the override.
|
||||
- `run-heavy <command>` runs a heavy command once it holds one of the machine's heavy slots, one per 4 CPUs, which every session shares; `run-heavy --help` states the slots, and the memory cap it gives a container that sets no limit.
|
||||
- `run-tests [<pytest args>...]` runs this repo's test suite, as Test says.
|
||||
@@ -134,7 +134,7 @@ Install puts only `loop-claude` and `loop-inbox` on `PATH`; call the others by t
|
||||
- `loop_state/` holds the files loop-claude and loop-inbox share: the question files (`waiting/`), the files a running loop lists itself in (`running_loops.py`, `session_lock.py`), their paths, and `hand_over.py`, which opens claude for `/talk`. A change to any of them changes both tools.
|
||||
- `machine_load/` takes the machine's heavy slots, its shared lock files, for `run_heavy`, and reads the memory share that loop-claude waits for. It also gives `mutate_changes` a slot's CPU count.
|
||||
- `mattermost_dm/` sends the human a Mattermost DM, for `mattermost-dm` and for loop-claude through `loop_effects`.
|
||||
- `model_routing/` holds the routing ladder, which loop-claude, `rule_hook`, both replay tools and `subagent-rung` read, and the list prices, which `transcript_archive` and `routing_figures` read.
|
||||
- `model_routing/` holds the routing ladder, which loop-claude, `rule_hook`, both replay tools and `subagent-rung` read, and the list prices, which `routing_figures` and `transcript_archive` read.
|
||||
- `rank_todo.py`, `settings_filter.py` and `statusline.py` are tools of one file each, tested beside them.
|
||||
- `records/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, stamps times in UTC, and replaces a file in one step, private where it is new.
|
||||
- `release_plan/` reads `todo.md`'s items table and release versions, says which release is next and whether it is ready to cut (`release_standing.py`), and ranks the plan (`ranking.py`, with its score formula in `formula.py` and its blocker order in `blockers.py`). `plan_file.py` reads, ranks, checks and writes a `todo.md`, for `larv_review`, `rank_todo.py`, `rule_hook` and `todo_item`.
|
||||
@@ -149,7 +149,7 @@ Install puts only `loop-claude` and `loop-inbox` on `PATH`; call the others by t
|
||||
|
||||
Test on the host with `bin/run-tests`, which runs pytest through `uvx`, pinned in `bin/run_tests/main.py`. It prints each failing test with its error and pytest's summary line, and saves the whole output to a file whose path it prints. It passes on any test paths or pytest options it is given, such as `bin/run-tests bin/chunk_pr -x`. It needs `uv` on `PATH`, which fetches pytest, pytest-xdist and mypy. pytest-xdist runs the tests in one worker per CPU of a heavy slot (`docs/decisions.md` → `bin/: the tools support only the latest stable Python, …`). `python3 -m unittest discover -s bin -p 'test_*.py'` runs the same tests one at a time. Test the bench's scripts with `python3 -m unittest discover -s workflow-bench -p 'test_*.py'`.
|
||||
|
||||
CI runs `ci.sh` on every pull request, and `main` merges nothing until it passes. The script runs the same tests on the committed HEAD in a pinned Python container; run it under `run-heavy` to reproduce a CI failure, with Docker on `PATH`.
|
||||
CI runs `ci.sh` on every pull request, and `main` doesn't merge anything until it passes. The script runs the same tests on the committed HEAD in a pinned Python container; run it under `run-heavy` to reproduce a CI failure, with Docker on `PATH`.
|
||||
|
||||
To mutation-test the lines a branch changes, run `~/.claude/bin/run-heavy ~/.claude/bin/mutate-changes --prefix 'uvx --python "$(command -v python3)" --with pytest==9.2.0 --from "mutmut>=3,<4"'` in its checkout.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ pull request only while its head is the commit checked out, so push before you r
|
||||
open pushes the branch, setting its upstream, and opens a draft pull request into origin's
|
||||
default branch. <title> is one line, as ~/.claude/CLAUDE.md → "Git, commits, and PRs"
|
||||
says, and <body> is the pull request's summary. Where the branch's pull request is open
|
||||
already, it opens no second one. On origin's default branch it refuses, pushing nothing.
|
||||
already, it opens no second one. On origin's default branch it refuses without pushing anything.
|
||||
sync fetches origin and merges its default branch into the branch checked out, where the
|
||||
branch does not hold it already, and prints which it did. Then, where the worktree's root
|
||||
holds a todo.md, it runs `rank_todo.py --check` on it and prints the result, failing where
|
||||
@@ -35,7 +35,7 @@ clean acts once the forge reports the branch's pull request merged at the com
|
||||
by any merge method. It fetches origin, fast-forwards the main checkout to origin's
|
||||
default branch, removes the chunk's worktree, then deletes the local branch; origin's
|
||||
branch stays. Where the worktree has uncommitted or untracked files, it refuses and
|
||||
touches nothing. Where the main checkout has uncommitted or untracked files, another
|
||||
doesn't touch anything. Where the main checkout has uncommitted or untracked files, another
|
||||
branch checked out, or commits origin's default branch lacks, it leaves that as it was,
|
||||
still removes the worktree and the branch, and fails saying why.
|
||||
|
||||
@@ -79,7 +79,7 @@ def main(argv: list[str]) -> int:
|
||||
|
||||
|
||||
def step(argv: list[str], run: Run) -> str | Failure:
|
||||
"""Run the step argv names: the line to print, or a Failure. A malformed argv runs nothing."""
|
||||
"""Run the step argv names: the line to print, or a Failure. A malformed argv doesn't run anything."""
|
||||
match argv:
|
||||
case ["open", title, *body] if len(body) <= 1 and not title.startswith("-"):
|
||||
return Failure("give a title of one line") if not one_line(title) else opened(title, body[0] if body else "", run)
|
||||
@@ -115,7 +115,7 @@ def checkout(run: Run) -> tuple[Forge, str, str] | Failure:
|
||||
|
||||
|
||||
def default_branch(run: Run) -> str | Failure:
|
||||
"""origin's default branch, such as main. Not git_cli's default_ref: that falls back to a local main or master and asks origin where nothing is recorded, but a pull request's base is origin's alone, and this tells the user how to record it."""
|
||||
"""origin's default branch, such as main. It differs from git_cli's default_ref, which falls back to a local main or master and asks origin where nothing is recorded. A pull request's base is origin's alone, so this fails instead, and says how to record it."""
|
||||
default = run(["git", "symbolic-ref", "--short", "refs/remotes/origin/HEAD"])
|
||||
if isinstance(default, Failure):
|
||||
return Failure(f"origin records no default branch; set it with `git remote set-head origin --auto`: {default.reason}")
|
||||
|
||||
@@ -117,7 +117,7 @@ def printed(shown: str, done: Ran | Failure) -> str:
|
||||
if isinstance(done, Failure):
|
||||
return f"`{shown}` {done.reason}"
|
||||
output = "\n".join(part.strip() for part in (done.stdout, done.stderr) if part.strip())
|
||||
return f"`{shown}` exited {done.code}, printing:\n\n{quoted(output)}" if output else f"`{shown}` exited {done.code}, printing nothing."
|
||||
return f"`{shown}` exited {done.code}, printing:\n\n{quoted(output)}" if output else f"`{shown}` exited {done.code}, printing no output."
|
||||
|
||||
|
||||
def levels(item: Item) -> str:
|
||||
|
||||
@@ -443,7 +443,7 @@ class Render(unittest.TestCase):
|
||||
said = rendered(item_facts(tools=[("rank_todo.py --check todo.md", Ran(0, "ranked\n", "")), ("reading-cost", Ran(1, "", "")), ("larv-review needs-scoring-run 3", Ran(1, "out\n", "err\n")), ("larv-review places", Failure("could not finish: timed out"))]))
|
||||
self.assertIn("`rank_todo.py --check todo.md` exited 0", said)
|
||||
self.assertIn("> ranked", said)
|
||||
self.assertIn("`reading-cost` exited 1, printing nothing", said)
|
||||
self.assertIn("`reading-cost` exited 1, printing no output", said)
|
||||
self.assertIn("> out\n> err", said)
|
||||
self.assertIn("`larv-review places` could not finish: timed out", said)
|
||||
|
||||
|
||||
@@ -231,7 +231,7 @@ def tool_result(part: object) -> ToolResult | None:
|
||||
def result_text(content: object) -> str:
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
# Claude Code may split one text across blocks, even inside a line, so the blocks join with nothing between them.
|
||||
# Claude Code may split one text across blocks, even inside a line, so the blocks join without anything between them.
|
||||
return "".join(part_text(part) or "" for part in content) if isinstance(content, list) else ""
|
||||
|
||||
|
||||
@@ -265,7 +265,7 @@ def rejected_until(event: dict) -> int | None:
|
||||
return resets_at if isinstance(resets_at, int) and not isinstance(resets_at, bool) else None
|
||||
|
||||
|
||||
# A count Claude Code left out or wrote as another type adds nothing, so one odd entry leaves the rest of a figure standing.
|
||||
# A count Claude Code left out or wrote as another type doesn't add anything, so one odd entry leaves the rest of a figure standing.
|
||||
def usage_count(usage: dict, field: str) -> int:
|
||||
"""One token count of a message's usage: {"input_tokens": 2, "cache_creation_input_tokens": 14541, "output_tokens": 8, ...}"""
|
||||
value = usage.get(field)
|
||||
|
||||
@@ -52,7 +52,7 @@ def segments(command: str) -> list[list[str]]:
|
||||
- a file descriptor before a redirect, as in `2>&1`, stays a word;
|
||||
- a heredoc's body is dropped;
|
||||
- variables, aliases and functions are not expanded;
|
||||
- a line with an unbalanced quote yields nothing."""
|
||||
- a line with an unbalanced quote doesn't yield anything."""
|
||||
text = uncommented(HEREDOC.sub(lambda heredoc: heredoc.group(0).split("\n", 1)[0], command))
|
||||
tokens = lexed(text)
|
||||
if tokens is None:
|
||||
|
||||
@@ -34,7 +34,7 @@ def deploy(checkout: str) -> str | Failure:
|
||||
if isinstance(git("merge-base", "--is-ancestor", "HEAD", tag, cwd=checkout), Failure):
|
||||
return Failure(f"{tag} doesn't hold the checkout's commit, so the move would drop work no release holds: cut a release that holds it first")
|
||||
settings_changed = isinstance(git("diff", "--quiet", "HEAD", tag, "--", "settings.json", cwd=checkout), Failure)
|
||||
# The tag holds each uncommitted change, so undoing them loses nothing, and lets the checkout below take the tag's copy.
|
||||
# The tag holds each uncommitted change, so undoing them doesn't lose anything, and lets the checkout below take the tag's copy.
|
||||
# A settings.json the clean filter shows unchanged may still differ on disk, by a model pick or Claude Code's key order, which checkout refuses to overwrite; the pick is asked for again below.
|
||||
tracked = git("ls-files", "--", "settings.json", cwd=checkout)
|
||||
if isinstance(tracked, Failure):
|
||||
|
||||
@@ -17,7 +17,7 @@ the move would drop work no release holds. Untracked files, such as Claude Code'
|
||||
memory, stay as they are.
|
||||
|
||||
--cut runs in the development clone, and refuses where that clone's origin isn't <checkout>'s. It cuts the next release, as chunk-preflight works it out, and
|
||||
refuses, changing nothing, unless that release holds no item in todo.md, its release review has
|
||||
refuses without changing anything, unless that release holds no item in todo.md, its release review has
|
||||
landed and changes are unreleased. The release names the version; the tool doesn't choose one.
|
||||
1. On a fresh origin/main, it renames ## Unreleased in CHANGELOG.md to the version, commits that
|
||||
titled `Cut release <version>`, and pushes it as the branch release-<version>. The clone's own
|
||||
|
||||
@@ -76,7 +76,7 @@ class Guarded(unittest.TestCase):
|
||||
|
||||
|
||||
class StalledOrigin(unittest.TestCase):
|
||||
"""A checkout whose origin, run through git's ext transport, answers nothing for 10 s, and which records no default branch for it."""
|
||||
"""A checkout whose origin, run through git's ext transport, doesn't answer for 10 s, and which records no default branch for it."""
|
||||
|
||||
def setUp(self):
|
||||
temp = tempfile.TemporaryDirectory()
|
||||
|
||||
@@ -43,7 +43,7 @@ docs/decisions.md opts out of the comprehension check is never due one.
|
||||
scoring-run-owed prints `No scoring run owed.` or exits 1 naming the runs the chunk owes and has
|
||||
not recorded. chunk-pr merge runs it and refuses to merge on exit 1, so a due line read earlier in
|
||||
the chunk can't stand in for it. It applies needs-scoring-run's rule to the checkout's todo.md and the committed history, against the
|
||||
merge base with the default branch, so a default branch that has moved on adds nothing: the items
|
||||
merge base with the default branch, so a default branch that has moved on doesn't add anything: the items
|
||||
the merge base's todo.md holds and the checkout's does not count as shipped. A chunk that empties a
|
||||
release owes two runs, any other due chunk one. The chunk has recorded the scoring runs that its
|
||||
committed docs/comprehension-history.md holds beyond the merge base's. A project that opts out or
|
||||
@@ -78,7 +78,7 @@ better, same or worse. Each place opens with the names from the seat's line that
|
||||
`, ` and followed by `: `. A dimension's vote counts as same unless one of the seat's places opens
|
||||
with that dimension and holds a `path:line` in backticks; the overall vote counts as same unless any
|
||||
place holds one. The printed votes say so. When the two counted overall votes differ, it appends
|
||||
nothing and prints `Split:`, saying to add the maintainability senior's and Junior A's lines.
|
||||
no line and prints `Split:`, saying to add the maintainability senior's and Junior A's lines.
|
||||
Otherwise the ruling is the overall vote that leads each other vote by two or more, else same. It
|
||||
prints `Ruling:` and appends the ruling.
|
||||
|
||||
@@ -169,7 +169,7 @@ from larv_review.reviewer_header import reviewer_header
|
||||
|
||||
|
||||
class Command(typing.NamedTuple):
|
||||
"""One subcommand: how the usage line spells it, whether the arguments after its name fit, and what it does with them. Where it ends in item IDs, ids_from is the number of arguments before them, so main can name the ones that aren't."""
|
||||
"""One subcommand: how the usage line spells it, whether the arguments after its name fit, and what it does with them. Where it ends in item IDs, ids_from is the number of arguments before them, so main can name the arguments that aren't IDs."""
|
||||
shape: str
|
||||
fits: typing.Callable[[list[str]], bool]
|
||||
run: typing.Callable[[list[str]], Failure | None]
|
||||
@@ -207,7 +207,7 @@ def run_reviewer_header(given: list[str]) -> Failure | None:
|
||||
return with_default_branch(lambda default: reviewer_header(default, given[0], given[1:] == ["--rerun"]), stale_ok=True)
|
||||
|
||||
|
||||
# One row per subcommand, in the order the usage line and the docstring above list them: main, USAGE and `--help` all read this table.
|
||||
# One row per subcommand: main, USAGE and `--help` all read this table.
|
||||
COMMANDS = {
|
||||
"review-folder": Command("review-folder", no_arguments, lambda given: make_review_folder()),
|
||||
"ruling-base": Command("ruling-base", no_arguments, lambda given: with_default_branch(show_ruling_base, stale_ok=False)),
|
||||
|
||||
@@ -10,7 +10,7 @@ from release_plan.todo_table import items, NoItemsTable
|
||||
def owed_scoring_run(default: str) -> Failure | None:
|
||||
"""A Failure saying which scoring runs the chunk owes and has not recorded; None where it owes none.
|
||||
|
||||
The chunk owes the runs `plan_reasons` calls, two where it empties a release, else one. It has recorded the scoring runs its committed history holds beyond those at the merge base, so a default branch that has moved on since the branch began counts for nothing. A project with no plan owes none. A missing run blocks the merge and a low score never does, which keeps the decision `review: the comprehension check records …`: it bars only scores from blocking.
|
||||
The chunk owes the runs `plan_reasons` calls, two where it empties a release, else one. It has recorded the scoring runs its committed history holds beyond those at the merge base, so a default branch that has moved on since the branch began doesn't count. A project with no plan owes none. A missing run blocks the merge and a low score never does, which keeps the decision `review: the comprehension check records …`: it bars only scores from blocking.
|
||||
"""
|
||||
checkout = top()
|
||||
if isinstance(checkout, Failure):
|
||||
|
||||
@@ -50,7 +50,7 @@ DIMENSION = re.compile(r"- \*\*([^*]+)\*\* — ")
|
||||
|
||||
|
||||
def score_bullets(skill: str) -> tuple[list[str], list[str]] | Failure:
|
||||
"""The six anchor bullets and the four dimension bullets of SKILL.md → "The score", without bold, each on one line. A bullet is one by how it opens, `- **<n> — ` or `- **<name>** — `, so other bullets and their order in the section change nothing."""
|
||||
"""The six anchor bullets and the four dimension bullets of SKILL.md → "The score", without bold, each on one line. A bullet is an anchor or a dimension by how it opens, `- **<n> — ` or `- **<name>** — `; the other bullets in the section are skipped, and the dimensions must come in the panel's order."""
|
||||
lines = [line for group in bullets(section(skill, "The score") or []) for line in group]
|
||||
anchors = [line.replace("**", "") for line in lines if ANCHOR.match(line)]
|
||||
dimensions = [(match[1], line) for line in lines if (match := DIMENSION.match(line))]
|
||||
|
||||
@@ -621,9 +621,9 @@ class LarvReviewTool(unittest.TestCase):
|
||||
history = "# Comprehension history\n\n"
|
||||
due = "Scoring run due: the chunk follows v0.0.2, cut with no scoring run since v0.0.1.\n"
|
||||
self.release("v0.0.2", "2026-10-20T12:00:00Z", {"docs/comprehension-history.md": history + paired("2026-10-21T12:00:00Z", "better") + "\n" + SCORING.replace("2026-10-08T12:00:00Z", "2026-10-09T12:00:00Z")})
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, due, "a ruling, and a run at the earlier tag's own second, end nothing")
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, due, "a ruling, and a run at the earlier tag's own second, don't end anything")
|
||||
self.release("v0.0.3", "2026-10-21T12:00:00Z", {"docs/comprehension-history.md": history + SCORING.replace("2026-10-08T12:00:00Z", "2026-10-30")})
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, due.replace("v0.0.2", "v0.0.3").replace("v0.0.1", "v0.0.2"), "a run whose time doesn't parse ends nothing")
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, due.replace("v0.0.2", "v0.0.3").replace("v0.0.1", "v0.0.2"), "a run whose time doesn't parse doesn't end anything")
|
||||
|
||||
def test_an_unmeasured_release_is_judged_by_the_default_branch_alone_and_names_a_release_over_its_candidate(self):
|
||||
self.release("v0.0.1", "2026-10-09T12:00:00Z")
|
||||
|
||||
@@ -10,7 +10,7 @@ def unmeasured_cut(default: str, checkout: str) -> str | None | Failure:
|
||||
|
||||
The reason ends once a run follows the earlier tag, so it asks only after a release was cut with none; it lets the chunk that finds the gap record the run.
|
||||
|
||||
A scoring run counts by the time its history heading records, against the tag's commit time; a heading whose time doesn't parse counts for nothing, so the run is asked for again.
|
||||
A scoring run counts by the time its history heading records, against the tag's commit time; a heading whose time doesn't parse doesn't count, so the run is asked for again.
|
||||
"""
|
||||
# Sorts v1.0.0-rc.1 below v1.0.0, as release.newest_tag does.
|
||||
tags = git("-c", "versionsort.suffix=-", "tag", "--list", "v*", "--merged", default, "--sort=-v:refname", cwd=checkout)
|
||||
|
||||
@@ -41,7 +41,7 @@ class Talked:
|
||||
|
||||
|
||||
def interrupt(question: str, answer_here: bool, session: WorkerSession, questions: pathlib.Path, effects: Effects) -> str | Talked | Stop:
|
||||
"""Prints question for the human and returns their answer: where answer_here, the first line typed at loop-claude's terminal or in loop-inbox, stripped, or Talked once they typed in the worker's claude session, which TALK opens; else Talked once they typed in that session, which opens at once on the terminal. loop-inbox lists the question meanwhile. A Stop where loop-claude is off a terminal, or they type nothing, or the input ends."""
|
||||
"""Prints question for the human and returns their answer: where answer_here, the first line typed at loop-claude's terminal or in loop-inbox, stripped, or Talked once they typed in the worker's claude session, which TALK opens; else Talked once they typed in that session, which opens at once on the terminal. loop-inbox lists the question meanwhile. A Stop where loop-claude is off a terminal, or they type no answer, or the input ends."""
|
||||
say(f"\n{question}", effects)
|
||||
on_terminal = effects.on_terminal()
|
||||
if on_terminal:
|
||||
@@ -60,7 +60,7 @@ def interrupt(question: str, answer_here: bool, session: WorkerSession, question
|
||||
|
||||
|
||||
def first_answer(ask: Ask, questions: pathlib.Path, effects: Effects) -> str | OpenOnTerminal | OpenInInbox | Stop:
|
||||
"""The first answer to ask, from loop-claude's terminal or loop-inbox: the line typed, stripped, or where claude is to open for it. A Stop where they type nothing, or the input ends."""
|
||||
"""The first answer to ask, from loop-claude's terminal or loop-inbox: the line typed, stripped, or where claude is to open for it. A Stop where they type no answer, or the input ends."""
|
||||
try:
|
||||
# Annotated since mypy infers input_or's T as object from the lambda.
|
||||
answer: str | Answer = effects.input_or(ASK_ANSWER, lambda: answered(questions, ask))
|
||||
@@ -110,4 +110,4 @@ def human_typed(transcript_path: str, held_open: typing.Callable[[], Stop | None
|
||||
entries = read_transcript(transcript_path)
|
||||
if isinstance(entries, Failure):
|
||||
return Stop(1, f"could not tell whether you typed anything: {entries.reason}")
|
||||
return None if human_turns(entries) > before else needs_human(ANSWER_QUESTION, "the interrupt ended with nothing typed")
|
||||
return None if human_turns(entries) > before else needs_human(ANSWER_QUESTION, "the interrupt ended without anything typed")
|
||||
|
||||
@@ -76,7 +76,7 @@ class Reply:
|
||||
answered_before: str = dataclasses.field(metadata={"description": "With interrupt true, a [human] line from earlier in the chunk that answered the question the worker asks now, quoted verbatim, without elisions. Empty otherwise."})
|
||||
left_open: str = dataclasses.field(metadata={"description": "With answered_before set, the words of that line that leave open what the question asks, quoted verbatim; the question ends with them. Empty otherwise."})
|
||||
question: str = dataclasses.field(metadata={"description": "With interrupt true, the single question to put to the human. With a quoted answer, the worker's question that the answer field answers. Empty otherwise."})
|
||||
# True opens claude's session at once, in place of the one-line answer (run.ask). The judge decides it, since loop-claude computes the plan's standing (release_standing.chunk_item) only when it routes a chunk on the ladder afresh: it would miss a release ready to cut in a resumed chunk or off the ladder. A wrong ruling costs the human one typed /talk, or answering in claude, which stops the loop if they leave it with nothing typed.
|
||||
# True opens claude's session at once, in place of the one-line answer (run.ask). The judge decides it, since loop-claude computes the plan's standing (release_standing.chunk_item) only when it routes a chunk on the ladder afresh: it would miss a release ready to cut in a resumed chunk or off the ladder. A wrong ruling costs the human one typed /talk, or answering in claude, which stops the loop if they leave it without typing anything.
|
||||
release_ready: bool = dataclasses.field(metadata={"description": "With interrupt true, whether the question is about a release ready to cut, which only the human may publish. False otherwise."})
|
||||
# Finds the answer that reverses a choice the worker took on its own recommendation. score-outcomes counts these answers per chunk: they are the counter-signal the 2026-11-08 entry of docs/decisions.md → `loop-claude: a judge decides each interrupt and each quoted answer…` stays valid by.
|
||||
reversed: str = dataclasses.field(metadata={"description": "The latest [human] line that overrides a choice the worker took on its own recommendation and recorded with its reason, quoted verbatim, without elisions. Empty where none does; a line that answers a question the worker left open doesn't override anything."})
|
||||
|
||||
@@ -46,7 +46,7 @@ def moment(stamp: str) -> datetime.datetime:
|
||||
return datetime.datetime.fromisoformat(stamp)
|
||||
|
||||
|
||||
# An outcome's stamp runs to the second, so a message written earlier in that second would count in both stretches. The worker writes none then, since the judge runs for seconds after its last message; a subagent or a /talk session still running may, which overcounts that message once.
|
||||
# An outcome's stamp runs to the second, so a message written earlier in that second would count in both stretches. The worker writes none then, since the judge runs for seconds after its last message. A subagent or a /talk session still running may write one, and that message is then counted twice.
|
||||
def later(timestamp: str, start: datetime.datetime) -> bool:
|
||||
"""Whether timestamp is after start; False where it is no time, or names no time zone to compare by."""
|
||||
try:
|
||||
|
||||
@@ -59,7 +59,7 @@ def raise_calls(root: pathlib.Path, new: list[Call], rung: Rung, loop: Loop) ->
|
||||
if not loop.effects.on_terminal():
|
||||
say("Not on a terminal: the calls are raised again at the next measurement.", loop.effects)
|
||||
return None
|
||||
prompt = f"Put these calls for a routing change from loop-claude's figures to the human, and change nothing until they answer:\n{bullets}The figures are in {root / archive.FIGURES}. The ladder, its steps and the delegation rule are docs/decisions.md → `routing:`, and a change to them is the human's. Record what they decide in this checkout, the development clone, as ~/.claude/CLAUDE.md says; the next chunk of this repo files the plan item for these calls."
|
||||
prompt = f"Put these calls for a routing change from loop-claude's figures to the human, and don't change anything until they answer:\n{bullets}The figures are in {root / archive.FIGURES}. The ladder, its steps and the delegation rule are docs/decisions.md → `routing:`, and a change to them is the human's. Record what they decide in this checkout, the development clone, as ~/.claude/CLAUDE.md says; the next chunk of this repo files the plan item for these calls."
|
||||
ask = Ask(loop.effects.new_id(), str(loop.session.id), loop.root, question, loop.effects.now(), OpenOnTerminal())
|
||||
with listed(loop.listed.folder, ask, loop.effects):
|
||||
unstarted = hand_over(ClaudeCommand(claude_args(rung, [prompt]), str(loop.paths.config_clone), {}), loop.effects)
|
||||
|
||||
@@ -122,7 +122,7 @@ Exit 2, when the human is needed to go on:
|
||||
- stdin or stdout is not a terminal, when a model must be picked or an interrupt answered
|
||||
- the input ends while a model is picked
|
||||
- an interrupt's answer is empty, or the input ends before one is typed
|
||||
- the claude opened for /talk, here or in loop-inbox, or for a release ready to cut, ends with
|
||||
nothing typed
|
||||
- the claude opened for /talk, here or in loop-inbox, or for a release ready to cut, ends without
|
||||
anything typed
|
||||
- the judge rules a chunk unfinished 6 cycles running; rerun to give it 6 more
|
||||
Rerunning resumes any open session."""
|
||||
|
||||
@@ -16,5 +16,5 @@ def read_premises(root: str) -> str | Failure:
|
||||
if len(bodies) > 1:
|
||||
return Failure(f"README.md has ## {name} more than once - no premises.")
|
||||
if not bodies[0]:
|
||||
return Failure(f"README.md has nothing under ## {name} - no premises.")
|
||||
return Failure(f"README.md has no text under ## {name} - no premises.")
|
||||
return f"## Goals\n{found['Goals'][0]}\n\n## Audience\n{found['Audience'][0]}"
|
||||
|
||||
@@ -342,7 +342,7 @@ class Premises(unittest.TestCase):
|
||||
b"# P\n## Goals\n1. g\xff\n## Audience\nme\n": "README.md in {root} is not UTF-8 on line 3 - no premises.",
|
||||
b"# P\n## Audience\nme\n": "README.md has no ## Goals heading - no premises.",
|
||||
b"# P\n## Goals\n1. g\n": "README.md has no ## Audience heading - no premises.",
|
||||
b"# P\n## Goals\n\n## Audience\nme\n": "README.md has nothing under ## Goals - no premises.",
|
||||
b"# P\n## Goals\n\n## Audience\nme\n": "README.md has no text under ## Goals - no premises.",
|
||||
b"# P\n## Goals\n1. g\n## Audience\nme\n## Audience\nyou\n": "README.md has ## Audience more than once - no premises.",
|
||||
}
|
||||
(checkout.root / "sub").mkdir()
|
||||
@@ -801,7 +801,7 @@ class Interrupt(unittest.TestCase):
|
||||
transcript.unlink()
|
||||
run = checkout.run([*asked, interrupt()], terminal_input="/talk\n", before_typing=ANSWERING)
|
||||
self.assertEqual((run.code, run.roles()), (2, ["worker", "judge", "interrupt"]))
|
||||
self.assertTrue(run.stderr.strip().endswith("the human is needed to answer the question above, but the interrupt ended with nothing typed"))
|
||||
self.assertTrue(run.stderr.strip().endswith("the human is needed to answer the question above, but the interrupt ended without anything typed"))
|
||||
|
||||
run = checkout.run([worker(say="A or B?"), judge(interrupt=True, evidence="A or B?", question="A or B?", vanish=True)], terminal_input="/talk\n", before_typing=ANSWERING)
|
||||
self.assertEqual((run.code, run.roles()), (1, ["worker", "judge"]))
|
||||
|
||||
@@ -862,7 +862,7 @@ class Interrupts(unittest.TestCase):
|
||||
self.assertEqual(answer, "Use B.")
|
||||
self.assertIn(f"loop-inbox cannot show this question: cannot make {self.paths.waiting}: File exists", said)
|
||||
stop, _ = self.put(FakeEffects(inputs=["/talk"], on_terminal_claude=[0]))
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended without anything typed"))
|
||||
|
||||
def test_makes_way_only_warns_where_loop_inbox_cannot_list_the_open_claude(self):
|
||||
self.paths.waiting.parent.mkdir(parents=True)
|
||||
@@ -892,7 +892,7 @@ class Interrupts(unittest.TestCase):
|
||||
send(self.paths.waiting, self.ask(), ClaudeEnded())
|
||||
fake = FakeEffects(inputs=[self.opening])
|
||||
stop, _ = self.put(fake, effects=dataclasses.replace(fake.effects(), alive=alive, sleep=ended))
|
||||
self.assertEqual((stop, looked, slept), (Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"), [INBOX], [POLL_SECONDS]))
|
||||
self.assertEqual((stop, looked, slept), (Stop(2, "the human is needed to answer the question above, but the interrupt ended without anything typed"), [INBOX], [POLL_SECONDS]))
|
||||
|
||||
def test_stops_when_loop_inbox_is_gone_with_nothing_typed(self):
|
||||
looked = []
|
||||
@@ -902,7 +902,7 @@ class Interrupts(unittest.TestCase):
|
||||
return pid == INBOX and len(looked) == 1
|
||||
fake = FakeEffects(inputs=[self.opening])
|
||||
stop, _ = self.put(fake, effects=dataclasses.replace(fake.effects(), alive=alive))
|
||||
self.assertEqual((stop, looked, fake.made("sleep")), (Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"), [INBOX, INBOX], [POLL_SECONDS]))
|
||||
self.assertEqual((stop, looked, fake.made("sleep")), (Stop(2, "the human is needed to answer the question above, but the interrupt ended without anything typed"), [INBOX, INBOX], [POLL_SECONDS]))
|
||||
|
||||
def test_lists_the_claude_open_on_its_terminal_in_loop_inbox_while_it_runs(self):
|
||||
for typed, answer_here in (([" /talk "], True), ([], False)):
|
||||
@@ -919,7 +919,7 @@ class Interrupts(unittest.TestCase):
|
||||
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[0])
|
||||
stop, _ = self.put(fake)
|
||||
self.assertEqual(fake.made("claude_on_terminal")[0][:4], ["--model", "m", "--effort", "high"])
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended without anything typed"))
|
||||
|
||||
def test_hands_the_claude_that_takes_the_terminal_the_workers_environment(self):
|
||||
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[0])
|
||||
@@ -994,7 +994,7 @@ class RoutingCheck(unittest.TestCase):
|
||||
self.assertIn(f"- {call}", (self.root / "routing-figures.md").read_text())
|
||||
((args),) = fake.made("claude_on_terminal")
|
||||
self.assertEqual(args[:6], ["--model", "m", "--effort", "high", "--permission-mode", "bypassPermissions"])
|
||||
self.assertIn(f"change nothing until they answer:\n- {call}", args[6])
|
||||
self.assertIn(f"don't change anything until they answer:\n- {call}", args[6])
|
||||
self.assertEqual(fake.terminal_environments, [{"CLAUDE_CODE_SANDBOXED": "1"}])
|
||||
self.assertEqual(fake.terminal_folders, [str(self.home / "code" / "claude-config")], "the development clone, since ~/.claude holds only a release")
|
||||
self.assertIn("Record what they decide in this checkout, the development clone", args[6])
|
||||
@@ -1051,7 +1051,7 @@ class RoutingCheck(unittest.TestCase):
|
||||
with contextlib.redirect_stderr(io.StringIO()) as said, unittest.mock.patch.object(state, "mark_raised", return_value=Failure("full")):
|
||||
self.assertIsNone(routing_check.raise_calls(self.root, new, Rung("m", "high"), self.loop(fake)))
|
||||
((args),) = fake.made("claude_on_terminal")
|
||||
self.assertIn(f"change nothing until they answer:\n- {new[0].sentence()}\n- {new[1].sentence()}\nThe figures", args[6])
|
||||
self.assertIn(f"don't change anything until they answer:\n- {new[0].sentence()}\n- {new[1].sentence()}\nThe figures", args[6])
|
||||
self.assertIn(f"call for a routing change:\n- {new[0].sentence()}\n- {new[1].sentence()}\n", said.getvalue())
|
||||
self.assertIn("routing figures: full", said.getvalue())
|
||||
[sent] = fake.dms
|
||||
|
||||
@@ -12,7 +12,7 @@ of its question as fits; the loop's terminal shows it whole. Answer as on the lo
|
||||
line is sent, and the rest is dropped; a longer answer goes through /talk.
|
||||
- /talk opens the worker's claude session here, with `claude --resume`, as /talk does on the loop's
|
||||
terminal. Once that claude ends, the loop goes on if you typed there, and the judge rules on what
|
||||
you typed; if you typed nothing, the loop stops.
|
||||
you typed; if you typed no answer, the loop stops.
|
||||
- Enter alone, or Esc, goes back to the list; unlike on the loop's terminal, it doesn't stop the loop.
|
||||
|
||||
The loop takes whichever answer comes first, here or on its terminal, so it keeps working with no
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""The files loop-claude and loop-inbox share: a running loop lists itself (running_loops.py, session_lock.py) and its question (waiting/), and both read the same paths (paths.py) and open claude for the human the same way (hand_over.py).
|
||||
|
||||
A change here changes both tools. It imports nothing from loop_claude/.
|
||||
A change here changes both tools. It doesn't import anything from loop_claude/.
|
||||
"""
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""What a message costs at list prices. The weights that split a chunk's recorded cost between its messages, and the prices a Spent block records beside what a stretch spent, so a price wrong here stays wrong in every outcome written after it."""
|
||||
"""What a message costs at list prices. They weight how a chunk's recorded cost splits between its messages, and a Spent block records them beside what a stretch spent, so a wrong price stays wrong in every outcome written after it."""
|
||||
import dataclasses
|
||||
|
||||
from claude_code.tokens import tokens_of
|
||||
|
||||
@@ -9,7 +9,7 @@ from model_routing.prices import PRICES
|
||||
|
||||
|
||||
class Prices(unittest.TestCase):
|
||||
def test_every_model_on_the_ladder_has_a_price_since_a_model_without_one_costs_nothing_in_a_figure(self):
|
||||
def test_every_model_on_the_ladder_has_a_price_since_a_model_without_one_is_left_out_of_every_figure(self):
|
||||
self.assertEqual(sorted({rung.model for rung in LADDER} - set(PRICES)), [])
|
||||
|
||||
|
||||
|
||||
+2
-2
@@ -9,7 +9,7 @@ sorts the items by release; within a release, question rows come first, then sec
|
||||
the rest, each group by score from highest, then by ID. Then it moves each blocker, an item
|
||||
another's Blocked by cell names, that ranks below an item it blocks to just above the first such
|
||||
item. Orders the details to match. Writes each item's R, and adds the Blocked by and R columns to a
|
||||
table without them. --check writes nothing and fails when the file is not ranked as it would be.
|
||||
table without them. --check doesn't write anything and fails when the file is not ranked as it would be.
|
||||
|
||||
A valid todo.md; ## Details and each item's detail are optional:
|
||||
|
||||
@@ -69,7 +69,7 @@ A valid todo.md; ## Details and each item's detail are optional:
|
||||
item starts, so no blocker may sit in a later release, and no item may block itself or form a
|
||||
loop. An ID at or above Next ID fails; one below it that the table no longer holds was finished
|
||||
or deleted, and is dropped.
|
||||
- ## Items holds the table and nothing else: an order goes in the scores or the Blocked by column.
|
||||
- ## Items holds only the table: an order goes in the scores or the Blocked by column.
|
||||
- The table holds the columns above. One missing L, E or I fails, and so does one with a column this
|
||||
version does not know, since writing it would drop that column.
|
||||
- A table in the old shape, with C, A, G and Goals columns in place of L, E, I and Moves, still
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""The formula todo.md states its score in: arithmetic over an item's scores, parsed so that it can hold nothing else, and evaluated to a float."""
|
||||
"""The formula todo.md states its score in: arithmetic over an item's scores, parsed so that it can't hold anything else, and evaluated to a float."""
|
||||
import ast
|
||||
import math
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ class Shape:
|
||||
"""A shape of the items table: its columns, and how its goals cell reads and writes."""
|
||||
# The score columns, in the order the table holds them.
|
||||
score_columns: tuple[str, ...]
|
||||
# The columns a reader does without: a missing BLOCKED_BY reads as blocked by nothing, a missing score column as that score unset, and a missing Score column as not yet ranked.
|
||||
# The columns a reader does without: a missing BLOCKED_BY reads as not blocked, a missing score column as that score unset, and a missing Score column as not yet ranked.
|
||||
optional: tuple[str, ...]
|
||||
# The columns rank_todo.py needs to rank a row, so no chunk starts on an item whose risk is unscored.
|
||||
risk_columns: tuple[str, ...]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import pathlib
|
||||
|
||||
# Each module here checks a convention of the repo's files, as a linter does: import order, citations, CI, settings. So it reads source or Markdown, which technical-principles.md bans in a test of a unit's behaviour.
|
||||
# Each module here checks a convention of the repo's own files, as a linter does, so it reads source or Markdown. technical-principles.md bans scanning source only in a test of a unit's behaviour.
|
||||
|
||||
# The checkout root.
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -24,7 +24,7 @@ then replays the rounds on each rung below, highest first: the project's model a
|
||||
effort, which the ladder leaves out, then each ladder rung down to the ladder's lowest. It stops at
|
||||
the first rung that doesn't qualify. A rung qualifies where it finds as many findings as reach the
|
||||
bar and misses no 🔴 that both replays on the project's rung found; a replay that failed on it
|
||||
finds nothing. For
|
||||
finds none. For
|
||||
each replay, claude on the project's rung matches the report to the archived findings, and counts
|
||||
a finding only where the report raises it as 🔴 or 🟡. The tool prints each rung's count, each 🔴
|
||||
it missed, and the lowest rung that, like each rung above it, qualifies. That rung is the evidence
|
||||
@@ -47,7 +47,7 @@ closes it with the header's line on the project principles.
|
||||
|
||||
Like the loop, it tries claude three times running before it counts a reply as failed. A failed
|
||||
review or match leaves its round out, except on a rung below the project's, where it finds
|
||||
nothing. Exit 1, saying why, on bad arguments, an unreadable archive or models file, no archived
|
||||
none. Exit 1, saying why, on bad arguments, an unreadable archive or models file, no archived
|
||||
round to replay, or a usage limit.
|
||||
"""
|
||||
import pathlib
|
||||
|
||||
@@ -23,7 +23,7 @@ def compared(reviewed: str, project: Rung, chosen: list[Round], found_on: typing
|
||||
if isinstance(found, Limited):
|
||||
return found
|
||||
runs.append(found)
|
||||
# A failed replay found nothing, which would lower the bar where it had CLAUDE.md and count as misses where it had none.
|
||||
# A failed replay found no findings, which would lower the bar where it had CLAUDE.md and count as misses where it had none.
|
||||
kept = rounds_without_failure(chosen, runs)
|
||||
first, second, third, without = ([found[run] for _, found in kept] for run in range(4))
|
||||
return comparison(reviewed, project, [one for one, _ in kept], (first, second, third), without, len(chosen) - len(kept))
|
||||
|
||||
@@ -64,7 +64,7 @@ def mean_share(rounds: list[Round], reference: Reference) -> Fraction:
|
||||
|
||||
|
||||
def needed(rounds: list[Round], reference: Reference) -> int:
|
||||
"""How many of rounds' findings a rung needs to find: the fewest that reach the project's rung's mean share less MARGIN. Zero or below, no rung is tried, since a rung that found nothing would qualify."""
|
||||
"""How many of rounds' findings a rung needs to find: the fewest that reach the project's rung's mean share less MARGIN. Zero or below, no rung is tried, since a rung that found none would qualify."""
|
||||
return math.ceil((mean_share(rounds, reference) - MARGIN) * held(rounds))
|
||||
|
||||
|
||||
@@ -118,7 +118,7 @@ def recalled(chosen: list[Round], project: Rung, replay_one: typing.Callable[[Ro
|
||||
on_rung: list[set[int] | Failure] | Limited = replayed_on(rounds, rung, replay_one, what)
|
||||
if isinstance(on_rung, Limited):
|
||||
return on_rung
|
||||
# A failed replay on a candidate found nothing, so it lowers that rung's count and never lifts it to what a rung needs.
|
||||
# A failed replay on a candidate found none, so it lowers that rung's count and never lifts it to what a rung needs.
|
||||
one = tried(rung, rounds, reference, [set() if isinstance(found, Failure) else found for found in on_rung])
|
||||
recalls.append(one)
|
||||
if not one.qualifies:
|
||||
|
||||
@@ -9,7 +9,7 @@ from records.utc_stamp import utc
|
||||
ARCHITECT = "Inherited architect"
|
||||
DECIDED_BY = "decided by"
|
||||
DIMENSIONS = ("Navigation", "Locality", "Shape", "Self-sufficiency", "Overall")
|
||||
# The comprehension floor, which every dimension and the overall should reach; docs/decisions.md → "comprehension floor" says it blocks nothing.
|
||||
# The comprehension floor, which every dimension and the overall should reach; docs/decisions.md → "comprehension floor" says it doesn't block anything.
|
||||
FLOOR = decimal.Decimal("7.0")
|
||||
HARDEST_FIRST = "hardest first"
|
||||
HEADING = re.compile(r"^([^,]+), (.+) at ([0-9a-f]{7,40})(?:, against ([0-9a-f]{7,40}))?$")
|
||||
|
||||
@@ -37,7 +37,7 @@ def starts_bare(agent: str, home: pathlib.Path) -> bool:
|
||||
|
||||
|
||||
def hook_reply(given: str, home: pathlib.Path) -> str:
|
||||
"""What the SubagentStart hook prints for its input given: the context for a reviewer that starts without the instruction files, nothing for any other agent."""
|
||||
"""What the SubagentStart hook prints for its input given: the context for a reviewer that starts without the instruction files, and no text for any other agent."""
|
||||
try:
|
||||
read = json.loads(given)
|
||||
except ValueError:
|
||||
|
||||
@@ -110,7 +110,7 @@ def entries(path: pathlib.Path) -> list[Entry]:
|
||||
|
||||
|
||||
def subagent(transcript: pathlib.Path) -> Subagent:
|
||||
"""The subagent, named by the .meta.json Claude Code writes beside its transcript: {"agentType": "Explore", "toolUseId": "toolu_01…", ...}. A meta missing or unreadable names nothing."""
|
||||
"""The subagent, named by the .meta.json Claude Code writes beside its transcript: {"agentType": "Explore", "toolUseId": "toolu_01…", ...}. A meta missing or unreadable doesn't name anything."""
|
||||
path = transcript.with_name(transcript.name.removesuffix(".jsonl") + ".meta.json")
|
||||
try:
|
||||
meta = json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
@@ -102,7 +102,7 @@ def answer(checked: Checked, what: str, now: float) -> int:
|
||||
if checked.blocked:
|
||||
return 2
|
||||
logs = [(breach, reason, log_path(breach.directory)) for breach, reason in checked.overridden]
|
||||
# Resolve every log before writing any, so a call that ends blocked has logged nothing.
|
||||
# Resolve every log before writing any, so a call that ends blocked hasn't logged anything.
|
||||
unloggable = [f"{breach.rule}: {log.reason}" for breach, _, log in logs if isinstance(log, Failure)]
|
||||
if not unloggable:
|
||||
for breach, reason, log in logs:
|
||||
|
||||
@@ -55,7 +55,7 @@ RULES: dict[RuleName, Rule] = {
|
||||
),
|
||||
# A reviewer only reports, so no marker waives it.
|
||||
"reviewer-report": Rule(
|
||||
f"a reviewer writes nothing but its report, with Write, into a file under ~/{REPORTS}/",
|
||||
f"a reviewer writes only its report, with Write, into a file under ~/{REPORTS}/",
|
||||
"a Write, Edit or MultiEdit call from inside an agent under agents/",
|
||||
overridable=False,
|
||||
),
|
||||
|
||||
@@ -860,9 +860,9 @@ class Hook(unittest.TestCase):
|
||||
for rule in ["action-pin", "commit-one-line", "docker-run-memory", "ignorefile-sorted", "image-pin", "no-ai-attribution", "repeat-read", "reviewer-report", "reviewer-rung", "round-cap", "round-title-counts", "run-heavy", "todo-rank-check"]:
|
||||
with self.subTest(rule):
|
||||
self.assertIn(f" {rule}\n The rule: ", helped)
|
||||
self.assertIn("reviewer-report\n The rule: a reviewer writes nothing but its report", helped)
|
||||
self.assertIn("reviewer-report\n The rule: a reviewer writes only its report", helped)
|
||||
self.assertRegex(helped, r" reviewer-report\n( .*\n){2} Takes no override\.\n")
|
||||
self.assertRegex(helped, r" image-pin\n( .*\n){2} \S", "a rule a marker waives says nothing of overrides")
|
||||
self.assertRegex(helped, r" image-pin\n( .*\n){2} \S", "a rule a marker waives doesn't say anything of overrides")
|
||||
|
||||
|
||||
class TodoRankCheck(unittest.TestCase):
|
||||
@@ -900,7 +900,7 @@ class TodoRankCheck(unittest.TestCase):
|
||||
|
||||
def test_its_own_rank_todo_failing_silently_names_the_script_and_exit(self):
|
||||
checkout = self.changed_plan(f"{self.folder}/clone", "import sys\nsys.exit(3)\n")
|
||||
self.assertEqual(self.details(checkout), [f"{checkout}/bin/rank_todo.py --check failed with exit 3 and printed nothing"])
|
||||
self.assertEqual(self.details(checkout), [f"{checkout}/bin/rank_todo.py --check failed with exit 3 and printed no output"])
|
||||
|
||||
def test_its_own_rank_todo_running_past_the_limit_is_named(self):
|
||||
checkout = self.changed_plan(f"{self.folder}/clone", "import signal\nsignal.pause()\n")
|
||||
|
||||
@@ -36,7 +36,7 @@ def check_failures(top: str, todo: pathlib.Path, own: pathlib.Path) -> list[str]
|
||||
return [f"{script} --check could not run: {failure}"]
|
||||
if ran.returncode == 0:
|
||||
return []
|
||||
return ran.stderr.splitlines() or [f"{script} --check failed with exit {ran.returncode} and printed nothing"]
|
||||
return ran.stderr.splitlines() or [f"{script} --check failed with exit {ran.returncode} and printed no output"]
|
||||
|
||||
|
||||
def same_repo(top: str, own: pathlib.Path) -> bool:
|
||||
|
||||
@@ -61,7 +61,7 @@ def main(argv: list[str], env: typing.Mapping[str, str], cpus: int | None, memor
|
||||
|
||||
|
||||
def capping(memory: Memory | None, slots: int, env: typing.Mapping[str, str]) -> dict[str, str]:
|
||||
"""What the command's environment needs for its docker to cap each container it makes at the slot's share of memory; nothing where the memory is unknown, as off Linux, or where PATH finds no docker to cap."""
|
||||
"""What the command's environment needs for its docker to cap each container it makes at the slot's share of memory; none where the memory is unknown, as off Linux, or where PATH finds no docker to cap."""
|
||||
path = env.get("PATH", "")
|
||||
if memory is None or shutil.which("docker", path=without_path_entry(path)) is None:
|
||||
return {}
|
||||
|
||||
@@ -61,7 +61,7 @@ def replayed(
|
||||
if isinstance(on_reference, Limited):
|
||||
return on_reference
|
||||
held = [(case, aim) for case, reply in zip(cases, on_reference) if (aim := reproduced(case, reply))]
|
||||
# With nothing to hold a rung to, no rung would miss anything.
|
||||
# Without anything to hold a rung to, no rung would miss anything.
|
||||
if not held:
|
||||
return Replay(on_reference, [], None)
|
||||
climbed: list[Tried[Case, Aim, Answer]] = []
|
||||
|
||||
@@ -111,7 +111,7 @@ UNMERGED = [
|
||||
"Chunks that left no merge commit:",
|
||||
"",
|
||||
"- Each is the run of commits on the default branch that share one commit time, since a rebase merge replays a pull request's commits at once. Its guesses come from the todo.md before its first commit.",
|
||||
"- Two pull requests merged in the same second read as one chunk, so a pull request that shipped nothing adds its lines, files and rounds to the chunk it shares a time with.",
|
||||
"- Two pull requests merged in the same second read as one chunk, so a pull request that shipped no item adds its lines, files and rounds to the chunk it shares a time with.",
|
||||
"- A run whose commits all kept their author time was never replayed: it was pushed or fast-forwarded as committed. It may hold only part of its pull request, so the medians and the rank correlations leave it out.",
|
||||
"- Rebase drops a round's commit that changed no file, so their rounds and findings are at least what history shows, and the medians count the rebased ones as shown.",
|
||||
]
|
||||
|
||||
@@ -48,7 +48,7 @@ SHOWN = ("L", "E", "I", "R", "S", "C")
|
||||
|
||||
|
||||
def results_of(chunk: BranchChunk, numstat: str, details: dict[int, str], tree: dict[str, int]) -> list[Result]:
|
||||
"""What one chunk measured for each item it shipped, from `git diff --numstat` of its change; none where it changed nothing but the plan and the records. details holds each item's detail in the todo.md the chunk started from, and tree each file's lines before the chunk, as file_lines reads them."""
|
||||
"""What one chunk measured for each item it shipped, from `git diff --numstat` of its change; none where it changed only the plan and the records. details holds each item's detail in the todo.md the chunk started from, and tree each file's lines before the chunk, as file_lines reads them."""
|
||||
changed = [row.split("\t", 2) for row in numstat.splitlines()]
|
||||
work = [row for row in changed if len(row) == 3 and row[2] not in NOT_WORK]
|
||||
if not work:
|
||||
|
||||
@@ -244,7 +244,7 @@ Reversals: the human's typed answers that reversed a choice the worker took on i
|
||||
def test_ends_with_how_it_reads_chunks_merged_by_rebase_only_where_it_measured_one(self):
|
||||
text = report_text([result(1, 5, 3, None, landing="rebase")], [], None, [])
|
||||
self.assertIn("\n\nChunks that left no merge commit:\n\n- Each is the run of commits", text.split("Median outcome")[1])
|
||||
self.assertIn("a pull request that shipped nothing adds its lines, files and rounds", text)
|
||||
self.assertIn("a pull request that shipped no item adds its lines, files and rounds", text)
|
||||
self.assertIn("at least what history shows, and the medians count the rebased ones as shown.", text)
|
||||
self.assertIn("Chunks that left no merge commit", report_text([result(1, 5, 3, None, landing="push")], [], None, []))
|
||||
self.assertNotIn("Chunks that left no merge commit", report_text([result(1, 5, 3, None)], [], None, []))
|
||||
|
||||
@@ -7,7 +7,7 @@ a defect, a level from 1 to 4 scored as for a todo.md item. It prints the rung t
|
||||
the model's cheapest rung, the project's rung, in the Agent tool's words, such as `model sonnet,
|
||||
effort high`. --plan, for a task that files or scores todo.md items, prints the project's rung where
|
||||
that L picks a lower one, as ~/.claude/rules/todo-items.md lets only the project's rung or above
|
||||
plan. For a model off the routing ladder, the tool prints nothing on stdout, says so on stderr and
|
||||
plan. For a model off the routing ladder, the tool doesn't print anything on stdout, says so on stderr and
|
||||
exits 0; launch the subagent with no model or effort.
|
||||
|
||||
Exit 1, saying why, on bad arguments.
|
||||
|
||||
@@ -12,20 +12,20 @@ The cells take the format rank_todo.py --help states; the letter flags take uppe
|
||||
add takes Next ID as the item's ID and raises Next ID by one. It writes the row, with the tagline
|
||||
in bold and each | in it escaped, and the detail it reads from stdin, if any, under ## Details.
|
||||
The detail may hold no ## or ### heading.
|
||||
It prints `Filed item <ID>.` It writes nothing where ranking fails or scores the item under the bar.
|
||||
It prints `Filed item <ID>.` It doesn't write anything where ranking fails or scores the item under the bar.
|
||||
|
||||
finish removes the item's row and its detail. With --changelog it adds the bullet first under
|
||||
CHANGELOG.md's ## Unreleased, or a ## heading that says unreleased, creating the section, or the
|
||||
file, where missing. It refuses where that section holds ### subsections: add the bullet by hand.
|
||||
Where CHANGELOG.md exists, finish needs --changelog or --no-changelog, so no bullet is left out by
|
||||
mistake. It prints `Finished item <ID>.` It writes nothing while docs/decisions.md cites the item.
|
||||
mistake. It prints `Finished item <ID>.` It doesn't write anything while docs/decisions.md cites the item.
|
||||
|
||||
show reads only todo.md. It prints the items table's header and divider, item <ID>'s row, and its detail
|
||||
from its ### heading to the next item's, sub-headings included. It fails, saying so, where todo.md has
|
||||
no row for <ID>.
|
||||
|
||||
Exit 2 on bad arguments, and 1, saying why, on any other failure. A message opening `not filed:`
|
||||
or `not finished:` means it wrote nothing.
|
||||
or `not finished:` means it didn't write anything.
|
||||
"""
|
||||
import argparse
|
||||
import pathlib
|
||||
@@ -127,7 +127,7 @@ def finish(args: argparse.Namespace, folder: pathlib.Path) -> str | Failure:
|
||||
def finish_texts(args: argparse.Namespace, todo: pathlib.Path, changelog: pathlib.Path) -> tuple[ranking.Ranked, str | None] | Failure:
|
||||
"""What finish writes: todo ranked without item args.id, and changelog's new text, None where finish leaves it as is."""
|
||||
if changelog.exists() and args.changelog is None and not args.no_changelog:
|
||||
return Failure("CHANGELOG.md exists beside todo.md; pass --changelog <bullet>, or --no-changelog where the repo publishes nothing")
|
||||
return Failure("CHANGELOG.md exists beside todo.md; pass --changelog <bullet>, or --no-changelog where the repo doesn't publish anything")
|
||||
texts = plan_file.read_all(todo)
|
||||
if isinstance(texts, Failure):
|
||||
return texts
|
||||
|
||||
@@ -33,7 +33,7 @@ JUDGE_COST_AT = re.compile(rf"^{re.escape(JUDGE)}(.+)\n{re.escape(JUDGE_COST)}([
|
||||
|
||||
# The claude-config release and commit that ran the cycle, on the line before JUDGE, such as `Config: v0.8.0 (<sha>)`.
|
||||
CONFIG = "Config: "
|
||||
# Opens the block of what the work since the previous outcome that recorded a spend spent; an outcome that spent nothing has none.
|
||||
# Opens the block of what the work since the previous outcome that recorded a spend spent; an outcome that didn't spend anything has none.
|
||||
SPENT = "Spent since "
|
||||
KINDS = "input, cache write 5m, cache write 1h, cache read, output"
|
||||
# Who spent, such as `worker`, `judge` or `Explore "Search the hooks"`, and the model they ran on.
|
||||
|
||||
@@ -67,7 +67,7 @@ def waited(file: str, pattern: str | None, timeout: float, clock: Clock = time)
|
||||
if text is None:
|
||||
return Failure(f"{file} did not appear in {timeout:g} s")
|
||||
last = line_in(text, None)
|
||||
wanted = f"no line matching `{pattern}`" if pattern is not None else "nothing"
|
||||
wanted = f"no line matching `{pattern}`" if pattern is not None else "no text"
|
||||
return Failure(f"{file} held {wanted} after {timeout:g} s" + (f"; its last line: {last}" if last is not None else ""))
|
||||
clock.sleep(min(POLL_SECONDS, left))
|
||||
|
||||
|
||||
@@ -90,7 +90,7 @@ class WaitFor(unittest.TestCase):
|
||||
def test_an_empty_file_gives_up_saying_it_held_nothing_and_no_last_line(self):
|
||||
self.file.write_text("")
|
||||
clock = Clock()
|
||||
self.assertEqual(main.waited(str(self.file), None, 3, clock), Failure(f"{self.file} held nothing after 3 s"))
|
||||
self.assertEqual(main.waited(str(self.file), None, 3, clock), Failure(f"{self.file} held no text after 3 s"))
|
||||
self.assertEqual(clock.now, 3)
|
||||
|
||||
def test_sleeps_only_the_time_left_before_the_timeout(self):
|
||||
|
||||
+14
-14
@@ -52,7 +52,7 @@ Settling a call is the human's. The chunk that settles the calls cites, in `docs
|
||||
|
||||
## bin/: a test parametrizes its cases with `subTest`, since pytest's `parametrize` doesn't reach a unittest `TestCase`
|
||||
|
||||
2026-11-12, lilleman, from the release review of 0.9.0. Applies `principles/technical-principles.md` → "Most code should have automated tests", whose "parametrize the cases" asks for one reported result per case. The tests are plain unittest, run by pytest, and pytest's `parametrize` doesn't work on a `TestCase` method; `subTest` is the unittest mechanism that does: each case reports and fails on its own, with its parameters named. A test body that loops over its cases for another reason, or holds a branch, is still a breach. Valid while the tests stay in plain unittest.
|
||||
2026-11-12, lilleman, from the release review of 0.9.0. Applies `principles/technical-principles.md` → "Most code should have automated tests", whose "parametrize the cases" asks for one reported result per case. The tests are plain unittest, run by pytest, and pytest's `parametrize` doesn't work on a `TestCase` method; `subTest` is the unittest mechanism that does: each case reports and fails on its own, with its parameters named. A test that loops over its cases itself, outside `subTest`, or holds a branch, is still a breach. Valid while the tests stay in plain unittest.
|
||||
|
||||
## bin/: an outcome is an entry of a `.outcomes.md` file in loop-claude's archive, a record is a document tools or people keep entries in, and a place is what a comprehension seat names
|
||||
|
||||
@@ -80,7 +80,7 @@ The cap is 5, `bin/release_plan/todo_table.py`'s `BUNDLE_MAX_R`, from the same c
|
||||
|
||||
## comprehension floor: every dimension and the overall at 7.0 or above is the target, and no score blocks a merge or release
|
||||
|
||||
2026-09-27, lilleman, restating the 2026-09-24 target and extending it to the overall. Serves goal 1. 2026-10-13, lilleman: the floor blocks nothing, and `review: the comprehension check records …` says why. Valid while the scoring run keeps its four seats.
|
||||
2026-09-27, lilleman, restating the 2026-09-24 target and extending it to the overall. Serves goal 1. 2026-10-13, lilleman: the floor doesn't block anything, and `review: the comprehension check records …` says why. Valid while the scoring run keeps its four seats.
|
||||
|
||||
## deploy: `~/.claude` holds only a release, which `deploy-release` moves it to; this repo is developed in `~/code/claude-config`, where loop-claude runs its chunks like any project's
|
||||
|
||||
@@ -116,7 +116,7 @@ Valid while `~/.claude` is a checkout of this repo.
|
||||
|
||||
2026-10-03, lilleman. Serves goal 1.2. The judge reads `~/.claude/CLAUDE.md`, each file under `~/.claude/rules/` and each file under `~/.claude/skills/larv-review/standards/` whole. The worker follows them, so a question they answer should not reach the human. Reading whole is safe, because the judge may quote only words that appear in them, and a list of sections would need keeping current.
|
||||
|
||||
2026-10-03, lilleman. Serves goal 1.2. The judge rules a chunk finished only once every answer the human typed in it is recorded. A finished chunk's session is deleted, and its answers with it, so an answer recorded nowhere would be asked again in a later chunk. Claude settled two choices lilleman's ruling left open. First, an answer counts as recorded where the files the judge reads hold it, `~/.claude/CLAUDE.md` included, or where the transcript shows the worker writing it into `README.md`'s premises, `AGENTS.md`, `docs/decisions.md`, `todo.md` or a code comment. A worker's write to `~/.claude/CLAUDE.md`, `~/.claude/rules/` or `~/.claude/skills/larv-review/standards/` does not count: the judge reads the live `~/.claude`, which holds the write only once it merges and the checkout is pulled. The judge cannot read a code comment or `todo.md`. Yet `CLAUDE.md` → "Working loop", "Ask, don't guess" puts a standing choice made in one place in a comment there. And the human's approval of a release's items, which "Working loop" requires, is the set of rows `todo.md` names for that release. The judge also reads the records once a cycle from the main checkout. That checkout may not yet hold an answer the worker wrote and merged. A judge that refused any of these would hold the chunk open every cycle. Second, only an answer a later chunk could need must be recorded, so an approval such as "merge it" holds nothing open. The worker's resume prompt states the rule, so a worker whose chunk is held open knows it.
|
||||
2026-10-03, lilleman. Serves goal 1.2. The judge rules a chunk finished only once every answer the human typed in it is recorded. A finished chunk's session is deleted, and its answers with it, so an answer recorded nowhere would be asked again in a later chunk. Claude settled two choices lilleman's ruling left open. First, an answer counts as recorded where the files the judge reads hold it, `~/.claude/CLAUDE.md` included, or where the transcript shows the worker writing it into `README.md`'s premises, `AGENTS.md`, `docs/decisions.md`, `todo.md` or a code comment. A worker's write to `~/.claude/CLAUDE.md`, `~/.claude/rules/` or `~/.claude/skills/larv-review/standards/` does not count: the judge reads the live `~/.claude`, which holds the write only once it merges and the checkout is pulled. The judge cannot read a code comment or `todo.md`. Yet `CLAUDE.md` → "Working loop", "Ask, don't guess" puts a standing choice made in one place in a comment there. And the human's approval of a release's items, which "Working loop" requires, is the set of rows `todo.md` names for that release. The judge also reads the records once a cycle from the main checkout. That checkout may not yet hold an answer the worker wrote and merged. A judge that refused any of these would hold the chunk open every cycle. Second, only an answer a later chunk could need must be recorded, so an approval such as "merge it" doesn't hold anything open. The worker's resume prompt states the rule, so a worker whose chunk is held open knows it.
|
||||
|
||||
2026-10-17, lilleman, todo item 168. The human answers an interrupt on loop-claude's terminal, and the worker resumes headless with the answer in a `-p` prompt. The judge reads that answer as a `[human]` line, not a `[loop]` one, so it holds the chunk open until the answer is recorded, as it does for an answer typed in claude.
|
||||
|
||||
@@ -244,13 +244,13 @@ Valid while no published standard sets either number, the reading-cost window st
|
||||
|
||||
## review: a fix for a 🔴 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟡, a 🟢, a ❓ or a prose finding reruns none; the stability pass doesn't rerun on a round's fixes where every 🔴 and 🟡 of that round asked only that a named mutant be killed or a named failing test pass, and the tests and the mutation command run clean on the fixes
|
||||
|
||||
2026-10-08, lilleman. Serves goal 1.4. An audit of the four costliest archived chunks found nearly every finding that mattered in a pass's first round; when every fix, a nit's included, re-entered review, later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes keep changing nothing a user would notice.
|
||||
2026-10-08, lilleman. Serves goal 1.4. An audit of the four costliest archived chunks found nearly every finding that mattered in a pass's first round; when every fix, a nit's included, re-entered review, later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes change only what a user wouldn't notice.
|
||||
|
||||
2026-10-13, lilleman through todo item 97. Serves goals 1.4 and 1.5. From round 2 a pass reviews only how the chunk answered its previous round, each fix and each dismissal with its reason, and reports only one that is plainly wrong or worse than the change it answered. A fix from another pass that re-enters it is still reviewed as a first round would review it, limited to its lines. Reviewing all that changed had a reviewer find new gaps in each round's fixes, so a pass ran to its cap. Valid while round-2 findings outside a round's answers keep changing nothing a user would notice.
|
||||
2026-10-13, lilleman through todo item 97. Serves goals 1.4 and 1.5. From round 2 a pass reviews only how the chunk answered its previous round, each fix and each dismissal with its reason, and reports only one that is plainly wrong or worse than the change it answered. A fix from another pass that re-enters it is still reviewed as a first round would review it, limited to its lines. Reviewing all that changed had a reviewer find new gaps in each round's fixes, so a pass ran to its cap. Valid while round-2 findings outside a round's answers change only what a user wouldn't notice.
|
||||
|
||||
2026-10-29, lilleman. Serves goals 1.4 and 1.5. A 🟡 fix no longer reruns its pass. One project's chunk on 2026-10-29, a README paragraph and plan rows, took 10 review rounds in 45 minutes, and three of its four second rounds answered only 🟡 findings, such as the wording of a question's options. Round 2 found 12 🔴 in about 200 runs since 2026-10-21, mostly in round 1's fixes of 🔴 findings, so a 🔴 fix still reruns its pass. A 🟡 fix that breaks something merges unreviewed by its pass; another pass whose row it meets for a 🔴 fix in the same round still reviews it. Valid while 🔴 findings in later chunks, fixing code a 🟡 fix wrote, stay rare.
|
||||
|
||||
2026-11-12, lilleman through todo item 308. Serves goal 1.4. Since the worker runs the mutation command before each stability round, a finding that asks only that a named mutant be killed or a named failing test pass has an answer a run can check. The archive from 2026-09-30 to 2026-11-12 holds 326 stability reports of round 2 or later, from 13 projects, and 81 of those reports open with BLOCK (`stability-skip-count/`). For each of the 326, the 🔴 and 🟡 lines of the report before it were matched on mutant and test-failure wording. Every such line matched in 5 of those earlier reports. Read by hand, 1 of the 5 asked only that a failing test pass, and the round 2 after each of the 5 approved. So none of the 81 BLOCKs fell in a round this skips. The skip needs every 🟡 to be of that kind too: in one chunk on 2026-11-12, round 1's only 🔴 was a red suite, and round 2 blocked on a 🟡 whose answer was wrong, which no test or mutant showed. A fix may not edit the named failing test, since a test made to pass that way shows nothing. Fixes from such a round still rerun every other pass whose row they meet. Valid while code that such a skipped round's fixes wrote is rarely what a later chunk's 🔴 findings fix.
|
||||
2026-11-12, lilleman through todo item 308. Serves goal 1.4. Since the worker runs the mutation command before each stability round, a finding that asks only that a named mutant be killed or a named failing test pass has an answer a run can check. The archive from 2026-09-30 to 2026-11-12 holds 326 stability reports of round 2 or later, from 13 projects, and 81 of those reports open with BLOCK (`stability-skip-count/`). For each of the 326, the 🔴 and 🟡 lines of the report before it were matched on mutant and test-failure wording. Every such line matched in 5 of those earlier reports. Read by hand, 1 of the 5 asked only that a failing test pass, and the round 2 after each of the 5 approved. So none of the 81 BLOCKs fell in a round this skips. The skip needs every 🟡 to be of that kind too: in one chunk on 2026-11-12, round 1's only 🔴 was a red suite, and round 2 blocked on a 🟡 whose answer was wrong, which no test or mutant showed. A fix may not edit the named failing test, since a test made to pass that way doesn't show anything. Fixes from such a round still rerun every other pass whose row they meet. Valid while code that such a skipped round's fixes wrote is rarely what a later chunk's 🔴 findings fix.
|
||||
|
||||
## review: a comment is prose in any file; a branch changing only Markdown no agent follows runs only the prose pass and, for README or docs an outside reader lands on, the product-owner pass; plan rows a chunk files from a finding, re-scores or moves get no review
|
||||
|
||||
@@ -272,11 +272,11 @@ Valid while the later fixes per finished chunk, which `score-outcomes` and the r
|
||||
|
||||
## review: a project decision may opt every chunk out of the architect, the product-owner pass or the comprehension check, and a project opting out of the comprehension check voids code health
|
||||
|
||||
2026-10-11, lilleman. Serves goal 1.4. A research repo's PR, in a repo with no consumer for the product-owner pass and no product for the architect, spent rounds of both, and of the comprehension check, on a probe. The opt-out is the project's to make, and only an entry titled `review opt-out:` makes it, so a project without one is reviewed as before (goal 1.3). No opted-out reviewer works against a goal or a technical principle: the principles audit, which has no opt-out, still judges KISS and every other principle, so the entry names no plan item that ends it. 2026-10-13, lilleman: an opt-out of the comprehension check no longer ends with a chunk that ships an item naming the floor, since the floor blocks nothing. 2026-10-18, lilleman, answering the review of PR #324: a project that opts out of the comprehension check voids code health in its README too: without the check the goal has no signal, and, never measured, it would weigh four times its weight forever. `rank_todo.py` refuses the plan of such a project until its README voids it. Valid while the principles audit judges every chunk that changes code, config or tests, and while the architect's and the product-owner's findings matter only to a product with consumers.
|
||||
2026-10-11, lilleman. Serves goal 1.4. A research repo's PR, in a repo with no consumer for the product-owner pass and no product for the architect, spent rounds of both, and of the comprehension check, on a probe. The opt-out is the project's to make, and only an entry titled `review opt-out:` makes it, so a project without one is reviewed as before (goal 1.3). No opted-out reviewer works against a goal or a technical principle: the principles audit, which has no opt-out, still judges KISS and every other principle, so the entry names no plan item that ends it. 2026-10-13, lilleman: an opt-out of the comprehension check no longer ends with a chunk that ships an item naming the floor, since the floor doesn't block anything. 2026-10-18, lilleman, answering the review of PR #324: a project that opts out of the comprehension check voids code health in its README too: without the check the goal has no signal, and, never measured, it would weigh four times its weight forever. `rank_todo.py` refuses the plan of such a project until its README voids it. Valid while the principles audit judges every chunk that changes code, config or tests, and while the architect's and the product-owner's findings matter only to a product with consumers.
|
||||
|
||||
## review: a project decision may suspend the comprehension check for the plan items it names
|
||||
|
||||
2026-10-04, lilleman. Serves goal 1.4. In a restructure, each step's scoring run measures code the next step deletes. It costs a run and, below the floor, blocks the step, yet says nothing about what the restructure buys. The scoring run after the last step measures that. 2026-10-08: the paired ruling replaced the scoring run in most chunks, and the suspension covers it too; the ruling after the last step compares with the commit measured before the first step. 2026-10-13, lilleman: the comprehension check blocks nothing now, so a suspension saves only the runs. Valid while such a decision names its items and the plan item that ends it.
|
||||
2026-10-04, lilleman. Serves goal 1.4. In a restructure, each step's scoring run measures code the next step deletes. It costs a run and, below the floor, blocks the step, yet doesn't say anything about what the restructure buys. The scoring run after the last step measures that. 2026-10-08: the paired ruling replaced the scoring run in most chunks, and the suspension covers it too; the ruling after the last step compares with the commit measured before the first step. 2026-10-13, lilleman: the comprehension check doesn't block anything now, so a suspension saves only the runs. Valid while such a decision names its items and the plan item that ends it.
|
||||
|
||||
## review: a release review fixes its findings in the release it reviews, and files under that release only a finding that needs a human answer, adds a feature or outlasts its five rounds
|
||||
|
||||
@@ -318,17 +318,17 @@ Claude Code's own subagent prompt tells a subagent to return findings rather tha
|
||||
|
||||
Before this change, reports took a median 19k tokens per chunk, 8% of the worker's peak context, and lines carrying a severity mark were 9–24% of a report (2026-10-08, over 68 archived transcripts). `review-replay` still counts each 🔴 and 🟡 by its line in the reviewer's reply, now a one-line summary. todo.md item 187 is to measure what the worker reads of the reports instead.
|
||||
|
||||
Valid while Claude Code passes `agent_type` to hooks inside a subagent. Without it, the rule refuses nothing, and a reviewer may write anywhere.
|
||||
Valid while Claude Code passes `agent_type` to hooks inside a subagent. Without it, the rule doesn't refuse anything, and a reviewer may write anywhere.
|
||||
|
||||
## review: stability and product-owner report only a defect, a broken requirement of the plan or the goals, a breach of a technical principle, or for stability a hard place the change touches; each architect finding names a later change the PR makes harder; prose runs only on a chunk that changes prose
|
||||
|
||||
2026-10-13, lilleman through todo item 97, implemented by claude. Serves goals 1.4 and 1.5. Anthropic's Claude Code best practices: "A reviewer prompted to find gaps will usually report some, even when the work is sound… Chasing every finding leads to over-engineering." 320 of this repo's 876 commits answered a review round. So stability and product-owner drop a finding that is none of these, and stability leaves the style of comments to prose, which reads every comment a chunk touches. An architect finding that names no later change it makes harder cannot be checked, so it is dropped. The architecture pass and the comprehension check keep their whole-project view, which guards goal 1.1 as a project grows. Valid while the findings these rules drop keep changing nothing a user would notice.
|
||||
2026-10-13, lilleman through todo item 97, implemented by claude. Serves goals 1.4 and 1.5. Anthropic's Claude Code best practices: "A reviewer prompted to find gaps will usually report some, even when the work is sound… Chasing every finding leads to over-engineering." 320 of this repo's 876 commits answered a review round. So stability and product-owner drop a finding that is none of these, and stability leaves the style of comments to prose, which reads every comment a chunk touches. An architect finding that names no later change it makes harder cannot be checked, so it is dropped. The architecture pass and the comprehension check keep their whole-project view, which guards goal 1.1 as a project grows. Valid while the findings these rules drop change only what a user wouldn't notice.
|
||||
|
||||
2026-10-28, lilleman: prose findings stay, since Claude's prose needs them, though prose's 105 first rounds since 2026-10-21 raised 498 🟡 and only 3 🔴.
|
||||
|
||||
## review: the comprehension check records how hard a project is to read, and no score blocks a merge: a paired ruling against the last measured commit in the release review, and the four-seat scoring run, once when an item names the floor and twice on one head when a release's items run out
|
||||
|
||||
2026-10-08, lilleman. Serves goals 1.1 and 1.4. Two independent four-seat runs moved up to 0.4 on near-identical code, so a fall within that noise forced rounds that changed nothing (PR #224). A seat reading both versions judges the difference on the same material, so the noise the versions share cancels. No conventional significance level lets a sign test settle with four votes, so the human chose a lead of two votes over every other ruling, same included. Two seats that agree settle it, so a PR that leaves the reading as it was costs two seats. A third seat alone can never settle it, so the third and fourth launch together. The scoring run also runs on a chunk shipping an item that names the floor, so the chunk that lifts a project to the floor can prove it. A chunk that removes a release's last item without shipping it runs the scoring run too, so no release ends unmeasured, unless `todo.md` marks that release renamed in a `## Renamed` section (`bin/release_plan/todo_table.py` → `renames`). Valid while seats reading both versions agree more often than two absolute runs do.
|
||||
2026-10-08, lilleman. Serves goals 1.1 and 1.4. Two independent four-seat runs moved up to 0.4 on near-identical code, so a fall within that noise forced rounds that didn't change anything (PR #224). A seat reading both versions judges the difference on the same material, so the noise the versions share cancels. No conventional significance level lets a sign test settle with four votes, so the human chose a lead of two votes over every other ruling, same included. Two seats that agree settle it, so a PR that leaves the reading as it was costs two seats. A third seat alone can never settle it, so the third and fourth launch together. The scoring run also runs on a chunk shipping an item that names the floor, so the chunk that lifts a project to the floor can prove it. A chunk that removes a release's last item without shipping it runs the scoring run too, so no release ends unmeasured, unless `todo.md` marks that release renamed in a `## Renamed` section (`bin/release_plan/todo_table.py` → `renames`). Valid while seats reading both versions agree more often than two absolute runs do.
|
||||
|
||||
2026-10-13, lilleman. No published study validates simulated reader panels as a measure of how easy code is to understand, and no test of whether the panel's signal beats its noise has run. So no ruling, score or floor blocks a merge. The runs go on, so the history keeps the scores and the places the seats found hard, for later measurement. The architect, the stability reviewer and a session filing an item read the newest places: the newest scoring run's and each later ruling's places a seat voted worse. Those are the places a chunk can improve. The check runs once per chunk, on the branch as review left it. A chunk that leaves a touched place as hard says why in its PR summary, not in a decline's record, which would outlive the place.
|
||||
|
||||
@@ -507,7 +507,7 @@ Valid while the archived verdicts show what a judge should return, and until a n
|
||||
|
||||
The rungs, least competent first: Sonnet 5.5 at high; Opus 5.5 at medium; Fable 5.2 at low and medium; Opus 5.5 at xhigh; Fable 5.2 at high and max. The 2026-11-11 note below put them so. Before it they were Sonnet 5.5 at high; Opus 5.5 at medium, high, xhigh and max; Fable 5.1 at max, after the 2026-10-29 note took Haiku 5.5 off. They first were: Haiku 5.5 at low, medium, high, xhigh and max; Sonnet 5.5 at high; Opus 5.5 at medium, high, xhigh and max; Fable 5.1 at max. The order and the cut come from the Artificial Analysis Intelligence Index (https://artificialanalysis.ai/leaderboards/models), which scores each model at each effort and gives a cost per task. A rung stays only where no cheaper rung scores as high, so a step up always buys more score (goal 1.4). The first cut, from the index read 2026-10-14, dropped Sonnet 5.5 at low, medium, xhigh and max, Opus 5.5 at low, and every Fable 5.1 rung, and every tie in the index fell among the dropped rungs. lilleman put Fable 5.1 at max back on top: the index scored it under Opus 5.5 at high, but Anthropic's model catalog names Fable the most capable model, and for the hardest tasks lilleman wants the highest quality at any price. Fable 5.2 at max tops the index outright, so that exception has ended.
|
||||
|
||||
A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. A model off the ladder routes nothing: claude runs it at the effort its settings give. A task's L steps its worker from the project's rung: 1 and 2 one down, 3 none, 4 one up. Its E + I steps its reviewers up from the project's rung: 4 or less none, 5 one up, 6 two up, 7 three up, 8 four up. A step past either end stops there. A worker errs more the more it changes and the newer that is, which L scores; review must look harder the further a defect would escape and the more it would break, which E + I scores (goals 1.1 and 1.4). Size raises L, so a large change runs its worker higher. The steps spread the values this repo's plan held when rescored on 2026-10-16, L 1 to 4 and E + I 2 to 7, so a worker on most items runs at or below the project's rung, and a review climbs only where a defect would escape far or break much. lilleman approved the steps on 2026-10-16 as a guess. `score-outcomes` reports how well L, E and I each order finished chunks by review rounds and 🔴 findings. A change to the ladder or the steps is lilleman's.
|
||||
A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. A model off the ladder doesn't route anything: claude runs it at the effort its settings give. A task's L steps its worker from the project's rung: 1 and 2 one down, 3 none, 4 one up. Its E + I steps its reviewers up from the project's rung: 4 or less none, 5 one up, 6 two up, 7 three up, 8 four up. A step past either end stops there. A worker errs more the more it changes and the newer that is, which L scores; review must look harder the further a defect would escape and the more it would break, which E + I scores (goals 1.1 and 1.4). Size raises L, so a large change runs its worker higher. The steps spread the values this repo's plan held when rescored on 2026-10-16, L 1 to 4 and E + I 2 to 7, so a worker on most items runs at or below the project's rung, and a review climbs only where a defect would escape far or break much. lilleman approved the steps on 2026-10-16 as a guess. `score-outcomes` reports how well L, E and I each order finished chunks by review rounds and 🔴 findings. A change to the ladder or the steps is lilleman's.
|
||||
|
||||
2026-10-21, lilleman, settling the routing calls stamped 2026-10-21T20:33:29Z: L 1 steps one rung down, not two as before, so an Opus 5.5 project runs no worker on Haiku 5.5. A delegated subagent takes the same steps, so an L 1 search moves off Haiku 5.5 too: lilleman chose one set of steps for chunk workers and subagents alike, so loop-claude, `subagent-rung` and rule-hook read one table (code health). No figure yet shows whether Sonnet 5.5 searches better than Haiku 5.5, so this costs more per search for a gain nobody has measured (goal 1.4). The calls came from loop-claude's routing figures, measured at cycle 165 over 46 chunks. Chunks started on Haiku 5.5 at max took a mean 7.8 review rounds past each pass's first, against 2.8 on Sonnet 5.5 at high, and 0.2 unfinished verdicts, against 0.1. The figures also called for moving workers off Sonnet 5.5 at high, whose chunks took 0.1 unfinished verdicts against 0.0 on Opus 5.5 at medium. lilleman kept L 2 on Sonnet 5.5, since that gap rests on one or two verdicts.
|
||||
|
||||
@@ -599,7 +599,7 @@ Valid while Claude Code loads `~/.claude/rules/*.md` in every project, a path-sc
|
||||
|
||||
2026-11-09, larv-review, through todo item 12. Serves goal 1.1: a merge no longer rests on the worker's word. larv-review's gate was the local run alone, which it skips when the last stability round left the suite green, so an unattended chunk could merge on the worker's own report. gitea.larvit.se's runners labelled `docker-host` run `.gitea/workflows/ci.yml`, which runs `ci.sh` on the committed HEAD in a pinned Python container; the script's header says how. A run takes about 3 minutes.
|
||||
|
||||
The protection lives in Gitea's settings, outside the repo: rule `main`, required status check `CI / tests (pull_request)`. A pull request has no status until CI queues its run, so `deploy-release --cut` waits for each check the protection requires rather than reading no status as no checks, for 30 minutes at most. CI runs no rule-hook check, so a `rule-override` in the PR body has nothing to override. Whoever adds a rule check to CI must also let it read the PR body's `rule-override`. Valid while gitea.larvit.se runs a runner labelled `docker-host`.
|
||||
The protection lives in Gitea's settings, outside the repo: rule `main`, required status check `CI / tests (pull_request)`. A pull request has no status until CI queues its run, so `deploy-release --cut` waits for each check the protection requires rather than reading no status as no checks, for 30 minutes at most. CI runs no rule-hook check, so a `rule-override` in the PR body doesn't have anything to override. Whoever adds a rule check to CI must also let it read the PR body's `rule-override`. Valid while gitea.larvit.se runs a runner labelled `docker-host`.
|
||||
|
||||
## tests: each test earns its keep as the technical principles' test bullet says, and the worker takes a test's expected values from its item, never from the code
|
||||
|
||||
@@ -633,7 +633,7 @@ Valid while the worker writes the tests that larv-review's reviewers hold to the
|
||||
|
||||
2026-10-02, lilleman, on the `principle` exemption. A 🟡 against a technical principle is exempt as `principle`, because its audience value scores low and the bar would drop every one. A 🟢 against a principle is scored like any other finding.
|
||||
|
||||
2026-10-15, lilleman: the scores follow a researched method, built by `todo.md` items 149 and 151 after PR #283 shipped the ranking, PR #296 the risk scores and PR #298 the anchors each project names for each level in `docs/score-anchors.md`; the evidence is in `docs/scoring-and-goals-research.md`. R is L + E + I − 2: the chance the change holds a defect, set first by its size; the chance the defect escapes tests, review and monitoring; and the worst impact dimension, each about ×10 per level. I5, irreversible, holds R at 7 or more, and I4 at 5 or more, since a severe enough consequence outranks how unlikely it is. R stays out of the ranking and sets how carefully an item is done; an item that lowers risk scores it as value. A task's L and E + I pick its rungs in place of R (2026-10-16, lilleman), as `routing: one ladder` records. There is no C column: what it scored beyond size, spread and novelty, which L already holds, is complexity, and that adds almost nothing once size is accounted for. There is no A column either, since the Audience says whose value the goals serve. Each README goal is `maintain`, which vetoes an item that breaks it, or `achieve`, with a signal, a counter-signal and a one-line reason, as this README's Goals mark them. Two parts are not built yet and hold once their items ship: each release fills by priority up to a capacity it states (item 149), and each project keeps 10 to 20 settled decisions to test its goals against (item 151).
|
||||
2026-10-15, lilleman: the scores follow a researched method, built by `todo.md` items 149 and 151 after PR #283 shipped the ranking, PR #296 the risk scores and PR #298 the anchors each project names for each level in `docs/score-anchors.md`; the evidence is in `docs/scoring-and-goals-research.md`. R is L + E + I − 2: the chance the change holds a defect, set first by its size; the chance the defect escapes tests, review and monitoring; and the worst impact dimension, each about ×10 per level. I5, irreversible, holds R at 7 or more, and I4 at 5 or more, since a severe enough consequence outranks how unlikely it is. R stays out of the ranking and sets how carefully an item is done; an item that lowers risk scores it as value. A task's L and E + I pick its rungs in place of R (2026-10-16, lilleman), as `routing: one ladder` records. There is no C column: what it scored beyond size, spread and novelty, which L already holds, is complexity, and that adds almost no information once size is accounted for. There is no A column either, since the Audience says whose value the goals serve. Each README goal is `maintain`, which vetoes an item that breaks it, or `achieve`, with a signal, a counter-signal and a one-line reason, as this README's Goals mark them. Two parts are not built yet and hold once their items ship: each release fills by priority up to a capacity it states (item 149), and each project keeps 10 to 20 settled decisions to test its goals against (item 151).
|
||||
|
||||
2026-10-16, PR #283: items rank by `Score = 100 * V / (2**S + F)`. The 100 lets a score read to one decimal. 2**S is the cost. V sums, over the goals an item's Moves cell names, each goal's weight times how far the item moves it, 1 to 10. A Moves column holds `1.1 by 8, 1.2 by 5`. An item names a maintain goal only as a defect restoring it, or by the confidence a test, gate leg or probe adds to it, as the defect paragraph below and the 2026-11-09 paragraph after it say.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Every study below predicts from the diff or history, none from a description, so
|
||||
6. **Live config or infrastructure as code.** About 70% of Google's outages come from changes to a live system (sre.google/sre-book/introduction). Syntax and configuration defects were 36.5–62.7% of IaC defects at four organisations (Rahman et al. 2018, arXiv:1809.07937).
|
||||
7. **New technology or AI-written code**, with mixed evidence (Pearce et al. 2022, S&P†; Perry et al. 2023, CCS†; Spracklen et al. 2025, USENIX Security).
|
||||
|
||||
Complexity metrics add almost nothing once size is accounted for (El Emam et al. 2001, TSE, doi:10.1109/32.935855†). So a separate difficulty score beside size, spread and novelty measures nothing new.
|
||||
Complexity metrics add almost no information once size is accounted for (El Emam et al. 2001, TSE, doi:10.1109/32.935855†). So a separate difficulty score beside size, spread and novelty measures nothing new.
|
||||
|
||||
## Value per cost, not a sum
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ The page lives at https://claude.ai/artifact/RJJwgRAke3viuqTYEgu1a9. To update i
|
||||
|
||||
## Who it is for
|
||||
|
||||
People in tech who want to build software factories, work with AI tools more efficiently, and stop fighting their agents. They know git, pull requests and the shell, and nothing about agent loops. Write for a tired reader on a Monday morning without coffee: one fact per sentence, each term explained before it is used, and no filler.
|
||||
People in tech who want to build software factories, work with AI tools more efficiently, and stop fighting their agents. They know git, pull requests and the shell, and don't know anything about agent loops. Write for a tired reader on a Monday morning without coffee: one fact per sentence, each term explained before it is used, and no filler.
|
||||
|
||||
## How the pages are built
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
* Prefer deep modules: a small, intuitive interface in front of a lot of hidden work.
|
||||
- A caller should know what a call does and what it returns, without knowing how it does it. `fs.writeFile("notes.txt", data)` hides opening the file, encoding, buffering, permissions and the disk itself. Yet anyone reading that line knows the outcome: the data is written, or the call fails and says why.
|
||||
- A module hides how it works, never what it depends on: what "Functional core, imperative shell" injects stays injected.
|
||||
- A shallow module costs more to learn than it saves. BANNED: a wrapper that only renames the function it calls; a module whose callers must make three calls in an order nothing but their memory enforces. GOOD: one call that does the whole job and handles the order itself.
|
||||
- A shallow module costs more to learn than it saves. BANNED: a wrapper that only renames the function it calls; a module whose callers must make three calls in an order only their memory enforces. GOOD: one call that does the whole job and handles the order itself.
|
||||
- When this pulls against "Explicit over implicit", the deep module wins, unless hiding the work makes it harder for someone new to the project to tell what the call does.
|
||||
* Make invalid states unrepresentable.
|
||||
- Example: using a time-only type in the database when we actually want to store a time of day introduces risk for errors.
|
||||
@@ -30,7 +30,7 @@
|
||||
* One owner per value; a reader can name every writer.
|
||||
- At every scale, from a variable to a table, file or volume: two components writing the same one are one component with a seam nobody declared.
|
||||
- The test: from the code in front of them, can a reader list every writer to this value, and the order those writers run in?
|
||||
- BANNED: a record passed to 24 functions, 13 of which mutate it and return nothing. GOOD: one type owns the value, the rest call it and take a result.
|
||||
- BANNED: a record passed to 24 functions, 13 of which mutate it and don't return anything. GOOD: one type owns the value, the rest call it and take a result.
|
||||
- BANNED: `fill(set)` then `adjust(set)`, where swapping the two lines changes the output. GOOD: `const filled = fill(); const adjusted = adjust(filled);`
|
||||
- BANNED: `stack` and `edges`, two arrays a comment says must stay in step. GOOD: `edges` derived from `stack`, or one array whose element type holds both.
|
||||
- Allowed, never flag: mutation that never leaves the function that made it; a loop counter; data shared and never written; a state machine with named states and one transition function.
|
||||
|
||||
@@ -123,7 +123,7 @@ The 5 release reviews hold 64 archived findings: architecture 47, principles 17.
|
||||
|
||||
## Candidate 3: override-reviewer folded into stability
|
||||
|
||||
The archive holds 38 override-reviewer runs. None has a 🔴 or 🟡, so there is nothing to recover. Nothing was replayed.
|
||||
The archive holds 38 override-reviewer runs. None has a 🔴 or 🟡, so there isn't anything to recover. Nothing was replayed.
|
||||
|
||||
**Verdict: no data.**
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ Hold all of these or the comparison doesn't prove anything.
|
||||
one. Seats that read different amounts produce numbers nothing can compare.
|
||||
- **Blind.** Deny the project's own conclusions — the plan, a decision log, prior review
|
||||
output — and never show a panelist the metric under test. An anchored reader is a wasted one.
|
||||
- **Check what the harness pre-loads, before launching.** `CLAUDE.md` may be nothing but
|
||||
- **Check what the harness pre-loads, before launching.** `CLAUDE.md` may hold only
|
||||
`@AGENTS.md`, and an `@file` import lands in a subagent's context ahead of its first tool call.
|
||||
That is harmless for a reader who was going to read the charter anyway, and fatal to the
|
||||
architects' README-and-tree-only step. Read the project's `CLAUDE.md` and resolve its imports
|
||||
@@ -121,7 +121,7 @@ Identical shape for all seven:
|
||||
2. The single unit they would least want to be asked to modify, and why.
|
||||
3. What they expected to be hard and found easy.
|
||||
4. Prose debt: which documentation they needed, what it cost to find, and — separately — which
|
||||
prose they read that told them nothing the code was not already saying. Both halves matter.
|
||||
prose they read that didn't tell them anything the code wasn't already saying. Both halves matter.
|
||||
Panels reliably find that the answers exist and are unfindable, which is a different defect
|
||||
from prose that should not exist at all, and the fixes are opposite: index the first, delete
|
||||
the second.
|
||||
|
||||
@@ -51,7 +51,7 @@ Report, in this shape:
|
||||
1. Ranked 5–8 hardest places, hardest first. Each: path:line and the unit's name, one to three sentences on what specifically defeated you (branches held at once, length, a misleading name, missing context, state mutated out of sight, a rule you had to hunt for elsewhere) and whether anything resolved it (a comment, the charter, a test) or it stayed opaque.
|
||||
2. The single unit you would least want to be asked to modify, and why.
|
||||
3. What you expected to be hard and found easy.
|
||||
4. Prose debt: documentation you needed and what it cost to find; separately, prose you read that told you nothing the code was not already saying.
|
||||
4. Prose debt: documentation you needed and what it cost to find; separately, prose you read that didn't tell you anything the code wasn't already saying.
|
||||
5. Navigation, Locality, Shape, Self-sufficiency, overall — each with one sentence naming the anchor it sits against and the concrete thing that holds it there. A score with no named cause is discarded.
|
||||
```
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ Default to NO comment. Delete one that is any of:
|
||||
|
||||
## Negation, history and limitations: name the action, or leave it out
|
||||
|
||||
Prose describing the code as it stands — a comment, a README, an `AGENTS.md`, an API spec — may narrate what is *not* done (a removed step, a call we don't make, a superseded version `we used to X, now Y`, a limitation) in one shape only: where a reader would otherwise **do** something harmful, and the sentence says what to do instead. GOOD: `wrap a direct ./tests/run.sh to put it under the same lock`. BANNED: `run.sh takes no host lock`. A limitation with nothing to do instead belongs in the PR or ticket, because **documenting a limitation is not fixing it**.
|
||||
Prose describing the code as it stands — a comment, a README, an `AGENTS.md`, an API spec — may narrate what is *not* done (a removed step, a call we don't make, a superseded version `we used to X, now Y`, a limitation) in one shape only: where a reader would otherwise **do** something harmful, and the sentence says what to do instead. GOOD: `wrap a direct ./tests/run.sh to put it under the same lock`. BANNED: `run.sh takes no host lock`. A limitation without a step to take instead belongs in the PR or ticket, because **documenting a limitation is not fixing it**.
|
||||
|
||||
## Links and pointers
|
||||
|
||||
|
||||
@@ -365,7 +365,7 @@ Weigh the widely used tmux-resurrect and tmux-continuum plugins against a script
|
||||
|
||||
### 267. Keep waiting when an interrupt gets an empty answer, at the terminal or in the claude opened for it, in place of stopping the loop.
|
||||
|
||||
11 runs in the archive ended with "Stopped: the human is needed to answer the question above, but the interrupt ended with nothing typed", each leaving its chunk open until the human restarted the loop. That stop comes from `human_typed` in `bin/loop_claude/cycle/interrupt.py`, after the claude opened for `/talk` or for a release ready to cut ends with no human turn. An empty line at the terminal stops the loop the same way, in `first_answer`. Item 268 removes one of the cases that opens claude.
|
||||
11 runs in the archive ended with "Stopped: the human is needed to answer the question above, but the interrupt ended without anything typed", each leaving its chunk open until the human restarted the loop. That stop comes from `human_typed` in `bin/loop_claude/cycle/interrupt.py`, after the claude opened for `/talk` or for a release ready to cut ends with no human turn. An empty line at the terminal stops the loop the same way, in `first_answer`. Item 268 removes one of the cases that opens claude.
|
||||
|
||||
### 4. Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
You compare two code-review processes, A and B, on one real chunk of work. Each process reviewed the same chunk three times, independently: runs A1–A3 and B1–B3. Rule whether B reviews at least as well as A, and whether A reviews at least as well as B.
|
||||
|
||||
Everything is under `{dir}`; read nothing outside it:
|
||||
Everything is under `{dir}`; don't read anything outside it:
|
||||
|
||||
- `truth-issues.md`: what the chunk's real review found and fixed, one row per issue.
|
||||
- `truth.diff`: the real review's fixes.
|
||||
|
||||
@@ -9,7 +9,7 @@ Report: write it to `{report}`.
|
||||
- `VERSION_DIR/skills/larv-review/SKILL.md` is larv-review. Do not invoke the installed `larv-review` skill.
|
||||
- Wherever a text cites `~/.claude/<path>`, read `VERSION_DIR/<path>` instead. `VERSION_DIR/CLAUDE.md` replaces `~/.claude/CLAUDE.md`, which you already have loaded: where the two differ, VERSION_DIR's wins.
|
||||
- Launch every reviewer named `<name>` under `~/.claude/agents/` as a `general-purpose` agent whose prompt opens with the body of `VERSION_DIR/agents/<name>.md` (frontmatter dropped), then the task. A reviewer the process names that has no file there gets the process's own description of it as its prompt.
|
||||
- `/security-review`: launch a `general-purpose` agent that invokes the Skill tool's `security-review`, edits nothing, and replies with the skill's report, or that the skill is unavailable. Where that skill is unavailable, emulate it with a `general-purpose` agent, told to review the branch diff for exploitable vulnerabilities and report only those at ≥0.8 confidence.
|
||||
- `/security-review`: launch a `general-purpose` agent that invokes the Skill tool's `security-review`, doesn't edit anything, and replies with the skill's report, or that the skill is unavailable. Where that skill is unavailable, emulate it with a `general-purpose` agent, told to review the branch diff for exploitable vulnerabilities and report only those at ≥0.8 confidence.
|
||||
|
||||
## Bench rules
|
||||
|
||||
@@ -18,7 +18,7 @@ Report: write it to `{report}`.
|
||||
- Run every git command through `python3 {review_git} <subcommand> <args>`, never bare `git`: that wrapper is the only shell command allowed. It runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit`. Where the process asks for another git command, do without it and list it under Ambiguities. Run no other code or test suite. Where the process asks for a run, such as the stability pass's mutation test, review by reading and list the skipped run under Ambiguities.
|
||||
- Skip the comprehension check; say so in the report.
|
||||
- No human is reachable. Where the process names a no-human branch, take it; where it names none, take your recommendation and list the question in the report.
|
||||
- Commit each round's fixes on `chunk` with `python3 {review_git} commit -m <message>`, messages prefixed `{case} - review: `. Touch nothing outside this repository except the report.
|
||||
- Commit each round's fixes on `chunk` with `python3 {review_git} commit -m <message>`, messages prefixed `{case} - review: `. Don't touch anything outside this repository except the report.
|
||||
|
||||
## Report
|
||||
|
||||
|
||||
Reference in New Issue
Block a user