Answer prose round 1: 0 🔴, 5 🟡 (0 misread); define the review standards at first use, retitle the worker decision, say where the judge reads the rules, and reword the citation, records and help comments
This commit is contained in:
@@ -46,7 +46,7 @@ Write the thing, not an introduction to it. State what holds and stop; name a re
|
||||
|
||||
Write negations as people say them: after a verb, or after its preposition, write `doesn't <verb> anything` or `anyone`, never `<verb> nothing` or `no one`. BANNED: `stops nothing`, `depends on no one`, `so answering in time costs nothing`. GOOD: `doesn't stop anything`, `doesn't depend on anyone`, `so answering in time doesn't cost anything`.
|
||||
|
||||
Reviewers hold prose to the full rules, with examples: `prose.md` in the project's review standards.
|
||||
Reviewers hold prose to the full rules, with examples: `prose.md` in the project's review standards, which "Topic rules" below defines.
|
||||
|
||||
## Public API specs
|
||||
- Text in public API specs (OpenAPI `description`/`title` fields, in a spec file or in code, and the generated api-docs) is read by external consumers. Never leak internal references: no ticket IDs, internal URLs, repo/system code-names, "see ticket X". Describe behaviour in consumer terms. GOOD: `Deprecated alias for paymentAccountId`. BANNED: `…will be removed (ABC-123)`.
|
||||
|
||||
@@ -11,7 +11,7 @@ You are an elite stability-focused code reviewer with deep expertise in defensiv
|
||||
## First Steps
|
||||
|
||||
1. **Scope the change.** You review every PR; a fresh invocation has no prior context, so determine the diff yourself. Diff the PR branch against its base (`gh pr diff`, or `git merge-base <base> HEAD` then `git diff`), and review the changed lines **plus enough surrounding code and callers** to judge correctness — read the whole changed file when practical; a diff alone hides whether a caller breaks. Read the PR description and, when the project uses an issue tracker (project-specific — check its docs) and the change references a ticket, that ticket: the diff is judged against its stated intent, and a scope mismatch in either direction is a finding — claimed-but-unimplemented, and implemented-but-unasked (#2).
|
||||
2. **Read every file in the folder your prompt's `Technical principles:` line names**, and `quality.md` in the folder its `Review standards:` line names. These are mandatory guidelines; all code MUST comply with each that applies. A file's `Applies to` line says what it covers; one without the line covers every change. Re-read each time — they change. Flag violations prominently. Where your prompt lacks either line, open your report by saying so.
|
||||
2. **Read every file in the folder your prompt's `Technical principles:` line names**, and `quality.md` in the folder your prompt's `Review standards:` line names. These are mandatory guidelines; all code MUST comply with each that applies. A file's `Applies to` line says what it covers; one without the line covers every change. Re-read each time — they change. Flag violations prominently. Where your prompt lacks either line, open your report by saying so.
|
||||
3. **Read README.md, AGENTS.md and `docs/decisions.md`** in the current project for project-specific conventions, build instructions, tooling, and the accepted trade-offs a reviewer should honor rather than re-flag. Note the documented test, lint, and static-analysis commands — you run them unless your prompt hands you a passing run on the head (step 4).
|
||||
4. **Verify, don't just read.** Where your prompt hands you the commit on which the worker's run of the test suite and static analysis last passed, and it is the head, don't run them again, and say so in Coverage & Verification. Otherwise run the test suite and static analysis/linters through the project's container tooling, unless `~/.claude/bin/project-config tooling-on-host` prints true or a decision puts them on the host; tear down test containers afterwards. Run the full suite under `~/.claude/bin/run-heavy`. Run it in the background or with the Bash tool's longest timeout: that timeout includes the wait for a slot. A full-suite run catches broken callers that diff-reading misses. When a suspected bug is cheap to confirm, reproduce it — a scratch script, a targeted test run — before reporting it. Make scratch work with Bash, since Write is for your report alone. Put it in a directory you create for this run, and delete it when done; don't delete anything else, since a directory the prompt names may hold other agents' files. Leave the working tree exactly as you found it. Anything you could not run goes in Coverage & Verification, never silently skipped.
|
||||
5. **Judge the surviving mutants.** Your prompt hands you what the project's mutation command printed on the change, with the commit it ran on: the mutants of changed lines that no test kills, or the message of a run that failed. Ignore a surviving mutant that leaves behaviour unchanged, or changes only message or log text no caller or user depends on. Report one 🟡 per test to add or strengthen, naming the surviving mutants it would catch. Where the prompt doesn't hand you any output, or names a commit other than the head, run the command yourself in the change's checkout, through run-heavy as in step 4: the one the project's README documents, else `~/.claude/bin/run-heavy ~/.claude/bin/mutate-changes --prefix '<how the project runs its tools>'`. `~/.claude/bin/mutate-changes --help` says which tool it runs for each language, how to set `--jobs`, and what it prints. Mutation-test the changed code that no run mutated, in a language mutate-changes doesn't cover or where a run failed, with a widely deployed mutation tool limited to the changed lines, else with mutants you make by hand. Report the wall time and mutant count in Coverage & Verification, or why nothing was mutated.
|
||||
|
||||
@@ -139,8 +139,8 @@ places prints into places.md, and lists the plan's items into
|
||||
plan.md, whatever store holds the plan; a file whose text is unchanged is left as it is. With --rerun, the
|
||||
header names the report of a relaunch one rung up, in the same round, and the report that asked for
|
||||
it. It exits 1 when the pass takes no more rounds, when `project-config` fails in the checkout, when
|
||||
either folder it names is not a folder with a file in it, or, with --rerun, when the report that
|
||||
asked is missing.
|
||||
the principles or standards folder is not a folder with a file in it, or, with --rerun, when the
|
||||
report that asked is missing.
|
||||
|
||||
reviewer-context is the SubagentStart hook. It reads the hook's input on stdin. For a reviewer
|
||||
whose definition under ~/.claude/agents/ has `omitClaudeMd: true`, and so starts without CLAUDE.md,
|
||||
|
||||
@@ -57,7 +57,7 @@ FINISHED_RULES = (
|
||||
"finished=true when the chunk is complete: merged, or nothing remains that the worker could do on "
|
||||
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}.\n\n"
|
||||
"An answer counts as recorded where the premises or the records above already hold it, the human's rules "
|
||||
"(~/.claude/CLAUDE.md, ~/.claude/rules/, the project's review standards) included. "
|
||||
"(~/.claude/CLAUDE.md, ~/.claude/rules/, the project's review standards, each headed by its path) included. "
|
||||
f"It also counts where the transcript shows the worker writing it into {ANSWER_DESTINATIONS}, "
|
||||
"but not into one of the human's rules.\n\n"
|
||||
"finished=false when work remains that a further run can "
|
||||
|
||||
@@ -9,7 +9,8 @@ from record_store.stores import store
|
||||
RECORD_FILES = ("AGENTS.md", "docs/decisions.md")
|
||||
# The setting in the project's config that places each of RECORD_FILES.
|
||||
RECORD_SETTINGS = ("working-rules", "decisions")
|
||||
# The human's rules: under ~/.claude, CLAUDE.md, then each .md file in the rules folder; then each .md file in the folder of review standards the project's config sets.
|
||||
# The human's rules, in the order the judge reads them: ~/.claude/CLAUDE.md, each .md file in ~/.claude/rules/,
|
||||
# then each .md file in the review standards folder the project's config sets.
|
||||
HUMAN_RULES_FILE = "CLAUDE.md"
|
||||
HUMAN_RULE_DIR = "rules"
|
||||
HUMAN_RULE_NAMES = (f"~/.claude/{HUMAN_RULES_FILE}", f"~/.claude/{HUMAN_RULE_DIR}/", "the project's review standards")
|
||||
|
||||
@@ -9,9 +9,9 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
from loop_claude.project.records import HUMAN_RULE_DIR
|
||||
from repo_checks import ROOT
|
||||
|
||||
# The review standards this repo ships, which a project's config names as `standards` where it keeps them.
|
||||
# The review standards this repo ships. A project's config may name another folder as `standards`.
|
||||
STANDARDS = "skills/larv-review/standards"
|
||||
# The folders whose files a citation names by path: the judge reads both, the standards where a project's config names them.
|
||||
# The folders whose files a citation may name by path. The judge reads both.
|
||||
RULE_DIRS = (HUMAN_RULE_DIR, STANDARDS)
|
||||
RULE_PATH = rf"(?:{'|'.join(map(re.escape, RULE_DIRS))})/[\w-]+\.md"
|
||||
NAMES = r'("[^"]+"(?:,? (?:and )?"[^"]+")*)'
|
||||
|
||||
+3
-3
@@ -43,7 +43,7 @@ Each project principle states one rule that the decisions below apply; their ent
|
||||
- todo.md: one table of items, scored by the formula it states, held to its bar and ranked by `bin/rank_todo.py`, which ranks each blocker above the items it blocks; each achieve goal weighs by its position, code health first and four times as heavy while the comprehension panel's overall is under the project's readability floor or unmeasured, then the goals file's goals, and a maintain goal weighs 0.5 per point of the confidence an item's test, gate leg or probe adds to it; the table's order is the plan's only order
|
||||
- todo.md: only a session or subagent on the project's rung or above scores or reshapes the plan; one below it drafts new items and changes to filed ones for a subagent at or above that rung to apply and score, and removes finished ones
|
||||
- todo.md: tools and agents read and change the plan as it stands only through `bin/plan_store` and `todo-item`; a tool that reads what a past commit held, ranks the file or checks its rank still parses `todo.md`
|
||||
- worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the standards in `skills/larv-review/standards/`
|
||||
- worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the project's review standards
|
||||
- workflow-bench/: the remote may show the bench cases' names and commit SHAs
|
||||
|
||||
# Working rules
|
||||
@@ -122,7 +122,7 @@ Valid while `~/.claude` is a checkout of this repo.
|
||||
|
||||
2026-10-03, lilleman. Serves goal 1.2. The judge reads `~/.claude/CLAUDE.md`, each file under `~/.claude/rules/` and each file in the project's review standards, the folder its config sets as `standards`, whole. The worker follows them, so a question they answer should not reach the human. Reading whole is safe, because the judge may quote only words that appear in them, and a list of sections would need keeping current.
|
||||
|
||||
2026-10-03, lilleman. Serves goal 1.2. The judge rules a chunk finished only once every answer the human typed in it is recorded. A finished chunk's session is deleted, and its answers with it, so an answer recorded nowhere would be asked again in a later chunk. Claude settled two choices lilleman's ruling left open. First, an answer counts as recorded where the files the judge reads hold it, `~/.claude/CLAUDE.md` included, or where the transcript shows the worker writing it into `README.md`'s premises, `AGENTS.md`, `docs/decisions.md`, `todo.md` or a code comment. A worker's write to `~/.claude/CLAUDE.md`, `~/.claude/rules/` or the project's review standards does not count: the judge reads the live copy, which holds the write only once it merges and the checkout is pulled. The judge cannot read a code comment or `todo.md`. Yet `CLAUDE.md` → "Working loop", "Ask, don't guess" puts a standing choice made in one place in a comment there. And the human's approval of a release's items, which "Working loop" requires, is the set of rows `todo.md` names for that release. The judge also reads the records once a cycle from the main checkout. That checkout may not yet hold an answer the worker wrote and merged. A judge that refused any of these would hold the chunk open every cycle. Second, only an answer a later chunk could need must be recorded, so an approval such as "merge it" doesn't hold anything open. The worker's resume prompt states the rule, so a worker whose chunk is held open knows it.
|
||||
2026-10-03, lilleman. Serves goal 1.2. The judge rules a chunk finished only once every answer the human typed in it is recorded. A finished chunk's session is deleted, and its answers with it, so an answer recorded nowhere would be asked again in a later chunk. Claude settled two choices lilleman's ruling left open. First, an answer counts as recorded where the files the judge reads hold it, `~/.claude/CLAUDE.md` included, or where the transcript shows the worker writing it into `README.md`'s premises, `AGENTS.md`, `docs/decisions.md`, `todo.md` or a code comment. A worker's write to `~/.claude/CLAUDE.md`, `~/.claude/rules/` or the project's review standards does not count: the judge reads those files where they are kept, not in the chunk's worktree, so they hold the write only once it merges and reaches there. The judge cannot read a code comment or `todo.md`. Yet `CLAUDE.md` → "Working loop", "Ask, don't guess" puts a standing choice made in one place in a comment there. And the human's approval of a release's items, which "Working loop" requires, is the set of rows `todo.md` names for that release. The judge also reads the records once a cycle from the main checkout. That checkout may not yet hold an answer the worker wrote and merged. A judge that refused any of these would hold the chunk open every cycle. Second, only an answer a later chunk could need must be recorded, so an approval such as "merge it" doesn't hold anything open. The worker's resume prompt states the rule, so a worker whose chunk is held open knows it.
|
||||
|
||||
2026-10-17, lilleman, todo item 168. The human answers an interrupt on loop-claude's terminal, and the worker resumes headless with the answer in a `-p` prompt. The judge reads that answer as a `[human]` line, not a `[loop]` one, so it holds the chunk open until the answer is recorded, as it does for an answer typed in claude.
|
||||
|
||||
@@ -732,7 +732,7 @@ Four choices the item left open:
|
||||
|
||||
`rules/todo-items.md` loads when a session itself reads or writes a `todo.md`, which a session that runs `todo-item` never does, so `CLAUDE.md` → "Topic rules" asks for it before an item is filed, scored, edited or moved. A reviewer without a shell reads the plan from the listing `larv-review reviewer-header` writes, which the header's `Plan:` line names. Valid while `todo-item` is the only command that changes a plan's items.
|
||||
|
||||
## worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the standards in `skills/larv-review/standards/`
|
||||
## worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the project's review standards
|
||||
|
||||
2026-10-13, lilleman through todo item 104, implemented by claude. Serves goals 1.4 and 1.5. Like red, green, refactor: the worker makes the item work, and the review passes raise it to the bar. A loop-claude worker loaded `CLAUDE.md` and every file under `rules/`, 4,247 words, and `CLAUDE.md` told it to read `technical-principles.md` too, 2,426 words more. The reviewers read the principles anyway, and a finding against one names it.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user