Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries
This commit is contained in:
@@ -36,6 +36,7 @@ Decisions live in `docs/decisions.md`:
|
||||
- rules/: `CLAUDE.md` holds the worker's rules that no file path can scope, and each topic's rules for the worker live in a `rules/` file that Claude Code loads by its `paths:` frontmatter
|
||||
- worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the standards in `skills/larv-review/standards/`
|
||||
- tests: each test earns its keep as the technical principles' test bullet says, and the worker takes a test's expected values from its item, never from the code
|
||||
- review: each reviewer under `agents/` writes its full report under `/tmp/larv-review/` and returns the worker its verdict and one line per finding; rule-hook refuses any other file call from it
|
||||
- review: stability and product-owner report only a defect, a broken requirement of the plan or the goals, a breach of a technical principle, or for stability a hard place the change touches; each architect finding names a later change the PR makes harder; prose runs only on a chunk that changes prose
|
||||
- review: the stability pass mutation-tests the lines a chunk changes, in place of measuring test coverage
|
||||
- routing: one ladder of (model, effort) rungs, ordered by the Artificial Analysis Intelligence Index with Fable 5.1 at max on top; a task's L picks its worker's rung and its E + I its reviewers', by fixed steps counted from the cheapest rung of the project's model
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Each reviewer under `agents/` writes its full report to a file under `/tmp/larv-review/`, which larv-review names in its prompt, and returns only its verdict and one line per finding, so a review takes less of the worker's context. The worker reads a finding's full entry before it fixes or answers a 🔴 or 🟡. Each reviewer now holds Write. `rule-hook` refuses a reviewer's Edit and MultiEdit, and any Write outside that folder, and takes no override.
|
||||
- `loop-claude` asks an interrupt's question on its own terminal and resumes the worker headless with your one-line answer, so the loop no longer waits for you to leave a claude session. Type `/talk` instead to open the worker's session in claude, as every interrupt did before, to read the diff or discuss first. A release ready to cut opens that session at once. Pressing Enter on an empty answer stops the loop, exit 2. Keys typed before the question shows are dropped, and a pasted answer keeps only its first line; type `/talk` for a longer one. The judge counts that answer as yours, like one typed in claude: the chunk stays open until the answer is recorded, where a later chunk could need it.
|
||||
- When a worker's run ends with a question and a request for a higher rung, `loop-claude` now steps the worker one rung up as the judge interrupts, or as it answers the question with a quote from `README.md` or another file it may quote. The judge must quote a passage showing that the escalation trigger the run cites fired. Before, the step was lost for an answered question, and the worker carried on from the answer on its old rung. The step has the same limits as the step on an unfinished verdict, and no later ruling of that cycle carries a second escalation. The archive's interrupt and quoted-answer outcomes now name the worker's rung.
|
||||
- A scoring run due because a release's items ran out costs four more seats: it is done twice on one head. `larv-review agreement` compares how often Mid A and the inherited architect vote alike in paired rulings with how often their overall scores move the same way between the two runs. After 10 such pairs it gives a verdict, and on `Assumption fails:` larv-review asks you whether to keep the paired ruling.
|
||||
|
||||
@@ -66,7 +66,7 @@ The rules only reviewers apply, for quality, comments, READMEs and prose, live i
|
||||
4. A review rules the work below standard: systems-architect's BLOCK, a finding against larv-review's `standards/quality.md`, or a vote `record-ruling` counts as worse, in the report that stands after any relaunch one rung up (`larv-review` §3 → "Rung up").
|
||||
|
||||
When one fires, hand the hard part, edits included, to a subagent on the rung loop-claude's prompt names, one above yours. Open the subagent's prompt with the marker loop-claude's prompt gives, then the trigger's evidence. Each later subagent on that task runs on that rung or higher, and each further escalation trigger moves the task one rung higher, up to the highest rung loop-claude's prompt names. Ask the human before going higher. Where loop-claude's prompt names no escalation, as on the ladder's top rung or off it, keep the task in your own thread.
|
||||
- Give a subagent only the access its job needs, and state the limits in its prompt. One that only reads, such as a prose review, gets no shell and no edits. One that probes a build tags the image as throwaway and removes it afterwards, writes only to the session scratchpad, and removes no image or file it did not create.
|
||||
- Give a subagent only the access its job needs, and state the limits in its prompt. One that only reads, such as a prose review, gets no shell and no edits; a reviewer under `agents/` also writes its report, under `/tmp/larv-review/`, where rule-hook confines it. One that probes a build tags the image as throwaway and removes it afterwards, writes only to the session scratchpad, and removes no image or file it did not create.
|
||||
- Check `README.md` first for build, test, install, and codegen commands, and fix one that misbehaves in the same change. Refresh `README.md` whenever you change what it describes, keeping it true, not longer.
|
||||
- Where `~/.claude/bin/rule-hook` does not check it, such as an image named in test code, a Kubernetes manifest, a `docker run` line or a file a Bash command writes, pin a container image to its full patch version and a GitHub Action to a semver tag.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: override-reviewer
|
||||
description: "Use this agent for larv-review's overrides pass."
|
||||
tools: Read, Grep, Glob
|
||||
tools: Read, Grep, Glob, Write
|
||||
---
|
||||
|
||||
You rule on each override of a rule `~/.claude/bin/rule-hook` enforces. The prompt gives you the lines `rule-hook overrides` printed, each with its rule and reason, and the branch diff. Read each rule and its stated exceptions in `~/.claude/bin/rule_hook/rules.py` → `RULES`, how an override is written and logged in the docstring of `~/.claude/bin/rule_hook/main.py`, and the project's `README.md` → "Goals" and "Audience".
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: principles-auditor
|
||||
description: "Use this agent for larv-review's principles pass: one yardstick checked against the whole project."
|
||||
tools: Read, Grep, Glob
|
||||
tools: Read, Grep, Glob, Write
|
||||
---
|
||||
|
||||
You check one yardstick against the whole project: every principle in the file you are handed, or every goal under the project's `README.md` → `## Goals`.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: "product-owner-reviewer"
|
||||
description: "Use this agent when you need a product-focused review of changes that affect external-facing surfaces (REST APIs, web UIs, CLI commands, public SDKs, webhooks, etc.) from an end-customer perspective. This agent evaluates ease of use, edge cases, persona fit, and reasonableness of usage paths — not internal implementation details unless they leak into the external experience. Invoke after a feature, endpoint, UI flow, or CLI command has been drafted or modified, or before opening a PR that changes user-facing behavior.\\n\\n<example>\\nContext: The developer just added a new REST endpoint for cancelling a subscription.\\nuser: \"I've added a DELETE /v4/subscriptions/{id} endpoint that cancels a subscription immediately.\"\\nassistant: \"Let me launch the product-owner-reviewer agent to evaluate this from an end-customer perspective.\"\\n<commentary>\\nA new external API surface was introduced. Use the Agent tool to launch product-owner-reviewer to check personas, edge cases (already cancelled? pending invoices? refund expectations?), and usability of the contract.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: The developer added a new CLI subcommand.\\nuser: \"Added `mytool deploy --env prod --force` to skip confirmation.\"\\nassistant: \"I'll use the Agent tool to launch the product-owner-reviewer agent to review the CLI ergonomics and edge cases.\"\\n<commentary>\\nCLI usage is an external surface. The product-owner-reviewer should evaluate persona fit (who uses --force? is it discoverable? what happens on partial failure?).\\n</commentary>\\n</example>\\n\\n<example>\\nContext: The developer modified a checkout flow in the web UI.\\nuser: \"I changed the checkout to require email verification before payment.\"\\nassistant: \"Let me launch the product-owner-reviewer agent to review this user-facing change.\"\\n<commentary>\\nWeb UI flow change directly affects end customers. Use product-owner-reviewer to assess friction, error states, recovery paths, and whether the change makes sense for the documented personas.\\n</commentary>\\n</example>"
|
||||
tools: Read, Grep, Glob
|
||||
tools: Read, Grep, Glob, Write
|
||||
---
|
||||
|
||||
You are a seasoned Product Owner with deep expertise in user experience, API design, and customer empathy. You have shipped consumer and B2B products across web, mobile, REST APIs, and developer-facing CLIs. Your superpower is putting yourself in the end customer's shoes and surfacing the friction, confusion, and edge cases that engineers tend to miss.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: prose-reviewer
|
||||
description: "Use this agent for larv-review's prose pass."
|
||||
tools: Read, Grep, Glob
|
||||
tools: Read, Grep, Glob, Write
|
||||
---
|
||||
|
||||
You read the prose as someone who did not write it: you see only what it says, never what the author meant.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: "stability-reviewer"
|
||||
description: "Use this agent when code has been written or modified and needs review for stability, bug prevention, and adherence to coding best practices. This includes after implementing new features, refactoring existing code, or fixing bugs. The agent focuses on defensive programming, error handling, edge cases, and compliance with the project's technical principles.\\n\\nExamples:\\n\\n- user: \"Please implement a retry mechanism for our API client\"\\n assistant: \"Here is the retry mechanism implementation: [code written]\"\\n Since significant code was written, use the Agent tool to launch the stability-reviewer agent to review the new code for stability and best practices.\\n assistant: \"Now let me use the stability-reviewer agent to review this implementation for stability concerns.\"\\n\\n- user: \"Refactor the payment processing module to use the new gateway\"\\n assistant: \"I've refactored the payment processing module. Let me now use the Agent tool to launch the stability-reviewer agent to check for any stability issues or violations of our technical principles.\"\\n\\n- user: \"Can you review what I just wrote?\"\\n assistant: \"I'll use the Agent tool to launch the stability-reviewer agent to thoroughly review your recent changes.\"\\n\\n- After any significant code change, proactively launch the stability-reviewer agent to catch issues before they reach production."
|
||||
color: orange
|
||||
tools: Bash, Read, Grep, Glob
|
||||
tools: Bash, Read, Grep, Glob, Write
|
||||
---
|
||||
|
||||
You are an elite stability-focused code reviewer with deep expertise in defensive programming, fault-tolerant system design, and production incident prevention. You have decades of experience identifying subtle bugs, race conditions, edge cases, and architectural weaknesses that lead to outages and unexpected behavior. You think like a chaos engineer — always asking "what could go wrong?" — and like a maintainer of ten years, always asking "what will this cost to live with?" There are no bugs, and no security holes, in code that doesn't exist.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: systems-architect
|
||||
description: "Use this agent when designing new systems, planning major features, reviewing architectural decisions in PRs, or evaluating the high-level structure of a codebase. It runs as a periodic, deliberate alignment review — not a per-PR reviewer; the stability-reviewer is the per-PR, code-level companion. This agent focuses on architectural patterns (functional core/imperative shell), the external contract with consumers, component boundaries, internal API design, testability seams and verification strategy (never individual tests), delivery strategy (merge gate with tests and static analysis, deployment path, dependency automation — as architecture, never pipeline configs or lint rules), naming, file system organization, the simplicity and reasonability of the core (questioning unjustified duplication and accidental complexity, demanding a stated reason for every repeated or complex structure), quantitative fits-in-your-head guardrails (per Code That Fits in Your Head), simpler near-equivalent solutions (trading a small, named behavior delta for a large complexity reduction), and long-term maintainability rather than implementation details. It also evaluates fitness for purpose — whether the chosen language, frameworks, and overall solution approach are the best fit for the project's goals and priorities — and treats missing or contradictory goals, priorities, audience, problem statement, scale and horizon, or domain vocabulary as the first thing to fix. It should be used proactively when starting new projects or significant new subsystems, and when reviewing PRs that introduce new architectural patterns or restructure existing ones.\\n\\n<example>\\nContext: The user is starting a new project and needs architectural guidance before writing code.\\nuser: \"I'm building a service that ingests events from multiple sources, deduplicates them, and forwards them to downstream systems.\"\\nassistant: \"Before we start implementing, let me use the Agent tool to launch the systems-architect agent to design the high-level architecture and data flows.\"\\n<commentary>\\nA new project is being started — use the systems-architect agent to deliberately design the architecture before any code is written.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: A PR has been opened that introduces a new module or restructures existing components.\\nuser: \"Here's the PR for the new payment processing subsystem: #4521\"\\nassistant: \"I'll use the Agent tool to launch the systems-architect agent to review the high-level architecture and component boundaries in this PR.\"\\n<commentary>\\nA PR introduces a new subsystem — the systems-architect agent should review architectural choices, not line-level details.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: The user is adding a significant new feature that crosses multiple components.\\nuser: \"We need to add real-time notifications across the web app, mobile app, and email.\"\\nassistant: \"This crosses several system boundaries. Let me use the Agent tool to launch the systems-architect agent to design how this should fit into the existing architecture.\"\\n<commentary>\\nA feature crossing multiple components needs deliberate architectural design — use the systems-architect agent proactively.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: The user asks for an architecture review but the project has no clearly stated goals or priorities.\\nuser: \"Can you review the architecture of our new sync service?\"\\nassistant: \"I'll use the Agent tool to launch the systems-architect agent. Because fitness-for-purpose depends on goals, it will first pin down the project's goals and priorities, then judge whether the language, frameworks, and approach actually serve them.\"\\n<commentary>\\nJudging whether an implementation is the best one requires explicit goals and priorities — the systems-architect agent drafts the missing goals and priorities and asks the user to confirm them before evaluating fitness.\\n</commentary>\\n</example>"
|
||||
tools: Read, Grep, Glob, WebFetch, WebSearch
|
||||
tools: Read, Grep, Glob, WebFetch, WebSearch, Write
|
||||
---
|
||||
You are an elite Systems Architect with decades of experience designing software systems that endure for as long as their horizon demands. Your philosophy is uncompromising on fundamentals: clarity, simplicity, and deliberate structural choices trump short-term convenience every time. You design systems people will read, understand, and extend for their whole recorded lifetime.
|
||||
|
||||
|
||||
@@ -11,6 +11,8 @@ class Call:
|
||||
tool: str
|
||||
given: dict[str, typing.Any]
|
||||
cwd: str
|
||||
# The subagent the call comes from, such as prose-reviewer; empty for the session's own agent.
|
||||
agent: str = ""
|
||||
|
||||
|
||||
def parsed_call(hook_input: object) -> Call | Failure:
|
||||
@@ -19,7 +21,8 @@ def parsed_call(hook_input: object) -> Call | Failure:
|
||||
tool, given, cwd = hook_input.get("tool_name"), hook_input.get("tool_input"), hook_input.get("cwd")
|
||||
if not isinstance(tool, str) or not isinstance(given, dict) or not isinstance(cwd, str):
|
||||
return Failure("the hook input has no string tool_name and cwd and object tool_input")
|
||||
return Call(tool, given, cwd)
|
||||
agent = hook_input.get("agent_type")
|
||||
return Call(tool, given, cwd, agent if isinstance(agent, str) else "")
|
||||
|
||||
|
||||
def text(given: dict[str, typing.Any], key: str) -> str:
|
||||
|
||||
@@ -41,6 +41,7 @@ from rule_hook.rules import Breach, command_breaches, file_breaches, Outcome, ru
|
||||
from git_command import commits
|
||||
from failure import Failure
|
||||
from rule_hook.override_log import Log, log_path, logged, print_overrides, write_override
|
||||
from rule_hook.reviewer_report import report_breaches
|
||||
from rule_hook.reviewer_rung import reviewer_breaches
|
||||
from shell_command import simple_commands
|
||||
from rule_hook.todo_check import todo_breaches
|
||||
@@ -81,7 +82,7 @@ def breaches_of(call: Call, home: str, environ: typing.Mapping[str, str]) -> tup
|
||||
return None
|
||||
before = read(path)
|
||||
added, after = applied(call, before)
|
||||
return added, path, file_breaches(path, before, added, after)
|
||||
return added, path, report_breaches(call, path) + file_breaches(path, before, added, after)
|
||||
|
||||
|
||||
def answer(judged: Outcome, what: str, now: float) -> int:
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
"""Refuse a reviewer's file call other than a Write of its report into the reports folder: skills/larv-review/review-findings.md → "Report"."""
|
||||
import os
|
||||
|
||||
from rule_hook.call import Call
|
||||
from rule_hook.reviewer_rung import REVIEWERS
|
||||
from rule_hook.rules import Breach
|
||||
|
||||
# Outside every checkout, so no project sees a report. A fixed path, since a headless session has no scratchpad.
|
||||
REPORTS = "/tmp/larv-review"
|
||||
|
||||
|
||||
def report_breaches(call: Call, path: str) -> list[Breach]:
|
||||
"""The breach of a file call from a reviewer that is not a Write under REPORTS; none from any other agent."""
|
||||
if call.agent not in REVIEWERS:
|
||||
return []
|
||||
written = os.path.realpath(os.path.join(call.cwd, path))
|
||||
if call.tool == "Write" and written.startswith(os.path.realpath(REPORTS) + "/"):
|
||||
return []
|
||||
return [Breach("reviewer-report", f"{call.agent} would {call.tool.lower()} {written}; write your report with Write to the path your prompt names, under {REPORTS}/", call.cwd)]
|
||||
@@ -7,7 +7,7 @@ from git_command import commits
|
||||
from round_titles import lacks_counts
|
||||
from shell_command import Simple
|
||||
|
||||
RuleName = typing.Literal["action-pin", "commit-one-line", "ignorefile-sorted", "image-pin", "no-ai-attribution", "reviewer-rung", "round-title-counts", "todo-rank-check"]
|
||||
RuleName = typing.Literal["action-pin", "commit-one-line", "ignorefile-sorted", "image-pin", "no-ai-attribution", "reviewer-report", "reviewer-rung", "round-title-counts", "todo-rank-check"]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -39,6 +39,12 @@ RULES: dict[RuleName, Rule] = {
|
||||
"no commit, PR, issue comment or file carries a Claude or Anthropic attribution, such as a co-author trailer or a generated-with line",
|
||||
"a git, gh or tea command, and any file written",
|
||||
),
|
||||
# A reviewer only reports, so no marker waives it.
|
||||
"reviewer-report": Rule(
|
||||
"a reviewer writes nothing but its report, with Write, into a file under /tmp/larv-review/, outside every checkout",
|
||||
"a Write, Edit or MultiEdit call from inside an agent under agents/",
|
||||
overridable=False,
|
||||
),
|
||||
# A rung is not a judgment call, and the refusal names the arguments to pass, so no marker waives it.
|
||||
"reviewer-rung": Rule(
|
||||
"a reviewer, a comprehension seat or the security pass's subagent runs on the rung loop-claude names for it: for a seat or the security pass the project's rung, and for a reviewer the highest of the project's rung, the worker's and the one the chunk's item's E + I picks, so a worker below the project's rung cannot hand a review to its own cheaper model",
|
||||
|
||||
@@ -233,6 +233,30 @@ class ReviewerRung(unittest.TestCase):
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "model": "haiku", "effort": "low"}, {}), [])
|
||||
|
||||
|
||||
class ReviewerReport(unittest.TestCase):
|
||||
def breached(self, tool: str, path: str, agent: str = "prose-reviewer", cwd: str = "/r") -> list[str]:
|
||||
given = {"file_path": path, "content": "BLOCK\n"} if tool == "Write" else {"file_path": path, "old_string": "a", "new_string": "b"}
|
||||
found = breaches_of(Call(tool, given, cwd, agent), "/h", {})
|
||||
return rules(found[2]) if found else []
|
||||
|
||||
def test_a_reviewer_writes_its_report_into_the_reports_folder(self):
|
||||
self.assertEqual(self.breached("Write", "/tmp/larv-review/.claude-item-73/prose-round-1.md"), [])
|
||||
self.assertEqual(self.breached("Write", "prose-round-1.md", cwd="/tmp/larv-review/x"), [], "a relative path resolves against the call's cwd")
|
||||
|
||||
def test_a_reviewer_writing_anywhere_else_is_refused(self):
|
||||
for path in ["/r/README.md", "/tmp/larv-review", "/tmp/larv-review-x/a.md", "/tmp/larv-review/../etc/a.md", "/tmp/a.md"]:
|
||||
with self.subTest(path):
|
||||
self.assertEqual(self.breached("Write", path), ["reviewer-report"])
|
||||
|
||||
def test_a_reviewer_edits_no_file_even_in_the_reports_folder(self):
|
||||
self.assertEqual(self.breached("Edit", "/tmp/larv-review/x/prose-round-1.md"), ["reviewer-report"])
|
||||
self.assertEqual(self.breached("MultiEdit", "/r/a.py", "stability-reviewer"), ["reviewer-report"])
|
||||
|
||||
def test_the_worker_and_other_subagents_write_anywhere(self):
|
||||
self.assertEqual(self.breached("Write", "/r/README.md", ""), [])
|
||||
self.assertEqual(self.breached("Write", "/r/README.md", "general-purpose"), [])
|
||||
|
||||
|
||||
class Hook(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.repo = tempfile.mkdtemp()
|
||||
@@ -307,6 +331,14 @@ class Hook(unittest.TestCase):
|
||||
call = {"cwd": self.repo, "tool_input": {"subagent_type": "stability-reviewer", "model": "opus", "effort": "medium", "prompt": "Review."}, "tool_name": "Agent"}
|
||||
self.assertEqual(hook(call, self.repo, ReviewerRung.ENVIRON).returncode, 0)
|
||||
|
||||
def test_a_reviewers_write_outside_the_reports_folder_is_refused_and_offers_no_override(self):
|
||||
call = {"agent_type": "prose-reviewer", "cwd": self.repo, "tool_input": {"file_path": f"{self.repo}/a.md", "content": "<!-- rule-override reviewer-report: trust me -->\n"}, "tool_name": "Write"}
|
||||
ran = hook(call, self.repo)
|
||||
self.assertEqual(ran.returncode, 2)
|
||||
self.assertIn("Blocked by rule reviewer-report: prose-reviewer would write ", ran.stderr)
|
||||
self.assertIn("/tmp/larv-review/", ran.stderr)
|
||||
self.assertNotIn("rule-override", ran.stderr)
|
||||
|
||||
def test_a_commit_override_is_logged_in_the_commit_s_checkout(self):
|
||||
outside = tempfile.mkdtemp()
|
||||
self.addCleanup(subprocess.run, ["rm", "-rf", outside])
|
||||
@@ -370,5 +402,5 @@ class Hook(unittest.TestCase):
|
||||
|
||||
def test_help_lists_every_rule(self):
|
||||
helped = subprocess.run([str(HOOK), "--help"], capture_output=True, check=True, env=isolated_env(), text=True).stdout
|
||||
for rule in ["action-pin", "commit-one-line", "ignorefile-sorted", "image-pin", "no-ai-attribution", "round-title-counts", "todo-rank-check"]:
|
||||
for rule in ["action-pin", "commit-one-line", "ignorefile-sorted", "image-pin", "no-ai-attribution", "reviewer-report", "round-title-counts", "todo-rank-check"]:
|
||||
self.assertIn(f" {rule}\n The rule: ", helped)
|
||||
|
||||
@@ -218,6 +218,16 @@ The decision `worker: a chunk's worker writes the simplest change…` has the wo
|
||||
|
||||
Valid while the worker writes the tests that larv-review's reviewers hold to the principles.
|
||||
|
||||
## review: each reviewer under `agents/` writes its full report under `/tmp/larv-review/` and returns the worker its verdict and one line per finding; rule-hook refuses any other file call from it
|
||||
|
||||
2026-10-18, the agent, shipping lilleman's plan item. Serves goal 1.4: a report the worker does not need line by line no longer fills its context. Goal 1.1 holds because the worker reads a finding's full entry before it answers a 🔴 or 🟡.
|
||||
|
||||
The worker names each report's file in the reviewer's prompt, per `skills/larv-review/SKILL.md` → "Reports", and `skills/larv-review/review-findings.md` → "Report" sets what the reviewer returns. Each reviewer holds Write for that. Claude Code passes a subagent's type to a PreToolUse hook as `agent_type`, so `bin/rule_hook/reviewer_report.py` refuses a reviewer's Edit, MultiEdit, and Write outside the folder; no marker overrides it. A reviewer's Bash, as stability-reviewer holds, is not confined. The folder is fixed, since loop-claude's headless sessions have no scratchpad, and sits outside every checkout, so no project sees a report. A general-purpose wrapper that launches the reviewer was not tried: Claude Code's documentation says a subagent cannot start another.
|
||||
|
||||
Before this change, reports took a median 19k tokens per chunk, 8% of the worker's peak context, and lines carrying a severity mark were 9–24% of a report (2026-10-08, over 68 archived transcripts). `stability-replay` still counts each 🔴 and 🟡 by its line in the reviewer's reply.
|
||||
|
||||
Valid while Claude Code passes `agent_type` to hooks inside a subagent.
|
||||
|
||||
## review: stability and product-owner report only a defect, a broken requirement of the plan or the goals, a breach of a technical principle, or for stability a hard place the change touches; each architect finding names a later change the PR makes harder; prose runs only on a chunk that changes prose
|
||||
|
||||
2026-10-13, lilleman through todo item 97, implemented by claude. Serves goals 1.4 and 1.5. Anthropic's Claude Code best practices: "A reviewer prompted to find gaps will usually report some, even when the work is sound… Chasing every finding leads to over-engineering." 320 of this repo's 876 commits answered a review round. So stability and product-owner drop a finding that is none of these, and stability leaves the style of comments to prose, which reads every comment a chunk touches. An architect finding that names no later change it makes harder cannot be checked, so it is dropped. The architecture pass and the comprehension check keep their whole-project view, which guards goal 1.1 as a project grows. Valid while the findings these rules drop keep changing nothing a user would notice.
|
||||
|
||||
@@ -636,3 +636,12 @@ Read most (opens and searches):
|
||||
## Reading cost, chunk 055546870eaf, finished 2026-10-17T20:46:45Z
|
||||
|
||||
100998 new input tokens before the first commit.
|
||||
|
||||
## Reading cost, chunk 6ab072ca20a3, finished 2026-10-17T22:04:13Z
|
||||
|
||||
248782 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `todo.md` 4
|
||||
- `rules/todo-items.md` 1
|
||||
|
||||
@@ -45,6 +45,8 @@ A comment is prose, not code. Passes run in this order: line-level polish on cod
|
||||
|
||||
Give a reviewer without Bash the branch diff and `git ls-files`'s output in its prompt: some harnesses grant it Read alone.
|
||||
|
||||
**Reports.** Name in each prompt to an agent under `~/.claude/agents/` the file its full report goes to: `/tmp/larv-review/<the worktree's folder name>/<pass>-round-<n>.md`, with the yardstick's name after the pass's for a principles-auditor, and `-rerun` before `.md` for a relaunch one rung up. It writes the report there and returns only its verdict and one line per finding. Read the report's entry for each 🔴 and 🟡 before you answer it. rule-hook refuses a reviewer's write anywhere else.
|
||||
|
||||
`~/.claude/bin/larv-review` holds this section's mechanical rules: the round count, the paired ruling's base and vote count, the scoring run's trigger, and the history's format and latest places, and its count of how often seats agree. Run it in the chunk's worktree; `~/.claude/bin/larv-review --help` states each rule. On exit 1, repair what its message names and run it again; when you cannot, ask the user.
|
||||
|
||||
**Precedent, then ask** — for the architecture pass and the full scan:
|
||||
@@ -64,7 +66,7 @@ Give a reviewer without Bash the branch diff and `git ls-files`'s output in its
|
||||
- **Round.** A round is one review by a pass's reviewer: the first, each re-entry, and the rerun after the architect returns Stage 0 questions ("Architecture" below). A relaunch one rung up ("Rung up" below) stays in its round.
|
||||
- **Auto-reviewer.** The external auto-reviewer (§4) is a pass of its own, each review it posts a round.
|
||||
- **Count.** Close each round with one commit that holds the round's fixes. Title it `Answer <pass> round <n>: <k> 🔴, <m> 🟡; …`: `<pass>` is the pass's name in lower case, such as `full scan`, and `<k>` and `<m>` are the 🔴 and 🟡 findings all the round's reviewers reported. Example: `Answer stability round 1: 1 🔴, 2 🟡; flag a deleted test`. A round that changed nothing gets an empty commit (`git commit --allow-empty`). Push the commits when §4 runs next, where there is a remote. `~/.claude/bin/larv-review round-count` prints each pass's count and marks a pass at the cap. The auto-reviewer's count is the number of its reviews on the PR, less its reviews of a `Record the comprehension check:` commit.
|
||||
- **Later rounds.** From round 2, an agent under `~/.claude/agents/` reviews only the answers to its last round's findings, and each fix from another pass that re-entered its pass. Prompt it with its last round's findings verbatim, each with its answer: the fix's diff, or the reason it was dismissed. Give it each other pass's fix as `git diff <the commit before that fix> <the fix's commit>`, since the fixes may be unpushed, with the finding it answered. It reports only an answer that is plainly wrong or worse than the change it answered, so its pass converges in its second round. Other reviewers review the branch as their tool does.
|
||||
- **Later rounds.** From round 2, an agent under `~/.claude/agents/` reviews only the answers to its last round's findings, and each fix from another pass that re-entered its pass. Prompt it with the path of its last round's report and each finding's answer: the fix's diff, or the reason it was dismissed. Give it each other pass's fix as `git diff <the commit before that fix> <the fix's commit>`, since the fixes may be unpushed, with the finding it answered. It reports only an answer that is plainly wrong or worse than the change it answered, so its pass converges in its second round. Other reviewers review the branch as their tool does.
|
||||
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or settled in the PR summary per "Comprehension check" → "Places", or, for the prose pass, fixed. Resolve that round's 🟢s and ❓s per "Findings" too.
|
||||
- **Cap.** A pass takes at most three rounds per chunk. Instead of a fourth, list what is open and ask the user. For the auto-reviewer, ask before the push that would summon its fourth, and hold the fixes unpushed until the answer.
|
||||
- **Answer.** A yes grants that pass one more round, after which the ask repeats; a no leaves the PR unmerged.
|
||||
@@ -78,7 +80,7 @@ Give a reviewer without Bash the branch diff and `git ls-files`'s output in its
|
||||
**Rung up.** A reviewer rules the work below standard when its report holds systems-architect's BLOCK or a finding against `standards/quality.md`. A comprehension seat does when `record-ruling` counts one of its votes as worse.
|
||||
|
||||
- **Ask.** Such a reviewer or seat may end its report asking for a rerun one rung up. Relaunch it only on the rung loop-claude's prompt names for it; where the prompt names none, the report stands.
|
||||
- **Relaunch.** Relaunch a reviewer on that rung with the same prompt and the report that asked; its report stands in place of that one, in the same round. Relaunch a seat with the same prompt alone; its line replaces the one that asked in the votes file, before `record-ruling` runs. A relaunch cannot ask again.
|
||||
- **Relaunch.** Relaunch a reviewer on that rung with the same prompt and the path of the report that asked; its report stands in place of that one, in the same round. Relaunch a seat with the same prompt alone; its line replaces the one that asked in the votes file, before `record-ruling` runs. A relaunch cannot ask again.
|
||||
- **Escalate.** A below-standard ruling in the report that stands is escalation trigger 4 in `~/.claude/CLAUDE.md` → "Working with the codebase".
|
||||
|
||||
**Findings.** A reviewer rates each finding 🔴 must fix, 🟡 should fix, 🟢 nit, ❓ question, or 🔀 escalation naming the reviewer whose concern it is.
|
||||
@@ -161,7 +163,7 @@ Launch prose-reviewer with what the diff touched and every section this review c
|
||||
1. Read the finished PR against the README goals: does anything in it work against one? It breaks a maintain goal → do not merge; redesign the change or drop its item. It works against an achieve goal → do not merge; ask the human, stating the conflict.
|
||||
2. Run the comprehension check (§3) where it runs; it runs once. When `origin/<default branch>` moves after that, return to §4, and keep the ruling and scoring run already recorded.
|
||||
3. Aligned, every pass done and everything green → **merge**. Follow the repo's merge method (`AGENTS.md`; ff-only where that is the convention). A repo withholds this in its `AGENTS.md` → leave the PR ready and unmerged, and say which line withheld it.
|
||||
4. Merged → remove the chunk's items from `todo.md` per `~/.claude/CLAUDE.md` → "Working loop".
|
||||
4. Merged → remove the chunk's items from `todo.md` per `~/.claude/CLAUDE.md` → "Working loop", and the chunk's folder under `/tmp/larv-review/`.
|
||||
5. Stop. The next chunk gets a fresh session.
|
||||
|
||||
## 6. Wrap up
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Words: `~/.claude/CLAUDE.md` → "Vocabulary".
|
||||
|
||||
A reviewer reads and reports; it edits nothing. Read `docs/decisions.md` and `todo.md` first: a finding declined or disproved in `docs/decisions.md` or in a comment at the code it concerns, or filed in `todo.md`, is not raised again, unless the decline works against a goal or a technical principle and names no `todo.md` item ending it.
|
||||
A reviewer reads and reports; it edits nothing, and writes only its report. Read `docs/decisions.md` and `todo.md` first: a finding declined or disproved in `docs/decisions.md` or in a comment at the code it concerns, or filed in `todo.md`, is not raised again, unless the decline works against a goal or a technical principle and names no `todo.md` item ending it.
|
||||
|
||||
## Severity
|
||||
|
||||
@@ -14,6 +14,17 @@ A reviewer reads and reports; it edits nothing. Read `docs/decisions.md` and `to
|
||||
|
||||
A finding against a technical principle names it and the file under `~/.claude/principles/` it lives in. A finding against `~/.claude/skills/larv-review/standards/quality.md` names that file and the rule it breaks.
|
||||
|
||||
## Report
|
||||
|
||||
Write your full report with the Write tool to the file your prompt names, under `/tmp/larv-review/`: the verdict, then every finding with its place, its evidence and its fix, as your agent file shapes them. Then return only:
|
||||
|
||||
- the verdict line
|
||||
- one line per finding: its severity mark, its place, and what is wrong in one sentence, such as `🟡 bin/rank_todo.py:311 place: a cycle among blockers recurses forever`
|
||||
- `Rerun one rung up`, where your report ends with it
|
||||
- the report's path
|
||||
|
||||
A prompt that names no file gets the full report as your reply.
|
||||
|
||||
## Verdict
|
||||
|
||||
A review's first line, derived from the findings:
|
||||
@@ -29,6 +40,6 @@ Where your report holds a finding against `~/.claude/skills/larv-review/standard
|
||||
## Rounds
|
||||
|
||||
- Handed neither earlier findings nor another pass's fix: surface everything.
|
||||
- Handed an earlier round's findings, each with its fix or the reason it was dismissed: review only those answers. Check each fix — re-run the scenario, never trust the claim — and each dismissal's reason. Report only a fix or a dismissal that is plainly wrong, or worse than the change it answered; a fix you would have written differently is not a finding.
|
||||
- Handed an earlier round's report, with each finding's fix or the reason it was dismissed: review only those answers. Check each fix — re-run the scenario, never trust the claim — and each dismissal's reason. Report only a fix or a dismissal that is plainly wrong, or worse than the change it answered; a fix you would have written differently is not a finding.
|
||||
- Handed a fix from another pass: review its diff as a first round would, limited to its lines.
|
||||
- Handed a report that asked to `Rerun one rung up`: confirm or dismiss each of its findings, with a reason, then review the rest of what your prompt scopes, as the round you rerun would. Do not ask again.
|
||||
|
||||
@@ -13,7 +13,6 @@
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 180 | 0.2.0 | | question | **Settle how a trigger that fires after a step-up at an interrupt or a quoted answer steps the worker, which today it never does.** | 2 | 2 | 2 | 4 | 3 | 1.1 by 3 | 4.1 |
|
||||
| 73 | 0.2.0 | | | **Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.** | 3 | 2 | 3 | 6 | 5 | 1.4 by 4 | 1.1 |
|
||||
| 159 | 0.2.0 | | decision | **Run each review pass on the lowest rung that a replay of its archived rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung.** | 4 | 2 | 3 | 7 | 6 | 1.4 by 7 | 1.1 |
|
||||
| 80 | 0.3.0 | | defect | **State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.** | 1 | 2 | 1 | 2 | 1 | 2 by 4 | 17.5 |
|
||||
| 21 | 0.3.0 | | defect | **Give `settings.json`'s model one writer, and sort its keys.** | 3 | 3 | 3 | 7 | 3 | 2 by 5 | 13.7 |
|
||||
@@ -98,19 +97,6 @@ Options:
|
||||
|
||||
Only the human can choose, and the answer revises that decision. Found by the stability review of PR #307.
|
||||
|
||||
### 73. Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.
|
||||
|
||||
2026-10-08, lilleman: rescored from A10/G8 to A4/G4 on the measurement below, so other items that lower token use go first.
|
||||
|
||||
Measured over the 68 transcripts in loop-claude's archive on 2026-10-08, matching each Agent call to its tool result and to any `<task-notification>` naming its tool-use id. No report reached the worker twice: 813 arrived once as the hand-back, and 363 once as a notification after Claude Code ran the agent in the background. Reports took a median 19k tokens per chunk, 8% of the worker's peak context (24% at most). Lines carrying a severity mark are 9–24% of a reviewer's report, and the worker would still open the file for each 🔴 or 🟡 it fixes. A third of the report volume is the comprehension seats', whose votes and places the worker copies into the history. File reads fill more of the worker's context: `sed -n` output is 30% of its Bash results and `grep` output 11%, which `reading-cost` targets.
|
||||
|
||||
The reports would live in the session's scratchpad, such as `<scratchpad>/larv-review/prose-round-1.md`, so no project's checkout sees them. A later round gets the path of that pass's earlier report in place of the findings pasted in. How a report reaches the file:
|
||||
|
||||
- **Write access.** Grant each reviewer Write. No reviewer under `agents/` holds it today, an agent's tool list cannot confine Write to one folder, and `~/.claude/CLAUDE.md` → "Working with the codebase" denies edits to a reviewer that only reads. Check whether a PreToolUse hook can tell a subagent's call from the worker's and deny a Write outside the scratchpad.
|
||||
- **A wrapper.** A general-purpose subagent launches the reviewer, saves the report and returns only the summary lines. Claude Code's documentation says a subagent cannot start another; drop this route unless a test shows one can.
|
||||
|
||||
This risks goal 1.1: a worker fixing from a one-line summary can miss context it needs.
|
||||
|
||||
### 159. Run each review pass on the lowest rung that a replay of its archived rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung.
|
||||
|
||||
lilleman asked, on 2026-10-16, for the prose pass to run on the worker's rung or lower, and the same day, answering PR #296, for every pass to drop below the project's rung only where a replay shows it may. Today `bin/loop_claude/cycle/prompts.py`'s `review_launches` keeps every reviewer at or above the project's rung whatever the item's E + I, so a docs-only chunk such as item 140's, routed to Haiku 5.5, runs its reviewers on Opus 5.5.
|
||||
|
||||
Reference in New Issue
Block a user