Merge pull request 'Give each review pass at most two rounds per chunk, with no third round and no ask for one' (#390) from item-246 into main
This commit was merged in pull request #390.
This commit is contained in:
@@ -23,8 +23,8 @@ Decisions live in `docs/decisions.md`:
|
||||
- reading cost: a rise is the last 5 chunks' median above the 5 before them, and files or refreshes one plan item
|
||||
- reading cost: larv-review measures each finished chunk from loop-claude's archive and appends the figure to `docs/reading-cost.md` in the next chunk's PR
|
||||
- review: `/security-review` is the security pass of record, except where the full scan runs on the same change and replaces it
|
||||
- review: a fix for a 🔴 or 🟡 reruns the pass that found it and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟢, a ❓ or a prose finding reruns none
|
||||
- review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human
|
||||
- review: a fix for a 🔴 or 🟡 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟢, a ❓ or a prose finding reruns none
|
||||
- review: a pass gets at most two rounds per chunk, with no third and no ask for one; the fixes answering round 2 merge once the gate passes
|
||||
- review: a pass skips only on a precedent: a decision, or a plan item crediting the pass, that the repo held before the chunk began
|
||||
- review: a project decision may opt every chunk out of the architect, the product-owner pass or the comprehension check, and a project opting out of the comprehension check voids code health
|
||||
- review: a project decision may suspend the comprehension check for the plan items it names
|
||||
@@ -70,4 +70,4 @@ The project values for `skills/comprehension-panel/seat-prompts.md`:
|
||||
- `{kind}`: a Claude Code configuration with an agent loop and review skills
|
||||
- `{domain}`: driving headless Claude Code sessions, agent review workflows
|
||||
- `{domain docs}`: the Claude Code documentation
|
||||
- `{3am question}`: a loop-claude run exits 1 overnight after the judge returned a verdict the loop rejected, or a larv-review run looped a reviewer a fourth time
|
||||
- `{3am question}`: a loop-claude run exits 1 overnight after the judge returned a verdict the loop rejected, or a larv-review run looped a reviewer a third time
|
||||
|
||||
+1
-1
@@ -2,6 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- larv-review now gives each review pass at most two rounds per chunk, where it gave three and then asked for more, and no longer asks: the fixes answering a pass's second round merge once the gate passes. rule-hook's new `round-cap` rule refuses a round commit that would take a pass past round 2, such as `Answer stability round 3: …`, and takes no override. It counts the pass's earlier round commits on the branch, so a third commit titled round 2 is refused too. `larv-review round-count` marks a pass at two rounds as at the cap.
|
||||
- rule-hook has a new rule, `repeat-read`, and now runs before each Read call too. It blocks a Read, or a Bash `cat`, `head` or `sed -n '<A>,<B>p'` of one file, when every line it asks for is already in the agent's context, and the block names those lines. The rule takes no override.
|
||||
- A line is in context when an earlier read's result in the agent's own transcript showed it since the last compaction, as the file holds it now. A subagent is judged on its own transcript, not its session's.
|
||||
- A read goes through when it asks for a line no earlier read showed, or one that changed since. A Bash output that Claude Code cut short shows no lines.
|
||||
@@ -19,7 +20,6 @@
|
||||
- Reviewers now mark a 🔴 or 🟡 `[misread]` where it arose because the implementer misread code that had already shipped. A round's commit title now counts the marked findings, as in `Answer stability round 1: 1 🔴, 2 🟡 (1 misread); …`, and rule-hook's `round-title-counts` blocks a round title without that count. `score-outcomes` shows the count on each finished item, and lists it after the medians for each chunk whose round titles carry it, newest first. A chunk whose round titles predate this change reads as unknown.
|
||||
- Sessions running at once no longer starve the machine. A new tool, `run-heavy <command>`, runs a heavy command, such as a full gate, mutation tests, an e2e run or a `docker build`, once it holds one of the machine's heavy slots: one per 4 CPUs, shared by every session. `CLAUDE.md` has every session run its heavy commands under it, and rule-hook blocks a `docker build` or a mutation run outside it. loop-claude now waits to start a worker while less than a quarter of the machine's memory is available.
|
||||
- larv-review's skill now says a commit answering the external auto-reviewer takes no `Answer <pass> round` title, and that `larv-review round-count` doesn't list the auto-reviewer. A worker who followed the skill used to get a "not a pass" warning.
|
||||
- rule-hook now blocks a larv-review round commit titled past round 3, such as `Answer stability round 4: …`, unless a `round-cap` override quotes the human's yes.
|
||||
- loop-claude's first prompt for a chunk that no plan item routed now also names the rung on which the worker launches its reviews for an item it takes in, as the prompt after a step-up or an unfinished verdict already did.
|
||||
- `loop-claude --help` now describes routing in two short paragraphs for the person running the loop. The rules it used to repeat are now stated once, in the code that enforces them.
|
||||
- loop-claude now archives the route on each `Worker:` line as a JSON record, such as `{"model": "claude-opus-5-5", "effort": "high", "item": 3, "likelihood": 4, "escape_and_impact": 5, "escalated_at": 12}`, and reads it back field by field. The line used to be a sentence, such as `claude-opus-5-5 at high effort, routed by item 3's L of 4 and E + I of 5`, which loop-claude parsed back word for word: rewording the sentence in the code would have made every archived line unreadable, and a rerun would have routed the open chunk afresh. A rerun and the routing figures still read the sentences archived since loop-claude routed on L. A rerun that finds a `Worker:` line it can't read now says so on stderr. The judge's reply now names the levels the worker took in `highest_likelihood_taken_in` and `highest_escape_and_impact_taken_in`, in place of `bundled` and `bundled_escape_and_impact`.
|
||||
|
||||
@@ -75,7 +75,7 @@ The rules only reviewers apply, for quality, comments, READMEs and prose, live i
|
||||
- Escalate a task that proves too hard for your rung. Four escalation triggers say so:
|
||||
1. A re-score of the item's L, made on the project's rung or above, picks a rung above yours.
|
||||
2. The same failing test or finding survives two fix attempts.
|
||||
3. A pass reports a 🔴 in round 2 or later.
|
||||
3. A pass reports a 🔴 in round 2.
|
||||
4. A review rules the work below standard: systems-architect's BLOCK, a finding against larv-review's `standards/quality.md`, or a vote `record-ruling` counts as worse, in the report that stands after any relaunch one rung up (`larv-review` §3 → "Rung up").
|
||||
|
||||
When one fires, hand the hard part, edits included, to a subagent on the rung loop-claude's prompt names, one above yours. Open the subagent's prompt with the marker loop-claude's prompt gives, then the trigger's evidence. Each later subagent on that task runs on that rung or higher, and each further escalation trigger moves the task one rung higher, up to the highest rung loop-claude's prompt names. Ask the human before going higher. Where loop-claude's prompt names no escalation, as on the ladder's top rung or off it, keep the task in your own thread.
|
||||
|
||||
+6
-3
@@ -11,6 +11,7 @@ class Commit:
|
||||
directory: str
|
||||
messages: list[str]
|
||||
from_file: bool
|
||||
amends: bool
|
||||
|
||||
|
||||
def commits(simples: list[Simple], home: str) -> list[Commit]:
|
||||
@@ -36,14 +37,16 @@ def git_subcommand(words: list[str], directory: str, home: str) -> tuple[str, li
|
||||
return None
|
||||
|
||||
|
||||
def commit_message(args: list[str]) -> tuple[list[str], bool]:
|
||||
"""git commit's messages, and whether it reads one from a file."""
|
||||
def commit_message(args: list[str]) -> tuple[list[str], bool, bool]:
|
||||
"""git commit's messages, whether it reads one from a file, and whether it amends."""
|
||||
messages = []
|
||||
from_file = False
|
||||
amends = False
|
||||
words = iter(args)
|
||||
for word in words:
|
||||
if word == "--":
|
||||
break
|
||||
amends = amends or word == "--amend"
|
||||
name, equals, value = word.partition("=")
|
||||
if name in ("--message", "--file"):
|
||||
given = value if equals else next(words, "")
|
||||
@@ -55,4 +58,4 @@ def commit_message(args: list[str]) -> tuple[list[str], bool]:
|
||||
continue
|
||||
messages += [given] if name == "--message" else []
|
||||
from_file = from_file or name == "--file"
|
||||
return messages, from_file
|
||||
return messages, from_file, amends
|
||||
|
||||
+18
-6
@@ -60,20 +60,32 @@ def default_branch(stale_ok: bool, timeout: float | None = None) -> tuple[str, s
|
||||
return ref if isinstance(ref, Failure) else (ref, note)
|
||||
|
||||
|
||||
def default_ref(cwd: str | None = None, timeout: float | None = None) -> str | Failure:
|
||||
"""The default branch's ref as last fetched, read in cwd, else in the current directory: `origin/<branch>` with that remote, else the local main or master. Asking origin, where the checkout records no default branch for it, fails after timeout seconds."""
|
||||
remotes = git("remote", cwd=cwd)
|
||||
def default_ref(cwd: str | None = None, timeout: float | None = None, guarded: bool = False) -> str | Failure:
|
||||
"""The default branch's ref as last fetched, read in cwd, else in the current directory: `origin/<branch>` with that remote, else the local main or master.
|
||||
|
||||
Where the checkout records no default branch for origin, it asks origin, which fails after timeout seconds. Guarded, it runs git as `git` does with guarded, and fails there instead of asking origin."""
|
||||
remotes = git("remote", cwd=cwd, guarded=guarded)
|
||||
if isinstance(remotes, Failure):
|
||||
return remotes
|
||||
if "origin" in remotes.split():
|
||||
recorded = git("symbolic-ref", "--short", "refs/remotes/origin/HEAD", cwd=cwd)
|
||||
return recorded.strip() if not isinstance(recorded, Failure) else origin_head(cwd, timeout)
|
||||
recorded = git("symbolic-ref", "--short", "refs/remotes/origin/HEAD", cwd=cwd, guarded=guarded)
|
||||
if not isinstance(recorded, Failure):
|
||||
return recorded.strip()
|
||||
return Failure("origin records no default branch") if guarded else origin_head(cwd, timeout)
|
||||
for branch in ("main", "master"):
|
||||
if not isinstance(git("rev-parse", "--quiet", "--verify", f"refs/heads/{branch}", cwd=cwd), Failure):
|
||||
if not isinstance(git("rev-parse", "--quiet", "--verify", f"refs/heads/{branch}", cwd=cwd, guarded=guarded), Failure):
|
||||
return branch
|
||||
return Failure("no origin remote, and no main or master branch to take as the default branch")
|
||||
|
||||
|
||||
def branch_titles(default: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False) -> list[str] | Failure:
|
||||
"""The titles of the branch's own commits since default, newest first, read in cwd, else in the current directory.
|
||||
|
||||
--first-parent leaves out the commits a merge brought in, such as those of a default branch newer than the one the checkout last fetched."""
|
||||
titles = git("log", "--first-parent", "--format=%s", f"{default}..HEAD", cwd=cwd, timeout=timeout, guarded=guarded)
|
||||
return titles if isinstance(titles, Failure) else titles.splitlines()
|
||||
|
||||
|
||||
def origin_head(cwd: str | None, timeout: float | None) -> str | Failure:
|
||||
"""The default branch origin names, for a clone made by `git init` and `git remote add`, which records none."""
|
||||
listed = git("ls-remote", "--symref", "origin", "HEAD", cwd=cwd, timeout=timeout)
|
||||
|
||||
@@ -22,10 +22,11 @@ measured commit is the default branch's newest merged change that adds a paired
|
||||
run to the history; a merge commit counts as one change against its first parent. With none, it is
|
||||
the merge base of HEAD and the default branch.
|
||||
|
||||
round-count prints each pass's round count among the commits not yet on the default branch whose
|
||||
titles open `Answer <pass> round <n>:`, ignoring case. The count is the highest <n>, or the number of
|
||||
such commits when that is more. At or past the cap of {cap} rounds, it says to ask the human before
|
||||
another round. It also lists each name in such a title that is not a pass.
|
||||
round-count prints each pass's round count among the branch's commits since the default branch
|
||||
whose titles open `Answer <pass> round <n>:`, ignoring case. It follows first parents, as
|
||||
rule-hook's round-cap does. The count is the highest <n>, or the number of such commits when that
|
||||
is more. At or past the cap of {cap} rounds, it says the pass takes no more rounds. It also lists
|
||||
each name in such a title that is not a pass.
|
||||
|
||||
needs-scoring-run takes the IDs of the todo.md items the chunk ships, none when it ships none, and
|
||||
prints why the chunk is due a scoring run, or `No scoring run due.` It is due when the chunk ships
|
||||
@@ -101,8 +102,8 @@ import re
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from git_repo import branch_titles, default_branch, git, top
|
||||
from round_titles import CAP, round_counts
|
||||
from git_repo import default_branch, git, top
|
||||
from failure import Failure
|
||||
from comprehension_history import FLOOR
|
||||
from larv_review.history_changes import history_changes
|
||||
@@ -194,10 +195,10 @@ def show_ruling_base(default: str) -> Failure | None:
|
||||
|
||||
|
||||
def round_count(default: str) -> Failure | None:
|
||||
titles = git("log", "--format=%s", f"{default}..HEAD")
|
||||
titles = branch_titles(default)
|
||||
if isinstance(titles, Failure):
|
||||
return titles
|
||||
for name, count in round_counts(titles.splitlines()).items():
|
||||
for name, count in round_counts(titles).items():
|
||||
print(f"{name}: {count}{cap_note(count) if name in PASSES else ', not a pass: check the commit title'}")
|
||||
return None
|
||||
|
||||
@@ -205,7 +206,7 @@ def round_count(default: str) -> Failure | None:
|
||||
def cap_note(count: int) -> str:
|
||||
if count < CAP:
|
||||
return ""
|
||||
return f", {'at' if count == CAP else 'past'} the cap: ask the human before another round"
|
||||
return f", {'at' if count == CAP else 'past'} the cap: no more rounds"
|
||||
|
||||
|
||||
def needs_scoring_run(default: str, ids: list[int]) -> Failure | None:
|
||||
|
||||
@@ -388,14 +388,21 @@ class LarvReviewTool(unittest.TestCase):
|
||||
self.commit("Answer stability round 1: a")
|
||||
self.commit("Answer stability round 2: b")
|
||||
self.commit("Answer prose round 1: c")
|
||||
self.commit("Answer stability round 3: d")
|
||||
ran = self.run_tool("round-count")
|
||||
self.assertIn("stability: 3, at the cap: ask the human before another round\n", ran.stdout)
|
||||
self.commit("Answer stability round 4: e")
|
||||
self.assertIn("stability: 4, past the cap: ask the human before another round\n", self.run_tool("round-count").stdout)
|
||||
self.assertIn("stability: 2, at the cap: no more rounds\n", ran.stdout)
|
||||
self.commit("Answer stability round 3: d")
|
||||
self.assertIn("stability: 3, past the cap: no more rounds\n", self.run_tool("round-count").stdout)
|
||||
self.assertIn("prose: 1\n", ran.stdout)
|
||||
self.assertIn("architecture: 0\n", ran.stdout)
|
||||
|
||||
def test_leaves_out_the_round_commits_of_a_branch_merged_in_as_the_hook_does(self):
|
||||
self.git("switch", "-q", "-c", "sibling")
|
||||
self.commit("Answer stability round 1: a")
|
||||
self.commit("Answer stability round 2: b")
|
||||
self.git("switch", "-q", "chunk")
|
||||
self.git("merge", "-q", "--no-ff", "-m", "Merge sibling", "sibling")
|
||||
self.assertIn("stability: 0\n", self.run_tool("round-count").stdout)
|
||||
|
||||
def test_says_whether_a_scoring_run_is_due_and_refuses_an_unknown_item(self):
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "1").stdout, "Scoring run due: the chunk leaves release 0.1.0 with no plan item.\n")
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, "No scoring run due.\n")
|
||||
@@ -583,7 +590,7 @@ class LarvReviewTool(unittest.TestCase):
|
||||
def test_states_its_rules_and_finds_origins_default_branch_where_the_clone_recorded_none(self):
|
||||
helped = self.run_tool("--help")
|
||||
self.assertEqual(helped.returncode, 0, helped.stderr)
|
||||
self.assertIn("cap of 3 rounds", helped.stdout)
|
||||
self.assertIn("cap of 2 rounds", helped.stdout)
|
||||
self.git("remote", "set-head", "origin", "--delete")
|
||||
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, "No scoring run due.\n")
|
||||
|
||||
|
||||
+6
-6
@@ -1,11 +1,11 @@
|
||||
"""The titles of larv-review's round commits, `Answer <pass> round <n>: <k> 🔴, <m> 🟡 (<j> misread); <summary>`: each pass's round count, and the findings the rounds reported. <j> counts the 🔴 and 🟡 findings a reviewer marked `[misread]`."""
|
||||
"""The titles of larv-review's round commits, `Answer <pass> round <n>: <k> 🔴, <m> 🟡 (<j> misread); <summary>`: each pass's round count against its cap, and the findings the rounds reported. <j> counts the 🔴 and 🟡 findings a reviewer marked `[misread]`."""
|
||||
import dataclasses
|
||||
import re
|
||||
|
||||
from review_passes import PASSES
|
||||
|
||||
# The most rounds a pass takes before the run asks the human, per `docs/decisions.md` → `review: a pass gets at most three rounds per chunk; …`. rule-hook's `round-cap` refuses a round title past it, and `larv-review round-count` marks a pass at or past it.
|
||||
CAP = 3
|
||||
# The most rounds a pass takes per chunk, per `docs/decisions.md` → `review: a pass gets at most two rounds per chunk, …`. rule-hook's `round-cap` refuses a round commit that `past_cap` finds takes its pass past it, and `larv-review round-count` marks a pass that reached it.
|
||||
CAP = 2
|
||||
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):", re.IGNORECASE)
|
||||
# What follows a round title's colon: the round's 🔴 and 🟡 counts, then how many of them a reviewer marked `[misread]`, a count that titles from before it lack, then `;` and its summary, if any.
|
||||
FINDINGS = re.compile(r" ([0-9]+) 🔴, ([0-9]+) 🟡(?: \(([0-9]+) misread\))?(?:;|$)")
|
||||
@@ -36,10 +36,10 @@ def lacks_counts(title: str) -> bool:
|
||||
return ROUND.match(title) is not None and ((counts := title_counts(title)) is None or counts.misread is None)
|
||||
|
||||
|
||||
def past_cap(title: str) -> bool:
|
||||
"""Whether title is a round title numbered past CAP."""
|
||||
def past_cap(title: str, earlier: list[str]) -> bool:
|
||||
"""Whether title is a round title that takes its pass past CAP, as round_counts counts title among the branch's earlier commit titles."""
|
||||
named = ROUND.match(title)
|
||||
return named is not None and int(named.group(2)) > CAP
|
||||
return named is not None and round_counts([*earlier, title])[named.group(1).lower()] > CAP
|
||||
|
||||
|
||||
def title_counts(title: str) -> Findings | None:
|
||||
|
||||
@@ -48,6 +48,7 @@ from failure import Failure
|
||||
from rule_hook.override_log import Log, log_path, logged, print_overrides, write_override
|
||||
from rule_hook.reviewer_report import report_breaches
|
||||
from rule_hook.reviewer_rung import reviewer_breaches
|
||||
from rule_hook.round_cap import round_cap_breaches
|
||||
from shell_command import simple_commands
|
||||
from rule_hook.todo_check import todo_breaches
|
||||
|
||||
@@ -111,7 +112,7 @@ def answer(checked: Checked, what: str, now: float) -> int:
|
||||
|
||||
def bash_breaches(command: str, cwd: str, home: str) -> list[Breach]:
|
||||
simples = simple_commands(command, cwd, home)
|
||||
return command_breaches(command, simples, home) + [breach for commit in commits(simples, home) for breach in todo_breaches(commit.directory)]
|
||||
return command_breaches(command, simples, home) + [breach for commit in commits(simples, home) for breach in round_cap_breaches(commit) + todo_breaches(commit.directory)]
|
||||
|
||||
|
||||
def read_breaches(call: Call, home: str) -> list[Breach]:
|
||||
|
||||
@@ -6,6 +6,7 @@ import sys
|
||||
import typing
|
||||
|
||||
from rule_hook.rules import Breach
|
||||
from git_repo import default_ref
|
||||
from rule_hook.git import existing, git
|
||||
from failure import Failure
|
||||
from utc_stamp import utc
|
||||
@@ -100,10 +101,10 @@ def print_overrides(directory: str) -> int:
|
||||
if unread:
|
||||
print(f"rule-hook overrides cannot read the branch's reflog, so it cannot tell which overrides are this chunk's: {unread.reason}", file=sys.stderr)
|
||||
return 1
|
||||
default = local_default_branch(directory)
|
||||
default = default_ref(directory, guarded=True)
|
||||
log = pathlib.Path(common) / LOG
|
||||
entries = [parsed_entry(line) for line in log.read_text(encoding="utf-8").splitlines()] if log.is_file() else []
|
||||
on_default = (lambda commit: holds(directory, default, commit)) if default else (lambda commit: False)
|
||||
on_default = (lambda commit: False) if isinstance(default, Failure) else (lambda commit: holds(directory, default, commit))
|
||||
for entry in entries:
|
||||
if entry and this_chunks(entry, set(str(reflog).split()), branch, on_default):
|
||||
print(entry.line(), end="")
|
||||
@@ -119,9 +120,5 @@ def unread_reflog(reflog: str | Failure, head: str | Failure) -> Failure | None:
|
||||
return None if head in reflog.split() else Failure("it does not hold HEAD")
|
||||
|
||||
|
||||
def local_default_branch(directory: str) -> str | None:
|
||||
return next((name for name in ("origin/HEAD", "main", "master") if not isinstance(git(directory, "rev-parse", "--verify", "--quiet", name), Failure)), None)
|
||||
|
||||
|
||||
def holds(directory: str, descendant: str, ancestor: str) -> bool:
|
||||
return bool(ancestor) and not isinstance(git(directory, "merge-base", "--is-ancestor", ancestor, descendant), Failure)
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
from rule_hook.rules import Breach
|
||||
from round_titles import CAP, past_cap, ROUND, round_counts
|
||||
from git_command import Commit
|
||||
from failure import Failure
|
||||
import git_repo
|
||||
|
||||
|
||||
def round_cap_breaches(commit: Commit) -> list[Breach]:
|
||||
"""A breach where the commit's title takes its pass past the cap, counted with the branch's earlier round commits."""
|
||||
if not any(ROUND.match(message) for message in commit.messages):
|
||||
return []
|
||||
earlier = earlier_titles(commit.directory, commit.amends)
|
||||
refused = [(named.group(1).lower(), message) for message in commit.messages if (named := ROUND.match(message)) and past_cap(message, earlier)]
|
||||
return [Breach("round-cap", refusal(name, message, earlier), commit.directory) for name, message in refused]
|
||||
|
||||
|
||||
def refusal(name: str, title: str, earlier: list[str]) -> str:
|
||||
"""Why title, a round title of the pass name, is refused, listing the pass's round commits counted, and how to commit the fixes in hand."""
|
||||
counted = [earlier_title for earlier_title in earlier if (named := ROUND.match(earlier_title)) and named.group(1).lower() == name]
|
||||
listed = ", ".join(f"`{counted_title}`" for counted_title in counted) or "none found"
|
||||
return f"this would be {name}'s round {round_counts([*counted, title])[name]}, past its {CAP}; {name}'s round commits on this branch: {listed}. Fold the fixes into its last round commit with `git commit --amend` where that is the newest commit, or commit them under a title that doesn't open `Answer <pass> round`; they merge once the gate passes"
|
||||
|
||||
|
||||
def earlier_titles(directory: str, amends: bool) -> list[str]:
|
||||
"""The branch's titles, less the one an --amend replaces; none where git cannot list them within 5 seconds or the checkout records no default branch for origin, so the title's own number still counts.
|
||||
|
||||
Run guarded, as the hook runs git, which neither fetches nor asks origin: the default branch is as last fetched."""
|
||||
default = git_repo.default_ref(directory, guarded=True)
|
||||
titles = default if isinstance(default, Failure) else git_repo.branch_titles(default, directory, timeout=5, guarded=True)
|
||||
if isinstance(titles, Failure):
|
||||
return []
|
||||
return titles[1:] if amends else titles
|
||||
@@ -3,7 +3,7 @@ import posixpath
|
||||
import re
|
||||
import typing
|
||||
|
||||
from round_titles import CAP, lacks_counts, past_cap
|
||||
from round_titles import CAP, lacks_counts
|
||||
from git_command import commits
|
||||
from rule_hook.heavy_command import heavy
|
||||
from shell_command import launched, Simple, tool_name
|
||||
@@ -64,9 +64,11 @@ RULES: dict[RuleName, Rule] = {
|
||||
"an Agent call, in a claude loop-claude started, whose subagent type is an agent under agents/, or whose prompt opens with seat-prompts.md's opening sentence or the one larv-review's Security section sets",
|
||||
overridable=False,
|
||||
),
|
||||
# The cap holds with no ask, so no marker waives it.
|
||||
"round-cap": Rule(
|
||||
f"a larv-review pass takes at most {CAP} rounds per chunk, so a commit titled `Answer <pass> round <n>:` with <n> past {CAP} waits for the human's yes, which the override reason quotes",
|
||||
"a git commit whose message opens `Answer <pass> round <n>:`",
|
||||
f"a larv-review pass takes at most {CAP} rounds per chunk, and none after them: the fixes answering its last round merge once the gate passes",
|
||||
"a git commit whose message opens `Answer <pass> round <n>:`, counted with the pass's round commits on the branch since the default branch as last fetched, less the one an --amend replaces",
|
||||
overridable=False,
|
||||
),
|
||||
"round-title-counts": Rule(
|
||||
"a larv-review round's commit title gives the counts of 🔴 and 🟡 findings the round reported, and how many of them a reviewer marked `[misread]`, at most the two together, such as `Answer stability round 1: 1 🔴, 2 🟡 (1 misread); flag a deleted test`, since score-outcomes counts them",
|
||||
@@ -150,8 +152,6 @@ def command_breaches(command: str, simples: list[Simple], home: str) -> list[Bre
|
||||
breaches.append(Breach("commit-one-line", "pass the message as one -m '<line>', not from a file", commit.directory))
|
||||
elif len(commit.messages) > 1 or any("\n" in message.strip() for message in commit.messages):
|
||||
breaches.append(Breach("commit-one-line", "pass the message as one -m '<line>', with no heredoc, second -m or newline", commit.directory))
|
||||
if any(past_cap(message) for message in commit.messages):
|
||||
breaches.append(Breach("round-cap", f"the pass has had its {CAP} rounds; ask the human, and quote the yes in the override", commit.directory))
|
||||
if any(lacks_counts(message) for message in commit.messages):
|
||||
breaches.append(Breach("round-title-counts", "open the summary after the round's colon with `<k> 🔴, <m> 🟡 (<j> misread);`, where <j> is at most <k> + <m>", commit.directory))
|
||||
return breaches
|
||||
|
||||
@@ -51,21 +51,25 @@ def git(cwd: str, *args: str) -> str:
|
||||
|
||||
class Commits(unittest.TestCase):
|
||||
def test_cd_and_dash_capital_c_move_the_commit(self):
|
||||
self.assertEqual(committed("cd a && git -C ~/b commit -am 'x'"), [Commit("/h/b", ["x"], False)])
|
||||
self.assertEqual(committed("cd a && git -C ~/b commit -am 'x'"), [Commit("/h/b", ["x"], False, False)])
|
||||
self.assertEqual(committed("cd a && git -c x=y -C b commit -m x")[0].directory, "/r/a/b")
|
||||
|
||||
def test_each_message_spelling_is_read(self):
|
||||
self.assertEqual(commit_message(["--message=a", "-m", "b=c", "--message", "c", "-mz", "-am", "d"]), (["a", "b=c", "c", "z", "d"], False))
|
||||
self.assertEqual(commit_message(["--message=a", "-m", "b=c", "--message", "c", "-mz", "-am", "d"]), (["a", "b=c", "c", "z", "d"], False, False))
|
||||
|
||||
def test_an_amend_is_seen(self):
|
||||
self.assertEqual([commit.amends for commit in committed("git commit --amend -m x && git commit -m y")], [True, False])
|
||||
self.assertEqual(commit_message(["--", "--amend"]), ([], False, False), "after --, --amend names a path")
|
||||
|
||||
def test_a_file_message_is_seen(self):
|
||||
self.assertEqual(commit_message(["-F", "-"]), ([], True))
|
||||
self.assertEqual(commit_message(["--file=x"]), ([], True))
|
||||
self.assertEqual(commit_message(["-F", "-"]), ([], True, False))
|
||||
self.assertEqual(commit_message(["--file=x"]), ([], True, False))
|
||||
|
||||
def test_other_git_commands_are_no_commit(self):
|
||||
self.assertEqual(committed("git log --grep commit; echo git commit"), [])
|
||||
|
||||
def test_a_line_break_or_a_comment_separates_commands(self):
|
||||
self.assertEqual(committed("cd a # go there\nGIT_EDITOR=true git commit -m x"), [Commit("/r/a", ["x"], False)])
|
||||
self.assertEqual(committed("cd a # go there\nGIT_EDITOR=true git commit -m x"), [Commit("/r/a", ["x"], False, False)])
|
||||
|
||||
def test_a_commit_after_a_heredoc_or_a_blank_line_is_seen(self):
|
||||
self.assertEqual(len(committed("cat <<EOF > notes\nhello\nEOF\ngit commit -m a")), 1)
|
||||
@@ -81,15 +85,15 @@ class Commits(unittest.TestCase):
|
||||
self.assertEqual(segments("echo hi > f arg"), [["echo", "hi", "arg"]])
|
||||
for redirect in [">", ">>", "<", "<<<", "<>", ">&", "<&", ">|", "&>", "&>>"]:
|
||||
with self.subTest(redirect):
|
||||
self.assertEqual(committed(f"git {redirect} /tmp/x commit -m hi"), [Commit("/r", ["hi"], False)])
|
||||
self.assertEqual(committed("git commit >&2 -m hi 2>&1"), [Commit("/r", ["hi"], False)])
|
||||
self.assertEqual(committed("> out git commit -m hi"), [Commit("/r", ["hi"], False)])
|
||||
self.assertEqual(committed(f"git {redirect} /tmp/x commit -m hi"), [Commit("/r", ["hi"], False, False)])
|
||||
self.assertEqual(committed("git commit >&2 -m hi 2>&1"), [Commit("/r", ["hi"], False, False)])
|
||||
self.assertEqual(committed("> out git commit -m hi"), [Commit("/r", ["hi"], False, False)])
|
||||
|
||||
def test_a_separator_glued_to_a_redirect_still_ends_the_command(self):
|
||||
self.assertEqual(segments("echo a > f; echo b | cat &> g\nls"), [["echo", "a"], ["echo", "b"], ["cat"], ["ls"]])
|
||||
for command in ["echo a &</dev/null git commit -m hi", "echo a;>f git commit -m hi"]:
|
||||
with self.subTest(command):
|
||||
self.assertEqual(committed(command), [Commit("/r", ["hi"], False)])
|
||||
self.assertEqual(committed(command), [Commit("/r", ["hi"], False, False)])
|
||||
|
||||
|
||||
class CommandBreaches(unittest.TestCase):
|
||||
@@ -128,18 +132,6 @@ class CommandBreaches(unittest.TestCase):
|
||||
with self.subTest(command):
|
||||
self.assertEqual(checked(command), [])
|
||||
|
||||
def test_a_round_title_past_round_3_breaks_round_cap(self):
|
||||
self.assertEqual(checked("git commit -m 'Answer stability round 4: 0 🔴, 1 🟡 (0 misread); fix the parser'"), ["round-cap"])
|
||||
self.assertEqual(checked("git commit -m 'Answer stability round 3: 0 🔴, 1 🟡 (0 misread); fix the parser'"), [])
|
||||
|
||||
def test_a_round_cap_breach_names_the_commit_s_checkout(self):
|
||||
command = "git -C /x commit -m 'Answer stability round 4: 0 🔴, 1 🟡 (0 misread); fix'"
|
||||
self.assertEqual([(breach.rule, breach.directory) for breach in command_breaches(command, simple_commands(command, "/r", "/h"), "/h")], [("round-cap", "/x")])
|
||||
|
||||
def test_an_override_quoting_the_human_s_yes_lets_a_round_past_the_cap_through(self):
|
||||
command = "git commit -m 'Answer stability round 4: 0 🔴, 1 🟡 (0 misread); fix' # rule-override round-cap: the human answered 'yes, one more'"
|
||||
self.assertFalse(check(command_breaches(command, simple_commands(command, "/r", "/h"), "/h"), command, False, True).blocked)
|
||||
|
||||
def test_an_attribution_breaks_only_in_a_git_gh_or_tea_command(self):
|
||||
self.assertEqual(checked(f"tea pr create --description '{TRAILER}'"), ["no-ai-attribution"])
|
||||
self.assertEqual(checked(f"grep -rn '{TRAILER}' ."), [])
|
||||
@@ -664,6 +656,54 @@ class Hook(unittest.TestCase):
|
||||
call = {"cwd": self.repo, "tool_input": {"file_path": f"{self.repo}/.gitignore", "new_string": "d", "old_string": "a"}, "tool_name": "Edit"}
|
||||
self.assertEqual(hook(call, self.repo).returncode, 2)
|
||||
|
||||
def round_commit(self, title: str, given: str = "") -> subprocess.CompletedProcess:
|
||||
return hook({"cwd": self.repo, "tool_input": {"command": f"git commit {given}-m '{title}'"}, "tool_name": "Bash"}, self.repo)
|
||||
|
||||
def test_a_round_commit_past_round_2_is_refused_and_offers_no_override(self):
|
||||
ran = self.round_commit("Answer stability round 3: 0 🔴, 1 🟡 (0 misread); fix # rule-override round-cap: the human said yes")
|
||||
self.assertEqual(ran.returncode, 2)
|
||||
self.assertIn("Blocked by rule round-cap: this would be stability's round 3, past its 2", ran.stderr)
|
||||
self.assertNotIn("rule-override", ran.stderr)
|
||||
self.assertEqual(self.round_commit("Answer stability round 2: 0 🔴, 1 🟡 (0 misread); fix").returncode, 0)
|
||||
|
||||
def test_a_third_round_commit_of_a_pass_on_the_branch_is_refused_whatever_its_number(self):
|
||||
for title in ["Answer stability round 1: 1 🔴, 0 🟡 (0 misread); a", "Answer prose round 1: 0 🔴, 1 🟡 (0 misread); b", "Answer stability round 2: 0 🔴, 1 🟡 (0 misread); c"]:
|
||||
git(self.repo, "commit", "-q", "--allow-empty", "-m", title)
|
||||
ran = self.round_commit("Answer stability round 2: 0 🔴, 0 🟡 (0 misread); d")
|
||||
self.assertEqual(ran.returncode, 2)
|
||||
self.assertIn("stability's round commits on this branch: `Answer stability round 2: 0 🔴, 1 🟡 (0 misread); c`, `Answer stability round 1: 1 🔴, 0 🟡 (0 misread); a`", ran.stderr)
|
||||
self.assertIn("`git commit --amend` where that is the newest commit", ran.stderr)
|
||||
self.assertEqual(self.round_commit("Answer prose round 2: 0 🔴, 0 🟡 (0 misread); d").returncode, 0)
|
||||
self.assertEqual(self.round_commit("Answer stability round 2: 0 🔴, 1 🟡 (0 misread); c, reworded", "--amend --allow-empty ").returncode, 0, "an amend replaces the commit it counts")
|
||||
git(self.repo, "commit", "-q", "--allow-empty", "-m", "fix a typo")
|
||||
self.assertEqual(self.round_commit("Answer stability round 2: 0 🔴, 1 🟡 (0 misread); e", "--amend --allow-empty ").returncode, 2, "an amend replaces the newest commit, not the oldest")
|
||||
|
||||
def test_a_stalled_origin_holds_up_neither_a_round_commit_nor_the_overrides_listing(self):
|
||||
git(self.repo, "config", "protocol.ext.allow", "always")
|
||||
git(self.repo, "remote", "add", "origin", "ext::sleep 30")
|
||||
self.assertEqual(self.round_commit("Answer stability round 1: 0 🔴, 1 🟡 (0 misread); a").returncode, 0)
|
||||
hook({"cwd": self.repo, "tool_input": {"command": "git commit -m a -m b # rule-override commit-one-line: a body is needed"}, "tool_name": "Bash"}, self.repo)
|
||||
listed = subprocess.run([str(HOOK), "overrides"], capture_output=True, check=True, cwd=self.repo, env=isolated_env(), text=True, timeout=10).stdout
|
||||
self.assertIn("\tcommit-one-line\ta body is needed\t", listed)
|
||||
|
||||
def test_round_commits_merged_in_from_the_default_branch_do_not_count(self):
|
||||
git(self.repo, "checkout", "-q", "main")
|
||||
for title in ["Answer stability round 1: 0 🔴, 1 🟡 (0 misread); a", "Answer stability round 2: 0 🔴, 1 🟡 (0 misread); b"]:
|
||||
git(self.repo, "commit", "-q", "--allow-empty", "-m", title)
|
||||
git(self.repo, "checkout", "-q", "chunk")
|
||||
git(self.repo, "merge", "-q", "--no-ff", "-m", "Merge main", "main")
|
||||
git(self.repo, "branch", "-f", "main", "main~2")
|
||||
self.assertEqual(self.round_commit("Answer stability round 1: 0 🔴, 1 🟡 (0 misread); c").returncode, 0, "a stale default branch still leaves another chunk's rounds out")
|
||||
|
||||
def test_a_stale_local_main_does_not_stand_in_for_a_default_branch_origin_left_unrecorded(self):
|
||||
git(self.repo, "checkout", "-q", "main")
|
||||
for title in ["Answer stability round 1: 0 🔴, 1 🟡 (0 misread); a", "Answer stability round 2: 0 🔴, 1 🟡 (0 misread); b"]:
|
||||
git(self.repo, "commit", "-q", "--allow-empty", "-m", title)
|
||||
git(self.repo, "checkout", "-q", "-B", "chunk", "main")
|
||||
git(self.repo, "branch", "-f", "main", "main~2")
|
||||
git(self.repo, "remote", "add", "origin", f"{self.repo}/gone.git")
|
||||
self.assertEqual(self.round_commit("Answer stability round 1: 0 🔴, 1 🟡 (0 misread); c").returncode, 0, "another chunk's rounds, fast-forwarded into the default branch, are not this chunk's")
|
||||
|
||||
def test_a_commit_while_an_unranked_todo_has_changes_is_blocked(self):
|
||||
pathlib.Path(self.repo, "docs").mkdir()
|
||||
for name, content in [("README.md", README), ("docs/decisions.md", DECISIONS), ("todo.md", TODO)]:
|
||||
|
||||
@@ -87,6 +87,11 @@ class StalledOrigin(unittest.TestCase):
|
||||
self.assertIsInstance(ref, Failure)
|
||||
self.assertIn("stopped after 0.5 s", ref.reason)
|
||||
|
||||
def test_reading_the_default_branch_guarded_asks_no_one_and_fails(self):
|
||||
ref = git_repo.default_ref(self.root, 0.5, guarded=True)
|
||||
self.assertIsInstance(ref, Failure)
|
||||
self.assertIn("origin records no default branch", ref.reason)
|
||||
|
||||
def test_fetching_the_default_branch_fails_after_the_timeout(self):
|
||||
with contextlib.chdir(self.root):
|
||||
found = git_repo.default_branch(stale_ok=True, timeout=0.5)
|
||||
|
||||
@@ -23,11 +23,16 @@ class Rounds(unittest.TestCase):
|
||||
|
||||
|
||||
class Cap(unittest.TestCase):
|
||||
def test_finds_a_round_title_past_round_3(self):
|
||||
self.assertTrue(past_cap("Answer stability round 4: 0 🔴, 1 🟡; fix"))
|
||||
self.assertTrue(past_cap("answer full scan round 10: x"))
|
||||
self.assertFalse(past_cap("Answer stability round 3: 0 🔴, 1 🟡; fix"))
|
||||
self.assertFalse(past_cap("Answer the 4 questions"))
|
||||
def test_finds_a_round_title_past_round_2(self):
|
||||
self.assertTrue(past_cap("Answer stability round 3: 0 🔴, 1 🟡; fix", []))
|
||||
self.assertTrue(past_cap("answer full scan round 10: x", []))
|
||||
self.assertFalse(past_cap("Answer stability round 2: 0 🔴, 1 🟡; fix", []))
|
||||
self.assertFalse(past_cap("Answer the 3 questions", []))
|
||||
|
||||
def test_counts_the_pass_s_earlier_round_commits(self):
|
||||
self.assertTrue(past_cap("Answer stability round 2: x", ["Answer stability round 2: y", "Answer stability round 1: z"]), "a third commit, though titled round 2")
|
||||
self.assertTrue(past_cap("Answer stability round 1: x", ["Answer Stability round 2: y", "answer stability round 1: z"]), "case is ignored")
|
||||
self.assertFalse(past_cap("Answer stability round 2: x", ["Answer prose round 1: y", "Answer principles round 2: z", "Answer stability round 1: w"]))
|
||||
|
||||
|
||||
class RoundFindings(unittest.TestCase):
|
||||
|
||||
@@ -2848,3 +2848,27 @@ Junior A, decided by:
|
||||
- Locality: `bin/rule_hook/repeat_read.py:29` `repeat_breaches`: needs lines through one past the last shown and a mtime formatted as a transcript stamp, both set up by main.py
|
||||
- Locality, Shape: `bin/routing_figures/measure.py:205` `results`: one line over the shared `tool_result`, where cde7720 held a private copy of the parsing
|
||||
- Shape: `bin/rule_hook/repeat_read.py:87` `results_on`: an unnamed four-field tuple unpacked by position, and `written` names a path here but the override text in main.py
|
||||
|
||||
## 2026-10-28T17:21:57Z, PR #390 at 3ef4790, against 47a4075
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | better | better | same | same | better |
|
||||
| Inherited architect | same | better | same | better | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Navigation, Locality, Overall: `bin/rule_hook/round_cap.py:8` `round_cap_breaches`: a refusal by round-cap is traced to a module named after the rule, wired like `todo_check.py`, where the base mixed it into `command_breaches`
|
||||
- Locality, Overall: `bin/git_repo.py:81` `branch_titles`: round-count and the hook read the branch's round titles through one function, so they agree by construction
|
||||
- Locality: `bin/rule_hook/override_log.py:104` `print_overrides`: the second definition of the default branch, `local_default_branch`, is gone
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality, Self-sufficiency, Overall: `bin/git_repo.py:81` `branch_titles`: the hook and round-count now count rounds one way, with first parents, where the base's two rules disagreed
|
||||
- Locality, Overall: `bin/rule_hook/override_log.py:104` `print_overrides`: one default-branch lookup instead of a quietly different local one
|
||||
- Self-sufficiency, Overall: `bin/rule_hook/round_cap.py:24` `earlier_titles`: the docstring states both fail-open cases, and the refusal names the counted commits and both ways out
|
||||
- Self-sufficiency: `skills/larv-review/SKILL.md:78` "Cap": the ask, yes, one-more-round loop is gone, though the bullet now packs four ideas
|
||||
- Locality: `bin/git_repo.py:63` `default_ref`: `guarded=True` means both safe git options and never asking origin, kept at each call site; where it rots first
|
||||
- Locality: `bin/rule_hook/main.py:113` `bash_breaches`: round-cap sits apart from its sibling round-title-counts in `command_breaches`, a split no comment states
|
||||
|
||||
+8
-8
@@ -145,19 +145,19 @@ Valid while no published standard sets either number, the reading-cost window st
|
||||
|
||||
2026-10-08, lilleman. Serves goal 1.4: run beside the full scan, `/security-review` started seconds apart from it each round and returned a near-identical report. Valid while the full scan covers what `/security-review` reports.
|
||||
|
||||
## review: a fix for a 🔴 or 🟡 reruns the pass that found it and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟢, a ❓ or a prose finding reruns none
|
||||
## review: a fix for a 🔴 or 🟡 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟢, a ❓ or a prose finding reruns none
|
||||
|
||||
2026-10-08, lilleman. Serves goal 1.4. An audit of the four costliest archived chunks found nearly every finding that mattered in a pass's first round; when every fix, a nit's included, re-entered review, later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes keep changing nothing a user would notice.
|
||||
|
||||
2026-10-13, lilleman through todo item 97. Serves goals 1.4 and 1.5. From round 2 a pass reviews only how the chunk answered its previous round, each fix and each dismissal with its reason, and reports only one that is plainly wrong or worse than the change it answered. A fix from another pass that re-enters it is still reviewed as a first round would review it, limited to its lines. Reviewing all that changed had a reviewer find new gaps in each round's fixes, so a pass ran to its cap. Valid while round-2 findings outside a round's answers keep changing nothing a user would notice.
|
||||
|
||||
## review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human
|
||||
## review: a pass gets at most two rounds per chunk, with no third and no ask for one; the fixes answering round 2 merge once the gate passes
|
||||
|
||||
2026-09-28, lilleman. Serves goal 1. One counter per pass per chunk, so no fix bounces between two passes without end. Valid while one human answers the ask, in the session or from the unmerged PR.
|
||||
2026-10-28, lilleman through todo item 246. Serves goals 1 and 1.4. One counter per pass per chunk, so no fix bounces between two passes without end. lilleman measured every repo's round titles since 2026-10-21. Round 1 ran 357 times and found 59 🔴. Round 2 ran about 200 times and found 12 🔴, mostly in round 1's fixes. Rounds 3 and later ran about 110 times and found no 🔴, at about $125 of review at list prices, and asks for a fourth or fifth round were 24 of the 59 interrupts. The cap had stood at three because a lower cap would ask the human more often; once nothing asks, that reason is gone. A rerun one rung up belongs to the round it reruns, so it doesn't count toward the cap: `review: a reviewer or comprehension seat that rules the work below standard`.
|
||||
|
||||
The cap stays at three, lilleman ruled through todo item 97: a lower cap would ask the human more often, against goal 1, and a third round reviews only the answers to round 2's findings, so it costs little. A rerun one rung up belongs to the round it reruns, so it counts toward no cap: `review: a reviewer or comprehension seat that rules the work below standard`.
|
||||
rule-hook's `round-cap` refuses a round commit that would take its pass past round 2, and no override waives it. It counts the pass's round commits on the branch as `larv-review round-count` does, so a third commit titled round 2 is refused too. The hook must not fetch, so it lists the branch's commits since the default branch as last fetched. It follows first parents, so the round commits that a merge of a newer default branch brought in don't count as this chunk's. It leaves out the commit an `--amend` replaces. Where git cannot list the branch's commits, within 5 seconds or because origin records no default branch, it counts the title's own number alone, so a slow or unset lookup doesn't block the commit.
|
||||
|
||||
rule-hook's `round-cap` refuses a round commit titled past round 3 unless its override reason quotes the human's yes (lilleman, todo item 214). It reads only the title's number, so a fourth commit titled round 3 passes it, while `larv-review round-count` marks that pass past the cap; `todo.md` item 220 closes the gap.
|
||||
Valid while the defects that later commits fix in a finished chunk's code, which `todo.md` item 247 counts, don't rise per chunk.
|
||||
|
||||
## review: a pass skips only on a precedent: a decision, or a plan item crediting the pass, that the repo held before the chunk began
|
||||
|
||||
@@ -187,7 +187,7 @@ Valid while the ladder orders its rungs by competence.
|
||||
|
||||
## review: an ask no user answers leaves the PR unmerged
|
||||
|
||||
2026-09-30, claude. Serves goal 1: the round cap, a BLOCK's restructure, Stage 0's questions and a goal conflict are the human's call, and an unmerged PR is what brings it to them; merging would make the call for them. Where an option reviews more — run the pass, fix a finding no goal lets the run decline — the run takes it; a ❓, or a fix changing a goal or a decision, is filed in `todo.md`, which reaches the human without holding the chunk. Valid while a human reads every unmerged PR an unattended run leaves.
|
||||
2026-09-30, claude. Serves goal 1: a BLOCK's restructure, Stage 0's questions and a goal conflict are the human's call, and an unmerged PR is what brings it to them; merging would make the call for them. Where an option reviews more — run the pass, fix a finding no goal lets the run decline — the run takes it; a ❓, or a fix changing a goal or a decision, is filed in `todo.md`, which reaches the human without holding the chunk. Valid while a human reads every unmerged PR an unattended run leaves.
|
||||
|
||||
## review: each reviewer under `agents/` writes its full report into the chunk's review folder under `~/.local/state/larv-review/` and returns the worker its verdict and one line per finding; rule-hook refuses its Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`
|
||||
|
||||
@@ -353,7 +353,7 @@ Valid while a model switch inside a session discards its prompt cache.
|
||||
|
||||
## rule-hook: a PreToolUse hook enforces each rule a program can check, a `# rule-override <rule>: <reason>` marker overrides one, and larv-review's overrides pass rules on each override before merge
|
||||
|
||||
2026-10-12, lilleman. Serves goals 1.2 and 1.3. Anthropic's best-practices page calls `CLAUDE.md` advisory and hooks deterministic, so a rule that must hold every time moves into `bin/rule-hook`, and `CLAUDE.md` keeps it only where the hook cannot check it. A Claude Code hook ties the check to one vendor ("Prefer the vendor-neutral option", `principles/technical-principles.md`), which is the target itself here: this repo configures Claude Code. What the tie buys: the hook blocks before a write lands, Edit and Write calls included, in every project with no per-repo install; a git `commit-msg` hook sees only commits, and CI sees them only after the push. The hook checks each write, so the agent fixes the line it just wrote; checking the staged diff at commit time would also cover files a Bash command writes, at the cost of later feedback. The marker's spelling was picked by a panel of three fresh readers, two to one. Each override is logged in the checkout's git directory, which is never pushed (goal 4). The plan had loop-claude's judge to rule on overrides inside a loop; lilleman approved larv-review's overrides pass in every run, because it rules before merge and an outrageous override still reaches the human through its ask. 2026-10-16, lilleman, answering plan item 166: `reviewer-rung` is the one rule no marker overrides; the comment above it in `bin/rule_hook/rules.py` says why. Valid while a PreToolUse hook that exits 2 blocks the call and hands its stderr to the model, and while an agent commits through Claude Code's Bash tool.
|
||||
2026-10-12, lilleman. Serves goals 1.2 and 1.3. Anthropic's best-practices page calls `CLAUDE.md` advisory and hooks deterministic, so a rule that must hold every time moves into `bin/rule-hook`, and `CLAUDE.md` keeps it only where the hook cannot check it. A Claude Code hook ties the check to one vendor ("Prefer the vendor-neutral option", `principles/technical-principles.md`), which is the target itself here: this repo configures Claude Code. What the tie buys: the hook blocks before a write lands, Edit and Write calls included, in every project with no per-repo install; a git `commit-msg` hook sees only commits, and CI sees them only after the push. The hook checks each write, so the agent fixes the line it just wrote; checking the staged diff at commit time would also cover files a Bash command writes, at the cost of later feedback. The marker's spelling was picked by a panel of three fresh readers, two to one. Each override is logged in the checkout's git directory, which is never pushed (goal 4). The plan had loop-claude's judge to rule on overrides inside a loop; lilleman approved larv-review's overrides pass in every run, because it rules before merge and an outrageous override still reaches the human through its ask. 2026-10-16, lilleman, answering plan item 166: no marker overrides `reviewer-rung`; the comment above it in `bin/rule_hook/rules.py` says why. Valid while a PreToolUse hook that exits 2 blocks the call and hands its stderr to the model, and while an agent commits through Claude Code's Bash tool.
|
||||
|
||||
## rules/: `CLAUDE.md` holds the worker's rules that no file path can scope, and each topic's rules for the worker live in a `rules/` file that Claude Code loads by its `paths:` frontmatter
|
||||
|
||||
@@ -433,7 +433,7 @@ A finding against a principle names it, so the worker reads only that one when i
|
||||
|
||||
The item also cut the rule that a session proposes new rules for `CLAUDE.md` and `rules/` and restructures them after each change: changes to these rules go through this repo's chunks and their reviews, like any other change.
|
||||
|
||||
Valid while review rounds per pass do not rise. A principle found late, such as functional core, can cost a rewrite within a pass's three rounds. `todo.md` item 105 compares rounds per pass and reading cost in loop-claude's archive for the five chunks before and after this one.
|
||||
Valid while review rounds per pass do not rise. A principle found late, such as functional core, can cost a rewrite within a pass's two rounds. `todo.md` item 105 compares rounds per pass and reading cost in loop-claude's archive for the five chunks before and after this one.
|
||||
|
||||
## workflow-bench/: the remote may show the bench cases' names and commit SHAs
|
||||
|
||||
|
||||
@@ -895,3 +895,7 @@ Read most (opens and searches):
|
||||
Read most (opens and searches):
|
||||
|
||||
- `rules/decisions.md` 1
|
||||
|
||||
## Reading cost, chunk c49a63a648d0, finished 2026-10-28T16:32:11Z
|
||||
|
||||
118558 new input tokens before the first commit.
|
||||
|
||||
@@ -71,14 +71,13 @@ Give a reviewer without Bash the branch diff and `git ls-files`'s output in its
|
||||
|
||||
**Round cap.**
|
||||
|
||||
- **Round.** A round is one review by a pass's reviewer: the first, each re-entry, and the rerun after the architect returns Stage 0 questions ("Architecture" below). A relaunch one rung up ("Rung up" below) stays in its round.
|
||||
- **Auto-reviewer.** The external auto-reviewer (§4) is a pass of its own, each review it posts a round. Its count is the number of its reviews on the PR, less its reviews of a `Record the comprehension check:` commit, and `round-count` doesn't list it. A commit that answers one of its rounds takes an ordinary title, not an `Answer <pass> round` one, and a round that changed nothing gets no commit.
|
||||
- **Round.** A round is one review by a pass's reviewer: the first and each re-entry. The rerun after the architect returns Stage 0 questions ("Architecture" below) and a relaunch one rung up ("Rung up" below) stay in their round.
|
||||
- **Auto-reviewer.** The external auto-reviewer (§4) is a pass of its own, each review it posts, up to the cap, a round. Its count is the number of its reviews on the PR, less its reviews of a `Record the comprehension check:` commit, and `round-count` doesn't list it. A commit that answers one of its rounds takes an ordinary title, not an `Answer <pass> round` one, and a round that changed nothing gets no commit.
|
||||
- **Count.** Close each round of a pass other than the auto-reviewer with one commit that holds the round's fixes. Title it `Answer <pass> round <n>: <k> 🔴, <m> 🟡 (<j> misread); …`: `<pass>` is the pass's name in lower case, such as `full scan`, `<k>` and `<m>` are the 🔴 and 🟡 findings all the round's reviewers reported, and `<j>` is how many of those they marked `[misread]` (review-findings.md → "Misread"); a reviewer that marks none, such as `/security-review`, adds 0. Example: `Answer stability round 1: 1 🔴, 2 🟡 (1 misread); flag a deleted test`. A round that changed nothing gets an empty commit (`git commit --allow-empty`). Push the commits when §4 runs next, where there is a remote. `~/.claude/bin/larv-review round-count` prints each pass's count and marks a pass at the cap.
|
||||
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or settled in the PR summary per "Comprehension check" → "Places", or, for the prose pass, fixed. Resolve that round's 🟢s and ❓s per "Findings" too.
|
||||
- **Cap.** A pass takes at most three rounds per chunk. Instead of a fourth, list what is open and ask the user. For the auto-reviewer, ask before the push that would summon its fourth, and hold the fixes unpushed until the answer.
|
||||
- **Answer.** A yes grants that pass one more round, after which the ask repeats; a no leaves the PR unmerged. rule-hook refuses a round commit titled past round 3 unless its `round-cap` override quotes the yes.
|
||||
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or settled in the PR summary per "Comprehension check" → "Places", or, for the prose pass and a pass at its cap, fixed. Resolve that round's 🟢s and ❓s per "Findings" too.
|
||||
- **Cap.** A pass takes at most two rounds per chunk; don't ask the user for a third. Answer its second round's findings per "Findings": those fixes merge once the gate passes, unreviewed by that pass. rule-hook's `round-cap` refuses a round commit that would take a pass past two, and no override waives it. A review the auto-reviewer posts after its second isn't a round: fix none of it and rerun no pass for it, but file a security finding it raises per `~/.claude/CLAUDE.md` → "Working loop", and any other 🔴 in `todo.md`.
|
||||
|
||||
**Re-entry.** A fix is a change no pass has seen. A fix for a 🟢, a ❓ or a prose finding reruns no pass, whatever it changes. A round's fixes for 🔴 and 🟡 findings re-enter together: run every pass that a row below names for any of them, a done pass included, in the waves of "Waves", then §4. A pass at its cap gets "Round cap" → "Cap"'s ask instead.
|
||||
**Re-entry.** A fix is a change no pass has seen. A fix for a 🟢, a ❓ or a prose finding reruns no pass, whatever it changes. A round's fixes for 🔴 and 🟡 findings re-enter together: run every pass that a row below names for any of them, a done pass included, in the waves of "Waves", then §4. A pass at its cap doesn't run again ("Round cap" → "Cap").
|
||||
|
||||
| The fix | Reruns |
|
||||
|---------|--------|
|
||||
@@ -172,7 +171,7 @@ Launch prose-reviewer with what the diff touched and every section this review c
|
||||
4. Watch the forge's checks until every one passes. A red check is a finding.
|
||||
5. No CI configured → the gate is local tests only; note it in the summary.
|
||||
6. Everything green → mark the PR ready, dropping any `WIP:`/`Draft:` title prefix.
|
||||
7. **External auto-reviewer**, when the repo has one — its config file (`.coderabbit.yaml` and the like), or its comments on earlier PRs; none → skip it, say so. Marking the PR ready is what summons it. Wait up to **10 minutes** for its first review, checking about once a minute; nothing by then → note it in the summary and carry on, never waiting twice. Its rounds, §3 → "Round cap". Once it has reviewed, each later return here, after a fix re-entered per §3, waits up to **5 minutes** for its re-review of the new commit.
|
||||
7. **External auto-reviewer**, when the repo has one — its config file (`.coderabbit.yaml` and the like), or its comments on earlier PRs; none → skip it, say so. Marking the PR ready is what summons it. Wait up to **10 minutes** for its first review, checking about once a minute; nothing by then → note it in the summary and carry on, never waiting twice. Its rounds, §3 → "Round cap". Once it has reviewed, each later return here before its cap, after a fix re-entered per §3, waits up to **5 minutes** for its re-review of the new commit.
|
||||
|
||||
## 5. Goal alignment, then merge
|
||||
|
||||
|
||||
@@ -12,7 +12,6 @@
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 246 | 0.4.0 | | owner | **Give each review pass at most two rounds per chunk, with no third round and no ask for one: a fix answering round 2 merges once the gate passes.** | 3 | 2 | 3 | 6 | 3 | 1.2 by 5, 1.5 by 3, 1.4 by 2 | 10.0 |
|
||||
| 244 | 0.4.0 | | owner | **Open every line loop-claude prints to its terminal with the machine's local time, as in `2026-10-28 11:46:01`.** | 2 | 2 | 1 | 3 | 3 | 1.5 by 2 | 1.9 |
|
||||
| 247 | 0.4.0 | | owner | **Count each finished chunk's escaped defects, the later commits that fix code it shipped, in `score-outcomes` and the routing figures.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4 | 1.7 |
|
||||
| 243 | 0.4.0 | | owner | **Show every loop-claude on the machine that waits for the human in one terminal app, and let the human answer each one there, `/talk` included.** | 4 | 2 | 3 | 7 | 6 | 1.5 by 4 | 0.9 |
|
||||
@@ -64,7 +63,6 @@
|
||||
| 77 | 0.6.0 | | | **Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.** | 4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 3.2 |
|
||||
| 4 | 0.6.0 | | | **Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.** | 3 | 2 | 3 | 6 | 3 | 1.1 by 3 | 3.2 |
|
||||
| 187 | 0.6.0 | | | **Count the tokens a worker spends reading reviewers' reports, so goal 1.4's signal shows whether report files lowered what a review costs the worker's context.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 3.0 |
|
||||
| 220 | 0.6.0 | | decision | **Have rule-hook also count a pass's earlier round commits on the branch, so a fourth commit titled round 3 is refused too.** | 3 | 2 | 3 | 6 | 3 | 1.4 by 3 | 3.0 |
|
||||
| 149 | 0.6.0 | | decision | **Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.** | 4 | 2 | 3 | 7 | 5 | 1.1 by 7 | 3.0 |
|
||||
| 165 | 0.6.0 | | | **Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, 1.4 by 3 | 2.9 |
|
||||
| 151 | 0.6.0 | | decision | **Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes.** | 3 | 2 | 2 | 5 | 5 | 1.2 by 7 | 2.9 |
|
||||
@@ -82,15 +80,6 @@
|
||||
|
||||
## Details
|
||||
|
||||
### 246. Give each review pass at most two rounds per chunk, with no third round and no ask for one: a fix answering round 2 merges once the gate passes.
|
||||
|
||||
lilleman's call, 2026-10-28, from a measurement of every repo's round titles since 2026-10-21. Round 1 ran 357 times and found 59 🔴. Round 2 ran about 200 times and found 12 🔴, mostly in round 1's fixes. Rounds 3 and later ran about 110 times and found no 🔴, at about $125 of review at list prices. Asks for a fourth or fifth round were 24 of the 59 interrupts in that time. Until this ships, lilleman answered no to a fourth principles round in PR #387, due only to tests a stability fix added after principles approved the code twice, and had the chunk merge without it.
|
||||
|
||||
- Replace `docs/decisions.md` → `review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human` with the new cap, and update its title in `AGENTS.md`. Its reason for three, that a lower cap asks the human more often, no longer holds once nothing asks.
|
||||
- Change `skills/larv-review/SKILL.md` → "Round cap" → "Cap", and every place it says three rounds or a fourth: the auto-reviewer included.
|
||||
- Have rule-hook's `round-cap` refuse a round commit titled past round 2, with no override. Settle item 220 inside this change, since it guards the same cap; drop it if this change makes it moot.
|
||||
- `bin/round_titles.py:4` `CAP` and `bin/rule_hook/rules.py:55` `round-title-counts` are places the comprehension seats found hard: the cap lives in SKILL.md prose while the code only counts. Make them easier to read.
|
||||
|
||||
### 244. Open every line loop-claude prints to its terminal with the machine's local time, as in `2026-10-28 11:46:01`.
|
||||
|
||||
lilleman's call, 2026-10-28, who also chose the spelling: `YYYY-MM-DD HH:MM:SS` in the machine's local time, never UTC, so the human sees when each line happened, such as how long a question has waited. Today about 27 `print` calls across `bin/loop_claude/` write bare lines to stderr. Route them through one function that adds the stamp, taking the time from `Effects.now` so tests can fix it. A line of a multi-line message, such as a question, gets the stamp only on its first line.
|
||||
@@ -350,10 +339,6 @@ Today the folder is the SHA-256 of the transcript's first timestamped line, whic
|
||||
|
||||
Before PR #310, reports reached the worker as the Agent call's result, a median 19k tokens per chunk (2026-10-08, 68 archived transcripts). Now a reviewer returns its verdict and one line per finding, and the worker reads a 🔴 or 🟡 finding's entry from the report file. `reading-cost` counts only tracked files (`bin/reading_cost/measure.py`), and `routing-figures` counts only the Agent call's result (`bin/routing_figures/measure.py`), so neither sees the report reads. Found by the architecture review of PR #310.
|
||||
|
||||
### 220. Have rule-hook also count a pass's earlier round commits on the branch, so a fourth commit titled round 3 is refused too.
|
||||
|
||||
Found by the principles pass of PR #360 (item 214). rule-hook's `round-cap` reads only the number in the title being committed (`bin/round_titles.py` `past_cap`), while `larv-review round-count` (`round_counts`) also counts repeated numbers, so a fourth commit titled `Answer stability round 3: …` passes the hook. Have `bash_breaches` in `bin/rule_hook/main.py` read the titles of `<default branch>..HEAD` in each round commit's checkout, with the hook's guarded git (`bin/rule_hook/git.py`), and refuse the commit when `round_counts(titles)` already holds 3 for its pass. The hook must not fetch from origin, as `git_repo.default_branch` does; it already has a lookup that needs no fetch, `local_default_branch` in `bin/rule_hook/override_log.py`. A `git commit --amend` must not count the commit it replaces. Hard part: the hook runs on every commit, so a slow or failing lookup must not block the commit. The change touches `bin/round_titles.py` `CAP`, which the comprehension seats found hard; make it easier to read.
|
||||
|
||||
### 149. Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.
|
||||
|
||||
A release's capacity is a sum of its items' cost, as the formula counts it. Each project states its own; the human sets the first value per project. Nothing is deleted by score.
|
||||
|
||||
Reference in New Issue
Block a user