Merge pull request 'Block a read of file lines already in the agent's context, unless a compaction dropped them or the file changed' (#387) from item-239 into main
This commit was merged in pull request #387.
This commit is contained in:
@@ -2,6 +2,11 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- rule-hook has a new rule, `repeat-read`, and now runs before each Read call too. It blocks a Read, or a Bash `cat`, `head` or `sed -n '<A>,<B>p'` of one file, when every line it asks for is already in the agent's context, and the block names those lines. The rule takes no override.
|
||||
- A line is in context when an earlier read's result in the agent's own transcript showed it since the last compaction, as the file holds it now. A subagent is judged on its own transcript, not its session's.
|
||||
- A read goes through when it asks for a line no earlier read showed, or one that changed since. A Bash output that Claude Code cut short shows no lines.
|
||||
- A Read also goes through until the agent has read the file with Read or written it with Edit, MultiEdit or Write, and again once the file changed after that. Edit, MultiEdit and Write refuse the file until then, and a Bash read does not count.
|
||||
- Where the context lacks lines the transcript shows, as after Claude Code cleared old tool results, `grep -n '' <file> | sed -n '<A>,<B>p'` reads them.
|
||||
- Install the latest stable Python, today 3.14, as `python3`: the tools and their tests no longer support 3.11 to 3.13. A full test run now takes about 29 s, where it took about 3.3 minutes. It runs once, on that Python, in four pytest-xdist workers, where it used to rerun the whole suite on 3.11.
|
||||
- `routing-figures.md` now reports dollars by project, subagent type, rung and tool, in place of tokens. Cache reads are included; the tokens left them out, though they were about a quarter of the cost. The dollars are what each worker transcript's last `cost-state` entry records for each model, split across the messages of the worker and its subagents in proportion to their cost at list prices. The judge's runs are not counted, since the archive doesn't hold them.
|
||||
- larv-review's principles pass now launches one auditor per round, which checks the README's goals, code health and every principles file that applies, so it reads the project once per round. It used to launch one auditor for the goals with code health and one for each principles file, each reading the whole project. Its report is `principles-round-<n>.md`, like every other pass's.
|
||||
|
||||
@@ -103,7 +103,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `rank_todo.py [--check] [todo.md]` scores and ranks a project's `todo.md`; `rank_todo.py --help` states the format.
|
||||
- `reading-cost [--reset] [record]` appends a figure for each chunk `loop-claude` finished to a project's `docs/reading-cost.md`, and says whether the figures rise; `reading-cost --help` states the measure.
|
||||
- `review-replay <pass> <model>` replays a review pass's archived rounds from the projects run on the model, on the code each reviewed, and prints the lowest rung below the project's that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again; `review-replay --help` states the replay. Each replayed reviewer skips every permission prompt, stability-reviewer has Bash, and an archived prompt may name paths on the host.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Bash, Edit, MultiEdit, Write and Agent call, and `rule-hook --help` states the rules and the override.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Agent, Bash, Edit, MultiEdit, Read and Write call, and `rule-hook --help` states the rules and the override.
|
||||
- `run-heavy <command>` runs a heavy command once it holds one of the machine's heavy slots, one per 4 CPUs, which every session shares; `run-heavy --help` states the slots, and the memory cap it gives a container that sets no limit.
|
||||
- `score-outcomes [L E I S]` lists the finished items whose guessed scores sit nearest the given ones, and the shipped items the project's `docs/score-anchors.md` names for each level of L, E and I, each with what its chunk measured; then the median outcome at each guessed R and S, and how well L, E and I each order the chunks by review rounds and 🔴 findings; then each chunk's 🔴 and 🟡 findings with how many a reviewer marked `[misread]`, code health's counter-signal; `score-outcomes --help` states the measures.
|
||||
- `settings_filter.py` is git's clean filter for `settings.json`: it drops `"model"` and sorts the keys. `.gitattributes` sets the filter `settings` on `settings.json`, and Install's `git config` line makes this script that filter.
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Entries of Claude Code's transcripts and of its stream-json and json output."""
|
||||
import json
|
||||
import typing
|
||||
|
||||
|
||||
def parse_entry(raw: str | bytes) -> dict | None:
|
||||
@@ -87,11 +88,35 @@ def part_text(part: object) -> str | None:
|
||||
return part["text"] if isinstance(part, dict) and part.get("type") == "text" and isinstance(part.get("text"), str) else None
|
||||
|
||||
|
||||
# {"type": "tool_use", "name": "Bash", "input": {"command": "echo hi"}}
|
||||
# {"type": "tool_use", "id": "toolu_01…", "name": "Bash", "input": {"command": "echo hi"}}
|
||||
def tool_call(part: object) -> tuple[object, object] | None:
|
||||
return (part.get("name"), part.get("input")) if isinstance(part, dict) and part.get("type") == "tool_use" else None
|
||||
|
||||
|
||||
def tool_use_id(part: object) -> str | None:
|
||||
return part["id"] if isinstance(part, dict) and part.get("type") == "tool_use" and isinstance(part.get("id"), str) else None
|
||||
|
||||
|
||||
class ToolResult(typing.NamedTuple):
|
||||
tool_use_id: str
|
||||
text: str
|
||||
failed: bool
|
||||
|
||||
|
||||
# {"type": "tool_result", "tool_use_id": "toolu_01…", "content": "hi"}, its content also as [{"type": "text", "text": "hi"}], and with "is_error": true where the call failed
|
||||
def tool_result(part: object) -> ToolResult | None:
|
||||
if not isinstance(part, dict) or part.get("type") != "tool_result":
|
||||
return None
|
||||
return ToolResult(str(part.get("tool_use_id")), result_text(part.get("content")), bool(part.get("is_error")))
|
||||
|
||||
|
||||
def result_text(content: object) -> str:
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
# Claude Code may split one text across blocks, even inside a line, so the blocks join with nothing between them.
|
||||
return "".join(part_text(part) or "" for part in content) if isinstance(content, list) else ""
|
||||
|
||||
|
||||
# stream-json's last line: {"type": "result", "subtype": "success", ...}
|
||||
def ends_stream(event: dict) -> bool:
|
||||
return event.get("type") == "result"
|
||||
|
||||
@@ -37,6 +37,11 @@ def session_files(transcript: str) -> list[str]:
|
||||
return sorted(glob.glob(glob.escape(transcript.removesuffix(".jsonl")) + "*"))
|
||||
|
||||
|
||||
# <session id>/subagents/agent-<agent id>.jsonl, beside <session id>.jsonl
|
||||
def subagent_transcript(session_transcript: str, agent_id: str) -> str:
|
||||
return os.path.join(session_transcript.removesuffix(".jsonl"), "subagents", f"agent-{agent_id}.jsonl")
|
||||
|
||||
|
||||
def delete_session(session: Session) -> Failure | None:
|
||||
path = transcript_file(session)
|
||||
if path is None:
|
||||
|
||||
@@ -54,6 +54,15 @@ class FirstTimestampedLine(unittest.TestCase):
|
||||
self.assertEqual(transcript.first_timestamped_line(b""), b"")
|
||||
|
||||
|
||||
class CurrentThread(unittest.TestCase):
|
||||
def test_a_parent_cycle_ends_the_thread(self):
|
||||
looped = [{"uuid": "a", "parentUuid": "b"}, {"uuid": "b", "parentUuid": "a"}]
|
||||
self.assertEqual([entry["uuid"] for entry in transcript.current_thread(looped)], ["a", "b"])
|
||||
|
||||
def test_entries_without_a_uuid_make_no_thread(self):
|
||||
self.assertEqual(transcript.current_thread([{"type": "summary"}]), [])
|
||||
|
||||
|
||||
class HumanTurns(unittest.TestCase):
|
||||
def test_counts_typed_turns_and_not_tool_results(self):
|
||||
self.assertEqual(transcript.human_turns([
|
||||
|
||||
@@ -21,6 +21,21 @@ def read_transcript(path: str) -> list[dict] | Failure:
|
||||
return Failure(f"could not read the transcript {path}: {failure.strerror}")
|
||||
|
||||
|
||||
def current_thread(entries: list[dict]) -> list[dict]:
|
||||
"""The entries of the transcript's current thread since its last compaction, oldest first: the newest entry with a uuid and its ancestors by parentUuid.
|
||||
|
||||
That leaves out a branch the session rewound, and the results of parallel tool calls that Claude Code chains on a side branch."""
|
||||
chained = [entry for entry in entries if isinstance(entry.get("uuid"), str)]
|
||||
by_uuid = {entry["uuid"]: entry for entry in chained}
|
||||
found: list[dict] = []
|
||||
entry = chained[-1] if chained else None
|
||||
# A compaction writes {"type": "system", "subtype": "compact_boundary", "parentUuid": null, ...}, so the thread starts there; the length check stops a parent cycle.
|
||||
while entry is not None and len(found) < len(chained):
|
||||
found.append(entry)
|
||||
entry = by_uuid.get(entry.get("parentUuid"))
|
||||
return found[::-1]
|
||||
|
||||
|
||||
# The lines before it, such as the session's title, are the same for every chunk of a project.
|
||||
def first_timestamped_line(raw: bytes) -> bytes:
|
||||
lines = raw.splitlines()
|
||||
|
||||
@@ -5,7 +5,7 @@ import re
|
||||
import statistics
|
||||
import typing
|
||||
|
||||
from claude_code.entries import assistant_parts, parts, tool_call
|
||||
from claude_code.entries import assistant_parts, parts, tool_call, tool_result
|
||||
from git_command import commits
|
||||
from model_routing.route import ESCALATION_MARKER, route_from
|
||||
from routing_figures.prices import listed
|
||||
@@ -204,19 +204,7 @@ def messages(entries: list[dict]) -> list[Message]:
|
||||
|
||||
def results(entries: list[dict]) -> dict[str, str]:
|
||||
"""Each tool result's text by the id of the call it answers."""
|
||||
found = {}
|
||||
for entry in entries:
|
||||
for part in parts(entry):
|
||||
if isinstance(part, dict) and part.get("type") == "tool_result":
|
||||
found[str(part.get("tool_use_id"))] = result_text(part.get("content"))
|
||||
return found
|
||||
|
||||
|
||||
# {"type": "tool_result", "content": "hi"} or {"type": "tool_result", "content": [{"type": "text", "text": "hi"}]}
|
||||
def result_text(body: object) -> str:
|
||||
if isinstance(body, str):
|
||||
return body
|
||||
return "".join(block["text"] for block in body if isinstance(block, dict) and isinstance(block.get("text"), str)) if isinstance(body, list) else ""
|
||||
return {result.tool_use_id: result.text for entry in entries for part in parts(entry) if (result := tool_result(part))}
|
||||
|
||||
|
||||
def rounds_past_first(worker: list[dict], home: str) -> int:
|
||||
|
||||
@@ -2,6 +2,7 @@ import dataclasses
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from claude_code.session import subagent_transcript
|
||||
|
||||
FILE_TOOLS = ("Edit", "MultiEdit", "Write")
|
||||
|
||||
@@ -13,6 +14,8 @@ class Call:
|
||||
cwd: str
|
||||
# The subagent the call comes from, such as prose-reviewer; empty for the session's own agent.
|
||||
agent: str = ""
|
||||
# The transcript of the agent that makes the call: a subagent's own, not its session's.
|
||||
transcript: str = ""
|
||||
|
||||
|
||||
def parsed_call(hook_input: object) -> Call | Failure:
|
||||
@@ -21,8 +24,12 @@ def parsed_call(hook_input: object) -> Call | Failure:
|
||||
tool, given, cwd = hook_input.get("tool_name"), hook_input.get("tool_input"), hook_input.get("cwd")
|
||||
if not isinstance(tool, str) or not isinstance(given, dict) or not isinstance(cwd, str):
|
||||
return Failure("the hook input has no string tool_name and cwd and object tool_input")
|
||||
agent = hook_input.get("agent_type")
|
||||
return Call(tool, given, cwd, agent if isinstance(agent, str) else "")
|
||||
agent, agent_id, transcript = hook_input.get("agent_type"), hook_input.get("agent_id"), hook_input.get("transcript_path")
|
||||
transcript = transcript if isinstance(transcript, str) else ""
|
||||
if isinstance(agent_id, str) and transcript:
|
||||
# Inside a subagent, transcript_path still names the session's transcript.
|
||||
transcript = subagent_transcript(transcript, agent_id)
|
||||
return Call(tool, given, cwd, agent if isinstance(agent, str) else "", transcript)
|
||||
|
||||
|
||||
def text(given: dict[str, typing.Any], key: str) -> str:
|
||||
|
||||
+39
-2
@@ -3,7 +3,7 @@
|
||||
usage: rule-hook (a PreToolUse hook: reads the call as JSON on stdin)
|
||||
rule-hook overrides (prints this chunk's overrides)
|
||||
|
||||
settings.json runs it before each Bash, Edit, MultiEdit, Write and Agent call, in every session and
|
||||
settings.json runs it before each Agent, Bash, Edit, MultiEdit, Read and Write call, in every session and
|
||||
subagent. The rules:
|
||||
|
||||
{rules}
|
||||
@@ -29,16 +29,21 @@ override cannot be logged; 0 otherwise. `rule-hook overrides` exits 1 when git c
|
||||
checkout or the branch's reflog. Exit 1, a non-blocking error to Claude Code, on input it
|
||||
cannot read.
|
||||
"""
|
||||
import datetime
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import stat
|
||||
import sys
|
||||
import time
|
||||
import typing
|
||||
|
||||
from rule_hook.call import applied, Call, FILE_TOOLS, parsed_call, text
|
||||
from rule_hook.repeat_read import asked_lines, repeat_breaches, shown_lines
|
||||
from rule_hook.rules import Breach, check, Checked, command_breaches, file_breaches, rules_help
|
||||
from git_command import commits
|
||||
from claude_code.transcript import current_thread, read_transcript
|
||||
from failure import Failure
|
||||
from rule_hook.override_log import Log, log_path, logged, print_overrides, write_override
|
||||
from rule_hook.reviewer_report import report_breaches
|
||||
@@ -76,7 +81,9 @@ def breaches_of(call: Call, home: str, environ: typing.Mapping[str, str]) -> tup
|
||||
return text(call.given, "prompt"), text(call.given, "subagent_type"), reviewer_breaches(call, environ)
|
||||
if call.tool == "Bash":
|
||||
command = text(call.given, "command")
|
||||
return command, command.split("\n", 1)[0], bash_breaches(command, call.cwd, home)
|
||||
return command, command.split("\n", 1)[0], bash_breaches(command, call.cwd, home) + read_breaches(call, home)
|
||||
if call.tool == "Read":
|
||||
return "", text(call.given, "file_path"), read_breaches(call, home)
|
||||
path = text(call.given, "file_path")
|
||||
if call.tool not in FILE_TOOLS or not path:
|
||||
return None
|
||||
@@ -107,6 +114,36 @@ def bash_breaches(command: str, cwd: str, home: str) -> list[Breach]:
|
||||
return command_breaches(command, simples, home) + [breach for commit in commits(simples, home) for breach in todo_breaches(commit.directory)]
|
||||
|
||||
|
||||
def read_breaches(call: Call, home: str) -> list[Breach]:
|
||||
"""The repeat-read breaches of a call, read from the calling agent's transcript and the asked file; none where either cannot be read, or the file is not a regular file."""
|
||||
asked = asked_lines(call.tool, call.given, call.cwd, home)
|
||||
changed = modified(asked.path) if asked else None
|
||||
entries = read_transcript(call.transcript) if changed and call.transcript else []
|
||||
if asked is None or changed is None or isinstance(entries, Failure):
|
||||
return []
|
||||
shown = shown_lines(current_thread(entries), call.tool, asked.path, changed, home)
|
||||
# No line past the last one shown is in context, so the file's lines through one past it tell as much as all of them, and a long file is not read whole.
|
||||
return repeat_breaches(asked, first_lines(asked.path, max(shown) + 1), shown, call.cwd) if shown else []
|
||||
|
||||
|
||||
def modified(path: str) -> str | None:
|
||||
"""When a regular file last changed, stamped as a transcript entry is; None for any other path, such as a FIFO, whose read could hang."""
|
||||
try:
|
||||
status = os.stat(path)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return datetime.datetime.fromtimestamp(status.st_mtime, datetime.UTC).isoformat(timespec="milliseconds").replace("+00:00", "Z") if stat.S_ISREG(status.st_mode) else None
|
||||
|
||||
|
||||
def first_lines(path: str, count: int) -> list[str]:
|
||||
"""The file's first count lines, without their line ends; none where it cannot be read as UTF-8."""
|
||||
try:
|
||||
with open(path, encoding="utf-8") as file:
|
||||
return [line.removesuffix("\n") for line in itertools.islice(file, count)]
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return []
|
||||
|
||||
|
||||
def read(path: str) -> str:
|
||||
try:
|
||||
return pathlib.Path(path).read_text(encoding="utf-8")
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
"""Whether a read asks for lines of a file already in the agent's context, for the `repeat-read` rule.
|
||||
|
||||
A line is in context when a read since the transcript thread's last compaction showed it at its line number, as the file holds it now.
|
||||
Against a Read, no line is in context when the file changed after the agent last read it with Read or wrote it with Edit, MultiEdit or Write, or when it never did."""
|
||||
import dataclasses
|
||||
import re
|
||||
|
||||
from rule_hook.rules import Breach
|
||||
from rule_hook.call import FILE_TOOLS, text
|
||||
from claude_code.entries import parts, tool_call, tool_result, tool_use_id
|
||||
from shell_command import resolved, simple_commands
|
||||
|
||||
# The lines a Read without a limit returns.
|
||||
READ_LIMIT = 2000
|
||||
# A line of a Read result: its number, then a tab, or an arrow in older Claude Code versions.
|
||||
NUMBERED = re.compile(r" *(\d+)[\t→](.*)", re.DOTALL)
|
||||
HEAD_COUNT = re.compile(r"-(?:n ?)?(\d+)")
|
||||
SED_RANGE = re.compile(r"(\d+)(?:,(\d+|\$))?p")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Asked:
|
||||
path: str
|
||||
first: int
|
||||
# None: to the file's end.
|
||||
last: int | None
|
||||
|
||||
|
||||
def repeat_breaches(asked: Asked, lines: list[str], shown: dict[int, str], cwd: str) -> list[Breach]:
|
||||
"""shown is what shown_lines found, and lines the asked file's lines now, read at least through one past the last shown line."""
|
||||
last = len(lines) if asked.last is None else min(asked.last, len(lines))
|
||||
if not in_context(asked.first, last, lines, shown):
|
||||
return []
|
||||
named = f"line {last} of {asked.path} is" if asked.first == last else f"lines {asked.first}-{last} of {asked.path} are"
|
||||
return [Breach("repeat-read", f"{named} in your context, unchanged since your last read of it", cwd)]
|
||||
|
||||
|
||||
def in_context(first: int, last: int, lines: list[str], shown: dict[int, str]) -> bool:
|
||||
return first <= last and all(shown.get(number) == lines[number - 1] for number in range(first, last + 1))
|
||||
|
||||
|
||||
def asked_lines(tool: object, given: object, cwd: str, home: str) -> Asked | None:
|
||||
"""The lines a tool call reads; None for any call but a Read and the Bash reads the rule names, such as one whose output goes elsewhere."""
|
||||
if not isinstance(given, dict):
|
||||
return None
|
||||
return read_asked(given, cwd, home) if tool == "Read" else shell_asked(text(given, "command"), cwd, home) if tool == "Bash" else None
|
||||
|
||||
|
||||
def read_asked(given: dict, cwd: str, home: str) -> Asked | None:
|
||||
path, offset, limit = text(given, "file_path"), given.get("offset"), given.get("limit")
|
||||
first = offset if isinstance(offset, int) and offset > 0 else 1
|
||||
return Asked(resolved(cwd, path, home), first, first + (limit if isinstance(limit, int) and limit > 0 else READ_LIMIT) - 1) if path else None
|
||||
|
||||
|
||||
# reading_cost/shell_reads.py parses a shell read too, loosely: a measure of reading counts any file a reader names, while a rule that blocks must match only a read whose lines it knows exactly.
|
||||
def shell_asked(command: str, cwd: str, home: str) -> Asked | None:
|
||||
simples = simple_commands(command, cwd, home) if not set("|<>") & set(command) else []
|
||||
if len(simples) != 1 or len(simples[0].words) < 2 or simples[0].words[-1].startswith("-"):
|
||||
return None
|
||||
(name, *options, path), directory = simples[0].words, simples[0].directory
|
||||
path = resolved(directory, path, home)
|
||||
if name == "cat" and not options:
|
||||
return Asked(path, 1, None)
|
||||
if name == "head" and (counted := HEAD_COUNT.fullmatch(" ".join(options) or "-10")):
|
||||
return Asked(path, 1, int(counted.group(1)))
|
||||
if name == "sed" and len(options) == 2 and options[0] == "-n" and (ranged := SED_RANGE.fullmatch(options[1])):
|
||||
start, end = ranged.groups()
|
||||
return Asked(path, int(start), None if end == "$" else int(end or start))
|
||||
return None
|
||||
|
||||
|
||||
def shown_lines(thread: list[dict], asker: str, path: str, changed: str, home: str) -> dict[int, str]:
|
||||
"""Each line number of path that a successful read in the thread showed, with the line its latest read showed; none against a Read when the file changed after the agent last read it with Read or wrote it with Edit, MultiEdit or Write, or when it never did.
|
||||
|
||||
changed is when the file last changed, stamped as the transcript stamps its entries, so the two compare as text."""
|
||||
used = results_on(thread, path, home)
|
||||
# Claude Code's Edit and Write refuse such a file until a Read, and a Bash read does not count as one.
|
||||
touched = max((stamp for tool, _, _, stamp in used if tool == "Read" or tool in FILE_TOOLS), default="")
|
||||
if asker == "Read" and changed > touched:
|
||||
return {}
|
||||
shown: dict[int, str] = {}
|
||||
for tool, asked, result, _ in used:
|
||||
shown |= result_lines(tool, asked, result) if asked else {}
|
||||
return shown
|
||||
|
||||
|
||||
def results_on(thread: list[dict], path: str, home: str) -> list[tuple[object, Asked | None, str, str]]:
|
||||
"""Each successful result in the thread of a call that read lines of path or wrote it: the call's tool, the lines it read (None for a write), the result's text and its entry's timestamp."""
|
||||
calls: dict[str, tuple[object, Asked | None]] = {}
|
||||
found: list[tuple[object, Asked | None, str, str]] = []
|
||||
for entry in thread:
|
||||
cwd = str(entry.get("cwd", ""))
|
||||
for part in parts(entry):
|
||||
if (call := tool_call(part)) and (called := tool_use_id(part)):
|
||||
asked = asked_lines(*call, cwd, home)
|
||||
if (asked.path if asked else written(*call, cwd, home)) == path:
|
||||
calls[called] = (call[0], asked)
|
||||
elif (result := tool_result(part)) and not result.failed and (use := calls.get(result.tool_use_id)):
|
||||
found.append((use[0], use[1], result.text, text(entry, "timestamp")))
|
||||
return found
|
||||
|
||||
|
||||
def written(tool: object, given: object, cwd: str, home: str) -> str:
|
||||
"""The file an Edit, MultiEdit or Write call writes; empty for any other call."""
|
||||
return resolved(cwd, path, home) if tool in FILE_TOOLS and isinstance(given, dict) and (path := text(given, "file_path")) else ""
|
||||
|
||||
|
||||
def result_lines(tool: object, asked: Asked, result: str) -> dict[int, str]:
|
||||
if tool == "Read":
|
||||
return {int(numbered.group(1)): numbered.group(2) for line in result.split("\n") if (numbered := NUMBERED.fullmatch(line))}
|
||||
# Claude Code cuts a long Bash output short with a note: before the output, "<persisted-output>\n" or "<truncated-output>\n" and a few more lines, or after it, "\n\n... [N lines truncated] ...".
|
||||
# The note's lines would take the numbers of the file's, so such an output shows no lines.
|
||||
if result.startswith(("<persisted-output>\n", "<truncated-output>\n")) or result.endswith(" lines truncated] ..."):
|
||||
return {}
|
||||
return {number: line for number, line in enumerate(result.split("\n"), asked.first) if asked.last is None or number <= asked.last}
|
||||
@@ -10,7 +10,7 @@ from shell_command import launched, Simple, tool_name
|
||||
from review_folder import REPORTS
|
||||
from docker_command import unlimited
|
||||
|
||||
RuleName = typing.Literal["action-pin", "commit-one-line", "docker-run-memory", "ignorefile-sorted", "image-pin", "no-ai-attribution", "reviewer-report", "reviewer-rung", "round-cap", "round-title-counts", "run-heavy", "todo-rank-check"]
|
||||
RuleName = typing.Literal["action-pin", "commit-one-line", "docker-run-memory", "ignorefile-sorted", "image-pin", "no-ai-attribution", "repeat-read", "reviewer-report", "reviewer-rung", "round-cap", "round-title-counts", "run-heavy", "todo-rank-check"]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -46,6 +46,12 @@ RULES: dict[RuleName, Rule] = {
|
||||
"no commit, PR, issue comment or file carries a Claude or Anthropic attribution, such as a co-author trailer or a generated-with line",
|
||||
"a git, gh or tea command, and any file written",
|
||||
),
|
||||
# A Read carries no text for a marker, and a misjudged read has a way through that the message names, so no marker waives it.
|
||||
"repeat-read": Rule(
|
||||
"an agent reads no lines of a file that its context already holds unchanged, since each read pays for them again; where the context lacks them although the transcript shows them, as after Claude Code cleared old tool results, read them with another command, such as `grep -n '' <file> | sed -n '<A>,<B>p'`; a Read goes through until the agent has read the file with Read or written it with Edit, MultiEdit or Write, and again once the file changed after that, since those refuse a file until then",
|
||||
"a Read call, and a Bash command that is one `cat <file>`, `head [-n <N> | -<N>] <file>` or `sed -n '<A>[,<B>]p' <file>`, with no pipe or redirect, against the reads in the calling agent's own transcript since its last compaction",
|
||||
overridable=False,
|
||||
),
|
||||
# A reviewer only reports, so no marker waives it.
|
||||
"reviewer-report": Rule(
|
||||
f"a reviewer writes nothing but its report, with Write, into a file under ~/{REPORTS}/",
|
||||
@@ -121,7 +127,7 @@ def check(breaches: list[Breach], written: str, markdown: bool, bash: bool) -> C
|
||||
for rule in sorted({breach.rule for breach in blocking}):
|
||||
details = "; ".join(breach.detail for breach in blocking if breach.rule == rule)
|
||||
if not RULES[rule].overridable:
|
||||
messages.append(f"Blocked by rule {rule}: {details}. The rule: {RULES[rule].holds}.")
|
||||
messages.append(f"Blocked by rule {rule}: {details}. The rule: {RULES[rule].holds}. No marker overrides it.")
|
||||
continue
|
||||
marker = f"<!-- rule-override {rule}: <reason> -->" if markdown else f"# rule-override {rule}: <reason>"
|
||||
messages.append(f"Blocked by rule {rule}: {details}. The rule: {RULES[rule].holds}. Where it does not fit this case, add `{marker}` to {where}; larv-review rules on the reason.")
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
@@ -9,9 +10,10 @@ import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from rule_hook.repeat_read import Asked, result_lines
|
||||
from rule_hook.rules import blocks, Breach, check, command_breaches, file_breaches, overrides
|
||||
from rule_hook.main import breaches_of
|
||||
from rule_hook.call import Call
|
||||
from rule_hook.main import breaches_of, modified, read_breaches
|
||||
from rule_hook.call import Call, parsed_call
|
||||
from git_command import Commit, commit_message, commits
|
||||
from test_rank_todo import DECISIONS, README, TODO
|
||||
from test_env import isolated_env
|
||||
@@ -40,7 +42,7 @@ def checked(command: str) -> list[str]:
|
||||
|
||||
|
||||
def hook(call: dict, cwd: str, env: dict[str, str] | None = None) -> subprocess.CompletedProcess:
|
||||
return subprocess.run([str(HOOK)], capture_output=True, cwd=cwd, env=isolated_env() | (env or {}), input=json.dumps(call), text=True)
|
||||
return subprocess.run([str(HOOK)], capture_output=True, cwd=cwd, env=isolated_env() | (env or {}), input=json.dumps(call), text=True, timeout=10)
|
||||
|
||||
|
||||
def git(cwd: str, *args: str) -> str:
|
||||
@@ -328,6 +330,242 @@ class ReviewerReport(unittest.TestCase):
|
||||
self.assertEqual(self.breached("Write", "/r/README.md", "general-purpose"), [])
|
||||
|
||||
|
||||
class RepeatRead(unittest.TestCase):
|
||||
LINES = [f"line {n}" for n in range(1, 31)]
|
||||
# The file is written at WRITTEN and each entry stamped READ_AT, unless a test changes the file at CHANGED and stamps a later entry AFTER_THE_CHANGE.
|
||||
WRITTEN = datetime.datetime(2026, 5, 1, tzinfo=datetime.UTC).timestamp()
|
||||
READ_AT = "2026-06-01T00:00:00.000Z"
|
||||
CHANGED = datetime.datetime(2026, 7, 1, tzinfo=datetime.UTC).timestamp()
|
||||
AFTER_THE_CHANGE = "2026-07-01T00:00:01.000Z"
|
||||
|
||||
def setUp(self):
|
||||
self.folder = tempfile.mkdtemp()
|
||||
self.addCleanup(subprocess.run, ["rm", "-rf", self.folder])
|
||||
self.file = f"{self.folder}/f.txt"
|
||||
self.write(self.LINES, self.WRITTEN)
|
||||
self.transcript = f"{self.folder}/s.jsonl"
|
||||
self.entries: list[dict] = []
|
||||
|
||||
def write(self, lines: list[str], at: float) -> None:
|
||||
pathlib.Path(self.file).write_text("\n".join(lines) + "\n")
|
||||
os.utime(self.file, (at, at))
|
||||
|
||||
def change(self, number: int) -> None:
|
||||
self.write([("changed" if n == number else line) for n, line in enumerate(self.LINES, 1)], self.CHANGED)
|
||||
|
||||
def add(self, entry: dict, parent: str | None = "last") -> str:
|
||||
uuid = f"u{len(self.entries)}"
|
||||
parent = (self.entries[-1]["uuid"] if self.entries else None) if parent == "last" else parent
|
||||
self.entries.append({"uuid": uuid, "parentUuid": parent, "cwd": self.folder, "timestamp": self.READ_AT} | entry)
|
||||
pathlib.Path(self.transcript).write_text("\n".join(json.dumps(entry) for entry in self.entries) + "\n")
|
||||
return uuid
|
||||
|
||||
def used(self, tool: str, given: dict, result: str, error: bool = False, at: str = READ_AT) -> None:
|
||||
tool_id = f"toolu_{len(self.entries)}"
|
||||
self.add({"type": "assistant", "timestamp": at, "message": {"content": [{"type": "tool_use", "id": tool_id, "name": tool, "input": given}]}})
|
||||
self.add({"type": "user", "timestamp": at, "message": {"content": [{"type": "tool_result", "tool_use_id": tool_id, "content": result, "is_error": error}]}})
|
||||
|
||||
def numbered(self, start: int, end: int) -> str:
|
||||
return "\n".join(f"{n}\t{self.LINES[n - 1]}" for n in range(start, end + 1))
|
||||
|
||||
def read(self, start: int, end: int) -> None:
|
||||
self.used("Read", {"file_path": self.file, "offset": start, "limit": end - start + 1}, self.numbered(start, end))
|
||||
|
||||
def shown(self, start: int, end: int) -> str:
|
||||
return "\n".join(self.LINES[start - 1:end])
|
||||
|
||||
def found(self, tool: str, given: dict, agent: str = "") -> list[Breach]:
|
||||
return read_breaches(Call(tool, given, self.folder, agent, self.transcript), "/h")
|
||||
|
||||
def test_a_read_of_lines_already_read_is_blocked_with_a_note(self):
|
||||
self.read(1, 10)
|
||||
found = self.found("Read", {"file_path": self.file, "offset": 2, "limit": 5})
|
||||
self.assertEqual(rules(found), ["repeat-read"])
|
||||
self.assertIn(f"lines 2-6 of {self.file} are in your context", found[0].detail)
|
||||
|
||||
def test_the_note_names_a_single_line_as_one(self):
|
||||
self.read(1, 10)
|
||||
self.assertIn(f"line 7 of {self.file} is in your context", self.found("Bash", {"command": "sed -n 7p f.txt"})[0].detail)
|
||||
|
||||
def test_lines_read_in_two_reads_are_in_context(self):
|
||||
self.read(1, 10)
|
||||
self.read(11, 20)
|
||||
self.assertEqual(rules(self.found("Read", {"file_path": self.file, "offset": 5, "limit": 10})), ["repeat-read"])
|
||||
|
||||
def test_a_read_past_what_was_read_goes_through(self):
|
||||
self.read(1, 10)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 5, "limit": 10}), [])
|
||||
|
||||
def test_a_read_without_a_limit_asks_for_the_whole_file(self):
|
||||
self.read(1, 10)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file}), [])
|
||||
|
||||
def test_a_read_result_in_the_older_arrow_format_counts(self):
|
||||
self.used("Read", {"file_path": self.file, "offset": 1, "limit": 10}, "\n".join(f"{n:6}→{self.LINES[n - 1]}" for n in range(1, 11)))
|
||||
self.assertEqual(rules(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10})), ["repeat-read"])
|
||||
|
||||
def test_a_compaction_lets_the_read_through(self):
|
||||
self.read(1, 10)
|
||||
self.add({"type": "system", "subtype": "compact_boundary"}, parent=None)
|
||||
self.add({"type": "user", "message": {"content": "This session is being continued"}})
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10}), [])
|
||||
|
||||
def test_a_changed_line_lets_the_read_through(self):
|
||||
self.read(1, 10)
|
||||
self.change(1)
|
||||
self.assertEqual(self.found("Bash", {"command": f"head {self.file}"}), [])
|
||||
|
||||
def test_a_change_in_the_last_asked_line_lets_the_read_through(self):
|
||||
self.read(1, 10)
|
||||
self.change(10)
|
||||
self.assertEqual(self.found("Bash", {"command": "sed -n 1,10p f.txt"}), [])
|
||||
|
||||
def test_lines_that_did_not_change_stay_in_context(self):
|
||||
self.read(1, 10)
|
||||
self.change(1)
|
||||
self.assertEqual(rules(self.found("Bash", {"command": "sed -n 2,10p f.txt"})), ["repeat-read"])
|
||||
|
||||
def test_a_read_without_an_offset_starts_at_line_1(self):
|
||||
self.read(2, 11)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "limit": 10}), [])
|
||||
|
||||
def test_calls_that_neither_read_nor_write_the_file_are_passed_over(self):
|
||||
self.used("Grep", {"pattern": "line", "path": self.folder}, "f.txt")
|
||||
self.used("Bash", {"command": "ls"}, "f.txt")
|
||||
self.read(1, 10)
|
||||
self.assertEqual(rules(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10})), ["repeat-read"])
|
||||
|
||||
# Claude Code's Edit refuses a file changed since the agent last read it with Read, or wrote it with Edit or Write, until it reads it again.
|
||||
def test_a_change_in_the_millisecond_of_the_last_read_counts_as_before_it(self):
|
||||
self.read(1, 10)
|
||||
at = datetime.datetime(2026, 6, 1, tzinfo=datetime.UTC).timestamp()
|
||||
os.utime(self.file, (at, at))
|
||||
self.assertEqual(rules(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10})), ["repeat-read"])
|
||||
|
||||
def test_a_file_s_change_is_stamped_as_the_transcript_stamps_its_entries(self):
|
||||
at = datetime.datetime(2026, 6, 1, 12, 30, 5, 250000, tzinfo=datetime.UTC).timestamp()
|
||||
os.utime(self.file, (at, at))
|
||||
self.assertEqual(modified(self.file), "2026-06-01T12:30:05.250Z")
|
||||
|
||||
def test_a_read_goes_through_once_the_file_changed_after_the_last_read(self):
|
||||
self.read(1, 10)
|
||||
self.change(30)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10}), [])
|
||||
|
||||
def test_a_refused_edit_does_not_count_as_reading_the_file(self):
|
||||
self.read(1, 10)
|
||||
self.change(30)
|
||||
self.used("Edit", {"file_path": self.file, "old_string": "line 5", "new_string": "five"}, "<tool_use_error>File has been modified since read, either by the user or by a linter. Read it again before attempting to write it.</tool_use_error>", error=True, at=self.AFTER_THE_CHANGE)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10}), [])
|
||||
|
||||
def test_an_edit_keeps_the_lines_it_left_unchanged_in_context(self):
|
||||
self.read(1, 10)
|
||||
self.change(30)
|
||||
self.used("Edit", {"file_path": self.file, "old_string": "line 30", "new_string": "changed"}, f"The file {self.file} has been updated successfully.", at=self.AFTER_THE_CHANGE)
|
||||
self.assertEqual(rules(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10})), ["repeat-read"])
|
||||
|
||||
def test_a_failed_read_holds_no_lines(self):
|
||||
self.used("Read", {"file_path": self.file, "offset": 1, "limit": 10}, self.numbered(1, 10), error=True)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10}), [])
|
||||
|
||||
def test_a_read_on_a_branch_the_session_rewound_holds_no_lines(self):
|
||||
start = self.add({"type": "user", "message": {"content": "go"}})
|
||||
self.read(1, 10)
|
||||
self.add({"type": "user", "message": {"content": "go again"}}, parent=start)
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 1, "limit": 10}), [])
|
||||
|
||||
def test_head_and_sed_of_lines_read_are_blocked(self):
|
||||
self.read(1, 20)
|
||||
for command in [f"head -n 5 {self.file}", f"head -n5 {self.file}", f"head -5 {self.file}", f"head {self.file}", "sed -n '3,8p' f.txt", "sed -n 7p f.txt"]:
|
||||
with self.subTest(command):
|
||||
self.assertEqual(rules(self.found("Bash", {"command": command})), ["repeat-read"])
|
||||
|
||||
def test_a_head_starts_at_line_1(self):
|
||||
self.read(2, 11)
|
||||
self.assertEqual(self.found("Bash", {"command": f"head {self.file}"}), [])
|
||||
|
||||
def test_a_cat_asks_for_the_whole_file(self):
|
||||
self.read(1, 20)
|
||||
self.assertEqual(self.found("Bash", {"command": f"cat {self.file}"}), [])
|
||||
|
||||
# Claude Code's Edit and Write take only a Read as having read the file.
|
||||
def test_a_read_goes_through_until_the_agent_read_the_file_with_read(self):
|
||||
self.used("Bash", {"command": f"sed -n '1,20p' {self.file}"}, self.shown(1, 20))
|
||||
self.assertEqual(self.found("Read", {"file_path": self.file, "offset": 4, "limit": 3}), [])
|
||||
|
||||
def test_a_bash_output_counts_against_a_read_once_the_agent_read_the_file_with_read(self):
|
||||
self.read(21, 30)
|
||||
self.used("Bash", {"command": f"sed -n '1,20p' {self.file}"}, self.shown(1, 20))
|
||||
self.assertEqual(rules(self.found("Read", {"file_path": self.file, "offset": 4, "limit": 3})), ["repeat-read"])
|
||||
|
||||
def test_a_cat_output_puts_every_line_in_context(self):
|
||||
self.used("Bash", {"command": f"cat {self.file}"}, self.shown(1, 30))
|
||||
self.assertEqual(rules(self.found("Bash", {"command": "sed -n '25,$p' f.txt"})), ["repeat-read"])
|
||||
|
||||
def test_a_relative_path_in_the_transcript_is_under_its_entry_s_cwd(self):
|
||||
self.used("Bash", {"command": "cat f.txt"}, self.shown(1, 30))
|
||||
self.assertEqual(rules(read_breaches(Call("Bash", {"command": f"head -n 5 {self.file}"}, "/", "", self.transcript), "/h")), ["repeat-read"])
|
||||
|
||||
def test_a_tilde_path_is_under_the_home_folder(self):
|
||||
self.used("Read", {"file_path": "~/f.txt", "offset": 1, "limit": 10}, self.numbered(1, 10))
|
||||
self.used("Bash", {"command": "cat ~/f.txt"}, self.shown(1, 30))
|
||||
for tool, given in [("Read", {"file_path": "~/f.txt", "offset": 1, "limit": 10}), ("Bash", {"command": "sed -n 1,30p ~/f.txt"})]:
|
||||
with self.subTest(tool):
|
||||
self.assertEqual(rules(read_breaches(Call(tool, given, "/", "", self.transcript), self.folder)), ["repeat-read"])
|
||||
|
||||
def test_other_commands_on_the_file_are_not_checked(self):
|
||||
self.used("Bash", {"command": f"cat {self.file}"}, self.shown(1, 30))
|
||||
for command in [f"cat {self.file} > copy", f"cat {self.file} | wc -l", f"cat {self.file} {self.file}", f"head -c 5 {self.file}", f"sed 's/a/b/' {self.file}"]:
|
||||
with self.subTest(command):
|
||||
self.assertEqual(self.found("Bash", {"command": command}), [])
|
||||
|
||||
# As Claude Code 2.1.312 writes a Bash output it cut short.
|
||||
def test_an_output_cut_short_shows_no_lines(self):
|
||||
for cut in [
|
||||
f"<persisted-output>\nOutput too large (37.9KB). Full output saved to: {self.folder}/tool-results/b2hymdt7w.txt\n\nPreview (first 2KB):\n{self.shown(1, 30)}\n...\n</persisted-output>",
|
||||
f"<truncated-output>\nOutput too large (37.9KB). It could not be saved, so only the first 2KB are shown; the rest was dropped. If the tool can page or filter its results, call it again for the part you need.\n\n{self.shown(1, 30)}\n</truncated-output>",
|
||||
self.shown(1, 5) + "\n\n... [25 lines truncated] ...",
|
||||
]:
|
||||
with self.subTest(cut[:20]):
|
||||
self.assertEqual(result_lines("Bash", Asked(self.file, 1, None), cut), {})
|
||||
|
||||
def test_a_fifo_is_never_opened(self):
|
||||
fifo = f"{self.folder}/fifo"
|
||||
os.mkfifo(fifo)
|
||||
self.used("Bash", {"command": f"cat {fifo}"}, "line 1")
|
||||
# Opened, the FIFO would block the hook, since it has no writer.
|
||||
ran = hook({"cwd": self.folder, "tool_name": "Bash", "tool_input": {"command": f"head -n 1 {fifo}"}, "transcript_path": self.transcript}, self.folder)
|
||||
self.assertEqual((ran.returncode, ran.stderr), (0, ""))
|
||||
|
||||
def called(self, agent_id: str = "") -> Call:
|
||||
given = {"cwd": self.folder, "tool_name": "Read", "tool_input": {"file_path": self.file, "offset": 1, "limit": 10}, "transcript_path": self.transcript}
|
||||
return parsed_call(given | ({"agent_id": agent_id, "agent_type": "Explore"} if agent_id else {}))
|
||||
|
||||
def test_a_subagent_s_transcript_is_beside_the_session_s(self):
|
||||
self.assertEqual(self.called("a1").transcript, f"{self.folder}/s/subagents/agent-a1.jsonl")
|
||||
|
||||
def test_the_session_s_reads_are_not_in_a_subagent_s_context(self):
|
||||
self.read(1, 10)
|
||||
self.assertEqual(read_breaches(self.called("a1"), "/h"), [])
|
||||
|
||||
def test_the_session_is_judged_on_its_own_transcript(self):
|
||||
self.read(1, 10)
|
||||
self.assertEqual(rules(read_breaches(self.called(), "/h")), ["repeat-read"])
|
||||
|
||||
def test_a_missing_transcript_lets_the_read_through(self):
|
||||
self.assertEqual(read_breaches(Call("Read", {"file_path": self.file}, self.folder, "", f"{self.folder}/none.jsonl"), "/h"), [])
|
||||
|
||||
def test_the_hook_blocks_a_repeated_read_without_offering_an_override(self):
|
||||
self.read(1, 10)
|
||||
self.used("Bash", {"command": f"head -n 5 {self.file}"}, self.shown(1, 5))
|
||||
for tool, given in [("Read", {"file_path": self.file, "offset": 1, "limit": 10}), ("Bash", {"command": f"head -n 5 {self.file}"})]:
|
||||
with self.subTest(tool):
|
||||
ran = hook({"cwd": self.folder, "tool_name": tool, "tool_input": given, "transcript_path": self.transcript}, self.folder)
|
||||
self.assertEqual(ran.returncode, 2)
|
||||
self.assertIn("Blocked by rule repeat-read: ", ran.stderr)
|
||||
self.assertNotIn("rule-override", ran.stderr)
|
||||
|
||||
|
||||
class Hook(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.repo = tempfile.mkdtemp()
|
||||
@@ -477,7 +715,7 @@ class Hook(unittest.TestCase):
|
||||
|
||||
def test_help_lists_every_rule(self):
|
||||
helped = subprocess.run([str(HOOK), "--help"], capture_output=True, check=True, env=isolated_env(), text=True).stdout
|
||||
for rule in ["action-pin", "commit-one-line", "docker-run-memory", "ignorefile-sorted", "image-pin", "no-ai-attribution", "reviewer-report", "reviewer-rung", "round-cap", "round-title-counts", "run-heavy", "todo-rank-check"]:
|
||||
for rule in ["action-pin", "commit-one-line", "docker-run-memory", "ignorefile-sorted", "image-pin", "no-ai-attribution", "repeat-read", "reviewer-report", "reviewer-rung", "round-cap", "round-title-counts", "run-heavy", "todo-rank-check"]:
|
||||
self.assertIn(f" {rule}\n The rule: ", helped)
|
||||
self.assertIn("reviewer-report\n The rule: a reviewer writes nothing but its report", helped)
|
||||
self.assertRegex(helped, r" reviewer-report\n( .*\n){2} Takes no override\.\n")
|
||||
|
||||
@@ -2808,3 +2808,43 @@ Inherited architect, decided by:
|
||||
- Locality: `bin/routing_figures/measure.py:125` `measured`: the one-line `lacking=` count is explained only in the `ChunkFigures` docstring and in `basis`
|
||||
- Shape: `bin/routing_figures/measure.py:85` `Message`: named fields replace unnamed `(at, tools, spent)` tuples and six positional arguments
|
||||
- Shape: `bin/routing_figures/prices.py:33` `listed`: list-price arithmetic gets its own small module with a name that fits
|
||||
|
||||
## 2026-10-28T16:23:54Z, PR #387 at 0447f2e, against cde7720
|
||||
|
||||
Ruling: same
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | same | worse | worse | same | worse |
|
||||
| Inherited architect | same | worse | same | same | same |
|
||||
| Maintainability senior | same | worse | same | same | same |
|
||||
| Junior A | same | worse | same | same | worse |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Locality, Shape, Overall: `bin/rule_hook/main.py:117` `read_breaches`: the repeat-read rule's file and transcript reads sit in main.py, apart from its logic in repeat_read.py, unlike the other rules, so changing the rule needs both files open
|
||||
- Locality: `bin/rule_hook/repeat_read.py:72` `shown_lines`: whether a Read is checked turns on a text comparison of the mtime with transcript stamps, which holds only while main.modified formats the mtime as Claude Code stamps entries
|
||||
- Locality: `bin/rule_hook/repeat_read.py:29` `repeat_breaches`: needs lines read one past the last shown line, a promise main.py's caller keeps with nothing enforcing it
|
||||
- Shape: `bin/rule_hook/repeat_read.py:87` `results_on`: returns unnamed four-part tuples that callers unpack by position
|
||||
- Shape: `bin/rule_hook/repeat_read.py:103` `written`: returns a path, while `written` in main.py names the text that may carry an override marker
|
||||
- Locality: `bin/rule_hook/main.py:84` `breaches_of`: a Bash call now also reads the transcript and the file system through read_breaches
|
||||
- Shape: `bin/claude_code/entries.py:100` `ToolResult`: tool-result parsing has one named home, which shrinks measure.py's `results` to one line
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality: `bin/rule_hook/main.py:117` `read_breaches`: the rule's I/O sits in main.py and its logic in repeat_read.py, with two contracts across the seam, the mtime stamp's format and reading one line past the last shown, that only docstrings in the other file state
|
||||
- Locality: `bin/rule_hook/repeat_read.py:72` `shown_lines`: `asker` switches on a Read-only cut-off whose reason lies in a comment and the rule text, the order-dependent part of the rule
|
||||
- Shape: `bin/rule_hook/repeat_read.py:87` `results_on`: untyped positional tuples and dense conditionals at :46 and :105, offset by the shared `ToolResult` parser in `bin/claude_code/entries.py:100` and measure.py's dropped duplicate
|
||||
|
||||
Maintainability senior, decided by:
|
||||
|
||||
- Locality: `bin/rule_hook/main.py:117` `read_breaches`: the rule's I/O sits in main.py and its logic in repeat_read.py; the count `max(shown) + 1` at :126 sets what an open-ended `cat` means in the other file
|
||||
- Locality: `bin/rule_hook/repeat_read.py:72` `shown_lines`: `changed > touched` compares stamps as text, which holds only while `modified` in main.py formats the mtime as the transcript does; `changed` and `asker` made it the slowest unit to read
|
||||
- Locality, Shape: `bin/claude_code/entries.py:107` `tool_result`: one named parser replaces measure.py's private `result_text`, though `measure.py:142` `agent_calls` still reads `part.get("id")` itself
|
||||
|
||||
Junior A, decided by:
|
||||
|
||||
- Overall, Locality: `bin/rule_hook/main.py:117` `read_breaches`: the rule's entry point, stat and file reads sit in main.py while its logic sits in repeat_read.py, so changing the rule needs both files
|
||||
- Locality: `bin/rule_hook/repeat_read.py:29` `repeat_breaches`: needs lines through one past the last shown and a mtime formatted as a transcript stamp, both set up by main.py
|
||||
- Locality, Shape: `bin/routing_figures/measure.py:205` `results`: one line over the shared `tool_result`, where cde7720 held a private copy of the parsing
|
||||
- Shape: `bin/rule_hook/repeat_read.py:87` `results_on`: an unnamed four-field tuple unpacked by position, and `written` names a path here but the override text in main.py
|
||||
|
||||
+1
-1
@@ -80,7 +80,7 @@ loop-claude hands each worker's claude `CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000`
|
||||
|
||||
Why 200k. Item 231 asked for the best means to lower what a chunk spends on context, in dollars, then hours, with no loss of quality. Measured from loop-claude's archive, the 90 chunks with messages after the routing figures' reset of 2026-10-21 cost $1,215 at the list prices read on 2026-10-26 (Opus 5.5: $4 per million input tokens, $5 and $8 for 5-minute and 1-hour cache writes, $0.20 for cache reads, $20 for output; no model but Haiku 5.5 charges more above 200k). That is 0.86 of what the chunks' `cost-state` entries sum to, since some chunks' subagent transcripts are not in the archive. Worker main threads cost $375 (31%): 1-hour cache writes $149, cache reads $153, output $74. Subagents cost $840: 5-minute cache writes $379, output $274, cache reads $187, with the principles auditors at $274 over 342 runs, the stability reviewers $179 and general-purpose subagents $192. A worker starts at 41k tokens of context and peaks at a median 167k, one in ten past 358k, the largest at 848k. The worker's messages above 200k were 32% of its messages and 55% of its cost.
|
||||
|
||||
What reads cost, and whether a tool cuts them, measured at lilleman's ask after those figures: no tool cut a reviewer's reading without losing a finding, except a scout, which kept the stability reviewers' findings but lost the auditors' 🔴 and slowed every round; so none of them ships here. lilleman chose on 2026-10-27: workers trial a reading rule (`todo.md` item 240), reviewers get no scout, and the principles audit keeps reading the project whole. Two measures of reading: the characters a thread read, 93M in all, about 27M tokens; and the new input tokens of the messages that took results in, charged to those results, which also count what arrives beside a result, such as the agent's own previous reply. By the second measure tool results caused $620 of the $1,215, and reads (Read, Bash reads, Grep, Glob and git's diff, show and log) $537 of that: the worker's reads $140 (Bash reads $115, Read $12) of its $190 of results, the principles auditors' $133 (Read $109, Grep $22) of $139, the stability reviewers' $97 (Bash reads $79) of $106, general-purpose subagents' $84 of $97, the prose reviewers' $32, the architects' $30 and the product-owner reviewers' $21. By the first measure the worker's reads are about $45. Precision is low everywhere: of the characters a principles auditor read, 13% were in a file its report names; a stability reviewer 8%; a prose reviewer 18%; an architect 11%; the worker 4% in a file it later edited. Reading is mostly to find things and to learn what a project holds. The worker's `sed` ranges are small, median 25 lines, one in ten over 80. Of the worker's read characters, 49% were of a file it had read earlier in the thread: 36% within three messages, paging through it, and 13% later, with the file unchanged, about $6 of the $45; a reviewer's later reads of an unchanged file it had read are 1 to 2% (`todo.md` item 239).
|
||||
What reads cost, and whether a tool cuts them, measured at lilleman's ask after those figures: no tool cut a reviewer's reading without losing a finding, except a scout, which kept the stability reviewers' findings but lost the auditors' 🔴 and slowed every round; so none of them ships here. lilleman chose on 2026-10-27: workers trial a reading rule (`todo.md` item 240), reviewers get no scout, and the principles audit keeps reading the project whole. Two measures of reading: the characters a thread read, 93M in all, about 27M tokens; and the new input tokens of the messages that took results in, charged to those results, which also count what arrives beside a result, such as the agent's own previous reply. By the second measure tool results caused $620 of the $1,215, and reads (Read, Bash reads, Grep, Glob and git's diff, show and log) $537 of that: the worker's reads $140 (Bash reads $115, Read $12) of its $190 of results, the principles auditors' $133 (Read $109, Grep $22) of $139, the stability reviewers' $97 (Bash reads $79) of $106, general-purpose subagents' $84 of $97, the prose reviewers' $32, the architects' $30 and the product-owner reviewers' $21. By the first measure the worker's reads are about $45. Precision is low everywhere: of the characters a principles auditor read, 13% were in a file its report names; a stability reviewer 8%; a prose reviewer 18%; an architect 11%; the worker 4% in a file it later edited. Reading is mostly to find things and to learn what a project holds. The worker's `sed` ranges are small, median 25 lines, one in ten over 80. Of the worker's read characters, 49% were of a file it had read earlier in the thread: 36% within three messages, paging through it, and 13% later, with the file unchanged, about $6 of the $45; a reviewer's later reads of an unchanged file it had read are 1 to 2%. rule-hook's `repeat-read` now blocks a read of lines already in context.
|
||||
|
||||
The trials: four stability rounds and five principles rounds archived on 2026-10-23, each holding 🔴 or 🟡 findings, replayed on the project's rung as `review-replay` replays them, with the reviewer's definition changed through `--agents`; and three worker-like tasks, a search, a rename and an explanation, run on fresh clones at the project's rung. The baseline, the reviewer as it is, found 4 of its own 6 archived findings in each pass, so a variant that finds 4 or 5 of 6 does what the same reviewer does twice. The options todo item 231 named, each simulated on the 90 chunks and, where lilleman asked, tried:
|
||||
|
||||
|
||||
@@ -887,3 +887,11 @@ Read most (opens and searches):
|
||||
## Reading cost, chunk b25d0b8f6319, finished 2026-10-28T11:11:03Z
|
||||
|
||||
221959 new input tokens before the first commit.
|
||||
|
||||
## Reading cost, chunk c596ec8546d0, finished 2026-10-28T11:37:19Z
|
||||
|
||||
83591 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `rules/decisions.md` 1
|
||||
|
||||
+1
-1
@@ -32,7 +32,7 @@
|
||||
"type": "command"
|
||||
}
|
||||
],
|
||||
"matcher": "Bash|Edit|MultiEdit|Write|Agent"
|
||||
"matcher": "Agent|Bash|Edit|MultiEdit|Read|Write"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -6,22 +6,24 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 248`
|
||||
`Next ID = 251`
|
||||
|
||||
## Items
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 246 | 0.4.0 | | owner | **Give each review pass at most two rounds per chunk, with no third round and no ask for one: a fix answering round 2 merges once the gate passes.** | 3 | 2 | 3 | 6 | 3 | 1.2 by 5, 1.5 by 3, 1.4 by 2 | 10.0 |
|
||||
| 239 | 0.4.0 | | owner | **Block a Read, `cat`, `sed` or `head` of a file range already in the session's context, with a note saying so, and let it through once a compaction dropped the range or the file changed.** | 3 | 2 | 3 | 6 | 4 | 1.4 by 3 | 2.0 |
|
||||
| 244 | 0.4.0 | | owner | **Open every line loop-claude prints to its terminal with the machine's local time, as in `2026-10-28 11:46:01`.** | 2 | 2 | 1 | 3 | 3 | 1.5 by 2 | 1.9 |
|
||||
| 247 | 0.4.0 | | owner | **Count each finished chunk's escaped defects, the later commits that fix code it shipped, in `score-outcomes` and the routing figures.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4 | 1.7 |
|
||||
| 243 | 0.4.0 | | owner | **Show every loop-claude on the machine that waits for the human in one terminal app, and let the human answer each one there, `/talk` included.** | 4 | 2 | 3 | 7 | 6 | 1.5 by 4 | 0.9 |
|
||||
| 226 | 0.5.0 | | question | **Answer whether rule-hook's `run-heavy` rule should also catch a mutation run or docker build behind `uv run`, `poetry run`, `python3 -m`, `sudo`, `docker image build` and `docker compose up --build`, or narrow the `machine load:` decision's wording.** | 2 | 2 | 3 | 5 | 3 | 1.2 by 2, 1.5 by 2 | 4.0 |
|
||||
| 248 | 0.5.0 | | question | **Answer whether the `rule-hook:` decision should name every rule that takes no override, `repeat-read`, `reviewer-report` and `reviewer-rung`, with each one's reason.** | 1 | 2 | 1 | 2 | 1 | 1.2 by 2 | 3.3 |
|
||||
| 250 | 0.5.0 | | | **Move the repeat-read rule's transcript, mtime and file reads from `bin/rule_hook/main.py` into `bin/rule_hook/repeat_read.py`, behind one `*_breaches` entry point like the other rules, and name the tuple `results_on` returns.** | 2 | 1 | 3 | 4 | 3 | code health by 3 | 13.5 |
|
||||
| 235 | 0.5.0 | | decision | **Compare the first 20 chunks whose worker ran under the 200k compaction window with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and file an item where any rose.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 2, 1.4 by 3 | 6.8 |
|
||||
| 167 | 0.5.0 | | | **Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.** | 3 | 2 | 3 | 6 | 2 | 1.1 by 4 | 5.6 |
|
||||
| 6 | 0.5.0 | | decision | **Move what `loop-claude` writes out of `~/.claude.json`, and ask the human about each write that must stay.** | 3 | 3 | 3 | 7 | 3 | 1.1 by 5 | 5.3 |
|
||||
| 240 | 0.5.0 | | decision | **Give loop-claude's worker prompt the reading rule "never read more than 80 lines of a file at once, nor take a tool output over 3,000 characters into your context", and keep it only where the 20 chunks after it hold the 20 before's review rounds, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk.** | 1 | 2 | 3 | 4 | 2 | 1.4 by 3 | 4.0 |
|
||||
| 249 | 0.5.0 | | | **Have rule-hook's Edit, MultiEdit and Write checks read only a regular file, so a FIFO or device as `file_path` no longer hangs or floods the hook.** | 1 | 1 | 3 | 3 | 2 | 1.5 by 2 | 2.6 |
|
||||
| 1 | 0.6.0 | | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 1 | 2 | 1 | 2 | 1 | | 0.0 |
|
||||
| 229 | 0.6.0 | | | **Keep the rule for which commands reach run-heavy's docker beside that docker in `bin/run_heavy/capped_docker.py`, so rule-hook's `docker-run-memory` asks it instead of repeating run-heavy's PATH rules.** | 2 | 1 | 3 | 4 | 2 | code health by 3 | 18.0 |
|
||||
| 219 | 0.6.0 | | | **Name the project's rung `project` in `bin/judge_replay/main.py`, `bin/subagent_rung/main.py` and `bin/review_replay/main.py`, in place of `anchor`, as `bin/model_routing/ladder.py` and `bin/loop_claude` name it.** | 1 | 1 | 3 | 3 | 2 | code health by 2 | 12.0 |
|
||||
@@ -82,17 +84,13 @@
|
||||
|
||||
### 246. Give each review pass at most two rounds per chunk, with no third round and no ask for one: a fix answering round 2 merges once the gate passes.
|
||||
|
||||
lilleman's call, 2026-10-28, from a measurement of every repo's round titles since 2026-10-21. Round 1 ran 357 times and found 59 🔴. Round 2 ran about 200 times and found 12 🔴, mostly in round 1's fixes. Rounds 3 and later ran about 110 times and found no 🔴, at about $125 of review at list prices. Asks for a fourth or fifth round were 24 of the 59 interrupts in that time.
|
||||
lilleman's call, 2026-10-28, from a measurement of every repo's round titles since 2026-10-21. Round 1 ran 357 times and found 59 🔴. Round 2 ran about 200 times and found 12 🔴, mostly in round 1's fixes. Rounds 3 and later ran about 110 times and found no 🔴, at about $125 of review at list prices. Asks for a fourth or fifth round were 24 of the 59 interrupts in that time. Until this ships, lilleman answered no to a fourth principles round in PR #387, due only to tests a stability fix added after principles approved the code twice, and had the chunk merge without it.
|
||||
|
||||
- Replace `docs/decisions.md` → `review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human` with the new cap, and update its title in `AGENTS.md`. Its reason for three, that a lower cap asks the human more often, no longer holds once nothing asks.
|
||||
- Change `skills/larv-review/SKILL.md` → "Round cap" → "Cap", and every place it says three rounds or a fourth: the auto-reviewer included.
|
||||
- Have rule-hook's `round-cap` refuse a round commit titled past round 2, with no override. Settle item 220 inside this change, since it guards the same cap; drop it if this change makes it moot.
|
||||
- `bin/round_titles.py:4` `CAP` and `bin/rule_hook/rules.py:55` `round-title-counts` are places the comprehension seats found hard: the cap lives in SKILL.md prose while the code only counts. Make them easier to read.
|
||||
|
||||
### 239. Block a Read, `cat`, `sed` or `head` of a file range already in the session's context, with a note saying so, and let it through once a compaction dropped the range or the file changed.
|
||||
|
||||
lilleman's call, 2026-10-27, filed from PR #376's measurement of loop-claude's archive since 2026-10-21: a read of a file the agent read earlier in the thread, more than three messages before and without editing it in between, is 13% of the worker's reading, about $6 of the $45 its reads cost, and 1 to 2% of a reviewer's. How many of those repeat a range exactly is unmeasured, so the figures are a ceiling. A hook on `Read` and on the Bash reads `cat`, `sed` and `head` blocks such a read with a short note that the content is already in context and unchanged. It lets the read through after a compaction, when the content has left the context, or when the file changed since the read. A PreToolUse hook gets a `transcript_path`, which holds the earlier reads and each `compact_boundary`, so it may need no state of its own; check whether a hook inside a subagent gets the subagent's own transcript.
|
||||
|
||||
### 244. Open every line loop-claude prints to its terminal with the machine's local time, as in `2026-10-28 11:46:01`.
|
||||
|
||||
lilleman's call, 2026-10-28, who also chose the spelling: `YYYY-MM-DD HH:MM:SS` in the machine's local time, never UTC, so the human sees when each line happened, such as how long a question has waited. Today about 27 `print` calls across `bin/loop_claude/` write bare lines to stderr. Route them through one function that adds the stamp, taking the time from `Effects.now` so tests can fix it. A line of a multi-line message, such as a question, gets the stamp only on its first line.
|
||||
@@ -118,6 +116,14 @@ Bench the app's command name and how it shows a question with readers seated as
|
||||
|
||||
Found by review of PR #370. Today `uv run mutmut run`, `poetry run mutmut run`, `python3 -m mutmut run`, `sudo docker build .`, `docker image build .` and `docker compose up --build` pass the rule, which `docs/decisions.md` → `machine load: …` and `RULES["run-heavy"]` describe as blocking docker builds and mutation runs. The rule's own text says a gate script isn't checked since no program can tell it from a light one, so the human decides where the line sits. Say what the rule's "checked" text promises either way. Where the answer widens the rule, the change touches `heavy` in `bin/rule_hook/heavy_command.py`, which the comprehension seats found hard: make it easier to read.
|
||||
|
||||
### 248. Answer whether the `rule-hook:` decision should name every rule that takes no override, `repeat-read`, `reviewer-report` and `reviewer-rung`, with each one's reason.
|
||||
|
||||
Filed from PR #387's architecture round 1. The decision's 2026-10-16 addendum says `reviewer-rung` is the one rule no marker overrides. `reviewer-report` already took no marker, and PR #387 adds `repeat-read`, so three rules take none, each with its reason in a comment above it in `bin/rule_hook/rules.py`; only `reviewer-report`'s is also in a decision, the `review:` one. A reader of the decision, the judge included, believes every other rule takes an override. The fix rewords the title and the addendum to name all three; it changes a decision, so it is lilleman's.
|
||||
|
||||
### 250. Move the repeat-read rule's transcript, mtime and file reads from `bin/rule_hook/main.py` into `bin/rule_hook/repeat_read.py`, behind one `*_breaches` entry point like the other rules, and name the tuple `results_on` returns.
|
||||
|
||||
Found by the paired ruling of PR #387: all four seats ruled Locality worse at `read_breaches`, `modified` and `first_lines` in `main.py`. The rule's reads sit there while its logic sits in `repeat_read.py`, and two contracts cross the seam: `modified` must stamp the mtime as the transcript stamps entries, since `shown_lines` compares the two as text, and `first_lines` must read one line past the last shown one, which `repeat_breaches` needs. Make these places easier to read: keep both contracts inside `repeat_read.py`, the way `todo_check.py` and `reviewer_report.py` do their own reads. Replace `results_on`'s positional four-part tuple with a named type, and rename `repeat_read.written`, which names a path while `written` in `main.py` names the text that may carry an override marker.
|
||||
|
||||
### 235. Compare the first 20 chunks whose worker ran under the 200k compaction window with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and file an item where any rose.
|
||||
|
||||
`docs/decisions.md` → `loop-claude: the worker's claude compacts its context before it fills a 200k window…` expects a saving of about 6% of all spend with no rise in the other figures. Measure from loop-claude's archive as that decision did: rounds past each pass's first and the 🔴, 🟡 and misread counts from the round commit titles, unfinished verdicts from the outcomes files, hours from the worker's first message to its last, and dollars at list prices, cache reads included, from the chunk's `cost-state` entry, which counts the compaction calls that no message's usage records; the routing figures now put such dollars in. A chunk ran under the window when its first worker started after that decision merged.
|
||||
@@ -134,6 +140,10 @@ Claude Code owns that file and writes it. `trust` reads it, changes it and write
|
||||
|
||||
lilleman's call, 2026-10-27, answering PR #376's trials, recorded in `docs/decisions.md` → `loop-claude: the worker's claude compacts its context before it fills a 200k window…`: three worker-like tasks each ran four ways, as workers run today, with an outline tool, with the LSP tool and under the rule, and only the rule cut reading, by 45% to 60%, at the same or lower cost and with the tasks done as well. The rule is for workers, so it goes into the prompt loop-claude gives a chunk's worker (`bin/loop_claude/cycle/prompts.py`), not into `CLAUDE.md`, which reviewers load too and where the one reviewer run under a like rule found 2 of 6 findings. A worker that reads less may misread more, so measure the 20 chunks after the rule merges against the 20 before, as item 235 measures the compaction window, and drop the rule where any of those figures rose.
|
||||
|
||||
### 249. Have rule-hook's Edit, MultiEdit and Write checks read only a regular file, so a FIFO or device as `file_path` no longer hangs or floods the hook.
|
||||
|
||||
Found by the stability review of PR #387. `read` in `bin/rule_hook/main.py`, which serves these checks, opens the target with `pathlib.Path(path).read_text` whatever it is. A FIFO without a writer hangs rule-hook until Claude Code's hook timeout, and a path such as `/dev/zero` makes it read without bound. rule-hook runs on every such call in every session. Reuse the check `modified` in the same file makes, `stat.S_ISREG` before any read, and have `read` return `""` for any other path. A test passes a FIFO and expects the check to finish.
|
||||
|
||||
### 1. Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.
|
||||
|
||||
It lives under `~/.claude`, which `.gitignore` keeps from the remote, so a lost disk loses every unattended run's record. Back it up only to a destination private to me and encrypted at rest: the worker runs with permissions bypassed, so the archived tool results can hold credentials it read.
|
||||
|
||||
Reference in New Issue
Block a user