Merge pull request 'Refuse a review launch below the rung loop-claude gives its worker' (#295) from item-163 into main
This commit was merged in pull request #295.
This commit is contained in:
@@ -8,6 +8,7 @@
|
||||
- An item that restores a maintain goal the project breaks today is exempt as `defect`, and its Moves may name that maintain goal, weighed as the heaviest achieve goal. A defect ranks by its score like any item. Only two kinds rank ahead of the rest of their release: questions first, since an answer may change other items, then security items.
|
||||
- The stability reviewer applies `principles/technical-principles.md` as written, in place of its own copies of it. Its test-data check now asks for UUID v7, where it asked for randomised UUIDs.
|
||||
- larv-review's principles audit is a pass of its own, and runs on every chunk that changes code, config or tests, not only on one with an architectural decision. A chunk of prose or planning alone skips it.
|
||||
- `rule-hook` refuses an Agent call that launches a reviewer, a comprehension seat or the security pass on a rung below the one `loop-claude` names for it, and takes no override: pass the model and effort the refusal names. A reviewer is held to the higher of the worker's rung and the project's; a seat or the security pass to the project's.
|
||||
|
||||
## 0.1.0
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `metric-bench [history]` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest; `metric-bench --help` states the metrics and the scores.
|
||||
- `rank_todo.py [--check] [todo.md]` scores and ranks a project's `todo.md`; `rank_todo.py --help` states the format.
|
||||
- `reading-cost [--reset] [record]` appends a figure for each chunk `loop-claude` finished to a project's `docs/reading-cost.md`, and says whether the figures rise; `reading-cost --help` states the measure.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Bash, Edit, MultiEdit and Write call, and `rule-hook --help` states the rules and the override.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Bash, Edit, MultiEdit, Write and Agent call, and `rule-hook --help` states the rules and the override.
|
||||
- `score-outcomes [R S C]` lists the finished items whose guessed R, S and C sit nearest the given scores, with what their chunks measured, and each score's median outcome; `score-outcomes --help` states the measures.
|
||||
- `stability-replay <model>` replays archived stability reviews of the projects run on the model, on the code each reviewed, and prints the lowest rung below the project's that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again; `stability-replay --help` states the replay. Run it only in a sandbox, like loop-claude: each replayed reviewer has Bash and skips every permission prompt, and an archived prompt may name paths on the host.
|
||||
- `statusline.py` renders Claude Code's status line; `settings.json` → `statusLine` runs it.
|
||||
|
||||
@@ -13,7 +13,7 @@ from loop_claude.project.checkout import worktrees
|
||||
ANSWER_QUESTION = "answer the question above"
|
||||
|
||||
|
||||
def interrupt(root: str, session_id: uuid.UUID, worker: Rung, path: str, question: str, paths: Paths, effects: Effects) -> Stop | None:
|
||||
def interrupt(root: str, session_id: uuid.UUID, worker: Rung, path: str, question: str, paths: Paths, env: dict[str, str], effects: Effects) -> Stop | None:
|
||||
print(f"\n{question}", file=sys.stderr)
|
||||
if stop := needs_terminal(ANSWER_QUESTION, effects.on_terminal()):
|
||||
return stop
|
||||
@@ -21,7 +21,7 @@ def interrupt(root: str, session_id: uuid.UUID, worker: Rung, path: str, questio
|
||||
if isinstance(entries, Failure):
|
||||
return Stop(1, entries.reason)
|
||||
before = human_turns(entries)
|
||||
if unstarted := hand_over(worker, ["--resume", str(session_id)], root, paths, effects):
|
||||
if unstarted := hand_over(worker, ["--resume", str(session_id)], root, paths, env, effects):
|
||||
return Stop(1, f"could not start claude for the interrupt: {unstarted.reason}")
|
||||
entries = read_transcript(path)
|
||||
if isinstance(entries, Failure):
|
||||
@@ -29,11 +29,11 @@ def interrupt(root: str, session_id: uuid.UUID, worker: Rung, path: str, questio
|
||||
return None if human_turns(entries) > before else needs_human(ANSWER_QUESTION, "the interrupt ended with nothing typed")
|
||||
|
||||
|
||||
def hand_over(rung: Rung, args: list[str], root: str, paths: Paths, effects: Effects) -> Failure | None:
|
||||
"""Runs claude on the terminal in root on rung, the checkout and each of its worktrees first marked trusted; the failure is why claude could not start."""
|
||||
def hand_over(rung: Rung, args: list[str], root: str, paths: Paths, env: dict[str, str], effects: Effects) -> Failure | None:
|
||||
"""Marks the checkout and each of its worktrees trusted, then runs claude on the terminal in root, on rung, with env added to its environment; the failure is why claude could not start."""
|
||||
trust(paths.claude_json, worktrees(root, effects))
|
||||
try:
|
||||
effects.claude_on_terminal([*rung.flags(), "--permission-mode", "bypassPermissions", *args], root)
|
||||
effects.claude_on_terminal([*rung.flags(), "--permission-mode", "bypassPermissions", *args], root, env)
|
||||
except OSError as failure:
|
||||
return Failure(str(failure.strerror))
|
||||
return None
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from model_routing.ladder import climbed, highest, LADDER, Rung
|
||||
from model_routing.ladder import climbed, LADDER, review_rung, Rung
|
||||
from model_routing.route import ESCALATION_MARKER, Route
|
||||
from loop_claude.routing import escalation_rungs
|
||||
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
|
||||
@@ -31,7 +31,7 @@ def off_the_projects_rung(route: Route, anchor: Rung | None) -> str:
|
||||
if anchor is None or route.rung == anchor:
|
||||
return ""
|
||||
# Below the project's rung, the reviewers stay on it to hold quality; above it, the seats score every chunk on one rung, and the security pass spends no more than the project chose.
|
||||
below = highest(route.rung, anchor) == anchor
|
||||
below = review_rung("reviewer", anchor, route.rung) == anchor
|
||||
launched = "each reviewer, each comprehension seat and the security pass's subagent" if below else "each comprehension seat and the security pass's subagent"
|
||||
planning = f" Launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan {anchor.model} <R+C>` prints." if below else ""
|
||||
return f" Your rung is {'below' if below else 'above'} the project's, {anchor}: launch {launched} with the Agent tool set to {anchor.agent_args()}. Run a full scan only from such a subagent, since the scan's agents run on the model that starts them.{planning}"
|
||||
@@ -56,8 +56,8 @@ def reruns(route: Route, anchor: Rung | None) -> str:
|
||||
if anchor is None or route.rung not in LADDER:
|
||||
return ""
|
||||
# The rungs off_the_projects_rung launches them on: the seats always on anchor, the reviewers on anchor only while the worker runs below it.
|
||||
reviewer = highest(route.rung, anchor)
|
||||
rerun = {name: climbed(rung, 1) for name, rung in (("reviewer", reviewer), ("comprehension seat", anchor)) if rung != LADDER[-1]}
|
||||
launched = (("reviewer", review_rung("reviewer", anchor, route.rung)), ("comprehension seat", review_rung("seat", anchor, route.rung)))
|
||||
rerun = {name: climbed(rung, 1) for name, rung in launched if rung != LADDER[-1]}
|
||||
if not rerun:
|
||||
return ""
|
||||
if len(set(rerun.values())) == 1:
|
||||
|
||||
@@ -19,6 +19,7 @@ from model_routing.route import Route
|
||||
from loop_claude.cycle.worker import run_worker, worker_args
|
||||
from model_routing.ladder import Rung
|
||||
from loop_claude.stop import Stop
|
||||
from model_routing import worker_rungs
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -84,12 +85,17 @@ def work(cycle: Cycle, answered: QuotedAnswer | None) -> WorkerRan | Stop:
|
||||
if stop := keep_answer(cycle, answered):
|
||||
return stop
|
||||
# worker_start runs inside the lambda: a failed attempt may have opened the session.
|
||||
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered), cycle.effects), cycle.effects):
|
||||
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered), worker_environment(cycle), cycle.effects), cycle.effects):
|
||||
return stop
|
||||
path = transcript_file(cycle.session)
|
||||
return WorkerRan(path) if path is not None else Stop(1, f"claude left no transcript for session {cycle.session.id}")
|
||||
|
||||
|
||||
def worker_environment(cycle: Cycle) -> dict[str, str]:
|
||||
"""The rungs the worker's claude is handed, which rule-hook holds its review launches to."""
|
||||
return worker_rungs.environment(cycle.anchor, cycle.route.rung)
|
||||
|
||||
|
||||
def worker_start(cycle: Cycle, answered: QuotedAnswer | None) -> list[str]:
|
||||
if answered is not None:
|
||||
return worker_args(cycle.session, cycle.route.rung, prompts.answered(answered.question, answered.answer), resume=True)
|
||||
@@ -118,7 +124,7 @@ def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) ->
|
||||
# Archived now, so the transcript the question was asked on is kept with it; the human's answer then grows the transcript.
|
||||
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.interrupted(question, ruling.evidence, str(judge_of(cycle))), cycle.effects.now()):
|
||||
return Stop(1, failure.reason)
|
||||
return interrupt(cycle.root, cycle.session.id, cycle.route.rung, transcript_path, question, cycle.paths, cycle.effects) or HumanAnswered()
|
||||
return interrupt(cycle.root, cycle.session.id, cycle.route.rung, transcript_path, question, cycle.paths, worker_environment(cycle), cycle.effects) or HumanAnswered()
|
||||
|
||||
|
||||
def question_for(ruling: Interrupt | QuotedAnswer) -> str:
|
||||
|
||||
@@ -9,9 +9,9 @@ from claude_code.session import Session
|
||||
PRINTED_TOOL_INPUT = 160
|
||||
|
||||
|
||||
def run_worker(root: str, args: list[str], effects: Effects) -> Failure | Limited | None:
|
||||
def run_worker(root: str, args: list[str], env: dict[str, str], effects: Effects) -> Failure | Limited | None:
|
||||
try:
|
||||
proc = effects.stream_claude(args, root)
|
||||
proc = effects.stream_claude(args, root, env)
|
||||
except OSError as failure:
|
||||
return Failure(f"could not start claude: {failure.strerror}")
|
||||
result: dict | None = None
|
||||
|
||||
@@ -17,7 +17,7 @@ class Streaming(typing.Protocol):
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Effects:
|
||||
claude: typing.Callable[[list[str], str, str], subprocess.CompletedProcess[str]]
|
||||
claude_on_terminal: typing.Callable[[list[str], str], int]
|
||||
claude_on_terminal: typing.Callable[[list[str], str, dict[str, str]], int]
|
||||
execv: typing.Callable[[str, list[str]], None]
|
||||
git: typing.Callable[[list[str], str], subprocess.CompletedProcess[str]]
|
||||
input: typing.Callable[[str], str]
|
||||
@@ -26,12 +26,12 @@ class Effects:
|
||||
on_terminal: typing.Callable[[], bool]
|
||||
python: typing.Callable[[list[str]], subprocess.CompletedProcess[str]]
|
||||
sleep: typing.Callable[[float], None]
|
||||
stream_claude: typing.Callable[[list[str], str], Streaming]
|
||||
stream_claude: typing.Callable[[list[str], str, dict[str, str]], Streaming]
|
||||
|
||||
|
||||
class StreamingClaude:
|
||||
def __init__(self, args: list[str], cwd: str) -> None:
|
||||
self.proc = subprocess.Popen(["claude", *args], cwd=cwd, encoding="utf-8", errors="replace", stdin=subprocess.DEVNULL, stdout=subprocess.PIPE)
|
||||
def __init__(self, args: list[str], cwd: str, env: dict[str, str]) -> None:
|
||||
self.proc = subprocess.Popen(["claude", *args], cwd=cwd, env={**os.environ, **env}, encoding="utf-8", errors="replace", stdin=subprocess.DEVNULL, stdout=subprocess.PIPE)
|
||||
|
||||
def lines(self) -> typing.Iterable[str]:
|
||||
with self.proc:
|
||||
@@ -44,7 +44,7 @@ class StreamingClaude:
|
||||
def system_effects() -> Effects:
|
||||
return Effects(
|
||||
claude=lambda args, cwd, stdin: subprocess.run(["claude", *args], capture_output=True, cwd=cwd, input=stdin, text=True),
|
||||
claude_on_terminal=lambda args, cwd: subprocess.run(["claude", *args], cwd=cwd).returncode,
|
||||
claude_on_terminal=lambda args, cwd, env: subprocess.run(["claude", *args], cwd=cwd, env={**os.environ, **env}).returncode,
|
||||
execv=os.execv,
|
||||
git=lambda args, cwd: subprocess.run(["git", *args], capture_output=True, cwd=cwd, text=True),
|
||||
input=input,
|
||||
|
||||
@@ -53,10 +53,12 @@ reviewer, each comprehension seat and the security pass in subagents on the chea
|
||||
is above, the prompt has the worker launch only each comprehension seat and the security pass on
|
||||
the cheapest rung. The prompt after each unfinished verdict
|
||||
says this again, since a step-up may have taken the worker above the cheapest rung. A reviewer or
|
||||
seat that rules the work below standard may ask to rerun one rung up; each prompt names that rung. The
|
||||
judge reports the highest R + C the worker states, and the next cycle steps up to the rung that
|
||||
R + C picks, never down. Every second unfinished verdict on a chunk steps it up one rung. So does
|
||||
an unfinished verdict on a worker that ended its run asking for a higher rung, once the judge quotes
|
||||
seat that rules the work below standard may ask to rerun one rung up; each prompt names that rung.
|
||||
loop-claude hands the worker's claude its own rung and the project's, in LOOP_CLAUDE_WORKER_RUNG
|
||||
and LOOP_CLAUDE_PROJECT_RUNG, and rule-hook refuses a review launched below the rung named for it.
|
||||
The judge reports the highest R + C the worker states, and the next
|
||||
cycle steps up to the rung that R + C picks, never down. Every second unfinished verdict on a chunk steps it up one rung.
|
||||
So does an unfinished verdict on a worker that ended its run asking for a higher rung, once the judge quotes
|
||||
the transcript showing that the escalation trigger it named fired
|
||||
(~/.claude/CLAUDE.md → "Working with the codebase" lists them). That step stops two rungs above the rung the chunk's item picks, and the
|
||||
worker's prompts offer it only while the worker is below that limit. Each
|
||||
|
||||
@@ -51,7 +51,7 @@ def raise_calls(root: pathlib.Path, new: list[str], rung: Rung, paths: Paths, ef
|
||||
print("Not on a terminal: the calls are raised again at the next measurement.", file=sys.stderr)
|
||||
return None
|
||||
prompt = f"Put these calls for a routing change from loop-claude's figures to the human, and change nothing until they answer:\n{listed}The figures are in {root / archive.FIGURES}. The ladder, its bands and the delegation rule are docs/decisions.md → `routing:`, and a change to them is the human's. Record what they decide as ~/.claude/CLAUDE.md says; the next chunk of this repo files the plan item for these calls."
|
||||
if unstarted := hand_over(rung, [prompt], str(paths.claude_dir), paths, effects):
|
||||
if unstarted := hand_over(rung, [prompt], str(paths.claude_dir), paths, {}, effects):
|
||||
return Stop(1, f"could not start claude for the routing calls: {unstarted.reason}")
|
||||
# Raised once the human has seen them, answered or not: an unanswered call reaches the plan through AGENTS.md, so it is not asked twice.
|
||||
if unmarked := archive.mark_raised(root, new, utc(effects.now())):
|
||||
|
||||
@@ -52,11 +52,13 @@ class FakeEffects:
|
||||
self.calls = []
|
||||
self.scripts = {"claude": list(claude), "claude_on_terminal": list(on_terminal_claude), "git": list(git), "input": list(inputs), "python": list(python), "stream_claude": list(streams)}
|
||||
self.terminal = on_terminal
|
||||
self.worker_environments = []
|
||||
self.terminal_environments = []
|
||||
|
||||
def effects(self):
|
||||
return Effects(
|
||||
claude=lambda args, cwd, stdin: self.answer("claude", (args, stdin)),
|
||||
claude_on_terminal=lambda args, cwd: self.answer("claude_on_terminal", args),
|
||||
claude_on_terminal=self.on_terminal_claude,
|
||||
execv=self.execv,
|
||||
git=lambda args, cwd: self.answer("git", args),
|
||||
input=lambda prompt: self.answer("input", prompt),
|
||||
@@ -65,9 +67,17 @@ class FakeEffects:
|
||||
on_terminal=lambda: self.terminal,
|
||||
python=lambda args: self.answer("python", args),
|
||||
sleep=lambda seconds: self.calls.append(("sleep", seconds)),
|
||||
stream_claude=lambda args, cwd: self.answer("stream_claude", args),
|
||||
stream_claude=self.stream,
|
||||
)
|
||||
|
||||
def on_terminal_claude(self, args, cwd, env):
|
||||
self.terminal_environments.append(env)
|
||||
return self.answer("claude_on_terminal", args)
|
||||
|
||||
def stream(self, args, cwd, env):
|
||||
self.worker_environments.append(env)
|
||||
return self.answer("stream_claude", args)
|
||||
|
||||
def answer(self, seam, what):
|
||||
self.calls.append((seam, what))
|
||||
answer = self.scripts[seam].pop(0)
|
||||
@@ -186,6 +196,23 @@ class Cycles(unittest.TestCase):
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\nJudge: judge-model\nWorker: worker-model\n"])
|
||||
self.assertIsNone(session.transcript_file(self.session))
|
||||
|
||||
def test_a_worker_is_handed_the_project_and_its_own_rung_in_its_environment(self):
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
|
||||
cycle = dataclasses.replace(self.cycle(fake), anchor=Rung("claude-opus-5-5", "medium"), route=Route(Rung("claude-haiku-5-5", "max"), 163, 5))
|
||||
self.assertEqual(run.run_cycle(cycle), Finished("working"))
|
||||
self.assertEqual(fake.worker_environments, [{"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort"}])
|
||||
|
||||
def test_the_claude_an_interrupt_resumes_is_handed_the_same_rungs_as_the_worker(self):
|
||||
def answer(args):
|
||||
with open(session.transcript_file(self.session), "a") as handle:
|
||||
handle.write(json.dumps(human("the second")) + "\n")
|
||||
return 0
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt"), judged("finished")], on_terminal_claude=[answer], git=[ran()])
|
||||
cycle = dataclasses.replace(self.cycle(fake), anchor=Rung("claude-opus-5-5", "medium"), route=Route(Rung("claude-haiku-5-5", "max"), 163, 5))
|
||||
self.assertEqual(run.run_cycle(cycle), Finished("working"))
|
||||
self.assertEqual(fake.terminal_environments, fake.worker_environments)
|
||||
self.assertEqual(fake.terminal_environments[0]["LOOP_CLAUDE_WORKER_RUNG"], "claude-haiku-5-5 at max effort")
|
||||
|
||||
def test_an_unfinished_chunks_transcript_is_archived_and_the_next_cycle_resumes_its_session_with_the_judges_evidence(self):
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("unfinished")] * 2)
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
|
||||
@@ -333,7 +360,7 @@ class Limits(unittest.TestCase):
|
||||
allowed = {"type": "rate_limit_event", "rate_limit_info": {"status": "allowed", "resetsAt": NOW + 9000}}
|
||||
limited = {"type": "result", "api_error_status": 429, "result": "You've hit your session limit"}
|
||||
fake = FakeEffects(streams=[Streamed(1, rejected[1], allowed, rejected[0], limited)])
|
||||
self.assertEqual(worker.run_worker("/r", [], fake.effects()), Limited(NOW + 600, "You've hit your session limit"))
|
||||
self.assertEqual(worker.run_worker("/r", [], {}, fake.effects()), Limited(NOW + 600, "You've hit your session limit"))
|
||||
|
||||
def test_a_limit_outranks_the_exit_code_and_its_message_fits_one_line(self):
|
||||
def limited(**said):
|
||||
@@ -357,7 +384,7 @@ class Limits(unittest.TestCase):
|
||||
class Interrupts(unittest.TestCase):
|
||||
def test_stops_off_a_terminal_before_starting_claude(self):
|
||||
fake = FakeEffects(on_terminal=False)
|
||||
stop = interrupt.interrupt("/r", uuid7(), Rung("m"), "/t", "Which one?", paths.paths_from(pathlib.Path("/h")), fake.effects())
|
||||
stop = interrupt.interrupt("/r", uuid7(), Rung("m"), "/t", "Which one?", paths.paths_from(pathlib.Path("/h")), {}, fake.effects())
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"))
|
||||
self.assertEqual(fake.calls, [])
|
||||
|
||||
@@ -366,10 +393,19 @@ class Interrupts(unittest.TestCase):
|
||||
path = pathlib.Path(home, "t.jsonl")
|
||||
path.write_text(json.dumps(human("a")) + "\n")
|
||||
fake = FakeEffects(on_terminal_claude=[0], git=[ran()])
|
||||
stop = interrupt.interrupt(home, uuid7(), Rung("m", "high"), str(path), "Which one?", paths.paths_from(pathlib.Path(home)), fake.effects())
|
||||
stop = interrupt.interrupt(home, uuid7(), Rung("m", "high"), str(path), "Which one?", paths.paths_from(pathlib.Path(home)), {}, fake.effects())
|
||||
self.assertEqual(fake.made("claude_on_terminal")[0][:4], ["--model", "m", "--effort", "high"])
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
|
||||
|
||||
def test_hands_the_claude_that_takes_the_terminal_the_workers_environment(self):
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
path = pathlib.Path(home, "t.jsonl")
|
||||
path.write_text(json.dumps(human("a")) + "\n")
|
||||
fake = FakeEffects(on_terminal_claude=[0], git=[ran()])
|
||||
handed = {"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort"}
|
||||
interrupt.interrupt(home, uuid7(), Rung("claude-haiku-5-5", "max"), str(path), "Which one?", paths.paths_from(pathlib.Path(home)), handed, fake.effects())
|
||||
self.assertEqual(fake.terminal_environments, [handed])
|
||||
|
||||
|
||||
class RoutingCheck(unittest.TestCase):
|
||||
def setUp(self):
|
||||
@@ -416,6 +452,7 @@ class RoutingCheck(unittest.TestCase):
|
||||
((args),) = fake.made("claude_on_terminal")
|
||||
self.assertEqual(args[:6], ["--model", "m", "--effort", "high", "--permission-mode", "bypassPermissions"])
|
||||
self.assertIn(f"change nothing until they answer:\n- {call}", args[6])
|
||||
self.assertEqual(fake.terminal_environments, [{}])
|
||||
(self.root / "routing-figures.md").write_text("Measured x, at cycle 0 across")
|
||||
self.assertEqual(self.check(FakeEffects())[0], None)
|
||||
self.assertNotIn(call, self.check(FakeEffects())[1])
|
||||
|
||||
@@ -1,5 +1,12 @@
|
||||
"""The routing ladder, and the rung an R + C picks from the project's rung: docs/decisions.md → `routing:`."""
|
||||
import dataclasses
|
||||
import re
|
||||
import typing
|
||||
|
||||
# How Rung.__str__ spells a rung that names an effort, and Rung.parse reads it back.
|
||||
WORDING = re.compile(r"(?P<model>\S+) at (?P<effort>\S+) effort")
|
||||
# What a review launch reviews: a reviewer, a comprehension seat or the security pass.
|
||||
ReviewKind = typing.Literal["reviewer", "seat", "security"]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -11,14 +18,32 @@ class Rung:
|
||||
def flags(self) -> list[str]:
|
||||
return ["--model", self.model, *(["--effort", self.effort] if self.effort else [])]
|
||||
|
||||
def family(self) -> str:
|
||||
"""The Agent tool's name for the model, such as opus for claude-opus-5-5."""
|
||||
return self.model.split("-")[1]
|
||||
|
||||
def agent_args(self) -> str:
|
||||
"""The rung in the Agent tool's words. The tool takes a family name, such as opus, and runs that family's latest model, which is the model each rung names."""
|
||||
return f"model {self.model.split('-')[1]}, effort {self.effort}"
|
||||
return f"model {self.family()}, effort {self.effort}"
|
||||
|
||||
# The archive records this wording, and model_routing.route.ROUTED reads it back to resume a chunk on its rung.
|
||||
@staticmethod
|
||||
def from_agent(model: str, effort: str) -> "Rung":
|
||||
"""The rung the Agent tool's model and effort name: a family such as opus is its ladder model, at the effort given or else its cheapest. A family off the ladder, or an effort off it, gives a rung that is not on LADDER."""
|
||||
family = next((rung for rung in LADDER if rung.family() == model), None)
|
||||
if family is None:
|
||||
return Rung(model, effort)
|
||||
return Rung(family.model, effort) if effort else family
|
||||
|
||||
# The one spelling of a rung. It is persisted, so changing it routes every stored chunk afresh: parse no longer reads their rungs.
|
||||
def __str__(self) -> str:
|
||||
return f"{self.model} at {self.effort} effort" if self.effort else self.model
|
||||
|
||||
@staticmethod
|
||||
def parse(text: str) -> "Rung | None":
|
||||
"""The rung that names an effort, which str wrote, whether or not it is on the ladder; None for any other text, such as a bare model."""
|
||||
found = WORDING.fullmatch(text)
|
||||
return Rung(found["model"], found["effort"]) if found else None
|
||||
|
||||
|
||||
# Least competent first.
|
||||
LADDER = (
|
||||
@@ -59,6 +84,11 @@ def stepped_up(rung: Rung) -> Rung:
|
||||
return climbed(rung, 1)
|
||||
|
||||
|
||||
def review_rung(kind: ReviewKind, project: Rung, worker: Rung) -> Rung:
|
||||
"""The rung a review runs on: a reviewer on the higher of the worker's rung and the project's; a seat or the security pass on the project's."""
|
||||
return highest(project, worker) if kind == "reviewer" else project
|
||||
|
||||
|
||||
def highest(*rungs: Rung) -> Rung:
|
||||
"""The most competent of the rungs; one off the ladder ranks below every rung on it."""
|
||||
return max(rungs, key=position)
|
||||
|
||||
@@ -6,7 +6,7 @@ from model_routing.ladder import Rung
|
||||
|
||||
# How a worker opens the prompt of a subagent it escalates a task to, `Escalation trigger 2: …`: docs/decisions.md → `routing: a worker hands a task one rung up`.
|
||||
ESCALATION_MARKER = "Escalation trigger"
|
||||
ROUTED = re.compile(r"(?P<model>\S+) at (?P<effort>\S+) effort(?:, routed by item (?P<item>[0-9]+)'s R \+ C of (?P<sum>[0-9]+))?")
|
||||
ROUTED = re.compile(r"(?P<rung>.+?)(?:, routed by item (?P<item>[0-9]+)'s R \+ C of (?P<sum>[0-9]+))?")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -30,4 +30,5 @@ def route_from(worker: str) -> Route | None:
|
||||
found = ROUTED.fullmatch(worker)
|
||||
if not found:
|
||||
return None
|
||||
return Route(Rung(found["model"], found["effort"]), int(found["item"] or 0), int(found["sum"] or 0))
|
||||
rung = Rung.parse(found["rung"])
|
||||
return Route(rung, int(found["item"] or 0), int(found["sum"] or 0)) if rung else None
|
||||
|
||||
@@ -4,7 +4,7 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from model_routing.ladder import floored, highest, LADDER, picked, project_rung, Rung, stepped_up
|
||||
from model_routing.ladder import floored, highest, LADDER, picked, project_rung, review_rung, Rung, stepped_up
|
||||
|
||||
OPUS = Rung("claude-opus-5-5", "medium")
|
||||
|
||||
@@ -51,6 +51,44 @@ class Ladder(unittest.TestCase):
|
||||
self.assertEqual(str(Rung("claude-opus-5-5", "high")), "claude-opus-5-5 at high effort")
|
||||
self.assertEqual(str(Rung("worker-model")), "worker-model")
|
||||
|
||||
def test_parses_what_str_wrote_for_each_rung_on_the_ladder(self):
|
||||
for rung in LADDER:
|
||||
with self.subTest(rung=rung):
|
||||
self.assertEqual(Rung.parse(str(rung)), rung)
|
||||
|
||||
def test_parses_a_rung_off_the_ladder_too_for_the_caller_to_judge(self):
|
||||
self.assertEqual(Rung.parse("claude-opus-5-5 at low effort"), Rung("claude-opus-5-5", "low"))
|
||||
|
||||
def test_parses_no_other_wording_as_a_rung(self):
|
||||
for text in ("", "claude-opus-5-5", "claude-opus-5-5 medium", "claude-opus-5-5 at medium", "at medium effort", "a b at medium effort"):
|
||||
with self.subTest(text=text):
|
||||
self.assertIsNone(Rung.parse(text))
|
||||
|
||||
def test_reads_the_agent_tools_words_back_as_a_rung(self):
|
||||
for rung in LADDER:
|
||||
with self.subTest(rung=rung):
|
||||
model, effort = (part.split(" ")[1] for part in rung.agent_args().split(", "))
|
||||
self.assertEqual(Rung.from_agent(model, effort), rung)
|
||||
|
||||
def test_reads_a_family_named_without_an_effort_as_its_cheapest_rung(self):
|
||||
self.assertEqual(Rung.from_agent("opus", ""), OPUS)
|
||||
self.assertEqual(Rung.from_agent("sonnet", ""), Rung("claude-sonnet-5-5", "high"))
|
||||
|
||||
def test_reads_an_unknown_family_or_an_effort_off_the_ladder_as_a_rung_off_it(self):
|
||||
for model, effort in (("gpt", "high"), ("gpt", ""), ("opus", "low"), ("haiku", "bogus")):
|
||||
with self.subTest(model=model, effort=effort):
|
||||
self.assertNotIn(Rung.from_agent(model, effort), LADDER)
|
||||
self.assertEqual(Rung.from_agent("opus", "low"), Rung("claude-opus-5-5", "low"))
|
||||
|
||||
def test_holds_a_reviewer_to_the_higher_of_the_workers_rung_and_the_projects_and_anything_else_to_the_projects(self):
|
||||
haiku, fable = LADDER[4], LADDER[-1]
|
||||
self.assertEqual(review_rung("reviewer", OPUS, haiku), OPUS)
|
||||
self.assertEqual(review_rung("reviewer", OPUS, fable), fable)
|
||||
for kind in ("seat", "security"):
|
||||
with self.subTest(kind=kind):
|
||||
self.assertEqual(review_rung(kind, OPUS, fable), OPUS)
|
||||
self.assertEqual(review_rung(kind, OPUS, haiku), OPUS)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
import unittest
|
||||
|
||||
from model_routing.worker_rungs import environment, handed
|
||||
from model_routing.ladder import Rung
|
||||
|
||||
|
||||
class Environment(unittest.TestCase):
|
||||
PROJECT = Rung("claude-opus-5-5", "medium")
|
||||
WORKER = Rung("claude-haiku-5-5", "max")
|
||||
|
||||
def test_the_project_and_the_worker_rung_are_named_in_the_environment(self):
|
||||
self.assertEqual(environment(self.PROJECT, self.WORKER), {"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort"})
|
||||
|
||||
def test_a_project_off_the_ladder_is_handed_an_empty_rung_so_none_is_inherited(self):
|
||||
self.assertEqual(environment(None, self.WORKER), {"LOOP_CLAUDE_PROJECT_RUNG": "", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort"})
|
||||
|
||||
def test_handed_reads_back_what_environment_named(self):
|
||||
self.assertEqual(handed(environment(self.PROJECT, self.WORKER)), (self.PROJECT, self.WORKER))
|
||||
self.assertEqual(handed(environment(None, self.WORKER)), (None, self.WORKER))
|
||||
|
||||
def test_a_missing_or_unknown_rung_reads_as_none(self):
|
||||
self.assertEqual(handed({}), (None, None))
|
||||
self.assertEqual(handed({"LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 bogus"}), (None, None))
|
||||
self.assertEqual(handed({"LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at bogus effort"}), (None, None))
|
||||
@@ -0,0 +1,24 @@
|
||||
"""The rungs loop-claude hands a worker's claude in its environment, so rule-hook holds each review launch to them: docs/decisions.md → `routing:`.
|
||||
|
||||
loop-claude sets these variables for its worker and for the claude an interrupt resumes; a human's shell does not, and rule-hook then checks no review launch."""
|
||||
import typing
|
||||
|
||||
from model_routing.ladder import LADDER, Rung
|
||||
|
||||
PROJECT_RUNG = "LOOP_CLAUDE_PROJECT_RUNG"
|
||||
WORKER_RUNG = "LOOP_CLAUDE_WORKER_RUNG"
|
||||
|
||||
|
||||
def environment(project: Rung | None, worker: Rung) -> dict[str, str]:
|
||||
"""The variables a worker's claude starts with. A project off the ladder gets an empty rung, so none is inherited from the environment."""
|
||||
return {PROJECT_RUNG: str(project) if project else "", WORKER_RUNG: str(worker)}
|
||||
|
||||
|
||||
def handed(environ: typing.Mapping[str, str]) -> tuple[Rung | None, Rung | None]:
|
||||
"""The project's and the worker's rungs, each None when unset or not on the ladder."""
|
||||
return on_the_ladder(environ.get(PROJECT_RUNG)), on_the_ladder(environ.get(WORKER_RUNG))
|
||||
|
||||
|
||||
def on_the_ladder(value: str | None) -> Rung | None:
|
||||
rung = Rung.parse(value or "")
|
||||
return rung if rung in LADDER else None
|
||||
@@ -0,0 +1,30 @@
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from rule_hook.reviewer_rung import REVIEWERS, SEAT, SECURITY_OPENER
|
||||
from repo_checks.test_python_floor import ROOT
|
||||
|
||||
OPENING = re.compile(r"^## Opening\n+```\n([^\n]+?\.)(?: |\n)", re.MULTILINE)
|
||||
REVIEWER_NAME = re.compile(r"\b[a-z]+(?:-[a-z]+)*-(?:reviewer|architect|auditor)\b")
|
||||
LARV_REVIEW = (ROOT / "skills" / "larv-review" / "SKILL.md").read_text(encoding="utf-8")
|
||||
|
||||
|
||||
class ReviewerLaunches(unittest.TestCase):
|
||||
"""rule-hook holds a review launch to its rung by recognising it; these fail when what it recognises drifts from what the skills launch."""
|
||||
|
||||
def test_the_reviewers_are_the_ones_larv_review_s_pass_table_names(self):
|
||||
passes = LARV_REVIEW.split("## 3. Passes", 1)[1].split("A comment is prose", 1)[0]
|
||||
self.assertEqual(set(REVIEWER_NAME.findall(passes)), REVIEWERS)
|
||||
|
||||
def test_the_seat_opening_is_the_sentence_that_opens_every_seat_prompt(self):
|
||||
opening = OPENING.search((ROOT / "skills" / "comprehension-panel" / "seat-prompts.md").read_text(encoding="utf-8"))
|
||||
self.assertIsNotNone(opening, "seat-prompts.md has a `## Opening` section whose code block opens with a sentence")
|
||||
self.assertEqual(SEAT, opening.group(1) if opening else "")
|
||||
|
||||
def test_the_security_opening_is_the_sentence_larv_review_has_the_security_pass_open_with(self):
|
||||
security = LARV_REVIEW.split("### Security", 1)[1].split("### Overrides", 1)[0]
|
||||
self.assertIn(SECURITY_OPENER, security)
|
||||
@@ -3,12 +3,12 @@
|
||||
usage: rule-hook (a PreToolUse hook: reads the call as JSON on stdin)
|
||||
rule-hook overrides (prints this chunk's overrides)
|
||||
|
||||
settings.json runs it before each Bash, Edit, MultiEdit and Write call, in every session and
|
||||
settings.json runs it before each Bash, Edit, MultiEdit, Write and Agent call, in every session and
|
||||
subagent. The rules:
|
||||
|
||||
{rules}
|
||||
|
||||
A file a Bash command writes, such as through `cat > Dockerfile`, is not checked.
|
||||
A file a Bash command writes, such as through `cat > Dockerfile`, is not checked. The reviewer-rung rule takes no override.
|
||||
|
||||
A call overrides a rule with `# rule-override <rule>: <reason>`:
|
||||
|
||||
@@ -34,12 +34,14 @@ import os
|
||||
import pathlib
|
||||
import sys
|
||||
import time
|
||||
import typing
|
||||
|
||||
from rule_hook.call import applied, Call, FILE_TOOLS, parsed_call, text
|
||||
from rule_hook.rules import Breach, command_breaches, file_breaches, Outcome, rules_help, verdict
|
||||
from git_command import commits
|
||||
from failure import Failure
|
||||
from rule_hook.override_log import Log, log_path, logged, print_overrides, write_override
|
||||
from rule_hook.reviewer_rung import reviewer_breaches
|
||||
from shell_command import simple_commands
|
||||
from rule_hook.todo_check import todo_breaches
|
||||
|
||||
@@ -60,15 +62,17 @@ def main(argv: list[str], stdin: str) -> int:
|
||||
if isinstance(call, Failure):
|
||||
print(f"rule-hook cannot read the hook input: {call.reason}", file=sys.stderr)
|
||||
return 1
|
||||
found = breaches_of(call, os.path.expanduser("~"))
|
||||
found = breaches_of(call, os.path.expanduser("~"), os.environ)
|
||||
if found is None:
|
||||
return 0
|
||||
written, what, breaches = found
|
||||
return answer(verdict(breaches, written, text(call.given, "file_path").endswith(".md"), call.tool == "Bash"), what, time.time())
|
||||
|
||||
|
||||
def breaches_of(call: Call, home: str) -> tuple[str, str, list[Breach]] | None:
|
||||
def breaches_of(call: Call, home: str, environ: typing.Mapping[str, str]) -> tuple[str, str, list[Breach]] | None:
|
||||
"""The text that may carry an override, what the log names the call by, and its breaches; None for a call no rule checks."""
|
||||
if call.tool == "Agent":
|
||||
return text(call.given, "prompt"), text(call.given, "subagent_type"), reviewer_breaches(call, environ)
|
||||
if call.tool == "Bash":
|
||||
command = text(call.given, "command")
|
||||
return command, command.split("\n", 1)[0], bash_breaches(command, call.cwd, home)
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Refuse an Agent call that launches a review below the rung it needs, worked out from the two rungs loop-claude hands its worker: docs/decisions.md → `routing:`."""
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from rule_hook.rules import Breach
|
||||
from rule_hook.call import Call, text
|
||||
from model_routing.worker_rungs import handed
|
||||
from model_routing.ladder import position, review_rung, ReviewKind, Rung
|
||||
|
||||
# A reviewer is an agent under agents/, as CLAUDE.md → "Vocabulary" defines it.
|
||||
REVIEWERS = frozenset(path.stem for path in (pathlib.Path(__file__).resolve().parents[2] / "agents").glob("*.md"))
|
||||
# The opening of every comprehension seat's prompt: skills/comprehension-panel/seat-prompts.md → "Opening".
|
||||
SEAT = "You are one seat of a comprehension panel."
|
||||
# The opening of the security pass's prompt: skills/larv-review/SKILL.md → "Security".
|
||||
SECURITY_OPENER = "You are the security pass for this chunk."
|
||||
SUBJECTS: dict[ReviewKind, str] = {"seat": "a comprehension seat", "security": "the security pass's subagent"}
|
||||
|
||||
|
||||
def reviewer_breaches(call: Call, environ: typing.Mapping[str, str]) -> list[Breach]:
|
||||
"""The breach of an Agent call that launches a review below its rung; none where loop-claude handed no rungs."""
|
||||
project, worker = handed(environ)
|
||||
kind = review_kind(call.given)
|
||||
if kind is None or project is None or worker is None:
|
||||
return []
|
||||
required = review_rung(kind, project, worker)
|
||||
named = named_rung(call.given, worker)
|
||||
if position(named) >= position(required):
|
||||
return []
|
||||
subject = text(call.given, "subagent_type") if kind == "reviewer" else SUBJECTS[kind]
|
||||
return [Breach("reviewer-rung", f"{subject} would run on {named}, below {required}: launch it with {required.agent_args()}", call.cwd)]
|
||||
|
||||
|
||||
def review_kind(given: dict[str, typing.Any]) -> ReviewKind | None:
|
||||
"""What a launch reviews, a reviewer, a comprehension seat or the security pass; None for any other subagent."""
|
||||
if text(given, "subagent_type") in REVIEWERS:
|
||||
return "reviewer"
|
||||
opening = text(given, "prompt").lstrip()
|
||||
if opening.startswith(SEAT):
|
||||
return "seat"
|
||||
if opening.startswith(SECURITY_OPENER):
|
||||
return "security"
|
||||
return None
|
||||
|
||||
|
||||
def named_rung(given: dict[str, typing.Any], worker: Rung) -> Rung:
|
||||
"""The rung an Agent call's model and effort name. A call naming no model runs on the worker's model, at the effort it names or else the worker's; one naming a model but no effort runs on that model's cheapest rung."""
|
||||
model, effort = text(given, "model"), text(given, "effort")
|
||||
if not model:
|
||||
return Rung(worker.model, effort or worker.effort)
|
||||
return Rung.from_agent(model, effort)
|
||||
+15
-4
@@ -7,13 +7,15 @@ from git_command import commits
|
||||
from round_titles import lacks_counts
|
||||
from shell_command import Simple
|
||||
|
||||
RuleName = typing.Literal["action-pin", "commit-one-line", "ignorefile-sorted", "image-pin", "no-ai-attribution", "round-title-counts", "todo-rank-check"]
|
||||
RuleName = typing.Literal["action-pin", "commit-one-line", "ignorefile-sorted", "image-pin", "no-ai-attribution", "reviewer-rung", "round-title-counts", "todo-rank-check"]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Rule:
|
||||
holds: str
|
||||
checked: str
|
||||
# False where no marker waives the rule: overrides() grants none for it.
|
||||
overridable: bool = True
|
||||
|
||||
|
||||
RULES: dict[RuleName, Rule] = {
|
||||
@@ -37,6 +39,12 @@ RULES: dict[RuleName, Rule] = {
|
||||
"no commit, PR, issue comment or file carries a Claude or Anthropic attribution, such as a co-author trailer or a generated-with line",
|
||||
"a git, gh or tea command, and any file written",
|
||||
),
|
||||
# A rung is not a judgment call, and the refusal names the arguments to pass, so no marker waives it.
|
||||
"reviewer-rung": Rule(
|
||||
"a reviewer, a comprehension seat or the security pass's subagent runs on the rung loop-claude names for it: the project's rung, or for a reviewer the worker's rung where that is higher, so a worker below the project's rung cannot hand a review to its own cheaper model",
|
||||
"an Agent call, in a claude loop-claude started, whose subagent type is an agent under agents/, or whose prompt opens with seat-prompts.md's opening sentence or the one larv-review's Security section sets",
|
||||
overridable=False,
|
||||
),
|
||||
"round-title-counts": Rule(
|
||||
"a larv-review round's commit title gives the counts of 🔴 and 🟡 findings the round reported, such as `Answer stability round 1: 1 🔴, 2 🟡; flag a deleted test`, since score-outcomes counts them",
|
||||
"a git commit whose message opens `Answer <pass> round <n>:`",
|
||||
@@ -75,10 +83,10 @@ class Outcome:
|
||||
|
||||
|
||||
def overrides(written: str) -> tuple[dict[str, str], list[str]]:
|
||||
"""The rules the text overrides with their reasons, and the markers that name no rule or give no reason."""
|
||||
"""The rules the text overrides with their reasons, and the markers that name no rule or give no reason; a marker for a rule that takes no override is neither."""
|
||||
given = [(rule, reason.strip().removesuffix("-->").strip()) for rule, reason in OVERRIDE.findall(written)]
|
||||
granted = {rule: reason for rule, reason in given if rule in RULES and reason}
|
||||
return granted, [f"rule-override {rule}: {reason}" for rule, reason in given if rule not in granted]
|
||||
granted = {rule: reason for rule, reason in given if rule in RULES and RULES[rule].overridable and reason}
|
||||
return granted, [f"rule-override {rule}: {reason}" for rule, reason in given if rule not in RULES or not reason]
|
||||
|
||||
|
||||
def verdict(breaches: list[Breach], written: str, markdown: bool, bash: bool) -> Outcome:
|
||||
@@ -88,6 +96,9 @@ def verdict(breaches: list[Breach], written: str, markdown: bool, bash: bool) ->
|
||||
where = "the command" if bash else "the text you write, as a comment line"
|
||||
for rule in sorted({breach.rule for breach in blocking}):
|
||||
details = "; ".join(breach.detail for breach in blocking if breach.rule == rule)
|
||||
if not RULES[rule].overridable:
|
||||
messages.append(f"Blocked by rule {rule}: {details}. The rule: {RULES[rule].holds}.")
|
||||
continue
|
||||
marker = f"<!-- rule-override {rule}: <reason> -->" if markdown else f"# rule-override {rule}: <reason>"
|
||||
messages.append(f"Blocked by rule {rule}: {details}. The rule: {RULES[rule].holds}. Where it does not fit this case, add `{marker}` to {where}; larv-review rules on the reason.")
|
||||
return Outcome(messages, bool(blocking), [(breach, granted[breach.rule]) for breach in breaches if breach.rule in granted])
|
||||
|
||||
@@ -9,10 +9,13 @@ import unittest
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from rule_hook.rules import blocks, Breach, command_breaches, file_breaches, overrides, verdict
|
||||
from rule_hook.main import breaches_of
|
||||
from rule_hook.call import Call
|
||||
from git_command import Commit, commit_message, commits
|
||||
from test_rank_todo import DECISIONS, README, TODO
|
||||
from test_env import isolated_env
|
||||
from rule_hook.override_log import LogLine, parsed_entry, this_chunks
|
||||
from rule_hook.reviewer_rung import reviewer_breaches
|
||||
from shell_command import simple_commands
|
||||
|
||||
HOOK = pathlib.Path(__file__).resolve().parents[1] / "rule-hook"
|
||||
@@ -32,8 +35,8 @@ def checked(command: str) -> list[str]:
|
||||
return rules(command_breaches(command, simple_commands(command, "/r", "/h"), "/h"))
|
||||
|
||||
|
||||
def hook(call: dict, cwd: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run([str(HOOK)], capture_output=True, cwd=cwd, env=isolated_env(), input=json.dumps(call), text=True)
|
||||
def hook(call: dict, cwd: str, env: dict[str, str] | None = None) -> subprocess.CompletedProcess:
|
||||
return subprocess.run([str(HOOK)], capture_output=True, cwd=cwd, env=isolated_env() | (env or {}), input=json.dumps(call), text=True)
|
||||
|
||||
|
||||
def git(cwd: str, *args: str) -> str:
|
||||
@@ -127,6 +130,10 @@ class Overrides(unittest.TestCase):
|
||||
def test_a_marker_names_a_known_rule_and_a_reason(self):
|
||||
self.assertEqual(overrides("x # rule-override image-pin: vendor ships floating tags only\n# rule-override nope: x\n# rule-override action-pin:"), ({"image-pin": "vendor ships floating tags only"}, ["rule-override nope: x", "rule-override action-pin: "]))
|
||||
|
||||
def test_a_marker_for_a_rule_that_takes_no_override_is_neither_granted_nor_malformed(self):
|
||||
self.assertEqual(overrides("# rule-override reviewer-rung: trust me"), ({}, []))
|
||||
self.assertEqual(overrides("# rule-override reviewer-rung: trust me\n# rule-override image-pin: a # rule-override nope: b"), ({"image-pin": "a"}, ["rule-override nope: b"]))
|
||||
|
||||
def test_two_markers_on_a_line_are_read_apart(self):
|
||||
self.assertEqual(overrides("git x # rule-override image-pin: a # rule-override nope: b")[1], ["rule-override nope: b"])
|
||||
|
||||
@@ -158,6 +165,64 @@ class OverrideLog(unittest.TestCase):
|
||||
self.assertEqual(this_chunks(LogLine("t", branch, commit, "image-pin", "r", "w"), {"base", "c1"}, "chunk", on_default), expected)
|
||||
|
||||
|
||||
class ReviewerRung(unittest.TestCase):
|
||||
# A worker routed to Haiku 5.5 at max, on a project whose rung is Opus 5.5 at medium.
|
||||
ENVIRON = {"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort"}
|
||||
# The same project, with a worker routed above it to Fable 5.1 at max.
|
||||
ABOVE = {"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-fable-5-1 at max effort"}
|
||||
|
||||
def refused(self, given: dict, environ: dict[str, str] = ENVIRON) -> list[str]:
|
||||
return rules(reviewer_breaches(Call("Agent", given, "/r"), environ))
|
||||
|
||||
def test_a_reviewer_below_the_project_rung_is_refused_naming_the_arguments_to_pass(self):
|
||||
breaches = reviewer_breaches(Call("Agent", {"subagent_type": "stability-reviewer", "model": "sonnet", "effort": "high", "prompt": "Review."}, "/r"), self.ENVIRON)
|
||||
self.assertEqual(rules(breaches), ["reviewer-rung"])
|
||||
self.assertIn("stability-reviewer", breaches[0].detail)
|
||||
self.assertIn("claude-sonnet-5-5 at high effort", breaches[0].detail)
|
||||
self.assertIn("model opus, effort medium", breaches[0].detail)
|
||||
|
||||
def test_a_reviewer_on_the_project_rung_or_a_rerun_above_it_passes(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "prose-reviewer", "model": "opus", "effort": "medium"}), [])
|
||||
self.assertEqual(self.refused({"subagent_type": "prose-reviewer", "model": "opus", "effort": "high"}), [])
|
||||
|
||||
def test_an_agent_calls_prompt_is_the_text_scanned_for_a_marker(self):
|
||||
call = Call("Agent", {"subagent_type": "prose-reviewer", "prompt": "Review. # rule-override reviewer-rung: trust me"}, "/r")
|
||||
self.assertEqual(breaches_of(call, "/h", {})[0], "Review. # rule-override reviewer-rung: trust me")
|
||||
|
||||
def test_a_call_naming_no_model_runs_on_the_workers_rung(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "prompt": "Review."}), ["reviewer-rung"])
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "effort": "medium"}), ["reviewer-rung"])
|
||||
|
||||
def test_a_model_named_without_an_effort_runs_on_its_cheapest_rung(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "prose-reviewer", "model": "opus"}), [])
|
||||
self.assertEqual(self.refused({"subagent_type": "prose-reviewer", "model": "sonnet"}), ["reviewer-rung"])
|
||||
|
||||
def test_an_effort_off_the_ladder_for_its_model_is_refused(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "prose-reviewer", "model": "opus", "effort": "low"}), ["reviewer-rung"])
|
||||
|
||||
def test_a_reviewer_is_held_to_the_workers_rung_where_that_is_higher(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "model": "opus", "effort": "medium"}, self.ABOVE), ["reviewer-rung"])
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "model": "fable", "effort": "max"}, self.ABOVE), [])
|
||||
|
||||
def test_a_seat_or_a_security_launch_is_held_to_the_project_rung_alone(self):
|
||||
seat = {"prompt": "You are one seat of a comprehension panel. Tools: Read, Grep, Glob.", "model": "opus", "effort": "medium"}
|
||||
self.assertEqual(self.refused(seat, self.ABOVE), [])
|
||||
self.assertEqual(self.refused({"prompt": "You are the security pass for this chunk. Invoke /security-review over the branch diff.", "model": "sonnet", "effort": "high"}), ["reviewer-rung"])
|
||||
self.assertEqual(self.refused({"prompt": "You are the security pass for this chunk. Run /claude-security scan this branch's changes at high.", "model": "opus", "effort": "medium"}), [])
|
||||
|
||||
def test_a_seat_named_below_the_project_rung_is_refused(self):
|
||||
self.assertEqual(self.refused({"prompt": "You are one seat of a comprehension panel.", "model": "sonnet", "effort": "high"}), ["reviewer-rung"])
|
||||
|
||||
def test_any_other_subagent_keeps_its_own_routing(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "Explore", "model": "haiku", "effort": "low", "prompt": "Find the callers of run_worker."}), [])
|
||||
self.assertEqual(self.refused({"prompt": "Note that security-review runs in CI.", "model": "haiku", "effort": "low"}), [])
|
||||
self.assertEqual(self.refused({"prompt": "Find where /security-review is named.", "model": "haiku", "effort": "low"}), [], "a mention without the opening sentence is no launch")
|
||||
self.assertEqual(self.refused({"prompt": "Quote the rule: 'You are the security pass for this chunk.' Then stop.", "model": "haiku", "effort": "low"}), [], "a quote midway is no launch")
|
||||
|
||||
def test_a_claude_loop_claude_did_not_start_is_not_checked(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "model": "haiku", "effort": "low"}, {}), [])
|
||||
|
||||
|
||||
class Hook(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.repo = tempfile.mkdtemp()
|
||||
@@ -218,6 +283,20 @@ class Hook(unittest.TestCase):
|
||||
self.assertEqual(hook({"cwd": self.repo, "tool_input": {"command": command}, "tool_name": "Bash"}, self.repo).returncode, 2)
|
||||
self.assertEqual(self.listed(), "")
|
||||
|
||||
def test_an_agent_call_below_its_rung_is_refused_and_offers_no_override(self):
|
||||
call = {"cwd": self.repo, "tool_input": {"subagent_type": "stability-reviewer", "model": "sonnet", "effort": "high", "prompt": "Review. # rule-override reviewer-rung: trust me"}, "tool_name": "Agent"}
|
||||
ran = hook(call, self.repo, ReviewerRung.ENVIRON)
|
||||
self.assertEqual(ran.returncode, 2)
|
||||
self.assertIn("Blocked by rule reviewer-rung", ran.stderr)
|
||||
self.assertIn("stability-reviewer", ran.stderr)
|
||||
self.assertIn("claude-sonnet-5-5 at high effort", ran.stderr)
|
||||
self.assertIn("model opus, effort medium", ran.stderr)
|
||||
self.assertNotIn("rule-override", ran.stderr)
|
||||
|
||||
def test_an_agent_call_at_its_rung_passes(self):
|
||||
call = {"cwd": self.repo, "tool_input": {"subagent_type": "stability-reviewer", "model": "opus", "effort": "medium", "prompt": "Review."}, "tool_name": "Agent"}
|
||||
self.assertEqual(hook(call, self.repo, ReviewerRung.ENVIRON).returncode, 0)
|
||||
|
||||
def test_a_commit_override_is_logged_in_the_commit_s_checkout(self):
|
||||
outside = tempfile.mkdtemp()
|
||||
self.addCleanup(subprocess.run, ["rm", "-rf", outside])
|
||||
|
||||
@@ -1961,3 +1961,26 @@ Inherited architect, decided by:
|
||||
- Locality, Shape: `bin/rank_todo.py:301` `unknown_columns`: the old-shape path is checked at six sites across `rank_todo.py` with its own reader and writer in `todo_table.py`, and nothing ties them together for whoever retires it
|
||||
- Shape: `bin/todo_table.py:16` `Moves`: the name lies for old-shape rows, and the goals cell is parsed by a callable stored on a dataclass
|
||||
- Locality: `bin/todo_table.py:41` `parse_moves`: the rules for a Moves cell are split between `parse_moves` and `rank_todo.py`'s `parse_item`
|
||||
|
||||
## 2026-10-16T05:16:12Z, PR #295 at 7ea23b1, against 1bff4ac
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | same | same | better | better | better |
|
||||
| Inherited architect | better | same | better | same | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Shape, Overall: `skills/larv-review/SKILL.md:36` `Principles`: the principles audit is a pass of its own, with its own row and trigger, so the exceptions the base needed for it are gone
|
||||
- Self-sufficiency, Shape: `bin/model_routing/ladder.py:87` `review_rung`: the rule that a reviewer runs on the higher rung is named, where the base wrote `highest` inline twice
|
||||
- Self-sufficiency: `bin/model_routing/ladder.py:42` `Rung.parse`: the spelling of a rung and its parser now sit beside `__str__`, where the base repeated the pattern in `route.py`
|
||||
- Locality: `bin/rule_hook/reviewer_rung.py:11` `REVIEWERS`: a list read from the disk at import, a hidden dependency that pulls towards worse
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Shape, Overall: `skills/larv-review/SKILL.md:36` `Principles`: a pass of its own, with none of the base's exceptions to carry
|
||||
- Navigation: `docs/decisions.md:239` `routing: a chunk's worker runs on the rung its item picks…`: the routing entry is split into titled entries, so a symptom leads to a heading
|
||||
- Shape: `bin/model_routing/ladder.py:87` `review_rung`: one named rule that the worker's prompt and the hook share
|
||||
- Locality: `bin/rule_hook/reviewer_rung.py:13` `SEAT`: the hook recognises a launch by a sentence in two markdown files, so rewording either turns the check off without a failure
|
||||
|
||||
@@ -486,3 +486,20 @@ Read most (opens and searches):
|
||||
- `bin/loop_claude/loop/routing_check.py` 1
|
||||
- `bin/loop_claude/project/records.py` 1
|
||||
- `bin/subagent_rung/main.py` 1
|
||||
|
||||
## Reading cost, chunk 9bac6d8e300d, finished 2026-10-16T03:41:21Z
|
||||
|
||||
189487 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `todo.md` 21
|
||||
- `agents/stability-reviewer.md` 6
|
||||
- `docs/decisions.md` 4
|
||||
- `CHANGELOG.md` 3
|
||||
- `skills/larv-review/SKILL.md` 3
|
||||
- `AGENTS.md` 2
|
||||
- `README.md` 2
|
||||
- `.gitignore` 1
|
||||
- `CLAUDE.md` 1
|
||||
- `principles/technical-principles.md` 1
|
||||
|
||||
+1
-1
@@ -36,7 +36,7 @@
|
||||
"hooks": {
|
||||
"PreToolUse": [
|
||||
{
|
||||
"matcher": "Bash|Edit|MultiEdit|Write",
|
||||
"matcher": "Bash|Edit|MultiEdit|Write|Agent",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
|
||||
@@ -122,7 +122,7 @@ The comprehension check records how hard the project is to read, and blocks no m
|
||||
|
||||
### Security
|
||||
|
||||
`/security-review` over the branch diff is the pass of record, except where the full scan runs on the same change and replaces it. Where the full scan fires but does not run — skipped on a precedent, declined at its cost prompt, or unavailable — `/security-review` runs. Launch a general-purpose subagent that invokes it through the Skill tool, edits nothing, and replies with the skill's report, or that the skill is unavailable. Invoked in your own context, the skill ends your turn on that report.
|
||||
`/security-review` over the branch diff is the pass of record, except where the full scan runs on the same change and replaces it. Where the full scan fires but does not run — skipped on a precedent, declined at its cost prompt, or unavailable — `/security-review` runs. Launch a general-purpose subagent that invokes it through the Skill tool, edits nothing, and replies with the skill's report, or that the skill is unavailable. Invoked in your own context, the skill ends your turn on that report. Open its prompt, and that of any subagent that runs the full scan, with `You are the security pass for this chunk.`: rule-hook reads that sentence as a security launch.
|
||||
|
||||
A verified HIGH inside this chunk's blast radius is fixed here, outside it on its own branch and PR with this chunk waiting behind it. Say which.
|
||||
|
||||
|
||||
@@ -6,19 +6,19 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 166`
|
||||
`Next ID = 168`
|
||||
|
||||
## Items
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | R | S | C | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|
|
||||
| 166 | 0.2.0 | | question | **Decide whether `docs/decisions.md` names rule-hook's refusal of a review launched below its rung, in `routing:` and in `rule-hook:`.** | 1 | 1 | 1 | 1.2 by 3 | 4.4 |
|
||||
| 146 | 0.2.0 | | decision | **Score R as L + E + I − 2 from anchored levels, with R at least 7 when I is 5 and at least 5 when I is 4; give L, E and I their own columns and have `rank_todo.py` compute R; drop C; and route a task's worker on its L and its reviewers on its E + I.** | 6 | 7 | 7 | 1.1 by 8, 1.2 by 8, 1.4 by 8 | 2.8 |
|
||||
| 152 | 0.2.0 | 146 | decision | **Anchor each score level in `rules/todo-items.md` with words, and each project's with two or three items it shipped, and have a scorer compare a new item against them.** | 3 | 4 | 4 | 1.1 by 7 | 6.4 |
|
||||
| 50 | 0.2.0 | | principle | **Grant the bench's runs only the tools a review needs, in place of skipping every permission.** | 2 | 2 | 3 | 1.1 by 3 | 5.5 |
|
||||
| 65 | 0.2.0 | | defect | **Archive each judge reply the loop rejects.** | 3 | 4 | 3 | 3 by 3 | 5.5 |
|
||||
| 153 | 0.2.0 | | decision | **Run the judge on the rung the highest R + C in its chunk picks, an item the worker takes in included.** | 3 | 4 | 5 | 1.2 by 5 | 3.0 |
|
||||
| 161 | 0.2.0 | | | **Have loop-claude run a preflight script before each fresh worker and put its findings in the worker's prompt, so the worker no longer spends tool calls finding the chunk's mechanical facts.** | 4 | 6 | 5 | 1.1 by 4, 1.4 by 6, 1.5 by 6 | 2.9 |
|
||||
| 163 | 0.2.0 | | | **Have `rule-hook` block an Agent call that launches a reviewer, a comprehension seat or the security pass's subagent below the rung loop-claude names for them, so a worker below the project's rung cannot hand a review to its own cheaper model.** | 4 | 5 | 6 | 1.1 by 5 | 2.7 |
|
||||
| 84 | 0.2.0 | | | **Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.** | 2 | 4 | 5 | 1.4 by 5 | 2.3 |
|
||||
| 159 | 0.2.0 | | | **Run the prose pass on the lowest rung that a replay of archived prose rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung.** | 3 | 5 | 5 | 1.4 by 5 | 1.4 |
|
||||
| 73 | 0.2.0 | | | **Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.** | 4 | 5 | 5 | 1.4 by 4 | 1.1 |
|
||||
@@ -39,6 +39,7 @@
|
||||
| 154 | 0.3.0 | | defect | **Have `bin/shell_command.py`'s `segments` keep a command's words together across a redirect, so `git > out commit` reads as one git commit.** | 3 | 2 | 3 | 1.1 by 6 | 10.9 |
|
||||
| 157 | 0.3.0 | | | **Split `docs/decisions.md` → `todo.md: one table of items …`'s 2026-10-02 paragraph into one paragraph per choice.** | 1 | 2 | 1 | 1.1 by 4, 1.4 by 4 | 10.9 |
|
||||
| 106 | 0.3.0 | | | **Make larv-review's "Do not over-read a clean result" cite a rule that says Renovate or Dependabot owns dependency CVEs, or drop the citation.** | 1 | 1 | 1 | 1.1 by 5 | 10.9 |
|
||||
| 167 | 0.3.0 | | | **Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.** | 2 | 2 | 3 | 1.1 by 4 | 7.3 |
|
||||
| 82 | 0.3.0 | | principle | **Sort `AGENTS.md`'s decision titles and `docs/decisions.md`'s entries by title.** | 1 | 4 | 2 | 1.1 by 4, 1.4 by 4 | 5.5 |
|
||||
| 162 | 0.3.0 | | owner | **Script each step that 0.2.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.** | 1 | 3 | 3 | 1.4 by 4, 1.5 by 4 | 4.9 |
|
||||
| 165 | 0.3.0 | | | **Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.** | 2 | 5 | 4 | 1.1 by 4, 1.4 by 3 | 3.0 |
|
||||
@@ -80,6 +81,10 @@
|
||||
|
||||
## Details
|
||||
|
||||
### 166. Decide whether `docs/decisions.md` names rule-hook's refusal of a review launched below its rung, in `routing:` and in `rule-hook:`.
|
||||
|
||||
Item 163 (PR #295) has rule-hook refuse a review launch below the rung loop-claude gives its worker. The routing decision still says loop-claude's prompt has the worker launch reviewers, comprehension seats and the security pass on the project's rung, and does not name the hook. Recommendation: amend that paragraph to say rule-hook refuses a launch below the rung too. The decision was held before the chunk began, so the human decides. The `rule-hook:` entry's title says a marker overrides one rule, and `reviewer-rung` takes none. The amendment need not give the reason no marker waives the rule: the comment above `reviewer-rung` in bin/rule_hook/rules.py records it.
|
||||
|
||||
### 146. Score R as L + E + I − 2 from anchored levels, with R at least 7 when I is 5 and at least 5 when I is 4; give L, E and I their own columns and have `rank_todo.py` compute R; drop C; and route a task's worker on its L and its reviewers on its E + I.
|
||||
|
||||
Each level is about ×10 of a real quantity; `docs/decisions.md` → `todo.md: …` says why. The anchors, for `rules/todo-items.md`:
|
||||
@@ -132,12 +137,6 @@ The preflight also lists the files and components the item's detail names, for t
|
||||
|
||||
Compute these in one tool and give loop-claude's prompt its output as facts, so the worker reads them rather than finding them. The reading the chunk needs, such as the decision an item changes, stays the worker's. After its first read of the code, the worker counts the item's size again: the lines, files and components it will change. Where that count would pick another rung, a subagent on the project's rung re-scores the item. A re-score that picks a rung above the worker's is escalation trigger 1. Measure the next few chunks' time and fresh tokens before the first edit against item 140's figures. `bin/loop_claude/cycle/prompts.py`'s `off_the_projects_rung` is a place the comprehension seats found hard: put the preflight's facts in a function of their own rather than another branch there.
|
||||
|
||||
### 163. Have `rule-hook` block an Agent call that launches a reviewer, a comprehension seat or the security pass's subagent below the rung loop-claude names for them, so a worker below the project's rung cannot hand a review to its own cheaper model.
|
||||
|
||||
`docs/decisions.md` → `routing: a chunk's worker runs on the rung its item picks…` keeps every reviewer, comprehension seat and the security pass's subagent at or above the project's rung. Only loop-claude's prompt holds that today. No agent under `agents/` sets `model:`, so an Agent call that names no model runs on the caller's: a Haiku worker that leaves out `model` once gets a Haiku review, and nothing reports it. On 2026-10-16, item 67's chunk passed `model=opus, effort=medium` on every such call, so no review has run low yet.
|
||||
|
||||
The hook sees only the tool call and its working directory, so loop-claude must hand it the project's rung and the worker's, such as in environment variables on the worker's `claude`. The hook then refuses an Agent call of a reviewer type in larv-review §3's table, or a seat or security launch, whose model and effort fall below the project's rung, counting a call that names no model as the worker's rung, and names the arguments to pass. A rerun one rung up, per `review: a reviewer or comprehension seat that rules the work below standard…`, passes, since it is higher. A subagent that is not a reviewer keeps routing by its own R + C.
|
||||
|
||||
### 84. Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.
|
||||
|
||||
Raised by the principles audit of PR #228: the `review: the comprehension check records …` entry in `docs/decisions.md` is valid only while that holds, and `docs/comprehension-history.md` keeps every vote, yet nothing reads the agreement rate. Scoring runs on near-identical code are rare now, so compare with those already in the history, such as PR #224's and PR #225's, or make a second run for the check.
|
||||
@@ -235,6 +234,10 @@ Raised by PR #283's prose pass. The paragraph runs about 330 words and holds six
|
||||
|
||||
Found by the goals audit of PR #245. `skills/larv-review/SKILL.md` §3 → "Security" says Renovate or Dependabot owns dependency CVEs per `CLAUDE.md` → "Working with the codebase", which says nothing about either.
|
||||
|
||||
### 167. Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.
|
||||
|
||||
rule-hook checks Agent calls (item 163) but no Skill calls. A worker that invokes `/security-review` in its own context, instead of in a subagent, runs the pass on its own rung, and nothing blocks it. Check Skill calls too: `breaches_of` in bin/rule_hook/main.py takes a Skill call whose skill is `security-review` or `claude-security` as a review launch, and settings.json's PreToolUse matcher gains Skill. Found by the systems-architect's review of PR #295.
|
||||
|
||||
### 82. Sort `AGENTS.md`'s decision titles and `docs/decisions.md`'s entries by title.
|
||||
|
||||
"Deterministic order", in `principles/technical-principles.md`, raised by the principles audit of PR #224: their order carries no meaning, so two chunks adding a decision conflict at the end, and a reader cannot find a title by scanning.
|
||||
@@ -365,7 +368,7 @@ While `rank_todo.py` ranks two shapes, projects follow two sets of scoring rules
|
||||
|
||||
### 155. Lower the reading cost of the files a chunk reads most before its first commit.
|
||||
|
||||
`bin/reading-cost` on 2026-10-16: the last 5 chunks' median is 146584 new input tokens before their first commit, the 5 before them 90662. Read most over the last 5, by opens and searches: `todo.md` 38, `docs/decisions.md` 22, `README.md` 15, `CHANGELOG.md` 11, `bin/test_rank_todo.py` 10, `skills/larv-review/SKILL.md` 8, `bin/loop_claude/test_loop_claude.py` 7, `AGENTS.md` 6, `bin/rank_todo.py` 6, `docs/scoring-and-goals-research.md` 5. The chunk that ships this item runs `bin/reading-cost --reset`.
|
||||
`bin/reading-cost` on 2026-10-16: the last 5 chunks' median is 189487 new input tokens before their first commit, the 5 before them 90662. Read most over the last 5, by opens and searches: `todo.md` 55, `docs/decisions.md` 23, `CHANGELOG.md` 12, `README.md` 12, `bin/test_rank_todo.py` 10, `AGENTS.md` 8, `bin/loop_claude/test_loop_claude.py` 7, `skills/larv-review/SKILL.md` 7, `agents/stability-reviewer.md` 6, `bin/rank_todo.py` 6. The chunk that ships this item runs `bin/reading-cost --reset`.
|
||||
|
||||
### 1. Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user