Refuse a security-review or claude-security Skill call from the worker's own context below the project's rung
This commit is contained in:
@@ -4,6 +4,7 @@
|
||||
|
||||
- A release review now fixes the findings of its architecture and principles passes in the release it reviews. Before, it filed each one in `todo.md`, so the release held the drift the review had found. It files only three kinds of finding, each under the release being reviewed: one whose fix needs a human answer, marked `question`; one whose fix adds a feature; and one whose fix the five rounds cannot finish. Until each is done, the release isn't ready to cut.
|
||||
- In a repo with no tag yet, `chunk-preflight` now calls a release finished once `todo.md` files no item under it: its release review is due, and no later release's item is taken. Before, it lost sight of such a release.
|
||||
- In a run `loop-claude` started, `rule-hook` now refuses a `/security-review` or `/claude-security` Skill call that the worker makes in its own context on a rung below the project's. The refusal says to invoke it from a subagent on the project's rung. Before, only an Agent call that launched the security pass was held to that rung.
|
||||
|
||||
## 0.4.0
|
||||
|
||||
|
||||
@@ -97,7 +97,7 @@ Install puts only `loop-claude` and `loop-inbox` on `PATH`; call the others by t
|
||||
- `rank_todo.py [--check] [todo.md]` scores and ranks a project's `todo.md`; `rank_todo.py --help` states the format.
|
||||
- `reading-cost [--reset] [record]` appends a figure for each chunk `loop-claude` finished to a project's `docs/reading-cost.md`, and says whether the figures rise; `reading-cost --help` states the measure.
|
||||
- `review-replay <pass> <model>` replays a review pass's archived rounds from the projects run on the model, on the code each reviewed, and prints the lowest rung below the project's that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again; `review-replay --help` states the replay. Each replayed reviewer skips every permission prompt, stability-reviewer has Bash, and an archived prompt may name paths on the host.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Agent, Bash, Edit, MultiEdit, Read and Write call, and `rule-hook --help` states the rules and the override.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Agent, Bash, Edit, MultiEdit, Read, Skill and Write call, and `rule-hook --help` states the rules and the override.
|
||||
- `run-heavy <command>` runs a heavy command once it holds one of the machine's heavy slots, one per 4 CPUs, which every session shares; `run-heavy --help` states the slots, and the memory cap it gives a container that sets no limit.
|
||||
- `score-outcomes [L E I S]` shows what each finished item's chunk measured beside the scores the item was guessed at, to score a new item by, and code health's two counter-signals per chunk, misread findings and later fixes; `score-outcomes --help` states the report.
|
||||
- `settings_filter.py` is git's clean filter for `settings.json`: it drops `"model"` and sorts the keys. `.gitattributes` sets the filter `settings` on `settings.json`, and Install's `git config` line makes this script that filter.
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
usage: rule-hook (a PreToolUse hook: reads the call as JSON on stdin)
|
||||
rule-hook overrides (prints this chunk's overrides)
|
||||
|
||||
settings.json runs it before each Agent, Bash, Edit, MultiEdit, Read and Write call, in every session and
|
||||
settings.json runs it before each Agent, Bash, Edit, MultiEdit, Read, Skill and Write call, in every session and
|
||||
subagent. The rules:
|
||||
|
||||
{rules}
|
||||
@@ -76,6 +76,8 @@ def breaches_of(call: Call, home: str, environ: typing.Mapping[str, str]) -> tup
|
||||
"""The text that may carry an override, what the log names the call by, and its breaches; None for a call no rule checks."""
|
||||
if call.tool == "Agent":
|
||||
return text(call.given, "prompt"), text(call.given, "subagent_type"), reviewer_breaches(call, environ)
|
||||
if call.tool == "Skill":
|
||||
return text(call.given, "args"), text(call.given, "skill"), reviewer_breaches(call, environ)
|
||||
if call.tool == "Bash":
|
||||
command = text(call.given, "command")
|
||||
return command, command.split("\n", 1)[0], bash_breaches(command, call.cwd, home) + read_breaches(call, home)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Refuse an Agent call that launches a review below the rung it needs, worked out from the rungs and the E + I loop-claude hands its worker: docs/decisions.md → `routing:`."""
|
||||
"""Refuse an Agent or Skill call that launches a review below the rung it needs, worked out from the rungs and the E + I loop-claude hands its worker: docs/decisions.md → `routing:`."""
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
@@ -18,24 +18,28 @@ SEAT = "You are one seat of a comprehension panel."
|
||||
# The opening of the security pass's prompt: skills/larv-review/SKILL.md → "Security".
|
||||
SECURITY_OPENER = "You are the security pass for this chunk."
|
||||
SUBJECTS: dict[ReviewKind, str] = {"seat": "a comprehension seat", "security": "the security pass's subagent"}
|
||||
# The skills that run the security pass, by their names with any plugin prefix cut.
|
||||
SECURITY_SKILLS = {"security-review", "claude-security"}
|
||||
# Markdown an agent follows, which counts as config: Claude Code's memory files, and the folders that hold Claude Code's skills, agents and rules and this repo's principles.
|
||||
INSTRUCTION_FILES = {"CLAUDE.md", "AGENTS.md"}
|
||||
INSTRUCTION_FOLDERS = {"agents", "principles", "rules", "skills"}
|
||||
|
||||
|
||||
def reviewer_breaches(call: Call, environ: typing.Mapping[str, str]) -> list[Breach]:
|
||||
"""The breach of an Agent call that launches a review below its rung; none where loop-claude handed no rungs."""
|
||||
"""The breach of an Agent or Skill call that launches a review below its rung; none where loop-claude handed no rungs."""
|
||||
project, worker, escape_and_impact = handed(environ)
|
||||
kind = review_kind(call.given)
|
||||
kind = security_skill(call) if call.tool == "Skill" else review_kind(call.given)
|
||||
if kind is None or project is None or worker is None:
|
||||
return []
|
||||
required = review_rungs(project, worker, escape_and_impact)[kind]
|
||||
# A branch that changes only Markdown no agent follows holds no code or config for a reviewer on a higher rung to catch a defect in: skills/larv-review/SKILL.md → "Markdown only".
|
||||
if kind in REVIEWERS and position(required) > position(project) and markdown_only(call.cwd):
|
||||
required = project
|
||||
named = named_rung(call.given, worker)
|
||||
named = worker if call.tool == "Skill" else named_rung(call.given, worker)
|
||||
if position(named) >= position(required):
|
||||
return []
|
||||
if call.tool == "Skill":
|
||||
return [Breach("reviewer-rung", f"{text(call.given, 'skill')} would run in this context on {named}, below {required}: invoke it from a subagent launched with {required.agent_args()}, its prompt opening with `{SECURITY_OPENER}`", call.cwd)]
|
||||
subject = SUBJECTS.get(kind) or text(call.given, "subagent_type")
|
||||
return [Breach("reviewer-rung", f"{subject} would run on {named}, below {required}: launch it with {required.agent_args()}", call.cwd)]
|
||||
|
||||
@@ -52,6 +56,12 @@ def review_kind(given: dict[str, typing.Any]) -> ReviewKind | None:
|
||||
return None
|
||||
|
||||
|
||||
def security_skill(call: Call) -> ReviewKind | None:
|
||||
"""The security pass, for a Skill call from the session's own agent that runs it; None for any other. A subagent's Skill call runs on the rung its Agent call was held to."""
|
||||
skill = text(call.given, "skill").lstrip("/").rsplit(":", 1)[-1]
|
||||
return "security" if skill in SECURITY_SKILLS and not call.agent else None
|
||||
|
||||
|
||||
def named_rung(given: dict[str, typing.Any], worker: Rung) -> Rung:
|
||||
"""The rung an Agent call's model and effort name. A call naming no model runs on the worker's model, at the effort it names or else the worker's; one naming a model but no effort runs on that model's cheapest rung."""
|
||||
model, effort = text(given, "model"), text(given, "effort")
|
||||
|
||||
@@ -61,7 +61,7 @@ RULES: dict[RuleName, Rule] = {
|
||||
# A rung is not a judgment call, and the refusal names the arguments to pass, so no marker waives it.
|
||||
"reviewer-rung": Rule(
|
||||
"a reviewer, a comprehension seat or the security pass's subagent runs on the rung loop-claude names for it: for a seat or the security pass the project's rung, and for a reviewer the highest of the project's rung, the worker's and the one the chunk's item's E + I picks, or the project's rung where the branch changes only Markdown no agent follows, so a worker below the project's rung cannot hand a review to its own cheaper model",
|
||||
"an Agent call, in a claude loop-claude started, whose subagent type is an agent under agents/, or whose prompt opens with seat-prompts.md's opening sentence or the one larv-review's Security section sets",
|
||||
"an Agent call, in a claude loop-claude started, whose subagent type is an agent under agents/, or whose prompt opens with seat-prompts.md's opening sentence or the one larv-review's Security section sets; and a Skill call there of `security-review` or `claude-security` from the session's own agent, which runs on the worker's rung",
|
||||
overridable=False,
|
||||
),
|
||||
# The cap holds with no ask, so no marker waives it.
|
||||
|
||||
@@ -315,6 +315,31 @@ class ReviewerRung(unittest.TestCase):
|
||||
def test_a_claude_loop_claude_did_not_start_is_not_checked(self):
|
||||
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "model": "haiku", "effort": "low"}, {}), [])
|
||||
|
||||
def test_a_security_skill_in_the_sessions_own_context_runs_on_the_workers_rung(self):
|
||||
for skill in ("security-review", "claude-security", "claude-security:claude-security"):
|
||||
with self.subTest(skill=skill):
|
||||
breaches = reviewer_breaches(Call("Skill", {"skill": skill}, "/r"), self.ENVIRON)
|
||||
self.assertEqual(rules(breaches), ["reviewer-rung"])
|
||||
self.assertIn("claude-sonnet-5-5 at high effort", breaches[0].detail)
|
||||
self.assertIn("model opus, effort medium", breaches[0].detail)
|
||||
self.assertIn("You are the security pass for this chunk.", breaches[0].detail)
|
||||
self.assertEqual(rules(reviewer_breaches(Call("Skill", {"skill": skill}, "/r"), self.ABOVE)), [])
|
||||
|
||||
def test_a_security_skill_inside_a_subagent_runs_on_the_rung_its_launch_was_held_to(self):
|
||||
self.assertEqual(rules(reviewer_breaches(Call("Skill", {"skill": "security-review"}, "/r", "general-purpose"), self.ENVIRON)), [])
|
||||
|
||||
def test_any_other_skill_is_not_checked(self):
|
||||
self.assertEqual(rules(reviewer_breaches(Call("Skill", {"skill": "simplify"}, "/r"), self.ENVIRON)), [])
|
||||
|
||||
def test_a_skill_calls_args_are_the_text_scanned_for_a_marker(self):
|
||||
found = breaches_of(Call("Skill", {"skill": "security-review", "args": "the branch # rule-override reviewer-rung: trust me"}, "/r"), "/h", {})
|
||||
self.assertEqual(found[:2], ("the branch # rule-override reviewer-rung: trust me", "security-review"))
|
||||
|
||||
def test_settings_run_the_hook_before_each_skill_call(self):
|
||||
settings = json.loads(pathlib.Path(__file__).resolve().parents[2].joinpath("settings.json").read_text())
|
||||
matchers = [entry["matcher"].split("|") for entry in settings["hooks"]["PreToolUse"]]
|
||||
self.assertTrue(any("Skill" in tools for tools in matchers))
|
||||
|
||||
|
||||
class ReviewerReport(unittest.TestCase):
|
||||
def breached(self, tool: str, path: str, agent: str = "prose-reviewer", cwd: str = "/r") -> list[str]:
|
||||
|
||||
@@ -923,3 +923,20 @@ Read most (opens and searches):
|
||||
## Reading cost, chunk cfd9ab4f5b1b, finished 2026-10-28T22:00:08Z
|
||||
|
||||
64114 new input tokens before the first commit.
|
||||
|
||||
## Reading cost, chunk d8367f5dd24a, finished 2026-10-30T12:08:14Z
|
||||
|
||||
144200 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `docs/decisions.md` 6
|
||||
- `todo.md` 6
|
||||
- `AGENTS.md` 4
|
||||
- `bin/test_chunk_pick.py` 4
|
||||
- `README.md` 3
|
||||
- `skills/larv-review/SKILL.md` 3
|
||||
- `bin/loop_claude/running_loops.py` 2
|
||||
- `bin/loop_claude/test_black_box.py` 2
|
||||
- `bin/loop_inbox/main.py` 2
|
||||
- `bin/loop_inbox/test_loop_inbox.py` 2
|
||||
|
||||
+1
-1
@@ -32,7 +32,7 @@
|
||||
"type": "command"
|
||||
}
|
||||
],
|
||||
"matcher": "Agent|Bash|Edit|MultiEdit|Read|Write"
|
||||
"matcher": "Agent|Bash|Edit|MultiEdit|Read|Skill|Write"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -12,7 +12,6 @@
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 167 | 0.5.0 | | | **Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.** | 3 | 2 | 3 | 6 | 2 | 1.1 by 4 | 5.6 |
|
||||
| 6 | 0.5.0 | | decision | **Move what `loop-claude` writes out of `~/.claude.json`, and ask the human about each write that must stay.** | 3 | 3 | 3 | 7 | 3 | 1.1 by 5 | 5.3 |
|
||||
| 257 | 0.5.0 | | | **Make `chunk-preflight` create the item's worktree and branch only when loop-claude starts a worker on it, so a run for the facts alone leaves nothing to remove.** | 3 | 2 | 3 | 6 | 4 | 1.5 by 2 | 1.3 |
|
||||
| 226 | 0.6.0 | | question | **Answer whether rule-hook's `run-heavy` rule should also catch a mutation run or docker build behind `uv run`, `poetry run`, `python3 -m`, `sudo`, `docker image build` and `docker compose up --build`, or narrow the `machine load:` decision's wording.** | 2 | 2 | 3 | 5 | 3 | 1.2 by 2, 1.5 by 2 | 4.0 |
|
||||
@@ -77,10 +76,6 @@
|
||||
|
||||
## Details
|
||||
|
||||
### 167. Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.
|
||||
|
||||
rule-hook checks Agent calls (item 163) but no Skill calls. A worker that invokes `/security-review` in its own context, instead of in a subagent, runs the pass on its own rung, and nothing blocks it. Check Skill calls too: `breaches_of` in bin/rule_hook/main.py takes a Skill call whose skill is `security-review` or `claude-security` as a review launch, and settings.json's PreToolUse matcher gains Skill. Found by the systems-architect's review of PR #295.
|
||||
|
||||
### 6. Move what `loop-claude` writes out of `~/.claude.json`, and ask the human about each write that must stay.
|
||||
|
||||
Claude Code owns that file and writes it. `trust` reads it, changes it and writes it back, so a write that lands in between is lost.
|
||||
|
||||
Reference in New Issue
Block a user