File items 513 to 517: prose negations, git in tests, shell mutation, secrets in Docker builds, security pass in worktrees #690
@@ -6,7 +6,7 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 513`
|
||||
`Next ID = 518`
|
||||
|
||||
## Items
|
||||
|
||||
@@ -43,6 +43,7 @@
|
||||
| 496 | 0.22.0 | 478, 491 | owner | **Rename the repo to the name item 478 picks, and rename what says Claude where it means any harness, such as `CLAUDE.md`.** | 3 | 2 | 3 | 6 | 6 | code health by 2 | 1.3 |
|
||||
| 494 | 0.22.0 | 477, 483 | owner | **Let a newcomer install, log in and run a first project from the README alone, with an example config, and test a fresh install in a container in the gate.** | 2 | 2 | 2 | 4 | 5 | 1.6 by 4 | 0.8 |
|
||||
| 446 | 0.23.0 | | question | **Kill the process a stuck check names where its verdict is `stuck`, once the human says the recorded verdicts match what they would have done.** | 2 | 2 | 3 | 5 | 3 | 1.5 by 3 | 1.0 |
|
||||
| 516 | 0.23.0 | | | **Refuse a `.dockerignore` that leaves out a secret-named entry the `.gitignore` beside it ignores, where a Dockerfile copies the whole build context.** | 2 | 2 | 3 | 5 | 3 | 4 by 4 | 5.0 |
|
||||
| 462 | 0.23.0 | | | **Run under `mutate-changes` the tests that today fail in mutmut's copy of the tree, so a mutant of a line only they cover is killed.** | 2 | 1 | 2 | 3 | 3 | code health by 3 | 4.7 |
|
||||
| 77 | 0.23.0 | | | **Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.** | 4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 2.1 |
|
||||
| 196 | 0.23.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 1.2 |
|
||||
@@ -86,9 +87,12 @@
|
||||
| 266 | 0.23.0 | | | **Move the chunks archived under the key `home/lilleman/.claude` to the key `claude-config`, so the routing figures and `score-outcomes` count this repo's chunks as one project.** | 3 | 2 | 2 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 339 | 0.23.0 | | | **Lower the reading cost of the files a chunk reads most before its first commit.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 341 | 0.23.0 | | | **Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 517 | 0.23.0 | | | **Let a subagent that ships a chunk in a worktree of another repo run larv-review's security pass on that worktree's diff.** | 2 | 2 | 3 | 5 | 3 | 1.5 by 3 | 1.0 |
|
||||
| 513 | 0.23.0 | | | **Refuse `nothing`, `no one` and `nobody` after a verb or a preposition in everything Claude writes: files, commit messages, pull requests and Jira text before they are written, and chat replies by sending them back to be reworded.** | 3 | 2 | 3 | 6 | 5 | 1.1 by 4 | 0.9 |
|
||||
| 181 | 0.23.0 | | principle | **Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through `prompts.rungs_held`.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 193 | 0.23.0 | | | **Narrow `numbered`'s mark to `Mark` by a typed lookup in place of `typing.cast`, so a mark added to the regex but not the lookup fails at run time, and one added to the lookup but not `Mark` fails mypy.** | 1 | 1 | 3 | 3 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 194 | 0.23.0 | | | **Make `test_waits_out_a_usage_limit_which_ends_a_run_of_failures` expect a usage-limit wait independent of the wall-clock time that passes during the run.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 514 | 0.23.0 | | | **Make stability-reviewer check that a test which runs git, or a script that runs git on another repository, isolates it from the caller's git environment.** | 1 | 2 | 2 | 3 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 298 | 0.23.0 | | | **Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.** | 3 | 3 | 3 | 7 | 4 | 1.4 by 2, 1.5 by 1 | 0.9 |
|
||||
| 265 | 0.23.0 | | | **Archive each wait that costs a chunk wall time, for a usage limit, low memory, a failed worker start or a heavy slot, with its start and end stamps, so goal 1.5's hours split waiting from work.** | 3 | 2 | 2 | 5 | 4 | 1.5 by 3 | 0.8 |
|
||||
| 475 | 0.23.0 | | | **Have stability-reviewer report, as a finding, each lookup that an input from outside the repo can make fail without an error naming what was missing.** | 1 | 1 | 2 | 2 | 2 | 1.1 by 2 | 0.8 |
|
||||
@@ -97,6 +101,7 @@
|
||||
| 169 | 0.23.0 | | | **Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.** | 3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 0.7 |
|
||||
| 447 | 0.23.0 | | | **Stop a headless claude call that runs past a time limit, and report it as a failure, so a hung judge or stuck check can't hold the loop.** | 2 | 2 | 2 | 4 | 3 | 1.5 by 2 | 0.7 |
|
||||
| 367 | 0.23.0 | | | **Check that every write to a Jira plan leaves it ranked, as rule-hook's `todo-rank-check` checks `todo.md` today.** | 2 | 2 | 3 | 5 | 3 | 1.6 by 2 | 0.7 |
|
||||
| 515 | 0.23.0 | | | **Mutate the shell lines a branch changes in `mutate-changes`, so a change to a shell script gets the same check as one in Python, Go or PHP.** | 3 | 2 | 2 | 5 | 5 | 1.1 by 2 | 0.5 |
|
||||
| 423 | 0.23.0 | | | **Skip the judge's model call on a worker's final message that the offline classifier decides is finished.** | 3 | 3 | 3 | 7 | 5 | 1.4 by 2 | 0.4 |
|
||||
| 197 | 0.23.0 | | | **Let the human edit what they type at loop-claude's terminal questions with the arrow keys, so Left moves the cursor in place of printing `^[[D`.** | 2 | 2 | 3 | 5 | 2 | 1.5 by 1 | 0.4 |
|
||||
| 1 | 0.24.0 | | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 1 | 2 | 1 | 2 | 1 | | 0.0 |
|
||||
@@ -257,6 +262,17 @@ Touches:
|
||||
- `bin/loop_effects/child_shells.py`: `stop` (small)
|
||||
- `bin/loop_claude/loop/usage.py`: `HELP` (small)
|
||||
|
||||
### 516. Refuse a `.dockerignore` that leaves out a secret-named entry the `.gitignore` beside it ignores, where a Dockerfile copies the whole build context.
|
||||
|
||||
An outside reviewer caught what our review missed: a repo's `.gitignore` ignored `auth.json`, Composer's credentials file, while its `.dockerignore` didn't, and its Dockerfiles run `COPY . .`. A developer's local `auth.json` could then reach the image. Today rule-hook checks ignore files only for sort order (`ignorefile-sorted`).
|
||||
|
||||
Add a rule-hook check on Write and Edit of `.gitignore`, `.dockerignore` and `Dockerfile*`, and a gate check:
|
||||
- where a Dockerfile in the repo copies the whole context (`COPY . <dest>` or `ADD . <dest>`),
|
||||
- each `.gitignore` entry whose name marks a secret (`.env`, `.env.*`, `auth.json`, `*.pem`, `*.key`, `id_*`, `credentials*`, `*secret*`)
|
||||
- must also be in `.dockerignore`, or be excluded by a pattern there.
|
||||
|
||||
Name the entry and the Dockerfile line. Source the secret-name list from a widely used one, such as GitHub's gitignore templates or gitleaks' default rules, and cite it in the decision that records the list.
|
||||
|
||||
### 462. Run under `mutate-changes` the tests that today fail in mutmut's copy of the tree, so a mutant of a line only they cover is killed.
|
||||
|
||||
Found while shipping item 461: `mutate-changes` printed `23 mutants on changed lines, 18 not killed. Left out the tests that fail without a mutant, in do_epic/test_main.py, larv_review/test_larv_review.py, larv_review/test_scoring_setup.py, plan/test_ranking.py, repo_checks/…, review_rules/test_review_passes.py, review_rules/test_reviewer_context.py, rule_hook/test_rule_hook.py, test_settings_filter.py`. All 18 survivors were in `bin/larv_review/scoring_setup.py`: `prepare_scoring`, whose tests are among those left out, so the chunk checked four of them by hand and left 14 unchecked. Find why each left-out test fails in mutmut's copy, such as a path to `skills/` or a git checkout it expects, and make it pass there, or report under the count which changed lines no remaining test covers.
|
||||
@@ -596,6 +612,31 @@ Touches:
|
||||
- `bin/agent_cycle/test_run.py`: `Cycles` (small)
|
||||
- `bin/loop_claude/loop/test_usage.py`: `HelpLimits` (small)
|
||||
|
||||
### 517. Let a subagent that ships a chunk in a worktree of another repo run larv-review's security pass on that worktree's diff.
|
||||
|
||||
On 2026-11-22, eleven subagents each shipped a chunk in a worktree of a different repo. Every one of them hit the same problem at the security pass:
|
||||
- `/security-review` reads the diff of the session's working directory, which a subagent inherits from the main session.
|
||||
- A `cd` inside the subagent doesn't reach the skill.
|
||||
- `EnterWorktree` refuses a worktree of another repo.
|
||||
|
||||
So each subagent stopped before opening its pull request, and the main session had to `cd` into each worktree and launch the security subagent from there. Every chunk waited on the main session. Some subagents also reviewed by hand, which larv-review forbids.
|
||||
|
||||
Make one of these work, and test it with a subagent in a worktree of another repo:
|
||||
- `EnterWorktree`, or a launch option, sets a subagent's working directory to any worktree path;
|
||||
- or larv-review's security launch names the worktree and the base ref, and the skill reviews `git diff <base>...HEAD` there.
|
||||
|
||||
Then larv-review §3 → "Security" says how a subagent in another repo's worktree runs the pass.
|
||||
|
||||
### 513. Refuse `nothing`, `no one` and `nobody` after a verb or a preposition in everything Claude writes: files, commit messages, pull requests and Jira text before they are written, and chat replies by sending them back to be reworded.
|
||||
|
||||
The human can't stand the `changes nothing` form, and `CLAUDE.md` → "Prose" bans it, yet it still slips through: on 2026-11-22 a draft goal read "a repo that calls the action and changes nothing keeps working". A prose rule only works when the model notices, so move the check into tools, which catch it every time.
|
||||
|
||||
- **rule-hook, before the write:** refuse a Write, Edit or MultiEdit whose new text in a prose file (`.md`, or a comment or docstring) holds the form. Do the same for a Bash call whose commit message, `chunk-pr`/`gh`/`tea` pull request text or `jira-rest` body holds it. Name the phrase and suggest the `doesn't <verb> anything` form.
|
||||
- **A Stop hook for chat:** chat can't be filtered before it shows, so a Stop hook reads the last assistant message from the transcript and, where it holds the form, blocks once with the phrase, so the reply is followed by a reworded one. Leave out code blocks and quoted text.
|
||||
- **The gate:** a check in `ci.sh` over the shipped prose, comments, docstrings and test names. Fix what it finds today, such as `bin/rule_hook/endless_wait.py` ("nothing bounds it"), `bin/rule_hook/reviewer_rung.py` ("nothing tells which rung") and the test names `test_a_blocked_call_overrides_nothing_yet` and `test_a_missing_field_breaks_nothing`.
|
||||
|
||||
The form is the word after another word inside a clause: `<word> nothing`, `<word> no one`, `<word> nobody`. A clause that opens with it, as in "Nothing changes", is fine. Bench the pattern on this repo's prose and a few client repos' commit history before it blocks: a false positive blocks every session's writes (I 3).
|
||||
|
||||
### 181. Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through `prompts.rungs_held`.
|
||||
|
||||
`bin/agent_cycle/test_run.py`'s `test_a_quoted_answer_that_cites_an_escalation_trigger_steps_the_worker_up_before_it_resumes_and_archives_the_step` builds its expected prompt from `prompts.rungs_held`, the code under test, so a bug there passes it. The technical principles' test bullet bans that shape. Assert the new rung and one review launch as literals, as the typed answer's step-up test does. Raised by the principles audit of PR #308.
|
||||
@@ -608,6 +649,18 @@ Found by review of PR #317. `bin/plan/goals_file.py`'s `numbered` writes `typing
|
||||
|
||||
Found by review of PR #317. The black-box test, `bin/loop_claude/test_black_box.py:609`, not the unit test of the same name in `bin/agent_cycle/test_retry.py`, which runs on a fake clock, sets `resets_at` from the test's `time.time()` and expects the wait, rounded to tens, to be 3720. When more than about 5 s pass between that read and the loop's, the wait rounds to 3710 and the test fails with no bug. Item 102 covers only replacing the `time.sleep` swap; this item changes the expected value, for example by asserting the wait against `resets_at + 120` minus the time the loop read, or a bound on it.
|
||||
|
||||
### 514. Make stability-reviewer check that a test which runs git, or a script that runs git on another repository, isolates it from the caller's git environment.
|
||||
|
||||
An outside reviewer caught what our review missed, twice in one day:
|
||||
- **A test inherited the environment of whoever ran it.** The test ran git through the script under test. The stability reviewer found one half: the developer's global config, where `commit.gpgsign=true` made every test error. The forge's reviewer found the other: a `GIT_DIR` set by a hook or an outer git command points git at another repository.
|
||||
- **A script did the same.** It ran `git -C <clone>` on another repository, so an inherited `GIT_DIR` or `GIT_WORK_TREE` sent its `rev-parse` and `checkout -B` to the caller's repository instead.
|
||||
|
||||
Add one check to the stability reviewer (`agents/stability-reviewer.md`), or to the testing principle it cites:
|
||||
- A test that runs git, or runs a script that does, sets `GIT_CONFIG_GLOBAL=/dev/null` and `GIT_CONFIG_NOSYSTEM=1`, and clears `GIT_DIR`, `GIT_WORK_TREE`, `GIT_INDEX_FILE`, `GIT_OBJECT_DIRECTORY` and `GIT_ALTERNATE_OBJECT_DIRECTORIES`.
|
||||
- A script that runs git on a repository other than its own clears the same repository variables first.
|
||||
|
||||
A finding names the test or script line and the variables it inherits.
|
||||
|
||||
### 298. Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.
|
||||
|
||||
153 `Judge cost:` entries total $109.76, a median of about $0.76 a verdict at every rung, and the cost follows the size of the records, not the transcript: about 76k characters in log and 188k in claude-config. In 224 of 251 recent verdicts (89%) the evidence sits in the worker's final message. A missed interrupt is the costly error, so the bar is zero changed interrupts and zero new rejections across the replay. The judge is about 4.5% of dollars, so the saving is small; each cycle also gets faster.
|
||||
@@ -660,6 +713,19 @@ Touches:
|
||||
|
||||
One of 0.13.0's history ports, per lilleman's call of 2026-11-12.
|
||||
|
||||
### 515. Mutate the shell lines a branch changes in `mutate-changes`, so a change to a shell script gets the same check as one in Python, Go or PHP.
|
||||
|
||||
Ten client chunks in one day changed only shell scripts. Each found that `mutate-changes` prints "Not mutated, as no tool covers their language" for `.sh` files. Each stability reviewer then mutated lines by hand: 10 to 26 mutants per chunk, judged by eye, with a different operator set each time.
|
||||
|
||||
First look for a maintained mutation tool for POSIX sh and Bash, and cite what was found. Where none is maintained, add a small built-in operator set to `mutate-changes`, applied only to changed lines:
|
||||
- negate a test (`[ ... ]`, `[[ ... ]]`, `test`);
|
||||
- swap `-n` and `-z`, `=` and `!=`, `&&` and `||`;
|
||||
- drop a command from a pipeline or a list;
|
||||
- empty a variable expansion's default (`${X:-y}` becomes `${X}`);
|
||||
- remove an `exit 1` or a `return 1`.
|
||||
|
||||
Run the repo's own gate per mutant, as for the other languages, and print each survivor.
|
||||
|
||||
### 423. Skip the judge's model call on a worker's final message that the offline classifier decides is finished.
|
||||
|
||||
On 2026-11-18 lilleman ruled that the classifier's miss rate of about 1 in 100 decided is close enough to no missed interrupt (`docs/decisions.md` → `routing: the offline classifier may decide finished, since lilleman ruled a miss rate of about 1 in 100 decided close enough to none; the judge's model keeps every other verdict kind`).
|
||||
|
||||
Reference in New Issue
Block a user