From 6953dcb8e075bdb26d9a5cb64c75c5a5fd1ca27f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 21 Nov 2026 00:54:24 +0100 Subject: [PATCH] Finish item 511 in the plan --- CHANGELOG.md | 1 + todo.md | 11 ----------- 2 files changed, 1 insertion(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f7c8b957..07ca85bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## Unreleased +- larv-review now runs each command that a changed doc line says prints something and hands the real output to the product-owner and prose reviewers, so a claim about output is checked by running it. - `do-epic` moves a parent issue whose sub-tasks a person filed: to done by `jira-finish-transition` once every sub-task is done, and before that by the transition the new setting `jira-sub-tasks-landed-transition` names, once each sub-task is done or in one of `jira-unblocking-statuses`. The transition must lead to one of those statuses. `--dry-run` prints each such move. - mutate-changes now mutates the Go lines a branch changes with Gremlins, as it does Python, Rust, TypeScript and PHP, and prints each mutant the tests do not kill. - A project's config may set `epic-branch`, such as `epic-branch = "{epic}"`: do-epic then keeps one working branch per epic, level with the default branch by merge commits, and each chunk branches off it and merges into it without waiting for a person, even under `merge = "human"`. `chunk-pr open-epic` opens the finished epic's pull request into the default branch for a person to merge. `review-request-comment` names a comment `chunk-pr` posts after each push to ask a forge's reviewer for a review, and `jira-pull-request-open-status` and `jira-pull-request-merged-status` name the statuses do-epic moves an issue to as its pull request opens and merges. diff --git a/todo.md b/todo.md index c0f8ad1d..ac026004 100644 --- a/todo.md +++ b/todo.md @@ -15,7 +15,6 @@ | 501 | 0.15.0 | | owner | **Ship `task-bench`, which plants a bug the tests catch at the places the comprehension seats found hardest, has Haiku fix it in a vanilla Claude Code at a base and a head, and prints for each side how often the fix passes, with its median tokens, turns, wall time and reads before it opens a file the bug is in.** | 4 | 2 | 1 | 5 | 9 | 1.1 by 5, code health by 4 | 0.6 | | 502 | 0.15.0 | 501 | owner | **Distil each `task-bench` run into a digest of short traces and decoys, the files most failing runs read but no fix touches, and have a stronger model turn it into plan items before the transcripts are deleted.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, code health by 4 | 4.8 | | 503 | 0.15.0 | 502 | owner | **Calibrate `task-bench` on fejkdata: bench a head against itself for the noise floor, then one finished chunk at its base and head, and record in a decision how many tasks and runs the bench needs.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3 | 1.2 | -| 511 | 0.15.0 | | | **Hand the product-owner and prose reviewers the real output of each command a changed doc line says prints something, so a claim about output is checked by running it, not by reading code.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 3 | 1.1 | | 509 | 0.16.0 | 503 | owner | **Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores.** | 3 | 2 | 3 | 6 | 5 | code health by 5, 1.1 by 3 | 5.6 | | 504 | 0.16.0 | 503 | owner | **Bench new features in `task-bench`, each a spec for Haiku with acceptance tests it never sees and a reference patch that proves it can be built, kept in a `bench-tasks` folder the project config names, starting with six for fejkdata.** | 3 | 2 | 1 | 4 | 6 | 1.1 by 4, code health by 3 | 2.6 | | 505 | 0.16.0 | 504 | owner | **Bench adding source data in `task-bench`: hand Haiku a fake register that a seeded script generates for a made-up country, and check the importer, the data and the licence entry it adds, starting with five tasks for fejkdata.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 | @@ -156,16 +155,6 @@ Touches: - `docs/decisions.md`: `## task-bench: how many tasks and runs a bench takes` - `bin/task_bench/main.py`: `main` (small) -### 511. Hand the product-owner and prose reviewers the real output of each command a changed doc line says prints something, so a claim about output is checked by running it, not by reading code. - -An outside reviewer found a README line, in a persona's description, that said a CLI error names the fix. Our product-owner pass checked the exit code claim against the source and corrected it, but passed the "names the fix" half, since the pass has Read, Grep and Glob and cannot run the command: the one-character input that triggers the error prints an error with no fix in it. A deterministic check would be better than a reviewer's reading, but no tool can tell which doc lines claim an output. The orchestrator can: where a doc line a chunk changes says what a command, flag or error prints, larv-review's text has it run that command in the chunk's worktree, through the project's container setup, and give each reviewer the real output beside the claim. Applies to every project whose docs state outputs; a reviewer still gets no shell. - -Touches: - -- `skills/larv-review/SKILL.md` -- `agents/product-owner-reviewer.md` -- `agents/prose-reviewer.md` - ### 509. Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores. It stays a report beside the panel until about 10 chunks show whether the two agree; whether it can replace the scoring run or gate a merge is the human's call then.