Merge remote-tracking branch 'origin/main' into item-500
This commit is contained in:
@@ -6,7 +6,7 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 511`
|
||||
`Next ID = 512`
|
||||
|
||||
## Items
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
| 502 | 0.15.0 | 501 | owner | **Distil each `task-bench` run into a digest of short traces and decoys, the files most failing runs read but no fix touches, and have a stronger model turn it into plan items before the transcripts are deleted.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, code health by 4 | 4.8 |
|
||||
| 469 | 0.15.0 | | | **Move a do-epic parent issue whose sub-tasks a person filed to Release Ready once every sub-task has landed, and to Done once every sub-task is done, by transitions its config names.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 4, 1.5 by 4, 1.6 by 3 | 3.1 |
|
||||
| 503 | 0.15.0 | 502 | owner | **Calibrate `task-bench` on fejkdata: bench a head against itself for the noise floor, then one finished chunk at its base and head, and record in a decision how many tasks and runs the bench needs.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3 | 1.2 |
|
||||
| 511 | 0.15.0 | | | **Hand the product-owner and prose reviewers the real output of each command a changed doc line says prints something, so a claim about output is checked by running it, not by reading code.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 3 | 1.1 |
|
||||
| 509 | 0.16.0 | 503 | owner | **Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores.** | 3 | 2 | 3 | 6 | 5 | code health by 5, 1.1 by 3 | 5.6 |
|
||||
| 504 | 0.16.0 | 503 | owner | **Bench new features in `task-bench`, each a spec for Haiku with acceptance tests it never sees and a reference patch that proves it can be built, kept in a `bench-tasks` folder the project config names, starting with six for fejkdata.** | 3 | 2 | 1 | 4 | 6 | 1.1 by 4, code health by 3 | 2.6 |
|
||||
| 505 | 0.16.0 | 504 | owner | **Bench adding source data in `task-bench`: hand Haiku a fake register that a seeded script generates for a made-up country, and check the importer, the data and the licence entry it adds, starting with five tasks for fejkdata.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 |
|
||||
@@ -200,6 +201,16 @@ Touches:
|
||||
- `docs/decisions.md`: `## task-bench: how many tasks and runs a bench takes`
|
||||
- `bin/task_bench/main.py`: `main` (small)
|
||||
|
||||
### 511. Hand the product-owner and prose reviewers the real output of each command a changed doc line says prints something, so a claim about output is checked by running it, not by reading code.
|
||||
|
||||
An outside reviewer found a README line, in a persona's description, that said a CLI error names the fix. Our product-owner pass checked the exit code claim against the source and corrected it, but passed the "names the fix" half, since the pass has Read, Grep and Glob and cannot run the command: the one-character input that triggers the error prints an error with no fix in it. A deterministic check would be better than a reviewer's reading, but no tool can tell which doc lines claim an output. The orchestrator can: where a doc line a chunk changes says what a command, flag or error prints, larv-review's text has it run that command in the chunk's worktree, through the project's container setup, and give each reviewer the real output beside the claim. Applies to every project whose docs state outputs; a reviewer still gets no shell.
|
||||
|
||||
Touches:
|
||||
|
||||
- `skills/larv-review/SKILL.md`
|
||||
- `agents/product-owner-reviewer.md`
|
||||
- `agents/prose-reviewer.md`
|
||||
|
||||
### 509. Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores.
|
||||
|
||||
It stays a report beside the panel until about 10 chunks show whether the two agree; whether it can replace the scoring run or gate a merge is the human's call then.
|
||||
|
||||
Reference in New Issue
Block a user