From 2a190c7700ee58d9e6a96c48dacb9257bc2750b6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 21 Nov 2026 13:28:23 +0100 Subject: [PATCH 1/3] File item 512: make stability-reviewer check that composed lookup keys cannot collide --- todo.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index 5ade30f5..5ba2af0b 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 0.1` -`Next ID = 512` +`Next ID = 513` ## Items @@ -18,6 +18,7 @@ | 505 | 0.16.0 | 504 | owner | **Bench adding source data in `task-bench`: hand Haiku a fake register that a seeded script generates for a made-up country, and check the importer, the data and the licence entry it adds, starting with five tasks for fejkdata.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 | | 508 | 0.16.0 | | owner | **Plant bugs for `task-bench` in Python, Rust, TypeScript and PHP, from the mutants `mutate-changes` runs for each, so this repo benches itself.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 | | 506 | 0.16.0 | 504 | owner | **Go over the bench features in each release review: reapply each reference patch, drop a feature that no longer breaks a goal, and add features until they cover the places the seats find hardest.** | 2 | 2 | 2 | 4 | 3 | 1.1 by 3 | 1.1 | +| 512 | 0.16.0 | | | **Make stability-reviewer check that keys built from path segments, or matched by prefix, cannot collide.** | 1 | 2 | 2 | 3 | 1 | 1.1 by 2 | 0.9 | | 507 | 0.16.0 | 504 | owner | **Bench rebuilding a feature the project shipped: revert its commit at the head, hand Haiku its item tagline, and check with the tests that commit added.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3 | 0.7 | | 477 | 0.17.0 | | question | **Rewrite `## Goals` and `## Audience` for a setup others install and run apart from their own `~/.claude`.** | 1 | 1 | 2 | 2 | 2 | | 0.0 | | 478 | 0.17.0 | | question | **Name the project, its repo and its two commands, today proposed as `do-release` and `do-epic`.** | 1 | 1 | 2 | 2 | 1 | | 0.0 | @@ -151,6 +152,13 @@ Each tool reports a killed mutant by file and line, and some by column; the plan A reference patch that no longer applies is rewritten by the review's model. A feature whose goal changed may have become real work, so it goes. New features follow item 504's rule: plausible for another audience, against one of this project's goals. +### 512. Make stability-reviewer check that keys built from path segments, or matched by prefix, cannot collide. + +An outside reviewer found two bugs in one chunk that our stability round passed. Both were lookups keyed by a path built from segments or matched by prefix, where two inputs give one key: a linked table that has a column of its own name took the column's row, and two modules whose namespaces nest (`ns` and `ns.sub`) were credited to the first prefix. Add a bullet beside the separator bullet in `agents/stability-reviewer.md` ยง5: where the change builds a lookup key by joining segments, or matches one by prefix, look for two inputs that give one key, such as a segment equal to its parent's name or a namespace that is a dot-prefix of another, and report the input the lookup credits to the wrong entry. + +Touches: +- `agents/stability-reviewer.md`: `### 5. Edge Cases & Input Validation` (small) + ### 507. Bench rebuilding a feature the project shipped: revert its commit at the head, hand Haiku its item tagline, and check with the tests that commit added. Such tasks cost nothing to write and are realistic. A commit whose revert doesn't apply cleanly at both the base and the head is skipped. From ec7a1613398b46785f08f06f055cda51987c92d3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 22 Nov 2026 15:50:23 +0100 Subject: [PATCH 2/3] Move item 503 to 0.15.1, since 0.15.0 is cut before its release review --- todo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/todo.md b/todo.md index 5ba2af0b..adcfef83 100644 --- a/todo.md +++ b/todo.md @@ -12,7 +12,7 @@ | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 503 | 0.15.0 | | owner | **Calibrate `task-bench` on fejkdata: bench a head against itself for the noise floor, then one finished chunk at its base and head, and record in a decision how many tasks and runs the bench needs.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3 | 1.2 | +| 503 | 0.15.1 | | owner | **Calibrate `task-bench` on fejkdata: bench a head against itself for the noise floor, then one finished chunk at its base and head, and record in a decision how many tasks and runs the bench needs.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3 | 1.2 | | 509 | 0.16.0 | 503 | owner | **Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores.** | 3 | 2 | 3 | 6 | 5 | code health by 5, 1.1 by 3 | 5.6 | | 504 | 0.16.0 | 503 | owner | **Bench new features in `task-bench`, each a spec for Haiku with acceptance tests it never sees and a reference patch that proves it can be built, kept in a `bench-tasks` folder the project config names, starting with six for fejkdata.** | 3 | 2 | 1 | 4 | 6 | 1.1 by 4, code health by 3 | 2.6 | | 505 | 0.16.0 | 504 | owner | **Bench adding source data in `task-bench`: hand Haiku a fake register that a seeded script generates for a made-up country, and check the importer, the data and the licence entry it adds, starting with five tasks for fejkdata.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 | From 0c2f2a5fd1a4011e99e899150f329bb5be44fb9f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 22 Nov 2026 15:52:21 +0100 Subject: [PATCH 3/3] Cut release 0.15.0 --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ecc68c25..1dc97850 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -## Unreleased +## 0.15.0 - `task-bench` writes `digest.md`, with each bug's decoys and up to two traces of its runs, and `task-bench gather` starts a claude on the project's rung that files a plan item for each place the digest shows to be hard, then removes the bench's folder. Each bench removes the folders earlier benches left, and `compress-archives` leaves lock files alone. - `task-bench` is new: it plants bugs that a Go project's tests catch, at the places the comprehension seats found hardest, and has Haiku fix each in a container at a base commit and at a head. It prints for each side how often the fix passes, with the median tokens, turns, wall time and reads before Haiku opens the file the bug is in. `task-bench --help` says how to run it.