Answer prose round 1: 0 🔴, 2 🟡 (0 misread); the bullet counts six chunks and the five that went on
CI / tests (pull_request) Successful in 3m53s
CI / tests (pull_request) Successful in 3m53s
This commit is contained in:
+1
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Add `workflow-bench/handoff_bench.py`, a bench of how a worker's step to a stronger model hands over the chunk, on five archived L 3 chunks. Opus 5.5 resuming Sonnet 5.5's session passed two and was partial on three, for a mean $0.68; a fresh session with a brief passed one and was partial on four, for $0.78. So a stepped worker still resumes its session.
|
||||
- Add `workflow-bench/handoff_bench.py`, a bench of how a worker's step to a stronger model hands over the chunk, on six archived L 3 chunks. Sonnet 5.5 left five unfinished at its budget. On those, Opus 5.5 resuming Sonnet 5.5's session passed two and was partial on three, for a mean $0.68; a fresh session with a brief passed one and was partial on four, for $0.78. So a stepped worker still resumes its session.
|
||||
- todo-item edit --blocked-by sets the "is blocked by" links of a Jira issue the loop filed.
|
||||
- A `Touches:` list in the description of a Jira issue is read as one in `todo.md` is, a path or name that holds a backtick included: the Jira description spells it in backticks with no padding spaces, and the prediction reader takes that spelling too.
|
||||
- `todo-item edit --detail` and `project-record save` leave a Jira description or a Confluence page exactly as it is, with its block ids and spacing paragraphs, where the text saved is the text a read showed. An unedited save over a page a teammate saved since still refuses.
|
||||
|
||||
+1
-1
@@ -578,7 +578,7 @@ The step is one rung up from the rung the worker would otherwise hold: its own,
|
||||
|
||||
The archive quotes the judge's passage under `Escalation:` in the unfinished verdict, the interrupt or the quoted answer. The step an unfinished verdict causes is archived after it. The step an interrupt or a quoted answer causes is archived with it, at once: the interrupt or answer names the rung the worker ran on until then, and a step-up archived beside it names the rung it runs on from then. So the routing figures charge each stretch to its own rung, and a loop that dies while the human answers resumes on the stepped rung. Each archived worker line names the mark once there is one, so a resumed loop keeps it. `bin/routing_figures/` counts none of these steps as an escalation by trigger, which it counts only from subagent prompts.
|
||||
|
||||
A stepped worker resumes its session, also when the step changes its model. A plan item, since dropped, proposed a fresh session with a brief of the chunk in its place, after arXiv 2608.24358 measured a fresh strong model ahead of one handed the weak model's transcript. `workflow-bench/handoff_bench.py` benched it on 2026-11-17, on L 3 items this repo shipped. Sonnet 5.5 at high effort started each of six and was stopped at $0.36. It stopped itself on item 163 as it reached the budget, which the bench reads as a finished run, so five items went on, each stopped after 9 to 13 turns. Opus 5.5 at medium effort then finished it three ways: resuming the session, resuming it after `/compact`, and in a fresh session with the item's text and a brief, which says that the unfinished change lies uncommitted in the clone. A judge that did not know the arm graded each change. It saw the item, the change that shipped for it, and the result of the tests run on the worker's tree.
|
||||
A stepped worker resumes its session, also when the step changes its model. A plan item, since dropped, proposed a fresh session with a brief of the chunk in its place, after arXiv 2608.24358 measured a fresh strong model ahead of one handed the weak model's transcript. `workflow-bench/handoff_bench.py` benched it on 2026-11-17, on L 3 items this repo shipped. Sonnet 5.5 at high effort started each of six and was stopped at $0.36. It stopped itself on item 163 as it reached the budget, which the bench reads as a finished run, so five items went on, each stopped after 9 to 13 turns. Opus 5.5 at medium effort then finished each of the five three ways: resuming the session, resuming it after `/compact`, and in a fresh session with the item's text and a brief, which says that the unfinished change lies uncommitted in the clone. A judge that did not know the arm graded each change. It saw the item, the change that shipped for it, and the result of the tests run on the worker's tree.
|
||||
|
||||
| arm | runs | pass | partial | fail | mean cost $ | mean seconds |
|
||||
|---|---|---|---|---|---|---|
|
||||
|
||||
Reference in New Issue
Block a user