Answer the first review round
This commit is contained in:
+4
-2
@@ -33,5 +33,7 @@
|
||||
!skills/larv-review/
|
||||
!skills/larv-review/*.md
|
||||
!workflow-bench/
|
||||
!workflow-bench/*.md
|
||||
!workflow-bench/*.py
|
||||
!workflow-bench/bench.py
|
||||
!workflow-bench/judge-prompt.md
|
||||
!workflow-bench/judge.py
|
||||
!workflow-bench/run-prompt.md
|
||||
|
||||
@@ -129,8 +129,8 @@ The rules every project follows, compose-not-entangle first: `~/.claude/principl
|
||||
- Use `gh` on GitHub remotes and `tea` on Gitea remotes (e.g. `gitea.larvit.se`) — same command shape: `tea pr list|create`, `tea issue`, `tea release`, `tea api`. Everything in this section applies to both.
|
||||
|
||||
## Handoffs
|
||||
- Write no handoff document. The repo holds everything the next session needs: the plan, the decisions, and every script, sketch or measurement they depend on. A file outside the repo is unreachable from another machine's checkout.
|
||||
- Four kinds stay out of the repo, in a memory file: a secret's value; where this machine keeps its copy of one; a quirk of the agent's own environment that a human contributor does not share; data the repo may not publish, unlicensed or holding PII.
|
||||
- Write no handoff document. The repo holds everything the next session needs: the plan, the decisions, and every script, sketch or measurement those depend on. A file outside the repo is unreachable from another machine's checkout.
|
||||
- Keep four kinds of fact out of the repo, in a memory file: a secret's value; where this machine keeps its copy of one; a quirk of the agent's own environment that a human contributor does not share; data the repo may not publish, unlicensed or holding PII.
|
||||
|
||||
## Web research
|
||||
- **A list or threshold a consumer relies on is sourced.** Deny-lists, defaults, limits: follow a published standard or a widely deployed tool, and cite it in the decision recording the choice; where none exists, ask. Never assemble one from names that seem plausible.
|
||||
|
||||
@@ -48,6 +48,6 @@ The panels on `rebuild` at 8585257 and on PR #182 named these under Self-suffici
|
||||
- Have `loop-claude` write the line to one file at the root of the transcript archive. The line names the project, the commit the chunk started from and the archived transcript's hash. It holds the worker's tokens from the transcript, the judge's from its JSON output, and the chunk's cycles and interrupts, which the loop counts.
|
||||
- End larv-review with a machine-readable summary, and have `loop-claude` copy it from the transcript onto the line: agents launched and rounds per pass; findings per pass by severity, inside the chunk's diff and outside it; and the comprehension scores when the panel ran.
|
||||
- **Write a `workflow-review` skill that reads the stats and files each change it proposes to the workflow as a plan item, and have `AGENTS.md` run it on the chunk shipping a release's last item.** The workflow is `CLAUDE.md`, `principles/`, the skills, the agents and `loop-claude`. The skill's signals: findings outside the chunk's diff and comprehension scores per release, both from the stats; fixes after merge, from git; and the human's corrections, from the archived transcripts. Goal 4.4.
|
||||
- **Bench about one chunk in ten against an open PR that changes the workflow, and have a blind claude rule which run better serves this repo's goal 4.** Redo the chunk under the PR, from the commit it started from. The ruling claude is not told which run is which. `loop-claude` picks the chunk at random, small diffs and big ones alike; with no such PR open, no bench runs. The same outcome with less code and prose, in the chunk or in the workflow, also counts as better. The PR merges only once three benches rule it better. A bench that rules it worse closes the PR and records, in `docs/workflow-learnings.md`, what the PR changed and why it lost. The ruling and both runs' numbers go on the chunk's stats line. Start from `workflow-bench/`, written for that bench's four fixed cases. Needs the stats line. Goals 4.1 and 4.4.
|
||||
- **Bench about one chunk in ten against an open PR that changes the workflow, and have a blind claude rule which run better serves this repo's goal 4.** Redo the chunk under the PR, from the commit it started from. The ruling claude is not told which run is which. `loop-claude` picks the chunk at random, small diffs and big ones alike; with no such PR open, no bench runs. The same outcome with less code and prose, in the chunk or in the workflow, also counts as better. The PR merges only once three benches rule it better. A bench that rules it worse closes the PR and records, in `docs/workflow-learnings.md`, what the PR changed and why it lost. The ruling and both runs' numbers go on the chunk's stats line. Start from `workflow-bench/`, written for the 2026-09-30 bench's four fixed cases. Needs the stats line. Goals 4.1 and 4.4.
|
||||
- **Bring the project to the comprehension floor on the four-seat panel.** Every dimension and the overall must reach 7.0, and none may fall below its best run on `rebuild`. A dimension under the floor gets its places filed under "Under the comprehension floor".
|
||||
- **Wait out a usage limit and retry however long it lasts, without counting it as a failed run.** Today a worker or judge that hits a usage limit fails three times a minute apart, and the loop exits 1, so an unattended loop dies at every limit. The transcript line Claude Code writes for it, marked `"error":"rate_limit"`, holds the reset time: `You've hit your session limit · resets 7:40pm (Europe/Stockholm)`. Sleep until 2 minutes past it. The judge runs without a transcript, so find what its JSON reply carries on a limit. Where no reset time can be read, retry every 10 minutes: a retry before the reset fails (https://platform.claude.com/docs/en/api/rate-limits), so polling sooner only spends requests. Print when the next try is due, and update `--help`'s exit 1. Goal 4.
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run larv-review bench runs: bench.py <case>-<version>-<n> ..."""
|
||||
"""Run larv-review bench runs: BENCH_SOURCES=<dir of the <case>-main clones> bench.py <case>-<version>-<n> ..."""
|
||||
import concurrent.futures
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
@@ -9,7 +10,7 @@ import sys
|
||||
import time
|
||||
|
||||
BENCH = pathlib.Path(__file__).resolve().parent
|
||||
SOURCES = BENCH.parent.parent / "2026-09-29" / "bench"
|
||||
SOURCES = pathlib.Path(os.environ["BENCH_SOURCES"])
|
||||
CASES = {
|
||||
"cjk": ("151bf38^", "6c49912"),
|
||||
"35b": ("692433b", "1d369ec"),
|
||||
|
||||
Reference in New Issue
Block a user