Answer prose round 1: precedent names what a pass skips on, the bench and floor-decision prose reads on one pass
This commit is contained in:
@@ -1,8 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run larv-review bench runs: BENCH_SOURCES=<dir> bench.py <case>-<version>-<n> ...
|
||||
|
||||
The 2026-09-30 bench had two versions of larv-review, main and rebuild, each review four real chunks,
|
||||
the cases, three times; compare.py then rules which version reviews better. Its inputs are untracked:
|
||||
The 2026-09-30 bench ran two versions of larv-review, main and rebuild. Each reviewed four real chunks,
|
||||
the cases, three times; compare.py then rules whether each version reviews at least as well as the
|
||||
other. The bench's inputs are untracked:
|
||||
- versions/<version>/ beside this script: a copy of ~/.claude at that version, which each run reads
|
||||
in place of ~/.claude (run-prompt.md).
|
||||
- $BENCH_SOURCES/<case>-main/: a clone of the case's project; CASES names the chunk's base and head
|
||||
|
||||
@@ -29,7 +29,7 @@ def runs(case, version):
|
||||
|
||||
|
||||
def blind(text):
|
||||
"""The report with every name that tells which version wrote it replaced."""
|
||||
"""The report, with VERSION_DIR in place of every name that tells which version wrote it."""
|
||||
return re.sub(r"\S*versions/(main|rebuild)\S*|\b(main|rebuild)\b(?=[`\s]*(process|version))", "VERSION_DIR", text)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user