3.0 KiB
3.0 KiB
You are one run of a bench comparing two versions of the larv-review process. Review the chunk in this repository (your working directory) exactly as the process below says, then write a report.
VERSION_DIR = {version_dir}
Chunk: commits {base}..chunk on branch chunk (head at start: {head}). Case {case}.
Report: write it to {report}.
The process
VERSION_DIR/skills/larv-review/SKILL.mdis larv-review. Do not invoke the installedlarv-reviewskill.- Wherever a text cites
~/.claude/<path>, readVERSION_DIR/<path>instead.VERSION_DIR/CLAUDE.mdreplaces~/.claude/CLAUDE.md, which you already have loaded: where the two differ, VERSION_DIR's wins. - Launch every reviewer named
<name>under~/.claude/agents/as ageneral-purposeagent whose prompt opens with the body ofVERSION_DIR/agents/<name>.md(frontmatter dropped), then the task. A reviewer the process names that has no file there gets the process's own description of it as its prompt. /security-review: launch ageneral-purposeagent that invokes the Skill tool'ssecurity-review, doesn't edit anything, and replies with the skill's report, or that the skill is unavailable. Where that skill is unavailable, emulate it with ageneral-purposeagent, told to review the branch diff for exploitable vulnerabilities and report only those at ≥0.8 confidence.
Bench rules
- Launch every agent in the foreground (
run_in_background: false): background agents die when this headless turn ends. - No remote and no forge: skip the PR, CI, the external auto-reviewer and the merge; review locally.
- Run every git command through
python3 {review_git} <subcommand> <args>, never baregit: that wrapper is the only shell command allowed. It runslog,diff,show,status,ls-files,addandcommit. Where the process asks for another git command, do without it and list it under Ambiguities. Run no other code or test suite. Where the process asks for a run, such as the stability pass's mutation test, review by reading and list the skipped run under Ambiguities. - Skip the comprehension check; say so in the report.
- No human is reachable. Where the process names a no-human branch, take it; where it names none, take your recommendation and list the question in the report.
- Commit each round's fixes on
chunkwithpython3 {review_git} commit -m <message>, messages prefixed{case} - review:. Don't touch anything outside this repository except the report.
Report
Sections, in order:
- Passes: a table of each pass, run or skipped, and why.
- Findings: per pass and round, each finding's severity, location, one-line summary and outcome (fixed in
<sha>, answered, declined, filed, disproved). - Questions for the human: each ask you could not put, with your recommendation.
- Totals: agents launched per pass, rounds per pass, commits.
- Ambiguities: every place the process text was unclear, contradicted itself, or could not be followed, and how you read it.