Answer stability round 4: 2 🔴, 1 🟡; the wrapper allows only listed options, takes the clone from the environment, and skips only a message its flag takes whole

This commit is contained in:
2026-10-16 22:30:24 +02:00
parent 4438ab04f3
commit b30c91aafc
9 changed files with 258 additions and 85 deletions
+2 -1
View File
@@ -88,5 +88,6 @@
!workflow-bench/bench.py
!workflow-bench/compare-prompt.md
!workflow-bench/compare.py
!workflow-bench/review-git.py
!workflow-bench/review_git.py
!workflow-bench/run-prompt.md
!workflow-bench/test_review_git.py
+1 -1
View File
@@ -2,7 +2,7 @@
## Unreleased
- `workflow-bench/bench.py` gives each run a fixed list of allowed tools, where it skipped every permission check. A run may launch agents, run `/security-review`, edit its clone (not the clone's `.git/` or `.claude/`) and its report, and read its clone and the copy of `~/.claude` it benches. For git it gets one command, `workflow-bench/review-git.py`, and no bare `git`: the wrapper runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit` under the clone with a scrubbed config, refusing an argument that names a path outside the clone and any option that reads a file, writes a file or runs a program. So no git argument, option, hook, redirect or shell substitution reaches beyond the clone. It loads no user settings, so `~/.claude/settings.json`'s hooks and extra directories do not apply to it. Every other call is denied. Expect a benched review that needs to run a project's code or tests to say in its report that it could not.
- `workflow-bench/bench.py` gives each run a fixed list of allowed tools. Before, it skipped every permission check. A run may launch agents, run `/security-review`, edit its clone (not the clone's `.git/` or `.claude/`) and its report, and read its clone and the copy of `~/.claude` it benches. For git it gets one command, `workflow-bench/review_git.py`, and no bare `git`. The wrapper runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit -m` on the run's clone, whatever the working directory. It allows only the options it lists for each, and refuses an argument that names a path outside the clone. git runs with no global or system config, hook, pager or editor. A run loads no user settings, so `~/.claude/settings.json`'s hooks and extra directories do not apply to it. Every other call is denied. Expect a benched review that needs another git command, or to run a project's code or tests, to say in its report that it could not.
- Every goal in `README.md` → Goals is marked `**Maintain:**`, a state to hold, or `**Achieve:**`, a direction to pursue. Each achieve goal has a signal, a counter-signal and a one-line reason under it. A change that breaks a maintain goal is redesigned or dropped, whatever it scores: `rules/todo-items.md` says so, and larv-review's goal alignment step no longer merges such a PR. `rules/goals-and-audience.md` states the form for every project's README. Expect larv-review to stop a PR on a maintain goal where it asked you before.
- Reshape every project's `todo.md` to the new shape below: until it is reshaped, that is the project's only work, as it is while README.md lacks Goals and Audience, and `rank_todo.py` says so. First move every machine to this release: an older release rejects the new formula, so it fails on every reshaped `todo.md`. Then mark the README's goals `**Maintain:**` or `**Achieve:**`, score each item's L, E and I, and set a bar on the new scale below the score of every item to keep, since the old bar would drop most items. Until then, an old-shape `todo.md` still ranks by its own formula and weights.
- `todo.md` ranks items by value per cost: this setup's own plan uses `Score = 100 * V / (2**S + 8)`, where 8 is a chunk's fixed cost, about one size-3 change, and `rank_todo.py --help` shows it. A Moves column replaces A, G and Goals, naming each achieve goal an item moves and how far, such as `1.1 by 8, 1.2 by 5`. V sums each named goal's weight times its move. `rank_todo.py` weighs each achieve goal by its position in README.md, the first heaviest, so the goal weights table goes. An item names the most specific goal it moves, never a goal beside its own sub-goal. `rules/todo-items.md` and `rank_todo.py --help` show the new shape.
+1 -1
View File
@@ -94,7 +94,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `statusline.py` renders Claude Code's status line; `settings.json` → `statusLine` runs it.
- `subagent-rung [--plan] <model> <L>` prints the Agent tool's model and effort for a subagent's task, picked by its L from the project's model, and with `--plan` never below the project's rung; `subagent-rung --help` states the rule.
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`. Test the bench's git wrapper with `python3 -m unittest discover -s workflow-bench -p 'test_*.py'`.
To mutation-test a changed module, copy `bin/` to a scratch directory (`cp -r bin "$scratch"`): mutmut writes files beside its config. In the copy, name the module and its tests in a `setup.cfg`, as below for `score_outcomes/outcomes.py`. Leave out each test class that starts a tool's launcher through `subprocess`: a mutated module fails to load in a process mutmut did not start.
+1 -1
View File
@@ -332,7 +332,7 @@ The maintainability senior and the inherited architect of PR #257's scoring run
"Most, if not all, code should have automated tests", in `principles/technical-principles.md`: `compare.blind()` keeps the comparison blind, and a version name it misses invalidates the bench silently.
The tests also cover `workflow-bench/bench.py`'s `permission_flags`, with a probe: a headless claude given its flags is denied a read of a file outside the granted trees, a `git -c alias.x=… x` and a `git diff --no-index`, and is allowed the review's own git commands.
The tests also cover `workflow-bench/bench.py`'s `permission_flags`, with a probe of a headless claude given its flags. It is denied a read outside the granted trees, a bare `git status`, and wrapper calls with `--output`, `--no-index`, a glued `-aF/abs`, `--fil=/abs` or a path outside the clone. A `cd <elsewhere> && python3 <wrapper> …` acts on the clone only. It is allowed the wrapper's `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit -m`. `workflow-bench/test_review_git.py` tests the wrapper's refusals directly.
### 4. Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.
+9 -6
View File
@@ -23,7 +23,7 @@ import sys
import time
BENCH = pathlib.Path(__file__).resolve().parent
REVIEW_GIT = BENCH / "review-git.py"
REVIEW_GIT = BENCH / "review_git.py"
SOURCES = pathlib.Path(os.environ["BENCH_SOURCES"])
CASES = {
"cjk": ("151bf38^", "6c49912"),
@@ -38,16 +38,19 @@ def permission_flags(clone, version_dir, report):
"""The tools a review needs, each limited to what it may touch; the run denies every other call."""
# A permission rule writes an absolute path as "//<path>"; a trailing "/**" matches everything under it.
trees = (f"/{clone}/**", f"/{version_dir}/**")
# git can reach outside the clone through an argument, an option or a hook, and a quote or redirect can
# smuggle one past a text pattern. So the run gets no bare git: review-git.py is its one shell command
# and confines git to the clone (see its docstring). The clone's own .git and .claude stay unwritable,
# lest an edit plant a hook or widen the run's tools; the user's settings would add ~/code to the trees.
# git can reach outside the clone through an argument, an option or a hook, so the run gets no bare git:
# review_git.py is its one shell command (see its docstring). The clone's own .git and .claude stay
# unwritable, lest an edit plant a hook or widen the run's tools.
allowed = [
"Agent", "Skill(security-review)", f"Edit(/{clone}/**)", f"Edit(/{report})",
# Claude Code, not the wrapper, refuses what a shell adds to this command: a redirect, `$(…)`, backticks,
# or a second command after `;`, `&` or `&&`. It lets through `cd <dir> && …`, which is why run() names
# the clone in REVIEW_GIT_CLONE, and a pipe into a reader such as `| head`.
f"Bash(python3 {REVIEW_GIT} *)",
*(f"{tool}({tree})" for tool in ("Read", "Grep", "Glob") for tree in trees),
]
denied = ["Bash(git *)", f"Edit(/{clone}/.git/**)", f"Edit(/{clone}/.claude/**)"]
# No "user" source: ~/.claude/settings.json would add ~/code to the trees and run its hooks.
return ["--setting-sources", "project,local", "--permission-mode", "dontAsk", "--permission-prompts", "none",
"--allowedTools", *allowed, "--disallowedTools", *denied]
@@ -101,7 +104,7 @@ def run(name):
subprocess.run(
["claude", "-p", prompt, *permission_flags(clone, version_dir, report),
"--model", "claude-opus-5-5", "--output-format", "stream-json", "--verbose"],
cwd=clone, stdout=out, stderr=subprocess.STDOUT)
cwd=clone, env={**os.environ, "REVIEW_GIT_CLONE": str(clone)}, stdout=out, stderr=subprocess.STDOUT)
if report.exists():
return f"{name}: done on attempt {attempt}"
time.sleep(max(1200, rate_limit_reset(log) + 120 - time.time()))
-73
View File
@@ -1,73 +0,0 @@
#!/usr/bin/env python3
"""Run one git command for a bench review, confined to the clone: review-git.py <subcommand> <args>...
The bench's review runs no bare git. It calls this wrapper, its one allowed command (bench.py's
permission_flags). The wrapper is git with everything that could reach outside the clone removed:
it accepts only the review's subcommands, each with no option that reads a file, writes a file or
runs a program; it refuses an argument that names a path outside the clone; and it runs git under
the clone with a scrubbed config, so no global or system config, hook, pager or external diff fires.
Quoting, redirects and `$` are the shell's, never git's, so they cannot smuggle a path past this.
The wrapper lives outside the clone, so the clone is not on Python's import path.
"""
import os
import subprocess
import sys
SUBCOMMANDS = {"log", "diff", "show", "status", "ls-files", "add", "commit"}
# Options that read a file, write a file or run a program; "--no-index" lets diff read any two paths.
DENY_EXACT = {"--no-index", "-t"}
DENY_PREFIX = ("--output", "-O", "--pathspec-from-file", "-F", "--file", "--template",
"--exclude-from", "-X", "--git-dir", "--work-tree", "--exec-path", "--ext-diff", "--textconv")
def reject(reason):
sys.exit(f"review-git.py: refused: {reason}")
def bad_option(token):
name = token.split("=", 1)[0]
return name in DENY_EXACT or any(name.startswith(prefix) for prefix in DENY_PREFIX)
def bad_path(clone, arg):
# A revision such as HEAD~1 or a..b is not a path; a path that escapes the clone is refused.
if arg.startswith("/") or arg.startswith("~"):
return True
if arg == ".." or any(part == ".." for part in arg.split("/")):
return True
target = os.path.realpath(os.path.join(clone, arg))
return os.path.lexists(target) and target != clone and not target.startswith(clone + os.sep)
def main(argv):
if len(argv) < 1 or argv[0] not in SUBCOMMANDS:
reject(f"subcommand must be one of {' '.join(sorted(SUBCOMMANDS))}")
sub, args = argv[0], argv[1:]
clone = os.path.realpath(os.getcwd())
operands_only = False # Everything after "--" is a path operand.
skip_message = False # commit's -m value is free text, never a path.
for token in args:
if skip_message:
skip_message = False
continue
if not operands_only and token == "--":
operands_only = True
elif not operands_only and token.startswith("-"):
if bad_option(token):
reject(f"option not allowed: {token}")
short_m = not token.startswith("--") and token.endswith("m") # -m, -am
if sub == "commit" and "=" not in token and (token == "--message" or short_m):
skip_message = True # the next token is the commit message, free text, never a path.
elif bad_path(clone, token):
reject(f"argument names a path outside the clone: {token}")
env = {**os.environ, "GIT_CONFIG_GLOBAL": "/dev/null", "GIT_CONFIG_SYSTEM": "/dev/null",
"GIT_CONFIG_NOSYSTEM": "1", "GIT_PAGER": "cat", "GIT_TERMINAL_PROMPT": "0"}
for name in ("GIT_EXTERNAL_DIFF", "GIT_SSH", "GIT_SSH_COMMAND", "GIT_PROXY_COMMAND", "GIT_CONFIG"):
env.pop(name, None)
git = ["git", "-c", "core.hooksPath=/dev/null", "-c", "core.fsmonitor=false",
"-c", "core.pager=cat", "-c", "protocol.ext.allow=never", sub, *args]
return subprocess.run(git, cwd=clone, env=env).returncode
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env python3
"""Run one git command for a bench review, confined to the clone: review_git.py <subcommand> <args>...
The bench's review runs no bare git: this wrapper is its one allowed shell command (bench.py's
permission_flags). bench.py names the clone in REVIEW_GIT_CLONE, and git runs there whatever the
working directory, with GIT_DIR and GIT_WORK_TREE pinned to it.
The wrapper refuses every subcommand and option ALLOWED does not list, so a cluster of short options
or a long option cut to a prefix is refused too. An option that takes a value takes it glued, as
`--format=%h`. A commit needs `-m`, whose value is the one free-text argument. Every other argument
must name no path outside the clone. git runs with no global or system config, hook, pager or editor.
The wrapper checks the arguments git receives, after the shell has removed quotes and expanded `$`,
so no quoting hides a path from it. It lives outside the clone, so a module planted in the clone
cannot replace `os` or `subprocess`.
"""
from collections.abc import Callable
import os
import re
import subprocess
import sys
OUTPUT = {"--name-only", "--name-status", "--no-color", "--stat", "--unified="}
# An entry ending in "=" allows that option with any glued value.
ALLOWED = {
"add": {"-A", "--all", "-u", "--update"},
"commit": {"-a", "--all"},
"diff": OUTPUT | {"--cached", "--check", "--staged", "--word-diff"},
"log": OUTPUT | {"--author=", "--decorate", "--follow", "--format=", "--graph", "--grep=", "--max-count=",
"--oneline", "-p", "--patch", "--pretty=", "--reverse", "--since=", "--until="},
"ls-files": {"--cached", "--deleted", "--exclude-standard", "--modified", "--others"},
"show": OUTPUT | {"--format=", "--no-patch", "--oneline", "--pretty=", "-s"},
"status": {"-b", "--branch", "--porcelain", "-s", "--short"},
}
# The commit identity, since no config supplies one.
BENCH_IDENTITY = {"GIT_AUTHOR_NAME": "bench", "GIT_AUTHOR_EMAIL": "bench@localhost",
"GIT_COMMITTER_NAME": "bench", "GIT_COMMITTER_EMAIL": "bench@localhost"}
GIT = ["git", "-c", "core.hooksPath=/dev/null", "-c", "core.fsmonitor=false", "-c", "core.pager=cat"]
def allowed_option(sub: str, token: str) -> bool:
options = ALLOWED[sub]
glued = any(token.startswith(option) for option in options if option.endswith("="))
return token in options or glued or (sub == "log" and re.fullmatch(r"-[0-9]+", token) is not None)
def outside_clone(token: str, escapes: Callable[[str], bool]) -> bool:
# A revision range such as a..b is checked side by side, so `..` cannot climb out of the clone:
# ../x splits into "" and "/x". A leading ":" is pathspec magic, which can name the repository's top.
sides = re.split(r"\.\.\.?", token)
return token.startswith(":") or escapes(token) or any(side.startswith(("/", "~")) or escapes(side) for side in sides)
def refusal(argv: list[str], escapes: Callable[[str], bool]) -> str | None:
"""Why the wrapper refuses `git <argv>`, or None; escapes(path) is whether path names an existing file outside the clone."""
if not argv or argv[0] not in ALLOWED:
return f"subcommand must be one of {' '.join(sorted(ALLOWED))}"
sub = argv[0]
has_message = operands_only = message_next = False
for token in argv[1:]:
if message_next:
message_next = False
elif operands_only or not token.startswith("-"):
if outside_clone(token, escapes):
return f"argument names a path outside the clone: {token}"
elif token == "--":
operands_only = True
elif sub == "commit" and token in ("-m", "-am", "--message"):
has_message = message_next = True
elif sub == "commit" and token.startswith(("-m", "--message=")):
has_message = True
elif not allowed_option(sub, token):
return f"option not allowed: {token}; {sub} takes {' '.join(sorted(ALLOWED[sub]))}"
if sub == "commit" and not has_message:
return "commit needs -m <message>"
return None
def git_env(environ: dict[str, str], clone: str) -> dict[str, str]:
"""The inherited environment without any variable that names a program or a repository, with git pinned to the clone."""
kept = {name: value for name, value in environ.items()
if not name.startswith("GIT_") and name not in ("EDITOR", "PAGER", "SSH_ASKPASS", "VISUAL")}
return {**kept, **BENCH_IDENTITY, "GIT_CONFIG_GLOBAL": "/dev/null", "GIT_CONFIG_NOSYSTEM": "1", "GIT_DIR": os.path.join(clone, ".git"),
"GIT_EDITOR": "true", "GIT_PAGER": "cat", "GIT_TERMINAL_PROMPT": "0", "GIT_WORK_TREE": clone}
def escapes(clone: str, path: str) -> bool:
target = os.path.realpath(os.path.join(clone, path))
return os.path.lexists(target) and os.path.commonpath([clone, target]) != clone
def refuse(reason: str) -> int:
print(f"review_git.py: refused: {reason}", file=sys.stderr)
return 2
def main(argv: list[str], environ: dict[str, str]) -> int:
if not environ.get("REVIEW_GIT_CLONE"):
return refuse("REVIEW_GIT_CLONE must name the clone")
clone = os.path.realpath(environ["REVIEW_GIT_CLONE"])
reason = refusal(argv, lambda path: escapes(clone, path))
if reason is not None:
return refuse(reason)
return subprocess.run([*GIT, *argv], cwd=clone, env=git_env(environ, clone)).returncode
if __name__ == "__main__":
sys.exit(main(sys.argv[1:], dict(os.environ)))
+2 -2
View File
@@ -15,10 +15,10 @@ Report: write it to `{report}`.
- Launch every agent in the foreground (`run_in_background: false`): background agents die when this headless turn ends.
- No remote and no forge: skip the PR, CI, the external auto-reviewer and the merge; review locally.
- Run every git command through `python3 {review_git} <subcommand> <args>`, never bare `git`: that wrapper is the only shell command allowed, and it confines git to this clone. Run no other code or test suite. Where the process asks for a run, such as the stability pass's mutation test, review by reading and list the skipped run under Ambiguities.
- Run every git command through `python3 {review_git} <subcommand> <args>`, never bare `git`: that wrapper is the only shell command allowed. It runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit`. Where the process asks for another git command, do without it and list it under Ambiguities. Run no other code or test suite. Where the process asks for a run, such as the stability pass's mutation test, review by reading and list the skipped run under Ambiguities.
- Skip the comprehension check; say so in the report.
- No human is reachable. Where the process names a no-human branch, take it; where it names none, take your recommendation and list the question in the report.
- Commit each round's fixes on `chunk` with `python3 {review_git} commit`, messages prefixed `{case} - review: `. Touch nothing outside this repository except the report.
- Commit each round's fixes on `chunk` with `python3 {review_git} commit -m <message>`, messages prefixed `{case} - review: `. Touch nothing outside this repository except the report.
## Report
+135
View File
@@ -0,0 +1,135 @@
import os
import pathlib
import subprocess
import sys
import tempfile
import unittest
from review_git import git_env, refusal
WRAPPER = pathlib.Path(__file__).resolve().parent / "review_git.py"
# Paths the fake filesystem resolves to an existing file outside the clone.
OUTSIDE = {"..", "link-out"}
def escapes(path: str) -> bool:
return path in OUTSIDE
class Refusal(unittest.TestCase):
def test_refuses_an_option_outside_the_subcommands_allowlist(self):
for argv in (
["commit", "-aF/abs/secret"], ["commit", "-t/abs/template", "-m", "x"], ["diff", "-RO/abs/out"],
["commit", "--fil=/abs/secret"], ["commit", "--templ=/abs/template", "-m", "x"],
["add", "--pathspec-from=/abs/list"], ["add", "--pathspec-f=list"], ["ls-files", "--exclude-f=/abs/x"],
["diff", "--ext"], ["ls-files", "-oX/abs/x"], ["diff", "--output=.git/config"], ["diff", "--no-index", "a", "b"],
["commit", "-S", "-m", "x"], ["commit", "--gpg-sign", "-m", "x"], ["commit", "-e", "-m", "x"],
["commit", "--edit", "-m", "x"], ["commit", "-c", "HEAD"], ["add", "-e"], ["add", "-p"], ["add", "-i"],
["add", "--patch"], ["log", "--help"],
):
with self.subTest(argv):
self.assertIn(argv[1], refusal(argv, escapes) or "")
def test_reads_an_option_after_a_message_glued_to_m(self):
for argv in (["commit", "-mm", "--pathspec-from-file=/secret"], ["commit", "-mfixm", "--pathspec-from-file=/secret"]):
with self.subTest(argv):
self.assertIn("--pathspec-from-file=/secret", refusal(argv, escapes) or "")
def test_refuses_a_commit_without_a_message(self):
for argv in (["commit"], ["commit", "-a"]):
with self.subTest(argv):
self.assertIsNotNone(refusal(argv, escapes))
def test_refuses_a_subcommand_outside_the_review(self):
for argv in ([], ["config", "core.pager", "sh"], ["-c", "alias.x=!sh", "x"], ["push"]):
with self.subTest(argv):
self.assertIsNotNone(refusal(argv, escapes))
def test_refuses_an_argument_naming_a_path_outside_the_clone(self):
for argv in (
["add", "/etc/passwd"], ["diff", "~/.ssh/id_rsa"], ["diff", "--", "../other/f"], ["add", ".."],
["show", "link-out"], ["diff", "HEAD..link-out"], ["diff", "a/../../b"], ["diff", "HEAD..~/x"],
["log", ":(top)"], ["log", ":/"], ["add", "--", "/etc/passwd"], ["commit", "-m", "x", "/etc/passwd"],
):
with self.subTest(argv):
self.assertIn(argv[-1], refusal(argv, escapes) or "")
def test_allows_the_reviews_own_commands(self):
for argv in (
["log", "--oneline", "abc123..chunk"], ["log", "-5"], ["log", "--max-count=3", "--format=%h %s"],
["log", "--stat", "HEAD~1..HEAD"], ["log", "--pretty=%H", "--", "src/a.py"], ["diff", "base...chunk", "--stat"],
["diff", "--cached"], ["diff", "--name-only", "HEAD~1"], ["diff", "--", "src"], ["show", "HEAD"],
["show", "--stat", "abc123"], ["show", "HEAD:README.md"], ["status"], ["status", "--short"], ["ls-files"],
["ls-files", "--others", "--exclude-standard"], ["add", "-A"], ["add", "--", "src/a.py"],
["commit", "-m", "case - review: fix"], ["commit", "-am", "fix"], ["commit", "-a", "-m", "fix"],
["commit", "-m", "/a message, never a path"], ["commit", "-mfix"], ["commit", "--message=fix"],
["commit", "--message", "fix"],
):
with self.subTest(argv):
self.assertIsNone(refusal(argv, escapes))
class GitEnv(unittest.TestCase):
def test_pins_git_to_the_clone_and_drops_every_program_the_environment_names(self):
inherited = {"PATH": "/bin", "GIT_DIR": "/other/.git", "GIT_EDITOR": "sh", "GIT_SEQUENCE_EDITOR": "sh",
"EDITOR": "sh", "VISUAL": "sh", "PAGER": "sh", "GIT_ASKPASS": "sh", "SSH_ASKPASS": "sh", "GIT_EXTERNAL_DIFF": "sh"}
env = git_env(inherited, "/clone")
self.assertEqual({name: env.get(name) for name in (*inherited, "GIT_WORK_TREE")}, {
"PATH": "/bin", "GIT_DIR": "/clone/.git", "GIT_EDITOR": "true", "GIT_SEQUENCE_EDITOR": None, "EDITOR": None,
"VISUAL": None, "PAGER": None, "GIT_ASKPASS": None, "SSH_ASKPASS": None, "GIT_EXTERNAL_DIFF": None,
"GIT_WORK_TREE": "/clone"})
def test_names_the_committer_whatever_the_inherited_environment_holds(self):
env = git_env({"GIT_AUTHOR_NAME": "someone", "EMAIL": "x@y"}, "/clone")
self.assertEqual([env[name] for name in ("GIT_AUTHOR_NAME", "GIT_COMMITTER_NAME")], ["bench", "bench"])
class Wrapper(unittest.TestCase):
def setUp(self):
scratch = tempfile.TemporaryDirectory()
self.addCleanup(scratch.cleanup)
self.root = pathlib.Path(scratch.name)
self.clone = self.root / "clone"
self.other = self.root / "other"
self.env = {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "REVIEW_GIT_"))}
self.env.update(EMAIL="bench@example.com", GIT_CONFIG_GLOBAL="/dev/null", GIT_CONFIG_NOSYSTEM="1")
for repo in (self.clone, self.other):
subprocess.run(["git", "init", "-q", "-b", "chunk", str(repo)], check=True, env=self.env)
(repo / "a.txt").write_text("one\n")
subprocess.run(["git", "-C", str(repo), "add", "a.txt"], check=True, env=self.env)
subprocess.run(["git", "-C", str(repo), "commit", "-qm", "start"], check=True, env=self.env)
def wrapper(self, *args, cwd, clone_env=True):
env = {**self.env, "REVIEW_GIT_CLONE": str(self.clone)} if clone_env else self.env
return subprocess.run([sys.executable, str(WRAPPER), *args], cwd=cwd, env=env, capture_output=True, text=True)
def subjects(self, repo):
return subprocess.run(["git", "-C", str(repo), "log", "--format=%s"], check=True, env=self.env,
capture_output=True, text=True).stdout.split("\n")[:-1]
def test_an_edit_is_added_and_committed_on_the_clone_whatever_the_working_directory(self):
(self.clone / "a.txt").write_text("two\n")
(self.other / "a.txt").write_text("two\n")
self.wrapper("add", "-A", cwd=self.other)
self.wrapper("commit", "-m", "case - review: fix", cwd=self.other)
self.assertEqual((self.subjects(self.clone), self.subjects(self.other)), (["case - review: fix", "start"], ["start"]))
def test_a_refused_output_option_leaves_git_config_unchanged(self):
config = (self.clone / ".git" / "config").read_bytes()
ran = self.wrapper("diff", "--output=.git/config", cwd=self.clone)
self.assertEqual((ran.returncode != 0, (self.clone / ".git" / "config").read_bytes()), (True, config))
def test_refuses_to_run_without_the_clone_named(self):
(self.clone / "a.txt").write_text("two\n")
ran = self.wrapper("diff", cwd=self.clone, clone_env=False)
self.assertEqual((ran.returncode != 0, ran.stdout), (True, ""))
def test_refuses_a_link_in_the_clone_to_a_file_outside_it(self):
(self.root / "secret").write_text("token\n")
(self.clone / "link-out").symlink_to(self.root / "secret")
ran = self.wrapper("diff", "--no-color", "link-out", cwd=self.clone)
self.assertEqual((ran.returncode != 0, "token" in ran.stdout), (True, False))
if __name__ == "__main__":
unittest.main()