Answer stability round 4: 2 🔴, 1 🟡; the wrapper allows only listed options, takes the clone from the environment, and skips only a message its flag takes whole
This commit is contained in:
+2
-1
@@ -88,5 +88,6 @@
|
||||
!workflow-bench/bench.py
|
||||
!workflow-bench/compare-prompt.md
|
||||
!workflow-bench/compare.py
|
||||
!workflow-bench/review-git.py
|
||||
!workflow-bench/review_git.py
|
||||
!workflow-bench/run-prompt.md
|
||||
!workflow-bench/test_review_git.py
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- `workflow-bench/bench.py` gives each run a fixed list of allowed tools, where it skipped every permission check. A run may launch agents, run `/security-review`, edit its clone (not the clone's `.git/` or `.claude/`) and its report, and read its clone and the copy of `~/.claude` it benches. For git it gets one command, `workflow-bench/review-git.py`, and no bare `git`: the wrapper runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit` under the clone with a scrubbed config, refusing an argument that names a path outside the clone and any option that reads a file, writes a file or runs a program. So no git argument, option, hook, redirect or shell substitution reaches beyond the clone. It loads no user settings, so `~/.claude/settings.json`'s hooks and extra directories do not apply to it. Every other call is denied. Expect a benched review that needs to run a project's code or tests to say in its report that it could not.
|
||||
- `workflow-bench/bench.py` gives each run a fixed list of allowed tools. Before, it skipped every permission check. A run may launch agents, run `/security-review`, edit its clone (not the clone's `.git/` or `.claude/`) and its report, and read its clone and the copy of `~/.claude` it benches. For git it gets one command, `workflow-bench/review_git.py`, and no bare `git`. The wrapper runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit -m` on the run's clone, whatever the working directory. It allows only the options it lists for each, and refuses an argument that names a path outside the clone. git runs with no global or system config, hook, pager or editor. A run loads no user settings, so `~/.claude/settings.json`'s hooks and extra directories do not apply to it. Every other call is denied. Expect a benched review that needs another git command, or to run a project's code or tests, to say in its report that it could not.
|
||||
- Every goal in `README.md` → Goals is marked `**Maintain:**`, a state to hold, or `**Achieve:**`, a direction to pursue. Each achieve goal has a signal, a counter-signal and a one-line reason under it. A change that breaks a maintain goal is redesigned or dropped, whatever it scores: `rules/todo-items.md` says so, and larv-review's goal alignment step no longer merges such a PR. `rules/goals-and-audience.md` states the form for every project's README. Expect larv-review to stop a PR on a maintain goal where it asked you before.
|
||||
- Reshape every project's `todo.md` to the new shape below: until it is reshaped, that is the project's only work, as it is while README.md lacks Goals and Audience, and `rank_todo.py` says so. First move every machine to this release: an older release rejects the new formula, so it fails on every reshaped `todo.md`. Then mark the README's goals `**Maintain:**` or `**Achieve:**`, score each item's L, E and I, and set a bar on the new scale below the score of every item to keep, since the old bar would drop most items. Until then, an old-shape `todo.md` still ranks by its own formula and weights.
|
||||
- `todo.md` ranks items by value per cost: this setup's own plan uses `Score = 100 * V / (2**S + 8)`, where 8 is a chunk's fixed cost, about one size-3 change, and `rank_todo.py --help` shows it. A Moves column replaces A, G and Goals, naming each achieve goal an item moves and how far, such as `1.1 by 8, 1.2 by 5`. V sums each named goal's weight times its move. `rank_todo.py` weighs each achieve goal by its position in README.md, the first heaviest, so the goal weights table goes. An item names the most specific goal it moves, never a goal beside its own sub-goal. `rules/todo-items.md` and `rank_todo.py --help` show the new shape.
|
||||
|
||||
@@ -94,7 +94,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `statusline.py` renders Claude Code's status line; `settings.json` → `statusLine` runs it.
|
||||
- `subagent-rung [--plan] <model> <L>` prints the Agent tool's model and effort for a subagent's task, picked by its L from the project's model, and with `--plan` never below the project's rung; `subagent-rung --help` states the rule.
|
||||
|
||||
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`
|
||||
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`. Test the bench's git wrapper with `python3 -m unittest discover -s workflow-bench -p 'test_*.py'`.
|
||||
|
||||
To mutation-test a changed module, copy `bin/` to a scratch directory (`cp -r bin "$scratch"`): mutmut writes files beside its config. In the copy, name the module and its tests in a `setup.cfg`, as below for `score_outcomes/outcomes.py`. Leave out each test class that starts a tool's launcher through `subprocess`: a mutated module fails to load in a process mutmut did not start.
|
||||
|
||||
|
||||
@@ -332,7 +332,7 @@ The maintainability senior and the inherited architect of PR #257's scoring run
|
||||
|
||||
"Most, if not all, code should have automated tests", in `principles/technical-principles.md`: `compare.blind()` keeps the comparison blind, and a version name it misses invalidates the bench silently.
|
||||
|
||||
The tests also cover `workflow-bench/bench.py`'s `permission_flags`, with a probe: a headless claude given its flags is denied a read of a file outside the granted trees, a `git -c alias.x=… x` and a `git diff --no-index`, and is allowed the review's own git commands.
|
||||
The tests also cover `workflow-bench/bench.py`'s `permission_flags`, with a probe of a headless claude given its flags. It is denied a read outside the granted trees, a bare `git status`, and wrapper calls with `--output`, `--no-index`, a glued `-aF/abs`, `--fil=/abs` or a path outside the clone. A `cd <elsewhere> && python3 <wrapper> …` acts on the clone only. It is allowed the wrapper's `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit -m`. `workflow-bench/test_review_git.py` tests the wrapper's refusals directly.
|
||||
|
||||
### 4. Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ import sys
|
||||
import time
|
||||
|
||||
BENCH = pathlib.Path(__file__).resolve().parent
|
||||
REVIEW_GIT = BENCH / "review-git.py"
|
||||
REVIEW_GIT = BENCH / "review_git.py"
|
||||
SOURCES = pathlib.Path(os.environ["BENCH_SOURCES"])
|
||||
CASES = {
|
||||
"cjk": ("151bf38^", "6c49912"),
|
||||
@@ -38,16 +38,19 @@ def permission_flags(clone, version_dir, report):
|
||||
"""The tools a review needs, each limited to what it may touch; the run denies every other call."""
|
||||
# A permission rule writes an absolute path as "//<path>"; a trailing "/**" matches everything under it.
|
||||
trees = (f"/{clone}/**", f"/{version_dir}/**")
|
||||
# git can reach outside the clone through an argument, an option or a hook, and a quote or redirect can
|
||||
# smuggle one past a text pattern. So the run gets no bare git: review-git.py is its one shell command
|
||||
# and confines git to the clone (see its docstring). The clone's own .git and .claude stay unwritable,
|
||||
# lest an edit plant a hook or widen the run's tools; the user's settings would add ~/code to the trees.
|
||||
# git can reach outside the clone through an argument, an option or a hook, so the run gets no bare git:
|
||||
# review_git.py is its one shell command (see its docstring). The clone's own .git and .claude stay
|
||||
# unwritable, lest an edit plant a hook or widen the run's tools.
|
||||
allowed = [
|
||||
"Agent", "Skill(security-review)", f"Edit(/{clone}/**)", f"Edit(/{report})",
|
||||
# Claude Code, not the wrapper, refuses what a shell adds to this command: a redirect, `$(…)`, backticks,
|
||||
# or a second command after `;`, `&` or `&&`. It lets through `cd <dir> && …`, which is why run() names
|
||||
# the clone in REVIEW_GIT_CLONE, and a pipe into a reader such as `| head`.
|
||||
f"Bash(python3 {REVIEW_GIT} *)",
|
||||
*(f"{tool}({tree})" for tool in ("Read", "Grep", "Glob") for tree in trees),
|
||||
]
|
||||
denied = ["Bash(git *)", f"Edit(/{clone}/.git/**)", f"Edit(/{clone}/.claude/**)"]
|
||||
# No "user" source: ~/.claude/settings.json would add ~/code to the trees and run its hooks.
|
||||
return ["--setting-sources", "project,local", "--permission-mode", "dontAsk", "--permission-prompts", "none",
|
||||
"--allowedTools", *allowed, "--disallowedTools", *denied]
|
||||
|
||||
@@ -101,7 +104,7 @@ def run(name):
|
||||
subprocess.run(
|
||||
["claude", "-p", prompt, *permission_flags(clone, version_dir, report),
|
||||
"--model", "claude-opus-5-5", "--output-format", "stream-json", "--verbose"],
|
||||
cwd=clone, stdout=out, stderr=subprocess.STDOUT)
|
||||
cwd=clone, env={**os.environ, "REVIEW_GIT_CLONE": str(clone)}, stdout=out, stderr=subprocess.STDOUT)
|
||||
if report.exists():
|
||||
return f"{name}: done on attempt {attempt}"
|
||||
time.sleep(max(1200, rate_limit_reset(log) + 120 - time.time()))
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run one git command for a bench review, confined to the clone: review-git.py <subcommand> <args>...
|
||||
|
||||
The bench's review runs no bare git. It calls this wrapper, its one allowed command (bench.py's
|
||||
permission_flags). The wrapper is git with everything that could reach outside the clone removed:
|
||||
it accepts only the review's subcommands, each with no option that reads a file, writes a file or
|
||||
runs a program; it refuses an argument that names a path outside the clone; and it runs git under
|
||||
the clone with a scrubbed config, so no global or system config, hook, pager or external diff fires.
|
||||
Quoting, redirects and `$` are the shell's, never git's, so they cannot smuggle a path past this.
|
||||
The wrapper lives outside the clone, so the clone is not on Python's import path.
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
SUBCOMMANDS = {"log", "diff", "show", "status", "ls-files", "add", "commit"}
|
||||
# Options that read a file, write a file or run a program; "--no-index" lets diff read any two paths.
|
||||
DENY_EXACT = {"--no-index", "-t"}
|
||||
DENY_PREFIX = ("--output", "-O", "--pathspec-from-file", "-F", "--file", "--template",
|
||||
"--exclude-from", "-X", "--git-dir", "--work-tree", "--exec-path", "--ext-diff", "--textconv")
|
||||
|
||||
|
||||
def reject(reason):
|
||||
sys.exit(f"review-git.py: refused: {reason}")
|
||||
|
||||
|
||||
def bad_option(token):
|
||||
name = token.split("=", 1)[0]
|
||||
return name in DENY_EXACT or any(name.startswith(prefix) for prefix in DENY_PREFIX)
|
||||
|
||||
|
||||
def bad_path(clone, arg):
|
||||
# A revision such as HEAD~1 or a..b is not a path; a path that escapes the clone is refused.
|
||||
if arg.startswith("/") or arg.startswith("~"):
|
||||
return True
|
||||
if arg == ".." or any(part == ".." for part in arg.split("/")):
|
||||
return True
|
||||
target = os.path.realpath(os.path.join(clone, arg))
|
||||
return os.path.lexists(target) and target != clone and not target.startswith(clone + os.sep)
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 1 or argv[0] not in SUBCOMMANDS:
|
||||
reject(f"subcommand must be one of {' '.join(sorted(SUBCOMMANDS))}")
|
||||
sub, args = argv[0], argv[1:]
|
||||
clone = os.path.realpath(os.getcwd())
|
||||
operands_only = False # Everything after "--" is a path operand.
|
||||
skip_message = False # commit's -m value is free text, never a path.
|
||||
for token in args:
|
||||
if skip_message:
|
||||
skip_message = False
|
||||
continue
|
||||
if not operands_only and token == "--":
|
||||
operands_only = True
|
||||
elif not operands_only and token.startswith("-"):
|
||||
if bad_option(token):
|
||||
reject(f"option not allowed: {token}")
|
||||
short_m = not token.startswith("--") and token.endswith("m") # -m, -am
|
||||
if sub == "commit" and "=" not in token and (token == "--message" or short_m):
|
||||
skip_message = True # the next token is the commit message, free text, never a path.
|
||||
elif bad_path(clone, token):
|
||||
reject(f"argument names a path outside the clone: {token}")
|
||||
env = {**os.environ, "GIT_CONFIG_GLOBAL": "/dev/null", "GIT_CONFIG_SYSTEM": "/dev/null",
|
||||
"GIT_CONFIG_NOSYSTEM": "1", "GIT_PAGER": "cat", "GIT_TERMINAL_PROMPT": "0"}
|
||||
for name in ("GIT_EXTERNAL_DIFF", "GIT_SSH", "GIT_SSH_COMMAND", "GIT_PROXY_COMMAND", "GIT_CONFIG"):
|
||||
env.pop(name, None)
|
||||
git = ["git", "-c", "core.hooksPath=/dev/null", "-c", "core.fsmonitor=false",
|
||||
"-c", "core.pager=cat", "-c", "protocol.ext.allow=never", sub, *args]
|
||||
return subprocess.run(git, cwd=clone, env=env).returncode
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,107 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run one git command for a bench review, confined to the clone: review_git.py <subcommand> <args>...
|
||||
|
||||
The bench's review runs no bare git: this wrapper is its one allowed shell command (bench.py's
|
||||
permission_flags). bench.py names the clone in REVIEW_GIT_CLONE, and git runs there whatever the
|
||||
working directory, with GIT_DIR and GIT_WORK_TREE pinned to it.
|
||||
|
||||
The wrapper refuses every subcommand and option ALLOWED does not list, so a cluster of short options
|
||||
or a long option cut to a prefix is refused too. An option that takes a value takes it glued, as
|
||||
`--format=%h`. A commit needs `-m`, whose value is the one free-text argument. Every other argument
|
||||
must name no path outside the clone. git runs with no global or system config, hook, pager or editor.
|
||||
The wrapper checks the arguments git receives, after the shell has removed quotes and expanded `$`,
|
||||
so no quoting hides a path from it. It lives outside the clone, so a module planted in the clone
|
||||
cannot replace `os` or `subprocess`.
|
||||
"""
|
||||
from collections.abc import Callable
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
OUTPUT = {"--name-only", "--name-status", "--no-color", "--stat", "--unified="}
|
||||
# An entry ending in "=" allows that option with any glued value.
|
||||
ALLOWED = {
|
||||
"add": {"-A", "--all", "-u", "--update"},
|
||||
"commit": {"-a", "--all"},
|
||||
"diff": OUTPUT | {"--cached", "--check", "--staged", "--word-diff"},
|
||||
"log": OUTPUT | {"--author=", "--decorate", "--follow", "--format=", "--graph", "--grep=", "--max-count=",
|
||||
"--oneline", "-p", "--patch", "--pretty=", "--reverse", "--since=", "--until="},
|
||||
"ls-files": {"--cached", "--deleted", "--exclude-standard", "--modified", "--others"},
|
||||
"show": OUTPUT | {"--format=", "--no-patch", "--oneline", "--pretty=", "-s"},
|
||||
"status": {"-b", "--branch", "--porcelain", "-s", "--short"},
|
||||
}
|
||||
# The commit identity, since no config supplies one.
|
||||
BENCH_IDENTITY = {"GIT_AUTHOR_NAME": "bench", "GIT_AUTHOR_EMAIL": "bench@localhost",
|
||||
"GIT_COMMITTER_NAME": "bench", "GIT_COMMITTER_EMAIL": "bench@localhost"}
|
||||
GIT = ["git", "-c", "core.hooksPath=/dev/null", "-c", "core.fsmonitor=false", "-c", "core.pager=cat"]
|
||||
|
||||
|
||||
def allowed_option(sub: str, token: str) -> bool:
|
||||
options = ALLOWED[sub]
|
||||
glued = any(token.startswith(option) for option in options if option.endswith("="))
|
||||
return token in options or glued or (sub == "log" and re.fullmatch(r"-[0-9]+", token) is not None)
|
||||
|
||||
|
||||
def outside_clone(token: str, escapes: Callable[[str], bool]) -> bool:
|
||||
# A revision range such as a..b is checked side by side, so `..` cannot climb out of the clone:
|
||||
# ../x splits into "" and "/x". A leading ":" is pathspec magic, which can name the repository's top.
|
||||
sides = re.split(r"\.\.\.?", token)
|
||||
return token.startswith(":") or escapes(token) or any(side.startswith(("/", "~")) or escapes(side) for side in sides)
|
||||
|
||||
|
||||
def refusal(argv: list[str], escapes: Callable[[str], bool]) -> str | None:
|
||||
"""Why the wrapper refuses `git <argv>`, or None; escapes(path) is whether path names an existing file outside the clone."""
|
||||
if not argv or argv[0] not in ALLOWED:
|
||||
return f"subcommand must be one of {' '.join(sorted(ALLOWED))}"
|
||||
sub = argv[0]
|
||||
has_message = operands_only = message_next = False
|
||||
for token in argv[1:]:
|
||||
if message_next:
|
||||
message_next = False
|
||||
elif operands_only or not token.startswith("-"):
|
||||
if outside_clone(token, escapes):
|
||||
return f"argument names a path outside the clone: {token}"
|
||||
elif token == "--":
|
||||
operands_only = True
|
||||
elif sub == "commit" and token in ("-m", "-am", "--message"):
|
||||
has_message = message_next = True
|
||||
elif sub == "commit" and token.startswith(("-m", "--message=")):
|
||||
has_message = True
|
||||
elif not allowed_option(sub, token):
|
||||
return f"option not allowed: {token}; {sub} takes {' '.join(sorted(ALLOWED[sub]))}"
|
||||
if sub == "commit" and not has_message:
|
||||
return "commit needs -m <message>"
|
||||
return None
|
||||
|
||||
|
||||
def git_env(environ: dict[str, str], clone: str) -> dict[str, str]:
|
||||
"""The inherited environment without any variable that names a program or a repository, with git pinned to the clone."""
|
||||
kept = {name: value for name, value in environ.items()
|
||||
if not name.startswith("GIT_") and name not in ("EDITOR", "PAGER", "SSH_ASKPASS", "VISUAL")}
|
||||
return {**kept, **BENCH_IDENTITY, "GIT_CONFIG_GLOBAL": "/dev/null", "GIT_CONFIG_NOSYSTEM": "1", "GIT_DIR": os.path.join(clone, ".git"),
|
||||
"GIT_EDITOR": "true", "GIT_PAGER": "cat", "GIT_TERMINAL_PROMPT": "0", "GIT_WORK_TREE": clone}
|
||||
|
||||
|
||||
def escapes(clone: str, path: str) -> bool:
|
||||
target = os.path.realpath(os.path.join(clone, path))
|
||||
return os.path.lexists(target) and os.path.commonpath([clone, target]) != clone
|
||||
|
||||
|
||||
def refuse(reason: str) -> int:
|
||||
print(f"review_git.py: refused: {reason}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
def main(argv: list[str], environ: dict[str, str]) -> int:
|
||||
if not environ.get("REVIEW_GIT_CLONE"):
|
||||
return refuse("REVIEW_GIT_CLONE must name the clone")
|
||||
clone = os.path.realpath(environ["REVIEW_GIT_CLONE"])
|
||||
reason = refusal(argv, lambda path: escapes(clone, path))
|
||||
if reason is not None:
|
||||
return refuse(reason)
|
||||
return subprocess.run([*GIT, *argv], cwd=clone, env=git_env(environ, clone)).returncode
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:], dict(os.environ)))
|
||||
@@ -15,10 +15,10 @@ Report: write it to `{report}`.
|
||||
|
||||
- Launch every agent in the foreground (`run_in_background: false`): background agents die when this headless turn ends.
|
||||
- No remote and no forge: skip the PR, CI, the external auto-reviewer and the merge; review locally.
|
||||
- Run every git command through `python3 {review_git} <subcommand> <args>`, never bare `git`: that wrapper is the only shell command allowed, and it confines git to this clone. Run no other code or test suite. Where the process asks for a run, such as the stability pass's mutation test, review by reading and list the skipped run under Ambiguities.
|
||||
- Run every git command through `python3 {review_git} <subcommand> <args>`, never bare `git`: that wrapper is the only shell command allowed. It runs `log`, `diff`, `show`, `status`, `ls-files`, `add` and `commit`. Where the process asks for another git command, do without it and list it under Ambiguities. Run no other code or test suite. Where the process asks for a run, such as the stability pass's mutation test, review by reading and list the skipped run under Ambiguities.
|
||||
- Skip the comprehension check; say so in the report.
|
||||
- No human is reachable. Where the process names a no-human branch, take it; where it names none, take your recommendation and list the question in the report.
|
||||
- Commit each round's fixes on `chunk` with `python3 {review_git} commit`, messages prefixed `{case} - review: `. Touch nothing outside this repository except the report.
|
||||
- Commit each round's fixes on `chunk` with `python3 {review_git} commit -m <message>`, messages prefixed `{case} - review: `. Touch nothing outside this repository except the report.
|
||||
|
||||
## Report
|
||||
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
import os
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from review_git import git_env, refusal
|
||||
|
||||
WRAPPER = pathlib.Path(__file__).resolve().parent / "review_git.py"
|
||||
# Paths the fake filesystem resolves to an existing file outside the clone.
|
||||
OUTSIDE = {"..", "link-out"}
|
||||
|
||||
|
||||
def escapes(path: str) -> bool:
|
||||
return path in OUTSIDE
|
||||
|
||||
|
||||
class Refusal(unittest.TestCase):
|
||||
def test_refuses_an_option_outside_the_subcommands_allowlist(self):
|
||||
for argv in (
|
||||
["commit", "-aF/abs/secret"], ["commit", "-t/abs/template", "-m", "x"], ["diff", "-RO/abs/out"],
|
||||
["commit", "--fil=/abs/secret"], ["commit", "--templ=/abs/template", "-m", "x"],
|
||||
["add", "--pathspec-from=/abs/list"], ["add", "--pathspec-f=list"], ["ls-files", "--exclude-f=/abs/x"],
|
||||
["diff", "--ext"], ["ls-files", "-oX/abs/x"], ["diff", "--output=.git/config"], ["diff", "--no-index", "a", "b"],
|
||||
["commit", "-S", "-m", "x"], ["commit", "--gpg-sign", "-m", "x"], ["commit", "-e", "-m", "x"],
|
||||
["commit", "--edit", "-m", "x"], ["commit", "-c", "HEAD"], ["add", "-e"], ["add", "-p"], ["add", "-i"],
|
||||
["add", "--patch"], ["log", "--help"],
|
||||
):
|
||||
with self.subTest(argv):
|
||||
self.assertIn(argv[1], refusal(argv, escapes) or "")
|
||||
|
||||
def test_reads_an_option_after_a_message_glued_to_m(self):
|
||||
for argv in (["commit", "-mm", "--pathspec-from-file=/secret"], ["commit", "-mfixm", "--pathspec-from-file=/secret"]):
|
||||
with self.subTest(argv):
|
||||
self.assertIn("--pathspec-from-file=/secret", refusal(argv, escapes) or "")
|
||||
|
||||
def test_refuses_a_commit_without_a_message(self):
|
||||
for argv in (["commit"], ["commit", "-a"]):
|
||||
with self.subTest(argv):
|
||||
self.assertIsNotNone(refusal(argv, escapes))
|
||||
|
||||
def test_refuses_a_subcommand_outside_the_review(self):
|
||||
for argv in ([], ["config", "core.pager", "sh"], ["-c", "alias.x=!sh", "x"], ["push"]):
|
||||
with self.subTest(argv):
|
||||
self.assertIsNotNone(refusal(argv, escapes))
|
||||
|
||||
def test_refuses_an_argument_naming_a_path_outside_the_clone(self):
|
||||
for argv in (
|
||||
["add", "/etc/passwd"], ["diff", "~/.ssh/id_rsa"], ["diff", "--", "../other/f"], ["add", ".."],
|
||||
["show", "link-out"], ["diff", "HEAD..link-out"], ["diff", "a/../../b"], ["diff", "HEAD..~/x"],
|
||||
["log", ":(top)"], ["log", ":/"], ["add", "--", "/etc/passwd"], ["commit", "-m", "x", "/etc/passwd"],
|
||||
):
|
||||
with self.subTest(argv):
|
||||
self.assertIn(argv[-1], refusal(argv, escapes) or "")
|
||||
|
||||
def test_allows_the_reviews_own_commands(self):
|
||||
for argv in (
|
||||
["log", "--oneline", "abc123..chunk"], ["log", "-5"], ["log", "--max-count=3", "--format=%h %s"],
|
||||
["log", "--stat", "HEAD~1..HEAD"], ["log", "--pretty=%H", "--", "src/a.py"], ["diff", "base...chunk", "--stat"],
|
||||
["diff", "--cached"], ["diff", "--name-only", "HEAD~1"], ["diff", "--", "src"], ["show", "HEAD"],
|
||||
["show", "--stat", "abc123"], ["show", "HEAD:README.md"], ["status"], ["status", "--short"], ["ls-files"],
|
||||
["ls-files", "--others", "--exclude-standard"], ["add", "-A"], ["add", "--", "src/a.py"],
|
||||
["commit", "-m", "case - review: fix"], ["commit", "-am", "fix"], ["commit", "-a", "-m", "fix"],
|
||||
["commit", "-m", "/a message, never a path"], ["commit", "-mfix"], ["commit", "--message=fix"],
|
||||
["commit", "--message", "fix"],
|
||||
):
|
||||
with self.subTest(argv):
|
||||
self.assertIsNone(refusal(argv, escapes))
|
||||
|
||||
|
||||
class GitEnv(unittest.TestCase):
|
||||
def test_pins_git_to_the_clone_and_drops_every_program_the_environment_names(self):
|
||||
inherited = {"PATH": "/bin", "GIT_DIR": "/other/.git", "GIT_EDITOR": "sh", "GIT_SEQUENCE_EDITOR": "sh",
|
||||
"EDITOR": "sh", "VISUAL": "sh", "PAGER": "sh", "GIT_ASKPASS": "sh", "SSH_ASKPASS": "sh", "GIT_EXTERNAL_DIFF": "sh"}
|
||||
env = git_env(inherited, "/clone")
|
||||
self.assertEqual({name: env.get(name) for name in (*inherited, "GIT_WORK_TREE")}, {
|
||||
"PATH": "/bin", "GIT_DIR": "/clone/.git", "GIT_EDITOR": "true", "GIT_SEQUENCE_EDITOR": None, "EDITOR": None,
|
||||
"VISUAL": None, "PAGER": None, "GIT_ASKPASS": None, "SSH_ASKPASS": None, "GIT_EXTERNAL_DIFF": None,
|
||||
"GIT_WORK_TREE": "/clone"})
|
||||
|
||||
def test_names_the_committer_whatever_the_inherited_environment_holds(self):
|
||||
env = git_env({"GIT_AUTHOR_NAME": "someone", "EMAIL": "x@y"}, "/clone")
|
||||
self.assertEqual([env[name] for name in ("GIT_AUTHOR_NAME", "GIT_COMMITTER_NAME")], ["bench", "bench"])
|
||||
|
||||
|
||||
class Wrapper(unittest.TestCase):
|
||||
def setUp(self):
|
||||
scratch = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(scratch.cleanup)
|
||||
self.root = pathlib.Path(scratch.name)
|
||||
self.clone = self.root / "clone"
|
||||
self.other = self.root / "other"
|
||||
self.env = {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "REVIEW_GIT_"))}
|
||||
self.env.update(EMAIL="bench@example.com", GIT_CONFIG_GLOBAL="/dev/null", GIT_CONFIG_NOSYSTEM="1")
|
||||
for repo in (self.clone, self.other):
|
||||
subprocess.run(["git", "init", "-q", "-b", "chunk", str(repo)], check=True, env=self.env)
|
||||
(repo / "a.txt").write_text("one\n")
|
||||
subprocess.run(["git", "-C", str(repo), "add", "a.txt"], check=True, env=self.env)
|
||||
subprocess.run(["git", "-C", str(repo), "commit", "-qm", "start"], check=True, env=self.env)
|
||||
|
||||
def wrapper(self, *args, cwd, clone_env=True):
|
||||
env = {**self.env, "REVIEW_GIT_CLONE": str(self.clone)} if clone_env else self.env
|
||||
return subprocess.run([sys.executable, str(WRAPPER), *args], cwd=cwd, env=env, capture_output=True, text=True)
|
||||
|
||||
def subjects(self, repo):
|
||||
return subprocess.run(["git", "-C", str(repo), "log", "--format=%s"], check=True, env=self.env,
|
||||
capture_output=True, text=True).stdout.split("\n")[:-1]
|
||||
|
||||
def test_an_edit_is_added_and_committed_on_the_clone_whatever_the_working_directory(self):
|
||||
(self.clone / "a.txt").write_text("two\n")
|
||||
(self.other / "a.txt").write_text("two\n")
|
||||
self.wrapper("add", "-A", cwd=self.other)
|
||||
self.wrapper("commit", "-m", "case - review: fix", cwd=self.other)
|
||||
self.assertEqual((self.subjects(self.clone), self.subjects(self.other)), (["case - review: fix", "start"], ["start"]))
|
||||
|
||||
def test_a_refused_output_option_leaves_git_config_unchanged(self):
|
||||
config = (self.clone / ".git" / "config").read_bytes()
|
||||
ran = self.wrapper("diff", "--output=.git/config", cwd=self.clone)
|
||||
self.assertEqual((ran.returncode != 0, (self.clone / ".git" / "config").read_bytes()), (True, config))
|
||||
|
||||
def test_refuses_to_run_without_the_clone_named(self):
|
||||
(self.clone / "a.txt").write_text("two\n")
|
||||
ran = self.wrapper("diff", cwd=self.clone, clone_env=False)
|
||||
self.assertEqual((ran.returncode != 0, ran.stdout), (True, ""))
|
||||
|
||||
def test_refuses_a_link_in_the_clone_to_a_file_outside_it(self):
|
||||
(self.root / "secret").write_text("token\n")
|
||||
(self.clone / "link-out").symlink_to(self.root / "secret")
|
||||
ran = self.wrapper("diff", "--no-color", "link-out", cwd=self.clone)
|
||||
self.assertEqual((ran.returncode != 0, "token" in ran.stdout), (True, False))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user