diff --git a/interop-tests/AGENTS.md b/interop-tests/AGENTS.md new file mode 100644 index 0000000..ad6f279 --- /dev/null +++ b/interop-tests/AGENTS.md @@ -0,0 +1,75 @@ +# interop-tests + +How the experiments in [PLAN.md](PLAN.md) are run and recorded. Read PLAN.md first, and the root +[AGENTS.md](../AGENTS.md) for the conventions all code here follows. + +## Layout + +| | | +| --- | --- | +| `compose..yaml` | Overlay on the root `compose.yaml`: the peer's services, a `capture` sidecar in the peer's network namespace, and `node` given `depends_on` the peer | +| `.test.ts` | `node:test` file driving our library against the peer, in the style of `test/`. Reads `PEER_HOST`/`PEER_PORT` from env, defaulting to the compose service name and 2775 | +| `peers//` | Dockerfile and config for a peer without a published image, or one that needs config files | +| `captures/` | pcapng and tshark JSON from a run. Gitignored | +| `findings/-.md` | What a phase found. Committed after every phase | +| `run.py` | Brings a peer up, runs its tests, stops the capture, tears down, analyses the capture | + +## Running + +```bash +./interop-tests/run.py # one peer, all its tests +./interop-tests/run.py --keep # leave the peer up for a manual look +``` + +`run.py` is the one spelling; it wraps + +```bash +docker compose -f compose.yaml -f interop-tests/compose..yaml run --rm --use-aliases node node --test interop-tests/.test.ts +docker compose -f compose.yaml -f interop-tests/compose..yaml stop capture +docker compose -f compose.yaml -f interop-tests/compose..yaml down -v +``` + +and then decodes `captures/.pcapng` with tshark (`-d tcp.port==,smpp -Y smpp -T json`), +printing the command histogram and the counts of `_ws.malformed` and error-severity `_ws.expert`. +Both counts must be zero for a phase to pass. + +## Rules for an experiment + +1. Peers run in Docker with full patch-version pins. Nothing is installed on the host. Node runs + only through the `node` service. +2. `src/` and `test/` are read-only during an experiment. A defect is recorded with a reproducer, + never fixed here — a fix is a separate change with a regression test in `test/`. +3. No git command that changes state: no add, commit, push, checkout, stash, reset. The + orchestrator commits after each phase. +4. A peer that will not come up is time-boxed: after about an hour of trying, record `blocked` with + everything tried, and stop. +5. `down -v` at the end of every run. Locally built images are kept, and their tag goes in the + findings. +6. Scratch files live outside the repo, in the directory the orchestrator names. +7. A finding says what happened, what the spec or the peer's docs say, and how to reproduce it. + Wording is neutral: a mismatch is a mismatch until a reader decides whose it is. + +## Findings file + +```markdown +# + +Date, images and tags, the commit of this repo, host Docker version. + +## Setup +Commands that worked, and what did not, so the next run starts where this one ended. + +## Scenarios +| Id (from PLAN.md) | Result (pass / fail / blocked / not run) | Evidence (test name, log line, tshark frame) | + +## Defects in @larvit/smpp +One subsection each: what happened, what the spec or the peer's docs say, reproducer (PDU hex or +test), severity. + +## Peer quirks +Behaviour of the peer worth knowing that is not our defect. + +## Open questions +``` + +Then set the phase's row in PLAN.md's Status table to `done` or `blocked`, linking the file.