From 80f9ed8e8b3a59c490199d7f9b6e45f16a9555a5 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Sun, 20 Sep 2026 21:54:12 +0000 Subject: [PATCH 001/966] Add renovate.json --- renovate.json | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 renovate.json diff --git a/renovate.json b/renovate.json new file mode 100644 index 0000000..5db72dd --- /dev/null +++ b/renovate.json @@ -0,0 +1,6 @@ +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "extends": [ + "config:recommended" + ] +} -- 2.52.0 From fab6ab4f2ed281b6dea3a1cf4de2716d314f82ee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 20 Sep 2026 23:55:23 +0200 Subject: [PATCH 002/966] Move the project to GitHub, where a stranger can file an issue and open a PR --- .github/workflows/test.yml | 8 ++++---- AGENTS.md | 13 ++++--------- README.md | 11 +++++++++-- cmd/fejkdata/main.go | 2 +- go.mod | 2 +- release-tooling/publish_release.py | 8 ++++---- todo.md | 6 +++--- 7 files changed, 26 insertions(+), 24 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 7569186..2ece4c7 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -51,7 +51,7 @@ jobs: run: docker build --build-arg GO_VERSION=1.22.12 . release: - name: Gitea release from CHANGELOG.md + name: Release from CHANGELOG.md needs: test if: github.event_name == 'push' runs-on: ubuntu-24.04 @@ -63,7 +63,7 @@ jobs: persist-credentials: false - run: python3 release-tooling/publish_release.py env: - GITEA_API_URL: ${{ github.api_url }} - GITEA_REPOSITORY: ${{ github.repository }} - GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }} + FORGE_API_URL: ${{ github.api_url }} + FORGE_REPOSITORY: ${{ github.repository }} + FORGE_TOKEN: ${{ secrets.GITHUB_TOKEN }} SHA: ${{ github.sha }} diff --git a/AGENTS.md b/AGENTS.md index e2b0b93..6cbd2a3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,16 +4,11 @@ - Tests first, in their own commit; the implementation follows in the next. A re-pin of seeded output or of `testdata/shipped_shape.txt` is its own commit. - A change under `data/` or to the shape pin adds its `CHANGELOG.md` entry under `Unreleased` in the same PR, and so does a change to a flag, an exit code, an exported name, a fence, a builtin or the lowest Go; what is major is the README's Versioning table. - One-line commit messages: no ticket prefix, no repo name, no authorship trailers. -- Merges are fast-forward only, which `tea pr merge` cannot spell — every style it - offers fails with "is it still open?", as does a merge whose required checks are - still pending, so read the checks before believing the style is the problem. Use - `tea api /repos/{owner}/{repo}/pulls//merge -f Do=fast-forward-only`. -- Gitea queues Actions runs that `/repos/{owner}/{repo}/actions/tasks` does not - list, so an empty task list says nothing about whether CI ran; read - `/repos/{owner}/{repo}/commits//status`. +- GitHub is where the project lives, so `gh` is the tool: merges are fast-forward + only, `gh pr merge --rebase` where the branch is already on top of main. - README goal 2 is gated at 7.0 on the `comprehension-panel` skill, and below it - nothing else merges — no feature, no category, no data — bar a security fix and a - dependency bump. `todo.md` carries the round: the nine-seat findings run at depth 1, + nothing else merges — no feature, no category, no data — bar a security fix, a + dependency bump and the infrastructure the round itself runs on. `todo.md` carries the round: the nine-seat findings run at depth 1, a PR per item it names, then the run again, until the score passes. At or above 7.0 every pull request scores with the four-seat run and answers it in one run. No merge lowers the last score. diff --git a/README.md b/README.md index 32464e6..b9b8f25 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ it as a Go library or the CLI — no data on disk, no dependencies, and a seed m reproducible. ```sh -go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest +go install github.com/larvit/fejkdata/cmd/fejkdata@latest fejkdata sv_SE.person # Sara Eriksson ``` @@ -262,7 +262,7 @@ Swedish region but `abbr` on a US one. ## Library ```sh -go get gitea.larvit.se/larvit/fejkdata # Go 1.22+ +go get github.com/larvit/fejkdata # Go 1.22+ ``` ```go @@ -854,6 +854,13 @@ the Development section below, and who ships a register the four above then draw by name. ## Decisions +- **GitHub is canonical, and the module path names it.** Goal 1 wants the usage the + tool earns and goal 2 expects extenders who did not write it; both need a stranger + to file an issue and open a pull request. Gitea has no anonymous issue, and no + cross-host pull request at all, so a contributor would need an account and a fork on + a personal instance. Valid while the project wants contribution from outside: the + workflow decided nothing, being portable already — `github.api_url` and + `secrets.GITHUB_TOKEN` resolve on either host, so only the names moved. - **Options and fields share one namespace.** `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a key, or prefixing options, would tax every template to guard against a diff --git a/cmd/fejkdata/main.go b/cmd/fejkdata/main.go index 327e5c3..4d39720 100644 --- a/cmd/fejkdata/main.go +++ b/cmd/fejkdata/main.go @@ -18,7 +18,7 @@ import ( "strconv" "strings" - "gitea.larvit.se/larvit/fejkdata" + "github.com/larvit/fejkdata" ) const usage = `Usage: fejkdata [flags] diff --git a/go.mod b/go.mod index d45362a..6b4bcce 100644 --- a/go.mod +++ b/go.mod @@ -1,3 +1,3 @@ -module gitea.larvit.se/larvit/fejkdata +module github.com/larvit/fejkdata go 1.22 diff --git a/release-tooling/publish_release.py b/release-tooling/publish_release.py index 313b84e..98cbf98 100644 --- a/release-tooling/publish_release.py +++ b/release-tooling/publish_release.py @@ -1,8 +1,8 @@ #!/usr/bin/env python3 -"""Publish the Gitea release that CHANGELOG.md's top heading names, tagging SHA. +"""Publish the release that CHANGELOG.md's top heading names, tagging SHA. A top heading of `[Unreleased]`, or a version already released, publishes nothing. -Env: GITEA_API_URL, GITEA_REPOSITORY (owner/repo), GITEA_TOKEN, SHA. +Env: FORGE_API_URL, FORGE_REPOSITORY (owner/repo), FORGE_TOKEN, SHA. """ import json @@ -17,9 +17,9 @@ HEADING = re.compile(r"^## \[([^\]]+)\].*?$\n?(.*?)(?=^## \[|\Z)", re.M | re.S) def request(path: str, data: dict | None = None): req = urllib.request.Request( - f"{os.environ['GITEA_API_URL']}/repos/{os.environ['GITEA_REPOSITORY']}/{path}", + f"{os.environ['FORGE_API_URL']}/repos/{os.environ['FORGE_REPOSITORY']}/{path}", data=json.dumps(data).encode() if data else None, - headers={"Authorization": f"token {os.environ['GITEA_TOKEN']}", "Content-Type": "application/json"}, + headers={"Authorization": f"token {os.environ['FORGE_TOKEN']}", "Content-Type": "application/json"}, ) with urllib.request.urlopen(req) as resp: return json.load(resp) diff --git a/todo.md b/todo.md index 76e2cce..3a813b5 100644 --- a/todo.md +++ b/todo.md @@ -173,9 +173,6 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide where the module is found from, before goal 1's announcement: the import - path is `gitea.larvit.se/larvit/fejkdata`, a developer looking for a fake-data - library searches GitHub and pkg.go.dev, and a module path is a major to change. - Decide whether `data/misc`'s 33 flat files gain a level before v1.0.0: a folder is a path segment, so `misc/net/tld` is a rename a consumer pays for, and goal 13 promises the directory keeps growing. @@ -241,6 +238,9 @@ because pairing a street with its exact postnummer rewrites shipped rows. ## Not release-bound +- Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub or retire it: + Gitea converts a repository to a mirror only by re-creating it, so until that runs + the copy there is a second owner of one history and goes stale from this commit. - Stop `/cache` collecting root-owned files, which today every command documented without `--user` leaves for the next `--user` one to trip over, and which the README answers with a chown a reader has to repeat. Put `--user` on all of them, or give the -- 2.52.0 From 12b4a095d38cd3967fd661fdaf2a612357cd9eee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 00:04:04 +0200 Subject: [PATCH 003/966] Let the test workflow be run by hand --- .github/workflows/test.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 2ece4c7..43bc4ec 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -4,6 +4,7 @@ on: pull_request: push: branches: [main] + workflow_dispatch: permissions: contents: read -- 2.52.0 From fad78a8ac07eaac057770a70d842b40553a954b6 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Sun, 20 Sep 2026 22:30:35 +0000 Subject: [PATCH 004/966] Update actions/checkout action to v7 --- .github/workflows/test.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 43bc4ec..485e03c 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -14,7 +14,7 @@ jobs: name: vet + fmt + tests runs-on: ubuntu-24.04 steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@v7 with: persist-credentials: false fetch-depth: 0 @@ -59,7 +59,7 @@ jobs: permissions: contents: write steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@v7 with: persist-credentials: false - run: python3 release-tooling/publish_release.py -- 2.52.0 From ed25368b935b3f9114ce0b43a73d42fe6033e6ef Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Mon, 21 Sep 2026 05:42:09 +0000 Subject: [PATCH 005/966] Update golang Docker tag to v1.27.1 --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 8f04a4a..2a9909c 100644 --- a/Dockerfile +++ b/Dockerfile @@ -2,7 +2,7 @@ # runs exactly this. Day-to-day, prefer `docker compose run` (bind-mounts source, # no rebuilds). GO_VERSION defaults to the latest stable Go; override it to test # the lowest supported version: docker build --build-arg GO_VERSION=1.22.12 . -ARG GO_VERSION=1.26.4 +ARG GO_VERSION=1.27.1 FROM golang:${GO_VERSION} WORKDIR /app -- 2.52.0 From 1b5ed2740605382fccce63b483fd3d3c944e9c53 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 07:49:14 +0200 Subject: [PATCH 006/966] Check formatting on the latest Go only, the toolchain that defines it --- .github/workflows/test.yml | 2 +- Dockerfile | 13 ++++++++----- README.md | 16 ++++++++++++---- 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 485e03c..888dc7c 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -49,7 +49,7 @@ jobs: # failure is version-specific. - name: Lowest supported Go if: ${{ !cancelled() && steps.changes.outputs.code == 'true' }} - run: docker build --build-arg GO_VERSION=1.22.12 . + run: docker build --build-arg GO_VERSION=1.22.12 --target portable . release: name: Release from CHANGELOG.md diff --git a/Dockerfile b/Dockerfile index 2a9909c..f6bbac1 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,9 +1,10 @@ # The merge gate: `docker build .` fails if vet, formatting or tests fail, and CI # runs exactly this. Day-to-day, prefer `docker compose run` (bind-mounts source, -# no rebuilds). GO_VERSION defaults to the latest stable Go; override it to test -# the lowest supported version: docker build --build-arg GO_VERSION=1.22.12 . +# no rebuilds). GO_VERSION defaults to the latest stable Go; the lowest supported +# version builds the `portable` stage, which every supported Go must pass: +# docker build --build-arg GO_VERSION=1.22.12 --target portable . ARG GO_VERSION=1.27.1 -FROM golang:${GO_VERSION} +FROM golang:${GO_VERSION} AS portable WORKDIR /app @@ -14,6 +15,8 @@ RUN go mod download COPY . . RUN go vet ./... && \ go run github.com/fzipp/gocyclo/cmd/gocyclo@v0.6.0 -over 14 -ignore _test . && \ - { unformatted="$(gofmt -l .)"; test -z "$unformatted" || \ - { echo "unformatted files:"; echo "$unformatted"; exit 1; }; } && \ go test -race ./... + +FROM portable +RUN unformatted="$(gofmt -l .)"; test -z "$unformatted" || \ + { echo "unformatted files:"; echo "$unformatted"; exit 1; } diff --git a/README.md b/README.md index b9b8f25..0eeec0f 100644 --- a/README.md +++ b/README.md @@ -940,6 +940,13 @@ the Development section below, and who ships a register the four above then draw - **Raising the lowest supported Go is a major.** A consumer building on it breaks, which is the one test every rule above applies; Go's convention of a minor is not followed. +- **The format check runs on the latest Go only.** `gofmt`'s output is the + toolchain's, not the code's: 1.27 stopped padding a map literal's values out to a + lone long key, so no source satisfies both it and 1.22's. A consumer on the lowest + supported Go depends on the code compiling and its tests passing there, which is + the `portable` stage, and never runs `gofmt` over this source, so the latest + toolchain alone defines the one canonical form goal 2 asks a reader to meet. Valid + while the lowest supported Go is not the latest. - **The changelog heading is the one spelling of a release; CI cuts the tag.** A tag pushed by hand is served by `go get` at once, so a tag whose commit lacks its heading is burnt, not fixed. The heading on a gate-passed `main` commit is the @@ -1367,12 +1374,13 @@ supported Go, and must pass before it can be merged — unless it changes none o the files the build and its tests read, nor the workflow itself, in which case it's skipped (see [Decisions](#decisions)). That build is the whole gate but the changelog check, which CI runs against the PR base — vet, complexity, format check -and tests — so run it locally before pushing: +and tests — so run it locally before pushing. The lowest Go builds the `portable` +stage, all of that bar the format check: ```sh -docker build . # latest -docker build --build-arg GO_VERSION=1.22.12 . # lowest supported -GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build +docker build . # latest +docker build --build-arg GO_VERSION=1.22.12 --target portable . # lowest supported +GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build ``` A change to the shipped data re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) -- 2.52.0 From b3869dd472229f3fc4fa6c1f427257b95bff769a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 07:49:15 +0200 Subject: [PATCH 007/966] Reformat the test files with Go 1.27's gofmt --- calc_test.go | 4 ++-- table_test.go | 40 ++++++++++++++++++++-------------------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/calc_test.go b/calc_test.go index 7a8b7bd..6c750de 100644 --- a/calc_test.go +++ b/calc_test.go @@ -250,8 +250,8 @@ func TestCalcOverANeverNumericOperandIsRejected(t *testing.T) { func TestCalcConstantZeroDivisorIsRejected(t *testing.T) { for src, want := range map[string]string{ - `"{calc(1/0)}"`: "divides by 0", - `"{calc(2/(1-1))}"`: "divides by (1 - 1)", + `"{calc(1/0)}"`: "divides by 0", + `"{calc(2/(1-1))}"`: "divides by (1 - 1)", `{"format":"{calc(x/y)}","x":"1","y":"0"}`: "divides by y", `{"format":"{calc(x/(y*2))}","x":"1","y":" 0 "}`: "divides by (y * 2)", `{"format":"{calc((a/0)+b)}","a":"1","b":"2"}`: "divides by 0", diff --git a/table_test.go b/table_test.go index 66529db..eded26e 100644 --- a/table_test.go +++ b/table_test.go @@ -460,26 +460,26 @@ func TestTableFences(t *testing.T) { "a table reaching its family through a template": {with(geo(), map[string]string{"addr.json": `"{/locality.name}"`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, "a table reaching its family through a repeat": {with(geo(), map[string]string{"addr.json": `{"format":"{/locality.name} ","repeat":3}`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, "a table reaching its family through a group": {with(geo(), map[string]string{"addr.json": `{"format":"{/locality.name}","drawGroup":"g"}`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, - "one row": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\nx\n"}, "one row"}, - "empty file": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": ""}, "header"}, - "parent is not a table": {map[string]string{"p.json": `"x"`, "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\tx\n"}, "not a table"}, - "parent does not exist": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\tx\n"}, `"p"`}, - "parent in another folder": {map[string]string{"g/p.json": `{"format":"{k}","rows":"p.tsv","key":"k"}`, "g/p.tsv": "k\nx\ny\n", "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\ty\n"}, `"p"`}, - "parent has no key": {map[string]string{"p.json": `{"format":"{k}","rows":"p.tsv"}`, "p.tsv": "k\nx\ny\n", "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\ty\n"}, "key"}, - "dangling link": {with(base, map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","parent":"region"}`, "t.tsv": "a\tregion\nx\t01\ny\t99\n"}), `"99"`}, - "childless parent row": {with(base, map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","parent":"region"}`, "t.tsv": "a\tregion\nx\t01\ny\t01\n"}), `"12"`}, - "parent cycle": {map[string]string{"a.json": `{"format":"{k}","rows":"a.tsv","key":"k","parent":"b"}`, "a.tsv": "k\tb\nx\tx\ny\ty\n", "b.json": `{"format":"{k}","rows":"b.tsv","key":"k","parent":"a"}`, "b.tsv": "k\ta\nx\tx\ny\ty\n"}, "cycle"}, - "child named like a column": {with(base, map[string]string{"name.json": `{"format":"{a}","rows":"name.tsv","parent":"region"}`, "name.tsv": "a\tregion\nx\t01\ny\t12\n"}), `"name"`}, - "rows nested in a field": {map[string]string{"t.json": `{"format":"{x}","x":{"format":"{a}","rows":"x.tsv"}}`, "x.tsv": "a\nx\ny\n"}, "category"}, - "rows in a choice item": {map[string]string{"t.json": `[{"format":"{a}","rows":"t.tsv"},"y"]`, "t.tsv": "a\nx\ny\n"}, "category"}, - "unknown table option": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","fields":"a"}`, "t.tsv": "a\nx\ny\n"}, "a table takes"}, - "repeat on a table": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","repeat":2}`, "t.tsv": "a\nx\ny\n"}, "a table takes"}, - "drawGroup on a table": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","drawGroup":"g"}`, "t.tsv": "a\nx\ny\n"}, "a table takes"}, - "option not a string": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","key":1}`, "t.tsv": "a\nx\ny\n"}, "string"}, - "format names no column": {map[string]string{"t.json": `{"format":"{b}","rows":"t.tsv"}`, "t.tsv": "a\nx\ny\n"}, `no column "b"`}, - "format reads into a column": {map[string]string{"t.json": `{"format":"{a.x}","rows":"t.tsv"}`, "t.tsv": "a\nx\ny\n"}, `"a"`}, - "a reference into a column": {with(base, map[string]string{"t.json": `"{/region.name.x}"`}), "column"}, - "a category referencing itself": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\n{/t.a}\ny\n"}, "names the category it sits in"}, + "one row": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\nx\n"}, "one row"}, + "empty file": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": ""}, "header"}, + "parent is not a table": {map[string]string{"p.json": `"x"`, "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\tx\n"}, "not a table"}, + "parent does not exist": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\tx\n"}, `"p"`}, + "parent in another folder": {map[string]string{"g/p.json": `{"format":"{k}","rows":"p.tsv","key":"k"}`, "g/p.tsv": "k\nx\ny\n", "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\ty\n"}, `"p"`}, + "parent has no key": {map[string]string{"p.json": `{"format":"{k}","rows":"p.tsv"}`, "p.tsv": "k\nx\ny\n", "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\ty\n"}, "key"}, + "dangling link": {with(base, map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","parent":"region"}`, "t.tsv": "a\tregion\nx\t01\ny\t99\n"}), `"99"`}, + "childless parent row": {with(base, map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","parent":"region"}`, "t.tsv": "a\tregion\nx\t01\ny\t01\n"}), `"12"`}, + "parent cycle": {map[string]string{"a.json": `{"format":"{k}","rows":"a.tsv","key":"k","parent":"b"}`, "a.tsv": "k\tb\nx\tx\ny\ty\n", "b.json": `{"format":"{k}","rows":"b.tsv","key":"k","parent":"a"}`, "b.tsv": "k\ta\nx\tx\ny\ty\n"}, "cycle"}, + "child named like a column": {with(base, map[string]string{"name.json": `{"format":"{a}","rows":"name.tsv","parent":"region"}`, "name.tsv": "a\tregion\nx\t01\ny\t12\n"}), `"name"`}, + "rows nested in a field": {map[string]string{"t.json": `{"format":"{x}","x":{"format":"{a}","rows":"x.tsv"}}`, "x.tsv": "a\nx\ny\n"}, "category"}, + "rows in a choice item": {map[string]string{"t.json": `[{"format":"{a}","rows":"t.tsv"},"y"]`, "t.tsv": "a\nx\ny\n"}, "category"}, + "unknown table option": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","fields":"a"}`, "t.tsv": "a\nx\ny\n"}, "a table takes"}, + "repeat on a table": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","repeat":2}`, "t.tsv": "a\nx\ny\n"}, "a table takes"}, + "drawGroup on a table": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","drawGroup":"g"}`, "t.tsv": "a\nx\ny\n"}, "a table takes"}, + "option not a string": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","key":1}`, "t.tsv": "a\nx\ny\n"}, "string"}, + "format names no column": {map[string]string{"t.json": `{"format":"{b}","rows":"t.tsv"}`, "t.tsv": "a\nx\ny\n"}, `no column "b"`}, + "format reads into a column": {map[string]string{"t.json": `{"format":"{a.x}","rows":"t.tsv"}`, "t.tsv": "a\nx\ny\n"}, `"a"`}, + "a reference into a column": {with(base, map[string]string{"t.json": `"{/region.name.x}"`}), "column"}, + "a category referencing itself": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\n{/t.a}\ny\n"}, "names the category it sits in"}, } for name, c := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, c.files))) -- 2.52.0 From 24a63f36e72bba3f3df13d7ad6311aeb77778de4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 07:49:17 +0200 Subject: [PATCH 008/966] Move compose's Go with the Dockerfile's, and let Renovate see it --- compose.yaml | 2 +- renovate.json | 13 +++++++++++++ todo.md | 5 +++++ 3 files changed, 19 insertions(+), 1 deletion(-) diff --git a/compose.yaml b/compose.yaml index 1c199aa..b331c8b 100644 --- a/compose.yaml +++ b/compose.yaml @@ -15,7 +15,7 @@ # supported Go with: GO_VERSION=1.22 docker compose run --rm test x-go: &go - image: golang:${GO_VERSION:-1.26.4} + image: golang:${GO_VERSION:-1.27.1} working_dir: /app volumes: - .:/app diff --git a/renovate.json b/renovate.json index 5db72dd..c613c8c 100644 --- a/renovate.json +++ b/renovate.json @@ -1,5 +1,18 @@ { "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "customManagers": [ + { + "customType": "regex", + "datasourceTemplate": "docker", + "description": "compose.yaml's Go, which the shell default hides from the docker-compose manager", + "managerFilePatterns": [ + "/^compose\\.yaml$/" + ], + "matchStrings": [ + "image: (?golang):\\$\\{GO_VERSION:-(?[^}]+)\\}" + ] + } + ], "extends": [ "config:recommended" ] diff --git a/todo.md b/todo.md index 3a813b5..b323736 100644 --- a/todo.md +++ b/todo.md @@ -247,5 +247,10 @@ because pairing a street with its exact postnummer rewrites shipped rows. `x-go` anchor a `user:` — that one needs a mechanism, since Compose interpolates from the process environment and neither `UID` nor `GID` is exported there, so `user: "${UID}:${GID}"` resolves to `":"`. +- Spell the latest supported Go once: `Dockerfile`'s `ARG GO_VERSION` and + `compose.yaml`'s `golang:${GO_VERSION:-…}` pin one version twice, and a bump that + moved only the first left `docker compose run --rm fmt` formatting to a `gofmt` the + gate rejects. A `renovate.json` custom manager reads the second today; drop it with + the duplication. - Publish a homepage with an in-browser generator: the library compiled to WebAssembly, so visitors generate as much data as they like in their own browser. -- 2.52.0 From d799bbcb0ba1625bcca4aa47273371d9aa072b6b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 07:50:34 +0200 Subject: [PATCH 009/966] Pin the lowest Go in compose's example to its patch version --- compose.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compose.yaml b/compose.yaml index b331c8b..7bdfd06 100644 --- a/compose.yaml +++ b/compose.yaml @@ -12,7 +12,7 @@ # ownership when run with `--user "$(id -u):$(id -g)"`. # # GO_VERSION picks the toolchain image (default: latest stable). Test the lowest -# supported Go with: GO_VERSION=1.22 docker compose run --rm test +# supported Go with: GO_VERSION=1.22.12 docker compose run --rm test x-go: &go image: golang:${GO_VERSION:-1.27.1} -- 2.52.0 From 0c322f50a5895c7523866ed655a5154524a717e5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 10:11:40 +0200 Subject: [PATCH 010/966] Prove the package doc's Vocabulary names symbols the package declares --- doc_test.go | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 doc_test.go diff --git a/doc_test.go b/doc_test.go new file mode 100644 index 0000000..30066e1 --- /dev/null +++ b/doc_test.go @@ -0,0 +1,114 @@ +package fejkdata + +import ( + "go/ast" + "go/parser" + "go/token" + "os" + "regexp" + "strings" + "testing" +) + +var backticked = regexp.MustCompile("`([A-Za-z][A-Za-z0-9]*(?:\\.[A-Za-z][A-Za-z0-9]*)?)`") + +// TestVocabularyNamesDeclaredSymbols proves every symbol the package doc's Vocabulary +// names is declared here, so a renamed unit renames its entry with it. +func TestVocabularyNamesDeclaredSymbols(t *testing.T) { + files := sourceFiles(t) + declared := declaredSymbols(files) + for _, m := range backticked.FindAllStringSubmatch(vocabulary(t, files), -1) { + if !declared[m[1]] { + t.Errorf("the Vocabulary names %s, which the package declares nowhere", m[1]) + } + } +} + +// sourceFiles parses the package's own files, the tests left out: the Vocabulary +// indexes what ships, not what proves it. +func sourceFiles(t *testing.T) []*ast.File { + t.Helper() + entries, err := os.ReadDir(".") + if err != nil { + t.Fatalf("read the package directory: %v", err) + } + fset := token.NewFileSet() + var files []*ast.File + for _, e := range entries { + name := e.Name() + if !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") { + continue + } + f, err := parser.ParseFile(fset, name, nil, parser.ParseComments) + if err != nil { + t.Fatalf("parse %s: %v", name, err) + } + files = append(files, f) + } + return files +} + +// vocabulary is the package doc from its Vocabulary heading on. +func vocabulary(t *testing.T, files []*ast.File) string { + t.Helper() + for _, f := range files { + if f.Doc == nil { + continue + } + if _, section, found := strings.Cut(f.Doc.Text(), "\n# Vocabulary\n"); found { + return section + } + } + t.Fatal("no package doc carries a Vocabulary heading, so fence, hold, pin and the rest are defined nowhere a reader of the code meets them") + return "" +} + +// declaredSymbols is every name the package declares, a method and a struct field +// under its type as well: fence, draws.pin, tableRead.whole. +func declaredSymbols(files []*ast.File) map[string]bool { + names := map[string]bool{} + for _, f := range files { + ast.Inspect(f, func(n ast.Node) bool { + switch d := n.(type) { + case *ast.FuncDecl: + names[d.Name.Name] = true + if d.Recv != nil { + names[receiverType(d.Recv)+"."+d.Name.Name] = true + } + case *ast.TypeSpec: + names[d.Name.Name] = true + addFields(names, d) + case *ast.ValueSpec: + for _, id := range d.Names { + names[id.Name] = true + } + } + return true + }) + } + return names +} + +func addFields(names map[string]bool, d *ast.TypeSpec) { + s, isStruct := d.Type.(*ast.StructType) + if !isStruct { + return + } + for _, field := range s.Fields.List { + for _, id := range field.Names { + names[d.Name.Name+"."+id.Name] = true + } + } +} + +func receiverType(recv *ast.FieldList) string { + e := recv.List[0].Type + if star, isPointer := e.(*ast.StarExpr); isPointer { + e = star.X + } + id, isIdent := e.(*ast.Ident) + if !isIdent { + return "" + } + return id.Name +} -- 2.52.0 From f56958007794b1a062b70ae1323b83e97d89a7cd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 10:13:34 +0200 Subject: [PATCH 011/966] Define draw, hold, pin, family, whole and fence in the package doc --- README.md | 1 + doc.go | 41 +++++++++++++++++++++++++++++++++++++++++ fejkdata.go | 12 ------------ todo.md | 3 --- 4 files changed, 42 insertions(+), 15 deletions(-) create mode 100644 doc.go diff --git a/README.md b/README.md index 0eeec0f..1f01cfe 100644 --- a/README.md +++ b/README.md @@ -1309,6 +1309,7 @@ the Development section below, and who ships a register the four above then draw ## Layout ``` +doc.go the package doc, and the vocabulary the rest of it uses for a render and its fences fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation table.go tables: the rows TSV, its options and links, row selection and draws diff --git a/doc.go b/doc.go new file mode 100644 index 0000000..16a8142 --- /dev/null +++ b/doc.go @@ -0,0 +1,41 @@ +// Package fejkdata generates fake data from recursive JSON templates. +// +// Data lives in JSON, not in Go. A generator starts from the shipped data set and +// layers any directories you add; folders and files become a dot-path namespace, +// then generate values by path: +// +// f, _ := fejkdata.New(fejkdata.WithSeed(42)) +// f.Fake("sv_SE.address") // "Järvedsvägen 43\n891 77 Järved" +// f.Fake("sv_SE.address.locality") // "Sundbyberg": a second call draws afresh +// +// Several sources merge in order, the last winning a name clash, so custom data +// layers over the built-ins. The JSON template format is documented in the README. +// +// # Vocabulary +// +// The words the rest of the package uses, and the unit that owns each: +// +// - draw — one pick from a choice, or one row taken from a table, and the value +// the pick produced: `pick` and `drawn` settle a choice, `table.draw` a row, and +// `draw` carries what a read produced. +// - expansion — one render of one format, by `expand`. A nested template is an +// expansion of its own, and so is each iteration of a `template.repeat`. +// - render — one `Fake` call or one record, started by `renderOnce` where nothing +// outer started it. It owns one `drawSet`, so it is what a reference draw spans. +// - hold — keeping one draw of a name, so every route to it reads that value. +// `template.held` lists what an expansion holds, `draws` keeps them and +// `readField` reads through it; a reference path is held for the whole render. +// - draw group — `template.drawGroup`, which gives the reference paths under it +// draws of their own inside the render. `drawSet` holds the unnamed group's and +// each named one's, and `drawScope` says which one a template renders in. +// - pin — fixing which row of a table the render uses. `draws.pin` pins it and the +// rows of its ancestors, and `draws.rowOf` draws one where none is pinned. +// - family — a table and every table reaching it through a chain of +// `table.parentT`, named by the root that chain ends at: `table.family`. +// - whole — a read that lands on a `row`, so the table's format renders every +// column of it: `tableRead.whole`. +// - fence — a load-time check refusing data that two routes to one draw would +// disagree in. Every one runs in `New`, which is why `Fake` cannot fail on +// loaded data: `heldCheck` over an expansion's held names, `drawCheck` over a +// render's reference paths, and `checkFamilies` over the rows they pin. +package fejkdata diff --git a/fejkdata.go b/fejkdata.go index 03a8e9c..9fbf0f0 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -1,15 +1,3 @@ -// Package fejkdata generates fake data from recursive JSON templates. -// -// Data lives in JSON, not in Go. A generator starts from the shipped data set and -// layers any directories you add; folders and files become a dot-path namespace, -// then generate values by path: -// -// f, _ := fejkdata.New(fejkdata.WithSeed(42)) -// f.Fake("sv_SE.address") // "Järvedsvägen 43\n891 77 Järved" -// f.Fake("sv_SE.address.locality") // "Sundbyberg": a second call draws afresh -// -// Several sources merge in order, the last winning a name clash, so custom data -// layers over the built-ins. The JSON template format is documented in the README. package fejkdata import ( diff --git a/todo.md b/todo.md index b323736..6ea95de 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Define the fence vocabulary where a reader of the code meets it: fence, hold, pin, - draw, draw group, expansion against render, family, whole. It exists only across five - README sections today, and every reader paid for it before the three files read at all. - Say on the `template` struct which pass fills which field, and that `compileFormat` runs a second time once `linkTemplateRefs` binds the references: nine of its fields are written by four other files, and the order lives only in `loadData`'s body. -- 2.52.0 From aed74498e6f6e3f0c5e66637a9cb700c6e43ca13 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 10:22:42 +0200 Subject: [PATCH 012/966] Name every render start, and let no backticked spelling go unchecked --- README.md | 2 +- doc.go | 19 ++++++++------ doc_test.go | 71 +++++++++++++++++++++++++++++------------------------ 3 files changed, 51 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index 1f01cfe..08132e0 100644 --- a/README.md +++ b/README.md @@ -1309,7 +1309,7 @@ the Development section below, and who ships a register the four above then draw ## Layout ``` -doc.go the package doc, and the vocabulary the rest of it uses for a render and its fences +doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation table.go tables: the rows TSV, its options and links, row selection and draws diff --git a/doc.go b/doc.go index 16a8142..667b40c 100644 --- a/doc.go +++ b/doc.go @@ -19,9 +19,11 @@ // the pick produced: `pick` and `drawn` settle a choice, `table.draw` a row, and // `draw` carries what a read produced. // - expansion — one render of one format, by `expand`. A nested template is an -// expansion of its own, and so is each iteration of a `template.repeat`. -// - render — one `Fake` call or one record, started by `renderOnce` where nothing -// outer started it. It owns one `drawSet`, so it is what a reference draw spans. +// expansion of its own. +// - render — what owns one `drawSet`, and so what one reference draw spans: +// `Generator.Fake`, `FakeRecord`, `FakeStruct`, `Template.Fake` and +// `RecordTemplate.Fake` each start one, as does `expandAnew` per iteration of a +// `template.repeat` — an iteration is an expansion and a render of its own. // - hold — keeping one draw of a name, so every route to it reads that value. // `template.held` lists what an expansion holds, `draws` keeps them and // `readField` reads through it; a reference path is held for the whole render. @@ -32,10 +34,11 @@ // rows of its ancestors, and `draws.rowOf` draws one where none is pinned. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. -// - whole — a read that lands on a `row`, so the table's format renders every -// column of it: `tableRead.whole`. +// - whole — a read that lands on a `row`, so the row renders through its table's +// format: `tableRead.whole` says a read did, `table.whole` is the node. // - fence — a load-time check refusing data that two routes to one draw would -// disagree in. Every one runs in `New`, which is why `Fake` cannot fail on -// loaded data: `heldCheck` over an expansion's held names, `drawCheck` over a -// render's reference paths, and `checkFamilies` over the rows they pin. +// disagree in, which is why a render cannot fail: `heldCheck` over an +// expansion's held names, `drawCheck` over a render's reference paths, and +// `checkFamilies` over the rows they pin. `New` runs them over the data set, +// `NewTemplate` and `FakeStruct` over what those compile. package fejkdata diff --git a/doc_test.go b/doc_test.go index 30066e1..56124f1 100644 --- a/doc_test.go +++ b/doc_test.go @@ -2,18 +2,20 @@ package fejkdata import ( "go/ast" + "go/build" "go/parser" "go/token" - "os" "regexp" "strings" "testing" ) -var backticked = regexp.MustCompile("`([A-Za-z][A-Za-z0-9]*(?:\\.[A-Za-z][A-Za-z0-9]*)?)`") +// Every backtick in the Vocabulary is a symbol, so an unrecognised spelling fails +// rather than going unchecked. +var backticked = regexp.MustCompile("`([^`]+)`") -// TestVocabularyNamesDeclaredSymbols proves every symbol the package doc's Vocabulary -// names is declared here, so a renamed unit renames its entry with it. +// TestVocabularyNamesDeclaredSymbols proves the package doc's Vocabulary names +// symbols this package declares, so a renamed unit renames its entry with it. func TestVocabularyNamesDeclaredSymbols(t *testing.T) { files := sourceFiles(t) declared := declaredSymbols(files) @@ -24,21 +26,15 @@ func TestVocabularyNamesDeclaredSymbols(t *testing.T) { } } -// sourceFiles parses the package's own files, the tests left out: the Vocabulary -// indexes what ships, not what proves it. func sourceFiles(t *testing.T) []*ast.File { t.Helper() - entries, err := os.ReadDir(".") + pkg, err := build.ImportDir(".", 0) if err != nil { t.Fatalf("read the package directory: %v", err) } fset := token.NewFileSet() var files []*ast.File - for _, e := range entries { - name := e.Name() - if !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") { - continue - } + for _, name := range pkg.GoFiles { f, err := parser.ParseFile(fset, name, nil, parser.ParseComments) if err != nil { t.Fatalf("parse %s: %v", name, err) @@ -48,7 +44,6 @@ func sourceFiles(t *testing.T) []*ast.File { return files } -// vocabulary is the package doc from its Vocabulary heading on. func vocabulary(t *testing.T, files []*ast.File) string { t.Helper() for _, f := range files { @@ -63,45 +58,57 @@ func vocabulary(t *testing.T, files []*ast.File) string { return "" } -// declaredSymbols is every name the package declares, a method and a struct field -// under its type as well: fence, draws.pin, tableRead.whole. +// declaredSymbols is every name the package declares at the top level, a method and +// a struct field keyed under its type: draws.pin, tableRead.whole. A name declared +// inside a function body is not one a Vocabulary entry may name. func declaredSymbols(files []*ast.File) map[string]bool { names := map[string]bool{} for _, f := range files { - ast.Inspect(f, func(n ast.Node) bool { - switch d := n.(type) { + for _, decl := range f.Decls { + switch d := decl.(type) { case *ast.FuncDecl: names[d.Name.Name] = true - if d.Recv != nil { - names[receiverType(d.Recv)+"."+d.Name.Name] = true - } - case *ast.TypeSpec: - names[d.Name.Name] = true - addFields(names, d) - case *ast.ValueSpec: - for _, id := range d.Names { - names[id.Name] = true + if recv := receiverType(d.Recv); recv != "" { + names[recv+"."+d.Name.Name] = true } + case *ast.GenDecl: + addSpecs(names, d) } - return true - }) + } } return names } -func addFields(names map[string]bool, d *ast.TypeSpec) { - s, isStruct := d.Type.(*ast.StructType) +func addSpecs(names map[string]bool, d *ast.GenDecl) { + for _, spec := range d.Specs { + switch s := spec.(type) { + case *ast.TypeSpec: + names[s.Name.Name] = true + addFields(names, s) + case *ast.ValueSpec: + for _, id := range s.Names { + names[id.Name] = true + } + } + } +} + +func addFields(names map[string]bool, s *ast.TypeSpec) { + st, isStruct := s.Type.(*ast.StructType) if !isStruct { return } - for _, field := range s.Fields.List { + for _, field := range st.Fields.List { for _, id := range field.Names { - names[d.Name.Name+"."+id.Name] = true + names[s.Name.Name+"."+id.Name] = true } } } func receiverType(recv *ast.FieldList) string { + if recv == nil || len(recv.List) == 0 { + return "" + } e := recv.List[0].Type if star, isPointer := e.(*ast.StarExpr); isPointer { e = star.X -- 2.52.0 From 2a6d1df1bf8c75cb727bb226b8b8a7523f6e179c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 10:27:04 +0200 Subject: [PATCH 013/966] Keep the vocabulary out of the page a consumer lands on --- README.md | 7 +++++++ doc.go | 8 +++++--- doc_test.go | 11 +++++------ 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 08132e0..f6590d4 100644 --- a/README.md +++ b/README.md @@ -861,6 +861,13 @@ the Development section below, and who ships a register the four above then draw a personal instance. Valid while the project wants contribution from outside: the workflow decided nothing, being portable already — `github.api_url` and `secrets.GITHUB_TOKEN` resolve on either host, so only the names moved. +- **The vocabulary sits below `doc.go`'s package clause, not in the package doc.** + Goal 1 is what a developer choosing a library reads first, and the package doc is + that page: a glossary of unexported units would fill two thirds of it with names + no consumer can call, and Go doc comments have no code markup, so each one would + render its backticks literally. Below the clause it reaches goal 2's reader — who + opens the file — and nothing else. Valid while the glossary names units a consumer + cannot reach; one that came to name the exported API would belong above the clause. - **Options and fields share one namespace.** `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a key, or prefixing options, would tax every template to guard against a diff --git a/doc.go b/doc.go index 667b40c..691ca0f 100644 --- a/doc.go +++ b/doc.go @@ -9,9 +9,12 @@ // f.Fake("sv_SE.address.locality") // "Sundbyberg": a second call draws afresh // // Several sources merge in order, the last winning a name clash, so custom data -// layers over the built-ins. The JSON template format is documented in the README. +// layers over the built-ins. The JSON template format is documented in the [README]. // -// # Vocabulary +// [README]: https://github.com/larvit/fejkdata#readme +package fejkdata + +// Vocabulary // // The words the rest of the package uses, and the unit that owns each: // @@ -41,4 +44,3 @@ // expansion's held names, `drawCheck` over a render's reference paths, and // `checkFamilies` over the rows they pin. `New` runs them over the data set, // `NewTemplate` and `FakeStruct` over what those compile. -package fejkdata diff --git a/doc_test.go b/doc_test.go index 56124f1..e544a37 100644 --- a/doc_test.go +++ b/doc_test.go @@ -47,14 +47,13 @@ func sourceFiles(t *testing.T) []*ast.File { func vocabulary(t *testing.T, files []*ast.File) string { t.Helper() for _, f := range files { - if f.Doc == nil { - continue - } - if _, section, found := strings.Cut(f.Doc.Text(), "\n# Vocabulary\n"); found { - return section + for _, c := range f.Comments { + if section, found := strings.CutPrefix(c.Text(), "Vocabulary\n"); found { + return section + } } } - t.Fatal("no package doc carries a Vocabulary heading, so fence, hold, pin and the rest are defined nowhere a reader of the code meets them") + t.Fatal("no comment opens with a Vocabulary heading, so fence, hold, pin and the rest are defined nowhere a reader of the code meets them") return "" } -- 2.52.0 From c6b1f3dc7739cc1ee8861ed0740079c8d1b04d52 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 10:33:31 +0200 Subject: [PATCH 014/966] Scan every Vocabulary comment, and fail when none names a symbol --- doc_test.go | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/doc_test.go b/doc_test.go index e544a37..258f7cd 100644 --- a/doc_test.go +++ b/doc_test.go @@ -14,12 +14,16 @@ import ( // rather than going unchecked. var backticked = regexp.MustCompile("`([^`]+)`") -// TestVocabularyNamesDeclaredSymbols proves the package doc's Vocabulary names -// symbols this package declares, so a renamed unit renames its entry with it. +// TestVocabularyNamesDeclaredSymbols proves the Vocabulary names symbols this +// package declares, so a renamed unit renames its entry with it. func TestVocabularyNamesDeclaredSymbols(t *testing.T) { files := sourceFiles(t) declared := declaredSymbols(files) - for _, m := range backticked.FindAllStringSubmatch(vocabulary(t, files), -1) { + named := backticked.FindAllStringSubmatch(vocabulary(t, files), -1) + if len(named) == 0 { + t.Fatal("the Vocabulary names no symbol, so nothing holds a renamed unit to its entry") + } + for _, m := range named { if !declared[m[1]] { t.Errorf("the Vocabulary names %s, which the package declares nowhere", m[1]) } @@ -46,15 +50,18 @@ func sourceFiles(t *testing.T) []*ast.File { func vocabulary(t *testing.T, files []*ast.File) string { t.Helper() + var found strings.Builder for _, f := range files { for _, c := range f.Comments { - if section, found := strings.CutPrefix(c.Text(), "Vocabulary\n"); found { - return section + if section, is := strings.CutPrefix(c.Text(), "Vocabulary\n\n"); is { + found.WriteString(section) } } } - t.Fatal("no comment opens with a Vocabulary heading, so fence, hold, pin and the rest are defined nowhere a reader of the code meets them") - return "" + if found.Len() == 0 { + t.Fatal("no comment opens with a Vocabulary heading, so fence, hold, pin and the rest are defined nowhere a reader of the code meets them") + } + return found.String() } // declaredSymbols is every name the package declares at the top level, a method and -- 2.52.0 From e583e42dfa8c5cc6befe847e5bad041e815cabf4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 10:38:57 +0200 Subject: [PATCH 015/966] Cut the vocabulary to what only it says, and fix what it got wrong --- README.md | 8 ++++---- doc.go | 38 +++++++++++++++----------------------- doc_test.go | 24 +++++++++++------------- todo.md | 3 +++ 4 files changed, 33 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index f6590d4..feddb0d 100644 --- a/README.md +++ b/README.md @@ -863,11 +863,11 @@ the Development section below, and who ships a register the four above then draw `secrets.GITHUB_TOKEN` resolve on either host, so only the names moved. - **The vocabulary sits below `doc.go`'s package clause, not in the package doc.** Goal 1 is what a developer choosing a library reads first, and the package doc is - that page: a glossary of unexported units would fill two thirds of it with names - no consumer can call, and Go doc comments have no code markup, so each one would + that page: the glossary would fill two thirds of it with the render path's units, + mostly unexported, and Go doc comments have no code markup, so every name would render its backticks literally. Below the clause it reaches goal 2's reader — who - opens the file — and nothing else. Valid while the glossary names units a consumer - cannot reach; one that came to name the exported API would belong above the clause. + opens the file — and nothing else. Valid while the glossary's subject is the render + path; one written to teach the exported API belongs above the clause. - **Options and fields share one namespace.** `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a key, or prefixing options, would tax every template to guard against a diff --git a/doc.go b/doc.go index 691ca0f..2f8e23d 100644 --- a/doc.go +++ b/doc.go @@ -16,31 +16,23 @@ package fejkdata // Vocabulary // -// The words the rest of the package uses, and the unit that owns each: -// -// - draw — one pick from a choice, or one row taken from a table, and the value -// the pick produced: `pick` and `drawn` settle a choice, `table.draw` a row, and -// `draw` carries what a read produced. -// - expansion — one render of one format, by `expand`. A nested template is an -// expansion of its own. +// - draw — one pick from a choice, or one row taken from a table, and the value a +// read produced: `pick`, `drawn`, `table.draw`, `draw`. +// - expansion — one render of one format: `expand`. // - render — what owns one `drawSet`, and so what one reference draw spans: -// `Generator.Fake`, `FakeRecord`, `FakeStruct`, `Template.Fake` and -// `RecordTemplate.Fake` each start one, as does `expandAnew` per iteration of a -// `template.repeat` — an iteration is an expansion and a render of its own. -// - hold — keeping one draw of a name, so every route to it reads that value. -// `template.held` lists what an expansion holds, `draws` keeps them and -// `readField` reads through it; a reference path is held for the whole render. -// - draw group — `template.drawGroup`, which gives the reference paths under it -// draws of their own inside the render. `drawSet` holds the unnamed group's and -// each named one's, and `drawScope` says which one a template renders in. -// - pin — fixing which row of a table the render uses. `draws.pin` pins it and the -// rows of its ancestors, and `draws.rowOf` draws one where none is pinned. +// `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` +// start one each, `FakeStruct` one per record, and `expandAnew` one per +// iteration of a `template.repeat`. +// - hold — keeping one draw of a name, so every route to it reads that value: +// `template.held`, `draws`, `readField`. +// - draw group — reference draws held apart inside one render: +// `template.drawGroup` as the data spells it, `template.drawGroupKey` as a +// render reads it, `drawSet`, `drawScope`. +// - pin — fixing which row of a table the render uses: `draws.pin`, `draws.rowOf`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. // - whole — a read that lands on a `row`, so the row renders through its table's // format: `tableRead.whole` says a read did, `table.whole` is the node. -// - fence — a load-time check refusing data that two routes to one draw would -// disagree in, which is why a render cannot fail: `heldCheck` over an -// expansion's held names, `drawCheck` over a render's reference paths, and -// `checkFamilies` over the rows they pin. `New` runs them over the data set, -// `NewTemplate` and `FakeStruct` over what those compile. +// - fence — a load-time check, so rendering a compiled tree cannot fail: `New` +// runs them over the data set, `NewTemplate` and `FakeStruct` over what those +// compile. The draw fences are `heldCheck`, `drawCheck` and `checkFamilies`. diff --git a/doc_test.go b/doc_test.go index 258f7cd..476c14d 100644 --- a/doc_test.go +++ b/doc_test.go @@ -10,12 +10,10 @@ import ( "testing" ) -// Every backtick in the Vocabulary is a symbol, so an unrecognised spelling fails -// rather than going unchecked. -var backticked = regexp.MustCompile("`([^`]+)`") +// Every backtick in the Vocabulary is a symbol. +var backticked = regexp.MustCompile("`([^`\n]+)`") -// TestVocabularyNamesDeclaredSymbols proves the Vocabulary names symbols this -// package declares, so a renamed unit renames its entry with it. +// TestVocabularyNamesDeclaredSymbols holds a renamed unit to its Vocabulary entry. func TestVocabularyNamesDeclaredSymbols(t *testing.T) { files := sourceFiles(t) declared := declaredSymbols(files) @@ -50,23 +48,23 @@ func sourceFiles(t *testing.T) []*ast.File { func vocabulary(t *testing.T, files []*ast.File) string { t.Helper() - var found strings.Builder + var sections strings.Builder for _, f := range files { for _, c := range f.Comments { - if section, is := strings.CutPrefix(c.Text(), "Vocabulary\n\n"); is { - found.WriteString(section) + section, found := strings.CutPrefix(c.Text(), "Vocabulary\n\n") + if found && c.Pos() > f.Package { + sections.WriteString(section) } } } - if found.Len() == 0 { - t.Fatal("no comment opens with a Vocabulary heading, so fence, hold, pin and the rest are defined nowhere a reader of the code meets them") + if sections.Len() == 0 { + t.Fatal("no comment below a package clause opens with a Vocabulary heading, so the words the package is written in are defined nowhere a reader of the code meets them") } - return found.String() + return sections.String() } // declaredSymbols is every name the package declares at the top level, a method and -// a struct field keyed under its type: draws.pin, tableRead.whole. A name declared -// inside a function body is not one a Vocabulary entry may name. +// a struct field keyed under its type: draws.pin, tableRead.whole. func declaredSymbols(files []*ast.File) map[string]bool { names := map[string]bool{} for _, f := range files { diff --git a/todo.md b/todo.md index 6ea95de..1bf159f 100644 --- a/todo.md +++ b/todo.md @@ -43,6 +43,9 @@ This round, in order: - Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go` hand-roll `drawSet{unnamed: draws{s: s}}` at three sites; name the lazy-map variant if the difference is deliberate. +- Run the package doc's Go example under test, as `readme_test.go` runs the README's + JSON blocks: `doc.go` and the README both print `Järvedsvägen 43\n891 77 Järved` + for seed 42, and nothing renders either, so a draw that moves leaves both false. ### Data -- 2.52.0 From 847236314df977fafc41517bc982f258815b00ea Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:01:16 +0200 Subject: [PATCH 016/966] Record PTS's fiction number series as sv_SE.phone's inert source --- docs/research/research-sources-se.md | 3 ++- todo.md | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/research/research-sources-se.md b/docs/research/research-sources-se.md index 6ebf7f5..1157dbf 100644 --- a/docs/research/research-sources-se.md +++ b/docs/research/research-sources-se.md @@ -15,7 +15,7 @@ Scope: names, identifiers, phone, company, plates/cars, words, dates/misc, other | Bolagsverket/SCB bulk company files | Not stated ("värdefulla datamängder", free, no agreement) | Derive stats only; do not embed raw | Cite | | Bankinfrastruktur clearing CSV | Not stated; accuracy disclaimer | Facts only; flag | Cite BSAB | | Bankgirot PDFs | "© Bankgirocentralen BGC AB. All rights reserved." (Informationsklass: Öppen) | No verbatim copy; facts only | — | -| PTS numbering plan PDF | Not stated (no licence on pts.se or in PDF) | Facts only; flag | Cite PTS | +| PTS numbering plan PDF, pts.se number pages | Not stated (no licence on pts.se or in PDF) | Facts only; flag | Cite PTS | | Transportstyrelsen blocked combinations | Not stated on page | Facts (a list of 99 codes); flag | Cite | | SALDO morphology (Språkbanken) | CC BY 4.0 | Yes | Cite: Språkbanken (2017) SALDO's morphology, DOI 10.23695/agcm-ny22 | | Språkbanken word statistics | CC BY 4.0 | Yes | Cite Språkbanken | @@ -90,6 +90,7 @@ Recommendation: embed SCB 2022 xlsx-derived lists (top-N per sheet with counts) ## 3. Phone - Source: PTS "The Swedish numbering plan for telephony according to ITU-T E.164", 2024-01-08 (verified, text extracted): https://pts.se/globalassets/globala-block/nummertillstand/the-swedish-numbering-plan-for-telephony-according-to-itu---2024-01-08.pdf. Swedish "nrplansammanstallning-2026-02-04.pdf" and https://nummer.pts.se/NbrPlanSearch are behind a Radware bot check (both curl and Playwright blocked). No licence stated by PTS anywhere found; pts.se says contact pts@pts.se for data. +- Reserved for fiction, PTS "Telefonnummer till böcker och filmer" (verified 2026-09-21; curl and Playwright are Radware-blocked, a rendering fetcher gets through): https://www.pts.se/internet-och-telefoni/telefonnummer-och-adressering/telefonnummer-till-bocker-och-filmer/ — mobile 070-1740605–1740699, geographic 08-46500400–46500499 (Stockholm), 031-3900600–3900699 (Göteborg), 040-6280400–6280499 (Malmö), 0980-319200–319299 (Kiruna). 495 numbers PTS holds back so they reach nobody, which is goal 14's never-reaching path for `sv_SE.phone`. Same licence status as the numbering plan: none stated, facts only, cite PTS. The mobile block fits the shipped `{a} {b} {c}` grouping (`174 06 05`–`99`), as do 031 and 040; Stockholm's 8 subscriber digits and Kiruna's 6 need a format of their own, and 0980 is not among the 20 shipped area codes. - 264 geographic area codes (PDF text yields 263 "Area code for" rows; one likely split across a page break — flag). Area code = trunk "0" + NDC of 1–3 digits (08 Stockholm, 031 Göteborg, 040 Malmö, 0480 Kalmar). Geographic N(S)N (NDC + subscriber) max 9 digits, min 7 (2-digit NDC) or 8 (3-digit NDC); Stockholm min 7. Each row in the PDF gives NDC, max/min length, "Area code for ", so a `{ndc, name}` list can be built from it. - Mobile: NDC 70, 72, 73, 76, 79 — N(S)N exactly 9 digits, i.e. 07X-XXX XX XX (10 digits with trunk 0). Others: 71 mobile broadband/M2M (13 digits), 74 paging, 75 personal numbering, 77 shared cost, 20 freephone, 10 location-independent (10 AXX XX XX, A=1–8), 378 M2M fixed (10 digits). - Formatting (Språkrådet/Isof frågelådan, via search excerpts): hyphen after area code, subscriber digits grouped 2–3 from the left: `08-668 01 50`, `031-123 45 67`, `0480-123 45`, `070-123 45 67` (also `0701-23 45 67` accepted). International: `+46 70 123 45 67` (drop trunk 0). diff --git a/todo.md b/todo.md index 1bf159f..cbf9c06 100644 --- a/todo.md +++ b/todo.md @@ -68,8 +68,10 @@ by a `data-import/` script, as README goal 12 asks. `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. - Audit the rest of the shipped set against goal 14 and give each a never-reaching - path where it lacks one: `phone` draws live PTS and NANP ranges, `bankgiro`, - `plusgiro` and `routing` draw live prefixes. + path where it lacks one: `phone` draws live PTS and NANP ranges — PTS's five + fiction series are the Swedish inert set, sourced in + [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one + still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. - Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold: ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it and goal 3 promises otherwise. -- 2.52.0 From 84d98011a7fe518c7d461ced1fd4b4393c591cee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:01:09 +0200 Subject: [PATCH 017/966] Fail when a template field names no pass that fills it --- doc_test.go | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) diff --git a/doc_test.go b/doc_test.go index 476c14d..6111631 100644 --- a/doc_test.go +++ b/doc_test.go @@ -123,3 +123,80 @@ func receiverType(recv *ast.FieldList) string { } return id.Name } + +// fillHeader opens the comment naming the pass that fills the fields below it. +const fillHeader = "Filled by " + +// TestTemplateFieldsNameTheirPass holds every template field to a header naming the +// pass that writes it. +func TestTemplateFieldsNameTheirPass(t *testing.T) { + files := sourceFiles(t) + declared := declaredSymbols(files) + st, headers := templateStruct(t, files) + for _, field := range st.Fields.List { + for _, id := range field.Names { + header := headerAbove(headers, id.Pos()) + if header == "" { + t.Errorf("template.%s sits under no %q header, so nothing says which pass fills it", id.Name, fillHeader) + continue + } + named := backticked.FindAllStringSubmatch(header, -1) + if len(named) == 0 { + t.Errorf("the header above template.%s names no function, so it names no pass", id.Name) + } + for _, m := range named { + if !declared[m[1]] { + t.Errorf("the header above template.%s names %s, which the package declares nowhere", id.Name, m[1]) + } + } + } + } +} + +// templateStruct is the template struct and the fill headers standing between its +// fields, in source order. +func templateStruct(t *testing.T, files []*ast.File) (*ast.StructType, []*ast.CommentGroup) { + t.Helper() + for _, f := range files { + for _, decl := range f.Decls { + d, isGen := decl.(*ast.GenDecl) + if !isGen { + continue + } + for _, spec := range d.Specs { + s, isType := spec.(*ast.TypeSpec) + if !isType || s.Name.Name != "template" { + continue + } + st, isStruct := s.Type.(*ast.StructType) + if !isStruct { + t.Fatal("template is not a struct") + } + return st, fillHeaders(f, st) + } + } + } + t.Fatal("the package declares no template struct") + return nil, nil +} + +func fillHeaders(f *ast.File, st *ast.StructType) []*ast.CommentGroup { + var found []*ast.CommentGroup + for _, c := range f.Comments { + if c.Pos() > st.Pos() && c.End() < st.End() && strings.HasPrefix(c.Text(), fillHeader) { + found = append(found, c) + } + } + return found +} + +// headerAbove is the last fill header standing above pos. +func headerAbove(headers []*ast.CommentGroup, pos token.Pos) string { + text := "" + for _, c := range headers { + if c.Pos() < pos { + text = c.Text() + } + } + return text +} -- 2.52.0 From 1e80fd54f6d973673d970fbd8cd4fda463dfcbfc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:02:50 +0200 Subject: [PATCH 018/966] Group the template fields under the pass that fills each --- node.go | 52 ++++++++++++++++++++++++++++++++-------------------- todo.md | 3 --- 2 files changed, 32 insertions(+), 23 deletions(-) diff --git a/node.go b/node.go index 56b7af7..56949ba 100644 --- a/node.go +++ b/node.go @@ -42,33 +42,45 @@ func (*null) isNode() {} // JSON string is a template with no fields. repeat (default 1) renders that format // that many times and joins the results with separator (default ""), each render // an independent pick. +// +// The groups below are filled in order: compiling the JSON writes the first and +// calls compileFormat for the second, then the reference link writes the third and +// calls compileFormat a second time, since a reference is a field the format reads +// and the first compile ran before any was bound. type template struct { - format string - fields map[string]node - repeat int - separator string - datatype DataType - ops []op // format compiled once (see compileOps); what expand walks - grow int // minimum output size, to size the render buffer - fixed bool // no op varies, so every render is lit - lit string // the whole output when fixed - refs map[string]refBinding // each reference the format reads -> what it is bound to + // Filled by `compileString`, `compileTemplate`, `table.compileFormat` and + // `checkCells`, from what the JSON says: + format string + fields map[string]node + repeat int + separator string + datatype DataType + drawGroup string // the draw group it draws in, as written; "" keeps its caller's + fromString bool // written as a JSON string rather than an object + record bool // compiled at the top without a repeat, so its fields are record columns + table *table // the table whose format this is, whose columns are the fields + cellOf *table // the table whose cell this is + cellRow int // the row the cell sits in + + // Filled by `template.compileFormat`, from the format over the fields above: + ops []op // what expand walks + grow int // minimum output size, to size the render buffer + fixed bool // no op varies, so every render is lit + lit string // the whole output when fixed // bound maps each field the format addresses by dotted path to one path token // reading it, which is the half of an overlap the fences name. nil when the // format takes no path. bound map[string]string // held is every name drawn once per expansion: the bound levels above, plus the // siblings a {calc()} reads. nil when the format holds nothing (see expand). - held map[string]bool - heldLocal bool // some held name is kept by the expansion itself, so expand makes its draws - fromString bool // written as a JSON string rather than an object - readsColumn *columnRead // set when the format is one reference alone reading a record's column - record bool // compiled at the top without a repeat, so its fields are record columns - table *table // the table whose format this is, whose columns are the fields - cellOf *table // the table whose cell this is - cellRow int // the row the cell sits in - drawGroup string // the draw group it draws in, as written; "" keeps its caller's - drawGroupKey string // its draw group keyed by its category once linked: what a render reads its reference paths under + held map[string]bool + heldLocal bool // some held name is kept by the expansion itself, so expand makes its draws + + // Filled by `linkRefs`, or `linkNodeRefs` for an inline template, from the + // assembled tree, which also add each bound reference to fields above: + refs map[string]refBinding // each reference the format reads -> what it is bound to + readsColumn *columnRead // set when the format is one reference alone reading a record's column + drawGroupKey string // its draw group keyed by its category: what a render reads its reference paths under } func (*template) isNode() {} diff --git a/todo.md b/todo.md index cbf9c06..8cc427d 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Say on the `template` struct which pass fills which field, and that `compileFormat` - runs a second time once `linkTemplateRefs` binds the references: nine of its fields - are written by four other files, and the order lives only in `loadData`'s body. - State on `rowSet` that it simulates `(*draws).pin` and `rowOf`, and prove the two agree with a test: the load-time walk reimplements the render-time pinning, nothing catches them drifting, and the failure is data that loads and then renders a family -- 2.52.0 From 1c753fcac9963a24a1eee0e04d8665b4a0251f6f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:11:14 +0200 Subject: [PATCH 019/966] Skip a trailing fill header, and catch an embedded field --- doc_test.go | 70 +++++++++++++++++++++++++++++++---------------------- 1 file changed, 41 insertions(+), 29 deletions(-) diff --git a/doc_test.go b/doc_test.go index 6111631..995af0f 100644 --- a/doc_test.go +++ b/doc_test.go @@ -15,20 +15,27 @@ var backticked = regexp.MustCompile("`([^`\n]+)`") // TestVocabularyNamesDeclaredSymbols holds a renamed unit to its Vocabulary entry. func TestVocabularyNamesDeclaredSymbols(t *testing.T) { - files := sourceFiles(t) + _, files := sourceFiles(t) declared := declaredSymbols(files) - named := backticked.FindAllStringSubmatch(vocabulary(t, files), -1) - if len(named) == 0 { + if namesDeclared(t, "the Vocabulary", vocabulary(t, files), declared) == 0 { t.Fatal("the Vocabulary names no symbol, so nothing holds a renamed unit to its entry") } - for _, m := range named { - if !declared[m[1]] { - t.Errorf("the Vocabulary names %s, which the package declares nowhere", m[1]) - } - } } -func sourceFiles(t *testing.T) []*ast.File { +// namesDeclared reports how many symbols text backticks, failing for each one the +// package declares nowhere. +func namesDeclared(t *testing.T, where, text string, declared map[string]bool) int { + t.Helper() + named := backticked.FindAllStringSubmatch(text, -1) + for _, m := range named { + if !declared[m[1]] { + t.Errorf("%s names %s, which the package declares nowhere", where, m[1]) + } + } + return len(named) +} + +func sourceFiles(t *testing.T) (*token.FileSet, []*ast.File) { t.Helper() pkg, err := build.ImportDir(".", 0) if err != nil { @@ -43,7 +50,7 @@ func sourceFiles(t *testing.T) []*ast.File { } files = append(files, f) } - return files + return fset, files } func vocabulary(t *testing.T, files []*ast.File) string { @@ -124,38 +131,34 @@ func receiverType(recv *ast.FieldList) string { return id.Name } -// fillHeader opens the comment naming the pass that fills the fields below it. const fillHeader = "Filled by " -// TestTemplateFieldsNameTheirPass holds every template field to a header naming the -// pass that writes it. +// TestTemplateFieldsNameTheirPass holds a renamed pass to the group it fills. func TestTemplateFieldsNameTheirPass(t *testing.T) { - files := sourceFiles(t) + fset, files := sourceFiles(t) declared := declaredSymbols(files) - st, headers := templateStruct(t, files) + doc, st, headers := templateStruct(t, fset, files) + namesDeclared(t, "the template doc", doc, declared) for _, field := range st.Fields.List { + if len(field.Names) == 0 { + t.Error("an embedded field of template sits under no header, so nothing says which pass fills it") + } for _, id := range field.Names { header := headerAbove(headers, id.Pos()) if header == "" { t.Errorf("template.%s sits under no %q header, so nothing says which pass fills it", id.Name, fillHeader) continue } - named := backticked.FindAllStringSubmatch(header, -1) - if len(named) == 0 { + if namesDeclared(t, "the header above template."+id.Name, header, declared) == 0 { t.Errorf("the header above template.%s names no function, so it names no pass", id.Name) } - for _, m := range named { - if !declared[m[1]] { - t.Errorf("the header above template.%s names %s, which the package declares nowhere", id.Name, m[1]) - } - } } } } -// templateStruct is the template struct and the fill headers standing between its -// fields, in source order. -func templateStruct(t *testing.T, files []*ast.File) (*ast.StructType, []*ast.CommentGroup) { +// templateStruct is the template struct, its doc, and the fill headers standing +// between its fields, in source order. +func templateStruct(t *testing.T, fset *token.FileSet, files []*ast.File) (string, *ast.StructType, []*ast.CommentGroup) { t.Helper() for _, f := range files { for _, decl := range f.Decls { @@ -172,18 +175,27 @@ func templateStruct(t *testing.T, files []*ast.File) (*ast.StructType, []*ast.Co if !isStruct { t.Fatal("template is not a struct") } - return st, fillHeaders(f, st) + return d.Doc.Text() + s.Doc.Text(), st, fillHeaders(fset, f, st) } } } t.Fatal("the package declares no template struct") - return nil, nil + return "", nil, nil } -func fillHeaders(f *ast.File, st *ast.StructType) []*ast.CommentGroup { +// fillHeaders skips a header trailing a field: it names that field, never the +// fields below it. +func fillHeaders(fset *token.FileSet, f *ast.File, st *ast.StructType) []*ast.CommentGroup { + trailing := map[int]bool{} + for _, field := range st.Fields.List { + trailing[fset.Position(field.End()).Line] = true + } var found []*ast.CommentGroup for _, c := range f.Comments { - if c.Pos() > st.Pos() && c.End() < st.End() && strings.HasPrefix(c.Text(), fillHeader) { + if c.Pos() < st.Pos() || c.End() > st.End() || trailing[fset.Position(c.Pos()).Line] { + continue + } + if strings.HasPrefix(c.Text(), fillHeader) { found = append(found, c) } } -- 2.52.0 From 311ed40d058edb39e7d5ae18333ac0cf70f72927 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:11:14 +0200 Subject: [PATCH 020/966] Say which pass fills the cell fields, and when the format recompiles --- node.go | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/node.go b/node.go index 56949ba..5b01c94 100644 --- a/node.go +++ b/node.go @@ -43,13 +43,11 @@ func (*null) isNode() {} // that many times and joins the results with separator (default ""), each render // an independent pick. // -// The groups below are filled in order: compiling the JSON writes the first and -// calls compileFormat for the second, then the reference link writes the third and -// calls compileFormat a second time, since a reference is a field the format reads -// and the first compile ran before any was bound. +// Each group below names the pass that fills it. What `template.compileFormat` +// fills is final only once `linkTemplateRefs` has run: where the format holds a +// reference, that binds it and compiles the format again. type template struct { - // Filled by `compileString`, `compileTemplate`, `table.compileFormat` and - // `checkCells`, from what the JSON says: + // Filled by `compileString`, `compileTemplate` and `table.compileFormat`: format string fields map[string]node repeat int @@ -59,10 +57,12 @@ type template struct { fromString bool // written as a JSON string rather than an object record bool // compiled at the top without a repeat, so its fields are record columns table *table // the table whose format this is, whose columns are the fields - cellOf *table // the table whose cell this is - cellRow int // the row the cell sits in - // Filled by `template.compileFormat`, from the format over the fields above: + // Filled by `checkCells`, after the cell's own format has compiled: + cellOf *table // the table whose cell this is + cellRow int // the row the cell sits in + + // Filled by `template.compileFormat`, from `template.format` and `template.refs`: ops []op // what expand walks grow int // minimum output size, to size the render buffer fixed bool // no op varies, so every render is lit @@ -76,8 +76,8 @@ type template struct { held map[string]bool heldLocal bool // some held name is kept by the expansion itself, so expand makes its draws - // Filled by `linkRefs`, or `linkNodeRefs` for an inline template, from the - // assembled tree, which also add each bound reference to fields above: + // Filled by `linkTemplateRefs` and `keyDrawGroup` from the assembled tree; + // `linkTemplateRefs` adds each bound reference to `template.fields` too: refs map[string]refBinding // each reference the format reads -> what it is bound to readsColumn *columnRead // set when the format is one reference alone reading a record's column drawGroupKey string // its draw group keyed by its category: what a render reads its reference paths under @@ -217,8 +217,8 @@ func compileString(s string) (node, error) { return t, nil } -// compileFormat compiles the format into ops once every field is in place, and -// applies the fences that need the compiled reads. +// compileFormat compiles the format into ops, and applies the fences that need the +// compiled reads. func (t *template) compileFormat() error { c := compileOps(t.format, t.refs) t.ops, t.grow, t.bound, t.held, t.heldLocal = c.ops, c.grow, c.bound, c.held, c.heldLocal -- 2.52.0 From dda470dc9c26358a180a126aa660592039088901 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:17:57 +0200 Subject: [PATCH 021/966] Hold the template doc to naming a pass, and name the check for what it does --- doc_test.go | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/doc_test.go b/doc_test.go index 995af0f..728f0f2 100644 --- a/doc_test.go +++ b/doc_test.go @@ -133,12 +133,15 @@ func receiverType(recv *ast.FieldList) string { const fillHeader = "Filled by " -// TestTemplateFieldsNameTheirPass holds a renamed pass to the group it fills. -func TestTemplateFieldsNameTheirPass(t *testing.T) { +// TestTemplateFieldGroupsNameADeclaredPass holds a renamed pass to the group it +// fills, and to the template doc. +func TestTemplateFieldGroupsNameADeclaredPass(t *testing.T) { fset, files := sourceFiles(t) declared := declaredSymbols(files) doc, st, headers := templateStruct(t, fset, files) - namesDeclared(t, "the template doc", doc, declared) + if namesDeclared(t, "the template doc", doc, declared) == 0 { + t.Error("the template doc names no pass, so nothing says when a field is final") + } for _, field := range st.Fields.List { if len(field.Names) == 0 { t.Error("an embedded field of template sits under no header, so nothing says which pass fills it") @@ -150,7 +153,7 @@ func TestTemplateFieldsNameTheirPass(t *testing.T) { continue } if namesDeclared(t, "the header above template."+id.Name, header, declared) == 0 { - t.Errorf("the header above template.%s names no function, so it names no pass", id.Name) + t.Errorf("the header above template.%s names no symbol, so it names no pass", id.Name) } } } -- 2.52.0 From a2074a455e3e83c6734c52dcc1e9b9309fdae2bf Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:17:57 +0200 Subject: [PATCH 022/966] Put the reference write beside fields, and cut the header restatement --- node.go | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/node.go b/node.go index 5b01c94..6a3f167 100644 --- a/node.go +++ b/node.go @@ -43,13 +43,13 @@ func (*null) isNode() {} // that many times and joins the results with separator (default ""), each render // an independent pick. // -// Each group below names the pass that fills it. What `template.compileFormat` -// fills is final only once `linkTemplateRefs` has run: where the format holds a -// reference, that binds it and compiles the format again. +// What `template.compileFormat` fills is final only once `linkTemplateRefs` has +// run: where the format holds a reference, that binds it and compiles the format +// again. type template struct { // Filled by `compileString`, `compileTemplate` and `table.compileFormat`: format string - fields map[string]node + fields map[string]node // `linkTemplateRefs` adds each bound reference here repeat int separator string datatype DataType @@ -76,8 +76,7 @@ type template struct { held map[string]bool heldLocal bool // some held name is kept by the expansion itself, so expand makes its draws - // Filled by `linkTemplateRefs` and `keyDrawGroup` from the assembled tree; - // `linkTemplateRefs` adds each bound reference to `template.fields` too: + // Filled by `linkTemplateRefs` and `keyDrawGroup`, from the assembled tree: refs map[string]refBinding // each reference the format reads -> what it is bound to readsColumn *columnRead // set when the format is one reference alone reading a record's column drawGroupKey string // its draw group keyed by its category: what a render reads its reference paths under -- 2.52.0 From 67ff2b258ef0778500517e606fd691b80b1a55bd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:20:56 +0200 Subject: [PATCH 023/966] Delete the duplicated recompile note and a miscount beside named --- graph.go | 4 +--- reference.go | 3 +-- todo.md | 10 +++++++++- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/graph.go b/graph.go index 79ad8bd..681cc9d 100644 --- a/graph.go +++ b/graph.go @@ -79,9 +79,7 @@ func contained(n node) []namedNode { } // named skips a bound {/path} key: it is a render edge, not containment, so using -// it as a path segment would report a node under a path that does not reach it. Only -// a template's fields hold bindings — loadDir skips a dot-prefixed entry, so a -// folder's children never carry the prefix — so this one skip serves both. +// it as a path segment would report a node under a path that does not reach it. func named(m map[string]node) []namedNode { out := make([]namedNode, 0, len(m)) for _, name := range sortedNames(m) { diff --git a/reference.go b/reference.go index a50a6cd..7b6a510 100644 --- a/reference.go +++ b/reference.go @@ -85,8 +85,7 @@ func linkRefs(root map[string]node) error { // linkTemplateRefs binds one template's references against root, refusing one that names the // category it sits in: a category is a unit, and a reference back into it describes a draw other -// than the fields beside it. A template with no reference is left untouched, so an inline format -// that references nothing costs only the refTokens scan. +// than the fields beside it. func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { names := refTokens(t.format) if len(names) == 0 { diff --git a/todo.md b/todo.md index 8cc427d..a774a92 100644 --- a/todo.md +++ b/todo.md @@ -196,7 +196,9 @@ by a `data-import/` script, as README goal 12 asks. README. Carry the timezone weight's premise in with it: weighting by GeoNames city population moved 300 seeded draws of `misc.territory[US].timezone` from 44 landing on the four zones most Americans live in to 278, and dropped `America/Indiana/Petersburg` - (pop. 2,400) from 17 draws to 0. + (pop. 2,400) from 17 draws to 0. Carry in the decision `linkTemplateRefs`'s doc holds + today, which no entry records: a category is a unit, so a reference back into it + describes a draw other than the fields beside it, which is what goal 3 asks for. - Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds: nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. @@ -237,6 +239,12 @@ because pairing a street with its exact postnummer rewrites shipped rows. ## Not release-bound +- Name goal 2's four criteria with the axes the panel scores, so a recorded score maps + back to the clause it came from: the goal spells them out in prose while `todo.md` + records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. +- Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates + Go at `Dockerfile:21`, and the shipped JSON under `data/` is two-space, so the rule as + written is one no Go file can break and every data file does. - Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub or retire it: Gitea converts a repository to a mirror only by re-creating it, so until that runs the copy there is a second owner of one history and goes stale from this commit. -- 2.52.0 From 8f74968371b566c3c46eae6313e979988c4d7e81 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 11:36:01 +0200 Subject: [PATCH 024/966] Require a fill header to name a function, not just a symbol --- doc_test.go | 65 +++++++++++++++++++++++++++++++++++------------------ 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/doc_test.go b/doc_test.go index 728f0f2..55d8b1a 100644 --- a/doc_test.go +++ b/doc_test.go @@ -6,6 +6,7 @@ import ( "go/parser" "go/token" "regexp" + "slices" "strings" "testing" ) @@ -17,22 +18,33 @@ var backticked = regexp.MustCompile("`([^`\n]+)`") func TestVocabularyNamesDeclaredSymbols(t *testing.T) { _, files := sourceFiles(t) declared := declaredSymbols(files) - if namesDeclared(t, "the Vocabulary", vocabulary(t, files), declared) == 0 { + if len(namesDeclared(t, "the Vocabulary", vocabulary(t, files), declared)) == 0 { t.Fatal("the Vocabulary names no symbol, so nothing holds a renamed unit to its entry") } } -// namesDeclared reports how many symbols text backticks, failing for each one the -// package declares nowhere. -func namesDeclared(t *testing.T, where, text string, declared map[string]bool) int { +// namesDeclared is every symbol text backticks, failing for each one the package +// declares nowhere. +func namesDeclared(t *testing.T, where, text string, declared map[string]bool) []string { t.Helper() - named := backticked.FindAllStringSubmatch(text, -1) - for _, m := range named { + var named []string + for _, m := range backticked.FindAllStringSubmatch(text, -1) { if !declared[m[1]] { t.Errorf("%s names %s, which the package declares nowhere", where, m[1]) } + named = append(named, m[1]) + } + return named +} + +// namesAPass fails unless one symbol text backticks is a function: a field names +// what a pass reads or writes, never the pass. +func namesAPass(t *testing.T, where, text string, declared, funcs map[string]bool) { + t.Helper() + named := namesDeclared(t, where, text, declared) + if !slices.ContainsFunc(named, func(n string) bool { return funcs[n] }) { + t.Errorf("%s names no function, so it names no pass", where) } - return len(named) } func sourceFiles(t *testing.T) (*token.FileSet, []*ast.File) { @@ -73,17 +85,30 @@ func vocabulary(t *testing.T, files []*ast.File) string { // declaredSymbols is every name the package declares at the top level, a method and // a struct field keyed under its type: draws.pin, tableRead.whole. func declaredSymbols(files []*ast.File) map[string]bool { + names := declaredFuncs(files) + for _, f := range files { + for _, decl := range f.Decls { + if d, isGen := decl.(*ast.GenDecl); isGen { + addSpecs(names, d) + } + } + } + return names +} + +// declaredFuncs is every function the package declares, a method keyed under its +// type: template.compileFormat. +func declaredFuncs(files []*ast.File) map[string]bool { names := map[string]bool{} for _, f := range files { for _, decl := range f.Decls { - switch d := decl.(type) { - case *ast.FuncDecl: - names[d.Name.Name] = true - if recv := receiverType(d.Recv); recv != "" { - names[recv+"."+d.Name.Name] = true - } - case *ast.GenDecl: - addSpecs(names, d) + d, isFunc := decl.(*ast.FuncDecl) + if !isFunc { + continue + } + names[d.Name.Name] = true + if recv := receiverType(d.Recv); recv != "" { + names[recv+"."+d.Name.Name] = true } } } @@ -137,11 +162,9 @@ const fillHeader = "Filled by " // fills, and to the template doc. func TestTemplateFieldGroupsNameADeclaredPass(t *testing.T) { fset, files := sourceFiles(t) - declared := declaredSymbols(files) + declared, funcs := declaredSymbols(files), declaredFuncs(files) doc, st, headers := templateStruct(t, fset, files) - if namesDeclared(t, "the template doc", doc, declared) == 0 { - t.Error("the template doc names no pass, so nothing says when a field is final") - } + namesAPass(t, "the template doc", doc, declared, funcs) for _, field := range st.Fields.List { if len(field.Names) == 0 { t.Error("an embedded field of template sits under no header, so nothing says which pass fills it") @@ -152,9 +175,7 @@ func TestTemplateFieldGroupsNameADeclaredPass(t *testing.T) { t.Errorf("template.%s sits under no %q header, so nothing says which pass fills it", id.Name, fillHeader) continue } - if namesDeclared(t, "the header above template."+id.Name, header, declared) == 0 { - t.Errorf("the header above template.%s names no symbol, so it names no pass", id.Name) - } + namesAPass(t, "the header above template."+id.Name, header, declared, funcs) } } } -- 2.52.0 From a1b11e7a5f917e0b602035794d8a6bd68d5b9cd1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:23:59 +0200 Subject: [PATCH 025/966] Prove the walk's row set against the render's pinning --- table_test.go | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/table_test.go b/table_test.go index eded26e..4ffbce7 100644 --- a/table_test.go +++ b/table_test.go @@ -963,3 +963,70 @@ func TestAmbiguousNameNamesARunnablePath(t *testing.T) { t.Fatalf("se.locality[Sandby] = %v, want its keys listed plainly", err) } } + +// TestRowSetSimulatesPinning proves the load-time walk's rowSet answers what +// (*draws).pin and pinned answer at render time: the rows one entry brings with it, +// the cells that entry rules out, and which two entries are alternatives. +func TestRowSetSimulatesPinning(t *testing.T) { + f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) + var tables []*table + for _, category := range []string{"region", "municipality", "locality"} { + tbl, isTable := f.categories[category].(*table) + if !isTable { + t.Fatalf("%s is not a table", category) + } + tables = append(tables, tbl) + } + for _, entered := range tables { + for row := 0; row < entered.rows(); row++ { + s := rowSet(nil).enter(entered, row) + var pinned draws + pinned.pin(entered, row) + assertRowSetAgrees(t, s, pinned, entered, row, tables) + } + } +} + +// assertRowSetAgrees compares the set entered with row of entered against the same +// row pinned, over every row of tables. +func assertRowSetAgrees(t *testing.T, s rowSet, pinned draws, entered *table, row int, tables []*table) { + t.Helper() + pinned.each(func(tbl *table, r int) { + got, in := s.rowOf(tbl) + if !in || got != r { + t.Errorf("entering %s leaves %s unset in the row set, though pinning it pins %s", entered.selectorSpelling(row), tbl.path, tbl.selectorSpelling(r)) + } + }) + for _, cand := range tables { + for cr := 0; cr < cand.rows(); cr++ { + var beside draws + beside.pin(entered, row) + refused := beside.pinRow(cand, cr) != nil + if got := s.excludes(&template{cellOf: cand, cellRow: cr}); got != refused { + t.Errorf("excludes(%s) = %v beside %s, but pinRow refuses it = %v", cand.selectorSpelling(cr), got, entered.selectorSpelling(row), refused) + } + a := drawAt{alt: s} + b := drawAt{alt: rowSet(nil).enter(cand, cr)} + if got := alternatives(a, b); got != refused { + t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) + } + } + } +} + +// TestCellsOfRowsThatNeverMeetAreNotCompared pins what the entered ancestors buy: a +// cell of region 01 and a cell of a municipality of region 12 never render together, +// so the reads inside them are not one render's to reconcile. +func TestCellsOfRowsThatNeverMeetAreNotCompared(t *testing.T) { + files := with(geo(), map[string]string{ + "addr.json": `{"format":"{city}","city":"Lund"}`, + "municipality.tsv": "code\tname\tregion\tpopulation\tnote\n0180\tStockholm\t01\t980000\t-\n0184\tSolna\t01\t85000\t-\n1280\tMalmö\t12\t360000\t{/addr.city}\n1281\tLund\t12\t130000\t-\n1480\tGöteborg\t14\t590000\t-\n", + "region.tsv": "code\tname\tpopulation\tnote\n01\tStockholms län\t2400000\t{/addr}\n12\tSkåne län\t1400000\t-\n14\tVästra Götalands län\t1750000\t-\n", + "x.json": `"{/region.note} {/municipality.note}"`, + }) + f, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, files)), WithSeed(1)) + if err != nil { + t.Fatalf("New = %v, want the two cells accepted", err) + } + fake(t, f, "x") +} -- 2.52.0 From 42fd72815675642277432674a3ced37640b24664 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:27:31 +0200 Subject: [PATCH 026/966] Enter the ancestor rows the render pins into the walk's row set --- draw.go | 18 +++++++++++++++--- todo.md | 4 ---- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/draw.go b/draw.go index b9cdd7d..21d0d55 100644 --- a/draw.go +++ b/draw.go @@ -301,7 +301,8 @@ type drawAt struct { // rowSet is the rows a walk entered, one per table: only one row of a table renders, so reads in // two rows of one table never meet, while reads in one row, across its columns and whatever they -// reach, do. +// reach, do. It simulates what the render pins: enter is (*draws).pin and rowOf is its pinned, and +// TestRowSetSimulatesPinning holds the two to one answer. type rowSet []tablePin func (s rowSet) rowOf(t *table) (int, bool) { @@ -313,12 +314,23 @@ func (s rowSet) rowOf(t *table) (int, bool) { return 0, false } -// enter is s with row r of t, where t is not in it yet; the set is copied, since walks branch. +// enter is s with row r of t and the ancestor rows it links to, where t is not in it yet; the set +// is copied, since walks branch. func (s rowSet) enter(t *table, r int) rowSet { if _, in := s.rowOf(t); in { return s } - out := append(append(make(rowSet, 0, len(s)+1), s...), tablePin{t, r}) + out := append(make(rowSet, 0, len(s)+1), s...) + for { + if _, in := out.rowOf(t); in { + break + } + out = append(out, tablePin{t, r}) + if t.parentT == nil { + break + } + t, r = t.parentT, t.parentRow(r) + } sort.Slice(out, func(i, j int) bool { return out[i].t.category < out[j].t.category }) return out } diff --git a/todo.md b/todo.md index a774a92..b43f905 100644 --- a/todo.md +++ b/todo.md @@ -21,10 +21,6 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- State on `rowSet` that it simulates `(*draws).pin` and `rowOf`, and prove the two - agree with a test: the load-time walk reimplements the render-time pinning, nothing - catches them drifting, and the failure is data that loads and then renders a family - that disagrees. - Put `draws` and its methods in one file: it is declared in `hold.go:262` and all eight of its methods sit in `family.go`, so a jump to the type lands where none of the behaviour is. -- 2.52.0 From 4d45e3ebdc1d120c8a463e049900fa3651638ff9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:39:19 +0200 Subject: [PATCH 027/966] Close the row set over ancestors only where the render pins the row --- CHANGELOG.md | 4 ++- README.md | 5 ++-- draw.go | 33 +++++++++++--------- table_test.go | 83 ++++++++++++++++++++++++++++----------------------- 4 files changed, 70 insertions(+), 55 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index de8b1c8..5556b4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,9 @@ replacement, and each removed path, column or flag. `parent`, a name repeating inside one parent row, a name spelling another row's key, a table named like a column of any table above it, a table whose format or cell references its own family, and, within one render and draw group, a path drawing a table another path selects a - row of, or two paths pinning different rows of one table. + row of, or two paths pinning different rows of one table. Two reads are held to one + draw only where a render can pin both their rows: a path read pins a row's ancestors + with it, while a table read whole draws a row and pins none. - `New` refuses a root choice of templates sharing one format and one set of string fields, naming the rows TSV to write instead. - `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and diff --git a/README.md b/README.md index feddb0d..cb29fa1 100644 --- a/README.md +++ b/README.md @@ -1140,8 +1140,9 @@ the Development section below, and who ships a register the four above then draw - **The rows of a table are alternatives.** Only one row renders, so a cell in one row and a cell in another never meet, and each may select its own row of another table; the cells of one row, and whatever they reach, do meet, and so does the - format beside them. The rule the fence applies is that two reads meet unless - they sit in two rows of one table, or in a row outside a selected ancestor's. A + format beside them. The rule the fence applies is that two reads meet unless they + sit in two rows of one table, or in rows no render pins at once: a path read pins a + row's ancestors with it, while a table read whole draws a row and pins none. A choice's items get no such treatment yet: two items selecting different rows are still refused. - **A table never reaches its own family, by any route.** A `repeat` iteration and a diff --git a/draw.go b/draw.go index 21d0d55..8f4a893 100644 --- a/draw.go +++ b/draw.go @@ -291,18 +291,20 @@ type drawWalk struct { seen map[drawVisit]bool } -// drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, and -// the table rows it entered. +// drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the +// table rows it entered, and the table a whole read draws a row of. type drawAt struct { group string route drawRoute alt rowSet + drawn *table } // rowSet is the rows a walk entered, one per table: only one row of a table renders, so reads in // two rows of one table never meet, while reads in one row, across its columns and whatever they -// reach, do. It simulates what the render pins: enter is (*draws).pin and rowOf is its pinned, and -// TestRowSetSimulatesPinning holds the two to one answer. +// reach, do. It stands in for the rows the render pins, so a row a path read pins enters with the +// ancestors (*draws).pin pins alongside it, while a table read whole is drawn a row of and pins +// none. type rowSet []tablePin func (s rowSet) rowOf(t *table) (int, bool) { @@ -314,24 +316,24 @@ func (s rowSet) rowOf(t *table) (int, bool) { return 0, false } -// enter is s with row r of t and the ancestor rows it links to, where t is not in it yet; the set -// is copied, since walks branch. -func (s rowSet) enter(t *table, r int) rowSet { +// enter is s with row r of t, where t is not in it yet, and, where pins says the render pins that +// row rather than draws it, the ancestor rows it links to. The set is copied, since walks branch. +func (s rowSet) enter(t *table, r int, pins bool) rowSet { if _, in := s.rowOf(t); in { return s } out := append(make(rowSet, 0, len(s)+1), s...) for { - if _, in := out.rowOf(t); in { - break - } out = append(out, tablePin{t, r}) - if t.parentT == nil { + if !pins || t.parentT == nil { break } t, r = t.parentT, t.parentRow(r) + if _, in := out.rowOf(t); in { + break + } } - sort.Slice(out, func(i, j int) bool { return out[i].t.category < out[j].t.category }) + sort.Slice(out, func(i, j int) bool { return out[i].t.path < out[j].t.path }) return out } @@ -394,12 +396,15 @@ func (w *drawWalk) walk(n node, at drawAt) { if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" { at.group = t.drawGroupKey } + if t, isTable := n.(*table); isTable { + at.drawn = t + } for _, e := range renderEdges(n) { if cell, isCell := e.to.(*template); isCell && cell.cellOf != nil { if at.alt.excludes(cell) { continue } - w.edge(n, e, drawAt{at.group, at.route, at.alt.enter(cell.cellOf, cell.cellRow)}) + w.edge(n, e, drawAt{at.group, at.route, at.alt.enter(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn), at.drawn}) continue } w.edge(n, e, at) @@ -430,7 +435,7 @@ func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { w.reads = append(w.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { at.alt = at.alt.enter(t, r) }) + tr.pins.each(func(t *table, r int) { at.alt = at.alt.enter(t, r, true) }) } } w.walk(e.to, at) diff --git a/table_test.go b/table_test.go index 4ffbce7..feb939a 100644 --- a/table_test.go +++ b/table_test.go @@ -964,9 +964,9 @@ func TestAmbiguousNameNamesARunnablePath(t *testing.T) { } } -// TestRowSetSimulatesPinning proves the load-time walk's rowSet answers what -// (*draws).pin and pinned answer at render time: the rows one entry brings with it, -// the cells that entry rules out, and which two entries are alternatives. +// TestRowSetSimulatesPinning proves the walk's row set answers what the render's pinning +// answers: a path read pins a row and the ancestors it links to, a whole read draws a row +// and pins none. func TestRowSetSimulatesPinning(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) var tables []*table @@ -979,54 +979,61 @@ func TestRowSetSimulatesPinning(t *testing.T) { } for _, entered := range tables { for row := 0; row < entered.rows(); row++ { - s := rowSet(nil).enter(entered, row) + if drawn := rowSet(nil).enter(entered, row, false); len(drawn) != 1 || drawn[0] != (tablePin{entered, row}) { + t.Errorf("a drawn %s enters %v, want its own row alone", entered.selectorSpelling(row), drawn) + } + s := rowSet(nil).enter(entered, row, true) var pinned draws + held := 0 pinned.pin(entered, row) - assertRowSetAgrees(t, s, pinned, entered, row, tables) - } - } -} - -// assertRowSetAgrees compares the set entered with row of entered against the same -// row pinned, over every row of tables. -func assertRowSetAgrees(t *testing.T, s rowSet, pinned draws, entered *table, row int, tables []*table) { - t.Helper() - pinned.each(func(tbl *table, r int) { - got, in := s.rowOf(tbl) - if !in || got != r { - t.Errorf("entering %s leaves %s unset in the row set, though pinning it pins %s", entered.selectorSpelling(row), tbl.path, tbl.selectorSpelling(r)) - } - }) - for _, cand := range tables { - for cr := 0; cr < cand.rows(); cr++ { - var beside draws - beside.pin(entered, row) - refused := beside.pinRow(cand, cr) != nil - if got := s.excludes(&template{cellOf: cand, cellRow: cr}); got != refused { - t.Errorf("excludes(%s) = %v beside %s, but pinRow refuses it = %v", cand.selectorSpelling(cr), got, entered.selectorSpelling(row), refused) + pinned.each(func(tbl *table, r int) { + held++ + if got, in := s.rowOf(tbl); !in || got != r { + t.Errorf("entering %s leaves %s unset, though pinning it pins %s", entered.selectorSpelling(row), tbl.path, tbl.selectorSpelling(r)) + } + }) + if held != len(s) { + t.Errorf("entering %s holds %d rows, though pinning it pins %d", entered.selectorSpelling(row), len(s), held) } - a := drawAt{alt: s} - b := drawAt{alt: rowSet(nil).enter(cand, cr)} - if got := alternatives(a, b); got != refused { - t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) + for _, cand := range tables { + for cr := 0; cr < cand.rows(); cr++ { + var beside draws + beside.pin(entered, row) + refused := beside.pinRow(cand, cr) != nil + if got := s.excludes(&template{cellOf: cand, cellRow: cr}); got != refused { + t.Errorf("excludes(%s) = %v beside %s, but pinRow refuses it = %v", cand.selectorSpelling(cr), got, entered.selectorSpelling(row), refused) + } + if got := alternatives(drawAt{alt: s}, drawAt{alt: rowSet(nil).enter(cand, cr, true)}); got != refused { + t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) + } + } } } } } -// TestCellsOfRowsThatNeverMeetAreNotCompared pins what the entered ancestors buy: a -// cell of region 01 and a cell of a municipality of region 12 never render together, -// so the reads inside them are not one render's to reconcile. -func TestCellsOfRowsThatNeverMeetAreNotCompared(t *testing.T) { - files := with(geo(), map[string]string{ - "addr.json": `{"format":"{city}","city":"Lund"}`, +// TestCellReadsMeetWhereTheRenderPairsTheRows pins which cells' reads are one render's to +// reconcile: a path read pins the row's ancestors, so a cell of region 01 and a cell of a +// municipality of region 12 never render together, while a whole read draws its row and pins +// no ancestor, so those same two cells do. +func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { + cells := with(geo(), map[string]string{ + "addr.json": `{"format":"{city}","city":["Lund","Malmö"]}`, "municipality.tsv": "code\tname\tregion\tpopulation\tnote\n0180\tStockholm\t01\t980000\t-\n0184\tSolna\t01\t85000\t-\n1280\tMalmö\t12\t360000\t{/addr.city}\n1281\tLund\t12\t130000\t-\n1480\tGöteborg\t14\t590000\t-\n", "region.tsv": "code\tname\tpopulation\tnote\n01\tStockholms län\t2400000\t{/addr}\n12\tSkåne län\t1400000\t-\n14\tVästra Götalands län\t1750000\t-\n", - "x.json": `"{/region.note} {/municipality.note}"`, }) - f, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, files)), WithSeed(1)) + paths := with(cells, map[string]string{"x.json": `"{/region.note} {/municipality.note}"`}) + f, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, paths)), WithSeed(1)) if err != nil { t.Fatalf("New = %v, want the two cells accepted", err) } fake(t, f, "x") + whole := with(cells, map[string]string{ + "municipality.json": `{"format":"{name}={note}","rows":"municipality.tsv","key":"code","name":"name","parent":"region","weight":"population"}`, + "region.json": `{"format":"{name}={note}","rows":"region.tsv","key":"code","name":"name","weight":"population"}`, + "x.json": `"{/region}|{/municipality}"`, + }) + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "reads a path into") { + t.Fatalf("New = %v, want the two drawn rows held to one draw of addr", err) + } } -- 2.52.0 From f7f49607553d63d3ddc045fcefa10008bca17807 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:44:32 +0200 Subject: [PATCH 028/966] Say the fence rule without overloading pin, and file the twice-drawn table --- CHANGELOG.md | 4 ++-- README.md | 4 ++-- draw.go | 6 ++++-- todo.md | 4 ++++ 4 files changed, 12 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5556b4c..1497e03 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,8 +19,8 @@ replacement, and each removed path, column or flag. table above it, a table whose format or cell references its own family, and, within one render and draw group, a path drawing a table another path selects a row of, or two paths pinning different rows of one table. Two reads are held to one - draw only where a render can pin both their rows: a path read pins a row's ancestors - with it, while a table read whole draws a row and pins none. + draw only where both their rows can render together: a path read pins a row's + ancestors with it, while a table read whole draws a row and pins none. - `New` refuses a root choice of templates sharing one format and one set of string fields, naming the rows TSV to write instead. - `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and diff --git a/README.md b/README.md index cb29fa1..bdec463 100644 --- a/README.md +++ b/README.md @@ -1141,8 +1141,8 @@ the Development section below, and who ships a register the four above then draw row and a cell in another never meet, and each may select its own row of another table; the cells of one row, and whatever they reach, do meet, and so does the format beside them. The rule the fence applies is that two reads meet unless they - sit in two rows of one table, or in rows no render pins at once: a path read pins a - row's ancestors with it, while a table read whole draws a row and pins none. A + sit in two rows of one table, or in rows that never render together: a path read + pins a row's ancestors with it, while a table read whole draws a row and pins none. A choice's items get no such treatment yet: two items selecting different rows are still refused. - **A table never reaches its own family, by any route.** A `repeat` iteration and a diff --git a/draw.go b/draw.go index 8f4a893..7e7726a 100644 --- a/draw.go +++ b/draw.go @@ -297,7 +297,7 @@ type drawAt struct { group string route drawRoute alt rowSet - drawn *table + drawn *table // only that table's own cells read it, which the own-family fence is what keeps true, so the visit keys leave it out } // rowSet is the rows a walk entered, one per table: only one row of a table renders, so reads in @@ -404,7 +404,9 @@ func (w *drawWalk) walk(n node, at drawAt) { if at.alt.excludes(cell) { continue } - w.edge(n, e, drawAt{at.group, at.route, at.alt.enter(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn), at.drawn}) + in := at + in.alt = at.alt.enter(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn) + w.edge(n, e, in) continue } w.edge(n, e, at) diff --git a/todo.md b/todo.md index b43f905..f42d777 100644 --- a/todo.md +++ b/todo.md @@ -153,6 +153,10 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Hold two whole reads of one table to one row, or refuse the pair: `{/region}|{/region}` + draws a row for each read, so cells of two rows render together while the fence counts + them alternatives and compares neither — a bare `{/addr}` in one row beside + `{/addr.city}` in another is the overlap it refuses everywhere else. - Name a spelling that works when a row selector misses: `misc.protocol[tcp]`, `misc.httpmethod[get]` and `misc.territory[se]` all answer "no row … has key or name" and stop there, where a case-insensitive match could name the row that exists, and a -- 2.52.0 From 1fdf23ecbd068b907670464e10fd7d0a92f502cb Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:50:47 +0200 Subject: [PATCH 029/966] State the pinned and whole rule once, and file what the errors leave unsaid --- CHANGELOG.md | 6 +++--- README.md | 13 +++++++------ todo.md | 24 ++++++++++++++++++++---- 3 files changed, 30 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1497e03..cdf85f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,9 +18,9 @@ replacement, and each removed path, column or flag. `parent`, a name repeating inside one parent row, a name spelling another row's key, a table named like a column of any table above it, a table whose format or cell references its own family, and, within one render and draw group, a path drawing a table another path selects a - row of, or two paths pinning different rows of one table. Two reads are held to one - draw only where both their rows can render together: a path read pins a row's - ancestors with it, while a table read whole draws a row and pins none. + row of, or two paths pinning different rows of one table. A read inside a cell is + weighed against another only where both rows can render together, which two rows of + one table, or rows under two different rows of an ancestor a path pinned, never can. - `New` refuses a root choice of templates sharing one format and one set of string fields, naming the rows TSV to write instead. - `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and diff --git a/README.md b/README.md index bdec463..502c753 100644 --- a/README.md +++ b/README.md @@ -1103,9 +1103,10 @@ the Development section below, and who ships a register the four above then draw a template beside the family under a selected row, is refused today; admitting it later is additive, since a refused spelling gains a meaning and no accepted one changes, so the door stays open for the address records the plan describes. -- **A table read into is pinned; a table rendered whole draws afresh.** A path into a - table pins its row for the render and group, as a reference path pins its level, - and a bare `{/city}` draws each time, as a bare reference does; so a bare table +- **A table read into is pinned; a table read whole draws afresh.** A path into a + table pins its row, and the ancestor rows it links to, for the render and group, as a + reference path pins its level, and a bare `{/city}` draws a row each time and pins no + row of its family, as a bare reference does; so a bare table beside a path into its family is refused like a bare reference beside a path into it. A bare table reference still counts as a read for a `drawGroup`, since the group is what draws it apart from the family's pins. @@ -1140,9 +1141,9 @@ the Development section below, and who ships a register the four above then draw - **The rows of a table are alternatives.** Only one row renders, so a cell in one row and a cell in another never meet, and each may select its own row of another table; the cells of one row, and whatever they reach, do meet, and so does the - format beside them. The rule the fence applies is that two reads meet unless they - sit in two rows of one table, or in rows that never render together: a path read - pins a row's ancestors with it, while a table read whole draws a row and pins none. A + format beside them. The rule the fence applies is that two reads meet unless they sit + in rows that never render together: two rows of one table, or rows under two different + rows of an ancestor a path pinned. A choice's items get no such treatment yet: two items selecting different rows are still refused. - **A table never reaches its own family, by any route.** A `repeat` iteration and a diff --git a/todo.md b/todo.md index f42d777..a49dbf9 100644 --- a/todo.md +++ b/todo.md @@ -153,10 +153,22 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI -- Hold two whole reads of one table to one row, or refuse the pair: `{/region}|{/region}` - draws a row for each read, so cells of two rows render together while the fence counts - them alternatives and compares neither — a bare `{/addr}` in one row beside - `{/addr.city}` in another is the overlap it refuses everywhere else. +- Refuse two whole reads of one table family in one render, which goal 5 promises is a + load error: `{/sel}|{/sel}` panics out of `Fake` with "a fence should have refused this + at New" where the two draws land on rows whose cells select different rows of another + table, and `{/geo.SE.locality}|{/geo.SE.municipality}` renders a locality outside the + municipality beside it. Each read draws its own row, so cells of two rows render + together, where the fence counts them alternatives and compares neither. +- Name the cell a refused read sits in: where both reads come from TSV cells the overlap + error says `{area} with {/addr} renders a level that {where} with {/addr.city} reads a + path into`, naming neither table, row nor column, so the author greps a register-sized + file — and "name the fields you want instead" rewrites a cell that is not the one to fix. +- Stop the family fence naming a rewrite that cannot work: where the drawn table's rows + select more than one row of the family, every `write {/terr[NO].name}` it offers fails + the same way, and only the `drawGroup` it also names is left. +- Spell a table one way in an error: `family.go` names `drawn.category` in one message and + `t.path` in the next, so one table is `territory` and `misc.territory`, and the short + spelling names no file where two folders hold that name. - Name a spelling that works when a row selector misses: `misc.protocol[tcp]`, `misc.httpmethod[get]` and `misc.territory[se]` all answer "no row … has key or name" and stop there, where a case-insensitive match could name the row that exists, and a @@ -172,6 +184,10 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle +- Decide whether the README's Audience names the author who writes categories under + `--data-path`: the four personas listed consume the shipped set and the contributor + ships a register upstream, while "Your own data" is a README section, `-d` is a + first-class flag, and every load error the draw fences raise lands on that author. - Decide whether `data/misc`'s 33 flat files gain a level before v1.0.0: a folder is a path segment, so `misc/net/tld` is a rename a consumer pays for, and goal 13 promises the directory keeps growing. -- 2.52.0 From ad7a62b67d05256b4a55d0ea5c63f72ad9a37968 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:56:19 +0200 Subject: [PATCH 030/966] Bound the alternatives claim to one read, and point the grammar at it --- CHANGELOG.md | 6 +++--- README.md | 26 +++++++++++++------------- todo.md | 15 ++++++++++----- 3 files changed, 26 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cdf85f5..ded2c19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,9 +18,9 @@ replacement, and each removed path, column or flag. `parent`, a name repeating inside one parent row, a name spelling another row's key, a table named like a column of any table above it, a table whose format or cell references its own family, and, within one render and draw group, a path drawing a table another path selects a - row of, or two paths pinning different rows of one table. A read inside a cell is - weighed against another only where both rows can render together, which two rows of - one table, or rows under two different rows of an ancestor a path pinned, never can. + row of, or two paths pinning different rows of one table. Two cells are weighed + against each other only where they can render together: not two rows of one table, and + not rows under different rows of a table a path pinned. - `New` refuses a root choice of templates sharing one format and one set of string fields, naming the rows TSV to write instead. - `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and diff --git a/README.md b/README.md index 502c753..f215461 100644 --- a/README.md +++ b/README.md @@ -546,7 +546,8 @@ to, a chain of parents that closes, a table named like a column of any table abo it, and a cell or format of a table that references a table of its own family, through any template, a `repeat` or a `drawGroup` included, since a row rendered whole would draw the family apart from itself: read the family from a template beside it, or add -the value as a column. +the value as a column. Which reads a render must reconcile, a cell's included, is +under [Decisions](#decisions). ### Options and fields @@ -1106,10 +1107,10 @@ the Development section below, and who ships a register the four above then draw - **A table read into is pinned; a table read whole draws afresh.** A path into a table pins its row, and the ancestor rows it links to, for the render and group, as a reference path pins its level, and a bare `{/city}` draws a row each time and pins no - row of its family, as a bare reference does; so a bare table - beside a path into its family is refused like a bare reference beside a path into - it. A bare table reference still counts as a read for a `drawGroup`, since the group - is what draws it apart from the family's pins. + row of its family, as a bare reference does; so a bare table beside a path into its + family is refused like a bare reference beside a path into it. A bare table reference + still counts as a read for a `drawGroup`, since the group is what draws it apart from + the family's pins. - **Every reference path into one family selects the same rows, per render and group.** A selector pins rows for the render, and a read that draws freely before it could pin a row the selector contradicts, so accepting both would make the result @@ -1138,14 +1139,13 @@ the Development section below, and who ships a register the four above then draw format and options, at the cost of holding the rows twice, which is what a category over a register with two natural formats asks for; a TSV nothing names stays a load error, since that one is a file forgotten rather than shared. -- **The rows of a table are alternatives.** Only one row renders, so a cell in one - row and a cell in another never meet, and each may select its own row of another - table; the cells of one row, and whatever they reach, do meet, and so does the - format beside them. The rule the fence applies is that two reads meet unless they sit - in rows that never render together: two rows of one table, or rows under two different - rows of an ancestor a path pinned. A - choice's items get no such treatment yet: two items selecting different rows - are still refused. +- **The rows of a table are alternatives.** Only one row renders per read, so within one + read a cell in one row and a cell in another never meet, and each may select its own + row of another table; the cells of one row, and whatever they reach, do meet, and so + does the format beside them. The rule the fence applies is that two reads meet unless + they sit in rows that never render together: rows falling under different rows of a + table a path pinned, two rows of that table itself included. A choice's items get no + such treatment yet: two items selecting different rows are still refused. - **A table never reaches its own family, by any route.** A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row that lists three localities from other regions is the output the family exists to prevent, so the own-family fence diff --git a/todo.md b/todo.md index a49dbf9..e768f7c 100644 --- a/todo.md +++ b/todo.md @@ -158,11 +158,13 @@ by a `data-import/` script, as README goal 12 asks. at New" where the two draws land on rows whose cells select different rows of another table, and `{/geo.SE.locality}|{/geo.SE.municipality}` renders a locality outside the municipality beside it. Each read draws its own row, so cells of two rows render - together, where the fence counts them alternatives and compares neither. -- Name the cell a refused read sits in: where both reads come from TSV cells the overlap - error says `{area} with {/addr} renders a level that {where} with {/addr.city} reads a - path into`, naming neither table, row nor column, so the author greps a register-sized - file — and "name the fields you want instead" rewrites a cell that is not the one to fix. + together, where the fence counts them alternatives and compares neither. Have the error + name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`, + as the README's rule on a rejected spelling asks. +- Name the row a refused read sits in: where both reads come from TSV cells the overlap + error names the category and the two columns but no row, so the author greps a + register-sized file for the cell — and "name the fields you want instead" rewrites a + cell that is not the one to fix. - Stop the family fence naming a rewrite that cannot work: where the drawn table's rows select more than one row of the family, every `write {/terr[NO].name}` it offers fails the same way, and only the `drawGroup` it also names is left. @@ -184,6 +186,9 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle +- Decide whether a goal 5 violation joins the comprehension gate's exception list, beside + a security fix and a dependency bump: the panic above waits behind the whole round at + 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile. - Decide whether the README's Audience names the author who writes categories under `--data-path`: the four personas listed consume the shipped set and the contributor ships a register upstream, while "Your own data" is a README section, `-d` is a -- 2.52.0 From d3f7207e742eb49dd9f49003da7320fef1a4da89 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 16:59:41 +0200 Subject: [PATCH 031/966] Cut the comments that restate the decision they sit under --- CHANGELOG.md | 4 ++-- README.md | 6 +++--- draw.go | 13 +++++-------- table_test.go | 12 +++++------- todo.md | 24 +++++++++++++----------- 5 files changed, 28 insertions(+), 31 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ded2c19..0dae6ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,8 +19,8 @@ replacement, and each removed path, column or flag. table above it, a table whose format or cell references its own family, and, within one render and draw group, a path drawing a table another path selects a row of, or two paths pinning different rows of one table. Two cells are weighed - against each other only where they can render together: not two rows of one table, and - not rows under different rows of a table a path pinned. + against each other only where they can render together: never two rows of one table, + nor rows under different rows of a table a path pinned. - `New` refuses a root choice of templates sharing one format and one set of string fields, naming the rows TSV to write instead. - `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and diff --git a/README.md b/README.md index f215461..de62a04 100644 --- a/README.md +++ b/README.md @@ -546,8 +546,8 @@ to, a chain of parents that closes, a table named like a column of any table abo it, and a cell or format of a table that references a table of its own family, through any template, a `repeat` or a `drawGroup` included, since a row rendered whole would draw the family apart from itself: read the family from a template beside it, or add -the value as a column. Which reads a render must reconcile, a cell's included, is -under [Decisions](#decisions). +the value as a column. A cell's reads are the render's too; which pairs are weighed +against each other is under [Decisions](#decisions). ### Options and fields @@ -1105,7 +1105,7 @@ the Development section below, and who ships a register the four above then draw later is additive, since a refused spelling gains a meaning and no accepted one changes, so the door stays open for the address records the plan describes. - **A table read into is pinned; a table read whole draws afresh.** A path into a - table pins its row, and the ancestor rows it links to, for the render and group, as a + table pins its row for the render and group, as a reference path pins its level, and a bare `{/city}` draws a row each time and pins no row of its family, as a bare reference does; so a bare table beside a path into its family is refused like a bare reference beside a path into it. A bare table reference diff --git a/draw.go b/draw.go index 7e7726a..daa1e61 100644 --- a/draw.go +++ b/draw.go @@ -297,14 +297,11 @@ type drawAt struct { group string route drawRoute alt rowSet - drawn *table // only that table's own cells read it, which the own-family fence is what keeps true, so the visit keys leave it out + drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true } -// rowSet is the rows a walk entered, one per table: only one row of a table renders, so reads in -// two rows of one table never meet, while reads in one row, across its columns and whatever they -// reach, do. It stands in for the rows the render pins, so a row a path read pins enters with the -// ancestors (*draws).pin pins alongside it, while a table read whole is drawn a row of and pins -// none. +// rowSet stands in at load for the rows (*draws).pin holds at render: a path read enters a row with +// its ancestors, a whole read enters the drawn row alone. type rowSet []tablePin func (s rowSet) rowOf(t *table) (int, bool) { @@ -316,8 +313,8 @@ func (s rowSet) rowOf(t *table) (int, bool) { return 0, false } -// enter is s with row r of t, where t is not in it yet, and, where pins says the render pins that -// row rather than draws it, the ancestor rows it links to. The set is copied, since walks branch. +// enter is s with row r of t, and with t's ancestors where the render pins that row rather than +// drawing it; the set is copied, since walks branch. func (s rowSet) enter(t *table, r int, pins bool) rowSet { if _, in := s.rowOf(t); in { return s diff --git a/table_test.go b/table_test.go index feb939a..2f871ed 100644 --- a/table_test.go +++ b/table_test.go @@ -964,9 +964,8 @@ func TestAmbiguousNameNamesARunnablePath(t *testing.T) { } } -// TestRowSetSimulatesPinning proves the walk's row set answers what the render's pinning -// answers: a path read pins a row and the ancestors it links to, a whole read draws a row -// and pins none. +// TestRowSetSimulatesPinning holds the walk's row set to the render's pinning: drift between +// the two is data that loads and then renders a family that disagrees. func TestRowSetSimulatesPinning(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) var tables []*table @@ -1012,10 +1011,9 @@ func TestRowSetSimulatesPinning(t *testing.T) { } } -// TestCellReadsMeetWhereTheRenderPairsTheRows pins which cells' reads are one render's to -// reconcile: a path read pins the row's ancestors, so a cell of region 01 and a cell of a -// municipality of region 12 never render together, while a whole read draws its row and pins -// no ancestor, so those same two cells do. +// TestCellReadsMeetWhereTheRenderPairsTheRows reads what the TSVs below do not show: region 01's +// note holds {/addr} and 1280's, under region 12, holds {/addr.city}, so the two meet only where +// no read pins a region. func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { cells := with(geo(), map[string]string{ "addr.json": `{"format":"{city}","city":["Lund","Malmö"]}`, diff --git a/todo.md b/todo.md index e768f7c..edebcc5 100644 --- a/todo.md +++ b/todo.md @@ -6,8 +6,8 @@ Ordered as the releases that ship it. ### Comprehension, until the panel scores 7.0 -Nothing else ships while the score is under 7.0 — no feature, no category, no data — -bar a security fix and a dependency bump, per README goal 2. This section is one +Nothing else ships while the score is under 7.0 — no feature, no category, no data — bar +what `AGENTS.md` excepts, per README goal 2. This section is one round, replayed: the `comprehension-panel` skill's nine-seat findings run at depth 1, a PR per item it names, then the run again, until the score passes. The run is the nine-seat one: the round is meant to buy architectural change, and the four-seat @@ -165,12 +165,13 @@ by a `data-import/` script, as README goal 12 asks. error names the category and the two columns but no row, so the author greps a register-sized file for the cell — and "name the fields you want instead" rewrites a cell that is not the one to fix. -- Stop the family fence naming a rewrite that cannot work: where the drawn table's rows - select more than one row of the family, every `write {/terr[NO].name}` it offers fails - the same way, and only the `drawGroup` it also names is left. -- Spell a table one way in an error: `family.go` names `drawn.category` in one message and - `t.path` in the next, so one table is `territory` and `misc.territory`, and the short - spelling names no file where two folders hold that name. +- Have the family fence offer only a rewrite that works: where the drawn table's rows select + more than one row of the family, every `write {/terr[NO].name}` it offers fails the same + way, so only the `drawGroup` it also names is left. +- Spell a table one way across `family.go`'s errors: `checkFamilyPair` names + `drawn.category`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one + table is `territory` and `misc.territory`, and the short spelling names no file where two + folders hold that name. - Name a spelling that works when a row selector misses: `misc.protocol[tcp]`, `misc.httpmethod[get]` and `misc.territory[se]` all answer "no row … has key or name" and stop there, where a case-insensitive match could name the row that exists, and a @@ -186,9 +187,10 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide whether a goal 5 violation joins the comprehension gate's exception list, beside - a security fix and a dependency bump: the panic above waits behind the whole round at - 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile. +- Decide whether a goal 5 violation joins the comprehension gate's exception list: the + `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and + `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a + repair restoring behaviour the repo already claims may already pass the gate as written. - Decide whether the README's Audience names the author who writes categories under `--data-path`: the four personas listed consume the shipped set and the contributor ships a register upstream, while "Your own data" is a README section, `-d` is a -- 2.52.0 From 35c4dc515080327d18a674270cb8a5957669d799 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 17:35:03 +0200 Subject: [PATCH 032/966] Put draws and its methods in one file --- README.md | 4 +- family.go | 130 ------------------------------------------------------ hold.go | 129 +++++++++++++++++++++++++++++++++++++++++++++++++++++ todo.md | 5 +-- 4 files changed, 132 insertions(+), 136 deletions(-) diff --git a/README.md b/README.md index de62a04..6d80e2e 100644 --- a/README.md +++ b/README.md @@ -1328,9 +1328,9 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold: one draw per expansion for paths and operands, and its fences +hold.go the hold: one draw per expansion for paths and operands, the rows a render pins, and its fences draw.go one reference draw per render and group: draw sets, the group option, and its fence -family.go a family of linked tables: the rows a render pins, and the fence over paths into one family +family.go a family of linked tables: what a reference path reads of one, and the fence over paths into it reference.go reference sigils, and binding references across the tree graph.go the render graph: edges, cycles, the repeat bound, tree walks builtins.go the {name()} function registry and its implementations diff --git a/family.go b/family.go index d984a7d..73443e4 100644 --- a/family.go +++ b/family.go @@ -2,139 +2,9 @@ package fejkdata import ( "fmt" - "sort" "strings" ) -// tablePin is one table's pinned row. -type tablePin struct { - t *table - row int -} - -// pinned is the row the render pinned for t, if any. -func (d *draws) pinned(t *table) (int, bool) { - for _, p := range d.pins[:d.npins] { - if p.t == t { - return p.row, true - } - } - r, ok := d.more[t] - return r, ok -} - -// mustRow is the row pinned for t, which the walk reaching a column pinned. -func (d *draws) mustRow(t *table) int { - r, ok := d.pinned(t) - if !ok { - panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) - } - return r -} - -// pin pins row r of t, and the rows of t's ancestors it links to. -func (d *draws) pin(t *table, r int) { - for { - if _, done := d.pinned(t); done { - return - } - if d.npins < len(d.pins) { - d.pins[d.npins] = tablePin{t, r} - d.npins++ - } else { - if d.more == nil { - d.more = map[*table]int{} - } - d.more[t] = r - } - if t.parentT == nil { - return - } - t, r = t.parentT, t.parentRow(r) - } -} - -// each calls fn for every pinned row, in pin order, the spilled ones by table name. -func (d *draws) each(fn func(t *table, r int)) { - for _, p := range d.pins[:d.npins] { - fn(p.t, p.row) - } - spilled := make([]*table, 0, len(d.more)) - for t := range d.more { - spilled = append(spilled, t) - } - sort.Slice(spilled, func(i, j int) bool { return spilled[i].category < spilled[j].category }) - for _, t := range spilled { - fn(t, d.more[t]) - } -} - -// rowOf is the render's row of t: the one pinned, else one drawn inside the -// nearest pinned ancestor — its parent drawn inside that first where the ancestor -// is further up — or over the whole table, and pinned with its ancestors. -func (d *draws) rowOf(t *table) int { - if r, ok := d.pinned(t); ok { - return r - } - r := -1 - for a := t.parentT; a != nil && r < 0; a = a.parentT { - if _, ok := d.pinned(a); !ok { - continue - } - if t.parentT != a { - d.rowOf(t.parentT) - } - pr, _ := d.pinned(t.parentT) - r = t.drawUnder(d.s, pr) - } - if r < 0 { - r = t.draw(d.s) - } - d.pin(t, r) - return r -} - -// pinRow pins row r of t where it agrees with the rows pinned before it. -func (d *draws) pinRow(t *table, r int) error { - if pr, ok := d.pinned(t); ok && pr != r { - return fmt.Errorf("%s and %s are two rows of %s", t.selectorSpelling(pr), t.selectorSpelling(r), t.path) - } - for a := t.parentT; a != nil; a = a.parentT { - if pa, ok := d.pinned(a); ok && !t.under(r, a, pa) { - return fmt.Errorf("%s is not inside %s", t.selectorSpelling(r), a.selectorSpelling(pa)) - } - } - d.pin(t, r) - return nil -} - -// selectRow pins the row a selector names. -func (d *draws) selectRow(t *table, sel string) error { - r, err := t.find(sel, d) - if err != nil { - return err - } - return d.pinRow(t, r) -} - -// inside keeps the rows of t that sit inside every pinned ancestor. -func (d *draws) inside(t *table, rows []int) []int { - for a := t.parentT; a != nil; a = a.parentT { - pa, ok := d.pinned(a) - if !ok { - continue - } - var kept []int - for _, r := range rows { - if t.under(r, a, pa) { - kept = append(kept, r) - } - } - rows = kept - } - return rows -} - // tableRead is what a reference path reads of a table family: the table its head // names, the rows its selectors pin, the tables it draws — those it walks with no // row pinned, and their unpinned ancestors — each selector's spelling, and whether diff --git a/hold.go b/hold.go index 3a82baf..4b89c82 100644 --- a/hold.go +++ b/hold.go @@ -255,6 +255,12 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) }) } +// tablePin is one table's pinned row. +type tablePin struct { + t *table + row int +} + // draws is what has already been drawn for held names: the variant each was drawn as, // so every path under it reads one row, and the draw each read made, by its one // spelling, so the same read written twice reads one draw. An expansion keeps one for @@ -274,6 +280,129 @@ type draw struct { null bool } +// pinned is the row the render pinned for t, if any. +func (d *draws) pinned(t *table) (int, bool) { + for _, p := range d.pins[:d.npins] { + if p.t == t { + return p.row, true + } + } + r, ok := d.more[t] + return r, ok +} + +// mustRow is the row pinned for t, which the walk reaching a column pinned. +func (d *draws) mustRow(t *table) int { + r, ok := d.pinned(t) + if !ok { + panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) + } + return r +} + +// pin pins row r of t, and the rows of t's ancestors it links to. +func (d *draws) pin(t *table, r int) { + for { + if _, done := d.pinned(t); done { + return + } + if d.npins < len(d.pins) { + d.pins[d.npins] = tablePin{t, r} + d.npins++ + } else { + if d.more == nil { + d.more = map[*table]int{} + } + d.more[t] = r + } + if t.parentT == nil { + return + } + t, r = t.parentT, t.parentRow(r) + } +} + +// each calls fn for every pinned row, in pin order, the spilled ones by table name. +func (d *draws) each(fn func(t *table, r int)) { + for _, p := range d.pins[:d.npins] { + fn(p.t, p.row) + } + spilled := make([]*table, 0, len(d.more)) + for t := range d.more { + spilled = append(spilled, t) + } + sort.Slice(spilled, func(i, j int) bool { return spilled[i].category < spilled[j].category }) + for _, t := range spilled { + fn(t, d.more[t]) + } +} + +// rowOf is the render's row of t: the one pinned, else one drawn inside the +// nearest pinned ancestor — its parent drawn inside that first where the ancestor +// is further up — or over the whole table, and pinned with its ancestors. +func (d *draws) rowOf(t *table) int { + if r, ok := d.pinned(t); ok { + return r + } + r := -1 + for a := t.parentT; a != nil && r < 0; a = a.parentT { + if _, ok := d.pinned(a); !ok { + continue + } + if t.parentT != a { + d.rowOf(t.parentT) + } + pr, _ := d.pinned(t.parentT) + r = t.drawUnder(d.s, pr) + } + if r < 0 { + r = t.draw(d.s) + } + d.pin(t, r) + return r +} + +// pinRow pins row r of t where it agrees with the rows pinned before it. +func (d *draws) pinRow(t *table, r int) error { + if pr, ok := d.pinned(t); ok && pr != r { + return fmt.Errorf("%s and %s are two rows of %s", t.selectorSpelling(pr), t.selectorSpelling(r), t.path) + } + for a := t.parentT; a != nil; a = a.parentT { + if pa, ok := d.pinned(a); ok && !t.under(r, a, pa) { + return fmt.Errorf("%s is not inside %s", t.selectorSpelling(r), a.selectorSpelling(pa)) + } + } + d.pin(t, r) + return nil +} + +// selectRow pins the row a selector names. +func (d *draws) selectRow(t *table, sel string) error { + r, err := t.find(sel, d) + if err != nil { + return err + } + return d.pinRow(t, r) +} + +// inside keeps the rows of t that sit inside every pinned ancestor. +func (d *draws) inside(t *table, rows []int) []int { + for a := t.parentT; a != nil; a = a.parentT { + pa, ok := d.pinned(a) + if !ok { + continue + } + var kept []int + for _, r := range rows { + if t.under(r, a, pa) { + kept = append(kept, r) + } + } + rows = kept + } + return rows +} + // readField renders one arm of a token. An arm's key is a sibling field or a // reference linkRefs bound into fields. A name the expansion holds — a level some // token addresses by dotted path, or a field an operand reads — is drawn once and diff --git a/todo.md b/todo.md index edebcc5..fd887d3 100644 --- a/todo.md +++ b/todo.md @@ -21,16 +21,13 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Put `draws` and its methods in one file: it is declared in `hold.go:262` and all - eight of its methods sit in `family.go`, so a jump to the type lands where none of - the behaviour is. - Split `hold.go`, `draw.go` and `family.go` so each file holds one phase, the load-time fences apart from the render-time machinery, changing no logic. - Move `nodeScope`, `checkScope` and `checkRenders` out of `graph.go` and refresh the README Layout lines for `graph.go` and `data.go`: the order every load fence runs in is the load path's spine, and no file name carries it. - Name the `draw` stem apart: the free `pinned` in `hold.go` is not `(*draws).pinned` - in `family.go`, `compileFormat` is a method on both `*template` and `*table`, and + beside it, `compileFormat` is a method on both `*template` and `*table`, and `draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, `drawVisit` and `drawRoute` no longer tell each other apart. - Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go` -- 2.52.0 From 22723a6df60352faad304204f91158aca9f5cd8a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 17:37:51 +0200 Subject: [PATCH 033/966] Point the round's remaining items at the files the move leaves them in --- todo.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index fd887d3..f2458cf 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,8 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Split `hold.go`, `draw.go` and `family.go` so each file holds one phase, the - load-time fences apart from the render-time machinery, changing no logic. +- Split `hold.go` and `draw.go` so each file holds one phase, the load-time fences + apart from the render-time machinery, changing no logic. - Move `nodeScope`, `checkScope` and `checkRenders` out of `graph.go` and refresh the README Layout lines for `graph.go` and `data.go`: the order every load fence runs in is the load path's spine, and no file name carries it. @@ -165,7 +165,7 @@ by a `data-import/` script, as README goal 12 asks. - Have the family fence offer only a rewrite that works: where the drawn table's rows select more than one row of the family, every `write {/terr[NO].name}` it offers fails the same way, so only the `drawGroup` it also names is left. -- Spell a table one way across `family.go`'s errors: `checkFamilyPair` names +- Spell a table one way across the family errors: `checkFamilyPair` names `drawn.category`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. -- 2.52.0 From 98fffe31bed6ee0252b5c145284ae4208ed0ceed Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 17:41:34 +0200 Subject: [PATCH 034/966] Name GitHub as the forge the release lands on, and docs/ in the Layout --- README.md | 5 +++-- todo.md | 9 +++++++++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6d80e2e..8d343b0 100644 --- a/README.md +++ b/README.md @@ -958,7 +958,7 @@ the Development section below, and who ships a register the four above then draw - **The changelog heading is the one spelling of a release; CI cuts the tag.** A tag pushed by hand is served by `go get` at once, so a tag whose commit lacks its heading is burnt, not fixed. The heading on a gate-passed `main` commit is the - trigger instead: the tag can land only there, and the Gitea release the same job + trigger instead: the tag can land only there, and the GitHub release the same job publishes keeps one text as its body and is where prebuilt binaries will attach. - **A `--data-path` override rebinds every reference to the category it replaces.** References bind against the merged tree, so once shipped data uses @@ -1344,6 +1344,7 @@ data.go data loading: fs.FS folders/files -> namespace tree, multi-sourc cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) +docs/ the register research behind the shipped data, and the survey of other libraries release-tooling/ the release CI publishes from the changelog heading testdata/ the pinned shipped shape (see Versioning) ``` @@ -1433,7 +1434,7 @@ docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/usera To release, head `CHANGELOG.md` with the version's section in place of `Unreleased` and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publishes -the Gitea release with the section as its body. A top heading of `[Unreleased]` +the GitHub release with the section as its body. A top heading of `[Unreleased]` publishes nothing. ## License diff --git a/todo.md b/todo.md index f2458cf..c7de840 100644 --- a/todo.md +++ b/todo.md @@ -262,6 +262,15 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Name goal 2's four criteria with the axes the panel scores, so a recorded score maps back to the clause it came from: the goal spells them out in prose while `todo.md` records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. +- Put the round's nine-seat score and the four-seat score every later pull request + takes on one scale, or say in `AGENTS.md` that they are two ratchets: it gates the + round on the first and every merge after 7.0 on the second, then holds one "last + score" across both, where the `comprehension-panel` skill has two independent panels + drift unless a calibration seat reads the anchor project without its score. +- Name the draw fences at one level in `doc.go`: the Vocabulary calls them `heldCheck`, + `drawCheck` and `checkFamilies`, where `checkFamilies` runs under `drawCheck` beside + `checkOwnFamily` and `checkColumnDraws`, which it names nowhere — so a reader looking + for the fence over a record's columns finds no entry reaching it. - Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates Go at `Dockerfile:21`, and the shipped JSON under `data/` is two-space, so the rule as written is one no Go file can break and every data file does. -- 2.52.0 From e0c82440b39b11bc6523ce1998176c45a2e396fa Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 18:36:31 +0200 Subject: [PATCH 035/966] Hold every source file to a README Layout line --- doc_test.go | 9 +++++++-- readme_test.go | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/doc_test.go b/doc_test.go index 55d8b1a..4952312 100644 --- a/doc_test.go +++ b/doc_test.go @@ -47,15 +47,20 @@ func namesAPass(t *testing.T, where, text string, declared, funcs map[string]boo } } -func sourceFiles(t *testing.T) (*token.FileSet, []*ast.File) { +func goFiles(t *testing.T) []string { t.Helper() pkg, err := build.ImportDir(".", 0) if err != nil { t.Fatalf("read the package directory: %v", err) } + return pkg.GoFiles +} + +func sourceFiles(t *testing.T) (*token.FileSet, []*ast.File) { + t.Helper() fset := token.NewFileSet() var files []*ast.File - for _, name := range pkg.GoFiles { + for _, name := range goFiles(t) { f, err := parser.ParseFile(fset, name, nil, parser.ParseComments) if err != nil { t.Fatalf("parse %s: %v", name, err) diff --git a/readme_test.go b/readme_test.go index c253170..a8238f8 100644 --- a/readme_test.go +++ b/readme_test.go @@ -4,12 +4,14 @@ import ( "encoding/json" "os" "regexp" + "slices" "strings" "testing" ) var ( jsonBlock = regexp.MustCompile("(?s)```json\n(.*?)```") + plainBlock = regexp.MustCompile("(?s)```\n(.*?)```") tsvBlock = regexp.MustCompile("(?s)```tsv\n(.*?)```") rowsFile = regexp.MustCompile(`"rows":\s*"([^"]+)"`) parentName = regexp.MustCompile(`"parent":\s*"([^"]+)"`) @@ -191,3 +193,38 @@ func TestReadmeTableExample(t *testing.T) { t.Fatalf("README table example as a record: %q, %v", r.CSVHeader(), err) } } + +// TestLayoutNamesEverySourceFile holds an added, renamed or split file to its Layout +// line, so the list a reader picks a file from stays the list on disk. +func TestLayoutNamesEverySourceFile(t *testing.T) { + src := readme(t) + i := strings.Index(src, "## Layout") + if i < 0 { + t.Fatal("README lost the Layout section") + } + block := plainBlock.FindStringSubmatch(src[i:]) + if block == nil { + t.Fatal("README Layout section has no block") + } + listed := map[string]bool{} + for _, line := range strings.Split(block[1], "\n") { + name, _, _ := strings.Cut(strings.TrimSpace(line), " ") + if strings.HasSuffix(name, ".go") { + listed[name] = true + } + } + for _, name := range goFiles(t) { + if !listed[name] { + t.Errorf("the README Layout names no %s, so nothing says what it holds", name) + } + delete(listed, name) + } + stale := make([]string, 0, len(listed)) + for name := range listed { + stale = append(stale, name) + } + slices.Sort(stale) + for _, name := range stale { + t.Errorf("the README Layout names %s, which the package does not hold", name) + } +} -- 2.52.0 From 6700a8ad9f8c172fe79b51030714d9ad49e3fefe Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 18:38:03 +0200 Subject: [PATCH 036/966] Split hold and draw so a file name says load or render --- README.md | 6 +- draw.go | 420 -------------------------------------------------- drawfence.go | 425 +++++++++++++++++++++++++++++++++++++++++++++++++++ hold.go | 250 ------------------------------ holdfence.go | 256 +++++++++++++++++++++++++++++++ todo.md | 7 +- 6 files changed, 688 insertions(+), 676 deletions(-) create mode 100644 drawfence.go create mode 100644 holdfence.go diff --git a/README.md b/README.md index 8d343b0..19a5df8 100644 --- a/README.md +++ b/README.md @@ -1328,8 +1328,10 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold: one draw per expansion for paths and operands, the rows a render pins, and its fences -draw.go one reference draw per render and group: draw sets, the group option, and its fence +hold.go the hold at render: one draw per expansion for paths and operands, and the rows it pins +holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways +draw.go one reference draw per render and group: the draw set a render reads its paths through +drawfence.go the load fences over reference draws: the drawGroup option, and the walk over a render's reads family.go a family of linked tables: what a reference path reads of one, and the fence over paths into it reference.go reference sigils, and binding references across the tree graph.go the render graph: edges, cycles, the repeat bound, tree walks diff --git a/draw.go b/draw.go index daa1e61..0fdb478 100644 --- a/draw.go +++ b/draw.go @@ -1,8 +1,6 @@ package fejkdata import ( - "fmt" - "sort" "strings" ) @@ -57,421 +55,3 @@ func (sc drawScope) draws(s *session) *draws { } return d } - -// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each -// iteration renders in no draw group. -func drawGroupOf(m map[string]any, repeat int) (string, error) { - v, ok := m["drawGroup"] - if !ok { - return "", nil - } - name, ok := v.(string) - switch { - case !ok: - return "", fmt.Errorf("drawGroup must be a string, got %T", v) - case name == "": - return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`) - case repeat > 1: - return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name) - } - return name, nil -} - -// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name -// is local to its category. -func (t *template) keyDrawGroup(category string) { - if t.drawGroup != "" { - t.drawGroupKey = category + "/" + t.drawGroup - } -} - -// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, -// short of a repeat or another draw group. -func checkNestedDrawGroup(fields map[string]node, group string) error { - if group == "" { - return nil - } - var walk func(path string, n node) error - walk = func(path string, n node) error { - t, isTemplate := n.(*template) - switch { - case isTemplate && t.drawGroup == group: - return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group) - case isTemplate && (t.drawGroup != "" || t.repeat > 1): - return nil - } - for _, c := range contained(n) { - if err := walk(join(path, c.name), c.node); err != nil { - return err - } - } - return nil - } - for _, name := range sortedNames(fields) { - if err := walk(name, fields[name]); err != nil { - return err - } - } - return nil -} - -// drawCheck fences each template of a scope as a render of its own, remembering which nodes read a -// reference path. -type drawCheck struct { - memo map[readsMemo]bool -} - -type readsMemo struct { - n node - stopAtGroup bool -} - -// checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference -// path inside a repeat or a nested draw group, which draw apart from it whatever it names. -func (c *drawCheck) checkDrawGroup(path string, n node) error { - if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) { - return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) - } - return nil -} - -func (c *drawCheck) checkDraws(path string, n node) error { - t, ok := n.(*template) - if !ok { - return nil - } - if err := checkOwnFamily(t); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - if !c.readsPath(t) { - return nil - } - w := newDrawWalk() - for _, e := range renderEdges(t) { - w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}}) - } - if err := w.check(); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - return nil -} - -// checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a row -// rendered whole draws its row without pinning it, so the family would draw apart -// from the row being rendered, whichever draws the reaching template holds. -func checkOwnFamily(t *template) error { - own := t.table - if own == nil { - own = t.cellOf - } - if own == nil { - return nil - } - seen := map[node]bool{} - var find func(n node) (renderEdge, *table, bool) - find = func(n node) (renderEdge, *table, bool) { - if seen[n] { - return renderEdge{}, nil, false - } - seen[n] = true - for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).fields[a.key].(*table); isTable && head.family() == own.family() { - return e, head, true - } - } - if e, head, found := find(e.to); found { - return e, head, true - } - } - return renderEdge{}, nil, false - } - if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a row of %s rendered whole would draw apart from; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) - } - return nil -} - -// checkRecordDraws fences the columns of a record — a template compiled at the top without a -// repeat — so a load proves the record view of it as well as the string view. -func (c *drawCheck) checkRecordDraws(path string, n node) error { - t, ok := n.(*template) - if !ok || !t.record { - return nil - } - // A record-only template's format renders nothing, so weigh the columns, not the format. - columns := recordColumns(t) - reads := false - for _, name := range columns { - reads = reads || c.readsPath(t.fields[name]) - } - if !reads { - return nil - } - if err := checkColumnDraws(t, columns); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - return nil -} - -// readsPath reports whether rendering n reads a reference path, short of a repeat, which renders -// over draws of its own. -func (c *drawCheck) readsPath(n node) bool { return c.reads(n, false) } - -// splitsDraws reports whether rendering n reads a reference path that n's own draw group answers -// for: one outside a repeat and outside a nested draw group, which hold their own draws. -func (c *drawCheck) splitsDraws(n node) bool { return c.reads(n, true) } - -// reads walks what rendering n renders for a reference path, stopping at a repeat — and at a nested -// draw group when stopAtGroup — since each holds draws of its own. -func (c *drawCheck) reads(n node, stopAtGroup bool) bool { - k := readsMemo{n, stopAtGroup} - if r, done := c.memo[k]; done { - return r - } - r := false - for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); (isRef && (len(a.tail) > 0 || readsTable(n, a))) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) { - r = true - break - } - } - if c.memo == nil { - c.memo = map[readsMemo]bool{} - } - c.memo[k] = r - return r -} - -func repeats(n node) bool { - t, isTemplate := n.(*template) - return isTemplate && t.repeat > 1 -} - -func grouped(n node) bool { - t, isTemplate := n.(*template) - return isTemplate && t.drawGroupKey != "" -} - -// readsTable reports whether a reference of n names a table, whose draw the render's -// group answers for even when read whole. -func readsTable(n node, a arm) bool { - t, isTemplate := n.(*template) - if !isTemplate { - return false - } - _, isTable := t.fields[a.key].(*table) - return isTable -} - -// refRead is the reference an edge of n reads; false when the edge reads none. -func refRead(n node, label string) (arm, bool) { - t, isTemplate := n.(*template) - if !isTemplate { - return arm{}, false - } - a := splitArm(label, t.refs) - return a, isRef(a.key) -} - -// checkColumnDraws fences a record's columns as one render. -func checkColumnDraws(t *template, columns []string) error { - w := newDrawWalk() - for _, name := range columns { - w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) - } - return w.check() -} - -// drawWalk gathers the references one render reads, by draw group, for check to compare. -type drawWalk struct { - reads []pathRead - read map[drawKey]bool - seen map[drawVisit]bool -} - -// drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the -// table rows it entered, and the table a whole read draws a row of. -type drawAt struct { - group string - route drawRoute - alt rowSet - drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true -} - -// rowSet stands in at load for the rows (*draws).pin holds at render: a path read enters a row with -// its ancestors, a whole read enters the drawn row alone. -type rowSet []tablePin - -func (s rowSet) rowOf(t *table) (int, bool) { - for _, p := range s { - if p.t == t { - return p.row, true - } - } - return 0, false -} - -// enter is s with row r of t, and with t's ancestors where the render pins that row rather than -// drawing it; the set is copied, since walks branch. -func (s rowSet) enter(t *table, r int, pins bool) rowSet { - if _, in := s.rowOf(t); in { - return s - } - out := append(make(rowSet, 0, len(s)+1), s...) - for { - out = append(out, tablePin{t, r}) - if !pins || t.parentT == nil { - break - } - t, r = t.parentT, t.parentRow(r) - if _, in := out.rowOf(t); in { - break - } - } - sort.Slice(out, func(i, j int) bool { return out[i].t.path < out[j].t.path }) - return out -} - -// key spells the set for a map, by the tables' identities. -func (s rowSet) key() string { - var b strings.Builder - for _, p := range s { - fmt.Fprintf(&b, "%p[%d]", p.t, p.row) - } - return b.String() -} - -// alternatives reports whether two reads sit in different rows of one table. -func alternatives(a, b drawAt) bool { - for _, p := range a.alt { - if r, in := b.alt.rowOf(p.t); in && r != p.row { - return true - } - } - return false -} - -// drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. -type drawRoute struct{ spelling, label string } - -// pathRead is one reference a render reads: a path, or a bare reference with no tail. -type pathRead struct { - at drawAt - a arm - tr *tableRead // set where the reference names a table -} - -type drawVisit struct { - n node - group string - alt string -} - -type drawKey struct { - group string - path string - alt string -} - -func newDrawWalk() *drawWalk { - return &drawWalk{read: map[drawKey]bool{}, seen: map[drawVisit]bool{}} -} - -// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops -// there. -func (w *drawWalk) walk(n node, at drawAt) { - v := drawVisit{n, at.group, at.alt.key()} - if w.seen[v] { - return - } - w.seen[v] = true - if repeats(n) { - return - } - if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" { - at.group = t.drawGroupKey - } - if t, isTable := n.(*table); isTable { - at.drawn = t - } - for _, e := range renderEdges(n) { - if cell, isCell := e.to.(*template); isCell && cell.cellOf != nil { - if at.alt.excludes(cell) { - continue - } - in := at - in.alt = at.alt.enter(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn) - w.edge(n, e, in) - continue - } - w.edge(n, e, at) - } -} - -// excludes reports whether a cell's row cannot render with the rows entered: another -// row of its table, or a row outside an entered ancestor's. -func (s rowSet) excludes(cell *template) bool { - for _, p := range s { - if p.t == cell.cellOf { - return p.row != cell.cellRow - } - if cell.cellOf.descends(p.t) && !cell.cellOf.under(cell.cellRow, p.t, p.row) { - return true - } - } - return false -} - -// edge records the reference an edge reads, then walks on with every row the read -// pins entered, so a selected row renders only its own cells. -func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { - if a, reads := refRead(from, e.label); reads { - tr := tableReadOf(from.(*template).fields[a.key], a, e.to) - if k := (drawKey{at.group, a.path, at.alt.key()}); !w.read[k] { - w.read[k] = true - w.reads = append(w.reads, pathRead{at, a, tr}) - } - if tr != nil { - tr.pins.each(func(t *table, r int) { at.alt = at.alt.enter(t, r, true) }) - } - } - w.walk(e.to, at) -} - -// check refuses what one draw per reference path cannot answer for: a read of a level beside a path -// another read takes into it, and two reads of one table family that select different rows. Reads -// are compared in path order, so which pair is reported does not vary. -func (w *drawWalk) check() error { - sort.SliceStable(w.reads, func(i, j int) bool { - if w.reads[i].at.group != w.reads[j].at.group { - return w.reads[i].at.group < w.reads[j].at.group - } - return w.reads[i].a.path < w.reads[j].a.path - }) - for i, level := range w.reads { - for _, into := range w.reads[i+1:] { - if into.at.group != level.at.group || alternatives(level.at, into.at) { - continue - } - if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.whole) { - return overlapError(level.at.route, level.a.name, into) - } - } - } - return checkFamilies(w.reads) -} - -func overlapError(route drawRoute, ref string, into pathRead) error { - return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name)) -} - -// spelled names the route, and the reference it reaches a draw by where its root edge is not that -// reference. -func (r drawRoute) spelled(ref string) string { - if ref == "" || ref == r.label { - return r.spelling - } - return fmt.Sprintf("%s with {%s}", r.spelling, ref) -} diff --git a/drawfence.go b/drawfence.go new file mode 100644 index 0000000..4e51eaf --- /dev/null +++ b/drawfence.go @@ -0,0 +1,425 @@ +package fejkdata + +import ( + "fmt" + "sort" + "strings" +) + +// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each +// iteration renders in no draw group. +func drawGroupOf(m map[string]any, repeat int) (string, error) { + v, ok := m["drawGroup"] + if !ok { + return "", nil + } + name, ok := v.(string) + switch { + case !ok: + return "", fmt.Errorf("drawGroup must be a string, got %T", v) + case name == "": + return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`) + case repeat > 1: + return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name) + } + return name, nil +} + +// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name +// is local to its category. +func (t *template) keyDrawGroup(category string) { + if t.drawGroup != "" { + t.drawGroupKey = category + "/" + t.drawGroup + } +} + +// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, +// short of a repeat or another draw group. +func checkNestedDrawGroup(fields map[string]node, group string) error { + if group == "" { + return nil + } + var walk func(path string, n node) error + walk = func(path string, n node) error { + t, isTemplate := n.(*template) + switch { + case isTemplate && t.drawGroup == group: + return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group) + case isTemplate && (t.drawGroup != "" || t.repeat > 1): + return nil + } + for _, c := range contained(n) { + if err := walk(join(path, c.name), c.node); err != nil { + return err + } + } + return nil + } + for _, name := range sortedNames(fields) { + if err := walk(name, fields[name]); err != nil { + return err + } + } + return nil +} + +// drawCheck fences each template of a scope as a render of its own, remembering which nodes read a +// reference path. +type drawCheck struct { + memo map[readsMemo]bool +} + +type readsMemo struct { + n node + stopAtGroup bool +} + +// checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference +// path inside a repeat or a nested draw group, which draw apart from it whatever it names. +func (c *drawCheck) checkDrawGroup(path string, n node) error { + if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) { + return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) + } + return nil +} + +func (c *drawCheck) checkDraws(path string, n node) error { + t, ok := n.(*template) + if !ok { + return nil + } + if err := checkOwnFamily(t); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + if !c.readsPath(t) { + return nil + } + w := newDrawWalk() + for _, e := range renderEdges(t) { + w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}}) + } + if err := w.check(); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + return nil +} + +// checkOwnFamily refuses a table's format or cell that reads, however many templates +// away and through a repeat or a draw group too, a table of its own family: a row +// rendered whole draws its row without pinning it, so the family would draw apart +// from the row being rendered, whichever draws the reaching template holds. +func checkOwnFamily(t *template) error { + own := t.table + if own == nil { + own = t.cellOf + } + if own == nil { + return nil + } + seen := map[node]bool{} + var find func(n node) (renderEdge, *table, bool) + find = func(n node) (renderEdge, *table, bool) { + if seen[n] { + return renderEdge{}, nil, false + } + seen[n] = true + for _, e := range renderEdges(n) { + if a, isRef := refRead(n, e.label); isRef { + if head, isTable := n.(*template).fields[a.key].(*table); isTable && head.family() == own.family() { + return e, head, true + } + } + if e, head, found := find(e.to); found { + return e, head, true + } + } + return renderEdge{}, nil, false + } + if e, head, found := find(t); found { + return fmt.Errorf("%s reads %s, a table of its own family, which a row of %s rendered whole would draw apart from; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + } + return nil +} + +// checkRecordDraws fences the columns of a record — a template compiled at the top without a +// repeat — so a load proves the record view of it as well as the string view. +func (c *drawCheck) checkRecordDraws(path string, n node) error { + t, ok := n.(*template) + if !ok || !t.record { + return nil + } + // A record-only template's format renders nothing, so weigh the columns, not the format. + columns := recordColumns(t) + reads := false + for _, name := range columns { + reads = reads || c.readsPath(t.fields[name]) + } + if !reads { + return nil + } + if err := checkColumnDraws(t, columns); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + return nil +} + +// readsPath reports whether rendering n reads a reference path, short of a repeat, which renders +// over draws of its own. +func (c *drawCheck) readsPath(n node) bool { return c.reads(n, false) } + +// splitsDraws reports whether rendering n reads a reference path that n's own draw group answers +// for: one outside a repeat and outside a nested draw group, which hold their own draws. +func (c *drawCheck) splitsDraws(n node) bool { return c.reads(n, true) } + +// reads walks what rendering n renders for a reference path, stopping at a repeat — and at a nested +// draw group when stopAtGroup — since each holds draws of its own. +func (c *drawCheck) reads(n node, stopAtGroup bool) bool { + k := readsMemo{n, stopAtGroup} + if r, done := c.memo[k]; done { + return r + } + r := false + for _, e := range renderEdges(n) { + if a, isRef := refRead(n, e.label); (isRef && (len(a.tail) > 0 || readsTable(n, a))) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) { + r = true + break + } + } + if c.memo == nil { + c.memo = map[readsMemo]bool{} + } + c.memo[k] = r + return r +} + +func repeats(n node) bool { + t, isTemplate := n.(*template) + return isTemplate && t.repeat > 1 +} + +func grouped(n node) bool { + t, isTemplate := n.(*template) + return isTemplate && t.drawGroupKey != "" +} + +// readsTable reports whether a reference of n names a table, whose draw the render's +// group answers for even when read whole. +func readsTable(n node, a arm) bool { + t, isTemplate := n.(*template) + if !isTemplate { + return false + } + _, isTable := t.fields[a.key].(*table) + return isTable +} + +// refRead is the reference an edge of n reads; false when the edge reads none. +func refRead(n node, label string) (arm, bool) { + t, isTemplate := n.(*template) + if !isTemplate { + return arm{}, false + } + a := splitArm(label, t.refs) + return a, isRef(a.key) +} + +// checkColumnDraws fences a record's columns as one render. +func checkColumnDraws(t *template, columns []string) error { + w := newDrawWalk() + for _, name := range columns { + w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) + } + return w.check() +} + +// drawWalk gathers the references one render reads, by draw group, for check to compare. +type drawWalk struct { + reads []pathRead + read map[drawKey]bool + seen map[drawVisit]bool +} + +// drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the +// table rows it entered, and the table a whole read draws a row of. +type drawAt struct { + group string + route drawRoute + alt rowSet + drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true +} + +// rowSet stands in at load for the rows (*draws).pin holds at render: a path read enters a row with +// its ancestors, a whole read enters the drawn row alone. +type rowSet []tablePin + +func (s rowSet) rowOf(t *table) (int, bool) { + for _, p := range s { + if p.t == t { + return p.row, true + } + } + return 0, false +} + +// enter is s with row r of t, and with t's ancestors where the render pins that row rather than +// drawing it; the set is copied, since walks branch. +func (s rowSet) enter(t *table, r int, pins bool) rowSet { + if _, in := s.rowOf(t); in { + return s + } + out := append(make(rowSet, 0, len(s)+1), s...) + for { + out = append(out, tablePin{t, r}) + if !pins || t.parentT == nil { + break + } + t, r = t.parentT, t.parentRow(r) + if _, in := out.rowOf(t); in { + break + } + } + sort.Slice(out, func(i, j int) bool { return out[i].t.path < out[j].t.path }) + return out +} + +// key spells the set for a map, by the tables' identities. +func (s rowSet) key() string { + var b strings.Builder + for _, p := range s { + fmt.Fprintf(&b, "%p[%d]", p.t, p.row) + } + return b.String() +} + +// alternatives reports whether two reads sit in different rows of one table. +func alternatives(a, b drawAt) bool { + for _, p := range a.alt { + if r, in := b.alt.rowOf(p.t); in && r != p.row { + return true + } + } + return false +} + +// drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. +type drawRoute struct{ spelling, label string } + +// pathRead is one reference a render reads: a path, or a bare reference with no tail. +type pathRead struct { + at drawAt + a arm + tr *tableRead // set where the reference names a table +} + +type drawVisit struct { + n node + group string + alt string +} + +type drawKey struct { + group string + path string + alt string +} + +func newDrawWalk() *drawWalk { + return &drawWalk{read: map[drawKey]bool{}, seen: map[drawVisit]bool{}} +} + +// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops +// there. +func (w *drawWalk) walk(n node, at drawAt) { + v := drawVisit{n, at.group, at.alt.key()} + if w.seen[v] { + return + } + w.seen[v] = true + if repeats(n) { + return + } + if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" { + at.group = t.drawGroupKey + } + if t, isTable := n.(*table); isTable { + at.drawn = t + } + for _, e := range renderEdges(n) { + if cell, isCell := e.to.(*template); isCell && cell.cellOf != nil { + if at.alt.excludes(cell) { + continue + } + in := at + in.alt = at.alt.enter(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn) + w.edge(n, e, in) + continue + } + w.edge(n, e, at) + } +} + +// excludes reports whether a cell's row cannot render with the rows entered: another +// row of its table, or a row outside an entered ancestor's. +func (s rowSet) excludes(cell *template) bool { + for _, p := range s { + if p.t == cell.cellOf { + return p.row != cell.cellRow + } + if cell.cellOf.descends(p.t) && !cell.cellOf.under(cell.cellRow, p.t, p.row) { + return true + } + } + return false +} + +// edge records the reference an edge reads, then walks on with every row the read +// pins entered, so a selected row renders only its own cells. +func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { + if a, reads := refRead(from, e.label); reads { + tr := tableReadOf(from.(*template).fields[a.key], a, e.to) + if k := (drawKey{at.group, a.path, at.alt.key()}); !w.read[k] { + w.read[k] = true + w.reads = append(w.reads, pathRead{at, a, tr}) + } + if tr != nil { + tr.pins.each(func(t *table, r int) { at.alt = at.alt.enter(t, r, true) }) + } + } + w.walk(e.to, at) +} + +// check refuses what one draw per reference path cannot answer for: a read of a level beside a path +// another read takes into it, and two reads of one table family that select different rows. Reads +// are compared in path order, so which pair is reported does not vary. +func (w *drawWalk) check() error { + sort.SliceStable(w.reads, func(i, j int) bool { + if w.reads[i].at.group != w.reads[j].at.group { + return w.reads[i].at.group < w.reads[j].at.group + } + return w.reads[i].a.path < w.reads[j].a.path + }) + for i, level := range w.reads { + for _, into := range w.reads[i+1:] { + if into.at.group != level.at.group || alternatives(level.at, into.at) { + continue + } + if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.whole) { + return overlapError(level.at.route, level.a.name, into) + } + } + } + return checkFamilies(w.reads) +} + +func overlapError(route drawRoute, ref string, into pathRead) error { + return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name)) +} + +// spelled names the route, and the reference it reaches a draw by where its root edge is not that +// reference. +func (r drawRoute) spelled(ref string) string { + if ref == "" || ref == r.label { + return r.spelling + } + return fmt.Sprintf("%s with {%s}", r.spelling, ref) +} diff --git a/hold.go b/hold.go index 4b89c82..a54e183 100644 --- a/hold.go +++ b/hold.go @@ -3,258 +3,8 @@ package fejkdata import ( "fmt" "sort" - "strings" ) -// heldCheck rejects every route to a held sibling name except the ones that read its -// draw. An expansion holds one draw of that name; anything else that renders it draws -// again, and the two disagree. checkNoOverlap settles the spellings within one format -// (a token, an operand); this settles the rest — a reference, whether it sits in that -// format or in anything the format renders, however deep. A reference path is held -// for the whole render instead, which drawCheck fences. -func heldCheck(path string, n node) error { - t, ok := n.(*template) - if !ok || len(t.held) == 0 { - return nil - } - readers := boundReaders(t.format, t.bound, t.refs) - for _, head := range heldHeads(t) { - if _, isPath := t.bound[head]; isPath && isRef(head) { - continue - } - if err := checkHeadHeld(t, head, readers); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - } - return nil -} - -// heldHeads lists a template's held names, operand heads first, then paths, each -// in name order: a level read both ways is reported by the operand's fence, and -// which overlap is reported does not vary. -func heldHeads(t *template) []string { - heads := make([]string, 0, len(t.held)) - for head := range t.held { - heads = append(heads, head) - } - sort.Slice(heads, func(i, j int) bool { - _, pi := t.bound[heads[i]] - _, pj := t.bound[heads[j]] - if pi != pj { - return !pi - } - return heads[i] < heads[j] - }) - return heads -} - -// pinned collects what the hold of head answers for: a path pins the levels it -// passes through and the leaf it lands on, an operand exactly the value its render -// produces. -func pinned(t *template, head string, readers []reader) map[node]bool { - held := map[node]bool{} - if _, isPath := t.bound[head]; !isPath { - operandDraw(t.fields[head], held) - return held - } - for _, r := range readers { - if a := splitArm(r.name, t.refs); a.key == head { - coverPath(t.fields[head], a.tail, held) - } - } - return held -} - -// checkHeadHeld rejects every route to what head's hold pins except the readers -// holding it. One seen set across the edges: a node that cannot reach the level -// cannot reach it by another route either, so it is walked once. -func checkHeadHeld(t *template, head string, readers []reader) error { - held := pinned(t, head, readers) - if len(held) == 0 { - return nil // a fixed head holds nothing to reach - } - reader, isPath := t.bound[head] - seen := map[node]bool{} - for _, e := range renderEdges(t) { - if splitArm(e.label, t.refs).key == head || !renders(e.to, held, seen) { - continue - } - if isPath { - return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), head, reader) - } - return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), head, operandReader(t, head)) - } - return nil -} - -// operandReader names the builtin whose operand holds head. -func operandReader(t *template, head string) string { - fn := "" - _ = eachToken(t.format, func(tok ftoken) error { - if tok.kind != 'b' || fn != "" { - return nil - } - if name, _, isFunc := funcCall(tok.body); isFunc { - for _, operand := range tokenOperands(tok.body) { - if splitArm(operand, t.refs).key == head { - fn = name - } - } - } - return nil - }) - return fn -} - -// coverPath collects what holding one path pins: every choice level the path -// passes through, whole, and the leaf it renders. -func coverPath(n node, tail []string, into map[node]bool) { - _, _ = walkPath(n, tail, pathWalk{ - choice: func(c *choice, _ []string) ([]node, error) { cover(c, into, false); return nil, nil }, - leaf: func(n node) error { cover(n, into, false); return nil }, - }) -} - -// cover collects a level and everything contained in it. A fixed string outside a -// choice is left out — it cannot disagree with itself — but inside one each -// variant carries its own, so there it counts. -func cover(n node, into map[node]bool, inChoice bool) { - if isFixed(n) && !inChoice { - return - } - into[n] = true - _, isChoice := n.(*choice) - for _, c := range contained(n) { - cover(c.node, into, inChoice || isChoice) - } -} - -// operandDraw collects what one held draw of an operand answers for: the operand -// and what rendering it settles inside itself. The builtin renders its operand -// whole, so that draw fixes every value the render produced, and a second route to -// any of them disagrees with it. -// -// The walk stops at a reference edge, which is where the operand's own value ends -// and a shared source begins: two names referencing one category are two draws, the -// same rule {word} {word} follows. -func operandDraw(n node, into map[node]bool) { - if isFixed(n) { - return - } - if into[n] { - return - } - into[n] = true - for _, e := range renderEdges(n) { - if isRef(e.label) { - continue - } - operandDraw(e.to, into) - } -} - -// isFixed is a string that varies nothing: fixed text with no fields to read into. -func isFixed(n node) bool { - t, ok := n.(*template) - return ok && t.fixed && len(t.fields) == 0 -} - -// renders reports whether rendering n can reach anything in want, following the -// same edges expand does. seen keeps a node shared by several routes from being -// walked twice; checkNoCycles has already proved the graph is a DAG, so the walk -// ends. -func renders(n node, want, seen map[node]bool) bool { - if want[n] { - return true - } - if seen[n] { - return false - } - seen[n] = true - for _, e := range renderEdges(n) { - if renders(e.to, want, seen) { - return true - } - } - return false -} - -// checkNoOverlap rejects a format that both renders a sibling level and reads a path -// into it — {p} beside {p.first}, {p.addr} beside {p.addr.city}. The path reads the -// level's held draw while rendering the level expands it afresh, so their values would -// disagree. Reads are compared in sorted order, so which pair is reported does not -// depend on where the tokens sit. -func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error { - names := boundReaders(format, bound, refs) - // Stable over one format-order scan, so two readers of one name (a token and a - // calc operand both naming "p") are reported as the format writes them. - sort.SliceStable(names, func(i, j int) bool { return names[i].path < names[j].path }) - for i, level := range names { - for _, path := range names[i+1:] { - if strings.HasPrefix(path.path, level.path+".") { - return fmt.Errorf("%s renders a level that {%s} reads a path into; name the fields you want instead", level.label, path.name) - } - } - } - return nil -} - -// reader is one way a format reaches a bound field: as written, by its one -// spelling, and how to name it. -type reader struct{ name, path, label string } - -// boundReaders lists every way a format reaches a bound sibling field, in the order the -// format writes them. An operand renders its field, so it names a level exactly -// as a token does; one scan finds both, which is what puts them in one order. -func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader { - var names []reader - _ = eachToken(format, func(t ftoken) error { - if t.kind != 'b' { - return nil - } - if fn, _, isFunc := funcCall(t.body); isFunc { - for _, operand := range tokenOperands(t.body) { - a := splitArm(operand, refs) - if _, isBound := bound[a.key]; isBound && !isRef(a.key) { - names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)}) - } - } - return nil - } - for _, a := range splitArms(t.body, refs) { - if _, isBound := bound[a.key]; isBound && !isRef(a.key) { - names = append(names, reader{a.name, a.path, "token {" + a.name + "}"}) - } - } - return nil - }) - return names -} - -// checkNoRepeatedRead rejects a bare token repeated on a held name: {w} {w} beside -// {uppercase(w)} would read one draw twice, where {w} {w} alone draws twice. The -// error names the single-token spelling. -func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) error { - count := map[string]int{} - return eachToken(format, func(t ftoken) error { - if t.kind != 'b' { - return nil - } - if _, _, isFunc := funcCall(t.body); isFunc { - return nil - } - for _, a := range splitArms(t.body, refs) { - if len(a.tail) > 0 || !c.held[a.key] { - continue - } - if count[a.key]++; count[a.key] > 1 { - return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.name, c.holder[a.key], a.key, a.name) - } - } - return nil - }) -} - // tablePin is one table's pinned row. type tablePin struct { t *table diff --git a/holdfence.go b/holdfence.go new file mode 100644 index 0000000..f1b114a --- /dev/null +++ b/holdfence.go @@ -0,0 +1,256 @@ +package fejkdata + +import ( + "fmt" + "sort" + "strings" +) + +// heldCheck rejects every route to a held sibling name except the ones that read its +// draw. An expansion holds one draw of that name; anything else that renders it draws +// again, and the two disagree. checkNoOverlap settles the spellings within one format +// (a token, an operand); this settles the rest — a reference, whether it sits in that +// format or in anything the format renders, however deep. A reference path is held +// for the whole render instead, which drawCheck fences. +func heldCheck(path string, n node) error { + t, ok := n.(*template) + if !ok || len(t.held) == 0 { + return nil + } + readers := boundReaders(t.format, t.bound, t.refs) + for _, head := range heldHeads(t) { + if _, isPath := t.bound[head]; isPath && isRef(head) { + continue + } + if err := checkHeadHeld(t, head, readers); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + } + return nil +} + +// heldHeads lists a template's held names, operand heads first, then paths, each +// in name order: a level read both ways is reported by the operand's fence, and +// which overlap is reported does not vary. +func heldHeads(t *template) []string { + heads := make([]string, 0, len(t.held)) + for head := range t.held { + heads = append(heads, head) + } + sort.Slice(heads, func(i, j int) bool { + _, pi := t.bound[heads[i]] + _, pj := t.bound[heads[j]] + if pi != pj { + return !pi + } + return heads[i] < heads[j] + }) + return heads +} + +// pinned collects what the hold of head answers for: a path pins the levels it +// passes through and the leaf it lands on, an operand exactly the value its render +// produces. +func pinned(t *template, head string, readers []reader) map[node]bool { + held := map[node]bool{} + if _, isPath := t.bound[head]; !isPath { + operandDraw(t.fields[head], held) + return held + } + for _, r := range readers { + if a := splitArm(r.name, t.refs); a.key == head { + coverPath(t.fields[head], a.tail, held) + } + } + return held +} + +// checkHeadHeld rejects every route to what head's hold pins except the readers +// holding it. One seen set across the edges: a node that cannot reach the level +// cannot reach it by another route either, so it is walked once. +func checkHeadHeld(t *template, head string, readers []reader) error { + held := pinned(t, head, readers) + if len(held) == 0 { + return nil // a fixed head holds nothing to reach + } + reader, isPath := t.bound[head] + seen := map[node]bool{} + for _, e := range renderEdges(t) { + if splitArm(e.label, t.refs).key == head || !renders(e.to, held, seen) { + continue + } + if isPath { + return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), head, reader) + } + return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), head, operandReader(t, head)) + } + return nil +} + +// operandReader names the builtin whose operand holds head. +func operandReader(t *template, head string) string { + fn := "" + _ = eachToken(t.format, func(tok ftoken) error { + if tok.kind != 'b' || fn != "" { + return nil + } + if name, _, isFunc := funcCall(tok.body); isFunc { + for _, operand := range tokenOperands(tok.body) { + if splitArm(operand, t.refs).key == head { + fn = name + } + } + } + return nil + }) + return fn +} + +// coverPath collects what holding one path pins: every choice level the path +// passes through, whole, and the leaf it renders. +func coverPath(n node, tail []string, into map[node]bool) { + _, _ = walkPath(n, tail, pathWalk{ + choice: func(c *choice, _ []string) ([]node, error) { cover(c, into, false); return nil, nil }, + leaf: func(n node) error { cover(n, into, false); return nil }, + }) +} + +// cover collects a level and everything contained in it. A fixed string outside a +// choice is left out — it cannot disagree with itself — but inside one each +// variant carries its own, so there it counts. +func cover(n node, into map[node]bool, inChoice bool) { + if isFixed(n) && !inChoice { + return + } + into[n] = true + _, isChoice := n.(*choice) + for _, c := range contained(n) { + cover(c.node, into, inChoice || isChoice) + } +} + +// operandDraw collects what one held draw of an operand answers for: the operand +// and what rendering it settles inside itself. The builtin renders its operand +// whole, so that draw fixes every value the render produced, and a second route to +// any of them disagrees with it. +// +// The walk stops at a reference edge, which is where the operand's own value ends +// and a shared source begins: two names referencing one category are two draws, the +// same rule {word} {word} follows. +func operandDraw(n node, into map[node]bool) { + if isFixed(n) { + return + } + if into[n] { + return + } + into[n] = true + for _, e := range renderEdges(n) { + if isRef(e.label) { + continue + } + operandDraw(e.to, into) + } +} + +// isFixed is a string that varies nothing: fixed text with no fields to read into. +func isFixed(n node) bool { + t, ok := n.(*template) + return ok && t.fixed && len(t.fields) == 0 +} + +// renders reports whether rendering n can reach anything in want, following the +// same edges expand does. seen keeps a node shared by several routes from being +// walked twice; checkNoCycles has already proved the graph is a DAG, so the walk +// ends. +func renders(n node, want, seen map[node]bool) bool { + if want[n] { + return true + } + if seen[n] { + return false + } + seen[n] = true + for _, e := range renderEdges(n) { + if renders(e.to, want, seen) { + return true + } + } + return false +} + +// checkNoOverlap rejects a format that both renders a sibling level and reads a path +// into it — {p} beside {p.first}, {p.addr} beside {p.addr.city}. The path reads the +// level's held draw while rendering the level expands it afresh, so their values would +// disagree. Reads are compared in sorted order, so which pair is reported does not +// depend on where the tokens sit. +func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error { + names := boundReaders(format, bound, refs) + // Stable over one format-order scan, so two readers of one name (a token and a + // calc operand both naming "p") are reported as the format writes them. + sort.SliceStable(names, func(i, j int) bool { return names[i].path < names[j].path }) + for i, level := range names { + for _, path := range names[i+1:] { + if strings.HasPrefix(path.path, level.path+".") { + return fmt.Errorf("%s renders a level that {%s} reads a path into; name the fields you want instead", level.label, path.name) + } + } + } + return nil +} + +// reader is one way a format reaches a bound field: as written, by its one +// spelling, and how to name it. +type reader struct{ name, path, label string } + +// boundReaders lists every way a format reaches a bound sibling field, in the order the +// format writes them. An operand renders its field, so it names a level exactly +// as a token does; one scan finds both, which is what puts them in one order. +func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader { + var names []reader + _ = eachToken(format, func(t ftoken) error { + if t.kind != 'b' { + return nil + } + if fn, _, isFunc := funcCall(t.body); isFunc { + for _, operand := range tokenOperands(t.body) { + a := splitArm(operand, refs) + if _, isBound := bound[a.key]; isBound && !isRef(a.key) { + names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)}) + } + } + return nil + } + for _, a := range splitArms(t.body, refs) { + if _, isBound := bound[a.key]; isBound && !isRef(a.key) { + names = append(names, reader{a.name, a.path, "token {" + a.name + "}"}) + } + } + return nil + }) + return names +} + +// checkNoRepeatedRead rejects a bare token repeated on a held name: {w} {w} beside +// {uppercase(w)} would read one draw twice, where {w} {w} alone draws twice. The +// error names the single-token spelling. +func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) error { + count := map[string]int{} + return eachToken(format, func(t ftoken) error { + if t.kind != 'b' { + return nil + } + if _, _, isFunc := funcCall(t.body); isFunc { + return nil + } + for _, a := range splitArms(t.body, refs) { + if len(a.tail) > 0 || !c.held[a.key] { + continue + } + if count[a.key]++; count[a.key] > 1 { + return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.name, c.holder[a.key], a.key, a.name) + } + } + return nil + }) +} diff --git a/todo.md b/todo.md index c7de840..bd4356c 100644 --- a/todo.md +++ b/todo.md @@ -21,13 +21,12 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Split `hold.go` and `draw.go` so each file holds one phase, the load-time fences - apart from the render-time machinery, changing no logic. - Move `nodeScope`, `checkScope` and `checkRenders` out of `graph.go` and refresh the README Layout lines for `graph.go` and `data.go`: the order every load fence runs in is the load path's spine, and no file name carries it. -- Name the `draw` stem apart: the free `pinned` in `hold.go` is not `(*draws).pinned` - beside it, `compileFormat` is a method on both `*template` and `*table`, and +- Name the `draw` stem apart: the free `pinned` in `holdfence.go` is not + `(*draws).pinned` in `hold.go`, `compileFormat` is a method on both `*template` and + `*table`, and `draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, `drawVisit` and `drawRoute` no longer tell each other apart. - Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go` -- 2.52.0 From 88be57ea4604d949a0da6ed59aa1735a57311118 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 18:44:33 +0200 Subject: [PATCH 037/966] Stop the Layout claiming the hold runs only at render --- README.md | 2 +- readme_test.go | 3 +-- todo.md | 8 ++++---- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 19a5df8..53197b7 100644 --- a/README.md +++ b/README.md @@ -1328,7 +1328,7 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold at render: one draw per expansion for paths and operands, and the rows it pins +hold.go the hold: one draw per expansion for paths and operands, and the rows it pins holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways draw.go one reference draw per render and group: the draw set a render reads its paths through drawfence.go the load fences over reference draws: the drawGroup option, and the walk over a render's reads diff --git a/readme_test.go b/readme_test.go index a8238f8..40b0ea2 100644 --- a/readme_test.go +++ b/readme_test.go @@ -194,8 +194,7 @@ func TestReadmeTableExample(t *testing.T) { } } -// TestLayoutNamesEverySourceFile holds an added, renamed or split file to its Layout -// line, so the list a reader picks a file from stays the list on disk. +// TestLayoutNamesEverySourceFile holds an added, renamed or split file to its Layout line. func TestLayoutNamesEverySourceFile(t *testing.T) { src := readme(t) i := strings.Index(src, "## Layout") diff --git a/todo.md b/todo.md index bd4356c..7024cd2 100644 --- a/todo.md +++ b/todo.md @@ -26,12 +26,12 @@ This round, in order: is the load path's spine, and no file name carries it. - Name the `draw` stem apart: the free `pinned` in `holdfence.go` is not `(*draws).pinned` in `hold.go`, `compileFormat` is a method on both `*template` and - `*table`, and - `draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, `drawVisit` and - `drawRoute` no longer tell each other apart. + `*table`, and `draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, + `drawVisit` and `drawRoute` no longer tell each other apart. - Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go` hand-roll `drawSet{unnamed: draws{s: s}}` at three sites; name the lazy-map variant - if the difference is deliberate. + if the difference is deliberate, and say whether `draw.go`, 57 lines after the split, + still earns a file of its own. - Run the package doc's Go example under test, as `readme_test.go` runs the README's JSON blocks: `doc.go` and the README both print `Järvedsvägen 43\n891 77 Järved` for seed 42, and nothing renders either, so a draw that moves leaves both false. -- 2.52.0 From 48495944fa43ece6f1998d17f8a484cced264c43 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 18:48:21 +0200 Subject: [PATCH 038/966] Leave the fence division to the decision that records it --- Dockerfile | 2 +- holdfence.go | 5 +---- readme_test.go | 1 - todo.md | 11 ++++++++--- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/Dockerfile b/Dockerfile index f6bbac1..579851e 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -# The merge gate: `docker build .` fails if vet, formatting or tests fail, and CI +# The merge gate: `docker build .` fails if vet, complexity, formatting or tests fail, and CI # runs exactly this. Day-to-day, prefer `docker compose run` (bind-mounts source, # no rebuilds). GO_VERSION defaults to the latest stable Go; the lowest supported # version builds the `portable` stage, which every supported Go must pass: diff --git a/holdfence.go b/holdfence.go index f1b114a..017b6bc 100644 --- a/holdfence.go +++ b/holdfence.go @@ -8,10 +8,7 @@ import ( // heldCheck rejects every route to a held sibling name except the ones that read its // draw. An expansion holds one draw of that name; anything else that renders it draws -// again, and the two disagree. checkNoOverlap settles the spellings within one format -// (a token, an operand); this settles the rest — a reference, whether it sits in that -// format or in anything the format renders, however deep. A reference path is held -// for the whole render instead, which drawCheck fences. +// again, and the two disagree. func heldCheck(path string, n node) error { t, ok := n.(*template) if !ok || len(t.held) == 0 { diff --git a/readme_test.go b/readme_test.go index 40b0ea2..cb300bb 100644 --- a/readme_test.go +++ b/readme_test.go @@ -194,7 +194,6 @@ func TestReadmeTableExample(t *testing.T) { } } -// TestLayoutNamesEverySourceFile holds an added, renamed or split file to its Layout line. func TestLayoutNamesEverySourceFile(t *testing.T) { src := readme(t) i := strings.Index(src, "## Layout") diff --git a/todo.md b/todo.md index 7024cd2..0c4cfa7 100644 --- a/todo.md +++ b/todo.md @@ -30,8 +30,8 @@ This round, in order: `drawVisit` and `drawRoute` no longer tell each other apart. - Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go` hand-roll `drawSet{unnamed: draws{s: s}}` at three sites; name the lazy-map variant - if the difference is deliberate, and say whether `draw.go`, 57 lines after the split, - still earns a file of its own. + if the difference is deliberate, and put the draw set in one file: `draw.go` holds 57 + lines after the split, and `render.go` two of the three sites. - Run the package doc's Go example under test, as `readme_test.go` runs the README's JSON blocks: `doc.go` and the README both print `Järvedsvägen 43\n891 77 Järved` for seed 42, and nothing renders either, so a draw that moves leaves both false. @@ -269,7 +269,12 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Name the draw fences at one level in `doc.go`: the Vocabulary calls them `heldCheck`, `drawCheck` and `checkFamilies`, where `checkFamilies` runs under `drawCheck` beside `checkOwnFamily` and `checkColumnDraws`, which it names nowhere — so a reader looking - for the fence over a record's columns finds no entry reaching it. + for the fence over a record's columns finds no entry reaching it. `holdfence.go` and + `drawfence.go` now index the fences by file, so deleting the sentence closes it too. +- Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word + in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a + second copy that drifts. Split the comprehension rule while there, which packs the + gate, its exceptions, the round and the at-or-above-7.0 regime into one line. - Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates Go at `Dockerfile:21`, and the shipped JSON under `data/` is two-space, so the rule as written is one no Go file can break and every data file does. -- 2.52.0 From 31d0c4fb450e27347d575d653160d47b6e01fced Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 18:52:02 +0200 Subject: [PATCH 039/966] Wrap the gate comment where its lines wrap --- Dockerfile | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Dockerfile b/Dockerfile index 579851e..9ab91ec 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,7 +1,8 @@ -# The merge gate: `docker build .` fails if vet, complexity, formatting or tests fail, and CI -# runs exactly this. Day-to-day, prefer `docker compose run` (bind-mounts source, -# no rebuilds). GO_VERSION defaults to the latest stable Go; the lowest supported -# version builds the `portable` stage, which every supported Go must pass: +# The merge gate: `docker build .` fails if vet, complexity, formatting or tests +# fail, and CI runs exactly this. Day-to-day, prefer `docker compose run` +# (bind-mounts source, no rebuilds). GO_VERSION defaults to the latest stable Go; +# the lowest supported version builds the `portable` stage, which every supported +# Go must pass: # docker build --build-arg GO_VERSION=1.22.12 --target portable . ARG GO_VERSION=1.27.1 FROM golang:${GO_VERSION} AS portable -- 2.52.0 From 11ecc1e3654fc142f6db5723574ae3302f23a3c9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 20:56:10 +0200 Subject: [PATCH 040/966] Put the load path and the fence order in one file --- README.md | 4 ++-- data.go | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ graph.go | 52 ---------------------------------------------------- todo.md | 3 --- 4 files changed, 54 insertions(+), 57 deletions(-) diff --git a/README.md b/README.md index 53197b7..6c2f345 100644 --- a/README.md +++ b/README.md @@ -1334,7 +1334,7 @@ draw.go one reference draw per render and group: the draw set a render r drawfence.go the load fences over reference draws: the drawGroup option, and the walk over a render's reads family.go a family of linked tables: what a reference path reads of one, and the fence over paths into it reference.go reference sigils, and binding references across the tree -graph.go the render graph: edges, cycles, the repeat bound, tree walks +graph.go the graph the fences walk: containment and render edges, and the cycle and repeat fences builtins.go the {name()} function registry and its implementations layout.go date and time layouts: the instants one is proved against, and the two samples checksum.go the check characters a derivation appends, and the IBAN they sit inside @@ -1342,7 +1342,7 @@ transform.go the builtins that rewrite an operand's value, and the ASCII fold calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype value.go the value proof: what a typed column or calc operand holds, checked at load -data.go data loading: fs.FS folders/files -> namespace tree, multi-source merge +data.go the load path: fs.FS folders/files -> namespace tree, multi-source merge, then the fences in order cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) diff --git a/data.go b/data.go index b6549b1..3ad356f 100644 --- a/data.go +++ b/data.go @@ -193,3 +193,55 @@ func mergeChildren(dst, src map[string]node) { dst[k] = v } } + +// nodeScope is the set of nodes one validation pass covers: a whole loaded tree, +// or a single inline node. +type nodeScope func(fn func(path string, n node) error) error + +func treeScope(root map[string]node) nodeScope { + return func(fn func(path string, n node) error) error { return walkNodes(root, fn) } +} + +func inlineScope(n node, label string) nodeScope { + return func(fn func(path string, m node) error) error { return eachNode(n, label, fn) } +} + +func checkScope(s nodeScope) error { + if err := checkColumns(s); err != nil { + return err + } + return checkRenders(s) +} + +// checkRenders runs the per-node fences over a scope, each over the whole scope +// before the next, so which of several broken nodes is reported does not depend on +// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks +// terminate. +func checkRenders(s nodeScope) error { + mem := reachMemo{} + if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { + return err + } + if err := s(heldCheck); err != nil { + return err + } + fence := &drawCheck{} + refs := false + if err := s(func(path string, n node) error { + if t, ok := n.(*template); ok && len(t.refs) > 0 { + refs = true + } + return fence.checkDrawGroup(path, n) + }); err != nil { + return err + } + if refs { + if err := s(fence.checkDraws); err != nil { + return err + } + if err := s(fence.checkRecordDraws); err != nil { + return err + } + } + return s((&valueProof{}).checkDatatype) +} diff --git a/graph.go b/graph.go index 681cc9d..d0fcf79 100644 --- a/graph.go +++ b/graph.go @@ -185,58 +185,6 @@ func pathLeaves(n node, tail []string) []node { return out } -// nodeScope is the set of nodes one validation pass covers: a whole loaded tree, -// or a single inline node. -type nodeScope func(fn func(path string, n node) error) error - -func treeScope(root map[string]node) nodeScope { - return func(fn func(path string, n node) error) error { return walkNodes(root, fn) } -} - -func inlineScope(n node, label string) nodeScope { - return func(fn func(path string, m node) error) error { return eachNode(n, label, fn) } -} - -func checkScope(s nodeScope) error { - if err := checkColumns(s); err != nil { - return err - } - return checkRenders(s) -} - -// checkRenders runs the per-node fences over a scope, each over the whole scope -// before the next, so which of several broken nodes is reported does not depend on -// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks -// terminate. -func checkRenders(s nodeScope) error { - mem := reachMemo{} - if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { - return err - } - if err := s(heldCheck); err != nil { - return err - } - fence := &drawCheck{} - refs := false - if err := s(func(path string, n node) error { - if t, ok := n.(*template); ok && len(t.refs) > 0 { - refs = true - } - return fence.checkDrawGroup(path, n) - }); err != nil { - return err - } - if refs { - if err := s(fence.checkDraws); err != nil { - return err - } - if err := s(fence.checkRecordDraws); err != nil { - return err - } - } - return s((&valueProof{}).checkDatatype) -} - type reachMemo map[node]int func (m reachMemo) of(n node) int { diff --git a/todo.md b/todo.md index 0c4cfa7..19215bf 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Move `nodeScope`, `checkScope` and `checkRenders` out of `graph.go` and refresh the - README Layout lines for `graph.go` and `data.go`: the order every load fence runs in - is the load path's spine, and no file name carries it. - Name the `draw` stem apart: the free `pinned` in `holdfence.go` is not `(*draws).pinned` in `hold.go`, `compileFormat` is a method on both `*template` and `*table`, and `draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, -- 2.52.0 From f94546edaa2f7c3ed11b2215e015c161f220574c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:00:27 +0200 Subject: [PATCH 041/966] Say what the graph and the load path each hold --- README.md | 4 ++-- data.go | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 6c2f345..6f8b984 100644 --- a/README.md +++ b/README.md @@ -1334,7 +1334,7 @@ draw.go one reference draw per render and group: the draw set a render r drawfence.go the load fences over reference draws: the drawGroup option, and the walk over a render's reads family.go a family of linked tables: what a reference path reads of one, and the fence over paths into it reference.go reference sigils, and binding references across the tree -graph.go the graph the fences walk: containment and render edges, and the cycle and repeat fences +graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences builtins.go the {name()} function registry and its implementations layout.go date and time layouts: the instants one is proved against, and the two samples checksum.go the check characters a derivation appends, and the IBAN they sit inside @@ -1342,7 +1342,7 @@ transform.go the builtins that rewrite an operand's value, and the ASCII fold calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype value.go the value proof: what a typed column or calc operand holds, checked at load -data.go the load path: fs.FS folders/files -> namespace tree, multi-source merge, then the fences in order +data.go the load path: fs.FS folders/files -> merged namespace tree, and the fence order every scope runs cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) diff --git a/data.go b/data.go index 3ad356f..5299de3 100644 --- a/data.go +++ b/data.go @@ -215,8 +215,8 @@ func checkScope(s nodeScope) error { // checkRenders runs the per-node fences over a scope, each over the whole scope // before the next, so which of several broken nodes is reported does not depend on -// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks -// terminate. +// the walk. Its walks terminate only where nothing renders itself, so run it over a +// loaded tree after checkNoCycles. func checkRenders(s nodeScope) error { mem := reachMemo{} if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { -- 2.52.0 From 106ad5d88167a15a29d345ab34f4b6423c74a2b9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:04:30 +0200 Subject: [PATCH 042/966] Count the fence passes the load path actually makes --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 6f8b984..2bdc176 100644 --- a/README.md +++ b/README.md @@ -747,11 +747,11 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance Each file is parsed, validated and weight-indexed once, in `New`. Proving the draw -fences adds one pass over the loaded tree, and walks what a render reads only where -data binds a reference, so a set that binds none pays for the pass alone. A `Fake` call -then costs about what its output costs: an unweighted pick is O(1) whatever the -list's length, a weighted one O(log n), and long formats, deep nesting and many -tokens add cost in proportion to the output. +fences adds a pass over the loaded tree per fence, and walks what a render reads only +where data binds a reference, so a set that binds none pays for the passes alone. A +`Fake` call then costs about what its output costs: an unweighted pick is O(1) +whatever the list's length, a weighted one O(log n), and long formats, deep nesting +and many tokens add cost in proportion to the output. ## Versioning @@ -1342,7 +1342,7 @@ transform.go the builtins that rewrite an operand's value, and the ASCII fold calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype value.go the value proof: what a typed column or calc operand holds, checked at load -data.go the load path: fs.FS folders/files -> merged namespace tree, and the fence order every scope runs +data.go the load path: fs.FS folders/files -> merged namespace tree, and the fence order a scope runs cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) -- 2.52.0 From b6f074a72a366fb491543775acfbfb6522a93f14 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:07:54 +0200 Subject: [PATCH 043/966] Say where a table's row index is really built --- README.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 2bdc176..c49bbd8 100644 --- a/README.md +++ b/README.md @@ -746,12 +746,14 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance -Each file is parsed, validated and weight-indexed once, in `New`. Proving the draw -fences adds a pass over the loaded tree per fence, and walks what a render reads only -where data binds a reference, so a set that binds none pays for the passes alone. A -`Fake` call then costs about what its output costs: an unweighted pick is O(1) -whatever the list's length, a weighted one O(log n), and long formats, deep nesting -and many tokens add cost in proportion to the output. +Each file is parsed, validated and weight-indexed once, in `New`, bar a table with a +name or a parent column, which indexes its rows on the first `Fake` that selects or +descends through it. Proving the draw fences adds a pass over the loaded tree per +fence, and walks what a render reads only where data binds a reference, so a set that +binds none pays for the passes alone. A `Fake` call then costs about what its output +costs: an unweighted pick is O(1) whatever the list's length, a weighted one +O(log n), and long formats, deep nesting and many tokens add cost in proportion to +the output. ## Versioning -- 2.52.0 From f54bead1ed62167b66e1d28ef579b6fdaca80e50 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:37:25 +0200 Subject: [PATCH 044/966] Fence a function that spells a method --- naming_test.go | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 naming_test.go diff --git a/naming_test.go b/naming_test.go new file mode 100644 index 0000000..04604fc --- /dev/null +++ b/naming_test.go @@ -0,0 +1,46 @@ +package fejkdata + +import ( + "go/ast" + "sort" + "testing" +) + +// TestNoFunctionSpellsAMethod holds a call to one unit. A method is written with +// its receiver and a function bare, so one name on both reads alike at a call site +// and greps as one unit; two types may share a method name, since the receiver +// standing before it says which. +func TestNoFunctionSpellsAMethod(t *testing.T) { + _, files := sourceFiles(t) + funcs, methods := declaredCalls(files) + names := make([]string, 0, len(funcs)) + for name := range funcs { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + if recv := methods[name]; recv != "" { + t.Errorf("func %s and %s.%s are two units under one name; name them apart", name, recv, name) + } + } +} + +// declaredCalls is every name the package declares as a function, and every name +// it declares as a method with one type declaring it. +func declaredCalls(files []*ast.File) (funcs map[string]bool, methods map[string]string) { + funcs, methods = map[string]bool{}, map[string]string{} + for _, f := range files { + for _, decl := range f.Decls { + d, isFunc := decl.(*ast.FuncDecl) + if !isFunc { + continue + } + if recv := receiverType(d.Recv); recv != "" { + methods[d.Name.Name] = recv + continue + } + funcs[d.Name.Name] = true + } + } + return funcs, methods +} -- 2.52.0 From d128621fc18165f5e3f42d2805851e547c738d5d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:41:44 +0200 Subject: [PATCH 045/966] Name a render's holds apart from the draw fence --- README.md | 2 +- doc.go | 8 +++---- draw.go | 57 ------------------------------------------------ drawfence.go | 16 +++++++------- family.go | 8 +++---- fejkdata.go | 2 +- hold.go | 36 +++++++++++++++--------------- hold_test.go | 2 +- holdfence.go | 6 ++--- holdset.go | 57 ++++++++++++++++++++++++++++++++++++++++++++++++ node.go | 2 +- path.go | 6 ++--- record.go | 16 +++++++------- render.go | 30 ++++++++++++------------- struct.go | 4 ++-- table.go | 9 ++++---- table_test.go | 4 ++-- template_test.go | 2 +- todo.md | 12 ++++------ 19 files changed, 138 insertions(+), 141 deletions(-) delete mode 100644 draw.go create mode 100644 holdset.go diff --git a/README.md b/README.md index c49bbd8..d7ac5d0 100644 --- a/README.md +++ b/README.md @@ -1332,7 +1332,7 @@ inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplat template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format hold.go the hold: one draw per expansion for paths and operands, and the rows it pins holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways -draw.go one reference draw per render and group: the draw set a render reads its paths through +holdset.go one reference draw per render and group: the hold set a render reads its paths through drawfence.go the load fences over reference draws: the drawGroup option, and the walk over a render's reads family.go a family of linked tables: what a reference path reads of one, and the fence over paths into it reference.go reference sigils, and binding references across the tree diff --git a/doc.go b/doc.go index 2f8e23d..1d8c2fe 100644 --- a/doc.go +++ b/doc.go @@ -19,16 +19,16 @@ package fejkdata // - draw — one pick from a choice, or one row taken from a table, and the value a // read produced: `pick`, `drawn`, `table.draw`, `draw`. // - expansion — one render of one format: `expand`. -// - render — what owns one `drawSet`, and so what one reference draw spans: +// - render — what owns one `holdSet`, and so what one reference draw spans: // `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. // - hold — keeping one draw of a name, so every route to it reads that value: -// `template.held`, `draws`, `readField`. +// `template.held`, `hold`, `readField`. // - draw group — reference draws held apart inside one render: // `template.drawGroup` as the data spells it, `template.drawGroupKey` as a -// render reads it, `drawSet`, `drawScope`. -// - pin — fixing which row of a table the render uses: `draws.pin`, `draws.rowOf`. +// render reads it, `holdSet`, `renderScope`. +// - pin — fixing which row of a table the render uses: `hold.pin`, `hold.rowOf`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. // - whole — a read that lands on a `row`, so the row renders through its table's diff --git a/draw.go b/draw.go deleted file mode 100644 index 0fdb478..0000000 --- a/draw.go +++ /dev/null @@ -1,57 +0,0 @@ -package fejkdata - -import ( - "strings" -) - -// drawSet is one render's reference draws: the unnamed draw group's, and each named one's. -type drawSet struct { - unnamed draws - named map[string]*draws -} - -// drawScope is where a render reads its reference paths: a draw set, in the draw group of the -// template rendering, and the row a table's format is rendering, which its columns read. -type drawScope struct { - set *drawSet - group string - t *table - row int -} - -// newDrawSet makes the unnamed group's maps where the set is declared, keeping them on that frame's -// stack for a render that reads through them; a zero drawSet makes them on its first read instead. -func newDrawSet(s *session) drawSet { - return drawSet{unnamed: draws{variant: map[string]node{}, value: map[string]draw{}, s: s}} -} - -// renderOnce renders n as one render, over draws of its own. -func renderOnce(s *session, n node) string { - set := drawSet{unnamed: draws{s: s}} - return render(s, n, drawScope{set: &set}) -} - -// in is the scope t renders in: its draw group where it names one, else its caller's. -func (sc drawScope) in(t *template) drawScope { - if t.drawGroupKey != "" { - sc.group = t.drawGroupKey - } - return sc -} - -// draws is the set's draws for sc's draw group. The session is passed in rather than -// read out of the set: copying it from there would leak the set's maps to the heap. -func (sc drawScope) draws(s *session) *draws { - if sc.group == "" { - return &sc.set.unnamed - } - d, drew := sc.set.named[sc.group] - if !drew { - if sc.set.named == nil { - sc.set.named = map[string]*draws{} - } - d = &draws{variant: map[string]node{}, value: map[string]draw{}, s: s} - sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's draw set, to the heap - } - return d -} diff --git a/drawfence.go b/drawfence.go index 4e51eaf..64a7e97 100644 --- a/drawfence.go +++ b/drawfence.go @@ -235,8 +235,8 @@ func checkColumnDraws(t *template, columns []string) error { // drawWalk gathers the references one render reads, by draw group, for check to compare. type drawWalk struct { reads []pathRead - read map[drawKey]bool - seen map[drawVisit]bool + read map[readKey]bool + seen map[nodeVisit]bool } // drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the @@ -248,7 +248,7 @@ type drawAt struct { drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true } -// rowSet stands in at load for the rows (*draws).pin holds at render: a path read enters a row with +// rowSet stands in at load for the rows (*hold).pin holds at render: a path read enters a row with // its ancestors, a whole read enters the drawn row alone. type rowSet []tablePin @@ -311,26 +311,26 @@ type pathRead struct { tr *tableRead // set where the reference names a table } -type drawVisit struct { +type nodeVisit struct { n node group string alt string } -type drawKey struct { +type readKey struct { group string path string alt string } func newDrawWalk() *drawWalk { - return &drawWalk{read: map[drawKey]bool{}, seen: map[drawVisit]bool{}} + return &drawWalk{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}} } // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops // there. func (w *drawWalk) walk(n node, at drawAt) { - v := drawVisit{n, at.group, at.alt.key()} + v := nodeVisit{n, at.group, at.alt.key()} if w.seen[v] { return } @@ -377,7 +377,7 @@ func (s rowSet) excludes(cell *template) bool { func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { if a, reads := refRead(from, e.label); reads { tr := tableReadOf(from.(*template).fields[a.key], a, e.to) - if k := (drawKey{at.group, a.path, at.alt.key()}); !w.read[k] { + if k := (readKey{at.group, a.path, at.alt.key()}); !w.read[k] { w.read[k] = true w.reads = append(w.reads, pathRead{at, a, tr}) } diff --git a/family.go b/family.go index 73443e4..a495e0c 100644 --- a/family.go +++ b/family.go @@ -11,7 +11,7 @@ import ( // it lands on a row rendered whole. type tableRead struct { head *table - pins draws + pins hold drawn map[*table]bool sels []tableSel whole bool @@ -125,7 +125,7 @@ func replayPairs(reads []pathRead) error { if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { continue } - var d draws + var d hold if err := r.tr.replay(&d); err != nil { return conflict(r, err) } @@ -142,7 +142,7 @@ func conflict(r pathRead, err error) error { } // replay pins the read's rows into d, where they agree with the rows pinned before. -func (r *tableRead) replay(d *draws) error { +func (r *tableRead) replay(d *hold) error { var err error r.pins.each(func(t *table, row int) { if err == nil { @@ -178,7 +178,7 @@ func checkFamilyPair(a, b pathRead) error { } // drawnOf is a table the read draws that pins holds a row of, if any. -func (r *tableRead) drawnOf(pins *draws) *table { +func (r *tableRead) drawnOf(pins *hold) *table { var found *table pins.each(func(t *table, _ int) { if found == nil && r.drawn[t] { diff --git a/fejkdata.go b/fejkdata.go index 9fbf0f0..3bcb894 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -35,7 +35,7 @@ type Generator struct { rand *session categories map[string]node root folder // the categories as the node a path walks from, owned here so a walk allocates none - set drawSet // one Fake's draws, owned here so a walk pinning rows keeps them off the heap + set holdSet // one Fake's hold set, owned here so a walk pinning rows keeps them off the heap records map[node]recordShape structs map[reflect.Type]structResult } diff --git a/hold.go b/hold.go index a54e183..555e583 100644 --- a/hold.go +++ b/hold.go @@ -11,11 +11,11 @@ type tablePin struct { row int } -// draws is what has already been drawn for held names: the variant each was drawn as, +// hold is what has already been drawn for held names: the variant each was drawn as, // so every path under it reads one row, and the draw each read made, by its one // spelling, so the same read written twice reads one draw. An expansion keeps one for // its sibling names, and a render one per group for its reference paths. -type draws struct { +type hold struct { variant map[string]node value map[string]draw pins [8]tablePin // the rows pinned, inline so a render over a country's five-deep geo tree stays off the heap @@ -31,7 +31,7 @@ type draw struct { } // pinned is the row the render pinned for t, if any. -func (d *draws) pinned(t *table) (int, bool) { +func (d *hold) pinned(t *table) (int, bool) { for _, p := range d.pins[:d.npins] { if p.t == t { return p.row, true @@ -42,7 +42,7 @@ func (d *draws) pinned(t *table) (int, bool) { } // mustRow is the row pinned for t, which the walk reaching a column pinned. -func (d *draws) mustRow(t *table) int { +func (d *hold) mustRow(t *table) int { r, ok := d.pinned(t) if !ok { panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) @@ -51,7 +51,7 @@ func (d *draws) mustRow(t *table) int { } // pin pins row r of t, and the rows of t's ancestors it links to. -func (d *draws) pin(t *table, r int) { +func (d *hold) pin(t *table, r int) { for { if _, done := d.pinned(t); done { return @@ -73,7 +73,7 @@ func (d *draws) pin(t *table, r int) { } // each calls fn for every pinned row, in pin order, the spilled ones by table name. -func (d *draws) each(fn func(t *table, r int)) { +func (d *hold) each(fn func(t *table, r int)) { for _, p := range d.pins[:d.npins] { fn(p.t, p.row) } @@ -90,7 +90,7 @@ func (d *draws) each(fn func(t *table, r int)) { // rowOf is the render's row of t: the one pinned, else one drawn inside the // nearest pinned ancestor — its parent drawn inside that first where the ancestor // is further up — or over the whole table, and pinned with its ancestors. -func (d *draws) rowOf(t *table) int { +func (d *hold) rowOf(t *table) int { if r, ok := d.pinned(t); ok { return r } @@ -113,7 +113,7 @@ func (d *draws) rowOf(t *table) int { } // pinRow pins row r of t where it agrees with the rows pinned before it. -func (d *draws) pinRow(t *table, r int) error { +func (d *hold) pinRow(t *table, r int) error { if pr, ok := d.pinned(t); ok && pr != r { return fmt.Errorf("%s and %s are two rows of %s", t.selectorSpelling(pr), t.selectorSpelling(r), t.path) } @@ -127,7 +127,7 @@ func (d *draws) pinRow(t *table, r int) error { } // selectRow pins the row a selector names. -func (d *draws) selectRow(t *table, sel string) error { +func (d *hold) selectRow(t *table, sel string) error { r, err := t.find(sel, d) if err != nil { return err @@ -136,7 +136,7 @@ func (d *draws) selectRow(t *table, sel string) error { } // inside keeps the rows of t that sit inside every pinned ancestor. -func (d *draws) inside(t *table, rows []int) []int { +func (d *hold) inside(t *table, rows []int) []int { for a := t.parentT; a != nil; a = a.parentT { pa, ok := d.pinned(a) if !ok { @@ -160,14 +160,14 @@ func (d *draws) inside(t *table, rows []int) []int { // gives one value, and a shown operand is the operand computed. Every other name is // drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and // linkRefs prove every step, so the walk cannot fail. -func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw { +func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { if !t.held[a.key] { if len(a.tail) > 0 { panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key)) } return draw{text: render(s, t.fields[a.key], sc)} } - d := readScope(s, held, sc, a) + d := readHold(s, held, sc, a) if r, done := d.value[a.path]; done { return r } @@ -202,26 +202,26 @@ func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw { return r } -// readScope is the draws a held read keeps its draw in: for a reference that reads a path, -// the render's draws for its group, so its draw spans the render; for a sibling, or a +// readHold is the hold a held read keeps its draw in: for a reference that reads a path, +// the render's hold for its group, so its draw spans the render; for a sibling, or a // reference read whole, held. -func readScope(s *session, held *draws, sc drawScope, a arm) *draws { +func readHold(s *session, held *hold, sc renderScope, a arm) *hold { if isRef(a.key) && len(a.tail) > 0 { - return sc.draws(s) + return sc.hold(s) } return held } // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. -func renderLeaf(s *session, n node, sc drawScope) draw { +func renderLeaf(s *session, n node, sc renderScope) draw { n = drawn(s, n) if _, isNull := n.(*null); isNull { return draw{null: true} } r := draw{text: render(s, n, sc)} if t, _ := n.(*template); t != nil && t.readsColumn != nil { - r.null = sc.in(t).draws(s).value[t.readsColumn.a.path].null + r.null = sc.in(t).hold(s).value[t.readsColumn.a.path].null } return r } diff --git a/hold_test.go b/hold_test.go index 53f6f9f..301feca 100644 --- a/hold_test.go +++ b/hold_test.go @@ -766,6 +766,6 @@ func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) { t.Fatal("not a template") } mustPanic(t, "unheld arm with a path", func() { - readField(engine(1).rand, tm, nil, drawScope{}, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) + readField(engine(1).rand, tm, nil, renderScope{}, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) }) } diff --git a/holdfence.go b/holdfence.go index 017b6bc..562c915 100644 --- a/holdfence.go +++ b/holdfence.go @@ -45,10 +45,10 @@ func heldHeads(t *template) []string { return heads } -// pinned collects what the hold of head answers for: a path pins the levels it +// heldNodes is what the hold of head answers for: a path pins the levels it // passes through and the leaf it lands on, an operand exactly the value its render // produces. -func pinned(t *template, head string, readers []reader) map[node]bool { +func heldNodes(t *template, head string, readers []reader) map[node]bool { held := map[node]bool{} if _, isPath := t.bound[head]; !isPath { operandDraw(t.fields[head], held) @@ -66,7 +66,7 @@ func pinned(t *template, head string, readers []reader) map[node]bool { // holding it. One seen set across the edges: a node that cannot reach the level // cannot reach it by another route either, so it is walked once. func checkHeadHeld(t *template, head string, readers []reader) error { - held := pinned(t, head, readers) + held := heldNodes(t, head, readers) if len(held) == 0 { return nil // a fixed head holds nothing to reach } diff --git a/holdset.go b/holdset.go new file mode 100644 index 0000000..69d82ca --- /dev/null +++ b/holdset.go @@ -0,0 +1,57 @@ +package fejkdata + +import ( + "strings" +) + +// holdSet is one render's reference draws: the unnamed draw group's, and each named one's. +type holdSet struct { + unnamed hold + named map[string]*hold +} + +// renderScope is where a render reads its reference paths: a hold set, in the draw group of the +// template rendering, and the row a table's format is rendering, which its columns read. +type renderScope struct { + set *holdSet + group string + t *table + row int +} + +// newHoldSet makes the unnamed group's maps where the set is declared, keeping them on that frame's +// stack for a render that reads through them; a zero holdSet makes them on its first read instead. +func newHoldSet(s *session) holdSet { + return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}, s: s}} +} + +// renderOnce renders n as one render, over draws of its own. +func renderOnce(s *session, n node) string { + set := holdSet{unnamed: hold{s: s}} + return render(s, n, renderScope{set: &set}) +} + +// in is the scope t renders in: its draw group where it names one, else its caller's. +func (sc renderScope) in(t *template) renderScope { + if t.drawGroupKey != "" { + sc.group = t.drawGroupKey + } + return sc +} + +// hold is the set's hold for sc's draw group. The session is passed in rather than +// read out of the set: copying it from there would leak the set's maps to the heap. +func (sc renderScope) hold(s *session) *hold { + if sc.group == "" { + return &sc.set.unnamed + } + d, drew := sc.set.named[sc.group] + if !drew { + if sc.set.named == nil { + sc.set.named = map[string]*hold{} + } + d = &hold{variant: map[string]node{}, value: map[string]draw{}, s: s} + sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's hold set, to the heap + } + return d +} diff --git a/node.go b/node.go index 6a3f167..c9bb65c 100644 --- a/node.go +++ b/node.go @@ -47,7 +47,7 @@ func (*null) isNode() {} // run: where the format holds a reference, that binds it and compiles the format // again. type template struct { - // Filled by `compileString`, `compileTemplate` and `table.compileFormat`: + // Filled by `compileString`, `compileTemplate` and `table.compileWhole`: format string fields map[string]node // `linkTemplateRefs` adds each bound reference here repeat int diff --git a/path.go b/path.go index 5d85b8b..20fa50e 100644 --- a/path.go +++ b/path.go @@ -135,7 +135,7 @@ type pathWalk struct { choice func(c *choice, rest []string) ([]node, error) level func(t *template, rest []string) error leaf func(n node) error - pins *draws + pins *hold } // walkPath descends tail from n and returns the node it ends at: a folder or @@ -272,7 +272,7 @@ func (t *table) step(tail []string) (column node, child *table, err error) { // readRow pins the row a path reads of t: the one its selector names, or, where the // walk draws, one drawn where the path reads into the table. -func readRow(d *draws, t *table, sel string, draw bool) error { +func readRow(d *hold, t *table, sel string, draw bool) error { if sel != "" { return d.selectRow(t, sel) } @@ -312,7 +312,7 @@ func unreachableInChoice(c *choice, want string) error { // carries a repeat or a drawGroup, which one draw of it could not apply. So a path // that validates here resolves on every render, and a typo is a New-time error. func checkPath(n node, tail []string, level string) error { - var pins draws + var pins hold _, err := walkPath(n, tail, pathWalk{ choice: func(c *choice, rest []string) ([]node, error) { if err := carriedByAll(c, rest); err != nil { diff --git a/record.go b/record.go index 9a2621d..1f61bf1 100644 --- a/record.go +++ b/record.go @@ -130,8 +130,8 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) } - set := newDrawSet(f.rand) - sc := drawScope{set: &set} + set := newHoldSet(f.rand) + sc := renderScope{set: &set} if t, isTable := n.(*table); isTable { if n, err = tableRecord(f.rand, t, tail, sc); err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) @@ -152,14 +152,14 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { // tableRecord walks a path's tail from a table to the table whose row is the record, // pinning the rows it selects or draws. -func tableRecord(s *session, t *table, tail []string, sc drawScope) (node, error) { +func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, error) { n, err := descend(s, t, tail, sc) if err != nil { return nil, err } switch n := n.(type) { case *table: - sc.draws(s).rowOf(n) + sc.hold(s).rowOf(n) return n, nil case *row: return n.t, nil @@ -204,8 +204,8 @@ type RecordTemplate struct { func (t *RecordTemplate) Fake() *Record { t.g.mu.Lock() defer t.g.mu.Unlock() - set := newDrawSet(t.g.rand) - return renderRecord(t.g.rand, t.t, t.columns, drawScope{set: &set}) + set := newHoldSet(t.g.rand) + return renderRecord(t.g.rand, t.t, t.columns, renderScope{set: &set}) } // NewRecordTemplate compiles an inline record — a JSON object with a format and @@ -262,10 +262,10 @@ func recordOf(n node) (*template, []Column, error) { // renderRecord draws each column once, in the name order recordOf fixed, as one render // over sc's draws; a table's columns read the row pinned there. -func renderRecord(s *session, t *template, columns []Column, sc drawScope) *Record { +func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.table != nil { - sc.t, sc.row = t.table, sc.draws(s).mustRow(t.table) + sc.t, sc.row = t.table, sc.hold(s).mustRow(t.table) } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { diff --git a/render.go b/render.go index a1c15c9..649b2c5 100644 --- a/render.go +++ b/render.go @@ -23,8 +23,8 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } - f.set = drawSet{unnamed: draws{s: f.rand}} - sc := drawScope{set: &f.set} + f.set = holdSet{unnamed: hold{s: f.rand}} + sc := renderScope{set: &f.set} f.root.children = f.categories n, err := descend(f.rand, &f.root, segments, sc) if err != nil { @@ -42,19 +42,19 @@ func (f *Generator) Fake(path string) (string, error) { // choice consumes no segment, so the rest of the path must be one every variant // carries before a variant is picked — a path that resolves at all resolves on // every call. -func descend(s *session, root node, segments []string, sc drawScope) (node, error) { +func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // Walked once without drawing first, so a path that fails moves no seeded stream. - var probe draws + var probe hold if _, err := walkPath(root, segments, pathWalk{pins: &probe}); err != nil { return nil, err } - return walkPath(root, segments, pathWalk{pins: sc.draws(s)}) + return walkPath(root, segments, pathWalk{pins: sc.hold(s)}) } // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over draws of its own. -func render(s *session, n node, sc drawScope) string { +func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: return render(s, pick(s, n), sc) @@ -64,11 +64,11 @@ func render(s *session, n node, sc drawScope) string { sc.t, sc.row = n, n.draw(s) return expand(s, n.format, sc) case *row: - sc.t, sc.row = n.t, sc.draws(s).mustRow(n.t) + sc.t, sc.row = n.t, sc.hold(s).mustRow(n.t) return expand(s, n.t.format, sc) case *column: if sc.t != n.t { - sc.t, sc.row = n.t, sc.draws(s).mustRow(n.t) + sc.t, sc.row = n.t, sc.hold(s).mustRow(n.t) } if cell := n.t.cellNode(sc.row, n.i); cell != nil { return render(s, cell, sc) @@ -97,19 +97,19 @@ func render(s *session, n node, sc drawScope) string { } // expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into -// render's loop, its draw set would move to the heap. +// render's loop, its hold set would move to the heap. // //go:noinline func expandAnew(s *session, t *template) string { - set := drawSet{unnamed: draws{s: s}} - return expand(s, t, drawScope{set: &set}) + set := holdSet{unnamed: hold{s: s}} + return expand(s, t, renderScope{set: &set}) } // pick selects one item. Uniform choices are O(1); weighted choices are an // O(log n) search over precomputed cumulative weights. compile guarantees a // non-empty choice and a finite positive total, so the index is always in range. // The session is concrete rather than the rng interface, which would make the -// walk that draws through it leak its draw set to the heap. +// walk that draws through it leak its hold set to the heap. func pick(s *session, c *choice) node { if c.cum == nil { return c.items[s.IntN(len(c.items))] @@ -119,15 +119,15 @@ func pick(s *session, c *choice) node { // expand renders a template's compiled ops. compile validated every token, so this // cannot fail. -func expand(s *session, t *template, sc drawScope) string { +func expand(s *session, t *template, sc renderScope) string { var b strings.Builder b.Grow(t.grow) // One draw per held name, for this expansion only: a nested template and each // repeat iteration get their own, since each is its own expansion. A reference // path reads the render's draws in sc instead. - var held *draws + var held *hold if t.heldLocal { - held = &draws{ + held = &hold{ variant: make(map[string]node, len(t.held)), value: make(map[string]draw, len(t.held)), } diff --git a/struct.go b/struct.go index 617a460..0a9cd91 100644 --- a/struct.go +++ b/struct.go @@ -349,8 +349,8 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro // fill draws the record into v's tagged fields, then each nested struct as a record of its own. func (s *structShape) fill(sess *session, v reflect.Value) { if s.record != nil { - set := newDrawSet(sess) - for i, c := range renderRecord(sess, s.record, s.columns, drawScope{set: &set}).columns { + set := newHoldSet(sess) + for i, c := range renderRecord(sess, s.record, s.columns, renderScope{set: &set}).columns { setColumn(fieldAt(v, s.fields[i]), c) } } diff --git a/table.go b/table.go index 987fee8..1227bcf 100644 --- a/table.go +++ b/table.go @@ -103,7 +103,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab if err := t.checkCells(); err != nil { return nil, fmt.Errorf("%s: %w", o.rows, err) } - if err := t.compileFormat(o.format); err != nil { + if err := t.compileWhole(o.format); err != nil { return nil, err } return t, nil @@ -321,8 +321,9 @@ func (t *table) checkCells() error { return nil } -// compileFormat compiles the format over the columns as its fields. -func (t *table) compileFormat(format string) error { +// compileWhole compiles the format a row renders whole through, over the columns +// as its fields. +func (t *table) compileWhole(format string) error { for _, name := range fieldTokens(format) { a := splitArm(name, nil) if _, ok := t.col[a.key]; !ok && !isRef(a.key) && a.key != "" { @@ -524,7 +525,7 @@ func (t *table) under(r int, a *table, pr int) bool { // find is the row a selector names: by key first, then by name, where a name // naming several rows resolves inside the ancestors pinned in d. -func (t *table) find(sel string, d *draws) (int, error) { +func (t *table) find(sel string, d *hold) (int, error) { if t.key < 0 && t.name < 0 { return 0, fmt.Errorf("%s has no key or name column to select a row by", t.path) } diff --git a/table_test.go b/table_test.go index 2f871ed..6b90f75 100644 --- a/table_test.go +++ b/table_test.go @@ -982,7 +982,7 @@ func TestRowSetSimulatesPinning(t *testing.T) { t.Errorf("a drawn %s enters %v, want its own row alone", entered.selectorSpelling(row), drawn) } s := rowSet(nil).enter(entered, row, true) - var pinned draws + var pinned hold held := 0 pinned.pin(entered, row) pinned.each(func(tbl *table, r int) { @@ -996,7 +996,7 @@ func TestRowSetSimulatesPinning(t *testing.T) { } for _, cand := range tables { for cr := 0; cr < cand.rows(); cr++ { - var beside draws + var beside hold beside.pin(entered, row) refused := beside.pinRow(cand, cr) != nil if got := s.excludes(&template{cellOf: cand, cellRow: cr}); got != refused { diff --git a/template_test.go b/template_test.go index da8f9ff..89f5e35 100644 --- a/template_test.go +++ b/template_test.go @@ -316,7 +316,7 @@ func TestGrowIsALowerBound(t *testing.T) { t.Fatalf("format %q did not compile to a template", format) } for i := 0; i < 50; i++ { - if got := len(expand(f.rand, tmpl, drawScope{set: &drawSet{}})); got < tmpl.grow { + if got := len(expand(f.rand, tmpl, renderScope{set: &holdSet{}})); got < tmpl.grow { t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) } } diff --git a/todo.md b/todo.md index 19215bf..a639190 100644 --- a/todo.md +++ b/todo.md @@ -21,14 +21,10 @@ name says whether a unit runs at `New` or at `Fake`. This round, in order: -- Name the `draw` stem apart: the free `pinned` in `holdfence.go` is not - `(*draws).pinned` in `hold.go`, `compileFormat` is a method on both `*template` and - `*table`, and `draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, - `drawVisit` and `drawRoute` no longer tell each other apart. -- Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go` - hand-roll `drawSet{unnamed: draws{s: s}}` at three sites; name the lazy-map variant - if the difference is deliberate, and put the draw set in one file: `draw.go` holds 57 - lines after the split, and `render.go` two of the three sites. +- Spell one fresh hold set one way: `newHoldSet` exists, and `render.go` and + `holdset.go` hand-roll `holdSet{unnamed: hold{s: s}}` at three sites; name the + lazy-map variant if the difference is deliberate, and put the hold set in one file: + `holdset.go` holds 57 lines after the split, and `render.go` two of the three sites. - Run the package doc's Go example under test, as `readme_test.go` runs the README's JSON blocks: `doc.go` and the README both print `Järvedsvägen 43\n891 77 Järved` for seed 42, and nothing renders either, so a draw that moves leaves both false. -- 2.52.0 From a52adc6b0d769d703865bab25d7ca458d327e2da Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:50:16 +0200 Subject: [PATCH 046/966] Widen the naming fence to the tests, and record what each stem names --- README.md | 13 ++++++++++++- doc_test.go | 13 +++++++++---- naming_test.go | 34 +++++++++++++++++++++------------- readme_test.go | 2 +- shape_test.go | 10 +++++----- template.go | 6 +++--- todo.md | 7 ++++--- 7 files changed, 55 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index d7ac5d0..04922b6 100644 --- a/README.md +++ b/README.md @@ -1042,7 +1042,7 @@ the Development section below, and who ships a register the four above then draw all, and making them anyway cost about a fifth of the cheapest render, so it makes them on the first read instead, at two heap allocations for a render that does share a draw. The allocation gate over a repeat of a reference path and over a named draw - group prices that, and pins the two measures that keep a draw set off the heap. + group prices that, and pins the two measures that keep a hold set off the heap. - **A category never references itself, and a record's fences run at load.** A category is one unit: a reference back into it — `{/users.first}` inside `users` — describes a draw other than the fields beside it, so `New` refuses it and the sibling path stays @@ -1077,6 +1077,17 @@ the Development section below, and who ships a register the four above then draw - **`Column` carries text, not a Go value.** `Value` is the rendered string beside `DataType` and `Null`, which each serializer writes as the load check proved it; a `Value any` would hand every caller a type switch. +- **`hold` names what a render keeps, `draw` what it draws.** Goal 2 wants a name to + reach one unit: `hold`, `holdSet` and `renderScope` are what a render holds while it + renders, and `draw`, `drawGroup` and `drawfence.go`'s units are the draws and the + load fence over them. So `holdset.go` is fenced by `drawfence.go`, which does not + pair by name as `hold.go` and `holdfence.go` do: that fence is over reference draws, + wherever a render keeps them. Valid while the load fences sit in files of their own. +- **A function may not spell a method; two types may.** A call writes a method with + its receiver and a function bare, so one name on both greps as one unit, which is + the reaching cost goal 2 counts, while the receiver before a shared method name + says which type answers. A type is outside the rule, never being called. + `TestNoFunctionSpellsAMethod` holds it over the package, its tests included. - **The package stays flat.** Go ties a package to one directory, so folders would split the API into packages. - **The performance gate asserts allocations, not wall-clock time.** `AllocsPerRun` diff --git a/doc_test.go b/doc_test.go index 4952312..77f9f23 100644 --- a/doc_test.go +++ b/doc_test.go @@ -47,20 +47,25 @@ func namesAPass(t *testing.T, where, text string, declared, funcs map[string]boo } } -func goFiles(t *testing.T) []string { +func packageDir(t *testing.T) *build.Package { t.Helper() pkg, err := build.ImportDir(".", 0) if err != nil { t.Fatalf("read the package directory: %v", err) } - return pkg.GoFiles + return pkg } func sourceFiles(t *testing.T) (*token.FileSet, []*ast.File) { + t.Helper() + return parseFiles(t, packageDir(t).GoFiles) +} + +func parseFiles(t *testing.T, names []string) (*token.FileSet, []*ast.File) { t.Helper() fset := token.NewFileSet() var files []*ast.File - for _, name := range goFiles(t) { + for _, name := range names { f, err := parser.ParseFile(fset, name, nil, parser.ParseComments) if err != nil { t.Fatalf("parse %s: %v", name, err) @@ -88,7 +93,7 @@ func vocabulary(t *testing.T, files []*ast.File) string { } // declaredSymbols is every name the package declares at the top level, a method and -// a struct field keyed under its type: draws.pin, tableRead.whole. +// a struct field keyed under its type: hold.pin, tableRead.whole. func declaredSymbols(files []*ast.File) map[string]bool { names := declaredFuncs(files) for _, f := range files { diff --git a/naming_test.go b/naming_test.go index 04604fc..075a58e 100644 --- a/naming_test.go +++ b/naming_test.go @@ -2,33 +2,41 @@ package fejkdata import ( "go/ast" + "slices" "sort" + "strings" "testing" ) -// TestNoFunctionSpellsAMethod holds a call to one unit. A method is written with -// its receiver and a function bare, so one name on both reads alike at a call site -// and greps as one unit; two types may share a method name, since the receiver -// standing before it says which. +// TestNoFunctionSpellsAMethod holds a call to one unit, over the tests as well, +// since a test file is this package too. func TestNoFunctionSpellsAMethod(t *testing.T) { - _, files := sourceFiles(t) - funcs, methods := declaredCalls(files) + funcs, methods := declaredCalls(namespaceFiles(t)) names := make([]string, 0, len(funcs)) for name := range funcs { names = append(names, name) } sort.Strings(names) for _, name := range names { - if recv := methods[name]; recv != "" { - t.Errorf("func %s and %s.%s are two units under one name; name them apart", name, recv, name) + if recv := methods[name]; len(recv) > 0 { + t.Errorf("func %s and %s are two units under one name; name them apart", name, strings.Join(recv, " and ")) } } } -// declaredCalls is every name the package declares as a function, and every name -// it declares as a method with one type declaring it. -func declaredCalls(files []*ast.File) (funcs map[string]bool, methods map[string]string) { - funcs, methods = map[string]bool{}, map[string]string{} +// namespaceFiles is every file declaring into the package namespace, its in-package +// tests included. +func namespaceFiles(t *testing.T) []*ast.File { + t.Helper() + pkg := packageDir(t) + _, files := parseFiles(t, slices.Concat(pkg.GoFiles, pkg.TestGoFiles)) + return files +} + +// declaredCalls is what each name is called as: a function, and the methods +// spelling it under the types declaring them. +func declaredCalls(files []*ast.File) (funcs map[string]bool, methods map[string][]string) { + funcs, methods = map[string]bool{}, map[string][]string{} for _, f := range files { for _, decl := range f.Decls { d, isFunc := decl.(*ast.FuncDecl) @@ -36,7 +44,7 @@ func declaredCalls(files []*ast.File) (funcs map[string]bool, methods map[string continue } if recv := receiverType(d.Recv); recv != "" { - methods[d.Name.Name] = recv + methods[d.Name.Name] = append(methods[d.Name.Name], recv+"."+d.Name.Name) continue } funcs[d.Name.Name] = true diff --git a/readme_test.go b/readme_test.go index cb300bb..49eebbc 100644 --- a/readme_test.go +++ b/readme_test.go @@ -211,7 +211,7 @@ func TestLayoutNamesEverySourceFile(t *testing.T) { listed[name] = true } } - for _, name := range goFiles(t) { + for _, name := range packageDir(t).GoFiles { if !listed[name] { t.Errorf("the README Layout names no %s, so nothing says what it holds", name) } diff --git a/shape_test.go b/shape_test.go index 665eb41..6107d07 100644 --- a/shape_test.go +++ b/shape_test.go @@ -42,14 +42,14 @@ func shippedShape(f *Generator) string { walk(join(prefix, name), n.children[name]) } case *choice: - facts[prefix] = reads(n) + facts[prefix] = readsFact(n) case *table: - facts[prefix] = "\tformat " + strconv.Quote(n.format.format) + tableFacts(n) + reads(n) + facts[prefix] = "\tformat " + strconv.Quote(n.format.format) + tableFacts(n) + readsFact(n) for _, name := range n.columns { facts[join(prefix, name)] = "\tstring" } case *template: - facts[prefix] = "\tformat " + strconv.Quote(n.format) + reads(n) + facts[prefix] = "\tformat " + strconv.Quote(n.format) + readsFact(n) if _, columns, err := recordOf(n); err == nil { for _, c := range columns { fact := "\t" + c.DataType.String() @@ -85,8 +85,8 @@ func tableFacts(t *table) string { return b.String() } -// reads names the categories any template under n references, sorted. -func reads(n node) string { +// readsFact names the categories any template under n references, sorted. +func readsFact(n node) string { set := map[string]bool{} var collect func(node) collect = func(n node) { diff --git a/template.go b/template.go index 41bf9b4..31f704d 100644 --- a/template.go +++ b/template.go @@ -369,7 +369,7 @@ type formatOps struct { heldLocal bool // some held name is kept by the expansion itself rather than the render's draws } -func (c *formatOps) hold(a arm, label string) { +func (c *formatOps) holdName(a arm, label string) { if c.held == nil { c.held = map[string]bool{} c.holder = map[string]string{} @@ -396,7 +396,7 @@ func (c *formatOps) function(body string, refs map[string]refBinding) { var operands []arm for _, operand := range tokenOperands(body) { a := splitArm(operand, refs) - c.hold(a, fmt.Sprintf("%s operand %q", name, operand)) + c.holdName(a, fmt.Sprintf("%s operand %q", name, operand)) operands = append(operands, a) } c.ops = append(c.ops, op{kind: 'b', call: builtins[name].prep(args), operands: operands}) @@ -406,7 +406,7 @@ func (c *formatOps) field(body string, refs map[string]refBinding) { arms := splitArms(body, refs) for _, a := range arms { if len(a.tail) > 0 { - c.hold(a, "token {"+a.name+"}") + c.holdName(a, "token {"+a.name+"}") } } c.ops = append(c.ops, op{kind: 'f', arms: arms}) diff --git a/todo.md b/todo.md index a639190..a31fd4d 100644 --- a/todo.md +++ b/todo.md @@ -15,9 +15,10 @@ scoring run names too little to steer one. A fresh run replaces the items below. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), eight of nine seats capped by Locality or Self-sufficiency, all -nine naming `hold.go`, `draw.go` and `family.go` first for reading cost: 1,138 lines in -which each file carries both a load-time fence and render-time machinery, so no file -name says whether a unit runs at `New` or at `Fake`. +nine naming `hold.go`, `draw.go` — now `holdset.go` and `drawfence.go` — and +`family.go` first for reading cost: 1,138 lines in which each file carries both a +load-time fence and render-time machinery, so no file name says whether a unit runs at +`New` or at `Fake`. This round, in order: -- 2.52.0 From 41dbbd3fdac5ca5049c221f6430aef76f9c184c4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:54:50 +0200 Subject: [PATCH 047/966] Say hold where the prose still said draws --- README.md | 6 +++--- holdset.go | 2 +- naming_test.go | 5 ++--- node.go | 2 +- perf_test.go | 4 ++-- record.go | 2 +- render.go | 4 ++-- table.go | 2 +- template.go | 2 +- todo.md | 4 ++-- 10 files changed, 16 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 04922b6..2c7e3c3 100644 --- a/README.md +++ b/README.md @@ -1080,9 +1080,9 @@ the Development section below, and who ships a register the four above then draw - **`hold` names what a render keeps, `draw` what it draws.** Goal 2 wants a name to reach one unit: `hold`, `holdSet` and `renderScope` are what a render holds while it renders, and `draw`, `drawGroup` and `drawfence.go`'s units are the draws and the - load fence over them. So `holdset.go` is fenced by `drawfence.go`, which does not - pair by name as `hold.go` and `holdfence.go` do: that fence is over reference draws, - wherever a render keeps them. Valid while the load fences sit in files of their own. + load fence over them, so `holdfence.go` fences the hold an expansion keeps and + `drawfence.go` the reference draws `holdset.go` keeps. Valid while the load fences + sit in files of their own. - **A function may not spell a method; two types may.** A call writes a method with its receiver and a function bare, so one name on both greps as one unit, which is the reaching cost goal 2 counts, while the receiver before a shared method name diff --git a/holdset.go b/holdset.go index 69d82ca..694107f 100644 --- a/holdset.go +++ b/holdset.go @@ -25,7 +25,7 @@ func newHoldSet(s *session) holdSet { return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}, s: s}} } -// renderOnce renders n as one render, over draws of its own. +// renderOnce renders n as one render, over a hold set of its own. func renderOnce(s *session, n node) string { set := holdSet{unnamed: hold{s: s}} return render(s, n, renderScope{set: &set}) diff --git a/naming_test.go b/naming_test.go index 075a58e..80d609a 100644 --- a/naming_test.go +++ b/naming_test.go @@ -8,8 +8,6 @@ import ( "testing" ) -// TestNoFunctionSpellsAMethod holds a call to one unit, over the tests as well, -// since a test file is this package too. func TestNoFunctionSpellsAMethod(t *testing.T) { funcs, methods := declaredCalls(namespaceFiles(t)) names := make([]string, 0, len(funcs)) @@ -19,7 +17,8 @@ func TestNoFunctionSpellsAMethod(t *testing.T) { sort.Strings(names) for _, name := range names { if recv := methods[name]; len(recv) > 0 { - t.Errorf("func %s and %s are two units under one name; name them apart", name, strings.Join(recv, " and ")) + sort.Strings(recv) + t.Errorf("func %s is spelled by %s too; name them apart", name, strings.Join(recv, " and ")) } } } diff --git a/node.go b/node.go index c9bb65c..afac977 100644 --- a/node.go +++ b/node.go @@ -74,7 +74,7 @@ type template struct { // held is every name drawn once per expansion: the bound levels above, plus the // siblings a {calc()} reads. nil when the format holds nothing (see expand). held map[string]bool - heldLocal bool // some held name is kept by the expansion itself, so expand makes its draws + heldLocal bool // some held name is kept by the expansion itself, so expand makes its hold // Filled by `linkTemplateRefs` and `keyDrawGroup`, from the assembled tree: refs map[string]refBinding // each reference the format reads -> what it is bound to diff --git a/perf_test.go b/perf_test.go index 771f46f..5864b55 100644 --- a/perf_test.go +++ b/perf_test.go @@ -69,12 +69,12 @@ func TestNoReferenceAllocRegression(t *testing.T) { t.Fatalf("New(%s): %v", s.name, err) } if allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") }); allocs > s.base*1.10 { - t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a draw set reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base) + t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a hold set reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base) } } } -// A table read pins a row in the render's draws and reads its cells in place. +// A table read pins a row in the render's hold and reads its cells in place. func TestNoTableAllocRegression(t *testing.T) { f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{ "region.json": {Data: []byte(`{"format":"{name}","rows":"region.tsv","key":"code","weight":"population"}`)}, diff --git a/record.go b/record.go index 1f61bf1..50fc7dd 100644 --- a/record.go +++ b/record.go @@ -261,7 +261,7 @@ func recordOf(n node) (*template, []Column, error) { } // renderRecord draws each column once, in the name order recordOf fixed, as one render -// over sc's draws; a table's columns read the row pinned there. +// over sc's hold; a table's columns read the row pinned there. func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.table != nil { diff --git a/render.go b/render.go index 649b2c5..2df2cc5 100644 --- a/render.go +++ b/render.go @@ -53,7 +53,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the -// render shares; each repeat iteration renders over draws of its own. +// render shares; each repeat iteration renders over a hold set of its own. func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: @@ -124,7 +124,7 @@ func expand(s *session, t *template, sc renderScope) string { b.Grow(t.grow) // One draw per held name, for this expansion only: a nested template and each // repeat iteration get their own, since each is its own expansion. A reference - // path reads the render's draws in sc instead. + // path reads the render's hold in sc instead. var held *hold if t.heldLocal { held = &hold{ diff --git a/table.go b/table.go index 1227bcf..e16afc9 100644 --- a/table.go +++ b/table.go @@ -463,7 +463,7 @@ func (t *table) indexed() *tableIndex { } // draw picks a row over the whole table. The session is concrete rather than the rng -// interface so that a walk holding the draws allocates nothing. +// interface so that a hold carrying it allocates nothing. func (t *table) draw(s *session) int { if t.cum == nil { return s.IntN(t.rows()) diff --git a/template.go b/template.go index 31f704d..f7862dc 100644 --- a/template.go +++ b/template.go @@ -366,7 +366,7 @@ type formatOps struct { bound map[string]string held map[string]bool holder map[string]string - heldLocal bool // some held name is kept by the expansion itself rather than the render's draws + heldLocal bool // some held name is kept by the expansion itself rather than the render's hold } func (c *formatOps) holdName(a arm, label string) { diff --git a/todo.md b/todo.md index a31fd4d..adbefb5 100644 --- a/todo.md +++ b/todo.md @@ -16,8 +16,8 @@ scoring run names too little to steer one. A fresh run replaces the items below. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), eight of nine seats capped by Locality or Self-sufficiency, all nine naming `hold.go`, `draw.go` — now `holdset.go` and `drawfence.go` — and -`family.go` first for reading cost: 1,138 lines in which each file carries both a -load-time fence and render-time machinery, so no file name says whether a unit runs at +`family.go` first for reading cost: 1,138 lines in which each file carried both a +load-time fence and render-time machinery, so no file name said whether a unit ran at `New` or at `Fake`. This round, in order: -- 2.52.0 From 3e5e01643f6d96b3d8a1ce708e51e0e1b7c8fa9c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 21:59:47 +0200 Subject: [PATCH 048/966] Say which files the panel's draw.go became --- table.go | 2 +- todo.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/table.go b/table.go index e16afc9..6060f07 100644 --- a/table.go +++ b/table.go @@ -463,7 +463,7 @@ func (t *table) indexed() *tableIndex { } // draw picks a row over the whole table. The session is concrete rather than the rng -// interface so that a hold carrying it allocates nothing. +// interface so that the walk that draws through it keeps its hold set off the heap. func (t *table) draw(s *session) int { if t.cum == nil { return s.IntN(t.rows()) diff --git a/todo.md b/todo.md index adbefb5..6df7c52 100644 --- a/todo.md +++ b/todo.md @@ -15,7 +15,7 @@ scoring run names too little to steer one. A fresh run replaces the items below. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), eight of nine seats capped by Locality or Self-sufficiency, all -nine naming `hold.go`, `draw.go` — now `holdset.go` and `drawfence.go` — and +nine naming `hold.go`, `draw.go` — now split into `holdset.go` and `drawfence.go` — and `family.go` first for reading cost: 1,138 lines in which each file carried both a load-time fence and render-time machinery, so no file name said whether a unit ran at `New` or at `Fake`. -- 2.52.0 From 19274ccb2b40217f2077313d0c504cd55d2e2ba3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 21 Sep 2026 22:02:28 +0200 Subject: [PATCH 049/966] Keep the rule, drop its second copy --- README.md | 20 +++++++++----------- naming_test.go | 4 ---- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 2c7e3c3..19b3e14 100644 --- a/README.md +++ b/README.md @@ -1077,17 +1077,15 @@ the Development section below, and who ships a register the four above then draw - **`Column` carries text, not a Go value.** `Value` is the rendered string beside `DataType` and `Null`, which each serializer writes as the load check proved it; a `Value any` would hand every caller a type switch. -- **`hold` names what a render keeps, `draw` what it draws.** Goal 2 wants a name to - reach one unit: `hold`, `holdSet` and `renderScope` are what a render holds while it - renders, and `draw`, `drawGroup` and `drawfence.go`'s units are the draws and the - load fence over them, so `holdfence.go` fences the hold an expansion keeps and - `drawfence.go` the reference draws `holdset.go` keeps. Valid while the load fences - sit in files of their own. -- **A function may not spell a method; two types may.** A call writes a method with - its receiver and a function bare, so one name on both greps as one unit, which is - the reaching cost goal 2 counts, while the receiver before a shared method name - says which type answers. A type is outside the rule, never being called. - `TestNoFunctionSpellsAMethod` holds it over the package, its tests included. +- **`hold` names what a draw is kept in, `draw` the draw itself.** Goal 2 wants a name + to reach one unit, so a unit takes the stem of what it is, and a file the stem of the + units it holds. +- **A function may not spell a method; two types may.** A call writes a method with its + receiver and a function bare, so one name on both greps as one unit, which is the + reaching cost goal 2 counts, while the receiver before a shared method name says + which type answers. `TestNoFunctionSpellsAMethod` holds it. A type is outside the + rule: `renderScope.hold` answers with a `*hold`, so the two spell one unit, which no + test can judge. - **The package stays flat.** Go ties a package to one directory, so folders would split the API into packages. - **The performance gate asserts allocations, not wall-clock time.** `AllocsPerRun` diff --git a/naming_test.go b/naming_test.go index 80d609a..53cc589 100644 --- a/naming_test.go +++ b/naming_test.go @@ -23,8 +23,6 @@ func TestNoFunctionSpellsAMethod(t *testing.T) { } } -// namespaceFiles is every file declaring into the package namespace, its in-package -// tests included. func namespaceFiles(t *testing.T) []*ast.File { t.Helper() pkg := packageDir(t) @@ -32,8 +30,6 @@ func namespaceFiles(t *testing.T) []*ast.File { return files } -// declaredCalls is what each name is called as: a function, and the methods -// spelling it under the types declaring them. func declaredCalls(files []*ast.File) (funcs map[string]bool, methods map[string][]string) { funcs, methods = map[string]bool{}, map[string][]string{} for _, f := range files { -- 2.52.0 From d0d9df52208f2c50616e87790974caa8c72a7d00 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 21:35:22 +0200 Subject: [PATCH 050/966] Name a fresh hold set's two spellings, eager and lazy --- holdset.go | 12 ++++++++---- record.go | 4 ++-- render.go | 4 ++-- struct.go | 2 +- todo.md | 4 ---- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/holdset.go b/holdset.go index 694107f..a6e4173 100644 --- a/holdset.go +++ b/holdset.go @@ -19,15 +19,19 @@ type renderScope struct { row int } -// newHoldSet makes the unnamed group's maps where the set is declared, keeping them on that frame's -// stack for a render that reads through them; a zero holdSet makes them on its first read instead. -func newHoldSet(s *session) holdSet { +// eagerHoldSet makes the unnamed group's maps where the set is declared, keeping them on that frame's +// stack for a render that reads through them; lazyHoldSet makes them on its first read instead. +func eagerHoldSet(s *session) holdSet { return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}, s: s}} } +func lazyHoldSet(s *session) holdSet { + return holdSet{unnamed: hold{s: s}} +} + // renderOnce renders n as one render, over a hold set of its own. func renderOnce(s *session, n node) string { - set := holdSet{unnamed: hold{s: s}} + set := lazyHoldSet(s) return render(s, n, renderScope{set: &set}) } diff --git a/record.go b/record.go index 50fc7dd..c98a892 100644 --- a/record.go +++ b/record.go @@ -130,7 +130,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) } - set := newHoldSet(f.rand) + set := eagerHoldSet(f.rand) sc := renderScope{set: &set} if t, isTable := n.(*table); isTable { if n, err = tableRecord(f.rand, t, tail, sc); err != nil { @@ -204,7 +204,7 @@ type RecordTemplate struct { func (t *RecordTemplate) Fake() *Record { t.g.mu.Lock() defer t.g.mu.Unlock() - set := newHoldSet(t.g.rand) + set := eagerHoldSet(t.g.rand) return renderRecord(t.g.rand, t.t, t.columns, renderScope{set: &set}) } diff --git a/render.go b/render.go index 2df2cc5..544685c 100644 --- a/render.go +++ b/render.go @@ -23,7 +23,7 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } - f.set = holdSet{unnamed: hold{s: f.rand}} + f.set = lazyHoldSet(f.rand) sc := renderScope{set: &f.set} f.root.children = f.categories n, err := descend(f.rand, &f.root, segments, sc) @@ -101,7 +101,7 @@ func render(s *session, n node, sc renderScope) string { // //go:noinline func expandAnew(s *session, t *template) string { - set := holdSet{unnamed: hold{s: s}} + set := lazyHoldSet(s) return expand(s, t, renderScope{set: &set}) } diff --git a/struct.go b/struct.go index 0a9cd91..de3ef95 100644 --- a/struct.go +++ b/struct.go @@ -349,7 +349,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro // fill draws the record into v's tagged fields, then each nested struct as a record of its own. func (s *structShape) fill(sess *session, v reflect.Value) { if s.record != nil { - set := newHoldSet(sess) + set := eagerHoldSet(sess) for i, c := range renderRecord(sess, s.record, s.columns, renderScope{set: &set}).columns { setColumn(fieldAt(v, s.fields[i]), c) } diff --git a/todo.md b/todo.md index 6df7c52..5f1c1fe 100644 --- a/todo.md +++ b/todo.md @@ -22,10 +22,6 @@ load-time fence and render-time machinery, so no file name said whether a unit r This round, in order: -- Spell one fresh hold set one way: `newHoldSet` exists, and `render.go` and - `holdset.go` hand-roll `holdSet{unnamed: hold{s: s}}` at three sites; name the - lazy-map variant if the difference is deliberate, and put the hold set in one file: - `holdset.go` holds 57 lines after the split, and `render.go` two of the three sites. - Run the package doc's Go example under test, as `readme_test.go` runs the README's JSON blocks: `doc.go` and the README both print `Järvedsvägen 43\n891 77 Järved` for seed 42, and nothing renders either, so a draw that moves leaves both false. -- 2.52.0 From 58f42dd8ca37bc7b3b2447aa7c45404d3e8f3636 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 21:35:22 +0200 Subject: [PATCH 051/966] Draw the grow test's hold set through lazyHoldSet --- template_test.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/template_test.go b/template_test.go index 89f5e35..bd5a9e3 100644 --- a/template_test.go +++ b/template_test.go @@ -316,7 +316,8 @@ func TestGrowIsALowerBound(t *testing.T) { t.Fatalf("format %q did not compile to a template", format) } for i := 0; i < 50; i++ { - if got := len(expand(f.rand, tmpl, renderScope{set: &holdSet{}})); got < tmpl.grow { + set := lazyHoldSet(f.rand) + if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.grow { t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) } } -- 2.52.0 From 34c069a844b5a5934b78642ae13c9f9af7ee9daa Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 21:46:57 +0200 Subject: [PATCH 052/966] Move expandAnew beside renderOnce in holdset.go --- holdset.go | 9 +++++++++ render.go | 9 --------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/holdset.go b/holdset.go index a6e4173..46f648a 100644 --- a/holdset.go +++ b/holdset.go @@ -35,6 +35,15 @@ func renderOnce(s *session, n node) string { return render(s, n, renderScope{set: &set}) } +// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into +// render's loop, its hold set would move to the heap. +// +//go:noinline +func expandAnew(s *session, t *template) string { + set := lazyHoldSet(s) + return expand(s, t, renderScope{set: &set}) +} + // in is the scope t renders in: its draw group where it names one, else its caller's. func (sc renderScope) in(t *template) renderScope { if t.drawGroupKey != "" { diff --git a/render.go b/render.go index 544685c..c51115c 100644 --- a/render.go +++ b/render.go @@ -96,15 +96,6 @@ func render(s *session, n node, sc renderScope) string { } } -// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into -// render's loop, its hold set would move to the heap. -// -//go:noinline -func expandAnew(s *session, t *template) string { - set := lazyHoldSet(s) - return expand(s, t, renderScope{set: &set}) -} - // pick selects one item. Uniform choices are O(1); weighted choices are an // O(log n) search over precomputed cumulative weights. compile guarantees a // non-empty choice and a finite positive total, so the index is always in range. -- 2.52.0 From 89e6b65e842c19a280d25563bb950dbfe790d441 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 21:50:05 +0200 Subject: [PATCH 053/966] File the named group's eager hold as a todo item --- todo.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/todo.md b/todo.md index 5f1c1fe..dfb8d2b 100644 --- a/todo.md +++ b/todo.md @@ -248,6 +248,9 @@ because pairing a street with its exact postnummer rewrites shipped rows. ## Not release-bound +- Make a named draw group's hold lazy, `&hold{s: s}` in `renderScope.hold`: it escapes + to the heap anyway, so its two eager maps buy no stack and cost two allocations per + group that may never read through them. Check it against the alloc tests. - Name goal 2's four criteria with the axes the panel scores, so a recorded score maps back to the clause it came from: the goal spells them out in prose while `todo.md` records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. -- 2.52.0 From 98edb1ba0106cce7b796f2dc4331fdc91296b57a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 22:06:09 +0200 Subject: [PATCH 054/966] Hold the Go examples' printed Fake values to what seed 42 renders --- doc_test.go | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/doc_test.go b/doc_test.go index 77f9f23..1fab4e4 100644 --- a/doc_test.go +++ b/doc_test.go @@ -5,8 +5,10 @@ import ( "go/build" "go/parser" "go/token" + "os" "regexp" "slices" + "strconv" "strings" "testing" ) @@ -246,3 +248,32 @@ func headerAbove(headers []*ast.CommentGroup, pos token.Pos) string { } return text } + +var printedFake = regexp.MustCompile(`f\.Fake\("([^"]+)"\)\s*// ("(?:[^"\\]|\\.)*")`) + +// TestGoExamplesPrintWhatTheyRender replays each Go example's printed Fake calls on seed 42. +func TestGoExamplesPrintWhatTheyRender(t *testing.T) { + for _, name := range []string{"doc.go", "README.md"} { + src, err := os.ReadFile(name) + if err != nil { + t.Fatal(err) + } + calls := printedFake.FindAllStringSubmatch(string(src), -1) + if calls == nil { + t.Errorf("%s prints no seeded Fake, so nothing holds its example to a render", name) + } + f, err := New(WithSeed(42)) + if err != nil { + t.Fatal(err) + } + for _, c := range calls { + want, err := strconv.Unquote(c[2]) + if err != nil { + t.Fatalf("%s prints %s: %v", name, c[2], err) + } + if got := fake(t, f, c[1]); got != want { + t.Errorf("%s prints Fake(%q) = %q for seed 42, it renders %q", name, c[1], want, got) + } + } + } +} -- 2.52.0 From f689afee09f55539f5659bfdd1b0eb440444cb91 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 22:06:09 +0200 Subject: [PATCH 055/966] Close the Go example item and queue the next panel run --- todo.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/todo.md b/todo.md index dfb8d2b..fd97a1d 100644 --- a/todo.md +++ b/todo.md @@ -22,9 +22,7 @@ load-time fence and render-time machinery, so no file name said whether a unit r This round, in order: -- Run the package doc's Go example under test, as `readme_test.go` runs the README's - JSON blocks: `doc.go` and the README both print `Järvedsvägen 43\n891 77 Järved` - for seed 42, and nothing renders either, so a draw that moves leaves both false. +- Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From f8d6109b9dcbba10660fe6b0d397ab382a364150 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 22:08:34 +0200 Subject: [PATCH 056/966] Fail a printed Fake value the Go example test cannot replay --- doc_test.go | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/doc_test.go b/doc_test.go index 1fab4e4..fe5b297 100644 --- a/doc_test.go +++ b/doc_test.go @@ -249,7 +249,10 @@ func headerAbove(headers []*ast.CommentGroup, pos token.Pos) string { return text } -var printedFake = regexp.MustCompile(`f\.Fake\("([^"]+)"\)\s*// ("(?:[^"\\]|\\.)*")`) +var ( + printedCall = regexp.MustCompile(`(?m)^.*Fake\w*\(.*//\s*".*$`) + printedFake = regexp.MustCompile(`\.Fake\("([^"]+)"\)\s*//\s*("(?:[^"\\]|\\.)*")`) +) // TestGoExamplesPrintWhatTheyRender replays each Go example's printed Fake calls on seed 42. func TestGoExamplesPrintWhatTheyRender(t *testing.T) { @@ -258,15 +261,19 @@ func TestGoExamplesPrintWhatTheyRender(t *testing.T) { if err != nil { t.Fatal(err) } - calls := printedFake.FindAllStringSubmatch(string(src), -1) - if calls == nil { + lines := printedCall.FindAllString(string(src), -1) + if lines == nil { t.Errorf("%s prints no seeded Fake, so nothing holds its example to a render", name) } f, err := New(WithSeed(42)) if err != nil { t.Fatal(err) } - for _, c := range calls { + for _, line := range lines { + c := printedFake.FindStringSubmatch(line) + if c == nil { + t.Fatalf("%s prints a value this test cannot replay; spell it f.Fake(\"path\") // \"value\": %s", name, line) + } want, err := strconv.Unquote(c[2]) if err != nil { t.Fatalf("%s prints %s: %v", name, c[2], err) -- 2.52.0 From 7c8d3c26052871feb58e54a65f56ae363cd6af14 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 22:21:20 +0200 Subject: [PATCH 057/966] Correct the comments and docs the panel found false --- README.md | 5 ++--- compose.yaml | 4 ++-- template.go | 7 ++++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 19b3e14..325c454 100644 --- a/README.md +++ b/README.md @@ -1239,9 +1239,8 @@ the Development section below, and who ships a register the four above then draw - **`misc` is what every locale shares.** A category whose facts differ by country belongs in that country's locale, read from the register that country's own records use; `misc` takes only sources that are international. NHTSA vPIC and - Mobility Sweden's registrations are national, so they build `en_US.car` and - `sv_SE.car`; `misc.car` waits for an international source rather than take one - of theirs. + Mobility Sweden's registrations are national, so they are for `en_US.car` and + `sv_SE.car`, and `misc.car` waits for an international source. - **A register's canonical spelling loses to the one its domain writes.** Where a source offers several spellings of one fact, the shipped one is what records in that domain carry. `misc.timezone` reads `zone.tab` and not the `zone1970.tab` diff --git a/compose.yaml b/compose.yaml index 7bdfd06..86595e4 100644 --- a/compose.yaml +++ b/compose.yaml @@ -4,8 +4,8 @@ # docker compose run --rm vet # docker compose run --rm build # -# The full gate — vet, formatting and tests — is `docker build .`, which is what -# CI runs. +# The full gate — vet, formatting, cyclomatic complexity and tests — is +# `docker build .`, which is what CI runs. # # Build caches persist in the `gocache` volume, so re-runs are fast and the # host tree stays clean. Commands that rewrite source (fmt, tidy) keep your diff --git a/template.go b/template.go index f7862dc..c92edd5 100644 --- a/template.go +++ b/template.go @@ -60,8 +60,8 @@ func eachToken(format string, fn func(ftoken) error) error { // builtin is a format-string function invoked as {name(args)}. It receives the // session (its rng, and the {seq()} counters), the output emitted so far in the // current expansion (for derivations such as a checksum over preceding digits), and -// the values of the operands it named (only calc names any). All must stay pure -// over (rng, emitted, args) so seeded output is reproducible; seq advances +// the values of the operands it named (calc and the transforms name them). All must +// stay pure over (rng, emitted, args) so seeded output is reproducible; seq advances // per-session counter state, which is itself deterministic. arity is the exact arg // count, or -1 for variadic (then check does all the validation). type builtin struct { @@ -121,7 +121,8 @@ func plural(n int) string { // checkFunc validates a function token at compile time: well-formed, naming a // known builtin, with the arg count that builtin takes and args its check accepts. -// fields is passed through for the one builtin (calc) that validates against them. +// fields is passed through for the builtins (calc, the transforms) that validate +// against them. func checkFunc(body string, fields map[string]node) error { name, args, ok := funcCall(body) if !ok { -- 2.52.0 From 021a95c1147768af0163e457bc67a47d6b54d220 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 22 Sep 2026 22:21:20 +0200 Subject: [PATCH 058/966] File the 2026-09-22 nine-seat panel run --- todo.md | 72 ++++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 59 insertions(+), 13 deletions(-) diff --git a/todo.md b/todo.md index fd97a1d..f4c2a06 100644 --- a/todo.md +++ b/todo.md @@ -14,14 +14,52 @@ nine-seat one: the round is meant to buy architectural change, and the four-seat scoring run names too little to steer one. A fresh run replaces the items below. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, -Self-sufficiency 5.7), eight of nine seats capped by Locality or Self-sufficiency, all -nine naming `hold.go`, `draw.go` — now split into `holdset.go` and `drawfence.go` — and -`family.go` first for reading cost: 1,138 lines in which each file carried both a -load-time fence and render-time machinery, so no file name said whether a unit ran at -`New` or at `Fake`. +Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9, +Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. Eight of nine +named `drawWalk.walk` and `rowSet` or `checkFamilies` hardest and least wanted to modify: +a load-time copy of the pinning `hold.pin` and `hold.rowOf` do at render, tied to them +by nothing but tests, whose premise lives only in the README's Decisions. This round, in order: +- Gather the family fence in `family.go`: `alternatives`, `rowSet` and `checkOwnFamily` + beside `checkFamilies`, and the `drawGroup` option parsing out of `drawfence.go`, so + one question about a family opens one file. +- Give render and the draw fences one pin model: a pin set `hold.pin`, `hold.rowOf` + and the fence's `rowSet` all use, so a change to how a render pins rows cannot leave + the fence proving the old rule, and `hold` stops being a proof scratchpad whenever its + session is nil. +- Move the README's Decisions section to `docs/decisions.md`, leaving a one-line index + of the titles in `AGENTS.md`, and drop the `AGENTS.md` line pointing decisions at the + README. Give each fence a bare link to the entry it enforces: every seat needed + "The rows of a table are alternatives", "The expansion hold and the render's draws + are two fences" or "A table read into is pinned" and found it by grepping 470 lines. + Carry the timezone weight's premise in with it: weighting by GeoNames city + population moved 300 seeded draws of `misc.territory[US].timezone` from 44 landing on + the four zones most Americans live in to 278, and dropped `America/Indiana/Petersburg` + (pop. 2,400) from 17 draws to 0. Carry in the decision `linkTemplateRefs`'s doc holds + today, which no entry records: a category is a unit, so a reference back into it + describes a draw other than the fields beside it, which is what goal 3 asks for. +- Keep reference bindings out of `template.fields`, so no walk over a template's fields + needs the `isRef` filter `named`, `paths`, `recordColumns` and `template.field` carry. +- Compile a template's format once: `compileFormat` runs again from `linkTemplateRefs`, + so `ops`, `bound` and `held` are provisional until linking and its fences run twice. +- Parse a format once and have the fences read `t.ops`: `eachToken` rescans the format + in `fieldTokens`, `operandTokens`, `boundReaders`, `renderEdges`, `loneRef`, + `operandReader`, `checkNoRepeatedRead` and `compileOps`, and function-versus-field is + decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. +- Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which + callbacks are nil and whether `pins.s` is, and `walkChoice` returns `nil, nil` where + none is set. +- Register a transform once: `transforms` restates the `lowercase`, `uppercase` and + `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while + render calls the second. Derive the numeric builtins `valueProof.template` lists in + its error text from `builtin.number` while there. +- Split `drawCheck.reads`'s six-term condition into named predicates. +- Rename what names two things: `template.go` holds the token grammar while `template` + is `node.go`'s type and `Template` `inline.go`'s; `kind` `'b'` is a brace body in + `ftoken` and a builtin in `op`; `hold`'s receiver is `d`; `drawAt.alt` holds the rows + a walk entered. - Run the nine-seat panel again and file what it names here. ### Data @@ -29,6 +67,10 @@ This round, in order: Shape: T = table, t = template, c = choice. Read every factual list from a register by a `data-import/` script, as README goal 12 asks. +- Place xlsx cells by their `r` reference in `data-import/xlsx.py`: a sheet omitting an + empty cell shifts every later column left. +- Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone + `{digits(1)}` at `New`, naming that spelling: both render the same run. - Add the remaining `misc` tables and templates, one row of the table below per chunk. - Read `misc.emoji` from the Unicode `emoji-test.txt` register, and `misc.car` from an international make and model source; vPIC and Mobility Sweden are national, so they @@ -137,6 +179,8 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Report the same error every load for a table with two bad options: + `readTableOptions` returns on the first in Go's map order. - Refuse two whole reads of one table family in one render, which goal 5 promises is a load error: `{/sel}|{/sel}` panics out of `Fake` with "a fence should have refused this at New" where the two draws land on rows whose cells select different rows of another @@ -198,14 +242,11 @@ by a `data-import/` script, as README goal 12 asks. ### Release -- Move the README's Decisions section to `docs/decisions.md`, leaving a one-line index - of the titles in `AGENTS.md`, and drop the `AGENTS.md` line pointing decisions at the - README. Carry the timezone weight's premise in with it: weighting by GeoNames city - population moved 300 seeded draws of `misc.territory[US].timezone` from 44 landing on - the four zones most Americans live in to 278, and dropped `America/Indiana/Petersburg` - (pop. 2,400) from 17 draws to 0. Carry in the decision `linkTemplateRefs`'s doc holds - today, which no entry records: a category is a unit, so a reference back into it - describes a draw other than the fields beside it, which is what goal 3 asks for. +- Read the tag through `git/ref/tags/{tag}` in `publish_release.py`: GitHub answers + `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a + release lands on a tag already pointing at another commit. +- Document `NewRecordTemplate` in the README's Library section, the one exported name + it does not name. - Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds: nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. @@ -246,6 +287,11 @@ because pairing a street with its exact postnummer rewrites shipped rows. ## Not release-bound +- Hold every change the charter says owes a `CHANGELOG.md` entry to one in CI: the + check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a + flag, an exit code, an exported name, a fence, a builtin and the lowest Go. +- Measure the allocation gate over the shipped `geo` trees too: `hold.pins [8]` is sized + for them, and `perf_test.go` checks a synthetic five-deep tree only. - Make a named draw group's hold lazy, `&hold{s: s}` in `renderScope.hold`: it escapes to the heap anyway, so its two eager maps buy no stack and cost two allocations per group that may never read through them. Check it against the alloc tests. -- 2.52.0 From def2a5f25dfc1154b5244958b952a7b202b77c8e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 23 Sep 2026 22:44:47 +0200 Subject: [PATCH 059/966] Gather the family fence in family.go --- README.md | 4 +- drawfence.go | 131 --------------------------------------------------- family.go | 105 +++++++++++++++++++++++++++++++++++++++++ node.go | 27 +++++++++++ todo.md | 3 -- 5 files changed, 134 insertions(+), 136 deletions(-) diff --git a/README.md b/README.md index 325c454..b1971de 100644 --- a/README.md +++ b/README.md @@ -1341,8 +1341,8 @@ template.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per expansion for paths and operands, and the rows it pins holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through -drawfence.go the load fences over reference draws: the drawGroup option, and the walk over a render's reads -family.go a family of linked tables: what a reference path reads of one, and the fence over paths into it +drawfence.go the load fences over reference draws: the walk over a render's reads +family.go a family of linked tables: what a reference path reads of one, the rows a walk enters, and the fences over paths into it reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences builtins.go the {name()} function registry and its implementations diff --git a/drawfence.go b/drawfence.go index 64a7e97..7ea756b 100644 --- a/drawfence.go +++ b/drawfence.go @@ -6,33 +6,6 @@ import ( "strings" ) -// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each -// iteration renders in no draw group. -func drawGroupOf(m map[string]any, repeat int) (string, error) { - v, ok := m["drawGroup"] - if !ok { - return "", nil - } - name, ok := v.(string) - switch { - case !ok: - return "", fmt.Errorf("drawGroup must be a string, got %T", v) - case name == "": - return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`) - case repeat > 1: - return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name) - } - return name, nil -} - -// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name -// is local to its category. -func (t *template) keyDrawGroup(category string) { - if t.drawGroup != "" { - t.drawGroupKey = category + "/" + t.drawGroup - } -} - // checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, // short of a repeat or another draw group. func checkNestedDrawGroup(fields map[string]node, group string) error { @@ -104,43 +77,6 @@ func (c *drawCheck) checkDraws(path string, n node) error { return nil } -// checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a row -// rendered whole draws its row without pinning it, so the family would draw apart -// from the row being rendered, whichever draws the reaching template holds. -func checkOwnFamily(t *template) error { - own := t.table - if own == nil { - own = t.cellOf - } - if own == nil { - return nil - } - seen := map[node]bool{} - var find func(n node) (renderEdge, *table, bool) - find = func(n node) (renderEdge, *table, bool) { - if seen[n] { - return renderEdge{}, nil, false - } - seen[n] = true - for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).fields[a.key].(*table); isTable && head.family() == own.family() { - return e, head, true - } - } - if e, head, found := find(e.to); found { - return e, head, true - } - } - return renderEdge{}, nil, false - } - if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a row of %s rendered whole would draw apart from; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) - } - return nil -} - // checkRecordDraws fences the columns of a record — a template compiled at the top without a // repeat — so a load proves the record view of it as well as the string view. func (c *drawCheck) checkRecordDraws(path string, n node) error { @@ -248,59 +184,6 @@ type drawAt struct { drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true } -// rowSet stands in at load for the rows (*hold).pin holds at render: a path read enters a row with -// its ancestors, a whole read enters the drawn row alone. -type rowSet []tablePin - -func (s rowSet) rowOf(t *table) (int, bool) { - for _, p := range s { - if p.t == t { - return p.row, true - } - } - return 0, false -} - -// enter is s with row r of t, and with t's ancestors where the render pins that row rather than -// drawing it; the set is copied, since walks branch. -func (s rowSet) enter(t *table, r int, pins bool) rowSet { - if _, in := s.rowOf(t); in { - return s - } - out := append(make(rowSet, 0, len(s)+1), s...) - for { - out = append(out, tablePin{t, r}) - if !pins || t.parentT == nil { - break - } - t, r = t.parentT, t.parentRow(r) - if _, in := out.rowOf(t); in { - break - } - } - sort.Slice(out, func(i, j int) bool { return out[i].t.path < out[j].t.path }) - return out -} - -// key spells the set for a map, by the tables' identities. -func (s rowSet) key() string { - var b strings.Builder - for _, p := range s { - fmt.Fprintf(&b, "%p[%d]", p.t, p.row) - } - return b.String() -} - -// alternatives reports whether two reads sit in different rows of one table. -func alternatives(a, b drawAt) bool { - for _, p := range a.alt { - if r, in := b.alt.rowOf(p.t); in && r != p.row { - return true - } - } - return false -} - // drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. type drawRoute struct{ spelling, label string } @@ -358,20 +241,6 @@ func (w *drawWalk) walk(n node, at drawAt) { } } -// excludes reports whether a cell's row cannot render with the rows entered: another -// row of its table, or a row outside an entered ancestor's. -func (s rowSet) excludes(cell *template) bool { - for _, p := range s { - if p.t == cell.cellOf { - return p.row != cell.cellRow - } - if cell.cellOf.descends(p.t) && !cell.cellOf.under(cell.cellRow, p.t, p.row) { - return true - } - } - return false -} - // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { diff --git a/family.go b/family.go index a495e0c..9403618 100644 --- a/family.go +++ b/family.go @@ -2,6 +2,7 @@ package fejkdata import ( "fmt" + "sort" "strings" ) @@ -187,3 +188,107 @@ func (r *tableRead) drawnOf(pins *hold) *table { }) return found } + +// checkOwnFamily refuses a table's format or cell that reads, however many templates +// away and through a repeat or a draw group too, a table of its own family: a row +// rendered whole draws its row without pinning it, so the family would draw apart +// from the row being rendered, whichever draws the reaching template holds. +func checkOwnFamily(t *template) error { + own := t.table + if own == nil { + own = t.cellOf + } + if own == nil { + return nil + } + seen := map[node]bool{} + var find func(n node) (renderEdge, *table, bool) + find = func(n node) (renderEdge, *table, bool) { + if seen[n] { + return renderEdge{}, nil, false + } + seen[n] = true + for _, e := range renderEdges(n) { + if a, isRef := refRead(n, e.label); isRef { + if head, isTable := n.(*template).fields[a.key].(*table); isTable && head.family() == own.family() { + return e, head, true + } + } + if e, head, found := find(e.to); found { + return e, head, true + } + } + return renderEdge{}, nil, false + } + if e, head, found := find(t); found { + return fmt.Errorf("%s reads %s, a table of its own family, which a row of %s rendered whole would draw apart from; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + } + return nil +} + +// rowSet stands in at load for the rows (*hold).pin holds at render: a path read enters a row with +// its ancestors, a whole read enters the drawn row alone. +type rowSet []tablePin + +func (s rowSet) rowOf(t *table) (int, bool) { + for _, p := range s { + if p.t == t { + return p.row, true + } + } + return 0, false +} + +// enter is s with row r of t, and with t's ancestors where the render pins that row rather than +// drawing it; the set is copied, since walks branch. +func (s rowSet) enter(t *table, r int, pins bool) rowSet { + if _, in := s.rowOf(t); in { + return s + } + out := append(make(rowSet, 0, len(s)+1), s...) + for { + out = append(out, tablePin{t, r}) + if !pins || t.parentT == nil { + break + } + t, r = t.parentT, t.parentRow(r) + if _, in := out.rowOf(t); in { + break + } + } + sort.Slice(out, func(i, j int) bool { return out[i].t.path < out[j].t.path }) + return out +} + +// key spells the set for a map, by the tables' identities. +func (s rowSet) key() string { + var b strings.Builder + for _, p := range s { + fmt.Fprintf(&b, "%p[%d]", p.t, p.row) + } + return b.String() +} + +// excludes reports whether a cell's row cannot render with the rows entered: another +// row of its table, or a row outside an entered ancestor's. +func (s rowSet) excludes(cell *template) bool { + for _, p := range s { + if p.t == cell.cellOf { + return p.row != cell.cellRow + } + if cell.cellOf.descends(p.t) && !cell.cellOf.under(cell.cellRow, p.t, p.row) { + return true + } + } + return false +} + +// alternatives reports whether two reads sit in different rows of one table. +func alternatives(a, b drawAt) bool { + for _, p := range a.alt { + if r, in := b.alt.rowOf(p.t); in && r != p.row { + return true + } + } + return false +} diff --git a/node.go b/node.go index afac977..cf3134b 100644 --- a/node.go +++ b/node.go @@ -433,6 +433,33 @@ func repeatOf(m map[string]any) (int, error) { return int(r), nil } +// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each +// iteration renders in no draw group. +func drawGroupOf(m map[string]any, repeat int) (string, error) { + v, ok := m["drawGroup"] + if !ok { + return "", nil + } + name, ok := v.(string) + switch { + case !ok: + return "", fmt.Errorf("drawGroup must be a string, got %T", v) + case name == "": + return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`) + case repeat > 1: + return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name) + } + return name, nil +} + +// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name +// is local to its category. +func (t *template) keyDrawGroup(category string) { + if t.drawGroup != "" { + t.drawGroupKey = category + "/" + t.drawGroup + } +} + // weightOf reads a node's "weight" (default 1) from its raw JSON form. Only // template objects carry weight; a present one must be finite and positive. func weightOf(raw any) (float64, error) { diff --git a/todo.md b/todo.md index f4c2a06..4b66f8b 100644 --- a/todo.md +++ b/todo.md @@ -22,9 +22,6 @@ by nothing but tests, whose premise lives only in the README's Decisions. This round, in order: -- Gather the family fence in `family.go`: `alternatives`, `rowSet` and `checkOwnFamily` - beside `checkFamilies`, and the `drawGroup` option parsing out of `drawfence.go`, so - one question about a family opens one file. - Give render and the draw fences one pin model: a pin set `hold.pin`, `hold.rowOf` and the fence's `rowSet` all use, so a change to how a render pins rows cannot leave the fence proving the old rule, and `hold` stops being a proof scratchpad whenever its -- 2.52.0 From 76e5efc73b1c45b25fcf3ef6b31d83750ec74fb0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 23 Sep 2026 22:57:21 +0200 Subject: [PATCH 060/966] Name the drawGroup fences and family.go where the prose indexes fences --- README.md | 2 +- todo.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index b1971de..64a72ac 100644 --- a/README.md +++ b/README.md @@ -1341,7 +1341,7 @@ template.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per expansion for paths and operands, and the rows it pins holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through -drawfence.go the load fences over reference draws: the walk over a render's reads +drawfence.go the load fences over reference draws: the drawGroup fences, and the walk over a render's reads family.go a family of linked tables: what a reference path reads of one, the rows a walk enters, and the fences over paths into it reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/todo.md b/todo.md index 4b66f8b..d889683 100644 --- a/todo.md +++ b/todo.md @@ -303,8 +303,8 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Name the draw fences at one level in `doc.go`: the Vocabulary calls them `heldCheck`, `drawCheck` and `checkFamilies`, where `checkFamilies` runs under `drawCheck` beside `checkOwnFamily` and `checkColumnDraws`, which it names nowhere — so a reader looking - for the fence over a record's columns finds no entry reaching it. `holdfence.go` and - `drawfence.go` now index the fences by file, so deleting the sentence closes it too. + for the fence over a record's columns finds no entry reaching it. `holdfence.go`, + `drawfence.go` and `family.go` now index the fences by file, so deleting the sentence closes it too. - Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a second copy that drifts. Split the comprehension rule while there, which packs the -- 2.52.0 From 7a772ea31399c9c572f8883b745c049f334c3581 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 23 Sep 2026 22:58:23 +0200 Subject: [PATCH 061/966] Name every fence drawfence.go holds, and correct the doc.go todo item's premise --- README.md | 2 +- todo.md | 8 +++----- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 64a72ac..17bd83b 100644 --- a/README.md +++ b/README.md @@ -1341,7 +1341,7 @@ template.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per expansion for paths and operands, and the rows it pins holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through -drawfence.go the load fences over reference draws: the drawGroup fences, and the walk over a render's reads +drawfence.go the load fences over reference draws: a drawGroup's, a render's and a record's columns, and the walk over what a render reads family.go a family of linked tables: what a reference path reads of one, the rows a walk enters, and the fences over paths into it reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/todo.md b/todo.md index d889683..a51120d 100644 --- a/todo.md +++ b/todo.md @@ -300,11 +300,9 @@ because pairing a street with its exact postnummer rewrites shipped rows. round on the first and every merge after 7.0 on the second, then holds one "last score" across both, where the `comprehension-panel` skill has two independent panels drift unless a calibration seat reads the anchor project without its score. -- Name the draw fences at one level in `doc.go`: the Vocabulary calls them `heldCheck`, - `drawCheck` and `checkFamilies`, where `checkFamilies` runs under `drawCheck` beside - `checkOwnFamily` and `checkColumnDraws`, which it names nowhere — so a reader looking - for the fence over a record's columns finds no entry reaching it. `holdfence.go`, - `drawfence.go` and `family.go` now index the fences by file, so deleting the sentence closes it too. +- Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and + a helper reached only through `drawWalk.check`, and misses `checkDrawGroup`, + `checkOwnFamily` and `checkColumnDraws`; the README's Layout indexes the fences by file. - Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a second copy that drifts. Split the comprehension rule while there, which packs the -- 2.52.0 From 968d79399d90fc6459dd24c216297569284ca3a3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 00:12:49 +0200 Subject: [PATCH 062/966] Hold the fence's entered rows to the render's pin set --- table_test.go | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/table_test.go b/table_test.go index 6b90f75..f9987b1 100644 --- a/table_test.go +++ b/table_test.go @@ -964,9 +964,10 @@ func TestAmbiguousNameNamesARunnablePath(t *testing.T) { } } -// TestRowSetSimulatesPinning holds the walk's row set to the render's pinning: drift between -// the two is data that loads and then renders a family that disagrees. -func TestRowSetSimulatesPinning(t *testing.T) { +// TestEnteredRowsAgreeWithPinning holds what a fence walk decides about the rows it entered to what +// pinning them refuses at render: drift between the two is data that loads and then renders a +// family that disagrees. +func TestEnteredRowsAgreeWithPinning(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) var tables []*table for _, category := range []string{"region", "municipality", "locality"} { @@ -976,33 +977,24 @@ func TestRowSetSimulatesPinning(t *testing.T) { } tables = append(tables, tbl) } + var none pinSet for _, entered := range tables { for row := 0; row < entered.rows(); row++ { - if drawn := rowSet(nil).enter(entered, row, false); len(drawn) != 1 || drawn[0] != (tablePin{entered, row}) { + var drawn []tablePin + alone := none.entered(entered, row, false) + alone.each(func(tbl *table, r int) { drawn = append(drawn, tablePin{tbl, r}) }) + if len(drawn) != 1 || drawn[0] != (tablePin{entered, row}) { t.Errorf("a drawn %s enters %v, want its own row alone", entered.selectorSpelling(row), drawn) } - s := rowSet(nil).enter(entered, row, true) - var pinned hold - held := 0 - pinned.pin(entered, row) - pinned.each(func(tbl *table, r int) { - held++ - if got, in := s.rowOf(tbl); !in || got != r { - t.Errorf("entering %s leaves %s unset, though pinning it pins %s", entered.selectorSpelling(row), tbl.path, tbl.selectorSpelling(r)) - } - }) - if held != len(s) { - t.Errorf("entering %s holds %d rows, though pinning it pins %d", entered.selectorSpelling(row), len(s), held) - } + s := none.entered(entered, row, true) for _, cand := range tables { for cr := 0; cr < cand.rows(); cr++ { - var beside hold - beside.pin(entered, row) + beside := s refused := beside.pinRow(cand, cr) != nil if got := s.excludes(&template{cellOf: cand, cellRow: cr}); got != refused { t.Errorf("excludes(%s) = %v beside %s, but pinRow refuses it = %v", cand.selectorSpelling(cr), got, entered.selectorSpelling(row), refused) } - if got := alternatives(drawAt{alt: s}, drawAt{alt: rowSet(nil).enter(cand, cr, true)}); got != refused { + if got := alternatives(drawAt{alt: s}, drawAt{alt: none.entered(cand, cr, true)}); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } } -- 2.52.0 From 3927dc9fa59bdae19408255eec25c292d370330d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 00:17:01 +0200 Subject: [PATCH 063/966] Give render and the draw fences one pin set --- README.md | 5 +- doc.go | 3 +- drawfence.go | 6 +- family.go | 79 +++---------------- graph.go | 2 +- hold.go | 151 +++---------------------------------- holdfence.go | 2 +- holdset.go | 21 ++---- path.go | 58 +++++++------- path_test.go | 6 +- pins.go | 192 +++++++++++++++++++++++++++++++++++++++++++++++ record.go | 8 +- render.go | 12 +-- struct.go | 2 +- table.go | 6 +- template_test.go | 2 +- todo.md | 8 +- 17 files changed, 279 insertions(+), 284 deletions(-) create mode 100644 pins.go diff --git a/README.md b/README.md index 17bd83b..78ca477 100644 --- a/README.md +++ b/README.md @@ -1338,11 +1338,12 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold: one draw per expansion for paths and operands, and the rows it pins +hold.go the hold: one draw per expansion for paths and operands holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through drawfence.go the load fences over reference draws: a drawGroup's, a render's and a record's columns, and the walk over what a render reads -family.go a family of linked tables: what a reference path reads of one, the rows a walk enters, and the fences over paths into it +family.go a family of linked tables: what a reference path reads of one, and the fences over paths into it +pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences builtins.go the {name()} function registry and its implementations diff --git a/doc.go b/doc.go index 1d8c2fe..ad963fd 100644 --- a/doc.go +++ b/doc.go @@ -28,7 +28,8 @@ package fejkdata // - draw group — reference draws held apart inside one render: // `template.drawGroup` as the data spells it, `template.drawGroupKey` as a // render reads it, `holdSet`, `renderScope`. -// - pin — fixing which row of a table the render uses: `hold.pin`, `hold.rowOf`. +// - pin — fixing which row of a table the render uses, which the draw fences replay: +// `pinSet`, `pinSet.pin`, `pinSet.rowOf`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. // - whole — a read that lands on a `row`, so the row renders through its table's diff --git a/drawfence.go b/drawfence.go index 7ea756b..824403e 100644 --- a/drawfence.go +++ b/drawfence.go @@ -180,7 +180,7 @@ type drawWalk struct { type drawAt struct { group string route drawRoute - alt rowSet + alt pinSet drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true } @@ -233,7 +233,7 @@ func (w *drawWalk) walk(n node, at drawAt) { continue } in := at - in.alt = at.alt.enter(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn) + in.alt = at.alt.entered(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn) w.edge(n, e, in) continue } @@ -251,7 +251,7 @@ func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { w.reads = append(w.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { at.alt = at.alt.enter(t, r, true) }) + tr.pins.each(func(t *table, r int) { at.alt = at.alt.entered(t, r, true) }) } } w.walk(e.to, at) diff --git a/family.go b/family.go index 9403618..c89e2a6 100644 --- a/family.go +++ b/family.go @@ -2,7 +2,6 @@ package fejkdata import ( "fmt" - "sort" "strings" ) @@ -12,7 +11,7 @@ import ( // it lands on a row rendered whole. type tableRead struct { head *table - pins hold + pins pinSet drawn map[*table]bool sels []tableSel whole bool @@ -34,7 +33,7 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { return nil } tr := &tableRead{head: t, drawn: map[*table]bool{}} - _, _ = walkPath(t, a.tail, pathWalk{pins: &tr.pins}) + _, _ = walkPath(nil, t, a.tail, pathWalk{pins: &tr.pins}) written := a.name[:len(a.name)-len(joinSegments(a.tail))] cur := t tr.draws(cur) @@ -126,7 +125,7 @@ func replayPairs(reads []pathRead) error { if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { continue } - var d hold + var d pinSet if err := r.tr.replay(&d); err != nil { return conflict(r, err) } @@ -143,7 +142,7 @@ func conflict(r pathRead, err error) error { } // replay pins the read's rows into d, where they agree with the rows pinned before. -func (r *tableRead) replay(d *hold) error { +func (r *tableRead) replay(d *pinSet) error { var err error r.pins.each(func(t *table, row int) { if err == nil { @@ -179,7 +178,7 @@ func checkFamilyPair(a, b pathRead) error { } // drawnOf is a table the read draws that pins holds a row of, if any. -func (r *tableRead) drawnOf(pins *hold) *table { +func (r *tableRead) drawnOf(pins *pinSet) *table { var found *table pins.each(func(t *table, _ int) { if found == nil && r.drawn[t] { @@ -226,69 +225,13 @@ func checkOwnFamily(t *template) error { return nil } -// rowSet stands in at load for the rows (*hold).pin holds at render: a path read enters a row with -// its ancestors, a whole read enters the drawn row alone. -type rowSet []tablePin - -func (s rowSet) rowOf(t *table) (int, bool) { - for _, p := range s { - if p.t == t { - return p.row, true - } - } - return 0, false -} - -// enter is s with row r of t, and with t's ancestors where the render pins that row rather than -// drawing it; the set is copied, since walks branch. -func (s rowSet) enter(t *table, r int, pins bool) rowSet { - if _, in := s.rowOf(t); in { - return s - } - out := append(make(rowSet, 0, len(s)+1), s...) - for { - out = append(out, tablePin{t, r}) - if !pins || t.parentT == nil { - break - } - t, r = t.parentT, t.parentRow(r) - if _, in := out.rowOf(t); in { - break - } - } - sort.Slice(out, func(i, j int) bool { return out[i].t.path < out[j].t.path }) - return out -} - -// key spells the set for a map, by the tables' identities. -func (s rowSet) key() string { - var b strings.Builder - for _, p := range s { - fmt.Fprintf(&b, "%p[%d]", p.t, p.row) - } - return b.String() -} - -// excludes reports whether a cell's row cannot render with the rows entered: another -// row of its table, or a row outside an entered ancestor's. -func (s rowSet) excludes(cell *template) bool { - for _, p := range s { - if p.t == cell.cellOf { - return p.row != cell.cellRow - } - if cell.cellOf.descends(p.t) && !cell.cellOf.under(cell.cellRow, p.t, p.row) { - return true - } - } - return false -} - // alternatives reports whether two reads sit in different rows of one table. func alternatives(a, b drawAt) bool { - for _, p := range a.alt { - if r, in := b.alt.rowOf(p.t); in && r != p.row { - return true + found := false + a.alt.each(func(t *table, r int) { + if br, in := b.alt.pinned(t); in && br != r { + found = true } - } - return false + }) + return found } diff --git a/graph.go b/graph.go index d0fcf79..6b1794d 100644 --- a/graph.go +++ b/graph.go @@ -178,7 +178,7 @@ func renderEdges(n node) []renderEdge { // already proved the tail resolves in every variant. func pathLeaves(n node, tail []string) []node { var out []node - _, _ = walkPath(n, tail, pathWalk{ + _, _ = walkPath(nil, n, tail, pathWalk{ choice: func(c *choice, _ []string) ([]node, error) { return c.items, nil }, leaf: func(n node) error { out = append(out, n); return nil }, }) diff --git a/hold.go b/hold.go index 555e583..ecb7f2a 100644 --- a/hold.go +++ b/hold.go @@ -2,26 +2,16 @@ package fejkdata import ( "fmt" - "sort" ) -// tablePin is one table's pinned row. -type tablePin struct { - t *table - row int -} - // hold is what has already been drawn for held names: the variant each was drawn as, // so every path under it reads one row, and the draw each read made, by its one -// spelling, so the same read written twice reads one draw. An expansion keeps one for -// its sibling names, and a render one per group for its reference paths. +// spelling, so the same read written twice reads one draw, and the table rows its reads pinned. An +// expansion keeps one for its sibling names, and a render one per group for its reference paths. type hold struct { variant map[string]node value map[string]draw - pins [8]tablePin // the rows pinned, inline so a render over a country's five-deep geo tree stays off the heap - npins int - more map[*table]int // the rows pinned past the inline eight - s *session // what draws a row; nil where a walk only proves selectors + pins pinSet } // draw is what one read drew: its text, and whether it landed on a null. @@ -30,129 +20,6 @@ type draw struct { null bool } -// pinned is the row the render pinned for t, if any. -func (d *hold) pinned(t *table) (int, bool) { - for _, p := range d.pins[:d.npins] { - if p.t == t { - return p.row, true - } - } - r, ok := d.more[t] - return r, ok -} - -// mustRow is the row pinned for t, which the walk reaching a column pinned. -func (d *hold) mustRow(t *table) int { - r, ok := d.pinned(t) - if !ok { - panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) - } - return r -} - -// pin pins row r of t, and the rows of t's ancestors it links to. -func (d *hold) pin(t *table, r int) { - for { - if _, done := d.pinned(t); done { - return - } - if d.npins < len(d.pins) { - d.pins[d.npins] = tablePin{t, r} - d.npins++ - } else { - if d.more == nil { - d.more = map[*table]int{} - } - d.more[t] = r - } - if t.parentT == nil { - return - } - t, r = t.parentT, t.parentRow(r) - } -} - -// each calls fn for every pinned row, in pin order, the spilled ones by table name. -func (d *hold) each(fn func(t *table, r int)) { - for _, p := range d.pins[:d.npins] { - fn(p.t, p.row) - } - spilled := make([]*table, 0, len(d.more)) - for t := range d.more { - spilled = append(spilled, t) - } - sort.Slice(spilled, func(i, j int) bool { return spilled[i].category < spilled[j].category }) - for _, t := range spilled { - fn(t, d.more[t]) - } -} - -// rowOf is the render's row of t: the one pinned, else one drawn inside the -// nearest pinned ancestor — its parent drawn inside that first where the ancestor -// is further up — or over the whole table, and pinned with its ancestors. -func (d *hold) rowOf(t *table) int { - if r, ok := d.pinned(t); ok { - return r - } - r := -1 - for a := t.parentT; a != nil && r < 0; a = a.parentT { - if _, ok := d.pinned(a); !ok { - continue - } - if t.parentT != a { - d.rowOf(t.parentT) - } - pr, _ := d.pinned(t.parentT) - r = t.drawUnder(d.s, pr) - } - if r < 0 { - r = t.draw(d.s) - } - d.pin(t, r) - return r -} - -// pinRow pins row r of t where it agrees with the rows pinned before it. -func (d *hold) pinRow(t *table, r int) error { - if pr, ok := d.pinned(t); ok && pr != r { - return fmt.Errorf("%s and %s are two rows of %s", t.selectorSpelling(pr), t.selectorSpelling(r), t.path) - } - for a := t.parentT; a != nil; a = a.parentT { - if pa, ok := d.pinned(a); ok && !t.under(r, a, pa) { - return fmt.Errorf("%s is not inside %s", t.selectorSpelling(r), a.selectorSpelling(pa)) - } - } - d.pin(t, r) - return nil -} - -// selectRow pins the row a selector names. -func (d *hold) selectRow(t *table, sel string) error { - r, err := t.find(sel, d) - if err != nil { - return err - } - return d.pinRow(t, r) -} - -// inside keeps the rows of t that sit inside every pinned ancestor. -func (d *hold) inside(t *table, rows []int) []int { - for a := t.parentT; a != nil; a = a.parentT { - pa, ok := d.pinned(a) - if !ok { - continue - } - var kept []int - for _, r := range rows { - if t.under(r, a, pa) { - kept = append(kept, r) - } - } - rows = kept - } - return rows -} - // readField renders one arm of a token. An arm's key is a sibling field or a // reference linkRefs bound into fields. A name the expansion holds — a level some // token addresses by dotted path, or a field an operand reads — is drawn once and @@ -167,11 +34,11 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw } return draw{text: render(s, t.fields[a.key], sc)} } - d := readHold(s, held, sc, a) + d := readHold(held, sc, a) if r, done := d.value[a.path]; done { return r } - leaf, err := walkPath(t.fields[a.key], a.tail, pathWalk{ + leaf, err := walkPath(s, t.fields[a.key], a.tail, pathWalk{ // Hold the draw at every level passed through, so two paths sharing a // prefix share it. choice: func(c *choice, rest []string) ([]node, error) { @@ -189,7 +56,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw } return []node{n}, nil }, - pins: d, + pins: &d.pins, }) if err != nil { panic(fmt.Sprintf("fejkdata: %q: %v; a fence should have refused this at New", a.name, err)) @@ -205,9 +72,9 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw // readHold is the hold a held read keeps its draw in: for a reference that reads a path, // the render's hold for its group, so its draw spans the render; for a sibling, or a // reference read whole, held. -func readHold(s *session, held *hold, sc renderScope, a arm) *hold { +func readHold(held *hold, sc renderScope, a arm) *hold { if isRef(a.key) && len(a.tail) > 0 { - return sc.hold(s) + return sc.hold() } return held } @@ -221,7 +88,7 @@ func renderLeaf(s *session, n node, sc renderScope) draw { } r := draw{text: render(s, n, sc)} if t, _ := n.(*template); t != nil && t.readsColumn != nil { - r.null = sc.in(t).hold(s).value[t.readsColumn.a.path].null + r.null = sc.in(t).hold().value[t.readsColumn.a.path].null } return r } diff --git a/holdfence.go b/holdfence.go index 562c915..63380d3 100644 --- a/holdfence.go +++ b/holdfence.go @@ -106,7 +106,7 @@ func operandReader(t *template, head string) string { // coverPath collects what holding one path pins: every choice level the path // passes through, whole, and the leaf it renders. func coverPath(n node, tail []string, into map[node]bool) { - _, _ = walkPath(n, tail, pathWalk{ + _, _ = walkPath(nil, n, tail, pathWalk{ choice: func(c *choice, _ []string) ([]node, error) { cover(c, into, false); return nil, nil }, leaf: func(n node) error { cover(n, into, false); return nil }, }) diff --git a/holdset.go b/holdset.go index 46f648a..303509e 100644 --- a/holdset.go +++ b/holdset.go @@ -20,18 +20,14 @@ type renderScope struct { } // eagerHoldSet makes the unnamed group's maps where the set is declared, keeping them on that frame's -// stack for a render that reads through them; lazyHoldSet makes them on its first read instead. -func eagerHoldSet(s *session) holdSet { - return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}, s: s}} -} - -func lazyHoldSet(s *session) holdSet { - return holdSet{unnamed: hold{s: s}} +// stack for a render that reads through them; a zero holdSet makes them on its first read instead. +func eagerHoldSet() holdSet { + return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}}} } // renderOnce renders n as one render, over a hold set of its own. func renderOnce(s *session, n node) string { - set := lazyHoldSet(s) + var set holdSet return render(s, n, renderScope{set: &set}) } @@ -40,7 +36,7 @@ func renderOnce(s *session, n node) string { // //go:noinline func expandAnew(s *session, t *template) string { - set := lazyHoldSet(s) + var set holdSet return expand(s, t, renderScope{set: &set}) } @@ -52,9 +48,8 @@ func (sc renderScope) in(t *template) renderScope { return sc } -// hold is the set's hold for sc's draw group. The session is passed in rather than -// read out of the set: copying it from there would leak the set's maps to the heap. -func (sc renderScope) hold(s *session) *hold { +// hold is the set's hold for sc's draw group. +func (sc renderScope) hold() *hold { if sc.group == "" { return &sc.set.unnamed } @@ -63,7 +58,7 @@ func (sc renderScope) hold(s *session) *hold { if sc.set.named == nil { sc.set.named = map[string]*hold{} } - d = &hold{variant: map[string]node{}, value: map[string]draw{}, s: s} + d = &hold{variant: map[string]node{}, value: map[string]draw{}} sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's hold set, to the heap } return d diff --git a/path.go b/path.go index 20fa50e..4b388d8 100644 --- a/path.go +++ b/path.go @@ -127,27 +127,27 @@ func names(segs []string) []string { // pathWalk is what one walk of a dotted path does at each kind of level: choice // returns the variants to continue into (none stops the walk), and where it is nil -// the walk draws one with the pins' session, or stops; level runs at each template -// a segment descends into; leaf runs where the tail ends; pins is where the walk -// pins the table rows it selects and draws, nil skipping tables. A nil action is -// skipped. A render's walk sets only pins, so it allocates nothing. +// the walk draws one, or stops; level runs at each template a segment descends +// into; leaf runs where the tail ends; pins is where the walk pins the table rows +// it selects and draws, nil skipping tables. A nil action is skipped. A render's +// walk sets only pins, so it allocates nothing. type pathWalk struct { choice func(c *choice, rest []string) ([]node, error) level func(t *template, rest []string) error leaf func(n node) error - pins *hold + pins *pinSet } // walkPath descends tail from n and returns the node it ends at: a folder or // template by its next segment, a choice by w.choice, which consumes no segment, a // table by walkTable. A missing segment is an error, so no walk reaches past what -// the data holds. -func walkPath(n node, tail []string, w pathWalk) (node, error) { +// the data holds. s draws rows and variants; nil, the walk only proves the path. +func walkPath(s *session, n node, tail []string, w pathWalk) (node, error) { if len(tail) == 0 { return n, w.atLeaf(n) } if t, isTable := n.(*table); isTable { - return walkTable(t, tail, w, false) + return walkTable(s, t, tail, w, false) } if isSelector(tail[0]) { return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(tail[0])) @@ -158,7 +158,7 @@ func walkPath(n node, tail []string, w pathWalk) (node, error) { if !ok { return nil, fmt.Errorf("no entry %q", tail[0]) } - return walkPath(child, tail[1:], w) + return walkPath(s, child, tail[1:], w) case *template: if w.level != nil { if err := w.level(n, tail); err != nil { @@ -169,9 +169,9 @@ func walkPath(n node, tail []string, w pathWalk) (node, error) { if !ok { return nil, fmt.Errorf("no field %q", tail[0]) } - return walkPath(child, tail[1:], w) + return walkPath(s, child, tail[1:], w) case *choice: - return walkChoice(n, tail, w) + return walkChoice(s, n, tail, w) case *column: return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", tail[0], n.t.columns[n.i]) } @@ -186,8 +186,8 @@ func (w pathWalk) atLeaf(n node) error { } // walkChoice continues a walk into the variants w.choice returns, or into one drawn -// by the pins' session where the walk names no choice action. -func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { +// by s where the walk names no choice action. +func walkChoice(s *session, c *choice, tail []string, w pathWalk) (node, error) { if w.choice == nil { if w.pins == nil { return nil, nil @@ -195,10 +195,10 @@ func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { if err := carriedByAll(c, tail); err != nil { return nil, err } - if w.pins.s == nil { // a probe: every variant carries the tail, so any one proves it - return walkPath(c.items[0], tail, w) + if s == nil { // a probe: every variant carries the tail, so any one proves it + return walkPath(s, c.items[0], tail, w) } - return walkPath(pick(w.pins.s, c), tail, w) + return walkPath(s, pick(s, c), tail, w) } next, err := w.choice(c, tail) if err != nil { @@ -206,7 +206,7 @@ func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { } var last node for _, item := range next { - if last, err = walkPath(item, tail, w); err != nil { + if last, err = walkPath(s, item, tail, w); err != nil { return nil, err } } @@ -217,7 +217,7 @@ func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { // linked table by name. The walk reads a row wherever a segment follows or the // table was reached from another; a table reached whole, with no selector, is left // to a render's own draw. -func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error) { +func walkTable(s *session, t *table, tail []string, w pathWalk, descended bool) (node, error) { sel, tail, err := t.selector(tail) if err != nil { return nil, err @@ -229,19 +229,19 @@ func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error // A selector further down pins this table by ancestry, so the walk draws only // where none follows; drawing first could pick a row the selector is not inside. if w.pins != nil { - if err := readRow(w.pins, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { + if err := readRow(s, w.pins, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { return nil, err } } switch { case len(tail) == 0 && sel == "" && !descended: - return walkPath(t, nil, w) + return walkPath(s, t, nil, w) case len(tail) == 0: - return walkPath(t.whole, nil, w) + return walkPath(s, t.whole, nil, w) case child != nil: - return walkTable(child, tail[1:], w, true) + return walkTable(s, child, tail[1:], w, true) } - return walkPath(column, tail[1:], w) + return walkPath(s, column, tail[1:], w) } // selector splits the selector a tail starts with from the rest of it. @@ -272,12 +272,12 @@ func (t *table) step(tail []string) (column node, child *table, err error) { // readRow pins the row a path reads of t: the one its selector names, or, where the // walk draws, one drawn where the path reads into the table. -func readRow(d *hold, t *table, sel string, draw bool) error { +func readRow(s *session, pins *pinSet, t *table, sel string, draw bool) error { if sel != "" { - return d.selectRow(t, sel) + return pins.selectRow(t, sel) } - if draw && d.s != nil { - d.rowOf(t) + if draw && s != nil { + pins.rowOf(s, t) } return nil } @@ -312,8 +312,8 @@ func unreachableInChoice(c *choice, want string) error { // carries a repeat or a drawGroup, which one draw of it could not apply. So a path // that validates here resolves on every render, and a typo is a New-time error. func checkPath(n node, tail []string, level string) error { - var pins hold - _, err := walkPath(n, tail, pathWalk{ + var pins pinSet + _, err := walkPath(nil, n, tail, pathWalk{ choice: func(c *choice, rest []string) ([]node, error) { if err := carriedByAll(c, rest); err != nil { return nil, err diff --git a/path_test.go b/path_test.go index 4a8ce24..f4260d0 100644 --- a/path_test.go +++ b/path_test.go @@ -13,14 +13,14 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) { level: func(tm *template, rest []string) error { seen = append(seen, "level:"+rest[0]); return nil }, leaf: func(n node) error { seen = append(seen, "leaf"); return nil }, } - if _, err := walkPath(n, []string{"a", "b"}, walk); err != nil { + if _, err := walkPath(nil, n, []string{"a", "b"}, walk); err != nil { t.Fatalf("walkPath(a.b) = %v", err) } if want := []string{"level:a", "level:b", "leaf"}; !slices.Equal(seen, want) { t.Errorf("walk visited %v, want %v", seen, want) } seen = nil - _, err := walkPath(n, []string{"a", "nope", "deeper"}, walk) + _, err := walkPath(nil, n, []string{"a", "nope", "deeper"}, walk) if err == nil || !strings.Contains(err.Error(), `no field "nope"`) { t.Errorf("walkPath(a.nope.deeper) = %v, want the missing segment named", err) } @@ -32,7 +32,7 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) { func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) var leaves []node - _, err := walkPath(n, []string{"f"}, pathWalk{ + _, err := walkPath(nil, n, []string{"f"}, pathWalk{ choice: func(c *choice, rest []string) ([]node, error) { if len(rest) != 1 || rest[0] != "f" { t.Errorf("choice saw rest %v, want [f]", rest) diff --git a/pins.go b/pins.go new file mode 100644 index 0000000..7014880 --- /dev/null +++ b/pins.go @@ -0,0 +1,192 @@ +package fejkdata + +import ( + "fmt" + "maps" + "sort" + "strings" +) + +// tablePin is one table's pinned row. +type tablePin struct { + t *table + row int +} + +// pinSet is the rows of tables fixed so far: by a render, by a path read replayed at load, and by +// a fence walk over the rows a render can enter. +type pinSet struct { + pins [8]tablePin // inline, so a render over a country's five-deep geo tree stays off the heap + n int + spill map[*table]int +} + +// pinned is the row pinned for t, if any. +func (p *pinSet) pinned(t *table) (int, bool) { + for _, q := range p.pins[:p.n] { + if q.t == t { + return q.row, true + } + } + r, ok := p.spill[t] + return r, ok +} + +// mustRow is the row pinned for t, which the walk reaching a column pinned. +func (p *pinSet) mustRow(t *table) int { + r, ok := p.pinned(t) + if !ok { + panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) + } + return r +} + +// add pins row r of t alone. +func (p *pinSet) add(t *table, r int) { + if p.n < len(p.pins) { + p.pins[p.n] = tablePin{t, r} + p.n++ + return + } + if p.spill == nil { + p.spill = map[*table]int{} + } + p.spill[t] = r +} + +// pin pins row r of t, and the rows of t's ancestors it links to. +func (p *pinSet) pin(t *table, r int) { + for { + if _, done := p.pinned(t); done { + return + } + p.add(t, r) + if t.parentT == nil { + return + } + t, r = t.parentT, t.parentRow(r) + } +} + +// each calls fn for every pinned row, in pin order, the spilled ones by table name. +func (p *pinSet) each(fn func(t *table, r int)) { + for _, q := range p.pins[:p.n] { + fn(q.t, q.row) + } + spilled := make([]*table, 0, len(p.spill)) + for t := range p.spill { + spilled = append(spilled, t) + } + sort.Slice(spilled, func(i, j int) bool { return spilled[i].category < spilled[j].category }) + for _, t := range spilled { + fn(t, p.spill[t]) + } +} + +// rowOf is the row of t: the one pinned, else one s draws inside the nearest pinned ancestor — its +// parent drawn inside that first where the ancestor is further up — or over the whole table, and +// pinned with its ancestors. +func (p *pinSet) rowOf(s *session, t *table) int { + if r, ok := p.pinned(t); ok { + return r + } + r := -1 + for a := t.parentT; a != nil && r < 0; a = a.parentT { + if _, ok := p.pinned(a); !ok { + continue + } + if t.parentT != a { + p.rowOf(s, t.parentT) + } + pr, _ := p.pinned(t.parentT) + r = t.drawUnder(s, pr) + } + if r < 0 { + r = t.draw(s) + } + p.pin(t, r) + return r +} + +// clash is the table whose pinned row keeps row r of t out: t itself pinned to another row, or the +// nearest ancestor pinned to a row r is not inside; nil where none does. +func (p *pinSet) clash(t *table, r int) *table { + if pr, ok := p.pinned(t); ok && pr != r { + return t + } + for a := t.parentT; a != nil; a = a.parentT { + if pa, ok := p.pinned(a); ok && !t.under(r, a, pa) { + return a + } + } + return nil +} + +// pinRow pins row r of t where it agrees with the rows pinned before it. +func (p *pinSet) pinRow(t *table, r int) error { + switch a := p.clash(t, r); { + case a == t: + pr, _ := p.pinned(t) + return fmt.Errorf("%s and %s are two rows of %s", t.selectorSpelling(pr), t.selectorSpelling(r), t.path) + case a != nil: + pa, _ := p.pinned(a) + return fmt.Errorf("%s is not inside %s", t.selectorSpelling(r), a.selectorSpelling(pa)) + } + p.pin(t, r) + return nil +} + +// selectRow pins the row a selector names. +func (p *pinSet) selectRow(t *table, sel string) error { + r, err := t.find(sel, p) + if err != nil { + return err + } + return p.pinRow(t, r) +} + +// inside keeps the rows of t that sit inside every pinned ancestor. +func (p *pinSet) inside(t *table, rows []int) []int { + for a := t.parentT; a != nil; a = a.parentT { + pa, ok := p.pinned(a) + if !ok { + continue + } + var kept []int + for _, r := range rows { + if t.under(r, a, pa) { + kept = append(kept, r) + } + } + rows = kept + } + return rows +} + +// entered is p with row r of t: pinned where the render pins it, alone where the render draws it +// to render whole. p is left as it was, since a fence walk branches. +func (p pinSet) entered(t *table, r int, pins bool) pinSet { + p.spill = maps.Clone(p.spill) + switch _, in := p.pinned(t); { + case pins: + p.pin(t, r) + case !in: + p.add(t, r) + } + return p +} + +// excludes reports whether a cell's row cannot render with the rows pinned. +func (p *pinSet) excludes(cell *template) bool { return p.clash(cell.cellOf, cell.cellRow) != nil } + +// key spells the set for a map, by the tables' identities in path order. +func (p *pinSet) key() string { + var pins []tablePin + p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) + sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) + var b strings.Builder + for _, q := range pins { + fmt.Fprintf(&b, "%p[%d]", q.t, q.row) + } + return b.String() +} diff --git a/record.go b/record.go index c98a892..9ef6a80 100644 --- a/record.go +++ b/record.go @@ -130,7 +130,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) } - set := eagerHoldSet(f.rand) + set := eagerHoldSet() sc := renderScope{set: &set} if t, isTable := n.(*table); isTable { if n, err = tableRecord(f.rand, t, tail, sc); err != nil { @@ -159,7 +159,7 @@ func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, err } switch n := n.(type) { case *table: - sc.hold(s).rowOf(n) + sc.hold().pins.rowOf(s, n) return n, nil case *row: return n.t, nil @@ -204,7 +204,7 @@ type RecordTemplate struct { func (t *RecordTemplate) Fake() *Record { t.g.mu.Lock() defer t.g.mu.Unlock() - set := eagerHoldSet(t.g.rand) + set := eagerHoldSet() return renderRecord(t.g.rand, t.t, t.columns, renderScope{set: &set}) } @@ -265,7 +265,7 @@ func recordOf(n node) (*template, []Column, error) { func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.table != nil { - sc.t, sc.row = t.table, sc.hold(s).mustRow(t.table) + sc.t, sc.row = t.table, sc.hold().pins.mustRow(t.table) } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { diff --git a/render.go b/render.go index c51115c..3ff95ba 100644 --- a/render.go +++ b/render.go @@ -23,7 +23,7 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } - f.set = lazyHoldSet(f.rand) + f.set = holdSet{} sc := renderScope{set: &f.set} f.root.children = f.categories n, err := descend(f.rand, &f.root, segments, sc) @@ -44,11 +44,11 @@ func (f *Generator) Fake(path string) (string, error) { // every call. func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // Walked once without drawing first, so a path that fails moves no seeded stream. - var probe hold - if _, err := walkPath(root, segments, pathWalk{pins: &probe}); err != nil { + var probe pinSet + if _, err := walkPath(nil, root, segments, pathWalk{pins: &probe}); err != nil { return nil, err } - return walkPath(root, segments, pathWalk{pins: sc.hold(s)}) + return walkPath(s, root, segments, pathWalk{pins: &sc.hold().pins}) } // render evaluates a compiled node to a string. compile validates every node up @@ -64,11 +64,11 @@ func render(s *session, n node, sc renderScope) string { sc.t, sc.row = n, n.draw(s) return expand(s, n.format, sc) case *row: - sc.t, sc.row = n.t, sc.hold(s).mustRow(n.t) + sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) return expand(s, n.t.format, sc) case *column: if sc.t != n.t { - sc.t, sc.row = n.t, sc.hold(s).mustRow(n.t) + sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) } if cell := n.t.cellNode(sc.row, n.i); cell != nil { return render(s, cell, sc) diff --git a/struct.go b/struct.go index de3ef95..aeb5e2c 100644 --- a/struct.go +++ b/struct.go @@ -349,7 +349,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro // fill draws the record into v's tagged fields, then each nested struct as a record of its own. func (s *structShape) fill(sess *session, v reflect.Value) { if s.record != nil { - set := eagerHoldSet(sess) + set := eagerHoldSet() for i, c := range renderRecord(sess, s.record, s.columns, renderScope{set: &set}).columns { setColumn(fieldAt(v, s.fields[i]), c) } diff --git a/table.go b/table.go index 6060f07..df254cb 100644 --- a/table.go +++ b/table.go @@ -524,8 +524,8 @@ func (t *table) under(r int, a *table, pr int) bool { } // find is the row a selector names: by key first, then by name, where a name -// naming several rows resolves inside the ancestors pinned in d. -func (t *table) find(sel string, d *hold) (int, error) { +// naming several rows resolves inside the ancestors pinned. +func (t *table) find(sel string, pins *pinSet) (int, error) { if t.key < 0 && t.name < 0 { return 0, fmt.Errorf("%s has no key or name column to select a row by", t.path) } @@ -534,7 +534,7 @@ func (t *table) find(sel string, d *hold) (int, error) { } rows := t.indexed().byName[sel] if len(rows) > 1 { - rows = d.inside(t, rows) + rows = pins.inside(t, rows) } switch len(rows) { case 1: diff --git a/template_test.go b/template_test.go index bd5a9e3..f4741cf 100644 --- a/template_test.go +++ b/template_test.go @@ -316,7 +316,7 @@ func TestGrowIsALowerBound(t *testing.T) { t.Fatalf("format %q did not compile to a template", format) } for i := 0; i < 50; i++ { - set := lazyHoldSet(f.rand) + var set holdSet if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.grow { t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) } diff --git a/todo.md b/todo.md index a51120d..378d08e 100644 --- a/todo.md +++ b/todo.md @@ -22,10 +22,6 @@ by nothing but tests, whose premise lives only in the README's Decisions. This round, in order: -- Give render and the draw fences one pin model: a pin set `hold.pin`, `hold.rowOf` - and the fence's `rowSet` all use, so a change to how a render pins rows cannot leave - the fence proving the old rule, and `hold` stops being a proof scratchpad whenever its - session is nil. - Move the README's Decisions section to `docs/decisions.md`, leaving a one-line index of the titles in `AGENTS.md`, and drop the `AGENTS.md` line pointing decisions at the README. Give each fence a bare link to the entry it enforces: every seat needed @@ -46,7 +42,7 @@ This round, in order: `operandReader`, `checkNoRepeatedRead` and `compileOps`, and function-versus-field is decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which - callbacks are nil and whether `pins.s` is, and `walkChoice` returns `nil, nil` where + callbacks are nil and whether its session is, and `walkChoice` returns `nil, nil` where none is set. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while @@ -287,7 +283,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Hold every change the charter says owes a `CHANGELOG.md` entry to one in CI: the check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -- Measure the allocation gate over the shipped `geo` trees too: `hold.pins [8]` is sized +- Measure the allocation gate over the shipped `geo` trees too: `pinSet.pins [8]` is sized for them, and `perf_test.go` checks a synthetic five-deep tree only. - Make a named draw group's hold lazy, `&hold{s: s}` in `renderScope.hold`: it escapes to the heap anyway, so its two eager maps buy no stack and cost two allocations per -- 2.52.0 From 85d9d88d0a071b43c50c0a7518ad01b969b99495 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 00:20:55 +0200 Subject: [PATCH 064/966] Hold the rows rowOf draws to what pinning refuses --- table_test.go | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/table_test.go b/table_test.go index f9987b1..b7f4e79 100644 --- a/table_test.go +++ b/table_test.go @@ -964,9 +964,9 @@ func TestAmbiguousNameNamesARunnablePath(t *testing.T) { } } -// TestEnteredRowsAgreeWithPinning holds what a fence walk decides about the rows it entered to what -// pinning them refuses at render: drift between the two is data that loads and then renders a -// family that disagrees. +// TestEnteredRowsAgreeWithPinning holds what a fence walk decides about the rows it entered, and +// the rows a render draws, to what pinning refuses: drift between them is data that loads and then +// renders a family that disagrees. func TestEnteredRowsAgreeWithPinning(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) var tables []*table @@ -988,12 +988,15 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { } s := none.entered(entered, row, true) for _, cand := range tables { + for i := 0; i < 20; i++ { + drawing := s + if r := drawing.rowOf(f.rand, cand); s.clash(cand, r) != nil { + t.Errorf("rowOf draws %s beside %s, which pinning it there refuses", cand.selectorSpelling(r), entered.selectorSpelling(row)) + } + } for cr := 0; cr < cand.rows(); cr++ { beside := s refused := beside.pinRow(cand, cr) != nil - if got := s.excludes(&template{cellOf: cand, cellRow: cr}); got != refused { - t.Errorf("excludes(%s) = %v beside %s, but pinRow refuses it = %v", cand.selectorSpelling(cr), got, entered.selectorSpelling(row), refused) - } if got := alternatives(drawAt{alt: s}, drawAt{alt: none.entered(cand, cr, true)}); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } -- 2.52.0 From 6b788e324af6e94273685416cf7cfd9ee9945ad7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 00:20:55 +0200 Subject: [PATCH 065/966] Say a fence walk's pin set holds a whole row alone, and correct todo.md's hold spelling --- pins.go | 2 +- todo.md | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/pins.go b/pins.go index 7014880..266a4a9 100644 --- a/pins.go +++ b/pins.go @@ -14,7 +14,7 @@ type tablePin struct { } // pinSet is the rows of tables fixed so far: by a render, by a path read replayed at load, and by -// a fence walk over the rows a render can enter. +// a fence walk over the rows a render can enter, which also holds a row rendered whole, alone. type pinSet struct { pins [8]tablePin // inline, so a render over a country's five-deep geo tree stays off the heap n int diff --git a/todo.md b/todo.md index 378d08e..e4ddc62 100644 --- a/todo.md +++ b/todo.md @@ -43,7 +43,8 @@ This round, in order: decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which callbacks are nil and whether its session is, and `walkChoice` returns `nil, nil` where - none is set. + none is set. The mode may live in `pathWalk`; the session may not, since beside `pins` it + moves the hold maps to the heap. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while render calls the second. Derive the numeric builtins `valueProof.template` lists in @@ -285,7 +286,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. flag, an exit code, an exported name, a fence, a builtin and the lowest Go. - Measure the allocation gate over the shipped `geo` trees too: `pinSet.pins [8]` is sized for them, and `perf_test.go` checks a synthetic five-deep tree only. -- Make a named draw group's hold lazy, `&hold{s: s}` in `renderScope.hold`: it escapes +- Make a named draw group's hold lazy, `&hold{}` in `renderScope.hold`: it escapes to the heap anyway, so its two eager maps buy no stack and cost two allocations per group that may never read through them. Check it against the alloc tests. - Name goal 2's four criteria with the axes the panel scores, so a recorded score maps -- 2.52.0 From 939f408ef8be8cc8e7c19259aa82f3cad1fee477 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 00:33:15 +0200 Subject: [PATCH 066/966] Give a fence walk's pin set one copy, and file the drawn-row half of the pin model --- family.go | 3 ++- pins.go | 8 +++++++- todo.md | 4 +++- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/family.go b/family.go index c89e2a6..c22474d 100644 --- a/family.go +++ b/family.go @@ -225,7 +225,8 @@ func checkOwnFamily(t *template) error { return nil } -// alternatives reports whether two reads sit in different rows of one table. +// alternatives reports whether two reads sit in different rows of one table. Not clash: a row drawn +// whole is held without its ancestors, since the render draws it over the whole table. func alternatives(a, b drawAt) bool { found := false a.alt.each(func(t *table, r int) { diff --git a/pins.go b/pins.go index 266a4a9..e1abce4 100644 --- a/pins.go +++ b/pins.go @@ -163,10 +163,16 @@ func (p *pinSet) inside(t *table, rows []int) []int { return rows } +// clone is a copy of p that pins apart from it; a plain copy shares the spill map. +func (p pinSet) clone() pinSet { + p.spill = maps.Clone(p.spill) + return p +} + // entered is p with row r of t: pinned where the render pins it, alone where the render draws it // to render whole. p is left as it was, since a fence walk branches. func (p pinSet) entered(t *table, r int, pins bool) pinSet { - p.spill = maps.Clone(p.spill) + p = p.clone() switch _, in := p.pinned(t); { case pins: p.pin(t, r) diff --git a/todo.md b/todo.md index e4ddc62..83fb7ed 100644 --- a/todo.md +++ b/todo.md @@ -43,7 +43,9 @@ This round, in order: decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which callbacks are nil and whether its session is, and `walkChoice` returns `nil, nil` where - none is set. The mode may live in `pathWalk`; the session may not, since beside `pins` it + none is set. Have the check mode report the tables a read draws, so `tableRead.drawn` + and `entered`'s `pins` flag come from the walk rather than a second copy of + `walkTable`'s draw condition and `rowOf`'s ancestor rule. The mode may live in `pathWalk`; the session may not, since beside `pins` it moves the hold maps to the heap. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while -- 2.52.0 From 30a39916b6a5da5ef5e5fa5f208a81b879cfed51 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 00:33:15 +0200 Subject: [PATCH 067/966] Copy the test's pin sets through clone --- table_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/table_test.go b/table_test.go index b7f4e79..b78fc61 100644 --- a/table_test.go +++ b/table_test.go @@ -989,13 +989,13 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { s := none.entered(entered, row, true) for _, cand := range tables { for i := 0; i < 20; i++ { - drawing := s + drawing := s.clone() if r := drawing.rowOf(f.rand, cand); s.clash(cand, r) != nil { t.Errorf("rowOf draws %s beside %s, which pinning it there refuses", cand.selectorSpelling(r), entered.selectorSpelling(row)) } } for cr := 0; cr < cand.rows(); cr++ { - beside := s + beside := s.clone() refused := beside.pinRow(cand, cr) != nil if got := alternatives(drawAt{alt: s}, drawAt{alt: none.entered(cand, cr, true)}); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) -- 2.52.0 From 00e6b7d357acffa3c95c88641eba08b9a406b0e9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 09:54:05 +0200 Subject: [PATCH 068/966] Name the pin set's inline array and entered's flag apart from pins, and file the draw half of the pin model as its own item --- doc_test.go | 2 +- drawfence.go | 2 +- pins.go | 21 +++++++++------------ todo.md | 12 ++++++++---- 4 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc_test.go b/doc_test.go index fe5b297..c805159 100644 --- a/doc_test.go +++ b/doc_test.go @@ -95,7 +95,7 @@ func vocabulary(t *testing.T, files []*ast.File) string { } // declaredSymbols is every name the package declares at the top level, a method and -// a struct field keyed under its type: hold.pin, tableRead.whole. +// a struct field keyed under its type: pinSet.pin, tableRead.whole. func declaredSymbols(files []*ast.File) map[string]bool { names := declaredFuncs(files) for _, f := range files { diff --git a/drawfence.go b/drawfence.go index 824403e..ecbffa4 100644 --- a/drawfence.go +++ b/drawfence.go @@ -229,7 +229,7 @@ func (w *drawWalk) walk(n node, at drawAt) { } for _, e := range renderEdges(n) { if cell, isCell := e.to.(*template); isCell && cell.cellOf != nil { - if at.alt.excludes(cell) { + if at.alt.clash(cell.cellOf, cell.cellRow) != nil { continue } in := at diff --git a/pins.go b/pins.go index e1abce4..9c5e469 100644 --- a/pins.go +++ b/pins.go @@ -16,14 +16,14 @@ type tablePin struct { // pinSet is the rows of tables fixed so far: by a render, by a path read replayed at load, and by // a fence walk over the rows a render can enter, which also holds a row rendered whole, alone. type pinSet struct { - pins [8]tablePin // inline, so a render over a country's five-deep geo tree stays off the heap - n int - spill map[*table]int + inline [8]tablePin // sized so a render over a country's five-deep geo tree stays off the heap + n int + spill map[*table]int } // pinned is the row pinned for t, if any. func (p *pinSet) pinned(t *table) (int, bool) { - for _, q := range p.pins[:p.n] { + for _, q := range p.inline[:p.n] { if q.t == t { return q.row, true } @@ -43,8 +43,8 @@ func (p *pinSet) mustRow(t *table) int { // add pins row r of t alone. func (p *pinSet) add(t *table, r int) { - if p.n < len(p.pins) { - p.pins[p.n] = tablePin{t, r} + if p.n < len(p.inline) { + p.inline[p.n] = tablePin{t, r} p.n++ return } @@ -70,7 +70,7 @@ func (p *pinSet) pin(t *table, r int) { // each calls fn for every pinned row, in pin order, the spilled ones by table name. func (p *pinSet) each(fn func(t *table, r int)) { - for _, q := range p.pins[:p.n] { + for _, q := range p.inline[:p.n] { fn(q.t, q.row) } spilled := make([]*table, 0, len(p.spill)) @@ -171,10 +171,10 @@ func (p pinSet) clone() pinSet { // entered is p with row r of t: pinned where the render pins it, alone where the render draws it // to render whole. p is left as it was, since a fence walk branches. -func (p pinSet) entered(t *table, r int, pins bool) pinSet { +func (p pinSet) entered(t *table, r int, pinsAncestors bool) pinSet { p = p.clone() switch _, in := p.pinned(t); { - case pins: + case pinsAncestors: p.pin(t, r) case !in: p.add(t, r) @@ -182,9 +182,6 @@ func (p pinSet) entered(t *table, r int, pins bool) pinSet { return p } -// excludes reports whether a cell's row cannot render with the rows pinned. -func (p *pinSet) excludes(cell *template) bool { return p.clash(cell.cellOf, cell.cellRow) != nil } - // key spells the set for a map, by the tables' identities in path order. func (p *pinSet) key() string { var pins []tablePin diff --git a/todo.md b/todo.md index 83fb7ed..a324f76 100644 --- a/todo.md +++ b/todo.md @@ -43,10 +43,14 @@ This round, in order: decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which callbacks are nil and whether its session is, and `walkChoice` returns `nil, nil` where - none is set. Have the check mode report the tables a read draws, so `tableRead.drawn` - and `entered`'s `pins` flag come from the walk rather than a second copy of - `walkTable`'s draw condition and `rowOf`'s ancestor rule. The mode may live in `pathWalk`; the session may not, since beside `pins` it - moves the hold maps to the heap. + none is set. The mode may live in `pathWalk`; the session may not, since beside `pins` + it moves the hold maps to the heap. +- Have the check mode report the tables a read draws, so `tableRead.drawn` and + `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and + `rowOf`'s ancestor rule: a change to how a render draws rows still leaves the fence + proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so + `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Needs the + item above. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while render calls the second. Derive the numeric builtins `valueProof.template` lists in -- 2.52.0 From bbcb259b02638136a2d137c783ab443989679321 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 10:01:40 +0200 Subject: [PATCH 069/966] Hold a spilled pin set apart from what enters beside it --- table_test.go | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/table_test.go b/table_test.go index b78fc61..f3d9cd8 100644 --- a/table_test.go +++ b/table_test.go @@ -1004,6 +1004,16 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { } } } + var spilled pinSet + for i := 0; i <= len(spilled.inline); i++ { + spilled = spilled.entered(&table{path: strconv.Itoa(i)}, 0, false) + } + _ = spilled.entered(&table{path: "past"}, 0, false) + n := 0 + spilled.each(func(*table, int) { n++ }) + if n != len(spilled.inline)+1 { + t.Errorf("entering a row beside a spilled pin set leaves it holding %d rows, want %d", n, len(spilled.inline)+1) + } } // TestCellReadsMeetWhereTheRenderPairsTheRows reads what the TSVs below do not show: region 01's -- 2.52.0 From a0fc1bc4f1ba14ae1855570a0e8d24ce314dafb3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 10:02:10 +0200 Subject: [PATCH 070/966] Order spilled pins by path, and correct todo.md's inline array name --- pins.go | 4 ++-- todo.md | 7 ++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/pins.go b/pins.go index 9c5e469..8d33cde 100644 --- a/pins.go +++ b/pins.go @@ -68,7 +68,7 @@ func (p *pinSet) pin(t *table, r int) { } } -// each calls fn for every pinned row, in pin order, the spilled ones by table name. +// each calls fn for every pinned row, in pin order, the spilled ones by path. func (p *pinSet) each(fn func(t *table, r int)) { for _, q := range p.inline[:p.n] { fn(q.t, q.row) @@ -77,7 +77,7 @@ func (p *pinSet) each(fn func(t *table, r int)) { for t := range p.spill { spilled = append(spilled, t) } - sort.Slice(spilled, func(i, j int) bool { return spilled[i].category < spilled[j].category }) + sort.Slice(spilled, func(i, j int) bool { return spilled[i].path < spilled[j].path }) for _, t := range spilled { fn(t, p.spill[t]) } diff --git a/todo.md b/todo.md index a324f76..8e81116 100644 --- a/todo.md +++ b/todo.md @@ -49,8 +49,9 @@ This round, in order: `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and `rowOf`'s ancestor rule: a change to how a render draws rows still leaves the fence proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so - `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Needs the - item above. + `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Settle a + cell of a municipality drawn whole beside a path pinning a region it is not in. Needs + the item above. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while render calls the second. Derive the numeric builtins `valueProof.template` lists in @@ -290,7 +291,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Hold every change the charter says owes a `CHANGELOG.md` entry to one in CI: the check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -- Measure the allocation gate over the shipped `geo` trees too: `pinSet.pins [8]` is sized +- Measure the allocation gate over the shipped `geo` trees too: `pinSet.inline [8]` is sized for them, and `perf_test.go` checks a synthetic five-deep tree only. - Make a named draw group's hold lazy, `&hold{}` in `renderScope.hold`: it escapes to the heap anyway, so its two eager maps buy no stack and cost two allocations per -- 2.52.0 From cc54cd6077fc5902217060d284ab4aeeb34c69bd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 10:06:28 +0200 Subject: [PATCH 071/966] Correct the prose the pin set made false, and delete what the code already says --- README.md | 2 +- hold.go | 8 ++++---- holdset.go | 3 --- path.go | 6 ++++-- pins.go | 7 ++----- todo.md | 14 ++++++-------- 6 files changed, 17 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 78ca477..c8d3bf5 100644 --- a/README.md +++ b/README.md @@ -1338,7 +1338,7 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold: one draw per expansion for paths and operands +hold.go the hold: one draw per held name, and reading a field through it holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through drawfence.go the load fences over reference draws: a drawGroup's, a render's and a record's columns, and the walk over what a render reads diff --git a/hold.go b/hold.go index ecb7f2a..b12dc21 100644 --- a/hold.go +++ b/hold.go @@ -4,10 +4,10 @@ import ( "fmt" ) -// hold is what has already been drawn for held names: the variant each was drawn as, -// so every path under it reads one row, and the draw each read made, by its one -// spelling, so the same read written twice reads one draw, and the table rows its reads pinned. An -// expansion keeps one for its sibling names, and a render one per group for its reference paths. +// hold is what has already been drawn for held names: the variant each was drawn as, so every +// path under it reads one row; the draw each read made, by its one spelling, so the same read +// written twice reads one draw; and the table rows those reads pinned. An expansion keeps one for +// its sibling names, and a render one per group for its reference paths. type hold struct { variant map[string]node value map[string]draw diff --git a/holdset.go b/holdset.go index 303509e..db1e022 100644 --- a/holdset.go +++ b/holdset.go @@ -19,8 +19,6 @@ type renderScope struct { row int } -// eagerHoldSet makes the unnamed group's maps where the set is declared, keeping them on that frame's -// stack for a render that reads through them; a zero holdSet makes them on its first read instead. func eagerHoldSet() holdSet { return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}}} } @@ -48,7 +46,6 @@ func (sc renderScope) in(t *template) renderScope { return sc } -// hold is the set's hold for sc's draw group. func (sc renderScope) hold() *hold { if sc.group == "" { return &sc.set.unnamed diff --git a/path.go b/path.go index 4b388d8..22226c7 100644 --- a/path.go +++ b/path.go @@ -127,7 +127,8 @@ func names(segs []string) []string { // pathWalk is what one walk of a dotted path does at each kind of level: choice // returns the variants to continue into (none stops the walk), and where it is nil -// the walk draws one, or stops; level runs at each template a segment descends +// the walk draws one with s, takes the first where s is nil, or stops where pins is +// nil too; level runs at each template a segment descends // into; leaf runs where the tail ends; pins is where the walk pins the table rows // it selects and draws, nil skipping tables. A nil action is skipped. A render's // walk sets only pins, so it allocates nothing. @@ -141,7 +142,8 @@ type pathWalk struct { // walkPath descends tail from n and returns the node it ends at: a folder or // template by its next segment, a choice by w.choice, which consumes no segment, a // table by walkTable. A missing segment is an error, so no walk reaches past what -// the data holds. s draws rows and variants; nil, the walk only proves the path. +// the data holds. s draws rows and variants; where it is nil, only selectors pin. +// s is a parameter: a pathWalk field beside pins moves the hold maps to the heap. func walkPath(s *session, n node, tail []string, w pathWalk) (node, error) { if len(tail) == 0 { return n, w.atLeaf(n) diff --git a/pins.go b/pins.go index 8d33cde..ca6b57c 100644 --- a/pins.go +++ b/pins.go @@ -7,21 +7,19 @@ import ( "strings" ) -// tablePin is one table's pinned row. type tablePin struct { t *table row int } -// pinSet is the rows of tables fixed so far: by a render, by a path read replayed at load, and by -// a fence walk over the rows a render can enter, which also holds a row rendered whole, alone. +// pinSet is the table rows fixed so far. A fence walk's set also holds a row rendered whole alone, +// which pin never does. type pinSet struct { inline [8]tablePin // sized so a render over a country's five-deep geo tree stays off the heap n int spill map[*table]int } -// pinned is the row pinned for t, if any. func (p *pinSet) pinned(t *table) (int, bool) { for _, q := range p.inline[:p.n] { if q.t == t { @@ -136,7 +134,6 @@ func (p *pinSet) pinRow(t *table, r int) error { return nil } -// selectRow pins the row a selector names. func (p *pinSet) selectRow(t *table, sel string) error { r, err := t.find(sel, p) if err != nil { diff --git a/todo.md b/todo.md index 8e81116..4383a05 100644 --- a/todo.md +++ b/todo.md @@ -15,10 +15,7 @@ scoring run names too little to steer one. A fresh run replaces the items below. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9, -Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. Eight of nine -named `drawWalk.walk` and `rowSet` or `checkFamilies` hardest and least wanted to modify: -a load-time copy of the pinning `hold.pin` and `hold.rowOf` do at render, tied to them -by nothing but tests, whose premise lives only in the README's Decisions. +Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: @@ -42,9 +39,8 @@ This round, in order: `operandReader`, `checkNoRepeatedRead` and `compileOps`, and function-versus-field is decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which - callbacks are nil and whether its session is, and `walkChoice` returns `nil, nil` where - none is set. The mode may live in `pathWalk`; the session may not, since beside `pins` - it moves the hold maps to the heap. + callbacks are nil and whether `walkPath`'s session is, and `walkChoice` returns + `nil, nil` where none is set. - Have the check mode report the tables a read draws, so `tableRead.drawn` and `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and `rowOf`'s ancestor rule: a change to how a render draws rows still leaves the fence @@ -59,7 +55,7 @@ This round, in order: - Split `drawCheck.reads`'s six-term condition into named predicates. - Rename what names two things: `template.go` holds the token grammar while `template` is `node.go`'s type and `Template` `inline.go`'s; `kind` `'b'` is a brace body in - `ftoken` and a builtin in `op`; `hold`'s receiver is `d`; `drawAt.alt` holds the rows + `ftoken` and a builtin in `op`; `drawAt.alt` holds the rows a walk entered. - Run the nine-seat panel again and file what it names here. @@ -307,6 +303,8 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and a helper reached only through `drawWalk.check`, and misses `checkDrawGroup`, `checkOwnFamily` and `checkColumnDraws`; the README's Layout indexes the fences by file. +- Cut the README's Layout block to the lines that say what a file name cannot: + `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. - Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a second copy that drifts. Split the comprehension rule while there, which packs the -- 2.52.0 From c77a2e2c9307ec5ada9d4fe8314f172bb430b674 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:11:12 +0200 Subject: [PATCH 072/966] Gate the allocations of paths read from one held draw --- perf_test.go | 4 ++++ table_test.go | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/perf_test.go b/perf_test.go index 5864b55..8392447 100644 --- a/perf_test.go +++ b/perf_test.go @@ -38,6 +38,10 @@ func TestNoRenderAllocRegression(t *testing.T) { {"nested depth 25", nestedJSON(25), 27}, {"nested depth 100", nestedJSON(100), 102}, {"wide 500 tokens", wideTokenJSON(500), 9}, + {"two paths from one held draw", `{"format":"{p.a} {p.b}","p":[{"format":"x","a":"1","b":"2"},{"format":"y","a":"A","b":"B"}]}`, 2}, + {"ten paths from one held draw", `{"format":"{r.a}{r.b}{r.c}{r.d}{r.e}{r.f}{r.g}{r.h}{r.i}{r.j}","r":[ + {"format":"x","a":"1","b":"2","c":"3","d":"4","e":"5","f":"6","g":"7","h":"8","i":"9","j":"0"}, + {"format":"y","a":"A","b":"B","c":"C","d":"D","e":"E","f":"F","g":"G","h":"H","i":"I","j":"J"}]}`, 6}, } for _, s := range shapes { f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"x.json": {Data: []byte(s.json)}})) diff --git a/table_test.go b/table_test.go index f3d9cd8..e5460ab 100644 --- a/table_test.go +++ b/table_test.go @@ -990,8 +990,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for _, cand := range tables { for i := 0; i < 20; i++ { drawing := s.clone() - if r := drawing.rowOf(f.rand, cand); s.clash(cand, r) != nil { - t.Errorf("rowOf draws %s beside %s, which pinning it there refuses", cand.selectorSpelling(r), entered.selectorSpelling(row)) + if r := (&pathDraws{s: f.rand}).row(&drawing, cand); s.clash(cand, r) != nil { + t.Errorf("a render draws %s beside %s, which pinning it there refuses", cand.selectorSpelling(r), entered.selectorSpelling(row)) } } for cr := 0; cr < cand.rows(); cr++ { -- 2.52.0 From 9cbd8841feb73a4c34f201b9fe3b07e7be022dee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:11:12 +0200 Subject: [PATCH 073/966] Draw a walk's rows and variants through pathDraws, off the pin set and the walk's signature --- doc.go | 2 +- family.go | 2 +- graph.go | 2 +- hold.go | 21 +-------- holdfence.go | 2 +- path.go | 118 +++++++++++++++++++++++++++++++++++++-------------- path_test.go | 6 +-- pins.go | 25 ----------- record.go | 2 +- render.go | 4 +- template.go | 18 ++++---- todo.md | 8 ++-- 12 files changed, 110 insertions(+), 100 deletions(-) diff --git a/doc.go b/doc.go index ad963fd..cbd54d0 100644 --- a/doc.go +++ b/doc.go @@ -29,7 +29,7 @@ package fejkdata // `template.drawGroup` as the data spells it, `template.drawGroupKey` as a // render reads it, `holdSet`, `renderScope`. // - pin — fixing which row of a table the render uses, which the draw fences replay: -// `pinSet`, `pinSet.pin`, `pinSet.rowOf`. +// `pinSet`, `pinSet.pin`, `pathDraws.row`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. // - whole — a read that lands on a `row`, so the row renders through its table's diff --git a/family.go b/family.go index c22474d..fad404e 100644 --- a/family.go +++ b/family.go @@ -33,7 +33,7 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { return nil } tr := &tableRead{head: t, drawn: map[*table]bool{}} - _, _ = walkPath(nil, t, a.tail, pathWalk{pins: &tr.pins}) + _, _ = walkPath(t, a.tail, pathWalk{pins: &tr.pins}) written := a.name[:len(a.name)-len(joinSegments(a.tail))] cur := t tr.draws(cur) diff --git a/graph.go b/graph.go index 6b1794d..d0fcf79 100644 --- a/graph.go +++ b/graph.go @@ -178,7 +178,7 @@ func renderEdges(n node) []renderEdge { // already proved the tail resolves in every variant. func pathLeaves(n node, tail []string) []node { var out []node - _, _ = walkPath(nil, n, tail, pathWalk{ + _, _ = walkPath(n, tail, pathWalk{ choice: func(c *choice, _ []string) ([]node, error) { return c.items, nil }, leaf: func(n node) error { out = append(out, n); return nil }, }) diff --git a/hold.go b/hold.go index b12dc21..9ad3629 100644 --- a/hold.go +++ b/hold.go @@ -38,26 +38,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if r, done := d.value[a.path]; done { return r } - leaf, err := walkPath(s, t.fields[a.key], a.tail, pathWalk{ - // Hold the draw at every level passed through, so two paths sharing a - // prefix share it. - choice: func(c *choice, rest []string) ([]node, error) { - key := a.key - if consumed := len(a.tail) - len(rest); consumed > 0 { - key = a.steps[consumed-1] - } - n, drew := d.variant[key] - if !drew { - n = drawn(s, c) - if d.variant == nil { - d.variant = map[string]node{} - } - d.variant[key] = n - } - return []node{n}, nil - }, - pins: &d.pins, - }) + leaf, err := walkPath(t.fields[a.key], a.tail, pathWalk{pins: &d.pins, draws: &pathDraws{s: s, held: d, a: &a}}) if err != nil { panic(fmt.Sprintf("fejkdata: %q: %v; a fence should have refused this at New", a.name, err)) } diff --git a/holdfence.go b/holdfence.go index 63380d3..562c915 100644 --- a/holdfence.go +++ b/holdfence.go @@ -106,7 +106,7 @@ func operandReader(t *template, head string) string { // coverPath collects what holding one path pins: every choice level the path // passes through, whole, and the leaf it renders. func coverPath(n node, tail []string, into map[node]bool) { - _, _ = walkPath(nil, n, tail, pathWalk{ + _, _ = walkPath(n, tail, pathWalk{ choice: func(c *choice, _ []string) ([]node, error) { cover(c, into, false); return nil, nil }, leaf: func(n node) error { cover(n, into, false); return nil }, }) diff --git a/path.go b/path.go index 22226c7..3fe216e 100644 --- a/path.go +++ b/path.go @@ -127,29 +127,40 @@ func names(segs []string) []string { // pathWalk is what one walk of a dotted path does at each kind of level: choice // returns the variants to continue into (none stops the walk), and where it is nil -// the walk draws one with s, takes the first where s is nil, or stops where pins is -// nil too; level runs at each template a segment descends +// the walk draws one with draws, takes the first where draws is nil, or stops where +// pins is nil too; level runs at each template a segment descends // into; leaf runs where the tail ends; pins is where the walk pins the table rows -// it selects and draws, nil skipping tables. A nil action is skipped. A render's -// walk sets only pins, so it allocates nothing. +// it selects and draws, nil skipping tables; draws is what draws the rows and +// variants the path reads, nil proving instead that the path resolves whichever +// are drawn. A nil action is skipped. A render's walk sets only pins and draws, so +// it allocates nothing. type pathWalk struct { choice func(c *choice, rest []string) ([]node, error) level func(t *template, rest []string) error leaf func(n node) error pins *pinSet + draws *pathDraws +} + +// pathDraws is what a walk draws with: the session, and, for a held read of a, the +// hold keeping the variant drawn at each level, so paths sharing a prefix share it. +// a is a pointer: held inline, its level keys take an eager hold set to the heap. +type pathDraws struct { + s *session + held *hold + a *arm } // walkPath descends tail from n and returns the node it ends at: a folder or // template by its next segment, a choice by w.choice, which consumes no segment, a // table by walkTable. A missing segment is an error, so no walk reaches past what -// the data holds. s draws rows and variants; where it is nil, only selectors pin. -// s is a parameter: a pathWalk field beside pins moves the hold maps to the heap. -func walkPath(s *session, n node, tail []string, w pathWalk) (node, error) { +// the data holds. +func walkPath(n node, tail []string, w pathWalk) (node, error) { if len(tail) == 0 { return n, w.atLeaf(n) } if t, isTable := n.(*table); isTable { - return walkTable(s, t, tail, w, false) + return walkTable(t, tail, w, false) } if isSelector(tail[0]) { return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(tail[0])) @@ -160,7 +171,7 @@ func walkPath(s *session, n node, tail []string, w pathWalk) (node, error) { if !ok { return nil, fmt.Errorf("no entry %q", tail[0]) } - return walkPath(s, child, tail[1:], w) + return walkPath(child, tail[1:], w) case *template: if w.level != nil { if err := w.level(n, tail); err != nil { @@ -171,9 +182,9 @@ func walkPath(s *session, n node, tail []string, w pathWalk) (node, error) { if !ok { return nil, fmt.Errorf("no field %q", tail[0]) } - return walkPath(s, child, tail[1:], w) + return walkPath(child, tail[1:], w) case *choice: - return walkChoice(s, n, tail, w) + return walkChoice(n, tail, w) case *column: return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", tail[0], n.t.columns[n.i]) } @@ -187,20 +198,20 @@ func (w pathWalk) atLeaf(n node) error { return nil } -// walkChoice continues a walk into the variants w.choice returns, or into one drawn -// by s where the walk names no choice action. -func walkChoice(s *session, c *choice, tail []string, w pathWalk) (node, error) { +// walkChoice continues a walk into the variants w.choice returns, or into one +// w.draws draws where the walk names no choice action. +func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { if w.choice == nil { if w.pins == nil { return nil, nil } - if err := carriedByAll(c, tail); err != nil { - return nil, err + if w.draws == nil { // a probe: every variant carries the tail, so any one proves it + if err := carriedByAll(c, tail); err != nil { + return nil, err + } + return walkPath(c.items[0], tail, w) } - if s == nil { // a probe: every variant carries the tail, so any one proves it - return walkPath(s, c.items[0], tail, w) - } - return walkPath(s, pick(s, c), tail, w) + return walkPath(w.draws.variant(c, tail), tail, w) } next, err := w.choice(c, tail) if err != nil { @@ -208,7 +219,7 @@ func walkChoice(s *session, c *choice, tail []string, w pathWalk) (node, error) } var last node for _, item := range next { - if last, err = walkPath(s, item, tail, w); err != nil { + if last, err = walkPath(item, tail, w); err != nil { return nil, err } } @@ -219,7 +230,7 @@ func walkChoice(s *session, c *choice, tail []string, w pathWalk) (node, error) // linked table by name. The walk reads a row wherever a segment follows or the // table was reached from another; a table reached whole, with no selector, is left // to a render's own draw. -func walkTable(s *session, t *table, tail []string, w pathWalk, descended bool) (node, error) { +func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error) { sel, tail, err := t.selector(tail) if err != nil { return nil, err @@ -231,19 +242,19 @@ func walkTable(s *session, t *table, tail []string, w pathWalk, descended bool) // A selector further down pins this table by ancestry, so the walk draws only // where none follows; drawing first could pick a row the selector is not inside. if w.pins != nil { - if err := readRow(s, w.pins, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { + if err := readRow(w.pins, w.draws, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { return nil, err } } switch { case len(tail) == 0 && sel == "" && !descended: - return walkPath(s, t, nil, w) + return walkPath(t, nil, w) case len(tail) == 0: - return walkPath(s, t.whole, nil, w) + return walkPath(t.whole, nil, w) case child != nil: - return walkTable(s, child, tail[1:], w, true) + return walkTable(child, tail[1:], w, true) } - return walkPath(s, column, tail[1:], w) + return walkPath(column, tail[1:], w) } // selector splits the selector a tail starts with from the rest of it. @@ -274,16 +285,59 @@ func (t *table) step(tail []string) (column node, child *table, err error) { // readRow pins the row a path reads of t: the one its selector names, or, where the // walk draws, one drawn where the path reads into the table. -func readRow(s *session, pins *pinSet, t *table, sel string, draw bool) error { +func readRow(p *pinSet, draws *pathDraws, t *table, sel string, draw bool) error { if sel != "" { - return pins.selectRow(t, sel) + return p.selectRow(t, sel) } - if draw && s != nil { - pins.rowOf(s, t) + if draw && draws != nil { + draws.row(p, t) } return nil } +// row is the render's row of t: the one pinned in p, else one drawn inside the +// nearest pinned ancestor — its parent drawn inside that first where the ancestor +// is further up — or over the whole table, and pinned with its ancestors. +func (d *pathDraws) row(p *pinSet, t *table) int { + if r, ok := p.pinned(t); ok { + return r + } + r := -1 + for a := t.parentT; a != nil && r < 0; a = a.parentT { + if _, ok := p.pinned(a); !ok { + continue + } + if t.parentT != a { + d.row(p, t.parentT) + } + pr, _ := p.pinned(t.parentT) + r = t.drawUnder(d.s, pr) + } + if r < 0 { + r = t.draw(d.s) + } + p.pin(t, r) + return r +} + +// variant is the variant of c the walk continues into: the one held for this level +// of the read, drawn once, where the walk holds its draws; else one drawn afresh. +func (d *pathDraws) variant(c *choice, rest []string) node { + if d.held == nil { + return pick(d.s, c) + } + key := d.a.levels[len(d.a.tail)-len(rest)] + n, drew := d.held.variant[key] + if !drew { + n = drawn(d.s, c) + if d.held.variant == nil { + d.held.variant = map[string]node{} + } + d.held.variant[key] = n + } + return n +} + // carriedByAll is the choice rule a path that must resolve on every call obeys: // the rest of the tail must be one every variant carries. func carriedByAll(c *choice, rest []string) error { @@ -315,7 +369,7 @@ func unreachableInChoice(c *choice, want string) error { // that validates here resolves on every render, and a typo is a New-time error. func checkPath(n node, tail []string, level string) error { var pins pinSet - _, err := walkPath(nil, n, tail, pathWalk{ + _, err := walkPath(n, tail, pathWalk{ choice: func(c *choice, rest []string) ([]node, error) { if err := carriedByAll(c, rest); err != nil { return nil, err diff --git a/path_test.go b/path_test.go index f4260d0..4a8ce24 100644 --- a/path_test.go +++ b/path_test.go @@ -13,14 +13,14 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) { level: func(tm *template, rest []string) error { seen = append(seen, "level:"+rest[0]); return nil }, leaf: func(n node) error { seen = append(seen, "leaf"); return nil }, } - if _, err := walkPath(nil, n, []string{"a", "b"}, walk); err != nil { + if _, err := walkPath(n, []string{"a", "b"}, walk); err != nil { t.Fatalf("walkPath(a.b) = %v", err) } if want := []string{"level:a", "level:b", "leaf"}; !slices.Equal(seen, want) { t.Errorf("walk visited %v, want %v", seen, want) } seen = nil - _, err := walkPath(nil, n, []string{"a", "nope", "deeper"}, walk) + _, err := walkPath(n, []string{"a", "nope", "deeper"}, walk) if err == nil || !strings.Contains(err.Error(), `no field "nope"`) { t.Errorf("walkPath(a.nope.deeper) = %v, want the missing segment named", err) } @@ -32,7 +32,7 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) { func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) var leaves []node - _, err := walkPath(nil, n, []string{"f"}, pathWalk{ + _, err := walkPath(n, []string{"f"}, pathWalk{ choice: func(c *choice, rest []string) ([]node, error) { if len(rest) != 1 || rest[0] != "f" { t.Errorf("choice saw rest %v, want [f]", rest) diff --git a/pins.go b/pins.go index ca6b57c..3badda9 100644 --- a/pins.go +++ b/pins.go @@ -81,31 +81,6 @@ func (p *pinSet) each(fn func(t *table, r int)) { } } -// rowOf is the row of t: the one pinned, else one s draws inside the nearest pinned ancestor — its -// parent drawn inside that first where the ancestor is further up — or over the whole table, and -// pinned with its ancestors. -func (p *pinSet) rowOf(s *session, t *table) int { - if r, ok := p.pinned(t); ok { - return r - } - r := -1 - for a := t.parentT; a != nil && r < 0; a = a.parentT { - if _, ok := p.pinned(a); !ok { - continue - } - if t.parentT != a { - p.rowOf(s, t.parentT) - } - pr, _ := p.pinned(t.parentT) - r = t.drawUnder(s, pr) - } - if r < 0 { - r = t.draw(s) - } - p.pin(t, r) - return r -} - // clash is the table whose pinned row keeps row r of t out: t itself pinned to another row, or the // nearest ancestor pinned to a row r is not inside; nil where none does. func (p *pinSet) clash(t *table, r int) *table { diff --git a/record.go b/record.go index 9ef6a80..b7e1067 100644 --- a/record.go +++ b/record.go @@ -159,7 +159,7 @@ func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, err } switch n := n.(type) { case *table: - sc.hold().pins.rowOf(s, n) + (&pathDraws{s: s}).row(&sc.hold().pins, n) return n, nil case *row: return n.t, nil diff --git a/render.go b/render.go index 3ff95ba..b0787ff 100644 --- a/render.go +++ b/render.go @@ -45,10 +45,10 @@ func (f *Generator) Fake(path string) (string, error) { func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // Walked once without drawing first, so a path that fails moves no seeded stream. var probe pinSet - if _, err := walkPath(nil, root, segments, pathWalk{pins: &probe}); err != nil { + if _, err := walkPath(root, segments, pathWalk{pins: &probe}); err != nil { return nil, err } - return walkPath(s, root, segments, pathWalk{pins: &sc.hold().pins}) + return walkPath(root, segments, pathWalk{pins: &sc.hold().pins, draws: &pathDraws{s: s}}) } // render evaluates a compiled node to a string. compile validates every node up diff --git a/template.go b/template.go index c92edd5..7ec28be 100644 --- a/template.go +++ b/template.go @@ -273,11 +273,11 @@ func fieldTokens(format string) []string { // tail is what makes the arm a bound draw: its head is drawn once per expansion // (see compileOps). type arm struct { - name string // as written, for messages - key string - tail []string - steps []string // key per level passed through; the head and leaf hold their own - path string // key and tail, the one spelling every way of writing this read shares + name string // as written, for messages + key string + tail []string + levels []string // the key at each level the tail passes through, the head's first + path string // key and tail, the one spelling every way of writing this read shares } // splitArm splits one name into key and tail. refs maps a reference to what @@ -302,11 +302,11 @@ func splitArm(name string, refs map[string]refBinding) arm { } func pathArm(name, key string, segs []string) arm { - var steps []string - for i := 0; i < len(segs)-1; i++ { // every level except the leaf's own - steps = append(steps, key+"."+strings.Join(segs[:i+1], ".")) + levels := []string{key} + for i := 0; i < len(segs)-1; i++ { + levels = append(levels, key+"."+strings.Join(segs[:i+1], ".")) } - return arm{name: name, key: key, tail: segs, steps: steps, path: key + "." + strings.Join(segs, ".")} + return arm{name: name, key: key, tail: segs, levels: levels, path: key + "." + strings.Join(segs, ".")} } // splitArms splits a token body's '|' alternatives. diff --git a/todo.md b/todo.md index 4383a05..2a87b86 100644 --- a/todo.md +++ b/todo.md @@ -39,12 +39,12 @@ This round, in order: `operandReader`, `checkNoRepeatedRead` and `compileOps`, and function-versus-field is decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which - callbacks are nil and whether `walkPath`'s session is, and `walkChoice` returns - `nil, nil` where none is set. + callbacks are nil and whether `draws` is, and `walkChoice` returns `nil, nil` where + none is set. - Have the check mode report the tables a read draws, so `tableRead.drawn` and `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and - `rowOf`'s ancestor rule: a change to how a render draws rows still leaves the fence - proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so + `pathDraws.row`'s ancestor rule: a change to how a render draws rows still leaves the + fence proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Settle a cell of a municipality drawn whole beside a path pinning a region it is not in. Needs the item above. -- 2.52.0 From 14dda205f0e176d84e35babe654d3754c109b9a1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:14:15 +0200 Subject: [PATCH 074/966] Draw the test's rows through table.drawIn --- table_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/table_test.go b/table_test.go index e5460ab..632f960 100644 --- a/table_test.go +++ b/table_test.go @@ -990,7 +990,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for _, cand := range tables { for i := 0; i < 20; i++ { drawing := s.clone() - if r := (&pathDraws{s: f.rand}).row(&drawing, cand); s.clash(cand, r) != nil { + if r := cand.drawIn(f.rand, &drawing); s.clash(cand, r) != nil { t.Errorf("a render draws %s beside %s, which pinning it there refuses", cand.selectorSpelling(r), entered.selectorSpelling(row)) } } -- 2.52.0 From a817f21842310f2bafeca3c7e2e0f0cd91730fc1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:15:23 +0200 Subject: [PATCH 075/966] Move the render's row draw beside the table's draws, as table.drawIn --- doc.go | 2 +- path.go | 34 ++++------------------------------ record.go | 2 +- table.go | 25 +++++++++++++++++++++++++ todo.md | 2 +- 5 files changed, 32 insertions(+), 33 deletions(-) diff --git a/doc.go b/doc.go index cbd54d0..7b39821 100644 --- a/doc.go +++ b/doc.go @@ -29,7 +29,7 @@ package fejkdata // `template.drawGroup` as the data spells it, `template.drawGroupKey` as a // render reads it, `holdSet`, `renderScope`. // - pin — fixing which row of a table the render uses, which the draw fences replay: -// `pinSet`, `pinSet.pin`, `pathDraws.row`. +// `pinSet`, `pinSet.pin`, `table.drawIn`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. // - whole — a read that lands on a `row`, so the row renders through its table's diff --git a/path.go b/path.go index 3fe216e..a65abd5 100644 --- a/path.go +++ b/path.go @@ -142,13 +142,12 @@ type pathWalk struct { draws *pathDraws } -// pathDraws is what a walk draws with: the session, and, for a held read of a, the -// hold keeping the variant drawn at each level, so paths sharing a prefix share it. -// a is a pointer: held inline, its level keys take an eager hold set to the heap. +// pathDraws is what a walk draws with: the session, and, for a held read, the hold +// keeping the variant drawn at each level of a, so paths sharing a prefix share it. type pathDraws struct { s *session held *hold - a *arm + a *arm // a pointer: its levels slice held here moves the render's hold set to the heap } // walkPath descends tail from n and returns the node it ends at: a folder or @@ -290,36 +289,11 @@ func readRow(p *pinSet, draws *pathDraws, t *table, sel string, draw bool) error return p.selectRow(t, sel) } if draw && draws != nil { - draws.row(p, t) + t.drawIn(draws.s, p) } return nil } -// row is the render's row of t: the one pinned in p, else one drawn inside the -// nearest pinned ancestor — its parent drawn inside that first where the ancestor -// is further up — or over the whole table, and pinned with its ancestors. -func (d *pathDraws) row(p *pinSet, t *table) int { - if r, ok := p.pinned(t); ok { - return r - } - r := -1 - for a := t.parentT; a != nil && r < 0; a = a.parentT { - if _, ok := p.pinned(a); !ok { - continue - } - if t.parentT != a { - d.row(p, t.parentT) - } - pr, _ := p.pinned(t.parentT) - r = t.drawUnder(d.s, pr) - } - if r < 0 { - r = t.draw(d.s) - } - p.pin(t, r) - return r -} - // variant is the variant of c the walk continues into: the one held for this level // of the read, drawn once, where the walk holds its draws; else one drawn afresh. func (d *pathDraws) variant(c *choice, rest []string) node { diff --git a/record.go b/record.go index b7e1067..7e4d868 100644 --- a/record.go +++ b/record.go @@ -159,7 +159,7 @@ func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, err } switch n := n.(type) { case *table: - (&pathDraws{s: s}).row(&sc.hold().pins, n) + n.drawIn(s, &sc.hold().pins) return n, nil case *row: return n.t, nil diff --git a/table.go b/table.go index df254cb..23a9081 100644 --- a/table.go +++ b/table.go @@ -487,6 +487,31 @@ func (t *table) drawUnder(s *session, pr int) int { return rows[pickCum(s, ix.childCum[k])] } +// drawIn is the render's row of t: the one pinned in p, else one drawn inside the nearest +// pinned ancestor — its parent drawn inside that first where the ancestor is further up — or +// over the whole table, and pinned with its ancestors. +func (t *table) drawIn(s *session, p *pinSet) int { + if r, ok := p.pinned(t); ok { + return r + } + r := -1 + for a := t.parentT; a != nil && r < 0; a = a.parentT { + if _, ok := p.pinned(a); !ok { + continue + } + if t.parentT != a { + t.parentT.drawIn(s, p) + } + pr, _ := p.pinned(t.parentT) + r = t.drawUnder(s, pr) + } + if r < 0 { + r = t.draw(s) + } + p.pin(t, r) + return r +} + // descendant is the table named name among those linked to t, at any depth. func (t *table) descendant(name string) *table { if c, ok := t.children[name]; ok { diff --git a/todo.md b/todo.md index 2a87b86..274ce22 100644 --- a/todo.md +++ b/todo.md @@ -43,7 +43,7 @@ This round, in order: none is set. - Have the check mode report the tables a read draws, so `tableRead.drawn` and `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and - `pathDraws.row`'s ancestor rule: a change to how a render draws rows still leaves the + `table.drawIn`'s ancestor rule: a change to how a render draws rows still leaves the fence proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Settle a cell of a municipality drawn whole beside a path pinning a region it is not in. Needs -- 2.52.0 From dc49fecd4b2838736f9564d78b2c719c313acfce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:20:03 +0200 Subject: [PATCH 076/966] Drop pathDraws' pointer rationale, and file the proved-path rule under the modes item --- path.go | 2 +- todo.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/path.go b/path.go index a65abd5..31b132b 100644 --- a/path.go +++ b/path.go @@ -147,7 +147,7 @@ type pathWalk struct { type pathDraws struct { s *session held *hold - a *arm // a pointer: its levels slice held here moves the render's hold set to the heap + a *arm } // walkPath descends tail from n and returns the node it ends at: a folder or diff --git a/todo.md b/todo.md index 274ce22..40d5d2d 100644 --- a/todo.md +++ b/todo.md @@ -40,7 +40,8 @@ This round, in order: decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which callbacks are nil and whether `draws` is, and `walkChoice` returns `nil, nil` where - none is set. + none is set. Make the render mode reachable only through a proved path: a walk that + draws no longer checks `carriedByAll`, trusting its caller's probe or `New`'s fences. - Have the check mode report the tables a read draws, so `tableRead.drawn` and `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and `table.drawIn`'s ancestor rule: a change to how a render draws rows still leaves the -- 2.52.0 From cbe982f52b356c55a0c7aa379a880288c8c0a164 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:22:32 +0200 Subject: [PATCH 077/966] Price two held paths at what the lowest supported Go allocates --- perf_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/perf_test.go b/perf_test.go index 8392447..4491566 100644 --- a/perf_test.go +++ b/perf_test.go @@ -38,7 +38,7 @@ func TestNoRenderAllocRegression(t *testing.T) { {"nested depth 25", nestedJSON(25), 27}, {"nested depth 100", nestedJSON(100), 102}, {"wide 500 tokens", wideTokenJSON(500), 9}, - {"two paths from one held draw", `{"format":"{p.a} {p.b}","p":[{"format":"x","a":"1","b":"2"},{"format":"y","a":"A","b":"B"}]}`, 2}, + {"two paths from one held draw", `{"format":"{p.a} {p.b}","p":[{"format":"x","a":"1","b":"2"},{"format":"y","a":"A","b":"B"}]}`, 3}, {"ten paths from one held draw", `{"format":"{r.a}{r.b}{r.c}{r.d}{r.e}{r.f}{r.g}{r.h}{r.i}{r.j}","r":[ {"format":"x","a":"1","b":"2","c":"3","d":"4","e":"5","f":"6","g":"7","h":"8","i":"9","j":"0"}, {"format":"y","a":"A","b":"B","c":"C","d":"D","e":"E","f":"F","g":"G","h":"H","i":"I","j":"J"}]}`, 6}, -- 2.52.0 From 9f00ad039e7bf1a9299bf252276570848075aa7f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:41:52 +0200 Subject: [PATCH 078/966] Move the decisions to docs/decisions.md, indexed in AGENTS.md and linked from the fences they back --- AGENTS.md | 91 ++++++- README.md | 485 +---------------------------------- docs/decisions.md | 640 ++++++++++++++++++++++++++++++++++++++++++++++ drawfence.go | 3 + family.go | 4 + fejkdata.go | 2 +- graph.go | 1 + holdfence.go | 3 + layout.go | 1 + node.go | 1 + reference.go | 4 +- table.go | 1 + todo.md | 13 +- value.go | 1 + 14 files changed, 759 insertions(+), 491 deletions(-) create mode 100644 docs/decisions.md diff --git a/AGENTS.md b/AGENTS.md index 6cbd2a3..6c05dd1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,9 +14,98 @@ lowers the last score. - Hard tabs. No comment by default; delete a restatement, a rationale, history, or a file preamble. - One spelling per result: reject the other at `New`, and let the error name the spelling to use. -- A standing choice a reader would relitigate goes under Decisions in the README, not in a comment. - Prose naming a source states what that source states: read the register's own field before paraphrasing it, and name the register you read before claiming none publishes a spelling. - A README example is a `json` block that loads and renders as a category; `readme_test.go` runs every one. - Cyclomatic complexity is gated at 14: the table-shaped dispatches (`linkParent`, `compileTemplate`, `renderEdges`) sit at it and stay whole; a function that would pass it is decomposed. + +# Decisions + +In [docs/decisions.md](docs/decisions.md): + +- GitHub is canonical, and the module path names it +- The vocabulary sits below `doc.go`'s package clause, not in the package doc +- Options and fields share one namespace +- `{a|b}` stays beside nested choices +- Flags follow getopt_long +- An argument is a template by its shape, not by a flag +- An inline template skips the cycle fence +- An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing +- A padded JSON argument is rejected, not trimmed +- `FakeTemplate` and `NewTemplate` both stay +- The shipped data is embedded, not discovered +- A bare reference draws each time; a reference path is held +- Reference sigils follow the filesystem +- A change to what exists is a major; a minor only adds +- Seeded output is promised within one version +- An error is a contract by what it names, not its bytes +- Raising the lowest supported Go is a major +- The format check runs on the latest Go only +- The changelog heading is the one spelling of a release; CI cuts the tag +- A `--data-path` override rebinds every reference to the category it replaces +- The repeat cap bounds renders, not bytes +- 64-bit targets only +- A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf` +- In data, a default written out and a constant spelled as a sample are load errors +- Samples say what they emit, transforms what they do +- A record is a template seen as columns; a Go struct is the one second schema +- A struct's records follow Go's field access, and compile on first use +- A render shares one reference draw per category, per group +- A draw group name is local to its category +- The expansion hold and the render's draws are two fences +- A record makes its draw maps up front, a `Fake` on its first read +- A category never references itself, and a record's fences run at load +- A record's column set is fixed before the first draw +- Null is a `null` item, not a rate +- A typed column holds one value, not composed text +- A column of one reference alone is the column it reads +- A typed column's calc is refused unless proven +- `Column` carries text, not a Go value +- `hold` names what a draw is kept in, `draw` the draw itself +- A function may not spell a method; two types may +- The package stays flat +- The performance gate asserts allocations, not wall-clock time +- Rows live in a TSV, the shape in JSON +- A selector is bracketed, and a dot inside it is literal +- `parent` names the link column and the table alike +- After a row, a path names a column or a linked table +- A table read into is pinned; a table read whole draws afresh +- Every reference path into one family selects the same rows, per render and group +- A parent row with no child row is a load error +- The choice-of-rows fence guards a data file's root, and requires string fields +- A table is a record of string columns +- The key index is built at load, the rest on first draw +- Two categories may name one TSV +- The rows of a table are alternatives +- A table never reaches its own family, by any route +- Tables carrying token cells stay small +- A path is walked once without drawing before it is walked for real +- A country's postal codes and streets are siblings under its locality +- A locale's `address` reads its country's `geo` tree, so the shipped set loads whole +- The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more +- A locale's `address` restates its country record's format +- A postort's kommun comes from its name, its tätort or its codes, never from distance +- A highway designation is not a street, and a US postal code belongs to the place holding most of its land inside places +- `--list` stays a plain list of paths +- A layout is always quoted +- A title is a table under `sex` +- A table owns the spelling of a selector on it +- No builtin reads the clock, so a date is bounded by days, never by an age +- A name column without a key resolves inside its parent +- `misc` is what every locale shares +- A register's canonical spelling loses to the one its domain writes +- `misc.territory` is the spine, and a `misc` table naming a territory links to it +- A table whose register publishes no frequency draws evenly +- `misc.timezone` weighs a zone by the people living in it +- `misc.port` selects by number, and carries no `name` +- `misc.car` is one flat table, not a make linked to its models +- `misc.territory` names its sovereign in a column, and there is no `misc.country` table +- `misc.loglevel` is a table keyed by the code, rendering POSIX's keyword +- `misc.tld` keys carry the leading dot, where other tables key on a bare code +- `misc.tld` is a table of its own, and `misc.territory.tld` stays a column +- `misc.territory` carries a currency code, it does not link to `misc.currency` +- An extension may name two media types +- The Swedish ids draw Skatteverket's test series +- The US given names come from a mirror of the SSA file +- `List` advertises direct descents only diff --git a/README.md b/README.md index c8d3bf5..08269f1 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,8 @@ spot. Its tokens reach the data by reference from the root — `{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike available. A path never contains a brace or a quote, and a bracket only as a selector after a name, so the two spellings cannot collide; which spellings an inline template -rejects, and what each names instead, is under [Decisions](#decisions). +rejects, and what each names instead, is under +[Decisions](docs/decisions.md#an-argument-is-a-template-by-its-shape-not-by-a-flag). | Flag | | |------|--| @@ -51,7 +52,7 @@ rejects, and what each names instead, is under [Decisions](#decisions). `--name value` and `--name=value` both work, a short flag's value attaches or follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see -[Decisions](#decisions); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing +[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing dir, unknown path), `2` misuse — a bad flag, an argument that names neither a template nor a path, or an inline template that does not compile. From a checkout: `go run ./cmd/fejkdata …`. @@ -144,7 +145,8 @@ category never references itself — `{/users.first}` or a bare `{/users}` insid describes a draw other than the fields beside it — so read a sibling as a path, and put a value two fields share in its own category and reference that. A field hold, transform or operand ties fields together within one column as always (see -[Correlated fields](#correlated-fields) and [Decisions](#decisions)). +[Correlated fields](#correlated-fields) and +[Decisions](docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load)). ## Data @@ -547,12 +549,12 @@ it, and a cell or format of a table that references a table of its own family, t any template, a `repeat` or a `drawGroup` included, since a row rendered whole would draw the family apart from itself: read the family from a template beside it, or add the value as a column. A cell's reads are the render's too; which pairs are weighed -against each other is under [Decisions](#decisions). +against each other is under [Decisions](docs/decisions.md#the-rows-of-a-table-are-alternatives). ### Options and fields `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options; -**any other key is a field** (see [Decisions](#decisions)), and `rows` makes a category +**any other key is a field** (see [Decisions](docs/decisions.md#options-and-fields-share-one-namespace)), and `rows` makes a category a [table](#table), so no template carries a field of that name. An object that does nothing a string can't — only a `format` — is rejected naming the string, as is a one-item choice naming its item. @@ -855,475 +857,6 @@ the Development section below, and who ships a register the four above then draw nothing, each on its own path. A fixture that looks nothing like production tests nothing; the caller who needs a value that can touch nothing asks for it by name. -## Decisions - -- **GitHub is canonical, and the module path names it.** Goal 1 wants the usage the - tool earns and goal 2 expects extenders who did not write it; both need a stranger - to file an issue and open a pull request. Gitea has no anonymous issue, and no - cross-host pull request at all, so a contributor would need an account and a fork on - a personal instance. Valid while the project wants contribution from outside: the - workflow decided nothing, being portable already — `github.api_url` and - `secrets.GITHUB_TOKEN` resolve on either host, so only the names moved. -- **The vocabulary sits below `doc.go`'s package clause, not in the package doc.** - Goal 1 is what a developer choosing a library reads first, and the package doc is - that page: the glossary would fill two thirds of it with the render path's units, - mostly unexported, and Go doc comments have no code markup, so every name would - render its backticks literally. Below the clause it reaches goal 2's reader — who - opens the file — and nothing else. Valid while the glossary's subject is the render - path; one written to teach the exported API belongs above the clause. -- **Options and fields share one namespace.** `format`, `weight`, `repeat`, - `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a - key, or prefixing options, would tax every template to guard against a - misspelt option. -- **`{a|b}` stays beside nested choices.** `[[…], […]]` picks the same way, but - its arms are anonymous; `{female|male}` keeps `person.female` addressable. -- **Flags follow getopt_long.** `--name value` and `--name=value` both work; a - short flag's value attaches or follows (`-s42`, `-s 42`) and short flags bundle - (`-hn 3`), as every shell user expects. A single-dash long flag is rejected - naming the double-dash spelling, and `-s=42` is rejected naming both short - spellings: `=` belongs to the long form, and reading `=42` as the value would - make `-d=./x` a directory named `=./x`. -- **An argument is a template by its shape, not by a flag.** A JSON object, array - or string, or a string carrying a `{` token, is an inline template; anything else - is a path. A name may not contain a brace, a bracket or a quote, so a path can - never collide with any of those spellings, and the leading `[` or `"` is gated on - valid JSON so a stray copied bracket never swallows an argument — it names - nothing, and says so. No `--template` flag is needed. Reserving the characters - whole — though only a leading one could collide — keeps one simple name rule - instead of a leading-position special case. The JSON string is what makes the - library's own advice reachable: the error for an object holding only a format - names `"…"`, and that spelling has to work where it is printed. An argument or struct - tag of one reference alone, `{/users}`, is refused naming the path `users`: both - render the same text, and only the path names a record. A folder-relative `{.name}` - or `{..name}` is refused naming `{/name}`, since an inline template sits in no - folder. `IsTemplate` exports the - rule, so the CLI, struct tags and any other caller read one. -- **An inline template skips the cycle fence.** `New` proves the loaded tree - acyclic, an inline node is a finite tree of its own, and nothing in the tree can - reference it, so no render of it reaches itself. Every other fence runs over both, - from one `checkScope`, except that struct tags leave column agreement to their Go - types. -- **An inline template that does not compile is misuse (exit 2), including a - reference that resolves to nothing** — the whole argument is the spelling under - test, and `NewTemplate` compiles, links and validates as one step. An unknown - *path* stays a runtime error (exit 1): there the argument is well-formed and only - the data is absent. -- **A padded JSON argument is rejected, not trimmed.** Padding is the one place the - two readings disagree — a format string renders it, JSON drops it — so the - spelling that renders is named rather than silently chosen. -- **`FakeTemplate` and `NewTemplate` both stay.** They reach the same value but not - at the same cost: `NewTemplate` pays the compile and validation once and renders - many times, `FakeTemplate` is the one-shot call, and `--repeat` is exactly the - case that needs the first. The pair is `regexp.MustCompile` and `regexp.Match`, - not two spellings of one result. -- **The shipped data is embedded, not discovered.** A directory a machine happens - to have would make `--seed 42` machine-dependent. Data still lives in `data/` - as JSON; `--data-path` layers over it. -- **A bare reference draws each time; a reference path is held.** `{/p} {/p}` - is two draws, as `{word} {word}` is, while every `{/p.first}` in one render reads - one draw, and a bare `{/p}` beside them is a load error: a bare token - is by contract an independent draw, a path pins its level, and a fresh draw of a - pinned level could show another row. A builtin's operand holds what it reads for - its expansion, references included, so `{uppercase(/p)} {/p}` is one draw — the - rule every operand follows. -- **Reference sigils follow the filesystem.** `/` is the root, `.` this file's - folder, `..` the folder above — what those spellings already mean to anyone who - has typed a path. A locale's files reach each other without naming the locale, - so a folder renames and copies without editing its references. -- **A change to what exists is a major; a minor only adds.** Data files, the CLI - and the Go API are the public API, and a consumer must be able to take a minor - without an edit — so an added column is a major, since it changes the CSV header - and the `INSERT` column list, as is a removed value, which changes what a fixture - holds, and a new option, which reserves a field name. One spelling per result - grows by tightening, so every fence invalidates some file. Each such release - names the rejected spelling and its replacement in the changelog and in the load - error, and that is the whole migration: a fence rejects one spelling with one - replacement, so the fix is local to each site. A fence that would need a - non-local rewrite ships a converter with its release instead. Before `v1.0.0` a - minor carries what a major would. -- **Seeded output is promised within one version.** Any edit to a category shifts - its stream and everything drawn after it, so a promise across versions would - freeze every shipped list; a fixture is re-pinned on a bump, as this repo's own are. -- **An error is a contract by what it names, not its bytes.** A script branches on - the exit code and reads the named path or spelling, so those hold; wording improves - in a minor. -- **Raising the lowest supported Go is a major.** A consumer building on it breaks, - which is the one test every rule above applies; Go's convention of a minor is not - followed. -- **The format check runs on the latest Go only.** `gofmt`'s output is the - toolchain's, not the code's: 1.27 stopped padding a map literal's values out to a - lone long key, so no source satisfies both it and 1.22's. A consumer on the lowest - supported Go depends on the code compiling and its tests passing there, which is - the `portable` stage, and never runs `gofmt` over this source, so the latest - toolchain alone defines the one canonical form goal 2 asks a reader to meet. Valid - while the lowest supported Go is not the latest. -- **The changelog heading is the one spelling of a release; CI cuts the tag.** A - tag pushed by hand is served by `go get` at once, so a tag whose commit lacks its - heading is burnt, not fixed. The heading on a gate-passed `main` commit is the - trigger instead: the tag can land only there, and the GitHub release the same job - publishes keeps one text as its body and is where prebuilt binaries will attach. -- **A `--data-path` override rebinds every reference to the category it - replaces.** References bind against the merged tree, so once shipped data uses - `{.person}`, a consumer's `sv_SE/person.json` is what every shipped reference - into `person` reads, and `New` fails on shipped data the consumer never wrote - when that file lacks a field those references read. Accepted: overriding is the - point of layering, the error names the reference and the field, and the fix is - the consumer's file carrying the fields the shipped tree reads. -- **The repeat cap bounds renders, not bytes.** A repeat, alone or nested, may - ask for at most 1 048 576 renders; how large each render is stays what the data - asked for, so `{hex(1048576)}` repeated to the cap is a terabyte, loaded without - complaint. A byte estimate would need every builtin to declare a width to fence - a shape no data comes near, and the harm lands on the author who wrote it. -- **64-bit targets only.** The gate builds amd64, and the buffer sizing a render - pre-computes (renders × bytes) assumes a 64-bit int; on a 32-bit target it could - overflow and panic. -- **A constant zero divisor is a load error; in a string column a divisor that is not - constant prints `Inf`.** `1/0` and a fixed `"0"` field are decidable, so they join - the never-numeric operand as a load error; the fold stops where an operand varies, - so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every - draw — catching it needs zero-absorbing algebra for a shape nobody writes. A - [typed column](#datatype) bounds its operands instead and refuses a divisor it - cannot keep from zero. -- **In data, a default written out and a constant spelled as a sample are load - errors.** `weight: 1`, `repeat: 1`, `separator: ""`, `datatype: "string"`, - `int(5,5)`, `float(1,1,2)`, - `+5` and `05` each spell what a shorter form already spells, so each is rejected - naming that form. The CLI's numbers follow the shell instead: `--seed 007` and - `--repeat +3` are 7 and 3, as every command line reads them. -- **Samples say what they emit, transforms what they do.** `{upper(2)}` is two - letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on - whether the argument looks like a number. -- **A record is a template seen as columns; a Go struct is the one second schema.** A - template's `format` composes its fields into one string; `FakeRecord` and - `--format` project the same fields as columns. Two views of one dataset, so a - record author writes the same JSON they already know, and a column is the same - field `Fake` renders by dotted path. The `format` is inert to a record — a - record-only template writes `"format": ""` — but it is compiled and fenced, so - a template that loads renders as whichever shape is asked for. `FakeStruct` takes - its columns from a struct instead, because a Go caller has already written that - schema: the fields name the columns and their types are the datatypes, so a tag - says only what to draw, and a `datatype` in it would be a second spelling of the - type. The Go type is a struct column's one datatype, so its items need not agree on - one among themselves; each must only hold that type. -- **A struct's records follow Go's field access, and compile on first use.** The - fields an embedded struct promotes are the struct's own — `e.First`, as - `encoding/json` and SQL mappers read them — so they are columns of its record and - share its draws; a tagged field that another field hides is refused, not dropped. A - named struct field is another entity and a record of its own. `fake:"-"` leaves a - struct field, embedded or named, unfilled, so no name may be `-`; a pointer back to - a struct already being filled is left alone, since filling it would never end. `New` cannot - see a caller's types, so the first `FakeStruct` for a type compiles its tags and the - answer, error included, is kept per type: a test's first call is its load, and no - `NewStruct` handle is needed, as the cache already compiles once. -- **A render shares one reference draw per category, per group.** Every reference - path into a category in one `Fake`, or one record, reads one draw of it, so a - value's facts agree across its fields, nested templates and columns alike — - `{/currency.code}` in one field and `{/currency.symbol}` in another name one - currency, whichever view renders them. A `repeat` iteration is a render of its - own, since repeating asks for another entity, and a [draw group](#draw-group) names further - entities within one render, so a payer and a payee are two groups over one - `person` rather than two copies of it. Only references share: a sibling field is - local to its own expansion, so a `first` column does not silently bind to a - `first` in the column next to it. -- **A draw group name is local to its category.** A category's groups are its own - entities, so a caller naming a group the same way never joins them by accident, - and renaming a group inside one file changes no render elsewhere. The unnamed - group still spans categories, since facts that belong together across categories - must agree. -- **The expansion hold and the render's draws are two fences.** One proves a sibling - path or an operand is reached only by its readers within an expansion, the other - does the same for reference paths across a render and its draw groups. They pin - different things — an operand pins the value its own render produced and stops at a - reference, a path pins every level it passes through — so one walk would carry both - rules and both scopes anyway, and tell them apart at every step. -- **A record makes its draw maps up front, a `Fake` on its first read.** A record's - columns always read through the render's draws, so making the maps where the set is - declared keeps them on that frame's stack. A `Fake` often reads no reference path at - all, and making them anyway cost about a fifth of the cheapest render, so it makes - them on the first read instead, at two heap allocations for a render that does share - a draw. The allocation gate over a repeat of a reference path and over a named draw - group prices that, and pins the two measures that keep a hold set off the heap. -- **A category never references itself, and a record's fences run at load.** A category - is one unit: a reference back into it — `{/users.first}` inside `users` — describes a - draw other than the fields beside it, so `New` refuses it and the sibling path stays - the one spelling for a field of one's own. A value two fields share goes in its own - category, which both reference. That settled, a record's column fences run at `New` - too, so a category that loads renders as whichever shape is asked for, and a reference - reaching back into a category through another one is refused there as the overlap it - is. Which reads those fences weigh differs on purpose: a record-only template's inert - format renders nothing, so a `drawGroup` on it can never matter and is refused, while - one on a rendering format can matter to a caller that bare-references it. -- **A record's column set is fixed before the first draw.** Only a category-level - template is a record: a path descending into a field, or naming a folder or a - choice, errors. A tail may pass through a choice whose variants carry different - fields, so the columns — and with them the CSV header written once ahead of every - row — would vary per draw. A fixed column set is what the CSV and `INSERT` - contracts rest on, so the restriction holds even where a particular choice would - happen to agree. -- **Null is a `null` item, not a rate.** A null is one more outcome of a column's - draw, so a choice's weights skew it like any other; a null-rate option would be a - second way to state odds. -- **A typed column holds one value, not composed text.** Its bounds come from a - literal or a call's arguments, so a load error names a real value, a range check is - one comparison, and `1{digits(2)}` is a second spelling of `{int(100,199)}`. -- **A column of one reference alone is the column it reads.** `{/src.score}` renders - exactly what `src.score` draws, so it takes that column's datatype and null rather - than restating them, and a `datatype` restating the one it takes is a second - spelling. Any other `datatype` still types the values — the one way to type a column - someone else wrote. -- **A typed column's calc is refused unless proven.** Operand bounds must keep each - divisor from zero and the result finite; what they cannot show is refused rather - than trusted, since a bare `NaN` breaks the JSON and SQL it lands in. -- **`Column` carries text, not a Go value.** `Value` is the rendered string beside - `DataType` and `Null`, which each serializer writes as the load check proved it; a - `Value any` would hand every caller a type switch. -- **`hold` names what a draw is kept in, `draw` the draw itself.** Goal 2 wants a name - to reach one unit, so a unit takes the stem of what it is, and a file the stem of the - units it holds. -- **A function may not spell a method; two types may.** A call writes a method with its - receiver and a function bare, so one name on both greps as one unit, which is the - reaching cost goal 2 counts, while the receiver before a shared method name says - which type answers. `TestNoFunctionSpellsAMethod` holds it. A type is outside the - rule: `renderScope.hold` answers with a `*hold`, so the two spell one unit, which no - test can judge. -- **The package stays flat.** Go ties a package to one directory, so folders would - split the API into packages. -- **The performance gate asserts allocations, not wall-clock time.** `AllocsPerRun` - is deterministic across machines, so a ±10% ceiling does not flake under CI load, - while time varies with the machine and its neighbours. A rendering slowdown - almost always costs an allocation too (a lost pre-size, a per-item map, an extra - copy). The benchmark suite (see Development) reports time for a human, not as a - pass/fail gate. -- **Rows live in a TSV, the shape in JSON.** `New` allocates once per node, so a - register of thirty thousand rows written as JSON objects would cost it a second; - a TSV is one allocation whose cells are substrings, and the JSON says only how a - row is composed. The TSV sits beside its category file, named by `rows`, so a - data directory stays a directory of categories, and one nothing names is a - load error rather than a file silently ignored. -- **A selector is bracketed, and a dot inside it is literal.** `municipality[0180]` - reads as selection to anyone who has indexed an array, and `[St. Louis]` keeps a - name whole where a colon or a dot-separated spelling could not; zsh needs the - brackets quoted, which the README's examples show. A name that spells another - row's key is refused at load rather than shadowed: a key names one row by - contract, so the name could never select its own, and the check is one lookup per - row against the key index that already exists. -- **`parent` names the link column and the table alike.** One word says both, so a - child table sits in its parent's folder and links on a column of the parent's - name; the geo plan wants exactly that, and a table needing another folder or - another column name would be asking for a second spelling. -- **After a row, a path names a column or a linked table.** `locality[Lund].address`, - a template beside the family under a selected row, is refused today; admitting it - later is additive, since a refused spelling gains a meaning and no accepted one - changes, so the door stays open for the address records the plan describes. -- **A table read into is pinned; a table read whole draws afresh.** A path into a - table pins its row for the render and group, as a - reference path pins its level, and a bare `{/city}` draws a row each time and pins no - row of its family, as a bare reference does; so a bare table beside a path into its - family is refused like a bare reference beside a path into it. A bare table reference - still counts as a read for a `drawGroup`, since the group is what draws it apart from - the family's pins. -- **Every reference path into one family selects the same rows, per render and - group.** A selector pins rows for the render, and a read that draws freely before it - could pin a row the selector contradicts, so accepting both would make the result - depend on which token rendered first. Requiring one selection per family per group - is checkable at load with no data lookup beyond the selectors themselves, and the - error names the rewrite. Two selectors naming one row by key and by name compare - equal, since the load check resolves them. -- **A parent row with no child row is a load error.** A descendant is drawn inside the - nearest pinned ancestor, so every ancestor row must lead to a row at every level - below it, or a render could find nothing to draw. The import script drops or fills - such rows; the alternative, falling back to a free draw, would break the consistency - the link exists for without saying so. -- **The choice-of-rows fence guards a data file's root, and requires string fields.** - A table is a category with a TSV beside its file, so only a root choice has the - spelling the fence names; a nested choice of same-shaped templates and an inline - one keep loading. Fields must all be strings because a cell is a string node: a - choice whose items carry a nested choice is not one table but two linked ones. -- **A table is a record of string columns.** Its columns are the CSV header and the - `INSERT` column list, fixed by the TSV header, so a table is a record by - construction; every column is a string until a typed column option earns its place. -- **The key index is built at load, the rest on first draw.** A link is proved - against the parent's keys and a key's uniqueness is a data mistake, so both are - load-time; the name index and the per-parent child lists serve only a draw or a - selection, so they wait for the first one, keeping `New` linear in the bytes read. -- **Two categories may name one TSV.** Each is a view of the file with its own - format and options, at the cost of holding the rows twice, which is what a - category over a register with two natural formats asks for; a TSV nothing names - stays a load error, since that one is a file forgotten rather than shared. -- **The rows of a table are alternatives.** Only one row renders per read, so within one - read a cell in one row and a cell in another never meet, and each may select its own - row of another table; the cells of one row, and whatever they reach, do meet, and so - does the format beside them. The rule the fence applies is that two reads meet unless - they sit in rows that never render together: rows falling under different rows of a - table a path pinned, two rows of that table itself included. A choice's items get no - such treatment yet: two items selecting different rows are still refused. -- **A table never reaches its own family, by any route.** A `repeat` iteration and a - `drawGroup` each draw apart on purpose, but a row that lists three localities from - other regions is the output the family exists to prevent, so the own-family fence - walks through both rather than stopping where the draw fences do. -- **Tables carrying token cells stay small.** The family fence compares the reads of - every pair of rows that can render together, so a table whose every row's cell - selects a row of another table loads in time quadratic in its rows: about a - second at four thousand rows. No shipped table carries such cells, and a register - is a column set rather than a set of references, so the fence is left as it is - until a real data set needs the indexed form. -- **A path is walked once without drawing before it is walked for real.** A path - that fails below its first level then moves no seeded stream, at the cost of one - draw-free walk per call, which allocates nothing. -- **A country's postal codes and streets are siblings under its locality.** No open - source pairs a Swedish street with its postnummer, and pairing the US through its - ZIPs would shape the two trees differently, so both draw inside the pinned - locality and an address agrees at that level. A street's own code is the exact - pairing to add when a source carries it. -- **A locale's `address` reads its country's `geo` tree, so the shipped set loads - whole.** `data/sv_SE` alone no longer loads: a test loads `data` and prefixes - the locale, and `--no-shipped-data -d` takes the whole `data` folder or a set of - one's own. -- **The default embed holds every Swedish postort the import can place and give a - street-delivery code and a street, and the US places of 25,000 or more.** Sweden - fits whole in 700 KB; every US place of 10,000 would pass a - megabyte and fetch 1,200 counties of TIGER files, so the threshold sits where the - two countries match in size, and `--min-population` and - `--streets-per-locality` on the import scripts build a fuller set. The two trees - add about 20 ms to `New`, which loads the shipped set in about 45 ms. -- **A locale's `address` restates its country record's format.** A record cannot - read another whole and keep its columns, so `sv_SE.address` names the same four - columns as `geo.SE.address`, each a reference into it, and the format appears - twice; a column is spelled the same in both, `street-number`, so the two never - disagree on a name. -- **A postort's kommun comes from its name, its tätort or its codes, never from - distance.** GeoNames leaves a fifth of Sweden's codes without a kommun and - carries stale spellings; the nearest code across a border named the wrong kommun - half the time it was tried, so a postort none of the three rules place is - dropped, as is one not cased like a place name. -- **A highway designation is not a street, and a US postal code belongs to the place - holding most of its land inside places.** `I- 55 Bus` and `US Hwy 1` carry the - most address ranges in many places and would head every address, so the import - drops names spelled as a route. A ZCTA goes to the place its largest in-place part - lies in, census-designated places left out since they never ship, and ships only - when that place does, so a few dozen places whose every code lies mostly in a - bigger neighbour ship no address; counting the land outside every place too would - drop a quarter of the places, whose codes straddle unincorporated land, for a - postal city the USPS mostly names the same way. -- **`--list` stays a plain list of paths.** It is what a script reads, so every line - has to be a path that `Fake` takes; a marker for the tables a `[selector]` follows, - or a legend above them, would make the output something to parse before use. - `--help` names the selector spelling instead, and the Table section teaches it. -- **A layout is always quoted.** A layout may carry the comma that separates - arguments, `'January 2, 2006'`, and one spelling for every layout beats a rule - about which ones need the quotes, so the bare spelling is refused naming the - quoted one. The layout is Go's reference time because the library renders with - it and a Go caller already knows it; its names are English, and a locale's own - month and weekday names are data. -- **A title is a table under `sex`.** A prefix drawn apart would put `Mr` on a record - whose `sex` column says `female`, which is the disagreement the record exists to - prevent; the tables this set already has are what a title needs, so `en_US.title` - links to `sex` as `first-name` does. Swedish has no everyday sexed honorific, so - `sv_SE.title` is a table as well but carries no `parent`. Its weight column is - `share`, not the `count` a name table carries, because the values are a curated - proportion rather than bearers anyone counted. -- **A table owns the spelling of a selector on it.** A reference reaches a table by a - path that carries no selector — `sv_SE.person.first` reads `first-name` through - `sex` — so the walk that resolved a name cannot say where a reader would type one. - The table's own location can, which is why it keeps its path, and why an ambiguity - error names `sv_SE.sex[f].first-name[Kim]` rather than the table's own name. -- **No builtin reads the clock, so a date is bounded by days, never by an age.** - An `age(min,max)` would make a seeded fixture change with the day it runs on, - which is what a seed exists to prevent; a birthdate for someone 20 to 60 is - `date(1966-01-01,2006-12-31,…)`, re-pinned as any fixture is. -- **A name column without a key resolves inside its parent.** A given name both - sexes carry is a row under each, so `name` cannot be the key; the parent's row - tells the two apart, `sex[f].first-name[Kim]`, the ambiguity error spells each - row inside its parent, and a name repeating inside one parent row is refused at - load, since nothing could then select it. -- **`misc` is what every locale shares.** A category whose facts differ by country - belongs in that country's locale, read from the register that country's own - records use; `misc` takes only sources that are international. NHTSA vPIC and - Mobility Sweden's registrations are national, so they are for `en_US.car` and - `sv_SE.car`, and `misc.car` waits for an international source. -- **A register's canonical spelling loses to the one its domain writes.** Where a - source offers several spellings of one fact, the shipped one is what records in - that domain carry. `misc.timezone` reads `zone.tab` and not the `zone1970.tab` - that supersedes it, because the latter keeps one zone per set of countries that - have agreed since 1970: it spells Sweden `Europe/Berlin`, and no Swedish system - writes that. For the same reason `misc.language` takes the ISO 639-2 register's - first synonym over CLDR, which says "Chinese, Mandarin" for `zh`. -- **`misc.territory` is the spine, and a `misc` table naming a territory links to - it.** Where every territory row has a child the column is a `parent`, and the - import drops the child rows whose territory the set does not ship — 17 of - `misc.timezone`'s, Antarctica's ten among them. Agreement across a record is - worth more than the last rows of a table. Where no such link can hold the fact - stays a column. Layer your own `misc.territory` over the shipped one and you - must layer `misc.timezone` too, or the link fails at load naming the row. -- **A table whose register publishes no frequency draws evenly.** `misc.httpmethod`, - `misc.port`, `misc.httpstatus`, `misc.mimetype` and `misc.tld` weigh every row alike, - so GET is a ninth of the methods drawn. Goal 10 keeps an authored fact out of a - sourced table, and no register publishes how often a method, a port or a TLD is used, - so a weight here would be invented. Where one exists it is read, as `misc.timezone` - reads GeoNames populations and `sv_SE.first-name` SCB bearers. -- **`misc.port` selects by number, and carries no `name`.** 29 of its services sit on - more than one port, `http-alt` on three, so `misc.port[http-alt]` could name no one - row. The number is what a port field holds anyway, and `misc.port[443].service` - reads the other way. -- **`misc.car` is one flat table, not a make linked to its models.** A row is a make - and a model drawn together, so no render pairs a Volvo with a RAV4. Two linked - tables would reach the same pairs and add a selector nothing asks for; a make alone - is `misc.car.make`. -- **`misc.territory` names its sovereign in a column, and there is no `misc.country` - table.** ISO 3166-1 codes territories, so `territory` is the honest name, and - `is_independent` in the register gives each one's state. A second table of the 195 - sovereigns would hold a `DK` row beside the territory `DK` row, both carrying - Denmark's capital, currency, flag and TLD — two owners for one fact, drifting at - the next import. Splitting the columns to avoid that is worse: put `capital` on - the territory alone and a country row can no longer name Copenhagen. The column - cannot be a `parent`, since a table never reaches its own family; a test proves - every value names a row instead. A territory the register records no state for - stands alone, which is `EH` alone, and naming one for it would be a claim - fejkdata has no business making. -- **`misc.loglevel` is a table keyed by the code, rendering POSIX's keyword.** A flat - list of names carries neither the code a PRI encodes nor a selector reaching it, and - goal 3 draws the two as one fact. The canonical spelling losing to the one its domain - writes, above, settles the rest: a configuration writes `info`, so RFC 5424's - `Informational` stays the `severity` column, and the code and the keyword hold the - two selector slots. -- **`misc.tld` keys carry the leading dot, where other tables key on a bare code.** - The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a - bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a - host with no separator. `misc.tld[se]` misses for it, which `todo.md` carries. -- **`misc.tld` is a table of its own, and `misc.territory.tld` stays a column.** A - `parent` demands a child for every parent row, so linking them would drop every root - zone row naming no territory, which is most of them, and goal 12 holds a sourced - table whole. The loader refuses the link outright anyway: `tld` is a column of - `misc.territory`, and a table may not be named like a column of its ancestor. -- **`misc.territory` carries a currency code, it does not link to `misc.currency`.** - A `parent` demands a child for every parent row, and ISO 4217 registers codes no - country's row can name: the funds codes (Mvdol, WIR Euro, US Dollar (Next day)), - and VED beside VES, both Venezuela's, of which a country row names one. Linking - would trade the register for the link, and `misc.territory.currency` already pairs a - country with its currency in one draw. -- **An extension may name two media types.** `.xml`, `.rtf`, `.sub`, `.mpp` and `.ac` - each name two rows of `misc.mimetype`. Separating them would mean dropping a - registered type, or naming one by an extension that is not its own — `.mpt` is - Project's template, not its document. Selecting such a name is an error listing - both keys; select by type instead. -- **The Swedish ids draw Skatteverket's test series.** A Luhn-valid personnummer - over a random birth number may be a living person's; 238 and 239 after any date - are blocked from assignment, so the shipped `personnummer` and - `samordningsnummer` use those. They sit in a `birth-number` table under `sex` - rather than as a column of it: the render's shared draw of the family is what - makes the number and the name agree on sex, and `sex` stays one shape across - locales instead of collecting every sex-keyed id fact. A samordningsnummer's - day, the birthday plus 60, is drawn from 61 to 88, valid in every month, rather - than computed from the date drawn. -- **The US given names come from a mirror of the SSA file.** ssa.gov refuses a - client outside the US, so `names-us.py` reads a GitHub copy that ends at 2020, - which a count over the births since 1930 barely feels; `--names` takes the - official zip. The SSA's placeholder rows are top-1000 entries that name nobody, so - the import drops them by name rather than by a rank a regeneration would move. -- **`List` advertises direct descents only.** `region.municipality.locality` is - listed, and `region.locality` resolves too but is not: the set of every descent - through a chain of five tables is every subsequence of it, and the direct chain is - the one a reader can predict from the tables' parents. ## Layout @@ -1357,7 +890,7 @@ data.go the load path: fs.FS folders/files -> merged namespace tree, and cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) -docs/ the register research behind the shipped data, and the survey of other libraries +docs/ the decision log, the register research behind the shipped data, and the survey of other libraries release-tooling/ the release CI publishes from the changelog heading testdata/ the pinned shipped shape (see Versioning) ``` @@ -1396,7 +929,7 @@ docker compose run --rm --user root --entrypoint chown test -R "$(id -u):$(id -g Every pull request runs `docker build .` against both the latest and the lowest supported Go, and must pass before it can be merged — unless it changes none of the files the build and its tests read, nor the workflow itself, in which case -it's skipped (see [Decisions](#decisions)). That build is the whole gate but the +it's skipped. That build is the whole gate but the changelog check, which CI runs against the PR base — vet, complexity, format check and tests — so run it locally before pushing. The lowest Go builds the `portable` stage, all of that bar the format check: diff --git a/docs/decisions.md b/docs/decisions.md new file mode 100644 index 0000000..5729990 --- /dev/null +++ b/docs/decisions.md @@ -0,0 +1,640 @@ +# Decisions + +## GitHub is canonical, and the module path names it + +Goal 1 wants the usage the tool earns and goal 2 expects extenders who did not write it; +both need a stranger to file an issue and open a pull request. Gitea has no anonymous +issue, and no cross-host pull request at all, so a contributor would need an account and +a fork on a personal instance. Valid while the project wants contribution from outside: +the workflow decided nothing, being portable already — `github.api_url` and +`secrets.GITHUB_TOKEN` resolve on either host, so only the names moved. + +## The vocabulary sits below `doc.go`'s package clause, not in the package doc + +Goal 1 is what a developer choosing a library reads first, and the package doc is that +page: the glossary would fill two thirds of it with the render path's units, mostly +unexported, and Go doc comments have no code markup, so every name would render its +backticks literally. Below the clause it reaches goal 2's reader — who opens the file — +and nothing else. Valid while the glossary's subject is the render path; one written to +teach the exported API belongs above the clause. + +## Options and fields share one namespace + +`format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are reserved; +every other key is a field. Nesting fields under a key, or prefixing options, would tax +every template to guard against a misspelt option. + +## `{a|b}` stays beside nested choices + +`[[…], […]]` picks the same way, but its arms are anonymous; `{female|male}` keeps +`person.female` addressable. + +## Flags follow getopt_long + +`--name value` and `--name=value` both work; a short flag's value attaches or follows +(`-s42`, `-s 42`) and short flags bundle (`-hn 3`), as every shell user expects. A +single-dash long flag is rejected naming the double-dash spelling, and `-s=42` is +rejected naming both short spellings: `=` belongs to the long form, and reading `=42` as +the value would make `-d=./x` a directory named `=./x`. + +## An argument is a template by its shape, not by a flag + +A JSON object, array or string, or a string carrying a `{` token, is an inline template; +anything else is a path. A name may not contain a brace, a bracket or a quote, so a path +can never collide with any of those spellings, and the leading `[` or `"` is gated on +valid JSON so a stray copied bracket never swallows an argument — it names nothing, and +says so. No `--template` flag is needed. Reserving the characters whole — though only a +leading one could collide — keeps one simple name rule instead of a leading-position +special case. The JSON string is what makes the library's own advice reachable: the +error for an object holding only a format names `"…"`, and that spelling has to work +where it is printed. An argument or struct tag of one reference alone, `{/users}`, is +refused naming the path `users`: both render the same text, and only the path names a +record. A folder-relative `{.name}` or `{..name}` is refused naming `{/name}`, since an +inline template sits in no folder. `IsTemplate` exports the rule, so the CLI, struct +tags and any other caller read one. + +## An inline template skips the cycle fence + +`New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and +nothing in the tree can reference it, so no render of it reaches itself. Every other +fence runs over both, from one `checkScope`, except that struct tags leave column +agreement to their Go types. + +## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing + +The whole argument is the spelling under test, and `NewTemplate` compiles, links and +validates as one step. An unknown *path* stays a runtime error (exit 1): there the +argument is well-formed and only the data is absent. + +## A padded JSON argument is rejected, not trimmed + +Padding is the one place the two readings disagree — a format string renders it, JSON +drops it — so the spelling that renders is named rather than silently chosen. + +## `FakeTemplate` and `NewTemplate` both stay + +They reach the same value but not at the same cost: `NewTemplate` pays the compile and +validation once and renders many times, `FakeTemplate` is the one-shot call, and +`--repeat` is exactly the case that needs the first. The pair is `regexp.MustCompile` +and `regexp.Match`, not two spellings of one result. + +## The shipped data is embedded, not discovered + +A directory a machine happens to have would make `--seed 42` machine-dependent. Data +still lives in `data/` as JSON; `--data-path` layers over it. + +## A bare reference draws each time; a reference path is held + +`{/p} {/p}` is two draws, as `{word} {word}` is, while every `{/p.first}` in one render +reads one draw, and a bare `{/p}` beside them is a load error: a bare token is by +contract an independent draw, a path pins its level, and a fresh draw of a pinned level +could show another row. A builtin's operand holds what it reads for its expansion, +references included, so `{uppercase(/p)} {/p}` is one draw — the rule every operand +follows. + +## Reference sigils follow the filesystem + +`/` is the root, `.` this file's folder, `..` the folder above — what those spellings +already mean to anyone who has typed a path. A locale's files reach each other without +naming the locale, so a folder renames and copies without editing its references. + +## A change to what exists is a major; a minor only adds + +Data files, the CLI and the Go API are the public API, and a consumer must be able to +take a minor without an edit — so an added column is a major, since it changes the CSV +header and the `INSERT` column list, as is a removed value, which changes what a fixture +holds, and a new option, which reserves a field name. One spelling per result grows by +tightening, so every fence invalidates some file. Each such release names the rejected +spelling and its replacement in the changelog and in the load error, and that is the +whole migration: a fence rejects one spelling with one replacement, so the fix is local +to each site. A fence that would need a non-local rewrite ships a converter with its +release instead. Before `v1.0.0` a minor carries what a major would. + +## Seeded output is promised within one version + +Any edit to a category shifts its stream and everything drawn after it, so a promise +across versions would freeze every shipped list; a fixture is re-pinned on a bump, as +this repo's own are. + +## An error is a contract by what it names, not its bytes + +A script branches on the exit code and reads the named path or spelling, so those hold; +wording improves in a minor. + +## Raising the lowest supported Go is a major + +A consumer building on it breaks, which is the one test every rule above applies; Go's +convention of a minor is not followed. + +## The format check runs on the latest Go only + +`gofmt`'s output is the toolchain's, not the code's: 1.27 stopped padding a map +literal's values out to a lone long key, so no source satisfies both it and 1.22's. A +consumer on the lowest supported Go depends on the code compiling and its tests passing +there, which is the `portable` stage, and never runs `gofmt` over this source, so the +latest toolchain alone defines the one canonical form goal 2 asks a reader to meet. +Valid while the lowest supported Go is not the latest. + +## The changelog heading is the one spelling of a release; CI cuts the tag + +A tag pushed by hand is served by `go get` at once, so a tag whose commit lacks its +heading is burnt, not fixed. The heading on a gate-passed `main` commit is the trigger +instead: the tag can land only there, and the GitHub release the same job publishes +keeps one text as its body and is where prebuilt binaries will attach. + +## A `--data-path` override rebinds every reference to the category it replaces + +References bind against the merged tree, so once shipped data uses `{.person}`, a +consumer's `sv_SE/person.json` is what every shipped reference into `person` reads, and +`New` fails on shipped data the consumer never wrote when that file lacks a field those +references read. Accepted: overriding is the point of layering, the error names the +reference and the field, and the fix is the consumer's file carrying the fields the +shipped tree reads. + +## The repeat cap bounds renders, not bytes + +A repeat, alone or nested, may ask for at most 1 048 576 renders; how large each render +is stays what the data asked for, so `{hex(1048576)}` repeated to the cap is a terabyte, +loaded without complaint. A byte estimate would need every builtin to declare a width to +fence a shape no data comes near, and the harm lands on the author who wrote it. + +## 64-bit targets only + +The gate builds amd64, and the buffer sizing a render pre-computes (renders × bytes) +assumes a 64-bit int; on a 32-bit target it could overflow and panic. + +## A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf` + +`1/0` and a fixed `"0"` field are decidable, so they join the never-numeric operand as a +load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed at `0` +and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing +algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands +instead and refuses a divisor it cannot keep from zero. + +## In data, a default written out and a constant spelled as a sample are load errors + +`weight: 1`, `repeat: 1`, `separator: ""`, `datatype: "string"`, `int(5,5)`, +`float(1,1,2)`, `+5` and `05` each spell what a shorter form already spells, so each is +rejected naming that form. The CLI's numbers follow the shell instead: `--seed 007` and +`--repeat +3` are 7 and 3, as every command line reads them. + +## Samples say what they emit, transforms what they do + +`{upper(2)}` is two letters, `{uppercase(x)}` is `x` upper-cased; one name for both +would turn on whether the argument looks like a number. + +## A record is a template seen as columns; a Go struct is the one second schema + +A template's `format` composes its fields into one string; `FakeRecord` and `--format` +project the same fields as columns. Two views of one dataset, so a record author writes +the same JSON they already know, and a column is the same field `Fake` renders by dotted +path. The `format` is inert to a record — a record-only template writes `"format": ""` — +but it is compiled and fenced, so a template that loads renders as whichever shape is +asked for. `FakeStruct` takes its columns from a struct instead, because a Go caller has +already written that schema: the fields name the columns and their types are the +datatypes, so a tag says only what to draw, and a `datatype` in it would be a second +spelling of the type. The Go type is a struct column's one datatype, so its items need +not agree on one among themselves; each must only hold that type. + +## A struct's records follow Go's field access, and compile on first use + +The fields an embedded struct promotes are the struct's own — `e.First`, as +`encoding/json` and SQL mappers read them — so they are columns of its record and share +its draws; a tagged field that another field hides is refused, not dropped. A named +struct field is another entity and a record of its own. `fake:"-"` leaves a struct +field, embedded or named, unfilled, so no name may be `-`; a pointer back to a struct +already being filled is left alone, since filling it would never end. `New` cannot see a +caller's types, so the first `FakeStruct` for a type compiles its tags and the answer, +error included, is kept per type: a test's first call is its load, and no `NewStruct` +handle is needed, as the cache already compiles once. + +## A render shares one reference draw per category, per group + +Every reference path into a category in one `Fake`, or one record, reads one draw of it, +so a value's facts agree across its fields, nested templates and columns alike — +`{/currency.code}` in one field and `{/currency.symbol}` in another name one currency, +whichever view renders them. A `repeat` iteration is a render of its own, since +repeating asks for another entity, and a [draw group](../README.md#draw-group) names further +entities within one render, so a payer and a payee are two groups over one `person` +rather than two copies of it. Only references share: a sibling field is local to its own +expansion, so a `first` column does not silently bind to a `first` in the column next to +it. + +## A draw group name is local to its category + +A category's groups are its own entities, so a caller naming a group the same way never +joins them by accident, and renaming a group inside one file changes no render +elsewhere. The unnamed group still spans categories, since facts that belong together +across categories must agree. + +## The expansion hold and the render's draws are two fences + +One proves a sibling path or an operand is reached only by its readers within an +expansion, the other does the same for reference paths across a render and its draw +groups. They pin different things — an operand pins the value its own render produced +and stops at a reference, a path pins every level it passes through — so one walk would +carry both rules and both scopes anyway, and tell them apart at every step. + +## A record makes its draw maps up front, a `Fake` on its first read + +A record's columns always read through the render's draws, so making the maps where the +set is declared keeps them on that frame's stack. A `Fake` often reads no reference path +at all, and making them anyway cost about a fifth of the cheapest render, so it makes +them on the first read instead, at two heap allocations for a render that does share a +draw. The allocation gate over a repeat of a reference path and over a named draw group +prices that, and pins the two measures that keep a hold set off the heap. + +## A category never references itself, and a record's fences run at load + +A category is one unit: a reference back into it — `{/users.first}` inside `users` — +describes a draw other than the fields beside it, where goal 3 asks facts that belong +together to come from one draw, so `New` refuses it and the sibling path stays the one +spelling for a field of one's own. A value two fields share goes in its own category, +which both reference. That settled, a record's column fences run at `New` too, so a +category that loads renders as whichever shape is asked for, and a reference reaching +back into a category through another one is refused there as the overlap it is. Which +reads those fences weigh differs on purpose: a record-only template's inert format +renders nothing, so a `drawGroup` on it can never matter and is refused, while one on a +rendering format can matter to a caller that bare-references it. + +## A record's column set is fixed before the first draw + +Only a category-level template is a record: a path descending into a field, or naming a +folder or a choice, errors. A tail may pass through a choice whose variants carry +different fields, so the columns — and with them the CSV header written once ahead of +every row — would vary per draw. A fixed column set is what the CSV and `INSERT` +contracts rest on, so the restriction holds even where a particular choice would happen +to agree. + +## Null is a `null` item, not a rate + +A null is one more outcome of a column's draw, so a choice's weights skew it like any +other; a null-rate option would be a second way to state odds. + +## A typed column holds one value, not composed text + +Its bounds come from a literal or a call's arguments, so a load error names a real +value, a range check is one comparison, and `1{digits(2)}` is a second spelling of +`{int(100,199)}`. + +## A column of one reference alone is the column it reads + +`{/src.score}` renders exactly what `src.score` draws, so it takes that column's +datatype and null rather than restating them, and a `datatype` restating the one it +takes is a second spelling. Any other `datatype` still types the values — the one way to +type a column someone else wrote. + +## A typed column's calc is refused unless proven + +Operand bounds must keep each divisor from zero and the result finite; what they cannot +show is refused rather than trusted, since a bare `NaN` breaks the JSON and SQL it lands +in. + +## `Column` carries text, not a Go value + +`Value` is the rendered string beside `DataType` and `Null`, which each serializer +writes as the load check proved it; a `Value any` would hand every caller a type switch. + +## `hold` names what a draw is kept in, `draw` the draw itself + +Goal 2 wants a name to reach one unit, so a unit takes the stem of what it is, and a +file the stem of the units it holds. + +## A function may not spell a method; two types may + +A call writes a method with its receiver and a function bare, so one name on both greps +as one unit, which is the reaching cost goal 2 counts, while the receiver before a +shared method name says which type answers. `TestNoFunctionSpellsAMethod` holds it. A +type is outside the rule: `renderScope.hold` answers with a `*hold`, so the two spell +one unit, which no test can judge. + +## The package stays flat + +Go ties a package to one directory, so folders would split the API into packages. + +## The performance gate asserts allocations, not wall-clock time + +`AllocsPerRun` is deterministic across machines, so a ±10% ceiling does not flake under +CI load, while time varies with the machine and its neighbours. A rendering slowdown +almost always costs an allocation too (a lost pre-size, a per-item map, an extra copy). +The benchmark suite (the README's Development) reports time for a human, not as a pass/fail gate. + +## Rows live in a TSV, the shape in JSON + +`New` allocates once per node, so a register of thirty thousand rows written as JSON +objects would cost it a second; a TSV is one allocation whose cells are substrings, and +the JSON says only how a row is composed. The TSV sits beside its category file, named +by `rows`, so a data directory stays a directory of categories, and one nothing names is +a load error rather than a file silently ignored. + +## A selector is bracketed, and a dot inside it is literal + +`municipality[0180]` reads as selection to anyone who has indexed an array, and `[St. +Louis]` keeps a name whole where a colon or a dot-separated spelling could not; zsh +needs the brackets quoted, which the README's examples show. A name that spells another +row's key is refused at load rather than shadowed: a key names one row by contract, so +the name could never select its own, and the check is one lookup per row against the key +index that already exists. + +## `parent` names the link column and the table alike + +One word says both, so a child table sits in its parent's folder and links on a column +of the parent's name; the geo plan wants exactly that, and a table needing another +folder or another column name would be asking for a second spelling. + +## After a row, a path names a column or a linked table + +`locality[Lund].address`, a template beside the family under a selected row, is refused +today; admitting it later is additive, since a refused spelling gains a meaning and no +accepted one changes, so the door stays open for the address records the plan describes. + +## A table read into is pinned; a table read whole draws afresh + +A path into a table pins its row for the render and group, as a reference path pins its +level, and a bare `{/city}` draws a row each time and pins no row of its family, as a +bare reference does; so a bare table beside a path into its family is refused like a +bare reference beside a path into it. A bare table reference still counts as a read for +a `drawGroup`, since the group is what draws it apart from the family's pins. + +## Every reference path into one family selects the same rows, per render and group + +A selector pins rows for the render, and a read that draws freely before it could pin a +row the selector contradicts, so accepting both would make the result depend on which +token rendered first. Requiring one selection per family per group is checkable at load +with no data lookup beyond the selectors themselves, and the error names the rewrite. +Two selectors naming one row by key and by name compare equal, since the load check +resolves them. + +## A parent row with no child row is a load error + +A descendant is drawn inside the nearest pinned ancestor, so every ancestor row must +lead to a row at every level below it, or a render could find nothing to draw. The +import script drops or fills such rows; the alternative, falling back to a free draw, +would break the consistency the link exists for without saying so. + +## The choice-of-rows fence guards a data file's root, and requires string fields + +A table is a category with a TSV beside its file, so only a root choice has the spelling +the fence names; a nested choice of same-shaped templates and an inline one keep +loading. Fields must all be strings because a cell is a string node: a choice whose +items carry a nested choice is not one table but two linked ones. + +## A table is a record of string columns + +Its columns are the CSV header and the `INSERT` column list, fixed by the TSV header, so +a table is a record by construction; every column is a string until a typed column +option earns its place. + +## The key index is built at load, the rest on first draw + +A link is proved against the parent's keys and a key's uniqueness is a data mistake, so +both are load-time; the name index and the per-parent child lists serve only a draw or a +selection, so they wait for the first one, keeping `New` linear in the bytes read. + +## Two categories may name one TSV + +Each is a view of the file with its own format and options, at the cost of holding the +rows twice, which is what a category over a register with two natural formats asks for; +a TSV nothing names stays a load error, since that one is a file forgotten rather than +shared. + +## The rows of a table are alternatives + +Only one row renders per read, so within one read a cell in one row and a cell in +another never meet, and each may select its own row of another table; the cells of one +row, and whatever they reach, do meet, and so does the format beside them. The rule the +fence applies is that two reads meet unless they sit in rows that never render together: +rows falling under different rows of a table a path pinned, two rows of that table +itself included. A choice's items get no such treatment yet: two items selecting +different rows are still refused. + +## A table never reaches its own family, by any route + +A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row that lists +three localities from other regions is the output the family exists to prevent, so the +own-family fence walks through both rather than stopping where the draw fences do. + +## Tables carrying token cells stay small + +The family fence compares the reads of every pair of rows that can render together, so a +table whose every row's cell selects a row of another table loads in time quadratic in +its rows: about a second at four thousand rows. No shipped table carries such cells, and +a register is a column set rather than a set of references, so the fence is left as it +is until a real data set needs the indexed form. + +## A path is walked once without drawing before it is walked for real + +A path that fails below its first level then moves no seeded stream, at the cost of one +draw-free walk per call, which allocates nothing. + +## A country's postal codes and streets are siblings under its locality + +No open source pairs a Swedish street with its postnummer, and pairing the US through +its ZIPs would shape the two trees differently, so both draw inside the pinned locality +and an address agrees at that level. A street's own code is the exact pairing to add +when a source carries it. + +## A locale's `address` reads its country's `geo` tree, so the shipped set loads whole + +`data/sv_SE` alone no longer loads: a test loads `data` and prefixes the locale, and +`--no-shipped-data -d` takes the whole `data` folder or a set of one's own. + +## The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more + +Sweden fits whole in 700 KB; every US place of 10,000 would pass a megabyte and fetch +1,200 counties of TIGER files, so the threshold sits where the two countries match in +size, and `--min-population` and `--streets-per-locality` on the import scripts build a +fuller set. The two trees add about 20 ms to `New`, which loads the shipped set in about +45 ms. + +## A locale's `address` restates its country record's format + +A record cannot read another whole and keep its columns, so `sv_SE.address` names the +same four columns as `geo.SE.address`, each a reference into it, and the format appears +twice; a column is spelled the same in both, `street-number`, so the two never disagree +on a name. + +## A postort's kommun comes from its name, its tätort or its codes, never from distance + +GeoNames leaves a fifth of Sweden's codes without a kommun and carries stale spellings; +the nearest code across a border named the wrong kommun half the time it was tried, so a +postort none of the three rules place is dropped, as is one not cased like a place name. + +## A highway designation is not a street, and a US postal code belongs to the place holding most of its land inside places + +`I- 55 Bus` and `US Hwy 1` carry the most address ranges in many places and would head +every address, so the import drops names spelled as a route. A ZCTA goes to the place +its largest in-place part lies in, census-designated places left out since they never +ship, and ships only when that place does, so a few dozen places whose every code lies +mostly in a bigger neighbour ship no address; counting the land outside every place too +would drop a quarter of the places, whose codes straddle unincorporated land, for a +postal city the USPS mostly names the same way. + +## `--list` stays a plain list of paths + +It is what a script reads, so every line has to be a path that `Fake` takes; a marker +for the tables a `[selector]` follows, or a legend above them, would make the output +something to parse before use. `--help` names the selector spelling instead, and the +Table section teaches it. + +## A layout is always quoted + +A layout may carry the comma that separates arguments, `'January 2, 2006'`, and one +spelling for every layout beats a rule about which ones need the quotes, so the bare +spelling is refused naming the quoted one. The layout is Go's reference time because the +library renders with it and a Go caller already knows it; its names are English, and a +locale's own month and weekday names are data. + +## A title is a table under `sex` + +A prefix drawn apart would put `Mr` on a record whose `sex` column says `female`, which +is the disagreement the record exists to prevent; the tables this set already has are +what a title needs, so `en_US.title` links to `sex` as `first-name` does. Swedish has no +everyday sexed honorific, so `sv_SE.title` is a table as well but carries no `parent`. +Its weight column is `share`, not the `count` a name table carries, because the values +are a curated proportion rather than bearers anyone counted. + +## A table owns the spelling of a selector on it + +A reference reaches a table by a path that carries no selector — `sv_SE.person.first` +reads `first-name` through `sex` — so the walk that resolved a name cannot say where a +reader would type one. The table's own location can, which is why it keeps its path, and +why an ambiguity error names `sv_SE.sex[f].first-name[Kim]` rather than the table's own +name. + +## No builtin reads the clock, so a date is bounded by days, never by an age + +An `age(min,max)` would make a seeded fixture change with the day it runs on, which is +what a seed exists to prevent; a birthdate for someone 20 to 60 is +`date(1966-01-01,2006-12-31,…)`, re-pinned as any fixture is. + +## A name column without a key resolves inside its parent + +A given name both sexes carry is a row under each, so `name` cannot be the key; the +parent's row tells the two apart, `sex[f].first-name[Kim]`, the ambiguity error spells +each row inside its parent, and a name repeating inside one parent row is refused at +load, since nothing could then select it. + +## `misc` is what every locale shares + +A category whose facts differ by country belongs in that country's locale, read from the +register that country's own records use; `misc` takes only sources that are +international. NHTSA vPIC and Mobility Sweden's registrations are national, so they are +for `en_US.car` and `sv_SE.car`, and `misc.car` waits for an international source. + +## A register's canonical spelling loses to the one its domain writes + +Where a source offers several spellings of one fact, the shipped one is what records in +that domain carry. `misc.timezone` reads `zone.tab` and not the `zone1970.tab` that +supersedes it, because the latter keeps one zone per set of countries that have agreed +since 1970: it spells Sweden `Europe/Berlin`, and no Swedish system writes that. For the +same reason `misc.language` takes the ISO 639-2 register's first synonym over CLDR, +which says "Chinese, Mandarin" for `zh`. + +## `misc.territory` is the spine, and a `misc` table naming a territory links to it + +Where every territory row has a child the column is a `parent`, and the import drops the +child rows whose territory the set does not ship — 17 of `misc.timezone`'s, Antarctica's +ten among them. Agreement across a record is worth more than the last rows of a table. +Where no such link can hold the fact stays a column. Layer your own `misc.territory` +over the shipped one and you must layer `misc.timezone` too, or the link fails at load +naming the row. + +## A table whose register publishes no frequency draws evenly + +`misc.httpmethod`, `misc.port`, `misc.httpstatus`, `misc.mimetype` and `misc.tld` weigh +every row alike, so GET is a ninth of the methods drawn. Goal 12 keeps an authored fact +out of a sourced table, and no register publishes how often a method, a port or a TLD is +used, so a weight here would be invented. Where one exists it is read, as +`misc.timezone` reads GeoNames populations and `sv_SE.first-name` SCB bearers. + +## `misc.timezone` weighs a zone by the people living in it + +The weight is the population GeoNames records in the zone's cities of 15,000 or more, +floored at 15,000, which goal 14 asks for: over 300 seeded draws of +`misc.territory[US].timezone`, the four zones most Americans live in took 278 where an +even weight gave them 44, and `America/Indiana/Petersburg`, a town of 2,400, fell from 17 +to 0. Valid while a draw weighted this way lands where people live. + +## `misc.port` selects by number, and carries no `name` + +29 of its services sit on more than one port, `http-alt` on three, so +`misc.port[http-alt]` could name no one row. The number is what a port field holds +anyway, and `misc.port[443].service` reads the other way. + +## `misc.car` is one flat table, not a make linked to its models + +A row is a make and a model drawn together, so no render pairs a Volvo with a RAV4. Two +linked tables would reach the same pairs and add a selector nothing asks for; a make +alone is `misc.car.make`. + +## `misc.territory` names its sovereign in a column, and there is no `misc.country` table + +ISO 3166-1 codes territories, so `territory` is the honest name, and `is_independent` in +the register gives each one's state. A second table of the 195 sovereigns would hold a +`DK` row beside the territory `DK` row, both carrying Denmark's capital, currency, flag +and TLD — two owners for one fact, drifting at the next import. Splitting the columns to +avoid that is worse: put `capital` on the territory alone and a country row can no +longer name Copenhagen. The column cannot be a `parent`, since a table never reaches its +own family; a test proves every value names a row instead. A territory the register +records no state for stands alone, which is `EH` alone, and naming one for it would be a +claim fejkdata has no business making. + +## `misc.loglevel` is a table keyed by the code, rendering POSIX's keyword + +A flat list of names carries neither the code a PRI encodes nor a selector reaching it, +and goal 3 draws the two as one fact. The canonical spelling losing to the one its +domain writes, above, settles the rest: a configuration writes `info`, so RFC 5424's +`Informational` stays the `severity` column, and the code and the keyword hold the two +selector slots. + +## `misc.tld` keys carry the leading dot, where other tables key on a bare code + +The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a +bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a host +with no separator. `misc.tld[se]` misses for it, which `todo.md` carries. + +## `misc.tld` is a table of its own, and `misc.territory.tld` stays a column + +A `parent` demands a child for every parent row, so linking them would drop every root +zone row naming no territory, which is most of them, and goal 12 holds a sourced table +whole. The loader refuses the link outright anyway: `tld` is a column of +`misc.territory`, and a table may not be named like a column of its ancestor. + +## `misc.territory` carries a currency code, it does not link to `misc.currency` + +A `parent` demands a child for every parent row, and ISO 4217 registers codes no +country's row can name: the funds codes (Mvdol, WIR Euro, US Dollar (Next day)), and VED +beside VES, both Venezuela's, of which a country row names one. Linking would trade the +register for the link, and `misc.territory.currency` already pairs a country with its +currency in one draw. + +## An extension may name two media types + +`.xml`, `.rtf`, `.sub`, `.mpp` and `.ac` each name two rows of `misc.mimetype`. +Separating them would mean dropping a registered type, or naming one by an extension +that is not its own — `.mpt` is Project's template, not its document. Selecting such a +name is an error listing both keys; select by type instead. + +## The Swedish ids draw Skatteverket's test series + +A Luhn-valid personnummer over a random birth number may be a living person's; 238 and +239 after any date are blocked from assignment, so the shipped `personnummer` and +`samordningsnummer` use those. They sit in a `birth-number` table under `sex` rather +than as a column of it: the render's shared draw of the family is what makes the number +and the name agree on sex, and `sex` stays one shape across locales instead of +collecting every sex-keyed id fact. A samordningsnummer's day, the birthday plus 60, is +drawn from 61 to 88, valid in every month, rather than computed from the date drawn. + +## The US given names come from a mirror of the SSA file + +ssa.gov refuses a client outside the US, so `names-us.py` reads a GitHub copy that ends +at 2020, which a count over the births since 1930 barely feels; `--names` takes the +official zip. The SSA's placeholder rows are top-1000 entries that name nobody, so the +import drops them by name rather than by a rank a regeneration would move. + +## `List` advertises direct descents only + +`region.municipality.locality` is listed, and `region.locality` resolves too but is not: +the set of every descent through a chain of five tables is every subsequence of it, and +the direct chain is the one a reader can predict from the tables' parents. diff --git a/drawfence.go b/drawfence.go index ecbffa4..1e30b7c 100644 --- a/drawfence.go +++ b/drawfence.go @@ -38,6 +38,8 @@ func checkNestedDrawGroup(fields map[string]node, group string) error { // drawCheck fences each template of a scope as a render of its own, remembering which nodes read a // reference path. +// docs/decisions.md#a-render-shares-one-reference-draw-per-category-per-group +// docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences type drawCheck struct { memo map[readsMemo]bool } @@ -79,6 +81,7 @@ func (c *drawCheck) checkDraws(path string, n node) error { // checkRecordDraws fences the columns of a record — a template compiled at the top without a // repeat — so a load proves the record view of it as well as the string view. +// docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load func (c *drawCheck) checkRecordDraws(path string, n node) error { t, ok := n.(*template) if !ok || !t.record { diff --git a/family.go b/family.go index fad404e..2bfd352 100644 --- a/family.go +++ b/family.go @@ -96,6 +96,7 @@ func (r *tableRead) selected(t *table) (tableSel, bool) { // read one consistent draw: a table rendered whole beside a path into the family, // a table one read draws that another pins, and two reads pinning different rows. // The reads come sorted by group and path, so which pair is reported does not vary. +// docs/decisions.md#every-reference-path-into-one-family-selects-the-same-rows-per-render-and-group func checkFamilies(reads []pathRead) error { for i, r := range reads { if r.tr == nil { @@ -155,6 +156,7 @@ func (r *tableRead) replay(d *pinSet) error { // checkFamilyPair refuses a table read whole beside a path into its family, and a // table one read draws that the other pins, since which token renders first would // then decide the row. +// docs/decisions.md#a-table-read-into-is-pinned-a-table-read-whole-draws-afresh func checkFamilyPair(a, b pathRead) error { for _, pair := range [][2]pathRead{{a, b}, {b, a}} { x, y := pair[0], pair[1] @@ -192,6 +194,7 @@ func (r *tableRead) drawnOf(pins *pinSet) *table { // away and through a repeat or a draw group too, a table of its own family: a row // rendered whole draws its row without pinning it, so the family would draw apart // from the row being rendered, whichever draws the reaching template holds. +// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route func checkOwnFamily(t *template) error { own := t.table if own == nil { @@ -227,6 +230,7 @@ func checkOwnFamily(t *template) error { // alternatives reports whether two reads sit in different rows of one table. Not clash: a row drawn // whole is held without its ancestors, since the render draws it over the whole table. +// docs/decisions.md#the-rows-of-a-table-are-alternatives func alternatives(a, b drawAt) bool { found := false a.alt.each(func(t *table, r int) { diff --git a/fejkdata.go b/fejkdata.go index 3bcb894..ac2fbfa 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -21,7 +21,7 @@ var shippedFS embed.FS // any one path; the CLI's --repeat shares it. const MaxRepeat = 1 << 20 -var _ [^uint(0)>>63 - 1]struct{} // 64-bit only, per the README's Decisions +var _ [^uint(0)>>63 - 1]struct{} // docs/decisions.md#64-bit-targets-only // ErrNoData is returned by New when no source is loaded at all. var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS") diff --git a/graph.go b/graph.go index d0fcf79..c17f881 100644 --- a/graph.go +++ b/graph.go @@ -206,6 +206,7 @@ func (m reachMemo) of(n node) int { // repeatCheck bounds the renders a repeat multiplies to along any root-to-leaf // path, so nested repeats cannot build what one repeat may not. +// docs/decisions.md#the-repeat-cap-bounds-renders-not-bytes func repeatCheck(path string, n node, mem reachMemo) error { if t, ok := n.(*template); ok && t.repeat > 1 && mem.of(n) > MaxRepeat { return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.of(n), MaxRepeat) diff --git a/holdfence.go b/holdfence.go index 562c915..cab1feb 100644 --- a/holdfence.go +++ b/holdfence.go @@ -9,6 +9,7 @@ import ( // heldCheck rejects every route to a held sibling name except the ones that read its // draw. An expansion holds one draw of that name; anything else that renders it draws // again, and the two disagree. +// docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences func heldCheck(path string, n node) error { t, ok := n.(*template) if !ok || len(t.held) == 0 { @@ -181,6 +182,7 @@ func renders(n node, want, seen map[node]bool) bool { // level's held draw while rendering the level expands it afresh, so their values would // disagree. Reads are compared in sorted order, so which pair is reported does not // depend on where the tokens sit. +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error { names := boundReaders(format, bound, refs) // Stable over one format-order scan, so two readers of one name (a token and a @@ -231,6 +233,7 @@ func boundReaders(format string, bound map[string]string, refs map[string]refBin // checkNoRepeatedRead rejects a bare token repeated on a held name: {w} {w} beside // {uppercase(w)} would read one draw twice, where {w} {w} alone draws twice. The // error names the single-token spelling. +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) error { count := map[string]int{} return eachToken(format, func(t ftoken) error { diff --git a/layout.go b/layout.go index a389bc8..69f26c1 100644 --- a/layout.go +++ b/layout.go @@ -34,6 +34,7 @@ func quotedLayout(a string) bool { } // layoutArg is the Go layout a quoted arg holds, refused when unquoted or constant. +// docs/decisions.md#a-layout-is-always-quoted func layoutArg(a string) (string, error) { if !quotedLayout(a) { bare := strings.Trim(a, `'"`) diff --git a/node.go b/node.go index cf3134b..4179d69 100644 --- a/node.go +++ b/node.go @@ -117,6 +117,7 @@ func compileCategory(v any, name string, files *categoryFiles) (node, error) { // checkNotRows refuses a choice of templates sharing one format and one set of // string fields: each item is a row, and the rows file is the spelling for that. +// docs/decisions.md#the-choice-of-rows-fence-guards-a-data-files-root-and-requires-string-fields func checkNotRows(items []any, name string) error { var format string var fields []string diff --git a/reference.go b/reference.go index 7b6a510..92e8559 100644 --- a/reference.go +++ b/reference.go @@ -84,8 +84,8 @@ func linkRefs(root map[string]node) error { } // linkTemplateRefs binds one template's references against root, refusing one that names the -// category it sits in: a category is a unit, and a reference back into it describes a draw other -// than the fields beside it. +// category it sits in. +// docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { names := refTokens(t.format) if len(names) == 0 { diff --git a/table.go b/table.go index 23a9081..f9c2431 100644 --- a/table.go +++ b/table.go @@ -358,6 +358,7 @@ func setTablePaths(root map[string]node) { // linkTables binds every table's parent to the table beside it, and proves the // links: a parent has a key, every link cell is one, every parent row is linked // to, no chain of parents closes, and no child is named like a parent's column. +// docs/decisions.md#a-parent-row-with-no-child-row-is-a-load-error func linkTables(root map[string]node) error { var walk func(dir string, children map[string]node) error walk = func(dir string, children map[string]node) error { diff --git a/todo.md b/todo.md index 40d5d2d..b1e9c44 100644 --- a/todo.md +++ b/todo.md @@ -19,17 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Move the README's Decisions section to `docs/decisions.md`, leaving a one-line index - of the titles in `AGENTS.md`, and drop the `AGENTS.md` line pointing decisions at the - README. Give each fence a bare link to the entry it enforces: every seat needed - "The rows of a table are alternatives", "The expansion hold and the render's draws - are two fences" or "A table read into is pinned" and found it by grepping 470 lines. - Carry the timezone weight's premise in with it: weighting by GeoNames city - population moved 300 seeded draws of `misc.territory[US].timezone` from 44 landing on - the four zones most Americans live in to 278, and dropped `America/Indiana/Petersburg` - (pop. 2,400) from 17 draws to 0. Carry in the decision `linkTemplateRefs`'s doc holds - today, which no entry records: a category is a unit, so a reference back into it - describes a draw other than the fields beside it, which is what goal 3 asks for. - Keep reference bindings out of `template.fields`, so no walk over a template's fields needs the `isRef` filter `named`, `paths`, `recordColumns` and `template.field` carry. - Compile a template's format once: `compileFormat` runs again from `linkTemplateRefs`, @@ -285,6 +274,8 @@ because pairing a street with its exact postnummer rewrites shipped rows. ## Not release-bound +- Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and + the premise it rests on; propose a goal for an entry that serves none. - Hold every change the charter says owes a `CHANGELOG.md` entry to one in CI: the check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. diff --git a/value.go b/value.go index c6f78ca..148e1b0 100644 --- a/value.go +++ b/value.go @@ -27,6 +27,7 @@ type valueProof struct { // checkDatatype rejects a typed column item some render of which is not text of its datatype, // and one restating the datatype of the column it is. +// docs/decisions.md#a-column-of-one-reference-alone-is-the-column-it-reads func (p *valueProof) checkDatatype(path string, n node) error { t, ok := n.(*template) if !ok || t.datatype == DataTypeString { -- 2.52.0 From 84b3252bfcc8d86b4d6bfaa2179c668a4e9299fe Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:45:53 +0200 Subject: [PATCH 079/966] Walk a template's fields in the shape test without skipping reference bindings --- shape_test.go | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/shape_test.go b/shape_test.go index 6107d07..e9d418e 100644 --- a/shape_test.go +++ b/shape_test.go @@ -104,10 +104,8 @@ func readsFact(n node) string { for _, b := range n.refs { set[strings.TrimPrefix(b.key, "/")] = true } - for name, field := range n.fields { - if !isRef(name) { - collect(field) - } + for _, field := range n.fields { + collect(field) } } } @@ -130,8 +128,9 @@ func TestShippedShapeNamesReads(t *testing.T) { "c": `{"format":"{v} {n}","n":[null,{"format":"{int(1,9)}","datatype":"integer"}],"v":["z","w"]}`, "d/pos": `["{.q}","{/b}"]`, "d/q": `"r"`, + "e": `"{/a}"`, })) - want := "a\tformat \"{x}\"\treads b c\na.x\tstring\nb\tformat \"y\"\nc\tformat \"{v} {n}\"\nc.n\tinteger null\nc.v\tstring\nd.pos\treads b d.q\nd.q\tformat \"r\"\n" + want := "a\tformat \"{x}\"\treads b c\na.x\tstring\nb\tformat \"y\"\nc\tformat \"{v} {n}\"\nc.n\tinteger null\nc.v\tstring\nd.pos\treads b d.q\nd.q\tformat \"r\"\ne\tformat \"{/a}\"\treads a\n" if got := shippedShape(f); got != want { t.Fatalf("shippedShape =\n%s\nwant\n%s", got, want) } -- 2.52.0 From 5aae86638b3f3c93c22c5fc5a803aa1659936fc7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 13:47:51 +0200 Subject: [PATCH 080/966] Keep reference bindings out of template.fields, in template.refHeads --- datatype.go | 2 +- drawfence.go | 6 +++--- family.go | 2 +- fejkdata.go | 3 --- graph.go | 9 ++------- hold.go | 6 +++--- holdfence.go | 4 ++-- node.go | 16 +++++++--------- path.go | 2 +- record.go | 15 +-------------- reference.go | 10 ++++------ todo.md | 2 -- value.go | 2 +- 13 files changed, 26 insertions(+), 53 deletions(-) diff --git a/datatype.go b/datatype.go index 36ba1ad..fae8097 100644 --- a/datatype.go +++ b/datatype.go @@ -92,7 +92,7 @@ func checkColumns(s nodeScope) error { if !ok || !t.record { return nil } - for _, name := range recordColumns(t) { + for _, name := range sortedNames(t.fields) { if err := check(path, name, t.fields[name]); err != nil { return err } diff --git a/drawfence.go b/drawfence.go index 1e30b7c..fb1e3f5 100644 --- a/drawfence.go +++ b/drawfence.go @@ -88,7 +88,7 @@ func (c *drawCheck) checkRecordDraws(path string, n node) error { return nil } // A record-only template's format renders nothing, so weigh the columns, not the format. - columns := recordColumns(t) + columns := sortedNames(t.fields) reads := false for _, name := range columns { reads = reads || c.readsPath(t.fields[name]) @@ -148,7 +148,7 @@ func readsTable(n node, a arm) bool { if !isTemplate { return false } - _, isTable := t.fields[a.key].(*table) + _, isTable := t.head(a.key).(*table) return isTable } @@ -248,7 +248,7 @@ func (w *drawWalk) walk(n node, at drawAt) { // pins entered, so a selected row renders only its own cells. func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { if a, reads := refRead(from, e.label); reads { - tr := tableReadOf(from.(*template).fields[a.key], a, e.to) + tr := tableReadOf(from.(*template).head(a.key), a, e.to) if k := (readKey{at.group, a.path, at.alt.key()}); !w.read[k] { w.read[k] = true w.reads = append(w.reads, pathRead{at, a, tr}) diff --git a/family.go b/family.go index 2bfd352..87ddc28 100644 --- a/family.go +++ b/family.go @@ -212,7 +212,7 @@ func checkOwnFamily(t *template) error { seen[n] = true for _, e := range renderEdges(n) { if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).fields[a.key].(*table); isTable && head.family() == own.family() { + if head, isTable := n.(*template).head(a.key).(*table); isTable && head.family() == own.family() { return e, head, true } } diff --git a/fejkdata.go b/fejkdata.go index ac2fbfa..f476679 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -146,9 +146,6 @@ func paths(n node) []string { case *template: out := []string{""} for _, name := range sortedNames(n.fields) { - if isRef(name) { // a binding, not a path segment - continue - } for _, p := range paths(n.fields[name]) { out = append(out, join(name, p)) } diff --git a/graph.go b/graph.go index c17f881..d68982e 100644 --- a/graph.go +++ b/graph.go @@ -78,14 +78,9 @@ func contained(n node) []namedNode { } } -// named skips a bound {/path} key: it is a render edge, not containment, so using -// it as a path segment would report a node under a path that does not reach it. func named(m map[string]node) []namedNode { out := make([]namedNode, 0, len(m)) for _, name := range sortedNames(m) { - if isRef(name) { - continue - } out = append(out, namedNode{name: name, node: m[name]}) } return out @@ -134,8 +129,8 @@ func renderEdges(n node) []renderEdge { var es []renderEdge add := func(name, operand string) { a := splitArm(name, n.refs) - c, ok := n.fields[a.key] - if !ok { + c := n.head(a.key) + if c == nil { return } for _, leaf := range pathLeaves(c, a.tail) { diff --git a/hold.go b/hold.go index 9ad3629..62b3ab1 100644 --- a/hold.go +++ b/hold.go @@ -21,7 +21,7 @@ type draw struct { } // readField renders one arm of a token. An arm's key is a sibling field or a -// reference linkRefs bound into fields. A name the expansion holds — a level some +// reference linkRefs bound into refHeads. A name the expansion holds — a level some // token addresses by dotted path, or a field an operand reads — is drawn once and // kept, so {place.postal-code} and {place.locality} read one row, either read twice // gives one value, and a shown operand is the operand computed. Every other name is @@ -32,13 +32,13 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if len(a.tail) > 0 { panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key)) } - return draw{text: render(s, t.fields[a.key], sc)} + return draw{text: render(s, t.head(a.key), sc)} } d := readHold(held, sc, a) if r, done := d.value[a.path]; done { return r } - leaf, err := walkPath(t.fields[a.key], a.tail, pathWalk{pins: &d.pins, draws: &pathDraws{s: s, held: d, a: &a}}) + leaf, err := walkPath(t.head(a.key), a.tail, pathWalk{pins: &d.pins, draws: &pathDraws{s: s, held: d, a: &a}}) if err != nil { panic(fmt.Sprintf("fejkdata: %q: %v; a fence should have refused this at New", a.name, err)) } diff --git a/holdfence.go b/holdfence.go index cab1feb..5eae660 100644 --- a/holdfence.go +++ b/holdfence.go @@ -52,12 +52,12 @@ func heldHeads(t *template) []string { func heldNodes(t *template, head string, readers []reader) map[node]bool { held := map[node]bool{} if _, isPath := t.bound[head]; !isPath { - operandDraw(t.fields[head], held) + operandDraw(t.head(head), held) return held } for _, r := range readers { if a := splitArm(r.name, t.refs); a.key == head { - coverPath(t.fields[head], a.tail, held) + coverPath(t.head(head), a.tail, held) } } return held diff --git a/node.go b/node.go index 4179d69..353acae 100644 --- a/node.go +++ b/node.go @@ -49,7 +49,7 @@ func (*null) isNode() {} type template struct { // Filled by `compileString`, `compileTemplate` and `table.compileWhole`: format string - fields map[string]node // `linkTemplateRefs` adds each bound reference here + fields map[string]node repeat int separator string datatype DataType @@ -78,19 +78,19 @@ type template struct { // Filled by `linkTemplateRefs` and `keyDrawGroup`, from the assembled tree: refs map[string]refBinding // each reference the format reads -> what it is bound to + refHeads map[string]node // each refBinding.key -> the category it names readsColumn *columnRead // set when the format is one reference alone reading a record's column drawGroupKey string // its draw group keyed by its category: what a render reads its reference paths under } func (*template) isNode() {} -// field is the node a path segment names; a binding is a render edge, not a field. -func (t *template) field(seg string) (node, bool) { - if isRef(seg) { - return nil, false +// head is the node an arm's key names: a sibling field, or a reference's category. +func (t *template) head(key string) node { + if isRef(key) { + return t.refHeads[key] } - n, ok := t.fields[seg] - return n, ok + return t.fields[key] } // compile converts parsed JSON — a category or an inline template — into a node tree, @@ -277,8 +277,6 @@ func compileChoice(items []any, pos position) (node, error) { } c.cum = cum } - // Computed before linkRefs binds references into the items: a binding is keyed - // by a reference sigil, which paths skips, so the set is the same after. c.shared = sharedPaths(c.items) return c, nil } diff --git a/path.go b/path.go index 31b132b..e80a60a 100644 --- a/path.go +++ b/path.go @@ -177,7 +177,7 @@ func walkPath(n node, tail []string, w pathWalk) (node, error) { return nil, err } } - child, ok := n.field(tail[0]) + child, ok := n.fields[tail[0]] if !ok { return nil, fmt.Errorf("no field %q", tail[0]) } diff --git a/record.go b/record.go index 7e4d868..2911f44 100644 --- a/record.go +++ b/record.go @@ -245,7 +245,7 @@ func recordOf(n node) (*template, []Column, error) { if !ok { return nil, nil, errors.New("names a choice, not a template; a record is a template whose fields are its columns") } - names := recordColumns(t) + names := sortedNames(t.fields) if len(names) == 0 { return nil, nil, ErrNoColumns } @@ -274,16 +274,3 @@ func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Re } return r } - -// recordColumns is the sorted non-reference field names — the columns a record -// projects. A {/path} binding is carried in fields under its root path, so only a -// name that is not a reference is a column. -func recordColumns(t *template) []string { - var names []string - for _, name := range sortedNames(t.fields) { - if !isRef(name) { - names = append(names, name) - } - } - return names -} diff --git a/reference.go b/reference.go index 92e8559..f1e34c7 100644 --- a/reference.go +++ b/reference.go @@ -68,7 +68,7 @@ func refSegments(name string, folder []string) ([]string, error) { } // linkRefs resolves every reference in the assembled tree. The head of the path — -// up to the category it names — is bound into the referring template's fields +// up to the category it names — is bound into the referring template's refHeads // under its root path, and the rest reads into it the way a sibling path does, so // a reference is held like a sibling and two spellings of one target are one // draw. It runs once, after all data is merged, so a reference sees the final @@ -91,10 +91,8 @@ func linkTemplateRefs(folder []string, path, category string, t *template, root if len(names) == 0 { return nil } - if t.fields == nil { - t.fields = map[string]node{} - } t.refs = make(map[string]refBinding, len(names)) + t.refHeads = make(map[string]node, len(names)) for _, name := range names { segments, err := refSegments(name, folder) if err != nil { @@ -111,7 +109,7 @@ func linkTemplateRefs(folder []string, path, category string, t *template, root if err := checkPath(target, tail, key); err != nil { return fmt.Errorf("%s: reference {%s}: %w", path, name, err) } - t.fields[key] = target + t.refHeads[key] = target t.refs[name] = refBinding{key, tail} } if err := t.compileFormat(); err != nil { @@ -134,7 +132,7 @@ func columnReadOf(t *template) *columnRead { return nil } a := splitArm(name, t.refs) - target, isTemplate := t.fields[a.key].(*template) + target, isTemplate := t.head(a.key).(*template) if !isTemplate || !target.record || len(a.tail) != 1 { return nil } diff --git a/todo.md b/todo.md index b1e9c44..5a5dbcb 100644 --- a/todo.md +++ b/todo.md @@ -19,8 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Keep reference bindings out of `template.fields`, so no walk over a template's fields - needs the `isRef` filter `named`, `paths`, `recordColumns` and `template.field` carry. - Compile a template's format once: `compileFormat` runs again from `linkTemplateRefs`, so `ops`, `bound` and `held` are provisional until linking and its fences run twice. - Parse a format once and have the fences read `t.ops`: `eachToken` rescans the format diff --git a/value.go b/value.go index 148e1b0..e952969 100644 --- a/value.go +++ b/value.go @@ -158,7 +158,7 @@ func (p *valueProof) template(t *template) proven { case !isFunc: var leaves []node for _, a := range splitArms(body, t.refs) { - leaves = append(leaves, pathLeaves(t.fields[a.key], a.tail)...) + leaves = append(leaves, pathLeaves(t.head(a.key), a.tail)...) } return p.unite(leaves) case name == "calc": -- 2.52.0 From 36714250f6c8571ed2c459d59f271169bbaacd88 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 19:22:43 +0200 Subject: [PATCH 081/966] Say an arm's key is what template.head resolves --- template.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/template.go b/template.go index 7ec28be..257edda 100644 --- a/template.go +++ b/template.go @@ -268,8 +268,8 @@ func fieldTokens(format string) []string { } // arm is one alternative of a {a|b} token or one operand, split into the key -// naming the node in a template's fields (a sibling field, or the head a -// reference is bound under) and the tail of a dotted path into it. A non-empty +// `template.head` resolves (a sibling field, or the head a reference is bound +// under) and the tail of a dotted path into it. A non-empty // tail is what makes the arm a bound draw: its head is drawn once per expansion // (see compileOps). type arm struct { -- 2.52.0 From d10cb9e9f4d3e688f0dbd8659e7c35b1e1af520c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 19:25:40 +0200 Subject: [PATCH 082/966] Leave an arm's key to template.head's doc, and reflow the arm comment --- hold.go | 3 +-- template.go | 7 +++---- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/hold.go b/hold.go index 62b3ab1..42c7d0f 100644 --- a/hold.go +++ b/hold.go @@ -20,8 +20,7 @@ type draw struct { null bool } -// readField renders one arm of a token. An arm's key is a sibling field or a -// reference linkRefs bound into refHeads. A name the expansion holds — a level some +// readField renders one arm of a token. A name the expansion holds — a level some // token addresses by dotted path, or a field an operand reads — is drawn once and // kept, so {place.postal-code} and {place.locality} read one row, either read twice // gives one value, and a shown operand is the operand computed. Every other name is diff --git a/template.go b/template.go index 257edda..ce6792c 100644 --- a/template.go +++ b/template.go @@ -268,10 +268,9 @@ func fieldTokens(format string) []string { } // arm is one alternative of a {a|b} token or one operand, split into the key -// `template.head` resolves (a sibling field, or the head a reference is bound -// under) and the tail of a dotted path into it. A non-empty -// tail is what makes the arm a bound draw: its head is drawn once per expansion -// (see compileOps). +// `template.head` resolves and the tail of a dotted path into it. A non-empty tail +// is what makes the arm a bound draw: its head is drawn once per expansion (see +// compileOps). type arm struct { name string // as written, for messages key string -- 2.52.0 From 526832fc3f475e793de5968231816900cce7be2b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 22:37:32 +0200 Subject: [PATCH 083/966] Compile a format holding a reference once, at link --- node.go | 16 ++++++++++------ table.go | 2 +- todo.md | 2 -- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/node.go b/node.go index 353acae..978efc8 100644 --- a/node.go +++ b/node.go @@ -42,10 +42,6 @@ func (*null) isNode() {} // JSON string is a template with no fields. repeat (default 1) renders that format // that many times and joins the results with separator (default ""), each render // an independent pick. -// -// What `template.compileFormat` fills is final only once `linkTemplateRefs` has -// run: where the format holds a reference, that binds it and compiles the format -// again. type template struct { // Filled by `compileString`, `compileTemplate` and `table.compileWhole`: format string @@ -211,12 +207,20 @@ func compileString(s string) (node, error) { return nil, err } t := &template{format: s, repeat: 1, fromString: true} - if err := t.compileFormat(); err != nil { + if err := t.compileUnbound(); err != nil { return nil, err } return t, nil } +// compileUnbound compiles a format holding no reference; `linkTemplateRefs` compiles the rest. +func (t *template) compileUnbound() error { + if len(refTokens(t.format)) > 0 { + return nil + } + return t.compileFormat() +} + // compileFormat compiles the format into ops, and applies the fences that need the // compiled reads. func (t *template) compileFormat() error { @@ -335,7 +339,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) { return nil, err } t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn} - if err := t.compileFormat(); err != nil { + if err := t.compileUnbound(); err != nil { return nil, err } return t, nil diff --git a/table.go b/table.go index f9c2431..54bf4ff 100644 --- a/table.go +++ b/table.go @@ -335,7 +335,7 @@ func (t *table) compileWhole(format string) error { } t.format = &template{format: format, fields: t.fields, repeat: 1, record: true, table: t} t.whole = &row{t} - return t.format.compileFormat() + return t.format.compileUnbound() } // setTablePaths gives every table the path a selector on it is written at, before a diff --git a/todo.md b/todo.md index 5a5dbcb..c01ef1d 100644 --- a/todo.md +++ b/todo.md @@ -19,8 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Compile a template's format once: `compileFormat` runs again from `linkTemplateRefs`, - so `ops`, `bound` and `held` are provisional until linking and its fences run twice. - Parse a format once and have the fences read `t.ops`: `eachToken` rescans the format in `fieldTokens`, `operandTokens`, `boundReaders`, `renderEdges`, `loneRef`, `operandReader`, `checkNoRepeatedRead` and `compileOps`, and function-versus-field is -- 2.52.0 From 1bf684aab21087eb0c5f23afc9d0737227837fef Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 22:42:04 +0200 Subject: [PATCH 084/966] Want a cell's link-time error to name its line --- table_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/table_test.go b/table_test.go index 632f960..f7ddb13 100644 --- a/table_test.go +++ b/table_test.go @@ -480,6 +480,7 @@ func TestTableFences(t *testing.T) { "format reads into a column": {map[string]string{"t.json": `{"format":"{a.x}","rows":"t.tsv"}`, "t.tsv": "a\nx\ny\n"}, `"a"`}, "a reference into a column": {with(base, map[string]string{"t.json": `"{/region.name.x}"`}), "column"}, "a category referencing itself": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\n{/t.a}\ny\n"}, "names the category it sits in"}, + "a cell repeating a reference": {with(base, map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\nx\n{/region} {/region} {uppercase(/region)}\n"}), "t.a, line 3: token {/region} is repeated"}, } for name, c := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, c.files))) -- 2.52.0 From 2e778f3b31c4a8d3e0f9c9465caa7a1f96615af3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 22:43:05 +0200 Subject: [PATCH 085/966] Name a cell's line in its link-time errors, and say where a format holding a reference compiles --- node.go | 9 ++++----- reference.go | 3 +++ table.go | 2 +- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/node.go b/node.go index 978efc8..8151be8 100644 --- a/node.go +++ b/node.go @@ -41,7 +41,7 @@ func (*null) isNode() {} // template renders a format string, substituting {tokens} from fields. A bare // JSON string is a template with no fields. repeat (default 1) renders that format // that many times and joins the results with separator (default ""), each render -// an independent pick. +// an independent pick. A format holding a reference compiles in `linkTemplateRefs`. type template struct { // Filled by `compileString`, `compileTemplate` and `table.compileWhole`: format string @@ -207,14 +207,13 @@ func compileString(s string) (node, error) { return nil, err } t := &template{format: s, repeat: 1, fromString: true} - if err := t.compileUnbound(); err != nil { + if err := t.compileRefFree(); err != nil { return nil, err } return t, nil } -// compileUnbound compiles a format holding no reference; `linkTemplateRefs` compiles the rest. -func (t *template) compileUnbound() error { +func (t *template) compileRefFree() error { if len(refTokens(t.format)) > 0 { return nil } @@ -339,7 +338,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) { return nil, err } t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn} - if err := t.compileUnbound(); err != nil { + if err := t.compileRefFree(); err != nil { return nil, err } return t, nil diff --git a/reference.go b/reference.go index f1e34c7..00d09e8 100644 --- a/reference.go +++ b/reference.go @@ -91,6 +91,9 @@ func linkTemplateRefs(folder []string, path, category string, t *template, root if len(names) == 0 { return nil } + if t.cellOf != nil { + path = fmt.Sprintf("%s, line %d", path, t.cellRow+2) + } t.refs = make(map[string]refBinding, len(names)) t.refHeads = make(map[string]node, len(names)) for _, name := range names { diff --git a/table.go b/table.go index 54bf4ff..e4cb88f 100644 --- a/table.go +++ b/table.go @@ -335,7 +335,7 @@ func (t *table) compileWhole(format string) error { } t.format = &template{format: format, fields: t.fields, repeat: 1, record: true, table: t} t.whole = &row{t} - return t.format.compileUnbound() + return t.format.compileRefFree() } // setTablePaths gives every table the path a selector on it is written at, before a -- 2.52.0 From 949b2e8db130e89c1cfa6fd14e6c0f76523c5721 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 22:43:19 +0200 Subject: [PATCH 086/966] Record that a format holding a reference is fenced at link, naming its path --- AGENTS.md | 1 + docs/decisions.md | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 6c05dd1..c6a6372 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -40,6 +40,7 @@ In [docs/decisions.md](docs/decisions.md): - A change to what exists is a major; a minor only adds - Seeded output is promised within one version - An error is a contract by what it names, not its bytes +- A format holding a reference is fenced at link, and its errors name its path - Raising the lowest supported Go is a major - The format check runs on the latest Go only - The changelog heading is the one spelling of a release; CI cuts the tag diff --git a/docs/decisions.md b/docs/decisions.md index 5729990..e4f8df1 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -121,6 +121,13 @@ this repo's own are. A script branches on the exit code and reads the named path or spelling, so those hold; wording improves in a minor. +## A format holding a reference is fenced at link, and its errors name its path + +2026-09-24, larv-review. Goal 2: a format compiles once, and one holding a reference +can compile only once bound, so its fences fire at link and name the category path as +every link error does, a cell's with its line; one without a reference still names its +file at compile. Valid while compilation needs the bound references. + ## Raising the lowest supported Go is a major A consumer building on it breaks, which is the one test every rule above applies; Go's -- 2.52.0 From 37f34e556f2e6e948b530db7ca95e0b627a7c8d4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 23:23:01 +0200 Subject: [PATCH 087/966] Want a parsed call to carry its builtin's operands --- calc_test.go | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/calc_test.go b/calc_test.go index 6c750de..608bff0 100644 --- a/calc_test.go +++ b/calc_test.go @@ -94,18 +94,18 @@ func TestCalcReproducible(t *testing.T) { } } -// TestTokenOperandsReadsOnlyAnOperandBuiltin pins what the helper answers for a body that -// is not a calc, and for one whose expression does not parse: nothing, either way. +// TestParsedCallReadsOnlyAnOperandBuiltin pins the operands a parsed call carries for a +// call that is not a calc, and for one whose expression does not parse: none, either way. // checkCalc is what reports a bad expression, so the callers that run after it // never meet one — but they must not have to depend on that order to be safe. -func TestTokenOperandsReadsOnlyAnOperandBuiltin(t *testing.T) { - for _, body := range []string{"plain", "luhn()", "calc()", "calc(1 +)", "calc(()"} { - if got := tokenOperands(body); got != nil { - t.Errorf("tokenOperands(%q) = %v, want none", body, got) +func TestParsedCallReadsOnlyAnOperandBuiltin(t *testing.T) { + for _, format := range []string{"{luhn()}", "{calc()}", "{calc(1 +)}", "{calc(()}"} { + if toks, err := parseFormat(format); err != nil || len(toks) != 1 || toks[0].names != nil { + t.Errorf("parseFormat(%q) = %+v, %v, want one call reading no operand", format, toks, err) } } - if got := tokenOperands("calc(net * qty)"); len(got) != 2 { - t.Errorf("tokenOperands(calc(net * qty)) = %v, want both operands", got) + if toks, err := parseFormat("{calc(net * qty)}"); err != nil || len(toks) != 1 || len(toks[0].names) != 2 { + t.Errorf("parseFormat({calc(net * qty)}) = %+v, %v, want both operands", toks, err) } } -- 2.52.0 From aa972bbd959861ad24c313f74ee4770d5e7f4d5b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 23:32:22 +0200 Subject: [PATCH 088/966] Want a lone reference whose selector holds a parenthesis refused as a template, and drop the parse test's stale order note --- calc_test.go | 2 -- inline_test.go | 39 ++++++++++++++++++++------------------- 2 files changed, 20 insertions(+), 21 deletions(-) diff --git a/calc_test.go b/calc_test.go index 608bff0..1c3dcab 100644 --- a/calc_test.go +++ b/calc_test.go @@ -96,8 +96,6 @@ func TestCalcReproducible(t *testing.T) { // TestParsedCallReadsOnlyAnOperandBuiltin pins the operands a parsed call carries for a // call that is not a calc, and for one whose expression does not parse: none, either way. -// checkCalc is what reports a bad expression, so the callers that run after it -// never meet one — but they must not have to depend on that order to be safe. func TestParsedCallReadsOnlyAnOperandBuiltin(t *testing.T) { for _, format := range []string{"{luhn()}", "{calc()}", "{calc(1 +)}", "{calc(()}"} { if toks, err := parseFormat(format); err != nil || len(toks) != 1 || toks[0].names != nil { diff --git a/inline_test.go b/inline_test.go index c2876d5..c5507e7 100644 --- a/inline_test.go +++ b/inline_test.go @@ -174,25 +174,26 @@ func TestIsTemplate(t *testing.T) { } } for arg, want := range map[string]string{ - "[abc]": `starts with "["`, - "[abc].field": `starts with "["`, - "x[1]y": `"]"`, - "x[[1]]": `"["`, - "x[]": "empty", - "a]b": `holds a "]"`, - "a}b": `holds a "}"`, - `"abc`: `holds a "\""`, - `"a]b`: `holds a "\""`, // the opener the reader typed, not the bracket behind it - "{/sv_SE.person.last}": "{/sv_SE.person.last} is the path sv_SE.person.last written as a template; write sv_SE.person.last", - `"{/sv_SE.person}"`: "write sv_SE.person", - "{.person.last}": "write person.last", - "/sv_SE.person": "write sv_SE.person", - ` "{/sv_SE.person}"`: "write sv_SE.person", - `{"format":"{/sv_SE.person}"}`: "write sv_SE.person", - "//sv_SE.person": "write sv_SE.person", - "{//sv_SE.person}": "{//sv_SE.person} is the path sv_SE.person written as a template; write sv_SE.person", - "/sv_SE/person": `path "sv_SE/person" contains "/"`, - "{/-}": `path "-" is reserved`, + "[abc]": `starts with "["`, + "[abc].field": `starts with "["`, + "x[1]y": `"]"`, + "x[[1]]": `"["`, + "x[]": "empty", + "a]b": `holds a "]"`, + "a}b": `holds a "}"`, + `"abc`: `holds a "\""`, + `"a]b`: `holds a "\""`, // the opener the reader typed, not the bracket behind it + "{/sv_SE.person.last}": "{/sv_SE.person.last} is the path sv_SE.person.last written as a template; write sv_SE.person.last", + `"{/sv_SE.person}"`: "write sv_SE.person", + "{.person.last}": "write person.last", + "/sv_SE.person": "write sv_SE.person", + ` "{/sv_SE.person}"`: "write sv_SE.person", + `{"format":"{/sv_SE.person}"}`: "write sv_SE.person", + "//sv_SE.person": "write sv_SE.person", + "{//sv_SE.person}": "{//sv_SE.person} is the path sv_SE.person written as a template; write sv_SE.person", + "/sv_SE/person": `path "sv_SE/person" contains "/"`, + "{/-}": `path "-" is reserved`, + "{/misc.currency[US Dollar (Next day)]}": "write misc.currency[US Dollar (Next day)]", } { if _, err := IsTemplate(arg); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("IsTemplate(%q) = %v; want it rejected naming %s", arg, err, want) -- 2.52.0 From 467cb6fb18af27cc3c1da76cb20d4d69cce68268 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 23:26:39 +0200 Subject: [PATCH 089/966] Parse a format once into tokens, and have the fences read its ops --- graph.go | 24 ++---- holdfence.go | 63 ++++++---------- inline.go | 13 +++- node.go | 25 +++++-- reference.go | 28 +++---- table.go | 22 ++++-- template.go | 207 +++++++++++++++++++++++++-------------------------- todo.md | 4 - value.go | 8 +- 9 files changed, 186 insertions(+), 208 deletions(-) diff --git a/graph.go b/graph.go index d68982e..b6d8359 100644 --- a/graph.go +++ b/graph.go @@ -127,31 +127,23 @@ func renderEdges(n node) []renderEdge { return es case *template: var es []renderEdge - add := func(name, operand string) { - a := splitArm(name, n.refs) + add := func(a arm, operand string) { c := n.head(a.key) if c == nil { return } for _, leaf := range pathLeaves(c, a.tail) { - es = append(es, renderEdge{leaf, name, operand}) + es = append(es, renderEdge{leaf, a.name, operand}) } } - _ = eachToken(n.format, func(t ftoken) error { - if t.kind != 'b' { - return nil + for _, o := range n.ops { + for _, a := range o.operands { + add(a, o.fn) } - if fn, _, isFunc := funcCall(t.body); isFunc { - for _, operand := range tokenOperands(t.body) { - add(operand, fn) - } - return nil + for _, a := range o.arms { + add(a, "") } - for _, name := range splitOutside(t.body, '|') { - add(name, "") - } - return nil - }) + } return es case *table: return []renderEdge{{to: n.format, label: "format"}} diff --git a/holdfence.go b/holdfence.go index 5eae660..1ab1102 100644 --- a/holdfence.go +++ b/holdfence.go @@ -15,7 +15,7 @@ func heldCheck(path string, n node) error { if !ok || len(t.held) == 0 { return nil } - readers := boundReaders(t.format, t.bound, t.refs) + readers := boundReaders(t.ops, t.bound) for _, head := range heldHeads(t) { if _, isPath := t.bound[head]; isPath && isRef(head) { continue @@ -87,21 +87,14 @@ func checkHeadHeld(t *template, head string, readers []reader) error { // operandReader names the builtin whose operand holds head. func operandReader(t *template, head string) string { - fn := "" - _ = eachToken(t.format, func(tok ftoken) error { - if tok.kind != 'b' || fn != "" { - return nil - } - if name, _, isFunc := funcCall(tok.body); isFunc { - for _, operand := range tokenOperands(tok.body) { - if splitArm(operand, t.refs).key == head { - fn = name - } + for _, o := range t.ops { + for _, a := range o.operands { + if a.key == head { + return o.fn } } - return nil - }) - return fn + } + return "" } // coverPath collects what holding one path pins: every choice level the path @@ -183,8 +176,8 @@ func renders(n node, want, seen map[node]bool) bool { // disagree. Reads are compared in sorted order, so which pair is reported does not // depend on where the tokens sit. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error { - names := boundReaders(format, bound, refs) +func checkNoOverlap(ops []op, bound map[string]string) error { + names := boundReaders(ops, bound) // Stable over one format-order scan, so two readers of one name (a token and a // calc operand both naming "p") are reported as the format writes them. sort.SliceStable(names, func(i, j int) bool { return names[i].path < names[j].path }) @@ -205,28 +198,20 @@ type reader struct{ name, path, label string } // boundReaders lists every way a format reaches a bound sibling field, in the order the // format writes them. An operand renders its field, so it names a level exactly // as a token does; one scan finds both, which is what puts them in one order. -func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader { +func boundReaders(ops []op, bound map[string]string) []reader { var names []reader - _ = eachToken(format, func(t ftoken) error { - if t.kind != 'b' { - return nil - } - if fn, _, isFunc := funcCall(t.body); isFunc { - for _, operand := range tokenOperands(t.body) { - a := splitArm(operand, refs) - if _, isBound := bound[a.key]; isBound && !isRef(a.key) { - names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)}) - } + for _, o := range ops { + for _, a := range o.operands { + if _, isBound := bound[a.key]; isBound && !isRef(a.key) { + names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", o.fn, a.name)}) } - return nil } - for _, a := range splitArms(t.body, refs) { + for _, a := range o.arms { if _, isBound := bound[a.key]; isBound && !isRef(a.key) { names = append(names, reader{a.name, a.path, "token {" + a.name + "}"}) } } - return nil - }) + } return names } @@ -234,16 +219,10 @@ func boundReaders(format string, bound map[string]string, refs map[string]refBin // {uppercase(w)} would read one draw twice, where {w} {w} alone draws twice. The // error names the single-token spelling. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) error { +func checkNoRepeatedRead(c formatOps) error { count := map[string]int{} - return eachToken(format, func(t ftoken) error { - if t.kind != 'b' { - return nil - } - if _, _, isFunc := funcCall(t.body); isFunc { - return nil - } - for _, a := range splitArms(t.body, refs) { + for _, o := range c.ops { + for _, a := range o.arms { if len(a.tail) > 0 || !c.held[a.key] { continue } @@ -251,6 +230,6 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.name, c.holder[a.key], a.key, a.name) } } - return nil - }) + } + return nil } diff --git a/inline.go b/inline.go index 1b23fe5..b44127e 100644 --- a/inline.go +++ b/inline.go @@ -107,8 +107,15 @@ func loneReference(arg string) (string, bool) { raw = m["format"] } format, isString := raw.(string) - body, lone := loneRef(format) - if !isString || !lone { + if !isString { + return "", false + } + toks, err := parseFormat(format) + if err != nil { + return "", false + } + body, lone := loneRef(toks) + if !lone { return "", false } if strings.HasPrefix(body, "/") { @@ -161,7 +168,7 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { return nil } t.keyDrawGroup("") - for _, name := range refTokens(t.format) { + for _, name := range refTokens(t.tokens) { sigil, rest, err := refShape(name) if err != nil { return fmt.Errorf("%s: reference {%s}: %w", path, name, err) diff --git a/node.go b/node.go index 8151be8..2ba3ef3 100644 --- a/node.go +++ b/node.go @@ -45,6 +45,7 @@ func (*null) isNode() {} type template struct { // Filled by `compileString`, `compileTemplate` and `table.compileWhole`: format string + tokens []formatToken fields map[string]node repeat int separator string @@ -203,10 +204,14 @@ func jsonKind(v any) string { } func compileString(s string) (node, error) { - if err := checkTokens(s, nil); err != nil { + toks, err := parseFormat(s) + if err != nil { return nil, err } - t := &template{format: s, repeat: 1, fromString: true} + if err := checkTokens(toks, nil); err != nil { + return nil, err + } + t := &template{format: s, tokens: toks, repeat: 1, fromString: true} if err := t.compileRefFree(); err != nil { return nil, err } @@ -214,7 +219,7 @@ func compileString(s string) (node, error) { } func (t *template) compileRefFree() error { - if len(refTokens(t.format)) > 0 { + if len(refTokens(t.tokens)) > 0 { return nil } return t.compileFormat() @@ -223,7 +228,7 @@ func (t *template) compileRefFree() error { // compileFormat compiles the format into ops, and applies the fences that need the // compiled reads. func (t *template) compileFormat() error { - c := compileOps(t.format, t.refs) + c := compileOps(t.tokens, t.refs) t.ops, t.grow, t.bound, t.held, t.heldLocal = c.ops, c.grow, c.bound, c.held, c.heldLocal t.fixed = true for _, o := range t.ops { @@ -234,10 +239,10 @@ func (t *template) compileFormat() error { if t.fixed && len(t.ops) == 1 { t.lit = t.ops[0].lit } - if err := checkNoOverlap(t.format, t.bound, t.refs); err != nil { + if err := checkNoOverlap(t.ops, t.bound); err != nil { return err } - return checkNoRepeatedRead(t.format, c, t.refs) + return checkNoRepeatedRead(c) } func compileChoice(items []any, pos position) (node, error) { @@ -331,13 +336,17 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" { return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format) } - if err := checkTokens(o.format, fields); err != nil { + toks, err := parseFormat(o.format) + if err != nil { + return nil, err + } + if err := checkTokens(toks, fields); err != nil { return nil, err } if err := checkNestedDrawGroup(fields, o.group); err != nil { return nil, err } - t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn} + t := &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn} if err := t.compileRefFree(); err != nil { return nil, err } diff --git a/reference.go b/reference.go index 00d09e8..be92a11 100644 --- a/reference.go +++ b/reference.go @@ -87,7 +87,7 @@ func linkRefs(root map[string]node) error { // category it sits in. // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { - names := refTokens(t.format) + names := refTokens(t.tokens) if len(names) == 0 { return nil } @@ -130,7 +130,7 @@ type columnRead struct { } func columnReadOf(t *template) *columnRead { - name, lone := loneRef(t.format) + name, lone := loneRef(t.tokens) if !lone || t.repeat != 1 { return nil } @@ -143,17 +143,11 @@ func columnReadOf(t *template) *columnRead { } // loneRef is the reference a format of one reference token and nothing else reads. -func loneRef(format string) (string, bool) { - units, body := 0, "" - err := eachToken(format, func(t ftoken) error { - units++ - body = t.body - return nil - }) - if err != nil || units != 1 || !isRef(body) || strings.ContainsAny(body, "|(") { +func loneRef(toks []formatToken) (string, bool) { + if len(toks) != 1 || toks[0].kind != 'f' || len(toks[0].names) != 1 || !isRef(toks[0].names[0]) { return "", false } - return body, true + return toks[0].names[0], true } // eachTemplate calls fn once per template, with the folder its category sits in @@ -220,13 +214,15 @@ func resolveCategory(root map[string]node, segments []string) (head []string, ta } // refTokens returns the reference names a format reads, as tokens or as operands. -func refTokens(format string) []string { +func refTokens(toks []formatToken) []string { var refs []string seen := map[string]bool{} - for _, name := range append(fieldTokens(format), operandTokens(format)...) { - if isRef(name) && !seen[name] { - seen[name] = true - refs = append(refs, name) + for _, tok := range toks { + for _, name := range tok.names { + if isRef(name) && !seen[name] { + seen[name] = true + refs = append(refs, name) + } } } return refs diff --git a/table.go b/table.go index e4cb88f..f61ebd7 100644 --- a/table.go +++ b/table.go @@ -324,16 +324,22 @@ func (t *table) checkCells() error { // compileWhole compiles the format a row renders whole through, over the columns // as its fields. func (t *table) compileWhole(format string) error { - for _, name := range fieldTokens(format) { - a := splitArm(name, nil) - if _, ok := t.col[a.key]; !ok && !isRef(a.key) && a.key != "" { - return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.columns) - } - } - if err := checkTokens(format, t.fields); err != nil { + toks, err := parseFormat(format) + if err != nil { return err } - t.format = &template{format: format, fields: t.fields, repeat: 1, record: true, table: t} + for _, tok := range toks { + for _, name := range tok.names { + a := splitArm(name, nil) + if _, ok := t.col[a.key]; tok.kind == 'f' && !ok && !isRef(a.key) && a.key != "" { + return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.columns) + } + } + } + if err := checkTokens(toks, t.fields); err != nil { + return err + } + t.format = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, record: true, table: t} t.whole = &row{t} return t.format.compileRefFree() } diff --git a/template.go b/template.go index ce6792c..5e9a863 100644 --- a/template.go +++ b/template.go @@ -57,6 +57,52 @@ func eachToken(format string, fn func(ftoken) error) error { return nil } +// formatToken is one parsed unit of a format: a literal run, a field alternation or a +// builtin call. +type formatToken struct { + kind byte // 'l' literal run, 'f' field alternation, 'b' builtin call + lit string // kind 'l' + body string // the braces' content, as written + fn string // kind 'b' + args []string // kind 'b' + names []string // kind 'f': the '|' arms; kind 'b': the operands its builtin reads +} + +// parseFormat is the one reading of a format's tokens: a '(' outside a selector makes +// a token a call. +func parseFormat(format string) ([]formatToken, error) { + var toks []formatToken + var lit strings.Builder + flush := func() { + if lit.Len() > 0 { + toks = append(toks, formatToken{kind: 'l', lit: lit.String()}) + lit.Reset() + } + } + err := eachToken(format, func(t ftoken) error { + if t.kind == 'l' { + lit.WriteRune(t.r) + return nil + } + flush() + if indexOutside(t.body, '(') < 0 { + toks = append(toks, formatToken{kind: 'f', body: t.body, names: splitOutside(t.body, '|')}) + return nil + } + name, args, ok := funcCall(t.body) + if !ok { + return fmt.Errorf("malformed function token {%s}", t.body) + } + toks = append(toks, formatToken{kind: 'b', body: t.body, fn: name, args: args, names: builtinOperands(name, args)}) + return nil + }) + if err != nil { + return nil, err + } + flush() + return toks, nil +} + // builtin is a format-string function invoked as {name(args)}. It receives the // session (its rng, and the {seq()} counters), the output emitted so far in the // current expansion (for derivations such as a checksum over preceding digits), and @@ -119,15 +165,11 @@ func plural(n int) string { return "s" } -// checkFunc validates a function token at compile time: well-formed, naming a -// known builtin, with the arg count that builtin takes and args its check accepts. -// fields is passed through for the builtins (calc, the transforms) that validate -// against them. -func checkFunc(body string, fields map[string]node) error { - name, args, ok := funcCall(body) - if !ok { - return fmt.Errorf("malformed function token {%s}", body) - } +// checkFunc validates a call at compile time: naming a known builtin, with the arg +// count that builtin takes and args its check accepts. fields is passed through for +// the builtins (calc, the transforms) that validate against them. +func checkFunc(tok formatToken, fields map[string]node) error { + body, name, args := tok.body, tok.fn, tok.args b, known := builtins[name] if !known { return fmt.Errorf("token {%s}: unknown function %q", body, name) @@ -143,34 +185,38 @@ func checkFunc(body string, fields map[string]node) error { return nil } -// checkTokens validates a format string the way expand scans it, so every -// "{token}" is balanced and names an existing field (or a known function). This -// makes a typo'd or dangling reference a New-time error, never a random -// render-time one. -func checkTokens(format string, fields map[string]node) error { - return eachToken(format, func(t ftoken) error { - if t.kind != 'b' { - return nil +// checkTokens proves every parsed token names an existing field or a known function, +// so a typo'd or dangling reference is a New-time error. +func checkTokens(toks []formatToken, fields map[string]node) error { + for _, t := range toks { + if err := checkToken(t, fields); err != nil { + return err } - if indexOutside(t.body, '(') >= 0 { // a function token, not a field - return checkFunc(t.body, fields) - } - names := splitOutside(t.body, '|') - for _, name := range names { + } + return nil +} + +func checkToken(t formatToken, fields map[string]node) error { + switch t.kind { + case 'b': + return checkFunc(t, fields) + case 'f': + for _, name := range t.names { if isRef(name) { if _, _, err := refShape(name); err != nil { return fmt.Errorf("token {%s}: %w", t.body, err) } continue // its target is checked at New (see linkRefs) } - if err := checkArm(name, fields, len(names) == 1); err != nil { + if err := checkArm(name, fields, len(t.names) == 1); err != nil { return fmt.Errorf("token {%s}: %w", t.body, err) } } // Last, so an arm broken on its own terms is reported as that: a repeat is // the consequence of such a mistake, not the mistake itself. - return checkNoRepeatedArm(t.body, names) - }) + return checkNoRepeatedArm(t.body, t.names) + } + return nil } // checkArm validates one sibling name or path against a template's fields. @@ -210,32 +256,16 @@ func hintableRef(name string) bool { return checkPathNames(name) == nil } -// tokenOperands lists the fields one {token} body reads as operands, empty for a -// field token or a builtin that reads none. -func tokenOperands(body string) []string { - name, args, ok := funcCall(body) - if !ok { - return nil - } +// builtinOperands lists the fields a call reads as operands, empty for a builtin that +// reads none, or a call checkFunc refuses. +func builtinOperands(name string, args []string) []string { b, known := builtins[name] - if !known || b.operands == nil { + if !known || b.operands == nil || (b.arity >= 0 && len(args) != b.arity) { return nil } return b.operands(args) } -// operandTokens lists every operand the builtins in a format read. -func operandTokens(format string) []string { - var names []string - _ = eachToken(format, func(t ftoken) error { - if t.kind == 'b' { - names = append(names, tokenOperands(t.body)...) - } - return nil - }) - return names -} - // checkNoRepeatedArm rejects {a|a|b}: an alternation picks its arms evenly, so a // repeated one is a second spelling of weight. The error names the spelling that // does skew a pick. @@ -253,20 +283,6 @@ func checkNoRepeatedArm(body string, names []string) error { return nil } -// fieldTokens returns the field and reference names a format renders via {name} -// or {a|..b} tokens (function tokens, which carry no field edges, are excluded). -// These are exactly the child nodes expand recurses into, through readField. -func fieldTokens(format string) []string { - var names []string - _ = eachToken(format, func(t ftoken) error { - if t.kind == 'b' && indexOutside(t.body, '(') < 0 { - names = append(names, splitOutside(t.body, '|')...) - } - return nil - }) - return names -} - // arm is one alternative of a {a|b} token or one operand, split into the key // `template.head` resolves and the tail of a dotted path into it. A non-empty tail // is what makes the arm a bound draw: its head is drawn once per expansion (see @@ -308,16 +324,6 @@ func pathArm(name, key string, segs []string) arm { return arm{name: name, key: key, tail: segs, levels: levels, path: key + "." + strings.Join(segs, ".")} } -// splitArms splits a token body's '|' alternatives. -func splitArms(body string, refs map[string]refBinding) []arm { - parts := splitOutside(body, '|') - arms := make([]arm, len(parts)) - for i, p := range parts { - arms[i] = splitArm(p, refs) - } - return arms -} - // checkSegments rejects an unfinished path: "{a.}" and "{a..b}" each have a // segment naming nothing. A field really named "" would otherwise make them // resolve, so a typo would read as a path that worked. @@ -345,9 +351,8 @@ type callFn func(s *session, emitted string, operands []string) string // or a builtin already bound to its args. compile builds these so render never // re-scans the format. type op struct { - kind byte // 'l' literal run, 'f' field alternation, 'b' builtin - lit string // kind 'l' - arms []arm // kind 'f': the '|' alternatives, split into key and path once + formatToken + arms []arm // kind 'f': the '|' alternatives, split into key and path once call callFn // operands are the fields the builtin reads, in the order its operands func // fixed; expand reads them before the call. nil for a builtin that reads none. @@ -391,53 +396,41 @@ func (c *formatOps) holdName(a arm, label string) { } } -func (c *formatOps) function(body string, refs map[string]refBinding) { - name, args, _ := funcCall(body) +func (c *formatOps) function(tok formatToken, refs map[string]refBinding) { var operands []arm - for _, operand := range tokenOperands(body) { + for _, operand := range tok.names { a := splitArm(operand, refs) - c.holdName(a, fmt.Sprintf("%s operand %q", name, operand)) + c.holdName(a, fmt.Sprintf("%s operand %q", tok.fn, operand)) operands = append(operands, a) } - c.ops = append(c.ops, op{kind: 'b', call: builtins[name].prep(args), operands: operands}) + c.ops = append(c.ops, op{formatToken: tok, call: builtins[tok.fn].prep(tok.args), operands: operands}) } -func (c *formatOps) field(body string, refs map[string]refBinding) { - arms := splitArms(body, refs) - for _, a := range arms { - if len(a.tail) > 0 { - c.holdName(a, "token {"+a.name+"}") +func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { + arms := make([]arm, len(tok.names)) + for i, name := range tok.names { + arms[i] = splitArm(name, refs) + if len(arms[i].tail) > 0 { + c.holdName(arms[i], "token {"+arms[i].name+"}") } } - c.ops = append(c.ops, op{kind: 'f', arms: arms}) + c.ops = append(c.ops, op{formatToken: tok, arms: arms}) } -// compileOps compiles a format string. Call checkTokens first: it is what proves -// the scan and every token are valid. -func compileOps(format string, refs map[string]refBinding) formatOps { +// compileOps compiles a parsed format. Call checkTokens first: it is what proves +// every token valid. +func compileOps(toks []formatToken, refs map[string]refBinding) formatOps { var c formatOps - var lit strings.Builder - flush := func() { - if lit.Len() > 0 { - c.grow += lit.Len() - c.ops = append(c.ops, op{kind: 'l', lit: lit.String()}) - lit.Reset() + for _, tok := range toks { + switch tok.kind { + case 'l': + c.grow += len(tok.lit) + c.ops = append(c.ops, op{formatToken: tok}) + case 'b': + c.function(tok, refs) + case 'f': + c.field(tok, refs) } } - _ = eachToken(format, func(t ftoken) error { - switch t.kind { - case 'l': - lit.WriteRune(t.r) - case 'b': - flush() - if _, _, isFunc := funcCall(t.body); isFunc { - c.function(t.body, refs) - } else { - c.field(t.body, refs) - } - } - return nil - }) - flush() return c } diff --git a/todo.md b/todo.md index c01ef1d..684cec7 100644 --- a/todo.md +++ b/todo.md @@ -19,10 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Parse a format once and have the fences read `t.ops`: `eachToken` rescans the format - in `fieldTokens`, `operandTokens`, `boundReaders`, `renderEdges`, `loneRef`, - `operandReader`, `checkNoRepeatedRead` and `compileOps`, and function-versus-field is - decided by `indexOutside(body, '(')` in two of them and by `funcCall` in the rest. - Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which callbacks are nil and whether `draws` is, and `walkChoice` returns `nil, nil` where none is set. Make the render mode reachable only through a proved path: a walk that diff --git a/value.go b/value.go index e952969..9b310c6 100644 --- a/value.go +++ b/value.go @@ -152,12 +152,12 @@ func (p *valueProof) template(t *template) proven { v.notOperand = notOneValue(t.format, "{int()}, {float()}, {seq()}, {digits()} or {calc()}") return v } - body := t.format[1 : len(t.format)-1] - name, args, isFunc := funcCall(body) + o := t.ops[0] + body, name, args := o.body, o.fn, o.args switch _, isTransform := transforms[name]; { - case !isFunc: + case o.kind == 'f': var leaves []node - for _, a := range splitArms(body, t.refs) { + for _, a := range o.arms { leaves = append(leaves, pathLeaves(t.head(a.key), a.tail)...) } return p.unite(leaves) -- 2.52.0 From 2a7499e93caa32fa20b87d6a15bce1e973f1a896 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 23:27:25 +0200 Subject: [PATCH 090/966] Parse and check a format in one call, keeping compileTemplate under the complexity gate --- node.go | 10 ++-------- template.go | 8 ++++++++ 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/node.go b/node.go index 2ba3ef3..cc84854 100644 --- a/node.go +++ b/node.go @@ -204,13 +204,10 @@ func jsonKind(v any) string { } func compileString(s string) (node, error) { - toks, err := parseFormat(s) + toks, err := parseChecked(s, nil) if err != nil { return nil, err } - if err := checkTokens(toks, nil); err != nil { - return nil, err - } t := &template{format: s, tokens: toks, repeat: 1, fromString: true} if err := t.compileRefFree(); err != nil { return nil, err @@ -336,13 +333,10 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" { return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format) } - toks, err := parseFormat(o.format) + toks, err := parseChecked(o.format, fields) if err != nil { return nil, err } - if err := checkTokens(toks, fields); err != nil { - return nil, err - } if err := checkNestedDrawGroup(fields, o.group); err != nil { return nil, err } diff --git a/template.go b/template.go index 5e9a863..e7e1d2e 100644 --- a/template.go +++ b/template.go @@ -185,6 +185,14 @@ func checkFunc(tok formatToken, fields map[string]node) error { return nil } +func parseChecked(format string, fields map[string]node) ([]formatToken, error) { + toks, err := parseFormat(format) + if err != nil { + return nil, err + } + return toks, checkTokens(toks, fields) +} + // checkTokens proves every parsed token names an existing field or a known function, // so a typo'd or dangling reference is a New-time error. func checkTokens(toks []formatToken, fields map[string]node) error { -- 2.52.0 From bb2bbe96979e359fdf4210d6a2fe84e493c9bc7a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 24 Sep 2026 23:33:05 +0200 Subject: [PATCH 091/966] Keep the transform's arity inside its operand func, skip calls in compileWhole's column check, and file the two tokens for renaming --- table.go | 5 ++++- template.go | 4 ++-- todo.md | 3 ++- transform.go | 3 +++ 4 files changed, 11 insertions(+), 4 deletions(-) diff --git a/table.go b/table.go index f61ebd7..f60d9cd 100644 --- a/table.go +++ b/table.go @@ -329,9 +329,12 @@ func (t *table) compileWhole(format string) error { return err } for _, tok := range toks { + if tok.kind != 'f' { + continue + } for _, name := range tok.names { a := splitArm(name, nil) - if _, ok := t.col[a.key]; tok.kind == 'f' && !ok && !isRef(a.key) && a.key != "" { + if _, ok := t.col[a.key]; !ok && !isRef(a.key) && a.key != "" { return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.columns) } } diff --git a/template.go b/template.go index e7e1d2e..b5f8a45 100644 --- a/template.go +++ b/template.go @@ -265,10 +265,10 @@ func hintableRef(name string) bool { } // builtinOperands lists the fields a call reads as operands, empty for a builtin that -// reads none, or a call checkFunc refuses. +// reads none or is unknown. func builtinOperands(name string, args []string) []string { b, known := builtins[name] - if !known || b.operands == nil || (b.arity >= 0 && len(args) != b.arity) { + if !known || b.operands == nil { return nil } return b.operands(args) diff --git a/todo.md b/todo.md index 684cec7..3bb0ae8 100644 --- a/todo.md +++ b/todo.md @@ -37,7 +37,8 @@ This round, in order: - Split `drawCheck.reads`'s six-term condition into named predicates. - Rename what names two things: `template.go` holds the token grammar while `template` is `node.go`'s type and `Template` `inline.go`'s; `kind` `'b'` is a brace body in - `ftoken` and a builtin in `op`; `drawAt.alt` holds the rows + `ftoken` and a builtin in `formatToken` and `op`; `tokens` is a template's parsed format + and a table's cell templates; `drawAt.alt` holds the rows a walk entered. - Run the nine-seat panel again and file what it names here. diff --git a/transform.go b/transform.go index 872917b..2b9fecd 100644 --- a/transform.go +++ b/transform.go @@ -45,6 +45,9 @@ func transformArg(fields map[string]node, a []string) error { return checkArm(leaf, fields, false) } func transformOperand(a []string) []string { + if len(a) != 1 { + return nil + } leaf, _, err := unwrapTransform(a[0]) if err != nil { return nil -- 2.52.0 From cc8acbc57bacb956833ecc5820dcdbed80e7e5fc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 07:56:09 +0200 Subject: [PATCH 092/966] Record the approval of the link-time fence decision --- docs/decisions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index e4f8df1..af50666 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -123,7 +123,7 @@ wording improves in a minor. ## A format holding a reference is fenced at link, and its errors name its path -2026-09-24, larv-review. Goal 2: a format compiles once, and one holding a reference +2026-09-24, larv-review; approved 2026-09-25 by lilleman. Goal 2: a format compiles once, and one holding a reference can compile only once bound, so its fences fire at link and name the category path as every link error does, a cell's with its line; one without a reference still names its file at compile. Valid while compilation needs the bound references. -- 2.52.0 From ed909dcf63badc8abea9ed437319c816903dad5c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:01:48 +0200 Subject: [PATCH 093/966] Want pathWalk's modes named, a cover stopping at a choice, and a draw of an unproved path panicking --- path_test.go | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/path_test.go b/path_test.go index 4a8ce24..2f7e058 100644 --- a/path_test.go +++ b/path_test.go @@ -1,6 +1,7 @@ package fejkdata import ( + "fmt" "slices" "strings" "testing" @@ -10,6 +11,8 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) { n := compiled(t, `{"format":"{a}","a":{"format":"{b}","b":"leaf"}}`) var seen []string walk := pathWalk{ + mode: walkEvery, + pins: &pinSet{}, level: func(tm *template, rest []string) error { seen = append(seen, "level:"+rest[0]); return nil }, leaf: func(n node) error { seen = append(seen, "leaf"); return nil }, } @@ -33,12 +36,8 @@ func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) var leaves []node _, err := walkPath(n, []string{"f"}, pathWalk{ - choice: func(c *choice, rest []string) ([]node, error) { - if len(rest) != 1 || rest[0] != "f" { - t.Errorf("choice saw rest %v, want [f]", rest) - } - return c.items, nil - }, + mode: walkEvery, + pins: &pinSet{}, leaf: func(n node) error { leaves = append(leaves, n); return nil }, }) if err != nil || len(leaves) != 2 { @@ -46,6 +45,27 @@ func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { } } +func TestWalkCoverStopsAtAChoice(t *testing.T) { + n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) + var leaves []node + _, err := walkPath(n, []string{"f"}, pathWalk{ + mode: walkCover, + leaf: func(n node) error { leaves = append(leaves, n); return nil }, + }) + if err != nil || len(leaves) != 1 || leaves[0] != n { + t.Fatalf("cover through a choice = %v, leaves %v, want the choice alone", err, leaves) + } +} + +func TestDrawPathPanicsOnAnUnprovedPath(t *testing.T) { + defer func() { + if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), `no field "f"`) { + t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the missing field", r) + } + }() + drawPath(compiled(t, `"plain"`), []string{"f"}, &pinSet{}, &pathDraws{s: engine(1).rand}) +} + func TestDeepDottedPath(t *testing.T) { // A 5-segment path descends through alternating object/array nodes; choices // on the path are single-variant, so it resolves deterministically. -- 2.52.0 From 5953c582248c4b130fa0d3352a48dc65d7cbb813 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:03:48 +0200 Subject: [PATCH 094/966] Name pathWalk's modes, and draw a path only through drawPath once it is proved --- family.go | 2 +- graph.go | 5 +-- hold.go | 6 +--- holdfence.go | 5 +-- path.go | 96 +++++++++++++++++++++++++++++----------------------- render.go | 4 +-- todo.md | 7 +--- 7 files changed, 61 insertions(+), 64 deletions(-) diff --git a/family.go b/family.go index 87ddc28..06675ea 100644 --- a/family.go +++ b/family.go @@ -33,7 +33,7 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { return nil } tr := &tableRead{head: t, drawn: map[*table]bool{}} - _, _ = walkPath(t, a.tail, pathWalk{pins: &tr.pins}) + _, _ = walkPath(t, a.tail, pathWalk{mode: walkProbe, pins: &tr.pins}) written := a.name[:len(a.name)-len(joinSegments(a.tail))] cur := t tr.draws(cur) diff --git a/graph.go b/graph.go index b6d8359..fc43333 100644 --- a/graph.go +++ b/graph.go @@ -165,10 +165,7 @@ func renderEdges(n node) []renderEdge { // already proved the tail resolves in every variant. func pathLeaves(n node, tail []string) []node { var out []node - _, _ = walkPath(n, tail, pathWalk{ - choice: func(c *choice, _ []string) ([]node, error) { return c.items, nil }, - leaf: func(n node) error { out = append(out, n); return nil }, - }) + _, _ = walkPath(n, tail, pathWalk{mode: walkEvery, pins: &pinSet{}, leaf: func(n node) error { out = append(out, n); return nil }}) return out } diff --git a/hold.go b/hold.go index 42c7d0f..2b579d4 100644 --- a/hold.go +++ b/hold.go @@ -37,11 +37,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if r, done := d.value[a.path]; done { return r } - leaf, err := walkPath(t.head(a.key), a.tail, pathWalk{pins: &d.pins, draws: &pathDraws{s: s, held: d, a: &a}}) - if err != nil { - panic(fmt.Sprintf("fejkdata: %q: %v; a fence should have refused this at New", a.name, err)) - } - r := renderLeaf(s, leaf, sc) + r := renderLeaf(s, drawPath(t.head(a.key), a.tail, &d.pins, &pathDraws{s: s, held: d, a: &a}), sc) if d.value == nil { d.value = map[string]draw{} } diff --git a/holdfence.go b/holdfence.go index 1ab1102..bef29f8 100644 --- a/holdfence.go +++ b/holdfence.go @@ -100,10 +100,7 @@ func operandReader(t *template, head string) string { // coverPath collects what holding one path pins: every choice level the path // passes through, whole, and the leaf it renders. func coverPath(n node, tail []string, into map[node]bool) { - _, _ = walkPath(n, tail, pathWalk{ - choice: func(c *choice, _ []string) ([]node, error) { cover(c, into, false); return nil, nil }, - leaf: func(n node) error { cover(n, into, false); return nil }, - }) + _, _ = walkPath(n, tail, pathWalk{mode: walkCover, leaf: func(n node) error { cover(n, into, false); return nil }}) } // cover collects a level and everything contained in it. A fixed string outside a diff --git a/path.go b/path.go index e80a60a..ec82450 100644 --- a/path.go +++ b/path.go @@ -125,21 +125,33 @@ func names(segs []string) []string { return out } -// pathWalk is what one walk of a dotted path does at each kind of level: choice -// returns the variants to continue into (none stops the walk), and where it is nil -// the walk draws one with draws, takes the first where draws is nil, or stops where -// pins is nil too; level runs at each template a segment descends -// into; leaf runs where the tail ends; pins is where the walk pins the table rows -// it selects and draws, nil skipping tables; draws is what draws the rows and -// variants the path reads, nil proving instead that the path resolves whichever -// are drawn. A nil action is skipped. A render's walk sets only pins and draws, so -// it allocates nothing. +// walkMode is what a walk does at a choice, and whether it reads the rows a path +// selects and draws. +type walkMode uint8 + +const ( + // walkCover stops at a choice, handing it whole to leaf, and reads no row. + walkCover walkMode = iota + // walkEvery proves every variant carries the rest of the path and walks each, + // pinning the rows it selects in pins. + walkEvery + // walkProbe proves as walkEvery does, walking one variant, so a path that + // resolves resolves whichever variants and rows are drawn. + walkProbe + // walkDraw draws the rows and variants a proved path reads; drawPath is its + // one entry. + walkDraw +) + +// pathWalk is one walk of a dotted path: level runs at each template a segment +// descends into, leaf where the walk ends, and a nil action is skipped. A render's +// walk sets only pins and draws, so it allocates nothing. type pathWalk struct { - choice func(c *choice, rest []string) ([]node, error) - level func(t *template, rest []string) error - leaf func(n node) error - pins *pinSet - draws *pathDraws + mode walkMode + level func(t *template, rest []string) error + leaf func(n node) error + pins *pinSet + draws *pathDraws } // pathDraws is what a walk draws with: the session, and, for a held read, the hold @@ -151,7 +163,7 @@ type pathDraws struct { } // walkPath descends tail from n and returns the node it ends at: a folder or -// template by its next segment, a choice by w.choice, which consumes no segment, a +// template by its next segment, a choice by its mode, which consumes no segment, a // table by walkTable. A missing segment is an error, so no walk reaches past what // the data holds. func walkPath(n node, tail []string, w pathWalk) (node, error) { @@ -197,27 +209,32 @@ func (w pathWalk) atLeaf(n node) error { return nil } -// walkChoice continues a walk into the variants w.choice returns, or into one -// w.draws draws where the walk names no choice action. +// drawPath draws the rows and variants a path reads, pinning them in pins. The path +// must be proved first, by a probe or by New's fences, so the walk cannot fail. +func drawPath(n node, tail []string, pins *pinSet, draws *pathDraws) node { + leaf, err := walkPath(n, tail, pathWalk{mode: walkDraw, pins: pins, draws: draws}) + if err != nil { + panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", joinSegments(tail), err)) + } + return leaf +} + func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { - if w.choice == nil { - if w.pins == nil { - return nil, nil - } - if w.draws == nil { // a probe: every variant carries the tail, so any one proves it - if err := carriedByAll(c, tail); err != nil { - return nil, err - } - return walkPath(c.items[0], tail, w) - } + switch w.mode { + case walkCover: + return c, w.atLeaf(c) + case walkDraw: return walkPath(w.draws.variant(c, tail), tail, w) } - next, err := w.choice(c, tail) - if err != nil { + if err := carriedByAll(c, tail); err != nil { return nil, err } + if w.mode == walkProbe { + return walkPath(c.items[0], tail, w) + } var last node - for _, item := range next { + for _, item := range c.items { + var err error if last, err = walkPath(item, tail, w); err != nil { return nil, err } @@ -240,8 +257,8 @@ func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error } // A selector further down pins this table by ancestry, so the walk draws only // where none follows; drawing first could pick a row the selector is not inside. - if w.pins != nil { - if err := readRow(w.pins, w.draws, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { + if w.mode != walkCover { + if err := readRow(w, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { return nil, err } } @@ -284,12 +301,12 @@ func (t *table) step(tail []string) (column node, child *table, err error) { // readRow pins the row a path reads of t: the one its selector names, or, where the // walk draws, one drawn where the path reads into the table. -func readRow(p *pinSet, draws *pathDraws, t *table, sel string, draw bool) error { +func readRow(w pathWalk, t *table, sel string, draw bool) error { if sel != "" { - return p.selectRow(t, sel) + return w.pins.selectRow(t, sel) } - if draw && draws != nil { - t.drawIn(draws.s, p) + if draw && w.mode == walkDraw { + t.drawIn(w.draws.s, w.pins) } return nil } @@ -344,12 +361,7 @@ func unreachableInChoice(c *choice, want string) error { func checkPath(n node, tail []string, level string) error { var pins pinSet _, err := walkPath(n, tail, pathWalk{ - choice: func(c *choice, rest []string) ([]node, error) { - if err := carriedByAll(c, rest); err != nil { - return nil, err - } - return c.items, nil - }, + mode: walkEvery, level: func(t *template, rest []string) error { name := join(level, strings.Join(tail[:len(tail)-len(rest)], ".")) switch { diff --git a/render.go b/render.go index b0787ff..ef02a75 100644 --- a/render.go +++ b/render.go @@ -45,10 +45,10 @@ func (f *Generator) Fake(path string) (string, error) { func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // Walked once without drawing first, so a path that fails moves no seeded stream. var probe pinSet - if _, err := walkPath(root, segments, pathWalk{pins: &probe}); err != nil { + if _, err := walkPath(root, segments, pathWalk{mode: walkProbe, pins: &probe}); err != nil { return nil, err } - return walkPath(root, segments, pathWalk{pins: &sc.hold().pins, draws: &pathDraws{s: s}}) + return drawPath(root, segments, &sc.hold().pins, &pathDraws{s: s}), nil } // render evaluates a compiled node to a string. compile validates every node up diff --git a/todo.md b/todo.md index 3bb0ae8..608460f 100644 --- a/todo.md +++ b/todo.md @@ -19,17 +19,12 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Name `pathWalk`'s modes: render, probe, check and cover are chosen today by which - callbacks are nil and whether `draws` is, and `walkChoice` returns `nil, nil` where - none is set. Make the render mode reachable only through a proved path: a walk that - draws no longer checks `carriedByAll`, trusting its caller's probe or `New`'s fences. - Have the check mode report the tables a read draws, so `tableRead.drawn` and `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and `table.drawIn`'s ancestor rule: a change to how a render draws rows still leaves the fence proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Settle a - cell of a municipality drawn whole beside a path pinning a region it is not in. Needs - the item above. + cell of a municipality drawn whole beside a path pinning a region it is not in. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while render calls the second. Derive the numeric builtins `valueProof.template` lists in -- 2.52.0 From 8f904ddd7409916fd337e14538085d016367a251 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:07:04 +0200 Subject: [PATCH 095/966] Want drawPath's panic to name the path as written --- path_test.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/path_test.go b/path_test.go index 2f7e058..805551f 100644 --- a/path_test.go +++ b/path_test.go @@ -59,11 +59,11 @@ func TestWalkCoverStopsAtAChoice(t *testing.T) { func TestDrawPathPanicsOnAnUnprovedPath(t *testing.T) { defer func() { - if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), `no field "f"`) { - t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the missing field", r) + if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), `plain.f: no field "f"`) { + t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the path and the missing field", r) } }() - drawPath(compiled(t, `"plain"`), []string{"f"}, &pinSet{}, &pathDraws{s: engine(1).rand}) + drawPath(compiled(t, `"plain"`), []string{"f"}, "plain.f", &pinSet{}, &pathDraws{s: engine(1).rand}) } func TestDeepDottedPath(t *testing.T) { -- 2.52.0 From eecfb7ab54a4cf6c529243c2d5c4e714256b6095 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:08:12 +0200 Subject: [PATCH 096/966] Pass drawPath the head its tail descends from --- path_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/path_test.go b/path_test.go index 805551f..8f1065d 100644 --- a/path_test.go +++ b/path_test.go @@ -63,7 +63,7 @@ func TestDrawPathPanicsOnAnUnprovedPath(t *testing.T) { t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the path and the missing field", r) } }() - drawPath(compiled(t, `"plain"`), []string{"f"}, "plain.f", &pinSet{}, &pathDraws{s: engine(1).rand}) + drawPath(compiled(t, `"plain"`), []string{"f"}, "plain", &pinSet{}, &pathDraws{s: engine(1).rand}) } func TestDeepDottedPath(t *testing.T) { -- 2.52.0 From c63b1c74dd382999cc1b9d08f0a6ac9a74dc2b49 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:08:43 +0200 Subject: [PATCH 097/966] Make walkEvery the zero mode, and have drawPath's panic name the head its tail descends from --- hold.go | 2 +- path.go | 14 +++++++------- render.go | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/hold.go b/hold.go index 2b579d4..7d7ccd7 100644 --- a/hold.go +++ b/hold.go @@ -37,7 +37,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if r, done := d.value[a.path]; done { return r } - r := renderLeaf(s, drawPath(t.head(a.key), a.tail, &d.pins, &pathDraws{s: s, held: d, a: &a}), sc) + r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &d.pins, &pathDraws{s: s, held: d, a: &a}), sc) if d.value == nil { d.value = map[string]draw{} } diff --git a/path.go b/path.go index ec82450..73da47e 100644 --- a/path.go +++ b/path.go @@ -130,11 +130,11 @@ func names(segs []string) []string { type walkMode uint8 const ( - // walkCover stops at a choice, handing it whole to leaf, and reads no row. - walkCover walkMode = iota // walkEvery proves every variant carries the rest of the path and walks each, // pinning the rows it selects in pins. - walkEvery + walkEvery walkMode = iota + // walkCover stops at a choice, handing it whole to leaf, and reads no row. + walkCover // walkProbe proves as walkEvery does, walking one variant, so a path that // resolves resolves whichever variants and rows are drawn. walkProbe @@ -144,8 +144,7 @@ const ( ) // pathWalk is one walk of a dotted path: level runs at each template a segment -// descends into, leaf where the walk ends, and a nil action is skipped. A render's -// walk sets only pins and draws, so it allocates nothing. +// descends into, leaf where the walk ends, and a nil action is skipped. type pathWalk struct { mode walkMode level func(t *template, rest []string) error @@ -211,10 +210,11 @@ func (w pathWalk) atLeaf(n node) error { // drawPath draws the rows and variants a path reads, pinning them in pins. The path // must be proved first, by a probe or by New's fences, so the walk cannot fail. -func drawPath(n node, tail []string, pins *pinSet, draws *pathDraws) node { +// head is what n is reached by, for the panic. +func drawPath(n node, tail []string, head string, pins *pinSet, draws *pathDraws) node { leaf, err := walkPath(n, tail, pathWalk{mode: walkDraw, pins: pins, draws: draws}) if err != nil { - panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", joinSegments(tail), err)) + panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(head, joinSegments(tail)), err)) } return leaf } diff --git a/render.go b/render.go index ef02a75..be0194d 100644 --- a/render.go +++ b/render.go @@ -48,7 +48,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er if _, err := walkPath(root, segments, pathWalk{mode: walkProbe, pins: &probe}); err != nil { return nil, err } - return drawPath(root, segments, &sc.hold().pins, &pathDraws{s: s}), nil + return drawPath(root, segments, "", &sc.hold().pins, &pathDraws{s: s}), nil } // render evaluates a compiled node to a string. compile validates every node up -- 2.52.0 From bf0f4e9d1fbbf40732b83b53afaf0201c468b9e7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:22:57 +0200 Subject: [PATCH 098/966] Want the probe to report the tables a draw pins, and a row rendered whole held apart from the pins --- table_test.go | 76 +++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 65 insertions(+), 11 deletions(-) diff --git a/table_test.go b/table_test.go index f7ddb13..4e5bf74 100644 --- a/table_test.go +++ b/table_test.go @@ -1,6 +1,7 @@ package fejkdata import ( + "maps" "os" "path/filepath" "reflect" @@ -967,7 +968,8 @@ func TestAmbiguousNameNamesARunnablePath(t *testing.T) { // TestEnteredRowsAgreeWithPinning holds what a fence walk decides about the rows it entered, and // the rows a render draws, to what pinning refuses: drift between them is data that loads and then -// renders a family that disagrees. +// renders a family that disagrees. A row rendered whole is drawn over the whole table, apart from +// every pin, so it is an alternative only to another row of its own draw. func TestEnteredRowsAgreeWithPinning(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) var tables []*table @@ -981,13 +983,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rows(); row++ { - var drawn []tablePin - alone := none.entered(entered, row, false) - alone.each(func(tbl *table, r int) { drawn = append(drawn, tablePin{tbl, r}) }) - if len(drawn) != 1 || drawn[0] != (tablePin{entered, row}) { - t.Errorf("a drawn %s enters %v, want its own row alone", entered.selectorSpelling(row), drawn) - } - s := none.entered(entered, row, true) + whole := drawAt{drawn: entered, whole: tablePin{entered, row}} + s := none.entered(entered, row) for _, cand := range tables { for i := 0; i < 20; i++ { drawing := s.clone() @@ -998,18 +995,26 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for cr := 0; cr < cand.rows(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil - if got := alternatives(drawAt{alt: s}, drawAt{alt: none.entered(cand, cr, true)}); got != refused { + pinned := drawAt{alt: none.entered(cand, cr)} + if got := alternatives(drawAt{alt: s}, pinned); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } + if alternatives(whole, pinned) || alternatives(pinned, whole) { + t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) + } + other := drawAt{drawn: cand, whole: tablePin{cand, cr}} + if want := cand == entered && cr != row; alternatives(whole, other) != want { + t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) + } } } } } var spilled pinSet for i := 0; i <= len(spilled.inline); i++ { - spilled = spilled.entered(&table{path: strconv.Itoa(i)}, 0, false) + spilled = spilled.entered(&table{path: strconv.Itoa(i)}, 0) } - _ = spilled.entered(&table{path: "past"}, 0, false) + _ = spilled.entered(&table{path: "past"}, 0) n := 0 spilled.each(func(*table, int) { n++ }) if n != len(spilled.inline)+1 { @@ -1017,6 +1022,55 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { } } +// TestProbeReportsTheTablesADrawPins holds the check mode to the render: the tables a probe says a +// path draws are those a draw of it pins beyond what its selectors pin. +func TestProbeReportsTheTablesADrawPins(t *testing.T) { + f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) + for _, path := range []string{ + "locality.code", + "municipality.locality.name", + "municipality[1281].locality.name", + "municipality.locality[L4].name", + "region", + "region.municipality.name", + "region[12].municipality.locality.code", + "region[12].name", + } { + segs, err := splitPath(path) + if err != nil { + t.Fatal(err) + } + head := f.categories[segs[0]].(*table) + var probed pinSet + drawn := map[*table]bool{} + if _, err := walkPath(head, segs[1:], pathWalk{mode: walkProbe, pins: &probed, drawn: drawn}); err != nil { + t.Fatalf("%s: %v", path, err) + } + for i := 0; i < 20; i++ { + var pins pinSet + drawPath(head, segs[1:], segs[0], &pins, &pathDraws{s: f.rand}) + want := map[*table]bool{} + pins.each(func(tbl *table, _ int) { + if _, selected := probed.pinned(tbl); !selected { + want[tbl] = true + } + }) + if !maps.Equal(drawn, want) { + t.Fatalf("%s: the probe reports %v drawn, a draw pins %v", path, categoriesOf(drawn), categoriesOf(want)) + } + } + } +} + +func categoriesOf(tables map[*table]bool) []string { + var out []string + for tbl := range tables { + out = append(out, tbl.category) + } + slices.Sort(out) + return out +} + // TestCellReadsMeetWhereTheRenderPairsTheRows reads what the TSVs below do not show: region 01's // note holds {/addr} and 1280's, under region 12, holds {/addr.city}, so the two meet only where // no read pins a region. -- 2.52.0 From 4a016c119cb183477f7bd0a1d24830f2aab3fd4a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:25:13 +0200 Subject: [PATCH 099/966] Have the probe report the tables a read draws, and hold a row rendered whole in drawAt apart from the pins --- docs/decisions.md | 4 +++- drawfence.go | 35 ++++++++++++++++++++++++----------- family.go | 21 ++++++--------------- path.go | 19 +++++++++++++------ pins.go | 37 ++++++++++++++++++------------------- todo.md | 6 ------ 6 files changed, 64 insertions(+), 58 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index af50666..edbb4bf 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -412,7 +412,9 @@ another never meet, and each may select its own row of another table; the cells row, and whatever they reach, do meet, and so does the format beside them. The rule the fence applies is that two reads meet unless they sit in rows that never render together: rows falling under different rows of a table a path pinned, two rows of that table -itself included. A choice's items get no such treatment yet: two items selecting +itself included. A table read whole draws its row apart from every pin, so its rows +are alternatives only to each other, and a cell of one meets a read pinning a region +the row is not in. A choice's items get no such treatment yet: two items selecting different rows are still refused. ## A table never reaches its own family, by any route diff --git a/drawfence.go b/drawfence.go index fb1e3f5..435ff69 100644 --- a/drawfence.go +++ b/drawfence.go @@ -179,12 +179,14 @@ type drawWalk struct { } // drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the -// table rows it entered, and the table a whole read draws a row of. +// table rows it pinned, the table a whole read draws a row of, and the row of it whose cells the +// walk is in. type drawAt struct { group string route drawRoute alt pinSet drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true + whole tablePin } // drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. @@ -216,7 +218,7 @@ func newDrawWalk() *drawWalk { // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops // there. func (w *drawWalk) walk(n node, at drawAt) { - v := nodeVisit{n, at.group, at.alt.key()} + v := nodeVisit{n, at.group, at.rowsKey()} if w.seen[v] { return } @@ -231,30 +233,41 @@ func (w *drawWalk) walk(n node, at drawAt) { at.drawn = t } for _, e := range renderEdges(n) { - if cell, isCell := e.to.(*template); isCell && cell.cellOf != nil { - if at.alt.clash(cell.cellOf, cell.cellRow) != nil { - continue - } + cell, isCell := e.to.(*template) + switch { + case !isCell || cell.cellOf == nil: + w.edge(n, e, at) + case cell.cellOf == at.drawn: in := at - in.alt = at.alt.entered(cell.cellOf, cell.cellRow, cell.cellOf != at.drawn) + in.whole = tablePin{cell.cellOf, cell.cellRow} + w.edge(n, e, in) + case at.alt.clash(cell.cellOf, cell.cellRow) == nil: + in := at + in.alt = at.alt.entered(cell.cellOf, cell.cellRow) w.edge(n, e, in) - continue } - w.edge(n, e, at) } } +// rowsKey spells the rows the walk stands in, pinned and whole, for a map. +func (at drawAt) rowsKey() string { + if at.whole.t == nil { + return at.alt.key() + } + return fmt.Sprintf("%s%p{%d}", at.alt.key(), at.whole.t, at.whole.row) +} + // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { if a, reads := refRead(from, e.label); reads { tr := tableReadOf(from.(*template).head(a.key), a, e.to) - if k := (readKey{at.group, a.path, at.alt.key()}); !w.read[k] { + if k := (readKey{at.group, a.path, at.rowsKey()}); !w.read[k] { w.read[k] = true w.reads = append(w.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { at.alt = at.alt.entered(t, r, true) }) + tr.pins.each(func(t *table, r int) { at.alt = at.alt.entered(t, r) }) } } w.walk(e.to, at) diff --git a/family.go b/family.go index 06675ea..1123871 100644 --- a/family.go +++ b/family.go @@ -33,33 +33,21 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { return nil } tr := &tableRead{head: t, drawn: map[*table]bool{}} - _, _ = walkPath(t, a.tail, pathWalk{mode: walkProbe, pins: &tr.pins}) + _, _ = walkPath(t, a.tail, pathWalk{mode: walkProbe, pins: &tr.pins, drawn: tr.drawn}) written := a.name[:len(a.name)-len(joinSegments(a.tail))] cur := t - tr.draws(cur) for i, seg := range a.tail { switch d := cur.descendant(seg); { case isSelector(seg): tr.sels = append(tr.sels, tableSel{cur, written + joinSegments(a.tail[:i+1])}) case d != nil: cur = d - tr.draws(cur) } } _, tr.whole = leaf.(*row) return tr } -// draws marks t and its ancestors drawn, up to the first the read pins. -func (r *tableRead) draws(t *table) { - for ; t != nil; t = t.parentT { - if _, pinned := r.pins.pinned(t); pinned { - return - } - r.drawn[t] = true - } -} - // joinSegments spells segments as a path: a selector attaches to the name before it. func joinSegments(segs []string) string { var b strings.Builder @@ -228,10 +216,13 @@ func checkOwnFamily(t *template) error { return nil } -// alternatives reports whether two reads sit in different rows of one table. Not clash: a row drawn -// whole is held without its ancestors, since the render draws it over the whole table. +// alternatives reports whether two reads sit in different rows of one table: rows the walks pinned, +// or rows of one whole draw. // docs/decisions.md#the-rows-of-a-table-are-alternatives func alternatives(a, b drawAt) bool { + if a.whole.t != nil && a.whole.t == b.whole.t && a.whole.row != b.whole.row { + return true + } found := false a.alt.each(func(t *table, r int) { if br, in := b.alt.pinned(t); in && br != r { diff --git a/path.go b/path.go index 73da47e..bedd88e 100644 --- a/path.go +++ b/path.go @@ -136,7 +136,8 @@ const ( // walkCover stops at a choice, handing it whole to leaf, and reads no row. walkCover // walkProbe proves as walkEvery does, walking one variant, so a path that - // resolves resolves whichever variants and rows are drawn. + // resolves resolves whichever variants and rows are drawn, and marks in drawn + // the tables a draw would pin. walkProbe // walkDraw draws the rows and variants a proved path reads; drawPath is its // one entry. @@ -150,6 +151,7 @@ type pathWalk struct { level func(t *template, rest []string) error leaf func(n node) error pins *pinSet + drawn map[*table]bool draws *pathDraws } @@ -299,14 +301,19 @@ func (t *table) step(tail []string) (column node, child *table, err error) { return nil, child, nil } -// readRow pins the row a path reads of t: the one its selector names, or, where the -// walk draws, one drawn where the path reads into the table. +// readRow pins the row a path reads of t: the one its selector names, or one drawn +// where the path reads into the table. func readRow(w pathWalk, t *table, sel string, draw bool) error { - if sel != "" { + switch { + case sel != "": return w.pins.selectRow(t, sel) - } - if draw && w.mode == walkDraw { + case !draw: + case w.mode == walkDraw: t.drawIn(w.draws.s, w.pins) + case w.drawn != nil: + for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { + w.drawn[t] = true + } } return nil } diff --git a/pins.go b/pins.go index 3badda9..f6acc46 100644 --- a/pins.go +++ b/pins.go @@ -12,8 +12,7 @@ type tablePin struct { row int } -// pinSet is the table rows fixed so far. A fence walk's set also holds a row rendered whole alone, -// which pin never does. +// pinSet is the table rows fixed so far. type pinSet struct { inline [8]tablePin // sized so a render over a country's five-deep geo tree stays off the heap n int @@ -39,7 +38,6 @@ func (p *pinSet) mustRow(t *table) int { return r } -// add pins row r of t alone. func (p *pinSet) add(t *table, r int) { if p.n < len(p.inline) { p.inline[p.n] = tablePin{t, r} @@ -54,18 +52,25 @@ func (p *pinSet) add(t *table, r int) { // pin pins row r of t, and the rows of t's ancestors it links to. func (p *pinSet) pin(t *table, r int) { - for { - if _, done := p.pinned(t); done { - return - } + for stop := p.nearestPinned(t); t != stop; t = t.parentT { p.add(t, r) - if t.parentT == nil { - return + if t.parentT != stop { + r = t.parentRow(r) } - t, r = t.parentT, t.parentRow(r) } } +// nearestPinned is the first of t and its ancestors p pins, where pinning a row of t stops; nil +// where none is. +func (p *pinSet) nearestPinned(t *table) *table { + for ; t != nil; t = t.parentT { + if _, ok := p.pinned(t); ok { + return t + } + } + return nil +} + // each calls fn for every pinned row, in pin order, the spilled ones by path. func (p *pinSet) each(fn func(t *table, r int)) { for _, q := range p.inline[:p.n] { @@ -141,16 +146,10 @@ func (p pinSet) clone() pinSet { return p } -// entered is p with row r of t: pinned where the render pins it, alone where the render draws it -// to render whole. p is left as it was, since a fence walk branches. -func (p pinSet) entered(t *table, r int, pinsAncestors bool) pinSet { +// entered is p with row r of t pinned. p is left as it was, since a fence walk branches. +func (p pinSet) entered(t *table, r int) pinSet { p = p.clone() - switch _, in := p.pinned(t); { - case pinsAncestors: - p.pin(t, r) - case !in: - p.add(t, r) - } + p.pin(t, r) return p } diff --git a/todo.md b/todo.md index 608460f..6abde67 100644 --- a/todo.md +++ b/todo.md @@ -19,12 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Have the check mode report the tables a read draws, so `tableRead.drawn` and - `entered`'s `pinsAncestors` flag stop restating `walkTable`'s draw condition and - `table.drawIn`'s ancestor rule: a change to how a render draws rows still leaves the - fence proving the old rule. Hold a row rendered whole in `drawAt` beside `drawn`, so - `pinSet` holds only what `pin` pins and `alternatives` needs no exception. Settle a - cell of a municipality drawn whole beside a path pinning a region it is not in. - Register a transform once: `transforms` restates the `lowercase`, `uppercase` and `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while render calls the second. Derive the numeric builtins `valueProof.template` lists in -- 2.52.0 From b0889faa1474b90cbf273ca60a1a0b43853f776e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:30:45 +0200 Subject: [PATCH 100/966] Want a whole draw nested in another kept apart from the outer draw's other rows, and the probe held over skip-level paths --- table_test.go | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/table_test.go b/table_test.go index 4e5bf74..8097a17 100644 --- a/table_test.go +++ b/table_test.go @@ -983,7 +983,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rows(); row++ { - whole := drawAt{drawn: entered, whole: tablePin{entered, row}} + whole := drawAt{drawn: entered} + whole.whole.add(entered, row) s := none.entered(entered, row) for _, cand := range tables { for i := 0; i < 20; i++ { @@ -1002,7 +1003,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { if alternatives(whole, pinned) || alternatives(pinned, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } - other := drawAt{drawn: cand, whole: tablePin{cand, cr}} + other := drawAt{drawn: cand} + other.whole.add(cand, cr) if want := cand == entered && cr != row; alternatives(whole, other) != want { t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) } @@ -1033,6 +1035,8 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { "municipality.locality[L4].name", "region", "region.municipality.name", + "region.locality.name", + "region[12].locality.code", "region[12].municipality.locality.code", "region[12].name", } { @@ -1094,4 +1098,14 @@ func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "reads a path into") { t.Fatalf("New = %v, want the two drawn rows held to one draw of addr", err) } + nested := with(whole, map[string]string{ + "municipality.tsv": "code\tname\tregion\tpopulation\tnote\n0180\tStockholm\t01\t980000\t-\n0184\tSolna\t01\t85000\t-\n1280\tMalmö\t12\t360000\t{/shop}\n1281\tLund\t12\t130000\t{/addr.city}\n1480\tGöteborg\t14\t590000\t-\n", + "region.tsv": "code\tname\tpopulation\tnote\n01\tStockholms län\t2400000\t-\n12\tSkåne län\t1400000\t-\n14\tVästra Götalands län\t1750000\t-\n", + "shop.json": `{"format":"{name}={note}","rows":"shop.tsv","key":"code","name":"name"}`, + "shop.tsv": "code\tname\tnote\nS1\tKiosk\t{/addr}\nS2\tBod\t-\n", + "x.json": `"{/municipality}"`, + }) + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, nested)), WithSeed(1)); err != nil { + t.Fatalf("New = %v, want 1280's shop and 1281's city kept apart, one draw of municipality rendering one of them", err) + } } -- 2.52.0 From 3500f8e1c30a140f5380bc4966024bab6df25dbf Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:31:25 +0200 Subject: [PATCH 101/966] Hold every whole draw's row a walk is in, so a nested whole draw keeps the outer one's --- docs/decisions.md | 6 +++--- drawfence.go | 16 ++++++---------- family.go | 13 +------------ pins.go | 11 +++++++++++ 4 files changed, 21 insertions(+), 25 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index edbb4bf..f7d0996 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -412,9 +412,9 @@ another never meet, and each may select its own row of another table; the cells row, and whatever they reach, do meet, and so does the format beside them. The rule the fence applies is that two reads meet unless they sit in rows that never render together: rows falling under different rows of a table a path pinned, two rows of that table -itself included. A table read whole draws its row apart from every pin, so its rows -are alternatives only to each other, and a cell of one meets a read pinning a region -the row is not in. A choice's items get no such treatment yet: two items selecting +itself included. A table read whole draws its row apart from every pin, so the rows of +one whole read are alternatives only to each other, and a cell of one meets a read +pinning an ancestor row the row is not under. A choice's items get no such treatment yet: two items selecting different rows are still refused. ## A table never reaches its own family, by any route diff --git a/drawfence.go b/drawfence.go index 435ff69..ec40eda 100644 --- a/drawfence.go +++ b/drawfence.go @@ -179,14 +179,14 @@ type drawWalk struct { } // drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the -// table rows it pinned, the table a whole read draws a row of, and the row of it whose cells the -// walk is in. +// table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose +// cells the walk is in. type drawAt struct { group string route drawRoute alt pinSet drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true - whole tablePin + whole pinSet } // drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. @@ -239,7 +239,8 @@ func (w *drawWalk) walk(n node, at drawAt) { w.edge(n, e, at) case cell.cellOf == at.drawn: in := at - in.whole = tablePin{cell.cellOf, cell.cellRow} + in.whole = at.whole.clone() + in.whole.add(cell.cellOf, cell.cellRow) w.edge(n, e, in) case at.alt.clash(cell.cellOf, cell.cellRow) == nil: in := at @@ -250,12 +251,7 @@ func (w *drawWalk) walk(n node, at drawAt) { } // rowsKey spells the rows the walk stands in, pinned and whole, for a map. -func (at drawAt) rowsKey() string { - if at.whole.t == nil { - return at.alt.key() - } - return fmt.Sprintf("%s%p{%d}", at.alt.key(), at.whole.t, at.whole.row) -} +func (at drawAt) rowsKey() string { return at.alt.key() + "|" + at.whole.key() } // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. diff --git a/family.go b/family.go index 1123871..55646fb 100644 --- a/family.go +++ b/family.go @@ -219,15 +219,4 @@ func checkOwnFamily(t *template) error { // alternatives reports whether two reads sit in different rows of one table: rows the walks pinned, // or rows of one whole draw. // docs/decisions.md#the-rows-of-a-table-are-alternatives -func alternatives(a, b drawAt) bool { - if a.whole.t != nil && a.whole.t == b.whole.t && a.whole.row != b.whole.row { - return true - } - found := false - a.alt.each(func(t *table, r int) { - if br, in := b.alt.pinned(t); in && br != r { - found = true - } - }) - return found -} +func alternatives(a, b drawAt) bool { return a.alt.differs(&b.alt) || a.whole.differs(&b.whole) } diff --git a/pins.go b/pins.go index f6acc46..4b5e668 100644 --- a/pins.go +++ b/pins.go @@ -153,6 +153,17 @@ func (p pinSet) entered(t *table, r int) pinSet { return p } +// differs reports whether p and q hold different rows of one table. +func (p *pinSet) differs(q *pinSet) bool { + found := false + p.each(func(t *table, r int) { + if qr, in := q.pinned(t); in && qr != r { + found = true + } + }) + return found +} + // key spells the set for a map, by the tables' identities in path order. func (p *pinSet) key() string { var pins []tablePin -- 2.52.0 From 57966b59eed37f2fab0fefff35af3ef87e379814 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:34:29 +0200 Subject: [PATCH 102/966] Say what makes rows alternatives without history, and drop what the whole-draw rule already implies --- docs/decisions.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index f7d0996..49baf2e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -409,13 +409,12 @@ shared. Only one row renders per read, so within one read a cell in one row and a cell in another never meet, and each may select its own row of another table; the cells of one -row, and whatever they reach, do meet, and so does the format beside them. The rule the -fence applies is that two reads meet unless they sit in rows that never render together: -rows falling under different rows of a table a path pinned, two rows of that table -itself included. A table read whole draws its row apart from every pin, so the rows of -one whole read are alternatives only to each other, and a cell of one meets a read -pinning an ancestor row the row is not under. A choice's items get no such treatment yet: two items selecting -different rows are still refused. +row, and whatever they reach, do meet, and so does the format beside them. Two reads +meet unless they sit under different rows of one pinned table; pinning a row pins its +ancestors too, so rows under different parents are alternatives as well. A table read +whole draws its row apart from every pin, so the rows of one whole read are +alternatives only to each other. A choice's items are not alternatives: two items +selecting different rows of one family are refused. ## A table never reaches its own family, by any route -- 2.52.0 From a048cdd73c4e84460745337292da3baa96870409 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:42:58 +0200 Subject: [PATCH 103/966] Want the calls a typed column's error offers listed in name order --- datatype_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/datatype_test.go b/datatype_test.go index 4ccb06c..61143b7 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -61,7 +61,7 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { {"a fraction", `{"format":"{v}","v":["1","1.5"],"datatype":"integer"}`, `"1.5" is not an integer`}, {"past int64", `{"format":"9223372036854775808","datatype":"integer"}`, "past the int64 range"}, {"a float sample", `{"format":"{float(0,1,2)}","datatype":"integer"}`, "{float(0,1,2)} prints a number, not an integer"}, - {"composed digits", `{"format":"1{digits(2)}","datatype":"integer"}`, "is not one value"}, + {"composed digits", `{"format":"1{digits(2)}","datatype":"integer"}`, "is not one value; write one literal or one {float()}, {int()}, {seq()} or {calc()}, or read one"}, {"a sign before a sample", `{"format":"-{int(1,9)}","datatype":"integer"}`, "is not one value"}, {"a repeat", `{"format":"{int(1,9)}","repeat":2,"separator":",","datatype":"integer"}`, "carries a repeat"}, {"through a reference", `{"format":"{/cat.code}","datatype":"integer"}`, `"2x" is not an integer`}, @@ -75,7 +75,7 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { {"a divisor that can be zero", `{"format":"{calc(a / b)}","a":"{int(1,9)}","b":"{int(0,9)}","datatype":"number"}`, "divides by b, which is not proven nonzero"}, {"an overflow", `{"format":"{calc(a * a)}","a":"{digits(200)}","datatype":"number"}`, "is not proven within 1e300"}, {"a division in an integer column", `{"format":"{calc(a / b)}","a":"{int(1,9)}","b":"{int(1,9)}","datatype":"integer"}`, "prints a number, not an integer"}, - {"a composed operand", `{"format":"{calc(n * 2)}","n":"{int(1,99)}.{digits(2)}","datatype":"number"}`, "{seq()}, {digits()} or {calc()}"}, + {"a composed operand", `{"format":"{calc(n * 2)}","n":"{int(1,99)}.{digits(2)}","datatype":"number"}`, "one {digits()}, {float()}, {int()}, {seq()} or {calc()}"}, {"a divisor that prints as zero", `{"format":"{calc(1 / b, 2)}","b":"{float(4.9999999999999994e-79,5e-79,78)}","datatype":"number"}`, "divides by b, which is not proven nonzero"}, {"a divisor that rounds to zero", `{"format":"{calc(a / b)}","a":"{int(1,9)}","b":"{float(0.1,1,0)}","datatype":"number"}`, "divides by b, which is not proven nonzero"}, {"a zero among a divisor's literals", `{"format":"{calc(a / b)}","a":"{int(1,9)}","b":["0","5"],"datatype":"number"}`, "divides by b, which is not proven nonzero"}, -- 2.52.0 From d81cfd575748f0e6ffafd278ae557cb63f9a729b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:43:58 +0200 Subject: [PATCH 104/966] Register a transform once, and derive the calls a typed column's error offers from builtin.number --- README.md | 2 +- builtins.go | 27 ++++++++++++--------------- template.go | 5 +++-- transform.go | 7 +++++++ value.go | 20 +++++++++++++++++--- 5 files changed, 40 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 08269f1..66dbb2d 100644 --- a/README.md +++ b/README.md @@ -413,7 +413,7 @@ datatype cannot hold is a load error naming it: ```text order.id: datatype integer: {digits(3)} prints text, not an integer -order.id: datatype integer: "1{digits(2)}" is not one value; write one literal or one {int()}, {float()}, {seq()} or {calc()}, or read one +order.id: datatype integer: "1{digits(2)}" is not one value; write one literal or one {float()}, {int()}, {seq()} or {calc()}, or read one ``` A column of one reference alone to another record's column — `"score": "{/src.score}"` diff --git a/builtins.go b/builtins.go index c6d6c0d..d960701 100644 --- a/builtins.go +++ b/builtins.go @@ -22,7 +22,7 @@ const ( // emitted so far in the current expansion (place them after their payload); // samples read only the rng. A time-based id (uuid v7, ulid) draws its timestamp // from the rng, not the wall clock, so seeded output stays reproducible. -var builtins = map[string]builtin{ +var builtins = withTransforms(map[string]builtin{ "luhn": {arity: 0, prep: derive(func(e string) string { return string(rune('0' + luhnCheck(e))) })}, "mod11": {arity: 0, prep: derive(mod11Check)}, "ean": {arity: 0, prep: derive(eanCheck)}, @@ -30,8 +30,8 @@ var builtins = map[string]builtin{ "ulid": {arity: 0, prep: sample(ulid)}, "nanoid": {arity: 1, check: posIntArg, prep: chars(nanoidAlphabet)}, "hex": {arity: 1, check: posIntArg, prep: chars(hexDigits)}, - "digits": {arity: 1, check: posIntArg, prep: chars("0123456789"), number: func(token string, a []string) proven { - return printing(token, DataTypeString, bounded(0, math.Pow(10, float64(atoi(a[0])))-1, true)) + "digits": {arity: 1, check: posIntArg, prep: chars("0123456789"), prints: DataTypeString, number: func(token string, prints DataType, a []string) proven { + return printing(token, prints, bounded(0, math.Pow(10, float64(atoi(a[0])))-1, true)) }}, "upper": {arity: 1, check: posIntArg, prep: chars("ABCDEFGHIJKLMNOPQRSTUVWXYZ")}, "lower": {arity: 1, check: posIntArg, prep: chars("abcdefghijklmnopqrstuvwxyz")}, @@ -44,27 +44,24 @@ var builtins = map[string]builtin{ "int": {arity: 2, check: intRangeArgs, prep: func(a []string) callFn { lo, span := atoi(a[0]), atoi(a[1])-atoi(a[0])+1 return func(s *session, _ string, _ []string) string { return strconv.Itoa(lo + s.IntN(span)) } - }, number: func(token string, a []string) proven { - return printing(token, DataTypeInteger, bounded(float64(atoi(a[0])), float64(atoi(a[1])), true)) + }, prints: DataTypeInteger, number: func(token string, prints DataType, a []string) proven { + return printing(token, prints, bounded(float64(atoi(a[0])), float64(atoi(a[1])), true)) }}, "float": {arity: 3, check: floatArgs, prep: func(a []string) callFn { lo, hi, dp := atof(a[0]), atof(a[1]), atoi(a[2]) return func(s *session, _ string, _ []string) string { return formatFloat(lo+s.Float64()*(hi-lo), dp) } - }, number: func(token string, a []string) proven { + }, prints: DataTypeNumber, number: func(token string, _ DataType, a []string) proven { return printedNumber(token, bounded(atof(a[0]), atof(a[1]), false), atoi(a[2])) }}, "iban": {arity: 1, check: ibanArg, prep: func(a []string) callFn { cc := a[0] return func(s *session, _ string, _ []string) string { return iban(s, cc) } }}, - "date": {arity: -1, check: dateArgs, prep: datePrep}, - "time": {arity: -1, check: timeArg, prep: timePrep}, - "calc": {arity: -1, check: checkCalc, prep: calcPrep, operands: calcOperands}, - "lowercase": {arity: 1, check: transformArg, prep: transformPrep(strings.ToLower), operands: transformOperand}, - "uppercase": {arity: 1, check: transformArg, prep: transformPrep(strings.ToUpper), operands: transformOperand}, - "ascii": {arity: 1, check: transformArg, prep: transformPrep(asciiFold), operands: transformOperand}, + "date": {arity: -1, check: dateArgs, prep: datePrep}, + "time": {arity: -1, check: timeArg, prep: timePrep}, + "calc": {arity: -1, check: checkCalc, prep: calcPrep, operands: calcOperands}, // seq is the one stateful builtin: a per-session counter from 1, advancing on // each call. An optional name selects an independent counter; no name uses the // default one. Deterministic by construction, so seeded output stays stable. @@ -76,10 +73,10 @@ var builtins = map[string]builtin{ return func(s *session, _ string, _ []string) string { return strconv.FormatUint(s.next(key), 10) } - }, number: func(token string, _ []string) proven { - return printing(token, DataTypeInteger, bounded(1, math.MaxInt64, true)) + }, prints: DataTypeInteger, number: func(token string, prints DataType, _ []string) proven { + return printing(token, prints, bounded(1, math.MaxInt64, true)) }}, -} +}) // derive and sample are the two argument-free builtin shapes: a derivation reads // the output emitted so far, a sample reads only the rng. chars is the shape of a diff --git a/template.go b/template.go index b5f8a45..d23dd0b 100644 --- a/template.go +++ b/template.go @@ -119,8 +119,9 @@ type builtin struct { // for a builtin that reads none. operands func(args []string) []string // number proves what a call prints, token its body: the bounds of its number and the - // datatype of its text; nil for a builtin whose text is no number. - number func(token string, args []string) proven + // datatype of its text, prints or one inside it; nil for a builtin whose text is no number. + prints DataType + number func(token string, prints DataType, args []string) proven } // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false diff --git a/transform.go b/transform.go index 2b9fecd..0515889 100644 --- a/transform.go +++ b/transform.go @@ -14,6 +14,13 @@ var transforms = map[string]func(string) string{ "uppercase": strings.ToUpper, } +func withTransforms(builtins map[string]builtin) map[string]builtin { + for name, fn := range transforms { + builtins[name] = builtin{arity: 1, check: transformArg, prep: transformPrep(fn), operands: transformOperand} + } + return builtins +} + // unwrapTransform peels nested transform calls off an operand arg, returning the // field it finally names and the transforms to apply, innermost last. func unwrapTransform(arg string) (leaf string, chain []func(string) string, err error) { diff --git a/value.go b/value.go index 9b310c6..d217fd0 100644 --- a/value.go +++ b/value.go @@ -4,6 +4,7 @@ import ( "fmt" "math" "regexp" + "slices" "strconv" "strings" ) @@ -148,8 +149,8 @@ func (p *valueProof) template(t *template) proven { case t.fixed: return literalValue(t.lit) case len(t.ops) != 1: - v := unproven(notOneValue(t.format, "{int()}, {float()}, {seq()} or {calc()}")) - v.notOperand = notOneValue(t.format, "{int()}, {float()}, {seq()}, {digits()} or {calc()}") + v := unproven(notOneValue(t.format, numberCalls(false))) + v.notOperand = notOneValue(t.format, numberCalls(true)) return v } o := t.ops[0] @@ -164,7 +165,7 @@ func (p *valueProof) template(t *template) proven { case name == "calc": return p.calc(t, body, args) case builtins[name].number != nil: - return builtins[name].number(body, args) + return builtins[name].number(body, builtins[name].prints, args) case isTransform: return unproven(fmt.Sprintf("{%s} rewrites text rather than printing a value; write the values it would print", body)) } @@ -287,6 +288,19 @@ func printing(token string, prints DataType, v proven) proven { return v } +// numberCalls lists the builtins printing a number, or with text also those whose text +// is one. +func numberCalls(text bool) string { + var calls []string + for name, b := range builtins { + if b.number != nil && (text || b.prints != DataTypeString) { + calls = append(calls, "{"+name+"()}") + } + } + slices.Sort(calls) + return strings.Join(calls, ", ") + " or {calc()}" +} + func notOneValue(format, calls string) string { return fmt.Sprintf("%q is not one value; write one literal or one %s, or read one", format, calls) } -- 2.52.0 From fbc8fe78c9f189870663694a21e35c7e56509fe1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 08:46:53 +0200 Subject: [PATCH 105/966] Refuse a name registered as a builtin and a transform, build the offered calls once, and drop the done item --- template.go | 2 +- todo.md | 4 ---- transform.go | 9 ++++++--- value.go | 6 ++++-- 4 files changed, 11 insertions(+), 10 deletions(-) diff --git a/template.go b/template.go index d23dd0b..7217f3f 100644 --- a/template.go +++ b/template.go @@ -120,8 +120,8 @@ type builtin struct { operands func(args []string) []string // number proves what a call prints, token its body: the bounds of its number and the // datatype of its text, prints or one inside it; nil for a builtin whose text is no number. - prints DataType number func(token string, prints DataType, args []string) proven + prints DataType } // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false diff --git a/todo.md b/todo.md index 6abde67..ce749a6 100644 --- a/todo.md +++ b/todo.md @@ -19,10 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Register a transform once: `transforms` restates the `lowercase`, `uppercase` and - `ascii` entries in `builtins`, and `valueProof.template` classifies by the first while - render calls the second. Derive the numeric builtins `valueProof.template` lists in - its error text from `builtin.number` while there. - Split `drawCheck.reads`'s six-term condition into named predicates. - Rename what names two things: `template.go` holds the token grammar while `template` is `node.go`'s type and `Template` `inline.go`'s; `kind` `'b'` is a brace body in diff --git a/transform.go b/transform.go index 0515889..f24be31 100644 --- a/transform.go +++ b/transform.go @@ -14,11 +14,14 @@ var transforms = map[string]func(string) string{ "uppercase": strings.ToUpper, } -func withTransforms(builtins map[string]builtin) map[string]builtin { +func withTransforms(calls map[string]builtin) map[string]builtin { for name, fn := range transforms { - builtins[name] = builtin{arity: 1, check: transformArg, prep: transformPrep(fn), operands: transformOperand} + if _, clash := calls[name]; clash { + panic("fejkdata: " + name + " is registered as a builtin and a transform") + } + calls[name] = builtin{arity: 1, check: transformArg, prep: transformPrep(fn), operands: transformOperand} } - return builtins + return calls } // unwrapTransform peels nested transform calls off an operand arg, returning the diff --git a/value.go b/value.go index d217fd0..94a1acf 100644 --- a/value.go +++ b/value.go @@ -149,8 +149,8 @@ func (p *valueProof) template(t *template) proven { case t.fixed: return literalValue(t.lit) case len(t.ops) != 1: - v := unproven(notOneValue(t.format, numberCalls(false))) - v.notOperand = notOneValue(t.format, numberCalls(true)) + v := unproven(notOneValue(t.format, typedCalls)) + v.notOperand = notOneValue(t.format, operandCalls) return v } o := t.ops[0] @@ -288,6 +288,8 @@ func printing(token string, prints DataType, v proven) proven { return v } +var typedCalls, operandCalls = numberCalls(false), numberCalls(true) + // numberCalls lists the builtins printing a number, or with text also those whose text // is one. func numberCalls(text bool) string { -- 2.52.0 From ab0a10115db14add699a305351ea8822a997c72c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 09:01:49 +0200 Subject: [PATCH 106/966] Split drawCheck.reads's condition into readsOnEdge and walksInto, and drop the done item --- drawfence.go | 15 ++++++++++++++- todo.md | 1 - 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/drawfence.go b/drawfence.go index ec40eda..400b925 100644 --- a/drawfence.go +++ b/drawfence.go @@ -119,7 +119,7 @@ func (c *drawCheck) reads(n node, stopAtGroup bool) bool { } r := false for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); (isRef && (len(a.tail) > 0 || readsTable(n, a))) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) { + if readsOnEdge(n, e.label) || (walksInto(e.to, stopAtGroup) && c.reads(e.to, stopAtGroup)) { r = true break } @@ -131,6 +131,19 @@ func (c *drawCheck) reads(n node, stopAtGroup bool) bool { return r } +// readsOnEdge reports whether an edge of n is itself a reference path: a reference with a tail, or +// one naming a table. +func readsOnEdge(n node, label string) bool { + a, isRef := refRead(n, label) + return isRef && (len(a.tail) > 0 || readsTable(n, a)) +} + +// walksInto reports whether reads walks on into to, which it does short of a repeat, and of a draw +// group when stopAtGroup. +func walksInto(to node, stopAtGroup bool) bool { + return !repeats(to) && !(stopAtGroup && grouped(to)) +} + func repeats(n node) bool { t, isTemplate := n.(*template) return isTemplate && t.repeat > 1 diff --git a/todo.md b/todo.md index ce749a6..7b9aa8c 100644 --- a/todo.md +++ b/todo.md @@ -19,7 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Split `drawCheck.reads`'s six-term condition into named predicates. - Rename what names two things: `template.go` holds the token grammar while `template` is `node.go`'s type and `Template` `inline.go`'s; `kind` `'b'` is a brace body in `ftoken` and a builtin in `formatToken` and `op`; `tokens` is a template's parsed format -- 2.52.0 From 29f1908e5a9bc6fdf8549c767143b3235fc97be2 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 09:04:36 +0200 Subject: [PATCH 107/966] Drop the comments restating readsOnEdge and walksInto --- drawfence.go | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drawfence.go b/drawfence.go index 400b925..3d5a400 100644 --- a/drawfence.go +++ b/drawfence.go @@ -131,15 +131,11 @@ func (c *drawCheck) reads(n node, stopAtGroup bool) bool { return r } -// readsOnEdge reports whether an edge of n is itself a reference path: a reference with a tail, or -// one naming a table. func readsOnEdge(n node, label string) bool { a, isRef := refRead(n, label) return isRef && (len(a.tail) > 0 || readsTable(n, a)) } -// walksInto reports whether reads walks on into to, which it does short of a repeat, and of a draw -// group when stopAtGroup. func walksInto(to node, stopAtGroup bool) bool { return !repeats(to) && !(stopAtGroup && grouped(to)) } -- 2.52.0 From 191186ec941aa28a25f4afad8f76eeec46fa24a7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 17:24:44 +0200 Subject: [PATCH 108/966] Record why each entry point to drawCheck.reads keeps its own doc --- AGENTS.md | 1 + docs/decisions.md | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index c6a6372..a2a2f82 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,3 +110,4 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only +- Each entry point to `drawCheck.reads` says what its caller gets diff --git a/docs/decisions.md b/docs/decisions.md index 49baf2e..be26579 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -646,3 +646,9 @@ import drops them by name rather than by a rank a regeneration would move. `region.municipality.locality` is listed, and `region.locality` resolves too but is not: the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. + +## Each entry point to `drawCheck.reads` says what its caller gets + +2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on +`readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, +though `reads` states the same stop rule. Valid while the two differ in where they stop. -- 2.52.0 From 005f4d7adc18bd0527399c4f574b0dcc6ed6f559 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 17:28:35 +0200 Subject: [PATCH 109/966] Rename template.go to format.go, a table's tokens to cellTemplates, drawAt.alt to pinned and ftoken's brace-body kind to 't', and drop the done item --- README.md | 2 +- drawfence.go | 22 ++++---- family.go | 2 +- template.go => format.go | 4 +- ...bility_test.go => format_stability_test.go | 0 template_test.go => format_test.go | 0 graph.go | 2 +- shape_test.go | 2 +- table.go | 50 +++++++++---------- table_test.go | 4 +- todo.md | 5 -- 11 files changed, 44 insertions(+), 49 deletions(-) rename template.go => format.go (99%) rename template_stability_test.go => format_stability_test.go (100%) rename template_test.go => format_test.go (100%) diff --git a/README.md b/README.md index 66dbb2d..01f76cf 100644 --- a/README.md +++ b/README.md @@ -870,7 +870,7 @@ render.go Fake and the recursive renderer (choices, format strings, expans record.go records: Record, the JSON/CSV/SQL serializers, and their entry points struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link -template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format +format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format hold.go the hold: one draw per held name, and reading a field through it holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through diff --git a/drawfence.go b/drawfence.go index 3d5a400..def7c1c 100644 --- a/drawfence.go +++ b/drawfence.go @@ -191,11 +191,11 @@ type drawWalk struct { // table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose // cells the walk is in. type drawAt struct { - group string - route drawRoute - alt pinSet - drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true - whole pinSet + group string + route drawRoute + pinned pinSet + drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true + whole pinSet } // drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. @@ -211,13 +211,13 @@ type pathRead struct { type nodeVisit struct { n node group string - alt string + rows string } type readKey struct { group string path string - alt string + rows string } func newDrawWalk() *drawWalk { @@ -251,16 +251,16 @@ func (w *drawWalk) walk(n node, at drawAt) { in.whole = at.whole.clone() in.whole.add(cell.cellOf, cell.cellRow) w.edge(n, e, in) - case at.alt.clash(cell.cellOf, cell.cellRow) == nil: + case at.pinned.clash(cell.cellOf, cell.cellRow) == nil: in := at - in.alt = at.alt.entered(cell.cellOf, cell.cellRow) + in.pinned = at.pinned.entered(cell.cellOf, cell.cellRow) w.edge(n, e, in) } } } // rowsKey spells the rows the walk stands in, pinned and whole, for a map. -func (at drawAt) rowsKey() string { return at.alt.key() + "|" + at.whole.key() } +func (at drawAt) rowsKey() string { return at.pinned.key() + "|" + at.whole.key() } // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. @@ -272,7 +272,7 @@ func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { w.reads = append(w.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { at.alt = at.alt.entered(t, r) }) + tr.pins.each(func(t *table, r int) { at.pinned = at.pinned.entered(t, r) }) } } w.walk(e.to, at) diff --git a/family.go b/family.go index 55646fb..18230d0 100644 --- a/family.go +++ b/family.go @@ -219,4 +219,4 @@ func checkOwnFamily(t *template) error { // alternatives reports whether two reads sit in different rows of one table: rows the walks pinned, // or rows of one whole draw. // docs/decisions.md#the-rows-of-a-table-are-alternatives -func alternatives(a, b drawAt) bool { return a.alt.differs(&b.alt) || a.whole.differs(&b.whole) } +func alternatives(a, b drawAt) bool { return a.pinned.differs(&b.pinned) || a.whole.differs(&b.whole) } diff --git a/template.go b/format.go similarity index 99% rename from template.go rename to format.go index 7217f3f..7224bce 100644 --- a/template.go +++ b/format.go @@ -8,7 +8,7 @@ import ( // ftoken is one unit of a scanned format string: a literal rune or the body of a // {…} token. type ftoken struct { - kind byte // 'l' literal rune, 'b' brace body + kind byte // 'l' literal rune, 't' token body r rune body string } @@ -38,7 +38,7 @@ func eachToken(format string, fn func(ftoken) error) error { if end >= len(rs) { return fmt.Errorf("unterminated '{' in %q", format) } - t.kind, t.body = 'b', string(rs[i+1:end]) + t.kind, t.body = 't', string(rs[i+1:end]) i = end case '}': if i+1 < len(rs) && rs[i+1] == '}' { diff --git a/template_stability_test.go b/format_stability_test.go similarity index 100% rename from template_stability_test.go rename to format_stability_test.go diff --git a/template_test.go b/format_test.go similarity index 100% rename from template_test.go rename to format_test.go diff --git a/graph.go b/graph.go index fc43333..d17b10c 100644 --- a/graph.go +++ b/graph.go @@ -63,7 +63,7 @@ func contained(n node) []namedNode { case *table: return append([]namedNode{{node: n.format}}, named(n.fields)...) case *column: - if len(n.t.tokens) == 0 { + if len(n.t.cellTemplates) == 0 { return nil } var out []namedNode diff --git a/shape_test.go b/shape_test.go index e9d418e..3b6c2c4 100644 --- a/shape_test.go +++ b/shape_test.go @@ -97,7 +97,7 @@ func readsFact(n node) string { } case *table: collect(n.format) - for _, cell := range n.tokens { + for _, cell := range n.cellTemplates { collect(cell) } case *template: diff --git a/table.go b/table.go index f60d9cd..b06fcd3 100644 --- a/table.go +++ b/table.go @@ -13,26 +13,26 @@ import ( // columns, each row is one draw, and the format renders the drawn row. A column is a // cell of the row a render pinned; a cell carrying tokens compiles to a string node. type table struct { - category string - path string // the category's path from the data root, which a selector is written at - file string - format *template // fields are the column nodes - columns []string - col map[string]int - fields map[string]node // column nodes, the format's fields - whole *row // the pinned row rendered by the format - cells []string // rows × columns, flat - tokens map[int]*template - key int // key, name, weight and parent are column indexes, or -1 - name int - weight int - parent int - cum []float64 // cumulative weights, nil when uniform - byKey map[string]int - parentT *table - children map[string]*table - once sync.Once - index tableIndex + category string + path string // the category's path from the data root, which a selector is written at + file string + format *template // fields are the column nodes + columns []string + col map[string]int + fields map[string]node // column nodes, the format's fields + whole *row // the pinned row rendered by the format + cells []string // rows × columns, flat + cellTemplates map[int]*template + key int // key, name, weight and parent are column indexes, or -1 + name int + weight int + parent int + cum []float64 // cumulative weights, nil when uniform + byKey map[string]int + parentT *table + children map[string]*table + once sync.Once + index tableIndex } // tableIndex is what selection and linked draws look up, built on the first draw @@ -64,7 +64,7 @@ func (t *table) cell(row, col int) string { return t.cells[row*len(t.columns)+co // cellNode is what a cell renders: its compiled template where it carries tokens, // else nil for its text. -func (t *table) cellNode(row, col int) *template { return t.tokens[row*len(t.columns)+col] } +func (t *table) cellNode(row, col int) *template { return t.cellTemplates[row*len(t.columns)+col] } // tableOptions are the keys a table object takes; every other key is refused. var tableOptions = []string{"format", "key", "name", "parent", "rows", "weight"} @@ -312,11 +312,11 @@ func (t *table) checkCells() error { if err != nil { return fmt.Errorf("line %d, %s: %w", row+2, t.columns[col], err) } - if t.tokens == nil { - t.tokens = map[int]*template{} + if t.cellTemplates == nil { + t.cellTemplates = map[int]*template{} } - t.tokens[i] = n.(*template) - t.tokens[i].cellOf, t.tokens[i].cellRow = t, row + t.cellTemplates[i] = n.(*template) + t.cellTemplates[i].cellOf, t.cellTemplates[i].cellRow = t, row } return nil } diff --git a/table_test.go b/table_test.go index 8097a17..8fce3e6 100644 --- a/table_test.go +++ b/table_test.go @@ -996,8 +996,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for cr := 0; cr < cand.rows(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil - pinned := drawAt{alt: none.entered(cand, cr)} - if got := alternatives(drawAt{alt: s}, pinned); got != refused { + pinned := drawAt{pinned: none.entered(cand, cr)} + if got := alternatives(drawAt{pinned: s}, pinned); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } if alternatives(whole, pinned) || alternatives(pinned, whole) { diff --git a/todo.md b/todo.md index 7b9aa8c..8ae15fc 100644 --- a/todo.md +++ b/todo.md @@ -19,11 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Rename what names two things: `template.go` holds the token grammar while `template` - is `node.go`'s type and `Template` `inline.go`'s; `kind` `'b'` is a brace body in - `ftoken` and a builtin in `formatToken` and `op`; `tokens` is a template's parsed format - and a table's cell templates; `drawAt.alt` holds the rows - a walk entered. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 068e2b73ca3e8f8d3234579fb6f44be0b0a278a9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 17:32:05 +0200 Subject: [PATCH 110/966] Rename drawAt.pinned to pins, the visit keys' rows to rowsKey and cellNode to cellTemplate, and file format_test.go's split --- drawfence.go | 30 +++++++++++++++--------------- family.go | 2 +- graph.go | 2 +- render.go | 2 +- table.go | 4 ++-- table_test.go | 6 +++--- todo.md | 3 +++ value.go | 2 +- 8 files changed, 27 insertions(+), 24 deletions(-) diff --git a/drawfence.go b/drawfence.go index def7c1c..aebaa52 100644 --- a/drawfence.go +++ b/drawfence.go @@ -191,11 +191,11 @@ type drawWalk struct { // table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose // cells the walk is in. type drawAt struct { - group string - route drawRoute - pinned pinSet - drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true - whole pinSet + group string + route drawRoute + pins pinSet + drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true + whole pinSet } // drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. @@ -209,15 +209,15 @@ type pathRead struct { } type nodeVisit struct { - n node - group string - rows string + n node + group string + rowsKey string } type readKey struct { - group string - path string - rows string + group string + path string + rowsKey string } func newDrawWalk() *drawWalk { @@ -251,16 +251,16 @@ func (w *drawWalk) walk(n node, at drawAt) { in.whole = at.whole.clone() in.whole.add(cell.cellOf, cell.cellRow) w.edge(n, e, in) - case at.pinned.clash(cell.cellOf, cell.cellRow) == nil: + case at.pins.clash(cell.cellOf, cell.cellRow) == nil: in := at - in.pinned = at.pinned.entered(cell.cellOf, cell.cellRow) + in.pins = at.pins.entered(cell.cellOf, cell.cellRow) w.edge(n, e, in) } } } // rowsKey spells the rows the walk stands in, pinned and whole, for a map. -func (at drawAt) rowsKey() string { return at.pinned.key() + "|" + at.whole.key() } +func (at drawAt) rowsKey() string { return at.pins.key() + "|" + at.whole.key() } // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. @@ -272,7 +272,7 @@ func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { w.reads = append(w.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { at.pinned = at.pinned.entered(t, r) }) + tr.pins.each(func(t *table, r int) { at.pins = at.pins.entered(t, r) }) } } w.walk(e.to, at) diff --git a/family.go b/family.go index 18230d0..4893597 100644 --- a/family.go +++ b/family.go @@ -219,4 +219,4 @@ func checkOwnFamily(t *template) error { // alternatives reports whether two reads sit in different rows of one table: rows the walks pinned, // or rows of one whole draw. // docs/decisions.md#the-rows-of-a-table-are-alternatives -func alternatives(a, b drawAt) bool { return a.pinned.differs(&b.pinned) || a.whole.differs(&b.whole) } +func alternatives(a, b drawAt) bool { return a.pins.differs(&b.pins) || a.whole.differs(&b.whole) } diff --git a/graph.go b/graph.go index d17b10c..09fcc11 100644 --- a/graph.go +++ b/graph.go @@ -68,7 +68,7 @@ func contained(n node) []namedNode { } var out []namedNode for r := 0; r < n.t.rows(); r++ { - if cell := n.t.cellNode(r, n.i); cell != nil { + if cell := n.t.cellTemplate(r, n.i); cell != nil { out = append(out, namedNode{node: cell}) } } diff --git a/render.go b/render.go index be0194d..4aea292 100644 --- a/render.go +++ b/render.go @@ -70,7 +70,7 @@ func render(s *session, n node, sc renderScope) string { if sc.t != n.t { sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) } - if cell := n.t.cellNode(sc.row, n.i); cell != nil { + if cell := n.t.cellTemplate(sc.row, n.i); cell != nil { return render(s, cell, sc) } return n.t.cell(sc.row, n.i) diff --git a/table.go b/table.go index b06fcd3..45471fb 100644 --- a/table.go +++ b/table.go @@ -62,9 +62,9 @@ func (t *table) rows() int { return len(t.cells) / len(t.columns) } func (t *table) cell(row, col int) string { return t.cells[row*len(t.columns)+col] } -// cellNode is what a cell renders: its compiled template where it carries tokens, +// cellTemplate is what a cell renders: its compiled template where it carries tokens, // else nil for its text. -func (t *table) cellNode(row, col int) *template { return t.cellTemplates[row*len(t.columns)+col] } +func (t *table) cellTemplate(row, col int) *template { return t.cellTemplates[row*len(t.columns)+col] } // tableOptions are the keys a table object takes; every other key is refused. var tableOptions = []string{"format", "key", "name", "parent", "rows", "weight"} diff --git a/table_test.go b/table_test.go index 8fce3e6..99838c8 100644 --- a/table_test.go +++ b/table_test.go @@ -996,11 +996,11 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for cr := 0; cr < cand.rows(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil - pinned := drawAt{pinned: none.entered(cand, cr)} - if got := alternatives(drawAt{pinned: s}, pinned); got != refused { + candAt := drawAt{pins: none.entered(cand, cr)} + if got := alternatives(drawAt{pins: s}, candAt); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } - if alternatives(whole, pinned) || alternatives(pinned, whole) { + if alternatives(whole, candAt) || alternatives(candAt, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } other := drawAt{drawn: cand} diff --git a/todo.md b/todo.md index 8ae15fc..e8de440 100644 --- a/todo.md +++ b/todo.md @@ -19,6 +19,9 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: +- Hold only the `{token}` grammar's tests in `format_test.go`: move the helpers seven + other test files use (`engine`, `parse`, `compiled`, `mustRender`) and the weight, + recursion and path-navigation tests to files named for what they test. - Run the nine-seat panel again and file what it names here. ### Data diff --git a/value.go b/value.go index 94a1acf..250bb25 100644 --- a/value.go +++ b/value.go @@ -103,7 +103,7 @@ func (p *valueProof) cells(c *column) proven { var v proven for r := 0; r < c.t.rows(); r++ { var w proven - if cell := c.t.cellNode(r, c.i); cell != nil { + if cell := c.t.cellTemplate(r, c.i); cell != nil { w = p.of(cell) } else { w = literalValue(c.t.cell(r, c.i)) -- 2.52.0 From 44a824ccf5f009e2f488c0c5da27253a6c277f1d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 18:15:21 +0200 Subject: [PATCH 111/966] Decide one name, one meaning, and file the names that break it --- AGENTS.md | 2 +- docs/decisions.md | 15 +++++++++------ todo.md | 5 +++++ 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a2a2f82..6f7c33c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -64,7 +64,7 @@ In [docs/decisions.md](docs/decisions.md): - A typed column's calc is refused unless proven - `Column` carries text, not a Go value - `hold` names what a draw is kept in, `draw` the draw itself -- A function may not spell a method; two types may +- One name, one meaning - The package stays flat - The performance gate asserts allocations, not wall-clock time - Rows live in a TSV, the shape in JSON diff --git a/docs/decisions.md b/docs/decisions.md index be26579..a50f7d6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -307,13 +307,16 @@ writes as the load check proved it; a `Value any` would hand every caller a type Goal 2 wants a name to reach one unit, so a unit takes the stem of what it is, and a file the stem of the units it holds. -## A function may not spell a method; two types may +## One name, one meaning -A call writes a method with its receiver and a function bare, so one name on both greps -as one unit, which is the reaching cost goal 2 counts, while the receiver before a -shared method name says which type answers. `TestNoFunctionSpellsAMethod` holds it. A -type is outside the rule: `renderScope.hold` answers with a `*hold`, so the two spell -one unit, which no test can judge. +Two declarations of any kind — function, method, field, type — share a name only where +they name one thing: an interface's method, `Fake` on each template kind, a field +holding what a same-named method returns, `renderScope.hold` answering with a `*hold`. +A bare function never shares a name with a method, since grep cannot tell a bare call +from a method's; `TestNoFunctionSpellsAMethod` holds that, and review judges the rest, +which is meaning no test can see. Serves goal 2: names that tell the truth, and reaching +the unit behind a symptom without asking a person. Decided 2026-09-25 by Lilleman; valid +while goal 2 counts reading cost. ## The package stays flat diff --git a/todo.md b/todo.md index e8de440..9f82197 100644 --- a/todo.md +++ b/todo.md @@ -22,6 +22,11 @@ This round, in order: - Hold only the `{token}` grammar's tests in `format_test.go`: move the helpers seven other test files use (`engine`, `parse`, `compiled`, `mustRender`) and the weight, recursion and path-navigation tests to files named for what they test. +- Give `rows` and `field` one meaning each, per the decision "One name, one meaning": + `categoryFiles.rows` returns a TSV file name where `table.rows` counts rows, and + `formatOps.field` compiles a `{field}` token where `structFields.field` reads a Go + struct field. Rename every other shared name that names two things — `walk`, + `column`, `expr` and `of` included. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 5e84b26a5fcd068eb8f91ce9154301f116e0caec Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 18:18:57 +0200 Subject: [PATCH 112/966] Hold only the {token} grammar's tests in format_test.go, the rest in files named for what they test, and drop the done item --- builtins_test.go | 70 ++++++++++++ calc_test.go | 19 ++++ fejkdata_test.go | 35 ++++++ format_test.go | 281 ++++------------------------------------------- node_test.go | 82 ++++++++++++++ path_test.go | 19 ++++ render_test.go | 56 ++++++++++ todo.md | 3 - 8 files changed, 301 insertions(+), 264 deletions(-) create mode 100644 node_test.go create mode 100644 render_test.go diff --git a/builtins_test.go b/builtins_test.go index adcf039..4389fb7 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -368,3 +368,73 @@ func TestBuiltinDateSpansOneDay(t *testing.T) { t.Fatalf("one day in a date-only layout = %v, want it named a constant", err) } } + +func TestClassBuiltins(t *testing.T) { + cases := map[string]*regexp.Regexp{ + `"{digits(1)}"`: regexp.MustCompile(`^[0-9]$`), + `"{digits(3)}"`: regexp.MustCompile(`^[0-9]{3}$`), + `"{int(1,9)}"`: regexp.MustCompile(`^[1-9]$`), + `"{upper(1)}"`: regexp.MustCompile(`^[A-Z]$`), + `"{lower(1)}"`: regexp.MustCompile(`^[a-z]$`), + `"{upper(2)}{lower(2)}"`: regexp.MustCompile(`^[A-Z]{2}[a-z]{2}$`), + } + f := engine(7) + for tmpl, re := range cases { + for i := 0; i < 100; i++ { + if got := mustRender(t, f, tmpl); !re.MatchString(got) { + t.Fatalf("%s produced %q, want %s", tmpl, got, re) + } + } + } +} + +func TestFunctionTokenLuhn(t *testing.T) { + // {luhn()} appends a Luhn check digit over the digits emitted so far in the + // current expansion (non-digits skipped but kept). It reads the output + // buffer, so a value is never re-rendered. Bodies are escaped to fix input. + f := engine(1) + cases := map[string]string{ + `"811218987{luhn()}"`: "8112189876", // personnummer body + `"7992739871{luhn()}"`: "79927398713", // classic Luhn vector + `"811218-987{luhn()}"`: "811218-9876", // '-' skipped, kept + `{"format":"{n}{luhn()}","n":"811218987"}`: "8112189876", // over a rendered token + } + for tmpl, want := range cases { + if got := mustRender(t, f, tmpl); got != want { + t.Fatalf("%s = %q, want %q", tmpl, got, want) + } + } +} + +func TestBuiltinCompileErrors(t *testing.T) { + for _, bad := range []string{ + `"{digits(0)}"`, // count must be positive + `"{upper(x)}"`, // count must be an integer + `"{lower()}"`, // wrong arity + `"{nope()}"`, // unknown function + `"{luhn(x)}"`, // function given args it takes none of + `"{int(1)}"`, // wrong arity + `"{int(a,b)}"`, // non-integer args + `"{int(5,1)}"`, // min > max + `"{hex(0)}"`, // count must be positive + `"{nanoid(-1)}"`, // negative count + `"{base64(0)}"`, // count must be positive + `"{float(1,2)}"`, // wrong arity + `"{float(1,2,-1)}"`, // negative decimals + `"{float(NaN,NaN,2)}"`, // bounds must be finite + `"{float(Inf,Inf,2)}"`, // same-sign infinities + `"{float(1,NaN,2)}"`, // one NaN bound + `"{float(-Inf,1,2)}"`, // one infinite bound + `"{digits(+5)}"`, // a count is a plain integer + `"{digits(05)}"`, // no leading zero + `"{int(+1,5)}"`, // a bound is a plain integer + `"{int(5,5)}"`, // a constant is written as text + `"{float(1,1,2)}"`, // a constant is written as text + `"{iban(US)}"`, // unsupported country + `"{seq(a,b)}"`, // seq takes at most one name + } { + if _, err := compile(parse(t, bad)); err == nil { + t.Errorf("compile(%s) = nil error, want error", bad) + } + } +} diff --git a/calc_test.go b/calc_test.go index 1c3dcab..0ae7cd8 100644 --- a/calc_test.go +++ b/calc_test.go @@ -270,3 +270,22 @@ func TestCalcConstantZeroDivisorIsRejected(t *testing.T) { } t.Fatal("a sometimes-zero divisor never printed +Inf in 50 draws") } + +func TestCalcCompileErrors(t *testing.T) { + for _, bad := range []string{ + `"{calc(1/0)}"`, // a constant zero divisor + `{"format":"{calc(x/y)}","x":"1","y":"0"}`, // a fixed zero divisor + `"{calc()}"`, // calc needs an expression + `"{calc(1 +)}"`, // dangling operator + `"{calc((1 + 2)}"`, // unbalanced parenthesis + `"{calc(1 2)}"`, // two operands, no operator + `"{calc(price)}"`, // operand names no field + `"{calc(1, 2, 3)}"`, // too many args + `"{calc(1, x)}"`, // decimals arg not an integer + `"{calc(1, -1)}"`, // decimals negative + } { + if _, err := compile(parse(t, bad)); err == nil { + t.Errorf("compile(%s) = nil error, want error", bad) + } + } +} diff --git a/fejkdata_test.go b/fejkdata_test.go index 18b5a59..bbefb3b 100644 --- a/fejkdata_test.go +++ b/fejkdata_test.go @@ -1,6 +1,7 @@ package fejkdata import ( + "encoding/json" "errors" "io/fs" "strings" @@ -92,3 +93,37 @@ func fakeTemplate(t *testing.T, f *Generator, s string) string { } return got } + +// engine builds a seeded generator with no loaded categories, for rendering tests. +func engine(seed uint64) *Generator { + s, err := newRand(seed, true) + if err != nil { + panic(err) + } + return &Generator{rand: s} +} + +// parse unmarshals a JSON template fragment into its dynamic form. +func parse(t *testing.T, s string) any { + t.Helper() + var v any + if err := json.Unmarshal([]byte(s), &v); err != nil { + t.Fatalf("parse %q: %v", s, err) + } + return v +} + +// compiled parses and compiles a JSON fragment into a node. +func compiled(t *testing.T, s string) node { + t.Helper() + n, err := compile(parse(t, s)) + if err != nil { + t.Fatalf("compile %q: %v", s, err) + } + return n +} + +func mustRender(t *testing.T, f *Generator, s string) string { + t.Helper() + return renderOnce(f.rand, compiled(t, s)) +} diff --git a/format_test.go b/format_test.go index f4741cf..0d466ef 100644 --- a/format_test.go +++ b/format_test.go @@ -1,47 +1,12 @@ package fejkdata import ( - "encoding/json" "reflect" "regexp" "strings" "testing" ) -// engine builds a seeded generator with no loaded categories, for rendering tests. -func engine(seed uint64) *Generator { - s, err := newRand(seed, true) - if err != nil { - panic(err) - } - return &Generator{rand: s} -} - -// parse unmarshals a JSON template fragment into its dynamic form. -func parse(t *testing.T, s string) any { - t.Helper() - var v any - if err := json.Unmarshal([]byte(s), &v); err != nil { - t.Fatalf("parse %q: %v", s, err) - } - return v -} - -// compiled parses and compiles a JSON fragment into a node. -func compiled(t *testing.T, s string) node { - t.Helper() - n, err := compile(parse(t, s)) - if err != nil { - t.Fatalf("compile %q: %v", s, err) - } - return n -} - -func mustRender(t *testing.T, f *Generator, s string) string { - t.Helper() - return renderOnce(f.rand, compiled(t, s)) -} - func TestStringIsAFormat(t *testing.T) { f := engine(1) for src, want := range map[string]string{ @@ -61,25 +26,6 @@ func TestStringIsAFormat(t *testing.T) { } } -func TestClassBuiltins(t *testing.T) { - cases := map[string]*regexp.Regexp{ - `"{digits(1)}"`: regexp.MustCompile(`^[0-9]$`), - `"{digits(3)}"`: regexp.MustCompile(`^[0-9]{3}$`), - `"{int(1,9)}"`: regexp.MustCompile(`^[1-9]$`), - `"{upper(1)}"`: regexp.MustCompile(`^[A-Z]$`), - `"{lower(1)}"`: regexp.MustCompile(`^[a-z]$`), - `"{upper(2)}{lower(2)}"`: regexp.MustCompile(`^[A-Z]{2}[a-z]{2}$`), - } - f := engine(7) - for tmpl, re := range cases { - for i := 0; i < 100; i++ { - if got := mustRender(t, f, tmpl); !re.MatchString(got) { - t.Fatalf("%s produced %q, want %s", tmpl, got, re) - } - } - } -} - func TestTextIsLiteral(t *testing.T) { if got := mustRender(t, engine(1), `{"format":"100 Main St #1 {x}","x":"A"}`); got != "100 Main St #1 A" { t.Fatalf("text = %q, want it verbatim", got) @@ -128,210 +74,17 @@ func TestAlternationPicksOneField(t *testing.T) { } } -func TestWeightZeroIsRejected(t *testing.T) { - _, err := compile(parse(t, `[{"format":"X","weight":0},"Y"]`)) - if err == nil || !strings.Contains(err.Error(), "weight 0") || !strings.Contains(err.Error(), "remove") { - t.Fatalf("compile(weight 0) = %v, want it rejected naming the fix", err) - } -} - -func TestWeightSkewsDistribution(t *testing.T) { - f := engine(9) - heavy := 0 - for i := 0; i < 1000; i++ { - if mustRender(t, f, `[{"format":"H","weight":10},"L"]`) == "H" { - heavy++ - } - } - if heavy < 800 { // expected ~909 - t.Fatalf("heavy variant chosen %d/1000, want a clear majority", heavy) - } -} - -func TestRepeatRendersFormatNTimes(t *testing.T) { - // repeat N concatenates N independent renders of the format ('x','y' are - // literal, not character classes). - if got := mustRender(t, engine(1), `{"format":"xy","repeat":3}`); got != "xyxyxy" { - t.Fatalf("repeat literal = %q, want xyxyxy", got) - } - // separator joins renders (N-1 of them, no trailing one). - if got := mustRender(t, engine(1), `{"format":"xy","repeat":3,"separator":"-"}`); got != "xy-xy-xy" { - t.Fatalf("repeat with separator = %q, want xy-xy-xy", got) - } - f, re := engine(7), regexp.MustCompile(`^[0-9]{4}$`) +func TestAlternationThreeWay(t *testing.T) { + f := engine(4) seen := map[string]bool{} - for i := 0; i < 50; i++ { - got := mustRender(t, f, `{"format":"{digits(1)}","repeat":4}`) - if !re.MatchString(got) { - t.Fatalf("repeat-4 = %q, want 4 digits", got) - } - seen[got] = true + for i := 0; i < 200; i++ { + seen[mustRender(t, f, `{"format":"{a|b|c}","a":"A","b":"B","c":"C"}`)] = true } - if len(seen) < 2 { - t.Fatalf("repeat never varied across renders: %v", seen) + if !seen["A"] || !seen["B"] || !seen["C"] || len(seen) != 3 { + t.Fatalf("3-way alternation produced %v, want A, B and C", seen) } } -func TestFunctionTokenLuhn(t *testing.T) { - // {luhn()} appends a Luhn check digit over the digits emitted so far in the - // current expansion (non-digits skipped but kept). It reads the output - // buffer, so a value is never re-rendered. Bodies are escaped to fix input. - f := engine(1) - cases := map[string]string{ - `"811218987{luhn()}"`: "8112189876", // personnummer body - `"7992739871{luhn()}"`: "79927398713", // classic Luhn vector - `"811218-987{luhn()}"`: "811218-9876", // '-' skipped, kept - `{"format":"{n}{luhn()}","n":"811218987"}`: "8112189876", // over a rendered token - } - for tmpl, want := range cases { - if got := mustRender(t, f, tmpl); got != want { - t.Fatalf("%s = %q, want %q", tmpl, got, want) - } - } -} - -func TestRecursionHasNoDepthLimit(t *testing.T) { - // Build {format:{a}, a:[{format:{a}, a:[ ... "deep" ]]}} 50 levels deep. - tmpl := `"deep"` - for i := 0; i < 50; i++ { - tmpl = `{"format":"{a}","a":` + tmpl + `}` - } - if got := mustRender(t, engine(1), tmpl); got != "deep" { - t.Fatalf("deep recursion = %q, want deep", got) - } -} - -func TestCompileErrors(t *testing.T) { - // Every structural problem is caught up front, at compile/New time, never - // deferred to a random render that happens to hit the bad branch. - for _, bad := range []string{ - `{"x":"Q"}`, // object without "format" - `{"format":"{y}","x":1}`, // a field is a bare number - `[1, 2]`, // a choice of numbers - `5`, // unsupported node type - `[]`, // empty choice - `"{x"`, // unterminated brace - `"x}y"`, // a lone } must be written }} - `["x"]`, // a one-item choice is its item - `["a","a"]`, // a repeated item is a weight - `{"format":"x"}`, // an object holding only a format is a string - `[{"format":"a","weight":1},"b"]`, // weight 1 is the default - `{"format":"{x}","x":"v","repeat":1}`, // repeat 1 is the default - `{"format":"{a{b}","a":"Q"}`, // a brace inside a token - `"{x}"`, // a bare string has no fields to name - `"{digits(0)}"`, // count must be positive - `"{upper(x)}"`, // count must be an integer - `"{lower()}"`, // wrong arity - `{"format":"{y}","x":"Q"}`, // token names a missing field - `"{}"`, // empty token name - `{"format":"{a|}","a":"Q"}`, // empty alternation segment - `[{"format":"A","weight":-1},"B"]`, // negative weight - `[{"format":"A","weight":0},{"format":"B","weight":0}]`, // weights sum to zero - `[{"format":"A","weight":1e308},{"format":"B","weight":1e308}]`, // weights overflow to +Inf - `{"format":"A","weight":"heavy"}`, // non-numeric weight - `{"format":"x","repeat":0}`, // repeat below 1 - `{"format":"x","repeat":-2}`, // negative repeat - `{"format":"x","repeat":1.5}`, // non-integer repeat - `{"format":"x","repeat":"two"}`, // non-numeric repeat - `{"format":"x","repeat":2,"separator":5}`, // non-string separator - `"{nope()}"`, // unknown function - `"{luhn(x)}"`, // function given args it takes none of - `"{luhn(}"`, // malformed function token - `"{int(1)}"`, // wrong arity - `"{int(a,b)}"`, // non-integer args - `"{int(5,1)}"`, // min > max - `"{hex(0)}"`, // count must be positive - `"{nanoid(-1)}"`, // negative count - `"{base64(0)}"`, // count must be positive - `"{float(1,2)}"`, // wrong arity - `"{float(1,2,-1)}"`, // negative decimals - `"{float(NaN,NaN,2)}"`, // bounds must be finite - `"{float(Inf,Inf,2)}"`, // same-sign infinities - `"{float(1,NaN,2)}"`, // one NaN bound - `"{float(-Inf,1,2)}"`, // one infinite bound - `"{digits(+5)}"`, // a count is a plain integer - `"{digits(05)}"`, // no leading zero - `"{int(+1,5)}"`, // a bound is a plain integer - `"{int(5,5)}"`, // a constant is written as text - `"{float(1,1,2)}"`, // a constant is written as text - `{"format":"{x}","x":"v","repeat":2,"separator":""}`, // separator "" is the default - `"{calc(1/0)}"`, // a constant zero divisor - `{"format":"{calc(x/y)}","x":"1","y":"0"}`, // a fixed zero divisor - `"{iban(US)}"`, // unsupported country - `"{seq(a,b)}"`, // seq takes at most one name - `"{calc()}"`, // calc needs an expression - `"{calc(1 +)}"`, // dangling operator - `"{calc((1 + 2)}"`, // unbalanced parenthesis - `"{calc(1 2)}"`, // two operands, no operator - `"{calc(price)}"`, // operand names no field - `"{calc(1, 2, 3)}"`, // too many args - `"{calc(1, x)}"`, // decimals arg not an integer - `"{calc(1, -1)}"`, // decimals negative - } { - if _, err := compile(parse(t, bad)); err == nil { - t.Errorf("compile(%s) = nil error, want error", bad) - } - } -} - -func TestFakePathNavigation(t *testing.T) { - f := engine(1) - f.categories = map[string]node{ - "addr": compiled(t, `{"format":"{street}","street":"Main"}`), - } - if got, err := f.Fake("addr"); err != nil || !strings.Contains(got, "Main") { - t.Fatalf("Fake(addr) = %q, %v", got, err) - } - if got, err := f.Fake("addr.street"); err != nil || got != "Main" { - t.Fatalf("Fake(addr.street) = %q, %v, want Main", got, err) - } - if _, err := f.Fake("addr.nope"); err == nil { - t.Error("Fake(addr.nope) = nil error, want missing-field error") - } - if _, err := f.Fake("missing"); err == nil { - t.Error("Fake(missing) = nil error, want unknown-category error") - } -} - -// TestGrowIsALowerBound pins the property that makes the pre-sized render buffer -// safe: grow must never exceed what expand emits. render multiplies it by repeat, -// so an over-estimate would amplify up to a million-fold. -func TestGrowIsALowerBound(t *testing.T) { - f := engine(3) - for _, format := range []string{ - "", - "plain literal", - "{digits(2)}-{int(1,9)}{int(1,9)}-{upper(2)}-{lower(2)}", - "01Aa# literal", - "{{}} {{{x}}}", - "Ö dag åäö 日本語", - "{x}{x}{x}", - "{hex(8)}-{int(10,99)}-{nanoid(5)}", - "9{d}{luhn()}", - "{a|b} and {a|b}", - } { - src := `{"format":` + quote(format) + `,"x":"1","a":"A","b":"B","d":"012345678901234"}` - tmpl, ok := compiled(t, src).(*template) - if !ok { - t.Fatalf("format %q did not compile to a template", format) - } - for i := 0; i < 50; i++ { - var set holdSet - if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.grow { - t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) - } - } - } -} - -func quote(s string) string { - b, err := json.Marshal(s) - if err != nil { - panic(err) - } - return string(b) -} - func TestHashIsLiteral(t *testing.T) { f := engine(1) cases := map[string]string{ @@ -357,14 +110,20 @@ func TestMultibyteFormat(t *testing.T) { } } -func TestAlternationThreeWay(t *testing.T) { - f := engine(4) - seen := map[string]bool{} - for i := 0; i < 200; i++ { - seen[mustRender(t, f, `{"format":"{a|b|c}","a":"A","b":"B","c":"C"}`)] = true - } - if !seen["A"] || !seen["B"] || !seen["C"] || len(seen) != 3 { - t.Fatalf("3-way alternation produced %v, want A, B and C", seen) +func TestFormatCompileErrors(t *testing.T) { + for _, bad := range []string{ + `"{x"`, // unterminated brace + `"x}y"`, // a lone } must be written }} + `{"format":"{a{b}","a":"Q"}`, // a brace inside a token + `"{x}"`, // a bare string has no fields to name + `{"format":"{y}","x":"Q"}`, // token names a missing field + `"{}"`, // empty token name + `{"format":"{a|}","a":"Q"}`, // empty alternation segment + `"{luhn(}"`, // malformed function token + } { + if _, err := compile(parse(t, bad)); err == nil { + t.Errorf("compile(%s) = nil error, want error", bad) + } } } diff --git a/node_test.go b/node_test.go new file mode 100644 index 0000000..5685849 --- /dev/null +++ b/node_test.go @@ -0,0 +1,82 @@ +package fejkdata + +import ( + "regexp" + "strings" + "testing" +) + +func TestWeightZeroIsRejected(t *testing.T) { + _, err := compile(parse(t, `[{"format":"X","weight":0},"Y"]`)) + if err == nil || !strings.Contains(err.Error(), "weight 0") || !strings.Contains(err.Error(), "remove") { + t.Fatalf("compile(weight 0) = %v, want it rejected naming the fix", err) + } +} + +func TestWeightSkewsDistribution(t *testing.T) { + f := engine(9) + heavy := 0 + for i := 0; i < 1000; i++ { + if mustRender(t, f, `[{"format":"H","weight":10},"L"]`) == "H" { + heavy++ + } + } + if heavy < 800 { // expected ~909 + t.Fatalf("heavy variant chosen %d/1000, want a clear majority", heavy) + } +} + +func TestRepeatRendersFormatNTimes(t *testing.T) { + // repeat N concatenates N independent renders of the format ('x','y' are + // literal, not character classes). + if got := mustRender(t, engine(1), `{"format":"xy","repeat":3}`); got != "xyxyxy" { + t.Fatalf("repeat literal = %q, want xyxyxy", got) + } + // separator joins renders (N-1 of them, no trailing one). + if got := mustRender(t, engine(1), `{"format":"xy","repeat":3,"separator":"-"}`); got != "xy-xy-xy" { + t.Fatalf("repeat with separator = %q, want xy-xy-xy", got) + } + f, re := engine(7), regexp.MustCompile(`^[0-9]{4}$`) + seen := map[string]bool{} + for i := 0; i < 50; i++ { + got := mustRender(t, f, `{"format":"{digits(1)}","repeat":4}`) + if !re.MatchString(got) { + t.Fatalf("repeat-4 = %q, want 4 digits", got) + } + seen[got] = true + } + if len(seen) < 2 { + t.Fatalf("repeat never varied across renders: %v", seen) + } +} + +func TestNodeCompileErrors(t *testing.T) { + // Every structural problem is caught up front, at compile/New time, never + // deferred to a random render that happens to hit the bad branch. + for _, bad := range []string{ + `{"x":"Q"}`, // object without "format" + `{"format":"{y}","x":1}`, // a field is a bare number + `[1, 2]`, // a choice of numbers + `5`, // unsupported node type + `[]`, // empty choice + `["x"]`, // a one-item choice is its item + `["a","a"]`, // a repeated item is a weight + `{"format":"x"}`, // an object holding only a format is a string + `[{"format":"a","weight":1},"b"]`, // weight 1 is the default + `{"format":"{x}","x":"v","repeat":1}`, // repeat 1 is the default + `[{"format":"A","weight":-1},"B"]`, // negative weight + `[{"format":"A","weight":0},{"format":"B","weight":0}]`, // weights sum to zero + `[{"format":"A","weight":1e308},{"format":"B","weight":1e308}]`, // weights overflow to +Inf + `{"format":"A","weight":"heavy"}`, // non-numeric weight + `{"format":"x","repeat":0}`, // repeat below 1 + `{"format":"x","repeat":-2}`, // negative repeat + `{"format":"x","repeat":1.5}`, // non-integer repeat + `{"format":"x","repeat":"two"}`, // non-numeric repeat + `{"format":"x","repeat":2,"separator":5}`, // non-string separator + `{"format":"{x}","x":"v","repeat":2,"separator":""}`, // separator "" is the default + } { + if _, err := compile(parse(t, bad)); err == nil { + t.Errorf("compile(%s) = nil error, want error", bad) + } + } +} diff --git a/path_test.go b/path_test.go index 8f1065d..a8a9fa6 100644 --- a/path_test.go +++ b/path_test.go @@ -170,3 +170,22 @@ func TestBindingKeyIsNotAPathSegment(t *testing.T) { t.Errorf("Fake(name./color) = %v, want a no-field error", err) } } + +func TestFakePathNavigation(t *testing.T) { + f := engine(1) + f.categories = map[string]node{ + "addr": compiled(t, `{"format":"{street}","street":"Main"}`), + } + if got, err := f.Fake("addr"); err != nil || !strings.Contains(got, "Main") { + t.Fatalf("Fake(addr) = %q, %v", got, err) + } + if got, err := f.Fake("addr.street"); err != nil || got != "Main" { + t.Fatalf("Fake(addr.street) = %q, %v, want Main", got, err) + } + if _, err := f.Fake("addr.nope"); err == nil { + t.Error("Fake(addr.nope) = nil error, want missing-field error") + } + if _, err := f.Fake("missing"); err == nil { + t.Error("Fake(missing) = nil error, want unknown-category error") + } +} diff --git a/render_test.go b/render_test.go new file mode 100644 index 0000000..4302a69 --- /dev/null +++ b/render_test.go @@ -0,0 +1,56 @@ +package fejkdata + +import ( + "encoding/json" + "testing" +) + +func TestRecursionHasNoDepthLimit(t *testing.T) { + // Build {format:{a}, a:[{format:{a}, a:[ ... "deep" ]]}} 50 levels deep. + tmpl := `"deep"` + for i := 0; i < 50; i++ { + tmpl = `{"format":"{a}","a":` + tmpl + `}` + } + if got := mustRender(t, engine(1), tmpl); got != "deep" { + t.Fatalf("deep recursion = %q, want deep", got) + } +} + +// TestGrowIsALowerBound pins the property that makes the pre-sized render buffer +// safe: grow must never exceed what expand emits. render multiplies it by repeat, +// so an over-estimate would amplify up to a million-fold. +func TestGrowIsALowerBound(t *testing.T) { + f := engine(3) + for _, format := range []string{ + "", + "plain literal", + "{digits(2)}-{int(1,9)}{int(1,9)}-{upper(2)}-{lower(2)}", + "01Aa# literal", + "{{}} {{{x}}}", + "Ö dag åäö 日本語", + "{x}{x}{x}", + "{hex(8)}-{int(10,99)}-{nanoid(5)}", + "9{d}{luhn()}", + "{a|b} and {a|b}", + } { + src := `{"format":` + quote(format) + `,"x":"1","a":"A","b":"B","d":"012345678901234"}` + tmpl, ok := compiled(t, src).(*template) + if !ok { + t.Fatalf("format %q did not compile to a template", format) + } + for i := 0; i < 50; i++ { + var set holdSet + if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.grow { + t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) + } + } + } +} + +func quote(s string) string { + b, err := json.Marshal(s) + if err != nil { + panic(err) + } + return string(b) +} diff --git a/todo.md b/todo.md index 9f82197..3e71e27 100644 --- a/todo.md +++ b/todo.md @@ -19,9 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Hold only the `{token}` grammar's tests in `format_test.go`: move the helpers seven - other test files use (`engine`, `parse`, `compiled`, `mustRender`) and the weight, - recursion and path-navigation tests to files named for what they test. - Give `rows` and `field` one meaning each, per the decision "One name, one meaning": `categoryFiles.rows` returns a TSV file name where `table.rows` counts rows, and `formatOps.field` compiles a `{field}` token where `structFields.field` reads a Go -- 2.52.0 From 4b087df57ef32dd91604a0d2b185f21afc1270dc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 18:21:32 +0200 Subject: [PATCH 113/966] Move the builtin argument tests to builtins_test.go and checkFunc's cases to format_test.go --- builtins_test.go | 20 ++++++++++++++------ format_test.go | 19 +++++-------------- 2 files changed, 19 insertions(+), 20 deletions(-) diff --git a/builtins_test.go b/builtins_test.go index 4389fb7..8d419fe 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -410,22 +410,16 @@ func TestBuiltinCompileErrors(t *testing.T) { for _, bad := range []string{ `"{digits(0)}"`, // count must be positive `"{upper(x)}"`, // count must be an integer - `"{lower()}"`, // wrong arity - `"{nope()}"`, // unknown function - `"{luhn(x)}"`, // function given args it takes none of - `"{int(1)}"`, // wrong arity `"{int(a,b)}"`, // non-integer args `"{int(5,1)}"`, // min > max `"{hex(0)}"`, // count must be positive `"{nanoid(-1)}"`, // negative count `"{base64(0)}"`, // count must be positive - `"{float(1,2)}"`, // wrong arity `"{float(1,2,-1)}"`, // negative decimals `"{float(NaN,NaN,2)}"`, // bounds must be finite `"{float(Inf,Inf,2)}"`, // same-sign infinities `"{float(1,NaN,2)}"`, // one NaN bound `"{float(-Inf,1,2)}"`, // one infinite bound - `"{digits(+5)}"`, // a count is a plain integer `"{digits(05)}"`, // no leading zero `"{int(+1,5)}"`, // a bound is a plain integer `"{int(5,5)}"`, // a constant is written as text @@ -438,3 +432,17 @@ func TestBuiltinCompileErrors(t *testing.T) { } } } + +func TestArgErrorsNameTheSpelling(t *testing.T) { + for src, want := range map[string]string{ + `"{float(1,2,02)}"`: "write 2", + `{"format":"{calc(a,02)}","a":"1"}`: "write 2", + `"{digits(+5)}"`: "write 5", + `"{hex(99999999999999999999)}"`: "exceeds the maximum", + `"{int(007,9)}"`: "write 7", + } { + if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("compile(%s) = %v, want an error saying %q", src, err, want) + } + } +} diff --git a/format_test.go b/format_test.go index 0d466ef..8df1f38 100644 --- a/format_test.go +++ b/format_test.go @@ -120,6 +120,11 @@ func TestFormatCompileErrors(t *testing.T) { `"{}"`, // empty token name `{"format":"{a|}","a":"Q"}`, // empty alternation segment `"{luhn(}"`, // malformed function token + `"{lower()}"`, // wrong arity + `"{nope()}"`, // unknown function + `"{luhn(x)}"`, // function given args it takes none of + `"{int(1)}"`, // wrong arity + `"{float(1,2)}"`, // wrong arity } { if _, err := compile(parse(t, bad)); err == nil { t.Errorf("compile(%s) = nil error, want error", bad) @@ -127,20 +132,6 @@ func TestFormatCompileErrors(t *testing.T) { } } -func TestArgErrorsNameTheSpelling(t *testing.T) { - for src, want := range map[string]string{ - `"{float(1,2,02)}"`: "write 2", - `{"format":"{calc(a,02)}","a":"1"}`: "write 2", - `"{digits(+5)}"`: "write 5", - `"{hex(99999999999999999999)}"`: "exceeds the maximum", - `"{int(007,9)}"`: "write 7", - } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { - t.Errorf("compile(%s) = %v, want an error saying %q", src, err, want) - } - } -} - // TestSplitArgsQuotesOutsideSelectors pins the arg grammar: a comma splits outside a // selector and outside a quoted layout, and a quote inside a selector is a name's text. func TestSplitArgsQuotesOutsideSelectors(t *testing.T) { -- 2.52.0 From 65091233a898043c4466afb2c3e5ad62f1958239 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 18:53:42 +0200 Subject: [PATCH 114/966] Give rows, field, walk, column, expr and of one meaning each: rowCount, readRows, gatherFields, addField, addTag, renderCount and the prove methods --- data.go | 4 ++-- datatype.go | 4 ++-- graph.go | 10 ++++----- struct.go | 22 +++++++++--------- table.go | 32 +++++++++++++------------- table_test.go | 4 ++-- todo.md | 8 +++---- value.go | 62 +++++++++++++++++++++++++-------------------------- 8 files changed, 72 insertions(+), 74 deletions(-) diff --git a/data.go b/data.go index 5299de3..08dab0a 100644 --- a/data.go +++ b/data.go @@ -124,8 +124,8 @@ type categoryFiles struct { tsv map[string]bool } -// rows reads the rows file a category names beside it. -func (c *categoryFiles) rows(name string) (string, error) { +// readRows reads the rows file a category names beside it. +func (c *categoryFiles) readRows(name string) (string, error) { if _, present := c.tsv[name]; !present { return "", fmt.Errorf("rows names %s, which is not beside it in %s", name, c.src.name(c.dir)) } diff --git a/datatype.go b/datatype.go index fae8097..31e1a0a 100644 --- a/datatype.go +++ b/datatype.go @@ -166,7 +166,7 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { switch { case da != DataTypeString && db != DataTypeString: return bothTyped(a, da, b, db) - case typed.datatype == DataTypeString && (&valueProof{}).columnItem(bare).not[want] != "": + case typed.datatype == DataTypeString && (&valueProof{}).proveColumnItem(bare).not[want] != "": return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, bare.format, dataTypeNouns[want], typed.format, asText(typed)) case bare.fromString: // an object may carry a weight, which this spelling would drop return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, bare.format, bare.format, want) @@ -184,7 +184,7 @@ func bothTyped(a *template, da DataType, b *template, db DataType) error { switch { case read.datatype != DataTypeString: return fmt.Errorf("its items hold %s and %s; a column holds one datatype", da, db) - case (&valueProof{}).columnItem(read).not[other] == "": + case (&valueProof{}).proveColumnItem(read).not[other] == "": return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", da, db, typedAs(read, other)) } return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", da, db, read.format, asText(read)) diff --git a/graph.go b/graph.go index 09fcc11..0ff962f 100644 --- a/graph.go +++ b/graph.go @@ -67,7 +67,7 @@ func contained(n node) []namedNode { return nil } var out []namedNode - for r := 0; r < n.t.rows(); r++ { + for r := 0; r < n.t.rowCount(); r++ { if cell := n.t.cellTemplate(r, n.i); cell != nil { out = append(out, namedNode{node: cell}) } @@ -171,13 +171,13 @@ func pathLeaves(n node, tail []string) []node { type reachMemo map[node]int -func (m reachMemo) of(n node) int { +func (m reachMemo) renderCount(n node) int { if r, done := m[n]; done { return r } r := 1 for _, e := range renderEdges(n) { - if c := m.of(e.to); c > r { + if c := m.renderCount(e.to); c > r { r = c } } @@ -192,8 +192,8 @@ func (m reachMemo) of(n node) int { // path, so nested repeats cannot build what one repeat may not. // docs/decisions.md#the-repeat-cap-bounds-renders-not-bytes func repeatCheck(path string, n node, mem reachMemo) error { - if t, ok := n.(*template); ok && t.repeat > 1 && mem.of(n) > MaxRepeat { - return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.of(n), MaxRepeat) + if t, ok := n.(*template); ok && t.repeat > 1 && mem.renderCount(n) > MaxRepeat { + return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.renderCount(n), MaxRepeat) } return nil } diff --git a/struct.go b/struct.go index aeb5e2c..193255a 100644 --- a/struct.go +++ b/struct.go @@ -105,7 +105,7 @@ func (sc *structCompile) record(t reflect.Type, label string) (*structShape, err sc.visiting[t] = true defer delete(sc.visiting, t) c := &structFields{structCompile: sc, t: t, label: label, tags: map[string]any{}, shape: &structShape{}} - if err := c.walk(t, nil); err != nil { + if err := c.gatherFields(t, nil); err != nil { return nil, err } if len(c.tags) > 0 { @@ -123,19 +123,19 @@ func (sc *structCompile) spend(label string) error { return fmt.Errorf(`%s: the struct fields reach more than %d structs; leave a struct field unfilled with fake:"-"`, label, maxStructs) } -// walk gathers the fields of struct type t, which sits at index within c.t. -func (c *structFields) walk(t reflect.Type, index []int) error { +// gatherFields gathers the fields of struct type t, which sits at index within c.t. +func (c *structFields) gatherFields(t reflect.Type, index []int) error { for i := 0; i < t.NumField(); i++ { sf := t.Field(i) sf.Index = append(index[:len(index):len(index)], i) - if err := c.field(sf); err != nil { + if err := c.addField(sf); err != nil { return err } } return nil } -func (c *structFields) field(sf reflect.StructField) error { +func (c *structFields) addField(sf reflect.StructField) error { tag, tagged := sf.Tag.Lookup("fake") elem := structOf(sf.Type) switch { @@ -145,7 +145,7 @@ func (c *structFields) field(sf reflect.StructField) error { } return nil case tagged: - return c.column(sf, tag) + return c.addTag(sf, tag) case elem == nil || c.visiting[elem]: return nil case sf.Anonymous: @@ -167,8 +167,8 @@ func structOf(t reflect.Type) reflect.Type { return t } -// column adds a tagged field's tag to the record, refusing one another field hides. -func (c *structFields) column(sf reflect.StructField, tag string) error { +// addTag adds a tagged field's tag to the record, refusing one another field hides. +func (c *structFields) addTag(sf reflect.StructField, tag string) error { if visible, ok := c.t.FieldByName(sf.Name); !ok || !slices.Equal(visible.Index, sf.Index) { return fmt.Errorf("%s.%s: hidden by another field named %s, so its fake tag cannot fill it; rename one", c.label, fieldPath(c.t, sf.Index), sf.Name) } @@ -196,7 +196,7 @@ func (c *structFields) embed(sf reflect.StructField, elem reflect.Type) error { c.visiting[elem] = true defer delete(c.visiting, elem) tags, nested := len(c.tags), len(c.shape.nested) - if err := c.walk(elem, sf.Index); err != nil { + if err := c.gatherFields(elem, sf.Index); err != nil { return err } if sf.Type.Kind() == reflect.Pointer && !sf.IsExported() && (len(c.tags) > tags || len(c.shape.nested) > nested) { @@ -327,7 +327,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro elem := ft if ft.Kind() == reflect.Pointer { elem = ft.Elem() - } else if p.column(column).null { + } else if p.proveColumn(column).null { return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft) } kind := columnKinds[elem.Kind()] @@ -335,7 +335,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro return nil } for _, it := range items { - v := p.columnItem(it) + v := p.proveColumnItem(it) if reason := v.not[kind.datatype]; reason != "" { return fmt.Errorf("%s (%s): %s", label, ft, reason) } diff --git a/table.go b/table.go index 45471fb..43ee53c 100644 --- a/table.go +++ b/table.go @@ -58,7 +58,7 @@ type row struct{ t *table } func (*row) isNode() {} -func (t *table) rows() int { return len(t.cells) / len(t.columns) } +func (t *table) rowCount() int { return len(t.cells) / len(t.columns) } func (t *table) cell(row, col int) string { return t.cells[row*len(t.columns)+col] } @@ -89,7 +89,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab if err != nil { return nil, err } - data, err := files.rows(o.rows) + data, err := files.readRows(o.rows) if err != nil { return nil, err } @@ -184,7 +184,7 @@ func (t *table) parseRows(data string) error { } rest = more } - if t.rows() < 2 { + if t.rowCount() < 2 { return fmt.Errorf("has one row, which is a template; write it as one") } return nil @@ -240,8 +240,8 @@ func (t *table) proveNamesInsideParent() error { if t.key >= 0 || t.name < 0 { return nil } - inside := make(map[string]int, t.rows()) - for r := 0; r < t.rows(); r++ { + inside := make(map[string]int, t.rowCount()) + for r := 0; r < t.rowCount(); r++ { k := t.cell(r, t.parent) + "\t" + t.cell(r, t.name) if first, dup := inside[k]; dup { return fmt.Errorf("%s line %d: name %q repeats line %d inside %s %q; a name selects one row inside its parent; drop one, or add a key column", t.file, r+2, t.cell(r, t.name), first+2, t.columns[t.parent], t.cell(r, t.parent)) @@ -256,8 +256,8 @@ func (t *table) indexKeys() error { if t.key < 0 { return nil } - t.byKey = make(map[string]int, t.rows()) - for r := 0; r < t.rows(); r++ { + t.byKey = make(map[string]int, t.rowCount()) + for r := 0; r < t.rowCount(); r++ { k := t.cell(r, t.key) if k == "" { return fmt.Errorf("%s line %d: the key is empty", t.file, r+2) @@ -267,7 +267,7 @@ func (t *table) indexKeys() error { } t.byKey[k] = r } - for r := 0; r < t.rows() && t.name >= 0; r++ { + for r := 0; r < t.rowCount() && t.name >= 0; r++ { if n := t.cell(r, t.name); t.byKey[n] != r { if other, isKey := t.byKey[n]; isKey { return fmt.Errorf("%s line %d: name %q is the key of line %d, which a selector reads first, so the name could never select this row", t.file, r+2, n, other+2) @@ -282,7 +282,7 @@ func (t *table) sumWeights() error { if t.weight < 0 { return nil } - t.cum = make([]float64, t.rows()) + t.cum = make([]float64, t.rowCount()) total := 0.0 for r := range t.cum { w, err := strconv.ParseFloat(t.cell(r, t.weight), 64) @@ -419,15 +419,15 @@ func (t *table) linkParent(path string, siblings map[string]node) error { return fmt.Errorf("%q is named like a column of %q, its ancestor, so %s.%s could read either; rename one", t.category, q.category, q.category, t.category) } } - linked := make(map[string]bool, p.rows()) - for r := 0; r < t.rows(); r++ { + linked := make(map[string]bool, p.rowCount()) + for r := 0; r < t.rowCount(); r++ { k := t.cell(r, t.parent) if _, ok := p.byKey[k]; !ok { return fmt.Errorf("%s line %d: %s %q is no key of %s", t.file, r+2, name, k, p.file) } linked[k] = true } - for r := 0; r < p.rows(); r++ { + for r := 0; r < p.rowCount(); r++ { if k := p.cell(r, p.key); !linked[k] { return fmt.Errorf("%s links no row to %s %q; every %s row needs one, or drop line %d of %s", t.file, name, k, name, r+2, p.file) } @@ -443,15 +443,15 @@ func (t *table) linkParent(path string, siblings map[string]node) error { func (t *table) indexed() *tableIndex { t.once.Do(func() { if t.name >= 0 { - t.index.byName = make(map[string][]int, t.rows()) - for r := 0; r < t.rows(); r++ { + t.index.byName = make(map[string][]int, t.rowCount()) + for r := 0; r < t.rowCount(); r++ { n := t.cell(r, t.name) t.index.byName[n] = append(t.index.byName[n], r) } } if t.parent >= 0 { t.index.children = map[string][]int{} - for r := 0; r < t.rows(); r++ { + for r := 0; r < t.rowCount(); r++ { k := t.cell(r, t.parent) t.index.children[k] = append(t.index.children[k], r) } @@ -476,7 +476,7 @@ func (t *table) indexed() *tableIndex { // interface so that the walk that draws through it keeps its hold set off the heap. func (t *table) draw(s *session) int { if t.cum == nil { - return s.IntN(t.rows()) + return s.IntN(t.rowCount()) } return pickCum(s, t.cum) } diff --git a/table_test.go b/table_test.go index 99838c8..1875906 100644 --- a/table_test.go +++ b/table_test.go @@ -982,7 +982,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { } var none pinSet for _, entered := range tables { - for row := 0; row < entered.rows(); row++ { + for row := 0; row < entered.rowCount(); row++ { whole := drawAt{drawn: entered} whole.whole.add(entered, row) s := none.entered(entered, row) @@ -993,7 +993,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { t.Errorf("a render draws %s beside %s, which pinning it there refuses", cand.selectorSpelling(r), entered.selectorSpelling(row)) } } - for cr := 0; cr < cand.rows(); cr++ { + for cr := 0; cr < cand.rowCount(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil candAt := drawAt{pins: none.entered(cand, cr)} diff --git a/todo.md b/todo.md index 3e71e27..78b582a 100644 --- a/todo.md +++ b/todo.md @@ -19,11 +19,9 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Give `rows` and `field` one meaning each, per the decision "One name, one meaning": - `categoryFiles.rows` returns a TSV file name where `table.rows` counts rows, and - `formatOps.field` compiles a `{field}` token where `structFields.field` reads a Go - struct field. Rename every other shared name that names two things — `walk`, - `column`, `expr` and `of` included. +- Rename the shared names left that name two things, per the decision "One name, one + meaning": judge each of `check`, `draw`, `drawn`, `head`, `key`, `name`, `named`, + `names`, `null`, `number`, `record` and `variant`. - Run the nine-seat panel again and file what it names here. ### Data diff --git a/value.go b/value.go index 250bb25..2d94b83 100644 --- a/value.go +++ b/value.go @@ -37,23 +37,23 @@ func (p *valueProof) checkDatatype(path string, n node) error { if d := readDatatype(t); d == t.datatype { return fmt.Errorf(`%s: %s takes datatype %s from the column it reads; drop "datatype"`, path, t.format, d) } - if reason := p.columnItem(t).not[t.datatype]; reason != "" { + if reason := p.proveColumnItem(t).not[t.datatype]; reason != "" { return fmt.Errorf("%s: datatype %s: %s", path, t.datatype, reason) } return nil } -// columnItem proves a column item: what it renders, or, when it is a column it reads, that column. -func (p *valueProof) columnItem(t *template) proven { +// proveColumnItem proves a column item: what it renders, or, when it is a column it reads, that column. +func (p *valueProof) proveColumnItem(t *template) proven { if t.readsColumn == nil { - return p.of(t) + return p.prove(t) } - return p.column(t.readsColumn.column) + return p.proveColumn(t.readsColumn.column) } -// column proves a column over what its items draw, a null item marking it null rather than +// proveColumn proves a column over what its items draw, a null item marking it null rather than // rendering "". -func (p *valueProof) column(n node) proven { +func (p *valueProof) proveColumn(n node) proven { if v, done := p.columns[n]; done { return v } @@ -63,7 +63,7 @@ func (p *valueProof) column(n node) proven { items, nullable := columnItems(n) var v proven for i, it := range items { - if w := p.columnItem(it); i == 0 { + if w := p.proveColumnItem(it); i == 0 { v = w } else { v = v.or(w) @@ -74,7 +74,7 @@ func (p *valueProof) column(n node) proven { return v } -func (p *valueProof) of(n node) proven { +func (p *valueProof) prove(n node) proven { if v, done := p.memo[n]; done { return v } @@ -84,11 +84,11 @@ func (p *valueProof) of(n node) proven { var v proven switch n := n.(type) { case *choice: - v = p.unite(n.items) + v = p.proveUnion(n.items) case *template: - v = p.template(n) + v = p.proveTemplate(n) case *column: - v = p.cells(n) + v = p.proveCells(n) case *row: v = unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", n.t.format.format, n.t.category)) default: @@ -98,13 +98,13 @@ func (p *valueProof) of(n node) proven { return v } -// cells proves a table column over every cell it may render. -func (p *valueProof) cells(c *column) proven { +// proveCells proves a table column over every cell it may render. +func (p *valueProof) proveCells(c *column) proven { var v proven - for r := 0; r < c.t.rows(); r++ { + for r := 0; r < c.t.rowCount(); r++ { var w proven if cell := c.t.cellTemplate(r, c.i); cell != nil { - w = p.of(cell) + w = p.prove(cell) } else { w = literalValue(c.t.cell(r, c.i)) } @@ -117,10 +117,10 @@ func (p *valueProof) cells(c *column) proven { return v } -func (p *valueProof) unite(nodes []node) proven { - v := p.of(nodes[0]) +func (p *valueProof) proveUnion(nodes []node) proven { + v := p.prove(nodes[0]) for _, n := range nodes[1:] { - v = v.or(p.of(n)) + v = v.or(p.prove(n)) } return v } @@ -140,9 +140,9 @@ func (v proven) or(w proven) proven { return v } -// template proves a template that renders one value: fixed text, or a format that is +// proveTemplate proves a template that renders one value: fixed text, or a format that is // one token alone. -func (p *valueProof) template(t *template) proven { +func (p *valueProof) proveTemplate(t *template) proven { switch { case t.repeat != 1: return unproven(fmt.Sprintf("%q carries a repeat, which composes text rather than one value", t.format)) @@ -161,9 +161,9 @@ func (p *valueProof) template(t *template) proven { for _, a := range o.arms { leaves = append(leaves, pathLeaves(t.head(a.key), a.tail)...) } - return p.unite(leaves) + return p.proveUnion(leaves) case name == "calc": - return p.calc(t, body, args) + return p.proveCalc(t, body, args) case builtins[name].number != nil: return builtins[name].number(body, builtins[name].prints, args) case isTransform: @@ -172,12 +172,12 @@ func (p *valueProof) template(t *template) proven { return printing(body, DataTypeString, proven{notOperand: fmt.Sprintf("{%s} prints text, not a number", body)}) } -func (p *valueProof) calc(t *template, body string, args []string) proven { +func (p *valueProof) proveCalc(t *template, body string, args []string) proven { expr, err := parseCalc(args[0]) if err != nil { panic(fmt.Sprintf("fejkdata: calc(%q) reached a proof unparsed: %v", args[0], err)) } - v, doubt := p.expr(expr, t.fields) + v, doubt := p.proveExpr(expr, t.fields) if doubt == "" && !(magnitude(v) <= calcLimit) { doubt = calcText(expr) + " is not proven within 1e300" } @@ -191,28 +191,28 @@ func (p *valueProof) calc(t *template, body string, args []string) proven { // math.MaxFloat64 that rounding in the bounds cannot hide an overflow. const calcLimit = 1e300 -// expr bounds a calc expression from its operands, or says why it cannot. -func (p *valueProof) expr(n calcNode, fields map[string]node) (proven, string) { +// proveExpr bounds a calc expression from its operands, or says why it cannot. +func (p *valueProof) proveExpr(n calcNode, fields map[string]node) (proven, string) { switch n := n.(type) { case calcNum: v := float64(n) return bounded(v, v, v == math.Trunc(v)), "" case calcVar: - v := p.of(fields[string(n)]) + v := p.prove(fields[string(n)]) if v.notOperand != "" { return proven{}, fmt.Sprintf("operand %q: %s", string(n), v.notOperand) } return proven{lo: v.lo, hi: v.hi, nonZero: v.nonZero, integral: v.integral}, "" case calcNeg: - v, doubt := p.expr(n.x, fields) + v, doubt := p.proveExpr(n.x, fields) v.lo, v.hi = -v.hi, -v.lo return v, doubt case calcBin: - l, doubt := p.expr(n.l, fields) + l, doubt := p.proveExpr(n.l, fields) if doubt != "" { return l, doubt } - r, doubt := p.expr(n.r, fields) + r, doubt := p.proveExpr(n.r, fields) if doubt != "" { return r, doubt } -- 2.52.0 From cfe8124cc393f44f1bc2882aaff54ff5ea60505d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 18:57:41 +0200 Subject: [PATCH 115/966] Rename addField to gatherField and reachMemo to renderCounts, and list every shared name left --- data.go | 2 +- graph.go | 6 +++--- struct.go | 6 +++--- todo.md | 5 +++-- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/data.go b/data.go index 08dab0a..c49c23c 100644 --- a/data.go +++ b/data.go @@ -218,7 +218,7 @@ func checkScope(s nodeScope) error { // the walk. Its walks terminate only where nothing renders itself, so run it over a // loaded tree after checkNoCycles. func checkRenders(s nodeScope) error { - mem := reachMemo{} + mem := renderCounts{} if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { return err } diff --git a/graph.go b/graph.go index 0ff962f..9c017ce 100644 --- a/graph.go +++ b/graph.go @@ -169,9 +169,9 @@ func pathLeaves(n node, tail []string) []node { return out } -type reachMemo map[node]int +type renderCounts map[node]int -func (m reachMemo) renderCount(n node) int { +func (m renderCounts) renderCount(n node) int { if r, done := m[n]; done { return r } @@ -191,7 +191,7 @@ func (m reachMemo) renderCount(n node) int { // repeatCheck bounds the renders a repeat multiplies to along any root-to-leaf // path, so nested repeats cannot build what one repeat may not. // docs/decisions.md#the-repeat-cap-bounds-renders-not-bytes -func repeatCheck(path string, n node, mem reachMemo) error { +func repeatCheck(path string, n node, mem renderCounts) error { if t, ok := n.(*template); ok && t.repeat > 1 && mem.renderCount(n) > MaxRepeat { return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.renderCount(n), MaxRepeat) } diff --git a/struct.go b/struct.go index 193255a..51f69e6 100644 --- a/struct.go +++ b/struct.go @@ -123,19 +123,19 @@ func (sc *structCompile) spend(label string) error { return fmt.Errorf(`%s: the struct fields reach more than %d structs; leave a struct field unfilled with fake:"-"`, label, maxStructs) } -// gatherFields gathers the fields of struct type t, which sits at index within c.t. +// t sits at index within c.t. func (c *structFields) gatherFields(t reflect.Type, index []int) error { for i := 0; i < t.NumField(); i++ { sf := t.Field(i) sf.Index = append(index[:len(index):len(index)], i) - if err := c.addField(sf); err != nil { + if err := c.gatherField(sf); err != nil { return err } } return nil } -func (c *structFields) addField(sf reflect.StructField) error { +func (c *structFields) gatherField(sf reflect.StructField) error { tag, tagged := sf.Tag.Lookup("fake") elem := structOf(sf.Type) switch { diff --git a/todo.md b/todo.md index 78b582a..e8b008c 100644 --- a/todo.md +++ b/todo.md @@ -20,8 +20,9 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: - Rename the shared names left that name two things, per the decision "One name, one - meaning": judge each of `check`, `draw`, `drawn`, `head`, `key`, `name`, `named`, - `names`, `null`, `number`, `record` and `variant`. + meaning": judge each of `check`, `children`, `column`, `columns`, `draw`, `drawn`, + `fields`, `format`, `head`, `index`, `key`, `n`, `name`, `named`, `names`, `null`, + `number`, `op`, `reads`, `record`, `root`, `row`, `t`, `variant` and `whole`. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From d6ac8ed9fd53296296298b6293b8601bb9810bae Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 19:10:18 +0200 Subject: [PATCH 116/966] Give check, children, column, columns, draw, drawn, fields, format, head, index, key, n, name, named, names, null, number, op, reads, record, root, row, t and whole one meaning each --- AGENTS.md | 2 +- builtins.go | 32 ++++---- builtins_test.go | 2 +- calc.go | 16 ++-- data.go | 24 +++--- datatype.go | 4 +- doc.go | 6 +- docs/decisions.md | 4 +- drawfence.go | 46 +++++------ family.go | 36 ++++----- fejkdata.go | 8 +- format.go | 36 ++++----- graph.go | 22 +++--- hold.go | 10 +-- hold_test.go | 2 +- holdfence.go | 12 +-- node.go | 14 ++-- path.go | 14 ++-- pins.go | 16 ++-- record.go | 32 ++++---- reference.go | 2 +- render.go | 12 +-- shape_test.go | 10 +-- struct.go | 50 ++++++------ table.go | 190 +++++++++++++++++++++++----------------------- table_test.go | 8 +- todo.md | 4 - transform.go | 2 +- value.go | 34 ++++----- 29 files changed, 324 insertions(+), 326 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6f7c33c..cd7bb56 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,4 +110,4 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only -- Each entry point to `drawCheck.reads` says what its caller gets +- Each entry point to `drawCheck.hasRead` says what its caller gets diff --git a/builtins.go b/builtins.go index d960701..dd8e72d 100644 --- a/builtins.go +++ b/builtins.go @@ -28,44 +28,44 @@ var builtins = withTransforms(map[string]builtin{ "ean": {arity: 0, prep: derive(eanCheck)}, "uuid": {arity: 0, prep: sample(uuidV7)}, "ulid": {arity: 0, prep: sample(ulid)}, - "nanoid": {arity: 1, check: posIntArg, prep: chars(nanoidAlphabet)}, - "hex": {arity: 1, check: posIntArg, prep: chars(hexDigits)}, - "digits": {arity: 1, check: posIntArg, prep: chars("0123456789"), prints: DataTypeString, number: func(token string, prints DataType, a []string) proven { + "nanoid": {arity: 1, checkArgs: posIntArg, prep: chars(nanoidAlphabet)}, + "hex": {arity: 1, checkArgs: posIntArg, prep: chars(hexDigits)}, + "digits": {arity: 1, checkArgs: posIntArg, prep: chars("0123456789"), prints: DataTypeString, proveNumber: func(token string, prints DataType, a []string) proven { return printing(token, prints, bounded(0, math.Pow(10, float64(atoi(a[0])))-1, true)) }}, - "upper": {arity: 1, check: posIntArg, prep: chars("ABCDEFGHIJKLMNOPQRSTUVWXYZ")}, - "lower": {arity: 1, check: posIntArg, prep: chars("abcdefghijklmnopqrstuvwxyz")}, - "base64": {arity: 1, check: posIntArg, prep: func(a []string) callFn { + "upper": {arity: 1, checkArgs: posIntArg, prep: chars("ABCDEFGHIJKLMNOPQRSTUVWXYZ")}, + "lower": {arity: 1, checkArgs: posIntArg, prep: chars("abcdefghijklmnopqrstuvwxyz")}, + "base64": {arity: 1, checkArgs: posIntArg, prep: func(a []string) callFn { n := atoi(a[0]) return func(s *session, _ string, _ []string) string { return base64.StdEncoding.EncodeToString(randBytes(s, n)) } }}, - "int": {arity: 2, check: intRangeArgs, prep: func(a []string) callFn { + "int": {arity: 2, checkArgs: intRangeArgs, prep: func(a []string) callFn { lo, span := atoi(a[0]), atoi(a[1])-atoi(a[0])+1 return func(s *session, _ string, _ []string) string { return strconv.Itoa(lo + s.IntN(span)) } - }, prints: DataTypeInteger, number: func(token string, prints DataType, a []string) proven { + }, prints: DataTypeInteger, proveNumber: func(token string, prints DataType, a []string) proven { return printing(token, prints, bounded(float64(atoi(a[0])), float64(atoi(a[1])), true)) }}, - "float": {arity: 3, check: floatArgs, prep: func(a []string) callFn { + "float": {arity: 3, checkArgs: floatArgs, prep: func(a []string) callFn { lo, hi, dp := atof(a[0]), atof(a[1]), atoi(a[2]) return func(s *session, _ string, _ []string) string { return formatFloat(lo+s.Float64()*(hi-lo), dp) } - }, prints: DataTypeNumber, number: func(token string, _ DataType, a []string) proven { + }, prints: DataTypeNumber, proveNumber: func(token string, _ DataType, a []string) proven { return printedNumber(token, bounded(atof(a[0]), atof(a[1]), false), atoi(a[2])) }}, - "iban": {arity: 1, check: ibanArg, prep: func(a []string) callFn { + "iban": {arity: 1, checkArgs: ibanArg, prep: func(a []string) callFn { cc := a[0] return func(s *session, _ string, _ []string) string { return iban(s, cc) } }}, - "date": {arity: -1, check: dateArgs, prep: datePrep}, - "time": {arity: -1, check: timeArg, prep: timePrep}, - "calc": {arity: -1, check: checkCalc, prep: calcPrep, operands: calcOperands}, + "date": {arity: -1, checkArgs: dateArgs, prep: datePrep}, + "time": {arity: -1, checkArgs: timeArg, prep: timePrep}, + "calc": {arity: -1, checkArgs: checkCalc, prep: calcPrep, operands: calcOperands}, // seq is the one stateful builtin: a per-session counter from 1, advancing on // each call. An optional name selects an independent counter; no name uses the // default one. Deterministic by construction, so seeded output stays stable. - "seq": {arity: -1, check: seqArg, prep: func(a []string) callFn { + "seq": {arity: -1, checkArgs: seqArg, prep: func(a []string) callFn { key := "" if len(a) == 1 { key = a[0] @@ -73,7 +73,7 @@ var builtins = withTransforms(map[string]builtin{ return func(s *session, _ string, _ []string) string { return strconv.FormatUint(s.next(key), 10) } - }, prints: DataTypeInteger, number: func(token string, prints DataType, _ []string) proven { + }, prints: DataTypeInteger, proveNumber: func(token string, prints DataType, _ []string) proven { return printing(token, prints, bounded(1, math.MaxInt64, true)) }}, }) diff --git a/builtins_test.go b/builtins_test.go index 8d419fe..b19c95d 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -194,7 +194,7 @@ func TestRegistryShapes(t *testing.T) { if b.prep == nil { t.Errorf("builtin %q has no prep: compileOps would call a nil func", name) } - if b.arity != 0 && b.check == nil { + if b.arity != 0 && b.checkArgs == nil { t.Errorf("builtin %q parses args in prep with no check", name) } } diff --git a/calc.go b/calc.go index eb0c2c4..a279978 100644 --- a/calc.go +++ b/calc.go @@ -19,8 +19,8 @@ type calcVar string // a sibling-field name, before indexVars places it type calcIdx int // an operand, by its position in the values expand read type calcNeg struct{ x calcNode } type calcBin struct { // a + - * / b - op byte - l, r calcNode + operator byte + l, r calcNode } func (n calcNum) eval([]string) float64 { return float64(n) } @@ -45,7 +45,7 @@ func (n calcNeg) eval(operands []string) float64 { return -n.x.eval(operands) } func (n calcBin) eval(operands []string) float64 { l, r := n.l.eval(operands), n.r.eval(operands) - switch n.op { + switch n.operator { case '+': return l + r case '-': @@ -100,7 +100,7 @@ func constantZeroDivisor(n calcNode, fields map[string]node) (string, bool) { case calcNeg: return constantZeroDivisor(n.x, fields) case calcBin: - if n.op == '/' { + if n.operator == '/' { if v, known := constantValue(n.r, fields); known && v == 0 { return calcText(n.r), true } @@ -134,7 +134,7 @@ func constantValue(n calcNode, fields map[string]node) (float64, bool) { if !lok || !rok { return 0, false } - return calcBin{n.op, calcNum(l), calcNum(r)}.eval(nil), true + return calcBin{n.operator, calcNum(l), calcNum(r)}.eval(nil), true } return 0, false } @@ -149,7 +149,7 @@ func calcText(n calcNode) string { case calcNeg: return "-" + calcText(n.x) case calcBin: - return "(" + calcText(n.l) + " " + string(n.op) + " " + calcText(n.r) + ")" + return "(" + calcText(n.l) + " " + string(n.operator) + " " + calcText(n.r) + ")" } return "?" } @@ -158,7 +158,7 @@ func calcText(n calcNode) string { // does not parse, or a choice of only such items. text is one such render. func neverNumeric(n node) (text string, never bool) { switch n := n.(type) { - case *null: + case *nullItem: return "", true case *template: if !n.fixed || n.repeat > 1 { @@ -220,7 +220,7 @@ func indexVars(n calcNode, at map[string]int) calcNode { case calcNeg: return calcNeg{indexVars(n.x, at)} case calcBin: - return calcBin{n.op, indexVars(n.l, at), indexVars(n.r, at)} + return calcBin{n.operator, indexVars(n.l, at), indexVars(n.r, at)} } return n } diff --git a/data.go b/data.go index c49c23c..3989539 100644 --- a/data.go +++ b/data.go @@ -17,10 +17,10 @@ type dataSource struct { label string onDisk bool path string - root string + dir string } -func (s dataSource) name(p string) string { +func (s dataSource) labelled(p string) string { if s.label == "" { return p } @@ -49,7 +49,7 @@ func loadData(sources []dataSource) (map[string]node, error) { return nil, fmt.Errorf("%s is not a directory", src.path) } } - dir := src.root + dir := src.dir if dir == "" { dir = "." } @@ -84,7 +84,7 @@ func loadData(sources []dataSource) (map[string]node, error) { func loadDir(src dataSource, dir string) (*folder, error) { entries, err := fs.ReadDir(src.fsys, dir) if err != nil { - return nil, fmt.Errorf("%s: %w", src.name(dir), err) + return nil, fmt.Errorf("%s: %w", src.labelled(dir), err) } g := &folder{children: map[string]node{}} files := &categoryFiles{src: src, dir: dir, tsv: map[string]bool{}} @@ -110,7 +110,7 @@ func loadDir(src dataSource, dir string) (*folder, error) { } for name, named := range files.tsv { if !named && !strings.HasPrefix(name, ".") { - return nil, fmt.Errorf("%s: no category names it in its rows; a table's rows file sits beside a category file naming it", src.name(path.Join(dir, name))) + return nil, fmt.Errorf("%s: no category names it in its rows; a table's rows file sits beside a category file naming it", src.labelled(path.Join(dir, name))) } } return g, nil @@ -127,12 +127,12 @@ type categoryFiles struct { // readRows reads the rows file a category names beside it. func (c *categoryFiles) readRows(name string) (string, error) { if _, present := c.tsv[name]; !present { - return "", fmt.Errorf("rows names %s, which is not beside it in %s", name, c.src.name(c.dir)) + return "", fmt.Errorf("rows names %s, which is not beside it in %s", name, c.src.labelled(c.dir)) } c.tsv[name] = true b, err := fs.ReadFile(c.src.fsys, path.Join(c.dir, name)) if err != nil { - return "", fmt.Errorf("%s: %w", c.src.name(path.Join(c.dir, name)), err) + return "", fmt.Errorf("%s: %w", c.src.labelled(path.Join(c.dir, name)), err) } return string(b), nil } @@ -147,7 +147,7 @@ func loadFolder(src dataSource, g *folder, full, name string) error { return nil } if err := checkName(name); err != nil { - return fmt.Errorf("%s: folder %w", src.name(full), err) + return fmt.Errorf("%s: folder %w", src.labelled(full), err) } g.children[name] = child return nil @@ -161,19 +161,19 @@ func loadFile(src dataSource, g *folder, full, file string, files *categoryFiles } name := strings.TrimSuffix(file, ".json") if err := checkName(name); err != nil { - return fmt.Errorf("%s: category %w", src.name(full), err) + return fmt.Errorf("%s: category %w", src.labelled(full), err) } b, err := fs.ReadFile(src.fsys, full) if err != nil { - return fmt.Errorf("%s: %w", src.name(full), err) + return fmt.Errorf("%s: %w", src.labelled(full), err) } var raw any if err := json.Unmarshal(b, &raw); err != nil { - return fmt.Errorf("%s: %w", src.name(full), err) + return fmt.Errorf("%s: %w", src.labelled(full), err) } n, err := compileCategory(raw, name, files) if err != nil { - return fmt.Errorf("%s: %w", src.name(full), err) + return fmt.Errorf("%s: %w", src.labelled(full), err) } g.children[name] = n return nil diff --git a/datatype.go b/datatype.go index 31e1a0a..b8f4610 100644 --- a/datatype.go +++ b/datatype.go @@ -89,7 +89,7 @@ func checkColumns(s nodeScope) error { } return s(func(path string, n node) error { t, ok := n.(*template) - if !ok || !t.record { + if !ok || !t.isRecord { return nil } for _, name := range sortedNames(t.fields) { @@ -149,7 +149,7 @@ func columnItems(n node) (items []*template, nullable bool) { } case *template: items = append(items, n) - case *null: + case *nullItem: nullable = true } } diff --git a/doc.go b/doc.go index 7b39821..eb4017f 100644 --- a/doc.go +++ b/doc.go @@ -17,7 +17,7 @@ package fejkdata // Vocabulary // // - draw — one pick from a choice, or one row taken from a table, and the value a -// read produced: `pick`, `drawn`, `table.draw`, `draw`. +// read produced: `pick`, `resolveChoice`, `table.drawRow`, `draw`. // - expansion — one render of one format: `expand`. // - render — what owns one `holdSet`, and so what one reference draw spans: // `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` @@ -32,8 +32,8 @@ package fejkdata // `pinSet`, `pinSet.pin`, `table.drawIn`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. -// - whole — a read that lands on a `row`, so the row renders through its table's -// format: `tableRead.whole` says a read did, `table.whole` is the node. +// - whole — a read that lands on a `tableRow`, so the row renders through its table's +// format: `tableRead.landsWhole` says a read did, `table.wholeRow` is the node. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those // compile. The draw fences are `heldCheck`, `drawCheck` and `checkFamilies`. diff --git a/docs/decisions.md b/docs/decisions.md index a50f7d6..2389f2e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -650,8 +650,8 @@ import drops them by name rather than by a rank a regeneration would move. the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. -## Each entry point to `drawCheck.reads` says what its caller gets +## Each entry point to `drawCheck.hasRead` says what its caller gets 2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on `readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, -though `reads` states the same stop rule. Valid while the two differ in where they stop. +though `hasRead` states the same stop rule. Valid while the two differ in where they stop. diff --git a/drawfence.go b/drawfence.go index aebaa52..d1ebc31 100644 --- a/drawfence.go +++ b/drawfence.go @@ -41,10 +41,10 @@ func checkNestedDrawGroup(fields map[string]node, group string) error { // docs/decisions.md#a-render-shares-one-reference-draw-per-category-per-group // docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences type drawCheck struct { - memo map[readsMemo]bool + memo map[hasReadMemo]bool } -type readsMemo struct { +type hasReadMemo struct { n node stopAtGroup bool } @@ -84,7 +84,7 @@ func (c *drawCheck) checkDraws(path string, n node) error { // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load func (c *drawCheck) checkRecordDraws(path string, n node) error { t, ok := n.(*template) - if !ok || !t.record { + if !ok || !t.isRecord { return nil } // A record-only template's format renders nothing, so weigh the columns, not the format. @@ -104,28 +104,28 @@ func (c *drawCheck) checkRecordDraws(path string, n node) error { // readsPath reports whether rendering n reads a reference path, short of a repeat, which renders // over draws of its own. -func (c *drawCheck) readsPath(n node) bool { return c.reads(n, false) } +func (c *drawCheck) readsPath(n node) bool { return c.hasRead(n, false) } // splitsDraws reports whether rendering n reads a reference path that n's own draw group answers // for: one outside a repeat and outside a nested draw group, which hold their own draws. -func (c *drawCheck) splitsDraws(n node) bool { return c.reads(n, true) } +func (c *drawCheck) splitsDraws(n node) bool { return c.hasRead(n, true) } -// reads walks what rendering n renders for a reference path, stopping at a repeat — and at a nested +// hasRead walks what rendering n renders for a reference path, stopping at a repeat — and at a nested // draw group when stopAtGroup — since each holds draws of its own. -func (c *drawCheck) reads(n node, stopAtGroup bool) bool { - k := readsMemo{n, stopAtGroup} +func (c *drawCheck) hasRead(n node, stopAtGroup bool) bool { + k := hasReadMemo{n, stopAtGroup} if r, done := c.memo[k]; done { return r } r := false for _, e := range renderEdges(n) { - if readsOnEdge(n, e.label) || (walksInto(e.to, stopAtGroup) && c.reads(e.to, stopAtGroup)) { + if readsOnEdge(n, e.label) || (walksInto(e.to, stopAtGroup) && c.hasRead(e.to, stopAtGroup)) { r = true break } } if c.memo == nil { - c.memo = map[readsMemo]bool{} + c.memo = map[hasReadMemo]bool{} } c.memo[k] = r return r @@ -191,11 +191,11 @@ type drawWalk struct { // table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose // cells the walk is in. type drawAt struct { - group string - route drawRoute - pins pinSet - drawn *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true - whole pinSet + group string + route drawRoute + pins pinSet + wholeTable *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true + wholePins pinSet } // drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. @@ -239,17 +239,17 @@ func (w *drawWalk) walk(n node, at drawAt) { at.group = t.drawGroupKey } if t, isTable := n.(*table); isTable { - at.drawn = t + at.wholeTable = t } for _, e := range renderEdges(n) { cell, isCell := e.to.(*template) switch { case !isCell || cell.cellOf == nil: w.edge(n, e, at) - case cell.cellOf == at.drawn: + case cell.cellOf == at.wholeTable: in := at - in.whole = at.whole.clone() - in.whole.add(cell.cellOf, cell.cellRow) + in.wholePins = at.wholePins.clone() + in.wholePins.add(cell.cellOf, cell.cellRow) w.edge(n, e, in) case at.pins.clash(cell.cellOf, cell.cellRow) == nil: in := at @@ -260,7 +260,7 @@ func (w *drawWalk) walk(n node, at drawAt) { } // rowsKey spells the rows the walk stands in, pinned and whole, for a map. -func (at drawAt) rowsKey() string { return at.pins.key() + "|" + at.whole.key() } +func (at drawAt) rowsKey() string { return at.pins.mapKey() + "|" + at.wholePins.mapKey() } // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. @@ -293,8 +293,8 @@ func (w *drawWalk) check() error { if into.at.group != level.at.group || alternatives(level.at, into.at) { continue } - if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.whole) { - return overlapError(level.at.route, level.a.name, into) + if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsWhole) { + return overlapError(level.at.route, level.a.spelling, into) } } } @@ -302,7 +302,7 @@ func (w *drawWalk) check() error { } func overlapError(route drawRoute, ref string, into pathRead) error { - return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name)) + return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) } // spelled names the route, and the reference it reaches a draw by where its root edge is not that diff --git a/family.go b/family.go index 4893597..a8003bf 100644 --- a/family.go +++ b/family.go @@ -10,11 +10,11 @@ import ( // row pinned, and their unpinned ancestors — each selector's spelling, and whether // it lands on a row rendered whole. type tableRead struct { - head *table - pins pinSet - drawn map[*table]bool - sels []tableSel - whole bool + headTable *table + pins pinSet + drawn map[*table]bool + sels []tableSel + landsWhole bool } // tableSel is one selector on the way: the table it selects a row of, and the path @@ -32,9 +32,9 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { if !isTable { return nil } - tr := &tableRead{head: t, drawn: map[*table]bool{}} + tr := &tableRead{headTable: t, drawn: map[*table]bool{}} _, _ = walkPath(t, a.tail, pathWalk{mode: walkProbe, pins: &tr.pins, drawn: tr.drawn}) - written := a.name[:len(a.name)-len(joinSegments(a.tail))] + written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] cur := t for i, seg := range a.tail { switch d := cur.descendant(seg); { @@ -44,7 +44,7 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { cur = d } } - _, tr.whole = leaf.(*row) + _, tr.landsWhole = leaf.(*tableRow) return tr } @@ -91,7 +91,7 @@ func checkFamilies(reads []pathRead) error { continue } for _, o := range reads[:i] { - if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.head.family() != r.tr.head.family() { + if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.family() != r.tr.headTable.family() { continue } if err := checkFamilyPair(o, r); err != nil { @@ -127,7 +127,7 @@ func replayPairs(reads []pathRead) error { } func conflict(r pathRead, err error) error { - return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.name), err) + return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) } // replay pins the read's rows into d, where they agree with the rows pinned before. @@ -149,19 +149,19 @@ func checkFamilyPair(a, b pathRead) error { for _, pair := range [][2]pathRead{{a, b}, {b, a}} { x, y := pair[0], pair[1] if len(x.a.tail) == 0 && len(y.a.tail) > 0 { - return overlapError(x.at.route, x.a.name, y) + return overlapError(x.at.route, x.a.spelling, y) } if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { - if s, ok := y.tr.selected(x.tr.head); ok && len(x.tr.sels) == 0 { + if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { tail := x.a.tail - if s.t != x.tr.head { - tail = append([]string{x.tr.head.category}, tail...) + if s.t != x.tr.headTable { + tail = append([]string{x.tr.headTable.category}, tail...) } return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.name), drawn.category, y.at.route.spelled(y.a.name), s.spelling, joinSegments(tail)) + x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) } return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.name), drawn.category, y.at.route.spelled(y.a.name)) + x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling)) } } return nil @@ -219,4 +219,6 @@ func checkOwnFamily(t *template) error { // alternatives reports whether two reads sit in different rows of one table: rows the walks pinned, // or rows of one whole draw. // docs/decisions.md#the-rows-of-a-table-are-alternatives -func alternatives(a, b drawAt) bool { return a.pins.differs(&b.pins) || a.whole.differs(&b.whole) } +func alternatives(a, b drawAt) bool { + return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) +} diff --git a/fejkdata.go b/fejkdata.go index f476679..ea89e72 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -99,7 +99,7 @@ func New(opts ...Option) (*Generator, error) { } var sources []dataSource if c.shipped { - sources = append(sources, dataSource{fsys: shippedFS, root: "data"}) + sources = append(sources, dataSource{fsys: shippedFS, dir: "data"}) } sources = append(sources, c.sources...) if len(sources) == 0 { @@ -151,11 +151,11 @@ func paths(n node) []string { } } return out - case *null: + case *nullItem: return []string{""} case *table: return tablePaths(n) - case *column, *row: + case *tableColumn, *tableRow: return []string{""} case *choice: out := []string{""} @@ -170,7 +170,7 @@ func paths(n node) []string { // tablePaths is a table's columns, then each table linked to it under its name: the // direct descents, a step at a time. func tablePaths(t *table) []string { - out := append([]string{""}, t.columns...) + out := append([]string{""}, t.header...) sort.Strings(out[1:]) children := make([]string, 0, len(t.children)) for name := range t.children { diff --git a/format.go b/format.go index 7224bce..fd3635a 100644 --- a/format.go +++ b/format.go @@ -109,19 +109,19 @@ func parseFormat(format string) ([]formatToken, error) { // the values of the operands it named (calc and the transforms name them). All must // stay pure over (rng, emitted, args) so seeded output is reproducible; seq advances // per-session counter state, which is itself deterministic. arity is the exact arg -// count, or -1 for variadic (then check does all the validation). +// count, or -1 for variadic (then checkArgs does all the validation). type builtin struct { arity int // prep parses validated args once, at compile time, into the closure expand calls. - prep func(args []string) callFn - check func(fields map[string]node, args []string) error + prep func(args []string) callFn + checkArgs func(fields map[string]node, args []string) error // operands names the fields the call reads, which expand renders for it; nil // for a builtin that reads none. operands func(args []string) []string - // number proves what a call prints, token its body: the bounds of its number and the + // proveNumber proves what a call prints, token its body: the bounds of its number and the // datatype of its text, prints or one inside it; nil for a builtin whose text is no number. - number func(token string, prints DataType, args []string) proven - prints DataType + proveNumber func(token string, prints DataType, args []string) proven + prints DataType } // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false @@ -178,8 +178,8 @@ func checkFunc(tok formatToken, fields map[string]node) error { if b.arity >= 0 && len(args) != b.arity { return fmt.Errorf("token {%s}: %s takes %d argument%s, got %d", body, name, b.arity, plural(b.arity), len(args)) } - if b.check != nil { - if err := b.check(fields, args); err != nil { + if b.checkArgs != nil { + if err := b.checkArgs(fields, args); err != nil { return fmt.Errorf("token {%s}: %w", body, err) } } @@ -297,11 +297,11 @@ func checkNoRepeatedArm(body string, names []string) error { // is what makes the arm a bound draw: its head is drawn once per expansion (see // compileOps). type arm struct { - name string // as written, for messages - key string - tail []string - levels []string // the key at each level the tail passes through, the head's first - path string // key and tail, the one spelling every way of writing this read shares + spelling string // as written, for messages + key string + tail []string + levels []string // the key at each level the tail passes through, the head's first + path string // key and tail, the one spelling every way of writing this read shares } // splitArm splits one name into key and tail. refs maps a reference to what @@ -314,13 +314,13 @@ func splitArm(name string, refs map[string]refBinding) arm { if bound { key = b.key } - return arm{name: name, key: key, path: key} + return arm{spelling: name, key: key, path: key} } return pathArm(name, b.key, b.tail) } segs, err := splitPath(name) if err != nil || len(segs) == 1 { - return arm{name: name, key: name, path: name} + return arm{spelling: name, key: name, path: name} } return pathArm(name, segs[0], segs[1:]) } @@ -330,7 +330,7 @@ func pathArm(name, key string, segs []string) arm { for i := 0; i < len(segs)-1; i++ { levels = append(levels, key+"."+strings.Join(segs[:i+1], ".")) } - return arm{name: name, key: key, tail: segs, levels: levels, path: key + "." + strings.Join(segs, ".")} + return arm{spelling: name, key: key, tail: segs, levels: levels, path: key + "." + strings.Join(segs, ".")} } // checkSegments rejects an unfinished path: "{a.}" and "{a..b}" each have a @@ -400,7 +400,7 @@ func (c *formatOps) holdName(a arm, label string) { c.bound = map[string]string{} } if _, named := c.bound[a.key]; !named { - c.bound[a.key] = a.name + c.bound[a.key] = a.spelling } } } @@ -420,7 +420,7 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { for i, name := range tok.names { arms[i] = splitArm(name, refs) if len(arms[i].tail) > 0 { - c.holdName(arms[i], "token {"+arms[i].name+"}") + c.holdName(arms[i], "token {"+arms[i].spelling+"}") } } c.ops = append(c.ops, op{formatToken: tok, arms: arms}) diff --git a/graph.go b/graph.go index 9c017ce..ec34bee 100644 --- a/graph.go +++ b/graph.go @@ -51,7 +51,7 @@ type namedNode struct { func contained(n node) []namedNode { switch n := n.(type) { case *folder: - return named(n.children) + return namedNodes(n.children) case *choice: out := make([]namedNode, len(n.items)) for i, it := range n.items { @@ -59,10 +59,10 @@ func contained(n node) []namedNode { } return out case *template: - return named(n.fields) + return namedNodes(n.fields) case *table: - return append([]namedNode{{node: n.format}}, named(n.fields)...) - case *column: + return append([]namedNode{{node: n.formatTemplate}}, namedNodes(n.fields)...) + case *tableColumn: if len(n.t.cellTemplates) == 0 { return nil } @@ -78,7 +78,7 @@ func contained(n node) []namedNode { } } -func named(m map[string]node) []namedNode { +func namedNodes(m map[string]node) []namedNode { out := make([]namedNode, 0, len(m)) for _, name := range sortedNames(m) { out = append(out, namedNode{name: name, node: m[name]}) @@ -133,7 +133,7 @@ func renderEdges(n node) []renderEdge { return } for _, leaf := range pathLeaves(c, a.tail) { - es = append(es, renderEdge{leaf, a.name, operand}) + es = append(es, renderEdge{leaf, a.spelling, operand}) } } for _, o := range n.ops { @@ -146,13 +146,13 @@ func renderEdges(n node) []renderEdge { } return es case *table: - return []renderEdge{{to: n.format, label: "format"}} - case *row: - return []renderEdge{{to: n.t.format, label: "format"}} - case *column: + return []renderEdge{{to: n.formatTemplate, label: "format"}} + case *tableRow: + return []renderEdge{{to: n.t.formatTemplate, label: "format"}} + case *tableColumn: var es []renderEdge for _, c := range contained(n) { - es = append(es, renderEdge{to: c.node, label: n.t.columns[n.i]}) + es = append(es, renderEdge{to: c.node, label: n.t.header[n.i]}) } return es default: diff --git a/hold.go b/hold.go index 7d7ccd7..d8f00f5 100644 --- a/hold.go +++ b/hold.go @@ -29,7 +29,7 @@ type draw struct { func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { if !t.held[a.key] { if len(a.tail) > 0 { - panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key)) + panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) } return draw{text: render(s, t.head(a.key), sc)} } @@ -58,8 +58,8 @@ func readHold(held *hold, sc renderScope, a arm) *hold { // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. func renderLeaf(s *session, n node, sc renderScope) draw { - n = drawn(s, n) - if _, isNull := n.(*null); isNull { + n = resolveChoice(s, n) + if _, isNull := n.(*nullItem); isNull { return draw{null: true} } r := draw{text: render(s, n, sc)} @@ -69,10 +69,10 @@ func renderLeaf(s *session, n node, sc renderScope) draw { return r } -// drawn resolves a choice to one variant, so a bound head is a concrete node the +// resolveChoice resolves a choice to one variant, so a bound head is a concrete node the // rest of the expansion shares. Nested choices unwrap too: a draw is one value, not // another set to pick from. -func drawn(s *session, n node) node { +func resolveChoice(s *session, n node) node { for c, ok := n.(*choice); ok; c, ok = n.(*choice) { n = pick(s, c) } diff --git a/hold_test.go b/hold_test.go index 301feca..06c691e 100644 --- a/hold_test.go +++ b/hold_test.go @@ -766,6 +766,6 @@ func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) { t.Fatal("not a template") } mustPanic(t, "unheld arm with a path", func() { - readField(engine(1).rand, tm, nil, renderScope{}, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) + readField(engine(1).rand, tm, nil, renderScope{}, arm{spelling: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) }) } diff --git a/holdfence.go b/holdfence.go index bef29f8..c547cde 100644 --- a/holdfence.go +++ b/holdfence.go @@ -56,7 +56,7 @@ func heldNodes(t *template, head string, readers []reader) map[node]bool { return held } for _, r := range readers { - if a := splitArm(r.name, t.refs); a.key == head { + if a := splitArm(r.spelling, t.refs); a.key == head { coverPath(t.head(head), a.tail, held) } } @@ -181,7 +181,7 @@ func checkNoOverlap(ops []op, bound map[string]string) error { for i, level := range names { for _, path := range names[i+1:] { if strings.HasPrefix(path.path, level.path+".") { - return fmt.Errorf("%s renders a level that {%s} reads a path into; name the fields you want instead", level.label, path.name) + return fmt.Errorf("%s renders a level that {%s} reads a path into; name the fields you want instead", level.label, path.spelling) } } } @@ -190,7 +190,7 @@ func checkNoOverlap(ops []op, bound map[string]string) error { // reader is one way a format reaches a bound field: as written, by its one // spelling, and how to name it. -type reader struct{ name, path, label string } +type reader struct{ spelling, path, label string } // boundReaders lists every way a format reaches a bound sibling field, in the order the // format writes them. An operand renders its field, so it names a level exactly @@ -200,12 +200,12 @@ func boundReaders(ops []op, bound map[string]string) []reader { for _, o := range ops { for _, a := range o.operands { if _, isBound := bound[a.key]; isBound && !isRef(a.key) { - names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", o.fn, a.name)}) + names = append(names, reader{a.spelling, a.path, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { if _, isBound := bound[a.key]; isBound && !isRef(a.key) { - names = append(names, reader{a.name, a.path, "token {" + a.name + "}"}) + names = append(names, reader{a.spelling, a.path, "token {" + a.spelling + "}"}) } } } @@ -224,7 +224,7 @@ func checkNoRepeatedRead(c formatOps) error { continue } if count[a.key]++; count[a.key] > 1 { - return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.name, c.holder[a.key], a.key, a.name) + return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.spelling, c.holder[a.key], a.key, a.spelling) } } } diff --git a/node.go b/node.go index cc84854..dff687b 100644 --- a/node.go +++ b/node.go @@ -33,10 +33,10 @@ type choice struct { func (*choice) isNode() {} -// null is a column's missing value, rendered ""; sized so two nulls are two map keys. -type null struct{ _ byte } +// nullItem is a column's missing value, rendered ""; sized so two nulls are two map keys. +type nullItem struct{ _ byte } -func (*null) isNode() {} +func (*nullItem) isNode() {} // template renders a format string, substituting {tokens} from fields. A bare // JSON string is a template with no fields. repeat (default 1) renders that format @@ -52,7 +52,7 @@ type template struct { datatype DataType drawGroup string // the draw group it draws in, as written; "" keeps its caller's fromString bool // written as a JSON string rather than an object - record bool // compiled at the top without a repeat, so its fields are record columns + isRecord bool // compiled at the top without a repeat, so its fields are record columns table *table // the table whose format this is, whose columns are the fields // Filled by `checkCells`, after the cell's own format has compiled: @@ -185,7 +185,7 @@ func compileItem(v any, pos position) (node, error) { if pos != inColumn { return nil, fmt.Errorf(`null is a record column's value; here it only renders "", so write ""`) } - return &null{}, nil + return &nullItem{}, nil default: return nil, fmt.Errorf("a template value must be a string, a list or an object, not %s", jsonKind(v)) } @@ -340,7 +340,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if err := checkNestedDrawGroup(fields, o.group); err != nil { return nil, err } - t := &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn} + t := &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn} if err := t.compileRefFree(); err != nil { return nil, err } @@ -526,7 +526,7 @@ func checkPathNames(path string) error { if err != nil { return err } - for _, seg := range names(segs) { + for _, seg := range nameSegments(segs) { if err := checkName(seg); err != nil { return fmt.Errorf("path %w", err) } diff --git a/path.go b/path.go index bedd88e..68e9860 100644 --- a/path.go +++ b/path.go @@ -114,8 +114,8 @@ func hasSelector(segs []string) bool { // selectorOf is the key or name a selector segment holds. func selectorOf(seg string) string { return seg[1 : len(seg)-1] } -// names is the segments of a path that are names, its selectors left out. -func names(segs []string) []string { +// nameSegments is the segments of a path that are names, its selectors left out. +func nameSegments(segs []string) []string { out := segs[:0:0] for _, s := range segs { if !isSelector(s) { @@ -197,8 +197,8 @@ func walkPath(n node, tail []string, w pathWalk) (node, error) { return walkPath(child, tail[1:], w) case *choice: return walkChoice(n, tail, w) - case *column: - return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", tail[0], n.t.columns[n.i]) + case *tableColumn: + return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", tail[0], n.t.header[n.i]) } return nil, fmt.Errorf("no field %q", tail[0]) } @@ -268,7 +268,7 @@ func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error case len(tail) == 0 && sel == "" && !descended: return walkPath(t, nil, w) case len(tail) == 0: - return walkPath(t.whole, nil, w) + return walkPath(t.wholeRow, nil, w) case child != nil: return walkTable(child, tail[1:], w, true) } @@ -293,7 +293,7 @@ func (t *table) step(tail []string) (column node, child *table, err error) { return nil, nil, nil } if i, ok := t.col[tail[0]]; ok { - return t.fields[t.columns[i]], nil, nil + return t.fields[t.header[i]], nil, nil } if child = t.descendant(tail[0]); child == nil { return nil, nil, fmt.Errorf("no column or linked table %q in %s", tail[0], t.category) @@ -327,7 +327,7 @@ func (d *pathDraws) variant(c *choice, rest []string) node { key := d.a.levels[len(d.a.tail)-len(rest)] n, drew := d.held.variant[key] if !drew { - n = drawn(d.s, c) + n = resolveChoice(d.s, c) if d.held.variant == nil { d.held.variant = map[string]node{} } diff --git a/pins.go b/pins.go index 4b5e668..c8951c9 100644 --- a/pins.go +++ b/pins.go @@ -15,12 +15,12 @@ type tablePin struct { // pinSet is the table rows fixed so far. type pinSet struct { inline [8]tablePin // sized so a render over a country's five-deep geo tree stays off the heap - n int + used int spill map[*table]int } func (p *pinSet) pinned(t *table) (int, bool) { - for _, q := range p.inline[:p.n] { + for _, q := range p.inline[:p.used] { if q.t == t { return q.row, true } @@ -39,9 +39,9 @@ func (p *pinSet) mustRow(t *table) int { } func (p *pinSet) add(t *table, r int) { - if p.n < len(p.inline) { - p.inline[p.n] = tablePin{t, r} - p.n++ + if p.used < len(p.inline) { + p.inline[p.used] = tablePin{t, r} + p.used++ return } if p.spill == nil { @@ -73,7 +73,7 @@ func (p *pinSet) nearestPinned(t *table) *table { // each calls fn for every pinned row, in pin order, the spilled ones by path. func (p *pinSet) each(fn func(t *table, r int)) { - for _, q := range p.inline[:p.n] { + for _, q := range p.inline[:p.used] { fn(q.t, q.row) } spilled := make([]*table, 0, len(p.spill)) @@ -164,8 +164,8 @@ func (p *pinSet) differs(q *pinSet) bool { return found } -// key spells the set for a map, by the tables' identities in path order. -func (p *pinSet) key() string { +// mapKey spells the set for a map, by the tables' identities in path order. +func (p *pinSet) mapKey() string { var pins []tablePin p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) diff --git a/record.go b/record.go index 2911f44..6b40b6f 100644 --- a/record.go +++ b/record.go @@ -141,13 +141,13 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { } shape := f.recordShapeOf(n) if errors.Is(shape.err, ErrNoColumns) { - ns := names(segments) + ns := nameSegments(segments) return nil, fmt.Errorf(`fejkdata: %s %w; render it as a column of one: {"format":"","%s":"{/%s}"}`, path, shape.err, ns[len(ns)-1], path) } if shape.err != nil { return nil, fmt.Errorf("fejkdata: %s %w", path, shape.err) } - return renderRecord(f.rand, shape.t, shape.columns, sc), nil + return renderRecord(f.rand, shape.template, shape.columns, sc), nil } // tableRecord walks a path's tail from a table to the table whose row is the record, @@ -161,10 +161,10 @@ func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, err case *table: n.drawIn(s, &sc.hold().pins) return n, nil - case *row: + case *tableRow: return n.t, nil - case *column: - return nil, fmt.Errorf("descends into %q, a column; a record is a table's row", n.t.columns[n.i]) + case *tableColumn: + return nil, fmt.Errorf("descends into %q, a column; a record is a table's row", n.t.header[n.i]) } return n, nil } @@ -172,9 +172,9 @@ func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, err // recordShape is what recordOf settled about a node: the template to project, its // columns, or why it is not a record. type recordShape struct { - t *template - columns []Column - err error + template *template + columns []Column + err error } // recordShapeOf fences a node once and remembers the answer. Callers hold the @@ -184,7 +184,7 @@ func (f *Generator) recordShapeOf(n node) recordShape { return shape } t, columns, err := recordOf(n) - shape := recordShape{t: t, columns: columns, err: err} + shape := recordShape{template: t, columns: columns, err: err} if f.records == nil { f.records = map[node]recordShape{} } @@ -195,9 +195,9 @@ func (f *Generator) recordShapeOf(n node) recordShape { // RecordTemplate is an inline record compiled, referenced and validated once, // ready to render many times with [RecordTemplate.Fake]. type RecordTemplate struct { - g *Generator - t *template - columns []Column + g *Generator + template *template + columns []Column } // Fake renders the record with one draw. @@ -205,7 +205,7 @@ func (t *RecordTemplate) Fake() *Record { t.g.mu.Lock() defer t.g.mu.Unlock() set := eagerHoldSet() - return renderRecord(t.g.rand, t.t, t.columns, renderScope{set: &set}) + return renderRecord(t.g.rand, t.template, t.columns, renderScope{set: &set}) } // NewRecordTemplate compiles an inline record — a JSON object with a format and @@ -219,7 +219,7 @@ func (f *Generator) NewRecordTemplate(input string) (*RecordTemplate, error) { if err != nil { return nil, fmt.Errorf("fejkdata: an inline record %w", err) } - return &RecordTemplate{g: f, t: tm, columns: columns}, nil + return &RecordTemplate{g: f, template: tm, columns: columns}, nil } // FakeRecordTemplate compiles and renders an inline record in one call. @@ -239,7 +239,7 @@ var ErrNoColumns = errors.New("has no fields, so no columns") // the template, fixed for every draw the caller goes on to make. func recordOf(n node) (*template, []Column, error) { if tb, isTable := n.(*table); isTable { - n = tb.format + n = tb.formatTemplate } t, ok := n.(*template) if !ok { @@ -249,7 +249,7 @@ func recordOf(n node) (*template, []Column, error) { if len(names) == 0 { return nil, nil, ErrNoColumns } - if !t.record { + if !t.isRecord { return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat) } columns := make([]Column, len(names)) diff --git a/reference.go b/reference.go index be92a11..3ab410a 100644 --- a/reference.go +++ b/reference.go @@ -136,7 +136,7 @@ func columnReadOf(t *template) *columnRead { } a := splitArm(name, t.refs) target, isTemplate := t.head(a.key).(*template) - if !isTemplate || !target.record || len(a.tail) != 1 { + if !isTemplate || !target.isRecord || len(a.tail) != 1 { return nil } return &columnRead{a: a, column: target.fields[a.tail[0]]} diff --git a/render.go b/render.go index 4aea292..be5a7c5 100644 --- a/render.go +++ b/render.go @@ -58,15 +58,15 @@ func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: return render(s, pick(s, n), sc) - case *null: + case *nullItem: return "" case *table: - sc.t, sc.row = n, n.draw(s) - return expand(s, n.format, sc) - case *row: + sc.t, sc.row = n, n.drawRow(s) + return expand(s, n.formatTemplate, sc) + case *tableRow: sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) - return expand(s, n.t.format, sc) - case *column: + return expand(s, n.t.formatTemplate, sc) + case *tableColumn: if sc.t != n.t { sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) } diff --git a/shape_test.go b/shape_test.go index 3b6c2c4..aae7dbd 100644 --- a/shape_test.go +++ b/shape_test.go @@ -44,8 +44,8 @@ func shippedShape(f *Generator) string { case *choice: facts[prefix] = readsFact(n) case *table: - facts[prefix] = "\tformat " + strconv.Quote(n.format.format) + tableFacts(n) + readsFact(n) - for _, name := range n.columns { + facts[prefix] = "\tformat " + strconv.Quote(n.formatTemplate.format) + tableFacts(n) + readsFact(n) + for _, name := range n.header { facts[join(prefix, name)] = "\tstring" } case *template: @@ -77,9 +77,9 @@ func tableFacts(t *table) string { for _, o := range []struct { name string col int - }{{"key", t.key}, {"name", t.name}, {"weight", t.weight}, {"parent", t.parent}} { + }{{"key", t.keyCol}, {"name", t.nameCol}, {"weight", t.weightCol}, {"parent", t.parentCol}} { if o.col >= 0 { - b.WriteString("\t" + o.name + " " + t.columns[o.col]) + b.WriteString("\t" + o.name + " " + t.header[o.col]) } } return b.String() @@ -96,7 +96,7 @@ func readsFact(n node) string { collect(it) } case *table: - collect(n.format) + collect(n.formatTemplate) for _, cell := range n.cellTemplates { collect(cell) } diff --git a/struct.go b/struct.go index 51f69e6..959fc07 100644 --- a/struct.go +++ b/struct.go @@ -49,8 +49,8 @@ func (f *Generator) structShapeOf(t reflect.Type) (*structShape, error) { if label == "" { label = "struct" } - sc := &structCompile{root: f.categories, visiting: map[reflect.Type]bool{}, structs: maxStructs} - shape, err := sc.record(t, label) + sc := &structCompile{categories: f.categories, visiting: map[reflect.Type]bool{}, structs: maxStructs} + shape, err := sc.compileShape(t, label) if err == nil && shape.empty() { err = fmt.Errorf("%s has no fake tags, so nothing to fill", t) } @@ -64,16 +64,16 @@ func (f *Generator) structShapeOf(t reflect.Type) (*structShape, error) { // structShape is a struct type compiled to fill: its tagged fields as one record, the field // index path each column fills, and the struct fields carrying tags of their own. type structShape struct { - record *template - columns []Column - fields [][]int - nested []nestedStruct + record *template + columns []Column + fieldIndexes [][]int + nested []nestedStruct } // nestedStruct is a named struct field, or a pointer to one, filled as a record of its own. type nestedStruct struct { - index []int - shape *structShape + fieldIndex []int + shape *structShape } func (s *structShape) empty() bool { return s.record == nil && len(s.nested) == 0 } @@ -82,9 +82,9 @@ func (s *structShape) empty() bool { return s.record == nil && len(s.nested) == // loaded tree, the types compiling or embedded above, so a pointer back to one is left alone // rather than filled without end, and how many more structs it may walk. type structCompile struct { - root map[string]node - visiting map[reflect.Type]bool - structs int + categories map[string]node + visiting map[reflect.Type]bool + structs int } // structFields gathers what one struct type fills: its tagged fields, those its embedded @@ -92,24 +92,24 @@ type structCompile struct { // records. type structFields struct { *structCompile - t reflect.Type + typ reflect.Type label string tags map[string]any shape *structShape } -func (sc *structCompile) record(t reflect.Type, label string) (*structShape, error) { +func (sc *structCompile) compileShape(t reflect.Type, label string) (*structShape, error) { if err := sc.spend(label); err != nil { return nil, err } sc.visiting[t] = true defer delete(sc.visiting, t) - c := &structFields{structCompile: sc, t: t, label: label, tags: map[string]any{}, shape: &structShape{}} + c := &structFields{structCompile: sc, typ: t, label: label, tags: map[string]any{}, shape: &structShape{}} if err := c.gatherFields(t, nil); err != nil { return nil, err } if len(c.tags) > 0 { - if err := c.shape.compileRecord(sc.root, t, label, c.tags); err != nil { + if err := c.shape.compileRecord(sc.categories, t, label, c.tags); err != nil { return nil, err } } @@ -169,8 +169,8 @@ func structOf(t reflect.Type) reflect.Type { // addTag adds a tagged field's tag to the record, refusing one another field hides. func (c *structFields) addTag(sf reflect.StructField, tag string) error { - if visible, ok := c.t.FieldByName(sf.Name); !ok || !slices.Equal(visible.Index, sf.Index) { - return fmt.Errorf("%s.%s: hidden by another field named %s, so its fake tag cannot fill it; rename one", c.label, fieldPath(c.t, sf.Index), sf.Name) + if visible, ok := c.typ.FieldByName(sf.Name); !ok || !slices.Equal(visible.Index, sf.Index) { + return fmt.Errorf("%s.%s: hidden by another field named %s, so its fake tag cannot fill it; rename one", c.label, fieldPath(c.typ, sf.Index), sf.Name) } v, err := tagValue(sf, tag) if err != nil { @@ -190,7 +190,7 @@ func fieldPath(t reflect.Type, index []int) string { } func (c *structFields) embed(sf reflect.StructField, elem reflect.Type) error { - if err := c.spend(c.label + "." + fieldPath(c.t, sf.Index)); err != nil { + if err := c.spend(c.label + "." + fieldPath(c.typ, sf.Index)); err != nil { return err } c.visiting[elem] = true @@ -200,13 +200,13 @@ func (c *structFields) embed(sf reflect.StructField, elem reflect.Type) error { return err } if sf.Type.Kind() == reflect.Pointer && !sf.IsExported() && (len(c.tags) > tags || len(c.shape.nested) > nested) { - return fmt.Errorf("%s.%s: an unexported embedded pointer field cannot be set, so the tags beneath it cannot fill; embed %s by value", c.label, fieldPath(c.t, sf.Index), elem) + return fmt.Errorf("%s.%s: an unexported embedded pointer field cannot be set, so the tags beneath it cannot fill; embed %s by value", c.label, fieldPath(c.typ, sf.Index), elem) } return nil } func (c *structFields) nest(sf reflect.StructField, elem reflect.Type) error { - nested, err := c.record(elem, c.label+"."+sf.Name) + nested, err := c.compileShape(elem, c.label+"."+sf.Name) if err != nil || nested.empty() { return err } @@ -266,13 +266,13 @@ func (s *structShape) compileRecord(root map[string]node, t reflect.Type, label return fmt.Errorf("%s: %w", label, err) } proof := &valueProof{} - s.fields = make([][]int, len(columns)) + s.fieldIndexes = make([][]int, len(columns)) for i, c := range columns { sf, _ := t.FieldByName(c.Name) if err := proof.checkField(label+"."+c.Name, sf.Type, record.fields[c.Name]); err != nil { return err } - s.fields[i] = sf.Index + s.fieldIndexes[i] = sf.Index } s.record, s.columns = record, columns return nil @@ -327,7 +327,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro elem := ft if ft.Kind() == reflect.Pointer { elem = ft.Elem() - } else if p.proveColumn(column).null { + } else if p.proveColumn(column).nullable { return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft) } kind := columnKinds[elem.Kind()] @@ -351,11 +351,11 @@ func (s *structShape) fill(sess *session, v reflect.Value) { if s.record != nil { set := eagerHoldSet() for i, c := range renderRecord(sess, s.record, s.columns, renderScope{set: &set}).columns { - setColumn(fieldAt(v, s.fields[i]), c) + setColumn(fieldAt(v, s.fieldIndexes[i]), c) } } for _, n := range s.nested { - field := fieldAt(v, n.index) + field := fieldAt(v, n.fieldIndex) if field.Kind() == reflect.Pointer { if field.IsNil() { field.Set(reflect.New(field.Type().Elem())) diff --git a/table.go b/table.go index 43ee53c..981de6f 100644 --- a/table.go +++ b/table.go @@ -13,58 +13,58 @@ import ( // columns, each row is one draw, and the format renders the drawn row. A column is a // cell of the row a render pinned; a cell carrying tokens compiles to a string node. type table struct { - category string - path string // the category's path from the data root, which a selector is written at - file string - format *template // fields are the column nodes - columns []string - col map[string]int - fields map[string]node // column nodes, the format's fields - whole *row // the pinned row rendered by the format - cells []string // rows × columns, flat - cellTemplates map[int]*template - key int // key, name, weight and parent are column indexes, or -1 - name int - weight int - parent int - cum []float64 // cumulative weights, nil when uniform - byKey map[string]int - parentT *table - children map[string]*table - once sync.Once - index tableIndex + category string + path string // the category's path from the data root, which a selector is written at + file string + formatTemplate *template // fields are the column nodes + header []string + col map[string]int + fields map[string]node // column nodes, the format's fields + wholeRow *tableRow // the pinned row rendered by the format + cells []string // rows × columns, flat + cellTemplates map[int]*template + keyCol int // keyCol through parentCol are -1 where the option is absent + nameCol int + weightCol int + parentCol int + cum []float64 // cumulative weights, nil when uniform + byKey map[string]int + parentT *table + children map[string]*table + once sync.Once + index tableIndex } // tableIndex is what selection and linked draws look up, built on the first draw // that needs it. type tableIndex struct { - byName map[string][]int - children map[string][]int // rows by parent key - childCum map[string][]float64 // cumulative weights per parent key, nil when uniform + byName map[string][]int + rowsByParent map[string][]int + childCum map[string][]float64 // cumulative weights per parent key, nil when uniform } func (*table) isNode() {} -// column is one column of a table, rendered as the cell of the row the render pinned. -type column struct { +// tableColumn is one column of a table, rendered as the cell of the row the render pinned. +type tableColumn struct { t *table i int } -func (*column) isNode() {} +func (*tableColumn) isNode() {} -// row is the row of a table the render pinned, rendered by the table's format. -type row struct{ t *table } +// tableRow is the row of a table the render pinned, rendered by the table's format. +type tableRow struct{ t *table } -func (*row) isNode() {} +func (*tableRow) isNode() {} -func (t *table) rowCount() int { return len(t.cells) / len(t.columns) } +func (t *table) rowCount() int { return len(t.cells) / len(t.header) } -func (t *table) cell(row, col int) string { return t.cells[row*len(t.columns)+col] } +func (t *table) cell(row, col int) string { return t.cells[row*len(t.header)+col] } // cellTemplate is what a cell renders: its compiled template where it carries tokens, // else nil for its text. -func (t *table) cellTemplate(row, col int) *template { return t.cellTemplates[row*len(t.columns)+col] } +func (t *table) cellTemplate(row, col int) *template { return t.cellTemplates[row*len(t.header)+col] } // tableOptions are the keys a table object takes; every other key is refused. var tableOptions = []string{"format", "key", "name", "parent", "rows", "weight"} @@ -93,7 +93,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab if err != nil { return nil, err } - t := &table{category: category, file: o.rows, key: -1, name: -1, weight: -1, parent: -1} + t := &table{category: category, file: o.rows, keyCol: -1, nameCol: -1, weightCol: -1, parentCol: -1} if err := t.parseRows(data); err != nil { return nil, fmt.Errorf("%s: %w", o.rows, err) } @@ -110,7 +110,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab } type tableOptionValues struct { - format, key, name, parent, rows, weight string + format, keyColumn, nameColumn, parentColumn, rows, weightColumn string } func readTableOptions(m map[string]any) (tableOptionValues, error) { @@ -120,7 +120,7 @@ func readTableOptions(m map[string]any) (tableOptionValues, error) { return o, fmt.Errorf("a table takes %s; %q is none of them", strings.Join(tableOptions, ", "), k) } } - for k, into := range map[string]*string{"format": &o.format, "key": &o.key, "name": &o.name, "parent": &o.parent, "rows": &o.rows, "weight": &o.weight} { + for k, into := range map[string]*string{"format": &o.format, "key": &o.keyColumn, "name": &o.nameColumn, "parent": &o.parentColumn, "rows": &o.rows, "weight": &o.weightColumn} { v, ok := m[k] if !ok { continue @@ -140,8 +140,8 @@ func readTableOptions(m map[string]any) (tableOptionValues, error) { if !strings.HasSuffix(o.rows, ".tsv") || strings.Contains(o.rows, "/") { return o, fmt.Errorf("rows names a .tsv file beside the category, not %q", o.rows) } - if o.key != "" && o.key == o.name { - return o, fmt.Errorf("name names the key column %q, which a selector already reads; drop it", o.name) + if o.keyColumn != "" && o.keyColumn == o.nameColumn { + return o, fmt.Errorf("name names the key column %q, which a selector already reads; drop it", o.nameColumn) } return o, nil } @@ -155,10 +155,10 @@ func (t *table) parseRows(data string) error { if header == "" { return fmt.Errorf("has no header line naming its columns") } - t.columns = strings.Split(header, "\t") - t.col = make(map[string]int, len(t.columns)) - t.fields = make(map[string]node, len(t.columns)) - for i, name := range t.columns { + t.header = strings.Split(header, "\t") + t.col = make(map[string]int, len(t.header)) + t.fields = make(map[string]node, len(t.header)) + for i, name := range t.header { if err := checkName(name); err != nil { return fmt.Errorf("column %w", err) } @@ -166,12 +166,12 @@ func (t *table) parseRows(data string) error { return fmt.Errorf("column %q is named twice", name) } t.col[name] = i - t.fields[name] = &column{t, i} + t.fields[name] = &tableColumn{t, i} } if header == data { return fmt.Errorf("has no rows below its header; a table is at least two rows") } - m := len(t.columns) + m := len(t.header) t.cells = make([]string, 0, m*(strings.Count(rest, "\n")+1)) for line := 2; ; line++ { row, more, found := strings.Cut(rest, "\n") @@ -195,7 +195,7 @@ func (t *table) appendRow(row string, line int) error { if row == "" { return fmt.Errorf("line %d is empty; every line below the header is a row", line) } - m := len(t.columns) + m := len(t.header) for n := 0; n < m; n++ { cell, next, tab := strings.Cut(row, "\t") if !tab && n < m-1 || tab && n == m-1 { @@ -213,17 +213,17 @@ func (t *table) bindOptions(o tableOptionValues) error { for _, opt := range []struct { name, value string into *int - }{{"key", o.key, &t.key}, {"name", o.name, &t.name}, {"parent", o.parent, &t.parent}, {"weight", o.weight, &t.weight}} { + }{{"key", o.keyColumn, &t.keyCol}, {"name", o.nameColumn, &t.nameCol}, {"parent", o.parentColumn, &t.parentCol}, {"weight", o.weightColumn, &t.weightCol}} { if opt.value == "" { continue } i, ok := t.col[opt.value] if !ok { - return fmt.Errorf("%s names no column %q of %s; the columns are %v", opt.name, opt.value, o.rows, t.columns) + return fmt.Errorf("%s names no column %q of %s; the columns are %v", opt.name, opt.value, o.rows, t.header) } *opt.into = i } - if t.name >= 0 && t.key < 0 && t.parent < 0 { + if t.nameCol >= 0 && t.keyCol < 0 && t.parentCol < 0 { return fmt.Errorf("name selects a row as a key does, and lists the rows it matches by their keys, so it needs a key column, or a parent inside which each name is one row; add key") } if err := t.indexKeys(); err != nil { @@ -237,14 +237,14 @@ func (t *table) bindOptions(o tableOptionValues) error { // proveNamesInsideParent proves a name without a key names one row inside its parent. func (t *table) proveNamesInsideParent() error { - if t.key >= 0 || t.name < 0 { + if t.keyCol >= 0 || t.nameCol < 0 { return nil } inside := make(map[string]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.parent) + "\t" + t.cell(r, t.name) + k := t.cell(r, t.parentCol) + "\t" + t.cell(r, t.nameCol) if first, dup := inside[k]; dup { - return fmt.Errorf("%s line %d: name %q repeats line %d inside %s %q; a name selects one row inside its parent; drop one, or add a key column", t.file, r+2, t.cell(r, t.name), first+2, t.columns[t.parent], t.cell(r, t.parent)) + return fmt.Errorf("%s line %d: name %q repeats line %d inside %s %q; a name selects one row inside its parent; drop one, or add a key column", t.file, r+2, t.cell(r, t.nameCol), first+2, t.header[t.parentCol], t.cell(r, t.parentCol)) } inside[k] = r } @@ -253,12 +253,12 @@ func (t *table) proveNamesInsideParent() error { // indexKeys proves every key names one row, and keeps the index a link is proved by. func (t *table) indexKeys() error { - if t.key < 0 { + if t.keyCol < 0 { return nil } t.byKey = make(map[string]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.key) + k := t.cell(r, t.keyCol) if k == "" { return fmt.Errorf("%s line %d: the key is empty", t.file, r+2) } @@ -267,8 +267,8 @@ func (t *table) indexKeys() error { } t.byKey[k] = r } - for r := 0; r < t.rowCount() && t.name >= 0; r++ { - if n := t.cell(r, t.name); t.byKey[n] != r { + for r := 0; r < t.rowCount() && t.nameCol >= 0; r++ { + if n := t.cell(r, t.nameCol); t.byKey[n] != r { if other, isKey := t.byKey[n]; isKey { return fmt.Errorf("%s line %d: name %q is the key of line %d, which a selector reads first, so the name could never select this row", t.file, r+2, n, other+2) } @@ -279,15 +279,15 @@ func (t *table) indexKeys() error { // sumWeights proves every weight is a positive number and builds the cumulative table. func (t *table) sumWeights() error { - if t.weight < 0 { + if t.weightCol < 0 { return nil } t.cum = make([]float64, t.rowCount()) total := 0.0 for r := range t.cum { - w, err := strconv.ParseFloat(t.cell(r, t.weight), 64) + w, err := strconv.ParseFloat(t.cell(r, t.weightCol), 64) if err != nil || math.IsInf(w, 0) || math.IsNaN(w) || w <= 0 { - return fmt.Errorf("%s line %d: weight %q is not a positive number", t.file, r+2, t.cell(r, t.weight)) + return fmt.Errorf("%s line %d: weight %q is not a positive number", t.file, r+2, t.cell(r, t.weightCol)) } total += w t.cum[r] = total @@ -301,16 +301,16 @@ func (t *table) sumWeights() error { // checkCells compiles every cell carrying a token, and fences what a selector reads. func (t *table) checkCells() error { for i, cell := range t.cells { - row, col := i/len(t.columns), i%len(t.columns) - if (col == t.key || col == t.name) && strings.ContainsAny(cell, inSelector) { - return fmt.Errorf("line %d: %s %q contains %q, which a selector cannot spell", row+2, t.columns[col], cell, cell[strings.IndexAny(cell, inSelector):][:1]) + row, col := i/len(t.header), i%len(t.header) + if (col == t.keyCol || col == t.nameCol) && strings.ContainsAny(cell, inSelector) { + return fmt.Errorf("line %d: %s %q contains %q, which a selector cannot spell", row+2, t.header[col], cell, cell[strings.IndexAny(cell, inSelector):][:1]) } if strings.IndexByte(cell, '{') < 0 && strings.IndexByte(cell, '}') < 0 { continue } n, err := compileString(cell) if err != nil { - return fmt.Errorf("line %d, %s: %w", row+2, t.columns[col], err) + return fmt.Errorf("line %d, %s: %w", row+2, t.header[col], err) } if t.cellTemplates == nil { t.cellTemplates = map[int]*template{} @@ -335,16 +335,16 @@ func (t *table) compileWhole(format string) error { for _, name := range tok.names { a := splitArm(name, nil) if _, ok := t.col[a.key]; !ok && !isRef(a.key) && a.key != "" { - return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.columns) + return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.header) } } } if err := checkTokens(toks, t.fields); err != nil { return err } - t.format = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, record: true, table: t} - t.whole = &row{t} - return t.format.compileRefFree() + t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, table: t} + t.wholeRow = &tableRow{t} + return t.formatTemplate.compileRefFree() } // setTablePaths gives every table the path a selector on it is written at, before a @@ -379,7 +379,7 @@ func linkTables(root map[string]node) error { return err } case *table: - if n.parent < 0 { + if n.parentCol < 0 { continue } if err := n.linkParent(path, children); err != nil { @@ -393,24 +393,24 @@ func linkTables(root map[string]node) error { } func (t *table) linkParent(path string, siblings map[string]node) error { - name := t.columns[t.parent] + name := t.header[t.parentCol] p, ok := siblings[name].(*table) switch { case siblings[name] == nil: return fmt.Errorf("parent %q names no table beside it", name) case !ok: return fmt.Errorf("parent %q is not a table; a link column reads a table's key", name) - case p.key < 0: + case p.keyCol < 0: return fmt.Errorf("parent %q has no key column to link to", name) } var ancestors []*table - for q, seen := p, map[*table]bool{t: true}; q != nil; q, _ = siblings[q.columns[q.parent]].(*table) { + for q, seen := p, map[*table]bool{t: true}; q != nil; q, _ = siblings[q.header[q.parentCol]].(*table) { if seen[q] { return fmt.Errorf("parent cycle: %s reaches itself through its parents", q.category) } seen[q] = true ancestors = append(ancestors, q) - if q.parent < 0 { + if q.parentCol < 0 { break } } @@ -421,14 +421,14 @@ func (t *table) linkParent(path string, siblings map[string]node) error { } linked := make(map[string]bool, p.rowCount()) for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.parent) + k := t.cell(r, t.parentCol) if _, ok := p.byKey[k]; !ok { return fmt.Errorf("%s line %d: %s %q is no key of %s", t.file, r+2, name, k, p.file) } linked[k] = true } for r := 0; r < p.rowCount(); r++ { - if k := p.cell(r, p.key); !linked[k] { + if k := p.cell(r, p.keyCol); !linked[k] { return fmt.Errorf("%s links no row to %s %q; every %s row needs one, or drop line %d of %s", t.file, name, k, name, r+2, p.file) } } @@ -442,25 +442,25 @@ func (t *table) linkParent(path string, siblings map[string]node) error { func (t *table) indexed() *tableIndex { t.once.Do(func() { - if t.name >= 0 { + if t.nameCol >= 0 { t.index.byName = make(map[string][]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { - n := t.cell(r, t.name) + n := t.cell(r, t.nameCol) t.index.byName[n] = append(t.index.byName[n], r) } } - if t.parent >= 0 { - t.index.children = map[string][]int{} + if t.parentCol >= 0 { + t.index.rowsByParent = map[string][]int{} for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.parent) - t.index.children[k] = append(t.index.children[k], r) + k := t.cell(r, t.parentCol) + t.index.rowsByParent[k] = append(t.index.rowsByParent[k], r) } if t.cum != nil { - t.index.childCum = make(map[string][]float64, len(t.index.children)) - for k, rows := range t.index.children { + t.index.childCum = make(map[string][]float64, len(t.index.rowsByParent)) + for k, rows := range t.index.rowsByParent { cum, total := make([]float64, len(rows)), 0.0 for i, r := range rows { - w, _ := strconv.ParseFloat(t.cell(r, t.weight), 64) // sumWeights proved it + w, _ := strconv.ParseFloat(t.cell(r, t.weightCol), 64) // sumWeights proved it total += w cum[i] = total } @@ -472,9 +472,9 @@ func (t *table) indexed() *tableIndex { return &t.index } -// draw picks a row over the whole table. The session is concrete rather than the rng +// drawRow picks a row over the whole table. The session is concrete rather than the rng // interface so that the walk that draws through it keeps its hold set off the heap. -func (t *table) draw(s *session) int { +func (t *table) drawRow(s *session) int { if t.cum == nil { return s.IntN(t.rowCount()) } @@ -489,8 +489,8 @@ func pickCum(s *session, cum []float64) int { // drawUnder picks a row among those linked to parent row pr. func (t *table) drawUnder(s *session, pr int) int { ix := t.indexed() - k := t.parentT.cell(pr, t.parentT.key) - rows := ix.children[k] + k := t.parentT.cell(pr, t.parentT.keyCol) + rows := ix.rowsByParent[k] if ix.childCum == nil { return rows[s.IntN(len(rows))] } @@ -516,7 +516,7 @@ func (t *table) drawIn(s *session, p *pinSet) int { r = t.drawUnder(s, pr) } if r < 0 { - r = t.draw(s) + r = t.drawRow(s) } p.pin(t, r) return r @@ -536,7 +536,7 @@ func (t *table) descendant(name string) *table { } // parentRow is the row of t's parent that row r links to. -func (t *table) parentRow(r int) int { return t.parentT.byKey[t.cell(r, t.parent)] } +func (t *table) parentRow(r int) int { return t.parentT.byKey[t.cell(r, t.parentCol)] } // descends reports whether a is an ancestor of t. func (t *table) descends(a *table) bool { @@ -561,7 +561,7 @@ func (t *table) under(r int, a *table, pr int) bool { // find is the row a selector names: by key first, then by name, where a name // naming several rows resolves inside the ancestors pinned. func (t *table) find(sel string, pins *pinSet) (int, error) { - if t.key < 0 && t.name < 0 { + if t.keyCol < 0 && t.nameCol < 0 { return 0, fmt.Errorf("%s has no key or name column to select a row by", t.path) } if r, ok := t.byKey[sel]; ok { @@ -585,14 +585,14 @@ func (t *table) find(sel string, pins *pinSet) (int, error) { func (t *table) ambiguous(sel string, rows []int) error { spellings := make([]string, len(rows)) for i, r := range rows { - if t.key < 0 { + if t.keyCol < 0 { spellings[i] = t.selectorSpelling(r) } else { - spellings[i] = t.cell(r, t.key) + spellings[i] = t.cell(r, t.keyCol) } } listed := strings.Join(spellings, ", ") - if t.key < 0 { + if t.keyCol < 0 { return fmt.Errorf("%q names %d rows of %s; select it inside its %s, one of %s", sel, len(rows), t.path, t.parentT.path, listed) } inside := "" @@ -606,10 +606,10 @@ func (t *table) ambiguous(sel string, rows []int) error { // by name inside its parent's row. func (t *table) selectorSpelling(r int) string { switch { - case t.key >= 0: - return t.path + "[" + t.cell(r, t.key) + "]" - case t.name >= 0: - return t.parentT.selectorSpelling(t.parentRow(r)) + "." + t.category + "[" + t.cell(r, t.name) + "]" + case t.keyCol >= 0: + return t.path + "[" + t.cell(r, t.keyCol) + "]" + case t.nameCol >= 0: + return t.parentT.selectorSpelling(t.parentRow(r)) + "." + t.category + "[" + t.cell(r, t.nameCol) + "]" } return fmt.Sprintf("%s line %d", t.file, r+2) } diff --git a/table_test.go b/table_test.go index 1875906..397c10a 100644 --- a/table_test.go +++ b/table_test.go @@ -983,8 +983,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rowCount(); row++ { - whole := drawAt{drawn: entered} - whole.whole.add(entered, row) + whole := drawAt{wholeTable: entered} + whole.wholePins.add(entered, row) s := none.entered(entered, row) for _, cand := range tables { for i := 0; i < 20; i++ { @@ -1003,8 +1003,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { if alternatives(whole, candAt) || alternatives(candAt, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } - other := drawAt{drawn: cand} - other.whole.add(cand, cr) + other := drawAt{wholeTable: cand} + other.wholePins.add(cand, cr) if want := cand == entered && cr != row; alternatives(whole, other) != want { t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) } diff --git a/todo.md b/todo.md index e8b008c..8ae15fc 100644 --- a/todo.md +++ b/todo.md @@ -19,10 +19,6 @@ Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. This round, in order: -- Rename the shared names left that name two things, per the decision "One name, one - meaning": judge each of `check`, `children`, `column`, `columns`, `draw`, `drawn`, - `fields`, `format`, `head`, `index`, `key`, `n`, `name`, `named`, `names`, `null`, - `number`, `op`, `reads`, `record`, `root`, `row`, `t`, `variant` and `whole`. - Run the nine-seat panel again and file what it names here. ### Data diff --git a/transform.go b/transform.go index f24be31..d5a3ae6 100644 --- a/transform.go +++ b/transform.go @@ -19,7 +19,7 @@ func withTransforms(calls map[string]builtin) map[string]builtin { if _, clash := calls[name]; clash { panic("fejkdata: " + name + " is registered as a builtin and a transform") } - calls[name] = builtin{arity: 1, check: transformArg, prep: transformPrep(fn), operands: transformOperand} + calls[name] = builtin{arity: 1, checkArgs: transformArg, prep: transformPrep(fn), operands: transformOperand} } return calls } diff --git a/value.go b/value.go index 2d94b83..1dd8b23 100644 --- a/value.go +++ b/value.go @@ -17,13 +17,13 @@ type proven struct { integral bool notOperand string // why some render reads as no finite number, the way calc reads it not [len(dataTypeNames)]string - null bool // some draw of a column is null + nullable bool // some draw of a column is null } // valueProof proves what typed columns and their calc operands hold, each node once per scope. type valueProof struct { - memo map[node]proven - columns map[node]proven + memo map[node]proven + columnMemo map[node]proven } // checkDatatype rejects a typed column item some render of which is not text of its datatype, @@ -54,11 +54,11 @@ func (p *valueProof) proveColumnItem(t *template) proven { // proveColumn proves a column over what its items draw, a null item marking it null rather than // rendering "". func (p *valueProof) proveColumn(n node) proven { - if v, done := p.columns[n]; done { + if v, done := p.columnMemo[n]; done { return v } - if p.columns == nil { - p.columns = map[node]proven{} + if p.columnMemo == nil { + p.columnMemo = map[node]proven{} } items, nullable := columnItems(n) var v proven @@ -69,8 +69,8 @@ func (p *valueProof) proveColumn(n node) proven { v = v.or(w) } } - v.null = v.null || nullable - p.columns[n] = v + v.nullable = v.nullable || nullable + p.columnMemo[n] = v return v } @@ -87,10 +87,10 @@ func (p *valueProof) prove(n node) proven { v = p.proveUnion(n.items) case *template: v = p.proveTemplate(n) - case *column: + case *tableColumn: v = p.proveCells(n) - case *row: - v = unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", n.t.format.format, n.t.category)) + case *tableRow: + v = unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", n.t.formatTemplate.format, n.t.category)) default: v = unproven(`it reads a null, which renders "" outside its own column`) } @@ -99,7 +99,7 @@ func (p *valueProof) prove(n node) proven { } // proveCells proves a table column over every cell it may render. -func (p *valueProof) proveCells(c *column) proven { +func (p *valueProof) proveCells(c *tableColumn) proven { var v proven for r := 0; r < c.t.rowCount(); r++ { var w proven @@ -128,7 +128,7 @@ func (p *valueProof) proveUnion(nodes []node) proven { // or is what a proof knows of a render that is either v or w. func (v proven) or(w proven) proven { v.lo, v.hi, v.nonZero = min(v.lo, w.lo), max(v.hi, w.hi), min(v.nonZero, w.nonZero) - v.integral, v.null = v.integral && w.integral, v.null || w.null + v.integral, v.nullable = v.integral && w.integral, v.nullable || w.nullable if v.notOperand == "" { v.notOperand = w.notOperand } @@ -164,8 +164,8 @@ func (p *valueProof) proveTemplate(t *template) proven { return p.proveUnion(leaves) case name == "calc": return p.proveCalc(t, body, args) - case builtins[name].number != nil: - return builtins[name].number(body, builtins[name].prints, args) + case builtins[name].proveNumber != nil: + return builtins[name].proveNumber(body, builtins[name].prints, args) case isTransform: return unproven(fmt.Sprintf("{%s} rewrites text rather than printing a value; write the values it would print", body)) } @@ -225,7 +225,7 @@ func (p *valueProof) proveExpr(n calcNode, fields map[string]node) (proven, stri func combine(n calcBin, l, r proven) (proven, string) { var v proven integral := l.integral && r.integral - switch n.op { + switch n.operator { case '+': v = bounded(l.lo+r.lo, l.hi+r.hi, integral) case '-': @@ -295,7 +295,7 @@ var typedCalls, operandCalls = numberCalls(false), numberCalls(true) func numberCalls(text bool) string { var calls []string for name, b := range builtins { - if b.number != nil && (text || b.prints != DataTypeString) { + if b.proveNumber != nil && (text || b.prints != DataTypeString) { calls = append(calls, "{"+name+"()}") } } -- 2.52.0 From c55c75a29e3bec0960a3ef3b2b39578f543c29d0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 19:16:57 +0200 Subject: [PATCH 117/966] Give dir, path, structs and r one meaning each, and name the table option indexes keyIndex through parentIndex --- data.go | 20 ++++++------- doc_test.go | 2 +- fejkdata.go | 4 +-- format.go | 12 ++++---- holdfence.go | 4 +-- shape_test.go | 2 +- struct.go | 14 ++++----- table.go | 80 +++++++++++++++++++++++++-------------------------- 8 files changed, 69 insertions(+), 69 deletions(-) diff --git a/data.go b/data.go index 3989539..3906661 100644 --- a/data.go +++ b/data.go @@ -13,11 +13,11 @@ import ( // label prefixes file names in errors; onDisk marks path as a directory that must // exist. type dataSource struct { - fsys fs.FS - label string - onDisk bool - path string - dir string + fsys fs.FS + label string + onDisk bool + diskPath string + baseDir string } func (s dataSource) labelled(p string) string { @@ -38,18 +38,18 @@ func loadData(sources []dataSource) (map[string]node, error) { root := map[string]node{} for _, src := range sources { if src.onDisk { - if src.path == "" { + if src.diskPath == "" { return nil, fmt.Errorf("a data path is empty") } - info, err := os.Stat(src.path) + info, err := os.Stat(src.diskPath) if err != nil { - return nil, fmt.Errorf("data path %s: %w", src.path, err) + return nil, fmt.Errorf("data path %s: %w", src.diskPath, err) } if !info.IsDir() { - return nil, fmt.Errorf("%s is not a directory", src.path) + return nil, fmt.Errorf("%s is not a directory", src.diskPath) } } - dir := src.dir + dir := src.baseDir if dir == "" { dir = "." } diff --git a/doc_test.go b/doc_test.go index c805159..b5d228d 100644 --- a/doc_test.go +++ b/doc_test.go @@ -95,7 +95,7 @@ func vocabulary(t *testing.T, files []*ast.File) string { } // declaredSymbols is every name the package declares at the top level, a method and -// a struct field keyed under its type: pinSet.pin, tableRead.whole. +// a struct field keyed under its type: pinSet.pin, tableRead.landsWhole. func declaredSymbols(files []*ast.File) map[string]bool { names := declaredFuncs(files) for _, f := range files { diff --git a/fejkdata.go b/fejkdata.go index ea89e72..d8bb4cc 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -72,7 +72,7 @@ func WithSeed(seed uint64) Option { // layer several; the last wins a name clash. func WithDataPath(dir string) Option { return func(c *config) { - c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, onDisk: true, path: dir}) + c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, onDisk: true, diskPath: dir}) } } @@ -99,7 +99,7 @@ func New(opts ...Option) (*Generator, error) { } var sources []dataSource if c.shipped { - sources = append(sources, dataSource{fsys: shippedFS, dir: "data"}) + sources = append(sources, dataSource{fsys: shippedFS, baseDir: "data"}) } sources = append(sources, c.sources...) if len(sources) == 0 { diff --git a/format.go b/format.go index fd3635a..1fac5ad 100644 --- a/format.go +++ b/format.go @@ -9,7 +9,7 @@ import ( // {…} token. type ftoken struct { kind byte // 'l' literal rune, 't' token body - r rune + char rune body string } @@ -24,7 +24,7 @@ func eachToken(format string, fn func(ftoken) error) error { switch c := rs[i]; c { case '{': if i+1 < len(rs) && rs[i+1] == '{' { - t.kind, t.r = 'l', '{' + t.kind, t.char = 'l', '{' i++ break } @@ -42,13 +42,13 @@ func eachToken(format string, fn func(ftoken) error) error { i = end case '}': if i+1 < len(rs) && rs[i+1] == '}' { - t.kind, t.r = 'l', '}' + t.kind, t.char = 'l', '}' i++ break } return fmt.Errorf("lone '}' in %q; a literal brace is written }}", format) default: - t.kind, t.r = 'l', c + t.kind, t.char = 'l', c } if err := fn(t); err != nil { return err @@ -81,7 +81,7 @@ func parseFormat(format string) ([]formatToken, error) { } err := eachToken(format, func(t ftoken) error { if t.kind == 'l' { - lit.WriteRune(t.r) + lit.WriteRune(t.char) return nil } flush() @@ -301,7 +301,7 @@ type arm struct { key string tail []string levels []string // the key at each level the tail passes through, the head's first - path string // key and tail, the one spelling every way of writing this read shares + path string // key and tail, the one path every way of writing this read shares } // splitArm splits one name into key and tail. refs maps a reference to what diff --git a/holdfence.go b/holdfence.go index c547cde..b6dea2f 100644 --- a/holdfence.go +++ b/holdfence.go @@ -188,8 +188,8 @@ func checkNoOverlap(ops []op, bound map[string]string) error { return nil } -// reader is one way a format reaches a bound field: as written, by its one -// spelling, and how to name it. +// reader is one way a format reaches a bound field: as written, by its path, +// and how to name it. type reader struct{ spelling, path, label string } // boundReaders lists every way a format reaches a bound sibling field, in the order the diff --git a/shape_test.go b/shape_test.go index aae7dbd..9747da2 100644 --- a/shape_test.go +++ b/shape_test.go @@ -77,7 +77,7 @@ func tableFacts(t *table) string { for _, o := range []struct { name string col int - }{{"key", t.keyCol}, {"name", t.nameCol}, {"weight", t.weightCol}, {"parent", t.parentCol}} { + }{{"key", t.keyIndex}, {"name", t.nameIndex}, {"weight", t.weightIndex}, {"parent", t.parentIndex}} { if o.col >= 0 { b.WriteString("\t" + o.name + " " + t.header[o.col]) } diff --git a/struct.go b/struct.go index 959fc07..66bfd61 100644 --- a/struct.go +++ b/struct.go @@ -49,7 +49,7 @@ func (f *Generator) structShapeOf(t reflect.Type) (*structShape, error) { if label == "" { label = "struct" } - sc := &structCompile{categories: f.categories, visiting: map[reflect.Type]bool{}, structs: maxStructs} + sc := &structCompile{categories: f.categories, visiting: map[reflect.Type]bool{}, structsLeft: maxStructs} shape, err := sc.compileShape(t, label) if err == nil && shape.empty() { err = fmt.Errorf("%s has no fake tags, so nothing to fill", t) @@ -82,9 +82,9 @@ func (s *structShape) empty() bool { return s.record == nil && len(s.nested) == // loaded tree, the types compiling or embedded above, so a pointer back to one is left alone // rather than filled without end, and how many more structs it may walk. type structCompile struct { - categories map[string]node - visiting map[reflect.Type]bool - structs int + categories map[string]node + visiting map[reflect.Type]bool + structsLeft int } // structFields gathers what one struct type fills: its tagged fields, those its embedded @@ -117,7 +117,7 @@ func (sc *structCompile) compileShape(t reflect.Type, label string) (*structShap } func (sc *structCompile) spend(label string) error { - if sc.structs--; sc.structs >= 0 { + if sc.structsLeft--; sc.structsLeft >= 0 { return nil } return fmt.Errorf(`%s: the struct fields reach more than %d structs; leave a struct field unfilled with fake:"-"`, label, maxStructs) @@ -252,13 +252,13 @@ func checkTaggedType(sf reflect.StructField) error { // compileRecord compiles the tagged fields of t as one record, and proves each column holds // only what its field's Go type can. -func (s *structShape) compileRecord(root map[string]node, t reflect.Type, label string, tags map[string]any) error { +func (s *structShape) compileRecord(categories map[string]node, t reflect.Type, label string, tags map[string]any) error { tags["format"] = "" n, err := compile(tags) if err != nil { return fmt.Errorf("%s: %w", label, err) } - if err := bindInline(n, label, root, checkRenders); err != nil { + if err := bindInline(n, label, categories, checkRenders); err != nil { return err } record, columns, err := recordOf(n) diff --git a/table.go b/table.go index 981de6f..5502e8d 100644 --- a/table.go +++ b/table.go @@ -23,10 +23,10 @@ type table struct { wholeRow *tableRow // the pinned row rendered by the format cells []string // rows × columns, flat cellTemplates map[int]*template - keyCol int // keyCol through parentCol are -1 where the option is absent - nameCol int - weightCol int - parentCol int + keyIndex int // keyIndex through parentIndex are column indexes, -1 where the option is absent + nameIndex int + weightIndex int + parentIndex int cum []float64 // cumulative weights, nil when uniform byKey map[string]int parentT *table @@ -93,7 +93,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab if err != nil { return nil, err } - t := &table{category: category, file: o.rows, keyCol: -1, nameCol: -1, weightCol: -1, parentCol: -1} + t := &table{category: category, file: o.rows, keyIndex: -1, nameIndex: -1, weightIndex: -1, parentIndex: -1} if err := t.parseRows(data); err != nil { return nil, fmt.Errorf("%s: %w", o.rows, err) } @@ -213,7 +213,7 @@ func (t *table) bindOptions(o tableOptionValues) error { for _, opt := range []struct { name, value string into *int - }{{"key", o.keyColumn, &t.keyCol}, {"name", o.nameColumn, &t.nameCol}, {"parent", o.parentColumn, &t.parentCol}, {"weight", o.weightColumn, &t.weightCol}} { + }{{"key", o.keyColumn, &t.keyIndex}, {"name", o.nameColumn, &t.nameIndex}, {"parent", o.parentColumn, &t.parentIndex}, {"weight", o.weightColumn, &t.weightIndex}} { if opt.value == "" { continue } @@ -223,7 +223,7 @@ func (t *table) bindOptions(o tableOptionValues) error { } *opt.into = i } - if t.nameCol >= 0 && t.keyCol < 0 && t.parentCol < 0 { + if t.nameIndex >= 0 && t.keyIndex < 0 && t.parentIndex < 0 { return fmt.Errorf("name selects a row as a key does, and lists the rows it matches by their keys, so it needs a key column, or a parent inside which each name is one row; add key") } if err := t.indexKeys(); err != nil { @@ -237,14 +237,14 @@ func (t *table) bindOptions(o tableOptionValues) error { // proveNamesInsideParent proves a name without a key names one row inside its parent. func (t *table) proveNamesInsideParent() error { - if t.keyCol >= 0 || t.nameCol < 0 { + if t.keyIndex >= 0 || t.nameIndex < 0 { return nil } inside := make(map[string]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.parentCol) + "\t" + t.cell(r, t.nameCol) + k := t.cell(r, t.parentIndex) + "\t" + t.cell(r, t.nameIndex) if first, dup := inside[k]; dup { - return fmt.Errorf("%s line %d: name %q repeats line %d inside %s %q; a name selects one row inside its parent; drop one, or add a key column", t.file, r+2, t.cell(r, t.nameCol), first+2, t.header[t.parentCol], t.cell(r, t.parentCol)) + return fmt.Errorf("%s line %d: name %q repeats line %d inside %s %q; a name selects one row inside its parent; drop one, or add a key column", t.file, r+2, t.cell(r, t.nameIndex), first+2, t.header[t.parentIndex], t.cell(r, t.parentIndex)) } inside[k] = r } @@ -253,12 +253,12 @@ func (t *table) proveNamesInsideParent() error { // indexKeys proves every key names one row, and keeps the index a link is proved by. func (t *table) indexKeys() error { - if t.keyCol < 0 { + if t.keyIndex < 0 { return nil } t.byKey = make(map[string]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.keyCol) + k := t.cell(r, t.keyIndex) if k == "" { return fmt.Errorf("%s line %d: the key is empty", t.file, r+2) } @@ -267,8 +267,8 @@ func (t *table) indexKeys() error { } t.byKey[k] = r } - for r := 0; r < t.rowCount() && t.nameCol >= 0; r++ { - if n := t.cell(r, t.nameCol); t.byKey[n] != r { + for r := 0; r < t.rowCount() && t.nameIndex >= 0; r++ { + if n := t.cell(r, t.nameIndex); t.byKey[n] != r { if other, isKey := t.byKey[n]; isKey { return fmt.Errorf("%s line %d: name %q is the key of line %d, which a selector reads first, so the name could never select this row", t.file, r+2, n, other+2) } @@ -279,15 +279,15 @@ func (t *table) indexKeys() error { // sumWeights proves every weight is a positive number and builds the cumulative table. func (t *table) sumWeights() error { - if t.weightCol < 0 { + if t.weightIndex < 0 { return nil } t.cum = make([]float64, t.rowCount()) total := 0.0 for r := range t.cum { - w, err := strconv.ParseFloat(t.cell(r, t.weightCol), 64) + w, err := strconv.ParseFloat(t.cell(r, t.weightIndex), 64) if err != nil || math.IsInf(w, 0) || math.IsNaN(w) || w <= 0 { - return fmt.Errorf("%s line %d: weight %q is not a positive number", t.file, r+2, t.cell(r, t.weightCol)) + return fmt.Errorf("%s line %d: weight %q is not a positive number", t.file, r+2, t.cell(r, t.weightIndex)) } total += w t.cum[r] = total @@ -302,7 +302,7 @@ func (t *table) sumWeights() error { func (t *table) checkCells() error { for i, cell := range t.cells { row, col := i/len(t.header), i%len(t.header) - if (col == t.keyCol || col == t.nameCol) && strings.ContainsAny(cell, inSelector) { + if (col == t.keyIndex || col == t.nameIndex) && strings.ContainsAny(cell, inSelector) { return fmt.Errorf("line %d: %s %q contains %q, which a selector cannot spell", row+2, t.header[col], cell, cell[strings.IndexAny(cell, inSelector):][:1]) } if strings.IndexByte(cell, '{') < 0 && strings.IndexByte(cell, '}') < 0 { @@ -379,7 +379,7 @@ func linkTables(root map[string]node) error { return err } case *table: - if n.parentCol < 0 { + if n.parentIndex < 0 { continue } if err := n.linkParent(path, children); err != nil { @@ -393,24 +393,24 @@ func linkTables(root map[string]node) error { } func (t *table) linkParent(path string, siblings map[string]node) error { - name := t.header[t.parentCol] + name := t.header[t.parentIndex] p, ok := siblings[name].(*table) switch { case siblings[name] == nil: return fmt.Errorf("parent %q names no table beside it", name) case !ok: return fmt.Errorf("parent %q is not a table; a link column reads a table's key", name) - case p.keyCol < 0: + case p.keyIndex < 0: return fmt.Errorf("parent %q has no key column to link to", name) } var ancestors []*table - for q, seen := p, map[*table]bool{t: true}; q != nil; q, _ = siblings[q.header[q.parentCol]].(*table) { + for q, seen := p, map[*table]bool{t: true}; q != nil; q, _ = siblings[q.header[q.parentIndex]].(*table) { if seen[q] { return fmt.Errorf("parent cycle: %s reaches itself through its parents", q.category) } seen[q] = true ancestors = append(ancestors, q) - if q.parentCol < 0 { + if q.parentIndex < 0 { break } } @@ -421,14 +421,14 @@ func (t *table) linkParent(path string, siblings map[string]node) error { } linked := make(map[string]bool, p.rowCount()) for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.parentCol) + k := t.cell(r, t.parentIndex) if _, ok := p.byKey[k]; !ok { return fmt.Errorf("%s line %d: %s %q is no key of %s", t.file, r+2, name, k, p.file) } linked[k] = true } for r := 0; r < p.rowCount(); r++ { - if k := p.cell(r, p.keyCol); !linked[k] { + if k := p.cell(r, p.keyIndex); !linked[k] { return fmt.Errorf("%s links no row to %s %q; every %s row needs one, or drop line %d of %s", t.file, name, k, name, r+2, p.file) } } @@ -442,17 +442,17 @@ func (t *table) linkParent(path string, siblings map[string]node) error { func (t *table) indexed() *tableIndex { t.once.Do(func() { - if t.nameCol >= 0 { + if t.nameIndex >= 0 { t.index.byName = make(map[string][]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { - n := t.cell(r, t.nameCol) + n := t.cell(r, t.nameIndex) t.index.byName[n] = append(t.index.byName[n], r) } } - if t.parentCol >= 0 { + if t.parentIndex >= 0 { t.index.rowsByParent = map[string][]int{} for r := 0; r < t.rowCount(); r++ { - k := t.cell(r, t.parentCol) + k := t.cell(r, t.parentIndex) t.index.rowsByParent[k] = append(t.index.rowsByParent[k], r) } if t.cum != nil { @@ -460,7 +460,7 @@ func (t *table) indexed() *tableIndex { for k, rows := range t.index.rowsByParent { cum, total := make([]float64, len(rows)), 0.0 for i, r := range rows { - w, _ := strconv.ParseFloat(t.cell(r, t.weightCol), 64) // sumWeights proved it + w, _ := strconv.ParseFloat(t.cell(r, t.weightIndex), 64) // sumWeights proved it total += w cum[i] = total } @@ -489,7 +489,7 @@ func pickCum(s *session, cum []float64) int { // drawUnder picks a row among those linked to parent row pr. func (t *table) drawUnder(s *session, pr int) int { ix := t.indexed() - k := t.parentT.cell(pr, t.parentT.keyCol) + k := t.parentT.cell(pr, t.parentT.keyIndex) rows := ix.rowsByParent[k] if ix.childCum == nil { return rows[s.IntN(len(rows))] @@ -536,7 +536,7 @@ func (t *table) descendant(name string) *table { } // parentRow is the row of t's parent that row r links to. -func (t *table) parentRow(r int) int { return t.parentT.byKey[t.cell(r, t.parentCol)] } +func (t *table) parentRow(r int) int { return t.parentT.byKey[t.cell(r, t.parentIndex)] } // descends reports whether a is an ancestor of t. func (t *table) descends(a *table) bool { @@ -561,7 +561,7 @@ func (t *table) under(r int, a *table, pr int) bool { // find is the row a selector names: by key first, then by name, where a name // naming several rows resolves inside the ancestors pinned. func (t *table) find(sel string, pins *pinSet) (int, error) { - if t.keyCol < 0 && t.nameCol < 0 { + if t.keyIndex < 0 && t.nameIndex < 0 { return 0, fmt.Errorf("%s has no key or name column to select a row by", t.path) } if r, ok := t.byKey[sel]; ok { @@ -585,14 +585,14 @@ func (t *table) find(sel string, pins *pinSet) (int, error) { func (t *table) ambiguous(sel string, rows []int) error { spellings := make([]string, len(rows)) for i, r := range rows { - if t.keyCol < 0 { + if t.keyIndex < 0 { spellings[i] = t.selectorSpelling(r) } else { - spellings[i] = t.cell(r, t.keyCol) + spellings[i] = t.cell(r, t.keyIndex) } } listed := strings.Join(spellings, ", ") - if t.keyCol < 0 { + if t.keyIndex < 0 { return fmt.Errorf("%q names %d rows of %s; select it inside its %s, one of %s", sel, len(rows), t.path, t.parentT.path, listed) } inside := "" @@ -606,10 +606,10 @@ func (t *table) ambiguous(sel string, rows []int) error { // by name inside its parent's row. func (t *table) selectorSpelling(r int) string { switch { - case t.keyCol >= 0: - return t.path + "[" + t.cell(r, t.keyCol) + "]" - case t.nameCol >= 0: - return t.parentT.selectorSpelling(t.parentRow(r)) + "." + t.category + "[" + t.cell(r, t.nameCol) + "]" + case t.keyIndex >= 0: + return t.path + "[" + t.cell(r, t.keyIndex) + "]" + case t.nameIndex >= 0: + return t.parentT.selectorSpelling(t.parentRow(r)) + "." + t.category + "[" + t.cell(r, t.nameIndex) + "]" } return fmt.Sprintf("%s line %d", t.file, r+2) } -- 2.52.0 From b38974adb2fc601bbd9ffec9eb385c92965cc34e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 19:17:35 +0200 Subject: [PATCH 118/966] Say path where hold and dataSource comments named the old spelling --- data.go | 2 +- hold.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/data.go b/data.go index 3906661..ed0542b 100644 --- a/data.go +++ b/data.go @@ -10,7 +10,7 @@ import ( ) // dataSource is one tree to load: an fs.FS and the directory in it to start from. -// label prefixes file names in errors; onDisk marks path as a directory that must +// label prefixes file names in errors; onDisk marks diskPath as a directory that must // exist. type dataSource struct { fsys fs.FS diff --git a/hold.go b/hold.go index d8f00f5..36909d9 100644 --- a/hold.go +++ b/hold.go @@ -5,7 +5,7 @@ import ( ) // hold is what has already been drawn for held names: the variant each was drawn as, so every -// path under it reads one row; the draw each read made, by its one spelling, so the same read +// path under it reads one row; the draw each read made, by its path, so the same read // written twice reads one draw; and the table rows those reads pinned. An expansion keeps one for // its sibling names, and a render one per group for its reference paths. type hold struct { -- 2.52.0 From c0cbb71d267284ce0d425656edf00779bbf45432 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 25 Sep 2026 19:18:23 +0200 Subject: [PATCH 119/966] Define what one meaning is in the naming decision --- docs/decisions.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 2389f2e..4abc5f2 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -310,11 +310,12 @@ file the stem of the units it holds. ## One name, one meaning Two declarations of any kind — function, method, field, type — share a name only where -they name one thing: an interface's method, `Fake` on each template kind, a field -holding what a same-named method returns, `renderScope.hold` answering with a `*hold`. -A bare function never shares a name with a method, since grep cannot tell a bare call -from a method's; `TestNoFunctionSpellsAMethod` holds that, and review judges the rest, -which is meaning no test can see. Serves goal 2: names that tell the truth, and reaching +one definition covers both: the same kind of value, counted or addressed the same way, so +a reader landing on the wrong one concludes nothing false. `Fake` on each template kind +renders one value from what it is called on; `renderScope.hold` returns the `*hold` it +names. A bare function never shares a name with a method: a search for the name returns +both, and a call site shows no receiver to tell them apart. `TestNoFunctionSpellsAMethod` +holds that, and review judges the rest. Serves goal 2: names that tell the truth, and reaching the unit behind a symptom without asking a person. Decided 2026-09-25 by Lilleman; valid while goal 2 counts reading cost. -- 2.52.0 From bee748a9327f40b77949432c26889c46a4177f49 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 13:38:26 +0200 Subject: [PATCH 120/966] File the 2026-09-26 nine-seat panel's findings, scored 5.7 --- todo.md | 52 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 49 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 8ae15fc..793e5f5 100644 --- a/todo.md +++ b/todo.md @@ -15,10 +15,49 @@ scoring run names too little to steer one. A fresh run replaces the items below. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9, -Self-sufficiency 5.6), every seat capped by Locality or Self-sufficiency. +Self-sufficiency 5.6); 5.7 on 2026-09-26 (Navigation 6.7, Locality 5.0, Shape 5.8, +Self-sufficiency 5.6), every seat capped by Locality, and all nine naming +`drawWalk.walk` among the hardest. This round, in order: +- State at each draw fence, in one line at the code, the rule it enforces, where a + `docs/decisions.md` link stands in for it today: the clash drop in `drawWalk.walk`, + `alternatives`, `replayPairs`' claim that pairs find every conflict, and + `heldCheck`'s skip of a reference path. Seven seats could not say why the fences + are complete without the decision bodies. +- Hold the draw fences to what render does: a test rendering seeded draws that fails + where a read the render makes falls outside what `drawWalk` gathered. Today the two + models agree by a comment on `renderEdges`, and a new render route drifts silently. +- Give "whole" one meaning: `doc.go` defines it as a read landing on a `tableRow`, + while `drawAt.wholeTable`, `checkFamilyPair` and `checkOwnFamily`'s comment use it for + a bare read landing on a `*table`, the one that draws without pinning. +- Give a table's current row one owner: `renderScope.t` and `row` hold it beside the + `pinSet`, and `renderLeaf` reads a column's null back from a hold `readField` wrote. +- Split `walkPath`'s four modes, so a caller does not select a behaviour by which of + `level`, `leaf`, `pins`, `drawn` and `draws` it leaves nil. +- Gather the family fence beside `drawWalk.check`: `checkFamilies`, `replayPairs`, + `checkFamilyPair` and `tableRead.replay` sit in `family.go` and `pins.go`, and one + rule spans seven files. +- Compile every format in one phase: a template holding a reference has nil `ops` + until `linkTemplateRefs`, so a check reading `ops` before link skips it silently. +- Run `New`, `NewTemplate` and `FakeStruct` through one bind pipeline that names + where they differ: each picks its own fences today. +- Make `disagreement` and `bothTyped` a table of cases: each swaps its operands and + runs a proof to pick an error's wording. +- Derive a column's datatype in one place: `columnItems` yields nothing for a + `tableColumn`, and six callers re-derive it from items. +- Give `head`, `index`, `bound` and `draw` one meaning each, name `table.family` + for the root it returns, and replace the token kind bytes `'l'`, `'t'`, `'f'` and + `'b'`, whose `'l'` means different things in `ftoken` and `formatToken`. +- Start every render from one spelling of an empty hold set: `Fake` uses + `holdSet{}`, the record and struct paths `eagerHoldSet()`. +- Correct the false comments: `funcCall` names `checkFunc` where `parseFormat` + reports, `struct.go`'s "t sits at index within c.t" names no field, `recordFormats` + calls ndjson "the same column", `Generator` says every other entry point takes `mu` + where `NewTemplate` and `NewRecordTemplate` do not, and `List` claims to be every + path `Fake` accepts, which a selector is too. Delete the dead `table.descends` and + the redundant `string(key)` in `checkNoRepeatedItem`. - Run the nine-seat panel again and file what it names here. ### Data @@ -138,8 +177,15 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI -- Report the same error every load for a table with two bad options: - `readTableOptions` returns on the first in Go's map order. +- Report the same error every load for a table with two bad options, and for a + folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the + first in Go's map order. +- Report a CRLF rows file holding only its header as having no rows: `parseRows` + answers "line 2 is empty". +- Refuse a bare table read in a typed column with a reason naming the table: + `prove`'s default arm says it reads a null. +- Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and + `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. - Refuse two whole reads of one table family in one render, which goal 5 promises is a load error: `{/sel}|{/sel}` panics out of `Fake` with "a fence should have refused this at New" where the two draws land on rows whose cells select different rows of another -- 2.52.0 From f0b92260eda74cbd0e75f59e4229c8f867a0e66a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 13:55:45 +0200 Subject: [PATCH 121/966] State the rule each draw fence enforces beside it: the cell drop in walk, alternatives, replayPairs and heldCheck --- drawfence.go | 3 ++- family.go | 10 ++++++---- holdfence.go | 2 +- todo.md | 5 ----- 4 files changed, 9 insertions(+), 11 deletions(-) diff --git a/drawfence.go b/drawfence.go index d1ebc31..77d9a25 100644 --- a/drawfence.go +++ b/drawfence.go @@ -225,7 +225,8 @@ func newDrawWalk() *drawWalk { } // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops -// there. +// there. A cell whose row the pins keep out has no case: a pinned table renders its pinned row +// alone, so that cell never renders on this route. func (w *drawWalk) walk(n node, at drawAt) { v := nodeVisit{n, at.group, at.rowsKey()} if w.seen[v] { diff --git a/family.go b/family.go index a8003bf..a4f9929 100644 --- a/family.go +++ b/family.go @@ -103,8 +103,9 @@ func checkFamilies(reads []pathRead) error { } // replayPairs replays every two reads of one draw group that can render together into -// one draws, the earlier read first. A pin conflicts with one earlier pin, never with a -// combination, so pairs find every conflict a full replay would. +// one draws, the earlier read first. Pairs find every conflict a full replay would: a +// read pins a row's ancestors with it, and clash judges a row against one pinned table, +// so a conflict is between two reads' pins and a third read adds nothing to it. func replayPairs(reads []pathRead) error { for i, r := range reads { if r.tr == nil { @@ -216,8 +217,9 @@ func checkOwnFamily(t *template) error { return nil } -// alternatives reports whether two reads sit in different rows of one table: rows the walks pinned, -// or rows of one whole draw. +// alternatives reports whether two reads never render together: one read renders one row of a +// table, so reads under different rows the walks pinned, or under different rows of one whole +// draw, never meet. // docs/decisions.md#the-rows-of-a-table-are-alternatives func alternatives(a, b drawAt) bool { return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) diff --git a/holdfence.go b/holdfence.go index b6dea2f..e249fb5 100644 --- a/holdfence.go +++ b/holdfence.go @@ -18,7 +18,7 @@ func heldCheck(path string, n node) error { readers := boundReaders(t.ops, t.bound) for _, head := range heldHeads(t) { if _, isPath := t.bound[head]; isPath && isRef(head) { - continue + continue // held for the render: drawCheck compares it across the render and its groups } if err := checkHeadHeld(t, head, readers); err != nil { return fmt.Errorf("%s: %w", path, err) diff --git a/todo.md b/todo.md index 793e5f5..8c1298b 100644 --- a/todo.md +++ b/todo.md @@ -21,11 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- State at each draw fence, in one line at the code, the rule it enforces, where a - `docs/decisions.md` link stands in for it today: the clash drop in `drawWalk.walk`, - `alternatives`, `replayPairs`' claim that pairs find every conflict, and - `heldCheck`'s skip of a reference path. Seven seats could not say why the fences - are complete without the decision bodies. - Hold the draw fences to what render does: a test rendering seeded draws that fails where a read the render makes falls outside what `drawWalk` gathered. Today the two models agree by a comment on `renderEdges`, and a new render route drifts silently. -- 2.52.0 From e3017801d1b89964083d0014e8d9c9193d25fc9b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 14:01:28 +0200 Subject: [PATCH 122/966] Cover the ancestor clash and name what drawCheck compares by --- drawfence.go | 3 ++- family.go | 8 ++++---- holdfence.go | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/drawfence.go b/drawfence.go index 77d9a25..9e0fa2a 100644 --- a/drawfence.go +++ b/drawfence.go @@ -226,7 +226,8 @@ func newDrawWalk() *drawWalk { // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops // there. A cell whose row the pins keep out has no case: a pinned table renders its pinned row -// alone, so that cell never renders on this route. +// alone and an unpinned one draws a row inside its nearest pinned ancestor's, so that cell never +// renders on this route. func (w *drawWalk) walk(n node, at drawAt) { v := nodeVisit{n, at.group, at.rowsKey()} if w.seen[v] { diff --git a/family.go b/family.go index a4f9929..05ca01b 100644 --- a/family.go +++ b/family.go @@ -102,10 +102,10 @@ func checkFamilies(reads []pathRead) error { return replayPairs(reads) } -// replayPairs replays every two reads of one draw group that can render together into -// one draws, the earlier read first. Pairs find every conflict a full replay would: a -// read pins a row's ancestors with it, and clash judges a row against one pinned table, -// so a conflict is between two reads' pins and a third read adds nothing to it. +// replayPairs replays every two reads that can render together into one draws, the earlier +// read first. Pairs find every conflict a full replay would: clash judges a row against one +// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry +// its rows' ancestors, so the pair of those two reads clashes the same way. func replayPairs(reads []pathRead) error { for i, r := range reads { if r.tr == nil { diff --git a/holdfence.go b/holdfence.go index e249fb5..f108c20 100644 --- a/holdfence.go +++ b/holdfence.go @@ -18,7 +18,7 @@ func heldCheck(path string, n node) error { readers := boundReaders(t.ops, t.bound) for _, head := range heldHeads(t) { if _, isPath := t.bound[head]; isPath && isRef(head) { - continue // held for the render: drawCheck compares it across the render and its groups + continue // held for the render: drawCheck compares every read of it, by path, across the render and its groups } if err := checkHeadHeld(t, head, readers); err != nil { return fmt.Errorf("%s: %w", path, err) -- 2.52.0 From fc8bb3424205db8d4ca707078b9c3cdcf199908b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 14:27:43 +0200 Subject: [PATCH 123/966] Want every reference a render reads among the reads drawWalk gathers for that render --- drawfence_test.go | 137 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 137 insertions(+) create mode 100644 drawfence_test.go diff --git a/drawfence_test.go b/drawfence_test.go new file mode 100644 index 0000000..b785070 --- /dev/null +++ b/drawfence_test.go @@ -0,0 +1,137 @@ +package fejkdata + +import ( + "strings" + "testing" +) + +// fenceCorpus layers over the shipped set what it lacks: draw groups, a repeat beside one, token +// cells selecting a row, a table read whole and records reading a family. +func fenceCorpus(t *testing.T) *Generator { + t.Helper() + dir := writeFiles(t, with(geo(), map[string]string{ + "addr.json": `{"format":"{a} {b} {c}","a":{"format":"{/locality.name}, {/municipality.name}"},"b":{"format":"{/region[12].name} {/place}","drawGroup":"g"},"c":{"format":"{/place.zip} ","repeat":2}}`, + "place.json": `{"format":"{zip} {name} {tag}","rows":"place.tsv","key":"name"}`, + "place.tsv": "name\tzip\ttag\nStockholm\t1{digits(2)} {digits(2)}\t{/w}\nTranås\t573 {digits(2)}\t{/region[12].name}\n", + "rec.json": `{"format":"","l":"{/locality.name}","m":"{/municipality.name}","t":"{/place[Stockholm].tag}"}`, + "w.json": `["x","y"]`, + })) + f, err := New(WithDataPath(dir), WithSeed(1)) + if err != nil { + t.Fatalf("New = %v", err) + } + return f +} + +type fenceRoot struct { + label string + t *template +} + +func fenceRoots(t *testing.T, f *Generator) []fenceRoot { + t.Helper() + var roots []fenceRoot + _ = walkNodes(f.categories, func(path string, n node) error { + if tm, ok := n.(*template); ok { + roots = append(roots, fenceRoot{path, tm}) + } + return nil + }) + inline, err := f.NewTemplate(`{"format":"{x}|{y}","x":{"format":"{/locality.name}","drawGroup":"g"},"y":"{/region[01].name}"}`) + if err != nil { + t.Fatalf("NewTemplate = %v", err) + } + record, err := f.NewRecordTemplate(`{"format":"","a":"{/locality.name}","b":"{/municipality.name}"}`) + if err != nil { + t.Fatalf("NewRecordTemplate = %v", err) + } + _ = eachNode(inline.n, "inline", func(path string, n node) error { + if tm, ok := n.(*template); ok { + roots = append(roots, fenceRoot{path, tm}) + } + return nil + }) + return append(roots, fenceRoot{"inline record", record.template}) +} + +func TestEveryReadARenderMakesIsGathered(t *testing.T) { + f := fenceCorpus(t) + for _, root := range fenceRoots(t, f) { + wantGathered(t, f.rand, root.label, renderDraws(root.t).reads, func() { renderRoot(f.rand, root.t) }) + if !root.t.isRecord { + continue + } + columns := sortedNames(root.t.fields) + wantGathered(t, f.rand, root.label+" as a record", columnDraws(root.t, columns).reads, func() { renderRecordRoot(f.rand, root.t) }) + } +} + +// wantGathered renders a root five times over the seeded stream and fails where a reference the +// render read is not among gathered: same draw group and path, under rows that can render together. +func wantGathered(t *testing.T, s *session, label string, gathered []pathRead, renderOnce func()) { + t.Helper() + var reads []pathRead + s.trace = &renderTrace{read: func(_ *template, group string, pins pinSet, a arm) { + if s.trace.depth == 0 && isRef(a.key) { + reads = append(reads, pathRead{at: drawAt{group: group, pins: pins}, a: a}) + } + }} + defer func() { s.trace = nil }() + for i := 0; i < 5; i++ { + renderOnce() + } + for _, r := range reads { + if !gathers(gathered, r) { + t.Errorf("%s: the render read {%s} in draw group %q with %s pinned, and drawWalk gathered no such read", label, r.a.spelling, r.at.group, pinned(&r.at.pins)) + } + } +} + +func gathers(gathered []pathRead, r pathRead) bool { + for _, g := range gathered { + if g.at.group == r.at.group && g.a.path == r.a.path && !g.at.pins.differs(&r.at.pins) { + return true + } + } + return false +} + +func pinned(p *pinSet) string { + var rows []string + p.each(func(t *table, r int) { rows = append(rows, t.selectorSpelling(r)) }) + if len(rows) == 0 { + return "no row" + } + return strings.Join(rows, ", ") +} + +// renderRoot renders t the way the render reaches it: a table's format through its table, a cell in +// its row, one iteration of a repeat in t's own draw group, and anything else as itself. +func renderRoot(s *session, t *template) { + var set holdSet + sc := renderScope{set: &set} + switch { + case t.table != nil: + render(s, t.table, sc) + case t.cellOf != nil: + sc.t, sc.row = t.cellOf, t.cellRow + render(s, t, sc) + case t.repeat > 1: + expand(s, t, sc.in(t)) + default: + render(s, t, sc) + } +} + +func renderRecordRoot(s *session, t *template) { + set := eagerHoldSet() + sc := renderScope{set: &set} + if t.table != nil { + t.table.drawIn(s, &sc.in(t).hold().pins) + } + _, columns, err := recordOf(t) + if err != nil { + panic(err) + } + renderRecord(s, t, columns, sc) +} -- 2.52.0 From 1a6111d67d4c3ed5d1b3982929b81d11030c570e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 14:36:39 +0200 Subject: [PATCH 124/966] Trace the reads a render makes, gather a render's draws in renderDraws and columnDraws, and drop the done item --- drawfence.go | 20 ++++++++++++++------ drawfence_test.go | 30 ++++++++++++++++++++---------- fejkdata.go | 1 + hold.go | 11 +++++++++++ holdset.go | 4 ++++ todo.md | 3 --- 6 files changed, 50 insertions(+), 19 deletions(-) diff --git a/drawfence.go b/drawfence.go index 9e0fa2a..9211afc 100644 --- a/drawfence.go +++ b/drawfence.go @@ -69,14 +69,19 @@ func (c *drawCheck) checkDraws(path string, n node) error { if !c.readsPath(t) { return nil } + if err := renderDraws(t).check(); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + return nil +} + +// renderDraws gathers the reads one render of t makes. +func renderDraws(t *template) *drawWalk { w := newDrawWalk() for _, e := range renderEdges(t) { w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}}) } - if err := w.check(); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - return nil + return w } // checkRecordDraws fences the columns of a record — a template compiled at the top without a @@ -172,12 +177,15 @@ func refRead(n node, label string) (arm, bool) { } // checkColumnDraws fences a record's columns as one render. -func checkColumnDraws(t *template, columns []string) error { +func checkColumnDraws(t *template, columns []string) error { return columnDraws(t, columns).check() } + +// columnDraws gathers the reads one record render of t's columns makes. +func columnDraws(t *template, columns []string) *drawWalk { w := newDrawWalk() for _, name := range columns { w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) } - return w.check() + return w } // drawWalk gathers the references one render reads, by draw group, for check to compare. diff --git a/drawfence_test.go b/drawfence_test.go index b785070..477eb47 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -10,7 +10,7 @@ import ( func fenceCorpus(t *testing.T) *Generator { t.Helper() dir := writeFiles(t, with(geo(), map[string]string{ - "addr.json": `{"format":"{a} {b} {c}","a":{"format":"{/locality.name}, {/municipality.name}"},"b":{"format":"{/region[12].name} {/place}","drawGroup":"g"},"c":{"format":"{/place.zip} ","repeat":2}}`, + "addr.json": `{"format":"{a} {b} {c}","a":"{/locality.name}, {/municipality.name}","b":{"format":"{/region[12].name} {/place}","drawGroup":"g"},"c":{"format":"{/place.zip} ","repeat":2}}`, "place.json": `{"format":"{zip} {name} {tag}","rows":"place.tsv","key":"name"}`, "place.tsv": "name\tzip\ttag\nStockholm\t1{digits(2)} {digits(2)}\t{/w}\nTranås\t573 {digits(2)}\t{/region[12].name}\n", "rec.json": `{"format":"","l":"{/locality.name}","m":"{/municipality.name}","t":"{/place[Stockholm].tag}"}`, @@ -67,13 +67,23 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { } // wantGathered renders a root five times over the seeded stream and fails where a reference the -// render read is not among gathered: same draw group and path, under rows that can render together. +// render read is not among gathered: same draw group and path, and where the read sits in a cell, +// gathered from that row or from no row of its table. func wantGathered(t *testing.T, s *session, label string, gathered []pathRead, renderOnce func()) { t.Helper() var reads []pathRead - s.trace = &renderTrace{read: func(_ *template, group string, pins pinSet, a arm) { - if s.trace.depth == 0 && isRef(a.key) { - reads = append(reads, pathRead{at: drawAt{group: group, pins: pins}, a: a}) + seen := map[readKey]bool{} + s.trace = &renderTrace{read: func(from *template, group string, a arm) { + if s.trace.depth != 0 || !isRef(a.key) { + return + } + r := pathRead{at: drawAt{group: group}, a: a} + if from.cellOf != nil { + r.at.pins.add(from.cellOf, from.cellRow) + } + if k := (readKey{group, a.path, r.at.rowsKey()}); !seen[k] { + seen[k] = true + reads = append(reads, r) } }} defer func() { s.trace = nil }() @@ -82,27 +92,27 @@ func wantGathered(t *testing.T, s *session, label string, gathered []pathRead, r } for _, r := range reads { if !gathers(gathered, r) { - t.Errorf("%s: the render read {%s} in draw group %q with %s pinned, and drawWalk gathered no such read", label, r.a.spelling, r.at.group, pinned(&r.at.pins)) + t.Errorf("%s: the render read {%s} in draw group %q from %s, and drawWalk gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) } } } func gathers(gathered []pathRead, r pathRead) bool { for _, g := range gathered { - if g.at.group == r.at.group && g.a.path == r.a.path && !g.at.pins.differs(&r.at.pins) { + if g.at.group == r.at.group && g.a.path == r.a.path && !g.at.pins.differs(&r.at.pins) && !g.at.wholePins.differs(&r.at.pins) { return true } } return false } -func pinned(p *pinSet) string { +func spellPins(p *pinSet) string { var rows []string p.each(func(t *table, r int) { rows = append(rows, t.selectorSpelling(r)) }) if len(rows) == 0 { - return "no row" + return "no cell" } - return strings.Join(rows, ", ") + return "the cell of " + strings.Join(rows, ", ") } // renderRoot renders t the way the render reaches it: a table's format through its table, a cell in diff --git a/fejkdata.go b/fejkdata.go index d8bb4cc..def390f 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -45,6 +45,7 @@ type Generator struct { type session struct { *rand.Rand counters map[string]uint64 + trace *renderTrace } func (s *session) next(key string) uint64 { diff --git a/hold.go b/hold.go index 36909d9..b5d752d 100644 --- a/hold.go +++ b/hold.go @@ -2,6 +2,7 @@ package fejkdata import ( "fmt" + "strings" ) // hold is what has already been drawn for held names: the variant each was drawn as, so every @@ -27,6 +28,9 @@ type draw struct { // drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and // linkRefs prove every step, so the walk cannot fail. func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { + if s.trace != nil { + s.trace.read(t, strings.Clone(sc.group), a) + } if !t.held[a.key] { if len(a.tail) > 0 { panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) @@ -45,6 +49,13 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw return r } +// renderTrace is a test's view of a render: each field a format reads, and how many repeat +// iterations, each a render of its own, the read sits inside. nil outside a test. +type renderTrace struct { + read func(from *template, group string, a arm) + depth int +} + // readHold is the hold a held read keeps its draw in: for a reference that reads a path, // the render's hold for its group, so its draw spans the render; for a sibling, or a // reference read whole, held. diff --git a/holdset.go b/holdset.go index db1e022..28e5ec4 100644 --- a/holdset.go +++ b/holdset.go @@ -35,6 +35,10 @@ func renderOnce(s *session, n node) string { //go:noinline func expandAnew(s *session, t *template) string { var set holdSet + if s.trace != nil { + s.trace.depth++ + defer func() { s.trace.depth-- }() + } return expand(s, t, renderScope{set: &set}) } diff --git a/todo.md b/todo.md index 8c1298b..bb5e0d4 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Hold the draw fences to what render does: a test rendering seeded draws that fails - where a read the render makes falls outside what `drawWalk` gathered. Today the two - models agree by a comment on `renderEdges`, and a new render route drifts silently. - Give "whole" one meaning: `doc.go` defines it as a read landing on a `tableRow`, while `drawAt.wholeTable`, `checkFamilyPair` and `checkOwnFamily`'s comment use it for a bare read landing on a `*table`, the one that draws without pinning. -- 2.52.0 From 9d8a52de68cde6d9ea700871b1ab186c125be479 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 14:51:06 +0200 Subject: [PATCH 125/966] Trace the row a render stands in, skip a fieldless record, and cover a cell reading through a template --- drawfence_test.go | 61 ++++++++++++++++++++++++----------------------- hold.go | 13 ++++++---- holdset.go | 4 ++-- 3 files changed, 41 insertions(+), 37 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index 477eb47..3387c02 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -5,16 +5,15 @@ import ( "testing" ) -// fenceCorpus layers over the shipped set what it lacks: draw groups, a repeat beside one, token -// cells selecting a row, a table read whole and records reading a family. func fenceCorpus(t *testing.T) *Generator { t.Helper() dir := writeFiles(t, with(geo(), map[string]string{ "addr.json": `{"format":"{a} {b} {c}","a":"{/locality.name}, {/municipality.name}","b":{"format":"{/region[12].name} {/place}","drawGroup":"g"},"c":{"format":"{/place.zip} ","repeat":2}}`, "place.json": `{"format":"{zip} {name} {tag}","rows":"place.tsv","key":"name"}`, - "place.tsv": "name\tzip\ttag\nStockholm\t1{digits(2)} {digits(2)}\t{/w}\nTranås\t573 {digits(2)}\t{/region[12].name}\n", + "place.tsv": "name\tzip\ttag\nStockholm\t1{digits(2)} {digits(2)}\t{/x}\nTranås\t573 {digits(2)}\t{/region[12].name}\n", "rec.json": `{"format":"","l":"{/locality.name}","m":"{/municipality.name}","t":"{/place[Stockholm].tag}"}`, "w.json": `["x","y"]`, + "x.json": `"{/w}"`, })) f, err := New(WithDataPath(dir), WithSeed(1)) if err != nil { @@ -28,12 +27,16 @@ type fenceRoot struct { t *template } -func fenceRoots(t *testing.T, f *Generator) []fenceRoot { +func fenceRoots(t *testing.T, f *Generator) ([]fenceRoot, map[string]*table) { t.Helper() var roots []fenceRoot + tables := map[string]*table{} _ = walkNodes(f.categories, func(path string, n node) error { - if tm, ok := n.(*template); ok { - roots = append(roots, fenceRoot{path, tm}) + switch n := n.(type) { + case *template: + roots = append(roots, fenceRoot{path, n}) + case *table: + tables[n.path] = n } return nil }) @@ -51,35 +54,39 @@ func fenceRoots(t *testing.T, f *Generator) []fenceRoot { } return nil }) - return append(roots, fenceRoot{"inline record", record.template}) + return append(roots, fenceRoot{"inline record", record.template}), tables } func TestEveryReadARenderMakesIsGathered(t *testing.T) { f := fenceCorpus(t) - for _, root := range fenceRoots(t, f) { - wantGathered(t, f.rand, root.label, renderDraws(root.t).reads, func() { renderRoot(f.rand, root.t) }) - if !root.t.isRecord { + roots, tables := fenceRoots(t, f) + for _, root := range roots { + wantGathered(t, f.rand, tables, root.label, renderDraws(root.t).reads, func() { renderRoot(f.rand, root.t) }) + if !root.t.isRecord || len(root.t.fields) == 0 { continue } - columns := sortedNames(root.t.fields) - wantGathered(t, f.rand, root.label+" as a record", columnDraws(root.t, columns).reads, func() { renderRecordRoot(f.rand, root.t) }) + _, columns, err := recordOf(root.t) + if err != nil { + t.Fatalf("%s: recordOf = %v", root.label, err) + } + wantGathered(t, f.rand, tables, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads, func() { renderRecordRoot(f.rand, root.t, columns) }) } } // wantGathered renders a root five times over the seeded stream and fails where a reference the -// render read is not among gathered: same draw group and path, and where the read sits in a cell, -// gathered from that row or from no row of its table. -func wantGathered(t *testing.T, s *session, label string, gathered []pathRead, renderOnce func()) { +// render read is not among gathered: same draw group and path, and where the render stood in a +// table's row, gathered from that row or from no row of the table. +func wantGathered(t *testing.T, s *session, tables map[string]*table, label string, gathered []pathRead, renderOnce func()) { t.Helper() var reads []pathRead seen := map[readKey]bool{} - s.trace = &renderTrace{read: func(from *template, group string, a arm) { - if s.trace.depth != 0 || !isRef(a.key) { + s.trace = &renderTrace{read: func(group, table string, row int, a arm) { + if s.trace.repeatDepth != 0 || !isRef(a.key) { return } r := pathRead{at: drawAt{group: group}, a: a} - if from.cellOf != nil { - r.at.pins.add(from.cellOf, from.cellRow) + if table != "" { + r.at.pins.add(tables[table], row) } if k := (readKey{group, a.path, r.at.rowsKey()}); !seen[k] { seen[k] = true @@ -110,13 +117,11 @@ func spellPins(p *pinSet) string { var rows []string p.each(func(t *table, r int) { rows = append(rows, t.selectorSpelling(r)) }) if len(rows) == 0 { - return "no cell" + return "no row" } - return "the cell of " + strings.Join(rows, ", ") + return "row " + strings.Join(rows, ", ") } -// renderRoot renders t the way the render reaches it: a table's format through its table, a cell in -// its row, one iteration of a repeat in t's own draw group, and anything else as itself. func renderRoot(s *session, t *template) { var set holdSet sc := renderScope{set: &set} @@ -127,21 +132,17 @@ func renderRoot(s *session, t *template) { sc.t, sc.row = t.cellOf, t.cellRow render(s, t, sc) case t.repeat > 1: - expand(s, t, sc.in(t)) + expand(s, t, sc) default: render(s, t, sc) } } -func renderRecordRoot(s *session, t *template) { +func renderRecordRoot(s *session, t *template, columns []Column) { set := eagerHoldSet() sc := renderScope{set: &set} if t.table != nil { - t.table.drawIn(s, &sc.in(t).hold().pins) - } - _, columns, err := recordOf(t) - if err != nil { - panic(err) + t.table.drawIn(s, &sc.hold().pins) } renderRecord(s, t, columns, sc) } diff --git a/hold.go b/hold.go index b5d752d..10ec227 100644 --- a/hold.go +++ b/hold.go @@ -29,7 +29,11 @@ type draw struct { // linkRefs prove every step, so the walk cannot fail. func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { if s.trace != nil { - s.trace.read(t, strings.Clone(sc.group), a) + var table string + if sc.t != nil { + table = strings.Clone(sc.t.path) + } + s.trace.read(strings.Clone(sc.group), table, sc.row, a) } if !t.held[a.key] { if len(a.tail) > 0 { @@ -49,11 +53,10 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw return r } -// renderTrace is a test's view of a render: each field a format reads, and how many repeat -// iterations, each a render of its own, the read sits inside. nil outside a test. +// renderTrace is a test's view of the fields a render reads; nil outside a test. type renderTrace struct { - read func(from *template, group string, a arm) - depth int + read func(group, table string, row int, a arm) + repeatDepth int } // readHold is the hold a held read keeps its draw in: for a reference that reads a path, diff --git a/holdset.go b/holdset.go index 28e5ec4..4cdc4fe 100644 --- a/holdset.go +++ b/holdset.go @@ -36,8 +36,8 @@ func renderOnce(s *session, n node) string { func expandAnew(s *session, t *template) string { var set holdSet if s.trace != nil { - s.trace.depth++ - defer func() { s.trace.depth-- }() + s.trace.repeatDepth++ + defer func() { s.trace.repeatDepth-- }() } return expand(s, t, renderScope{set: &set}) } -- 2.52.0 From 55fb1ec42e5707679003affdd510374c74b4ec57 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 14:57:04 +0200 Subject: [PATCH 126/966] Render every row of a table root pinned, and drop the comments restating renderDraws and columnDraws --- drawfence.go | 2 -- drawfence_test.go | 6 ++++++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drawfence.go b/drawfence.go index 9211afc..5adad1d 100644 --- a/drawfence.go +++ b/drawfence.go @@ -75,7 +75,6 @@ func (c *drawCheck) checkDraws(path string, n node) error { return nil } -// renderDraws gathers the reads one render of t makes. func renderDraws(t *template) *drawWalk { w := newDrawWalk() for _, e := range renderEdges(t) { @@ -179,7 +178,6 @@ func refRead(n node, label string) (arm, bool) { // checkColumnDraws fences a record's columns as one render. func checkColumnDraws(t *template, columns []string) error { return columnDraws(t, columns).check() } -// columnDraws gathers the reads one record render of t's columns makes. func columnDraws(t *template, columns []string) *drawWalk { w := newDrawWalk() for _, name := range columns { diff --git a/drawfence_test.go b/drawfence_test.go index 3387c02..91e6bfb 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -128,6 +128,12 @@ func renderRoot(s *session, t *template) { switch { case t.table != nil: render(s, t.table, sc) + for r := 0; r < t.table.rowCount(); r++ { + var set holdSet + sc := renderScope{set: &set} + sc.hold().pins.pin(t.table, r) + render(s, t.table.wholeRow, sc) + } case t.cellOf != nil: sc.t, sc.row = t.cellOf, t.cellRow render(s, t, sc) -- 2.52.0 From 203b235fdd65237957afee56b823857b68089452 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 15:04:00 +0200 Subject: [PATCH 127/966] State the copy rule on renderScope once, and say the allocation gate prices what keeps a hold set off the heap --- docs/decisions.md | 2 +- drawfence_test.go | 3 --- holdset.go | 6 ++++-- perf_test.go | 2 +- 4 files changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 4abc5f2..721f41f 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -249,7 +249,7 @@ set is declared keeps them on that frame's stack. A `Fake` often reads no refere at all, and making them anyway cost about a fifth of the cheapest render, so it makes them on the first read instead, at two heap allocations for a render that does share a draw. The allocation gate over a repeat of a reference path and over a named draw group -prices that, and pins the two measures that keep a hold set off the heap. +prices that, and pins what keeps a hold set off the heap. ## A category never references itself, and a record's fences run at load diff --git a/drawfence_test.go b/drawfence_test.go index 91e6bfb..6d28bf7 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -73,9 +73,6 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { } } -// wantGathered renders a root five times over the seeded stream and fails where a reference the -// render read is not among gathered: same draw group and path, and where the render stood in a -// table's row, gathered from that row or from no row of the table. func wantGathered(t *testing.T, s *session, tables map[string]*table, label string, gathered []pathRead, renderOnce func()) { t.Helper() var reads []pathRead diff --git a/holdset.go b/holdset.go index 4cdc4fe..ab91287 100644 --- a/holdset.go +++ b/holdset.go @@ -11,7 +11,9 @@ type holdSet struct { } // renderScope is where a render reads its reference paths: a hold set, in the draw group of the -// template rendering, and the row a table's format is rendering, which its columns read. +// template rendering, and the row a table's format is rendering, which its columns read. It passes +// by value, and what is read from it reaches a map key, an interface or a func value only as a +// copy; else sc, and with it every render's hold set, moves to the heap. type renderScope struct { set *holdSet group string @@ -60,7 +62,7 @@ func (sc renderScope) hold() *hold { sc.set.named = map[string]*hold{} } d = &hold{variant: map[string]node{}, value: map[string]draw{}} - sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's hold set, to the heap + sc.set.named[strings.Clone(sc.group)] = d } return d } diff --git a/perf_test.go b/perf_test.go index 4491566..7336cf6 100644 --- a/perf_test.go +++ b/perf_test.go @@ -55,7 +55,7 @@ func TestNoRenderAllocRegression(t *testing.T) { } } -// The repeat shape prices both escape measures: dropping either costs an alloc an iteration. +// The repeat shape prices what keeps a hold set off the heap: a copy dropped costs an alloc an iteration. func TestNoReferenceAllocRegression(t *testing.T) { word := `{"format":"{w}","w":["alpha","beta","gamma","delta"]}` for _, s := range []struct { -- 2.52.0 From c1b928a3f53ced245144f3364d670b76fe8ba0e9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 15:22:30 +0200 Subject: [PATCH 128/966] Give whole one meaning, the bare table read, and name the pinned row pinnedRow, landsRow and compileRowFormat --- doc.go | 5 +++-- doc_test.go | 2 +- drawfence.go | 2 +- drawfence_test.go | 2 +- family.go | 12 ++++++------ hold.go | 2 +- node.go | 2 +- path.go | 2 +- table.go | 14 +++++++------- table_test.go | 2 +- todo.md | 3 --- 11 files changed, 23 insertions(+), 25 deletions(-) diff --git a/doc.go b/doc.go index eb4017f..f6a59dd 100644 --- a/doc.go +++ b/doc.go @@ -32,8 +32,9 @@ package fejkdata // `pinSet`, `pinSet.pin`, `table.drawIn`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.family`. -// - whole — a read that lands on a `tableRow`, so the row renders through its table's -// format: `tableRead.landsWhole` says a read did, `table.wholeRow` is the node. +// - whole — a read of a table with no selector and no descent, {/city}, landing on +// the `table` itself: it draws a row apart from every pin, so the fences keep the rows +// its cells stand in as `drawAt.wholePins`, beside `drawAt.wholeTable`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those // compile. The draw fences are `heldCheck`, `drawCheck` and `checkFamilies`. diff --git a/doc_test.go b/doc_test.go index b5d228d..ce32fff 100644 --- a/doc_test.go +++ b/doc_test.go @@ -95,7 +95,7 @@ func vocabulary(t *testing.T, files []*ast.File) string { } // declaredSymbols is every name the package declares at the top level, a method and -// a struct field keyed under its type: pinSet.pin, tableRead.landsWhole. +// a struct field keyed under its type: pinSet.pin, tableRead.landsRow. func declaredSymbols(files []*ast.File) map[string]bool { names := declaredFuncs(files) for _, f := range files { diff --git a/drawfence.go b/drawfence.go index 5adad1d..f8c5dd7 100644 --- a/drawfence.go +++ b/drawfence.go @@ -301,7 +301,7 @@ func (w *drawWalk) check() error { if into.at.group != level.at.group || alternatives(level.at, into.at) { continue } - if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsWhole) { + if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { return overlapError(level.at.route, level.a.spelling, into) } } diff --git a/drawfence_test.go b/drawfence_test.go index 6d28bf7..bb6004d 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -129,7 +129,7 @@ func renderRoot(s *session, t *template) { var set holdSet sc := renderScope{set: &set} sc.hold().pins.pin(t.table, r) - render(s, t.table.wholeRow, sc) + render(s, t.table.pinnedRow, sc) } case t.cellOf != nil: sc.t, sc.row = t.cellOf, t.cellRow diff --git a/family.go b/family.go index 05ca01b..3b0c0fe 100644 --- a/family.go +++ b/family.go @@ -10,11 +10,11 @@ import ( // row pinned, and their unpinned ancestors — each selector's spelling, and whether // it lands on a row rendered whole. type tableRead struct { - headTable *table - pins pinSet - drawn map[*table]bool - sels []tableSel - landsWhole bool + headTable *table + pins pinSet + drawn map[*table]bool + sels []tableSel + landsRow bool } // tableSel is one selector on the way: the table it selects a row of, and the path @@ -44,7 +44,7 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { cur = d } } - _, tr.landsWhole = leaf.(*tableRow) + _, tr.landsRow = leaf.(*tableRow) return tr } diff --git a/hold.go b/hold.go index 10ec227..cee9335 100644 --- a/hold.go +++ b/hold.go @@ -61,7 +61,7 @@ type renderTrace struct { // readHold is the hold a held read keeps its draw in: for a reference that reads a path, // the render's hold for its group, so its draw spans the render; for a sibling, or a -// reference read whole, held. +// bare reference, held. func readHold(held *hold, sc renderScope, a arm) *hold { if isRef(a.key) && len(a.tail) > 0 { return sc.hold() diff --git a/node.go b/node.go index dff687b..62c791d 100644 --- a/node.go +++ b/node.go @@ -43,7 +43,7 @@ func (*nullItem) isNode() {} // that many times and joins the results with separator (default ""), each render // an independent pick. A format holding a reference compiles in `linkTemplateRefs`. type template struct { - // Filled by `compileString`, `compileTemplate` and `table.compileWhole`: + // Filled by `compileString`, `compileTemplate` and `table.compileRowFormat`: format string tokens []formatToken fields map[string]node diff --git a/path.go b/path.go index 68e9860..2853225 100644 --- a/path.go +++ b/path.go @@ -268,7 +268,7 @@ func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error case len(tail) == 0 && sel == "" && !descended: return walkPath(t, nil, w) case len(tail) == 0: - return walkPath(t.wholeRow, nil, w) + return walkPath(t.pinnedRow, nil, w) case child != nil: return walkTable(child, tail[1:], w, true) } diff --git a/table.go b/table.go index 5502e8d..d577ba0 100644 --- a/table.go +++ b/table.go @@ -20,8 +20,8 @@ type table struct { header []string col map[string]int fields map[string]node // column nodes, the format's fields - wholeRow *tableRow // the pinned row rendered by the format - cells []string // rows × columns, flat + pinnedRow *tableRow + cells []string // rows × columns, flat cellTemplates map[int]*template keyIndex int // keyIndex through parentIndex are column indexes, -1 where the option is absent nameIndex int @@ -103,7 +103,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab if err := t.checkCells(); err != nil { return nil, fmt.Errorf("%s: %w", o.rows, err) } - if err := t.compileWhole(o.format); err != nil { + if err := t.compileRowFormat(o.format); err != nil { return nil, err } return t, nil @@ -321,9 +321,9 @@ func (t *table) checkCells() error { return nil } -// compileWhole compiles the format a row renders whole through, over the columns -// as its fields. -func (t *table) compileWhole(format string) error { +// compileRowFormat compiles the format a row renders through, over the columns as its +// fields. +func (t *table) compileRowFormat(format string) error { toks, err := parseFormat(format) if err != nil { return err @@ -343,7 +343,7 @@ func (t *table) compileWhole(format string) error { return err } t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, table: t} - t.wholeRow = &tableRow{t} + t.pinnedRow = &tableRow{t} return t.formatTemplate.compileRefFree() } diff --git a/table_test.go b/table_test.go index 397c10a..c1376d6 100644 --- a/table_test.go +++ b/table_test.go @@ -599,7 +599,7 @@ func TestTableRowsAreAlternatives(t *testing.T) { "a cell reaching a table whose rows are alternatives": {"country.tsv": "alpha2\tname\tmoney\nFI\tFinland\t{/y.sym}\nSE\tSweden\t{/cur[SEK].sym}\n"}, "a cell agreeing with the row it selects": {"country.tsv": "alpha2\tname\tmoney\nFI\tFinland\t{/y[1].sym} {/cur[SEK].sym}\nSE\tSweden\t{/cur[SEK].sym}\n"}, "a selected row's cell beside the row it agrees with": {"z.json": `"{/country[SE].money} {/cur[SEK].sym}"`}, - "a selected row whole beside the row it agrees with": {"z.json": `"{/country[SE]} {/cur[SEK].sym}"`}, + "a selected row beside the row it agrees with": {"z.json": `"{/country[SE]} {/cur[SEK].sym}"`}, "a column under a selected ancestor": { "city.json": `{"format":"{name} {money}","rows":"city.tsv","key":"name","parent":"country"}`, "city.tsv": "name\tcountry\tmoney\nHelsinki\tFI\t{/cur[EUR].sym}\nMalmö\tSE\t{/cur[SEK].sym}\nLund\tSE\t{/cur[SEK].sym}\n", diff --git a/todo.md b/todo.md index bb5e0d4..f323474 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Give "whole" one meaning: `doc.go` defines it as a read landing on a `tableRow`, - while `drawAt.wholeTable`, `checkFamilyPair` and `checkOwnFamily`'s comment use it for - a bare read landing on a `*table`, the one that draws without pinning. - Give a table's current row one owner: `renderScope.t` and `row` hold it beside the `pinSet`, and `renderLeaf` reads a column's null back from a hold `readField` wrote. - Split `walkPath`'s four modes, so a caller does not select a behaviour by which of -- 2.52.0 From d71edd5ede91ada239d1fd50884b84cd4a742256 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 15:27:18 +0200 Subject: [PATCH 129/966] Say tableRead.landsRow lands on a tableRow --- family.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/family.go b/family.go index 3b0c0fe..3857e3a 100644 --- a/family.go +++ b/family.go @@ -8,7 +8,7 @@ import ( // tableRead is what a reference path reads of a table family: the table its head // names, the rows its selectors pin, the tables it draws — those it walks with no // row pinned, and their unpinned ancestors — each selector's spelling, and whether -// it lands on a row rendered whole. +// it lands on a `tableRow`. type tableRead struct { headTable *table pins pinSet -- 2.52.0 From 8312b44a435c4c1d5c5561f707fda5aae1f9963c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 15:32:44 +0200 Subject: [PATCH 130/966] Say a whole read draws the family apart, drop the decision the bare-reference one already states, and link the naming test to its decision --- AGENTS.md | 1 - README.md | 6 +++--- docs/decisions.md | 8 -------- family.go | 10 +++++----- naming_test.go | 1 + todo.md | 5 +++++ 6 files changed, 14 insertions(+), 17 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index cd7bb56..a5c182e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,7 +71,6 @@ In [docs/decisions.md](docs/decisions.md): - A selector is bracketed, and a dot inside it is literal - `parent` names the link column and the table alike - After a row, a path names a column or a linked table -- A table read into is pinned; a table read whole draws afresh - Every reference path into one family selects the same rows, per render and group - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields diff --git a/README.md b/README.md index 01f76cf..ae5d18a 100644 --- a/README.md +++ b/README.md @@ -546,9 +546,9 @@ table another path in the group selects a row of, `{/city.name}` beside also refuses a link cell that is no key of the parent, a parent row no child links to, a chain of parents that closes, a table named like a column of any table above it, and a cell or format of a table that references a table of its own family, through -any template, a `repeat` or a `drawGroup` included, since a row rendered whole would -draw the family apart from itself: read the family from a template beside it, or add -the value as a column. A cell's reads are the render's too; which pairs are weighed +any template, a `repeat` or a `drawGroup` included, since a whole read of the table +would draw the family apart from the row it renders: read the family from a template +beside it, or add the value as a column. A cell's reads are the render's too; which pairs are weighed against each other is under [Decisions](docs/decisions.md#the-rows-of-a-table-are-alternatives). ### Options and fields diff --git a/docs/decisions.md b/docs/decisions.md index 721f41f..dd0f0d4 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -359,14 +359,6 @@ folder or another column name would be asking for a second spelling. today; admitting it later is additive, since a refused spelling gains a meaning and no accepted one changes, so the door stays open for the address records the plan describes. -## A table read into is pinned; a table read whole draws afresh - -A path into a table pins its row for the render and group, as a reference path pins its -level, and a bare `{/city}` draws a row each time and pins no row of its family, as a -bare reference does; so a bare table beside a path into its family is refused like a -bare reference beside a path into it. A bare table reference still counts as a read for -a `drawGroup`, since the group is what draws it apart from the family's pins. - ## Every reference path into one family selects the same rows, per render and group A selector pins rows for the render, and a read that draws freely before it could pin a diff --git a/family.go b/family.go index 3857e3a..938e9d1 100644 --- a/family.go +++ b/family.go @@ -145,7 +145,7 @@ func (r *tableRead) replay(d *pinSet) error { // checkFamilyPair refuses a table read whole beside a path into its family, and a // table one read draws that the other pins, since which token renders first would // then decide the row. -// docs/decisions.md#a-table-read-into-is-pinned-a-table-read-whole-draws-afresh +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held func checkFamilyPair(a, b pathRead) error { for _, pair := range [][2]pathRead{{a, b}, {b, a}} { x, y := pair[0], pair[1] @@ -180,9 +180,9 @@ func (r *tableRead) drawnOf(pins *pinSet) *table { } // checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a row -// rendered whole draws its row without pinning it, so the family would draw apart -// from the row being rendered, whichever draws the reaching template holds. +// away and through a repeat or a draw group too, a table of its own family: a whole +// read draws its row without pinning it, so the family would draw apart from the row +// it renders, whichever draws the reaching template holds. // docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route func checkOwnFamily(t *template) error { own := t.table @@ -212,7 +212,7 @@ func checkOwnFamily(t *template) error { return renderEdge{}, nil, false } if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a row of %s rendered whole would draw apart from; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + return fmt.Errorf("%s reads %s, a table of its own family, which a whole read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) } return nil } diff --git a/naming_test.go b/naming_test.go index 53cc589..f73e58b 100644 --- a/naming_test.go +++ b/naming_test.go @@ -8,6 +8,7 @@ import ( "testing" ) +// docs/decisions.md#one-name-one-meaning func TestNoFunctionSpellsAMethod(t *testing.T) { funcs, methods := declaredCalls(namespaceFiles(t)) names := make([]string, 0, len(funcs)) diff --git a/todo.md b/todo.md index f323474..5a91cc3 100644 --- a/todo.md +++ b/todo.md @@ -47,6 +47,11 @@ This round, in order: where `NewTemplate` and `NewRecordTemplate` do not, and `List` claims to be every path `Fake` accepts, which a selector is too. Delete the dead `table.descends` and the redundant `string(key)` in `checkNoRepeatedItem`. +- Keep only the standing choice in the decision "The rows of a table are alternatives", + a choice's items refused as alternatives, and give it the goal it serves: the rest + defines `alternatives`, which its comment in `family.go` already does, and the README's + "which pairs are weighed against each other" sends a reader to the log for that + definition. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 9babea96f65f9a85bdc789ea95dba478c1ff1871 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 15:35:11 +0200 Subject: [PATCH 131/966] Name the bare read in the own-family error and its README sentence, the word the README defines --- README.md | 6 +++--- family.go | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index ae5d18a..5d9556b 100644 --- a/README.md +++ b/README.md @@ -546,9 +546,9 @@ table another path in the group selects a row of, `{/city.name}` beside also refuses a link cell that is no key of the parent, a parent row no child links to, a chain of parents that closes, a table named like a column of any table above it, and a cell or format of a table that references a table of its own family, through -any template, a `repeat` or a `drawGroup` included, since a whole read of the table -would draw the family apart from the row it renders: read the family from a template -beside it, or add the value as a column. A cell's reads are the render's too; which pairs are weighed +any template, a `repeat` or a `drawGroup` included, since a bare read of the referencing +table draws its row without pinning it, so the family would draw apart from that row: +read the family from a template beside it, or add the value as a column. A cell's reads are the render's too; which pairs are weighed against each other is under [Decisions](docs/decisions.md#the-rows-of-a-table-are-alternatives). ### Options and fields diff --git a/family.go b/family.go index 938e9d1..22477a7 100644 --- a/family.go +++ b/family.go @@ -212,7 +212,7 @@ func checkOwnFamily(t *template) error { return renderEdge{}, nil, false } if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a whole read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) } return nil } -- 2.52.0 From 538114a92684d81417e1bac1a77919e66edff3d4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 26 Sep 2026 15:40:03 +0200 Subject: [PATCH 132/966] Rewrap the own-family paragraph --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5d9556b..28c288d 100644 --- a/README.md +++ b/README.md @@ -548,8 +548,9 @@ to, a chain of parents that closes, a table named like a column of any table abo it, and a cell or format of a table that references a table of its own family, through any template, a `repeat` or a `drawGroup` included, since a bare read of the referencing table draws its row without pinning it, so the family would draw apart from that row: -read the family from a template beside it, or add the value as a column. A cell's reads are the render's too; which pairs are weighed -against each other is under [Decisions](docs/decisions.md#the-rows-of-a-table-are-alternatives). +read the family from a template beside it, or add the value as a column. A cell's +reads are the render's too; which pairs are weighed against each other is under +[Decisions](docs/decisions.md#the-rows-of-a-table-are-alternatives). ### Options and fields -- 2.52.0 From fb7a30e505730860155d35687f5a4275be9446f4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 15:00:16 +0200 Subject: [PATCH 133/966] Make goal 4 ask for intuitive templates, and decide a selected row is fixed, not drawn --- AGENTS.md | 2 +- README.md | 9 ++++++--- docs/decisions.md | 25 ++++++++++++++++--------- family.go | 1 - todo.md | 24 ++++++++++++++++-------- 5 files changed, 39 insertions(+), 22 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a5c182e..57c2f88 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,7 +71,7 @@ In [docs/decisions.md](docs/decisions.md): - A selector is bracketed, and a dot inside it is literal - `parent` names the link column and the table alike - After a row, a path names a column or a linked table -- Every reference path into one family selects the same rows, per render and group +- A selected row is fixed, not drawn, and an unnamed read beside it is refused - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields - A table is a record of string columns diff --git a/README.md b/README.md index 28c288d..cdf4b8d 100644 --- a/README.md +++ b/README.md @@ -823,9 +823,12 @@ the Development section below, and who ships a register the four above then draw 3. **Valid by construction** — every value passes the check its real consumer applies; facts that belong together come from one draw, within a value and across categories. -4. **Text means what it says** — a format renders as written; only `{…}` varies, - random characters included (`{digits(3)}`). One spelling per result; the wrong - one is a load error naming the right one. +4. **Templates are intuitive, and text means what it says** — a template does what + its writer expects on first reading. A format renders as written; only `{…}` + varies, random characters included (`{digits(3)}`), and a selected row, `[SE]`, + is that row wherever it is written. Where a reader could expect two outputs, the + template is a load error naming a spelling for each. One spelling per result; the + wrong one is a load error naming the right one. 5. **Every mistake is a load error** — `New` rejects the data and `NewTemplate` the inline template; on a loaded generator `Fake` fails only for an unknown path, `FakeStruct` only for a non-struct argument or a type its tags do not diff --git a/docs/decisions.md b/docs/decisions.md index dd0f0d4..79f24fa 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -359,14 +359,22 @@ folder or another column name would be asking for a second spelling. today; admitting it later is additive, since a refused spelling gains a meaning and no accepted one changes, so the door stays open for the address records the plan describes. -## Every reference path into one family selects the same rows, per render and group +## A selected row is fixed, not drawn, and an unnamed read beside it is refused -A selector pins rows for the render, and a read that draws freely before it could pin a -row the selector contradicts, so accepting both would make the result depend on which -token rendered first. Requiring one selection per family per group is checkable at load -with no data lookup beyond the selectors themselves, and the error names the rewrite. -Two selectors naming one row by key and by name compare equal, since the load check -resolves them. +Decided 2026-09-27 by the maintainer, for goal 4; valid while a selector names exactly one row. + +- Selected rows never conflict: `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` + renders `Stockholm / Helsinki`, and so may two items of one choice. +- What a path draws beneath a selected row is one draw per selection, shared by every + path through it: `{/geo.SE.region[12].locality.name}` twice names one locality. +- A read naming no row of a table whose family a selection in the render and group + names is refused, naming the selected path and a `drawGroup`: beside + `{/misc.territory[SE].capital}`, a writer reading `{/misc.territory.currency}` expects + `SEK`, and `Stockholm pays in EUR` would pass review. It stays refused beside two + selections, since adding `[FI]` to that template would otherwise make it load. + +Checked against three simulated template writers from the Audience, who agreed on each +case but the last, where two expected a render and all three accepted the refusal. ## A parent row with no child row is a load error @@ -409,8 +417,7 @@ row, and whatever they reach, do meet, and so does the format beside them. Two r meet unless they sit under different rows of one pinned table; pinning a row pins its ancestors too, so rows under different parents are alternatives as well. A table read whole draws its row apart from every pin, so the rows of one whole read are -alternatives only to each other. A choice's items are not alternatives: two items -selecting different rows of one family are refused. +alternatives only to each other. ## A table never reaches its own family, by any route diff --git a/family.go b/family.go index 22477a7..e870057 100644 --- a/family.go +++ b/family.go @@ -84,7 +84,6 @@ func (r *tableRead) selected(t *table) (tableSel, bool) { // read one consistent draw: a table rendered whole beside a path into the family, // a table one read draws that another pins, and two reads pinning different rows. // The reads come sorted by group and path, so which pair is reported does not vary. -// docs/decisions.md#every-reference-path-into-one-family-selects-the-same-rows-per-render-and-group func checkFamilies(reads []pathRead) error { for i, r := range reads { if r.tr == nil { diff --git a/todo.md b/todo.md index 5a91cc3..9f8553b 100644 --- a/todo.md +++ b/todo.md @@ -47,11 +47,9 @@ This round, in order: where `NewTemplate` and `NewRecordTemplate` do not, and `List` claims to be every path `Fake` accepts, which a selector is too. Delete the dead `table.descends` and the redundant `string(key)` in `checkNoRepeatedItem`. -- Keep only the standing choice in the decision "The rows of a table are alternatives", - a choice's items refused as alternatives, and give it the goal it serves: the rest - defines `alternatives`, which its comment in `family.go` already does, and the README's - "which pairs are weighed against each other" sends a reader to the log for that - definition. +- Delete the decision "The rows of a table are alternatives": it defines `alternatives`, + which its comment in `family.go` already does, and the README's "which pairs are + weighed against each other" sends a reader to the log for that definition. - Run the nine-seat panel again and file what it names here. ### Data @@ -59,6 +57,8 @@ This round, in order: Shape: T = table, t = template, c = choice. Read every factual list from a register by a `data-import/` script, as README goal 12 asks. +- Strip the whitespace `data-import/territory.py` copies from its register: + `misc.territory[CW].capital` renders ` Willemstad`, with a leading space. - Place xlsx cells by their `r` reference in `data-import/xlsx.py`: a sheet omitting an empty cell shifts every later column left. - Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone @@ -171,6 +171,13 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Render selected rows as the decision "A selected row is fixed, not drawn, and an + unnamed read beside it is refused" states: `New` refuses + `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice + whose items select different rows. Have the refusal of an unnamed read name a + selected path that loads: beside `{/geo.SE.locality[Lund].name}`, + `{/geo.SE.municipality.name}` names none today, and the one that holds selects from + the top, `{/geo.SE.municipality[Lund].name}`. - Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the first in Go's map order. @@ -192,9 +199,6 @@ by a `data-import/` script, as README goal 12 asks. error names the category and the two columns but no row, so the author greps a register-sized file for the cell — and "name the fields you want instead" rewrites a cell that is not the one to fix. -- Have the family fence offer only a rewrite that works: where the drawn table's rows select - more than one row of the family, every `write {/terr[NO].name}` it offers fails the same - way, so only the `drawGroup` it also names is left. - Spell a table one way across the family errors: `checkFamilyPair` names `drawn.category`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two @@ -214,6 +218,10 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle +- Decide whether a path steps from a row up to its parent, so + `{/geo.SE.locality[Lund].municipality.name}` reads Lund's municipality: three simulated + template writers all expected it to, per goal 4, and today it fails with "municipality + is a column, and a cell holds no fields". Adding it only accepts a refused spelling. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 98b25177e72edc7eef07c818970d79ccb7bec441 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 15:02:46 +0200 Subject: [PATCH 134/966] Name goal 4's check and the rewrite the refusal offers, scope the render-draw decision to unselected paths, and state the step-up clash --- README.md | 13 +++++++------ docs/decisions.md | 25 +++++++++++++++---------- todo.md | 25 +++++++++++++++---------- 3 files changed, 37 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index cdf4b8d..a1d80a6 100644 --- a/README.md +++ b/README.md @@ -823,12 +823,13 @@ the Development section below, and who ships a register the four above then draw 3. **Valid by construction** — every value passes the check its real consumer applies; facts that belong together come from one draw, within a value and across categories. -4. **Templates are intuitive, and text means what it says** — a template does what - its writer expects on first reading. A format renders as written; only `{…}` - varies, random characters included (`{digits(3)}`), and a selected row, `[SE]`, - is that row wherever it is written. Where a reader could expect two outputs, the - template is a load error naming a spelling for each. One spelling per result; the - wrong one is a load error naming the right one. +4. **Templates are intuitive, and text means what it says** — a template renders what + every writer reading it would expect, judged by simulated writers from the Audience. + A format renders as written; only `{…}` varies, random characters included + (`{digits(3)}`), and a selected row, `[SE]`, is that row wherever it is written. + Where two writers could expect different outputs, the template is a load error + naming a spelling for each. One spelling per result; the wrong one is a load error + naming the right one. 5. **Every mistake is a load error** — `New` rejects the data and `NewTemplate` the inline template; on a loaded generator `Fake` fails only for an unknown path, `FakeStruct` only for a non-struct argument or a type its tags do not diff --git a/docs/decisions.md b/docs/decisions.md index 79f24fa..b67e5dd 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -217,7 +217,8 @@ handle is needed, as the cache already compiles once. ## A render shares one reference draw per category, per group -Every reference path into a category in one `Fake`, or one record, reads one draw of it, +Every reference path into a category that selects no row, in one `Fake` or one record, +reads one draw of it, so a value's facts agree across its fields, nested templates and columns alike — `{/currency.code}` in one field and `{/currency.symbol}` in another name one currency, whichever view renders them. A `repeat` iteration is a render of its own, since @@ -361,17 +362,21 @@ accepted one changes, so the door stays open for the address records the plan de ## A selected row is fixed, not drawn, and an unnamed read beside it is refused -Decided 2026-09-27 by the maintainer, for goal 4; valid while a selector names exactly one row. +Decided 2026-09-27 by the maintainer, for goals 3 and 4; valid while a selector names +exactly one row. - Selected rows never conflict: `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` - renders `Stockholm / Helsinki`, and so may two items of one choice. -- What a path draws beneath a selected row is one draw per selection, shared by every - path through it: `{/geo.SE.region[12].locality.name}` twice names one locality. -- A read naming no row of a table whose family a selection in the render and group - names is refused, naming the selected path and a `drawGroup`: beside - `{/misc.territory[SE].capital}`, a writer reading `{/misc.territory.currency}` expects - `SEK`, and `Stockholm pays in EUR` would pass review. It stays refused beside two - selections, since adding `[FI]` to that template would otherwise make it load. + renders `Stockholm / Helsinki`, and two items of one choice may select different rows. +- What a path draws beneath a selected row is one draw per selection, per render and + group, shared by every path through it: `{/geo.SE.municipality[1281].locality.name}` + twice names one locality. `[1281]` and `[Lund]` select one row, so they are one + selection. +- A read whose path selects no row is refused beside a selection in its family, in the + same render and group, naming the rewrite through each selection and a `drawGroup`: + beside `{/misc.territory[SE].capital}`, a writer reading `{/misc.territory.currency}` + expects `SEK`, so the error names `{/misc.territory[SE].currency}`, since + `Stockholm pays in EUR` would pass review. A second selection does not make the + unnamed read load. Checked against three simulated template writers from the Audience, who agreed on each case but the last, where two expected a render and all three accepted the refusal. diff --git a/todo.md b/todo.md index 9f8553b..0e2b51d 100644 --- a/todo.md +++ b/todo.md @@ -48,8 +48,9 @@ This round, in order: path `Fake` accepts, which a selector is too. Delete the dead `table.descends` and the redundant `string(key)` in `checkNoRepeatedItem`. - Delete the decision "The rows of a table are alternatives": it defines `alternatives`, - which its comment in `family.go` already does, and the README's "which pairs are - weighed against each other" sends a reader to the log for that definition. + which its comment in `family.go` already does. Point the README's "which pairs are + weighed against each other" at that comment's rule, stated in the README's Table + section, where it sends a reader to the log for the definition today. - Run the nine-seat panel again and file what it names here. ### Data @@ -174,10 +175,11 @@ by a `data-import/` script, as README goal 12 asks. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice - whose items select different rows. Have the refusal of an unnamed read name a - selected path that loads: beside `{/geo.SE.locality[Lund].name}`, - `{/geo.SE.municipality.name}` names none today, and the one that holds selects from - the top, `{/geo.SE.municipality[Lund].name}`. + whose items select different rows. Have the refusal of an unnamed read name the + rewrite through each selection, selecting an ancestor of the selected row by key: + beside `{/geo.SE.locality[Lund].name}`, `{/geo.SE.municipality.name}` names none + today, where `{/geo.SE.municipality[1281].name}` holds. Scope the README's "A reference + path into a category is held" to a path that selects no row. - Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the first in Go's map order. @@ -218,10 +220,13 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide whether a path steps from a row up to its parent, so - `{/geo.SE.locality[Lund].municipality.name}` reads Lund's municipality: three simulated - template writers all expected it to, per goal 4, and today it fails with "municipality - is a column, and a cell holds no fields". Adding it only accepts a refused spelling. +- Decide how a path steps from a row up to its parent row: three simulated template + writers expected `{/geo.SE.locality[Lund].municipality.name}` to read Lund's + municipality, which goal 4 asks for, but `.municipality` after a row is already the + link column's cell, `city[Oslo].country` rendering `NO`. So either the name means the + cell where it ends a path and the parent row where a segment follows, which the + decisions "One name, one meaning" and "`parent` names the link column and the table + alike" have to allow, or the step up takes a spelling of its own. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 430bdbcaae1874f0ae694a4b8ae5e6b4f76a9484 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 15:03:27 +0200 Subject: [PATCH 135/966] Rewrap the render-draw decision --- docs/decisions.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index b67e5dd..db319fb 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -218,10 +218,9 @@ handle is needed, as the cache already compiles once. ## A render shares one reference draw per category, per group Every reference path into a category that selects no row, in one `Fake` or one record, -reads one draw of it, -so a value's facts agree across its fields, nested templates and columns alike — -`{/currency.code}` in one field and `{/currency.symbol}` in another name one currency, -whichever view renders them. A `repeat` iteration is a render of its own, since +reads one draw of it, so a value's facts agree across its fields, nested templates and +columns alike — `{/currency.code}` in one field and `{/currency.symbol}` in another name +one currency, whichever view renders them. A `repeat` iteration is a render of its own, since repeating asks for another entity, and a [draw group](../README.md#draw-group) names further entities within one render, so a payer and a payee are two groups over one `person` rather than two copies of it. Only references share: a sibling field is local to its own -- 2.52.0 From e6a1a7aa8b28c94e953b2f46bb457dbba37d2423 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 15:04:13 +0200 Subject: [PATCH 136/966] State the step-up question's options, their version cost and the spelling each leaves --- todo.md | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/todo.md b/todo.md index 0e2b51d..b84abf9 100644 --- a/todo.md +++ b/todo.md @@ -220,13 +220,24 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide how a path steps from a row up to its parent row: three simulated template - writers expected `{/geo.SE.locality[Lund].municipality.name}` to read Lund's - municipality, which goal 4 asks for, but `.municipality` after a row is already the - link column's cell, `city[Oslo].country` rendering `NO`. So either the name means the - cell where it ends a path and the parent row where a segment follows, which the - decisions "One name, one meaning" and "`parent` names the link column and the table - alike" have to allow, or the step up takes a spelling of its own. +- Decide how a path steps from a row up to its parent row, one level per step, chained. + Three simulated template writers expected `{/geo.SE.locality[Lund].municipality.name}` + to read Lund's municipality; today it is a load error, since `.municipality` after a + row is the link column's cell, `city[Oslo].country` rendering `NO`. The options: + - The name means the parent row where a segment follows, and the cell where the path + ends. It gives a refused spelling a meaning, so a minor, and extends "After a row, a + path names a column or a linked table" from child tables to the parent. + - The name always means the parent row, rendering through its format where the path + ends: `city[Oslo].country` renders `Norway`, the cell stays reachable through the + parent's key column. A change + to what exists, so a major, and a record's `country` column then holds the key while + the path renders the name. + - Goal 4 refuses a path ending on the name, since a writer could expect the code or the + name, naming the parent's key column and `.name`. A major. + + Either of the first two gives `locality[Lund].municipality.name` and + `municipality[1281].name` one result, so goal 4's one spelling per result refuses one + of them, and the rewrite the selected-row item above names follows that choice. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 543b21d99f1d0f955ceb8ae06bd270db433296dc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 15:06:34 +0200 Subject: [PATCH 137/966] Say every step-up option leaves two spellings of one result --- todo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/todo.md b/todo.md index b84abf9..e292442 100644 --- a/todo.md +++ b/todo.md @@ -235,7 +235,7 @@ by a `data-import/` script, as README goal 12 asks. - Goal 4 refuses a path ending on the name, since a writer could expect the code or the name, naming the parent's key column and `.name`. A major. - Either of the first two gives `locality[Lund].municipality.name` and + Each option gives `locality[Lund].municipality.name` and `municipality[1281].name` one result, so goal 4's one spelling per result refuses one of them, and the rewrite the selected-row item above names follows that choice. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the -- 2.52.0 From f8e9d9a2809c1d2dfd29c1efd0af57c00a0ec4db Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:41:31 +0200 Subject: [PATCH 138/966] Answer the step-up question at goal level: a path renders what every writer expects, so a link column steps up to its parent and may not end a path --- AGENTS.md | 1 + README.md | 5 +++-- docs/decisions.md | 18 ++++++++++++++++++ todo.md | 31 +++++++++---------------------- 4 files changed, 31 insertions(+), 24 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 57c2f88..a33d167 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,6 +72,7 @@ In [docs/decisions.md](docs/decisions.md): - `parent` names the link column and the table alike - After a row, a path names a column or a linked table - A selected row is fixed, not drawn, and an unnamed read beside it is refused +- A link column after a row steps up to the parent row, and a path may not end on it - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields - A table is a record of string columns diff --git a/README.md b/README.md index a1d80a6..55ac723 100644 --- a/README.md +++ b/README.md @@ -823,8 +823,9 @@ the Development section below, and who ships a register the four above then draw 3. **Valid by construction** — every value passes the check its real consumer applies; facts that belong together come from one draw, within a value and across categories. -4. **Templates are intuitive, and text means what it says** — a template renders what - every writer reading it would expect, judged by simulated writers from the Audience. +4. **Templates are intuitive, and text means what it says** — a template, and every + path in it, renders what every writer reading it would expect, judged by simulated + writers from the Audience. A format renders as written; only `{…}` varies, random characters included (`{digits(3)}`), and a selected row, `[SE]`, is that row wherever it is written. Where two writers could expect different outputs, the template is a load error diff --git a/docs/decisions.md b/docs/decisions.md index db319fb..fe68bb7 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -380,6 +380,24 @@ exactly one row. Checked against three simulated template writers from the Audience, who agreed on each case but the last, where two expected a render and all three accepted the refusal. +## A link column after a row steps up to the parent row, and a path may not end on it + +Decided 2026-09-27 by the maintainer, for goal 4; valid while a link column holds the +parent's key. + +- Where a segment follows, the link column's name reads the parent row, one level per + step, chained: `city[Oslo].country.name` renders `Norway`, and + `geo.SE.locality[Lund].municipality.region.name` Lund's region. +- A path ending on it is refused, naming the parent's key column and the columns its + format reads: `city[Oslo].country` could be `NO` or `Norway`, so the error names + `city[Oslo].country.alpha2` and `city[Oslo].country.name`. +- A route through another row is not a second spelling of a selection: + `city[Oslo].country.name` and `country[NO].name` both load, as `[1281]` and `[Lund]` do. + +Checked against three simulated template writers from the Audience: all three read the +parent row where a segment followed, and they split between the code and the name where +the path ended. + ## A parent row with no child row is a load error A descendant is drawn inside the nearest pinned ancestor, so every ancestor row must diff --git a/todo.md b/todo.md index e292442..4e9cc85 100644 --- a/todo.md +++ b/todo.md @@ -172,14 +172,19 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Step up from a row to its parent as the decision "A link column after a row steps up + to the parent row, and a path may not end on it" states: `city[Oslo].country.name` is + a load error today, and `city[Oslo].country` renders the cell `NO`. Correct the + README's `city[Oslo].country` example to match. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice whose items select different rows. Have the refusal of an unnamed read name the - rewrite through each selection, selecting an ancestor of the selected row by key: - beside `{/geo.SE.locality[Lund].name}`, `{/geo.SE.municipality.name}` names none - today, where `{/geo.SE.municipality[1281].name}` holds. Scope the README's "A reference - path into a category is held" to a path that selects no row. + rewrite through each selection: beside `{/geo.SE.locality[Lund].name}`, + `{/geo.SE.municipality.name}` names none today, where + `{/geo.SE.locality[Lund].municipality.name}` holds once the step up above lands. + Scope the README's "A reference path into a category is held" to a path that + selects no row. - Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the first in Go's map order. @@ -220,24 +225,6 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide how a path steps from a row up to its parent row, one level per step, chained. - Three simulated template writers expected `{/geo.SE.locality[Lund].municipality.name}` - to read Lund's municipality; today it is a load error, since `.municipality` after a - row is the link column's cell, `city[Oslo].country` rendering `NO`. The options: - - The name means the parent row where a segment follows, and the cell where the path - ends. It gives a refused spelling a meaning, so a minor, and extends "After a row, a - path names a column or a linked table" from child tables to the parent. - - The name always means the parent row, rendering through its format where the path - ends: `city[Oslo].country` renders `Norway`, the cell stays reachable through the - parent's key column. A change - to what exists, so a major, and a record's `country` column then holds the key while - the path renders the name. - - Goal 4 refuses a path ending on the name, since a writer could expect the code or the - name, naming the parent's key column and `.name`. A major. - - Each option gives `locality[Lund].municipality.name` and - `municipality[1281].name` one result, so goal 4's one spelling per result refuses one - of them, and the rewrite the selected-row item above names follows that choice. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 4ec20e6e3aaa358466d05594cb28e84a303315a0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:44:41 +0200 Subject: [PATCH 139/966] Scope the step up to drawn rows too, name the format rewrite and what Fake, List and records do, and leave the second-spelling question open --- README.md | 11 +++++------ docs/decisions.md | 20 ++++++++++++-------- todo.md | 10 ++++++++-- 3 files changed, 25 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 55ac723..b021c71 100644 --- a/README.md +++ b/README.md @@ -823,17 +823,16 @@ the Development section below, and who ships a register the four above then draw 3. **Valid by construction** — every value passes the check its real consumer applies; facts that belong together come from one draw, within a value and across categories. -4. **Templates are intuitive, and text means what it says** — a template, and every - path in it, renders what every writer reading it would expect, judged by simulated - writers from the Audience. - A format renders as written; only `{…}` varies, random characters included +4. **Templates are intuitive, and text means what it says** — a template or a path, + and every path in a template, renders what every writer reading it would expect, + judged by simulated writers from the Audience. A format renders as written; only `{…}` varies, random characters included (`{digits(3)}`), and a selected row, `[SE]`, is that row wherever it is written. Where two writers could expect different outputs, the template is a load error naming a spelling for each. One spelling per result; the wrong one is a load error naming the right one. 5. **Every mistake is a load error** — `New` rejects the data and `NewTemplate` - the inline template; on a loaded generator `Fake` fails only for an unknown - path, `FakeStruct` only for a non-struct argument or a type its tags do not + the inline template; on a loaded generator `Fake` fails only for a path that + names nothing or could name two things, `FakeStruct` only for a non-struct argument or a type its tags do not describe, with the same error every call, and `Template.Fake` cannot fail at all. 6. **Zero to a value in one command** — `go install`, then `fejkdata sv_SE.person`: no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position; diff --git a/docs/decisions.md b/docs/decisions.md index fe68bb7..525d82b 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -383,16 +383,20 @@ case but the last, where two expected a render and all three accepted the refusa ## A link column after a row steps up to the parent row, and a path may not end on it Decided 2026-09-27 by the maintainer, for goal 4; valid while a link column holds the -parent's key. +parent's key. It extends [After a row, a path names a column or a linked +table](#after-a-row-a-path-names-a-column-or-a-linked-table) from child tables to the +parent. A row is selected or drawn alike. - Where a segment follows, the link column's name reads the parent row, one level per - step, chained: `city[Oslo].country.name` renders `Norway`, and - `geo.SE.locality[Lund].municipality.region.name` Lund's region. -- A path ending on it is refused, naming the parent's key column and the columns its - format reads: `city[Oslo].country` could be `NO` or `Norway`, so the error names - `city[Oslo].country.alpha2` and `city[Oslo].country.name`. -- A route through another row is not a second spelling of a selection: - `city[Oslo].country.name` and `country[NO].name` both load, as `[1281]` and `[Lund]` do. + step, chained: `city[Oslo].country.name` renders `Norway`, `locality.municipality.name` + the drawn locality's municipality, and `geo.SE.locality[Lund].municipality.region.name` + Lund's region. The parent row of a selected row is fixed as the selection is, so + `city[Oslo].country.city.name` draws a city of Norway, which may not be Oslo. +- A path ending on it is refused at `New`, `NewTemplate` and `Fake` alike, naming the + parent's key column and the parent's format spelled as paths: with a `country` format + of `{name} ({alpha2})`, `city[Oslo].country` names `city[Oslo].country.alpha2` and + `{/city[Oslo].country.name} ({/city[Oslo].country.alpha2})`. `List` leaves such a path + out, and a record's link column still holds the key, since it is a column, not a path. Checked against three simulated template writers from the Audience: all three read the parent row where a segment followed, and they split between the code and the name where diff --git a/todo.md b/todo.md index 4e9cc85..4894ab2 100644 --- a/todo.md +++ b/todo.md @@ -174,8 +174,10 @@ by a `data-import/` script, as README goal 12 asks. - Step up from a row to its parent as the decision "A link column after a row steps up to the parent row, and a path may not end on it" states: `city[Oslo].country.name` is - a load error today, and `city[Oslo].country` renders the cell `NO`. Correct the - README's `city[Oslo].country` example to match. + a load error today, and `city[Oslo].country` renders the cell `NO`, as + `locality.municipality` does, which `List` advertises. Replace the README's + `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, + rendering `Norway`. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice @@ -225,6 +227,10 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle +- Decide whether a route through another row is a second spelling of one result: + `city[Oslo].country.name` and `country[NO].name` always render `Norway`, and goal 4 + refuses a second spelling, while `country[NO]` and `country[Norway]` both select one + row today. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 55eacf03193a08f277f6b35c7c4cd602f4a7bee0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:45:50 +0200 Subject: [PATCH 140/966] Leave a path stepping up and back down an open question, and say List stops advertising a path ending on a link column --- docs/decisions.md | 3 +-- todo.md | 4 +++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 525d82b..1e4f9f3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -390,8 +390,7 @@ parent. A row is selected or drawn alike. - Where a segment follows, the link column's name reads the parent row, one level per step, chained: `city[Oslo].country.name` renders `Norway`, `locality.municipality.name` the drawn locality's municipality, and `geo.SE.locality[Lund].municipality.region.name` - Lund's region. The parent row of a selected row is fixed as the selection is, so - `city[Oslo].country.city.name` draws a city of Norway, which may not be Oslo. + Lund's region. - A path ending on it is refused at `New`, `NewTemplate` and `Fake` alike, naming the parent's key column and the parent's format spelled as paths: with a `country` format of `{name} ({alpha2})`, `city[Oslo].country` names `city[Oslo].country.alpha2` and diff --git a/todo.md b/todo.md index 4894ab2..1e7397f 100644 --- a/todo.md +++ b/todo.md @@ -175,7 +175,7 @@ by a `data-import/` script, as README goal 12 asks. - Step up from a row to its parent as the decision "A link column after a row steps up to the parent row, and a path may not end on it" states: `city[Oslo].country.name` is a load error today, and `city[Oslo].country` renders the cell `NO`, as - `locality.municipality` does, which `List` advertises. Replace the README's + `locality.municipality` does, which `List` stops advertising. Replace the README's `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. - Render selected rows as the decision "A selected row is fixed, not drawn, and an @@ -227,6 +227,8 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle +- Decide what a path stepping up and back down renders, per goal 4's writer check: + `city[Oslo].country.city.name` could be Oslo or a city of Norway drawn afresh. - Decide whether a route through another row is a second spelling of one result: `city[Oslo].country.name` and `country[NO].name` always render `Norway`, and goal 4 refuses a second spelling, while `country[NO]` and `country[Norway]` both select one -- 2.52.0 From 003ce9deb4ccfcc5a16a35621a06e396d1a3abdb Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:55:51 +0200 Subject: [PATCH 141/966] Decide a selector names a row by its key alone, a route through another row is another path, and a step up may not step back down --- AGENTS.md | 1 + docs/decisions.md | 15 +++++++++++++-- todo.md | 14 +++++++------- 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a33d167..c519a01 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,6 +73,7 @@ In [docs/decisions.md](docs/decisions.md): - After a row, a path names a column or a linked table - A selected row is fixed, not drawn, and an unnamed read beside it is refused - A link column after a row steps up to the parent row, and a path may not end on it +- A selector names a row by its key alone - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields - A table is a record of string columns diff --git a/docs/decisions.md b/docs/decisions.md index 1e4f9f3..ff220c4 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -368,8 +368,7 @@ exactly one row. renders `Stockholm / Helsinki`, and two items of one choice may select different rows. - What a path draws beneath a selected row is one draw per selection, per render and group, shared by every path through it: `{/geo.SE.municipality[1281].locality.name}` - twice names one locality. `[1281]` and `[Lund]` select one row, so they are one - selection. + twice names one locality. - A read whose path selects no row is refused beside a selection in its family, in the same render and group, naming the rewrite through each selection and a `drawGroup`: beside `{/misc.territory[SE].capital}`, a writer reading `{/misc.territory.currency}` @@ -396,11 +395,23 @@ parent. A row is selected or drawn alike. of `{name} ({alpha2})`, `city[Oslo].country` names `city[Oslo].country.alpha2` and `{/city[Oslo].country.name} ({/city[Oslo].country.alpha2})`. `List` leaves such a path out, and a record's link column still holds the key, since it is a column, not a path. +- A path may not step back down into a child table after a step up: + `city[Oslo].country.city.name` could be Oslo or a city of Norway drawn afresh, so it is + refused, naming `city[Oslo].name` and `country[NO].city.name`. +- A route through another row is another path to the same data, not a second spelling: + `city[Oslo].country.name` and `country[NO].name` both load. Checked against three simulated template writers from the Audience: all three read the parent row where a segment followed, and they split between the code and the name where the path ended. +## A selector names a row by its key alone + +Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result. A table with +a `key` selects by it alone: `country[NO]` loads, and `country[Norway]` is refused +naming `[NO]`, since a name could spell another row's key. A table with no `key` +selects by its `name` inside its parent, which is the key it has. + ## A parent row with no child row is a load error A descendant is drawn inside the nearest pinned ancestor, so every ancestor row must diff --git a/todo.md b/todo.md index 1e7397f..1545961 100644 --- a/todo.md +++ b/todo.md @@ -174,10 +174,16 @@ by a `data-import/` script, as README goal 12 asks. - Step up from a row to its parent as the decision "A link column after a row steps up to the parent row, and a path may not end on it" states: `city[Oslo].country.name` is - a load error today, and `city[Oslo].country` renders the cell `NO`, as + a load error today, `city[Oslo].country.city.name` draws a city, and + `city[Oslo].country` renders the cell `NO`, as `locality.municipality` does, which `List` stops advertising. Replace the README's `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. +- Select a row by its key alone, as the decision "A selector names a row by its key + alone" states: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row + today, where the error should name `[1281]` and `[SE]`. Refuse a `name` option beside + `key`, which then selects nothing, drop it from the shipped tables, and state the + README's selectors and `CHANGELOG.md`'s "a path selects a row by key or name" to match. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice @@ -227,12 +233,6 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide what a path stepping up and back down renders, per goal 4's writer check: - `city[Oslo].country.city.name` could be Oslo or a city of Norway drawn afresh. -- Decide whether a route through another row is a second spelling of one result: - `city[Oslo].country.name` and `country[NO].name` always render `Norway`, and goal 4 - refuses a second spelling, while `country[NO]` and `country[Norway]` both select one - row today. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From ae487142ba97cbad1037294c8ff53e81718aaee8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:56:01 +0200 Subject: [PATCH 142/966] Drop the false claim that a step back down draws today --- todo.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/todo.md b/todo.md index 1545961..6210e3f 100644 --- a/todo.md +++ b/todo.md @@ -173,10 +173,10 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI - Step up from a row to its parent as the decision "A link column after a row steps up - to the parent row, and a path may not end on it" states: `city[Oslo].country.name` is - a load error today, `city[Oslo].country.city.name` draws a city, and - `city[Oslo].country` renders the cell `NO`, as - `locality.municipality` does, which `List` stops advertising. Replace the README's + to the parent row, and a path may not end on it" states, refusing a step back down: + `city[Oslo].country.name` is a load error today, and `city[Oslo].country` renders the + cell `NO`, as `locality.municipality` does, which `List` stops advertising. Replace the + README's `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. - Select a row by its key alone, as the decision "A selector names a row by its key -- 2.52.0 From e435f8129604de6011de4e747575c36c7e27f410 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:56:07 +0200 Subject: [PATCH 143/966] Rewrap the step-up item --- todo.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 6210e3f..c7ef426 100644 --- a/todo.md +++ b/todo.md @@ -176,9 +176,8 @@ by a `data-import/` script, as README goal 12 asks. to the parent row, and a path may not end on it" states, refusing a step back down: `city[Oslo].country.name` is a load error today, and `city[Oslo].country` renders the cell `NO`, as `locality.municipality` does, which `List` stops advertising. Replace the - README's - `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, - rendering `Norway`. + README's `city[Oslo].country` example with + `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. - Select a row by its key alone, as the decision "A selector names a row by its key alone" states: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today, where the error should name `[1281]` and `[SE]`. Refuse a `name` option beside -- 2.52.0 From 8c4ee8362e0cf2fe8e5be81d9ec2536021f66e2f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:59:37 +0200 Subject: [PATCH 144/966] Name the near miss by the row whose cell spells it, retire the rules the key-only selector makes false, and ask which key each table keeps --- docs/decisions.md | 21 ++++++++++----------- todo.md | 29 ++++++++++++++++++----------- 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index ff220c4..3b7788d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -342,10 +342,7 @@ a load error rather than a file silently ignored. `municipality[0180]` reads as selection to anyone who has indexed an array, and `[St. Louis]` keeps a name whole where a colon or a dot-separated spelling could not; zsh -needs the brackets quoted, which the README's examples show. A name that spells another -row's key is refused at load rather than shadowed: a key names one row by contract, so -the name could never select its own, and the check is one lookup per row against the key -index that already exists. +needs the brackets quoted, which the README's examples show. ## `parent` names the link column and the table alike @@ -397,7 +394,9 @@ parent. A row is selected or drawn alike. out, and a record's link column still holds the key, since it is a column, not a path. - A path may not step back down into a child table after a step up: `city[Oslo].country.city.name` could be Oslo or a city of Norway drawn afresh, so it is - refused, naming `city[Oslo].name` and `country[NO].city.name`. + refused, naming the path to the row it stepped up from, `city[Oslo].name`, and the path + down from the parent, `country[NO].city.name`; after a drawn row, `city.country.city.name` + names `city.name` and `country.city.name`. - A route through another row is another path to the same data, not a second spelling: `city[Oslo].country.name` and `country[NO].name` both load. @@ -407,10 +406,11 @@ the path ended. ## A selector names a row by its key alone -Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result. A table with -a `key` selects by it alone: `country[NO]` loads, and `country[Norway]` is refused -naming `[NO]`, since a name could spell another row's key. A table with no `key` -selects by its `name` inside its parent, which is the key it has. +Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; valid while +every table either has a `key` or names its rows inside a parent. A table with a `key` +selects by it alone: `country[NO]` loads, and `country[Norway]` is refused naming +`[NO]`, found by the row whose cell spells the selector. A table with no `key` selects by +its `name` inside its parent, which is the key it has. ## A parent row with no child row is a load error @@ -632,8 +632,7 @@ claim fejkdata has no business making. A flat list of names carries neither the code a PRI encodes nor a selector reaching it, and goal 3 draws the two as one fact. The canonical spelling losing to the one its domain writes, above, settles the rest: a configuration writes `info`, so RFC 5424's -`Informational` stays the `severity` column, and the code and the keyword hold the two -selector slots. +`Informational` stays the `severity` column. ## `misc.tld` keys carry the leading dot, where other tables key on a bare code diff --git a/todo.md b/todo.md index c7ef426..9d2c2ca 100644 --- a/todo.md +++ b/todo.md @@ -179,10 +179,16 @@ by a `data-import/` script, as README goal 12 asks. README's `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. - Select a row by its key alone, as the decision "A selector names a row by its key - alone" states: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row - today, where the error should name `[1281]` and `[SE]`. Refuse a `name` option beside - `key`, which then selects nothing, drop it from the shipped tables, and state the - README's selectors and `CHANGELOG.md`'s "a path selects a row by key or name" to match. + alone" states, once the open question on each table's key is settled: + `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. + - Have a selector that misses name the key of the row whose cell spells it, matched + without case, else say the table selects by its key: `misc.territory[Sweden]` and + `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, `misc.tld[se]` + `[.se]`, and `misc.loglevel[Error]` the row a column spells `Error` in. + - Refuse a `name` option beside `key`, which then selects nothing, and drop it from + the shipped tables. + - Rewrite what selects by name: README.md lines 20, 174, 183, 193, 198, 255, 257, + 260, 473-491 and 499-504, and CHANGELOG.md lines 13-17 and 56. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice @@ -217,13 +223,6 @@ by a `data-import/` script, as README goal 12 asks. `drawn.category`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. -- Name a spelling that works when a row selector misses: `misc.protocol[tcp]`, - `misc.httpmethod[get]` and `misc.territory[se]` all answer "no row … has key or name" - and stop there, where a case-insensitive match could name the row that exists, and a - table with no `name` column could say it selects by key alone. `misc.tld[se]` misses - on the leading dot its keys carry rather than on case, and `misc.loglevel[Error]` - misses although a column carries `Error`, so the near miss is worth naming whatever - shape it takes. - Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text @@ -232,6 +231,14 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle +- Decide which spelling each shipped table selects by, now a selector names its key + alone. Keeping today's key loses `geo.US.region[IL].locality[Springfield]` to a + numeric GEOID, `misc.loglevel[err]` to `[3]`, `misc.tld[.рф]` to `[.xn--p1ai]`, + `misc.currency[Euro]`, `misc.protocol`'s second spelling and + `geo.SE.region[Skåne län]`. The tables carrying both today: `geo.SE` and `geo.US` + region, municipality and US locality, `misc` currency, httpstatus, language, + loglevel, mimetype, protocol, territory and tld, and both locales' `sex`. Decide too + whether the `name` option, left meaning "the key inside a parent", keeps its name. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 32f8774ddb902fc332c0b01555042a2422184ad0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 16:59:48 +0200 Subject: [PATCH 145/966] Plan the removal of the name-spells-a-key check --- todo.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index 9d2c2ca..7768d26 100644 --- a/todo.md +++ b/todo.md @@ -185,8 +185,9 @@ by a `data-import/` script, as README goal 12 asks. without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, `misc.tld[se]` `[.se]`, and `misc.loglevel[Error]` the row a column spells `Error` in. - - Refuse a `name` option beside `key`, which then selects nothing, and drop it from - the shipped tables. + - Refuse a `name` option beside `key`, which then selects nothing, drop it from the + shipped tables, and delete the load check refusing a name that spells another row's + key. - Rewrite what selects by name: README.md lines 20, 174, 183, 193, 198, 255, 257, 260, 473-491 and 499-504, and CHANGELOG.md lines 13-17 and 56. - Render selected rows as the decision "A selected row is fixed, not drawn, and an -- 2.52.0 From 6603a45dc927d58ff16af834c65c202316477463 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 17:00:43 +0200 Subject: [PATCH 146/966] Name every matching row's key on a near miss, and anchor the rewrite list on the phrases it replaces --- todo.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/todo.md b/todo.md index 7768d26..3117b8d 100644 --- a/todo.md +++ b/todo.md @@ -181,15 +181,21 @@ by a `data-import/` script, as README goal 12 asks. - Select a row by its key alone, as the decision "A selector names a row by its key alone" states, once the open question on each table's key is settled: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. - - Have a selector that misses name the key of the row whose cell spells it, matched - without case, else say the table selects by its key: `misc.territory[Sweden]` and - `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, `misc.tld[se]` - `[.se]`, and `misc.loglevel[Error]` the row a column spells `Error` in. + - Have a selector that misses name the key of every row whose cell spells it, + matched without case, else say the table selects by its key: + `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` + its key, and `misc.loglevel[Error]` the row a column spells `Error` in. - Refuse a `name` option beside `key`, which then selects nothing, drop it from the shipped tables, and delete the load check refusing a name that spells another row's key. - - Rewrite what selects by name: README.md lines 20, 174, 183, 193, 198, 255, 257, - 260, 473-491 and 499-504, and CHANGELOG.md lines 13-17 and 56. + - Rewrite what selects by name: in README.md, "selected by key or name", + `misc.currency[Euro]`, "selects by either spelling", `misc.tld[.рф]`, + `misc.loglevel[err]` and "the code and the keyword do", the `geo` table's "by code + or name", "by USPS abbreviation or name" and "by GEOID or name", + `geo.SE.region[Skåne län]`, `geo.US.region[IL].locality[Springfield]`, "selection by + name", the `name` option's paragraph and "`[key]` or `[name]`"; in CHANGELOG.md, "a + path selects a row by key or name", "a name spelling another row's key" and + `misc.tld[.рф]`. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice @@ -240,6 +246,8 @@ by a `data-import/` script, as README goal 12 asks. region, municipality and US locality, `misc` currency, httpstatus, language, loglevel, mimetype, protocol, territory and tld, and both locales' `sex`. Decide too whether the `name` option, left meaning "the key inside a parent", keeps its name. + Record each answer in `docs/decisions.md`, amending the `misc.tld` and `misc.loglevel` + entries it changes. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 7acb2136ec1511c869fc2a443efeb727175949c2 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 23:09:26 +0200 Subject: [PATCH 147/966] Key every table on one code, never a free-form name, and plan each table's key --- AGENTS.md | 5 ++--- docs/decisions.md | 34 +++++++++++++++++----------------- todo.md | 39 +++++++++++++++++---------------------- 3 files changed, 36 insertions(+), 42 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c519a01..ec26f1d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,7 +73,7 @@ In [docs/decisions.md](docs/decisions.md): - After a row, a path names a column or a linked table - A selected row is fixed, not drawn, and an unnamed read beside it is refused - A link column after a row steps up to the parent row, and a path may not end on it -- A selector names a row by its key alone +- A table selects by one key column, a code, never a free-form name - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields - A table is a record of string columns @@ -94,7 +94,6 @@ In [docs/decisions.md](docs/decisions.md): - A title is a table under `sex` - A table owns the spelling of a selector on it - No builtin reads the clock, so a date is bounded by days, never by an age -- A name column without a key resolves inside its parent - `misc` is what every locale shares - A register's canonical spelling loses to the one its domain writes - `misc.territory` is the spine, and a `misc` table naming a territory links to it @@ -103,7 +102,7 @@ In [docs/decisions.md](docs/decisions.md): - `misc.port` selects by number, and carries no `name` - `misc.car` is one flat table, not a make linked to its models - `misc.territory` names its sovereign in a column, and there is no `misc.country` table -- `misc.loglevel` is a table keyed by the code, rendering POSIX's keyword +- `misc.loglevel` is a table keyed by POSIX's keyword, carrying the code - `misc.tld` keys carry the leading dot, where other tables key on a bare code - `misc.tld` is a table of its own, and `misc.territory.tld` stays a column - `misc.territory` carries a currency code, it does not link to `misc.currency` diff --git a/docs/decisions.md b/docs/decisions.md index 3b7788d..33e2767 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -385,8 +385,8 @@ parent. A row is selected or drawn alike. - Where a segment follows, the link column's name reads the parent row, one level per step, chained: `city[Oslo].country.name` renders `Norway`, `locality.municipality.name` - the drawn locality's municipality, and `geo.SE.locality[Lund].municipality.region.name` - Lund's region. + the drawn locality's municipality, and + `geo.US.locality[1714000].municipality.region.name` Chicago's state. - A path ending on it is refused at `New`, `NewTemplate` and `Fake` alike, naming the parent's key column and the parent's format spelled as paths: with a `country` format of `{name} ({alpha2})`, `city[Oslo].country` names `city[Oslo].country.alpha2` and @@ -404,13 +404,21 @@ Checked against three simulated template writers from the Audience: all three re parent row where a segment followed, and they split between the code and the name where the path ended. -## A selector names a row by its key alone +## A table selects by one key column, a code, never a free-form name Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; valid while -every table either has a `key` or names its rows inside a parent. A table with a `key` -selects by it alone: `country[NO]` loads, and `country[Norway]` is refused naming -`[NO]`, found by the row whose cell spells the selector. A table with no `key` selects by -its `name` inside its parent, which is the key it has. +the key is the code a user writes, as a database keys a table. `country[NO]` loads, and +`country[Norway]` is refused naming `[NO]`, found by the row whose cell spells the +selector. The key is the friendliest code the register holds, `misc.territory[SE]` and +`misc.currency[EUR]`, and only where it holds none an id the import assigns and never +moves to another row. There is no `name` option. + +- `misc.loglevel` keys on the keyword, `[err]`, which a configuration writes. +- `misc.tld` keys on the ASCII form, `[.xn--p1ai]`, the one DNS carries. +- `geo.US.locality` keys on the Census GEOID, `[1714000]` for Chicago. +- `geo.SE.locality` keys on an assigned id: a postort has no code of its own. +- A name table, `first-name`, `last-name` and `title`, carries no key, so it is drawn + from and never selected. ## A parent row with no child row is a load error @@ -546,8 +554,7 @@ are a curated proportion rather than bearers anyone counted. A reference reaches a table by a path that carries no selector — `sv_SE.person.first` reads `first-name` through `sex` — so the walk that resolved a name cannot say where a reader would type one. The table's own location can, which is why it keeps its path, and -why an ambiguity error names `sv_SE.sex[f].first-name[Kim]` rather than the table's own -name. +why an error names `geo.SE.municipality[1281]` rather than `municipality[1281]`. ## No builtin reads the clock, so a date is bounded by days, never by an age @@ -555,13 +562,6 @@ An `age(min,max)` would make a seeded fixture change with the day it runs on, wh what a seed exists to prevent; a birthdate for someone 20 to 60 is `date(1966-01-01,2006-12-31,…)`, re-pinned as any fixture is. -## A name column without a key resolves inside its parent - -A given name both sexes carry is a row under each, so `name` cannot be the key; the -parent's row tells the two apart, `sex[f].first-name[Kim]`, the ambiguity error spells -each row inside its parent, and a name repeating inside one parent row is refused at -load, since nothing could then select it. - ## `misc` is what every locale shares A category whose facts differ by country belongs in that country's locale, read from the @@ -627,7 +627,7 @@ own family; a test proves every value names a row instead. A territory the regis records no state for stands alone, which is `EH` alone, and naming one for it would be a claim fejkdata has no business making. -## `misc.loglevel` is a table keyed by the code, rendering POSIX's keyword +## `misc.loglevel` is a table keyed by POSIX's keyword, carrying the code A flat list of names carries neither the code a PRI encodes nor a selector reaching it, and goal 3 draws the two as one fact. The canonical spelling losing to the one its diff --git a/todo.md b/todo.md index 3117b8d..a90176e 100644 --- a/todo.md +++ b/todo.md @@ -178,23 +178,28 @@ by a `data-import/` script, as README goal 12 asks. cell `NO`, as `locality.municipality` does, which `List` stops advertising. Replace the README's `city[Oslo].country` example with `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. -- Select a row by its key alone, as the decision "A selector names a row by its key - alone" states, once the open question on each table's key is settled: - `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. +- Key every table as the decision "A table selects by one key column, a code, never a + free-form name" states: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select + a row today. - Have a selector that misses name the key of every row whose cell spells it, matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` - its key, and `misc.loglevel[Error]` the row a column spells `Error` in. - - Refuse a `name` option beside `key`, which then selects nothing, drop it from the - shipped tables, and delete the load check refusing a name that spells another row's - key. + its key, and `misc.loglevel[Error]` `[err]`. + - Delete the `name` option, and the load checks it carries: a name spelling another + row's key, and a name repeating inside its parent. + - Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on an id `data-import/geo-se.py` + assigns and keeps across imports, linking `street` and `postal-code` by it. + - Drop `key` from `last-name` and `sv_SE.title`. - Rewrite what selects by name: in README.md, "selected by key or name", `misc.currency[Euro]`, "selects by either spelling", `misc.tld[.рф]`, - `misc.loglevel[err]` and "the code and the keyword do", the `geo` table's "by code - or name", "by USPS abbreviation or name" and "by GEOID or name", - `geo.SE.region[Skåne län]`, `geo.US.region[IL].locality[Springfield]`, "selection by - name", the `name` option's paragraph and "`[key]` or `[name]`"; in CHANGELOG.md, "a - path selects a row by key or name", "a name spelling another row's key" and + `misc.loglevel[3]`, `misc.loglevel[err]` and "the code and the keyword do", the `geo` + table's "by code or name", "by USPS abbreviation or name", "by GEOID or name" and + "by name", `geo.SE.locality[Lund]`, `geo.SE.region[Skåne län]`, + `geo.US.region[IL].locality[Springfield]`, "selection by name", + `first-name[Taylor]`, `first-name[Kim]`, `sv_SE.title[dr]`, the `name` option's + paragraph and "`[key]` or `[name]`"; `locality[Lund]` in docs/decisions.md; in + CHANGELOG.md, "a path selects a row by key or name", "a name spelling another row's + key", "a name repeating inside one parent row", "A `name` without a `key`" and `misc.tld[.рф]`. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses @@ -238,16 +243,6 @@ by a `data-import/` script, as README goal 12 asks. ### Open questions to settle -- Decide which spelling each shipped table selects by, now a selector names its key - alone. Keeping today's key loses `geo.US.region[IL].locality[Springfield]` to a - numeric GEOID, `misc.loglevel[err]` to `[3]`, `misc.tld[.рф]` to `[.xn--p1ai]`, - `misc.currency[Euro]`, `misc.protocol`'s second spelling and - `geo.SE.region[Skåne län]`. The tables carrying both today: `geo.SE` and `geo.US` - region, municipality and US locality, `misc` currency, httpstatus, language, - loglevel, mimetype, protocol, territory and tld, and both locales' `sex`. Decide too - whether the `name` option, left meaning "the key inside a parent", keeps its name. - Record each answer in `docs/decisions.md`, amending the `misc.tld` and `misc.loglevel` - entries it changes. - Decide whether a goal 5 violation joins the comprehension gate's exception list: the `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a -- 2.52.0 From 03a48e378cf21a9154c077104143a8789d234eae Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 23:15:52 +0200 Subject: [PATCH 148/966] Key only a table a user selects into, state how a postort keeps its id, retire what the name option served, and make the doc sweep a grep --- AGENTS.md | 1 - docs/decisions.md | 28 ++++++++++------------------ todo.md | 35 +++++++++++++++-------------------- 3 files changed, 25 insertions(+), 39 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ec26f1d..691412c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -99,7 +99,6 @@ In [docs/decisions.md](docs/decisions.md): - `misc.territory` is the spine, and a `misc` table naming a territory links to it - A table whose register publishes no frequency draws evenly - `misc.timezone` weighs a zone by the people living in it -- `misc.port` selects by number, and carries no `name` - `misc.car` is one flat table, not a make linked to its models - `misc.territory` names its sovereign in a column, and there is no `misc.country` table - `misc.loglevel` is a table keyed by POSIX's keyword, carrying the code diff --git a/docs/decisions.md b/docs/decisions.md index 33e2767..1513312 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -410,15 +410,14 @@ Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; vali the key is the code a user writes, as a database keys a table. `country[NO]` loads, and `country[Norway]` is refused naming `[NO]`, found by the row whose cell spells the selector. The key is the friendliest code the register holds, `misc.territory[SE]` and -`misc.currency[EUR]`, and only where it holds none an id the import assigns and never -moves to another row. There is no `name` option. +`misc.currency[EUR]`. There is no `name` option, and a table a user has no row of to +select, a name table, `street`, `car` or `useragent`, carries no key and is drawn only. -- `misc.loglevel` keys on the keyword, `[err]`, which a configuration writes. -- `misc.tld` keys on the ASCII form, `[.xn--p1ai]`, the one DNS carries. - `geo.US.locality` keys on the Census GEOID, `[1714000]` for Chicago. -- `geo.SE.locality` keys on an assigned id: a postort has no code of its own. -- A name table, `first-name`, `last-name` and `title`, carries no key, so it is drawn - from and never selected. +- `geo.SE.locality` keys on an integer the import assigns, since a postort has no code of + its own: the import reads the previous TSV and keeps each postort's id by kommun and + name, a new postort takes the next unused integer, and a removed or renamed one retires + its id for good. ## A parent row with no child row is a load error @@ -443,8 +442,7 @@ option earns its place. ## The key index is built at load, the rest on first draw A link is proved against the parent's keys and a key's uniqueness is a data mistake, so -both are load-time; the name index and the per-parent child lists serve only a draw or a -selection, so they wait for the first one, keeping `New` linear in the bytes read. +both are load-time; the per-parent child lists serve only a draw, so they wait for the first one, keeping `New` linear in the bytes read. ## Two categories may name one TSV @@ -551,9 +549,9 @@ are a curated proportion rather than bearers anyone counted. ## A table owns the spelling of a selector on it -A reference reaches a table by a path that carries no selector — `sv_SE.person.first` -reads `first-name` through `sex` — so the walk that resolved a name cannot say where a -reader would type one. The table's own location can, which is why it keeps its path, and +A reference reaches a table by a path that carries no selector — `sv_SE.address` reads +`geo.SE.locality` through its template — so the walk that reached it cannot say where +a reader would type a selector. The table's own location can, which is why it keeps its path, and why an error names `geo.SE.municipality[1281]` rather than `municipality[1281]`. ## No builtin reads the clock, so a date is bounded by days, never by an age @@ -603,12 +601,6 @@ floored at 15,000, which goal 14 asks for: over 300 seeded draws of even weight gave them 44, and `America/Indiana/Petersburg`, a town of 2,400, fell from 17 to 0. Valid while a draw weighted this way lands where people live. -## `misc.port` selects by number, and carries no `name` - -29 of its services sit on more than one port, `http-alt` on three, so -`misc.port[http-alt]` could name no one row. The number is what a port field holds -anyway, and `misc.port[443].service` reads the other way. - ## `misc.car` is one flat table, not a make linked to its models A row is a make and a model drawn together, so no render pairs a Volvo with a RAV4. Two diff --git a/todo.md b/todo.md index a90176e..34f3f91 100644 --- a/todo.md +++ b/todo.md @@ -101,7 +101,7 @@ by a `data-import/` script, as README goal 12 asks. rewrite that returns a different value where the read it conflicts with sits inside another category, and report a struct column's draw conflict with the path spelling a tag takes. -- Let `geo.SE.locality[Lund].address` descend from a selected row into the template +- Let `geo.US.locality[1714000].address` descend from a selected row into the template beside the family: require the path step to reach a sibling category, and seed every draw group of the render from the outer selector's pins. - Read `email.local`'s and `username`'s handles from the shipped name tables, which @@ -185,29 +185,24 @@ by a `data-import/` script, as README goal 12 asks. matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, and `misc.loglevel[Error]` `[err]`. - - Delete the `name` option, and the load checks it carries: a name spelling another - row's key, and a name repeating inside its parent. - - Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on an id `data-import/geo-se.py` - assigns and keeps across imports, linking `street` and `postal-code` by it. - - Drop `key` from `last-name` and `sv_SE.title`. - - Rewrite what selects by name: in README.md, "selected by key or name", - `misc.currency[Euro]`, "selects by either spelling", `misc.tld[.рф]`, - `misc.loglevel[3]`, `misc.loglevel[err]` and "the code and the keyword do", the `geo` - table's "by code or name", "by USPS abbreviation or name", "by GEOID or name" and - "by name", `geo.SE.locality[Lund]`, `geo.SE.region[Skåne län]`, - `geo.US.region[IL].locality[Springfield]`, "selection by name", - `first-name[Taylor]`, `first-name[Kim]`, `sv_SE.title[dr]`, the `name` option's - paragraph and "`[key]` or `[name]`"; `locality[Lund]` in docs/decisions.md; in - CHANGELOG.md, "a path selects a row by key or name", "a name spelling another row's - key", "a name repeating inside one parent row", "A `name` without a `key`" and - `misc.tld[.рф]`. + - Delete the `name` option and the load checks it carries, a name spelling another + row's key and a name repeating inside its parent, and have `New` refuse a table + still carrying `name`, naming `key`. + - Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on the integer + `data-import/geo-se.py` assigns as the decision states, linking `street` and + `postal-code` by it. + - Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. + - Leave no selection by name in README.md, CHANGELOG.md or docs/: no selector spelling + a free-form name, `geo.SE.locality[Lund]`, `misc.currency[Euro]`, `first-name[Kim]` + and the rest, no "key or name", "by name" or "either" wording about selectors, and no + `name` option in an example. - Render selected rows as the decision "A selected row is fixed, not drawn, and an unnamed read beside it is refused" states: `New` refuses `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice whose items select different rows. Have the refusal of an unnamed read name the - rewrite through each selection: beside `{/geo.SE.locality[Lund].name}`, - `{/geo.SE.municipality.name}` names none today, where - `{/geo.SE.locality[Lund].municipality.name}` holds once the step up above lands. + rewrite through each selection: beside `{/geo.US.locality[1714000].name}`, + `{/geo.US.municipality.name}` names none today, where + `{/geo.US.locality[1714000].municipality.name}` holds once the step up above lands. Scope the README's "A reference path into a category is held" to a path that selects no row. - Report the same error every load for a table with two bad options, and for a -- 2.52.0 From 52292d366981bb712d29c845144f5970098094b6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 23:16:58 +0200 Subject: [PATCH 149/966] Number the first postort ids in name order, keep them by the unique name, and rewrap --- docs/decisions.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 1513312..31f3626 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -410,14 +410,14 @@ Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; vali the key is the code a user writes, as a database keys a table. `country[NO]` loads, and `country[Norway]` is refused naming `[NO]`, found by the row whose cell spells the selector. The key is the friendliest code the register holds, `misc.territory[SE]` and -`misc.currency[EUR]`. There is no `name` option, and a table a user has no row of to -select, a name table, `street`, `car` or `useragent`, carries no key and is drawn only. +`misc.currency[EUR]`. There is no `name` option, and a table nothing selects into, a name +table, `street`, `car` or `useragent`, carries no key and is only drawn. - `geo.US.locality` keys on the Census GEOID, `[1714000]` for Chicago. - `geo.SE.locality` keys on an integer the import assigns, since a postort has no code of - its own: the import reads the previous TSV and keeps each postort's id by kommun and - name, a new postort takes the next unused integer, and a removed or renamed one retires - its id for good. + its own. The first import numbers the postorter in name order; later imports keep each + id by the postort's name, unique across Sweden as the key it replaces, give a new + postort the next unused integer, and retire a removed or renamed one's id for good. ## A parent row with no child row is a load error @@ -442,7 +442,8 @@ option earns its place. ## The key index is built at load, the rest on first draw A link is proved against the parent's keys and a key's uniqueness is a data mistake, so -both are load-time; the per-parent child lists serve only a draw, so they wait for the first one, keeping `New` linear in the bytes read. +both are load-time; the per-parent child lists serve only a draw, so they wait for the +first one, keeping `New` linear in the bytes read. ## Two categories may name one TSV @@ -551,8 +552,8 @@ are a curated proportion rather than bearers anyone counted. A reference reaches a table by a path that carries no selector — `sv_SE.address` reads `geo.SE.locality` through its template — so the walk that reached it cannot say where -a reader would type a selector. The table's own location can, which is why it keeps its path, and -why an error names `geo.SE.municipality[1281]` rather than `municipality[1281]`. +a reader would type a selector. The table's own location can, which is why it keeps its +path, and why an error names `geo.SE.municipality[1281]` rather than `municipality[1281]`. ## No builtin reads the clock, so a date is bounded by days, never by an age -- 2.52.0 From 3b77ed6b065fb73ba6b3e22b702cfd587a7b7c50 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 23:48:31 +0200 Subject: [PATCH 150/966] Give a table's current row one owner, the pin set, and return a column's null from its read --- drawfence_test.go | 3 --- hold.go | 32 +++++++++++++++++++++----------- holdset.go | 16 +++++++++++----- record.go | 3 --- render.go | 13 ++++++------- todo.md | 2 -- 6 files changed, 38 insertions(+), 31 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index bb6004d..3b6c23a 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -131,9 +131,6 @@ func renderRoot(s *session, t *template) { sc.hold().pins.pin(t.table, r) render(s, t.table.pinnedRow, sc) } - case t.cellOf != nil: - sc.t, sc.row = t.cellOf, t.cellRow - render(s, t, sc) case t.repeat > 1: expand(s, t, sc) default: diff --git a/hold.go b/hold.go index cee9335..312fae8 100644 --- a/hold.go +++ b/hold.go @@ -29,11 +29,7 @@ type draw struct { // linkRefs prove every step, so the walk cannot fail. func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { if s.trace != nil { - var table string - if sc.t != nil { - table = strings.Clone(sc.t.path) - } - s.trace.read(strings.Clone(sc.group), table, sc.row, a) + traceRead(s.trace, t, sc, a) } if !t.held[a.key] { if len(a.tail) > 0 { @@ -45,6 +41,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if r, done := d.value[a.path]; done { return r } + sc.pins = &d.pins r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &d.pins, &pathDraws{s: s, held: d, a: &a}), sc) if d.value == nil { d.value = map[string]draw{} @@ -59,6 +56,18 @@ type renderTrace struct { repeatDepth int } +func traceRead(trace *renderTrace, t *template, sc renderScope, a arm) { + var table string + var row int + switch { + case t.cellOf != nil: + table, row = t.cellOf.path, t.cellRow + case t.table != nil: + table, row = t.table.path, sc.rows().mustRow(t.table) + } + trace.read(strings.Clone(sc.group), strings.Clone(table), row, a) +} + // readHold is the hold a held read keeps its draw in: for a reference that reads a path, // the render's hold for its group, so its draw spans the render; for a sibling, or a // bare reference, held. @@ -73,14 +82,15 @@ func readHold(held *hold, sc renderScope, a arm) *hold { // reference alone whose read drew null. func renderLeaf(s *session, n node, sc renderScope) draw { n = resolveChoice(s, n) - if _, isNull := n.(*nullItem); isNull { + switch leaf := n.(type) { + case *nullItem: return draw{null: true} + case *template: + if leaf.readsColumn != nil { + return readField(s, leaf, nil, sc.in(leaf), leaf.readsColumn.a) + } } - r := draw{text: render(s, n, sc)} - if t, _ := n.(*template); t != nil && t.readsColumn != nil { - r.null = sc.in(t).hold().value[t.readsColumn.a.path].null - } - return r + return draw{text: render(s, n, sc)} } // resolveChoice resolves a choice to one variant, so a bound head is a concrete node the diff --git a/holdset.go b/holdset.go index ab91287..bae0d47 100644 --- a/holdset.go +++ b/holdset.go @@ -11,14 +11,13 @@ type holdSet struct { } // renderScope is where a render reads its reference paths: a hold set, in the draw group of the -// template rendering, and the row a table's format is rendering, which its columns read. It passes -// by value, and what is read from it reaches a map key, an interface or a func value only as a -// copy; else sc, and with it every render's hold set, moves to the heap. +// template rendering, and the pin set a table's columns read their row from: a whole read's own, +// else the hold's. It passes by value, and what is read from it reaches a map key, an interface or +// a func value only as a copy; else sc, and with it every render's hold set, moves to the heap. type renderScope struct { set *holdSet group string - t *table - row int + pins *pinSet } func eagerHoldSet() holdSet { @@ -52,6 +51,13 @@ func (sc renderScope) in(t *template) renderScope { return sc } +func (sc renderScope) rows() *pinSet { + if sc.pins != nil { + return sc.pins + } + return &sc.hold().pins +} + func (sc renderScope) hold() *hold { if sc.group == "" { return &sc.set.unnamed diff --git a/record.go b/record.go index 6b40b6f..f8016bd 100644 --- a/record.go +++ b/record.go @@ -264,9 +264,6 @@ func recordOf(n node) (*template, []Column, error) { // over sc's hold; a table's columns read the row pinned there. func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) - if t.table != nil { - sc.t, sc.row = t.table, sc.hold().pins.mustRow(t.table) - } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { column := renderLeaf(s, t.fields[r.columns[i].Name], sc) diff --git a/render.go b/render.go index be5a7c5..b3cc7d6 100644 --- a/render.go +++ b/render.go @@ -61,19 +61,18 @@ func render(s *session, n node, sc renderScope) string { case *nullItem: return "" case *table: - sc.t, sc.row = n, n.drawRow(s) + var own pinSet + own.add(n, n.drawRow(s)) + sc.pins = &own return expand(s, n.formatTemplate, sc) case *tableRow: - sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) return expand(s, n.t.formatTemplate, sc) case *tableColumn: - if sc.t != n.t { - sc.t, sc.row = n.t, sc.hold().pins.mustRow(n.t) - } - if cell := n.t.cellTemplate(sc.row, n.i); cell != nil { + row := sc.rows().mustRow(n.t) + if cell := n.t.cellTemplate(row, n.i); cell != nil { return render(s, cell, sc) } - return n.t.cell(sc.row, n.i) + return n.t.cell(row, n.i) case *template: sc = sc.in(n) if n.repeat == 1 { diff --git a/todo.md b/todo.md index 34f3f91..625b2d2 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Give a table's current row one owner: `renderScope.t` and `row` hold it beside the - `pinSet`, and `renderLeaf` reads a column's null back from a hold `readField` wrote. - Split `walkPath`'s four modes, so a caller does not select a behaviour by which of `level`, `leaf`, `pins`, `drawn` and `draws` it leaves nil. - Gather the family fence beside `drawWalk.check`: `checkFamilies`, `replayPairs`, -- 2.52.0 From c0754abfca5b4ddfb429b85a4f6bd09c9425001c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 23:54:16 +0200 Subject: [PATCH 151/966] Want a table format's calc over its own column to render --- table_test.go | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/table_test.go b/table_test.go index c1376d6..3218eae 100644 --- a/table_test.go +++ b/table_test.go @@ -110,6 +110,22 @@ func TestTableRendersRowsAndColumns(t *testing.T) { t.Fatalf("column %q is %s null=%v, want a string", c.Name, c.DataType, c.Null) } } + calc := newGenerator(t, writeFiles(t, map[string]string{ + "r.json": `{"format":"{name} {calc(population * 2)}","rows":"r.tsv","key":"code"}`, + "r.tsv": "code\tname\tpopulation\n01\tA\t10\n02\tB\t4\n", + "w.json": `"{/r}"`, + })) + for i := 0; i < 20; i++ { + if got := fake(t, calc, "r"); got != "A 20" && got != "B 8" { + t.Fatalf("r = %q, want a row doubling its population", got) + } + if got := fake(t, calc, "r[01]"); got != "A 20" { + t.Fatalf("r[01] = %q, want A 20", got) + } + if got := fake(t, calc, "w"); got != "A 20" && got != "B 8" { + t.Fatalf("w = %q, want a row doubling its population", got) + } + } } func TestTableWeightSkewsTheDraw(t *testing.T) { -- 2.52.0 From 270aa41dfbc83b9398b3d62198be18048b2e7add Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 27 Sep 2026 23:55:28 +0200 Subject: [PATCH 152/966] Point a read at the pins it drew only where it walked a path --- hold.go | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hold.go b/hold.go index 312fae8..70033d2 100644 --- a/hold.go +++ b/hold.go @@ -41,7 +41,9 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if r, done := d.value[a.path]; done { return r } - sc.pins = &d.pins + if len(a.tail) > 0 { + sc.pins = &d.pins + } r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &d.pins, &pathDraws{s: s, held: d, a: &a}), sc) if d.value == nil { d.value = map[string]draw{} -- 2.52.0 From 9126e0e07d86a7cc630d73d9fa06c4d8b64e0937 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:10:15 +0200 Subject: [PATCH 153/966] Want walkPath's modes as walkers of their own, each holding only what it uses --- path_test.go | 46 +++++++++++++--------------------------------- table_test.go | 8 ++++---- 2 files changed, 17 insertions(+), 37 deletions(-) diff --git a/path_test.go b/path_test.go index a8a9fa6..3cdfa4a 100644 --- a/path_test.go +++ b/path_test.go @@ -9,51 +9,31 @@ import ( func TestWalkPathStopsAtAMissingSegment(t *testing.T) { n := compiled(t, `{"format":"{a}","a":{"format":"{b}","b":"leaf"}}`) - var seen []string - walk := pathWalk{ - mode: walkEvery, - pins: &pinSet{}, - level: func(tm *template, rest []string) error { seen = append(seen, "level:"+rest[0]); return nil }, - leaf: func(n node) error { seen = append(seen, "leaf"); return nil }, + if leaves := pathLeaves(n, []string{"a", "b"}); len(leaves) != 1 { + t.Fatalf("pathLeaves(a.b) = %v, want the one leaf", leaves) } - if _, err := walkPath(n, []string{"a", "b"}, walk); err != nil { - t.Fatalf("walkPath(a.b) = %v", err) - } - if want := []string{"level:a", "level:b", "leaf"}; !slices.Equal(seen, want) { - t.Errorf("walk visited %v, want %v", seen, want) - } - seen = nil - _, err := walkPath(n, []string{"a", "nope", "deeper"}, walk) + w := &proveWalk{tail: []string{"a", "nope", "deeper"}} + _, err := w.walk(n, w.tail) if err == nil || !strings.Contains(err.Error(), `no field "nope"`) { - t.Errorf("walkPath(a.nope.deeper) = %v, want the missing segment named", err) + t.Errorf("walk(a.nope.deeper) = %v, want the missing segment named", err) } - if slices.Contains(seen, "leaf") { - t.Errorf("walk reached a leaf past a missing segment: %v", seen) + if len(w.leaves) > 0 { + t.Errorf("walk reached a leaf past a missing segment: %v", w.leaves) } } func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) - var leaves []node - _, err := walkPath(n, []string{"f"}, pathWalk{ - mode: walkEvery, - pins: &pinSet{}, - leaf: func(n node) error { leaves = append(leaves, n); return nil }, - }) - if err != nil || len(leaves) != 2 { - t.Fatalf("walkPath through a choice = %v, %d leaves, want both variants' f", err, len(leaves)) + if leaves := pathLeaves(n, []string{"f"}); len(leaves) != 2 { + t.Fatalf("pathLeaves through a choice = %v, want both variants' f", leaves) } } func TestWalkCoverStopsAtAChoice(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) - var leaves []node - _, err := walkPath(n, []string{"f"}, pathWalk{ - mode: walkCover, - leaf: func(n node) error { leaves = append(leaves, n); return nil }, - }) - if err != nil || len(leaves) != 1 || leaves[0] != n { - t.Fatalf("cover through a choice = %v, leaves %v, want the choice alone", err, leaves) + leaf, err := (&coverWalk{into: map[node]bool{}}).walk(n, []string{"f"}) + if err != nil || leaf != n { + t.Fatalf("cover through a choice = %v, %v, want the choice itself", leaf, err) } } @@ -63,7 +43,7 @@ func TestDrawPathPanicsOnAnUnprovedPath(t *testing.T) { t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the path and the missing field", r) } }() - drawPath(compiled(t, `"plain"`), []string{"f"}, "plain", &pinSet{}, &pathDraws{s: engine(1).rand}) + drawPath(compiled(t, `"plain"`), []string{"f"}, "plain", &renderWalk{s: engine(1).rand, pins: &pinSet{}}) } func TestDeepDottedPath(t *testing.T) { diff --git a/table_test.go b/table_test.go index 3218eae..ed80b16 100644 --- a/table_test.go +++ b/table_test.go @@ -1061,14 +1061,14 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { t.Fatal(err) } head := f.categories[segs[0]].(*table) - var probed pinSet - drawn := map[*table]bool{} - if _, err := walkPath(head, segs[1:], pathWalk{mode: walkProbe, pins: &probed, drawn: drawn}); err != nil { + probe := &probeWalk{} + if _, err := probe.walk(head, segs[1:]); err != nil { t.Fatalf("%s: %v", path, err) } + probed, drawn := probe.pins, probe.drawn for i := 0; i < 20; i++ { var pins pinSet - drawPath(head, segs[1:], segs[0], &pins, &pathDraws{s: f.rand}) + drawPath(head, segs[1:], segs[0], &renderWalk{s: f.rand, pins: &pins}) want := map[*table]bool{} pins.each(func(tbl *table, _ int) { if _, selected := probed.pinned(tbl); !selected { -- 2.52.0 From 94eb7aa81f5395fb405fac2a061010018fbe9b52 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:19:43 +0200 Subject: [PATCH 154/966] Split walkPath's four modes into walkers of their own, sharing how a path steps --- family.go | 5 +- graph.go | 6 +- hold.go | 2 +- holdfence.go | 2 +- path.go | 424 ++++++++++++++++++++++++++++++--------------------- reference.go | 4 +- render.go | 5 +- 7 files changed, 261 insertions(+), 187 deletions(-) diff --git a/family.go b/family.go index e870057..af1ac51 100644 --- a/family.go +++ b/family.go @@ -32,8 +32,9 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { if !isTable { return nil } - tr := &tableRead{headTable: t, drawn: map[*table]bool{}} - _, _ = walkPath(t, a.tail, pathWalk{mode: walkProbe, pins: &tr.pins, drawn: tr.drawn}) + probe := &probeWalk{} + _, _ = probe.walk(t, a.tail) + tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] cur := t for i, seg := range a.tail { diff --git a/graph.go b/graph.go index ec34bee..c128b0b 100644 --- a/graph.go +++ b/graph.go @@ -164,9 +164,9 @@ func renderEdges(n node) []renderEdge { // contributes every variant, since any of them may be the one drawn. checkPath has // already proved the tail resolves in every variant. func pathLeaves(n node, tail []string) []node { - var out []node - _, _ = walkPath(n, tail, pathWalk{mode: walkEvery, pins: &pinSet{}, leaf: func(n node) error { out = append(out, n); return nil }}) - return out + w := &proveWalk{tail: tail} + _, _ = w.walk(n, tail) + return w.leaves } type renderCounts map[node]int diff --git a/hold.go b/hold.go index 70033d2..9c20384 100644 --- a/hold.go +++ b/hold.go @@ -44,7 +44,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if len(a.tail) > 0 { sc.pins = &d.pins } - r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &d.pins, &pathDraws{s: s, held: d, a: &a}), sc) + r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &renderWalk{s: s, pins: &d.pins, held: d, a: &a}), sc) if d.value == nil { d.value = map[string]draw{} } diff --git a/holdfence.go b/holdfence.go index f108c20..27b35c4 100644 --- a/holdfence.go +++ b/holdfence.go @@ -100,7 +100,7 @@ func operandReader(t *template, head string) string { // coverPath collects what holding one path pins: every choice level the path // passes through, whole, and the leaf it renders. func coverPath(n node, tail []string, into map[node]bool) { - _, _ = walkPath(n, tail, pathWalk{mode: walkCover, leaf: func(n node) error { cover(n, into, false); return nil }}) + _, _ = (&coverWalk{into: into}).walk(n, tail) } // cover collects a level and everything contained in it. A fixed string outside a diff --git a/path.go b/path.go index 2853225..87a4aee 100644 --- a/path.go +++ b/path.go @@ -125,154 +125,282 @@ func nameSegments(segs []string) []string { return out } -// walkMode is what a walk does at a choice, and whether it reads the rows a path -// selects and draws. -type walkMode uint8 - -const ( - // walkEvery proves every variant carries the rest of the path and walks each, - // pinning the rows it selects in pins. - walkEvery walkMode = iota - // walkCover stops at a choice, handing it whole to leaf, and reads no row. - walkCover - // walkProbe proves as walkEvery does, walking one variant, so a path that - // resolves resolves whichever variants and rows are drawn, and marks in drawn - // the tables a draw would pin. - walkProbe - // walkDraw draws the rows and variants a proved path reads; drawPath is its - // one entry. - walkDraw -) - -// pathWalk is one walk of a dotted path: level runs at each template a segment -// descends into, leaf where the walk ends, and a nil action is skipped. -type pathWalk struct { - mode walkMode - level func(t *template, rest []string) error - leaf func(n node) error - pins *pinSet - drawn map[*table]bool - draws *pathDraws -} - -// pathDraws is what a walk draws with: the session, and, for a held read, the hold -// keeping the variant drawn at each level of a, so paths sharing a prefix share it. -type pathDraws struct { - s *session - held *hold - a *arm -} - -// walkPath descends tail from n and returns the node it ends at: a folder or -// template by its next segment, a choice by its mode, which consumes no segment, a -// table by walkTable. A missing segment is an error, so no walk reaches past what -// the data holds. -func walkPath(n node, tail []string, w pathWalk) (node, error) { - if len(tail) == 0 { - return n, w.atLeaf(n) - } - if t, isTable := n.(*table); isTable { - return walkTable(t, tail, w, false) - } - if isSelector(tail[0]) { - return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(tail[0])) +// stepInto is what seg names under n: a folder's entry or a template's field. A +// missing segment is an error, so no walk reaches past what the data holds. +func stepInto(n node, seg string) (node, error) { + if isSelector(seg) { + return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(seg)) } switch n := n.(type) { case *folder: - child, ok := n.children[tail[0]] - if !ok { - return nil, fmt.Errorf("no entry %q", tail[0]) + if child, ok := n.children[seg]; ok { + return child, nil } - return walkPath(child, tail[1:], w) + return nil, fmt.Errorf("no entry %q", seg) case *template: - if w.level != nil { - if err := w.level(n, tail); err != nil { + if child, ok := n.fields[seg]; ok { + return child, nil + } + case *tableColumn: + return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", seg, n.t.header[n.i]) + } + return nil, fmt.Errorf("no field %q", seg) +} + +// tableRoute is how a path passes one table: the row it reads, by selector or +// drawn, and what it goes on into, a linked table it descends to among them. +type tableRoute struct { + sel string + draw bool + next node + rest []string + descends bool +} + +// route is how tail passes t, reached from another table where descended. The path +// reads a row wherever a segment follows or the table was reached from another; a +// table reached whole, with no selector, is left to a render's own draw. +func (t *table) route(tail []string, descended bool) (tableRoute, error) { + sel, tail, err := t.selector(tail) + if err != nil { + return tableRoute{}, err + } + column, child, err := t.step(tail) + if err != nil { + return tableRoute{}, err + } + // A selector further down pins this table by ancestry, so the walk draws only + // where none follows; drawing first could pick a row the selector is not inside. + r := tableRoute{sel: sel, draw: sel == "" && (descended || len(tail) > 0) && !hasSelector(tail)} + switch { + case len(tail) == 0 && sel == "" && !descended: + r.next = t + case len(tail) == 0: + r.next = t.pinnedRow + case child != nil: + r.next, r.rest, r.descends = child, tail[1:], true + default: + r.next, r.rest = column, tail[1:] + } + return r, nil +} + +// proveWalk proves a path resolves whichever way the draws go: every variant of a +// choice carries the rest of it, and is walked, a selector names a row inside the +// rows selected before it, and no level read carries a repeat or a drawGroup. It +// collects every leaf the path may render; level and tail name a refused level. +type proveWalk struct { + pins pinSet + level string + tail []string + leaves []node +} + +func (w *proveWalk) walk(n node, tail []string) (node, error) { + for descended := false; len(tail) > 0 || descended; { + var err error + switch x := n.(type) { + case *choice: + return w.walkEvery(x, tail) + case *table: + var r tableRoute + if r, err = x.route(tail, descended); err != nil { + return nil, err + } + if r.sel != "" { + if err := w.pins.selectRow(x, r.sel); err != nil { + return nil, err + } + } + n, tail, descended = r.next, r.rest, r.descends + continue + case *template: + if err := w.enter(x, tail); err != nil { return nil, err } } - child, ok := n.fields[tail[0]] - if !ok { - return nil, fmt.Errorf("no field %q", tail[0]) + if n, err = stepInto(n, tail[0]); err != nil { + return nil, err } - return walkPath(child, tail[1:], w) - case *choice: - return walkChoice(n, tail, w) - case *tableColumn: - return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", tail[0], n.t.header[n.i]) + tail = tail[1:] } - return nil, fmt.Errorf("no field %q", tail[0]) + w.leaves = append(w.leaves, n) + return n, nil } -func (w pathWalk) atLeaf(n node) error { - if w.leaf != nil { - return w.leaf(n) - } - return nil -} - -// drawPath draws the rows and variants a path reads, pinning them in pins. The path -// must be proved first, by a probe or by New's fences, so the walk cannot fail. -// head is what n is reached by, for the panic. -func drawPath(n node, tail []string, head string, pins *pinSet, draws *pathDraws) node { - leaf, err := walkPath(n, tail, pathWalk{mode: walkDraw, pins: pins, draws: draws}) - if err != nil { - panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(head, joinSegments(tail)), err)) - } - return leaf -} - -func walkChoice(c *choice, tail []string, w pathWalk) (node, error) { - switch w.mode { - case walkCover: - return c, w.atLeaf(c) - case walkDraw: - return walkPath(w.draws.variant(c, tail), tail, w) - } +func (w *proveWalk) walkEvery(c *choice, tail []string) (node, error) { if err := carriedByAll(c, tail); err != nil { return nil, err } - if w.mode == walkProbe { - return walkPath(c.items[0], tail, w) - } var last node for _, item := range c.items { var err error - if last, err = walkPath(item, tail, w); err != nil { + if last, err = w.walk(item, tail); err != nil { return nil, err } } return last, nil } -// walkTable descends tail from a table: a selector first, then a column or a -// linked table by name. The walk reads a row wherever a segment follows or the -// table was reached from another; a table reached whole, with no selector, is left -// to a render's own draw. -func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error) { - sel, tail, err := t.selector(tail) - if err != nil { - return nil, err +func (w *proveWalk) enter(t *template, rest []string) error { + name := join(w.level, strings.Join(w.tail[:len(w.tail)-len(rest)], ".")) + switch { + case t.repeat > 1: + return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", name) + case t.drawGroup != "": + return fmt.Errorf("the level %q carries a drawGroup, which a path reading into it cannot apply", name) } - column, child, err := t.step(tail) - if err != nil { - return nil, err - } - // A selector further down pins this table by ancestry, so the walk draws only - // where none follows; drawing first could pick a row the selector is not inside. - if w.mode != walkCover { - if err := readRow(w, t, sel, (descended || len(tail) > 0) && !hasSelector(tail)); err != nil { + return nil +} + +// coverWalk collects into what holding a path pins: every choice level it passes +// through, whole, and the leaf it renders. It stops at a choice and reads no row. +type coverWalk struct{ into map[node]bool } + +func (w *coverWalk) walk(n node, tail []string) (node, error) { + for descended := false; len(tail) > 0 || descended; { + var err error + switch x := n.(type) { + case *choice: + cover(x, w.into, false) + return x, nil + case *table: + var r tableRoute + if r, err = x.route(tail, descended); err != nil { + return nil, err + } + n, tail, descended = r.next, r.rest, r.descends + continue + } + if n, err = stepInto(n, tail[0]); err != nil { return nil, err } + tail = tail[1:] } - switch { - case len(tail) == 0 && sel == "" && !descended: - return walkPath(t, nil, w) - case len(tail) == 0: - return walkPath(t.pinnedRow, nil, w) - case child != nil: - return walkTable(child, tail[1:], w, true) + cover(n, w.into, false) + return n, nil +} + +// probeWalk proves a path as proveWalk does, walking one variant of a choice, so a +// path that resolves resolves whichever variants and rows are drawn. It draws +// nothing, marking in drawn the tables a draw would pin. +type probeWalk struct { + pins pinSet + drawn map[*table]bool +} + +func (w *probeWalk) walk(n node, tail []string) (node, error) { + for descended := false; len(tail) > 0 || descended; { + var err error + switch x := n.(type) { + case *choice: + if err := carriedByAll(x, tail); err != nil { + return nil, err + } + n = x.items[0] + continue + case *table: + var r tableRoute + if r, err = x.route(tail, descended); err != nil { + return nil, err + } + if err := w.readRow(x, r); err != nil { + return nil, err + } + n, tail, descended = r.next, r.rest, r.descends + continue + } + if n, err = stepInto(n, tail[0]); err != nil { + return nil, err + } + tail = tail[1:] } - return walkPath(column, tail[1:], w) + return n, nil +} + +func (w *probeWalk) readRow(t *table, r tableRoute) error { + if r.sel != "" { + return w.pins.selectRow(t, r.sel) + } + if !r.draw { + return nil + } + if w.drawn == nil { + w.drawn = map[*table]bool{} + } + for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { + w.drawn[t] = true + } + return nil +} + +// renderWalk draws the rows and variants a proved path reads, pinning them in pins. +// For a held read, held keeps the variant drawn at each level of a, so paths +// sharing a prefix share it. +type renderWalk struct { + s *session + pins *pinSet + held *hold + a *arm +} + +// drawPath walks w over a path proved first, by a probe or by New's fences, so the +// walk cannot fail. head is what n is reached by, for the panic. +func drawPath(n node, tail []string, head string, w *renderWalk) node { + leaf, err := w.walk(n, tail) + if err != nil { + panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(head, joinSegments(tail)), err)) + } + return leaf +} + +func (w *renderWalk) walk(n node, tail []string) (node, error) { + for descended := false; len(tail) > 0 || descended; { + var err error + switch x := n.(type) { + case *choice: + n = w.variant(x, tail) + continue + case *table: + var r tableRoute + if r, err = x.route(tail, descended); err != nil { + return nil, err + } + switch { + case r.sel != "": + if err := w.pins.selectRow(x, r.sel); err != nil { + return nil, err + } + case r.draw: + x.drawIn(w.s, w.pins) + } + n, tail, descended = r.next, r.rest, r.descends + continue + } + if n, err = stepInto(n, tail[0]); err != nil { + return nil, err + } + tail = tail[1:] + } + return n, nil +} + +// variant is the variant of c the walk continues into: the one held for this level +// of the read, drawn once, where the walk holds its draws; else one drawn afresh. +func (w *renderWalk) variant(c *choice, rest []string) node { + if w.held == nil { + return pick(w.s, c) + } + key := w.a.levels[len(w.a.tail)-len(rest)] + n, drew := w.held.variant[key] + if !drew { + n = resolveChoice(w.s, c) + if w.held.variant == nil { + w.held.variant = map[string]node{} + } + w.held.variant[key] = n + } + return n } // selector splits the selector a tail starts with from the rest of it. @@ -301,41 +429,6 @@ func (t *table) step(tail []string) (column node, child *table, err error) { return nil, child, nil } -// readRow pins the row a path reads of t: the one its selector names, or one drawn -// where the path reads into the table. -func readRow(w pathWalk, t *table, sel string, draw bool) error { - switch { - case sel != "": - return w.pins.selectRow(t, sel) - case !draw: - case w.mode == walkDraw: - t.drawIn(w.draws.s, w.pins) - case w.drawn != nil: - for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { - w.drawn[t] = true - } - } - return nil -} - -// variant is the variant of c the walk continues into: the one held for this level -// of the read, drawn once, where the walk holds its draws; else one drawn afresh. -func (d *pathDraws) variant(c *choice, rest []string) node { - if d.held == nil { - return pick(d.s, c) - } - key := d.a.levels[len(d.a.tail)-len(rest)] - n, drew := d.held.variant[key] - if !drew { - n = resolveChoice(d.s, c) - if d.held.variant == nil { - d.held.variant = map[string]node{} - } - d.held.variant[key] = n - } - return n -} - // carriedByAll is the choice rule a path that must resolve on every call obeys: // the rest of the tail must be one every variant carries. func carriedByAll(c *choice, rest []string) error { @@ -360,26 +453,9 @@ func unreachableInChoice(c *choice, want string) error { return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } -// checkPath proves a dotted tail resolves whichever way the draws go — a choice -// must carry the rest of the path in the set every variant shares, a selector must -// name one row inside the rows selected before it — and that no level a path reads -// carries a repeat or a drawGroup, which one draw of it could not apply. So a path -// that validates here resolves on every render, and a typo is a New-time error. +// checkPath proves a dotted tail as proveWalk does, so a path that validates here +// resolves on every render, and a typo is a New-time error. func checkPath(n node, tail []string, level string) error { - var pins pinSet - _, err := walkPath(n, tail, pathWalk{ - mode: walkEvery, - level: func(t *template, rest []string) error { - name := join(level, strings.Join(tail[:len(tail)-len(rest)], ".")) - switch { - case t.repeat > 1: - return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", name) - case t.drawGroup != "": - return fmt.Errorf("the level %q carries a drawGroup, which a path reading into it cannot apply", name) - } - return nil - }, - pins: &pins, - }) + _, err := (&proveWalk{level: level, tail: tail}).walk(n, tail) return err } diff --git a/reference.go b/reference.go index 3ab410a..9b4c041 100644 --- a/reference.go +++ b/reference.go @@ -187,9 +187,7 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t } // resolveCategory walks a dotted path through the folders to the category it -// names, returning that head, the node, and the tail left to read into it. A -// descent of its own rather than a walkPath: it walks folders only and returns -// where they end, not a leaf. +// names, returning that head, the node, and the tail left to read into it. func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) { var n node = &folder{children: root} i := 0 diff --git a/render.go b/render.go index b3cc7d6..bc25236 100644 --- a/render.go +++ b/render.go @@ -44,11 +44,10 @@ func (f *Generator) Fake(path string) (string, error) { // every call. func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // Walked once without drawing first, so a path that fails moves no seeded stream. - var probe pinSet - if _, err := walkPath(root, segments, pathWalk{mode: walkProbe, pins: &probe}); err != nil { + if _, err := (&probeWalk{}).walk(root, segments); err != nil { return nil, err } - return drawPath(root, segments, "", &sc.hold().pins, &pathDraws{s: s}), nil + return drawPath(root, segments, "", &renderWalk{s: s, pins: &sc.hold().pins}), nil } // render evaluates a compiled node to a string. compile validates every node up -- 2.52.0 From 8fdd33b3b4fb8081c23c1f49ff0b699cad0c772b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:20:38 +0200 Subject: [PATCH 155/966] Drop the walk-modes item from the plan --- todo.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/todo.md b/todo.md index 625b2d2..2876deb 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Split `walkPath`'s four modes, so a caller does not select a behaviour by which of - `level`, `leaf`, `pins`, `drawn` and `draws` it leaves nil. - Gather the family fence beside `drawWalk.check`: `checkFamilies`, `replayPairs`, `checkFamilyPair` and `tableRead.replay` sit in `family.go` and `pins.go`, and one rule spans seven files. -- 2.52.0 From 8c856b6510f18ae9849b2b0135e1dbcf46750a3c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:26:01 +0200 Subject: [PATCH 156/966] Want a column of a drawn row read without the probe allocating --- perf_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/perf_test.go b/perf_test.go index 7336cf6..83c477f 100644 --- a/perf_test.go +++ b/perf_test.go @@ -93,6 +93,7 @@ func TestNoTableAllocRegression(t *testing.T) { base float64 }{ {"a table drawn", "region", 2}, + {"a column of a drawn row", "region.name", 1}, {"a row selected", "region[12]", 2}, {"two columns of one draw", "x", 10}, } { -- 2.52.0 From 917c93564f27ccb5543f4ddabb2cb07f8db32ded Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:26:33 +0200 Subject: [PATCH 157/966] Want the path walks named after their entry points --- path_test.go | 10 +++++----- table_test.go | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/path_test.go b/path_test.go index 3cdfa4a..cbcb487 100644 --- a/path_test.go +++ b/path_test.go @@ -7,12 +7,12 @@ import ( "testing" ) -func TestWalkPathStopsAtAMissingSegment(t *testing.T) { +func TestPathCheckStopsAtAMissingSegment(t *testing.T) { n := compiled(t, `{"format":"{a}","a":{"format":"{b}","b":"leaf"}}`) if leaves := pathLeaves(n, []string{"a", "b"}); len(leaves) != 1 { t.Fatalf("pathLeaves(a.b) = %v, want the one leaf", leaves) } - w := &proveWalk{tail: []string{"a", "nope", "deeper"}} + w := &pathCheck{tail: []string{"a", "nope", "deeper"}} _, err := w.walk(n, w.tail) if err == nil || !strings.Contains(err.Error(), `no field "nope"`) { t.Errorf("walk(a.nope.deeper) = %v, want the missing segment named", err) @@ -22,7 +22,7 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) { } } -func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { +func TestPathCheckChoiceConsumesNoSegment(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) if leaves := pathLeaves(n, []string{"f"}); len(leaves) != 2 { t.Fatalf("pathLeaves through a choice = %v, want both variants' f", leaves) @@ -31,7 +31,7 @@ func TestWalkPathChoiceConsumesNoSegment(t *testing.T) { func TestWalkCoverStopsAtAChoice(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) - leaf, err := (&coverWalk{into: map[node]bool{}}).walk(n, []string{"f"}) + leaf, err := (&pathCover{into: map[node]bool{}}).walk(n, []string{"f"}) if err != nil || leaf != n { t.Fatalf("cover through a choice = %v, %v, want the choice itself", leaf, err) } @@ -43,7 +43,7 @@ func TestDrawPathPanicsOnAnUnprovedPath(t *testing.T) { t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the path and the missing field", r) } }() - drawPath(compiled(t, `"plain"`), []string{"f"}, "plain", &renderWalk{s: engine(1).rand, pins: &pinSet{}}) + drawPath(compiled(t, `"plain"`), []string{"f"}, "plain", &pathDraw{s: engine(1).rand, pins: &pinSet{}}) } func TestDeepDottedPath(t *testing.T) { diff --git a/table_test.go b/table_test.go index ed80b16..b83f0dd 100644 --- a/table_test.go +++ b/table_test.go @@ -1061,14 +1061,14 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { t.Fatal(err) } head := f.categories[segs[0]].(*table) - probe := &probeWalk{} + probe := &pathProbe{drawn: map[*table]bool{}} if _, err := probe.walk(head, segs[1:]); err != nil { t.Fatalf("%s: %v", path, err) } probed, drawn := probe.pins, probe.drawn for i := 0; i < 20; i++ { var pins pinSet - drawPath(head, segs[1:], segs[0], &renderWalk{s: f.rand, pins: &pins}) + drawPath(head, segs[1:], segs[0], &pathDraw{s: f.rand, pins: &pins}) want := map[*table]bool{} pins.each(func(tbl *table, _ int) { if _, selected := probed.pinned(tbl); !selected { -- 2.52.0 From 68033bd42f0b2ace2366e6bd1d1ea2533f4e0f2e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:26:33 +0200 Subject: [PATCH 158/966] Name the path walks after their entry points, keep the probe allocation-free, and record why the walks are four loops --- AGENTS.md | 1 + docs/decisions.md | 9 +++++++++ family.go | 2 +- graph.go | 2 +- hold.go | 2 +- holdfence.go | 2 +- path.go | 42 +++++++++++++++++++----------------------- render.go | 4 ++-- 8 files changed, 35 insertions(+), 29 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 691412c..439b0ce 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,3 +110,4 @@ In [docs/decisions.md](docs/decisions.md): - The US given names come from a mirror of the SSA file - `List` advertises direct descents only - Each entry point to `drawCheck.hasRead` says what its caller gets +- The four path walks are four loops diff --git a/docs/decisions.md b/docs/decisions.md index 31f3626..faf3af2 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -683,3 +683,12 @@ the direct chain is the one a reader can predict from the tables' parents. 2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on `readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, though `hasRead` states the same stop rule. Valid while the two differ in where they stop. + +## The four path walks are four loops + +2026-09-28, larv-review on systems-architect's finding; awaiting approval. Goals 2 and +11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over `stepInto` and +`table.route` themselves. One walk calling them through an interface or a type parameter +leaks the walker and the pins it holds, an allocation per reference read. Valid while Go's +escape analysis treats a call through an interface or a type parameter as leaking its +receiver. diff --git a/family.go b/family.go index af1ac51..0051deb 100644 --- a/family.go +++ b/family.go @@ -32,7 +32,7 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { if !isTable { return nil } - probe := &probeWalk{} + probe := &pathProbe{drawn: map[*table]bool{}} _, _ = probe.walk(t, a.tail) tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] diff --git a/graph.go b/graph.go index c128b0b..25c40ce 100644 --- a/graph.go +++ b/graph.go @@ -164,7 +164,7 @@ func renderEdges(n node) []renderEdge { // contributes every variant, since any of them may be the one drawn. checkPath has // already proved the tail resolves in every variant. func pathLeaves(n node, tail []string) []node { - w := &proveWalk{tail: tail} + w := &pathCheck{tail: tail} _, _ = w.walk(n, tail) return w.leaves } diff --git a/hold.go b/hold.go index 9c20384..ec51a05 100644 --- a/hold.go +++ b/hold.go @@ -44,7 +44,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if len(a.tail) > 0 { sc.pins = &d.pins } - r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &renderWalk{s: s, pins: &d.pins, held: d, a: &a}), sc) + r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, pins: &d.pins, held: d, a: &a}), sc) if d.value == nil { d.value = map[string]draw{} } diff --git a/holdfence.go b/holdfence.go index 27b35c4..cbbd04f 100644 --- a/holdfence.go +++ b/holdfence.go @@ -100,7 +100,7 @@ func operandReader(t *template, head string) string { // coverPath collects what holding one path pins: every choice level the path // passes through, whole, and the leaf it renders. func coverPath(n node, tail []string, into map[node]bool) { - _, _ = (&coverWalk{into: into}).walk(n, tail) + _, _ = (&pathCover{into: into}).walk(n, tail) } // cover collects a level and everything contained in it. A fixed string outside a diff --git a/path.go b/path.go index 87a4aee..02ab643 100644 --- a/path.go +++ b/path.go @@ -185,18 +185,18 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { return r, nil } -// proveWalk proves a path resolves whichever way the draws go: every variant of a +// pathCheck proves a path resolves whichever way the draws go: every variant of a // choice carries the rest of it, and is walked, a selector names a row inside the // rows selected before it, and no level read carries a repeat or a drawGroup. It // collects every leaf the path may render; level and tail name a refused level. -type proveWalk struct { +type pathCheck struct { pins pinSet level string tail []string leaves []node } -func (w *proveWalk) walk(n node, tail []string) (node, error) { +func (w *pathCheck) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error switch x := n.(type) { @@ -228,7 +228,7 @@ func (w *proveWalk) walk(n node, tail []string) (node, error) { return n, nil } -func (w *proveWalk) walkEvery(c *choice, tail []string) (node, error) { +func (w *pathCheck) walkEvery(c *choice, tail []string) (node, error) { if err := carriedByAll(c, tail); err != nil { return nil, err } @@ -242,7 +242,7 @@ func (w *proveWalk) walkEvery(c *choice, tail []string) (node, error) { return last, nil } -func (w *proveWalk) enter(t *template, rest []string) error { +func (w *pathCheck) enter(t *template, rest []string) error { name := join(w.level, strings.Join(w.tail[:len(w.tail)-len(rest)], ".")) switch { case t.repeat > 1: @@ -253,11 +253,10 @@ func (w *proveWalk) enter(t *template, rest []string) error { return nil } -// coverWalk collects into what holding a path pins: every choice level it passes -// through, whole, and the leaf it renders. It stops at a choice and reads no row. -type coverWalk struct{ into map[node]bool } +// pathCover is coverPath's walk: it stops at a choice and reads no row. +type pathCover struct{ into map[node]bool } -func (w *coverWalk) walk(n node, tail []string) (node, error) { +func (w *pathCover) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error switch x := n.(type) { @@ -281,15 +280,15 @@ func (w *coverWalk) walk(n node, tail []string) (node, error) { return n, nil } -// probeWalk proves a path as proveWalk does, walking one variant of a choice, so a +// pathProbe proves a path resolves, walking one variant of a choice, so a // path that resolves resolves whichever variants and rows are drawn. It draws // nothing, marking in drawn the tables a draw would pin. -type probeWalk struct { +type pathProbe struct { pins pinSet drawn map[*table]bool } -func (w *probeWalk) walk(n node, tail []string) (node, error) { +func (w *pathProbe) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error switch x := n.(type) { @@ -318,26 +317,23 @@ func (w *probeWalk) walk(n node, tail []string) (node, error) { return n, nil } -func (w *probeWalk) readRow(t *table, r tableRoute) error { +func (w *pathProbe) readRow(t *table, r tableRoute) error { if r.sel != "" { return w.pins.selectRow(t, r.sel) } if !r.draw { return nil } - if w.drawn == nil { - w.drawn = map[*table]bool{} - } for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { w.drawn[t] = true } return nil } -// renderWalk draws the rows and variants a proved path reads, pinning them in pins. +// pathDraw draws the rows and variants a proved path reads, pinning them in pins. // For a held read, held keeps the variant drawn at each level of a, so paths // sharing a prefix share it. -type renderWalk struct { +type pathDraw struct { s *session pins *pinSet held *hold @@ -346,7 +342,7 @@ type renderWalk struct { // drawPath walks w over a path proved first, by a probe or by New's fences, so the // walk cannot fail. head is what n is reached by, for the panic. -func drawPath(n node, tail []string, head string, w *renderWalk) node { +func drawPath(n node, tail []string, head string, w *pathDraw) node { leaf, err := w.walk(n, tail) if err != nil { panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(head, joinSegments(tail)), err)) @@ -354,7 +350,7 @@ func drawPath(n node, tail []string, head string, w *renderWalk) node { return leaf } -func (w *renderWalk) walk(n node, tail []string) (node, error) { +func (w *pathDraw) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error switch x := n.(type) { @@ -387,7 +383,7 @@ func (w *renderWalk) walk(n node, tail []string) (node, error) { // variant is the variant of c the walk continues into: the one held for this level // of the read, drawn once, where the walk holds its draws; else one drawn afresh. -func (w *renderWalk) variant(c *choice, rest []string) node { +func (w *pathDraw) variant(c *choice, rest []string) node { if w.held == nil { return pick(w.s, c) } @@ -453,9 +449,9 @@ func unreachableInChoice(c *choice, want string) error { return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } -// checkPath proves a dotted tail as proveWalk does, so a path that validates here +// checkPath proves a dotted tail as pathCheck does, so a path that validates here // resolves on every render, and a typo is a New-time error. func checkPath(n node, tail []string, level string) error { - _, err := (&proveWalk{level: level, tail: tail}).walk(n, tail) + _, err := (&pathCheck{level: level, tail: tail}).walk(n, tail) return err } diff --git a/render.go b/render.go index bc25236..d5a1b20 100644 --- a/render.go +++ b/render.go @@ -44,10 +44,10 @@ func (f *Generator) Fake(path string) (string, error) { // every call. func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // Walked once without drawing first, so a path that fails moves no seeded stream. - if _, err := (&probeWalk{}).walk(root, segments); err != nil { + if _, err := (&pathProbe{drawn: map[*table]bool{}}).walk(root, segments); err != nil { return nil, err } - return drawPath(root, segments, "", &renderWalk{s: s, pins: &sc.hold().pins}), nil + return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.hold().pins}), nil } // render evaluates a compiled node to a string. compile validates every node up -- 2.52.0 From e66a5f9b4b28bbdab3fb4c412981617a9e12d1b0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:28:25 +0200 Subject: [PATCH 159/966] Name the cover test after its walk --- path_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/path_test.go b/path_test.go index cbcb487..80fe041 100644 --- a/path_test.go +++ b/path_test.go @@ -29,7 +29,7 @@ func TestPathCheckChoiceConsumesNoSegment(t *testing.T) { } } -func TestWalkCoverStopsAtAChoice(t *testing.T) { +func TestPathCoverStopsAtAChoice(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) leaf, err := (&pathCover{into: map[node]bool{}}).walk(n, []string{"f"}) if err != nil || leaf != n { -- 2.52.0 From 1073232ca7b95148bfbaec814c12e3025449187e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:28:25 +0200 Subject: [PATCH 160/966] Record why the probe has no entry function --- docs/decisions.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index faf3af2..a5b1d4f 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -688,7 +688,8 @@ though `hasRead` states the same stop rule. Valid while the two differ in where 2026-09-28, larv-review on systems-architect's finding; awaiting approval. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over `stepInto` and -`table.route` themselves. One walk calling them through an interface or a type parameter +`table.route` themselves, and `descend` and `tableReadOf` build their probe in place, so +its `drawn` map stays on the stack. One walk calling them through an interface or a type parameter leaks the walker and the pins it holds, an allocation per reference read. Valid while Go's escape analysis treats a call through an interface or a type parameter as leaking its receiver. -- 2.52.0 From d95045f4acf41659c40c8a2ee4223eed45cf263a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:35:52 +0200 Subject: [PATCH 161/966] Want a selector on a repeating template refused as a selector, and the probe held to a descent ending on a linked table --- path_test.go | 2 +- table_test.go | 13 ++++++++----- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/path_test.go b/path_test.go index 80fe041..cd11e10 100644 --- a/path_test.go +++ b/path_test.go @@ -13,7 +13,7 @@ func TestPathCheckStopsAtAMissingSegment(t *testing.T) { t.Fatalf("pathLeaves(a.b) = %v, want the one leaf", leaves) } w := &pathCheck{tail: []string{"a", "nope", "deeper"}} - _, err := w.walk(n, w.tail) + _, err := w.run(n) if err == nil || !strings.Contains(err.Error(), `no field "nope"`) { t.Errorf("walk(a.nope.deeper) = %v, want the missing segment named", err) } diff --git a/table_test.go b/table_test.go index b83f0dd..32f869b 100644 --- a/table_test.go +++ b/table_test.go @@ -352,12 +352,13 @@ func TestTableSelectorInAReference(t *testing.T) { } } for name, c := range map[string]struct{ json, want string }{ - "unknown row": {`"{/region[99].name}"`, `"99"`}, - "ambiguous name": {`"{/locality[Sandby].name}"`, "L7, L8"}, - "not inside": {`"{/region[12].municipality[0180].name}"`, "not inside"}, - "selector on template": {`"{/x[1].a}"`, "not a table"}, + "unknown row": {`"{/region[99].name}"`, `"99"`}, + "ambiguous name": {`"{/locality[Sandby].name}"`, "L7, L8"}, + "not inside": {`"{/region[12].municipality[0180].name}"`, "not inside"}, + "selector on template": {`"{/x[1].a}"`, "not a table"}, + "selector on repeating template": {`"{/rep[1]}"`, "not a table"}, } { - _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, with(files, map[string]string{"bad.json": c.json})))) + _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, with(files, map[string]string{"bad.json": c.json, "rep.json": `{"format":"x","repeat":2}`})))) if err == nil || !strings.Contains(err.Error(), c.want) { t.Errorf("%s: New = %v, want an error mentioning %q", name, err, c.want) } @@ -1050,8 +1051,10 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { "municipality[1281].locality.name", "municipality.locality[L4].name", "region", + "region.municipality", "region.municipality.name", "region.locality.name", + "region[12].locality", "region[12].locality.code", "region[12].municipality.locality.code", "region[12].name", -- 2.52.0 From 060fb42fee83f9443e738e5fde1a407aa3090bf7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:36:33 +0200 Subject: [PATCH 162/966] Refuse a selector on a template before its level fence, give pathCheck one entry over its own tail, and scope the probe decision to descend --- docs/decisions.md | 10 +++++----- graph.go | 2 +- path.go | 7 ++++++- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index a5b1d4f..5575527 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -688,8 +688,8 @@ though `hasRead` states the same stop rule. Valid while the two differ in where 2026-09-28, larv-review on systems-architect's finding; awaiting approval. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over `stepInto` and -`table.route` themselves, and `descend` and `tableReadOf` build their probe in place, so -its `drawn` map stays on the stack. One walk calling them through an interface or a type parameter -leaks the walker and the pins it holds, an allocation per reference read. Valid while Go's -escape analysis treats a call through an interface or a type parameter as leaking its -receiver. +`table.route` themselves, and `descend` builds its probe in place, so the probe's +`drawn` map stays on the stack. One walk calling them through an interface or a type +parameter leaks the walker and the pins it holds, an allocation per reference read. +Valid while Go's escape analysis treats a call through an interface or a type parameter +as leaking its receiver. diff --git a/graph.go b/graph.go index 25c40ce..bc7856d 100644 --- a/graph.go +++ b/graph.go @@ -165,7 +165,7 @@ func renderEdges(n node) []renderEdge { // already proved the tail resolves in every variant. func pathLeaves(n node, tail []string) []node { w := &pathCheck{tail: tail} - _, _ = w.walk(n, tail) + _, _ = w.run(n) return w.leaves } diff --git a/path.go b/path.go index 02ab643..ee3d118 100644 --- a/path.go +++ b/path.go @@ -196,6 +196,8 @@ type pathCheck struct { leaves []node } +func (w *pathCheck) run(n node) (node, error) { return w.walk(n, w.tail) } + func (w *pathCheck) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error @@ -215,6 +217,9 @@ func (w *pathCheck) walk(n node, tail []string) (node, error) { n, tail, descended = r.next, r.rest, r.descends continue case *template: + if isSelector(tail[0]) { + break + } if err := w.enter(x, tail); err != nil { return nil, err } @@ -452,6 +457,6 @@ func unreachableInChoice(c *choice, want string) error { // checkPath proves a dotted tail as pathCheck does, so a path that validates here // resolves on every render, and a typo is a New-time error. func checkPath(n node, tail []string, level string) error { - _, err := (&pathCheck{level: level, tail: tail}).walk(n, tail) + _, err := (&pathCheck{level: level, tail: tail}).run(n) return err } -- 2.52.0 From 79cacb28232c5cc1a3c475b5d236b2e3b2a3a5eb Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:38:59 +0200 Subject: [PATCH 163/966] Record that the probe's drawn map is never nil --- docs/decisions.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index 5575527..fe5b4e6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -689,7 +689,8 @@ though `hasRead` states the same stop rule. Valid while the two differ in where 2026-09-28, larv-review on systems-architect's finding; awaiting approval. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over `stepInto` and `table.route` themselves, and `descend` builds its probe in place, so the probe's -`drawn` map stays on the stack. One walk calling them through an interface or a type +`drawn` map stays on the stack; its caller always builds that map, since no walker +picks a behaviour by a nil field. One walk calling them through an interface or a type parameter leaks the walker and the pins it holds, an allocation per reference read. Valid while Go's escape analysis treats a call through an interface or a type parameter as leaking its receiver. -- 2.52.0 From e9bb89f044a01d7590864187dee03c09df48c7a6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 00:43:20 +0200 Subject: [PATCH 164/966] Correct and trim the comments around the path walks --- README.md | 2 +- docs/decisions.md | 6 +++--- family.go | 4 ++-- hold.go | 3 +-- holdfence.go | 4 ++-- node.go | 2 +- path.go | 24 ++++++++++-------------- render.go | 10 +++------- 8 files changed, 23 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index b021c71..d39b9aa 100644 --- a/README.md +++ b/README.md @@ -870,7 +870,7 @@ doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation table.go tables: the rows TSV, its options and links, row selection and draws -path.go the dotted-path walk with its selectors, and proving a path resolves +path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake and the recursive renderer (choices, format strings, expansions) record.go records: Record, the JSON/CSV/SQL serializers, and their entry points struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype diff --git a/docs/decisions.md b/docs/decisions.md index fe5b4e6..867a86d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -325,7 +325,7 @@ Go ties a package to one directory, so folders would split the API into packages ## The performance gate asserts allocations, not wall-clock time -`AllocsPerRun` is deterministic across machines, so a ±10% ceiling does not flake under +`AllocsPerRun` is deterministic across machines, so a 10% ceiling does not flake under CI load, while time varies with the machine and its neighbours. A rendering slowdown almost always costs an allocation too (a lost pre-size, a per-item map, an extra copy). The benchmark suite (the README's Development) reports time for a human, not as a pass/fail gate. @@ -689,8 +689,8 @@ though `hasRead` states the same stop rule. Valid while the two differ in where 2026-09-28, larv-review on systems-architect's finding; awaiting approval. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over `stepInto` and `table.route` themselves, and `descend` builds its probe in place, so the probe's -`drawn` map stays on the stack; its caller always builds that map, since no walker -picks a behaviour by a nil field. One walk calling them through an interface or a type +`drawn` map stays on the stack; its caller always builds that map, so `readRow` never +tests it for nil. One walk calling them through an interface or a type parameter leaks the walker and the pins it holds, an allocation per reference read. Valid while Go's escape analysis treats a call through an interface or a type parameter as leaking its receiver. diff --git a/family.go b/family.go index 0051deb..f941f8a 100644 --- a/family.go +++ b/family.go @@ -24,8 +24,8 @@ type tableSel struct { spelling string } -// tableReadOf replays a reference's selectors through the walk a render uses, so -// two paths pinning one row by different routes compare equal. checkPath proved +// tableReadOf replays a reference path through pathProbe, so two paths pinning one +// row by different routes compare equal. checkPath proved // each selector names a row. func tableReadOf(head node, a arm, leaf node) *tableRead { t, isTable := head.(*table) diff --git a/hold.go b/hold.go index ec51a05..81ee2a0 100644 --- a/hold.go +++ b/hold.go @@ -25,8 +25,7 @@ type draw struct { // token addresses by dotted path, or a field an operand reads — is drawn once and // kept, so {place.postal-code} and {place.locality} read one row, either read twice // gives one value, and a shown operand is the operand computed. Every other name is -// drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and -// linkRefs prove every step, so the walk cannot fail. +// drawn afresh, so {word} {word} still draws twice. func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { if s.trace != nil { traceRead(s.trace, t, sc, a) diff --git a/holdfence.go b/holdfence.go index cbbd04f..b839621 100644 --- a/holdfence.go +++ b/holdfence.go @@ -97,8 +97,8 @@ func operandReader(t *template, head string) string { return "" } -// coverPath collects what holding one path pins: every choice level the path -// passes through, whole, and the leaf it renders. +// coverPath collects what holding one path pins: the first choice level it passes, +// whole, else the leaf it renders; it reads no row. func coverPath(n node, tail []string, into map[node]bool) { _, _ = (&pathCover{into: into}).walk(n, tail) } diff --git a/node.go b/node.go index 62c791d..b6c274a 100644 --- a/node.go +++ b/node.go @@ -23,7 +23,7 @@ func (*folder) isNode() {} // choice picks one of its items. cum holds cumulative weights for a weighted // pick; when nil the choice is uniform and selection is O(1). shared is the set of -// relative dot paths every item can address, so descend and List both read the one +// relative dot paths every item can address, so carriedByAll and List both read the one // answer to what a path may reach through this choice. type choice struct { items []node diff --git a/path.go b/path.go index ee3d118..4966a32 100644 --- a/path.go +++ b/path.go @@ -125,8 +125,7 @@ func nameSegments(segs []string) []string { return out } -// stepInto is what seg names under n: a folder's entry or a template's field. A -// missing segment is an error, so no walk reaches past what the data holds. +// stepInto is what seg names under n: a folder's entry or a template's field. func stepInto(n node, seg string) (node, error) { if isSelector(seg) { return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(seg)) @@ -148,7 +147,7 @@ func stepInto(n node, seg string) (node, error) { } // tableRoute is how a path passes one table: the row it reads, by selector or -// drawn, and what it goes on into, a linked table it descends to among them. +// drawn, what it goes on into, and whether that is a linked table. type tableRoute struct { sel string draw bool @@ -188,7 +187,7 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { // pathCheck proves a path resolves whichever way the draws go: every variant of a // choice carries the rest of it, and is walked, a selector names a row inside the // rows selected before it, and no level read carries a repeat or a drawGroup. It -// collects every leaf the path may render; level and tail name a refused level. +// collects every leaf the path may render; level names the head in its errors. type pathCheck struct { pins pinSet level string @@ -258,7 +257,6 @@ func (w *pathCheck) enter(t *template, rest []string) error { return nil } -// pathCover is coverPath's walk: it stops at a choice and reads no row. type pathCover struct{ into map[node]bool } func (w *pathCover) walk(n node, tail []string) (node, error) { @@ -285,9 +283,8 @@ func (w *pathCover) walk(n node, tail []string) (node, error) { return n, nil } -// pathProbe proves a path resolves, walking one variant of a choice, so a -// path that resolves resolves whichever variants and rows are drawn. It draws -// nothing, marking in drawn the tables a draw would pin. +// pathProbe proves a path resolves without drawing: carriedByAll lets one variant +// of a choice stand for all. It marks in drawn the tables a draw would pin. type pathProbe struct { pins pinSet drawn map[*table]bool @@ -345,8 +342,8 @@ type pathDraw struct { a *arm } -// drawPath walks w over a path proved first, by a probe or by New's fences, so the -// walk cannot fail. head is what n is reached by, for the panic. +// drawPath walks w over a path proved first, by a probe or at load, so the walk +// cannot fail. head names n in the panic. func drawPath(n node, tail []string, head string, w *pathDraw) node { leaf, err := w.walk(n, tail) if err != nil { @@ -440,8 +437,7 @@ func carriedByAll(c *choice, rest []string) error { } // unreachableInChoice reports that a path cannot step through this choice, listing -// what every variant does carry. It reads the precomputed set, so a failing path -// costs no more than a rendering one. +// what every variant does carry. func unreachableInChoice(c *choice, want string) error { if len(c.shared) == 0 { return fmt.Errorf("no variant of this %d-way choice carries %q", len(c.items), want) @@ -454,8 +450,8 @@ func unreachableInChoice(c *choice, want string) error { return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } -// checkPath proves a dotted tail as pathCheck does, so a path that validates here -// resolves on every render, and a typo is a New-time error. +// checkPath runs pathCheck over a path read at load; level names the head in its +// errors. func checkPath(n node, tail []string, level string) error { _, err := (&pathCheck{level: level, tail: tail}).run(n) return err diff --git a/render.go b/render.go index d5a1b20..c7b100b 100644 --- a/render.go +++ b/render.go @@ -36,14 +36,10 @@ func (f *Generator) Fake(path string) (string, error) { return render(f.rand, n, sc), nil } -// descend walks named fields to the node a path names, pinning the table rows it -// selects or draws in sc. It is the one render-side step that can fail, because the -// path comes from the caller and may name a field or a row that does not exist. A -// choice consumes no segment, so the rest of the path must be one every variant -// carries before a variant is picked — a path that resolves at all resolves on -// every call. +// descend walks a caller's path to the node it names, pinning in sc the rows it +// selects or draws. func descend(s *session, root node, segments []string, sc renderScope) (node, error) { - // Walked once without drawing first, so a path that fails moves no seeded stream. + // docs/decisions.md#a-path-is-walked-once-without-drawing-before-it-is-walked-for-real if _, err := (&pathProbe{drawn: map[*table]bool{}}).walk(root, segments); err != nil { return nil, err } -- 2.52.0 From 2d506e79eba6344ff59fbb0066bcb7e8dbbd6879 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:31:26 +0200 Subject: [PATCH 165/966] Approve the four-loops decision --- docs/decisions.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 867a86d..1f43aa4 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -686,11 +686,11 @@ though `hasRead` states the same stop rule. Valid while the two differ in where ## The four path walks are four loops -2026-09-28, larv-review on systems-architect's finding; awaiting approval. Goals 2 and -11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over `stepInto` and -`table.route` themselves, and `descend` builds its probe in place, so the probe's -`drawn` map stays on the stack; its caller always builds that map, so `readRow` never -tests it for nil. One walk calling them through an interface or a type -parameter leaks the walker and the pins it holds, an allocation per reference read. -Valid while Go's escape analysis treats a call through an interface or a type parameter -as leaking its receiver. +2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. +Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over +`stepInto` and `table.route` themselves, and `descend` builds its probe in place, so the +probe's `drawn` map stays on the stack; its caller always builds that map, so `readRow` +never tests it for nil. One walk calling them through an interface or a type parameter +leaks the walker and the pins it holds, an allocation per reference read. Valid while +Go's escape analysis treats a call through an interface or a type parameter as leaking +its receiver. -- 2.52.0 From 4a52085e8f4039b34bef78237d82f90d36b0007b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:37:32 +0200 Subject: [PATCH 166/966] Gather the family fence beside drawWalk.check in drawfence.go --- README.md | 3 +- drawfence.go | 200 +++++++++++++++++++++++++++++++++++++++++++++ family.go | 226 --------------------------------------------------- path.go | 12 +++ table.go | 8 ++ todo.md | 5 +- 6 files changed, 222 insertions(+), 232 deletions(-) delete mode 100644 family.go diff --git a/README.md b/README.md index d39b9aa..e9e5301 100644 --- a/README.md +++ b/README.md @@ -879,8 +879,7 @@ format.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per held name, and reading a field through it holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through -drawfence.go the load fences over reference draws: a drawGroup's, a render's and a record's columns, and the walk over what a render reads -family.go a family of linked tables: what a reference path reads of one, and the fences over paths into it +drawfence.go the load fences over reference draws: a drawGroup's, a render's, a record's columns and a table family's, and the walk over what a render reads pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/drawfence.go b/drawfence.go index f8c5dd7..f54da29 100644 --- a/drawfence.go +++ b/drawfence.go @@ -309,6 +309,90 @@ func (w *drawWalk) check() error { return checkFamilies(w.reads) } +// checkFamilies refuses reads of one table family in one draw group that cannot +// read one consistent draw: a table rendered whole beside a path into the family, +// a table one read draws that another pins, and two reads pinning different rows. +// The reads come sorted by group and path, so which pair is reported does not vary. +func checkFamilies(reads []pathRead) error { + for i, r := range reads { + if r.tr == nil { + continue + } + for _, o := range reads[:i] { + if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.family() != r.tr.headTable.family() { + continue + } + if err := checkFamilyPair(o, r); err != nil { + return err + } + } + } + return replayPairs(reads) +} + +// checkFamilyPair refuses a table read whole beside a path into its family, and a +// table one read draws that the other pins, since which token renders first would +// then decide the row. +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held +func checkFamilyPair(a, b pathRead) error { + for _, pair := range [][2]pathRead{{a, b}, {b, a}} { + x, y := pair[0], pair[1] + if len(x.a.tail) == 0 && len(y.a.tail) > 0 { + return overlapError(x.at.route, x.a.spelling, y) + } + if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { + if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { + tail := x.a.tail + if s.t != x.tr.headTable { + tail = append([]string{x.tr.headTable.category}, tail...) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", + x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", + x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling)) + } + } + return nil +} + +// replayPairs replays every two reads that can render together into one draws, the earlier +// read first. Pairs find every conflict a full replay would: clash judges a row against one +// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry +// its rows' ancestors, so the pair of those two reads clashes the same way. +func replayPairs(reads []pathRead) error { + for i, r := range reads { + if r.tr == nil { + continue + } + for _, o := range reads[i+1:] { + if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { + continue + } + var d pinSet + if err := r.tr.replay(&d); err != nil { + return conflict(r, err) + } + if err := o.tr.replay(&d); err != nil { + return conflict(o, err) + } + } + } + return nil +} + +func conflict(r pathRead, err error) error { + return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) +} + +// alternatives reports whether two reads never render together: one read renders one row of a +// table, so reads under different rows the walks pinned, or under different rows of one whole +// draw, never meet. +// docs/decisions.md#the-rows-of-a-table-are-alternatives +func alternatives(a, b drawAt) bool { + return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) +} + func overlapError(route drawRoute, ref string, into pathRead) error { return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) } @@ -321,3 +405,119 @@ func (r drawRoute) spelled(ref string) string { } return fmt.Sprintf("%s with {%s}", r.spelling, ref) } + +// tableRead is what a reference path reads of a table family: the table its head +// names, the rows its selectors pin, the tables it draws — those it walks with no +// row pinned, and their unpinned ancestors — each selector's spelling, and whether +// it lands on a `tableRow`. +type tableRead struct { + headTable *table + pins pinSet + drawn map[*table]bool + sels []tableSel + landsRow bool +} + +// tableSel is one selector on the way: the table it selects a row of, and the path +// as written up to and including it. +type tableSel struct { + t *table + spelling string +} + +// tableReadOf replays a reference path through pathProbe, so two paths pinning one +// row by different routes compare equal. checkPath proved +// each selector names a row. +func tableReadOf(head node, a arm, leaf node) *tableRead { + t, isTable := head.(*table) + if !isTable { + return nil + } + probe := &pathProbe{drawn: map[*table]bool{}} + _, _ = probe.walk(t, a.tail) + tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} + written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] + cur := t + for i, seg := range a.tail { + switch d := cur.descendant(seg); { + case isSelector(seg): + tr.sels = append(tr.sels, tableSel{cur, written + joinSegments(a.tail[:i+1])}) + case d != nil: + cur = d + } + } + _, tr.landsRow = leaf.(*tableRow) + return tr +} + +// selected is the selector in r on t, or on the nearest ancestor of t it selects. +func (r *tableRead) selected(t *table) (tableSel, bool) { + for ; t != nil; t = t.parentT { + for _, s := range r.sels { + if s.t == t { + return s, true + } + } + } + return tableSel{}, false +} + +// drawnOf is a table the read draws that pins holds a row of, if any. +func (r *tableRead) drawnOf(pins *pinSet) *table { + var found *table + pins.each(func(t *table, _ int) { + if found == nil && r.drawn[t] { + found = t + } + }) + return found +} + +// replay pins the read's rows into d, where they agree with the rows pinned before. +func (r *tableRead) replay(d *pinSet) error { + var err error + r.pins.each(func(t *table, row int) { + if err == nil { + err = d.pinRow(t, row) + } + }) + return err +} + +// checkOwnFamily refuses a table's format or cell that reads, however many templates +// away and through a repeat or a draw group too, a table of its own family: a whole +// read draws its row without pinning it, so the family would draw apart from the row +// it renders, whichever draws the reaching template holds. +// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route +func checkOwnFamily(t *template) error { + own := t.table + if own == nil { + own = t.cellOf + } + if own == nil { + return nil + } + seen := map[node]bool{} + var find func(n node) (renderEdge, *table, bool) + find = func(n node) (renderEdge, *table, bool) { + if seen[n] { + return renderEdge{}, nil, false + } + seen[n] = true + for _, e := range renderEdges(n) { + if a, isRef := refRead(n, e.label); isRef { + if head, isTable := n.(*template).head(a.key).(*table); isTable && head.family() == own.family() { + return e, head, true + } + } + if e, head, found := find(e.to); found { + return e, head, true + } + } + return renderEdge{}, nil, false + } + if e, head, found := find(t); found { + return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + } + return nil +} diff --git a/family.go b/family.go deleted file mode 100644 index f941f8a..0000000 --- a/family.go +++ /dev/null @@ -1,226 +0,0 @@ -package fejkdata - -import ( - "fmt" - "strings" -) - -// tableRead is what a reference path reads of a table family: the table its head -// names, the rows its selectors pin, the tables it draws — those it walks with no -// row pinned, and their unpinned ancestors — each selector's spelling, and whether -// it lands on a `tableRow`. -type tableRead struct { - headTable *table - pins pinSet - drawn map[*table]bool - sels []tableSel - landsRow bool -} - -// tableSel is one selector on the way: the table it selects a row of, and the path -// as written up to and including it. -type tableSel struct { - t *table - spelling string -} - -// tableReadOf replays a reference path through pathProbe, so two paths pinning one -// row by different routes compare equal. checkPath proved -// each selector names a row. -func tableReadOf(head node, a arm, leaf node) *tableRead { - t, isTable := head.(*table) - if !isTable { - return nil - } - probe := &pathProbe{drawn: map[*table]bool{}} - _, _ = probe.walk(t, a.tail) - tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} - written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] - cur := t - for i, seg := range a.tail { - switch d := cur.descendant(seg); { - case isSelector(seg): - tr.sels = append(tr.sels, tableSel{cur, written + joinSegments(a.tail[:i+1])}) - case d != nil: - cur = d - } - } - _, tr.landsRow = leaf.(*tableRow) - return tr -} - -// joinSegments spells segments as a path: a selector attaches to the name before it. -func joinSegments(segs []string) string { - var b strings.Builder - for _, s := range segs { - if b.Len() > 0 && !isSelector(s) { - b.WriteByte('.') - } - b.WriteString(s) - } - return b.String() -} - -// family is the table a chain of parents ends at. -func (t *table) family() *table { - for t.parentT != nil { - t = t.parentT - } - return t -} - -// selected is the selector in r on t, or on the nearest ancestor of t it selects. -func (r *tableRead) selected(t *table) (tableSel, bool) { - for ; t != nil; t = t.parentT { - for _, s := range r.sels { - if s.t == t { - return s, true - } - } - } - return tableSel{}, false -} - -// checkFamilies refuses reads of one table family in one draw group that cannot -// read one consistent draw: a table rendered whole beside a path into the family, -// a table one read draws that another pins, and two reads pinning different rows. -// The reads come sorted by group and path, so which pair is reported does not vary. -func checkFamilies(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[:i] { - if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.family() != r.tr.headTable.family() { - continue - } - if err := checkFamilyPair(o, r); err != nil { - return err - } - } - } - return replayPairs(reads) -} - -// replayPairs replays every two reads that can render together into one draws, the earlier -// read first. Pairs find every conflict a full replay would: clash judges a row against one -// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry -// its rows' ancestors, so the pair of those two reads clashes the same way. -func replayPairs(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[i+1:] { - if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { - continue - } - var d pinSet - if err := r.tr.replay(&d); err != nil { - return conflict(r, err) - } - if err := o.tr.replay(&d); err != nil { - return conflict(o, err) - } - } - } - return nil -} - -func conflict(r pathRead, err error) error { - return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) -} - -// replay pins the read's rows into d, where they agree with the rows pinned before. -func (r *tableRead) replay(d *pinSet) error { - var err error - r.pins.each(func(t *table, row int) { - if err == nil { - err = d.pinRow(t, row) - } - }) - return err -} - -// checkFamilyPair refuses a table read whole beside a path into its family, and a -// table one read draws that the other pins, since which token renders first would -// then decide the row. -// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkFamilyPair(a, b pathRead) error { - for _, pair := range [][2]pathRead{{a, b}, {b, a}} { - x, y := pair[0], pair[1] - if len(x.a.tail) == 0 && len(y.a.tail) > 0 { - return overlapError(x.at.route, x.a.spelling, y) - } - if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { - if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { - tail := x.a.tail - if s.t != x.tr.headTable { - tail = append([]string{x.tr.headTable.category}, tail...) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling)) - } - } - return nil -} - -// drawnOf is a table the read draws that pins holds a row of, if any. -func (r *tableRead) drawnOf(pins *pinSet) *table { - var found *table - pins.each(func(t *table, _ int) { - if found == nil && r.drawn[t] { - found = t - } - }) - return found -} - -// checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a whole -// read draws its row without pinning it, so the family would draw apart from the row -// it renders, whichever draws the reaching template holds. -// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route -func checkOwnFamily(t *template) error { - own := t.table - if own == nil { - own = t.cellOf - } - if own == nil { - return nil - } - seen := map[node]bool{} - var find func(n node) (renderEdge, *table, bool) - find = func(n node) (renderEdge, *table, bool) { - if seen[n] { - return renderEdge{}, nil, false - } - seen[n] = true - for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).head(a.key).(*table); isTable && head.family() == own.family() { - return e, head, true - } - } - if e, head, found := find(e.to); found { - return e, head, true - } - } - return renderEdge{}, nil, false - } - if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) - } - return nil -} - -// alternatives reports whether two reads never render together: one read renders one row of a -// table, so reads under different rows the walks pinned, or under different rows of one whole -// draw, never meet. -// docs/decisions.md#the-rows-of-a-table-are-alternatives -func alternatives(a, b drawAt) bool { - return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) -} diff --git a/path.go b/path.go index 4966a32..17f9640 100644 --- a/path.go +++ b/path.go @@ -456,3 +456,15 @@ func checkPath(n node, tail []string, level string) error { _, err := (&pathCheck{level: level, tail: tail}).run(n) return err } + +// joinSegments spells segments as a path: a selector attaches to the name before it. +func joinSegments(segs []string) string { + var b strings.Builder + for _, s := range segs { + if b.Len() > 0 && !isSelector(s) { + b.WriteByte('.') + } + b.WriteString(s) + } + return b.String() +} diff --git a/table.go b/table.go index d577ba0..f11c4d9 100644 --- a/table.go +++ b/table.go @@ -613,3 +613,11 @@ func (t *table) selectorSpelling(r int) string { } return fmt.Sprintf("%s line %d", t.file, r+2) } + +// family is the table a chain of parents ends at. +func (t *table) family() *table { + for t.parentT != nil { + t = t.parentT + } + return t +} diff --git a/todo.md b/todo.md index 2876deb..3b58261 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Gather the family fence beside `drawWalk.check`: `checkFamilies`, `replayPairs`, - `checkFamilyPair` and `tableRead.replay` sit in `family.go` and `pins.go`, and one - rule spans seven files. - Compile every format in one phase: a template holding a reference has nil `ops` until `linkTemplateRefs`, so a check reading `ops` before link skips it silently. - Run `New`, `NewTemplate` and `FakeStruct` through one bind pipeline that names @@ -44,7 +41,7 @@ This round, in order: path `Fake` accepts, which a selector is too. Delete the dead `table.descends` and the redundant `string(key)` in `checkNoRepeatedItem`. - Delete the decision "The rows of a table are alternatives": it defines `alternatives`, - which its comment in `family.go` already does. Point the README's "which pairs are + which its comment in `drawfence.go` already does. Point the README's "which pairs are weighed against each other" at that comment's rule, stated in the README's Table section, where it sends a reader to the log for the definition today. - Run the nine-seat panel again and file what it names here. -- 2.52.0 From f6ac63f3abf508387359194f5aaae7e7858d8373 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:39:36 +0200 Subject: [PATCH 167/966] Place checkOwnFamily beside checkDraws, its caller --- drawfence.go | 76 ++++++++++++++++++++++++++-------------------------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/drawfence.go b/drawfence.go index f54da29..01adc1e 100644 --- a/drawfence.go +++ b/drawfence.go @@ -75,6 +75,44 @@ func (c *drawCheck) checkDraws(path string, n node) error { return nil } +// checkOwnFamily refuses a table's format or cell that reads, however many templates +// away and through a repeat or a draw group too, a table of its own family: a whole +// read draws its row without pinning it, so the family would draw apart from the row +// it renders, whichever draws the reaching template holds. +// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route +func checkOwnFamily(t *template) error { + own := t.table + if own == nil { + own = t.cellOf + } + if own == nil { + return nil + } + seen := map[node]bool{} + var find func(n node) (renderEdge, *table, bool) + find = func(n node) (renderEdge, *table, bool) { + if seen[n] { + return renderEdge{}, nil, false + } + seen[n] = true + for _, e := range renderEdges(n) { + if a, isRef := refRead(n, e.label); isRef { + if head, isTable := n.(*template).head(a.key).(*table); isTable && head.family() == own.family() { + return e, head, true + } + } + if e, head, found := find(e.to); found { + return e, head, true + } + } + return renderEdge{}, nil, false + } + if e, head, found := find(t); found { + return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + } + return nil +} + func renderDraws(t *template) *drawWalk { w := newDrawWalk() for _, e := range renderEdges(t) { @@ -483,41 +521,3 @@ func (r *tableRead) replay(d *pinSet) error { }) return err } - -// checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a whole -// read draws its row without pinning it, so the family would draw apart from the row -// it renders, whichever draws the reaching template holds. -// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route -func checkOwnFamily(t *template) error { - own := t.table - if own == nil { - own = t.cellOf - } - if own == nil { - return nil - } - seen := map[node]bool{} - var find func(n node) (renderEdge, *table, bool) - find = func(n node) (renderEdge, *table, bool) { - if seen[n] { - return renderEdge{}, nil, false - } - seen[n] = true - for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).head(a.key).(*table); isTable && head.family() == own.family() { - return e, head, true - } - } - if e, head, found := find(e.to); found { - return e, head, true - } - } - return renderEdge{}, nil, false - } - if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) - } - return nil -} -- 2.52.0 From e0969e69e7197d4d3f8e3473e815d7cc08647d00 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:48:24 +0200 Subject: [PATCH 168/966] Want every format compiled at link, not only one holding a reference --- node_test.go | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/node_test.go b/node_test.go index 5685849..f0cee3d 100644 --- a/node_test.go +++ b/node_test.go @@ -80,3 +80,37 @@ func TestNodeCompileErrors(t *testing.T) { } } } + +func TestEveryFormatCompilesAtLink(t *testing.T) { + n, err := compile(parse(t, `{"format":"{a} {b}","a":{"format":"x{c}","c":["1","2"]},"b":"{/w}"}`)) + if err != nil { + t.Fatal(err) + } + w, err := compile(parse(t, `["p","q"]`)) + if err != nil { + t.Fatal(err) + } + opsBy := func() map[string]bool { + compiled := map[string]bool{} + _ = eachNode(n, "t", func(path string, m node) error { + if tm, ok := m.(*template); ok { + compiled[path+" "+tm.format] = tm.ops != nil + } + return nil + }) + return compiled + } + for format, has := range opsBy() { + if has { + t.Errorf("%s compiled before link", format) + } + } + if err := bindInline(n, "t", map[string]node{"w": w}, func(nodeScope) error { return nil }); err != nil { + t.Fatal(err) + } + for format, has := range opsBy() { + if !has { + t.Errorf("%s not compiled at link", format) + } + } +} -- 2.52.0 From e11ee9ec0e061f3ee38a32c4c3db3c15c596f78c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:53:08 +0200 Subject: [PATCH 169/966] Link every test fragment before asserting on it --- builtins_test.go | 12 ++++++------ calc_test.go | 8 ++++---- datatype_test.go | 4 ++-- fejkdata_test.go | 14 ++++++++++++-- format_test.go | 6 +++--- hold_test.go | 4 ++-- node_test.go | 4 ++-- one_spelling_test.go | 8 ++++---- transform_test.go | 2 +- 9 files changed, 36 insertions(+), 26 deletions(-) diff --git a/builtins_test.go b/builtins_test.go index b19c95d..ee72939 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -142,7 +142,7 @@ func TestBuiltinArgLimits(t *testing.T) { `{"format":"{x}","x":"1","repeat":2000000000}`, // repeat -> multi-GB join } for _, tmpl := range bad { - if _, err := compile(parse(t, tmpl)); err == nil { + if _, err := linked(t, tmpl); err == nil { t.Errorf("compile(%s) = nil error, want a limit rejection", tmpl) } } @@ -226,12 +226,12 @@ func mustPanic(t *testing.T, name string, call func()) { func TestClassBuiltinArgs(t *testing.T) { for _, bad := range []string{`"{digits(0)}"`, `"{digits(2000000000)}"`, `"{upper(-1)}"`, `"{lower(x)}"`, `"{digits()}"`, `"{upper(1,2)}"`} { - if _, err := compile(parse(t, bad)); err == nil { + if _, err := linked(t, bad); err == nil { t.Errorf("compile(%s) = nil error, want the arg rejected", bad) } } for _, ok := range []string{`"{digits(1048576)}"`, `"{upper(1)}"`, `"{lower(26)}"`} { - if _, err := compile(parse(t, ok)); err != nil { + if _, err := linked(t, ok); err != nil { t.Errorf("compile(%s) = %v", ok, err) } } @@ -313,7 +313,7 @@ func TestBuiltinDateArgs(t *testing.T) { `"{time('x')}"`: "text", `"{date(1990-01-01,1990-12-31,'15:04')}"`: "time('15:04')", } { - _, err := compile(parse(t, tmpl)) + _, err := linked(t, tmpl) if err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error mentioning %q", tmpl, err, want) } @@ -427,7 +427,7 @@ func TestBuiltinCompileErrors(t *testing.T) { `"{iban(US)}"`, // unsupported country `"{seq(a,b)}"`, // seq takes at most one name } { - if _, err := compile(parse(t, bad)); err == nil { + if _, err := linked(t, bad); err == nil { t.Errorf("compile(%s) = nil error, want error", bad) } } @@ -441,7 +441,7 @@ func TestArgErrorsNameTheSpelling(t *testing.T) { `"{hex(99999999999999999999)}"`: "exceeds the maximum", `"{int(007,9)}"`: "write 7", } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error saying %q", src, err, want) } } diff --git a/calc_test.go b/calc_test.go index 0ae7cd8..a43c159 100644 --- a/calc_test.go +++ b/calc_test.go @@ -229,7 +229,7 @@ func TestCalcOverANeverNumericOperandIsRejected(t *testing.T) { `{"format":"{calc(x * 2)}","x":["a","b"]}`: `"x"`, `{"format":"{calc(x + y)}","x":"1","y":"{{2}}"}`: `"{2}"`, } { - _, err := compile(parse(t, src)) + _, err := linked(t, src) if err == nil || !strings.Contains(err.Error(), want) || !strings.Contains(err.Error(), "never a number") { t.Errorf("compile(%s) = %v, want the operand rejected naming %s", src, err, want) } @@ -240,7 +240,7 @@ func TestCalcOverANeverNumericOperandIsRejected(t *testing.T) { `{"format":"{calc(x * 2)}","x":["1","abc"]}`, `{"format":"{calc(x * 2)}","x":"{digits(2)}"}`, } { - if _, err := compile(parse(t, ok)); err != nil { + if _, err := linked(t, ok); err != nil { t.Errorf("compile(%s) = %v, want it accepted", ok, err) } } @@ -255,7 +255,7 @@ func TestCalcConstantZeroDivisorIsRejected(t *testing.T) { `{"format":"{calc((a/0)+b)}","a":"1","b":"2"}`: "divides by 0", `{"format":"{calc(a/-0)}","a":"1"}`: "divides by -0", } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want the constant zero divisor rejected naming %q", src, err, want) } } @@ -284,7 +284,7 @@ func TestCalcCompileErrors(t *testing.T) { `"{calc(1, x)}"`, // decimals arg not an integer `"{calc(1, -1)}"`, // decimals negative } { - if _, err := compile(parse(t, bad)); err == nil { + if _, err := linked(t, bad); err == nil { t.Errorf("compile(%s) = nil error, want error", bad) } } diff --git a/datatype_test.go b/datatype_test.go index 61143b7..46824bb 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -18,7 +18,7 @@ func TestDatatypeAndNullSitOnlyInAColumn(t *testing.T) { `{"format":"","age":[null,{"format":"{int(18,99)}","datatype":"integer","weight":9}]}`, `{"format":"","pick":[[null,"a"],"b"]}`, } { - if _, err := compile(parse(t, src)); err != nil { + if _, err := linked(t, src); err != nil { t.Errorf("compile(%s) = %v, want a column to take a datatype and null", src, err) } } @@ -32,7 +32,7 @@ func TestDatatypeAndNullSitOnlyInAColumn(t *testing.T) { `null`: `so write ""`, `{"format":"{p}","p":{"format":"{x}","x":[null,"a"]}}`: `so write ""`, } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error containing %q", src, err, want) } } diff --git a/fejkdata_test.go b/fejkdata_test.go index bbefb3b..f108855 100644 --- a/fejkdata_test.go +++ b/fejkdata_test.go @@ -113,10 +113,20 @@ func parse(t *testing.T, s string) any { return v } -// compiled parses and compiles a JSON fragment into a node. -func compiled(t *testing.T, s string) node { +// linked compiles a JSON fragment and links it against no data. +func linked(t *testing.T, s string) (node, error) { t.Helper() n, err := compile(parse(t, s)) + if err != nil { + return nil, err + } + return n, linkNodeRefs(inlineScope(n, "template"), nil) +} + +// compiled parses, compiles and links a JSON fragment into a node. +func compiled(t *testing.T, s string) node { + t.Helper() + n, err := linked(t, s) if err != nil { t.Fatalf("compile %q: %v", s, err) } diff --git a/format_test.go b/format_test.go index 8df1f38..eae7414 100644 --- a/format_test.go +++ b/format_test.go @@ -21,7 +21,7 @@ func TestStringIsAFormat(t *testing.T) { if got := mustRender(t, f, `"{digits(3)}"`); !regexp.MustCompile(`^[0-9]{3}$`).MatchString(got) { t.Errorf(`render("{digits(3)}") = %q, want three digits`, got) } - if _, err := compile(parse(t, `"{x}"`)); err == nil || !strings.Contains(err.Error(), `no field "x"`) { + if _, err := linked(t, `"{x}"`); err == nil || !strings.Contains(err.Error(), `no field "x"`) { t.Errorf(`compile("{x}") = %v, want a no-field error`, err) } } @@ -51,7 +51,7 @@ func TestBraceEscapes(t *testing.T) { `{"format":"{a{b}","a":"Q"}`: "'{'", `"{x"`: "unterminated", } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want) } } @@ -126,7 +126,7 @@ func TestFormatCompileErrors(t *testing.T) { `"{int(1)}"`, // wrong arity `"{float(1,2)}"`, // wrong arity } { - if _, err := compile(parse(t, bad)); err == nil { + if _, err := linked(t, bad); err == nil { t.Errorf("compile(%s) = nil error, want error", bad) } } diff --git a/hold_test.go b/hold_test.go index 06c691e..5419a42 100644 --- a/hold_test.go +++ b/hold_test.go @@ -743,7 +743,7 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { `{"format":"{n} + {n} = {calc(n * 2)}","n":["1","2"]}`: "write {n} once", `{"format":"{p.a} {q} {q} {lowercase(q)}","p":{"format":"{a}","a":"1"},"q":["A","B"]}`: "write {q} once", } { - _, err := compile(parse(t, src)) + _, err := linked(t, src) if err == nil || !strings.Contains(err.Error(), want) || !strings.Contains(err.Error(), "holds") { t.Errorf("compile(%s) = %v, want the repeated token rejected naming %s", src, err, want) } @@ -754,7 +754,7 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00"],"qty":["3","7"]}`, `{"format":"{uppercase(w)} {uppercase(w)}","w":["a","b"]}`, } { - if _, err := compile(parse(t, ok)); err != nil { + if _, err := linked(t, ok); err != nil { t.Errorf("compile(%s) = %v, want it accepted", ok, err) } } diff --git a/node_test.go b/node_test.go index f0cee3d..f939618 100644 --- a/node_test.go +++ b/node_test.go @@ -7,7 +7,7 @@ import ( ) func TestWeightZeroIsRejected(t *testing.T) { - _, err := compile(parse(t, `[{"format":"X","weight":0},"Y"]`)) + _, err := linked(t, `[{"format":"X","weight":0},"Y"]`) if err == nil || !strings.Contains(err.Error(), "weight 0") || !strings.Contains(err.Error(), "remove") { t.Fatalf("compile(weight 0) = %v, want it rejected naming the fix", err) } @@ -75,7 +75,7 @@ func TestNodeCompileErrors(t *testing.T) { `{"format":"x","repeat":2,"separator":5}`, // non-string separator `{"format":"{x}","x":"v","repeat":2,"separator":""}`, // separator "" is the default } { - if _, err := compile(parse(t, bad)); err == nil { + if _, err := linked(t, bad); err == nil { t.Errorf("compile(%s) = nil error, want error", bad) } } diff --git a/one_spelling_test.go b/one_spelling_test.go index 6f1f559..8673873 100644 --- a/one_spelling_test.go +++ b/one_spelling_test.go @@ -7,7 +7,7 @@ import ( func TestOneItemChoiceIsRejected(t *testing.T) { for _, src := range []string{`["x"]`, `[{"format":"{d}","d":"x"}]`, `{"format":"{w}","w":["only"]}`} { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), "one-item choice") { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), "one-item choice") { t.Errorf("compile(%s) = %v, want the one-item choice rejected", src, err) } } @@ -20,11 +20,11 @@ func TestRepeatedChoiceItemIsRejected(t *testing.T) { `{"format":"{w}","w":["", "", "x"]}`: `{ "format": "", "weight": 2 }`, `{"format":"","w":[null,null,"a"]}`: "a null takes no weight", } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error naming %s", src, err, want) } } - if _, err := compile(parse(t, `[{"format":"a","weight":2}, "b"]`)); err != nil { + if _, err := linked(t, `[{"format":"a","weight":2}, "b"]`); err != nil { t.Errorf("compile(weighted a, b) = %v", err) } } @@ -41,7 +41,7 @@ func TestInertObjectIsRejected(t *testing.T) { `{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":""}}`: `drawGroup "" is the default`, `{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":1}}`: "drawGroup must be a string", } { - if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { + if _, err := linked(t, src); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want) } } diff --git a/transform_test.go b/transform_test.go index 740212d..668ea6f 100644 --- a/transform_test.go +++ b/transform_test.go @@ -46,7 +46,7 @@ func TestTransformArgs(t *testing.T) { `{"format":"{lowercase(hex(2))}","x":"v"}`, `{"format":"{lowercase(x)} {x.a}","x":{"format":"{a}","a":"1"}}`, } { - if _, err := compile(parse(t, bad)); err == nil { + if _, err := linked(t, bad); err == nil { t.Errorf("compile(%s) = nil error, want it rejected", bad) } } -- 2.52.0 From 306b88ef5690679e790e04e0f03742d3955155a3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:53:08 +0200 Subject: [PATCH 170/966] Compile every format at link --- AGENTS.md | 2 +- calc.go | 15 ++++++++++----- docs/decisions.md | 10 +++++----- holdfence.go | 6 +++++- node.go | 41 +++++++++++++---------------------------- reference.go | 33 +++++++++++++++++++-------------- render.go | 4 ++-- table.go | 2 +- todo.md | 2 -- value.go | 5 +++-- 10 files changed, 59 insertions(+), 61 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 439b0ce..1dcbb74 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -40,7 +40,7 @@ In [docs/decisions.md](docs/decisions.md): - A change to what exists is a major; a minor only adds - Seeded output is promised within one version - An error is a contract by what it names, not its bytes -- A format holding a reference is fenced at link, and its errors name its path +- Every format is fenced at link, and its errors name its path - Raising the lowest supported Go is a major - The format check runs on the latest Go only - The changelog heading is the one spelling of a release; CI cuts the tag diff --git a/calc.go b/calc.go index a279978..60801fb 100644 --- a/calc.go +++ b/calc.go @@ -120,10 +120,14 @@ func constantValue(n calcNode, fields map[string]node) (float64, bool) { return float64(n), true case calcVar: t, ok := fields[string(n)].(*template) - if !ok || !t.fixed || t.repeat > 1 { + if !ok || t.repeat > 1 { return 0, false } - v, err := strconv.ParseFloat(strings.TrimSpace(t.lit), 64) + lit, fixed := t.fixedText() + if !fixed { + return 0, false + } + v, err := strconv.ParseFloat(strings.TrimSpace(lit), 64) return v, err == nil case calcNeg: v, ok := constantValue(n.x, fields) @@ -161,11 +165,12 @@ func neverNumeric(n node) (text string, never bool) { case *nullItem: return "", true case *template: - if !n.fixed || n.repeat > 1 { + lit, fixed := n.fixedText() + if !fixed || n.repeat > 1 { return "", false } - if _, err := strconv.ParseFloat(strings.TrimSpace(n.lit), 64); err != nil { - return n.lit, true + if _, err := strconv.ParseFloat(strings.TrimSpace(lit), 64); err != nil { + return lit, true } case *choice: for _, it := range n.items { diff --git a/docs/decisions.md b/docs/decisions.md index 1f43aa4..bbb1731 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -121,12 +121,12 @@ this repo's own are. A script branches on the exit code and reads the named path or spelling, so those hold; wording improves in a minor. -## A format holding a reference is fenced at link, and its errors name its path +## Every format is fenced at link, and its errors name its path -2026-09-24, larv-review; approved 2026-09-25 by lilleman. Goal 2: a format compiles once, and one holding a reference -can compile only once bound, so its fences fire at link and name the category path as -every link error does, a cell's with its line; one without a reference still names its -file at compile. Valid while compilation needs the bound references. +2026-09-28, larv-review; approval pending. Goal 2: every format compiles in one phase, +at link, since one holding a reference can compile only once bound, so no check reads +compiled ops a format lacks yet. Its fences name the category path as every link error +does, a cell's with its line. Valid while compilation needs the bound references. ## Raising the lowest supported Go is a major diff --git a/holdfence.go b/holdfence.go index b839621..e3e749f 100644 --- a/holdfence.go +++ b/holdfence.go @@ -144,7 +144,11 @@ func operandDraw(n node, into map[node]bool) { // isFixed is a string that varies nothing: fixed text with no fields to read into. func isFixed(n node) bool { t, ok := n.(*template) - return ok && t.fixed && len(t.fields) == 0 + if !ok { + return false + } + _, fixed := t.fixedText() + return fixed && len(t.fields) == 0 } // renders reports whether rendering n can reach anything in want, following the diff --git a/node.go b/node.go index b6c274a..a0cc34c 100644 --- a/node.go +++ b/node.go @@ -41,7 +41,7 @@ func (*nullItem) isNode() {} // template renders a format string, substituting {tokens} from fields. A bare // JSON string is a template with no fields. repeat (default 1) renders that format // that many times and joins the results with separator (default ""), each render -// an independent pick. A format holding a reference compiles in `linkTemplateRefs`. +// an independent pick. Every format compiles in `linkTemplateRefs`. type template struct { // Filled by `compileString`, `compileTemplate` and `table.compileRowFormat`: format string @@ -60,10 +60,8 @@ type template struct { cellRow int // the row the cell sits in // Filled by `template.compileFormat`, from `template.format` and `template.refs`: - ops []op // what expand walks - grow int // minimum output size, to size the render buffer - fixed bool // no op varies, so every render is lit - lit string // the whole output when fixed + ops []op // what expand walks + grow int // minimum output size, to size the render buffer // bound maps each field the format addresses by dotted path to one path token // reading it, which is the half of an overlap the fences name. nil when the // format takes no path. @@ -208,18 +206,18 @@ func compileString(s string) (node, error) { if err != nil { return nil, err } - t := &template{format: s, tokens: toks, repeat: 1, fromString: true} - if err := t.compileRefFree(); err != nil { - return nil, err - } - return t, nil + return &template{format: s, tokens: toks, repeat: 1, fromString: true}, nil } -func (t *template) compileRefFree() error { - if len(refTokens(t.tokens)) > 0 { - return nil +// fixedText is the whole output of a format holding no token, which every render is. +func (t *template) fixedText() (string, bool) { + switch { + case len(t.tokens) == 0: + return "", true + case len(t.tokens) == 1 && t.tokens[0].kind == 'l': + return t.tokens[0].lit, true } - return t.compileFormat() + return "", false } // compileFormat compiles the format into ops, and applies the fences that need the @@ -227,15 +225,6 @@ func (t *template) compileRefFree() error { func (t *template) compileFormat() error { c := compileOps(t.tokens, t.refs) t.ops, t.grow, t.bound, t.held, t.heldLocal = c.ops, c.grow, c.bound, c.held, c.heldLocal - t.fixed = true - for _, o := range t.ops { - if o.kind != 'l' { - t.fixed = false - } - } - if t.fixed && len(t.ops) == 1 { - t.lit = t.ops[0].lit - } if err := checkNoOverlap(t.ops, t.bound); err != nil { return err } @@ -340,11 +329,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if err := checkNestedDrawGroup(fields, o.group); err != nil { return nil, err } - t := &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn} - if err := t.compileRefFree(); err != nil { - return nil, err - } - return t, nil + return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn}, nil } // templateOptions is what a template object's option keys say. diff --git a/reference.go b/reference.go index 9b4c041..c8e6882 100644 --- a/reference.go +++ b/reference.go @@ -67,9 +67,9 @@ func refSegments(name string, folder []string) ([]string, error) { return append(append([]string{}, base...), segs...), nil } -// linkRefs resolves every reference in the assembled tree. The head of the path — -// up to the category it names — is bound into the referring template's refHeads -// under its root path, and the rest reads into it the way a sibling path does, so +// linkRefs resolves every reference in the assembled tree and compiles every format. +// The head of the path — up to the category it names — is bound into the referring +// template's refHeads under its root path, and the rest reads into it the way a sibling path does, so // a reference is held like a sibling and two spellings of one target are one // draw. It runs once, after all data is merged, so a reference sees the final // (override-resolved) tree. A path that is unknown, names a folder, or reads a @@ -83,17 +83,26 @@ func linkRefs(root map[string]node) error { }) } -// linkTemplateRefs binds one template's references against root, refusing one that names the -// category it sits in. -// docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load +// linkTemplateRefs binds one template's references against root, then compiles its format. func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { - names := refTokens(t.tokens) - if len(names) == 0 { - return nil - } if t.cellOf != nil { path = fmt.Sprintf("%s, line %d", path, t.cellRow+2) } + if names := refTokens(t.tokens); len(names) > 0 { + if err := t.bindRefs(folder, path, category, names, root); err != nil { + return err + } + } + if err := t.compileFormat(); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + t.readsColumn = columnReadOf(t) + return nil +} + +// bindRefs refuses a reference naming the category t sits in. +// docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load +func (t *template) bindRefs(folder []string, path, category string, names []string, root map[string]node) error { t.refs = make(map[string]refBinding, len(names)) t.refHeads = make(map[string]node, len(names)) for _, name := range names { @@ -115,10 +124,6 @@ func linkTemplateRefs(folder []string, path, category string, t *template, root t.refHeads[key] = target t.refs[name] = refBinding{key, tail} } - if err := t.compileFormat(); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - t.readsColumn = columnReadOf(t) return nil } diff --git a/render.go b/render.go index c7b100b..1275386 100644 --- a/render.go +++ b/render.go @@ -71,8 +71,8 @@ func render(s *session, n node, sc renderScope) string { case *template: sc = sc.in(n) if n.repeat == 1 { - if n.fixed { - return n.lit + if lit, fixed := n.fixedText(); fixed { + return lit } return expand(s, n, sc) } diff --git a/table.go b/table.go index f11c4d9..9c407d9 100644 --- a/table.go +++ b/table.go @@ -344,7 +344,7 @@ func (t *table) compileRowFormat(format string) error { } t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, table: t} t.pinnedRow = &tableRow{t} - return t.formatTemplate.compileRefFree() + return nil } // setTablePaths gives every table the path a selector on it is written at, before a diff --git a/todo.md b/todo.md index 3b58261..8232804 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Compile every format in one phase: a template holding a reference has nil `ops` - until `linkTemplateRefs`, so a check reading `ops` before link skips it silently. - Run `New`, `NewTemplate` and `FakeStruct` through one bind pipeline that names where they differ: each picks its own fences today. - Make `disagreement` and `bothTyped` a table of cases: each swaps its operands and diff --git a/value.go b/value.go index 1dd8b23..32718e4 100644 --- a/value.go +++ b/value.go @@ -143,11 +143,12 @@ func (v proven) or(w proven) proven { // proveTemplate proves a template that renders one value: fixed text, or a format that is // one token alone. func (p *valueProof) proveTemplate(t *template) proven { + lit, fixed := t.fixedText() switch { case t.repeat != 1: return unproven(fmt.Sprintf("%q carries a repeat, which composes text rather than one value", t.format)) - case t.fixed: - return literalValue(t.lit) + case fixed: + return literalValue(lit) case len(t.ops) != 1: v := unproven(notOneValue(t.format, typedCalls)) v.notOperand = notOneValue(t.format, operandCalls) -- 2.52.0 From 0dee3e01f8a1460fd03858b6bd4fa7baffb794f6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 02:55:42 +0200 Subject: [PATCH 171/966] Want a fence error to name its category path --- hold_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hold_test.go b/hold_test.go index 5419a42..0decffd 100644 --- a/hold_test.go +++ b/hold_test.go @@ -56,8 +56,8 @@ func TestRenderingALevelAndReadingIntoItIsRejected(t *testing.T) { t.Errorf("%s: New = nil error, want the overlapping tokens rejected", name) continue } - if !strings.Contains(err.Error(), "reads a path into") { - t.Errorf("%s: New = %v, want it to name the overlap", name, err) + if !strings.HasPrefix(err.Error(), "fejkdata: cat: ") || !strings.Contains(err.Error(), "reads a path into") { + t.Errorf("%s: New = %v, want it to name the category path and the overlap", name, err) } } // The same path twice is one spelling, so it stays legal. -- 2.52.0 From dbddb0845a125655b17527502f3bcc4f75c549cd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 03:06:46 +0200 Subject: [PATCH 172/966] Name the compiled-flag map and fail on its walk --- node_test.go | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/node_test.go b/node_test.go index f939618..83f1634 100644 --- a/node_test.go +++ b/node_test.go @@ -90,17 +90,19 @@ func TestEveryFormatCompilesAtLink(t *testing.T) { if err != nil { t.Fatal(err) } - opsBy := func() map[string]bool { + compiledByPath := func() map[string]bool { compiled := map[string]bool{} - _ = eachNode(n, "t", func(path string, m node) error { + if err := eachNode(n, "t", func(path string, m node) error { if tm, ok := m.(*template); ok { compiled[path+" "+tm.format] = tm.ops != nil } return nil - }) + }); err != nil { + t.Fatal(err) + } return compiled } - for format, has := range opsBy() { + for format, has := range compiledByPath() { if has { t.Errorf("%s compiled before link", format) } @@ -108,7 +110,7 @@ func TestEveryFormatCompilesAtLink(t *testing.T) { if err := bindInline(n, "t", map[string]node{"w": w}, func(nodeScope) error { return nil }); err != nil { t.Fatal(err) } - for format, has := range opsBy() { + for format, has := range compiledByPath() { if !has { t.Errorf("%s not compiled at link", format) } -- 2.52.0 From aee932dc934bae5a7b44d34578f8486db3c2e43a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 03:06:46 +0200 Subject: [PATCH 173/966] Rewrap the linkRefs comment --- reference.go | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/reference.go b/reference.go index c8e6882..d5cf94e 100644 --- a/reference.go +++ b/reference.go @@ -69,12 +69,12 @@ func refSegments(name string, folder []string) ([]string, error) { // linkRefs resolves every reference in the assembled tree and compiles every format. // The head of the path — up to the category it names — is bound into the referring -// template's refHeads under its root path, and the rest reads into it the way a sibling path does, so -// a reference is held like a sibling and two spellings of one target are one -// draw. It runs once, after all data is merged, so a reference sees the final -// (override-resolved) tree. A path that is unknown, names a folder, or reads a -// field not every variant carries fails here, keeping a bad reference a New-time -// error, never a random render-time one. +// template's refHeads under its root path, and the rest reads into it the way a +// sibling path does, so a reference is held like a sibling and two spellings of one +// target are one draw. It runs once, after all data is merged, so a reference sees +// the final (override-resolved) tree. A path that is unknown, names a folder, or +// reads a field not every variant carries fails here, keeping a bad reference a +// New-time error, never a random render-time one. func linkRefs(root map[string]node) error { return eachTemplate(root, func(folder []string, path string, t *template) error { category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") -- 2.52.0 From 6e0f302d20e2cf1949e42d361a1f8535074f2ffc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 03:07:47 +0200 Subject: [PATCH 174/966] Plan naming the file a link error comes from --- todo.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/todo.md b/todo.md index 8232804..312f48e 100644 --- a/todo.md +++ b/todo.md @@ -213,6 +213,9 @@ by a `data-import/` script, as README goal 12 asks. together, where the fence counts them alternatives and compares neither. Have the error name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`, as the README's rule on a rejected spelling asks. +- Name the file a link error comes from beside its category path, `sub.x + (/d/a/sub/x.json): …`: with several `--data-path` layers the author has to work out + which directory won, while a parse error on the same file names it. - Name the row a refused read sits in: where both reads come from TSV cells the overlap error names the category and the two columns but no row, so the author greps a register-sized file for the cell — and "name the fields you want instead" rewrites a -- 2.52.0 From ecb90f828f13721593d656a5314491265b5f5996 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 03:09:30 +0200 Subject: [PATCH 175/966] Correct the comments and decision around link-time compilation --- docs/decisions.md | 8 ++++---- node.go | 6 +++--- reference.go | 15 +++++---------- 3 files changed, 12 insertions(+), 17 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index bbb1731..c1d176e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -123,10 +123,10 @@ wording improves in a minor. ## Every format is fenced at link, and its errors name its path -2026-09-28, larv-review; approval pending. Goal 2: every format compiles in one phase, -at link, since one holding a reference can compile only once bound, so no check reads -compiled ops a format lacks yet. Its fences name the category path as every link error -does, a cell's with its line. Valid while compilation needs the bound references. +2026-09-28, larv-review; approval pending. Goal 2. Every format compiles once, at +link: one holding a reference compiles only once bound, and compiling all there +leaves no check reading ops a format lacks yet. A link error names the category +path, a table cell's with its line. Valid while compilation needs the bound references. ## Raising the lowest supported Go is a major diff --git a/node.go b/node.go index a0cc34c..bb0975c 100644 --- a/node.go +++ b/node.go @@ -55,11 +55,11 @@ type template struct { isRecord bool // compiled at the top without a repeat, so its fields are record columns table *table // the table whose format this is, whose columns are the fields - // Filled by `checkCells`, after the cell's own format has compiled: + // Filled by `checkCells`: cellOf *table // the table whose cell this is cellRow int // the row the cell sits in - // Filled by `template.compileFormat`, from `template.format` and `template.refs`: + // Filled by `template.compileFormat` at link, from `template.tokens` and `template.refs`: ops []op // what expand walks grow int // minimum output size, to size the render buffer // bound maps each field the format addresses by dotted path to one path token @@ -209,7 +209,7 @@ func compileString(s string) (node, error) { return &template{format: s, tokens: toks, repeat: 1, fromString: true}, nil } -// fixedText is the whole output of a format holding no token, which every render is. +// fixedText is one render's output when the format holds no token; repeat is the caller's. func (t *template) fixedText() (string, bool) { switch { case len(t.tokens) == 0: diff --git a/reference.go b/reference.go index d5cf94e..fceff0f 100644 --- a/reference.go +++ b/reference.go @@ -67,14 +67,10 @@ func refSegments(name string, folder []string) ([]string, error) { return append(append([]string{}, base...), segs...), nil } -// linkRefs resolves every reference in the assembled tree and compiles every format. -// The head of the path — up to the category it names — is bound into the referring -// template's refHeads under its root path, and the rest reads into it the way a -// sibling path does, so a reference is held like a sibling and two spellings of one -// target are one draw. It runs once, after all data is merged, so a reference sees -// the final (override-resolved) tree. A path that is unknown, names a folder, or -// reads a field not every variant carries fails here, keeping a bad reference a -// New-time error, never a random render-time one. +// linkRefs binds every reference and compiles every format, once all data is merged, +// so a reference sees the override-resolved tree. A reference's head binds into +// refHeads under its root path and its tail reads like a sibling path, so two +// spellings of one target are one draw. func linkRefs(root map[string]node) error { return eachTemplate(root, func(folder []string, path string, t *template) error { category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") @@ -83,7 +79,6 @@ func linkRefs(root map[string]node) error { }) } -// linkTemplateRefs binds one template's references against root, then compiles its format. func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { if t.cellOf != nil { path = fmt.Sprintf("%s, line %d", path, t.cellRow+2) @@ -100,7 +95,7 @@ func linkTemplateRefs(folder []string, path, category string, t *template, root return nil } -// bindRefs refuses a reference naming the category t sits in. +// bindRefs fills refs and refHeads, refusing a reference to t's own category: // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load func (t *template) bindRefs(folder []string, path, category string, names []string, root map[string]node) error { t.refs = make(map[string]refBinding, len(names)) -- 2.52.0 From 9ea8f844211439776ef612321148320e51050ead Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 10:21:01 +0200 Subject: [PATCH 176/966] Approve and reword the link-time checks decision --- AGENTS.md | 2 +- docs/decisions.md | 12 +++++++----- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 1dcbb74..cea2259 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -40,7 +40,7 @@ In [docs/decisions.md](docs/decisions.md): - A change to what exists is a major; a minor only adds - Seeded output is promised within one version - An error is a contract by what it names, not its bytes -- Every format is fenced at link, and its errors name its path +- Formats are checked after all data is loaded, so their errors name the category - Raising the lowest supported Go is a major - The format check runs on the latest Go only - The changelog heading is the one spelling of a release; CI cuts the tag diff --git a/docs/decisions.md b/docs/decisions.md index c1d176e..5cc3b00 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -121,12 +121,14 @@ this repo's own are. A script branches on the exit code and reads the named path or spelling, so those hold; wording improves in a minor. -## Every format is fenced at link, and its errors name its path +## Formats are checked after all data is loaded, so their errors name the category -2026-09-28, larv-review; approval pending. Goal 2. Every format compiles once, at -link: one holding a reference compiles only once bound, and compiling all there -leaves no check reading ops a format lacks yet. A link error names the category -path, a table cell's with its line. Valid while compilation needs the bound references. +2026-09-28, larv-review; approved 2026-09-28 by lilleman. Goal 2. Loading reads every +file first, then connects references across them. A format holding a reference can only +be checked once it is connected, so every format is checked then, and all checks run at +one moment. An error from those checks names the category path you would type (`sub.x`), +or a table cell's category and line, not the file. Valid while checking a format needs +its references connected. ## Raising the lowest supported Go is a major -- 2.52.0 From cfe653de4d13a841f212f6767cc6edf23d7488f7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 10:32:28 +0200 Subject: [PATCH 177/966] Run New, NewTemplate and FakeStruct through one bind pipeline naming where they differ --- README.md | 2 +- data.go | 57 +++++++++++++++++++++++++++++++++++------------ docs/decisions.md | 2 +- inline.go | 18 +++++++-------- node_test.go | 2 +- struct.go | 4 +++- todo.md | 2 -- 7 files changed, 58 insertions(+), 29 deletions(-) diff --git a/README.md b/README.md index e9e5301..5eb803b 100644 --- a/README.md +++ b/README.md @@ -890,7 +890,7 @@ transform.go the builtins that rewrite an operand's value, and the ASCII fold calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype value.go the value proof: what a typed column or calc operand holds, checked at load -data.go the load path: fs.FS folders/files -> merged namespace tree, and the fence order a scope runs +data.go the load path: fs.FS folders/files -> merged namespace tree, and the bind pipeline every entry point runs cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) diff --git a/data.go b/data.go index ed0542b..46c9cda 100644 --- a/data.go +++ b/data.go @@ -62,17 +62,7 @@ func loadData(sources []dataSource) (map[string]node, error) { if len(root) == 0 { return nil, fmt.Errorf("no .json data found") } - setTablePaths(root) - if err := linkTables(root); err != nil { - return nil, err - } - if err := linkRefs(root); err != nil { - return nil, err - } - if err := checkNoCycles(root); err != nil { - return nil, err - } - if err := checkScope(treeScope(root)); err != nil { + if err := treeBinding(root).bind(); err != nil { return nil, err } return root, nil @@ -206,11 +196,50 @@ func inlineScope(n node, label string) nodeScope { return func(fn func(path string, m node) error) error { return eachNode(n, label, fn) } } -func checkScope(s nodeScope) error { - if err := checkColumns(s); err != nil { +// binding is one entry point's nodes on their way through bind, the load pipeline New, +// NewTemplate and FakeStruct share; its fields are where they differ. +type binding struct { + scope nodeScope + // link binds the references: a loaded tree's relative to each category's folder, an + // inline node's from the root alone. + link func() error + // noCycles refuses a reference cycle, which only a loaded tree can close: nothing + // references an inline node. + noCycles func() error + // typedByGo marks a struct's columns, whose datatypes its Go types set and checkField + // proves, so checkColumns skips them. + typedByGo bool +} + +func treeBinding(root map[string]node) binding { + return binding{ + scope: treeScope(root), + link: func() error { + setTablePaths(root) + if err := linkTables(root); err != nil { + return err + } + return linkRefs(root) + }, + noCycles: func() error { return checkNoCycles(root) }, + } +} + +func (b binding) bind() error { + if err := b.link(); err != nil { return err } - return checkRenders(s) + if b.noCycles != nil { + if err := b.noCycles(); err != nil { + return err + } + } + if !b.typedByGo { + if err := checkColumns(b.scope); err != nil { + return err + } + } + return checkRenders(b.scope) } // checkRenders runs the per-node fences over a scope, each over the whole scope diff --git a/docs/decisions.md b/docs/decisions.md index 5cc3b00..1c697e6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -57,7 +57,7 @@ tags and any other caller read one. `New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and nothing in the tree can reference it, so no render of it reaches itself. Every other -fence runs over both, from one `checkScope`, except that struct tags leave column +fence runs over both, from one `binding.bind`, except that struct tags leave column agreement to their Go types. ## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing diff --git a/inline.go b/inline.go index b44127e..29ecf11 100644 --- a/inline.go +++ b/inline.go @@ -32,7 +32,7 @@ func (f *Generator) NewTemplate(input string) (*Template, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - if err := bindInline(n, "template", f.categories, checkScope); err != nil { + if err := inlineBinding(n, "template", f.categories).bind(); err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } return &Template{g: f, n: n}, nil @@ -146,17 +146,17 @@ func inputValue(input string) (any, error) { return raw, nil } -// bindInline links an inline node's references against root and runs check over it, naming its -// nodes from label. -func bindInline(n node, label string, root map[string]node, check func(nodeScope) error) error { +// inlineBinding binds an inline node against root, naming its nodes from label. +func inlineBinding(n node, label string, root map[string]node) binding { + scope := inlineScope(n, label) + return binding{scope: scope, link: func() error { return linkInline(n, label, scope, root) }} +} + +func linkInline(n node, label string, scope nodeScope, root map[string]node) error { if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) } - scope := inlineScope(n, label) - if err := linkNodeRefs(scope, root); err != nil { - return err - } - return check(scope) + return linkNodeRefs(scope, root) } // linkNodeRefs binds the references in an inline node's templates against the diff --git a/node_test.go b/node_test.go index 83f1634..b8e1759 100644 --- a/node_test.go +++ b/node_test.go @@ -107,7 +107,7 @@ func TestEveryFormatCompilesAtLink(t *testing.T) { t.Errorf("%s compiled before link", format) } } - if err := bindInline(n, "t", map[string]node{"w": w}, func(nodeScope) error { return nil }); err != nil { + if err := linkInline(n, "t", inlineScope(n, "t"), map[string]node{"w": w}); err != nil { t.Fatal(err) } for format, has := range compiledByPath() { diff --git a/struct.go b/struct.go index 66bfd61..251f199 100644 --- a/struct.go +++ b/struct.go @@ -258,7 +258,9 @@ func (s *structShape) compileRecord(categories map[string]node, t reflect.Type, if err != nil { return fmt.Errorf("%s: %w", label, err) } - if err := bindInline(n, label, categories, checkRenders); err != nil { + b := inlineBinding(n, label, categories) + b.typedByGo = true + if err := b.bind(); err != nil { return err } record, columns, err := recordOf(n) diff --git a/todo.md b/todo.md index 312f48e..e9e2783 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Run `New`, `NewTemplate` and `FakeStruct` through one bind pipeline that names - where they differ: each picks its own fences today. - Make `disagreement` and `bothTyped` a table of cases: each swaps its operands and runs a proof to pick an error's wording. - Derive a column's datatype in one place: `columnItems` yields nothing for a -- 2.52.0 From 224cf12fdd6c3545d9c22951807da08bf69aecfa Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 10:36:35 +0200 Subject: [PATCH 178/966] Link the compile-at-link fragment as every other test does --- node_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/node_test.go b/node_test.go index b8e1759..f994f27 100644 --- a/node_test.go +++ b/node_test.go @@ -107,7 +107,7 @@ func TestEveryFormatCompilesAtLink(t *testing.T) { t.Errorf("%s compiled before link", format) } } - if err := linkInline(n, "t", inlineScope(n, "t"), map[string]node{"w": w}); err != nil { + if err := linkNodeRefs(inlineScope(n, "t"), map[string]node{"w": w}); err != nil { t.Fatal(err) } for format, has := range compiledByPath() { -- 2.52.0 From 01a60d06f1d6dc4ffb2c3b63fc88c8337928fbb8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 10:37:26 +0200 Subject: [PATCH 179/966] Name the inline drawGroup refusal as a binding's reference fence, beside the cycle fence --- data.go | 23 ++++++++--------------- docs/decisions.md | 3 ++- inline.go | 17 +++++++++-------- 3 files changed, 19 insertions(+), 24 deletions(-) diff --git a/data.go b/data.go index 46c9cda..52e94ec 100644 --- a/data.go +++ b/data.go @@ -200,14 +200,10 @@ func inlineScope(n node, label string) nodeScope { // NewTemplate and FakeStruct share; its fields are where they differ. type binding struct { scope nodeScope - // link binds the references: a loaded tree's relative to each category's folder, an - // inline node's from the root alone. - link func() error - // noCycles refuses a reference cycle, which only a loaded tree can close: nothing - // references an inline node. - noCycles func() error - // typedByGo marks a struct's columns, whose datatypes its Go types set and checkField - // proves, so checkColumns skips them. + link func() error + // A loaded tree's cycle fence; an inline node's refusal of a drawGroup at its root. + referenceFence func() error + // checkField proves a struct's columns instead. typedByGo bool } @@ -221,7 +217,7 @@ func treeBinding(root map[string]node) binding { } return linkRefs(root) }, - noCycles: func() error { return checkNoCycles(root) }, + referenceFence: func() error { return checkNoCycles(root) }, } } @@ -229,10 +225,8 @@ func (b binding) bind() error { if err := b.link(); err != nil { return err } - if b.noCycles != nil { - if err := b.noCycles(); err != nil { - return err - } + if err := b.referenceFence(); err != nil { + return err } if !b.typedByGo { if err := checkColumns(b.scope); err != nil { @@ -244,8 +238,7 @@ func (b binding) bind() error { // checkRenders runs the per-node fences over a scope, each over the whole scope // before the next, so which of several broken nodes is reported does not depend on -// the walk. Its walks terminate only where nothing renders itself, so run it over a -// loaded tree after checkNoCycles. +// the walk. Its walks terminate only where nothing renders itself. func checkRenders(s nodeScope) error { mem := renderCounts{} if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { diff --git a/docs/decisions.md b/docs/decisions.md index 1c697e6..3544f20 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -58,7 +58,8 @@ tags and any other caller read one. `New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and nothing in the tree can reference it, so no render of it reaches itself. Every other fence runs over both, from one `binding.bind`, except that struct tags leave column -agreement to their Go types. +agreement to their Go types, and an inline node refuses a `drawGroup` at its root, +which nothing can name. ## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing diff --git a/inline.go b/inline.go index 29ecf11..3685670 100644 --- a/inline.go +++ b/inline.go @@ -146,17 +146,18 @@ func inputValue(input string) (any, error) { return raw, nil } -// inlineBinding binds an inline node against root, naming its nodes from label. func inlineBinding(n node, label string, root map[string]node) binding { scope := inlineScope(n, label) - return binding{scope: scope, link: func() error { return linkInline(n, label, scope, root) }} -} - -func linkInline(n node, label string, scope nodeScope, root map[string]node) error { - if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { - return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) + return binding{ + scope: scope, + link: func() error { return linkNodeRefs(scope, root) }, + referenceFence: func() error { + if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { + return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) + } + return nil + }, } - return linkNodeRefs(scope, root) } // linkNodeRefs binds the references in an inline node's templates against the -- 2.52.0 From 2254b33f0e7fd6585148fee36328de37b823b3de Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 10:40:35 +0200 Subject: [PATCH 180/966] Keep the binding's prose to what its code cannot say --- data.go | 11 +++++------ docs/decisions.md | 5 +---- 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/data.go b/data.go index 52e94ec..f08d07a 100644 --- a/data.go +++ b/data.go @@ -196,14 +196,13 @@ func inlineScope(n node, label string) nodeScope { return func(fn func(path string, m node) error) error { return eachNode(n, label, fn) } } -// binding is one entry point's nodes on their way through bind, the load pipeline New, -// NewTemplate and FakeStruct share; its fields are where they differ. +// binding is one scope's way through bind, the one load pipeline; its fields are where +// scopes differ. type binding struct { - scope nodeScope - link func() error - // A loaded tree's cycle fence; an inline node's refusal of a drawGroup at its root. + scope nodeScope + link func() error referenceFence func() error - // checkField proves a struct's columns instead. + // Set by a caller that proves the columns against their Go types itself. typedByGo bool } diff --git a/docs/decisions.md b/docs/decisions.md index 3544f20..6304093 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -56,10 +56,7 @@ tags and any other caller read one. ## An inline template skips the cycle fence `New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and -nothing in the tree can reference it, so no render of it reaches itself. Every other -fence runs over both, from one `binding.bind`, except that struct tags leave column -agreement to their Go types, and an inline node refuses a `drawGroup` at its root, -which nothing can name. +nothing in the tree can reference it, so no render of it reaches itself. ## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing -- 2.52.0 From 6afe16ba0173795e87028e0fb0950f07823f585e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 10:55:49 +0200 Subject: [PATCH 181/966] Pick a column disagreement's wording from one swap, one proof and one switch --- datatype.go | 42 ++++++++++++++++-------------------------- todo.md | 2 -- 2 files changed, 16 insertions(+), 28 deletions(-) diff --git a/datatype.go b/datatype.go index b8f4610..028cb65 100644 --- a/datatype.go +++ b/datatype.go @@ -157,37 +157,27 @@ func columnItems(n node) (items []*template, nullable bool) { return items, nullable } -// disagreement names the fix for two items of one column holding different datatypes. +// disagreement names the fix for two items of one column holding different datatypes. The item to +// fix declares none, and is text where the other reads a typed column. func disagreement(a *template, da DataType, b *template, db DataType) error { - bare, typed, want := b, a, da - if da == DataTypeString { - bare, typed, want = a, b, db + fix, has, other, want := a, da, b, db + if da != DataTypeString && (a.datatype != DataTypeString || db == DataTypeString) { + fix, has, other, want = b, db, a, da } + proven := (&valueProof{}).proveColumnItem(fix).not[want] == "" switch { - case da != DataTypeString && db != DataTypeString: - return bothTyped(a, da, b, db) - case typed.datatype == DataTypeString && (&valueProof{}).proveColumnItem(bare).not[want] != "": - return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, bare.format, dataTypeNouns[want], typed.format, asText(typed)) - case bare.fromString: // an object may carry a weight, which this spelling would drop - return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, bare.format, bare.format, want) - } - return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, bare.format, want, want) -} - -// bothTyped names the fix for two items holding different datatypes: the one taking its datatype -// from the column it reads is typed as the other where its values prove it, else read as text. -func bothTyped(a *template, da DataType, b *template, db DataType) error { - read, other := a, db - if a.datatype != DataTypeString { - read, other = b, da - } - switch { - case read.datatype != DataTypeString: + case fix.datatype != DataTypeString: return fmt.Errorf("its items hold %s and %s; a column holds one datatype", da, db) - case (&valueProof{}).proveColumnItem(read).not[other] == "": - return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", da, db, typedAs(read, other)) + case has != DataTypeString && proven: + return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", da, db, typedAs(fix, want)) + case has != DataTypeString: + return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", da, db, fix.format, asText(fix)) + case other.datatype == DataTypeString && !proven: + return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) + case fix.fromString: // an object may carry a weight, which this spelling would drop + return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, fix.format, fix.format, want) } - return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", da, db, read.format, asText(read)) + return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, fix.format, want, want) } // typedAs names the spelling giving a column-read item datatype d, keeping the other keys an object diff --git a/todo.md b/todo.md index e9e2783..0a08c1e 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Make `disagreement` and `bothTyped` a table of cases: each swaps its operands and - runs a proof to pick an error's wording. - Derive a column's datatype in one place: `columnItems` yields nothing for a `tableColumn`, and six callers re-derive it from items. - Give `head`, `index`, `bound` and `draw` one meaning each, name `table.family` -- 2.52.0 From 86118e27101cb82af3fdddd2f9b105ffc678417b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:00:39 +0200 Subject: [PATCH 182/966] Name the both-declared case in disagreement's comment --- datatype.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/datatype.go b/datatype.go index 028cb65..80e1ede 100644 --- a/datatype.go +++ b/datatype.go @@ -158,7 +158,7 @@ func columnItems(n node) (items []*template, nullable bool) { } // disagreement names the fix for two items of one column holding different datatypes. The item to -// fix declares none, and is text where the other reads a typed column. +// fix declares none unless both do, and is text where the other reads a typed column. func disagreement(a *template, da DataType, b *template, db DataType) error { fix, has, other, want := a, da, b, db if da != DataTypeString && (a.datatype != DataTypeString || db == DataTypeString) { -- 2.52.0 From b2ffac53178ab03a8f6272b7c290f59190186917 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:15:09 +0200 Subject: [PATCH 183/966] Derive a column's datatype in columnDatatype alone --- datatype.go | 47 +++++++++++++++++------------------------------ record.go | 3 +-- todo.md | 2 -- value.go | 4 ++-- 4 files changed, 20 insertions(+), 36 deletions(-) diff --git a/datatype.go b/datatype.go index 80e1ede..5d2cc13 100644 --- a/datatype.go +++ b/datatype.go @@ -82,8 +82,14 @@ func checkColumns(s nodeScope) error { } } } - if _, err := columnDatatype(column); err != nil { - return fmt.Errorf("%s: field %q: %w", path, name, err) + if len(items) == 0 { + return nil + } + first := itemDatatype(items[0]) + for _, it := range items[1:] { + if d := itemDatatype(it); d != first { + return fmt.Errorf("%s: field %q: %w", path, name, disagreement(items[0], first, it, d)) + } } return nil } @@ -101,41 +107,22 @@ func checkColumns(s nodeScope) error { }) } -// columnDatatype is the datatype a column's items hold. They must agree, since a -// column holds one; a column only ever null is a string. -func columnDatatype(n node) (DataType, error) { +// columnDatatype is the datatype a column holds, which its first item decides: checkColumns refuses +// a column whose items disagree. A table's column, like one only ever null, has no item and is a string. +func columnDatatype(n node) DataType { items, _ := columnItems(n) if len(items) == 0 { - return DataTypeString, nil + return DataTypeString } - first := itemDatatype(items[0]) - for _, t := range items[1:] { - if d := itemDatatype(t); d != first { - return first, disagreement(items[0], first, t, d) - } - } - return first, nil + return itemDatatype(items[0]) } -// itemDatatype is the datatype a column item declares, else that of the column it is. +// itemDatatype is the datatype a column item declares, else that of the column it reads. func itemDatatype(t *template) DataType { - if t.datatype != DataTypeString { - return t.datatype + if t.datatype == DataTypeString && t.readsColumn != nil { + return columnDatatype(t.readsColumn.column) } - return readDatatype(t) -} - -// readDatatype is the datatype of the column t is, reading it by one reference alone; a string -// when t reads none. -func readDatatype(t *template) DataType { - if t.readsColumn == nil { - return DataTypeString - } - items, _ := columnItems(t.readsColumn.column) - if len(items) == 0 { - return DataTypeString - } - return itemDatatype(items[0]) // checkColumns refuses that column where its items disagree + return t.datatype } // columnItems is a column's template items, its choices unwrapped, and whether one is null. diff --git a/record.go b/record.go index f8016bd..67ef410 100644 --- a/record.go +++ b/record.go @@ -254,8 +254,7 @@ func recordOf(n node) (*template, []Column, error) { } columns := make([]Column, len(names)) for i, name := range names { - datatype, _ := columnDatatype(t.fields[name]) // checkColumns refused items that disagree wherever DataType is read - columns[i] = Column{Name: name, DataType: datatype} + columns[i] = Column{Name: name, DataType: columnDatatype(t.fields[name])} } return t, columns, nil } diff --git a/todo.md b/todo.md index 0a08c1e..ce50901 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Derive a column's datatype in one place: `columnItems` yields nothing for a - `tableColumn`, and six callers re-derive it from items. - Give `head`, `index`, `bound` and `draw` one meaning each, name `table.family` for the root it returns, and replace the token kind bytes `'l'`, `'t'`, `'f'` and `'b'`, whose `'l'` means different things in `ftoken` and `formatToken`. diff --git a/value.go b/value.go index 32718e4..89de4d8 100644 --- a/value.go +++ b/value.go @@ -34,8 +34,8 @@ func (p *valueProof) checkDatatype(path string, n node) error { if !ok || t.datatype == DataTypeString { return nil } - if d := readDatatype(t); d == t.datatype { - return fmt.Errorf(`%s: %s takes datatype %s from the column it reads; drop "datatype"`, path, t.format, d) + if r := t.readsColumn; r != nil && columnDatatype(r.column) == t.datatype { + return fmt.Errorf(`%s: %s takes datatype %s from the column it reads; drop "datatype"`, path, t.format, t.datatype) } if reason := p.proveColumnItem(t).not[t.datatype]; reason != "" { return fmt.Errorf("%s: datatype %s: %s", path, t.datatype, reason) -- 2.52.0 From c6a0ef2e989f6909a74d18e63234f3a7402075b7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:18:14 +0200 Subject: [PATCH 184/966] Read checkColumns' first datatype through columnDatatype --- datatype.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/datatype.go b/datatype.go index 5d2cc13..8dbf0c3 100644 --- a/datatype.go +++ b/datatype.go @@ -85,7 +85,7 @@ func checkColumns(s nodeScope) error { if len(items) == 0 { return nil } - first := itemDatatype(items[0]) + first := columnDatatype(column) for _, it := range items[1:] { if d := itemDatatype(it); d != first { return fmt.Errorf("%s: field %q: %w", path, name, disagreement(items[0], first, it, d)) -- 2.52.0 From cb064362d1a67d3cb09b0bb5d1595808a296153e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:36:04 +0200 Subject: [PATCH 185/966] Name the tests' held reads held, keeping bound for a numeric bound --- bench_test.go | 16 ++++++++-------- hold_test.go | 26 +++++++++++++------------- loading_test.go | 2 +- reference_test.go | 4 ++-- 4 files changed, 24 insertions(+), 24 deletions(-) diff --git a/bench_test.go b/bench_test.go index 3df242a..585b132 100644 --- a/bench_test.go +++ b/bench_test.go @@ -57,29 +57,29 @@ func BenchmarkRepeat(b *testing.B) { benchPath(b, dir, "many") } -// BenchmarkBound measures a format that binds a head and reads two paths from it — -// what a correlated pair costs against BenchmarkUnbound, the same output drawn from +// BenchmarkHeld measures a format that holds a head and reads two paths from it — +// what a correlated pair costs against BenchmarkUnheld, the same output drawn from // two independent fields. -func BenchmarkBound(b *testing.B) { +func BenchmarkHeld(b *testing.B) { dir := tmpData(b, "addr", `{"format":"{place.postal-code} {place.locality}","place":[{"format":"{locality}","locality":"Stockholm","postal-code":"1{digits(2)} {digits(2)}"},{"format":"{locality}","locality":"Tranås","postal-code":"573 {digits(2)}"}]}`) benchPath(b, dir, "addr") } -func BenchmarkUnbound(b *testing.B) { +func BenchmarkUnheld(b *testing.B) { dir := tmpData(b, "addr", `{"format":"{postal-code} {locality}","postal-code":["1{digits(2)} {digits(2)}","573 {digits(2)}"],"locality":["Stockholm","Tranås"]}`) benchPath(b, dir, "addr") } -// BenchmarkBoundDeep reads two paths through an intermediate level, the shape the +// BenchmarkHeldDeep reads two paths through an intermediate level, the shape the // depth question is about. -func BenchmarkBoundDeep(b *testing.B) { +func BenchmarkHeldDeep(b *testing.B) { dir := tmpData(b, "addr", `{"format":"{p.addr.city} {p.addr.zip}","p":{"format":"{addr}","addr":{"format":"{city}","city":["Stockholm","Tranås"],"zip":"1{digits(2)} {digits(2)}"}}}`) benchPath(b, dir, "addr") } -// BenchmarkBoundWide reads ten paths from one row: the point where the maps an +// BenchmarkHeldWide reads ten paths from one row: the point where the maps an // expansion holds outgrow a single bucket. -func BenchmarkBoundWide(b *testing.B) { +func BenchmarkHeldWide(b *testing.B) { dir := tmpData(b, "row", `{"format":"{r.a}{r.b}{r.c}{r.d}{r.e}{r.f}{r.g}{r.h}{r.i}{r.j}","r":[ {"format":"x","a":"1","b":"2","c":"3","d":"4","e":"5","f":"6","g":"7","h":"8","i":"9","j":"0"}, {"format":"y","a":"A","b":"B","c":"C","d":"D","e":"E","f":"F","g":"G","h":"H","i":"I","j":"J"}]}`) diff --git a/hold_test.go b/hold_test.go index 0decffd..a0aeee4 100644 --- a/hold_test.go +++ b/hold_test.go @@ -68,8 +68,8 @@ func TestRenderingALevelAndReadingIntoItIsRejected(t *testing.T) { } } -func TestCalcOperandNamingABoundLevelIsRejected(t *testing.T) { - // A calc operand renders its field, so naming a bound level in one is the same +func TestCalcOperandNamingAHeldLevelIsRejected(t *testing.T) { + // A calc operand renders its field, so naming a held level in one is the same // overlap as a bare token: {calc(item * 1)} renders what {item.price} reads a // path into, and the two disagree. _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ @@ -87,8 +87,8 @@ func TestCalcOperandNamingABoundLevelIsRejected(t *testing.T) { } } -func TestReferenceNamingABoundLevelIsRejected(t *testing.T) { - // A reference can name a bound level from the data root, which renders it +func TestReferenceNamingAHeldLevelIsRejected(t *testing.T) { + // A reference can name a held level from the data root, which renders it // afresh beside the path that reads its held draw — the same overlap by // another spelling. // A category never references itself, so each reference back into cat sits in a @@ -102,7 +102,7 @@ func TestReferenceNamingABoundLevelIsRejected(t *testing.T) { "cat": `{"format":"{p.addr}|{/hop}","p":{"format":"x","addr":["A","B","C","D"]}}`, "hop": `"{/cat.p.addr}"`, }, - // The reference need not sit in the format that binds: any field it renders + // The reference need not sit in the format that holds the level: any field it renders // reaches the level just the same, however deep. "reference from a sibling field": { "cat": `{"format":"{p.first}|{inner}","p":[` + @@ -122,7 +122,7 @@ func TestReferenceNamingABoundLevelIsRejected(t *testing.T) { "cat": `{"format":"{p.first} {/surname}","p":{"format":"{first}","first":["Anna","Bo"]}}`, "surname": `["Eriksson","Lindqvist"]`, }))); err != nil { - t.Errorf("New = %v, want a reference outside the bound level accepted", err) + t.Errorf("New = %v, want a reference outside the held level accepted", err) } } @@ -179,7 +179,7 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { }, `{q} renders "net"`, }, - // A reference ending at the operand binds the choice itself, not a variant, + // A reference ending at the operand holds the choice itself, not a variant, // so wrapping the operand in one changes nothing. "a reference to an operand wrapped in a choice": { map[string]string{ @@ -344,7 +344,7 @@ func TestALevelAPathNeverRendersIsAccepted(t *testing.T) { } func TestADeepDiamondChainLoads(t *testing.T) { - // A diamond chain is 2^n routes through n nodes. The search past a bound level + // A diamond chain is 2^n routes through n nodes. The search past a held level // must walk each node once, not once per route, or a deep chain never loads. files := map[string]string{"l0": `"x"`} for i := 1; i <= 30; i++ { @@ -359,7 +359,7 @@ func TestADeepDiamondChainLoads(t *testing.T) { func TestASharedNodeIsWalkedOnce(t *testing.T) { // Two fields reaching one node make the render graph a diamond, not a tree. - // The search past a bound level must take that in its stride rather than walk + // The search past a held level must take that in its stride rather than walk // the shared node once per route. f, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ "cat": `{"format":"{p.first}|{q}","p":{"format":"{first}","first":["Anna","Bo"]},` + @@ -399,7 +399,7 @@ func TestReferenceToAMatchingStringIsAccepted(t *testing.T) { } func TestNestedChoiceDrawsOneVariant(t *testing.T) { - // A choice item may itself be a choice, so drawing a bound head unwraps until + // A choice item may itself be a choice, so drawing a held head unwraps until // it reaches a value. Stopping at one level leaves a choice where the walk // expects a template. f := engine(21) @@ -478,7 +478,7 @@ func TestFieldWithoutAPathTokenStillDrawsEachTime(t *testing.T) { } } -func TestTwoBoundHeadsDrawIndependently(t *testing.T) { +func TestTwoHeldHeadsDrawIndependently(t *testing.T) { // Binding is per head, so two correlated sets in one format compose instead // of needing a cross product of variants. f := engine(4) @@ -538,7 +538,7 @@ func TestNestedTemplateKeepsItsOwnBinding(t *testing.T) { } } -func TestBoundDrawIsSeedStable(t *testing.T) { +func TestHeldDrawIsSeedStable(t *testing.T) { tmpl := places("{place.postal-code} {place.locality}") a := mustRender(t, engine(42), tmpl) b := mustRender(t, engine(42), tmpl) @@ -721,7 +721,7 @@ func TestCycleThroughAPathTokenIsRejected(t *testing.T) { } } -func TestBoundPathIsReachableByFake(t *testing.T) { +func TestHeldPathIsReachableByFake(t *testing.T) { // Binding changes how a format reads a sibling, not what List and Fake offer: // the sub-fields stay addressable on their own. dir := writeData(t, map[string]string{"address": places("{place.postal-code} {place.locality}")}) diff --git a/loading_test.go b/loading_test.go index daeefad..d8cdac8 100644 --- a/loading_test.go +++ b/loading_test.go @@ -19,7 +19,7 @@ func TestList(t *testing.T) { "person": `{"format":"{first} {last}","first":"A","last":"B"}`, "word": `["x", "y"]`, "geo/city": `"Z"`, - // A bound {/path} reference is a render edge, not an addressable field. + // A {/path} reference is a render edge, not an addressable field. "greeting": `{"format":"hej {/person.first} and {own}","own":"x"}`, // Only the fields every variant carries are addressable, so "extra" is not. "coin": `[{"format":"{code}","code":"A","name":"Aa"},{"format":"{code}","code":"B","name":"Bb","extra":"x"}]`, diff --git a/reference_test.go b/reference_test.go index e4e1705..5ea685f 100644 --- a/reference_test.go +++ b/reference_test.go @@ -102,7 +102,7 @@ func TestReferenceErrors(t *testing.T) { "a category in a subfolder referencing itself": {"sv_SE/a": `"x{/sv_SE.a}"`}, "mutual cycle within a subfolder": {"sv_SE/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/sv_SE.a}"`}, "mutual cycle across two folders": {"en_US/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/en_US.a}"`}, - // ".." is reserved for bound references, so an authored key using it would + // ".." is reserved for references, so an authored key using it would // name a node nothing can reach and nothing would validate. "field key using the reference prefix": {"cat": `{"format":"hi","..x":"{/nope}"}`}, } @@ -187,7 +187,7 @@ func TestNewErrorIsDeterministic(t *testing.T) { } // TestNewErrorPathIsCanonical pins the node path a load error names: a choice arm -// adds no segment, and a bound {/path} reference is not a containment segment at +// adds no segment, and a {/path} reference is not a containment segment at // all, so a bad reference is reported against the node that holds it. func TestNewErrorPathIsCanonical(t *testing.T) { cases := []struct { -- 2.52.0 From affbb9ac6f10c30234213f4e27bdb212b43a1520 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:36:04 +0200 Subject: [PATCH 186/966] Give head, index, bound and draw one meaning each, name familyRoot, and type the token kinds --- doc.go | 6 ++-- drawfence.go | 4 +-- format.go | 97 ++++++++++++++++++++++++++++------------------------ graph.go | 4 +-- hold.go | 20 +++++------ holdfence.go | 74 +++++++++++++++++++-------------------- holdset.go | 4 +-- node.go | 14 ++++---- path.go | 6 ++-- reference.go | 12 +++---- render.go | 8 ++--- table.go | 40 +++++++++++----------- todo.md | 3 -- value.go | 2 +- 14 files changed, 149 insertions(+), 145 deletions(-) diff --git a/doc.go b/doc.go index f6a59dd..8109d45 100644 --- a/doc.go +++ b/doc.go @@ -16,8 +16,8 @@ package fejkdata // Vocabulary // -// - draw — one pick from a choice, or one row taken from a table, and the value a -// read produced: `pick`, `resolveChoice`, `table.drawRow`, `draw`. +// - draw — one pick from a choice, or one row taken from a table: `pick`, +// `resolveChoice`, `table.drawRow`. // - expansion — one render of one format: `expand`. // - render — what owns one `holdSet`, and so what one reference draw spans: // `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` @@ -31,7 +31,7 @@ package fejkdata // - pin — fixing which row of a table the render uses, which the draw fences replay: // `pinSet`, `pinSet.pin`, `table.drawIn`. // - family — a table and every table reaching it through a chain of -// `table.parentT`, named by the root that chain ends at: `table.family`. +// `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // - whole — a read of a table with no selector and no descent, {/city}, landing on // the `table` itself: it draws a row apart from every pin, so the fences keep the rows // its cells stand in as `drawAt.wholePins`, beside `drawAt.wholeTable`. diff --git a/drawfence.go b/drawfence.go index 01adc1e..2988095 100644 --- a/drawfence.go +++ b/drawfence.go @@ -97,7 +97,7 @@ func checkOwnFamily(t *template) error { seen[n] = true for _, e := range renderEdges(n) { if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).head(a.key).(*table); isTable && head.family() == own.family() { + if head, isTable := n.(*template).head(a.key).(*table); isTable && head.familyRoot() == own.familyRoot() { return e, head, true } } @@ -357,7 +357,7 @@ func checkFamilies(reads []pathRead) error { continue } for _, o := range reads[:i] { - if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.family() != r.tr.headTable.family() { + if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { continue } if err := checkFamilyPair(o, r); err != nil { diff --git a/format.go b/format.go index 1fac5ad..7fcc43c 100644 --- a/format.go +++ b/format.go @@ -8,9 +8,9 @@ import ( // ftoken is one unit of a scanned format string: a literal rune or the body of a // {…} token. type ftoken struct { - kind byte // 'l' literal rune, 't' token body - char rune - body string + isToken bool // a {…} body, else a literal rune + char rune + body string } // eachToken scans a format string once and calls fn for each unit, the single @@ -24,7 +24,7 @@ func eachToken(format string, fn func(ftoken) error) error { switch c := rs[i]; c { case '{': if i+1 < len(rs) && rs[i+1] == '{' { - t.kind, t.char = 'l', '{' + t.char = '{' i++ break } @@ -38,17 +38,17 @@ func eachToken(format string, fn func(ftoken) error) error { if end >= len(rs) { return fmt.Errorf("unterminated '{' in %q", format) } - t.kind, t.body = 't', string(rs[i+1:end]) + t.isToken, t.body = true, string(rs[i+1:end]) i = end case '}': if i+1 < len(rs) && rs[i+1] == '}' { - t.kind, t.char = 'l', '}' + t.char = '}' i++ break } return fmt.Errorf("lone '}' in %q; a literal brace is written }}", format) default: - t.kind, t.char = 'l', c + t.char = c } if err := fn(t); err != nil { return err @@ -57,17 +57,24 @@ func eachToken(format string, fn func(ftoken) error) error { return nil } -// formatToken is one parsed unit of a format: a literal run, a field alternation or a -// builtin call. +// formatToken is one parsed unit of a format. type formatToken struct { - kind byte // 'l' literal run, 'f' field alternation, 'b' builtin call - lit string // kind 'l' + kind tokenKind + lit string // literalRun body string // the braces' content, as written - fn string // kind 'b' - args []string // kind 'b' - names []string // kind 'f': the '|' arms; kind 'b': the operands its builtin reads + fn string // builtinCall + args []string // builtinCall + names []string // fieldAlternation: the '|' arms; builtinCall: the operands its builtin reads } +type tokenKind uint8 + +const ( + builtinCall tokenKind = iota + fieldAlternation + literalRun +) + // parseFormat is the one reading of a format's tokens: a '(' outside a selector makes // a token a call. func parseFormat(format string) ([]formatToken, error) { @@ -75,25 +82,25 @@ func parseFormat(format string) ([]formatToken, error) { var lit strings.Builder flush := func() { if lit.Len() > 0 { - toks = append(toks, formatToken{kind: 'l', lit: lit.String()}) + toks = append(toks, formatToken{kind: literalRun, lit: lit.String()}) lit.Reset() } } err := eachToken(format, func(t ftoken) error { - if t.kind == 'l' { + if !t.isToken { lit.WriteRune(t.char) return nil } flush() if indexOutside(t.body, '(') < 0 { - toks = append(toks, formatToken{kind: 'f', body: t.body, names: splitOutside(t.body, '|')}) + toks = append(toks, formatToken{kind: fieldAlternation, body: t.body, names: splitOutside(t.body, '|')}) return nil } name, args, ok := funcCall(t.body) if !ok { return fmt.Errorf("malformed function token {%s}", t.body) } - toks = append(toks, formatToken{kind: 'b', body: t.body, fn: name, args: args, names: builtinOperands(name, args)}) + toks = append(toks, formatToken{kind: builtinCall, body: t.body, fn: name, args: args, names: builtinOperands(name, args)}) return nil }) if err != nil { @@ -207,9 +214,9 @@ func checkTokens(toks []formatToken, fields map[string]node) error { func checkToken(t formatToken, fields map[string]node) error { switch t.kind { - case 'b': + case builtinCall: return checkFunc(t, fields) - case 'f': + case fieldAlternation: for _, name := range t.names { if isRef(name) { if _, _, err := refShape(name); err != nil { @@ -236,7 +243,7 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { if err := checkSegments(a); err != nil { return err } - head, ok := fields[a.key] + field, ok := fields[a.key] if !ok { if a.key == "" { return fmt.Errorf("a name is never empty, so this token can name no field") @@ -253,7 +260,7 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { } return fmt.Errorf("no field %q", a.key) } - if err := checkPath(head, a.tail, a.key); err != nil { + if err := checkPath(field, a.tail, a.key); err != nil { return fmt.Errorf("field %q: %w", a.key, err) } return nil @@ -294,24 +301,24 @@ func checkNoRepeatedArm(body string, names []string) error { // arm is one alternative of a {a|b} token or one operand, split into the key // `template.head` resolves and the tail of a dotted path into it. A non-empty tail -// is what makes the arm a bound draw: its head is drawn once per expansion (see +// is what makes the arm a held draw: its head is drawn once per expansion (see // compileOps). type arm struct { spelling string // as written, for messages key string tail []string - levels []string // the key at each level the tail passes through, the head's first + levels []string // the key at each level the tail passes through, the arm's key first path string // key and tail, the one path every way of writing this read shares } // splitArm splits one name into key and tail. refs maps a reference to what -// linkRefs bound it to; before linking, a reference is whole. +// linkRefs resolved it to; before linking, a reference is whole. func splitArm(name string, refs map[string]refBinding) arm { if isRef(name) { - b, bound := refs[name] - if !bound || len(b.tail) == 0 { + b, linked := refs[name] + if !linked || len(b.tail) == 0 { key := name - if bound { + if linked { key = b.key } return arm{spelling: name, key: key, path: key} @@ -351,17 +358,17 @@ func checkSegments(a arm) error { return nil } -// callFn is a builtin bound to one call site: its args already parsed. It reads the +// callFn is a builtin prepared for one call site: its args already parsed. It reads the // output emitted so far in the current expansion (a derivation's payload) and the // values of the operands it named, which expand read for it. type callFn func(s *session, emitted string, operands []string) string // op is one compiled unit of a format string: a literal run, a field alternation, -// or a builtin already bound to its args. compile builds these so render never +// or a builtin already prepared with its args. compile builds these so render never // re-scans the format. type op struct { formatToken - arms []arm // kind 'f': the '|' alternatives, split into key and path once + arms []arm // fieldAlternation: the '|' alternatives, split into key and path once call callFn // operands are the fields the builtin reads, in the order its operands func // fixed; expand reads them before the call. nil for a builtin that reads none. @@ -369,18 +376,18 @@ type op struct { } // formatOps is a compiled format: its ops, the size of its literal text (to size -// the render buffer), and the names drawn once per expansion. bound maps each level +// the render buffer), and the names drawn once per expansion. pathLevels maps each level // a path reads into to the first such path; held is every such level plus the // fields an operand reads; holder maps each held name to the first reader holding // it, for error messages. The maps are nil when the format holds nothing, so data // that holds nothing carries no render-time cost. type formatOps struct { - ops []op - grow int - bound map[string]string - held map[string]bool - holder map[string]string - heldLocal bool // some held name is kept by the expansion itself rather than the render's hold + ops []op + grow int + pathLevels map[string]string + held map[string]bool + holder map[string]string + heldLocal bool // some held name is kept by the expansion itself rather than the render's hold } func (c *formatOps) holdName(a arm, label string) { @@ -396,11 +403,11 @@ func (c *formatOps) holdName(a arm, label string) { c.holder[a.key] = label } if len(a.tail) > 0 { - if c.bound == nil { - c.bound = map[string]string{} + if c.pathLevels == nil { + c.pathLevels = map[string]string{} } - if _, named := c.bound[a.key]; !named { - c.bound[a.key] = a.spelling + if _, named := c.pathLevels[a.key]; !named { + c.pathLevels[a.key] = a.spelling } } } @@ -432,12 +439,12 @@ func compileOps(toks []formatToken, refs map[string]refBinding) formatOps { var c formatOps for _, tok := range toks { switch tok.kind { - case 'l': + case literalRun: c.grow += len(tok.lit) c.ops = append(c.ops, op{formatToken: tok}) - case 'b': + case builtinCall: c.function(tok, refs) - case 'f': + case fieldAlternation: c.field(tok, refs) } } diff --git a/graph.go b/graph.go index bc7856d..fbaee34 100644 --- a/graph.go +++ b/graph.go @@ -18,7 +18,7 @@ func walkNodes(root map[string]node, fn func(path string, n node) error) error { } // eachNode visits n and every node contained within it once, passing the dot path -// that reaches each. It never crosses a reference edge — a bound {/path} field is +// that reaches each. It never crosses a reference edge — a {/path} reference is // skipped — so a single inline node is walked on its own. func eachNode(n node, path string, fn func(path string, n node) error) error { seen := map[node]bool{} @@ -105,7 +105,7 @@ type renderEdge struct { operand string } -// reached names an edge as the author spelled it, the vocabulary boundReaders uses +// reached names an edge as the author spelled it, the vocabulary levelReaders uses // for the sibling fence. func (e renderEdge) reached() string { if e.operand != "" { diff --git a/hold.go b/hold.go index 81ee2a0..18e1236 100644 --- a/hold.go +++ b/hold.go @@ -11,12 +11,12 @@ import ( // its sibling names, and a render one per group for its reference paths. type hold struct { variant map[string]node - value map[string]draw + value map[string]readValue pins pinSet } -// draw is what one read drew: its text, and whether it landed on a null. -type draw struct { +// readValue is what one read drew: its text, and whether it landed on a null. +type readValue struct { text string null bool } @@ -26,7 +26,7 @@ type draw struct { // kept, so {place.postal-code} and {place.locality} read one row, either read twice // gives one value, and a shown operand is the operand computed. Every other name is // drawn afresh, so {word} {word} still draws twice. -func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw { +func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { if s.trace != nil { traceRead(s.trace, t, sc, a) } @@ -34,7 +34,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw if len(a.tail) > 0 { panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) } - return draw{text: render(s, t.head(a.key), sc)} + return readValue{text: render(s, t.head(a.key), sc)} } d := readHold(held, sc, a) if r, done := d.value[a.path]; done { @@ -45,7 +45,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) draw } r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, pins: &d.pins, held: d, a: &a}), sc) if d.value == nil { - d.value = map[string]draw{} + d.value = map[string]readValue{} } d.value[a.path] = r return r @@ -81,20 +81,20 @@ func readHold(held *hold, sc renderScope, a arm) *hold { // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. -func renderLeaf(s *session, n node, sc renderScope) draw { +func renderLeaf(s *session, n node, sc renderScope) readValue { n = resolveChoice(s, n) switch leaf := n.(type) { case *nullItem: - return draw{null: true} + return readValue{null: true} case *template: if leaf.readsColumn != nil { return readField(s, leaf, nil, sc.in(leaf), leaf.readsColumn.a) } } - return draw{text: render(s, n, sc)} + return readValue{text: render(s, n, sc)} } -// resolveChoice resolves a choice to one variant, so a bound head is a concrete node the +// resolveChoice resolves a choice to one variant, so a held head is a concrete node the // rest of the expansion shares. Nested choices unwrap too: a draw is one value, not // another set to pick from. func resolveChoice(s *session, n node) node { diff --git a/holdfence.go b/holdfence.go index e3e749f..1327f7b 100644 --- a/holdfence.go +++ b/holdfence.go @@ -15,81 +15,81 @@ func heldCheck(path string, n node) error { if !ok || len(t.held) == 0 { return nil } - readers := boundReaders(t.ops, t.bound) - for _, head := range heldHeads(t) { - if _, isPath := t.bound[head]; isPath && isRef(head) { + readers := levelReaders(t.ops, t.pathLevels) + for _, name := range heldNames(t) { + if _, isPath := t.pathLevels[name]; isPath && isRef(name) { continue // held for the render: drawCheck compares every read of it, by path, across the render and its groups } - if err := checkHeadHeld(t, head, readers); err != nil { + if err := checkNameHeld(t, name, readers); err != nil { return fmt.Errorf("%s: %w", path, err) } } return nil } -// heldHeads lists a template's held names, operand heads first, then paths, each +// heldNames lists a template's held names, operands' first, then paths', each // in name order: a level read both ways is reported by the operand's fence, and // which overlap is reported does not vary. -func heldHeads(t *template) []string { - heads := make([]string, 0, len(t.held)) - for head := range t.held { - heads = append(heads, head) +func heldNames(t *template) []string { + names := make([]string, 0, len(t.held)) + for name := range t.held { + names = append(names, name) } - sort.Slice(heads, func(i, j int) bool { - _, pi := t.bound[heads[i]] - _, pj := t.bound[heads[j]] + sort.Slice(names, func(i, j int) bool { + _, pi := t.pathLevels[names[i]] + _, pj := t.pathLevels[names[j]] if pi != pj { return !pi } - return heads[i] < heads[j] + return names[i] < names[j] }) - return heads + return names } -// heldNodes is what the hold of head answers for: a path pins the levels it +// heldNodes is what the hold of name answers for: a path pins the levels it // passes through and the leaf it lands on, an operand exactly the value its render // produces. -func heldNodes(t *template, head string, readers []reader) map[node]bool { +func heldNodes(t *template, name string, readers []reader) map[node]bool { held := map[node]bool{} - if _, isPath := t.bound[head]; !isPath { - operandDraw(t.head(head), held) + if _, isPath := t.pathLevels[name]; !isPath { + operandDraw(t.head(name), held) return held } for _, r := range readers { - if a := splitArm(r.spelling, t.refs); a.key == head { - coverPath(t.head(head), a.tail, held) + if a := splitArm(r.spelling, t.refs); a.key == name { + coverPath(t.head(name), a.tail, held) } } return held } -// checkHeadHeld rejects every route to what head's hold pins except the readers +// checkNameHeld rejects every route to what name's hold pins except the readers // holding it. One seen set across the edges: a node that cannot reach the level // cannot reach it by another route either, so it is walked once. -func checkHeadHeld(t *template, head string, readers []reader) error { - held := heldNodes(t, head, readers) +func checkNameHeld(t *template, name string, readers []reader) error { + held := heldNodes(t, name, readers) if len(held) == 0 { return nil // a fixed head holds nothing to reach } - reader, isPath := t.bound[head] + reader, isPath := t.pathLevels[name] seen := map[node]bool{} for _, e := range renderEdges(t) { - if splitArm(e.label, t.refs).key == head || !renders(e.to, held, seen) { + if splitArm(e.label, t.refs).key == name || !renders(e.to, held, seen) { continue } if isPath { - return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), head, reader) + return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), name, reader) } - return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), head, operandReader(t, head)) + return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), name, operandReader(t, name)) } return nil } -// operandReader names the builtin whose operand holds head. -func operandReader(t *template, head string) string { +// operandReader names the builtin whose operand holds name. +func operandReader(t *template, name string) string { for _, o := range t.ops { for _, a := range o.operands { - if a.key == head { + if a.key == name { return o.fn } } @@ -177,8 +177,8 @@ func renders(n node, want, seen map[node]bool) bool { // disagree. Reads are compared in sorted order, so which pair is reported does not // depend on where the tokens sit. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkNoOverlap(ops []op, bound map[string]string) error { - names := boundReaders(ops, bound) +func checkNoOverlap(ops []op, pathLevels map[string]string) error { + names := levelReaders(ops, pathLevels) // Stable over one format-order scan, so two readers of one name (a token and a // calc operand both naming "p") are reported as the format writes them. sort.SliceStable(names, func(i, j int) bool { return names[i].path < names[j].path }) @@ -192,23 +192,23 @@ func checkNoOverlap(ops []op, bound map[string]string) error { return nil } -// reader is one way a format reaches a bound field: as written, by its path, +// reader is one way a format reaches a path level: as written, by its path, // and how to name it. type reader struct{ spelling, path, label string } -// boundReaders lists every way a format reaches a bound sibling field, in the order the +// levelReaders lists every way a format reaches a sibling path level, in the order the // format writes them. An operand renders its field, so it names a level exactly // as a token does; one scan finds both, which is what puts them in one order. -func boundReaders(ops []op, bound map[string]string) []reader { +func levelReaders(ops []op, pathLevels map[string]string) []reader { var names []reader for _, o := range ops { for _, a := range o.operands { - if _, isBound := bound[a.key]; isBound && !isRef(a.key) { + if _, isLevel := pathLevels[a.key]; isLevel && !isRef(a.key) { names = append(names, reader{a.spelling, a.path, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { - if _, isBound := bound[a.key]; isBound && !isRef(a.key) { + if _, isLevel := pathLevels[a.key]; isLevel && !isRef(a.key) { names = append(names, reader{a.spelling, a.path, "token {" + a.spelling + "}"}) } } diff --git a/holdset.go b/holdset.go index bae0d47..b03fb4a 100644 --- a/holdset.go +++ b/holdset.go @@ -21,7 +21,7 @@ type renderScope struct { } func eagerHoldSet() holdSet { - return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]draw{}}} + return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]readValue{}}} } // renderOnce renders n as one render, over a hold set of its own. @@ -67,7 +67,7 @@ func (sc renderScope) hold() *hold { if sc.set.named == nil { sc.set.named = map[string]*hold{} } - d = &hold{variant: map[string]node{}, value: map[string]draw{}} + d = &hold{variant: map[string]node{}, value: map[string]readValue{}} sc.set.named[strings.Clone(sc.group)] = d } return d diff --git a/node.go b/node.go index bb0975c..8e9f1a1 100644 --- a/node.go +++ b/node.go @@ -62,17 +62,17 @@ type template struct { // Filled by `template.compileFormat` at link, from `template.tokens` and `template.refs`: ops []op // what expand walks grow int // minimum output size, to size the render buffer - // bound maps each field the format addresses by dotted path to one path token + // pathLevels maps each field the format addresses by dotted path to one path token // reading it, which is the half of an overlap the fences name. nil when the // format takes no path. - bound map[string]string - // held is every name drawn once per expansion: the bound levels above, plus the + pathLevels map[string]string + // held is every name drawn once per expansion: the path levels above, plus the // siblings a {calc()} reads. nil when the format holds nothing (see expand). held map[string]bool heldLocal bool // some held name is kept by the expansion itself, so expand makes its hold // Filled by `linkTemplateRefs` and `keyDrawGroup`, from the assembled tree: - refs map[string]refBinding // each reference the format reads -> what it is bound to + refs map[string]refBinding // each reference the format reads -> what it resolves to refHeads map[string]node // each refBinding.key -> the category it names readsColumn *columnRead // set when the format is one reference alone reading a record's column drawGroupKey string // its draw group keyed by its category: what a render reads its reference paths under @@ -214,7 +214,7 @@ func (t *template) fixedText() (string, bool) { switch { case len(t.tokens) == 0: return "", true - case len(t.tokens) == 1 && t.tokens[0].kind == 'l': + case len(t.tokens) == 1 && t.tokens[0].kind == literalRun: return t.tokens[0].lit, true } return "", false @@ -224,8 +224,8 @@ func (t *template) fixedText() (string, bool) { // compiled reads. func (t *template) compileFormat() error { c := compileOps(t.tokens, t.refs) - t.ops, t.grow, t.bound, t.held, t.heldLocal = c.ops, c.grow, c.bound, c.held, c.heldLocal - if err := checkNoOverlap(t.ops, t.bound); err != nil { + t.ops, t.grow, t.pathLevels, t.held, t.heldLocal = c.ops, c.grow, c.pathLevels, c.held, c.heldLocal + if err := checkNoOverlap(t.ops, t.pathLevels); err != nil { return err } return checkNoRepeatedRead(c) diff --git a/path.go b/path.go index 17f9640..e07156b 100644 --- a/path.go +++ b/path.go @@ -343,11 +343,11 @@ type pathDraw struct { } // drawPath walks w over a path proved first, by a probe or at load, so the walk -// cannot fail. head names n in the panic. -func drawPath(n node, tail []string, head string, w *pathDraw) node { +// cannot fail. level names n in the panic. +func drawPath(n node, tail []string, level string, w *pathDraw) node { leaf, err := w.walk(n, tail) if err != nil { - panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(head, joinSegments(tail)), err)) + panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(level, joinSegments(tail)), err)) } return leaf } diff --git a/reference.go b/reference.go index fceff0f..089c8cb 100644 --- a/reference.go +++ b/reference.go @@ -9,7 +9,7 @@ import ( // data root, {.a} from the folder this file sits in, {..a} from the folder above. func isRef(name string) bool { return strings.HasPrefix(name, ".") || strings.HasPrefix(name, "/") } -// refBinding is what a reference was bound to: the head key its category is held +// refBinding is what a reference resolves to: the key its category is held // under, and the tail read into it. type refBinding struct { key string @@ -105,11 +105,11 @@ func (t *template) bindRefs(folder []string, path, category string, names []stri if err != nil { return fmt.Errorf("%s: reference {%s}: %w", path, name, err) } - head, target, tail, err := resolveCategory(root, segments) + categorySegs, target, tail, err := resolveCategory(root, segments) if err != nil { return fmt.Errorf("%s: reference {%s}: %w", path, name, err) } - key := "/" + strings.Join(head, ".") + key := "/" + strings.Join(categorySegs, ".") if category != "" && key == "/"+category { return fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) } @@ -144,7 +144,7 @@ func columnReadOf(t *template) *columnRead { // loneRef is the reference a format of one reference token and nothing else reads. func loneRef(toks []formatToken) (string, bool) { - if len(toks) != 1 || toks[0].kind != 'f' || len(toks[0].names) != 1 || !isRef(toks[0].names[0]) { + if len(toks) != 1 || toks[0].kind != fieldAlternation || len(toks[0].names) != 1 || !isRef(toks[0].names[0]) { return "", false } return toks[0].names[0], true @@ -187,8 +187,8 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t } // resolveCategory walks a dotted path through the folders to the category it -// names, returning that head, the node, and the tail left to read into it. -func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) { +// names, returning that category's segments, the node, and the tail left to read into it. +func resolveCategory(root map[string]node, segments []string) (category []string, target node, tail []string, err error) { var n node = &folder{children: root} i := 0 for ; i < len(segments); i++ { diff --git a/render.go b/render.go index 1275386..de13d17 100644 --- a/render.go +++ b/render.go @@ -114,17 +114,17 @@ func expand(s *session, t *template, sc renderScope) string { if t.heldLocal { held = &hold{ variant: make(map[string]node, len(t.held)), - value: make(map[string]draw, len(t.held)), + value: make(map[string]readValue, len(t.held)), } } for i := range t.ops { o := &t.ops[i] switch o.kind { - case 'l': + case literalRun: b.WriteString(o.lit) - case 'f': + case fieldAlternation: b.WriteString(readField(s, t, held, sc, o.arms[s.IntN(len(o.arms))]).text) - case 'b': + case builtinCall: // Read before the call, so the value a calc computes is the value the // format showed. calcVars fixed the order op.operands holds. var operands []string diff --git a/table.go b/table.go index 9c407d9..78e8383 100644 --- a/table.go +++ b/table.go @@ -32,12 +32,12 @@ type table struct { parentT *table children map[string]*table once sync.Once - index tableIndex + lookup rowLookup } -// tableIndex is what selection and linked draws look up, built on the first draw +// rowLookup is what selection and linked draws look up, built on the first draw // that needs it. -type tableIndex struct { +type rowLookup struct { byName map[string][]int rowsByParent map[string][]int childCum map[string][]float64 // cumulative weights per parent key, nil when uniform @@ -226,7 +226,7 @@ func (t *table) bindOptions(o tableOptionValues) error { if t.nameIndex >= 0 && t.keyIndex < 0 && t.parentIndex < 0 { return fmt.Errorf("name selects a row as a key does, and lists the rows it matches by their keys, so it needs a key column, or a parent inside which each name is one row; add key") } - if err := t.indexKeys(); err != nil { + if err := t.mapKeys(); err != nil { return err } if err := t.proveNamesInsideParent(); err != nil { @@ -251,8 +251,8 @@ func (t *table) proveNamesInsideParent() error { return nil } -// indexKeys proves every key names one row, and keeps the index a link is proved by. -func (t *table) indexKeys() error { +// mapKeys proves every key names one row, and keeps the map a link is proved by. +func (t *table) mapKeys() error { if t.keyIndex < 0 { return nil } @@ -329,7 +329,7 @@ func (t *table) compileRowFormat(format string) error { return err } for _, tok := range toks { - if tok.kind != 'f' { + if tok.kind != fieldAlternation { continue } for _, name := range tok.names { @@ -440,36 +440,36 @@ func (t *table) linkParent(path string, siblings map[string]node) error { return nil } -func (t *table) indexed() *tableIndex { +func (t *table) builtLookup() *rowLookup { t.once.Do(func() { if t.nameIndex >= 0 { - t.index.byName = make(map[string][]int, t.rowCount()) + t.lookup.byName = make(map[string][]int, t.rowCount()) for r := 0; r < t.rowCount(); r++ { n := t.cell(r, t.nameIndex) - t.index.byName[n] = append(t.index.byName[n], r) + t.lookup.byName[n] = append(t.lookup.byName[n], r) } } if t.parentIndex >= 0 { - t.index.rowsByParent = map[string][]int{} + t.lookup.rowsByParent = map[string][]int{} for r := 0; r < t.rowCount(); r++ { k := t.cell(r, t.parentIndex) - t.index.rowsByParent[k] = append(t.index.rowsByParent[k], r) + t.lookup.rowsByParent[k] = append(t.lookup.rowsByParent[k], r) } if t.cum != nil { - t.index.childCum = make(map[string][]float64, len(t.index.rowsByParent)) - for k, rows := range t.index.rowsByParent { + t.lookup.childCum = make(map[string][]float64, len(t.lookup.rowsByParent)) + for k, rows := range t.lookup.rowsByParent { cum, total := make([]float64, len(rows)), 0.0 for i, r := range rows { w, _ := strconv.ParseFloat(t.cell(r, t.weightIndex), 64) // sumWeights proved it total += w cum[i] = total } - t.index.childCum[k] = cum + t.lookup.childCum[k] = cum } } } }) - return &t.index + return &t.lookup } // drawRow picks a row over the whole table. The session is concrete rather than the rng @@ -488,7 +488,7 @@ func pickCum(s *session, cum []float64) int { // drawUnder picks a row among those linked to parent row pr. func (t *table) drawUnder(s *session, pr int) int { - ix := t.indexed() + ix := t.builtLookup() k := t.parentT.cell(pr, t.parentT.keyIndex) rows := ix.rowsByParent[k] if ix.childCum == nil { @@ -567,7 +567,7 @@ func (t *table) find(sel string, pins *pinSet) (int, error) { if r, ok := t.byKey[sel]; ok { return r, nil } - rows := t.indexed().byName[sel] + rows := t.builtLookup().byName[sel] if len(rows) > 1 { rows = pins.inside(t, rows) } @@ -614,8 +614,8 @@ func (t *table) selectorSpelling(r int) string { return fmt.Sprintf("%s line %d", t.file, r+2) } -// family is the table a chain of parents ends at. -func (t *table) family() *table { +// familyRoot is the table a chain of parents ends at. +func (t *table) familyRoot() *table { for t.parentT != nil { t = t.parentT } diff --git a/todo.md b/todo.md index ce50901..f6fbef1 100644 --- a/todo.md +++ b/todo.md @@ -21,9 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Give `head`, `index`, `bound` and `draw` one meaning each, name `table.family` - for the root it returns, and replace the token kind bytes `'l'`, `'t'`, `'f'` and - `'b'`, whose `'l'` means different things in `ftoken` and `formatToken`. - Start every render from one spelling of an empty hold set: `Fake` uses `holdSet{}`, the record and struct paths `eagerHoldSet()`. - Correct the false comments: `funcCall` names `checkFunc` where `parseFormat` diff --git a/value.go b/value.go index 89de4d8..4b405c6 100644 --- a/value.go +++ b/value.go @@ -157,7 +157,7 @@ func (p *valueProof) proveTemplate(t *template) proven { o := t.ops[0] body, name, args := o.body, o.fn, o.args switch _, isTransform := transforms[name]; { - case o.kind == 'f': + case o.kind == fieldAlternation: var leaves []node for _, a := range o.arms { leaves = append(leaves, pathLeaves(t.head(a.key), a.tail)...) -- 2.52.0 From 0225f169246690f48ff3ff5fa6fc716225814615 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:42:56 +0200 Subject: [PATCH 187/966] Name the tests' holds, header and row lookup by their one meaning, and drop the dead bound_test.go pointer --- calc_test.go | 2 +- hold_test.go | 16 ++++++++-------- perf_test.go | 2 +- table_test.go | 44 ++++++++++++++++++++++---------------------- 4 files changed, 32 insertions(+), 32 deletions(-) diff --git a/calc_test.go b/calc_test.go index a43c159..9f47d9c 100644 --- a/calc_test.go +++ b/calc_test.go @@ -125,7 +125,7 @@ func TestCalcGuardsPanic(t *testing.T) { // TestCalcOperandReadsTheExpansionsDraw pins the one-draw rule for a calc operand. // A field the format renders and a calc reads is drawn once per expansion, so the // operand shown is the operand computed — the correlation the dotted-path rule -// already gives a level (see bound_test.go), applied to a plain sibling. +// already gives a level, applied to a plain sibling. func TestCalcOperandReadsTheExpansionsDraw(t *testing.T) { dir := writeData(t, map[string]string{ "inv": `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00","100.00"],"qty":["2","3","7"]}`, diff --git a/hold_test.go b/hold_test.go index a0aeee4..f6ede64 100644 --- a/hold_test.go +++ b/hold_test.go @@ -117,7 +117,7 @@ func TestReferenceNamingAHeldLevelIsRejected(t *testing.T) { t.Errorf("%s: New = %v, want the reference rejected as an overlap", name, err) } } - // A reference to anything this format does not bind is untouched. + // A reference to anything this format does not hold is untouched. if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ "cat": `{"format":"{p.first} {/surname}","p":{"format":"{first}","first":["Anna","Bo"]}}`, "surname": `["Eriksson","Lindqvist"]`, @@ -479,7 +479,7 @@ func TestFieldWithoutAPathTokenStillDrawsEachTime(t *testing.T) { } func TestTwoHeldHeadsDrawIndependently(t *testing.T) { - // Binding is per head, so two correlated sets in one format compose instead + // A hold is per head, so two correlated sets in one format compose instead // of needing a cross product of variants. f := engine(4) seen := map[string]bool{} @@ -495,8 +495,8 @@ func TestTwoHeldHeadsDrawIndependently(t *testing.T) { } } -func TestRepeatRedrawsTheBinding(t *testing.T) { - // The binding lives for one expansion, so each repetition is a fresh draw — +func TestRepeatRedrawsTheHold(t *testing.T) { + // The hold lives for one expansion, so each repetition is a fresh draw — // and each repetition is internally consistent. f := engine(5) tmpl := strings.Replace(places("{place.postal-code} {place.locality}"), `"format":"{place.postal-code} {place.locality}"`, @@ -522,8 +522,8 @@ func TestRepeatRedrawsTheBinding(t *testing.T) { } } -func TestNestedTemplateKeepsItsOwnBinding(t *testing.T) { - // A binding belongs to the expansion that made it: an inner template's own +func TestNestedTemplateKeepsItsOwnHold(t *testing.T) { + // A hold belongs to the expansion that made it: an inner template's own // place is not the outer one's. f := engine(6) tmpl := `{"format":"{place.v}{inner}", @@ -534,7 +534,7 @@ func TestNestedTemplateKeepsItsOwnBinding(t *testing.T) { seen[mustRender(t, f, tmpl)] = true } if !seen["AB"] && !seen["BA"] { - t.Fatalf("400 draws produced %v, want the inner binding to be independent", seen) + t.Fatalf("400 draws produced %v, want the inner hold to be independent", seen) } } @@ -722,7 +722,7 @@ func TestCycleThroughAPathTokenIsRejected(t *testing.T) { } func TestHeldPathIsReachableByFake(t *testing.T) { - // Binding changes how a format reads a sibling, not what List and Fake offer: + // Holding changes how a format reads a sibling, not what List and Fake offer: // the sub-fields stay addressable on their own. dir := writeData(t, map[string]string{"address": places("{place.postal-code} {place.locality}")}) f, err := New(WithoutShippedData(), WithDataPath(dir), WithSeed(9)) diff --git a/perf_test.go b/perf_test.go index 83c477f..03389c1 100644 --- a/perf_test.go +++ b/perf_test.go @@ -98,7 +98,7 @@ func TestNoTableAllocRegression(t *testing.T) { {"two columns of one draw", "x", 10}, } { if allocs := testing.AllocsPerRun(10000, func() { f.Fake(s.path) }); allocs > s.base*1.10 { - t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a row index built per draw is the usual cause", s.name, allocs, s.base*1.10, s.base) + t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a row lookup built per draw is the usual cause", s.name, allocs, s.base*1.10, s.base) } } // Five linked tables, the depth a country's geo tree has: every pin must stay inline. diff --git a/table_test.go b/table_test.go index 32f869b..1ad0eef 100644 --- a/table_test.go +++ b/table_test.go @@ -819,13 +819,13 @@ func TestShippedTables(t *testing.T) { // TestEveryTerritoryNamesAShippedCountry proves what no parent can: the sovereign a // territory names is a row of the same table, which a link would make a cycle. func TestEveryTerritoryNamesAShippedCountry(t *testing.T) { - head, rows := shippedRows(t, "territory.tsv", "alpha2", "country") + header, rows := shippedRows(t, "territory.tsv", "alpha2", "country") keys := map[string]bool{} for _, r := range rows { - keys[r[head["alpha2"]]] = true + keys[r[header["alpha2"]]] = true } for i, r := range rows { - if c := r[head["country"]]; !keys[c] { + if c := r[header["country"]]; !keys[c] { t.Errorf("territory.tsv line %d: country %q is no alpha2 of the table", i+2, c) } } @@ -835,34 +835,34 @@ func TestEveryTerritoryNamesAShippedCountry(t *testing.T) { // every Chromium fork carries Chrome's token, so a mis-ordered pattern relabels a // row rather than dropping it. func TestEveryUserAgentColumnAgreesWithItsString(t *testing.T) { - head, rows := shippedRows(t, "useragent.tsv", "browser", "device", "os", "ua") + header, rows := shippedRows(t, "useragent.tsv", "browser", "device", "os", "ua") for _, token := range []struct{ in, browser string }{ {"Edg", "Edge"}, {"OPR/", "Opera"}, {"SamsungBrowser/", "Samsung Internet"}, {"CriOS/", "Chrome"}, } { for i, r := range rows { - if strings.Contains(r[head["ua"]], token.in) && r[head["browser"]] != token.browser { - t.Errorf("useragent.tsv line %d: %q carries %q but is labelled %q", i+2, r[head["ua"]], token.in, r[head["browser"]]) + if strings.Contains(r[header["ua"]], token.in) && r[header["browser"]] != token.browser { + t.Errorf("useragent.tsv line %d: %q carries %q but is labelled %q", i+2, r[header["ua"]], token.in, r[header["browser"]]) } } } for i, r := range rows { - if r[head["device"]] == "mobile" && r[head["os"]] != "Android" && r[head["os"]] != "iOS" { - t.Errorf("useragent.tsv line %d: a mobile row runs %q", i+2, r[head["os"]]) + if r[header["device"]] == "mobile" && r[header["os"]] != "Android" && r[header["os"]] != "iOS" { + t.Errorf("useragent.tsv line %d: a mobile row runs %q", i+2, r[header["os"]]) } } } // TestEveryTimezoneRowIsWellFormed scans the file rather than drawing: the draw is -// population-weighted, so it samples the populous head and leaves most rows unrendered. +// population-weighted, so it samples the populous zones and leaves most rows unrendered. func TestEveryTimezoneRowIsWellFormed(t *testing.T) { - head, rows := shippedRows(t, "timezone.tsv", "offset", "territory", "weight", "zone") + header, rows := shippedRows(t, "timezone.tsv", "offset", "territory", "weight", "zone") offset := regexp.MustCompile(`^[+-](0\d|1[0-4]):[0-5]\d$`) for i, r := range rows { - if !offset.MatchString(r[head["offset"]]) { - t.Errorf("timezone.tsv line %d: offset %q is no ±HH:MM", i+2, r[head["offset"]]) + if !offset.MatchString(r[header["offset"]]) { + t.Errorf("timezone.tsv line %d: offset %q is no ±HH:MM", i+2, r[header["offset"]]) } - if w, err := strconv.Atoi(r[head["weight"]]); err != nil || w < 1 { - t.Errorf("timezone.tsv line %d: weight %q is not a positive number", i+2, r[head["weight"]]) + if w, err := strconv.Atoi(r[header["weight"]]); err != nil || w < 1 { + t.Errorf("timezone.tsv line %d: weight %q is not a positive number", i+2, r[header["weight"]]) } } } @@ -871,10 +871,10 @@ func TestEveryTimezoneRowIsWellFormed(t *testing.T) { // most of 1438 rows unrendered, and the register wraps a right-to-left label in bidi // marks that belong to its display, not to the label. func TestEveryTldRowSpellsItsKey(t *testing.T) { - head, rows := shippedRows(t, "tld.tsv", "tld", "type", "unicode") + header, rows := shippedRows(t, "tld.tsv", "tld", "type", "unicode") types := map[string]bool{"country-code": true, "generic": true, "generic-restricted": true, "infrastructure": true, "sponsored": true} for i, r := range rows { - key, kind, shown := r[head["tld"]], r[head["type"]], r[head["unicode"]] + key, kind, shown := r[header["tld"]], r[header["type"]], r[header["unicode"]] if !types[kind] { t.Errorf("tld.tsv line %d: %s is typed %q, which the register assigns no TLD", i+2, key, kind) } @@ -900,24 +900,24 @@ func shippedRows(t *testing.T, file string, want ...string) (map[string]int, [][ t.Fatal(err) } lines := strings.Split(strings.TrimSuffix(string(data), "\n"), "\n") - head := map[string]int{} + header := map[string]int{} for i, name := range strings.Split(lines[0], "\t") { - head[name] = i + header[name] = i } for _, name := range want { - if _, ok := head[name]; !ok { + if _, ok := header[name]; !ok { t.Fatalf("%s columns %v, want %v", file, lines[0], want) } } rows := make([][]string, 0, len(lines)-1) for i, line := range lines[1:] { cells := strings.Split(line, "\t") - if len(cells) != len(head) { - t.Fatalf("%s line %d has %d cells, want %d", file, i+2, len(cells), len(head)) + if len(cells) != len(header) { + t.Fatalf("%s line %d has %d cells, want %d", file, i+2, len(cells), len(header)) } rows = append(rows, cells) } - return head, rows + return header, rows } // TestNamedTableWithoutAKeyResolvesInsideItsParent pins a table whose rows are -- 2.52.0 From 8f86de2187917e3989b8c0a2614f3f4d2810c153 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:44:06 +0200 Subject: [PATCH 188/966] Name pathKeys, pathKeyReaders, proveKeys and readValue's hold for what they hold, and give no token kind to the zero value --- AGENTS.md | 4 ++-- docs/decisions.md | 4 ++-- format.go | 26 +++++++++++++------------- graph.go | 2 +- hold.go | 4 ++-- holdfence.go | 26 +++++++++++++------------- node.go | 10 +++++----- reference.go | 2 +- table.go | 14 +++++++------- 9 files changed, 46 insertions(+), 46 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index cea2259..613c3c7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,7 +63,7 @@ In [docs/decisions.md](docs/decisions.md): - A column of one reference alone is the column it reads - A typed column's calc is refused unless proven - `Column` carries text, not a Go value -- `hold` names what a draw is kept in, `draw` the draw itself +- `hold` names what a draw is kept in, `readValue` what a read drew - One name, one meaning - The package stays flat - The performance gate asserts allocations, not wall-clock time @@ -77,7 +77,7 @@ In [docs/decisions.md](docs/decisions.md): - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields - A table is a record of string columns -- The key index is built at load, the rest on first draw +- The key map is built at load, the rest on first draw - Two categories may name one TSV - The rows of a table are alternatives - A table never reaches its own family, by any route diff --git a/docs/decisions.md b/docs/decisions.md index 6304093..0dd9e0b 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -302,7 +302,7 @@ in. `Value` is the rendered string beside `DataType` and `Null`, which each serializer writes as the load check proved it; a `Value any` would hand every caller a type switch. -## `hold` names what a draw is kept in, `draw` the draw itself +## `hold` names what a draw is kept in, `readValue` what a read drew Goal 2 wants a name to reach one unit, so a unit takes the stem of what it is, and a file the stem of the units it holds. @@ -439,7 +439,7 @@ Its columns are the CSV header and the `INSERT` column list, fixed by the TSV he a table is a record by construction; every column is a string until a typed column option earns its place. -## The key index is built at load, the rest on first draw +## The key map is built at load, the rest on first draw A link is proved against the parent's keys and a key's uniqueness is a data mistake, so both are load-time; the per-parent child lists serve only a draw, so they wait for the diff --git a/format.go b/format.go index 7fcc43c..16ef085 100644 --- a/format.go +++ b/format.go @@ -70,7 +70,7 @@ type formatToken struct { type tokenKind uint8 const ( - builtinCall tokenKind = iota + builtinCall tokenKind = iota + 1 fieldAlternation literalRun ) @@ -376,18 +376,18 @@ type op struct { } // formatOps is a compiled format: its ops, the size of its literal text (to size -// the render buffer), and the names drawn once per expansion. pathLevels maps each level -// a path reads into to the first such path; held is every such level plus the +// the render buffer), and the names drawn once per expansion. pathKeys maps each key +// a path starts from to the first such path; held is every such key plus the // fields an operand reads; holder maps each held name to the first reader holding // it, for error messages. The maps are nil when the format holds nothing, so data // that holds nothing carries no render-time cost. type formatOps struct { - ops []op - grow int - pathLevels map[string]string - held map[string]bool - holder map[string]string - heldLocal bool // some held name is kept by the expansion itself rather than the render's hold + ops []op + grow int + pathKeys map[string]string + held map[string]bool + holder map[string]string + heldLocal bool // some held name is kept by the expansion itself rather than the render's hold } func (c *formatOps) holdName(a arm, label string) { @@ -403,11 +403,11 @@ func (c *formatOps) holdName(a arm, label string) { c.holder[a.key] = label } if len(a.tail) > 0 { - if c.pathLevels == nil { - c.pathLevels = map[string]string{} + if c.pathKeys == nil { + c.pathKeys = map[string]string{} } - if _, named := c.pathLevels[a.key]; !named { - c.pathLevels[a.key] = a.spelling + if _, named := c.pathKeys[a.key]; !named { + c.pathKeys[a.key] = a.spelling } } } diff --git a/graph.go b/graph.go index fbaee34..252bcba 100644 --- a/graph.go +++ b/graph.go @@ -105,7 +105,7 @@ type renderEdge struct { operand string } -// reached names an edge as the author spelled it, the vocabulary levelReaders uses +// reached names an edge as the author spelled it, the vocabulary pathKeyReaders uses // for the sibling fence. func (e renderEdge) reached() string { if e.operand != "" { diff --git a/hold.go b/hold.go index 18e1236..a09951d 100644 --- a/hold.go +++ b/hold.go @@ -6,8 +6,8 @@ import ( ) // hold is what has already been drawn for held names: the variant each was drawn as, so every -// path under it reads one row; the draw each read made, by its path, so the same read -// written twice reads one draw; and the table rows those reads pinned. An expansion keeps one for +// path under it reads one row; the value each read produced, by its path, so the same read +// written twice reads one value; and the table rows those reads pinned. An expansion keeps one for // its sibling names, and a render one per group for its reference paths. type hold struct { variant map[string]node diff --git a/holdfence.go b/holdfence.go index 1327f7b..59be3f0 100644 --- a/holdfence.go +++ b/holdfence.go @@ -15,9 +15,9 @@ func heldCheck(path string, n node) error { if !ok || len(t.held) == 0 { return nil } - readers := levelReaders(t.ops, t.pathLevels) + readers := pathKeyReaders(t.ops, t.pathKeys) for _, name := range heldNames(t) { - if _, isPath := t.pathLevels[name]; isPath && isRef(name) { + if _, isPath := t.pathKeys[name]; isPath && isRef(name) { continue // held for the render: drawCheck compares every read of it, by path, across the render and its groups } if err := checkNameHeld(t, name, readers); err != nil { @@ -36,8 +36,8 @@ func heldNames(t *template) []string { names = append(names, name) } sort.Slice(names, func(i, j int) bool { - _, pi := t.pathLevels[names[i]] - _, pj := t.pathLevels[names[j]] + _, pi := t.pathKeys[names[i]] + _, pj := t.pathKeys[names[j]] if pi != pj { return !pi } @@ -51,7 +51,7 @@ func heldNames(t *template) []string { // produces. func heldNodes(t *template, name string, readers []reader) map[node]bool { held := map[node]bool{} - if _, isPath := t.pathLevels[name]; !isPath { + if _, isPath := t.pathKeys[name]; !isPath { operandDraw(t.head(name), held) return held } @@ -71,7 +71,7 @@ func checkNameHeld(t *template, name string, readers []reader) error { if len(held) == 0 { return nil // a fixed head holds nothing to reach } - reader, isPath := t.pathLevels[name] + reader, isPath := t.pathKeys[name] seen := map[node]bool{} for _, e := range renderEdges(t) { if splitArm(e.label, t.refs).key == name || !renders(e.to, held, seen) { @@ -177,8 +177,8 @@ func renders(n node, want, seen map[node]bool) bool { // disagree. Reads are compared in sorted order, so which pair is reported does not // depend on where the tokens sit. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkNoOverlap(ops []op, pathLevels map[string]string) error { - names := levelReaders(ops, pathLevels) +func checkNoOverlap(ops []op, pathKeys map[string]string) error { + names := pathKeyReaders(ops, pathKeys) // Stable over one format-order scan, so two readers of one name (a token and a // calc operand both naming "p") are reported as the format writes them. sort.SliceStable(names, func(i, j int) bool { return names[i].path < names[j].path }) @@ -192,23 +192,23 @@ func checkNoOverlap(ops []op, pathLevels map[string]string) error { return nil } -// reader is one way a format reaches a path level: as written, by its path, +// reader is one way a format reaches a path key: as written, by its path, // and how to name it. type reader struct{ spelling, path, label string } -// levelReaders lists every way a format reaches a sibling path level, in the order the +// pathKeyReaders lists every way a format reaches a sibling path key, in the order the // format writes them. An operand renders its field, so it names a level exactly // as a token does; one scan finds both, which is what puts them in one order. -func levelReaders(ops []op, pathLevels map[string]string) []reader { +func pathKeyReaders(ops []op, pathKeys map[string]string) []reader { var names []reader for _, o := range ops { for _, a := range o.operands { - if _, isLevel := pathLevels[a.key]; isLevel && !isRef(a.key) { + if _, isPathKey := pathKeys[a.key]; isPathKey && !isRef(a.key) { names = append(names, reader{a.spelling, a.path, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { - if _, isLevel := pathLevels[a.key]; isLevel && !isRef(a.key) { + if _, isPathKey := pathKeys[a.key]; isPathKey && !isRef(a.key) { names = append(names, reader{a.spelling, a.path, "token {" + a.spelling + "}"}) } } diff --git a/node.go b/node.go index 8e9f1a1..6ff24de 100644 --- a/node.go +++ b/node.go @@ -62,11 +62,11 @@ type template struct { // Filled by `template.compileFormat` at link, from `template.tokens` and `template.refs`: ops []op // what expand walks grow int // minimum output size, to size the render buffer - // pathLevels maps each field the format addresses by dotted path to one path token + // pathKeys maps each key a dotted path in the format starts from to one path token // reading it, which is the half of an overlap the fences name. nil when the // format takes no path. - pathLevels map[string]string - // held is every name drawn once per expansion: the path levels above, plus the + pathKeys map[string]string + // held is every name drawn once per expansion: the path keys above, plus the // siblings a {calc()} reads. nil when the format holds nothing (see expand). held map[string]bool heldLocal bool // some held name is kept by the expansion itself, so expand makes its hold @@ -224,8 +224,8 @@ func (t *template) fixedText() (string, bool) { // compiled reads. func (t *template) compileFormat() error { c := compileOps(t.tokens, t.refs) - t.ops, t.grow, t.pathLevels, t.held, t.heldLocal = c.ops, c.grow, c.pathLevels, c.held, c.heldLocal - if err := checkNoOverlap(t.ops, t.pathLevels); err != nil { + t.ops, t.grow, t.pathKeys, t.held, t.heldLocal = c.ops, c.grow, c.pathKeys, c.held, c.heldLocal + if err := checkNoOverlap(t.ops, t.pathKeys); err != nil { return err } return checkNoRepeatedRead(c) diff --git a/reference.go b/reference.go index 089c8cb..4814a10 100644 --- a/reference.go +++ b/reference.go @@ -188,7 +188,7 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t // resolveCategory walks a dotted path through the folders to the category it // names, returning that category's segments, the node, and the tail left to read into it. -func resolveCategory(root map[string]node, segments []string) (category []string, target node, tail []string, err error) { +func resolveCategory(root map[string]node, segments []string) (categorySegs []string, target node, tail []string, err error) { var n node = &folder{children: root} i := 0 for ; i < len(segments); i++ { diff --git a/table.go b/table.go index 78e8383..7ba2b2b 100644 --- a/table.go +++ b/table.go @@ -226,7 +226,7 @@ func (t *table) bindOptions(o tableOptionValues) error { if t.nameIndex >= 0 && t.keyIndex < 0 && t.parentIndex < 0 { return fmt.Errorf("name selects a row as a key does, and lists the rows it matches by their keys, so it needs a key column, or a parent inside which each name is one row; add key") } - if err := t.mapKeys(); err != nil { + if err := t.proveKeys(); err != nil { return err } if err := t.proveNamesInsideParent(); err != nil { @@ -251,8 +251,8 @@ func (t *table) proveNamesInsideParent() error { return nil } -// mapKeys proves every key names one row, and keeps the map a link is proved by. -func (t *table) mapKeys() error { +// proveKeys proves every key names one row, and keeps the map a link is proved by. +func (t *table) proveKeys() error { if t.keyIndex < 0 { return nil } @@ -488,13 +488,13 @@ func pickCum(s *session, cum []float64) int { // drawUnder picks a row among those linked to parent row pr. func (t *table) drawUnder(s *session, pr int) int { - ix := t.builtLookup() + lookup := t.builtLookup() k := t.parentT.cell(pr, t.parentT.keyIndex) - rows := ix.rowsByParent[k] - if ix.childCum == nil { + rows := lookup.rowsByParent[k] + if lookup.childCum == nil { return rows[s.IntN(len(rows))] } - return rows[pickCum(s, ix.childCum[k])] + return rows[pickCum(s, lookup.childCum[k])] } // drawIn is the render's row of t: the one pinned in p, else one drawn inside the nearest -- 2.52.0 From ce469cb3a34561438c03608b1d4afcb4859048d5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 11:52:40 +0200 Subject: [PATCH 189/966] State the row lookup whole in the key-map decision --- docs/decisions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 0dd9e0b..d66bd23 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -442,8 +442,8 @@ option earns its place. ## The key map is built at load, the rest on first draw A link is proved against the parent's keys and a key's uniqueness is a data mistake, so -both are load-time; the per-parent child lists serve only a draw, so they wait for the -first one, keeping `New` linear in the bytes read. +both are load-time; the row lookup, by name and by parent, serves only a draw, so it waits +for the first one, keeping `New` linear in the bytes read. ## Two categories may name one TSV -- 2.52.0 From 1d73f1dc906edaf5ab5b4365ccf00ce888a25278 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 18:45:07 +0200 Subject: [PATCH 190/966] Start the record test root from the zero hold set --- drawfence_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drawfence_test.go b/drawfence_test.go index 3b6c23a..d4e0eb9 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -139,7 +139,7 @@ func renderRoot(s *session, t *template) { } func renderRecordRoot(s *session, t *template, columns []Column) { - set := eagerHoldSet() + var set holdSet sc := renderScope{set: &set} if t.table != nil { t.table.drawIn(s, &sc.hold().pins) -- 2.52.0 From 1b9cf00a264cd36116e6fa90ea5aea1900f0417e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 18:45:07 +0200 Subject: [PATCH 191/966] Start every render from the zero hold set --- AGENTS.md | 2 +- docs/decisions.md | 15 ++++++++------- holdset.go | 4 ---- record.go | 4 ++-- struct.go | 2 +- todo.md | 2 -- 6 files changed, 12 insertions(+), 17 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 613c3c7..8fc3f98 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,7 +55,7 @@ In [docs/decisions.md](docs/decisions.md): - A render shares one reference draw per category, per group - A draw group name is local to its category - The expansion hold and the render's draws are two fences -- A record makes its draw maps up front, a `Fake` on its first read +- A hold set makes its draw maps on the first read - A category never references itself, and a record's fences run at load - A record's column set is fixed before the first draw - Null is a `null` item, not a rate diff --git a/docs/decisions.md b/docs/decisions.md index d66bd23..8358cc6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -242,14 +242,15 @@ groups. They pin different things — an operand pins the value its own render p and stops at a reference, a path pins every level it passes through — so one walk would carry both rules and both scopes anyway, and tell them apart at every step. -## A record makes its draw maps up front, a `Fake` on its first read +## A hold set makes its draw maps on the first read -A record's columns always read through the render's draws, so making the maps where the -set is declared keeps them on that frame's stack. A `Fake` often reads no reference path -at all, and making them anyway cost about a fifth of the cheapest render, so it makes -them on the first read instead, at two heap allocations for a render that does share a -draw. The allocation gate over a repeat of a reference path and over a named draw group -prices that, and pins what keeps a hold set off the heap. +Every render starts from the zero `holdSet`, per goal 2's one spelling. Making the maps +where the set is declared kept a record's on its frame's stack, but a `Fake`'s set lives +on the `Generator`, so there it cost every render two heap allocations and about half the +cheapest one's time, measured 2026-09-28. Lazily, a render reading a reference path pays +those two allocations, a record or struct included, and one reading none pays nothing; +goal 11 holds either way. The allocation gate over a repeat of a reference path and over +a named draw group prices that, and pins what keeps a hold set off the heap. ## A category never references itself, and a record's fences run at load diff --git a/holdset.go b/holdset.go index b03fb4a..83dbab0 100644 --- a/holdset.go +++ b/holdset.go @@ -20,10 +20,6 @@ type renderScope struct { pins *pinSet } -func eagerHoldSet() holdSet { - return holdSet{unnamed: hold{variant: map[string]node{}, value: map[string]readValue{}}} -} - // renderOnce renders n as one render, over a hold set of its own. func renderOnce(s *session, n node) string { var set holdSet diff --git a/record.go b/record.go index 67ef410..b0c2627 100644 --- a/record.go +++ b/record.go @@ -130,7 +130,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) } - set := eagerHoldSet() + var set holdSet sc := renderScope{set: &set} if t, isTable := n.(*table); isTable { if n, err = tableRecord(f.rand, t, tail, sc); err != nil { @@ -204,7 +204,7 @@ type RecordTemplate struct { func (t *RecordTemplate) Fake() *Record { t.g.mu.Lock() defer t.g.mu.Unlock() - set := eagerHoldSet() + var set holdSet return renderRecord(t.g.rand, t.template, t.columns, renderScope{set: &set}) } diff --git a/struct.go b/struct.go index 251f199..da98175 100644 --- a/struct.go +++ b/struct.go @@ -351,7 +351,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro // fill draws the record into v's tagged fields, then each nested struct as a record of its own. func (s *structShape) fill(sess *session, v reflect.Value) { if s.record != nil { - set := eagerHoldSet() + var set holdSet for i, c := range renderRecord(sess, s.record, s.columns, renderScope{set: &set}).columns { setColumn(fieldAt(v, s.fieldIndexes[i]), c) } diff --git a/todo.md b/todo.md index f6fbef1..615586a 100644 --- a/todo.md +++ b/todo.md @@ -21,8 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Start every render from one spelling of an empty hold set: `Fake` uses - `holdSet{}`, the record and struct paths `eagerHoldSet()`. - Correct the false comments: `funcCall` names `checkFunc` where `parseFormat` reports, `struct.go`'s "t sits at index within c.t" names no field, `recordFormats` calls ndjson "the same column", `Generator` says every other entry point takes `mu` -- 2.52.0 From a1852076fe95cf626c1b3e0e1fa505c5b2cdd7f7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 18:53:47 +0200 Subject: [PATCH 192/966] Start a named draw group from an empty hold, and state the decision's goal and cost exactly --- docs/decisions.md | 13 ++++++------- holdset.go | 2 +- todo.md | 3 --- 3 files changed, 7 insertions(+), 11 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 8358cc6..56d0024 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -244,13 +244,12 @@ carry both rules and both scopes anyway, and tell them apart at every step. ## A hold set makes its draw maps on the first read -Every render starts from the zero `holdSet`, per goal 2's one spelling. Making the maps -where the set is declared kept a record's on its frame's stack, but a `Fake`'s set lives -on the `Generator`, so there it cost every render two heap allocations and about half the -cheapest one's time, measured 2026-09-28. Lazily, a render reading a reference path pays -those two allocations, a record or struct included, and one reading none pays nothing; -goal 11 holds either way. The allocation gate over a repeat of a reference path and over -a named draw group prices that, and pins what keeps a hold set off the heap. +Every render and every draw group starts from an empty hold, since goal 2 prices a reader +meeting two spellings of one empty set. Making the maps where the set is declared kept a +record's on its frame's stack, but a `Fake`'s set lives on the `Generator`, so there it +cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09-28. +Lazily, a render reading a reference path pays those two allocations, a record or struct +included, and one reading none pays nothing; goal 11 holds either way. ## A category never references itself, and a record's fences run at load diff --git a/holdset.go b/holdset.go index 83dbab0..eeba3cb 100644 --- a/holdset.go +++ b/holdset.go @@ -63,7 +63,7 @@ func (sc renderScope) hold() *hold { if sc.set.named == nil { sc.set.named = map[string]*hold{} } - d = &hold{variant: map[string]node{}, value: map[string]readValue{}} + d = &hold{} sc.set.named[strings.Clone(sc.group)] = d } return d diff --git a/todo.md b/todo.md index 615586a..296a607 100644 --- a/todo.md +++ b/todo.md @@ -300,9 +300,6 @@ because pairing a street with its exact postnummer rewrites shipped rows. flag, an exit code, an exported name, a fence, a builtin and the lowest Go. - Measure the allocation gate over the shipped `geo` trees too: `pinSet.inline [8]` is sized for them, and `perf_test.go` checks a synthetic five-deep tree only. -- Make a named draw group's hold lazy, `&hold{}` in `renderScope.hold`: it escapes - to the heap anyway, so its two eager maps buy no stack and cost two allocations per - group that may never read through them. Check it against the alloc tests. - Name goal 2's four criteria with the axes the panel scores, so a recorded score maps back to the clause it came from: the goal spells them out in prose while `todo.md` records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. -- 2.52.0 From e4d075af6c5e14f4a0960ef08e7ed0c2aaf16e12 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 18:56:07 +0200 Subject: [PATCH 193/966] Name the hold set in the decision's first line --- docs/decisions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 56d0024..e7ea0bd 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -244,8 +244,8 @@ carry both rules and both scopes anyway, and tell them apart at every step. ## A hold set makes its draw maps on the first read -Every render and every draw group starts from an empty hold, since goal 2 prices a reader -meeting two spellings of one empty set. Making the maps where the set is declared kept a +Every render starts from an empty hold set, its draw groups included, since goal 2 prices +a reader meeting two spellings of one empty set. Making the maps where the set is declared kept a record's on its frame's stack, but a `Fake`'s set lives on the `Generator`, so there it cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09-28. Lazily, a render reading a reference path pays those two allocations, a record or struct -- 2.52.0 From a84997d2f4506475e722f3e821e85ca3b15bf53f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 20:26:20 +0200 Subject: [PATCH 194/966] Settle the open questions through the goals and audience --- AGENTS.md | 6 ++++-- README.md | 15 ++++++++++----- todo.md | 48 ++++++++++-------------------------------------- 3 files changed, 24 insertions(+), 45 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8fc3f98..6a4caf4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,10 +8,12 @@ only, `gh pr merge --rebase` where the branch is already on top of main. - README goal 2 is gated at 7.0 on the `comprehension-panel` skill, and below it nothing else merges — no feature, no category, no data — bar a security fix, a - dependency bump and the infrastructure the round itself runs on. `todo.md` carries the round: the nine-seat findings run at depth 1, + dependency bump, a repair of a promise goals 3 to 5 make, and the infrastructure + the round itself runs on. `todo.md` carries the round: the nine-seat findings run at depth 1, a PR per item it names, then the run again, until the score passes. At or above 7.0 every pull request scores with the four-seat run and answers it in one run. No merge - lowers the last score. + lowers the last score its own panel recorded: the nine-seat round and the four-seat + run are two ratchets, never compared with each other. - Hard tabs. No comment by default; delete a restatement, a rationale, history, or a file preamble. - One spelling per result: reject the other at `New`, and let the error name the spelling to use. - Prose naming a source states what that source states: read the register's own field diff --git a/README.md b/README.md index 5eb803b..36f9948 100644 --- a/README.md +++ b/README.md @@ -801,9 +801,11 @@ App developers writing tests and fixtures, in Go and at a shell: - a **Go test author**, filling a struct with `FakeStruct` - a **hand fixture author**, one value at a shell - a **validator-facing author**, who needs a value a real checker accepts +- a **data author**, who writes categories under `--data-path` and meets every load + error the fences raise and a **contributor**, who reads [`todo.md`](todo.md), [`AGENTS.md`](AGENTS.md) and -the Development section below, and who ships a register the four above then draw from. +the Development section below, and who ships a register the five above then draw from. ## Goals @@ -819,14 +821,17 @@ the Development section below, and who ships a register the four above then draw holding the rest, names that tell the truth, and a file that stands with no second document open beside it. A simulated reader panel scores those four, the score is a ratchet no release lowers, and [`AGENTS.md`](AGENTS.md) says what each pull request - owes it. -3. **Valid by construction** — every value passes the check its real consumer - applies; facts that belong together come from one draw, within a value and - across categories. + owes it. Below the floor it holds back new capability, never a repair of a promise + goals 3 to 5 already make, as long as the round that lifts the score stays planned. +3. **Valid by construction** — every value passes the check a current release of + its real consumer applies; facts that belong together come from one draw, within + a value and across categories. 4. **Templates are intuitive, and text means what it says** — a template or a path, and every path in a template, renders what every writer reading it would expect, judged by simulated writers from the Audience. A format renders as written; only `{…}` varies, random characters included (`{digits(3)}`), and a selected row, `[SE]`, is that row wherever it is written. + A name is changed whenever a more intuitive one is found; the Versioning table + prices the change, never blocks it. Where two writers could expect different outputs, the template is a load error naming a spelling for each. One spelling per result; the wrong one is a load error naming the right one. diff --git a/todo.md b/todo.md index 296a607..b23ebbd 100644 --- a/todo.md +++ b/todo.md @@ -73,9 +73,8 @@ by a `data-import/` script, as README goal 12 asks. - Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100: the format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. -- Settle `en_US.ip` and `sv_SE.ip`, today byte-identical: either one `misc.ip` as the - decision "`misc` is what every locale shares" asks, or a decision saying why an - address that carries no locale stays per-locale. +- Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`, per goal 4's + one spelling per result. - Accept a middle name, and draw a shipped `personnummer` inside a *selected* sex; both want a draw group sharing its family's pins. Stop the conflict error naming a rewrite that returns a different value where the read it conflicts with sits inside @@ -88,8 +87,13 @@ by a `data-import/` script, as README goal 12 asks. goal 12 asks for and the hand-written list they use today does not meet. Keep the handle shape: `username` shortens a surname to `ahl` or `sjo`, and the US one draws unisex given names, neither of which falls out of the tables unaided. -- Decide whether `email.local` and `username` share one list: a reference between - shipped categories breaks a consumer once tagged, so it rides a 0.(x+1).0 after that. +- Draw `email.local` from `username`, so the two share one handle list. +- Rename `misc.territory.country` to `sovereign`, per goal 4: it holds a code, and + `country` collides with the `misc.country` path it replaced. +- Add `misc.browser` as the parent of `misc.useragent`, so + `misc.browser[Chrome].useragent` resolves, per goal 3. +- Group `data/misc` into folders where a group name makes a path easier to guess, per + goal 4. `misc` (locale-neutral) @@ -219,33 +223,6 @@ by a `data-import/` script, as README goal 12 asks. everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. -### Open questions to settle - -- Decide whether a goal 5 violation joins the comprehension gate's exception list: the - `{/sel}|{/sel}` panic waits behind the whole round at 5.9, and - `{/geo.SE.locality}|{/geo.SE.municipality}` mispairs the shipped data meanwhile, though a - repair restoring behaviour the repo already claims may already pass the gate as written. -- Decide whether the README's Audience names the author who writes categories under - `--data-path`: the four personas listed consume the shipped set and the contributor - ships a register upstream, while "Your own data" is a README section, `-d` is a - first-class flag, and every load error the draw fences raise lands on that author. -- Decide whether `data/misc`'s 33 flat files gain a level before v1.0.0: a folder is a - path segment, so `misc/net/tld` is a rename a consumer pays for, and goal 13 promises - the directory keeps growing. -- Decide whether `misc.browser` becomes a parent of `misc.useragent`, so - `misc.browser[Chrome].useragent` resolves. Adding a `parent` after v0.1.0 breaks a - consumer, so it rides a 0.(x+1).0. -- Decide whether `misc.timezone` keeps every zone tzdb names: one added recently, - `Europe/Kyiv` among them, is rejected by a consumer resolving it against older - tzdata, and the population weight makes that zone likelier, not rarer. Today the - README names the action a reader takes instead; narrowing what ships is the - alternative. -- Decide whether `misc.territory.country` is renamed `sovereign`: it holds a code, and - `country` collides with the `misc.country` path it replaced. A rename after v0.1.0 - breaks a consumer, so it rides a 0.(x+1).0. -- Decide whether `parent: territory` stays, given a `--data-path` override of - `misc.territory` now fails `New` unless `misc.timezone` is overridden with it. - ### Release - Read the tag through `git/ref/tags/{tag}` in `publish_release.py`: GitHub answers @@ -303,11 +280,6 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Name goal 2's four criteria with the axes the panel scores, so a recorded score maps back to the clause it came from: the goal spells them out in prose while `todo.md` records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. -- Put the round's nine-seat score and the four-seat score every later pull request - takes on one scale, or say in `AGENTS.md` that they are two ratchets: it gates the - round on the first and every merge after 7.0 on the second, then holds one "last - score" across both, where the `comprehension-panel` skill has two independent panels - drift unless a calibration seat reads the anchor project without its score. - Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and a helper reached only through `drawWalk.check`, and misses `checkDrawGroup`, `checkOwnFamily` and `checkColumnDraws`; the README's Layout indexes the fences by file. @@ -320,7 +292,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates Go at `Dockerfile:21`, and the shipped JSON under `data/` is two-space, so the rule as written is one no Go file can break and every data file does. -- Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub or retire it: +- Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub: Gitea converts a repository to a mirror only by re-creating it, so until that runs the copy there is a second owner of one history and goes stale from this commit. - Stop `/cache` collecting root-owned files, which today every command documented -- 2.52.0 From 21b48c8c4195c5af2622e1fb4ded3766759d8b54 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 21:29:54 +0200 Subject: [PATCH 195/966] Correct the false comments and delete dead code --- cmd/fejkdata/main.go | 2 +- fejkdata.go | 6 +++--- format.go | 2 +- node.go | 2 +- struct.go | 2 +- table.go | 10 ---------- todo.md | 6 ------ 7 files changed, 7 insertions(+), 23 deletions(-) diff --git a/cmd/fejkdata/main.go b/cmd/fejkdata/main.go index 4d39720..04f8d86 100644 --- a/cmd/fejkdata/main.go +++ b/cmd/fejkdata/main.go @@ -224,7 +224,7 @@ type recordFormat struct { } // recordFormats is every --format that writes records. json frames the records -// as one array document; ndjson is the same column, one object per line. +// as one array document; ndjson writes one object per line. var recordFormats = map[string]recordFormat{ "csv": {header: (*fejkdata.Record).CSVHeader, line: func(r *fejkdata.Record, _ string) string { return r.CSVLine() }, sep: "\n"}, "json": {line: jsonLine, open: "[", close: "]", sep: ",\n"}, diff --git a/fejkdata.go b/fejkdata.go index def390f..f18fa4f 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -28,8 +28,8 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD // Generator generates fake data from a loaded namespace tree. Create one with [New]. // It is safe for concurrent use; a seeded sequence is reproducible only when drawn -// from one goroutine. The compiled tree is immutable after [New], so List reads it -// without a lock; mu guards the rest, which every other entry point takes. +// from one goroutine. The compiled tree is immutable after [New], so List, NewTemplate +// and NewRecordTemplate read it without a lock; mu guards the rest, which every render takes. type Generator struct { mu sync.Mutex rand *session @@ -120,7 +120,7 @@ func New(opts ...Option) (*Generator, error) { // List returns the sorted dotted paths Fake can render: every category, the dotted // fields within a template, and folder segments. A choice consumes no segment, so a // path continues through one only where every variant carries it, which is the -// rule Fake applies too: List is the set of paths Fake accepts. +// rule Fake applies too. A selector is left out: Fake also takes `[key]` after a table. func (f *Generator) List() []string { var out []string for _, name := range sortedNames(f.categories) { diff --git a/format.go b/format.go index 16ef085..941bc17 100644 --- a/format.go +++ b/format.go @@ -133,7 +133,7 @@ type builtin struct { // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false // for a plain field or alternation body. A '(' without a trailing ')' yields -// ok=false; checkFunc reports it as malformed at compile time. +// ok=false; parseFormat reports it as malformed. func funcCall(body string) (name string, args []string, ok bool) { lp := indexOutside(body, '(') if lp < 0 || !strings.HasSuffix(body, ")") { diff --git a/node.go b/node.go index 6ff24de..f4a689c 100644 --- a/node.go +++ b/node.go @@ -298,7 +298,7 @@ func checkNoRepeatedItem(items []any) error { } return fmt.Errorf("choice item %d repeats item %d; skew the odds with a weight on one of them instead", i, j) } - seen[string(key)] = i + seen[key] = i } return nil } diff --git a/struct.go b/struct.go index da98175..67f5643 100644 --- a/struct.go +++ b/struct.go @@ -123,7 +123,7 @@ func (sc *structCompile) spend(label string) error { return fmt.Errorf(`%s: the struct fields reach more than %d structs; leave a struct field unfilled with fake:"-"`, label, maxStructs) } -// t sits at index within c.t. +// t sits at index within c.typ. func (c *structFields) gatherFields(t reflect.Type, index []int) error { for i := 0; i < t.NumField(); i++ { sf := t.Field(i) diff --git a/table.go b/table.go index 7ba2b2b..65b2ec8 100644 --- a/table.go +++ b/table.go @@ -538,16 +538,6 @@ func (t *table) descendant(name string) *table { // parentRow is the row of t's parent that row r links to. func (t *table) parentRow(r int) int { return t.parentT.byKey[t.cell(r, t.parentIndex)] } -// descends reports whether a is an ancestor of t. -func (t *table) descends(a *table) bool { - for p := t.parentT; p != nil; p = p.parentT { - if p == a { - return true - } - } - return false -} - // under reports whether row r of t sits inside row pr of ancestor a. func (t *table) under(r int, a *table, pr int) bool { for c, row := t, r; c.parentT != nil; c, row = c.parentT, c.parentRow(row) { diff --git a/todo.md b/todo.md index b23ebbd..711f715 100644 --- a/todo.md +++ b/todo.md @@ -21,12 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Correct the false comments: `funcCall` names `checkFunc` where `parseFormat` - reports, `struct.go`'s "t sits at index within c.t" names no field, `recordFormats` - calls ndjson "the same column", `Generator` says every other entry point takes `mu` - where `NewTemplate` and `NewRecordTemplate` do not, and `List` claims to be every - path `Fake` accepts, which a selector is too. Delete the dead `table.descends` and - the redundant `string(key)` in `checkNoRepeatedItem`. - Delete the decision "The rows of a table are alternatives": it defines `alternatives`, which its comment in `drawfence.go` already does. Point the README's "which pairs are weighed against each other" at that comment's rule, stated in the README's Table -- 2.52.0 From dc0f9fa2d6d2397e870514cab9db7cb14723ef2e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 21:33:45 +0200 Subject: [PATCH 196/966] Drop the comment claims that name callers and List's gaps --- fejkdata.go | 2 +- format.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index f18fa4f..41672d7 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -120,7 +120,7 @@ func New(opts ...Option) (*Generator, error) { // List returns the sorted dotted paths Fake can render: every category, the dotted // fields within a template, and folder segments. A choice consumes no segment, so a // path continues through one only where every variant carries it, which is the -// rule Fake applies too. A selector is left out: Fake also takes `[key]` after a table. +// rule Fake applies too. func (f *Generator) List() []string { var out []string for _, name := range sortedNames(f.categories) { diff --git a/format.go b/format.go index 941bc17..5cca79d 100644 --- a/format.go +++ b/format.go @@ -133,7 +133,7 @@ type builtin struct { // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false // for a plain field or alternation body. A '(' without a trailing ')' yields -// ok=false; parseFormat reports it as malformed. +// ok=false. func funcCall(body string) (name string, args []string, ok bool) { lp := indexOutside(body, '(') if lp < 0 || !strings.HasSuffix(body, ")") { -- 2.52.0 From 7eb1b2473ffc6125077a8d9f1ad1a184709fd635 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 21:47:29 +0200 Subject: [PATCH 197/966] State in the README which reads never render together, and drop the decision restating alternatives --- AGENTS.md | 1 - README.md | 5 +++-- docs/decisions.md | 10 ---------- drawfence.go | 1 - todo.md | 4 ---- 5 files changed, 3 insertions(+), 18 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6a4caf4..50d5998 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -81,7 +81,6 @@ In [docs/decisions.md](docs/decisions.md): - A table is a record of string columns - The key map is built at load, the rest on first draw - Two categories may name one TSV -- The rows of a table are alternatives - A table never reaches its own family, by any route - Tables carrying token cells stay small - A path is walked once without drawing before it is walked for real diff --git a/README.md b/README.md index 36f9948..739817b 100644 --- a/README.md +++ b/README.md @@ -549,8 +549,9 @@ it, and a cell or format of a table that references a table of its own family, t any template, a `repeat` or a `drawGroup` included, since a bare read of the referencing table draws its row without pinning it, so the family would draw apart from that row: read the family from a template beside it, or add the value as a column. A cell's -reads are the render's too; which pairs are weighed against each other is under -[Decisions](docs/decisions.md#the-rows-of-a-table-are-alternatives). +reads are the render's too, weighed against every read that renders with them: one read +renders one row, so reads under different rows of one pinned table, whose ancestors pin +with it, or of one table read whole, never render together. ### Options and fields diff --git a/docs/decisions.md b/docs/decisions.md index e7ea0bd..0d4dc26 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -452,16 +452,6 @@ rows twice, which is what a category over a register with two natural formats as a TSV nothing names stays a load error, since that one is a file forgotten rather than shared. -## The rows of a table are alternatives - -Only one row renders per read, so within one read a cell in one row and a cell in -another never meet, and each may select its own row of another table; the cells of one -row, and whatever they reach, do meet, and so does the format beside them. Two reads -meet unless they sit under different rows of one pinned table; pinning a row pins its -ancestors too, so rows under different parents are alternatives as well. A table read -whole draws its row apart from every pin, so the rows of one whole read are -alternatives only to each other. - ## A table never reaches its own family, by any route A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row that lists diff --git a/drawfence.go b/drawfence.go index 2988095..efe4602 100644 --- a/drawfence.go +++ b/drawfence.go @@ -426,7 +426,6 @@ func conflict(r pathRead, err error) error { // alternatives reports whether two reads never render together: one read renders one row of a // table, so reads under different rows the walks pinned, or under different rows of one whole // draw, never meet. -// docs/decisions.md#the-rows-of-a-table-are-alternatives func alternatives(a, b drawAt) bool { return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) } diff --git a/todo.md b/todo.md index 711f715..6a779b9 100644 --- a/todo.md +++ b/todo.md @@ -21,10 +21,6 @@ Self-sufficiency 5.6), every seat capped by Locality, and all nine naming This round, in order: -- Delete the decision "The rows of a table are alternatives": it defines `alternatives`, - which its comment in `drawfence.go` already does. Point the README's "which pairs are - weighed against each other" at that comment's rule, stated in the README's Table - section, where it sends a reader to the log for the definition today. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 805a2d11fc72519439b533f84797ed60b253325f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 21:50:54 +0200 Subject: [PATCH 198/966] Name the selector syntax the alternatives rule turns on --- README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 739817b..ecdf5ad 100644 --- a/README.md +++ b/README.md @@ -549,9 +549,10 @@ it, and a cell or format of a table that references a table of its own family, t any template, a `repeat` or a `drawGroup` included, since a bare read of the referencing table draws its row without pinning it, so the family would draw apart from that row: read the family from a template beside it, or add the value as a column. A cell's -reads are the render's too, weighed against every read that renders with them: one read -renders one row, so reads under different rows of one pinned table, whose ancestors pin -with it, or of one table read whole, never render together. +reads are the render's too, weighed against every read that renders with them. A +reference renders one row of a table, so reads under different selected rows of it, or +of their ancestors, never render together, and nor do reads under different rows of one +reference that selects none. ### Options and fields -- 2.52.0 From 836ab8ddc481108e3b692b285ddeec6aa91d1be1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 21:51:55 +0200 Subject: [PATCH 199/966] State the alternatives rule by the row a cell sits in --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index ecdf5ad..59d6b6f 100644 --- a/README.md +++ b/README.md @@ -550,9 +550,9 @@ any template, a `repeat` or a `drawGroup` included, since a bare read of the ref table draws its row without pinning it, so the family would draw apart from that row: read the family from a template beside it, or add the value as a column. A cell's reads are the render's too, weighed against every read that renders with them. A -reference renders one row of a table, so reads under different selected rows of it, or -of their ancestors, never render together, and nor do reads under different rows of one -reference that selects none. +reference renders one row of a table, so reads in cells of different rows of it, or of +different rows of their ancestors, never render together, whether `[key]` selects the row +or the reference draws it. ### Options and fields -- 2.52.0 From 3a639c93b515598b9215179ceede5c8c19591ed4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:05:24 +0200 Subject: [PATCH 200/966] File the 2026-09-28 nine-seat panel's findings, scored 5.7 --- todo.md | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 6a779b9..8c6c796 100644 --- a/todo.md +++ b/todo.md @@ -17,10 +17,43 @@ Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9, Self-sufficiency 5.6); 5.7 on 2026-09-26 (Navigation 6.7, Locality 5.0, Shape 5.8, Self-sufficiency 5.6), every seat capped by Locality, and all nine naming -`drawWalk.walk` among the hardest. +`drawWalk.walk` among the hardest; 5.7 on 2026-09-28 (Navigation 6.7, Locality 5.1, +Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming +`drawWalk.walk` and the four path walks among the hardest. This round, in order: +- Hold `replayPairs`' claim that pairs find every conflict a full replay would to a + test replaying every read set whole, over the fixtures and the shipped data: six + seats could only take the comment's proof on trust. +- Key `drawWalk`'s visits by the pins, not `rowsKey`'s `%p` strings, and make leaving + `wholeTable` out of the key fail loudly where `checkOwnFamily` has not run: the walk is + sound only because another fence ran first, and nothing at the walk enforces it. +- Split `hold`'s two lifetimes, the expansion's sibling hold and the render's group + hold, into two types, and stop `readField` rewriting `sc.pins`: which pins a column + reads depends on which distant caller last overrode them, the cost seven seats named. +- Return each phase's output from compile, link and format compile, not fill + `template`'s 25 fields in place from four files: five seats had to know which phase a + function runs in to know which fields are valid yet. +- Make `renderEdges` panic on a node it does not handle, as `render` does: its + `default: return nil` silently exempts a new node kind from every fence, and nothing + at `expand` says `renderEdges` mirrors it. +- Carry the arm on `renderEdge` so `graph.go`, `drawfence.go` and `holdfence.go` stop + re-parsing `label` through `splitArm`, whose key differs before and after link. +- Give `pathCover` a doc comment and say in each path walk's comment how it differs from + `pathCheck` at a choice and a table: every seat diffed the four loops by eye. +- Make `disagreement` a table keyed by the two datatypes and the proof, dropping the + operand swap: six seats still needed a truth table to see which item it blames. +- Move `session.trace` and `renderTrace.repeatDepth` off the render path: they serve + `drawfence_test.go` alone. +- Give `Fake` a local hold set like `FakeRecord`, `RecordTemplate.Fake` and + `FakeStruct`, or state at `Generator.set` why it alone needs one: four seats read the + field's heap claim as contradicted by the other three. +- Spell every "a fence should have refused this" panic with one greppable phrase and the + table's `path`: the 17 panics word it differently. +- Correct the false comments: `List` claims folder segments, which `paths` never lists, + and `renderEdge.reached` claims `pathKeyReaders`' vocabulary, which spells a token + `token {x}`. Drop `rng`'s "for the reason below", which points at `pick`. - Run the nine-seat panel again and file what it names here. ### Data @@ -166,8 +199,8 @@ by a `data-import/` script, as README goal 12 asks. `data-import/geo-se.py` assigns as the decision states, linking `street` and `postal-code` by it. - Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. - - Leave no selection by name in README.md, CHANGELOG.md or docs/: no selector spelling - a free-form name, `geo.SE.locality[Lund]`, `misc.currency[Euro]`, `first-name[Kim]` + - Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`: no + selector spelling a free-form name, `geo.SE.locality[Lund]`, `misc.currency[Euro]`, `first-name[Kim]` and the rest, no "key or name", "by name" or "either" wording about selectors, and no `name` option in an example. - Render selected rows as the decision "A selected row is fixed, not drawn, and an -- 2.52.0 From 9ae65d13a05e3319d125040f8aa96c19c7166f81 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:05:53 +0200 Subject: [PATCH 201/966] Name the panics the round respells, and drop a quoted panic no code prints --- todo.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/todo.md b/todo.md index 8c6c796..4fca912 100644 --- a/todo.md +++ b/todo.md @@ -49,8 +49,9 @@ This round, in order: - Give `Fake` a local hold set like `FakeRecord`, `RecordTemplate.Fake` and `FakeStruct`, or state at `Generator.set` why it alone needs one: four seats read the field's heap claim as contradicted by the other three. -- Spell every "a fence should have refused this" panic with one greppable phrase and the - table's `path`: the 17 panics word it differently. +- Spell every panic an earlier check makes unreachable with one greppable phrase: the + 17 say "reached prep unvalidated", "should have been proved", "with no row pinned" + and more, so a user's report greps to nothing. - Correct the false comments: `List` claims folder segments, which `paths` never lists, and `renderEdge.reached` claims `pathKeyReaders`' vocabulary, which spells a token `token {x}`. Drop `rng`'s "for the reason below", which points at `pick`. @@ -222,9 +223,9 @@ by a `data-import/` script, as README goal 12 asks. - Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. - Refuse two whole reads of one table family in one render, which goal 5 promises is a - load error: `{/sel}|{/sel}` panics out of `Fake` with "a fence should have refused this - at New" where the two draws land on rows whose cells select different rows of another - table, and `{/geo.SE.locality}|{/geo.SE.municipality}` renders a locality outside the + load error: `{/sel}|{/sel}` panics out of `Fake` where the two draws land on rows whose + cells select different rows of another table, and + `{/geo.SE.locality}|{/geo.SE.municipality}` renders a locality outside the municipality beside it. Each read draws its own row, so cells of two rows render together, where the fence counts them alternatives and compares neither. Have the error name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`, -- 2.52.0 From 0825819e4420049bf81ea4e16ece7ab0451127f8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:20:14 +0200 Subject: [PATCH 202/966] Hold replayPairs to a whole replay of every read set rendering together --- drawfence_test.go | 114 ++++++++++++++++++++++++++++++++++++++++++++++ todo.md | 3 -- 2 files changed, 114 insertions(+), 3 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index d4e0eb9..6f9a0aa 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -146,3 +146,117 @@ func renderRecordRoot(s *session, t *template, columns []Column) { } renderRecord(s, t, columns, sc) } + +func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { + corpus := fenceCorpus(t) + roots, _ := fenceRoots(t, corpus) + shipped, err := New() + if err != nil { + t.Fatalf("New() = %v", err) + } + _ = walkNodes(shipped.categories, func(path string, n node) error { + if tm, ok := n.(*template); ok { + roots = append(roots, fenceRoot{path, tm}) + } + return nil + }) + for _, root := range roots { + wantPairsFindAll(t, root.label, renderDraws(root.t).reads) + if root.t.isRecord && len(root.t.fields) > 0 { + wantPairsFindAll(t, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads) + } + } +} + +func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { + f, err := New(WithDataPath(writeFiles(t, siblings()))) + if err != nil { + t.Fatalf("New = %v", err) + } + var reads []pathRead + _ = walkNodes(f.categories, func(_ string, n node) error { + tb, isTable := n.(*table) + if !isTable || tb.keyIndex < 0 || tb.familyRoot().path != "region" { + return nil + } + for r := 0; r < tb.rowCount(); r++ { + inline, err := f.NewTemplate(`"{/` + tb.selectorSpelling(r) + `}"`) + if err != nil { + t.Fatalf("NewTemplate(%s) = %v", tb.selectorSpelling(r), err) + } + root, isTemplate := inline.n.(*template) + if !isTemplate { + t.Fatalf("NewTemplate(%s) compiled to %T", tb.selectorSpelling(r), inline.n) + } + reads = append(reads, renderDraws(root).reads...) + } + return nil + }) + conflicts := 0 + for _, a := range reads { + for _, b := range reads { + for _, c := range reads { + triple := []pathRead{a, b, c} + wantPairsFindAll(t, a.a.spelling+", "+b.a.spelling+", "+c.a.spelling, triple) + if replayPairs(triple) != nil { + conflicts++ + } + } + } + } + if conflicts == 0 { + t.Error("no triple of selected rows conflicts, so the comparison proved nothing") + } +} + +func wantPairsFindAll(t *testing.T, label string, reads []pathRead) { + t.Helper() + var tabled []pathRead + for _, r := range reads { + if r.tr != nil { + tabled = append(tabled, r) + } + } + pairs := replayPairs(reads) + var whole error + eachRenderSet(tabled, func(set []pathRead) { + var d pinSet + for _, r := range set { + if err := r.tr.replay(&d); err != nil && whole == nil { + whole = err + } + } + }) + if (pairs == nil) != (whole == nil) { + t.Errorf("%s: replayPairs = %v, a whole replay of every read set rendering together = %v", label, pairs, whole) + } +} + +// eachRenderSet calls fn with every maximal set of reads that render together, by Bron–Kerbosch. +func eachRenderSet(reads []pathRead, fn func([]pathRead)) { + together := func(a, b pathRead) bool { return a.at.group == b.at.group && !alternatives(a.at, b.at) } + var grow func(set, candidates, excluded []pathRead) + grow = func(set, candidates, excluded []pathRead) { + if len(candidates) == 0 && len(excluded) == 0 { + fn(set) + return + } + for len(candidates) > 0 { + r := candidates[0] + var nextC, nextX []pathRead + for _, c := range candidates[1:] { + if together(r, c) { + nextC = append(nextC, c) + } + } + for _, x := range excluded { + if together(r, x) { + nextX = append(nextX, x) + } + } + grow(append(set[:len(set):len(set)], r), nextC, nextX) + candidates, excluded = candidates[1:], append(excluded, r) + } + } + grow(nil, reads, nil) +} diff --git a/todo.md b/todo.md index 4fca912..ffa2a4f 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Hold `replayPairs`' claim that pairs find every conflict a full replay would to a - test replaying every read set whole, over the fixtures and the shipped data: six - seats could only take the comment's proof on trust. - Key `drawWalk`'s visits by the pins, not `rowsKey`'s `%p` strings, and make leaving `wholeTable` out of the key fail loudly where `checkOwnFamily` has not run: the walk is sound only because another fence ran first, and nothing at the walk enforces it. -- 2.52.0 From 19033bf2e57cdbcc79a2a682cb0bf59ea2a473d4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:26:35 +0200 Subject: [PATCH 203/966] Fail the whole-replay comparison where it compared no read set --- drawfence_test.go | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index 6f9a0aa..56fa605 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -160,12 +160,16 @@ func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { } return nil }) + compared := 0 for _, root := range roots { - wantPairsFindAll(t, root.label, renderDraws(root.t).reads) + compared += wantPairsFindAll(t, root.label, renderDraws(root.t).reads) if root.t.isRecord && len(root.t.fields) > 0 { - wantPairsFindAll(t, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads) + compared += wantPairsFindAll(t, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads) } } + if compared == 0 { + t.Error("no read set rendering together holds two table reads, so the comparison proved nothing") + } } func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { @@ -209,7 +213,8 @@ func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { } } -func wantPairsFindAll(t *testing.T, label string, reads []pathRead) { +// wantPairsFindAll counts the read sets rendering together that hold two table reads or more. +func wantPairsFindAll(t *testing.T, label string, reads []pathRead) int { t.Helper() var tabled []pathRead for _, r := range reads { @@ -219,7 +224,11 @@ func wantPairsFindAll(t *testing.T, label string, reads []pathRead) { } pairs := replayPairs(reads) var whole error + compared := 0 eachRenderSet(tabled, func(set []pathRead) { + if len(set) > 1 { + compared++ + } var d pinSet for _, r := range set { if err := r.tr.replay(&d); err != nil && whole == nil { @@ -230,6 +239,7 @@ func wantPairsFindAll(t *testing.T, label string, reads []pathRead) { if (pairs == nil) != (whole == nil) { t.Errorf("%s: replayPairs = %v, a whole replay of every read set rendering together = %v", label, pairs, whole) } + return compared } // eachRenderSet calls fn with every maximal set of reads that render together, by Bron–Kerbosch. -- 2.52.0 From df230e286fb6708de9530079d01eb09fcb10a8b3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:41:24 +0200 Subject: [PATCH 204/966] Key the traced reads by their table row, not drawAt.rowsKey --- drawfence_test.go | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index 56fa605..a803e85 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -76,7 +76,11 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { func wantGathered(t *testing.T, s *session, tables map[string]*table, label string, gathered []pathRead, renderOnce func()) { t.Helper() var reads []pathRead - seen := map[readKey]bool{} + type traceKey struct { + group, path, table string + row int + } + seen := map[traceKey]bool{} s.trace = &renderTrace{read: func(group, table string, row int, a arm) { if s.trace.repeatDepth != 0 || !isRef(a.key) { return @@ -85,7 +89,7 @@ func wantGathered(t *testing.T, s *session, tables map[string]*table, label stri if table != "" { r.at.pins.add(tables[table], row) } - if k := (readKey{group, a.path, r.at.rowsKey()}); !seen[k] { + if k := (traceKey{group, a.path, table, row}); !seen[k] { seen[k] = true reads = append(reads, r) } -- 2.52.0 From d03c92ddbfa339e1c2d6196ad264d38ed1241209 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:57:46 +0200 Subject: [PATCH 205/966] Pin that the fence walk reaches a table's cells by every route --- drawfence_test.go | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/drawfence_test.go b/drawfence_test.go index a803e85..2dc2223 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -3,6 +3,7 @@ package fejkdata import ( "strings" "testing" + "testing/fstest" ) func fenceCorpus(t *testing.T) *Generator { @@ -274,3 +275,35 @@ func eachRenderSet(reads []pathRead, fn func([]pathRead)) { } grow(nil, reads, nil) } + +func TestDrawWalkWalksACellByEveryRoute(t *testing.T) { + fsys := fstest.MapFS{ + "g.json": {Data: []byte(`{"format":"{/x.v}","drawGroup":"g2"}`)}, + "r.json": {Data: []byte(`"{/g} {/x}"`)}, + "x.json": {Data: []byte(`{"format":"{/g}","rows":"x.tsv","key":"code"}`)}, + "x.tsv": {Data: []byte("code\tv\n1\t{/z}\n2\t{/z.a}\n")}, + "z.json": {Data: []byte(`{"format":"{a}","a":"hi"}`)}, + } + g, err := loadDir(dataSource{fsys: fsys}, ".") + if err != nil { + t.Fatalf("loadDir = %v", err) + } + if err := treeBinding(g.children).link(); err != nil { + t.Fatalf("link = %v", err) + } + r, isTemplate := g.children["r"].(*template) + if !isTemplate { + t.Fatalf("r is a %T, want a template", g.children["r"]) + } + var pinned, whole bool + for _, read := range renderDraws(r).reads { + if read.a.path != "/z" { + continue + } + pinned = pinned || read.at.pins.used > 0 + whole = whole || read.at.wholePins.used > 0 + } + if !pinned || !whole { + t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root and from inside {/x}", pinned, whole) + } +} -- 2.52.0 From 88f762077f504815a823dffb36f8e7529d019fb0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 22:42:07 +0200 Subject: [PATCH 206/966] Key drawWalk's visits by interned pin sets and the whole table --- drawfence.go | 59 ++++++++++++++++++++++++++++++++++++++-------------- pins.go | 13 ------------ todo.md | 3 --- 3 files changed, 43 insertions(+), 32 deletions(-) diff --git a/drawfence.go b/drawfence.go index efe4602..b46aff1 100644 --- a/drawfence.go +++ b/drawfence.go @@ -226,9 +226,10 @@ func columnDraws(t *template, columns []string) *drawWalk { // drawWalk gathers the references one render reads, by draw group, for check to compare. type drawWalk struct { - reads []pathRead - read map[readKey]bool - seen map[nodeVisit]bool + reads []pathRead + read map[readKey]bool + seen map[nodeVisit]bool + pinIDs map[pinsLink]pinsID } // drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the @@ -238,7 +239,7 @@ type drawAt struct { group string route drawRoute pins pinSet - wholeTable *table // left out of the visit keys: only this table's own cells compare against it, which the own-family fence keeps true + wholeTable *table wholePins pinSet } @@ -253,19 +254,48 @@ type pathRead struct { } type nodeVisit struct { - n node - group string - rowsKey string + n node + group string + wholeTable *table + pins pinsID + wholePins pinsID } type readKey struct { - group string - path string - rowsKey string + group string + path string + pins pinsID + wholePins pinsID +} + +// pinsID names a pin set within one walk: two sets pinning the same rows share one. +type pinsID int + +// pinsLink is a set's last pin, in path order, and the id of the set before it. +type pinsLink struct { + prev pinsID + pin tablePin } func newDrawWalk() *drawWalk { - return &drawWalk{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}} + return &drawWalk{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}, pinIDs: map[pinsLink]pinsID{}} +} + +func (w *drawWalk) pinsID(p *pinSet) pinsID { + var pins []tablePin + p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) + sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) + id := pinsID(0) + for _, q := range pins { + l := pinsLink{id, q} + next, ok := w.pinIDs[l] + if !ok { + next = pinsID(len(w.pinIDs) + 1) + w.pinIDs[l] = next + } + id = next + } + return id } // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops @@ -273,7 +303,7 @@ func newDrawWalk() *drawWalk { // alone and an unpinned one draws a row inside its nearest pinned ancestor's, so that cell never // renders on this route. func (w *drawWalk) walk(n node, at drawAt) { - v := nodeVisit{n, at.group, at.rowsKey()} + v := nodeVisit{n, at.group, at.wholeTable, w.pinsID(&at.pins), w.pinsID(&at.wholePins)} if w.seen[v] { return } @@ -305,15 +335,12 @@ func (w *drawWalk) walk(n node, at drawAt) { } } -// rowsKey spells the rows the walk stands in, pinned and whole, for a map. -func (at drawAt) rowsKey() string { return at.pins.mapKey() + "|" + at.wholePins.mapKey() } - // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { if a, reads := refRead(from, e.label); reads { tr := tableReadOf(from.(*template).head(a.key), a, e.to) - if k := (readKey{at.group, a.path, at.rowsKey()}); !w.read[k] { + if k := (readKey{at.group, a.path, w.pinsID(&at.pins), w.pinsID(&at.wholePins)}); !w.read[k] { w.read[k] = true w.reads = append(w.reads, pathRead{at, a, tr}) } diff --git a/pins.go b/pins.go index c8951c9..f16cee0 100644 --- a/pins.go +++ b/pins.go @@ -4,7 +4,6 @@ import ( "fmt" "maps" "sort" - "strings" ) type tablePin struct { @@ -163,15 +162,3 @@ func (p *pinSet) differs(q *pinSet) bool { }) return found } - -// mapKey spells the set for a map, by the tables' identities in path order. -func (p *pinSet) mapKey() string { - var pins []tablePin - p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) - sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) - var b strings.Builder - for _, q := range pins { - fmt.Fprintf(&b, "%p[%d]", q.t, q.row) - } - return b.String() -} diff --git a/todo.md b/todo.md index ffa2a4f..37eb540 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Key `drawWalk`'s visits by the pins, not `rowsKey`'s `%p` strings, and make leaving - `wholeTable` out of the key fail loudly where `checkOwnFamily` has not run: the walk is - sound only because another fence ran first, and nothing at the walk enforces it. - Split `hold`'s two lifetimes, the expansion's sibling hold and the render's group hold, into two types, and stop `readField` rewriting `sc.pins`: which pins a column reads depends on which distant caller last overrode them, the cost seven seats named. -- 2.52.0 From 194f194f8181d0a76fe193deaef50dc07b9f28a9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:09:13 +0200 Subject: [PATCH 207/966] Split the expansion's hold from the group's, and read a table's row from where the table is entered --- doc.go | 2 +- drawfence_test.go | 2 +- hold.go | 52 ++++++++++++++++++++++++----------------------- holdset.go | 37 ++++++++++++++++++++++----------- path.go | 6 +++--- record.go | 3 +++ render.go | 8 +++----- 7 files changed, 63 insertions(+), 47 deletions(-) diff --git a/doc.go b/doc.go index 8109d45..e079331 100644 --- a/doc.go +++ b/doc.go @@ -24,7 +24,7 @@ package fejkdata // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. // - hold — keeping one draw of a name, so every route to it reads that value: -// `template.held`, `hold`, `readField`. +// `template.held`, `hold`, `groupHold`, `readField`. // - draw group — reference draws held apart inside one render: // `template.drawGroup` as the data spells it, `template.drawGroupKey` as a // render reads it, `holdSet`, `renderScope`. diff --git a/drawfence_test.go b/drawfence_test.go index 2dc2223..6475138 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -134,7 +134,7 @@ func renderRoot(s *session, t *template) { var set holdSet sc := renderScope{set: &set} sc.hold().pins.pin(t.table, r) - render(s, t.table.pinnedRow, sc) + render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.hold().pins)) } case t.repeat > 1: expand(s, t, sc) diff --git a/hold.go b/hold.go index a09951d..8b507fb 100644 --- a/hold.go +++ b/hold.go @@ -5,14 +5,19 @@ import ( "strings" ) -// hold is what has already been drawn for held names: the variant each was drawn as, so every -// path under it reads one row; the value each read produced, by its path, so the same read -// written twice reads one value; and the table rows those reads pinned. An expansion keeps one for -// its sibling names, and a render one per group for its reference paths. +// hold is what one expansion has drawn for the names it holds: the variant each was drawn as, so +// every path under it reads one variant; and the value each read produced, by its path, so the +// same read written twice reads one value. type hold struct { variant map[string]node value map[string]readValue - pins pinSet +} + +// groupHold is what one render's draw group has drawn for its reference paths: a hold, and the +// table rows those paths pinned. +type groupHold struct { + hold + pins pinSet } // readValue is what one read drew: its text, and whether it landed on a null. @@ -25,7 +30,9 @@ type readValue struct { // token addresses by dotted path, or a field an operand reads — is drawn once and // kept, so {place.postal-code} and {place.locality} read one row, either read twice // gives one value, and a shown operand is the operand computed. Every other name is -// drawn afresh, so {word} {word} still draws twice. +// drawn afresh, so {word} {word} still draws twice. A reference path is kept in the +// render's hold for its group, so its draw spans the render; every other held name in +// held, the expansion's. func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { if s.trace != nil { traceRead(s.trace, t, sc, a) @@ -36,18 +43,23 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV } return readValue{text: render(s, t.head(a.key), sc)} } - d := readHold(held, sc, a) - if r, done := d.value[a.path]; done { + w := pathDraw{s: s, held: held, a: &a} + if isRef(a.key) && len(a.tail) > 0 { + group := sc.hold() + w.held, w.pins = &group.hold, &group.pins + } + if r, done := w.held.value[a.path]; done { return r } - if len(a.tail) > 0 { - sc.pins = &d.pins + leaf := drawPath(t.head(a.key), a.tail, a.key, &w) + if w.pins != nil { + sc = sc.at(leaf, w.pins) } - r := renderLeaf(s, drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, pins: &d.pins, held: d, a: &a}), sc) - if d.value == nil { - d.value = map[string]readValue{} + r := renderLeaf(s, leaf, sc) + if w.held.value == nil { + w.held.value = map[string]readValue{} } - d.value[a.path] = r + w.held.value[a.path] = r return r } @@ -64,21 +76,11 @@ func traceRead(trace *renderTrace, t *template, sc renderScope, a arm) { case t.cellOf != nil: table, row = t.cellOf.path, t.cellRow case t.table != nil: - table, row = t.table.path, sc.rows().mustRow(t.table) + table, row = t.table.path, sc.rowOf(t.table) } trace.read(strings.Clone(sc.group), strings.Clone(table), row, a) } -// readHold is the hold a held read keeps its draw in: for a reference that reads a path, -// the render's hold for its group, so its draw spans the render; for a sibling, or a -// bare reference, held. -func readHold(held *hold, sc renderScope, a arm) *hold { - if isRef(a.key) && len(a.tail) > 0 { - return sc.hold() - } - return held -} - // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. func renderLeaf(s *session, n node, sc renderScope) readValue { diff --git a/holdset.go b/holdset.go index eeba3cb..c798226 100644 --- a/holdset.go +++ b/holdset.go @@ -1,23 +1,23 @@ package fejkdata import ( + "fmt" "strings" ) // holdSet is one render's reference draws: the unnamed draw group's, and each named one's. type holdSet struct { - unnamed hold - named map[string]*hold + unnamed groupHold + named map[string]*groupHold } // renderScope is where a render reads its reference paths: a hold set, in the draw group of the -// template rendering, and the pin set a table's columns read their row from: a whole read's own, -// else the hold's. It passes by value, and what is read from it reaches a map key, an interface or +// template rendering; and row, the row of the table rendering, which its columns read. It passes by value, and what is read from it reaches a map key, an interface or // a func value only as a copy; else sc, and with it every render's hold set, moves to the heap. type renderScope struct { set *holdSet group string - pins *pinSet + row tablePin } // renderOnce renders n as one render, over a hold set of its own. @@ -47,23 +47,36 @@ func (sc renderScope) in(t *template) renderScope { return sc } -func (sc renderScope) rows() *pinSet { - if sc.pins != nil { - return sc.pins +// at is the scope n, the leaf of a path that pinned its rows in pins, renders in: where n is a +// table's row or column, at the row pins holds for that table. +func (sc renderScope) at(n node, pins *pinSet) renderScope { + switch n := n.(type) { + case *tableRow: + sc.row = tablePin{n.t, pins.mustRow(n.t)} + case *tableColumn: + sc.row = tablePin{n.t, pins.mustRow(n.t)} } - return &sc.hold().pins + return sc } -func (sc renderScope) hold() *hold { +// rowOf is the row of t its columns render from. +func (sc renderScope) rowOf(t *table) int { + if sc.row.t != t { + panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) + } + return sc.row.row +} + +func (sc renderScope) hold() *groupHold { if sc.group == "" { return &sc.set.unnamed } d, drew := sc.set.named[sc.group] if !drew { if sc.set.named == nil { - sc.set.named = map[string]*hold{} + sc.set.named = map[string]*groupHold{} } - d = &hold{} + d = &groupHold{} sc.set.named[strings.Clone(sc.group)] = d } return d diff --git a/path.go b/path.go index e07156b..139fe2f 100644 --- a/path.go +++ b/path.go @@ -332,9 +332,9 @@ func (w *pathProbe) readRow(t *table, r tableRoute) error { return nil } -// pathDraw draws the rows and variants a proved path reads, pinning them in pins. -// For a held read, held keeps the variant drawn at each level of a, so paths -// sharing a prefix share it. +// pathDraw draws the rows and variants a proved path reads, pinning the rows in pins, which +// only a path from a reference needs: a table is a category. For a held read, held keeps the +// variant drawn at each level of a, so paths sharing a prefix share it. type pathDraw struct { s *session pins *pinSet diff --git a/record.go b/record.go index b0c2627..4ca8d0c 100644 --- a/record.go +++ b/record.go @@ -263,6 +263,9 @@ func recordOf(n node) (*template, []Column, error) { // over sc's hold; a table's columns read the row pinned there. func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) + if t.table != nil { + sc = sc.at(t.table.pinnedRow, &sc.hold().pins) + } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { column := renderLeaf(s, t.fields[r.columns[i].Name], sc) diff --git a/render.go b/render.go index de13d17..8e9a7c9 100644 --- a/render.go +++ b/render.go @@ -33,7 +33,7 @@ func (f *Generator) Fake(path string) (string, error) { if _, ok := n.(*folder); ok { return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) } - return render(f.rand, n, sc), nil + return render(f.rand, n, sc.at(n, &sc.hold().pins)), nil } // descend walks a caller's path to the node it names, pinning in sc the rows it @@ -56,14 +56,12 @@ func render(s *session, n node, sc renderScope) string { case *nullItem: return "" case *table: - var own pinSet - own.add(n, n.drawRow(s)) - sc.pins = &own + sc.row = tablePin{n, n.drawRow(s)} return expand(s, n.formatTemplate, sc) case *tableRow: return expand(s, n.t.formatTemplate, sc) case *tableColumn: - row := sc.rows().mustRow(n.t) + row := sc.rowOf(n.t) if cell := n.t.cellTemplate(row, n.i); cell != nil { return render(s, cell, sc) } -- 2.52.0 From 54d933ecc2d255ac8f2eafb6728730051b2b518b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:18:25 +0200 Subject: [PATCH 208/966] Name renderScope.groupHold for what it returns, correct pathDraw's pins comment, and drop the done round item --- docs/decisions.md | 2 +- drawfence_test.go | 6 +++--- hold.go | 2 +- holdset.go | 7 ++++--- path.go | 6 +++--- record.go | 4 ++-- render.go | 4 ++-- todo.md | 3 --- 8 files changed, 16 insertions(+), 18 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 0d4dc26..f0a3be8 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -312,7 +312,7 @@ file the stem of the units it holds. Two declarations of any kind — function, method, field, type — share a name only where one definition covers both: the same kind of value, counted or addressed the same way, so a reader landing on the wrong one concludes nothing false. `Fake` on each template kind -renders one value from what it is called on; `renderScope.hold` returns the `*hold` it +renders one value from what it is called on; `renderScope.groupHold` returns the `*groupHold` it names. A bare function never shares a name with a method: a search for the name returns both, and a call site shows no receiver to tell them apart. `TestNoFunctionSpellsAMethod` holds that, and review judges the rest. Serves goal 2: names that tell the truth, and reaching diff --git a/drawfence_test.go b/drawfence_test.go index 6475138..c426bc4 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -133,8 +133,8 @@ func renderRoot(s *session, t *template) { for r := 0; r < t.table.rowCount(); r++ { var set holdSet sc := renderScope{set: &set} - sc.hold().pins.pin(t.table, r) - render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.hold().pins)) + sc.groupHold().pins.pin(t.table, r) + render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.groupHold().pins)) } case t.repeat > 1: expand(s, t, sc) @@ -147,7 +147,7 @@ func renderRecordRoot(s *session, t *template, columns []Column) { var set holdSet sc := renderScope{set: &set} if t.table != nil { - t.table.drawIn(s, &sc.hold().pins) + t.table.drawIn(s, &sc.groupHold().pins) } renderRecord(s, t, columns, sc) } diff --git a/hold.go b/hold.go index 8b507fb..02ff5d5 100644 --- a/hold.go +++ b/hold.go @@ -45,7 +45,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV } w := pathDraw{s: s, held: held, a: &a} if isRef(a.key) && len(a.tail) > 0 { - group := sc.hold() + group := sc.groupHold() w.held, w.pins = &group.hold, &group.pins } if r, done := w.held.value[a.path]; done { diff --git a/holdset.go b/holdset.go index c798226..eacf77f 100644 --- a/holdset.go +++ b/holdset.go @@ -12,8 +12,9 @@ type holdSet struct { } // renderScope is where a render reads its reference paths: a hold set, in the draw group of the -// template rendering; and row, the row of the table rendering, which its columns read. It passes by value, and what is read from it reaches a map key, an interface or -// a func value only as a copy; else sc, and with it every render's hold set, moves to the heap. +// template rendering; and row, the row of the table rendering, which its columns read. It passes +// by value, and what is read from it reaches a map key, an interface or a func value only as a +// copy; else sc, and with it every render's hold set, moves to the heap. type renderScope struct { set *holdSet group string @@ -67,7 +68,7 @@ func (sc renderScope) rowOf(t *table) int { return sc.row.row } -func (sc renderScope) hold() *groupHold { +func (sc renderScope) groupHold() *groupHold { if sc.group == "" { return &sc.set.unnamed } diff --git a/path.go b/path.go index 139fe2f..29574ed 100644 --- a/path.go +++ b/path.go @@ -332,9 +332,9 @@ func (w *pathProbe) readRow(t *table, r tableRoute) error { return nil } -// pathDraw draws the rows and variants a proved path reads, pinning the rows in pins, which -// only a path from a reference needs: a table is a category. For a held read, held keeps the -// variant drawn at each level of a, so paths sharing a prefix share it. +// pathDraw draws the rows and variants a proved path reads, pinning the rows in pins; pins is nil +// for a sibling path, which never crosses a table, since a table is only a category. For a held +// read, held keeps the variant drawn at each level of a, so paths sharing a prefix share it. type pathDraw struct { s *session pins *pinSet diff --git a/record.go b/record.go index 4ca8d0c..d0b4a79 100644 --- a/record.go +++ b/record.go @@ -159,7 +159,7 @@ func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, err } switch n := n.(type) { case *table: - n.drawIn(s, &sc.hold().pins) + n.drawIn(s, &sc.groupHold().pins) return n, nil case *tableRow: return n.t, nil @@ -264,7 +264,7 @@ func recordOf(n node) (*template, []Column, error) { func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.table != nil { - sc = sc.at(t.table.pinnedRow, &sc.hold().pins) + sc = sc.at(t.table.pinnedRow, &sc.groupHold().pins) } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { diff --git a/render.go b/render.go index 8e9a7c9..9bffb12 100644 --- a/render.go +++ b/render.go @@ -33,7 +33,7 @@ func (f *Generator) Fake(path string) (string, error) { if _, ok := n.(*folder); ok { return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) } - return render(f.rand, n, sc.at(n, &sc.hold().pins)), nil + return render(f.rand, n, sc.at(n, &sc.groupHold().pins)), nil } // descend walks a caller's path to the node it names, pinning in sc the rows it @@ -43,7 +43,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er if _, err := (&pathProbe{drawn: map[*table]bool{}}).walk(root, segments); err != nil { return nil, err } - return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.hold().pins}), nil + return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.groupHold().pins}), nil } // render evaluates a compiled node to a string. compile validates every node up diff --git a/todo.md b/todo.md index 37eb540..3420edc 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Split `hold`'s two lifetimes, the expansion's sibling hold and the render's group - hold, into two types, and stop `readField` rewriting `sc.pins`: which pins a column - reads depends on which distant caller last overrode them, the cost seven seats named. - Return each phase's output from compile, link and format compile, not fill `template`'s 25 fields in place from four files: five seats had to know which phase a function runs in to know which fields are valid yet. -- 2.52.0 From 243094e4abb9391e260a293df6b035a83e2f926f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:19:17 +0200 Subject: [PATCH 209/966] Name every Fake receiver and fold the function-method rule into its test in the one-name decision --- docs/decisions.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index f0a3be8..8eedcfd 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -309,15 +309,15 @@ file the stem of the units it holds. ## One name, one meaning -Two declarations of any kind — function, method, field, type — share a name only where -one definition covers both: the same kind of value, counted or addressed the same way, so -a reader landing on the wrong one concludes nothing false. `Fake` on each template kind -renders one value from what it is called on; `renderScope.groupHold` returns the `*groupHold` it -names. A bare function never shares a name with a method: a search for the name returns -both, and a call site shows no receiver to tell them apart. `TestNoFunctionSpellsAMethod` -holds that, and review judges the rest. Serves goal 2: names that tell the truth, and reaching -the unit behind a symptom without asking a person. Decided 2026-09-25 by Lilleman; valid -while goal 2 counts reading cost. +Two declarations of any kind — function, method, field, type — share a name only where one +definition covers both: the same kind of value, counted or addressed the same way, so a +reader landing on the wrong one concludes nothing false. `Fake` renders one value on +`Generator`, `Template` and `RecordTemplate`; `renderScope.groupHold` returns the +`*groupHold` it names. `TestNoFunctionSpellsAMethod` keeps a bare function off a method's +name, since a search returns both and a call site shows no receiver; review judges the +rest. Serves goal 2: names that tell the truth, and reaching the unit behind a symptom +without asking a person. Decided 2026-09-25 by Lilleman; valid while goal 2 counts reading +cost. ## The package stays flat -- 2.52.0 From e257dc0e6e6e96ce742be8692da26e0004bb7342 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:37:32 +0200 Subject: [PATCH 210/966] Return link's and format compile's outputs as templateLink and formatOps, assigned once in linkTemplate --- data.go | 2 +- datatype.go | 6 ++--- doc.go | 4 ++-- drawfence.go | 30 ++++++++++++++----------- graph.go | 2 +- hold.go | 10 ++++----- holdfence.go | 24 ++++++++++---------- holdset.go | 4 ++-- inline.go | 3 +-- node.go | 55 ++++++++++++++++++--------------------------- node_test.go | 2 +- reference.go | 61 +++++++++++++++++++++++++++++--------------------- render.go | 14 ++++++------ render_test.go | 4 ++-- shape_test.go | 2 +- table.go | 5 ++--- value.go | 10 ++++----- 17 files changed, 120 insertions(+), 118 deletions(-) diff --git a/data.go b/data.go index f08d07a..a4cb217 100644 --- a/data.go +++ b/data.go @@ -249,7 +249,7 @@ func checkRenders(s nodeScope) error { fence := &drawCheck{} refs := false if err := s(func(path string, n node) error { - if t, ok := n.(*template); ok && len(t.refs) > 0 { + if t, ok := n.(*template); ok && len(t.link.refs) > 0 { refs = true } return fence.checkDrawGroup(path, n) diff --git a/datatype.go b/datatype.go index 8dbf0c3..aba4a9b 100644 --- a/datatype.go +++ b/datatype.go @@ -76,7 +76,7 @@ func checkColumns(s nodeScope) error { checked[column] = true items, _ := columnItems(column) for _, it := range items { - if r := it.readsColumn; r != nil { + if r := it.link.readsColumn; r != nil { if err := check(r.a.key[1:], r.a.tail[0], r.column); err != nil { return err } @@ -119,8 +119,8 @@ func columnDatatype(n node) DataType { // itemDatatype is the datatype a column item declares, else that of the column it reads. func itemDatatype(t *template) DataType { - if t.datatype == DataTypeString && t.readsColumn != nil { - return columnDatatype(t.readsColumn.column) + if t.datatype == DataTypeString && t.link.readsColumn != nil { + return columnDatatype(t.link.readsColumn.column) } return t.datatype } diff --git a/doc.go b/doc.go index e079331..a518fdf 100644 --- a/doc.go +++ b/doc.go @@ -24,9 +24,9 @@ package fejkdata // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. // - hold — keeping one draw of a name, so every route to it reads that value: -// `template.held`, `hold`, `groupHold`, `readField`. +// `formatOps.held`, `hold`, `groupHold`, `readField`. // - draw group — reference draws held apart inside one render: -// `template.drawGroup` as the data spells it, `template.drawGroupKey` as a +// `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a // render reads it, `holdSet`, `renderScope`. // - pin — fixing which row of a table the render uses, which the draw fences replay: // `pinSet`, `pinSet.pin`, `table.drawIn`. diff --git a/drawfence.go b/drawfence.go index b46aff1..eb08c3f 100644 --- a/drawfence.go +++ b/drawfence.go @@ -83,7 +83,7 @@ func (c *drawCheck) checkDraws(path string, n node) error { func checkOwnFamily(t *template) error { own := t.table if own == nil { - own = t.cellOf + own = t.cell.table } if own == nil { return nil @@ -116,7 +116,7 @@ func checkOwnFamily(t *template) error { func renderDraws(t *template) *drawWalk { w := newDrawWalk() for _, e := range renderEdges(t) { - w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}}) + w.edge(t, e, drawAt{group: t.link.drawGroupKey, route: drawRoute{e.reached(), e.label}}) } return w } @@ -189,7 +189,7 @@ func repeats(n node) bool { func grouped(n node) bool { t, isTemplate := n.(*template) - return isTemplate && t.drawGroupKey != "" + return isTemplate && t.link.drawGroupKey != "" } // readsTable reports whether a reference of n names a table, whose draw the render's @@ -209,7 +209,7 @@ func refRead(n node, label string) (arm, bool) { if !isTemplate { return arm{}, false } - a := splitArm(label, t.refs) + a := splitArm(label, t.link.refs) return a, isRef(a.key) } @@ -219,7 +219,7 @@ func checkColumnDraws(t *template, columns []string) error { return columnDraws( func columnDraws(t *template, columns []string) *drawWalk { w := newDrawWalk() for _, name := range columns { - w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) + w.walk(t.fields[name], drawAt{group: t.link.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) } return w } @@ -311,25 +311,29 @@ func (w *drawWalk) walk(n node, at drawAt) { if repeats(n) { return } - if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" { - at.group = t.drawGroupKey + if t, isTemplate := n.(*template); isTemplate && t.link.drawGroupKey != "" { + at.group = t.link.drawGroupKey } if t, isTable := n.(*table); isTable { at.wholeTable = t } for _, e := range renderEdges(n) { - cell, isCell := e.to.(*template) + to, isTemplate := e.to.(*template) + cell := cellSite{} + if isTemplate { + cell = to.cell + } switch { - case !isCell || cell.cellOf == nil: + case cell.table == nil: w.edge(n, e, at) - case cell.cellOf == at.wholeTable: + case cell.table == at.wholeTable: in := at in.wholePins = at.wholePins.clone() - in.wholePins.add(cell.cellOf, cell.cellRow) + in.wholePins.add(cell.table, cell.row) w.edge(n, e, in) - case at.pins.clash(cell.cellOf, cell.cellRow) == nil: + case at.pins.clash(cell.table, cell.row) == nil: in := at - in.pins = at.pins.entered(cell.cellOf, cell.cellRow) + in.pins = at.pins.entered(cell.table, cell.row) w.edge(n, e, in) } } diff --git a/graph.go b/graph.go index 252bcba..5eab557 100644 --- a/graph.go +++ b/graph.go @@ -136,7 +136,7 @@ func renderEdges(n node) []renderEdge { es = append(es, renderEdge{leaf, a.spelling, operand}) } } - for _, o := range n.ops { + for _, o := range n.compiled.ops { for _, a := range o.operands { add(a, o.fn) } diff --git a/hold.go b/hold.go index 02ff5d5..54b553b 100644 --- a/hold.go +++ b/hold.go @@ -37,7 +37,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV if s.trace != nil { traceRead(s.trace, t, sc, a) } - if !t.held[a.key] { + if !t.compiled.held[a.key] { if len(a.tail) > 0 { panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) } @@ -73,8 +73,8 @@ func traceRead(trace *renderTrace, t *template, sc renderScope, a arm) { var table string var row int switch { - case t.cellOf != nil: - table, row = t.cellOf.path, t.cellRow + case t.cell.table != nil: + table, row = t.cell.table.path, t.cell.row case t.table != nil: table, row = t.table.path, sc.rowOf(t.table) } @@ -89,8 +89,8 @@ func renderLeaf(s *session, n node, sc renderScope) readValue { case *nullItem: return readValue{null: true} case *template: - if leaf.readsColumn != nil { - return readField(s, leaf, nil, sc.in(leaf), leaf.readsColumn.a) + if leaf.link.readsColumn != nil { + return readField(s, leaf, nil, sc.in(leaf), leaf.link.readsColumn.a) } } return readValue{text: render(s, n, sc)} diff --git a/holdfence.go b/holdfence.go index 59be3f0..83a14fb 100644 --- a/holdfence.go +++ b/holdfence.go @@ -12,12 +12,12 @@ import ( // docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences func heldCheck(path string, n node) error { t, ok := n.(*template) - if !ok || len(t.held) == 0 { + if !ok || len(t.compiled.held) == 0 { return nil } - readers := pathKeyReaders(t.ops, t.pathKeys) + readers := pathKeyReaders(t.compiled.ops, t.compiled.pathKeys) for _, name := range heldNames(t) { - if _, isPath := t.pathKeys[name]; isPath && isRef(name) { + if _, isPath := t.compiled.pathKeys[name]; isPath && isRef(name) { continue // held for the render: drawCheck compares every read of it, by path, across the render and its groups } if err := checkNameHeld(t, name, readers); err != nil { @@ -31,13 +31,13 @@ func heldCheck(path string, n node) error { // in name order: a level read both ways is reported by the operand's fence, and // which overlap is reported does not vary. func heldNames(t *template) []string { - names := make([]string, 0, len(t.held)) - for name := range t.held { + names := make([]string, 0, len(t.compiled.held)) + for name := range t.compiled.held { names = append(names, name) } sort.Slice(names, func(i, j int) bool { - _, pi := t.pathKeys[names[i]] - _, pj := t.pathKeys[names[j]] + _, pi := t.compiled.pathKeys[names[i]] + _, pj := t.compiled.pathKeys[names[j]] if pi != pj { return !pi } @@ -51,12 +51,12 @@ func heldNames(t *template) []string { // produces. func heldNodes(t *template, name string, readers []reader) map[node]bool { held := map[node]bool{} - if _, isPath := t.pathKeys[name]; !isPath { + if _, isPath := t.compiled.pathKeys[name]; !isPath { operandDraw(t.head(name), held) return held } for _, r := range readers { - if a := splitArm(r.spelling, t.refs); a.key == name { + if a := splitArm(r.spelling, t.link.refs); a.key == name { coverPath(t.head(name), a.tail, held) } } @@ -71,10 +71,10 @@ func checkNameHeld(t *template, name string, readers []reader) error { if len(held) == 0 { return nil // a fixed head holds nothing to reach } - reader, isPath := t.pathKeys[name] + reader, isPath := t.compiled.pathKeys[name] seen := map[node]bool{} for _, e := range renderEdges(t) { - if splitArm(e.label, t.refs).key == name || !renders(e.to, held, seen) { + if splitArm(e.label, t.link.refs).key == name || !renders(e.to, held, seen) { continue } if isPath { @@ -87,7 +87,7 @@ func checkNameHeld(t *template, name string, readers []reader) error { // operandReader names the builtin whose operand holds name. func operandReader(t *template, name string) string { - for _, o := range t.ops { + for _, o := range t.compiled.ops { for _, a := range o.operands { if a.key == name { return o.fn diff --git a/holdset.go b/holdset.go index eacf77f..93be3fa 100644 --- a/holdset.go +++ b/holdset.go @@ -42,8 +42,8 @@ func expandAnew(s *session, t *template) string { // in is the scope t renders in: its draw group where it names one, else its caller's. func (sc renderScope) in(t *template) renderScope { - if t.drawGroupKey != "" { - sc.group = t.drawGroupKey + if t.link.drawGroupKey != "" { + sc.group = t.link.drawGroupKey } return sc } diff --git a/inline.go b/inline.go index 3685670..91749e0 100644 --- a/inline.go +++ b/inline.go @@ -168,7 +168,6 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { if !ok { return nil } - t.keyDrawGroup("") for _, name := range refTokens(t.tokens) { sigil, rest, err := refShape(name) if err != nil { @@ -178,6 +177,6 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest) } } - return linkTemplateRefs(nil, path, "", t, root) + return linkTemplate(nil, path, "", t, root) }) } diff --git a/node.go b/node.go index f4a689c..352f96b 100644 --- a/node.go +++ b/node.go @@ -41,9 +41,10 @@ func (*nullItem) isNode() {} // template renders a format string, substituting {tokens} from fields. A bare // JSON string is a template with no fields. repeat (default 1) renders that format // that many times and joins the results with separator (default ""), each render -// an independent pick. Every format compiles in `linkTemplateRefs`. +// an independent pick. Every format compiles in `linkTemplate`. type template struct { - // Filled by `compileString`, `compileTemplate` and `table.compileRowFormat`: + // Filled by `compileString`, `compileTemplate`, `table.checkCells` and + // `table.compileRowFormat`: format string tokens []formatToken fields map[string]node @@ -54,24 +55,21 @@ type template struct { fromString bool // written as a JSON string rather than an object isRecord bool // compiled at the top without a repeat, so its fields are record columns table *table // the table whose format this is, whose columns are the fields + cell cellSite - // Filled by `checkCells`: - cellOf *table // the table whose cell this is - cellRow int // the row the cell sits in + // Filled by `linkTemplate`, from the assembled tree: + link templateLink + compiled formatOps +} - // Filled by `template.compileFormat` at link, from `template.tokens` and `template.refs`: - ops []op // what expand walks - grow int // minimum output size, to size the render buffer - // pathKeys maps each key a dotted path in the format starts from to one path token - // reading it, which is the half of an overlap the fences name. nil when the - // format takes no path. - pathKeys map[string]string - // held is every name drawn once per expansion: the path keys above, plus the - // siblings a {calc()} reads. nil when the format holds nothing (see expand). - held map[string]bool - heldLocal bool // some held name is kept by the expansion itself, so expand makes its hold +// cellSite is the table cell a template is compiled from; table is nil for any other template. +type cellSite struct { + table *table + row int +} - // Filled by `linkTemplateRefs` and `keyDrawGroup`, from the assembled tree: +// templateLink is what a template's references resolve to in the assembled tree. +type templateLink struct { refs map[string]refBinding // each reference the format reads -> what it resolves to refHeads map[string]node // each refBinding.key -> the category it names readsColumn *columnRead // set when the format is one reference alone reading a record's column @@ -83,7 +81,7 @@ func (*template) isNode() {} // head is the node an arm's key names: a sibling field, or a reference's category. func (t *template) head(key string) node { if isRef(key) { - return t.refHeads[key] + return t.link.refHeads[key] } return t.fields[key] } @@ -220,15 +218,14 @@ func (t *template) fixedText() (string, bool) { return "", false } -// compileFormat compiles the format into ops, and applies the fences that need the +// compileFormat compiles a format into ops, and applies the fences that need the // compiled reads. -func (t *template) compileFormat() error { - c := compileOps(t.tokens, t.refs) - t.ops, t.grow, t.pathKeys, t.held, t.heldLocal = c.ops, c.grow, c.pathKeys, c.held, c.heldLocal - if err := checkNoOverlap(t.ops, t.pathKeys); err != nil { - return err +func compileFormat(toks []formatToken, refs map[string]refBinding) (formatOps, error) { + c := compileOps(toks, refs) + if err := checkNoOverlap(c.ops, c.pathKeys); err != nil { + return c, err } - return checkNoRepeatedRead(c) + return c, checkNoRepeatedRead(c) } func compileChoice(items []any, pos position) (node, error) { @@ -442,14 +439,6 @@ func drawGroupOf(m map[string]any, repeat int) (string, error) { return name, nil } -// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name -// is local to its category. -func (t *template) keyDrawGroup(category string) { - if t.drawGroup != "" { - t.drawGroupKey = category + "/" + t.drawGroup - } -} - // weightOf reads a node's "weight" (default 1) from its raw JSON form. Only // template objects carry weight; a present one must be finite and positive. func weightOf(raw any) (float64, error) { diff --git a/node_test.go b/node_test.go index f994f27..225f496 100644 --- a/node_test.go +++ b/node_test.go @@ -94,7 +94,7 @@ func TestEveryFormatCompilesAtLink(t *testing.T) { compiled := map[string]bool{} if err := eachNode(n, "t", func(path string, m node) error { if tm, ok := m.(*template); ok { - compiled[path+" "+tm.format] = tm.ops != nil + compiled[path+" "+tm.format] = tm.compiled.ops != nil } return nil }); err != nil { diff --git a/reference.go b/reference.go index 4814a10..0563604 100644 --- a/reference.go +++ b/reference.go @@ -74,52 +74,63 @@ func refSegments(name string, folder []string) ([]string, error) { func linkRefs(root map[string]node) error { return eachTemplate(root, func(folder []string, path string, t *template) error { category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") - t.keyDrawGroup(category) - return linkTemplateRefs(folder, path, category, t, root) + return linkTemplate(folder, path, category, t, root) }) } -func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { - if t.cellOf != nil { - path = fmt.Sprintf("%s, line %d", path, t.cellRow+2) +// linkTemplate resolves t's references in category, "" for an inline template, and +// compiles its format against them. +func linkTemplate(folder []string, path, category string, t *template, root map[string]node) error { + if t.cell.table != nil { + path = fmt.Sprintf("%s, line %d", path, t.cell.row+2) } - if names := refTokens(t.tokens); len(names) > 0 { - if err := t.bindRefs(folder, path, category, names, root); err != nil { - return err - } + link, err := t.resolveLink(folder, path, category, root) + if err != nil { + return err } - if err := t.compileFormat(); err != nil { + compiled, err := compileFormat(t.tokens, link.refs) + if err != nil { return fmt.Errorf("%s: %w", path, err) } - t.readsColumn = columnReadOf(t) + t.link, t.compiled = link, compiled return nil } -// bindRefs fills refs and refHeads, refusing a reference to t's own category: +// resolveLink binds every reference t reads, refusing one to t's own category, and keys +// t's draw group by that category, so a name is local to it: // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load -func (t *template) bindRefs(folder []string, path, category string, names []string, root map[string]node) error { - t.refs = make(map[string]refBinding, len(names)) - t.refHeads = make(map[string]node, len(names)) +func (t *template) resolveLink(folder []string, path, category string, root map[string]node) (templateLink, error) { + var link templateLink + if t.drawGroup != "" { + link.drawGroupKey = category + "/" + t.drawGroup + } + names := refTokens(t.tokens) + if len(names) == 0 { + return link, nil + } + link.refs = make(map[string]refBinding, len(names)) + link.refHeads = make(map[string]node, len(names)) for _, name := range names { segments, err := refSegments(name, folder) if err != nil { - return fmt.Errorf("%s: reference {%s}: %w", path, name, err) + return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } categorySegs, target, tail, err := resolveCategory(root, segments) if err != nil { - return fmt.Errorf("%s: reference {%s}: %w", path, name, err) + return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } key := "/" + strings.Join(categorySegs, ".") if category != "" && key == "/"+category { - return fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) + return link, fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) } if err := checkPath(target, tail, key); err != nil { - return fmt.Errorf("%s: reference {%s}: %w", path, name, err) + return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } - t.refHeads[key] = target - t.refs[name] = refBinding{key, tail} + link.refHeads[key] = target + link.refs[name] = refBinding{key, tail} } - return nil + link.readsColumn = columnReadOf(t, link) + return link, nil } // columnRead is a record's column read by a format of that one reference alone, which is the @@ -129,13 +140,13 @@ type columnRead struct { column node } -func columnReadOf(t *template) *columnRead { +func columnReadOf(t *template, link templateLink) *columnRead { name, lone := loneRef(t.tokens) if !lone || t.repeat != 1 { return nil } - a := splitArm(name, t.refs) - target, isTemplate := t.head(a.key).(*template) + a := splitArm(name, link.refs) + target, isTemplate := link.refHeads[a.key].(*template) if !isTemplate || !target.isRecord || len(a.tail) != 1 { return nil } diff --git a/render.go b/render.go index 9bffb12..5fe866a 100644 --- a/render.go +++ b/render.go @@ -75,7 +75,7 @@ func render(s *session, n node, sc renderScope) string { return expand(s, n, sc) } var b strings.Builder - b.Grow(n.repeat * (n.grow + len(n.separator))) + b.Grow(n.repeat * (n.compiled.grow + len(n.separator))) for i := 0; i < n.repeat; i++ { if i > 0 { b.WriteString(n.separator) @@ -104,19 +104,19 @@ func pick(s *session, c *choice) node { // cannot fail. func expand(s *session, t *template, sc renderScope) string { var b strings.Builder - b.Grow(t.grow) + b.Grow(t.compiled.grow) // One draw per held name, for this expansion only: a nested template and each // repeat iteration get their own, since each is its own expansion. A reference // path reads the render's hold in sc instead. var held *hold - if t.heldLocal { + if t.compiled.heldLocal { held = &hold{ - variant: make(map[string]node, len(t.held)), - value: make(map[string]readValue, len(t.held)), + variant: make(map[string]node, len(t.compiled.held)), + value: make(map[string]readValue, len(t.compiled.held)), } } - for i := range t.ops { - o := &t.ops[i] + for i := range t.compiled.ops { + o := &t.compiled.ops[i] switch o.kind { case literalRun: b.WriteString(o.lit) diff --git a/render_test.go b/render_test.go index 4302a69..d0da7aa 100644 --- a/render_test.go +++ b/render_test.go @@ -40,8 +40,8 @@ func TestGrowIsALowerBound(t *testing.T) { } for i := 0; i < 50; i++ { var set holdSet - if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.grow { - t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) + if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.compiled.grow { + t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.compiled.grow) } } } diff --git a/shape_test.go b/shape_test.go index 9747da2..bc72377 100644 --- a/shape_test.go +++ b/shape_test.go @@ -101,7 +101,7 @@ func readsFact(n node) string { collect(cell) } case *template: - for _, b := range n.refs { + for _, b := range n.link.refs { set[strings.TrimPrefix(b.key, "/")] = true } for _, field := range n.fields { diff --git a/table.go b/table.go index 65b2ec8..4c9165d 100644 --- a/table.go +++ b/table.go @@ -308,15 +308,14 @@ func (t *table) checkCells() error { if strings.IndexByte(cell, '{') < 0 && strings.IndexByte(cell, '}') < 0 { continue } - n, err := compileString(cell) + toks, err := parseChecked(cell, nil) if err != nil { return fmt.Errorf("line %d, %s: %w", row+2, t.header[col], err) } if t.cellTemplates == nil { t.cellTemplates = map[int]*template{} } - t.cellTemplates[i] = n.(*template) - t.cellTemplates[i].cellOf, t.cellTemplates[i].cellRow = t, row + t.cellTemplates[i] = &template{format: cell, tokens: toks, repeat: 1, fromString: true, cell: cellSite{t, row}} } return nil } diff --git a/value.go b/value.go index 4b405c6..3a764d8 100644 --- a/value.go +++ b/value.go @@ -34,7 +34,7 @@ func (p *valueProof) checkDatatype(path string, n node) error { if !ok || t.datatype == DataTypeString { return nil } - if r := t.readsColumn; r != nil && columnDatatype(r.column) == t.datatype { + if r := t.link.readsColumn; r != nil && columnDatatype(r.column) == t.datatype { return fmt.Errorf(`%s: %s takes datatype %s from the column it reads; drop "datatype"`, path, t.format, t.datatype) } if reason := p.proveColumnItem(t).not[t.datatype]; reason != "" { @@ -45,10 +45,10 @@ func (p *valueProof) checkDatatype(path string, n node) error { // proveColumnItem proves a column item: what it renders, or, when it is a column it reads, that column. func (p *valueProof) proveColumnItem(t *template) proven { - if t.readsColumn == nil { + if t.link.readsColumn == nil { return p.prove(t) } - return p.proveColumn(t.readsColumn.column) + return p.proveColumn(t.link.readsColumn.column) } // proveColumn proves a column over what its items draw, a null item marking it null rather than @@ -149,12 +149,12 @@ func (p *valueProof) proveTemplate(t *template) proven { return unproven(fmt.Sprintf("%q carries a repeat, which composes text rather than one value", t.format)) case fixed: return literalValue(lit) - case len(t.ops) != 1: + case len(t.compiled.ops) != 1: v := unproven(notOneValue(t.format, typedCalls)) v.notOperand = notOneValue(t.format, operandCalls) return v } - o := t.ops[0] + o := t.compiled.ops[0] body, name, args := o.body, o.fn, o.args switch _, isTransform := transforms[name]; { case o.kind == fieldAlternation: -- 2.52.0 From 6158eb0f46a58d342705850bb5f208b5b73fb4e3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:41:22 +0200 Subject: [PATCH 211/966] Build a table cell's template through compileString, and drop the done round item --- node.go | 12 ++++++++---- table.go | 4 ++-- todo.md | 3 --- 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/node.go b/node.go index 352f96b..c6b5a2c 100644 --- a/node.go +++ b/node.go @@ -68,7 +68,7 @@ type cellSite struct { row int } -// templateLink is what a template's references resolve to in the assembled tree. +// templateLink is what a template resolves to in the assembled tree. type templateLink struct { refs map[string]refBinding // each reference the format reads -> what it resolves to refHeads map[string]node // each refBinding.key -> the category it names @@ -172,7 +172,11 @@ func compileAt(v any, pos position) (node, error) { func compileItem(v any, pos position) (node, error) { switch v := v.(type) { case string: - return compileString(v) + t, err := compileString(v, cellSite{}) + if err != nil { + return nil, err + } + return t, nil case []any: return compileChoice(v, pos) case map[string]any: @@ -199,12 +203,12 @@ func jsonKind(v any) string { return fmt.Sprintf("%T", v) } -func compileString(s string) (node, error) { +func compileString(s string, cell cellSite) (*template, error) { toks, err := parseChecked(s, nil) if err != nil { return nil, err } - return &template{format: s, tokens: toks, repeat: 1, fromString: true}, nil + return &template{format: s, tokens: toks, repeat: 1, fromString: true, cell: cell}, nil } // fixedText is one render's output when the format holds no token; repeat is the caller's. diff --git a/table.go b/table.go index 4c9165d..d407d26 100644 --- a/table.go +++ b/table.go @@ -308,14 +308,14 @@ func (t *table) checkCells() error { if strings.IndexByte(cell, '{') < 0 && strings.IndexByte(cell, '}') < 0 { continue } - toks, err := parseChecked(cell, nil) + n, err := compileString(cell, cellSite{t, row}) if err != nil { return fmt.Errorf("line %d, %s: %w", row+2, t.header[col], err) } if t.cellTemplates == nil { t.cellTemplates = map[int]*template{} } - t.cellTemplates[i] = &template{format: cell, tokens: toks, repeat: 1, fromString: true, cell: cellSite{t, row}} + t.cellTemplates[i] = n } return nil } diff --git a/todo.md b/todo.md index 3420edc..25716c2 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Return each phase's output from compile, link and format compile, not fill - `template`'s 25 fields in place from four files: five seats had to know which phase a - function runs in to know which fields are valid yet. - Make `renderEdges` panic on a node it does not handle, as `render` does: its `default: return nil` silently exempts a new node kind from every fence, and nothing at `expand` says `renderEdges` mirrors it. -- 2.52.0 From 62ae658b4d1c7ede20bc46b760397b5b932b2810 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:42:14 +0200 Subject: [PATCH 212/966] Drop checkCells from the compile fill header --- node.go | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/node.go b/node.go index c6b5a2c..105bb53 100644 --- a/node.go +++ b/node.go @@ -43,8 +43,7 @@ func (*nullItem) isNode() {} // that many times and joins the results with separator (default ""), each render // an independent pick. Every format compiles in `linkTemplate`. type template struct { - // Filled by `compileString`, `compileTemplate`, `table.checkCells` and - // `table.compileRowFormat`: + // Filled by `compileString`, `compileTemplate` and `table.compileRowFormat`: format string tokens []formatToken fields map[string]node -- 2.52.0 From 446a3f98d62fc791906f9eac86c7ee0decc690d8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:56:34 +0200 Subject: [PATCH 213/966] Test that renderEdges panics on a node kind it does not handle --- render_test.go | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/render_test.go b/render_test.go index d0da7aa..5fda506 100644 --- a/render_test.go +++ b/render_test.go @@ -2,6 +2,8 @@ package fejkdata import ( "encoding/json" + "fmt" + "strings" "testing" ) @@ -54,3 +56,21 @@ func quote(s string) string { } return string(b) } + +type unhandledNode struct{} + +func (*unhandledNode) isNode() {} + +func TestRenderEdgesPanicsOnAnUnhandledNode(t *testing.T) { + for _, n := range []node{&folder{}, &nullItem{}} { + if es := renderEdges(n); len(es) != 0 { + t.Errorf("renderEdges(%T) = %v, want no edges", n, es) + } + } + defer func() { + if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), "*fejkdata.unhandledNode") { + t.Errorf("renderEdges(unhandledNode) recovered %v, want a panic naming the node kind", r) + } + }() + renderEdges(&unhandledNode{}) +} -- 2.52.0 From d19c5d4719f9ff210ba814ca519f0359163dea19 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 28 Sep 2026 23:58:14 +0200 Subject: [PATCH 214/966] Make renderEdges panic on a node kind it does not handle, and drop the done round item --- graph.go | 6 ++++-- render.go | 1 + todo.md | 3 --- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/graph.go b/graph.go index 5eab557..d47e196 100644 --- a/graph.go +++ b/graph.go @@ -114,7 +114,7 @@ func (e renderEdge) reached() string { return "{" + e.label + "}" } -// renderEdges lists the children rendering n recurses into, mirroring expand: a +// renderEdges lists the children rendering n recurses into, mirroring render: a // choice's items, and a template's field/reference tokens plus its operands. A // folder renders nothing, so it has no edges. func renderEdges(n node) []renderEdge { @@ -155,8 +155,10 @@ func renderEdges(n node) []renderEdge { es = append(es, renderEdge{to: c.node, label: n.t.header[n.i]}) } return es - default: + case *folder, *nullItem: return nil + default: + panic(fmt.Sprintf("fejkdata: renderEdges has no case for node %T", n)) } } diff --git a/render.go b/render.go index 5fe866a..b3b776c 100644 --- a/render.go +++ b/render.go @@ -49,6 +49,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over a hold set of its own. +// renderEdges mirrors this switch for the fences, so a new node kind goes in both. func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: diff --git a/todo.md b/todo.md index 25716c2..a295900 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Make `renderEdges` panic on a node it does not handle, as `render` does: its - `default: return nil` silently exempts a new node kind from every fence, and nothing - at `expand` says `renderEdges` mirrors it. - Carry the arm on `renderEdge` so `graph.go`, `drawfence.go` and `holdfence.go` stop re-parsing `label` through `splitArm`, whose key differs before and after link. - Give `pathCover` a doc comment and say in each path walk's comment how it differs from -- 2.52.0 From 181e0bbb6e3a74fbac29442d73af5b52d892e559 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:01:03 +0200 Subject: [PATCH 215/966] Keep the renderEdges test to the panic it names --- render_test.go | 5 ----- 1 file changed, 5 deletions(-) diff --git a/render_test.go b/render_test.go index 5fda506..478f1a0 100644 --- a/render_test.go +++ b/render_test.go @@ -62,11 +62,6 @@ type unhandledNode struct{} func (*unhandledNode) isNode() {} func TestRenderEdgesPanicsOnAnUnhandledNode(t *testing.T) { - for _, n := range []node{&folder{}, &nullItem{}} { - if es := renderEdges(n); len(es) != 0 { - t.Errorf("renderEdges(%T) = %v, want no edges", n, es) - } - } defer func() { if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), "*fejkdata.unhandledNode") { t.Errorf("renderEdges(unhandledNode) recovered %v, want a panic naming the node kind", r) -- 2.52.0 From 010bcd15bcc2b12683e7e381e4638101ee2048ca Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:15:54 +0200 Subject: [PATCH 216/966] Carry the linked arm on renderEdge and reader, and drop the done round item --- drawfence.go | 24 +++++++----------------- graph.go | 13 +++++++++---- holdfence.go | 26 ++++++++++++++------------ todo.md | 2 -- 4 files changed, 30 insertions(+), 35 deletions(-) diff --git a/drawfence.go b/drawfence.go index eb08c3f..86c5192 100644 --- a/drawfence.go +++ b/drawfence.go @@ -96,8 +96,8 @@ func checkOwnFamily(t *template) error { } seen[n] = true for _, e := range renderEdges(n) { - if a, isRef := refRead(n, e.label); isRef { - if head, isTable := n.(*template).head(a.key).(*table); isTable && head.familyRoot() == own.familyRoot() { + if e.readsRef() { + if head, isTable := n.(*template).head(e.read.key).(*table); isTable && head.familyRoot() == own.familyRoot() { return e, head, true } } @@ -161,7 +161,7 @@ func (c *drawCheck) hasRead(n node, stopAtGroup bool) bool { } r := false for _, e := range renderEdges(n) { - if readsOnEdge(n, e.label) || (walksInto(e.to, stopAtGroup) && c.hasRead(e.to, stopAtGroup)) { + if readsOnEdge(n, e) || (walksInto(e.to, stopAtGroup) && c.hasRead(e.to, stopAtGroup)) { r = true break } @@ -173,9 +173,8 @@ func (c *drawCheck) hasRead(n node, stopAtGroup bool) bool { return r } -func readsOnEdge(n node, label string) bool { - a, isRef := refRead(n, label) - return isRef && (len(a.tail) > 0 || readsTable(n, a)) +func readsOnEdge(n node, e renderEdge) bool { + return e.readsRef() && (len(e.read.tail) > 0 || readsTable(n, e.read)) } func walksInto(to node, stopAtGroup bool) bool { @@ -203,16 +202,6 @@ func readsTable(n node, a arm) bool { return isTable } -// refRead is the reference an edge of n reads; false when the edge reads none. -func refRead(n node, label string) (arm, bool) { - t, isTemplate := n.(*template) - if !isTemplate { - return arm{}, false - } - a := splitArm(label, t.link.refs) - return a, isRef(a.key) -} - // checkColumnDraws fences a record's columns as one render. func checkColumnDraws(t *template, columns []string) error { return columnDraws(t, columns).check() } @@ -342,7 +331,8 @@ func (w *drawWalk) walk(n node, at drawAt) { // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { - if a, reads := refRead(from, e.label); reads { + if e.readsRef() { + a := e.read tr := tableReadOf(from.(*template).head(a.key), a, e.to) if k := (readKey{at.group, a.path, w.pinsID(&at.pins), w.pinsID(&at.wholePins)}); !w.read[k] { w.read[k] = true diff --git a/graph.go b/graph.go index d47e196..ef02180 100644 --- a/graph.go +++ b/graph.go @@ -96,15 +96,20 @@ func sortedNames(m map[string]node) []string { } // renderEdge is a child a node renders into, labelled by what reaches it (a field -// name, reference, or choice index) for a readable cycle report. operand names -// the builtin when the label is its operand rather than a token, so an error can -// name it the way the author wrote it. +// name, reference, or choice index) for a readable cycle report. read is the linked +// arm a template's edge reads, zero on any other. operand names the builtin when the +// label is its operand rather than a token, so an error can name it the way the +// author wrote it. type renderEdge struct { to node label string + read arm operand string } +// readsRef reports whether the edge reads a reference. +func (e renderEdge) readsRef() bool { return isRef(e.read.key) } + // reached names an edge as the author spelled it, the vocabulary pathKeyReaders uses // for the sibling fence. func (e renderEdge) reached() string { @@ -133,7 +138,7 @@ func renderEdges(n node) []renderEdge { return } for _, leaf := range pathLeaves(c, a.tail) { - es = append(es, renderEdge{leaf, a.spelling, operand}) + es = append(es, renderEdge{leaf, a.spelling, a, operand}) } } for _, o := range n.compiled.ops { diff --git a/holdfence.go b/holdfence.go index 83a14fb..a91d6aa 100644 --- a/holdfence.go +++ b/holdfence.go @@ -56,8 +56,8 @@ func heldNodes(t *template, name string, readers []reader) map[node]bool { return held } for _, r := range readers { - if a := splitArm(r.spelling, t.link.refs); a.key == name { - coverPath(t.head(name), a.tail, held) + if r.a.key == name { + coverPath(t.head(name), r.a.tail, held) } } return held @@ -74,7 +74,7 @@ func checkNameHeld(t *template, name string, readers []reader) error { reader, isPath := t.compiled.pathKeys[name] seen := map[node]bool{} for _, e := range renderEdges(t) { - if splitArm(e.label, t.link.refs).key == name || !renders(e.to, held, seen) { + if e.read.key == name || !renders(e.to, held, seen) { continue } if isPath { @@ -134,7 +134,7 @@ func operandDraw(n node, into map[node]bool) { } into[n] = true for _, e := range renderEdges(n) { - if isRef(e.label) { + if e.readsRef() { continue } operandDraw(e.to, into) @@ -181,20 +181,22 @@ func checkNoOverlap(ops []op, pathKeys map[string]string) error { names := pathKeyReaders(ops, pathKeys) // Stable over one format-order scan, so two readers of one name (a token and a // calc operand both naming "p") are reported as the format writes them. - sort.SliceStable(names, func(i, j int) bool { return names[i].path < names[j].path }) + sort.SliceStable(names, func(i, j int) bool { return names[i].a.path < names[j].a.path }) for i, level := range names { for _, path := range names[i+1:] { - if strings.HasPrefix(path.path, level.path+".") { - return fmt.Errorf("%s renders a level that {%s} reads a path into; name the fields you want instead", level.label, path.spelling) + if strings.HasPrefix(path.a.path, level.a.path+".") { + return fmt.Errorf("%s renders a level that {%s} reads a path into; name the fields you want instead", level.label, path.a.spelling) } } } return nil } -// reader is one way a format reaches a path key: as written, by its path, -// and how to name it. -type reader struct{ spelling, path, label string } +// reader is one way a format reaches a path key, and how to name it. +type reader struct { + a arm + label string +} // pathKeyReaders lists every way a format reaches a sibling path key, in the order the // format writes them. An operand renders its field, so it names a level exactly @@ -204,12 +206,12 @@ func pathKeyReaders(ops []op, pathKeys map[string]string) []reader { for _, o := range ops { for _, a := range o.operands { if _, isPathKey := pathKeys[a.key]; isPathKey && !isRef(a.key) { - names = append(names, reader{a.spelling, a.path, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) + names = append(names, reader{a, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { if _, isPathKey := pathKeys[a.key]; isPathKey && !isRef(a.key) { - names = append(names, reader{a.spelling, a.path, "token {" + a.spelling + "}"}) + names = append(names, reader{a, "token {" + a.spelling + "}"}) } } } diff --git a/todo.md b/todo.md index a295900..d63b1c4 100644 --- a/todo.md +++ b/todo.md @@ -23,8 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Carry the arm on `renderEdge` so `graph.go`, `drawfence.go` and `holdfence.go` stop - re-parsing `label` through `splitArm`, whose key differs before and after link. - Give `pathCover` a doc comment and say in each path walk's comment how it differs from `pathCheck` at a choice and a table: every seat diffed the four loops by eye. - Make `disagreement` a table keyed by the two datatypes and the proof, dropping the -- 2.52.0 From e09a0434a31018acfef1b9afd22b1d254b97182a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:18:50 +0200 Subject: [PATCH 217/966] Drop the comment restating readsRef --- graph.go | 1 - 1 file changed, 1 deletion(-) diff --git a/graph.go b/graph.go index ef02180..10d7e65 100644 --- a/graph.go +++ b/graph.go @@ -107,7 +107,6 @@ type renderEdge struct { operand string } -// readsRef reports whether the edge reads a reference. func (e renderEdge) readsRef() bool { return isRef(e.read.key) } // reached names an edge as the author spelled it, the vocabulary pathKeyReaders uses -- 2.52.0 From 1542867c71c75ff029bd8896e409b69697ca83d9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:31:24 +0200 Subject: [PATCH 218/966] Say how each path walk differs from pathCheck at a choice and a table, and drop the done round item --- path.go | 10 ++++++++-- todo.md | 2 -- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/path.go b/path.go index 29574ed..3fd8b5b 100644 --- a/path.go +++ b/path.go @@ -257,6 +257,9 @@ func (w *pathCheck) enter(t *template, rest []string) error { return nil } +// pathCover is coverPath's walk. Where pathCheck walks every variant of a choice, it +// covers the first choice it reaches whole and stops; at a table it follows the route +// and selects no row. type pathCover struct{ into map[node]bool } func (w *pathCover) walk(n node, tail []string) (node, error) { @@ -283,8 +286,9 @@ func (w *pathCover) walk(n node, tail []string) (node, error) { return n, nil } -// pathProbe proves a path resolves without drawing: carriedByAll lets one variant -// of a choice stand for all. It marks in drawn the tables a draw would pin. +// pathProbe proves a path resolves without drawing. Where pathCheck walks every +// variant of a choice, it walks the first, which carriedByAll lets stand for all; at a +// table it selects as pathCheck does and marks in drawn the tables a draw would pin. type pathProbe struct { pins pinSet drawn map[*table]bool @@ -335,6 +339,8 @@ func (w *pathProbe) readRow(t *table, r tableRoute) error { // pathDraw draws the rows and variants a proved path reads, pinning the rows in pins; pins is nil // for a sibling path, which never crosses a table, since a table is only a category. For a held // read, held keeps the variant drawn at each level of a, so paths sharing a prefix share it. +// Where pathCheck walks every variant of a choice, it walks the one drawn; at a table it selects +// as pathCheck does and draws the row the route asks for. type pathDraw struct { s *session pins *pinSet diff --git a/todo.md b/todo.md index d63b1c4..e0f7276 100644 --- a/todo.md +++ b/todo.md @@ -23,8 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Give `pathCover` a doc comment and say in each path walk's comment how it differs from - `pathCheck` at a choice and a table: every seat diffed the four loops by eye. - Make `disagreement` a table keyed by the two datatypes and the proof, dropping the operand swap: six seats still needed a truth table to see which item it blames. - Move `session.trace` and `renderTrace.repeatDepth` off the render path: they serve -- 2.52.0 From ac0d177939d3261e926a54f0f6d130858c2eeb08 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:45:22 +0200 Subject: [PATCH 219/966] Pin each column-datatype disagreement with its items in either order --- datatype_test.go | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/datatype_test.go b/datatype_test.go index 46824bb..a10da13 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -54,6 +54,10 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { {"a datatype a typed column's values reject", `{"format":"{/src.score}","datatype":"boolean"}`, "{int(1,9)} prints an integer, not a boolean"}, {"two typed columns it reads", `["{/src.score}","{/src.flag}"]`, `so to read "{/src.score}" as text, write {"format":"{text}","text":"{/src.score}"}`}, {"a typed column read that holds the datatype declared beside it", `[{"format":"1.5","datatype":"number"},"{/src.score}"]`, `so write "{/src.score}" as {"format":"{/src.score}","datatype":"number"}`}, + {"text before a typed column it reads", `["n/a","{/src.score}"]`, `to read that column as text, write {"format":"{text}","text":"{/src.score}"}`}, + {"a weighted item beside a typed column it reads", `["{/src.score}",{"format":"5","weight":2}]`, `give it "datatype": "integer"`}, + {"a typed column read before the datatype it holds", `["{/src.score}",{"format":"1.5","datatype":"number"}]`, `so write "{/src.score}" as {"format":"{/src.score}","datatype":"number"}`}, + {"a typed column read beside a datatype it does not hold", `["{/src.score}",{"format":"true","datatype":"boolean"}]`, `so to read "{/src.score}" as text, write {"format":"{text}","text":"{/src.score}"}`}, {"text beside a weighted typed column read", `[{"format":"{/src.score}","weight":3},"n/a"]`, `to read that column as text, set its "format" to "{text}" and add "text": "{/src.score}"`}, {"a value of the column it reads", `{"format":"{/src.code}","datatype":"integer"}`, `"2x" is not an integer`}, {"a null read into text", `{"format":"{x}","x":"{/src.score}","datatype":"integer"}`, "reads a null"}, -- 2.52.0 From ef6116f795086ab89b483fd2b7dcf01988964996 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:47:50 +0200 Subject: [PATCH 220/966] Make disagreement a table keyed by the two item kinds and the proof, and drop the done round item --- datatype.go | 95 ++++++++++++++++++++++++++++++++++++++++++----------- todo.md | 2 -- 2 files changed, 76 insertions(+), 21 deletions(-) diff --git a/datatype.go b/datatype.go index aba4a9b..033905a 100644 --- a/datatype.go +++ b/datatype.go @@ -144,27 +144,84 @@ func columnItems(n node) (items []*template, nullable bool) { return items, nullable } -// disagreement names the fix for two items of one column holding different datatypes. The item to -// fix declares none unless both do, and is text where the other reads a typed column. -func disagreement(a *template, da DataType, b *template, db DataType) error { - fix, has, other, want := a, da, b, db - if da != DataTypeString && (a.datatype != DataTypeString || db == DataTypeString) { - fix, has, other, want = b, db, a, da - } - proven := (&valueProof{}).proveColumnItem(fix).not[want] == "" +// itemKind is how a column item comes by its datatype: none, read from a typed column, or declared. +type itemKind int + +const ( + kindText itemKind = iota + kindReads + kindDeclares +) + +func kindOf(t *template) itemKind { switch { - case fix.datatype != DataTypeString: - return fmt.Errorf("its items hold %s and %s; a column holds one datatype", da, db) - case has != DataTypeString && proven: - return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", da, db, typedAs(fix, want)) - case has != DataTypeString: - return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", da, db, fix.format, asText(fix)) - case other.datatype == DataTypeString && !proven: - return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) - case fix.fromString: // an object may carry a weight, which this spelling would drop - return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, fix.format, fix.format, want) + case t.datatype != DataTypeString: + return kindDeclares + case itemDatatype(t) != DataTypeString: + return kindReads } - return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, fix.format, want, want) + return kindText +} + +// disagreeing is two items of one column holding different datatypes: fix is the one to change, +// and want the datatype of the other. +type disagreeing struct { + first, second DataType + fix, other *template + want DataType +} + +// disagreementKey is the two items' kinds and whether fix's values prove to hold want. +type disagreementKey struct { + fix, other itemKind + proven bool +} + +var disagreements = map[disagreementKey]func(d disagreeing) error{ + {kindDeclares, kindDeclares, false}: holdOne, + {kindDeclares, kindDeclares, true}: holdOne, + {kindReads, kindDeclares, false}: readAsText, + {kindReads, kindDeclares, true}: readTyped, + {kindReads, kindReads, false}: readAsText, + {kindReads, kindReads, true}: readTyped, + {kindText, kindDeclares, false}: declare, + {kindText, kindDeclares, true}: declare, + {kindText, kindReads, false}: otherAsText, + {kindText, kindReads, true}: declare, +} + +// disagreement names the fix for items a and b of one column holding different datatypes, fixing +// the one that declares least, a when both declare alike. +func disagreement(a *template, da DataType, b *template, db DataType) error { + d := disagreeing{first: da, second: db, fix: a, other: b, want: db} + if kindOf(b) < kindOf(a) { + d.fix, d.other, d.want = b, a, da + } + proven := (&valueProof{}).proveColumnItem(d.fix).not[d.want] == "" + return disagreements[disagreementKey{kindOf(d.fix), kindOf(d.other), proven}](d) +} + +func holdOne(d disagreeing) error { + return fmt.Errorf("its items hold %s and %s; a column holds one datatype", d.first, d.second) +} + +func readTyped(d disagreeing) error { + return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", d.first, d.second, typedAs(d.fix, d.want)) +} + +func readAsText(d disagreeing) error { + return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", d.first, d.second, d.fix.format, asText(d.fix)) +} + +func otherAsText(d disagreeing) error { + return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, d.fix.format, dataTypeNouns[d.want], d.other.format, asText(d.other)) +} + +func declare(d disagreeing) error { + if d.fix.fromString { // an object may carry a weight, which this spelling would drop + return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, d.fix.format, d.fix.format, d.want) + } + return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, d.fix.format, d.want, d.want) } // typedAs names the spelling giving a column-read item datatype d, keeping the other keys an object diff --git a/todo.md b/todo.md index e0f7276..d5139ba 100644 --- a/todo.md +++ b/todo.md @@ -23,8 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Make `disagreement` a table keyed by the two datatypes and the proof, dropping the - operand swap: six seats still needed a truth table to see which item it blames. - Move `session.trace` and `renderTrace.repeatDepth` off the render path: they serve `drawfence_test.go` alone. - Give `Fake` a local hold set like `FakeRecord`, `RecordTemplate.Fake` and -- 2.52.0 From 7a6ca476a8bb0f5ce5e28f14749a931dd7406f36 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:50:48 +0200 Subject: [PATCH 221/966] Pin the proven disagreements of two declared and two read datatypes --- datatype_test.go | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/datatype_test.go b/datatype_test.go index a10da13..4393dde 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -41,7 +41,7 @@ func TestDatatypeAndNullSitOnlyInAColumn(t *testing.T) { func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { tree := map[string]string{ "cat": `[{"format":"{code}","code":"200"},{"format":"{code}","code":"2x","note":"n"}]`, - "src": `{"format":"","code":[null,"200","2x"],"flag":{"format":"{b}","b":["true","false"],"datatype":"boolean"},"score":[null,{"format":"{int(1,9)}","datatype":"integer"}]}`, + "src": `{"format":"","code":[null,"200","2x"],"flag":{"format":"{b}","b":["true","false"],"datatype":"boolean"},"ratio":{"format":"{float(0,9,0)}","datatype":"number"},"score":[null,{"format":"{int(1,9)}","datatype":"integer"}]}`, } for _, c := range []struct{ name, column, want string }{ {"an item beside a typed one", `[{"format":"1","datatype":"integer"},"x"]`, `write it as {"format":"x","datatype":"integer"}`}, @@ -54,6 +54,8 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { {"a datatype a typed column's values reject", `{"format":"{/src.score}","datatype":"boolean"}`, "{int(1,9)} prints an integer, not a boolean"}, {"two typed columns it reads", `["{/src.score}","{/src.flag}"]`, `so to read "{/src.score}" as text, write {"format":"{text}","text":"{/src.score}"}`}, {"a typed column read that holds the datatype declared beside it", `[{"format":"1.5","datatype":"number"},"{/src.score}"]`, `so write "{/src.score}" as {"format":"{/src.score}","datatype":"number"}`}, + {"two datatypes the first holds both of", `[{"format":"1","datatype":"integer"},{"format":"1.5","datatype":"number"}]`, "its items hold integer and number; a column holds one datatype"}, + {"a typed column read holding the datatype of another", `["{/src.ratio}","{/src.score}"]`, `so write "{/src.ratio}" as {"format":"{/src.ratio}","datatype":"integer"}`}, {"text before a typed column it reads", `["n/a","{/src.score}"]`, `to read that column as text, write {"format":"{text}","text":"{/src.score}"}`}, {"a weighted item beside a typed column it reads", `["{/src.score}",{"format":"5","weight":2}]`, `give it "datatype": "integer"`}, {"a typed column read before the datatype it holds", `["{/src.score}",{"format":"1.5","datatype":"number"}]`, `so write "{/src.score}" as {"format":"{/src.score}","datatype":"number"}`}, -- 2.52.0 From fd4906846deb83681d6e5c257cd0624f5be04938 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:51:19 +0200 Subject: [PATCH 222/966] Name disagreement's item pair itemPair --- datatype.go | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/datatype.go b/datatype.go index 033905a..ba5c678 100644 --- a/datatype.go +++ b/datatype.go @@ -163,9 +163,9 @@ func kindOf(t *template) itemKind { return kindText } -// disagreeing is two items of one column holding different datatypes: fix is the one to change, +// itemPair is two items of one column holding different datatypes: fix is the one to change, // and want the datatype of the other. -type disagreeing struct { +type itemPair struct { first, second DataType fix, other *template want DataType @@ -177,7 +177,7 @@ type disagreementKey struct { proven bool } -var disagreements = map[disagreementKey]func(d disagreeing) error{ +var disagreements = map[disagreementKey]func(d itemPair) error{ {kindDeclares, kindDeclares, false}: holdOne, {kindDeclares, kindDeclares, true}: holdOne, {kindReads, kindDeclares, false}: readAsText, @@ -193,7 +193,7 @@ var disagreements = map[disagreementKey]func(d disagreeing) error{ // disagreement names the fix for items a and b of one column holding different datatypes, fixing // the one that declares least, a when both declare alike. func disagreement(a *template, da DataType, b *template, db DataType) error { - d := disagreeing{first: da, second: db, fix: a, other: b, want: db} + d := itemPair{first: da, second: db, fix: a, other: b, want: db} if kindOf(b) < kindOf(a) { d.fix, d.other, d.want = b, a, da } @@ -201,23 +201,23 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { return disagreements[disagreementKey{kindOf(d.fix), kindOf(d.other), proven}](d) } -func holdOne(d disagreeing) error { +func holdOne(d itemPair) error { return fmt.Errorf("its items hold %s and %s; a column holds one datatype", d.first, d.second) } -func readTyped(d disagreeing) error { +func readTyped(d itemPair) error { return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", d.first, d.second, typedAs(d.fix, d.want)) } -func readAsText(d disagreeing) error { +func readAsText(d itemPair) error { return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", d.first, d.second, d.fix.format, asText(d.fix)) } -func otherAsText(d disagreeing) error { +func otherAsText(d itemPair) error { return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, d.fix.format, dataTypeNouns[d.want], d.other.format, asText(d.other)) } -func declare(d disagreeing) error { +func declare(d itemPair) error { if d.fix.fromString { // an object may carry a weight, which this spelling would drop return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, d.fix.format, d.fix.format, d.want) } -- 2.52.0 From 64f339b0dbc69ca0cfb5393ab8da0b2dcb549121 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 00:53:06 +0200 Subject: [PATCH 223/966] Pin holdOne's disagreement by the end of the error, which the other two extend --- datatype_test.go | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/datatype_test.go b/datatype_test.go index 4393dde..9ba95aa 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -46,7 +46,6 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { for _, c := range []struct{ name, column, want string }{ {"an item beside a typed one", `[{"format":"1","datatype":"integer"},"x"]`, `write it as {"format":"x","datatype":"integer"}`}, {"a weighted item beside a typed one", `[{"format":"1","datatype":"integer"},{"format":"2","weight":3}]`, `give it "datatype": "integer"`}, - {"items of two datatypes", `[{"format":"1","datatype":"integer"},{"format":"true","datatype":"boolean"}]`, "a column holds one datatype"}, {"an item beside a typed column it reads", `["{/src.score}","5"]`, `write it as {"format":"5","datatype":"integer"}`}, {"a typed item beside a string column it reads", `["{/src.code}",{"format":"1","datatype":"integer"}]`, `write it as {"format":"{/src.code}","datatype":"integer"}`}, {"text beside a typed column it reads", `["{/src.score}","n/a"]`, `to read that column as text, write {"format":"{text}","text":"{/src.score}"}`}, @@ -54,7 +53,6 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { {"a datatype a typed column's values reject", `{"format":"{/src.score}","datatype":"boolean"}`, "{int(1,9)} prints an integer, not a boolean"}, {"two typed columns it reads", `["{/src.score}","{/src.flag}"]`, `so to read "{/src.score}" as text, write {"format":"{text}","text":"{/src.score}"}`}, {"a typed column read that holds the datatype declared beside it", `[{"format":"1.5","datatype":"number"},"{/src.score}"]`, `so write "{/src.score}" as {"format":"{/src.score}","datatype":"number"}`}, - {"two datatypes the first holds both of", `[{"format":"1","datatype":"integer"},{"format":"1.5","datatype":"number"}]`, "its items hold integer and number; a column holds one datatype"}, {"a typed column read holding the datatype of another", `["{/src.ratio}","{/src.score}"]`, `so write "{/src.ratio}" as {"format":"{/src.ratio}","datatype":"integer"}`}, {"text before a typed column it reads", `["n/a","{/src.score}"]`, `to read that column as text, write {"format":"{text}","text":"{/src.score}"}`}, {"a weighted item beside a typed column it reads", `["{/src.score}",{"format":"5","weight":2}]`, `give it "datatype": "integer"`}, @@ -108,6 +106,18 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) { t.Errorf("%s: NewTemplate = %v, want the inline template refused the same way", c.name, err) } } + for column, want := range map[string]string{ + `[{"format":"1","datatype":"integer"},{"format":"true","datatype":"boolean"}]`: "its items hold integer and boolean; a column holds one datatype", + `[{"format":"1","datatype":"integer"},{"format":"1.5","datatype":"number"}]`: "its items hold integer and number; a column holds one datatype", + } { + row := `{"format":"","col":` + column + `}` + if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"row": row}))); err == nil || !strings.HasSuffix(err.Error(), want) { + t.Errorf("items of two declared datatypes %s: New = %v, want an error ending %q", column, err, want) + } + if _, err := newGenerator(t, writeData(t, tree)).NewTemplate(row); err == nil || !strings.HasSuffix(err.Error(), want) { + t.Errorf("items of two declared datatypes %s: NewTemplate = %v, want an error ending %q", column, err, want) + } + } _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ "a": `{"format":"","c":["{/b.x}",{"format":"1","datatype":"integer"}]}`, "b": `{"format":"","x":["2",{"format":"1","datatype":"integer"}]}`, -- 2.52.0 From 77f79de44322128cdb5101278c25e535084df31f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:10:10 +0200 Subject: [PATCH 224/966] Trace a render's reads through the hold set the test makes, dropping session.trace and repeatDepth --- drawfence_test.go | 25 ++++++++++++------------- fejkdata.go | 1 - hold.go | 13 +++++-------- holdset.go | 8 +++----- todo.md | 2 -- 5 files changed, 20 insertions(+), 29 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index c426bc4..03dca96 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -62,7 +62,7 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { f := fenceCorpus(t) roots, tables := fenceRoots(t, f) for _, root := range roots { - wantGathered(t, f.rand, tables, root.label, renderDraws(root.t).reads, func() { renderRoot(f.rand, root.t) }) + wantGathered(t, tables, root.label, renderDraws(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) if !root.t.isRecord || len(root.t.fields) == 0 { continue } @@ -70,11 +70,11 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { if err != nil { t.Fatalf("%s: recordOf = %v", root.label, err) } - wantGathered(t, f.rand, tables, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads, func() { renderRecordRoot(f.rand, root.t, columns) }) + wantGathered(t, tables, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) } } -func wantGathered(t *testing.T, s *session, tables map[string]*table, label string, gathered []pathRead, renderOnce func()) { +func wantGathered(t *testing.T, tables map[string]*table, label string, gathered []pathRead, renderOnce func(renderTrace)) { t.Helper() var reads []pathRead type traceKey struct { @@ -82,8 +82,8 @@ func wantGathered(t *testing.T, s *session, tables map[string]*table, label stri row int } seen := map[traceKey]bool{} - s.trace = &renderTrace{read: func(group, table string, row int, a arm) { - if s.trace.repeatDepth != 0 || !isRef(a.key) { + trace := func(group, table string, row int, a arm) { + if !isRef(a.key) { return } r := pathRead{at: drawAt{group: group}, a: a} @@ -94,10 +94,9 @@ func wantGathered(t *testing.T, s *session, tables map[string]*table, label stri seen[k] = true reads = append(reads, r) } - }} - defer func() { s.trace = nil }() + } for i := 0; i < 5; i++ { - renderOnce() + renderOnce(trace) } for _, r := range reads { if !gathers(gathered, r) { @@ -124,14 +123,14 @@ func spellPins(p *pinSet) string { return "row " + strings.Join(rows, ", ") } -func renderRoot(s *session, t *template) { - var set holdSet +func renderRoot(s *session, t *template, trace renderTrace) { + set := holdSet{trace: trace} sc := renderScope{set: &set} switch { case t.table != nil: render(s, t.table, sc) for r := 0; r < t.table.rowCount(); r++ { - var set holdSet + set := holdSet{trace: trace} sc := renderScope{set: &set} sc.groupHold().pins.pin(t.table, r) render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.groupHold().pins)) @@ -143,8 +142,8 @@ func renderRoot(s *session, t *template) { } } -func renderRecordRoot(s *session, t *template, columns []Column) { - var set holdSet +func renderRecordRoot(s *session, t *template, columns []Column, trace renderTrace) { + set := holdSet{trace: trace} sc := renderScope{set: &set} if t.table != nil { t.table.drawIn(s, &sc.groupHold().pins) diff --git a/fejkdata.go b/fejkdata.go index 41672d7..c47e10e 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -45,7 +45,6 @@ type Generator struct { type session struct { *rand.Rand counters map[string]uint64 - trace *renderTrace } func (s *session) next(key string) uint64 { diff --git a/hold.go b/hold.go index 54b553b..0b30533 100644 --- a/hold.go +++ b/hold.go @@ -34,8 +34,8 @@ type readValue struct { // render's hold for its group, so its draw spans the render; every other held name in // held, the expansion's. func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { - if s.trace != nil { - traceRead(s.trace, t, sc, a) + if sc.set.trace != nil { + traceRead(sc.set.trace, t, sc, a) } if !t.compiled.held[a.key] { if len(a.tail) > 0 { @@ -64,12 +64,9 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV } // renderTrace is a test's view of the fields a render reads; nil outside a test. -type renderTrace struct { - read func(group, table string, row int, a arm) - repeatDepth int -} +type renderTrace func(group, table string, row int, a arm) -func traceRead(trace *renderTrace, t *template, sc renderScope, a arm) { +func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { var table string var row int switch { @@ -78,7 +75,7 @@ func traceRead(trace *renderTrace, t *template, sc renderScope, a arm) { case t.table != nil: table, row = t.table.path, sc.rowOf(t.table) } - trace.read(strings.Clone(sc.group), strings.Clone(table), row, a) + trace(strings.Clone(sc.group), strings.Clone(table), row, a) } // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one diff --git a/holdset.go b/holdset.go index 93be3fa..810b8d2 100644 --- a/holdset.go +++ b/holdset.go @@ -5,10 +5,12 @@ import ( "strings" ) -// holdSet is one render's reference draws: the unnamed draw group's, and each named one's. +// holdSet is one render's reference draws: the unnamed draw group's, and each named one's; and a +// test's trace of its reads. type holdSet struct { unnamed groupHold named map[string]*groupHold + trace renderTrace } // renderScope is where a render reads its reference paths: a hold set, in the draw group of the @@ -33,10 +35,6 @@ func renderOnce(s *session, n node) string { //go:noinline func expandAnew(s *session, t *template) string { var set holdSet - if s.trace != nil { - s.trace.repeatDepth++ - defer func() { s.trace.repeatDepth-- }() - } return expand(s, t, renderScope{set: &set}) } diff --git a/todo.md b/todo.md index d5139ba..f0027c2 100644 --- a/todo.md +++ b/todo.md @@ -23,8 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Move `session.trace` and `renderTrace.repeatDepth` off the render path: they serve - `drawfence_test.go` alone. - Give `Fake` a local hold set like `FakeRecord`, `RecordTemplate.Fake` and `FakeStruct`, or state at `Generator.set` why it alone needs one: four seats read the field's heap claim as contradicted by the other three. -- 2.52.0 From 5a87dc04d31e76c46d3be57a4a440dfff0699eb6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:15:22 +0200 Subject: [PATCH 225/966] Pin readField's unheld-path panic past the trace check, and fail the gathered-reads test when nothing traced --- drawfence_test.go | 13 +++++++++---- hold_test.go | 2 +- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index 03dca96..7bfb00c 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -61,8 +61,9 @@ func fenceRoots(t *testing.T, f *Generator) ([]fenceRoot, map[string]*table) { func TestEveryReadARenderMakesIsGathered(t *testing.T) { f := fenceCorpus(t) roots, tables := fenceRoots(t, f) + traced := 0 for _, root := range roots { - wantGathered(t, tables, root.label, renderDraws(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) + traced += wantGathered(t, tables, root.label, renderDraws(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) if !root.t.isRecord || len(root.t.fields) == 0 { continue } @@ -70,11 +71,14 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { if err != nil { t.Fatalf("%s: recordOf = %v", root.label, err) } - wantGathered(t, tables, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) + traced += wantGathered(t, tables, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) + } + if traced == 0 { + t.Error("no render traced a reference read, so the comparison proved nothing") } } -func wantGathered(t *testing.T, tables map[string]*table, label string, gathered []pathRead, renderOnce func(renderTrace)) { +func wantGathered(t *testing.T, tables map[string]*table, label string, gathered []pathRead, renderWith func(renderTrace)) int { t.Helper() var reads []pathRead type traceKey struct { @@ -96,13 +100,14 @@ func wantGathered(t *testing.T, tables map[string]*table, label string, gathered } } for i := 0; i < 5; i++ { - renderOnce(trace) + renderWith(trace) } for _, r := range reads { if !gathers(gathered, r) { t.Errorf("%s: the render read {%s} in draw group %q from %s, and drawWalk gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) } } + return len(reads) } func gathers(gathered []pathRead, r pathRead) bool { diff --git a/hold_test.go b/hold_test.go index f6ede64..a2d4830 100644 --- a/hold_test.go +++ b/hold_test.go @@ -766,6 +766,6 @@ func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) { t.Fatal("not a template") } mustPanic(t, "unheld arm with a path", func() { - readField(engine(1).rand, tm, nil, renderScope{}, arm{spelling: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) + readField(engine(1).rand, tm, nil, renderScope{set: &holdSet{}}, arm{spelling: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) }) } -- 2.52.0 From 3af4526ef6767c72d77f094e2c1d94bcdbef2e0d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:30:42 +0200 Subject: [PATCH 226/966] Give Fake a local hold set, as the other renders have, and drop the done round item --- fejkdata.go | 3 +-- render.go | 4 ++-- todo.md | 3 --- 3 files changed, 3 insertions(+), 7 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index c47e10e..7999c23 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -34,8 +34,7 @@ type Generator struct { mu sync.Mutex rand *session categories map[string]node - root folder // the categories as the node a path walks from, owned here so a walk allocates none - set holdSet // one Fake's hold set, owned here so a walk pinning rows keeps them off the heap + root folder // the categories as the node a path walks from, owned here so a walk allocates none records map[node]recordShape structs map[reflect.Type]structResult } diff --git a/render.go b/render.go index b3b776c..b6490d6 100644 --- a/render.go +++ b/render.go @@ -23,8 +23,8 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } - f.set = holdSet{} - sc := renderScope{set: &f.set} + var set holdSet + sc := renderScope{set: &set} f.root.children = f.categories n, err := descend(f.rand, &f.root, segments, sc) if err != nil { diff --git a/todo.md b/todo.md index f0027c2..c13eacd 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Give `Fake` a local hold set like `FakeRecord`, `RecordTemplate.Fake` and - `FakeStruct`, or state at `Generator.set` why it alone needs one: four seats read the - field's heap claim as contradicted by the other three. - Spell every panic an earlier check makes unreachable with one greppable phrase: the 17 say "reached prep unvalidated", "should have been proved", "with no row pinned" and more, so a user's report greps to nothing. -- 2.52.0 From 3aa747de02ac011f7960b27393c9610f948adbbd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:36:53 +0200 Subject: [PATCH 227/966] Require every panic to report through internalError, one greppable phrase --- builtins_test.go | 5 +++-- naming_test.go | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/builtins_test.go b/builtins_test.go index ee72939..6e2aa60 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -2,6 +2,7 @@ package fejkdata import ( "encoding/base64" + "fmt" "regexp" "strconv" "strings" @@ -217,8 +218,8 @@ func TestArgGuardsPanic(t *testing.T) { func mustPanic(t *testing.T, name string, call func()) { t.Helper() defer func() { - if recover() == nil { - t.Errorf("%s: no panic, want the invariant reported", name) + if r := recover(); r == nil || !strings.HasPrefix(fmt.Sprint(r), "fejkdata: internal error: ") { + t.Errorf("%s: recovered %v, want the invariant reported as an internal error", name, r) } }() call() diff --git a/naming_test.go b/naming_test.go index f73e58b..b7c3e38 100644 --- a/naming_test.go +++ b/naming_test.go @@ -24,6 +24,27 @@ func TestNoFunctionSpellsAMethod(t *testing.T) { } } +func TestEveryPanicIsAnInternalError(t *testing.T) { + fset, files := sourceFiles(t) + for _, f := range files { + ast.Inspect(f, func(n ast.Node) bool { + call, isCall := n.(*ast.CallExpr) + if !isCall || !isIdent(call.Fun, "panic") { + return true + } + if arg, isArgCall := call.Args[0].(*ast.CallExpr); !isArgCall || !isIdent(arg.Fun, "internalError") { + t.Errorf("%s: panic with internalError(…), so every invariant break greps to one phrase", fset.Position(call.Pos())) + } + return true + }) + } +} + +func isIdent(e ast.Expr, name string) bool { + id, isID := e.(*ast.Ident) + return isID && id.Name == name +} + func namespaceFiles(t *testing.T) []*ast.File { t.Helper() pkg := packageDir(t) -- 2.52.0 From 514d738b0879d1acb1af4a260e64fab510cd5e6d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:38:45 +0200 Subject: [PATCH 228/966] Report every invariant break through internalError --- builtins.go | 4 ++-- calc.go | 6 +++--- fejkdata.go | 4 ++++ graph.go | 2 +- hold.go | 7 ++----- holdset.go | 7 ++----- layout.go | 6 +++--- path.go | 2 +- pins.go | 2 +- render.go | 2 +- struct.go | 2 +- transform.go | 4 ++-- value.go | 4 ++-- 13 files changed, 25 insertions(+), 27 deletions(-) diff --git a/builtins.go b/builtins.go index dd8e72d..31c3916 100644 --- a/builtins.go +++ b/builtins.go @@ -115,7 +115,7 @@ func formatFloat(v float64, dp int) string { func atoi(s string) int { n, err := strconv.Atoi(s) if err != nil { - panic(fmt.Sprintf("fejkdata: builtin arg %q reached prep unvalidated: %v", s, err)) + panic(internalError("builtin arg %q reached prep unvalidated: %v", s, err)) } return n } @@ -124,7 +124,7 @@ func atoi(s string) int { func atof(s string) float64 { f, err := strconv.ParseFloat(s, 64) if err != nil { - panic(fmt.Sprintf("fejkdata: builtin arg %q reached prep unvalidated: %v", s, err)) + panic(internalError("builtin arg %q reached prep unvalidated: %v", s, err)) } return f } diff --git a/calc.go b/calc.go index 60801fb..827fde1 100644 --- a/calc.go +++ b/calc.go @@ -38,7 +38,7 @@ func (n calcIdx) eval(operands []string) float64 { // than returning NaN, so a node kind indexVars forgets is a stack trace and not a // silently wrong number. func (n calcVar) eval([]string) float64 { - panic(fmt.Sprintf("fejkdata: calc operand %q was never placed", string(n))) + panic(internalError("calc operand %q was never placed", string(n))) } func (n calcNeg) eval(operands []string) float64 { return -n.x.eval(operands) } @@ -191,7 +191,7 @@ func neverNumeric(n node) (text string, never bool) { func calcPrep(args []string) callFn { expr, err := parseCalc(args[0]) if err != nil { // a nil AST would be a nil dereference per render, with no message - panic(fmt.Sprintf("fejkdata: calc(%q) reached prep unparsed: %v", args[0], err)) + panic(internalError("calc(%q) reached prep unparsed: %v", args[0], err)) } at := make(map[string]int) for i, name := range calcVars(expr) { @@ -219,7 +219,7 @@ func indexVars(n calcNode, at map[string]int) calcNode { case calcVar: i, placed := at[string(n)] if !placed { // calcVars named every operand, so a miss means the two disagree - panic(fmt.Sprintf("fejkdata: calc operand %q is not among the names read for it", string(n))) + panic(internalError("calc operand %q is not among the names read for it", string(n))) } return calcIdx(i) case calcNeg: diff --git a/fejkdata.go b/fejkdata.go index 7999c23..25c0666 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -240,3 +240,7 @@ func newRand(seed uint64, seeded bool) (*session, error) { } return &session{Rand: r, counters: map[string]uint64{}}, nil } + +func internalError(format string, a ...any) string { + return "fejkdata: internal error: " + fmt.Sprintf(format, a...) +} diff --git a/graph.go b/graph.go index 10d7e65..7d30b9e 100644 --- a/graph.go +++ b/graph.go @@ -162,7 +162,7 @@ func renderEdges(n node) []renderEdge { case *folder, *nullItem: return nil default: - panic(fmt.Sprintf("fejkdata: renderEdges has no case for node %T", n)) + panic(internalError("renderEdges has no case for node %T", n)) } } diff --git a/hold.go b/hold.go index 0b30533..590ee5b 100644 --- a/hold.go +++ b/hold.go @@ -1,9 +1,6 @@ package fejkdata -import ( - "fmt" - "strings" -) +import "strings" // hold is what one expansion has drawn for the names it holds: the variant each was drawn as, so // every path under it reads one variant; and the value each read produced, by its path, so the @@ -39,7 +36,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV } if !t.compiled.held[a.key] { if len(a.tail) > 0 { - panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) + panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) } return readValue{text: render(s, t.head(a.key), sc)} } diff --git a/holdset.go b/holdset.go index 810b8d2..d0a2e0c 100644 --- a/holdset.go +++ b/holdset.go @@ -1,9 +1,6 @@ package fejkdata -import ( - "fmt" - "strings" -) +import "strings" // holdSet is one render's reference draws: the unnamed draw group's, and each named one's; and a // test's trace of its reads. @@ -61,7 +58,7 @@ func (sc renderScope) at(n node, pins *pinSet) renderScope { // rowOf is the row of t its columns render from. func (sc renderScope) rowOf(t *table) int { if sc.row.t != t { - panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) + panic(internalError("a column of %s is rendered with no row pinned", t.category)) } return sc.row.row } diff --git a/layout.go b/layout.go index 69f26c1..af98d7e 100644 --- a/layout.go +++ b/layout.go @@ -54,7 +54,7 @@ func layoutArg(a string) (string, error) { func layoutOf(a string) string { layout, err := layoutArg(a) if err != nil { - panic(fmt.Sprintf("fejkdata: builtin arg %q reached prep unvalidated: %v", a, err)) + panic(internalError("builtin arg %q reached prep unvalidated: %v", a, err)) } return layout } @@ -128,11 +128,11 @@ func timeArg(_ map[string]node, a []string) error { func datePrep(a []string) callFn { from, err := time.Parse(dayLayout, a[0]) if err != nil { - panic(fmt.Sprintf("fejkdata: builtin arg %q reached prep unvalidated: %v", a[0], err)) + panic(internalError("builtin arg %q reached prep unvalidated: %v", a[0], err)) } to, err := time.Parse(dayLayout, a[1]) if err != nil { - panic(fmt.Sprintf("fejkdata: builtin arg %q reached prep unvalidated: %v", a[1], err)) + panic(internalError("builtin arg %q reached prep unvalidated: %v", a[1], err)) } layout, start, span := layoutOf(a[2]), from.Unix(), int(to.Unix()-from.Unix())+86400 return func(s *session, _ string, _ []string) string { diff --git a/path.go b/path.go index 3fd8b5b..771248c 100644 --- a/path.go +++ b/path.go @@ -353,7 +353,7 @@ type pathDraw struct { func drawPath(n node, tail []string, level string, w *pathDraw) node { leaf, err := w.walk(n, tail) if err != nil { - panic(fmt.Sprintf("fejkdata: %s: %v; the path should have been proved before it was drawn", join(level, joinSegments(tail)), err)) + panic(internalError("%s: %v; the path should have been proved before it was drawn", join(level, joinSegments(tail)), err)) } return leaf } diff --git a/pins.go b/pins.go index f16cee0..5866e3b 100644 --- a/pins.go +++ b/pins.go @@ -32,7 +32,7 @@ func (p *pinSet) pinned(t *table) (int, bool) { func (p *pinSet) mustRow(t *table) int { r, ok := p.pinned(t) if !ok { - panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category)) + panic(internalError("a column of %s is rendered with no row pinned", t.category)) } return r } diff --git a/render.go b/render.go index b6490d6..739ce15 100644 --- a/render.go +++ b/render.go @@ -85,7 +85,7 @@ func render(s *session, n node, sc renderScope) string { } return b.String() default: - panic(fmt.Sprintf("fejkdata: uncompiled node %T", n)) + panic(internalError("uncompiled node %T", n)) } } diff --git a/struct.go b/struct.go index 67f5643..a6f3df1 100644 --- a/struct.go +++ b/struct.go @@ -422,6 +422,6 @@ func setText(field reflect.Value, text string) { field.SetUint(n) } if err != nil { - panic(fmt.Sprintf("fejkdata: %q reached a %s field unproven: %v", text, field.Type(), err)) + panic(internalError("%q reached a %s field unproven: %v", text, field.Type(), err)) } } diff --git a/transform.go b/transform.go index d5a3ae6..e5f965e 100644 --- a/transform.go +++ b/transform.go @@ -17,7 +17,7 @@ var transforms = map[string]func(string) string{ func withTransforms(calls map[string]builtin) map[string]builtin { for name, fn := range transforms { if _, clash := calls[name]; clash { - panic("fejkdata: " + name + " is registered as a builtin and a transform") + panic(internalError("%s is registered as a builtin and a transform", name)) } calls[name] = builtin{arity: 1, checkArgs: transformArg, prep: transformPrep(fn), operands: transformOperand} } @@ -68,7 +68,7 @@ func transformPrep(outer func(string) string) func([]string) callFn { return func(a []string) callFn { _, chain, err := unwrapTransform(a[0]) if err != nil { - panic(fmt.Sprintf("fejkdata: transform arg %q reached prep unvalidated: %v", a[0], err)) + panic(internalError("transform arg %q reached prep unvalidated: %v", a[0], err)) } return func(_ *session, _ string, operands []string) string { v := operands[0] diff --git a/value.go b/value.go index 3a764d8..0c73593 100644 --- a/value.go +++ b/value.go @@ -176,7 +176,7 @@ func (p *valueProof) proveTemplate(t *template) proven { func (p *valueProof) proveCalc(t *template, body string, args []string) proven { expr, err := parseCalc(args[0]) if err != nil { - panic(fmt.Sprintf("fejkdata: calc(%q) reached a proof unparsed: %v", args[0], err)) + panic(internalError("calc(%q) reached a proof unparsed: %v", args[0], err)) } v, doubt := p.proveExpr(expr, t.fields) if doubt == "" && !(magnitude(v) <= calcLimit) { @@ -219,7 +219,7 @@ func (p *valueProof) proveExpr(n calcNode, fields map[string]node) (proven, stri } return combine(n, l, r) } - panic(fmt.Sprintf("fejkdata: calc node %T has no bound", n)) + panic(internalError("calc node %T has no bound", n)) } // combine bounds one operation from the bounds of its sides. -- 2.52.0 From 125fb1e9aac11c3e62b8c9331a15c8b6d060283e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:38:45 +0200 Subject: [PATCH 229/966] Drop the done panic-phrase round item --- todo.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/todo.md b/todo.md index c13eacd..a0a7d51 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Spell every panic an earlier check makes unreachable with one greppable phrase: the - 17 say "reached prep unvalidated", "should have been proved", "with no row pinned" - and more, so a user's report greps to nothing. - Correct the false comments: `List` claims folder segments, which `paths` never lists, and `renderEdge.reached` claims `pathKeyReaders`' vocabulary, which spells a token `token {x}`. Drop `rng`'s "for the reason below", which points at `pick`. -- 2.52.0 From e88c7e0170cb8df8b8bfe1f59dfd4950cb0a4742 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:41:24 +0200 Subject: [PATCH 230/966] Say mustPanic wants internalError's phrase --- builtins_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/builtins_test.go b/builtins_test.go index 6e2aa60..d093483 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -213,8 +213,8 @@ func TestArgGuardsPanic(t *testing.T) { } } -// mustPanic fails unless call panics, which is what separates a reported -// invariant break from a silently wrong value. +// mustPanic fails unless call panics with internalError's phrase, which is what +// separates a reported invariant break from a silently wrong value. func mustPanic(t *testing.T, name string, call func()) { t.Helper() defer func() { -- 2.52.0 From 4c5466cad0c51eaafee0833f8de558b5a47ebf8e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:41:24 +0200 Subject: [PATCH 231/966] Spell rowOf's panic apart from mustRow's --- holdset.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/holdset.go b/holdset.go index d0a2e0c..37080ff 100644 --- a/holdset.go +++ b/holdset.go @@ -58,7 +58,7 @@ func (sc renderScope) at(n node, pins *pinSet) renderScope { // rowOf is the row of t its columns render from. func (sc renderScope) rowOf(t *table) int { if sc.row.t != t { - panic(internalError("a column of %s is rendered with no row pinned", t.category)) + panic(internalError("a column of %s renders in a scope holding no row of it", t.category)) } return sc.row.row } -- 2.52.0 From a9f35a1ab346bb30836d622af32963f4cb7ea220 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:53:20 +0200 Subject: [PATCH 232/966] Correct List's, reached's and rng's false comments --- fejkdata.go | 5 ++--- graph.go | 3 +-- render.go | 2 +- todo.md | 3 --- 4 files changed, 4 insertions(+), 9 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index 25c0666..7d00957 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -116,9 +116,8 @@ func New(opts ...Option) (*Generator, error) { } // List returns the sorted dotted paths Fake can render: every category, the dotted -// fields within a template, and folder segments. A choice consumes no segment, so a -// path continues through one only where every variant carries it, which is the -// rule Fake applies too. +// fields within a template. A choice consumes no segment, so a path continues +// through one only where every variant carries it, which is the rule Fake applies too. func (f *Generator) List() []string { var out []string for _, name := range sortedNames(f.categories) { diff --git a/graph.go b/graph.go index 7d30b9e..20e5b74 100644 --- a/graph.go +++ b/graph.go @@ -109,8 +109,7 @@ type renderEdge struct { func (e renderEdge) readsRef() bool { return isRef(e.read.key) } -// reached names an edge as the author spelled it, the vocabulary pathKeyReaders uses -// for the sibling fence. +// reached names an edge as the author spelled it. func (e renderEdge) reached() string { if e.operand != "" { return fmt.Sprintf("%s operand %q", e.operand, e.label) diff --git a/render.go b/render.go index 739ce15..3b864ff 100644 --- a/render.go +++ b/render.go @@ -6,7 +6,7 @@ import ( ) // rng is the randomness a builtin sample draws from; *rand.Rand satisfies it. The -// render path takes the concrete *session instead, for the reason below. +// render path takes the concrete *session instead. type rng interface { IntN(n int) int Float64() float64 diff --git a/todo.md b/todo.md index a0a7d51..b73c349 100644 --- a/todo.md +++ b/todo.md @@ -23,9 +23,6 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming This round, in order: -- Correct the false comments: `List` claims folder segments, which `paths` never lists, - and `renderEdge.reached` claims `pathKeyReaders`' vocabulary, which spells a token - `token {x}`. Drop `rng`'s "for the reason below", which points at `pick`. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 7b01a7fd6850e5540e2cda6708a191b5ce30c687 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 01:55:00 +0200 Subject: [PATCH 233/966] Say what List returns: fields, columns, linked tables, direct descents only --- README.md | 2 +- fejkdata.go | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 59d6b6f..3a6df86 100644 --- a/README.md +++ b/README.md @@ -273,7 +273,7 @@ if err != nil { log.Fatal(err) } v, err := f.Fake("sv_SE.address") // "Järvedsvägen 43\n891 77 Järved" -paths := f.List() // every path Fake accepts, sorted +paths := f.List() // the paths Fake renders, sorted; direct descents only v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once v = t.Fake() // render many times, no re-parse diff --git a/fejkdata.go b/fejkdata.go index 7d00957..73d54ab 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -115,9 +115,9 @@ func New(opts ...Option) (*Generator, error) { return &Generator{rand: rng, categories: cats}, nil } -// List returns the sorted dotted paths Fake can render: every category, the dotted -// fields within a template. A choice consumes no segment, so a path continues -// through one only where every variant carries it, which is the rule Fake applies too. +// List returns the sorted dotted paths Fake renders: each category and every field, +// column and linked table below it, a direct descent at a time. A choice consumes no +// segment, so a path continues through one only where every variant carries it. func (f *Generator) List() []string { var out []string for _, name := range sortedNames(f.categories) { -- 2.52.0 From 501ea274ad602d8b63d651c802052d6e768b937e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:19:00 +0200 Subject: [PATCH 234/966] File the 2026-09-29 nine-seat panel's findings, scored 5.8 --- todo.md | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index b73c349..dc7f8de 100644 --- a/todo.md +++ b/todo.md @@ -19,10 +19,40 @@ Self-sufficiency 5.6); 5.7 on 2026-09-26 (Navigation 6.7, Locality 5.0, Shape 5. Self-sufficiency 5.6), every seat capped by Locality, and all nine naming `drawWalk.walk` among the hardest; 5.7 on 2026-09-28 (Navigation 6.7, Locality 5.1, Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming -`drawWalk.walk` and the four path walks among the hardest. +`drawWalk.walk` and the four path walks among the hardest; 5.8 on 2026-09-29 +(Navigation 6.8, Locality 5.3, Shape 5.8, Self-sufficiency 5.7), seven seats capped +by Locality, all nine naming `drawWalk.walk` hardest and eight the unit they would +least modify. This round, in order: +- Make every switch over node kinds panic through `internalError` on a kind it does not + list, as `render` and `renderEdges` do, naming the kinds it skips: `contained` and + `paths` return nil. Correct `render`'s "a new node kind goes in both", which misses + the rest. +- Rename the load-time `draw*` types (`drawWalk`, `drawAt`, `drawRoute`, `drawCheck`) + to say they read and draw nothing, and move `drawWalk` and `tableRead` out of + `drawfence.go`: a grep for "draw" lands in render and check code alike. +- Spell `drawWalk.walk`'s dropped cell as a case of its switch: a clashing row's cell + falls out with no arm, which four seats named. +- Define `descended` where `table.route` takes it: five seats reverse-engineered it from + the `draw` expression and the four walks' loop condition. +- Keep a template's table in one field: `template.table` and `template.cell.table` both + answer which table it belongs to, and `checkOwnFamily` and `traceRead` branch on both. +- Stop `readField` dereferencing a nil `held`: `renderLeaf` passes nil, safe only + because a `readsColumn` arm is a reference path with a tail, which nothing there says. +- Test that `disagreements` holds every key `disagreement` can build, and panic through + `internalError` on a miss: a missing key is a nil func call today. +- Name `printedNumber`'s `dp` cases, and say on `builtin` what `prints` and + `proveNumber` a new builtin sets: the extender copied `digits` by analogy. +- Rename the names that mislead: `table.pinnedRow` is the row node, not a pinned row; + `table.category` is a bare name beside the dotted `category` of `resolveLink`; + `checkRenders` renders nothing; `fieldAlternation` is a plain `{name}` too; and + `referenceFence` is the cycle check for a tree and a drawGroup check inline. +- Correct the false comments: `node` says rendering never re-sums weights, which + `builtLookup` does on first draw; `Generator` claims immutable after `New`, while + `Fake` sets `root.children` each call, which belongs in `New`; `renders` credits + `checkNoCycles`, which an inline binding never runs. - Run the nine-seat panel again and file what it names here. ### Data @@ -30,6 +60,10 @@ This round, in order: Shape: T = table, t = template, c = choice. Read every factual list from a register by a `data-import/` script, as README goal 12 asks. +- Fetch NVDB through `source.fetch` in `data-import/geo-se.py`: `nvdb_segments` calls + urllib itself, with no retry and no User-Agent. +- Test that every `geo` tree holds the five table names and the `address` columns the + README says port across countries: nothing checks it, and a new country breaks it silently. - Strip the whitespace `data-import/territory.py` copies from its register: `misc.territory[CW].capital` renders ` Willemstad`, with a leading space. - Place xlsx cells by their `r` reference in `data-import/xlsx.py`: a sheet omitting an @@ -148,6 +182,16 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Compare `calcParser.binary`'s operator as a rune: `byte(p.rs[p.pos])` reads U+012B + `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. +- Stop the `columnKinds` suggestion naming a field's own kind or a narrower one: an + `int64` or `float64` field unproven in range is told to become itself, and a `uint64` + one to become `int64`. +- Say in the README's Linked tables that the first table a render reads decides whose + weights govern the family: `geo.SE.address` reads `street` first, so it draws by + street segments and the region, kommun and postort populations never apply. +- Parse `defaultTable`'s path with the library's grammar: the CLI counts brackets of its + own, so a change to the selector grammar desyncs the default `--table` silently. - Step up from a row to its parent as the decision "A link column after a row steps up to the parent row, and a path may not end on it" states, refusing a step back down: `city[Oslo].country.name` is a load error today, and `city[Oslo].country` renders the -- 2.52.0 From a82e937412347659567708a8c942fc298964fe1c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:31:10 +0200 Subject: [PATCH 235/966] Test that contained and paths panic on a node kind they do not list --- render_test.go | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/render_test.go b/render_test.go index 478f1a0..402d6bb 100644 --- a/render_test.go +++ b/render_test.go @@ -61,11 +61,20 @@ type unhandledNode struct{} func (*unhandledNode) isNode() {} -func TestRenderEdgesPanicsOnAnUnhandledNode(t *testing.T) { - defer func() { - if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), "*fejkdata.unhandledNode") { - t.Errorf("renderEdges(unhandledNode) recovered %v, want a panic naming the node kind", r) - } - }() - renderEdges(&unhandledNode{}) +func TestNodeSwitchesPanicOnAnUnhandledNode(t *testing.T) { + for name, call := range map[string]func(){ + "contained": func() { contained(&unhandledNode{}) }, + "paths": func() { paths(&unhandledNode{}) }, + "render": func() { render(nil, &unhandledNode{}, renderScope{}) }, + "renderEdges": func() { renderEdges(&unhandledNode{}) }, + } { + t.Run(name, func(t *testing.T) { + defer func() { + if r := recover(); r == nil || !strings.HasPrefix(fmt.Sprint(r), "fejkdata: internal error: ") || !strings.Contains(fmt.Sprint(r), "*fejkdata.unhandledNode") { + t.Errorf("recovered %v, want an internal error naming the node kind", r) + } + }() + call() + }) + } } -- 2.52.0 From 38837b86a830c7c0dd9ccc4b262b38c346a69587 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:31:52 +0200 Subject: [PATCH 236/966] Panic through internalError on a node kind contained or paths does not list --- fejkdata.go | 3 ++- graph.go | 4 +++- render.go | 3 ++- todo.md | 4 ---- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index 73d54ab..21a75cc 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -161,8 +161,9 @@ func paths(n node) []string { out = append(out, p) } return out + default: + panic(internalError("paths has no case for node %T", n)) } - return nil } // tablePaths is a table's columns, then each table linked to it under its name: the diff --git a/graph.go b/graph.go index 20e5b74..fee1b54 100644 --- a/graph.go +++ b/graph.go @@ -73,8 +73,10 @@ func contained(n node) []namedNode { } } return out - default: + case *nullItem, *tableRow: return nil + default: + panic(internalError("contained has no case for node %T", n)) } } diff --git a/render.go b/render.go index 3b864ff..90e9827 100644 --- a/render.go +++ b/render.go @@ -49,7 +49,8 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over a hold set of its own. -// renderEdges mirrors this switch for the fences, so a new node kind goes in both. +// renderEdges mirrors this switch for the fences. A new node kind goes in every switch +// that panics through internalError on a kind it does not list. func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: diff --git a/todo.md b/todo.md index dc7f8de..ddab1d6 100644 --- a/todo.md +++ b/todo.md @@ -26,10 +26,6 @@ least modify. This round, in order: -- Make every switch over node kinds panic through `internalError` on a kind it does not - list, as `render` and `renderEdges` do, naming the kinds it skips: `contained` and - `paths` return nil. Correct `render`'s "a new node kind goes in both", which misses - the rest. - Rename the load-time `draw*` types (`drawWalk`, `drawAt`, `drawRoute`, `drawCheck`) to say they read and draw nothing, and move `drawWalk` and `tableRead` out of `drawfence.go`: a grep for "draw" lands in render and check code alike. -- 2.52.0 From f523283463eff8f973d57809859e7d4705badfd9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:35:02 +0200 Subject: [PATCH 237/966] Assert the node-kind panics through mustPanic --- render_test.go | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/render_test.go b/render_test.go index 402d6bb..4aa67ae 100644 --- a/render_test.go +++ b/render_test.go @@ -2,8 +2,6 @@ package fejkdata import ( "encoding/json" - "fmt" - "strings" "testing" ) @@ -68,13 +66,6 @@ func TestNodeSwitchesPanicOnAnUnhandledNode(t *testing.T) { "render": func() { render(nil, &unhandledNode{}, renderScope{}) }, "renderEdges": func() { renderEdges(&unhandledNode{}) }, } { - t.Run(name, func(t *testing.T) { - defer func() { - if r := recover(); r == nil || !strings.HasPrefix(fmt.Sprint(r), "fejkdata: internal error: ") || !strings.Contains(fmt.Sprint(r), "*fejkdata.unhandledNode") { - t.Errorf("recovered %v, want an internal error naming the node kind", r) - } - }() - call() - }) + mustPanic(t, name, call) } } -- 2.52.0 From b41ce7679f39a4f13670b42c3330913f4d73eb06 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:35:02 +0200 Subject: [PATCH 238/966] State the node-kind switch rule on node --- node.go | 3 ++- render.go | 3 +-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/node.go b/node.go index 105bb53..e488658 100644 --- a/node.go +++ b/node.go @@ -11,7 +11,8 @@ import ( // node is a compiled element of the namespace tree: a folder, choice, null, // template, table, column or row. Compiling JSON into these once (see compile) -// means rendering never re-inspects the raw JSON or re-sums weights. +// means rendering never re-inspects the raw JSON or re-sums weights. A new kind goes in +// every switch that panics through internalError on a kind it does not list. type node interface{ isNode() } // folder is a namespace of named children, built from a directory of JSON files diff --git a/render.go b/render.go index 90e9827..e8cf1c9 100644 --- a/render.go +++ b/render.go @@ -49,8 +49,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over a hold set of its own. -// renderEdges mirrors this switch for the fences. A new node kind goes in every switch -// that panics through internalError on a kind it does not list. +// renderEdges mirrors this switch for the fences. func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: -- 2.52.0 From 1cefa18039435fb4ed31505901efe8a72b829fce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:36:07 +0200 Subject: [PATCH 239/966] Name the four node-kind switches on node --- node.go | 4 ++-- render.go | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/node.go b/node.go index e488658..5f6f754 100644 --- a/node.go +++ b/node.go @@ -11,8 +11,8 @@ import ( // node is a compiled element of the namespace tree: a folder, choice, null, // template, table, column or row. Compiling JSON into these once (see compile) -// means rendering never re-inspects the raw JSON or re-sums weights. A new kind goes in -// every switch that panics through internalError on a kind it does not list. +// means rendering never re-inspects the raw JSON or re-sums weights. A new kind needs a +// case in contained, paths, render and renderEdges; each panics on a kind it does not list. type node interface{ isNode() } // folder is a namespace of named children, built from a directory of JSON files diff --git a/render.go b/render.go index e8cf1c9..0ac5970 100644 --- a/render.go +++ b/render.go @@ -49,7 +49,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over a hold set of its own. -// renderEdges mirrors this switch for the fences. +// A child this switch renders is one renderEdges must list too, or the fences miss it. func render(s *session, n node, sc renderScope) string { switch n := n.(type) { case *choice: -- 2.52.0 From 2d82f34cd4f7b71b845eb809f5c151ab9827b778 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:51:35 +0200 Subject: [PATCH 240/966] Rename the load-time draw types to the read survey and fence, and move the survey to readsurvey.go --- AGENTS.md | 2 +- README.md | 3 +- data.go | 2 +- doc.go | 4 +- docs/decisions.md | 2 +- drawfence.go | 261 +++------------------------------------------- drawfence_test.go | 16 +-- holdfence.go | 2 +- readsurvey.go | 241 ++++++++++++++++++++++++++++++++++++++++++ table_test.go | 8 +- todo.md | 7 +- 11 files changed, 276 insertions(+), 272 deletions(-) create mode 100644 readsurvey.go diff --git a/AGENTS.md b/AGENTS.md index 50d5998..8ce3b36 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,5 +110,5 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only -- Each entry point to `drawCheck.hasRead` says what its caller gets +- Each entry point to `readFence.hasRead` says what its caller gets - The four path walks are four loops diff --git a/README.md b/README.md index 3a6df86..b7edb18 100644 --- a/README.md +++ b/README.md @@ -886,7 +886,8 @@ format.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per held name, and reading a field through it holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through -drawfence.go the load fences over reference draws: a drawGroup's, a render's, a record's columns and a table family's, and the walk over what a render reads +drawfence.go the load fences over reference draws: a drawGroup's, a render's, a record's columns and a table family's +readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, for the draw fences pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/data.go b/data.go index a4cb217..62a9af4 100644 --- a/data.go +++ b/data.go @@ -246,7 +246,7 @@ func checkRenders(s nodeScope) error { if err := s(heldCheck); err != nil { return err } - fence := &drawCheck{} + fence := &readFence{} refs := false if err := s(func(path string, n node) error { if t, ok := n.(*template); ok && len(t.link.refs) > 0 { diff --git a/doc.go b/doc.go index a518fdf..79dfc95 100644 --- a/doc.go +++ b/doc.go @@ -34,7 +34,7 @@ package fejkdata // `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // - whole — a read of a table with no selector and no descent, {/city}, landing on // the `table` itself: it draws a row apart from every pin, so the fences keep the rows -// its cells stand in as `drawAt.wholePins`, beside `drawAt.wholeTable`. +// its cells stand in as `surveyAt.wholePins`, beside `surveyAt.wholeTable`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those -// compile. The draw fences are `heldCheck`, `drawCheck` and `checkFamilies`. +// compile. The draw fences are `heldCheck`, `readFence` and `checkFamilies`. diff --git a/docs/decisions.md b/docs/decisions.md index 8eedcfd..470ea3d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -668,7 +668,7 @@ import drops them by name rather than by a rank a regeneration would move. the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. -## Each entry point to `drawCheck.hasRead` says what its caller gets +## Each entry point to `readFence.hasRead` says what its caller gets 2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on `readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, diff --git a/drawfence.go b/drawfence.go index 86c5192..b1a30bd 100644 --- a/drawfence.go +++ b/drawfence.go @@ -36,11 +36,11 @@ func checkNestedDrawGroup(fields map[string]node, group string) error { return nil } -// drawCheck fences each template of a scope as a render of its own, remembering which nodes read a +// readFence fences each template of a scope as a render of its own, remembering which nodes read a // reference path. // docs/decisions.md#a-render-shares-one-reference-draw-per-category-per-group // docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences -type drawCheck struct { +type readFence struct { memo map[hasReadMemo]bool } @@ -51,14 +51,14 @@ type hasReadMemo struct { // checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference // path inside a repeat or a nested draw group, which draw apart from it whatever it names. -func (c *drawCheck) checkDrawGroup(path string, n node) error { +func (c *readFence) checkDrawGroup(path string, n node) error { if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) { return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) } return nil } -func (c *drawCheck) checkDraws(path string, n node) error { +func (c *readFence) checkDraws(path string, n node) error { t, ok := n.(*template) if !ok { return nil @@ -69,7 +69,7 @@ func (c *drawCheck) checkDraws(path string, n node) error { if !c.readsPath(t) { return nil } - if err := renderDraws(t).check(); err != nil { + if err := surveyRender(t).check(); err != nil { return fmt.Errorf("%s: %w", path, err) } return nil @@ -113,18 +113,10 @@ func checkOwnFamily(t *template) error { return nil } -func renderDraws(t *template) *drawWalk { - w := newDrawWalk() - for _, e := range renderEdges(t) { - w.edge(t, e, drawAt{group: t.link.drawGroupKey, route: drawRoute{e.reached(), e.label}}) - } - return w -} - // checkRecordDraws fences the columns of a record — a template compiled at the top without a // repeat — so a load proves the record view of it as well as the string view. // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load -func (c *drawCheck) checkRecordDraws(path string, n node) error { +func (c *readFence) checkRecordDraws(path string, n node) error { t, ok := n.(*template) if !ok || !t.isRecord { return nil @@ -146,15 +138,15 @@ func (c *drawCheck) checkRecordDraws(path string, n node) error { // readsPath reports whether rendering n reads a reference path, short of a repeat, which renders // over draws of its own. -func (c *drawCheck) readsPath(n node) bool { return c.hasRead(n, false) } +func (c *readFence) readsPath(n node) bool { return c.hasRead(n, false) } // splitsDraws reports whether rendering n reads a reference path that n's own draw group answers // for: one outside a repeat and outside a nested draw group, which hold their own draws. -func (c *drawCheck) splitsDraws(n node) bool { return c.hasRead(n, true) } +func (c *readFence) splitsDraws(n node) bool { return c.hasRead(n, true) } // hasRead walks what rendering n renders for a reference path, stopping at a repeat — and at a nested // draw group when stopAtGroup — since each holds draws of its own. -func (c *drawCheck) hasRead(n node, stopAtGroup bool) bool { +func (c *readFence) hasRead(n node, stopAtGroup bool) bool { k := hasReadMemo{n, stopAtGroup} if r, done := c.memo[k]; done { return r @@ -203,152 +195,12 @@ func readsTable(n node, a arm) bool { } // checkColumnDraws fences a record's columns as one render. -func checkColumnDraws(t *template, columns []string) error { return columnDraws(t, columns).check() } - -func columnDraws(t *template, columns []string) *drawWalk { - w := newDrawWalk() - for _, name := range columns { - w.walk(t.fields[name], drawAt{group: t.link.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) - } - return w -} - -// drawWalk gathers the references one render reads, by draw group, for check to compare. -type drawWalk struct { - reads []pathRead - read map[readKey]bool - seen map[nodeVisit]bool - pinIDs map[pinsLink]pinsID -} - -// drawAt is where a walk stands: the draw group it draws in, how the render's root reached it, the -// table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose -// cells the walk is in. -type drawAt struct { - group string - route drawRoute - pins pinSet - wholeTable *table - wholePins pinSet -} - -// drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. -type drawRoute struct{ spelling, label string } - -// pathRead is one reference a render reads: a path, or a bare reference with no tail. -type pathRead struct { - at drawAt - a arm - tr *tableRead // set where the reference names a table -} - -type nodeVisit struct { - n node - group string - wholeTable *table - pins pinsID - wholePins pinsID -} - -type readKey struct { - group string - path string - pins pinsID - wholePins pinsID -} - -// pinsID names a pin set within one walk: two sets pinning the same rows share one. -type pinsID int - -// pinsLink is a set's last pin, in path order, and the id of the set before it. -type pinsLink struct { - prev pinsID - pin tablePin -} - -func newDrawWalk() *drawWalk { - return &drawWalk{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}, pinIDs: map[pinsLink]pinsID{}} -} - -func (w *drawWalk) pinsID(p *pinSet) pinsID { - var pins []tablePin - p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) - sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) - id := pinsID(0) - for _, q := range pins { - l := pinsLink{id, q} - next, ok := w.pinIDs[l] - if !ok { - next = pinsID(len(w.pinIDs) + 1) - w.pinIDs[l] = next - } - id = next - } - return id -} - -// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops -// there. A cell whose row the pins keep out has no case: a pinned table renders its pinned row -// alone and an unpinned one draws a row inside its nearest pinned ancestor's, so that cell never -// renders on this route. -func (w *drawWalk) walk(n node, at drawAt) { - v := nodeVisit{n, at.group, at.wholeTable, w.pinsID(&at.pins), w.pinsID(&at.wholePins)} - if w.seen[v] { - return - } - w.seen[v] = true - if repeats(n) { - return - } - if t, isTemplate := n.(*template); isTemplate && t.link.drawGroupKey != "" { - at.group = t.link.drawGroupKey - } - if t, isTable := n.(*table); isTable { - at.wholeTable = t - } - for _, e := range renderEdges(n) { - to, isTemplate := e.to.(*template) - cell := cellSite{} - if isTemplate { - cell = to.cell - } - switch { - case cell.table == nil: - w.edge(n, e, at) - case cell.table == at.wholeTable: - in := at - in.wholePins = at.wholePins.clone() - in.wholePins.add(cell.table, cell.row) - w.edge(n, e, in) - case at.pins.clash(cell.table, cell.row) == nil: - in := at - in.pins = at.pins.entered(cell.table, cell.row) - w.edge(n, e, in) - } - } -} - -// edge records the reference an edge reads, then walks on with every row the read -// pins entered, so a selected row renders only its own cells. -func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { - if e.readsRef() { - a := e.read - tr := tableReadOf(from.(*template).head(a.key), a, e.to) - if k := (readKey{at.group, a.path, w.pinsID(&at.pins), w.pinsID(&at.wholePins)}); !w.read[k] { - w.read[k] = true - w.reads = append(w.reads, pathRead{at, a, tr}) - } - if tr != nil { - tr.pins.each(func(t *table, r int) { at.pins = at.pins.entered(t, r) }) - } - } - w.walk(e.to, at) -} +func checkColumnDraws(t *template, columns []string) error { return surveyColumns(t, columns).check() } // check refuses what one draw per reference path cannot answer for: a read of a level beside a path // another read takes into it, and two reads of one table family that select different rows. Reads // are compared in path order, so which pair is reported does not vary. -func (w *drawWalk) check() error { +func (w *readSurvey) check() error { sort.SliceStable(w.reads, func(i, j int) bool { if w.reads[i].at.group != w.reads[j].at.group { return w.reads[i].at.group < w.reads[j].at.group @@ -447,97 +299,10 @@ func conflict(r pathRead, err error) error { // alternatives reports whether two reads never render together: one read renders one row of a // table, so reads under different rows the walks pinned, or under different rows of one whole // draw, never meet. -func alternatives(a, b drawAt) bool { +func alternatives(a, b surveyAt) bool { return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) } -func overlapError(route drawRoute, ref string, into pathRead) error { +func overlapError(route readRoute, ref string, into pathRead) error { return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) } - -// spelled names the route, and the reference it reaches a draw by where its root edge is not that -// reference. -func (r drawRoute) spelled(ref string) string { - if ref == "" || ref == r.label { - return r.spelling - } - return fmt.Sprintf("%s with {%s}", r.spelling, ref) -} - -// tableRead is what a reference path reads of a table family: the table its head -// names, the rows its selectors pin, the tables it draws — those it walks with no -// row pinned, and their unpinned ancestors — each selector's spelling, and whether -// it lands on a `tableRow`. -type tableRead struct { - headTable *table - pins pinSet - drawn map[*table]bool - sels []tableSel - landsRow bool -} - -// tableSel is one selector on the way: the table it selects a row of, and the path -// as written up to and including it. -type tableSel struct { - t *table - spelling string -} - -// tableReadOf replays a reference path through pathProbe, so two paths pinning one -// row by different routes compare equal. checkPath proved -// each selector names a row. -func tableReadOf(head node, a arm, leaf node) *tableRead { - t, isTable := head.(*table) - if !isTable { - return nil - } - probe := &pathProbe{drawn: map[*table]bool{}} - _, _ = probe.walk(t, a.tail) - tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} - written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] - cur := t - for i, seg := range a.tail { - switch d := cur.descendant(seg); { - case isSelector(seg): - tr.sels = append(tr.sels, tableSel{cur, written + joinSegments(a.tail[:i+1])}) - case d != nil: - cur = d - } - } - _, tr.landsRow = leaf.(*tableRow) - return tr -} - -// selected is the selector in r on t, or on the nearest ancestor of t it selects. -func (r *tableRead) selected(t *table) (tableSel, bool) { - for ; t != nil; t = t.parentT { - for _, s := range r.sels { - if s.t == t { - return s, true - } - } - } - return tableSel{}, false -} - -// drawnOf is a table the read draws that pins holds a row of, if any. -func (r *tableRead) drawnOf(pins *pinSet) *table { - var found *table - pins.each(func(t *table, _ int) { - if found == nil && r.drawn[t] { - found = t - } - }) - return found -} - -// replay pins the read's rows into d, where they agree with the rows pinned before. -func (r *tableRead) replay(d *pinSet) error { - var err error - r.pins.each(func(t *table, row int) { - if err == nil { - err = d.pinRow(t, row) - } - }) - return err -} diff --git a/drawfence_test.go b/drawfence_test.go index 7bfb00c..069246e 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -63,7 +63,7 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { roots, tables := fenceRoots(t, f) traced := 0 for _, root := range roots { - traced += wantGathered(t, tables, root.label, renderDraws(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) + traced += wantGathered(t, tables, root.label, surveyRender(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) if !root.t.isRecord || len(root.t.fields) == 0 { continue } @@ -71,7 +71,7 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { if err != nil { t.Fatalf("%s: recordOf = %v", root.label, err) } - traced += wantGathered(t, tables, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) + traced += wantGathered(t, tables, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) } if traced == 0 { t.Error("no render traced a reference read, so the comparison proved nothing") @@ -90,7 +90,7 @@ func wantGathered(t *testing.T, tables map[string]*table, label string, gathered if !isRef(a.key) { return } - r := pathRead{at: drawAt{group: group}, a: a} + r := pathRead{at: surveyAt{group: group}, a: a} if table != "" { r.at.pins.add(tables[table], row) } @@ -104,7 +104,7 @@ func wantGathered(t *testing.T, tables map[string]*table, label string, gathered } for _, r := range reads { if !gathers(gathered, r) { - t.Errorf("%s: the render read {%s} in draw group %q from %s, and drawWalk gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) + t.Errorf("%s: the render read {%s} in draw group %q from %s, and readSurvey gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) } } return len(reads) @@ -171,9 +171,9 @@ func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { }) compared := 0 for _, root := range roots { - compared += wantPairsFindAll(t, root.label, renderDraws(root.t).reads) + compared += wantPairsFindAll(t, root.label, surveyRender(root.t).reads) if root.t.isRecord && len(root.t.fields) > 0 { - compared += wantPairsFindAll(t, root.label+" as a record", columnDraws(root.t, sortedNames(root.t.fields)).reads) + compared += wantPairsFindAll(t, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads) } } if compared == 0 { @@ -201,7 +201,7 @@ func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { if !isTemplate { t.Fatalf("NewTemplate(%s) compiled to %T", tb.selectorSpelling(r), inline.n) } - reads = append(reads, renderDraws(root).reads...) + reads = append(reads, surveyRender(root).reads...) } return nil }) @@ -300,7 +300,7 @@ func TestDrawWalkWalksACellByEveryRoute(t *testing.T) { t.Fatalf("r is a %T, want a template", g.children["r"]) } var pinned, whole bool - for _, read := range renderDraws(r).reads { + for _, read := range surveyRender(r).reads { if read.a.path != "/z" { continue } diff --git a/holdfence.go b/holdfence.go index a91d6aa..2a41d2c 100644 --- a/holdfence.go +++ b/holdfence.go @@ -18,7 +18,7 @@ func heldCheck(path string, n node) error { readers := pathKeyReaders(t.compiled.ops, t.compiled.pathKeys) for _, name := range heldNames(t) { if _, isPath := t.compiled.pathKeys[name]; isPath && isRef(name) { - continue // held for the render: drawCheck compares every read of it, by path, across the render and its groups + continue // held for the render: readFence compares every read of it, by path, across the render and its groups } if err := checkNameHeld(t, name, readers); err != nil { return fmt.Errorf("%s: %w", path, err) diff --git a/readsurvey.go b/readsurvey.go new file mode 100644 index 0000000..b41c166 --- /dev/null +++ b/readsurvey.go @@ -0,0 +1,241 @@ +package fejkdata + +import ( + "fmt" + "sort" +) + +// readSurvey gathers, at load, the references one render reads, by draw group, for check to compare. +type readSurvey struct { + reads []pathRead + read map[readKey]bool + seen map[nodeVisit]bool + pinIDs map[pinsLink]pinsID +} + +// surveyAt is where a survey stands: the draw group it draws in, how the render's root reached it, the +// table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose +// cells the walk is in. +type surveyAt struct { + group string + route readRoute + pins pinSet + wholeTable *table + wholePins pinSet +} + +// readRoute is how a render reaches a read: as its author spells it, and the root edge's label. +type readRoute struct{ spelling, label string } + +// pathRead is one reference a render reads: a path, or a bare reference with no tail. +type pathRead struct { + at surveyAt + a arm + tr *tableRead // set where the reference names a table +} + +type nodeVisit struct { + n node + group string + wholeTable *table + pins pinsID + wholePins pinsID +} + +type readKey struct { + group string + path string + pins pinsID + wholePins pinsID +} + +// pinsID names a pin set within one walk: two sets pinning the same rows share one. +type pinsID int + +// pinsLink is a set's last pin, in path order, and the id of the set before it. +type pinsLink struct { + prev pinsID + pin tablePin +} + +func newReadSurvey() *readSurvey { + return &readSurvey{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}, pinIDs: map[pinsLink]pinsID{}} +} + +func surveyRender(t *template) *readSurvey { + w := newReadSurvey() + for _, e := range renderEdges(t) { + w.edge(t, e, surveyAt{group: t.link.drawGroupKey, route: readRoute{e.reached(), e.label}}) + } + return w +} + +func surveyColumns(t *template, columns []string) *readSurvey { + w := newReadSurvey() + for _, name := range columns { + w.walk(t.fields[name], surveyAt{group: t.link.drawGroupKey, route: readRoute{spelling: fmt.Sprintf("column %q", name)}}) + } + return w +} + +func (w *readSurvey) pinsID(p *pinSet) pinsID { + var pins []tablePin + p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) + sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) + id := pinsID(0) + for _, q := range pins { + l := pinsLink{id, q} + next, ok := w.pinIDs[l] + if !ok { + next = pinsID(len(w.pinIDs) + 1) + w.pinIDs[l] = next + } + id = next + } + return id +} + +// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops +// there. A cell whose row the pins keep out has no case: a pinned table renders its pinned row +// alone and an unpinned one draws a row inside its nearest pinned ancestor's, so that cell never +// renders on this route. +func (w *readSurvey) walk(n node, at surveyAt) { + v := nodeVisit{n, at.group, at.wholeTable, w.pinsID(&at.pins), w.pinsID(&at.wholePins)} + if w.seen[v] { + return + } + w.seen[v] = true + if repeats(n) { + return + } + if t, isTemplate := n.(*template); isTemplate && t.link.drawGroupKey != "" { + at.group = t.link.drawGroupKey + } + if t, isTable := n.(*table); isTable { + at.wholeTable = t + } + for _, e := range renderEdges(n) { + to, isTemplate := e.to.(*template) + cell := cellSite{} + if isTemplate { + cell = to.cell + } + switch { + case cell.table == nil: + w.edge(n, e, at) + case cell.table == at.wholeTable: + in := at + in.wholePins = at.wholePins.clone() + in.wholePins.add(cell.table, cell.row) + w.edge(n, e, in) + case at.pins.clash(cell.table, cell.row) == nil: + in := at + in.pins = at.pins.entered(cell.table, cell.row) + w.edge(n, e, in) + } + } +} + +// edge records the reference an edge reads, then walks on with every row the read +// pins entered, so a selected row renders only its own cells. +func (w *readSurvey) edge(from node, e renderEdge, at surveyAt) { + if e.readsRef() { + a := e.read + tr := tableReadOf(from.(*template).head(a.key), a, e.to) + if k := (readKey{at.group, a.path, w.pinsID(&at.pins), w.pinsID(&at.wholePins)}); !w.read[k] { + w.read[k] = true + w.reads = append(w.reads, pathRead{at, a, tr}) + } + if tr != nil { + tr.pins.each(func(t *table, r int) { at.pins = at.pins.entered(t, r) }) + } + } + w.walk(e.to, at) +} + +// spelled names the route, and the reference it reaches a draw by where its root edge is not that +// reference. +func (r readRoute) spelled(ref string) string { + if ref == "" || ref == r.label { + return r.spelling + } + return fmt.Sprintf("%s with {%s}", r.spelling, ref) +} + +// tableRead is what a reference path reads of a table family: the table its head +// names, the rows its selectors pin, the tables it draws — those it walks with no +// row pinned, and their unpinned ancestors — each selector's spelling, and whether +// it lands on a `tableRow`. +type tableRead struct { + headTable *table + pins pinSet + drawn map[*table]bool + sels []tableSel + landsRow bool +} + +// tableSel is one selector on the way: the table it selects a row of, and the path +// as written up to and including it. +type tableSel struct { + t *table + spelling string +} + +// tableReadOf replays a reference path through pathProbe, so two paths pinning one +// row by different routes compare equal. checkPath proved +// each selector names a row. +func tableReadOf(head node, a arm, leaf node) *tableRead { + t, isTable := head.(*table) + if !isTable { + return nil + } + probe := &pathProbe{drawn: map[*table]bool{}} + _, _ = probe.walk(t, a.tail) + tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} + written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] + cur := t + for i, seg := range a.tail { + switch d := cur.descendant(seg); { + case isSelector(seg): + tr.sels = append(tr.sels, tableSel{cur, written + joinSegments(a.tail[:i+1])}) + case d != nil: + cur = d + } + } + _, tr.landsRow = leaf.(*tableRow) + return tr +} + +// selected is the selector in r on t, or on the nearest ancestor of t it selects. +func (r *tableRead) selected(t *table) (tableSel, bool) { + for ; t != nil; t = t.parentT { + for _, s := range r.sels { + if s.t == t { + return s, true + } + } + } + return tableSel{}, false +} + +// drawnOf is a table the read draws that pins holds a row of, if any. +func (r *tableRead) drawnOf(pins *pinSet) *table { + var found *table + pins.each(func(t *table, _ int) { + if found == nil && r.drawn[t] { + found = t + } + }) + return found +} + +// replay pins the read's rows into d, where they agree with the rows pinned before. +func (r *tableRead) replay(d *pinSet) error { + var err error + r.pins.each(func(t *table, row int) { + if err == nil { + err = d.pinRow(t, row) + } + }) + return err +} diff --git a/table_test.go b/table_test.go index 1ad0eef..c05b2ee 100644 --- a/table_test.go +++ b/table_test.go @@ -1000,7 +1000,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rowCount(); row++ { - whole := drawAt{wholeTable: entered} + whole := surveyAt{wholeTable: entered} whole.wholePins.add(entered, row) s := none.entered(entered, row) for _, cand := range tables { @@ -1013,14 +1013,14 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for cr := 0; cr < cand.rowCount(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil - candAt := drawAt{pins: none.entered(cand, cr)} - if got := alternatives(drawAt{pins: s}, candAt); got != refused { + candAt := surveyAt{pins: none.entered(cand, cr)} + if got := alternatives(surveyAt{pins: s}, candAt); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } if alternatives(whole, candAt) || alternatives(candAt, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } - other := drawAt{wholeTable: cand} + other := surveyAt{wholeTable: cand} other.wholePins.add(cand, cr) if want := cand == entered && cr != row; alternatives(whole, other) != want { t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) diff --git a/todo.md b/todo.md index ddab1d6..41aa6ec 100644 --- a/todo.md +++ b/todo.md @@ -26,10 +26,7 @@ least modify. This round, in order: -- Rename the load-time `draw*` types (`drawWalk`, `drawAt`, `drawRoute`, `drawCheck`) - to say they read and draw nothing, and move `drawWalk` and `tableRead` out of - `drawfence.go`: a grep for "draw" lands in render and check code alike. -- Spell `drawWalk.walk`'s dropped cell as a case of its switch: a clashing row's cell +- Spell `readSurvey.walk`'s dropped cell as a case of its switch: a clashing row's cell falls out with no arm, which four seats named. - Define `descended` where `table.route` takes it: five seats reverse-engineered it from the `draw` expression and the four walks' loop condition. @@ -313,7 +310,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. back to the clause it came from: the goal spells them out in prose while `todo.md` records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. - Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and - a helper reached only through `drawWalk.check`, and misses `checkDrawGroup`, + a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`, `checkOwnFamily` and `checkColumnDraws`; the README's Layout indexes the fences by file. - Cut the README's Layout block to the lines that say what a file name cannot: `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -- 2.52.0 From f7cfc753942ffd9f5f82a53e06f2e788792bcbbd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:54:53 +0200 Subject: [PATCH 241/966] Name the fence drawFence and the route surveyRoute, and move the survey tests to readsurvey_test.go --- AGENTS.md | 2 +- data.go | 2 +- doc.go | 2 +- docs/decisions.md | 2 +- drawfence.go | 34 +++++------ drawfence_test.go | 136 +------------------------------------------- holdfence.go | 2 +- readsurvey.go | 54 +++++++++--------- readsurvey_test.go | 137 +++++++++++++++++++++++++++++++++++++++++++++ 9 files changed, 187 insertions(+), 184 deletions(-) create mode 100644 readsurvey_test.go diff --git a/AGENTS.md b/AGENTS.md index 8ce3b36..a66402a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,5 +110,5 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only -- Each entry point to `readFence.hasRead` says what its caller gets +- Each entry point to `drawFence.hasRead` says what its caller gets - The four path walks are four loops diff --git a/data.go b/data.go index 62a9af4..b370355 100644 --- a/data.go +++ b/data.go @@ -246,7 +246,7 @@ func checkRenders(s nodeScope) error { if err := s(heldCheck); err != nil { return err } - fence := &readFence{} + fence := &drawFence{} refs := false if err := s(func(path string, n node) error { if t, ok := n.(*template); ok && len(t.link.refs) > 0 { diff --git a/doc.go b/doc.go index 79dfc95..787891b 100644 --- a/doc.go +++ b/doc.go @@ -37,4 +37,4 @@ package fejkdata // its cells stand in as `surveyAt.wholePins`, beside `surveyAt.wholeTable`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those -// compile. The draw fences are `heldCheck`, `readFence` and `checkFamilies`. +// compile. The draw fences are `heldCheck`, `drawFence` and `checkFamilies`. diff --git a/docs/decisions.md b/docs/decisions.md index 470ea3d..927f672 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -668,7 +668,7 @@ import drops them by name rather than by a rank a regeneration would move. the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. -## Each entry point to `readFence.hasRead` says what its caller gets +## Each entry point to `drawFence.hasRead` says what its caller gets 2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on `readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, diff --git a/drawfence.go b/drawfence.go index b1a30bd..2ca53bf 100644 --- a/drawfence.go +++ b/drawfence.go @@ -36,11 +36,11 @@ func checkNestedDrawGroup(fields map[string]node, group string) error { return nil } -// readFence fences each template of a scope as a render of its own, remembering which nodes read a +// drawFence fences each template of a scope as a render of its own, remembering which nodes read a // reference path. // docs/decisions.md#a-render-shares-one-reference-draw-per-category-per-group // docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences -type readFence struct { +type drawFence struct { memo map[hasReadMemo]bool } @@ -51,14 +51,14 @@ type hasReadMemo struct { // checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference // path inside a repeat or a nested draw group, which draw apart from it whatever it names. -func (c *readFence) checkDrawGroup(path string, n node) error { +func (c *drawFence) checkDrawGroup(path string, n node) error { if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) { return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) } return nil } -func (c *readFence) checkDraws(path string, n node) error { +func (c *drawFence) checkDraws(path string, n node) error { t, ok := n.(*template) if !ok { return nil @@ -116,7 +116,7 @@ func checkOwnFamily(t *template) error { // checkRecordDraws fences the columns of a record — a template compiled at the top without a // repeat — so a load proves the record view of it as well as the string view. // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load -func (c *readFence) checkRecordDraws(path string, n node) error { +func (c *drawFence) checkRecordDraws(path string, n node) error { t, ok := n.(*template) if !ok || !t.isRecord { return nil @@ -138,15 +138,15 @@ func (c *readFence) checkRecordDraws(path string, n node) error { // readsPath reports whether rendering n reads a reference path, short of a repeat, which renders // over draws of its own. -func (c *readFence) readsPath(n node) bool { return c.hasRead(n, false) } +func (c *drawFence) readsPath(n node) bool { return c.hasRead(n, false) } // splitsDraws reports whether rendering n reads a reference path that n's own draw group answers // for: one outside a repeat and outside a nested draw group, which hold their own draws. -func (c *readFence) splitsDraws(n node) bool { return c.hasRead(n, true) } +func (c *drawFence) splitsDraws(n node) bool { return c.hasRead(n, true) } // hasRead walks what rendering n renders for a reference path, stopping at a repeat — and at a nested // draw group when stopAtGroup — since each holds draws of its own. -func (c *readFence) hasRead(n node, stopAtGroup bool) bool { +func (c *drawFence) hasRead(n node, stopAtGroup bool) bool { k := hasReadMemo{n, stopAtGroup} if r, done := c.memo[k]; done { return r @@ -200,15 +200,15 @@ func checkColumnDraws(t *template, columns []string) error { return surveyColumn // check refuses what one draw per reference path cannot answer for: a read of a level beside a path // another read takes into it, and two reads of one table family that select different rows. Reads // are compared in path order, so which pair is reported does not vary. -func (w *readSurvey) check() error { - sort.SliceStable(w.reads, func(i, j int) bool { - if w.reads[i].at.group != w.reads[j].at.group { - return w.reads[i].at.group < w.reads[j].at.group +func (s *readSurvey) check() error { + sort.SliceStable(s.reads, func(i, j int) bool { + if s.reads[i].at.group != s.reads[j].at.group { + return s.reads[i].at.group < s.reads[j].at.group } - return w.reads[i].a.path < w.reads[j].a.path + return s.reads[i].a.path < s.reads[j].a.path }) - for i, level := range w.reads { - for _, into := range w.reads[i+1:] { + for i, level := range s.reads { + for _, into := range s.reads[i+1:] { if into.at.group != level.at.group || alternatives(level.at, into.at) { continue } @@ -217,7 +217,7 @@ func (w *readSurvey) check() error { } } } - return checkFamilies(w.reads) + return checkFamilies(s.reads) } // checkFamilies refuses reads of one table family in one draw group that cannot @@ -303,6 +303,6 @@ func alternatives(a, b surveyAt) bool { return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) } -func overlapError(route readRoute, ref string, into pathRead) error { +func overlapError(route surveyRoute, ref string, into pathRead) error { return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) } diff --git a/drawfence_test.go b/drawfence_test.go index 069246e..f690f1b 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -1,10 +1,6 @@ package fejkdata -import ( - "strings" - "testing" - "testing/fstest" -) +import "testing" func fenceCorpus(t *testing.T) *Generator { t.Helper() @@ -58,104 +54,6 @@ func fenceRoots(t *testing.T, f *Generator) ([]fenceRoot, map[string]*table) { return append(roots, fenceRoot{"inline record", record.template}), tables } -func TestEveryReadARenderMakesIsGathered(t *testing.T) { - f := fenceCorpus(t) - roots, tables := fenceRoots(t, f) - traced := 0 - for _, root := range roots { - traced += wantGathered(t, tables, root.label, surveyRender(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) - if !root.t.isRecord || len(root.t.fields) == 0 { - continue - } - _, columns, err := recordOf(root.t) - if err != nil { - t.Fatalf("%s: recordOf = %v", root.label, err) - } - traced += wantGathered(t, tables, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) - } - if traced == 0 { - t.Error("no render traced a reference read, so the comparison proved nothing") - } -} - -func wantGathered(t *testing.T, tables map[string]*table, label string, gathered []pathRead, renderWith func(renderTrace)) int { - t.Helper() - var reads []pathRead - type traceKey struct { - group, path, table string - row int - } - seen := map[traceKey]bool{} - trace := func(group, table string, row int, a arm) { - if !isRef(a.key) { - return - } - r := pathRead{at: surveyAt{group: group}, a: a} - if table != "" { - r.at.pins.add(tables[table], row) - } - if k := (traceKey{group, a.path, table, row}); !seen[k] { - seen[k] = true - reads = append(reads, r) - } - } - for i := 0; i < 5; i++ { - renderWith(trace) - } - for _, r := range reads { - if !gathers(gathered, r) { - t.Errorf("%s: the render read {%s} in draw group %q from %s, and readSurvey gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) - } - } - return len(reads) -} - -func gathers(gathered []pathRead, r pathRead) bool { - for _, g := range gathered { - if g.at.group == r.at.group && g.a.path == r.a.path && !g.at.pins.differs(&r.at.pins) && !g.at.wholePins.differs(&r.at.pins) { - return true - } - } - return false -} - -func spellPins(p *pinSet) string { - var rows []string - p.each(func(t *table, r int) { rows = append(rows, t.selectorSpelling(r)) }) - if len(rows) == 0 { - return "no row" - } - return "row " + strings.Join(rows, ", ") -} - -func renderRoot(s *session, t *template, trace renderTrace) { - set := holdSet{trace: trace} - sc := renderScope{set: &set} - switch { - case t.table != nil: - render(s, t.table, sc) - for r := 0; r < t.table.rowCount(); r++ { - set := holdSet{trace: trace} - sc := renderScope{set: &set} - sc.groupHold().pins.pin(t.table, r) - render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.groupHold().pins)) - } - case t.repeat > 1: - expand(s, t, sc) - default: - render(s, t, sc) - } -} - -func renderRecordRoot(s *session, t *template, columns []Column, trace renderTrace) { - set := holdSet{trace: trace} - sc := renderScope{set: &set} - if t.table != nil { - t.table.drawIn(s, &sc.groupHold().pins) - } - renderRecord(s, t, columns, sc) -} - func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { corpus := fenceCorpus(t) roots, _ := fenceRoots(t, corpus) @@ -279,35 +177,3 @@ func eachRenderSet(reads []pathRead, fn func([]pathRead)) { } grow(nil, reads, nil) } - -func TestDrawWalkWalksACellByEveryRoute(t *testing.T) { - fsys := fstest.MapFS{ - "g.json": {Data: []byte(`{"format":"{/x.v}","drawGroup":"g2"}`)}, - "r.json": {Data: []byte(`"{/g} {/x}"`)}, - "x.json": {Data: []byte(`{"format":"{/g}","rows":"x.tsv","key":"code"}`)}, - "x.tsv": {Data: []byte("code\tv\n1\t{/z}\n2\t{/z.a}\n")}, - "z.json": {Data: []byte(`{"format":"{a}","a":"hi"}`)}, - } - g, err := loadDir(dataSource{fsys: fsys}, ".") - if err != nil { - t.Fatalf("loadDir = %v", err) - } - if err := treeBinding(g.children).link(); err != nil { - t.Fatalf("link = %v", err) - } - r, isTemplate := g.children["r"].(*template) - if !isTemplate { - t.Fatalf("r is a %T, want a template", g.children["r"]) - } - var pinned, whole bool - for _, read := range surveyRender(r).reads { - if read.a.path != "/z" { - continue - } - pinned = pinned || read.at.pins.used > 0 - whole = whole || read.at.wholePins.used > 0 - } - if !pinned || !whole { - t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root and from inside {/x}", pinned, whole) - } -} diff --git a/holdfence.go b/holdfence.go index 2a41d2c..53a54ef 100644 --- a/holdfence.go +++ b/holdfence.go @@ -18,7 +18,7 @@ func heldCheck(path string, n node) error { readers := pathKeyReaders(t.compiled.ops, t.compiled.pathKeys) for _, name := range heldNames(t) { if _, isPath := t.compiled.pathKeys[name]; isPath && isRef(name) { - continue // held for the render: readFence compares every read of it, by path, across the render and its groups + continue // held for the render: drawFence compares every read of it, by path, across the render and its groups } if err := checkNameHeld(t, name, readers); err != nil { return fmt.Errorf("%s: %w", path, err) diff --git a/readsurvey.go b/readsurvey.go index b41c166..20a8ce7 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -13,19 +13,19 @@ type readSurvey struct { pinIDs map[pinsLink]pinsID } -// surveyAt is where a survey stands: the draw group it draws in, how the render's root reached it, the +// surveyAt is where a survey stands: the draw group its reads draw in, how the render's root reached it, the // table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose // cells the walk is in. type surveyAt struct { group string - route readRoute + route surveyRoute pins pinSet wholeTable *table wholePins pinSet } -// readRoute is how a render reaches a read: as its author spells it, and the root edge's label. -type readRoute struct{ spelling, label string } +// surveyRoute is how a render reaches a draw: as its author spells it, and the root edge's label. +type surveyRoute struct{ spelling, label string } // pathRead is one reference a render reads: a path, or a bare reference with no tail. type pathRead struct { @@ -63,32 +63,32 @@ func newReadSurvey() *readSurvey { } func surveyRender(t *template) *readSurvey { - w := newReadSurvey() + s := newReadSurvey() for _, e := range renderEdges(t) { - w.edge(t, e, surveyAt{group: t.link.drawGroupKey, route: readRoute{e.reached(), e.label}}) + s.edge(t, e, surveyAt{group: t.link.drawGroupKey, route: surveyRoute{e.reached(), e.label}}) } - return w + return s } func surveyColumns(t *template, columns []string) *readSurvey { - w := newReadSurvey() + s := newReadSurvey() for _, name := range columns { - w.walk(t.fields[name], surveyAt{group: t.link.drawGroupKey, route: readRoute{spelling: fmt.Sprintf("column %q", name)}}) + s.walk(t.fields[name], surveyAt{group: t.link.drawGroupKey, route: surveyRoute{spelling: fmt.Sprintf("column %q", name)}}) } - return w + return s } -func (w *readSurvey) pinsID(p *pinSet) pinsID { +func (s *readSurvey) pinsID(p *pinSet) pinsID { var pins []tablePin p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) id := pinsID(0) for _, q := range pins { l := pinsLink{id, q} - next, ok := w.pinIDs[l] + next, ok := s.pinIDs[l] if !ok { - next = pinsID(len(w.pinIDs) + 1) - w.pinIDs[l] = next + next = pinsID(len(s.pinIDs) + 1) + s.pinIDs[l] = next } id = next } @@ -99,12 +99,12 @@ func (w *readSurvey) pinsID(p *pinSet) pinsID { // there. A cell whose row the pins keep out has no case: a pinned table renders its pinned row // alone and an unpinned one draws a row inside its nearest pinned ancestor's, so that cell never // renders on this route. -func (w *readSurvey) walk(n node, at surveyAt) { - v := nodeVisit{n, at.group, at.wholeTable, w.pinsID(&at.pins), w.pinsID(&at.wholePins)} - if w.seen[v] { +func (s *readSurvey) walk(n node, at surveyAt) { + v := nodeVisit{n, at.group, at.wholeTable, s.pinsID(&at.pins), s.pinsID(&at.wholePins)} + if s.seen[v] { return } - w.seen[v] = true + s.seen[v] = true if repeats(n) { return } @@ -122,40 +122,40 @@ func (w *readSurvey) walk(n node, at surveyAt) { } switch { case cell.table == nil: - w.edge(n, e, at) + s.edge(n, e, at) case cell.table == at.wholeTable: in := at in.wholePins = at.wholePins.clone() in.wholePins.add(cell.table, cell.row) - w.edge(n, e, in) + s.edge(n, e, in) case at.pins.clash(cell.table, cell.row) == nil: in := at in.pins = at.pins.entered(cell.table, cell.row) - w.edge(n, e, in) + s.edge(n, e, in) } } } // edge records the reference an edge reads, then walks on with every row the read // pins entered, so a selected row renders only its own cells. -func (w *readSurvey) edge(from node, e renderEdge, at surveyAt) { +func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { if e.readsRef() { a := e.read tr := tableReadOf(from.(*template).head(a.key), a, e.to) - if k := (readKey{at.group, a.path, w.pinsID(&at.pins), w.pinsID(&at.wholePins)}); !w.read[k] { - w.read[k] = true - w.reads = append(w.reads, pathRead{at, a, tr}) + if k := (readKey{at.group, a.path, s.pinsID(&at.pins), s.pinsID(&at.wholePins)}); !s.read[k] { + s.read[k] = true + s.reads = append(s.reads, pathRead{at, a, tr}) } if tr != nil { tr.pins.each(func(t *table, r int) { at.pins = at.pins.entered(t, r) }) } } - w.walk(e.to, at) + s.walk(e.to, at) } // spelled names the route, and the reference it reaches a draw by where its root edge is not that // reference. -func (r readRoute) spelled(ref string) string { +func (r surveyRoute) spelled(ref string) string { if ref == "" || ref == r.label { return r.spelling } diff --git a/readsurvey_test.go b/readsurvey_test.go new file mode 100644 index 0000000..0c39001 --- /dev/null +++ b/readsurvey_test.go @@ -0,0 +1,137 @@ +package fejkdata + +import ( + "strings" + "testing" + "testing/fstest" +) + +func TestEveryReadARenderMakesIsGathered(t *testing.T) { + f := fenceCorpus(t) + roots, tables := fenceRoots(t, f) + traced := 0 + for _, root := range roots { + traced += wantGathered(t, tables, root.label, surveyRender(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) + if !root.t.isRecord || len(root.t.fields) == 0 { + continue + } + _, columns, err := recordOf(root.t) + if err != nil { + t.Fatalf("%s: recordOf = %v", root.label, err) + } + traced += wantGathered(t, tables, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) + } + if traced == 0 { + t.Error("no render traced a reference read, so the comparison proved nothing") + } +} + +func wantGathered(t *testing.T, tables map[string]*table, label string, gathered []pathRead, renderWith func(renderTrace)) int { + t.Helper() + var reads []pathRead + type traceKey struct { + group, path, table string + row int + } + seen := map[traceKey]bool{} + trace := func(group, table string, row int, a arm) { + if !isRef(a.key) { + return + } + r := pathRead{at: surveyAt{group: group}, a: a} + if table != "" { + r.at.pins.add(tables[table], row) + } + if k := (traceKey{group, a.path, table, row}); !seen[k] { + seen[k] = true + reads = append(reads, r) + } + } + for i := 0; i < 5; i++ { + renderWith(trace) + } + for _, r := range reads { + if !gathers(gathered, r) { + t.Errorf("%s: the render read {%s} in draw group %q from %s, and readSurvey gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) + } + } + return len(reads) +} + +func gathers(gathered []pathRead, r pathRead) bool { + for _, g := range gathered { + if g.at.group == r.at.group && g.a.path == r.a.path && !g.at.pins.differs(&r.at.pins) && !g.at.wholePins.differs(&r.at.pins) { + return true + } + } + return false +} + +func spellPins(p *pinSet) string { + var rows []string + p.each(func(t *table, r int) { rows = append(rows, t.selectorSpelling(r)) }) + if len(rows) == 0 { + return "no row" + } + return "row " + strings.Join(rows, ", ") +} + +func renderRoot(s *session, t *template, trace renderTrace) { + set := holdSet{trace: trace} + sc := renderScope{set: &set} + switch { + case t.table != nil: + render(s, t.table, sc) + for r := 0; r < t.table.rowCount(); r++ { + set := holdSet{trace: trace} + sc := renderScope{set: &set} + sc.groupHold().pins.pin(t.table, r) + render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.groupHold().pins)) + } + case t.repeat > 1: + expand(s, t, sc) + default: + render(s, t, sc) + } +} + +func renderRecordRoot(s *session, t *template, columns []Column, trace renderTrace) { + set := holdSet{trace: trace} + sc := renderScope{set: &set} + if t.table != nil { + t.table.drawIn(s, &sc.groupHold().pins) + } + renderRecord(s, t, columns, sc) +} + +func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { + fsys := fstest.MapFS{ + "g.json": {Data: []byte(`{"format":"{/x.v}","drawGroup":"g2"}`)}, + "r.json": {Data: []byte(`"{/g} {/x}"`)}, + "x.json": {Data: []byte(`{"format":"{/g}","rows":"x.tsv","key":"code"}`)}, + "x.tsv": {Data: []byte("code\tv\n1\t{/z}\n2\t{/z.a}\n")}, + "z.json": {Data: []byte(`{"format":"{a}","a":"hi"}`)}, + } + g, err := loadDir(dataSource{fsys: fsys}, ".") + if err != nil { + t.Fatalf("loadDir = %v", err) + } + if err := treeBinding(g.children).link(); err != nil { + t.Fatalf("link = %v", err) + } + r, isTemplate := g.children["r"].(*template) + if !isTemplate { + t.Fatalf("r is a %T, want a template", g.children["r"]) + } + var pinned, whole bool + for _, read := range surveyRender(r).reads { + if read.a.path != "/z" { + continue + } + pinned = pinned || read.at.pins.used > 0 + whole = whole || read.at.wholePins.used > 0 + } + if !pinned || !whole { + t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root and from inside {/x}", pinned, whole) + } +} -- 2.52.0 From ecc7833e04af90e1b594a2edd9576f0fa0a068ce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 02:57:46 +0200 Subject: [PATCH 242/966] Name drawFence's receiver f, and say a survey route reaches a read --- drawfence.go | 28 ++++++++++++++-------------- readsurvey.go | 4 ++-- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/drawfence.go b/drawfence.go index 2ca53bf..77b1758 100644 --- a/drawfence.go +++ b/drawfence.go @@ -51,14 +51,14 @@ type hasReadMemo struct { // checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference // path inside a repeat or a nested draw group, which draw apart from it whatever it names. -func (c *drawFence) checkDrawGroup(path string, n node) error { - if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) { +func (f *drawFence) checkDrawGroup(path string, n node) error { + if t, ok := n.(*template); ok && t.drawGroup != "" && !f.splitsDraws(t) { return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) } return nil } -func (c *drawFence) checkDraws(path string, n node) error { +func (f *drawFence) checkDraws(path string, n node) error { t, ok := n.(*template) if !ok { return nil @@ -66,7 +66,7 @@ func (c *drawFence) checkDraws(path string, n node) error { if err := checkOwnFamily(t); err != nil { return fmt.Errorf("%s: %w", path, err) } - if !c.readsPath(t) { + if !f.readsPath(t) { return nil } if err := surveyRender(t).check(); err != nil { @@ -116,7 +116,7 @@ func checkOwnFamily(t *template) error { // checkRecordDraws fences the columns of a record — a template compiled at the top without a // repeat — so a load proves the record view of it as well as the string view. // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load -func (c *drawFence) checkRecordDraws(path string, n node) error { +func (f *drawFence) checkRecordDraws(path string, n node) error { t, ok := n.(*template) if !ok || !t.isRecord { return nil @@ -125,7 +125,7 @@ func (c *drawFence) checkRecordDraws(path string, n node) error { columns := sortedNames(t.fields) reads := false for _, name := range columns { - reads = reads || c.readsPath(t.fields[name]) + reads = reads || f.readsPath(t.fields[name]) } if !reads { return nil @@ -138,30 +138,30 @@ func (c *drawFence) checkRecordDraws(path string, n node) error { // readsPath reports whether rendering n reads a reference path, short of a repeat, which renders // over draws of its own. -func (c *drawFence) readsPath(n node) bool { return c.hasRead(n, false) } +func (f *drawFence) readsPath(n node) bool { return f.hasRead(n, false) } // splitsDraws reports whether rendering n reads a reference path that n's own draw group answers // for: one outside a repeat and outside a nested draw group, which hold their own draws. -func (c *drawFence) splitsDraws(n node) bool { return c.hasRead(n, true) } +func (f *drawFence) splitsDraws(n node) bool { return f.hasRead(n, true) } // hasRead walks what rendering n renders for a reference path, stopping at a repeat — and at a nested // draw group when stopAtGroup — since each holds draws of its own. -func (c *drawFence) hasRead(n node, stopAtGroup bool) bool { +func (f *drawFence) hasRead(n node, stopAtGroup bool) bool { k := hasReadMemo{n, stopAtGroup} - if r, done := c.memo[k]; done { + if r, done := f.memo[k]; done { return r } r := false for _, e := range renderEdges(n) { - if readsOnEdge(n, e) || (walksInto(e.to, stopAtGroup) && c.hasRead(e.to, stopAtGroup)) { + if readsOnEdge(n, e) || (walksInto(e.to, stopAtGroup) && f.hasRead(e.to, stopAtGroup)) { r = true break } } - if c.memo == nil { - c.memo = map[hasReadMemo]bool{} + if f.memo == nil { + f.memo = map[hasReadMemo]bool{} } - c.memo[k] = r + f.memo[k] = r return r } diff --git a/readsurvey.go b/readsurvey.go index 20a8ce7..5a9bd62 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -24,7 +24,7 @@ type surveyAt struct { wholePins pinSet } -// surveyRoute is how a render reaches a draw: as its author spells it, and the root edge's label. +// surveyRoute is how a render reaches a read: as its author spells it, and the root edge's label. type surveyRoute struct{ spelling, label string } // pathRead is one reference a render reads: a path, or a bare reference with no tail. @@ -153,7 +153,7 @@ func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { s.walk(e.to, at) } -// spelled names the route, and the reference it reaches a draw by where its root edge is not that +// spelled names the route, and the reference it reaches a read by where its root edge is not that // reference. func (r surveyRoute) spelled(ref string) string { if ref == "" || ref == r.label { -- 2.52.0 From 4e4254fa8480ec90495505b37d246361173a24ae Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:00:31 +0200 Subject: [PATCH 243/966] Title the stem decision by its rule, and point doc.go's whole at surveyAt --- AGENTS.md | 2 +- doc.go | 4 ++-- docs/decisions.md | 6 +++--- todo.md | 1 + 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a66402a..1d51302 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -65,7 +65,7 @@ In [docs/decisions.md](docs/decisions.md): - A column of one reference alone is the column it reads - A typed column's calc is refused unless proven - `Column` carries text, not a Go value -- `hold` names what a draw is kept in, `readValue` what a read drew +- A unit takes the stem of what it is, and a file the stem of the units it holds - One name, one meaning - The package stays flat - The performance gate asserts allocations, not wall-clock time diff --git a/doc.go b/doc.go index 787891b..b5f8528 100644 --- a/doc.go +++ b/doc.go @@ -33,8 +33,8 @@ package fejkdata // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // - whole — a read of a table with no selector and no descent, {/city}, landing on -// the `table` itself: it draws a row apart from every pin, so the fences keep the rows -// its cells stand in as `surveyAt.wholePins`, beside `surveyAt.wholeTable`. +// the `table` itself, so it draws a row apart from every pin: +// `surveyAt.wholeTable`, `surveyAt.wholePins`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those // compile. The draw fences are `heldCheck`, `drawFence` and `checkFamilies`. diff --git a/docs/decisions.md b/docs/decisions.md index 927f672..cb62999 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -302,10 +302,10 @@ in. `Value` is the rendered string beside `DataType` and `Null`, which each serializer writes as the load check proved it; a `Value any` would hand every caller a type switch. -## `hold` names what a draw is kept in, `readValue` what a read drew +## A unit takes the stem of what it is, and a file the stem of the units it holds -Goal 2 wants a name to reach one unit, so a unit takes the stem of what it is, and a -file the stem of the units it holds. +Goal 2 wants a name to reach one unit: `hold` is what a draw is kept in, `readValue` what +a read drew. ## One name, one meaning diff --git a/todo.md b/todo.md index 41aa6ec..86501a4 100644 --- a/todo.md +++ b/todo.md @@ -312,6 +312,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`, `checkOwnFamily` and `checkColumnDraws`; the README's Layout indexes the fences by file. + Say "fences" in the `pin` entry then, which names the draw fences too. - Cut the README's Layout block to the lines that say what a file name cannot: `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. - Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word -- 2.52.0 From 38fbc476789c5f752b6f8426689e84ea2c5817cb Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:15:51 +0200 Subject: [PATCH 244/966] Spell readSurvey.walk's dropped cell as a default arm --- readsurvey.go | 7 ++++--- todo.md | 2 -- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/readsurvey.go b/readsurvey.go index 5a9bd62..9d718be 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -96,9 +96,7 @@ func (s *readSurvey) pinsID(p *pinSet) pinsID { } // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops -// there. A cell whose row the pins keep out has no case: a pinned table renders its pinned row -// alone and an unpinned one draws a row inside its nearest pinned ancestor's, so that cell never -// renders on this route. +// there. func (s *readSurvey) walk(n node, at surveyAt) { v := nodeVisit{n, at.group, at.wholeTable, s.pinsID(&at.pins), s.pinsID(&at.wholePins)} if s.seen[v] { @@ -132,6 +130,9 @@ func (s *readSurvey) walk(n node, at surveyAt) { in := at in.pins = at.pins.entered(cell.table, cell.row) s.edge(n, e, in) + default: + // A pinned table renders its pinned row alone, and an unpinned one draws a row inside its + // nearest pinned ancestor's, so a cell of a row the pins clash with never renders here. } } } diff --git a/todo.md b/todo.md index 86501a4..ef8e9c9 100644 --- a/todo.md +++ b/todo.md @@ -26,8 +26,6 @@ least modify. This round, in order: -- Spell `readSurvey.walk`'s dropped cell as a case of its switch: a clashing row's cell - falls out with no arm, which four seats named. - Define `descended` where `table.route` takes it: five seats reverse-engineered it from the `draw` expression and the four walks' loop condition. - Keep a template's table in one field: `template.table` and `template.cell.table` both -- 2.52.0 From 0ad94089e2722be9a938f00124a88aee17530b44 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:19:25 +0200 Subject: [PATCH 245/966] Say on which route readSurvey.walk's dropped cell never renders --- readsurvey.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/readsurvey.go b/readsurvey.go index 9d718be..0b01e8a 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -132,7 +132,7 @@ func (s *readSurvey) walk(n node, at surveyAt) { s.edge(n, e, in) default: // A pinned table renders its pinned row alone, and an unpinned one draws a row inside its - // nearest pinned ancestor's, so a cell of a row the pins clash with never renders here. + // nearest pinned ancestor's, so a cell of a row the pins clash with never renders on this route. } } } -- 2.52.0 From 05bc56fe75049299b2b3efcfc4fbf0c77a8dc247 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:24:44 +0200 Subject: [PATCH 246/966] Define descended where table.route takes it --- path.go | 7 ++++--- todo.md | 2 -- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/path.go b/path.go index 771248c..2e015cb 100644 --- a/path.go +++ b/path.go @@ -156,9 +156,10 @@ type tableRoute struct { descends bool } -// route is how tail passes t, reached from another table where descended. The path -// reads a row wherever a segment follows or the table was reached from another; a -// table reached whole, with no selector, is left to a render's own draw. +// route is how tail passes t. descended means the previous route stepped into t +// from a row of an ancestor table, so t reads a row even where tail is empty; a +// table reached otherwise, with no selector and an empty tail, is left to a render's +// own draw. func (t *table) route(tail []string, descended bool) (tableRoute, error) { sel, tail, err := t.selector(tail) if err != nil { diff --git a/todo.md b/todo.md index ef8e9c9..158eaaf 100644 --- a/todo.md +++ b/todo.md @@ -26,8 +26,6 @@ least modify. This round, in order: -- Define `descended` where `table.route` takes it: five seats reverse-engineered it from - the `draw` expression and the four walks' loop condition. - Keep a template's table in one field: `template.table` and `template.cell.table` both answer which table it belongs to, and `checkOwnFamily` and `traceRead` branch on both. - Stop `readField` dereferencing a nil `held`: `renderLeaf` passes nil, safe only -- 2.52.0 From e9cdfe9edda72f87a6ee8b0aefb630e4fec380ef Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:30:19 +0200 Subject: [PATCH 247/966] Keep a template's table in one field, tableSite --- drawfence.go | 5 +---- hold.go | 12 ++++++------ node.go | 18 ++++++++++-------- readsurvey.go | 6 +++--- readsurvey_test.go | 15 ++++++++------- record.go | 4 ++-- reference.go | 4 ++-- table.go | 4 ++-- todo.md | 2 -- 9 files changed, 34 insertions(+), 36 deletions(-) diff --git a/drawfence.go b/drawfence.go index 77b1758..078b1bf 100644 --- a/drawfence.go +++ b/drawfence.go @@ -81,10 +81,7 @@ func (f *drawFence) checkDraws(path string, n node) error { // it renders, whichever draws the reaching template holds. // docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route func checkOwnFamily(t *template) error { - own := t.table - if own == nil { - own = t.cell.table - } + own := t.site.table if own == nil { return nil } diff --git a/hold.go b/hold.go index 590ee5b..3bfec8b 100644 --- a/hold.go +++ b/hold.go @@ -65,12 +65,12 @@ type renderTrace func(group, table string, row int, a arm) func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { var table string - var row int - switch { - case t.cell.table != nil: - table, row = t.cell.table.path, t.cell.row - case t.table != nil: - table, row = t.table.path, sc.rowOf(t.table) + row := t.site.row + if t.site.table != nil { + table = t.site.table.path + } + if row == formatRow { + row = sc.rowOf(t.site.table) } trace(strings.Clone(sc.group), strings.Clone(table), row, a) } diff --git a/node.go b/node.go index 5f6f754..28cecfb 100644 --- a/node.go +++ b/node.go @@ -54,20 +54,22 @@ type template struct { drawGroup string // the draw group it draws in, as written; "" keeps its caller's fromString bool // written as a JSON string rather than an object isRecord bool // compiled at the top without a repeat, so its fields are record columns - table *table // the table whose format this is, whose columns are the fields - cell cellSite + site tableSite // Filled by `linkTemplate`, from the assembled tree: link templateLink compiled formatOps } -// cellSite is the table cell a template is compiled from; table is nil for any other template. -type cellSite struct { +// tableSite is the table a template belongs to, as its format or one of its cells; table is nil +// for any other template. +type tableSite struct { table *table - row int + row int // the cell's row, or formatRow for the format, whose columns are the fields } +const formatRow = -1 + // templateLink is what a template resolves to in the assembled tree. type templateLink struct { refs map[string]refBinding // each reference the format reads -> what it resolves to @@ -172,7 +174,7 @@ func compileAt(v any, pos position) (node, error) { func compileItem(v any, pos position) (node, error) { switch v := v.(type) { case string: - t, err := compileString(v, cellSite{}) + t, err := compileString(v, tableSite{}) if err != nil { return nil, err } @@ -203,12 +205,12 @@ func jsonKind(v any) string { return fmt.Sprintf("%T", v) } -func compileString(s string, cell cellSite) (*template, error) { +func compileString(s string, site tableSite) (*template, error) { toks, err := parseChecked(s, nil) if err != nil { return nil, err } - return &template{format: s, tokens: toks, repeat: 1, fromString: true, cell: cell}, nil + return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site}, nil } // fixedText is one render's output when the format holds no token; repeat is the caller's. diff --git a/readsurvey.go b/readsurvey.go index 0b01e8a..fb43df6 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -114,9 +114,9 @@ func (s *readSurvey) walk(n node, at surveyAt) { } for _, e := range renderEdges(n) { to, isTemplate := e.to.(*template) - cell := cellSite{} - if isTemplate { - cell = to.cell + cell := tableSite{} + if isTemplate && to.site.row != formatRow { + cell = to.site } switch { case cell.table == nil: diff --git a/readsurvey_test.go b/readsurvey_test.go index 0c39001..07d6186 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -80,13 +80,14 @@ func renderRoot(s *session, t *template, trace renderTrace) { set := holdSet{trace: trace} sc := renderScope{set: &set} switch { - case t.table != nil: - render(s, t.table, sc) - for r := 0; r < t.table.rowCount(); r++ { + case t.site.row == formatRow: + tbl := t.site.table + render(s, tbl, sc) + for r := 0; r < tbl.rowCount(); r++ { set := holdSet{trace: trace} sc := renderScope{set: &set} - sc.groupHold().pins.pin(t.table, r) - render(s, t.table.pinnedRow, sc.at(t.table.pinnedRow, &sc.groupHold().pins)) + sc.groupHold().pins.pin(tbl, r) + render(s, tbl.pinnedRow, sc.at(tbl.pinnedRow, &sc.groupHold().pins)) } case t.repeat > 1: expand(s, t, sc) @@ -98,8 +99,8 @@ func renderRoot(s *session, t *template, trace renderTrace) { func renderRecordRoot(s *session, t *template, columns []Column, trace renderTrace) { set := holdSet{trace: trace} sc := renderScope{set: &set} - if t.table != nil { - t.table.drawIn(s, &sc.groupHold().pins) + if t.site.row == formatRow { + t.site.table.drawIn(s, &sc.groupHold().pins) } renderRecord(s, t, columns, sc) } diff --git a/record.go b/record.go index d0b4a79..4b4c666 100644 --- a/record.go +++ b/record.go @@ -263,8 +263,8 @@ func recordOf(n node) (*template, []Column, error) { // over sc's hold; a table's columns read the row pinned there. func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) - if t.table != nil { - sc = sc.at(t.table.pinnedRow, &sc.groupHold().pins) + if t.site.row == formatRow { + sc = sc.at(t.site.table.pinnedRow, &sc.groupHold().pins) } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { diff --git a/reference.go b/reference.go index 0563604..e218229 100644 --- a/reference.go +++ b/reference.go @@ -81,8 +81,8 @@ func linkRefs(root map[string]node) error { // linkTemplate resolves t's references in category, "" for an inline template, and // compiles its format against them. func linkTemplate(folder []string, path, category string, t *template, root map[string]node) error { - if t.cell.table != nil { - path = fmt.Sprintf("%s, line %d", path, t.cell.row+2) + if t.site.table != nil && t.site.row != formatRow { + path = fmt.Sprintf("%s, line %d", path, t.site.row+2) } link, err := t.resolveLink(folder, path, category, root) if err != nil { diff --git a/table.go b/table.go index d407d26..0d41a73 100644 --- a/table.go +++ b/table.go @@ -308,7 +308,7 @@ func (t *table) checkCells() error { if strings.IndexByte(cell, '{') < 0 && strings.IndexByte(cell, '}') < 0 { continue } - n, err := compileString(cell, cellSite{t, row}) + n, err := compileString(cell, tableSite{t, row}) if err != nil { return fmt.Errorf("line %d, %s: %w", row+2, t.header[col], err) } @@ -341,7 +341,7 @@ func (t *table) compileRowFormat(format string) error { if err := checkTokens(toks, t.fields); err != nil { return err } - t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, table: t} + t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} t.pinnedRow = &tableRow{t} return nil } diff --git a/todo.md b/todo.md index 158eaaf..703bb27 100644 --- a/todo.md +++ b/todo.md @@ -26,8 +26,6 @@ least modify. This round, in order: -- Keep a template's table in one field: `template.table` and `template.cell.table` both - answer which table it belongs to, and `checkOwnFamily` and `traceRead` branch on both. - Stop `readField` dereferencing a nil `held`: `renderLeaf` passes nil, safe only because a `readsColumn` arm is a reference path with a tail, which nothing there says. - Test that `disagreements` holds every key `disagreement` can build, and panic through -- 2.52.0 From e77a9371db281df7b02a690f1942d0063a60f2b5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:32:25 +0200 Subject: [PATCH 248/966] Cut tableSite.row's stray clause --- node.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/node.go b/node.go index 28cecfb..46c751c 100644 --- a/node.go +++ b/node.go @@ -65,7 +65,7 @@ type template struct { // for any other template. type tableSite struct { table *table - row int // the cell's row, or formatRow for the format, whose columns are the fields + row int // the cell's row, or formatRow for the format } const formatRow = -1 -- 2.52.0 From 5ce0ce147ef61b7b7bf2342779bf9cd647e47724 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:38:29 +0200 Subject: [PATCH 249/966] Read a reference path through readReference, which takes no held --- hold.go | 44 +++++++++++++++++++++++++++----------------- todo.md | 2 -- 2 files changed, 27 insertions(+), 19 deletions(-) diff --git a/hold.go b/hold.go index 3bfec8b..07e4239 100644 --- a/hold.go +++ b/hold.go @@ -25,12 +25,13 @@ type readValue struct { // readField renders one arm of a token. A name the expansion holds — a level some // token addresses by dotted path, or a field an operand reads — is drawn once and -// kept, so {place.postal-code} and {place.locality} read one row, either read twice -// gives one value, and a shown operand is the operand computed. Every other name is -// drawn afresh, so {word} {word} still draws twice. A reference path is kept in the -// render's hold for its group, so its draw spans the render; every other held name in -// held, the expansion's. +// kept in held, so {place.postal-code} and {place.locality} read one row, either read +// twice gives one value, and a shown operand is the operand computed. Every other name +// is drawn afresh, so {word} {word} still draws twice. func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { + if isRef(a.key) && len(a.tail) > 0 { + return readReference(s, t, sc, a) + } if sc.set.trace != nil { traceRead(sc.set.trace, t, sc, a) } @@ -40,23 +41,32 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV } return readValue{text: render(s, t.head(a.key), sc)} } - w := pathDraw{s: s, held: held, a: &a} - if isRef(a.key) && len(a.tail) > 0 { - group := sc.groupHold() - w.held, w.pins = &group.hold, &group.pins + return readHeld(s, t, held, nil, sc, a) +} + +// readReference reads a reference path, kept in the render's hold for its group, so its draw +// spans the render. +func readReference(s *session, t *template, sc renderScope, a arm) readValue { + if sc.set.trace != nil { + traceRead(sc.set.trace, t, sc, a) } - if r, done := w.held.value[a.path]; done { + group := sc.groupHold() + return readHeld(s, t, &group.hold, &group.pins, sc, a) +} + +func readHeld(s *session, t *template, held *hold, pins *pinSet, sc renderScope, a arm) readValue { + if r, done := held.value[a.path]; done { return r } - leaf := drawPath(t.head(a.key), a.tail, a.key, &w) - if w.pins != nil { - sc = sc.at(leaf, w.pins) + leaf := drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, held: held, pins: pins, a: &a}) + if pins != nil { + sc = sc.at(leaf, pins) } r := renderLeaf(s, leaf, sc) - if w.held.value == nil { - w.held.value = map[string]readValue{} + if held.value == nil { + held.value = map[string]readValue{} } - w.held.value[a.path] = r + held.value[a.path] = r return r } @@ -84,7 +94,7 @@ func renderLeaf(s *session, n node, sc renderScope) readValue { return readValue{null: true} case *template: if leaf.link.readsColumn != nil { - return readField(s, leaf, nil, sc.in(leaf), leaf.link.readsColumn.a) + return readReference(s, leaf, sc.in(leaf), leaf.link.readsColumn.a) } } return readValue{text: render(s, n, sc)} diff --git a/todo.md b/todo.md index 703bb27..9540234 100644 --- a/todo.md +++ b/todo.md @@ -26,8 +26,6 @@ least modify. This round, in order: -- Stop `readField` dereferencing a nil `held`: `renderLeaf` passes nil, safe only - because a `readsColumn` arm is a reference path with a tail, which nothing there says. - Test that `disagreements` holds every key `disagreement` can build, and panic through `internalError` on a miss: a missing key is a nil func call today. - Name `printedNumber`'s `dp` cases, and say on `builtin` what `prints` and -- 2.52.0 From a9ec8b3632477f97b1948e8268280aa778a6db69 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:42:33 +0200 Subject: [PATCH 250/966] Test that readField reports a held arm with no hold as an internal error --- hold_test.go | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/hold_test.go b/hold_test.go index a2d4830..5121c13 100644 --- a/hold_test.go +++ b/hold_test.go @@ -760,12 +760,18 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { } } -func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) { - tm, ok := compiled(t, `{"format":"{w}","w":{"format":"{x}","x":"1"}}`).(*template) - if !ok { - t.Fatal("not a template") +func TestReadFieldPanicsOnAPathItCannotHold(t *testing.T) { + a := arm{spelling: "w.x", key: "w", tail: []string{"x"}, path: "w.x"} + for name, format := range map[string]string{ + "unheld arm with a path": `{"format":"{w}","w":{"format":"{x}","x":"1"}}`, + "held arm with no hold": `{"format":"{w.x}","w":{"format":"{x}","x":"1"}}`, + } { + tm, ok := compiled(t, format).(*template) + if !ok { + t.Fatal("not a template") + } + mustPanic(t, name, func() { + readField(engine(1).rand, tm, nil, renderScope{set: &holdSet{}}, a) + }) } - mustPanic(t, "unheld arm with a path", func() { - readField(engine(1).rand, tm, nil, renderScope{set: &holdSet{}}, arm{spelling: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) - }) } -- 2.52.0 From 58e26c6d6bb5f45c5bdc232e92e82ed1c3109967 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:43:11 +0200 Subject: [PATCH 251/966] Report a held arm with no hold as an internal error, and name readHeld as the hold --- doc.go | 2 +- hold.go | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/doc.go b/doc.go index b5f8528..efe7a17 100644 --- a/doc.go +++ b/doc.go @@ -24,7 +24,7 @@ package fejkdata // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. // - hold — keeping one draw of a name, so every route to it reads that value: -// `formatOps.held`, `hold`, `groupHold`, `readField`. +// `formatOps.held`, `hold`, `groupHold`, `readHeld`. // - draw group — reference draws held apart inside one render: // `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a // render reads it, `holdSet`, `renderScope`. diff --git a/hold.go b/hold.go index 07e4239..d275c7f 100644 --- a/hold.go +++ b/hold.go @@ -41,6 +41,9 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV } return readValue{text: render(s, t.head(a.key), sc)} } + if held == nil { + panic(internalError("%q reads a name the expansion holds, with no hold to keep it in", a.spelling)) + } return readHeld(s, t, held, nil, sc, a) } -- 2.52.0 From 3131ecf2d660f248274e37f5346aa24fb4122712 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:46:35 +0200 Subject: [PATCH 252/966] Say which held names the expansion keeps, and cut formatOps restatements --- format.go | 12 +++++------- hold.go | 8 ++++---- 2 files changed, 9 insertions(+), 11 deletions(-) diff --git a/format.go b/format.go index 5cca79d..42043b5 100644 --- a/format.go +++ b/format.go @@ -375,19 +375,17 @@ type op struct { operands []arm } -// formatOps is a compiled format: its ops, the size of its literal text (to size -// the render buffer), and the names drawn once per expansion. pathKeys maps each key -// a path starts from to the first such path; held is every such key plus the -// fields an operand reads; holder maps each held name to the first reader holding -// it, for error messages. The maps are nil when the format holds nothing, so data -// that holds nothing carries no render-time cost. +// formatOps is a compiled format: its ops, the size of its literal text, and the +// names it holds. pathKeys maps each key a path starts from to the first such path; +// held is every such key plus the fields an operand reads; holder maps each held +// name to the first reader holding it, for error messages. type formatOps struct { ops []op grow int pathKeys map[string]string held map[string]bool holder map[string]string - heldLocal bool // some held name is kept by the expansion itself rather than the render's hold + heldLocal bool // some held name is not a reference path, so expand makes a hold } func (c *formatOps) holdName(a arm, label string) { diff --git a/hold.go b/hold.go index d275c7f..579d0d8 100644 --- a/hold.go +++ b/hold.go @@ -24,10 +24,10 @@ type readValue struct { } // readField renders one arm of a token. A name the expansion holds — a level some -// token addresses by dotted path, or a field an operand reads — is drawn once and -// kept in held, so {place.postal-code} and {place.locality} read one row, either read -// twice gives one value, and a shown operand is the operand computed. Every other name -// is drawn afresh, so {word} {word} still draws twice. +// token addresses by a dotted path that is not a reference, or a field an operand +// reads — is drawn once and kept in held, so {place.postal-code} and {place.locality} +// read one row, either read twice gives one value, and a shown operand is the operand +// computed. Every other name is drawn afresh, so {word} {word} still draws twice. func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { if isRef(a.key) && len(a.tail) > 0 { return readReference(s, t, sc, a) -- 2.52.0 From 526b4c84fd87b93627690ae8f8bf6669d9e1cc29 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:51:03 +0200 Subject: [PATCH 253/966] Test that disagreements holds every key disagreement builds, and a miss panics as an internal error --- datatype_test.go | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/datatype_test.go b/datatype_test.go index 9ba95aa..b08aac0 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -220,3 +220,20 @@ func TestNullColumn(t *testing.T) { t.Errorf("List() = %v, want the null column row.gone, which Fake accepts", f.List()) } } + +func TestDisagreementsHoldEveryKeyDisagreementBuilds(t *testing.T) { + kinds := []itemKind{kindText, kindReads, kindDeclares} + for _, fix := range kinds { + for _, other := range kinds[fix:] { + if other == kindText { + continue + } + for _, proven := range []bool{false, true} { + if disagreementFix(disagreementKey{fix, other, proven}) == nil { + t.Errorf("disagreementFix(%v, %v, %v) = nil", fix, other, proven) + } + } + } + } + mustPanic(t, "two text items", func() { disagreementFix(disagreementKey{kindText, kindText, false}) }) +} -- 2.52.0 From c304dae30371240cf49e73088d80db73024cd578 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:51:44 +0200 Subject: [PATCH 254/966] Panic through internalError on a disagreement key with no fix --- datatype.go | 10 +++++++++- todo.md | 2 -- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/datatype.go b/datatype.go index ba5c678..f6d8b64 100644 --- a/datatype.go +++ b/datatype.go @@ -198,7 +198,15 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { d.fix, d.other, d.want = b, a, da } proven := (&valueProof{}).proveColumnItem(d.fix).not[d.want] == "" - return disagreements[disagreementKey{kindOf(d.fix), kindOf(d.other), proven}](d) + return disagreementFix(disagreementKey{kindOf(d.fix), kindOf(d.other), proven})(d) +} + +func disagreementFix(k disagreementKey) func(d itemPair) error { + fix, ok := disagreements[k] + if !ok { + panic(internalError("no disagreement names the fix for %+v", k)) + } + return fix } func holdOne(d itemPair) error { diff --git a/todo.md b/todo.md index 9540234..e53919f 100644 --- a/todo.md +++ b/todo.md @@ -26,8 +26,6 @@ least modify. This round, in order: -- Test that `disagreements` holds every key `disagreement` can build, and panic through - `internalError` on a miss: a missing key is a nil func call today. - Name `printedNumber`'s `dp` cases, and say on `builtin` what `prints` and `proveNumber` a new builtin sets: the extender copied `digits` by analogy. - Rename the names that mislead: `table.pinnedRow` is the row node, not a pinned row; -- 2.52.0 From e74457a4704740f7b1bd5ac6b308c9843a7f7ae2 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:54:31 +0200 Subject: [PATCH 255/966] Let disagreementFix alone judge a key the test builds --- datatype_test.go | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/datatype_test.go b/datatype_test.go index b08aac0..95206ee 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -229,9 +229,7 @@ func TestDisagreementsHoldEveryKeyDisagreementBuilds(t *testing.T) { continue } for _, proven := range []bool{false, true} { - if disagreementFix(disagreementKey{fix, other, proven}) == nil { - t.Errorf("disagreementFix(%v, %v, %v) = nil", fix, other, proven) - } + disagreementFix(disagreementKey{fix, other, proven}) } } } -- 2.52.0 From be9ab3efcfdd57ce9d4579f0165ff7549983e271 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 03:54:31 +0200 Subject: [PATCH 256/966] Panic through internalError on a disagreement key mapped to nil too --- datatype.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/datatype.go b/datatype.go index f6d8b64..d9e3424 100644 --- a/datatype.go +++ b/datatype.go @@ -202,8 +202,8 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { } func disagreementFix(k disagreementKey) func(d itemPair) error { - fix, ok := disagreements[k] - if !ok { + fix := disagreements[k] + if fix == nil { panic(internalError("no disagreement names the fix for %+v", k)) } return fix -- 2.52.0 From 0dfa615b54f3f0a929dc3d7a6608437294d4296a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:10:36 +0200 Subject: [PATCH 257/966] Name printedNumber's dp cases, and say what prints and proveNumber a builtin sets --- builtins.go | 4 +++- calc.go | 6 +++--- format.go | 10 +++++++--- todo.md | 2 -- value.go | 21 ++++++++++++++------- 5 files changed, 27 insertions(+), 16 deletions(-) diff --git a/builtins.go b/builtins.go index 31c3916..4523b11 100644 --- a/builtins.go +++ b/builtins.go @@ -16,6 +16,8 @@ import ( const ( maxLen = 1 << 20 maxDecimals = 1024 + // shortestDecimals prints the fewest digits that read back as the same float64. + shortestDecimals = -1 ) // builtins is the registry of {name(args)} functions. Derivations read the digits @@ -100,7 +102,7 @@ func chars(alphabet string) func([]string) callFn { const hexDigits = "0123456789abcdef" -// formatFloat prints v to dp decimals, -1 for the shortest form, and a zero unsigned. +// formatFloat prints v to dp decimals, or shortestDecimals, and a zero unsigned. func formatFloat(v float64, dp int) string { s := strconv.FormatFloat(v, 'f', dp, 64) if strings.HasPrefix(s, "-") && strings.Trim(s, "-0.") == "" { diff --git a/calc.go b/calc.go index 827fde1..3b5e497 100644 --- a/calc.go +++ b/calc.go @@ -187,7 +187,7 @@ func neverNumeric(n node) (text string, never bool) { // calcPrep parses the expression and decimals once, at compile time, and places each // operand name at the position expand will read it into. checkCalc proved both args -// valid, so no step here can fail; dp -1 prints the minimal form. +// valid, so no step here can fail. func calcPrep(args []string) callFn { expr, err := parseCalc(args[0]) if err != nil { // a nil AST would be a nil dereference per render, with no message @@ -204,12 +204,12 @@ func calcPrep(args []string) callFn { } } -// calcDecimals is a calc's decimals count, or -1 for the shortest form. +// calcDecimals is a calc's decimals count, or shortestDecimals where it names none. func calcDecimals(args []string) int { if len(args) == 2 { return atoi(args[1]) } - return -1 + return shortestDecimals } // indexVars replaces each operand name with its position in the values expand reads. diff --git a/format.go b/format.go index 42043b5..b448968 100644 --- a/format.go +++ b/format.go @@ -125,10 +125,14 @@ type builtin struct { // operands names the fields the call reads, which expand renders for it; nil // for a builtin that reads none. operands func(args []string) []string - // proveNumber proves what a call prints, token its body: the bounds of its number and the - // datatype of its text, prints or one inside it; nil for a builtin whose text is no number. + // proveNumber bounds the number a call's text reads as, token its body, and says which + // datatypes that text is not; set it where every render reads as a finite number, which + // makes the call a calc operand, and leave it nil otherwise. proveNumber func(token string, prints DataType, args []string) proven - prints DataType + // prints is the datatype every render's text is, which a typed column holds the call to: + // DataTypeString where it reads as a number but no column should type it, as digits' + // leading zeros; unset where proveNumber is nil. + prints DataType } // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false diff --git a/todo.md b/todo.md index e53919f..9f08fd1 100644 --- a/todo.md +++ b/todo.md @@ -26,8 +26,6 @@ least modify. This round, in order: -- Name `printedNumber`'s `dp` cases, and say on `builtin` what `prints` and - `proveNumber` a new builtin sets: the extender copied `digits` by analogy. - Rename the names that mislead: `table.pinnedRow` is the row node, not a pinned row; `table.category` is a bare name beside the dotted `category` of `resolveLink`; `checkRenders` renders nothing; `fieldAlternation` is a plain `{name}` too; and diff --git a/value.go b/value.go index 0c73593..5ec2bf6 100644 --- a/value.go +++ b/value.go @@ -261,16 +261,23 @@ func bounded(lo, hi float64, integral bool) proven { func magnitude(v proven) float64 { return math.Max(math.Abs(v.lo), math.Abs(v.hi)) } -// printedNumber is what a token printing v to dp decimals holds: an integer when whole -// and within int64, else a number. +// printedNumber is what a token printing v to dp decimals holds. func printedNumber(token string, v proven, dp int) proven { - if dp >= 0 { - half, _ := strconv.ParseFloat("5e-"+strconv.Itoa(dp+1), 64) - v = proven{lo: v.lo - half, hi: v.hi + half, nonZero: math.Max(0, v.nonZero-half), integral: v.integral || dp == 0} - } - if dp != 0 && !(dp < 0 && v.integral) { + if dp == shortestDecimals { + if v.integral { // a whole value prints with no point + return printedInteger(token, v) + } return printing(token, DataTypeNumber, v) } + half, _ := strconv.ParseFloat("5e-"+strconv.Itoa(dp+1), 64) + v = proven{lo: v.lo - half, hi: v.hi + half, nonZero: math.Max(0, v.nonZero-half), integral: v.integral || dp == 0} + if dp == 0 { + return printedInteger(token, v) + } + return printing(token, DataTypeNumber, v) +} + +func printedInteger(token string, v proven) proven { v = printing(token, DataTypeInteger, v) if !(magnitude(v) < math.MaxInt64) { v.not[DataTypeInteger] = fmt.Sprintf("{%s} is not proven within int64", token) -- 2.52.0 From 4d35ead2a4504f38d301c7867db74cac9e7cec7d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:12:52 +0200 Subject: [PATCH 258/966] Say prints is handed to proveNumber, and give shortestDecimals its own const --- builtins.go | 5 +++-- format.go | 6 +++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/builtins.go b/builtins.go index 4523b11..f4be6ea 100644 --- a/builtins.go +++ b/builtins.go @@ -16,10 +16,11 @@ import ( const ( maxLen = 1 << 20 maxDecimals = 1024 - // shortestDecimals prints the fewest digits that read back as the same float64. - shortestDecimals = -1 ) +// shortestDecimals prints the fewest digits that read back as the same float64. +const shortestDecimals = -1 + // builtins is the registry of {name(args)} functions. Derivations read the digits // emitted so far in the current expansion (place them after their payload); // samples read only the rng. A time-based id (uuid v7, ulid) draws its timestamp diff --git a/format.go b/format.go index b448968..cce7f09 100644 --- a/format.go +++ b/format.go @@ -129,9 +129,9 @@ type builtin struct { // datatypes that text is not; set it where every render reads as a finite number, which // makes the call a calc operand, and leave it nil otherwise. proveNumber func(token string, prints DataType, args []string) proven - // prints is the datatype every render's text is, which a typed column holds the call to: - // DataTypeString where it reads as a number but no column should type it, as digits' - // leading zeros; unset where proveNumber is nil. + // prints is the datatype a call's text is, handed to proveNumber: DataTypeString where it + // reads as a number no column should type, as digits' leading zeros; unset where + // proveNumber is nil. prints DataType } -- 2.52.0 From 29f3821a2157832cdbaa07d438d4a60d04272a68 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:20:28 +0200 Subject: [PATCH 259/966] Rename pinnedRow, table.category, checkRenders, fieldAlternation and referenceFence to what they are --- data.go | 16 ++++++++-------- drawfence.go | 8 ++++---- format.go | 12 ++++++------ holdset.go | 2 +- inline.go | 2 +- path.go | 6 +++--- pins.go | 2 +- readsurvey_test.go | 2 +- record.go | 2 +- reference.go | 2 +- render.go | 2 +- table.go | 22 +++++++++++----------- table_test.go | 2 +- value.go | 4 ++-- 14 files changed, 42 insertions(+), 42 deletions(-) diff --git a/data.go b/data.go index b370355..019d678 100644 --- a/data.go +++ b/data.go @@ -199,9 +199,9 @@ func inlineScope(n node, label string) nodeScope { // binding is one scope's way through bind, the one load pipeline; its fields are where // scopes differ. type binding struct { - scope nodeScope - link func() error - referenceFence func() error + scope nodeScope + link func() error + scopeFence func() error // Set by a caller that proves the columns against their Go types itself. typedByGo bool } @@ -216,7 +216,7 @@ func treeBinding(root map[string]node) binding { } return linkRefs(root) }, - referenceFence: func() error { return checkNoCycles(root) }, + scopeFence: func() error { return checkNoCycles(root) }, } } @@ -224,7 +224,7 @@ func (b binding) bind() error { if err := b.link(); err != nil { return err } - if err := b.referenceFence(); err != nil { + if err := b.scopeFence(); err != nil { return err } if !b.typedByGo { @@ -232,13 +232,13 @@ func (b binding) bind() error { return err } } - return checkRenders(b.scope) + return checkNodeFences(b.scope) } -// checkRenders runs the per-node fences over a scope, each over the whole scope +// checkNodeFences runs the per-node fences over a scope, each over the whole scope // before the next, so which of several broken nodes is reported does not depend on // the walk. Its walks terminate only where nothing renders itself. -func checkRenders(s nodeScope) error { +func checkNodeFences(s nodeScope) error { mem := renderCounts{} if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { return err diff --git a/drawfence.go b/drawfence.go index 078b1bf..9561589 100644 --- a/drawfence.go +++ b/drawfence.go @@ -105,7 +105,7 @@ func checkOwnFamily(t *template) error { return renderEdge{}, nil, false } if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.category, own.category) + return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.segment, own.segment) } return nil } @@ -252,13 +252,13 @@ func checkFamilyPair(a, b pathRead) error { if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { tail := x.a.tail if s.t != x.tr.headTable { - tail = append([]string{x.tr.headTable.category}, tail...) + tail = append([]string{x.tr.headTable.segment}, tail...) } return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) + x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) } return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.category, y.at.route.spelled(y.a.spelling)) + x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling)) } } return nil diff --git a/format.go b/format.go index cce7f09..18d947b 100644 --- a/format.go +++ b/format.go @@ -64,14 +64,14 @@ type formatToken struct { body string // the braces' content, as written fn string // builtinCall args []string // builtinCall - names []string // fieldAlternation: the '|' arms; builtinCall: the operands its builtin reads + names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads } type tokenKind uint8 const ( builtinCall tokenKind = iota + 1 - fieldAlternation + nameRead literalRun ) @@ -93,7 +93,7 @@ func parseFormat(format string) ([]formatToken, error) { } flush() if indexOutside(t.body, '(') < 0 { - toks = append(toks, formatToken{kind: fieldAlternation, body: t.body, names: splitOutside(t.body, '|')}) + toks = append(toks, formatToken{kind: nameRead, body: t.body, names: splitOutside(t.body, '|')}) return nil } name, args, ok := funcCall(t.body) @@ -220,7 +220,7 @@ func checkToken(t formatToken, fields map[string]node) error { switch t.kind { case builtinCall: return checkFunc(t, fields) - case fieldAlternation: + case nameRead: for _, name := range t.names { if isRef(name) { if _, _, err := refShape(name); err != nil { @@ -372,7 +372,7 @@ type callFn func(s *session, emitted string, operands []string) string // re-scans the format. type op struct { formatToken - arms []arm // fieldAlternation: the '|' alternatives, split into key and path once + arms []arm // nameRead: the '|' alternatives, split into key and path once call callFn // operands are the fields the builtin reads, in the order its operands func // fixed; expand reads them before the call. nil for a builtin that reads none. @@ -446,7 +446,7 @@ func compileOps(toks []formatToken, refs map[string]refBinding) formatOps { c.ops = append(c.ops, op{formatToken: tok}) case builtinCall: c.function(tok, refs) - case fieldAlternation: + case nameRead: c.field(tok, refs) } } diff --git a/holdset.go b/holdset.go index 37080ff..0b2bdd9 100644 --- a/holdset.go +++ b/holdset.go @@ -58,7 +58,7 @@ func (sc renderScope) at(n node, pins *pinSet) renderScope { // rowOf is the row of t its columns render from. func (sc renderScope) rowOf(t *table) int { if sc.row.t != t { - panic(internalError("a column of %s renders in a scope holding no row of it", t.category)) + panic(internalError("a column of %s renders in a scope holding no row of it", t.segment)) } return sc.row.row } diff --git a/inline.go b/inline.go index 91749e0..06ca0c4 100644 --- a/inline.go +++ b/inline.go @@ -151,7 +151,7 @@ func inlineBinding(n node, label string, root map[string]node) binding { return binding{ scope: scope, link: func() error { return linkNodeRefs(scope, root) }, - referenceFence: func() error { + scopeFence: func() error { if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) } diff --git a/path.go b/path.go index 2e015cb..9d2d791 100644 --- a/path.go +++ b/path.go @@ -176,7 +176,7 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { case len(tail) == 0 && sel == "" && !descended: r.next = t case len(tail) == 0: - r.next = t.pinnedRow + r.next = t.rowNode case child != nil: r.next, r.rest, r.descends = child, tail[1:], true default: @@ -415,7 +415,7 @@ func (t *table) selector(tail []string) (sel string, rest []string, err error) { } sel, rest = selectorOf(tail[0]), tail[1:] if len(rest) > 0 && isSelector(rest[0]) { - return "", nil, fmt.Errorf("%s[%s] is selected twice; one selector names its row", t.category, sel) + return "", nil, fmt.Errorf("%s[%s] is selected twice; one selector names its row", t.segment, sel) } return sel, rest, nil } @@ -429,7 +429,7 @@ func (t *table) step(tail []string) (column node, child *table, err error) { return t.fields[t.header[i]], nil, nil } if child = t.descendant(tail[0]); child == nil { - return nil, nil, fmt.Errorf("no column or linked table %q in %s", tail[0], t.category) + return nil, nil, fmt.Errorf("no column or linked table %q in %s", tail[0], t.segment) } return nil, child, nil } diff --git a/pins.go b/pins.go index 5866e3b..498ffc6 100644 --- a/pins.go +++ b/pins.go @@ -32,7 +32,7 @@ func (p *pinSet) pinned(t *table) (int, bool) { func (p *pinSet) mustRow(t *table) int { r, ok := p.pinned(t) if !ok { - panic(internalError("a column of %s is rendered with no row pinned", t.category)) + panic(internalError("a column of %s is rendered with no row pinned", t.segment)) } return r } diff --git a/readsurvey_test.go b/readsurvey_test.go index 07d6186..bcf71e2 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -87,7 +87,7 @@ func renderRoot(s *session, t *template, trace renderTrace) { set := holdSet{trace: trace} sc := renderScope{set: &set} sc.groupHold().pins.pin(tbl, r) - render(s, tbl.pinnedRow, sc.at(tbl.pinnedRow, &sc.groupHold().pins)) + render(s, tbl.rowNode, sc.at(tbl.rowNode, &sc.groupHold().pins)) } case t.repeat > 1: expand(s, t, sc) diff --git a/record.go b/record.go index 4b4c666..b69b31f 100644 --- a/record.go +++ b/record.go @@ -264,7 +264,7 @@ func recordOf(n node) (*template, []Column, error) { func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.site.row == formatRow { - sc = sc.at(t.site.table.pinnedRow, &sc.groupHold().pins) + sc = sc.at(t.site.table.rowNode, &sc.groupHold().pins) } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { diff --git a/reference.go b/reference.go index e218229..668b1db 100644 --- a/reference.go +++ b/reference.go @@ -155,7 +155,7 @@ func columnReadOf(t *template, link templateLink) *columnRead { // loneRef is the reference a format of one reference token and nothing else reads. func loneRef(toks []formatToken) (string, bool) { - if len(toks) != 1 || toks[0].kind != fieldAlternation || len(toks[0].names) != 1 || !isRef(toks[0].names[0]) { + if len(toks) != 1 || toks[0].kind != nameRead || len(toks[0].names) != 1 || !isRef(toks[0].names[0]) { return "", false } return toks[0].names[0], true diff --git a/render.go b/render.go index 0ac5970..3a33a6f 100644 --- a/render.go +++ b/render.go @@ -121,7 +121,7 @@ func expand(s *session, t *template, sc renderScope) string { switch o.kind { case literalRun: b.WriteString(o.lit) - case fieldAlternation: + case nameRead: b.WriteString(readField(s, t, held, sc, o.arms[s.IntN(len(o.arms))]).text) case builtinCall: // Read before the call, so the value a calc computes is the value the diff --git a/table.go b/table.go index 0d41a73..a0da290 100644 --- a/table.go +++ b/table.go @@ -13,14 +13,14 @@ import ( // columns, each row is one draw, and the format renders the drawn row. A column is a // cell of the row a render pinned; a cell carrying tokens compiles to a string node. type table struct { - category string + segment string // path's last segment path string // the category's path from the data root, which a selector is written at file string formatTemplate *template // fields are the column nodes header []string col map[string]int fields map[string]node // column nodes, the format's fields - pinnedRow *tableRow + rowNode *tableRow cells []string // rows × columns, flat cellTemplates map[int]*template keyIndex int // keyIndex through parentIndex are column indexes, -1 where the option is absent @@ -84,7 +84,7 @@ func isTableOption(name string) bool { const inSelector = `[]{}"|` // compileTable compiles a category object naming a rows file into a table. -func compileTable(m map[string]any, category string, files *categoryFiles) (*table, error) { +func compileTable(m map[string]any, segment string, files *categoryFiles) (*table, error) { o, err := readTableOptions(m) if err != nil { return nil, err @@ -93,7 +93,7 @@ func compileTable(m map[string]any, category string, files *categoryFiles) (*tab if err != nil { return nil, err } - t := &table{category: category, file: o.rows, keyIndex: -1, nameIndex: -1, weightIndex: -1, parentIndex: -1} + t := &table{segment: segment, file: o.rows, keyIndex: -1, nameIndex: -1, weightIndex: -1, parentIndex: -1} if err := t.parseRows(data); err != nil { return nil, fmt.Errorf("%s: %w", o.rows, err) } @@ -328,7 +328,7 @@ func (t *table) compileRowFormat(format string) error { return err } for _, tok := range toks { - if tok.kind != fieldAlternation { + if tok.kind != nameRead { continue } for _, name := range tok.names { @@ -342,7 +342,7 @@ func (t *table) compileRowFormat(format string) error { return err } t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} - t.pinnedRow = &tableRow{t} + t.rowNode = &tableRow{t} return nil } @@ -405,7 +405,7 @@ func (t *table) linkParent(path string, siblings map[string]node) error { var ancestors []*table for q, seen := p, map[*table]bool{t: true}; q != nil; q, _ = siblings[q.header[q.parentIndex]].(*table) { if seen[q] { - return fmt.Errorf("parent cycle: %s reaches itself through its parents", q.category) + return fmt.Errorf("parent cycle: %s reaches itself through its parents", q.segment) } seen[q] = true ancestors = append(ancestors, q) @@ -414,8 +414,8 @@ func (t *table) linkParent(path string, siblings map[string]node) error { } } for _, q := range ancestors { - if _, clash := q.col[t.category]; clash { - return fmt.Errorf("%q is named like a column of %q, its ancestor, so %s.%s could read either; rename one", t.category, q.category, q.category, t.category) + if _, clash := q.col[t.segment]; clash { + return fmt.Errorf("%q is named like a column of %q, its ancestor, so %s.%s could read either; rename one", t.segment, q.segment, q.segment, t.segment) } } linked := make(map[string]bool, p.rowCount()) @@ -435,7 +435,7 @@ func (t *table) linkParent(path string, siblings map[string]node) error { if p.children == nil { p.children = map[string]*table{} } - p.children[t.category] = t + p.children[t.segment] = t return nil } @@ -598,7 +598,7 @@ func (t *table) selectorSpelling(r int) string { case t.keyIndex >= 0: return t.path + "[" + t.cell(r, t.keyIndex) + "]" case t.nameIndex >= 0: - return t.parentT.selectorSpelling(t.parentRow(r)) + "." + t.category + "[" + t.cell(r, t.nameIndex) + "]" + return t.parentT.selectorSpelling(t.parentRow(r)) + "." + t.segment + "[" + t.cell(r, t.nameIndex) + "]" } return fmt.Sprintf("%s line %d", t.file, r+2) } diff --git a/table_test.go b/table_test.go index c05b2ee..fc5a85f 100644 --- a/table_test.go +++ b/table_test.go @@ -1088,7 +1088,7 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { func categoriesOf(tables map[*table]bool) []string { var out []string for tbl := range tables { - out = append(out, tbl.category) + out = append(out, tbl.segment) } slices.Sort(out) return out diff --git a/value.go b/value.go index 5ec2bf6..243d0bd 100644 --- a/value.go +++ b/value.go @@ -90,7 +90,7 @@ func (p *valueProof) prove(n node) proven { case *tableColumn: v = p.proveCells(n) case *tableRow: - v = unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", n.t.formatTemplate.format, n.t.category)) + v = unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", n.t.formatTemplate.format, n.t.segment)) default: v = unproven(`it reads a null, which renders "" outside its own column`) } @@ -157,7 +157,7 @@ func (p *valueProof) proveTemplate(t *template) proven { o := t.compiled.ops[0] body, name, args := o.body, o.fn, o.args switch _, isTransform := transforms[name]; { - case o.kind == fieldAlternation: + case o.kind == nameRead: var leaves []node for _, a := range o.arms { leaves = append(leaves, pathLeaves(t.head(a.key), a.tail)...) -- 2.52.0 From f0992e7151d257202b7de8c66d17ffcfd0f02344 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:22:39 +0200 Subject: [PATCH 260/966] Rename categoriesOf to segmentsOf --- table_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/table_test.go b/table_test.go index fc5a85f..c2d5749 100644 --- a/table_test.go +++ b/table_test.go @@ -1079,13 +1079,13 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { } }) if !maps.Equal(drawn, want) { - t.Fatalf("%s: the probe reports %v drawn, a draw pins %v", path, categoriesOf(drawn), categoriesOf(want)) + t.Fatalf("%s: the probe reports %v drawn, a draw pins %v", path, segmentsOf(drawn), segmentsOf(want)) } } } } -func categoriesOf(tables map[*table]bool) []string { +func segmentsOf(tables map[*table]bool) []string { var out []string for tbl := range tables { out = append(out, tbl.segment) -- 2.52.0 From fe2fea5da29fe23391cbb6097d9245e5ecf5679a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:22:39 +0200 Subject: [PATCH 261/966] Drop the last field-alternation wording, say checkNodeFences holds the fences every binding needs, and tick the rename off --- data.go | 6 +++--- format.go | 2 +- todo.md | 4 ---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/data.go b/data.go index 019d678..c3fd37a 100644 --- a/data.go +++ b/data.go @@ -235,9 +235,9 @@ func (b binding) bind() error { return checkNodeFences(b.scope) } -// checkNodeFences runs the per-node fences over a scope, each over the whole scope -// before the next, so which of several broken nodes is reported does not depend on -// the walk. Its walks terminate only where nothing renders itself. +// checkNodeFences runs the per-node fences every binding needs over a scope, each +// over the whole scope before the next, so which of several broken nodes is reported +// does not depend on the walk. Its walks terminate only where nothing renders itself. func checkNodeFences(s nodeScope) error { mem := renderCounts{} if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { diff --git a/format.go b/format.go index 18d947b..133bb7b 100644 --- a/format.go +++ b/format.go @@ -367,7 +367,7 @@ func checkSegments(a arm) error { // values of the operands it named, which expand read for it. type callFn func(s *session, emitted string, operands []string) string -// op is one compiled unit of a format string: a literal run, a field alternation, +// op is one compiled unit of a format string: a literal run, a name read, // or a builtin already prepared with its args. compile builds these so render never // re-scans the format. type op struct { diff --git a/todo.md b/todo.md index 9f08fd1..052f2b3 100644 --- a/todo.md +++ b/todo.md @@ -26,10 +26,6 @@ least modify. This round, in order: -- Rename the names that mislead: `table.pinnedRow` is the row node, not a pinned row; - `table.category` is a bare name beside the dotted `category` of `resolveLink`; - `checkRenders` renders nothing; `fieldAlternation` is a plain `{name}` too; and - `referenceFence` is the cycle check for a tree and a drawGroup check inline. - Correct the false comments: `node` says rendering never re-sums weights, which `builtLookup` does on first draw; `Generator` claims immutable after `New`, while `Fake` sets `root.children` each call, which belongs in `New`; `renders` credits -- 2.52.0 From 3047627715f998d5662022fcacc4fc35d3897111 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:25:07 +0200 Subject: [PATCH 262/966] Call the body funcCall refuses a name-read body --- format.go | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/format.go b/format.go index 133bb7b..f232179 100644 --- a/format.go +++ b/format.go @@ -136,8 +136,7 @@ type builtin struct { } // funcCall splits a "{token}" body shaped name(args) into its parts; ok is false -// for a plain field or alternation body. A '(' without a trailing ')' yields -// ok=false. +// for a name-read body. A '(' without a trailing ')' yields ok=false. func funcCall(body string) (name string, args []string, ok bool) { lp := indexOutside(body, '(') if lp < 0 || !strings.HasSuffix(body, ")") { -- 2.52.0 From d519a61f581546abf31957fb7114e8fa052d5a63 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:25:49 +0200 Subject: [PATCH 263/966] Keep Fake as the one example of a shared name --- docs/decisions.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index cb62999..22ffd7f 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -311,9 +311,8 @@ a read drew. Two declarations of any kind — function, method, field, type — share a name only where one definition covers both: the same kind of value, counted or addressed the same way, so a -reader landing on the wrong one concludes nothing false. `Fake` renders one value on -`Generator`, `Template` and `RecordTemplate`; `renderScope.groupHold` returns the -`*groupHold` it names. `TestNoFunctionSpellsAMethod` keeps a bare function off a method's +reader landing on the wrong one concludes nothing false, as `Fake` renders one value on +`Generator`, `Template` and `RecordTemplate`. `TestNoFunctionSpellsAMethod` keeps a bare function off a method's name, since a search returns both and a call site shows no receiver; review judges the rest. Serves goal 2: names that tell the truth, and reaching the unit behind a symptom without asking a person. Decided 2026-09-25 by Lilleman; valid while goal 2 counts reading -- 2.52.0 From c422c9e436bb737dc10dc00ec9ba41953053312d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:34:50 +0200 Subject: [PATCH 264/966] Build the walk root in New, drop the categories copy, and correct the node and renders comments --- drawfence_test.go | 6 +++--- fejkdata.go | 17 ++++++++--------- holdfence.go | 5 ++--- inline.go | 2 +- node.go | 2 +- path_test.go | 6 +++--- record.go | 2 +- render.go | 1 - shape_test.go | 4 ++-- struct.go | 2 +- table_test.go | 4 ++-- todo.md | 4 ---- 12 files changed, 24 insertions(+), 31 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index f690f1b..83ffe6f 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -28,7 +28,7 @@ func fenceRoots(t *testing.T, f *Generator) ([]fenceRoot, map[string]*table) { t.Helper() var roots []fenceRoot tables := map[string]*table{} - _ = walkNodes(f.categories, func(path string, n node) error { + _ = walkNodes(f.root.children, func(path string, n node) error { switch n := n.(type) { case *template: roots = append(roots, fenceRoot{path, n}) @@ -61,7 +61,7 @@ func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { if err != nil { t.Fatalf("New() = %v", err) } - _ = walkNodes(shipped.categories, func(path string, n node) error { + _ = walkNodes(shipped.root.children, func(path string, n node) error { if tm, ok := n.(*template); ok { roots = append(roots, fenceRoot{path, tm}) } @@ -85,7 +85,7 @@ func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { t.Fatalf("New = %v", err) } var reads []pathRead - _ = walkNodes(f.categories, func(_ string, n node) error { + _ = walkNodes(f.root.children, func(_ string, n node) error { tb, isTable := n.(*table) if !isTable || tb.keyIndex < 0 || tb.familyRoot().path != "region" { return nil diff --git a/fejkdata.go b/fejkdata.go index 21a75cc..5c2c232 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -31,12 +31,11 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD // from one goroutine. The compiled tree is immutable after [New], so List, NewTemplate // and NewRecordTemplate read it without a lock; mu guards the rest, which every render takes. type Generator struct { - mu sync.Mutex - rand *session - categories map[string]node - root folder // the categories as the node a path walks from, owned here so a walk allocates none - records map[node]recordShape - structs map[reflect.Type]structResult + mu sync.Mutex + rand *session + root folder // the categories as the node a path walks from, owned here so a walk allocates none + records map[node]recordShape + structs map[reflect.Type]structResult } // session is one generator's mutable render state: the seeded rng plus the {seq()} @@ -112,7 +111,7 @@ func New(opts ...Option) (*Generator, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - return &Generator{rand: rng, categories: cats}, nil + return &Generator{rand: rng, root: folder{children: cats}}, nil } // List returns the sorted dotted paths Fake renders: each category and every field, @@ -120,8 +119,8 @@ func New(opts ...Option) (*Generator, error) { // segment, so a path continues through one only where every variant carries it. func (f *Generator) List() []string { var out []string - for _, name := range sortedNames(f.categories) { - for _, p := range paths(f.categories[name]) { + for _, name := range sortedNames(f.root.children) { + for _, p := range paths(f.root.children[name]) { out = append(out, join(name, p)) } } diff --git a/holdfence.go b/holdfence.go index 53a54ef..627a79e 100644 --- a/holdfence.go +++ b/holdfence.go @@ -152,9 +152,8 @@ func isFixed(n node) bool { } // renders reports whether rendering n can reach anything in want, following the -// same edges expand does. seen keeps a node shared by several routes from being -// walked twice; checkNoCycles has already proved the graph is a DAG, so the walk -// ends. +// same edges expand does. seen keeps a node from being walked twice, so the walk +// ends across a cycle too. func renders(n node, want, seen map[node]bool) bool { if want[n] { return true diff --git a/inline.go b/inline.go index 06ca0c4..b090ecd 100644 --- a/inline.go +++ b/inline.go @@ -32,7 +32,7 @@ func (f *Generator) NewTemplate(input string) (*Template, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - if err := inlineBinding(n, "template", f.categories).bind(); err != nil { + if err := inlineBinding(n, "template", f.root.children).bind(); err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } return &Template{g: f, n: n}, nil diff --git a/node.go b/node.go index 46c751c..56bd89a 100644 --- a/node.go +++ b/node.go @@ -11,7 +11,7 @@ import ( // node is a compiled element of the namespace tree: a folder, choice, null, // template, table, column or row. Compiling JSON into these once (see compile) -// means rendering never re-inspects the raw JSON or re-sums weights. A new kind needs a +// means rendering never re-inspects the raw JSON. A new kind needs a // case in contained, paths, render and renderEdges; each panics on a kind it does not list. type node interface{ isNode() } diff --git a/path_test.go b/path_test.go index cd11e10..fde337b 100644 --- a/path_test.go +++ b/path_test.go @@ -50,7 +50,7 @@ func TestDeepDottedPath(t *testing.T) { // A 5-segment path descends through alternating object/array nodes; choices // on the path are single-variant, so it resolves deterministically. f := engine(1) - f.categories = map[string]node{ + f.root.children = map[string]node{ "deep": compiled(t, `{"format":"{a}","a":{"format":"{b}","b":{"format":"{c}","c":{"format":"{d}","d":"leaf"}}}}`), } if got, err := f.Fake("deep.a.b.c.d"); err != nil || got != "leaf" { @@ -64,7 +64,7 @@ func TestDeepDottedPath(t *testing.T) { func TestDescendIntoStringErrors(t *testing.T) { f := engine(1) - f.categories = map[string]node{"greeting": compiled(t, `"hej"`)} + f.root.children = map[string]node{"greeting": compiled(t, `"hej"`)} if _, err := f.Fake("greeting.extra"); err == nil || !strings.Contains(err.Error(), `no field "extra"`) { t.Fatalf("Fake(greeting.extra) = %v, want a no-field error", err) } @@ -153,7 +153,7 @@ func TestBindingKeyIsNotAPathSegment(t *testing.T) { func TestFakePathNavigation(t *testing.T) { f := engine(1) - f.categories = map[string]node{ + f.root.children = map[string]node{ "addr": compiled(t, `{"format":"{street}","street":"Main"}`), } if got, err := f.Fake("addr"); err != nil || !strings.Contains(got, "Main") { diff --git a/record.go b/record.go index b69b31f..0298571 100644 --- a/record.go +++ b/record.go @@ -126,7 +126,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - _, n, tail, err := resolveCategory(f.categories, segments) + _, n, tail, err := resolveCategory(f.root.children, segments) if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) } diff --git a/render.go b/render.go index 3a33a6f..efc37dc 100644 --- a/render.go +++ b/render.go @@ -25,7 +25,6 @@ func (f *Generator) Fake(path string) (string, error) { } var set holdSet sc := renderScope{set: &set} - f.root.children = f.categories n, err := descend(f.rand, &f.root, segments, sc) if err != nil { return "", fmt.Errorf("fejkdata: %s: %w", path, err) diff --git a/shape_test.go b/shape_test.go index bc72377..010b2fe 100644 --- a/shape_test.go +++ b/shape_test.go @@ -61,8 +61,8 @@ func shippedShape(f *Generator) string { } } } - for _, name := range sortedNames(f.categories) { - walk(name, f.categories[name]) + for _, name := range sortedNames(f.root.children) { + walk(name, f.root.children[name]) } var b strings.Builder for _, p := range f.List() { diff --git a/struct.go b/struct.go index a6f3df1..8f6b82a 100644 --- a/struct.go +++ b/struct.go @@ -49,7 +49,7 @@ func (f *Generator) structShapeOf(t reflect.Type) (*structShape, error) { if label == "" { label = "struct" } - sc := &structCompile{categories: f.categories, visiting: map[reflect.Type]bool{}, structsLeft: maxStructs} + sc := &structCompile{categories: f.root.children, visiting: map[reflect.Type]bool{}, structsLeft: maxStructs} shape, err := sc.compileShape(t, label) if err == nil && shape.empty() { err = fmt.Errorf("%s has no fake tags, so nothing to fill", t) diff --git a/table_test.go b/table_test.go index c2d5749..3bb3a3f 100644 --- a/table_test.go +++ b/table_test.go @@ -991,7 +991,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) var tables []*table for _, category := range []string{"region", "municipality", "locality"} { - tbl, isTable := f.categories[category].(*table) + tbl, isTable := f.root.children[category].(*table) if !isTable { t.Fatalf("%s is not a table", category) } @@ -1063,7 +1063,7 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { if err != nil { t.Fatal(err) } - head := f.categories[segs[0]].(*table) + head := f.root.children[segs[0]].(*table) probe := &pathProbe{drawn: map[*table]bool{}} if _, err := probe.walk(head, segs[1:]); err != nil { t.Fatalf("%s: %v", path, err) diff --git a/todo.md b/todo.md index 052f2b3..d813adc 100644 --- a/todo.md +++ b/todo.md @@ -26,10 +26,6 @@ least modify. This round, in order: -- Correct the false comments: `node` says rendering never re-sums weights, which - `builtLookup` does on first draw; `Generator` claims immutable after `New`, while - `Fake` sets `root.children` each call, which belongs in `New`; `renders` credits - `checkNoCycles`, which an inline binding never runs. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 16604de74c528926ec9c48ea1baac6519e733705 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:37:19 +0200 Subject: [PATCH 265/966] Name the table lookup the Generator doc's fixed tree excepts, and drop the renders restatement --- fejkdata.go | 5 +++-- holdfence.go | 3 +-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index 5c2c232..8fa244a 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -28,8 +28,9 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD // Generator generates fake data from a loaded namespace tree. Create one with [New]. // It is safe for concurrent use; a seeded sequence is reproducible only when drawn -// from one goroutine. The compiled tree is immutable after [New], so List, NewTemplate -// and NewRecordTemplate read it without a lock; mu guards the rest, which every render takes. +// from one goroutine. The compiled tree is fixed after [New] bar each table's row lookup, +// which its sync.Once builds on first use, so List, NewTemplate and NewRecordTemplate +// read it without a lock; mu guards the rest, which every render takes. type Generator struct { mu sync.Mutex rand *session diff --git a/holdfence.go b/holdfence.go index 627a79e..9343523 100644 --- a/holdfence.go +++ b/holdfence.go @@ -152,8 +152,7 @@ func isFixed(n node) bool { } // renders reports whether rendering n can reach anything in want, following the -// same edges expand does. seen keeps a node from being walked twice, so the walk -// ends across a cycle too. +// same edges expand does. func renders(n node, want, seen map[node]bool) bool { if want[n] { return true -- 2.52.0 From 6bd2b155d2e4a40a28ab9f27807deda3c29691f0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:40:07 +0200 Subject: [PATCH 266/966] Move the Generator's lock-free-reader invariant from its godoc onto mu --- fejkdata.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index 8fa244a..d913c5f 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -28,10 +28,11 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD // Generator generates fake data from a loaded namespace tree. Create one with [New]. // It is safe for concurrent use; a seeded sequence is reproducible only when drawn -// from one goroutine. The compiled tree is fixed after [New] bar each table's row lookup, -// which its sync.Once builds on first use, so List, NewTemplate and NewRecordTemplate -// read it without a lock; mu guards the rest, which every render takes. +// from one goroutine. type Generator struct { + // mu guards what a render touches. The tree under root is fixed after New bar each + // table's row lookup, built under its sync.Once, so List, NewTemplate and + // NewRecordTemplate read it without mu. mu sync.Mutex rand *session root folder // the categories as the node a path walks from, owned here so a walk allocates none -- 2.52.0 From 83f8c4549365fba0233e79fae13ac23bf2049e4a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 04:42:36 +0200 Subject: [PATCH 267/966] Name the fields mu guards --- fejkdata.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fejkdata.go b/fejkdata.go index d913c5f..d13a7be 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -30,7 +30,7 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD // It is safe for concurrent use; a seeded sequence is reproducible only when drawn // from one goroutine. type Generator struct { - // mu guards what a render touches. The tree under root is fixed after New bar each + // mu guards rand, records and structs. The tree under root is fixed after New bar each // table's row lookup, built under its sync.Once, so List, NewTemplate and // NewRecordTemplate read it without mu. mu sync.Mutex -- 2.52.0 From 5b313a44bb50f2143ca5c98426597d0d0ab2ed96 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 08:43:37 +0200 Subject: [PATCH 268/966] Plan the promise that a Generator's renders run one at a time --- todo.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/todo.md b/todo.md index d813adc..6866133 100644 --- a/todo.md +++ b/todo.md @@ -248,6 +248,8 @@ by a `data-import/` script, as README goal 12 asks. ## v0.2.0 +- **Promise in `Generator`'s godoc that its renders run one at a time**, so a caller + wanting parallel throughput makes one generator per goroutine. - Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`: every US place of 10,000 and more streets per locality, built by `--min-population` and `--streets-per-locality`. The default embed stays under ~1 MB -- 2.52.0 From d8887e0ea12ca78c46fb18dc4cc174bf355a460f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 08:58:19 +0200 Subject: [PATCH 269/966] File the second 2026-09-29 nine-seat panel's findings, scored 5.7 --- todo.md | 41 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 5 deletions(-) diff --git a/todo.md b/todo.md index 6866133..858e485 100644 --- a/todo.md +++ b/todo.md @@ -22,10 +22,39 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming `drawWalk.walk` and the four path walks among the hardest; 5.8 on 2026-09-29 (Navigation 6.8, Locality 5.3, Shape 5.8, Self-sufficiency 5.7), seven seats capped by Locality, all nine naming `drawWalk.walk` hardest and eight the unit they would -least modify. +least modify; 5.7 on 2026-09-29 again (Navigation 6.6, Locality 5.2, Shape 5.9, +Self-sufficiency 5.8), four seats capped by Locality, six of the seven unit seats +ranking `readSurvey.walk` hardest and all seven the unit they would least modify. This round, in order: +- Move `readSurvey`'s verdict — `check`, `checkFamilies`, `checkFamilyPair`, + `replayPairs`, `alternatives`, `conflict` and `overlapError` — from `drawfence.go` + into `readsurvey.go`: both architects looked for it beside the survey. +- Record each selector's table and spelling in `pathProbe.readRow`, and delete + `tableReadOf`'s second walk over the tail: it steps by `descendant` apart from + `route`, so a route that steps up to a parent row credits a selector to the wrong table. +- Move the render's read path — `readField`, `readReference`, `readHeld`, `traceRead`, + `renderLeaf`, `resolveChoice`, `renderOnce` and `expandAnew` — into `render.go`, and + merge what `hold.go` and `holdset.go` keep into one file: both architects followed one read + across three files. +- Give `prove`, `stepInto` and `columnItems` a case per node kind and panic through + `internalError` on the rest, and list every such switch in `node`'s doc: `prove`'s + default tells a typed column reading a bare table that it reads a null, and five seats + found `node` naming four switches of seven. +- Replace `disagreements`' ten keys with the rule they encode: a declared fix holds one + datatype, a read one is `readTyped` where proven and `readAsText` otherwise, a text one + is `otherAsText` beside an unproven read and `declare` otherwise. Five seats could not + see why each key gets its fix. +- Drop `table.fields` for `formatTemplate.fields`: one map held in two fields. +- Rename `arm.key` to `head`, as `template.head` reads it, leaving `key` the table's key + column; give `renderScope.row` a type that is not `tablePin`, since it pins nothing; + and name `value.go` by the `valueProof` it holds. +- Delete the prose three or more seats found empty: "kept whole on purpose" + (`format.go`, `calc.go`), the second copy of the concrete-session rationale + (`table.drawRow`, `pick`), the doc comments restating `isSelector`, `selectorOf`, + `checkPath`, `checkColumnDraws`, `compileTable` and `isOption`, and the `compose.yaml` + and `Dockerfile` preambles restating the README's Development. - Run the nine-seat panel again and file what it names here. ### Data @@ -203,8 +232,8 @@ by a `data-import/` script, as README goal 12 asks. first in Go's map order. - Report a CRLF rows file holding only its header as having no rows: `parseRows` answers "line 2 is empty". -- Refuse a bare table read in a typed column with a reason naming the table: - `prove`'s default arm says it reads a null. +- Refuse a bare table read in a typed column with a reason naming the table, once + `prove` has a case per node kind. - Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. - Refuse two whole reads of one table family in one render, which goal 5 promises is a @@ -237,8 +266,10 @@ by a `data-import/` script, as README goal 12 asks. - Read the tag through `git/ref/tags/{tag}` in `publish_release.py`: GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. -- Document `NewRecordTemplate` in the README's Library section, the one exported name - it does not name. +- Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the + README's Library section, which names none of them. +- Build on a manual run of `test.yml`: `workflow_dispatch` leaves `github.event.before` + empty, so the diff compares `HEAD` with itself and skips `docker build`. - Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds: nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. -- 2.52.0 From 2f8d43194020d39bacbeb8718fa541131deedd65 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 08:58:41 +0200 Subject: [PATCH 270/966] Correct the seat counts --- todo.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/todo.md b/todo.md index 858e485..b9f9112 100644 --- a/todo.md +++ b/todo.md @@ -40,17 +40,17 @@ This round, in order: across three files. - Give `prove`, `stepInto` and `columnItems` a case per node kind and panic through `internalError` on the rest, and list every such switch in `node`'s doc: `prove`'s - default tells a typed column reading a bare table that it reads a null, and five seats - found `node` naming four switches of seven. + default tells a typed column reading a bare table that it reads a null, and six seats + found `node` naming four of them. - Replace `disagreements`' ten keys with the rule they encode: a declared fix holds one datatype, a read one is `readTyped` where proven and `readAsText` otherwise, a text one - is `otherAsText` beside an unproven read and `declare` otherwise. Five seats could not + is `otherAsText` beside an unproven read and `declare` otherwise. Six seats could not see why each key gets its fix. - Drop `table.fields` for `formatTemplate.fields`: one map held in two fields. - Rename `arm.key` to `head`, as `template.head` reads it, leaving `key` the table's key column; give `renderScope.row` a type that is not `tablePin`, since it pins nothing; and name `value.go` by the `valueProof` it holds. -- Delete the prose three or more seats found empty: "kept whole on purpose" +- Delete the prose the seats found empty: "kept whole on purpose" (`format.go`, `calc.go`), the second copy of the concrete-session rationale (`table.drawRow`, `pick`), the doc comments restating `isSelector`, `selectorOf`, `checkPath`, `checkColumnDraws`, `compileTable` and `isOption`, and the `compose.yaml` -- 2.52.0 From c78f9618cfd6ea68c1218de0eb99a35a01f6c829 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:02:02 +0200 Subject: [PATCH 271/966] Move readSurvey's verdict from drawfence.go into readsurvey.go --- README.md | 4 +- drawfence.go | 116 +------------------------------------------------- readsurvey.go | 111 +++++++++++++++++++++++++++++++++++++++++++++++ todo.md | 3 -- 4 files changed, 114 insertions(+), 120 deletions(-) diff --git a/README.md b/README.md index b7edb18..bbb09a1 100644 --- a/README.md +++ b/README.md @@ -886,8 +886,8 @@ format.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per held name, and reading a field through it holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways holdset.go one reference draw per render and group: the hold set a render reads its paths through -drawfence.go the load fences over reference draws: a drawGroup's, a render's, a record's columns and a table family's -readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, for the draw fences +drawfence.go the load fences over reference draws: a drawGroup's, a table's own family, and a render's and a record's columns through the survey +readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/drawfence.go b/drawfence.go index 9561589..6080817 100644 --- a/drawfence.go +++ b/drawfence.go @@ -1,10 +1,6 @@ package fejkdata -import ( - "fmt" - "sort" - "strings" -) +import "fmt" // checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, // short of a repeat or another draw group. @@ -193,113 +189,3 @@ func readsTable(n node, a arm) bool { // checkColumnDraws fences a record's columns as one render. func checkColumnDraws(t *template, columns []string) error { return surveyColumns(t, columns).check() } - -// check refuses what one draw per reference path cannot answer for: a read of a level beside a path -// another read takes into it, and two reads of one table family that select different rows. Reads -// are compared in path order, so which pair is reported does not vary. -func (s *readSurvey) check() error { - sort.SliceStable(s.reads, func(i, j int) bool { - if s.reads[i].at.group != s.reads[j].at.group { - return s.reads[i].at.group < s.reads[j].at.group - } - return s.reads[i].a.path < s.reads[j].a.path - }) - for i, level := range s.reads { - for _, into := range s.reads[i+1:] { - if into.at.group != level.at.group || alternatives(level.at, into.at) { - continue - } - if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { - return overlapError(level.at.route, level.a.spelling, into) - } - } - } - return checkFamilies(s.reads) -} - -// checkFamilies refuses reads of one table family in one draw group that cannot -// read one consistent draw: a table rendered whole beside a path into the family, -// a table one read draws that another pins, and two reads pinning different rows. -// The reads come sorted by group and path, so which pair is reported does not vary. -func checkFamilies(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[:i] { - if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { - continue - } - if err := checkFamilyPair(o, r); err != nil { - return err - } - } - } - return replayPairs(reads) -} - -// checkFamilyPair refuses a table read whole beside a path into its family, and a -// table one read draws that the other pins, since which token renders first would -// then decide the row. -// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkFamilyPair(a, b pathRead) error { - for _, pair := range [][2]pathRead{{a, b}, {b, a}} { - x, y := pair[0], pair[1] - if len(x.a.tail) == 0 && len(y.a.tail) > 0 { - return overlapError(x.at.route, x.a.spelling, y) - } - if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { - if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { - tail := x.a.tail - if s.t != x.tr.headTable { - tail = append([]string{x.tr.headTable.segment}, tail...) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling)) - } - } - return nil -} - -// replayPairs replays every two reads that can render together into one draws, the earlier -// read first. Pairs find every conflict a full replay would: clash judges a row against one -// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry -// its rows' ancestors, so the pair of those two reads clashes the same way. -func replayPairs(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[i+1:] { - if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { - continue - } - var d pinSet - if err := r.tr.replay(&d); err != nil { - return conflict(r, err) - } - if err := o.tr.replay(&d); err != nil { - return conflict(o, err) - } - } - } - return nil -} - -func conflict(r pathRead, err error) error { - return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) -} - -// alternatives reports whether two reads never render together: one read renders one row of a -// table, so reads under different rows the walks pinned, or under different rows of one whole -// draw, never meet. -func alternatives(a, b surveyAt) bool { - return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) -} - -func overlapError(route surveyRoute, ref string, into pathRead) error { - return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) -} diff --git a/readsurvey.go b/readsurvey.go index fb43df6..9490a3f 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -3,6 +3,7 @@ package fejkdata import ( "fmt" "sort" + "strings" ) // readSurvey gathers, at load, the references one render reads, by draw group, for check to compare. @@ -240,3 +241,113 @@ func (r *tableRead) replay(d *pinSet) error { }) return err } + +// check refuses what one draw per reference path cannot answer for: a read of a level beside a path +// another read takes into it, and two reads of one table family that select different rows. Reads +// are compared in path order, so which pair is reported does not vary. +func (s *readSurvey) check() error { + sort.SliceStable(s.reads, func(i, j int) bool { + if s.reads[i].at.group != s.reads[j].at.group { + return s.reads[i].at.group < s.reads[j].at.group + } + return s.reads[i].a.path < s.reads[j].a.path + }) + for i, level := range s.reads { + for _, into := range s.reads[i+1:] { + if into.at.group != level.at.group || alternatives(level.at, into.at) { + continue + } + if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { + return overlapError(level.at.route, level.a.spelling, into) + } + } + } + return checkFamilies(s.reads) +} + +// checkFamilies refuses reads of one table family in one draw group that cannot +// read one consistent draw: a table rendered whole beside a path into the family, +// a table one read draws that another pins, and two reads pinning different rows. +// The reads come sorted by group and path, so which pair is reported does not vary. +func checkFamilies(reads []pathRead) error { + for i, r := range reads { + if r.tr == nil { + continue + } + for _, o := range reads[:i] { + if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { + continue + } + if err := checkFamilyPair(o, r); err != nil { + return err + } + } + } + return replayPairs(reads) +} + +// checkFamilyPair refuses a table read whole beside a path into its family, and a +// table one read draws that the other pins, since which token renders first would +// then decide the row. +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held +func checkFamilyPair(a, b pathRead) error { + for _, pair := range [][2]pathRead{{a, b}, {b, a}} { + x, y := pair[0], pair[1] + if len(x.a.tail) == 0 && len(y.a.tail) > 0 { + return overlapError(x.at.route, x.a.spelling, y) + } + if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { + if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { + tail := x.a.tail + if s.t != x.tr.headTable { + tail = append([]string{x.tr.headTable.segment}, tail...) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", + x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", + x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling)) + } + } + return nil +} + +// replayPairs replays every two reads that can render together into one draws, the earlier +// read first. Pairs find every conflict a full replay would: clash judges a row against one +// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry +// its rows' ancestors, so the pair of those two reads clashes the same way. +func replayPairs(reads []pathRead) error { + for i, r := range reads { + if r.tr == nil { + continue + } + for _, o := range reads[i+1:] { + if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { + continue + } + var d pinSet + if err := r.tr.replay(&d); err != nil { + return conflict(r, err) + } + if err := o.tr.replay(&d); err != nil { + return conflict(o, err) + } + } + } + return nil +} + +func conflict(r pathRead, err error) error { + return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) +} + +// alternatives reports whether two reads never render together: one read renders one row of a +// table, so reads under different rows the walks pinned, or under different rows of one whole +// draw, never meet. +func alternatives(a, b surveyAt) bool { + return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) +} + +func overlapError(route surveyRoute, ref string, into pathRead) error { + return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) +} diff --git a/todo.md b/todo.md index b9f9112..8009fd1 100644 --- a/todo.md +++ b/todo.md @@ -28,9 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Move `readSurvey`'s verdict — `check`, `checkFamilies`, `checkFamilyPair`, - `replayPairs`, `alternatives`, `conflict` and `overlapError` — from `drawfence.go` - into `readsurvey.go`: both architects looked for it beside the survey. - Record each selector's table and spelling in `pathProbe.readRow`, and delete `tableReadOf`'s second walk over the tail: it steps by `descendant` apart from `route`, so a route that steps up to a parent row credits a selector to the wrong table. -- 2.52.0 From 2499e71809a448aeda20b912b7a56225469b06b0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:17:31 +0200 Subject: [PATCH 272/966] Assert the table each probed selector is credited to --- table_test.go | 52 +++++++++++++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 24 deletions(-) diff --git a/table_test.go b/table_test.go index 3bb3a3f..98dadc3 100644 --- a/table_test.go +++ b/table_test.go @@ -1042,36 +1042,40 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { } // TestProbeReportsTheTablesADrawPins holds the check mode to the render: the tables a probe says a -// path draws are those a draw of it pins beyond what its selectors pin. +// path draws are those a draw of it pins beyond what its selectors pin, and each selector is credited +// to the table it selects a row of. func TestProbeReportsTheTablesADrawPins(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) - for _, path := range []string{ - "locality.code", - "municipality.locality.name", - "municipality[1281].locality.name", - "municipality.locality[L4].name", - "region", - "region.municipality", - "region.municipality.name", - "region.locality.name", - "region[12].locality", - "region[12].locality.code", - "region[12].municipality.locality.code", - "region[12].name", + for _, c := range []struct{ path, sels string }{ + {"locality.code", ""}, + {"municipality.locality.name", ""}, + {"municipality[1281].locality.name", "municipality=municipality[1281]"}, + {"municipality.locality[L4].name", "locality=municipality.locality[L4]"}, + {"region", ""}, + {"region.municipality", ""}, + {"region.municipality.name", ""}, + {"region.locality.name", ""}, + {"region[12].locality", "region=region[12]"}, + {"region[12].locality.code", "region=region[12]"}, + {"region[12].locality[L4].name", "region=region[12] locality=region[12].locality[L4]"}, + {"region[12].municipality.locality.code", "region=region[12]"}, + {"region[12].municipality[1281].locality.code", "region=region[12] municipality=region[12].municipality[1281]"}, + {"region[12].name", "region=region[12]"}, } { - segs, err := splitPath(path) - if err != nil { - t.Fatal(err) + a := splitArm(c.path, nil) + head := f.root.children[a.key].(*table) + tr := tableReadOf(head, a, nil) + var sels []string + for _, s := range tr.sels { + sels = append(sels, s.t.segment+"="+s.spelling) } - head := f.root.children[segs[0]].(*table) - probe := &pathProbe{drawn: map[*table]bool{}} - if _, err := probe.walk(head, segs[1:]); err != nil { - t.Fatalf("%s: %v", path, err) + if got := strings.Join(sels, " "); got != c.sels { + t.Errorf("%s: the probe credits the selectors %q, want %q", c.path, got, c.sels) } - probed, drawn := probe.pins, probe.drawn + probed, drawn := tr.pins, tr.drawn for i := 0; i < 20; i++ { var pins pinSet - drawPath(head, segs[1:], segs[0], &pathDraw{s: f.rand, pins: &pins}) + drawPath(head, a.tail, a.key, &pathDraw{s: f.rand, pins: &pins}) want := map[*table]bool{} pins.each(func(tbl *table, _ int) { if _, selected := probed.pinned(tbl); !selected { @@ -1079,7 +1083,7 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { } }) if !maps.Equal(drawn, want) { - t.Fatalf("%s: the probe reports %v drawn, a draw pins %v", path, segmentsOf(drawn), segmentsOf(want)) + t.Fatalf("%s: the probe reports %v drawn, a draw pins %v", c.path, segmentsOf(drawn), segmentsOf(want)) } } } -- 2.52.0 From 2e553d29f7b3f0024430d4dac3de3b813c06e493 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:20:20 +0200 Subject: [PATCH 273/966] Record each selector's table and spelling in pathProbe.readRow --- path.go | 30 ++++++++++++++++++++---------- readsurvey.go | 17 +++-------------- render.go | 2 +- todo.md | 3 --- 4 files changed, 24 insertions(+), 28 deletions(-) diff --git a/path.go b/path.go index 9d2d791..cf62df0 100644 --- a/path.go +++ b/path.go @@ -289,10 +289,13 @@ func (w *pathCover) walk(n node, tail []string) (node, error) { // pathProbe proves a path resolves without drawing. Where pathCheck walks every // variant of a choice, it walks the first, which carriedByAll lets stand for all; at a -// table it selects as pathCheck does and marks in drawn the tables a draw would pin. +// table it selects as pathCheck does. Probing an arm, it records what the read takes: +// in drawn the tables a draw would pin, and in sels each selector's table and spelling. type pathProbe struct { pins pinSet + a *arm drawn map[*table]bool + sels []tableSel } func (w *pathProbe) walk(n node, tail []string) (node, error) { @@ -310,7 +313,7 @@ func (w *pathProbe) walk(n node, tail []string) (node, error) { if r, err = x.route(tail, descended); err != nil { return nil, err } - if err := w.readRow(x, r); err != nil { + if err := w.readRow(x, tail, r); err != nil { return nil, err } n, tail, descended = r.next, r.rest, r.descends @@ -324,15 +327,22 @@ func (w *pathProbe) walk(n node, tail []string) (node, error) { return n, nil } -func (w *pathProbe) readRow(t *table, r tableRoute) error { - if r.sel != "" { +func (w *pathProbe) readRow(t *table, tail []string, r tableRoute) error { + switch { + case r.sel != "": + if w.a != nil { + walked := len(w.a.tail) - len(tail) + 1 + written := w.a.spelling[:len(w.a.spelling)-len(joinSegments(w.a.tail))] + w.sels = append(w.sels, tableSel{t, written + joinSegments(w.a.tail[:walked])}) + } return w.pins.selectRow(t, r.sel) - } - if !r.draw { - return nil - } - for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { - w.drawn[t] = true + case r.draw && w.a != nil: + if w.drawn == nil { + w.drawn = map[*table]bool{} + } + for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { + w.drawn[t] = true + } } return nil } diff --git a/readsurvey.go b/readsurvey.go index 9490a3f..e2e94e6 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -191,21 +191,10 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { if !isTable { return nil } - probe := &pathProbe{drawn: map[*table]bool{}} + probe := &pathProbe{a: &a} _, _ = probe.walk(t, a.tail) - tr := &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn} - written := a.spelling[:len(a.spelling)-len(joinSegments(a.tail))] - cur := t - for i, seg := range a.tail { - switch d := cur.descendant(seg); { - case isSelector(seg): - tr.sels = append(tr.sels, tableSel{cur, written + joinSegments(a.tail[:i+1])}) - case d != nil: - cur = d - } - } - _, tr.landsRow = leaf.(*tableRow) - return tr + _, landsRow := leaf.(*tableRow) + return &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn, sels: probe.sels, landsRow: landsRow} } // selected is the selector in r on t, or on the nearest ancestor of t it selects. diff --git a/render.go b/render.go index efc37dc..a83b9ef 100644 --- a/render.go +++ b/render.go @@ -39,7 +39,7 @@ func (f *Generator) Fake(path string) (string, error) { // selects or draws. func descend(s *session, root node, segments []string, sc renderScope) (node, error) { // docs/decisions.md#a-path-is-walked-once-without-drawing-before-it-is-walked-for-real - if _, err := (&pathProbe{drawn: map[*table]bool{}}).walk(root, segments); err != nil { + if _, err := (&pathProbe{}).walk(root, segments); err != nil { return nil, err } return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.groupHold().pins}), nil diff --git a/todo.md b/todo.md index 8009fd1..98f9537 100644 --- a/todo.md +++ b/todo.md @@ -28,9 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Record each selector's table and spelling in `pathProbe.readRow`, and delete - `tableReadOf`'s second walk over the tail: it steps by `descendant` apart from - `route`, so a route that steps up to a parent row credits a selector to the wrong table. - Move the render's read path — `readField`, `readReference`, `readHeld`, `traceRead`, `renderLeaf`, `resolveChoice`, `renderOnce` and `expandAnew` — into `render.go`, and merge what `hold.go` and `holdset.go` keep into one file: both architects followed one read -- 2.52.0 From 9ab18c575a94dc9fcc7a6ca27540b0fd7a35b33a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:30:00 +0200 Subject: [PATCH 274/966] Move the render's read path into render.go and merge holdset.go into hold.go --- README.md | 5 +- hold.go | 136 ++++++++++++++++++++++------------------------------- holdset.go | 79 ------------------------------- render.go | 102 ++++++++++++++++++++++++++++++++++++++++ todo.md | 4 -- 5 files changed, 159 insertions(+), 167 deletions(-) delete mode 100644 holdset.go diff --git a/README.md b/README.md index bbb09a1..a3f55e2 100644 --- a/README.md +++ b/README.md @@ -878,14 +878,13 @@ fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation table.go tables: the rows TSV, its options and links, row selection and draws path.go the dotted-path walks with their selectors, and proving a path resolves -render.go Fake and the recursive renderer (choices, format strings, expansions) +render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make record.go records: Record, the JSON/CSV/SQL serializers, and their entry points struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold: one draw per held name, and reading a field through it +hold.go the hold, one draw per held name, and the hold set and scope a render reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways -holdset.go one reference draw per render and group: the hold set a render reads its paths through drawfence.go the load fences over reference draws: a drawGroup's, a table's own family, and a render's and a record's columns through the survey readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them pins.go the pin set: the table rows a render fixes, which the draw fences replay diff --git a/hold.go b/hold.go index 579d0d8..a97fb26 100644 --- a/hold.go +++ b/hold.go @@ -23,92 +23,66 @@ type readValue struct { null bool } -// readField renders one arm of a token. A name the expansion holds — a level some -// token addresses by a dotted path that is not a reference, or a field an operand -// reads — is drawn once and kept in held, so {place.postal-code} and {place.locality} -// read one row, either read twice gives one value, and a shown operand is the operand -// computed. Every other name is drawn afresh, so {word} {word} still draws twice. -func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { - if isRef(a.key) && len(a.tail) > 0 { - return readReference(s, t, sc, a) - } - if sc.set.trace != nil { - traceRead(sc.set.trace, t, sc, a) - } - if !t.compiled.held[a.key] { - if len(a.tail) > 0 { - panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) - } - return readValue{text: render(s, t.head(a.key), sc)} - } - if held == nil { - panic(internalError("%q reads a name the expansion holds, with no hold to keep it in", a.spelling)) - } - return readHeld(s, t, held, nil, sc, a) -} - -// readReference reads a reference path, kept in the render's hold for its group, so its draw -// spans the render. -func readReference(s *session, t *template, sc renderScope, a arm) readValue { - if sc.set.trace != nil { - traceRead(sc.set.trace, t, sc, a) - } - group := sc.groupHold() - return readHeld(s, t, &group.hold, &group.pins, sc, a) -} - -func readHeld(s *session, t *template, held *hold, pins *pinSet, sc renderScope, a arm) readValue { - if r, done := held.value[a.path]; done { - return r - } - leaf := drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, held: held, pins: pins, a: &a}) - if pins != nil { - sc = sc.at(leaf, pins) - } - r := renderLeaf(s, leaf, sc) - if held.value == nil { - held.value = map[string]readValue{} - } - held.value[a.path] = r - return r -} - // renderTrace is a test's view of the fields a render reads; nil outside a test. type renderTrace func(group, table string, row int, a arm) -func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { - var table string - row := t.site.row - if t.site.table != nil { - table = t.site.table.path - } - if row == formatRow { - row = sc.rowOf(t.site.table) - } - trace(strings.Clone(sc.group), strings.Clone(table), row, a) +// holdSet is one render's reference draws: the unnamed draw group's, and each named one's; and a +// test's trace of its reads. +type holdSet struct { + unnamed groupHold + named map[string]*groupHold + trace renderTrace } -// renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one -// reference alone whose read drew null. -func renderLeaf(s *session, n node, sc renderScope) readValue { - n = resolveChoice(s, n) - switch leaf := n.(type) { - case *nullItem: - return readValue{null: true} - case *template: - if leaf.link.readsColumn != nil { - return readReference(s, leaf, sc.in(leaf), leaf.link.readsColumn.a) +// renderScope is where a render reads its reference paths: a hold set, in the draw group of the +// template rendering; and row, the row of the table rendering, which its columns read. It passes +// by value, and what is read from it reaches a map key, an interface or a func value only as a +// copy; else sc, and with it every render's hold set, moves to the heap. +type renderScope struct { + set *holdSet + group string + row tablePin +} + +// in is the scope t renders in: its draw group where it names one, else its caller's. +func (sc renderScope) in(t *template) renderScope { + if t.link.drawGroupKey != "" { + sc.group = t.link.drawGroupKey + } + return sc +} + +// at is the scope n, the leaf of a path that pinned its rows in pins, renders in: where n is a +// table's row or column, at the row pins holds for that table. +func (sc renderScope) at(n node, pins *pinSet) renderScope { + switch n := n.(type) { + case *tableRow: + sc.row = tablePin{n.t, pins.mustRow(n.t)} + case *tableColumn: + sc.row = tablePin{n.t, pins.mustRow(n.t)} + } + return sc +} + +// rowOf is the row of t its columns render from. +func (sc renderScope) rowOf(t *table) int { + if sc.row.t != t { + panic(internalError("a column of %s renders in a scope holding no row of it", t.segment)) + } + return sc.row.row +} + +func (sc renderScope) groupHold() *groupHold { + if sc.group == "" { + return &sc.set.unnamed + } + d, drew := sc.set.named[sc.group] + if !drew { + if sc.set.named == nil { + sc.set.named = map[string]*groupHold{} } + d = &groupHold{} + sc.set.named[strings.Clone(sc.group)] = d } - return readValue{text: render(s, n, sc)} -} - -// resolveChoice resolves a choice to one variant, so a held head is a concrete node the -// rest of the expansion shares. Nested choices unwrap too: a draw is one value, not -// another set to pick from. -func resolveChoice(s *session, n node) node { - for c, ok := n.(*choice); ok; c, ok = n.(*choice) { - n = pick(s, c) - } - return n + return d } diff --git a/holdset.go b/holdset.go deleted file mode 100644 index 0b2bdd9..0000000 --- a/holdset.go +++ /dev/null @@ -1,79 +0,0 @@ -package fejkdata - -import "strings" - -// holdSet is one render's reference draws: the unnamed draw group's, and each named one's; and a -// test's trace of its reads. -type holdSet struct { - unnamed groupHold - named map[string]*groupHold - trace renderTrace -} - -// renderScope is where a render reads its reference paths: a hold set, in the draw group of the -// template rendering; and row, the row of the table rendering, which its columns read. It passes -// by value, and what is read from it reaches a map key, an interface or a func value only as a -// copy; else sc, and with it every render's hold set, moves to the heap. -type renderScope struct { - set *holdSet - group string - row tablePin -} - -// renderOnce renders n as one render, over a hold set of its own. -func renderOnce(s *session, n node) string { - var set holdSet - return render(s, n, renderScope{set: &set}) -} - -// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into -// render's loop, its hold set would move to the heap. -// -//go:noinline -func expandAnew(s *session, t *template) string { - var set holdSet - return expand(s, t, renderScope{set: &set}) -} - -// in is the scope t renders in: its draw group where it names one, else its caller's. -func (sc renderScope) in(t *template) renderScope { - if t.link.drawGroupKey != "" { - sc.group = t.link.drawGroupKey - } - return sc -} - -// at is the scope n, the leaf of a path that pinned its rows in pins, renders in: where n is a -// table's row or column, at the row pins holds for that table. -func (sc renderScope) at(n node, pins *pinSet) renderScope { - switch n := n.(type) { - case *tableRow: - sc.row = tablePin{n.t, pins.mustRow(n.t)} - case *tableColumn: - sc.row = tablePin{n.t, pins.mustRow(n.t)} - } - return sc -} - -// rowOf is the row of t its columns render from. -func (sc renderScope) rowOf(t *table) int { - if sc.row.t != t { - panic(internalError("a column of %s renders in a scope holding no row of it", t.segment)) - } - return sc.row.row -} - -func (sc renderScope) groupHold() *groupHold { - if sc.group == "" { - return &sc.set.unnamed - } - d, drew := sc.set.named[sc.group] - if !drew { - if sc.set.named == nil { - sc.set.named = map[string]*groupHold{} - } - d = &groupHold{} - sc.set.named[strings.Clone(sc.group)] = d - } - return d -} diff --git a/render.go b/render.go index a83b9ef..c2433e8 100644 --- a/render.go +++ b/render.go @@ -45,6 +45,12 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.groupHold().pins}), nil } +// renderOnce renders n as one render, over a hold set of its own. +func renderOnce(s *session, n node) string { + var set holdSet + return render(s, n, renderScope{set: &set}) +} + // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over a hold set of its own. @@ -88,6 +94,15 @@ func render(s *session, n node, sc renderScope) string { } } +// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into +// render's loop, its hold set would move to the heap. +// +//go:noinline +func expandAnew(s *session, t *template) string { + var set holdSet + return expand(s, t, renderScope{set: &set}) +} + // pick selects one item. Uniform choices are O(1); weighted choices are an // O(log n) search over precomputed cumulative weights. compile guarantees a // non-empty choice and a finite positive total, so the index is always in range. @@ -137,3 +152,90 @@ func expand(s *session, t *template, sc renderScope) string { } return b.String() } + +// readField renders one arm of a token. A name the expansion holds — a level some +// token addresses by a dotted path that is not a reference, or a field an operand +// reads — is drawn once and kept in held, so {place.postal-code} and {place.locality} +// read one row, either read twice gives one value, and a shown operand is the operand +// computed. Every other name is drawn afresh, so {word} {word} still draws twice. +func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { + if isRef(a.key) && len(a.tail) > 0 { + return readReference(s, t, sc, a) + } + if sc.set.trace != nil { + traceRead(sc.set.trace, t, sc, a) + } + if !t.compiled.held[a.key] { + if len(a.tail) > 0 { + panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) + } + return readValue{text: render(s, t.head(a.key), sc)} + } + if held == nil { + panic(internalError("%q reads a name the expansion holds, with no hold to keep it in", a.spelling)) + } + return readHeld(s, t, held, nil, sc, a) +} + +// readReference reads a reference path, kept in the render's hold for its group, so its draw +// spans the render. +func readReference(s *session, t *template, sc renderScope, a arm) readValue { + if sc.set.trace != nil { + traceRead(sc.set.trace, t, sc, a) + } + group := sc.groupHold() + return readHeld(s, t, &group.hold, &group.pins, sc, a) +} + +func readHeld(s *session, t *template, held *hold, pins *pinSet, sc renderScope, a arm) readValue { + if r, done := held.value[a.path]; done { + return r + } + leaf := drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, held: held, pins: pins, a: &a}) + if pins != nil { + sc = sc.at(leaf, pins) + } + r := renderLeaf(s, leaf, sc) + if held.value == nil { + held.value = map[string]readValue{} + } + held.value[a.path] = r + return r +} + +func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { + var table string + row := t.site.row + if t.site.table != nil { + table = t.site.table.path + } + if row == formatRow { + row = sc.rowOf(t.site.table) + } + trace(strings.Clone(sc.group), strings.Clone(table), row, a) +} + +// renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one +// reference alone whose read drew null. +func renderLeaf(s *session, n node, sc renderScope) readValue { + n = resolveChoice(s, n) + switch leaf := n.(type) { + case *nullItem: + return readValue{null: true} + case *template: + if leaf.link.readsColumn != nil { + return readReference(s, leaf, sc.in(leaf), leaf.link.readsColumn.a) + } + } + return readValue{text: render(s, n, sc)} +} + +// resolveChoice resolves a choice to one variant, so a held head is a concrete node the +// rest of the expansion shares. Nested choices unwrap too: a draw is one value, not +// another set to pick from. +func resolveChoice(s *session, n node) node { + for c, ok := n.(*choice); ok; c, ok = n.(*choice) { + n = pick(s, c) + } + return n +} diff --git a/todo.md b/todo.md index 98f9537..3ecf6de 100644 --- a/todo.md +++ b/todo.md @@ -28,10 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Move the render's read path — `readField`, `readReference`, `readHeld`, `traceRead`, - `renderLeaf`, `resolveChoice`, `renderOnce` and `expandAnew` — into `render.go`, and - merge what `hold.go` and `holdset.go` keep into one file: both architects followed one read - across three files. - Give `prove`, `stepInto` and `columnItems` a case per node kind and panic through `internalError` on the rest, and list every such switch in `node`'s doc: `prove`'s default tells a typed column reading a bare table that it reads a null, and six seats -- 2.52.0 From 64bb5c7004df6af79db76235ac722ee2ed32cab5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:31:16 +0200 Subject: [PATCH 275/966] Separate the hold's gloss from the hold set in README's layout --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a3f55e2..c9cae0b 100644 --- a/README.md +++ b/README.md @@ -883,7 +883,7 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold, one draw per held name, and the hold set and scope a render reads its reference paths through +hold.go the hold: one draw per held name; and the hold set and scope a render reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways drawfence.go the load fences over reference draws: a drawGroup's, a table's own family, and a render's and a record's columns through the survey readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them -- 2.52.0 From d01c914e7f2cd4333cadae13455ef013428f493c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:36:14 +0200 Subject: [PATCH 276/966] Assert prove, stepInto and columnItems panic on an unhandled node, and a bare table read is refused as a row --- render_test.go | 3 +++ table_test.go | 10 ++++++++++ 2 files changed, 13 insertions(+) diff --git a/render_test.go b/render_test.go index 4aa67ae..8f0e5cd 100644 --- a/render_test.go +++ b/render_test.go @@ -61,10 +61,13 @@ func (*unhandledNode) isNode() {} func TestNodeSwitchesPanicOnAnUnhandledNode(t *testing.T) { for name, call := range map[string]func(){ + "columnItems": func() { columnItems(&unhandledNode{}) }, "contained": func() { contained(&unhandledNode{}) }, "paths": func() { paths(&unhandledNode{}) }, + "prove": func() { (&valueProof{}).prove(&unhandledNode{}) }, "render": func() { render(nil, &unhandledNode{}, renderScope{}) }, "renderEdges": func() { renderEdges(&unhandledNode{}) }, + "stepInto": func() { _, _ = stepInto(&unhandledNode{}, "x") }, } { mustPanic(t, name, call) } diff --git a/table_test.go b/table_test.go index 98dadc3..8a3d656 100644 --- a/table_test.go +++ b/table_test.go @@ -1132,3 +1132,13 @@ func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { t.Fatalf("New = %v, want 1280's shop and 1281's city kept apart, one draw of municipality rendering one of them", err) } } + +func TestDatatypeRefusesARowRead(t *testing.T) { + for _, format := range []string{"{/region}", "{/region[01]}"} { + files := with(geo(), map[string]string{"row.json": `{"format":"","col":{"format":"` + format + `","datatype":"integer"}}`}) + want := `"{name}" renders a row of region, which is composed text` + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, files))); err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("%s: New = %v, want an error containing %q", format, err, want) + } + } +} -- 2.52.0 From 16e4f4ebbbe92150243b3b399c2e76516a1d7c67 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:37:32 +0200 Subject: [PATCH 277/966] Give prove, stepInto and columnItems a case per node kind, panicking on the rest --- datatype.go | 3 +++ node.go | 5 +++-- path.go | 3 +++ todo.md | 6 ------ value.go | 6 +++++- 5 files changed, 14 insertions(+), 9 deletions(-) diff --git a/datatype.go b/datatype.go index d9e3424..7fad539 100644 --- a/datatype.go +++ b/datatype.go @@ -138,6 +138,9 @@ func columnItems(n node) (items []*template, nullable bool) { items = append(items, n) case *nullItem: nullable = true + case *tableColumn: + default: + panic(internalError("columnItems has no case for node %T", n)) } } collect(n) diff --git a/node.go b/node.go index 56bd89a..77ab03d 100644 --- a/node.go +++ b/node.go @@ -11,8 +11,9 @@ import ( // node is a compiled element of the namespace tree: a folder, choice, null, // template, table, column or row. Compiling JSON into these once (see compile) -// means rendering never re-inspects the raw JSON. A new kind needs a -// case in contained, paths, render and renderEdges; each panics on a kind it does not list. +// means rendering never re-inspects the raw JSON. A new kind needs a case in +// columnItems, contained, paths, prove, render, renderEdges and stepInto; each +// panics on a kind it does not list. type node interface{ isNode() } // folder is a namespace of named children, built from a directory of JSON files diff --git a/path.go b/path.go index cf62df0..206ec95 100644 --- a/path.go +++ b/path.go @@ -142,6 +142,9 @@ func stepInto(n node, seg string) (node, error) { } case *tableColumn: return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", seg, n.t.header[n.i]) + case *nullItem: + default: + panic(internalError("stepInto has no case for node %T", n)) } return nil, fmt.Errorf("no field %q", seg) } diff --git a/todo.md b/todo.md index 3ecf6de..40b1f4f 100644 --- a/todo.md +++ b/todo.md @@ -28,10 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Give `prove`, `stepInto` and `columnItems` a case per node kind and panic through - `internalError` on the rest, and list every such switch in `node`'s doc: `prove`'s - default tells a typed column reading a bare table that it reads a null, and six seats - found `node` naming four of them. - Replace `disagreements`' ten keys with the rule they encode: a declared fix holds one datatype, a read one is `readTyped` where proven and `readAsText` otherwise, a text one is `otherAsText` beside an unproven read and `declare` otherwise. Six seats could not @@ -222,8 +218,6 @@ by a `data-import/` script, as README goal 12 asks. first in Go's map order. - Report a CRLF rows file holding only its header as having no rows: `parseRows` answers "line 2 is empty". -- Refuse a bare table read in a typed column with a reason naming the table, once - `prove` has a case per node kind. - Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. - Refuse two whole reads of one table family in one render, which goal 5 promises is a diff --git a/value.go b/value.go index 243d0bd..ef82019 100644 --- a/value.go +++ b/value.go @@ -89,10 +89,14 @@ func (p *valueProof) prove(n node) proven { v = p.proveTemplate(n) case *tableColumn: v = p.proveCells(n) + case *table: + v = p.prove(n.rowNode) case *tableRow: v = unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", n.t.formatTemplate.format, n.t.segment)) - default: + case *nullItem: v = unproven(`it reads a null, which renders "" outside its own column`) + default: + panic(internalError("prove has no case for node %T", n)) } p.memo[n] = v return v -- 2.52.0 From 14ced0e6426c5aa9a2b39db0ddb44a359f8b302d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:42:47 +0200 Subject: [PATCH 278/966] File prove's composed-text reason for a one-column table format --- todo.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/todo.md b/todo.md index 40b1f4f..0ec0cbe 100644 --- a/todo.md +++ b/todo.md @@ -218,6 +218,9 @@ by a `data-import/` script, as README goal 12 asks. first in Go's map order. - Report a CRLF rows file holding only its header as having no rows: `parseRows` answers "line 2 is empty". +- Stop `prove` calling every row read composed text: a table whose format is one + column, `{code}`, renders one cell, yet a typed column reading it is refused as + "composed text". - Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. - Refuse two whole reads of one table family in one render, which goal 5 promises is a -- 2.52.0 From e494d61647328d0746af68baa6496a7ac3a5ed60 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:43:24 +0200 Subject: [PATCH 279/966] Fold the product-owner's wording findings into the row-read refusal item --- todo.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 0ec0cbe..7c9446f 100644 --- a/todo.md +++ b/todo.md @@ -218,9 +218,10 @@ by a `data-import/` script, as README goal 12 asks. first in Go's map order. - Report a CRLF rows file holding only its header as having no rows: `parseRows` answers "line 2 is empty". -- Stop `prove` calling every row read composed text: a table whose format is one - column, `{code}`, renders one cell, yet a typed column reading it is refused as - "composed text". +- Reword `prove`'s refusal of a typed column reading a row, `{/region}`: say the + format is the table's own (`region's format "{name}"`), name the read to write, + `{/region.}` with its columns, and stop calling a one-column format, + `{code}`, composed text. - Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. - Refuse two whole reads of one table family in one render, which goal 5 promises is a -- 2.52.0 From c15067b9428c3e4a2f7d206eef5226a26aaa60f2 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:57:13 +0200 Subject: [PATCH 280/966] Assert disagreementFix takes the two kinds and the proof, and refuses two text items --- datatype_test.go | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/datatype_test.go b/datatype_test.go index 95206ee..b6ceb2b 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -221,17 +221,6 @@ func TestNullColumn(t *testing.T) { } } -func TestDisagreementsHoldEveryKeyDisagreementBuilds(t *testing.T) { - kinds := []itemKind{kindText, kindReads, kindDeclares} - for _, fix := range kinds { - for _, other := range kinds[fix:] { - if other == kindText { - continue - } - for _, proven := range []bool{false, true} { - disagreementFix(disagreementKey{fix, other, proven}) - } - } - } - mustPanic(t, "two text items", func() { disagreementFix(disagreementKey{kindText, kindText, false}) }) +func TestDisagreementFixRefusesTwoTextItems(t *testing.T) { + mustPanic(t, "two text items", func() { disagreementFix(kindText, kindText, false) }) } -- 2.52.0 From e8469f1f89382de277abba4731483741c668f118 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 09:58:01 +0200 Subject: [PATCH 281/966] Replace disagreements' ten keys with the rule they encode --- datatype.go | 41 ++++++++++++++++------------------------- todo.md | 4 ---- 2 files changed, 16 insertions(+), 29 deletions(-) diff --git a/datatype.go b/datatype.go index 7fad539..81a1f36 100644 --- a/datatype.go +++ b/datatype.go @@ -174,25 +174,6 @@ type itemPair struct { want DataType } -// disagreementKey is the two items' kinds and whether fix's values prove to hold want. -type disagreementKey struct { - fix, other itemKind - proven bool -} - -var disagreements = map[disagreementKey]func(d itemPair) error{ - {kindDeclares, kindDeclares, false}: holdOne, - {kindDeclares, kindDeclares, true}: holdOne, - {kindReads, kindDeclares, false}: readAsText, - {kindReads, kindDeclares, true}: readTyped, - {kindReads, kindReads, false}: readAsText, - {kindReads, kindReads, true}: readTyped, - {kindText, kindDeclares, false}: declare, - {kindText, kindDeclares, true}: declare, - {kindText, kindReads, false}: otherAsText, - {kindText, kindReads, true}: declare, -} - // disagreement names the fix for items a and b of one column holding different datatypes, fixing // the one that declares least, a when both declare alike. func disagreement(a *template, da DataType, b *template, db DataType) error { @@ -201,15 +182,25 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { d.fix, d.other, d.want = b, a, da } proven := (&valueProof{}).proveColumnItem(d.fix).not[d.want] == "" - return disagreementFix(disagreementKey{kindOf(d.fix), kindOf(d.other), proven})(d) + return disagreementFix(kindOf(d.fix), kindOf(d.other), proven)(d) } -func disagreementFix(k disagreementKey) func(d itemPair) error { - fix := disagreements[k] - if fix == nil { - panic(internalError("no disagreement names the fix for %+v", k)) +// disagreementFix is the fix for item fix beside item other, where proven says fix's values hold +// the other's datatype. +func disagreementFix(fix, other itemKind, proven bool) func(d itemPair) error { + switch { + case fix == kindDeclares: + return holdOne + case fix == kindReads && proven: + return readTyped + case fix == kindReads: + return readAsText + case other == kindText: + panic(internalError("two text items hold one datatype, so they never disagree")) + case other == kindReads && !proven: + return otherAsText } - return fix + return declare } func holdOne(d itemPair) error { diff --git a/todo.md b/todo.md index 7c9446f..a72d389 100644 --- a/todo.md +++ b/todo.md @@ -28,10 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Replace `disagreements`' ten keys with the rule they encode: a declared fix holds one - datatype, a read one is `readTyped` where proven and `readAsText` otherwise, a text one - is `otherAsText` beside an unproven read and `declare` otherwise. Six seats could not - see why each key gets its fix. - Drop `table.fields` for `formatTemplate.fields`: one map held in two fields. - Rename `arm.key` to `head`, as `template.head` reads it, leaving `key` the table's key column; give `renderScope.row` a type that is not `tablePin`, since it pins nothing; -- 2.52.0 From ade2a7855f2d8cb904ab36056b71b3631b5ecf28 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:00:12 +0200 Subject: [PATCH 282/966] Trim disagreementFix's comment to the proof it defines --- datatype.go | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/datatype.go b/datatype.go index 81a1f36..6837775 100644 --- a/datatype.go +++ b/datatype.go @@ -185,8 +185,7 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { return disagreementFix(kindOf(d.fix), kindOf(d.other), proven)(d) } -// disagreementFix is the fix for item fix beside item other, where proven says fix's values hold -// the other's datatype. +// proven says fix's values hold other's datatype. func disagreementFix(fix, other itemKind, proven bool) func(d itemPair) error { switch { case fix == kindDeclares: -- 2.52.0 From 30d0f84ea4cd7d109de3f0e151062da974fa64e9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:06:00 +0200 Subject: [PATCH 283/966] Drop table.fields for formatTemplate.fields --- graph.go | 2 +- path.go | 4 ++-- table.go | 11 ++++++----- todo.md | 1 - 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/graph.go b/graph.go index fee1b54..bbb226b 100644 --- a/graph.go +++ b/graph.go @@ -61,7 +61,7 @@ func contained(n node) []namedNode { case *template: return namedNodes(n.fields) case *table: - return append([]namedNode{{node: n.formatTemplate}}, namedNodes(n.fields)...) + return append([]namedNode{{node: n.formatTemplate}}, namedNodes(n.formatTemplate.fields)...) case *tableColumn: if len(n.t.cellTemplates) == 0 { return nil diff --git a/path.go b/path.go index 206ec95..ead0746 100644 --- a/path.go +++ b/path.go @@ -438,8 +438,8 @@ func (t *table) step(tail []string) (column node, child *table, err error) { if len(tail) == 0 { return nil, nil, nil } - if i, ok := t.col[tail[0]]; ok { - return t.fields[t.header[i]], nil, nil + if column, ok := t.formatTemplate.fields[tail[0]]; ok { + return column, nil, nil } if child = t.descendant(tail[0]); child == nil { return nil, nil, fmt.Errorf("no column or linked table %q in %s", tail[0], t.segment) diff --git a/table.go b/table.go index a0da290..a5c70e8 100644 --- a/table.go +++ b/table.go @@ -19,7 +19,6 @@ type table struct { formatTemplate *template // fields are the column nodes header []string col map[string]int - fields map[string]node // column nodes, the format's fields rowNode *tableRow cells []string // rows × columns, flat cellTemplates map[int]*template @@ -157,7 +156,6 @@ func (t *table) parseRows(data string) error { } t.header = strings.Split(header, "\t") t.col = make(map[string]int, len(t.header)) - t.fields = make(map[string]node, len(t.header)) for i, name := range t.header { if err := checkName(name); err != nil { return fmt.Errorf("column %w", err) @@ -166,7 +164,6 @@ func (t *table) parseRows(data string) error { return fmt.Errorf("column %q is named twice", name) } t.col[name] = i - t.fields[name] = &tableColumn{t, i} } if header == data { return fmt.Errorf("has no rows below its header; a table is at least two rows") @@ -338,10 +335,14 @@ func (t *table) compileRowFormat(format string) error { } } } - if err := checkTokens(toks, t.fields); err != nil { + fields := make(map[string]node, len(t.header)) + for i, name := range t.header { + fields[name] = &tableColumn{t, i} + } + if err := checkTokens(toks, fields); err != nil { return err } - t.formatTemplate = &template{format: format, tokens: toks, fields: t.fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} + t.formatTemplate = &template{format: format, tokens: toks, fields: fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} t.rowNode = &tableRow{t} return nil } diff --git a/todo.md b/todo.md index a72d389..9607b20 100644 --- a/todo.md +++ b/todo.md @@ -28,7 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Drop `table.fields` for `formatTemplate.fields`: one map held in two fields. - Rename `arm.key` to `head`, as `template.head` reads it, leaving `key` the table's key column; give `renderScope.row` a type that is not `tablePin`, since it pins nothing; and name `value.go` by the `valueProof` it holds. -- 2.52.0 From a7d77bd3adbca012b874786a65dff85c02efd6e1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:11:12 +0200 Subject: [PATCH 284/966] Rename arm.key to head, give renderScope.row its own type, and value.go to valueproof.go --- README.md | 2 +- datatype.go | 2 +- drawfence.go | 4 +-- format.go | 66 +++++++++++++++++++-------------------- graph.go | 4 +-- hold.go | 13 +++++--- hold_test.go | 2 +- holdfence.go | 38 +++++++++++----------- node.go | 12 +++---- readsurvey.go | 2 +- readsurvey_test.go | 2 +- reference.go | 2 +- render.go | 12 +++---- table.go | 4 +-- table_test.go | 4 +-- todo.md | 3 -- value.go => valueproof.go | 2 +- 17 files changed, 88 insertions(+), 86 deletions(-) rename value.go => valueproof.go (99%) diff --git a/README.md b/README.md index c9cae0b..5867104 100644 --- a/README.md +++ b/README.md @@ -896,7 +896,7 @@ checksum.go the check characters a derivation appends, and the IBAN they sit transform.go the builtins that rewrite an operand's value, and the ASCII folding calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype -value.go the value proof: what a typed column or calc operand holds, checked at load +valueproof.go the value proof: what a typed column or calc operand holds, checked at load data.go the load path: fs.FS folders/files -> merged namespace tree, and the bind pipeline every entry point runs cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc diff --git a/datatype.go b/datatype.go index 6837775..9930975 100644 --- a/datatype.go +++ b/datatype.go @@ -77,7 +77,7 @@ func checkColumns(s nodeScope) error { items, _ := columnItems(column) for _, it := range items { if r := it.link.readsColumn; r != nil { - if err := check(r.a.key[1:], r.a.tail[0], r.column); err != nil { + if err := check(r.a.head[1:], r.a.tail[0], r.column); err != nil { return err } } diff --git a/drawfence.go b/drawfence.go index 6080817..325695e 100644 --- a/drawfence.go +++ b/drawfence.go @@ -90,7 +90,7 @@ func checkOwnFamily(t *template) error { seen[n] = true for _, e := range renderEdges(n) { if e.readsRef() { - if head, isTable := n.(*template).head(e.read.key).(*table); isTable && head.familyRoot() == own.familyRoot() { + if head, isTable := n.(*template).head(e.read.head).(*table); isTable && head.familyRoot() == own.familyRoot() { return e, head, true } } @@ -183,7 +183,7 @@ func readsTable(n node, a arm) bool { if !isTemplate { return false } - _, isTable := t.head(a.key).(*table) + _, isTable := t.head(a.head).(*table) return isTable } diff --git a/format.go b/format.go index f232179..d9849c5 100644 --- a/format.go +++ b/format.go @@ -246,25 +246,25 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { if err := checkSegments(a); err != nil { return err } - field, ok := fields[a.key] + field, ok := fields[a.head] if !ok { - if a.key == "" { + if a.head == "" { return fmt.Errorf("a name is never empty, so this token can name no field") } - if isOption(a.key) { - return fmt.Errorf("%q is an option and can never be a field", a.key) + if isOption(a.head) { + return fmt.Errorf("%q is an option and can never be a field", a.head) } if len(fields) == 0 { hint := "" if wholeToken && hintableRef(name) { hint = fmt.Sprintf(" — write {/%s} to reference the data", name) } - return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.key, hint) + return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint) } - return fmt.Errorf("no field %q", a.key) + return fmt.Errorf("no field %q", a.head) } - if err := checkPath(field, a.tail, a.key); err != nil { - return fmt.Errorf("field %q: %w", a.key, err) + if err := checkPath(field, a.tail, a.head); err != nil { + return fmt.Errorf("field %q: %w", a.head, err) } return nil } @@ -302,45 +302,45 @@ func checkNoRepeatedArm(body string, names []string) error { return nil } -// arm is one alternative of a {a|b} token or one operand, split into the key +// arm is one alternative of a {a|b} token or one operand, split into the head // `template.head` resolves and the tail of a dotted path into it. A non-empty tail // is what makes the arm a held draw: its head is drawn once per expansion (see // compileOps). type arm struct { spelling string // as written, for messages - key string + head string tail []string - levels []string // the key at each level the tail passes through, the arm's key first - path string // key and tail, the one path every way of writing this read shares + levels []string // the path at each level the tail passes through, the head first + path string // head and tail, the one path every way of writing this read shares } -// splitArm splits one name into key and tail. refs maps a reference to what +// splitArm splits one name into head and tail. refs maps a reference to what // linkRefs resolved it to; before linking, a reference is whole. func splitArm(name string, refs map[string]refBinding) arm { if isRef(name) { b, linked := refs[name] if !linked || len(b.tail) == 0 { - key := name + head := name if linked { - key = b.key + head = b.key } - return arm{spelling: name, key: key, path: key} + return arm{spelling: name, head: head, path: head} } return pathArm(name, b.key, b.tail) } segs, err := splitPath(name) if err != nil || len(segs) == 1 { - return arm{spelling: name, key: name, path: name} + return arm{spelling: name, head: name, path: name} } return pathArm(name, segs[0], segs[1:]) } -func pathArm(name, key string, segs []string) arm { - levels := []string{key} +func pathArm(name, head string, segs []string) arm { + levels := []string{head} for i := 0; i < len(segs)-1; i++ { - levels = append(levels, key+"."+strings.Join(segs[:i+1], ".")) + levels = append(levels, head+"."+strings.Join(segs[:i+1], ".")) } - return arm{spelling: name, key: key, tail: segs, levels: levels, path: key + "." + strings.Join(segs, ".")} + return arm{spelling: name, head: head, tail: segs, levels: levels, path: head + "." + strings.Join(segs, ".")} } // checkSegments rejects an unfinished path: "{a.}" and "{a..b}" each have a @@ -350,7 +350,7 @@ func checkSegments(a arm) error { if len(a.tail) == 0 { return nil } - if a.key == "" { + if a.head == "" { return fmt.Errorf("path has an empty segment") } for _, seg := range a.tail { @@ -379,13 +379,13 @@ type op struct { } // formatOps is a compiled format: its ops, the size of its literal text, and the -// names it holds. pathKeys maps each key a path starts from to the first such path; -// held is every such key plus the fields an operand reads; holder maps each held +// names it holds. pathHeads maps each head a path starts from to the first such path; +// held is every such head plus the fields an operand reads; holder maps each held // name to the first reader holding it, for error messages. type formatOps struct { ops []op grow int - pathKeys map[string]string + pathHeads map[string]string held map[string]bool holder map[string]string heldLocal bool // some held name is not a reference path, so expand makes a hold @@ -396,19 +396,19 @@ func (c *formatOps) holdName(a arm, label string) { c.held = map[string]bool{} c.holder = map[string]string{} } - c.held[a.key] = true - if !isRef(a.key) || len(a.tail) == 0 { + c.held[a.head] = true + if !isRef(a.head) || len(a.tail) == 0 { c.heldLocal = true } - if _, named := c.holder[a.key]; !named { - c.holder[a.key] = label + if _, named := c.holder[a.head]; !named { + c.holder[a.head] = label } if len(a.tail) > 0 { - if c.pathKeys == nil { - c.pathKeys = map[string]string{} + if c.pathHeads == nil { + c.pathHeads = map[string]string{} } - if _, named := c.pathKeys[a.key]; !named { - c.pathKeys[a.key] = a.spelling + if _, named := c.pathHeads[a.head]; !named { + c.pathHeads[a.head] = a.spelling } } } diff --git a/graph.go b/graph.go index bbb226b..d99fb26 100644 --- a/graph.go +++ b/graph.go @@ -109,7 +109,7 @@ type renderEdge struct { operand string } -func (e renderEdge) readsRef() bool { return isRef(e.read.key) } +func (e renderEdge) readsRef() bool { return isRef(e.read.head) } // reached names an edge as the author spelled it. func (e renderEdge) reached() string { @@ -133,7 +133,7 @@ func renderEdges(n node) []renderEdge { case *template: var es []renderEdge add := func(a arm, operand string) { - c := n.head(a.key) + c := n.head(a.head) if c == nil { return } diff --git a/hold.go b/hold.go index a97fb26..dfaf116 100644 --- a/hold.go +++ b/hold.go @@ -41,7 +41,12 @@ type holdSet struct { type renderScope struct { set *holdSet group string - row tablePin + row renderedRow +} + +type renderedRow struct { + t *table + index int } // in is the scope t renders in: its draw group where it names one, else its caller's. @@ -57,9 +62,9 @@ func (sc renderScope) in(t *template) renderScope { func (sc renderScope) at(n node, pins *pinSet) renderScope { switch n := n.(type) { case *tableRow: - sc.row = tablePin{n.t, pins.mustRow(n.t)} + sc.row = renderedRow{n.t, pins.mustRow(n.t)} case *tableColumn: - sc.row = tablePin{n.t, pins.mustRow(n.t)} + sc.row = renderedRow{n.t, pins.mustRow(n.t)} } return sc } @@ -69,7 +74,7 @@ func (sc renderScope) rowOf(t *table) int { if sc.row.t != t { panic(internalError("a column of %s renders in a scope holding no row of it", t.segment)) } - return sc.row.row + return sc.row.index } func (sc renderScope) groupHold() *groupHold { diff --git a/hold_test.go b/hold_test.go index 5121c13..b3621d5 100644 --- a/hold_test.go +++ b/hold_test.go @@ -761,7 +761,7 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { } func TestReadFieldPanicsOnAPathItCannotHold(t *testing.T) { - a := arm{spelling: "w.x", key: "w", tail: []string{"x"}, path: "w.x"} + a := arm{spelling: "w.x", head: "w", tail: []string{"x"}, path: "w.x"} for name, format := range map[string]string{ "unheld arm with a path": `{"format":"{w}","w":{"format":"{x}","x":"1"}}`, "held arm with no hold": `{"format":"{w.x}","w":{"format":"{x}","x":"1"}}`, diff --git a/holdfence.go b/holdfence.go index 9343523..534e547 100644 --- a/holdfence.go +++ b/holdfence.go @@ -15,9 +15,9 @@ func heldCheck(path string, n node) error { if !ok || len(t.compiled.held) == 0 { return nil } - readers := pathKeyReaders(t.compiled.ops, t.compiled.pathKeys) + readers := pathHeadReaders(t.compiled.ops, t.compiled.pathHeads) for _, name := range heldNames(t) { - if _, isPath := t.compiled.pathKeys[name]; isPath && isRef(name) { + if _, isPath := t.compiled.pathHeads[name]; isPath && isRef(name) { continue // held for the render: drawFence compares every read of it, by path, across the render and its groups } if err := checkNameHeld(t, name, readers); err != nil { @@ -36,8 +36,8 @@ func heldNames(t *template) []string { names = append(names, name) } sort.Slice(names, func(i, j int) bool { - _, pi := t.compiled.pathKeys[names[i]] - _, pj := t.compiled.pathKeys[names[j]] + _, pi := t.compiled.pathHeads[names[i]] + _, pj := t.compiled.pathHeads[names[j]] if pi != pj { return !pi } @@ -51,12 +51,12 @@ func heldNames(t *template) []string { // produces. func heldNodes(t *template, name string, readers []reader) map[node]bool { held := map[node]bool{} - if _, isPath := t.compiled.pathKeys[name]; !isPath { + if _, isPath := t.compiled.pathHeads[name]; !isPath { operandDraw(t.head(name), held) return held } for _, r := range readers { - if r.a.key == name { + if r.a.head == name { coverPath(t.head(name), r.a.tail, held) } } @@ -71,10 +71,10 @@ func checkNameHeld(t *template, name string, readers []reader) error { if len(held) == 0 { return nil // a fixed head holds nothing to reach } - reader, isPath := t.compiled.pathKeys[name] + reader, isPath := t.compiled.pathHeads[name] seen := map[node]bool{} for _, e := range renderEdges(t) { - if e.read.key == name || !renders(e.to, held, seen) { + if e.read.head == name || !renders(e.to, held, seen) { continue } if isPath { @@ -89,7 +89,7 @@ func checkNameHeld(t *template, name string, readers []reader) error { func operandReader(t *template, name string) string { for _, o := range t.compiled.ops { for _, a := range o.operands { - if a.key == name { + if a.head == name { return o.fn } } @@ -175,8 +175,8 @@ func renders(n node, want, seen map[node]bool) bool { // disagree. Reads are compared in sorted order, so which pair is reported does not // depend on where the tokens sit. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkNoOverlap(ops []op, pathKeys map[string]string) error { - names := pathKeyReaders(ops, pathKeys) +func checkNoOverlap(ops []op, pathHeads map[string]string) error { + names := pathHeadReaders(ops, pathHeads) // Stable over one format-order scan, so two readers of one name (a token and a // calc operand both naming "p") are reported as the format writes them. sort.SliceStable(names, func(i, j int) bool { return names[i].a.path < names[j].a.path }) @@ -190,25 +190,25 @@ func checkNoOverlap(ops []op, pathKeys map[string]string) error { return nil } -// reader is one way a format reaches a path key, and how to name it. +// reader is one way a format reaches a path head, and how to name it. type reader struct { a arm label string } -// pathKeyReaders lists every way a format reaches a sibling path key, in the order the +// pathHeadReaders lists every way a format reaches a sibling path head, in the order the // format writes them. An operand renders its field, so it names a level exactly // as a token does; one scan finds both, which is what puts them in one order. -func pathKeyReaders(ops []op, pathKeys map[string]string) []reader { +func pathHeadReaders(ops []op, pathHeads map[string]string) []reader { var names []reader for _, o := range ops { for _, a := range o.operands { - if _, isPathKey := pathKeys[a.key]; isPathKey && !isRef(a.key) { + if _, isPathHead := pathHeads[a.head]; isPathHead && !isRef(a.head) { names = append(names, reader{a, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { - if _, isPathKey := pathKeys[a.key]; isPathKey && !isRef(a.key) { + if _, isPathHead := pathHeads[a.head]; isPathHead && !isRef(a.head) { names = append(names, reader{a, "token {" + a.spelling + "}"}) } } @@ -224,11 +224,11 @@ func checkNoRepeatedRead(c formatOps) error { count := map[string]int{} for _, o := range c.ops { for _, a := range o.arms { - if len(a.tail) > 0 || !c.held[a.key] { + if len(a.tail) > 0 || !c.held[a.head] { continue } - if count[a.key]++; count[a.key] > 1 { - return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.spelling, c.holder[a.key], a.key, a.spelling) + if count[a.head]++; count[a.head] > 1 { + return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.spelling, c.holder[a.head], a.head, a.spelling) } } } diff --git a/node.go b/node.go index 77ab03d..fc53753 100644 --- a/node.go +++ b/node.go @@ -81,12 +81,12 @@ type templateLink struct { func (*template) isNode() {} -// head is the node an arm's key names: a sibling field, or a reference's category. -func (t *template) head(key string) node { - if isRef(key) { - return t.link.refHeads[key] +// head is the node an arm's head names: a sibling field, or a reference's category. +func (t *template) head(name string) node { + if isRef(name) { + return t.link.refHeads[name] } - return t.fields[key] + return t.fields[name] } // compile converts parsed JSON — a category or an inline template — into a node tree, @@ -229,7 +229,7 @@ func (t *template) fixedText() (string, bool) { // compiled reads. func compileFormat(toks []formatToken, refs map[string]refBinding) (formatOps, error) { c := compileOps(toks, refs) - if err := checkNoOverlap(c.ops, c.pathKeys); err != nil { + if err := checkNoOverlap(c.ops, c.pathHeads); err != nil { return c, err } return c, checkNoRepeatedRead(c) diff --git a/readsurvey.go b/readsurvey.go index e2e94e6..8f5106e 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -143,7 +143,7 @@ func (s *readSurvey) walk(n node, at surveyAt) { func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { if e.readsRef() { a := e.read - tr := tableReadOf(from.(*template).head(a.key), a, e.to) + tr := tableReadOf(from.(*template).head(a.head), a, e.to) if k := (readKey{at.group, a.path, s.pinsID(&at.pins), s.pinsID(&at.wholePins)}); !s.read[k] { s.read[k] = true s.reads = append(s.reads, pathRead{at, a, tr}) diff --git a/readsurvey_test.go b/readsurvey_test.go index bcf71e2..5e67d68 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -35,7 +35,7 @@ func wantGathered(t *testing.T, tables map[string]*table, label string, gathered } seen := map[traceKey]bool{} trace := func(group, table string, row int, a arm) { - if !isRef(a.key) { + if !isRef(a.head) { return } r := pathRead{at: surveyAt{group: group}, a: a} diff --git a/reference.go b/reference.go index 668b1db..03e2902 100644 --- a/reference.go +++ b/reference.go @@ -146,7 +146,7 @@ func columnReadOf(t *template, link templateLink) *columnRead { return nil } a := splitArm(name, link.refs) - target, isTemplate := link.refHeads[a.key].(*template) + target, isTemplate := link.refHeads[a.head].(*template) if !isTemplate || !target.isRecord || len(a.tail) != 1 { return nil } diff --git a/render.go b/render.go index c2433e8..b8821e4 100644 --- a/render.go +++ b/render.go @@ -62,7 +62,7 @@ func render(s *session, n node, sc renderScope) string { case *nullItem: return "" case *table: - sc.row = tablePin{n, n.drawRow(s)} + sc.row = renderedRow{n, n.drawRow(s)} return expand(s, n.formatTemplate, sc) case *tableRow: return expand(s, n.t.formatTemplate, sc) @@ -159,17 +159,17 @@ func expand(s *session, t *template, sc renderScope) string { // read one row, either read twice gives one value, and a shown operand is the operand // computed. Every other name is drawn afresh, so {word} {word} still draws twice. func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { - if isRef(a.key) && len(a.tail) > 0 { + if isRef(a.head) && len(a.tail) > 0 { return readReference(s, t, sc, a) } if sc.set.trace != nil { traceRead(sc.set.trace, t, sc, a) } - if !t.compiled.held[a.key] { + if !t.compiled.held[a.head] { if len(a.tail) > 0 { - panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.key)) + panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.head)) } - return readValue{text: render(s, t.head(a.key), sc)} + return readValue{text: render(s, t.head(a.head), sc)} } if held == nil { panic(internalError("%q reads a name the expansion holds, with no hold to keep it in", a.spelling)) @@ -191,7 +191,7 @@ func readHeld(s *session, t *template, held *hold, pins *pinSet, sc renderScope, if r, done := held.value[a.path]; done { return r } - leaf := drawPath(t.head(a.key), a.tail, a.key, &pathDraw{s: s, held: held, pins: pins, a: &a}) + leaf := drawPath(t.head(a.head), a.tail, a.head, &pathDraw{s: s, held: held, pins: pins, a: &a}) if pins != nil { sc = sc.at(leaf, pins) } diff --git a/table.go b/table.go index a5c70e8..feee148 100644 --- a/table.go +++ b/table.go @@ -330,8 +330,8 @@ func (t *table) compileRowFormat(format string) error { } for _, name := range tok.names { a := splitArm(name, nil) - if _, ok := t.col[a.key]; !ok && !isRef(a.key) && a.key != "" { - return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.header) + if _, ok := t.col[a.head]; !ok && !isRef(a.head) && a.head != "" { + return fmt.Errorf("format names no column %q of %s; the columns are %v", a.head, t.file, t.header) } } } diff --git a/table_test.go b/table_test.go index 8a3d656..22c03f6 100644 --- a/table_test.go +++ b/table_test.go @@ -1063,7 +1063,7 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { {"region[12].name", "region=region[12]"}, } { a := splitArm(c.path, nil) - head := f.root.children[a.key].(*table) + head := f.root.children[a.head].(*table) tr := tableReadOf(head, a, nil) var sels []string for _, s := range tr.sels { @@ -1075,7 +1075,7 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { probed, drawn := tr.pins, tr.drawn for i := 0; i < 20; i++ { var pins pinSet - drawPath(head, a.tail, a.key, &pathDraw{s: f.rand, pins: &pins}) + drawPath(head, a.tail, a.head, &pathDraw{s: f.rand, pins: &pins}) want := map[*table]bool{} pins.each(func(tbl *table, _ int) { if _, selected := probed.pinned(tbl); !selected { diff --git a/todo.md b/todo.md index 9607b20..e042842 100644 --- a/todo.md +++ b/todo.md @@ -28,9 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Rename `arm.key` to `head`, as `template.head` reads it, leaving `key` the table's key - column; give `renderScope.row` a type that is not `tablePin`, since it pins nothing; - and name `value.go` by the `valueProof` it holds. - Delete the prose the seats found empty: "kept whole on purpose" (`format.go`, `calc.go`), the second copy of the concrete-session rationale (`table.drawRow`, `pick`), the doc comments restating `isSelector`, `selectorOf`, diff --git a/value.go b/valueproof.go similarity index 99% rename from value.go rename to valueproof.go index ef82019..9c46f5b 100644 --- a/value.go +++ b/valueproof.go @@ -164,7 +164,7 @@ func (p *valueProof) proveTemplate(t *template) proven { case o.kind == nameRead: var leaves []node for _, a := range o.arms { - leaves = append(leaves, pathLeaves(t.head(a.key), a.tail)...) + leaves = append(leaves, pathLeaves(t.head(a.head), a.tail)...) } return p.proveUnion(leaves) case name == "calc": -- 2.52.0 From 6275c6d41e1b9151672954a88b5772dadd8b9143 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:14:00 +0200 Subject: [PATCH 285/966] Rename refBinding.key to head, and the stale key wording beside it --- drawfence.go | 4 ++-- format.go | 6 +++--- node.go | 2 +- reference.go | 14 +++++++------- shape_test.go | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/drawfence.go b/drawfence.go index 325695e..42910b3 100644 --- a/drawfence.go +++ b/drawfence.go @@ -90,8 +90,8 @@ func checkOwnFamily(t *template) error { seen[n] = true for _, e := range renderEdges(n) { if e.readsRef() { - if head, isTable := n.(*template).head(e.read.head).(*table); isTable && head.familyRoot() == own.familyRoot() { - return e, head, true + if read, isTable := n.(*template).head(e.read.head).(*table); isTable && read.familyRoot() == own.familyRoot() { + return e, read, true } } if e, head, found := find(e.to); found { diff --git a/format.go b/format.go index d9849c5..98ab09b 100644 --- a/format.go +++ b/format.go @@ -322,11 +322,11 @@ func splitArm(name string, refs map[string]refBinding) arm { if !linked || len(b.tail) == 0 { head := name if linked { - head = b.key + head = b.head } return arm{spelling: name, head: head, path: head} } - return pathArm(name, b.key, b.tail) + return pathArm(name, b.head, b.tail) } segs, err := splitPath(name) if err != nil || len(segs) == 1 { @@ -371,7 +371,7 @@ type callFn func(s *session, emitted string, operands []string) string // re-scans the format. type op struct { formatToken - arms []arm // nameRead: the '|' alternatives, split into key and path once + arms []arm // nameRead: the '|' alternatives, split into head and tail once call callFn // operands are the fields the builtin reads, in the order its operands func // fixed; expand reads them before the call. nil for a builtin that reads none. diff --git a/node.go b/node.go index fc53753..9fc2867 100644 --- a/node.go +++ b/node.go @@ -74,7 +74,7 @@ const formatRow = -1 // templateLink is what a template resolves to in the assembled tree. type templateLink struct { refs map[string]refBinding // each reference the format reads -> what it resolves to - refHeads map[string]node // each refBinding.key -> the category it names + refHeads map[string]node // each refBinding.head -> the category it names readsColumn *columnRead // set when the format is one reference alone reading a record's column drawGroupKey string // its draw group keyed by its category: what a render reads its reference paths under } diff --git a/reference.go b/reference.go index 03e2902..62c477c 100644 --- a/reference.go +++ b/reference.go @@ -9,10 +9,10 @@ import ( // data root, {.a} from the folder this file sits in, {..a} from the folder above. func isRef(name string) bool { return strings.HasPrefix(name, ".") || strings.HasPrefix(name, "/") } -// refBinding is what a reference resolves to: the key its category is held +// refBinding is what a reference resolves to: the head its category is held // under, and the tail read into it. type refBinding struct { - key string + head string tail []string } @@ -119,15 +119,15 @@ func (t *template) resolveLink(folder []string, path, category string, root map[ if err != nil { return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } - key := "/" + strings.Join(categorySegs, ".") - if category != "" && key == "/"+category { + head := "/" + strings.Join(categorySegs, ".") + if category != "" && head == "/"+category { return link, fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) } - if err := checkPath(target, tail, key); err != nil { + if err := checkPath(target, tail, head); err != nil { return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } - link.refHeads[key] = target - link.refs[name] = refBinding{key, tail} + link.refHeads[head] = target + link.refs[name] = refBinding{head, tail} } link.readsColumn = columnReadOf(t, link) return link, nil diff --git a/shape_test.go b/shape_test.go index 010b2fe..50d48f2 100644 --- a/shape_test.go +++ b/shape_test.go @@ -102,7 +102,7 @@ func readsFact(n node) string { } case *template: for _, b := range n.link.refs { - set[strings.TrimPrefix(b.key, "/")] = true + set[strings.TrimPrefix(b.head, "/")] = true } for _, field := range n.fields { collect(field) -- 2.52.0 From 64705fcf0fd74766acbda80ababe5acbb6787f2d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:15:39 +0200 Subject: [PATCH 286/966] Name checkOwnFamily's found table familyTable --- drawfence.go | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drawfence.go b/drawfence.go index 42910b3..dba895f 100644 --- a/drawfence.go +++ b/drawfence.go @@ -90,18 +90,18 @@ func checkOwnFamily(t *template) error { seen[n] = true for _, e := range renderEdges(n) { if e.readsRef() { - if read, isTable := n.(*template).head(e.read.head).(*table); isTable && read.familyRoot() == own.familyRoot() { - return e, read, true + if familyTable, isTable := n.(*template).head(e.read.head).(*table); isTable && familyTable.familyRoot() == own.familyRoot() { + return e, familyTable, true } } - if e, head, found := find(e.to); found { - return e, head, true + if e, familyTable, found := find(e.to); found { + return e, familyTable, true } } return renderEdge{}, nil, false } - if e, head, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), head.segment, own.segment) + if e, familyTable, found := find(t); found { + return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), familyTable.segment, own.segment) } return nil } -- 2.52.0 From eebf2aa62bdfad066dc3b0a109158aef8e2d64d6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:30:13 +0200 Subject: [PATCH 287/966] Delete the comments and preambles the panel found empty --- Dockerfile | 6 ------ calc.go | 1 - compose.yaml | 16 ---------------- drawfence.go | 1 - format.go | 3 +-- node.go | 2 -- path.go | 4 ---- render.go | 5 ++--- table.go | 4 +--- todo.md | 7 +------ 10 files changed, 5 insertions(+), 44 deletions(-) diff --git a/Dockerfile b/Dockerfile index 9ab91ec..32b3220 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,9 +1,3 @@ -# The merge gate: `docker build .` fails if vet, complexity, formatting or tests -# fail, and CI runs exactly this. Day-to-day, prefer `docker compose run` -# (bind-mounts source, no rebuilds). GO_VERSION defaults to the latest stable Go; -# the lowest supported version builds the `portable` stage, which every supported -# Go must pass: -# docker build --build-arg GO_VERSION=1.22.12 --target portable . ARG GO_VERSION=1.27.1 FROM golang:${GO_VERSION} AS portable diff --git a/calc.go b/calc.go index 3b5e497..8179cfb 100644 --- a/calc.go +++ b/calc.go @@ -322,7 +322,6 @@ func (p *calcParser) binary(next func() (calcNode, error), ops ...byte) (calcNod } } -// factor is a table-shaped scanner, one case per token kind, kept whole on purpose. func (p *calcParser) factor() (calcNode, error) { p.space() if p.pos >= len(p.rs) { diff --git a/compose.yaml b/compose.yaml index 86595e4..91aee47 100644 --- a/compose.yaml +++ b/compose.yaml @@ -1,19 +1,3 @@ -# Docker-based dev workflow — no local tooling beyond Docker is required. -# -# docker compose run --rm test -# docker compose run --rm vet -# docker compose run --rm build -# -# The full gate — vet, formatting, cyclomatic complexity and tests — is -# `docker build .`, which is what CI runs. -# -# Build caches persist in the `gocache` volume, so re-runs are fast and the -# host tree stays clean. Commands that rewrite source (fmt, tidy) keep your -# ownership when run with `--user "$(id -u):$(id -g)"`. -# -# GO_VERSION picks the toolchain image (default: latest stable). Test the lowest -# supported Go with: GO_VERSION=1.22.12 docker compose run --rm test - x-go: &go image: golang:${GO_VERSION:-1.27.1} working_dir: /app diff --git a/drawfence.go b/drawfence.go index dba895f..27cef1a 100644 --- a/drawfence.go +++ b/drawfence.go @@ -187,5 +187,4 @@ func readsTable(n node, a arm) bool { return isTable } -// checkColumnDraws fences a record's columns as one render. func checkColumnDraws(t *template, columns []string) error { return surveyColumns(t, columns).check() } diff --git a/format.go b/format.go index 98ab09b..7cac363 100644 --- a/format.go +++ b/format.go @@ -15,8 +15,7 @@ type ftoken struct { // eachToken scans a format string once and calls fn for each unit, the single // source of truth for how braces are read: "{{" and "}}" are literal braces, a "{" -// opens a token that must reach its "}", and a lone "}" is an error. A table-shaped -// scanner, one case per rune kind, kept whole on purpose. +// opens a token that must reach its "}", and a lone "}" is an error. func eachToken(format string, fn func(ftoken) error) error { rs := []rune(format) for i := 0; i < len(rs); i++ { diff --git a/node.go b/node.go index 9fc2867..3315751 100644 --- a/node.go +++ b/node.go @@ -515,8 +515,6 @@ func checkPathNames(path string) error { return nil } -// isOption reports whether a template key configures the node instead of naming a -// field. These names can never be fields. func isOption(name string) bool { switch name { case "datatype", "drawGroup", "format", "repeat", "separator", "weight": diff --git a/path.go b/path.go index ead0746..4319091 100644 --- a/path.go +++ b/path.go @@ -99,7 +99,6 @@ func splitOutside(s string, c byte) []string { } } -// isSelector reports whether a segment is a [key or name] rather than a name. func isSelector(seg string) bool { return strings.HasPrefix(seg, "[") } func hasSelector(segs []string) bool { @@ -111,7 +110,6 @@ func hasSelector(segs []string) bool { return false } -// selectorOf is the key or name a selector segment holds. func selectorOf(seg string) string { return seg[1 : len(seg)-1] } // nameSegments is the segments of a path that are names, its selectors left out. @@ -470,8 +468,6 @@ func unreachableInChoice(c *choice, want string) error { return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } -// checkPath runs pathCheck over a path read at load; level names the head in its -// errors. func checkPath(n node, tail []string, level string) error { _, err := (&pathCheck{level: level, tail: tail}).run(n) return err diff --git a/render.go b/render.go index b8821e4..be3abb9 100644 --- a/render.go +++ b/render.go @@ -6,7 +6,8 @@ import ( ) // rng is the randomness a builtin sample draws from; *rand.Rand satisfies it. The -// render path takes the concrete *session instead. +// render path takes the concrete *session instead, which keeps the hold set of the walk +// drawing through it off the heap. type rng interface { IntN(n int) int Float64() float64 @@ -106,8 +107,6 @@ func expandAnew(s *session, t *template) string { // pick selects one item. Uniform choices are O(1); weighted choices are an // O(log n) search over precomputed cumulative weights. compile guarantees a // non-empty choice and a finite positive total, so the index is always in range. -// The session is concrete rather than the rng interface, which would make the -// walk that draws through it leak its hold set to the heap. func pick(s *session, c *choice) node { if c.cum == nil { return c.items[s.IntN(len(c.items))] diff --git a/table.go b/table.go index feee148..1e81db0 100644 --- a/table.go +++ b/table.go @@ -82,7 +82,6 @@ func isTableOption(name string) bool { // token arm. const inSelector = `[]{}"|` -// compileTable compiles a category object naming a rows file into a table. func compileTable(m map[string]any, segment string, files *categoryFiles) (*table, error) { o, err := readTableOptions(m) if err != nil { @@ -472,8 +471,7 @@ func (t *table) builtLookup() *rowLookup { return &t.lookup } -// drawRow picks a row over the whole table. The session is concrete rather than the rng -// interface so that the walk that draws through it keeps its hold set off the heap. +// drawRow picks a row over the whole table. func (t *table) drawRow(s *session) int { if t.cum == nil { return s.IntN(t.rowCount()) diff --git a/todo.md b/todo.md index e042842..d9f3dcb 100644 --- a/todo.md +++ b/todo.md @@ -28,11 +28,6 @@ ranking `readSurvey.walk` hardest and all seven the unit they would least modify This round, in order: -- Delete the prose the seats found empty: "kept whole on purpose" - (`format.go`, `calc.go`), the second copy of the concrete-session rationale - (`table.drawRow`, `pick`), the doc comments restating `isSelector`, `selectorOf`, - `checkPath`, `checkColumnDraws`, `compileTable` and `isOption`, and the `compose.yaml` - and `Dockerfile` preambles restating the README's Development. - Run the nine-seat panel again and file what it names here. ### Data @@ -313,7 +308,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. second copy that drifts. Split the comprehension rule while there, which packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line. - Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates - Go at `Dockerfile:21`, and the shipped JSON under `data/` is two-space, so the rule as + Go at `Dockerfile:16`, and the shipped JSON under `data/` is two-space, so the rule as written is one no Go file can break and every data file does. - Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub: Gitea converts a repository to a mirror only by re-creating it, so until that runs -- 2.52.0 From 21140042d34c20255bc1f9396f20cc059548c63f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:43:11 +0200 Subject: [PATCH 288/966] File the third 2026-09-29 nine-seat panel's findings, scored 6.1 --- todo.md | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index d9f3dcb..c91878e 100644 --- a/todo.md +++ b/todo.md @@ -24,10 +24,40 @@ Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming by Locality, all nine naming `drawWalk.walk` hardest and eight the unit they would least modify; 5.7 on 2026-09-29 again (Navigation 6.6, Locality 5.2, Shape 5.9, Self-sufficiency 5.8), four seats capped by Locality, six of the seven unit seats -ranking `readSurvey.walk` hardest and all seven the unit they would least modify. +ranking `readSurvey.walk` hardest and all seven the unit they would least modify; 6.1 +on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-sufficiency +6.1), every seat's lowest dimension Locality, all seven unit seats ranking the +`readSurvey` walk or check hardest and the unit they would least modify. This round, in order: +- Key the survey's visits and reads by a pin set's rows spelled as a string, and delete + `pinsID`, `pinsLink` and `readSurvey.pinIDs`: five seats stalled on the interning. + Start `replayPairs`' pin set as a clone of the earlier read's pins, deleting the + replay into an empty set, which cannot fail since `pinRow` built those pins. +- Move `checkOwnFamily` into `readsurvey.go` beside `checkFamilies`, and + `checkNestedDrawGroup` into `node.go` beside `drawGroupOf`, the compile step calling + it; delete `checkColumnDraws` for `surveyColumns(…).check()` at its caller. Both + architects found the family fence in two files, and draw-group checks of two + lifecycles in one. +- Replace `formatOps`' `held`, `holder` and `pathHeads` with one map from a held name + to its first reader and the path it starts, if any: four seats traced + `formatOps.holdName` writing four fields at once. +- Give `arm` its head as written, set where `splitArm` splits, and build + `pathProbe.readRow`'s selector spelling from it: six seats found the slice of + `arm.spelling` by `len(joinSegments(tail))` resting on an invariant nothing states. +- Pick `disagreement`'s fix through a named `itemKind` comparison: six seats could not + see that the kinds' declaration order is the rule. +- Give `tableSite` one test for a cell, and have `traceRead` report no row outside a + table: the zero value reads as row 0, a real cell, so each reader tests `table` first. +- Rename `ftoken` to `scanUnit`, beside `formatToken`, and `session` to + `generatorState`: three seats confused the token types, and both architects read + `session` as per-render state. +- State at `pinSet.pin` that its caller proved r sits inside the nearest pinned + ancestor, which `pinRow` and `drawIn` do. +- Delete the prose two or more seats found empty: `loadData`'s comment, which also names + `linkRefs` where `bind` runs, `table.drawRow`'s, `table.selector`'s, and `render.go`'s + second and third "compile validated" notes. - Run the nine-seat panel again and file what it names here. ### Data @@ -55,6 +85,8 @@ by a `data-import/` script, as README goal 12 asks. - Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too. - Fill the 398 Swedish localities weighted 200 from SCB småorter. +- List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list + of what each locale carries, which names neither. - Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default, per goal 14: 17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's @@ -200,6 +232,11 @@ by a `data-import/` script, as README goal 12 asks. `{/geo.US.locality[1714000].municipality.name}` holds once the step up above lands. Scope the README's "A reference path into a category is held" to a path that selects no row. +- Refuse a struct tag's `datatype` naming the Go type that sets it before proving its + values, as the README's Library section promises: `datatype: boolean` over + `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. +- Call `templateError` a compile failure: its doc says render failure, and it wraps + what `NewTemplate` and `NewRecordTemplate` refuse. - Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the first in Go's map order. -- 2.52.0 From 704837bf0b018c0dd324fdb2f7fd34491bbab024 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 10:52:38 +0200 Subject: [PATCH 289/966] File the one-loop path walk experiment as the next round item --- todo.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/todo.md b/todo.md index c91878e..1a414fe 100644 --- a/todo.md +++ b/todo.md @@ -31,6 +31,13 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: +- Collapse the four path walks into one loop switching on a plain mode field, check, + cover, probe or draw, with no call through an interface or a type parameter, and run + it against `perf_test.go`'s allocation gate. It holds: replace the decision "The four + path walks are four loops" with one recording a single loop, and land the collapse; + all nine seats named the four loops among the hardest. It fails: keep the loops, and + rewrite that decision's rationale to match `descend` and `pathProbe.readRow` today, + where `readRow` makes `drawn` on first use. - Key the survey's visits and reads by a pin set's rows spelled as a string, and delete `pinsID`, `pinsLink` and `readSurvey.pinIDs`: five seats stalled on the interning. Start `replayPairs`' pin set as a clone of the earlier read's pins, deleting the -- 2.52.0 From 79e63829c46a3e374564fcf61a04117e7bf142a3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:01:16 +0200 Subject: [PATCH 290/966] Keep the four path walks: one loop fails the allocation gate --- docs/decisions.md | 16 +++++++++------- todo.md | 7 ------- 2 files changed, 9 insertions(+), 14 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 22ffd7f..a4dcd47 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -676,10 +676,12 @@ though `hasRead` states the same stop rule. Valid while the two differ in where ## The four path walks are four loops 2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. -Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` each loop over -`stepInto` and `table.route` themselves, and `descend` builds its probe in place, so the -probe's `drawn` map stays on the stack; its caller always builds that map, so `readRow` -never tests it for nil. One walk calling them through an interface or a type parameter -leaks the walker and the pins it holds, an allocation per reference read. Valid while -Go's escape analysis treats a call through an interface or a type parameter as leaking -its receiver. +Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. +Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` stay four loops. +One loop fails `perf_test.go`: the check's leaves, the probe's selectors and the errors +naming a level leak the walker's contents, and Go tracks a struct's fields as one, so +the draw's pins, arm and hold leak with them. Held by pointer, a repeat of a reference +path rose from 66 to 106 allocations; copied into the walker by value, +`BenchmarkHeld` rose from 5 to 9. A call through an interface or a type parameter leaks +the walker the same way. Valid while Go's escape analysis tracks a struct's fields as +one and treats such a call as leaking its receiver. diff --git a/todo.md b/todo.md index 1a414fe..c91878e 100644 --- a/todo.md +++ b/todo.md @@ -31,13 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Collapse the four path walks into one loop switching on a plain mode field, check, - cover, probe or draw, with no call through an interface or a type parameter, and run - it against `perf_test.go`'s allocation gate. It holds: replace the decision "The four - path walks are four loops" with one recording a single loop, and land the collapse; - all nine seats named the four loops among the hardest. It fails: keep the loops, and - rewrite that decision's rationale to match `descend` and `pathProbe.readRow` today, - where `readRow` makes `drawn` on first use. - Key the survey's visits and reads by a pin set's rows spelled as a string, and delete `pinsID`, `pinsLink` and `readSurvey.pinIDs`: five seats stalled on the interning. Start `replayPairs`' pin set as a clone of the earlier read's pins, deleting the -- 2.52.0 From 4772fa7df3df3327530f718e65d0b18c7fb745d2 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:06:57 +0200 Subject: [PATCH 291/966] Key the survey by a pin set's rows spelled as a string --- pins.go | 13 ++++++++++++ readsurvey.go | 56 ++++++++++++--------------------------------------- todo.md | 4 ---- 3 files changed, 26 insertions(+), 47 deletions(-) diff --git a/pins.go b/pins.go index 498ffc6..0418ecb 100644 --- a/pins.go +++ b/pins.go @@ -4,6 +4,7 @@ import ( "fmt" "maps" "sort" + "strings" ) type tablePin struct { @@ -85,6 +86,18 @@ func (p *pinSet) each(fn func(t *table, r int)) { } } +// key spells the pinned rows, so two sets pinning the same rows share one. +func (p *pinSet) key() string { + var pins []tablePin + p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) + sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) + var b strings.Builder + for _, q := range pins { + fmt.Fprintf(&b, "%q%d", q.t.path, q.row) + } + return b.String() +} + // clash is the table whose pinned row keeps row r of t out: t itself pinned to another row, or the // nearest ancestor pinned to a row r is not inside; nil where none does. func (p *pinSet) clash(t *table, r int) *table { diff --git a/readsurvey.go b/readsurvey.go index 8f5106e..6c48de5 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -8,10 +8,9 @@ import ( // readSurvey gathers, at load, the references one render reads, by draw group, for check to compare. type readSurvey struct { - reads []pathRead - read map[readKey]bool - seen map[nodeVisit]bool - pinIDs map[pinsLink]pinsID + reads []pathRead + read map[readKey]bool + seen map[nodeVisit]bool } // surveyAt is where a survey stands: the draw group its reads draw in, how the render's root reached it, the @@ -39,28 +38,19 @@ type nodeVisit struct { n node group string wholeTable *table - pins pinsID - wholePins pinsID + pins string + wholePins string } type readKey struct { group string path string - pins pinsID - wholePins pinsID -} - -// pinsID names a pin set within one walk: two sets pinning the same rows share one. -type pinsID int - -// pinsLink is a set's last pin, in path order, and the id of the set before it. -type pinsLink struct { - prev pinsID - pin tablePin + pins string + wholePins string } func newReadSurvey() *readSurvey { - return &readSurvey{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}, pinIDs: map[pinsLink]pinsID{}} + return &readSurvey{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}} } func surveyRender(t *template) *readSurvey { @@ -79,27 +69,10 @@ func surveyColumns(t *template, columns []string) *readSurvey { return s } -func (s *readSurvey) pinsID(p *pinSet) pinsID { - var pins []tablePin - p.each(func(t *table, r int) { pins = append(pins, tablePin{t, r}) }) - sort.Slice(pins, func(i, j int) bool { return pins[i].t.path < pins[j].t.path }) - id := pinsID(0) - for _, q := range pins { - l := pinsLink{id, q} - next, ok := s.pinIDs[l] - if !ok { - next = pinsID(len(s.pinIDs) + 1) - s.pinIDs[l] = next - } - id = next - } - return id -} - // walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops // there. func (s *readSurvey) walk(n node, at surveyAt) { - v := nodeVisit{n, at.group, at.wholeTable, s.pinsID(&at.pins), s.pinsID(&at.wholePins)} + v := nodeVisit{n, at.group, at.wholeTable, at.pins.key(), at.wholePins.key()} if s.seen[v] { return } @@ -144,7 +117,7 @@ func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { if e.readsRef() { a := e.read tr := tableReadOf(from.(*template).head(a.head), a, e.to) - if k := (readKey{at.group, a.path, s.pinsID(&at.pins), s.pinsID(&at.wholePins)}); !s.read[k] { + if k := (readKey{at.group, a.path, at.pins.key(), at.wholePins.key()}); !s.read[k] { s.read[k] = true s.reads = append(s.reads, pathRead{at, a, tr}) } @@ -301,8 +274,8 @@ func checkFamilyPair(a, b pathRead) error { return nil } -// replayPairs replays every two reads that can render together into one draws, the earlier -// read first. Pairs find every conflict a full replay would: clash judges a row against one +// replayPairs replays, of every two reads that can render together, the later into the +// earlier's pins. Pairs find every conflict a full replay would: clash judges a row against one // pinned table, and the read that pinned it holds that pin itself, since a read's pins carry // its rows' ancestors, so the pair of those two reads clashes the same way. func replayPairs(reads []pathRead) error { @@ -314,10 +287,7 @@ func replayPairs(reads []pathRead) error { if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { continue } - var d pinSet - if err := r.tr.replay(&d); err != nil { - return conflict(r, err) - } + d := r.tr.pins.clone() if err := o.tr.replay(&d); err != nil { return conflict(o, err) } diff --git a/todo.md b/todo.md index c91878e..05a2df2 100644 --- a/todo.md +++ b/todo.md @@ -31,10 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Key the survey's visits and reads by a pin set's rows spelled as a string, and delete - `pinsID`, `pinsLink` and `readSurvey.pinIDs`: five seats stalled on the interning. - Start `replayPairs`' pin set as a clone of the earlier read's pins, deleting the - replay into an empty set, which cannot fail since `pinRow` built those pins. - Move `checkOwnFamily` into `readsurvey.go` beside `checkFamilies`, and `checkNestedDrawGroup` into `node.go` beside `drawGroupOf`, the compile step calling it; delete `checkColumnDraws` for `surveyColumns(…).check()` at its caller. Both -- 2.52.0 From 7baee64cc300bb5ec251a423d0dc13f9d8dcf903 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:13:33 +0200 Subject: [PATCH 292/966] Move each draw fence beside the check it pairs with, and inline checkColumnDraws --- README.md | 6 ++--- drawfence.go | 69 +-------------------------------------------------- node.go | 30 ++++++++++++++++++++++ readsurvey.go | 35 ++++++++++++++++++++++++++ todo.md | 7 +----- 5 files changed, 70 insertions(+), 77 deletions(-) diff --git a/README.md b/README.md index 5867104..9b865f9 100644 --- a/README.md +++ b/README.md @@ -875,7 +875,7 @@ the Development section below, and who ships a register the five above then draw ``` doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, the embedded data set, List -node.go the node model and JSON -> node compilation +node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make @@ -885,8 +885,8 @@ inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplat format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format hold.go the hold: one draw per held name; and the hold set and scope a render reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways -drawfence.go the load fences over reference draws: a drawGroup's, a table's own family, and a render's and a record's columns through the survey -readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them +drawfence.go the load fences over reference draws: a drawGroup's, and a render's and a record's columns through the survey +readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them, a table reaching its own family included pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/drawfence.go b/drawfence.go index 27cef1a..8f46f5d 100644 --- a/drawfence.go +++ b/drawfence.go @@ -2,36 +2,6 @@ package fejkdata import "fmt" -// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, -// short of a repeat or another draw group. -func checkNestedDrawGroup(fields map[string]node, group string) error { - if group == "" { - return nil - } - var walk func(path string, n node) error - walk = func(path string, n node) error { - t, isTemplate := n.(*template) - switch { - case isTemplate && t.drawGroup == group: - return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group) - case isTemplate && (t.drawGroup != "" || t.repeat > 1): - return nil - } - for _, c := range contained(n) { - if err := walk(join(path, c.name), c.node); err != nil { - return err - } - } - return nil - } - for _, name := range sortedNames(fields) { - if err := walk(name, fields[name]); err != nil { - return err - } - } - return nil -} - // drawFence fences each template of a scope as a render of its own, remembering which nodes read a // reference path. // docs/decisions.md#a-render-shares-one-reference-draw-per-category-per-group @@ -71,41 +41,6 @@ func (f *drawFence) checkDraws(path string, n node) error { return nil } -// checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a whole -// read draws its row without pinning it, so the family would draw apart from the row -// it renders, whichever draws the reaching template holds. -// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route -func checkOwnFamily(t *template) error { - own := t.site.table - if own == nil { - return nil - } - seen := map[node]bool{} - var find func(n node) (renderEdge, *table, bool) - find = func(n node) (renderEdge, *table, bool) { - if seen[n] { - return renderEdge{}, nil, false - } - seen[n] = true - for _, e := range renderEdges(n) { - if e.readsRef() { - if familyTable, isTable := n.(*template).head(e.read.head).(*table); isTable && familyTable.familyRoot() == own.familyRoot() { - return e, familyTable, true - } - } - if e, familyTable, found := find(e.to); found { - return e, familyTable, true - } - } - return renderEdge{}, nil, false - } - if e, familyTable, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), familyTable.segment, own.segment) - } - return nil -} - // checkRecordDraws fences the columns of a record — a template compiled at the top without a // repeat — so a load proves the record view of it as well as the string view. // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load @@ -123,7 +58,7 @@ func (f *drawFence) checkRecordDraws(path string, n node) error { if !reads { return nil } - if err := checkColumnDraws(t, columns); err != nil { + if err := surveyColumns(t, columns).check(); err != nil { return fmt.Errorf("%s: %w", path, err) } return nil @@ -186,5 +121,3 @@ func readsTable(n node, a arm) bool { _, isTable := t.head(a.head).(*table) return isTable } - -func checkColumnDraws(t *template, columns []string) error { return surveyColumns(t, columns).check() } diff --git a/node.go b/node.go index 3315751..ebb52b2 100644 --- a/node.go +++ b/node.go @@ -427,6 +427,36 @@ func repeatOf(m map[string]any) (int, error) { return int(r), nil } +// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, +// short of a repeat or another draw group. +func checkNestedDrawGroup(fields map[string]node, group string) error { + if group == "" { + return nil + } + var walk func(path string, n node) error + walk = func(path string, n node) error { + t, isTemplate := n.(*template) + switch { + case isTemplate && t.drawGroup == group: + return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group) + case isTemplate && (t.drawGroup != "" || t.repeat > 1): + return nil + } + for _, c := range contained(n) { + if err := walk(join(path, c.name), c.node); err != nil { + return err + } + } + return nil + } + for _, name := range sortedNames(fields) { + if err := walk(name, fields[name]); err != nil { + return err + } + } + return nil +} + // drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each // iteration renders in no draw group. func drawGroupOf(m map[string]any, repeat int) (string, error) { diff --git a/readsurvey.go b/readsurvey.go index 6c48de5..e08dba1 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -248,6 +248,41 @@ func checkFamilies(reads []pathRead) error { return replayPairs(reads) } +// checkOwnFamily refuses a table's format or cell that reads, however many templates +// away and through a repeat or a draw group too, a table of its own family: a whole +// read draws its row without pinning it, so the family would draw apart from the row +// it renders, whichever draws the reaching template holds. +// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route +func checkOwnFamily(t *template) error { + own := t.site.table + if own == nil { + return nil + } + seen := map[node]bool{} + var find func(n node) (renderEdge, *table, bool) + find = func(n node) (renderEdge, *table, bool) { + if seen[n] { + return renderEdge{}, nil, false + } + seen[n] = true + for _, e := range renderEdges(n) { + if e.readsRef() { + if familyTable, isTable := n.(*template).head(e.read.head).(*table); isTable && familyTable.familyRoot() == own.familyRoot() { + return e, familyTable, true + } + } + if e, familyTable, found := find(e.to); found { + return e, familyTable, true + } + } + return renderEdge{}, nil, false + } + if e, familyTable, found := find(t); found { + return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), familyTable.segment, own.segment) + } + return nil +} + // checkFamilyPair refuses a table read whole beside a path into its family, and a // table one read draws that the other pins, since which token renders first would // then decide the row. diff --git a/todo.md b/todo.md index 05a2df2..793ffdf 100644 --- a/todo.md +++ b/todo.md @@ -31,11 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Move `checkOwnFamily` into `readsurvey.go` beside `checkFamilies`, and - `checkNestedDrawGroup` into `node.go` beside `drawGroupOf`, the compile step calling - it; delete `checkColumnDraws` for `surveyColumns(…).check()` at its caller. Both - architects found the family fence in two files, and draw-group checks of two - lifecycles in one. - Replace `formatOps`' `held`, `holder` and `pathHeads` with one map from a held name to its first reader and the path it starts, if any: four seats traced `formatOps.holdName` writing four fields at once. @@ -332,7 +327,7 @@ because pairing a street with its exact postnummer rewrites shipped rows. records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. - Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`, - `checkOwnFamily` and `checkColumnDraws`; the README's Layout indexes the fences by file. + `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by file. Say "fences" in the `pin` entry then, which names the draw fences too. - Cut the README's Layout block to the lines that say what a file name cannot: `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -- 2.52.0 From 1b219e333affca393a827c67cb234feea2760cb6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:14:43 +0200 Subject: [PATCH 293/966] Word readsurvey.go's Layout line so the own-family fence is not the survey's verdict --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9b865f9..7e46d43 100644 --- a/README.md +++ b/README.md @@ -886,7 +886,7 @@ format.go the {token} grammar: scanning, tokens, operands, validation, com hold.go the hold: one draw per held name; and the hold set and scope a render reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways drawfence.go the load fences over reference draws: a drawGroup's, and a render's and a record's columns through the survey -readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them, a table reaching its own family included +readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them; and the fence on a table reaching its own family pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences -- 2.52.0 From bb70688f1d558e9208547260dc4b37b04b63ca2b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:20:09 +0200 Subject: [PATCH 294/966] Hold each name in one formatOps map, beside its first reader and path --- format.go | 34 ++++++++++++++++------------------ holdfence.go | 31 ++++++++++++++++--------------- node.go | 2 +- render.go | 2 +- todo.md | 3 --- 5 files changed, 34 insertions(+), 38 deletions(-) diff --git a/format.go b/format.go index 7cac363..2d25a8a 100644 --- a/format.go +++ b/format.go @@ -378,38 +378,36 @@ type op struct { } // formatOps is a compiled format: its ops, the size of its literal text, and the -// names it holds. pathHeads maps each head a path starts from to the first such path; -// held is every such head plus the fields an operand reads; holder maps each held -// name to the first reader holding it, for error messages. +// names it holds: every head a path starts from, plus the fields an operand reads. type formatOps struct { ops []op grow int - pathHeads map[string]string - held map[string]bool - holder map[string]string + held map[string]heldName heldLocal bool // some held name is not a reference path, so expand makes a hold } +// heldName is how a format first reaches a held name: the reader holding it, for +// error messages, and the first path starting from it, "" where none does. +type heldName struct { + holder string + path string +} + func (c *formatOps) holdName(a arm, label string) { if c.held == nil { - c.held = map[string]bool{} - c.holder = map[string]string{} + c.held = map[string]heldName{} } - c.held[a.head] = true if !isRef(a.head) || len(a.tail) == 0 { c.heldLocal = true } - if _, named := c.holder[a.head]; !named { - c.holder[a.head] = label + h, named := c.held[a.head] + if !named { + h.holder = label } - if len(a.tail) > 0 { - if c.pathHeads == nil { - c.pathHeads = map[string]string{} - } - if _, named := c.pathHeads[a.head]; !named { - c.pathHeads[a.head] = a.spelling - } + if len(a.tail) > 0 && h.path == "" { + h.path = a.spelling } + c.held[a.head] = h } func (c *formatOps) function(tok formatToken, refs map[string]refBinding) { diff --git a/holdfence.go b/holdfence.go index 534e547..ca30e1a 100644 --- a/holdfence.go +++ b/holdfence.go @@ -15,9 +15,9 @@ func heldCheck(path string, n node) error { if !ok || len(t.compiled.held) == 0 { return nil } - readers := pathHeadReaders(t.compiled.ops, t.compiled.pathHeads) + readers := pathHeadReaders(t.compiled) for _, name := range heldNames(t) { - if _, isPath := t.compiled.pathHeads[name]; isPath && isRef(name) { + if t.compiled.held[name].path != "" && isRef(name) { continue // held for the render: drawFence compares every read of it, by path, across the render and its groups } if err := checkNameHeld(t, name, readers); err != nil { @@ -36,8 +36,8 @@ func heldNames(t *template) []string { names = append(names, name) } sort.Slice(names, func(i, j int) bool { - _, pi := t.compiled.pathHeads[names[i]] - _, pj := t.compiled.pathHeads[names[j]] + pi := t.compiled.held[names[i]].path != "" + pj := t.compiled.held[names[j]].path != "" if pi != pj { return !pi } @@ -51,7 +51,7 @@ func heldNames(t *template) []string { // produces. func heldNodes(t *template, name string, readers []reader) map[node]bool { held := map[node]bool{} - if _, isPath := t.compiled.pathHeads[name]; !isPath { + if t.compiled.held[name].path == "" { operandDraw(t.head(name), held) return held } @@ -71,13 +71,13 @@ func checkNameHeld(t *template, name string, readers []reader) error { if len(held) == 0 { return nil // a fixed head holds nothing to reach } - reader, isPath := t.compiled.pathHeads[name] + reader := t.compiled.held[name].path seen := map[node]bool{} for _, e := range renderEdges(t) { if e.read.head == name || !renders(e.to, held, seen) { continue } - if isPath { + if reader != "" { return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), name, reader) } return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), name, operandReader(t, name)) @@ -175,8 +175,8 @@ func renders(n node, want, seen map[node]bool) bool { // disagree. Reads are compared in sorted order, so which pair is reported does not // depend on where the tokens sit. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkNoOverlap(ops []op, pathHeads map[string]string) error { - names := pathHeadReaders(ops, pathHeads) +func checkNoOverlap(c formatOps) error { + names := pathHeadReaders(c) // Stable over one format-order scan, so two readers of one name (a token and a // calc operand both naming "p") are reported as the format writes them. sort.SliceStable(names, func(i, j int) bool { return names[i].a.path < names[j].a.path }) @@ -199,16 +199,16 @@ type reader struct { // pathHeadReaders lists every way a format reaches a sibling path head, in the order the // format writes them. An operand renders its field, so it names a level exactly // as a token does; one scan finds both, which is what puts them in one order. -func pathHeadReaders(ops []op, pathHeads map[string]string) []reader { +func pathHeadReaders(c formatOps) []reader { var names []reader - for _, o := range ops { + for _, o := range c.ops { for _, a := range o.operands { - if _, isPathHead := pathHeads[a.head]; isPathHead && !isRef(a.head) { + if c.held[a.head].path != "" && !isRef(a.head) { names = append(names, reader{a, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { - if _, isPathHead := pathHeads[a.head]; isPathHead && !isRef(a.head) { + if c.held[a.head].path != "" && !isRef(a.head) { names = append(names, reader{a, "token {" + a.spelling + "}"}) } } @@ -224,11 +224,12 @@ func checkNoRepeatedRead(c formatOps) error { count := map[string]int{} for _, o := range c.ops { for _, a := range o.arms { - if len(a.tail) > 0 || !c.held[a.head] { + h, held := c.held[a.head] + if len(a.tail) > 0 || !held { continue } if count[a.head]++; count[a.head] > 1 { - return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.spelling, c.holder[a.head], a.head, a.spelling) + return fmt.Errorf("token {%s} is repeated, and %s holds %q to one draw per expansion; write {%s} once", a.spelling, h.holder, a.head, a.spelling) } } } diff --git a/node.go b/node.go index ebb52b2..03542a1 100644 --- a/node.go +++ b/node.go @@ -229,7 +229,7 @@ func (t *template) fixedText() (string, bool) { // compiled reads. func compileFormat(toks []formatToken, refs map[string]refBinding) (formatOps, error) { c := compileOps(toks, refs) - if err := checkNoOverlap(c.ops, c.pathHeads); err != nil { + if err := checkNoOverlap(c); err != nil { return c, err } return c, checkNoRepeatedRead(c) diff --git a/render.go b/render.go index be3abb9..ce34353 100644 --- a/render.go +++ b/render.go @@ -164,7 +164,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV if sc.set.trace != nil { traceRead(sc.set.trace, t, sc, a) } - if !t.compiled.held[a.head] { + if _, expansionHolds := t.compiled.held[a.head]; !expansionHolds { if len(a.tail) > 0 { panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.head)) } diff --git a/todo.md b/todo.md index 793ffdf..aac1e3b 100644 --- a/todo.md +++ b/todo.md @@ -31,9 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Replace `formatOps`' `held`, `holder` and `pathHeads` with one map from a held name - to its first reader and the path it starts, if any: four seats traced - `formatOps.holdName` writing four fields at once. - Give `arm` its head as written, set where `splitArm` splits, and build `pathProbe.readRow`'s selector spelling from it: six seats found the slice of `arm.spelling` by `len(joinSegments(tail))` resting on an invariant nothing states. -- 2.52.0 From bad15a5bdb21690018847e9823aecc16f7ea130d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:22:29 +0200 Subject: [PATCH 295/966] Name formatOps' held entry firstReach, and checkNameHeld's path spelling path --- format.go | 12 ++++++------ holdfence.go | 6 +++--- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/format.go b/format.go index 2d25a8a..97d2704 100644 --- a/format.go +++ b/format.go @@ -382,26 +382,26 @@ type op struct { type formatOps struct { ops []op grow int - held map[string]heldName + held map[string]firstReach heldLocal bool // some held name is not a reference path, so expand makes a hold } -// heldName is how a format first reaches a held name: the reader holding it, for +// firstReach is how a format first reaches a held name: the reader holding it, for // error messages, and the first path starting from it, "" where none does. -type heldName struct { +type firstReach struct { holder string path string } func (c *formatOps) holdName(a arm, label string) { if c.held == nil { - c.held = map[string]heldName{} + c.held = map[string]firstReach{} } if !isRef(a.head) || len(a.tail) == 0 { c.heldLocal = true } - h, named := c.held[a.head] - if !named { + h, seen := c.held[a.head] + if !seen { h.holder = label } if len(a.tail) > 0 && h.path == "" { diff --git a/holdfence.go b/holdfence.go index ca30e1a..b754fdd 100644 --- a/holdfence.go +++ b/holdfence.go @@ -71,14 +71,14 @@ func checkNameHeld(t *template, name string, readers []reader) error { if len(held) == 0 { return nil // a fixed head holds nothing to reach } - reader := t.compiled.held[name].path + path := t.compiled.held[name].path seen := map[node]bool{} for _, e := range renderEdges(t) { if e.read.head == name || !renders(e.to, held, seen) { continue } - if reader != "" { - return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), name, reader) + if path != "" { + return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), name, path) } return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), name, operandReader(t, name)) } -- 2.52.0 From df4d4001f7456b20db7db6be7ac36130a88f9e37 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:39:14 +0200 Subject: [PATCH 296/966] Test a relative reference's selector spelling in the selection fence's rewrite --- table_test.go | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/table_test.go b/table_test.go index 22c03f6..8c4ab5e 100644 --- a/table_test.go +++ b/table_test.go @@ -367,11 +367,12 @@ func TestTableSelectorInAReference(t *testing.T) { func TestTableSelectionFences(t *testing.T) { rejected := map[string]struct{ json, want string }{ - "bare beside a path": {`"{/region} {/region.name}"`, "reads a path into"}, - "bare beside a linked path": {`"{/region} {/locality.name}"`, "drawGroup"}, - "selected beside unselected": {`"{/region[12].municipality.name} {/municipality.code}"`, "{/region[12].municipality.code}"}, - "two selections": {`"{/region[12].name} {/region[14].name}"`, "drawGroup"}, - "selection beside a descendant": {`"{/region[12].name} {/locality.code}"`, "{/region[12].locality.code}"}, + "bare beside a path": {`"{/region} {/region.name}"`, "reads a path into"}, + "bare beside a linked path": {`"{/region} {/locality.name}"`, "drawGroup"}, + "selected beside unselected": {`"{/region[12].municipality.name} {/municipality.code}"`, "{/region[12].municipality.code}"}, + "relative selected beside unselected": {`"{.region[12].municipality.name} {.municipality.code}"`, "{.region[12].municipality.code}"}, + "two selections": {`"{/region[12].name} {/region[14].name}"`, "drawGroup"}, + "selection beside a descendant": {`"{/region[12].name} {/locality.code}"`, "{/region[12].locality.code}"}, } for name, c := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, with(geo(), map[string]string{"x.json": c.json})))) -- 2.52.0 From 6b17205159e562afc1ef96ebe95dd5da8ac014d9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:39:14 +0200 Subject: [PATCH 297/966] Give arm its head as written, and spell readRow's selector from it --- format.go | 21 ++++++++++++--------- path.go | 3 +-- todo.md | 3 --- 3 files changed, 13 insertions(+), 14 deletions(-) diff --git a/format.go b/format.go index 97d2704..415d80e 100644 --- a/format.go +++ b/format.go @@ -306,11 +306,12 @@ func checkNoRepeatedArm(body string, names []string) error { // is what makes the arm a held draw: its head is drawn once per expansion (see // compileOps). type arm struct { - spelling string // as written, for messages - head string - tail []string - levels []string // the path at each level the tail passes through, the head first - path string // head and tail, the one path every way of writing this read shares + spelling string // as written, for messages + head string + writtenHead string // head as written, sigil included + tail []string + levels []string // the path at each level the tail passes through, the head first + path string // head and tail, the one path every way of writing this read shares } // splitArm splits one name into head and tail. refs maps a reference to what @@ -325,21 +326,23 @@ func splitArm(name string, refs map[string]refBinding) arm { } return arm{spelling: name, head: head, path: head} } - return pathArm(name, b.head, b.tail) + sigil, rest, _ := refShape(name) // linkRefs proved it + written, _ := splitPath(rest) + return pathArm(name, b.head, sigil+joinSegments(written[:len(written)-len(b.tail)]), b.tail) } segs, err := splitPath(name) if err != nil || len(segs) == 1 { return arm{spelling: name, head: name, path: name} } - return pathArm(name, segs[0], segs[1:]) + return pathArm(name, segs[0], segs[0], segs[1:]) } -func pathArm(name, head string, segs []string) arm { +func pathArm(name, head, writtenHead string, segs []string) arm { levels := []string{head} for i := 0; i < len(segs)-1; i++ { levels = append(levels, head+"."+strings.Join(segs[:i+1], ".")) } - return arm{spelling: name, head: head, tail: segs, levels: levels, path: head + "." + strings.Join(segs, ".")} + return arm{spelling: name, head: head, writtenHead: writtenHead, tail: segs, levels: levels, path: head + "." + strings.Join(segs, ".")} } // checkSegments rejects an unfinished path: "{a.}" and "{a..b}" each have a diff --git a/path.go b/path.go index 4319091..b8b253f 100644 --- a/path.go +++ b/path.go @@ -333,8 +333,7 @@ func (w *pathProbe) readRow(t *table, tail []string, r tableRoute) error { case r.sel != "": if w.a != nil { walked := len(w.a.tail) - len(tail) + 1 - written := w.a.spelling[:len(w.a.spelling)-len(joinSegments(w.a.tail))] - w.sels = append(w.sels, tableSel{t, written + joinSegments(w.a.tail[:walked])}) + w.sels = append(w.sels, tableSel{t, joinSegments(append([]string{w.a.writtenHead}, w.a.tail[:walked]...))}) } return w.pins.selectRow(t, r.sel) case r.draw && w.a != nil: diff --git a/todo.md b/todo.md index aac1e3b..f27cfca 100644 --- a/todo.md +++ b/todo.md @@ -31,9 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Give `arm` its head as written, set where `splitArm` splits, and build - `pathProbe.readRow`'s selector spelling from it: six seats found the slice of - `arm.spelling` by `len(joinSegments(tail))` resting on an invariant nothing states. - Pick `disagreement`'s fix through a named `itemKind` comparison: six seats could not see that the kinds' declaration order is the rule. - Give `tableSite` one test for a cell, and have `traceRead` report no row outside a -- 2.52.0 From b75e517bda626fade0d7e9dbcbe39f7e3ec11da1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:42:43 +0200 Subject: [PATCH 298/966] Name resolveLink as what proves splitArm's reference, and the tail it takes --- format.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/format.go b/format.go index 415d80e..ae043e3 100644 --- a/format.go +++ b/format.go @@ -326,7 +326,7 @@ func splitArm(name string, refs map[string]refBinding) arm { } return arm{spelling: name, head: head, path: head} } - sigil, rest, _ := refShape(name) // linkRefs proved it + sigil, rest, _ := refShape(name) // resolveLink proved it, and took b.tail as a suffix of its segments written, _ := splitPath(rest) return pathArm(name, b.head, sigil+joinSegments(written[:len(written)-len(b.tail)]), b.tail) } -- 2.52.0 From a4547472ec4d1ebb979fa8cff0bf5ae10fedff65 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:56:51 +0200 Subject: [PATCH 299/966] Pick disagreement's fix through itemKind.declaresLess --- datatype.go | 7 ++++++- todo.md | 2 -- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/datatype.go b/datatype.go index 9930975..f04ce26 100644 --- a/datatype.go +++ b/datatype.go @@ -148,6 +148,7 @@ func columnItems(n node) (items []*template, nullable bool) { } // itemKind is how a column item comes by its datatype: none, read from a typed column, or declared. +// The kinds are declared from the one that declares least. type itemKind int const ( @@ -156,6 +157,10 @@ const ( kindDeclares ) +func (k itemKind) declaresLess(than itemKind) bool { + return k < than +} + func kindOf(t *template) itemKind { switch { case t.datatype != DataTypeString: @@ -178,7 +183,7 @@ type itemPair struct { // the one that declares least, a when both declare alike. func disagreement(a *template, da DataType, b *template, db DataType) error { d := itemPair{first: da, second: db, fix: a, other: b, want: db} - if kindOf(b) < kindOf(a) { + if kindOf(b).declaresLess(kindOf(a)) { d.fix, d.other, d.want = b, a, da } proven := (&valueProof{}).proveColumnItem(d.fix).not[d.want] == "" diff --git a/todo.md b/todo.md index f27cfca..84fa028 100644 --- a/todo.md +++ b/todo.md @@ -31,8 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Pick `disagreement`'s fix through a named `itemKind` comparison: six seats could not - see that the kinds' declaration order is the rule. - Give `tableSite` one test for a cell, and have `traceRead` report no row outside a table: the zero value reads as row 0, a real cell, so each reader tests `table` first. - Rename `ftoken` to `scanUnit`, beside `formatToken`, and `session` to -- 2.52.0 From d50136223a8e60931de74a1b81e127721524d3aa Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 11:58:44 +0200 Subject: [PATCH 300/966] Rank itemKind through a free declaresLess, named in the type's comment --- datatype.go | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/datatype.go b/datatype.go index f04ce26..07005f0 100644 --- a/datatype.go +++ b/datatype.go @@ -147,8 +147,7 @@ func columnItems(n node) (items []*template, nullable bool) { return items, nullable } -// itemKind is how a column item comes by its datatype: none, read from a typed column, or declared. -// The kinds are declared from the one that declares least. +// itemKind is how a column item comes by its datatype, ranked by declaresLess in this order: none, read from a typed column, or declared. type itemKind int const ( @@ -157,8 +156,8 @@ const ( kindDeclares ) -func (k itemKind) declaresLess(than itemKind) bool { - return k < than +func declaresLess(a, b itemKind) bool { + return a < b } func kindOf(t *template) itemKind { @@ -183,7 +182,7 @@ type itemPair struct { // the one that declares least, a when both declare alike. func disagreement(a *template, da DataType, b *template, db DataType) error { d := itemPair{first: da, second: db, fix: a, other: b, want: db} - if kindOf(b).declaresLess(kindOf(a)) { + if declaresLess(kindOf(b), kindOf(a)) { d.fix, d.other, d.want = b, a, da } proven := (&valueProof{}).proveColumnItem(d.fix).not[d.want] == "" -- 2.52.0 From 4fcf45b5d0312e91c4fdf548cf5438d8a6537d5d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:12:42 +0200 Subject: [PATCH 301/966] Trace a render's read with the row it renders, none outside a table --- drawfence_test.go | 14 +++++--------- readsurvey_test.go | 20 ++++++++++---------- 2 files changed, 15 insertions(+), 19 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index 83ffe6f..701a3a4 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -24,16 +24,12 @@ type fenceRoot struct { t *template } -func fenceRoots(t *testing.T, f *Generator) ([]fenceRoot, map[string]*table) { +func fenceRoots(t *testing.T, f *Generator) []fenceRoot { t.Helper() var roots []fenceRoot - tables := map[string]*table{} _ = walkNodes(f.root.children, func(path string, n node) error { - switch n := n.(type) { - case *template: - roots = append(roots, fenceRoot{path, n}) - case *table: - tables[n.path] = n + if tm, ok := n.(*template); ok { + roots = append(roots, fenceRoot{path, tm}) } return nil }) @@ -51,12 +47,12 @@ func fenceRoots(t *testing.T, f *Generator) ([]fenceRoot, map[string]*table) { } return nil }) - return append(roots, fenceRoot{"inline record", record.template}), tables + return append(roots, fenceRoot{"inline record", record.template}) } func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { corpus := fenceCorpus(t) - roots, _ := fenceRoots(t, corpus) + roots := fenceRoots(t, corpus) shipped, err := New() if err != nil { t.Fatalf("New() = %v", err) diff --git a/readsurvey_test.go b/readsurvey_test.go index 5e67d68..992fd11 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -8,10 +8,10 @@ import ( func TestEveryReadARenderMakesIsGathered(t *testing.T) { f := fenceCorpus(t) - roots, tables := fenceRoots(t, f) + roots := fenceRoots(t, f) traced := 0 for _, root := range roots { - traced += wantGathered(t, tables, root.label, surveyRender(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) + traced += wantGathered(t, root.label, surveyRender(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) if !root.t.isRecord || len(root.t.fields) == 0 { continue } @@ -19,30 +19,30 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { if err != nil { t.Fatalf("%s: recordOf = %v", root.label, err) } - traced += wantGathered(t, tables, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) + traced += wantGathered(t, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) } if traced == 0 { t.Error("no render traced a reference read, so the comparison proved nothing") } } -func wantGathered(t *testing.T, tables map[string]*table, label string, gathered []pathRead, renderWith func(renderTrace)) int { +func wantGathered(t *testing.T, label string, gathered []pathRead, renderWith func(renderTrace)) int { t.Helper() var reads []pathRead type traceKey struct { - group, path, table string - row int + group, path string + row renderedRow } seen := map[traceKey]bool{} - trace := func(group, table string, row int, a arm) { + trace := func(group string, row renderedRow, a arm) { if !isRef(a.head) { return } r := pathRead{at: surveyAt{group: group}, a: a} - if table != "" { - r.at.pins.add(tables[table], row) + if row.t != nil { + r.at.pins.add(row.t, row.index) } - if k := (traceKey{group, a.path, table, row}); !seen[k] { + if k := (traceKey{group, a.path, row}); !seen[k] { seen[k] = true reads = append(reads, r) } -- 2.52.0 From 8793681d8abae0b142d1f050ea069cdac5974291 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:13:33 +0200 Subject: [PATCH 302/966] Test a table cell through tableSite.isCell, and trace no row outside a table --- hold.go | 2 +- node.go | 4 ++++ readsurvey.go | 6 +++--- reference.go | 2 +- render.go | 15 +++++++-------- todo.md | 2 -- 6 files changed, 16 insertions(+), 15 deletions(-) diff --git a/hold.go b/hold.go index dfaf116..e63e517 100644 --- a/hold.go +++ b/hold.go @@ -24,7 +24,7 @@ type readValue struct { } // renderTrace is a test's view of the fields a render reads; nil outside a test. -type renderTrace func(group, table string, row int, a arm) +type renderTrace func(group string, row renderedRow, a arm) // holdSet is one render's reference draws: the unnamed draw group's, and each named one's; and a // test's trace of its reads. diff --git a/node.go b/node.go index 03542a1..c10069b 100644 --- a/node.go +++ b/node.go @@ -71,6 +71,10 @@ type tableSite struct { const formatRow = -1 +func (s tableSite) isCell() bool { + return s.table != nil && s.row != formatRow +} + // templateLink is what a template resolves to in the assembled tree. type templateLink struct { refs map[string]refBinding // each reference the format reads -> what it resolves to diff --git a/readsurvey.go b/readsurvey.go index e08dba1..531e8c7 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -88,12 +88,12 @@ func (s *readSurvey) walk(n node, at surveyAt) { } for _, e := range renderEdges(n) { to, isTemplate := e.to.(*template) - cell := tableSite{} - if isTemplate && to.site.row != formatRow { + var cell tableSite + if isTemplate { cell = to.site } switch { - case cell.table == nil: + case !cell.isCell(): s.edge(n, e, at) case cell.table == at.wholeTable: in := at diff --git a/reference.go b/reference.go index 62c477c..861104e 100644 --- a/reference.go +++ b/reference.go @@ -81,7 +81,7 @@ func linkRefs(root map[string]node) error { // linkTemplate resolves t's references in category, "" for an inline template, and // compiles its format against them. func linkTemplate(folder []string, path, category string, t *template, root map[string]node) error { - if t.site.table != nil && t.site.row != formatRow { + if t.site.isCell() { path = fmt.Sprintf("%s, line %d", path, t.site.row+2) } link, err := t.resolveLink(folder, path, category, root) diff --git a/render.go b/render.go index ce34353..2c5d8db 100644 --- a/render.go +++ b/render.go @@ -203,15 +203,14 @@ func readHeld(s *session, t *template, held *hold, pins *pinSet, sc renderScope, } func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { - var table string - row := t.site.row - if t.site.table != nil { - table = t.site.table.path + var row renderedRow + switch { + case t.site.isCell(): + row = renderedRow{t.site.table, t.site.row} + case t.site.table != nil: + row = renderedRow{t.site.table, sc.rowOf(t.site.table)} } - if row == formatRow { - row = sc.rowOf(t.site.table) - } - trace(strings.Clone(sc.group), strings.Clone(table), row, a) + trace(strings.Clone(sc.group), row, a) } // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one diff --git a/todo.md b/todo.md index 84fa028..5215fb9 100644 --- a/todo.md +++ b/todo.md @@ -31,8 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Give `tableSite` one test for a cell, and have `traceRead` report no row outside a - table: the zero value reads as row 0, a real cell, so each reader tests `table` first. - Rename `ftoken` to `scanUnit`, beside `formatToken`, and `session` to `generatorState`: three seats confused the token types, and both architects read `session` as per-render state. -- 2.52.0 From aa2890b479d01e949fa4abc2f4e06b0d4f14ecee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:15:43 +0200 Subject: [PATCH 303/966] Spell traceRead's format case as the other format tests do --- render.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/render.go b/render.go index 2c5d8db..a03cd73 100644 --- a/render.go +++ b/render.go @@ -207,7 +207,7 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { switch { case t.site.isCell(): row = renderedRow{t.site.table, t.site.row} - case t.site.table != nil: + case t.site.row == formatRow: row = renderedRow{t.site.table, sc.rowOf(t.site.table)} } trace(strings.Clone(sc.group), row, a) -- 2.52.0 From 14194a775d7d8b1fe6efeba99c36bf7956620522 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:31:52 +0200 Subject: [PATCH 304/966] Rename ftoken to scanUnit and session to generatorState --- builtins.go | 18 +++++++++--------- builtins_test.go | 10 +++++----- calc.go | 2 +- fejkdata.go | 12 ++++++------ format.go | 16 ++++++++-------- layout.go | 4 ++-- path.go | 2 +- readsurvey_test.go | 4 ++-- record.go | 4 ++-- render.go | 24 ++++++++++++------------ struct.go | 6 +++--- table.go | 8 ++++---- todo.md | 3 --- transform.go | 2 +- 14 files changed, 56 insertions(+), 59 deletions(-) diff --git a/builtins.go b/builtins.go index f4be6ea..58b7e1d 100644 --- a/builtins.go +++ b/builtins.go @@ -40,19 +40,19 @@ var builtins = withTransforms(map[string]builtin{ "lower": {arity: 1, checkArgs: posIntArg, prep: chars("abcdefghijklmnopqrstuvwxyz")}, "base64": {arity: 1, checkArgs: posIntArg, prep: func(a []string) callFn { n := atoi(a[0]) - return func(s *session, _ string, _ []string) string { + return func(s *generatorState, _ string, _ []string) string { return base64.StdEncoding.EncodeToString(randBytes(s, n)) } }}, "int": {arity: 2, checkArgs: intRangeArgs, prep: func(a []string) callFn { lo, span := atoi(a[0]), atoi(a[1])-atoi(a[0])+1 - return func(s *session, _ string, _ []string) string { return strconv.Itoa(lo + s.IntN(span)) } + return func(s *generatorState, _ string, _ []string) string { return strconv.Itoa(lo + s.IntN(span)) } }, prints: DataTypeInteger, proveNumber: func(token string, prints DataType, a []string) proven { return printing(token, prints, bounded(float64(atoi(a[0])), float64(atoi(a[1])), true)) }}, "float": {arity: 3, checkArgs: floatArgs, prep: func(a []string) callFn { lo, hi, dp := atof(a[0]), atof(a[1]), atoi(a[2]) - return func(s *session, _ string, _ []string) string { + return func(s *generatorState, _ string, _ []string) string { return formatFloat(lo+s.Float64()*(hi-lo), dp) } }, prints: DataTypeNumber, proveNumber: func(token string, _ DataType, a []string) proven { @@ -60,12 +60,12 @@ var builtins = withTransforms(map[string]builtin{ }}, "iban": {arity: 1, checkArgs: ibanArg, prep: func(a []string) callFn { cc := a[0] - return func(s *session, _ string, _ []string) string { return iban(s, cc) } + return func(s *generatorState, _ string, _ []string) string { return iban(s, cc) } }}, "date": {arity: -1, checkArgs: dateArgs, prep: datePrep}, "time": {arity: -1, checkArgs: timeArg, prep: timePrep}, "calc": {arity: -1, checkArgs: checkCalc, prep: calcPrep, operands: calcOperands}, - // seq is the one stateful builtin: a per-session counter from 1, advancing on + // seq is the one stateful builtin: a per-generator counter from 1, advancing on // each call. An optional name selects an independent counter; no name uses the // default one. Deterministic by construction, so seeded output stays stable. "seq": {arity: -1, checkArgs: seqArg, prep: func(a []string) callFn { @@ -73,7 +73,7 @@ var builtins = withTransforms(map[string]builtin{ if len(a) == 1 { key = a[0] } - return func(s *session, _ string, _ []string) string { + return func(s *generatorState, _ string, _ []string) string { return strconv.FormatUint(s.next(key), 10) } }, prints: DataTypeInteger, proveNumber: func(token string, prints DataType, _ []string) proven { @@ -86,18 +86,18 @@ var builtins = withTransforms(map[string]builtin{ // sample of n characters drawn from an alphabet. func derive(f func(emitted string) string) func([]string) callFn { return func([]string) callFn { - return func(_ *session, emitted string, _ []string) string { return f(emitted) } + return func(_ *generatorState, emitted string, _ []string) string { return f(emitted) } } } func sample(f func(rng) string) func([]string) callFn { return func([]string) callFn { - return func(s *session, _ string, _ []string) string { return f(s) } + return func(s *generatorState, _ string, _ []string) string { return f(s) } } } func chars(alphabet string) func([]string) callFn { return func(a []string) callFn { n := atoi(a[0]) - return func(s *session, _ string, _ []string) string { return randChars(s, n, alphabet) } + return func(s *generatorState, _ string, _ []string) string { return randChars(s, n, alphabet) } } } diff --git a/builtins_test.go b/builtins_test.go index d093483..5c30158 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -102,10 +102,10 @@ func TestBuiltinChecksums(t *testing.T) { } } -// TestBuiltinSeqPerSession pins seq's contract: a counter from 1, advancing on +// TestBuiltinSeqPerGenerator pins seq's contract: a counter from 1, advancing on // each call, named counters independent, and the whole thing scoped to one -// session so a fresh Generator restarts at 1. -func TestBuiltinSeqPerSession(t *testing.T) { +// generatorState so a fresh Generator restarts at 1. +func TestBuiltinSeqPerGenerator(t *testing.T) { f := engine(1) for i := 1; i <= 5; i++ { if got := mustRender(t, f, `"{seq()}"`); got != strconv.Itoa(i) { @@ -122,9 +122,9 @@ func TestBuiltinSeqPerSession(t *testing.T) { if got := mustRender(t, engine(2), `{"format":"{seq()}","repeat":3,"separator":","}`); got != "1,2,3" { t.Fatalf("repeat seq = %q, want 1,2,3", got) } - // ...and a fresh session restarts from 1. + // ...and a fresh generator restarts from 1. if got := mustRender(t, engine(2), `"{seq()}"`); got != "1" { - t.Fatalf("new session seq = %q, want 1", got) + t.Fatalf("new generator seq = %q, want 1", got) } } diff --git a/calc.go b/calc.go index 8179cfb..04879ef 100644 --- a/calc.go +++ b/calc.go @@ -199,7 +199,7 @@ func calcPrep(args []string) callFn { } placed := indexVars(expr, at) dp := calcDecimals(args) - return func(_ *session, _ string, operands []string) string { + return func(_ *generatorState, _ string, operands []string) string { return formatFloat(placed.eval(operands), dp) } } diff --git a/fejkdata.go b/fejkdata.go index d13a7be..5c1c17d 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -34,20 +34,20 @@ type Generator struct { // table's row lookup, built under its sync.Once, so List, NewTemplate and // NewRecordTemplate read it without mu. mu sync.Mutex - rand *session + rand *generatorState root folder // the categories as the node a path walks from, owned here so a walk allocates none records map[node]recordShape structs map[reflect.Type]structResult } -// session is one generator's mutable render state: the seeded rng plus the {seq()} +// generatorState is one generator's state across its renders: the seeded rng plus the {seq()} // counters. -type session struct { +type generatorState struct { *rand.Rand counters map[string]uint64 } -func (s *session) next(key string) uint64 { +func (s *generatorState) next(key string) uint64 { s.counters[key]++ return s.counters[key] } @@ -228,7 +228,7 @@ func join(prefix, name string) string { // randomBytes seeds an unseeded generator. var randomBytes = crand.Read -func newRand(seed uint64, seeded bool) (*session, error) { +func newRand(seed uint64, seeded bool) (*generatorState, error) { var r *rand.Rand if seeded { r = rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15)) @@ -239,7 +239,7 @@ func newRand(seed uint64, seeded bool) (*session, error) { } r = rand.New(rand.NewPCG(binary.LittleEndian.Uint64(b[:8]), binary.LittleEndian.Uint64(b[8:]))) } - return &session{Rand: r, counters: map[string]uint64{}}, nil + return &generatorState{Rand: r, counters: map[string]uint64{}}, nil } func internalError(format string, a ...any) string { diff --git a/format.go b/format.go index ae043e3..9498152 100644 --- a/format.go +++ b/format.go @@ -5,9 +5,9 @@ import ( "strings" ) -// ftoken is one unit of a scanned format string: a literal rune or the body of a +// scanUnit is one unit of a scanned format string: a literal rune or the body of a // {…} token. -type ftoken struct { +type scanUnit struct { isToken bool // a {…} body, else a literal rune char rune body string @@ -16,10 +16,10 @@ type ftoken struct { // eachToken scans a format string once and calls fn for each unit, the single // source of truth for how braces are read: "{{" and "}}" are literal braces, a "{" // opens a token that must reach its "}", and a lone "}" is an error. -func eachToken(format string, fn func(ftoken) error) error { +func eachToken(format string, fn func(scanUnit) error) error { rs := []rune(format) for i := 0; i < len(rs); i++ { - var t ftoken + var t scanUnit switch c := rs[i]; c { case '{': if i+1 < len(rs) && rs[i+1] == '{' { @@ -85,7 +85,7 @@ func parseFormat(format string) ([]formatToken, error) { lit.Reset() } } - err := eachToken(format, func(t ftoken) error { + err := eachToken(format, func(t scanUnit) error { if !t.isToken { lit.WriteRune(t.char) return nil @@ -110,11 +110,11 @@ func parseFormat(format string) ([]formatToken, error) { } // builtin is a format-string function invoked as {name(args)}. It receives the -// session (its rng, and the {seq()} counters), the output emitted so far in the +// generatorState (its rng, and the {seq()} counters), the output emitted so far in the // current expansion (for derivations such as a checksum over preceding digits), and // the values of the operands it named (calc and the transforms name them). All must // stay pure over (rng, emitted, args) so seeded output is reproducible; seq advances -// per-session counter state, which is itself deterministic. arity is the exact arg +// per-generator counter state, which is itself deterministic. arity is the exact arg // count, or -1 for variadic (then checkArgs does all the validation). type builtin struct { arity int @@ -366,7 +366,7 @@ func checkSegments(a arm) error { // callFn is a builtin prepared for one call site: its args already parsed. It reads the // output emitted so far in the current expansion (a derivation's payload) and the // values of the operands it named, which expand read for it. -type callFn func(s *session, emitted string, operands []string) string +type callFn func(s *generatorState, emitted string, operands []string) string // op is one compiled unit of a format string: a literal run, a name read, // or a builtin already prepared with its args. compile builds these so render never diff --git a/layout.go b/layout.go index af98d7e..cb1a61c 100644 --- a/layout.go +++ b/layout.go @@ -135,13 +135,13 @@ func datePrep(a []string) callFn { panic(internalError("builtin arg %q reached prep unvalidated: %v", a[1], err)) } layout, start, span := layoutOf(a[2]), from.Unix(), int(to.Unix()-from.Unix())+86400 - return func(s *session, _ string, _ []string) string { + return func(s *generatorState, _ string, _ []string) string { return time.Unix(start+int64(s.IntN(span)), 0).UTC().Format(layout) } } func timePrep(a []string) callFn { layout := layoutOf(a[0]) - return func(s *session, _ string, _ []string) string { + return func(s *generatorState, _ string, _ []string) string { return time.Unix(int64(s.IntN(86400)), 0).UTC().Format(layout) } } diff --git a/path.go b/path.go index b8b253f..02037ab 100644 --- a/path.go +++ b/path.go @@ -353,7 +353,7 @@ func (w *pathProbe) readRow(t *table, tail []string, r tableRoute) error { // Where pathCheck walks every variant of a choice, it walks the one drawn; at a table it selects // as pathCheck does and draws the row the route asks for. type pathDraw struct { - s *session + s *generatorState pins *pinSet held *hold a *arm diff --git a/readsurvey_test.go b/readsurvey_test.go index 992fd11..0c5f780 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -76,7 +76,7 @@ func spellPins(p *pinSet) string { return "row " + strings.Join(rows, ", ") } -func renderRoot(s *session, t *template, trace renderTrace) { +func renderRoot(s *generatorState, t *template, trace renderTrace) { set := holdSet{trace: trace} sc := renderScope{set: &set} switch { @@ -96,7 +96,7 @@ func renderRoot(s *session, t *template, trace renderTrace) { } } -func renderRecordRoot(s *session, t *template, columns []Column, trace renderTrace) { +func renderRecordRoot(s *generatorState, t *template, columns []Column, trace renderTrace) { set := holdSet{trace: trace} sc := renderScope{set: &set} if t.site.row == formatRow { diff --git a/record.go b/record.go index 0298571..e8dee4f 100644 --- a/record.go +++ b/record.go @@ -152,7 +152,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { // tableRecord walks a path's tail from a table to the table whose row is the record, // pinning the rows it selects or draws. -func tableRecord(s *session, t *table, tail []string, sc renderScope) (node, error) { +func tableRecord(s *generatorState, t *table, tail []string, sc renderScope) (node, error) { n, err := descend(s, t, tail, sc) if err != nil { return nil, err @@ -261,7 +261,7 @@ func recordOf(n node) (*template, []Column, error) { // renderRecord draws each column once, in the name order recordOf fixed, as one render // over sc's hold; a table's columns read the row pinned there. -func renderRecord(s *session, t *template, columns []Column, sc renderScope) *Record { +func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.site.row == formatRow { sc = sc.at(t.site.table.rowNode, &sc.groupHold().pins) diff --git a/render.go b/render.go index a03cd73..e05ca5c 100644 --- a/render.go +++ b/render.go @@ -6,7 +6,7 @@ import ( ) // rng is the randomness a builtin sample draws from; *rand.Rand satisfies it. The -// render path takes the concrete *session instead, which keeps the hold set of the walk +// render path takes the concrete *generatorState instead, which keeps the hold set of the walk // drawing through it off the heap. type rng interface { IntN(n int) int @@ -38,7 +38,7 @@ func (f *Generator) Fake(path string) (string, error) { // descend walks a caller's path to the node it names, pinning in sc the rows it // selects or draws. -func descend(s *session, root node, segments []string, sc renderScope) (node, error) { +func descend(s *generatorState, root node, segments []string, sc renderScope) (node, error) { // docs/decisions.md#a-path-is-walked-once-without-drawing-before-it-is-walked-for-real if _, err := (&pathProbe{}).walk(root, segments); err != nil { return nil, err @@ -47,7 +47,7 @@ func descend(s *session, root node, segments []string, sc renderScope) (node, er } // renderOnce renders n as one render, over a hold set of its own. -func renderOnce(s *session, n node) string { +func renderOnce(s *generatorState, n node) string { var set holdSet return render(s, n, renderScope{set: &set}) } @@ -56,7 +56,7 @@ func renderOnce(s *session, n node) string { // front, so rendering a compiled tree cannot fail. sc holds the reference draws the // render shares; each repeat iteration renders over a hold set of its own. // A child this switch renders is one renderEdges must list too, or the fences miss it. -func render(s *session, n node, sc renderScope) string { +func render(s *generatorState, n node, sc renderScope) string { switch n := n.(type) { case *choice: return render(s, pick(s, n), sc) @@ -99,7 +99,7 @@ func render(s *session, n node, sc renderScope) string { // render's loop, its hold set would move to the heap. // //go:noinline -func expandAnew(s *session, t *template) string { +func expandAnew(s *generatorState, t *template) string { var set holdSet return expand(s, t, renderScope{set: &set}) } @@ -107,7 +107,7 @@ func expandAnew(s *session, t *template) string { // pick selects one item. Uniform choices are O(1); weighted choices are an // O(log n) search over precomputed cumulative weights. compile guarantees a // non-empty choice and a finite positive total, so the index is always in range. -func pick(s *session, c *choice) node { +func pick(s *generatorState, c *choice) node { if c.cum == nil { return c.items[s.IntN(len(c.items))] } @@ -116,7 +116,7 @@ func pick(s *session, c *choice) node { // expand renders a template's compiled ops. compile validated every token, so this // cannot fail. -func expand(s *session, t *template, sc renderScope) string { +func expand(s *generatorState, t *template, sc renderScope) string { var b strings.Builder b.Grow(t.compiled.grow) // One draw per held name, for this expansion only: a nested template and each @@ -157,7 +157,7 @@ func expand(s *session, t *template, sc renderScope) string { // reads — is drawn once and kept in held, so {place.postal-code} and {place.locality} // read one row, either read twice gives one value, and a shown operand is the operand // computed. Every other name is drawn afresh, so {word} {word} still draws twice. -func readField(s *session, t *template, held *hold, sc renderScope, a arm) readValue { +func readField(s *generatorState, t *template, held *hold, sc renderScope, a arm) readValue { if isRef(a.head) && len(a.tail) > 0 { return readReference(s, t, sc, a) } @@ -178,7 +178,7 @@ func readField(s *session, t *template, held *hold, sc renderScope, a arm) readV // readReference reads a reference path, kept in the render's hold for its group, so its draw // spans the render. -func readReference(s *session, t *template, sc renderScope, a arm) readValue { +func readReference(s *generatorState, t *template, sc renderScope, a arm) readValue { if sc.set.trace != nil { traceRead(sc.set.trace, t, sc, a) } @@ -186,7 +186,7 @@ func readReference(s *session, t *template, sc renderScope, a arm) readValue { return readHeld(s, t, &group.hold, &group.pins, sc, a) } -func readHeld(s *session, t *template, held *hold, pins *pinSet, sc renderScope, a arm) readValue { +func readHeld(s *generatorState, t *template, held *hold, pins *pinSet, sc renderScope, a arm) readValue { if r, done := held.value[a.path]; done { return r } @@ -215,7 +215,7 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. -func renderLeaf(s *session, n node, sc renderScope) readValue { +func renderLeaf(s *generatorState, n node, sc renderScope) readValue { n = resolveChoice(s, n) switch leaf := n.(type) { case *nullItem: @@ -231,7 +231,7 @@ func renderLeaf(s *session, n node, sc renderScope) readValue { // resolveChoice resolves a choice to one variant, so a held head is a concrete node the // rest of the expansion shares. Nested choices unwrap too: a draw is one value, not // another set to pick from. -func resolveChoice(s *session, n node) node { +func resolveChoice(s *generatorState, n node) node { for c, ok := n.(*choice); ok; c, ok = n.(*choice) { n = pick(s, c) } diff --git a/struct.go b/struct.go index 8f6b82a..f51f218 100644 --- a/struct.go +++ b/struct.go @@ -349,10 +349,10 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro } // fill draws the record into v's tagged fields, then each nested struct as a record of its own. -func (s *structShape) fill(sess *session, v reflect.Value) { +func (s *structShape) fill(state *generatorState, v reflect.Value) { if s.record != nil { var set holdSet - for i, c := range renderRecord(sess, s.record, s.columns, renderScope{set: &set}).columns { + for i, c := range renderRecord(state, s.record, s.columns, renderScope{set: &set}).columns { setColumn(fieldAt(v, s.fieldIndexes[i]), c) } } @@ -364,7 +364,7 @@ func (s *structShape) fill(sess *session, v reflect.Value) { } field = field.Elem() } - n.shape.fill(sess, field) + n.shape.fill(state, field) } } diff --git a/table.go b/table.go index 1e81db0..71c4fc8 100644 --- a/table.go +++ b/table.go @@ -472,20 +472,20 @@ func (t *table) builtLookup() *rowLookup { } // drawRow picks a row over the whole table. -func (t *table) drawRow(s *session) int { +func (t *table) drawRow(s *generatorState) int { if t.cum == nil { return s.IntN(t.rowCount()) } return pickCum(s, t.cum) } -func pickCum(s *session, cum []float64) int { +func pickCum(s *generatorState, cum []float64) int { x := s.Float64() * cum[len(cum)-1] return min(sort.Search(len(cum), func(i int) bool { return cum[i] > x }), len(cum)-1) // x can round up to the total } // drawUnder picks a row among those linked to parent row pr. -func (t *table) drawUnder(s *session, pr int) int { +func (t *table) drawUnder(s *generatorState, pr int) int { lookup := t.builtLookup() k := t.parentT.cell(pr, t.parentT.keyIndex) rows := lookup.rowsByParent[k] @@ -498,7 +498,7 @@ func (t *table) drawUnder(s *session, pr int) int { // drawIn is the render's row of t: the one pinned in p, else one drawn inside the nearest // pinned ancestor — its parent drawn inside that first where the ancestor is further up — or // over the whole table, and pinned with its ancestors. -func (t *table) drawIn(s *session, p *pinSet) int { +func (t *table) drawIn(s *generatorState, p *pinSet) int { if r, ok := p.pinned(t); ok { return r } diff --git a/todo.md b/todo.md index 5215fb9..970d455 100644 --- a/todo.md +++ b/todo.md @@ -31,9 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Rename `ftoken` to `scanUnit`, beside `formatToken`, and `session` to - `generatorState`: three seats confused the token types, and both architects read - `session` as per-render state. - State at `pinSet.pin` that its caller proved r sits inside the nearest pinned ancestor, which `pinRow` and `drawIn` do. - Delete the prose two or more seats found empty: `loadData`'s comment, which also names diff --git a/transform.go b/transform.go index e5f965e..64d4d11 100644 --- a/transform.go +++ b/transform.go @@ -70,7 +70,7 @@ func transformPrep(outer func(string) string) func([]string) callFn { if err != nil { panic(internalError("transform arg %q reached prep unvalidated: %v", a[0], err)) } - return func(_ *session, _ string, operands []string) string { + return func(_ *generatorState, _ string, operands []string) string { v := operands[0] for i := len(chain) - 1; i >= 0; i-- { v = chain[i](v) -- 2.52.0 From 186443f883d514fded6a148766619e4235b5d776 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:34:36 +0200 Subject: [PATCH 305/966] Rename eachToken to eachScanUnit, and narrow generatorState to draw state --- fejkdata.go | 4 ++-- format.go | 44 ++++++++++++++++++++++---------------------- 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index 5c1c17d..fce5cc6 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -40,8 +40,8 @@ type Generator struct { structs map[reflect.Type]structResult } -// generatorState is one generator's state across its renders: the seeded rng plus the {seq()} -// counters. +// generatorState is one generator's draw state, kept across its renders: the seeded rng plus +// the {seq()} counters. type generatorState struct { *rand.Rand counters map[string]uint64 diff --git a/format.go b/format.go index 9498152..599ec75 100644 --- a/format.go +++ b/format.go @@ -13,17 +13,17 @@ type scanUnit struct { body string } -// eachToken scans a format string once and calls fn for each unit, the single +// eachScanUnit scans a format string once and calls fn for each unit, the single // source of truth for how braces are read: "{{" and "}}" are literal braces, a "{" // opens a token that must reach its "}", and a lone "}" is an error. -func eachToken(format string, fn func(scanUnit) error) error { +func eachScanUnit(format string, fn func(scanUnit) error) error { rs := []rune(format) for i := 0; i < len(rs); i++ { - var t scanUnit + var u scanUnit switch c := rs[i]; c { case '{': if i+1 < len(rs) && rs[i+1] == '{' { - t.char = '{' + u.char = '{' i++ break } @@ -37,19 +37,19 @@ func eachToken(format string, fn func(scanUnit) error) error { if end >= len(rs) { return fmt.Errorf("unterminated '{' in %q", format) } - t.isToken, t.body = true, string(rs[i+1:end]) + u.isToken, u.body = true, string(rs[i+1:end]) i = end case '}': if i+1 < len(rs) && rs[i+1] == '}' { - t.char = '}' + u.char = '}' i++ break } return fmt.Errorf("lone '}' in %q; a literal brace is written }}", format) default: - t.char = c + u.char = c } - if err := fn(t); err != nil { + if err := fn(u); err != nil { return err } } @@ -85,21 +85,21 @@ func parseFormat(format string) ([]formatToken, error) { lit.Reset() } } - err := eachToken(format, func(t scanUnit) error { - if !t.isToken { - lit.WriteRune(t.char) + err := eachScanUnit(format, func(u scanUnit) error { + if !u.isToken { + lit.WriteRune(u.char) return nil } flush() - if indexOutside(t.body, '(') < 0 { - toks = append(toks, formatToken{kind: nameRead, body: t.body, names: splitOutside(t.body, '|')}) + if indexOutside(u.body, '(') < 0 { + toks = append(toks, formatToken{kind: nameRead, body: u.body, names: splitOutside(u.body, '|')}) return nil } - name, args, ok := funcCall(t.body) + name, args, ok := funcCall(u.body) if !ok { - return fmt.Errorf("malformed function token {%s}", t.body) + return fmt.Errorf("malformed function token {%s}", u.body) } - toks = append(toks, formatToken{kind: builtinCall, body: t.body, fn: name, args: args, names: builtinOperands(name, args)}) + toks = append(toks, formatToken{kind: builtinCall, body: u.body, fn: name, args: args, names: builtinOperands(name, args)}) return nil }) if err != nil { @@ -110,12 +110,12 @@ func parseFormat(format string) ([]formatToken, error) { } // builtin is a format-string function invoked as {name(args)}. It receives the -// generatorState (its rng, and the {seq()} counters), the output emitted so far in the -// current expansion (for derivations such as a checksum over preceding digits), and -// the values of the operands it named (calc and the transforms name them). All must -// stay pure over (rng, emitted, args) so seeded output is reproducible; seq advances -// per-generator counter state, which is itself deterministic. arity is the exact arg -// count, or -1 for variadic (then checkArgs does all the validation). +// generatorState, the output emitted so far in the current expansion (for derivations +// such as a checksum over preceding digits), and the values of the operands it named +// (calc and the transforms name them). All must stay pure over (rng, emitted, args) so +// seeded output is reproducible; seq advances per-generator counter state, which is +// itself deterministic. arity is the exact arg count, or -1 for variadic (then +// checkArgs does all the validation). type builtin struct { arity int // prep parses validated args once, at compile time, into the closure expand calls. -- 2.52.0 From 8079157329e51f28b90768fec11c4aa99892b3f0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:35:03 +0200 Subject: [PATCH 306/966] File the newRand rename in todo.md --- todo.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/todo.md b/todo.md index 970d455..3895d93 100644 --- a/todo.md +++ b/todo.md @@ -167,6 +167,9 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`: both name + half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` + shadows the `rng` interface. - Compare `calcParser.binary`'s operator as a rune: `byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. - Stop the `columnKinds` suggestion naming a field's own kind or a narrower one: an -- 2.52.0 From 322c8dead7258e331e3fcb02e381d25dc971447a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 12:51:00 +0200 Subject: [PATCH 307/966] State pinSet.pin's precondition, and file readSurvey.edge's unchecked pins --- pins.go | 4 +++- todo.md | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/pins.go b/pins.go index 0418ecb..c866373 100644 --- a/pins.go +++ b/pins.go @@ -50,7 +50,9 @@ func (p *pinSet) add(t *table, r int) { p.spill[t] = r } -// pin pins row r of t, and the rows of t's ancestors it links to. +// pin pins row r of t, and the rows of t's ancestors it links to. It checks nothing, so r must +// agree with the row p pins of nearestPinned(t): ask clash first, as pinRow does, or draw r inside +// it, as drawIn does. func (p *pinSet) pin(t *table, r int) { for stop := p.nearestPinned(t); t != stop; t = t.parentT { p.add(t, r) diff --git a/todo.md b/todo.md index 3895d93..f7ebddb 100644 --- a/todo.md +++ b/todo.md @@ -31,8 +31,8 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- State at `pinSet.pin` that its caller proved r sits inside the nearest pinned - ancestor, which `pinRow` and `drawIn` do. +- Ask `clash` before `readSurvey.edge` enters a read's pins, or refuse the clash there: + `pin` checks nothing, and `replayPairs` compares reads within one draw group only. - Delete the prose two or more seats found empty: `loadData`'s comment, which also names `linkRefs` where `bind` runs, `table.drawRow`'s, `table.selector`'s, and `render.go`'s second and third "compile validated" notes. -- 2.52.0 From 25857fb660d139169262d9d1f3d7340ffa1b3de4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:06:18 +0200 Subject: [PATCH 308/966] Test a selected row reaching its family's other row through a draw group --- table_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/table_test.go b/table_test.go index 8c4ab5e..d247f38 100644 --- a/table_test.go +++ b/table_test.go @@ -479,6 +479,7 @@ func TestTableFences(t *testing.T) { "a table reaching its family through a template": {with(geo(), map[string]string{"addr.json": `"{/locality.name}"`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, "a table reaching its family through a repeat": {with(geo(), map[string]string{"addr.json": `{"format":"{/locality.name} ","repeat":3}`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, "a table reaching its family through a group": {with(geo(), map[string]string{"addr.json": `{"format":"{/locality.name}","drawGroup":"g"}`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, + "a selected row reaching its family's other row": {with(geo(), map[string]string{"a.json": `"{/region[01].note}"`, "addr.json": `{"format":"{/region[12].name}","drawGroup":"g"}`, "region.tsv": "code\tname\tpopulation\ttimezone\tnote\n01\tStockholms län\t2400000\tEurope/Stockholm\t{/addr}\n12\tSkåne län\t1400000\tEurope/Stockholm\t-\n14\tVästra Götalands län\t1750000\tEurope/Stockholm\t-\n"}), "family"}, "one row": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": "a\nx\n"}, "one row"}, "empty file": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv"}`, "t.tsv": ""}, "header"}, "parent is not a table": {map[string]string{"p.json": `"x"`, "t.json": `{"format":"{a}","rows":"t.tsv","parent":"p"}`, "t.tsv": "a\tp\nx\tx\ny\tx\n"}, "not a table"}, -- 2.52.0 From 6cb2d143d67f4d2750eef62f96ffb6377fbc3761 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:09:19 +0200 Subject: [PATCH 309/966] Refuse own-family reads before any survey, and panic on a clash where readSurvey.edge enters a read's pins --- data.go | 3 +++ drawfence.go | 10 +++++++++- readsurvey.go | 7 ++++++- 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/data.go b/data.go index c3fd37a..6c1b81b 100644 --- a/data.go +++ b/data.go @@ -257,6 +257,9 @@ func checkNodeFences(s nodeScope) error { return err } if refs { + if err := s(checkOwnFamilies); err != nil { + return err + } if err := s(fence.checkDraws); err != nil { return err } diff --git a/drawfence.go b/drawfence.go index 8f46f5d..8caffb7 100644 --- a/drawfence.go +++ b/drawfence.go @@ -24,7 +24,7 @@ func (f *drawFence) checkDrawGroup(path string, n node) error { return nil } -func (f *drawFence) checkDraws(path string, n node) error { +func checkOwnFamilies(path string, n node) error { t, ok := n.(*template) if !ok { return nil @@ -32,6 +32,14 @@ func (f *drawFence) checkDraws(path string, n node) error { if err := checkOwnFamily(t); err != nil { return fmt.Errorf("%s: %w", path, err) } + return nil +} + +func (f *drawFence) checkDraws(path string, n node) error { + t, ok := n.(*template) + if !ok { + return nil + } if !f.readsPath(t) { return nil } diff --git a/readsurvey.go b/readsurvey.go index 531e8c7..e77903c 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -122,7 +122,12 @@ func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { s.reads = append(s.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { at.pins = at.pins.entered(t, r) }) + tr.pins.each(func(t *table, r int) { + if c := at.pins.clash(t, r); c != nil { + panic(internalError("%s reads %s past a pinned row of %s, which checkOwnFamily refuses", e.reached(), t.selectorSpelling(r), c.path)) + } + at.pins = at.pins.entered(t, r) + }) } } s.walk(e.to, at) -- 2.52.0 From 0fde439d8baa4a6dba89792c1845cda6f61701f0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:09:19 +0200 Subject: [PATCH 310/966] Drop the done readSurvey.edge item from todo.md --- todo.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/todo.md b/todo.md index f7ebddb..fb77657 100644 --- a/todo.md +++ b/todo.md @@ -31,8 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Ask `clash` before `readSurvey.edge` enters a read's pins, or refuse the clash there: - `pin` checks nothing, and `replayPairs` compares reads within one draw group only. - Delete the prose two or more seats found empty: `loadData`'s comment, which also names `linkRefs` where `bind` runs, `table.drawRow`'s, `table.selector`'s, and `render.go`'s second and third "compile validated" notes. -- 2.52.0 From 94c8bd229db654a1d54d2265bc3c4c6e24f5c1d0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:14:15 +0200 Subject: [PATCH 311/966] Give checkOwnFamily the scope walk's signature, and drop its wrapper --- data.go | 2 +- drawfence.go | 11 ----------- readsurvey.go | 9 +++++---- 3 files changed, 6 insertions(+), 16 deletions(-) diff --git a/data.go b/data.go index 6c1b81b..f12e9c5 100644 --- a/data.go +++ b/data.go @@ -257,7 +257,7 @@ func checkNodeFences(s nodeScope) error { return err } if refs { - if err := s(checkOwnFamilies); err != nil { + if err := s(checkOwnFamily); err != nil { return err } if err := s(fence.checkDraws); err != nil { diff --git a/drawfence.go b/drawfence.go index 8caffb7..4d34014 100644 --- a/drawfence.go +++ b/drawfence.go @@ -24,17 +24,6 @@ func (f *drawFence) checkDrawGroup(path string, n node) error { return nil } -func checkOwnFamilies(path string, n node) error { - t, ok := n.(*template) - if !ok { - return nil - } - if err := checkOwnFamily(t); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - return nil -} - func (f *drawFence) checkDraws(path string, n node) error { t, ok := n.(*template) if !ok { diff --git a/readsurvey.go b/readsurvey.go index e77903c..634ca83 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -258,11 +258,12 @@ func checkFamilies(reads []pathRead) error { // read draws its row without pinning it, so the family would draw apart from the row // it renders, whichever draws the reaching template holds. // docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route -func checkOwnFamily(t *template) error { - own := t.site.table - if own == nil { +func checkOwnFamily(path string, n node) error { + t, isTemplate := n.(*template) + if !isTemplate || t.site.table == nil { return nil } + own := t.site.table seen := map[node]bool{} var find func(n node) (renderEdge, *table, bool) find = func(n node) (renderEdge, *table, bool) { @@ -283,7 +284,7 @@ func checkOwnFamily(t *template) error { return renderEdge{}, nil, false } if e, familyTable, found := find(t); found { - return fmt.Errorf("%s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", e.reached(), familyTable.segment, own.segment) + return fmt.Errorf("%s: %s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", path, e.reached(), familyTable.segment, own.segment) } return nil } -- 2.52.0 From e1769f20a24b747c8262e23235f67dae8834ede7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:28:27 +0200 Subject: [PATCH 312/966] Delete loadData's, drawRow's and selector's comments, and render.go's second and third compile notes --- data.go | 7 ------- path.go | 1 - render.go | 6 ++---- table.go | 1 - todo.md | 3 --- 5 files changed, 2 insertions(+), 16 deletions(-) diff --git a/data.go b/data.go index f12e9c5..28bd9be 100644 --- a/data.go +++ b/data.go @@ -27,13 +27,6 @@ func (s dataSource) labelled(p string) string { return path.Join(s.label, p) } -// loadData loads every source into one namespace tree and returns its root -// children. A directory becomes a folder; each *.json file in it compiles to a node -// keyed by its base name (address.json -> "address"); each subdirectory becomes a -// nested folder, so folders turn into dot-path segments. Sources merge left to right: -// matching folders merge by their children, and any other clash is won by the last -// source loaded. Once merged, linkRefs binds every reference against the -// final tree. func loadData(sources []dataSource) (map[string]node, error) { root := map[string]node{} for _, src := range sources { diff --git a/path.go b/path.go index 02037ab..59b7c16 100644 --- a/path.go +++ b/path.go @@ -418,7 +418,6 @@ func (w *pathDraw) variant(c *choice, rest []string) node { return n } -// selector splits the selector a tail starts with from the rest of it. func (t *table) selector(tail []string) (sel string, rest []string, err error) { if len(tail) == 0 || !isSelector(tail[0]) { return "", tail, nil diff --git a/render.go b/render.go index e05ca5c..86a3aa2 100644 --- a/render.go +++ b/render.go @@ -105,8 +105,7 @@ func expandAnew(s *generatorState, t *template) string { } // pick selects one item. Uniform choices are O(1); weighted choices are an -// O(log n) search over precomputed cumulative weights. compile guarantees a -// non-empty choice and a finite positive total, so the index is always in range. +// O(log n) search over precomputed cumulative weights. func pick(s *generatorState, c *choice) node { if c.cum == nil { return c.items[s.IntN(len(c.items))] @@ -114,8 +113,7 @@ func pick(s *generatorState, c *choice) node { return c.items[pickCum(s, c.cum)] } -// expand renders a template's compiled ops. compile validated every token, so this -// cannot fail. +// expand renders a template's compiled ops. func expand(s *generatorState, t *template, sc renderScope) string { var b strings.Builder b.Grow(t.compiled.grow) diff --git a/table.go b/table.go index 71c4fc8..6a7e9cd 100644 --- a/table.go +++ b/table.go @@ -471,7 +471,6 @@ func (t *table) builtLookup() *rowLookup { return &t.lookup } -// drawRow picks a row over the whole table. func (t *table) drawRow(s *generatorState) int { if t.cum == nil { return s.IntN(t.rowCount()) diff --git a/todo.md b/todo.md index fb77657..107ef1b 100644 --- a/todo.md +++ b/todo.md @@ -31,9 +31,6 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic This round, in order: -- Delete the prose two or more seats found empty: `loadData`'s comment, which also names - `linkRefs` where `bind` runs, `table.drawRow`'s, `table.selector`'s, and `render.go`'s - second and third "compile validated" notes. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 57d9698b0be26bc649b7e1d9db0c3fdee5a4a79d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:39:33 +0200 Subject: [PATCH 313/966] File the fourth 2026-09-29 nine-seat panel's findings, scored 5.8 --- todo.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index 107ef1b..72d0bc4 100644 --- a/todo.md +++ b/todo.md @@ -27,10 +27,34 @@ Self-sufficiency 5.8), four seats capped by Locality, six of the seven unit seat ranking `readSurvey.walk` hardest and all seven the unit they would least modify; 6.1 on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-sufficiency 6.1), every seat's lowest dimension Locality, all seven unit seats ranking the -`readSurvey` walk or check hardest and the unit they would least modify. +`readSurvey` walk or check hardest and the unit they would least modify; 5.8 on +2026-09-29 a fourth time (Navigation 6.7, Locality 5.0, Shape 5.7, Self-sufficiency +5.7), every seat's lowest dimension Locality, alone or tied, all seven unit seats +ranking `readSurvey.walk` and `edge` hardest and the unit they would least modify. This round, in order: +- Make `readSurvey.edge` sound whatever order `checkNodeFences` runs its fences in: + report the pin clash as a load error, or have `checkDraws` run `checkOwnFamily` over + the scope before its survey, and delete the panic naming `checkOwnFamily`. All nine + seats found the survey sound only by a call order stated in that panic's text. +- Test that `render` and `renderEdges` agree on every node kind, and each of the + switches `node`'s comment lists, so a kind one of them misses fails a test: five + seats, both architects among them, found the mirror held by comments alone. +- Choose `disagreement`'s error directly, deleting `disagreementFix`'s switch returning + a builder and `declaresLess` for `<`: seven seats traced the matrix for a message, + and `disagreementFix`'s comment describes its parameter. +- Give the expansion hold and the render's group hold two names: `hold`, `held`, + `heldLocal`, `holdSet` and `groupHold` spell both, and four seats could not tell + which fence guards which. +- Word the three overlap refusals apart, `checkNameHeld`'s, `checkNoOverlap`'s and + `overlapError`'s: three seats grepped one message to three fences. +- Compile a format in `format.go`: `linkTemplate` calls `compileFormat` in `node.go`, + which calls `compileOps` and `holdfence.go`'s two checks, and both architects placed + the step wrong. +- Delete the "so which … is reported does not vary" clause at its five sites past + `checkNodeFences`, and `checkRecordDraws`' "A record-only template's format renders + nothing", which runs over templates with a format too. - Run the nine-seat panel again and file what it names here. ### Data @@ -58,6 +82,8 @@ by a `data-import/` script, as README goal 12 asks. - Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too. - Fill the 398 Swedish localities weighted 200 from SCB småorter. +- Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they + are: neither names a `weight`, so a bare `sex` draws evenly. - List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries, which names neither. - Give `url` and `email` a path that draws only domains nobody can register, keeping @@ -213,6 +239,8 @@ by a `data-import/` script, as README goal 12 asks. `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. - Call `templateError` a compile failure: its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. +- Name the node a selector follows in `stepInto`'s refusal: `sv_SE.person[1].first` + answers `1 is not a table`. - Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the first in Go's map order. -- 2.52.0 From acbb8f0b244f5e2f1ed27921f9683e2cab853f5c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:39:57 +0200 Subject: [PATCH 314/966] Say the group hold embeds the expansion hold in the round's naming item --- todo.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 72d0bc4..1811fdc 100644 --- a/todo.md +++ b/todo.md @@ -44,9 +44,9 @@ This round, in order: - Choose `disagreement`'s error directly, deleting `disagreementFix`'s switch returning a builder and `declaresLess` for `<`: seven seats traced the matrix for a message, and `disagreementFix`'s comment describes its parameter. -- Give the expansion hold and the render's group hold two names: `hold`, `held`, - `heldLocal`, `holdSet` and `groupHold` spell both, and four seats could not tell - which fence guards which. +- Give the expansion hold and the render's group hold two names: `groupHold` embeds + `hold`, `held`, `heldLocal` and `holdSet` share its stem, and four seats could not + tell which fence guards which. - Word the three overlap refusals apart, `checkNameHeld`'s, `checkNoOverlap`'s and `overlapError`'s: three seats grepped one message to three fences. - Compile a format in `format.go`: `linkTemplate` calls `compileFormat` in `node.go`, -- 2.52.0 From 4c42c60651c199b3e590b2cbe7298e9d896316ce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:42:19 +0200 Subject: [PATCH 315/966] Test that readSurvey reports a pin clash as an error --- readsurvey_test.go | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/readsurvey_test.go b/readsurvey_test.go index 0c5f780..d36cf28 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -136,3 +136,27 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root and from inside {/x}", pinned, whole) } } + +func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { + fsys := fstest.MapFS{ + "r.json": {Data: []byte(`"{/y[a].v}"`)}, + "x.json": {Data: []byte(`{"format":"{v}","rows":"x.tsv","key":"code"}`)}, + "x.tsv": {Data: []byte("code\tv\n1\tb\n2\tc\n")}, + "y.json": {Data: []byte(`{"format":"{v}","rows":"y.tsv","key":"code","parent":"x"}`)}, + "y.tsv": {Data: []byte("code\tv\tx\na\t{/x[2].v}\t1\nb\tz\t2\n")}, + } + g, err := loadDir(dataSource{fsys: fsys}, ".") + if err != nil { + t.Fatalf("loadDir = %v", err) + } + if err := treeBinding(g.children).link(); err != nil { + t.Fatalf("link = %v", err) + } + r, isTemplate := g.children["r"].(*template) + if !isTemplate { + t.Fatalf("r is a %T, want a template", g.children["r"]) + } + if err := surveyRender(r).check(); err == nil || !strings.Contains(err.Error(), "two rows of") { + t.Errorf("check = %v, want y[a]'s row of x clashing with x[2] named, whatever order the fences run in", err) + } +} -- 2.52.0 From b3cc37be261493a6bd3a8ccc595dea3204c0bab7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:44:08 +0200 Subject: [PATCH 316/966] Report a pin clash in readSurvey.edge as a load error, not a panic --- readsurvey.go | 23 +++++++++++++++-------- todo.md | 4 ---- 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/readsurvey.go b/readsurvey.go index 634ca83..a40e81b 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -11,6 +11,7 @@ type readSurvey struct { reads []pathRead read map[readKey]bool seen map[nodeVisit]bool + clash error } // surveyAt is where a survey stands: the draw group its reads draw in, how the render's root reached it, the @@ -122,12 +123,14 @@ func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { s.reads = append(s.reads, pathRead{at, a, tr}) } if tr != nil { - tr.pins.each(func(t *table, r int) { - if c := at.pins.clash(t, r); c != nil { - panic(internalError("%s reads %s past a pinned row of %s, which checkOwnFamily refuses", e.reached(), t.selectorSpelling(r), c.path)) + pins := at.pins.clone() + if err := tr.replay(&pins); err != nil { + if s.clash == nil { + s.clash = conflict(pathRead{at, a, tr}, err) } - at.pins = at.pins.entered(t, r) - }) + return + } + at.pins = pins } } s.walk(e.to, at) @@ -209,10 +212,14 @@ func (r *tableRead) replay(d *pinSet) error { return err } -// check refuses what one draw per reference path cannot answer for: a read of a level beside a path -// another read takes into it, and two reads of one table family that select different rows. Reads -// are compared in path order, so which pair is reported does not vary. +// check refuses what one draw per reference path cannot answer for: a read selecting a row its route +// pinned another of, a read of a level beside a path another read takes into it, and two reads of one +// table family that select different rows. Reads are compared in path order, so which pair is +// reported does not vary. func (s *readSurvey) check() error { + if s.clash != nil { + return s.clash + } sort.SliceStable(s.reads, func(i, j int) bool { if s.reads[i].at.group != s.reads[j].at.group { return s.reads[i].at.group < s.reads[j].at.group diff --git a/todo.md b/todo.md index 1811fdc..3033c48 100644 --- a/todo.md +++ b/todo.md @@ -34,10 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Make `readSurvey.edge` sound whatever order `checkNodeFences` runs its fences in: - report the pin clash as a load error, or have `checkDraws` run `checkOwnFamily` over - the scope before its survey, and delete the panic naming `checkOwnFamily`. All nine - seats found the survey sound only by a call order stated in that panic's text. - Test that `render` and `renderEdges` agree on every node kind, and each of the switches `node`'s comment lists, so a kind one of them misses fails a test: five seats, both architects among them, found the mirror held by comments alone. -- 2.52.0 From 64040c745fe9a37b13b247701646a51b7c4edbbd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 13:46:23 +0200 Subject: [PATCH 317/966] Give readSurvey's pin clash checkOwnFamily's advice --- readsurvey.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/readsurvey.go b/readsurvey.go index a40e81b..e422d23 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -126,7 +126,7 @@ func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { pins := at.pins.clone() if err := tr.replay(&pins); err != nil { if s.clash == nil { - s.clash = conflict(pathRead{at, a, tr}, err) + s.clash = fmt.Errorf("%s: %w; read the family from a template beside it, or add the value as a column", at.route.spelled(a.spelling), err) } return } -- 2.52.0 From 9d5b81d2f5e278e66ac2631db094d7097a7dc81f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:05:54 +0200 Subject: [PATCH 318/966] Test that every node switch handles every node kind, and render and renderEdges agree --- render_test.go | 94 ++++++++++++++++++++++++++++++++++++++++++++------ todo.md | 3 -- 2 files changed, 83 insertions(+), 14 deletions(-) diff --git a/render_test.go b/render_test.go index 8f0e5cd..6642a18 100644 --- a/render_test.go +++ b/render_test.go @@ -2,6 +2,9 @@ package fejkdata import ( "encoding/json" + "go/ast" + "slices" + "sort" "testing" ) @@ -59,16 +62,85 @@ type unhandledNode struct{} func (*unhandledNode) isNode() {} -func TestNodeSwitchesPanicOnAnUnhandledNode(t *testing.T) { - for name, call := range map[string]func(){ - "columnItems": func() { columnItems(&unhandledNode{}) }, - "contained": func() { contained(&unhandledNode{}) }, - "paths": func() { paths(&unhandledNode{}) }, - "prove": func() { (&valueProof{}).prove(&unhandledNode{}) }, - "render": func() { render(nil, &unhandledNode{}, renderScope{}) }, - "renderEdges": func() { renderEdges(&unhandledNode{}) }, - "stepInto": func() { _, _ = stepInto(&unhandledNode{}, "x") }, - } { - mustPanic(t, name, call) +// nodeSwitches are the switches node's comment lists, each over a scope holding row 0 +// of the sample table. +func nodeSwitches(tbl *table) map[string]func(node) { + return map[string]func(node){ + "columnItems": func(n node) { columnItems(n) }, + "contained": func(n node) { contained(n) }, + "paths": func(n node) { paths(n) }, + "prove": func(n node) { (&valueProof{}).prove(n) }, + "render": func(n node) { + var set holdSet + render(engine(1).rand, n, renderScope{set: &set, row: renderedRow{tbl, 0}}) + }, + "renderEdges": func(n node) { renderEdges(n) }, + "stepInto": func(n node) { _, _ = stepInto(n, "x") }, + } +} + +// nodeSwitchSkips is the kinds a switch is never handed: its callers step past them first. +var nodeSwitchSkips = map[string][]string{ + "columnItems": {"folder", "table", "tableRow"}, + "prove": {"folder"}, + "render": {"folder"}, + "stepInto": {"choice", "table", "tableRow"}, +} + +func TestNodeSwitchesHandleEveryKind(t *testing.T) { + f := newGenerator(t, writeFiles(t, map[string]string{ + "r.json": `{"format":"{name}","rows":"r.tsv","key":"code"}`, + "r.tsv": "code\tname\n01\t{digits(2)}\n02\tB\n", + })) + tbl := f.root.children["r"].(*table) + samples := map[string]node{ + "choice": compiled(t, `["a","b"]`), + "folder": &f.root, + "nullItem": &nullItem{}, + "table": tbl, + "tableColumn": tbl.formatTemplate.fields["name"], + "tableRow": tbl.rowNode, + "template": compiled(t, `{"format":"{x}","x":"1"}`), + } + var kinds []string + _, files := sourceFiles(t) + for _, file := range files { + for _, decl := range file.Decls { + if d, isFunc := decl.(*ast.FuncDecl); isFunc && d.Name.Name == "isNode" { + kinds = append(kinds, receiverType(d.Recv)) + } + } + } + sort.Strings(kinds) + sampled := make([]string, 0, len(samples)) + for kind := range samples { + sampled = append(sampled, kind) + } + sort.Strings(sampled) + if !slices.Equal(kinds, sampled) { + t.Fatalf("node kinds %v, samples %v; give every kind a sample here", kinds, sampled) + } + for name, call := range nodeSwitches(tbl) { + mustPanic(t, name+" on an unhandled node", func() { call(&unhandledNode{}) }) + for _, kind := range kinds { + if slices.Contains(nodeSwitchSkips[name], kind) { + mustPanic(t, name+" on a skipped "+kind, func() { call(samples[kind]) }) + continue + } + func() { + defer func() { + if r := recover(); r != nil { + t.Errorf("%s on %s: %v", name, kind, r) + } + }() + call(samples[kind]) + }() + } + } + for kind, n := range samples { + recurses := kind != "nullItem" && !slices.Contains(nodeSwitchSkips["render"], kind) + if edges := renderEdges(n); recurses != (len(edges) > 0) { + t.Errorf("%s: render recurses into it %v, renderEdges lists %d edges; a kind render recurses into has edges, and no other", kind, recurses, len(edges)) + } } } diff --git a/todo.md b/todo.md index 3033c48..d7ed416 100644 --- a/todo.md +++ b/todo.md @@ -34,9 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Test that `render` and `renderEdges` agree on every node kind, and each of the - switches `node`'s comment lists, so a kind one of them misses fails a test: five - seats, both architects among them, found the mirror held by comments alone. - Choose `disagreement`'s error directly, deleting `disagreementFix`'s switch returning a builder and `declaresLess` for `<`: seven seats traced the matrix for a message, and `disagreementFix`'s comment describes its parameter. -- 2.52.0 From 4ee0d61a6410182228090fa50fc97f836f0a0bbf Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:11:19 +0200 Subject: [PATCH 319/966] Name each node switch sample by its type --- render_test.go | 55 ++++++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 29 deletions(-) diff --git a/render_test.go b/render_test.go index 6642a18..28ab84a 100644 --- a/render_test.go +++ b/render_test.go @@ -3,8 +3,8 @@ package fejkdata import ( "encoding/json" "go/ast" + "reflect" "slices" - "sort" "testing" ) @@ -62,23 +62,6 @@ type unhandledNode struct{} func (*unhandledNode) isNode() {} -// nodeSwitches are the switches node's comment lists, each over a scope holding row 0 -// of the sample table. -func nodeSwitches(tbl *table) map[string]func(node) { - return map[string]func(node){ - "columnItems": func(n node) { columnItems(n) }, - "contained": func(n node) { contained(n) }, - "paths": func(n node) { paths(n) }, - "prove": func(n node) { (&valueProof{}).prove(n) }, - "render": func(n node) { - var set holdSet - render(engine(1).rand, n, renderScope{set: &set, row: renderedRow{tbl, 0}}) - }, - "renderEdges": func(n node) { renderEdges(n) }, - "stepInto": func(n node) { _, _ = stepInto(n, "x") }, - } -} - // nodeSwitchSkips is the kinds a switch is never handed: its callers step past them first. var nodeSwitchSkips = map[string][]string{ "columnItems": {"folder", "table", "tableRow"}, @@ -93,14 +76,17 @@ func TestNodeSwitchesHandleEveryKind(t *testing.T) { "r.tsv": "code\tname\n01\t{digits(2)}\n02\tB\n", })) tbl := f.root.children["r"].(*table) - samples := map[string]node{ - "choice": compiled(t, `["a","b"]`), - "folder": &f.root, - "nullItem": &nullItem{}, - "table": tbl, - "tableColumn": tbl.formatTemplate.fields["name"], - "tableRow": tbl.rowNode, - "template": compiled(t, `{"format":"{x}","x":"1"}`), + samples := map[string]node{} + for _, n := range []node{ + compiled(t, `["a","b"]`), + compiled(t, `{"format":"{x}","x":"1"}`), + &f.root, + &nullItem{}, + tbl, + tbl.formatTemplate.fields["name"], + tbl.rowNode, + } { + samples[reflect.TypeOf(n).Elem().Name()] = n } var kinds []string _, files := sourceFiles(t) @@ -111,16 +97,27 @@ func TestNodeSwitchesHandleEveryKind(t *testing.T) { } } } - sort.Strings(kinds) + slices.Sort(kinds) sampled := make([]string, 0, len(samples)) for kind := range samples { sampled = append(sampled, kind) } - sort.Strings(sampled) + slices.Sort(sampled) if !slices.Equal(kinds, sampled) { t.Fatalf("node kinds %v, samples %v; give every kind a sample here", kinds, sampled) } - for name, call := range nodeSwitches(tbl) { + for name, call := range map[string]func(node){ + "columnItems": func(n node) { columnItems(n) }, + "contained": func(n node) { contained(n) }, + "paths": func(n node) { paths(n) }, + "prove": func(n node) { (&valueProof{}).prove(n) }, + "render": func(n node) { + var set holdSet + render(engine(1).rand, n, renderScope{set: &set, row: renderedRow{tbl, 0}}) + }, + "renderEdges": func(n node) { renderEdges(n) }, + "stepInto": func(n node) { _, _ = stepInto(n, "x") }, + } { mustPanic(t, name+" on an unhandled node", func() { call(&unhandledNode{}) }) for _, kind := range kinds { if slices.Contains(nodeSwitchSkips[name], kind) { -- 2.52.0 From 8fb3695fa988b2cf3aa44c59c26bee8409cdcf6b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:12:36 +0200 Subject: [PATCH 320/966] Delete node's switch list, which the kind test now holds --- node.go | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/node.go b/node.go index c10069b..ccf9262 100644 --- a/node.go +++ b/node.go @@ -11,9 +11,7 @@ import ( // node is a compiled element of the namespace tree: a folder, choice, null, // template, table, column or row. Compiling JSON into these once (see compile) -// means rendering never re-inspects the raw JSON. A new kind needs a case in -// columnItems, contained, paths, prove, render, renderEdges and stepInto; each -// panics on a kind it does not list. +// means rendering never re-inspects the raw JSON. type node interface{ isNode() } // folder is a namespace of named children, built from a directory of JSON files -- 2.52.0 From ccc079a9caa1a6c4b77013b7bb57e1d3597fd6ce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:26:47 +0200 Subject: [PATCH 321/966] Test disagreement's two-text panic through disagreement itself --- datatype_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/datatype_test.go b/datatype_test.go index b6ceb2b..416f6bf 100644 --- a/datatype_test.go +++ b/datatype_test.go @@ -221,6 +221,6 @@ func TestNullColumn(t *testing.T) { } } -func TestDisagreementFixRefusesTwoTextItems(t *testing.T) { - mustPanic(t, "two text items", func() { disagreementFix(kindText, kindText, false) }) +func TestDisagreementRefusesTwoTextItems(t *testing.T) { + mustPanic(t, "two text items", func() { _ = disagreement(&template{}, DataTypeString, &template{}, DataTypeString) }) } -- 2.52.0 From 40f1129c23b90f70357858e433fbde92deac95b8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:27:34 +0200 Subject: [PATCH 322/966] Choose disagreement's error directly, ranking itemKind with < --- datatype.go | 73 +++++++++++++---------------------------------------- todo.md | 3 --- 2 files changed, 18 insertions(+), 58 deletions(-) diff --git a/datatype.go b/datatype.go index 07005f0..1cc5251 100644 --- a/datatype.go +++ b/datatype.go @@ -147,7 +147,7 @@ func columnItems(n node) (items []*template, nullable bool) { return items, nullable } -// itemKind is how a column item comes by its datatype, ranked by declaresLess in this order: none, read from a typed column, or declared. +// itemKind is how a column item comes by its datatype, ranked in this order: none, read from a typed column, or declared. type itemKind int const ( @@ -156,10 +156,6 @@ const ( kindDeclares ) -func declaresLess(a, b itemKind) bool { - return a < b -} - func kindOf(t *template) itemKind { switch { case t.datatype != DataTypeString: @@ -170,63 +166,30 @@ func kindOf(t *template) itemKind { return kindText } -// itemPair is two items of one column holding different datatypes: fix is the one to change, -// and want the datatype of the other. -type itemPair struct { - first, second DataType - fix, other *template - want DataType -} - // disagreement names the fix for items a and b of one column holding different datatypes, fixing // the one that declares least, a when both declare alike. func disagreement(a *template, da DataType, b *template, db DataType) error { - d := itemPair{first: da, second: db, fix: a, other: b, want: db} - if declaresLess(kindOf(b), kindOf(a)) { - d.fix, d.other, d.want = b, a, da + fix, other, want := a, b, db + if kindOf(b) < kindOf(a) { + fix, other, want = b, a, da } - proven := (&valueProof{}).proveColumnItem(d.fix).not[d.want] == "" - return disagreementFix(kindOf(d.fix), kindOf(d.other), proven)(d) -} - -// proven says fix's values hold other's datatype. -func disagreementFix(fix, other itemKind, proven bool) func(d itemPair) error { + held := fmt.Sprintf("its items hold %s and %s; a column holds one datatype", da, db) + proven := (&valueProof{}).proveColumnItem(fix).not[want] == "" switch { - case fix == kindDeclares: - return holdOne - case fix == kindReads && proven: - return readTyped - case fix == kindReads: - return readAsText - case other == kindText: + case kindOf(fix) == kindDeclares: + return errors.New(held) + case kindOf(fix) == kindReads && proven: + return fmt.Errorf("%s, so %s", held, typedAs(fix, want)) + case kindOf(fix) == kindReads: + return fmt.Errorf("%s, so to read %q as text, %s", held, fix.format, asText(fix)) + case kindOf(other) == kindText: panic(internalError("two text items hold one datatype, so they never disagree")) - case other == kindReads && !proven: - return otherAsText + case kindOf(other) == kindReads && !proven: + return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) + case fix.fromString: // an object may carry a weight, which this spelling would drop + return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, fix.format, fix.format, want) } - return declare -} - -func holdOne(d itemPair) error { - return fmt.Errorf("its items hold %s and %s; a column holds one datatype", d.first, d.second) -} - -func readTyped(d itemPair) error { - return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", d.first, d.second, typedAs(d.fix, d.want)) -} - -func readAsText(d itemPair) error { - return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", d.first, d.second, d.fix.format, asText(d.fix)) -} - -func otherAsText(d itemPair) error { - return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, d.fix.format, dataTypeNouns[d.want], d.other.format, asText(d.other)) -} - -func declare(d itemPair) error { - if d.fix.fromString { // an object may carry a weight, which this spelling would drop - return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, d.fix.format, d.fix.format, d.want) - } - return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, d.fix.format, d.want, d.want) + return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, fix.format, want, want) } // typedAs names the spelling giving a column-read item datatype d, keeping the other keys an object diff --git a/todo.md b/todo.md index d7ed416..b9c9d49 100644 --- a/todo.md +++ b/todo.md @@ -34,9 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Choose `disagreement`'s error directly, deleting `disagreementFix`'s switch returning - a builder and `declaresLess` for `<`: seven seats traced the matrix for a message, - and `disagreementFix`'s comment describes its parameter. - Give the expansion hold and the render's group hold two names: `groupHold` embeds `hold`, `held`, `heldLocal` and `holdSet` share its stem, and four seats could not tell which fence guards which. -- 2.52.0 From b99966fe19dbb6653655b294511e7cec913ae41d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:38:36 +0200 Subject: [PATCH 323/966] Name the render's draws apart from the expansion hold: drawMemo, groupDraws, renderDraws --- AGENTS.md | 2 +- README.md | 2 +- doc.go | 12 ++++---- docs/decisions.md | 10 +++---- hold.go => draws.go | 40 +++++++++++++------------- format.go | 4 +-- hold_test.go | 2 +- path.go | 20 ++++++------- perf_test.go | 6 ++-- readsurvey_test.go | 18 ++++++------ record.go | 12 ++++---- render.go | 70 ++++++++++++++++++++++----------------------- render_test.go | 8 +++--- struct.go | 4 +-- todo.md | 3 -- 15 files changed, 106 insertions(+), 107 deletions(-) rename hold.go => draws.go (66%) diff --git a/AGENTS.md b/AGENTS.md index 1d51302..23128d9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,7 +57,7 @@ In [docs/decisions.md](docs/decisions.md): - A render shares one reference draw per category, per group - A draw group name is local to its category - The expansion hold and the render's draws are two fences -- A hold set makes its draw maps on the first read +- A render's draws make their maps on the first read - A category never references itself, and a record's fences run at load - A record's column set is fixed before the first draw - Null is a `null` item, not a rate diff --git a/README.md b/README.md index 7e46d43..dac4db8 100644 --- a/README.md +++ b/README.md @@ -883,7 +883,7 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -hold.go the hold: one draw per held name; and the hold set and scope a render reads its reference paths through +draws.go the memo a hold or draw group keeps its draws in; and a render's draws and the scope it reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways drawfence.go the load fences over reference draws: a drawGroup's, and a render's and a record's columns through the survey readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them; and the fence on a table reaching its own family diff --git a/doc.go b/doc.go index efe7a17..6180994 100644 --- a/doc.go +++ b/doc.go @@ -19,15 +19,17 @@ package fejkdata // - draw — one pick from a choice, or one row taken from a table: `pick`, // `resolveChoice`, `table.drawRow`. // - expansion — one render of one format: `expand`. -// - render — what owns one `holdSet`, and so what one reference draw spans: +// - render — what owns one `renderDraws`, and so what one reference draw spans: // `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. -// - hold — keeping one draw of a name, so every route to it reads that value: -// `formatOps.held`, `hold`, `groupHold`, `readHeld`. -// - draw group — reference draws held apart inside one render: +// - hold — one expansion keeping one draw of a name, so every route to it reads +// that value: `formatOps.held`, `formatOps.keepsHold`, `heldCheck`. +// - memo — what a hold or a draw group keeps its draws in, the variant drawn at +// each level and the value each path read: `drawMemo`, `readMemo`. +// - draw group — reference draws kept apart inside one render: // `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a -// render reads it, `holdSet`, `renderScope`. +// render reads it, `renderDraws`, `groupDraws`, `renderScope`. // - pin — fixing which row of a table the render uses, which the draw fences replay: // `pinSet`, `pinSet.pin`, `table.drawIn`. // - family — a table and every table reaching it through a chain of diff --git a/docs/decisions.md b/docs/decisions.md index a4dcd47..4acfdcf 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -242,11 +242,11 @@ groups. They pin different things — an operand pins the value its own render p and stops at a reference, a path pins every level it passes through — so one walk would carry both rules and both scopes anyway, and tell them apart at every step. -## A hold set makes its draw maps on the first read +## A render's draws make their maps on the first read -Every render starts from an empty hold set, its draw groups included, since goal 2 prices -a reader meeting two spellings of one empty set. Making the maps where the set is declared kept a -record's on its frame's stack, but a `Fake`'s set lives on the `Generator`, so there it +Every render starts from empty draws, its draw groups included, since goal 2 prices +a reader meeting two spellings of one empty set. Making the maps where the draws are declared kept a +record's on its frame's stack, but a `Fake`'s draws live on the `Generator`, so there it cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09-28. Lazily, a render reading a reference path pays those two allocations, a record or struct included, and one reading none pays nothing; goal 11 holds either way. @@ -304,7 +304,7 @@ writes as the load check proved it; a `Value any` would hand every caller a type ## A unit takes the stem of what it is, and a file the stem of the units it holds -Goal 2 wants a name to reach one unit: `hold` is what a draw is kept in, `readValue` what +Goal 2 wants a name to reach one unit: `drawMemo` is what a draw is kept in, `readValue` what a read drew. ## One name, one meaning diff --git a/hold.go b/draws.go similarity index 66% rename from hold.go rename to draws.go index e63e517..eb5e4f7 100644 --- a/hold.go +++ b/draws.go @@ -2,18 +2,18 @@ package fejkdata import "strings" -// hold is what one expansion has drawn for the names it holds: the variant each was drawn as, so +// drawMemo is what a hold or a draw group has drawn: the variant each level was drawn as, so // every path under it reads one variant; and the value each read produced, by its path, so the // same read written twice reads one value. -type hold struct { +type drawMemo struct { variant map[string]node value map[string]readValue } -// groupHold is what one render's draw group has drawn for its reference paths: a hold, and the +// groupDraws is what one render's draw group has drawn for its reference paths: a memo, and the // table rows those paths pinned. -type groupHold struct { - hold +type groupDraws struct { + memo drawMemo pins pinSet } @@ -26,20 +26,20 @@ type readValue struct { // renderTrace is a test's view of the fields a render reads; nil outside a test. type renderTrace func(group string, row renderedRow, a arm) -// holdSet is one render's reference draws: the unnamed draw group's, and each named one's; and a -// test's trace of its reads. -type holdSet struct { - unnamed groupHold - named map[string]*groupHold +// renderDraws is one render's reference draws: the unnamed draw group's, and each named one's; +// and a test's trace of its reads. +type renderDraws struct { + unnamed groupDraws + named map[string]*groupDraws trace renderTrace } -// renderScope is where a render reads its reference paths: a hold set, in the draw group of the +// renderScope is where a render reads its reference paths: its reference draws, in the draw group of the // template rendering; and row, the row of the table rendering, which its columns read. It passes // by value, and what is read from it reaches a map key, an interface or a func value only as a -// copy; else sc, and with it every render's hold set, moves to the heap. +// copy; else sc, and with it every render's draws, move to the heap. type renderScope struct { - set *holdSet + draws *renderDraws group string row renderedRow } @@ -77,17 +77,17 @@ func (sc renderScope) rowOf(t *table) int { return sc.row.index } -func (sc renderScope) groupHold() *groupHold { +func (sc renderScope) groupDraws() *groupDraws { if sc.group == "" { - return &sc.set.unnamed + return &sc.draws.unnamed } - d, drew := sc.set.named[sc.group] + d, drew := sc.draws.named[sc.group] if !drew { - if sc.set.named == nil { - sc.set.named = map[string]*groupHold{} + if sc.draws.named == nil { + sc.draws.named = map[string]*groupDraws{} } - d = &groupHold{} - sc.set.named[strings.Clone(sc.group)] = d + d = &groupDraws{} + sc.draws.named[strings.Clone(sc.group)] = d } return d } diff --git a/format.go b/format.go index 599ec75..369917d 100644 --- a/format.go +++ b/format.go @@ -386,7 +386,7 @@ type formatOps struct { ops []op grow int held map[string]firstReach - heldLocal bool // some held name is not a reference path, so expand makes a hold + keepsHold bool // some held name is not a reference path, so expand makes a hold } // firstReach is how a format first reaches a held name: the reader holding it, for @@ -401,7 +401,7 @@ func (c *formatOps) holdName(a arm, label string) { c.held = map[string]firstReach{} } if !isRef(a.head) || len(a.tail) == 0 { - c.heldLocal = true + c.keepsHold = true } h, seen := c.held[a.head] if !seen { diff --git a/hold_test.go b/hold_test.go index b3621d5..df296dc 100644 --- a/hold_test.go +++ b/hold_test.go @@ -771,7 +771,7 @@ func TestReadFieldPanicsOnAPathItCannotHold(t *testing.T) { t.Fatal("not a template") } mustPanic(t, name, func() { - readField(engine(1).rand, tm, nil, renderScope{set: &holdSet{}}, a) + readField(engine(1).rand, tm, nil, renderScope{draws: &renderDraws{}}, a) }) } } diff --git a/path.go b/path.go index 59b7c16..3ce85a2 100644 --- a/path.go +++ b/path.go @@ -348,14 +348,14 @@ func (w *pathProbe) readRow(t *table, tail []string, r tableRoute) error { } // pathDraw draws the rows and variants a proved path reads, pinning the rows in pins; pins is nil -// for a sibling path, which never crosses a table, since a table is only a category. For a held -// read, held keeps the variant drawn at each level of a, so paths sharing a prefix share it. +// for a sibling path, which never crosses a table, since a table is only a category. For a memoized +// read, memo keeps the variant drawn at each level of a, so paths sharing a prefix share it. // Where pathCheck walks every variant of a choice, it walks the one drawn; at a table it selects // as pathCheck does and draws the row the route asks for. type pathDraw struct { s *generatorState pins *pinSet - held *hold + memo *drawMemo a *arm } @@ -400,20 +400,20 @@ func (w *pathDraw) walk(n node, tail []string) (node, error) { return n, nil } -// variant is the variant of c the walk continues into: the one held for this level -// of the read, drawn once, where the walk holds its draws; else one drawn afresh. +// variant is the variant of c the walk continues into: the one memo keeps for this level +// of the read, drawn once, where the walk has a memo; else one drawn afresh. func (w *pathDraw) variant(c *choice, rest []string) node { - if w.held == nil { + if w.memo == nil { return pick(w.s, c) } key := w.a.levels[len(w.a.tail)-len(rest)] - n, drew := w.held.variant[key] + n, drew := w.memo.variant[key] if !drew { n = resolveChoice(w.s, c) - if w.held.variant == nil { - w.held.variant = map[string]node{} + if w.memo.variant == nil { + w.memo.variant = map[string]node{} } - w.held.variant[key] = n + w.memo.variant[key] = n } return n } diff --git a/perf_test.go b/perf_test.go index 03389c1..8a0d2c7 100644 --- a/perf_test.go +++ b/perf_test.go @@ -55,7 +55,7 @@ func TestNoRenderAllocRegression(t *testing.T) { } } -// The repeat shape prices what keeps a hold set off the heap: a copy dropped costs an alloc an iteration. +// The repeat shape prices what keeps a render's draws off the heap: a copy dropped costs an alloc an iteration. func TestNoReferenceAllocRegression(t *testing.T) { word := `{"format":"{w}","w":["alpha","beta","gamma","delta"]}` for _, s := range []struct { @@ -73,12 +73,12 @@ func TestNoReferenceAllocRegression(t *testing.T) { t.Fatalf("New(%s): %v", s.name, err) } if allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") }); allocs > s.base*1.10 { - t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a hold set reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base) + t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a render's draws reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base) } } } -// A table read pins a row in the render's hold and reads its cells in place. +// A table read pins a row in the render's draws and reads its cells in place. func TestNoTableAllocRegression(t *testing.T) { f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{ "region.json": {Data: []byte(`{"format":"{name}","rows":"region.tsv","key":"code","weight":"population"}`)}, diff --git a/readsurvey_test.go b/readsurvey_test.go index d36cf28..a8733b0 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -77,17 +77,17 @@ func spellPins(p *pinSet) string { } func renderRoot(s *generatorState, t *template, trace renderTrace) { - set := holdSet{trace: trace} - sc := renderScope{set: &set} + draws := renderDraws{trace: trace} + sc := renderScope{draws: &draws} switch { case t.site.row == formatRow: tbl := t.site.table render(s, tbl, sc) for r := 0; r < tbl.rowCount(); r++ { - set := holdSet{trace: trace} - sc := renderScope{set: &set} - sc.groupHold().pins.pin(tbl, r) - render(s, tbl.rowNode, sc.at(tbl.rowNode, &sc.groupHold().pins)) + draws := renderDraws{trace: trace} + sc := renderScope{draws: &draws} + sc.groupDraws().pins.pin(tbl, r) + render(s, tbl.rowNode, sc.at(tbl.rowNode, &sc.groupDraws().pins)) } case t.repeat > 1: expand(s, t, sc) @@ -97,10 +97,10 @@ func renderRoot(s *generatorState, t *template, trace renderTrace) { } func renderRecordRoot(s *generatorState, t *template, columns []Column, trace renderTrace) { - set := holdSet{trace: trace} - sc := renderScope{set: &set} + draws := renderDraws{trace: trace} + sc := renderScope{draws: &draws} if t.site.row == formatRow { - t.site.table.drawIn(s, &sc.groupHold().pins) + t.site.table.drawIn(s, &sc.groupDraws().pins) } renderRecord(s, t, columns, sc) } diff --git a/record.go b/record.go index e8dee4f..77737c8 100644 --- a/record.go +++ b/record.go @@ -130,8 +130,8 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) } - var set holdSet - sc := renderScope{set: &set} + var draws renderDraws + sc := renderScope{draws: &draws} if t, isTable := n.(*table); isTable { if n, err = tableRecord(f.rand, t, tail, sc); err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) @@ -159,7 +159,7 @@ func tableRecord(s *generatorState, t *table, tail []string, sc renderScope) (no } switch n := n.(type) { case *table: - n.drawIn(s, &sc.groupHold().pins) + n.drawIn(s, &sc.groupDraws().pins) return n, nil case *tableRow: return n.t, nil @@ -204,8 +204,8 @@ type RecordTemplate struct { func (t *RecordTemplate) Fake() *Record { t.g.mu.Lock() defer t.g.mu.Unlock() - var set holdSet - return renderRecord(t.g.rand, t.template, t.columns, renderScope{set: &set}) + var draws renderDraws + return renderRecord(t.g.rand, t.template, t.columns, renderScope{draws: &draws}) } // NewRecordTemplate compiles an inline record — a JSON object with a format and @@ -264,7 +264,7 @@ func recordOf(n node) (*template, []Column, error) { func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.site.row == formatRow { - sc = sc.at(t.site.table.rowNode, &sc.groupHold().pins) + sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins) } r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { diff --git a/render.go b/render.go index 86a3aa2..e6f83f7 100644 --- a/render.go +++ b/render.go @@ -6,7 +6,7 @@ import ( ) // rng is the randomness a builtin sample draws from; *rand.Rand satisfies it. The -// render path takes the concrete *generatorState instead, which keeps the hold set of the walk +// render path takes the concrete *generatorState instead, which keeps the draws of the walk // drawing through it off the heap. type rng interface { IntN(n int) int @@ -24,8 +24,8 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } - var set holdSet - sc := renderScope{set: &set} + var draws renderDraws + sc := renderScope{draws: &draws} n, err := descend(f.rand, &f.root, segments, sc) if err != nil { return "", fmt.Errorf("fejkdata: %s: %w", path, err) @@ -33,7 +33,7 @@ func (f *Generator) Fake(path string) (string, error) { if _, ok := n.(*folder); ok { return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) } - return render(f.rand, n, sc.at(n, &sc.groupHold().pins)), nil + return render(f.rand, n, sc.at(n, &sc.groupDraws().pins)), nil } // descend walks a caller's path to the node it names, pinning in sc the rows it @@ -43,18 +43,18 @@ func descend(s *generatorState, root node, segments []string, sc renderScope) (n if _, err := (&pathProbe{}).walk(root, segments); err != nil { return nil, err } - return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.groupHold().pins}), nil + return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.groupDraws().pins}), nil } -// renderOnce renders n as one render, over a hold set of its own. +// renderOnce renders n as one render, over draws of its own. func renderOnce(s *generatorState, n node) string { - var set holdSet - return render(s, n, renderScope{set: &set}) + var draws renderDraws + return render(s, n, renderScope{draws: &draws}) } // render evaluates a compiled node to a string. compile validates every node up // front, so rendering a compiled tree cannot fail. sc holds the reference draws the -// render shares; each repeat iteration renders over a hold set of its own. +// render shares; each repeat iteration renders over draws of its own. // A child this switch renders is one renderEdges must list too, or the fences miss it. func render(s *generatorState, n node, sc renderScope) string { switch n := n.(type) { @@ -96,12 +96,12 @@ func render(s *generatorState, n node, sc renderScope) string { } // expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into -// render's loop, its hold set would move to the heap. +// render's loop, its draws would move to the heap. // //go:noinline func expandAnew(s *generatorState, t *template) string { - var set holdSet - return expand(s, t, renderScope{set: &set}) + var draws renderDraws + return expand(s, t, renderScope{draws: &draws}) } // pick selects one item. Uniform choices are O(1); weighted choices are an @@ -119,10 +119,10 @@ func expand(s *generatorState, t *template, sc renderScope) string { b.Grow(t.compiled.grow) // One draw per held name, for this expansion only: a nested template and each // repeat iteration get their own, since each is its own expansion. A reference - // path reads the render's hold in sc instead. - var held *hold - if t.compiled.heldLocal { - held = &hold{ + // path reads its draw group's memo in sc instead. + var hold *drawMemo + if t.compiled.keepsHold { + hold = &drawMemo{ variant: make(map[string]node, len(t.compiled.held)), value: make(map[string]readValue, len(t.compiled.held)), } @@ -133,7 +133,7 @@ func expand(s *generatorState, t *template, sc renderScope) string { case literalRun: b.WriteString(o.lit) case nameRead: - b.WriteString(readField(s, t, held, sc, o.arms[s.IntN(len(o.arms))]).text) + b.WriteString(readField(s, t, hold, sc, o.arms[s.IntN(len(o.arms))]).text) case builtinCall: // Read before the call, so the value a calc computes is the value the // format showed. calcVars fixed the order op.operands holds. @@ -141,7 +141,7 @@ func expand(s *generatorState, t *template, sc renderScope) string { if len(o.operands) > 0 { operands = make([]string, len(o.operands)) for j, a := range o.operands { - operands[j] = readField(s, t, held, sc, a).text + operands[j] = readField(s, t, hold, sc, a).text } } b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far @@ -152,15 +152,15 @@ func expand(s *generatorState, t *template, sc renderScope) string { // readField renders one arm of a token. A name the expansion holds — a level some // token addresses by a dotted path that is not a reference, or a field an operand -// reads — is drawn once and kept in held, so {place.postal-code} and {place.locality} +// reads — is drawn once and kept in hold, so {place.postal-code} and {place.locality} // read one row, either read twice gives one value, and a shown operand is the operand // computed. Every other name is drawn afresh, so {word} {word} still draws twice. -func readField(s *generatorState, t *template, held *hold, sc renderScope, a arm) readValue { +func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue { if isRef(a.head) && len(a.tail) > 0 { return readReference(s, t, sc, a) } - if sc.set.trace != nil { - traceRead(sc.set.trace, t, sc, a) + if sc.draws.trace != nil { + traceRead(sc.draws.trace, t, sc, a) } if _, expansionHolds := t.compiled.held[a.head]; !expansionHolds { if len(a.tail) > 0 { @@ -168,35 +168,35 @@ func readField(s *generatorState, t *template, held *hold, sc renderScope, a arm } return readValue{text: render(s, t.head(a.head), sc)} } - if held == nil { + if hold == nil { panic(internalError("%q reads a name the expansion holds, with no hold to keep it in", a.spelling)) } - return readHeld(s, t, held, nil, sc, a) + return readMemo(s, t, hold, nil, sc, a) } -// readReference reads a reference path, kept in the render's hold for its group, so its draw +// readReference reads a reference path, kept in its draw group's memo, so its draw // spans the render. func readReference(s *generatorState, t *template, sc renderScope, a arm) readValue { - if sc.set.trace != nil { - traceRead(sc.set.trace, t, sc, a) + if sc.draws.trace != nil { + traceRead(sc.draws.trace, t, sc, a) } - group := sc.groupHold() - return readHeld(s, t, &group.hold, &group.pins, sc, a) + group := sc.groupDraws() + return readMemo(s, t, &group.memo, &group.pins, sc, a) } -func readHeld(s *generatorState, t *template, held *hold, pins *pinSet, sc renderScope, a arm) readValue { - if r, done := held.value[a.path]; done { +func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc renderScope, a arm) readValue { + if r, done := memo.value[a.path]; done { return r } - leaf := drawPath(t.head(a.head), a.tail, a.head, &pathDraw{s: s, held: held, pins: pins, a: &a}) + leaf := drawPath(t.head(a.head), a.tail, a.head, &pathDraw{s: s, memo: memo, pins: pins, a: &a}) if pins != nil { sc = sc.at(leaf, pins) } r := renderLeaf(s, leaf, sc) - if held.value == nil { - held.value = map[string]readValue{} + if memo.value == nil { + memo.value = map[string]readValue{} } - held.value[a.path] = r + memo.value[a.path] = r return r } diff --git a/render_test.go b/render_test.go index 28ab84a..6bdea1d 100644 --- a/render_test.go +++ b/render_test.go @@ -42,8 +42,8 @@ func TestGrowIsALowerBound(t *testing.T) { t.Fatalf("format %q did not compile to a template", format) } for i := 0; i < 50; i++ { - var set holdSet - if got := len(expand(f.rand, tmpl, renderScope{set: &set})); got < tmpl.compiled.grow { + var draws renderDraws + if got := len(expand(f.rand, tmpl, renderScope{draws: &draws})); got < tmpl.compiled.grow { t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.compiled.grow) } } @@ -112,8 +112,8 @@ func TestNodeSwitchesHandleEveryKind(t *testing.T) { "paths": func(n node) { paths(n) }, "prove": func(n node) { (&valueProof{}).prove(n) }, "render": func(n node) { - var set holdSet - render(engine(1).rand, n, renderScope{set: &set, row: renderedRow{tbl, 0}}) + var draws renderDraws + render(engine(1).rand, n, renderScope{draws: &draws, row: renderedRow{tbl, 0}}) }, "renderEdges": func(n node) { renderEdges(n) }, "stepInto": func(n node) { _, _ = stepInto(n, "x") }, diff --git a/struct.go b/struct.go index f51f218..eb38a56 100644 --- a/struct.go +++ b/struct.go @@ -351,8 +351,8 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro // fill draws the record into v's tagged fields, then each nested struct as a record of its own. func (s *structShape) fill(state *generatorState, v reflect.Value) { if s.record != nil { - var set holdSet - for i, c := range renderRecord(state, s.record, s.columns, renderScope{set: &set}).columns { + var draws renderDraws + for i, c := range renderRecord(state, s.record, s.columns, renderScope{draws: &draws}).columns { setColumn(fieldAt(v, s.fieldIndexes[i]), c) } } diff --git a/todo.md b/todo.md index b9c9d49..f0a2a90 100644 --- a/todo.md +++ b/todo.md @@ -34,9 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Give the expansion hold and the render's group hold two names: `groupHold` embeds - `hold`, `held`, `heldLocal` and `holdSet` share its stem, and four seats could not - tell which fence guards which. - Word the three overlap refusals apart, `checkNameHeld`'s, `checkNoOverlap`'s and `overlapError`'s: three seats grepped one message to three fences. - Compile a format in `format.go`: `linkTemplate` calls `compileFormat` in `node.go`, -- 2.52.0 From f1b0cda818b8f4501765663814b214ab8c34750e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:41:46 +0200 Subject: [PATCH 324/966] Say a draw group holds a reference path, and correct the stale hold wording --- doc.go | 5 +++-- docs/decisions.md | 2 +- draws.go | 2 +- record.go | 2 +- 4 files changed, 6 insertions(+), 5 deletions(-) diff --git a/doc.go b/doc.go index 6180994..949e257 100644 --- a/doc.go +++ b/doc.go @@ -23,8 +23,9 @@ package fejkdata // `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. -// - hold — one expansion keeping one draw of a name, so every route to it reads -// that value: `formatOps.held`, `formatOps.keepsHold`, `heldCheck`. +// - hold — keeping one draw of a name, so every route to it reads that value: +// an expansion holds a sibling path's head and an operand's field, its draw +// group a reference path's: `formatOps.held`, `formatOps.keepsHold`, `heldCheck`. // - memo — what a hold or a draw group keeps its draws in, the variant drawn at // each level and the value each path read: `drawMemo`, `readMemo`. // - draw group — reference draws kept apart inside one render: diff --git a/docs/decisions.md b/docs/decisions.md index 4acfdcf..4dcd1b5 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -680,7 +680,7 @@ Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` stay four loops. One loop fails `perf_test.go`: the check's leaves, the probe's selectors and the errors naming a level leak the walker's contents, and Go tracks a struct's fields as one, so -the draw's pins, arm and hold leak with them. Held by pointer, a repeat of a reference +the draw's pins, arm and memo leak with them. Held by pointer, a repeat of a reference path rose from 66 to 106 allocations; copied into the walker by value, `BenchmarkHeld` rose from 5 to 9. A call through an interface or a type parameter leaks the walker the same way. Valid while Go's escape analysis tracks a struct's fields as diff --git a/draws.go b/draws.go index eb5e4f7..a7f5894 100644 --- a/draws.go +++ b/draws.go @@ -37,7 +37,7 @@ type renderDraws struct { // renderScope is where a render reads its reference paths: its reference draws, in the draw group of the // template rendering; and row, the row of the table rendering, which its columns read. It passes // by value, and what is read from it reaches a map key, an interface or a func value only as a -// copy; else sc, and with it every render's draws, move to the heap. +// copy; else sc, and with it every render's draws, moves to the heap. type renderScope struct { draws *renderDraws group string diff --git a/record.go b/record.go index 77737c8..4d76149 100644 --- a/record.go +++ b/record.go @@ -260,7 +260,7 @@ func recordOf(n node) (*template, []Column, error) { } // renderRecord draws each column once, in the name order recordOf fixed, as one render -// over sc's hold; a table's columns read the row pinned there. +// over sc's draws; a table's columns read the row pinned there. func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.site.row == formatRow { -- 2.52.0 From 487f60c6af51182c3f5245eb11aab34f40b85710 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:48:43 +0200 Subject: [PATCH 325/966] Pin each overlap refusal to its own wording --- hold_test.go | 10 +++++----- inline_test.go | 4 ++-- record_test.go | 4 ++-- reference_test.go | 6 +++--- struct_test.go | 2 +- table_test.go | 4 ++-- 6 files changed, 15 insertions(+), 15 deletions(-) diff --git a/hold_test.go b/hold_test.go index df296dc..8f375b5 100644 --- a/hold_test.go +++ b/hold_test.go @@ -56,7 +56,7 @@ func TestRenderingALevelAndReadingIntoItIsRejected(t *testing.T) { t.Errorf("%s: New = nil error, want the overlapping tokens rejected", name) continue } - if !strings.HasPrefix(err.Error(), "fejkdata: cat: ") || !strings.Contains(err.Error(), "reads a path into") { + if !strings.HasPrefix(err.Error(), "fejkdata: cat: ") || !strings.Contains(err.Error(), "renders a level that") { t.Errorf("%s: New = %v, want it to name the category path and the overlap", name, err) } } @@ -76,7 +76,7 @@ func TestCalcOperandNamingAHeldLevelIsRejected(t *testing.T) { "cat": `{"format":"{item.price}|{calc(item * 1)}","item":[` + `{"format":"{price}","price":"10"},{"format":"{price}","price":"20"}]}`, }))) - if err == nil || !strings.Contains(err.Error(), "reads a path into") { + if err == nil || !strings.Contains(err.Error(), "renders a level that") { t.Fatalf("New = %v, want the calc operand rejected as an overlap", err) } // Arithmetic over fields no path names is untouched. @@ -113,7 +113,7 @@ func TestReferenceNamingAHeldLevelIsRejected(t *testing.T) { } for name, files := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files))) - if err == nil || !strings.Contains(err.Error(), "reads a path into") { + if err == nil || !strings.Contains(err.Error(), "by a second route") { t.Errorf("%s: New = %v, want the reference rejected as an overlap", name, err) } } @@ -147,7 +147,7 @@ func TestALevelRenderedOnlyByAPathTokenIsHeld(t *testing.T) { `"q":{"format":"{x}","x":["1","2"]}}`, "hop": `"{/thing.q}"`, }))) - if err == nil || !strings.Contains(err.Error(), "reads a path into") { + if err == nil || !strings.Contains(err.Error(), "by a second route") { t.Fatalf("New = %v, want the second route to q rejected", err) } } @@ -308,7 +308,7 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) { `"q":{"format":"{y}","y":["1","2"]}}`, "hop": `"{/cat.q}"`, }, - "reads a path into", + "by a second route", }, } for name, c := range rejected { diff --git a/inline_test.go b/inline_test.go index c5507e7..ffb4572 100644 --- a/inline_test.go +++ b/inline_test.go @@ -108,8 +108,8 @@ func TestFakeTemplateErrors(t *testing.T) { {`name: {/no.such.path}`, "no entry"}, {`name: {..nope}`, "write {/nope}"}, {`{"format":"x"}`, "is a string"}, - {`{/misc.territory} {/misc.territory.alpha2}`, "renders a level that {/misc.territory.alpha2} reads a path into; name the fields you want instead, or draw them apart with a drawGroup"}, - {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "reads a path into"}, + {`{/misc.territory} {/misc.territory.alpha2}`, "reads whole the reference draw that {/misc.territory.alpha2} reads a path into; name the fields you want instead, or draw them apart with a drawGroup"}, + {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "reads whole the reference draw"}, {`{"format":"{/misc.territory.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) diff --git a/record_test.go b/record_test.go index 00f00a4..e91ff2c 100644 --- a/record_test.go +++ b/record_test.go @@ -220,7 +220,7 @@ func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) { {"a column reaching back through another category", `{"format":"","whole":"{/mid}","inner":"{/cat.a.b}"}`}, } { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}))) - if err == nil || !strings.Contains(err.Error(), "reads a path into") { + if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { t.Errorf("%s: New = %v, want the overlap refused at load, the way one format is", c.name, err) continue } @@ -236,7 +236,7 @@ func TestInlineRecordRejectsOverlappingColumns(t *testing.T) { }) f := newGenerator(t, dir, WithSeed(1)) _, err := f.FakeRecordTemplate(`{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`) - if err == nil || !strings.Contains(err.Error(), "reads a path into") { + if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { t.Fatalf("inline record over an overlapping pair = %v, want the inline entry point to refuse it too", err) } } diff --git a/reference_test.go b/reference_test.go index 5ea685f..b0d032c 100644 --- a/reference_test.go +++ b/reference_test.go @@ -271,7 +271,7 @@ func TestReferenceOverlapIsRejected(t *testing.T) { "a bare reference beside a path into what it never renders": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`, } { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row}))) - if err == nil || !strings.Contains(err.Error(), "reads a path into") { + if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { t.Errorf("%s: New = %v, want the overlap rejected", name, err) } } @@ -280,7 +280,7 @@ func TestReferenceOverlapIsRejected(t *testing.T) { if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ "row": `{"format":"{p.first} {/hop}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`, "hop": `"{/row.p.last}"`, - }))); err == nil || !strings.Contains(err.Error(), "reads a path into") { + }))); err == nil || !strings.Contains(err.Error(), "by a second route") { t.Errorf("New = %v, want a sibling path beside a reference into it rejected", err) } if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ @@ -413,7 +413,7 @@ func TestSpellingsOfOneReferenceAreOneLevel(t *testing.T) { "sv_SE/person": person, "sv_SE/mail": `"{.person} <{/sv_SE.person.first}>"`, }))) - if err == nil || !strings.Contains(err.Error(), "reads a path into") || !strings.Contains(err.Error(), "{.person}") { + if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") || !strings.Contains(err.Error(), "{.person}") { t.Errorf("New = %v, want the bare spelling rejected beside the path spelling", err) } dir := writeData(t, map[string]string{ diff --git a/struct_test.go b/struct_test.go index fd48f8f..aa626cb 100644 --- a/struct_test.go +++ b/struct_test.go @@ -296,7 +296,7 @@ func TestFakeStructErrors(t *testing.T) { {&struct { A string `fake:"trip.leg"` B string `fake:"trip.leg.to"` - }{}, "reads a path into"}, + }{}, "reads whole the reference draw"}, {&struct { Trip struct { A int `fake:"{digits(3)}"` diff --git a/table_test.go b/table_test.go index d247f38..d642d8f 100644 --- a/table_test.go +++ b/table_test.go @@ -367,7 +367,7 @@ func TestTableSelectorInAReference(t *testing.T) { func TestTableSelectionFences(t *testing.T) { rejected := map[string]struct{ json, want string }{ - "bare beside a path": {`"{/region} {/region.name}"`, "reads a path into"}, + "bare beside a path": {`"{/region} {/region.name}"`, "reads whole the reference draw"}, "bare beside a linked path": {`"{/region} {/locality.name}"`, "drawGroup"}, "selected beside unselected": {`"{/region[12].municipality.name} {/municipality.code}"`, "{/region[12].municipality.code}"}, "relative selected beside unselected": {`"{.region[12].municipality.name} {.municipality.code}"`, "{.region[12].municipality.code}"}, @@ -1120,7 +1120,7 @@ func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { "region.json": `{"format":"{name}={note}","rows":"region.tsv","key":"code","name":"name","weight":"population"}`, "x.json": `"{/region}|{/municipality}"`, }) - if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "reads a path into") { + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { t.Fatalf("New = %v, want the two drawn rows held to one draw of addr", err) } nested := with(whole, map[string]string{ -- 2.52.0 From 2e57a38e0b91f8ca048c3a90e8d208a6a408f35a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:50:08 +0200 Subject: [PATCH 326/966] Word the second-route and reference-draw overlap refusals apart from the format's own --- holdfence.go | 2 +- readsurvey.go | 2 +- todo.md | 2 -- 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/holdfence.go b/holdfence.go index b754fdd..3248072 100644 --- a/holdfence.go +++ b/holdfence.go @@ -78,7 +78,7 @@ func checkNameHeld(t *template, name string, readers []reader) error { continue } if path != "" { - return fmt.Errorf("%s renders %q, which {%s} reads a path into; name the fields you want instead", e.reached(), name, path) + return fmt.Errorf("%s reaches %q by a second route, where {%s} holds it to one draw; name the fields you want instead", e.reached(), name, path) } return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), name, operandReader(t, name)) } diff --git a/readsurvey.go b/readsurvey.go index e422d23..09735ce 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -356,5 +356,5 @@ func alternatives(a, b surveyAt) bool { } func overlapError(route surveyRoute, ref string, into pathRead) error { - return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) + return fmt.Errorf("%s reads whole the reference draw that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) } diff --git a/todo.md b/todo.md index f0a2a90..80e5e76 100644 --- a/todo.md +++ b/todo.md @@ -34,8 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Word the three overlap refusals apart, `checkNameHeld`'s, `checkNoOverlap`'s and - `overlapError`'s: three seats grepped one message to three fences. - Compile a format in `format.go`: `linkTemplate` calls `compileFormat` in `node.go`, which calls `compileOps` and `holdfence.go`'s two checks, and both architects placed the step wrong. -- 2.52.0 From 7cb07bfc5dfe4d34e75f314d86e9493bcbf6cfaa Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:53:14 +0200 Subject: [PATCH 327/966] Pin the reference-draw overlap to wording true of a path and of a family table --- inline_test.go | 4 ++-- record_test.go | 4 ++-- reference_test.go | 4 ++-- struct_test.go | 2 +- table_test.go | 6 +++--- 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/inline_test.go b/inline_test.go index ffb4572..879b5ea 100644 --- a/inline_test.go +++ b/inline_test.go @@ -108,8 +108,8 @@ func TestFakeTemplateErrors(t *testing.T) { {`name: {/no.such.path}`, "no entry"}, {`name: {..nope}`, "write {/nope}"}, {`{"format":"x"}`, "is a string"}, - {`{/misc.territory} {/misc.territory.alpha2}`, "reads whole the reference draw that {/misc.territory.alpha2} reads a path into; name the fields you want instead, or draw them apart with a drawGroup"}, - {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "reads whole the reference draw"}, + {`{/misc.territory} {/misc.territory.alpha2}`, "and {/misc.territory.alpha2} share one reference draw, one read whole and one by a path; name the fields you want instead, or draw them apart with a drawGroup"}, + {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "share one reference draw"}, {`{"format":"{/misc.territory.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) diff --git a/record_test.go b/record_test.go index e91ff2c..17d2ebb 100644 --- a/record_test.go +++ b/record_test.go @@ -220,7 +220,7 @@ func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) { {"a column reaching back through another category", `{"format":"","whole":"{/mid}","inner":"{/cat.a.b}"}`}, } { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}))) - if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { + if err == nil || !strings.Contains(err.Error(), "share one reference draw") { t.Errorf("%s: New = %v, want the overlap refused at load, the way one format is", c.name, err) continue } @@ -236,7 +236,7 @@ func TestInlineRecordRejectsOverlappingColumns(t *testing.T) { }) f := newGenerator(t, dir, WithSeed(1)) _, err := f.FakeRecordTemplate(`{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`) - if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { + if err == nil || !strings.Contains(err.Error(), "share one reference draw") { t.Fatalf("inline record over an overlapping pair = %v, want the inline entry point to refuse it too", err) } } diff --git a/reference_test.go b/reference_test.go index b0d032c..15af699 100644 --- a/reference_test.go +++ b/reference_test.go @@ -271,7 +271,7 @@ func TestReferenceOverlapIsRejected(t *testing.T) { "a bare reference beside a path into what it never renders": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`, } { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row}))) - if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { + if err == nil || !strings.Contains(err.Error(), "share one reference draw") { t.Errorf("%s: New = %v, want the overlap rejected", name, err) } } @@ -413,7 +413,7 @@ func TestSpellingsOfOneReferenceAreOneLevel(t *testing.T) { "sv_SE/person": person, "sv_SE/mail": `"{.person} <{/sv_SE.person.first}>"`, }))) - if err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") || !strings.Contains(err.Error(), "{.person}") { + if err == nil || !strings.Contains(err.Error(), "share one reference draw") || !strings.Contains(err.Error(), "{.person}") { t.Errorf("New = %v, want the bare spelling rejected beside the path spelling", err) } dir := writeData(t, map[string]string{ diff --git a/struct_test.go b/struct_test.go index aa626cb..8300c71 100644 --- a/struct_test.go +++ b/struct_test.go @@ -296,7 +296,7 @@ func TestFakeStructErrors(t *testing.T) { {&struct { A string `fake:"trip.leg"` B string `fake:"trip.leg.to"` - }{}, "reads whole the reference draw"}, + }{}, "share one reference draw"}, {&struct { Trip struct { A int `fake:"{digits(3)}"` diff --git a/table_test.go b/table_test.go index d642d8f..541c6f0 100644 --- a/table_test.go +++ b/table_test.go @@ -367,8 +367,8 @@ func TestTableSelectorInAReference(t *testing.T) { func TestTableSelectionFences(t *testing.T) { rejected := map[string]struct{ json, want string }{ - "bare beside a path": {`"{/region} {/region.name}"`, "reads whole the reference draw"}, - "bare beside a linked path": {`"{/region} {/locality.name}"`, "drawGroup"}, + "bare beside a path": {`"{/region} {/region.name}"`, "share one reference draw"}, + "bare beside a linked path": {`"{/region} {/locality.name}"`, "share one reference draw"}, "selected beside unselected": {`"{/region[12].municipality.name} {/municipality.code}"`, "{/region[12].municipality.code}"}, "relative selected beside unselected": {`"{.region[12].municipality.name} {.municipality.code}"`, "{.region[12].municipality.code}"}, "two selections": {`"{/region[12].name} {/region[14].name}"`, "drawGroup"}, @@ -1120,7 +1120,7 @@ func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { "region.json": `{"format":"{name}={note}","rows":"region.tsv","key":"code","name":"name","weight":"population"}`, "x.json": `"{/region}|{/municipality}"`, }) - if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "reads whole the reference draw") { + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "share one reference draw") { t.Fatalf("New = %v, want the two drawn rows held to one draw of addr", err) } nested := with(whole, map[string]string{ -- 2.52.0 From 5c8e7d66d0d97a4fddd6d5418f21dbaa507ebf63 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:53:45 +0200 Subject: [PATCH 328/966] Word the reference-draw overlap so it holds for a path and a family table --- readsurvey.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/readsurvey.go b/readsurvey.go index 09735ce..a8e0059 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -356,5 +356,5 @@ func alternatives(a, b surveyAt) bool { } func overlapError(route surveyRoute, ref string, into pathRead) error { - return fmt.Errorf("%s reads whole the reference draw that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) + return fmt.Errorf("%s and %s share one reference draw, one read whole and one by a path; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) } -- 2.52.0 From 68dc2e594a9d9c255127f40a3dbc1ea828d91381 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:58:32 +0200 Subject: [PATCH 329/966] Pin the reference-draw overlap to naming the disagreement and the read a drawGroup goes on --- inline_test.go | 4 ++-- record_test.go | 4 ++-- reference_test.go | 20 +++++++++++++------- struct_test.go | 2 +- table_test.go | 6 +++--- 5 files changed, 21 insertions(+), 15 deletions(-) diff --git a/inline_test.go b/inline_test.go index 879b5ea..fb97109 100644 --- a/inline_test.go +++ b/inline_test.go @@ -108,8 +108,8 @@ func TestFakeTemplateErrors(t *testing.T) { {`name: {/no.such.path}`, "no entry"}, {`name: {..nope}`, "write {/nope}"}, {`{"format":"x"}`, "is a string"}, - {`{/misc.territory} {/misc.territory.alpha2}`, "and {/misc.territory.alpha2} share one reference draw, one read whole and one by a path; name the fields you want instead, or draw them apart with a drawGroup"}, - {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "share one reference draw"}, + {`{/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads a path through; name the fields you want instead, or move {/misc.territory.alpha2} into a field with a drawGroup"}, + {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "renders its own draw of what"}, {`{"format":"{/misc.territory.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) diff --git a/record_test.go b/record_test.go index 17d2ebb..bee5950 100644 --- a/record_test.go +++ b/record_test.go @@ -220,7 +220,7 @@ func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) { {"a column reaching back through another category", `{"format":"","whole":"{/mid}","inner":"{/cat.a.b}"}`}, } { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}))) - if err == nil || !strings.Contains(err.Error(), "share one reference draw") { + if err == nil || !strings.Contains(err.Error(), "renders its own draw of what") { t.Errorf("%s: New = %v, want the overlap refused at load, the way one format is", c.name, err) continue } @@ -236,7 +236,7 @@ func TestInlineRecordRejectsOverlappingColumns(t *testing.T) { }) f := newGenerator(t, dir, WithSeed(1)) _, err := f.FakeRecordTemplate(`{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`) - if err == nil || !strings.Contains(err.Error(), "share one reference draw") { + if err == nil || !strings.Contains(err.Error(), "renders its own draw of what") { t.Fatalf("inline record over an overlapping pair = %v, want the inline entry point to refuse it too", err) } } diff --git a/reference_test.go b/reference_test.go index 15af699..3f5d5a6 100644 --- a/reference_test.go +++ b/reference_test.go @@ -271,10 +271,14 @@ func TestReferenceOverlapIsRejected(t *testing.T) { "a bare reference beside a path into what it never renders": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`, } { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row}))) - if err == nil || !strings.Contains(err.Error(), "share one reference draw") { + if err == nil || !strings.Contains(err.Error(), "renders its own draw of what") { t.Errorf("%s: New = %v, want the overlap rejected", name, err) } } + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`}))) + if err == nil || !strings.HasSuffix(err.Error(), "or give {b} a drawGroup") { + t.Errorf("New = %v, want the drawGroup named on the path's field", err) + } // A sibling path and a reference into the level it holds are the same overlap, // and the reference reaches it from a category row renders. if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ @@ -283,11 +287,13 @@ func TestReferenceOverlapIsRejected(t *testing.T) { }))); err == nil || !strings.Contains(err.Error(), "by a second route") { t.Errorf("New = %v, want a sibling path beside a reference into it rejected", err) } - if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ - "cat": cat, - "row": `{"format":"{a} {b}","a":{"format":"{/cat.p}","drawGroup":"g"},"b":"{/cat.p.first}"}`, - }))); err != nil { - t.Errorf("New = %v, want a level and a path into it accepted in groups of their own", err) + for _, row := range []string{ + `{"format":"{a} {b}","a":{"format":"{/cat.p}","drawGroup":"g"},"b":"{/cat.p.first}"}`, + `{"format":"{a} {b}","a":"{/cat}","b":{"format":"{/cat.p.first}","drawGroup":"g"}}`, + } { + if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row}))); err != nil { + t.Errorf("New(%s) = %v, want a level and a path into it accepted in groups of their own", row, err) + } } } @@ -413,7 +419,7 @@ func TestSpellingsOfOneReferenceAreOneLevel(t *testing.T) { "sv_SE/person": person, "sv_SE/mail": `"{.person} <{/sv_SE.person.first}>"`, }))) - if err == nil || !strings.Contains(err.Error(), "share one reference draw") || !strings.Contains(err.Error(), "{.person}") { + if err == nil || !strings.Contains(err.Error(), "renders its own draw of what") || !strings.Contains(err.Error(), "{.person}") { t.Errorf("New = %v, want the bare spelling rejected beside the path spelling", err) } dir := writeData(t, map[string]string{ diff --git a/struct_test.go b/struct_test.go index 8300c71..b2b579f 100644 --- a/struct_test.go +++ b/struct_test.go @@ -296,7 +296,7 @@ func TestFakeStructErrors(t *testing.T) { {&struct { A string `fake:"trip.leg"` B string `fake:"trip.leg.to"` - }{}, "share one reference draw"}, + }{}, "renders its own draw of what"}, {&struct { Trip struct { A int `fake:"{digits(3)}"` diff --git a/table_test.go b/table_test.go index 541c6f0..d7b8be5 100644 --- a/table_test.go +++ b/table_test.go @@ -367,8 +367,8 @@ func TestTableSelectorInAReference(t *testing.T) { func TestTableSelectionFences(t *testing.T) { rejected := map[string]struct{ json, want string }{ - "bare beside a path": {`"{/region} {/region.name}"`, "share one reference draw"}, - "bare beside a linked path": {`"{/region} {/locality.name}"`, "share one reference draw"}, + "bare beside a path": {`"{/region} {/region.name}"`, "renders its own draw of what"}, + "bare beside a linked path": {`"{/region} {/locality.name}"`, "renders its own draw of what"}, "selected beside unselected": {`"{/region[12].municipality.name} {/municipality.code}"`, "{/region[12].municipality.code}"}, "relative selected beside unselected": {`"{.region[12].municipality.name} {.municipality.code}"`, "{.region[12].municipality.code}"}, "two selections": {`"{/region[12].name} {/region[14].name}"`, "drawGroup"}, @@ -1120,7 +1120,7 @@ func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { "region.json": `{"format":"{name}={note}","rows":"region.tsv","key":"code","name":"name","weight":"population"}`, "x.json": `"{/region}|{/municipality}"`, }) - if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "share one reference draw") { + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "renders its own draw of what") { t.Fatalf("New = %v, want the two drawn rows held to one draw of addr", err) } nested := with(whole, map[string]string{ -- 2.52.0 From ba8bf06c004ef0135e8216e70dcda8057d067b49 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 14:59:16 +0200 Subject: [PATCH 330/966] Name the reference-draw overlap's disagreement and the read a drawGroup goes on --- readsurvey.go | 6 +++++- todo.md | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/readsurvey.go b/readsurvey.go index a8e0059..11ae00f 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -356,5 +356,9 @@ func alternatives(a, b surveyAt) bool { } func overlapError(route surveyRoute, ref string, into pathRead) error { - return fmt.Errorf("%s and %s share one reference draw, one read whole and one by a path; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.spelling)) + apart := fmt.Sprintf("move {%s} into a field with a drawGroup", into.a.spelling) + if into.a.spelling != into.at.route.label { + apart = fmt.Sprintf("give %s a drawGroup", into.at.route.spelling) + } + return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead, or %s", route.spelled(ref), into.at.route.spelled(into.a.spelling), apart) } diff --git a/todo.md b/todo.md index 80e5e76..6d460a3 100644 --- a/todo.md +++ b/todo.md @@ -222,6 +222,10 @@ by a `data-import/` script, as README goal 12 asks. - Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises: `datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. +- Name the spelling to use in `checkNameHeld`'s and `checkNoOverlap`'s refusals, as the + README's hold section promises: both answer "name the fields you want instead", and + `{n} reaches "p" by a second route` names neither the read the route ends on, + `{/row.p.last}`, nor the spelling to write, `{p.last}`. - Call `templateError` a compile failure: its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. - Name the node a selector follows in `stepInto`'s refusal: `sv_SE.person[1].first` -- 2.52.0 From 82ea5994580dcf446f314d0f635c07ecac0a6258 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:05:13 +0200 Subject: [PATCH 331/966] Pin the drawGroup advice to a field that splits the pair, or none --- reference_test.go | 11 ++++++++--- table_test.go | 4 ++-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/reference_test.go b/reference_test.go index 3f5d5a6..da4f750 100644 --- a/reference_test.go +++ b/reference_test.go @@ -275,9 +275,14 @@ func TestReferenceOverlapIsRejected(t *testing.T) { t.Errorf("%s: New = %v, want the overlap rejected", name, err) } } - _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`}))) - if err == nil || !strings.HasSuffix(err.Error(), "or give {b} a drawGroup") { - t.Errorf("New = %v, want the drawGroup named on the path's field", err) + for row, suffix := range map[string]string{ + `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`: "or give {b} a drawGroup", + `{"format":"{a}","a":"{/cat} {/cat.p.first}"}`: "name the fields you want instead", + } { + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row}))) + if err == nil || !strings.HasSuffix(err.Error(), suffix) { + t.Errorf("New(%s) = %v, want it to end %q", row, err, suffix) + } } // A sibling path and a reference into the level it holds are the same overlap, // and the reference reaches it from a category row renders. diff --git a/table_test.go b/table_test.go index d7b8be5..38a0e86 100644 --- a/table_test.go +++ b/table_test.go @@ -1120,8 +1120,8 @@ func TestCellReadsMeetWhereTheRenderPairsTheRows(t *testing.T) { "region.json": `{"format":"{name}={note}","rows":"region.tsv","key":"code","name":"name","weight":"population"}`, "x.json": `"{/region}|{/municipality}"`, }) - if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.Contains(err.Error(), "renders its own draw of what") { - t.Fatalf("New = %v, want the two drawn rows held to one draw of addr", err) + if _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, whole)), WithSeed(1)); err == nil || !strings.HasSuffix(err.Error(), "or move {/municipality} into a field with a drawGroup") { + t.Fatalf("New = %v, want the two drawn rows held to one draw of addr, and the drawGroup named on a field around the read", err) } nested := with(whole, map[string]string{ "municipality.tsv": "code\tname\tregion\tpopulation\tnote\n0180\tStockholm\t01\t980000\t-\n0184\tSolna\t01\t85000\t-\n1280\tMalmö\t12\t360000\t{/shop}\n1281\tLund\t12\t130000\t{/addr.city}\n1480\tGöteborg\t14\t590000\t-\n", -- 2.52.0 From d943bdca166f27b534d60e343a583c723a8a8fee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:05:45 +0200 Subject: [PATCH 332/966] Name a drawGroup only on a field that splits the pair, and a field around a reference --- readsurvey.go | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/readsurvey.go b/readsurvey.go index 11ae00f..c819800 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -356,9 +356,13 @@ func alternatives(a, b surveyAt) bool { } func overlapError(route surveyRoute, ref string, into pathRead) error { - apart := fmt.Sprintf("move {%s} into a field with a drawGroup", into.a.spelling) - if into.a.spelling != into.at.route.label { - apart = fmt.Sprintf("give %s a drawGroup", into.at.route.spelling) + apart := "" + switch { + case route.spelling == into.at.route.spelling: + case isRef(into.at.route.label): + apart = fmt.Sprintf(", or move {%s} into a field with a drawGroup", into.at.route.label) + default: + apart = fmt.Sprintf(", or give %s a drawGroup", into.at.route.spelling) } - return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead, or %s", route.spelled(ref), into.at.route.spelled(into.a.spelling), apart) + return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead%s", route.spelled(ref), into.at.route.spelled(into.a.spelling), apart) } -- 2.52.0 From 9d9d72c892fd9abd0fe52d059a75cdd59aff182c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:08:03 +0200 Subject: [PATCH 333/966] File the drawGroup advice for two reads under one field --- todo.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/todo.md b/todo.md index 6d460a3..2d7f7ed 100644 --- a/todo.md +++ b/todo.md @@ -226,6 +226,9 @@ by a `data-import/` script, as README goal 12 asks. README's hold section promises: both answer "name the fields you want instead", and `{n} reaches "p" by a second route` names neither the read the route ends on, `{/row.p.last}`, nor the spelling to write, `{p.last}`. +- Offer `move {Y} into a field with a drawGroup` in `overlapError` where both reads share + one field: `"a":"{/cat} {/cat.p.first}"` ends at "name the fields you want instead", + while the same pair in a format offers the move, which loads there too. - Call `templateError` a compile failure: its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. - Name the node a selector follows in `stepInto`'s refusal: `sv_SE.person[1].first` -- 2.52.0 From c2ac7bf84399a2a398e244ccbd8377a77968c04e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:08:59 +0200 Subject: [PATCH 334/966] File the missing spellings and the linked-tables paragraph split --- todo.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 2d7f7ed..29aff58 100644 --- a/todo.md +++ b/todo.md @@ -223,9 +223,12 @@ by a `data-import/` script, as README goal 12 asks. values, as the README's Library section promises: `datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. - Name the spelling to use in `checkNameHeld`'s and `checkNoOverlap`'s refusals, as the - README's hold section promises: both answer "name the fields you want instead", and - `{n} reaches "p" by a second route` names neither the read the route ends on, - `{/row.p.last}`, nor the spelling to write, `{p.last}`. + README's hold section promises: both answer "name the fields you want instead", or + "reach it one way" for an operand, and `{n} reaches "p" by a second route` names + neither the read the route ends on, `{/row.p.last}`, nor the spelling to write, `{p.last}`. +- Split the README's linked-tables paragraph opening "A path descends from a row", 23 + lines holding five rules: move its load errors that are no overlap to the table + section, and drop its bare `{/city}` refusal, which the hold section states. - Offer `move {Y} into a field with a drawGroup` in `overlapError` where both reads share one field: `"a":"{/cat} {/cat.p.first}"` ends at "name the fields you want instead", while the same pair in a format offers the move, which loads there too. -- 2.52.0 From ba43eb1c914272e2a8f2a39a79f559fa2e1621e6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:27:21 +0200 Subject: [PATCH 335/966] Compile a format in format.go --- format.go | 10 ++++++++++ node.go | 10 ---------- todo.md | 3 --- 3 files changed, 10 insertions(+), 13 deletions(-) diff --git a/format.go b/format.go index 369917d..3a93418 100644 --- a/format.go +++ b/format.go @@ -434,6 +434,16 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { c.ops = append(c.ops, op{formatToken: tok, arms: arms}) } +// compileFormat compiles a format into ops, and applies the fences that need the +// compiled reads. +func compileFormat(toks []formatToken, refs map[string]refBinding) (formatOps, error) { + c := compileOps(toks, refs) + if err := checkNoOverlap(c); err != nil { + return c, err + } + return c, checkNoRepeatedRead(c) +} + // compileOps compiles a parsed format. Call checkTokens first: it is what proves // every token valid. func compileOps(toks []formatToken, refs map[string]refBinding) formatOps { diff --git a/node.go b/node.go index ccf9262..bf158b9 100644 --- a/node.go +++ b/node.go @@ -227,16 +227,6 @@ func (t *template) fixedText() (string, bool) { return "", false } -// compileFormat compiles a format into ops, and applies the fences that need the -// compiled reads. -func compileFormat(toks []formatToken, refs map[string]refBinding) (formatOps, error) { - c := compileOps(toks, refs) - if err := checkNoOverlap(c); err != nil { - return c, err - } - return c, checkNoRepeatedRead(c) -} - func compileChoice(items []any, pos position) (node, error) { itemPos := inFormat if pos == inColumn { diff --git a/todo.md b/todo.md index 29aff58..a02fb17 100644 --- a/todo.md +++ b/todo.md @@ -34,9 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Compile a format in `format.go`: `linkTemplate` calls `compileFormat` in `node.go`, - which calls `compileOps` and `holdfence.go`'s two checks, and both architects placed - the step wrong. - Delete the "so which … is reported does not vary" clause at its five sites past `checkNodeFences`, and `checkRecordDraws`' "A record-only template's format renders nothing", which runs over templates with a format too. -- 2.52.0 From 789c47d0ee49e01f2aefd81a1463f4d68f57a503 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:29:01 +0200 Subject: [PATCH 336/966] Name a real method in declaredFuncs' example --- doc_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc_test.go b/doc_test.go index ce32fff..9dd978c 100644 --- a/doc_test.go +++ b/doc_test.go @@ -109,7 +109,7 @@ func declaredSymbols(files []*ast.File) map[string]bool { } // declaredFuncs is every function the package declares, a method keyed under its -// type: template.compileFormat. +// type: template.fixedText. func declaredFuncs(files []*ast.File) map[string]bool { names := map[string]bool{} for _, f := range files { -- 2.52.0 From 9b050994f5352d06d9d81828f89c9e1c3ba77c20 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:43:46 +0200 Subject: [PATCH 337/966] Delete the reported-order clauses and checkRecordDraws' record-only comment --- drawfence.go | 1 - graph.go | 3 +-- holdfence.go | 6 ++---- node.go | 3 +-- readsurvey.go | 5 ++--- todo.md | 3 --- 6 files changed, 6 insertions(+), 15 deletions(-) diff --git a/drawfence.go b/drawfence.go index 4d34014..34acf2e 100644 --- a/drawfence.go +++ b/drawfence.go @@ -46,7 +46,6 @@ func (f *drawFence) checkRecordDraws(path string, n node) error { if !ok || !t.isRecord { return nil } - // A record-only template's format renders nothing, so weigh the columns, not the format. columns := sortedNames(t.fields) reads := false for _, name := range columns { diff --git a/graph.go b/graph.go index d99fb26..868f75a 100644 --- a/graph.go +++ b/graph.go @@ -6,8 +6,7 @@ import ( ) // walkNodes calls fn once per contained node, passing the dot path that reaches it, -// visiting keys in sorted order so which of several broken nodes gets reported does -// not depend on map iteration. +// visiting keys in sorted order. func walkNodes(root map[string]node, fn func(path string, n node) error) error { for _, name := range sortedNames(root) { if err := eachNode(root[name], name, fn); err != nil { diff --git a/holdfence.go b/holdfence.go index 3248072..6f5bfd0 100644 --- a/holdfence.go +++ b/holdfence.go @@ -28,8 +28,7 @@ func heldCheck(path string, n node) error { } // heldNames lists a template's held names, operands' first, then paths', each -// in name order: a level read both ways is reported by the operand's fence, and -// which overlap is reported does not vary. +// in name order: a level read both ways is reported by the operand's fence. func heldNames(t *template) []string { names := make([]string, 0, len(t.compiled.held)) for name := range t.compiled.held { @@ -172,8 +171,7 @@ func renders(n node, want, seen map[node]bool) bool { // checkNoOverlap rejects a format that both renders a sibling level and reads a path // into it — {p} beside {p.first}, {p.addr} beside {p.addr.city}. The path reads the // level's held draw while rendering the level expands it afresh, so their values would -// disagree. Reads are compared in sorted order, so which pair is reported does not -// depend on where the tokens sit. +// disagree. // docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held func checkNoOverlap(c formatOps) error { names := pathHeadReaders(c) diff --git a/node.go b/node.go index bf158b9..69044e0 100644 --- a/node.go +++ b/node.go @@ -371,8 +371,7 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) { return o, nil } -// compileFields compiles every non-option key of a template object, in name order -// so which of several bad fields is reported does not vary. +// compileFields compiles every non-option key of a template object, in name order. func compileFields(m map[string]any, pos position) (map[string]node, error) { fields := make(map[string]node, len(m)) keys := make([]string, 0, len(m)) diff --git a/readsurvey.go b/readsurvey.go index c819800..5844166 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -214,8 +214,7 @@ func (r *tableRead) replay(d *pinSet) error { // check refuses what one draw per reference path cannot answer for: a read selecting a row its route // pinned another of, a read of a level beside a path another read takes into it, and two reads of one -// table family that select different rows. Reads are compared in path order, so which pair is -// reported does not vary. +// table family that select different rows. func (s *readSurvey) check() error { if s.clash != nil { return s.clash @@ -242,7 +241,7 @@ func (s *readSurvey) check() error { // checkFamilies refuses reads of one table family in one draw group that cannot // read one consistent draw: a table rendered whole beside a path into the family, // a table one read draws that another pins, and two reads pinning different rows. -// The reads come sorted by group and path, so which pair is reported does not vary. +// The reads come sorted by group and path. func checkFamilies(reads []pathRead) error { for i, r := range reads { if r.tr == nil { diff --git a/todo.md b/todo.md index a02fb17..1d4808a 100644 --- a/todo.md +++ b/todo.md @@ -34,9 +34,6 @@ ranking `readSurvey.walk` and `edge` hardest and the unit they would least modif This round, in order: -- Delete the "so which … is reported does not vary" clause at its five sites past - `checkNodeFences`, and `checkRecordDraws`' "A record-only template's format renders - nothing", which runs over templates with a format too. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 6ee7ee42348e35e1fdf127b2473e929b882a3764 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:44:50 +0200 Subject: [PATCH 338/966] Delete the sort-order restatements on walkNodes and compileFields --- graph.go | 3 +-- node.go | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/graph.go b/graph.go index 868f75a..4992dc9 100644 --- a/graph.go +++ b/graph.go @@ -5,8 +5,7 @@ import ( "sort" ) -// walkNodes calls fn once per contained node, passing the dot path that reaches it, -// visiting keys in sorted order. +// walkNodes calls fn once per contained node, passing the dot path that reaches it. func walkNodes(root map[string]node, fn func(path string, n node) error) error { for _, name := range sortedNames(root) { if err := eachNode(root[name], name, fn); err != nil { diff --git a/node.go b/node.go index 69044e0..0725b7d 100644 --- a/node.go +++ b/node.go @@ -371,7 +371,7 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) { return o, nil } -// compileFields compiles every non-option key of a template object, in name order. +// compileFields compiles every non-option key of a template object. func compileFields(m map[string]any, pos position) (map[string]node, error) { fields := make(map[string]node, len(m)) keys := make([]string, 0, len(m)) -- 2.52.0 From dd2ba5696e0ae103e212454c6d2d176f170b3c8d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 15:57:13 +0200 Subject: [PATCH 339/966] File the fifth 2026-09-29 nine-seat panel's findings, scored 5.8 --- todo.md | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 5 deletions(-) diff --git a/todo.md b/todo.md index 1d4808a..9264ae1 100644 --- a/todo.md +++ b/todo.md @@ -30,10 +30,54 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic `readSurvey` walk or check hardest and the unit they would least modify; 5.8 on 2026-09-29 a fourth time (Navigation 6.7, Locality 5.0, Shape 5.7, Self-sufficiency 5.7), every seat's lowest dimension Locality, alone or tied, all seven unit seats -ranking `readSurvey.walk` and `edge` hardest and the unit they would least modify. +ranking `readSurvey.walk` and `edge` hardest and the unit they would least modify; 5.8 +on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-sufficiency +5.8), every seat's lowest dimension Locality, alone or tied, all seven unit seats ranking +`readSurvey.walk` hardest and the unit they would least modify. This round, in order: +- Put the draw fence in one file: `drawFence.checkDraws` in `drawfence.go` hands + `surveyRender(t).check()` the verdict, whose refusals live in `readsurvey.go`, and + eight seats grepped one fence error across `holdfence.go`, `drawfence.go`, + `readsurvey.go` and `pins.go`. +- Split `formatOps.held` into the names an expansion holds and the reference paths a + render's draws hold, deleting `keepsHold` and `heldCheck`'s skip of a reference path: + one map carries both, told apart by `isRef` and `firstReach.path == ""`, and six seats + hunted for which fence owns which. +- Give `surveyAt`'s whole read one field, its table and pins together, rename + `alternatives` for what it answers, two reads that never render together, and say at + `check` why a level landing on a row may sit beside a path into it: five seats held + `surveyAt`'s five fields at once, two misread `alternatives`, and two found no reason + for the `landsRow` exemption. +- Switch `disagreement` on the fixed item's kind, with the other item inside each arm, + and rename its `proven` local, which shadows the `proven` type: seven seats read the + seven-arm switch as a truth table without the table. +- Say at `checkNodeFences` that its walks recurse unguarded, so a caller runs + `checkNoCycles` first, and compute `refs` apart from `checkDrawGroup`'s pass: five + seats reconstructed the order, and the inline binding runs no cycle check. +- Give `tableSite` an `isFormat` beside `isCell`, so `renderRecord` and `traceRead` stop + comparing `row` with `formatRow` alone, which holds only because a template outside a + table has row 0: four seats found the invariant unstated. +- Move each unit to its owner's file, `valueProof.checkField` to `valueproof.go` and + `coverPath` and `pathLeaves` beside their walks in `path.go`, and rename `checkPath`, + which differs from the `pathCheck` type it wraps by word order alone: both architects + placed them wrong. +- Correct the claims readers followed wrong: `IsTemplate`'s doc says an argument holding + a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; `arm`'s + says a tail's head is drawn once per expansion, where a reference path is held per + render; the README says rows are indexed on the first draw, where `New` builds the key + map; and `loadDir`'s says it builds the tree deterministically, where it reports an + unnamed rows file in map order. +- Delete the comments the panel found restating their code or arranging the file: + `node`'s second sentence, `op`'s "compile builds these so render never re-scans the + format", `checkCalc`'s last sentence, `calcVar.eval`'s four lines, `checkNoCycles`' + grey and black, and the docs on `expand`, `readRows`, `splitOutside`, `reservedList` + and `writesRecords`. +- Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and + a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`, + `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by file. + Say "fences" in the `pin` entry then, which names the draw fences too. - Run the nine-seat panel again and file what it names here. ### Data @@ -233,6 +277,11 @@ by a `data-import/` script, as README goal 12 asks. - Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the first in Go's map order. +- Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that + is not there: `loadDir` indexes hidden TSVs and skips them in two places, so one loads + where its comment says a hidden file is never data. +- Print `--help`'s repeat range from `fejkdata.MaxRepeat`: the usage spells + `1..1048576` a second time. - Report a CRLF rows file holding only its header as having no rows: `parseRows` answers "line 2 is empty". - Reword `prove`'s refusal of a typed column reading a row, `{/region}`: say the @@ -327,10 +376,6 @@ because pairing a street with its exact postnummer rewrites shipped rows. - Name goal 2's four criteria with the axes the panel scores, so a recorded score maps back to the clause it came from: the goal spells them out in prose while `todo.md` records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. -- Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and - a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`, - `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by file. - Say "fences" in the `pin` entry then, which names the draw fences too. - Cut the README's Layout block to the lines that say what a file name cannot: `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. - Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word -- 2.52.0 From ff54251e9b789b0c9996b7a507a26d2aba37f958 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:10:16 +0200 Subject: [PATCH 340/966] Put the draw fence and its refusals in drawfence.go --- README.md | 4 +- drawfence.go | 160 +++++++++++++++++++++++++++++++++++++++++++++- readsurvey.go | 171 +++----------------------------------------------- todo.md | 4 -- 4 files changed, 171 insertions(+), 168 deletions(-) diff --git a/README.md b/README.md index dac4db8..a2e1888 100644 --- a/README.md +++ b/README.md @@ -885,8 +885,8 @@ inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplat format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format draws.go the memo a hold or draw group keeps its draws in; and a render's draws and the scope it reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways -drawfence.go the load fences over reference draws: a drawGroup's, and a render's and a record's columns through the survey -readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family, and its verdict on them; and the fence on a table reaching its own family +drawfence.go the load fences over reference draws and their refusals: a drawGroup's, a render's and a record's columns judged on the survey, and a table reaching its own family +readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/drawfence.go b/drawfence.go index 34acf2e..a6bbee4 100644 --- a/drawfence.go +++ b/drawfence.go @@ -1,6 +1,10 @@ package fejkdata -import "fmt" +import ( + "fmt" + "sort" + "strings" +) // drawFence fences each template of a scope as a render of its own, remembering which nodes read a // reference path. @@ -60,6 +64,42 @@ func (f *drawFence) checkRecordDraws(path string, n node) error { return nil } +// checkOwnFamily refuses a table's format or cell that reads, however many templates +// away and through a repeat or a draw group too, a table of its own family: a whole +// read draws its row without pinning it, so the family would draw apart from the row +// it renders, whichever draws the reaching template holds. +// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route +func checkOwnFamily(path string, n node) error { + t, isTemplate := n.(*template) + if !isTemplate || t.site.table == nil { + return nil + } + own := t.site.table + seen := map[node]bool{} + var find func(n node) (renderEdge, *table, bool) + find = func(n node) (renderEdge, *table, bool) { + if seen[n] { + return renderEdge{}, nil, false + } + seen[n] = true + for _, e := range renderEdges(n) { + if e.readsRef() { + if familyTable, isTable := n.(*template).head(e.read.head).(*table); isTable && familyTable.familyRoot() == own.familyRoot() { + return e, familyTable, true + } + } + if e, familyTable, found := find(e.to); found { + return e, familyTable, true + } + } + return renderEdge{}, nil, false + } + if e, familyTable, found := find(t); found { + return fmt.Errorf("%s: %s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", path, e.reached(), familyTable.segment, own.segment) + } + return nil +} + // readsPath reports whether rendering n reads a reference path, short of a repeat, which renders // over draws of its own. func (f *drawFence) readsPath(n node) bool { return f.hasRead(n, false) } @@ -117,3 +157,121 @@ func readsTable(n node, a arm) bool { _, isTable := t.head(a.head).(*table) return isTable } + +// check refuses what one draw per reference path cannot answer for: a read selecting a row its route +// pinned another of, a read of a level beside a path another read takes into it, and two reads of one +// table family that select different rows. +func (s *readSurvey) check() error { + if c := s.clash; c != nil { + return fmt.Errorf("%s: %w; read the family from a template beside it, or add the value as a column", c.route.spelled(c.spelling), c.err) + } + sort.SliceStable(s.reads, func(i, j int) bool { + if s.reads[i].at.group != s.reads[j].at.group { + return s.reads[i].at.group < s.reads[j].at.group + } + return s.reads[i].a.path < s.reads[j].a.path + }) + for i, level := range s.reads { + for _, into := range s.reads[i+1:] { + if into.at.group != level.at.group || alternatives(level.at, into.at) { + continue + } + if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { + return overlapError(level.at.route, level.a.spelling, into) + } + } + } + return checkFamilies(s.reads) +} + +// checkFamilies refuses reads of one table family in one draw group that cannot +// read one consistent draw: a table rendered whole beside a path into the family, +// a table one read draws that another pins, and two reads pinning different rows. +// The reads come sorted by group and path. +func checkFamilies(reads []pathRead) error { + for i, r := range reads { + if r.tr == nil { + continue + } + for _, o := range reads[:i] { + if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { + continue + } + if err := checkFamilyPair(o, r); err != nil { + return err + } + } + } + return replayPairs(reads) +} + +// checkFamilyPair refuses a table read whole beside a path into its family, and a +// table one read draws that the other pins, since which token renders first would +// then decide the row. +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held +func checkFamilyPair(a, b pathRead) error { + for _, pair := range [][2]pathRead{{a, b}, {b, a}} { + x, y := pair[0], pair[1] + if len(x.a.tail) == 0 && len(y.a.tail) > 0 { + return overlapError(x.at.route, x.a.spelling, y) + } + if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { + if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { + tail := x.a.tail + if s.t != x.tr.headTable { + tail = append([]string{x.tr.headTable.segment}, tail...) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", + x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", + x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling)) + } + } + return nil +} + +// replayPairs replays, of every two reads that can render together, the later into the +// earlier's pins. Pairs find every conflict a full replay would: clash judges a row against one +// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry +// its rows' ancestors, so the pair of those two reads clashes the same way. +func replayPairs(reads []pathRead) error { + for i, r := range reads { + if r.tr == nil { + continue + } + for _, o := range reads[i+1:] { + if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { + continue + } + d := r.tr.pins.clone() + if err := o.tr.replay(&d); err != nil { + return conflict(o, err) + } + } + } + return nil +} + +func conflict(r pathRead, err error) error { + return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) +} + +// alternatives reports whether two reads never render together: one read renders one row of a +// table, so reads under different rows the walks pinned, or under different rows of one whole +// draw, never meet. +func alternatives(a, b surveyAt) bool { + return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) +} + +func overlapError(route surveyRoute, ref string, into pathRead) error { + apart := "" + switch { + case route.spelling == into.at.route.spelling: + case isRef(into.at.route.label): + apart = fmt.Sprintf(", or move {%s} into a field with a drawGroup", into.at.route.label) + default: + apart = fmt.Sprintf(", or give %s a drawGroup", into.at.route.spelling) + } + return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead%s", route.spelled(ref), into.at.route.spelled(into.a.spelling), apart) +} diff --git a/readsurvey.go b/readsurvey.go index 5844166..7cca85c 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -1,17 +1,20 @@ package fejkdata -import ( - "fmt" - "sort" - "strings" -) +import "fmt" // readSurvey gathers, at load, the references one render reads, by draw group, for check to compare. type readSurvey struct { reads []pathRead read map[readKey]bool seen map[nodeVisit]bool - clash error + clash *pinClash +} + +// pinClash is the first read whose selectors clash with a row its route pinned. +type pinClash struct { + route surveyRoute + spelling string + err error } // surveyAt is where a survey stands: the draw group its reads draw in, how the render's root reached it, the @@ -126,7 +129,7 @@ func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { pins := at.pins.clone() if err := tr.replay(&pins); err != nil { if s.clash == nil { - s.clash = fmt.Errorf("%s: %w; read the family from a template beside it, or add the value as a column", at.route.spelled(a.spelling), err) + s.clash = &pinClash{at.route, a.spelling, err} } return } @@ -211,157 +214,3 @@ func (r *tableRead) replay(d *pinSet) error { }) return err } - -// check refuses what one draw per reference path cannot answer for: a read selecting a row its route -// pinned another of, a read of a level beside a path another read takes into it, and two reads of one -// table family that select different rows. -func (s *readSurvey) check() error { - if s.clash != nil { - return s.clash - } - sort.SliceStable(s.reads, func(i, j int) bool { - if s.reads[i].at.group != s.reads[j].at.group { - return s.reads[i].at.group < s.reads[j].at.group - } - return s.reads[i].a.path < s.reads[j].a.path - }) - for i, level := range s.reads { - for _, into := range s.reads[i+1:] { - if into.at.group != level.at.group || alternatives(level.at, into.at) { - continue - } - if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { - return overlapError(level.at.route, level.a.spelling, into) - } - } - } - return checkFamilies(s.reads) -} - -// checkFamilies refuses reads of one table family in one draw group that cannot -// read one consistent draw: a table rendered whole beside a path into the family, -// a table one read draws that another pins, and two reads pinning different rows. -// The reads come sorted by group and path. -func checkFamilies(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[:i] { - if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { - continue - } - if err := checkFamilyPair(o, r); err != nil { - return err - } - } - } - return replayPairs(reads) -} - -// checkOwnFamily refuses a table's format or cell that reads, however many templates -// away and through a repeat or a draw group too, a table of its own family: a whole -// read draws its row without pinning it, so the family would draw apart from the row -// it renders, whichever draws the reaching template holds. -// docs/decisions.md#a-table-never-reaches-its-own-family-by-any-route -func checkOwnFamily(path string, n node) error { - t, isTemplate := n.(*template) - if !isTemplate || t.site.table == nil { - return nil - } - own := t.site.table - seen := map[node]bool{} - var find func(n node) (renderEdge, *table, bool) - find = func(n node) (renderEdge, *table, bool) { - if seen[n] { - return renderEdge{}, nil, false - } - seen[n] = true - for _, e := range renderEdges(n) { - if e.readsRef() { - if familyTable, isTable := n.(*template).head(e.read.head).(*table); isTable && familyTable.familyRoot() == own.familyRoot() { - return e, familyTable, true - } - } - if e, familyTable, found := find(e.to); found { - return e, familyTable, true - } - } - return renderEdge{}, nil, false - } - if e, familyTable, found := find(t); found { - return fmt.Errorf("%s: %s reads %s, a table of its own family, which a bare read of %s would draw apart from the row it renders; read the family from a template beside it, or add the value as a column", path, e.reached(), familyTable.segment, own.segment) - } - return nil -} - -// checkFamilyPair refuses a table read whole beside a path into its family, and a -// table one read draws that the other pins, since which token renders first would -// then decide the row. -// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkFamilyPair(a, b pathRead) error { - for _, pair := range [][2]pathRead{{a, b}, {b, a}} { - x, y := pair[0], pair[1] - if len(x.a.tail) == 0 && len(y.a.tail) > 0 { - return overlapError(x.at.route, x.a.spelling, y) - } - if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { - if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { - tail := x.a.tail - if s.t != x.tr.headTable { - tail = append([]string{x.tr.headTable.segment}, tail...) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling)) - } - } - return nil -} - -// replayPairs replays, of every two reads that can render together, the later into the -// earlier's pins. Pairs find every conflict a full replay would: clash judges a row against one -// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry -// its rows' ancestors, so the pair of those two reads clashes the same way. -func replayPairs(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[i+1:] { - if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { - continue - } - d := r.tr.pins.clone() - if err := o.tr.replay(&d); err != nil { - return conflict(o, err) - } - } - } - return nil -} - -func conflict(r pathRead, err error) error { - return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) -} - -// alternatives reports whether two reads never render together: one read renders one row of a -// table, so reads under different rows the walks pinned, or under different rows of one whole -// draw, never meet. -func alternatives(a, b surveyAt) bool { - return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) -} - -func overlapError(route surveyRoute, ref string, into pathRead) error { - apart := "" - switch { - case route.spelling == into.at.route.spelling: - case isRef(into.at.route.label): - apart = fmt.Sprintf(", or move {%s} into a field with a drawGroup", into.at.route.label) - default: - apart = fmt.Sprintf(", or give %s a drawGroup", into.at.route.spelling) - } - return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead%s", route.spelled(ref), into.at.route.spelled(into.a.spelling), apart) -} diff --git a/todo.md b/todo.md index 9264ae1..1236cb0 100644 --- a/todo.md +++ b/todo.md @@ -37,10 +37,6 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Put the draw fence in one file: `drawFence.checkDraws` in `drawfence.go` hands - `surveyRender(t).check()` the verdict, whose refusals live in `readsurvey.go`, and - eight seats grepped one fence error across `holdfence.go`, `drawfence.go`, - `readsurvey.go` and `pins.go`. - Split `formatOps.held` into the names an expansion holds and the reference paths a render's draws hold, deleting `keepsHold` and `heldCheck`'s skip of a reference path: one map carries both, told apart by `isRef` and `firstReach.path == ""`, and six seats -- 2.52.0 From 14952d0c0685c04149464cdbe6cd8f96f8d8ba37 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:17:08 +0200 Subject: [PATCH 341/966] Test that a repeated bare reference beside its path names the draw fence's refusal --- inline_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/inline_test.go b/inline_test.go index fb97109..c908c50 100644 --- a/inline_test.go +++ b/inline_test.go @@ -110,6 +110,7 @@ func TestFakeTemplateErrors(t *testing.T) { {`{"format":"x"}`, "is a string"}, {`{/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads a path through; name the fields you want instead, or move {/misc.territory.alpha2} into a field with a drawGroup"}, {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "renders its own draw of what"}, + {`{/misc.territory} {/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads"}, {`{"format":"{/misc.territory.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) -- 2.52.0 From dee1e186e3d6cb97be70066a5121d063fbed298b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:18:13 +0200 Subject: [PATCH 342/966] Hold only an expansion's names in formatOps.held, deleting keepsHold --- doc.go | 3 ++- format.go | 16 ++++++++-------- holdfence.go | 7 ++----- render.go | 2 +- todo.md | 4 ---- 5 files changed, 13 insertions(+), 19 deletions(-) diff --git a/doc.go b/doc.go index 949e257..8d1b118 100644 --- a/doc.go +++ b/doc.go @@ -25,7 +25,8 @@ package fejkdata // iteration of a `template.repeat`. // - hold — keeping one draw of a name, so every route to it reads that value: // an expansion holds a sibling path's head and an operand's field, its draw -// group a reference path's: `formatOps.held`, `formatOps.keepsHold`, `heldCheck`. +// group a reference path's: `formatOps.held` and `heldCheck` an expansion's, +// `readReference` a draw group's. // - memo — what a hold or a draw group keeps its draws in, the variant drawn at // each level and the value each path read: `drawMemo`, `readMemo`. // - draw group — reference draws kept apart inside one render: diff --git a/format.go b/format.go index 3a93418..7f64949 100644 --- a/format.go +++ b/format.go @@ -381,12 +381,12 @@ type op struct { } // formatOps is a compiled format: its ops, the size of its literal text, and the -// names it holds: every head a path starts from, plus the fields an operand reads. +// names its expansion holds: every head a path that is not a reference starts from, +// plus the fields an operand reads. type formatOps struct { - ops []op - grow int - held map[string]firstReach - keepsHold bool // some held name is not a reference path, so expand makes a hold + ops []op + grow int + held map[string]firstReach } // firstReach is how a format first reaches a held name: the reader holding it, for @@ -397,12 +397,12 @@ type firstReach struct { } func (c *formatOps) holdName(a arm, label string) { + if isRef(a.head) && len(a.tail) > 0 { + return + } if c.held == nil { c.held = map[string]firstReach{} } - if !isRef(a.head) || len(a.tail) == 0 { - c.keepsHold = true - } h, seen := c.held[a.head] if !seen { h.holder = label diff --git a/holdfence.go b/holdfence.go index 6f5bfd0..9b03056 100644 --- a/holdfence.go +++ b/holdfence.go @@ -17,9 +17,6 @@ func heldCheck(path string, n node) error { } readers := pathHeadReaders(t.compiled) for _, name := range heldNames(t) { - if t.compiled.held[name].path != "" && isRef(name) { - continue // held for the render: drawFence compares every read of it, by path, across the render and its groups - } if err := checkNameHeld(t, name, readers); err != nil { return fmt.Errorf("%s: %w", path, err) } @@ -201,12 +198,12 @@ func pathHeadReaders(c formatOps) []reader { var names []reader for _, o := range c.ops { for _, a := range o.operands { - if c.held[a.head].path != "" && !isRef(a.head) { + if c.held[a.head].path != "" { names = append(names, reader{a, fmt.Sprintf("%s operand %q", o.fn, a.spelling)}) } } for _, a := range o.arms { - if c.held[a.head].path != "" && !isRef(a.head) { + if c.held[a.head].path != "" { names = append(names, reader{a, "token {" + a.spelling + "}"}) } } diff --git a/render.go b/render.go index e6f83f7..08da588 100644 --- a/render.go +++ b/render.go @@ -121,7 +121,7 @@ func expand(s *generatorState, t *template, sc renderScope) string { // repeat iteration get their own, since each is its own expansion. A reference // path reads its draw group's memo in sc instead. var hold *drawMemo - if t.compiled.keepsHold { + if len(t.compiled.held) > 0 { hold = &drawMemo{ variant: make(map[string]node, len(t.compiled.held)), value: make(map[string]readValue, len(t.compiled.held)), diff --git a/todo.md b/todo.md index 1236cb0..457c85e 100644 --- a/todo.md +++ b/todo.md @@ -37,10 +37,6 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Split `formatOps.held` into the names an expansion holds and the reference paths a - render's draws hold, deleting `keepsHold` and `heldCheck`'s skip of a reference path: - one map carries both, told apart by `isRef` and `firstReach.path == ""`, and six seats - hunted for which fence owns which. - Give `surveyAt`'s whole read one field, its table and pins together, rename `alternatives` for what it answers, two reads that never render together, and say at `check` why a level landing on a row may sit beside a path into it: five seats held -- 2.52.0 From acbc6943b8f3820fb2c6e00a0c9dc767f9c8f96a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:24:10 +0200 Subject: [PATCH 343/966] Spell a reference path as arm.isRefPath, and say where each tail's head is held --- format.go | 10 +++++++--- render.go | 2 +- todo.md | 4 +--- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/format.go b/format.go index 7f64949..abc146d 100644 --- a/format.go +++ b/format.go @@ -303,8 +303,8 @@ func checkNoRepeatedArm(body string, names []string) error { // arm is one alternative of a {a|b} token or one operand, split into the head // `template.head` resolves and the tail of a dotted path into it. A non-empty tail -// is what makes the arm a held draw: its head is drawn once per expansion (see -// compileOps). +// makes the arm a held draw: a reference path's for the render, any other for the +// expansion. type arm struct { spelling string // as written, for messages head string @@ -314,6 +314,10 @@ type arm struct { path string // head and tail, the one path every way of writing this read shares } +func (a arm) isRefPath() bool { + return isRef(a.head) && len(a.tail) > 0 +} + // splitArm splits one name into head and tail. refs maps a reference to what // linkRefs resolved it to; before linking, a reference is whole. func splitArm(name string, refs map[string]refBinding) arm { @@ -397,7 +401,7 @@ type firstReach struct { } func (c *formatOps) holdName(a arm, label string) { - if isRef(a.head) && len(a.tail) > 0 { + if a.isRefPath() { return } if c.held == nil { diff --git a/render.go b/render.go index 08da588..b5c5da9 100644 --- a/render.go +++ b/render.go @@ -156,7 +156,7 @@ func expand(s *generatorState, t *template, sc renderScope) string { // read one row, either read twice gives one value, and a shown operand is the operand // computed. Every other name is drawn afresh, so {word} {word} still draws twice. func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue { - if isRef(a.head) && len(a.tail) > 0 { + if a.isRefPath() { return readReference(s, t, sc, a) } if sc.draws.trace != nil { diff --git a/todo.md b/todo.md index 457c85e..1b4f966 100644 --- a/todo.md +++ b/todo.md @@ -56,9 +56,7 @@ This round, in order: which differs from the `pathCheck` type it wraps by word order alone: both architects placed them wrong. - Correct the claims readers followed wrong: `IsTemplate`'s doc says an argument holding - a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; `arm`'s - says a tail's head is drawn once per expansion, where a reference path is held per - render; the README says rows are indexed on the first draw, where `New` builds the key + a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; the README says rows are indexed on the first draw, where `New` builds the key map; and `loadDir`'s says it builds the tree deterministically, where it reports an unnamed rows file in map order. - Delete the comments the panel found restating their code or arranging the file: -- 2.52.0 From e84522f244b3f5825bfda0897654484049623852 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:27:44 +0200 Subject: [PATCH 344/966] Scope the held-reference decision to a draw group, and give both hold decisions their goal --- docs/decisions.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 4dcd1b5..ecdcc42 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -83,12 +83,12 @@ still lives in `data/` as JSON; `--data-path` layers over it. ## A bare reference draws each time; a reference path is held -`{/p} {/p}` is two draws, as `{word} {word}` is, while every `{/p.first}` in one render -reads one draw, and a bare `{/p}` beside them is a load error: a bare token is by -contract an independent draw, a path pins its level, and a fresh draw of a pinned level -could show another row. A builtin's operand holds what it reads for its expansion, -references included, so `{uppercase(/p)} {/p}` is one draw — the rule every operand -follows. +Goal 4: `{/p} {/p}` is two draws, as `{word} {word}` is, while every `{/p.first}` in +one render and draw group reads one draw, and a bare `{/p}` beside them is a load error, +as `{p}` beside `{p.first}` is: a bare token is by contract an independent draw, a path +pins its level, and a fresh draw of a pinned level could show another row. A builtin's +operand holds what it reads for its expansion, references included, so +`{uppercase(/p)} {/p}` is one draw, and `{/p}` written twice beside it is a load error. ## Reference sigils follow the filesystem @@ -236,11 +236,11 @@ across categories must agree. ## The expansion hold and the render's draws are two fences -One proves a sibling path or an operand is reached only by its readers within an -expansion, the other does the same for reference paths across a render and its draw -groups. They pin different things — an operand pins the value its own render produced -and stops at a reference, a path pins every level it passes through — so one walk would -carry both rules and both scopes anyway, and tell them apart at every step. +Goal 2: one proves a sibling path or an operand is reached only by its readers within +an expansion, the other that reference paths, whole-table reads and a table family's +rows agree across a render and its draw groups. Their scopes differ and only the second +tracks rows and pins, so one walk would carry both scopes and tell them apart at every +step. ## A render's draws make their maps on the first read -- 2.52.0 From 401141bb66ee59bfd4d29fae31f6b38ae4a6330d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:29:54 +0200 Subject: [PATCH 345/966] Test that the operand refusal names the operand as written and the spelling to use --- hold_test.go | 2 +- inline_test.go | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/hold_test.go b/hold_test.go index 8f375b5..8efadfe 100644 --- a/hold_test.go +++ b/hold_test.go @@ -232,7 +232,7 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { } for name, c := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, c.files))) - if err == nil || !strings.Contains(err.Error(), "a {calc()} also reads") { + if err == nil || !strings.Contains(err.Error(), "holds to one draw for this expansion; drop ") { t.Errorf("%s: New = %v, want the second route to the operand rejected", name, err) continue } diff --git a/inline_test.go b/inline_test.go index c908c50..c46920c 100644 --- a/inline_test.go +++ b/inline_test.go @@ -110,7 +110,8 @@ func TestFakeTemplateErrors(t *testing.T) { {`{"format":"x"}`, "is a string"}, {`{/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads a path through; name the fields you want instead, or move {/misc.territory.alpha2} into a field with a drawGroup"}, {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "renders its own draw of what"}, - {`{/misc.territory} {/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads"}, + {`{/misc.territory} {/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads a path through; name the fields you want instead, or move {/misc.territory.alpha2} into a field with a drawGroup"}, + {`{"format":"{uppercase(/misc.territory)} {y}","y":"{/misc.territory}"}`, `{y} renders "/misc.territory", which {uppercase(/misc.territory)} holds to one draw for this expansion; drop {y}, or write {/misc.territory} in this format to read that draw`}, {`{"format":"{/misc.territory.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) -- 2.52.0 From 316c69c063dab10b971121b785ecbd8f6b6c5285 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:30:31 +0200 Subject: [PATCH 346/966] Name the operand refusal's call as written, and the spelling that reads its draw --- holdfence.go | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/holdfence.go b/holdfence.go index 9b03056..c895670 100644 --- a/holdfence.go +++ b/holdfence.go @@ -76,21 +76,22 @@ func checkNameHeld(t *template, name string, readers []reader) error { if path != "" { return fmt.Errorf("%s reaches %q by a second route, where {%s} holds it to one draw; name the fields you want instead", e.reached(), name, path) } - return fmt.Errorf("%s renders %q, which a {%s()} also reads; reach it one way so it is drawn once", e.reached(), name, operandReader(t, name)) + call, operand := operandReader(t, name) + return fmt.Errorf("%s renders %q, which {%s} holds to one draw for this expansion; drop %s, or write {%s} in this format to read that draw", e.reached(), name, call, e.reached(), operand) } return nil } -// operandReader names the builtin whose operand holds name. -func operandReader(t *template, name string) string { +// operandReader spells the call whose operand holds name, and the operand, as written. +func operandReader(t *template, name string) (call, operand string) { for _, o := range t.compiled.ops { for _, a := range o.operands { if a.head == name { - return o.fn + return o.body, a.spelling } } } - return "" + return "", "" } // coverPath collects what holding one path pins: the first choice level it passes, -- 2.52.0 From c4911e15bc36ee5ce5aab5d501491595211e95dd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:30:41 +0200 Subject: [PATCH 347/966] Narrow the refusal-spelling item to the path refusals --- todo.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/todo.md b/todo.md index 1b4f966..b8d927f 100644 --- a/todo.md +++ b/todo.md @@ -250,10 +250,9 @@ by a `data-import/` script, as README goal 12 asks. - Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises: `datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. -- Name the spelling to use in `checkNameHeld`'s and `checkNoOverlap`'s refusals, as the - README's hold section promises: both answer "name the fields you want instead", or - "reach it one way" for an operand, and `{n} reaches "p" by a second route` names - neither the read the route ends on, `{/row.p.last}`, nor the spelling to write, `{p.last}`. +- Name the spelling to use in `checkNameHeld`'s path refusal and `checkNoOverlap`'s, as + the README's hold section promises: both answer "name the fields you want instead", + and `{n} reaches "p" by a second route` names neither the read the route ends on, `{/row.p.last}`, nor the spelling to write, `{p.last}`. - Split the README's linked-tables paragraph opening "A path descends from a row", 23 lines holding five rules: move its load errors that are no overlap to the table section, and drop its bare `{/city}` refusal, which the hold section states. -- 2.52.0 From 5880487f47fdfa32c806386ceda88b477801316f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:33:11 +0200 Subject: [PATCH 348/966] Test that the operand refusal names the holding operand, and an operand route's fix --- hold_test.go | 4 ++-- inline_test.go | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/hold_test.go b/hold_test.go index 8efadfe..06e39ab 100644 --- a/hold_test.go +++ b/hold_test.go @@ -197,7 +197,7 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { `"b":"{/hop}"}`, "hop": `"{/cat.a}"`, }, - `calc operand "b" renders "a"`, + `calc operand "b" renders "a", which {calc(a + b, 0)} holds to one draw for this expansion; stop calc operand "b" reaching "a"`, }, // Reaching *into* the operand is the same shape as naming it: the draw the // calc holds settled that value too. The token spelling {net.v} is already @@ -232,7 +232,7 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { } for name, c := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, c.files))) - if err == nil || !strings.Contains(err.Error(), "holds to one draw for this expansion; drop ") { + if err == nil || !strings.Contains(err.Error(), "holds to one draw for this expansion; ") { t.Errorf("%s: New = %v, want the second route to the operand rejected", name, err) continue } diff --git a/inline_test.go b/inline_test.go index c46920c..6bc6e14 100644 --- a/inline_test.go +++ b/inline_test.go @@ -112,6 +112,7 @@ func TestFakeTemplateErrors(t *testing.T) { {`{"format":"{/misc.territory.alpha2} {x}","x":"{/misc.territory}"}`, "renders its own draw of what"}, {`{/misc.territory} {/misc.territory} {/misc.territory.alpha2}`, "{/misc.territory} renders its own draw of what {/misc.territory.alpha2} reads a path through; name the fields you want instead, or move {/misc.territory.alpha2} into a field with a drawGroup"}, {`{"format":"{uppercase(/misc.territory)} {y}","y":"{/misc.territory}"}`, `{y} renders "/misc.territory", which {uppercase(/misc.territory)} holds to one draw for this expansion; drop {y}, or write {/misc.territory} in this format to read that draw`}, + {`{"format":"{uppercase(/misc.territory.alpha2)} {lowercase(/misc.territory)} {y}","y":"{/misc.territory}"}`, "which {lowercase(/misc.territory)} holds to one draw for this expansion; drop {y}, or write {/misc.territory} in"}, {`{"format":"{/misc.territory.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) -- 2.52.0 From fe40e2ca951ea34a874edd04dd222f657ff63426 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:34:05 +0200 Subject: [PATCH 349/966] Name the holding operand in the operand refusal, and an operand route's fix --- holdfence.go | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/holdfence.go b/holdfence.go index c895670..a86e9a3 100644 --- a/holdfence.go +++ b/holdfence.go @@ -77,6 +77,9 @@ func checkNameHeld(t *template, name string, readers []reader) error { return fmt.Errorf("%s reaches %q by a second route, where {%s} holds it to one draw; name the fields you want instead", e.reached(), name, path) } call, operand := operandReader(t, name) + if e.operand != "" { + return fmt.Errorf("%s renders %q, which {%s} holds to one draw for this expansion; stop %s reaching %q", e.reached(), name, call, e.reached(), name) + } return fmt.Errorf("%s renders %q, which {%s} holds to one draw for this expansion; drop %s, or write {%s} in this format to read that draw", e.reached(), name, call, e.reached(), operand) } return nil @@ -86,7 +89,7 @@ func checkNameHeld(t *template, name string, readers []reader) error { func operandReader(t *template, name string) (call, operand string) { for _, o := range t.compiled.ops { for _, a := range o.operands { - if a.head == name { + if a.head == name && !a.isRefPath() { return o.body, a.spelling } } -- 2.52.0 From 672263421ab9f1c51e429dc52c077251d6131b83 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 16:38:20 +0200 Subject: [PATCH 350/966] File the operand route's missing spelling under the refusal-spelling item --- todo.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index b8d927f..6f1dfbd 100644 --- a/todo.md +++ b/todo.md @@ -252,7 +252,10 @@ by a `data-import/` script, as README goal 12 asks. `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. - Name the spelling to use in `checkNameHeld`'s path refusal and `checkNoOverlap`'s, as the README's hold section promises: both answer "name the fields you want instead", - and `{n} reaches "p" by a second route` names neither the read the route ends on, `{/row.p.last}`, nor the spelling to write, `{p.last}`. + and `{n} reaches "p" by a second route` names neither the read the route ends on, + `{/row.p.last}`, nor the spelling to write, `{p.last}`. Where an operand route's value + is the held name, offer it as the operand: `lowercase operand "y" renders + "/misc.territory"` ends at "stop", where `{lowercase(/misc.territory)}` loads. - Split the README's linked-tables paragraph opening "A path descends from a row", 23 lines holding five rules: move its load errors that are no overlap to the table section, and drop its bare `{/city}` refusal, which the hold section states. -- 2.52.0 From dbac4c5ad89f574c571802a6f917994ee8d08857 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 18:41:07 +0200 Subject: [PATCH 351/966] Date every decision from the history that first names it, and credit it to Lilleman --- docs/decisions.md | 172 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) diff --git a/docs/decisions.md b/docs/decisions.md index ecdcc42..4928ea6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -2,6 +2,8 @@ ## GitHub is canonical, and the module path names it +2026-09-20, Lilleman auf Larv. + Goal 1 wants the usage the tool earns and goal 2 expects extenders who did not write it; both need a stranger to file an issue and open a pull request. Gitea has no anonymous issue, and no cross-host pull request at all, so a contributor would need an account and @@ -11,6 +13,8 @@ the workflow decided nothing, being portable already — `github.api_url` and ## The vocabulary sits below `doc.go`'s package clause, not in the package doc +2026-09-21, Lilleman auf Larv. + Goal 1 is what a developer choosing a library reads first, and the package doc is that page: the glossary would fill two thirds of it with the render path's units, mostly unexported, and Go doc comments have no code markup, so every name would render its @@ -20,17 +24,23 @@ teach the exported API belongs above the clause. ## Options and fields share one namespace +2026-09-02, Lilleman auf Larv. + `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a key, or prefixing options, would tax every template to guard against a misspelt option. ## `{a|b}` stays beside nested choices +2026-09-02, Lilleman auf Larv. + `[[…], […]]` picks the same way, but its arms are anonymous; `{female|male}` keeps `person.female` addressable. ## Flags follow getopt_long +2026-09-02, Lilleman auf Larv. + `--name value` and `--name=value` both work; a short flag's value attaches or follows (`-s42`, `-s 42`) and short flags bundle (`-hn 3`), as every shell user expects. A single-dash long flag is rejected naming the double-dash spelling, and `-s=42` is @@ -39,6 +49,8 @@ the value would make `-d=./x` a directory named `=./x`. ## An argument is a template by its shape, not by a flag +2026-09-03, Lilleman auf Larv. + A JSON object, array or string, or a string carrying a `{` token, is an inline template; anything else is a path. A name may not contain a brace, a bracket or a quote, so a path can never collide with any of those spellings, and the leading `[` or `"` is gated on @@ -55,22 +67,30 @@ tags and any other caller read one. ## An inline template skips the cycle fence +2026-09-03, Lilleman auf Larv. + `New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and nothing in the tree can reference it, so no render of it reaches itself. ## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing +2026-09-24, Lilleman auf Larv. + The whole argument is the spelling under test, and `NewTemplate` compiles, links and validates as one step. An unknown *path* stays a runtime error (exit 1): there the argument is well-formed and only the data is absent. ## A padded JSON argument is rejected, not trimmed +2026-09-03, Lilleman auf Larv. + Padding is the one place the two readings disagree — a format string renders it, JSON drops it — so the spelling that renders is named rather than silently chosen. ## `FakeTemplate` and `NewTemplate` both stay +2026-09-03, Lilleman auf Larv. + They reach the same value but not at the same cost: `NewTemplate` pays the compile and validation once and renders many times, `FakeTemplate` is the one-shot call, and `--repeat` is exactly the case that needs the first. The pair is `regexp.MustCompile` @@ -78,11 +98,15 @@ and `regexp.Match`, not two spellings of one result. ## The shipped data is embedded, not discovered +2026-09-02, Lilleman auf Larv. + A directory a machine happens to have would make `--seed 42` machine-dependent. Data still lives in `data/` as JSON; `--data-path` layers over it. ## A bare reference draws each time; a reference path is held +2026-09-02, Lilleman auf Larv. + Goal 4: `{/p} {/p}` is two draws, as `{word} {word}` is, while every `{/p.first}` in one render and draw group reads one draw, and a bare `{/p}` beside them is a load error, as `{p}` beside `{p.first}` is: a bare token is by contract an independent draw, a path @@ -92,12 +116,16 @@ operand holds what it reads for its expansion, references included, so ## Reference sigils follow the filesystem +2026-09-02, Lilleman auf Larv. + `/` is the root, `.` this file's folder, `..` the folder above — what those spellings already mean to anyone who has typed a path. A locale's files reach each other without naming the locale, so a folder renames and copies without editing its references. ## A change to what exists is a major; a minor only adds +2026-09-16, Lilleman auf Larv. + Data files, the CLI and the Go API are the public API, and a consumer must be able to take a minor without an edit — so an added column is a major, since it changes the CSV header and the `INSERT` column list, as is a removed value, which changes what a fixture @@ -110,17 +138,23 @@ release instead. Before `v1.0.0` a minor carries what a major would. ## Seeded output is promised within one version +2026-09-16, Lilleman auf Larv. + Any edit to a category shifts its stream and everything drawn after it, so a promise across versions would freeze every shipped list; a fixture is re-pinned on a bump, as this repo's own are. ## An error is a contract by what it names, not its bytes +2026-09-16, Lilleman auf Larv. + A script branches on the exit code and reads the named path or spelling, so those hold; wording improves in a minor. ## Formats are checked after all data is loaded, so their errors name the category +2026-09-28, Lilleman auf Larv. + 2026-09-28, larv-review; approved 2026-09-28 by lilleman. Goal 2. Loading reads every file first, then connects references across them. A format holding a reference can only be checked once it is connected, so every format is checked then, and all checks run at @@ -130,11 +164,15 @@ its references connected. ## Raising the lowest supported Go is a major +2026-09-16, Lilleman auf Larv. + A consumer building on it breaks, which is the one test every rule above applies; Go's convention of a minor is not followed. ## The format check runs on the latest Go only +2026-09-21, Lilleman auf Larv. + `gofmt`'s output is the toolchain's, not the code's: 1.27 stopped padding a map literal's values out to a lone long key, so no source satisfies both it and 1.22's. A consumer on the lowest supported Go depends on the code compiling and its tests passing @@ -144,6 +182,8 @@ Valid while the lowest supported Go is not the latest. ## The changelog heading is the one spelling of a release; CI cuts the tag +2026-09-16, Lilleman auf Larv. + A tag pushed by hand is served by `go get` at once, so a tag whose commit lacks its heading is burnt, not fixed. The heading on a gate-passed `main` commit is the trigger instead: the tag can land only there, and the GitHub release the same job publishes @@ -151,6 +191,8 @@ keeps one text as its body and is where prebuilt binaries will attach. ## A `--data-path` override rebinds every reference to the category it replaces +2026-09-24, Lilleman auf Larv. + References bind against the merged tree, so once shipped data uses `{.person}`, a consumer's `sv_SE/person.json` is what every shipped reference into `person` reads, and `New` fails on shipped data the consumer never wrote when that file lacks a field those @@ -160,6 +202,8 @@ shipped tree reads. ## The repeat cap bounds renders, not bytes +2026-09-02, Lilleman auf Larv. + A repeat, alone or nested, may ask for at most 1 048 576 renders; how large each render is stays what the data asked for, so `{hex(1048576)}` repeated to the cap is a terabyte, loaded without complaint. A byte estimate would need every builtin to declare a width to @@ -167,11 +211,15 @@ fence a shape no data comes near, and the harm lands on the author who wrote it. ## 64-bit targets only +2026-09-02, Lilleman auf Larv. + The gate builds amd64, and the buffer sizing a render pre-computes (renders × bytes) assumes a 64-bit int; on a 32-bit target it could overflow and panic. ## A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf` +2026-09-24, Lilleman auf Larv. + `1/0` and a fixed `"0"` field are decidable, so they join the never-numeric operand as a load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing @@ -180,6 +228,8 @@ instead and refuses a divisor it cannot keep from zero. ## In data, a default written out and a constant spelled as a sample are load errors +2026-09-24, Lilleman auf Larv. + `weight: 1`, `repeat: 1`, `separator: ""`, `datatype: "string"`, `int(5,5)`, `float(1,1,2)`, `+5` and `05` each spell what a shorter form already spells, so each is rejected naming that form. The CLI's numbers follow the shell instead: `--seed 007` and @@ -187,11 +237,15 @@ rejected naming that form. The CLI's numbers follow the shell instead: `--seed 0 ## Samples say what they emit, transforms what they do +2026-09-02, Lilleman auf Larv. + `{upper(2)}` is two letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on whether the argument looks like a number. ## A record is a template seen as columns; a Go struct is the one second schema +2026-09-15, Lilleman auf Larv. + A template's `format` composes its fields into one string; `FakeRecord` and `--format` project the same fields as columns. Two views of one dataset, so a record author writes the same JSON they already know, and a column is the same field `Fake` renders by dotted @@ -205,6 +259,8 @@ not agree on one among themselves; each must only hold that type. ## A struct's records follow Go's field access, and compile on first use +2026-09-15, Lilleman auf Larv. + The fields an embedded struct promotes are the struct's own — `e.First`, as `encoding/json` and SQL mappers read them — so they are columns of its record and share its draws; a tagged field that another field hides is refused, not dropped. A named @@ -217,6 +273,8 @@ handle is needed, as the cache already compiles once. ## A render shares one reference draw per category, per group +2026-09-15, Lilleman auf Larv. + Every reference path into a category that selects no row, in one `Fake` or one record, reads one draw of it, so a value's facts agree across its fields, nested templates and columns alike — `{/currency.code}` in one field and `{/currency.symbol}` in another name @@ -229,6 +287,8 @@ it. ## A draw group name is local to its category +2026-09-16, Lilleman auf Larv. + A category's groups are its own entities, so a caller naming a group the same way never joins them by accident, and renaming a group inside one file changes no render elsewhere. The unnamed group still spans categories, since facts that belong together @@ -236,6 +296,8 @@ across categories must agree. ## The expansion hold and the render's draws are two fences +2026-09-16, Lilleman auf Larv. + Goal 2: one proves a sibling path or an operand is reached only by its readers within an expansion, the other that reference paths, whole-table reads and a table family's rows agree across a render and its draw groups. Their scopes differ and only the second @@ -244,6 +306,8 @@ step. ## A render's draws make their maps on the first read +2026-09-29, Lilleman auf Larv. + Every render starts from empty draws, its draw groups included, since goal 2 prices a reader meeting two spellings of one empty set. Making the maps where the draws are declared kept a record's on its frame's stack, but a `Fake`'s draws live on the `Generator`, so there it @@ -253,6 +317,8 @@ included, and one reading none pays nothing; goal 11 holds either way. ## A category never references itself, and a record's fences run at load +2026-09-16, Lilleman auf Larv. + A category is one unit: a reference back into it — `{/users.first}` inside `users` — describes a draw other than the fields beside it, where goal 3 asks facts that belong together to come from one draw, so `New` refuses it and the sibling path stays the one @@ -266,6 +332,8 @@ rendering format can matter to a caller that bare-references it. ## A record's column set is fixed before the first draw +2026-09-04, Lilleman auf Larv. + Only a category-level template is a record: a path descending into a field, or naming a folder or a choice, errors. A tail may pass through a choice whose variants carry different fields, so the columns — and with them the CSV header written once ahead of @@ -275,17 +343,23 @@ to agree. ## Null is a `null` item, not a rate +2026-09-15, Lilleman auf Larv. + A null is one more outcome of a column's draw, so a choice's weights skew it like any other; a null-rate option would be a second way to state odds. ## A typed column holds one value, not composed text +2026-09-15, Lilleman auf Larv. + Its bounds come from a literal or a call's arguments, so a load error names a real value, a range check is one comparison, and `1{digits(2)}` is a second spelling of `{int(100,199)}`. ## A column of one reference alone is the column it reads +2026-09-15, Lilleman auf Larv. + `{/src.score}` renders exactly what `src.score` draws, so it takes that column's datatype and null rather than restating them, and a `datatype` restating the one it takes is a second spelling. Any other `datatype` still types the values — the one way to @@ -293,22 +367,30 @@ type a column someone else wrote. ## A typed column's calc is refused unless proven +2026-09-15, Lilleman auf Larv. + Operand bounds must keep each divisor from zero and the result finite; what they cannot show is refused rather than trusted, since a bare `NaN` breaks the JSON and SQL it lands in. ## `Column` carries text, not a Go value +2026-09-15, Lilleman auf Larv. + `Value` is the rendered string beside `DataType` and `Null`, which each serializer writes as the load check proved it; a `Value any` would hand every caller a type switch. ## A unit takes the stem of what it is, and a file the stem of the units it holds +2026-09-29, Lilleman auf Larv. + Goal 2 wants a name to reach one unit: `drawMemo` is what a draw is kept in, `readValue` what a read drew. ## One name, one meaning +2026-09-25, Lilleman auf Larv. + Two declarations of any kind — function, method, field, type — share a name only where one definition covers both: the same kind of value, counted or addressed the same way, so a reader landing on the wrong one concludes nothing false, as `Fake` renders one value on @@ -320,10 +402,14 @@ cost. ## The package stays flat +2026-09-15, Lilleman auf Larv. + Go ties a package to one directory, so folders would split the API into packages. ## The performance gate asserts allocations, not wall-clock time +2026-09-03, Lilleman auf Larv. + `AllocsPerRun` is deterministic across machines, so a 10% ceiling does not flake under CI load, while time varies with the machine and its neighbours. A rendering slowdown almost always costs an allocation too (a lost pre-size, a per-item map, an extra copy). @@ -331,6 +417,8 @@ The benchmark suite (the README's Development) reports time for a human, not as ## Rows live in a TSV, the shape in JSON +2026-09-17, Lilleman auf Larv. + `New` allocates once per node, so a register of thirty thousand rows written as JSON objects would cost it a second; a TSV is one allocation whose cells are substrings, and the JSON says only how a row is composed. The TSV sits beside its category file, named @@ -339,24 +427,32 @@ a load error rather than a file silently ignored. ## A selector is bracketed, and a dot inside it is literal +2026-09-17, Lilleman auf Larv. + `municipality[0180]` reads as selection to anyone who has indexed an array, and `[St. Louis]` keeps a name whole where a colon or a dot-separated spelling could not; zsh needs the brackets quoted, which the README's examples show. ## `parent` names the link column and the table alike +2026-09-17, Lilleman auf Larv. + One word says both, so a child table sits in its parent's folder and links on a column of the parent's name; the geo plan wants exactly that, and a table needing another folder or another column name would be asking for a second spelling. ## After a row, a path names a column or a linked table +2026-09-17, Lilleman auf Larv. + `locality[Lund].address`, a template beside the family under a selected row, is refused today; admitting it later is additive, since a refused spelling gains a meaning and no accepted one changes, so the door stays open for the address records the plan describes. ## A selected row is fixed, not drawn, and an unnamed read beside it is refused +2026-09-27, Lilleman auf Larv. + Decided 2026-09-27 by the maintainer, for goals 3 and 4; valid while a selector names exactly one row. @@ -377,6 +473,8 @@ case but the last, where two expected a render and all three accepted the refusa ## A link column after a row steps up to the parent row, and a path may not end on it +2026-09-27, Lilleman auf Larv. + Decided 2026-09-27 by the maintainer, for goal 4; valid while a link column holds the parent's key. It extends [After a row, a path names a column or a linked table](#after-a-row-a-path-names-a-column-or-a-linked-table) from child tables to the @@ -405,6 +503,8 @@ the path ended. ## A table selects by one key column, a code, never a free-form name +2026-09-27, Lilleman auf Larv. + Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; valid while the key is the code a user writes, as a database keys a table. `country[NO]` loads, and `country[Norway]` is refused naming `[NO]`, found by the row whose cell spells the @@ -420,6 +520,8 @@ table, `street`, `car` or `useragent`, carries no key and is only drawn. ## A parent row with no child row is a load error +2026-09-17, Lilleman auf Larv. + A descendant is drawn inside the nearest pinned ancestor, so every ancestor row must lead to a row at every level below it, or a render could find nothing to draw. The import script drops or fills such rows; the alternative, falling back to a free draw, @@ -427,6 +529,8 @@ would break the consistency the link exists for without saying so. ## The choice-of-rows fence guards a data file's root, and requires string fields +2026-09-17, Lilleman auf Larv. + A table is a category with a TSV beside its file, so only a root choice has the spelling the fence names; a nested choice of same-shaped templates and an inline one keep loading. Fields must all be strings because a cell is a string node: a choice whose @@ -434,18 +538,24 @@ items carry a nested choice is not one table but two linked ones. ## A table is a record of string columns +2026-09-17, Lilleman auf Larv. + Its columns are the CSV header and the `INSERT` column list, fixed by the TSV header, so a table is a record by construction; every column is a string until a typed column option earns its place. ## The key map is built at load, the rest on first draw +2026-09-28, Lilleman auf Larv. + A link is proved against the parent's keys and a key's uniqueness is a data mistake, so both are load-time; the row lookup, by name and by parent, serves only a draw, so it waits for the first one, keeping `New` linear in the bytes read. ## Two categories may name one TSV +2026-09-17, Lilleman auf Larv. + Each is a view of the file with its own format and options, at the cost of holding the rows twice, which is what a category over a register with two natural formats asks for; a TSV nothing names stays a load error, since that one is a file forgotten rather than @@ -453,12 +563,16 @@ shared. ## A table never reaches its own family, by any route +2026-09-17, Lilleman auf Larv. + A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row that lists three localities from other regions is the output the family exists to prevent, so the own-family fence walks through both rather than stopping where the draw fences do. ## Tables carrying token cells stay small +2026-09-17, Lilleman auf Larv. + The family fence compares the reads of every pair of rows that can render together, so a table whose every row's cell selects a row of another table loads in time quadratic in its rows: about a second at four thousand rows. No shipped table carries such cells, and @@ -467,11 +581,15 @@ is until a real data set needs the indexed form. ## A path is walked once without drawing before it is walked for real +2026-09-17, Lilleman auf Larv. + A path that fails below its first level then moves no seeded stream, at the cost of one draw-free walk per call, which allocates nothing. ## A country's postal codes and streets are siblings under its locality +2026-09-18, Lilleman auf Larv. + No open source pairs a Swedish street with its postnummer, and pairing the US through its ZIPs would shape the two trees differently, so both draw inside the pinned locality and an address agrees at that level. A street's own code is the exact pairing to add @@ -479,11 +597,15 @@ when a source carries it. ## A locale's `address` reads its country's `geo` tree, so the shipped set loads whole +2026-09-24, Lilleman auf Larv. + `data/sv_SE` alone no longer loads: a test loads `data` and prefixes the locale, and `--no-shipped-data -d` takes the whole `data` folder or a set of one's own. ## The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more +2026-09-24, Lilleman auf Larv. + Sweden fits whole in 700 KB; every US place of 10,000 would pass a megabyte and fetch 1,200 counties of TIGER files, so the threshold sits where the two countries match in size, and `--min-population` and `--streets-per-locality` on the import scripts build a @@ -492,6 +614,8 @@ fuller set. The two trees add about 20 ms to `New`, which loads the shipped set ## A locale's `address` restates its country record's format +2026-09-18, Lilleman auf Larv. + A record cannot read another whole and keep its columns, so `sv_SE.address` names the same four columns as `geo.SE.address`, each a reference into it, and the format appears twice; a column is spelled the same in both, `street-number`, so the two never disagree @@ -499,12 +623,16 @@ on a name. ## A postort's kommun comes from its name, its tätort or its codes, never from distance +2026-09-24, Lilleman auf Larv. + GeoNames leaves a fifth of Sweden's codes without a kommun and carries stale spellings; the nearest code across a border named the wrong kommun half the time it was tried, so a postort none of the three rules place is dropped, as is one not cased like a place name. ## A highway designation is not a street, and a US postal code belongs to the place holding most of its land inside places +2026-09-24, Lilleman auf Larv. + `I- 55 Bus` and `US Hwy 1` carry the most address ranges in many places and would head every address, so the import drops names spelled as a route. A ZCTA goes to the place its largest in-place part lies in, census-designated places left out since they never @@ -515,6 +643,8 @@ postal city the USPS mostly names the same way. ## `--list` stays a plain list of paths +2026-09-18, Lilleman auf Larv. + It is what a script reads, so every line has to be a path that `Fake` takes; a marker for the tables a `[selector]` follows, or a legend above them, would make the output something to parse before use. `--help` names the selector spelling instead, and the @@ -522,6 +652,8 @@ Table section teaches it. ## A layout is always quoted +2026-09-18, Lilleman auf Larv. + A layout may carry the comma that separates arguments, `'January 2, 2006'`, and one spelling for every layout beats a rule about which ones need the quotes, so the bare spelling is refused naming the quoted one. The layout is Go's reference time because the @@ -530,6 +662,8 @@ locale's own month and weekday names are data. ## A title is a table under `sex` +2026-09-18, Lilleman auf Larv. + A prefix drawn apart would put `Mr` on a record whose `sex` column says `female`, which is the disagreement the record exists to prevent; the tables this set already has are what a title needs, so `en_US.title` links to `sex` as `first-name` does. Swedish has no @@ -539,6 +673,8 @@ are a curated proportion rather than bearers anyone counted. ## A table owns the spelling of a selector on it +2026-09-18, Lilleman auf Larv. + A reference reaches a table by a path that carries no selector — `sv_SE.address` reads `geo.SE.locality` through its template — so the walk that reached it cannot say where a reader would type a selector. The table's own location can, which is why it keeps its @@ -546,12 +682,16 @@ path, and why an error names `geo.SE.municipality[1281]` rather than `municipali ## No builtin reads the clock, so a date is bounded by days, never by an age +2026-09-18, Lilleman auf Larv. + An `age(min,max)` would make a seeded fixture change with the day it runs on, which is what a seed exists to prevent; a birthdate for someone 20 to 60 is `date(1966-01-01,2006-12-31,…)`, re-pinned as any fixture is. ## `misc` is what every locale shares +2026-09-18, Lilleman auf Larv. + A category whose facts differ by country belongs in that country's locale, read from the register that country's own records use; `misc` takes only sources that are international. NHTSA vPIC and Mobility Sweden's registrations are national, so they are @@ -559,6 +699,8 @@ for `en_US.car` and `sv_SE.car`, and `misc.car` waits for an international sourc ## A register's canonical spelling loses to the one its domain writes +2026-09-18, Lilleman auf Larv. + Where a source offers several spellings of one fact, the shipped one is what records in that domain carry. `misc.timezone` reads `zone.tab` and not the `zone1970.tab` that supersedes it, because the latter keeps one zone per set of countries that have agreed @@ -568,6 +710,8 @@ which says "Chinese, Mandarin" for `zh`. ## `misc.territory` is the spine, and a `misc` table naming a territory links to it +2026-09-24, Lilleman auf Larv. + Where every territory row has a child the column is a `parent`, and the import drops the child rows whose territory the set does not ship — 17 of `misc.timezone`'s, Antarctica's ten among them. Agreement across a record is worth more than the last rows of a table. @@ -577,6 +721,8 @@ naming the row. ## A table whose register publishes no frequency draws evenly +2026-09-20, Lilleman auf Larv. + `misc.httpmethod`, `misc.port`, `misc.httpstatus`, `misc.mimetype` and `misc.tld` weigh every row alike, so GET is a ninth of the methods drawn. Goal 12 keeps an authored fact out of a sourced table, and no register publishes how often a method, a port or a TLD is @@ -585,6 +731,8 @@ used, so a weight here would be invented. Where one exists it is read, as ## `misc.timezone` weighs a zone by the people living in it +2026-09-24, Lilleman auf Larv. + The weight is the population GeoNames records in the zone's cities of 15,000 or more, floored at 15,000, which goal 14 asks for: over 300 seeded draws of `misc.territory[US].timezone`, the four zones most Americans live in took 278 where an @@ -593,12 +741,16 @@ to 0. Valid while a draw weighted this way lands where people live. ## `misc.car` is one flat table, not a make linked to its models +2026-09-18, Lilleman auf Larv. + A row is a make and a model drawn together, so no render pairs a Volvo with a RAV4. Two linked tables would reach the same pairs and add a selector nothing asks for; a make alone is `misc.car.make`. ## `misc.territory` names its sovereign in a column, and there is no `misc.country` table +2026-09-24, Lilleman auf Larv. + ISO 3166-1 codes territories, so `territory` is the honest name, and `is_independent` in the register gives each one's state. A second table of the 195 sovereigns would hold a `DK` row beside the territory `DK` row, both carrying Denmark's capital, currency, flag @@ -611,6 +763,8 @@ claim fejkdata has no business making. ## `misc.loglevel` is a table keyed by POSIX's keyword, carrying the code +2026-09-27, Lilleman auf Larv. + A flat list of names carries neither the code a PRI encodes nor a selector reaching it, and goal 3 draws the two as one fact. The canonical spelling losing to the one its domain writes, above, settles the rest: a configuration writes `info`, so RFC 5424's @@ -618,12 +772,16 @@ domain writes, above, settles the rest: a configuration writes `info`, so RFC 54 ## `misc.tld` keys carry the leading dot, where other tables key on a bare code +2026-09-20, Lilleman auf Larv. + The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a host with no separator. `misc.tld[se]` misses for it, which `todo.md` carries. ## `misc.tld` is a table of its own, and `misc.territory.tld` stays a column +2026-09-20, Lilleman auf Larv. + A `parent` demands a child for every parent row, so linking them would drop every root zone row naming no territory, which is most of them, and goal 12 holds a sourced table whole. The loader refuses the link outright anyway: `tld` is a column of @@ -631,6 +789,8 @@ whole. The loader refuses the link outright anyway: `tld` is a column of ## `misc.territory` carries a currency code, it does not link to `misc.currency` +2026-09-18, Lilleman auf Larv. + A `parent` demands a child for every parent row, and ISO 4217 registers codes no country's row can name: the funds codes (Mvdol, WIR Euro, US Dollar (Next day)), and VED beside VES, both Venezuela's, of which a country row names one. Linking would trade the @@ -639,6 +799,8 @@ currency in one draw. ## An extension may name two media types +2026-09-18, Lilleman auf Larv. + `.xml`, `.rtf`, `.sub`, `.mpp` and `.ac` each name two rows of `misc.mimetype`. Separating them would mean dropping a registered type, or naming one by an extension that is not its own — `.mpt` is Project's template, not its document. Selecting such a @@ -646,6 +808,8 @@ name is an error listing both keys; select by type instead. ## The Swedish ids draw Skatteverket's test series +2026-09-18, Lilleman auf Larv. + A Luhn-valid personnummer over a random birth number may be a living person's; 238 and 239 after any date are blocked from assignment, so the shipped `personnummer` and `samordningsnummer` use those. They sit in a `birth-number` table under `sex` rather @@ -656,6 +820,8 @@ drawn from 61 to 88, valid in every month, rather than computed from the date dr ## The US given names come from a mirror of the SSA file +2026-09-18, Lilleman auf Larv. + ssa.gov refuses a client outside the US, so `names-us.py` reads a GitHub copy that ends at 2020, which a count over the births since 1930 barely feels; `--names` takes the official zip. The SSA's placeholder rows are top-1000 entries that name nobody, so the @@ -663,18 +829,24 @@ import drops them by name rather than by a rank a regeneration would move. ## `List` advertises direct descents only +2026-09-17, Lilleman auf Larv. + `region.municipality.locality` is listed, and `region.locality` resolves too but is not: the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. ## Each entry point to `drawFence.hasRead` says what its caller gets +2026-09-29, Lilleman auf Larv. + 2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on `readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, though `hasRead` states the same stop rule. Valid while the two differ in where they stop. ## The four path walks are four loops +2026-09-28, Lilleman auf Larv. + 2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` stay four loops. -- 2.52.0 From 0e8c054a4694b71528029d72ce9aa1542c8829be Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 18:41:07 +0200 Subject: [PATCH 352/966] Check CodeRabbit once at ready, and never wait for it below 10 stars --- AGENTS.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 23128d9..f5d169e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,6 +6,9 @@ - One-line commit messages: no ticket prefix, no repo name, no authorship trailers. - GitHub is where the project lives, so `gh` is the tool: merges are fast-forward only, `gh pr merge --rebase` where the branch is already on top of main. +- CodeRabbit skips every pull request until the repository has 10 GitHub stars: check + its status once when a PR is marked ready, answer a review if one is there, and never + wait for one. - README goal 2 is gated at 7.0 on the `comprehension-panel` skill, and below it nothing else merges — no feature, no category, no data — bar a security fix, a dependency bump, a repair of a promise goals 3 to 5 make, and the infrastructure -- 2.52.0 From 85cc000d23aa2d869aabd110c1a75c9a2a82a73a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:11:11 +0200 Subject: [PATCH 353/966] Drop the backfilled date where a decision already records its own --- docs/decisions.md | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 4928ea6..559a0ae 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -153,8 +153,6 @@ wording improves in a minor. ## Formats are checked after all data is loaded, so their errors name the category -2026-09-28, Lilleman auf Larv. - 2026-09-28, larv-review; approved 2026-09-28 by lilleman. Goal 2. Loading reads every file first, then connects references across them. A format holding a reference can only be checked once it is connected, so every format is checked then, and all checks run at @@ -389,8 +387,6 @@ a read drew. ## One name, one meaning -2026-09-25, Lilleman auf Larv. - Two declarations of any kind — function, method, field, type — share a name only where one definition covers both: the same kind of value, counted or addressed the same way, so a reader landing on the wrong one concludes nothing false, as `Fake` renders one value on @@ -451,8 +447,6 @@ accepted one changes, so the door stays open for the address records the plan de ## A selected row is fixed, not drawn, and an unnamed read beside it is refused -2026-09-27, Lilleman auf Larv. - Decided 2026-09-27 by the maintainer, for goals 3 and 4; valid while a selector names exactly one row. @@ -473,8 +467,6 @@ case but the last, where two expected a render and all three accepted the refusa ## A link column after a row steps up to the parent row, and a path may not end on it -2026-09-27, Lilleman auf Larv. - Decided 2026-09-27 by the maintainer, for goal 4; valid while a link column holds the parent's key. It extends [After a row, a path names a column or a linked table](#after-a-row-a-path-names-a-column-or-a-linked-table) from child tables to the @@ -503,8 +495,6 @@ the path ended. ## A table selects by one key column, a code, never a free-form name -2026-09-27, Lilleman auf Larv. - Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; valid while the key is the code a user writes, as a database keys a table. `country[NO]` loads, and `country[Norway]` is refused naming `[NO]`, found by the row whose cell spells the @@ -837,16 +827,12 @@ the direct chain is the one a reader can predict from the tables' parents. ## Each entry point to `drawFence.hasRead` says what its caller gets -2026-09-29, Lilleman auf Larv. - 2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on `readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, though `hasRead` states the same stop rule. Valid while the two differ in where they stop. ## The four path walks are four loops -2026-09-28, Lilleman auf Larv. - 2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` stay four loops. -- 2.52.0 From 45bdd10c9b282bdf9d330620720970e8ec5e87ba Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:11:47 +0200 Subject: [PATCH 354/966] Record that a cell may read a reference, and plan the round as the approved restructure --- AGENTS.md | 6 +++-- docs/decisions.md | 9 +++++++ todo.md | 68 +++++++++++++++++++++++++++++++---------------- 3 files changed, 58 insertions(+), 25 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f5d169e..9dcab2d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,8 +12,9 @@ - README goal 2 is gated at 7.0 on the `comprehension-panel` skill, and below it nothing else merges — no feature, no category, no data — bar a security fix, a dependency bump, a repair of a promise goals 3 to 5 make, and the infrastructure - the round itself runs on. `todo.md` carries the round: the nine-seat findings run at depth 1, - a PR per item it names, then the run again, until the score passes. At or above 7.0 + the round itself runs on. `todo.md` carries the round: a PR per item it plans, the nine-seat + findings at depth 1 or a restructure the maintainer approved, then the nine-seat run + again, until the score passes. At or above 7.0 every pull request scores with the four-seat run and answers it in one run. No merge lowers the last score its own panel recorded: the nine-seat round and the four-seat run are two ratchets, never compared with each other. @@ -85,6 +86,7 @@ In [docs/decisions.md](docs/decisions.md): - The key map is built at load, the rest on first draw - Two categories may name one TSV - A table never reaches its own family, by any route +- A cell may read a reference, and each row its own - Tables carrying token cells stay small - A path is walked once without drawing before it is walked for real - A country's postal codes and streets are siblings under its locality diff --git a/docs/decisions.md b/docs/decisions.md index 559a0ae..1261403 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -559,6 +559,15 @@ A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row tha three localities from other regions is the output the family exists to prevent, so the own-family fence walks through both rather than stopping where the draw fences do. +## A cell may read a reference, and each row its own + +2026-09-29, Lilleman auf Larv. + +Goal 3: facts that belong together come from one draw, so a shop row's `phone` cell reads +`{/sv_SE.phone}` on a Swedish shop and `{/en_US.phone}` on a US one, and the table's +format, the same for every row, cannot say that. A restructure for goal 2 keeps the fences +that prove such cells safe. Valid while a table's rows need generators of their own. + ## Tables carrying token cells stay small 2026-09-17, Lilleman auf Larv. diff --git a/todo.md b/todo.md index 6f1dfbd..a7c89cf 100644 --- a/todo.md +++ b/todo.md @@ -7,11 +7,10 @@ Ordered as the releases that ship it. ### Comprehension, until the panel scores 7.0 Nothing else ships while the score is under 7.0 — no feature, no category, no data — bar -what `AGENTS.md` excepts, per README goal 2. This section is one -round, replayed: the `comprehension-panel` skill's nine-seat findings run at depth 1, -a PR per item it names, then the run again, until the score passes. The run is the -nine-seat one: the round is meant to buy architectural change, and the four-seat -scoring run names too little to steer one. A fresh run replaces the items below. +what `AGENTS.md` excepts, per README goal 2. Nine rounds of the nine-seat findings held +the score between 5.7 and 6.1, so this round is a restructure the maintainer approved on +2026-09-29: a PR per item, then the nine-seat run again. The run is the nine-seat one, +since the four-seat scoring run names too little to steer an architectural change. Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9, @@ -37,37 +36,60 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Give `surveyAt`'s whole read one field, its table and pins together, rename - `alternatives` for what it answers, two reads that never render together, and say at - `check` why a level landing on a row may sit beside a path into it: five seats held - `surveyAt`'s five fields at once, two misread `alternatives`, and two found no reason - for the `landsRow` exemption. +- Compute what rendering a node reads as a fold memoised per node, `reads(n)`, deduped + per node, and have `surveyRender` and `surveyColumns` call it, deleting + `readSurvey.walk`, `edge`, `nodeVisit` and `seen`: every run's seats named the walk + hardest. Prototyped green with the allocation gate held; without the per-node dedup + `TestADeepDiamondChainLoads` doubles its list per level. +- Have `checkDrawGroup`, `checkDraws` and `checkRecordDraws` query the fold, deleting + `drawFence.hasRead`, its helpers and the decision "Each entry point to + `drawFence.hasRead` says what its caller gets". +- Move the fold to `reads.go` and the verdicts to `drawfence.go`, deleting + `readsurvey.go`; name a read's context its `group`, `route` and `branches`, deleting + `surveyAt`, and let the README's Layout and `doc.go` follow. +- Name what a read sits under `branches`, the rows pinned with their ancestors and the + rows of whole draws without, both load-bearing, and fold `check`, `checkFamilies`, + `replayPairs`, `checkFamilyPair` and `alternatives` into one pairwise loop over + `coRender` and `conflict`. Report a pin clash lowest by route and spelling, so which of + two is named stops following the walk, and say at `conflict` why a level landing on a + row may sit beside a path into it. +- Compile each path once at link into steps holding no pointer, with its leaves and + cover, and have `renderEdges`, `heldNodes`, `valueProof` and `tableReadOf` read them, + deleting `pathCover`, `coverPath` and `pathLeaves`. Prototyped green, allocations + identical. +- Draw every path through one interpreter over those steps, `drawSteps`, with `descend` + probing into a stack buffer, deleting `pathDraw`, and replace the decision "The four + path walks are four loops": the prototype held `perf_test.go` once a step held no + pointer, the probe returned its slice and the interpreter took `levels`. +- Fold `checkNoOverlap` and `checkNoRepeatedRead` into `heldCheck`, so `compileFormat` + compiles and nothing more. +- Fix at compile how `readField` reads an arm, `arm.kind`, deleting its two panics and + its `held` lookup per read. - Switch `disagreement` on the fixed item's kind, with the other item inside each arm, and rename its `proven` local, which shadows the `proven` type: seven seats read the seven-arm switch as a truth table without the table. - Say at `checkNodeFences` that its walks recurse unguarded, so a caller runs - `checkNoCycles` first, and compute `refs` apart from `checkDrawGroup`'s pass: five - seats reconstructed the order, and the inline binding runs no cycle check. + `checkNoCycles` first: five seats reconstructed the order, and the inline binding runs + no cycle check. - Give `tableSite` an `isFormat` beside `isCell`, so `renderRecord` and `traceRead` stop comparing `row` with `formatRow` alone, which holds only because a template outside a table has row 0: four seats found the invariant unstated. -- Move each unit to its owner's file, `valueProof.checkField` to `valueproof.go` and - `coverPath` and `pathLeaves` beside their walks in `path.go`, and rename `checkPath`, - which differs from the `pathCheck` type it wraps by word order alone: both architects - placed them wrong. +- Move `valueProof.checkField` to `valueproof.go`, and rename `checkPath`, which differs + from the `pathCheck` type it wraps by word order alone: both architects placed them + wrong. - Correct the claims readers followed wrong: `IsTemplate`'s doc says an argument holding - a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; the README says rows are indexed on the first draw, where `New` builds the key - map; and `loadDir`'s says it builds the tree deterministically, where it reports an - unnamed rows file in map order. + a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; the + README says rows are indexed on the first draw, where `New` builds the key map; and + `loadDir`'s says it builds the tree deterministically, where it reports an unnamed + rows file in map order. - Delete the comments the panel found restating their code or arranging the file: `node`'s second sentence, `op`'s "compile builds these so render never re-scans the format", `checkCalc`'s last sentence, `calcVar.eval`'s four lines, `checkNoCycles`' grey and black, and the docs on `expand`, `readRows`, `splitOutside`, `reservedList` and `writesRecords`. -- Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and - a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`, - `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by file. - Say "fences" in the `pin` entry then, which names the draw fences too. +- Delete `doc.go`'s "The draw fences are …" sentence, which misses `checkDrawGroup`, + `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by + file. Say "fences" in the `pin` entry then, which names the draw fences too. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 73fa0130114a38fc550a7f67a9c1f315b485d17f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:16:48 +0200 Subject: [PATCH 355/966] Survey tests: reach a table's cells through its own format, the shape New lets through --- readsurvey_test.go | 4 ++-- table_test.go | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/readsurvey_test.go b/readsurvey_test.go index a8733b0..c2d0098 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -109,7 +109,7 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { fsys := fstest.MapFS{ "g.json": {Data: []byte(`{"format":"{/x.v}","drawGroup":"g2"}`)}, "r.json": {Data: []byte(`"{/g} {/x}"`)}, - "x.json": {Data: []byte(`{"format":"{/g}","rows":"x.tsv","key":"code"}`)}, + "x.json": {Data: []byte(`{"format":"{v}","rows":"x.tsv","key":"code"}`)}, "x.tsv": {Data: []byte("code\tv\n1\t{/z}\n2\t{/z.a}\n")}, "z.json": {Data: []byte(`{"format":"{a}","a":"hi"}`)}, } @@ -133,7 +133,7 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { whole = whole || read.at.wholePins.used > 0 } if !pinned || !whole { - t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root and from inside {/x}", pinned, whole) + t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root, and {/x} renders them whole", pinned, whole) } } diff --git a/table_test.go b/table_test.go index 38a0e86..7af0cc4 100644 --- a/table_test.go +++ b/table_test.go @@ -1002,7 +1002,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rowCount(); row++ { - whole := surveyAt{wholeTable: entered} + whole := surveyAt{} whole.wholePins.add(entered, row) s := none.entered(entered, row) for _, cand := range tables { @@ -1022,7 +1022,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { if alternatives(whole, candAt) || alternatives(candAt, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } - other := surveyAt{wholeTable: cand} + other := surveyAt{} other.wholePins.add(cand, cr) if want := cand == entered && cr != row; alternatives(whole, other) != want { t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) -- 2.52.0 From 6fdd8bd09b8cf75d733a38d07d55373a34f90979 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:18:13 +0200 Subject: [PATCH 356/966] Test that a pin clash below a nested template names the root's route --- readsurvey_test.go | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/readsurvey_test.go b/readsurvey_test.go index c2d0098..8ab26a3 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -140,6 +140,7 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { fsys := fstest.MapFS{ "r.json": {Data: []byte(`"{/y[a].v}"`)}, + "n.json": {Data: []byte(`{"format":"{f}","f":"{/y[a].v}"}`)}, "x.json": {Data: []byte(`{"format":"{v}","rows":"x.tsv","key":"code"}`)}, "x.tsv": {Data: []byte("code\tv\n1\tb\n2\tc\n")}, "y.json": {Data: []byte(`{"format":"{v}","rows":"y.tsv","key":"code","parent":"x"}`)}, @@ -159,4 +160,11 @@ func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { if err := surveyRender(r).check(); err == nil || !strings.Contains(err.Error(), "two rows of") { t.Errorf("check = %v, want y[a]'s row of x clashing with x[2] named, whatever order the fences run in", err) } + n, isTemplate := g.children["n"].(*template) + if !isTemplate { + t.Fatalf("n is a %T, want a template", g.children["n"]) + } + if err := surveyRender(n).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with {/x[2].v}: x[1] and x[2] are two rows of x") { + t.Errorf("check = %v, want the clash below {f} named by the route the root reaches it by", err) + } } -- 2.52.0 From 4cd6e9f767e71732c7855f0a41040e11fa363a25 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:21:00 +0200 Subject: [PATCH 357/966] Gather a render's reads as a fold memoised per node, deleting readSurvey.walk and edge --- doc.go | 3 +- readsurvey.go | 284 ++++++++++++++++++++++++++++++++++---------------- todo.md | 5 - 3 files changed, 197 insertions(+), 95 deletions(-) diff --git a/doc.go b/doc.go index 8d1b118..38a3e51 100644 --- a/doc.go +++ b/doc.go @@ -37,8 +37,7 @@ package fejkdata // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // - whole — a read of a table with no selector and no descent, {/city}, landing on -// the `table` itself, so it draws a row apart from every pin: -// `surveyAt.wholeTable`, `surveyAt.wholePins`. +// the `table` itself, so it draws a row apart from every pin: `surveyAt.wholePins`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those // compile. The draw fences are `heldCheck`, `drawFence` and `checkFamilies`. diff --git a/readsurvey.go b/readsurvey.go index 7cca85c..6097d98 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -2,11 +2,9 @@ package fejkdata import "fmt" -// readSurvey gathers, at load, the references one render reads, by draw group, for check to compare. +// readSurvey is what one render reads, gathered at load for check to compare. type readSurvey struct { reads []pathRead - read map[readKey]bool - seen map[nodeVisit]bool clash *pinClash } @@ -17,15 +15,14 @@ type pinClash struct { err error } -// surveyAt is where a survey stands: the draw group its reads draw in, how the render's root reached it, the -// table rows it pinned, the table a whole read draws a row of, and the rows of whole draws whose -// cells the walk is in. +// surveyAt is where a read sits in its render: the draw group it draws in ("" until the +// root names one), how the render's root reached it, the rows pinned above it, and the rows +// of whole draws whose cells it sits in. type surveyAt struct { - group string - route surveyRoute - pins pinSet - wholeTable *table - wholePins pinSet + group string + route surveyRoute + pins pinSet + wholePins pinSet } // surveyRoute is how a render reaches a read: as its author spells it, and the root edge's label. @@ -33,17 +30,10 @@ type surveyRoute struct{ spelling, label string } // pathRead is one reference a render reads: a path, or a bare reference with no tail. type pathRead struct { - at surveyAt - a arm - tr *tableRead // set where the reference names a table -} - -type nodeVisit struct { - n node - group string - wholeTable *table - pins string - wholePins string + at surveyAt + a arm + tr *tableRead // set where the reference names a table + clash error // set where its selectors clash with a row its route pinned } type readKey struct { @@ -51,92 +41,210 @@ type readKey struct { path string pins string wholePins string + clash bool } -func newReadSurvey() *readSurvey { - return &readSurvey{read: map[readKey]bool{}, seen: map[nodeVisit]bool{}} +// readFold gathers, once per node, the reference reads rendering it makes. +type readFold struct { + memo map[node][]pathRead } +func newReadFold() *readFold { return &readFold{memo: map[node][]pathRead{}} } + func surveyRender(t *template) *readSurvey { - s := newReadSurvey() - for _, e := range renderEdges(t) { - s.edge(t, e, surveyAt{group: t.link.drawGroupKey, route: surveyRoute{e.reached(), e.label}}) - } - return s + f := newReadFold() + return survey(f.rootReads(t), t.link.drawGroupKey) } func surveyColumns(t *template, columns []string) *readSurvey { - s := newReadSurvey() - for _, name := range columns { - s.walk(t.fields[name], surveyAt{group: t.link.drawGroupKey, route: surveyRoute{spelling: fmt.Sprintf("column %q", name)}}) + f := newReadFold() + return survey(f.columnReads(t, columns), t.link.drawGroupKey) +} + +// rootReads is what rendering t as a render of its own reads, each read named by the edge of t +// that reaches it, and in "" where no template below t names a draw group. +func (f *readFold) rootReads(t *template) []pathRead { + var reads []pathRead + for _, e := range renderEdges(t) { + reads = append(reads, f.viaEdge(t, e, surveyRoute{e.reached(), e.label})...) } + return reads +} + +// columnReads is what rendering t's columns as one record reads. +func (f *readFold) columnReads(t *template, columns []string) []pathRead { + var reads []pathRead + for _, name := range columns { + reads = append(reads, routed(f.reads(t.fields[name]), surveyRoute{spelling: fmt.Sprintf("column %q", name)})...) + } + return reads +} + +// survey is the fold's result as a render in group: every read not inside a nested draw group +// draws in it, and one read gathered by several routes is kept once. +func survey(reads []pathRead, group string) *readSurvey { + s := &readSurvey{} + for _, r := range reads { + if r.clash != nil { + if s.clash == nil { + s.clash = &pinClash{r.at.route, r.a.spelling, r.clash} + } + continue + } + if r.at.group == "" { + r.at.group = group + } + s.reads = append(s.reads, r) + } + s.reads = distinct(s.reads) return s } -// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops -// there. -func (s *readSurvey) walk(n node, at surveyAt) { - v := nodeVisit{n, at.group, at.wholeTable, at.pins.key(), at.wholePins.key()} - if s.seen[v] { - return +// reads is what rendering n reads, short of a repeat, which renders over draws of its own: each +// read with the rows pinned above it inside n, and in its draw group where a template inside n +// names one. +func (f *readFold) reads(n node) []pathRead { + if r, done := f.memo[n]; done { + return r } - s.seen[v] = true - if repeats(n) { - return - } - if t, isTemplate := n.(*template); isTemplate && t.link.drawGroupKey != "" { - at.group = t.link.drawGroupKey - } - if t, isTable := n.(*table); isTable { - at.wholeTable = t - } - for _, e := range renderEdges(n) { - to, isTemplate := e.to.(*template) - var cell tableSite - if isTemplate { - cell = to.site + var out []pathRead + switch n := n.(type) { + case *choice: + for _, it := range n.items { + out = append(out, f.reads(it)...) } - switch { - case !cell.isCell(): - s.edge(n, e, at) - case cell.table == at.wholeTable: - in := at - in.wholePins = at.wholePins.clone() - in.wholePins.add(cell.table, cell.row) - s.edge(n, e, in) - case at.pins.clash(cell.table, cell.row) == nil: - in := at - in.pins = at.pins.entered(cell.table, cell.row) - s.edge(n, e, in) - default: - // A pinned table renders its pinned row alone, and an unpinned one draws a row inside its - // nearest pinned ancestor's, so a cell of a row the pins clash with never renders on this route. + case *template: + if n.repeat > 1 { + break } + for _, e := range renderEdges(n) { + out = append(out, f.viaEdge(n, e, surveyRoute{})...) + } + if n.link.drawGroupKey != "" { + for i := range out { + if out[i].at.group == "" { + out[i].at.group = n.link.drawGroupKey + } + } + } + case *table: + out = f.rowReads(n, func(p *surveyAt, row int) bool { + p.wholePins = p.wholePins.clone() + p.wholePins.add(n, row) + return true + }) + case *tableRow: + out = f.rowReads(n.t, func(p *surveyAt, row int) bool { return p.enter(n.t, row) }) + case *tableColumn: + out = f.cellReads(n, func(p *surveyAt, row int) bool { return p.enter(n.t, row) }) } + out = distinct(out) + f.memo[n] = out + return out } -// edge records the reference an edge reads, then walks on with every row the read -// pins entered, so a selected row renders only its own cells. -func (s *readSurvey) edge(from node, e renderEdge, at surveyAt) { - if e.readsRef() { - a := e.read - tr := tableReadOf(from.(*template).head(a.head), a, e.to) - if k := (readKey{at.group, a.path, at.pins.key(), at.wholePins.key()}); !s.read[k] { - s.read[k] = true - s.reads = append(s.reads, pathRead{at, a, tr}) - } - if tr != nil { - pins := at.pins.clone() - if err := tr.replay(&pins); err != nil { - if s.clash == nil { - s.clash = &pinClash{at.route, a.spelling, err} - } - return - } - at.pins = pins +// distinct keeps one of every read gathered by several routes, so a diamond of templates +// contributes each read once however many routes reach it. +func distinct(reads []pathRead) []pathRead { + seen := make(map[readKey]bool, len(reads)) + out := reads[:0:0] + for _, r := range reads { + k := readKey{r.at.group, r.a.path, r.at.pins.key(), r.at.wholePins.key(), r.clash != nil} + if !seen[k] { + seen[k] = true + out = append(out, r) } } - s.walk(e.to, at) + return out +} + +// viaEdge is what rendering e from n reads: the reference e reads, then what its leaf renders +// under the rows that reference pins. +func (f *readFold) viaEdge(n node, e renderEdge, route surveyRoute) []pathRead { + if !e.readsRef() { + return routed(f.reads(e.to), route) + } + a := e.read + tr := tableReadOf(n.(*template).head(a.head), a, e.to) + out := []pathRead{{at: surveyAt{route: route}, a: a, tr: tr}} + below := f.reads(e.to) + if tr != nil { + below = pinBelow(below, tr) + } + return append(out, routed(below, route)...) +} + +// pinBelow is the reads below a table read, once its selected rows are pinned above them: a read +// under a cell those rows keep out is dropped, since that cell never renders on this route, and +// a read whose own selectors clash with them carries that clash. +func pinBelow(reads []pathRead, tr *tableRead) []pathRead { + kept := reads[:0:0] + for _, r := range reads { + pins := r.at.pins.clone() + if err := tr.replay(&pins); err != nil { + continue + } + if r.tr != nil && r.clash == nil { + check := pins.clone() + r.clash = r.tr.replay(&check) + } + r.at.pins = pins + kept = append(kept, r) + } + return kept +} + +// rowReads is what the rows of t read through the columns its format renders. A format reaching +// its own cells through another category is checkOwnFamily's refusal, which runs first. +func (f *readFold) rowReads(t *table, tag func(*surveyAt, int) bool) []pathRead { + var out []pathRead + for _, e := range renderEdges(t.formatTemplate) { + if c, isColumn := e.to.(*tableColumn); isColumn { + out = append(out, f.cellReads(c, tag)...) + } + } + return out +} + +// cellReads is what the cells of column c read, each tagged with its row. +func (f *readFold) cellReads(c *tableColumn, tag func(*surveyAt, int) bool) []pathRead { + var out []pathRead + for r := 0; r < c.t.rowCount(); r++ { + cell := c.t.cellTemplate(r, c.i) + if cell == nil { + continue + } + for _, read := range f.reads(cell) { + if tag(&read.at, r) { + out = append(out, read) + } + } + } + return out +} + +// enter pins row r of t above a read, or reports that the rows already pinned keep it out. +func (at *surveyAt) enter(t *table, r int) bool { + if at.pins.clash(t, r) != nil { + return false + } + at.pins = at.pins.entered(t, r) + return true +} + +// routed names the route the render's root reaches reads by, where none is named yet. +func routed(reads []pathRead, route surveyRoute) []pathRead { + if route == (surveyRoute{}) { + return reads + } + out := make([]pathRead, len(reads)) + for i, r := range reads { + if r.at.route == (surveyRoute{}) { + r.at.route = route + } + out[i] = r + } + return out } // spelled names the route, and the reference it reaches a read by where its root edge is not that diff --git a/todo.md b/todo.md index a7c89cf..e633882 100644 --- a/todo.md +++ b/todo.md @@ -36,11 +36,6 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Compute what rendering a node reads as a fold memoised per node, `reads(n)`, deduped - per node, and have `surveyRender` and `surveyColumns` call it, deleting - `readSurvey.walk`, `edge`, `nodeVisit` and `seen`: every run's seats named the walk - hardest. Prototyped green with the allocation gate held; without the per-node dedup - `TestADeepDiamondChainLoads` doubles its list per level. - Have `checkDrawGroup`, `checkDraws` and `checkRecordDraws` query the fold, deleting `drawFence.hasRead`, its helpers and the decision "Each entry point to `drawFence.hasRead` says what its caller gets". -- 2.52.0 From fc5f1e868533f0cdc6d25223ae4e4d7ce476ffab Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:28:51 +0200 Subject: [PATCH 358/966] Test that a table format's own reference reads are fenced, whole or by row --- drawfence_test.go | 2 +- table_test.go | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drawfence_test.go b/drawfence_test.go index 701a3a4..d0e810b 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -6,7 +6,7 @@ func fenceCorpus(t *testing.T) *Generator { t.Helper() dir := writeFiles(t, with(geo(), map[string]string{ "addr.json": `{"format":"{a} {b} {c}","a":"{/locality.name}, {/municipality.name}","b":{"format":"{/region[12].name} {/place}","drawGroup":"g"},"c":{"format":"{/place.zip} ","repeat":2}}`, - "place.json": `{"format":"{zip} {name} {tag}","rows":"place.tsv","key":"name"}`, + "place.json": `{"format":"{zip} {name} {tag} {/w}","rows":"place.tsv","key":"name"}`, "place.tsv": "name\tzip\ttag\nStockholm\t1{digits(2)} {digits(2)}\t{/x}\nTranås\t573 {digits(2)}\t{/region[12].name}\n", "rec.json": `{"format":"","l":"{/locality.name}","m":"{/municipality.name}","t":"{/place[Stockholm].tag}"}`, "w.json": `["x","y"]`, diff --git a/table_test.go b/table_test.go index 7af0cc4..9cce34e 100644 --- a/table_test.go +++ b/table_test.go @@ -465,6 +465,9 @@ func TestTableFences(t *testing.T) { "a name that is another row's key": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","key":"a","name":"n"}`, "t.tsv": "a\tn\nx\ty\ny\tz\n"}, `"y"`}, "a cell reading its family": {with(geo(), map[string]string{"locality.tsv": "code\tname\tmunicipality\tnote\nL1\tStockholm\t0180\t{/municipality.code}\nL2\tSolna\t0184\t-\nL3\tMalmö\t1280\t-\nL4\tLund\t1281\t-\nL5\tGöteborg\t1480\t-\n"}), "family"}, "a format reading its family": {with(geo(), map[string]string{"locality.json": `{"format":"{name} {/region.name}","rows":"locality.tsv","key":"code","name":"name","parent":"municipality"}`}), "family"}, + "a format's reference beside a path into it": {map[string]string{"x.json": `{"format":"{v} {/z}","rows":"x.tsv","key":"code"}`, "x.tsv": "code\tv\n1\ta\n2\tb\n", "z.json": `{"format":"{a}","a":["p","q"]}`, "r.json": `"{/x} {/z.a}"`}, "renders its own draw of what {/z.a}"}, + "a selected row's format selecting another row": {map[string]string{"x.json": `{"format":"{v} {/y[1].w}","rows":"x.tsv","key":"code"}`, "x.tsv": "code\tv\n1\ta\n2\tb\n", "y.json": `{"format":"{w}","rows":"y.tsv","key":"k"}`, "y.tsv": "k\tw\n1\tm\n2\tn\n", "r.json": `"{/x[1]} {/y[2].w}"`}, "two rows of y"}, + "a whole table's format selecting another row": {map[string]string{"x.json": `{"format":"{v} {/y[1].w}","rows":"x.tsv","key":"code"}`, "x.tsv": "code\tv\n1\ta\n2\tb\n", "y.json": `{"format":"{w}","rows":"y.tsv","key":"k"}`, "y.tsv": "k\tw\n1\tm\n2\tn\n", "r.json": `"{/x} {/y[2].w}"`}, "two rows of y"}, "a descendant named like an ancestor's column": {with(geo(), map[string]string{"region.tsv": "code\tname\tpopulation\tlocality\n01\tStockholms län\t2400000\tx\n12\tSkåne län\t1400000\ty\n14\tVästra Götalands län\t1750000\tz\n"}), `"locality"`}, "weight not a number": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","weight":"w"}`, "t.tsv": "a\tw\nx\tmany\ny\t2\n"}, `"many"`}, "weight zero": {map[string]string{"t.json": `{"format":"{a}","rows":"t.tsv","weight":"w"}`, "t.tsv": "a\tw\nx\t0\ny\t2\n"}, "0"}, -- 2.52.0 From 8d5c7e0adf91fd5b6ba274b41314c96549a59248 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 20:30:36 +0200 Subject: [PATCH 359/966] Fold a table format's own reference reads, and tag only its own columns' cells with a row --- readsurvey.go | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/readsurvey.go b/readsurvey.go index 6097d98..fd7e26d 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -114,7 +114,7 @@ func (f *readFold) reads(n node) []pathRead { out = append(out, f.reads(it)...) } case *template: - if n.repeat > 1 { + if repeats(n) { break } for _, e := range renderEdges(n) { @@ -194,14 +194,17 @@ func pinBelow(reads []pathRead, tr *tableRead) []pathRead { return kept } -// rowReads is what the rows of t read through the columns its format renders. A format reaching -// its own cells through another category is checkOwnFamily's refusal, which runs first. +// rowReads is what the rows of t read: the cells of the columns its format renders, each tagged +// with its row, and the format's other reads, which every row shares. A format reaching its own +// cells through another category is checkOwnFamily's refusal, which runs first. func (f *readFold) rowReads(t *table, tag func(*surveyAt, int) bool) []pathRead { var out []pathRead for _, e := range renderEdges(t.formatTemplate) { - if c, isColumn := e.to.(*tableColumn); isColumn { + if c, isColumn := e.to.(*tableColumn); isColumn && !e.readsRef() { out = append(out, f.cellReads(c, tag)...) + continue } + out = append(out, f.viaEdge(t.formatTemplate, e, surveyRoute{})...) } return out } -- 2.52.0 From 78b08b699ffa037a132af649ff4462142847004f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:16:50 +0200 Subject: [PATCH 360/966] Have the draw fence query the read fold, deleting hasRead and its helpers --- AGENTS.md | 1 - docs/decisions.md | 6 --- drawfence.go | 97 +++++++++++++++-------------------------------- todo.md | 3 -- 4 files changed, 31 insertions(+), 76 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9dcab2d..f6c7ca9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -115,5 +115,4 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only -- Each entry point to `drawFence.hasRead` says what its caller gets - The four path walks are four loops diff --git a/docs/decisions.md b/docs/decisions.md index 1261403..7cdcb04 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -834,12 +834,6 @@ import drops them by name rather than by a rank a regeneration would move. the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. -## Each entry point to `drawFence.hasRead` says what its caller gets - -2026-09-25, larv-review; approved 2026-09-25 by lilleman. Goal 2: a reader lands on -`readsPath` or `splitsDraws` from its caller, so each doc states where that walk stops, -though `hasRead` states the same stop rule. Valid while the two differ in where they stop. - ## The four path walks are four loops 2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. diff --git a/drawfence.go b/drawfence.go index a6bbee4..2eb6fd5 100644 --- a/drawfence.go +++ b/drawfence.go @@ -6,26 +6,34 @@ import ( "strings" ) -// drawFence fences each template of a scope as a render of its own, remembering which nodes read a -// reference path. +// drawFence fences each template of a scope as a render of its own, over one fold of what every +// node reads. // docs/decisions.md#a-render-shares-one-reference-draw-per-category-per-group // docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences type drawFence struct { - memo map[hasReadMemo]bool + fold *readFold } -type hasReadMemo struct { - n node - stopAtGroup bool +func (f *drawFence) reads() *readFold { + if f.fold == nil { + f.fold = newReadFold() + } + return f.fold } // checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference // path inside a repeat or a nested draw group, which draw apart from it whatever it names. func (f *drawFence) checkDrawGroup(path string, n node) error { - if t, ok := n.(*template); ok && t.drawGroup != "" && !f.splitsDraws(t) { - return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) + t, ok := n.(*template) + if !ok || t.drawGroup == "" { + return nil } - return nil + for _, r := range f.reads().rootReads(t) { + if r.at.group == "" && r.draws() { + return nil + } + } + return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) } func (f *drawFence) checkDraws(path string, n node) error { @@ -33,10 +41,11 @@ func (f *drawFence) checkDraws(path string, n node) error { if !ok { return nil } - if !f.readsPath(t) { + reads := f.reads().rootReads(t) + if !anyDraws(reads) { return nil } - if err := surveyRender(t).check(); err != nil { + if err := survey(reads, t.link.drawGroupKey).check(); err != nil { return fmt.Errorf("%s: %w", path, err) } return nil @@ -50,15 +59,11 @@ func (f *drawFence) checkRecordDraws(path string, n node) error { if !ok || !t.isRecord { return nil } - columns := sortedNames(t.fields) - reads := false - for _, name := range columns { - reads = reads || f.readsPath(t.fields[name]) - } - if !reads { + reads := f.reads().columnReads(t, sortedNames(t.fields)) + if !anyDraws(reads) { return nil } - if err := surveyColumns(t, columns).check(); err != nil { + if err := survey(reads, t.link.drawGroupKey).check(); err != nil { return fmt.Errorf("%s: %w", path, err) } return nil @@ -100,41 +105,17 @@ func checkOwnFamily(path string, n node) error { return nil } -// readsPath reports whether rendering n reads a reference path, short of a repeat, which renders -// over draws of its own. -func (f *drawFence) readsPath(n node) bool { return f.hasRead(n, false) } +// draws reports whether a read's draw is its group's to answer for: a path, or a table read whole, +// whose row the group draws. +func (r pathRead) draws() bool { return len(r.a.tail) > 0 || r.tr != nil } -// splitsDraws reports whether rendering n reads a reference path that n's own draw group answers -// for: one outside a repeat and outside a nested draw group, which hold their own draws. -func (f *drawFence) splitsDraws(n node) bool { return f.hasRead(n, true) } - -// hasRead walks what rendering n renders for a reference path, stopping at a repeat — and at a nested -// draw group when stopAtGroup — since each holds draws of its own. -func (f *drawFence) hasRead(n node, stopAtGroup bool) bool { - k := hasReadMemo{n, stopAtGroup} - if r, done := f.memo[k]; done { - return r - } - r := false - for _, e := range renderEdges(n) { - if readsOnEdge(n, e) || (walksInto(e.to, stopAtGroup) && f.hasRead(e.to, stopAtGroup)) { - r = true - break +func anyDraws(reads []pathRead) bool { + for _, r := range reads { + if r.draws() { + return true } } - if f.memo == nil { - f.memo = map[hasReadMemo]bool{} - } - f.memo[k] = r - return r -} - -func readsOnEdge(n node, e renderEdge) bool { - return e.readsRef() && (len(e.read.tail) > 0 || readsTable(n, e.read)) -} - -func walksInto(to node, stopAtGroup bool) bool { - return !repeats(to) && !(stopAtGroup && grouped(to)) + return false } func repeats(n node) bool { @@ -142,22 +123,6 @@ func repeats(n node) bool { return isTemplate && t.repeat > 1 } -func grouped(n node) bool { - t, isTemplate := n.(*template) - return isTemplate && t.link.drawGroupKey != "" -} - -// readsTable reports whether a reference of n names a table, whose draw the render's -// group answers for even when read whole. -func readsTable(n node, a arm) bool { - t, isTemplate := n.(*template) - if !isTemplate { - return false - } - _, isTable := t.head(a.head).(*table) - return isTable -} - // check refuses what one draw per reference path cannot answer for: a read selecting a row its route // pinned another of, a read of a level beside a path another read takes into it, and two reads of one // table family that select different rows. diff --git a/todo.md b/todo.md index e633882..d0c5822 100644 --- a/todo.md +++ b/todo.md @@ -36,9 +36,6 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Have `checkDrawGroup`, `checkDraws` and `checkRecordDraws` query the fold, deleting - `drawFence.hasRead`, its helpers and the decision "Each entry point to - `drawFence.hasRead` says what its caller gets". - Move the fold to `reads.go` and the verdicts to `drawfence.go`, deleting `readsurvey.go`; name a read's context its `group`, `route` and `branches`, deleting `surveyAt`, and let the README's Layout and `doc.go` follow. -- 2.52.0 From b7bbf40366a579b7960e7aa1a8a42110640f310b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:17:26 +0200 Subject: [PATCH 361/966] Move surveyRender and surveyColumns, used only by tests, into readsurvey_test.go --- readsurvey.go | 10 ---------- readsurvey_test.go | 10 ++++++++++ 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/readsurvey.go b/readsurvey.go index fd7e26d..ebf3736 100644 --- a/readsurvey.go +++ b/readsurvey.go @@ -51,16 +51,6 @@ type readFold struct { func newReadFold() *readFold { return &readFold{memo: map[node][]pathRead{}} } -func surveyRender(t *template) *readSurvey { - f := newReadFold() - return survey(f.rootReads(t), t.link.drawGroupKey) -} - -func surveyColumns(t *template, columns []string) *readSurvey { - f := newReadFold() - return survey(f.columnReads(t, columns), t.link.drawGroupKey) -} - // rootReads is what rendering t as a render of its own reads, each read named by the edge of t // that reaches it, and in "" where no template below t names a draw group. func (f *readFold) rootReads(t *template) []pathRead { diff --git a/readsurvey_test.go b/readsurvey_test.go index 8ab26a3..184981e 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -168,3 +168,13 @@ func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { t.Errorf("check = %v, want the clash below {f} named by the route the root reaches it by", err) } } + +func surveyRender(t *template) *readSurvey { + f := newReadFold() + return survey(f.rootReads(t), t.link.drawGroupKey) +} + +func surveyColumns(t *template, columns []string) *readSurvey { + f := newReadFold() + return survey(f.columnReads(t, columns), t.link.drawGroupKey) +} -- 2.52.0 From a222fb985e447b2d6405b6c39b9a53190fb44658 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:26:47 +0200 Subject: [PATCH 362/966] Share one read fold across each oracle's corpus, as the draw fence does --- drawfence_test.go | 8 +++++--- readsurvey_test.go | 17 ++++++++--------- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index d0e810b..1e30a0d 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -64,10 +64,11 @@ func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { return nil }) compared := 0 + fold := newReadFold() for _, root := range roots { - compared += wantPairsFindAll(t, root.label, surveyRender(root.t).reads) + compared += wantPairsFindAll(t, root.label, surveyRender(fold, root.t).reads) if root.t.isRecord && len(root.t.fields) > 0 { - compared += wantPairsFindAll(t, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads) + compared += wantPairsFindAll(t, root.label+" as a record", surveyColumns(fold, root.t, sortedNames(root.t.fields)).reads) } } if compared == 0 { @@ -81,6 +82,7 @@ func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { t.Fatalf("New = %v", err) } var reads []pathRead + fold := newReadFold() _ = walkNodes(f.root.children, func(_ string, n node) error { tb, isTable := n.(*table) if !isTable || tb.keyIndex < 0 || tb.familyRoot().path != "region" { @@ -95,7 +97,7 @@ func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { if !isTemplate { t.Fatalf("NewTemplate(%s) compiled to %T", tb.selectorSpelling(r), inline.n) } - reads = append(reads, surveyRender(root).reads...) + reads = append(reads, surveyRender(fold, root).reads...) } return nil }) diff --git a/readsurvey_test.go b/readsurvey_test.go index 184981e..f7a7ee0 100644 --- a/readsurvey_test.go +++ b/readsurvey_test.go @@ -10,8 +10,9 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { f := fenceCorpus(t) roots := fenceRoots(t, f) traced := 0 + fold := newReadFold() for _, root := range roots { - traced += wantGathered(t, root.label, surveyRender(root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) + traced += wantGathered(t, root.label, surveyRender(fold, root.t).reads, func(trace renderTrace) { renderRoot(f.rand, root.t, trace) }) if !root.t.isRecord || len(root.t.fields) == 0 { continue } @@ -19,7 +20,7 @@ func TestEveryReadARenderMakesIsGathered(t *testing.T) { if err != nil { t.Fatalf("%s: recordOf = %v", root.label, err) } - traced += wantGathered(t, root.label+" as a record", surveyColumns(root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) + traced += wantGathered(t, root.label+" as a record", surveyColumns(fold, root.t, sortedNames(root.t.fields)).reads, func(trace renderTrace) { renderRecordRoot(f.rand, root.t, columns, trace) }) } if traced == 0 { t.Error("no render traced a reference read, so the comparison proved nothing") @@ -125,7 +126,7 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { t.Fatalf("r is a %T, want a template", g.children["r"]) } var pinned, whole bool - for _, read := range surveyRender(r).reads { + for _, read := range surveyRender(newReadFold(), r).reads { if read.a.path != "/z" { continue } @@ -157,24 +158,22 @@ func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { if !isTemplate { t.Fatalf("r is a %T, want a template", g.children["r"]) } - if err := surveyRender(r).check(); err == nil || !strings.Contains(err.Error(), "two rows of") { + if err := surveyRender(newReadFold(), r).check(); err == nil || !strings.Contains(err.Error(), "two rows of") { t.Errorf("check = %v, want y[a]'s row of x clashing with x[2] named, whatever order the fences run in", err) } n, isTemplate := g.children["n"].(*template) if !isTemplate { t.Fatalf("n is a %T, want a template", g.children["n"]) } - if err := surveyRender(n).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with {/x[2].v}: x[1] and x[2] are two rows of x") { + if err := surveyRender(newReadFold(), n).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with {/x[2].v}: x[1] and x[2] are two rows of x") { t.Errorf("check = %v, want the clash below {f} named by the route the root reaches it by", err) } } -func surveyRender(t *template) *readSurvey { - f := newReadFold() +func surveyRender(f *readFold, t *template) *readSurvey { return survey(f.rootReads(t), t.link.drawGroupKey) } -func surveyColumns(t *template, columns []string) *readSurvey { - f := newReadFold() +func surveyColumns(f *readFold, t *template, columns []string) *readSurvey { return survey(f.columnReads(t, columns), t.link.drawGroupKey) } -- 2.52.0 From 42c443bd5a4782d7ce7002264ddeee8a54c90469 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:27:23 +0200 Subject: [PATCH 363/966] Build the draw fence's fold with the fence, deleting its lazy accessor --- data.go | 2 +- drawfence.go | 13 +++---------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/data.go b/data.go index 28bd9be..5ee88f1 100644 --- a/data.go +++ b/data.go @@ -239,7 +239,7 @@ func checkNodeFences(s nodeScope) error { if err := s(heldCheck); err != nil { return err } - fence := &drawFence{} + fence := &drawFence{fold: newReadFold()} refs := false if err := s(func(path string, n node) error { if t, ok := n.(*template); ok && len(t.link.refs) > 0 { diff --git a/drawfence.go b/drawfence.go index 2eb6fd5..4d0a6a5 100644 --- a/drawfence.go +++ b/drawfence.go @@ -14,13 +14,6 @@ type drawFence struct { fold *readFold } -func (f *drawFence) reads() *readFold { - if f.fold == nil { - f.fold = newReadFold() - } - return f.fold -} - // checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference // path inside a repeat or a nested draw group, which draw apart from it whatever it names. func (f *drawFence) checkDrawGroup(path string, n node) error { @@ -28,7 +21,7 @@ func (f *drawFence) checkDrawGroup(path string, n node) error { if !ok || t.drawGroup == "" { return nil } - for _, r := range f.reads().rootReads(t) { + for _, r := range f.fold.rootReads(t) { if r.at.group == "" && r.draws() { return nil } @@ -41,7 +34,7 @@ func (f *drawFence) checkDraws(path string, n node) error { if !ok { return nil } - reads := f.reads().rootReads(t) + reads := f.fold.rootReads(t) if !anyDraws(reads) { return nil } @@ -59,7 +52,7 @@ func (f *drawFence) checkRecordDraws(path string, n node) error { if !ok || !t.isRecord { return nil } - reads := f.reads().columnReads(t, sortedNames(t.fields)) + reads := f.fold.columnReads(t, sortedNames(t.fields)) if !anyDraws(reads) { return nil } -- 2.52.0 From 4952ae94429463973dbd33602566cf3f32fb543e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:37:08 +0200 Subject: [PATCH 364/966] Checkpoint the round with a nine-seat run before the path steps --- todo.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/todo.md b/todo.md index d0c5822..59efc5e 100644 --- a/todo.md +++ b/todo.md @@ -45,6 +45,8 @@ This round, in order: `coRender` and `conflict`. Report a pin clash lowest by route and spelling, so which of two is named stops following the walk, and say at `conflict` why a level landing on a row may sit beside a path into it. +- Run the nine-seat panel as a checkpoint, and file its scores above: where Locality has + not risen from 5.2, stop and bring the scores to the maintainer before the path steps. - Compile each path once at link into steps holding no pointer, with its leaves and cover, and have `renderEdges`, `heldNodes`, `valueProof` and `tableReadOf` read them, deleting `pathCover`, `coverPath` and `pathLeaves`. Prototyped green, allocations -- 2.52.0 From 9e786ccf39ab9643880fa7956e32ff2187c468ac Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:44:39 +0200 Subject: [PATCH 365/966] Move the read fold to reads.go and the survey's verdict to drawfence.go, deleting readsurvey.go --- README.md | 4 +-- drawfence.go | 33 +++++++++++++++++++++ drawfence_test.go | 46 ++++++++++++++++++++++++++++- readsurvey.go => reads.go | 33 --------------------- readsurvey_test.go => reads_test.go | 40 ------------------------- 5 files changed, 80 insertions(+), 76 deletions(-) rename readsurvey.go => reads.go (91%) rename readsurvey_test.go => reads_test.go (71%) diff --git a/README.md b/README.md index a2e1888..0bb4537 100644 --- a/README.md +++ b/README.md @@ -885,8 +885,8 @@ inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplat format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format draws.go the memo a hold or draw group keeps its draws in; and a render's draws and the scope it reads its reference paths through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways -drawfence.go the load fences over reference draws and their refusals: a drawGroup's, a render's and a record's columns judged on the survey, and a table reaching its own family -readsurvey.go the load-time survey of what a render reads, and what a path reads of a table family +drawfence.go the load fences over reference draws and their refusals: a drawGroup's, a render's and a record's columns judged on what they read, and a table reaching its own family +reads.go the load-time fold of what a render reads, and what a path reads of a table family pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and binding references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences diff --git a/drawfence.go b/drawfence.go index 4d0a6a5..926ed57 100644 --- a/drawfence.go +++ b/drawfence.go @@ -14,6 +14,39 @@ type drawFence struct { fold *readFold } +// readSurvey is what one render reads, gathered at load for check to compare. +type readSurvey struct { + reads []pathRead + clash *pinClash +} + +// pinClash is the first read whose selectors clash with a row its route pinned. +type pinClash struct { + route surveyRoute + spelling string + err error +} + +// survey is the fold's result as a render in group: every read not inside a nested draw group +// draws in it, and one read gathered by several routes is kept once. +func survey(reads []pathRead, group string) *readSurvey { + s := &readSurvey{} + for _, r := range reads { + if r.clash != nil { + if s.clash == nil { + s.clash = &pinClash{r.at.route, r.a.spelling, r.clash} + } + continue + } + if r.at.group == "" { + r.at.group = group + } + s.reads = append(s.reads, r) + } + s.reads = distinct(s.reads) + return s +} + // checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference // path inside a repeat or a nested draw group, which draw apart from it whatever it names. func (f *drawFence) checkDrawGroup(path string, n node) error { diff --git a/drawfence_test.go b/drawfence_test.go index 1e30a0d..bed94b0 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -1,6 +1,10 @@ package fejkdata -import "testing" +import ( + "strings" + "testing" + "testing/fstest" +) func fenceCorpus(t *testing.T) *Generator { t.Helper() @@ -175,3 +179,43 @@ func eachRenderSet(reads []pathRead, fn func([]pathRead)) { } grow(nil, reads, nil) } + +func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { + fsys := fstest.MapFS{ + "r.json": {Data: []byte(`"{/y[a].v}"`)}, + "n.json": {Data: []byte(`{"format":"{f}","f":"{/y[a].v}"}`)}, + "x.json": {Data: []byte(`{"format":"{v}","rows":"x.tsv","key":"code"}`)}, + "x.tsv": {Data: []byte("code\tv\n1\tb\n2\tc\n")}, + "y.json": {Data: []byte(`{"format":"{v}","rows":"y.tsv","key":"code","parent":"x"}`)}, + "y.tsv": {Data: []byte("code\tv\tx\na\t{/x[2].v}\t1\nb\tz\t2\n")}, + } + g, err := loadDir(dataSource{fsys: fsys}, ".") + if err != nil { + t.Fatalf("loadDir = %v", err) + } + if err := treeBinding(g.children).link(); err != nil { + t.Fatalf("link = %v", err) + } + r, isTemplate := g.children["r"].(*template) + if !isTemplate { + t.Fatalf("r is a %T, want a template", g.children["r"]) + } + if err := surveyRender(newReadFold(), r).check(); err == nil || !strings.Contains(err.Error(), "two rows of") { + t.Errorf("check = %v, want y[a]'s row of x clashing with x[2] named, whatever order the fences run in", err) + } + n, isTemplate := g.children["n"].(*template) + if !isTemplate { + t.Fatalf("n is a %T, want a template", g.children["n"]) + } + if err := surveyRender(newReadFold(), n).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with {/x[2].v}: x[1] and x[2] are two rows of x") { + t.Errorf("check = %v, want the clash below {f} named by the route the root reaches it by", err) + } +} + +func surveyRender(f *readFold, t *template) *readSurvey { + return survey(f.rootReads(t), t.link.drawGroupKey) +} + +func surveyColumns(f *readFold, t *template, columns []string) *readSurvey { + return survey(f.columnReads(t, columns), t.link.drawGroupKey) +} diff --git a/readsurvey.go b/reads.go similarity index 91% rename from readsurvey.go rename to reads.go index ebf3736..d56e33d 100644 --- a/readsurvey.go +++ b/reads.go @@ -2,19 +2,6 @@ package fejkdata import "fmt" -// readSurvey is what one render reads, gathered at load for check to compare. -type readSurvey struct { - reads []pathRead - clash *pinClash -} - -// pinClash is the first read whose selectors clash with a row its route pinned. -type pinClash struct { - route surveyRoute - spelling string - err error -} - // surveyAt is where a read sits in its render: the draw group it draws in ("" until the // root names one), how the render's root reached it, the rows pinned above it, and the rows // of whole draws whose cells it sits in. @@ -70,26 +57,6 @@ func (f *readFold) columnReads(t *template, columns []string) []pathRead { return reads } -// survey is the fold's result as a render in group: every read not inside a nested draw group -// draws in it, and one read gathered by several routes is kept once. -func survey(reads []pathRead, group string) *readSurvey { - s := &readSurvey{} - for _, r := range reads { - if r.clash != nil { - if s.clash == nil { - s.clash = &pinClash{r.at.route, r.a.spelling, r.clash} - } - continue - } - if r.at.group == "" { - r.at.group = group - } - s.reads = append(s.reads, r) - } - s.reads = distinct(s.reads) - return s -} - // reads is what rendering n reads, short of a repeat, which renders over draws of its own: each // read with the rows pinned above it inside n, and in its draw group where a template inside n // names one. diff --git a/readsurvey_test.go b/reads_test.go similarity index 71% rename from readsurvey_test.go rename to reads_test.go index f7a7ee0..3b33c7e 100644 --- a/readsurvey_test.go +++ b/reads_test.go @@ -137,43 +137,3 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root, and {/x} renders them whole", pinned, whole) } } - -func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { - fsys := fstest.MapFS{ - "r.json": {Data: []byte(`"{/y[a].v}"`)}, - "n.json": {Data: []byte(`{"format":"{f}","f":"{/y[a].v}"}`)}, - "x.json": {Data: []byte(`{"format":"{v}","rows":"x.tsv","key":"code"}`)}, - "x.tsv": {Data: []byte("code\tv\n1\tb\n2\tc\n")}, - "y.json": {Data: []byte(`{"format":"{v}","rows":"y.tsv","key":"code","parent":"x"}`)}, - "y.tsv": {Data: []byte("code\tv\tx\na\t{/x[2].v}\t1\nb\tz\t2\n")}, - } - g, err := loadDir(dataSource{fsys: fsys}, ".") - if err != nil { - t.Fatalf("loadDir = %v", err) - } - if err := treeBinding(g.children).link(); err != nil { - t.Fatalf("link = %v", err) - } - r, isTemplate := g.children["r"].(*template) - if !isTemplate { - t.Fatalf("r is a %T, want a template", g.children["r"]) - } - if err := surveyRender(newReadFold(), r).check(); err == nil || !strings.Contains(err.Error(), "two rows of") { - t.Errorf("check = %v, want y[a]'s row of x clashing with x[2] named, whatever order the fences run in", err) - } - n, isTemplate := g.children["n"].(*template) - if !isTemplate { - t.Fatalf("n is a %T, want a template", g.children["n"]) - } - if err := surveyRender(newReadFold(), n).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with {/x[2].v}: x[1] and x[2] are two rows of x") { - t.Errorf("check = %v, want the clash below {f} named by the route the root reaches it by", err) - } -} - -func surveyRender(f *readFold, t *template) *readSurvey { - return survey(f.rootReads(t), t.link.drawGroupKey) -} - -func surveyColumns(f *readFold, t *template, columns []string) *readSurvey { - return survey(f.columnReads(t, columns), t.link.drawGroupKey) -} -- 2.52.0 From 2b61433258f53d7636aa1ab72adcf77e60f06eac Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:46:25 +0200 Subject: [PATCH 366/966] Name a read's context its group, route and branches, deleting surveyAt --- doc.go | 2 +- drawfence.go | 40 ++++++++++++++-------------- drawfence_test.go | 2 +- reads.go | 67 ++++++++++++++++++++++++----------------------- reads_test.go | 14 +++++----- table_test.go | 8 +++--- todo.md | 3 --- 7 files changed, 67 insertions(+), 69 deletions(-) diff --git a/doc.go b/doc.go index 38a3e51..a075162 100644 --- a/doc.go +++ b/doc.go @@ -37,7 +37,7 @@ package fejkdata // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // - whole — a read of a table with no selector and no descent, {/city}, landing on -// the `table` itself, so it draws a row apart from every pin: `surveyAt.wholePins`. +// the `table` itself, so it draws a row apart from every pin: `branches.wholePins`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those // compile. The draw fences are `heldCheck`, `drawFence` and `checkFamilies`. diff --git a/drawfence.go b/drawfence.go index 926ed57..bbb2b4b 100644 --- a/drawfence.go +++ b/drawfence.go @@ -34,12 +34,12 @@ func survey(reads []pathRead, group string) *readSurvey { for _, r := range reads { if r.clash != nil { if s.clash == nil { - s.clash = &pinClash{r.at.route, r.a.spelling, r.clash} + s.clash = &pinClash{r.route, r.a.spelling, r.clash} } continue } - if r.at.group == "" { - r.at.group = group + if r.group == "" { + r.group = group } s.reads = append(s.reads, r) } @@ -55,7 +55,7 @@ func (f *drawFence) checkDrawGroup(path string, n node) error { return nil } for _, r := range f.fold.rootReads(t) { - if r.at.group == "" && r.draws() { + if r.group == "" && r.draws() { return nil } } @@ -157,18 +157,18 @@ func (s *readSurvey) check() error { return fmt.Errorf("%s: %w; read the family from a template beside it, or add the value as a column", c.route.spelled(c.spelling), c.err) } sort.SliceStable(s.reads, func(i, j int) bool { - if s.reads[i].at.group != s.reads[j].at.group { - return s.reads[i].at.group < s.reads[j].at.group + if s.reads[i].group != s.reads[j].group { + return s.reads[i].group < s.reads[j].group } return s.reads[i].a.path < s.reads[j].a.path }) for i, level := range s.reads { for _, into := range s.reads[i+1:] { - if into.at.group != level.at.group || alternatives(level.at, into.at) { + if into.group != level.group || alternatives(level.branches, into.branches) { continue } if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { - return overlapError(level.at.route, level.a.spelling, into) + return overlapError(level.route, level.a.spelling, into) } } } @@ -185,7 +185,7 @@ func checkFamilies(reads []pathRead) error { continue } for _, o := range reads[:i] { - if o.tr == nil || o.at.group != r.at.group || alternatives(o.at, r.at) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { + if o.tr == nil || o.group != r.group || alternatives(o.branches, r.branches) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { continue } if err := checkFamilyPair(o, r); err != nil { @@ -204,7 +204,7 @@ func checkFamilyPair(a, b pathRead) error { for _, pair := range [][2]pathRead{{a, b}, {b, a}} { x, y := pair[0], pair[1] if len(x.a.tail) == 0 && len(y.a.tail) > 0 { - return overlapError(x.at.route, x.a.spelling, y) + return overlapError(x.route, x.a.spelling, y) } if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { @@ -213,10 +213,10 @@ func checkFamilyPair(a, b pathRead) error { tail = append([]string{x.tr.headTable.segment}, tail...) } return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) + x.route.spelled(x.a.spelling), drawn.segment, y.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) } return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.at.route.spelled(x.a.spelling), drawn.segment, y.at.route.spelled(y.a.spelling)) + x.route.spelled(x.a.spelling), drawn.segment, y.route.spelled(y.a.spelling)) } } return nil @@ -232,7 +232,7 @@ func replayPairs(reads []pathRead) error { continue } for _, o := range reads[i+1:] { - if o.tr == nil || o.at.group != r.at.group || alternatives(r.at, o.at) { + if o.tr == nil || o.group != r.group || alternatives(r.branches, o.branches) { continue } d := r.tr.pins.clone() @@ -245,24 +245,24 @@ func replayPairs(reads []pathRead) error { } func conflict(r pathRead, err error) error { - return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.at.route.spelled(r.a.spelling), err) + return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.route.spelled(r.a.spelling), err) } // alternatives reports whether two reads never render together: one read renders one row of a // table, so reads under different rows the walks pinned, or under different rows of one whole // draw, never meet. -func alternatives(a, b surveyAt) bool { +func alternatives(a, b branches) bool { return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) } func overlapError(route surveyRoute, ref string, into pathRead) error { apart := "" switch { - case route.spelling == into.at.route.spelling: - case isRef(into.at.route.label): - apart = fmt.Sprintf(", or move {%s} into a field with a drawGroup", into.at.route.label) + case route.spelling == into.route.spelling: + case isRef(into.route.label): + apart = fmt.Sprintf(", or move {%s} into a field with a drawGroup", into.route.label) default: - apart = fmt.Sprintf(", or give %s a drawGroup", into.at.route.spelling) + apart = fmt.Sprintf(", or give %s a drawGroup", into.route.spelling) } - return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead%s", route.spelled(ref), into.at.route.spelled(into.a.spelling), apart) + return fmt.Errorf("%s renders its own draw of what %s reads a path through; name the fields you want instead%s", route.spelled(ref), into.route.spelled(into.a.spelling), apart) } diff --git a/drawfence_test.go b/drawfence_test.go index bed94b0..4916102 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -153,7 +153,7 @@ func wantPairsFindAll(t *testing.T, label string, reads []pathRead) int { // eachRenderSet calls fn with every maximal set of reads that render together, by Bron–Kerbosch. func eachRenderSet(reads []pathRead, fn func([]pathRead)) { - together := func(a, b pathRead) bool { return a.at.group == b.at.group && !alternatives(a.at, b.at) } + together := func(a, b pathRead) bool { return a.group == b.group && !alternatives(a.branches, b.branches) } var grow func(set, candidates, excluded []pathRead) grow = func(set, candidates, excluded []pathRead) { if len(candidates) == 0 && len(excluded) == 0 { diff --git a/reads.go b/reads.go index d56e33d..89bdb97 100644 --- a/reads.go +++ b/reads.go @@ -2,25 +2,26 @@ package fejkdata import "fmt" -// surveyAt is where a read sits in its render: the draw group it draws in ("" until the -// root names one), how the render's root reached it, the rows pinned above it, and the rows -// of whole draws whose cells it sits in. -type surveyAt struct { - group string - route surveyRoute +// surveyRoute is how a render reaches a read: as its author spells it, and the root edge's label. +type surveyRoute struct{ spelling, label string } + +// branches is the rows a read sits under: those pinned above it, and those of whole draws whose +// cells it sits in. +type branches struct { pins pinSet wholePins pinSet } -// surveyRoute is how a render reaches a read: as its author spells it, and the root edge's label. -type surveyRoute struct{ spelling, label string } - -// pathRead is one reference a render reads: a path, or a bare reference with no tail. +// pathRead is one reference a render reads, a path or a bare reference with no tail: in its draw +// group ("" until the root names one), by the route the render's root reached it, under its +// branches. type pathRead struct { - at surveyAt - a arm - tr *tableRead // set where the reference names a table - clash error // set where its selectors clash with a row its route pinned + group string + route surveyRoute + branches branches + a arm + tr *tableRead // set where the reference names a table + clash error // set where its selectors clash with a row its route pinned } type readKey struct { @@ -79,21 +80,21 @@ func (f *readFold) reads(n node) []pathRead { } if n.link.drawGroupKey != "" { for i := range out { - if out[i].at.group == "" { - out[i].at.group = n.link.drawGroupKey + if out[i].group == "" { + out[i].group = n.link.drawGroupKey } } } case *table: - out = f.rowReads(n, func(p *surveyAt, row int) bool { - p.wholePins = p.wholePins.clone() - p.wholePins.add(n, row) + out = f.rowReads(n, func(b *branches, row int) bool { + b.wholePins = b.wholePins.clone() + b.wholePins.add(n, row) return true }) case *tableRow: - out = f.rowReads(n.t, func(p *surveyAt, row int) bool { return p.enter(n.t, row) }) + out = f.rowReads(n.t, func(b *branches, row int) bool { return b.enter(n.t, row) }) case *tableColumn: - out = f.cellReads(n, func(p *surveyAt, row int) bool { return p.enter(n.t, row) }) + out = f.cellReads(n, func(b *branches, row int) bool { return b.enter(n.t, row) }) } out = distinct(out) f.memo[n] = out @@ -106,7 +107,7 @@ func distinct(reads []pathRead) []pathRead { seen := make(map[readKey]bool, len(reads)) out := reads[:0:0] for _, r := range reads { - k := readKey{r.at.group, r.a.path, r.at.pins.key(), r.at.wholePins.key(), r.clash != nil} + k := readKey{r.group, r.a.path, r.branches.pins.key(), r.branches.wholePins.key(), r.clash != nil} if !seen[k] { seen[k] = true out = append(out, r) @@ -123,7 +124,7 @@ func (f *readFold) viaEdge(n node, e renderEdge, route surveyRoute) []pathRead { } a := e.read tr := tableReadOf(n.(*template).head(a.head), a, e.to) - out := []pathRead{{at: surveyAt{route: route}, a: a, tr: tr}} + out := []pathRead{{route: route, a: a, tr: tr}} below := f.reads(e.to) if tr != nil { below = pinBelow(below, tr) @@ -137,7 +138,7 @@ func (f *readFold) viaEdge(n node, e renderEdge, route surveyRoute) []pathRead { func pinBelow(reads []pathRead, tr *tableRead) []pathRead { kept := reads[:0:0] for _, r := range reads { - pins := r.at.pins.clone() + pins := r.branches.pins.clone() if err := tr.replay(&pins); err != nil { continue } @@ -145,7 +146,7 @@ func pinBelow(reads []pathRead, tr *tableRead) []pathRead { check := pins.clone() r.clash = r.tr.replay(&check) } - r.at.pins = pins + r.branches.pins = pins kept = append(kept, r) } return kept @@ -154,7 +155,7 @@ func pinBelow(reads []pathRead, tr *tableRead) []pathRead { // rowReads is what the rows of t read: the cells of the columns its format renders, each tagged // with its row, and the format's other reads, which every row shares. A format reaching its own // cells through another category is checkOwnFamily's refusal, which runs first. -func (f *readFold) rowReads(t *table, tag func(*surveyAt, int) bool) []pathRead { +func (f *readFold) rowReads(t *table, tag func(*branches, int) bool) []pathRead { var out []pathRead for _, e := range renderEdges(t.formatTemplate) { if c, isColumn := e.to.(*tableColumn); isColumn && !e.readsRef() { @@ -167,7 +168,7 @@ func (f *readFold) rowReads(t *table, tag func(*surveyAt, int) bool) []pathRead } // cellReads is what the cells of column c read, each tagged with its row. -func (f *readFold) cellReads(c *tableColumn, tag func(*surveyAt, int) bool) []pathRead { +func (f *readFold) cellReads(c *tableColumn, tag func(*branches, int) bool) []pathRead { var out []pathRead for r := 0; r < c.t.rowCount(); r++ { cell := c.t.cellTemplate(r, c.i) @@ -175,7 +176,7 @@ func (f *readFold) cellReads(c *tableColumn, tag func(*surveyAt, int) bool) []pa continue } for _, read := range f.reads(cell) { - if tag(&read.at, r) { + if tag(&read.branches, r) { out = append(out, read) } } @@ -184,11 +185,11 @@ func (f *readFold) cellReads(c *tableColumn, tag func(*surveyAt, int) bool) []pa } // enter pins row r of t above a read, or reports that the rows already pinned keep it out. -func (at *surveyAt) enter(t *table, r int) bool { - if at.pins.clash(t, r) != nil { +func (b *branches) enter(t *table, r int) bool { + if b.pins.clash(t, r) != nil { return false } - at.pins = at.pins.entered(t, r) + b.pins = b.pins.entered(t, r) return true } @@ -199,8 +200,8 @@ func routed(reads []pathRead, route surveyRoute) []pathRead { } out := make([]pathRead, len(reads)) for i, r := range reads { - if r.at.route == (surveyRoute{}) { - r.at.route = route + if r.route == (surveyRoute{}) { + r.route = route } out[i] = r } diff --git a/reads_test.go b/reads_test.go index 3b33c7e..2d0ffe7 100644 --- a/reads_test.go +++ b/reads_test.go @@ -39,9 +39,9 @@ func wantGathered(t *testing.T, label string, gathered []pathRead, renderWith fu if !isRef(a.head) { return } - r := pathRead{at: surveyAt{group: group}, a: a} + r := pathRead{group: group, a: a} if row.t != nil { - r.at.pins.add(row.t, row.index) + r.branches.pins.add(row.t, row.index) } if k := (traceKey{group, a.path, row}); !seen[k] { seen[k] = true @@ -53,7 +53,7 @@ func wantGathered(t *testing.T, label string, gathered []pathRead, renderWith fu } for _, r := range reads { if !gathers(gathered, r) { - t.Errorf("%s: the render read {%s} in draw group %q from %s, and readSurvey gathered no such read", label, r.a.spelling, r.at.group, spellPins(&r.at.pins)) + t.Errorf("%s: the render read {%s} in draw group %q from %s, and the fold gathered no such read", label, r.a.spelling, r.group, spellPins(&r.branches.pins)) } } return len(reads) @@ -61,7 +61,7 @@ func wantGathered(t *testing.T, label string, gathered []pathRead, renderWith fu func gathers(gathered []pathRead, r pathRead) bool { for _, g := range gathered { - if g.at.group == r.at.group && g.a.path == r.a.path && !g.at.pins.differs(&r.at.pins) && !g.at.wholePins.differs(&r.at.pins) { + if g.group == r.group && g.a.path == r.a.path && !g.branches.pins.differs(&r.branches.pins) && !g.branches.wholePins.differs(&r.branches.pins) { return true } } @@ -106,7 +106,7 @@ func renderRecordRoot(s *generatorState, t *template, columns []Column, trace re renderRecord(s, t, columns, sc) } -func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { +func TestReadFoldWalksACellByEveryRoute(t *testing.T) { fsys := fstest.MapFS{ "g.json": {Data: []byte(`{"format":"{/x.v}","drawGroup":"g2"}`)}, "r.json": {Data: []byte(`"{/g} {/x}"`)}, @@ -130,8 +130,8 @@ func TestReadSurveyWalksACellByEveryRoute(t *testing.T) { if read.a.path != "/z" { continue } - pinned = pinned || read.at.pins.used > 0 - whole = whole || read.at.wholePins.used > 0 + pinned = pinned || read.branches.pins.used > 0 + whole = whole || read.branches.wholePins.used > 0 } if !pinned || !whole { t.Errorf("{/z} gathered under a pinned row %v and a whole read's row %v, want both: {/g} reaches x's cells from the root, and {/x} renders them whole", pinned, whole) diff --git a/table_test.go b/table_test.go index 9cce34e..779c127 100644 --- a/table_test.go +++ b/table_test.go @@ -1005,7 +1005,7 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rowCount(); row++ { - whole := surveyAt{} + whole := branches{} whole.wholePins.add(entered, row) s := none.entered(entered, row) for _, cand := range tables { @@ -1018,14 +1018,14 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for cr := 0; cr < cand.rowCount(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil - candAt := surveyAt{pins: none.entered(cand, cr)} - if got := alternatives(surveyAt{pins: s}, candAt); got != refused { + candAt := branches{pins: none.entered(cand, cr)} + if got := alternatives(branches{pins: s}, candAt); got != refused { t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } if alternatives(whole, candAt) || alternatives(candAt, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } - other := surveyAt{} + other := branches{} other.wholePins.add(cand, cr) if want := cand == entered && cr != row; alternatives(whole, other) != want { t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) diff --git a/todo.md b/todo.md index 59efc5e..88c46b7 100644 --- a/todo.md +++ b/todo.md @@ -36,9 +36,6 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Move the fold to `reads.go` and the verdicts to `drawfence.go`, deleting - `readsurvey.go`; name a read's context its `group`, `route` and `branches`, deleting - `surveyAt`, and let the README's Layout and `doc.go` follow. - Name what a read sits under `branches`, the rows pinned with their ancestors and the rows of whole draws without, both load-bearing, and fold `check`, `checkFamilies`, `replayPairs`, `checkFamilyPair` and `alternatives` into one pairwise loop over -- 2.52.0 From 2306834995f079a65242af7f601457fe746ecb4f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 29 Sep 2026 21:49:37 +0200 Subject: [PATCH 367/966] Name the fold's route type readRoute, and reword the next item to what it still asks --- drawfence.go | 4 ++-- reads.go | 24 ++++++++++++------------ todo.md | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/drawfence.go b/drawfence.go index bbb2b4b..914d6d2 100644 --- a/drawfence.go +++ b/drawfence.go @@ -22,7 +22,7 @@ type readSurvey struct { // pinClash is the first read whose selectors clash with a row its route pinned. type pinClash struct { - route surveyRoute + route readRoute spelling string err error } @@ -255,7 +255,7 @@ func alternatives(a, b branches) bool { return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) } -func overlapError(route surveyRoute, ref string, into pathRead) error { +func overlapError(route readRoute, ref string, into pathRead) error { apart := "" switch { case route.spelling == into.route.spelling: diff --git a/reads.go b/reads.go index 89bdb97..a5aef4c 100644 --- a/reads.go +++ b/reads.go @@ -2,8 +2,8 @@ package fejkdata import "fmt" -// surveyRoute is how a render reaches a read: as its author spells it, and the root edge's label. -type surveyRoute struct{ spelling, label string } +// readRoute is how a render reaches a read: as its author spells it, and the root edge's label. +type readRoute struct{ spelling, label string } // branches is the rows a read sits under: those pinned above it, and those of whole draws whose // cells it sits in. @@ -17,7 +17,7 @@ type branches struct { // branches. type pathRead struct { group string - route surveyRoute + route readRoute branches branches a arm tr *tableRead // set where the reference names a table @@ -44,7 +44,7 @@ func newReadFold() *readFold { return &readFold{memo: map[node][]pathRead{}} } func (f *readFold) rootReads(t *template) []pathRead { var reads []pathRead for _, e := range renderEdges(t) { - reads = append(reads, f.viaEdge(t, e, surveyRoute{e.reached(), e.label})...) + reads = append(reads, f.viaEdge(t, e, readRoute{e.reached(), e.label})...) } return reads } @@ -53,7 +53,7 @@ func (f *readFold) rootReads(t *template) []pathRead { func (f *readFold) columnReads(t *template, columns []string) []pathRead { var reads []pathRead for _, name := range columns { - reads = append(reads, routed(f.reads(t.fields[name]), surveyRoute{spelling: fmt.Sprintf("column %q", name)})...) + reads = append(reads, routed(f.reads(t.fields[name]), readRoute{spelling: fmt.Sprintf("column %q", name)})...) } return reads } @@ -76,7 +76,7 @@ func (f *readFold) reads(n node) []pathRead { break } for _, e := range renderEdges(n) { - out = append(out, f.viaEdge(n, e, surveyRoute{})...) + out = append(out, f.viaEdge(n, e, readRoute{})...) } if n.link.drawGroupKey != "" { for i := range out { @@ -118,7 +118,7 @@ func distinct(reads []pathRead) []pathRead { // viaEdge is what rendering e from n reads: the reference e reads, then what its leaf renders // under the rows that reference pins. -func (f *readFold) viaEdge(n node, e renderEdge, route surveyRoute) []pathRead { +func (f *readFold) viaEdge(n node, e renderEdge, route readRoute) []pathRead { if !e.readsRef() { return routed(f.reads(e.to), route) } @@ -162,7 +162,7 @@ func (f *readFold) rowReads(t *table, tag func(*branches, int) bool) []pathRead out = append(out, f.cellReads(c, tag)...) continue } - out = append(out, f.viaEdge(t.formatTemplate, e, surveyRoute{})...) + out = append(out, f.viaEdge(t.formatTemplate, e, readRoute{})...) } return out } @@ -194,13 +194,13 @@ func (b *branches) enter(t *table, r int) bool { } // routed names the route the render's root reaches reads by, where none is named yet. -func routed(reads []pathRead, route surveyRoute) []pathRead { - if route == (surveyRoute{}) { +func routed(reads []pathRead, route readRoute) []pathRead { + if route == (readRoute{}) { return reads } out := make([]pathRead, len(reads)) for i, r := range reads { - if r.route == (surveyRoute{}) { + if r.route == (readRoute{}) { r.route = route } out[i] = r @@ -210,7 +210,7 @@ func routed(reads []pathRead, route surveyRoute) []pathRead { // spelled names the route, and the reference it reaches a read by where its root edge is not that // reference. -func (r surveyRoute) spelled(ref string) string { +func (r readRoute) spelled(ref string) string { if ref == "" || ref == r.label { return r.spelling } diff --git a/todo.md b/todo.md index 88c46b7..069b245 100644 --- a/todo.md +++ b/todo.md @@ -36,7 +36,7 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Name what a read sits under `branches`, the rows pinned with their ancestors and the +- Say at `branches` what its two sets hold, the rows pinned with their ancestors and the rows of whole draws without, both load-bearing, and fold `check`, `checkFamilies`, `replayPairs`, `checkFamilyPair` and `alternatives` into one pairwise loop over `coRender` and `conflict`. Report a pin clash lowest by route and spelling, so which of -- 2.52.0 From 4cba01fc7a052535e556363efd148d8e0c634c2d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 07:53:07 +0200 Subject: [PATCH 368/966] Test the draw fence's pairs through check and coRender, and that the lowest route's pin clash is named --- drawfence_test.go | 23 +++++++++++++++-------- table_test.go | 20 ++++++++++---------- 2 files changed, 25 insertions(+), 18 deletions(-) diff --git a/drawfence_test.go b/drawfence_test.go index 4916102..113a316 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -54,7 +54,7 @@ func fenceRoots(t *testing.T, f *Generator) []fenceRoot { return append(roots, fenceRoot{"inline record", record.template}) } -func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { +func TestPairsFindWhatAWholeReplayFinds(t *testing.T) { corpus := fenceCorpus(t) roots := fenceRoots(t, corpus) shipped, err := New() @@ -80,7 +80,7 @@ func TestReplayPairsFindsWhatAWholeReplayFinds(t *testing.T) { } } -func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { +func TestPairsFindWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { f, err := New(WithDataPath(writeFiles(t, siblings()))) if err != nil { t.Fatalf("New = %v", err) @@ -111,7 +111,7 @@ func TestReplayPairsFindsWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { for _, c := range reads { triple := []pathRead{a, b, c} wantPairsFindAll(t, a.a.spelling+", "+b.a.spelling+", "+c.a.spelling, triple) - if replayPairs(triple) != nil { + if (&readSurvey{reads: triple}).check() != nil { conflicts++ } } @@ -131,7 +131,7 @@ func wantPairsFindAll(t *testing.T, label string, reads []pathRead) int { tabled = append(tabled, r) } } - pairs := replayPairs(reads) + pairs := (&readSurvey{reads: reads}).check() var whole error compared := 0 eachRenderSet(tabled, func(set []pathRead) { @@ -146,14 +146,13 @@ func wantPairsFindAll(t *testing.T, label string, reads []pathRead) int { } }) if (pairs == nil) != (whole == nil) { - t.Errorf("%s: replayPairs = %v, a whole replay of every read set rendering together = %v", label, pairs, whole) + t.Errorf("%s: check = %v, a whole replay of every read set rendering together = %v", label, pairs, whole) } return compared } // eachRenderSet calls fn with every maximal set of reads that render together, by Bron–Kerbosch. func eachRenderSet(reads []pathRead, fn func([]pathRead)) { - together := func(a, b pathRead) bool { return a.group == b.group && !alternatives(a.branches, b.branches) } var grow func(set, candidates, excluded []pathRead) grow = func(set, candidates, excluded []pathRead) { if len(candidates) == 0 && len(excluded) == 0 { @@ -164,12 +163,12 @@ func eachRenderSet(reads []pathRead, fn func([]pathRead)) { r := candidates[0] var nextC, nextX []pathRead for _, c := range candidates[1:] { - if together(r, c) { + if coRender(r, c) { nextC = append(nextC, c) } } for _, x := range excluded { - if together(r, x) { + if coRender(r, x) { nextX = append(nextX, x) } } @@ -183,6 +182,7 @@ func eachRenderSet(reads []pathRead, fn func([]pathRead)) { func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { fsys := fstest.MapFS{ "r.json": {Data: []byte(`"{/y[a].v}"`)}, + "m.json": {Data: []byte(`{"format":"{g}{f}","f":"{/y[a].v}","g":"{/y[a].v}"}`)}, "n.json": {Data: []byte(`{"format":"{f}","f":"{/y[a].v}"}`)}, "x.json": {Data: []byte(`{"format":"{v}","rows":"x.tsv","key":"code"}`)}, "x.tsv": {Data: []byte("code\tv\n1\tb\n2\tc\n")}, @@ -210,6 +210,13 @@ func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { if err := surveyRender(newReadFold(), n).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with {/x[2].v}: x[1] and x[2] are two rows of x") { t.Errorf("check = %v, want the clash below {f} named by the route the root reaches it by", err) } + m, isTemplate := g.children["m"].(*template) + if !isTemplate { + t.Fatalf("m is a %T, want a template", g.children["m"]) + } + if err := surveyRender(newReadFold(), m).check(); err == nil || !strings.HasPrefix(err.Error(), "{f} with") { + t.Errorf("check = %v, want the clash of the lowest route named, {f}, whichever the walk reaches first", err) + } } func surveyRender(f *readFold, t *template) *readSurvey { diff --git a/table_test.go b/table_test.go index 779c127..1aca9ec 100644 --- a/table_test.go +++ b/table_test.go @@ -1005,8 +1005,8 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { var none pinSet for _, entered := range tables { for row := 0; row < entered.rowCount(); row++ { - whole := branches{} - whole.wholePins.add(entered, row) + whole := pathRead{} + whole.branches.wholePins.add(entered, row) s := none.entered(entered, row) for _, cand := range tables { for i := 0; i < 20; i++ { @@ -1018,17 +1018,17 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { for cr := 0; cr < cand.rowCount(); cr++ { beside := s.clone() refused := beside.pinRow(cand, cr) != nil - candAt := branches{pins: none.entered(cand, cr)} - if got := alternatives(branches{pins: s}, candAt); got != refused { - t.Errorf("alternatives(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) + candAt := pathRead{branches: branches{pins: none.entered(cand, cr)}} + if got := coRender(pathRead{branches: branches{pins: s}}, candAt); got == refused { + t.Errorf("coRender(%s, %s) = %v, but pinning both refuses = %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), got, refused) } - if alternatives(whole, candAt) || alternatives(candAt, whole) { + if !coRender(whole, candAt) || !coRender(candAt, whole) { t.Errorf("%s rendered whole is an alternative to %s pinned, though the whole draw ignores the pin", entered.selectorSpelling(row), cand.selectorSpelling(cr)) } - other := branches{} - other.wholePins.add(cand, cr) - if want := cand == entered && cr != row; alternatives(whole, other) != want { - t.Errorf("alternatives(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), !want, want) + other := pathRead{} + other.branches.wholePins.add(cand, cr) + if want := cand == entered && cr != row; coRender(whole, other) == want { + t.Errorf("coRender(%s, %s), both rendered whole, = %v, want %v", entered.selectorSpelling(row), cand.selectorSpelling(cr), want, !want) } } } -- 2.52.0 From d34513336d5442ba177b83e3493ef1ac369943ba Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 07:55:37 +0200 Subject: [PATCH 369/966] Fold the draw fence's pair checks into one loop over coRender and conflict, naming the lowest pin clash --- doc.go | 2 +- drawfence.go | 148 +++++++++++++++++++++++---------------------------- reads.go | 6 ++- todo.md | 10 +--- 4 files changed, 73 insertions(+), 93 deletions(-) diff --git a/doc.go b/doc.go index a075162..639dfd5 100644 --- a/doc.go +++ b/doc.go @@ -40,4 +40,4 @@ package fejkdata // the `table` itself, so it draws a row apart from every pin: `branches.wholePins`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those -// compile. The draw fences are `heldCheck`, `drawFence` and `checkFamilies`. +// compile. The draw fences are `heldCheck` and `drawFence`. diff --git a/drawfence.go b/drawfence.go index 914d6d2..5702944 100644 --- a/drawfence.go +++ b/drawfence.go @@ -20,21 +20,30 @@ type readSurvey struct { clash *pinClash } -// pinClash is the first read whose selectors clash with a row its route pinned. +// pinClash is the read, lowest by how it is named, whose selectors clash with a row its route pinned. type pinClash struct { route readRoute spelling string err error } +func (c *pinClash) named() string { return c.route.spelled(c.spelling) } + +func (c *pinClash) before(o *pinClash) bool { + if c.named() != o.named() { + return c.named() < o.named() + } + return c.err.Error() < o.err.Error() +} + // survey is the fold's result as a render in group: every read not inside a nested draw group // draws in it, and one read gathered by several routes is kept once. func survey(reads []pathRead, group string) *readSurvey { s := &readSurvey{} for _, r := range reads { if r.clash != nil { - if s.clash == nil { - s.clash = &pinClash{r.route, r.a.spelling, r.clash} + if c := (&pinClash{r.route, r.a.spelling, r.clash}); s.clash == nil || c.before(s.clash) { + s.clash = c } continue } @@ -150,11 +159,10 @@ func repeats(n node) bool { } // check refuses what one draw per reference path cannot answer for: a read selecting a row its route -// pinned another of, a read of a level beside a path another read takes into it, and two reads of one -// table family that select different rows. +// pinned another of, and any two reads that render together yet conflict. func (s *readSurvey) check() error { if c := s.clash; c != nil { - return fmt.Errorf("%s: %w; read the family from a template beside it, or add the value as a column", c.route.spelled(c.spelling), c.err) + return fmt.Errorf("%s: %w; read the family from a template beside it, or add the value as a column", c.named(), c.err) } sort.SliceStable(s.reads, func(i, j int) bool { if s.reads[i].group != s.reads[j].group { @@ -162,97 +170,73 @@ func (s *readSurvey) check() error { } return s.reads[i].a.path < s.reads[j].a.path }) - for i, level := range s.reads { - for _, into := range s.reads[i+1:] { - if into.group != level.group || alternatives(level.branches, into.branches) { + for i, a := range s.reads { + for _, b := range s.reads[i+1:] { + if !coRender(a, b) { continue } - if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.landsRow) { - return overlapError(level.route, level.a.spelling, into) - } - } - } - return checkFamilies(s.reads) -} - -// checkFamilies refuses reads of one table family in one draw group that cannot -// read one consistent draw: a table rendered whole beside a path into the family, -// a table one read draws that another pins, and two reads pinning different rows. -// The reads come sorted by group and path. -func checkFamilies(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[:i] { - if o.tr == nil || o.group != r.group || alternatives(o.branches, r.branches) || o.tr.headTable.familyRoot() != r.tr.headTable.familyRoot() { - continue - } - if err := checkFamilyPair(o, r); err != nil { + if err := conflict(a, b); err != nil { return err } } } - return replayPairs(reads) + return nil } -// checkFamilyPair refuses a table read whole beside a path into its family, and a -// table one read draws that the other pins, since which token renders first would -// then decide the row. -// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held -func checkFamilyPair(a, b pathRead) error { +// coRender reports whether two reads can render together: in one draw group, and under no two +// rows of one table, since one read renders one row of it. +func coRender(a, b pathRead) bool { + return a.group == b.group && !a.branches.pins.differs(&b.branches.pins) && !a.branches.wholePins.differs(&b.branches.wholePins) +} + +// conflict refuses two reads rendering together, a before b by path, that one draw per reference +// path cannot answer for. Pairs find every conflict a replay of all the reads would: a row clashes +// with one pinned table, and the read that pinned it holds that pin itself, with its ancestors. +func conflict(a, b pathRead) error { for _, pair := range [][2]pathRead{{a, b}, {b, a}} { - x, y := pair[0], pair[1] - if len(x.a.tail) == 0 && len(y.a.tail) > 0 { - return overlapError(x.route, x.a.spelling, y) - } - if drawn := x.tr.drawnOf(&y.tr.pins); drawn != nil { - if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { - tail := x.a.tail - if s.t != x.tr.headTable { - tail = append([]string{x.tr.headTable.segment}, tail...) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", - x.route.spelled(x.a.spelling), drawn.segment, y.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) - } - return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", - x.route.spelled(x.a.spelling), drawn.segment, y.route.spelled(y.a.spelling)) + if err := drawsApart(pair[0], pair[1]); err != nil { + return err } } + if a.tr == nil || b.tr == nil { + return nil + } + d := a.tr.pins.clone() + if err := b.tr.replay(&d); err != nil { + return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", b.route.spelled(b.a.spelling), err) + } return nil } -// replayPairs replays, of every two reads that can render together, the later into the -// earlier's pins. Pairs find every conflict a full replay would: clash judges a row against one -// pinned table, and the read that pinned it holds that pin itself, since a read's pins carry -// its rows' ancestors, so the pair of those two reads clashes the same way. -func replayPairs(reads []pathRead) error { - for i, r := range reads { - if r.tr == nil { - continue - } - for _, o := range reads[i+1:] { - if o.tr == nil || o.group != r.group || alternatives(r.branches, o.branches) { - continue - } - d := r.tr.pins.clone() - if err := o.tr.replay(&d); err != nil { - return conflict(o, err) - } - } +// drawsApart refuses x drawing apart from what y reads: a level beside a path y takes into it, a +// table x reads whole beside a path y takes into its family, and a table x draws that y pins, since +// which token renders first would then decide the row. A level landing on a row renders the row its +// selectors pin, the one a path into it reads too. +// docs/decisions.md#a-bare-reference-draws-each-time-a-reference-path-is-held +func drawsApart(x, y pathRead) error { + if strings.HasPrefix(y.a.path, x.a.path+".") && !(x.tr != nil && x.tr.landsRow) { + return overlapError(x.route, x.a.spelling, y) } - return nil -} - -func conflict(r pathRead, err error) error { - return fmt.Errorf("%s: %w; select the same rows in every path into the family, or draw them apart with a drawGroup", r.route.spelled(r.a.spelling), err) -} - -// alternatives reports whether two reads never render together: one read renders one row of a -// table, so reads under different rows the walks pinned, or under different rows of one whole -// draw, never meet. -func alternatives(a, b branches) bool { - return a.pins.differs(&b.pins) || a.wholePins.differs(&b.wholePins) + if x.tr == nil || y.tr == nil || x.tr.headTable.familyRoot() != y.tr.headTable.familyRoot() { + return nil + } + if len(x.a.tail) == 0 && len(y.a.tail) > 0 { + return overlapError(x.route, x.a.spelling, y) + } + drawn := x.tr.drawnOf(&y.tr.pins) + if drawn == nil { + return nil + } + if s, ok := y.tr.selected(x.tr.headTable); ok && len(x.tr.sels) == 0 { + tail := x.a.tail + if s.t != x.tr.headTable { + tail = append([]string{x.tr.headTable.segment}, tail...) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; write {%s.%s}, or draw them apart with a drawGroup", + x.route.spelled(x.a.spelling), drawn.segment, y.route.spelled(y.a.spelling), s.spelling, joinSegments(tail)) + } + return fmt.Errorf("%s draws %s, which %s selects a row of; select that row in both, or draw them apart with a drawGroup", + x.route.spelled(x.a.spelling), drawn.segment, y.route.spelled(y.a.spelling)) } func overlapError(route readRoute, ref string, into pathRead) error { diff --git a/reads.go b/reads.go index a5aef4c..00a340c 100644 --- a/reads.go +++ b/reads.go @@ -5,8 +5,10 @@ import "fmt" // readRoute is how a render reaches a read: as its author spells it, and the root edge's label. type readRoute struct{ spelling, label string } -// branches is the rows a read sits under: those pinned above it, and those of whole draws whose -// cells it sits in. +// branches is the rows a read sits under, where two reads under different rows of one table never +// render together. pins holds the rows pinned above it with their ancestors, which a pair of reads +// needs to find what a replay of all of them does; wholePins the rows of whole draws whose cells it +// sits in, without ancestors, since each whole draw draws apart from every other. type branches struct { pins pinSet wholePins pinSet diff --git a/todo.md b/todo.md index 069b245..d8ab305 100644 --- a/todo.md +++ b/todo.md @@ -36,12 +36,6 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic This round, in order: -- Say at `branches` what its two sets hold, the rows pinned with their ancestors and the - rows of whole draws without, both load-bearing, and fold `check`, `checkFamilies`, - `replayPairs`, `checkFamilyPair` and `alternatives` into one pairwise loop over - `coRender` and `conflict`. Report a pin clash lowest by route and spelling, so which of - two is named stops following the walk, and say at `conflict` why a level landing on a - row may sit beside a path into it. - Run the nine-seat panel as a checkpoint, and file its scores above: where Locality has not risen from 5.2, stop and bring the scores to the maintainer before the path steps. - Compile each path once at link into steps holding no pointer, with its leaves and @@ -310,8 +304,8 @@ by a `data-import/` script, as README goal 12 asks. error names the category and the two columns but no row, so the author greps a register-sized file for the cell — and "name the fields you want instead" rewrites a cell that is not the one to fix. -- Spell a table one way across the family errors: `checkFamilyPair` names - `drawn.category`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one +- Spell a table one way across the family errors: `drawsApart` names + `drawn.segment`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. - Let a table column carry a `datatype`, so `--format json` writes -- 2.52.0 From b9f7ba04ead44e536142e574614777cf562169de Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 08:00:46 +0200 Subject: [PATCH 370/966] File the decision on which of several load mistakes an error names --- todo.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/todo.md b/todo.md index d8ab305..f748ed9 100644 --- a/todo.md +++ b/todo.md @@ -297,6 +297,10 @@ by a `data-import/` script, as README goal 12 asks. together, where the fence counts them alternatives and compares neither. Have the error name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`, as the README's rule on a rejected spelling asks. +- Decide, in "An error is a contract by what it names", which of several mistakes a + load names, and link the entry to goal 5, which it serves and does not name. The + fence names the lowest by route and path, so a data edit can move which one is named; + proposed: fixed within a version, as seeded output is, and free to change in any release. - Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`: with several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. -- 2.52.0 From 49e47d610377477991dc90c28c97416aa5968711 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 11:03:51 +0200 Subject: [PATCH 371/966] Rewrite goal 5 as an aim, and plan a load reporting every mistake at once --- AGENTS.md | 1 + README.md | 10 ++++++---- docs/decisions.md | 12 ++++++++++-- todo.md | 7 +++---- 4 files changed, 20 insertions(+), 10 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f6c7ca9..7bd10f9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,6 +46,7 @@ In [docs/decisions.md](docs/decisions.md): - A change to what exists is a major; a minor only adds - Seeded output is promised within one version - An error is a contract by what it names, not its bytes +- A load reports every mistake at once - Formats are checked after all data is loaded, so their errors name the category - Raising the lowest supported Go is a major - The format check runs on the latest Go only diff --git a/README.md b/README.md index 0bb4537..80b7546 100644 --- a/README.md +++ b/README.md @@ -291,6 +291,11 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el | `WithDataFS(fsys)` | layer an `fs.FS`, such as your own `embed.FS` | | `WithoutShippedData()` | load only what you give | +`New` refuses a mistake in the data and `NewTemplate` one in the template, so on a +loaded generator `Fake` fails only for a path that names nothing or could name two +things, `FakeStruct` only for a non-struct argument or a type its tags do not describe, +with the same error every call, and `Template.Fake` cannot fail at all. + A `*Record` carries its columns via `Columns()` — each a `Column` of `Name`, `DataType`, rendered `Value` and `Null` — and serializes them with `JSON()` (one object), `CSVHeader()`/`CSVLine()`, or `SQLInsert(table)` — the shapes the @@ -837,10 +842,7 @@ the Development section below, and who ships a register the five above then draw Where two writers could expect different outputs, the template is a load error naming a spelling for each. One spelling per result; the wrong one is a load error naming the right one. -5. **Every mistake is a load error** — `New` rejects the data and `NewTemplate` - the inline template; on a loaded generator `Fake` fails only for a path that - names nothing or could name two things, `FakeStruct` only for a non-struct argument or a type its tags do not - describe, with the same error every call, and `Template.Fake` cannot fail at all. +5. **Errors are up front and easy to understand.** 6. **Zero to a value in one command** — `go install`, then `fejkdata sv_SE.person`: no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position; the first custom template needs no escape and no option. diff --git a/docs/decisions.md b/docs/decisions.md index 7cdcb04..bc66c4d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -148,8 +148,16 @@ this repo's own are. 2026-09-16, Lilleman auf Larv. -A script branches on the exit code and reads the named path or spelling, so those hold; -wording improves in a minor. +Goal 5. A script branches on the exit code and reads the named path or spelling, so +those hold; wording improves in a minor. + +## A load reports every mistake at once + +2026-09-30, Lilleman auf Larv. Goal 5. An author learns everything wrong with the data +in one load, not one mistake per run: `New` and `NewTemplate` report every mistake in one +error, ordered by the path each names. A mistake following only from another, such as a +reference into a category that did not compile, is left out, so each one names something +to fix. Valid while a load checks the whole set before its first render. ## Formats are checked after all data is loaded, so their errors name the category diff --git a/todo.md b/todo.md index f748ed9..d73406a 100644 --- a/todo.md +++ b/todo.md @@ -208,6 +208,9 @@ by a `data-import/` script, as README goal 12 asks. ### Library and CLI +- Report every mistake a load finds in one error, as the decision "A load reports every + mistake at once" states: `New` and `NewTemplate` stop at the first, so data holding + two mistakes takes two runs to fix. Split it into sub-items before starting. - Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`: both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` shadows the `rng` interface. @@ -297,10 +300,6 @@ by a `data-import/` script, as README goal 12 asks. together, where the fence counts them alternatives and compares neither. Have the error name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`, as the README's rule on a rejected spelling asks. -- Decide, in "An error is a contract by what it names", which of several mistakes a - load names, and link the entry to goal 5, which it serves and does not name. The - fence names the lowest by route and path, so a data edit can move which one is named; - proposed: fixed within a version, as seeded output is, and free to change in any release. - Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`: with several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. -- 2.52.0 From f5f4812403b2ae57acd03edcfd36cdf404e0ac6b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 11:04:49 +0200 Subject: [PATCH 372/966] Split the Library's failure list, spell up front one way, and name what the item's split decides --- README.md | 12 +++++++----- todo.md | 4 +++- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 80b7546..b6192cc 100644 --- a/README.md +++ b/README.md @@ -291,10 +291,12 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el | `WithDataFS(fsys)` | layer an `fs.FS`, such as your own `embed.FS` | | `WithoutShippedData()` | load only what you give | -`New` refuses a mistake in the data and `NewTemplate` one in the template, so on a -loaded generator `Fake` fails only for a path that names nothing or could name two -things, `FakeStruct` only for a non-struct argument or a type its tags do not describe, -with the same error every call, and `Template.Fake` cannot fail at all. +`New` refuses a mistake in the data, and `NewTemplate` one in the template. On a loaded +generator, each with the same error every call: + +- `Fake` fails only for a path that names nothing or could name two things. +- `FakeStruct` fails only for a non-struct argument or a type its tags do not describe. +- `Template.Fake` cannot fail. A `*Record` carries its columns via `Columns()` — each a `Column` of `Name`, `DataType`, rendered `Value` and `Null` — and serializes them with `JSON()` @@ -852,7 +854,7 @@ the Development section below, and who ships a register the five above then draw 10. **Docs index the grammar** — every syntax feature is a heading; every example runs under test and shows its output; a rule is stated once. 11. **Fast enough to be free** — a value renders in about a microsecond and `New` - parses and validates the whole set once upfront, so generating fixtures stays + parses and validates the whole set once up front, so generating fixtures stays noise against a test's own runtime. 12. **Data is sourced, or on its way there** — a shipped fact, a name, place, code, id or classification, is read from a register or open dataset by a diff --git a/todo.md b/todo.md index d73406a..2357157 100644 --- a/todo.md +++ b/todo.md @@ -210,7 +210,9 @@ by a `data-import/` script, as README goal 12 asks. - Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states: `New` and `NewTemplate` stop at the first, so data holding - two mistakes takes two runs to fix. Split it into sub-items before starting. + two mistakes takes two runs to fix. Split it into sub-items before starting, one of + them the shape a Go caller iterates and the CLI prints, and reword the README's + "`New` refuses a mistake in the data" to every mistake. - Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`: both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` shadows the `rng` interface. -- 2.52.0 From a898df62c12a0809599bd6f2bb2c91013a8c9170 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 11:07:36 +0200 Subject: [PATCH 373/966] Exempt below the floor a repair of documented behaviour, where goal 5 no longer promises one --- AGENTS.md | 2 +- README.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7bd10f9..f834c41 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,7 +11,7 @@ wait for one. - README goal 2 is gated at 7.0 on the `comprehension-panel` skill, and below it nothing else merges — no feature, no category, no data — bar a security fix, a - dependency bump, a repair of a promise goals 3 to 5 make, and the infrastructure + dependency bump, a repair of behaviour the README documents, and the infrastructure the round itself runs on. `todo.md` carries the round: a PR per item it plans, the nine-seat findings at depth 1 or a restructure the maintainer approved, then the nine-seat run again, until the score passes. At or above 7.0 diff --git a/README.md b/README.md index b6192cc..5040d41 100644 --- a/README.md +++ b/README.md @@ -830,8 +830,8 @@ the Development section below, and who ships a register the five above then draw holding the rest, names that tell the truth, and a file that stands with no second document open beside it. A simulated reader panel scores those four, the score is a ratchet no release lowers, and [`AGENTS.md`](AGENTS.md) says what each pull request - owes it. Below the floor it holds back new capability, never a repair of a promise - goals 3 to 5 already make, as long as the round that lifts the score stays planned. + owes it. Below the floor it holds back new capability, never a repair of behaviour + this README already documents, as long as the round that lifts the score stays planned. 3. **Valid by construction** — every value passes the check a current release of its real consumer applies; facts that belong together come from one draw, within a value and across categories. -- 2.52.0 From e54d1c06d6bde8d782c644308b6c2f4da1838ef4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 11:07:59 +0200 Subject: [PATCH 374/966] Pin the same-error promise to the calls that fail, and add an inline template's order to the item --- README.md | 8 +++++--- todo.md | 3 ++- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 5040d41..5ae2e24 100644 --- a/README.md +++ b/README.md @@ -292,10 +292,12 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el | `WithoutShippedData()` | load only what you give | `New` refuses a mistake in the data, and `NewTemplate` one in the template. On a loaded -generator, each with the same error every call: +generator: -- `Fake` fails only for a path that names nothing or could name two things. -- `FakeStruct` fails only for a non-struct argument or a type its tags do not describe. +- `Fake` fails only for a path that names nothing or could name two things, with the + same error every call. +- `FakeStruct` fails only for a non-struct argument or a type its tags do not describe, + with the same error every call. - `Template.Fake` cannot fail. A `*Record` carries its columns via `Columns()` — each a `Column` of `Name`, diff --git a/todo.md b/todo.md index 2357157..859ac9a 100644 --- a/todo.md +++ b/todo.md @@ -211,7 +211,8 @@ by a `data-import/` script, as README goal 12 asks. - Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states: `New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into sub-items before starting, one of - them the shape a Go caller iterates and the CLI prints, and reword the README's + them the shape a Go caller iterates and the CLI prints, and the order of an + inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. - Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`: both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` -- 2.52.0 From 54c7d481444f0845567c826ad03b95f542b5007f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 15:09:53 +0200 Subject: [PATCH 375/966] File the nine-seat checkpoint, which held Locality at 5.2, and stop the round for the maintainer's call --- todo.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 859ac9a..f6b23dc 100644 --- a/todo.md +++ b/todo.md @@ -32,12 +32,19 @@ on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-suffic ranking `readSurvey.walk` and `edge` hardest and the unit they would least modify; 5.8 on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-sufficiency 5.8), every seat's lowest dimension Locality, alone or tied, all seven unit seats ranking -`readSurvey.walk` hardest and the unit they would least modify. +`readSurvey.walk` hardest and the unit they would least modify; 5.7 on 2026-09-30, the +checkpoint (Navigation 6.8, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), every seat's +lowest dimension Locality, alone or tied, all seven unit seats ranking `drawsApart` or +`readFold.reads` hardest and five `drawsApart` the unit they would least modify. This round, in order: -- Run the nine-seat panel as a checkpoint, and file its scores above: where Locality has - not risen from 5.2, stop and bring the scores to the maintainer before the path steps. +- **Take the maintainer's call on the path steps below**: the checkpoint held Locality at + 5.2, so the round stops here until they decide. +- Give `readFold.reads` a `default` that panics, as every other switch over node kinds + does: a new kind contributes no reads and the draw fence passes it silently. +- Correct `rowReads`' "checkOwnFamily's refusal, which runs first": `checkNodeFences` + runs `checkDrawGroup`, which folds through it, before `checkOwnFamily`. - Compile each path once at link into steps holding no pointer, with its leaves and cover, and have `renderEdges`, `heldNodes`, `valueProof` and `tableReadOf` read them, deleting `pathCover`, `coverPath` and `pathLeaves`. Prototyped green, allocations -- 2.52.0 From 3d1b153a9134e7724b0a37ff28d44595d66136d5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 15:50:44 +0200 Subject: [PATCH 376/966] Record the maintainer's call to go on with the path steps --- todo.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index f6b23dc..41a234d 100644 --- a/todo.md +++ b/todo.md @@ -35,12 +35,11 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic `readSurvey.walk` hardest and the unit they would least modify; 5.7 on 2026-09-30, the checkpoint (Navigation 6.8, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), every seat's lowest dimension Locality, alone or tied, all seven unit seats ranking `drawsApart` or -`readFold.reads` hardest and five `drawsApart` the unit they would least modify. +`readFold.reads` hardest and five `drawsApart` the unit they would least modify. The +maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- **Take the maintainer's call on the path steps below**: the checkpoint held Locality at - 5.2, so the round stops here until they decide. - Give `readFold.reads` a `default` that panics, as every other switch over node kinds does: a new kind contributes no reads and the draw fence passes it silently. - Correct `rowReads`' "checkOwnFamily's refusal, which runs first": `checkNodeFences` -- 2.52.0 From 3240f05027fa72b5bcbcc06f8973532b65ee69fc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 16:56:24 +0200 Subject: [PATCH 377/966] Require readFold.reads to panic on a node kind it has no case for --- render_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/render_test.go b/render_test.go index 6bdea1d..c30bdb5 100644 --- a/render_test.go +++ b/render_test.go @@ -111,6 +111,7 @@ func TestNodeSwitchesHandleEveryKind(t *testing.T) { "contained": func(n node) { contained(n) }, "paths": func(n node) { paths(n) }, "prove": func(n node) { (&valueProof{}).prove(n) }, + "reads": func(n node) { newReadFold().reads(n) }, "render": func(n node) { var draws renderDraws render(engine(1).rand, n, renderScope{draws: &draws, row: renderedRow{tbl, 0}}) -- 2.52.0 From e55b9cc89ba9f19f10f32dd028fdea188658ef9f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 16:57:54 +0200 Subject: [PATCH 378/966] Panic in readFold.reads on a node kind it has no case for --- reads.go | 3 +++ todo.md | 2 -- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/reads.go b/reads.go index 00a340c..10890f0 100644 --- a/reads.go +++ b/reads.go @@ -97,6 +97,9 @@ func (f *readFold) reads(n node) []pathRead { out = f.rowReads(n.t, func(b *branches, row int) bool { return b.enter(n.t, row) }) case *tableColumn: out = f.cellReads(n, func(b *branches, row int) bool { return b.enter(n.t, row) }) + case *folder, *nullItem: + default: + panic(internalError("reads has no case for node %T", n)) } out = distinct(out) f.memo[n] = out diff --git a/todo.md b/todo.md index 41a234d..cc227ea 100644 --- a/todo.md +++ b/todo.md @@ -40,8 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Give `readFold.reads` a `default` that panics, as every other switch over node kinds - does: a new kind contributes no reads and the draw fence passes it silently. - Correct `rowReads`' "checkOwnFamily's refusal, which runs first": `checkNodeFences` runs `checkDrawGroup`, which folds through it, before `checkOwnFamily`. - Compile each path once at link into steps holding no pointer, with its leaves and -- 2.52.0 From a7c99dfb09724300cc93de68a0b3811169c13bbb Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 17:01:52 +0200 Subject: [PATCH 379/966] Say rowReads' own-family refusal runs before any survey, not first --- reads.go | 3 ++- todo.md | 2 -- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/reads.go b/reads.go index 10890f0..a2ffdc5 100644 --- a/reads.go +++ b/reads.go @@ -159,7 +159,8 @@ func pinBelow(reads []pathRead, tr *tableRead) []pathRead { // rowReads is what the rows of t read: the cells of the columns its format renders, each tagged // with its row, and the format's other reads, which every row shares. A format reaching its own -// cells through another category is checkOwnFamily's refusal, which runs first. +// cells through another category is checkOwnFamily's refusal, which runs before any survey of +// these reads. func (f *readFold) rowReads(t *table, tag func(*branches, int) bool) []pathRead { var out []pathRead for _, e := range renderEdges(t.formatTemplate) { diff --git a/todo.md b/todo.md index cc227ea..c750647 100644 --- a/todo.md +++ b/todo.md @@ -40,8 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Correct `rowReads`' "checkOwnFamily's refusal, which runs first": `checkNodeFences` - runs `checkDrawGroup`, which folds through it, before `checkOwnFamily`. - Compile each path once at link into steps holding no pointer, with its leaves and cover, and have `renderEdges`, `heldNodes`, `valueProof` and `tableReadOf` read them, deleting `pathCover`, `coverPath` and `pathLeaves`. Prototyped green, allocations -- 2.52.0 From 0f58784a1ba01428fbba272551b925d1f4c06307 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 17:06:17 +0200 Subject: [PATCH 380/966] Require each path's leaves, cover and table steps from compilePath --- path_test.go | 13 ++++++------- table_test.go | 1 + 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/path_test.go b/path_test.go index fde337b..9866f2f 100644 --- a/path_test.go +++ b/path_test.go @@ -9,8 +9,8 @@ import ( func TestPathCheckStopsAtAMissingSegment(t *testing.T) { n := compiled(t, `{"format":"{a}","a":{"format":"{b}","b":"leaf"}}`) - if leaves := pathLeaves(n, []string{"a", "b"}); len(leaves) != 1 { - t.Fatalf("pathLeaves(a.b) = %v, want the one leaf", leaves) + if leaves := compilePath(n, []string{"a", "b"}).leaves; len(leaves) != 1 { + t.Fatalf("compilePath(a.b).leaves = %v, want the one leaf", leaves) } w := &pathCheck{tail: []string{"a", "nope", "deeper"}} _, err := w.run(n) @@ -24,16 +24,15 @@ func TestPathCheckStopsAtAMissingSegment(t *testing.T) { func TestPathCheckChoiceConsumesNoSegment(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) - if leaves := pathLeaves(n, []string{"f"}); len(leaves) != 2 { - t.Fatalf("pathLeaves through a choice = %v, want both variants' f", leaves) + if leaves := compilePath(n, []string{"f"}).leaves; len(leaves) != 2 { + t.Fatalf("compilePath through a choice = %v, want both variants' f", leaves) } } func TestPathCoverStopsAtAChoice(t *testing.T) { n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`) - leaf, err := (&pathCover{into: map[node]bool{}}).walk(n, []string{"f"}) - if err != nil || leaf != n { - t.Fatalf("cover through a choice = %v, %v, want the choice itself", leaf, err) + if cover := compilePath(n, []string{"f"}).cover; cover != n { + t.Fatalf("cover through a choice = %v, want the choice itself", cover) } } diff --git a/table_test.go b/table_test.go index 1aca9ec..1fcdb36 100644 --- a/table_test.go +++ b/table_test.go @@ -1069,6 +1069,7 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { } { a := splitArm(c.path, nil) head := f.root.children[a.head].(*table) + a.steps = compilePath(head, a.tail).steps tr := tableReadOf(head, a, nil) var sels []string for _, s := range tr.sels { -- 2.52.0 From 6b9755b3efcd9440c49a767c3275c93f0330d271 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 17:09:50 +0200 Subject: [PATCH 381/966] Compile each path's steps, leaves and cover once at link, deleting pathCover, coverPath and pathLeaves --- AGENTS.md | 2 +- docs/decisions.md | 5 +- format.go | 3 + graph.go | 15 +---- holdfence.go | 8 +-- path.go | 158 +++++++++++++++++++++++++++------------------- reads.go | 28 ++++++-- reference.go | 23 +++++++ todo.md | 8 +-- valueproof.go | 2 +- 10 files changed, 149 insertions(+), 103 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f834c41..24ee4ad 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -116,4 +116,4 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only -- The four path walks are four loops +- The path walks are separate loops diff --git a/docs/decisions.md b/docs/decisions.md index bc66c4d..2ab0cd1 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -842,11 +842,12 @@ import drops them by name rather than by a rank a regeneration would move. the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. -## The four path walks are four loops +## The path walks are separate loops 2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. -Goals 2 and 11: `pathCheck`, `pathCover`, `pathProbe` and `pathDraw` stay four loops. +Goals 2 and 11: `pathCheck`, `pathProbe` and `pathDraw` stay separate loops; `pathCover` +became the cover `pathCheck` compiles, on 2026-09-30. One loop fails `perf_test.go`: the check's leaves, the probe's selectors and the errors naming a level leak the walker's contents, and Go tracks a struct's fields as one, so the draw's pins, arm and memo leak with them. Held by pointer, a repeat of a reference diff --git a/format.go b/format.go index abc146d..825f52b 100644 --- a/format.go +++ b/format.go @@ -312,6 +312,9 @@ type arm struct { tail []string levels []string // the path at each level the tail passes through, the head first path string // head and tail, the one path every way of writing this read shares + steps []pathStep + leaves []node // every node the path may land on, one per variant it passes + cover node // what holding the path pins: the first choice it passes, else its leaf } func (a arm) isRefPath() bool { diff --git a/graph.go b/graph.go index 4992dc9..88ee6fe 100644 --- a/graph.go +++ b/graph.go @@ -131,11 +131,7 @@ func renderEdges(n node) []renderEdge { case *template: var es []renderEdge add := func(a arm, operand string) { - c := n.head(a.head) - if c == nil { - return - } - for _, leaf := range pathLeaves(c, a.tail) { + for _, leaf := range a.leaves { es = append(es, renderEdge{leaf, a.spelling, a, operand}) } } @@ -165,15 +161,6 @@ func renderEdges(n node) []renderEdge { } } -// pathLeaves lists what a token's dotted tail renders: a choice on the way -// contributes every variant, since any of them may be the one drawn. checkPath has -// already proved the tail resolves in every variant. -func pathLeaves(n node, tail []string) []node { - w := &pathCheck{tail: tail} - _, _ = w.run(n) - return w.leaves -} - type renderCounts map[node]int func (m renderCounts) renderCount(n node) int { diff --git a/holdfence.go b/holdfence.go index a86e9a3..fe19439 100644 --- a/holdfence.go +++ b/holdfence.go @@ -53,7 +53,7 @@ func heldNodes(t *template, name string, readers []reader) map[node]bool { } for _, r := range readers { if r.a.head == name { - coverPath(t.head(name), r.a.tail, held) + cover(r.a.cover, held, false) } } return held @@ -97,12 +97,6 @@ func operandReader(t *template, name string) (call, operand string) { return "", "" } -// coverPath collects what holding one path pins: the first choice level it passes, -// whole, else the leaf it renders; it reads no row. -func coverPath(n node, tail []string, into map[node]bool) { - _, _ = (&pathCover{into: into}).walk(n, tail) -} - // cover collects a level and everything contained in it. A fixed string outside a // choice is left out — it cannot disagree with itself — but inside one each // variant carries its own, so there it counts. diff --git a/path.go b/path.go index 3ce85a2..9659ce8 100644 --- a/path.go +++ b/path.go @@ -186,34 +186,70 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { return r, nil } +// pathStep is one step of a compiled path, taken at the node the steps before it +// reached; at indexes the tail segment the step consumes. A step holds no pointer, so a +// caller's steps stay on its stack. +type pathStep struct { + kind stepKind + at int + name string // stepField: the field; stepSelect: the selector; stepColumn, stepChild: the column or table stepped to + row int // stepSelect: the row the selector names +} + +type stepKind uint8 + +const ( + stepField stepKind = iota + 1 // into a field of the template at this level, a choice there resolved first + stepSelect // pin the selected row of the table + stepDraw // draw a row of the table inside the pins + stepRow // land on the table's row node + stepColumn // into a column of the table + stepChild // into a table linked to the table +) + // pathCheck proves a path resolves whichever way the draws go: every variant of a // choice carries the rest of it, and is walked, a selector names a row inside the // rows selected before it, and no level read carries a repeat or a drawGroup. It -// collects every leaf the path may render; level names the head in its errors. +// compiles the steps a draw takes, every leaf the path may render, and as cover the +// first choice it passes, else the leaf; level names the head in its errors. type pathCheck struct { pins pinSet level string tail []string + steps []pathStep leaves []node + cover node } -func (w *pathCheck) run(n node) (node, error) { return w.walk(n, w.tail) } +func (w *pathCheck) run(n node) (node, error) { + leaf, err := w.walk(n, w.tail) + if err == nil && w.cover == nil { + w.cover = leaf + } + return leaf, err +} + +func (w *pathCheck) emit(st pathStep, tail []string) { + st.at = len(w.tail) - len(tail) + w.steps = append(w.steps, st) +} func (w *pathCheck) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error switch x := n.(type) { case *choice: + if w.cover == nil { + w.cover = x + } return w.walkEvery(x, tail) case *table: var r tableRoute if r, err = x.route(tail, descended); err != nil { return nil, err } - if r.sel != "" { - if err := w.pins.selectRow(x, r.sel); err != nil { - return nil, err - } + if err := w.tableSteps(x, r, tail); err != nil { + return nil, err } n, tail, descended = r.next, r.rest, r.descends continue @@ -228,23 +264,58 @@ func (w *pathCheck) walk(n node, tail []string) (node, error) { if n, err = stepInto(n, tail[0]); err != nil { return nil, err } + w.emit(pathStep{kind: stepField, name: tail[0]}, tail) tail = tail[1:] } w.leaves = append(w.leaves, n) return n, nil } +func (w *pathCheck) tableSteps(t *table, r tableRoute, tail []string) error { + if r.sel != "" { + if err := w.pins.selectRow(t, r.sel); err != nil { + return err + } + row, _ := w.pins.pinned(t) + w.emit(pathStep{kind: stepSelect, name: r.sel, row: row}, tail) + } + if r.draw { + w.emit(pathStep{kind: stepDraw}, tail) + } + switch next := r.next.(type) { + case *table: + if next != t { + w.emit(pathStep{kind: stepChild, name: next.segment}, tail) + } + case *tableRow: + w.emit(pathStep{kind: stepRow}, tail) + case *tableColumn: + w.emit(pathStep{kind: stepColumn, name: t.header[next.i]}, tail) + default: + panic(internalError("%s: a route onto %T", t.segment, r.next)) + } + return nil +} + +// walkEvery walks every variant and keeps the first one's steps: every variant carries +// the rest of the path, so each compiles to the same steps past the choice. func (w *pathCheck) walkEvery(c *choice, tail []string) (node, error) { if err := carriedByAll(c, tail); err != nil { return nil, err } var last node - for _, item := range c.items { + before, first := w.steps[:len(w.steps):len(w.steps)], w.steps + for i, item := range c.items { var err error + w.steps = before if last, err = w.walk(item, tail); err != nil { return nil, err } + if i == 0 { + first = w.steps + } } + w.steps = first return last, nil } @@ -259,45 +330,10 @@ func (w *pathCheck) enter(t *template, rest []string) error { return nil } -// pathCover is coverPath's walk. Where pathCheck walks every variant of a choice, it -// covers the first choice it reaches whole and stops; at a table it follows the route -// and selects no row. -type pathCover struct{ into map[node]bool } - -func (w *pathCover) walk(n node, tail []string) (node, error) { - for descended := false; len(tail) > 0 || descended; { - var err error - switch x := n.(type) { - case *choice: - cover(x, w.into, false) - return x, nil - case *table: - var r tableRoute - if r, err = x.route(tail, descended); err != nil { - return nil, err - } - n, tail, descended = r.next, r.rest, r.descends - continue - } - if n, err = stepInto(n, tail[0]); err != nil { - return nil, err - } - tail = tail[1:] - } - cover(n, w.into, false) - return n, nil -} - // pathProbe proves a path resolves without drawing. Where pathCheck walks every // variant of a choice, it walks the first, which carriedByAll lets stand for all; at a -// table it selects as pathCheck does. Probing an arm, it records what the read takes: -// in drawn the tables a draw would pin, and in sels each selector's table and spelling. -type pathProbe struct { - pins pinSet - a *arm - drawn map[*table]bool - sels []tableSel -} +// table it selects as pathCheck does. +type pathProbe struct{ pins pinSet } func (w *pathProbe) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { @@ -314,8 +350,10 @@ func (w *pathProbe) walk(n node, tail []string) (node, error) { if r, err = x.route(tail, descended); err != nil { return nil, err } - if err := w.readRow(x, tail, r); err != nil { - return nil, err + if r.sel != "" { + if err := w.pins.selectRow(x, r.sel); err != nil { + return nil, err + } } n, tail, descended = r.next, r.rest, r.descends continue @@ -328,25 +366,6 @@ func (w *pathProbe) walk(n node, tail []string) (node, error) { return n, nil } -func (w *pathProbe) readRow(t *table, tail []string, r tableRoute) error { - switch { - case r.sel != "": - if w.a != nil { - walked := len(w.a.tail) - len(tail) + 1 - w.sels = append(w.sels, tableSel{t, joinSegments(append([]string{w.a.writtenHead}, w.a.tail[:walked]...))}) - } - return w.pins.selectRow(t, r.sel) - case r.draw && w.a != nil: - if w.drawn == nil { - w.drawn = map[*table]bool{} - } - for stop := w.pins.nearestPinned(t); t != stop; t = t.parentT { - w.drawn[t] = true - } - } - return nil -} - // pathDraw draws the rows and variants a proved path reads, pinning the rows in pins; pins is nil // for a sibling path, which never crosses a table, since a table is only a category. For a memoized // read, memo keeps the variant drawn at each level of a, so paths sharing a prefix share it. @@ -471,6 +490,15 @@ func checkPath(n node, tail []string, level string) error { return err } +// compilePath compiles a path checkPath proved. +func compilePath(n node, tail []string) pathCheck { + w := pathCheck{tail: tail} + if _, err := w.run(n); err != nil { + panic(internalError("%s: %v; the path should have been proved before it was compiled", joinSegments(tail), err)) + } + return w +} + // joinSegments spells segments as a path: a selector attaches to the name before it. func joinSegments(segs []string) string { var b strings.Builder diff --git a/reads.go b/reads.go index a2ffdc5..e8bf049 100644 --- a/reads.go +++ b/reads.go @@ -242,18 +242,32 @@ type tableSel struct { spelling string } -// tableReadOf replays a reference path through pathProbe, so two paths pinning one -// row by different routes compare equal. checkPath proved -// each selector names a row. +// tableReadOf follows a reference path's steps for what it takes of a table family, so +// two paths pinning one row by different routes compare equal. func tableReadOf(head node, a arm, leaf node) *tableRead { t, isTable := head.(*table) if !isTable { return nil } - probe := &pathProbe{a: &a} - _, _ = probe.walk(t, a.tail) - _, landsRow := leaf.(*tableRow) - return &tableRead{headTable: t, pins: probe.pins, drawn: probe.drawn, sels: probe.sels, landsRow: landsRow} + r := &tableRead{headTable: t} + _, r.landsRow = leaf.(*tableRow) + for _, st := range a.steps { + switch st.kind { + case stepSelect: + r.pins.pin(t, st.row) + r.sels = append(r.sels, tableSel{t, joinSegments(append([]string{a.writtenHead}, a.tail[:st.at+1]...))}) + case stepDraw: + if r.drawn == nil { + r.drawn = map[*table]bool{} + } + for d, stop := t, r.pins.nearestPinned(t); d != stop; d = d.parentT { + r.drawn[d] = true + } + case stepChild: + t = t.descendant(st.name) + } + } + return r } // selected is the selector in r on t, or on the nearest ancestor of t it selects. diff --git a/reference.go b/reference.go index 861104e..833b0f7 100644 --- a/reference.go +++ b/reference.go @@ -93,9 +93,32 @@ func linkTemplate(folder []string, path, category string, t *template, root map[ return fmt.Errorf("%s: %w", path, err) } t.link, t.compiled = link, compiled + compileArms(t) return nil } +func compileArms(t *template) { + if r := t.link.readsColumn; r != nil { + compileArm(t, &r.a) + } + for i := range t.compiled.ops { + o := &t.compiled.ops[i] + for j := range o.operands { + compileArm(t, &o.operands[j]) + } + for j := range o.arms { + compileArm(t, &o.arms[j]) + } + } +} + +func compileArm(t *template, a *arm) { + if head := t.head(a.head); head != nil { + w := compilePath(head, a.tail) + a.steps, a.leaves, a.cover = w.steps, w.leaves, w.cover + } +} + // resolveLink binds every reference t reads, refusing one to t's own category, and keys // t's draw group by that category, so a name is local to it: // docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load diff --git a/todo.md b/todo.md index c750647..db711ec 100644 --- a/todo.md +++ b/todo.md @@ -40,13 +40,9 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Compile each path once at link into steps holding no pointer, with its leaves and - cover, and have `renderEdges`, `heldNodes`, `valueProof` and `tableReadOf` read them, - deleting `pathCover`, `coverPath` and `pathLeaves`. Prototyped green, allocations - identical. - Draw every path through one interpreter over those steps, `drawSteps`, with `descend` - probing into a stack buffer, deleting `pathDraw`, and replace the decision "The four - path walks are four loops": the prototype held `perf_test.go` once a step held no + probing into a stack buffer, deleting `pathDraw`, and replace the decision "The path + walks are separate loops": the prototype held `perf_test.go` once a step held no pointer, the probe returned its slice and the interpreter took `levels`. - Fold `checkNoOverlap` and `checkNoRepeatedRead` into `heldCheck`, so `compileFormat` compiles and nothing more. diff --git a/valueproof.go b/valueproof.go index 9c46f5b..0ac6217 100644 --- a/valueproof.go +++ b/valueproof.go @@ -164,7 +164,7 @@ func (p *valueProof) proveTemplate(t *template) proven { case o.kind == nameRead: var leaves []node for _, a := range o.arms { - leaves = append(leaves, pathLeaves(t.head(a.head), a.tail)...) + leaves = append(leaves, a.leaves...) } return p.proveUnion(leaves) case name == "calc": -- 2.52.0 From c29885e31fc0e5ba7df7b9c778dc57ffe388cfd1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 19:56:21 +0200 Subject: [PATCH 382/966] Require a compiled step to sit at the segment it consumes, and name the table read's test for it --- table_test.go | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/table_test.go b/table_test.go index 1fcdb36..823ebea 100644 --- a/table_test.go +++ b/table_test.go @@ -1046,10 +1046,10 @@ func TestEnteredRowsAgreeWithPinning(t *testing.T) { } } -// TestProbeReportsTheTablesADrawPins holds the check mode to the render: the tables a probe says a -// path draws are those a draw of it pins beyond what its selectors pin, and each selector is credited -// to the table it selects a row of. -func TestProbeReportsTheTablesADrawPins(t *testing.T) { +// TestTableReadReportsTheTablesADrawPins holds a path's compiled steps to the render: the tables +// its table read says it draws are those a draw of it pins beyond what its selectors pin, each +// selector is credited to the table it selects a row of, and a step naming a segment sits at it. +func TestTableReadReportsTheTablesADrawPins(t *testing.T) { f := newGenerator(t, writeFiles(t, geo()), WithSeed(1)) for _, c := range []struct{ path, sels string }{ {"locality.code", ""}, @@ -1070,26 +1070,40 @@ func TestProbeReportsTheTablesADrawPins(t *testing.T) { a := splitArm(c.path, nil) head := f.root.children[a.head].(*table) a.steps = compilePath(head, a.tail).steps + for _, st := range a.steps { + seg := "" + switch st.kind { + case stepSelect: + seg = "[" + st.name + "]" + case stepChild, stepColumn: + seg = st.name + default: + continue + } + if st.at >= len(a.tail) || a.tail[st.at] != seg { + t.Errorf("%s: step %q sits at %d, want the segment it consumes", c.path, seg, st.at) + } + } tr := tableReadOf(head, a, nil) var sels []string for _, s := range tr.sels { sels = append(sels, s.t.segment+"="+s.spelling) } if got := strings.Join(sels, " "); got != c.sels { - t.Errorf("%s: the probe credits the selectors %q, want %q", c.path, got, c.sels) + t.Errorf("%s: the table read credits the selectors %q, want %q", c.path, got, c.sels) } - probed, drawn := tr.pins, tr.drawn + selected, drawn := tr.pins, tr.drawn for i := 0; i < 20; i++ { var pins pinSet drawPath(head, a.tail, a.head, &pathDraw{s: f.rand, pins: &pins}) want := map[*table]bool{} pins.each(func(tbl *table, _ int) { - if _, selected := probed.pinned(tbl); !selected { + if _, isSelected := selected.pinned(tbl); !isSelected { want[tbl] = true } }) if !maps.Equal(drawn, want) { - t.Fatalf("%s: the probe reports %v drawn, a draw pins %v", c.path, segmentsOf(drawn), segmentsOf(want)) + t.Fatalf("%s: the table read reports %v drawn, a draw pins %v", c.path, segmentsOf(drawn), segmentsOf(want)) } } } -- 2.52.0 From d279408ba523093729f1ba5e27a7813591db92d1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 19:57:29 +0200 Subject: [PATCH 383/966] Step a table's routes past its selector, and panic on an arm with no bound head --- docs/decisions.md | 5 ++--- path.go | 1 + reference.go | 8 +++++--- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 2ab0cd1..2aef6aa 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -846,9 +846,8 @@ the direct chain is the one a reader can predict from the tables' parents. 2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. -Goals 2 and 11: `pathCheck`, `pathProbe` and `pathDraw` stay separate loops; `pathCover` -became the cover `pathCheck` compiles, on 2026-09-30. -One loop fails `perf_test.go`: the check's leaves, the probe's selectors and the errors +Goals 2 and 11: `pathCheck`, `pathProbe` and `pathDraw` stay separate loops. +One loop fails `perf_test.go`: the check's leaves and the errors naming a level leak the walker's contents, and Go tracks a struct's fields as one, so the draw's pins, arm and memo leak with them. Held by pointer, a repeat of a reference path rose from 66 to 106 allocations; copied into the walker by value, diff --git a/path.go b/path.go index 9659ce8..a7c10a6 100644 --- a/path.go +++ b/path.go @@ -278,6 +278,7 @@ func (w *pathCheck) tableSteps(t *table, r tableRoute, tail []string) error { } row, _ := w.pins.pinned(t) w.emit(pathStep{kind: stepSelect, name: r.sel, row: row}, tail) + tail = tail[1:] } if r.draw { w.emit(pathStep{kind: stepDraw}, tail) diff --git a/reference.go b/reference.go index 833b0f7..f2ac8cb 100644 --- a/reference.go +++ b/reference.go @@ -113,10 +113,12 @@ func compileArms(t *template) { } func compileArm(t *template, a *arm) { - if head := t.head(a.head); head != nil { - w := compilePath(head, a.tail) - a.steps, a.leaves, a.cover = w.steps, w.leaves, w.cover + head := t.head(a.head) + if head == nil { + panic(internalError("{%s} reads a head nothing bound", a.spelling)) } + w := compilePath(head, a.tail) + a.steps, a.leaves, a.cover = w.steps, w.leaves, w.cover } // resolveLink binds every reference t reads, refusing one to t's own category, and keys -- 2.52.0 From 1397d1438a5eb07efc7ce7679b1a2cf0d255d6f5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 19:57:49 +0200 Subject: [PATCH 384/966] Say a compiled step holds no node, where its name is a string --- path.go | 2 +- todo.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/path.go b/path.go index a7c10a6..bff2e7a 100644 --- a/path.go +++ b/path.go @@ -187,7 +187,7 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { } // pathStep is one step of a compiled path, taken at the node the steps before it -// reached; at indexes the tail segment the step consumes. A step holds no pointer, so a +// reached; at indexes the tail segment the step consumes. A step holds no node, so a // caller's steps stay on its stack. type pathStep struct { kind stepKind diff --git a/todo.md b/todo.md index db711ec..c019b27 100644 --- a/todo.md +++ b/todo.md @@ -43,7 +43,7 @@ This round, in order: - Draw every path through one interpreter over those steps, `drawSteps`, with `descend` probing into a stack buffer, deleting `pathDraw`, and replace the decision "The path walks are separate loops": the prototype held `perf_test.go` once a step held no - pointer, the probe returned its slice and the interpreter took `levels`. + node, the probe returned its slice and the interpreter took `levels`. - Fold `checkNoOverlap` and `checkNoRepeatedRead` into `heldCheck`, so `compileFormat` compiles and nothing more. - Fix at compile how `readField` reads an arm, `arm.kind`, deleting its two panics and -- 2.52.0 From d1722a8706c2f3699be8d9661ac25d9261e8d113 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:13:08 +0200 Subject: [PATCH 385/966] Pin the seeded bytes of paths through a choice, a selector and a drawn row --- format_stability_test.go | 63 ++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 21 deletions(-) diff --git a/format_stability_test.go b/format_stability_test.go index e0e584f..8028182 100644 --- a/format_stability_test.go +++ b/format_stability_test.go @@ -1,34 +1,55 @@ package fejkdata -import "testing" +import ( + "os" + "path/filepath" + "testing" +) // TestSeededOutputIsStable pins the exact bytes a seeded generator emits for each // piece of the token grammar, so a change to how a format is scanned or rendered // cannot quietly shift the rng stream that reproducibility depends on. func TestSeededOutputIsStable(t *testing.T) { dir := writeData(t, map[string]string{ - "alt": `{"format":"{a|b}","a":"A","b":"B"}`, - "calc": `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00","100.00"],"qty":["2","3","7"]}`, - "classes": `"{digits(2)}-{int(1,9)}{int(1,9)}-{upper(2)}-{lower(2)}"`, - "escapes": `{"format":"01Aa#{x}","x":"!"}`, - "funcs": `"{hex(6)} {int(10,99)} {float(0,1,3)} {nanoid(5)} {seq()}"`, - "nested": `{"format":"{outer}","outer":{"format":"{inner}-{digits(2)}","inner":"i"}}`, - "ref": `"see {/alt}"`, - "repeat": `{"format":"{w}","repeat":4,"separator":",","w":["x","y","z"]}`, - "sums": `{"format":"9{d}{luhn()} {e}{ean()} {m}{mod11()}","d":"012345678901234","e":"123456789012","m":"12345678"}`, - "weights": `[{"format":"big","weight":9},"tiny"]`, + "alt": `{"format":"{a|b}","a":"A","b":"B"}`, + "calc": `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00","100.00"],"qty":["2","3","7"]}`, + "classes": `"{digits(2)}-{int(1,9)}{int(1,9)}-{upper(2)}-{lower(2)}"`, + "escapes": `{"format":"01Aa#{x}","x":"!"}`, + "funcs": `"{hex(6)} {int(10,99)} {float(0,1,3)} {nanoid(5)} {seq()}"`, + "municipality": `{"format":"{name}","rows":"municipality.tsv","key":"code","name":"name","parent":"region"}`, + "nested": `{"format":"{outer}","outer":{"format":"{inner}-{digits(2)}","inner":"i"}}`, + "paths": `{"format":"{p.a}{p.b}","p":[{"format":"x","a":"1","b":"2"},{"format":"y","a":"A","b":"B"},{"format":"z","a":"a","b":"b"}]}`, + "ref": `"see {/alt}"`, + "refpath": `"{/region.municipality.name} in {/region.name}"`, + "region": `{"format":"{name}","rows":"region.tsv","key":"code","name":"name","weight":"people"}`, + "repeat": `{"format":"{w}","repeat":4,"separator":",","w":["x","y","z"]}`, + "sums": `{"format":"9{d}{luhn()} {e}{ean()} {m}{mod11()}","d":"012345678901234","e":"123456789012","m":"12345678"}`, + "weights": `[{"format":"big","weight":9},"tiny"]`, }) + for name, rows := range map[string]string{ + "municipality": "code\tname\tregion\n0180\tStockholm\t01\n0184\tSolna\t01\n1280\tMalmö\t12\n1281\tLund\t12\n", + "region": "code\tname\tpeople\n01\tStockholms län\t3\n12\tSkåne län\t1\n", + } { + if err := os.WriteFile(filepath.Join(dir, name+".tsv"), []byte(rows), 0o644); err != nil { + t.Fatal(err) + } + } want := map[string][]string{ - "alt": {"A", "A", "B", "A"}, - "calc": {"100.00 x 3 = 300.00", "100.00 x 3 = 300.00", "100.00 x 7 = 700.00", "5.00 x 3 = 15.00"}, - "classes": {"49-74-VY-gj", "38-68-RP-xo", "47-68-IB-hs", "91-89-LP-gk"}, - "escapes": {"01Aa#!", "01Aa#!", "01Aa#!", "01Aa#!"}, - "funcs": {"0016a2 33 0.649 k4Kwj 1", "a00c07 84 0.175 6UjGv 2", "f70eb8 12 0.390 p5p6g 3", "9048a3 90 0.531 I3Aqv 4"}, - "nested": {"i-73", "i-23", "i-67", "i-85"}, - "ref": {"see A", "see A", "see A", "see B"}, - "repeat": {"z,y,z,y", "z,z,y,y", "z,z,x,z", "x,z,y,y"}, - "sums": {"90123456789012348 1234567890124 123456780", "90123456789012348 1234567890124 123456780", "90123456789012348 1234567890124 123456780", "90123456789012348 1234567890124 123456780"}, - "weights": {"big", "big", "big", "big"}, + "alt": {"A", "A", "B", "A"}, + "calc": {"100.00 x 3 = 300.00", "100.00 x 3 = 300.00", "100.00 x 7 = 700.00", "5.00 x 3 = 15.00"}, + "classes": {"49-74-VY-gj", "38-68-RP-xo", "47-68-IB-hs", "91-89-LP-gk"}, + "escapes": {"01Aa#!", "01Aa#!", "01Aa#!", "01Aa#!"}, + "funcs": {"0016a2 33 0.649 k4Kwj 1", "a00c07 84 0.175 6UjGv 2", "f70eb8 12 0.390 p5p6g 3", "9048a3 90 0.531 I3Aqv 4"}, + "nested": {"i-73", "i-23", "i-67", "i-85"}, + "paths": {"ab", "ab", "AB", "AB"}, + "paths.p.a": {"A", "a", "a", "A"}, + "ref": {"see A", "see A", "see A", "see B"}, + "refpath": {"Solna in Stockholms län", "Malmö in Skåne län", "Stockholm in Stockholms län", "Malmö in Skåne län"}, + "region.municipality[0184].name": {"Solna", "Solna", "Solna", "Solna"}, + "region[12].municipality.name": {"Malmö", "Malmö", "Lund", "Malmö"}, + "repeat": {"z,y,z,y", "z,z,y,y", "z,z,x,z", "x,z,y,y"}, + "sums": {"90123456789012348 1234567890124 123456780", "90123456789012348 1234567890124 123456780", "90123456789012348 1234567890124 123456780", "90123456789012348 1234567890124 123456780"}, + "weights": {"big", "big", "big", "big"}, } for path := range want { f := newGenerator(t, dir, WithSeed(42)) -- 2.52.0 From d5d1e86b1c351973be9a6ae9fde8ae7e165fc6ae Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:04:39 +0200 Subject: [PATCH 386/966] Draw a path's compiled steps through drawSteps in the tests --- path_test.go | 8 ++++---- table_test.go | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/path_test.go b/path_test.go index 9866f2f..2110263 100644 --- a/path_test.go +++ b/path_test.go @@ -36,13 +36,13 @@ func TestPathCoverStopsAtAChoice(t *testing.T) { } } -func TestDrawPathPanicsOnAnUnprovedPath(t *testing.T) { +func TestDrawStepsPanicsOnAStepItsNodeLacks(t *testing.T) { defer func() { - if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), `plain.f: no field "f"`) { - t.Errorf("drawPath(plain, f) recovered %v, want a panic naming the path and the missing field", r) + if r := recover(); r == nil || !strings.Contains(fmt.Sprint(r), `no field "f"`) { + t.Errorf("drawSteps(plain, f) recovered %v, want a panic naming the missing field", r) } }() - drawPath(compiled(t, `"plain"`), []string{"f"}, "plain", &pathDraw{s: engine(1).rand, pins: &pinSet{}}) + drawSteps(engine(1).rand, compiled(t, `"plain"`), []pathStep{{kind: stepField, name: "f"}}, &pinSet{}, nil, nil) } func TestDeepDottedPath(t *testing.T) { diff --git a/table_test.go b/table_test.go index 823ebea..cc3e23c 100644 --- a/table_test.go +++ b/table_test.go @@ -1095,7 +1095,7 @@ func TestTableReadReportsTheTablesADrawPins(t *testing.T) { selected, drawn := tr.pins, tr.drawn for i := 0; i < 20; i++ { var pins pinSet - drawPath(head, a.tail, a.head, &pathDraw{s: f.rand, pins: &pins}) + drawSteps(f.rand, head, a.steps, &pins, nil, nil) want := map[*table]bool{} pins.each(func(tbl *table, _ int) { if _, isSelected := selected.pinned(tbl); !isSelected { -- 2.52.0 From 96d9e2ab24bf57cdbae1ffa854897748c25ef83e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:07:10 +0200 Subject: [PATCH 387/966] Draw every path through drawSteps over its compiled steps, deleting pathDraw and pathProbe --- AGENTS.md | 2 +- docs/decisions.md | 21 +++--- path.go | 177 +++++++++++++++++++++------------------------- render.go | 8 ++- todo.md | 4 -- 5 files changed, 98 insertions(+), 114 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 24ee4ad..b5d3bfc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -116,4 +116,4 @@ In [docs/decisions.md](docs/decisions.md): - The Swedish ids draw Skatteverket's test series - The US given names come from a mirror of the SSA file - `List` advertises direct descents only -- The path walks are separate loops +- A path draws through its compiled steps diff --git a/docs/decisions.md b/docs/decisions.md index 2aef6aa..e5fc422 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -842,15 +842,14 @@ import drops them by name rather than by a rank a regeneration would move. the set of every descent through a chain of five tables is every subsequence of it, and the direct chain is the one a reader can predict from the tables' parents. -## The path walks are separate loops +## A path draws through its compiled steps -2026-09-28, larv-review on systems-architect's finding; approved 2026-09-28 by lilleman. -Re-tested 2026-09-29 by collapsing them into one loop switching on a mode field. -Goals 2 and 11: `pathCheck`, `pathProbe` and `pathDraw` stay separate loops. -One loop fails `perf_test.go`: the check's leaves and the errors -naming a level leak the walker's contents, and Go tracks a struct's fields as one, so -the draw's pins, arm and memo leak with them. Held by pointer, a repeat of a reference -path rose from 66 to 106 allocations; copied into the walker by value, -`BenchmarkHeld` rose from 5 to 9. A call through an interface or a type parameter leaks -the walker the same way. Valid while Go's escape analysis tracks a struct's fields as -one and treats such a call as leaking its receiver. +2026-09-30, larv-review on the comprehension round the maintainer approved on 2026-09-29; +replaces "The path walks are separate loops". +Goals 2 and 11: `drawSteps` draws every path from its steps, compiled at link by +`pathCheck` or, for a caller's path, by `probePath` into a stack buffer. The check and +the probe stay two loops: one loop leaks the check's leaves and errors with the draw's +state, since Go tracks a struct's fields as one, and a repeat of a reference path rose +from 66 to 106 allocations. `drawSteps` takes its state as parameters and a step holds +no node, so no benchmark gained an allocation. Valid while Go's escape analysis tracks +a struct's fields as one. diff --git a/path.go b/path.go index bff2e7a..b08c7fb 100644 --- a/path.go +++ b/path.go @@ -229,11 +229,6 @@ func (w *pathCheck) run(n node) (node, error) { return leaf, err } -func (w *pathCheck) emit(st pathStep, tail []string) { - st.at = len(w.tail) - len(tail) - w.steps = append(w.steps, st) -} - func (w *pathCheck) walk(n node, tail []string) (node, error) { for descended := false; len(tail) > 0 || descended; { var err error @@ -248,7 +243,7 @@ func (w *pathCheck) walk(n node, tail []string) (node, error) { if r, err = x.route(tail, descended); err != nil { return nil, err } - if err := w.tableSteps(x, r, tail); err != nil { + if w.steps, err = routeSteps(w.steps, &w.pins, x, r, len(w.tail)-len(tail)); err != nil { return nil, err } n, tail, descended = r.next, r.rest, r.descends @@ -264,38 +259,40 @@ func (w *pathCheck) walk(n node, tail []string) (node, error) { if n, err = stepInto(n, tail[0]); err != nil { return nil, err } - w.emit(pathStep{kind: stepField, name: tail[0]}, tail) + w.steps = append(w.steps, pathStep{kind: stepField, at: len(w.tail) - len(tail), name: tail[0]}) tail = tail[1:] } w.leaves = append(w.leaves, n) return n, nil } -func (w *pathCheck) tableSteps(t *table, r tableRoute, tail []string) error { +// routeSteps appends the steps r takes past t, the first at at, pinning in pins the row +// it selects. +func routeSteps(steps []pathStep, pins *pinSet, t *table, r tableRoute, at int) ([]pathStep, error) { if r.sel != "" { - if err := w.pins.selectRow(t, r.sel); err != nil { - return err + if err := pins.selectRow(t, r.sel); err != nil { + return steps, err } - row, _ := w.pins.pinned(t) - w.emit(pathStep{kind: stepSelect, name: r.sel, row: row}, tail) - tail = tail[1:] + row, _ := pins.pinned(t) + steps = append(steps, pathStep{kind: stepSelect, at: at, name: r.sel, row: row}) + at++ } if r.draw { - w.emit(pathStep{kind: stepDraw}, tail) + steps = append(steps, pathStep{kind: stepDraw, at: at}) } switch next := r.next.(type) { case *table: if next != t { - w.emit(pathStep{kind: stepChild, name: next.segment}, tail) + steps = append(steps, pathStep{kind: stepChild, at: at, name: next.segment}) } case *tableRow: - w.emit(pathStep{kind: stepRow}, tail) + steps = append(steps, pathStep{kind: stepRow, at: at}) case *tableColumn: - w.emit(pathStep{kind: stepColumn, name: t.header[next.i]}, tail) + steps = append(steps, pathStep{kind: stepColumn, at: at, name: t.header[next.i]}) default: panic(internalError("%s: a route onto %T", t.segment, r.next)) } - return nil + return steps, nil } // walkEvery walks every variant and keeps the first one's steps: every variant carries @@ -331,13 +328,12 @@ func (w *pathCheck) enter(t *template, rest []string) error { return nil } -// pathProbe proves a path resolves without drawing. Where pathCheck walks every -// variant of a choice, it walks the first, which carriedByAll lets stand for all; at a -// table it selects as pathCheck does. -type pathProbe struct{ pins pinSet } - -func (w *pathProbe) walk(n node, tail []string) (node, error) { - for descended := false; len(tail) > 0 || descended; { +// probePath proves a path resolves without drawing, appending to steps the steps a +// draw takes. Where pathCheck walks every variant of a choice, it walks the first, +// which carriedByAll lets stand for all; at a table it selects as pathCheck does. +func probePath(n node, tail []string, steps []pathStep) ([]pathStep, error) { + var pins pinSet + for full, descended := len(tail), false; len(tail) > 0 || descended; { var err error switch x := n.(type) { case *choice: @@ -351,89 +347,80 @@ func (w *pathProbe) walk(n node, tail []string) (node, error) { if r, err = x.route(tail, descended); err != nil { return nil, err } - if r.sel != "" { - if err := w.pins.selectRow(x, r.sel); err != nil { - return nil, err - } - } - n, tail, descended = r.next, r.rest, r.descends - continue - } - if n, err = stepInto(n, tail[0]); err != nil { - return nil, err - } - tail = tail[1:] - } - return n, nil -} - -// pathDraw draws the rows and variants a proved path reads, pinning the rows in pins; pins is nil -// for a sibling path, which never crosses a table, since a table is only a category. For a memoized -// read, memo keeps the variant drawn at each level of a, so paths sharing a prefix share it. -// Where pathCheck walks every variant of a choice, it walks the one drawn; at a table it selects -// as pathCheck does and draws the row the route asks for. -type pathDraw struct { - s *generatorState - pins *pinSet - memo *drawMemo - a *arm -} - -// drawPath walks w over a path proved first, by a probe or at load, so the walk -// cannot fail. level names n in the panic. -func drawPath(n node, tail []string, level string, w *pathDraw) node { - leaf, err := w.walk(n, tail) - if err != nil { - panic(internalError("%s: %v; the path should have been proved before it was drawn", join(level, joinSegments(tail)), err)) - } - return leaf -} - -func (w *pathDraw) walk(n node, tail []string) (node, error) { - for descended := false; len(tail) > 0 || descended; { - var err error - switch x := n.(type) { - case *choice: - n = w.variant(x, tail) - continue - case *table: - var r tableRoute - if r, err = x.route(tail, descended); err != nil { + if steps, err = routeSteps(steps, &pins, x, r, full-len(tail)); err != nil { return nil, err } - switch { - case r.sel != "": - if err := w.pins.selectRow(x, r.sel); err != nil { - return nil, err - } - case r.draw: - x.drawIn(w.s, w.pins) - } n, tail, descended = r.next, r.rest, r.descends continue } if n, err = stepInto(n, tail[0]); err != nil { return nil, err } + steps = append(steps, pathStep{kind: stepField, at: full - len(tail), name: tail[0]}) tail = tail[1:] } - return n, nil + return steps, nil } -// variant is the variant of c the walk continues into: the one memo keeps for this level -// of the read, drawn once, where the walk has a memo; else one drawn afresh. -func (w *pathDraw) variant(c *choice, rest []string) node { - if w.memo == nil { - return pick(w.s, c) - } - key := w.a.levels[len(w.a.tail)-len(rest)] - n, drew := w.memo.variant[key] - if !drew { - n = resolveChoice(w.s, c) - if w.memo.variant == nil { - w.memo.variant = map[string]node{} +// drawSteps draws the rows and variants a path's steps read from n, pinning the rows in pins; +// pins is nil for a sibling path, which never crosses a table, since a table is only a category. +// For a memoized read, memo keeps the variant drawn at each of levels, so paths sharing a prefix +// share it. +func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo *drawMemo, levels []string) node { + for _, st := range steps { + if c, ok := n.(*choice); ok { + n = drawVariant(s, c, memo, levels, st.at) } - w.memo.variant[key] = n + if st.kind == stepField { + var err error + if n, err = stepInto(n, st.name); err != nil { + panic(internalError("%v; a path's steps should have been compiled from the node they are drawn from", err)) + } + continue + } + t, ok := n.(*table) + if !ok { + panic(internalError("a table's step %q on %T", st.name, n)) + } + n = t.drawStep(s, st, pins) + } + return n +} + +func (t *table) drawStep(s *generatorState, st pathStep, pins *pinSet) node { + switch st.kind { + case stepSelect: + if err := pins.pinRow(t, st.row); err != nil { + panic(internalError("%s[%s]: %v", t.segment, st.name, err)) + } + return t + case stepDraw: + t.drawIn(s, pins) + return t + case stepRow: + return t.rowNode + case stepColumn: + return t.formatTemplate.fields[st.name] + case stepChild: + return t.descendant(st.name) + } + panic(internalError("drawStep has no case for step kind %d", st.kind)) +} + +// drawVariant is the variant of c a draw continues into at the step at: the one memo keeps for +// that level, drawn once, where the draw has a memo; else one drawn afresh. +func drawVariant(s *generatorState, c *choice, memo *drawMemo, levels []string, at int) node { + if memo == nil { + return resolveChoice(s, c) + } + level := levels[at] + n, drew := memo.variant[level] + if !drew { + n = resolveChoice(s, c) + if memo.variant == nil { + memo.variant = map[string]node{} + } + memo.variant[level] = n } return n } diff --git a/render.go b/render.go index b5c5da9..3982afd 100644 --- a/render.go +++ b/render.go @@ -40,10 +40,12 @@ func (f *Generator) Fake(path string) (string, error) { // selects or draws. func descend(s *generatorState, root node, segments []string, sc renderScope) (node, error) { // docs/decisions.md#a-path-is-walked-once-without-drawing-before-it-is-walked-for-real - if _, err := (&pathProbe{}).walk(root, segments); err != nil { + var buf [16]pathStep + steps, err := probePath(root, segments, buf[:0]) + if err != nil { return nil, err } - return drawPath(root, segments, "", &pathDraw{s: s, pins: &sc.groupDraws().pins}), nil + return drawSteps(s, root, steps, &sc.groupDraws().pins, nil, nil), nil } // renderOnce renders n as one render, over draws of its own. @@ -188,7 +190,7 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r if r, done := memo.value[a.path]; done { return r } - leaf := drawPath(t.head(a.head), a.tail, a.head, &pathDraw{s: s, memo: memo, pins: pins, a: &a}) + leaf := drawSteps(s, t.head(a.head), a.steps, pins, memo, a.levels) if pins != nil { sc = sc.at(leaf, pins) } diff --git a/todo.md b/todo.md index c019b27..eefa87d 100644 --- a/todo.md +++ b/todo.md @@ -40,10 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Draw every path through one interpreter over those steps, `drawSteps`, with `descend` - probing into a stack buffer, deleting `pathDraw`, and replace the decision "The path - walks are separate loops": the prototype held `perf_test.go` once a step held no - node, the probe returned its slice and the interpreter took `levels`. - Fold `checkNoOverlap` and `checkNoRepeatedRead` into `heldCheck`, so `compileFormat` compiles and nothing more. - Fix at compile how `readField` reads an arm, `arm.kind`, deleting its two panics and -- 2.52.0 From 270ba93fbe372aae003b10bf25a935b293ae3573 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:13:49 +0200 Subject: [PATCH 388/966] Name a failed step's index in drawSteps' panics, and quote the decision's measurement as the experiment it was --- docs/decisions.md | 11 ++++++----- path.go | 9 +++++---- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index e5fc422..cc12a65 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -591,7 +591,7 @@ is until a real data set needs the indexed form. 2026-09-17, Lilleman auf Larv. A path that fails below its first level then moves no seeded stream, at the cost of one -draw-free walk per call, which allocates nothing. +draw-free walk per call, which allocates nothing up to 16 steps. ## A country's postal codes and streets are siblings under its locality @@ -848,8 +848,9 @@ the direct chain is the one a reader can predict from the tables' parents. replaces "The path walks are separate loops". Goals 2 and 11: `drawSteps` draws every path from its steps, compiled at link by `pathCheck` or, for a caller's path, by `probePath` into a stack buffer. The check and -the probe stay two loops: one loop leaks the check's leaves and errors with the draw's -state, since Go tracks a struct's fields as one, and a repeat of a reference path rose -from 66 to 106 allocations. `drawSteps` takes its state as parameters and a step holds -no node, so no benchmark gained an allocation. Valid while Go's escape analysis tracks +the probe stay two loops: on 2026-09-29 one loop over all three walks, switching on a +mode field, leaked the check's leaves and errors with the draw's pins, arm and memo, +since Go tracks a struct's fields as one, and a repeat of a reference path rose from 66 +to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node, +so no benchmark gained an allocation. Valid while Go's escape analysis tracks a struct's fields as one. diff --git a/path.go b/path.go index b08c7fb..c4f1307 100644 --- a/path.go +++ b/path.go @@ -187,7 +187,8 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { } // pathStep is one step of a compiled path, taken at the node the steps before it -// reached; at indexes the tail segment the step consumes. A step holds no node, so a +// reached, a choice there resolved first; at indexes the tail where it is taken, and +// only a table's step sits past the tail's end. A step holds no node, so a // caller's steps stay on its stack. type pathStep struct { kind stepKind @@ -199,7 +200,7 @@ type pathStep struct { type stepKind uint8 const ( - stepField stepKind = iota + 1 // into a field of the template at this level, a choice there resolved first + stepField stepKind = iota + 1 // into a field of the template at this level stepSelect // pin the selected row of the table stepDraw // draw a row of the table inside the pins stepRow // land on the table's row node @@ -374,13 +375,13 @@ func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo * if st.kind == stepField { var err error if n, err = stepInto(n, st.name); err != nil { - panic(internalError("%v; a path's steps should have been compiled from the node they are drawn from", err)) + panic(internalError("step %d: %v; a path's steps should have been compiled from the node they are drawn from", st.at, err)) } continue } t, ok := n.(*table) if !ok { - panic(internalError("a table's step %q on %T", st.name, n)) + panic(internalError("step %d: a table's step of kind %d on %T", st.at, st.kind, n)) } n = t.drawStep(s, st, pins) } -- 2.52.0 From 220129584e8f13a90e6ba1f640aafc433bae6ae4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:14:52 +0200 Subject: [PATCH 389/966] Write the seeded pin's data through writeFiles --- format_stability_test.go | 46 ++++++++++++++++------------------------ 1 file changed, 18 insertions(+), 28 deletions(-) diff --git a/format_stability_test.go b/format_stability_test.go index 8028182..3cfb0c9 100644 --- a/format_stability_test.go +++ b/format_stability_test.go @@ -1,39 +1,29 @@ package fejkdata -import ( - "os" - "path/filepath" - "testing" -) +import "testing" // TestSeededOutputIsStable pins the exact bytes a seeded generator emits for each // piece of the token grammar, so a change to how a format is scanned or rendered // cannot quietly shift the rng stream that reproducibility depends on. func TestSeededOutputIsStable(t *testing.T) { - dir := writeData(t, map[string]string{ - "alt": `{"format":"{a|b}","a":"A","b":"B"}`, - "calc": `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00","100.00"],"qty":["2","3","7"]}`, - "classes": `"{digits(2)}-{int(1,9)}{int(1,9)}-{upper(2)}-{lower(2)}"`, - "escapes": `{"format":"01Aa#{x}","x":"!"}`, - "funcs": `"{hex(6)} {int(10,99)} {float(0,1,3)} {nanoid(5)} {seq()}"`, - "municipality": `{"format":"{name}","rows":"municipality.tsv","key":"code","name":"name","parent":"region"}`, - "nested": `{"format":"{outer}","outer":{"format":"{inner}-{digits(2)}","inner":"i"}}`, - "paths": `{"format":"{p.a}{p.b}","p":[{"format":"x","a":"1","b":"2"},{"format":"y","a":"A","b":"B"},{"format":"z","a":"a","b":"b"}]}`, - "ref": `"see {/alt}"`, - "refpath": `"{/region.municipality.name} in {/region.name}"`, - "region": `{"format":"{name}","rows":"region.tsv","key":"code","name":"name","weight":"people"}`, - "repeat": `{"format":"{w}","repeat":4,"separator":",","w":["x","y","z"]}`, - "sums": `{"format":"9{d}{luhn()} {e}{ean()} {m}{mod11()}","d":"012345678901234","e":"123456789012","m":"12345678"}`, - "weights": `[{"format":"big","weight":9},"tiny"]`, + dir := writeFiles(t, map[string]string{ + "alt.json": `{"format":"{a|b}","a":"A","b":"B"}`, + "calc.json": `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00","100.00"],"qty":["2","3","7"]}`, + "classes.json": `"{digits(2)}-{int(1,9)}{int(1,9)}-{upper(2)}-{lower(2)}"`, + "escapes.json": `{"format":"01Aa#{x}","x":"!"}`, + "funcs.json": `"{hex(6)} {int(10,99)} {float(0,1,3)} {nanoid(5)} {seq()}"`, + "municipality.json": `{"format":"{name}","rows":"municipality.tsv","key":"code","name":"name","parent":"region"}`, + "municipality.tsv": "code\tname\tregion\n0180\tStockholm\t01\n0184\tSolna\t01\n1280\tMalmö\t12\n1281\tLund\t12\n", + "nested.json": `{"format":"{outer}","outer":{"format":"{inner}-{digits(2)}","inner":"i"}}`, + "paths.json": `{"format":"{p.a}{p.b}","p":[{"format":"x","a":"1","b":"2"},{"format":"y","a":"A","b":"B"},{"format":"z","a":"a","b":"b"}]}`, + "ref.json": `"see {/alt}"`, + "refpath.json": `"{/region.municipality.name} in {/region.name}"`, + "region.json": `{"format":"{name}","rows":"region.tsv","key":"code","name":"name","weight":"people"}`, + "region.tsv": "code\tname\tpeople\n01\tStockholms län\t3\n12\tSkåne län\t1\n", + "repeat.json": `{"format":"{w}","repeat":4,"separator":",","w":["x","y","z"]}`, + "sums.json": `{"format":"9{d}{luhn()} {e}{ean()} {m}{mod11()}","d":"012345678901234","e":"123456789012","m":"12345678"}`, + "weights.json": `[{"format":"big","weight":9},"tiny"]`, }) - for name, rows := range map[string]string{ - "municipality": "code\tname\tregion\n0180\tStockholm\t01\n0184\tSolna\t01\n1280\tMalmö\t12\n1281\tLund\t12\n", - "region": "code\tname\tpeople\n01\tStockholms län\t3\n12\tSkåne län\t1\n", - } { - if err := os.WriteFile(filepath.Join(dir, name+".tsv"), []byte(rows), 0o644); err != nil { - t.Fatal(err) - } - } want := map[string][]string{ "alt": {"A", "A", "B", "A"}, "calc": {"100.00 x 3 = 300.00", "100.00 x 3 = 300.00", "100.00 x 7 = 700.00", "5.00 x 3 = 15.00"}, -- 2.52.0 From 4bf8deddb4b3547119407e23eb65eca8057ed612 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:17:18 +0200 Subject: [PATCH 390/966] Name the goals behind probing a path before drawing it, and cut comments restating their code --- docs/decisions.md | 5 +++-- path.go | 8 +++----- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index cc12a65..8b13829 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -590,8 +590,9 @@ is until a real data set needs the indexed form. 2026-09-17, Lilleman auf Larv. -A path that fails below its first level then moves no seeded stream, at the cost of one -draw-free walk per call, which allocates nothing up to 16 steps. +Goals 8 and 11: a path that fails moves no seeded stream, at the cost of one draw-free +walk per call, compiling its steps into a stack buffer of 16. Valid while that walk +costs little against the draw it guards. ## A country's postal codes and streets are siblings under its locality diff --git a/path.go b/path.go index c4f1307..b2ae912 100644 --- a/path.go +++ b/path.go @@ -200,12 +200,12 @@ type pathStep struct { type stepKind uint8 const ( - stepField stepKind = iota + 1 // into a field of the template at this level + stepField stepKind = iota + 1 // into a folder's entry or a template's field stepSelect // pin the selected row of the table stepDraw // draw a row of the table inside the pins stepRow // land on the table's row node - stepColumn // into a column of the table - stepChild // into a table linked to the table + stepColumn + stepChild ) // pathCheck proves a path resolves whichever way the draws go: every variant of a @@ -408,8 +408,6 @@ func (t *table) drawStep(s *generatorState, st pathStep, pins *pinSet) node { panic(internalError("drawStep has no case for step kind %d", st.kind)) } -// drawVariant is the variant of c a draw continues into at the step at: the one memo keeps for -// that level, drawn once, where the draw has a memo; else one drawn afresh. func drawVariant(s *generatorState, c *choice, memo *drawMemo, levels []string, at int) node { if memo == nil { return resolveChoice(s, c) -- 2.52.0 From 4ea7cb48a522dde63ab7a8db09e8623048eff0ed Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:22:19 +0200 Subject: [PATCH 391/966] Check a repeated held token through NewTemplate, where the load fences run --- hold_test.go | 9 +++++---- transform_test.go | 5 +++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/hold_test.go b/hold_test.go index 06e39ab..560bb6f 100644 --- a/hold_test.go +++ b/hold_test.go @@ -737,15 +737,16 @@ func TestHeldPathIsReachableByFake(t *testing.T) { } func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { + f := newGenerator(t, writeData(t, map[string]string{"w": `"a"`})) for src, want := range map[string]string{ `{"format":"{w} {w} {uppercase(w)}","w":["a","b"]}`: "write {w} once", `{"format":"{w} {w|x} {uppercase(w)}","w":["a","b"],"x":["c","d"]}`: "write {w} once", `{"format":"{n} + {n} = {calc(n * 2)}","n":["1","2"]}`: "write {n} once", `{"format":"{p.a} {q} {q} {lowercase(q)}","p":{"format":"{a}","a":"1"},"q":["A","B"]}`: "write {q} once", } { - _, err := linked(t, src) + _, err := f.NewTemplate(src) if err == nil || !strings.Contains(err.Error(), want) || !strings.Contains(err.Error(), "holds") { - t.Errorf("compile(%s) = %v, want the repeated token rejected naming %s", src, err, want) + t.Errorf("NewTemplate(%s) = %v, want the repeated token rejected naming %s", src, err, want) } } for _, ok := range []string{ @@ -754,8 +755,8 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { `{"format":"{net} x {qty} = {calc(net * qty, 2)}","net":["19.99","5.00"],"qty":["3","7"]}`, `{"format":"{uppercase(w)} {uppercase(w)}","w":["a","b"]}`, } { - if _, err := linked(t, ok); err != nil { - t.Errorf("compile(%s) = %v, want it accepted", ok, err) + if _, err := f.NewTemplate(ok); err != nil { + t.Errorf("NewTemplate(%s) = %v, want it accepted", ok, err) } } } diff --git a/transform_test.go b/transform_test.go index 668ea6f..b6b2b38 100644 --- a/transform_test.go +++ b/transform_test.go @@ -39,6 +39,7 @@ func TestTransformOverAReference(t *testing.T) { } func TestTransformArgs(t *testing.T) { + f := newGenerator(t, writeData(t, map[string]string{"x": `"v"`})) for _, bad := range []string{ `{"format":"{lowercase()}","x":"v"}`, `{"format":"{lowercase(nope)}","x":"v"}`, @@ -46,8 +47,8 @@ func TestTransformArgs(t *testing.T) { `{"format":"{lowercase(hex(2))}","x":"v"}`, `{"format":"{lowercase(x)} {x.a}","x":{"format":"{a}","a":"1"}}`, } { - if _, err := linked(t, bad); err == nil { - t.Errorf("compile(%s) = nil error, want it rejected", bad) + if _, err := f.NewTemplate(bad); err == nil { + t.Errorf("NewTemplate(%s) = nil error, want it rejected", bad) } } } -- 2.52.0 From 317486df0acb811d44d19ebcb9eaaa47bcf706b0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:23:25 +0200 Subject: [PATCH 392/966] Fold checkNoOverlap and checkNoRepeatedRead into heldCheck, so compileFormat compiles and nothing more --- builtins_test.go | 6 +++--- format.go | 14 ++------------ holdfence.go | 15 ++++++++++++++- node.go | 8 ++++++++ reference.go | 11 ++--------- 5 files changed, 29 insertions(+), 25 deletions(-) diff --git a/builtins_test.go b/builtins_test.go index 5c30158..eb4d5fd 100644 --- a/builtins_test.go +++ b/builtins_test.go @@ -188,12 +188,12 @@ func ibanValid(s string) bool { return rem == 1 } -// TestRegistryShapes pins the builtin contract compileOps relies on: every entry +// TestRegistryShapes pins the builtin contract compileFormat relies on: every entry // supplies prep, and args parsed at compile only behind a check. func TestRegistryShapes(t *testing.T) { for name, b := range builtins { if b.prep == nil { - t.Errorf("builtin %q has no prep: compileOps would call a nil func", name) + t.Errorf("builtin %q has no prep: compileFormat would call a nil func", name) } if b.arity != 0 && b.checkArgs == nil { t.Errorf("builtin %q parses args in prep with no check", name) @@ -203,7 +203,7 @@ func TestRegistryShapes(t *testing.T) { // TestArgGuardsPanic pins the guards that report a builtin arg its check should // have rejected. No data reaches them — checkFunc runs a builtin's check before -// compileOps ever calls prep — so they are exercised directly. +// compileFormat ever calls prep — so they are exercised directly. func TestArgGuardsPanic(t *testing.T) { for name, call := range map[string]func(){ "atoi on an unvalidated arg": func() { atoi("nope") }, diff --git a/format.go b/format.go index 825f52b..2d3d4e3 100644 --- a/format.go +++ b/format.go @@ -441,19 +441,9 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { c.ops = append(c.ops, op{formatToken: tok, arms: arms}) } -// compileFormat compiles a format into ops, and applies the fences that need the -// compiled reads. -func compileFormat(toks []formatToken, refs map[string]refBinding) (formatOps, error) { - c := compileOps(toks, refs) - if err := checkNoOverlap(c); err != nil { - return c, err - } - return c, checkNoRepeatedRead(c) -} - -// compileOps compiles a parsed format. Call checkTokens first: it is what proves +// compileFormat compiles a parsed format. Call checkTokens first: it is what proves // every token valid. -func compileOps(toks []formatToken, refs map[string]refBinding) formatOps { +func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps { var c formatOps for _, tok := range toks { switch tok.kind { diff --git a/holdfence.go b/holdfence.go index fe19439..ad1ae91 100644 --- a/holdfence.go +++ b/holdfence.go @@ -15,10 +15,23 @@ func heldCheck(path string, n node) error { if !ok || len(t.compiled.held) == 0 { return nil } + if err := checkHeld(t); err != nil { + return fmt.Errorf("%s: %w", t.site.label(path), err) + } + return nil +} + +func checkHeld(t *template) error { + if err := checkNoOverlap(t.compiled); err != nil { + return err + } + if err := checkNoRepeatedRead(t.compiled); err != nil { + return err + } readers := pathHeadReaders(t.compiled) for _, name := range heldNames(t) { if err := checkNameHeld(t, name, readers); err != nil { - return fmt.Errorf("%s: %w", path, err) + return err } } return nil diff --git a/node.go b/node.go index 0725b7d..b36ba47 100644 --- a/node.go +++ b/node.go @@ -73,6 +73,14 @@ func (s tableSite) isCell() bool { return s.table != nil && s.row != formatRow } +// label names the template at path in an error, a cell by its line in the rows file. +func (s tableSite) label(path string) string { + if s.isCell() { + return fmt.Sprintf("%s, line %d", path, s.row+2) + } + return path +} + // templateLink is what a template resolves to in the assembled tree. type templateLink struct { refs map[string]refBinding // each reference the format reads -> what it resolves to diff --git a/reference.go b/reference.go index f2ac8cb..93c58c5 100644 --- a/reference.go +++ b/reference.go @@ -81,18 +81,11 @@ func linkRefs(root map[string]node) error { // linkTemplate resolves t's references in category, "" for an inline template, and // compiles its format against them. func linkTemplate(folder []string, path, category string, t *template, root map[string]node) error { - if t.site.isCell() { - path = fmt.Sprintf("%s, line %d", path, t.site.row+2) - } - link, err := t.resolveLink(folder, path, category, root) + link, err := t.resolveLink(folder, t.site.label(path), category, root) if err != nil { return err } - compiled, err := compileFormat(t.tokens, link.refs) - if err != nil { - return fmt.Errorf("%s: %w", path, err) - } - t.link, t.compiled = link, compiled + t.link, t.compiled = link, compileFormat(t.tokens, link.refs) compileArms(t) return nil } -- 2.52.0 From 71feec7db976252e62e4d85fedd2484729b0374c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:25:00 +0200 Subject: [PATCH 393/966] Drop the done heldCheck fold from the plan --- todo.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/todo.md b/todo.md index eefa87d..0b417aa 100644 --- a/todo.md +++ b/todo.md @@ -40,8 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Fold `checkNoOverlap` and `checkNoRepeatedRead` into `heldCheck`, so `compileFormat` - compiles and nothing more. - Fix at compile how `readField` reads an arm, `arm.kind`, deleting its two panics and its `held` lookup per read. - Switch `disagreement` on the fixed item's kind, with the other item inside each arm, -- 2.52.0 From c80d5016f616deaf0c5ca980d974484815b35908 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:29:24 +0200 Subject: [PATCH 394/966] Say heldCheck also rejects a held draw read two ways, and name its helper checkTemplateHold --- holdfence.go | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/holdfence.go b/holdfence.go index ad1ae91..a605187 100644 --- a/holdfence.go +++ b/holdfence.go @@ -6,22 +6,22 @@ import ( "strings" ) -// heldCheck rejects every route to a held sibling name except the ones that read its -// draw. An expansion holds one draw of that name; anything else that renders it draws -// again, and the two disagree. +// heldCheck rejects a format reading one held draw two ways, and every route to a held +// name except the readers of its draw. An expansion holds one draw of that name; +// anything else that renders it draws again, and the two disagree. // docs/decisions.md#the-expansion-hold-and-the-renders-draws-are-two-fences func heldCheck(path string, n node) error { t, ok := n.(*template) if !ok || len(t.compiled.held) == 0 { return nil } - if err := checkHeld(t); err != nil { + if err := checkTemplateHold(t); err != nil { return fmt.Errorf("%s: %w", t.site.label(path), err) } return nil } -func checkHeld(t *template) error { +func checkTemplateHold(t *template) error { if err := checkNoOverlap(t.compiled); err != nil { return err } -- 2.52.0 From efbad0f32558b938d4711240f30a9d01110ecc5b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:33:52 +0200 Subject: [PATCH 395/966] Require compile to fix how each arm is read, in place of readField's panics --- hold_test.go | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/hold_test.go b/hold_test.go index 560bb6f..acab7ff 100644 --- a/hold_test.go +++ b/hold_test.go @@ -761,18 +761,24 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) { } } -func TestReadFieldPanicsOnAPathItCannotHold(t *testing.T) { - a := arm{spelling: "w.x", head: "w", tail: []string{"x"}, path: "w.x"} - for name, format := range map[string]string{ - "unheld arm with a path": `{"format":"{w}","w":{"format":"{x}","x":"1"}}`, - "held arm with no hold": `{"format":"{w.x}","w":{"format":"{x}","x":"1"}}`, - } { - tm, ok := compiled(t, format).(*template) - if !ok { - t.Fatal("not a template") +func TestCompileFixesHowEachArmIsRead(t *testing.T) { + f, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ + "cat": `{"format":"{w} {w} {p.x} {/other.y} {n} {uppercase(n)}","w":["a","b"],"n":["c","d"],"p":{"format":"{x}","x":"1"}}`, + "other": `{"format":"{y}","y":"2"}`, + }))) + if err != nil { + t.Fatal(err) + } + tm, ok := f.root.children["cat"].(*template) + if !ok { + t.Fatal("cat is not a template") + } + want := map[string]armKind{"/other.y": refPathRead, "n": heldRead, "p.x": heldRead, "w": freshRead} + for _, o := range tm.compiled.ops { + for _, a := range append(o.arms, o.operands...) { + if a.kind != want[a.spelling] { + t.Errorf("{%s} compiles to kind %d, want %d", a.spelling, a.kind, want[a.spelling]) + } } - mustPanic(t, name, func() { - readField(engine(1).rand, tm, nil, renderScope{draws: &renderDraws{}}, a) - }) } } -- 2.52.0 From 26b35abe7064cbdc85050103103957a17afaa742 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:35:41 +0200 Subject: [PATCH 396/966] Fix at compile how readField reads an arm, deleting its two panics and its held lookup per read --- format.go | 10 ++++++++++ reference.go | 6 ++++++ render.go | 10 ++-------- 3 files changed, 18 insertions(+), 8 deletions(-) diff --git a/format.go b/format.go index 2d3d4e3..c9f44d6 100644 --- a/format.go +++ b/format.go @@ -315,8 +315,18 @@ type arm struct { steps []pathStep leaves []node // every node the path may land on, one per variant it passes cover node // what holding the path pins: the first choice it passes, else its leaf + kind armKind } +// armKind is how expand reads an arm, fixed at compile. +type armKind uint8 + +const ( + freshRead armKind = iota // drawn afresh at every read + heldRead // drawn once per expansion, kept in its hold + refPathRead // drawn once per draw group, kept in its memo +) + func (a arm) isRefPath() bool { return isRef(a.head) && len(a.tail) > 0 } diff --git a/reference.go b/reference.go index 93c58c5..8eae402 100644 --- a/reference.go +++ b/reference.go @@ -112,6 +112,12 @@ func compileArm(t *template, a *arm) { } w := compilePath(head, a.tail) a.steps, a.leaves, a.cover = w.steps, w.leaves, w.cover + switch _, held := t.compiled.held[a.head]; { + case a.isRefPath(): + a.kind = refPathRead + case held: + a.kind = heldRead + } } // resolveLink binds every reference t reads, refusing one to t's own category, and keys diff --git a/render.go b/render.go index 3982afd..abc02b6 100644 --- a/render.go +++ b/render.go @@ -158,21 +158,15 @@ func expand(s *generatorState, t *template, sc renderScope) string { // read one row, either read twice gives one value, and a shown operand is the operand // computed. Every other name is drawn afresh, so {word} {word} still draws twice. func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue { - if a.isRefPath() { + if a.kind == refPathRead { return readReference(s, t, sc, a) } if sc.draws.trace != nil { traceRead(sc.draws.trace, t, sc, a) } - if _, expansionHolds := t.compiled.held[a.head]; !expansionHolds { - if len(a.tail) > 0 { - panic(internalError("%q reads a path into %q, which the expansion does not hold", a.spelling, a.head)) - } + if a.kind == freshRead { return readValue{text: render(s, t.head(a.head), sc)} } - if hold == nil { - panic(internalError("%q reads a name the expansion holds, with no hold to keep it in", a.spelling)) - } return readMemo(s, t, hold, nil, sc, a) } -- 2.52.0 From 55b127856b53dd209fbc5165b0b9cc9c7e244c9f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:35:41 +0200 Subject: [PATCH 397/966] Drop the done readField item from the plan --- todo.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/todo.md b/todo.md index 0b417aa..9273f7a 100644 --- a/todo.md +++ b/todo.md @@ -40,8 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Fix at compile how `readField` reads an arm, `arm.kind`, deleting its two panics and - its `held` lookup per read. - Switch `disagreement` on the fixed item's kind, with the other item inside each arm, and rename its `proven` local, which shadows the `proven` type: seven seats read the seven-arm switch as a truth table without the table. -- 2.52.0 From faa4bba89895b0f4300b30973c10bc31c81ec756 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:37:47 +0200 Subject: [PATCH 398/966] Fail the arm kind test on an arm it lists no kind for --- hold_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hold_test.go b/hold_test.go index acab7ff..8bc7e23 100644 --- a/hold_test.go +++ b/hold_test.go @@ -776,8 +776,8 @@ func TestCompileFixesHowEachArmIsRead(t *testing.T) { want := map[string]armKind{"/other.y": refPathRead, "n": heldRead, "p.x": heldRead, "w": freshRead} for _, o := range tm.compiled.ops { for _, a := range append(o.arms, o.operands...) { - if a.kind != want[a.spelling] { - t.Errorf("{%s} compiles to kind %d, want %d", a.spelling, a.kind, want[a.spelling]) + if k, ok := want[a.spelling]; !ok || a.kind != k { + t.Errorf("{%s} compiles to kind %d, want %d", a.spelling, a.kind, k) } } } -- 2.52.0 From 57f48631071f4cc2ace9cc4a4e2a1df5503df86b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:51:35 +0200 Subject: [PATCH 399/966] Switch disagreement on the fixed item's kind, and rename its proven local to fits --- datatype.go | 26 ++++++++++++++++---------- todo.md | 3 --- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/datatype.go b/datatype.go index 1cc5251..432a7e9 100644 --- a/datatype.go +++ b/datatype.go @@ -174,19 +174,25 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { fix, other, want = b, a, da } held := fmt.Sprintf("its items hold %s and %s; a column holds one datatype", da, db) - proven := (&valueProof{}).proveColumnItem(fix).not[want] == "" - switch { - case kindOf(fix) == kindDeclares: + fits := (&valueProof{}).proveColumnItem(fix).not[want] == "" + switch kindOf(fix) { + case kindDeclares: return errors.New(held) - case kindOf(fix) == kindReads && proven: - return fmt.Errorf("%s, so %s", held, typedAs(fix, want)) - case kindOf(fix) == kindReads: + case kindReads: + if fits { + return fmt.Errorf("%s, so %s", held, typedAs(fix, want)) + } return fmt.Errorf("%s, so to read %q as text, %s", held, fix.format, asText(fix)) - case kindOf(other) == kindText: + } + switch kindOf(other) { + case kindText: panic(internalError("two text items hold one datatype, so they never disagree")) - case kindOf(other) == kindReads && !proven: - return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) - case fix.fromString: // an object may carry a weight, which this spelling would drop + case kindReads: + if !fits { + return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) + } + } + if fix.fromString { // an object may carry a weight, which this spelling would drop return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, fix.format, fix.format, want) } return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, fix.format, want, want) diff --git a/todo.md b/todo.md index 9273f7a..29a3956 100644 --- a/todo.md +++ b/todo.md @@ -40,9 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Switch `disagreement` on the fixed item's kind, with the other item inside each arm, - and rename its `proven` local, which shadows the `proven` type: seven seats read the - seven-arm switch as a truth table without the table. - Say at `checkNodeFences` that its walks recurse unguarded, so a caller runs `checkNoCycles` first: five seats reconstructed the order, and the inline binding runs no cycle check. -- 2.52.0 From 4ed4b10fa23cecbdc844eb197925669858df67b0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 20:57:48 +0200 Subject: [PATCH 400/966] Nest disagreement's other-item switch in the fixed item's text arm --- datatype.go | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/datatype.go b/datatype.go index 432a7e9..9754938 100644 --- a/datatype.go +++ b/datatype.go @@ -183,19 +183,21 @@ func disagreement(a *template, da DataType, b *template, db DataType) error { return fmt.Errorf("%s, so %s", held, typedAs(fix, want)) } return fmt.Errorf("%s, so to read %q as text, %s", held, fix.format, asText(fix)) - } - switch kindOf(other) { case kindText: - panic(internalError("two text items hold one datatype, so they never disagree")) - case kindReads: - if !fits { - return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) + switch kindOf(other) { + case kindText: + panic(internalError("two text items hold one datatype, so they never disagree")) + case kindReads: + if !fits { + return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, fix.format, dataTypeNouns[want], other.format, asText(other)) + } } + if fix.fromString { // an object may carry a weight, which this spelling would drop + return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, fix.format, fix.format, want) + } + return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, fix.format, want, want) } - if fix.fromString { // an object may carry a weight, which this spelling would drop - return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, fix.format, fix.format, want) - } - return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, fix.format, want, want) + panic(internalError("disagreement has no case for item kind %d", kindOf(fix))) } // typedAs names the spelling giving a column-read item datatype d, keeping the other keys an object -- 2.52.0 From e9b8216116ed00d8f3b5984e0e339dfbccb22dc1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:32:49 +0200 Subject: [PATCH 401/966] Say at checkNodeFences that its scope's cycles are refused first --- data.go | 4 +++- todo.md | 3 --- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/data.go b/data.go index 5ee88f1..4583343 100644 --- a/data.go +++ b/data.go @@ -230,7 +230,9 @@ func (b binding) bind() error { // checkNodeFences runs the per-node fences every binding needs over a scope, each // over the whole scope before the next, so which of several broken nodes is reported -// does not depend on the walk. Its walks terminate only where nothing renders itself. +// does not depend on the walk. Its walks recurse unguarded, so the scope's cycles are +// refused first: a tree by checkNoCycles, and an inline node closes none, since nothing +// references it. func checkNodeFences(s nodeScope) error { mem := renderCounts{} if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { diff --git a/todo.md b/todo.md index 29a3956..ca75024 100644 --- a/todo.md +++ b/todo.md @@ -40,9 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Say at `checkNodeFences` that its walks recurse unguarded, so a caller runs - `checkNoCycles` first: five seats reconstructed the order, and the inline binding runs - no cycle check. - Give `tableSite` an `isFormat` beside `isCell`, so `renderRecord` and `traceRead` stop comparing `row` with `formatRow` alone, which holds only because a template outside a table has row 0: four seats found the invariant unstated. -- 2.52.0 From 492772b995de1d33a7db7add1ec4ac6cf1c53e46 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:37:10 +0200 Subject: [PATCH 402/966] Give tableSite an isFormat beside isCell --- node.go | 4 ++++ record.go | 2 +- render.go | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/node.go b/node.go index b36ba47..3a0f739 100644 --- a/node.go +++ b/node.go @@ -73,6 +73,10 @@ func (s tableSite) isCell() bool { return s.table != nil && s.row != formatRow } +func (s tableSite) isFormat() bool { + return s.table != nil && s.row == formatRow +} + // label names the template at path in an error, a cell by its line in the rows file. func (s tableSite) label(path string) string { if s.isCell() { diff --git a/record.go b/record.go index 4d76149..3b001e7 100644 --- a/record.go +++ b/record.go @@ -263,7 +263,7 @@ func recordOf(n node) (*template, []Column, error) { // over sc's draws; a table's columns read the row pinned there. func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) - if t.site.row == formatRow { + if t.site.isFormat() { sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins) } r := &Record{columns: append([]Column(nil), columns...)} diff --git a/render.go b/render.go index abc02b6..9ef18d7 100644 --- a/render.go +++ b/render.go @@ -201,7 +201,7 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { switch { case t.site.isCell(): row = renderedRow{t.site.table, t.site.row} - case t.site.row == formatRow: + case t.site.isFormat(): row = renderedRow{t.site.table, sc.rowOf(t.site.table)} } trace(strings.Clone(sc.group), row, a) -- 2.52.0 From 16d6497396d938dac7faafdc8cb7fa01db8d388d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:37:10 +0200 Subject: [PATCH 403/966] Read isFormat in the render test helpers --- reads_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/reads_test.go b/reads_test.go index 2d0ffe7..29dd106 100644 --- a/reads_test.go +++ b/reads_test.go @@ -81,7 +81,7 @@ func renderRoot(s *generatorState, t *template, trace renderTrace) { draws := renderDraws{trace: trace} sc := renderScope{draws: &draws} switch { - case t.site.row == formatRow: + case t.site.isFormat(): tbl := t.site.table render(s, tbl, sc) for r := 0; r < tbl.rowCount(); r++ { @@ -100,7 +100,7 @@ func renderRoot(s *generatorState, t *template, trace renderTrace) { func renderRecordRoot(s *generatorState, t *template, columns []Column, trace renderTrace) { draws := renderDraws{trace: trace} sc := renderScope{draws: &draws} - if t.site.row == formatRow { + if t.site.isFormat() { t.site.table.drawIn(s, &sc.groupDraws().pins) } renderRecord(s, t, columns, sc) -- 2.52.0 From cb7b43609f70a38cdc965d8dbcc56f409de8d0b9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:38:32 +0200 Subject: [PATCH 404/966] Drop the done isFormat item from the plan --- todo.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/todo.md b/todo.md index ca75024..3db5960 100644 --- a/todo.md +++ b/todo.md @@ -40,9 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Give `tableSite` an `isFormat` beside `isCell`, so `renderRecord` and `traceRead` stop - comparing `row` with `formatRow` alone, which holds only because a template outside a - table has row 0: four seats found the invariant unstated. - Move `valueProof.checkField` to `valueproof.go`, and rename `checkPath`, which differs from the `pathCheck` type it wraps by word order alone: both architects placed them wrong. -- 2.52.0 From 698ac036d58f9ad1b828d826f7f22bf8d1b3429d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:43:24 +0200 Subject: [PATCH 405/966] Move checkField to valueproof.go and rename checkPath to checkResolves --- README.md | 2 +- format.go | 2 +- path.go | 4 ++-- reference.go | 2 +- struct.go | 31 ------------------------------- todo.md | 3 --- valueproof.go | 32 ++++++++++++++++++++++++++++++++ 7 files changed, 37 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index 5ae2e24..ce80187 100644 --- a/README.md +++ b/README.md @@ -902,7 +902,7 @@ checksum.go the check characters a derivation appends, and the IBAN they sit transform.go the builtins that rewrite an operand's value, and the ASCII folding calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype -valueproof.go the value proof: what a typed column or calc operand holds, checked at load +valueproof.go the value proof: what a typed column, calc operand or struct field holds, checked at load data.go the load path: fs.FS folders/files -> merged namespace tree, and the bind pipeline every entry point runs cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc diff --git a/format.go b/format.go index c9f44d6..db76043 100644 --- a/format.go +++ b/format.go @@ -262,7 +262,7 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { } return fmt.Errorf("no field %q", a.head) } - if err := checkPath(field, a.tail, a.head); err != nil { + if err := checkResolves(field, a.tail, a.head); err != nil { return fmt.Errorf("field %q: %w", a.head, err) } return nil diff --git a/path.go b/path.go index b2ae912..5891486 100644 --- a/path.go +++ b/path.go @@ -472,12 +472,12 @@ func unreachableInChoice(c *choice, want string) error { return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } -func checkPath(n node, tail []string, level string) error { +func checkResolves(n node, tail []string, level string) error { _, err := (&pathCheck{level: level, tail: tail}).run(n) return err } -// compilePath compiles a path checkPath proved. +// compilePath compiles a path checkResolves proved. func compilePath(n node, tail []string) pathCheck { w := pathCheck{tail: tail} if _, err := w.run(n); err != nil { diff --git a/reference.go b/reference.go index 8eae402..92e65f3 100644 --- a/reference.go +++ b/reference.go @@ -147,7 +147,7 @@ func (t *template) resolveLink(folder []string, path, category string, root map[ if category != "" && head == "/"+category { return link, fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) } - if err := checkPath(target, tail, head); err != nil { + if err := checkResolves(target, tail, head); err != nil { return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } link.refHeads[head] = target diff --git a/struct.go b/struct.go index eb38a56..3016a68 100644 --- a/struct.go +++ b/struct.go @@ -317,37 +317,6 @@ func (k columnKind) holds(v proven) bool { return v.lo >= k.lo && v.hi <= k.hi } -// checkField rejects a column a field of Go type ft cannot fill: a datatype, which the Go type -// sets, a null outside a pointer, or a value its kind's datatype or range refuses. -func (p *valueProof) checkField(label string, ft reflect.Type, column node) error { - items, _ := columnItems(column) - for _, it := range items { - if it.datatype != DataTypeString { - return fmt.Errorf("%s: its Go type %s sets the datatype; drop \"datatype\"", label, ft) - } - } - elem := ft - if ft.Kind() == reflect.Pointer { - elem = ft.Elem() - } else if p.proveColumn(column).nullable { - return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft) - } - kind := columnKinds[elem.Kind()] - if kind.datatype == DataTypeString { - return nil - } - for _, it := range items { - v := p.proveColumnItem(it) - if reason := v.not[kind.datatype]; reason != "" { - return fmt.Errorf("%s (%s): %s", label, ft, reason) - } - if !kind.holds(v) { - return fmt.Errorf("%s (%s): %q is not proven within %s; narrow it to that range, or make the field %s", label, ft, it.format, elem.Kind(), kind.wider) - } - } - return nil -} - // fill draws the record into v's tagged fields, then each nested struct as a record of its own. func (s *structShape) fill(state *generatorState, v reflect.Value) { if s.record != nil { diff --git a/todo.md b/todo.md index 3db5960..22e5dec 100644 --- a/todo.md +++ b/todo.md @@ -40,9 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Move `valueProof.checkField` to `valueproof.go`, and rename `checkPath`, which differs - from the `pathCheck` type it wraps by word order alone: both architects placed them - wrong. - Correct the claims readers followed wrong: `IsTemplate`'s doc says an argument holding a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; the README says rows are indexed on the first draw, where `New` builds the key map; and diff --git a/valueproof.go b/valueproof.go index 0ac6217..6a8bf3e 100644 --- a/valueproof.go +++ b/valueproof.go @@ -3,6 +3,7 @@ package fejkdata import ( "fmt" "math" + "reflect" "regexp" "slices" "strconv" @@ -43,6 +44,37 @@ func (p *valueProof) checkDatatype(path string, n node) error { return nil } +// checkField rejects a column a field of Go type ft cannot fill: a datatype, which the Go type +// sets, a null outside a pointer, or a value its kind's datatype or range refuses. +func (p *valueProof) checkField(label string, ft reflect.Type, column node) error { + items, _ := columnItems(column) + for _, it := range items { + if it.datatype != DataTypeString { + return fmt.Errorf("%s: its Go type %s sets the datatype; drop \"datatype\"", label, ft) + } + } + elem := ft + if ft.Kind() == reflect.Pointer { + elem = ft.Elem() + } else if p.proveColumn(column).nullable { + return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft) + } + kind := columnKinds[elem.Kind()] + if kind.datatype == DataTypeString { + return nil + } + for _, it := range items { + v := p.proveColumnItem(it) + if reason := v.not[kind.datatype]; reason != "" { + return fmt.Errorf("%s (%s): %s", label, ft, reason) + } + if !kind.holds(v) { + return fmt.Errorf("%s (%s): %q is not proven within %s; narrow it to that range, or make the field %s", label, ft, it.format, elem.Kind(), kind.wider) + } + } + return nil +} + // proveColumnItem proves a column item: what it renders, or, when it is a column it reads, that column. func (p *valueProof) proveColumnItem(t *template) proven { if t.link.readsColumn == nil { -- 2.52.0 From 0b5124d4d5869f93bb4e0bc67bf31f33a7017fc3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:45:26 +0200 Subject: [PATCH 406/966] Rename checkResolves to checkPathResolves --- format.go | 2 +- path.go | 4 ++-- reference.go | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/format.go b/format.go index db76043..25c7d08 100644 --- a/format.go +++ b/format.go @@ -262,7 +262,7 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { } return fmt.Errorf("no field %q", a.head) } - if err := checkResolves(field, a.tail, a.head); err != nil { + if err := checkPathResolves(field, a.tail, a.head); err != nil { return fmt.Errorf("field %q: %w", a.head, err) } return nil diff --git a/path.go b/path.go index 5891486..335670c 100644 --- a/path.go +++ b/path.go @@ -472,12 +472,12 @@ func unreachableInChoice(c *choice, want string) error { return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } -func checkResolves(n node, tail []string, level string) error { +func checkPathResolves(n node, tail []string, level string) error { _, err := (&pathCheck{level: level, tail: tail}).run(n) return err } -// compilePath compiles a path checkResolves proved. +// compilePath compiles a path checkPathResolves proved. func compilePath(n node, tail []string) pathCheck { w := pathCheck{tail: tail} if _, err := w.run(n); err != nil { diff --git a/reference.go b/reference.go index 92e65f3..80080e8 100644 --- a/reference.go +++ b/reference.go @@ -147,7 +147,7 @@ func (t *template) resolveLink(folder []string, path, category string, root map[ if category != "" && head == "/"+category { return link, fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) } - if err := checkResolves(target, tail, head); err != nil { + if err := checkPathResolves(target, tail, head); err != nil { return link, fmt.Errorf("%s: reference {%s}: %w", path, name, err) } link.refHeads[head] = target -- 2.52.0 From 4fb81a56729f5c554703ac47a09b0c958e9a222c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:50:50 +0200 Subject: [PATCH 407/966] Correct IsTemplate's, loadDir's and the README's row-index claims --- README.md | 4 ++-- data.go | 3 +-- inline.go | 8 ++++---- todo.md | 5 ----- 4 files changed, 7 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index ce80187..af454d3 100644 --- a/README.md +++ b/README.md @@ -491,8 +491,8 @@ table, `{/misc.currency[SEK].symbol}` on one row and `{/misc.currency[EUR].symbo on the next, since only one row renders. `New` proves the header, the options and every cell token, and refuses a TSV no category names, a key that is empty or repeats, a weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`, -`"` or `|`, which a selector cannot spell; the rows are indexed on the first draw that -selects one. A `name` needs a `key`, since a name naming several rows is reported by +`"` or `|`, which a selector cannot spell; `New` maps the keys, and the lookups by name +and by parent wait for the first draw that selects one. A `name` needs a `key`, since a name naming several rows is reported by their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]` is settled by the `sex` selected before it and a name repeating inside one parent row is refused; a name spelling another row's key is refused, since the key would diff --git a/data.go b/data.go index 4583343..e698ae9 100644 --- a/data.go +++ b/data.go @@ -61,8 +61,7 @@ func loadData(sources []dataSource) (map[string]node, error) { return root, nil } -// loadDir compiles one directory into a folder. fs.ReadDir yields entries sorted -// by name, so the tree is built deterministically. Empty subdirectories (no JSON +// loadDir compiles one directory into a folder. Empty subdirectories (no JSON // anywhere under them) are skipped rather than added as empty namespaces. func loadDir(src dataSource, dir string) (*folder, error) { entries, err := fs.ReadDir(src.fsys, dir) diff --git a/inline.go b/inline.go index b090ecd..1b13b61 100644 --- a/inline.go +++ b/inline.go @@ -50,10 +50,10 @@ func (f *Generator) FakeTemplate(input string) (string, error) { } // IsTemplate reports whether arg is an inline template rather than a path, by its shape: a { -// token, or a JSON object, array or string, is a template, and anything else is a path. A -// name never holds a bracket, a brace or a quote, so an arg holding one that is not valid -// JSON names neither, and errors; so do a template of one reference alone, which is a path -// written as a template, and a path written with a leading /. +// token, or a JSON object, array or string, is a template, and anything else is a path. An +// arg that is not valid JSON errors where it holds a } or a quote, or starts with [, since no +// path may; so do a template of one reference alone, which is a path written as a template, +// and a path written with a leading /. func IsTemplate(arg string) (bool, error) { inline, err := isTemplate(arg) if err != nil { diff --git a/todo.md b/todo.md index 22e5dec..9755137 100644 --- a/todo.md +++ b/todo.md @@ -40,11 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Correct the claims readers followed wrong: `IsTemplate`'s doc says an argument holding - a bracket that is not valid JSON errors, where `misc.territory[SE]` is a path; the - README says rows are indexed on the first draw, where `New` builds the key map; and - `loadDir`'s says it builds the tree deterministically, where it reports an unnamed - rows file in map order. - Delete the comments the panel found restating their code or arranging the file: `node`'s second sentence, `op`'s "compile builds these so render never re-scans the format", `checkCalc`'s last sentence, `calcVar.eval`'s four lines, `checkNoCycles`' -- 2.52.0 From a24f52a1f0588fd597a3d1e08a2f538165464c7c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:52:57 +0200 Subject: [PATCH 408/966] Say which input errors in IsTemplate, and which columns wait for the first draw --- README.md | 10 +++++----- inline.go | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index af454d3..354a080 100644 --- a/README.md +++ b/README.md @@ -491,8 +491,8 @@ table, `{/misc.currency[SEK].symbol}` on one row and `{/misc.currency[EUR].symbo on the next, since only one row renders. `New` proves the header, the options and every cell token, and refuses a TSV no category names, a key that is empty or repeats, a weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`, -`"` or `|`, which a selector cannot spell; `New` maps the keys, and the lookups by name -and by parent wait for the first draw that selects one. A `name` needs a `key`, since a name naming several rows is reported by +`"` or `|`, which a selector cannot spell; `New` maps the keys, and a name or parent column +is mapped on the first draw that selects by name or descends through the table. A `name` needs a `key`, since a name naming several rows is reported by their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]` is settled by the `sex` selected before it and a name repeating inside one parent row is refused; a name spelling another row's key is refused, since the key would @@ -760,9 +760,9 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance -Each file is parsed, validated and weight-indexed once, in `New`, bar a table with a -name or a parent column, which indexes its rows on the first `Fake` that selects or -descends through it. Proving the draw fences adds a pass over the loaded tree per +Each file is parsed, validated and weight-indexed once, in `New`, bar a table's name and +parent columns, which are mapped on the first `Fake` that selects by name or descends +through it. Proving the draw fences adds a pass over the loaded tree per fence, and walks what a render reads only where data binds a reference, so a set that binds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one diff --git a/inline.go b/inline.go index 1b13b61..f03fda9 100644 --- a/inline.go +++ b/inline.go @@ -49,11 +49,11 @@ func (f *Generator) FakeTemplate(input string) (string, error) { return t.Fake(), nil } -// IsTemplate reports whether arg is an inline template rather than a path, by its shape: a { -// token, or a JSON object, array or string, is a template, and anything else is a path. An -// arg that is not valid JSON errors where it holds a } or a quote, or starts with [, since no -// path may; so do a template of one reference alone, which is a path written as a template, -// and a path written with a leading /. +// IsTemplate reports whether arg is an inline template rather than a path, by its shape: an +// arg holding a {, or a valid JSON array or string, is a template, and anything else is a +// path. A path errors where it holds a } or a quote, starts with [, or is no valid path; so +// do a template of one reference alone, which is a path written as a template, and a path +// written with a leading /. func IsTemplate(arg string) (bool, error) { inline, err := isTemplate(arg) if err != nil { -- 2.52.0 From a9448d1a9974e9fff0e5d93a53bf0128d683583f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:57:52 +0200 Subject: [PATCH 409/966] Delete comments restating their code, and doc.go's partial list of draw fences --- calc.go | 8 +------- cmd/fejkdata/main.go | 1 - data.go | 1 - doc.go | 4 ++-- format.go | 3 +-- graph.go | 4 +--- node.go | 4 +--- path.go | 1 - render.go | 1 - todo.md | 8 -------- 10 files changed, 6 insertions(+), 29 deletions(-) diff --git a/calc.go b/calc.go index 04879ef..081fd52 100644 --- a/calc.go +++ b/calc.go @@ -33,10 +33,6 @@ func (n calcIdx) eval(operands []string) float64 { return v } -// eval on an unplaced name cannot happen: calcPrep runs indexVars over every -// expression it compiles, so only a calcIdx reaches a render. It panics rather -// than returning NaN, so a node kind indexVars forgets is a stack trace and not a -// silently wrong number. func (n calcVar) eval([]string) float64 { panic(internalError("calc operand %q was never placed", string(n))) } @@ -58,9 +54,7 @@ func (n calcBin) eval(operands []string) float64 { } // checkCalc validates a calc token at compile time: a parseable expression whose -// operands all name existing fields, and an optional non-negative integer dp. It -// is a builtin check (fields first), so calc dispatches through the registry like -// every other {name(args)} function. +// operands all name existing fields, and an optional non-negative integer dp. func checkCalc(fields map[string]node, args []string) error { if len(args) < 1 || len(args) > 2 { return fmt.Errorf("calc takes an expression and an optional decimals count, got %d args", len(args)) diff --git a/cmd/fejkdata/main.go b/cmd/fejkdata/main.go index 04f8d86..36e116f 100644 --- a/cmd/fejkdata/main.go +++ b/cmd/fejkdata/main.go @@ -234,7 +234,6 @@ var recordFormats = map[string]recordFormat{ func jsonLine(r *fejkdata.Record, _ string) string { return r.JSON() } -// writesRecords reports whether the format writes records rather than plain text. func (in invocation) writesRecords() bool { return in.format != "text" } diff --git a/data.go b/data.go index e698ae9..942fb4a 100644 --- a/data.go +++ b/data.go @@ -106,7 +106,6 @@ type categoryFiles struct { tsv map[string]bool } -// readRows reads the rows file a category names beside it. func (c *categoryFiles) readRows(name string) (string, error) { if _, present := c.tsv[name]; !present { return "", fmt.Errorf("rows names %s, which is not beside it in %s", name, c.src.labelled(c.dir)) diff --git a/doc.go b/doc.go index 639dfd5..2f0fd96 100644 --- a/doc.go +++ b/doc.go @@ -32,7 +32,7 @@ package fejkdata // - draw group — reference draws kept apart inside one render: // `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a // render reads it, `renderDraws`, `groupDraws`, `renderScope`. -// - pin — fixing which row of a table the render uses, which the draw fences replay: +// - pin — fixing which row of a table the render uses, which the fences replay: // `pinSet`, `pinSet.pin`, `table.drawIn`. // - family — a table and every table reaching it through a chain of // `table.parentT`, named by the root that chain ends at: `table.familyRoot`. @@ -40,4 +40,4 @@ package fejkdata // the `table` itself, so it draws a row apart from every pin: `branches.wholePins`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // runs them over the data set, `NewTemplate` and `FakeStruct` over what those -// compile. The draw fences are `heldCheck` and `drawFence`. +// compile. diff --git a/format.go b/format.go index 25c7d08..e962806 100644 --- a/format.go +++ b/format.go @@ -386,8 +386,7 @@ func checkSegments(a arm) error { type callFn func(s *generatorState, emitted string, operands []string) string // op is one compiled unit of a format string: a literal run, a name read, -// or a builtin already prepared with its args. compile builds these so render never -// re-scans the format. +// or a builtin already prepared with its args. type op struct { formatToken arms []arm // nameRead: the '|' alternatives, split into head and tail once diff --git a/graph.go b/graph.go index 88ee6fe..5eea0dc 100644 --- a/graph.go +++ b/graph.go @@ -193,9 +193,7 @@ func repeatCheck(path string, n node, mem renderCounts) error { // checkNoCycles rejects a reference cycle: a node whose rendering can reach itself // — directly, mutually, or through a chain — never terminates, so it must fail at // New rather than stack-overflow at render. It is a depth-first walk of the render -// graph (renderEdges); grey marks nodes on the current path so a back-edge to one -// is the cycle, while black lets a shared node (a DAG, not a cycle) be skipped. -// Every node is a root: a field its parent's format never renders is still reachable +// graph (renderEdges). Every node is a root: a field its parent's format never renders is still reachable // by dot path, so a cycle in one would otherwise reach render and be fatal there. func checkNoCycles(root map[string]node) error { const ( diff --git a/node.go b/node.go index 3a0f739..841a324 100644 --- a/node.go +++ b/node.go @@ -10,8 +10,7 @@ import ( ) // node is a compiled element of the namespace tree: a folder, choice, null, -// template, table, column or row. Compiling JSON into these once (see compile) -// means rendering never re-inspects the raw JSON. +// template, table, column or row. type node interface{ isNode() } // folder is a namespace of named children, built from a directory of JSON files @@ -513,7 +512,6 @@ func weightOf(raw any) (float64, error) { // than where it would be unreachable. const reservedInName = ".|({}/[]\"" -// reservedList spells reservedInName for an error message, so the two cannot drift. var reservedList = strings.Join(strings.Split(reservedInName, ""), " ") // checkName rejects a name the dot path, {token} and JSON grammars cannot spell, or a struct diff --git a/path.go b/path.go index 335670c..ed99016 100644 --- a/path.go +++ b/path.go @@ -87,7 +87,6 @@ func indexOutside(s string, c byte) int { return -1 } -// splitOutside splits s on c outside any [selector]. func splitOutside(s string, c byte) []string { var parts []string for { diff --git a/render.go b/render.go index 9ef18d7..b9d3901 100644 --- a/render.go +++ b/render.go @@ -115,7 +115,6 @@ func pick(s *generatorState, c *choice) node { return c.items[pickCum(s, c.cum)] } -// expand renders a template's compiled ops. func expand(s *generatorState, t *template, sc renderScope) string { var b strings.Builder b.Grow(t.compiled.grow) diff --git a/todo.md b/todo.md index 9755137..5afafa7 100644 --- a/todo.md +++ b/todo.md @@ -40,14 +40,6 @@ maintainer chose on 2026-09-30 to go on with the path steps regardless. This round, in order: -- Delete the comments the panel found restating their code or arranging the file: - `node`'s second sentence, `op`'s "compile builds these so render never re-scans the - format", `checkCalc`'s last sentence, `calcVar.eval`'s four lines, `checkNoCycles`' - grey and black, and the docs on `expand`, `readRows`, `splitOutside`, `reservedList` - and `writesRecords`. -- Delete `doc.go`'s "The draw fences are …" sentence, which misses `checkDrawGroup`, - `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by - file. Say "fences" in the `pin` entry then, which names the draw fences too. - Run the nine-seat panel again and file what it names here. ### Data -- 2.52.0 From 5cb919eb3c4801d4dab8632528aa547779e83589 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 21:58:08 +0200 Subject: [PATCH 410/966] Rewrap checkNoCycles' doc --- graph.go | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/graph.go b/graph.go index 5eea0dc..098fe5a 100644 --- a/graph.go +++ b/graph.go @@ -193,8 +193,9 @@ func repeatCheck(path string, n node, mem renderCounts) error { // checkNoCycles rejects a reference cycle: a node whose rendering can reach itself // — directly, mutually, or through a chain — never terminates, so it must fail at // New rather than stack-overflow at render. It is a depth-first walk of the render -// graph (renderEdges). Every node is a root: a field its parent's format never renders is still reachable -// by dot path, so a cycle in one would otherwise reach render and be fatal there. +// graph (renderEdges). Every node is a root: a field its parent's format never +// renders is still reachable by dot path, so a cycle in one would otherwise reach +// render and be fatal there. func checkNoCycles(root map[string]node) error { const ( grey = 1 -- 2.52.0 From c1d367bbc0b489342d6b9c4bf8613f3f6da7e5f6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 22:08:46 +0200 Subject: [PATCH 411/966] File the nine-seat run after the path steps, which held the score at 5.7, and stop the round for the maintainer's call --- todo.md | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index 5afafa7..eb3693e 100644 --- a/todo.md +++ b/todo.md @@ -36,10 +36,41 @@ on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-suffic checkpoint (Navigation 6.8, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), every seat's lowest dimension Locality, alone or tied, all seven unit seats ranking `drawsApart` or `readFold.reads` hardest and five `drawsApart` the unit they would least modify. The -maintainer chose on 2026-09-30 to go on with the path steps regardless. +maintainer chose on 2026-09-30 to go on with the path steps regardless; 5.7 on +2026-09-30 after them (Navigation 7.0, Locality 5.1, Shape 5.9, Self-sufficiency 5.7), +every seat's lowest dimension Locality, alone or tied, six of the seven unit seats +ranking `drawsApart` hardest and all seven the unit they would least modify. This round, in order: +- **Take the maintainer's call on the items below**: the approved restructure is done and + left the score at 5.7, so the round stops here until they decide. +- **Split `drawsApart` into one named check per refusal, and show at `conflict` why pairs + find what a replay would**: all seven unit seats would least modify it, holding four + refusals over `landsRow`, `familyRoot`, `drawnOf` and `selected` at once, and four found + the pairs claim credible only through `TestPairsFindWhatAWholeReplayFinds`. +- **Walk a path in one function**: `probePath` copies `pathCheck.walk`'s loop and + `routeSteps` call, differing at a choice and in skipping `enter`, and seven seats and + both architects had to diff them by eye. +- **Name what `branches` and `pinBelow` hold and drop**: seven seats ranked + `readFold.reads` second, decoding `pins` against `wholePins` through `doc.go` and + finding `pinBelow`'s silent `continue` only by tracing. +- **Fill `arm` in one place**: `splitArm` runs before the link and again after it, and + `compileArm` finishes it from `reference.go`, so five seats traced three phases and both + architects found format compilation under `linkTemplate`. +- **Rename `holdfence.go`'s `cover`, and say at `heldNodes` why a path hold and an + operand hold answer for different nodes**: `arm.cover` means something else, and six + seats held both walks beside `renderEdges` to see why they differ. +- **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their + own, stating there that `checkOwnFamily` runs before the draw surveys**: both architects + found the fence order last, and the order is stated only at `rowReads`. +- **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`**: + the inherited architect's 3am trace crossed four files for one draw. +- **Test that every node kind reaches each switch over node kinds**: ten switches, from + `render` to `columnItems`, must agree, and only `render`'s comment says so. +- **Rename the load-time read fold apart from the render's reads**: `reads.go`, `pathRead` + and `readFold` share a word with `readField` and `readValue`, and `graph.go` holds + `renderEdges`, the edge model every fence walks. - Run the nine-seat panel again and file what it names here. ### Data @@ -250,6 +281,8 @@ by a `data-import/` script, as README goal 12 asks. - Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there: `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. +- Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states: + `probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. - Print `--help`'s repeat range from `fejkdata.MaxRepeat`: the usage spells `1..1048576` a second time. - Report a CRLF rows file holding only its header as having no rows: `parseRows` -- 2.52.0 From 2d5e7da8dccfc54f63ba5307a06563ed63ae823b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 30 Sep 2026 22:09:14 +0200 Subject: [PATCH 412/966] Count the seats behind each filed panel item as the reports name them --- todo.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/todo.md b/todo.md index eb3693e..0de08b3 100644 --- a/todo.md +++ b/todo.md @@ -47,12 +47,12 @@ This round, in order: left the score at 5.7, so the round stops here until they decide. - **Split `drawsApart` into one named check per refusal, and show at `conflict` why pairs find what a replay would**: all seven unit seats would least modify it, holding four - refusals over `landsRow`, `familyRoot`, `drawnOf` and `selected` at once, and four found - the pairs claim credible only through `TestPairsFindWhatAWholeReplayFinds`. + refusals over `landsRow`, `familyRoot`, `drawnOf` and `selected` at once, and two opened + `TestPairsFindWhatAWholeReplayFinds` to believe the pairs claim. - **Walk a path in one function**: `probePath` copies `pathCheck.walk`'s loop and `routeSteps` call, differing at a choice and in skipping `enter`, and seven seats and - both architects had to diff them by eye. -- **Name what `branches` and `pinBelow` hold and drop**: seven seats ranked + both architects named the pair. +- **Name what `branches` and `pinBelow` hold and drop**: six seats ranked `readFold.reads` second, decoding `pins` against `wholePins` through `doc.go` and finding `pinBelow`'s silent `continue` only by tracing. - **Fill `arm` in one place**: `splitArm` runs before the link and again after it, and @@ -62,12 +62,12 @@ This round, in order: operand hold answer for different nodes**: `arm.cover` means something else, and six seats held both walks beside `renderEdges` to see why they differ. - **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their - own, stating there that `checkOwnFamily` runs before the draw surveys**: both architects - found the fence order last, and the order is stated only at `rowReads`. + own, stating there that `checkOwnFamily` runs before the draw surveys**: the greenfield + architect found the pipeline last, and the order is stated only at `rowReads`. - **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`**: the inherited architect's 3am trace crossed four files for one draw. - **Test that every node kind reaches each switch over node kinds**: ten switches, from - `render` to `columnItems`, must agree, and only `render`'s comment says so. + `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. - **Rename the load-time read fold apart from the render's reads**: `reads.go`, `pathRead` and `readFold` share a word with `readField` and `readValue`, and `graph.go` holds `renderEdges`, the edge model every fence walks. -- 2.52.0 From f43064ed97f02d3a94ad61df48e67e2e9bb23dca Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:21:29 +0200 Subject: [PATCH 413/966] Rewrite the goals as plain one-line aims, with draws kept only by name and paths stepping up with .. --- README.md | 115 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 60 insertions(+), 55 deletions(-) diff --git a/README.md b/README.md index 354a080..c5fe2ee 100644 --- a/README.md +++ b/README.md @@ -174,6 +174,14 @@ zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.sym `misc.territory[SE].capital` and `misc.currency[Euro].symbol` select a row; `car` and `useragent` carry no key or name, so they are drawn from rather than selected in. +A fact is read from its source by a script under [`data-import/`](data-import), as +goal 12 asks. Where no script exists yet, a small hand-written set ships, and +[`todo.md`](todo.md) carries the step that replaces it. A script drops a source row +only by a rule it states. A category is added in proportion to how many real schemas +store it, per goal 13. Per goal 14, a category whose values could reach something real, +such as a domain anyone may register, also gets a path that draws only values that +reach nothing; `todo.md` carries the categories still missing one. + `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's other entries, WebDAV and DeltaV among them, do not ship. Its `safe` and @@ -760,6 +768,8 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance +Goal 11 aims at about a microsecond per value. + Each file is parsed, validated and weight-indexed once, in `New`, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends through it. Proving the draw fences adds a pass over the loaded tree per @@ -820,61 +830,52 @@ the Development section below, and who ships a register the five above then draw ## Goals -1. **The best fake-data tool available, and used because it is** — a developer - choosing a seed-data library or CLI, in any language, should find this the one - worth switching to, and every goal below it is what that costs. Merit is only - half the goal: once the grammar, the data and the CLI are complete and stable, - it ships where those developers install from and is announced where they read. -2. **Understanding the code is part of the product** — it is a compiler with static - fences behind a small API, it will grow, and the people who extend it will not be - the people who wrote it. So reading cost is judged alongside behaviour: reaching - the unit behind a symptom without asking a person, changing one piece without - holding the rest, names that tell the truth, and a file that stands with no second - document open beside it. A simulated reader panel scores those four, the score is a - ratchet no release lowers, and [`AGENTS.md`](AGENTS.md) says what each pull request - owes it. Below the floor it holds back new capability, never a repair of behaviour - this README already documents, as long as the round that lifts the score stays planned. -3. **Valid by construction** — every value passes the check a current release of - its real consumer applies; facts that belong together come from one draw, within - a value and across categories. -4. **Templates are intuitive, and text means what it says** — a template or a path, - and every path in a template, renders what every writer reading it would expect, - judged by simulated writers from the Audience. A format renders as written; only `{…}` varies, random characters included - (`{digits(3)}`), and a selected row, `[SE]`, is that row wherever it is written. - A name is changed whenever a more intuitive one is found; the Versioning table - prices the change, never blocks it. - Where two writers could expect different outputs, the template is a load error - naming a spelling for each. One spelling per result; the wrong one is a load error - naming the right one. -5. **Errors are up front and easy to understand.** -6. **Zero to a value in one command** — `go install`, then `fejkdata sv_SE.person`: - no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position; - the first custom template needs no escape and no option. -7. **Data lives in JSON** — a builtin only for what data can't express. -8. **Reproducible** — seed in, same stream out; no builtin reads a clock. -9. **Zero dependencies** — standard library only. -10. **Docs index the grammar** — every syntax feature is a heading; every example - runs under test and shows its output; a rule is stated once. -11. **Fast enough to be free** — a value renders in about a microsecond and `New` - parses and validates the whole set once up front, so generating fixtures stays - noise against a test's own runtime. -12. **Data is sourced, or on its way there** — a shipped fact, a name, place, - code, id or classification, is read from a register or open dataset by a - [`data-import/`](data-import) script wherever one exists to read; where none - does yet a small hand-written set ships and [`todo.md`](todo.md) carries the - step that replaces it. Only non-factual copy stays authored. A sourced table - holds the rows its source holds: none is added by hand, and one is dropped - only by a rule the script states. -13. **Breadth follows what most systems store** — a category is added in proportion - to how many real schemas hold it: names, addresses, phones, ids, money and - timestamps before anything domain-specific, and a catalogue serving one niche - waits behind everything serving many. -14. **Realism is the default, inertness is selectable** — where a value could reach - something real, a domain anyone may register or an account a bank could issue, - the realistic breadth ships *and* so does the subset that provably reaches - nothing, each on its own path. A fixture that looks nothing like production - tests nothing; the caller who needs a value that can touch nothing asks for it - by name. +1. **The best fake-data tool there is, and the one developers reach for.** + 1. You can install it the way you install your other tools, and people hear about + it where developers read. +2. **The code is easy to understand.** + 1. You can go from a bug to the code behind it without asking anyone. + 2. You can change one part without keeping the rest in your head. + 3. Every name says what the thing really is. + 4. Each file makes sense on its own, without another document open beside it. +3. **Every value is valid.** It passes the same check the real system would apply, + such as the checksum in a Swedish personal number. + 1. Values that belong together come from the same random pick. You choose which + ones by giving that pick a name. +4. **Templates are easy to write and read.** A template is the text you write to + describe a value, such as `{first} {last}`. + 1. Everything outside `{…}` prints exactly as written. + 2. Each `{…}` makes a new random pick. + 3. A name keeps one pick. `{/person as p}` picks a person and prints nothing. + `{p}` then prints that person, and `{p.last}` their last name. + 4. A path works like a folder path: `.` goes down a level, and `..` goes up one. + 5. `[SE]` selects one row of a table, and means that row wherever you write it. + 6. If two people could read a template differently, loading it fails, and the + error shows how to write each meaning. + 7. Each result has one way to write it. Any other way fails to load, and the error + shows the right one. + 8. When a clearer name turns up, it replaces the old one, even if that needs a new + major version. +5. **Mistakes are caught when the data loads, and the error says plainly what is + wrong.** +6. **One command gets you a value.** + 1. Flags work like in other command-line tools, and can go anywhere on the line. + 2. Your first template of your own needs no escaping and no options. +7. **Data is JSON files. Code is only for what JSON can't express.** +8. **The same seed always gives the same output.** +9. **It depends on nothing but Go's standard library.** +10. **The docs are a map of the template language.** + 1. Every feature has its own heading. + 2. Every example runs as a test and shows its output. + 3. Each rule is stated in one place. +11. **It is fast enough that you never notice it.** +12. **Every fact comes from an official or open source,** such as a government + register. + 1. A table holds every row its source holds, and nothing added by hand. +13. **The most common data comes first,** such as names, addresses, phone numbers and + dates. +14. **Values look real by default. You can also ask for values that can never reach + anything real,** such as a domain nobody can register. ## Layout @@ -995,6 +996,10 @@ docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/tld.p docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/useragent.py ``` +Goal 2 is scored by a panel of simulated readers, the `comprehension-panel` skill, and +[`AGENTS.md`](AGENTS.md) says what the score asks of a pull request. Goal 4 is checked +against simulated template writers from the Audience. + To release, head `CHANGELOG.md` with the version's section in place of `Unreleased` and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publishes the GitHub release with the section as its body. A top heading of `[Unreleased]` -- 2.52.0 From e8e1a2e2c13dfad62e871a8b710ccbde1c995896 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:23:59 +0200 Subject: [PATCH 414/966] Mark the goals as aims, and keep the moved text true where it landed --- README.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index c5fe2ee..5fe26cb 100644 --- a/README.md +++ b/README.md @@ -177,10 +177,10 @@ zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.sym A fact is read from its source by a script under [`data-import/`](data-import), as goal 12 asks. Where no script exists yet, a small hand-written set ships, and [`todo.md`](todo.md) carries the step that replaces it. A script drops a source row -only by a rule it states. A category is added in proportion to how many real schemas -store it, per goal 13. Per goal 14, a category whose values could reach something real, -such as a domain anyone may register, also gets a path that draws only values that -reach nothing; `todo.md` carries the categories still missing one. +only by a rule it states. Copy that states no fact, such as `word` and `sentence`, stays +hand-written. A category is added in proportion to how many real schemas store it, per +goal 13. Goal 14's paths that draw only values reaching nothing are planned per category +in `todo.md`; until one ships, pin a value such as `example.com`. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's @@ -830,6 +830,8 @@ the Development section below, and who ships a register the five above then draw ## Goals +Where the project is heading; the sections above document what ships today. + 1. **The best fake-data tool there is, and the one developers reach for.** 1. You can install it the way you install your other tools, and people hear about it where developers read. @@ -996,9 +998,9 @@ docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/tld.p docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/useragent.py ``` -Goal 2 is scored by a panel of simulated readers, the `comprehension-panel` skill, and -[`AGENTS.md`](AGENTS.md) says what the score asks of a pull request. Goal 4 is checked -against simulated template writers from the Audience. +The maintainer scores goal 2 with a panel of simulated readers, and checks goal 4 +against simulated template writers from the Audience; [`AGENTS.md`](AGENTS.md) says +what the score asks of a pull request. To release, head `CHANGELOG.md` with the version's section in place of `Unreleased` and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publishes -- 2.52.0 From fe102d413a93c896ffc1788358b4c29276ebe2fc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:26:25 +0200 Subject: [PATCH 415/966] Put each moved sentence where its reader looks, the condition first --- README.md | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 5fe26cb..2da052c 100644 --- a/README.md +++ b/README.md @@ -174,13 +174,12 @@ zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.sym `misc.territory[SE].capital` and `misc.currency[Euro].symbol` select a row; `car` and `useragent` carry no key or name, so they are drawn from rather than selected in. -A fact is read from its source by a script under [`data-import/`](data-import), as -goal 12 asks. Where no script exists yet, a small hand-written set ships, and -[`todo.md`](todo.md) carries the step that replaces it. A script drops a source row -only by a rule it states. Copy that states no fact, such as `word` and `sentence`, stays -hand-written. A category is added in proportion to how many real schemas store it, per -goal 13. Goal 14's paths that draw only values reaching nothing are planned per category -in `todo.md`; until one ships, pin a value such as `example.com`. +Where a script under [`data-import/`](data-import) reads a source, the facts come from +it, and it drops a source row only by a rule it states; elsewhere a small hand-written +set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 12). Copy +that states no fact, such as `word` and `sentence`, stays hand-written. Where a category +has no path that draws only values reaching nothing real (goal 14), pin a value such as +`example.com`; `todo.md` plans the path. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's @@ -768,7 +767,7 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance -Goal 11 aims at about a microsecond per value. +The target is about a microsecond per value (goal 11). Each file is parsed, validated and weight-indexed once, in `New`, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends @@ -830,7 +829,7 @@ the Development section below, and who ships a register the five above then draw ## Goals -Where the project is heading; the sections above document what ships today. +Where the project is heading; the sections before Audience document what ships today. 1. **The best fake-data tool there is, and the one developers reach for.** 1. You can install it the way you install your other tools, and people hear about @@ -960,6 +959,10 @@ docker build --build-arg GO_VERSION=1.22.12 --target portable . # lowest support GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build ``` +Goal 2 is scored by a panel of simulated readers, and goal 4 checked against simulated +template writers from the Audience; [`AGENTS.md`](AGENTS.md) says when each runs and +what the score asks of a pull request. + A change to the shipped data re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) in its own commit: @@ -998,10 +1001,6 @@ docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/tld.p docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/useragent.py ``` -The maintainer scores goal 2 with a panel of simulated readers, and checks goal 4 -against simulated template writers from the Audience; [`AGENTS.md`](AGENTS.md) says -what the score asks of a pull request. - To release, head `CHANGELOG.md` with the version's section in place of `Unreleased` and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publishes the GitHub release with the section as its body. A top heading of `[Unreleased]` -- 2.52.0 From 02f35e262557ae9a05d2b25626886e53b265a486 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:26:34 +0200 Subject: [PATCH 416/966] Claim of AGENTS.md only what it says --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 2da052c..9354d8a 100644 --- a/README.md +++ b/README.md @@ -959,9 +959,9 @@ docker build --build-arg GO_VERSION=1.22.12 --target portable . # lowest support GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build ``` -Goal 2 is scored by a panel of simulated readers, and goal 4 checked against simulated -template writers from the Audience; [`AGENTS.md`](AGENTS.md) says when each runs and -what the score asks of a pull request. +Goal 2 is scored by a panel of simulated readers, and [`AGENTS.md`](AGENTS.md) says +what the score asks of a pull request. Goal 4 is checked against simulated template +writers from the Audience. A change to the shipped data re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) in its own commit: -- 2.52.0 From 83c2a4758ccc04ef29c17c8a278188bc45a11bd5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:27:40 +0200 Subject: [PATCH 417/966] Name the script and the source apart, and keep pin for re-pinning --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9354d8a..271714e 100644 --- a/README.md +++ b/README.md @@ -175,11 +175,11 @@ zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.sym `useragent` carry no key or name, so they are drawn from rather than selected in. Where a script under [`data-import/`](data-import) reads a source, the facts come from -it, and it drops a source row only by a rule it states; elsewhere a small hand-written +that source, and the script drops a row only by a rule it states; elsewhere a small hand-written set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 12). Copy that states no fact, such as `word` and `sentence`, stays hand-written. Where a category -has no path that draws only values reaching nothing real (goal 14), pin a value such as -`example.com`; `todo.md` plans the path. +has no path that draws only values reaching nothing real (goal 14), write a fixed value such +as `example.com` into the template; `todo.md` plans the path. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's -- 2.52.0 From c3d47e5b21e480e308dd7c929c35750caf5a0fec Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:52:57 +0200 Subject: [PATCH 418/966] Reshape todo.md into the scored table, planning the draw restructure first --- docs/decisions.md | 20 ++ todo.md | 669 +++++++++++++++++++++++----------------------- 2 files changed, 357 insertions(+), 332 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 8b13829..52cdfc3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -114,6 +114,8 @@ pins its level, and a fresh draw of a pinned level could show another row. A bui operand holds what it reads for its expansion, references included, so `{uppercase(/p)} {/p}` is one draw, and `{/p}` written twice beside it is a load error. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## Reference sigils follow the filesystem 2026-09-02, Lilleman auf Larv. @@ -291,6 +293,8 @@ rather than two copies of it. Only references share: a sibling field is local to expansion, so a `first` column does not silently bind to a `first` in the column next to it. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## A draw group name is local to its category 2026-09-16, Lilleman auf Larv. @@ -300,6 +304,8 @@ joins them by accident, and renaming a group inside one file changes no render elsewhere. The unnamed group still spans categories, since facts that belong together across categories must agree. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## The expansion hold and the render's draws are two fences 2026-09-16, Lilleman auf Larv. @@ -310,6 +316,8 @@ rows agree across a render and its draw groups. Their scopes differ and only the tracks rows and pins, so one walk would carry both scopes and tell them apart at every step. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## A render's draws make their maps on the first read 2026-09-29, Lilleman auf Larv. @@ -321,6 +329,8 @@ cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09 Lazily, a render reading a reference path pays those two allocations, a record or struct included, and one reading none pays nothing; goal 11 holds either way. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## A category never references itself, and a record's fences run at load 2026-09-16, Lilleman auf Larv. @@ -473,6 +483,8 @@ exactly one row. Checked against three simulated template writers from the Audience, who agreed on each case but the last, where two expected a render and all three accepted the refusal. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## A link column after a row steps up to the parent row, and a path may not end on it Decided 2026-09-27 by the maintainer, for goal 4; valid while a link column holds the @@ -501,6 +513,8 @@ Checked against three simulated template writers from the Audience: all three re parent row where a segment followed, and they split between the code and the name where the path ended. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 2 ends it. + ## A table selects by one key column, a code, never a free-form name Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; valid while @@ -567,6 +581,8 @@ A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row tha three localities from other regions is the output the family exists to prevent, so the own-family fence walks through both rather than stopping where the draw fences do. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## A cell may read a reference, and each row its own 2026-09-29, Lilleman auf Larv. @@ -576,6 +592,8 @@ Goal 3: facts that belong together come from one draw, so a shop row's `phone` c format, the same for every row, cannot say that. A restructure for goal 2 keeps the fences that prove such cells safe. Valid while a table's rows need generators of their own. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## Tables carrying token cells stay small 2026-09-17, Lilleman auf Larv. @@ -586,6 +604,8 @@ its rows: about a second at four thousand rows. No shipped table carries such ce a register is a column set rather than a set of references, so the fence is left as it is until a real data set needs the indexed form. +Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. + ## A path is walked once without drawing before it is walked for real 2026-09-17, Lilleman auf Larv. diff --git a/todo.md b/todo.md index 0de08b3..a8e94d0 100644 --- a/todo.md +++ b/todo.md @@ -1,149 +1,261 @@ -# Plan +# todo -Ordered as the releases that ship it. +## Scoring -## v0.1.0 +`Score = -R - S/4 + 2*A + 2*G*W` -### Comprehension, until the panel scores 7.0 +`Bar = 9` -Nothing else ships while the score is under 7.0 — no feature, no category, no data — bar -what `AGENTS.md` excepts, per README goal 2. Nine rounds of the nine-seat findings held -the score between 5.7 and 6.1, so this round is a restructure the maintainer approved on -2026-09-29: a PR per item, then the nine-seat run again. The run is the nine-seat one, -since the four-seat scoring run names too little to steer an architectural change. +`Next ID = 62` -Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9, -Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9, -Self-sufficiency 5.6); 5.7 on 2026-09-26 (Navigation 6.7, Locality 5.0, Shape 5.8, -Self-sufficiency 5.6), every seat capped by Locality, and all nine naming -`drawWalk.walk` among the hardest; 5.7 on 2026-09-28 (Navigation 6.7, Locality 5.1, -Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming -`drawWalk.walk` and the four path walks among the hardest; 5.8 on 2026-09-29 -(Navigation 6.8, Locality 5.3, Shape 5.8, Self-sufficiency 5.7), seven seats capped -by Locality, all nine naming `drawWalk.walk` hardest and eight the unit they would -least modify; 5.7 on 2026-09-29 again (Navigation 6.6, Locality 5.2, Shape 5.9, -Self-sufficiency 5.8), four seats capped by Locality, six of the seven unit seats -ranking `readSurvey.walk` hardest and all seven the unit they would least modify; 6.1 -on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-sufficiency -6.1), every seat's lowest dimension Locality, all seven unit seats ranking the -`readSurvey` walk or check hardest and the unit they would least modify; 5.8 on -2026-09-29 a fourth time (Navigation 6.7, Locality 5.0, Shape 5.7, Self-sufficiency -5.7), every seat's lowest dimension Locality, alone or tied, all seven unit seats -ranking `readSurvey.walk` and `edge` hardest and the unit they would least modify; 5.8 -on 2026-09-29 a fifth time (Navigation 6.9, Locality 5.2, Shape 6.1, Self-sufficiency -5.8), every seat's lowest dimension Locality, alone or tied, all seven unit seats ranking -`readSurvey.walk` hardest and the unit they would least modify; 5.7 on 2026-09-30, the -checkpoint (Navigation 6.8, Locality 5.2, Shape 5.9, Self-sufficiency 5.7), every seat's -lowest dimension Locality, alone or tied, all seven unit seats ranking `drawsApart` or -`readFold.reads` hardest and five `drawsApart` the unit they would least modify. The -maintainer chose on 2026-09-30 to go on with the path steps regardless; 5.7 on -2026-09-30 after them (Navigation 7.0, Locality 5.1, Shape 5.9, Self-sufficiency 5.7), -every seat's lowest dimension Locality, alone or tied, six of the seven unit seats -ranking `drawsApart` hardest and all seven the unit they would least modify. +| Goal | W | +|---|---| +| 1 | 1.00 | +| 1.1 | 0.97 | +| 2 | 0.94 | +| 2.1 | 0.91 | +| 2.2 | 0.88 | +| 2.3 | 0.85 | +| 2.4 | 0.82 | +| 3 | 0.79 | +| 3.1 | 0.76 | +| 4 | 0.74 | +| 4.1 | 0.71 | +| 4.2 | 0.68 | +| 4.3 | 0.65 | +| 4.4 | 0.62 | +| 4.5 | 0.59 | +| 4.6 | 0.56 | +| 4.7 | 0.53 | +| 4.8 | 0.50 | +| 5 | 0.47 | +| 6 | 0.44 | +| 6.1 | 0.41 | +| 6.2 | 0.38 | +| 7 | 0.35 | +| 8 | 0.32 | +| 9 | 0.29 | +| 10 | 0.26 | +| 10.1 | 0.24 | +| 10.2 | 0.21 | +| 10.3 | 0.18 | +| 11 | 0.15 | +| 12 | 0.12 | +| 12.1 | 0.09 | +| 13 | 0.06 | +| 14 | 0.03 | -This round, in order: +## Items -- **Take the maintainer's call on the items below**: the approved restructure is done and - left the score at 5.7, so the round stops here until they decide. -- **Split `drawsApart` into one named check per refusal, and show at `conflict` why pairs - find what a replay would**: all seven unit seats would least modify it, holding four - refusals over `landsRow`, `familyRoot`, `drawnOf` and `selected` at once, and two opened - `TestPairsFindWhatAWholeReplayFinds` to believe the pairs claim. -- **Walk a path in one function**: `probePath` copies `pathCheck.walk`'s loop and - `routeSteps` call, differing at a choice and in skipping `enter`, and seven seats and - both architects named the pair. -- **Name what `branches` and `pinBelow` hold and drop**: six seats ranked - `readFold.reads` second, decoding `pins` against `wholePins` through `doc.go` and - finding `pinBelow`'s silent `continue` only by tracing. -- **Fill `arm` in one place**: `splitArm` runs before the link and again after it, and - `compileArm` finishes it from `reference.go`, so five seats traced three phases and both - architects found format compilation under `linkTemplate`. -- **Rename `holdfence.go`'s `cover`, and say at `heldNodes` why a path hold and an - operand hold answer for different nodes**: `arm.cover` means something else, and six - seats held both walks beside `renderEdges` to see why they differ. -- **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their - own, stating there that `checkOwnFamily` runs before the draw surveys**: the greenfield - architect found the pipeline last, and the order is stated only at `rowReads`. -- **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`**: - the inherited architect's 3am trace crossed four files for one draw. -- **Test that every node kind reaches each switch over node kinds**: ten switches, from - `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. -- **Rename the load-time read fold apart from the render's reads**: `reads.go`, `pathRead` - and `readFold` share a word with `readField` and `readValue`, and `graph.go` holds - `renderEdges`, the edge model every fence walks. -- Run the nine-seat panel again and file what it names here. +| ID | Release | Exempt | Item | R | S | A | G | Goals | Score | +|---|---|---|---|---|---|---|---|---|---| +| 51 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.0 | +| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 24.2 | +| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 23.4 | +| 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 22.2 | +| 24 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 22.1 | +| 50 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.1 | +| 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.2 | +| 25 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 20.1 | +| 32 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 20.1 | +| 13 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 19.8 | +| 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.8 | +| 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 4.4 | 18.7 | +| 23 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 17.8 | +| 27 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a selected sex, by name.** | 4 | 4 | 6 | 7 | 3.1 | 17.6 | +| 47 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 16.2 | +| 19 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 15.9 | +| 48 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.4 | +| 46 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 15.1 | +| 34 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 15.0 | +| 20 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 15.0 | +| 38 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 14.7 | +| 18 | 0.1.0 | | **Give every shipped category building-block columns, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 14.4 | +| 43 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 14.4 | +| 49 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 14.1 | +| 28 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 13.9 | +| 30 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 13.4 | +| 7 | 0.1.0 | | **Walk a path in one function: fold `probePath` into `pathCheck.walk`.** | 4 | 4 | 3 | 7 | 2.2 | 13.3 | +| 14 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 13.0 | +| 11 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 12.8 | +| 16 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of the table below per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 12.4 | +| 12 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 12.4 | +| 45 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 12.1 | +| 17 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 11.8 | +| 9 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | +| 10 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | +| 29 | 0.1.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 11.5 | +| 39 | 0.1.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 11.1 | +| 26 | 0.1.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 10.9 | +| 15 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 10.6 | +| 22 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 10.5 | +| 36 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 41 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 44 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 31 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 10.2 | +| 37 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 10.1 | +| 42 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 10.1 | +| 33 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 10.0 | +| 21 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 9.9 | +| 8 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 9.6 | +| 35 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | +| 40 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | +| 55 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 17.7 | +| 54 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 14.5 | +| 52 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 11.3 | +| 58 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 10.9 | +| 53 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 10.6 | +| 59 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 10.3 | +| 57 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 10.2 | +| 56 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 9.8 | +| 60 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 29.7 | +| 61 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.1 | -### Data +## Details -Shape: T = table, t = template, c = choice. Read every factual list from a register -by a `data-import/` script, as README goal 12 asks. +### 51. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. -- Fetch NVDB through `source.fetch` in `data-import/geo-se.py`: `nvdb_segments` calls - urllib itself, with no retry and no User-Agent. -- Test that every `geo` tree holds the five table names and the `address` columns the - README says port across countries: nothing checks it, and a new country breaks it silently. -- Strip the whitespace `data-import/territory.py` copies from its register: - `misc.territory[CW].capital` renders ` Willemstad`, with a leading space. -- Place xlsx cells by their `r` reference in `data-import/xlsx.py`: a sheet omitting an - empty cell shifts every later column left. -- Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone - `{digits(1)}` at `New`, naming that spelling: both render the same run. -- Add the remaining `misc` tables and templates, one row of the table below per chunk. -- Read `misc.emoji` from the Unicode `emoji-test.txt` register, and `misc.car` from an - international make and model source; vPIC and Mobility Sweden are national, so they - build `en_US.car` and `sv_SE.car`. -- Add the remaining locale categories: company, phone, finance, vehicle, words. -- Give every shipped category building-block columns, `misc.uuid` → `variant` among - them, and one record shape across the national ids; re-pin the shape. -- Give the address records one column set across countries: `region` and - `municipality` as columns on `geo.SE.address` too. -- Fill the 398 Swedish localities weighted 200 from SCB småorter. -- Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they - are: neither names a `weight`, so a bare `sex` draws evenly. -- List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list - of what each locale carries, which names neither. -- Give `url` and `email` a path that draws only domains nobody can register, keeping - the wide set as the default, per goal 14: 17 of the 40 distinct ones shipped today - sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's - `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` - provably reach nothing. -- Audit the rest of the shipped set against goal 14 and give each a never-reaching - path where it lacks one: `phone` draws live PTS and NANP ranges — PTS's five - fiction series are the Swedish inert set, sourced in - [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one - still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. -- Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does - not hold: ISO 3166 reserves it for Western Sahara and the root zone has never been - delegated it, so no resolver answers for it and goal 3 promises otherwise. -- Draw `en_US.phone`'s `exch` as a NANP central office code: `{int(100,999)}` renders - a leading 1 in about an eighth of draws, which libphonenumber rejects, and - `TestShippedUSPhone` proves the shape rather than the rule. Assert the rule with it. -- Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop - drawing birthdates that reach 100: the format hard-codes `-`, and the 1930 floor - makes the oldest draws invalid from 2031. -- Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`, per goal 4's - one spelling per result. -- Accept a middle name, and draw a shipped `personnummer` inside a *selected* sex; - both want a draw group sharing its family's pins. Stop the conflict error naming a - rewrite that returns a different value where the read it conflicts with sits inside - another category, and report a struct column's draw conflict with the path spelling - a tag takes. -- Let `geo.US.locality[1714000].address` descend from a selected row into the template - beside the family: require the path step to reach a sibling category, and seed every - draw group of the render from the outer selector's pins. -- Read `email.local`'s and `username`'s handles from the shipped name tables, which - goal 12 asks for and the hand-written list they use today does not meet. Keep the - handle shape: `username` shortens a surname to `ahl` or `sjo`, and the US one draws - unisex given names, neither of which falls out of the tables unaided. -- Draw `email.local` from `username`, so the two share one handle list. -- Rename `misc.territory.country` to `sovereign`, per goal 4: it holds a code, and - `country` collides with the `misc.country` path it replaced. -- Add `misc.browser` as the parent of `misc.useragent`, so - `misc.browser[Chrome].useragent` resolves, per goal 3. -- Group `data/misc` into folders where a group name makes a path easier to guess, per - goal 4. +GoReleaser attaches them to the release the tag workflow publishes. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. -`misc` (locale-neutral) +### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. + +- A binding prints nothing: `{/person as p}{p}` renders `Erik Lind`, and `{p.last}` beside it `Lind`. +- A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. +- A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read. +- A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line. +- Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace. +- Lands before item 4 changes what an unnamed `{…}` does, so the shipped data can move to names first (item 3). + +### 3. Rewrite the shipped categories so facts that belong together come from one named pick. + +- `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. +- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. +- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. +- Re-pin seeded output in its own commit. + +### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. + +- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G`, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup` and the decisions that reverse with them. The remaining load checks are names, paths through names, cycles and the repeat cap. +- Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. +- Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. +- Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. + +### 24. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. + +`{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. + +### 50. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. + +Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. + +### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. + +Needs item 4, and runs as item 6 decides. + +### 25. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. + +The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. + +### 32. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. + +`isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. + +### 13. Strip the whitespace `data-import/territory.py` copies from its register. + +`misc.territory[CW].capital` renders ` Willemstad`, with a leading space. + +### 6. Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4. + +Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: + +| Seat | Run 1 | Run 2 | +|---|---|---| +| Junior A | 5 | 5 | +| Junior B | 4 | 5 | +| Mid A | 6 | 5 | +| Mid B | 6 | 6 | +| Senior domain | 6 | 5 | +| Senior maintainability | 6 | 6 | +| Senior extender | 6 | 5 | +| Architect greenfield | 6 | 6 | +| Architect inherited | 6 | 6 | +| Mean (Navigation / Locality / Shape / Self-sufficiency) | 5.7 (7.0 / 5.1 / 5.9 / 5.7) | 5.4 (6.3 / 4.8 / 5.8 / 5.3) | + +Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Every score since 2026-09-20 (5.7–6.1) sits inside this spread, and `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. + +### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. + +- `..country` steps up to the nearest ancestor table named `country`, from a selected row, a drawn row or a name bound to a row: `{c..country.name}`. +- After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. +- `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. +- A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. +- Replaces the decision "A link column after a row steps up to the parent row, and a path may not end on it". + +### 23. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. + +ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it. + +### 27. Accept a middle name, and draw a `personnummer` whose sex matches a selected sex, by name. + +Needs item 4. Report a struct column's draw conflict with the path spelling a tag takes. + +### 47. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. + +Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. + +### 48. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. + +GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. + +### 46. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. + +With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. + +### 34. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. + +`New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. + +### 20. Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are. + +Neither names a `weight`, so a bare `sex` draws evenly. + +### 38. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. + +`geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. Steps: +- Have a selector that misses name the key of every row whose cell spells it, matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, and `misc.loglevel[Error]` `[err]`. +- Delete the `name` option and the load checks it carries, and have `New` refuse a table still carrying `name`, naming `key`. +- Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on the integer `data-import/geo-se.py` assigns as the decision states, linking `street` and `postal-code` by it. +- Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. +- Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`. + +### 18. Give every shipped category building-block columns, and one record shape across the national ids; re-pin the shape. + +`misc.uuid` → `variant` among them. + +### 43. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. + +`probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. + +### 28. Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`. + +Require the path step to reach a sibling category. + +### 7. Walk a path in one function: fold `probePath` into `pathCheck.walk`. + +`probePath` copies `pathCheck.walk`'s loop and `routeSteps` call, differing at a choice and in skipping `enter`; seven seats and both architects named the pair. After item 4. + +### 14. Place xlsx cells by their `r` reference in `data-import/xlsx.py`. + +A sheet omitting an empty cell shifts every later column left. + +### 11. Test that every node kind reaches each switch over node kinds. + +Ten switches, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. `tableReadOf` switches over `stepKind` with no default, so a new step kind passes it silently. + +### 16. Add the remaining `misc` tables and templates, one row of the table below per chunk. + +Read every factual list from a register by a `data-import/` script. Shape: T = table, t = template, c = choice. | Category | Shape | Source | Licence | |---|---|---|---| @@ -164,6 +276,18 @@ by a `data-import/` script, as README goal 12 asks. | `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | | `direction`, `continent`, `ulid` | c/t | — | — | +### 12. Test that every `geo` tree holds the five table names and the `address` columns that port across countries. + +Nothing checks it, and a new country breaks it silently. + +### 45. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. + +Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. + +### 17. Add the remaining locale categories: company, phone, finance, vehicle, words. + +Shape: T = table, t = template, c = choice. + `sv_SE` ([source research](docs/research/research-sources-se.md)) | Category | Shape | Source | Licence | @@ -194,213 +318,94 @@ by a `data-import/` script, as README goal 12 asks. | `word`, `sentence`, `paragraph` | T+t | Moby POS or WordNet | public domain / WordNet | | month and weekday names, `holiday` | t+T | CLDR en | Unicode | -### Builtins the data cannot express +### 9. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. -- Add `{isin()}` (Luhn over letters expanded to digits), `{cusip()}`, `{aba()}` - (3-7-1 weights) and `{vin()}` (position 9 over the whole; a sample taking the WMI, - since the check sits mid-string). -- Add `{base64url(n)}` for JWT shapes. -- Add a `unix` layout to `date()` once something needs it. +The greenfield architect found the fence pipeline last. After item 4, which shrinks it. -### Library and CLI +### 10. Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`. -- Report every mistake a load finds in one error, as the decision "A load reports every - mistake at once" states: `New` and `NewTemplate` stop at the first, so data holding - two mistakes takes two runs to fix. Split it into sub-items before starting, one of - them the shape a Go caller iterates and the CLI prints, and the order of an - inline template's mistakes, which name no path; reword the README's - "`New` refuses a mistake in the data" to every mistake. -- Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`: both name - half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` - shadows the `rng` interface. -- Compare `calcParser.binary`'s operator as a rune: `byte(p.rs[p.pos])` reads U+012B - `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. -- Stop the `columnKinds` suggestion naming a field's own kind or a narrower one: an - `int64` or `float64` field unproven in range is told to become itself, and a `uint64` - one to become `int64`. -- Say in the README's Linked tables that the first table a render reads decides whose - weights govern the family: `geo.SE.address` reads `street` first, so it draws by - street segments and the region, kommun and postort populations never apply. -- Parse `defaultTable`'s path with the library's grammar: the CLI counts brackets of its - own, so a change to the selector grammar desyncs the default `--table` silently. -- Step up from a row to its parent as the decision "A link column after a row steps up - to the parent row, and a path may not end on it" states, refusing a step back down: - `city[Oslo].country.name` is a load error today, and `city[Oslo].country` renders the - cell `NO`, as `locality.municipality` does, which `List` stops advertising. Replace the - README's `city[Oslo].country` example with - `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`. -- Key every table as the decision "A table selects by one key column, a code, never a - free-form name" states: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select - a row today. - - Have a selector that misses name the key of every row whose cell spells it, - matched without case, else say the table selects by its key: - `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` - its key, and `misc.loglevel[Error]` `[err]`. - - Delete the `name` option and the load checks it carries, a name spelling another - row's key and a name repeating inside its parent, and have `New` refuse a table - still carrying `name`, naming `key`. - - Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on the integer - `data-import/geo-se.py` assigns as the decision states, linking `street` and - `postal-code` by it. - - Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. - - Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`: no - selector spelling a free-form name, `geo.SE.locality[Lund]`, `misc.currency[Euro]`, `first-name[Kim]` - and the rest, no "key or name", "by name" or "either" wording about selectors, and no - `name` option in an example. -- Render selected rows as the decision "A selected row is fixed, not drawn, and an - unnamed read beside it is refused" states: `New` refuses - `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice - whose items select different rows. Have the refusal of an unnamed read name the - rewrite through each selection: beside `{/geo.US.locality[1714000].name}`, - `{/geo.US.municipality.name}` names none today, where - `{/geo.US.locality[1714000].municipality.name}` holds once the step up above lands. - Scope the README's "A reference path into a category is held" to a path that - selects no row. -- Refuse a struct tag's `datatype` naming the Go type that sets it before proving its - values, as the README's Library section promises: `datatype: boolean` over - `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. -- Name the spelling to use in `checkNameHeld`'s path refusal and `checkNoOverlap`'s, as - the README's hold section promises: both answer "name the fields you want instead", - and `{n} reaches "p" by a second route` names neither the read the route ends on, - `{/row.p.last}`, nor the spelling to write, `{p.last}`. Where an operand route's value - is the held name, offer it as the operand: `lowercase operand "y" renders - "/misc.territory"` ends at "stop", where `{lowercase(/misc.territory)}` loads. -- Split the README's linked-tables paragraph opening "A path descends from a row", 23 - lines holding five rules: move its load errors that are no overlap to the table - section, and drop its bare `{/city}` refusal, which the hold section states. -- Offer `move {Y} into a field with a drawGroup` in `overlapError` where both reads share - one field: `"a":"{/cat} {/cat.p.first}"` ends at "name the fields you want instead", - while the same pair in a format offers the move, which loads there too. -- Call `templateError` a compile failure: its doc says render failure, and it wraps - what `NewTemplate` and `NewRecordTemplate` refuse. -- Name the node a selector follows in `stepInto`'s refusal: `sv_SE.person[1].first` - answers `1 is not a table`. -- Report the same error every load for a table with two bad options, and for a - folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the - first in Go's map order. -- Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that - is not there: `loadDir` indexes hidden TSVs and skips them in two places, so one loads - where its comment says a hidden file is never data. -- Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states: - `probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. -- Print `--help`'s repeat range from `fejkdata.MaxRepeat`: the usage spells - `1..1048576` a second time. -- Report a CRLF rows file holding only its header as having no rows: `parseRows` - answers "line 2 is empty". -- Reword `prove`'s refusal of a typed column reading a row, `{/region}`: say the - format is the table's own (`region's format "{name}"`), name the read to write, - `{/region.}` with its columns, and stop calling a one-column format, - `{code}`, composed text. -- Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and - `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither. -- Refuse two whole reads of one table family in one render, which goal 5 promises is a - load error: `{/sel}|{/sel}` panics out of `Fake` where the two draws land on rows whose - cells select different rows of another table, and - `{/geo.SE.locality}|{/geo.SE.municipality}` renders a locality outside the - municipality beside it. Each read draws its own row, so cells of two rows render - together, where the fence counts them alternatives and compares neither. Have the error - name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`, - as the README's rule on a rejected spelling asks. -- Name the file a link error comes from beside its category path, `sub.x - (/d/a/sub/x.json): …`: with several `--data-path` layers the author has to work out - which directory won, while a parse error on the same file names it. -- Name the row a refused read sits in: where both reads come from TSV cells the overlap - error names the category and the two columns but no row, so the author greps a - register-sized file for the cell — and "name the fields you want instead" rewrites a - cell that is not the one to fix. -- Spell a table one way across the family errors: `drawsApart` names - `drawn.segment`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one - table is `territory` and `misc.territory`, and the short spelling names no file where two - folders hold that name. -- Let a table column carry a `datatype`, so `--format json` writes - `"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only - a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text - everywhere else. Typing a shipped column changes what `json` and `sql` write, so the - capability is a minor and applying it to `misc.httpmethod` is a major. +The inherited architect's 3am trace crossed four files for one draw. -### Release +### 29. Rename `misc.territory.country` to `sovereign`. -- Read the tag through `git/ref/tags/{tag}` in `publish_release.py`: GitHub answers - `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a - release lands on a tag already pointing at another commit. -- Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the - README's Library section, which names none of them. -- Build on a manual run of `test.yml`: `workflow_dispatch` leaves `github.event.before` - empty, so the diff compares `HEAD` with itself and skips `docker build`. -- Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds: nothing has shipped, so - "no longer paths", "where it used to fail" and "where it used to be an empty string" - describe versions no reader can have installed. -- Ship prebuilt binaries so the CLI needs no Go: GoReleaser attaching them to the Gitea - release the tag workflow publishes, a container image, Homebrew and Scoop. A checkout - build prints `devel` for `--version`; the binaries carry the stamped tag. +It holds a code, and `country` collides with the `misc.country` path it replaced. -## v0.2.0 +### 39. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. -- **Promise in `Generator`'s godoc that its renders run one at a time**, so a caller - wanting parallel throughput makes one generator per goroutine. -- Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip - for `--data-path`: every US place of 10,000 and more streets per locality, built by - `--min-population` and `--streets-per-locality`. The default embed stays under ~1 MB - per country and `New` under ~50 ms. -- Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, with - person, address, phone, national id, company and date names each, and the `geo/` - trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE their `address` reads by reference; - AU and CA get a tree with no locale of their own. -- Add `{btc()}` and `{eth()}`, which need sha256 and keccak for Base58Check and EIP-55. -- Pair a street with its exact postnummer, which needs an application to Lantmäteriet; - today a street goes to the nearest postal code centroid. It rewrites shipped rows, so - it is a major once 1.0 is cut and a minor before that. +`datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. -## v0.3.0 +### 15. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. -Names no items yet, so planning it is a chunk of its own: it takes the additive work -that lands after v0.2.0, and it has to add something, since a minor only adds. Its job -is to be the release with no breaking change that v1.0.0 waits on — v0.2.0 cannot be, -because pairing a street with its exact postnummer rewrites shipped rows. +Both render the same run. -## v1.0.0 +### 22. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. -- Cut it once the shipped data is in its record shape and one full minor has shipped - with no breaking change, per the README's Versioning table: v0.1.0 settles the - record shape and v0.3.0 is the quiet minor. -- Announce it where a developer choosing a fake-data tool already reads, which is - goal 1's second half and waits for the tag: complete, stable, installable without - Go. Write the README's first screen for someone deciding in a minute. +17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. -## Not release-bound +### 36. Compare `calcParser.binary`'s operator as a rune. -- Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and - the premise it rests on; propose a goal for an entry that serves none. -- Hold every change the charter says owes a `CHANGELOG.md` entry to one in CI: the - check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a - flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -- Measure the allocation gate over the shipped `geo` trees too: `pinSet.inline [8]` is sized - for them, and `perf_test.go` checks a synthetic five-deep tree only. -- Name goal 2's four criteria with the axes the panel scores, so a recorded score maps - back to the clause it came from: the goal spells them out in prose while `todo.md` - records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`. -- Cut the README's Layout block to the lines that say what a file name cannot: - `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -- Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word - in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a - second copy that drifts. Split the comprehension rule while there, which packs the - gate, its exceptions, the round and the at-or-above-7.0 regime into one line. -- Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates - Go at `Dockerfile:16`, and the shipped JSON under `data/` is two-space, so the rule as - written is one no Go file can break and every data file does. -- Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub: - Gitea converts a repository to a mirror only by re-creating it, so until that runs - the copy there is a second owner of one history and goes stale from this commit. -- Stop `/cache` collecting root-owned files, which today every command documented - without `--user` leaves for the next `--user` one to trip over, and which the README - answers with a chown a reader has to repeat. Put `--user` on all of them, or give the - `x-go` anchor a `user:` — that one needs a mechanism, since Compose interpolates from - the process environment and neither `UID` nor `GID` is exported there, so - `user: "${UID}:${GID}"` resolves to `":"`. -- Spell the latest supported Go once: `Dockerfile`'s `ARG GO_VERSION` and - `compose.yaml`'s `golang:${GO_VERSION:-…}` pin one version twice, and a bump that - moved only the first left `docker compose run --rm fmt` formatting to a `gofmt` the - gate rejects. A `renovate.json` custom manager reads the second today; drop it with - the duplication. -- Publish a homepage with an in-browser generator: the library compiled to WebAssembly, - so visitors generate as much data as they like in their own browser. +`byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. + +### 41. Name the node a selector follows in `stepInto`'s refusal. + +`sv_SE.person[1].first` answers `1 is not a table`. + +### 44. Report a CRLF rows file holding only its header as having no rows. + +`parseRows` answers "line 2 is empty". + +### 37. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. + +An `int64` or `float64` field unproven in range is told to become itself, and a `uint64` one to become `int64`. + +### 42. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. + +`readTableOptions` and `loadDir` return on the first in Go's map order. + +### 8. Fill `arm` in one place. + +`splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. After item 4. + +### 35. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. + +Both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` shadows the `rng` interface. + +### 40. Call `templateError` a compile failure in its doc. + +Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. + +### 55. Pair a street with its exact postnummer. + +Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. + +### 54. Add `{btc()}` and `{eth()}`. + +They need sha256 and keccak for Base58Check and EIP-55. + +### 52. Promise in `Generator`'s godoc that its renders run one at a time. + +So a caller wanting parallel throughput makes one generator per goroutine. + +### 58. Cut the README's Layout block to the lines that say what a file name cannot. + +`calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. + +### 53. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. + +Person, address, phone, national id, company and date names each, and the `geo/` trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE their `address` reads by reference; AU and CA get a tree with no locale of their own. + +### 59. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. + +The round procedure stands in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. + +### 57. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. + +The check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. + +### 56. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. + +Propose a goal for an entry that serves none. + +### 60. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. + +The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; v0.2.0 rewrites shipped rows, so a v0.3.0 with no breaking change comes first. -- 2.52.0 From a260feefe8b5779ee97bdb7c1806c8ac1e5788fc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 00:58:21 +0200 Subject: [PATCH 419/966] Answer the prose review: decision lines by goal, item details true after the restructure, defects kept --- docs/decisions.md | 24 +++-- todo.md | 240 ++++++++++++++++++++++++---------------------- 2 files changed, 138 insertions(+), 126 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 52cdfc3..d84fd0c 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -30,6 +30,8 @@ teach the exported API belongs above the clause. every other key is a field. Nesting fields under a key, or prefixing options, would tax every template to guard against a misspelt option. +`todo.md` item 4 deletes `drawGroup`, which this reserves. + ## `{a|b}` stays beside nested choices 2026-09-02, Lilleman auf Larv. @@ -114,7 +116,7 @@ pins its level, and a fresh draw of a pinned level could show another row. A bui operand holds what it reads for its expansion, references included, so `{uppercase(/p)} {/p}` is one draw, and `{/p}` written twice beside it is a load error. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +Goal 4.2 overrides this; `todo.md` item 4 replaces it. ## Reference sigils follow the filesystem @@ -293,7 +295,7 @@ rather than two copies of it. Only references share: a sibling field is local to expansion, so a `first` column does not silently bind to a `first` in the column next to it. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +Goal 4.2 overrides this; `todo.md` item 4 replaces it. ## A draw group name is local to its category @@ -304,7 +306,7 @@ joins them by accident, and renaming a group inside one file changes no render elsewhere. The unnamed group still spans categories, since facts that belong together across categories must agree. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +`todo.md` item 4 deletes what this decides. ## The expansion hold and the render's draws are two fences @@ -316,7 +318,7 @@ rows agree across a render and its draw groups. Their scopes differ and only the tracks rows and pins, so one walk would carry both scopes and tell them apart at every step. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +`todo.md` item 4 deletes what this decides. ## A render's draws make their maps on the first read @@ -329,7 +331,7 @@ cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09 Lazily, a render reading a reference path pays those two allocations, a record or struct included, and one reading none pays nothing; goal 11 holds either way. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +`todo.md` item 4 deletes what this decides. ## A category never references itself, and a record's fences run at load @@ -346,6 +348,8 @@ reads those fences weigh differs on purpose: a record-only template's inert form renders nothing, so a `drawGroup` on it can never matter and is refused, while one on a rendering format can matter to a caller that bare-references it. +`todo.md` item 4 deletes the `drawGroup` refusal this names. + ## A record's column set is fixed before the first draw 2026-09-04, Lilleman auf Larv. @@ -483,7 +487,7 @@ exactly one row. Checked against three simulated template writers from the Audience, who agreed on each case but the last, where two expected a render and all three accepted the refusal. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +Goal 4.2 overrides its shared draw beneath a selection and its refusal of an unnamed read; `todo.md` item 4 replaces them. ## A link column after a row steps up to the parent row, and a path may not end on it @@ -513,7 +517,7 @@ Checked against three simulated template writers from the Audience: all three re parent row where a segment followed, and they split between the code and the name where the path ended. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 2 ends it. +Goal 4.4 overrides this; `todo.md` item 2 replaces it. ## A table selects by one key column, a code, never a free-form name @@ -581,7 +585,7 @@ A `repeat` iteration and a `drawGroup` each draw apart on purpose, but a row tha three localities from other regions is the output the family exists to prevent, so the own-family fence walks through both rather than stopping where the draw fences do. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +`todo.md` item 4 deletes what this decides. ## A cell may read a reference, and each row its own @@ -592,7 +596,7 @@ Goal 3: facts that belong together come from one draw, so a shop row's `phone` c format, the same for every row, cannot say that. A restructure for goal 2 keeps the fences that prove such cells safe. Valid while a table's rows need generators of their own. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +`todo.md` item 4 deletes the fences this keeps; a cell may still read a reference. ## Tables carrying token cells stay small @@ -604,7 +608,7 @@ its rows: about a second at four thousand rows. No shipped table carries such ce a register is a column set rather than a set of references, so the fence is left as it is until a real data set needs the indexed form. -Works against goal 4's every-`{…}`-is-a-new-pick aims until `todo.md` item 4 ends it. +`todo.md` item 4 deletes what this decides. ## A path is walked once without drawing before it is walked for real diff --git a/todo.md b/todo.md index a8e94d0..8930f12 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 62` +`Next ID = 64` | Goal | W | |---|---| @@ -49,103 +49,103 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 51 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.0 | +| 53 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.0 | | 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 24.2 | | 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 23.4 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 22.2 | -| 24 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 22.1 | -| 50 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.1 | +| 23 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 22.1 | +| 52 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.1 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.2 | -| 25 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 20.1 | -| 32 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 20.1 | -| 13 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 19.8 | +| 24 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 20.1 | +| 31 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 20.1 | +| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 19.8 | | 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.8 | | 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 4.4 | 18.7 | -| 23 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 17.8 | -| 27 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a selected sex, by name.** | 4 | 4 | 6 | 7 | 3.1 | 17.6 | -| 47 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 16.2 | -| 19 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 15.9 | -| 48 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.4 | -| 46 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 15.1 | -| 34 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 15.0 | -| 20 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 15.0 | -| 38 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 14.7 | -| 18 | 0.1.0 | | **Give every shipped category building-block columns, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 14.4 | +| 22 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 17.8 | +| 26 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 17.6 | +| 48 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 16.2 | +| 18 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 15.9 | +| 49 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.4 | +| 47 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 15.1 | +| 33 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 15.0 | +| 19 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 15.0 | +| 37 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 14.7 | +| 17 | 0.1.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 14.4 | | 43 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 14.4 | -| 49 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 14.1 | -| 28 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 13.9 | -| 30 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 13.4 | -| 7 | 0.1.0 | | **Walk a path in one function: fold `probePath` into `pathCheck.walk`.** | 4 | 4 | 3 | 7 | 2.2 | 13.3 | -| 14 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 13.0 | -| 11 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 12.8 | -| 16 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of the table below per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 12.4 | -| 12 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 12.4 | +| 50 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 14.1 | +| 27 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 13.9 | +| 29 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 13.4 | +| 13 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 13.0 | +| 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 12.8 | +| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 12.4 | +| 11 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 12.4 | | 45 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 12.1 | -| 17 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 11.8 | -| 9 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | -| 10 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | -| 29 | 0.1.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 11.5 | -| 39 | 0.1.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 11.1 | -| 26 | 0.1.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 10.9 | -| 15 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 10.6 | -| 22 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 10.5 | -| 36 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 41 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 11.8 | +| 8 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | +| 9 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | +| 28 | 0.1.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 11.5 | +| 38 | 0.1.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 11.1 | +| 25 | 0.1.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 10.9 | +| 14 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 10.6 | +| 21 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 10.5 | +| 35 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 40 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 10.4 | | 44 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 31 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 10.2 | -| 37 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 10.1 | -| 42 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 10.1 | -| 33 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 10.0 | -| 21 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 9.9 | -| 8 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 9.6 | -| 35 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | -| 40 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | -| 55 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 17.7 | -| 54 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 14.5 | -| 52 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 11.3 | -| 58 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 10.9 | -| 53 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 10.6 | -| 59 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 10.3 | -| 57 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 10.2 | -| 56 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 9.8 | -| 60 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 29.7 | -| 61 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.1 | +| 30 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 10.2 | +| 36 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 10.1 | +| 46 | 0.1.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 10.1 | +| 41 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 10.1 | +| 32 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 10.0 | +| 20 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 9.9 | +| 7 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 9.6 | +| 34 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | +| 39 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | +| 51 | 0.1.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | +| 42 | 0.1.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 7.4 | +| 57 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 17.7 | +| 56 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 14.5 | +| 54 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 11.3 | +| 60 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 10.9 | +| 55 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 10.6 | +| 61 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 10.3 | +| 59 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 10.2 | +| 58 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 9.8 | +| 62 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 29.7 | +| 63 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.1 | ## Details -### 51. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. +### 53. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. GoReleaser attaches them to the release the tag workflow publishes. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. ### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. -- A binding prints nothing: `{/person as p}{p}` renders `Erik Lind`, and `{p.last}` beside it `Lind`. - A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. - A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read. - A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line. - Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace. -- Lands before item 4 changes what an unnamed `{…}` does, so the shipped data can move to names first (item 3). ### 3. Rewrite the shipped categories so facts that belong together come from one named pick. - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. - `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. -- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. +- Needs item 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G`, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. -- Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup` and the decisions that reverse with them. The remaining load checks are names, paths through names, cycles and the repeat cap. +- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. - Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. -### 24. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. +### 23. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. `{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. -### 50. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. +### 52. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. @@ -153,15 +153,15 @@ Nothing has shipped, so "no longer paths", "where it used to fail" and "where it Needs item 4, and runs as item 6 decides. -### 25. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. +### 24. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. -### 32. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. +### 31. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. `isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. -### 13. Strip the whitespace `data-import/territory.py` copies from its register. +### 12. Strip the whitespace `data-import/territory.py` copies from its register. `misc.territory[CW].capital` renders ` Willemstad`, with a leading space. @@ -182,45 +182,45 @@ Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: | Architect inherited | 6 | 6 | | Mean (Navigation / Locality / Shape / Self-sufficiency) | 5.7 (7.0 / 5.1 / 5.9 / 5.7) | 5.4 (6.3 / 4.8 / 5.8 / 5.3) | -Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Every score since 2026-09-20 (5.7–6.1) sits inside this spread, and `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. +Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Scores since 2026-09-20 range 5.7–6.1, wider than this 0.3 spread by only 0.1, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. The last recorded nine-seat score is 5.7, on 2026-09-30; the ratchet reads it from here until this item is decided. ### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. -- `..country` steps up to the nearest ancestor table named `country`, from a selected row, a drawn row or a name bound to a row: `{c..country.name}`. +- `..` goes up one level, to the parent table, and the name after it is that parent's: `{c..country.name}`, and from a locality `{l..municipality..region.name}`. Naming any other table is a load error naming the parent. +- It steps up from a selected row, a drawn row or a name bound to a row. - After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. - `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. - A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. -- Replaces the decision "A link column after a row steps up to the parent row, and a path may not end on it". -### 23. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. +### 22. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it. -### 27. Accept a middle name, and draw a `personnummer` whose sex matches a selected sex, by name. +### 26. Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick. -Needs item 4. Report a struct column's draw conflict with the path spelling a tag takes. +Needs item 4. -### 47. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. +### 48. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. -### 48. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. +### 49. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. -### 46. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. +### 47. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. -### 34. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. +### 33. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. `New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. -### 20. Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are. +### 19. Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are. Neither names a `weight`, so a bare `sex` draws evenly. -### 38. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. +### 37. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. Steps: - Have a selector that misses name the key of every row whose cell spells it, matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, and `misc.loglevel[Error]` `[err]`. @@ -229,33 +229,29 @@ Neither names a `weight`, so a bare `sex` draws evenly. - Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. - Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`. -### 18. Give every shipped category building-block columns, and one record shape across the national ids; re-pin the shape. +### 17. Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape. -`misc.uuid` → `variant` among them. +`misc.uuid` gains `variant`, for one. ### 43. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. `probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. -### 28. Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`. +### 27. Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`. Require the path step to reach a sibling category. -### 7. Walk a path in one function: fold `probePath` into `pathCheck.walk`. - -`probePath` copies `pathCheck.walk`'s loop and `routeSteps` call, differing at a choice and in skipping `enter`; seven seats and both architects named the pair. After item 4. - -### 14. Place xlsx cells by their `r` reference in `data-import/xlsx.py`. +### 13. Place xlsx cells by their `r` reference in `data-import/xlsx.py`. A sheet omitting an empty cell shifts every later column left. -### 11. Test that every node kind reaches each switch over node kinds. +### 10. Test that every node kind reaches each switch over node kinds. -Ten switches, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. `tableReadOf` switches over `stepKind` with no default, so a new step kind passes it silently. +After item 4; recount the switches then. Today ten, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. -### 16. Add the remaining `misc` tables and templates, one row of the table below per chunk. +### 15. Add the remaining `misc` tables and templates, one row of its detail's table per chunk. -Read every factual list from a register by a `data-import/` script. Shape: T = table, t = template, c = choice. +Shape: T = table, t = template, c = choice. | Category | Shape | Source | Licence | |---|---|---|---| @@ -276,7 +272,7 @@ Read every factual list from a register by a `data-import/` script. Shape: T = t | `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | | `direction`, `continent`, `ulid` | c/t | — | — | -### 12. Test that every `geo` tree holds the five table names and the `address` columns that port across countries. +### 11. Test that every `geo` tree holds the five table names and the `address` columns that port across countries. Nothing checks it, and a new country breaks it silently. @@ -284,7 +280,7 @@ Nothing checks it, and a new country breaks it silently. Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. -### 17. Add the remaining locale categories: company, phone, finance, vehicle, words. +### 16. Add the remaining locale categories: company, phone, finance, vehicle, words. Shape: T = table, t = template, c = choice. @@ -318,35 +314,35 @@ Shape: T = table, t = template, c = choice. | `word`, `sentence`, `paragraph` | T+t | Moby POS or WordNet | public domain / WordNet | | month and weekday names, `holiday` | t+T | CLDR en | Unicode | -### 9. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. +### 8. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. -The greenfield architect found the fence pipeline last. After item 4, which shrinks it. +The greenfield architect found the fence pipeline last. After item 4, which shrinks the pipeline. -### 10. Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`. +### 9. Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`. The inherited architect's 3am trace crossed four files for one draw. -### 29. Rename `misc.territory.country` to `sovereign`. +### 28. Rename `misc.territory.country` to `sovereign`. It holds a code, and `country` collides with the `misc.country` path it replaced. -### 39. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. +### 38. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. `datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. -### 15. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. +### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. Both render the same run. -### 22. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. +### 21. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. 17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. -### 36. Compare `calcParser.binary`'s operator as a rune. +### 35. Compare `calcParser.binary`'s operator as a rune. `byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. -### 41. Name the node a selector follows in `stepInto`'s refusal. +### 40. Name the node a selector follows in `stepInto`'s refusal. `sv_SE.person[1].first` answers `1 is not a table`. @@ -354,58 +350,70 @@ Both render the same run. `parseRows` answers "line 2 is empty". -### 37. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. +### 36. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. An `int64` or `float64` field unproven in range is told to become itself, and a `uint64` one to become `int64`. -### 42. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. +### 46. Spell a table one way across the errors that name it. + +`pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. + +### 41. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. `readTableOptions` and `loadDir` return on the first in Go's map order. -### 8. Fill `arm` in one place. +### 7. Fill `arm` in one place. `splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. After item 4. -### 35. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. +### 34. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. Both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` shadows the `rng` interface. -### 40. Call `templateError` a compile failure in its doc. +### 39. Call `templateError` a compile failure in its doc. Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. -### 55. Pair a street with its exact postnummer. +### 51. Build on a manual run of `test.yml`. + +`workflow_dispatch` leaves `github.event.before` empty, so the diff compares `HEAD` with itself and skips `docker build`. + +### 42. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. + +`loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. + +### 57. Pair a street with its exact postnummer. Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. -### 54. Add `{btc()}` and `{eth()}`. +### 56. Add `{btc()}` and `{eth()}`. They need sha256 and keccak for Base58Check and EIP-55. -### 52. Promise in `Generator`'s godoc that its renders run one at a time. +### 54. Promise in `Generator`'s godoc that its renders run one at a time. -So a caller wanting parallel throughput makes one generator per goroutine. +A caller wanting parallel throughput then makes one generator per goroutine. -### 58. Cut the README's Layout block to the lines that say what a file name cannot. +### 60. Cut the README's Layout block to the lines that say what a file name cannot. `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -### 53. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. +### 55. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. -Person, address, phone, national id, company and date names each, and the `geo/` trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE their `address` reads by reference; AU and CA get a tree with no locale of their own. +Each locale carries person, address, phone, national id, company and date names. Their `address` reads the `geo/` trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE by reference; AU and CA get a tree with no locale. -### 59. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. +### 61. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. -The round procedure stands in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. +The `gh` rule opens with the README's GitHub decision, so it is a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. -### 57. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. +### 59. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. The check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -### 56. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. +### 58. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. Propose a goal for an entry that serves none. -### 60. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. +### 62. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; v0.2.0 rewrites shipped rows, so a v0.3.0 with no breaking change comes first. -- 2.52.0 From ae70c31d9673e6a8fa2b20c3f5da0ccbc441593e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:01:10 +0200 Subject: [PATCH 420/966] Answer prose round 2: one spelling for a dependency, the recorded score explained, the tld miss filed --- docs/decisions.md | 2 +- todo.md | 109 ++++++++++++++++++++++++---------------------- 2 files changed, 58 insertions(+), 53 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index d84fd0c..c25f0bb 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -808,7 +808,7 @@ domain writes, above, settles the rest: a configuration writes `info`, so RFC 54 The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a host -with no separator. `misc.tld[se]` misses for it, which `todo.md` carries. +with no separator. `misc.tld[se]` misses for it, which `todo.md` item 40 ends. ## `misc.tld` is a table of its own, and `misc.territory.tld` stays a column diff --git a/todo.md b/todo.md index 8930f12..f5e5021 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 64` +`Next ID = 65` | Goal | W | |---|---| @@ -49,12 +49,12 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 53 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.0 | +| 54 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.0 | | 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 24.2 | | 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 23.4 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 22.2 | | 23 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 22.1 | -| 52 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.1 | +| 53 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.1 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.2 | | 24 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 20.1 | | 31 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 20.1 | @@ -63,23 +63,24 @@ | 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 4.4 | 18.7 | | 22 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 17.8 | | 26 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 17.6 | -| 48 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 16.2 | +| 49 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 16.2 | | 18 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 15.9 | -| 49 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.4 | -| 47 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 15.1 | +| 50 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.4 | +| 48 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 15.1 | | 33 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 15.0 | | 19 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 15.0 | | 37 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 14.7 | | 17 | 0.1.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 14.4 | -| 43 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 14.4 | -| 50 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 14.1 | +| 44 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 14.4 | +| 51 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 14.1 | | 27 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 13.9 | | 29 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 13.4 | | 13 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 13.0 | | 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 12.8 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 12.4 | | 11 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 12.4 | -| 45 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 12.1 | +| 40 | 0.1.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or name it.** | 1 | 1 | 4 | 6 | 5 | 12.4 | +| 46 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 12.1 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 11.8 | | 8 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | | 9 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | @@ -89,33 +90,33 @@ | 14 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 10.6 | | 21 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 10.5 | | 35 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 40 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 44 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 41 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 10.4 | +| 45 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 10.4 | | 30 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 10.2 | | 36 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 10.1 | -| 46 | 0.1.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 10.1 | -| 41 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 10.1 | +| 47 | 0.1.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 10.1 | +| 42 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 10.1 | | 32 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 10.0 | | 20 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 9.9 | | 7 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 9.6 | | 34 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | | 39 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | -| 51 | 0.1.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | -| 42 | 0.1.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 7.4 | -| 57 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 17.7 | -| 56 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 14.5 | -| 54 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 11.3 | -| 60 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 10.9 | -| 55 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 10.6 | -| 61 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 10.3 | -| 59 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 10.2 | -| 58 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 9.8 | -| 62 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 29.7 | -| 63 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.1 | +| 52 | 0.1.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | +| 43 | 0.1.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 7.4 | +| 58 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 17.7 | +| 57 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 14.5 | +| 55 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 11.3 | +| 61 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 10.9 | +| 56 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 10.6 | +| 62 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 10.3 | +| 60 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 10.2 | +| 59 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 9.8 | +| 63 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 29.7 | +| 64 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.1 | ## Details -### 53. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. +### 54. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. GoReleaser attaches them to the release the tag workflow publishes. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. @@ -145,7 +146,7 @@ GoReleaser attaches them to the release the tag workflow publishes. A checkout b `{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. -### 52. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. +### 53. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. @@ -182,7 +183,7 @@ Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: | Architect inherited | 6 | 6 | | Mean (Navigation / Locality / Shape / Self-sufficiency) | 5.7 (7.0 / 5.1 / 5.9 / 5.7) | 5.4 (6.3 / 4.8 / 5.8 / 5.3) | -Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Scores since 2026-09-20 range 5.7–6.1, wider than this 0.3 spread by only 0.1, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. The last recorded nine-seat score is 5.7, on 2026-09-30; the ratchet reads it from here until this item is decided. +Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Run 1 is the recorded score; run 2 reran the same code to measure noise. A spread of 0.3 is close to the whole 5.7–6.1 range the recorded scores span since 2026-09-20, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. Until this item is decided, that rule compares against 5.7, from 2026-09-30. ### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. @@ -200,15 +201,15 @@ ISO 3166 reserves it for Western Sahara and the root zone has never been delegat Needs item 4. -### 48. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. +### 49. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. -### 49. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. +### 50. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. -### 47. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. +### 48. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. @@ -233,7 +234,7 @@ Neither names a `weight`, so a bare `sex` draws evenly. `misc.uuid` gains `variant`, for one. -### 43. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. +### 44. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. `probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. @@ -247,7 +248,7 @@ A sheet omitting an empty cell shifts every later column left. ### 10. Test that every node kind reaches each switch over node kinds. -After item 4; recount the switches then. Today ten, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. +Needs item 4; recount the switches then. Today ten, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. ### 15. Add the remaining `misc` tables and templates, one row of its detail's table per chunk. @@ -276,7 +277,11 @@ Shape: T = table, t = template, c = choice. Nothing checks it, and a new country breaks it silently. -### 45. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. +### 40. Have `misc.tld[se]` select the row keyed `.se`, or name it. + +The decision "`misc.tld` keys carry the leading dot, where other tables key on a bare code" leaves `misc.tld[se]` missing today. + +### 46. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. @@ -316,7 +321,7 @@ Shape: T = table, t = template, c = choice. ### 8. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. -The greenfield architect found the fence pipeline last. After item 4, which shrinks the pipeline. +The greenfield architect found the fence pipeline last. Needs item 4, which shrinks the pipeline. ### 9. Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`. @@ -342,11 +347,11 @@ Both render the same run. `byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. -### 40. Name the node a selector follows in `stepInto`'s refusal. +### 41. Name the node a selector follows in `stepInto`'s refusal. `sv_SE.person[1].first` answers `1 is not a table`. -### 44. Report a CRLF rows file holding only its header as having no rows. +### 45. Report a CRLF rows file holding only its header as having no rows. `parseRows` answers "line 2 is empty". @@ -354,17 +359,17 @@ Both render the same run. An `int64` or `float64` field unproven in range is told to become itself, and a `uint64` one to become `int64`. -### 46. Spell a table one way across the errors that name it. +### 47. Spell a table one way across the errors that name it. `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. -### 41. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. +### 42. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. `readTableOptions` and `loadDir` return on the first in Go's map order. ### 7. Fill `arm` in one place. -`splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. After item 4. +`splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. Needs item 4. ### 34. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. @@ -374,46 +379,46 @@ Both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. -### 51. Build on a manual run of `test.yml`. +### 52. Build on a manual run of `test.yml`. `workflow_dispatch` leaves `github.event.before` empty, so the diff compares `HEAD` with itself and skips `docker build`. -### 42. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. +### 43. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. -### 57. Pair a street with its exact postnummer. +### 58. Pair a street with its exact postnummer. Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. -### 56. Add `{btc()}` and `{eth()}`. +### 57. Add `{btc()}` and `{eth()}`. They need sha256 and keccak for Base58Check and EIP-55. -### 54. Promise in `Generator`'s godoc that its renders run one at a time. +### 55. Promise in `Generator`'s godoc that its renders run one at a time. A caller wanting parallel throughput then makes one generator per goroutine. -### 60. Cut the README's Layout block to the lines that say what a file name cannot. +### 61. Cut the README's Layout block to the lines that say what a file name cannot. `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -### 55. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. +### 56. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. Each locale carries person, address, phone, national id, company and date names. Their `address` reads the `geo/` trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE by reference; AU and CA get a tree with no locale. -### 61. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. +### 62. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. -The `gh` rule opens with the README's GitHub decision, so it is a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. +The `gh` rule restates the decision "GitHub is canonical, and the module path names it", so it is a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. -### 59. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. +### 60. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. The check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -### 58. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. +### 59. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. Propose a goal for an entry that serves none. -### 62. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. +### 63. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; v0.2.0 rewrites shipped rows, so a v0.3.0 with no breaking change comes first. -- 2.52.0 From 94f1678c3abeee98483e4380f85f4294af5f30f5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:02:08 +0200 Subject: [PATCH 421/966] Answer prose round 3: the gh rule's restated half, item 40's tagline, items plural --- todo.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/todo.md b/todo.md index f5e5021..662d6c3 100644 --- a/todo.md +++ b/todo.md @@ -79,7 +79,7 @@ | 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 12.8 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 12.4 | | 11 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 12.4 | -| 40 | 0.1.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or name it.** | 1 | 1 | 4 | 6 | 5 | 12.4 | +| 40 | 0.1.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 5 | 12.4 | | 46 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 12.1 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 11.8 | | 8 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | @@ -131,7 +131,7 @@ GoReleaser attaches them to the release the tag workflow publishes. A checkout b - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. - `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. -- Needs item 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. +- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. @@ -277,7 +277,7 @@ Shape: T = table, t = template, c = choice. Nothing checks it, and a new country breaks it silently. -### 40. Have `misc.tld[se]` select the row keyed `.se`, or name it. +### 40. Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`. The decision "`misc.tld` keys carry the leading dot, where other tables key on a bare code" leaves `misc.tld[se]` missing today. @@ -409,7 +409,7 @@ Each locale carries person, address, phone, national id, company and date names. ### 62. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. -The `gh` rule restates the decision "GitHub is canonical, and the module path names it", so it is a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. +The `gh` rule opens by restating the decision "GitHub is canonical, and the module path names it", a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. ### 60. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. -- 2.52.0 From 724937e9ae84525d038c1f8f31fe53815c25f5ee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:11:23 +0200 Subject: [PATCH 422/966] Weigh each goal 0.05 below the one before it, a sub-goal as its goal --- docs/decisions.md | 2 +- todo.md | 477 ++++++++++++++++++++++++---------------------- 2 files changed, 253 insertions(+), 226 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index c25f0bb..e0b20ba 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -808,7 +808,7 @@ domain writes, above, settles the rest: a configuration writes `info`, so RFC 54 The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a host -with no separator. `misc.tld[se]` misses for it, which `todo.md` item 40 ends. +with no separator. `misc.tld[se]` misses for it, which `todo.md` item 45 ends. ## `misc.tld` is a table of its own, and `misc.territory.tld` stays a column diff --git a/todo.md b/todo.md index 662d6c3..ba9e6cc 100644 --- a/todo.md +++ b/todo.md @@ -6,117 +6,124 @@ `Bar = 9` -`Next ID = 65` +`Next ID = 72` | Goal | W | |---|---| | 1 | 1.00 | -| 1.1 | 0.97 | -| 2 | 0.94 | -| 2.1 | 0.91 | -| 2.2 | 0.88 | -| 2.3 | 0.85 | -| 2.4 | 0.82 | -| 3 | 0.79 | -| 3.1 | 0.76 | -| 4 | 0.74 | -| 4.1 | 0.71 | -| 4.2 | 0.68 | -| 4.3 | 0.65 | -| 4.4 | 0.62 | -| 4.5 | 0.59 | -| 4.6 | 0.56 | -| 4.7 | 0.53 | -| 4.8 | 0.50 | -| 5 | 0.47 | -| 6 | 0.44 | -| 6.1 | 0.41 | -| 6.2 | 0.38 | -| 7 | 0.35 | -| 8 | 0.32 | -| 9 | 0.29 | -| 10 | 0.26 | -| 10.1 | 0.24 | -| 10.2 | 0.21 | -| 10.3 | 0.18 | -| 11 | 0.15 | -| 12 | 0.12 | -| 12.1 | 0.09 | -| 13 | 0.06 | -| 14 | 0.03 | +| 1.1 | 1.00 | +| 2 | 0.95 | +| 2.1 | 0.95 | +| 2.2 | 0.95 | +| 2.3 | 0.95 | +| 2.4 | 0.95 | +| 3 | 0.90 | +| 3.1 | 0.90 | +| 4 | 0.85 | +| 4.1 | 0.85 | +| 4.2 | 0.85 | +| 4.3 | 0.85 | +| 4.4 | 0.85 | +| 4.5 | 0.85 | +| 4.6 | 0.85 | +| 4.7 | 0.85 | +| 4.8 | 0.85 | +| 5 | 0.80 | +| 6 | 0.75 | +| 6.1 | 0.75 | +| 6.2 | 0.75 | +| 7 | 0.70 | +| 8 | 0.65 | +| 9 | 0.60 | +| 10 | 0.55 | +| 10.1 | 0.55 | +| 10.2 | 0.55 | +| 10.3 | 0.55 | +| 11 | 0.50 | +| 12 | 0.45 | +| 12.1 | 0.45 | +| 13 | 0.40 | +| 14 | 0.35 | ## Items | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 54 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.0 | -| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 24.2 | -| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 23.4 | -| 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 22.2 | -| 23 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 22.1 | -| 53 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.1 | -| 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.2 | -| 24 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 20.1 | -| 31 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 20.1 | -| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 19.8 | -| 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.8 | -| 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 4.4 | 18.7 | -| 22 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 17.8 | -| 26 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 17.6 | -| 49 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 16.2 | -| 18 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 15.9 | -| 50 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.4 | -| 48 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 15.1 | -| 33 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 15.0 | -| 19 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 15.0 | -| 37 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 14.7 | -| 17 | 0.1.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 14.4 | -| 44 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 14.4 | -| 51 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 14.1 | -| 27 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 13.9 | -| 29 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 13.4 | -| 13 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 13.0 | -| 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 12.8 | -| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 12.4 | -| 11 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 12.4 | -| 40 | 0.1.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 5 | 12.4 | -| 46 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 12.1 | -| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 11.8 | -| 8 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | -| 9 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 11.6 | -| 28 | 0.1.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 11.5 | -| 38 | 0.1.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 11.1 | -| 25 | 0.1.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 10.9 | -| 14 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 10.6 | -| 21 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 10.5 | -| 35 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 41 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 45 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 10.4 | -| 30 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 10.2 | -| 36 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 10.1 | -| 47 | 0.1.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 10.1 | -| 42 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 10.1 | -| 32 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 10.0 | -| 20 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 9.9 | -| 7 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 9.6 | -| 34 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | -| 39 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 9.2 | -| 52 | 0.1.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | -| 43 | 0.1.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 7.4 | -| 58 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 17.7 | -| 57 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 14.5 | -| 55 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 11.3 | -| 61 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 10.9 | -| 56 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 10.6 | -| 62 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 10.3 | -| 60 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 10.2 | -| 59 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 9.8 | -| 63 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 29.7 | -| 64 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.1 | +| 60 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.5 | +| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 26.7 | +| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 25.9 | +| 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 25.0 | +| 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 23.9 | +| 59 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | +| 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 4.4 | 22.4 | +| 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 21.9 | +| 35 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 21.6 | +| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 21.4 | +| 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.3 | +| 37 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 20.3 | +| 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.9 | +| 42 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 19.9 | +| 54 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 19.7 | +| 28 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 19.6 | +| 24 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 19.4 | +| 57 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 18.2 | +| 55 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 17.6 | +| 18 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 17.2 | +| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 17.1 | +| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 16.6 | +| 32 | 0.1.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 16.4 | +| 45 | 0.1.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 5 | 16.4 | +| 20 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 16.3 | +| 52 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 16.1 | +| 56 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | +| 17 | 0.1.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 15.7 | +| 49 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 15.7 | +| 22 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 15.6 | +| 29 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 15.2 | +| 43 | 0.1.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 15.1 | +| 33 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 15.1 | +| 47 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 14.7 | +| 27 | 0.1.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 14.7 | +| 14 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 14.4 | +| 39 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 14.4 | +| 46 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 14.4 | +| 51 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 14.4 | +| 13 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 14.3 | +| 40 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 14.1 | +| 53 | 0.1.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 14.1 | +| 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 13.6 | +| 11 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 13.5 | +| 36 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 13.5 | +| 21 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 13.4 | +| 23 | 0.1.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 14 | 12.4 | +| 8 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | +| 9 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | +| 31 | 0.1.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | +| 41 | 0.1.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | +| 34 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 11.2 | +| 48 | 0.1.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 10.8 | +| 19 | 0.1.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 12.1 | 10.4 | +| 7 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 10.4 | +| 38 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | +| 44 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | +| 30 | 0.1.0 | | **Read `email.local`'s and `username`'s handles from the shipped name tables.** | 3 | 4 | 4 | 6 | 12 | 9.4 | +| 50 | 0.1.0 | | **Print `--help`'s repeat range from `fejkdata.MaxRepeat`.** | 1 | 1 | 2 | 6 | 10.3 | 9.4 | +| 58 | 0.1.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | +| 65 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 19.9 | +| 64 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 15.8 | +| 63 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 15.4 | +| 61 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 14.2 | +| 67 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 13.8 | +| 62 | 0.2.0 | | **Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`.** | 5 | 8 | 7 | 6 | 13, 11 | 13.0 | +| 68 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 12.2 | +| 69 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 11.9 | +| 66 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 11.4 | +| 70 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 30.2 | +| 71 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.5 | ## Details -### 54. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. +### 60. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. GoReleaser attaches them to the release the tag workflow publishes. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. @@ -142,23 +149,27 @@ GoReleaser attaches them to the release the tag workflow publishes. A checkout b - Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. - Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. -### 23. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. +### 25. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. `{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. -### 53. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. +### 59. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. -### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. +### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. -Needs item 4, and runs as item 6 decides. +- `..` goes up one level, to the parent table, and the name after it is that parent's: `{c..country.name}`, and from a locality `{l..municipality..region.name}`. Naming any other table is a load error naming the parent. +- It steps up from a selected row, a drawn row or a name bound to a row. +- After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. +- `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. +- A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. -### 24. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. +### 26. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. -### 31. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. +### 35. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. `isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. @@ -166,6 +177,14 @@ The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid fro `misc.territory[CW].capital` renders ` Willemstad`, with a leading space. +### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. + +Needs item 4, and runs as item 6 decides. + +### 37. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. + +`New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. + ### 6. Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4. Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: @@ -185,43 +204,7 @@ Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Run 1 is the recorded score; run 2 reran the same code to measure noise. A spread of 0.3 is close to the whole 5.7–6.1 range the recorded scores span since 2026-09-20, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. Until this item is decided, that rule compares against 5.7, from 2026-09-30. -### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. - -- `..` goes up one level, to the parent table, and the name after it is that parent's: `{c..country.name}`, and from a locality `{l..municipality..region.name}`. Naming any other table is a load error naming the parent. -- It steps up from a selected row, a drawn row or a name bound to a row. -- After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. -- `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. -- A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. - -### 22. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. - -ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it. - -### 26. Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick. - -Needs item 4. - -### 49. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. - -Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. - -### 50. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. - -GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. - -### 48. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. - -With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. - -### 33. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. - -`New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. - -### 19. Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are. - -Neither names a `weight`, so a bare `sex` draws evenly. - -### 37. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. +### 42. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. Steps: - Have a selector that misses name the key of every row whose cell spells it, matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, and `misc.loglevel[Error]` `[err]`. @@ -230,25 +213,21 @@ Neither names a `weight`, so a bare `sex` draws evenly. - Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. - Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`. -### 17. Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape. +### 54. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. -`misc.uuid` gains `variant`, for one. +With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. -### 44. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. +### 28. Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick. -`probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. +Needs item 4. -### 27. Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`. +### 24. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. -Require the path step to reach a sibling category. +ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it. -### 13. Place xlsx cells by their `r` reference in `data-import/xlsx.py`. +### 55. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. -A sheet omitting an empty cell shifts every later column left. - -### 10. Test that every node kind reaches each switch over node kinds. - -Needs item 4; recount the switches then. Today ten, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. +Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. ### 15. Add the remaining `misc` tables and templates, one row of its detail's table per chunk. @@ -273,18 +252,6 @@ Shape: T = table, t = template, c = choice. | `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | | `direction`, `continent`, `ulid` | c/t | — | — | -### 11. Test that every `geo` tree holds the five table names and the `address` columns that port across countries. - -Nothing checks it, and a new country breaks it silently. - -### 40. Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`. - -The decision "`misc.tld` keys carry the leading dot, where other tables key on a bare code" leaves `misc.tld[se]` missing today. - -### 46. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. - -Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. - ### 16. Add the remaining locale categories: company, phone, finance, vehicle, words. Shape: T = table, t = template, c = choice. @@ -319,6 +286,90 @@ Shape: T = table, t = template, c = choice. | `word`, `sentence`, `paragraph` | T+t | Moby POS or WordNet | public domain / WordNet | | month and weekday names, `holiday` | t+T | CLDR en | Unicode | +### 32. Rename `misc.territory.country` to `sovereign`. + +It holds a code, and `country` collides with the `misc.country` path it replaced. + +### 45. Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`. + +The decision "`misc.tld` keys carry the leading dot, where other tables key on a bare code" leaves `misc.tld[se]` missing today. + +### 20. Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are. + +Neither names a `weight`, so a bare `sex` draws evenly. + +### 52. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. + +Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. + +### 56. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. + +GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. + +### 17. Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape. + +`misc.uuid` gains `variant`, for one. + +### 49. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. + +`probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. + +### 22. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. + +17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. + +### 29. Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`. + +Require the path step to reach a sibling category. + +### 43. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. + +`datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. + +### 47. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. + +`readTableOptions` and `loadDir` return on the first in Go's map order. + +### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. + +Both render the same run. + +### 39. Compare `calcParser.binary`'s operator as a rune. + +`byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. + +### 46. Name the node a selector follows in `stepInto`'s refusal. + +`sv_SE.person[1].first` answers `1 is not a table`. + +### 51. Report a CRLF rows file holding only its header as having no rows. + +`parseRows` answers "line 2 is empty". + +### 13. Place xlsx cells by their `r` reference in `data-import/xlsx.py`. + +A sheet omitting an empty cell shifts every later column left. + +### 40. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. + +An `int64` or `float64` field unproven in range is told to become itself, and a `uint64` one to become `int64`. + +### 53. Spell a table one way across the errors that name it. + +`pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. + +### 10. Test that every node kind reaches each switch over node kinds. + +Needs item 4; recount the switches then. Today ten, from `render` to `columnItems`, must agree, and only `render` and `renderEdges` say so. + +### 11. Test that every `geo` tree holds the five table names and the `address` columns that port across countries. + +Nothing checks it, and a new country breaks it silently. + +### 23. Give every other shipped category that could reach something real a path that never does. + +`phone` draws live PTS and NANP ranges — PTS's five fiction series are the Swedish inert set, sourced in [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. + ### 8. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. The greenfield architect found the fence pipeline last. Needs item 4, which shrinks the pipeline. @@ -327,98 +378,74 @@ The greenfield architect found the fence pipeline last. Needs item 4, which shri The inherited architect's 3am trace crossed four files for one draw. -### 28. Rename `misc.territory.country` to `sovereign`. +### 41. Parse `defaultTable`'s path with the library's grammar. -It holds a code, and `country` collides with the `misc.country` path it replaced. +The CLI counts brackets of its own, so a change to the selector grammar desyncs the default `--table` silently. -### 38. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. +### 48. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. -`datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. - -### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. - -Both render the same run. - -### 21. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. - -17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. - -### 35. Compare `calcParser.binary`'s operator as a rune. - -`byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. - -### 41. Name the node a selector follows in `stepInto`'s refusal. - -`sv_SE.person[1].first` answers `1 is not a table`. - -### 45. Report a CRLF rows file holding only its header as having no rows. - -`parseRows` answers "line 2 is empty". - -### 36. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. - -An `int64` or `float64` field unproven in range is told to become itself, and a `uint64` one to become `int64`. - -### 47. Spell a table one way across the errors that name it. - -`pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. - -### 42. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. - -`readTableOptions` and `loadDir` return on the first in Go's map order. +`loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. ### 7. Fill `arm` in one place. `splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. Needs item 4. -### 34. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. +### 38. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. Both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` shadows the `rng` interface. -### 39. Call `templateError` a compile failure in its doc. +### 44. Call `templateError` a compile failure in its doc. Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. -### 52. Build on a manual run of `test.yml`. +### 30. Read `email.local`'s and `username`'s handles from the shipped name tables. + +Keep the handle shape: `username` shortens a surname to `ahl` or `sjo`, and the US one draws unisex given names, neither of which falls out of the tables unaided. + +### 50. Print `--help`'s repeat range from `fejkdata.MaxRepeat`. + +The usage spells `1..1048576` a second time. + +### 58. Build on a manual run of `test.yml`. `workflow_dispatch` leaves `github.event.before` empty, so the diff compares `HEAD` with itself and skips `docker build`. -### 43. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. - -`loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. - -### 58. Pair a street with its exact postnummer. +### 65. Pair a street with its exact postnummer. Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. -### 57. Add `{btc()}` and `{eth()}`. +### 64. Add `{btc()}` and `{eth()}`. They need sha256 and keccak for Base58Check and EIP-55. -### 55. Promise in `Generator`'s godoc that its renders run one at a time. - -A caller wanting parallel throughput then makes one generator per goroutine. - -### 61. Cut the README's Layout block to the lines that say what a file name cannot. - -`calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. - -### 56. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. +### 63. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. Each locale carries person, address, phone, national id, company and date names. Their `address` reads the `geo/` trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE by reference; AU and CA get a tree with no locale. -### 62. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. +### 61. Promise in `Generator`'s godoc that its renders run one at a time. -The `gh` rule opens by restating the decision "GitHub is canonical, and the module path names it", a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. +A caller wanting parallel throughput then makes one generator per goroutine. -### 60. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. +### 67. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. The check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -### 59. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. +### 62. Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`. + +Every US place of 10,000 and more streets per locality, built by `--min-population` and `--streets-per-locality`. The default embed stays under ~1 MB per country and `New` under ~50 ms. + +### 68. Cut the README's Layout block to the lines that say what a file name cannot. + +`calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. + +### 69. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. + +The `gh` rule opens by restating the decision "GitHub is canonical, and the module path names it", a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. + +### 66. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. Propose a goal for an entry that serves none. -### 63. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. +### 70. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; v0.2.0 rewrites shipped rows, so a v0.3.0 with no breaking change comes first. -- 2.52.0 From 0c1e380bede341bae1d8a382489114ad34d780ee Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:18:38 +0200 Subject: [PATCH 423/966] Ship v0.1.0 before packaging, tier the releases by score, and keep the Gitea mirror by decision --- AGENTS.md | 1 + docs/decisions.md | 9 +++ todo.md | 137 ++++++++++++++++++++++++---------------------- 3 files changed, 81 insertions(+), 66 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b5d3bfc..4bcf150 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,6 +31,7 @@ In [docs/decisions.md](docs/decisions.md): - GitHub is canonical, and the module path names it +- The Gitea copy stays, as a pull mirror of GitHub - The vocabulary sits below `doc.go`'s package clause, not in the package doc - Options and fields share one namespace - `{a|b}` stays beside nested choices diff --git a/docs/decisions.md b/docs/decisions.md index e0b20ba..b33d25c 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -11,6 +11,15 @@ a fork on a personal instance. Valid while the project wants contribution from o the workflow decided nothing, being portable already — `github.api_url` and `secrets.GITHUB_TOKEN` resolve on either host, so only the names moved. + +## The Gitea copy stays, as a pull mirror of GitHub + +2026-10-03, decided by the maintainer. Technical principle: one owner per value +(`~/.claude/principles/technical-principles.md`). GitHub owns the history, and +`gitea.larvit.se/larvit/fejkdata` stays as a read-only pull mirror of it, so it is never +a second owner. Until `todo.md` item 70 converts it, the Gitea copy goes stale. Valid +while the maintainer keeps a Gitea copy. + ## The vocabulary sits below `doc.go`'s package clause, not in the package doc 2026-09-21, Lilleman auf Larv. diff --git a/todo.md b/todo.md index ba9e6cc..09faa2d 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 72` +`Next ID = 73` | Goal | W | |---|---| @@ -49,7 +49,6 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 60 | 0.1.0 | | **Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go.** | 3 | 6 | 8 | 9 | 1.1, 6 | 29.5 | | 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 26.7 | | 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 25.9 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 25.0 | @@ -61,72 +60,70 @@ | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 21.4 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.3 | | 37 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 20.3 | -| 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.9 | -| 42 | 0.1.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 19.9 | -| 54 | 0.1.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 19.7 | -| 28 | 0.1.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 19.6 | -| 24 | 0.1.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 19.4 | -| 57 | 0.1.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 18.2 | -| 55 | 0.1.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 17.6 | -| 18 | 0.1.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 17.2 | -| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 17.1 | -| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 16.6 | -| 32 | 0.1.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 16.4 | -| 45 | 0.1.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 5 | 16.4 | -| 20 | 0.1.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 16.3 | -| 52 | 0.1.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 16.1 | -| 56 | 0.1.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | -| 17 | 0.1.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 15.7 | -| 49 | 0.1.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 15.7 | -| 22 | 0.1.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 15.6 | -| 29 | 0.1.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 15.2 | -| 43 | 0.1.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 15.1 | -| 33 | 0.1.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 15.1 | -| 47 | 0.1.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 14.7 | -| 27 | 0.1.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 14.7 | -| 14 | 0.1.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 14.4 | -| 39 | 0.1.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 14.4 | -| 46 | 0.1.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 14.4 | -| 51 | 0.1.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 14.4 | -| 13 | 0.1.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 14.3 | -| 40 | 0.1.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 14.1 | -| 53 | 0.1.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 14.1 | -| 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 13.6 | -| 11 | 0.1.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 13.5 | -| 36 | 0.1.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 13.5 | -| 21 | 0.1.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 13.4 | -| 23 | 0.1.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 14 | 12.4 | -| 8 | 0.1.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | -| 9 | 0.1.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | -| 31 | 0.1.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | -| 41 | 0.1.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | -| 34 | 0.1.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 11.2 | -| 48 | 0.1.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 10.8 | -| 19 | 0.1.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 12.1 | 10.4 | -| 7 | 0.1.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 10.4 | -| 38 | 0.1.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | -| 44 | 0.1.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | -| 30 | 0.1.0 | | **Read `email.local`'s and `username`'s handles from the shipped name tables.** | 3 | 4 | 4 | 6 | 12 | 9.4 | -| 50 | 0.1.0 | | **Print `--help`'s repeat range from `fejkdata.MaxRepeat`.** | 1 | 1 | 2 | 6 | 10.3 | 9.4 | -| 58 | 0.1.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | -| 65 | 0.2.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 19.9 | -| 64 | 0.2.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 15.8 | -| 63 | 0.2.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 15.4 | -| 61 | 0.2.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 14.2 | -| 67 | 0.2.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 13.8 | -| 62 | 0.2.0 | | **Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`.** | 5 | 8 | 7 | 6 | 13, 11 | 13.0 | -| 68 | 0.2.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 12.2 | -| 69 | 0.2.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 11.9 | -| 66 | 0.2.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 11.4 | -| 70 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 30.2 | -| 71 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.5 | +| 60 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 6 | 29.2 | +| 6 | 0.2.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.9 | +| 42 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 19.9 | +| 54 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 19.7 | +| 28 | 0.2.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 19.6 | +| 24 | 0.2.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 19.4 | +| 57 | 0.2.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 18.2 | +| 55 | 0.2.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 17.6 | +| 18 | 0.2.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 17.2 | +| 15 | 0.2.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 17.1 | +| 16 | 0.2.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 16.6 | +| 32 | 0.2.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 16.4 | +| 45 | 0.2.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 5 | 16.4 | +| 20 | 0.2.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 16.3 | +| 52 | 0.2.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 16.1 | +| 56 | 0.2.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | +| 17 | 0.2.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 15.7 | +| 49 | 0.2.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 15.7 | +| 22 | 0.2.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 15.6 | +| 29 | 0.2.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 15.2 | +| 43 | 0.2.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 15.1 | +| 33 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 15.1 | +| 47 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 14.7 | +| 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 14.7 | +| 14 | 0.2.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 14.4 | +| 39 | 0.2.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 14.4 | +| 46 | 0.2.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 14.4 | +| 51 | 0.2.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 14.4 | +| 13 | 0.2.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 14.3 | +| 40 | 0.2.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 14.1 | +| 53 | 0.2.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 14.1 | +| 10 | 0.3.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 13.6 | +| 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 13.5 | +| 36 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 13.5 | +| 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 13.4 | +| 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 14 | 12.4 | +| 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | +| 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | +| 31 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | +| 41 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | +| 34 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 11.2 | +| 48 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 10.8 | +| 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 12.1 | 10.4 | +| 7 | 0.3.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 10.4 | +| 38 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | +| 44 | 0.3.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | +| 30 | 0.3.0 | | **Read `email.local`'s and `username`'s handles from the shipped name tables.** | 3 | 4 | 4 | 6 | 12 | 9.4 | +| 50 | 0.3.0 | | **Print `--help`'s repeat range from `fejkdata.MaxRepeat`.** | 1 | 1 | 2 | 6 | 10.3 | 9.4 | +| 58 | 0.3.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | +| 65 | 0.4.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 19.9 | +| 64 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 15.8 | +| 63 | 0.4.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 15.4 | +| 61 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 14.2 | +| 67 | 0.4.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 13.8 | +| 62 | 0.4.0 | | **Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`.** | 5 | 8 | 7 | 6 | 13, 11 | 13.0 | +| 68 | 0.4.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 12.2 | +| 69 | 0.4.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 11.9 | +| 66 | 0.4.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 11.4 | +| 70 | 0.4.0 | decision | **Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub.** | 2 | 1 | 1 | 3 | 1.1 | 5.8 | +| 71 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 30.2 | +| 72 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.5 | ## Details -### 60. Ship prebuilt binaries, a container image, Homebrew and Scoop, so the CLI needs no Go. - -GoReleaser attaches them to the release the tag workflow publishes. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. - ### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. - A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. @@ -185,6 +182,10 @@ Needs item 4, and runs as item 6 decides. `New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. +### 60. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. + +GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. + ### 6. Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4. Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: @@ -446,6 +447,10 @@ The `gh` rule opens by restating the decision "GitHub is canonical, and the modu Propose a goal for an entry that serves none. -### 70. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. +### 70. Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub. + +Gitea converts a repository to a mirror only by re-creating it, so until that runs the copy there is a second owner of one history. + +### 71. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; v0.2.0 rewrites shipped rows, so a v0.3.0 with no breaking change comes first. -- 2.52.0 From d980e84779aedba3d54324d74676e9c4985455bf Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:18:50 +0200 Subject: [PATCH 424/966] Point v1.0.0's quiet minor at the release that rewrites shipped rows --- todo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/todo.md b/todo.md index 09faa2d..4a46c2f 100644 --- a/todo.md +++ b/todo.md @@ -453,4 +453,4 @@ Gitea converts a repository to a mirror only by re-creating it, so until that ru ### 71. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. -The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; v0.2.0 rewrites shipped rows, so a v0.3.0 with no breaking change comes first. +The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; pairing a street with its exact postnummer rewrites shipped rows, so a minor with no breaking change follows it first. -- 2.52.0 From c7f98858211cdb2fa06e312e7d3fdaec0a52f02d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:23:34 +0200 Subject: [PATCH 425/966] Add goal 2, enough built-in data for the ten most-spoken languages and the Nordics, and renumber every goal reference --- AGENTS.md | 2 +- DATA-LICENSES.md | 2 +- README.md | 40 ++- docs/decisions.md | 62 ++-- docs/research/research-sources-se.md | 2 +- todo.md | 513 +++++++++++++-------------- 6 files changed, 309 insertions(+), 312 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 4bcf150..88a90b3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,7 +9,7 @@ - CodeRabbit skips every pull request until the repository has 10 GitHub stars: check its status once when a PR is marked ready, answer a review if one is there, and never wait for one. -- README goal 2 is gated at 7.0 on the `comprehension-panel` skill, and below it +- README goal 3 is gated at 7.0 on the `comprehension-panel` skill, and below it nothing else merges — no feature, no category, no data — bar a security fix, a dependency bump, a repair of behaviour the README documents, and the infrastructure the round itself runs on. `todo.md` carries the round: a PR per item it plans, the nine-seat diff --git a/DATA-LICENSES.md b/DATA-LICENSES.md index 8da2f33..368b611 100644 --- a/DATA-LICENSES.md +++ b/DATA-LICENSES.md @@ -14,7 +14,7 @@ Every shipped dataset, its source, its licence and the attribution it asks for. | `sv_SE/first-name.tsv`, `last-name.tsv` | [SCB](https://www.scb.se/) names with at least two bearers, 31 December 2022 | CC0 1.0 | "Källa: SCB" | `data-import/names-se.py` | | `en_US/first-name.tsv` | [SSA](https://www.ssa.gov/oact/babynames/) baby names, births 1930 to 2020, through [hackerb9/ssa-baby-names](https://github.com/hackerb9/ssa-baby-names) | public domain | none required | `data-import/names-us.py` | | `en_US/last-name.tsv` | Census Bureau surnames occurring 100 or more times, 2010 | public domain | none required | `data-import/names-us.py` | -| `sv_SE/sex.tsv`, `sv_SE/birth-number.tsv`, `sv_SE/title.tsv`, `en_US/sex.tsv`, `en_US/title.tsv`, `misc/car.tsv` | curated; Skatteverket's test birth numbers are facts, and `car.tsv` waits for an international source (goal 12) | — | — | — | +| `sv_SE/sex.tsv`, `sv_SE/birth-number.tsv`, `sv_SE/title.tsv`, `en_US/sex.tsv`, `en_US/title.tsv`, `misc/car.tsv` | curated; Skatteverket's test birth numbers are facts, and `car.tsv` waits for an international source (goal 13) | — | — | — | | `misc/currency.tsv` | [datasets/currency-codes](https://github.com/datasets/currency-codes); symbols from [Unicode CLDR](https://github.com/unicode-org/cldr) `en.xml` and `root.xml` | PDDL 1.0; [Unicode License v3](https://www.unicode.org/license.txt) | CLDR: "Copyright © 1991-2025 Unicode, Inc. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other countries." | `data-import/currency.py` | | `misc/httpmethod.tsv` | [IANA HTTP Method Registry](https://www.iana.org/assignments/http-methods/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpmethod.py` | | `misc/httpstatus.tsv` | [IANA HTTP Status Code Registry](https://www.iana.org/assignments/http-status-codes/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpstatus.py` | diff --git a/README.md b/README.md index 271714e..00b1082 100644 --- a/README.md +++ b/README.md @@ -176,9 +176,9 @@ zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.sym Where a script under [`data-import/`](data-import) reads a source, the facts come from that source, and the script drops a row only by a rule it states; elsewhere a small hand-written -set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 12). Copy +set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 13). Copy that states no fact, such as `word` and `sentence`, stays hand-written. Where a category -has no path that draws only values reaching nothing real (goal 14), write a fixed value such +has no path that draws only values reaching nothing real (goal 15), write a fixed value such as `example.com` into the template; `todo.md` plans the path. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. @@ -767,7 +767,7 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance -The target is about a microsecond per value (goal 11). +The target is about a microsecond per value (goal 12). Each file is parsed, validated and weight-indexed once, in `New`, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends @@ -834,16 +834,20 @@ Where the project is heading; the sections before Audience document what ships t 1. **The best fake-data tool there is, and the one developers reach for.** 1. You can install it the way you install your other tools, and people hear about it where developers read. -2. **The code is easy to understand.** +2. **Enough data ships built in to fill a real test database, with no download.** + 1. It covers the ten most-spoken languages and the Nordic countries. + 2. Each locale covers the common categories: names, addresses, phone numbers, ids, + companies and dates. +3. **The code is easy to understand.** 1. You can go from a bug to the code behind it without asking anyone. 2. You can change one part without keeping the rest in your head. 3. Every name says what the thing really is. 4. Each file makes sense on its own, without another document open beside it. -3. **Every value is valid.** It passes the same check the real system would apply, +4. **Every value is valid.** It passes the same check the real system would apply, such as the checksum in a Swedish personal number. 1. Values that belong together come from the same random pick. You choose which ones by giving that pick a name. -4. **Templates are easy to write and read.** A template is the text you write to +5. **Templates are easy to write and read.** A template is the text you write to describe a value, such as `{first} {last}`. 1. Everything outside `{…}` prints exactly as written. 2. Each `{…}` makes a new random pick. @@ -857,25 +861,25 @@ Where the project is heading; the sections before Audience document what ships t shows the right one. 8. When a clearer name turns up, it replaces the old one, even if that needs a new major version. -5. **Mistakes are caught when the data loads, and the error says plainly what is +6. **Mistakes are caught when the data loads, and the error says plainly what is wrong.** -6. **One command gets you a value.** +7. **One command gets you a value.** 1. Flags work like in other command-line tools, and can go anywhere on the line. 2. Your first template of your own needs no escaping and no options. -7. **Data is JSON files. Code is only for what JSON can't express.** -8. **The same seed always gives the same output.** -9. **It depends on nothing but Go's standard library.** -10. **The docs are a map of the template language.** +8. **Data is JSON files. Code is only for what JSON can't express.** +9. **The same seed always gives the same output.** +10. **It depends on nothing but Go's standard library.** +11. **The docs are a map of the template language.** 1. Every feature has its own heading. 2. Every example runs as a test and shows its output. 3. Each rule is stated in one place. -11. **It is fast enough that you never notice it.** -12. **Every fact comes from an official or open source,** such as a government +12. **It is fast enough that you never notice it.** +13. **Every fact comes from an official or open source,** such as a government register. 1. A table holds every row its source holds, and nothing added by hand. -13. **The most common data comes first,** such as names, addresses, phone numbers and +14. **The most common data comes first,** such as names, addresses, phone numbers and dates. -14. **Values look real by default. You can also ask for values that can never reach +15. **Values look real by default. You can also ask for values that can never reach anything real,** such as a domain nobody can register. ## Layout @@ -959,8 +963,8 @@ docker build --build-arg GO_VERSION=1.22.12 --target portable . # lowest support GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build ``` -Goal 2 is scored by a panel of simulated readers, and [`AGENTS.md`](AGENTS.md) says -what the score asks of a pull request. Goal 4 is checked against simulated template +Goal 3 is scored by a panel of simulated readers, and [`AGENTS.md`](AGENTS.md) says +what the score asks of a pull request. Goal 5 is checked against simulated template writers from the Audience. A change to the shipped data re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) diff --git a/docs/decisions.md b/docs/decisions.md index b33d25c..2eb6010 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -4,7 +4,7 @@ 2026-09-20, Lilleman auf Larv. -Goal 1 wants the usage the tool earns and goal 2 expects extenders who did not write it; +Goal 1 wants the usage the tool earns and goal 3 expects extenders who did not write it; both need a stranger to file an issue and open a pull request. Gitea has no anonymous issue, and no cross-host pull request at all, so a contributor would need an account and a fork on a personal instance. Valid while the project wants contribution from outside: @@ -17,7 +17,7 @@ the workflow decided nothing, being portable already — `github.api_url` and 2026-10-03, decided by the maintainer. Technical principle: one owner per value (`~/.claude/principles/technical-principles.md`). GitHub owns the history, and `gitea.larvit.se/larvit/fejkdata` stays as a read-only pull mirror of it, so it is never -a second owner. Until `todo.md` item 70 converts it, the Gitea copy goes stale. Valid +a second owner. Until `todo.md` item 68 converts it, the Gitea copy goes stale. Valid while the maintainer keeps a Gitea copy. ## The vocabulary sits below `doc.go`'s package clause, not in the package doc @@ -27,7 +27,7 @@ while the maintainer keeps a Gitea copy. Goal 1 is what a developer choosing a library reads first, and the package doc is that page: the glossary would fill two thirds of it with the render path's units, mostly unexported, and Go doc comments have no code markup, so every name would render its -backticks literally. Below the clause it reaches goal 2's reader — who opens the file — +backticks literally. Below the clause it reaches goal 3's reader — who opens the file — and nothing else. Valid while the glossary's subject is the render path; one written to teach the exported API belongs above the clause. @@ -118,14 +118,14 @@ still lives in `data/` as JSON; `--data-path` layers over it. 2026-09-02, Lilleman auf Larv. -Goal 4: `{/p} {/p}` is two draws, as `{word} {word}` is, while every `{/p.first}` in +Goal 5: `{/p} {/p}` is two draws, as `{word} {word}` is, while every `{/p.first}` in one render and draw group reads one draw, and a bare `{/p}` beside them is a load error, as `{p}` beside `{p.first}` is: a bare token is by contract an independent draw, a path pins its level, and a fresh draw of a pinned level could show another row. A builtin's operand holds what it reads for its expansion, references included, so `{uppercase(/p)} {/p}` is one draw, and `{/p}` written twice beside it is a load error. -Goal 4.2 overrides this; `todo.md` item 4 replaces it. +Goal 5.2 overrides this; `todo.md` item 4 replaces it. ## Reference sigils follow the filesystem @@ -161,12 +161,12 @@ this repo's own are. 2026-09-16, Lilleman auf Larv. -Goal 5. A script branches on the exit code and reads the named path or spelling, so +Goal 6. A script branches on the exit code and reads the named path or spelling, so those hold; wording improves in a minor. ## A load reports every mistake at once -2026-09-30, Lilleman auf Larv. Goal 5. An author learns everything wrong with the data +2026-09-30, Lilleman auf Larv. Goal 6. An author learns everything wrong with the data in one load, not one mistake per run: `New` and `NewTemplate` report every mistake in one error, ordered by the path each names. A mistake following only from another, such as a reference into a category that did not compile, is left out, so each one names something @@ -174,7 +174,7 @@ to fix. Valid while a load checks the whole set before its first render. ## Formats are checked after all data is loaded, so their errors name the category -2026-09-28, larv-review; approved 2026-09-28 by lilleman. Goal 2. Loading reads every +2026-09-28, larv-review; approved 2026-09-28 by lilleman. Goal 3. Loading reads every file first, then connects references across them. A format holding a reference can only be checked once it is connected, so every format is checked then, and all checks run at one moment. An error from those checks names the category path you would type (`sub.x`), @@ -196,7 +196,7 @@ convention of a minor is not followed. literal's values out to a lone long key, so no source satisfies both it and 1.22's. A consumer on the lowest supported Go depends on the code compiling and its tests passing there, which is the `portable` stage, and never runs `gofmt` over this source, so the -latest toolchain alone defines the one canonical form goal 2 asks a reader to meet. +latest toolchain alone defines the one canonical form goal 3 asks a reader to meet. Valid while the lowest supported Go is not the latest. ## The changelog heading is the one spelling of a release; CI cuts the tag @@ -304,7 +304,7 @@ rather than two copies of it. Only references share: a sibling field is local to expansion, so a `first` column does not silently bind to a `first` in the column next to it. -Goal 4.2 overrides this; `todo.md` item 4 replaces it. +Goal 5.2 overrides this; `todo.md` item 4 replaces it. ## A draw group name is local to its category @@ -321,7 +321,7 @@ across categories must agree. 2026-09-16, Lilleman auf Larv. -Goal 2: one proves a sibling path or an operand is reached only by its readers within +Goal 3: one proves a sibling path or an operand is reached only by its readers within an expansion, the other that reference paths, whole-table reads and a table family's rows agree across a render and its draw groups. Their scopes differ and only the second tracks rows and pins, so one walk would carry both scopes and tell them apart at every @@ -333,12 +333,12 @@ step. 2026-09-29, Lilleman auf Larv. -Every render starts from empty draws, its draw groups included, since goal 2 prices +Every render starts from empty draws, its draw groups included, since goal 3 prices a reader meeting two spellings of one empty set. Making the maps where the draws are declared kept a record's on its frame's stack, but a `Fake`'s draws live on the `Generator`, so there it cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09-28. Lazily, a render reading a reference path pays those two allocations, a record or struct -included, and one reading none pays nothing; goal 11 holds either way. +included, and one reading none pays nothing; goal 12 holds either way. `todo.md` item 4 deletes what this decides. @@ -347,7 +347,7 @@ included, and one reading none pays nothing; goal 11 holds either way. 2026-09-16, Lilleman auf Larv. A category is one unit: a reference back into it — `{/users.first}` inside `users` — -describes a draw other than the fields beside it, where goal 3 asks facts that belong +describes a draw other than the fields beside it, where goal 4 asks facts that belong together to come from one draw, so `New` refuses it and the sibling path stays the one spelling for a field of one's own. A value two fields share goes in its own category, which both reference. That settled, a record's column fences run at `New` too, so a @@ -413,7 +413,7 @@ writes as the load check proved it; a `Value any` would hand every caller a type 2026-09-29, Lilleman auf Larv. -Goal 2 wants a name to reach one unit: `drawMemo` is what a draw is kept in, `readValue` what +Goal 3 wants a name to reach one unit: `drawMemo` is what a draw is kept in, `readValue` what a read drew. ## One name, one meaning @@ -423,8 +423,8 @@ definition covers both: the same kind of value, counted or addressed the same wa reader landing on the wrong one concludes nothing false, as `Fake` renders one value on `Generator`, `Template` and `RecordTemplate`. `TestNoFunctionSpellsAMethod` keeps a bare function off a method's name, since a search returns both and a call site shows no receiver; review judges the -rest. Serves goal 2: names that tell the truth, and reaching the unit behind a symptom -without asking a person. Decided 2026-09-25 by Lilleman; valid while goal 2 counts reading +rest. Serves goal 3: names that tell the truth, and reaching the unit behind a symptom +without asking a person. Decided 2026-09-25 by Lilleman; valid while goal 3 counts reading cost. ## The package stays flat @@ -478,7 +478,7 @@ accepted one changes, so the door stays open for the address records the plan de ## A selected row is fixed, not drawn, and an unnamed read beside it is refused -Decided 2026-09-27 by the maintainer, for goals 3 and 4; valid while a selector names +Decided 2026-09-27 by the maintainer, for goals 4 and 5; valid while a selector names exactly one row. - Selected rows never conflict: `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` @@ -496,11 +496,11 @@ exactly one row. Checked against three simulated template writers from the Audience, who agreed on each case but the last, where two expected a render and all three accepted the refusal. -Goal 4.2 overrides its shared draw beneath a selection and its refusal of an unnamed read; `todo.md` item 4 replaces them. +Goal 5.2 overrides its shared draw beneath a selection and its refusal of an unnamed read; `todo.md` item 4 replaces them. ## A link column after a row steps up to the parent row, and a path may not end on it -Decided 2026-09-27 by the maintainer, for goal 4; valid while a link column holds the +Decided 2026-09-27 by the maintainer, for goal 5; valid while a link column holds the parent's key. It extends [After a row, a path names a column or a linked table](#after-a-row-a-path-names-a-column-or-a-linked-table) from child tables to the parent. A row is selected or drawn alike. @@ -526,11 +526,11 @@ Checked against three simulated template writers from the Audience: all three re parent row where a segment followed, and they split between the code and the name where the path ended. -Goal 4.4 overrides this; `todo.md` item 2 replaces it. +Goal 5.4 overrides this; `todo.md` item 2 replaces it. ## A table selects by one key column, a code, never a free-form name -Decided 2026-09-27 by the maintainer, for goal 4's one spelling per result; valid while +Decided 2026-09-27 by the maintainer, for goal 5's one spelling per result; valid while the key is the code a user writes, as a database keys a table. `country[NO]` loads, and `country[Norway]` is refused naming `[NO]`, found by the row whose cell spells the selector. The key is the friendliest code the register holds, `misc.territory[SE]` and @@ -600,9 +600,9 @@ own-family fence walks through both rather than stopping where the draw fences d 2026-09-29, Lilleman auf Larv. -Goal 3: facts that belong together come from one draw, so a shop row's `phone` cell reads +Goal 4: facts that belong together come from one draw, so a shop row's `phone` cell reads `{/sv_SE.phone}` on a Swedish shop and `{/en_US.phone}` on a US one, and the table's -format, the same for every row, cannot say that. A restructure for goal 2 keeps the fences +format, the same for every row, cannot say that. A restructure for goal 3 keeps the fences that prove such cells safe. Valid while a table's rows need generators of their own. `todo.md` item 4 deletes the fences this keeps; a cell may still read a reference. @@ -623,7 +623,7 @@ is until a real data set needs the indexed form. 2026-09-17, Lilleman auf Larv. -Goals 8 and 11: a path that fails moves no seeded stream, at the cost of one draw-free +Goals 9 and 12: a path that fails moves no seeded stream, at the cost of one draw-free walk per call, compiling its steps into a stack buffer of 16. Valid while that walk costs little against the draw it guards. @@ -765,7 +765,7 @@ naming the row. 2026-09-20, Lilleman auf Larv. `misc.httpmethod`, `misc.port`, `misc.httpstatus`, `misc.mimetype` and `misc.tld` weigh -every row alike, so GET is a ninth of the methods drawn. Goal 12 keeps an authored fact +every row alike, so GET is a ninth of the methods drawn. Goal 13 keeps an authored fact out of a sourced table, and no register publishes how often a method, a port or a TLD is used, so a weight here would be invented. Where one exists it is read, as `misc.timezone` reads GeoNames populations and `sv_SE.first-name` SCB bearers. @@ -775,7 +775,7 @@ used, so a weight here would be invented. Where one exists it is read, as 2026-09-24, Lilleman auf Larv. The weight is the population GeoNames records in the zone's cities of 15,000 or more, -floored at 15,000, which goal 14 asks for: over 300 seeded draws of +floored at 15,000, which goal 15 asks for: over 300 seeded draws of `misc.territory[US].timezone`, the four zones most Americans live in took 278 where an even weight gave them 44, and `America/Indiana/Petersburg`, a town of 2,400, fell from 17 to 0. Valid while a draw weighted this way lands where people live. @@ -807,7 +807,7 @@ claim fejkdata has no business making. 2026-09-27, Lilleman auf Larv. A flat list of names carries neither the code a PRI encodes nor a selector reaching it, -and goal 3 draws the two as one fact. The canonical spelling losing to the one its +and goal 4 draws the two as one fact. The canonical spelling losing to the one its domain writes, above, settles the rest: a configuration writes `info`, so RFC 5424's `Informational` stays the `severity` column. @@ -817,14 +817,14 @@ domain writes, above, settles the rest: a configuration writes `info`, so RFC 54 The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a host -with no separator. `misc.tld[se]` misses for it, which `todo.md` item 45 ends. +with no separator. `misc.tld[se]` misses for it, which `todo.md` item 44 ends. ## `misc.tld` is a table of its own, and `misc.territory.tld` stays a column 2026-09-20, Lilleman auf Larv. A `parent` demands a child for every parent row, so linking them would drop every root -zone row naming no territory, which is most of them, and goal 12 holds a sourced table +zone row naming no territory, which is most of them, and goal 13 holds a sourced table whole. The loader refuses the link outright anyway: `tld` is a column of `misc.territory`, and a table may not be named like a column of its ancestor. @@ -880,7 +880,7 @@ the direct chain is the one a reader can predict from the tables' parents. 2026-09-30, larv-review on the comprehension round the maintainer approved on 2026-09-29; replaces "The path walks are separate loops". -Goals 2 and 11: `drawSteps` draws every path from its steps, compiled at link by +Goals 3 and 12: `drawSteps` draws every path from its steps, compiled at link by `pathCheck` or, for a caller's path, by `probePath` into a stack buffer. The check and the probe stay two loops: on 2026-09-29 one loop over all three walks, switching on a mode field, leaked the check's leaves and errors with the draw's pins, arm and memo, diff --git a/docs/research/research-sources-se.md b/docs/research/research-sources-se.md index 1157dbf..a2d1948 100644 --- a/docs/research/research-sources-se.md +++ b/docs/research/research-sources-se.md @@ -90,7 +90,7 @@ Recommendation: embed SCB 2022 xlsx-derived lists (top-N per sheet with counts) ## 3. Phone - Source: PTS "The Swedish numbering plan for telephony according to ITU-T E.164", 2024-01-08 (verified, text extracted): https://pts.se/globalassets/globala-block/nummertillstand/the-swedish-numbering-plan-for-telephony-according-to-itu---2024-01-08.pdf. Swedish "nrplansammanstallning-2026-02-04.pdf" and https://nummer.pts.se/NbrPlanSearch are behind a Radware bot check (both curl and Playwright blocked). No licence stated by PTS anywhere found; pts.se says contact pts@pts.se for data. -- Reserved for fiction, PTS "Telefonnummer till böcker och filmer" (verified 2026-09-21; curl and Playwright are Radware-blocked, a rendering fetcher gets through): https://www.pts.se/internet-och-telefoni/telefonnummer-och-adressering/telefonnummer-till-bocker-och-filmer/ — mobile 070-1740605–1740699, geographic 08-46500400–46500499 (Stockholm), 031-3900600–3900699 (Göteborg), 040-6280400–6280499 (Malmö), 0980-319200–319299 (Kiruna). 495 numbers PTS holds back so they reach nobody, which is goal 14's never-reaching path for `sv_SE.phone`. Same licence status as the numbering plan: none stated, facts only, cite PTS. The mobile block fits the shipped `{a} {b} {c}` grouping (`174 06 05`–`99`), as do 031 and 040; Stockholm's 8 subscriber digits and Kiruna's 6 need a format of their own, and 0980 is not among the 20 shipped area codes. +- Reserved for fiction, PTS "Telefonnummer till böcker och filmer" (verified 2026-09-21; curl and Playwright are Radware-blocked, a rendering fetcher gets through): https://www.pts.se/internet-och-telefoni/telefonnummer-och-adressering/telefonnummer-till-bocker-och-filmer/ — mobile 070-1740605–1740699, geographic 08-46500400–46500499 (Stockholm), 031-3900600–3900699 (Göteborg), 040-6280400–6280499 (Malmö), 0980-319200–319299 (Kiruna). 495 numbers PTS holds back so they reach nobody, which is goal 15's never-reaching path for `sv_SE.phone`. Same licence status as the numbering plan: none stated, facts only, cite PTS. The mobile block fits the shipped `{a} {b} {c}` grouping (`174 06 05`–`99`), as do 031 and 040; Stockholm's 8 subscriber digits and Kiruna's 6 need a format of their own, and 0980 is not among the 20 shipped area codes. - 264 geographic area codes (PDF text yields 263 "Area code for" rows; one likely split across a page break — flag). Area code = trunk "0" + NDC of 1–3 digits (08 Stockholm, 031 Göteborg, 040 Malmö, 0480 Kalmar). Geographic N(S)N (NDC + subscriber) max 9 digits, min 7 (2-digit NDC) or 8 (3-digit NDC); Stockholm min 7. Each row in the PDF gives NDC, max/min length, "Area code for ", so a `{ndc, name}` list can be built from it. - Mobile: NDC 70, 72, 73, 76, 79 — N(S)N exactly 9 digits, i.e. 07X-XXX XX XX (10 digits with trunk 0). Others: 71 mobile broadband/M2M (13 digits), 74 paging, 75 personal numbering, 77 shared cost, 20 freephone, 10 location-independent (10 AXX XX XX, A=1–8), 378 M2M fixed (10 digits). - Formatting (Språkrådet/Isof frågelådan, via search excerpts): hyphen after area code, subscriber digits grouped 2–3 from the left: `08-668 01 50`, `031-123 45 67`, `0480-123 45`, `070-123 45 67` (also `0701-23 45 67` accepted). International: `+46 70 123 45 67` (drop trunk 0). diff --git a/todo.md b/todo.md index 4a46c2f..a0a2590 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 73` +`Next ID = 71` | Goal | W | |---|---| @@ -15,244 +15,116 @@ | 2 | 0.95 | | 2.1 | 0.95 | | 2.2 | 0.95 | -| 2.3 | 0.95 | -| 2.4 | 0.95 | | 3 | 0.90 | | 3.1 | 0.90 | +| 3.2 | 0.90 | +| 3.3 | 0.90 | +| 3.4 | 0.90 | | 4 | 0.85 | | 4.1 | 0.85 | -| 4.2 | 0.85 | -| 4.3 | 0.85 | -| 4.4 | 0.85 | -| 4.5 | 0.85 | -| 4.6 | 0.85 | -| 4.7 | 0.85 | -| 4.8 | 0.85 | | 5 | 0.80 | +| 5.1 | 0.80 | +| 5.2 | 0.80 | +| 5.3 | 0.80 | +| 5.4 | 0.80 | +| 5.5 | 0.80 | +| 5.6 | 0.80 | +| 5.7 | 0.80 | +| 5.8 | 0.80 | | 6 | 0.75 | -| 6.1 | 0.75 | -| 6.2 | 0.75 | | 7 | 0.70 | +| 7.1 | 0.70 | +| 7.2 | 0.70 | | 8 | 0.65 | | 9 | 0.60 | | 10 | 0.55 | -| 10.1 | 0.55 | -| 10.2 | 0.55 | -| 10.3 | 0.55 | | 11 | 0.50 | +| 11.1 | 0.50 | +| 11.2 | 0.50 | +| 11.3 | 0.50 | | 12 | 0.45 | -| 12.1 | 0.45 | | 13 | 0.40 | +| 13.1 | 0.40 | | 14 | 0.35 | +| 15 | 0.30 | ## Items | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 4.3, 3.1 | 26.7 | -| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 3.1 | 25.9 | -| 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 4.2, 3.1 | 25.0 | -| 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 3 | 23.9 | -| 59 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | -| 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 4.4 | 22.4 | -| 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 3 | 21.9 | -| 35 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 3 | 21.6 | -| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 3 | 21.4 | -| 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 2 | 21.3 | -| 37 | 0.1.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 5 | 20.3 | -| 60 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 6 | 29.2 | -| 6 | 0.2.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 2 | 19.9 | -| 42 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 4.7 | 19.9 | -| 54 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 5 | 19.7 | -| 28 | 0.2.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 3.1 | 19.6 | -| 24 | 0.2.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 3 | 19.4 | -| 57 | 0.2.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 10 | 18.2 | -| 55 | 0.2.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 3 | 17.6 | -| 18 | 0.2.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 4 | 17.2 | -| 15 | 0.2.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 7 | 13, 12 | 17.1 | -| 16 | 0.2.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 7 | 13 | 16.6 | -| 32 | 0.2.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 4.8 | 16.4 | -| 45 | 0.2.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 5 | 16.4 | -| 20 | 0.2.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 3 | 16.3 | -| 52 | 0.2.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 5 | 16.1 | -| 56 | 0.2.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | -| 17 | 0.2.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 4 | 15.7 | -| 49 | 0.2.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 4 | 15.7 | -| 22 | 0.2.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 14 | 15.6 | -| 29 | 0.2.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 4 | 15.2 | -| 43 | 0.2.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 5 | 15.1 | -| 33 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 3.1 | 15.1 | -| 47 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 5, 8 | 14.7 | -| 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 4.7 | 14.7 | -| 14 | 0.2.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 4.7 | 14.4 | -| 39 | 0.2.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 5 | 14.4 | -| 46 | 0.2.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 5 | 14.4 | -| 51 | 0.2.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 5 | 14.4 | -| 13 | 0.2.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 3 | 14.3 | -| 40 | 0.2.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 5 | 14.1 | -| 53 | 0.2.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 5 | 14.1 | -| 10 | 0.3.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 2.2 | 13.6 | -| 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 3 | 13.5 | -| 36 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 7 | 13.5 | -| 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 10 | 13.4 | -| 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 14 | 12.4 | -| 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | -| 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 2.1 | 12.0 | -| 31 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | -| 41 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 4.7 | 12.0 | -| 34 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 4 | 11.2 | -| 48 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 5 | 10.8 | -| 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 12.1 | 10.4 | -| 7 | 0.3.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 2.2 | 10.4 | -| 38 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | -| 44 | 0.3.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 2.3 | 10.2 | -| 30 | 0.3.0 | | **Read `email.local`'s and `username`'s handles from the shipped name tables.** | 3 | 4 | 4 | 6 | 12 | 9.4 | -| 50 | 0.3.0 | | **Print `--help`'s repeat range from `fejkdata.MaxRepeat`.** | 1 | 1 | 2 | 6 | 10.3 | 9.4 | -| 58 | 0.3.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | -| 65 | 0.4.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 3.1 | 19.9 | -| 64 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 3 | 15.8 | -| 63 | 0.4.0 | | **Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees.** | 4 | 9 | 8 | 7 | 13 | 15.4 | -| 61 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 10 | 14.2 | -| 67 | 0.4.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 4.8 | 13.8 | -| 62 | 0.4.0 | | **Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`.** | 5 | 8 | 7 | 6 | 13, 11 | 13.0 | -| 68 | 0.4.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 2.4 | 12.2 | -| 69 | 0.4.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 2.4 | 11.9 | -| 66 | 0.4.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 2.4 | 11.4 | -| 70 | 0.4.0 | decision | **Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub.** | 2 | 1 | 1 | 3 | 1.1 | 5.8 | -| 71 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 30.2 | -| 72 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.5 | +| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | +| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | +| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 5.3, 4.1 | 25.8 | +| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | +| 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | +| 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 4 | 23.1 | +| 57 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | +| 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 5.4 | 21.6 | +| 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 4 | 21.1 | +| 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | +| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | +| 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | +| 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 3 | 19.1 | +| 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | +| 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | +| 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | +| 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | +| 28 | 0.2.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 4.1 | 18.9 | +| 24 | 0.2.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 4 | 18.6 | +| 55 | 0.2.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 11 | 17.5 | +| 53 | 0.2.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 4 | 16.9 | +| 18 | 0.2.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 5 | 16.6 | +| 44 | 0.2.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 6 | 15.8 | +| 54 | 0.2.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | +| 31 | 0.2.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 5.8 | 15.7 | +| 20 | 0.2.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 4 | 15.7 | +| 50 | 0.2.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 6 | 15.5 | +| 17 | 0.2.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 5 | 15.1 | +| 48 | 0.2.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 5 | 15.1 | +| 22 | 0.2.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 15 | 14.8 | +| 29 | 0.2.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 5 | 14.6 | +| 42 | 0.2.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 6 | 14.5 | +| 32 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 4.1 | 14.4 | +| 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | +| 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | +| 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | +| 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | +| 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | +| 49 | 0.3.0 | | **Report a CRLF rows file holding only its header as having no rows.** | 1 | 1 | 3 | 6 | 6 | 13.8 | +| 13 | 0.3.0 | defect | **Place xlsx cells by their `r` reference in `data-import/xlsx.py`.** | 2 | 2 | 3 | 6 | 4 | 13.7 | +| 39 | 0.3.0 | | **Stop the `columnKinds` suggestion naming a field's own kind or a narrower one.** | 1 | 2 | 3 | 6 | 6 | 13.5 | +| 51 | 0.3.0 | | **Spell a table one way across the errors that name it.** | 1 | 2 | 3 | 6 | 6 | 13.5 | +| 10 | 0.3.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 3.2 | 13.1 | +| 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 4 | 13.0 | +| 35 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 8 | 13.0 | +| 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 11 | 12.8 | +| 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 15 | 11.7 | +| 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | +| 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | +| 30 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | +| 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | +| 33 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 5 | 10.8 | +| 47 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 6 | 10.2 | +| 7 | 0.3.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 3.2 | 9.8 | +| 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 13.1 | 9.8 | +| 37 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | +| 43 | 0.3.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | +| 56 | 0.3.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | +| 61 | 0.4.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | +| 63 | 0.4.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 4.1 | 19.1 | +| 62 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 4 | 15.2 | +| 59 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 11 | 13.8 | +| 65 | 0.4.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 5.8 | 13.2 | +| 60 | 0.4.0 | | **Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`.** | 5 | 8 | 7 | 6 | 14, 12 | 12.4 | +| 66 | 0.4.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 3.4 | 11.8 | +| 67 | 0.4.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 3.4 | 11.3 | +| 64 | 0.4.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 3.4 | 10.8 | +| 68 | 0.4.0 | decision | **Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub.** | 2 | 1 | 1 | 3 | 1.1 | 5.8 | +| 69 | 1.0.0 | | **Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute.** | 1 | 3 | 7 | 9 | 1.1 | 30.2 | +| 70 | 1.0.0 | | **Publish a homepage with an in-browser generator, the library compiled to WebAssembly.** | 3 | 6 | 6 | 6 | 1.1 | 19.5 | ## Details -### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. - -- A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. -- A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read. -- A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line. -- Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace. - -### 3. Rewrite the shipped categories so facts that belong together come from one named pick. - -- `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. -- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. -- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. -- Re-pin seeded output in its own commit. - -### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. - -- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. -- Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. -- Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. -- Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. -- Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. - -### 25. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. - -`{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. - -### 59. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. - -Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. - -### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. - -- `..` goes up one level, to the parent table, and the name after it is that parent's: `{c..country.name}`, and from a locality `{l..municipality..region.name}`. Naming any other table is a load error naming the parent. -- It steps up from a selected row, a drawn row or a name bound to a row. -- After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. -- `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. -- A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. - -### 26. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. - -The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. - -### 35. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. - -`isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. - -### 12. Strip the whitespace `data-import/territory.py` copies from its register. - -`misc.territory[CW].capital` renders ` Willemstad`, with a leading space. - -### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. - -Needs item 4, and runs as item 6 decides. - -### 37. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. - -`New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. - -### 60. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. - -GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. - -### 6. Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4. - -Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: - -| Seat | Run 1 | Run 2 | -|---|---|---| -| Junior A | 5 | 5 | -| Junior B | 4 | 5 | -| Mid A | 6 | 5 | -| Mid B | 6 | 6 | -| Senior domain | 6 | 5 | -| Senior maintainability | 6 | 6 | -| Senior extender | 6 | 5 | -| Architect greenfield | 6 | 6 | -| Architect inherited | 6 | 6 | -| Mean (Navigation / Locality / Shape / Self-sufficiency) | 5.7 (7.0 / 5.1 / 5.9 / 5.7) | 5.4 (6.3 / 4.8 / 5.8 / 5.3) | - -Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Run 1 is the recorded score; run 2 reran the same code to measure noise. A spread of 0.3 is close to the whole 5.7–6.1 range the recorded scores span since 2026-09-20, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. Until this item is decided, that rule compares against 5.7, from 2026-09-30. - -### 42. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. - -`geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. Steps: -- Have a selector that misses name the key of every row whose cell spells it, matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, and `misc.loglevel[Error]` `[err]`. -- Delete the `name` option and the load checks it carries, and have `New` refuse a table still carrying `name`, naming `key`. -- Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on the integer `data-import/geo-se.py` assigns as the decision states, linking `street` and `postal-code` by it. -- Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. -- Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`. - -### 54. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. - -With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. - -### 28. Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick. - -Needs item 4. - -### 24. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. - -ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it. - -### 55. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. - -Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. - -### 15. Add the remaining `misc` tables and templates, one row of its detail's table per chunk. - -Shape: T = table, t = template, c = choice. - -| Category | Shape | Source | Licence | -|---|---|---|---| -| `ip` v4 documentation ranges, private, CIDR; `ipv6` `2001:db8::/32`; `mac` locally administered | t | RFC 5737, 1918, 9637 | facts | -| `creditcard` per network: IIN, length, CVV, expiry | T+t | network rules, `{luhn()}` | facts | -| `bic`, `isin`, `cusip` | t | structure rules | facts | -| `ean`, `upc`, `gtin`, `isbn`, `issn`, `imei`, `asin`, `sku` | t | check-digit rules, GS1 demo prefix 952 | facts | -| `product` name, adjective, material, department, `garmentsize` | T/c | LLM-written lists | — | -| `airport` IATA, ICAO, territory, municipality; `airline`; `aircraft`; `flight`, `seat`, `pnr` | T+t | OurAirports, Wikidata | public domain, CC0 | -| `element`, `unit`, `planet` | T | PubChem, BIPM, NASA | public domain, CC BY 4.0 | -| `animal` by class, `plant` | T | Wikidata | CC0 | -| `bloodtype` weighted (global) | c | AABB | facts | -| `color` named, hex, rgb, hsl, cmyk | T+t | CSS Color 4, XKCD | W3C, CC0 | -| `programminglanguage`, `os`, `browser`, `database` column/type/engine, `fileext` | T/c | Linguist, curated | MIT | -| `commit` message, branch, `sha` | t | LLM-written, `{hex(40)}` | — | -| `password`, `hash`, `jwt`, `slug`, `hostname`, `avatar`, `placeholderimage` | t | shapes | — | -| `book` title, author; `genre`; `instrument`; `dish`, `ingredient` | T | Gutenberg catalog, MusicBrainz, USDA | CC0 | -| `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | -| `direction`, `continent`, `ulid` | c/t | — | — | - ### 16. Add the remaining locale categories: company, phone, finance, vehicle, words. Shape: T = table, t = template, c = choice. @@ -287,31 +159,160 @@ Shape: T = table, t = template, c = choice. | `word`, `sentence`, `paragraph` | T+t | Moby POS or WordNet | public domain / WordNet | | month and weekday names, `holiday` | t+T | CLDR en | Unicode | -### 32. Rename `misc.territory.country` to `sovereign`. +### 15. Add the remaining `misc` tables and templates, one row of its detail's table per chunk. -It holds a code, and `country` collides with the `misc.country` path it replaced. +Shape: T = table, t = template, c = choice. -### 45. Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`. +| Category | Shape | Source | Licence | +|---|---|---|---| +| `ip` v4 documentation ranges, private, CIDR; `ipv6` `2001:db8::/32`; `mac` locally administered | t | RFC 5737, 1918, 9637 | facts | +| `creditcard` per network: IIN, length, CVV, expiry | T+t | network rules, `{luhn()}` | facts | +| `bic`, `isin`, `cusip` | t | structure rules | facts | +| `ean`, `upc`, `gtin`, `isbn`, `issn`, `imei`, `asin`, `sku` | t | check-digit rules, GS1 demo prefix 952 | facts | +| `product` name, adjective, material, department, `garmentsize` | T/c | LLM-written lists | — | +| `airport` IATA, ICAO, territory, municipality; `airline`; `aircraft`; `flight`, `seat`, `pnr` | T+t | OurAirports, Wikidata | public domain, CC0 | +| `element`, `unit`, `planet` | T | PubChem, BIPM, NASA | public domain, CC BY 4.0 | +| `animal` by class, `plant` | T | Wikidata | CC0 | +| `bloodtype` weighted (global) | c | AABB | facts | +| `color` named, hex, rgb, hsl, cmyk | T+t | CSS Color 4, XKCD | W3C, CC0 | +| `programminglanguage`, `os`, `browser`, `database` column/type/engine, `fileext` | T/c | Linguist, curated | MIT | +| `commit` message, branch, `sha` | t | LLM-written, `{hex(40)}` | — | +| `password`, `hash`, `jwt`, `slug`, `hostname`, `avatar`, `placeholderimage` | t | shapes | — | +| `book` title, author; `genre`; `instrument`; `dish`, `ingredient` | T | Gutenberg catalog, MusicBrainz, USDA | CC0 | +| `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | +| `direction`, `continent`, `ulid` | c/t | — | — | + +### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. + +- A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. +- A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read. +- A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line. +- Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace. + +### 3. Rewrite the shipped categories so facts that belong together come from one named pick. + +- `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. +- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. +- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. +- Re-pin seeded output in its own commit. + +### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. + +- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. +- Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. +- Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. +- Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. + +### 25. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. + +`{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. + +### 57. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. + +Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. + +### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. + +- `..` goes up one level, to the parent table, and the name after it is that parent's: `{c..country.name}`, and from a locality `{l..municipality..region.name}`. Naming any other table is a load error naming the parent. +- It steps up from a selected row, a drawn row or a name bound to a row. +- After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. +- `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. +- A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. + +### 26. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. + +The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. + +### 34. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. + +`isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. + +### 12. Strip the whitespace `data-import/territory.py` copies from its register. + +`misc.territory[CW].capital` renders ` Willemstad`, with a leading space. + +### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. + +Needs item 4, and runs as item 6 decides. + +### 6. Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4. + +Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: + +| Seat | Run 1 | Run 2 | +|---|---|---| +| Junior A | 5 | 5 | +| Junior B | 4 | 5 | +| Mid A | 6 | 5 | +| Mid B | 6 | 6 | +| Senior domain | 6 | 5 | +| Senior maintainability | 6 | 6 | +| Senior extender | 6 | 5 | +| Architect greenfield | 6 | 6 | +| Architect inherited | 6 | 6 | +| Mean (Navigation / Locality / Shape / Self-sufficiency) | 5.7 (7.0 / 5.1 / 5.9 / 5.7) | 5.4 (6.3 / 4.8 / 5.8 / 5.3) | + +Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Run 1 is the recorded score; run 2 reran the same code to measure noise. A spread of 0.3 is close to the whole 5.7–6.1 range the recorded scores span since 2026-09-20, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. Until this item is decided, that rule compares against 5.7, from 2026-09-30. + +### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. + +GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. + +### 36. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. + +`New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. + +### 41. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. + +`geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. Steps: +- Have a selector that misses name the key of every row whose cell spells it, matched without case, else say the table selects by its key: `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]` its key, and `misc.loglevel[Error]` `[err]`. +- Delete the `name` option and the load checks it carries, and have `New` refuse a table still carrying `name`, naming `key`. +- Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on the integer `data-import/geo-se.py` assigns as the decision states, linking `street` and `postal-code` by it. +- Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`. +- Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`. + +### 52. Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`. + +With several `--data-path` layers the author has to work out which directory won, while a parse error on the same file names it. + +### 28. Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick. + +Needs item 4. + +### 24. Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold. + +ISO 3166 reserves it for Western Sahara and the root zone has never been delegated it, so no resolver answers for it. + +### 53. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. + +Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text everywhere else. Typing a shipped column changes what `json` and `sql` write, so the capability is a minor and applying it to `misc.httpmethod` is a major. + +### 44. Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`. The decision "`misc.tld` keys carry the leading dot, where other tables key on a bare code" leaves `misc.tld[se]` missing today. +### 54. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. + +GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. + +### 31. Rename `misc.territory.country` to `sovereign`. + +It holds a code, and `country` collides with the `misc.country` path it replaced. + ### 20. Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are. Neither names a `weight`, so a bare `sex` draws evenly. -### 52. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. +### 50. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. -### 56. Read the tag through `git/ref/tags/{tag}` in `publish_release.py`. - -GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a release lands on a tag already pointing at another commit. - ### 17. Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape. `misc.uuid` gains `variant`, for one. -### 49. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. +### 48. Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states. `probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. @@ -323,11 +324,11 @@ GitHub answers `tags/{tag}` with 404 whatever exists, so the burnt-version guard Require the path step to reach a sibling category. -### 43. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. +### 42. Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises. `datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. -### 47. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. +### 46. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. `readTableOptions` and `loadDir` return on the first in Go's map order. @@ -335,15 +336,15 @@ Require the path step to reach a sibling category. Both render the same run. -### 39. Compare `calcParser.binary`'s operator as a rune. +### 38. Compare `calcParser.binary`'s operator as a rune. `byte(p.rs[p.pos])` reads U+012B `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`. -### 46. Name the node a selector follows in `stepInto`'s refusal. +### 45. Name the node a selector follows in `stepInto`'s refusal. `sv_SE.person[1].first` answers `1 is not a table`. -### 51. Report a CRLF rows file holding only its header as having no rows. +### 49. Report a CRLF rows file holding only its header as having no rows. `parseRows` answers "line 2 is empty". @@ -351,11 +352,11 @@ Both render the same run. A sheet omitting an empty cell shifts every later column left. -### 40. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. +### 39. Stop the `columnKinds` suggestion naming a field's own kind or a narrower one. An `int64` or `float64` field unproven in range is told to become itself, and a `uint64` one to become `int64`. -### 53. Spell a table one way across the errors that name it. +### 51. Spell a table one way across the errors that name it. `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one table is `territory` and `misc.territory`, and the short spelling names no file where two folders hold that name. @@ -379,11 +380,11 @@ The greenfield architect found the fence pipeline last. Needs item 4, which shri The inherited architect's 3am trace crossed four files for one draw. -### 41. Parse `defaultTable`'s path with the library's grammar. +### 40. Parse `defaultTable`'s path with the library's grammar. The CLI counts brackets of its own, so a change to the selector grammar desyncs the default `--table` silently. -### 48. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. +### 47. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. @@ -391,66 +392,58 @@ The CLI counts brackets of its own, so a change to the selector grammar desyncs `splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. Needs item 4. -### 38. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. +### 37. Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`. Both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s local `rng` shadows the `rng` interface. -### 44. Call `templateError` a compile failure in its doc. +### 43. Call `templateError` a compile failure in its doc. Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. -### 30. Read `email.local`'s and `username`'s handles from the shipped name tables. - -Keep the handle shape: `username` shortens a surname to `ahl` or `sjo`, and the US one draws unisex given names, neither of which falls out of the tables unaided. - -### 50. Print `--help`'s repeat range from `fejkdata.MaxRepeat`. - -The usage spells `1..1048576` a second time. - -### 58. Build on a manual run of `test.yml`. +### 56. Build on a manual run of `test.yml`. `workflow_dispatch` leaves `github.event.before` empty, so the diff compares `HEAD` with itself and skips `docker build`. -### 65. Pair a street with its exact postnummer. +### 61. Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country. + +Pick the ten from a published ranking of languages by total speakers, such as Ethnologue's, each in the country where it has the most speakers, and record the ranking in the decision that names them. The Nordic ones are sv_SE (shipped), nb_NO, da_DK, fi_FI and is_IS. Each locale carries goal 2.2's categories, and its `address` reads its country's `geo/` tree by reference. + +### 63. Pair a street with its exact postnummer. Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. -### 64. Add `{btc()}` and `{eth()}`. +### 62. Add `{btc()}` and `{eth()}`. They need sha256 and keccak for Base58Check and EIP-55. -### 63. Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, and their `geo/` trees. - -Each locale carries person, address, phone, national id, company and date names. Their `address` reads the `geo/` trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE by reference; AU and CA get a tree with no locale. - -### 61. Promise in `Generator`'s godoc that its renders run one at a time. +### 59. Promise in `Generator`'s godoc that its renders run one at a time. A caller wanting parallel throughput then makes one generator per goroutine. -### 67. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. +### 65. Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI. The check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -### 62. Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`. +### 60. Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`. Every US place of 10,000 and more streets per locality, built by `--min-population` and `--streets-per-locality`. The default embed stays under ~1 MB per country and `New` under ~50 ms. -### 68. Cut the README's Layout block to the lines that say what a file name cannot. +### 66. Cut the README's Layout block to the lines that say what a file name cannot. `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -### 69. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. +### 67. Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses. The `gh` rule opens by restating the decision "GitHub is canonical, and the module path names it", a second copy that drifts. The comprehension rule packs the gate, its exceptions, the round and the at-or-above-7.0 regime into one line, and uses "depth 1", "nine-seat", "four-seat" and "answers it in one run" without saying what they mean. -### 66. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. +### 64. Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on. Propose a goal for an entry that serves none. -### 70. Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub. +### 68. Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub. Gitea converts a repository to a mirror only by re-creating it, so until that runs the copy there is a second owner of one history. -### 71. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. +### 69. Announce v1.0.0 where a developer choosing a fake-data tool already reads, with a README first screen for someone deciding in a minute. The human cuts v1.0.0 once the shipped data is in its record shape and one full minor has shipped with no breaking change, per the README's Versioning table; pairing a street with its exact postnummer rewrites shipped rows, so a minor with no breaking change follows it first. -- 2.52.0 From b7679a58032d1bed52025ae984db1ea23e41af4c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:26:57 +0200 Subject: [PATCH 426/966] Ship the Nordic and most-spoken locales, and the remaining misc and locale categories, in v0.1.0 --- todo.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/todo.md b/todo.md index a0a2590..451cb0a 100644 --- a/todo.md +++ b/todo.md @@ -52,6 +52,7 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| +| 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | | 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 5.3, 4.1 | 25.8 | @@ -110,7 +111,6 @@ | 37 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | | 43 | 0.3.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | | 56 | 0.3.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | -| 61 | 0.4.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | | 63 | 0.4.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 4.1 | 19.1 | | 62 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 4 | 15.2 | | 59 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 11 | 13.8 | @@ -125,6 +125,10 @@ ## Details +### 61. Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country. + +Pick the ten from a published ranking of languages by total speakers, such as Ethnologue's, each in the country where it has the most speakers, and record the ranking in the decision that names them. The Nordic ones are sv_SE (shipped), nb_NO, da_DK, fi_FI and is_IS. Each locale carries goal 2.2's categories, and its `address` reads its country's `geo/` tree by reference. + ### 16. Add the remaining locale categories: company, phone, finance, vehicle, words. Shape: T = table, t = template, c = choice. @@ -404,10 +408,6 @@ Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTempl `workflow_dispatch` leaves `github.event.before` empty, so the diff compares `HEAD` with itself and skips `docker build`. -### 61. Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country. - -Pick the ten from a published ranking of languages by total speakers, such as Ethnologue's, each in the country where it has the most speakers, and record the ranking in the decision that names them. The Nordic ones are sv_SE (shipped), nb_NO, da_DK, fi_FI and is_IS. Each locale carries goal 2.2's categories, and its `address` reads its country's `geo/` tree by reference. - ### 63. Pair a street with its exact postnummer. Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. -- 2.52.0 From 30358eb7ff1a1e80a34548fca1cb66dfd5373ada Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:39:59 +0200 Subject: [PATCH 427/966] Test binding a pick to a name with {x as n} --- name_test.go | 212 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 name_test.go diff --git a/name_test.go b/name_test.go new file mode 100644 index 0000000..5107422 --- /dev/null +++ b/name_test.go @@ -0,0 +1,212 @@ +package fejkdata + +import ( + "strings" + "testing" +) + +// nameTables is a region table linked to by a municipality table. +func nameTables() map[string]string { + return map[string]string{ + "region.json": `{"format":"{name}","rows":"region.tsv","key":"code"}`, + "region.tsv": "code\tname\n01\tStockholms län\n12\tSkåne län\n", + "municipality.json": `{"format":"{name}","rows":"municipality.tsv","key":"code","parent":"region"}`, + "municipality.tsv": "code\tname\tregion\n0180\tStockholm\t01\n0184\tSolna\t01\n1280\tMalmö\t12\n1281\tLund\t12\n", + } +} + +var municipalityCode = map[string]string{"Stockholm": "0180", "Solna": "0184", "Malmö": "1280", "Lund": "1281"} + +var municipalityRegion = map[string]string{"Stockholm": "Stockholms län", "Solna": "Stockholms län", "Malmö": "Skåne län", "Lund": "Skåne län"} + +func TestANameIsOnePickOfEverythingUnderIt(t *testing.T) { + files := nameTables() + files["card.json"] = `"{/region as r}{r.name}|{r.municipality.name}|{r.municipality.code}|{r}"` + f := newGenerator(t, writeFiles(t, files), WithSeed(3)) + for i := 0; i < 200; i++ { + got := strings.Split(fake(t, f, "card"), "|") + region, municipality, code, whole := got[0], got[1], got[2], got[3] + if municipalityCode[municipality] != code || municipalityRegion[municipality] != region || whole != region { + t.Fatalf("card = %q, want one region, and one municipality inside it", got) + } + } +} + +func TestANameReadWholeAgreesWithItsPaths(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first} {last}","first":["Ada","Bo","Cy"],"last":["Byron","Ek","Lind"]}`, + "card": `"{/person as p}{p}|{p.last}|{p.first}"`, + }) + f := newGenerator(t, dir, WithSeed(5)) + for i := 0; i < 200; i++ { + got := strings.Split(fake(t, f, "card"), "|") + if got[0] != got[2]+" "+got[1] { + t.Fatalf("card = %q, want the whole person to match its parts", got) + } + } +} + +func TestANameDrawsEachFieldUnderItOnce(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `{"format":"{w}-{w}","w":["a","b","c","d","e","f","g","h"]}`, + "card": `"{/word as w}{w}"`, + }) + f := newGenerator(t, dir, WithSeed(7)) + for i := 0; i < 100; i++ { + if got := fake(t, f, "card"); got[0] != got[2] { + t.Fatalf("card = %q, want one draw of w under the name", got) + } + } +} + +func TestABindingPrintsNothing(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `["a","b"]`, + "card": `"<{/word as w}>{w}"`, + }) + f := newGenerator(t, dir, WithSeed(1)) + if got := fake(t, f, "card"); got != "<>a" && got != "<>b" { + t.Fatalf("card = %q, want the binding to print nothing", got) + } +} + +func TestSiblingFieldsAndColumnsReadOneName(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first} {last}","first":["Ada","Bo","Cy"],"last":["Byron","Ek","Lind"]}`, + "row": `{"format":"{first} {last}","first":"{/person as p}{p.first}","last":"{p.last}","whole":"{p}"}`, + }) + f := newGenerator(t, dir, WithSeed(9)) + for i := 0; i < 100; i++ { + r, err := f.FakeRecord("row") + if err != nil { + t.Fatal(err) + } + c := r.Columns() + if c[2].Value != c[0].Value+" "+c[1].Value { + t.Fatalf("columns = %v, want one person across them", c) + } + if got := strings.Fields(fake(t, f, "row")); len(got) != 2 { + t.Fatalf("row = %q, want a first and a last name", got) + } + } +} + +func TestAFieldReadsTheNameItsCategoryBinds(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first} {last}","first":"Ada","last":"Byron"}`, + "row": `{"format":"{first}","first":"{/person as p}{p.first}","last":"{p.last}"}`, + }) + f := newGenerator(t, dir, WithSeed(1)) + if got := fake(t, f, "row.last"); got != "Byron" { + t.Fatalf("row.last = %q, want Byron", got) + } +} + +func TestANameInsideARepeatPicksEachIteration(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `["a","b","c","d","e","f","g","h"]`, + "line": `{"format":"{each}","each":{"format":"{/word as w}{w}{w}","repeat":40,"separator":" "}}`, + }) + f := newGenerator(t, dir, WithSeed(11)) + seen := map[string]bool{} + for _, pair := range strings.Fields(fake(t, f, "line")) { + if pair[0] != pair[1] { + t.Fatalf("pair %q, want one pick per iteration", pair) + } + seen[pair] = true + } + if len(seen) < 2 { + t.Fatalf("pairs %v, want each iteration to pick anew", seen) + } +} + +func TestANameOutsideARepeatKeepsItsPick(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `["a","b","c","d","e","f","g","h"]`, + "line": `{"format":"{/word as w}{each}","each":{"format":"{w}","repeat":40,"separator":" "}}`, + }) + f := newGenerator(t, dir, WithSeed(13)) + got := strings.Fields(fake(t, f, "line")) + for _, w := range got { + if w != got[0] { + t.Fatalf("line = %q, want one pick on every line", got) + } + } +} + +func TestTwoRendersPickTwice(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `["a","b","c","d","e","f","g","h"]`, + "one": `"{/word as w}{w}"`, + "two": `"{/one}{/one}{/one}{/one}{/one}{/one}"`, + }) + f := newGenerator(t, dir, WithSeed(17)) + got := fake(t, f, "two") + if strings.Count(got, got[:1]) == len(got) { + t.Fatalf("two = %q, want each bare reference to its own pick", got) + } +} + +func TestANameInAnInlineTemplate(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first} {last}","first":["Ada","Bo","Cy"],"last":["Byron","Ek","Lind"]}`, + }) + f := newGenerator(t, dir, WithSeed(19)) + for i := 0; i < 50; i++ { + got := strings.Split(fakeTemplate(t, f, "{/person as p}{p}|{p.first} {p.last}"), "|") + if got[0] != got[1] { + t.Fatalf("template = %q, want one person", got) + } + } +} + +func TestNameErrors(t *testing.T) { + cases := []struct { + name string + files map[string]string + want string + }{ + {"bound twice", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{y}","x":"{/word as w}{w}","y":"{/word as w}"}`}, + `name "w" is bound twice in one scope`}, + {"bound outside the repeat too", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":{"format":"{/word as w}{w}","repeat":2}}`}, + `name "w" is bound outside this repeat already`}, + {"a field too", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{w}","w":"x"}`}, + `name "w" is a field too`}, + {"an option", map[string]string{"word": `["a","b"]`, "card": `"{/word as format}{format}"`}, + `"format" is an option and can never be a name`}, + {"no reference", map[string]string{"card": `{"format":"{word as w}{w}","word":["a","b"]}`}, + `"word" is no reference`}, + {"read by nothing", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}"`}, + `nothing reads name "w"`}, + {"unbound", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":{"format":"{/word as w}{w}","repeat":2}}`}, + `no field "w"`}, + {"padded", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}{w}"`}, + `write {/word as w}`}, + {"selector after a name", map[string]string{"region.json": nameTables()["region.json"], "card": `"{/region as r}{r.municipality[0180].name}"`}, + `a path through name "r" selects no row`}, + {"in a table's format", map[string]string{"word": `["a","b"]`, "t.json": `{"format":"{/word as w}{w}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\nx\ny\n"}, + "a table binds no name"}, + {"in a field's name", map[string]string{"card": `{"format":"{a as b}","a as b":"x"}`}, + `contains " as "`}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + files := map[string]string{} + for name, body := range c.files { + if !strings.Contains(name, ".") { + name += ".json" + } + files[name] = body + } + if c.name == "selector after a name" { + for k, v := range nameTables() { + files[k] = v + } + } + _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, files))) + if err == nil || !strings.Contains(err.Error(), c.want) { + t.Fatalf("New() = %v, want an error holding %q", err, c.want) + } + }) + } +} -- 2.52.0 From ba9477f2e5607af9b0124134b103165ca4f55f5f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:51:14 +0200 Subject: [PATCH 428/966] Test a name bound from a folder, through a reference path, and across struct tags --- name_test.go | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/name_test.go b/name_test.go index 5107422..2ed818c 100644 --- a/name_test.go +++ b/name_test.go @@ -210,3 +210,51 @@ func TestNameErrors(t *testing.T) { }) } } + +func TestANameBindsAReferenceFromItsFolder(t *testing.T) { + dir := writeData(t, map[string]string{ + "sv/word": `["a","b","c","d","e","f","g","h"]`, + "sv/card": `"{.word as w}{w}{w}"`, + }) + f := newGenerator(t, dir, WithSeed(23)) + for i := 0; i < 50; i++ { + if got := fake(t, f, "sv.card"); got[0] != got[1] { + t.Fatalf("sv.card = %q, want one pick", got) + } + } +} + +func TestANameBindsAReferencePath(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first}","first":["Ada","Bo","Cy","Di"]}`, + "card": `"{/person.first as f}{f}|{f}"`, + }) + f := newGenerator(t, dir, WithSeed(29)) + for i := 0; i < 50; i++ { + got := strings.Split(fake(t, f, "card"), "|") + if got[0] != got[1] { + t.Fatalf("card = %q, want one pick", got) + } + } +} + +type namedPerson struct { + First string `fake:"{/person as p}{p.first}"` + Whole string `fake:"{p}"` +} + +func TestStructTagsReadOneName(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first} {last}","first":["Ada","Bo","Cy"],"last":["Byron","Ek","Lind"]}`, + }) + f := newGenerator(t, dir, WithSeed(31)) + for i := 0; i < 50; i++ { + var p namedPerson + if err := f.FakeStruct(&p); err != nil { + t.Fatal(err) + } + if !strings.HasPrefix(p.Whole, p.First+" ") { + t.Fatalf("struct = %+v, want one person", p) + } + } +} -- 2.52.0 From 23e426016decfa20b63cdc458ee5aab1c3476f5f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:51:14 +0200 Subject: [PATCH 429/966] Bind a pick to a name with {ref as n}, and read it with {n} and {n.path} --- AGENTS.md | 1 + CHANGELOG.md | 8 ++ README.md | 30 ++++- doc.go | 5 +- docs/decisions.md | 13 ++ draws.go | 16 ++- format.go | 114 ++++++++++++---- inline.go | 10 +- name.go | 331 ++++++++++++++++++++++++++++++++++++++++++++++ node.go | 35 +++-- path.go | 16 +++ record.go | 6 +- reference.go | 13 +- render.go | 39 ++++-- table.go | 23 +++- todo.md | 12 +- 16 files changed, 609 insertions(+), 63 deletions(-) create mode 100644 name.go diff --git a/AGENTS.md b/AGENTS.md index 88a90b3..6babe68 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -118,3 +118,4 @@ In [docs/decisions.md](docs/decisions.md): - The US given names come from a mirror of the SSA file - `List` advertises direct descents only - A path draws through its compiled steps +- A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read diff --git a/CHANGELOG.md b/CHANGELOG.md index 0dae6ef..230d153 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,14 @@ replacement, and each removed path, column or flag. ### Added +- A name keeps one pick: `{/misc.territory as t}` binds one row of `misc.territory` to `t` + and prints nothing, and `{t}`, `{t.capital}` and `{t.calling-code}` read that one row. + A name lives in the category binding it, so every field and record column of the + category reads one pick, and a name a `repeat` binds picks again on every iteration. + Refused at `New`: a name bound twice in one category, or inside a `repeat` and outside + it; a name that is a field or an option; a binding of anything but a reference; a + binding nothing reads; a path through a name that selects a row; and a binding in a + table. A field, column, category or folder name may not contain ` as `. - First release: the CLI, the library and the shipped data set. - Table categories: a category JSON naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and `parent`; a path selects a row by key or name, diff --git a/README.md b/README.md index 00b1082..6ca5188 100644 --- a/README.md +++ b/README.md @@ -359,7 +359,8 @@ Every character is literal except a `{…}` token: | Token | Renders | |-------|---------| -| `{name}` | the sibling field `name` | +| `{name}` | the sibling field `name`, or the pick bound to the name ([Names](#names)) | +| `{ref as name}` | nothing: binds one pick of the reference `ref` to `name` ([Names](#names)) | | `{name.field}` | `field` of one draw of `name` ([Correlated fields](#correlated-fields)) | | `{a\|b}` | one of the named fields, even odds | | `{fn(args)}` | a builtin ([Functions](#functions)) | @@ -574,7 +575,8 @@ or the reference draws it. `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options; **any other key is a field** (see [Decisions](docs/decisions.md#options-and-fields-share-one-namespace)), and `rows` makes a category -a [table](#table), so no template carries a field of that name. An object that does nothing a +a [table](#table), so no template carries a field of that name. A [name](#names) shares the +namespace too. An object that does nothing a string can't — only a `format` — is rejected naming the string, as is a one-item choice naming its item. @@ -696,6 +698,29 @@ unknown, names a folder, has no folder above, reads a field not every variant of a choice carries, or names the category the reference sits in, and a reference that leads back to its own value, directly, mutually or through a chain. +### Names + +`{ref as n}` binds one pick of a reference to the name `n` and prints nothing. `{n}` then +prints that pick, and `{n.path}` reads a path through it, so every read of `n` describes one +row: + +```json +"{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}" +``` + +Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn +once, so `{n}` and `{n.path}` agree, and a field `n` renders twice prints the same text twice. +A name is drawn on its first read, and lives in the category binding it: any field of the +category reads it, and a record's columns read one pick. Each render of the category picks +anew, and so does each bare reference to it. A name a `repeat` binds picks again on every +iteration, and one bound outside the repeat keeps its pick on every line. + +Rejected at load: a name bound twice in one category, or both inside a `repeat` and outside +it; a name that is a field or an option; a binding of anything but a reference; a binding +nothing reads; a path through a name that selects a row, which the reference selects instead; +and a binding in a table's format or cell. A field, column, category or folder name may not +contain ` as `. + ### Draw group A template may carry `drawGroup` to hold its reference draws apart: every reference path @@ -889,6 +914,7 @@ doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws +name.go names: {ref as n} bindings, the scopes they live in, and the picks a render keeps under them path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make record.go records: Record, the JSON/CSV/SQL serializers, and their entry points diff --git a/doc.go b/doc.go index 2f0fd96..7841405 100644 --- a/doc.go +++ b/doc.go @@ -27,7 +27,10 @@ package fejkdata // an expansion holds a sibling path's head and an operand's field, its draw // group a reference path's: `formatOps.held` and `heldCheck` an expansion's, // `readReference` a draw group's. -// - memo — what a hold or a draw group keeps its draws in, the variant drawn at +// - named pick — one draw of a reference a binding token, {ref as n}, gives a name, kept for one +// render of the scope it is bound in, a category or a repeat iteration, and drawn on its +// first read: `nameBinding`, `nameScope`, `namedPick`, `pickFrame`. +// - memo — what a hold, a draw group or a named pick keeps its draws in, the variant drawn at // each level and the value each path read: `drawMemo`, `readMemo`. // - draw group — reference draws kept apart inside one render: // `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a diff --git a/docs/decisions.md b/docs/decisions.md index 2eb6010..73658b5 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -888,3 +888,16 @@ since Go tracks a struct's fields as one, and a repeat of a reference path rose to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node, so no benchmark gained an allocation. Valid while Go's escape analysis tracks a struct's fields as one. + +## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read + +2026-10-03, larv-review. Goals 5.3 and 5.6. +`bindNames` scopes a name at compile, and a render keeps one `pickFrame` per scope. One +field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside +`"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category is the smallest unit holding every field and record column. The +scope is lexical: a category referencing another never sees its names, so what a category +renders never depends on who references it. A pick is drawn on its first read, so a record's +columns, rendered in name order, read one pick whichever binds it. A read entering a category +past its root, `Fake("cat.field")` or `{/cat.field}`, keeps the category's frame in the memo of +that read: per render and group, as the read's own draw is. Valid while a name is read only +inside the category binding it. diff --git a/draws.go b/draws.go index a7f5894..f49728b 100644 --- a/draws.go +++ b/draws.go @@ -2,12 +2,14 @@ package fejkdata import "strings" -// drawMemo is what a hold or a draw group has drawn: the variant each level was drawn as, so -// every path under it reads one variant; and the value each read produced, by its path, so the -// same read written twice reads one value. +// drawMemo is what a hold, a draw group or a named pick has drawn: the variant each level was +// drawn as, so every path under it reads one variant; the value each read produced, by its path, +// so the same read written twice reads one value; and the frame of each name scope a read +// entered past its owner. type drawMemo struct { variant map[string]node value map[string]readValue + frames map[*nameScope]*pickFrame } // groupDraws is what one render's draw group has drawn for its reference paths: a memo, and the @@ -27,11 +29,12 @@ type readValue struct { type renderTrace func(group string, row renderedRow, a arm) // renderDraws is one render's reference draws: the unnamed draw group's, and each named one's; -// and a test's trace of its reads. +// a test's trace of its reads; and the frames of the name scopes rendering. type renderDraws struct { unnamed groupDraws named map[string]*groupDraws trace renderTrace + frames *frameStack } // renderScope is where a render reads its reference paths: its reference draws, in the draw group of the @@ -42,6 +45,11 @@ type renderScope struct { draws *renderDraws group string row renderedRow + entry *drawMemo // the memo of the read that entered what renders + // pick is the named pick what renders is part of, at pickKey, its path from the name; nil + // outside one. + pick *namedPick + pickKey string } type renderedRow struct { diff --git a/format.go b/format.go index e962806..185be82 100644 --- a/format.go +++ b/format.go @@ -1,6 +1,7 @@ package fejkdata import ( + "errors" "fmt" "strings" ) @@ -63,7 +64,8 @@ type formatToken struct { body string // the braces' content, as written fn string // builtinCall args []string // builtinCall - names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads + names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads; nameBind: the reference it binds + bound string // nameBind: the name } type tokenKind uint8 @@ -72,8 +74,12 @@ const ( builtinCall tokenKind = iota + 1 nameRead literalRun + nameBind ) +// asWord separates a binding's reference from the name it binds: {/person as p}. +const asWord = " as " + // parseFormat is the one reading of a format's tokens: a '(' outside a selector makes // a token a call. func parseFormat(format string) ([]formatToken, error) { @@ -91,6 +97,10 @@ func parseFormat(format string) ([]formatToken, error) { return nil } flush() + if ref, name, binds := cutOutside(u.body, asWord); binds && indexOutside(u.body, '(') < 0 { + toks = append(toks, formatToken{kind: nameBind, body: u.body, names: []string{ref}, bound: name}) + return nil + } if indexOutside(u.body, '(') < 0 { toks = append(toks, formatToken{kind: nameRead, body: u.body, names: splitOutside(u.body, '|')}) return nil @@ -195,51 +205,103 @@ func checkFunc(tok formatToken, fields map[string]node) error { return nil } -func parseChecked(format string, fields map[string]node) ([]formatToken, error) { +func parseChecked(format string, fields map[string]node) ([]formatToken, []unboundRead, error) { toks, err := parseFormat(format) if err != nil { - return nil, err + return nil, nil, err } - return toks, checkTokens(toks, fields) + unbound, err := checkTokens(toks, fields) + return toks, unbound, err } +// unboundRead is a token reading a head no field holds, which only a name bound around the +// template can answer; err is the refusal where none does. +type unboundRead struct { + head string + err error +} + +// fieldMiss is a head no field of the template holds. +type fieldMiss struct { + head string + err error +} + +func (m *fieldMiss) Error() string { return m.err.Error() } + // checkTokens proves every parsed token names an existing field or a known function, -// so a typo'd or dangling reference is a New-time error. -func checkTokens(toks []formatToken, fields map[string]node) error { +// so a typo'd or dangling reference is a New-time error. A head no field holds comes +// back unbound, for bindNames to look up among the names. +func checkTokens(toks []formatToken, fields map[string]node) ([]unboundRead, error) { + var unbound []unboundRead for _, t := range toks { - if err := checkToken(t, fields); err != nil { - return err + u, err := checkToken(t, fields) + if err != nil { + return nil, err } + unbound = append(unbound, u...) } - return nil + return unbound, nil } -func checkToken(t formatToken, fields map[string]node) error { +func checkToken(t formatToken, fields map[string]node) ([]unboundRead, error) { switch t.kind { case builtinCall: - return checkFunc(t, fields) + return nil, checkFunc(t, fields) + case nameBind: + return nil, checkBind(t) case nameRead: + var unbound []unboundRead for _, name := range t.names { if isRef(name) { if _, _, err := refShape(name); err != nil { - return fmt.Errorf("token {%s}: %w", t.body, err) + return nil, fmt.Errorf("token {%s}: %w", t.body, err) } continue // its target is checked at New (see linkRefs) } - if err := checkArm(name, fields, len(t.names) == 1); err != nil { - return fmt.Errorf("token {%s}: %w", t.body, err) + err := checkArm(name, fields, len(t.names) == 1) + var miss *fieldMiss + if errors.As(err, &miss) { + unbound = append(unbound, unboundRead{miss.head, fmt.Errorf("token {%s}: %w", t.body, err)}) + continue + } + if err != nil { + return nil, fmt.Errorf("token {%s}: %w", t.body, err) } } // Last, so an arm broken on its own terms is reported as that: a repeat is // the consequence of such a mistake, not the mistake itself. - return checkNoRepeatedArm(t.body, t.names) + return unbound, checkNoRepeatedArm(t.body, t.names) + } + return nil, nil +} + +// checkBind proves a {ref as name} token binds a reference to a name no other token +// could read as something else. +func checkBind(t formatToken) error { + ref, name := t.names[0], t.bound + if trimmed := strings.TrimSpace(ref) + asWord + strings.TrimSpace(name); trimmed != t.body { + return fmt.Errorf("token {%s}: write {%s}", t.body, trimmed) + } + if !isRef(ref) { + return fmt.Errorf("token {%s}: %q is no reference; a name binds a pick of a reference, which starts with /, . or ..", t.body, ref) + } + if _, _, err := refShape(ref); err != nil { + return fmt.Errorf("token {%s}: %w", t.body, err) + } + if isOption(name) { + return fmt.Errorf("token {%s}: %q is an option and can never be a name", t.body, name) + } + if err := checkName(name); err != nil { + return fmt.Errorf("token {%s}: name %w", t.body, err) } return nil } // checkArm validates one sibling name or path against a template's fields. // wholeToken says the name is the token's entire body, so {/name} would render -// the same value and can be offered as the reference spelling. +// the same value and can be offered as the reference spelling. A head no field +// holds is a *fieldMiss. func checkArm(name string, fields map[string]node, wholeToken bool) error { a := splitArm(name, nil) if err := checkSegments(a); err != nil { @@ -258,9 +320,9 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { if wholeToken && hintableRef(name) { hint = fmt.Sprintf(" — write {/%s} to reference the data", name) } - return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint) + return &fieldMiss{a.head, fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint)} } - return fmt.Errorf("no field %q", a.head) + return &fieldMiss{a.head, fmt.Errorf("no field %q", a.head)} } if err := checkPathResolves(field, a.tail, a.head); err != nil { return fmt.Errorf("field %q: %w", a.head, err) @@ -316,6 +378,7 @@ type arm struct { leaves []node // every node the path may land on, one per variant it passes cover node // what holding the path pins: the first choice it passes, else its leaf kind armKind + bind *nameBinding // namedRead: the binding of the name it reads through } // armKind is how expand reads an arm, fixed at compile. @@ -325,6 +388,7 @@ const ( freshRead armKind = iota // drawn afresh at every read heldRead // drawn once per expansion, kept in its hold refPathRead // drawn once per draw group, kept in its memo + namedRead // read through a name, kept in its pick ) func (a arm) isRefPath() bool { @@ -439,10 +503,14 @@ func (c *formatOps) function(tok formatToken, refs map[string]refBinding) { c.ops = append(c.ops, op{formatToken: tok, call: builtins[tok.fn].prep(tok.args), operands: operands}) } -func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { +func (c *formatOps) field(tok formatToken, refs map[string]refBinding, isName func(string) bool) { arms := make([]arm, len(tok.names)) for i, name := range tok.names { arms[i] = splitArm(name, refs) + if isName(arms[i].head) { + arms[i].kind = namedRead + continue + } if len(arms[i].tail) > 0 { c.holdName(arms[i], "token {"+arms[i].spelling+"}") } @@ -450,9 +518,9 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { c.ops = append(c.ops, op{formatToken: tok, arms: arms}) } -// compileFormat compiles a parsed format. Call checkTokens first: it is what proves -// every token valid. -func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps { +// compileFormat compiles a parsed format, a binding to nothing, since it prints nothing. Call +// checkTokens first: it is what proves every token valid. +func compileFormat(toks []formatToken, refs map[string]refBinding, isName func(head string) bool) formatOps { var c formatOps for _, tok := range toks { switch tok.kind { @@ -462,7 +530,7 @@ func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps { case builtinCall: c.function(tok, refs) case nameRead: - c.field(tok, refs) + c.field(tok, refs, isName) } } return c diff --git a/inline.go b/inline.go index f03fda9..e7fec5c 100644 --- a/inline.go +++ b/inline.go @@ -163,7 +163,7 @@ func inlineBinding(n node, label string, root map[string]node) binding { // linkNodeRefs binds the references in an inline node's templates against the // loaded tree. func linkNodeRefs(scope nodeScope, root map[string]node) error { - return scope(func(path string, m node) error { + if err := scope(func(path string, m node) error { t, ok := m.(*template) if !ok { return nil @@ -178,5 +178,13 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { } } return linkTemplate(nil, path, "", t, root) + }); err != nil { + return err + } + return scope(func(path string, m node) error { + if t, ok := m.(*template); ok { + return linkNames(path, t) + } + return nil }) } diff --git a/name.go b/name.go new file mode 100644 index 0000000..b87b7e7 --- /dev/null +++ b/name.go @@ -0,0 +1,331 @@ +package fejkdata + +import ( + "fmt" + "strings" +) + +// nameScope is where a name is bound: a category, or one iteration of a repeat inside it. +// Every token of the category outside a nested repeat reads the category's names, so one +// field may bind a name and another read it. +type nameScope struct { + up *nameScope + owner node // the category's root, or the repeat + bindings map[string]*nameBinding + order []*nameBinding +} + +// nameBinding is one {ref as name} token: the scope it binds name in, its index among +// that scope's bindings, and, once linked, the head node and tail its reference reads. +type nameBinding struct { + name string + ref string + body string + where string // the fields reaching the binding template, as a compile error spells them + scope *nameScope + index int + binder *template + read bool + head node + tail []string +} + +func (sc *nameScope) lookup(name string) *nameBinding { + for ; sc != nil; sc = sc.up { + if b, ok := sc.bindings[name]; ok { + return b + } + } + return nil +} + +// bindNames gives every template of a compiled category or inline template the scope its +// names live in, and refuses a name bound twice along one chain of scopes, a name a field +// spells too, a read no field or name answers, and a binding nothing reads. +func bindNames(root node) error { + var scopes []*nameScope + var gather func(n node, scope *nameScope, where string) error + gather = func(n node, scope *nameScope, where string) error { + if t, isTemplate := n.(*template); isTemplate { + if t.repeat > 1 && n != root { + scope = &nameScope{up: scope, owner: t} + scopes = append(scopes, scope) + } + t.scope = scope + for _, tok := range t.tokens { + if tok.kind != nameBind { + continue + } + if err := scope.bind(tok, t, where); err != nil { + return err + } + } + } + return eachContained(n, where, func(c node, where string) error { return gather(c, scope, where) }) + } + top := &nameScope{owner: root} + scopes = append(scopes, top) + if err := gather(root, top, ""); err != nil { + return err + } + if err := resolveReads(root, ""); err != nil { + return err + } + for _, sc := range scopes { + if err := sc.close(); err != nil { + return err + } + } + return nil +} + +// close refuses a binding nothing reads, and gives the owner of a scope binding names the scope. +func (sc *nameScope) close() error { + for _, b := range sc.order { + if !b.read { + return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) + } + } + if len(sc.order) == 0 { + return nil + } + switch o := sc.owner.(type) { + case *template: + o.ownScope = sc + case *choice: + o.ownScope = sc + } + return nil +} + +// eachContained calls fn for each node n contains, with where naming the fields reaching it as +// compile errors do. +func eachContained(n node, where string, fn func(c node, where string) error) error { + for _, c := range contained(n) { + next := where + if c.name != "" { + next += fmt.Sprintf("field %q: ", c.name) + } + if err := fn(c.node, next); err != nil { + return err + } + } + return nil +} + +func (sc *nameScope) bind(tok formatToken, binder *template, where string) error { + if b, twice := sc.bindings[tok.bound]; twice { + return fmt.Errorf("%stoken {%s}: name %q is bound twice in one scope, by {%s} too; rename one", where, tok.body, tok.bound, b.body) + } + if b := sc.up.lookup(tok.bound); b != nil { + return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat already, by {%s}; rename one", where, tok.body, tok.bound, b.body) + } + b := &nameBinding{name: tok.bound, ref: tok.names[0], body: tok.body, where: where, scope: sc, index: len(sc.order), binder: binder} + if sc.bindings == nil { + sc.bindings = map[string]*nameBinding{} + } + sc.bindings[b.name] = b + sc.order = append(sc.order, b) + return nil +} + +// resolveReads answers each unbound read under n with a name its template sees, fields before the +// format as compile reports them, and refuses a field spelling a name. +func resolveReads(n node, where string) error { + if err := eachContained(n, where, resolveReads); err != nil { + return err + } + t, isTemplate := n.(*template) + if !isTemplate { + return nil + } + for _, name := range sortedNames(t.fields) { + if b := t.scope.lookup(name); b != nil { + return fmt.Errorf("%stoken {%s}: name %q is a field too; options, fields and names share one namespace, so rename one", b.where, b.body, name) + } + } + for _, u := range t.unbound { + b := t.scope.lookup(u.head) + if b == nil { + return fmt.Errorf("%s%w", where, u.err) + } + b.read = true + } + return nil +} + +// isName reports whether a head t reads is a name: neither a reference nor a field. +func (t *template) isName(head string) bool { + return !isRef(head) && t.fields[head] == nil && t.scope.lookup(head) != nil +} + +// linkNames compiles t's reads of a name as paths from the head its binding's reference names, once +// every template is linked, so the binder has resolved that reference. +func linkNames(path string, t *template) error { + for i := range t.compiled.ops { + for j := range t.compiled.ops[i].arms { + a := &t.compiled.ops[i].arms[j] + if a.kind != namedRead { + continue + } + if err := linkName(t, a); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), a.spelling, err) + } + } + } + return nil +} + +func linkName(t *template, a *arm) error { + b := t.scope.lookup(a.head) + if b.head == nil { + ref := b.binder.link.refs[b.ref] + b.head, b.tail = b.binder.link.refHeads[ref.head], ref.tail + } + if hasSelector(a.tail) { + return fmt.Errorf("a path through name %q selects no row; select it in the reference %q binds", a.head, a.head) + } + full := append(b.tail[:len(b.tail):len(b.tail)], a.tail...) + if err := checkPathResolves(b.head, full, a.head); err != nil { + return err + } + w := compilePath(b.head, full) + a.bind, a.steps, a.leaves, a.cover = b, w.steps, w.leaves, w.cover + a.levels = make([]string, len(full)+1) + for i := range a.levels { + a.levels[i] = joinSegments(full[:i]) + } + a.path = joinSegments(full) + return nil +} + +// namedPick is one draw of a name: the variant drawn at each level under it, the value each +// path through it read, keyed by the path from the name, and the table rows they pinned. +type namedPick struct { + memo drawMemo + pins pinSet +} + +// pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. +type pickFrame struct { + scope *nameScope + picks []namedPick +} + +func newPickFrame(scope *nameScope) *pickFrame { + return &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))} +} + +// frameStack is the frames of the name scopes rendering, innermost last. A render's draws point +// at it, and a repeat iteration's share it: holding the frames in the draws, or in a renderScope, +// would move every render's draws to the heap. +type frameStack struct { + frames []*pickFrame +} + +// pushFrame opens a frame of scope until popFrame closes it, returning the mark popFrame takes. +func (d *renderDraws) pushFrame(scope *nameScope) int { + if d.frames == nil { + d.frames = &frameStack{} + } + mark := len(d.frames.frames) + d.frames.frames = append(d.frames.frames, newPickFrame(scope)) + return mark +} + +func (d *renderDraws) popFrame(mark int) { d.frames.frames = d.frames.frames[:mark] } + +// pickOf is b's pick in the frame rendering its scope. A read entering the scope past its owner, +// as Fake("cat.field") or {/cat.field} do, keeps the frame in the memo of that read. +func (sc renderScope) pickOf(b *nameBinding) *namedPick { + if stack := sc.draws.frames; stack != nil { + for i := len(stack.frames) - 1; i >= 0; i-- { + if f := stack.frames[i]; f.scope == b.scope { + return &f.picks[b.index] + } + } + } + memo := sc.entry + if memo == nil { + memo = &sc.groupDraws().memo + } + f, ok := memo.frames[b.scope] + if !ok { + f = newPickFrame(b.scope) + if memo.frames == nil { + memo.frames = map[*nameScope]*pickFrame{} + } + memo.frames[b.scope] = f + } + return &f.picks[b.index] +} + +func readName(s *generatorState, sc renderScope, a arm) readValue { + return readPicked(s, sc.pickOf(a.bind), a.bind.head, a.steps, a.levels, a.path, sc) +} + +// drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one. +func (sc renderScope) drawRowOf(s *generatorState, t *table) int { + if sc.pick != nil { + return t.drawIn(s, &sc.pick.pins) + } + return t.drawRow(s) +} + +// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey: once per pick, +// by its path from the name. +func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue { + levels := make([]string, len(a.levels)) + for i, l := range a.levels { + levels[i] = underKey(sc.pickKey, l) + } + return readPicked(s, sc.pick, t.head(a.head), a.steps, levels, underKey(sc.pickKey, a.path), sc) +} + +// underKey is the key of path under the pick key prefix. It never returns prefix itself, which a +// memo would keep: anything a renderScope holds reaching the heap moves every render's draws there. +func underKey(prefix, path string) string { + if prefix == "" { + return path + } + return prefix + "." + path +} + +// readPicked draws the path key names under pick p, once, and renders what it lands on as part of p. +func readPicked(s *generatorState, p *namedPick, head node, steps []pathStep, levels []string, key string, sc renderScope) readValue { + if r, done := p.memo.value[key]; done { + return r + } + leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) + mark := -1 + if c, isChoice := leaf.(*choice); isChoice { + if c.ownScope != nil { + mark = sc.draws.pushFrame(c.ownScope) + } + leaf = p.memo.variantOf(s, c, key) + } + sc = sc.at(leaf, &p.pins) + sc.pick, sc.pickKey, sc.entry = p, key, &p.memo + r := renderLeaf(s, leaf, sc) + if mark >= 0 { + sc.draws.popFrame(mark) + } + if p.memo.value == nil { + p.memo.value = map[string]readValue{} + } + p.memo.value[key] = r + return r +} + +// variantOf is the variant of c drawn at level, drawn now where none was. +func (m *drawMemo) variantOf(s *generatorState, c *choice, level string) node { + n, drew := m.variant[level] + if !drew { + n = resolveChoice(s, c) + if m.variant == nil { + m.variant = map[string]node{} + } + m.variant[strings.Clone(level)] = n + } + return n +} diff --git a/node.go b/node.go index 841a324..9920cc7 100644 --- a/node.go +++ b/node.go @@ -25,9 +25,10 @@ func (*folder) isNode() {} // relative dot paths every item can address, so carriedByAll and List both read the one // answer to what a path may reach through this choice. type choice struct { - items []node - cum []float64 - shared map[string]bool + items []node + cum []float64 + shared map[string]bool + ownScope *nameScope // set on a category's root binding names, by bindNames } func (*choice) isNode() {} @@ -53,6 +54,11 @@ type template struct { fromString bool // written as a JSON string rather than an object isRecord bool // compiled at the top without a repeat, so its fields are record columns site tableSite + unbound []unboundRead // the heads its tokens read that no field holds + + // Filled by `bindNames`, from the compiled category: + scope *nameScope // where its tokens look a name up + ownScope *nameScope // the scope it renders a frame of: a category's, on its root, or a repeat's, per iteration // Filled by `linkTemplate`, from the assembled tree: link templateLink @@ -105,7 +111,11 @@ func (t *template) head(name string) node { // compile converts parsed JSON — a category or an inline template — into a node tree, // validating structure up front. func compile(v any) (node, error) { - return compileAt(v, atTop) + n, err := compileAt(v, atTop) + if err != nil { + return nil, err + } + return n, bindNames(n) } // compileCategory compiles a data file's value, which may be a table over a rows @@ -220,11 +230,17 @@ func jsonKind(v any) string { } func compileString(s string, site tableSite) (*template, error) { - toks, err := parseChecked(s, nil) + toks, unbound, err := parseChecked(s, nil) if err != nil { return nil, err } - return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site}, nil + if site.table != nil { + if err := checkTableToks(toks, unbound); err != nil { + return nil, err + } + unbound = nil + } + return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site, unbound: unbound}, nil } // fixedText is one render's output when the format holds no token; repeat is the caller's. @@ -329,14 +345,14 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" { return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format) } - toks, err := parseChecked(o.format, fields) + toks, unbound, err := parseChecked(o.format, fields) if err != nil { return nil, err } if err := checkNestedDrawGroup(fields, o.group); err != nil { return nil, err } - return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn}, nil + return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn, unbound: unbound}, nil } // templateOptions is what a template object's option keys say. @@ -525,6 +541,9 @@ func checkName(name string) error { if name == "-" { return fmt.Errorf(`%q is reserved: the struct tag fake:"-" leaves a field unfilled, so no tag could read it; rename it`, name) } + if strings.Contains(name, asWord) { + return fmt.Errorf("%q contains %q, which a token reads as binding a name; rename it", name, asWord) + } if i := strings.IndexAny(name, reservedInName); i >= 0 { return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path, {token} and JSON grammars reserve", name, name[i:i+1], reservedList) diff --git a/path.go b/path.go index ed99016..0666214 100644 --- a/path.go +++ b/path.go @@ -87,6 +87,22 @@ func indexOutside(s string, c byte) int { return -1 } +// cutOutside cuts s around the first sep outside a [selector]. +func cutOutside(s, sep string) (before, after string, found bool) { + depth := 0 + for i := 0; i < len(s); i++ { + switch { + case s[i] == '[': + depth++ + case s[i] == ']' && depth > 0: + depth-- + case depth == 0 && strings.HasPrefix(s[i:], sep): + return s[:i], s[i+len(sep):], true + } + } + return s, "", false +} + func splitOutside(s string, c byte) []string { var parts []string for { diff --git a/record.go b/record.go index 3b001e7..85ab697 100644 --- a/record.go +++ b/record.go @@ -22,7 +22,7 @@ type Column struct { // Record is one record rendered from a template: every direct field is a column, // listed in name order. Each column is its own expansion, so a sibling field is // local to it, while a reference that reads a path is drawn once for the whole -// record, per group. +// record, per group, and a name once for the whole record. type Record struct { columns []Column } @@ -263,6 +263,10 @@ func recordOf(n node) (*template, []Column, error) { // over sc's draws; a table's columns read the row pinned there. func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) + if t.ownScope != nil { + mark := sc.draws.pushFrame(t.ownScope) + defer sc.draws.popFrame(mark) + } if t.site.isFormat() { sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins) } diff --git a/reference.go b/reference.go index 80080e8..784362c 100644 --- a/reference.go +++ b/reference.go @@ -72,10 +72,13 @@ func refSegments(name string, folder []string) ([]string, error) { // refHeads under its root path and its tail reads like a sibling path, so two // spellings of one target are one draw. func linkRefs(root map[string]node) error { - return eachTemplate(root, func(folder []string, path string, t *template) error { + if err := eachTemplate(root, func(folder []string, path string, t *template) error { category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") return linkTemplate(folder, path, category, t, root) - }) + }); err != nil { + return err + } + return eachTemplate(root, func(_ []string, path string, t *template) error { return linkNames(path, t) }) } // linkTemplate resolves t's references in category, "" for an inline template, and @@ -85,7 +88,7 @@ func linkTemplate(folder []string, path, category string, t *template, root map[ if err != nil { return err } - t.link, t.compiled = link, compileFormat(t.tokens, link.refs) + t.link, t.compiled = link, compileFormat(t.tokens, link.refs, t.isName) compileArms(t) return nil } @@ -100,7 +103,9 @@ func compileArms(t *template) { compileArm(t, &o.operands[j]) } for j := range o.arms { - compileArm(t, &o.arms[j]) + if o.arms[j].kind != namedRead { + compileArm(t, &o.arms[j]) + } } } } diff --git a/render.go b/render.go index b9d3901..c703bc1 100644 --- a/render.go +++ b/render.go @@ -61,11 +61,15 @@ func renderOnce(s *generatorState, n node) string { func render(s *generatorState, n node, sc renderScope) string { switch n := n.(type) { case *choice: + if n.ownScope != nil { + mark := sc.draws.pushFrame(n.ownScope) + defer sc.draws.popFrame(mark) + } return render(s, pick(s, n), sc) case *nullItem: return "" case *table: - sc.row = renderedRow{n, n.drawRow(s)} + sc.row = renderedRow{n, sc.drawRowOf(s, n)} return expand(s, n.formatTemplate, sc) case *tableRow: return expand(s, n.t.formatTemplate, sc) @@ -78,6 +82,10 @@ func render(s *generatorState, n node, sc renderScope) string { case *template: sc = sc.in(n) if n.repeat == 1 { + if n.ownScope != nil { + mark := sc.draws.pushFrame(n.ownScope) + defer sc.draws.popFrame(mark) + } if lit, fixed := n.fixedText(); fixed { return lit } @@ -89,7 +97,7 @@ func render(s *generatorState, n node, sc renderScope) string { if i > 0 { b.WriteString(n.separator) } - b.WriteString(expandAnew(s, n)) + b.WriteString(expandAnew(s, n, sc.draws.frames)) } return b.String() default: @@ -97,12 +105,15 @@ func render(s *generatorState, n node, sc renderScope) string { } } -// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into -// render's loop, its draws would move to the heap. +// expandAnew expands one repeat iteration of t as a render of its own, in no group, inside the +// name scopes of frames. Inlined into render's loop, its draws would move to the heap. // //go:noinline -func expandAnew(s *generatorState, t *template) string { - var draws renderDraws +func expandAnew(s *generatorState, t *template, frames *frameStack) string { + draws := renderDraws{frames: frames} + if t.ownScope != nil { + defer draws.popFrame(draws.pushFrame(t.ownScope)) + } return expand(s, t, renderScope{draws: &draws}) } @@ -155,7 +166,8 @@ func expand(s *generatorState, t *template, sc renderScope) string { // token addresses by a dotted path that is not a reference, or a field an operand // reads — is drawn once and kept in hold, so {place.postal-code} and {place.locality} // read one row, either read twice gives one value, and a shown operand is the operand -// computed. Every other name is drawn afresh, so {word} {word} still draws twice. +// computed. A field of a template rendering as part of a named pick is kept in the pick. +// Every other field is drawn afresh, so {word} {word} still draws twice. func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue { if a.kind == refPathRead { return readReference(s, t, sc, a) @@ -163,7 +175,13 @@ func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a if sc.draws.trace != nil { traceRead(sc.draws.trace, t, sc, a) } - if a.kind == freshRead { + switch { + case a.kind == namedRead: + return readName(s, sc, a) + case sc.pick != nil && !isRef(a.head): + return readUnder(s, t, sc, a) + case a.kind == freshRead: + sc.pick = nil return readValue{text: render(s, t.head(a.head), sc)} } return readMemo(s, t, hold, nil, sc, a) @@ -187,6 +205,7 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r if pins != nil { sc = sc.at(leaf, pins) } + sc.pick, sc.entry = nil, memo r := renderLeaf(s, leaf, sc) if memo.value == nil { memo.value = map[string]readValue{} @@ -209,6 +228,10 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. func renderLeaf(s *generatorState, n node, sc renderScope) readValue { + if c, isChoice := n.(*choice); isChoice && c.ownScope != nil { + mark := sc.draws.pushFrame(c.ownScope) + defer sc.draws.popFrame(mark) + } n = resolveChoice(s, n) switch leaf := n.(type) { case *nullItem: diff --git a/table.go b/table.go index 6a7e9cd..1375324 100644 --- a/table.go +++ b/table.go @@ -323,6 +323,9 @@ func (t *table) compileRowFormat(format string) error { if err != nil { return err } + if err := checkTableToks(toks, nil); err != nil { + return err + } for _, tok := range toks { if tok.kind != nameRead { continue @@ -338,14 +341,32 @@ func (t *table) compileRowFormat(format string) error { for i, name := range t.header { fields[name] = &tableColumn{t, i} } - if err := checkTokens(toks, fields); err != nil { + unbound, err := checkTokens(toks, fields) + if err != nil { return err } + if len(unbound) > 0 { + return unbound[0].err + } t.formatTemplate = &template{format: format, tokens: toks, fields: fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} t.rowNode = &tableRow{t} return nil } +// checkTableToks refuses a table's format or cell binding a name, or reading a head no column +// holds: a table holds no names. +func checkTableToks(toks []formatToken, unbound []unboundRead) error { + for _, tok := range toks { + if tok.kind == nameBind { + return fmt.Errorf("token {%s}: a table binds no name; bind it in a template reading the table", tok.body) + } + } + if len(unbound) > 0 { + return unbound[0].err + } + return nil +} + // setTablePaths gives every table the path a selector on it is written at, before a // link or a draw can name one. func setTablePaths(root map[string]node) { diff --git a/todo.md b/todo.md index 451cb0a..a3a8ad9 100644 --- a/todo.md +++ b/todo.md @@ -55,7 +55,6 @@ | 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | -| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 5.3, 4.1 | 25.8 | | 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | | 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 4 | 23.1 | @@ -186,23 +185,16 @@ Shape: T = table, t = template, c = choice. | `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | | `direction`, `continent`, `ulid` | c/t | — | — | -### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. - -- A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. -- A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read. -- A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line. -- Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace. - ### 3. Rewrite the shipped categories so facts that belong together come from one named pick. - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. - `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. -- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. +- Needs item 2. Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Needs items 2 and 3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. -- 2.52.0 From 40e7ab46f6eaf3d6b7e684572f20f5bc0692ac37 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:02:32 +0200 Subject: [PATCH 430/966] Test a name entered past its category, a shadow written first, a binding in a choice, and a name as a transform operand --- name_test.go | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 67 insertions(+), 2 deletions(-) diff --git a/name_test.go b/name_test.go index 2ed818c..9111c7d 100644 --- a/name_test.go +++ b/name_test.go @@ -167,11 +167,15 @@ func TestNameErrors(t *testing.T) { want string }{ {"bound twice", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{y}","x":"{/word as w}{w}","y":"{/word as w}"}`}, - `name "w" is bound twice in one scope`}, + `name "w" is bound twice outside any repeat`}, + {"bound inside a repeat written first", map[string]string{"word": `["a","b"]`, "card": `{"format":"{a}{z}","a":{"format":"{/word as w}{w}","repeat":2},"z":"{/word as w}{w}"}`}, + `name "w" is bound outside this repeat already`}, + {"bound in a choice's item", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":["{/word as w}","b"]}`}, + `a choice's item binds no name`}, {"bound outside the repeat too", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":{"format":"{/word as w}{w}","repeat":2}}`}, `name "w" is bound outside this repeat already`}, {"a field too", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{w}","w":"x"}`}, - `name "w" is a field too`}, + `name "w" is a field of the root template too`}, {"an option", map[string]string{"word": `["a","b"]`, "card": `"{/word as format}{format}"`}, `"format" is an option and can never be a name`}, {"no reference", map[string]string{"card": `{"format":"{word as w}{w}","word":["a","b"]}`}, @@ -258,3 +262,64 @@ func TestStructTagsReadOneName(t *testing.T) { } } } + +func TestANameEnteredPastItsCategoryAgreesWithItsWhole(t *testing.T) { + dir := writeData(t, map[string]string{ + "given": `["Ada","Bo","Cy","Di","Ed","Flo"]`, + "person": `{"format":"{/given as g}{g}","first":"{g}"}`, + "card": `"{/person as p}{p}|{p.first}"`, + }) + f := newGenerator(t, dir, WithSeed(37)) + for i := 0; i < 100; i++ { + got := strings.Split(fake(t, f, "card"), "|") + if got[0] != got[1] { + t.Fatalf("card = %q, want {p} and {p.first} to read one pick of g", got) + } + } +} + +func TestANameKeepsItsPickInARepeatReachedByPath(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `["a","b","c","d","e","f","g","h"]`, + "line": `{"format":"{/word as w}{each}","each":{"format":"{w}","repeat":40,"separator":" "}}`, + "far": `"{/line.each}"`, + }) + f := newGenerator(t, dir, WithSeed(41)) + for _, path := range []string{"line.each", "far"} { + got := strings.Fields(fake(t, f, path)) + for _, w := range got { + if w != got[0] { + t.Fatalf("%s = %q, want one pick on every line", path, got) + } + } + } +} + +func TestAHeldPathReadsTheNameItsFieldReads(t *testing.T) { + dir := writeFiles(t, map[string]string{ + "person.json": `{"format":"{first} {last}","rows":"person.tsv"}`, + "person.tsv": "first\tlast\nAda\tByron\nBo\tEk\nCy\tLind\n", + "cat.json": `{"format":"{/person as n}{field}","field":{"format":"{n.first} {x.y}","x":{"format":"{y}","y":"{n.last}"}}}`, + }) + f := newGenerator(t, dir, WithSeed(43)) + people := map[string]bool{"Ada Byron": true, "Bo Ek": true, "Cy Lind": true} + for i := 0; i < 100; i++ { + if got := fake(t, f, "cat.field"); !people[got] { + t.Fatalf("cat.field = %q, want one person read through n", got) + } + } +} + +func TestANameIsATransformOperand(t *testing.T) { + dir := writeData(t, map[string]string{ + "person": `{"format":"{first}","first":["Åsa","Bo","Cy"]}`, + "card": `"{/person as p}{p.first} <{lowercase(ascii(p.first))}>"`, + }) + f := newGenerator(t, dir, WithSeed(47)) + want := map[string]bool{"Åsa ": true, "Bo ": true, "Cy ": true} + for i := 0; i < 50; i++ { + if got := fake(t, f, "card"); !want[got] { + t.Fatalf("card = %q, want the operand to read the pick", got) + } + } +} -- 2.52.0 From d0c477ec7818a337f465cf19565517f23553d31b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:05:02 +0200 Subject: [PATCH 431/966] Keep a name's frame with the read entering its category, refuse a binding in a choice's item, and read a name as a transform operand --- CHANGELOG.md | 11 +-- README.md | 15 ++-- docs/decisions.md | 17 +++-- draws.go | 6 +- format.go | 16 +++- name.go | 186 ++++++++++++++++++++++++++++------------------ node.go | 7 +- record.go | 3 +- reference.go | 4 +- render.go | 54 +++++++------- todo.md | 9 ++- 11 files changed, 198 insertions(+), 130 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 230d153..581ffa8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,11 +11,12 @@ replacement, and each removed path, column or flag. - A name keeps one pick: `{/misc.territory as t}` binds one row of `misc.territory` to `t` and prints nothing, and `{t}`, `{t.capital}` and `{t.calling-code}` read that one row. A name lives in the category binding it, so every field and record column of the - category reads one pick, and a name a `repeat` binds picks again on every iteration. - Refused at `New`: a name bound twice in one category, or inside a `repeat` and outside - it; a name that is a field or an option; a binding of anything but a reference; a - binding nothing reads; a path through a name that selects a row; and a binding in a - table. A field, column, category or folder name may not contain ` as `. + category reads one pick, a transform reads it as `{lowercase(t.name)}`, and a name a + `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice in + one category, or inside a `repeat` and outside it; a name that is a field or an option; + a binding of anything but a reference; a binding in a choice's item; a binding nothing + reads; a path through a name that selects a row; and a binding in a table. A field, + column, category or folder name may not contain ` as `. - First release: the CLI, the library and the shipped data set. - Table categories: a category JSON naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and `parent`; a path selects a row by key or name, diff --git a/README.md b/README.md index 6ca5188..12fa13d 100644 --- a/README.md +++ b/README.md @@ -656,7 +656,7 @@ fail, except in a [typed column](#datatype), which must prove neither happens. ### Transforms `{lowercase(x)}`, `{uppercase(x)}` and `{ascii(x)}` rewrite the value of `x` — a -field, a path or a `..path` — and nest. `ascii` folds Latin letters (`Åsa Öberg` +field, a path, a `..path` or a [name](#names) — and nest. `ascii` folds Latin letters (`Åsa Öberg` → `Asa Oberg`) and drops any other non-ASCII rune. `x` is held ([One draw, one spelling](#one-draw-one-spelling)), so an email built from a name matches the name beside it: @@ -709,17 +709,20 @@ row: ``` Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn -once, so `{n}` and `{n.path}` agree, and a field `n` renders twice prints the same text twice. +once, so `{n}` and `{n.path}` agree, a [transform](#transforms) of `n.path` reads the same +value, and a field `n` renders twice prints the same text twice. A name is a pick of its own: +`{t.name}` and `{/misc.territory.name}` beside it are two draws. + A name is drawn on its first read, and lives in the category binding it: any field of the category reads it, and a record's columns read one pick. Each render of the category picks anew, and so does each bare reference to it. A name a `repeat` binds picks again on every iteration, and one bound outside the repeat keeps its pick on every line. Rejected at load: a name bound twice in one category, or both inside a `repeat` and outside -it; a name that is a field or an option; a binding of anything but a reference; a binding -nothing reads; a path through a name that selects a row, which the reference selects instead; -and a binding in a table's format or cell. A field, column, category or folder name may not -contain ` as `. +it; a name that is a field or an option; a binding of anything but a reference; a binding in +a choice's item, which every other item would leave unbound; a binding nothing reads; a path +through a name that selects a row, which the reference selects instead; and a binding in a +table's format or cell. A field, column, category or folder name may not contain ` as `. ### Draw group diff --git a/docs/decisions.md b/docs/decisions.md index 73658b5..3f44575 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -894,10 +894,13 @@ a struct's fields as one. 2026-10-03, larv-review. Goals 5.3 and 5.6. `bindNames` scopes a name at compile, and a render keeps one `pickFrame` per scope. One field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside -`"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category is the smallest unit holding every field and record column. The -scope is lexical: a category referencing another never sees its names, so what a category -renders never depends on who references it. A pick is drawn on its first read, so a record's -columns, rendered in name order, read one pick whichever binds it. A read entering a category -past its root, `Fake("cat.field")` or `{/cat.field}`, keeps the category's frame in the memo of -that read: per render and group, as the read's own draw is. Valid while a name is read only -inside the category binding it. +`"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category +is the smallest unit holding every field and record column. A choice's item binds no name, +since a read outside the item would see a binding another item leaves out. The scope is +lexical: a read entering a category, by a reference or through a name, sees none of the +frames rendering above it, so what a category renders never depends on who references it. A +pick is drawn on its first read, so a record's columns, rendered in name order, read one pick +whichever binds it. A read entering a category keeps the frame of each scope it enters in its +own memo, and the category's root renders that frame when the read lands on the root, so +`{n}` and `{n.path}` read one pick: per render and group for a reference path, per pick for a +path through a name, and per render for `Fake("cat.field")`. diff --git a/draws.go b/draws.go index f49728b..3695b45 100644 --- a/draws.go +++ b/draws.go @@ -45,7 +45,11 @@ type renderScope struct { draws *renderDraws group string row renderedRow - entry *drawMemo // the memo of the read that entered what renders + // entry is the memo of the read that entered the category rendering, nil where none did; + // base, the depth of the frame stack when it entered, below which no frame is the + // category's. + entry *drawMemo + base int // pick is the named pick what renders is part of, at pickKey, its path from the name; nil // outside one. pick *namedPick diff --git a/format.go b/format.go index 185be82..5003366 100644 --- a/format.go +++ b/format.go @@ -247,7 +247,12 @@ func checkTokens(toks []formatToken, fields map[string]node) ([]unboundRead, err func checkToken(t formatToken, fields map[string]node) ([]unboundRead, error) { switch t.kind { case builtinCall: - return nil, checkFunc(t, fields) + err := checkFunc(t, fields) + var miss *fieldMiss + if errors.As(err, &miss) { + return []unboundRead{{miss.head, err}}, nil + } + return nil, err case nameBind: return nil, checkBind(t) case nameRead: @@ -493,10 +498,15 @@ func (c *formatOps) holdName(a arm, label string) { c.held[a.head] = h } -func (c *formatOps) function(tok formatToken, refs map[string]refBinding) { +func (c *formatOps) function(tok formatToken, refs map[string]refBinding, isName func(string) bool) { var operands []arm for _, operand := range tok.names { a := splitArm(operand, refs) + if isName(a.head) { + a.kind = namedRead + operands = append(operands, a) + continue + } c.holdName(a, fmt.Sprintf("%s operand %q", tok.fn, operand)) operands = append(operands, a) } @@ -528,7 +538,7 @@ func compileFormat(toks []formatToken, refs map[string]refBinding, isName func(h c.grow += len(tok.lit) c.ops = append(c.ops, op{formatToken: tok}) case builtinCall: - c.function(tok, refs) + c.function(tok, refs, isName) case nameRead: c.field(tok, refs, isName) } diff --git a/name.go b/name.go index b87b7e7..fb4dc6a 100644 --- a/name.go +++ b/name.go @@ -40,34 +40,38 @@ func (sc *nameScope) lookup(name string) *nameBinding { } // bindNames gives every template of a compiled category or inline template the scope its -// names live in, and refuses a name bound twice along one chain of scopes, a name a field -// spells too, a read no field or name answers, and a binding nothing reads. +// names live in, and refuses a name bound twice along one chain of scopes, a binding in a +// choice's item, a name a field spells too, a read no field or name answers, and a binding +// nothing reads. func bindNames(root node) error { - var scopes []*nameScope - var gather func(n node, scope *nameScope, where string) error - gather = func(n node, scope *nameScope, where string) error { + top := &nameScope{owner: root} + scopes := []*nameScope{top} + var gather func(n node, scope *nameScope, inChoice bool, where string) error + gather = func(n node, scope *nameScope, inChoice bool, where string) error { if t, isTemplate := n.(*template); isTemplate { if t.repeat > 1 && n != root { scope = &nameScope{up: scope, owner: t} scopes = append(scopes, scope) + inChoice = false } t.scope = scope - for _, tok := range t.tokens { - if tok.kind != nameBind { - continue - } - if err := scope.bind(tok, t, where); err != nil { - return err - } + if err := scope.bindAll(t, inChoice, where); err != nil { + return err } } - return eachContained(n, where, func(c node, where string) error { return gather(c, scope, where) }) + _, isChoice := n.(*choice) + return eachContained(n, where, func(c node, where string) error { return gather(c, scope, inChoice || isChoice, where) }) } - top := &nameScope{owner: root} - scopes = append(scopes, top) - if err := gather(root, top, ""); err != nil { + if err := gather(root, top, false, ""); err != nil { return err } + for _, sc := range scopes[1:] { + for _, b := range sc.order { + if outer := sc.up.lookup(b.name); outer != nil { + return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat already, by {%s}; rename one", b.where, b.body, b.name, outer.body) + } + } + } if err := resolveReads(root, ""); err != nil { return err } @@ -79,6 +83,37 @@ func bindNames(root node) error { return nil } +// bindAll binds every name t's tokens bind. A choice's item binds none: a read outside the item +// would find the name bound by an item that may never be drawn. +func (sc *nameScope) bindAll(t *template, inChoice bool, where string) error { + for _, tok := range t.tokens { + if tok.kind != nameBind { + continue + } + if inChoice { + return fmt.Errorf("%stoken {%s}: a choice's item binds no name, since every other item would leave it unbound; bind it outside the choice", where, tok.body) + } + if b, twice := sc.bindings[tok.bound]; twice { + return fmt.Errorf("%stoken {%s}: name %q is bound twice %s, by {%s} too; rename one", where, tok.body, tok.bound, sc.spelled(), b.body) + } + b := &nameBinding{name: tok.bound, ref: tok.names[0], body: tok.body, where: where, scope: sc, index: len(sc.order), binder: t} + if sc.bindings == nil { + sc.bindings = map[string]*nameBinding{} + } + sc.bindings[b.name] = b + sc.order = append(sc.order, b) + } + return nil +} + +// spelled names the scope in an error. +func (sc *nameScope) spelled() string { + if sc.up == nil { + return "outside any repeat" + } + return "in one repeat" +} + // close refuses a binding nothing reads, and gives the owner of a scope binding names the scope. func (sc *nameScope) close() error { for _, b := range sc.order { @@ -86,14 +121,8 @@ func (sc *nameScope) close() error { return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) } } - if len(sc.order) == 0 { - return nil - } - switch o := sc.owner.(type) { - case *template: - o.ownScope = sc - case *choice: - o.ownScope = sc + if t, isTemplate := sc.owner.(*template); isTemplate && len(sc.order) > 0 { + t.ownScope = sc } return nil } @@ -113,22 +142,6 @@ func eachContained(n node, where string, fn func(c node, where string) error) er return nil } -func (sc *nameScope) bind(tok formatToken, binder *template, where string) error { - if b, twice := sc.bindings[tok.bound]; twice { - return fmt.Errorf("%stoken {%s}: name %q is bound twice in one scope, by {%s} too; rename one", where, tok.body, tok.bound, b.body) - } - if b := sc.up.lookup(tok.bound); b != nil { - return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat already, by {%s}; rename one", where, tok.body, tok.bound, b.body) - } - b := &nameBinding{name: tok.bound, ref: tok.names[0], body: tok.body, where: where, scope: sc, index: len(sc.order), binder: binder} - if sc.bindings == nil { - sc.bindings = map[string]*nameBinding{} - } - sc.bindings[b.name] = b - sc.order = append(sc.order, b) - return nil -} - // resolveReads answers each unbound read under n with a name its template sees, fields before the // format as compile reports them, and refuses a field spelling a name. func resolveReads(n node, where string) error { @@ -141,7 +154,11 @@ func resolveReads(n node, where string) error { } for _, name := range sortedNames(t.fields) { if b := t.scope.lookup(name); b != nil { - return fmt.Errorf("%stoken {%s}: name %q is a field too; options, fields and names share one namespace, so rename one", b.where, b.body, name) + holder := "the root template" + if where != "" { + holder = strings.TrimSuffix(where, ": ") + } + return fmt.Errorf("%stoken {%s}: name %q is a field of %s too; options, fields and names share one namespace, so rename one", b.where, b.body, name, holder) } } for _, u := range t.unbound { @@ -163,13 +180,15 @@ func (t *template) isName(head string) bool { // every template is linked, so the binder has resolved that reference. func linkNames(path string, t *template) error { for i := range t.compiled.ops { - for j := range t.compiled.ops[i].arms { - a := &t.compiled.ops[i].arms[j] - if a.kind != namedRead { - continue - } - if err := linkName(t, a); err != nil { - return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), a.spelling, err) + o := &t.compiled.ops[i] + for _, reads := range [][]arm{o.arms, o.operands} { + for j := range reads { + if reads[j].kind != namedRead { + continue + } + if err := linkName(t, &reads[j]); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), o.body, err) + } } } } @@ -223,23 +242,58 @@ type frameStack struct { frames []*pickFrame } -// pushFrame opens a frame of scope until popFrame closes it, returning the mark popFrame takes. -func (d *renderDraws) pushFrame(scope *nameScope) int { +// pushFrame opens f until popFrame closes it, returning the mark popFrame takes. +func (d *renderDraws) pushFrame(f *pickFrame) int { + mark := d.depth() if d.frames == nil { d.frames = &frameStack{} } - mark := len(d.frames.frames) - d.frames.frames = append(d.frames.frames, newPickFrame(scope)) + d.frames.frames = append(d.frames.frames, f) return mark } func (d *renderDraws) popFrame(mark int) { d.frames.frames = d.frames.frames[:mark] } -// pickOf is b's pick in the frame rendering its scope. A read entering the scope past its owner, -// as Fake("cat.field") or {/cat.field} do, keeps the frame in the memo of that read. +func (d *renderDraws) depth() int { + if d.frames == nil { + return 0 + } + return len(d.frames.frames) +} + +// entering is sc as a read entering another category sees it: none of the frames rendering, and +// memo, which keeps the frame of each scope the read enters past its owner. +func (sc renderScope) entering(memo *drawMemo) renderScope { + sc.base, sc.entry, sc.pick = sc.draws.depth(), memo, nil + return sc +} + +// frameOf is the frame a render of scope's owner opens: the one the read entering the category +// keeps, so {n} and {n.path} read one frame, else a fresh one. +func (sc renderScope) frameOf(scope *nameScope) *pickFrame { + if sc.entry == nil { + return newPickFrame(scope) + } + return sc.entry.frameOf(scope) +} + +func (m *drawMemo) frameOf(scope *nameScope) *pickFrame { + f, ok := m.frames[scope] + if !ok { + f = newPickFrame(scope) + if m.frames == nil { + m.frames = map[*nameScope]*pickFrame{} + } + m.frames[scope] = f + } + return f +} + +// pickOf is b's pick: in the frame rendering its scope since the read entering the category, else +// in the frame that read keeps, as for Fake("cat.field") or {/cat.field}. func (sc renderScope) pickOf(b *nameBinding) *namedPick { if stack := sc.draws.frames; stack != nil { - for i := len(stack.frames) - 1; i >= 0; i-- { + for i := len(stack.frames) - 1; i >= sc.base; i-- { if f := stack.frames[i]; f.scope == b.scope { return &f.picks[b.index] } @@ -249,19 +303,12 @@ func (sc renderScope) pickOf(b *nameBinding) *namedPick { if memo == nil { memo = &sc.groupDraws().memo } - f, ok := memo.frames[b.scope] - if !ok { - f = newPickFrame(b.scope) - if memo.frames == nil { - memo.frames = map[*nameScope]*pickFrame{} - } - memo.frames[b.scope] = f - } - return &f.picks[b.index] + return &memo.frameOf(b.scope).picks[b.index] } func readName(s *generatorState, sc renderScope, a arm) readValue { - return readPicked(s, sc.pickOf(a.bind), a.bind.head, a.steps, a.levels, a.path, sc) + p := sc.pickOf(a.bind) + return readPicked(s, p, a.bind.head, a.steps, a.levels, a.path, sc.entering(&p.memo)) } // drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one. @@ -297,19 +344,12 @@ func readPicked(s *generatorState, p *namedPick, head node, steps []pathStep, le return r } leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) - mark := -1 if c, isChoice := leaf.(*choice); isChoice { - if c.ownScope != nil { - mark = sc.draws.pushFrame(c.ownScope) - } leaf = p.memo.variantOf(s, c, key) } sc = sc.at(leaf, &p.pins) - sc.pick, sc.pickKey, sc.entry = p, key, &p.memo + sc.pick, sc.pickKey = p, key r := renderLeaf(s, leaf, sc) - if mark >= 0 { - sc.draws.popFrame(mark) - } if p.memo.value == nil { p.memo.value = map[string]readValue{} } diff --git a/node.go b/node.go index 9920cc7..6438566 100644 --- a/node.go +++ b/node.go @@ -25,10 +25,9 @@ func (*folder) isNode() {} // relative dot paths every item can address, so carriedByAll and List both read the one // answer to what a path may reach through this choice. type choice struct { - items []node - cum []float64 - shared map[string]bool - ownScope *nameScope // set on a category's root binding names, by bindNames + items []node + cum []float64 + shared map[string]bool } func (*choice) isNode() {} diff --git a/record.go b/record.go index 85ab697..ede1fe8 100644 --- a/record.go +++ b/record.go @@ -264,8 +264,7 @@ func recordOf(n node) (*template, []Column, error) { func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) if t.ownScope != nil { - mark := sc.draws.pushFrame(t.ownScope) - defer sc.draws.popFrame(mark) + defer sc.draws.popFrame(sc.draws.pushFrame(sc.frameOf(t.ownScope))) } if t.site.isFormat() { sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins) diff --git a/reference.go b/reference.go index 784362c..93a064f 100644 --- a/reference.go +++ b/reference.go @@ -100,7 +100,9 @@ func compileArms(t *template) { for i := range t.compiled.ops { o := &t.compiled.ops[i] for j := range o.operands { - compileArm(t, &o.operands[j]) + if o.operands[j].kind != namedRead { + compileArm(t, &o.operands[j]) + } } for j := range o.arms { if o.arms[j].kind != namedRead { diff --git a/render.go b/render.go index c703bc1..779a6a3 100644 --- a/render.go +++ b/render.go @@ -61,10 +61,6 @@ func renderOnce(s *generatorState, n node) string { func render(s *generatorState, n node, sc renderScope) string { switch n := n.(type) { case *choice: - if n.ownScope != nil { - mark := sc.draws.pushFrame(n.ownScope) - defer sc.draws.popFrame(mark) - } return render(s, pick(s, n), sc) case *nullItem: return "" @@ -83,38 +79,48 @@ func render(s *generatorState, n node, sc renderScope) string { sc = sc.in(n) if n.repeat == 1 { if n.ownScope != nil { - mark := sc.draws.pushFrame(n.ownScope) - defer sc.draws.popFrame(mark) + defer sc.draws.popFrame(sc.draws.pushFrame(sc.frameOf(n.ownScope))) } if lit, fixed := n.fixedText(); fixed { return lit } return expand(s, n, sc) } - var b strings.Builder - b.Grow(n.repeat * (n.compiled.grow + len(n.separator))) - for i := 0; i < n.repeat; i++ { - if i > 0 { - b.WriteString(n.separator) - } - b.WriteString(expandAnew(s, n, sc.draws.frames)) - } - return b.String() + return renderRepeat(s, n, sc) default: panic(internalError("uncompiled node %T", n)) } } +// renderRepeat renders each iteration of t as a render of its own, inside the name scopes +// rendering t, where a name bound outside t keeps its pick: a scope sc's category was entered +// past keeps its frame in that read's memo, else in sc's render's. +func renderRepeat(s *generatorState, t *template, sc renderScope) string { + entry := sc.entry + if entry == nil { + entry = &sc.groupDraws().memo + } + var b strings.Builder + b.Grow(t.repeat * (t.compiled.grow + len(t.separator))) + for i := 0; i < t.repeat; i++ { + if i > 0 { + b.WriteString(t.separator) + } + b.WriteString(expandAnew(s, t, sc.draws.frames, sc.base, entry)) + } + return b.String() +} + // expandAnew expands one repeat iteration of t as a render of its own, in no group, inside the -// name scopes of frames. Inlined into render's loop, its draws would move to the heap. +// name scopes of frames from base. Inlined into render's loop, its draws would move to the heap. // //go:noinline -func expandAnew(s *generatorState, t *template, frames *frameStack) string { +func expandAnew(s *generatorState, t *template, frames *frameStack, base int, entry *drawMemo) string { draws := renderDraws{frames: frames} if t.ownScope != nil { - defer draws.popFrame(draws.pushFrame(t.ownScope)) + defer draws.popFrame(draws.pushFrame(newPickFrame(t.ownScope))) } - return expand(s, t, renderScope{draws: &draws}) + return expand(s, t, renderScope{draws: &draws, base: base, entry: entry}) } // pick selects one item. Uniform choices are O(1); weighted choices are an @@ -180,8 +186,9 @@ func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a return readName(s, sc, a) case sc.pick != nil && !isRef(a.head): return readUnder(s, t, sc, a) + case a.kind == freshRead && isRef(a.head): + return readValue{text: render(s, t.head(a.head), sc.entering(nil))} case a.kind == freshRead: - sc.pick = nil return readValue{text: render(s, t.head(a.head), sc)} } return readMemo(s, t, hold, nil, sc, a) @@ -203,9 +210,8 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r } leaf := drawSteps(s, t.head(a.head), a.steps, pins, memo, a.levels) if pins != nil { - sc = sc.at(leaf, pins) + sc = sc.at(leaf, pins).entering(memo) } - sc.pick, sc.entry = nil, memo r := renderLeaf(s, leaf, sc) if memo.value == nil { memo.value = map[string]readValue{} @@ -228,10 +234,6 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. func renderLeaf(s *generatorState, n node, sc renderScope) readValue { - if c, isChoice := n.(*choice); isChoice && c.ownScope != nil { - mark := sc.draws.pushFrame(c.ownScope) - defer sc.draws.popFrame(mark) - } n = resolveChoice(s, n) switch leaf := n.(type) { case *nullItem: diff --git a/todo.md b/todo.md index a3a8ad9..50c92d8 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 71` +`Next ID = 72` | Goal | W | |---|---| @@ -65,6 +65,7 @@ | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 3 | 19.1 | +| 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | @@ -194,7 +195,7 @@ Shape: T = table, t = template, c = choice. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 2 and 3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Needs items 2, 3 and 71. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. @@ -251,6 +252,10 @@ Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Run 1 is the recorded score; run 2 reran the same code to measure noise. A spread of 0.3 is close to the whole 5.7–6.1 range the recorded scores span since 2026-09-20, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. Until this item is decided, that rule compares against 5.7, from 2026-09-30. +### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. + +`checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. + ### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. -- 2.52.0 From 30d18bf48582a152d51e5635e5d41d3e0bf37b60 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:12:24 +0200 Subject: [PATCH 432/966] Test one name across draw groups, two names on one category, and a repeat of a name's allocations --- name_test.go | 30 ++++++++++++++++++++++++++++++ perf_test.go | 1 + 2 files changed, 31 insertions(+) diff --git a/name_test.go b/name_test.go index 9111c7d..e3a07d7 100644 --- a/name_test.go +++ b/name_test.go @@ -323,3 +323,33 @@ func TestANameIsATransformOperand(t *testing.T) { } } } + +func TestOneRenderReadsOneNameAcrossDrawGroups(t *testing.T) { + dir := writeData(t, map[string]string{ + "word": `["a","b","c","d","e","f","g","h"]`, + "other": `{"format":"{x}","x":["1","2"]}`, + "cat": `{"format":"{/word as w}{a}","a":{"format":"{w}|{b}","b":{"format":"{w}{/other.x}","drawGroup":"g"}}}`, + }) + f := newGenerator(t, dir, WithSeed(59)) + for i := 0; i < 50; i++ { + if got := fake(t, f, "cat.a"); got[0] != got[2] { + t.Fatalf("cat.a = %q, want one pick of w in both draw groups", got) + } + } +} + +func TestTwoNamesOnOneCategoryAreTwoPicks(t *testing.T) { + dir := writeData(t, map[string]string{ + "given": `{"format":"{first}","first":["Ada","Bo","Cy","Di","Ed","Flo","Gus","Hal"]}`, + "person": `"{/given.first}"`, + "card": `"{/person as p}{/person as q}{p}|{q}"`, + }) + f := newGenerator(t, dir, WithSeed(53)) + for i := 0; i < 50; i++ { + got := strings.Split(fake(t, f, "card"), "|") + if got[0] != got[1] { + return + } + } + t.Fatal("card printed one person for p and q in 50 renders, want two picks") +} diff --git a/perf_test.go b/perf_test.go index 8a0d2c7..92cc738 100644 --- a/perf_test.go +++ b/perf_test.go @@ -64,6 +64,7 @@ func TestNoReferenceAllocRegression(t *testing.T) { }{ {"a repeat of a reference path", `{"format":"{r}","r":{"format":"{/word.w}","repeat":20,"separator":", "}}`, 66}, {"a named draw group", `{"format":"{a}","a":{"format":"{/word.w}","drawGroup":"g"}}`, 11}, + {"a repeat of a name", `{"format":"{r}","r":{"format":"{/word as n}{n.w}","repeat":20,"separator":", "}}`, 205}, } { f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{ "word.json": {Data: []byte(word)}, -- 2.52.0 From 4362905b82cb0feaacc9ae20fd984750fdd6de5f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:12:24 +0200 Subject: [PATCH 433/966] Fix a render's fallback frame to its unnamed group, and draw a pick's references in draw groups of its own --- CHANGELOG.md | 4 ++-- README.md | 7 ++++--- docs/decisions.md | 3 ++- draws.go | 19 ++++++++++++------- name.go | 19 ++++++++++++++----- render.go | 8 ++------ 6 files changed, 36 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 581ffa8..f41a89e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,8 +12,8 @@ replacement, and each removed path, column or flag. and prints nothing, and `{t}`, `{t.capital}` and `{t.calling-code}` read that one row. A name lives in the category binding it, so every field and record column of the category reads one pick, a transform reads it as `{lowercase(t.name)}`, and a name a - `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice in - one category, or inside a `repeat` and outside it; a name that is a field or an option; + `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice + outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a name that is a field or an option; a binding of anything but a reference; a binding in a choice's item; a binding nothing reads; a path through a name that selects a row; and a binding in a table. A field, column, category or folder name may not contain ` as `. diff --git a/README.md b/README.md index 12fa13d..96f071f 100644 --- a/README.md +++ b/README.md @@ -711,15 +711,16 @@ row: Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn once, so `{n}` and `{n.path}` agree, a [transform](#transforms) of `n.path` reads the same value, and a field `n` renders twice prints the same text twice. A name is a pick of its own: -`{t.name}` and `{/misc.territory.name}` beside it are two draws. +`{t.name}` and `{/misc.territory.name}` beside it are two draws, as are two names bound to one +category. A name is drawn on its first read, and lives in the category binding it: any field of the category reads it, and a record's columns read one pick. Each render of the category picks anew, and so does each bare reference to it. A name a `repeat` binds picks again on every iteration, and one bound outside the repeat keeps its pick on every line. -Rejected at load: a name bound twice in one category, or both inside a `repeat` and outside -it; a name that is a field or an option; a binding of anything but a reference; a binding in +Rejected at load: a name bound twice outside any `repeat`, twice in one `repeat`, or both +inside a `repeat` and outside it; a name that is a field or an option; a binding of anything but a reference; a binding in a choice's item, which every other item would leave unbound; a binding nothing reads; a path through a name that selects a row, which the reference selects instead; and a binding in a table's format or cell. A field, column, category or folder name may not contain ` as `. diff --git a/docs/decisions.md b/docs/decisions.md index 3f44575..40bd461 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -903,4 +903,5 @@ pick is drawn on its first read, so a record's columns, rendered in name order, whichever binds it. A read entering a category keeps the frame of each scope it enters in its own memo, and the category's root renders that frame when the read lands on the root, so `{n}` and `{n.path}` read one pick: per render and group for a reference path, per pick for a -path through a name, and per render for `Fake("cat.field")`. +path through a name, and per render for `Fake("cat.field")`. A reference read under a pick +draws in the pick's own draw groups, so two names bound to one category are two picks of it. diff --git a/draws.go b/draws.go index 3695b45..38072b4 100644 --- a/draws.go +++ b/draws.go @@ -51,9 +51,10 @@ type renderScope struct { entry *drawMemo base int // pick is the named pick what renders is part of, at pickKey, its path from the name; nil - // outside one. + // outside one. refs is the draw groups of the last pick entered, nil before one. pick *namedPick pickKey string + refs *renderDraws } type renderedRow struct { @@ -90,16 +91,20 @@ func (sc renderScope) rowOf(t *table) int { } func (sc renderScope) groupDraws() *groupDraws { - if sc.group == "" { - return &sc.draws.unnamed + draws := sc.draws + if sc.refs != nil { + draws = sc.refs } - d, drew := sc.draws.named[sc.group] + if sc.group == "" { + return &draws.unnamed + } + d, drew := draws.named[sc.group] if !drew { - if sc.draws.named == nil { - sc.draws.named = map[string]*groupDraws{} + if draws.named == nil { + draws.named = map[string]*groupDraws{} } d = &groupDraws{} - sc.draws.named[strings.Clone(sc.group)] = d + draws.named[strings.Clone(sc.group)] = d } return d } diff --git a/name.go b/name.go index fb4dc6a..5f40896 100644 --- a/name.go +++ b/name.go @@ -219,10 +219,12 @@ func linkName(t *template, a *arm) error { } // namedPick is one draw of a name: the variant drawn at each level under it, the value each -// path through it read, keyed by the path from the name, and the table rows they pinned. +// path through it read, keyed by the path from the name, and the table rows they pinned; and the +// draw groups of the references read under it, so two names on one category are two picks of it. type namedPick struct { memo drawMemo pins pinSet + refs renderDraws } // pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. @@ -299,15 +301,22 @@ func (sc renderScope) pickOf(b *nameBinding) *namedPick { } } } - memo := sc.entry - if memo == nil { - memo = &sc.groupDraws().memo + return &sc.entryMemo().frameOf(b.scope).picks[b.index] +} + +// entryMemo is the memo of the read that entered the category rendering, else the unnamed draw +// group's of the render: one memo for a render begun below a category's root, whatever draw group +// its templates name. +func (sc renderScope) entryMemo() *drawMemo { + if sc.entry != nil { + return sc.entry } - return &memo.frameOf(b.scope).picks[b.index] + return &sc.draws.unnamed.memo } func readName(s *generatorState, sc renderScope, a arm) readValue { p := sc.pickOf(a.bind) + sc.refs, sc.group = &p.refs, "" return readPicked(s, p, a.bind.head, a.steps, a.levels, a.path, sc.entering(&p.memo)) } diff --git a/render.go b/render.go index 779a6a3..78c6848 100644 --- a/render.go +++ b/render.go @@ -93,13 +93,9 @@ func render(s *generatorState, n node, sc renderScope) string { } // renderRepeat renders each iteration of t as a render of its own, inside the name scopes -// rendering t, where a name bound outside t keeps its pick: a scope sc's category was entered -// past keeps its frame in that read's memo, else in sc's render's. +// rendering t, where a name bound outside t keeps its pick in the frame sc.entryMemo keeps. func renderRepeat(s *generatorState, t *template, sc renderScope) string { - entry := sc.entry - if entry == nil { - entry = &sc.groupDraws().memo - } + entry := sc.entryMemo() var b strings.Builder b.Grow(t.repeat * (t.compiled.grow + len(t.separator))) for i := 0; i < t.repeat; i++ { -- 2.52.0 From df63d0f562ae1823e61b19950f9bed66b0b38435 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:18:21 +0200 Subject: [PATCH 434/966] Draw a pick's references in the draw group of the read of the name, where the draw fences survey them --- README.md | 5 +++-- docs/decisions.md | 3 ++- draws.go | 17 ++++++----------- name.go | 7 +++---- name_test.go | 16 ---------------- 5 files changed, 14 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index 96f071f..5a2ddc2 100644 --- a/README.md +++ b/README.md @@ -711,8 +711,9 @@ row: Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn once, so `{n}` and `{n.path}` agree, a [transform](#transforms) of `n.path` reads the same value, and a field `n` renders twice prints the same text twice. A name is a pick of its own: -`{t.name}` and `{/misc.territory.name}` beside it are two draws, as are two names bound to one -category. +`{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path read under a +name draws in the draw group of the read of the name, so give the reads of two names bound to +one category a [draw group](#draw-group) each to draw what it references apart. A name is drawn on its first read, and lives in the category binding it: any field of the category reads it, and a record's columns read one pick. Each render of the category picks diff --git a/docs/decisions.md b/docs/decisions.md index 40bd461..db0f4e0 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -904,4 +904,5 @@ whichever binds it. A read entering a category keeps the frame of each scope it own memo, and the category's root renders that frame when the read lands on the root, so `{n}` and `{n.path}` read one pick: per render and group for a reference path, per pick for a path through a name, and per render for `Fake("cat.field")`. A reference read under a pick -draws in the pick's own draw groups, so two names bound to one category are two picks of it. +draws in the draw group of the read of the name, the group the draw fences survey it in; a pick +with draw groups of its own would draw where no fence looks. diff --git a/draws.go b/draws.go index 38072b4..3695b45 100644 --- a/draws.go +++ b/draws.go @@ -51,10 +51,9 @@ type renderScope struct { entry *drawMemo base int // pick is the named pick what renders is part of, at pickKey, its path from the name; nil - // outside one. refs is the draw groups of the last pick entered, nil before one. + // outside one. pick *namedPick pickKey string - refs *renderDraws } type renderedRow struct { @@ -91,20 +90,16 @@ func (sc renderScope) rowOf(t *table) int { } func (sc renderScope) groupDraws() *groupDraws { - draws := sc.draws - if sc.refs != nil { - draws = sc.refs - } if sc.group == "" { - return &draws.unnamed + return &sc.draws.unnamed } - d, drew := draws.named[sc.group] + d, drew := sc.draws.named[sc.group] if !drew { - if draws.named == nil { - draws.named = map[string]*groupDraws{} + if sc.draws.named == nil { + sc.draws.named = map[string]*groupDraws{} } d = &groupDraws{} - draws.named[strings.Clone(sc.group)] = d + sc.draws.named[strings.Clone(sc.group)] = d } return d } diff --git a/name.go b/name.go index 5f40896..0a841ac 100644 --- a/name.go +++ b/name.go @@ -219,12 +219,12 @@ func linkName(t *template, a *arm) error { } // namedPick is one draw of a name: the variant drawn at each level under it, the value each -// path through it read, keyed by the path from the name, and the table rows they pinned; and the -// draw groups of the references read under it, so two names on one category are two picks of it. +// path through it read, keyed by the path from the name, and the table rows they pinned. A +// reference read under it draws in the draw group of the read of the name, which the draw +// fences survey it in. type namedPick struct { memo drawMemo pins pinSet - refs renderDraws } // pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. @@ -316,7 +316,6 @@ func (sc renderScope) entryMemo() *drawMemo { func readName(s *generatorState, sc renderScope, a arm) readValue { p := sc.pickOf(a.bind) - sc.refs, sc.group = &p.refs, "" return readPicked(s, p, a.bind.head, a.steps, a.levels, a.path, sc.entering(&p.memo)) } diff --git a/name_test.go b/name_test.go index e3a07d7..d664e1f 100644 --- a/name_test.go +++ b/name_test.go @@ -337,19 +337,3 @@ func TestOneRenderReadsOneNameAcrossDrawGroups(t *testing.T) { } } } - -func TestTwoNamesOnOneCategoryAreTwoPicks(t *testing.T) { - dir := writeData(t, map[string]string{ - "given": `{"format":"{first}","first":["Ada","Bo","Cy","Di","Ed","Flo","Gus","Hal"]}`, - "person": `"{/given.first}"`, - "card": `"{/person as p}{/person as q}{p}|{q}"`, - }) - f := newGenerator(t, dir, WithSeed(53)) - for i := 0; i < 50; i++ { - got := strings.Split(fake(t, f, "card"), "|") - if got[0] != got[1] { - return - } - } - t.Fatal("card printed one person for p and q in 50 renders, want two picks") -} -- 2.52.0 From 685e7bb27cb5850a0df6f2377555e2a167374d9e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:29:21 +0200 Subject: [PATCH 435/966] Test a name keeping only what its paths address, the second spelling of a reference, and names in tables and file names --- name_test.go | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) diff --git a/name_test.go b/name_test.go index d664e1f..5e176e9 100644 --- a/name_test.go +++ b/name_test.go @@ -46,26 +46,32 @@ func TestANameReadWholeAgreesWithItsPaths(t *testing.T) { } } -func TestANameDrawsEachFieldUnderItOnce(t *testing.T) { +func TestANameKeepsWhatItsPathsAddress(t *testing.T) { dir := writeData(t, map[string]string{ "word": `{"format":"{w}-{w}","w":["a","b","c","d","e","f","g","h"]}`, - "card": `"{/word as w}{w}"`, + "card": `"{/word as n}{n}|{n}"`, }) f := newGenerator(t, dir, WithSeed(7)) + differ := false for i := 0; i < 100; i++ { - if got := fake(t, f, "card"); got[0] != got[2] { - t.Fatalf("card = %q, want one draw of w under the name", got) + got := strings.Split(fake(t, f, "card"), "|") + if got[0] != got[1] { + t.Fatalf("card = %q, want one pick of n", got) } + differ = differ || got[0][0] != got[0][2] + } + if !differ { + t.Fatal("{w}-{w} under a name drew w once in 100 renders, want each {w} a pick of its own") } } func TestABindingPrintsNothing(t *testing.T) { dir := writeData(t, map[string]string{ "word": `["a","b"]`, - "card": `"<{/word as w}>{w}"`, + "card": `"<{/word as w}>{w}{w}"`, }) f := newGenerator(t, dir, WithSeed(1)) - if got := fake(t, f, "card"); got != "<>a" && got != "<>b" { + if got := fake(t, f, "card"); got != "<>aa" && got != "<>bb" { t.Fatalf("card = %q, want the binding to print nothing", got) } } @@ -137,12 +143,12 @@ func TestANameOutsideARepeatKeepsItsPick(t *testing.T) { func TestTwoRendersPickTwice(t *testing.T) { dir := writeData(t, map[string]string{ "word": `["a","b","c","d","e","f","g","h"]`, - "one": `"{/word as w}{w}"`, + "one": `"{/word as w}{w}{w}"`, "two": `"{/one}{/one}{/one}{/one}{/one}{/one}"`, }) f := newGenerator(t, dir, WithSeed(17)) got := fake(t, f, "two") - if strings.Count(got, got[:1]) == len(got) { + if strings.Count(got, got[:1]) == len(got) || got[0] != got[1] { t.Fatalf("two = %q, want each bare reference to its own pick", got) } } @@ -182,6 +188,20 @@ func TestNameErrors(t *testing.T) { `"word" is no reference`}, {"read by nothing", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}"`}, `nothing reads name "w"`}, + {"read once whole", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":"{w}"}`}, + `write {/word}`}, + {"rendered twice where a path reads it", map[string]string{"word": `{"format":"{w}-{w}","w":["a","b"]}`, "card": `"{/word as n}{n}|{n.w}"`}, + `{n} renders field "w" twice`}, + {"a binding in a cell", map[string]string{"word": `["a","b"]`, "t.json": `{"format":"{k}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\tc\nx\t{/word as w}{w}{w}\ny\tz\n"}, + "a table binds no name"}, + {"in a category's name", map[string]string{"a as b": `"x"`}, + `contains " as "`}, + {"in a folder's name", map[string]string{"a as b/c": `"x"`}, + `contains " as "`}, + {"in a column's name", map[string]string{"t.json": `{"format":"{k}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\ta as b\nx\t1\ny\t2\n"}, + `contains " as "`}, + {"read outside the repeat binding it", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":{"format":"{/word as w}{w}","repeat":2}}`}, + `name "w" is bound inside the repeat at field "x"`}, {"unbound", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":{"format":"{/word as w}{w}","repeat":2}}`}, `no field "w"`}, {"padded", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}{w}"`}, -- 2.52.0 From b1aed332e798356ea6c02786d13e8975c646d228 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 02:36:12 +0200 Subject: [PATCH 436/966] Keep in a pick only what reads of its name address, look frames up on the stack alone, and refuse a name read once whole --- CHANGELOG.md | 10 +- README.md | 28 ++-- doc.go | 6 +- docs/decisions.md | 29 ++-- draws.go | 22 ++- format.go | 102 ++++++------- inline.go | 15 +- name.go | 357 ++++++++++++++++++++++++++++++++++------------ node.go | 11 +- record.go | 4 +- reference.go | 14 +- render.go | 39 +++-- table.go | 4 +- todo.md | 30 +++- transform.go | 15 +- 15 files changed, 455 insertions(+), 231 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f41a89e..ae3f359 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,10 +13,12 @@ replacement, and each removed path, column or flag. A name lives in the category binding it, so every field and record column of the category reads one pick, a transform reads it as `{lowercase(t.name)}`, and a name a `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice - outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a name that is a field or an option; - a binding of anything but a reference; a binding in a choice's item; a binding nothing - reads; a path through a name that selects a row; and a binding in a table. A field, - column, category or folder name may not contain ` as `. + outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a + name that is a field or an option; a binding of anything but a reference; a binding in a + choice's item; a binding nothing reads, or one read once whole, which the reference + spells; a path through a name that selects a row; a read of a name rendering a field + twice where a path through the name reads it; and a binding in a table. A field, column, + category or folder name may not contain ` as `. - First release: the CLI, the library and the shipped data set. - Table categories: a category JSON naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and `parent`; a path selects a row by key or name, diff --git a/README.md b/README.md index 5a2ddc2..10db0ff 100644 --- a/README.md +++ b/README.md @@ -708,12 +708,16 @@ row: "{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}" ``` -Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn -once, so `{n}` and `{n.path}` agree, a [transform](#transforms) of `n.path` reads the same -value, and a field `n` renders twice prints the same text twice. A name is a pick of its own: -`{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path read under a -name draws in the draw group of the read of the name, so give the reads of two names bound to -one category a [draw group](#draw-group) each to draw what it references apart. +Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. A name keeps each level a +`{n.path}` reads, and what it lands on, so `{n}` and `{n.path}` agree, and a +[transform](#transforms) of `n.path` reads the same value. Everything else under the name draws +as it would anywhere: `{w}-{w}` in the category `n` names draws `w` twice, so `{n.w}` beside +`{n}` is refused there, since it could read either draw. + +A name is a pick of its own: `{t.name}` and `{/misc.territory.name}` beside it are two draws. A +reference path the picked category reads is still held for the render as usual, so two names +bound to `sv_SE.person`, whose fields read `{.first-name.name}`, print one person unless their +reads sit in two [draw groups](#draw-group). A name is drawn on its first read, and lives in the category binding it: any field of the category reads it, and a record's columns read one pick. Each render of the category picks @@ -721,10 +725,12 @@ anew, and so does each bare reference to it. A name a `repeat` binds picks again iteration, and one bound outside the repeat keeps its pick on every line. Rejected at load: a name bound twice outside any `repeat`, twice in one `repeat`, or both -inside a `repeat` and outside it; a name that is a field or an option; a binding of anything but a reference; a binding in -a choice's item, which every other item would leave unbound; a binding nothing reads; a path -through a name that selects a row, which the reference selects instead; and a binding in a -table's format or cell. A field, column, category or folder name may not contain ` as `. +inside a `repeat` and outside it; a name that is a field or an option; a binding of anything +but a reference; a binding in a choice's item, which every other item would leave unbound; a +binding nothing reads, or one read once whole outside a `repeat`, which the reference spells; +a path through a name that selects a row, which the reference selects instead; a read of a name +rendering a field twice where a path through the name reads it; and a binding in a table's +format or cell. A field, column, category or folder name may not contain ` as `. ### Draw group @@ -926,7 +932,7 @@ record.go records: Record, the JSON/CSV/SQL serializers, and their entry p struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format -draws.go the memo a hold or draw group keeps its draws in; and a render's draws and the scope it reads its reference paths through +draws.go the memo a hold, draw group or named pick keeps its draws in; and a render's draws and the scope it reads its reference paths and names through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways drawfence.go the load fences over reference draws and their refusals: a drawGroup's, a render's and a record's columns judged on what they read, and a table reaching its own family reads.go the load-time fold of what a render reads, and what a path reads of a table family diff --git a/doc.go b/doc.go index 7841405..1461d4b 100644 --- a/doc.go +++ b/doc.go @@ -27,9 +27,9 @@ package fejkdata // an expansion holds a sibling path's head and an operand's field, its draw // group a reference path's: `formatOps.held` and `heldCheck` an expansion's, // `readReference` a draw group's. -// - named pick — one draw of a reference a binding token, {ref as n}, gives a name, kept for one -// render of the scope it is bound in, a category or a repeat iteration, and drawn on its -// first read: `nameBinding`, `nameScope`, `namedPick`, `pickFrame`. +// - named pick — the draw a token such as {/person as p} binds to a name. It is drawn on its +// first read and kept while its scope renders: a category, or one repeat iteration. +// `nameBinding`, `nameScope`, `namedPick`, `pickFrame`. // - memo — what a hold, a draw group or a named pick keeps its draws in, the variant drawn at // each level and the value each path read: `drawMemo`, `readMemo`. // - draw group — reference draws kept apart inside one render: diff --git a/docs/decisions.md b/docs/decisions.md index db0f4e0..286ea38 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -662,6 +662,8 @@ same four columns as `geo.SE.address`, each a reference into it, and the format twice; a column is spelled the same in both, `street-number`, so the two never disagree on a name. +`todo.md` item 3 replaces it. + ## A postort's kommun comes from its name, its tätort or its codes, never from distance 2026-09-24, Lilleman auf Larv. @@ -896,13 +898,20 @@ a struct's fields as one. field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside `"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category is the smallest unit holding every field and record column. A choice's item binds no name, -since a read outside the item would see a binding another item leaves out. The scope is -lexical: a read entering a category, by a reference or through a name, sees none of the -frames rendering above it, so what a category renders never depends on who references it. A -pick is drawn on its first read, so a record's columns, rendered in name order, read one pick -whichever binds it. A read entering a category keeps the frame of each scope it enters in its -own memo, and the category's root renders that frame when the read lands on the root, so -`{n}` and `{n.path}` read one pick: per render and group for a reference path, per pick for a -path through a name, and per render for `Fake("cat.field")`. A reference read under a pick -draws in the draw group of the read of the name, the group the draw fences survey it in; a pick -with draw groups of its own would draw where no fence looks. +since a read outside the item would see a binding another item leaves out. A pick is drawn on +its first read, so a record's columns, rendered in name order, read one pick whichever binds +it. + +The scope is lexical: a read entering a category, by a reference, through a name or from +`Fake`, sees none of the frames rendering above it, so what a category renders never depends +on who references it. That read opens the frame of each scope around what it lands on, taken +from its own memo, so every read entering by one memo, `{n}` and `{n.path}` alike, reads one +pick: per render and group for a reference path, per pick for a path through a name, per +render for `Fake("cat.field")`. A frame is looked up on the stack of frames rendering, and +nowhere else. + +A pick keeps only the levels a read of the name addresses, so a template draws the same way +under a name as anywhere else (goal 5.2). A reference read under a pick draws in the draw group +of the read of the name, the group the draw fences survey it in, so two names bound to one +category share what it reads by reference path. `todo.md` item 4 ends that: once every `{…}` +draws afresh, nothing is held for the render. diff --git a/draws.go b/draws.go index 3695b45..e0aa063 100644 --- a/draws.go +++ b/draws.go @@ -7,9 +7,9 @@ import "strings" // so the same read written twice reads one value; and the frame of each name scope a read // entered past its owner. type drawMemo struct { - variant map[string]node - value map[string]readValue - frames map[*nameScope]*pickFrame + variant map[string]node + value map[string]readValue + enteredFrames map[*nameScope]*pickFrame } // groupDraws is what one render's draw group has drawn for its reference paths: a memo, and the @@ -31,10 +31,10 @@ type renderTrace func(group string, row renderedRow, a arm) // renderDraws is one render's reference draws: the unnamed draw group's, and each named one's; // a test's trace of its reads; and the frames of the name scopes rendering. type renderDraws struct { - unnamed groupDraws - named map[string]*groupDraws - trace renderTrace - frames *frameStack + unnamed groupDraws + named map[string]*groupDraws + trace renderTrace + frameStack *frameStack } // renderScope is where a render reads its reference paths: its reference draws, in the draw group of the @@ -45,11 +45,9 @@ type renderScope struct { draws *renderDraws group string row renderedRow - // entry is the memo of the read that entered the category rendering, nil where none did; - // base, the depth of the frame stack when it entered, below which no frame is the - // category's. - entry *drawMemo - base int + // base is the depth of the frame stack when the read rendering the category entered it, + // below which no frame is the category's. + base int // pick is the named pick what renders is part of, at pickKey, its path from the name; nil // outside one. pick *namedPick diff --git a/format.go b/format.go index 5003366..8bf18e6 100644 --- a/format.go +++ b/format.go @@ -1,7 +1,6 @@ package fejkdata import ( - "errors" "fmt" "strings" ) @@ -64,8 +63,9 @@ type formatToken struct { body string // the braces' content, as written fn string // builtinCall args []string // builtinCall - names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads; nameBind: the reference it binds - bound string // nameBind: the name + names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads + // nameBind: the reference it binds, and the name. + boundRef, bound string } type tokenKind uint8 @@ -98,7 +98,7 @@ func parseFormat(format string) ([]formatToken, error) { } flush() if ref, name, binds := cutOutside(u.body, asWord); binds && indexOutside(u.body, '(') < 0 { - toks = append(toks, formatToken{kind: nameBind, body: u.body, names: []string{ref}, bound: name}) + toks = append(toks, formatToken{kind: nameBind, body: u.body, boundRef: ref, bound: name}) return nil } if indexOutside(u.body, '(') < 0 { @@ -215,20 +215,15 @@ func parseChecked(format string, fields map[string]node) ([]formatToken, []unbou } // unboundRead is a token reading a head no field holds, which only a name bound around the -// template can answer; err is the refusal where none does. +// template can answer; err is the refusal where none does. whole is a read of the name itself, +// no path into it, and operand a builtin's read of it. type unboundRead struct { - head string - err error + head string + whole bool + operand bool + err error } -// fieldMiss is a head no field of the template holds. -type fieldMiss struct { - head string - err error -} - -func (m *fieldMiss) Error() string { return m.err.Error() } - // checkTokens proves every parsed token names an existing field or a known function, // so a typo'd or dangling reference is a New-time error. A head no field holds comes // back unbound, for bindNames to look up among the names. @@ -247,32 +242,16 @@ func checkTokens(toks []formatToken, fields map[string]node) ([]unboundRead, err func checkToken(t formatToken, fields map[string]node) ([]unboundRead, error) { switch t.kind { case builtinCall: - err := checkFunc(t, fields) - var miss *fieldMiss - if errors.As(err, &miss) { - return []unboundRead{{miss.head, err}}, nil + if err := checkFunc(t, fields); err != nil { + return nil, err } - return nil, err + return checkReads(t, fields, true) case nameBind: return nil, checkBind(t) case nameRead: - var unbound []unboundRead - for _, name := range t.names { - if isRef(name) { - if _, _, err := refShape(name); err != nil { - return nil, fmt.Errorf("token {%s}: %w", t.body, err) - } - continue // its target is checked at New (see linkRefs) - } - err := checkArm(name, fields, len(t.names) == 1) - var miss *fieldMiss - if errors.As(err, &miss) { - unbound = append(unbound, unboundRead{miss.head, fmt.Errorf("token {%s}: %w", t.body, err)}) - continue - } - if err != nil { - return nil, fmt.Errorf("token {%s}: %w", t.body, err) - } + unbound, err := checkReads(t, fields, false) + if err != nil { + return nil, err } // Last, so an arm broken on its own terms is reported as that: a repeat is // the consequence of such a mistake, not the mistake itself. @@ -281,10 +260,35 @@ func checkToken(t formatToken, fields map[string]node) ([]unboundRead, error) { return nil, nil } +// checkReads proves each name t reads, an arm or an operand, is a reference or a path into a +// field, returning those whose head no field holds. +func checkReads(t formatToken, fields map[string]node, operands bool) ([]unboundRead, error) { + var unbound []unboundRead + for _, name := range t.names { + if isRef(name) { + if _, _, err := refShape(name); err != nil { + return nil, fmt.Errorf("token {%s}: %w", t.body, err) + } + continue // its target is checked at New (see linkRefs) + } + missing, err := checkArm(name, fields, !operands && len(t.names) == 1) + if err == nil { + continue + } + err = fmt.Errorf("token {%s}: %w", t.body, err) + if !missing { + return nil, err + } + a := splitArm(name, nil) + unbound = append(unbound, unboundRead{head: a.head, whole: len(a.tail) == 0, operand: operands, err: err}) + } + return unbound, nil +} + // checkBind proves a {ref as name} token binds a reference to a name no other token // could read as something else. func checkBind(t formatToken) error { - ref, name := t.names[0], t.bound + ref, name := t.boundRef, t.bound if trimmed := strings.TrimSpace(ref) + asWord + strings.TrimSpace(name); trimmed != t.body { return fmt.Errorf("token {%s}: write {%s}", t.body, trimmed) } @@ -305,34 +309,34 @@ func checkBind(t formatToken) error { // checkArm validates one sibling name or path against a template's fields. // wholeToken says the name is the token's entire body, so {/name} would render -// the same value and can be offered as the reference spelling. A head no field -// holds is a *fieldMiss. -func checkArm(name string, fields map[string]node, wholeToken bool) error { +// the same value and can be offered as the reference spelling. missing says no +// field holds the head, which a name may yet answer. +func checkArm(name string, fields map[string]node, wholeToken bool) (missing bool, err error) { a := splitArm(name, nil) if err := checkSegments(a); err != nil { - return err + return false, err } field, ok := fields[a.head] if !ok { if a.head == "" { - return fmt.Errorf("a name is never empty, so this token can name no field") + return false, fmt.Errorf("a name is never empty, so this token can name no field") } if isOption(a.head) { - return fmt.Errorf("%q is an option and can never be a field", a.head) + return false, fmt.Errorf("%q is an option and can never be a field", a.head) } if len(fields) == 0 { hint := "" if wholeToken && hintableRef(name) { hint = fmt.Sprintf(" — write {/%s} to reference the data", name) } - return &fieldMiss{a.head, fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint)} + return true, fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint) } - return &fieldMiss{a.head, fmt.Errorf("no field %q", a.head)} + return true, fmt.Errorf("no field %q", a.head) } if err := checkPathResolves(field, a.tail, a.head); err != nil { - return fmt.Errorf("field %q: %w", a.head, err) + return false, fmt.Errorf("field %q: %w", a.head, err) } - return nil + return false, nil } // hintableRef reports whether {/name} is a reference the grammar accepts, so the @@ -383,7 +387,7 @@ type arm struct { leaves []node // every node the path may land on, one per variant it passes cover node // what holding the path pins: the first choice it passes, else its leaf kind armKind - bind *nameBinding // namedRead: the binding of the name it reads through + named *nameBinding // namedRead: the binding of the name it reads through } // armKind is how expand reads an arm, fixed at compile. diff --git a/inline.go b/inline.go index e7fec5c..924a348 100644 --- a/inline.go +++ b/inline.go @@ -181,10 +181,15 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { }); err != nil { return err } - return scope(func(path string, m node) error { - if t, ok := m.(*template); ok { - return linkNames(path, t) + for _, pass := range []func(string, *template) error{linkNames, checkNameReads} { + if err := scope(func(path string, m node) error { + if t, ok := m.(*template); ok { + return pass(path, t) + } + return nil + }); err != nil { + return err } - return nil - }) + } + return nil } diff --git a/name.go b/name.go index 0a841ac..03f7cc8 100644 --- a/name.go +++ b/name.go @@ -2,6 +2,7 @@ package fejkdata import ( "fmt" + "sort" "strings" ) @@ -16,7 +17,8 @@ type nameScope struct { } // nameBinding is one {ref as name} token: the scope it binds name in, its index among -// that scope's bindings, and, once linked, the head node and tail its reference reads. +// that scope's bindings, its uses, and, once linked, the head node and tail its +// reference reads and the keys its reads address. type nameBinding struct { name string ref string @@ -25,9 +27,18 @@ type nameBinding struct { scope *nameScope index int binder *template - read bool + uses []nameUse head node tail []string + // addressed is every key a read of the name lands on or passes, the spelling of the + // first read reaching it beside it; a pick keeps the draws at these keys, and only these. + addressed map[string]string +} + +// nameUse is one read of a name: whole, with no path into it, an operand of a builtin, or in a +// repeat nested inside the name's scope. +type nameUse struct { + whole, operand, nested bool } func (sc *nameScope) lookup(name string) *nameBinding { @@ -42,22 +53,25 @@ func (sc *nameScope) lookup(name string) *nameBinding { // bindNames gives every template of a compiled category or inline template the scope its // names live in, and refuses a name bound twice along one chain of scopes, a binding in a // choice's item, a name a field spells too, a read no field or name answers, and a binding -// nothing reads. +// read by nothing, or once whole, which its reference spells. func bindNames(root node) error { top := &nameScope{owner: root} scopes := []*nameScope{top} var gather func(n node, scope *nameScope, inChoice bool, where string) error gather = func(n node, scope *nameScope, inChoice bool, where string) error { - if t, isTemplate := n.(*template); isTemplate { + switch t := n.(type) { + case *template: if t.repeat > 1 && n != root { scope = &nameScope{up: scope, owner: t} scopes = append(scopes, scope) inChoice = false } - t.scope = scope + t.nameScope = scope if err := scope.bindAll(t, inChoice, where); err != nil { return err } + case *choice: + t.nameScope = scope } _, isChoice := n.(*choice) return eachContained(n, where, func(c node, where string) error { return gather(c, scope, inChoice || isChoice, where) }) @@ -72,11 +86,11 @@ func bindNames(root node) error { } } } - if err := resolveReads(root, ""); err != nil { + if err := resolveReads(root, "", scopes); err != nil { return err } for _, sc := range scopes { - if err := sc.close(); err != nil { + if err := sc.settle(); err != nil { return err } } @@ -96,7 +110,7 @@ func (sc *nameScope) bindAll(t *template, inChoice bool, where string) error { if b, twice := sc.bindings[tok.bound]; twice { return fmt.Errorf("%stoken {%s}: name %q is bound twice %s, by {%s} too; rename one", where, tok.body, tok.bound, sc.spelled(), b.body) } - b := &nameBinding{name: tok.bound, ref: tok.names[0], body: tok.body, where: where, scope: sc, index: len(sc.order), binder: t} + b := &nameBinding{name: tok.bound, ref: tok.boundRef, body: tok.body, where: where, scope: sc, index: len(sc.order), binder: t} if sc.bindings == nil { sc.bindings = map[string]*nameBinding{} } @@ -114,15 +128,24 @@ func (sc *nameScope) spelled() string { return "in one repeat" } -// close refuses a binding nothing reads, and gives the owner of a scope binding names the scope. -func (sc *nameScope) close() error { +// settle refuses a binding read by nothing, or read once whole, outside a repeat, where its +// reference spells the same pick; and hands a scope binding names to its owner, which renders a +// frame of it. +func (sc *nameScope) settle() error { for _, b := range sc.order { - if !b.read { + if len(b.uses) == 0 { return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) } + if r := b.uses[0]; len(b.uses) == 1 && r.whole && !r.nested { + spelling := "{" + b.ref + "}" + if r.operand { + spelling = b.ref + } + return fmt.Errorf("%stoken {%s}: name %q is read once, so it keeps nothing; write %s where it is read, and drop the token", b.where, b.body, b.name, spelling) + } } if t, isTemplate := sc.owner.(*template); isTemplate && len(sc.order) > 0 { - t.ownScope = sc + t.ownNameScope = sc } return nil } @@ -143,9 +166,10 @@ func eachContained(n node, where string, fn func(c node, where string) error) er } // resolveReads answers each unbound read under n with a name its template sees, fields before the -// format as compile reports them, and refuses a field spelling a name. -func resolveReads(n node, where string) error { - if err := eachContained(n, where, resolveReads); err != nil { +// format as compile reports them, and refuses a field spelling a name. scopes are every scope of +// the category, which a refusal searches for a name bound where the read cannot see it. +func resolveReads(n node, where string, scopes []*nameScope) error { + if err := eachContained(n, where, func(c node, where string) error { return resolveReads(c, where, scopes) }); err != nil { return err } t, isTemplate := n.(*template) @@ -153,7 +177,7 @@ func resolveReads(n node, where string) error { return nil } for _, name := range sortedNames(t.fields) { - if b := t.scope.lookup(name); b != nil { + if b := t.nameScope.lookup(name); b != nil { holder := "the root template" if where != "" { holder = strings.TrimSuffix(where, ": ") @@ -162,32 +186,52 @@ func resolveReads(n node, where string) error { } } for _, u := range t.unbound { - b := t.scope.lookup(u.head) + b := t.nameScope.lookup(u.head) if b == nil { - return fmt.Errorf("%s%w", where, u.err) + return fmt.Errorf("%s%w%s", where, u.err, boundElsewhere(u.head, scopes)) } - b.read = true + b.uses = append(b.uses, nameUse{whole: u.whole, operand: u.operand, nested: t.nameScope != b.scope}) } return nil } -// isName reports whether a head t reads is a name: neither a reference nor a field. -func (t *template) isName(head string) bool { - return !isRef(head) && t.fields[head] == nil && t.scope.lookup(head) != nil +// boundElsewhere names the repeat binding name, for a read outside it. +func boundElsewhere(name string, scopes []*nameScope) string { + for _, sc := range scopes { + if b, ok := sc.bindings[name]; ok { + return fmt.Sprintf("; name %q is bound inside the repeat at %s, which a read outside the repeat cannot see", name, strings.TrimSuffix(b.where, ": ")) + } + } + return "" } -// linkNames compiles t's reads of a name as paths from the head its binding's reference names, once -// every template is linked, so the binder has resolved that reference. -func linkNames(path string, t *template) error { +// isName reports whether a head t reads is a name: neither a reference nor a field. +func (t *template) isName(head string) bool { + return !isRef(head) && t.fields[head] == nil && t.nameScope.lookup(head) != nil +} + +// linkBindings gives each binding t's tokens make the head and tail its reference resolved to. +func linkBindings(t *template) { + for _, tok := range t.tokens { + if tok.kind != nameBind { + continue + } + b := t.nameScope.bindings[tok.bound] + ref := t.link.refs[b.ref] + b.head, b.tail = t.link.refHeads[ref.head], ref.tail + } +} + +// namedReads calls fn with every read of a name t's format makes, and the token holding it. +func namedReads(t *template, fn func(o *op, a *arm) error) error { for i := range t.compiled.ops { o := &t.compiled.ops[i] for _, reads := range [][]arm{o.arms, o.operands} { for j := range reads { - if reads[j].kind != namedRead { - continue - } - if err := linkName(t, &reads[j]); err != nil { - return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), o.body, err) + if reads[j].kind == namedRead { + if err := fn(o, &reads[j]); err != nil { + return err + } } } } @@ -195,12 +239,19 @@ func linkNames(path string, t *template) error { return nil } +// linkNames compiles t's reads of a name as paths from the head its binding's reference names, once +// every template is linked, so each binder has resolved that reference. +func linkNames(path string, t *template) error { + return namedReads(t, func(o *op, a *arm) error { + if err := linkName(t, a); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), o.body, err) + } + return nil + }) +} + func linkName(t *template, a *arm) error { - b := t.scope.lookup(a.head) - if b.head == nil { - ref := b.binder.link.refs[b.ref] - b.head, b.tail = b.binder.link.refHeads[ref.head], ref.tail - } + b := t.nameScope.lookup(a.head) if hasSelector(a.tail) { return fmt.Errorf("a path through name %q selects no row; select it in the reference %q binds", a.head, a.head) } @@ -209,22 +260,96 @@ func linkName(t *template, a *arm) error { return err } w := compilePath(b.head, full) - a.bind, a.steps, a.leaves, a.cover = b, w.steps, w.leaves, w.cover + a.named, a.steps, a.leaves, a.cover = b, w.steps, w.leaves, w.cover a.levels = make([]string, len(full)+1) for i := range a.levels { a.levels[i] = joinSegments(full[:i]) } a.path = joinSegments(full) + if b.addressed == nil { + b.addressed = map[string]string{} + } + for _, key := range append(a.levels[len(b.tail):], a.path) { + if _, seen := b.addressed[key]; !seen { + b.addressed[key] = a.spelling + } + } return nil } -// namedPick is one draw of a name: the variant drawn at each level under it, the value each -// path through it read, keyed by the path from the name, and the table rows they pinned. A -// reference read under it draws in the draw group of the read of the name, which the draw -// fences survey it in. +// checkNameReads refuses a read of a name rendering, under the pick, a field it renders twice where +// a path through the name reads one draw of it: the pick keeps one draw there, and the two +// renders would disagree with it. +func checkNameReads(path string, t *template) error { + return namedReads(t, func(o *op, a *arm) error { + for _, leaf := range a.leaves { + if err := a.named.checkOnce(a.spelling, leaf, a.path); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), o.body, err) + } + } + return nil + }) +} + +// checkOnce walks n, rendering at key under b's pick, into each level a read of b addresses. +func (b *nameBinding) checkOnce(read string, n node, key string) error { + switch n := n.(type) { + case *choice: + for _, item := range n.items { + if err := b.checkOnce(read, item, key); err != nil { + return err + } + } + case *template: + return b.checkTemplateOnce(read, n, key) + } + return nil +} + +func (b *nameBinding) checkTemplateOnce(read string, t *template, key string) error { + fresh := map[string]int{} + var into []arm + for _, o := range t.compiled.ops { + for _, a := range append(o.arms[:len(o.arms):len(o.arms)], o.operands...) { + if isRef(a.head) || a.kind == namedRead { + continue + } + if _, kept := b.addressed[join(key, a.path)]; kept { + into = append(into, a) + } + if a.kind == freshRead { + fresh[a.head]++ + } + } + } + heads := make([]string, 0, len(fresh)) + for head := range fresh { + heads = append(heads, head) + } + sort.Strings(heads) + for _, head := range heads { + if by, kept := b.addressed[join(key, head)]; kept && fresh[head] > 1 { + return fmt.Errorf("{%s} renders field %q twice, so {%s} cannot say which draw it reads; drop one of them", read, head, by) + } + } + for _, a := range into { + for _, leaf := range a.leaves { + if err := b.checkOnce(read, leaf, join(key, a.path)); err != nil { + return err + } + } + } + return nil +} + +// namedPick is one draw of a name: the variant drawn at each level a read of it addresses, the +// value each read there produced, keyed by the path from the name, and the table rows they +// pinned. A reference read under it draws in the draw group of the read of the name, which the +// draw fences survey it in. type namedPick struct { - memo drawMemo - pins pinSet + named *nameBinding + memo drawMemo + pins pinSet } // pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. @@ -234,7 +359,11 @@ type pickFrame struct { } func newPickFrame(scope *nameScope) *pickFrame { - return &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))} + f := &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))} + for i, b := range scope.order { + f.picks[i].named = b + } + return f } // frameStack is the frames of the name scopes rendering, innermost last. A render's draws point @@ -244,79 +373,104 @@ type frameStack struct { frames []*pickFrame } -// pushFrame opens f until popFrame closes it, returning the mark popFrame takes. +// pushFrame opens f until popFrames closes it, returning the mark popFrames takes. func (d *renderDraws) pushFrame(f *pickFrame) int { mark := d.depth() - if d.frames == nil { - d.frames = &frameStack{} + if d.frameStack == nil { + d.frameStack = &frameStack{} } - d.frames.frames = append(d.frames.frames, f) + d.frameStack.frames = append(d.frameStack.frames, f) return mark } -func (d *renderDraws) popFrame(mark int) { d.frames.frames = d.frames.frames[:mark] } +func (d *renderDraws) popFrames(mark int) { + if d.frameStack != nil { + d.frameStack.frames = d.frameStack.frames[:mark] + } +} func (d *renderDraws) depth() int { - if d.frames == nil { + if d.frameStack == nil { return 0 } - return len(d.frames.frames) + return len(d.frameStack.frames) } -// entering is sc as a read entering another category sees it: none of the frames rendering, and -// memo, which keeps the frame of each scope the read enters past its owner. -func (sc renderScope) entering(memo *drawMemo) renderScope { - sc.base, sc.entry, sc.pick = sc.draws.depth(), memo, nil - return sc -} - -// frameOf is the frame a render of scope's owner opens: the one the read entering the category -// keeps, so {n} and {n.path} read one frame, else a fresh one. -func (sc renderScope) frameOf(scope *nameScope) *pickFrame { - if sc.entry == nil { - return newPickFrame(scope) +// enter is sc as a read entering a category sees it, landing on n: none of the frames rendering, +// and, from memo, the frame of each scope binding names around n, so every read entering with +// memo reads one pick of each name. It returns the mark that closes those frames. +func (sc renderScope) enter(n node, memo *drawMemo) (renderScope, int) { + sc.base, sc.pick = sc.draws.depth(), nil + for scope := scopeAround(n); scope != nil; scope = scope.up { + if len(scope.order) > 0 { + sc.draws.pushFrame(memo.enteredFrame(scope)) + } } - return sc.entry.frameOf(scope) + return sc, sc.base } -func (m *drawMemo) frameOf(scope *nameScope) *pickFrame { - f, ok := m.frames[scope] +// scopeAround is the innermost name scope a render of n reads names in, short of the frames n +// renders itself, one per iteration of a repeat. +func scopeAround(n node) *nameScope { + switch n := n.(type) { + case *template: + if n.repeat > 1 && n.nameScope != nil && n.nameScope.owner == n { + return n.nameScope.up + } + return n.nameScope + case *choice: + return n.nameScope + } + return nil +} + +func (m *drawMemo) enteredFrame(scope *nameScope) *pickFrame { + f, ok := m.enteredFrames[scope] if !ok { f = newPickFrame(scope) - if m.frames == nil { - m.frames = map[*nameScope]*pickFrame{} + if m.enteredFrames == nil { + m.enteredFrames = map[*nameScope]*pickFrame{} } - m.frames[scope] = f + m.enteredFrames[scope] = f } return f } -// pickOf is b's pick: in the frame rendering its scope since the read entering the category, else -// in the frame that read keeps, as for Fake("cat.field") or {/cat.field}. -func (sc renderScope) pickOf(b *nameBinding) *namedPick { - if stack := sc.draws.frames; stack != nil { +// renderFrame opens a fresh frame of t's scope, where t binds names and no read entering the +// category opened one, returning the mark that closes it, or -1. +func (sc renderScope) renderFrame(t *template) int { + if t.ownNameScope == nil || sc.frameOf(t.ownNameScope) != nil { + return -1 + } + return sc.draws.pushFrame(newPickFrame(t.ownNameScope)) +} + +// frameOf is the frame of scope rendering since the read entering the category, nil where none is. +func (sc renderScope) frameOf(scope *nameScope) *pickFrame { + if stack := sc.draws.frameStack; stack != nil { for i := len(stack.frames) - 1; i >= sc.base; i-- { - if f := stack.frames[i]; f.scope == b.scope { - return &f.picks[b.index] + if f := stack.frames[i]; f.scope == scope { + return f } } } - return &sc.entryMemo().frameOf(b.scope).picks[b.index] -} - -// entryMemo is the memo of the read that entered the category rendering, else the unnamed draw -// group's of the render: one memo for a render begun below a category's root, whatever draw group -// its templates name. -func (sc renderScope) entryMemo() *drawMemo { - if sc.entry != nil { - return sc.entry - } - return &sc.draws.unnamed.memo + return nil } func readName(s *generatorState, sc renderScope, a arm) readValue { - p := sc.pickOf(a.bind) - return readPicked(s, p, a.bind.head, a.steps, a.levels, a.path, sc.entering(&p.memo)) + f := sc.frameOf(a.named.scope) + if f == nil { + panic(internalError("name %q is read where no frame of its scope renders", a.named.name)) + } + p := &f.picks[a.named.index] + if r, done := p.memo.value[a.path]; done { + return r + } + leaf := p.draw(s, a.named.head, a.steps, a.levels, a.path) + sc, mark := sc.enter(leaf, &p.memo) + r := p.renderAt(s, leaf, a.path, sc) + sc.draws.popFrames(mark) + return r } // drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one. @@ -327,14 +481,25 @@ func (sc renderScope) drawRowOf(s *generatorState, t *table) int { return t.drawRow(s) } -// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey: once per pick, -// by its path from the name. +// keeps reports whether the pick sc renders part of keeps a, a read of t: whether a read of the +// name addresses the level a starts at. +func (sc renderScope) keeps(a arm) bool { + _, kept := sc.pick.named.addressed[underKey(sc.pickKey, a.head)] + return kept +} + +// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey, where a read of +// the name addresses it: once per pick, by its path from the name. func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue { + p, key := sc.pick, underKey(sc.pickKey, a.path) + if r, done := p.memo.value[key]; done { + return r + } levels := make([]string, len(a.levels)) for i, l := range a.levels { levels[i] = underKey(sc.pickKey, l) } - return readPicked(s, sc.pick, t.head(a.head), a.steps, levels, underKey(sc.pickKey, a.path), sc) + return p.renderAt(s, p.draw(s, t.head(a.head), a.steps, levels, key), key, sc) } // underKey is the key of path under the pick key prefix. It never returns prefix itself, which a @@ -346,15 +511,17 @@ func underKey(prefix, path string) string { return prefix + "." + path } -// readPicked draws the path key names under pick p, once, and renders what it lands on as part of p. -func readPicked(s *generatorState, p *namedPick, head node, steps []pathStep, levels []string, key string, sc renderScope) readValue { - if r, done := p.memo.value[key]; done { - return r - } +// draw draws the path key names under p, its variant at key kept too. +func (p *namedPick) draw(s *generatorState, head node, steps []pathStep, levels []string, key string) node { leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) if c, isChoice := leaf.(*choice); isChoice { leaf = p.memo.variantOf(s, c, key) } + return leaf +} + +// renderAt renders leaf, drawn at key, as part of p, and keeps what it rendered. +func (p *namedPick) renderAt(s *generatorState, leaf node, key string, sc renderScope) readValue { sc = sc.at(leaf, &p.pins) sc.pick, sc.pickKey = p, key r := renderLeaf(s, leaf, sc) diff --git a/node.go b/node.go index 6438566..4f91007 100644 --- a/node.go +++ b/node.go @@ -25,9 +25,10 @@ func (*folder) isNode() {} // relative dot paths every item can address, so carriedByAll and List both read the one // answer to what a path may reach through this choice. type choice struct { - items []node - cum []float64 - shared map[string]bool + items []node + cum []float64 + shared map[string]bool + nameScope *nameScope // where its items look a name up, set by bindNames } func (*choice) isNode() {} @@ -56,8 +57,8 @@ type template struct { unbound []unboundRead // the heads its tokens read that no field holds // Filled by `bindNames`, from the compiled category: - scope *nameScope // where its tokens look a name up - ownScope *nameScope // the scope it renders a frame of: a category's, on its root, or a repeat's, per iteration + nameScope *nameScope // where its tokens look a name up + ownNameScope *nameScope // the scope it renders a frame of: a category's, on its root, or a repeat's, per iteration // Filled by `linkTemplate`, from the assembled tree: link templateLink diff --git a/record.go b/record.go index ede1fe8..d8b2247 100644 --- a/record.go +++ b/record.go @@ -263,8 +263,8 @@ func recordOf(n node) (*template, []Column, error) { // over sc's draws; a table's columns read the row pinned there. func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) - if t.ownScope != nil { - defer sc.draws.popFrame(sc.draws.pushFrame(sc.frameOf(t.ownScope))) + if mark := sc.renderFrame(t); mark >= 0 { + defer sc.draws.popFrames(mark) } if t.site.isFormat() { sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins) diff --git a/reference.go b/reference.go index 93a064f..23bd9ab 100644 --- a/reference.go +++ b/reference.go @@ -78,7 +78,10 @@ func linkRefs(root map[string]node) error { }); err != nil { return err } - return eachTemplate(root, func(_ []string, path string, t *template) error { return linkNames(path, t) }) + if err := eachTemplate(root, func(_ []string, path string, t *template) error { return linkNames(path, t) }); err != nil { + return err + } + return eachTemplate(root, func(_ []string, path string, t *template) error { return checkNameReads(path, t) }) } // linkTemplate resolves t's references in category, "" for an inline template, and @@ -89,6 +92,7 @@ func linkTemplate(folder []string, path, category string, t *template, root map[ return err } t.link, t.compiled = link, compileFormat(t.tokens, link.refs, t.isName) + linkBindings(t) compileArms(t) return nil } @@ -253,12 +257,16 @@ func resolveCategory(root map[string]node, segments []string) (categorySegs []st return segments[:i], n, segments[i:], nil } -// refTokens returns the reference names a format reads, as tokens or as operands. +// refTokens returns the reference names a format reads, as tokens or as operands, or binds. func refTokens(toks []formatToken) []string { var refs []string seen := map[string]bool{} for _, tok := range toks { - for _, name := range tok.names { + names := tok.names + if tok.kind == nameBind { + names = []string{tok.boundRef} + } + for _, name := range names { if isRef(name) && !seen[name] { seen[name] = true refs = append(refs, name) diff --git a/render.go b/render.go index 78c6848..39eb8df 100644 --- a/render.go +++ b/render.go @@ -33,7 +33,9 @@ func (f *Generator) Fake(path string) (string, error) { if _, ok := n.(*folder); ok { return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) } - return render(f.rand, n, sc.at(n, &sc.groupDraws().pins)), nil + group := sc.groupDraws() + sc, _ = sc.at(n, &group.pins).enter(n, &group.memo) + return render(f.rand, n, sc), nil } // descend walks a caller's path to the node it names, pinning in sc the rows it @@ -78,8 +80,8 @@ func render(s *generatorState, n node, sc renderScope) string { case *template: sc = sc.in(n) if n.repeat == 1 { - if n.ownScope != nil { - defer sc.draws.popFrame(sc.draws.pushFrame(sc.frameOf(n.ownScope))) + if mark := sc.renderFrame(n); mark >= 0 { + defer sc.draws.popFrames(mark) } if lit, fixed := n.fixedText(); fixed { return lit @@ -93,30 +95,29 @@ func render(s *generatorState, n node, sc renderScope) string { } // renderRepeat renders each iteration of t as a render of its own, inside the name scopes -// rendering t, where a name bound outside t keeps its pick in the frame sc.entryMemo keeps. +// rendering t, where a name bound outside t keeps its pick. func renderRepeat(s *generatorState, t *template, sc renderScope) string { - entry := sc.entryMemo() var b strings.Builder b.Grow(t.repeat * (t.compiled.grow + len(t.separator))) for i := 0; i < t.repeat; i++ { if i > 0 { b.WriteString(t.separator) } - b.WriteString(expandAnew(s, t, sc.draws.frames, sc.base, entry)) + b.WriteString(expandAnew(s, t, sc.draws.frameStack, sc.base)) } return b.String() } // expandAnew expands one repeat iteration of t as a render of its own, in no group, inside the -// name scopes of frames from base. Inlined into render's loop, its draws would move to the heap. +// name scopes of stack from base. Inlined into render's loop, its draws would move to the heap. // //go:noinline -func expandAnew(s *generatorState, t *template, frames *frameStack, base int, entry *drawMemo) string { - draws := renderDraws{frames: frames} - if t.ownScope != nil { - defer draws.popFrame(draws.pushFrame(newPickFrame(t.ownScope))) +func expandAnew(s *generatorState, t *template, stack *frameStack, base int) string { + draws := renderDraws{frameStack: stack} + if t.ownNameScope != nil { + defer draws.popFrames(draws.pushFrame(newPickFrame(t.ownNameScope))) } - return expand(s, t, renderScope{draws: &draws, base: base, entry: entry}) + return expand(s, t, renderScope{draws: &draws, base: base}) } // pick selects one item. Uniform choices are O(1); weighted choices are an @@ -180,10 +181,14 @@ func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a switch { case a.kind == namedRead: return readName(s, sc, a) - case sc.pick != nil && !isRef(a.head): + case sc.pick != nil && !isRef(a.head) && sc.keeps(a): return readUnder(s, t, sc, a) + } + sc.pick = nil + switch { case a.kind == freshRead && isRef(a.head): - return readValue{text: render(s, t.head(a.head), sc.entering(nil))} + sc.base = sc.draws.depth() + return readValue{text: render(s, t.head(a.head), sc)} case a.kind == freshRead: return readValue{text: render(s, t.head(a.head), sc)} } @@ -205,10 +210,14 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r return r } leaf := drawSteps(s, t.head(a.head), a.steps, pins, memo, a.levels) + mark := -1 if pins != nil { - sc = sc.at(leaf, pins).entering(memo) + sc, mark = sc.at(leaf, pins).enter(leaf, memo) } r := renderLeaf(s, leaf, sc) + if mark >= 0 { + sc.draws.popFrames(mark) + } if memo.value == nil { memo.value = map[string]readValue{} } diff --git a/table.go b/table.go index 1375324..983765b 100644 --- a/table.go +++ b/table.go @@ -345,8 +345,8 @@ func (t *table) compileRowFormat(format string) error { if err != nil { return err } - if len(unbound) > 0 { - return unbound[0].err + if err := checkTableToks(nil, unbound); err != nil { + return err } t.formatTemplate = &template{format: format, tokens: toks, fields: fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} t.rowNode = &tableRow{t} diff --git a/todo.md b/todo.md index 50c92d8..27d41e5 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 72` +`Next ID = 76` | Goal | W | |---|---| @@ -55,7 +55,7 @@ | 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | -| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | +| 3 | 0.1.0 | decision | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | | 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 4 | 23.1 | | 57 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | @@ -88,6 +88,8 @@ | 32 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 4.1 | 14.4 | | 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | | 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | +| 73 | 0.2.0 | question | **Decide whether the default embed's partial Swedish and US place tables stand against goal 13.1, or have the goals say which goal wins.** | 1 | 1 | 3 | 6 | 13.1 | 9.6 | +| 74 | 0.2.0 | question | **Decide whether `misc.timezone` may drop the 17 zones of territories `misc.territory` leaves out, against goal 13.1, or have the goals say which goal wins.** | 1 | 1 | 3 | 6 | 13.1 | 9.6 | | 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | | 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | | 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | @@ -99,6 +101,7 @@ | 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 4 | 13.0 | | 35 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 8 | 13.0 | | 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 11 | 12.8 | +| 75 | 0.3.0 | | **Run every README `sh` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 5 | 4 | 7 | 11.2 | 11.8 | | 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 15 | 11.7 | | 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | | 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | @@ -106,6 +109,7 @@ | 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 33 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 5 | 10.8 | | 47 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 6 | 10.2 | +| 72 | 0.3.0 | principle | **Fail the merge gate when test coverage falls below the last recorded figure.** | 2 | 3 | 2 | 5 | 3 | 10.2 | | 7 | 0.3.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 3.2 | 9.8 | | 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 13.1 | 9.8 | | 37 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | @@ -189,16 +193,16 @@ Shape: T = table, t = template, c = choice. ### 3. Rewrite the shipped categories so facts that belong together come from one named pick. - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. -- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. +- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once and print it whole as `{a}`, so the country record's format appears once, and the decision "A locale's `address` restates its country record's format" is revised. - Needs item 2. Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 2, 3 and 71. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Needs items 2, 3 and 71. After it, `{/person} ({/person.first})` renders two people, as does `{/sv_SE.person as a}{/sv_SE.person as b}{a} & {b}`, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. -- Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. +- Rewrites the README's References, Draw group, Correlated fields, Linked tables, Names and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. - Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. ### 25. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. @@ -333,6 +337,14 @@ Require the path step to reach a sibling category. `readTableOptions` and `loadDir` return on the first in Go's map order. +### 73. Decide whether the default embed's partial Swedish and US place tables stand against goal 13.1, or have the goals say which goal wins. + +The decision "The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more" ships less than its registers hold, and names no item that ends it. Item 60 ships the full registers as packs, beside the partial default. + +### 74. Decide whether `misc.timezone` may drop the 17 zones of territories `misc.territory` leaves out, against goal 13.1, or have the goals say which goal wins. + +The decision "`misc.territory` is the spine, and a `misc` table naming a territory links to it" serves goal 4.1 and drops those tzdb rows, and names no item that ends it. + ### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. Both render the same run. @@ -369,6 +381,10 @@ Needs item 4; recount the switches then. Today ten, from `render` to `columnItem Nothing checks it, and a new country breaks it silently. +### 75. Run every README `sh` example as a test, and compare each "Renders" line to a seeded render. + +`readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables) and compares no "Renders e.g." output, so goal 11.2 is not met. + ### 23. Give every other shipped category that could reach something real a path that never does. `phone` draws live PTS and NANP ranges — PTS's five fiction series are the Swedish inert set, sourced in [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. @@ -389,6 +405,10 @@ The CLI counts brackets of its own, so a change to the selector grammar desyncs `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. +### 72. Fail the merge gate when test coverage falls below the last recorded figure. + +`technical-principles.md` holds that test coverage should not decline. `docker compose run --rm cover` prints the figure, and neither the `Dockerfile` gate nor `.github/workflows/test.yml` checks it. + ### 7. Fill `arm` in one place. `splitArm` runs before the link and again after it, and `compileArm` finishes it from `reference.go`, so five seats traced three phases. Needs item 4. diff --git a/transform.go b/transform.go index 64d4d11..17d7f4e 100644 --- a/transform.go +++ b/transform.go @@ -43,16 +43,11 @@ func unwrapTransform(arg string) (leaf string, chain []func(string) string, err arg = args[0] } } -func transformArg(fields map[string]node, a []string) error { - leaf, _, err := unwrapTransform(a[0]) - if err != nil { - return err - } - if isRef(leaf) { - _, _, err := refShape(leaf) - return err - } - return checkArm(leaf, fields, false) + +// transformArg proves a transform's chain; checkReads proves the operand it reads. +func transformArg(_ map[string]node, a []string) error { + _, _, err := unwrapTransform(a[0]) + return err } func transformOperand(a []string) []string { if len(a) != 1 { -- 2.52.0 From 05cf5313397005b0e28ab02e1cc52bba6bd76ff4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:05:36 +0200 Subject: [PATCH 437/966] Test a name read across draw groups or in a repeat, read once through a path, and misspelt --- name_test.go | 25 +++++++++++++------------ perf_test.go | 2 +- 2 files changed, 14 insertions(+), 13 deletions(-) diff --git a/name_test.go b/name_test.go index 5e176e9..856c42a 100644 --- a/name_test.go +++ b/name_test.go @@ -175,11 +175,11 @@ func TestNameErrors(t *testing.T) { {"bound twice", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{y}","x":"{/word as w}{w}","y":"{/word as w}"}`}, `name "w" is bound twice outside any repeat`}, {"bound inside a repeat written first", map[string]string{"word": `["a","b"]`, "card": `{"format":"{a}{z}","a":{"format":"{/word as w}{w}","repeat":2},"z":"{/word as w}{w}"}`}, - `name "w" is bound outside this repeat already`}, + `name "w" is bound outside this repeat too`}, {"bound in a choice's item", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":["{/word as w}","b"]}`}, `a choice's item binds no name`}, {"bound outside the repeat too", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":{"format":"{/word as w}{w}","repeat":2}}`}, - `name "w" is bound outside this repeat already`}, + `name "w" is bound outside this repeat too`}, {"a field too", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{w}","w":"x"}`}, `name "w" is a field of the root template too`}, {"an option", map[string]string{"word": `["a","b"]`, "card": `"{/word as format}{format}"`}, @@ -188,6 +188,14 @@ func TestNameErrors(t *testing.T) { `"word" is no reference`}, {"read by nothing", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}"`}, `nothing reads name "w"`}, + {"read once through a path", map[string]string{"word": `{"format":"{w}","w":["a","b"]}`, "card": `"{/word as n}{n.w}"`}, + `write {/word.w}`}, + {"read in two draw groups", map[string]string{"nm.json": `{"format":"{first}","rows":"nm.tsv"}`, "nm.tsv": "first\tlast\nAda\tByron\nBo\tEk\n", "person": `{"format":"{first} {last}","first":"{/nm.first}","last":"{/nm.last}"}`, "card": `{"format":"{b}|{a}","a":"{/person as p}{p}","b":{"format":"{p.first}","drawGroup":"g"}}`}, + `name "p" is read in two draw groups`}, + {"read in a repeat", map[string]string{"nm.json": `{"format":"{first}","rows":"nm.tsv"}`, "nm.tsv": "first\tlast\nAda\tByron\nBo\tEk\n", "person": `{"format":"{first} {last}","first":"{/nm.first}","last":"{/nm.last}"}`, "card": `{"format":"{/person as p}{p}{each}","each":{"format":"{p.first}","repeat":2}}`}, + `name "p" is read inside a repeat`}, + {"a misspelt name", map[string]string{"word": `["a","b"]`, "card": `"{/word as p}{p}{p}{pp}"`}, + `no field or name "pp"; the names bound here are [p]`}, {"read once whole", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":"{w}"}`}, `write {/word}`}, {"rendered twice where a path reads it", map[string]string{"word": `{"format":"{w}-{w}","w":["a","b"]}`, "card": `"{/word as n}{n}|{n.w}"`}, @@ -201,13 +209,11 @@ func TestNameErrors(t *testing.T) { {"in a column's name", map[string]string{"t.json": `{"format":"{k}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\ta as b\nx\t1\ny\t2\n"}, `contains " as "`}, {"read outside the repeat binding it", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":{"format":"{/word as w}{w}","repeat":2}}`}, - `name "w" is bound inside the repeat at field "x"`}, - {"unbound", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":{"format":"{/word as w}{w}","repeat":2}}`}, - `no field "w"`}, + `no field "w"; name "w" is bound inside the repeat at field "x"`}, {"padded", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}{w}"`}, `write {/word as w}`}, - {"selector after a name", map[string]string{"region.json": nameTables()["region.json"], "card": `"{/region as r}{r.municipality[0180].name}"`}, - `a path through name "r" selects no row`}, + {"selector after a name", map[string]string{"region.json": nameTables()["region.json"], "region.tsv": nameTables()["region.tsv"], "municipality.json": nameTables()["municipality.json"], "municipality.tsv": nameTables()["municipality.tsv"], "card": `"{/region as r}{r.municipality[0180].name}"`}, + `read it directly, {/region.municipality[0180].name}`}, {"in a table's format", map[string]string{"word": `["a","b"]`, "t.json": `{"format":"{/word as w}{w}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\nx\ny\n"}, "a table binds no name"}, {"in a field's name", map[string]string{"card": `{"format":"{a as b}","a as b":"x"}`}, @@ -222,11 +228,6 @@ func TestNameErrors(t *testing.T) { } files[name] = body } - if c.name == "selector after a name" { - for k, v := range nameTables() { - files[k] = v - } - } _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, files))) if err == nil || !strings.Contains(err.Error(), c.want) { t.Fatalf("New() = %v, want an error holding %q", err, c.want) diff --git a/perf_test.go b/perf_test.go index 92cc738..178ffc7 100644 --- a/perf_test.go +++ b/perf_test.go @@ -64,7 +64,7 @@ func TestNoReferenceAllocRegression(t *testing.T) { }{ {"a repeat of a reference path", `{"format":"{r}","r":{"format":"{/word.w}","repeat":20,"separator":", "}}`, 66}, {"a named draw group", `{"format":"{a}","a":{"format":"{/word.w}","drawGroup":"g"}}`, 11}, - {"a repeat of a name", `{"format":"{r}","r":{"format":"{/word as n}{n.w}","repeat":20,"separator":", "}}`, 205}, + {"a repeat of a name", `{"format":"{r}","r":{"format":"{/word as n}{n.w}{n.w}","repeat":20,"separator":", "}}`, 205}, } { f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{ "word.json": {Data: []byte(word)}, -- 2.52.0 From 740f2bc3bd713f831e77a35b002d663abab9002a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:08:40 +0200 Subject: [PATCH 438/966] Refuse a name read across draw groups or in a repeat where it reads a reference path, or read once; name the spelling in each refusal --- CHANGELOG.md | 7 +-- README.md | 26 ++++++---- docs/decisions.md | 9 ++-- format.go | 13 +++-- name.go | 128 +++++++++++++++++++++++++++++++++++++--------- name_test.go | 2 +- 6 files changed, 138 insertions(+), 47 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ae3f359..56596d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,9 +15,10 @@ replacement, and each removed path, column or flag. `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a name that is a field or an option; a binding of anything but a reference; a binding in a - choice's item; a binding nothing reads, or one read once whole, which the reference - spells; a path through a name that selects a row; a read of a name rendering a field - twice where a path through the name reads it; and a binding in a table. A field, column, + choice's item; a binding nothing reads, or one read once, which the reference spells; a + path through a name that selects a row; a read of a name rendering a field twice where a + path through the name reads it; reads of a name in two draw groups, or inside a + `repeat`, where what it names reads a reference path; and a binding in a table. A field, column, category or folder name may not contain ` as `. - First release: the CLI, the library and the shipped data set. - Table categories: a category JSON naming a `rows` TSV beside it, with the options diff --git a/README.md b/README.md index 10db0ff..049ba02 100644 --- a/README.md +++ b/README.md @@ -711,8 +711,8 @@ row: Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. A name keeps each level a `{n.path}` reads, and what it lands on, so `{n}` and `{n.path}` agree, and a [transform](#transforms) of `n.path` reads the same value. Everything else under the name draws -as it would anywhere: `{w}-{w}` in the category `n` names draws `w` twice, so `{n.w}` beside -`{n}` is refused there, since it could read either draw. +as it would anywhere. So if the category `n` names has the format `{w}-{w}`, it draws `w` +twice, and `{n.w}` beside `{n}` is refused, since it could read either draw. A name is a pick of its own: `{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path the picked category reads is still held for the render as usual, so two names @@ -724,13 +724,21 @@ category reads it, and a record's columns read one pick. Each render of the cate anew, and so does each bare reference to it. A name a `repeat` binds picks again on every iteration, and one bound outside the repeat keeps its pick on every line. -Rejected at load: a name bound twice outside any `repeat`, twice in one `repeat`, or both -inside a `repeat` and outside it; a name that is a field or an option; a binding of anything -but a reference; a binding in a choice's item, which every other item would leave unbound; a -binding nothing reads, or one read once whole outside a `repeat`, which the reference spells; -a path through a name that selects a row, which the reference selects instead; a read of a name -rendering a field twice where a path through the name reads it; and a binding in a table's -format or cell. A field, column, category or folder name may not contain ` as `. +Rejected at load, each error naming what to write instead: + +- a name bound twice outside any `repeat`, twice in one `repeat`, or both inside a `repeat` + and outside it; +- a name that is a field or an option; +- a binding of anything but a reference; +- a binding in a choice's item, which every other item would leave unbound; +- a binding nothing reads, or read once outside a `repeat`, which the reference spells: + `{/word as w}{w}` is `{/word}`, and `{/word as w}{w.x}` is `{/word.x}`; +- a path through a name that selects a row; +- a read of a name rendering a field twice, where a path through the name reads it; +- reads of a name in two draw groups, or inside a `repeat`, where what it names reads a + reference path, which each draw group and iteration draws apart; +- a binding in a table's format or cell; +- a field, column, category or folder name containing ` as `. ### Draw group diff --git a/docs/decisions.md b/docs/decisions.md index 286ea38..e9ff0d6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -893,7 +893,7 @@ a struct's fields as one. ## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read -2026-10-03, larv-review. Goals 5.3 and 5.6. +2026-10-03, larv-review in PR #145, for Lilleman auf Larv. Goals 5.3 and 5.6. `bindNames` scopes a name at compile, and a render keeps one `pickFrame` per scope. One field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside `"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category @@ -913,5 +913,8 @@ nowhere else. A pick keeps only the levels a read of the name addresses, so a template draws the same way under a name as anywhere else (goal 5.2). A reference read under a pick draws in the draw group of the read of the name, the group the draw fences survey it in, so two names bound to one -category share what it reads by reference path. `todo.md` item 4 ends that: once every `{…}` -draws afresh, nothing is held for the render. +category share what it reads by reference path. Where what a name names reads a reference +path, its reads stay in one draw group and outside any repeat: a reference path is held per +draw group and per iteration, so what the pick kept and what it renders afresh would come from +two draws of it. `todo.md` item 4 ends both: once every `{…}` draws afresh, nothing is held for +the render. diff --git a/format.go b/format.go index 8bf18e6..c175ce0 100644 --- a/format.go +++ b/format.go @@ -215,13 +215,12 @@ func parseChecked(format string, fields map[string]node) ([]formatToken, []unbou } // unboundRead is a token reading a head no field holds, which only a name bound around the -// template can answer; err is the refusal where none does. whole is a read of the name itself, -// no path into it, and operand a builtin's read of it. +// template can answer; err is the refusal where none does. tail is the path read into the name, +// "" for the name itself, operand marks a builtin's read of it, and body is the token's. type unboundRead struct { - head string - whole bool - operand bool - err error + head, tail, body string + operand bool + err error } // checkTokens proves every parsed token names an existing field or a known function, @@ -280,7 +279,7 @@ func checkReads(t formatToken, fields map[string]node, operands bool) ([]unbound return nil, err } a := splitArm(name, nil) - unbound = append(unbound, unboundRead{head: a.head, whole: len(a.tail) == 0, operand: operands, err: err}) + unbound = append(unbound, unboundRead{head: a.head, tail: joinSegments(a.tail), body: t.body, operand: operands, err: err}) } return unbound, nil } diff --git a/name.go b/name.go index 03f7cc8..e07d119 100644 --- a/name.go +++ b/name.go @@ -35,10 +35,12 @@ type nameBinding struct { addressed map[string]string } -// nameUse is one read of a name: whole, with no path into it, an operand of a builtin, or in a -// repeat nested inside the name's scope. +// nameUse is one read of a name: the path it reads into the name, "" for the name itself; whether +// a builtin reads it as an operand; the draw group its template renders in; and whether it sits in +// a repeat nested inside the name's scope. type nameUse struct { - whole, operand, nested bool + tail, group string + operand, nested bool } func (sc *nameScope) lookup(name string) *nameBinding { @@ -82,11 +84,11 @@ func bindNames(root node) error { for _, sc := range scopes[1:] { for _, b := range sc.order { if outer := sc.up.lookup(b.name); outer != nil { - return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat already, by {%s}; rename one", b.where, b.body, b.name, outer.body) + return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat too, by {%s}; rename one", b.where, b.body, b.name, outer.body) } } } - if err := resolveReads(root, "", scopes); err != nil { + if err := resolveReads(root, "", "", scopes); err != nil { return err } for _, sc := range scopes { @@ -105,7 +107,7 @@ func (sc *nameScope) bindAll(t *template, inChoice bool, where string) error { continue } if inChoice { - return fmt.Errorf("%stoken {%s}: a choice's item binds no name, since every other item would leave it unbound; bind it outside the choice", where, tok.body) + return fmt.Errorf("%stoken {%s}: a choice's item binds no name, since every other item would leave it unbound; bind it outside the choice, or move the item into a category of its own and reference that", where, tok.body) } if b, twice := sc.bindings[tok.bound]; twice { return fmt.Errorf("%stoken {%s}: name %q is bound twice %s, by {%s} too; rename one", where, tok.body, tok.bound, sc.spelled(), b.body) @@ -122,7 +124,7 @@ func (sc *nameScope) bindAll(t *template, inChoice bool, where string) error { // spelled names the scope in an error. func (sc *nameScope) spelled() string { - if sc.up == nil { + if t, isTemplate := sc.owner.(*template); sc.up == nil && !(isTemplate && t.repeat > 1) { return "outside any repeat" } return "in one repeat" @@ -136,10 +138,13 @@ func (sc *nameScope) settle() error { if len(b.uses) == 0 { return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) } - if r := b.uses[0]; len(b.uses) == 1 && r.whole && !r.nested { - spelling := "{" + b.ref + "}" - if r.operand { - spelling = b.ref + if r := b.uses[0]; len(b.uses) == 1 && !r.nested { + spelling := b.ref + if r.tail != "" { + spelling += "." + r.tail + } + if !r.operand { + spelling = "{" + spelling + "}" } return fmt.Errorf("%stoken {%s}: name %q is read once, so it keeps nothing; write %s where it is read, and drop the token", b.where, b.body, b.name, spelling) } @@ -168,11 +173,14 @@ func eachContained(n node, where string, fn func(c node, where string) error) er // resolveReads answers each unbound read under n with a name its template sees, fields before the // format as compile reports them, and refuses a field spelling a name. scopes are every scope of // the category, which a refusal searches for a name bound where the read cannot see it. -func resolveReads(n node, where string, scopes []*nameScope) error { - if err := eachContained(n, where, func(c node, where string) error { return resolveReads(c, where, scopes) }); err != nil { +func resolveReads(n node, where, group string, scopes []*nameScope) error { + t, isTemplate := n.(*template) + if isTemplate && t.drawGroup != "" { + group = t.drawGroup + } + if err := eachContained(n, where, func(c node, where string) error { return resolveReads(c, where, group, scopes) }); err != nil { return err } - t, isTemplate := n.(*template) if !isTemplate { return nil } @@ -188,21 +196,32 @@ func resolveReads(n node, where string, scopes []*nameScope) error { for _, u := range t.unbound { b := t.nameScope.lookup(u.head) if b == nil { - return fmt.Errorf("%s%w%s", where, u.err, boundElsewhere(u.head, scopes)) + return unresolved(where, u, t.nameScope, scopes) } - b.uses = append(b.uses, nameUse{whole: u.whole, operand: u.operand, nested: t.nameScope != b.scope}) + b.uses = append(b.uses, nameUse{tail: u.tail, group: group, operand: u.operand, nested: t.nameScope != b.scope}) } return nil } -// boundElsewhere names the repeat binding name, for a read outside it. -func boundElsewhere(name string, scopes []*nameScope) string { - for _, sc := range scopes { - if b, ok := sc.bindings[name]; ok { - return fmt.Sprintf("; name %q is bound inside the repeat at %s, which a read outside the repeat cannot see", name, strings.TrimSuffix(b.where, ": ")) +// unresolved is the refusal of u, a read no field or name answers: naming the names the read sees, +// else the repeat binding the name where the read cannot see it. +func unresolved(where string, u unboundRead, seen *nameScope, scopes []*nameScope) error { + var visible []string + for sc := seen; sc != nil; sc = sc.up { + for _, b := range sc.order { + visible = append(visible, b.name) } } - return "" + if len(visible) > 0 { + sort.Strings(visible) + return fmt.Errorf("%stoken {%s}: no field or name %q; the names bound here are %v", where, u.body, u.head, visible) + } + for _, sc := range scopes { + if b, ok := sc.bindings[u.head]; ok { + return fmt.Errorf("%s%w; name %q is bound inside the repeat at %s, which a read outside the repeat cannot see; bind it outside the repeat to read one pick on every line", where, u.err, u.head, strings.TrimSuffix(b.where, ": ")) + } + } + return fmt.Errorf("%s%w", where, u.err) } // isName reports whether a head t reads is a name: neither a reference nor a field. @@ -253,7 +272,7 @@ func linkNames(path string, t *template) error { func linkName(t *template, a *arm) error { b := t.nameScope.lookup(a.head) if hasSelector(a.tail) { - return fmt.Errorf("a path through name %q selects no row; select it in the reference %q binds", a.head, a.head) + return fmt.Errorf("a path through name %q selects no row; read it directly, {%s.%s}, or bind the row to a name of its own", a.head, b.ref, joinSegments(a.tail)) } full := append(b.tail[:len(b.tail):len(b.tail)], a.tail...) if err := checkPathResolves(b.head, full, a.head); err != nil { @@ -281,6 +300,14 @@ func linkName(t *template, a *arm) error { // a path through the name reads one draw of it: the pick keeps one draw there, and the two // renders would disagree with it. func checkNameReads(path string, t *template) error { + for _, tok := range t.tokens { + if tok.kind != nameBind { + continue + } + if err := t.nameScope.bindings[tok.bound].checkUses(); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), tok.body, err) + } + } return namedReads(t, func(o *op, a *arm) error { for _, leaf := range a.leaves { if err := a.named.checkOnce(a.spelling, leaf, a.path); err != nil { @@ -291,6 +318,59 @@ func checkNameReads(path string, t *template) error { }) } +// checkUses refuses reads of b in two draw groups, or inside a repeat, where what b names reads a +// reference path: that path is held per draw group and per iteration, so what the pick kept and +// what it renders afresh would come from two draws of it. todo.md item 4 ends the hold. +func (b *nameBinding) checkUses() error { + if !readsHeld(b.head, map[node]bool{}) { + return nil + } + for _, u := range b.uses { + if u.nested { + return fmt.Errorf("name %q is read inside a repeat, and what it names reads a reference path, which each iteration draws apart; bind the name inside the repeat", b.name) + } + if u.group != b.uses[0].group { + return fmt.Errorf("name %q is read in two draw groups, %s and %s, and what it names reads a reference path, which each draw group draws apart; read the name in one draw group", b.name, groupSpelling(b.uses[0].group), groupSpelling(u.group)) + } + } + return nil +} + +func groupSpelling(group string) string { + if group == "" { + return "the unnamed one" + } + return fmt.Sprintf("%q", group) +} + +// readsHeld reports whether rendering n, or any field under it, can read a reference path. +func readsHeld(n node, seen map[node]bool) bool { + if seen[n] { + return false + } + seen[n] = true + if t, isTemplate := n.(*template); isTemplate { + for _, o := range t.compiled.ops { + for _, a := range append(o.arms[:len(o.arms):len(o.arms)], o.operands...) { + if a.kind == refPathRead { + return true + } + } + } + } + for _, c := range contained(n) { + if readsHeld(c.node, seen) { + return true + } + } + for _, e := range renderEdges(n) { + if readsHeld(e.to, seen) { + return true + } + } + return false +} + // checkOnce walks n, rendering at key under b's pick, into each level a read of b addresses. func (b *nameBinding) checkOnce(read string, n node, key string) error { switch n := n.(type) { @@ -329,7 +409,7 @@ func (b *nameBinding) checkTemplateOnce(read string, t *template, key string) er sort.Strings(heads) for _, head := range heads { if by, kept := b.addressed[join(key, head)]; kept && fresh[head] > 1 { - return fmt.Errorf("{%s} renders field %q twice, so {%s} cannot say which draw it reads; drop one of them", read, head, by) + return fmt.Errorf("{%s} renders field %q twice, so {%s} cannot say which draw it reads; drop {%s} or {%s}", read, head, by, read, by) } } for _, a := range into { diff --git a/name_test.go b/name_test.go index 856c42a..a2c6089 100644 --- a/name_test.go +++ b/name_test.go @@ -212,7 +212,7 @@ func TestNameErrors(t *testing.T) { `no field "w"; name "w" is bound inside the repeat at field "x"`}, {"padded", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}{w}"`}, `write {/word as w}`}, - {"selector after a name", map[string]string{"region.json": nameTables()["region.json"], "region.tsv": nameTables()["region.tsv"], "municipality.json": nameTables()["municipality.json"], "municipality.tsv": nameTables()["municipality.tsv"], "card": `"{/region as r}{r.municipality[0180].name}"`}, + {"selector after a name", map[string]string{"region.json": nameTables()["region.json"], "region.tsv": nameTables()["region.tsv"], "municipality.json": nameTables()["municipality.json"], "municipality.tsv": nameTables()["municipality.tsv"], "card": `"{/region as r}{r.name} {r.municipality[0180].name}"`}, `read it directly, {/region.municipality[0180].name}`}, {"in a table's format", map[string]string{"word": `["a","b"]`, "t.json": `{"format":"{/word as w}{w}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\nx\ny\n"}, "a table binds no name"}, -- 2.52.0 From c5468d613f90a23f3822edbf58f2f6c52aa4fc83 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:14:04 +0200 Subject: [PATCH 439/966] Test that a name read once through a path loads, and an empty name is refused --- name_test.go | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/name_test.go b/name_test.go index a2c6089..302fbf4 100644 --- a/name_test.go +++ b/name_test.go @@ -188,14 +188,16 @@ func TestNameErrors(t *testing.T) { `"word" is no reference`}, {"read by nothing", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}"`}, `nothing reads name "w"`}, - {"read once through a path", map[string]string{"word": `{"format":"{w}","w":["a","b"]}`, "card": `"{/word as n}{n.w}"`}, - `write {/word.w}`}, {"read in two draw groups", map[string]string{"nm.json": `{"format":"{first}","rows":"nm.tsv"}`, "nm.tsv": "first\tlast\nAda\tByron\nBo\tEk\n", "person": `{"format":"{first} {last}","first":"{/nm.first}","last":"{/nm.last}"}`, "card": `{"format":"{b}|{a}","a":"{/person as p}{p}","b":{"format":"{p.first}","drawGroup":"g"}}`}, `name "p" is read in two draw groups`}, {"read in a repeat", map[string]string{"nm.json": `{"format":"{first}","rows":"nm.tsv"}`, "nm.tsv": "first\tlast\nAda\tByron\nBo\tEk\n", "person": `{"format":"{first} {last}","first":"{/nm.first}","last":"{/nm.last}"}`, "card": `{"format":"{/person as p}{p}{each}","each":{"format":"{p.first}","repeat":2}}`}, `name "p" is read inside a repeat`}, {"a misspelt name", map[string]string{"word": `["a","b"]`, "card": `"{/word as p}{p}{p}{pp}"`}, `no field or name "pp"; the names bound here are [p]`}, + {"read once through a path beside its reference path", map[string]string{"word": `{"format":"{w}","w":["a","b"]}`, "card": `"{/word as a}{a.w} {/word.w}"`}, + ""}, + {"an empty name", map[string]string{"word": `["a","b"]`, "card": `"{/word as }"`}, + "a binding names nothing"}, {"read once whole", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":"{w}"}`}, `write {/word}`}, {"rendered twice where a path reads it", map[string]string{"word": `{"format":"{w}-{w}","w":["a","b"]}`, "card": `"{/word as n}{n}|{n.w}"`}, @@ -229,6 +231,12 @@ func TestNameErrors(t *testing.T) { files[name] = body } _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, files))) + if c.want == "" { + if err != nil { + t.Fatalf("New() = %v, want it to load", err) + } + return + } if err == nil || !strings.Contains(err.Error(), c.want) { t.Fatalf("New() = %v, want an error holding %q", err, c.want) } -- 2.52.0 From 0b0370936a2798187b0cdc7b6760e420df0246ce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:14:58 +0200 Subject: [PATCH 440/966] Refuse a name read once only when read whole, and name every way out in the scope and draw-group refusals --- CHANGELOG.md | 2 +- README.md | 4 ++-- format.go | 3 +++ name.go | 29 +++++++++++++++-------------- 4 files changed, 21 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 56596d4..d7bc4b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,7 +15,7 @@ replacement, and each removed path, column or flag. `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a name that is a field or an option; a binding of anything but a reference; a binding in a - choice's item; a binding nothing reads, or one read once, which the reference spells; a + choice's item; a binding nothing reads, or one read once whole, which the reference spells; a path through a name that selects a row; a read of a name rendering a field twice where a path through the name reads it; reads of a name in two draw groups, or inside a `repeat`, where what it names reads a reference path; and a binding in a table. A field, column, diff --git a/README.md b/README.md index 049ba02..3af9d2b 100644 --- a/README.md +++ b/README.md @@ -731,8 +731,8 @@ Rejected at load, each error naming what to write instead: - a name that is a field or an option; - a binding of anything but a reference; - a binding in a choice's item, which every other item would leave unbound; -- a binding nothing reads, or read once outside a `repeat`, which the reference spells: - `{/word as w}{w}` is `{/word}`, and `{/word as w}{w.x}` is `{/word.x}`; +- a binding nothing reads, or read once whole outside a `repeat`, which the reference + spells: `{/word as w}{w}` is `{/word}`; - a path through a name that selects a row; - a read of a name rendering a field twice, where a path through the name reads it; - reads of a name in two draw groups, or inside a `repeat`, where what it names reads a diff --git a/format.go b/format.go index c175ce0..e457a5c 100644 --- a/format.go +++ b/format.go @@ -297,6 +297,9 @@ func checkBind(t formatToken) error { if _, _, err := refShape(ref); err != nil { return fmt.Errorf("token {%s}: %w", t.body, err) } + if name == "" { + return fmt.Errorf("token {%s}: a binding names nothing; write {%s as n}", t.body, ref) + } if isOption(name) { return fmt.Errorf("token {%s}: %q is an option and can never be a name", t.body, name) } diff --git a/name.go b/name.go index e07d119..fcba65e 100644 --- a/name.go +++ b/name.go @@ -138,11 +138,8 @@ func (sc *nameScope) settle() error { if len(b.uses) == 0 { return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) } - if r := b.uses[0]; len(b.uses) == 1 && !r.nested { + if r := b.uses[0]; len(b.uses) == 1 && r.tail == "" && !r.nested { spelling := b.ref - if r.tail != "" { - spelling += "." + r.tail - } if !r.operand { spelling = "{" + spelling + "}" } @@ -203,9 +200,18 @@ func resolveReads(n node, where, group string, scopes []*nameScope) error { return nil } -// unresolved is the refusal of u, a read no field or name answers: naming the names the read sees, -// else the repeat binding the name where the read cannot see it. +// unresolved is the refusal of u, a read no field or name answers: naming the repeat binding the +// name where the read cannot see it, else the names the read sees. func unresolved(where string, u unboundRead, seen *nameScope, scopes []*nameScope) error { + for _, sc := range scopes { + if b, ok := sc.bindings[u.head]; ok { + at := strings.TrimSuffix(b.where, ": ") + if at == "" { + at = "the root template" + } + return fmt.Errorf("%s%w; name %q is bound inside the repeat at %s, which a read outside the repeat cannot see; bind it outside the repeat to read one pick on every line", where, u.err, u.head, at) + } + } var visible []string for sc := seen; sc != nil; sc = sc.up { for _, b := range sc.order { @@ -216,11 +222,6 @@ func unresolved(where string, u unboundRead, seen *nameScope, scopes []*nameScop sort.Strings(visible) return fmt.Errorf("%stoken {%s}: no field or name %q; the names bound here are %v", where, u.body, u.head, visible) } - for _, sc := range scopes { - if b, ok := sc.bindings[u.head]; ok { - return fmt.Errorf("%s%w; name %q is bound inside the repeat at %s, which a read outside the repeat cannot see; bind it outside the repeat to read one pick on every line", where, u.err, u.head, strings.TrimSuffix(b.where, ": ")) - } - } return fmt.Errorf("%s%w", where, u.err) } @@ -272,7 +273,7 @@ func linkNames(path string, t *template) error { func linkName(t *template, a *arm) error { b := t.nameScope.lookup(a.head) if hasSelector(a.tail) { - return fmt.Errorf("a path through name %q selects no row; read it directly, {%s.%s}, or bind the row to a name of its own", a.head, b.ref, joinSegments(a.tail)) + return fmt.Errorf("a path through name %q may not select a row; read it directly, {%s.%s}, or bind the row to a name of its own", a.head, b.ref, joinSegments(a.tail)) } full := append(b.tail[:len(b.tail):len(b.tail)], a.tail...) if err := checkPathResolves(b.head, full, a.head); err != nil { @@ -327,10 +328,10 @@ func (b *nameBinding) checkUses() error { } for _, u := range b.uses { if u.nested { - return fmt.Errorf("name %q is read inside a repeat, and what it names reads a reference path, which each iteration draws apart; bind the name inside the repeat", b.name) + return fmt.Errorf("name %q is read inside a repeat, and what it names reads a reference path, which each iteration draws apart; bind the name inside the repeat, or bind a second name there", b.name) } if u.group != b.uses[0].group { - return fmt.Errorf("name %q is read in two draw groups, %s and %s, and what it names reads a reference path, which each draw group draws apart; read the name in one draw group", b.name, groupSpelling(b.uses[0].group), groupSpelling(u.group)) + return fmt.Errorf("name %q is read in two draw groups, %s and %s, and what it names reads a reference path, which each draw group draws apart; read the name in one draw group, or bind a name in each", b.name, groupSpelling(b.uses[0].group), groupSpelling(u.group)) } } return nil -- 2.52.0 From dbc2b7e7ebb45bd6b474cbb4f518ce9335bce095 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:17:33 +0200 Subject: [PATCH 441/966] Test a name bound to a reference path read once whole, and where an unseen name is bound --- name_test.go | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/name_test.go b/name_test.go index 302fbf4..d8799b0 100644 --- a/name_test.go +++ b/name_test.go @@ -193,9 +193,13 @@ func TestNameErrors(t *testing.T) { {"read in a repeat", map[string]string{"nm.json": `{"format":"{first}","rows":"nm.tsv"}`, "nm.tsv": "first\tlast\nAda\tByron\nBo\tEk\n", "person": `{"format":"{first} {last}","first":"{/nm.first}","last":"{/nm.last}"}`, "card": `{"format":"{/person as p}{p}{each}","each":{"format":"{p.first}","repeat":2}}`}, `name "p" is read inside a repeat`}, {"a misspelt name", map[string]string{"word": `["a","b"]`, "card": `"{/word as p}{p}{p}{pp}"`}, - `no field or name "pp"; the names bound here are [p]`}, + `no field or name "pp"; the names bound here are "p"`}, {"read once through a path beside its reference path", map[string]string{"word": `{"format":"{w}","w":["a","b"]}`, "card": `"{/word as a}{a.w} {/word.w}"`}, ""}, + {"read once whole, bound to a reference path", map[string]string{"word": `{"format":"{w}","w":["a","b"]}`, "card": `"{/word.w as c}{c} {/word.w}"`}, + ""}, + {"read outside a repeat in the root choice", map[string]string{"word": `["a","b"]`, "card": `[{"format":"{/word as p}{p}{p}","repeat":2},"{p}"]`}, + `name "p" is bound at an item of the root choice, inside a repeat`}, {"an empty name", map[string]string{"word": `["a","b"]`, "card": `"{/word as }"`}, "a binding names nothing"}, {"read once whole", map[string]string{"word": `["a","b"]`, "card": `{"format":"{/word as w}{x}","x":"{w}"}`}, @@ -211,7 +215,7 @@ func TestNameErrors(t *testing.T) { {"in a column's name", map[string]string{"t.json": `{"format":"{k}","rows":"t.tsv","key":"k"}`, "t.tsv": "k\ta as b\nx\t1\ny\t2\n"}, `contains " as "`}, {"read outside the repeat binding it", map[string]string{"word": `["a","b"]`, "card": `{"format":"{x}{w}","x":{"format":"{/word as w}{w}","repeat":2}}`}, - `no field "w"; name "w" is bound inside the repeat at field "x"`}, + `no field "w"; name "w" is bound at field "x", inside a repeat`}, {"padded", map[string]string{"word": `["a","b"]`, "card": `"{/word as w}{w}"`}, `write {/word as w}`}, {"selector after a name", map[string]string{"region.json": nameTables()["region.json"], "region.tsv": nameTables()["region.tsv"], "municipality.json": nameTables()["municipality.json"], "municipality.tsv": nameTables()["municipality.tsv"], "card": `"{/region as r}{r.name} {r.municipality[0180].name}"`}, -- 2.52.0 From 32124c5b1ebf95d0085aa7c7e0c98bf76ab81f6b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:18:32 +0200 Subject: [PATCH 442/966] Refuse a read-once name only where it binds a whole category, and say where an unseen name is bound --- CHANGELOG.md | 2 +- README.md | 4 ++-- name.go | 32 ++++++++++++++++++-------------- 3 files changed, 21 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d7bc4b5..afe8dfd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,7 +15,7 @@ replacement, and each removed path, column or flag. `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a name that is a field or an option; a binding of anything but a reference; a binding in a - choice's item; a binding nothing reads, or one read once whole, which the reference spells; a + choice's item; a binding nothing reads, or one of a whole category read once whole, which the bare reference spells; a path through a name that selects a row; a read of a name rendering a field twice where a path through the name reads it; reads of a name in two draw groups, or inside a `repeat`, where what it names reads a reference path; and a binding in a table. A field, column, diff --git a/README.md b/README.md index 3af9d2b..c279450 100644 --- a/README.md +++ b/README.md @@ -731,8 +731,8 @@ Rejected at load, each error naming what to write instead: - a name that is a field or an option; - a binding of anything but a reference; - a binding in a choice's item, which every other item would leave unbound; -- a binding nothing reads, or read once whole outside a `repeat`, which the reference - spells: `{/word as w}{w}` is `{/word}`; +- a binding nothing reads, or a binding of a whole category read once whole where it is + bound, which the bare reference spells: `{/word as w}{w}` is `{/word}`; - a path through a name that selects a row; - a read of a name rendering a field twice, where a path through the name reads it; - reads of a name in two draw groups, or inside a `repeat`, where what it names reads a diff --git a/name.go b/name.go index fcba65e..3248c19 100644 --- a/name.go +++ b/name.go @@ -3,6 +3,7 @@ package fejkdata import ( "fmt" "sort" + "strconv" "strings" ) @@ -130,21 +131,13 @@ func (sc *nameScope) spelled() string { return "in one repeat" } -// settle refuses a binding read by nothing, or read once whole, outside a repeat, where its -// reference spells the same pick; and hands a scope binding names to its owner, which renders a -// frame of it. +// settle refuses a binding read by nothing, and hands a scope binding names to its owner, which +// renders a frame of it. func (sc *nameScope) settle() error { for _, b := range sc.order { if len(b.uses) == 0 { return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) } - if r := b.uses[0]; len(b.uses) == 1 && r.tail == "" && !r.nested { - spelling := b.ref - if !r.operand { - spelling = "{" + spelling + "}" - } - return fmt.Errorf("%stoken {%s}: name %q is read once, so it keeps nothing; write %s where it is read, and drop the token", b.where, b.body, b.name, spelling) - } } if t, isTemplate := sc.owner.(*template); isTemplate && len(sc.order) > 0 { t.ownNameScope = sc @@ -207,9 +200,9 @@ func unresolved(where string, u unboundRead, seen *nameScope, scopes []*nameScop if b, ok := sc.bindings[u.head]; ok { at := strings.TrimSuffix(b.where, ": ") if at == "" { - at = "the root template" + at = "an item of the root choice" } - return fmt.Errorf("%s%w; name %q is bound inside the repeat at %s, which a read outside the repeat cannot see; bind it outside the repeat to read one pick on every line", where, u.err, u.head, at) + return fmt.Errorf("%s%w; name %q is bound at %s, inside a repeat, which a read outside the repeat cannot see; bind it outside the repeat to read one pick on every line", where, u.err, u.head, at) } } var visible []string @@ -220,7 +213,10 @@ func unresolved(where string, u unboundRead, seen *nameScope, scopes []*nameScop } if len(visible) > 0 { sort.Strings(visible) - return fmt.Errorf("%stoken {%s}: no field or name %q; the names bound here are %v", where, u.body, u.head, visible) + for i, name := range visible { + visible[i] = strconv.Quote(name) + } + return fmt.Errorf("%stoken {%s}: no field or name %q; the names bound here are %s", where, u.body, u.head, strings.Join(visible, ", ")) } return fmt.Errorf("%s%w", where, u.err) } @@ -319,10 +315,18 @@ func checkNameReads(path string, t *template) error { }) } -// checkUses refuses reads of b in two draw groups, or inside a repeat, where what b names reads a +// checkUses refuses a binding of a category read once whole, which the bare reference spells, and +// reads of b in two draw groups, or inside a repeat, where what b names reads a // reference path: that path is held per draw group and per iteration, so what the pick kept and // what it renders afresh would come from two draws of it. todo.md item 4 ends the hold. func (b *nameBinding) checkUses() error { + if r := b.uses[0]; len(b.uses) == 1 && r.tail == "" && !r.nested && len(b.tail) == 0 { + spelling := b.ref + if !r.operand { + spelling = "{" + spelling + "}" + } + return fmt.Errorf("name %q is read once, whole, which the bare reference draws the same way; write %s where it is read, and drop the token", b.name, spelling) + } if !readsHeld(b.head, map[node]bool{}) { return nil } -- 2.52.0 From 62214370ed9b7e71eb1a4cc66bfcac3fe005fbc4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:23:21 +0200 Subject: [PATCH 443/966] Make the names prose true of the code and readable on one pass, and state each refusal once --- CHANGELOG.md | 16 ++++------------ README.md | 18 +++++++++--------- doc.go | 11 ++++++----- docs/decisions.md | 17 +++++++++-------- draws.go | 8 ++++---- format.go | 14 ++++++-------- name.go | 30 +++++++++++------------------- record.go | 2 +- render.go | 4 ++-- todo.md | 6 +++--- 10 files changed, 55 insertions(+), 71 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index afe8dfd..420e1a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,18 +8,10 @@ replacement, and each removed path, column or flag. ### Added -- A name keeps one pick: `{/misc.territory as t}` binds one row of `misc.territory` to `t` - and prints nothing, and `{t}`, `{t.capital}` and `{t.calling-code}` read that one row. - A name lives in the category binding it, so every field and record column of the - category reads one pick, a transform reads it as `{lowercase(t.name)}`, and a name a - `repeat` binds picks again on every iteration. Refused at `New`: a name bound twice - outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; a - name that is a field or an option; a binding of anything but a reference; a binding in a - choice's item; a binding nothing reads, or one of a whole category read once whole, which the bare reference spells; a - path through a name that selects a row; a read of a name rendering a field twice where a - path through the name reads it; reads of a name in two draw groups, or inside a - `repeat`, where what it names reads a reference path; and a binding in a table. A field, column, - category or folder name may not contain ` as `. +- Names: `{/misc.territory as t}` binds one row of `misc.territory` to `t` and prints + nothing; `{t}`, `{t.capital}` and `{lowercase(t.name)}` read that one row. A name lives + in the category binding it, and a `repeat` binding it picks again on every iteration. + What `New` refuses is listed under [Names](README.md#names). - First release: the CLI, the library and the shipped data set. - Table categories: a category JSON naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and `parent`; a path selects a row by key or name, diff --git a/README.md b/README.md index c279450..1f0e5bc 100644 --- a/README.md +++ b/README.md @@ -708,23 +708,23 @@ row: "{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}" ``` -Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. A name keeps each level a -`{n.path}` reads, and what it lands on, so `{n}` and `{n.path}` agree, and a -[transform](#transforms) of `n.path` reads the same value. Everything else under the name draws -as it would anywhere. So if the category `n` names has the format `{w}-{w}`, it draws `w` -twice, and `{n.w}` beside `{n}` is refused, since it could read either draw. +Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. A name keeps what each `{n.path}` +lands on and every level it passes, so `{n}` and `{n.path}` agree, and so does a +[transform](#transforms) of `n.path`. Everything else under the name draws as it would +anywhere: if `n`'s category has the format `{w}-{w}`, it draws `w` twice, and `{n.w}` beside +`{n}` is refused, since it could read either draw. A name is a pick of its own: `{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path the picked category reads is still held for the render as usual, so two names -bound to `sv_SE.person`, whose fields read `{.first-name.name}`, print one person unless their -reads sit in two [draw groups](#draw-group). +bound to `sv_SE.person`, whose fields read `{.first-name.name}`, print the same first and last +name unless their reads sit in two [draw groups](#draw-group). A name is drawn on its first read, and lives in the category binding it: any field of the category reads it, and a record's columns read one pick. Each render of the category picks anew, and so does each bare reference to it. A name a `repeat` binds picks again on every iteration, and one bound outside the repeat keeps its pick on every line. -Rejected at load, each error naming what to write instead: +Refused at `New`, each error naming what to write instead: - a name bound twice outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; @@ -734,7 +734,7 @@ Rejected at load, each error naming what to write instead: - a binding nothing reads, or a binding of a whole category read once whole where it is bound, which the bare reference spells: `{/word as w}{w}` is `{/word}`; - a path through a name that selects a row; -- a read of a name rendering a field twice, where a path through the name reads it; +- `{n}` beside `{n.w}` where `n`'s category renders `w` twice, as in `{w}-{w}`; - reads of a name in two draw groups, or inside a `repeat`, where what it names reads a reference path, which each draw group and iteration draws apart; - a binding in a table's format or cell; diff --git a/doc.go b/doc.go index 1461d4b..176183e 100644 --- a/doc.go +++ b/doc.go @@ -27,11 +27,12 @@ package fejkdata // an expansion holds a sibling path's head and an operand's field, its draw // group a reference path's: `formatOps.held` and `heldCheck` an expansion's, // `readReference` a draw group's. -// - named pick — the draw a token such as {/person as p} binds to a name. It is drawn on its -// first read and kept while its scope renders: a category, or one repeat iteration. -// `nameBinding`, `nameScope`, `namedPick`, `pickFrame`. -// - memo — what a hold, a draw group or a named pick keeps its draws in, the variant drawn at -// each level and the value each path read: `drawMemo`, `readMemo`. +// - named pick — the draw a token such as {/person as p} binds to a name, drawn +// on its first read and kept while its scope renders: a category, or one +// repeat iteration: `nameBinding`, `nameScope`, `namedPick`, `pickFrame`. +// - memo — what a hold, a draw group or a named pick keeps its draws in, the +// variant drawn at each level and the value each path read: `drawMemo`, +// `readMemo`. // - draw group — reference draws kept apart inside one render: // `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a // render reads it, `renderDraws`, `groupDraws`, `renderScope`. diff --git a/docs/decisions.md b/docs/decisions.md index e9ff0d6..7572ab8 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -662,7 +662,8 @@ same four columns as `geo.SE.address`, each a reference into it, and the format twice; a column is spelled the same in both, `street-number`, so the two never disagree on a name. -`todo.md` item 3 replaces it. +`todo.md` item 3 revises this decision: the locale `address` prints `{a}` whole, so the +format appears once. ## A postort's kommun comes from its name, its tätort or its codes, never from distance @@ -902,13 +903,11 @@ since a read outside the item would see a binding another item leaves out. A pic its first read, so a record's columns, rendered in name order, read one pick whichever binds it. -The scope is lexical: a read entering a category, by a reference, through a name or from -`Fake`, sees none of the frames rendering above it, so what a category renders never depends -on who references it. That read opens the frame of each scope around what it lands on, taken -from its own memo, so every read entering by one memo, `{n}` and `{n.path}` alike, reads one -pick: per render and group for a reference path, per pick for a path through a name, per -render for `Fake("cat.field")`. A frame is looked up on the stack of frames rendering, and -nowhere else. +The scope is lexical. A read entering a category, by a reference, through a name or from +`Fake`, sees no frame (one render of a scope's picks) opened above it, so a category renders +the same whoever references it. That read opens the frames around where it lands from its own +memo, so reads sharing a memo, `{n}` and `{n.path}` alike, read one pick. A reference path's +memo lasts per render and draw group, a name's per pick, and `Fake("cat.field")`'s per render. A pick keeps only the levels a read of the name addresses, so a template draws the same way under a name as anywhere else (goal 5.2). A reference read under a pick draws in the draw group @@ -918,3 +917,5 @@ path, its reads stay in one draw group and outside any repeat: a reference path draw group and per iteration, so what the pick kept and what it renders afresh would come from two draws of it. `todo.md` item 4 ends both: once every `{…}` draws afresh, nothing is held for the render. + +Valid while names are read only inside the category binding them. diff --git a/draws.go b/draws.go index e0aa063..bfa6c2b 100644 --- a/draws.go +++ b/draws.go @@ -4,8 +4,8 @@ import "strings" // drawMemo is what a hold, a draw group or a named pick has drawn: the variant each level was // drawn as, so every path under it reads one variant; the value each read produced, by its path, -// so the same read written twice reads one value; and the frame of each name scope a read -// entered past its owner. +// so the same read written twice reads one value; and the frames a read opens for the name +// scopes around where it lands (enter). type drawMemo struct { variant map[string]node value map[string]readValue @@ -48,8 +48,8 @@ type renderScope struct { // base is the depth of the frame stack when the read rendering the category entered it, // below which no frame is the category's. base int - // pick is the named pick what renders is part of, at pickKey, its path from the name; nil - // outside one. + // pick is the named pick being rendered, nil outside one; pickKey is the path from the name + // to what renders. pick *namedPick pickKey string } diff --git a/format.go b/format.go index e457a5c..c1459c1 100644 --- a/format.go +++ b/format.go @@ -77,7 +77,6 @@ const ( nameBind ) -// asWord separates a binding's reference from the name it binds: {/person as p}. const asWord = " as " // parseFormat is the one reading of a format's tokens: a '(' outside a selector makes @@ -223,9 +222,8 @@ type unboundRead struct { err error } -// checkTokens proves every parsed token names an existing field or a known function, -// so a typo'd or dangling reference is a New-time error. A head no field holds comes -// back unbound, for bindNames to look up among the names. +// checkTokens proves every token's grammar, functions and field paths, and returns the reads +// whose head no field holds, for bindNames to look up among the names. func checkTokens(toks []formatToken, fields map[string]node) ([]unboundRead, error) { var unbound []unboundRead for _, t := range toks { @@ -284,8 +282,8 @@ func checkReads(t formatToken, fields map[string]node, operands bool) ([]unbound return unbound, nil } -// checkBind proves a {ref as name} token binds a reference to a name no other token -// could read as something else. +// checkBind proves a {ref as name} token is spelled once, binds a reference, and names a valid +// name that is no option. func checkBind(t formatToken) error { ref, name := t.boundRef, t.bound if trimmed := strings.TrimSpace(ref) + asWord + strings.TrimSpace(name); trimmed != t.body { @@ -301,7 +299,7 @@ func checkBind(t formatToken) error { return fmt.Errorf("token {%s}: a binding names nothing; write {%s as n}", t.body, ref) } if isOption(name) { - return fmt.Errorf("token {%s}: %q is an option and can never be a name", t.body, name) + return fmt.Errorf("token {%s}: %q is an option and can never be a name; rename it", t.body, name) } if err := checkName(name); err != nil { return fmt.Errorf("token {%s}: name %w", t.body, err) @@ -534,7 +532,7 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding, isName fu c.ops = append(c.ops, op{formatToken: tok, arms: arms}) } -// compileFormat compiles a parsed format, a binding to nothing, since it prints nothing. Call +// compileFormat compiles a parsed format; a {ref as n} token compiles to no op. Call // checkTokens first: it is what proves every token valid. func compileFormat(toks []formatToken, refs map[string]refBinding, isName func(head string) bool) formatOps { var c formatOps diff --git a/name.go b/name.go index 3248c19..d213c08 100644 --- a/name.go +++ b/name.go @@ -7,9 +7,9 @@ import ( "strings" ) -// nameScope is where a name is bound: a category, or one iteration of a repeat inside it. -// Every token of the category outside a nested repeat reads the category's names, so one -// field may bind a name and another read it. +// nameScope is where a name is bound: a category, or one iteration of a repeat inside it. A +// token sees its scope's names and every enclosing scope's, so one field may bind a name and +// another read it. type nameScope struct { up *nameScope owner node // the category's root, or the repeat @@ -17,9 +17,6 @@ type nameScope struct { order []*nameBinding } -// nameBinding is one {ref as name} token: the scope it binds name in, its index among -// that scope's bindings, its uses, and, once linked, the head node and tail its -// reference reads and the keys its reads address. type nameBinding struct { name string ref string @@ -56,7 +53,7 @@ func (sc *nameScope) lookup(name string) *nameBinding { // bindNames gives every template of a compiled category or inline template the scope its // names live in, and refuses a name bound twice along one chain of scopes, a binding in a // choice's item, a name a field spells too, a read no field or name answers, and a binding -// read by nothing, or once whole, which its reference spells. +// read by nothing. func bindNames(root node) error { top := &nameScope{owner: root} scopes := []*nameScope{top} @@ -100,8 +97,7 @@ func bindNames(root node) error { return nil } -// bindAll binds every name t's tokens bind. A choice's item binds none: a read outside the item -// would find the name bound by an item that may never be drawn. +// bindAll binds every name t's tokens bind. func (sc *nameScope) bindAll(t *template, inChoice bool, where string) error { for _, tok := range t.tokens { if tok.kind != nameBind { @@ -221,7 +217,6 @@ func unresolved(where string, u unboundRead, seen *nameScope, scopes []*nameScop return fmt.Errorf("%s%w", where, u.err) } -// isName reports whether a head t reads is a name: neither a reference nor a field. func (t *template) isName(head string) bool { return !isRef(head) && t.fields[head] == nil && t.nameScope.lookup(head) != nil } @@ -315,10 +310,8 @@ func checkNameReads(path string, t *template) error { }) } -// checkUses refuses a binding of a category read once whole, which the bare reference spells, and -// reads of b in two draw groups, or inside a repeat, where what b names reads a -// reference path: that path is held per draw group and per iteration, so what the pick kept and -// what it renders afresh would come from two draws of it. todo.md item 4 ends the hold. +// checkUses refuses a binding of a category read once whole, and reads of b in two draw groups, +// or inside a repeat, where what b names reads a reference path. func (b *nameBinding) checkUses() error { if r := b.uses[0]; len(b.uses) == 1 && r.tail == "" && !r.nested && len(b.tail) == 0 { spelling := b.ref @@ -481,9 +474,9 @@ func (d *renderDraws) depth() int { return len(d.frameStack.frames) } -// enter is sc as a read entering a category sees it, landing on n: none of the frames rendering, -// and, from memo, the frame of each scope binding names around n, so every read entering with -// memo reads one pick of each name. It returns the mark that closes those frames. +// enter hides the frames already rendering from a read landing on n, and opens from memo a frame +// for each name scope around n, so every read sharing memo reads one pick of each name. It +// returns the mark that closes those frames. func (sc renderScope) enter(n node, memo *drawMemo) (renderScope, int) { sc.base, sc.pick = sc.draws.depth(), nil for scope := scopeAround(n); scope != nil; scope = scope.up { @@ -566,8 +559,7 @@ func (sc renderScope) drawRowOf(s *generatorState, t *table) int { return t.drawRow(s) } -// keeps reports whether the pick sc renders part of keeps a, a read of t: whether a read of the -// name addresses the level a starts at. +// keeps reports whether a read of sc's name addresses the level a starts at. func (sc renderScope) keeps(a arm) bool { _, kept := sc.pick.named.addressed[underKey(sc.pickKey, a.head)] return kept diff --git a/record.go b/record.go index d8b2247..5d5db99 100644 --- a/record.go +++ b/record.go @@ -22,7 +22,7 @@ type Column struct { // Record is one record rendered from a template: every direct field is a column, // listed in name order. Each column is its own expansion, so a sibling field is // local to it, while a reference that reads a path is drawn once for the whole -// record, per group, and a name once for the whole record. +// record, per group, and a pick bound by {ref as n} once for the whole record. type Record struct { columns []Column } diff --git a/render.go b/render.go index 39eb8df..b921b40 100644 --- a/render.go +++ b/render.go @@ -109,7 +109,7 @@ func renderRepeat(s *generatorState, t *template, sc renderScope) string { } // expandAnew expands one repeat iteration of t as a render of its own, in no group, inside the -// name scopes of stack from base. Inlined into render's loop, its draws would move to the heap. +// name scopes of stack from base. Inlined into renderRepeat's loop, its draws would move to the heap. // //go:noinline func expandAnew(s *generatorState, t *template, stack *frameStack, base int) string { @@ -169,7 +169,7 @@ func expand(s *generatorState, t *template, sc renderScope) string { // token addresses by a dotted path that is not a reference, or a field an operand // reads — is drawn once and kept in hold, so {place.postal-code} and {place.locality} // read one row, either read twice gives one value, and a shown operand is the operand -// computed. A field of a template rendering as part of a named pick is kept in the pick. +// computed. A field a read of a name addresses is kept in that name's pick. // Every other field is drawn afresh, so {word} {word} still draws twice. func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue { if a.kind == refPathRead { diff --git a/todo.md b/todo.md index 27d41e5..919a074 100644 --- a/todo.md +++ b/todo.md @@ -193,13 +193,13 @@ Shape: T = table, t = template, c = choice. ### 3. Rewrite the shipped categories so facts that belong together come from one named pick. - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. -- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once and print it whole as `{a}`, so the country record's format appears once, and the decision "A locale's `address` restates its country record's format" is revised. +- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. They print it whole as `{a}`, so the country record's format appears once; revise the decision "A locale's `address` restates its country record's format". - Needs item 2. Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 2, 3 and 71. After it, `{/person} ({/person.first})` renders two people, as does `{/sv_SE.person as a}{/sv_SE.person as b}{a} & {b}`, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Needs items 2, 3 and 71. After it, `{/person} ({/person.first})` renders two people, as does `{/sv_SE.person as a}{/sv_SE.person as b}{a} & {b}`. `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables, Names and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. @@ -339,7 +339,7 @@ Require the path step to reach a sibling category. ### 73. Decide whether the default embed's partial Swedish and US place tables stand against goal 13.1, or have the goals say which goal wins. -The decision "The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more" ships less than its registers hold, and names no item that ends it. Item 60 ships the full registers as packs, beside the partial default. +The decision "The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more" serves goal 2's built-in data with no download, ships less than its registers hold, and names no item that ends it. Item 60 ships the full registers as packs, beside the partial default. ### 74. Decide whether `misc.timezone` may drop the 17 zones of territories `misc.territory` leaves out, against goal 13.1, or have the goals say which goal wins. -- 2.52.0 From c955d35bbe04c096ea2c3f7323fb1b473e5242a0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:25:35 +0200 Subject: [PATCH 444/966] Make checkNameReads and enter read on one pass, and keep name for names --- README.md | 4 ++-- doc.go | 2 +- name.go | 9 ++++----- render.go | 2 +- 4 files changed, 8 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 1f0e5bc..8f80dfd 100644 --- a/README.md +++ b/README.md @@ -716,11 +716,11 @@ anywhere: if `n`'s category has the format `{w}-{w}`, it draws `w` twice, and `{ A name is a pick of its own: `{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path the picked category reads is still held for the render as usual, so two names -bound to `sv_SE.person`, whose fields read `{.first-name.name}`, print the same first and last +bound to `sv_SE.person`, whose fields read `{.first-name.name}` and `{.last-name.name}`, print the same first and last name unless their reads sit in two [draw groups](#draw-group). A name is drawn on its first read, and lives in the category binding it: any field of the -category reads it, and a record's columns read one pick. Each render of the category picks +category may read it, and a record's columns read one pick. Each render of the category picks anew, and so does each bare reference to it. A name a `repeat` binds picks again on every iteration, and one bound outside the repeat keeps its pick on every line. diff --git a/doc.go b/doc.go index 176183e..1b608ed 100644 --- a/doc.go +++ b/doc.go @@ -23,7 +23,7 @@ package fejkdata // `Generator.Fake`, `FakeRecord`, `Template.Fake` and `RecordTemplate.Fake` // start one each, `FakeStruct` one per record, and `expandAnew` one per // iteration of a `template.repeat`. -// - hold — keeping one draw of a name, so every route to it reads that value: +// - hold — keeping one draw of a field, so every route to it reads that value: // an expansion holds a sibling path's head and an operand's field, its draw // group a reference path's: `formatOps.held` and `heldCheck` an expansion's, // `readReference` a draw group's. diff --git a/name.go b/name.go index d213c08..1228790 100644 --- a/name.go +++ b/name.go @@ -288,9 +288,8 @@ func linkName(t *template, a *arm) error { return nil } -// checkNameReads refuses a read of a name rendering, under the pick, a field it renders twice where -// a path through the name reads one draw of it: the pick keeps one draw there, and the two -// renders would disagree with it. +// checkNameReads refuses each binding of t that checkUses refuses, and a read such as {n} whose +// pick renders twice a field another read, {n.w}, reads once: the pick keeps one draw of it. func checkNameReads(path string, t *template) error { for _, tok := range t.tokens { if tok.kind != nameBind { @@ -474,8 +473,8 @@ func (d *renderDraws) depth() int { return len(d.frameStack.frames) } -// enter hides the frames already rendering from a read landing on n, and opens from memo a frame -// for each name scope around n, so every read sharing memo reads one pick of each name. It +// enter starts a read landing on n: the read sees no frame opened before it, and gets from memo a +// frame for each name scope around n, so reads sharing memo read one pick of each name. It // returns the mark that closes those frames. func (sc renderScope) enter(n node, memo *drawMemo) (renderScope, int) { sc.base, sc.pick = sc.draws.depth(), nil diff --git a/render.go b/render.go index b921b40..855afcf 100644 --- a/render.go +++ b/render.go @@ -165,7 +165,7 @@ func expand(s *generatorState, t *template, sc renderScope) string { return b.String() } -// readField renders one arm of a token. A name the expansion holds — a level some +// readField renders one arm of a token. A field the expansion holds — a level some // token addresses by a dotted path that is not a reference, or a field an operand // reads — is drawn once and kept in hold, so {place.postal-code} and {place.locality} // read one row, either read twice gives one value, and a shown operand is the operand -- 2.52.0 From 8c3ee104f6fd7e641341141ef4aa7f2b0cc9bafc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:25:41 +0200 Subject: [PATCH 445/966] Rewrap the Names paragraph --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8f80dfd..c34c82d 100644 --- a/README.md +++ b/README.md @@ -716,8 +716,8 @@ anywhere: if `n`'s category has the format `{w}-{w}`, it draws `w` twice, and `{ A name is a pick of its own: `{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path the picked category reads is still held for the render as usual, so two names -bound to `sv_SE.person`, whose fields read `{.first-name.name}` and `{.last-name.name}`, print the same first and last -name unless their reads sit in two [draw groups](#draw-group). +bound to `sv_SE.person`, whose fields read `{.first-name.name}` and `{.last-name.name}`, print +the same first and last name unless their reads sit in two [draw groups](#draw-group). A name is drawn on its first read, and lives in the category binding it: any field of the category may read it, and a record's columns read one pick. Each render of the category picks -- 2.52.0 From 2bb795e08c1a12d075d7db6aab1fc9de468ce727 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:16:57 +0200 Subject: [PATCH 446/966] Split the picks and the name fences out of name.go, state the repeat refusal and the naming rule where they hold, and draw a memo's variants one way --- README.md | 22 +-- docs/decisions.md | 7 +- name.go | 333 ---------------------------------------------- namefence.go | 138 +++++++++++++++++++ node.go | 6 +- path.go | 11 +- pick.go | 209 +++++++++++++++++++++++++++++ render.go | 3 +- table.go | 14 +- todo.md | 1 + 10 files changed, 375 insertions(+), 369 deletions(-) create mode 100644 namefence.go create mode 100644 pick.go diff --git a/README.md b/README.md index c34c82d..be03692 100644 --- a/README.md +++ b/README.md @@ -156,8 +156,8 @@ work with no data on disk. A directory is a namespace: each JSON file is a category named after the file, each subdirectory a dot-path segment, so `mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one. Sources merge in order; matching folders combine, any other clash is won by the -last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`, nor be -`-`, which a struct tag reserves; dot-prefixed entries are skipped, so a data directory can also be a checkout. +last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"`, `/` or ` as `, +which binds a [name](#names), nor be `-`, which a struct tag reserves; dot-prefixed entries are skipped, so a data directory can also be a checkout. Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, @@ -735,10 +735,10 @@ Refused at `New`, each error naming what to write instead: bound, which the bare reference spells: `{/word as w}{w}` is `{/word}`; - a path through a name that selects a row; - `{n}` beside `{n.w}` where `n`'s category renders `w` twice, as in `{w}-{w}`; -- reads of a name in two draw groups, or inside a `repeat`, where what it names reads a - reference path, which each draw group and iteration draws apart; -- a binding in a table's format or cell; -- a field, column, category or folder name containing ` as `. +- reads of a name in two draw groups, or inside a `repeat` nested where the name is bound, + where what it names reads a reference path, which each draw group and iteration draws + apart; +- a binding in a table's format or cell. ### Draw group @@ -816,8 +816,8 @@ The target is about a microsecond per value (goal 12). Each file is parsed, validated and weight-indexed once, in `New`, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends through it. Proving the draw fences adds a pass over the loaded tree per -fence, and walks what a render reads only where data binds a reference, so a set that -binds none pays for the passes alone. A `Fake` call then costs about what its output +fence, and walks what a render reads only where data holds a reference, so a set that +holds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one O(log n), and long formats, deep nesting and many tokens add cost in proportion to the output. @@ -933,7 +933,9 @@ doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws -name.go names: {ref as n} bindings, the scopes they live in, and the picks a render keeps under them +name.go names: {ref as n} bindings, the scopes they live in, and the reads they answer +namefence.go the load fences over names: reads in two draw groups or a nested repeat, and a field rendered twice under a pick +pick.go a render's named picks: the frames of the name scopes rendering, and the reads kept under a name path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make record.go records: Record, the JSON/CSV/SQL serializers, and their entry points @@ -945,7 +947,7 @@ holdfence.go the load fences over the hold: the routes and spellings that rea drawfence.go the load fences over reference draws and their refusals: a drawGroup's, a render's and a record's columns judged on what they read, and a table reaching its own family reads.go the load-time fold of what a render reads, and what a path reads of a table family pins.go the pin set: the table rows a render fixes, which the draw fences replay -reference.go reference sigils, and binding references across the tree +reference.go reference sigils, and resolving references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences builtins.go the {name()} function registry and its implementations layout.go date and time layouts: the instants one is proved against, and the two samples diff --git a/docs/decisions.md b/docs/decisions.md index 7572ab8..b97fed0 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -905,15 +905,16 @@ it. The scope is lexical. A read entering a category, by a reference, through a name or from `Fake`, sees no frame (one render of a scope's picks) opened above it, so a category renders -the same whoever references it. That read opens the frames around where it lands from its own -memo, so reads sharing a memo, `{n}` and `{n.path}` alike, read one pick. A reference path's +the same whoever references it. A reference path, a read through a name or `Fake` opens the +frames around where it lands from its own memo, so reads sharing a memo, `{n}` and `{n.path}` +alike, read one pick; a bare reference lands on a category's root, which opens fresh ones. A reference path's memo lasts per render and draw group, a name's per pick, and `Fake("cat.field")`'s per render. A pick keeps only the levels a read of the name addresses, so a template draws the same way under a name as anywhere else (goal 5.2). A reference read under a pick draws in the draw group of the read of the name, the group the draw fences survey it in, so two names bound to one category share what it reads by reference path. Where what a name names reads a reference -path, its reads stay in one draw group and outside any repeat: a reference path is held per +path, its reads stay in one draw group and outside any repeat nested where it is bound: a reference path is held per draw group and per iteration, so what the pick kept and what it renders afresh would come from two draws of it. `todo.md` item 4 ends both: once every `{…}` draws afresh, nothing is held for the render. diff --git a/name.go b/name.go index 1228790..5c6ae05 100644 --- a/name.go +++ b/name.go @@ -287,336 +287,3 @@ func linkName(t *template, a *arm) error { } return nil } - -// checkNameReads refuses each binding of t that checkUses refuses, and a read such as {n} whose -// pick renders twice a field another read, {n.w}, reads once: the pick keeps one draw of it. -func checkNameReads(path string, t *template) error { - for _, tok := range t.tokens { - if tok.kind != nameBind { - continue - } - if err := t.nameScope.bindings[tok.bound].checkUses(); err != nil { - return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), tok.body, err) - } - } - return namedReads(t, func(o *op, a *arm) error { - for _, leaf := range a.leaves { - if err := a.named.checkOnce(a.spelling, leaf, a.path); err != nil { - return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), o.body, err) - } - } - return nil - }) -} - -// checkUses refuses a binding of a category read once whole, and reads of b in two draw groups, -// or inside a repeat, where what b names reads a reference path. -func (b *nameBinding) checkUses() error { - if r := b.uses[0]; len(b.uses) == 1 && r.tail == "" && !r.nested && len(b.tail) == 0 { - spelling := b.ref - if !r.operand { - spelling = "{" + spelling + "}" - } - return fmt.Errorf("name %q is read once, whole, which the bare reference draws the same way; write %s where it is read, and drop the token", b.name, spelling) - } - if !readsHeld(b.head, map[node]bool{}) { - return nil - } - for _, u := range b.uses { - if u.nested { - return fmt.Errorf("name %q is read inside a repeat, and what it names reads a reference path, which each iteration draws apart; bind the name inside the repeat, or bind a second name there", b.name) - } - if u.group != b.uses[0].group { - return fmt.Errorf("name %q is read in two draw groups, %s and %s, and what it names reads a reference path, which each draw group draws apart; read the name in one draw group, or bind a name in each", b.name, groupSpelling(b.uses[0].group), groupSpelling(u.group)) - } - } - return nil -} - -func groupSpelling(group string) string { - if group == "" { - return "the unnamed one" - } - return fmt.Sprintf("%q", group) -} - -// readsHeld reports whether rendering n, or any field under it, can read a reference path. -func readsHeld(n node, seen map[node]bool) bool { - if seen[n] { - return false - } - seen[n] = true - if t, isTemplate := n.(*template); isTemplate { - for _, o := range t.compiled.ops { - for _, a := range append(o.arms[:len(o.arms):len(o.arms)], o.operands...) { - if a.kind == refPathRead { - return true - } - } - } - } - for _, c := range contained(n) { - if readsHeld(c.node, seen) { - return true - } - } - for _, e := range renderEdges(n) { - if readsHeld(e.to, seen) { - return true - } - } - return false -} - -// checkOnce walks n, rendering at key under b's pick, into each level a read of b addresses. -func (b *nameBinding) checkOnce(read string, n node, key string) error { - switch n := n.(type) { - case *choice: - for _, item := range n.items { - if err := b.checkOnce(read, item, key); err != nil { - return err - } - } - case *template: - return b.checkTemplateOnce(read, n, key) - } - return nil -} - -func (b *nameBinding) checkTemplateOnce(read string, t *template, key string) error { - fresh := map[string]int{} - var into []arm - for _, o := range t.compiled.ops { - for _, a := range append(o.arms[:len(o.arms):len(o.arms)], o.operands...) { - if isRef(a.head) || a.kind == namedRead { - continue - } - if _, kept := b.addressed[join(key, a.path)]; kept { - into = append(into, a) - } - if a.kind == freshRead { - fresh[a.head]++ - } - } - } - heads := make([]string, 0, len(fresh)) - for head := range fresh { - heads = append(heads, head) - } - sort.Strings(heads) - for _, head := range heads { - if by, kept := b.addressed[join(key, head)]; kept && fresh[head] > 1 { - return fmt.Errorf("{%s} renders field %q twice, so {%s} cannot say which draw it reads; drop {%s} or {%s}", read, head, by, read, by) - } - } - for _, a := range into { - for _, leaf := range a.leaves { - if err := b.checkOnce(read, leaf, join(key, a.path)); err != nil { - return err - } - } - } - return nil -} - -// namedPick is one draw of a name: the variant drawn at each level a read of it addresses, the -// value each read there produced, keyed by the path from the name, and the table rows they -// pinned. A reference read under it draws in the draw group of the read of the name, which the -// draw fences survey it in. -type namedPick struct { - named *nameBinding - memo drawMemo - pins pinSet -} - -// pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. -type pickFrame struct { - scope *nameScope - picks []namedPick -} - -func newPickFrame(scope *nameScope) *pickFrame { - f := &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))} - for i, b := range scope.order { - f.picks[i].named = b - } - return f -} - -// frameStack is the frames of the name scopes rendering, innermost last. A render's draws point -// at it, and a repeat iteration's share it: holding the frames in the draws, or in a renderScope, -// would move every render's draws to the heap. -type frameStack struct { - frames []*pickFrame -} - -// pushFrame opens f until popFrames closes it, returning the mark popFrames takes. -func (d *renderDraws) pushFrame(f *pickFrame) int { - mark := d.depth() - if d.frameStack == nil { - d.frameStack = &frameStack{} - } - d.frameStack.frames = append(d.frameStack.frames, f) - return mark -} - -func (d *renderDraws) popFrames(mark int) { - if d.frameStack != nil { - d.frameStack.frames = d.frameStack.frames[:mark] - } -} - -func (d *renderDraws) depth() int { - if d.frameStack == nil { - return 0 - } - return len(d.frameStack.frames) -} - -// enter starts a read landing on n: the read sees no frame opened before it, and gets from memo a -// frame for each name scope around n, so reads sharing memo read one pick of each name. It -// returns the mark that closes those frames. -func (sc renderScope) enter(n node, memo *drawMemo) (renderScope, int) { - sc.base, sc.pick = sc.draws.depth(), nil - for scope := scopeAround(n); scope != nil; scope = scope.up { - if len(scope.order) > 0 { - sc.draws.pushFrame(memo.enteredFrame(scope)) - } - } - return sc, sc.base -} - -// scopeAround is the innermost name scope a render of n reads names in, short of the frames n -// renders itself, one per iteration of a repeat. -func scopeAround(n node) *nameScope { - switch n := n.(type) { - case *template: - if n.repeat > 1 && n.nameScope != nil && n.nameScope.owner == n { - return n.nameScope.up - } - return n.nameScope - case *choice: - return n.nameScope - } - return nil -} - -func (m *drawMemo) enteredFrame(scope *nameScope) *pickFrame { - f, ok := m.enteredFrames[scope] - if !ok { - f = newPickFrame(scope) - if m.enteredFrames == nil { - m.enteredFrames = map[*nameScope]*pickFrame{} - } - m.enteredFrames[scope] = f - } - return f -} - -// renderFrame opens a fresh frame of t's scope, where t binds names and no read entering the -// category opened one, returning the mark that closes it, or -1. -func (sc renderScope) renderFrame(t *template) int { - if t.ownNameScope == nil || sc.frameOf(t.ownNameScope) != nil { - return -1 - } - return sc.draws.pushFrame(newPickFrame(t.ownNameScope)) -} - -// frameOf is the frame of scope rendering since the read entering the category, nil where none is. -func (sc renderScope) frameOf(scope *nameScope) *pickFrame { - if stack := sc.draws.frameStack; stack != nil { - for i := len(stack.frames) - 1; i >= sc.base; i-- { - if f := stack.frames[i]; f.scope == scope { - return f - } - } - } - return nil -} - -func readName(s *generatorState, sc renderScope, a arm) readValue { - f := sc.frameOf(a.named.scope) - if f == nil { - panic(internalError("name %q is read where no frame of its scope renders", a.named.name)) - } - p := &f.picks[a.named.index] - if r, done := p.memo.value[a.path]; done { - return r - } - leaf := p.draw(s, a.named.head, a.steps, a.levels, a.path) - sc, mark := sc.enter(leaf, &p.memo) - r := p.renderAt(s, leaf, a.path, sc) - sc.draws.popFrames(mark) - return r -} - -// drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one. -func (sc renderScope) drawRowOf(s *generatorState, t *table) int { - if sc.pick != nil { - return t.drawIn(s, &sc.pick.pins) - } - return t.drawRow(s) -} - -// keeps reports whether a read of sc's name addresses the level a starts at. -func (sc renderScope) keeps(a arm) bool { - _, kept := sc.pick.named.addressed[underKey(sc.pickKey, a.head)] - return kept -} - -// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey, where a read of -// the name addresses it: once per pick, by its path from the name. -func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue { - p, key := sc.pick, underKey(sc.pickKey, a.path) - if r, done := p.memo.value[key]; done { - return r - } - levels := make([]string, len(a.levels)) - for i, l := range a.levels { - levels[i] = underKey(sc.pickKey, l) - } - return p.renderAt(s, p.draw(s, t.head(a.head), a.steps, levels, key), key, sc) -} - -// underKey is the key of path under the pick key prefix. It never returns prefix itself, which a -// memo would keep: anything a renderScope holds reaching the heap moves every render's draws there. -func underKey(prefix, path string) string { - if prefix == "" { - return path - } - return prefix + "." + path -} - -// draw draws the path key names under p, its variant at key kept too. -func (p *namedPick) draw(s *generatorState, head node, steps []pathStep, levels []string, key string) node { - leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) - if c, isChoice := leaf.(*choice); isChoice { - leaf = p.memo.variantOf(s, c, key) - } - return leaf -} - -// renderAt renders leaf, drawn at key, as part of p, and keeps what it rendered. -func (p *namedPick) renderAt(s *generatorState, leaf node, key string, sc renderScope) readValue { - sc = sc.at(leaf, &p.pins) - sc.pick, sc.pickKey = p, key - r := renderLeaf(s, leaf, sc) - if p.memo.value == nil { - p.memo.value = map[string]readValue{} - } - p.memo.value[key] = r - return r -} - -// variantOf is the variant of c drawn at level, drawn now where none was. -func (m *drawMemo) variantOf(s *generatorState, c *choice, level string) node { - n, drew := m.variant[level] - if !drew { - n = resolveChoice(s, c) - if m.variant == nil { - m.variant = map[string]node{} - } - m.variant[strings.Clone(level)] = n - } - return n -} diff --git a/namefence.go b/namefence.go new file mode 100644 index 0000000..52dac90 --- /dev/null +++ b/namefence.go @@ -0,0 +1,138 @@ +package fejkdata + +import ( + "fmt" + "slices" + "sort" +) + +// checkNameReads refuses each binding of t that checkUses refuses, and a read such as {n} whose +// pick renders twice a field another read, {n.w}, reads once: the pick keeps one draw of it. +func checkNameReads(path string, t *template) error { + for _, tok := range t.tokens { + if tok.kind != nameBind { + continue + } + if err := t.nameScope.bindings[tok.bound].checkUses(); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), tok.body, err) + } + } + return namedReads(t, func(o *op, a *arm) error { + for _, leaf := range a.leaves { + if err := a.named.checkOnce(a.spelling, leaf, a.path); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), o.body, err) + } + } + return nil + }) +} + +// checkUses refuses a binding of a category read once whole, and reads of b in two draw groups, +// or inside a repeat, where what b names reads a reference path. +func (b *nameBinding) checkUses() error { + if r := b.uses[0]; len(b.uses) == 1 && r.tail == "" && !r.nested && len(b.tail) == 0 { + spelling := b.ref + if !r.operand { + spelling = "{" + spelling + "}" + } + return fmt.Errorf("name %q is read once, whole, which the bare reference draws the same way; write %s where it is read, and drop the token", b.name, spelling) + } + if !readsHeld(b.head, map[node]bool{}) { + return nil + } + for _, u := range b.uses { + if u.nested { + return fmt.Errorf("name %q is read inside a repeat, and what it names reads a reference path, which each iteration draws apart; bind the name inside the repeat, or bind a second name there", b.name) + } + if u.group != b.uses[0].group { + return fmt.Errorf("name %q is read in two draw groups, %s and %s, and what it names reads a reference path, which each draw group draws apart; read the name in one draw group, or bind a name in each", b.name, groupSpelling(b.uses[0].group), groupSpelling(u.group)) + } + } + return nil +} + +func groupSpelling(group string) string { + if group == "" { + return "the unnamed one" + } + return fmt.Sprintf("%q", group) +} + +// readsHeld reports whether rendering n, or any field under it, can read a reference path. +func readsHeld(n node, seen map[node]bool) bool { + if seen[n] { + return false + } + seen[n] = true + if t, isTemplate := n.(*template); isTemplate { + for _, o := range t.compiled.ops { + for _, a := range slices.Concat(o.arms, o.operands) { + if a.kind == refPathRead { + return true + } + } + } + } + for _, c := range contained(n) { + if readsHeld(c.node, seen) { + return true + } + } + for _, e := range renderEdges(n) { + if readsHeld(e.to, seen) { + return true + } + } + return false +} + +// checkOnce walks n, rendering at key under b's pick, into each level a read of b addresses. +func (b *nameBinding) checkOnce(read string, n node, key string) error { + switch n := n.(type) { + case *choice: + for _, item := range n.items { + if err := b.checkOnce(read, item, key); err != nil { + return err + } + } + case *template: + return b.checkTemplateOnce(read, n, key) + } + return nil +} + +func (b *nameBinding) checkTemplateOnce(read string, t *template, key string) error { + fresh := map[string]int{} + var into []arm + for _, o := range t.compiled.ops { + for _, a := range slices.Concat(o.arms, o.operands) { + if isRef(a.head) || a.kind == namedRead { + continue + } + if _, kept := b.addressed[join(key, a.path)]; kept { + into = append(into, a) + } + if a.kind == freshRead { + fresh[a.head]++ + } + } + } + heads := make([]string, 0, len(fresh)) + for head := range fresh { + heads = append(heads, head) + } + sort.Strings(heads) + for _, head := range heads { + if by, kept := b.addressed[join(key, head)]; kept && fresh[head] > 1 { + return fmt.Errorf("{%s} renders field %q twice, so {%s} cannot say which draw it reads; drop {%s} or {%s}", read, head, by, read, by) + } + } + for _, a := range into { + for _, leaf := range a.leaves { + if err := b.checkOnce(read, leaf, join(key, a.path)); err != nil { + return err + } + } + } + return nil +} diff --git a/node.go b/node.go index 4f91007..15487e4 100644 --- a/node.go +++ b/node.go @@ -235,10 +235,12 @@ func compileString(s string, site tableSite) (*template, error) { return nil, err } if site.table != nil { - if err := checkTableToks(toks, unbound); err != nil { + if err := refuseTableBinding(toks); err != nil { return nil, err } - unbound = nil + if len(unbound) > 0 { + return nil, unbound[0].err + } } return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site, unbound: unbound}, nil } diff --git a/path.go b/path.go index 0666214..eabadcb 100644 --- a/path.go +++ b/path.go @@ -427,16 +427,7 @@ func drawVariant(s *generatorState, c *choice, memo *drawMemo, levels []string, if memo == nil { return resolveChoice(s, c) } - level := levels[at] - n, drew := memo.variant[level] - if !drew { - n = resolveChoice(s, c) - if memo.variant == nil { - memo.variant = map[string]node{} - } - memo.variant[level] = n - } - return n + return memo.variantOf(s, c, levels[at]) } func (t *table) selector(tail []string) (sel string, rest []string, err error) { diff --git a/pick.go b/pick.go new file mode 100644 index 0000000..95ff8e7 --- /dev/null +++ b/pick.go @@ -0,0 +1,209 @@ +package fejkdata + +// namedPick is one draw of a name: the variant drawn at each level a read of it addresses, the +// value each read there produced, keyed by the path from the name, and the table rows they +// pinned. A reference read under it draws in the draw group of the read of the name, which the +// draw fences survey it in. +type namedPick struct { + named *nameBinding + memo drawMemo + pins pinSet +} + +// pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. +type pickFrame struct { + scope *nameScope + picks []namedPick +} + +func newPickFrame(scope *nameScope) *pickFrame { + f := &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))} + for i, b := range scope.order { + f.picks[i].named = b + } + return f +} + +// frameStack is the frames of the name scopes rendering, innermost last. A render's draws point +// at it, and a repeat iteration's share it: holding the frames in the draws, or in a renderScope, +// would move every render's draws to the heap. +type frameStack struct { + frames []*pickFrame +} + +// pushFrame opens f until popFrames closes it, returning the mark popFrames takes. +func (d *renderDraws) pushFrame(f *pickFrame) int { + mark := d.depth() + if d.frameStack == nil { + d.frameStack = &frameStack{} + } + d.frameStack.frames = append(d.frameStack.frames, f) + return mark +} + +func (d *renderDraws) popFrames(mark int) { + if d.frameStack != nil { + d.frameStack.frames = d.frameStack.frames[:mark] + } +} + +func (d *renderDraws) depth() int { + if d.frameStack == nil { + return 0 + } + return len(d.frameStack.frames) +} + +// enter starts a read landing on n: the read sees no frame opened before it, and gets from memo a +// frame for each name scope around n, so reads sharing memo read one pick of each name. It +// returns the mark that closes those frames. +func (sc renderScope) enter(n node, memo *drawMemo) (renderScope, int) { + sc = sc.entering() + for scope := scopeAround(n); scope != nil; scope = scope.up { + if len(scope.order) > 0 { + sc.draws.pushFrame(memo.enteredFrame(scope)) + } + } + return sc, sc.base +} + +// entering is sc as a read entering a category sees it: no frame opened before it, and no pick. +func (sc renderScope) entering() renderScope { + sc.base, sc.pick = sc.draws.depth(), nil + return sc +} + +// scopeAround is the innermost name scope a render of n reads names in, short of the frames n +// renders itself, one per iteration of a repeat. +func scopeAround(n node) *nameScope { + switch n := n.(type) { + case *template: + if n.repeat > 1 && n.nameScope != nil && n.nameScope.owner == n { + return n.nameScope.up + } + return n.nameScope + case *choice: + return n.nameScope + } + return nil +} + +func (m *drawMemo) enteredFrame(scope *nameScope) *pickFrame { + f, ok := m.enteredFrames[scope] + if !ok { + f = newPickFrame(scope) + if m.enteredFrames == nil { + m.enteredFrames = map[*nameScope]*pickFrame{} + } + m.enteredFrames[scope] = f + } + return f +} + +// renderFrame opens a fresh frame of t's scope, where t binds names and no read entering the +// category opened one, returning the mark that closes it, or -1. +func (sc renderScope) renderFrame(t *template) int { + if t.ownNameScope == nil || sc.frameOf(t.ownNameScope) != nil { + return -1 + } + return sc.draws.pushFrame(newPickFrame(t.ownNameScope)) +} + +// frameOf is the frame of scope rendering since the read entering the category, nil where none is. +func (sc renderScope) frameOf(scope *nameScope) *pickFrame { + if stack := sc.draws.frameStack; stack != nil { + for i := len(stack.frames) - 1; i >= sc.base; i-- { + if f := stack.frames[i]; f.scope == scope { + return f + } + } + } + return nil +} + +func readName(s *generatorState, sc renderScope, a arm) readValue { + f := sc.frameOf(a.named.scope) + if f == nil { + panic(internalError("name %q is read where no frame of its scope renders", a.named.name)) + } + p := &f.picks[a.named.index] + if r, done := p.memo.value[a.path]; done { + return r + } + leaf := p.draw(s, a.named.head, a.steps, a.levels, a.path) + sc, mark := sc.enter(leaf, &p.memo) + r := p.renderAt(s, leaf, a.path, sc) + sc.draws.popFrames(mark) + return r +} + +// drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one. +func (sc renderScope) drawRowOf(s *generatorState, t *table) int { + if sc.pick != nil { + return t.drawIn(s, &sc.pick.pins) + } + return t.drawRow(s) +} + +// keeps reports whether a read of sc's name addresses the level a starts at. +func (sc renderScope) keeps(a arm) bool { + _, kept := sc.pick.named.addressed[underKey(sc.pickKey, a.head)] + return kept +} + +// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey, where a read of +// the name addresses it: once per pick, by its path from the name. +func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue { + p, key := sc.pick, underKey(sc.pickKey, a.path) + if r, done := p.memo.value[key]; done { + return r + } + levels := make([]string, len(a.levels)) + for i, l := range a.levels { + levels[i] = underKey(sc.pickKey, l) + } + return p.renderAt(s, p.draw(s, t.head(a.head), a.steps, levels, key), key, sc) +} + +// underKey is the key of path under the pick key prefix. It never returns prefix itself, which a +// memo would keep: anything a renderScope holds reaching the heap moves every render's draws there. +func underKey(prefix, path string) string { + if prefix == "" { + return path + } + return prefix + "." + path +} + +// draw draws the path key names under p, its variant at key kept too. +func (p *namedPick) draw(s *generatorState, head node, steps []pathStep, levels []string, key string) node { + leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) + if c, isChoice := leaf.(*choice); isChoice { + leaf = p.memo.variantOf(s, c, key) + } + return leaf +} + +// renderAt renders leaf, drawn at key, as part of p, and keeps what it rendered. +func (p *namedPick) renderAt(s *generatorState, leaf node, key string, sc renderScope) readValue { + sc = sc.at(leaf, &p.pins) + sc.pick, sc.pickKey = p, key + r := renderLeaf(s, leaf, sc) + if p.memo.value == nil { + p.memo.value = map[string]readValue{} + } + p.memo.value[key] = r + return r +} + +// variantOf is the variant of c drawn at level, drawn now where none was. +func (m *drawMemo) variantOf(s *generatorState, c *choice, level string) node { + n, drew := m.variant[level] + if !drew { + n = resolveChoice(s, c) + if m.variant == nil { + m.variant = map[string]node{} + } + m.variant[level] = n + } + return n +} diff --git a/render.go b/render.go index 855afcf..2943416 100644 --- a/render.go +++ b/render.go @@ -187,8 +187,7 @@ func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a sc.pick = nil switch { case a.kind == freshRead && isRef(a.head): - sc.base = sc.draws.depth() - return readValue{text: render(s, t.head(a.head), sc)} + return readValue{text: render(s, t.head(a.head), sc.entering())} case a.kind == freshRead: return readValue{text: render(s, t.head(a.head), sc)} } diff --git a/table.go b/table.go index 983765b..c23a513 100644 --- a/table.go +++ b/table.go @@ -323,7 +323,7 @@ func (t *table) compileRowFormat(format string) error { if err != nil { return err } - if err := checkTableToks(toks, nil); err != nil { + if err := refuseTableBinding(toks); err != nil { return err } for _, tok := range toks { @@ -345,25 +345,21 @@ func (t *table) compileRowFormat(format string) error { if err != nil { return err } - if err := checkTableToks(nil, unbound); err != nil { - return err + if len(unbound) > 0 { + return unbound[0].err } t.formatTemplate = &template{format: format, tokens: toks, fields: fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} t.rowNode = &tableRow{t} return nil } -// checkTableToks refuses a table's format or cell binding a name, or reading a head no column -// holds: a table holds no names. -func checkTableToks(toks []formatToken, unbound []unboundRead) error { +// refuseTableBinding refuses a table's format or cell binding a name. +func refuseTableBinding(toks []formatToken) error { for _, tok := range toks { if tok.kind == nameBind { return fmt.Errorf("token {%s}: a table binds no name; bind it in a template reading the table", tok.body) } } - if len(unbound) > 0 { - return unbound[0].err - } return nil } diff --git a/todo.md b/todo.md index 919a074..cc197a4 100644 --- a/todo.md +++ b/todo.md @@ -203,6 +203,7 @@ Shape: T = table, t = template, c = choice. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables, Names and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. +- Widens the refusal of a name read once to every binding, a path into it or a reference path bound included, since `{n.x}` read once is then `{/ref.x}`, and deletes the names fence over draw groups and nested repeats (`checkUses`, `readsHeld`). - Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. ### 25. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. -- 2.52.0 From 3328d56cb2dc4b8ed2acc1d3281f404da0719466 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:21:32 +0200 Subject: [PATCH 447/966] Name namefence.go by what it holds --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index be03692..1777f00 100644 --- a/README.md +++ b/README.md @@ -934,7 +934,7 @@ fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws name.go names: {ref as n} bindings, the scopes they live in, and the reads they answer -namefence.go the load fences over names: reads in two draw groups or a nested repeat, and a field rendered twice under a pick +namefence.go the load fences over names pick.go a render's named picks: the frames of the name scopes rendering, and the reads kept under a name path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make -- 2.52.0 From c569bf907f91e16007e4b718197ebf6364dfea9f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:54:34 +0200 Subject: [PATCH 448/966] Count a lower panel score as a lower score, noise or not, and record the named-pick decision's approval --- AGENTS.md | 3 ++- docs/decisions.md | 2 +- todo.md | 22 +--------------------- 3 files changed, 4 insertions(+), 23 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6babe68..1de290c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,8 @@ again, until the score passes. At or above 7.0 every pull request scores with the four-seat run and answers it in one run. No merge lowers the last score its own panel recorded: the nine-seat round and the four-seat - run are two ratchets, never compared with each other. + run are two ratchets, never compared with each other. A lower score blocks the merge + even where it may be panel noise: fix what the seats name until the score rises. - Hard tabs. No comment by default; delete a restatement, a rationale, history, or a file preamble. - One spelling per result: reject the other at `New`, and let the error name the spelling to use. - Prose naming a source states what that source states: read the register's own field diff --git a/docs/decisions.md b/docs/decisions.md index b97fed0..ee79b42 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -894,7 +894,7 @@ a struct's fields as one. ## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read -2026-10-03, larv-review in PR #145, for Lilleman auf Larv. Goals 5.3 and 5.6. +2026-10-03, larv-review in PR #145; approved by Lilleman auf Larv. Goals 5.3 and 5.6. `bindNames` scopes a name at compile, and a render keeps one `pickFrame` per scope. One field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside `"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category diff --git a/todo.md b/todo.md index cc197a4..c587109 100644 --- a/todo.md +++ b/todo.md @@ -64,7 +64,6 @@ | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | -| 6 | 0.1.0 | question | **Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4.** | 1 | 1 | 3 | 8 | 3 | 19.1 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | @@ -236,26 +235,7 @@ The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid fro ### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. -Needs item 4, and runs as item 6 decides. - -### 6. Decide how the comprehension gate treats panel noise: identical code scored 5.7 and 5.4. - -Two nine-seat runs on `5cb919e`, same briefs, same model, 2026-09-30: - -| Seat | Run 1 | Run 2 | -|---|---|---| -| Junior A | 5 | 5 | -| Junior B | 4 | 5 | -| Mid A | 6 | 5 | -| Mid B | 6 | 6 | -| Senior domain | 6 | 5 | -| Senior maintainability | 6 | 6 | -| Senior extender | 6 | 5 | -| Architect greenfield | 6 | 6 | -| Architect inherited | 6 | 6 | -| Mean (Navigation / Locality / Shape / Self-sufficiency) | 5.7 (7.0 / 5.1 / 5.9 / 5.7) | 5.4 (6.3 / 4.8 / 5.8 / 5.3) | - -Both runs named the same hardest units, `drawsApart` and `readFold.reads`. Run 1 is the recorded score; run 2 reran the same code to measure noise. A spread of 0.3 is close to the whole 5.7–6.1 range the recorded scores span since 2026-09-20, so `AGENTS.md`'s "no merge lowers the last score" can block or pass a chunk by chance. Until this item is decided, that rule compares against 5.7, from 2026-09-30. +Needs item 4. ### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. -- 2.52.0 From 568c8758885a9126d4b31a7901dd01820d312bd0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:01:36 +0200 Subject: [PATCH 449/966] Let misc.timezone drop the zones of unshipped territories: agreement wins over the last rows --- docs/decisions.md | 3 ++- todo.md | 5 ----- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index ee79b42..76cc49a 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -758,7 +758,8 @@ which says "Chinese, Mandarin" for `zh`. Where every territory row has a child the column is a `parent`, and the import drops the child rows whose territory the set does not ship — 17 of `misc.timezone`'s, Antarctica's -ten among them. Agreement across a record is worth more than the last rows of a table. +ten among them. Agreement across a record is worth more than the last rows of a table: +goal 4.1 wins over goal 13.1 here, as the maintainer confirmed on 2026-10-03. Where no such link can hold the fact stays a column. Layer your own `misc.territory` over the shipped one and you must layer `misc.timezone` too, or the link fails at load naming the row. diff --git a/todo.md b/todo.md index c587109..6a0e63f 100644 --- a/todo.md +++ b/todo.md @@ -88,7 +88,6 @@ | 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | | 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | | 73 | 0.2.0 | question | **Decide whether the default embed's partial Swedish and US place tables stand against goal 13.1, or have the goals say which goal wins.** | 1 | 1 | 3 | 6 | 13.1 | 9.6 | -| 74 | 0.2.0 | question | **Decide whether `misc.timezone` may drop the 17 zones of territories `misc.territory` leaves out, against goal 13.1, or have the goals say which goal wins.** | 1 | 1 | 3 | 6 | 13.1 | 9.6 | | 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | | 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | | 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | @@ -322,10 +321,6 @@ Require the path step to reach a sibling category. The decision "The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more" serves goal 2's built-in data with no download, ships less than its registers hold, and names no item that ends it. Item 60 ships the full registers as packs, beside the partial default. -### 74. Decide whether `misc.timezone` may drop the 17 zones of territories `misc.territory` leaves out, against goal 13.1, or have the goals say which goal wins. - -The decision "`misc.territory` is the spine, and a `misc` table naming a territory links to it" serves goal 4.1 and drops those tzdb rows, and names no item that ends it. - ### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. Both render the same run. -- 2.52.0 From 1d6b96143f1975bd9799809626f5d414476675ce Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:47:19 +0200 Subject: [PATCH 450/966] Plan on-demand shipped data first: a build-time index, data modules for the library, and population-weighted 10% geo slices --- docs/decisions.md | 5 +++++ todo.md | 27 ++++++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 76cc49a..84e24f3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -114,6 +114,9 @@ and `regexp.Match`, not two spellings of one result. A directory a machine happens to have would make `--seed 42` machine-dependent. Data still lives in `data/` as JSON; `--data-path` layers over it. +`todo.md` items 76 and 77 revise it: shipped data loads on demand from an index built at +build time, and a library imports it as Go modules. + ## A bare reference draws each time; a reference path is held 2026-09-02, Lilleman auf Larv. @@ -653,6 +656,8 @@ size, and `--min-population` and `--streets-per-locality` on the import scripts fuller set. The two trees add about 20 ms to `New`, which loads the shipped set in about 45 ms. +Withdrawn by the maintainer on 2026-10-03; `todo.md` item 78 replaces it. + ## A locale's `address` restates its country record's format 2026-09-18, Lilleman auf Larv. diff --git a/todo.md b/todo.md index 6a0e63f..c5c2a54 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 76` +`Next ID = 79` | Goal | W | |---|---| @@ -52,6 +52,9 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| +| 76 | 0.1.0 | decision | **Generate the index of shipped categories at build time, and parse and link a shipped category only when a template or path first reaches it; a `--data-path` still loads whole.** | 4 | 8 | 10 | 10 | 2, 12 | 33.0 | +| 77 | 0.1.0 | decision | **Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI.** | 5 | 8 | 10 | 10 | 2, 1.1 | 33.0 | +| 78 | 0.1.0 | decision | **Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population.** | 6 | 7 | 10 | 10 | 2, 13.1 | 31.2 | | 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | @@ -87,7 +90,6 @@ | 32 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 4.1 | 14.4 | | 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | | 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | -| 73 | 0.2.0 | question | **Decide whether the default embed's partial Swedish and US place tables stand against goal 13.1, or have the goals say which goal wins.** | 1 | 1 | 3 | 6 | 13.1 | 9.6 | | 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | | 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | | 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | @@ -117,7 +119,6 @@ | 62 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 4 | 15.2 | | 59 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 11 | 13.8 | | 65 | 0.4.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 5.8 | 13.2 | -| 60 | 0.4.0 | | **Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`.** | 5 | 8 | 7 | 6 | 14, 12 | 12.4 | | 66 | 0.4.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 3.4 | 11.8 | | 67 | 0.4.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 3.4 | 11.3 | | 64 | 0.4.0 | | **Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and the premise it rests on.** | 1 | 4 | 1 | 6 | 3.4 | 10.8 | @@ -127,6 +128,18 @@ ## Details +### 76. Generate the index of shipped categories at build time, and parse and link a shipped category only when a template or path first reaches it; a `--data-path` still loads whole. + +A restructure the maintainer approved on 2026-10-03, so it merges below goal 3's floor and goes first; the round's gates are adjusted to it afterwards. `go generate` writes an index of every shipped category, path and table, embedded beside the data. `New` reads the index; a template, `Fake` path or record compiles the categories it reaches, transitively, and keeps them. CI loads every shipped category eagerly, so shipped data never fails for a user. Revise the decisions this touches, "The shipped data is embedded, not discovered" among them, and the README's Performance section. + +### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. + +Needs item 76. The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Check goal 2's wording against this, and propose a rewording to the maintainer if it no longer says what ships where. + +### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. + +Needs item 77. Ten streets per place crowds 500 Stockholm customers onto ten streets. `geo-se.py` and `geo-us.py` keep 10% of each place's streets, at least 10. `geo-us.py` draws 10% of every Census place, down to the smallest village, picking each with odds by population, and every table weights a place by its population, so a draw is not mostly tiny villages. Replaces the decision "The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more", which the maintainer withdrew on 2026-10-03. Re-pin seeded output and the shape in their own commits. + ### 61. Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country. Pick the ten from a published ranking of languages by total speakers, such as Ethnologue's, each in the country where it has the most speakers, and record the ranking in the decision that names them. The Nordic ones are sv_SE (shipped), nb_NO, da_DK, fi_FI and is_IS. Each locale carries goal 2.2's categories, and its `address` reads its country's `geo/` tree by reference. @@ -317,10 +330,6 @@ Require the path step to reach a sibling category. `readTableOptions` and `loadDir` return on the first in Go's map order. -### 73. Decide whether the default embed's partial Swedish and US place tables stand against goal 13.1, or have the goals say which goal wins. - -The decision "The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more" serves goal 2's built-in data with no download, ships less than its registers hold, and names no item that ends it. Item 60 ships the full registers as packs, beside the partial default. - ### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. Both render the same run. @@ -417,10 +426,6 @@ A caller wanting parallel throughput then makes one generator per goroutine. The check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a flag, an exit code, an exported name, a fence, a builtin and the lowest Go. -### 60. Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip for `--data-path`. - -Every US place of 10,000 and more streets per locality, built by `--min-population` and `--streets-per-locality`. The default embed stays under ~1 MB per country and `New` under ~50 ms. - ### 66. Cut the README's Layout block to the lines that say what a file name cannot. `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names. -- 2.52.0 From cff6533e7bacee6ec1734765ad7e90d089bc6550 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:55:00 +0200 Subject: [PATCH 451/966] Say in goal 2 that the CLI carries all the data and a library imports what it needs --- README.md | 3 ++- todo.md | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 1777f00..6a08a07 100644 --- a/README.md +++ b/README.md @@ -878,7 +878,8 @@ Where the project is heading; the sections before Audience document what ships t 1. **The best fake-data tool there is, and the one developers reach for.** 1. You can install it the way you install your other tools, and people hear about it where developers read. -2. **Enough data ships built in to fill a real test database, with no download.** +2. **Enough data ships built in to fill a real test database: the CLI carries all of it, a + library imports what it needs, and nothing is fetched at run time.** 1. It covers the ten most-spoken languages and the Nordic countries. 2. Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates. diff --git a/todo.md b/todo.md index c5c2a54..fa72a2d 100644 --- a/todo.md +++ b/todo.md @@ -134,7 +134,7 @@ A restructure the maintainer approved on 2026-10-03, so it merges below goal 3's ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -Needs item 76. The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Check goal 2's wording against this, and propose a rewording to the maintainer if it no longer says what ships where. +Needs item 76. The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where since 2026-10-03. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. -- 2.52.0 From 75597664e0aabf353ca8e234fcb8f0376b9884cc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:07:36 +0200 Subject: [PATCH 452/966] Test that shipped categories load when first reached, from a current index, and render as the whole load --- drawfence_test.go | 5 +- shape_test.go | 6 +- shipped_test.go | 209 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 211 insertions(+), 9 deletions(-) create mode 100644 shipped_test.go diff --git a/drawfence_test.go b/drawfence_test.go index 113a316..85bddbf 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -57,10 +57,7 @@ func fenceRoots(t *testing.T, f *Generator) []fenceRoot { func TestPairsFindWhatAWholeReplayFinds(t *testing.T) { corpus := fenceCorpus(t) roots := fenceRoots(t, corpus) - shipped, err := New() - if err != nil { - t.Fatalf("New() = %v", err) - } + shipped := newShippedWhole(t) _ = walkNodes(shipped.root.children, func(path string, n node) error { if tm, ok := n.(*template); ok { roots = append(roots, fenceRoot{path, tm}) diff --git a/shape_test.go b/shape_test.go index 50d48f2..a912b67 100644 --- a/shape_test.go +++ b/shape_test.go @@ -12,11 +12,7 @@ const shapePin = "testdata/shipped_shape.txt" // REPIN=1 rewrites the pin. func TestShippedShapeIsPinned(t *testing.T) { - f, err := New(WithSeed(1)) - if err != nil { - t.Fatal(err) - } - got := shippedShape(f) + got := shippedShape(newShippedWhole(t, WithSeed(1))) if os.Getenv("REPIN") == "1" { if err := os.WriteFile(shapePin, []byte(got), 0o644); err != nil { t.Fatal(err) diff --git a/shipped_test.go b/shipped_test.go new file mode 100644 index 0000000..13145dc --- /dev/null +++ b/shipped_test.go @@ -0,0 +1,209 @@ +package fejkdata + +import ( + "bytes" + "fmt" + "go/format" + "io/fs" + "os" + "reflect" + "slices" + "sort" + "sync" + "testing" +) + +const shippedIndexFile = "shippedindex.go" + +// newShippedWhole loads every shipped category in New, as a --data-path load does, so a test +// reads the whole shipped tree. +func newShippedWhole(t testing.TB, opts ...Option) *Generator { + t.Helper() + data, err := fs.Sub(shippedFS, "data") + if err != nil { + t.Fatal(err) + } + f, err := New(append([]Option{WithoutShippedData(), WithDataFS(data)}, opts...)...) + if err != nil { + t.Fatalf("New() = %v", err) + } + return f +} + +// loadedCategories is the dot path of every category f has parsed and linked. +func loadedCategories(f *Generator) []string { + var out []string + var walk func(prefix string, g *folder) + walk = func(prefix string, g *folder) { + for _, name := range sortedNames(g.children) { + if sub, isFolder := g.children[name].(*folder); isFolder { + walk(join(prefix, name), sub) + continue + } + out = append(out, join(prefix, name)) + } + } + walk("", &f.root) + return out +} + +// REPIN=1 rewrites the index. +func TestShippedIndexIsCurrent(t *testing.T) { + got, err := shippedIndexSource(newShippedWhole(t)) + if err != nil { + t.Fatal(err) + } + if os.Getenv("REPIN") == "1" { + if err := os.WriteFile(shippedIndexFile, got, 0o644); err != nil { + t.Fatal(err) + } + return + } + want, err := os.ReadFile(shippedIndexFile) + if err != nil { + t.Fatal(err) + } + if string(got) != string(want) { + t.Fatalf("%s is stale for data/; rewrite it with REPIN=1", shippedIndexFile) + } +} + +func shippedIndexSource(whole *Generator) ([]byte, error) { + var b bytes.Buffer + b.WriteString("// Code generated by TestShippedIndexIsCurrent with REPIN=1. DO NOT EDIT.\n\npackage fejkdata\n\nvar shippedIndex = map[string]shippedCategory{\n") + for _, s := range categorySites(&whole.root) { + fmt.Fprintf(&b, "%q: {", s.path) + if t, isTable := s.n.(*table); isTable && t.parentIndex >= 0 { + fmt.Fprintf(&b, "parent: %q, ", t.header[t.parentIndex]) + } + ps := paths(s.n) + sort.Strings(ps) + fmt.Fprintf(&b, "paths: %#v},\n", ps) + } + b.WriteString("}\n") + return format.Source(b.Bytes()) +} + +func TestNewLoadsNoShippedCategory(t *testing.T) { + f, err := New() + if err != nil { + t.Fatalf("New() = %v", err) + } + if got := loadedCategories(f); len(got) > 0 { + t.Fatalf("New() loaded %v", got) + } +} + +func TestReachingACategoryLoadsWhatItReads(t *testing.T) { + for _, tc := range []struct { + reach func(f *Generator) error + want, absent []string + }{ + { + reach: func(f *Generator) error { _, err := f.Fake("sv_SE.person.first"); return err }, + want: []string{"sv_SE.first-name", "sv_SE.last-name", "sv_SE.person", "sv_SE.sex", "sv_SE.title"}, + }, + { + reach: func(f *Generator) error { _, err := f.Fake("geo.SE.region"); return err }, + want: []string{"geo.SE.locality", "geo.SE.municipality", "geo.SE.postal-code", "geo.SE.region", "geo.SE.street"}, + absent: []string{"geo.SE.address", "geo.US.region"}, + }, + { + reach: func(f *Generator) error { _, err := f.FakeTemplate("{/misc.territory[SE].name}"); return err }, + want: []string{"misc.territory", "misc.timezone"}, + absent: []string{"misc.tld"}, + }, + { + reach: func(f *Generator) error { _, err := f.FakeRecord("misc.uuid"); return err }, + want: []string{"misc.uuid"}, + absent: []string{"misc.objectid"}, + }, + { + reach: func(f *Generator) error { + var v struct { + Name string `fake:"sv_SE.first-name"` + } + return f.FakeStruct(&v) + }, + want: []string{"sv_SE.first-name"}, + absent: []string{"sv_SE.person"}, + }, + } { + f, err := New() + if err != nil { + t.Fatalf("New() = %v", err) + } + if err := tc.reach(f); err != nil { + t.Fatal(err) + } + got := loadedCategories(f) + for _, w := range tc.want { + if !slices.Contains(got, w) { + t.Errorf("loaded %v, want %s among them", got, w) + } + } + for _, a := range tc.absent { + if slices.Contains(got, a) { + t.Errorf("loaded %v, want %s left unloaded", got, a) + } + } + } +} + +func TestADataPathLoadsTheShippedSetWhole(t *testing.T) { + f, err := New(WithDataPath(writeData(t, map[string]string{"x": `"y"`}))) + if err != nil { + t.Fatalf("New() = %v", err) + } + if got, want := loadedCategories(f), loadedCategories(newShippedWhole(t, WithDataPath(writeData(t, map[string]string{"x": `"y"`})))); !reflect.DeepEqual(got, want) { + t.Fatalf("loaded %v, want %v", got, want) + } +} + +func TestOnDemandRendersAsTheWholeLoad(t *testing.T) { + whole := newShippedWhole(t) + paths := whole.List() + lazy, err := New() + if err != nil { + t.Fatalf("New() = %v", err) + } + if got := lazy.List(); !reflect.DeepEqual(got, paths) { + t.Fatalf("List() on demand = %v, want %v", got, paths) + } + a, err := New(WithSeed(7)) + if err != nil { + t.Fatal(err) + } + b := newShippedWhole(t, WithSeed(7)) + for _, p := range paths { + for range 3 { + got, gotErr := a.Fake(p) + want, wantErr := b.Fake(p) + if got != want || (gotErr == nil) != (wantErr == nil) { + t.Fatalf("Fake(%q) on demand = %q, %v; whole = %q, %v", p, got, gotErr, want, wantErr) + } + } + } +} + +func TestListRunsBesideAnOnDemandLoad(t *testing.T) { + f, err := New() + if err != nil { + t.Fatalf("New() = %v", err) + } + var wg sync.WaitGroup + for _, p := range []string{"sv_SE.person", "geo.US.address", "misc.territory"} { + wg.Add(2) + go func() { + defer wg.Done() + if _, err := f.Fake(p); err != nil { + t.Error(err) + } + }() + go func() { + defer wg.Done() + f.List() + }() + } + wg.Wait() +} -- 2.52.0 From 5667cd5dc31dd9427bd3a9263de4304d1b8f1557 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:12:44 +0200 Subject: [PATCH 453/966] Load a shipped category on the first call reaching it, from an index generated from data/ --- AGENTS.md | 1 + README.md | 14 ++-- data.go | 76 ++++++++++++++++----- docs/decisions.md | 15 ++++- drawfence_test.go | 8 +-- fejkdata.go | 54 ++++++++------- graph.go | 14 +--- inline.go | 13 +++- node.go | 5 +- reads_test.go | 2 +- record.go | 3 + reference.go | 53 +++++---------- render.go | 3 + shipped.go | 168 ++++++++++++++++++++++++++++++++++++++++++++++ shippedindex.go | 81 ++++++++++++++++++++++ struct.go | 10 +-- table.go | 44 ++++-------- 17 files changed, 422 insertions(+), 142 deletions(-) create mode 100644 shipped.go create mode 100644 shippedindex.go diff --git a/AGENTS.md b/AGENTS.md index 1de290c..b38477a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -43,6 +43,7 @@ In [docs/decisions.md](docs/decisions.md): - A padded JSON argument is rejected, not trimmed - `FakeTemplate` and `NewTemplate` both stay - The shipped data is embedded, not discovered +- The shipped set alone loads a category on the first call reaching it; beside a `--data-path` it loads whole - A bare reference draws each time; a reference path is held - Reference sigils follow the filesystem - A change to what exists is a major; a minor only adds diff --git a/README.md b/README.md index 6a08a07..79365b0 100644 --- a/README.md +++ b/README.md @@ -813,9 +813,11 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan The target is about a microsecond per value (goal 12). -Each file is parsed, validated and weight-indexed once, in `New`, bar a table's name and -parent columns, which are mapped on the first `Fake` that selects by name or descends -through it. Proving the draw fences adds a pass over the loaded tree per +Each file is parsed, validated and weight-indexed once, bar a table's name and parent +columns, which are mapped on the first `Fake` that selects by name or descends through +it. With the shipped set alone, a category loads on the first call reaching it, with +every category it reads, and `List` reads an index; beside a `--data-path`, every +category loads in `New`. Proving the draw fences adds a pass over the loaded tree per fence, and walks what a render reads only where data holds a reference, so a set that holds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one @@ -931,7 +933,9 @@ Where the project is heading; the sections before Audience document what ships t ``` doc.go the package doc, and the vocabulary the package is written in -fejkdata.go Generator, New, options, the embedded data set, List +fejkdata.go Generator, New, options, List +shipped.go the embedded data set, and loading a shipped category on the first call reaching it +shippedindex.go the index of shipped categories, generated from data/ node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws name.go names: {ref as n} bindings, the scopes they live in, and the reads they answer @@ -1016,7 +1020,7 @@ what the score asks of a pull request. Goal 5 is checked against simulated templ writers from the Audience. A change to the shipped data re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) -in its own commit: +and regenerates [`shippedindex.go`](shippedindex.go) in its own commit: ```sh REPIN=1 docker compose run --rm --user "$(id -u):$(id -g)" test diff --git a/data.go b/data.go index 942fb4a..9a88b9a 100644 --- a/data.go +++ b/data.go @@ -55,7 +55,7 @@ func loadData(sources []dataSource) (map[string]node, error) { if len(root) == 0 { return nil, fmt.Errorf("no .json data found") } - if err := treeBinding(root).bind(); err != nil { + if err := categoryBinding(categorySites(&folder{children: root}), root).bind(); err != nil { return nil, err } return root, nil @@ -69,12 +69,7 @@ func loadDir(src dataSource, dir string) (*folder, error) { return nil, fmt.Errorf("%s: %w", src.labelled(dir), err) } g := &folder{children: map[string]node{}} - files := &categoryFiles{src: src, dir: dir, tsv: map[string]bool{}} - for _, e := range entries { - if strings.HasSuffix(e.Name(), ".tsv") && !e.IsDir() { - files.tsv[e.Name()] = false - } - } + files := newCategoryFiles(src, dir, entries) for _, e := range entries { if strings.HasPrefix(e.Name(), ".") { // hidden: a checkout or an editor's file, never data continue @@ -106,6 +101,16 @@ type categoryFiles struct { tsv map[string]bool } +func newCategoryFiles(src dataSource, dir string, entries []fs.DirEntry) *categoryFiles { + files := &categoryFiles{src: src, dir: dir, tsv: map[string]bool{}} + for _, e := range entries { + if strings.HasSuffix(e.Name(), ".tsv") && !e.IsDir() { + files.tsv[e.Name()] = false + } + } + return files +} + func (c *categoryFiles) readRows(name string) (string, error) { if _, present := c.tsv[name]; !present { return "", fmt.Errorf("rows names %s, which is not beside it in %s", name, c.src.labelled(c.dir)) @@ -175,12 +180,45 @@ func mergeChildren(dst, src map[string]node) { } } -// nodeScope is the set of nodes one validation pass covers: a whole loaded tree, -// or a single inline node. +// nodeScope is the set of nodes one validation pass covers: the categories one load +// binds, or a single inline node. type nodeScope func(fn func(path string, n node) error) error -func treeScope(root map[string]node) nodeScope { - return func(fn func(path string, n node) error) error { return walkNodes(root, fn) } +// categorySite is a loaded category and where it sits: the folder holding it, that +// folder's path, and its own dot path. +type categorySite struct { + dir []string + path string + in *folder + n node +} + +// categorySites lists every category under root, folders and names in sorted order. +func categorySites(root *folder) []categorySite { + var out []categorySite + var walk func(dir []string, g *folder) + walk = func(dir []string, g *folder) { + for _, name := range sortedNames(g.children) { + if sub, isFolder := g.children[name].(*folder); isFolder { + walk(append(dir[:len(dir):len(dir)], name), sub) + continue + } + out = append(out, categorySite{dir: dir, path: join(strings.Join(dir, "."), name), in: g, n: g.children[name]}) + } + } + walk(nil, root) + return out +} + +func sitesScope(sites []categorySite) nodeScope { + return func(fn func(path string, n node) error) error { + for _, s := range sites { + if err := eachNode(s.n, s.path, fn); err != nil { + return err + } + } + return nil + } } func inlineScope(n node, label string) nodeScope { @@ -197,17 +235,19 @@ type binding struct { typedByGo bool } -func treeBinding(root map[string]node) binding { +// categoryBinding binds the categories of one load, their references resolving +// against root. +func categoryBinding(sites []categorySite, root map[string]node) binding { return binding{ - scope: treeScope(root), + scope: sitesScope(sites), link: func() error { - setTablePaths(root) - if err := linkTables(root); err != nil { + setTablePaths(sites) + if err := linkTables(sites); err != nil { return err } - return linkRefs(root) + return linkRefs(sites, root) }, - scopeFence: func() error { return checkNoCycles(root) }, + scopeFence: func() error { return checkNoCycles(sites) }, } } @@ -229,7 +269,7 @@ func (b binding) bind() error { // checkNodeFences runs the per-node fences every binding needs over a scope, each // over the whole scope before the next, so which of several broken nodes is reported // does not depend on the walk. Its walks recurse unguarded, so the scope's cycles are -// refused first: a tree by checkNoCycles, and an inline node closes none, since nothing +// refused first: categories by checkNoCycles, and an inline node closes none, since nothing // references it. func checkNodeFences(s nodeScope) error { mem := renderCounts{} diff --git a/docs/decisions.md b/docs/decisions.md index 84e24f3..6c6159a 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -114,8 +114,19 @@ and `regexp.Match`, not two spellings of one result. A directory a machine happens to have would make `--seed 42` machine-dependent. Data still lives in `data/` as JSON; `--data-path` layers over it. -`todo.md` items 76 and 77 revise it: shipped data loads on demand from an index built at -build time, and a library imports it as Go modules. +`todo.md` item 77 revises it: a library imports the shipped data as Go modules. + +## The shipped set alone loads a category on the first call reaching it; beside a `--data-path` it loads whole + +2026-10-03, approved by the maintainer as the restructure of `todo.md` item 76. Goal 12: a +`New` pays nothing for the categories a run never reaches, which matters once the shipped +set grows to hundreds of MB. A category loads with every category it references and its +whole table family, and is bound as a whole load binds, so it renders as it would from +`New`. `List` reads `shippedindex.go`, generated from `data/` with `REPIN=1`, and +`TestShippedIndexIsCurrent` fails while it is stale. CI loads the whole shipped set, so a +shipped category never fails at first reach. Goal 6: beside a `--data-path` or a +`WithDataFS`, every category loads in `New`, so each mistake in a source the user wrote is +`New`'s error. Valid while the shipped set is proven whole in CI. ## A bare reference draws each time; a reference path is held diff --git a/drawfence_test.go b/drawfence_test.go index 85bddbf..9a1c220 100644 --- a/drawfence_test.go +++ b/drawfence_test.go @@ -31,7 +31,7 @@ type fenceRoot struct { func fenceRoots(t *testing.T, f *Generator) []fenceRoot { t.Helper() var roots []fenceRoot - _ = walkNodes(f.root.children, func(path string, n node) error { + _ = sitesScope(categorySites(&f.root))(func(path string, n node) error { if tm, ok := n.(*template); ok { roots = append(roots, fenceRoot{path, tm}) } @@ -58,7 +58,7 @@ func TestPairsFindWhatAWholeReplayFinds(t *testing.T) { corpus := fenceCorpus(t) roots := fenceRoots(t, corpus) shipped := newShippedWhole(t) - _ = walkNodes(shipped.root.children, func(path string, n node) error { + _ = sitesScope(categorySites(&shipped.root))(func(path string, n node) error { if tm, ok := n.(*template); ok { roots = append(roots, fenceRoot{path, tm}) } @@ -84,7 +84,7 @@ func TestPairsFindWhatAWholeReplayOfSelectedRowsFinds(t *testing.T) { } var reads []pathRead fold := newReadFold() - _ = walkNodes(f.root.children, func(_ string, n node) error { + _ = sitesScope(categorySites(&f.root))(func(_ string, n node) error { tb, isTable := n.(*table) if !isTable || tb.keyIndex < 0 || tb.familyRoot().path != "region" { return nil @@ -190,7 +190,7 @@ func TestReadSurveyReportsAPinClashAsAnError(t *testing.T) { if err != nil { t.Fatalf("loadDir = %v", err) } - if err := treeBinding(g.children).link(); err != nil { + if err := categoryBinding(categorySites(g), g.children).link(); err != nil { t.Fatalf("link = %v", err) } r, isTemplate := g.children["r"].(*template) diff --git a/fejkdata.go b/fejkdata.go index fce5cc6..4dcacb2 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -2,7 +2,6 @@ package fejkdata import ( crand "crypto/rand" - "embed" "encoding/binary" "errors" "fmt" @@ -14,9 +13,6 @@ import ( "sync" ) -//go:embed data -var shippedFS embed.FS - // MaxRepeat caps a repeat, and caps the renders nested repeats multiply to along // any one path; the CLI's --repeat shares it. const MaxRepeat = 1 << 20 @@ -30,9 +26,8 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD // It is safe for concurrent use; a seeded sequence is reproducible only when drawn // from one goroutine. type Generator struct { - // mu guards rand, records and structs. The tree under root is fixed after New bar each - // table's row lookup, built under its sync.Once, so List, NewTemplate and - // NewRecordTemplate read it without mu. + // mu guards rand, records, structs and root, which loads a shipped category on the + // first call reaching it. mu sync.Mutex rand *generatorState root folder // the categories as the node a path walks from, owned here so a walk allocates none @@ -97,15 +92,7 @@ func New(opts ...Option) (*Generator, error) { for _, opt := range opts { opt(&c) } - var sources []dataSource - if c.shipped { - sources = append(sources, dataSource{fsys: shippedFS, baseDir: "data"}) - } - sources = append(sources, c.sources...) - if len(sources) == 0 { - return nil, fmt.Errorf("fejkdata: %w", ErrNoData) - } - cats, err := loadData(sources) + root, err := c.load() if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } @@ -113,19 +100,35 @@ func New(opts ...Option) (*Generator, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - return &Generator{rand: rng, root: folder{children: cats}}, nil + return &Generator{rand: rng, root: root}, nil +} + +// load is the tree New starts from. The shipped data set alone loads each category on the +// first call reaching it; beside any other source, every category loads here, so each +// mistake in that source is New's error. +func (c config) load() (folder, error) { + if c.shipped && len(c.sources) == 0 { + return shelve(), nil + } + var sources []dataSource + if c.shipped { + sources = append(sources, shippedSource) + } + sources = append(sources, c.sources...) + if len(sources) == 0 { + return folder{}, ErrNoData + } + cats, err := loadData(sources) + return folder{children: cats}, err } // List returns the sorted dotted paths Fake renders: each category and every field, // column and linked table below it, a direct descent at a time. A choice consumes no // segment, so a path continues through one only where every variant carries it. func (f *Generator) List() []string { - var out []string - for _, name := range sortedNames(f.root.children) { - for _, p := range paths(f.root.children[name]) { - out = append(out, join(name, p)) - } - } + f.mu.Lock() + defer f.mu.Unlock() + out := paths(&f.root) sort.Strings(out) return out } @@ -141,6 +144,11 @@ func paths(n node) []string { out = append(out, join(name, p)) } } + for name, c := range n.unloaded { + for _, p := range c.paths { + out = append(out, join(name, p)) + } + } return out case *template: out := []string{""} diff --git a/graph.go b/graph.go index 098fe5a..3d40630 100644 --- a/graph.go +++ b/graph.go @@ -5,16 +5,6 @@ import ( "sort" ) -// walkNodes calls fn once per contained node, passing the dot path that reaches it. -func walkNodes(root map[string]node, fn func(path string, n node) error) error { - for _, name := range sortedNames(root) { - if err := eachNode(root[name], name, fn); err != nil { - return err - } - } - return nil -} - // eachNode visits n and every node contained within it once, passing the dot path // that reaches each. It never crosses a reference edge — a {/path} reference is // skipped — so a single inline node is walked on its own. @@ -196,7 +186,7 @@ func repeatCheck(path string, n node, mem renderCounts) error { // graph (renderEdges). Every node is a root: a field its parent's format never // renders is still reachable by dot path, so a cycle in one would otherwise reach // render and be fatal there. -func checkNoCycles(root map[string]node) error { +func checkNoCycles(sites []categorySite) error { const ( grey = 1 black = 2 @@ -219,5 +209,5 @@ func checkNoCycles(root map[string]node) error { color[n] = black return nil } - return walkNodes(root, func(path string, n node) error { return visit(n, path) }) + return sitesScope(sites)(func(path string, n node) error { return visit(n, path) }) } diff --git a/inline.go b/inline.go index 924a348..9547e08 100644 --- a/inline.go +++ b/inline.go @@ -32,7 +32,9 @@ func (f *Generator) NewTemplate(input string) (*Template, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - if err := inlineBinding(n, "template", f.root.children).bind(); err != nil { + f.mu.Lock() + defer f.mu.Unlock() + if err := inlineBinding(n, "template", &f.root).bind(); err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } return &Template{g: f, n: n}, nil @@ -146,11 +148,16 @@ func inputValue(input string) (any, error) { return raw, nil } -func inlineBinding(n node, label string, root map[string]node) binding { +func inlineBinding(n node, label string, root *folder) binding { scope := inlineScope(n, label) return binding{ scope: scope, - link: func() error { return linkNodeRefs(scope, root) }, + link: func() error { + if err := loadShipped(root, shippedReads(root, nil, scope)); err != nil { + return err + } + return linkNodeRefs(scope, root.children) + }, scopeFence: func() error { if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) diff --git a/node.go b/node.go index 15487e4..6a9c3f2 100644 --- a/node.go +++ b/node.go @@ -16,7 +16,10 @@ type node interface{ isNode() } // folder is a namespace of named children, built from a directory of JSON files // and subdirectories. It has no value of its own: descend into a named child by // dot path; rendering one is an error (see Fake). -type folder struct{ children map[string]node } +type folder struct { + children map[string]node + unloaded map[string]shippedCategory // the shipped categories here a call has not reached yet +} func (*folder) isNode() {} diff --git a/reads_test.go b/reads_test.go index 29dd106..0d2ca07 100644 --- a/reads_test.go +++ b/reads_test.go @@ -118,7 +118,7 @@ func TestReadFoldWalksACellByEveryRoute(t *testing.T) { if err != nil { t.Fatalf("loadDir = %v", err) } - if err := treeBinding(g.children).link(); err != nil { + if err := categoryBinding(categorySites(g), g.children).link(); err != nil { t.Fatalf("link = %v", err) } r, isTemplate := g.children["r"].(*template) diff --git a/record.go b/record.go index 5d5db99..40b8c31 100644 --- a/record.go +++ b/record.go @@ -126,6 +126,9 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } + if err := f.loadShippedAt(segments); err != nil { + return nil, fmt.Errorf("fejkdata: %w", err) + } _, n, tail, err := resolveCategory(f.root.children, segments) if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) diff --git a/reference.go b/reference.go index 23bd9ab..c6c3af9 100644 --- a/reference.go +++ b/reference.go @@ -71,17 +71,16 @@ func refSegments(name string, folder []string) ([]string, error) { // so a reference sees the override-resolved tree. A reference's head binds into // refHeads under its root path and its tail reads like a sibling path, so two // spellings of one target are one draw. -func linkRefs(root map[string]node) error { - if err := eachTemplate(root, func(folder []string, path string, t *template) error { - category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") - return linkTemplate(folder, path, category, t, root) +func linkRefs(sites []categorySite, root map[string]node) error { + if err := eachTemplate(sites, func(s categorySite, path string, t *template) error { + return linkTemplate(s.dir, path, s.path, t, root) }); err != nil { return err } - if err := eachTemplate(root, func(_ []string, path string, t *template) error { return linkNames(path, t) }); err != nil { + if err := eachTemplate(sites, func(_ categorySite, path string, t *template) error { return linkNames(path, t) }); err != nil { return err } - return eachTemplate(root, func(_ []string, path string, t *template) error { return checkNameReads(path, t) }) + return eachTemplate(sites, func(_ categorySite, path string, t *template) error { return checkNameReads(path, t) }) } // linkTemplate resolves t's references in category, "" for an inline template, and @@ -196,40 +195,20 @@ func loneRef(toks []formatToken) (string, bool) { return toks[0].names[0], true } -// eachTemplate calls fn once per template, with the folder its category sits in -// and the dot path reaching it, folders and names in sorted order. -func eachTemplate(root map[string]node, fn func(folder []string, path string, t *template) error) error { - var inCategory func(folder []string, path string, n node) error - inCategory = func(folder []string, path string, n node) error { - if t, ok := n.(*template); ok { - if err := fn(folder, path, t); err != nil { - return err +// eachTemplate calls fn once per template of the categories, with the category it sits +// in and the dot path reaching it. +func eachTemplate(sites []categorySite, fn func(s categorySite, path string, t *template) error) error { + for _, s := range sites { + if err := eachNode(s.n, s.path, func(path string, n node) error { + if t, isTemplate := n.(*template); isTemplate { + return fn(s, path, t) } + return nil + }); err != nil { + return err } - for _, c := range contained(n) { - if err := inCategory(folder, join(path, c.name), c.node); err != nil { - return err - } - } - return nil } - var inFolder func(dir []string, children map[string]node) error - inFolder = func(dir []string, children map[string]node) error { - for _, name := range sortedNames(children) { - path := join(strings.Join(dir, "."), name) - if g, ok := children[name].(*folder); ok { - if err := inFolder(append(dir[:len(dir):len(dir)], name), g.children); err != nil { - return err - } - continue - } - if err := inCategory(dir, path, children[name]); err != nil { - return err - } - } - return nil - } - return inFolder(nil, root) + return nil } // resolveCategory walks a dotted path through the folders to the category it diff --git a/render.go b/render.go index 2943416..2d306cf 100644 --- a/render.go +++ b/render.go @@ -24,6 +24,9 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } + if err := f.loadShippedAt(segments); err != nil { + return "", fmt.Errorf("fejkdata: %w", err) + } var draws renderDraws sc := renderScope{draws: &draws} n, err := descend(f.rand, &f.root, segments, sc) diff --git a/shipped.go b/shipped.go new file mode 100644 index 0000000..1cabd5f --- /dev/null +++ b/shipped.go @@ -0,0 +1,168 @@ +package fejkdata + +import ( + "embed" + "fmt" + "io/fs" + "path" + "sort" + "strings" +) + +//go:embed data +var shippedFS embed.FS + +var shippedSource = dataSource{fsys: shippedFS, baseDir: "data"} + +// shippedCategory is a shipped category as shippedindex.go lists it: the table beside it +// that it links to, "" for none, and the paths List advertises below it. +type shippedCategory struct { + parent string + paths []string +} + +// shelve builds the folders of the shipped index, with every category in them unloaded. +func shelve() folder { + root := folder{children: map[string]node{}} + for p, c := range shippedIndex { + segs := strings.Split(p, ".") + g := &root + for _, seg := range segs[:len(segs)-1] { + sub, isFolder := g.children[seg].(*folder) + if !isFolder { + sub = &folder{children: map[string]node{}} + g.children[seg] = sub + } + g = sub + } + if g.unloaded == nil { + g.unloaded = map[string]shippedCategory{} + } + g.unloaded[segs[len(segs)-1]] = c + } + return root +} + +// pendingLoad is an unloaded shipped category: the folder holding it, that folder's path, +// and its name there. +type pendingLoad struct { + dir []string + in *folder + name string +} + +// unloadedAt is the unloaded shipped category a path from root names, or descends into. +func unloadedAt(root *folder, segs []string) (pendingLoad, bool) { + g := root + for i, seg := range segs { + switch n := g.children[seg].(type) { + case *folder: + g = n + continue + case nil: + if _, unloaded := g.unloaded[seg]; unloaded { + return pendingLoad{dir: segs[:i:i], in: g, name: seg}, true + } + } + return pendingLoad{}, false + } + return pendingLoad{}, false +} + +// loadShippedAt loads the shipped category a caller's path names or descends into, unless +// it is loaded already. +func (f *Generator) loadShippedAt(segs []string) error { + if p, unloaded := unloadedAt(&f.root, segs); unloaded { + return loadShipped(&f.root, []pendingLoad{p}) + } + return nil +} + +// shippedReads is every unloaded shipped category the templates of scope reference, each +// reference read from the folder dir. +func shippedReads(root *folder, dir []string, scope nodeScope) []pendingLoad { + var out []pendingLoad + _ = scope(func(_ string, n node) error { + t, isTemplate := n.(*template) + if !isTemplate { + return nil + } + for _, name := range refTokens(t.tokens) { + segs, err := refSegments(name, dir) + if err != nil { + continue // the link reports it + } + if p, unloaded := unloadedAt(root, segs); unloaded { + out = append(out, p) + } + } + return nil + }) + return out +} + +// loadShipped loads the shipped categories wanted, with every category they read and the +// whole table family of each, then binds what it loaded, as New binds a whole load. A failed +// bind leaves every category it loaded unloaded again. +func loadShipped(root *folder, wanted []pendingLoad) (err error) { + var loaded []pendingLoad + var entries []shippedCategory + defer func() { + if err == nil { + return + } + for i, p := range loaded { + delete(p.in.children, p.name) + p.in.unloaded[p.name] = entries[i] + } + }() + var sites []categorySite + for queue := wanted; len(queue) > 0; queue = queue[1:] { + p := queue[0] + c, unloaded := p.in.unloaded[p.name] + if !unloaded { + continue + } + site, err := p.load() + if err != nil { + return err + } + loaded, entries = append(loaded, p), append(entries, c) + sites = append(sites, site) + queue = append(queue, shippedReads(root, p.dir, sitesScope([]categorySite{site}))...) + queue = append(queue, p.family(c)...) + } + sort.Slice(sites, func(i, j int) bool { return sites[i].path < sites[j].path }) + return categoryBinding(sites, root.children).bind() +} + +// load parses and compiles the category, and moves it from its folder's unloaded +// categories to its children. +func (p pendingLoad) load() (categorySite, error) { + dir := path.Join(append([]string{shippedSource.baseDir}, p.dir...)...) + entries, err := fs.ReadDir(shippedSource.fsys, dir) + if err != nil { + return categorySite{}, fmt.Errorf("%s: %w", dir, err) + } + file := p.name + ".json" + if err := loadFile(shippedSource, p.in, path.Join(dir, file), file, newCategoryFiles(shippedSource, dir, entries)); err != nil { + return categorySite{}, err + } + delete(p.in.unloaded, p.name) + return categorySite{dir: p.dir, path: join(strings.Join(p.dir, "."), p.name), in: p.in, n: p.in.children[p.name]}, nil +} + +// family is the tables beside p that a table family links it to: its parent, and every +// table whose parent it is. +func (p pendingLoad) family(c shippedCategory) []pendingLoad { + var out []pendingLoad + if c.parent != "" { + out = append(out, pendingLoad{dir: p.dir, in: p.in, name: c.parent}) + } + for name, sibling := range p.in.unloaded { + if sibling.parent == p.name { + out = append(out, pendingLoad{dir: p.dir, in: p.in, name: name}) + } + } + return out +} diff --git a/shippedindex.go b/shippedindex.go new file mode 100644 index 0000000..e9fb3a7 --- /dev/null +++ b/shippedindex.go @@ -0,0 +1,81 @@ +// Code generated by TestShippedIndexIsCurrent with REPIN=1. DO NOT EDIT. + +package fejkdata + +var shippedIndex = map[string]shippedCategory{ + "en_US.address": {paths: []string{"", "locality", "postal-code", "region", "street", "street-number"}}, + "en_US.color": {paths: []string{""}}, + "en_US.company": {paths: []string{"", "base", "suffix"}}, + "en_US.date": {paths: []string{""}}, + "en_US.email": {paths: []string{"", "domain", "local", "local.n"}}, + "en_US.first-name": {parent: "sex", paths: []string{"", "count", "name", "sex"}}, + "en_US.ip": {paths: []string{""}}, + "en_US.itin": {paths: []string{"", "area", "group", "serial"}}, + "en_US.last-name": {paths: []string{"", "count", "name"}}, + "en_US.person": {paths: []string{"", "first", "last", "prefix", "sex"}}, + "en_US.phone": {paths: []string{"", "area", "exch", "line"}}, + "en_US.price": {paths: []string{"", "amt", "cents"}}, + "en_US.sentence": {paths: []string{"", "adj", "noun", "prep", "verb"}}, + "en_US.sex": {paths: []string{"", "code", "first-name", "first-name.count", "first-name.name", "first-name.sex", "name", "title", "title.name", "title.sex", "title.share"}}, + "en_US.ssn": {paths: []string{"", "area", "group", "serial"}}, + "en_US.time": {paths: []string{""}}, + "en_US.title": {parent: "sex", paths: []string{"", "name", "sex", "share"}}, + "en_US.url": {paths: []string{"", "host", "path"}}, + "en_US.username": {paths: []string{"", "n"}}, + "en_US.version": {paths: []string{"", "n", "pre", "suffix"}}, + "en_US.word": {paths: []string{""}}, + "geo.SE.address": {paths: []string{"", "locality", "postal-code", "street", "street-number"}}, + "geo.SE.locality": {parent: "municipality", paths: []string{"", "lat", "lon", "municipality", "name", "population", "postal-code", "postal-code.code", "postal-code.locality", "street", "street.locality", "street.name", "street.segments"}}, + "geo.SE.municipality": {parent: "region", paths: []string{"", "code", "locality", "locality.lat", "locality.lon", "locality.municipality", "locality.name", "locality.population", "locality.postal-code", "locality.postal-code.code", "locality.postal-code.locality", "locality.street", "locality.street.locality", "locality.street.name", "locality.street.segments", "name", "population", "region"}}, + "geo.SE.postal-code": {parent: "locality", paths: []string{"", "code", "locality"}}, + "geo.SE.region": {paths: []string{"", "code", "municipality", "municipality.code", "municipality.locality", "municipality.locality.lat", "municipality.locality.lon", "municipality.locality.municipality", "municipality.locality.name", "municipality.locality.population", "municipality.locality.postal-code", "municipality.locality.postal-code.code", "municipality.locality.postal-code.locality", "municipality.locality.street", "municipality.locality.street.locality", "municipality.locality.street.name", "municipality.locality.street.segments", "municipality.name", "municipality.population", "municipality.region", "name", "population", "timezone"}}, + "geo.SE.street": {parent: "locality", paths: []string{"", "locality", "name", "segments"}}, + "geo.US.address": {paths: []string{"", "locality", "postal-code", "region", "street", "street-number"}}, + "geo.US.locality": {parent: "municipality", paths: []string{"", "code", "lat", "lon", "municipality", "name", "population", "postal-code", "postal-code.addresses", "postal-code.code", "postal-code.locality", "street", "street.addresses", "street.locality", "street.name"}}, + "geo.US.municipality": {parent: "region", paths: []string{"", "code", "locality", "locality.code", "locality.lat", "locality.lon", "locality.municipality", "locality.name", "locality.population", "locality.postal-code", "locality.postal-code.addresses", "locality.postal-code.code", "locality.postal-code.locality", "locality.street", "locality.street.addresses", "locality.street.locality", "locality.street.name", "name", "population", "region"}}, + "geo.US.postal-code": {parent: "locality", paths: []string{"", "addresses", "code", "locality"}}, + "geo.US.region": {paths: []string{"", "abbr", "code", "municipality", "municipality.code", "municipality.locality", "municipality.locality.code", "municipality.locality.lat", "municipality.locality.lon", "municipality.locality.municipality", "municipality.locality.name", "municipality.locality.population", "municipality.locality.postal-code", "municipality.locality.postal-code.addresses", "municipality.locality.postal-code.code", "municipality.locality.postal-code.locality", "municipality.locality.street", "municipality.locality.street.addresses", "municipality.locality.street.locality", "municipality.locality.street.name", "municipality.name", "municipality.population", "municipality.region", "name", "population", "timezone"}}, + "geo.US.street": {parent: "locality", paths: []string{"", "addresses", "locality", "name"}}, + "misc.car": {paths: []string{"", "make", "model"}}, + "misc.coordinate": {paths: []string{"", "lat", "lon"}}, + "misc.creditcard": {paths: []string{"", "d"}}, + "misc.currency": {paths: []string{"", "code", "decimals", "name", "numeric", "symbol"}}, + "misc.datetime": {paths: []string{""}}, + "misc.emoji": {paths: []string{""}}, + "misc.httpmethod": {paths: []string{"", "idempotent", "method", "safe"}}, + "misc.httpstatus": {paths: []string{"", "code", "reason"}}, + "misc.language": {paths: []string{"", "code", "code3", "name"}}, + "misc.loglevel": {paths: []string{"", "code", "keyword", "severity"}}, + "misc.mac": {paths: []string{""}}, + "misc.mimetype": {paths: []string{"", "ext", "type"}}, + "misc.objectid": {paths: []string{""}}, + "misc.port": {paths: []string{"", "number", "service"}}, + "misc.protocol": {paths: []string{"", "keyword", "name", "number"}}, + "misc.territory": {paths: []string{"", "alpha2", "alpha3", "calling-code", "capital", "country", "currency", "flag", "languages", "name", "numeric", "timezone", "timezone.offset", "timezone.territory", "timezone.weight", "timezone.zone", "tld"}}, + "misc.timezone": {parent: "territory", paths: []string{"", "offset", "territory", "weight", "zone"}}, + "misc.tld": {paths: []string{"", "tld", "type", "unicode"}}, + "misc.useragent": {paths: []string{"", "browser", "device", "os", "ua"}}, + "misc.uuid": {paths: []string{"", "variant"}}, + "sv_SE.address": {paths: []string{"", "locality", "postal-code", "street", "street-number"}}, + "sv_SE.birth-number": {parent: "sex", paths: []string{"", "number", "sex"}}, + "sv_SE.color": {paths: []string{""}}, + "sv_SE.company": {paths: []string{"", "base", "suffix"}}, + "sv_SE.date": {paths: []string{""}}, + "sv_SE.email": {paths: []string{"", "domain", "local", "local.n"}}, + "sv_SE.first-name": {parent: "sex", paths: []string{"", "count", "name", "sex"}}, + "sv_SE.ip": {paths: []string{""}}, + "sv_SE.last-name": {paths: []string{"", "count", "name"}}, + "sv_SE.person": {paths: []string{"", "first", "last", "prefix", "sex"}}, + "sv_SE.personnummer": {paths: []string{""}}, + "sv_SE.phone": {paths: []string{"", "a", "b", "c", "prefix"}}, + "sv_SE.price": {paths: []string{"", "amt", "ore"}}, + "sv_SE.samordningsnummer": {paths: []string{""}}, + "sv_SE.sentence": {paths: []string{"", "adj", "noun", "prep", "verb"}}, + "sv_SE.sex": {paths: []string{"", "birth-number", "birth-number.number", "birth-number.sex", "code", "first-name", "first-name.count", "first-name.name", "first-name.sex", "name"}}, + "sv_SE.time": {paths: []string{""}}, + "sv_SE.title": {paths: []string{"", "name", "share"}}, + "sv_SE.url": {paths: []string{"", "host", "path"}}, + "sv_SE.username": {paths: []string{"", "n"}}, + "sv_SE.version": {paths: []string{"", "n", "pre", "suffix"}}, + "sv_SE.word": {paths: []string{""}}, +} diff --git a/struct.go b/struct.go index 3016a68..0ee799a 100644 --- a/struct.go +++ b/struct.go @@ -49,7 +49,7 @@ func (f *Generator) structShapeOf(t reflect.Type) (*structShape, error) { if label == "" { label = "struct" } - sc := &structCompile{categories: f.root.children, visiting: map[reflect.Type]bool{}, structsLeft: maxStructs} + sc := &structCompile{root: &f.root, visiting: map[reflect.Type]bool{}, structsLeft: maxStructs} shape, err := sc.compileShape(t, label) if err == nil && shape.empty() { err = fmt.Errorf("%s has no fake tags, so nothing to fill", t) @@ -82,7 +82,7 @@ func (s *structShape) empty() bool { return s.record == nil && len(s.nested) == // loaded tree, the types compiling or embedded above, so a pointer back to one is left alone // rather than filled without end, and how many more structs it may walk. type structCompile struct { - categories map[string]node + root *folder visiting map[reflect.Type]bool structsLeft int } @@ -109,7 +109,7 @@ func (sc *structCompile) compileShape(t reflect.Type, label string) (*structShap return nil, err } if len(c.tags) > 0 { - if err := c.shape.compileRecord(sc.categories, t, label, c.tags); err != nil { + if err := c.shape.compileRecord(sc.root, t, label, c.tags); err != nil { return nil, err } } @@ -252,13 +252,13 @@ func checkTaggedType(sf reflect.StructField) error { // compileRecord compiles the tagged fields of t as one record, and proves each column holds // only what its field's Go type can. -func (s *structShape) compileRecord(categories map[string]node, t reflect.Type, label string, tags map[string]any) error { +func (s *structShape) compileRecord(root *folder, t reflect.Type, label string, tags map[string]any) error { tags["format"] = "" n, err := compile(tags) if err != nil { return fmt.Errorf("%s: %w", label, err) } - b := inlineBinding(n, label, categories) + b := inlineBinding(n, label, root) b.typedByGo = true if err := b.bind(); err != nil { return err diff --git a/table.go b/table.go index c23a513..97e256d 100644 --- a/table.go +++ b/table.go @@ -365,47 +365,29 @@ func refuseTableBinding(toks []formatToken) error { // setTablePaths gives every table the path a selector on it is written at, before a // link or a draw can name one. -func setTablePaths(root map[string]node) { - var walk func(dir string, children map[string]node) - walk = func(dir string, children map[string]node) { - for name, child := range children { - switch n := child.(type) { - case *folder: - walk(join(dir, name), n.children) - case *table: - n.path = join(dir, name) - } +func setTablePaths(sites []categorySite) { + for _, s := range sites { + if t, isTable := s.n.(*table); isTable { + t.path = s.path } } - walk("", root) } // linkTables binds every table's parent to the table beside it, and proves the // links: a parent has a key, every link cell is one, every parent row is linked // to, no chain of parents closes, and no child is named like a parent's column. // docs/decisions.md#a-parent-row-with-no-child-row-is-a-load-error -func linkTables(root map[string]node) error { - var walk func(dir string, children map[string]node) error - walk = func(dir string, children map[string]node) error { - for _, name := range sortedNames(children) { - path := join(dir, name) - switch n := children[name].(type) { - case *folder: - if err := walk(path, n.children); err != nil { - return err - } - case *table: - if n.parentIndex < 0 { - continue - } - if err := n.linkParent(path, children); err != nil { - return fmt.Errorf("%s: %w", path, err) - } - } +func linkTables(sites []categorySite) error { + for _, s := range sites { + t, isTable := s.n.(*table) + if !isTable || t.parentIndex < 0 { + continue + } + if err := t.linkParent(s.path, s.in.children); err != nil { + return fmt.Errorf("%s: %w", s.path, err) } - return nil } - return walk("", root) + return nil } func (t *table) linkParent(path string, siblings map[string]node) error { -- 2.52.0 From 81bb9791647196e8852c0c5a602bb43b42aac3e7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:20:36 +0200 Subject: [PATCH 454/966] Test that every shipped category loads alone into a fresh New --- shipped_test.go | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/shipped_test.go b/shipped_test.go index 13145dc..84497ad 100644 --- a/shipped_test.go +++ b/shipped_test.go @@ -33,17 +33,9 @@ func newShippedWhole(t testing.TB, opts ...Option) *Generator { // loadedCategories is the dot path of every category f has parsed and linked. func loadedCategories(f *Generator) []string { var out []string - var walk func(prefix string, g *folder) - walk = func(prefix string, g *folder) { - for _, name := range sortedNames(g.children) { - if sub, isFolder := g.children[name].(*folder); isFolder { - walk(join(prefix, name), sub) - continue - } - out = append(out, join(prefix, name)) - } + for _, s := range categorySites(&f.root) { + out = append(out, s.path) } - walk("", &f.root) return out } @@ -70,7 +62,7 @@ func TestShippedIndexIsCurrent(t *testing.T) { func shippedIndexSource(whole *Generator) ([]byte, error) { var b bytes.Buffer - b.WriteString("// Code generated by TestShippedIndexIsCurrent with REPIN=1. DO NOT EDIT.\n\npackage fejkdata\n\nvar shippedIndex = map[string]shippedCategory{\n") + b.WriteString("// Code generated by TestShippedIndexIsCurrent with REPIN=1. DO NOT EDIT.\n\npackage fejkdata\n\nvar shippedIndex = map[string]shippedEntry{\n") for _, s := range categorySites(&whole.root) { fmt.Fprintf(&b, "%q: {", s.path) if t, isTable := s.n.(*table); isTable && t.parentIndex >= 0 { @@ -186,6 +178,18 @@ func TestOnDemandRendersAsTheWholeLoad(t *testing.T) { } } +func TestEveryShippedCategoryLoadsAlone(t *testing.T) { + for p := range shippedIndex { + f, err := New() + if err != nil { + t.Fatalf("New() = %v", err) + } + if _, err := f.Fake(p); err != nil { + t.Errorf("Fake(%q) on a fresh New() = %v", p, err) + } + } +} + func TestListRunsBesideAnOnDemandLoad(t *testing.T) { f, err := New() if err != nil { -- 2.52.0 From 8b3bbc4003643b60a78cf69a4335b1d829098550 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:20:36 +0200 Subject: [PATCH 455/966] Load shipped categories before binding, panic on an index that misses a read, and name the index entry and unloaded category plainly --- AGENTS.md | 2 +- docs/decisions.md | 8 ++-- fejkdata.go | 6 +-- inline.go | 12 ++--- node.go | 2 +- record.go | 4 +- render.go | 4 +- shipped.go | 112 +++++++++++++++++++++------------------------- shippedindex.go | 2 +- struct.go | 3 +- 10 files changed, 70 insertions(+), 85 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b38477a..bc4a4a9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -43,7 +43,7 @@ In [docs/decisions.md](docs/decisions.md): - A padded JSON argument is rejected, not trimmed - `FakeTemplate` and `NewTemplate` both stay - The shipped data is embedded, not discovered -- The shipped set alone loads a category on the first call reaching it; beside a `--data-path` it loads whole +- The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` - A bare reference draws each time; a reference path is held - Reference sigils follow the filesystem - A change to what exists is a major; a minor only adds diff --git a/docs/decisions.md b/docs/decisions.md index 6c6159a..07c17d6 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -116,17 +116,17 @@ still lives in `data/` as JSON; `--data-path` layers over it. `todo.md` item 77 revises it: a library imports the shipped data as Go modules. -## The shipped set alone loads a category on the first call reaching it; beside a `--data-path` it loads whole +## The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` 2026-10-03, approved by the maintainer as the restructure of `todo.md` item 76. Goal 12: a `New` pays nothing for the categories a run never reaches, which matters once the shipped set grows to hundreds of MB. A category loads with every category it references and its whole table family, and is bound as a whole load binds, so it renders as it would from `New`. `List` reads `shippedindex.go`, generated from `data/` with `REPIN=1`, and -`TestShippedIndexIsCurrent` fails while it is stale. CI loads the whole shipped set, so a -shipped category never fails at first reach. Goal 6: beside a `--data-path` or a +`TestShippedIndexIsCurrent` fails while it is stale. CI loads the whole shipped set, and +loads each shipped category alone into a fresh `New`, so none fails at first reach. Goal 6: beside a `--data-path` or a `WithDataFS`, every category loads in `New`, so each mistake in a source the user wrote is -`New`'s error. Valid while the shipped set is proven whole in CI. +`New`'s error. Valid while CI proves the shipped set whole and each category alone. ## A bare reference draws each time; a reference path is held diff --git a/fejkdata.go b/fejkdata.go index 4dcacb2..8108780 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -108,7 +108,7 @@ func New(opts ...Option) (*Generator, error) { // mistake in that source is New's error. func (c config) load() (folder, error) { if c.shipped && len(c.sources) == 0 { - return shelve(), nil + return unloadedTree(), nil } var sources []dataSource if c.shipped { @@ -144,8 +144,8 @@ func paths(n node) []string { out = append(out, join(name, p)) } } - for name, c := range n.unloaded { - for _, p := range c.paths { + for name, e := range n.unloaded { + for _, p := range e.paths { out = append(out, join(name, p)) } } diff --git a/inline.go b/inline.go index 9547e08..f0adbd7 100644 --- a/inline.go +++ b/inline.go @@ -34,7 +34,8 @@ func (f *Generator) NewTemplate(input string) (*Template, error) { } f.mu.Lock() defer f.mu.Unlock() - if err := inlineBinding(n, "template", &f.root).bind(); err != nil { + loadShipped(&f.root, unloadedReads(&f.root, nil, inlineScope(n, "template"))) + if err := inlineBinding(n, "template", f.root.children).bind(); err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } return &Template{g: f, n: n}, nil @@ -148,16 +149,11 @@ func inputValue(input string) (any, error) { return raw, nil } -func inlineBinding(n node, label string, root *folder) binding { +func inlineBinding(n node, label string, root map[string]node) binding { scope := inlineScope(n, label) return binding{ scope: scope, - link: func() error { - if err := loadShipped(root, shippedReads(root, nil, scope)); err != nil { - return err - } - return linkNodeRefs(scope, root.children) - }, + link: func() error { return linkNodeRefs(scope, root) }, scopeFence: func() error { if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) diff --git a/node.go b/node.go index 6a9c3f2..822ab3a 100644 --- a/node.go +++ b/node.go @@ -18,7 +18,7 @@ type node interface{ isNode() } // dot path; rendering one is an error (see Fake). type folder struct { children map[string]node - unloaded map[string]shippedCategory // the shipped categories here a call has not reached yet + unloaded map[string]shippedEntry // the shipped categories here a call has not reached yet } func (*folder) isNode() {} diff --git a/record.go b/record.go index 40b8c31..b459301 100644 --- a/record.go +++ b/record.go @@ -126,9 +126,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { if err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } - if err := f.loadShippedAt(segments); err != nil { - return nil, fmt.Errorf("fejkdata: %w", err) - } + f.loadShippedAt(segments) _, n, tail, err := resolveCategory(f.root.children, segments) if err != nil { return nil, fmt.Errorf("fejkdata: %s: %w", path, err) diff --git a/render.go b/render.go index 2d306cf..d08dc04 100644 --- a/render.go +++ b/render.go @@ -24,9 +24,7 @@ func (f *Generator) Fake(path string) (string, error) { if err != nil { return "", fmt.Errorf("fejkdata: %w", err) } - if err := f.loadShippedAt(segments); err != nil { - return "", fmt.Errorf("fejkdata: %w", err) - } + f.loadShippedAt(segments) var draws renderDraws sc := renderScope{draws: &draws} n, err := descend(f.rand, &f.root, segments, sc) diff --git a/shipped.go b/shipped.go index 1cabd5f..a7c7525 100644 --- a/shipped.go +++ b/shipped.go @@ -14,17 +14,17 @@ var shippedFS embed.FS var shippedSource = dataSource{fsys: shippedFS, baseDir: "data"} -// shippedCategory is a shipped category as shippedindex.go lists it: the table beside it -// that it links to, "" for none, and the paths List advertises below it. -type shippedCategory struct { +// shippedEntry is a shipped category's entry in shippedindex.go: the table beside it that +// it links to, "" for none, and the paths List advertises below it. +type shippedEntry struct { parent string paths []string } -// shelve builds the folders of the shipped index, with every category in them unloaded. -func shelve() folder { +// unloadedTree builds the folders of the shipped index, with every category in them unloaded. +func unloadedTree() folder { root := folder{children: map[string]node{}} - for p, c := range shippedIndex { + for p, e := range shippedIndex { segs := strings.Split(p, ".") g := &root for _, seg := range segs[:len(segs)-1] { @@ -36,23 +36,23 @@ func shelve() folder { g = sub } if g.unloaded == nil { - g.unloaded = map[string]shippedCategory{} + g.unloaded = map[string]shippedEntry{} } - g.unloaded[segs[len(segs)-1]] = c + g.unloaded[segs[len(segs)-1]] = e } return root } -// pendingLoad is an unloaded shipped category: the folder holding it, that folder's path, -// and its name there. -type pendingLoad struct { +// unloadedCategory is a shipped category not loaded yet: the folder holding it, that +// folder's path, and its name there. +type unloadedCategory struct { dir []string in *folder name string } // unloadedAt is the unloaded shipped category a path from root names, or descends into. -func unloadedAt(root *folder, segs []string) (pendingLoad, bool) { +func unloadedAt(root *folder, segs []string) (unloadedCategory, bool) { g := root for i, seg := range segs { switch n := g.children[seg].(type) { @@ -61,27 +61,26 @@ func unloadedAt(root *folder, segs []string) (pendingLoad, bool) { continue case nil: if _, unloaded := g.unloaded[seg]; unloaded { - return pendingLoad{dir: segs[:i:i], in: g, name: seg}, true + return unloadedCategory{dir: segs[:i:i], in: g, name: seg}, true } } - return pendingLoad{}, false + return unloadedCategory{}, false } - return pendingLoad{}, false + return unloadedCategory{}, false } // loadShippedAt loads the shipped category a caller's path names or descends into, unless // it is loaded already. -func (f *Generator) loadShippedAt(segs []string) error { - if p, unloaded := unloadedAt(&f.root, segs); unloaded { - return loadShipped(&f.root, []pendingLoad{p}) +func (f *Generator) loadShippedAt(segs []string) { + if u, unloaded := unloadedAt(&f.root, segs); unloaded { + loadShipped(&f.root, []unloadedCategory{u}) } - return nil } -// shippedReads is every unloaded shipped category the templates of scope reference, each +// unloadedReads is every unloaded shipped category the templates of scope reference, each // reference read from the folder dir. -func shippedReads(root *folder, dir []string, scope nodeScope) []pendingLoad { - var out []pendingLoad +func unloadedReads(root *folder, dir []string, scope nodeScope) []unloadedCategory { + var out []unloadedCategory _ = scope(func(_ string, n node) error { t, isTemplate := n.(*template) if !isTemplate { @@ -92,8 +91,8 @@ func shippedReads(root *folder, dir []string, scope nodeScope) []pendingLoad { if err != nil { continue // the link reports it } - if p, unloaded := unloadedAt(root, segs); unloaded { - out = append(out, p) + if u, unloaded := unloadedAt(root, segs); unloaded { + out = append(out, u) } } return nil @@ -102,66 +101,59 @@ func shippedReads(root *folder, dir []string, scope nodeScope) []pendingLoad { } // loadShipped loads the shipped categories wanted, with every category they read and the -// whole table family of each, then binds what it loaded, as New binds a whole load. A failed -// bind leaves every category it loaded unloaded again. -func loadShipped(root *folder, wanted []pendingLoad) (err error) { - var loaded []pendingLoad - var entries []shippedCategory - defer func() { - if err == nil { - return - } - for i, p := range loaded { - delete(p.in.children, p.name) - p.in.unloaded[p.name] = entries[i] - } - }() +// whole table family of each, then binds what it loaded, as New binds a whole load. +// TestEveryShippedCategoryLoadsAlone proves each closure, so a failure is the index's. +func loadShipped(root *folder, wanted []unloadedCategory) { var sites []categorySite for queue := wanted; len(queue) > 0; queue = queue[1:] { - p := queue[0] - c, unloaded := p.in.unloaded[p.name] + u := queue[0] + e, unloaded := u.in.unloaded[u.name] if !unloaded { continue } - site, err := p.load() + site, err := u.load() if err != nil { - return err + panic(internalError("shipped %s: %v", u.name, err)) } - loaded, entries = append(loaded, p), append(entries, c) sites = append(sites, site) - queue = append(queue, shippedReads(root, p.dir, sitesScope([]categorySite{site}))...) - queue = append(queue, p.family(c)...) + queue = append(queue, unloadedReads(root, u.dir, sitesScope([]categorySite{site}))...) + queue = append(queue, u.family(e)...) + } + if len(sites) == 0 { + return } sort.Slice(sites, func(i, j int) bool { return sites[i].path < sites[j].path }) - return categoryBinding(sites, root.children).bind() + if err := categoryBinding(sites, root.children).bind(); err != nil { + panic(internalError("binding shipped categories: %v", err)) + } } // load parses and compiles the category, and moves it from its folder's unloaded // categories to its children. -func (p pendingLoad) load() (categorySite, error) { - dir := path.Join(append([]string{shippedSource.baseDir}, p.dir...)...) +func (u unloadedCategory) load() (categorySite, error) { + dir := path.Join(append([]string{shippedSource.baseDir}, u.dir...)...) entries, err := fs.ReadDir(shippedSource.fsys, dir) if err != nil { return categorySite{}, fmt.Errorf("%s: %w", dir, err) } - file := p.name + ".json" - if err := loadFile(shippedSource, p.in, path.Join(dir, file), file, newCategoryFiles(shippedSource, dir, entries)); err != nil { + file := u.name + ".json" + if err := loadFile(shippedSource, u.in, path.Join(dir, file), file, newCategoryFiles(shippedSource, dir, entries)); err != nil { return categorySite{}, err } - delete(p.in.unloaded, p.name) - return categorySite{dir: p.dir, path: join(strings.Join(p.dir, "."), p.name), in: p.in, n: p.in.children[p.name]}, nil + delete(u.in.unloaded, u.name) + return categorySite{dir: u.dir, path: join(strings.Join(u.dir, "."), u.name), in: u.in, n: u.in.children[u.name]}, nil } -// family is the tables beside p that a table family links it to: its parent, and every +// family is the tables beside u that a table family links it to: its parent, and every // table whose parent it is. -func (p pendingLoad) family(c shippedCategory) []pendingLoad { - var out []pendingLoad - if c.parent != "" { - out = append(out, pendingLoad{dir: p.dir, in: p.in, name: c.parent}) +func (u unloadedCategory) family(e shippedEntry) []unloadedCategory { + var out []unloadedCategory + if e.parent != "" { + out = append(out, unloadedCategory{dir: u.dir, in: u.in, name: e.parent}) } - for name, sibling := range p.in.unloaded { - if sibling.parent == p.name { - out = append(out, pendingLoad{dir: p.dir, in: p.in, name: name}) + for name, sibling := range u.in.unloaded { + if sibling.parent == u.name { + out = append(out, unloadedCategory{dir: u.dir, in: u.in, name: name}) } } return out diff --git a/shippedindex.go b/shippedindex.go index e9fb3a7..6e73c43 100644 --- a/shippedindex.go +++ b/shippedindex.go @@ -2,7 +2,7 @@ package fejkdata -var shippedIndex = map[string]shippedCategory{ +var shippedIndex = map[string]shippedEntry{ "en_US.address": {paths: []string{"", "locality", "postal-code", "region", "street", "street-number"}}, "en_US.color": {paths: []string{""}}, "en_US.company": {paths: []string{"", "base", "suffix"}}, diff --git a/struct.go b/struct.go index 0ee799a..0f74bf7 100644 --- a/struct.go +++ b/struct.go @@ -258,7 +258,8 @@ func (s *structShape) compileRecord(root *folder, t reflect.Type, label string, if err != nil { return fmt.Errorf("%s: %w", label, err) } - b := inlineBinding(n, label, root) + loadShipped(root, unloadedReads(root, nil, inlineScope(n, label))) + b := inlineBinding(n, label, root.children) b.typedByGo = true if err := b.bind(); err != nil { return err -- 2.52.0 From 77fb7b69f6daf00f6934a9d8ce7a71ad1e994ee6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:31:39 +0200 Subject: [PATCH 456/966] Benchmark a fresh generator's first Fake, where shipped loading now costs --- bench_test.go | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/bench_test.go b/bench_test.go index 585b132..9595833 100644 --- a/bench_test.go +++ b/bench_test.go @@ -86,6 +86,19 @@ func BenchmarkHeldWide(b *testing.B) { benchPath(b, dir, "row") } +func BenchmarkFirstFake(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + f, err := New(WithSeed(1)) + if err != nil { + b.Fatal(err) + } + if _, err := f.Fake("sv_SE.address"); err != nil { + b.Fatal(err) + } + } +} + func BenchmarkNew(b *testing.B) { b.ReportAllocs() for i := 0; i < b.N; i++ { -- 2.52.0 From c3b462ec0a11c7a1630a63e6ff902246c92c0001 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:31:39 +0200 Subject: [PATCH 457/966] Answer architecture round 1: one inline load-and-bind, go generate for the index, sorted unloaded names, item 79 named in the decision, and the audit's other findings filed --- AGENTS.md | 2 +- README.md | 10 +++++---- compose.yaml | 4 ++++ doc.go | 3 ++- docs/decisions.md | 10 +++++---- fejkdata.go | 4 ++-- inline.go | 15 +++++++------ shipped.go | 34 +++++++++++++++++++---------- shipped_test.go | 8 +++++-- struct.go | 5 +---- todo.md | 54 ++++++++++++++++++++++++++++++++++++----------- 11 files changed, 102 insertions(+), 47 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index bc4a4a9..60731c5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -94,7 +94,7 @@ In [docs/decisions.md](docs/decisions.md): - Tables carrying token cells stay small - A path is walked once without drawing before it is walked for real - A country's postal codes and streets are siblings under its locality -- A locale's `address` reads its country's `geo` tree, so the shipped set loads whole +- A locale's `address` reads its country's `geo` tree, so a locale folder never loads alone - The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more - A locale's `address` restates its country record's format - A postort's kommun comes from its name, its tätort or its codes, never from distance diff --git a/README.md b/README.md index 79365b0..7992aa8 100644 --- a/README.md +++ b/README.md @@ -988,8 +988,9 @@ docker compose run --rm dev # interactive shell Commands that rewrite source keep your file ownership when run with `--user`: ```sh -docker compose run --rm --user "$(id -u):$(id -g)" fmt # gofmt -w . -docker compose run --rm --user "$(id -u):$(id -g)" tidy # go mod tidy +docker compose run --rm --user "$(id -u):$(id -g)" fmt # gofmt -w . +docker compose run --rm --user "$(id -u):$(id -g)" generate # go generate: shippedindex.go from data/ +docker compose run --rm --user "$(id -u):$(id -g)" tidy # go mod tidy ``` A `--user` command answering `permission denied` on `/cache` has met root-owned @@ -1019,8 +1020,9 @@ Goal 3 is scored by a panel of simulated readers, and [`AGENTS.md`](AGENTS.md) s what the score asks of a pull request. Goal 5 is checked against simulated template writers from the Audience. -A change to the shipped data re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) -and regenerates [`shippedindex.go`](shippedindex.go) in its own commit: +A change to the shipped data regenerates [`shippedindex.go`](shippedindex.go) with +`generate` above, and re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) +in its own commit: ```sh REPIN=1 docker compose run --rm --user "$(id -u):$(id -g)" test diff --git a/compose.yaml b/compose.yaml index 91aee47..e5682fc 100644 --- a/compose.yaml +++ b/compose.yaml @@ -23,6 +23,10 @@ services: <<: *go command: go test -run XXX -bench . -benchmem ./... + generate: + <<: *go + command: go generate ./... + vet: <<: *go command: go vet ./... diff --git a/doc.go b/doc.go index 1b608ed..0c9067e 100644 --- a/doc.go +++ b/doc.go @@ -43,5 +43,6 @@ package fejkdata // - whole — a read of a table with no selector and no descent, {/city}, landing on // the `table` itself, so it draws a row apart from every pin: `branches.wholePins`. // - fence — a load-time check, so rendering a compiled tree cannot fail: `New` -// runs them over the data set, `NewTemplate` and `FakeStruct` over what those +// runs them over the sources it loads, the first call reaching a shipped category +// over what that call loads, and `NewTemplate` and `FakeStruct` over what those // compile. diff --git a/docs/decisions.md b/docs/decisions.md index 07c17d6..56e5613 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -118,15 +118,17 @@ still lives in `data/` as JSON; `--data-path` layers over it. ## The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` -2026-10-03, approved by the maintainer as the restructure of `todo.md` item 76. Goal 12: a +2026-10-03, a restructure the maintainer approved. Goal 12: a `New` pays nothing for the categories a run never reaches, which matters once the shipped set grows to hundreds of MB. A category loads with every category it references and its whole table family, and is bound as a whole load binds, so it renders as it would from -`New`. `List` reads `shippedindex.go`, generated from `data/` with `REPIN=1`, and +`New`. `List` reads `shippedindex.go`, which `go generate` writes from `data/`, and `TestShippedIndexIsCurrent` fails while it is stale. CI loads the whole shipped set, and loads each shipped category alone into a fresh `New`, so none fails at first reach. Goal 6: beside a `--data-path` or a `WithDataFS`, every category loads in `New`, so each mistake in a source the user wrote is -`New`'s error. Valid while CI proves the shipped set whole and each category alone. +`New`'s error, though it parses the whole shipped set, against goal 12, until +`todo.md` item 79 loads only what the user's sources reach. Valid while CI proves the shipped set +whole and each category alone. ## A bare reference draws each time; a reference path is held @@ -650,7 +652,7 @@ its ZIPs would shape the two trees differently, so both draw inside the pinned l and an address agrees at that level. A street's own code is the exact pairing to add when a source carries it. -## A locale's `address` reads its country's `geo` tree, so the shipped set loads whole +## A locale's `address` reads its country's `geo` tree, so a locale folder never loads alone 2026-09-24, Lilleman auf Larv. diff --git a/fejkdata.go b/fejkdata.go index 8108780..d2e8887 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -144,8 +144,8 @@ func paths(n node) []string { out = append(out, join(name, p)) } } - for name, e := range n.unloaded { - for _, p := range e.paths { + for _, name := range sortedEntries(n.unloaded) { + for _, p := range n.unloaded[name].paths { out = append(out, join(name, p)) } } diff --git a/inline.go b/inline.go index f0adbd7..5e2a2a8 100644 --- a/inline.go +++ b/inline.go @@ -34,8 +34,7 @@ func (f *Generator) NewTemplate(input string) (*Template, error) { } f.mu.Lock() defer f.mu.Unlock() - loadShipped(&f.root, unloadedReads(&f.root, nil, inlineScope(n, "template"))) - if err := inlineBinding(n, "template", f.root.children).bind(); err != nil { + if err := bindInline(&f.root, n, "template", false); err != nil { return nil, fmt.Errorf("fejkdata: %w", err) } return &Template{g: f, n: n}, nil @@ -149,18 +148,22 @@ func inputValue(input string) (any, error) { return raw, nil } -func inlineBinding(n node, label string, root map[string]node) binding { +// bindInline loads the shipped categories n reads, then binds n against root; typedByGo is +// binding's. +func bindInline(root *folder, n node, label string, typedByGo bool) error { scope := inlineScope(n, label) + loadShipped(root, unloadedReads(root, nil, scope)) return binding{ - scope: scope, - link: func() error { return linkNodeRefs(scope, root) }, + scope: scope, + link: func() error { return linkNodeRefs(scope, root.children) }, + typedByGo: typedByGo, scopeFence: func() error { if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) } return nil }, - } + }.bind() } // linkNodeRefs binds the references in an inline node's templates against the diff --git a/shipped.go b/shipped.go index a7c7525..5e6336d 100644 --- a/shipped.go +++ b/shipped.go @@ -9,6 +9,8 @@ import ( "strings" ) +//go:generate env REPIN=1 go test -run ^TestShippedIndexIsCurrent$ . + //go:embed data var shippedFS embed.FS @@ -43,6 +45,15 @@ func unloadedTree() folder { return root } +func sortedEntries(m map[string]shippedEntry) []string { + names := make([]string, 0, len(m)) + for name := range m { + names = append(names, name) + } + sort.Strings(names) + return names +} + // unloadedCategory is a shipped category not loaded yet: the folder holding it, that // folder's path, and its name there. type unloadedCategory struct { @@ -70,7 +81,8 @@ func unloadedAt(root *folder, segs []string) (unloadedCategory, bool) { } // loadShippedAt loads the shipped category a caller's path names or descends into, unless -// it is loaded already. +// it is loaded already. An entry point walking root calls it first: a walk that loaded +// would load mid-render. func (f *Generator) loadShippedAt(segs []string) { if u, unloaded := unloadedAt(&f.root, segs); unloaded { loadShipped(&f.root, []unloadedCategory{u}) @@ -100,8 +112,8 @@ func unloadedReads(root *folder, dir []string, scope nodeScope) []unloadedCatego return out } -// loadShipped loads the shipped categories wanted, with every category they read and the -// whole table family of each, then binds what it loaded, as New binds a whole load. +// loadShipped loads the shipped categories wanted, with every category they read and every +// table linked to one, then binds what it loaded, as New binds a whole load. // TestEveryShippedCategoryLoadsAlone proves each closure, so a failure is the index's. func loadShipped(root *folder, wanted []unloadedCategory) { var sites []categorySite @@ -113,18 +125,18 @@ func loadShipped(root *folder, wanted []unloadedCategory) { } site, err := u.load() if err != nil { - panic(internalError("shipped %s: %v", u.name, err)) + panic(internalError("shipped %s: %v; after a change under data/, regenerate shippedindex.go", join(strings.Join(u.dir, "."), u.name), err)) } sites = append(sites, site) queue = append(queue, unloadedReads(root, u.dir, sitesScope([]categorySite{site}))...) - queue = append(queue, u.family(e)...) + queue = append(queue, u.linkedTables(e)...) } if len(sites) == 0 { return } sort.Slice(sites, func(i, j int) bool { return sites[i].path < sites[j].path }) if err := categoryBinding(sites, root.children).bind(); err != nil { - panic(internalError("binding shipped categories: %v", err)) + panic(internalError("binding shipped categories: %v; after a change under data/, regenerate shippedindex.go", err)) } } @@ -144,15 +156,15 @@ func (u unloadedCategory) load() (categorySite, error) { return categorySite{dir: u.dir, path: join(strings.Join(u.dir, "."), u.name), in: u.in, n: u.in.children[u.name]}, nil } -// family is the tables beside u that a table family links it to: its parent, and every -// table whose parent it is. -func (u unloadedCategory) family(e shippedEntry) []unloadedCategory { +// linkedTables is the tables beside u that link to it or that it links to: its parent, and +// every table whose parent it is. +func (u unloadedCategory) linkedTables(e shippedEntry) []unloadedCategory { var out []unloadedCategory if e.parent != "" { out = append(out, unloadedCategory{dir: u.dir, in: u.in, name: e.parent}) } - for name, sibling := range u.in.unloaded { - if sibling.parent == u.name { + for _, name := range sortedEntries(u.in.unloaded) { + if u.in.unloaded[name].parent == u.name { out = append(out, unloadedCategory{dir: u.dir, in: u.in, name: name}) } } diff --git a/shipped_test.go b/shipped_test.go index 84497ad..3c8819c 100644 --- a/shipped_test.go +++ b/shipped_test.go @@ -51,12 +51,16 @@ func TestShippedIndexIsCurrent(t *testing.T) { } return } - want, err := os.ReadFile(shippedIndexFile) + file, err := os.ReadFile(shippedIndexFile) + if err != nil { + t.Fatal(err) + } + want, err := format.Source(file) // docs/decisions.md#the-format-check-runs-on-the-latest-go-only if err != nil { t.Fatal(err) } if string(got) != string(want) { - t.Fatalf("%s is stale for data/; rewrite it with REPIN=1", shippedIndexFile) + t.Fatalf("%s is stale for data/; regenerate it with go generate", shippedIndexFile) } } diff --git a/struct.go b/struct.go index 0f74bf7..a27388c 100644 --- a/struct.go +++ b/struct.go @@ -258,10 +258,7 @@ func (s *structShape) compileRecord(root *folder, t reflect.Type, label string, if err != nil { return fmt.Errorf("%s: %w", label, err) } - loadShipped(root, unloadedReads(root, nil, inlineScope(n, label))) - b := inlineBinding(n, label, root.children) - b.typedByGo = true - if err := b.bind(); err != nil { + if err := bindInline(root, n, label, true); err != nil { return err } record, columns, err := recordOf(n) diff --git a/todo.md b/todo.md index fa72a2d..d261058 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 79` +`Next ID = 86` | Goal | W | |---|---| @@ -52,7 +52,6 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 76 | 0.1.0 | decision | **Generate the index of shipped categories at build time, and parse and link a shipped category only when a template or path first reaches it; a `--data-path` still loads whole.** | 4 | 8 | 10 | 10 | 2, 12 | 33.0 | | 77 | 0.1.0 | decision | **Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI.** | 5 | 8 | 10 | 10 | 2, 1.1 | 33.0 | | 78 | 0.1.0 | decision | **Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population.** | 6 | 7 | 10 | 10 | 2, 13.1 | 31.2 | | 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | @@ -68,7 +67,9 @@ | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | +| 79 | 0.1.0 | decision | **Beside a `--data-path`, load only the shipped categories the user's sources override or reach, as the shipped set alone does.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | +| 81 | 0.2.0 | question | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 4, 6 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | | 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | @@ -81,6 +82,7 @@ | 54 | 0.2.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | | 31 | 0.2.0 | | **Rename `misc.territory.country` to `sovereign`.** | 3 | 2 | 4 | 7 | 5.8 | 15.7 | | 20 | 0.2.0 | defect | **Weight `sv_SE.sex` and `en_US.sex` by bearers, as the README's Data section says they are.** | 2 | 2 | 4 | 6 | 4 | 15.7 | +| 80 | 0.2.0 | defect | **Walk a path through one per-step function that both `pathCheck.walk` and `probePath` call.** | 4 | 4 | 4 | 7 | 3.2 | 15.6 | | 50 | 0.2.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 6 | 15.5 | | 17 | 0.2.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 5 | 15.1 | | 48 | 0.2.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 5 | 15.1 | @@ -90,6 +92,8 @@ | 32 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 4.1 | 14.4 | | 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | | 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | +| 85 | 0.2.0 | question | **Reword goal 9 to promise one output per seed and version, as the decision "Seeded output is promised within one version" reads it.** | 1 | 1 | 2 | 5 | 9 | 8.8 | +| 84 | 0.2.0 | question | **Reword goal 8 so a table's rows in a TSV beside its JSON meet it, or end the decision "Rows live in a TSV, the shape in JSON".** | 1 | 1 | 1 | 6 | 8 | 8.6 | | 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | | 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | | 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | @@ -101,12 +105,12 @@ | 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 4 | 13.0 | | 35 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 8 | 13.0 | | 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 11 | 12.8 | -| 75 | 0.3.0 | | **Run every README `sh` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 5 | 4 | 7 | 11.2 | 11.8 | | 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 15 | 11.7 | | 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | | 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | | 30 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | +| 75 | 0.3.0 | | **Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 6 | 4 | 7 | 11.2 | 11.5 | | 33 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 5 | 10.8 | | 47 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 6 | 10.2 | | 72 | 0.3.0 | principle | **Fail the merge gate when test coverage falls below the last recorded figure.** | 2 | 3 | 2 | 5 | 3 | 10.2 | @@ -114,7 +118,9 @@ | 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 13.1 | 9.8 | | 37 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | | 43 | 0.3.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | +| 82 | 0.3.0 | principle | **Pass `newRand` its entropy source, so no test swaps the package's `randomBytes`.** | 1 | 1 | 1 | 5 | 3.2 | 9.8 | | 56 | 0.3.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | +| 83 | 0.3.0 | principle | **Read a name binding's head and tail from its binder's link, and delete `linkBindings`.** | 4 | 3 | 1 | 6 | 3.2 | 8.1 | | 63 | 0.4.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 4.1 | 19.1 | | 62 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 4 | 15.2 | | 59 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 11 | 13.8 | @@ -128,13 +134,9 @@ ## Details -### 76. Generate the index of shipped categories at build time, and parse and link a shipped category only when a template or path first reaches it; a `--data-path` still loads whole. - -A restructure the maintainer approved on 2026-10-03, so it merges below goal 3's floor and goes first; the round's gates are adjusted to it afterwards. `go generate` writes an index of every shipped category, path and table, embedded beside the data. `New` reads the index; a template, `Fake` path or record compiles the categories it reaches, transitively, and keeps them. CI loads every shipped category eagerly, so shipped data never fails for a user. Revise the decisions this touches, "The shipped data is embedded, not discovered" among them, and the README's Performance section. - ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -Needs item 76. The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where since 2026-10-03. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where since 2026-10-03. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. @@ -253,10 +255,18 @@ Needs item 4. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. +### 79. Beside a `--data-path`, load only the shipped categories the user's sources override or reach, as the shipped set alone does. + +The decision "The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, and the shipped ones whose reads reach those, so the index would carry each category's reads. + ### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. +### 81. Refuse at load every `calc` divisor not proven nonzero, in a string column too. + +The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row, against goals 4 and 6; `valueproof.go` already proves a typed column's divisor nonzero. Changing the decision is the maintainer's call. + ### 36. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. `New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. @@ -302,6 +312,10 @@ It holds a code, and `country` collides with the `misc.country` path it replaced Neither names a `weight`, so a bare `sex` draws evenly. +### 80. Walk a path through one per-step function that both `pathCheck.walk` and `probePath` call. + +The decision "A path draws through its compiled steps" keeps two loops over one rule for passing a choice, a table or a field, and they have drifted: `walk` refuses a `repeat` or `drawGroup` level through `enter`, `probePath` does not (item 48). Technical principle: DRY. + ### 50. Reword `prove`'s refusal of a typed column reading a row, `{/region}`. Say the format is the table's own (`region's format "{name}"`), name the read to write, `{/region.}` with its columns, and stop calling a one-column format, `{code}`, composed text. @@ -330,6 +344,14 @@ Require the path step to reach a sibling category. `readTableOptions` and `loadDir` return on the first in Go's map order. +### 85. Reword goal 9 to promise one output per seed and version, as the decision "Seeded output is promised within one version" reads it. + +Goal 9 says the same seed always gives the same output; the decision promises it within one version. + +### 84. Reword goal 8 so a table's rows in a TSV beside its JSON meet it, or end the decision "Rows live in a TSV, the shape in JSON". + +Goal 8 says data is JSON files, and the decision serves goal 12, which ranks below it. Proposed wording: "Data is JSON files, a table's rows a TSV beside them. Code is only for what JSON can't express." + ### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. Both render the same run. @@ -366,10 +388,6 @@ Needs item 4; recount the switches then. Today ten, from `render` to `columnItem Nothing checks it, and a new country breaks it silently. -### 75. Run every README `sh` example as a test, and compare each "Renders" line to a seeded render. - -`readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables) and compares no "Renders e.g." output, so goal 11.2 is not met. - ### 23. Give every other shipped category that could reach something real a path that never does. `phone` draws live PTS and NANP ranges — PTS's five fiction series are the Swedish inert set, sourced in [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. @@ -386,6 +404,10 @@ The inherited architect's 3am trace crossed four files for one draw. The CLI counts brackets of its own, so a change to the selector grammar desyncs the default `--table` silently. +### 75. Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render. + +`readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables), no `go` example (Library) and no `text` block of error text, and compares no "Renders e.g." output, so goal 11.2 is not met. + ### 47. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. @@ -406,10 +428,18 @@ Both name half of a `generatorState`, hiding its `{seq()}` counters, and `New`'s Its doc says render failure, and it wraps what `NewTemplate` and `NewRecordTemplate` refuse. +### 82. Pass `newRand` its entropy source, so no test swaps the package's `randomBytes`. + +`fejkdata_test.go` swaps the global `randomBytes` to fail seeding, so every `New` in the package reads the failing source while that test runs. Technical principle: functional core, imperative shell. + ### 56. Build on a manual run of `test.yml`. `workflow_dispatch` leaves `github.event.before` empty, so the diff compares `HEAD` with itself and skips `docker build`. +### 83. Read a name binding's head and tail from its binder's link, and delete `linkBindings`. + +`linkBindings` writes `nameBinding.head` and `tail` in `linkRefs`'s first pass and `linkName` reads them in its second, so swapping the passes hands `linkName` a nil head. Technical principle: one owner per value. + ### 63. Pair a street with its exact postnummer. Needs an application to Lantmäteriet; today a street goes to the nearest postal code centroid. It rewrites shipped rows, so it is a major once 1.0 is cut and a minor before that. -- 2.52.0 From 3bb8628e3930444fcf3c9616b83df0388a9bb521 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:38:26 +0200 Subject: [PATCH 458/966] Answer architecture round 2: one folder walk and one sorted-keys helper, REGENERATE for the index, and the held decisions naming the items that end them --- AGENTS.md | 2 +- README.md | 4 ++-- docs/decisions.md | 11 +++++++---- fejkdata.go | 2 +- graph.go | 2 +- node.go | 7 +------ reference.go | 42 ++++++++++++++++++++++++------------------ shipped.go | 38 ++++++++++++-------------------------- shipped_test.go | 4 ++-- 9 files changed, 51 insertions(+), 61 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 60731c5..ecae776 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # Rules -- Go runs only through `docker compose run --rm `; the merge gate is `docker build .` plus CI's changelog check. +- Go runs only through `docker compose run --rm `; the merge gate is `docker build .` plus CI's changelog check. - Tests first, in their own commit; the implementation follows in the next. A re-pin of seeded output or of `testdata/shipped_shape.txt` is its own commit. - A change under `data/` or to the shape pin adds its `CHANGELOG.md` entry under `Unreleased` in the same PR, and so does a change to a flag, an exit code, an exported name, a fence, a builtin or the lowest Go; what is major is the README's Versioning table. - One-line commit messages: no ticket prefix, no repo name, no authorship trailers. diff --git a/README.md b/README.md index 7992aa8..86e1fa7 100644 --- a/README.md +++ b/README.md @@ -816,8 +816,8 @@ The target is about a microsecond per value (goal 12). Each file is parsed, validated and weight-indexed once, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends through it. With the shipped set alone, a category loads on the first call reaching it, with -every category it reads, and `List` reads an index; beside a `--data-path`, every -category loads in `New`. Proving the draw fences adds a pass over the loaded tree per +every category it reads and every table linked to it, and `List` reads an index for what +has not loaded yet; beside a `--data-path`, every category loads in `New`. Proving the draw fences adds a pass over the loaded tree per fence, and walks what a render reads only where data holds a reference, so a set that holds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one diff --git a/docs/decisions.md b/docs/decisions.md index 56e5613..bf4669b 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -171,7 +171,7 @@ release instead. Before `v1.0.0` a minor carries what a major would. Any edit to a category shifts its stream and everything drawn after it, so a promise across versions would freeze every shipped list; a fixture is re-pinned on a bump, as -this repo's own are. +this repo's own are. Goal 9 reads wider than this until `todo.md` item 85 rewords it. ## An error is a contract by what it names, not its bytes @@ -259,7 +259,8 @@ assumes a 64-bit int; on a 32-bit target it could overflow and panic. load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands -instead and refuses a divisor it cannot keep from zero. +instead and refuses a divisor it cannot keep from zero. `Inf` in a string column works +against goals 4 and 6; `todo.md` item 81 asks the maintainer whether to refuse it. ## In data, a default written out and a constant spelled as a sample are load errors @@ -466,7 +467,8 @@ The benchmark suite (the README's Development) reports time for a human, not as objects would cost it a second; a TSV is one allocation whose cells are substrings, and the JSON says only how a row is composed. The TSV sits beside its category file, named by `rows`, so a data directory stays a directory of categories, and one nothing names is -a load error rather than a file silently ignored. +a load error rather than a file silently ignored. A TSV works against goal 8's "Data is +JSON files" until `todo.md` item 84 rewords the goal or ends this decision. ## A selector is bracketed, and a dot inside it is literal @@ -909,7 +911,8 @@ mode field, leaked the check's leaves and errors with the draw's pins, arm and m since Go tracks a struct's fields as one, and a repeat of a reference path rose from 66 to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node, so no benchmark gained an allocation. Valid while Go's escape analysis tracks -a struct's fields as one. +a struct's fields as one. The two loops have drifted, against DRY, until `todo.md` item 80 +walks both through one per-step function. ## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read diff --git a/fejkdata.go b/fejkdata.go index d2e8887..cfccc77 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -144,7 +144,7 @@ func paths(n node) []string { out = append(out, join(name, p)) } } - for _, name := range sortedEntries(n.unloaded) { + for _, name := range sortedNames(n.unloaded) { for _, p := range n.unloaded[name].paths { out = append(out, join(name, p)) } diff --git a/graph.go b/graph.go index 3d40630..a9cf92c 100644 --- a/graph.go +++ b/graph.go @@ -76,7 +76,7 @@ func namedNodes(m map[string]node) []namedNode { return out } -func sortedNames(m map[string]node) []string { +func sortedNames[V any](m map[string]V) []string { names := make([]string, 0, len(m)) for name := range m { names = append(names, name) diff --git a/node.go b/node.go index 822ab3a..01fbf58 100644 --- a/node.go +++ b/node.go @@ -406,12 +406,7 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) { // compileFields compiles every non-option key of a template object. func compileFields(m map[string]any, pos position) (map[string]node, error) { fields := make(map[string]node, len(m)) - keys := make([]string, 0, len(m)) - for k := range m { - keys = append(keys, k) - } - sort.Strings(keys) - for _, k := range keys { + for _, k := range sortedNames(m) { if isOption(k) { continue } diff --git a/reference.go b/reference.go index c6c3af9..fc717c6 100644 --- a/reference.go +++ b/reference.go @@ -214,26 +214,32 @@ func eachTemplate(sites []categorySite, fn func(s categorySite, path string, t * // resolveCategory walks a dotted path through the folders to the category it // names, returning that category's segments, the node, and the tail left to read into it. func resolveCategory(root map[string]node, segments []string) (categorySegs []string, target node, tail []string, err error) { - var n node = &folder{children: root} - i := 0 - for ; i < len(segments); i++ { - g, ok := n.(*folder) - if !ok { - break - } - if isSelector(segments[i]) { - return nil, nil, nil, fmt.Errorf("%s is a folder, not a table, so it has no row to select", strings.Join(segments[:i], ".")) - } - child, ok := g.children[segments[i]] - if !ok { - return nil, nil, nil, fmt.Errorf("no entry %q", segments[i]) - } - n = child - } - if _, ok := n.(*folder); ok { + g, i := folderAt(&folder{children: root}, segments) + switch { + case i == len(segments): return nil, nil, nil, fmt.Errorf("names a folder, not a value") + case isSelector(segments[i]): + return nil, nil, nil, fmt.Errorf("%s is a folder, not a table, so it has no row to select", strings.Join(segments[:i], ".")) } - return segments[:i], n, segments[i:], nil + n, ok := g.children[segments[i]] + if !ok { + return nil, nil, nil, fmt.Errorf("no entry %q", segments[i]) + } + return segments[:i+1], n, segments[i+1:], nil +} + +// folderAt is the deepest folder a path from root walks into, and the index of the first +// segment naming no folder in it, len(segs) where every segment does. +func folderAt(root *folder, segs []string) (*folder, int) { + g := root + for i, seg := range segs { + sub, isFolder := g.children[seg].(*folder) + if !isFolder { + return g, i + } + g = sub + } + return g, len(segs) } // refTokens returns the reference names a format reads, as tokens or as operands, or binds. diff --git a/shipped.go b/shipped.go index 5e6336d..0783f4c 100644 --- a/shipped.go +++ b/shipped.go @@ -9,7 +9,7 @@ import ( "strings" ) -//go:generate env REPIN=1 go test -run ^TestShippedIndexIsCurrent$ . +//go:generate env REGENERATE=1 go test -run ^TestShippedIndexIsCurrent$ . //go:embed data var shippedFS embed.FS @@ -17,7 +17,8 @@ var shippedFS embed.FS var shippedSource = dataSource{fsys: shippedFS, baseDir: "data"} // shippedEntry is a shipped category's entry in shippedindex.go: the table beside it that -// it links to, "" for none, and the paths List advertises below it. +// it links to, "" for none, and the paths List advertises below it. After changing it, empty +// shippedIndex's literal by hand so the package builds, then run generate. type shippedEntry struct { parent string paths []string @@ -45,15 +46,6 @@ func unloadedTree() folder { return root } -func sortedEntries(m map[string]shippedEntry) []string { - names := make([]string, 0, len(m)) - for name := range m { - names = append(names, name) - } - sort.Strings(names) - return names -} - // unloadedCategory is a shipped category not loaded yet: the folder holding it, that // folder's path, and its name there. type unloadedCategory struct { @@ -64,25 +56,19 @@ type unloadedCategory struct { // unloadedAt is the unloaded shipped category a path from root names, or descends into. func unloadedAt(root *folder, segs []string) (unloadedCategory, bool) { - g := root - for i, seg := range segs { - switch n := g.children[seg].(type) { - case *folder: - g = n - continue - case nil: - if _, unloaded := g.unloaded[seg]; unloaded { - return unloadedCategory{dir: segs[:i:i], in: g, name: seg}, true - } - } + g, i := folderAt(root, segs) + if i == len(segs) { return unloadedCategory{}, false } - return unloadedCategory{}, false + if _, unloaded := g.unloaded[segs[i]]; !unloaded { + return unloadedCategory{}, false + } + return unloadedCategory{dir: segs[:i:i], in: g, name: segs[i]}, true } // loadShippedAt loads the shipped category a caller's path names or descends into, unless -// it is loaded already. An entry point walking root calls it first: a walk that loaded -// would load mid-render. +// it is loaded already. An entry point calls it before walking root: a walk is a query and +// loads nothing. func (f *Generator) loadShippedAt(segs []string) { if u, unloaded := unloadedAt(&f.root, segs); unloaded { loadShipped(&f.root, []unloadedCategory{u}) @@ -163,7 +149,7 @@ func (u unloadedCategory) linkedTables(e shippedEntry) []unloadedCategory { if e.parent != "" { out = append(out, unloadedCategory{dir: u.dir, in: u.in, name: e.parent}) } - for _, name := range sortedEntries(u.in.unloaded) { + for _, name := range sortedNames(u.in.unloaded) { if u.in.unloaded[name].parent == u.name { out = append(out, unloadedCategory{dir: u.dir, in: u.in, name: name}) } diff --git a/shipped_test.go b/shipped_test.go index 3c8819c..61c9f45 100644 --- a/shipped_test.go +++ b/shipped_test.go @@ -39,13 +39,13 @@ func loadedCategories(f *Generator) []string { return out } -// REPIN=1 rewrites the index. +// REGENERATE=1, which go generate sets, rewrites the index. func TestShippedIndexIsCurrent(t *testing.T) { got, err := shippedIndexSource(newShippedWhole(t)) if err != nil { t.Fatal(err) } - if os.Getenv("REPIN") == "1" { + if os.Getenv("REGENERATE") == "1" { if err := os.WriteFile(shippedIndexFile, got, 0o644); err != nil { t.Fatal(err) } -- 2.52.0 From cc8797744bf3c913a3a15515f78d117840f6c3a9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:46:28 +0200 Subject: [PATCH 459/966] Answer architecture round 3: the generated header names go generate, one sorted-keys helper everywhere, one site constructor, and an inline folder reference refused before anything loads --- data.go | 6 +++++- fejkdata.go | 7 +------ inline.go | 23 ++++++++++++++++++----- namefence.go | 8 +------- naming_test.go | 7 +------ path.go | 7 +------ shape_test.go | 8 +------- shipped.go | 2 +- shipped_test.go | 4 ++-- shippedindex.go | 2 +- 10 files changed, 32 insertions(+), 42 deletions(-) diff --git a/data.go b/data.go index 9a88b9a..cb31d96 100644 --- a/data.go +++ b/data.go @@ -203,13 +203,17 @@ func categorySites(root *folder) []categorySite { walk(append(dir[:len(dir):len(dir)], name), sub) continue } - out = append(out, categorySite{dir: dir, path: join(strings.Join(dir, "."), name), in: g, n: g.children[name]}) + out = append(out, siteIn(dir, g, name)) } } walk(nil, root) return out } +func siteIn(dir []string, in *folder, name string) categorySite { + return categorySite{dir: dir, path: join(strings.Join(dir, "."), name), in: in, n: in.children[name]} +} + func sitesScope(sites []categorySite) nodeScope { return func(fn func(path string, n node) error) error { for _, s := range sites { diff --git a/fejkdata.go b/fejkdata.go index cfccc77..660cbb5 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -180,12 +180,7 @@ func paths(n node) []string { func tablePaths(t *table) []string { out := append([]string{""}, t.header...) sort.Strings(out[1:]) - children := make([]string, 0, len(t.children)) - for name := range t.children { - children = append(children, name) - } - sort.Strings(children) - for _, name := range children { + for _, name := range sortedNames(t.children) { for _, p := range paths(t.children[name]) { out = append(out, join(name, p)) } diff --git a/inline.go b/inline.go index 5e2a2a8..14d4e6d 100644 --- a/inline.go +++ b/inline.go @@ -152,6 +152,9 @@ func inputValue(input string) (any, error) { // binding's. func bindInline(root *folder, n node, label string, typedByGo bool) error { scope := inlineScope(n, label) + if err := refuseFolderRefs(scope); err != nil { + return err + } loadShipped(root, unloadedReads(root, nil, scope)) return binding{ scope: scope, @@ -166,10 +169,9 @@ func bindInline(root *folder, n node, label string, typedByGo bool) error { }.bind() } -// linkNodeRefs binds the references in an inline node's templates against the -// loaded tree. -func linkNodeRefs(scope nodeScope, root map[string]node) error { - if err := scope(func(path string, m node) error { +// refuseFolderRefs refuses a reference an inline node writes from a folder, which it has none of. +func refuseFolderRefs(scope nodeScope) error { + return scope(func(path string, m node) error { t, ok := m.(*template) if !ok { return nil @@ -183,7 +185,18 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest) } } - return linkTemplate(nil, path, "", t, root) + return nil + }) +} + +// linkNodeRefs binds the references in an inline node's templates against the +// loaded tree. +func linkNodeRefs(scope nodeScope, root map[string]node) error { + if err := scope(func(path string, m node) error { + if t, ok := m.(*template); ok { + return linkTemplate(nil, path, "", t, root) + } + return nil }); err != nil { return err } diff --git a/namefence.go b/namefence.go index 52dac90..f99765d 100644 --- a/namefence.go +++ b/namefence.go @@ -3,7 +3,6 @@ package fejkdata import ( "fmt" "slices" - "sort" ) // checkNameReads refuses each binding of t that checkUses refuses, and a read such as {n} whose @@ -117,12 +116,7 @@ func (b *nameBinding) checkTemplateOnce(read string, t *template, key string) er } } } - heads := make([]string, 0, len(fresh)) - for head := range fresh { - heads = append(heads, head) - } - sort.Strings(heads) - for _, head := range heads { + for _, head := range sortedNames(fresh) { if by, kept := b.addressed[join(key, head)]; kept && fresh[head] > 1 { return fmt.Errorf("{%s} renders field %q twice, so {%s} cannot say which draw it reads; drop {%s} or {%s}", read, head, by, read, by) } diff --git a/naming_test.go b/naming_test.go index b7c3e38..84c80ea 100644 --- a/naming_test.go +++ b/naming_test.go @@ -11,12 +11,7 @@ import ( // docs/decisions.md#one-name-one-meaning func TestNoFunctionSpellsAMethod(t *testing.T) { funcs, methods := declaredCalls(namespaceFiles(t)) - names := make([]string, 0, len(funcs)) - for name := range funcs { - names = append(names, name) - } - sort.Strings(names) - for _, name := range names { + for _, name := range sortedNames(funcs) { if recv := methods[name]; len(recv) > 0 { sort.Strings(recv) t.Errorf("func %s is spelled by %s too; name them apart", name, strings.Join(recv, " and ")) diff --git a/path.go b/path.go index eabadcb..bcb8747 100644 --- a/path.go +++ b/path.go @@ -2,7 +2,6 @@ package fejkdata import ( "fmt" - "sort" "strings" ) @@ -470,11 +469,7 @@ func unreachableInChoice(c *choice, want string) error { if len(c.shared) == 0 { return fmt.Errorf("no variant of this %d-way choice carries %q", len(c.items), want) } - offered := make([]string, 0, len(c.shared)) - for p := range c.shared { - offered = append(offered, p) - } - sort.Strings(offered) + offered := sortedNames(c.shared) return fmt.Errorf("not every variant of this %d-way choice carries %q; all carry %v", len(c.items), want, offered) } diff --git a/shape_test.go b/shape_test.go index a912b67..42e6299 100644 --- a/shape_test.go +++ b/shape_test.go @@ -2,7 +2,6 @@ package fejkdata import ( "os" - "sort" "strconv" "strings" "testing" @@ -109,12 +108,7 @@ func readsFact(n node) string { if len(set) == 0 { return "" } - keys := make([]string, 0, len(set)) - for k := range set { - keys = append(keys, k) - } - sort.Strings(keys) - return "\treads " + strings.Join(keys, " ") + return "\treads " + strings.Join(sortedNames(set), " ") } func TestShippedShapeNamesReads(t *testing.T) { diff --git a/shipped.go b/shipped.go index 0783f4c..ffba5b5 100644 --- a/shipped.go +++ b/shipped.go @@ -139,7 +139,7 @@ func (u unloadedCategory) load() (categorySite, error) { return categorySite{}, err } delete(u.in.unloaded, u.name) - return categorySite{dir: u.dir, path: join(strings.Join(u.dir, "."), u.name), in: u.in, n: u.in.children[u.name]}, nil + return siteIn(u.dir, u.in, u.name), nil } // linkedTables is the tables beside u that link to it or that it links to: its parent, and diff --git a/shipped_test.go b/shipped_test.go index 61c9f45..a40992f 100644 --- a/shipped_test.go +++ b/shipped_test.go @@ -66,7 +66,7 @@ func TestShippedIndexIsCurrent(t *testing.T) { func shippedIndexSource(whole *Generator) ([]byte, error) { var b bytes.Buffer - b.WriteString("// Code generated by TestShippedIndexIsCurrent with REPIN=1. DO NOT EDIT.\n\npackage fejkdata\n\nvar shippedIndex = map[string]shippedEntry{\n") + b.WriteString("// Code generated by go generate. DO NOT EDIT.\n\npackage fejkdata\n\nvar shippedIndex = map[string]shippedEntry{\n") for _, s := range categorySites(&whole.root) { fmt.Fprintf(&b, "%q: {", s.path) if t, isTable := s.n.(*table); isTable && t.parentIndex >= 0 { @@ -183,7 +183,7 @@ func TestOnDemandRendersAsTheWholeLoad(t *testing.T) { } func TestEveryShippedCategoryLoadsAlone(t *testing.T) { - for p := range shippedIndex { + for _, p := range sortedNames(shippedIndex) { f, err := New() if err != nil { t.Fatalf("New() = %v", err) diff --git a/shippedindex.go b/shippedindex.go index 6e73c43..32ef0e2 100644 --- a/shippedindex.go +++ b/shippedindex.go @@ -1,4 +1,4 @@ -// Code generated by TestShippedIndexIsCurrent with REPIN=1. DO NOT EDIT. +// Code generated by go generate. DO NOT EDIT. package fejkdata -- 2.52.0 From 590681ed24e602a62eaf611a37eac488f2ff2a2a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:52:29 +0200 Subject: [PATCH 460/966] Answer stability and product-owner: New's godoc says where the load lands, plainer Performance wording, regenerate before re-pinning, and no redundant sort --- README.md | 12 ++++++------ fejkdata.go | 3 ++- shipped.go | 2 -- shipped_test.go | 2 +- 4 files changed, 9 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 86e1fa7..8dfab77 100644 --- a/README.md +++ b/README.md @@ -815,9 +815,9 @@ The target is about a microsecond per value (goal 12). Each file is parsed, validated and weight-indexed once, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends through -it. With the shipped set alone, a category loads on the first call reaching it, with -every category it reads and every table linked to it, and `List` reads an index for what -has not loaded yet; beside a `--data-path`, every category loads in `New`. Proving the draw fences adds a pass over the loaded tree per +it. With the shipped set alone, `New` loads nothing: a category loads on the first call +reaching it, with every category it reads and every table linked to it. Beside +`--data-path`, `WithDataPath` or `WithDataFS`, every category loads in `New`. Proving the draw fences adds a pass over the loaded tree per fence, and walks what a render reads only where data holds a reference, so a set that holds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one @@ -1020,9 +1020,9 @@ Goal 3 is scored by a panel of simulated readers, and [`AGENTS.md`](AGENTS.md) s what the score asks of a pull request. Goal 5 is checked against simulated template writers from the Audience. -A change to the shipped data regenerates [`shippedindex.go`](shippedindex.go) with -`generate` above, and re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) -in its own commit: +A change to the shipped data runs `generate` above first and commits +[`shippedindex.go`](shippedindex.go) with the change, then re-pins +[`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) in its own commit: ```sh REPIN=1 docker compose run --rm --user "$(id -u):$(id -g)" test diff --git a/fejkdata.go b/fejkdata.go index 660cbb5..1dc09c5 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -86,7 +86,8 @@ func WithoutShippedData() Option { // in order with the last winning a name clash. Each JSON file becomes a category // named after the file (address.json -> "address") and each subdirectory a // namespace segment. It errors on a missing directory, invalid JSON, invalid data, -// or no data at all. +// or no data at all. The shipped set alone loads each category on the first call +// reaching it; beside WithDataPath or WithDataFS, every category loads here. func New(opts ...Option) (*Generator, error) { c := config{shipped: true} for _, opt := range opts { diff --git a/shipped.go b/shipped.go index ffba5b5..3754537 100644 --- a/shipped.go +++ b/shipped.go @@ -5,7 +5,6 @@ import ( "fmt" "io/fs" "path" - "sort" "strings" ) @@ -120,7 +119,6 @@ func loadShipped(root *folder, wanted []unloadedCategory) { if len(sites) == 0 { return } - sort.Slice(sites, func(i, j int) bool { return sites[i].path < sites[j].path }) if err := categoryBinding(sites, root.children).bind(); err != nil { panic(internalError("binding shipped categories: %v; after a change under data/, regenerate shippedindex.go", err)) } diff --git a/shipped_test.go b/shipped_test.go index a40992f..63cf222 100644 --- a/shipped_test.go +++ b/shipped_test.go @@ -60,7 +60,7 @@ func TestShippedIndexIsCurrent(t *testing.T) { t.Fatal(err) } if string(got) != string(want) { - t.Fatalf("%s is stale for data/; regenerate it with go generate", shippedIndexFile) + t.Fatalf("%s is stale for data/; regenerate it: docker compose run --rm --user \"$(id -u):$(id -g)\" generate", shippedIndexFile) } } -- 2.52.0 From a9eb337ec7df584c8d523249bd7460e603e482eb Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:52:37 +0200 Subject: [PATCH 461/966] Say in config.load only what New's godoc does not --- fejkdata.go | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fejkdata.go b/fejkdata.go index 1dc09c5..2632eb2 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -104,9 +104,8 @@ func New(opts ...Option) (*Generator, error) { return &Generator{rand: rng, root: root}, nil } -// load is the tree New starts from. The shipped data set alone loads each category on the -// first call reaching it; beside any other source, every category loads here, so each -// mistake in that source is New's error. +// load is the tree New starts from: beside a source of the caller's, every category loads +// here, so each mistake in that source is New's error. func (c config) load() (folder, error) { if c.shipped && len(c.sources) == 0 { return unloadedTree(), nil -- 2.52.0 From 53431ab908ac00c28a05da449e3e4d59f05b344d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:57:04 +0200 Subject: [PATCH 462/966] Answer prose round 1: plainer decision titles and body, comments that say what the code cannot, and claims New no longer makes --- AGENTS.md | 4 ++-- README.md | 13 +++++++------ doc.go | 6 ++---- docs/decisions.md | 39 ++++++++++++++++++++++----------------- fejkdata.go | 12 ++++++------ inline.go | 6 +++--- shipped.go | 18 +++++++++--------- shipped_test.go | 2 +- todo.md | 8 ++++---- 9 files changed, 56 insertions(+), 52 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ecae776..3381f37 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -43,7 +43,7 @@ In [docs/decisions.md](docs/decisions.md): - A padded JSON argument is rejected, not trimmed - `FakeTemplate` and `NewTemplate` both stay - The shipped data is embedded, not discovered -- The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` +- With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New` - A bare reference draws each time; a reference path is held - Reference sigils follow the filesystem - A change to what exists is a major; a minor only adds @@ -94,7 +94,7 @@ In [docs/decisions.md](docs/decisions.md): - Tables carrying token cells stay small - A path is walked once without drawing before it is walked for real - A country's postal codes and streets are siblings under its locality -- A locale's `address` reads its country's `geo` tree, so a locale folder never loads alone +- A locale's `address` reads its country's `geo` tree, so a locale folder is no data set on its own - The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more - A locale's `address` restates its country record's format - A postort's kommun comes from its name, its tätort or its codes, never from distance diff --git a/README.md b/README.md index 8dfab77..94ee6cb 100644 --- a/README.md +++ b/README.md @@ -499,7 +499,7 @@ table, `{/misc.currency[SEK].symbol}` on one row and `{/misc.currency[EUR].symbo on the next, since only one row renders. `New` proves the header, the options and every cell token, and refuses a TSV no category names, a key that is empty or repeats, a weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`, -`"` or `|`, which a selector cannot spell; `New` maps the keys, and a name or parent column +`"` or `|`, which a selector cannot spell; the table's load maps the keys, and a name or parent column is mapped on the first draw that selects by name or descends through the table. A `name` needs a `key`, since a name naming several rows is reported by their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]` is settled by the `sex` selected before it and a name repeating inside one parent row @@ -813,11 +813,12 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan The target is about a microsecond per value (goal 12). -Each file is parsed, validated and weight-indexed once, bar a table's name and parent -columns, which are mapped on the first `Fake` that selects by name or descends through -it. With the shipped set alone, `New` loads nothing: a category loads on the first call -reaching it, with every category it reads and every table linked to it. Beside -`--data-path`, `WithDataPath` or `WithDataFS`, every category loads in `New`. Proving the draw fences adds a pass over the loaded tree per +With only the shipped set, `New` loads nothing: a category loads on the first `Fake`, +`FakeRecord`, `FakeStruct` or template that reads it, with every category it reads and +its whole table family. Beside `--data-path`, `WithDataPath` or `WithDataFS`, every +category loads in `New`. Each file is parsed, validated and weight-indexed once, bar a +table's name and parent columns, which are mapped on the first `Fake` that selects by +name or descends through it. Proving the draw fences adds a pass over the loaded tree per fence, and walks what a render reads only where data holds a reference, so a set that holds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one diff --git a/doc.go b/doc.go index 0c9067e..12bc008 100644 --- a/doc.go +++ b/doc.go @@ -42,7 +42,5 @@ package fejkdata // `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // - whole — a read of a table with no selector and no descent, {/city}, landing on // the `table` itself, so it draws a row apart from every pin: `branches.wholePins`. -// - fence — a load-time check, so rendering a compiled tree cannot fail: `New` -// runs them over the sources it loads, the first call reaching a shipped category -// over what that call loads, and `NewTemplate` and `FakeStruct` over what those -// compile. +// - fence — a load-time check, run over each category, template or struct as it +// loads or compiles, so rendering a compiled tree cannot fail. diff --git a/docs/decisions.md b/docs/decisions.md index bf4669b..d18abba 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -80,7 +80,7 @@ tags and any other caller read one. 2026-09-03, Lilleman auf Larv. -`New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and +Each load proves what it loads acyclic, an inline node is a finite tree of its own, and nothing in the tree can reference it, so no render of it reaches itself. ## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing @@ -116,19 +116,23 @@ still lives in `data/` as JSON; `--data-path` layers over it. `todo.md` item 77 revises it: a library imports the shipped data as Go modules. -## The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` +## With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New` -2026-10-03, a restructure the maintainer approved. Goal 12: a -`New` pays nothing for the categories a run never reaches, which matters once the shipped -set grows to hundreds of MB. A category loads with every category it references and its -whole table family, and is bound as a whole load binds, so it renders as it would from -`New`. `List` reads `shippedindex.go`, which `go generate` writes from `data/`, and -`TestShippedIndexIsCurrent` fails while it is stale. CI loads the whole shipped set, and -loads each shipped category alone into a fresh `New`, so none fails at first reach. Goal 6: beside a `--data-path` or a -`WithDataFS`, every category loads in `New`, so each mistake in a source the user wrote is -`New`'s error, though it parses the whole shipped set, against goal 12, until -`todo.md` item 79 loads only what the user's sources reach. Valid while CI proves the shipped set -whole and each category alone. +2026-10-03, a restructure the maintainer approved. + +Goal 12: `New` pays nothing for a category a run never reaches, which matters once the +shipped set grows to hundreds of MB. A category loads with every category it references +and its whole table family, and binds as a whole load does, so it renders the same as +after a whole load. `List` reads `shippedindex.go`, which `go generate` writes from +`data/`; `TestShippedIndexIsCurrent` fails while the index is stale. CI loads the whole +shipped set, and each shipped category alone into a fresh `New`, so none fails at first +reach. + +Goal 6: beside a `--data-path`, `WithDataPath` or `WithDataFS`, every category loads in +`New`, so each mistake in the user's data is `New`'s error. That parses the whole shipped +set, against goal 12, until `todo.md` item 79 ends it. + +Valid while CI proves the shipped set whole and each category alone. ## A bare reference draws each time; a reference path is held @@ -171,7 +175,8 @@ release instead. Before `v1.0.0` a minor carries what a major would. Any edit to a category shifts its stream and everything drawn after it, so a promise across versions would freeze every shipped list; a fixture is re-pinned on a bump, as -this repo's own are. Goal 9 reads wider than this until `todo.md` item 85 rewords it. +this repo's own are. This works against goal 9's "always" until `todo.md` item 85 +rewords the goal. ## An error is a contract by what it names, not its bytes @@ -654,7 +659,7 @@ its ZIPs would shape the two trees differently, so both draw inside the pinned l and an address agrees at that level. A street's own code is the exact pairing to add when a source carries it. -## A locale's `address` reads its country's `geo` tree, so a locale folder never loads alone +## A locale's `address` reads its country's `geo` tree, so a locale folder is no data set on its own 2026-09-24, Lilleman auf Larv. @@ -911,8 +916,8 @@ mode field, leaked the check's leaves and errors with the draw's pins, arm and m since Go tracks a struct's fields as one, and a repeat of a reference path rose from 66 to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node, so no benchmark gained an allocation. Valid while Go's escape analysis tracks -a struct's fields as one. The two loops have drifted, against DRY, until `todo.md` item 80 -walks both through one per-step function. +a struct's fields as one. Two loops over one rule work against DRY, and they already +disagree on a repeat level; `todo.md` item 80 walks both through one per-step function. ## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read diff --git a/fejkdata.go b/fejkdata.go index 2632eb2..d3c8bdb 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -22,11 +22,11 @@ var _ [^uint(0)>>63 - 1]struct{} // docs/decisions.md#64-bit-targets-only // ErrNoData is returned by New when no source is loaded at all. var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS") -// Generator generates fake data from a loaded namespace tree. Create one with [New]. +// Generator generates fake data from a namespace tree. Create one with [New]. // It is safe for concurrent use; a seeded sequence is reproducible only when drawn // from one goroutine. type Generator struct { - // mu guards rand, records, structs and root, which loads a shipped category on the + // mu guards rand, records, structs and root, which gains a shipped category on the // first call reaching it. mu sync.Mutex rand *generatorState @@ -86,8 +86,8 @@ func WithoutShippedData() Option { // in order with the last winning a name clash. Each JSON file becomes a category // named after the file (address.json -> "address") and each subdirectory a // namespace segment. It errors on a missing directory, invalid JSON, invalid data, -// or no data at all. The shipped set alone loads each category on the first call -// reaching it; beside WithDataPath or WithDataFS, every category loads here. +// or no data at all. With only the shipped data, a category loads on the first call +// reaching it; with WithDataPath or WithDataFS, every category loads here. func New(opts ...Option) (*Generator, error) { c := config{shipped: true} for _, opt := range opts { @@ -104,8 +104,8 @@ func New(opts ...Option) (*Generator, error) { return &Generator{rand: rng, root: root}, nil } -// load is the tree New starts from: beside a source of the caller's, every category loads -// here, so each mistake in that source is New's error. +// load is the tree New starts from. +// docs/decisions.md#with-only-the-shipped-set-a-category-loads-on-the-first-call-reaching-it-beside-a---data-path-every-category-loads-in-new func (c config) load() (folder, error) { if c.shipped && len(c.sources) == 0 { return unloadedTree(), nil diff --git a/inline.go b/inline.go index 14d4e6d..df9b4d5 100644 --- a/inline.go +++ b/inline.go @@ -148,8 +148,7 @@ func inputValue(input string) (any, error) { return raw, nil } -// bindInline loads the shipped categories n reads, then binds n against root; typedByGo is -// binding's. +// bindInline loads the shipped categories n reads, then binds n against root. func bindInline(root *folder, n node, label string, typedByGo bool) error { scope := inlineScope(n, label) if err := refuseFolderRefs(scope); err != nil { @@ -169,7 +168,8 @@ func bindInline(root *folder, n node, label string, typedByGo bool) error { }.bind() } -// refuseFolderRefs refuses a reference an inline node writes from a folder, which it has none of. +// refuseFolderRefs refuses each reference in scope not written from the root, {/x}: an +// inline node sits in no folder. func refuseFolderRefs(scope nodeScope) error { return scope(func(path string, m node) error { t, ok := m.(*template) diff --git a/shipped.go b/shipped.go index 3754537..14810a8 100644 --- a/shipped.go +++ b/shipped.go @@ -15,8 +15,8 @@ var shippedFS embed.FS var shippedSource = dataSource{fsys: shippedFS, baseDir: "data"} -// shippedEntry is a shipped category's entry in shippedindex.go: the table beside it that -// it links to, "" for none, and the paths List advertises below it. After changing it, empty +// shippedEntry is a shipped category's entry in shippedindex.go: its parent table, "" for +// none, and the paths List advertises below it. After changing it, empty // shippedIndex's literal by hand so the package builds, then run generate. type shippedEntry struct { parent string @@ -97,9 +97,10 @@ func unloadedReads(root *folder, dir []string, scope nodeScope) []unloadedCatego return out } -// loadShipped loads the shipped categories wanted, with every category they read and every -// table linked to one, then binds what it loaded, as New binds a whole load. -// TestEveryShippedCategoryLoadsAlone proves each closure, so a failure is the index's. +// loadShipped loads the shipped categories wanted, with every category they read and the +// family of each, then binds what it loaded as a whole load binds. +// TestEveryShippedCategoryLoadsAlone reaches each category, so a failure here means a stale +// index. func loadShipped(root *folder, wanted []unloadedCategory) { var sites []categorySite for queue := wanted; len(queue) > 0; queue = queue[1:] { @@ -124,8 +125,8 @@ func loadShipped(root *folder, wanted []unloadedCategory) { } } -// load parses and compiles the category, and moves it from its folder's unloaded -// categories to its children. +// load parses and compiles the category, and moves it from its folder's unloaded map to +// the folder's children. func (u unloadedCategory) load() (categorySite, error) { dir := path.Join(append([]string{shippedSource.baseDir}, u.dir...)...) entries, err := fs.ReadDir(shippedSource.fsys, dir) @@ -140,8 +141,7 @@ func (u unloadedCategory) load() (categorySite, error) { return siteIn(u.dir, u.in, u.name), nil } -// linkedTables is the tables beside u that link to it or that it links to: its parent, and -// every table whose parent it is. +// linkedTables is u's parent beside it, and every table beside it whose parent u is. func (u unloadedCategory) linkedTables(e shippedEntry) []unloadedCategory { var out []unloadedCategory if e.parent != "" { diff --git a/shipped_test.go b/shipped_test.go index 63cf222..645496b 100644 --- a/shipped_test.go +++ b/shipped_test.go @@ -39,7 +39,7 @@ func loadedCategories(f *Generator) []string { return out } -// REGENERATE=1, which go generate sets, rewrites the index. +// REGENERATE=1 rewrites the index. func TestShippedIndexIsCurrent(t *testing.T) { got, err := shippedIndexSource(newShippedWhole(t)) if err != nil { diff --git a/todo.md b/todo.md index d261058..d782e8a 100644 --- a/todo.md +++ b/todo.md @@ -67,7 +67,7 @@ | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | -| 79 | 0.1.0 | decision | **Beside a `--data-path`, load only the shipped categories the user's sources override or reach, as the shipped set alone does.** | 6 | 6 | 7 | 8 | 12 | 13.7 | +| 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` only the user's categories, the shipped ones they override and the shipped ones they read; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 81 | 0.2.0 | question | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 4, 6 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | @@ -136,7 +136,7 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where since 2026-10-03. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. @@ -255,9 +255,9 @@ Needs item 4. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. -### 79. Beside a `--data-path`, load only the shipped categories the user's sources override or reach, as the shipped set alone does. +### 79. Beside a `--data-path`, load in `New` only the user's categories, the shipped ones they override and the shipped ones they read; load the rest on first reach. -The decision "The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, and the shipped ones whose reads reach those, so the index would carry each category's reads. +The decision "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, the shipped ones they read, and the shipped ones whose reads reach an overridden category, so the index would carry each category's reads. ### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. -- 2.52.0 From f60804098653f60c3ed96ecff81cf0b951046909 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:59:14 +0200 Subject: [PATCH 463/966] Answer prose round 2: item 79 names every set it loads, New maps a data author's keys, and fewer comments restating the code --- README.md | 2 +- docs/decisions.md | 4 ++-- shipped.go | 12 +++++------- todo.md | 4 ++-- 4 files changed, 10 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 94ee6cb..14317e9 100644 --- a/README.md +++ b/README.md @@ -499,7 +499,7 @@ table, `{/misc.currency[SEK].symbol}` on one row and `{/misc.currency[EUR].symbo on the next, since only one row renders. `New` proves the header, the options and every cell token, and refuses a TSV no category names, a key that is empty or repeats, a weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`, -`"` or `|`, which a selector cannot spell; the table's load maps the keys, and a name or parent column +`"` or `|`, which a selector cannot spell; `New` maps the keys, and a name or parent column is mapped on the first draw that selects by name or descends through the table. A `name` needs a `key`, since a name naming several rows is reported by their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]` is settled by the `sex` selected before it and a name repeating inside one parent row diff --git a/docs/decisions.md b/docs/decisions.md index d18abba..1e36208 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -673,8 +673,8 @@ when a source carries it. Sweden fits whole in 700 KB; every US place of 10,000 would pass a megabyte and fetch 1,200 counties of TIGER files, so the threshold sits where the two countries match in size, and `--min-population` and `--streets-per-locality` on the import scripts build a -fuller set. The two trees add about 20 ms to `New`, which loads the shipped set in about -45 ms. +fuller set. The two trees add about 20 ms to loading the shipped set whole, which took +about 45 ms. Withdrawn by the maintainer on 2026-10-03; `todo.md` item 78 replaces it. diff --git a/shipped.go b/shipped.go index 14810a8..27a9c1e 100644 --- a/shipped.go +++ b/shipped.go @@ -45,8 +45,7 @@ func unloadedTree() folder { return root } -// unloadedCategory is a shipped category not loaded yet: the folder holding it, that -// folder's path, and its name there. +// unloadedCategory is a shipped category not loaded yet. type unloadedCategory struct { dir []string in *folder @@ -65,9 +64,8 @@ func unloadedAt(root *folder, segs []string) (unloadedCategory, bool) { return unloadedCategory{dir: segs[:i:i], in: g, name: segs[i]}, true } -// loadShippedAt loads the shipped category a caller's path names or descends into, unless -// it is loaded already. An entry point calls it before walking root: a walk is a query and -// loads nothing. +// loadShippedAt loads the shipped category a caller's path names or descends into. An entry +// point calls it before walking root: a walk is a query and loads nothing. func (f *Generator) loadShippedAt(segs []string) { if u, unloaded := unloadedAt(&f.root, segs); unloaded { loadShipped(&f.root, []unloadedCategory{u}) @@ -97,8 +95,8 @@ func unloadedReads(root *folder, dir []string, scope nodeScope) []unloadedCatego return out } -// loadShipped loads the shipped categories wanted, with every category they read and the -// family of each, then binds what it loaded as a whole load binds. +// loadShipped loads the shipped categories wanted, every category they read, and the table +// family of each one it loads, then binds them as a whole load binds. // TestEveryShippedCategoryLoadsAlone reaches each category, so a failure here means a stale // index. func loadShipped(root *folder, wanted []unloadedCategory) { diff --git a/todo.md b/todo.md index d782e8a..a9b231b 100644 --- a/todo.md +++ b/todo.md @@ -67,7 +67,7 @@ | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | -| 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` only the user's categories, the shipped ones they override and the shipped ones they read; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | +| 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 81 | 0.2.0 | question | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 4, 6 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | @@ -255,7 +255,7 @@ Needs item 4. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. -### 79. Beside a `--data-path`, load in `New` only the user's categories, the shipped ones they override and the shipped ones they read; load the rest on first reach. +### 79. Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach. The decision "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, the shipped ones they read, and the shipped ones whose reads reach an overridden category, so the index would carry each category's reads. -- 2.52.0 From 1538d1ab2c0a7a03fd2640498503a01ca74ca538 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 18:00:05 +0200 Subject: [PATCH 464/966] Say that loadShipped follows the reads of everything it loads, and what the 45 ms holds --- docs/decisions.md | 4 ++-- shipped.go | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 1e36208..4401bbf 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -673,8 +673,8 @@ when a source carries it. Sweden fits whole in 700 KB; every US place of 10,000 would pass a megabyte and fetch 1,200 counties of TIGER files, so the threshold sits where the two countries match in size, and `--min-population` and `--streets-per-locality` on the import scripts build a -fuller set. The two trees add about 20 ms to loading the shipped set whole, which took -about 45 ms. +fuller set. Loading the whole shipped set took about 45 ms, about 20 ms of it the two +trees. Withdrawn by the maintainer on 2026-10-03; `todo.md` item 78 replaces it. diff --git a/shipped.go b/shipped.go index 27a9c1e..b8aafd7 100644 --- a/shipped.go +++ b/shipped.go @@ -95,8 +95,8 @@ func unloadedReads(root *folder, dir []string, scope nodeScope) []unloadedCatego return out } -// loadShipped loads the shipped categories wanted, every category they read, and the table -// family of each one it loads, then binds them as a whole load binds. +// loadShipped loads the shipped categories wanted and, for each category it loads, every +// category it reads and its table family, then binds all it loaded as a whole load binds. // TestEveryShippedCategoryLoadsAlone reaches each category, so a failure here means a stale // index. func loadShipped(root *folder, wanted []unloadedCategory) { -- 2.52.0 From fa49e8ad2bd44185ee852a39ea5f36520cf73f68 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 11:56:53 +0200 Subject: [PATCH 465/966] Refuse a template that could render an invalid value, count TSV tables as data, and promise one output per seed and version --- README.md | 5 +++-- docs/decisions.md | 8 +++----- todo.md | 17 ++++------------- 3 files changed, 10 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 14317e9..7226686 100644 --- a/README.md +++ b/README.md @@ -911,11 +911,12 @@ Where the project is heading; the sections before Audience document what ships t major version. 6. **Mistakes are caught when the data loads, and the error says plainly what is wrong.** + 1. A template that could render an invalid value fails to load instead. 7. **One command gets you a value.** 1. Flags work like in other command-line tools, and can go anywhere on the line. 2. Your first template of your own needs no escaping and no options. -8. **Data is JSON files. Code is only for what JSON can't express.** -9. **The same seed always gives the same output.** +8. **Data is JSON files and TSV tables. Code is only for what they can't express.** +9. **The same seed and version always give the same output.** 10. **It depends on nothing but Go's standard library.** 11. **The docs are a map of the template language.** 1. Every feature has its own heading. diff --git a/docs/decisions.md b/docs/decisions.md index 4401bbf..1fa4d36 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -175,8 +175,7 @@ release instead. Before `v1.0.0` a minor carries what a major would. Any edit to a category shifts its stream and everything drawn after it, so a promise across versions would freeze every shipped list; a fixture is re-pinned on a bump, as -this repo's own are. This works against goal 9's "always" until `todo.md` item 85 -rewords the goal. +this repo's own are. ## An error is a contract by what it names, not its bytes @@ -265,7 +264,7 @@ load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands instead and refuses a divisor it cannot keep from zero. `Inf` in a string column works -against goals 4 and 6; `todo.md` item 81 asks the maintainer whether to refuse it. +against goal 6.1 until `todo.md` item 81 refuses it there too. ## In data, a default written out and a constant spelled as a sample are load errors @@ -472,8 +471,7 @@ The benchmark suite (the README's Development) reports time for a human, not as objects would cost it a second; a TSV is one allocation whose cells are substrings, and the JSON says only how a row is composed. The TSV sits beside its category file, named by `rows`, so a data directory stays a directory of categories, and one nothing names is -a load error rather than a file silently ignored. A TSV works against goal 8's "Data is -JSON files" until `todo.md` item 84 rewords the goal or ends this decision. +a load error rather than a file silently ignored. ## A selector is bracketed, and a dot inside it is literal diff --git a/todo.md b/todo.md index a9b231b..2b4c972 100644 --- a/todo.md +++ b/todo.md @@ -32,6 +32,7 @@ | 5.7 | 0.80 | | 5.8 | 0.80 | | 6 | 0.75 | +| 6.1 | 0.75 | | 7 | 0.70 | | 7.1 | 0.70 | | 7.2 | 0.70 | @@ -69,7 +70,7 @@ | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | -| 81 | 0.2.0 | question | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 4, 6 | 20.6 | +| 81 | 0.2.0 | decision | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | | 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | @@ -92,8 +93,6 @@ | 32 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 4.1 | 14.4 | | 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | | 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | -| 85 | 0.2.0 | question | **Reword goal 9 to promise one output per seed and version, as the decision "Seeded output is promised within one version" reads it.** | 1 | 1 | 2 | 5 | 9 | 8.8 | -| 84 | 0.2.0 | question | **Reword goal 8 so a table's rows in a TSV beside its JSON meet it, or end the decision "Rows live in a TSV, the shape in JSON".** | 1 | 1 | 1 | 6 | 8 | 8.6 | | 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | | 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | | 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | @@ -136,7 +135,7 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree and one for `misc`; and a library passes a module to `New`, `New(WithData(svse.Data))`, with no registry a module's import fills, per the technical principles' dependency injection and no package globals. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. @@ -265,7 +264,7 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 81. Refuse at load every `calc` divisor not proven nonzero, in a string column too. -The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row, against goals 4 and 6; `valueproof.go` already proves a typed column's divisor nonzero. Changing the decision is the maintainer's call. +The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row, against goal 6.1; `valueproof.go` already proves a typed column's divisor nonzero. Revise the decision when this lands. ### 36. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. @@ -344,14 +343,6 @@ Require the path step to reach a sibling category. `readTableOptions` and `loadDir` return on the first in Go's map order. -### 85. Reword goal 9 to promise one output per seed and version, as the decision "Seeded output is promised within one version" reads it. - -Goal 9 says the same seed always gives the same output; the decision promises it within one version. - -### 84. Reword goal 8 so a table's rows in a TSV beside its JSON meet it, or end the decision "Rows live in a TSV, the shape in JSON". - -Goal 8 says data is JSON files, and the decision serves goal 12, which ranks below it. Proposed wording: "Data is JSON files, a table's rows a TSV beside them. Code is only for what JSON can't express." - ### 14. Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling. Both render the same run. -- 2.52.0 From 9cb0ed32d31713ccb04bd45d91c87881ab93cd1b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 11:58:50 +0200 Subject: [PATCH 466/966] Answer prose and product-owner round 1: item 81 covers NaN too, the seed decision folds into goal 9, and each decision names its goal --- AGENTS.md | 1 - README.md | 2 +- docs/decisions.md | 18 +++++------------- todo.md | 8 ++++---- 4 files changed, 10 insertions(+), 19 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3381f37..a1918b0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,7 +47,6 @@ In [docs/decisions.md](docs/decisions.md): - A bare reference draws each time; a reference path is held - Reference sigils follow the filesystem - A change to what exists is a major; a minor only adds -- Seeded output is promised within one version - An error is a contract by what it names, not its bytes - A load reports every mistake at once - Formats are checked after all data is loaded, so their errors name the category diff --git a/README.md b/README.md index 7226686..f2df02e 100644 --- a/README.md +++ b/README.md @@ -293,7 +293,7 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el | Option | | |--------|--| -| `WithSeed(n)` | same seed, same data: identical sequence | +| `WithSeed(n)` | same seed, version and data: identical sequence | | `WithDataPath(dir)` | layer a directory; repeat to layer several, the last wins a clash | | `WithDataFS(fsys)` | layer an `fs.FS`, such as your own `embed.FS` | | `WithoutShippedData()` | load only what you give | diff --git a/docs/decisions.md b/docs/decisions.md index 1fa4d36..2d8e701 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -112,7 +112,7 @@ and `regexp.Match`, not two spellings of one result. 2026-09-02, Lilleman auf Larv. A directory a machine happens to have would make `--seed 42` machine-dependent. Data -still lives in `data/` as JSON; `--data-path` layers over it. +still lives in `data/`; `--data-path` layers over it. `todo.md` item 77 revises it: a library imports the shipped data as Go modules. @@ -169,14 +169,6 @@ whole migration: a fence rejects one spelling with one replacement, so the fix i to each site. A fence that would need a non-local rewrite ships a converter with its release instead. Before `v1.0.0` a minor carries what a major would. -## Seeded output is promised within one version - -2026-09-16, Lilleman auf Larv. - -Any edit to a category shifts its stream and everything drawn after it, so a promise -across versions would freeze every shipped list; a fixture is re-pinned on a bump, as -this repo's own are. - ## An error is a contract by what it names, not its bytes 2026-09-16, Lilleman auf Larv. @@ -263,8 +255,8 @@ assumes a 64-bit int; on a 32-bit target it could overflow and panic. load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands -instead and refuses a divisor it cannot keep from zero. `Inf` in a string column works -against goal 6.1 until `todo.md` item 81 refuses it there too. +instead and refuses a divisor it cannot keep from zero. `Inf` and `NaN` in a string column +work against goals 6.1 and 4 until `todo.md` item 81 refuses them there too. ## In data, a default written out and a constant spelled as a sample are load errors @@ -465,13 +457,13 @@ The benchmark suite (the README's Development) reports time for a human, not as ## Rows live in a TSV, the shape in JSON -2026-09-17, Lilleman auf Larv. +2026-09-17, Lilleman auf Larv. Goals 8 and 12. `New` allocates once per node, so a register of thirty thousand rows written as JSON objects would cost it a second; a TSV is one allocation whose cells are substrings, and the JSON says only how a row is composed. The TSV sits beside its category file, named by `rows`, so a data directory stays a directory of categories, and one nothing names is -a load error rather than a file silently ignored. +a load error. ## A selector is bracketed, and a dot inside it is literal diff --git a/todo.md b/todo.md index 2b4c972..3a5d952 100644 --- a/todo.md +++ b/todo.md @@ -70,7 +70,7 @@ | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | -| 81 | 0.2.0 | decision | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | +| 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | | 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | @@ -135,7 +135,7 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree and one for `misc`; and a library passes a module to `New`, `New(WithData(svse.Data))`, with no registry a module's import fills, per the technical principles' dependency injection and no package globals. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree and one for `misc`; and a module registers nothing when imported: the library passes it to `New`, as in `New(WithData(svse.Data))`, per the technical principles' dependency injection and no package globals. The entry for `docs/decisions.md` is written when this ships. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. @@ -262,9 +262,9 @@ The decision "With only the shipped set, a category loads on the first call reac GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. -### 81. Refuse at load every `calc` divisor not proven nonzero, in a string column too. +### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. -The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row, against goal 6.1; `valueproof.go` already proves a typed column's divisor nonzero. Revise the decision when this lands. +The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row, and an operand that is sometimes not a number prints `NaN`, against goals 6.1 and 4; `valueproof.go` already proves a typed column's divisor nonzero. ### 36. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. -- 2.52.0 From fba410c83f7f84dd38dfeff6c58420183b6f1e3b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 12:16:19 +0200 Subject: [PATCH 467/966] Narrow goal 6.1 to values a template's own rules forbid, and name the data in goal 9 --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f2df02e..ddb48bf 100644 --- a/README.md +++ b/README.md @@ -911,12 +911,13 @@ Where the project is heading; the sections before Audience document what ships t major version. 6. **Mistakes are caught when the data loads, and the error says plainly what is wrong.** - 1. A template that could render an invalid value fails to load instead. + 1. A template that could render a value its own rules forbid, such as `NaN` or `Inf`, + fails to load instead. 7. **One command gets you a value.** 1. Flags work like in other command-line tools, and can go anywhere on the line. 2. Your first template of your own needs no escaping and no options. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** -9. **The same seed and version always give the same output.** +9. **The same seed, version and data always give the same output.** 10. **It depends on nothing but Go's standard library.** 11. **The docs are a map of the template language.** 1. Every feature has its own heading. -- 2.52.0 From 665c532f8cb489025e23dc6c2e2feffe36c28e98 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 12:17:56 +0200 Subject: [PATCH 468/966] Answer prose and product-owner round 2: one idea per sentence in items 77 and 81, item 77's open API points, and NaN explained where the decision names it --- README.md | 5 ++--- docs/decisions.md | 4 ++-- todo.md | 4 ++-- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index ddb48bf..e769614 100644 --- a/README.md +++ b/README.md @@ -843,9 +843,8 @@ a minor only adds, and a major is the only release that changes what exists. A patch changes no row of this table: performance, docs, or a fix inside a promised behaviour that changes no value, path, format or spelling. -Seeded output is a promise within one version: same seed, same version, same -data, same output. Any release may shift a stream, since a value added to a list -moves every draw after it, so pin fixtures per version. An error's wording may +Any release may shift a seeded stream, since a value added to a list moves every +draw after it, so pin fixtures per version. An error's wording may improve in a minor; the path, rejected spelling and replacement it names may not. Before `v1.0.0` a minor is the breaking unit: `0.(x+1).0` may carry a major's diff --git a/docs/decisions.md b/docs/decisions.md index 2d8e701..6a80527 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -255,8 +255,8 @@ assumes a 64-bit int; on a 32-bit target it could overflow and panic. load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands -instead and refuses a divisor it cannot keep from zero. `Inf` and `NaN` in a string column -work against goals 6.1 and 4 until `todo.md` item 81 refuses them there too. +instead and refuses a divisor it cannot keep from zero. An operand that is sometimes not a +number prints `NaN` the same way. `Inf` and `NaN` in a string column work against goals 6.1 and 4 until `todo.md` item 81 refuses them there too. ## In data, a default written out and a constant spelled as a sample are load errors diff --git a/todo.md b/todo.md index 3a5d952..e336bb2 100644 --- a/todo.md +++ b/todo.md @@ -135,7 +135,7 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree and one for `misc`; and a module registers nothing when imported: the library passes it to `New`, as in `New(WithData(svse.Data))`, per the technical principles' dependency injection and no package globals. The entry for `docs/decisions.md` is written when this ships. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported; the library passes it to `New`, as in `New(WithData(svse.Data))`, per the technical principles "Compose, do not entangle" and "Explicit over implicit". The entry for `docs/decisions.md` is written when this ships. Still to decide: what a bare `New()` loads, whether a module's data is an `fs.FS` passed to `WithDataFS` so no second option exists, and whether `WithoutShippedData` goes. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. @@ -264,7 +264,7 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. -The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row, and an operand that is sometimes not a number prints `NaN`, against goals 6.1 and 4; `valueproof.go` already proves a typed column's divisor nonzero. +The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row. An operand that is sometimes not a number prints `NaN`. Both work against goals 6.1 and 4. `valueproof.go` already proves a typed column's divisor nonzero. ### 36. Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states. -- 2.52.0 From fd313b76b6bfaf13acf0bb65a2f898f798a28059 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 12:18:01 +0200 Subject: [PATCH 469/966] Wrap the zero-divisor decision's last lines --- docs/decisions.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index 6a80527..f9ed312 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -256,7 +256,8 @@ load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands instead and refuses a divisor it cannot keep from zero. An operand that is sometimes not a -number prints `NaN` the same way. `Inf` and `NaN` in a string column work against goals 6.1 and 4 until `todo.md` item 81 refuses them there too. +number prints `NaN` the same way. `Inf` and `NaN` in a string column work against goals +6.1 and 4 until `todo.md` item 81 refuses them there too. ## In data, a default written out and a constant spelled as a sample are load errors -- 2.52.0 From b2406ef4bcd0dfa0c5e82d5910bde2e58fcb31e4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 12:19:06 +0200 Subject: [PATCH 470/966] Answer prose round 3: say where NaN prints, and leave item 77's option name open --- docs/decisions.md | 7 ++++--- todo.md | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index f9ed312..d2b3a28 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -255,9 +255,10 @@ assumes a 64-bit int; on a 32-bit target it could overflow and panic. load error; the fold stops where an operand varies, so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every draw — catching it needs zero-absorbing algebra for a shape nobody writes. A [typed column](../README.md#datatype) bounds its operands -instead and refuses a divisor it cannot keep from zero. An operand that is sometimes not a -number prints `NaN` the same way. `Inf` and `NaN` in a string column work against goals -6.1 and 4 until `todo.md` item 81 refuses them there too. +instead and refuses a divisor it cannot keep from zero. In a string column, an operand that +is sometimes not a number loads, and prints `NaN` on each draw where it is not one. `Inf` +and `NaN` in a string column work against goals 6.1 and 4 until `todo.md` item 81 +refuses them there too. ## In data, a default written out and a constant spelled as a sample are load errors diff --git a/todo.md b/todo.md index e336bb2..acaadf2 100644 --- a/todo.md +++ b/todo.md @@ -135,7 +135,7 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported; the library passes it to `New`, as in `New(WithData(svse.Data))`, per the technical principles "Compose, do not entangle" and "Explicit over implicit". The entry for `docs/decisions.md` is written when this ships. Still to decide: what a bare `New()` loads, whether a module's data is an `fs.FS` passed to `WithDataFS` so no second option exists, and whether `WithoutShippedData` goes. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported; the library passes it to `New` as an option, per the technical principles "Compose, do not entangle" and "Explicit over implicit". Still to decide: what a bare `New()` loads; whether a module's data is an `fs.FS` passed to the existing `WithDataFS` or a new option such as `WithData(svse.Data)`; and whether `WithoutShippedData` goes. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. -- 2.52.0 From 48894a55e84d8d20f6128a2af4b4782a7e01b224 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 12:19:14 +0200 Subject: [PATCH 471/966] Give the CLI's --seed row the conditions the library's WithSeed row states --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e769614..55495c2 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ rejects, and what each names instead, is under |------|--| | `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash | | `--no-shipped-data` | load only the `--data-path` directories | -| `-s`, `--seed N` | reproducible output | +| `-s`, `--seed N` | same seed, version and data: identical output | | `-n`, `--repeat N` | render the value N times (up to 1048576), each an independent draw, streamed | | `--separator S` | between repeated values (default a newline) | | `--format F` | `text` (default), `json`, `ndjson`, `csv` or `sql` — a record's columns, one record per row (json frames them as an array) | -- 2.52.0 From 0bedcd47e2769afc5603f71c3840ce76f7784d16 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 16:55:57 +0200 Subject: [PATCH 472/966] Say a patch may change what a seed draws, so seeded fixtures pin to one version --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 55495c2..9179c19 100644 --- a/README.md +++ b/README.md @@ -841,10 +841,10 @@ a minor only adds, and a major is the only release that changes what exists. | Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option | A patch changes no row of this table: performance, docs, or a fix inside a promised -behaviour that changes no value, path, format or spelling. +behaviour that adds or removes no value and changes no path, format or spelling. -Any release may shift a seeded stream, since a value added to a list moves every -draw after it, so pin fixtures per version. An error's wording may +Any release, a patch too, may change what a seed draws, so pin seeded fixtures to one +version. An error's wording may improve in a minor; the path, rejected spelling and replacement it names may not. Before `v1.0.0` a minor is the breaking unit: `0.(x+1).0` may carry a major's -- 2.52.0 From 6eb9880a3fbccbcfd9fd3d1f58a5566d540a4b1e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:11:27 +0200 Subject: [PATCH 473/966] Name the version in every seed promise, and say to regenerate seeded fixtures when it changes --- README.md | 13 ++++++++----- cmd/fejkdata/main.go | 2 +- fejkdata.go | 4 ++-- todo.md | 2 +- 4 files changed, 12 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 9179c19..d77e181 100644 --- a/README.md +++ b/README.md @@ -829,8 +829,9 @@ the output. Semver tags on `main`, `v0.1.0` first; one version covers the shipped data, the library and the CLI, and [`CHANGELOG.md`](CHANGELOG.md) names what each release -changed. A consumer's data, code and scripts keep working across a minor or a patch: -a minor only adds, and a major is the only release that changes what exists. +changed. A consumer's data, code and scripts keep working across a minor or a patch, +except output pinned to a seed. A minor only adds, and a major is the only release that +changes what exists. | Surface | Major | Minor | |---------|-------|-------| @@ -843,9 +844,11 @@ a minor only adds, and a major is the only release that changes what exists. A patch changes no row of this table: performance, docs, or a fix inside a promised behaviour that adds or removes no value and changes no path, format or spelling. -Any release, a patch too, may change what a seed draws, so pin seeded fixtures to one -version. An error's wording may -improve in a minor; the path, rejected spelling and replacement it names may not. +Any release, a patch too, may change what a seed draws, so regenerate seeded fixtures +whenever the fejkdata version changes. + +An error's wording may improve in a minor; the path, rejected spelling and replacement +it names may not. Before `v1.0.0` a minor is the breaking unit: `0.(x+1).0` may carry a major's changes, each named in the changelog, and a `0.x.y` patch may not. `v1.0.0` is diff --git a/cmd/fejkdata/main.go b/cmd/fejkdata/main.go index 36e116f..49b621c 100644 --- a/cmd/fejkdata/main.go +++ b/cmd/fejkdata/main.go @@ -48,7 +48,7 @@ array, one JSON object per line, one CSV row (after a header), or one INSERT. --list list the paths the data offers, then exit --no-shipped-data load only the --data-path directories -n, --repeat N render the value N times, 1..1048576 (default 1) - -s, --seed N seed for reproducible output + -s, --seed N seed: the same seed, version and data give identical output --separator S string between repeated values (default newline) --table T the INSERT target for --format sql (default: the path's last segment, or records for an inline template) --version print the version, then exit diff --git a/fejkdata.go b/fejkdata.go index d3c8bdb..7ca6300 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -57,8 +57,8 @@ type config struct { // Option configures a [Generator]. type Option func(*config) -// WithSeed makes output reproducible: two generators with the same seed and data -// emit identical sequences. +// WithSeed makes output reproducible: two generators with the same seed, version and +// data emit identical sequences. func WithSeed(seed uint64) Option { return func(c *config) { c.seed, c.seeded = seed, true } } diff --git a/todo.md b/todo.md index acaadf2..7a3c2b8 100644 --- a/todo.md +++ b/todo.md @@ -135,7 +135,7 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported; the library passes it to `New` as an option, per the technical principles "Compose, do not entangle" and "Explicit over implicit". Still to decide: what a bare `New()` loads; whether a module's data is an `fs.FS` passed to the existing `WithDataFS` or a new option such as `WithData(svse.Data)`; and whether `WithoutShippedData` goes. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported; the code importing it passes it to `New` as an option, per the technical principles "Compose, do not entangle" and "Explicit over implicit". Still to decide: what a bare `New()` loads; whether a module's data is an `fs.FS` passed to the existing `WithDataFS` or a new option such as `WithData(svse.Data)`; and whether `WithoutShippedData` goes. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. -- 2.52.0 From bded47846ded1db1c0cb0fcc575dffd4d6f68a7b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:13:33 +0200 Subject: [PATCH 474/966] State the seed exception once, with pin-then-regenerate advice, and word --seed's help as the README's row --- README.md | 11 +++++------ cmd/fejkdata/main.go | 2 +- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index d77e181..75625d0 100644 --- a/README.md +++ b/README.md @@ -829,9 +829,11 @@ the output. Semver tags on `main`, `v0.1.0` first; one version covers the shipped data, the library and the CLI, and [`CHANGELOG.md`](CHANGELOG.md) names what each release -changed. A consumer's data, code and scripts keep working across a minor or a patch, -except output pinned to a seed. A minor only adds, and a major is the only release that -changes what exists. +changed. A consumer's data, code and scripts keep working across a minor or a patch. A +minor only adds, and a major is the only release that changes what exists. Seeded output +is the exception: any release, a patch too, may change what a seed draws. Where you +compare seeded output, install one exact version (`go install …@vX.Y.Z`; `go.mod` already +pins one), and regenerate seeded fixtures and expected values when you raise it. | Surface | Major | Minor | |---------|-------|-------| @@ -844,9 +846,6 @@ changes what exists. A patch changes no row of this table: performance, docs, or a fix inside a promised behaviour that adds or removes no value and changes no path, format or spelling. -Any release, a patch too, may change what a seed draws, so regenerate seeded fixtures -whenever the fejkdata version changes. - An error's wording may improve in a minor; the path, rejected spelling and replacement it names may not. diff --git a/cmd/fejkdata/main.go b/cmd/fejkdata/main.go index 49b621c..2299410 100644 --- a/cmd/fejkdata/main.go +++ b/cmd/fejkdata/main.go @@ -48,7 +48,7 @@ array, one JSON object per line, one CSV row (after a header), or one INSERT. --list list the paths the data offers, then exit --no-shipped-data load only the --data-path directories -n, --repeat N render the value N times, 1..1048576 (default 1) - -s, --seed N seed: the same seed, version and data give identical output + -s, --seed N same seed, version and data: identical output --separator S string between repeated values (default newline) --table T the INSERT target for --format sql (default: the path's last segment, or records for an inline template) --version print the version, then exit -- 2.52.0 From 5bd022fe89cc70d013d8973bae2816f02964f0a4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:14:55 +0200 Subject: [PATCH 475/966] Spell the exact-version install in full, and say which version to raise --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 75625d0..f57ae5a 100644 --- a/README.md +++ b/README.md @@ -832,8 +832,9 @@ library and the CLI, and [`CHANGELOG.md`](CHANGELOG.md) names what each release changed. A consumer's data, code and scripts keep working across a minor or a patch. A minor only adds, and a major is the only release that changes what exists. Seeded output is the exception: any release, a patch too, may change what a seed draws. Where you -compare seeded output, install one exact version (`go install …@vX.Y.Z`; `go.mod` already -pins one), and regenerate seeded fixtures and expected values when you raise it. +compare seeded output, install one exact version +(`go install github.com/larvit/fejkdata/cmd/fejkdata@vX.Y.Z`; `go.mod` already pins one), +and regenerate seeded fixtures and expected values when you raise that version. | Surface | Major | Minor | |---------|-------|-------| -- 2.52.0 From 7ac8442bcf5a3d47b4562e3e7a08b640ad388324 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:31:40 +0200 Subject: [PATCH 476/966] Test that a US exchange is an assignable NANP code and that no shipped cell is padded --- data_test.go | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/data_test.go b/data_test.go index 7dbe162..a92c7b8 100644 --- a/data_test.go +++ b/data_test.go @@ -1,6 +1,8 @@ package fejkdata import ( + "io/fs" + "os" "regexp" "strconv" "strings" @@ -328,13 +330,43 @@ func TestShippedPersonHasParts(t *testing.T) { } } +// NANPA's central office code records mark N11, 555, 950, 958, 959 and 976 unassignable. func TestShippedUSPhone(t *testing.T) { f := newGenerator(t, "data", WithSeed(11)) - re := regexp.MustCompile(`^(\(\d{3}\) \d{3}-\d{4}|\d{3}-\d{3}-\d{4})$`) - for i := 0; i < 50; i++ { - if n := fake(t, f, "en_US.phone"); !re.MatchString(n) { + re := regexp.MustCompile(`^(?:\(\d{3}\) |\d{3}-)([2-9]\d{2})-\d{4}$`) + unassignable := map[string]bool{"555": true, "950": true, "958": true, "959": true, "976": true} + for i := 0; i < 2000; i++ { + n := fake(t, f, "en_US.phone") + m := re.FindStringSubmatch(n) + if m == nil { t.Fatalf("phone %q does not match %s", n, re) } + if exch := m[1]; exch[1:] == "11" || unassignable[exch] { + t.Fatalf("phone %q has the unassignable exchange %s", n, exch) + } + } +} + +func TestShippedTableCellsAreTrimmed(t *testing.T) { + err := fs.WalkDir(os.DirFS("data"), ".", func(path string, d fs.DirEntry, err error) error { + if err != nil || !strings.HasSuffix(path, ".tsv") { + return err + } + b, err := fs.ReadFile(os.DirFS("data"), path) + if err != nil { + return err + } + for i, line := range strings.Split(string(b), "\n") { + for _, cell := range strings.Split(line, "\t") { + if cell != strings.TrimSpace(cell) { + t.Errorf("data/%s:%d: cell %q is padded", path, i+1, cell) + } + } + } + return nil + }) + if err != nil { + t.Fatal(err) } } -- 2.52.0 From 563237a45a40dcbbaa2b1cba4c6cb3cad2ef24f3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:33:16 +0200 Subject: [PATCH 477/966] Draw a US exchange from NANPA's assignable codes, and strip every imported cell --- CHANGELOG.md | 2 ++ data-import/tsv.py | 4 ++-- data/en_US/phone.json | 36 ++++++++++++++++++++++++++++++++++-- data/misc/currency.tsv | 2 +- data/misc/territory.tsv | 2 +- 5 files changed, 40 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 420e1a9..09777f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,6 +90,8 @@ replacement, and each removed path, column or flag. and carries the same three. `person.prefix` is null where a person has no title, where it used to be an empty string, so `--format sql` writes `NULL` and a `string` struct field reading it becomes `*string`. +- `en_US.phone` draws its exchange, `exch`, from the central office codes NANPA can + assign: `200` to `999`, never `N11`, `555`, `950`, `958`, `959` or `976`. - `ErrNoColumns` is exported, so a caller can tell the one record fence a path can answer from the rest. - An error names a spelling that runs: a layout is named single-quoted and free of diff --git a/data-import/tsv.py b/data-import/tsv.py index 51ea6ea..333e9ab 100644 --- a/data-import/tsv.py +++ b/data-import/tsv.py @@ -5,10 +5,10 @@ from pathlib import Path def write(path, columns, rows): - """Write the rows as a TSV; the table needs two rows, and every cell must be non-empty and free of tabs, newlines and braces.""" + """Write the rows as a TSV; the table needs two rows, and every cell is stripped, and must be non-empty and free of tabs, newlines and braces.""" lines = ["\t".join(columns)] for row in rows: - cells = [str(row[c]) for c in columns] + cells = [str(row[c]).strip() for c in columns] if not all(cells) or any(re.search(r"[\t\n{}]", c) for c in cells): raise ValueError(f"{path}: a cell is empty or holds a tab, newline or brace: {row}") lines.append("\t".join(cells)) diff --git a/data/en_US/phone.json b/data/en_US/phone.json index b47298b..a5e9f9e 100644 --- a/data/en_US/phone.json +++ b/data/en_US/phone.json @@ -3,13 +3,45 @@ "format": "({area}) {exch}-{line}", "weight": 2, "area": ["202", "212", "213", "305", "312", "404", "415", "469", "503", "512", "602", "617", "646", "702", "718", "720", "737", "773", "786", "808", "917"], - "exch": "{int(100,999)}", + "exch": [ + { "format": "{int(2,9)}0{digits(1)}", "weight": 80 }, + { "format": "{int(2,9)}1{int(2,9)}", "weight": 64 }, + { "format": "{int(2,9)}10", "weight": 8 }, + { "format": "{int(2,4)}{int(2,9)}{digits(1)}", "weight": 240 }, + { "format": "{int(6,8)}{int(2,9)}{digits(1)}", "weight": 240 }, + { "format": "5{int(2,4)}{digits(1)}", "weight": 30 }, + { "format": "5{int(6,9)}{digits(1)}", "weight": 40 }, + { "format": "55{int(0,4)}", "weight": 5 }, + { "format": "55{int(6,9)}", "weight": 4 }, + { "format": "9{int(2,4)}{digits(1)}", "weight": 30 }, + { "format": "9{int(8,9)}{digits(1)}", "weight": 20 }, + { "format": "95{int(1,7)}", "weight": 7 }, + { "format": "96{digits(1)}", "weight": 10 }, + { "format": "97{int(0,5)}", "weight": 6 }, + { "format": "97{int(7,9)}", "weight": 3 } + ], "line": "{digits(4)}" }, { "format": "{area}-{exch}-{line}", "area": ["202", "212", "213", "305", "312", "404", "415", "469", "503", "512", "602", "617", "646", "702", "718", "720", "737", "773", "786", "808", "917"], - "exch": "{int(100,999)}", + "exch": [ + { "format": "{int(2,9)}0{digits(1)}", "weight": 80 }, + { "format": "{int(2,9)}1{int(2,9)}", "weight": 64 }, + { "format": "{int(2,9)}10", "weight": 8 }, + { "format": "{int(2,4)}{int(2,9)}{digits(1)}", "weight": 240 }, + { "format": "{int(6,8)}{int(2,9)}{digits(1)}", "weight": 240 }, + { "format": "5{int(2,4)}{digits(1)}", "weight": 30 }, + { "format": "5{int(6,9)}{digits(1)}", "weight": 40 }, + { "format": "55{int(0,4)}", "weight": 5 }, + { "format": "55{int(6,9)}", "weight": 4 }, + { "format": "9{int(2,4)}{digits(1)}", "weight": 30 }, + { "format": "9{int(8,9)}{digits(1)}", "weight": 20 }, + { "format": "95{int(1,7)}", "weight": 7 }, + { "format": "96{digits(1)}", "weight": 10 }, + { "format": "97{int(0,5)}", "weight": 6 }, + { "format": "97{int(7,9)}", "weight": 3 } + ], "line": "{digits(4)}" } ] diff --git a/data/misc/currency.tsv b/data/misc/currency.tsv index 0e9f584..dacb23d 100644 --- a/data/misc/currency.tsv +++ b/data/misc/currency.tsv @@ -71,7 +71,7 @@ JPY 0 Yen 392 ¥ KES 2 Kenyan Shilling 404 KES KGS 2 Som 417 ⃀ KHR 2 Riel 116 ៛ -KMF 0 Comorian Franc 174 CF +KMF 0 Comorian Franc 174 CF KPW 2 North Korean Won 408 ₩ KRW 0 Won 410 ₩ KWD 3 Kuwaiti Dinar 414 KWD diff --git a/data/misc/territory.tsv b/data/misc/territory.tsv index 787996d..dc760cd 100644 --- a/data/misc/territory.tsv +++ b/data/misc/territory.tsv @@ -48,7 +48,7 @@ CO COL 57 Bogota CO COP 🇨🇴 es-CO Colombia 170 .co CR CRI 506 San Jose CR CRC 🇨🇷 es-CR,en Costa Rica 188 .cr CU CUB 53 Havana CU CUP 🇨🇺 es-CU,pap Cuba 192 .cu CV CPV 238 Praia CV CVE 🇨🇻 pt-CV Cape Verde 132 .cv -CW CUW 599 Willemstad NL XCG 🇨🇼 nl,pap Curaçao 531 .cw +CW CUW 599 Willemstad NL XCG 🇨🇼 nl,pap Curaçao 531 .cw CX CXR 61 Flying Fish Cove AU AUD 🇨🇽 en,zh,ms-CX Christmas Island 162 .cx CY CYP 357 Nicosia CY EUR 🇨🇾 el-CY,tr-CY,en Cyprus 196 .cy CZ CZE 420 Prague CZ CZK 🇨🇿 cs,sk Czechia 203 .cz -- 2.52.0 From 7ac63caeb86879df8cb20fc6df72e3dddeb2a4ac Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:33:30 +0200 Subject: [PATCH 478/966] Close items 25 and 12 --- todo.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/todo.md b/todo.md index 7a3c2b8..23dd2d0 100644 --- a/todo.md +++ b/todo.md @@ -60,12 +60,10 @@ | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | | 3 | 0.1.0 | decision | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | -| 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 4 | 23.1 | | 57 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | | 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 5.4 | 21.6 | | 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 4 | 21.1 | | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | -| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | @@ -218,10 +216,6 @@ Shape: T = table, t = template, c = choice. - Widens the refusal of a name read once to every binding, a path into it or a reference path bound included, since `{n.x}` read once is then `{/ref.x}`, and deletes the names fence over draw groups and nested repeats (`checkUses`, `readsHeld`). - Re-pin seeded output and `testdata/shipped_shape.txt` in their own commits; the CHANGELOG names the grammar change. -### 25. Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`. - -`{int(100,999)}` renders a leading 1 in about an eighth of draws, which libphonenumber rejects, and the test proves the shape rather than the rule. - ### 57. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds. Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. @@ -242,10 +236,6 @@ The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid fro `isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. -### 12. Strip the whitespace `data-import/territory.py` copies from its register. - -`misc.territory[CW].capital` renders ` Willemstad`, with a leading space. - ### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. Needs item 4. -- 2.52.0 From 951dd881695b5863d058716c5583d0804521ab01 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:38:02 +0200 Subject: [PATCH 479/966] Answer stability round 1: draw the exchange from its own category, en_US.phone-exchange, and file the area list's copy --- CHANGELOG.md | 4 ++-- README.md | 2 +- data/en_US/phone-exchange.json | 17 ++++++++++++++++ data/en_US/phone.json | 36 ++-------------------------------- shippedindex.go | 1 + todo.md | 7 ++++++- 6 files changed, 29 insertions(+), 38 deletions(-) create mode 100644 data/en_US/phone-exchange.json diff --git a/CHANGELOG.md b/CHANGELOG.md index 09777f2..ae5e331 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,8 +90,8 @@ replacement, and each removed path, column or flag. and carries the same three. `person.prefix` is null where a person has no title, where it used to be an empty string, so `--format sql` writes `NULL` and a `string` struct field reading it becomes `*string`. -- `en_US.phone` draws its exchange, `exch`, from the central office codes NANPA can - assign: `200` to `999`, never `N11`, `555`, `950`, `958`, `959` or `976`. +- `en_US.phone-exchange` draws a central office code NANPA can assign: `200` to `999`, + never `N11`, `555`, `950`, `958`, `959` or `976`. `en_US.phone` draws its `exch` from it. - `ErrNoColumns` is exported, so a caller can tell the one record fence a path can answer from the rest. - An error names a spelling that runs: a layout is named single-quoted and free of diff --git a/README.md b/README.md index f57ae5a..201ac22 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,7 @@ which binds a [name](#names), nor be `-`, which a struct tag reserves; dot-prefi Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, `username`, `version` and `word`, formatted per locale; `sv_SE` adds -`personnummer` and `samordningsnummer`, `en_US` adds `ssn` and `itin`. `misc` +`personnummer` and `samordningsnummer`, `en_US` adds `ssn`, `itin` and `phone-exchange`. `misc` carries `car`, `coordinate`, `creditcard` (Luhn-valid), `currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpmethod`, `httpstatus`, `language` (ISO 639-1, with its 639-2/T code), `loglevel` (syslog severity), `mac`, `mimetype`, `objectid`, diff --git a/data/en_US/phone-exchange.json b/data/en_US/phone-exchange.json new file mode 100644 index 0000000..8966fca --- /dev/null +++ b/data/en_US/phone-exchange.json @@ -0,0 +1,17 @@ +[ + { "format": "{int(2,9)}0{digits(1)}", "weight": 80 }, + { "format": "{int(2,9)}1{int(2,9)}", "weight": 64 }, + { "format": "{int(2,9)}10", "weight": 8 }, + { "format": "{int(2,4)}{int(2,9)}{digits(1)}", "weight": 240 }, + { "format": "{int(6,8)}{int(2,9)}{digits(1)}", "weight": 240 }, + { "format": "5{int(2,4)}{digits(1)}", "weight": 30 }, + { "format": "5{int(6,9)}{digits(1)}", "weight": 40 }, + { "format": "55{int(0,4)}", "weight": 5 }, + { "format": "55{int(6,9)}", "weight": 4 }, + { "format": "9{int(2,4)}{digits(1)}", "weight": 30 }, + { "format": "9{int(8,9)}{digits(1)}", "weight": 20 }, + { "format": "95{int(1,7)}", "weight": 7 }, + { "format": "96{digits(1)}", "weight": 10 }, + { "format": "97{int(0,5)}", "weight": 6 }, + { "format": "97{int(7,9)}", "weight": 3 } +] diff --git a/data/en_US/phone.json b/data/en_US/phone.json index a5e9f9e..30d4483 100644 --- a/data/en_US/phone.json +++ b/data/en_US/phone.json @@ -3,45 +3,13 @@ "format": "({area}) {exch}-{line}", "weight": 2, "area": ["202", "212", "213", "305", "312", "404", "415", "469", "503", "512", "602", "617", "646", "702", "718", "720", "737", "773", "786", "808", "917"], - "exch": [ - { "format": "{int(2,9)}0{digits(1)}", "weight": 80 }, - { "format": "{int(2,9)}1{int(2,9)}", "weight": 64 }, - { "format": "{int(2,9)}10", "weight": 8 }, - { "format": "{int(2,4)}{int(2,9)}{digits(1)}", "weight": 240 }, - { "format": "{int(6,8)}{int(2,9)}{digits(1)}", "weight": 240 }, - { "format": "5{int(2,4)}{digits(1)}", "weight": 30 }, - { "format": "5{int(6,9)}{digits(1)}", "weight": 40 }, - { "format": "55{int(0,4)}", "weight": 5 }, - { "format": "55{int(6,9)}", "weight": 4 }, - { "format": "9{int(2,4)}{digits(1)}", "weight": 30 }, - { "format": "9{int(8,9)}{digits(1)}", "weight": 20 }, - { "format": "95{int(1,7)}", "weight": 7 }, - { "format": "96{digits(1)}", "weight": 10 }, - { "format": "97{int(0,5)}", "weight": 6 }, - { "format": "97{int(7,9)}", "weight": 3 } - ], + "exch": "{.phone-exchange}", "line": "{digits(4)}" }, { "format": "{area}-{exch}-{line}", "area": ["202", "212", "213", "305", "312", "404", "415", "469", "503", "512", "602", "617", "646", "702", "718", "720", "737", "773", "786", "808", "917"], - "exch": [ - { "format": "{int(2,9)}0{digits(1)}", "weight": 80 }, - { "format": "{int(2,9)}1{int(2,9)}", "weight": 64 }, - { "format": "{int(2,9)}10", "weight": 8 }, - { "format": "{int(2,4)}{int(2,9)}{digits(1)}", "weight": 240 }, - { "format": "{int(6,8)}{int(2,9)}{digits(1)}", "weight": 240 }, - { "format": "5{int(2,4)}{digits(1)}", "weight": 30 }, - { "format": "5{int(6,9)}{digits(1)}", "weight": 40 }, - { "format": "55{int(0,4)}", "weight": 5 }, - { "format": "55{int(6,9)}", "weight": 4 }, - { "format": "9{int(2,4)}{digits(1)}", "weight": 30 }, - { "format": "9{int(8,9)}{digits(1)}", "weight": 20 }, - { "format": "95{int(1,7)}", "weight": 7 }, - { "format": "96{digits(1)}", "weight": 10 }, - { "format": "97{int(0,5)}", "weight": 6 }, - { "format": "97{int(7,9)}", "weight": 3 } - ], + "exch": "{.phone-exchange}", "line": "{digits(4)}" } ] diff --git a/shippedindex.go b/shippedindex.go index 32ef0e2..f6a96ea 100644 --- a/shippedindex.go +++ b/shippedindex.go @@ -14,6 +14,7 @@ var shippedIndex = map[string]shippedEntry{ "en_US.last-name": {paths: []string{"", "count", "name"}}, "en_US.person": {paths: []string{"", "first", "last", "prefix", "sex"}}, "en_US.phone": {paths: []string{"", "area", "exch", "line"}}, + "en_US.phone-exchange": {paths: []string{""}}, "en_US.price": {paths: []string{"", "amt", "cents"}}, "en_US.sentence": {paths: []string{"", "adj", "noun", "prep", "verb"}}, "en_US.sex": {paths: []string{"", "code", "first-name", "first-name.count", "first-name.name", "first-name.sex", "name", "title", "title.name", "title.sex", "title.share"}}, diff --git a/todo.md b/todo.md index 23dd2d0..eb5c2f9 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 86` +`Next ID = 87` | Goal | W | |---|---| @@ -108,6 +108,7 @@ | 30 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 75 | 0.3.0 | | **Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 6 | 4 | 7 | 11.2 | 11.5 | +| 86 | 0.3.0 | | **Read `en_US.phone`'s area codes from one category, as its exchange reads `en_US.phone-exchange`.** | 1 | 2 | 2 | 5 | 3.2 | 11.5 | | 33 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 5 | 10.8 | | 47 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 6 | 10.2 | | 72 | 0.3.0 | principle | **Fail the merge gate when test coverage falls below the last recorded figure.** | 2 | 3 | 2 | 5 | 3 | 10.2 | @@ -389,6 +390,10 @@ The CLI counts brackets of its own, so a change to the selector grammar desyncs `readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables), no `go` example (Library) and no `text` block of error text, and compares no "Renders e.g." output, so goal 11.2 is not met. +### 86. Read `en_US.phone`'s area codes from one category, as its exchange reads `en_US.phone-exchange`. + +Both phone formats carry the same 21-code `area` list, so an edit to one copy can miss the other. + ### 47. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. -- 2.52.0 From 2f712d6ec016bb9f474ed9c2db61353eeee13cda Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:38:02 +0200 Subject: [PATCH 480/966] Re-pin the shape for en_US.phone-exchange --- testdata/shipped_shape.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/testdata/shipped_shape.txt b/testdata/shipped_shape.txt index b7cab31..9cb563c 100644 --- a/testdata/shipped_shape.txt +++ b/testdata/shipped_shape.txt @@ -30,7 +30,8 @@ en_US.person.first string en_US.person.last string en_US.person.prefix string null en_US.person.sex string -en_US.phone +en_US.phone reads en_US.phone-exchange +en_US.phone-exchange en_US.phone.area en_US.phone.exch en_US.phone.line -- 2.52.0 From 59e3ebe65d60c873bb1658faa5859811a395db5e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:42:46 +0200 Subject: [PATCH 481/966] Answer product-owner round 1: name the field exchange, and say in plain words what phone-exchange draws --- CHANGELOG.md | 5 +++-- README.md | 21 +++++++++++---------- data/en_US/phone.json | 8 ++++---- shippedindex.go | 2 +- 4 files changed, 19 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ae5e331..a35c4ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,8 +90,9 @@ replacement, and each removed path, column or flag. and carries the same three. `person.prefix` is null where a person has no title, where it used to be an empty string, so `--format sql` writes `NULL` and a `string` struct field reading it becomes `*string`. -- `en_US.phone-exchange` draws a central office code NANPA can assign: `200` to `999`, - never `N11`, `555`, `950`, `958`, `959` or `976`. `en_US.phone` draws its `exch` from it. +- `en_US.phone-exchange` draws the three digits after a US area code, as NANPA assigns + them: `200` to `999`, never `211` to `911` ending in `11`, nor `555`, `950`, `958`, + `959` or `976`. `en_US.phone` draws its `exchange` from it. - `ErrNoColumns` is exported, so a caller can tell the one record fence a path can answer from the rest. - An error names a spelling that runs: a layout is named single-quoted and free of diff --git a/README.md b/README.md index 201ac22..6e9b4ee 100644 --- a/README.md +++ b/README.md @@ -161,16 +161,17 @@ which binds a [name](#names), nor be `-`, which a struct tag reserves; dot-prefi Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, -`username`, `version` and `word`, formatted per locale; `sv_SE` adds -`personnummer` and `samordningsnummer`, `en_US` adds `ssn`, `itin` and `phone-exchange`. `misc` -carries `car`, `coordinate`, `creditcard` (Luhn-valid), `currency` (ISO 4217), -`datetime` (RFC 3339), `emoji`, `httpmethod`, `httpstatus`, `language` (ISO 639-1, -with its 639-2/T code), `loglevel` (syslog severity), `mac`, `mimetype`, `objectid`, -`port`, `protocol`, `territory` (ISO 3166-1), `timezone` (IANA), `tld` (IANA root -zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.symbol`, -`misc.territory.alpha2`, `misc.httpstatus.code` — which `--list` shows. `car`, -`currency`, `httpmethod`, `httpstatus`, `language`, `loglevel`, `mimetype`, `port`, -`protocol`, `territory`, `timezone`, `tld` and `useragent` are [tables](#table), so +`username`, `version` and `word`, formatted per locale; `sv_SE` adds `personnummer` +and `samordningsnummer`, `en_US` adds `ssn`, `itin` and `phone-exchange` (the three +digits after the area code). `misc` carries `car`, `coordinate`, `creditcard` +(Luhn-valid), `currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpmethod`, +`httpstatus`, `language` (ISO 639-1, with its 639-2/T code), `loglevel` (syslog +severity), `mac`, `mimetype`, `objectid`, `port`, `protocol`, `territory` (ISO +3166-1), `timezone` (IANA), `tld` (IANA root zone), `useragent` and `uuid` (v4). Many +carry sub-fields — `misc.currency.symbol`, `misc.territory.alpha2`, +`misc.httpstatus.code` — which `--list` shows. `car`, `currency`, `httpmethod`, +`httpstatus`, `language`, `loglevel`, `mimetype`, `port`, `protocol`, `territory`, +`timezone`, `tld` and `useragent` are [tables](#table), so `misc.territory[SE].capital` and `misc.currency[Euro].symbol` select a row; `car` and `useragent` carry no key or name, so they are drawn from rather than selected in. diff --git a/data/en_US/phone.json b/data/en_US/phone.json index 30d4483..74ddd35 100644 --- a/data/en_US/phone.json +++ b/data/en_US/phone.json @@ -1,15 +1,15 @@ [ { - "format": "({area}) {exch}-{line}", + "format": "({area}) {exchange}-{line}", "weight": 2, "area": ["202", "212", "213", "305", "312", "404", "415", "469", "503", "512", "602", "617", "646", "702", "718", "720", "737", "773", "786", "808", "917"], - "exch": "{.phone-exchange}", + "exchange": "{.phone-exchange}", "line": "{digits(4)}" }, { - "format": "{area}-{exch}-{line}", + "format": "{area}-{exchange}-{line}", "area": ["202", "212", "213", "305", "312", "404", "415", "469", "503", "512", "602", "617", "646", "702", "718", "720", "737", "773", "786", "808", "917"], - "exch": "{.phone-exchange}", + "exchange": "{.phone-exchange}", "line": "{digits(4)}" } ] diff --git a/shippedindex.go b/shippedindex.go index f6a96ea..a14a807 100644 --- a/shippedindex.go +++ b/shippedindex.go @@ -13,7 +13,7 @@ var shippedIndex = map[string]shippedEntry{ "en_US.itin": {paths: []string{"", "area", "group", "serial"}}, "en_US.last-name": {paths: []string{"", "count", "name"}}, "en_US.person": {paths: []string{"", "first", "last", "prefix", "sex"}}, - "en_US.phone": {paths: []string{"", "area", "exch", "line"}}, + "en_US.phone": {paths: []string{"", "area", "exchange", "line"}}, "en_US.phone-exchange": {paths: []string{""}}, "en_US.price": {paths: []string{"", "amt", "cents"}}, "en_US.sentence": {paths: []string{"", "adj", "noun", "prep", "verb"}}, -- 2.52.0 From 965d671707ce40a402717fb4912b6b80c73a27e8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:42:46 +0200 Subject: [PATCH 482/966] Re-pin the shape for en_US.phone.exchange --- testdata/shipped_shape.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/testdata/shipped_shape.txt b/testdata/shipped_shape.txt index 9cb563c..135865f 100644 --- a/testdata/shipped_shape.txt +++ b/testdata/shipped_shape.txt @@ -33,7 +33,7 @@ en_US.person.sex string en_US.phone reads en_US.phone-exchange en_US.phone-exchange en_US.phone.area -en_US.phone.exch +en_US.phone.exchange en_US.phone.line en_US.price format "${amt}.{cents}" en_US.price.amt string -- 2.52.0 From 1af5abf762dd7015809d968e2e56b9df0471cf3d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:46:48 +0200 Subject: [PATCH 483/966] Answer prose round 1: say NANPA may assign the codes, spell the test's exchange in full, and file the record paragraph's split --- CHANGELOG.md | 6 +++--- data-import/tsv.py | 2 +- data_test.go | 4 ++-- todo.md | 13 +++++++++---- 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a35c4ec..7def0fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,9 +90,9 @@ replacement, and each removed path, column or flag. and carries the same three. `person.prefix` is null where a person has no title, where it used to be an empty string, so `--format sql` writes `NULL` and a `string` struct field reading it becomes `*string`. -- `en_US.phone-exchange` draws the three digits after a US area code, as NANPA assigns - them: `200` to `999`, never `211` to `911` ending in `11`, nor `555`, `950`, `958`, - `959` or `976`. `en_US.phone` draws its `exchange` from it. +- `en_US.phone-exchange` draws the three digits after a US area code from the codes + NANPA may assign: `200` to `999`, except `211`, `311` … `911`, `555`, `950`, `958`, + `959` and `976`. `en_US.phone` draws its `exchange` from it. - `ErrNoColumns` is exported, so a caller can tell the one record fence a path can answer from the rest. - An error names a spelling that runs: a layout is named single-quoted and free of diff --git a/data-import/tsv.py b/data-import/tsv.py index 333e9ab..eeebacc 100644 --- a/data-import/tsv.py +++ b/data-import/tsv.py @@ -5,7 +5,7 @@ from pathlib import Path def write(path, columns, rows): - """Write the rows as a TSV; the table needs two rows, and every cell is stripped, and must be non-empty and free of tabs, newlines and braces.""" + """Write the rows as a TSV of at least two rows, each cell stripped, non-empty and free of tabs, newlines and braces.""" lines = ["\t".join(columns)] for row in rows: cells = [str(row[c]).strip() for c in columns] diff --git a/data_test.go b/data_test.go index a92c7b8..8f9b0d0 100644 --- a/data_test.go +++ b/data_test.go @@ -341,8 +341,8 @@ func TestShippedUSPhone(t *testing.T) { if m == nil { t.Fatalf("phone %q does not match %s", n, re) } - if exch := m[1]; exch[1:] == "11" || unassignable[exch] { - t.Fatalf("phone %q has the unassignable exchange %s", n, exch) + if exchange := m[1]; exchange[1:] == "11" || unassignable[exchange] { + t.Fatalf("phone %q has the unassignable exchange %s", n, exchange) } } } diff --git a/todo.md b/todo.md index eb5c2f9..578a955 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 87` +`Next ID = 88` | Goal | W | |---|---| @@ -65,6 +65,7 @@ | 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 4 | 21.1 | | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | +| 87 | 0.1.0 | | **Split the README's record paragraph into one paragraph per rule, and name the shapes and the hold it refers to.** | 1 | 2 | 4 | 6 | 3, 11.3 | 17.3 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | @@ -108,7 +109,7 @@ | 30 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 75 | 0.3.0 | | **Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 6 | 4 | 7 | 11.2 | 11.5 | -| 86 | 0.3.0 | | **Read `en_US.phone`'s area codes from one category, as its exchange reads `en_US.phone-exchange`.** | 1 | 2 | 2 | 5 | 3.2 | 11.5 | +| 86 | 0.3.0 | | **Read both `en_US.phone` formats' area codes from one category, as their exchange reads `en_US.phone-exchange`.** | 1 | 2 | 2 | 5 | 3.2 | 11.5 | | 33 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 5 | 10.8 | | 47 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 6 | 10.2 | | 72 | 0.3.0 | principle | **Fail the merge gate when test coverage falls below the last recorded figure.** | 2 | 3 | 2 | 5 | 3 | 10.2 | @@ -171,7 +172,7 @@ Shape: T = table, t = template, c = choice. |---|---|---|---| | `ein` valid ranges | t | IRS | facts | | `company` names, suffix, `naics` | t+T | SEC tickers for patterns, NAICS 2022 | public domain | -| `phone` NANP with valid NPA/NXX | t | NANPA rules | facts | +| `phone` NANP with valid NPA (the NXX ships as `en_US.phone-exchange`) | t | NANPA rules | facts | | `routing` ABA with check, `bankaccount` | t | Fed prefix ranges | facts | | `licenseplate` per state | T | hand-authored patterns | facts | | `car` make, model | T | NHTSA vPIC | public domain | @@ -241,6 +242,10 @@ The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid fro Needs item 4. +### 87. Split the README's record paragraph into one paragraph per rule, and name the shapes and the hold it refers to. + +Needs item 4, which rewrites the draw-group rule this paragraph states. The paragraph under Records, from "The columns are the point" to "Correlated fields", holds six rules, so "put a value two fields share in its own category" is hard to find. "Renders as either shape" names no shape nearby, and "a field hold" is no term the README defines. + ### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. @@ -390,7 +395,7 @@ The CLI counts brackets of its own, so a change to the selector grammar desyncs `readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables), no `go` example (Library) and no `text` block of error text, and compares no "Renders e.g." output, so goal 11.2 is not met. -### 86. Read `en_US.phone`'s area codes from one category, as its exchange reads `en_US.phone-exchange`. +### 86. Read both `en_US.phone` formats' area codes from one category, as their exchange reads `en_US.phone-exchange`. Both phone formats carry the same 21-code `area` list, so an edit to one copy can miss the other. -- 2.52.0 From 663b20b77a401ff111576a3884d7199e54926b6a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 17:47:48 +0200 Subject: [PATCH 484/966] Answer prose round 2: say which paragraph item 87 splits, in the words it fixes, under goal 11 --- todo.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/todo.md b/todo.md index 578a955..81516e1 100644 --- a/todo.md +++ b/todo.md @@ -65,9 +65,9 @@ | 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 4 | 21.1 | | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | -| 87 | 0.1.0 | | **Split the README's record paragraph into one paragraph per rule, and name the shapes and the hold it refers to.** | 1 | 2 | 4 | 6 | 3, 11.3 | 17.3 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | +| 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 11, 11.3 | 12.5 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | @@ -242,10 +242,6 @@ The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid fro Needs item 4. -### 87. Split the README's record paragraph into one paragraph per rule, and name the shapes and the hold it refers to. - -Needs item 4, which rewrites the draw-group rule this paragraph states. The paragraph under Records, from "The columns are the point" to "Correlated fields", holds six rules, so "put a value two fields share in its own category" is hard to find. "Renders as either shape" names no shape nearby, and "a field hold" is no term the README defines. - ### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. @@ -254,6 +250,10 @@ Needs item 4, which rewrites the draw-group rule this paragraph states. The para The decision "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, the shipped ones they read, and the shipped ones whose reads reach an overridden category, so the index would carry each category's reads. +### 87. Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold". + +Needs item 4, which rewrites the draw-group rule this paragraph states. The last paragraph under Records, from "A record written only to emit columns" to the Decisions link, holds seven rules, so "put a value two fields share in its own category" is hard to find. "Renders as either shape" names no shape nearby, and the README defines no "field hold". + ### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. -- 2.52.0 From 61e759a489ebc3f7065e5a858eb3e87da9a079fe Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 19:29:32 +0200 Subject: [PATCH 485/966] Plan item 26 for 0.2.0: the + separator from a date the caller gives, exploring how a render takes input first --- todo.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/todo.md b/todo.md index 81516e1..c8898a5 100644 --- a/todo.md +++ b/todo.md @@ -62,13 +62,13 @@ | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | | 57 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | | 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 5.4 | 21.6 | -| 26 | 0.1.0 | defect | **Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100.** | 2 | 2 | 5 | 8 | 4 | 21.1 | | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 11, 11.3 | 12.5 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | +| 26 | 0.2.0 | defect | **Draw a `sv_SE.personnummer` aged 100 or more with the `+` separator Skatteverket spells, its age counted from a date the caller gives.** | 3 | 4 | 7 | 8 | 4 | 23.6 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | @@ -230,10 +230,6 @@ Nothing has shipped, so "no longer paths", "where it used to fail" and "where it - `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. - A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. -### 26. Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop drawing birthdates that reach 100. - -The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. - ### 34. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. `isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. @@ -258,6 +254,10 @@ Needs item 4, which rewrites the draw-group rule this paragraph states. The last GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. +### 26. Draw a `sv_SE.personnummer` aged 100 or more with the `+` separator Skatteverket spells, its age counted from a date the caller gives. + +The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the day it is read, which the decision "No builtin reads the clock, so a date is bounded by days, never by an age" keeps out of the generator. When this item is picked, first explore how a caller gives a render input data, such as today's date: inside the template, as a plain option like `WithNow` and `--now`, or both. Goal 9 holds only while that date is an input, never the clock read silently. + ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. The decision "A constant zero divisor is a load error; in a string column a divisor that is not constant prints `Inf`" lets `{calc(a/(b*c))}` with `b` fixed at `"0"` write `Inf` into every row. An operand that is sometimes not a number prints `NaN`. Both work against goals 6.1 and 4. `valueproof.go` already proves a typed column's divisor nonzero. -- 2.52.0 From 4a2e161cfcea851ea9c01a210451ea7783dc365a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 19:31:16 +0200 Subject: [PATCH 486/966] Answer prose round 1: the + from the year its holder turns 100, in both Swedish ids, from 2030 --- todo.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index c8898a5..630dbf2 100644 --- a/todo.md +++ b/todo.md @@ -68,7 +68,7 @@ | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 11, 11.3 | 12.5 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | -| 26 | 0.2.0 | defect | **Draw a `sv_SE.personnummer` aged 100 or more with the `+` separator Skatteverket spells, its age counted from a date the caller gives.** | 3 | 4 | 7 | 8 | 4 | 23.6 | +| 26 | 0.2.0 | defect | **Spell a `sv_SE.personnummer` and `sv_SE.samordningsnummer` with the `+` Skatteverket uses from the year its holder turns 100, counted from a date the caller gives.** | 3 | 4 | 7 | 8 | 4 | 23.6 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | @@ -254,9 +254,9 @@ Needs item 4, which rewrites the draw-group rule this paragraph states. The last GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. -### 26. Draw a `sv_SE.personnummer` aged 100 or more with the `+` separator Skatteverket spells, its age counted from a date the caller gives. +### 26. Spell a `sv_SE.personnummer` and `sv_SE.samordningsnummer` with the `+` Skatteverket uses from the year its holder turns 100, counted from a date the caller gives. -The format hard-codes `-`, and the 1930 floor makes the oldest draws invalid from 2031. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the day it is read, which the decision "No builtin reads the clock, so a date is bounded by days, never by an age" keeps out of the generator. When this item is picked, first explore how a caller gives a render input data, such as today's date: inside the template, as a plain option like `WithNow` and `--now`, or both. Goal 9 holds only while that date is an input, never the clock read silently. +Both formats hard-code `-`, and their earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. When this item is picked, first explore how a caller passes data, such as today's date, to a render: in the template, as a plain option like `WithNow` and `--now`, or both. ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. -- 2.52.0 From ef164354ab05fdbf1315c3d24d0c5d7717f49ae8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 19:32:21 +0200 Subject: [PATCH 487/966] Answer prose round 2: state the + for personnummer, as SKV 704 does, and check samordningsnummer when the item is picked --- todo.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index 630dbf2..d2e4fba 100644 --- a/todo.md +++ b/todo.md @@ -68,7 +68,7 @@ | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 11, 11.3 | 12.5 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | -| 26 | 0.2.0 | defect | **Spell a `sv_SE.personnummer` and `sv_SE.samordningsnummer` with the `+` Skatteverket uses from the year its holder turns 100, counted from a date the caller gives.** | 3 | 4 | 7 | 8 | 4 | 23.6 | +| 26 | 0.2.0 | defect | **Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives.** | 3 | 4 | 7 | 8 | 4 | 23.6 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | @@ -254,9 +254,9 @@ Needs item 4, which rewrites the draw-group rule this paragraph states. The last GoReleaser attaches the binaries to the release the tag workflow publishes, and builds deb and rpm packages, an Alpine APK and an AUR package beside Homebrew and Scoop. A checkout build prints `devel` for `--version`; the binaries carry the stamped tag. v0.1.0 ships first, so it can be tested before it is packaged. -### 26. Spell a `sv_SE.personnummer` and `sv_SE.samordningsnummer` with the `+` Skatteverket uses from the year its holder turns 100, counted from a date the caller gives. +### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. -Both formats hard-code `-`, and their earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. When this item is picked, first explore how a caller passes data, such as today's date, to a render: in the template, as a plain option like `WithNow` and `--now`, or both. +The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. `sv_SE.samordningsnummer` hard-codes `-` from the same date: check whether folkbokföringslagen 18 a § gives it the same `+`, since Skatteverket's SKV 704 states the rule for personnummer only. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. When this item is picked, first explore how a caller passes data, such as today's date, to a render: in the template, as a plain option like `WithNow` and `--now`, or both. ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. -- 2.52.0 From 84763399aa44ae8047450c1c204026f23299df54 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 19:33:27 +0200 Subject: [PATCH 488/966] Answer product-owner round 1: plan the + so it ships before 2030, on request and in a data author's own category, and file the 12-digit form --- todo.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index d2e4fba..b6227d4 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 88` +`Next ID = 89` | Goal | W | |---|---| @@ -68,9 +68,10 @@ | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 11, 11.3 | 12.5 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | -| 26 | 0.2.0 | defect | **Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives.** | 3 | 4 | 7 | 8 | 4 | 23.6 | +| 26 | 0.2.0 | defect | **Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives.** | 4 | 6 | 7 | 8 | 4 | 22.1 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | +| 88 | 0.2.0 | | **Add a 12-digit `YYYYMMDDNNNC` form of `sv_SE.personnummer` and `sv_SE.samordningsnummer`, the form Skatteverket's test series and many systems store.** | 2 | 3 | 6 | 6 | 4 | 19.4 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | | 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | | 28 | 0.2.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 4.1 | 18.9 | @@ -256,7 +257,13 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. -The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. `sv_SE.samordningsnummer` hard-codes `-` from the same date: check whether folkbokföringslagen 18 a § gives it the same `+`, since Skatteverket's SKV 704 states the rule for personnummer only. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. When this item is picked, first explore how a caller passes data, such as today's date, to a render: in the template, as a plain option like `WithNow` and `--now`, or both. +The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. Steps: +- Explore how a caller passes data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Bench the spelling; `--now` reads like the clock, and only the year counts. +- Decide and document what renders when the caller gives no date, keeping seeded output stable. +- Reach birthdates back past 100 years, as Skatteverket's test series does from 1890, so a `+` ships before 2030, and decide what share of draws carries one. +- Give a path that always draws a `+` number for the date given, so a test can ask for a centenarian. +- Spell the separator with a builtin or construct a data author's own category can use too, and update the README's personnummer recipe and its "a day in 1930–2025" count. +- Check whether Lag (2022:1697) om samordningsnummer gives `sv_SE.samordningsnummer`, which also hard-codes `-` from 1930, the same `+`; Skatteverket's SKV 704 states the rule for personnummer only. ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. @@ -266,6 +273,10 @@ The decision "A constant zero divisor is a load error; in a string column a divi `New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. +### 88. Add a 12-digit `YYYYMMDDNNNC` form of `sv_SE.personnummer` and `sv_SE.samordningsnummer`, the form Skatteverket's test series and many systems store. + +A system storing the century avoids the `+` altogether, so a fixture in that form tests the other half of the problem item 26 tests. + ### 41. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select a row today. Steps: -- 2.52.0 From 4bf1ee292a48476786d7ef2557053ab165492d63 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 19:34:40 +0200 Subject: [PATCH 489/966] Answer prose round 3 and product-owner round 2: one reading per step, no birthdate after the given date, and the 12-digit claim scoped to testpersonnummer --- todo.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/todo.md b/todo.md index b6227d4..a10d061 100644 --- a/todo.md +++ b/todo.md @@ -71,7 +71,7 @@ | 26 | 0.2.0 | defect | **Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives.** | 4 | 6 | 7 | 8 | 4 | 22.1 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | -| 88 | 0.2.0 | | **Add a 12-digit `YYYYMMDDNNNC` form of `sv_SE.personnummer` and `sv_SE.samordningsnummer`, the form Skatteverket's test series and many systems store.** | 2 | 3 | 6 | 6 | 4 | 19.4 | +| 88 | 0.2.0 | | **Add a 12-digit `YYYYMMDDNNNC` form of `sv_SE.personnummer` and `sv_SE.samordningsnummer`, the form Skatteverket's testpersonnummer series and many systems store.** | 2 | 3 | 6 | 6 | 4 | 19.4 | | 41 | 0.2.0 | | **Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states.** | 6 | 7 | 7 | 8 | 5.7 | 19.1 | | 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | | 28 | 0.2.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 4.1 | 18.9 | @@ -259,11 +259,11 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. Steps: - Explore how a caller passes data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Bench the spelling; `--now` reads like the clock, and only the year counts. -- Decide and document what renders when the caller gives no date, keeping seeded output stable. -- Reach birthdates back past 100 years, as Skatteverket's test series does from 1890, so a `+` ships before 2030, and decide what share of draws carries one. +- Decide and document what renders when the caller gives no date, so the output never depends on the day it runs. +- Reach birthdates back past 100 years, as Skatteverket's test series does from 1890, with birth numbers 980/981 before 1900, so a `+` ships before 2030; draw no birthdate after the given date, and decide what share of draws carries a `+`, such as enough that a few hundred rows hold one. - Give a path that always draws a `+` number for the date given, so a test can ask for a centenarian. -- Spell the separator with a builtin or construct a data author's own category can use too, and update the README's personnummer recipe and its "a day in 1930–2025" count. -- Check whether Lag (2022:1697) om samordningsnummer gives `sv_SE.samordningsnummer`, which also hard-codes `-` from 1930, the same `+`; Skatteverket's SKV 704 states the rule for personnummer only. +- Spell the separator with a builtin, or another construct, that a data author's own category can use too, and update the README's personnummer recipe and its "a day in 1930–2025" count. +- `sv_SE.samordningsnummer` also hard-codes `-` and draws from 1930. Check whether Lag (2022:1697) om samordningsnummer gives it the same `+`; folkbokföringslagen 18 § states the rule for personnummer only. ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. @@ -273,9 +273,9 @@ The decision "A constant zero divisor is a load error; in a string column a divi `New` and `NewTemplate` stop at the first, so data holding two mistakes takes two runs to fix. Split it into items before starting, one of them the shape a Go caller iterates and the CLI prints, and the order of an inline template's mistakes, which name no path; reword the README's "`New` refuses a mistake in the data" to every mistake. -### 88. Add a 12-digit `YYYYMMDDNNNC` form of `sv_SE.personnummer` and `sv_SE.samordningsnummer`, the form Skatteverket's test series and many systems store. +### 88. Add a 12-digit `YYYYMMDDNNNC` form of `sv_SE.personnummer` and `sv_SE.samordningsnummer`, the form Skatteverket's testpersonnummer series and many systems store. -A system storing the century avoids the `+` altogether, so a fixture in that form tests the other half of the problem item 26 tests. +A system that stores the 12-digit form never meets the `+`, so it needs a fixture in that form. ### 41. Key every table by one key column, as the decision "A table selects by one key column, a code, never a free-form name" states. -- 2.52.0 From 32c1fb6c266946778c5394dca73758b010d40f16 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 20:27:07 +0200 Subject: [PATCH 490/966] Record the maintainer's input-data direction in item 26, and the age bands that weigh a birthdate --- docs/research/research-age-bands.md | 10 ++++++++++ todo.md | 14 +++++++------- 2 files changed, 17 insertions(+), 7 deletions(-) create mode 100644 docs/research/research-age-bands.md diff --git a/docs/research/research-age-bands.md b/docs/research/research-age-bands.md new file mode 100644 index 0000000..bd10dac --- /dev/null +++ b/docs/research/research-age-bands.md @@ -0,0 +1,10 @@ +# Population by age band — research 2026-10-04 + +Share of the population in four bands, from each country's register, for weighting a drawn birthdate by age (todo.md item 26). Computed from single-year ages. + +| Country | Date | Total | 0–17 | 18–64 | 65–99 | 100+ | Source | +|---|---|---|---|---|---|---|---| +| SE | 2024-12-31 | 10,587,710 | 20.36% | 58.80% | 20.81% | 0.028% (2,961) | SCB Statistikdatabasen, table BE0101A/BefolkningNy, CC0 1.0 | +| US | 2024-07-01 | 340,110,988 | 21.50% | 60.51% | 17.96% | 0.029% (98,034) | Census Bureau, [nc-est2024-agesex-res.csv](https://www2.census.gov/programs-surveys/popest/datasets/2020-2024/national/asrh/nc-est2024-agesex-res.csv), public domain | + +At 0.03%, a fixture of 10,000 rows holds about three people aged 100 or more. diff --git a/todo.md b/todo.md index a10d061..3aba974 100644 --- a/todo.md +++ b/todo.md @@ -257,13 +257,13 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. -The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. Steps: -- Explore how a caller passes data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Bench the spelling; `--now` reads like the clock, and only the year counts. -- Decide and document what renders when the caller gives no date, so the output never depends on the day it runs. -- Reach birthdates back past 100 years, as Skatteverket's test series does from 1890, with birth numbers 980/981 before 1900, so a `+` ships before 2030; draw no birthdate after the given date, and decide what share of draws carries a `+`, such as enough that a few hundred rows hold one. -- Give a path that always draws a `+` number for the date given, so a test can ask for a centenarian. -- Spell the separator with a builtin, or another construct, that a data author's own category can use too, and update the README's personnummer recipe and its "a day in 1930–2025" count. -- `sv_SE.samordningsnummer` also hard-codes `-` and draws from 1930. Check whether Lag (2022:1697) om samordningsnummer gives it the same `+`; folkbokföringslagen 18 § states the rule for personnummer only. +The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. Explore the first step before the rest: +- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Bench the spelling; `--now` reads like the clock, and only the year counts. The same input can narrow what is drawn: only adults, only children, only 65 and over, or a mix. That also covers a test asking for a centenarian. +- The maintainer's direction, 2026-10-04: the library requires every input that can change, such as the date and the randomness, and the CLI defaults to the system's when its flag is absent. Today `New` without `WithSeed` draws its own randomness, so this reaches beyond the date. +- Draw birthdates far enough back that a `+` can appear today. Read in 2026, the oldest shipped birthdate is 96 years old, so no draw carries a `+` until 2030. Skatteverket's test series reaches back to 1890, with birth numbers 980/981 before 1900. Draw no birthdate after the given date. +- Weight the ages by the territory's population, so the share of `+` numbers follows its demography: [research-age-bands.md](docs/research/research-age-bands.md). +- Let a data author's own category spell the separator too, through a builtin or a function a data module provides (item 77), and update the README's personnummer recipe and its "a day in 1930–2025" count. +- Research: `sv_SE.samordningsnummer` also hard-codes `-` and draws from 1930. Check whether Lag (2022:1697) om samordningsnummer gives it the same `+`; folkbokföringslagen 18 § states the rule for personnummer only. ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. -- 2.52.0 From f3fcf569ad8edae2be5bb5a3b48a838cd5749b19 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 20:28:50 +0200 Subject: [PATCH 491/966] Answer prose round 1: say item 26 revises the clock decision, fold the direction into its first step, and leave narrowing and the seed to their own items --- docs/research/research-age-bands.md | 2 +- todo.md | 7 +++---- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/research/research-age-bands.md b/docs/research/research-age-bands.md index bd10dac..756ec77 100644 --- a/docs/research/research-age-bands.md +++ b/docs/research/research-age-bands.md @@ -1,6 +1,6 @@ # Population by age band — research 2026-10-04 -Share of the population in four bands, from each country's register, for weighting a drawn birthdate by age (todo.md item 26). Computed from single-year ages. +Share of the population in four bands, from each country's statistics office, for weighting a drawn birthdate by age. Computed from single-year ages. | Country | Date | Total | 0–17 | 18–64 | 65–99 | 100+ | Source | |---|---|---|---|---|---|---|---| diff --git a/todo.md b/todo.md index 3aba974..fbf9f14 100644 --- a/todo.md +++ b/todo.md @@ -257,10 +257,9 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. -The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. Explore the first step before the rest: -- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Bench the spelling; `--now` reads like the clock, and only the year counts. The same input can narrow what is drawn: only adults, only children, only 65 and over, or a mix. That also covers a test asking for a centenarian. -- The maintainer's direction, 2026-10-04: the library requires every input that can change, such as the date and the randomness, and the CLI defaults to the system's when its flag is absent. Today `New` without `WithSeed` draws its own randomness, so this reaches beyond the date. -- Draw birthdates far enough back that a `+` can appear today. Read in 2026, the oldest shipped birthdate is 96 years old, so no draw carries a `+` until 2030. Skatteverket's test series reaches back to 1890, with birth numbers 980/981 before 1900. Draw no birthdate after the given date. +The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. This item revises it: once the caller gives the date, an age is a bound, and the CLI's default date makes output change with the day it runs unless the flag is given. Do the first step before the rest: +- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. The library requires every input that can change, such as the date and the randomness; the CLI defaults to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. Requiring the randomness changes `New`, which today seeds itself without `WithSeed`; split that into its own item when this one is picked. +- Draw birthdates back past 100 years before the given date, so a `+` can appear. Read in 2026, the oldest shipped birthdate is 96 years old, so no draw carries a `+` until 2030. Skatteverket's test series reaches back to 1890, with birth numbers 980/981 before 1900. Draw no birthdate after the given date. - Weight the ages by the territory's population, so the share of `+` numbers follows its demography: [research-age-bands.md](docs/research/research-age-bands.md). - Let a data author's own category spell the separator too, through a builtin or a function a data module provides (item 77), and update the README's personnummer recipe and its "a day in 1930–2025" count. - Research: `sv_SE.samordningsnummer` also hard-codes `-` and draws from 1930. Check whether Lag (2022:1697) om samordningsnummer gives it the same `+`; folkbokföringslagen 18 § states the rule for personnummer only. -- 2.52.0 From dc48ae11a5b77ffc3a16c7361909ce6e6ca3796d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 20:29:36 +0200 Subject: [PATCH 492/966] Answer prose round 2: state the library's required inputs as a step --- todo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/todo.md b/todo.md index fbf9f14..543eddb 100644 --- a/todo.md +++ b/todo.md @@ -258,7 +258,7 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. This item revises it: once the caller gives the date, an age is a bound, and the CLI's default date makes output change with the day it runs unless the flag is given. Do the first step before the rest: -- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. The library requires every input that can change, such as the date and the randomness; the CLI defaults to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. Requiring the randomness changes `New`, which today seeds itself without `WithSeed`; split that into its own item when this one is picked. +- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Make the library require every input that can change, such as the date and the randomness, and let the CLI default each to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. Requiring the randomness changes `New`, which today seeds itself without `WithSeed`; split that into its own item when this one is picked. - Draw birthdates back past 100 years before the given date, so a `+` can appear. Read in 2026, the oldest shipped birthdate is 96 years old, so no draw carries a `+` until 2030. Skatteverket's test series reaches back to 1890, with birth numbers 980/981 before 1900. Draw no birthdate after the given date. - Weight the ages by the territory's population, so the share of `+` numbers follows its demography: [research-age-bands.md](docs/research/research-age-bands.md). - Let a data author's own category spell the separator too, through a builtin or a function a data module provides (item 77), and update the README's personnummer recipe and its "a day in 1930–2025" count. -- 2.52.0 From 90be0ee6d88bd1eac767895b9032cb47f5a69b95 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 22:07:37 +0200 Subject: [PATCH 493/966] Make everything replaceable goal 9, require changing inputs in the library and default them in the CLI, and record item 77's answers --- DATA-LICENSES.md | 2 +- README.md | 41 ++++++++++---- docs/decisions.md | 20 +++---- docs/research/research-sources-se.md | 2 +- todo.md | 81 +++++++++++++++++----------- 5 files changed, 93 insertions(+), 53 deletions(-) diff --git a/DATA-LICENSES.md b/DATA-LICENSES.md index 368b611..f8e527f 100644 --- a/DATA-LICENSES.md +++ b/DATA-LICENSES.md @@ -14,7 +14,7 @@ Every shipped dataset, its source, its licence and the attribution it asks for. | `sv_SE/first-name.tsv`, `last-name.tsv` | [SCB](https://www.scb.se/) names with at least two bearers, 31 December 2022 | CC0 1.0 | "Källa: SCB" | `data-import/names-se.py` | | `en_US/first-name.tsv` | [SSA](https://www.ssa.gov/oact/babynames/) baby names, births 1930 to 2020, through [hackerb9/ssa-baby-names](https://github.com/hackerb9/ssa-baby-names) | public domain | none required | `data-import/names-us.py` | | `en_US/last-name.tsv` | Census Bureau surnames occurring 100 or more times, 2010 | public domain | none required | `data-import/names-us.py` | -| `sv_SE/sex.tsv`, `sv_SE/birth-number.tsv`, `sv_SE/title.tsv`, `en_US/sex.tsv`, `en_US/title.tsv`, `misc/car.tsv` | curated; Skatteverket's test birth numbers are facts, and `car.tsv` waits for an international source (goal 13) | — | — | — | +| `sv_SE/sex.tsv`, `sv_SE/birth-number.tsv`, `sv_SE/title.tsv`, `en_US/sex.tsv`, `en_US/title.tsv`, `misc/car.tsv` | curated; Skatteverket's test birth numbers are facts, and `car.tsv` waits for an international source (goal 14) | — | — | — | | `misc/currency.tsv` | [datasets/currency-codes](https://github.com/datasets/currency-codes); symbols from [Unicode CLDR](https://github.com/unicode-org/cldr) `en.xml` and `root.xml` | PDDL 1.0; [Unicode License v3](https://www.unicode.org/license.txt) | CLDR: "Copyright © 1991-2025 Unicode, Inc. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other countries." | `data-import/currency.py` | | `misc/httpmethod.tsv` | [IANA HTTP Method Registry](https://www.iana.org/assignments/http-methods/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpmethod.py` | | `misc/httpstatus.tsv` | [IANA HTTP Status Code Registry](https://www.iana.org/assignments/http-status-codes/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpstatus.py` | diff --git a/README.md b/README.md index 6e9b4ee..e8f51b1 100644 --- a/README.md +++ b/README.md @@ -177,9 +177,9 @@ carry sub-fields — `misc.currency.symbol`, `misc.territory.alpha2`, Where a script under [`data-import/`](data-import) reads a source, the facts come from that source, and the script drops a row only by a rule it states; elsewhere a small hand-written -set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 13). Copy +set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 14). Copy that states no fact, such as `word` and `sentence`, stays hand-written. Where a category -has no path that draws only values reaching nothing real (goal 15), write a fixed value such +has no path that draws only values reaching nothing real (goal 16), write a fixed value such as `example.com` into the template; `todo.md` plans the path. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. @@ -812,7 +812,7 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance -The target is about a microsecond per value (goal 12). +The target is about a microsecond per value (goal 13). With only the shipped set, `New` loads nothing: a category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads it, with every category it reads and @@ -889,6 +889,8 @@ Where the project is heading; the sections before Audience document what ships t 1. It covers the ten most-spoken languages and the Nordic countries. 2. Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates. + 3. A library gets only the data and functions it hands to `New`, and fejkdata's own + load the same way as anyone else's. 3. **The code is easy to understand.** 1. You can go from a bug to the code behind it without asking anyone. 2. You can change one part without keeping the rest in your head. @@ -916,23 +918,42 @@ Where the project is heading; the sections before Audience document what ships t wrong.** 1. A template that could render a value its own rules forbid, such as `NaN` or `Inf`, fails to load instead. + 2. Data that reads something nobody provided fails to load, and the error names the + package that provides it by default. 7. **One command gets you a value.** 1. Flags work like in other command-line tools, and can go anywhere on the line. 2. Your first template of your own needs no escaping and no options. + 3. The CLI fills in what you leave out with a sane default: all the shipped data, the + system's randomness and today's date. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** -9. **The same seed, version and data always give the same output.** -10. **It depends on nothing but Go's standard library.** -11. **The docs are a map of the template language.** +9. **Everything is replaceable.** Data, functions, and the packages they depend on can + come from fejkdata or anyone else. + 1. A package names the packages it reads by default, and anything providing the same + paths can stand in for one. + 2. Loading two things under one name fails, in the library and the CLI alike, unless + you say the second replaces the first. + 3. The template language's own functions stay in the core, so a template means the + same in every program. + 4. A function loaded from a package can ask for the input it needs, and gets the + randomness and the date the generator uses. + 5. A CLI carrying your own packages is a few lines of Go, and works just like + `fejkdata`. +10. **The same seed, version and data always give the same output.** + 1. A library call takes everything that can change its output from its caller: data, + randomness and the date. +11. **It depends on nothing but Go's standard library.** +12. **The docs are a map of the template language.** 1. Every feature has its own heading. 2. Every example runs as a test and shows its output. 3. Each rule is stated in one place. -12. **It is fast enough that you never notice it.** -13. **Every fact comes from an official or open source,** such as a government +13. **It is fast enough that you never notice with the shipped data, and its cost grows + with the data a render reaches, not with all the data loaded.** +14. **Every fact comes from an official or open source,** such as a government register. 1. A table holds every row its source holds, and nothing added by hand. -14. **The most common data comes first,** such as names, addresses, phone numbers and +15. **The most common data comes first,** such as names, addresses, phone numbers and dates. -15. **Values look real by default. You can also ask for values that can never reach +16. **Values look real by default. You can also ask for values that can never reach anything real,** such as a domain nobody can register. ## Layout diff --git a/docs/decisions.md b/docs/decisions.md index d2b3a28..050e1ad 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -120,7 +120,7 @@ still lives in `data/`; `--data-path` layers over it. 2026-10-03, a restructure the maintainer approved. -Goal 12: `New` pays nothing for a category a run never reaches, which matters once the +Goal 13: `New` pays nothing for a category a run never reaches, which matters once the shipped set grows to hundreds of MB. A category loads with every category it references and its whole table family, and binds as a whole load does, so it renders the same as after a whole load. `List` reads `shippedindex.go`, which `go generate` writes from @@ -130,7 +130,7 @@ reach. Goal 6: beside a `--data-path`, `WithDataPath` or `WithDataFS`, every category loads in `New`, so each mistake in the user's data is `New`'s error. That parses the whole shipped -set, against goal 12, until `todo.md` item 79 ends it. +set, against goal 13, until `todo.md` item 79 ends it. Valid while CI proves the shipped set whole and each category alone. @@ -353,7 +353,7 @@ a reader meeting two spellings of one empty set. Making the maps where the draws record's on its frame's stack, but a `Fake`'s draws live on the `Generator`, so there it cost the cheapest render two heap allocations and 240→390 ns, measured 2026-09-28. Lazily, a render reading a reference path pays those two allocations, a record or struct -included, and one reading none pays nothing; goal 12 holds either way. +included, and one reading none pays nothing; goal 13 holds either way. `todo.md` item 4 deletes what this decides. @@ -459,7 +459,7 @@ The benchmark suite (the README's Development) reports time for a human, not as ## Rows live in a TSV, the shape in JSON -2026-09-17, Lilleman auf Larv. Goals 8 and 12. +2026-09-17, Lilleman auf Larv. Goals 8 and 13. `New` allocates once per node, so a register of thirty thousand rows written as JSON objects would cost it a second; a TSV is one allocation whose cells are substrings, and @@ -638,7 +638,7 @@ is until a real data set needs the indexed form. 2026-09-17, Lilleman auf Larv. -Goals 9 and 12: a path that fails moves no seeded stream, at the cost of one draw-free +Goals 10 and 13: a path that fails moves no seeded stream, at the cost of one draw-free walk per call, compiling its steps into a stack buffer of 16. Valid while that walk costs little against the draw it guards. @@ -776,7 +776,7 @@ which says "Chinese, Mandarin" for `zh`. Where every territory row has a child the column is a `parent`, and the import drops the child rows whose territory the set does not ship — 17 of `misc.timezone`'s, Antarctica's ten among them. Agreement across a record is worth more than the last rows of a table: -goal 4.1 wins over goal 13.1 here, as the maintainer confirmed on 2026-10-03. +goal 4.1 wins over goal 14.1 here, as the maintainer confirmed on 2026-10-03. Where no such link can hold the fact stays a column. Layer your own `misc.territory` over the shipped one and you must layer `misc.timezone` too, or the link fails at load naming the row. @@ -786,7 +786,7 @@ naming the row. 2026-09-20, Lilleman auf Larv. `misc.httpmethod`, `misc.port`, `misc.httpstatus`, `misc.mimetype` and `misc.tld` weigh -every row alike, so GET is a ninth of the methods drawn. Goal 13 keeps an authored fact +every row alike, so GET is a ninth of the methods drawn. Goal 14 keeps an authored fact out of a sourced table, and no register publishes how often a method, a port or a TLD is used, so a weight here would be invented. Where one exists it is read, as `misc.timezone` reads GeoNames populations and `sv_SE.first-name` SCB bearers. @@ -796,7 +796,7 @@ used, so a weight here would be invented. Where one exists it is read, as 2026-09-24, Lilleman auf Larv. The weight is the population GeoNames records in the zone's cities of 15,000 or more, -floored at 15,000, which goal 15 asks for: over 300 seeded draws of +floored at 15,000, which goal 16 asks for: over 300 seeded draws of `misc.territory[US].timezone`, the four zones most Americans live in took 278 where an even weight gave them 44, and `America/Indiana/Petersburg`, a town of 2,400, fell from 17 to 0. Valid while a draw weighted this way lands where people live. @@ -845,7 +845,7 @@ with no separator. `misc.tld[se]` misses for it, which `todo.md` item 44 ends. 2026-09-20, Lilleman auf Larv. A `parent` demands a child for every parent row, so linking them would drop every root -zone row naming no territory, which is most of them, and goal 13 holds a sourced table +zone row naming no territory, which is most of them, and goal 14 holds a sourced table whole. The loader refuses the link outright anyway: `tld` is a column of `misc.territory`, and a table may not be named like a column of its ancestor. @@ -901,7 +901,7 @@ the direct chain is the one a reader can predict from the tables' parents. 2026-09-30, larv-review on the comprehension round the maintainer approved on 2026-09-29; replaces "The path walks are separate loops". -Goals 3 and 12: `drawSteps` draws every path from its steps, compiled at link by +Goals 3 and 13: `drawSteps` draws every path from its steps, compiled at link by `pathCheck` or, for a caller's path, by `probePath` into a stack buffer. The check and the probe stay two loops: on 2026-09-29 one loop over all three walks, switching on a mode field, leaked the check's leaves and errors with the draw's pins, arm and memo, diff --git a/docs/research/research-sources-se.md b/docs/research/research-sources-se.md index a2d1948..9ce0dbb 100644 --- a/docs/research/research-sources-se.md +++ b/docs/research/research-sources-se.md @@ -90,7 +90,7 @@ Recommendation: embed SCB 2022 xlsx-derived lists (top-N per sheet with counts) ## 3. Phone - Source: PTS "The Swedish numbering plan for telephony according to ITU-T E.164", 2024-01-08 (verified, text extracted): https://pts.se/globalassets/globala-block/nummertillstand/the-swedish-numbering-plan-for-telephony-according-to-itu---2024-01-08.pdf. Swedish "nrplansammanstallning-2026-02-04.pdf" and https://nummer.pts.se/NbrPlanSearch are behind a Radware bot check (both curl and Playwright blocked). No licence stated by PTS anywhere found; pts.se says contact pts@pts.se for data. -- Reserved for fiction, PTS "Telefonnummer till böcker och filmer" (verified 2026-09-21; curl and Playwright are Radware-blocked, a rendering fetcher gets through): https://www.pts.se/internet-och-telefoni/telefonnummer-och-adressering/telefonnummer-till-bocker-och-filmer/ — mobile 070-1740605–1740699, geographic 08-46500400–46500499 (Stockholm), 031-3900600–3900699 (Göteborg), 040-6280400–6280499 (Malmö), 0980-319200–319299 (Kiruna). 495 numbers PTS holds back so they reach nobody, which is goal 15's never-reaching path for `sv_SE.phone`. Same licence status as the numbering plan: none stated, facts only, cite PTS. The mobile block fits the shipped `{a} {b} {c}` grouping (`174 06 05`–`99`), as do 031 and 040; Stockholm's 8 subscriber digits and Kiruna's 6 need a format of their own, and 0980 is not among the 20 shipped area codes. +- Reserved for fiction, PTS "Telefonnummer till böcker och filmer" (verified 2026-09-21; curl and Playwright are Radware-blocked, a rendering fetcher gets through): https://www.pts.se/internet-och-telefoni/telefonnummer-och-adressering/telefonnummer-till-bocker-och-filmer/ — mobile 070-1740605–1740699, geographic 08-46500400–46500499 (Stockholm), 031-3900600–3900699 (Göteborg), 040-6280400–6280499 (Malmö), 0980-319200–319299 (Kiruna). 495 numbers PTS holds back so they reach nobody, which is goal 16's never-reaching path for `sv_SE.phone`. Same licence status as the numbering plan: none stated, facts only, cite PTS. The mobile block fits the shipped `{a} {b} {c}` grouping (`174 06 05`–`99`), as do 031 and 040; Stockholm's 8 subscriber digits and Kiruna's 6 need a format of their own, and 0980 is not among the 20 shipped area codes. - 264 geographic area codes (PDF text yields 263 "Area code for" rows; one likely split across a page break — flag). Area code = trunk "0" + NDC of 1–3 digits (08 Stockholm, 031 Göteborg, 040 Malmö, 0480 Kalmar). Geographic N(S)N (NDC + subscriber) max 9 digits, min 7 (2-digit NDC) or 8 (3-digit NDC); Stockholm min 7. Each row in the PDF gives NDC, max/min length, "Area code for ", so a `{ndc, name}` list can be built from it. - Mobile: NDC 70, 72, 73, 76, 79 — N(S)N exactly 9 digits, i.e. 07X-XXX XX XX (10 digits with trunk 0). Others: 71 mobile broadband/M2M (13 digits), 74 paging, 75 personal numbering, 77 shared cost, 20 freephone, 10 location-independent (10 AXX XX XX, A=1–8), 378 M2M fixed (10 digits). - Formatting (Språkrådet/Isof frågelådan, via search excerpts): hyphen after area code, subscriber digits grouped 2–3 from the left: `08-668 01 50`, `031-123 45 67`, `0480-123 45`, `070-123 45 67` (also `0701-23 45 67` accepted). International: `+46 70 123 45 67` (drop trunk 0). diff --git a/todo.md b/todo.md index 543eddb..0e13efc 100644 --- a/todo.md +++ b/todo.md @@ -15,6 +15,7 @@ | 2 | 0.95 | | 2.1 | 0.95 | | 2.2 | 0.95 | +| 2.3 | 0.95 | | 3 | 0.90 | | 3.1 | 0.90 | | 3.2 | 0.90 | @@ -33,31 +34,40 @@ | 5.8 | 0.80 | | 6 | 0.75 | | 6.1 | 0.75 | +| 6.2 | 0.75 | | 7 | 0.70 | | 7.1 | 0.70 | | 7.2 | 0.70 | +| 7.3 | 0.70 | | 8 | 0.65 | | 9 | 0.60 | +| 9.1 | 0.60 | +| 9.2 | 0.60 | +| 9.3 | 0.60 | +| 9.4 | 0.60 | +| 9.5 | 0.60 | | 10 | 0.55 | +| 10.1 | 0.55 | | 11 | 0.50 | -| 11.1 | 0.50 | -| 11.2 | 0.50 | -| 11.3 | 0.50 | | 12 | 0.45 | +| 12.1 | 0.45 | +| 12.2 | 0.45 | +| 12.3 | 0.45 | | 13 | 0.40 | -| 13.1 | 0.40 | | 14 | 0.35 | +| 14.1 | 0.35 | | 15 | 0.30 | +| 16 | 0.25 | ## Items | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 77 | 0.1.0 | decision | **Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI.** | 5 | 8 | 10 | 10 | 2, 1.1 | 33.0 | -| 78 | 0.1.0 | decision | **Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population.** | 6 | 7 | 10 | 10 | 2, 13.1 | 31.2 | +| 77 | 0.1.0 | decision | **Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI.** | 5 | 10 | 10 | 10 | 2, 1.1, 9 | 32.5 | +| 78 | 0.1.0 | decision | **Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population.** | 6 | 7 | 10 | 10 | 2, 14.1 | 31.2 | | 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | -| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | -| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | +| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 15 | 28.1 | +| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 15, 14 | 25.9 | | 3 | 0.1.0 | decision | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | | 57 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | @@ -65,8 +75,8 @@ | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | -| 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 | -| 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 11, 11.3 | 12.5 | +| 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 13 | 12.9 | +| 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 12, 12.3 | 11.9 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 26 | 0.2.0 | defect | **Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives.** | 4 | 6 | 7 | 8 | 4 | 22.1 | | 81 | 0.2.0 | decision | **Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 6.1, 4 | 20.6 | @@ -76,8 +86,8 @@ | 52 | 0.2.0 | | **Name the file a link error comes from beside its category path, `sub.x (/d/a/sub/x.json): …`.** | 1 | 2 | 5 | 7 | 6 | 19.0 | | 28 | 0.2.0 | | **Accept a middle name, and draw a `personnummer` whose sex matches a sex selected through a named pick.** | 4 | 4 | 6 | 7 | 4.1 | 18.9 | | 24 | 0.2.0 | defect | **Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does not hold.** | 1 | 1 | 4 | 7 | 4 | 18.6 | -| 55 | 0.2.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 11 | 17.5 | | 53 | 0.2.0 | | **Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.** | 4 | 5 | 6 | 6 | 4 | 16.9 | +| 55 | 0.2.0 | | **Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the README's Library section.** | 1 | 2 | 6 | 7 | 12 | 16.8 | | 18 | 0.2.0 | | **Give the address records one column set across countries: `region` and `municipality` as columns on `geo.SE.address` too.** | 4 | 4 | 6 | 6 | 5 | 16.6 | | 44 | 0.2.0 | decision | **Have `misc.tld[se]` select the row keyed `.se`, or have its miss name `[.se]`.** | 1 | 1 | 4 | 6 | 6 | 15.8 | | 54 | 0.2.0 | defect | **Read the tag through `git/ref/tags/{tag}` in `publish_release.py`.** | 2 | 1 | 3 | 6 | 1.1 | 15.8 | @@ -87,12 +97,12 @@ | 50 | 0.2.0 | | **Reword `prove`'s refusal of a typed column reading a row, `{/region}`.** | 1 | 2 | 4 | 6 | 6 | 15.5 | | 17 | 0.2.0 | | **Give every shipped category a column for each part its format composes, and one record shape across the national ids; re-pin the shape.** | 5 | 6 | 6 | 6 | 5 | 15.1 | | 48 | 0.2.0 | defect | **Refuse a `Fake` path into a level carrying a `repeat`, as the README's Correlated fields section states.** | 2 | 2 | 4 | 6 | 5 | 15.1 | -| 22 | 0.2.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 15 | 14.8 | | 29 | 0.2.0 | | **Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`.** | 4 | 4 | 5 | 6 | 5 | 14.6 | | 42 | 0.2.0 | | **Refuse a struct tag's `datatype` naming the Go type that sets it before proving its values, as the README's Library section promises.** | 2 | 2 | 4 | 6 | 6 | 14.5 | | 32 | 0.2.0 | | **Add `misc.browser` as the parent of `misc.useragent`, so `misc.browser[Chrome].useragent` resolves.** | 3 | 3 | 4 | 6 | 4.1 | 14.4 | | 27 | 0.2.0 | | **Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`.** | 3 | 2 | 4 | 6 | 5.7 | 14.1 | -| 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 9 | 14.0 | +| 22 | 0.2.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 16 | 14.0 | +| 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 10 | 14.0 | | 14 | 0.3.0 | | **Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone `{digits(1)}` at `New`, naming that spelling.** | 3 | 3 | 4 | 6 | 5.7 | 13.9 | | 38 | 0.3.0 | defect | **Compare `calcParser.binary`'s operator as a rune.** | 1 | 1 | 3 | 6 | 6 | 13.8 | | 45 | 0.3.0 | | **Name the node a selector follows in `stepInto`'s refusal.** | 1 | 1 | 3 | 6 | 6 | 13.8 | @@ -103,27 +113,27 @@ | 10 | 0.3.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 3.2 | 13.1 | | 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 4 | 13.0 | | 35 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 8 | 13.0 | -| 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 11 | 12.8 | -| 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 15 | 11.7 | +| 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 12 | 12.2 | | 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | | 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | | 30 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | -| 75 | 0.3.0 | | **Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 6 | 4 | 7 | 11.2 | 11.5 | | 86 | 0.3.0 | | **Read both `en_US.phone` formats' area codes from one category, as their exchange reads `en_US.phone-exchange`.** | 1 | 2 | 2 | 5 | 3.2 | 11.5 | +| 23 | 0.3.0 | | **Give every other shipped category that could reach something real a path that never does.** | 3 | 6 | 6 | 7 | 16 | 11.0 | +| 75 | 0.3.0 | | **Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render.** | 2 | 6 | 4 | 7 | 12.2 | 10.8 | | 33 | 0.3.0 | | **Group `data/misc` into folders where a group name makes a path easier to guess.** | 4 | 5 | 4 | 5 | 5 | 10.8 | | 47 | 0.3.0 | defect | **Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there.** | 1 | 1 | 2 | 5 | 6 | 10.2 | | 72 | 0.3.0 | principle | **Fail the merge gate when test coverage falls below the last recorded figure.** | 2 | 3 | 2 | 5 | 3 | 10.2 | | 7 | 0.3.0 | | **Fill `arm` in one place.** | 4 | 4 | 2 | 6 | 3.2 | 9.8 | -| 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 13.1 | 9.8 | | 37 | 0.3.0 | | **Rename `newRand` to `newGeneratorState` and `Generator.rand` to `state`.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | | 43 | 0.3.0 | | **Call `templateError` a compile failure in its doc.** | 1 | 1 | 1 | 5 | 3.3 | 9.8 | | 82 | 0.3.0 | principle | **Pass `newRand` its entropy source, so no test swaps the package's `randomBytes`.** | 1 | 1 | 1 | 5 | 3.2 | 9.8 | +| 19 | 0.3.0 | | **Fill the 398 Swedish localities weighted 200 from SCB småorter.** | 2 | 4 | 4 | 6 | 14.1 | 9.2 | | 56 | 0.3.0 | defect | **Build on a manual run of `test.yml`.** | 1 | 1 | 1 | 4 | 1 | 8.8 | | 83 | 0.3.0 | principle | **Read a name binding's head and tail from its binder's link, and delete `linkBindings`.** | 4 | 3 | 1 | 6 | 3.2 | 8.1 | | 63 | 0.4.0 | | **Pair a street with its exact postnummer.** | 3 | 6 | 5 | 8 | 4.1 | 19.1 | | 62 | 0.4.0 | | **Add `{btc()}` and `{eth()}`.** | 2 | 4 | 4 | 6 | 4 | 15.2 | -| 59 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 11 | 13.8 | +| 59 | 0.4.0 | | **Promise in `Generator`'s godoc that its renders run one at a time.** | 1 | 1 | 5 | 5 | 12 | 13.2 | | 65 | 0.4.0 | | **Hold every change `AGENTS.md` says owes a `CHANGELOG.md` entry to one in CI.** | 2 | 3 | 4 | 5 | 5.8 | 13.2 | | 66 | 0.4.0 | | **Cut the README's Layout block to the lines that say what a file name cannot.** | 1 | 1 | 2 | 5 | 3.4 | 11.8 | | 67 | 0.4.0 | | **Cut `AGENTS.md` to the rules only it states, and explain every term its comprehension rule uses.** | 1 | 2 | 1 | 6 | 3.4 | 11.3 | @@ -136,7 +146,16 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Goal 2 says what ships where. The maintainer decided on 2026-10-04: one module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported; the code importing it passes it to `New` as an option, per the technical principles "Compose, do not entangle" and "Explicit over implicit". Still to decide: what a bare `New()` loads; whether a module's data is an `fs.FS` passed to the existing `WithDataFS` or a new option such as `WithData(svse.Data)`; and whether `WithoutShippedData` goes. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. +The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. The maintainer decided on 2026-10-04, under goals 2.3, 6.2, 7.3, 9 and 10.1: +- One module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported. +- A bare `New()` loads no data, and its error names the option to add. `WithoutShippedData` goes. +- A data module is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and several load at once. A module carrying functions loads through an option of its own; gaining one is a breaking change for that module's users. +- A module names, in a manifest, what it reads and the module providing each by default. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. +- Two sources defining one name fail to load, unless an option says the second replaces the first. The CLI's `--data-path` takes `--replace` to override a shipped category. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". +- The template language's builtins stay in the core. A module's function asks for the inputs it needs through one generic mechanism, and gets the randomness and the date the generator uses. The README strongly encourages functions with no side effects of their own. +- The CLI is a library call, so a custom CLI carrying other modules is a few lines of Go and works just like `fejkdata`. A README section shows it with example code. + +Split it into items before starting. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. @@ -333,10 +352,6 @@ Say the format is the table's own (`region's format "{name}"`), name the read to `probePath` skips `pathCheck.enter`, so `Fake("x.a")` under `"repeat":3` renders one draw. -### 22. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. - -17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. - ### 29. Let a path go from a selected row into the template beside its family: `geo.US.locality[1714000].address`. Require the path step to reach a sibling category. @@ -345,6 +360,10 @@ Require the path step to reach a sibling category. `datatype: boolean` over `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`. +### 22. Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default. + +17 of the 40 distinct ones shipped today sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost` provably reach nothing. + ### 46. Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files. `readTableOptions` and `loadDir` return on the first in Go's map order. @@ -385,10 +404,6 @@ Needs item 4; recount the switches then. Today ten, from `render` to `columnItem Nothing checks it, and a new country breaks it silently. -### 23. Give every other shipped category that could reach something real a path that never does. - -`phone` draws live PTS and NANP ranges — PTS's five fiction series are the Swedish inert set, sourced in [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. - ### 8. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. The greenfield architect found the fence pipeline last. Needs item 4, which shrinks the pipeline. @@ -401,14 +416,18 @@ The inherited architect's 3am trace crossed four files for one draw. The CLI counts brackets of its own, so a change to the selector grammar desyncs the default `--table` silently. -### 75. Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render. - -`readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables), no `go` example (Library) and no `text` block of error text, and compares no "Renders e.g." output, so goal 11.2 is not met. - ### 86. Read both `en_US.phone` formats' area codes from one category, as their exchange reads `en_US.phone-exchange`. Both phone formats carry the same 21-code `area` list, so an edit to one copy can miss the other. +### 23. Give every other shipped category that could reach something real a path that never does. + +`phone` draws live PTS and NANP ranges — PTS's five fiction series are the Swedish inert set, sourced in [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes. + +### 75. Run every README `sh`, `go` and `text` example as a test, and compare each "Renders" line to a seeded render. + +`readme_test.go` loads and renders each `json` block, but runs no `sh` example (CLI, Records, Table, Linked tables), no `go` example (Library) and no `text` block of error text, and compares no "Renders e.g." output, so goal 12.2 is not met. + ### 47. Refuse a category naming a hidden rows file, `"rows": ".x.tsv"`, as a rows file that is not there. `loadDir` indexes hidden TSVs and skips them in two places, so one loads where its comment says a hidden file is never data. -- 2.52.0 From eee7572360a03cd9e3f066635859d598bb271c29 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 22:10:23 +0200 Subject: [PATCH 494/966] Answer prose round 1 and product-owner round 1: modules not packages, goal 13 split, the decisions item 77 ends, and a manifest naming every default it reaches --- README.md | 25 ++++++++++++------------- docs/decisions.md | 4 ++++ todo.md | 17 ++++++++++------- 3 files changed, 26 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index e8f51b1..d1d3c57 100644 --- a/README.md +++ b/README.md @@ -889,8 +889,7 @@ Where the project is heading; the sections before Audience document what ships t 1. It covers the ten most-spoken languages and the Nordic countries. 2. Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates. - 3. A library gets only the data and functions it hands to `New`, and fejkdata's own - load the same way as anyone else's. + 3. A library gets only the data and the module functions it is handed. 3. **The code is easy to understand.** 1. You can go from a bug to the code behind it without asking anyone. 2. You can change one part without keeping the rest in your head. @@ -919,35 +918,35 @@ Where the project is heading; the sections before Audience document what ships t 1. A template that could render a value its own rules forbid, such as `NaN` or `Inf`, fails to load instead. 2. Data that reads something nobody provided fails to load, and the error names the - package that provides it by default. + module that provides it by default, or says that none does. 7. **One command gets you a value.** 1. Flags work like in other command-line tools, and can go anywhere on the line. 2. Your first template of your own needs no escaping and no options. 3. The CLI fills in what you leave out with a sane default: all the shipped data, the system's randomness and today's date. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** -9. **Everything is replaceable.** Data, functions, and the packages they depend on can - come from fejkdata or anyone else. - 1. A package names the packages it reads by default, and anything providing the same +9. **Everything is replaceable.** Data, functions and the modules they come in can come + from fejkdata or anyone else. + 1. A module names the modules it reads by default, and anything providing the same paths can stand in for one. 2. Loading two things under one name fails, in the library and the CLI alike, unless you say the second replaces the first. - 3. The template language's own functions stay in the core, so a template means the - same in every program. - 4. A function loaded from a package can ask for the input it needs, and gets the + 3. The builtins ship with fejkdata itself, so a template means the same in every + program. + 4. A function loaded from a module can ask for the input it needs, and gets the randomness and the date the generator uses. - 5. A CLI carrying your own packages is a few lines of Go, and works just like + 5. A CLI carrying your own modules is a few lines of Go, and works just like `fejkdata`. 10. **The same seed, version and data always give the same output.** - 1. A library call takes everything that can change its output from its caller: data, + 1. In the library, the caller supplies everything that can change the output: data, randomness and the date. 11. **It depends on nothing but Go's standard library.** 12. **The docs are a map of the template language.** 1. Every feature has its own heading. 2. Every example runs as a test and shows its output. 3. Each rule is stated in one place. -13. **It is fast enough that you never notice with the shipped data, and its cost grows - with the data a render reaches, not with all the data loaded.** +13. **It is fast enough that you never notice it with the shipped data.** + 1. A render's cost grows only with the data it reaches. 14. **Every fact comes from an official or open source,** such as a government register. 1. A table holds every row its source holds, and nothing added by hand. diff --git a/docs/decisions.md b/docs/decisions.md index 050e1ad..555ac48 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -134,6 +134,8 @@ set, against goal 13, until `todo.md` item 79 ends it. Valid while CI proves the shipped set whole and each category alone. +Works against goal 2.3 until `todo.md` item 77 ends it. + ## A bare reference draws each time; a reference path is held 2026-09-02, Lilleman auf Larv. @@ -231,6 +233,8 @@ references read. Accepted: overriding is the point of layering, the error names reference and the field, and the fix is the consumer's file carrying the fields the shipped tree reads. +Works against goal 9.2 until `todo.md` item 77 ends it. + ## The repeat cap bounds renders, not bytes 2026-09-02, Lilleman auf Larv. diff --git a/todo.md b/todo.md index 0e13efc..269e66d 100644 --- a/todo.md +++ b/todo.md @@ -54,6 +54,7 @@ | 12.2 | 0.45 | | 12.3 | 0.45 | | 13 | 0.40 | +| 13.1 | 0.40 | | 14 | 0.35 | | 14.1 | 0.35 | | 15 | 0.30 | @@ -146,14 +147,14 @@ ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. -The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. The maintainer decided on 2026-10-04, under goals 2.3, 6.2, 7.3, 9 and 10.1: +The CLI binary may grow to hundreds of MB. The maintainer decided on 2026-10-04, under goals 2.3, 6.2, 7.3, 9 and 10.1: - One module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported. -- A bare `New()` loads no data, and its error names the option to add. `WithoutShippedData` goes. -- A data module is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and several load at once. A module carrying functions loads through an option of its own; gaining one is a breaking change for that module's users. -- A module names, in a manifest, what it reads and the module providing each by default. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. -- Two sources defining one name fail to load, unless an option says the second replaces the first. The CLI's `--data-path` takes `--replace` to override a shipped category. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". -- The template language's builtins stay in the core. A module's function asks for the inputs it needs through one generic mechanism, and gets the randomness and the date the generator uses. The README strongly encourages functions with no side effects of their own. -- The CLI is a library call, so a custom CLI carrying other modules is a few lines of Go and works just like `fejkdata`. A README section shows it with example code. +- A bare `New()` loads no data and fails, naming the option to add. `WithoutShippedData` goes. +- A data module is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. +- A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. +- Two sources defining one name fail to load, unless an option says the second replaces the first. In the CLI, a flag standing on its own, benched with the data and hand fixture authors, does the same for a `--data-path`, so flags still go anywhere on the line. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". +- The builtins stay in fejkdata itself. A module's function asks for the inputs it needs through one generic mechanism, and gets the randomness and the date the generator uses. The README asks for functions with no side effects of their own. +- The CLI is a library call, so a custom CLI carrying other modules is a few lines of Go and works just like `fejkdata`. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. Split it into items before starting. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. @@ -264,6 +265,8 @@ Needs item 4. ### 79. Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach. +Needs item 77, which makes an override fail unless it says it replaces. + The decision "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, the shipped ones they read, and the shipped ones whose reads reach an overridden category, so the index would carry each category's reads. ### 87. Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold". -- 2.52.0 From 9240d90547865a9a15284e08e29523be8e0c96ad Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 22:30:15 +0200 Subject: [PATCH 495/966] Answer prose round 2 and product-owner round 2: flexible and extensible, the date in the seed promise, the custom-CLI builder, and the seed required beside item 77 --- README.md | 23 +++++++++++++---------- docs/decisions.md | 2 +- todo.md | 9 +++++---- 3 files changed, 19 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index d1d3c57..529a004 100644 --- a/README.md +++ b/README.md @@ -873,9 +873,10 @@ App developers writing tests and fixtures, in Go and at a shell: - a **validator-facing author**, who needs a value a real checker accepts - a **data author**, who writes categories under `--data-path` and meets every load error the fences raise +- a **custom-CLI builder**, who ships a CLI like `fejkdata` carrying modules of their own and a **contributor**, who reads [`todo.md`](todo.md), [`AGENTS.md`](AGENTS.md) and -the Development section below, and who ships a register the five above then draw from. +the Development section below, and who ships a register the six above then draw from. ## Goals @@ -925,19 +926,21 @@ Where the project is heading; the sections before Audience document what ships t 3. The CLI fills in what you leave out with a sane default: all the shipped data, the system's randomness and today's date. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** -9. **Everything is replaceable.** Data, functions and the modules they come in can come - from fejkdata or anyone else. - 1. A module names the modules it reads by default, and anything providing the same +9. **Flexible and extensible.** + 1. Data and functions come in modules, from fejkdata or anyone else, and fejkdata's + modules load as anyone else's do. + 2. A module names the modules it reads by default, and anything providing the same paths can stand in for one. - 2. Loading two things under one name fails, in the library and the CLI alike, unless - you say the second replaces the first. - 3. The builtins ship with fejkdata itself, so a template means the same in every + 3. Loading two things under one name fails, in the library and the CLI alike, unless + you say the second replaces the first, and then every read of that name reads + the second. + 4. The builtins ship with fejkdata itself, so a template means the same in every program. - 4. A function loaded from a module can ask for the input it needs, and gets the + 5. A function loaded from a module can ask for the input it needs, and gets the randomness and the date the generator uses. - 5. A CLI carrying your own modules is a few lines of Go, and works just like + 6. A CLI carrying your own modules is a few lines of Go, and works just like `fejkdata`. -10. **The same seed, version and data always give the same output.** +10. **The same seed, date, data and versions always give the same output.** 1. In the library, the caller supplies everything that can change the output: data, randomness and the date. 11. **It depends on nothing but Go's standard library.** diff --git a/docs/decisions.md b/docs/decisions.md index 555ac48..836732d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -233,7 +233,7 @@ references read. Accepted: overriding is the point of layering, the error names reference and the field, and the fix is the consumer's file carrying the fields the shipped tree reads. -Works against goal 9.2 until `todo.md` item 77 ends it. +Works against goal 9.3 until `todo.md` item 77 ends it. ## The repeat cap bounds renders, not bytes diff --git a/todo.md b/todo.md index 269e66d..81241b9 100644 --- a/todo.md +++ b/todo.md @@ -46,6 +46,7 @@ | 9.3 | 0.60 | | 9.4 | 0.60 | | 9.5 | 0.60 | +| 9.6 | 0.60 | | 10 | 0.55 | | 10.1 | 0.55 | | 11 | 0.50 | @@ -149,12 +150,12 @@ The CLI binary may grow to hundreds of MB. The maintainer decided on 2026-10-04, under goals 2.3, 6.2, 7.3, 9 and 10.1: - One module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported. -- A bare `New()` loads no data and fails, naming the option to add. `WithoutShippedData` goes. +- A bare `New()` loads no data and fails, naming the option to add, and `New` without `WithSeed` fails the same way; the CLI seeds from the system. `WithoutShippedData` goes. - A data module is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. - A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. -- Two sources defining one name fail to load, unless an option says the second replaces the first. In the CLI, a flag standing on its own, benched with the data and hand fixture authors, does the same for a `--data-path`, so flags still go anywhere on the line. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". +- Two sources defining one name fail to load, unless an option says the second replaces the first; every read of that name, a shipped module's included, then reads the second. In the CLI, a flag standing on its own, benched with the data and hand fixture authors, does the same for a `--data-path`, so flags still go anywhere on the line. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". - The builtins stay in fejkdata itself. A module's function asks for the inputs it needs through one generic mechanism, and gets the randomness and the date the generator uses. The README asks for functions with no side effects of their own. -- The CLI is a library call, so a custom CLI carrying other modules is a few lines of Go and works just like `fejkdata`. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. +- The CLI is a library call, and one exported list names every shipped module; `fejkdata`'s own `main` passes that list, so a custom CLI carrying other modules is the same few lines of Go, and no code exists only for custom CLIs. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. Split it into items before starting. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. @@ -280,7 +281,7 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. This item revises it: once the caller gives the date, an age is a bound, and the CLI's default date makes output change with the day it runs unless the flag is given. Do the first step before the rest: -- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Make the library require every input that can change, such as the date and the randomness, and let the CLI default each to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. Requiring the randomness changes `New`, which today seeds itself without `WithSeed`; split that into its own item when this one is picked. +- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Make the library require the date, as item 77 makes it require the randomness, and let the CLI default it to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. - Draw birthdates back past 100 years before the given date, so a `+` can appear. Read in 2026, the oldest shipped birthdate is 96 years old, so no draw carries a `+` until 2030. Skatteverket's test series reaches back to 1890, with birth numbers 980/981 before 1900. Draw no birthdate after the given date. - Weight the ages by the territory's population, so the share of `+` numbers follows its demography: [research-age-bands.md](docs/research/research-age-bands.md). - Let a data author's own category spell the separator too, through a builtin or a function a data module provides (item 77), and update the README's personnummer recipe and its "a day in 1930–2025" count. -- 2.52.0 From b2002bb51dd40050a0efb81514e10d74881b3f89 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:00:19 +0200 Subject: [PATCH 496/966] Answer prose round 2 and product-owner round 2: module explained at first use, the seed as item 89, modules in lockstep, and a second break of New for the date --- README.md | 12 +++++++----- docs/decisions.md | 2 +- todo.md | 17 ++++++++++++----- 3 files changed, 20 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 529a004..f600449 100644 --- a/README.md +++ b/README.md @@ -873,7 +873,8 @@ App developers writing tests and fixtures, in Go and at a shell: - a **validator-facing author**, who needs a value a real checker accepts - a **data author**, who writes categories under `--data-path` and meets every load error the fences raise -- a **custom-CLI builder**, who ships a CLI like `fejkdata` carrying modules of their own +- a **custom-CLI builder**, who ships a CLI like `fejkdata` with data and functions of + their own and a **contributor**, who reads [`todo.md`](todo.md), [`AGENTS.md`](AGENTS.md) and the Development section below, and who ships a register the six above then draw from. @@ -890,7 +891,8 @@ Where the project is heading; the sections before Audience document what ships t 1. It covers the ten most-spoken languages and the Nordic countries. 2. Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates. - 3. A library gets only the data and the module functions it is handed. + 3. Data and functions come in Go modules, and a library gets only the modules it is + handed. 3. **The code is easy to understand.** 1. You can go from a bug to the code behind it without asking anyone. 2. You can change one part without keeping the rest in your head. @@ -927,8 +929,8 @@ Where the project is heading; the sections before Audience document what ships t system's randomness and today's date. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** 9. **Flexible and extensible.** - 1. Data and functions come in modules, from fejkdata or anyone else, and fejkdata's - modules load as anyone else's do. + 1. Modules come from fejkdata or anyone else, and fejkdata's load as anyone else's + do. 2. A module names the modules it reads by default, and anything providing the same paths can stand in for one. 3. Loading two things under one name fails, in the library and the CLI alike, unless @@ -942,7 +944,7 @@ Where the project is heading; the sections before Audience document what ships t `fejkdata`. 10. **The same seed, date, data and versions always give the same output.** 1. In the library, the caller supplies everything that can change the output: data, - randomness and the date. + seed and date. 11. **It depends on nothing but Go's standard library.** 12. **The docs are a map of the template language.** 1. Every feature has its own heading. diff --git a/docs/decisions.md b/docs/decisions.md index 836732d..0606206 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -233,7 +233,7 @@ references read. Accepted: overriding is the point of layering, the error names reference and the field, and the fix is the consumer's file carrying the fields the shipped tree reads. -Works against goal 9.3 until `todo.md` item 77 ends it. +Its override needs no request, against goal 9.3, until `todo.md` item 77 ends it. ## The repeat cap bounds renders, not bytes diff --git a/todo.md b/todo.md index 81241b9..709214c 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 89` +`Next ID = 90` | Goal | W | |---|---| @@ -77,6 +77,7 @@ | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | +| 89 | 0.1.0 | | **Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.** | 3 | 3 | 6 | 7 | 10.1 | 16.0 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 13 | 12.9 | | 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 12, 12.3 | 11.9 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | @@ -150,12 +151,13 @@ The CLI binary may grow to hundreds of MB. The maintainer decided on 2026-10-04, under goals 2.3, 6.2, 7.3, 9 and 10.1: - One module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported. -- A bare `New()` loads no data and fails, naming the option to add, and `New` without `WithSeed` fails the same way; the CLI seeds from the system. `WithoutShippedData` goes. +- A bare `New()` loads no data and fails, naming the option to add. `WithoutShippedData` goes. - A data module is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. -- A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. +- A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. A manifest is optional: a `--data-path` folder without one is a module, and a read nothing provides fails naming the module that provides it by default. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. +- Every shipped module releases in lockstep, under one version number, so the README's one version still covers the library, the CLI and the data. CI cuts one tag per module per release, which revises the decision "The changelog heading is the one spelling of a release; CI cuts the tag". - Two sources defining one name fail to load, unless an option says the second replaces the first; every read of that name, a shipped module's included, then reads the second. In the CLI, a flag standing on its own, benched with the data and hand fixture authors, does the same for a `--data-path`, so flags still go anywhere on the line. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". - The builtins stay in fejkdata itself. A module's function asks for the inputs it needs through one generic mechanism, and gets the randomness and the date the generator uses. The README asks for functions with no side effects of their own. -- The CLI is a library call, and one exported list names every shipped module; `fejkdata`'s own `main` passes that list, so a custom CLI carrying other modules is the same few lines of Go, and no code exists only for custom CLIs. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. +- The CLI is a library call. One exported list names every shipped module, and `fejkdata`'s own `main` passes it to that call. The list lives where no library user's module graph pulls in every shipped module, and the architect places it. A custom CLI is the same few lines with other modules added, and no code exists only for custom CLIs. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. Split it into items before starting. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. @@ -264,6 +266,10 @@ Needs item 4. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. +### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system. + +Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0 beside item 77, so `New` breaks once before anyone depends on it. The Library section's options table shows `WithSeed` as required. + ### 79. Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach. Needs item 77, which makes an override fail unless it says it replaces. @@ -281,10 +287,11 @@ GoReleaser attaches the binaries to the release the tag workflow publishes, and ### 26. Spell `sv_SE.personnummer` with the `+` Skatteverket uses from the year the holder turns 100, judged by a date the caller gives. The format hard-codes `-`, and its earliest birthdate, 1930-01-01, makes the oldest draws invalid from 2030. A system holding Swedish personnummer often breaks on the `+`, so a fixture carrying one is a valuable test. Whether a number takes `+` depends on the date it is read, and the decision "No builtin reads the clock, so a date is bounded by days, never by an age" forbids a builtin from reading that date off the clock. This item revises it: once the caller gives the date, an age is a bound, and the CLI's default date makes output change with the day it runs unless the flag is given. Do the first step before the rest: -- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Make the library require the date, as item 77 makes it require the randomness, and let the CLI default it to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. +- Explore how a caller passes input data, such as the date a number is read on, to a render: in the template, as a plain option, or both. Make the library require the date, as item 89 makes it require the seed, and let the CLI default it to the system's when its flag is absent. Bench the spelling; `--now` reads like the clock. Shape the input so it can later narrow the draws to adults, children, 65 and over, or a mix, and so let a test ask for a centenarian; narrowing itself waits for its own item. - Draw birthdates back past 100 years before the given date, so a `+` can appear. Read in 2026, the oldest shipped birthdate is 96 years old, so no draw carries a `+` until 2030. Skatteverket's test series reaches back to 1890, with birth numbers 980/981 before 1900. Draw no birthdate after the given date. - Weight the ages by the territory's population, so the share of `+` numbers follows its demography: [research-age-bands.md](docs/research/research-age-bands.md). - Let a data author's own category spell the separator too, through a builtin or a function a data module provides (item 77), and update the README's personnummer recipe and its "a day in 1930–2025" count. +- Rewrite the README's `--seed` and `WithSeed` rows to goal 10's wording, and name the new required option in the changelog: `New` breaks a second time here, after item 89, as the maintainer accepted. - Research: `sv_SE.samordningsnummer` also hard-codes `-` and draws from 1930. Check whether Lag (2022:1697) om samordningsnummer gives it the same `+`; folkbokföringslagen 18 § states the rule for personnummer only. ### 81. Refuse at load every `calc` operand not proven numeric and every divisor not proven nonzero, in a string column too. -- 2.52.0 From f3d0959cae2ec5a8d637ab1481236c65b58919a4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:01:32 +0200 Subject: [PATCH 497/966] Answer prose round 3 and product-owner round 3: a module is a Go module or a data folder, a manifest names its fejkdata version, and the Versioning rewrite planned --- README.md | 6 +++--- docs/decisions.md | 2 +- todo.md | 7 ++++--- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index f600449..0b2a5a2 100644 --- a/README.md +++ b/README.md @@ -891,7 +891,7 @@ Where the project is heading; the sections before Audience document what ships t 1. It covers the ten most-spoken languages and the Nordic countries. 2. Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates. - 3. Data and functions come in Go modules, and a library gets only the modules it is + 3. The shipped data comes in Go modules, and a library gets only the modules it is handed. 3. **The code is easy to understand.** 1. You can go from a bug to the code behind it without asking anyone. @@ -929,8 +929,8 @@ Where the project is heading; the sections before Audience document what ships t system's randomness and today's date. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** 9. **Flexible and extensible.** - 1. Modules come from fejkdata or anyone else, and fejkdata's load as anyone else's - do. + 1. Data and functions come in modules, a Go module or a data folder, from fejkdata + or anyone else, and fejkdata's load as anyone else's do. 2. A module names the modules it reads by default, and anything providing the same paths can stand in for one. 3. Loading two things under one name fails, in the library and the CLI alike, unless diff --git a/docs/decisions.md b/docs/decisions.md index 0606206..9a5406e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -233,7 +233,7 @@ references read. Accepted: overriding is the point of layering, the error names reference and the field, and the fix is the consumer's file carrying the fields the shipped tree reads. -Its override needs no request, against goal 9.3, until `todo.md` item 77 ends it. +A `--data-path` category replaces a shipped one without saying so, against goal 9.3, until `todo.md` item 77 ends it. ## The repeat cap bounds renders, not bytes diff --git a/todo.md b/todo.md index 709214c..24aae26 100644 --- a/todo.md +++ b/todo.md @@ -153,11 +153,12 @@ The CLI binary may grow to hundreds of MB. The maintainer decided on 2026-10-04, - One module per locale, one per country's `geo/` tree, and one for `misc`. A module registers nothing when imported. - A bare `New()` loads no data and fails, naming the option to add. `WithoutShippedData` goes. - A data module is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. -- A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. A manifest is optional: a `--data-path` folder without one is a module, and a read nothing provides fails naming the module that provides it by default. Nothing loads a default on its own: loading fails naming it, and anything providing the same paths stands in. +- A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. A manifest is optional: a `--data-path` folder without one is a module. A read nothing provides fails, naming the module that provides that path by default or saying none does. Nothing loads a default on its own, and anything providing the same paths stands in. +- A module's manifest names the fejkdata version it was built for, and a mismatch fails in `New`, naming the `go get` line that aligns them. Each shipped module requires the core at its own version. - Every shipped module releases in lockstep, under one version number, so the README's one version still covers the library, the CLI and the data. CI cuts one tag per module per release, which revises the decision "The changelog heading is the one spelling of a release; CI cuts the tag". - Two sources defining one name fail to load, unless an option says the second replaces the first; every read of that name, a shipped module's included, then reads the second. In the CLI, a flag standing on its own, benched with the data and hand fixture authors, does the same for a `--data-path`, so flags still go anywhere on the line. This revises the decisions "A `--data-path` override rebinds every reference to the category it replaces" and "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`". - The builtins stay in fejkdata itself. A module's function asks for the inputs it needs through one generic mechanism, and gets the randomness and the date the generator uses. The README asks for functions with no side effects of their own. -- The CLI is a library call. One exported list names every shipped module, and `fejkdata`'s own `main` passes it to that call. The list lives where no library user's module graph pulls in every shipped module, and the architect places it. A custom CLI is the same few lines with other modules added, and no code exists only for custom CLIs. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. +- The CLI is a library call. One exported list names every shipped module, and `fejkdata`'s own `main` passes it to that call. The list lives where no library user's module graph pulls in every shipped module, and the architect places it. A custom CLI is the same few lines with other modules added, and no code exists only for custom CLIs. A README section shows it with example code, and the README's Library quick start shows the whole import block for one locale. The README's Versioning section gives each module's tag spelling, and says to pin every fejkdata module at one version. Split it into items before starting. Before the module API ships, a panel of the README's audience personas tries pulling in data with it. @@ -268,7 +269,7 @@ Needs item 4. ### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system. -Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0 beside item 77, so `New` breaks once before anyone depends on it. The Library section's options table shows `WithSeed` as required. +Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. This item ships in 0.1.0 beside item 77, so `New` breaks once before anyone depends on it. Mark `WithSeed` required in the Library section's options table. ### 79. Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach. -- 2.52.0 From d160de3ee149ba06857e8f54d8aa9bdf1d8d84c3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:41:25 +0200 Subject: [PATCH 498/966] Scrap the flat-package decision, plan item 90's folder restructure before the panel, and suspend scoring until item 5 --- AGENTS.md | 2 +- docs/decisions.md | 10 ++++------ todo.md | 19 +++++++------------ 3 files changed, 12 insertions(+), 19 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a1918b0..9f0ea48 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -74,7 +74,6 @@ In [docs/decisions.md](docs/decisions.md): - `Column` carries text, not a Go value - A unit takes the stem of what it is, and a file the stem of the units it holds - One name, one meaning -- The package stays flat - The performance gate asserts allocations, not wall-clock time - Rows live in a TSV, the shape in JSON - A selector is bracketed, and a dot inside it is literal @@ -120,3 +119,4 @@ In [docs/decisions.md](docs/decisions.md): - `List` advertises direct descents only - A path draws through its compiled steps - A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read +- comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship under it with no scoring run, and the nine-seat panel, item 5, measures them diff --git a/docs/decisions.md b/docs/decisions.md index 9a5406e..c30dd74 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -446,12 +446,6 @@ rest. Serves goal 3: names that tell the truth, and reaching the unit behind a s without asking a person. Decided 2026-09-25 by Lilleman; valid while goal 3 counts reading cost. -## The package stays flat - -2026-09-15, Lilleman auf Larv. - -Go ties a package to one directory, so folders would split the API into packages. - ## The performance gate asserts allocations, not wall-clock time 2026-09-03, Lilleman auf Larv. @@ -943,3 +937,7 @@ two draws of it. `todo.md` item 4 ends both: once every `{…}` draws afresh, no the render. Valid while names are read only inside the category binding them. + +## comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship under it with no scoring run, and the nine-seat panel, item 5, measures them + +2026-10-04, Lilleman auf Larv. Serves goal 3. Nine nine-seat panels held the score between 5.2 and 6.1, Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Each step of that restructure would score code the next step removes, so no scoring run or panel runs until items 2, 71, 3, 4 and 90 have merged, in that order. Item 5 then ends the suspension: it runs the nine-seat panel and larv-review's scoring run. Valid until item 5 runs. diff --git a/todo.md b/todo.md index 24aae26..f8680b7 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 90` +`Next ID = 91` | Goal | W | |---|---| @@ -78,6 +78,7 @@ | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 89 | 0.1.0 | | **Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.** | 3 | 3 | 6 | 7 | 10.1 | 16.0 | +| 90 | 0.1.0 | | **Restructure the code into folders named for what they hold: the public API stays in the root package, and the implementation moves into `internal/` packages.** | 6 | 9 | 3 | 9 | 3.2, 3.4 | 13.9 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 13 | 12.9 | | 87 | 0.1.0 | | **Split the README's last Records paragraph into one paragraph per rule, name the two shapes "either shape" means, and define or replace "a field hold".** | 1 | 2 | 4 | 6 | 12, 12.3 | 11.9 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | @@ -117,8 +118,6 @@ | 11 | 0.3.0 | | **Test that every `geo` tree holds the five table names and the `address` columns that port across countries.** | 1 | 2 | 3 | 5 | 4 | 13.0 | | 35 | 0.3.0 | | **Add `{base64url(n)}` for JWT shapes.** | 1 | 2 | 4 | 5 | 8 | 13.0 | | 21 | 0.3.0 | | **List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list of what each locale carries.** | 1 | 1 | 4 | 6 | 12 | 12.2 | -| 8 | 0.3.0 | | **Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | -| 9 | 0.3.0 | | **Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`.** | 1 | 2 | 2 | 5 | 3.1 | 11.5 | | 30 | 0.3.0 | | **Draw `email.local` from `username`, so the two share one handle list.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 40 | 0.3.0 | | **Parse `defaultTable`'s path with the library's grammar.** | 2 | 2 | 3 | 5 | 5.7 | 11.5 | | 86 | 0.3.0 | | **Read both `en_US.phone` formats' area codes from one category, as their exchange reads `en_US.phone-exchange`.** | 1 | 2 | 2 | 5 | 3.2 | 11.5 | @@ -261,7 +260,7 @@ Nothing has shipped, so "no longer paths", "where it used to fail" and "where it ### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. -Needs item 4. +Needs items 2, 71, 3, 4 and 90, in that order. The decision "comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship under it with no scoring run, and the nine-seat panel, item 5, measures them" holds every scoring run and panel until then. ### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. @@ -271,6 +270,10 @@ Needs item 4. Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. This item ships in 0.1.0 beside item 77, so `New` breaks once before anyone depends on it. Mark `WithSeed` required in the Library section's options table. +### 90. Restructure the code into folders named for what they hold: the public API stays in the root package, and the implementation moves into `internal/` packages. + +Needs item 4, and comes before item 5's panel. The maintainer scrapped the decision "The package stays flat" on 2026-10-04: its premise, that folders split the API into packages, does not hold for `internal/` packages, which no other module can import. Each package is a folder named for what it holds, and what one package uses from another is exported on purpose. It takes in two file moves the panel asked for: `binding`, `bind` and `checkNodeFences` out of `data.go`, and `table.route`, `selector`, `step` and `drawStep` out of `path.go`. The architect proposes the folders and their names, and the maintainer approves them before code moves. + ### 79. Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach. Needs item 77, which makes an override fail unless it says it replaces. @@ -416,14 +419,6 @@ Needs item 4; recount the switches then. Today ten, from `render` to `columnItem Nothing checks it, and a new country breaks it silently. -### 8. Move `binding`, `bind` and `checkNodeFences` out of `data.go` into a file of their own. - -The greenfield architect found the fence pipeline last. Needs item 4, which shrinks the pipeline. - -### 9. Move `table.route`, `selector`, `step` and `drawStep` from `path.go` to `table.go`. - -The inherited architect's 3am trace crossed four files for one draw. - ### 40. Parse `defaultTable`'s path with the library's grammar. The CLI counts brackets of its own, so a change to the selector grammar desyncs the default `--table` silently. -- 2.52.0 From e14c9d0f4c64e422e671d1c4b7ceffb162d2c6f1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:43:05 +0200 Subject: [PATCH 499/966] Answer prose round 1: item 5 scores the restructure, which is the round the comprehension rule lets merge, and item 90 says what internal/ buys --- AGENTS.md | 2 +- docs/decisions.md | 4 ++-- todo.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9f0ea48..a64f3c8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -119,4 +119,4 @@ In [docs/decisions.md](docs/decisions.md): - `List` advertises direct descents only - A path draws through its compiled steps - A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read -- comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship under it with no scoring run, and the nine-seat panel, item 5, measures them +- comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them diff --git a/docs/decisions.md b/docs/decisions.md index c30dd74..f9d6468 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -938,6 +938,6 @@ the render. Valid while names are read only inside the category binding them. -## comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship under it with no scoring run, and the nine-seat panel, item 5, measures them +## comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them -2026-10-04, Lilleman auf Larv. Serves goal 3. Nine nine-seat panels held the score between 5.2 and 6.1, Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Each step of that restructure would score code the next step removes, so no scoring run or panel runs until items 2, 71, 3, 4 and 90 have merged, in that order. Item 5 then ends the suspension: it runs the nine-seat panel and larv-review's scoring run. Valid until item 5 runs. +2026-10-04, Lilleman auf Larv. Serves goal 3. The nine-seat panels filed in `todo.md` from 2026-09-22 to 2026-09-30, in its git history, held the score between 5.2 and 6.1, with Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Each of items 2, 71, 3, 4 and 90 rewrites code that a later one removes, so a scoring run between them would score code about to go. They are the round `AGENTS.md`'s comprehension rule lets merge below the floor, a restructure the maintainer approved. Item 5 ends the suspension: it runs the nine-seat panel and larv-review's scoring run, which writes this entry's first scores. Valid until item 5 runs. diff --git a/todo.md b/todo.md index f8680b7..f3f6acb 100644 --- a/todo.md +++ b/todo.md @@ -260,7 +260,7 @@ Nothing has shipped, so "no longer paths", "where it used to fail" and "where it ### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. -Needs items 2, 71, 3, 4 and 90, in that order. The decision "comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship under it with no scoring run, and the nine-seat panel, item 5, measures them" holds every scoring run and panel until then. +Needs items 2, 71, 3, 4 and 90. The decision "comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them" suspends every scoring run and panel until then. ### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. @@ -272,7 +272,7 @@ Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` see ### 90. Restructure the code into folders named for what they hold: the public API stays in the root package, and the implementation moves into `internal/` packages. -Needs item 4, and comes before item 5's panel. The maintainer scrapped the decision "The package stays flat" on 2026-10-04: its premise, that folders split the API into packages, does not hold for `internal/` packages, which no other module can import. Each package is a folder named for what it holds, and what one package uses from another is exported on purpose. It takes in two file moves the panel asked for: `binding`, `bind` and `checkNodeFences` out of `data.go`, and `table.route`, `selector`, `step` and `drawStep` out of `path.go`. The architect proposes the folders and their names, and the maintainer approves them before code moves. +Needs item 4, and comes before item 5's panel. Packages under `internal/` leave the API in one package: no other module can import them. A package exports only what another package uses. It takes in two file moves the nine-seat panel asked for: `binding`, `bind` and `checkNodeFences` out of `data.go`, and `table.route`, `selector`, `step` and `drawStep` out of `path.go`. The systems-architect proposes the folders and their names, and the maintainer approves them before code moves. ### 79. Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach. -- 2.52.0 From bad5beb42ba581811e95a515e1e2fdc0dc99b210 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:44:12 +0200 Subject: [PATCH 500/966] Answer prose round 2: items 4 and 90 remove what the earlier items touch, and the scoring run writes the first scores --- docs/decisions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index f9d6468..b3a7bd1 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -940,4 +940,4 @@ Valid while names are read only inside the category binding them. ## comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them -2026-10-04, Lilleman auf Larv. Serves goal 3. The nine-seat panels filed in `todo.md` from 2026-09-22 to 2026-09-30, in its git history, held the score between 5.2 and 6.1, with Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Each of items 2, 71, 3, 4 and 90 rewrites code that a later one removes, so a scoring run between them would score code about to go. They are the round `AGENTS.md`'s comprehension rule lets merge below the floor, a restructure the maintainer approved. Item 5 ends the suspension: it runs the nine-seat panel and larv-review's scoring run, which writes this entry's first scores. Valid until item 5 runs. +2026-10-04, Lilleman auf Larv. Serves goal 3. The nine-seat panels in `todo.md`'s git history, 2026-09-22 to 2026-09-30, held the score between 5.2 and 6.1, with Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Items 4 and 90 delete or move what the earlier items touch, so a scoring run before item 90 merges would score code about to go. They are the round `AGENTS.md`'s comprehension rule lets merge below the floor, a restructure the maintainer approved. No scoring run or panel runs until item 5, which runs the nine-seat panel and larv-review's scoring run. The scoring run writes this entry's first scores. Valid until item 5 runs. -- 2.52.0 From 5707c697d854c17bcb1623ed03bb19e42cefc686 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:44:37 +0200 Subject: [PATCH 501/966] Answer prose round 3: name the five items the round lets merge --- docs/decisions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index b3a7bd1..d81b87e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -940,4 +940,4 @@ Valid while names are read only inside the category binding them. ## comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them -2026-10-04, Lilleman auf Larv. Serves goal 3. The nine-seat panels in `todo.md`'s git history, 2026-09-22 to 2026-09-30, held the score between 5.2 and 6.1, with Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Items 4 and 90 delete or move what the earlier items touch, so a scoring run before item 90 merges would score code about to go. They are the round `AGENTS.md`'s comprehension rule lets merge below the floor, a restructure the maintainer approved. No scoring run or panel runs until item 5, which runs the nine-seat panel and larv-review's scoring run. The scoring run writes this entry's first scores. Valid until item 5 runs. +2026-10-04, Lilleman auf Larv. Serves goal 3. The nine-seat panels in `todo.md`'s git history, 2026-09-22 to 2026-09-30, held the score between 5.2 and 6.1, with Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Items 4 and 90 delete or move what the earlier items touch, so a scoring run before item 90 merges would score code about to go. Items 2, 71, 3, 4 and 90 are the round `AGENTS.md`'s comprehension rule lets merge below the floor, a restructure the maintainer approved. No scoring run or panel runs until item 5, which runs the nine-seat panel and larv-review's scoring run. The scoring run writes this entry's first scores. Valid until item 5 runs. -- 2.52.0 From 07c66c57fc5572ce406968564b21cf472549be89 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:51:10 +0200 Subject: [PATCH 502/966] Test that .. steps up from a row to the row it links to --- hold_test.go | 4 +-- table_test.go | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/hold_test.go b/hold_test.go index 8bc7e23..3dca823 100644 --- a/hold_test.go +++ b/hold_test.go @@ -689,12 +689,12 @@ func TestDeepPathsUnderOneHeadStayIndependentWhereTheyDiverge(t *testing.T) { } func TestEmptyPathSegmentIsRejected(t *testing.T) { - // "{a.}" and "{a..b}" are unfinished paths, and the segment naming nothing is + // "{a.}" and "{a...b}" are unfinished paths, and the segment naming nothing is // reported as that rather than as a missing field. An empty name is rejected // where it is authored, so no data can make these resolve. rejected := map[string]string{ "trailing dot": `{"format":"[{a.}]","a":"x"}`, - "double dot": `{"format":"[{a..b}]","a":"x"}`, + "triple dot": `{"format":"[{a...b}]","a":"x"}`, } for name, file := range rejected { _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file}))) diff --git a/table_test.go b/table_test.go index cc3e23c..2e852bd 100644 --- a/table_test.go +++ b/table_test.go @@ -235,6 +235,102 @@ func TestTableDescendsToALinkedTable(t *testing.T) { } } +func TestTableStepsUpToTheRowItLinksTo(t *testing.T) { + files := with(geo(), map[string]string{ + "drawn.json": `"{/locality.code}|{/locality..municipality.code}|{/locality..municipality..region.code}"`, + "named.json": `"{/locality as l}{l.code}|{l..municipality.code}|{l..municipality..region.code}"`, + "down.json": `"{/locality.code}|{/locality..municipality.locality.code}"`, + "named-down.json": `"{/locality as l}{l.code}|{l..municipality.locality.code}"`, + "record.json": `{"format":"","l":"{/locality.code}","m":"{/locality..municipality.code}"}`, + }) + f := newGenerator(t, writeFiles(t, files), WithSeed(4)) + for path, want := range map[string]string{ + "locality[L4]..municipality.name": "Lund", + "locality[L4]..municipality": "Lund", + "locality[L4]..municipality..region.name": "Skåne län", + "municipality[1281]..region.code": "12", + "region[12].municipality[1281].locality[L4]..municipality.code": "1281", + "locality[L4].municipality": "1281", + } { + if got := fake(t, f, path); got != want { + t.Errorf("Fake(%q) = %q, want %q", path, got, want) + } + } + if got := fakeTemplate(t, f, "{/locality[L4]..municipality.name}"); got != "Lund" { + t.Errorf("{/locality[L4]..municipality.name} = %q, want Lund", got) + } + afresh := map[string]bool{} + for i := 0; i < 200; i++ { + for _, path := range []string{"drawn", "named"} { + parts := strings.Split(fake(t, f, path), "|") + if municipalityOf[parts[0]] != parts[1] || regionOf[parts[1]] != parts[2] { + t.Fatalf("%s = %v, want the locality's own municipality and region", path, parts) + } + } + if l := fake(t, f, "locality[L4]..municipality.locality.code"); l != "L4" && l != "L7" { + t.Fatalf("locality[L4]..municipality.locality.code = %q, outside Lund", l) + } else { + afresh[l] = true + } + if m := fake(t, f, "locality[L4]..municipality..region.municipality.code"); regionOf[m] != "12" { + t.Fatalf("locality[L4]..municipality..region.municipality.code = %q, outside 12", m) + } + for _, path := range []string{"down", "named-down"} { + parts := strings.Split(fake(t, f, path), "|") + if municipalityOf[parts[0]] != municipalityOf[parts[1]] { + t.Fatalf("%s = %v, want a locality of the first one's municipality", path, parts) + } + } + down, err := f.FakeRecord("locality[L4]..municipality.locality") + if err != nil { + t.Fatal(err) + } + if c := down.Columns(); c[0].Value != "L4" && c[0].Value != "L7" { + t.Fatalf("record locality[L4]..municipality.locality = %v, outside Lund", c) + } else { + afresh["record "+c[0].Value] = true + } + r, err := f.FakeRecord("record") + if err != nil { + t.Fatal(err) + } + if c := r.Columns(); municipalityOf[c[0].Value] != c[1].Value { + t.Fatalf("record = %v, want the locality's own municipality", c) + } + } + if !afresh["L4"] || !afresh["L7"] || !afresh["record L4"] || !afresh["record L7"] { + t.Fatalf("locality[L4]..municipality.locality.code drew only %v in 200 renders, want a locality of Lund drawn afresh", afresh) + } + for _, p := range f.List() { + if strings.Contains(p, "..") { + t.Fatalf("List() advertises %q; it lists direct descents only", p) + } + } +} + +func TestTableStepUpRefusals(t *testing.T) { + for path, want := range map[string]string{ + "locality[L4]..region.name": "..municipality", + "region[12]..municipality.name": "no parent", + "locality[L4]..": "..municipality", + "locality[L4]..municipality[0180]": "selects its rows before", + "locality[L4].name..municipality": "column", + "x..y": "not a table", + } { + f := newGenerator(t, writeFiles(t, with(geo(), map[string]string{"x.json": `{"format":"{y}","y":"a"}`}))) + if _, err := f.Fake(path); err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("Fake(%q) = %v, want an error mentioning %s", path, err, want) + } + _, err := New(WithoutShippedData(), WithDataPath(writeFiles(t, with(geo(), map[string]string{ + "x.json": `{"format":"{y}","y":"a"}`, + "t.json": strconv.Quote("{/" + path + "}"), + })))) + if err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("{/%s}: New = %v, want an error mentioning %s", path, err, want) + } + } +} + func TestLinkedTablesDrawConsistently(t *testing.T) { spellings := map[string]string{ "leaf first": `{"format":"{l}|{m}|{r}","l":"{/locality.code}","m":"{/municipality.code}","r":"{/region.code}"}`, -- 2.52.0 From 8b76d51f2ce06b02842c0f0fc95c08f7ecdb778e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:56:45 +0200 Subject: [PATCH 503/966] Step up from a row to the row it links to with .. --- format.go | 2 +- path.go | 79 ++++++++++++++++++++++++++++++++++++++++++++++--------- pick.go | 22 +++++++++------- pins.go | 11 ++++++++ reads.go | 20 +++++++++++--- render.go | 11 +++++--- 6 files changed, 115 insertions(+), 30 deletions(-) diff --git a/format.go b/format.go index c1459c1..4013b37 100644 --- a/format.go +++ b/format.go @@ -435,7 +435,7 @@ func pathArm(name, head, writtenHead string, segs []string) arm { return arm{spelling: name, head: head, writtenHead: writtenHead, tail: segs, levels: levels, path: head + "." + strings.Join(segs, ".")} } -// checkSegments rejects an unfinished path: "{a.}" and "{a..b}" each have a +// checkSegments rejects an unfinished path: "{a.}" and "{a...b}" each have a // segment naming nothing. A field really named "" would otherwise make them // resolve, so a typo would read as a path that worked. func checkSegments(a arm) error { diff --git a/path.go b/path.go index bcb8747..399ec84 100644 --- a/path.go +++ b/path.go @@ -6,8 +6,8 @@ import ( ) // splitPath splits a dotted path into segments, a selector — [key or name] after a -// table's name — becoming a segment of its own, brackets kept. A dot inside a -// selector is part of the key or name. +// table's name — becoming a segment of its own, brackets kept, and so does each "..". A +// dot inside a selector is part of the key or name. func splitPath(path string) ([]string, error) { segs := make([]string, 0, strings.Count(path, ".")+2*strings.Count(path, "[")+1) start, open := 0, -1 @@ -24,11 +24,13 @@ func splitPath(path string) ([]string, error) { return nil, err } segs = append(segs, path[start:i+1]) - start, open = i+2, -1 - i++ + open = -1 + segs, i = stepUpAt(segs, path, i+1) + start = i + 1 case '.': if open < 0 { segs = append(segs, path[start:i]) + segs, i = stepUpAt(segs, path, i) start = i + 1 } } @@ -36,12 +38,21 @@ func splitPath(path string) ([]string, error) { if open >= 0 { return nil, fmt.Errorf(`%q opens a selector with "[" and never closes it with "]"`, path) } - if start <= len(path) { + if start < len(path) || start == len(path) && (len(segs) == 0 || segs[len(segs)-1] != "..") { segs = append(segs, path[start:]) } return segs, nil } +// stepUpAt appends a ".." segment where the dot at i is the first of two, returning the +// index of the last dot it consumed. +func stepUpAt(segs []string, path string, i int) ([]string, int) { + if i+1 < len(path) && path[i+1] == '.' { + return append(segs, ".."), i + 1 + } + return segs, i +} + // checkOpen refuses a "[" at i that opens no selector: one inside a selector, or one // following no name. func checkOpen(path string, i, start, open int) error { @@ -139,8 +150,14 @@ func nameSegments(segs []string) []string { // stepInto is what seg names under n: a folder's entry or a template's field. func stepInto(n node, seg string) (node, error) { - if isSelector(seg) { + switch { + case isSelector(seg): return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(seg)) + case seg == "..": + if c, ok := n.(*tableColumn); ok { + return nil, fmt.Errorf(`".." steps up from a table's row, and %q is a column`, c.t.header[c.i]) + } + return nil, fmt.Errorf(`".." steps up from a table's row to the row it links to, and what it follows is not a table`) } switch n := n.(type) { case *folder: @@ -169,6 +186,7 @@ type tableRoute struct { next node rest []string descends bool + up bool } // route is how tail passes t. descended means the previous route stepped into t @@ -180,6 +198,12 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { if err != nil { return tableRoute{}, err } + if len(tail) > 0 && tail[0] == ".." { + if err := t.stepUp(tail[1:]); err != nil { + return tableRoute{}, err + } + return tableRoute{sel: sel, draw: sel == "", next: t.parentT, rest: tail[2:], descends: true, up: true}, nil + } column, child, err := t.step(tail) if err != nil { return tableRoute{}, err @@ -200,6 +224,20 @@ func (t *table) route(tail []string, descended bool) (tableRoute, error) { return r, nil } +// stepUp proves a ".." after a row of t can step up: t has a parent, rest names it, and no +// selector follows, since a row selected below the parent row could lie outside it. +func (t *table) stepUp(rest []string) error { + switch { + case t.parentT == nil: + return fmt.Errorf(`%s has no parent table for ".." to step up to`, t.segment) + case len(rest) == 0 || rest[0] != t.parentT.segment: + return fmt.Errorf(`".." steps up from %s to its parent table, so name that next: ..%s`, t.segment, t.parentT.segment) + case hasSelector(rest): + return fmt.Errorf(`a path selects its rows before a "..", since a row selected after it could lie outside the row it steps up to; start a path at the table to select from it`) + } + return nil +} + // pathStep is one step of a compiled path, taken at the node the steps before it // reached, a choice there resolved first; at indexes the tail where it is taken, and // only a table's step sits past the tail's end. A step holds no node, so a @@ -220,6 +258,7 @@ const ( stepRow // land on the table's row node stepColumn stepChild + stepParent // up to the table's parent, at the row its own row links to ) // pathCheck proves a path resolves whichever way the draws go: every variant of a @@ -297,7 +336,10 @@ func routeSteps(steps []pathStep, pins *pinSet, t *table, r tableRoute, at int) } switch next := r.next.(type) { case *table: - if next != t { + switch { + case r.up: + steps = append(steps, pathStep{kind: stepParent, at: at}) + case next != t: steps = append(steps, pathStep{kind: stepChild, at: at, name: next.segment}) } case *tableRow: @@ -380,8 +422,10 @@ func probePath(n node, tail []string, steps []pathStep) ([]pathStep, error) { // drawSteps draws the rows and variants a path's steps read from n, pinning the rows in pins; // pins is nil for a sibling path, which never crosses a table, since a table is only a category. // For a memoized read, memo keeps the variant drawn at each of levels, so paths sharing a prefix -// share it. -func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo *drawMemo, levels []string) node { +// share it. It returns the leaf and the pins its row is in: a step down after a step up draws +// afresh, into pins of its own. +func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo *drawMemo, levels []string) (node, *pinSet) { + var climbed *table for _, st := range steps { if c, ok := n.(*choice); ok { n = drawVariant(s, c, memo, levels, st.at) @@ -397,9 +441,15 @@ func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo * if !ok { panic(internalError("step %d: a table's step of kind %d on %T", st.at, st.kind, n)) } + switch { + case st.kind == stepParent: + climbed = t.parentT + case st.kind == stepChild && climbed != nil: + pins, climbed = pins.above(climbed), nil + } n = t.drawStep(s, st, pins) } - return n + return n, pins } func (t *table) drawStep(s *generatorState, st pathStep, pins *pinSet) node { @@ -418,6 +468,8 @@ func (t *table) drawStep(s *generatorState, st pathStep, pins *pinSet) node { return t.formatTemplate.fields[st.name] case stepChild: return t.descendant(st.name) + case stepParent: + return t.parentT } panic(internalError("drawStep has no case for step kind %d", st.kind)) } @@ -487,11 +539,12 @@ func compilePath(n node, tail []string) pathCheck { return w } -// joinSegments spells segments as a path: a selector attaches to the name before it. +// joinSegments spells segments as a path: a selector attaches to the name before it, and +// so do a ".." and the name after it. func joinSegments(segs []string) string { var b strings.Builder - for _, s := range segs { - if b.Len() > 0 && !isSelector(s) { + for i, s := range segs { + if b.Len() > 0 && !isSelector(s) && s != ".." && segs[i-1] != ".." { b.WriteByte('.') } b.WriteString(s) diff --git a/pick.go b/pick.go index 95ff8e7..7fbe12e 100644 --- a/pick.go +++ b/pick.go @@ -130,9 +130,9 @@ func readName(s *generatorState, sc renderScope, a arm) readValue { if r, done := p.memo.value[a.path]; done { return r } - leaf := p.draw(s, a.named.head, a.steps, a.levels, a.path) + leaf, pins := p.draw(s, a.named.head, a.steps, a.levels, a.path) sc, mark := sc.enter(leaf, &p.memo) - r := p.renderAt(s, leaf, a.path, sc) + r := p.renderAt(s, leaf, pins, a.path, sc) sc.draws.popFrames(mark) return r } @@ -162,7 +162,8 @@ func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue for i, l := range a.levels { levels[i] = underKey(sc.pickKey, l) } - return p.renderAt(s, p.draw(s, t.head(a.head), a.steps, levels, key), key, sc) + leaf, pins := p.draw(s, t.head(a.head), a.steps, levels, key) + return p.renderAt(s, leaf, pins, key, sc) } // underKey is the key of path under the pick key prefix. It never returns prefix itself, which a @@ -174,18 +175,19 @@ func underKey(prefix, path string) string { return prefix + "." + path } -// draw draws the path key names under p, its variant at key kept too. -func (p *namedPick) draw(s *generatorState, head node, steps []pathStep, levels []string, key string) node { - leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) +// draw draws the path key names under p, its variant at key kept too, returning the leaf and the +// pins its row is in. +func (p *namedPick) draw(s *generatorState, head node, steps []pathStep, levels []string, key string) (node, *pinSet) { + leaf, pins := drawSteps(s, head, steps, &p.pins, &p.memo, levels) if c, isChoice := leaf.(*choice); isChoice { leaf = p.memo.variantOf(s, c, key) } - return leaf + return leaf, pins } -// renderAt renders leaf, drawn at key, as part of p, and keeps what it rendered. -func (p *namedPick) renderAt(s *generatorState, leaf node, key string, sc renderScope) readValue { - sc = sc.at(leaf, &p.pins) +// renderAt renders leaf, drawn at key into pins, as part of p, and keeps what it rendered. +func (p *namedPick) renderAt(s *generatorState, leaf node, pins *pinSet, key string, sc renderScope) readValue { + sc = sc.at(leaf, pins) sc.pick, sc.pickKey = p, key r := renderLeaf(s, leaf, sc) if p.memo.value == nil { diff --git a/pins.go b/pins.go index c866373..c7f4fd3 100644 --- a/pins.go +++ b/pins.go @@ -154,6 +154,17 @@ func (p *pinSet) inside(t *table, rows []int) []int { return rows } +// above is a pin set holding the rows p pins of t and its ancestors, and none below them. +func (p *pinSet) above(t *table) *pinSet { + q := new(pinSet) + for ; t != nil; t = t.parentT { + if r, ok := p.pinned(t); ok { + q.add(t, r) + } + } + return q +} + // clone is a copy of p that pins apart from it; a plain copy shares the spill map. func (p pinSet) clone() pinSet { p.spill = maps.Clone(p.spill) diff --git a/reads.go b/reads.go index e8bf049..0c050be 100644 --- a/reads.go +++ b/reads.go @@ -144,7 +144,7 @@ func pinBelow(reads []pathRead, tr *tableRead) []pathRead { kept := reads[:0:0] for _, r := range reads { pins := r.branches.pins.clone() - if err := tr.replay(&pins); err != nil { + if err := tr.replayBelow(&pins); err != nil { continue } if r.tr != nil && r.clash == nil { @@ -230,6 +230,7 @@ func (r readRoute) spelled(ref string) string { type tableRead struct { headTable *table pins pinSet + leafPins *pinSet // the rows the leaf renders in: pins, short of the rows below a step up drawn map[*table]bool sels []tableSel landsRow bool @@ -250,9 +251,17 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { return nil } r := &tableRead{headTable: t} + r.leafPins = &r.pins _, r.landsRow = leaf.(*tableRow) + climbed := false for _, st := range a.steps { + if climbed && st.kind != stepChild && st.kind != stepParent { + continue + } switch st.kind { + case stepParent: + t, climbed = t.parentT, true + r.leafPins = r.leafPins.above(t) case stepSelect: r.pins.pin(t, st.row) r.sels = append(r.sels, tableSel{t, joinSegments(append([]string{a.writtenHead}, a.tail[:st.at+1]...))}) @@ -294,9 +303,14 @@ func (r *tableRead) drawnOf(pins *pinSet) *table { } // replay pins the read's rows into d, where they agree with the rows pinned before. -func (r *tableRead) replay(d *pinSet) error { +func (r *tableRead) replay(d *pinSet) error { return replayPins(&r.pins, d) } + +// replayBelow pins into d the rows the read's leaf renders in. +func (r *tableRead) replayBelow(d *pinSet) error { return replayPins(r.leafPins, d) } + +func replayPins(from, d *pinSet) error { var err error - r.pins.each(func(t *table, row int) { + from.each(func(t *table, row int) { if err == nil { err = d.pinRow(t, row) } diff --git a/render.go b/render.go index d08dc04..889617c 100644 --- a/render.go +++ b/render.go @@ -48,7 +48,12 @@ func descend(s *generatorState, root node, segments []string, sc renderScope) (n if err != nil { return nil, err } - return drawSteps(s, root, steps, &sc.groupDraws().pins, nil, nil), nil + group := sc.groupDraws() + n, pins := drawSteps(s, root, steps, &group.pins, nil, nil) + // A caller's path is its render's first draw, so the rows a step down after a step up drew + // are all the render pins. + group.pins = *pins + return n, nil } // renderOnce renders n as one render, over draws of its own. @@ -209,10 +214,10 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r if r, done := memo.value[a.path]; done { return r } - leaf := drawSteps(s, t.head(a.head), a.steps, pins, memo, a.levels) + leaf, leafPins := drawSteps(s, t.head(a.head), a.steps, pins, memo, a.levels) mark := -1 if pins != nil { - sc, mark = sc.at(leaf, pins).enter(leaf, memo) + sc, mark = sc.at(leaf, leafPins).enter(leaf, memo) } r := renderLeaf(s, leaf, sc) if mark >= 0 { -- 2.52.0 From 4b5506d07b694b66ed7fa756e6c705b0b1ed3768 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sun, 4 Oct 2026 23:58:50 +0200 Subject: [PATCH 504/966] Document .., drop the step-up decision item 2 replaces, and close item 2 --- AGENTS.md | 3 +-- CHANGELOG.md | 4 ++++ README.md | 20 ++++++++++++++++++++ docs/decisions.md | 32 +------------------------------- todo.md | 15 +++------------ 5 files changed, 29 insertions(+), 45 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a64f3c8..f9974c9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -80,7 +80,6 @@ In [docs/decisions.md](docs/decisions.md): - `parent` names the link column and the table alike - After a row, a path names a column or a linked table - A selected row is fixed, not drawn, and an unnamed read beside it is refused -- A link column after a row steps up to the parent row, and a path may not end on it - A table selects by one key column, a code, never a free-form name - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields @@ -119,4 +118,4 @@ In [docs/decisions.md](docs/decisions.md): - `List` advertises direct descents only - A path draws through its compiled steps - A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read -- comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them +- comprehension floor: every dimension and the overall at 7.0 or above; items 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them diff --git a/CHANGELOG.md b/CHANGELOG.md index 7def0fd..b888b2d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,10 @@ replacement, and each removed path, column or flag. ### Added +- Step up: `..` after a row steps up to the row its link column names, so + `{/city[Oslo]..country.name}` renders `Norway`, and `.` after it steps down again, + drawing afresh. `New` refuses a `..` naming anything but the parent table, and a + selector after a `..`. - Names: `{/misc.territory as t}` binds one row of `misc.territory` to `t` and prints nothing; `{t}`, `{t.capital}` and `{lowercase(t.name)}` read that one row. A name lives in the category binding it, and a `repeat` binding it picks again on every iteration. diff --git a/README.md b/README.md index 0b2a5a2..cd53345 100644 --- a/README.md +++ b/README.md @@ -572,6 +572,26 @@ reference renders one row of a table, so reads in cells of different rows of it, different rows of their ancestors, never render together, whether `[key]` selects the row or the reference draws it. +### Step up + +`..` after a row steps up to the row its link column names, and the parent table's name +follows it, as a folder's does in a filesystem path: + +```sh +fejkdata -d ./mydata 'city[Oslo]..country.name' # Norway +fejkdata -d ./mydata 'city..country' # a city drawn, then its country: Sweden (SE) +fejkdata -d ./mydata 'city[Göteborg]..country.city' # Stockholm or Göteborg, drawn afresh +fejkdata -d ./mydata '{/city as c}{c}, {c..country.name}' # Oslo, Norway +``` + +It steps up from a selected row, a drawn row or a [name](#names) bound to a row, one level +per `..`: `{l..municipality..region.name}`. After it, `.` steps down again, drawing +inside the row stepped up to. `.country` after a row still reads the link column's cell, +`NO`. A `..` that names anything but the parent table is an error naming the parent, and +so is a selector after a `..`, which could name a row outside the one stepped up to: +select rows before the `..`. `--list` advertises no path with a `..`. At the start of a +reference, `..` is the folder above ([References](#references)). + ### Options and fields `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options; diff --git a/docs/decisions.md b/docs/decisions.md index d81b87e..dbe5b36 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -511,36 +511,6 @@ case but the last, where two expected a render and all three accepted the refusa Goal 5.2 overrides its shared draw beneath a selection and its refusal of an unnamed read; `todo.md` item 4 replaces them. -## A link column after a row steps up to the parent row, and a path may not end on it - -Decided 2026-09-27 by the maintainer, for goal 5; valid while a link column holds the -parent's key. It extends [After a row, a path names a column or a linked -table](#after-a-row-a-path-names-a-column-or-a-linked-table) from child tables to the -parent. A row is selected or drawn alike. - -- Where a segment follows, the link column's name reads the parent row, one level per - step, chained: `city[Oslo].country.name` renders `Norway`, `locality.municipality.name` - the drawn locality's municipality, and - `geo.US.locality[1714000].municipality.region.name` Chicago's state. -- A path ending on it is refused at `New`, `NewTemplate` and `Fake` alike, naming the - parent's key column and the parent's format spelled as paths: with a `country` format - of `{name} ({alpha2})`, `city[Oslo].country` names `city[Oslo].country.alpha2` and - `{/city[Oslo].country.name} ({/city[Oslo].country.alpha2})`. `List` leaves such a path - out, and a record's link column still holds the key, since it is a column, not a path. -- A path may not step back down into a child table after a step up: - `city[Oslo].country.city.name` could be Oslo or a city of Norway drawn afresh, so it is - refused, naming the path to the row it stepped up from, `city[Oslo].name`, and the path - down from the parent, `country[NO].city.name`; after a drawn row, `city.country.city.name` - names `city.name` and `country.city.name`. -- A route through another row is another path to the same data, not a second spelling: - `city[Oslo].country.name` and `country[NO].name` both load. - -Checked against three simulated template writers from the Audience: all three read the -parent row where a segment followed, and they split between the code and the name where -the path ended. - -Goal 5.4 overrides this; `todo.md` item 2 replaces it. - ## A table selects by one key column, a code, never a free-form name Decided 2026-09-27 by the maintainer, for goal 5's one spelling per result; valid while @@ -938,6 +908,6 @@ the render. Valid while names are read only inside the category binding them. -## comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them +## comprehension floor: every dimension and the overall at 7.0 or above; items 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them 2026-10-04, Lilleman auf Larv. Serves goal 3. The nine-seat panels in `todo.md`'s git history, 2026-09-22 to 2026-09-30, held the score between 5.2 and 6.1, with Locality lowest at every seat, and the hardest units are the draw fences item 4 deletes. Items 4 and 90 delete or move what the earlier items touch, so a scoring run before item 90 merges would score code about to go. Items 2, 71, 3, 4 and 90 are the round `AGENTS.md`'s comprehension rule lets merge below the floor, a restructure the maintainer approved. No scoring run or panel runs until item 5, which runs the nine-seat panel and larv-review's scoring run. The scoring run writes this entry's first scores. Valid until item 5 runs. diff --git a/todo.md b/todo.md index f3f6acb..c0cfe6d 100644 --- a/todo.md +++ b/todo.md @@ -73,7 +73,6 @@ | 3 | 0.1.0 | decision | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | | 57 | 0.1.0 | | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 | -| 2 | 0.1.0 | decision | **Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`.** | 4 | 5 | 7 | 8 | 5.4 | 21.6 | | 34 | 0.1.0 | | **Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`.** | 2 | 4 | 6 | 7 | 4 | 20.9 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | @@ -230,12 +229,12 @@ Shape: T = table, t = template, c = choice. - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. - `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. They print it whole as `{a}`, so the country record's format appears once; revise the decision "A locale's `address` restates its country record's format". -- Needs item 2. Every shipped value must still pass its consumer's check before and after item 4. +- Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 2, 3 and 71. After it, `{/person} ({/person.first})` renders two people, as does `{/sv_SE.person as a}{/sv_SE.person as b}{a} & {b}`. `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Needs items 3 and 71. After it, `{/person} ({/person.first})` renders two people, as does `{/sv_SE.person as a}{/sv_SE.person as b}{a} & {b}`. `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables, Names and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides. @@ -246,21 +245,13 @@ Shape: T = table, t = template, c = choice. Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed. -### 2. Step up from a row to the row it links to with `..`: `{/city[Oslo]..country.name}` renders `Norway`, and `city[Oslo].country` stays the cell `NO`. - -- `..` goes up one level, to the parent table, and the name after it is that parent's: `{c..country.name}`, and from a locality `{l..municipality..region.name}`. Naming any other table is a load error naming the parent. -- It steps up from a selected row, a drawn row or a name bound to a row. -- After `..`, `.` steps down again: `{c..country.city.name}` is a city of `c`'s country, drawn afresh. -- `.country` after a row keeps reading the link column's cell, so the README's `city[Oslo].country` example stays true. -- A `Fake` path and `--list` take `..` too; `List` keeps advertising direct descents only. - ### 34. Add `{isin()}`, `{cusip()}`, `{aba()}` and `{vin()}`. `isin`: Luhn over letters expanded to digits. `aba`: 3-7-1 weights. `vin`: position 9 over the whole; a sample taking the WMI, since the check sits mid-string. ### 5. Run the nine-seat comprehension panel after the draw restructure, and file what it names. -Needs items 2, 71, 3, 4 and 90. The decision "comprehension floor: every dimension and the overall at 7.0 or above; items 2, 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them" suspends every scoring run and panel until then. +Needs items 71, 3, 4 and 90. The decision "comprehension floor: every dimension and the overall at 7.0 or above; items 71, 3, 4 and 90 ship with no scoring run, and item 5 scores them" suspends every scoring run and panel until then. ### 71. Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints. -- 2.52.0 From 1ec2bee098edc4258a0957259915bccaf10f3f24 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 5 Oct 2026 00:05:58 +0200 Subject: [PATCH 505/966] Answer architecture round 1: share a step down's draw per draw group and pick, own descend's pins, name the spelling to write, and record the .. decision --- AGENTS.md | 1 + README.md | 16 +++++++++++----- docs/decisions.md | 16 ++++++++++++++++ draws.go | 22 ++++++++++++++++++++-- path.go | 10 +++++----- reads.go | 20 +++++++++++--------- record.go | 8 +++++--- render.go | 19 ++++++++----------- table_test.go | 21 +++++++++++++++------ 9 files changed, 92 insertions(+), 41 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f9974c9..9df76c5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -80,6 +80,7 @@ In [docs/decisions.md](docs/decisions.md): - `parent` names the link column and the table alike - After a row, a path names a column or a linked table - A selected row is fixed, not drawn, and an unnamed read beside it is refused +- `..` steps up to the parent row, and a step down after it draws once per draw group or pick - A table selects by one key column, a code, never a free-form name - A parent row with no child row is a load error - The choice-of-rows fence guards a data file's root, and requires string fields diff --git a/README.md b/README.md index cd53345..a961750 100644 --- a/README.md +++ b/README.md @@ -585,11 +585,17 @@ fejkdata -d ./mydata '{/city as c}{c}, {c..country.name}' # Oslo, Norway ``` It steps up from a selected row, a drawn row or a [name](#names) bound to a row, one level -per `..`: `{l..municipality..region.name}`. After it, `.` steps down again, drawing -inside the row stepped up to. `.country` after a row still reads the link column's cell, -`NO`. A `..` that names anything but the parent table is an error naming the parent, and -so is a selector after a `..`, which could name a row outside the one stepped up to: -select rows before the `..`. `--list` advertises no path with a `..`. At the start of a +per `..`: `{l..municipality..region.name}`. `.country` after a row still reads the link +column's cell, `NO`. + +After a `..`, `.` steps down again, drawing afresh inside the row stepped up to, apart from +the row the path stepped up from. Paths stepping down from one `..` read one draw, as one +reference path does: `{/city..country.city.name}` and `{/city..country.city.population}` +describe one city, within a render and [draw group](#draw-group), or within a name's pick. + +A `..` that names anything but the parent table is an error naming the parent. A selector +after a `..` is an error naming the table to select from, since it could name a row outside +the one stepped up to. `--list` advertises no path with a `..`. At the start of a reference, `..` is the folder above ([References](#references)). ### Options and fields diff --git a/docs/decisions.md b/docs/decisions.md index dbe5b36..058a33f 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -511,6 +511,22 @@ case but the last, where two expected a render and all three accepted the refusa Goal 5.2 overrides its shared draw beneath a selection and its refusal of an unnamed read; `todo.md` item 4 replaces them. +## `..` steps up to the parent row, and a step down after it draws once per draw group or pick + +2026-10-05, larv-review on `todo.md` item 2. Goal 5.4 has `..` go up one level, and goal 5.2 +has each `{…}` make a new pick, which `todo.md` item 4 brings to every path. + +- `..` names the parent table next, `city..country`, as a folder path names the folder it + enters. `.country` after a row stays the link column's cell. +- A step down after a `..` draws afresh inside the row stepped up to (`drawSteps`), and the + draw fences leave it out (`tableReadOf`), since it pins none of the render's rows. Paths + stepping down from one `..` share that draw through their memo (`drawMemo.stepDownPins`), + as reference paths share a draw today, so two reads of one step down describe one row. +- A route through another row is another path to the same data, not a second spelling: + `city[Oslo]..country.name` and `country[NO].name` both load. + +Valid until item 4 makes every `{…}` draw afresh, which revises the shared step down. + ## A table selects by one key column, a code, never a free-form name Decided 2026-09-27 by the maintainer, for goal 5's one spelling per result; valid while diff --git a/draws.go b/draws.go index bfa6c2b..1dd17cc 100644 --- a/draws.go +++ b/draws.go @@ -4,12 +4,30 @@ import "strings" // drawMemo is what a hold, a draw group or a named pick has drawn: the variant each level was // drawn as, so every path under it reads one variant; the value each read produced, by its path, -// so the same read written twice reads one value; and the frames a read opens for the name -// scopes around where it lands (enter). +// so the same read written twice reads one value; the frames a read opens for the name +// scopes around where it lands (enter); and the rows drawn below each step up stepped down from. type drawMemo struct { variant map[string]node value map[string]readValue enteredFrames map[*nameScope]*pickFrame + steppedDown map[string]*pinSet +} + +// stepDownPins is the pins a path steps down into after stepping up to t, at the level at: kept +// in m where there is one, so every path stepping down there reads one draw. +func (m *drawMemo) stepDownPins(pins *pinSet, t *table, levels []string, at int) *pinSet { + if m == nil { + return pins.above(t) + } + p, ok := m.steppedDown[levels[at]] + if !ok { + p = pins.above(t) + if m.steppedDown == nil { + m.steppedDown = map[string]*pinSet{} + } + m.steppedDown[levels[at]] = p + } + return p } // groupDraws is what one render's draw group has drawn for its reference paths: a memo, and the diff --git a/path.go b/path.go index 399ec84..6247c82 100644 --- a/path.go +++ b/path.go @@ -155,9 +155,9 @@ func stepInto(n node, seg string) (node, error) { return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(seg)) case seg == "..": if c, ok := n.(*tableColumn); ok { - return nil, fmt.Errorf(`".." steps up from a table's row, and %q is a column`, c.t.header[c.i]) + return nil, fmt.Errorf(`".." steps up from a table's row, and %q is a column; put the ".." right after the row`, c.t.header[c.i]) } - return nil, fmt.Errorf(`".." steps up from a table's row to the row it links to, and what it follows is not a table`) + return nil, fmt.Errorf(`".." steps up from a table's row to the row it links to, and what it follows is not a table; step up only from a table`) } switch n := n.(type) { case *folder: @@ -233,7 +233,7 @@ func (t *table) stepUp(rest []string) error { case len(rest) == 0 || rest[0] != t.parentT.segment: return fmt.Errorf(`".." steps up from %s to its parent table, so name that next: ..%s`, t.segment, t.parentT.segment) case hasSelector(rest): - return fmt.Errorf(`a path selects its rows before a "..", since a row selected after it could lie outside the row it steps up to; start a path at the table to select from it`) + return fmt.Errorf(`a path selects its rows before a "..", since a row selected after it could lie outside the row it steps up to; select from the table instead: %s`, joinSegments(append([]string{t.parentT.path}, rest[1:]...))) } return nil } @@ -423,7 +423,7 @@ func probePath(n node, tail []string, steps []pathStep) ([]pathStep, error) { // pins is nil for a sibling path, which never crosses a table, since a table is only a category. // For a memoized read, memo keeps the variant drawn at each of levels, so paths sharing a prefix // share it. It returns the leaf and the pins its row is in: a step down after a step up draws -// afresh, into pins of its own. +// afresh, into pins of its own, which memo shares with every path stepping down there. func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo *drawMemo, levels []string) (node, *pinSet) { var climbed *table for _, st := range steps { @@ -445,7 +445,7 @@ func drawSteps(s *generatorState, n node, steps []pathStep, pins *pinSet, memo * case st.kind == stepParent: climbed = t.parentT case st.kind == stepChild && climbed != nil: - pins, climbed = pins.above(climbed), nil + pins, climbed = memo.stepDownPins(pins, climbed, levels, st.at), nil } n = t.drawStep(s, st, pins) } diff --git a/reads.go b/reads.go index 0c050be..4acf8fe 100644 --- a/reads.go +++ b/reads.go @@ -230,7 +230,7 @@ func (r readRoute) spelled(ref string) string { type tableRead struct { headTable *table pins pinSet - leafPins *pinSet // the rows the leaf renders in: pins, short of the rows below a step up + leafPins *pinSet // the rows the leaf renders in: pins, short of the rows a step down after a step up draws afresh drawn map[*table]bool sels []tableSel landsRow bool @@ -253,27 +253,29 @@ func tableReadOf(head node, a arm, leaf node) *tableRead { r := &tableRead{headTable: t} r.leafPins = &r.pins _, r.landsRow = leaf.(*tableRow) - climbed := false + var climbed *table for _, st := range a.steps { - if climbed && st.kind != stepChild && st.kind != stepParent { - continue - } switch st.kind { case stepParent: - t, climbed = t.parentT, true - r.leafPins = r.leafPins.above(t) + t, climbed = t.parentT, t.parentT + case stepChild: + if climbed != nil { + r.leafPins, climbed = r.leafPins.above(climbed), nil + } + t = t.descendant(st.name) case stepSelect: r.pins.pin(t, st.row) r.sels = append(r.sels, tableSel{t, joinSegments(append([]string{a.writtenHead}, a.tail[:st.at+1]...))}) case stepDraw: + if r.leafPins != &r.pins { + continue // a step down after a step up draws apart from the render's pins + } if r.drawn == nil { r.drawn = map[*table]bool{} } for d, stop := t, r.pins.nearestPinned(t); d != stop; d = d.parentT { r.drawn[d] = true } - case stepChild: - t = t.descendant(st.name) } } return r diff --git a/record.go b/record.go index b459301..27186a3 100644 --- a/record.go +++ b/record.go @@ -152,15 +152,17 @@ func (f *Generator) FakeRecord(path string) (*Record, error) { } // tableRecord walks a path's tail from a table to the table whose row is the record, -// pinning the rows it selects or draws. +// pinning in sc the rows it selects or draws. func tableRecord(s *generatorState, t *table, tail []string, sc renderScope) (node, error) { - n, err := descend(s, t, tail, sc) + n, pins, err := descend(s, t, tail) if err != nil { return nil, err } + group := sc.groupDraws() + group.pins = pins switch n := n.(type) { case *table: - n.drawIn(s, &sc.groupDraws().pins) + n.drawIn(s, &group.pins) return n, nil case *tableRow: return n.t, nil diff --git a/render.go b/render.go index 889617c..d862e96 100644 --- a/render.go +++ b/render.go @@ -27,7 +27,7 @@ func (f *Generator) Fake(path string) (string, error) { f.loadShippedAt(segments) var draws renderDraws sc := renderScope{draws: &draws} - n, err := descend(f.rand, &f.root, segments, sc) + n, pins, err := descend(f.rand, &f.root, segments) if err != nil { return "", fmt.Errorf("fejkdata: %s: %w", path, err) } @@ -35,25 +35,22 @@ func (f *Generator) Fake(path string) (string, error) { return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) } group := sc.groupDraws() + group.pins = pins sc, _ = sc.at(n, &group.pins).enter(n, &group.memo) return render(f.rand, n, sc), nil } -// descend walks a caller's path to the node it names, pinning in sc the rows it -// selects or draws. -func descend(s *generatorState, root node, segments []string, sc renderScope) (node, error) { +// descend walks a caller's path to the node it names, returning the rows its leaf renders in. +func descend(s *generatorState, root node, segments []string) (node, pinSet, error) { // docs/decisions.md#a-path-is-walked-once-without-drawing-before-it-is-walked-for-real var buf [16]pathStep steps, err := probePath(root, segments, buf[:0]) if err != nil { - return nil, err + return nil, pinSet{}, err } - group := sc.groupDraws() - n, pins := drawSteps(s, root, steps, &group.pins, nil, nil) - // A caller's path is its render's first draw, so the rows a step down after a step up drew - // are all the render pins. - group.pins = *pins - return n, nil + var pins pinSet + n, leafPins := drawSteps(s, root, steps, &pins, nil, nil) + return n, *leafPins, nil } // renderOnce renders n as one render, over draws of its own. diff --git a/table_test.go b/table_test.go index 2e852bd..68d0748 100644 --- a/table_test.go +++ b/table_test.go @@ -237,11 +237,13 @@ func TestTableDescendsToALinkedTable(t *testing.T) { func TestTableStepsUpToTheRowItLinksTo(t *testing.T) { files := with(geo(), map[string]string{ - "drawn.json": `"{/locality.code}|{/locality..municipality.code}|{/locality..municipality..region.code}"`, - "named.json": `"{/locality as l}{l.code}|{l..municipality.code}|{l..municipality..region.code}"`, - "down.json": `"{/locality.code}|{/locality..municipality.locality.code}"`, - "named-down.json": `"{/locality as l}{l.code}|{l..municipality.locality.code}"`, - "record.json": `{"format":"","l":"{/locality.code}","m":"{/locality..municipality.code}"}`, + "drawn.json": `"{/locality.code}|{/locality..municipality.code}|{/locality..municipality..region.code}"`, + "named.json": `"{/locality as l}{l.code}|{l..municipality.code}|{l..municipality..region.code}"`, + "down.json": `"{/locality.code}|{/locality..municipality.locality.code}"`, + "named-down.json": `"{/locality as l}{l.code}|{l..municipality.locality.code}"`, + "twice.json": `"{/locality..municipality.locality.code}|{/locality..municipality.locality.name}"`, + "named-twice.json": `"{/locality as l}{l..municipality.locality.code}|{l..municipality.locality.name}"`, + "record.json": `{"format":"","l":"{/locality.code}","m":"{/locality..municipality.code}"}`, }) f := newGenerator(t, writeFiles(t, files), WithSeed(4)) for path, want := range map[string]string{ @@ -259,6 +261,7 @@ func TestTableStepsUpToTheRowItLinksTo(t *testing.T) { if got := fakeTemplate(t, f, "{/locality[L4]..municipality.name}"); got != "Lund" { t.Errorf("{/locality[L4]..municipality.name} = %q, want Lund", got) } + names := map[string]string{"L1": "Stockholm", "L2": "Solna", "L3": "Malmö", "L4": "Lund", "L5": "Göteborg", "L6": "Hisingen", "L7": "Sandby", "L8": "Sandby"} afresh := map[string]bool{} for i := 0; i < 200; i++ { for _, path := range []string{"drawn", "named"} { @@ -281,6 +284,12 @@ func TestTableStepsUpToTheRowItLinksTo(t *testing.T) { t.Fatalf("%s = %v, want a locality of the first one's municipality", path, parts) } } + for _, path := range []string{"twice", "named-twice"} { + parts := strings.Split(fake(t, f, path), "|") + if names[parts[0]] != parts[1] { + t.Fatalf("%s = %v, want both reads of one step down to read one locality", path, parts) + } + } down, err := f.FakeRecord("locality[L4]..municipality.locality") if err != nil { t.Fatal(err) @@ -313,7 +322,7 @@ func TestTableStepUpRefusals(t *testing.T) { "locality[L4]..region.name": "..municipality", "region[12]..municipality.name": "no parent", "locality[L4]..": "..municipality", - "locality[L4]..municipality[0180]": "selects its rows before", + "locality[L4]..municipality[0180]": "municipality[0180]", "locality[L4].name..municipality": "column", "x..y": "not a table", } { -- 2.52.0 From 1551087ddf0742082939ad5380c5d9d8c799e02f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 5 Oct 2026 00:09:06 +0200 Subject: [PATCH 506/966] Answer architecture round 2: say the suggested path starts at the data root --- path.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/path.go b/path.go index 6247c82..f388da4 100644 --- a/path.go +++ b/path.go @@ -233,7 +233,7 @@ func (t *table) stepUp(rest []string) error { case len(rest) == 0 || rest[0] != t.parentT.segment: return fmt.Errorf(`".." steps up from %s to its parent table, so name that next: ..%s`, t.segment, t.parentT.segment) case hasSelector(rest): - return fmt.Errorf(`a path selects its rows before a "..", since a row selected after it could lie outside the row it steps up to; select from the table instead: %s`, joinSegments(append([]string{t.parentT.path}, rest[1:]...))) + return fmt.Errorf(`a path selects its rows before a "..", since a row selected after it could lie outside the row it steps up to; select from the table instead, its path from the data root: %s`, joinSegments(append([]string{t.parentT.path}, rest[1:]...))) } return nil } -- 2.52.0 From 406232e8f91e6c40971133f41958f2d08646cf39 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Mon, 5 Oct 2026 00:15:13 +0200 Subject: [PATCH 507/966] Answer stability round 1 and product-owner round 1: spell .. in the fence's and the link column's advice, take it in struct tags, and show the whole climb --- README.md | 4 ++-- cmd/fejkdata/main.go | 3 ++- drawfence.go | 4 ++-- path.go | 26 ++++++++++++++++++++++---- table_test.go | 17 +++++++++++++++++ 5 files changed, 45 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index a961750..c2b9ce0 100644 --- a/README.md +++ b/README.md @@ -588,8 +588,8 @@ It steps up from a selected row, a drawn row or a [name](#names) bound to a row, per `..`: `{l..municipality..region.name}`. `.country` after a row still reads the link column's cell, `NO`. -After a `..`, `.` steps down again, drawing afresh inside the row stepped up to, apart from -the row the path stepped up from. Paths stepping down from one `..` read one draw, as one +After a `..`, `.` steps down again, drawing afresh inside the row stepped up to, +independent of the row the path stepped up from, which it may draw again. Paths stepping down from one `..` read one draw, as one reference path does: `{/city..country.city.name}` and `{/city..country.city.population}` describe one city, within a render and [draw group](#draw-group), or within a name's pick. diff --git a/cmd/fejkdata/main.go b/cmd/fejkdata/main.go index 2299410..c14aa09 100644 --- a/cmd/fejkdata/main.go +++ b/cmd/fejkdata/main.go @@ -24,7 +24,8 @@ import ( const usage = `Usage: fejkdata [flags] a category, or a dotted path into one (person, person.last); - a table's row by key or name: 'misc.territory[SE]', 'misc.territory[Sweden].tld' + a table's row by key or name: 'misc.territory[SE]', 'misc.territory[Sweden].tld'; + .. steps up to the row a row links to: 'geo.SE.locality..municipality.name'