Compare commits

...

39 Commits

Author SHA1 Message Date
lilleman 5d981a7304 Drop the deferred section, superseded by todo.md
Tests / vet + fmt + tests (pull_request) Successful in 4s
Tests / vet + fmt + tests (push) Successful in 4s
2026-09-05 15:30:02 +02:00
lilleman b17106afab Record the release checklist and the later data de-duplication
Tests / vet + fmt + tests (pull_request) Successful in 4s
Tests / vet + fmt + tests (push) Successful in 4s
2026-09-05 12:12:58 +02:00
lilleman 314ba49b9d README redaction
Tests / vet + fmt + tests (pull_request) Successful in 1m2s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 23:05:27 +02:00
lilleman 19aae8bbfe Name the workflow in the gate-skip note
Tests / vet + fmt + tests (pull_request) Successful in 1m0s
2026-09-03 22:40:31 +02:00
lilleman 2d28220869 Diff the whole push range and de-duplicate the gate-skip note
Tests / vet + fmt + tests (pull_request) Failing after 3s
2026-09-03 22:36:48 +02:00
lilleman cc15bdd1d9 Run the merge gate only for changes its tests read
Tests / vet + fmt + tests (pull_request) Successful in 1m4s
2026-09-03 22:30:02 +02:00
lilleman 466ed5a4d9 Minor README update
Tests / vet + fmt + tests (pull_request) Successful in 1m4s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 22:09:42 +02:00
lilleman ce22c83e7d Cut the review-found narration: two helper invariants, one line each
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 21:53:51 +02:00
lilleman 0d2a080b85 Add the fast-enough goal and record the allocation-gate decision
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-03 21:46:22 +02:00
lilleman 296058fb80 Tests: allocation ceilings for deeply nested and wide renders 2026-09-03 21:46:22 +02:00
lilleman 44601c90ab Offer the reference hint only where it would render the same value, and name the guard set in the help
Tests / vet + fmt + tests (pull_request) Successful in 57s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 21:12:04 +02:00
lilleman 8ed8d9bb51 Tests: no hint where a reference would mean something else, and every reference the help advertises resolves 2026-09-03 21:11:11 +02:00
lilleman 1cd422352b Cut the comments the README already records, and record the cycle-fence and exit-code rules there
Tests / vet + fmt + tests (pull_request) Successful in 58s
2026-09-03 21:00:42 +02:00
lilleman 0cdbdb7bf4 Fix the help example, read padding past the shape gate, catch a stray closing brace, and stop the fieldless hint naming a spelling that fails 2026-09-03 20:58:01 +02:00
lilleman d8048df169 Tests: padded and stray-brace arguments, the inline held fence, a template under concurrent use, and the fieldless-token hint 2026-09-03 20:57:25 +02:00
lilleman 71cccaf4c9 Reject a leading quote that opens no JSON, and give the inline walk one home
Tests / vet + fmt + tests (pull_request) Successful in 55s
2026-09-03 20:43:04 +02:00
lilleman 704d18d109 Tests: a leading quote that opens no JSON names neither a template nor a path 2026-09-03 20:42:34 +02:00
lilleman 44c91668f9 Give the inline pipeline its own file, and record the shape, padding and template-API decisions
Tests / vet + fmt + tests (pull_request) Successful in 55s
2026-09-03 20:37:44 +02:00
lilleman 6916894ab5 One CLI classifier settling the argument shape before load; JSON strings are templates and a quote is reserved in names 2026-09-03 20:34:42 +02:00
lilleman 80e8d72bb7 One fence sequence over a node scope, and inline templates reject the folder sigils 2026-09-03 20:34:42 +02:00
lilleman 7c7bee5cd9 Tests: one fence sequence, inline folder sigils rejected, JSON string arguments, quoted names, and shape misuse before load 2026-09-03 20:31:49 +02:00
lilleman 841513bb27 Lead the README with what it does and a quick example, move Goals beside Decisions
Tests / vet + fmt + tests (pull_request) Successful in 57s
2026-09-03 20:14:11 +02:00
lilleman 873b7678ba Address reviewer findings: reference hint, consumer-terms type errors, misuse exit code, immutability and parity docs
Tests / vet + fmt + tests (pull_request) Successful in 57s
2026-09-03 19:21:42 +02:00
lilleman 4d58c93757 Tests: template misuse exits 2, and a bare token hints its reference spelling 2026-09-03 19:21:42 +02:00
lilleman 305d8f8edd Address stability nits: doc grammar, scalar-JSON note, bracket-rule rationale
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-03 17:53:05 +02:00
lilleman d8ece597c8 Reserve brackets in names, add compile-once NewTemplate, let --repeat reuse it, record scope artifacts
Tests / vet + fmt + tests (pull_request) Failing after 16s
2026-09-03 17:48:20 +02:00
lilleman 47ec883bb0 Tests: reusable NewTemplate, and bracket names rejected at load 2026-09-03 17:48:20 +02:00
lilleman 031a3ec7d6 Resolve architect findings: gate [ on json.Valid, fix the root-relative README claim, test isTemplate
Tests / vet + fmt + tests (pull_request) Failing after 16s
2026-09-03 17:40:29 +02:00
lilleman e08435afa7 Document inline templates: goal 4, the CLI and library, the shape-based disambiguation
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-03 17:30:09 +02:00
lilleman 3a9f3a4758 Route inline templates to FakeTemplate at the CLI 2026-09-03 17:29:07 +02:00
lilleman 4399b8b89f Tests for inline templates at the CLI 2026-09-03 17:28:14 +02:00
lilleman 3bc3e35045 Add FakeTemplate: compile an inline template and render it against the loaded tree 2026-09-03 17:27:54 +02:00
lilleman 5ba38a8281 Tests for FakeTemplate: inline format strings, JSON templates, references, and their errors 2026-09-03 17:27:51 +02:00
lilleman addba9abf7 Give maxLen its own literal and repeats MaxRepeat; arg errors name the spelling and the range; write's comment says what Flush reports; assert 64-bit at compile time
Tests / vet + fmt + tests (pull_request) Successful in 59s
Tests / vet + fmt + tests (push) Successful in 6s
2026-09-02 19:25:19 +02:00
lilleman a9552ed966 Tests for every branch of the constant-divisor fold, arg errors naming the spelling, and shell numbers at the CLI
Tests / vet + fmt + tests (pull_request) Failing after 43s
2026-09-02 19:24:46 +02:00
lilleman c5304bb6e7 Record the repeat cap's reach, 64-bit only, the constant-divisor rule and the rejected default spellings; correct the repeat cap comment
Tests / vet + fmt + tests (pull_request) Successful in 58s
2026-09-02 19:09:50 +02:00
lilleman 08c7ef9b7f Keep the real data-path error, reject an empty data path, and report a crypto/rand failure instead of seeding zero
Tests / vet + fmt + tests (pull_request) Successful in 58s
2026-09-02 19:09:29 +02:00
lilleman ae0527a26e Tests for the real data-path error, an empty data path, and a crypto/rand failure
Tests / vet + fmt + tests (pull_request) Failing after 16s
2026-09-02 19:09:19 +02:00
lilleman 4f9285ab5c Reject non-finite float bounds, unplain integer args, constant samples, a constant zero divisor and the default separator; a binding key is no path segment; ascii keeps DEL; an unheld path panics
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-02 19:08:32 +02:00
26 changed files with 1083 additions and 217 deletions
+17 -1
View File
@@ -16,12 +16,28 @@ jobs:
- uses: actions/checkout@v6
with:
persist-credentials: false
fetch-depth: 0
- name: Relevant changes
id: changes
run: |
if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then
git fetch --no-tags origin "$GITHUB_BASE_REF"
base="origin/$GITHUB_BASE_REF"
else
base='${{ github.event.before }}'
fi
if git diff --quiet "$base"...HEAD -- '*.go' go.mod go.sum Dockerfile .dockerignore .github data README.md; then
echo "code=false" >> "$GITHUB_OUTPUT"
else
echo "code=true" >> "$GITHUB_OUTPUT"
fi
# `docker build` streams the context to the daemon. A compose bind-mount of
# `.` mounts an empty host dir instead, because the job is itself a container.
- name: Latest supported Go
if: ${{ steps.changes.outputs.code == 'true' }}
run: docker build .
# Runs even when the step above failed, so a red build says whether the
# failure is version-specific.
- name: Lowest supported Go
if: ${{ !cancelled() }}
if: ${{ !cancelled() && steps.changes.outputs.code == 'true' }}
run: docker build --build-arg GO_VERSION=1.22.12 .
-1
View File
@@ -1,4 +1,3 @@
.claude
*.out
__pycache__/
todo.md
-8
View File
@@ -8,11 +8,3 @@
- A standing choice a reader would relitigate goes under Decisions in the README, not in a comment.
- A README example is a `json` block that loads and renders as a category; `readme_test.go` runs every one.
- Cyclomatic complexity is gated at 14: the table-shaped dispatches (`eachToken`, `calc.factor`, `walkPath`) sit at 1314 and stay whole; anything else that reaches 14 is decomposed.
# Deferred
- **Shipped-data de-duplication (2026-09-02).** `email.json`'s `local` is a
drifted copy of `username.json`, and no shipped file yet uses a held path, an
operand or a reference. Fixed in the data fill before the first tag, when the
shipped set is rewritten anyway; premise: nothing depends on the shipped data's
shape until then. Not raised in review before that.
+119 -38
View File
@@ -1,32 +1,17 @@
# fejkdata
A Go library and CLI for generating locale-aware fake data from JSON templates.
Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).
Locale-aware fake data for tests and fixtures, generated from JSON templates. Use
it as a Go library or the CLI — no data on disk, no dependencies, and a seed makes output
reproducible.
## Goals
1. **Valid by construction** — every value passes the check its real consumer
applies; facts that belong together come from one draw, within a value and
across categories.
2. **Text means what it says** — a format renders as written; only `{…}` varies,
random characters included (`{digits(3)}`). One spelling per result; the wrong
one is a load error naming the right one.
3. **Every mistake is a load error**`New` rejects; `Fake` on a loaded generator
fails only for an unknown path.
4. **Zero to a value in one command**`go install`, then `fejkdata sv_SE.person`:
no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position;
the first custom template needs no escape and no option.
5. **Data lives in JSON** — a builtin only for what data can't express.
6. **Reproducible** — seed in, same stream out; no builtin reads a clock.
7. **Zero dependencies** — standard library only.
8. **Docs index the grammar** — every syntax feature is a heading; every example
runs under test and shows its output; a rule is stated once.
```sh
go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest
fejkdata sv_SE.person # Sara Eriksson
```
## CLI
```sh
go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest
fejkdata sv_SE.person # Sara Eriksson
fejkdata sv_SE.person.last # Eriksson
fejkdata --seed 42 sv_SE.address # the same address every run
@@ -34,17 +19,27 @@ fejkdata -n 3 --separator ', ' sv_SE.word # nät, barn, sol
fejkdata --list # every path the data offers
fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data
fejkdata --no-shipped-data -d ./mydata --list # only your data
fejkdata 'name: {/sv_SE.person.last}' # name: <a surname> — an inline template
fejkdata '{"format":"name: {x}","x":["bosse","lina"]}' # name: bosse or name: lina
```
A path names a category, or a field inside one: each dot segment descends one
level — folders, then the category (a JSON file), then fields.
level — folders, then the category (a JSON file), then fields. An argument that is
a JSON object, array or string, or that carries a `{` token, is instead an
**inline template**: a format string or a JSON value compiled and rendered on the
spot. Its tokens reach the data by reference from the root —
`{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike
available. An inline template sits in no folder, so the folder-relative `{.name}`
and `{..name}` are rejected naming the root spelling. A path never contains a
brace, a bracket or a quote, so the two cannot collide (see
[Decisions](#decisions)).
| Flag | |
|------|--|
| `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash |
| `--no-shipped-data` | load only the `--data-path` directories |
| `-s`, `--seed N` | reproducible output |
| `-n`, `--repeat N` | render the path N times (up to 1048576), each an independent draw, streamed |
| `-n`, `--repeat N` | render the value N times (up to 1048576), each an independent draw, streamed |
| `--separator S` | between repeated values (default a newline) |
| `--list` | print every path, then exit |
| `--version`, `-h`, `--help` | print, then exit |
@@ -52,7 +47,9 @@ level — folders, then the category (a JSON file), then fields.
`--name value` and `--name=value` both work, a short flag's value attaches or
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
[Decisions](#decisions); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing
dir, unknown path), `2` misuse. From a checkout: `go run ./cmd/fejkdata …`.
dir, unknown path), `2` misuse — a bad flag, an argument that names neither a
template nor a path, or an inline template that does not compile. From a checkout:
`go run ./cmd/fejkdata …`.
### Your own data
@@ -89,6 +86,9 @@ if err != nil {
}
v, err := f.Fake("sv_SE.address") // "Kungsvägen 68\n379 17 Stockholm"
paths := f.List() // every path Fake accepts, sorted
v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call
t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once
v = t.Fake() // render many times, no re-parse
```
| Option | |
@@ -110,8 +110,8 @@ work with no data on disk. A directory is a namespace: each JSON file is a
category named after the file, each subdirectory a dot-path segment, so
`mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one.
Sources merge in order; matching folders combine, any other clash is won by the
last loaded. Names may not use `.`, `|`, `(`, `{`, `}` or `/`; dot-prefixed entries
are skipped, so a data directory can also be a checkout.
last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`;
dot-prefixed entries are skipped, so a data directory can also be a checkout.
Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`,
`person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`,
@@ -180,9 +180,9 @@ many times — each an independent draw — joined by `separator` (default `""`)
{ "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] }
```
Renders e.g. `bar foo baz`. A `separator` without a `repeat` is rejected, and so
is a `repeat` that multiplies to more than 1 048 576 renders along any path of
nested repeats.
Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat`,
a `separator` of `""` (the default), and a `repeat` that multiplies to more than
1 048 576 renders along any path of nested repeats.
### Options and fields
@@ -194,9 +194,11 @@ choice naming its item.
### Functions
A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad
count, range, country or expression fails fast, and a length, count or decimal
place beyond a sane maximum is rejected, so a fat-fingered `hex(2000000000)`
never tries to allocate gigabytes. Every builtin draws only from the seed — a
count, range, country or expression fails fast; an integer is written plain
(`5`, not `+5` or `05`); bounds are finite; a sample that could only ever emit one
value (`int(5,5)`, `float(1,1,2)`) is rejected naming the text to write instead;
and a length, count or decimal place beyond a sane maximum is rejected, so a
fat-fingered `hex(2000000000)` never tries to allocate gigabytes. Every builtin draws only from the seed — a
time-based id takes its timestamp from the rng, not the clock — so seeded output
stays reproducible.
@@ -243,8 +245,10 @@ hyphenated field can't be an operand.
```
Renders e.g. `19.99 x 3 = 59.97`. An operand that can never be a number (`"abc"`,
or a choice of such) is rejected at load; one that sometimes is not yields `NaN`,
and a division by zero `Inf` — both print rather than fail.
or a choice of such) is rejected at load, as is a division by a constant zero
(`1/0`, or a fixed `"0"` field); an operand that sometimes is not a number yields
`NaN`, and a division by one that is not constant `Inf` — both print rather than
fail.
### Transforms
@@ -337,6 +341,29 @@ then costs about what its output costs: an unweighted pick is O(1) whatever the
list's length, a weighted one O(log n), and long formats, deep nesting and many
tokens add cost in proportion to the output.
## Goals
1. **Valid by construction** — every value passes the check its real consumer
applies; facts that belong together come from one draw, within a value and
across categories.
2. **Text means what it says** — a format renders as written; only `{…}` varies,
random characters included (`{digits(3)}`). One spelling per result; the wrong
one is a load error naming the right one.
3. **Every mistake is a load error**`New` rejects the data and `NewTemplate`
the inline template; on a loaded generator `Fake` fails only for an unknown
path, and `Template.Fake` cannot fail at all.
4. **Zero to a value in one command**`go install`, then `fejkdata sv_SE.person`:
no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position;
the first custom template needs no escape and no option.
5. **Data lives in JSON** — a builtin only for what data can't express.
6. **Reproducible** — seed in, same stream out; no builtin reads a clock.
7. **Zero dependencies** — standard library only.
8. **Docs index the grammar** — every syntax feature is a heading; every example
runs under test and shows its output; a rule is stated once.
9. **Fast enough to be free** — a value renders in about a microsecond and `New`
parses and validates the whole set once upfront, so generating fixtures stays
noise against a test's own runtime.
## Decisions
- **Options and fields share one namespace.** `format`, `weight`, `repeat` and
@@ -352,6 +379,33 @@ tokens add cost in proportion to the output.
naming the double-dash spelling, and `-s=42` is rejected naming both short
spellings: `=` belongs to the long form, and reading `=42` as the value would
make `-d=./x` a directory named `=./x`.
- **An argument is a template by its shape, not by a flag.** A JSON object, array
or string, or a string carrying a `{` token, is an inline template; anything else
is a path. A name may not contain a brace, a bracket or a quote, so a path can
never collide with any of those spellings, and the leading `[` or `"` is gated on
valid JSON so a stray copied bracket never swallows an argument — it names
nothing, and says so. No `--template` flag is needed. Reserving the characters
whole — though only a leading one could collide — keeps one simple name rule
instead of a leading-position special case. The JSON string is what makes the
library's own advice reachable: the error for an object holding only a format
names `"…"`, and that spelling has to work where it is printed.
- **An inline template skips the cycle fence, and only that one.** `New` proves the
loaded tree acyclic, an inline node is a finite tree of its own, and nothing in
the tree can reference it, so no render of it reaches itself. Every other fence
runs over both, from one `checkScope`.
- **An inline template that does not compile is misuse (exit 2), including a
reference that resolves to nothing** — the whole argument is the spelling under
test, and `NewTemplate` compiles, links and validates as one step. An unknown
*path* stays a runtime error (exit 1): there the argument is well-formed and only
the data is absent.
- **A padded JSON argument is rejected, not trimmed.** Padding is the one place the
two readings disagree — a format string renders it, JSON drops it — so the
spelling that renders is named rather than silently chosen.
- **`FakeTemplate` and `NewTemplate` both stay.** They reach the same value but not
at the same cost: `NewTemplate` pays the compile and validation once and renders
many times, `FakeTemplate` is the one-shot call, and `--repeat` is exactly the
case that needs the first. The pair is `regexp.MustCompile` and `regexp.Match`,
not two spellings of one result.
- **The shipped data is embedded, not discovered.** A directory a machine happens
to have would make `--seed 42` machine-dependent. Data still lives in `data/`
as JSON; `--data-path` layers over it.
@@ -379,9 +433,33 @@ tokens add cost in proportion to the output.
when that file lacks a field those references read. Accepted: overriding is the
point of layering, the error names the reference and the field, and the fix is
the consumer's file carrying the fields the shipped tree reads.
- **The repeat cap bounds renders, not bytes.** A repeat, alone or nested, may
ask for at most 1 048 576 renders; how large each render is stays what the data
asked for, so `{hex(1048576)}` repeated to the cap is a terabyte, loaded without
complaint. A byte estimate would need every builtin to declare a width to fence
a shape no data comes near, and the harm lands on the author who wrote it.
- **64-bit targets only.** The gate builds amd64, and the buffer sizing a render
pre-computes (renders × bytes) assumes a 64-bit int; on a 32-bit target it could
overflow and panic.
- **A constant zero divisor is a load error; a divisor that is not constant prints
`Inf`.** `1/0` and a fixed `"0"` field are decidable, so they join the
never-numeric operand as a load error; the fold stops where an operand varies,
so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every
draw — catching it needs zero-absorbing algebra for a shape nobody writes.
- **In data, a default written out and a constant spelled as a sample are load
errors.** `weight: 1`, `repeat: 1`, `separator: ""`, `int(5,5)`, `float(1,1,2)`,
`+5` and `05` each spell what a shorter form already spells, so each is rejected
naming that form. The CLI's numbers follow the shell instead: `--seed 007` and
`--repeat +3` are 7 and 3, as every command line reads them.
- **Samples say what they emit, transforms what they do.** `{upper(2)}` is two
letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on
whether the argument looks like a number.
- **The performance gate asserts allocations, not wall-clock time.** `AllocsPerRun`
is deterministic across machines, so a ±10% ceiling does not flake under CI load,
while time varies with the machine and its neighbours. A rendering slowdown
almost always costs an allocation too (a lost pre-size, a per-item map, an extra
copy). The benchmark suite (see Development) reports time for a human, not as a
pass/fail gate.
## Development
@@ -406,8 +484,10 @@ docker compose run --rm --user "$(id -u):$(id -g)" tidy # go mod tidy
```
Every pull request runs `docker build .` against both the latest and the lowest
supported Go, and must pass before it can be merged. That build is the whole
gate — vet, complexity, format check and tests — so run it locally before pushing:
supported Go, and must pass before it can be merged — unless it changes none of
the files the build and its tests read, nor the workflow itself, in which case
it's skipped (see [Decisions](#decisions)). That build is the whole gate — vet,
complexity, format check and tests — so run it locally before pushing:
```sh
docker build . # latest
@@ -422,6 +502,7 @@ fejkdata.go Generator, New, options, the embedded data set, List
node.go the node model and JSON -> node compilation
path.go the dotted-path walk, and proving a path resolves
render.go Fake and the recursive renderer (choices, format strings, expansions)
inline.go inline templates: Template, NewTemplate, FakeTemplate, and their compile and link
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
hold.go the hold: one draw per expansion for paths and operands, and its fences
reference.go reference sigils, and binding references across the tree
@@ -435,4 +516,4 @@ data/ shipped data (JSON), embedded at build: locale folders + a misc
## License
MIT — see [LICENSE](LICENSE).
MIT — see [LICENSE](LICENSE). Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).
+57 -20
View File
@@ -2,6 +2,7 @@ package fejkdata
import (
"encoding/base64"
"errors"
"fmt"
"math"
"strconv"
@@ -9,12 +10,12 @@ import (
"unicode"
)
// maxLen caps sample output lengths (hex, nanoid, base64) and the renders a repeat
// multiplies to along any path; maxDecimals caps float/calc decimal places. So a
// fat-fingered or overflowing argument fails at New instead of trying to allocate
// gigabytes — or panicking — at render.
// maxLen caps sample output lengths (hex, nanoid, base64, digits, upper, lower)
// and maxDecimals float/calc decimal places, so a fat-fingered or overflowing
// argument fails at New instead of trying to allocate gigabytes — or panicking —
// at render.
const (
maxLen = MaxRepeat
maxLen = 1 << 20
maxDecimals = 1024
)
@@ -132,7 +133,7 @@ func transformArg(fields map[string]node, a []string) error {
_, _, err := refShape(leaf)
return err
}
return checkArm(leaf, fields)
return checkArm(leaf, fields, false)
}
func transformOperand(a []string) []string {
@@ -176,7 +177,7 @@ var asciiFolds = map[rune]string{
func asciiFold(s string) string {
var b strings.Builder
for _, r := range s {
if r < unicode.MaxASCII {
if r <= unicode.MaxASCII {
b.WriteRune(r)
} else {
b.WriteString(asciiFolds[r])
@@ -221,10 +222,31 @@ func randChars(r rng, n int, alphabet string) string {
return string(b)
}
// plainInt parses an integer arg written the one way: no sign, no leading zero.
func plainInt(s string) (int, error) {
n, err := strconv.Atoi(s)
if errors.Is(err, strconv.ErrRange) {
return 0, fmt.Errorf("%q is past the integer range: %w", s, err)
}
if err != nil {
return 0, fmt.Errorf("%q is not an integer", s)
}
if strconv.Itoa(n) != s {
return 0, fmt.Errorf("%q is not a plain integer; write %d", s, n)
}
return n, nil
}
func posIntArg(_ map[string]node, a []string) error {
n, err := strconv.Atoi(a[0])
if err != nil || n < 1 {
return fmt.Errorf("count %q must be a positive integer", a[0])
n, err := plainInt(a[0])
if errors.Is(err, strconv.ErrRange) {
return fmt.Errorf("count %q exceeds the maximum %d", a[0], maxLen)
}
if err != nil {
return fmt.Errorf("count %w", err)
}
if n < 1 {
return fmt.Errorf("count %q must be positive", a[0])
}
if n > maxLen {
return fmt.Errorf("count %d exceeds the maximum %d", n, maxLen)
@@ -233,14 +255,20 @@ func posIntArg(_ map[string]node, a []string) error {
}
func intRangeArgs(_ map[string]node, a []string) error {
lo, e1 := strconv.Atoi(a[0])
hi, e2 := strconv.Atoi(a[1])
if e1 != nil || e2 != nil {
return fmt.Errorf("int(min,max) needs integer args, got %q,%q", a[0], a[1])
lo, err := plainInt(a[0])
if err != nil {
return fmt.Errorf("int(min,max): min %w", err)
}
hi, err := plainInt(a[1])
if err != nil {
return fmt.Errorf("int(min,max): max %w", err)
}
if lo > hi {
return fmt.Errorf("int(min,max): min %d > max %d", lo, hi)
}
if lo == hi {
return fmt.Errorf("int(%d,%d) is the constant %d; write it as text", lo, hi, lo)
}
if uint64(hi)-uint64(lo) >= uint64(math.MaxInt64) { // span hi-lo+1 would overflow int -> IntN panic
return fmt.Errorf("int(min,max): range %d..%d is too wide", lo, hi)
}
@@ -250,19 +278,28 @@ func intRangeArgs(_ map[string]node, a []string) error {
func floatArgs(_ map[string]node, a []string) error {
lo, e1 := strconv.ParseFloat(a[0], 64)
hi, e2 := strconv.ParseFloat(a[1], 64)
dp, e3 := strconv.Atoi(a[2])
if e1 != nil || e2 != nil || e3 != nil {
return fmt.Errorf("float(min,max,dp) needs numeric args, got %q,%q,%q", a[0], a[1], a[2])
if e1 != nil || e2 != nil {
return fmt.Errorf("float(min,max,dp) needs numeric bounds, got %q,%q", a[0], a[1])
}
dp, err := plainInt(a[2])
if err != nil {
return fmt.Errorf("float(min,max,dp): decimals %w", err)
}
if math.IsNaN(lo) || math.IsNaN(hi) || math.IsInf(lo, 0) || math.IsInf(hi, 0) {
return fmt.Errorf("float(min,max,dp) needs finite bounds, got %q,%q", a[0], a[1])
}
if lo > hi {
return fmt.Errorf("float(min,max,dp): min %v > max %v", lo, hi)
}
if math.IsInf(hi-lo, 0) { // an overflowing span would render as "+Inf"
return fmt.Errorf("float(min,max,dp): range %v..%v is too wide", lo, hi)
}
if dp < 0 || dp > maxDecimals {
return fmt.Errorf("float(min,max,dp): decimals %d out of range 0..%d", dp, maxDecimals)
}
if lo == hi {
return fmt.Errorf("float(%s,%s,%d) is the constant %q; write it as text", a[0], a[1], dp, strconv.FormatFloat(lo, 'f', dp, 64))
}
if math.IsInf(hi-lo, 0) { // an overflowing span would render as "+Inf"
return fmt.Errorf("float(min,max,dp): range %v..%v is too wide", lo, hi)
}
return nil
}
+70 -2
View File
@@ -78,14 +78,82 @@ func checkCalc(fields map[string]node, args []string) error {
return fmt.Errorf("calc(%q): operand %q is never a number: it renders %q", args[0], name, text)
}
}
if divisor, zero := constantZeroDivisor(expr, fields); zero {
return fmt.Errorf("calc(%q) divides by %s, which is always zero", args[0], divisor)
}
if len(args) == 2 {
if dp, err := strconv.Atoi(args[1]); err != nil || dp < 0 || dp > maxDecimals {
return fmt.Errorf("calc decimals %q must be an integer in 0..%d", args[1], maxDecimals)
dp, err := plainInt(args[1])
if err != nil {
return fmt.Errorf("calc decimals %w", err)
}
if dp < 0 || dp > maxDecimals {
return fmt.Errorf("calc decimals %d must be in 0..%d", dp, maxDecimals)
}
}
return nil
}
// constantZeroDivisor finds a division whose right side is a constant zero: number
// literals and fixed operands folded, anything that varies left unknown.
func constantZeroDivisor(n calcNode, fields map[string]node) (string, bool) {
switch n := n.(type) {
case calcNeg:
return constantZeroDivisor(n.x, fields)
case calcBin:
if n.op == '/' {
if v, known := constantValue(n.r, fields); known && v == 0 {
return calcText(n.r), true
}
}
if d, zero := constantZeroDivisor(n.l, fields); zero {
return d, true
}
return constantZeroDivisor(n.r, fields)
}
return "", false
}
// constantValue evaluates an expression whose every operand is fixed.
func constantValue(n calcNode, fields map[string]node) (float64, bool) {
switch n := n.(type) {
case calcNum:
return float64(n), true
case calcVar:
t, ok := fields[string(n)].(*template)
if !ok || !t.fixed || t.repeat > 1 {
return 0, false
}
v, err := strconv.ParseFloat(strings.TrimSpace(t.lit), 64)
return v, err == nil
case calcNeg:
v, ok := constantValue(n.x, fields)
return -v, ok
case calcBin:
l, lok := constantValue(n.l, fields)
r, rok := constantValue(n.r, fields)
if !lok || !rok {
return 0, false
}
return calcBin{n.op, calcNum(l), calcNum(r)}.eval(nil), true
}
return 0, false
}
// calcText spells an expression node the way an author would read it.
func calcText(n calcNode) string {
switch n := n.(type) {
case calcNum:
return strconv.FormatFloat(float64(n), 'f', -1, 64)
case calcVar:
return string(n)
case calcNeg:
return "-" + calcText(n.x)
case calcBin:
return "(" + calcText(n.l) + " " + string(n.op) + " " + calcText(n.r) + ")"
}
return "?"
}
// neverNumeric reports a node no render of which is a number: fixed text that does
// not parse, or a choice of only such items. text is one such render.
func neverNumeric(n node) (text string, never bool) {
+13 -3
View File
@@ -247,11 +247,21 @@ func TestCalcOverANeverNumericOperandIsRejected(t *testing.T) {
}
func TestCalcConstantZeroDivisorIsRejected(t *testing.T) {
for _, bad := range []string{`"{calc(1/0)}"`, `"{calc(2/(1-1))}"`, `{"format":"{calc(x/y)}","x":"1","y":"0"}`, `{"format":"{calc(x/(y*2))}","x":"1","y":" 0 "}`} {
if _, err := compile(parse(t, bad)); err == nil || !strings.Contains(err.Error(), "zero") {
t.Errorf("compile(%s) = %v, want the constant zero divisor rejected", bad, err)
for src, want := range map[string]string{
`"{calc(1/0)}"`: "divides by 0",
`"{calc(2/(1-1))}"`: "divides by (1 - 1)",
`{"format":"{calc(x/y)}","x":"1","y":"0"}`: "divides by y",
`{"format":"{calc(x/(y*2))}","x":"1","y":" 0 "}`: "divides by (y * 2)",
`{"format":"{calc((a/0)+b)}","a":"1","b":"2"}`: "divides by 0",
`{"format":"{calc(a/-0)}","a":"1"}`: "divides by -0",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want the constant zero divisor rejected naming %q", src, err, want)
}
}
if _, err := compile(parse(t, `{"format":"{calc(a/(b*c))}","a":"1","b":"0","c":["1","2"]}`)); err != nil {
t.Errorf("compile(a/(b*c)) = %v, want a divisor that varies accepted", err)
}
f := engine(1)
for i := 0; i < 50; i++ {
if got := mustRender(t, f, `{"format":"{calc(x/y)}","x":"1","y":["0","1"]}`); got == "+Inf" {
+79 -19
View File
@@ -9,6 +9,7 @@ package main
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"io"
@@ -20,21 +21,30 @@ import (
"gitea.larvit.se/larvit/fejkdata"
)
const usage = `Usage: fejkdata [flags] <path>
const usage = `Usage: fejkdata [flags] <path|template>
<path> a category, or a dotted path into one (person, person.last)
<template> a format string or JSON value to render inline, e.g.
'name: {/sv_SE.person.last}' or '{"format":"{x}","x":["bosse","lina"]}'
An argument containing a { token, or a JSON object, array or string, is a
template; any other argument is a path (a path never contains a brace, a bracket
or a quote). Templates reach the data by reference from the root —
{/sv_SE.person.last} — whether the data is shipped or layered with --data-path. An
argument carrying a bracket, a closing brace or a quote but no valid JSON names
neither.
-d, --data-path D a data directory to layer over the shipped data (repeatable; last wins on a clash)
-h, --help print this help, then exit
--list list the paths the data offers, then exit
--no-shipped-data load only the --data-path directories
-n, --repeat N render the path N times, 1..1048576 (default 1)
-n, --repeat N render the value N times, 1..1048576 (default 1)
-s, --seed N seed for reproducible output
--separator S string between repeated values (default newline)
--version print the version, then exit
Flags may come before or after <path>; -- ends the flags. A short flag's value
attaches or follows (-n3, -n 3); short flags bundle (-hn 3).
Flags may come before or after <path|template>; -- ends the flags. A short flag's
value attaches or follows (-n3, -n 3); short flags bundle (-hn 3).
`
type invocation struct {
@@ -186,18 +196,22 @@ func parseArgs(argv []string) (invocation, error) {
return in, nil
}
// check rejects a flag combination that cannot run.
func (in invocation) check() error {
// check rejects a flag combination or an argument that cannot run, and reports
// what the argument names, so its shape is settled before any data is read.
func (in invocation) check() (argKind, error) {
if in.list && len(in.paths) > 0 {
return errors.New("--list takes no path")
return argPath, errors.New("--list takes no path")
}
if in.list && (in.repeatSet || in.separatorSet) {
return errors.New("--list takes no --repeat or --separator")
return argPath, errors.New("--list takes no --repeat or --separator")
}
if !in.list && len(in.paths) != 1 {
return fmt.Errorf("expected one path, got %d", len(in.paths))
if in.list {
return argPath, nil
}
return nil
if len(in.paths) != 1 {
return argPath, fmt.Errorf("expected one path or template, got %d", len(in.paths))
}
return classify(in.paths[0])
}
func (in invocation) options() []fejkdata.Option {
@@ -214,13 +228,23 @@ func (in invocation) options() []fejkdata.Option {
return opts
}
// write streams the path's renders to w, repeat of them joined by the separator
// and ended by a newline. A path that renders once renders every time, so the only
// failure comes before anything is written.
func (in invocation) write(f *fejkdata.Generator, w io.Writer) error {
// write streams the argument's renders to w, repeat of them joined by the
// separator and ended by a newline. A value that renders once renders every time,
// so a render failure comes before anything is written; a write failure surfaces
// from Flush, bufio keeping the first one.
func (in invocation) write(f *fejkdata.Generator, kind argKind, w io.Writer) error {
arg := in.paths[0]
draw := func() (string, error) { return f.Fake(arg) }
if kind == argTemplate {
t, err := f.NewTemplate(arg)
if err != nil {
return templateError{err}
}
draw = func() (string, error) { return t.Fake(), nil }
}
out := bufio.NewWriter(w)
for i := 0; i < in.repeat; i++ {
v, err := f.Fake(in.paths[0])
v, err := draw()
if err != nil {
return err
}
@@ -233,6 +257,36 @@ func (in invocation) write(f *fejkdata.Generator, w io.Writer) error {
return out.Flush()
}
// templateError marks a render failure that is the argument's own fault — an
// inline template that does not compile. run reports it as misuse (exit 2, with a
// pointer to --help), unlike an unknown path, which is a runtime error (exit 1).
type templateError struct{ error }
func (e templateError) Unwrap() error { return e.error }
type argKind int
const (
argPath argKind = iota
argTemplate
)
// classify reads what a positional argument names by its shape: a { token, or a
// JSON object, array or string, is an inline template; anything else is a path.
func classify(arg string) (argKind, error) {
if strings.ContainsRune(arg, '{') || (isJSONStart(strings.TrimSpace(arg)) && json.Valid([]byte(arg))) {
return argTemplate, nil
}
if i := strings.IndexAny(arg, `[]}"`); i >= 0 {
return argPath, fmt.Errorf("%q holds a %q, which no path may, and it is not valid JSON, so it names no template either", arg, arg[i:i+1])
}
return argPath, nil
}
func isJSONStart(arg string) bool {
return strings.HasPrefix(arg, "[") || strings.HasPrefix(arg, `"`)
}
func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) }
// run returns the exit code: 0 ok, 1 runtime error, 2 misuse.
@@ -249,7 +303,8 @@ func run(args []string, stdout, stderr io.Writer) int {
fmt.Fprintln(stdout, "fejkdata "+buildVersion())
return 0
}
if err := in.check(); err != nil {
kind, err := in.check()
if err != nil {
return misuse(stderr, err)
}
f, err := fejkdata.New(in.options()...)
@@ -266,7 +321,11 @@ func run(args []string, stdout, stderr io.Writer) int {
}
return 0
}
if err := in.write(f, stdout); err != nil {
if err := in.write(f, kind, stdout); err != nil {
var te templateError
if errors.As(err, &te) {
return misuse(stderr, te.error)
}
fmt.Fprintln(stderr, err)
return 1
}
@@ -274,7 +333,8 @@ func run(args []string, stdout, stderr io.Writer) int {
}
func misuse(stderr io.Writer, err error) int {
fmt.Fprintf(stderr, "fejkdata: %v\ntry 'fejkdata --help'\n", err)
// A library error already names the program, so the prefix is not doubled.
fmt.Fprintf(stderr, "fejkdata: %s\ntry 'fejkdata --help'\n", strings.TrimPrefix(err.Error(), "fejkdata: "))
return 2
}
+102
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
)
@@ -301,6 +302,92 @@ func TestRunShippedDataByDefault(t *testing.T) {
}
}
func TestClassify(t *testing.T) {
for arg, want := range map[string]argKind{
"sv_SE.person": argPath,
"person.last": argPath,
"name: {x}": argTemplate, // a { token: a path can never carry a brace
`{"format":"x"}`: argTemplate,
`["a","b"]`: argTemplate, // a JSON array carries no brace
`[1, 2]`: argTemplate,
` ["a","b"]`: argTemplate, // padding is the template's own error, not a shape verdict
`"hello"`: argTemplate, // a JSON string, the spelling a format-only object names
} {
got, err := classify(arg)
if err != nil || got != want {
t.Errorf("classify(%q) = %v, %v; want %v", arg, got, err, want)
}
}
for arg, want := range map[string]string{
"[abc]": `holds a "["`,
"[abc].field": `holds a "["`,
"x[1]": `holds a "["`,
"a]b": `holds a "]"`,
"a}b": `holds a "}"`,
`"abc`: `holds a "\""`,
`"a]b`: `holds a "\""`, // the opener the reader typed, not the bracket behind it
} {
_, err := classify(arg)
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("classify(%q) = %v; want it rejected naming %s", arg, err, want)
}
}
}
func TestUsageReferencesResolve(t *testing.T) {
for _, token := range regexp.MustCompile(`\{/[^}]+\}`).FindAllString(usage, -1) {
code, out, errb := runOut("--seed", "1", token)
if code != 0 || strings.TrimSpace(out) == "" {
t.Errorf("usage advertises %s: run = %d, %q, stderr %q", token, code, out, errb)
}
}
}
func TestRunShapeMisuseBeforeLoad(t *testing.T) {
code, _, errb := runOut("--no-shipped-data", "[abc]")
if code != 2 || !strings.Contains(errb, "[abc]") || strings.Contains(errb, "--data-path") {
t.Fatalf("shape misuse with no data = %d, %q; want the shape error before any load", code, errb)
}
}
func TestRunInlineTemplate(t *testing.T) {
code, out, errb := runOut("--seed", "1", "name: {/sv_SE.person.last}")
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline format string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", `{"format":"name: {x}","x":["bosse","lina"]}`)
if code != 0 || (out != "name: bosse\n" && out != "name: lina\n") {
t.Fatalf("inline JSON template = %d, %q, want one name, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", `"name: {/sv_SE.person.last}"`)
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline JSON string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", "-n", "2", `{digits(1)}`)
if code != 0 || len(strings.Split(strings.TrimRight(out, "\n"), "\n")) != 2 {
t.Fatalf("inline template with --repeat = %d, %q, stderr %q", code, out, errb)
}
}
func TestRunTemplateMisuse(t *testing.T) {
for arg, want := range map[string]string{
"{bad": "unterminated",
"[red,green]": "names no template either",
`{"format":"x"}`: "is a string",
"{/no.such.path}": "no entry",
"x[1]": "names no template either",
` ["a","b"] `: "may not be padded",
} {
code, out, errb := runOut("--seed", "1", arg)
if code != 2 || out != "" || !strings.Contains(errb, "try 'fejkdata --help'") || !strings.Contains(errb, want) {
t.Errorf("run(%q) = %d, %q, %q; want misuse naming %q and --help", arg, code, out, errb, want)
}
if strings.Contains(errb, "fejkdata: fejkdata:") {
t.Errorf("run(%q) doubled the program prefix: %q", arg, errb)
}
}
}
func TestRunNoShippedData(t *testing.T) {
code, list, errb := runOut("--no-shipped-data", "-d", svSE, "--list")
if code != 0 {
@@ -355,3 +442,18 @@ func TestRunUnknownFlagIsNamedByRune(t *testing.T) {
t.Errorf("run(-ä) = %d, %q, want the flag named whole", code, errb)
}
}
func TestRunEmptyDataPathIsNamed(t *testing.T) {
code, _, errb := runOut("--data-path=", "sv_SE.word")
if code != 1 || !strings.Contains(errb, "empty") {
t.Errorf("run(--data-path=) = %d, %q, want the empty path named", code, errb)
}
}
func TestRunNumbersFollowTheShell(t *testing.T) {
_, want, _ := runOut("--seed", "7", "--repeat", "3", "sv_SE.word")
code, got, errb := runOut("--seed", "007", "--repeat", "+3", "sv_SE.word")
if code != 0 || got != want {
t.Errorf("run(--seed 007 --repeat +3) = %d, %q, stderr %q; want the same as --seed 7 --repeat 3 %q", code, got, errb, want)
}
}
+13 -12
View File
@@ -10,13 +10,14 @@ import (
)
// dataSource is one tree to load: an fs.FS and the directory in it to start from.
// label prefixes file names in errors; path, when set, is a directory on disk that
// must exist.
// label prefixes file names in errors; onDisk marks path as a directory that must
// exist.
type dataSource struct {
fsys fs.FS
label string
path string
root string
fsys fs.FS
label string
onDisk bool
path string
root string
}
func (s dataSource) name(p string) string {
@@ -36,10 +37,13 @@ func (s dataSource) name(p string) string {
func loadData(sources []dataSource) (map[string]node, error) {
root := map[string]node{}
for _, src := range sources {
if src.path != "" {
if src.onDisk {
if src.path == "" {
return nil, fmt.Errorf("a data path is empty")
}
info, err := os.Stat(src.path)
if err != nil {
return nil, fmt.Errorf("%s: no such directory", src.path)
return nil, err
}
if !info.IsDir() {
return nil, fmt.Errorf("%s is not a directory", src.path)
@@ -64,10 +68,7 @@ func loadData(sources []dataSource) (map[string]node, error) {
if err := checkNoCycles(root); err != nil {
return nil, err
}
if err := checkRepeatReach(root); err != nil {
return nil, err
}
if err := checkBoundLevelsHeld(root); err != nil {
if err := checkScope(treeScope(root)); err != nil {
return nil, err
}
return root, nil
+24 -9
View File
@@ -28,15 +28,19 @@ import (
//go:embed data
var shippedFS embed.FS
// MaxRepeat caps a repeat, and the renders nested repeats multiply to along any path.
// MaxRepeat caps a repeat, and the renders nested repeats multiply to along any
// path; the CLI's --repeat shares it.
const MaxRepeat = 1 << 20
var _ [^uint(0)>>63 - 1]struct{} // 64-bit only, per the README's Decisions
// ErrNoData is returned by New when no source is loaded at all.
var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS")
// Generator generates fake data from a loaded namespace tree. Create one with [New].
// It is safe for concurrent use; a seeded sequence is reproducible only when drawn
// from one goroutine.
// from one goroutine. The compiled tree is immutable after [New], and Fake,
// NewTemplate and List read it concurrently without a lock.
type Generator struct {
mu sync.Mutex
rand *session
@@ -75,7 +79,7 @@ func WithSeed(seed uint64) Option {
// layer several; the last wins a name clash.
func WithDataPath(dir string) Option {
return func(c *config) {
c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, path: dir})
c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, onDisk: true, path: dir})
}
}
@@ -112,7 +116,11 @@ func New(opts ...Option) (*Generator, error) {
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Generator{rand: newRand(c.seed, c.seeded), categories: cats}, nil
rng, err := newRand(c.seed, c.seeded)
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Generator{rand: rng, categories: cats}, nil
}
// List returns the sorted dotted paths Fake can render: every category, the dotted
@@ -203,12 +211,19 @@ func join(prefix, name string) string {
return prefix + "." + name
}
func newRand(seed uint64, seeded bool) *session {
r := rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15))
if !seeded {
// randomBytes seeds an unseeded generator.
var randomBytes = crand.Read
func newRand(seed uint64, seeded bool) (*session, error) {
var r *rand.Rand
if seeded {
r = rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15))
} else {
var b [16]byte
_, _ = crand.Read(b[:])
if _, err := randomBytes(b[:]); err != nil {
return nil, fmt.Errorf("seeding from crypto/rand: %w", err)
}
r = rand.New(rand.NewPCG(binary.LittleEndian.Uint64(b[:8]), binary.LittleEndian.Uint64(b[8:])))
}
return &session{Rand: r, counters: map[string]uint64{}}
return &session{Rand: r, counters: map[string]uint64{}}, nil
}
+22 -2
View File
@@ -1,8 +1,11 @@
package fejkdata
import (
"errors"
"io/fs"
"strings"
"testing"
"testing/fstest"
)
// newGenerator creates a generator over a single data directory, failing on
@@ -38,8 +41,25 @@ func fake(t *testing.T, f *Generator, path string) string {
func TestNewMissingDirectory(t *testing.T) {
_, err := New(WithoutShippedData(), WithDataPath("data/de_DE"))
if err == nil || !strings.Contains(err.Error(), "de_DE") {
t.Fatalf("New(missing) error = %v, want it to name the path", err)
if err == nil || !strings.Contains(err.Error(), "de_DE") || !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("New(missing) error = %v, want the real error, naming the path", err)
}
}
func TestNewRejectsAnEmptyDataPath(t *testing.T) {
_, err := New(WithoutShippedData(), WithDataPath(""))
if err == nil || !strings.Contains(err.Error(), "empty") {
t.Fatalf("New(WithDataPath(\"\")) = %v, want the empty path named", err)
}
}
func TestNewReportsAnEntropyFailure(t *testing.T) {
saved := randomBytes
randomBytes = func([]byte) (int, error) { return 0, errors.New("no entropy") }
defer func() { randomBytes = saved }()
_, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"w.json": {Data: []byte(`"x"`)}}))
if err == nil || !strings.Contains(err.Error(), "no entropy") {
t.Fatalf("New() without entropy = %v, want the failure reported", err)
}
}
+69 -40
View File
@@ -10,29 +10,36 @@ import (
// visiting keys in sorted order so which of several broken nodes gets reported does
// not depend on map iteration.
func walkNodes(root map[string]node, fn func(path string, n node) error) error {
seen := map[node]bool{}
var visit func(string, node) error
visit = func(path string, n node) error {
if n == nil || seen[n] {
return nil
}
seen[n] = true
if err := fn(path, n); err != nil {
for _, name := range sortedNames(root) {
if err := eachNode(root[name], name, fn); err != nil {
return err
}
for _, c := range contained(n) {
}
return nil
}
// eachNode visits n and every node contained within it once, passing the dot path
// that reaches each. It never crosses a reference edge — a bound {/path} field is
// skipped — so a single inline node is walked on its own.
func eachNode(n node, path string, fn func(path string, n node) error) error {
seen := map[node]bool{}
var visit func(string, node) error
visit = func(path string, m node) error {
if m == nil || seen[m] {
return nil
}
seen[m] = true
if err := fn(path, m); err != nil {
return err
}
for _, c := range contained(m) {
if err := visit(join(path, c.name), c.node); err != nil {
return err
}
}
return nil
}
for _, name := range sortedNames(root) {
if err := visit(name, root[name]); err != nil {
return err
}
}
return nil
return visit(path, n)
}
// namedNode is a contained child and the segment reaching it; a choice's items carry
@@ -158,34 +165,56 @@ func pathLeaves(n node, tail []string) []node {
return out
}
// checkRepeatReach bounds the renders a repeat multiplies to along any root-to-leaf
// path, so nested repeats cannot build what one repeat may not. It runs after
// checkNoCycles, whose guarantee is what lets the walk terminate.
func checkRepeatReach(root map[string]node) error {
reach := map[node]int{}
var of func(n node) int
of = func(n node) int {
if r, done := reach[n]; done {
return r
}
r := 1
for _, e := range renderEdges(n) {
if c := of(e.to); c > r {
r = c
}
}
if t, ok := n.(*template); ok {
r *= t.repeat
}
reach[n] = r
// nodeScope is the set of nodes one validation pass covers: a whole loaded tree,
// or a single inline node.
type nodeScope func(fn func(path string, n node) error) error
func treeScope(root map[string]node) nodeScope {
return func(fn func(path string, n node) error) error { return walkNodes(root, fn) }
}
func inlineScope(n node) nodeScope {
return func(fn func(path string, m node) error) error { return eachNode(n, "template", fn) }
}
// checkScope runs the per-node fences over a scope, each over the whole scope
// before the next, so which of several broken nodes is reported does not depend on
// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks
// terminate.
func checkScope(s nodeScope) error {
mem := reachMemo{}
if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil {
return err
}
return s(heldCheck)
}
type reachMemo map[node]int
func (m reachMemo) of(n node) int {
if r, done := m[n]; done {
return r
}
return walkNodes(root, func(path string, n node) error {
if t, ok := n.(*template); ok && t.repeat > 1 && of(n) > maxLen {
return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, of(n), maxLen)
r := 1
for _, e := range renderEdges(n) {
if c := m.of(e.to); c > r {
r = c
}
return nil
})
}
if t, ok := n.(*template); ok {
r *= t.repeat
}
m[n] = r
return r
}
// repeatCheck bounds the renders a repeat multiplies to along any root-to-leaf
// path, so nested repeats cannot build what one repeat may not.
func repeatCheck(path string, n node, mem reachMemo) error {
if t, ok := n.(*template); ok && t.repeat > 1 && mem.of(n) > MaxRepeat {
return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.of(n), MaxRepeat)
}
return nil
}
// checkNoCycles rejects a reference cycle: a node whose rendering can reach itself
+17 -18
View File
@@ -6,27 +6,23 @@ import (
"strings"
)
// checkBoundLevelsHeld rejects every route to a held name except the ones that read
// its draw. An expansion holds one draw of that name; anything else that renders it
// draws again, and the two disagree. checkNoOverlap settles the spellings within one
// heldCheck rejects every route to a held name except the ones that read its draw.
// An expansion holds one draw of that name; anything else that renders it draws
// again, and the two disagree. checkNoOverlap settles the spellings within one
// format (a token, an operand); this settles the rest — a reference, whether it
// sits in that format or in anything the format renders, however deep.
//
// It runs after checkNoCycles, whose guarantee is what lets the walk terminate.
func checkBoundLevelsHeld(root map[string]node) error {
return walkNodes(root, func(path string, n node) error {
t, ok := n.(*template)
if !ok || len(t.held) == 0 {
return nil
}
readers := boundReaders(t.format, t.bound, t.refs)
for _, head := range heldHeads(t) {
if err := checkHeadHeld(t, head, readers); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
}
func heldCheck(path string, n node) error {
t, ok := n.(*template)
if !ok || len(t.held) == 0 {
return nil
})
}
readers := boundReaders(t.format, t.bound, t.refs)
for _, head := range heldHeads(t) {
if err := checkHeadHeld(t, head, readers); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
}
return nil
}
// heldHeads lists a template's held names, operand heads first, then paths, each
@@ -273,6 +269,9 @@ type draws struct {
// linkRefs prove every step, so the walk cannot fail.
func readField(s *session, t *template, held *draws, a arm) string {
if !t.held[a.key] {
if len(a.tail) > 0 {
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
}
return render(s, t.fields[a.key])
}
if v, read := held.value[a.path]; read {
+88
View File
@@ -0,0 +1,88 @@
package fejkdata
import (
"encoding/json"
"fmt"
"strings"
)
// Template is an inline template compiled, referenced and validated against a
// generator's loaded data once, ready to render many times with [Template.Fake].
// It is safe for concurrent use: Fake serializes on its generator's lock, so a
// seeded sequence is reproducible only when a generator — and its templates — are
// drawn from one goroutine.
type Template struct {
g *Generator
n node
}
// Fake renders the template with one draw.
func (t *Template) Fake() string {
t.g.mu.Lock()
defer t.g.mu.Unlock()
return render(t.g.rand, t.n)
}
// NewTemplate compiles an inline template — a format string or a JSON value — and
// binds its references against the loaded tree, so repeated renders pay the
// compile and validation once. It shares [New]'s guarantees: a bad template errors
// here, and rendering cannot fail.
func (f *Generator) NewTemplate(input string) (*Template, error) {
n, err := compileInput(input)
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
scope := inlineScope(n)
if err := linkNodeRefs(scope, f.categories); err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
if err := checkScope(scope); err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Template{g: f, n: n}, nil
}
// FakeTemplate compiles and renders an inline template in one call. It is
// [NewTemplate] then [Template.Fake]; to render the same template many times, hold
// the *Template and call its Fake.
func (f *Generator) FakeTemplate(input string) (string, error) {
t, err := f.NewTemplate(input)
if err != nil {
return "", err
}
return t.Fake(), nil
}
// compileInput compiles an inline template: a JSON value, or a bare format string
// when the input is not JSON.
func compileInput(input string) (node, error) {
var raw any
if err := json.Unmarshal([]byte(input), &raw); err != nil {
return compile(input)
}
if trimmed := strings.TrimSpace(input); trimmed != input {
return nil, fmt.Errorf("a JSON template may not be padded with spaces, which a format string would render; write %s", trimmed)
}
return compile(raw)
}
// linkNodeRefs binds the references in an inline node's templates against the
// loaded tree.
func linkNodeRefs(scope nodeScope, root map[string]node) error {
return scope(func(path string, m node) error {
t, ok := m.(*template)
if !ok {
return nil
}
for _, name := range refTokens(t.format) {
sigil, rest, err := refShape(name)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
if sigil != "/" {
return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest)
}
}
return linkTemplateRefs(nil, path, t, root)
})
}
+159
View File
@@ -0,0 +1,159 @@
package fejkdata
import (
"strings"
"testing"
)
func tmpl(t *testing.T, f *Generator, input string) string {
t.Helper()
s, err := f.FakeTemplate(input)
if err != nil {
t.Fatalf("FakeTemplate(%q): %v", input, err)
}
return s
}
func shipped(t *testing.T, opts ...Option) *Generator {
t.Helper()
f, err := New(append([]Option{WithSeed(1)}, opts...)...)
if err != nil {
t.Fatal(err)
}
return f
}
func TestFakeTemplateFormatString(t *testing.T) {
f := shipped(t)
got := tmpl(t, f, "name: {/sv_SE.person.last}")
if !strings.HasPrefix(got, "name: ") || strings.HasSuffix(got, " ") || strings.Contains(got, "{") {
t.Fatalf("FakeTemplate = %q, want a rendered last name after the prefix", got)
}
}
func TestFakeTemplateJSONObject(t *testing.T) {
f := shipped(t)
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[tmpl(t, f, `{"format":"name: {x}","x":["bosse","lina"]}`)] = true
}
if !seen["name: bosse"] || !seen["name: lina"] || len(seen) != 2 {
t.Fatalf("JSON template produced %v, want both names", seen)
}
}
func TestFakeTemplateJSONArray(t *testing.T) {
f := shipped(t)
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[tmpl(t, f, `["foo","bar","baz"]`)] = true
}
if len(seen) != 3 {
t.Fatalf("JSON array choice produced %v, want three items", seen)
}
}
func TestFakeTemplateCorrelatedReferences(t *testing.T) {
dir := writeData(t, map[string]string{
"person": `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"}]`,
})
f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 100; i++ {
got := tmpl(t, f, "{/person.first} {/person.last}")
if got != "Ada Lovelace" && got != "Bo Ek" {
t.Fatalf("correlated references = %q, want one person's first and last", got)
}
}
}
func TestFakeTemplateDeterministic(t *testing.T) {
a, b := shipped(t), shipped(t)
for i := 0; i < 20; i++ {
in := "row: {/misc.uuid} {digits(3)}"
if x, y := tmpl(t, a, in), tmpl(t, b, in); x != y {
t.Fatalf("same seed diverged: %q != %q", x, y)
}
}
}
func TestFieldlessTokenHint(t *testing.T) {
f := shipped(t)
_, err := f.FakeTemplate(`{sv_SE.person.last}`)
if err == nil || !strings.Contains(err.Error(), "write {/sv_SE.person.last}") {
t.Fatalf("FakeTemplate(bare token) = %v, want a hint naming {/sv_SE.person.last}", err)
}
_, err = f.FakeTemplate(`{"format":"{x}","repeat":2}`)
if err == nil || !strings.Contains(err.Error(), `this template has none — write {/x}`) {
t.Errorf("FakeTemplate(fieldless object) = %v, want the hint without calling it a bare string", err)
}
// A hint is only a drop-in where the name is the whole token: {/x} inside a
// transform or an alternation renders a different value, so none is offered.
for _, input := range []string{`{ /sv_SE.person.last }`, "{lowercase(x)}", "{x|y}"} {
_, err := f.FakeTemplate(input)
if err == nil || strings.Contains(err.Error(), "write {") {
t.Errorf("FakeTemplate(%q) = %v, want no hint naming a spelling that means something else", input, err)
}
}
}
func TestFakeTemplateErrors(t *testing.T) {
f := shipped(t)
for _, c := range []struct {
input string
want string
}{
{`{"x":"Q"}`, "missing string \"format\""},
{`"{x}"`, `no field "x"`},
{`"{digits(0)}"`, "must be positive"},
{`name: {/no.such.path}`, "no entry"},
{`name: {..nope}`, "write {/nope}"},
{`{"format":"x"}`, "is a string"},
{`{/misc.country} {/misc.country.alpha2}`, "renders a level"},
{`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"},
} {
_, err := f.FakeTemplate(c.input)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("FakeTemplate(%q) = %v, want an error containing %q", c.input, err, c.want)
}
}
}
func TestFakeTemplateJSONString(t *testing.T) {
f := shipped(t)
got := tmpl(t, f, `"name: {/sv_SE.person.last}"`)
if !strings.HasPrefix(got, "name: ") || strings.Contains(got, "{") {
t.Fatalf("FakeTemplate(JSON string) = %q, want a rendered last name after the prefix", got)
}
}
func TestPaddedJSONIsRejected(t *testing.T) {
f := shipped(t)
in := `{"format":"{x}","x":["a","b"]}`
_, err := f.NewTemplate(" " + in + " ")
if err == nil || !strings.Contains(err.Error(), "write "+in) {
t.Fatalf("NewTemplate(padded JSON) = %v, want an error naming the unpadded spelling", err)
}
}
func TestNewTemplateReusable(t *testing.T) {
f := shipped(t)
reusable, err := f.NewTemplate(`{digits(2)}`)
if err != nil {
t.Fatalf("NewTemplate: %v", err)
}
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[reusable.Fake()] = true
}
if len(seen) < 2 {
t.Fatalf("Template.Fake() repeated %v, want varied draws from one compile", seen)
}
}
func TestFakeTemplateRepeatBound(t *testing.T) {
f := shipped(t)
_, err := f.FakeTemplate(`{"format":"{x}","repeat":200,"x":{"format":"{y}","repeat":200,"y":{"format":"z","repeat":200}}}`)
if err == nil || !strings.Contains(err.Error(), "maximum") {
t.Errorf("nested repeat over the cap = %v, want it rejected naming the maximum", err)
}
}
+12
View File
@@ -275,6 +275,18 @@ func TestNewErrors(t *testing.T) {
map[string]string{"a|b": `"1"`},
`category "a|b" contains "|"`,
},
"field name with a bracket": {
map[string]string{"a": `{"format":"{x}","x":"1","b[c":"2"}`},
`field "b[c" contains "["`,
},
"category name with a bracket": {
map[string]string{"[abc]": `"1"`},
`category "[abc]" contains "["`,
},
"field name with a quote": {
map[string]string{"a": `{"format":"{x}","x":"1","b\"c":"2"}`},
`field "b\"c" contains "\""`,
},
// An empty name is not a path segment, so List never offered it — while a
// bare {}, a trailing dot in Fake("a.") and a {/a.} reference all reached
// it. The engine accepted spellings it would never advertise.
+37 -11
View File
@@ -57,6 +57,15 @@ type template struct {
func (*template) isNode() {}
// field is the node a path segment names; a binding is a render edge, not a field.
func (t *template) field(seg string) (node, bool) {
if isRef(seg) {
return nil, false
}
n, ok := t.fields[seg]
return n, ok
}
// compile converts parsed JSON into a node tree, validating structure up front.
// Only a choice's items carry a weight, so one here would be inert whatever its type.
func compile(v any) (node, error) {
@@ -78,10 +87,24 @@ func compileItem(v any) (node, error) {
case map[string]any:
return compileTemplate(v)
default:
return nil, fmt.Errorf("unsupported node type %T", v)
return nil, fmt.Errorf("a template value must be a string, a list or an object, not %s", jsonKind(v))
}
}
// jsonKind names a JSON value a template cannot hold, in the data format's own
// terms rather than the decoding library's.
func jsonKind(v any) string {
switch v.(type) {
case float64:
return "a number"
case bool:
return "a boolean"
case nil:
return "null"
}
return fmt.Sprintf("%T", v)
}
func compileString(s string) (node, error) {
if err := checkTokens(s, nil); err != nil {
return nil, err
@@ -229,6 +252,9 @@ func readOptions(m map[string]any) (templateOptions, error) {
if repeat == 1 {
return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1")
}
if o.separator == "" {
return o, fmt.Errorf("separator \"\" is the default, so it has no effect; drop it")
}
}
_, o.weighted = m["weight"]
return o, nil
@@ -276,8 +302,8 @@ func repeatOf(m map[string]any) (int, error) {
if r == 1 {
return 0, fmt.Errorf("repeat 1 is the default, so it has no effect; drop it")
}
if r > maxLen { // cap so a fat-fingered repeat can't build a multi-GB string
return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, maxLen)
if r > MaxRepeat { // caps the renders one repeat asks for; repeatCheck bounds what nested ones multiply to
return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, MaxRepeat)
}
return int(r), nil
}
@@ -311,23 +337,23 @@ func weightOf(raw any) (float64, error) {
// reservedInName is what a category, folder or field name may not contain: a dot
// separates the segments of a path, '|' the arms of a token, '(' opens a function
// call, braces delimit the token and '/' starts a reference. A name carrying one is
// reachable by no format, so it is rejected where it is authored rather than at
// the token that cannot reach it.
const reservedInName = ".|({}/"
// call, braces delimit the token, '/' starts a reference, and brackets and a quote
// open a JSON value. A name carrying one is rejected where it is authored rather
// than where it would be unreachable.
const reservedInName = ".|({}/[]\""
// reservedList spells reservedInName for an error message, so the two cannot drift.
var reservedList = strings.Join(strings.Split(reservedInName, ""), " ")
// checkName rejects a name the dot path and {token} grammars cannot spell. Both a
// category or folder and a field go through it, so there is one answer to what a
// name may contain.
// checkName rejects a name the dot path, {token} and JSON grammars cannot spell.
// Both a category or folder and a field go through it, so there is one answer to
// what a name may contain.
func checkName(name string) error {
if name == "" {
return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name)
}
if i := strings.IndexAny(name, reservedInName); i >= 0 {
return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path and {token} grammars reserve",
return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path, {token} and JSON grammars reserve",
name, name[i:i+1], reservedList)
}
return nil
+10
View File
@@ -47,3 +47,13 @@ func TestInertObjectIsRejected(t *testing.T) {
}
}
}
func TestInlineFolderSigilsAreRejected(t *testing.T) {
f := shipped(t)
for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} {
_, err := f.NewTemplate(input)
if err == nil || !strings.Contains(err.Error(), "write {/sv_SE.person.last}") {
t.Errorf("NewTemplate(%q) = %v, want an error naming the root spelling", input, err)
}
}
}
+1 -1
View File
@@ -40,7 +40,7 @@ func walkPath(n node, tail []string, w pathWalk) error {
return err
}
}
child, ok := n.fields[tail[0]]
child, ok := n.field(tail[0])
if !ok {
return fmt.Errorf("no field %q", tail[0])
}
+61
View File
@@ -0,0 +1,61 @@
package fejkdata
import (
"fmt"
"strings"
"testing"
"testing/fstest"
)
// One Fake call descends depth levels: each level's "a" is the next template down.
func nestedJSON(depth int) string {
s := `"leaf"`
for i := 0; i < depth; i++ {
s = fmt.Sprintf(`{"format":"{a}","a":%s}`, s)
}
return s
}
// One Fake call expands n sibling tokens.
func wideTokenJSON(n int) string {
var toks, fields strings.Builder
for i := 0; i < n; i++ {
fmt.Fprintf(&toks, "{f%d}", i)
if i > 0 {
fields.WriteByte(',')
}
fmt.Fprintf(&fields, `"f%d":"x"`, i)
}
return fmt.Sprintf(`{"format":"%s",%s}`, toks.String(), fields.String())
}
func TestNoRenderAllocRegression(t *testing.T) {
shapes := []struct {
name string
json string
base float64
}{
{"nested depth 25", nestedJSON(25), 27},
{"nested depth 100", nestedJSON(100), 102},
{"wide 500 tokens", wideTokenJSON(500), 9},
}
for _, s := range shapes {
f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"x.json": {Data: []byte(s.json)}}))
if err != nil {
t.Fatalf("New(%s): %v", s.name, err)
}
allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") })
if allocs > s.base*1.10 {
t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); bump the baseline only as a deliberate change", s.name, allocs, s.base*1.10, s.base)
}
}
}
func BenchmarkNestedDepth25(b *testing.B) { benchPath(b, tmpData(b, "deep", nestedJSON(25)), "deep") }
func BenchmarkNestedDepth100(b *testing.B) { benchPath(b, tmpData(b, "deep", nestedJSON(100)), "deep") }
func BenchmarkWideTokens100(b *testing.B) {
benchPath(b, tmpData(b, "wide", wideTokenJSON(100)), "wide")
}
func BenchmarkWideTokens500(b *testing.B) {
benchPath(b, tmpData(b, "wide", wideTokenJSON(500)), "wide")
}
+35 -28
View File
@@ -72,37 +72,44 @@ func refSegments(name string, folder []string) ([]string, error) {
// error, never a random render-time one.
func linkRefs(root map[string]node) error {
return eachTemplate(root, func(folder []string, path string, t *template) error {
names := refTokens(t.format)
if len(names) == 0 {
return nil
}
if t.fields == nil {
t.fields = map[string]node{}
}
t.refs = make(map[string]refBinding, len(names))
for _, name := range names {
segments, err := refSegments(name, folder)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
head, target, tail, err := resolveRef(root, segments)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
key := "/" + strings.Join(head, ".")
if err := checkPath(target, tail, key); err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
t.fields[key] = target
t.refs[name] = refBinding{key, tail}
}
if err := t.compileFormat(); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
return linkTemplateRefs(folder, path, t, root)
})
}
// linkTemplateRefs binds one template's references against root. A template with
// none is left untouched, so an inline format that references nothing costs only
// the refTokens scan.
func linkTemplateRefs(folder []string, path string, t *template, root map[string]node) error {
names := refTokens(t.format)
if len(names) == 0 {
return nil
}
if t.fields == nil {
t.fields = map[string]node{}
}
t.refs = make(map[string]refBinding, len(names))
for _, name := range names {
segments, err := refSegments(name, folder)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
head, target, tail, err := resolveRef(root, segments)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
key := "/" + strings.Join(head, ".")
if err := checkPath(target, tail, key); err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
t.fields[key] = target
t.refs[name] = refBinding{key, tail}
}
if err := t.compileFormat(); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
// eachTemplate calls fn once per template, with the folder its category sits in
// and the dot path reaching it, folders and names in sorted order.
func eachTemplate(root map[string]node, fn func(folder []string, path string, t *template) error) error {
+6
View File
@@ -108,6 +108,12 @@ func TestFakeIsSafeForConcurrentUse(t *testing.T) {
return
}
f.List()
tmpl, err := f.NewTemplate("{/sv_SE.person.last}")
if err != nil {
t.Error(err)
return
}
tmpl.Fake()
}
}()
}
+22 -2
View File
@@ -140,7 +140,7 @@ func checkTokens(format string, fields map[string]node) error {
}
continue // its target is checked at New (see linkRefs)
}
if err := checkArm(name, fields); err != nil {
if err := checkArm(name, fields, len(names) == 1); err != nil {
return fmt.Errorf("token {%s}: %w", t.body, err)
}
}
@@ -151,7 +151,9 @@ func checkTokens(format string, fields map[string]node) error {
}
// checkArm validates one sibling name or path against a template's fields.
func checkArm(name string, fields map[string]node) error {
// wholeToken says the name is the token's entire body, so {/name} would render
// the same value and can be offered as the reference spelling.
func checkArm(name string, fields map[string]node, wholeToken bool) error {
a := splitArm(name, nil)
if err := checkSegments(a); err != nil {
return err
@@ -164,6 +166,13 @@ func checkArm(name string, fields map[string]node) error {
if isOption(a.key) {
return fmt.Errorf("%q is an option and can never be a field", a.key)
}
if len(fields) == 0 {
hint := ""
if wholeToken && hintableRef(name) {
hint = fmt.Sprintf(" — write {/%s} to reference the data", name)
}
return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.key, hint)
}
return fmt.Errorf("no field %q", a.key)
}
if err := checkPath(head, a.tail, a.key); err != nil {
@@ -172,6 +181,17 @@ func checkArm(name string, fields map[string]node) error {
return nil
}
// hintableRef reports whether {/name} is a reference the grammar accepts, so the
// hint never names a spelling that fails too.
func hintableRef(name string) bool {
for _, seg := range strings.Split(name, ".") {
if checkName(seg) != nil {
return false
}
}
return true
}
// tokenOperands lists the fields one {token} body reads as operands, empty for a
// field token or a builtin that reads none.
func tokenOperands(body string) []string {
+21 -1
View File
@@ -8,7 +8,13 @@ import (
)
// engine builds a seeded generator with no loaded categories, for rendering tests.
func engine(seed uint64) *Generator { return &Generator{rand: newRand(seed, true)} }
func engine(seed uint64) *Generator {
s, err := newRand(seed, true)
if err != nil {
panic(err)
}
return &Generator{rand: s}
}
// parse unmarshals a JSON template fragment into its dynamic form.
func parse(t *testing.T, s string) any {
@@ -359,3 +365,17 @@ func TestAlternationThreeWay(t *testing.T) {
t.Fatalf("3-way alternation produced %v, want A, B and C", seen)
}
}
func TestArgErrorsNameTheSpelling(t *testing.T) {
for src, want := range map[string]string{
`"{float(1,2,02)}"`: "write 2",
`{"format":"{calc(a,02)}","a":"1"}`: "write 2",
`"{digits(+5)}"`: "write 5",
`"{hex(99999999999999999999)}"`: "exceeds the maximum",
`"{int(007,9)}"`: "write 7",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error saying %q", src, err, want)
}
}
}
+28
View File
@@ -0,0 +1,28 @@
# Release checklist
What to settle before the first tag, then the work that follows in a later,
data-heavy release.
## Before the first release — settle the record flag and API contract
- `--format` vocabulary — confirm `text`, `json`, `ndjson`, `csv`, `sql`; the
`json`-as-array vs `ndjson`-as-lines split; `--table` (the SQL INSERT target);
the `--separator` rejection on record formats; and the exit codes (misuse 2,
runtime 1).
- Library surface — confirm `Record`, `FakeRecord`, `NewRecordTemplate`,
`RecordTemplate`, `Column`/`Columns()`, and the `JSON()`, `CSVHeader()`,
`CSVLine()`, `SQLInsert()` serializers.
- Typed scalars — columns are strings today (`"42"`, quoted SQL). Confirm that
stays out of scope, or add a per-column `kind` before the tag.
- Struct-filling — `fake:"..."` tags (reflection over an arbitrary struct) stay
out of scope; `Columns()` hands the caller the values to map themselves.
Confirm.
- Independent reference draw — within one record every tailed reference to a
category is one draw, with no spelling for "these columns should disagree".
Confirm the per-record contract, or add the spelling.
## Later, in a data-heavy release
- Shipped-data de-duplication — `email.json`'s `local` is a drifted copy of
`username.json`; fold it in when the shipped set grows and we add lots more
data.