Compare commits

..

1 Commits

28 changed files with 218 additions and 1101 deletions
+1 -17
View File
@@ -16,28 +16,12 @@ jobs:
- uses: actions/checkout@v6 - uses: actions/checkout@v6
with: with:
persist-credentials: false persist-credentials: false
fetch-depth: 0
- name: Relevant changes
id: changes
run: |
if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then
git fetch --no-tags origin "$GITHUB_BASE_REF"
base="origin/$GITHUB_BASE_REF"
else
base='${{ github.event.before }}'
fi
if git diff --quiet "$base"...HEAD -- '*.go' go.mod go.sum Dockerfile .dockerignore .github data README.md; then
echo "code=false" >> "$GITHUB_OUTPUT"
else
echo "code=true" >> "$GITHUB_OUTPUT"
fi
# `docker build` streams the context to the daemon. A compose bind-mount of # `docker build` streams the context to the daemon. A compose bind-mount of
# `.` mounts an empty host dir instead, because the job is itself a container. # `.` mounts an empty host dir instead, because the job is itself a container.
- name: Latest supported Go - name: Latest supported Go
if: ${{ steps.changes.outputs.code == 'true' }}
run: docker build . run: docker build .
# Runs even when the step above failed, so a red build says whether the # Runs even when the step above failed, so a red build says whether the
# failure is version-specific. # failure is version-specific.
- name: Lowest supported Go - name: Lowest supported Go
if: ${{ !cancelled() && steps.changes.outputs.code == 'true' }} if: ${{ !cancelled() }}
run: docker build --build-arg GO_VERSION=1.22.12 . run: docker build --build-arg GO_VERSION=1.22.12 .
+2 -1
View File
@@ -1,3 +1,4 @@
.claude .claude
*.out *.out
__pycache__/ __pycache__/
todo.md
+8
View File
@@ -8,3 +8,11 @@
- A standing choice a reader would relitigate goes under Decisions in the README, not in a comment. - A standing choice a reader would relitigate goes under Decisions in the README, not in a comment.
- A README example is a `json` block that loads and renders as a category; `readme_test.go` runs every one. - A README example is a `json` block that loads and renders as a category; `readme_test.go` runs every one.
- Cyclomatic complexity is gated at 14: the table-shaped dispatches (`eachToken`, `calc.factor`, `walkPath`) sit at 1314 and stay whole; anything else that reaches 14 is decomposed. - Cyclomatic complexity is gated at 14: the table-shaped dispatches (`eachToken`, `calc.factor`, `walkPath`) sit at 1314 and stay whole; anything else that reaches 14 is decomposed.
# Deferred
- **Shipped-data de-duplication (2026-09-02).** `email.json`'s `local` is a
drifted copy of `username.json`, and no shipped file yet uses a held path, an
operand or a reference. Fixed in the data fill before the first tag, when the
shipped set is rewritten anyway; premise: nothing depends on the shipped data's
shape until then. Not raised in review before that.
+38 -119
View File
@@ -1,17 +1,32 @@
# fejkdata # fejkdata
Locale-aware fake data for tests and fixtures, generated from JSON templates. Use A Go library and CLI for generating locale-aware fake data from JSON templates.
it as a Go library or the CLI — no data on disk, no dependencies, and a seed makes output Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).
reproducible.
```sh ## Goals
go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest
fejkdata sv_SE.person # Sara Eriksson 1. **Valid by construction** — every value passes the check its real consumer
``` applies; facts that belong together come from one draw, within a value and
across categories.
2. **Text means what it says** — a format renders as written; only `{…}` varies,
random characters included (`{digits(3)}`). One spelling per result; the wrong
one is a load error naming the right one.
3. **Every mistake is a load error**`New` rejects; `Fake` on a loaded generator
fails only for an unknown path.
4. **Zero to a value in one command**`go install`, then `fejkdata sv_SE.person`:
no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position;
the first custom template needs no escape and no option.
5. **Data lives in JSON** — a builtin only for what data can't express.
6. **Reproducible** — seed in, same stream out; no builtin reads a clock.
7. **Zero dependencies** — standard library only.
8. **Docs index the grammar** — every syntax feature is a heading; every example
runs under test and shows its output; a rule is stated once.
## CLI ## CLI
```sh ```sh
go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest
fejkdata sv_SE.person # Sara Eriksson fejkdata sv_SE.person # Sara Eriksson
fejkdata sv_SE.person.last # Eriksson fejkdata sv_SE.person.last # Eriksson
fejkdata --seed 42 sv_SE.address # the same address every run fejkdata --seed 42 sv_SE.address # the same address every run
@@ -19,27 +34,17 @@ fejkdata -n 3 --separator ', ' sv_SE.word # nät, barn, sol
fejkdata --list # every path the data offers fejkdata --list # every path the data offers
fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data
fejkdata --no-shipped-data -d ./mydata --list # only your data fejkdata --no-shipped-data -d ./mydata --list # only your data
fejkdata 'name: {/sv_SE.person.last}' # name: <a surname> — an inline template
fejkdata '{"format":"name: {x}","x":["bosse","lina"]}' # name: bosse or name: lina
``` ```
A path names a category, or a field inside one: each dot segment descends one A path names a category, or a field inside one: each dot segment descends one
level — folders, then the category (a JSON file), then fields. An argument that is level — folders, then the category (a JSON file), then fields.
a JSON object, array or string, or that carries a `{` token, is instead an
**inline template**: a format string or a JSON value compiled and rendered on the
spot. Its tokens reach the data by reference from the root —
`{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike
available. An inline template sits in no folder, so the folder-relative `{.name}`
and `{..name}` are rejected naming the root spelling. A path never contains a
brace, a bracket or a quote, so the two cannot collide (see
[Decisions](#decisions)).
| Flag | | | Flag | |
|------|--| |------|--|
| `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash | | `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash |
| `--no-shipped-data` | load only the `--data-path` directories | | `--no-shipped-data` | load only the `--data-path` directories |
| `-s`, `--seed N` | reproducible output | | `-s`, `--seed N` | reproducible output |
| `-n`, `--repeat N` | render the value N times (up to 1048576), each an independent draw, streamed | | `-n`, `--repeat N` | render the path N times (up to 1048576), each an independent draw, streamed |
| `--separator S` | between repeated values (default a newline) | | `--separator S` | between repeated values (default a newline) |
| `--list` | print every path, then exit | | `--list` | print every path, then exit |
| `--version`, `-h`, `--help` | print, then exit | | `--version`, `-h`, `--help` | print, then exit |
@@ -47,9 +52,7 @@ brace, a bracket or a quote, so the two cannot collide (see
`--name value` and `--name=value` both work, a short flag's value attaches or `--name value` and `--name=value` both work, a short flag's value attaches or
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
[Decisions](#decisions); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing [Decisions](#decisions); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing
dir, unknown path), `2` misuse — a bad flag, an argument that names neither a dir, unknown path), `2` misuse. From a checkout: `go run ./cmd/fejkdata …`.
template nor a path, or an inline template that does not compile. From a checkout:
`go run ./cmd/fejkdata …`.
### Your own data ### Your own data
@@ -86,9 +89,6 @@ if err != nil {
} }
v, err := f.Fake("sv_SE.address") // "Kungsvägen 68\n379 17 Stockholm" v, err := f.Fake("sv_SE.address") // "Kungsvägen 68\n379 17 Stockholm"
paths := f.List() // every path Fake accepts, sorted paths := f.List() // every path Fake accepts, sorted
v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call
t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once
v = t.Fake() // render many times, no re-parse
``` ```
| Option | | | Option | |
@@ -110,8 +110,8 @@ work with no data on disk. A directory is a namespace: each JSON file is a
category named after the file, each subdirectory a dot-path segment, so category named after the file, each subdirectory a dot-path segment, so
`mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one. `mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one.
Sources merge in order; matching folders combine, any other clash is won by the Sources merge in order; matching folders combine, any other clash is won by the
last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`; last loaded. Names may not use `.`, `|`, `(`, `{`, `}` or `/`; dot-prefixed entries
dot-prefixed entries are skipped, so a data directory can also be a checkout. are skipped, so a data directory can also be a checkout.
Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`, Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`,
`person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`, `person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`,
@@ -180,9 +180,9 @@ many times — each an independent draw — joined by `separator` (default `""`)
{ "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] } { "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] }
``` ```
Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat`, Renders e.g. `bar foo baz`. A `separator` without a `repeat` is rejected, and so
a `separator` of `""` (the default), and a `repeat` that multiplies to more than is a `repeat` that multiplies to more than 1 048 576 renders along any path of
1 048 576 renders along any path of nested repeats. nested repeats.
### Options and fields ### Options and fields
@@ -194,11 +194,9 @@ choice naming its item.
### Functions ### Functions
A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad
count, range, country or expression fails fast; an integer is written plain count, range, country or expression fails fast, and a length, count or decimal
(`5`, not `+5` or `05`); bounds are finite; a sample that could only ever emit one place beyond a sane maximum is rejected, so a fat-fingered `hex(2000000000)`
value (`int(5,5)`, `float(1,1,2)`) is rejected naming the text to write instead; never tries to allocate gigabytes. Every builtin draws only from the seed — a
and a length, count or decimal place beyond a sane maximum is rejected, so a
fat-fingered `hex(2000000000)` never tries to allocate gigabytes. Every builtin draws only from the seed — a
time-based id takes its timestamp from the rng, not the clock — so seeded output time-based id takes its timestamp from the rng, not the clock — so seeded output
stays reproducible. stays reproducible.
@@ -245,10 +243,8 @@ hyphenated field can't be an operand.
``` ```
Renders e.g. `19.99 x 3 = 59.97`. An operand that can never be a number (`"abc"`, Renders e.g. `19.99 x 3 = 59.97`. An operand that can never be a number (`"abc"`,
or a choice of such) is rejected at load, as is a division by a constant zero or a choice of such) is rejected at load; one that sometimes is not yields `NaN`,
(`1/0`, or a fixed `"0"` field); an operand that sometimes is not a number yields and a division by zero `Inf` — both print rather than fail.
`NaN`, and a division by one that is not constant `Inf` — both print rather than
fail.
### Transforms ### Transforms
@@ -341,29 +337,6 @@ then costs about what its output costs: an unweighted pick is O(1) whatever the
list's length, a weighted one O(log n), and long formats, deep nesting and many list's length, a weighted one O(log n), and long formats, deep nesting and many
tokens add cost in proportion to the output. tokens add cost in proportion to the output.
## Goals
1. **Valid by construction** — every value passes the check its real consumer
applies; facts that belong together come from one draw, within a value and
across categories.
2. **Text means what it says** — a format renders as written; only `{…}` varies,
random characters included (`{digits(3)}`). One spelling per result; the wrong
one is a load error naming the right one.
3. **Every mistake is a load error**`New` rejects the data and `NewTemplate`
the inline template; on a loaded generator `Fake` fails only for an unknown
path, and `Template.Fake` cannot fail at all.
4. **Zero to a value in one command**`go install`, then `fejkdata sv_SE.person`:
no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position;
the first custom template needs no escape and no option.
5. **Data lives in JSON** — a builtin only for what data can't express.
6. **Reproducible** — seed in, same stream out; no builtin reads a clock.
7. **Zero dependencies** — standard library only.
8. **Docs index the grammar** — every syntax feature is a heading; every example
runs under test and shows its output; a rule is stated once.
9. **Fast enough to be free** — a value renders in about a microsecond and `New`
parses and validates the whole set once upfront, so generating fixtures stays
noise against a test's own runtime.
## Decisions ## Decisions
- **Options and fields share one namespace.** `format`, `weight`, `repeat` and - **Options and fields share one namespace.** `format`, `weight`, `repeat` and
@@ -379,33 +352,6 @@ tokens add cost in proportion to the output.
naming the double-dash spelling, and `-s=42` is rejected naming both short naming the double-dash spelling, and `-s=42` is rejected naming both short
spellings: `=` belongs to the long form, and reading `=42` as the value would spellings: `=` belongs to the long form, and reading `=42` as the value would
make `-d=./x` a directory named `=./x`. make `-d=./x` a directory named `=./x`.
- **An argument is a template by its shape, not by a flag.** A JSON object, array
or string, or a string carrying a `{` token, is an inline template; anything else
is a path. A name may not contain a brace, a bracket or a quote, so a path can
never collide with any of those spellings, and the leading `[` or `"` is gated on
valid JSON so a stray copied bracket never swallows an argument — it names
nothing, and says so. No `--template` flag is needed. Reserving the characters
whole — though only a leading one could collide — keeps one simple name rule
instead of a leading-position special case. The JSON string is what makes the
library's own advice reachable: the error for an object holding only a format
names `"…"`, and that spelling has to work where it is printed.
- **An inline template skips the cycle fence, and only that one.** `New` proves the
loaded tree acyclic, an inline node is a finite tree of its own, and nothing in
the tree can reference it, so no render of it reaches itself. Every other fence
runs over both, from one `checkScope`.
- **An inline template that does not compile is misuse (exit 2), including a
reference that resolves to nothing** — the whole argument is the spelling under
test, and `NewTemplate` compiles, links and validates as one step. An unknown
*path* stays a runtime error (exit 1): there the argument is well-formed and only
the data is absent.
- **A padded JSON argument is rejected, not trimmed.** Padding is the one place the
two readings disagree — a format string renders it, JSON drops it — so the
spelling that renders is named rather than silently chosen.
- **`FakeTemplate` and `NewTemplate` both stay.** They reach the same value but not
at the same cost: `NewTemplate` pays the compile and validation once and renders
many times, `FakeTemplate` is the one-shot call, and `--repeat` is exactly the
case that needs the first. The pair is `regexp.MustCompile` and `regexp.Match`,
not two spellings of one result.
- **The shipped data is embedded, not discovered.** A directory a machine happens - **The shipped data is embedded, not discovered.** A directory a machine happens
to have would make `--seed 42` machine-dependent. Data still lives in `data/` to have would make `--seed 42` machine-dependent. Data still lives in `data/`
as JSON; `--data-path` layers over it. as JSON; `--data-path` layers over it.
@@ -433,33 +379,9 @@ tokens add cost in proportion to the output.
when that file lacks a field those references read. Accepted: overriding is the when that file lacks a field those references read. Accepted: overriding is the
point of layering, the error names the reference and the field, and the fix is point of layering, the error names the reference and the field, and the fix is
the consumer's file carrying the fields the shipped tree reads. the consumer's file carrying the fields the shipped tree reads.
- **The repeat cap bounds renders, not bytes.** A repeat, alone or nested, may
ask for at most 1 048 576 renders; how large each render is stays what the data
asked for, so `{hex(1048576)}` repeated to the cap is a terabyte, loaded without
complaint. A byte estimate would need every builtin to declare a width to fence
a shape no data comes near, and the harm lands on the author who wrote it.
- **64-bit targets only.** The gate builds amd64, and the buffer sizing a render
pre-computes (renders × bytes) assumes a 64-bit int; on a 32-bit target it could
overflow and panic.
- **A constant zero divisor is a load error; a divisor that is not constant prints
`Inf`.** `1/0` and a fixed `"0"` field are decidable, so they join the
never-numeric operand as a load error; the fold stops where an operand varies,
so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every
draw — catching it needs zero-absorbing algebra for a shape nobody writes.
- **In data, a default written out and a constant spelled as a sample are load
errors.** `weight: 1`, `repeat: 1`, `separator: ""`, `int(5,5)`, `float(1,1,2)`,
`+5` and `05` each spell what a shorter form already spells, so each is rejected
naming that form. The CLI's numbers follow the shell instead: `--seed 007` and
`--repeat +3` are 7 and 3, as every command line reads them.
- **Samples say what they emit, transforms what they do.** `{upper(2)}` is two - **Samples say what they emit, transforms what they do.** `{upper(2)}` is two
letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on
whether the argument looks like a number. whether the argument looks like a number.
- **The performance gate asserts allocations, not wall-clock time.** `AllocsPerRun`
is deterministic across machines, so a ±10% ceiling does not flake under CI load,
while time varies with the machine and its neighbours. A rendering slowdown
almost always costs an allocation too (a lost pre-size, a per-item map, an extra
copy). The benchmark suite (see Development) reports time for a human, not as a
pass/fail gate.
## Development ## Development
@@ -484,10 +406,8 @@ docker compose run --rm --user "$(id -u):$(id -g)" tidy # go mod tidy
``` ```
Every pull request runs `docker build .` against both the latest and the lowest Every pull request runs `docker build .` against both the latest and the lowest
supported Go, and must pass before it can be merged — unless it changes none of supported Go, and must pass before it can be merged. That build is the whole
the files the build and its tests read, nor the workflow itself, in which case gate — vet, complexity, format check and tests — so run it locally before pushing:
it's skipped (see [Decisions](#decisions)). That build is the whole gate — vet,
complexity, format check and tests — so run it locally before pushing:
```sh ```sh
docker build . # latest docker build . # latest
@@ -502,7 +422,6 @@ fejkdata.go Generator, New, options, the embedded data set, List
node.go the node model and JSON -> node compilation node.go the node model and JSON -> node compilation
path.go the dotted-path walk, and proving a path resolves path.go the dotted-path walk, and proving a path resolves
render.go Fake and the recursive renderer (choices, format strings, expansions) render.go Fake and the recursive renderer (choices, format strings, expansions)
inline.go inline templates: Template, NewTemplate, FakeTemplate, and their compile and link
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
hold.go the hold: one draw per expansion for paths and operands, and its fences hold.go the hold: one draw per expansion for paths and operands, and its fences
reference.go reference sigils, and binding references across the tree reference.go reference sigils, and binding references across the tree
@@ -516,4 +435,4 @@ data/ shipped data (JSON), embedded at build: locale folders + a misc
## License ## License
MIT — see [LICENSE](LICENSE). Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes). MIT — see [LICENSE](LICENSE).
+20 -57
View File
@@ -2,7 +2,6 @@ package fejkdata
import ( import (
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"math" "math"
"strconv" "strconv"
@@ -10,12 +9,12 @@ import (
"unicode" "unicode"
) )
// maxLen caps sample output lengths (hex, nanoid, base64, digits, upper, lower) // maxLen caps sample output lengths (hex, nanoid, base64) and the renders a repeat
// and maxDecimals float/calc decimal places, so a fat-fingered or overflowing // multiplies to along any path; maxDecimals caps float/calc decimal places. So a
// argument fails at New instead of trying to allocate gigabytes — or panicking — // fat-fingered or overflowing argument fails at New instead of trying to allocate
// at render. // gigabytes — or panicking — at render.
const ( const (
maxLen = 1 << 20 maxLen = MaxRepeat
maxDecimals = 1024 maxDecimals = 1024
) )
@@ -133,7 +132,7 @@ func transformArg(fields map[string]node, a []string) error {
_, _, err := refShape(leaf) _, _, err := refShape(leaf)
return err return err
} }
return checkArm(leaf, fields, false) return checkArm(leaf, fields)
} }
func transformOperand(a []string) []string { func transformOperand(a []string) []string {
@@ -177,7 +176,7 @@ var asciiFolds = map[rune]string{
func asciiFold(s string) string { func asciiFold(s string) string {
var b strings.Builder var b strings.Builder
for _, r := range s { for _, r := range s {
if r <= unicode.MaxASCII { if r < unicode.MaxASCII {
b.WriteRune(r) b.WriteRune(r)
} else { } else {
b.WriteString(asciiFolds[r]) b.WriteString(asciiFolds[r])
@@ -222,31 +221,10 @@ func randChars(r rng, n int, alphabet string) string {
return string(b) return string(b)
} }
// plainInt parses an integer arg written the one way: no sign, no leading zero.
func plainInt(s string) (int, error) {
n, err := strconv.Atoi(s)
if errors.Is(err, strconv.ErrRange) {
return 0, fmt.Errorf("%q is past the integer range: %w", s, err)
}
if err != nil {
return 0, fmt.Errorf("%q is not an integer", s)
}
if strconv.Itoa(n) != s {
return 0, fmt.Errorf("%q is not a plain integer; write %d", s, n)
}
return n, nil
}
func posIntArg(_ map[string]node, a []string) error { func posIntArg(_ map[string]node, a []string) error {
n, err := plainInt(a[0]) n, err := strconv.Atoi(a[0])
if errors.Is(err, strconv.ErrRange) { if err != nil || n < 1 {
return fmt.Errorf("count %q exceeds the maximum %d", a[0], maxLen) return fmt.Errorf("count %q must be a positive integer", a[0])
}
if err != nil {
return fmt.Errorf("count %w", err)
}
if n < 1 {
return fmt.Errorf("count %q must be positive", a[0])
} }
if n > maxLen { if n > maxLen {
return fmt.Errorf("count %d exceeds the maximum %d", n, maxLen) return fmt.Errorf("count %d exceeds the maximum %d", n, maxLen)
@@ -255,20 +233,14 @@ func posIntArg(_ map[string]node, a []string) error {
} }
func intRangeArgs(_ map[string]node, a []string) error { func intRangeArgs(_ map[string]node, a []string) error {
lo, err := plainInt(a[0]) lo, e1 := strconv.Atoi(a[0])
if err != nil { hi, e2 := strconv.Atoi(a[1])
return fmt.Errorf("int(min,max): min %w", err) if e1 != nil || e2 != nil {
} return fmt.Errorf("int(min,max) needs integer args, got %q,%q", a[0], a[1])
hi, err := plainInt(a[1])
if err != nil {
return fmt.Errorf("int(min,max): max %w", err)
} }
if lo > hi { if lo > hi {
return fmt.Errorf("int(min,max): min %d > max %d", lo, hi) return fmt.Errorf("int(min,max): min %d > max %d", lo, hi)
} }
if lo == hi {
return fmt.Errorf("int(%d,%d) is the constant %d; write it as text", lo, hi, lo)
}
if uint64(hi)-uint64(lo) >= uint64(math.MaxInt64) { // span hi-lo+1 would overflow int -> IntN panic if uint64(hi)-uint64(lo) >= uint64(math.MaxInt64) { // span hi-lo+1 would overflow int -> IntN panic
return fmt.Errorf("int(min,max): range %d..%d is too wide", lo, hi) return fmt.Errorf("int(min,max): range %d..%d is too wide", lo, hi)
} }
@@ -278,28 +250,19 @@ func intRangeArgs(_ map[string]node, a []string) error {
func floatArgs(_ map[string]node, a []string) error { func floatArgs(_ map[string]node, a []string) error {
lo, e1 := strconv.ParseFloat(a[0], 64) lo, e1 := strconv.ParseFloat(a[0], 64)
hi, e2 := strconv.ParseFloat(a[1], 64) hi, e2 := strconv.ParseFloat(a[1], 64)
if e1 != nil || e2 != nil { dp, e3 := strconv.Atoi(a[2])
return fmt.Errorf("float(min,max,dp) needs numeric bounds, got %q,%q", a[0], a[1]) if e1 != nil || e2 != nil || e3 != nil {
} return fmt.Errorf("float(min,max,dp) needs numeric args, got %q,%q,%q", a[0], a[1], a[2])
dp, err := plainInt(a[2])
if err != nil {
return fmt.Errorf("float(min,max,dp): decimals %w", err)
}
if math.IsNaN(lo) || math.IsNaN(hi) || math.IsInf(lo, 0) || math.IsInf(hi, 0) {
return fmt.Errorf("float(min,max,dp) needs finite bounds, got %q,%q", a[0], a[1])
} }
if lo > hi { if lo > hi {
return fmt.Errorf("float(min,max,dp): min %v > max %v", lo, hi) return fmt.Errorf("float(min,max,dp): min %v > max %v", lo, hi)
} }
if dp < 0 || dp > maxDecimals {
return fmt.Errorf("float(min,max,dp): decimals %d out of range 0..%d", dp, maxDecimals)
}
if lo == hi {
return fmt.Errorf("float(%s,%s,%d) is the constant %q; write it as text", a[0], a[1], dp, strconv.FormatFloat(lo, 'f', dp, 64))
}
if math.IsInf(hi-lo, 0) { // an overflowing span would render as "+Inf" if math.IsInf(hi-lo, 0) { // an overflowing span would render as "+Inf"
return fmt.Errorf("float(min,max,dp): range %v..%v is too wide", lo, hi) return fmt.Errorf("float(min,max,dp): range %v..%v is too wide", lo, hi)
} }
if dp < 0 || dp > maxDecimals {
return fmt.Errorf("float(min,max,dp): decimals %d out of range 0..%d", dp, maxDecimals)
}
return nil return nil
} }
+2 -70
View File
@@ -78,82 +78,14 @@ func checkCalc(fields map[string]node, args []string) error {
return fmt.Errorf("calc(%q): operand %q is never a number: it renders %q", args[0], name, text) return fmt.Errorf("calc(%q): operand %q is never a number: it renders %q", args[0], name, text)
} }
} }
if divisor, zero := constantZeroDivisor(expr, fields); zero {
return fmt.Errorf("calc(%q) divides by %s, which is always zero", args[0], divisor)
}
if len(args) == 2 { if len(args) == 2 {
dp, err := plainInt(args[1]) if dp, err := strconv.Atoi(args[1]); err != nil || dp < 0 || dp > maxDecimals {
if err != nil { return fmt.Errorf("calc decimals %q must be an integer in 0..%d", args[1], maxDecimals)
return fmt.Errorf("calc decimals %w", err)
}
if dp < 0 || dp > maxDecimals {
return fmt.Errorf("calc decimals %d must be in 0..%d", dp, maxDecimals)
} }
} }
return nil return nil
} }
// constantZeroDivisor finds a division whose right side is a constant zero: number
// literals and fixed operands folded, anything that varies left unknown.
func constantZeroDivisor(n calcNode, fields map[string]node) (string, bool) {
switch n := n.(type) {
case calcNeg:
return constantZeroDivisor(n.x, fields)
case calcBin:
if n.op == '/' {
if v, known := constantValue(n.r, fields); known && v == 0 {
return calcText(n.r), true
}
}
if d, zero := constantZeroDivisor(n.l, fields); zero {
return d, true
}
return constantZeroDivisor(n.r, fields)
}
return "", false
}
// constantValue evaluates an expression whose every operand is fixed.
func constantValue(n calcNode, fields map[string]node) (float64, bool) {
switch n := n.(type) {
case calcNum:
return float64(n), true
case calcVar:
t, ok := fields[string(n)].(*template)
if !ok || !t.fixed || t.repeat > 1 {
return 0, false
}
v, err := strconv.ParseFloat(strings.TrimSpace(t.lit), 64)
return v, err == nil
case calcNeg:
v, ok := constantValue(n.x, fields)
return -v, ok
case calcBin:
l, lok := constantValue(n.l, fields)
r, rok := constantValue(n.r, fields)
if !lok || !rok {
return 0, false
}
return calcBin{n.op, calcNum(l), calcNum(r)}.eval(nil), true
}
return 0, false
}
// calcText spells an expression node the way an author would read it.
func calcText(n calcNode) string {
switch n := n.(type) {
case calcNum:
return strconv.FormatFloat(float64(n), 'f', -1, 64)
case calcVar:
return string(n)
case calcNeg:
return "-" + calcText(n.x)
case calcBin:
return "(" + calcText(n.l) + " " + string(n.op) + " " + calcText(n.r) + ")"
}
return "?"
}
// neverNumeric reports a node no render of which is a number: fixed text that does // neverNumeric reports a node no render of which is a number: fixed text that does
// not parse, or a choice of only such items. text is one such render. // not parse, or a choice of only such items. text is one such render.
func neverNumeric(n node) (text string, never bool) { func neverNumeric(n node) (text string, never bool) {
+3 -13
View File
@@ -247,21 +247,11 @@ func TestCalcOverANeverNumericOperandIsRejected(t *testing.T) {
} }
func TestCalcConstantZeroDivisorIsRejected(t *testing.T) { func TestCalcConstantZeroDivisorIsRejected(t *testing.T) {
for src, want := range map[string]string{ for _, bad := range []string{`"{calc(1/0)}"`, `"{calc(2/(1-1))}"`, `{"format":"{calc(x/y)}","x":"1","y":"0"}`, `{"format":"{calc(x/(y*2))}","x":"1","y":" 0 "}`} {
`"{calc(1/0)}"`: "divides by 0", if _, err := compile(parse(t, bad)); err == nil || !strings.Contains(err.Error(), "zero") {
`"{calc(2/(1-1))}"`: "divides by (1 - 1)", t.Errorf("compile(%s) = %v, want the constant zero divisor rejected", bad, err)
`{"format":"{calc(x/y)}","x":"1","y":"0"}`: "divides by y",
`{"format":"{calc(x/(y*2))}","x":"1","y":" 0 "}`: "divides by (y * 2)",
`{"format":"{calc((a/0)+b)}","a":"1","b":"2"}`: "divides by 0",
`{"format":"{calc(a/-0)}","a":"1"}`: "divides by -0",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want the constant zero divisor rejected naming %q", src, err, want)
} }
} }
if _, err := compile(parse(t, `{"format":"{calc(a/(b*c))}","a":"1","b":"0","c":["1","2"]}`)); err != nil {
t.Errorf("compile(a/(b*c)) = %v, want a divisor that varies accepted", err)
}
f := engine(1) f := engine(1)
for i := 0; i < 50; i++ { for i := 0; i < 50; i++ {
if got := mustRender(t, f, `{"format":"{calc(x/y)}","x":"1","y":["0","1"]}`); got == "+Inf" { if got := mustRender(t, f, `{"format":"{calc(x/y)}","x":"1","y":["0","1"]}`); got == "+Inf" {
+19 -79
View File
@@ -9,7 +9,6 @@ package main
import ( import (
"bufio" "bufio"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -21,30 +20,21 @@ import (
"gitea.larvit.se/larvit/fejkdata" "gitea.larvit.se/larvit/fejkdata"
) )
const usage = `Usage: fejkdata [flags] <path|template> const usage = `Usage: fejkdata [flags] <path>
<path> a category, or a dotted path into one (person, person.last) <path> a category, or a dotted path into one (person, person.last)
<template> a format string or JSON value to render inline, e.g.
'name: {/sv_SE.person.last}' or '{"format":"{x}","x":["bosse","lina"]}'
An argument containing a { token, or a JSON object, array or string, is a
template; any other argument is a path (a path never contains a brace, a bracket
or a quote). Templates reach the data by reference from the root —
{/sv_SE.person.last} — whether the data is shipped or layered with --data-path. An
argument carrying a bracket, a closing brace or a quote but no valid JSON names
neither.
-d, --data-path D a data directory to layer over the shipped data (repeatable; last wins on a clash) -d, --data-path D a data directory to layer over the shipped data (repeatable; last wins on a clash)
-h, --help print this help, then exit -h, --help print this help, then exit
--list list the paths the data offers, then exit --list list the paths the data offers, then exit
--no-shipped-data load only the --data-path directories --no-shipped-data load only the --data-path directories
-n, --repeat N render the value N times, 1..1048576 (default 1) -n, --repeat N render the path N times, 1..1048576 (default 1)
-s, --seed N seed for reproducible output -s, --seed N seed for reproducible output
--separator S string between repeated values (default newline) --separator S string between repeated values (default newline)
--version print the version, then exit --version print the version, then exit
Flags may come before or after <path|template>; -- ends the flags. A short flag's Flags may come before or after <path>; -- ends the flags. A short flag's value
value attaches or follows (-n3, -n 3); short flags bundle (-hn 3). attaches or follows (-n3, -n 3); short flags bundle (-hn 3).
` `
type invocation struct { type invocation struct {
@@ -196,22 +186,18 @@ func parseArgs(argv []string) (invocation, error) {
return in, nil return in, nil
} }
// check rejects a flag combination or an argument that cannot run, and reports // check rejects a flag combination that cannot run.
// what the argument names, so its shape is settled before any data is read. func (in invocation) check() error {
func (in invocation) check() (argKind, error) {
if in.list && len(in.paths) > 0 { if in.list && len(in.paths) > 0 {
return argPath, errors.New("--list takes no path") return errors.New("--list takes no path")
} }
if in.list && (in.repeatSet || in.separatorSet) { if in.list && (in.repeatSet || in.separatorSet) {
return argPath, errors.New("--list takes no --repeat or --separator") return errors.New("--list takes no --repeat or --separator")
} }
if in.list { if !in.list && len(in.paths) != 1 {
return argPath, nil return fmt.Errorf("expected one path, got %d", len(in.paths))
} }
if len(in.paths) != 1 { return nil
return argPath, fmt.Errorf("expected one path or template, got %d", len(in.paths))
}
return classify(in.paths[0])
} }
func (in invocation) options() []fejkdata.Option { func (in invocation) options() []fejkdata.Option {
@@ -228,23 +214,13 @@ func (in invocation) options() []fejkdata.Option {
return opts return opts
} }
// write streams the argument's renders to w, repeat of them joined by the // write streams the path's renders to w, repeat of them joined by the separator
// separator and ended by a newline. A value that renders once renders every time, // and ended by a newline. A path that renders once renders every time, so the only
// so a render failure comes before anything is written; a write failure surfaces // failure comes before anything is written.
// from Flush, bufio keeping the first one. func (in invocation) write(f *fejkdata.Generator, w io.Writer) error {
func (in invocation) write(f *fejkdata.Generator, kind argKind, w io.Writer) error {
arg := in.paths[0]
draw := func() (string, error) { return f.Fake(arg) }
if kind == argTemplate {
t, err := f.NewTemplate(arg)
if err != nil {
return templateError{err}
}
draw = func() (string, error) { return t.Fake(), nil }
}
out := bufio.NewWriter(w) out := bufio.NewWriter(w)
for i := 0; i < in.repeat; i++ { for i := 0; i < in.repeat; i++ {
v, err := draw() v, err := f.Fake(in.paths[0])
if err != nil { if err != nil {
return err return err
} }
@@ -257,36 +233,6 @@ func (in invocation) write(f *fejkdata.Generator, kind argKind, w io.Writer) err
return out.Flush() return out.Flush()
} }
// templateError marks a render failure that is the argument's own fault — an
// inline template that does not compile. run reports it as misuse (exit 2, with a
// pointer to --help), unlike an unknown path, which is a runtime error (exit 1).
type templateError struct{ error }
func (e templateError) Unwrap() error { return e.error }
type argKind int
const (
argPath argKind = iota
argTemplate
)
// classify reads what a positional argument names by its shape: a { token, or a
// JSON object, array or string, is an inline template; anything else is a path.
func classify(arg string) (argKind, error) {
if strings.ContainsRune(arg, '{') || (isJSONStart(strings.TrimSpace(arg)) && json.Valid([]byte(arg))) {
return argTemplate, nil
}
if i := strings.IndexAny(arg, `[]}"`); i >= 0 {
return argPath, fmt.Errorf("%q holds a %q, which no path may, and it is not valid JSON, so it names no template either", arg, arg[i:i+1])
}
return argPath, nil
}
func isJSONStart(arg string) bool {
return strings.HasPrefix(arg, "[") || strings.HasPrefix(arg, `"`)
}
func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) }
// run returns the exit code: 0 ok, 1 runtime error, 2 misuse. // run returns the exit code: 0 ok, 1 runtime error, 2 misuse.
@@ -303,8 +249,7 @@ func run(args []string, stdout, stderr io.Writer) int {
fmt.Fprintln(stdout, "fejkdata "+buildVersion()) fmt.Fprintln(stdout, "fejkdata "+buildVersion())
return 0 return 0
} }
kind, err := in.check() if err := in.check(); err != nil {
if err != nil {
return misuse(stderr, err) return misuse(stderr, err)
} }
f, err := fejkdata.New(in.options()...) f, err := fejkdata.New(in.options()...)
@@ -321,11 +266,7 @@ func run(args []string, stdout, stderr io.Writer) int {
} }
return 0 return 0
} }
if err := in.write(f, kind, stdout); err != nil { if err := in.write(f, stdout); err != nil {
var te templateError
if errors.As(err, &te) {
return misuse(stderr, te.error)
}
fmt.Fprintln(stderr, err) fmt.Fprintln(stderr, err)
return 1 return 1
} }
@@ -333,8 +274,7 @@ func run(args []string, stdout, stderr io.Writer) int {
} }
func misuse(stderr io.Writer, err error) int { func misuse(stderr io.Writer, err error) int {
// A library error already names the program, so the prefix is not doubled. fmt.Fprintf(stderr, "fejkdata: %v\ntry 'fejkdata --help'\n", err)
fmt.Fprintf(stderr, "fejkdata: %s\ntry 'fejkdata --help'\n", strings.TrimPrefix(err.Error(), "fejkdata: "))
return 2 return 2
} }
-102
View File
@@ -4,7 +4,6 @@ import (
"bytes" "bytes"
"os" "os"
"path/filepath" "path/filepath"
"regexp"
"strings" "strings"
"testing" "testing"
) )
@@ -302,92 +301,6 @@ func TestRunShippedDataByDefault(t *testing.T) {
} }
} }
func TestClassify(t *testing.T) {
for arg, want := range map[string]argKind{
"sv_SE.person": argPath,
"person.last": argPath,
"name: {x}": argTemplate, // a { token: a path can never carry a brace
`{"format":"x"}`: argTemplate,
`["a","b"]`: argTemplate, // a JSON array carries no brace
`[1, 2]`: argTemplate,
` ["a","b"]`: argTemplate, // padding is the template's own error, not a shape verdict
`"hello"`: argTemplate, // a JSON string, the spelling a format-only object names
} {
got, err := classify(arg)
if err != nil || got != want {
t.Errorf("classify(%q) = %v, %v; want %v", arg, got, err, want)
}
}
for arg, want := range map[string]string{
"[abc]": `holds a "["`,
"[abc].field": `holds a "["`,
"x[1]": `holds a "["`,
"a]b": `holds a "]"`,
"a}b": `holds a "}"`,
`"abc`: `holds a "\""`,
`"a]b`: `holds a "\""`, // the opener the reader typed, not the bracket behind it
} {
_, err := classify(arg)
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("classify(%q) = %v; want it rejected naming %s", arg, err, want)
}
}
}
func TestUsageReferencesResolve(t *testing.T) {
for _, token := range regexp.MustCompile(`\{/[^}]+\}`).FindAllString(usage, -1) {
code, out, errb := runOut("--seed", "1", token)
if code != 0 || strings.TrimSpace(out) == "" {
t.Errorf("usage advertises %s: run = %d, %q, stderr %q", token, code, out, errb)
}
}
}
func TestRunShapeMisuseBeforeLoad(t *testing.T) {
code, _, errb := runOut("--no-shipped-data", "[abc]")
if code != 2 || !strings.Contains(errb, "[abc]") || strings.Contains(errb, "--data-path") {
t.Fatalf("shape misuse with no data = %d, %q; want the shape error before any load", code, errb)
}
}
func TestRunInlineTemplate(t *testing.T) {
code, out, errb := runOut("--seed", "1", "name: {/sv_SE.person.last}")
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline format string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", `{"format":"name: {x}","x":["bosse","lina"]}`)
if code != 0 || (out != "name: bosse\n" && out != "name: lina\n") {
t.Fatalf("inline JSON template = %d, %q, want one name, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", `"name: {/sv_SE.person.last}"`)
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline JSON string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", "-n", "2", `{digits(1)}`)
if code != 0 || len(strings.Split(strings.TrimRight(out, "\n"), "\n")) != 2 {
t.Fatalf("inline template with --repeat = %d, %q, stderr %q", code, out, errb)
}
}
func TestRunTemplateMisuse(t *testing.T) {
for arg, want := range map[string]string{
"{bad": "unterminated",
"[red,green]": "names no template either",
`{"format":"x"}`: "is a string",
"{/no.such.path}": "no entry",
"x[1]": "names no template either",
` ["a","b"] `: "may not be padded",
} {
code, out, errb := runOut("--seed", "1", arg)
if code != 2 || out != "" || !strings.Contains(errb, "try 'fejkdata --help'") || !strings.Contains(errb, want) {
t.Errorf("run(%q) = %d, %q, %q; want misuse naming %q and --help", arg, code, out, errb, want)
}
if strings.Contains(errb, "fejkdata: fejkdata:") {
t.Errorf("run(%q) doubled the program prefix: %q", arg, errb)
}
}
}
func TestRunNoShippedData(t *testing.T) { func TestRunNoShippedData(t *testing.T) {
code, list, errb := runOut("--no-shipped-data", "-d", svSE, "--list") code, list, errb := runOut("--no-shipped-data", "-d", svSE, "--list")
if code != 0 { if code != 0 {
@@ -442,18 +355,3 @@ func TestRunUnknownFlagIsNamedByRune(t *testing.T) {
t.Errorf("run(-ä) = %d, %q, want the flag named whole", code, errb) t.Errorf("run(-ä) = %d, %q, want the flag named whole", code, errb)
} }
} }
func TestRunEmptyDataPathIsNamed(t *testing.T) {
code, _, errb := runOut("--data-path=", "sv_SE.word")
if code != 1 || !strings.Contains(errb, "empty") {
t.Errorf("run(--data-path=) = %d, %q, want the empty path named", code, errb)
}
}
func TestRunNumbersFollowTheShell(t *testing.T) {
_, want, _ := runOut("--seed", "7", "--repeat", "3", "sv_SE.word")
code, got, errb := runOut("--seed", "007", "--repeat", "+3", "sv_SE.word")
if code != 0 || got != want {
t.Errorf("run(--seed 007 --repeat +3) = %d, %q, stderr %q; want the same as --seed 7 --repeat 3 %q", code, got, errb, want)
}
}
+12 -13
View File
@@ -10,14 +10,13 @@ import (
) )
// dataSource is one tree to load: an fs.FS and the directory in it to start from. // dataSource is one tree to load: an fs.FS and the directory in it to start from.
// label prefixes file names in errors; onDisk marks path as a directory that must // label prefixes file names in errors; path, when set, is a directory on disk that
// exist. // must exist.
type dataSource struct { type dataSource struct {
fsys fs.FS fsys fs.FS
label string label string
onDisk bool path string
path string root string
root string
} }
func (s dataSource) name(p string) string { func (s dataSource) name(p string) string {
@@ -37,13 +36,10 @@ func (s dataSource) name(p string) string {
func loadData(sources []dataSource) (map[string]node, error) { func loadData(sources []dataSource) (map[string]node, error) {
root := map[string]node{} root := map[string]node{}
for _, src := range sources { for _, src := range sources {
if src.onDisk { if src.path != "" {
if src.path == "" {
return nil, fmt.Errorf("a data path is empty")
}
info, err := os.Stat(src.path) info, err := os.Stat(src.path)
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("%s: no such directory", src.path)
} }
if !info.IsDir() { if !info.IsDir() {
return nil, fmt.Errorf("%s is not a directory", src.path) return nil, fmt.Errorf("%s is not a directory", src.path)
@@ -68,7 +64,10 @@ func loadData(sources []dataSource) (map[string]node, error) {
if err := checkNoCycles(root); err != nil { if err := checkNoCycles(root); err != nil {
return nil, err return nil, err
} }
if err := checkScope(treeScope(root)); err != nil { if err := checkRepeatReach(root); err != nil {
return nil, err
}
if err := checkBoundLevelsHeld(root); err != nil {
return nil, err return nil, err
} }
return root, nil return root, nil
+9 -24
View File
@@ -28,19 +28,15 @@ import (
//go:embed data //go:embed data
var shippedFS embed.FS var shippedFS embed.FS
// MaxRepeat caps a repeat, and the renders nested repeats multiply to along any // MaxRepeat caps a repeat, and the renders nested repeats multiply to along any path.
// path; the CLI's --repeat shares it.
const MaxRepeat = 1 << 20 const MaxRepeat = 1 << 20
var _ [^uint(0)>>63 - 1]struct{} // 64-bit only, per the README's Decisions
// ErrNoData is returned by New when no source is loaded at all. // ErrNoData is returned by New when no source is loaded at all.
var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS") var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS")
// Generator generates fake data from a loaded namespace tree. Create one with [New]. // Generator generates fake data from a loaded namespace tree. Create one with [New].
// It is safe for concurrent use; a seeded sequence is reproducible only when drawn // It is safe for concurrent use; a seeded sequence is reproducible only when drawn
// from one goroutine. The compiled tree is immutable after [New], and Fake, // from one goroutine.
// NewTemplate and List read it concurrently without a lock.
type Generator struct { type Generator struct {
mu sync.Mutex mu sync.Mutex
rand *session rand *session
@@ -79,7 +75,7 @@ func WithSeed(seed uint64) Option {
// layer several; the last wins a name clash. // layer several; the last wins a name clash.
func WithDataPath(dir string) Option { func WithDataPath(dir string) Option {
return func(c *config) { return func(c *config) {
c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, onDisk: true, path: dir}) c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, path: dir})
} }
} }
@@ -116,11 +112,7 @@ func New(opts ...Option) (*Generator, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err) return nil, fmt.Errorf("fejkdata: %w", err)
} }
rng, err := newRand(c.seed, c.seeded) return &Generator{rand: newRand(c.seed, c.seeded), categories: cats}, nil
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Generator{rand: rng, categories: cats}, nil
} }
// List returns the sorted dotted paths Fake can render: every category, the dotted // List returns the sorted dotted paths Fake can render: every category, the dotted
@@ -211,19 +203,12 @@ func join(prefix, name string) string {
return prefix + "." + name return prefix + "." + name
} }
// randomBytes seeds an unseeded generator. func newRand(seed uint64, seeded bool) *session {
var randomBytes = crand.Read r := rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15))
if !seeded {
func newRand(seed uint64, seeded bool) (*session, error) {
var r *rand.Rand
if seeded {
r = rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15))
} else {
var b [16]byte var b [16]byte
if _, err := randomBytes(b[:]); err != nil { _, _ = crand.Read(b[:])
return nil, fmt.Errorf("seeding from crypto/rand: %w", err)
}
r = rand.New(rand.NewPCG(binary.LittleEndian.Uint64(b[:8]), binary.LittleEndian.Uint64(b[8:]))) r = rand.New(rand.NewPCG(binary.LittleEndian.Uint64(b[:8]), binary.LittleEndian.Uint64(b[8:])))
} }
return &session{Rand: r, counters: map[string]uint64{}}, nil return &session{Rand: r, counters: map[string]uint64{}}
} }
+2 -22
View File
@@ -1,11 +1,8 @@
package fejkdata package fejkdata
import ( import (
"errors"
"io/fs"
"strings" "strings"
"testing" "testing"
"testing/fstest"
) )
// newGenerator creates a generator over a single data directory, failing on // newGenerator creates a generator over a single data directory, failing on
@@ -41,25 +38,8 @@ func fake(t *testing.T, f *Generator, path string) string {
func TestNewMissingDirectory(t *testing.T) { func TestNewMissingDirectory(t *testing.T) {
_, err := New(WithoutShippedData(), WithDataPath("data/de_DE")) _, err := New(WithoutShippedData(), WithDataPath("data/de_DE"))
if err == nil || !strings.Contains(err.Error(), "de_DE") || !errors.Is(err, fs.ErrNotExist) { if err == nil || !strings.Contains(err.Error(), "de_DE") {
t.Fatalf("New(missing) error = %v, want the real error, naming the path", err) t.Fatalf("New(missing) error = %v, want it to name the path", err)
}
}
func TestNewRejectsAnEmptyDataPath(t *testing.T) {
_, err := New(WithoutShippedData(), WithDataPath(""))
if err == nil || !strings.Contains(err.Error(), "empty") {
t.Fatalf("New(WithDataPath(\"\")) = %v, want the empty path named", err)
}
}
func TestNewReportsAnEntropyFailure(t *testing.T) {
saved := randomBytes
randomBytes = func([]byte) (int, error) { return 0, errors.New("no entropy") }
defer func() { randomBytes = saved }()
_, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"w.json": {Data: []byte(`"x"`)}}))
if err == nil || !strings.Contains(err.Error(), "no entropy") {
t.Fatalf("New() without entropy = %v, want the failure reported", err)
} }
} }
+36 -65
View File
@@ -10,36 +10,29 @@ import (
// visiting keys in sorted order so which of several broken nodes gets reported does // visiting keys in sorted order so which of several broken nodes gets reported does
// not depend on map iteration. // not depend on map iteration.
func walkNodes(root map[string]node, fn func(path string, n node) error) error { func walkNodes(root map[string]node, fn func(path string, n node) error) error {
for _, name := range sortedNames(root) {
if err := eachNode(root[name], name, fn); err != nil {
return err
}
}
return nil
}
// eachNode visits n and every node contained within it once, passing the dot path
// that reaches each. It never crosses a reference edge — a bound {/path} field is
// skipped — so a single inline node is walked on its own.
func eachNode(n node, path string, fn func(path string, n node) error) error {
seen := map[node]bool{} seen := map[node]bool{}
var visit func(string, node) error var visit func(string, node) error
visit = func(path string, m node) error { visit = func(path string, n node) error {
if m == nil || seen[m] { if n == nil || seen[n] {
return nil return nil
} }
seen[m] = true seen[n] = true
if err := fn(path, m); err != nil { if err := fn(path, n); err != nil {
return err return err
} }
for _, c := range contained(m) { for _, c := range contained(n) {
if err := visit(join(path, c.name), c.node); err != nil { if err := visit(join(path, c.name), c.node); err != nil {
return err return err
} }
} }
return nil return nil
} }
return visit(path, n) for _, name := range sortedNames(root) {
if err := visit(name, root[name]); err != nil {
return err
}
}
return nil
} }
// namedNode is a contained child and the segment reaching it; a choice's items carry // namedNode is a contained child and the segment reaching it; a choice's items carry
@@ -165,56 +158,34 @@ func pathLeaves(n node, tail []string) []node {
return out return out
} }
// nodeScope is the set of nodes one validation pass covers: a whole loaded tree, // checkRepeatReach bounds the renders a repeat multiplies to along any root-to-leaf
// or a single inline node. // path, so nested repeats cannot build what one repeat may not. It runs after
type nodeScope func(fn func(path string, n node) error) error // checkNoCycles, whose guarantee is what lets the walk terminate.
func checkRepeatReach(root map[string]node) error {
func treeScope(root map[string]node) nodeScope { reach := map[node]int{}
return func(fn func(path string, n node) error) error { return walkNodes(root, fn) } var of func(n node) int
} of = func(n node) int {
if r, done := reach[n]; done {
func inlineScope(n node) nodeScope { return r
return func(fn func(path string, m node) error) error { return eachNode(n, "template", fn) } }
} r := 1
for _, e := range renderEdges(n) {
// checkScope runs the per-node fences over a scope, each over the whole scope if c := of(e.to); c > r {
// before the next, so which of several broken nodes is reported does not depend on r = c
// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks }
// terminate. }
func checkScope(s nodeScope) error { if t, ok := n.(*template); ok {
mem := reachMemo{} r *= t.repeat
if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil { }
return err reach[n] = r
}
return s(heldCheck)
}
type reachMemo map[node]int
func (m reachMemo) of(n node) int {
if r, done := m[n]; done {
return r return r
} }
r := 1 return walkNodes(root, func(path string, n node) error {
for _, e := range renderEdges(n) { if t, ok := n.(*template); ok && t.repeat > 1 && of(n) > maxLen {
if c := m.of(e.to); c > r { return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, of(n), maxLen)
r = c
} }
} return nil
if t, ok := n.(*template); ok { })
r *= t.repeat
}
m[n] = r
return r
}
// repeatCheck bounds the renders a repeat multiplies to along any root-to-leaf
// path, so nested repeats cannot build what one repeat may not.
func repeatCheck(path string, n node, mem reachMemo) error {
if t, ok := n.(*template); ok && t.repeat > 1 && mem.of(n) > MaxRepeat {
return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.of(n), MaxRepeat)
}
return nil
} }
// checkNoCycles rejects a reference cycle: a node whose rendering can reach itself // checkNoCycles rejects a reference cycle: a node whose rendering can reach itself
+18 -17
View File
@@ -6,23 +6,27 @@ import (
"strings" "strings"
) )
// heldCheck rejects every route to a held name except the ones that read its draw. // checkBoundLevelsHeld rejects every route to a held name except the ones that read
// An expansion holds one draw of that name; anything else that renders it draws // its draw. An expansion holds one draw of that name; anything else that renders it
// again, and the two disagree. checkNoOverlap settles the spellings within one // draws again, and the two disagree. checkNoOverlap settles the spellings within one
// format (a token, an operand); this settles the rest — a reference, whether it // format (a token, an operand); this settles the rest — a reference, whether it
// sits in that format or in anything the format renders, however deep. // sits in that format or in anything the format renders, however deep.
func heldCheck(path string, n node) error { //
t, ok := n.(*template) // It runs after checkNoCycles, whose guarantee is what lets the walk terminate.
if !ok || len(t.held) == 0 { func checkBoundLevelsHeld(root map[string]node) error {
return nil return walkNodes(root, func(path string, n node) error {
} t, ok := n.(*template)
readers := boundReaders(t.format, t.bound, t.refs) if !ok || len(t.held) == 0 {
for _, head := range heldHeads(t) { return nil
if err := checkHeadHeld(t, head, readers); err != nil {
return fmt.Errorf("%s: %w", path, err)
} }
} readers := boundReaders(t.format, t.bound, t.refs)
return nil for _, head := range heldHeads(t) {
if err := checkHeadHeld(t, head, readers); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
}
return nil
})
} }
// heldHeads lists a template's held names, operand heads first, then paths, each // heldHeads lists a template's held names, operand heads first, then paths, each
@@ -269,9 +273,6 @@ type draws struct {
// linkRefs prove every step, so the walk cannot fail. // linkRefs prove every step, so the walk cannot fail.
func readField(s *session, t *template, held *draws, a arm) string { func readField(s *session, t *template, held *draws, a arm) string {
if !t.held[a.key] { if !t.held[a.key] {
if len(a.tail) > 0 {
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
}
return render(s, t.fields[a.key]) return render(s, t.fields[a.key])
} }
if v, read := held.value[a.path]; read { if v, read := held.value[a.path]; read {
-10
View File
@@ -711,13 +711,3 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) {
} }
} }
} }
func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) {
tm, ok := compiled(t, `{"format":"{w}","w":{"format":"{x}","x":"1"}}`).(*template)
if !ok {
t.Fatal("not a template")
}
mustPanic(t, "unheld arm with a path", func() {
readField(engine(1).rand, tm, nil, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"})
})
}
-88
View File
@@ -1,88 +0,0 @@
package fejkdata
import (
"encoding/json"
"fmt"
"strings"
)
// Template is an inline template compiled, referenced and validated against a
// generator's loaded data once, ready to render many times with [Template.Fake].
// It is safe for concurrent use: Fake serializes on its generator's lock, so a
// seeded sequence is reproducible only when a generator — and its templates — are
// drawn from one goroutine.
type Template struct {
g *Generator
n node
}
// Fake renders the template with one draw.
func (t *Template) Fake() string {
t.g.mu.Lock()
defer t.g.mu.Unlock()
return render(t.g.rand, t.n)
}
// NewTemplate compiles an inline template — a format string or a JSON value — and
// binds its references against the loaded tree, so repeated renders pay the
// compile and validation once. It shares [New]'s guarantees: a bad template errors
// here, and rendering cannot fail.
func (f *Generator) NewTemplate(input string) (*Template, error) {
n, err := compileInput(input)
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
scope := inlineScope(n)
if err := linkNodeRefs(scope, f.categories); err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
if err := checkScope(scope); err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Template{g: f, n: n}, nil
}
// FakeTemplate compiles and renders an inline template in one call. It is
// [NewTemplate] then [Template.Fake]; to render the same template many times, hold
// the *Template and call its Fake.
func (f *Generator) FakeTemplate(input string) (string, error) {
t, err := f.NewTemplate(input)
if err != nil {
return "", err
}
return t.Fake(), nil
}
// compileInput compiles an inline template: a JSON value, or a bare format string
// when the input is not JSON.
func compileInput(input string) (node, error) {
var raw any
if err := json.Unmarshal([]byte(input), &raw); err != nil {
return compile(input)
}
if trimmed := strings.TrimSpace(input); trimmed != input {
return nil, fmt.Errorf("a JSON template may not be padded with spaces, which a format string would render; write %s", trimmed)
}
return compile(raw)
}
// linkNodeRefs binds the references in an inline node's templates against the
// loaded tree.
func linkNodeRefs(scope nodeScope, root map[string]node) error {
return scope(func(path string, m node) error {
t, ok := m.(*template)
if !ok {
return nil
}
for _, name := range refTokens(t.format) {
sigil, rest, err := refShape(name)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
if sigil != "/" {
return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest)
}
}
return linkTemplateRefs(nil, path, t, root)
})
}
-159
View File
@@ -1,159 +0,0 @@
package fejkdata
import (
"strings"
"testing"
)
func tmpl(t *testing.T, f *Generator, input string) string {
t.Helper()
s, err := f.FakeTemplate(input)
if err != nil {
t.Fatalf("FakeTemplate(%q): %v", input, err)
}
return s
}
func shipped(t *testing.T, opts ...Option) *Generator {
t.Helper()
f, err := New(append([]Option{WithSeed(1)}, opts...)...)
if err != nil {
t.Fatal(err)
}
return f
}
func TestFakeTemplateFormatString(t *testing.T) {
f := shipped(t)
got := tmpl(t, f, "name: {/sv_SE.person.last}")
if !strings.HasPrefix(got, "name: ") || strings.HasSuffix(got, " ") || strings.Contains(got, "{") {
t.Fatalf("FakeTemplate = %q, want a rendered last name after the prefix", got)
}
}
func TestFakeTemplateJSONObject(t *testing.T) {
f := shipped(t)
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[tmpl(t, f, `{"format":"name: {x}","x":["bosse","lina"]}`)] = true
}
if !seen["name: bosse"] || !seen["name: lina"] || len(seen) != 2 {
t.Fatalf("JSON template produced %v, want both names", seen)
}
}
func TestFakeTemplateJSONArray(t *testing.T) {
f := shipped(t)
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[tmpl(t, f, `["foo","bar","baz"]`)] = true
}
if len(seen) != 3 {
t.Fatalf("JSON array choice produced %v, want three items", seen)
}
}
func TestFakeTemplateCorrelatedReferences(t *testing.T) {
dir := writeData(t, map[string]string{
"person": `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"}]`,
})
f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 100; i++ {
got := tmpl(t, f, "{/person.first} {/person.last}")
if got != "Ada Lovelace" && got != "Bo Ek" {
t.Fatalf("correlated references = %q, want one person's first and last", got)
}
}
}
func TestFakeTemplateDeterministic(t *testing.T) {
a, b := shipped(t), shipped(t)
for i := 0; i < 20; i++ {
in := "row: {/misc.uuid} {digits(3)}"
if x, y := tmpl(t, a, in), tmpl(t, b, in); x != y {
t.Fatalf("same seed diverged: %q != %q", x, y)
}
}
}
func TestFieldlessTokenHint(t *testing.T) {
f := shipped(t)
_, err := f.FakeTemplate(`{sv_SE.person.last}`)
if err == nil || !strings.Contains(err.Error(), "write {/sv_SE.person.last}") {
t.Fatalf("FakeTemplate(bare token) = %v, want a hint naming {/sv_SE.person.last}", err)
}
_, err = f.FakeTemplate(`{"format":"{x}","repeat":2}`)
if err == nil || !strings.Contains(err.Error(), `this template has none — write {/x}`) {
t.Errorf("FakeTemplate(fieldless object) = %v, want the hint without calling it a bare string", err)
}
// A hint is only a drop-in where the name is the whole token: {/x} inside a
// transform or an alternation renders a different value, so none is offered.
for _, input := range []string{`{ /sv_SE.person.last }`, "{lowercase(x)}", "{x|y}"} {
_, err := f.FakeTemplate(input)
if err == nil || strings.Contains(err.Error(), "write {") {
t.Errorf("FakeTemplate(%q) = %v, want no hint naming a spelling that means something else", input, err)
}
}
}
func TestFakeTemplateErrors(t *testing.T) {
f := shipped(t)
for _, c := range []struct {
input string
want string
}{
{`{"x":"Q"}`, "missing string \"format\""},
{`"{x}"`, `no field "x"`},
{`"{digits(0)}"`, "must be positive"},
{`name: {/no.such.path}`, "no entry"},
{`name: {..nope}`, "write {/nope}"},
{`{"format":"x"}`, "is a string"},
{`{/misc.country} {/misc.country.alpha2}`, "renders a level"},
{`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"},
} {
_, err := f.FakeTemplate(c.input)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("FakeTemplate(%q) = %v, want an error containing %q", c.input, err, c.want)
}
}
}
func TestFakeTemplateJSONString(t *testing.T) {
f := shipped(t)
got := tmpl(t, f, `"name: {/sv_SE.person.last}"`)
if !strings.HasPrefix(got, "name: ") || strings.Contains(got, "{") {
t.Fatalf("FakeTemplate(JSON string) = %q, want a rendered last name after the prefix", got)
}
}
func TestPaddedJSONIsRejected(t *testing.T) {
f := shipped(t)
in := `{"format":"{x}","x":["a","b"]}`
_, err := f.NewTemplate(" " + in + " ")
if err == nil || !strings.Contains(err.Error(), "write "+in) {
t.Fatalf("NewTemplate(padded JSON) = %v, want an error naming the unpadded spelling", err)
}
}
func TestNewTemplateReusable(t *testing.T) {
f := shipped(t)
reusable, err := f.NewTemplate(`{digits(2)}`)
if err != nil {
t.Fatalf("NewTemplate: %v", err)
}
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[reusable.Fake()] = true
}
if len(seen) < 2 {
t.Fatalf("Template.Fake() repeated %v, want varied draws from one compile", seen)
}
}
func TestFakeTemplateRepeatBound(t *testing.T) {
f := shipped(t)
_, err := f.FakeTemplate(`{"format":"{x}","repeat":200,"x":{"format":"{y}","repeat":200,"y":{"format":"z","repeat":200}}}`)
if err == nil || !strings.Contains(err.Error(), "maximum") {
t.Errorf("nested repeat over the cap = %v, want it rejected naming the maximum", err)
}
}
-12
View File
@@ -275,18 +275,6 @@ func TestNewErrors(t *testing.T) {
map[string]string{"a|b": `"1"`}, map[string]string{"a|b": `"1"`},
`category "a|b" contains "|"`, `category "a|b" contains "|"`,
}, },
"field name with a bracket": {
map[string]string{"a": `{"format":"{x}","x":"1","b[c":"2"}`},
`field "b[c" contains "["`,
},
"category name with a bracket": {
map[string]string{"[abc]": `"1"`},
`category "[abc]" contains "["`,
},
"field name with a quote": {
map[string]string{"a": `{"format":"{x}","x":"1","b\"c":"2"}`},
`field "b\"c" contains "\""`,
},
// An empty name is not a path segment, so List never offered it — while a // An empty name is not a path segment, so List never offered it — while a
// bare {}, a trailing dot in Fake("a.") and a {/a.} reference all reached // bare {}, a trailing dot in Fake("a.") and a {/a.} reference all reached
// it. The engine accepted spellings it would never advertise. // it. The engine accepted spellings it would never advertise.
+11 -37
View File
@@ -57,15 +57,6 @@ type template struct {
func (*template) isNode() {} func (*template) isNode() {}
// field is the node a path segment names; a binding is a render edge, not a field.
func (t *template) field(seg string) (node, bool) {
if isRef(seg) {
return nil, false
}
n, ok := t.fields[seg]
return n, ok
}
// compile converts parsed JSON into a node tree, validating structure up front. // compile converts parsed JSON into a node tree, validating structure up front.
// Only a choice's items carry a weight, so one here would be inert whatever its type. // Only a choice's items carry a weight, so one here would be inert whatever its type.
func compile(v any) (node, error) { func compile(v any) (node, error) {
@@ -87,24 +78,10 @@ func compileItem(v any) (node, error) {
case map[string]any: case map[string]any:
return compileTemplate(v) return compileTemplate(v)
default: default:
return nil, fmt.Errorf("a template value must be a string, a list or an object, not %s", jsonKind(v)) return nil, fmt.Errorf("unsupported node type %T", v)
} }
} }
// jsonKind names a JSON value a template cannot hold, in the data format's own
// terms rather than the decoding library's.
func jsonKind(v any) string {
switch v.(type) {
case float64:
return "a number"
case bool:
return "a boolean"
case nil:
return "null"
}
return fmt.Sprintf("%T", v)
}
func compileString(s string) (node, error) { func compileString(s string) (node, error) {
if err := checkTokens(s, nil); err != nil { if err := checkTokens(s, nil); err != nil {
return nil, err return nil, err
@@ -252,9 +229,6 @@ func readOptions(m map[string]any) (templateOptions, error) {
if repeat == 1 { if repeat == 1 {
return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1") return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1")
} }
if o.separator == "" {
return o, fmt.Errorf("separator \"\" is the default, so it has no effect; drop it")
}
} }
_, o.weighted = m["weight"] _, o.weighted = m["weight"]
return o, nil return o, nil
@@ -302,8 +276,8 @@ func repeatOf(m map[string]any) (int, error) {
if r == 1 { if r == 1 {
return 0, fmt.Errorf("repeat 1 is the default, so it has no effect; drop it") return 0, fmt.Errorf("repeat 1 is the default, so it has no effect; drop it")
} }
if r > MaxRepeat { // caps the renders one repeat asks for; repeatCheck bounds what nested ones multiply to if r > maxLen { // cap so a fat-fingered repeat can't build a multi-GB string
return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, MaxRepeat) return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, maxLen)
} }
return int(r), nil return int(r), nil
} }
@@ -337,23 +311,23 @@ func weightOf(raw any) (float64, error) {
// reservedInName is what a category, folder or field name may not contain: a dot // reservedInName is what a category, folder or field name may not contain: a dot
// separates the segments of a path, '|' the arms of a token, '(' opens a function // separates the segments of a path, '|' the arms of a token, '(' opens a function
// call, braces delimit the token, '/' starts a reference, and brackets and a quote // call, braces delimit the token and '/' starts a reference. A name carrying one is
// open a JSON value. A name carrying one is rejected where it is authored rather // reachable by no format, so it is rejected where it is authored rather than at
// than where it would be unreachable. // the token that cannot reach it.
const reservedInName = ".|({}/[]\"" const reservedInName = ".|({}/"
// reservedList spells reservedInName for an error message, so the two cannot drift. // reservedList spells reservedInName for an error message, so the two cannot drift.
var reservedList = strings.Join(strings.Split(reservedInName, ""), " ") var reservedList = strings.Join(strings.Split(reservedInName, ""), " ")
// checkName rejects a name the dot path, {token} and JSON grammars cannot spell. // checkName rejects a name the dot path and {token} grammars cannot spell. Both a
// Both a category or folder and a field go through it, so there is one answer to // category or folder and a field go through it, so there is one answer to what a
// what a name may contain. // name may contain.
func checkName(name string) error { func checkName(name string) error {
if name == "" { if name == "" {
return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name) return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name)
} }
if i := strings.IndexAny(name, reservedInName); i >= 0 { if i := strings.IndexAny(name, reservedInName); i >= 0 {
return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path, {token} and JSON grammars reserve", return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path and {token} grammars reserve",
name, name[i:i+1], reservedList) name, name[i:i+1], reservedList)
} }
return nil return nil
+5 -16
View File
@@ -30,12 +30,11 @@ func TestRepeatedChoiceItemIsRejected(t *testing.T) {
func TestInertObjectIsRejected(t *testing.T) { func TestInertObjectIsRejected(t *testing.T) {
for src, want := range map[string]string{ for src, want := range map[string]string{
`{"format":"Malmö"}`: `write "Malmö"`, `{"format":"Malmö"}`: `write "Malmö"`,
`{"format":"{digits(3)}"}`: `write "{digits(3)}"`, `{"format":"{digits(3)}"}`: `write "{digits(3)}"`,
`[{"format":"a","weight":1},"b"]`: "weight 1", `[{"format":"a","weight":1},"b"]`: "weight 1",
`{"format":"{x}","x":"v","repeat":1}`: "repeat 1", `{"format":"{x}","x":"v","repeat":1}`: "repeat 1",
`{"format":"{x}","x":"v","separator":","}`: "separator", `{"format":"{x}","x":"v","separator":","}`: "separator",
`{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default",
} { } {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want) t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want)
@@ -47,13 +46,3 @@ func TestInertObjectIsRejected(t *testing.T) {
} }
} }
} }
func TestInlineFolderSigilsAreRejected(t *testing.T) {
f := shipped(t)
for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} {
_, err := f.NewTemplate(input)
if err == nil || !strings.Contains(err.Error(), "write {/sv_SE.person.last}") {
t.Errorf("NewTemplate(%q) = %v, want an error naming the root spelling", input, err)
}
}
}
+1 -1
View File
@@ -40,7 +40,7 @@ func walkPath(n node, tail []string, w pathWalk) error {
return err return err
} }
} }
child, ok := n.field(tail[0]) child, ok := n.fields[tail[0]]
if !ok { if !ok {
return fmt.Errorf("no field %q", tail[0]) return fmt.Errorf("no field %q", tail[0])
} }
-61
View File
@@ -1,61 +0,0 @@
package fejkdata
import (
"fmt"
"strings"
"testing"
"testing/fstest"
)
// One Fake call descends depth levels: each level's "a" is the next template down.
func nestedJSON(depth int) string {
s := `"leaf"`
for i := 0; i < depth; i++ {
s = fmt.Sprintf(`{"format":"{a}","a":%s}`, s)
}
return s
}
// One Fake call expands n sibling tokens.
func wideTokenJSON(n int) string {
var toks, fields strings.Builder
for i := 0; i < n; i++ {
fmt.Fprintf(&toks, "{f%d}", i)
if i > 0 {
fields.WriteByte(',')
}
fmt.Fprintf(&fields, `"f%d":"x"`, i)
}
return fmt.Sprintf(`{"format":"%s",%s}`, toks.String(), fields.String())
}
func TestNoRenderAllocRegression(t *testing.T) {
shapes := []struct {
name string
json string
base float64
}{
{"nested depth 25", nestedJSON(25), 27},
{"nested depth 100", nestedJSON(100), 102},
{"wide 500 tokens", wideTokenJSON(500), 9},
}
for _, s := range shapes {
f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"x.json": {Data: []byte(s.json)}}))
if err != nil {
t.Fatalf("New(%s): %v", s.name, err)
}
allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") })
if allocs > s.base*1.10 {
t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); bump the baseline only as a deliberate change", s.name, allocs, s.base*1.10, s.base)
}
}
}
func BenchmarkNestedDepth25(b *testing.B) { benchPath(b, tmpData(b, "deep", nestedJSON(25)), "deep") }
func BenchmarkNestedDepth100(b *testing.B) { benchPath(b, tmpData(b, "deep", nestedJSON(100)), "deep") }
func BenchmarkWideTokens100(b *testing.B) {
benchPath(b, tmpData(b, "wide", wideTokenJSON(100)), "wide")
}
func BenchmarkWideTokens500(b *testing.B) {
benchPath(b, tmpData(b, "wide", wideTokenJSON(500)), "wide")
}
+28 -35
View File
@@ -72,42 +72,35 @@ func refSegments(name string, folder []string) ([]string, error) {
// error, never a random render-time one. // error, never a random render-time one.
func linkRefs(root map[string]node) error { func linkRefs(root map[string]node) error {
return eachTemplate(root, func(folder []string, path string, t *template) error { return eachTemplate(root, func(folder []string, path string, t *template) error {
return linkTemplateRefs(folder, path, t, root) names := refTokens(t.format)
}) if len(names) == 0 {
} return nil
}
// linkTemplateRefs binds one template's references against root. A template with if t.fields == nil {
// none is left untouched, so an inline format that references nothing costs only t.fields = map[string]node{}
// the refTokens scan. }
func linkTemplateRefs(folder []string, path string, t *template, root map[string]node) error { t.refs = make(map[string]refBinding, len(names))
names := refTokens(t.format) for _, name := range names {
if len(names) == 0 { segments, err := refSegments(name, folder)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
head, target, tail, err := resolveRef(root, segments)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
key := "/" + strings.Join(head, ".")
if err := checkPath(target, tail, key); err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
t.fields[key] = target
t.refs[name] = refBinding{key, tail}
}
if err := t.compileFormat(); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil return nil
} })
if t.fields == nil {
t.fields = map[string]node{}
}
t.refs = make(map[string]refBinding, len(names))
for _, name := range names {
segments, err := refSegments(name, folder)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
head, target, tail, err := resolveRef(root, segments)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
key := "/" + strings.Join(head, ".")
if err := checkPath(target, tail, key); err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
t.fields[key] = target
t.refs[name] = refBinding{key, tail}
}
if err := t.compileFormat(); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
} }
// eachTemplate calls fn once per template, with the folder its category sits in // eachTemplate calls fn once per template, with the folder its category sits in
-6
View File
@@ -108,12 +108,6 @@ func TestFakeIsSafeForConcurrentUse(t *testing.T) {
return return
} }
f.List() f.List()
tmpl, err := f.NewTemplate("{/sv_SE.person.last}")
if err != nil {
t.Error(err)
return
}
tmpl.Fake()
} }
}() }()
} }
+2 -22
View File
@@ -140,7 +140,7 @@ func checkTokens(format string, fields map[string]node) error {
} }
continue // its target is checked at New (see linkRefs) continue // its target is checked at New (see linkRefs)
} }
if err := checkArm(name, fields, len(names) == 1); err != nil { if err := checkArm(name, fields); err != nil {
return fmt.Errorf("token {%s}: %w", t.body, err) return fmt.Errorf("token {%s}: %w", t.body, err)
} }
} }
@@ -151,9 +151,7 @@ func checkTokens(format string, fields map[string]node) error {
} }
// checkArm validates one sibling name or path against a template's fields. // checkArm validates one sibling name or path against a template's fields.
// wholeToken says the name is the token's entire body, so {/name} would render func checkArm(name string, fields map[string]node) error {
// the same value and can be offered as the reference spelling.
func checkArm(name string, fields map[string]node, wholeToken bool) error {
a := splitArm(name, nil) a := splitArm(name, nil)
if err := checkSegments(a); err != nil { if err := checkSegments(a); err != nil {
return err return err
@@ -166,13 +164,6 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error {
if isOption(a.key) { if isOption(a.key) {
return fmt.Errorf("%q is an option and can never be a field", a.key) return fmt.Errorf("%q is an option and can never be a field", a.key)
} }
if len(fields) == 0 {
hint := ""
if wholeToken && hintableRef(name) {
hint = fmt.Sprintf(" — write {/%s} to reference the data", name)
}
return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.key, hint)
}
return fmt.Errorf("no field %q", a.key) return fmt.Errorf("no field %q", a.key)
} }
if err := checkPath(head, a.tail, a.key); err != nil { if err := checkPath(head, a.tail, a.key); err != nil {
@@ -181,17 +172,6 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error {
return nil return nil
} }
// hintableRef reports whether {/name} is a reference the grammar accepts, so the
// hint never names a spelling that fails too.
func hintableRef(name string) bool {
for _, seg := range strings.Split(name, ".") {
if checkName(seg) != nil {
return false
}
}
return true
}
// tokenOperands lists the fields one {token} body reads as operands, empty for a // tokenOperands lists the fields one {token} body reads as operands, empty for a
// field token or a builtin that reads none. // field token or a builtin that reads none.
func tokenOperands(body string) []string { func tokenOperands(body string) []string {
+1 -21
View File
@@ -8,13 +8,7 @@ import (
) )
// engine builds a seeded generator with no loaded categories, for rendering tests. // engine builds a seeded generator with no loaded categories, for rendering tests.
func engine(seed uint64) *Generator { func engine(seed uint64) *Generator { return &Generator{rand: newRand(seed, true)} }
s, err := newRand(seed, true)
if err != nil {
panic(err)
}
return &Generator{rand: s}
}
// parse unmarshals a JSON template fragment into its dynamic form. // parse unmarshals a JSON template fragment into its dynamic form.
func parse(t *testing.T, s string) any { func parse(t *testing.T, s string) any {
@@ -365,17 +359,3 @@ func TestAlternationThreeWay(t *testing.T) {
t.Fatalf("3-way alternation produced %v, want A, B and C", seen) t.Fatalf("3-way alternation produced %v, want A, B and C", seen)
} }
} }
func TestArgErrorsNameTheSpelling(t *testing.T) {
for src, want := range map[string]string{
`"{float(1,2,02)}"`: "write 2",
`{"format":"{calc(a,02)}","a":"1"}`: "write 2",
`"{digits(+5)}"`: "write 5",
`"{hex(99999999999999999999)}"`: "exceeds the maximum",
`"{int(007,9)}"`: "write 7",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error saying %q", src, err, want)
}
}
}
-28
View File
@@ -1,28 +0,0 @@
# Release checklist
What to settle before the first tag, then the work that follows in a later,
data-heavy release.
## Before the first release — settle the record flag and API contract
- `--format` vocabulary — confirm `text`, `json`, `ndjson`, `csv`, `sql`; the
`json`-as-array vs `ndjson`-as-lines split; `--table` (the SQL INSERT target);
the `--separator` rejection on record formats; and the exit codes (misuse 2,
runtime 1).
- Library surface — confirm `Record`, `FakeRecord`, `NewRecordTemplate`,
`RecordTemplate`, `Column`/`Columns()`, and the `JSON()`, `CSVHeader()`,
`CSVLine()`, `SQLInsert()` serializers.
- Typed scalars — columns are strings today (`"42"`, quoted SQL). Confirm that
stays out of scope, or add a per-column `kind` before the tag.
- Struct-filling — `fake:"..."` tags (reflection over an arbitrary struct) stay
out of scope; `Columns()` hands the caller the values to map themselves.
Confirm.
- Independent reference draw — within one record every tailed reference to a
category is one draw, with no spelling for "these columns should disagree".
Confirm the per-record contract, or add the spelling.
## Later, in a data-heavy release
- Shipped-data de-duplication — `email.json`'s `local` is a drifted copy of
`username.json`; fold it in when the shipped set grows and we add lots more
data.
-6
View File
@@ -51,9 +51,3 @@ func TestTransformArgs(t *testing.T) {
} }
} }
} }
func TestAsciiKeepsDEL(t *testing.T) {
if got := mustRender(t, engine(1), `{"format":"{ascii(x)}","x":"a\u007fb"}`); got != "a\u007fb" {
t.Errorf("ascii over DEL = %q, want it kept: DEL is ASCII", got)
}
}