Compare commits

..

57 Commits

Author SHA1 Message Date
lilleman b223fcd717 Move deferred and out-of-scope items to todo.md
Tests / vet + fmt + tests (pull_request) Successful in 1m38s
2026-09-05 15:29:37 +02:00
lilleman f75a9be5b4 Frame json as an array and add ndjson
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-04 10:48:28 +02:00
lilleman 80524a73b7 Tests: json is an array document and ndjson is one object per line 2026-09-04 10:48:28 +02:00
lilleman 2310cb1e1a Name every spelling the self-reference fence refuses, and cut the doubled comment
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-04 09:53:46 +02:00
lilleman 85a4cd4475 Refuse a column pointing back at its record however it is spelled, and qualify the shared-draw claim to path reads
Tests / vet + fmt + tests (pull_request) Successful in 1m2s
2026-09-04 09:47:55 +02:00
lilleman b99526aeb8 Tests: a record read whole or as an operand still points back at itself, and Record under concurrent use 2026-09-04 09:46:39 +02:00
lilleman 79827a66d6 Fence a record once per node, refuse a column reading its own record, and keep a bare reference independent as an operand
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-04 09:32:29 +02:00
lilleman 78572ccbbc Tests: a self-referencing column, a bare reference as an operand, and every route the overlap fence must reach 2026-09-04 09:27:52 +02:00
lilleman c612c9e734 Refuse overlapping reference reads across a record's columns, keep an empty column a CSV row, and reject an empty --table
Tests / vet + fmt + tests (pull_request) Successful in 1m4s
2026-09-04 09:10:03 +02:00
lilleman be5dfdf977 Tests: overlapping reference columns rejected, every CSV shape round-tripped, an empty column still a row, and an empty --table refused 2026-09-04 09:08:23 +02:00
lilleman 83e11074ac Fix the column set once, drop the unreachable separator override, and name the record predicate apart from the format table
Tests / vet + fmt + tests (pull_request) Successful in 1m2s
2026-09-04 08:52:08 +02:00
lilleman 59159d4f5d Tests: the record fence reads correctly for an inline record too 2026-09-04 08:51:00 +02:00
lilleman 7ee7f1788b Record the SQL dialect, the newline-in-a-column shape, the fixed column set and the string view's divergence
Tests / vet + fmt + tests (pull_request) Successful in 1m1s
2026-09-04 08:43:12 +02:00
lilleman 8cb8cf6334 Reject a record's top-level repeat, share one record fence between both entry points, and give the renderer one spelling 2026-09-04 08:40:58 +02:00
lilleman 54e6865b11 Tests: columns as the record's public name, and a top-level repeat rejected at both record entry points 2026-09-04 08:38:03 +02:00
lilleman ee90337834 Fix the Records examples and pin their seeded output
Tests / vet + fmt + tests (pull_request) Successful in 1m2s
2026-09-04 07:39:28 +02:00
lilleman d237c15d77 Pin bare-reference independence and tighten the shared-draw comments
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-03 23:29:10 +02:00
lilleman 93ac610192 Reject field descent in Record and pin the shared-draw scope
Tests / vet + fmt + tests (pull_request) Successful in 1m2s
2026-09-03 23:22:43 +02:00
lilleman 6a8aa96b20 Quote SQL identifiers and share reference draws across record columns
Tests / vet + fmt + tests (pull_request) Successful in 1m2s
2026-09-03 23:12:45 +02:00
lilleman 7383e8d0ca Test record output and its serializers
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-03 22:56:56 +02:00
lilleman ca40fe0c2c Add record output to JSON, CSV and SQL 2026-09-03 22:56:56 +02:00
lilleman 466ed5a4d9 Minor README update
Tests / vet + fmt + tests (pull_request) Successful in 1m4s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 22:09:42 +02:00
lilleman ce22c83e7d Cut the review-found narration: two helper invariants, one line each
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 21:53:51 +02:00
lilleman 0d2a080b85 Add the fast-enough goal and record the allocation-gate decision
Tests / vet + fmt + tests (pull_request) Successful in 1m3s
2026-09-03 21:46:22 +02:00
lilleman 296058fb80 Tests: allocation ceilings for deeply nested and wide renders 2026-09-03 21:46:22 +02:00
lilleman 44601c90ab Offer the reference hint only where it would render the same value, and name the guard set in the help
Tests / vet + fmt + tests (pull_request) Successful in 57s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-03 21:12:04 +02:00
lilleman 8ed8d9bb51 Tests: no hint where a reference would mean something else, and every reference the help advertises resolves 2026-09-03 21:11:11 +02:00
lilleman 1cd422352b Cut the comments the README already records, and record the cycle-fence and exit-code rules there
Tests / vet + fmt + tests (pull_request) Successful in 58s
2026-09-03 21:00:42 +02:00
lilleman 0cdbdb7bf4 Fix the help example, read padding past the shape gate, catch a stray closing brace, and stop the fieldless hint naming a spelling that fails 2026-09-03 20:58:01 +02:00
lilleman d8048df169 Tests: padded and stray-brace arguments, the inline held fence, a template under concurrent use, and the fieldless-token hint 2026-09-03 20:57:25 +02:00
lilleman 71cccaf4c9 Reject a leading quote that opens no JSON, and give the inline walk one home
Tests / vet + fmt + tests (pull_request) Successful in 55s
2026-09-03 20:43:04 +02:00
lilleman 704d18d109 Tests: a leading quote that opens no JSON names neither a template nor a path 2026-09-03 20:42:34 +02:00
lilleman 44c91668f9 Give the inline pipeline its own file, and record the shape, padding and template-API decisions
Tests / vet + fmt + tests (pull_request) Successful in 55s
2026-09-03 20:37:44 +02:00
lilleman 6916894ab5 One CLI classifier settling the argument shape before load; JSON strings are templates and a quote is reserved in names 2026-09-03 20:34:42 +02:00
lilleman 80e8d72bb7 One fence sequence over a node scope, and inline templates reject the folder sigils 2026-09-03 20:34:42 +02:00
lilleman 7c7bee5cd9 Tests: one fence sequence, inline folder sigils rejected, JSON string arguments, quoted names, and shape misuse before load 2026-09-03 20:31:49 +02:00
lilleman 841513bb27 Lead the README with what it does and a quick example, move Goals beside Decisions
Tests / vet + fmt + tests (pull_request) Successful in 57s
2026-09-03 20:14:11 +02:00
lilleman 873b7678ba Address reviewer findings: reference hint, consumer-terms type errors, misuse exit code, immutability and parity docs
Tests / vet + fmt + tests (pull_request) Successful in 57s
2026-09-03 19:21:42 +02:00
lilleman 4d58c93757 Tests: template misuse exits 2, and a bare token hints its reference spelling 2026-09-03 19:21:42 +02:00
lilleman 305d8f8edd Address stability nits: doc grammar, scalar-JSON note, bracket-rule rationale
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-03 17:53:05 +02:00
lilleman d8ece597c8 Reserve brackets in names, add compile-once NewTemplate, let --repeat reuse it, record scope artifacts
Tests / vet + fmt + tests (pull_request) Failing after 16s
2026-09-03 17:48:20 +02:00
lilleman 47ec883bb0 Tests: reusable NewTemplate, and bracket names rejected at load 2026-09-03 17:48:20 +02:00
lilleman 031a3ec7d6 Resolve architect findings: gate [ on json.Valid, fix the root-relative README claim, test isTemplate
Tests / vet + fmt + tests (pull_request) Failing after 16s
2026-09-03 17:40:29 +02:00
lilleman e08435afa7 Document inline templates: goal 4, the CLI and library, the shape-based disambiguation
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-03 17:30:09 +02:00
lilleman 3a9f3a4758 Route inline templates to FakeTemplate at the CLI 2026-09-03 17:29:07 +02:00
lilleman 4399b8b89f Tests for inline templates at the CLI 2026-09-03 17:28:14 +02:00
lilleman 3bc3e35045 Add FakeTemplate: compile an inline template and render it against the loaded tree 2026-09-03 17:27:54 +02:00
lilleman 5ba38a8281 Tests for FakeTemplate: inline format strings, JSON templates, references, and their errors 2026-09-03 17:27:51 +02:00
lilleman addba9abf7 Give maxLen its own literal and repeats MaxRepeat; arg errors name the spelling and the range; write's comment says what Flush reports; assert 64-bit at compile time
Tests / vet + fmt + tests (pull_request) Successful in 59s
Tests / vet + fmt + tests (push) Successful in 6s
2026-09-02 19:25:19 +02:00
lilleman a9552ed966 Tests for every branch of the constant-divisor fold, arg errors naming the spelling, and shell numbers at the CLI
Tests / vet + fmt + tests (pull_request) Failing after 43s
2026-09-02 19:24:46 +02:00
lilleman c5304bb6e7 Record the repeat cap's reach, 64-bit only, the constant-divisor rule and the rejected default spellings; correct the repeat cap comment
Tests / vet + fmt + tests (pull_request) Successful in 58s
2026-09-02 19:09:50 +02:00
lilleman 08c7ef9b7f Keep the real data-path error, reject an empty data path, and report a crypto/rand failure instead of seeding zero
Tests / vet + fmt + tests (pull_request) Successful in 58s
2026-09-02 19:09:29 +02:00
lilleman ae0527a26e Tests for the real data-path error, an empty data path, and a crypto/rand failure
Tests / vet + fmt + tests (pull_request) Failing after 16s
2026-09-02 19:09:19 +02:00
lilleman 4f9285ab5c Reject non-finite float bounds, unplain integer args, constant samples, a constant zero divisor and the default separator; a binding key is no path segment; ascii keeps DEL; an unheld path panics
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-02 19:08:32 +02:00
lilleman 7b28dc36d9 Tests for finite float bounds, plain integer args, constant samples and divisors, the default separator, binding keys in paths, DEL in ascii, and an unheld path
Tests / vet + fmt + tests (pull_request) Failing after 42s
2026-09-02 19:08:24 +02:00
lilleman 3d3efbb642 Bound --repeat at MaxRepeat and stream the renders; name an unknown flag by its rune; --list rejects an inert --repeat or --separator
Tests / vet + fmt + tests (pull_request) Successful in 56s
2026-09-02 19:06:20 +02:00
lilleman 5a75df8034 Tests for a bounded, streamed --repeat, rune-named flags, and --list with inert flags 2026-09-02 19:06:15 +02:00
30 changed files with 2308 additions and 273 deletions
-8
View File
@@ -8,11 +8,3 @@
- A standing choice a reader would relitigate goes under Decisions in the README, not in a comment. - A standing choice a reader would relitigate goes under Decisions in the README, not in a comment.
- A README example is a `json` block that loads and renders as a category; `readme_test.go` runs every one. - A README example is a `json` block that loads and renders as a category; `readme_test.go` runs every one.
- Cyclomatic complexity is gated at 14: the table-shaped dispatches (`eachToken`, `calc.factor`, `walkPath`) sit at 1314 and stay whole; anything else that reaches 14 is decomposed. - Cyclomatic complexity is gated at 14: the table-shaped dispatches (`eachToken`, `calc.factor`, `walkPath`) sit at 1314 and stay whole; anything else that reaches 14 is decomposed.
# Deferred
- **Shipped-data de-duplication (2026-09-02).** `email.json`'s `local` is a
drifted copy of `username.json`, and no shipped file yet uses a held path, an
operand or a reference. Fixed in the data fill before the first tag, when the
shipped set is rewritten anyway; premise: nothing depends on the shipped data's
shape until then. Not raised in review before that.
+219 -36
View File
@@ -1,32 +1,17 @@
# fejkdata # fejkdata
A Go library and CLI for generating locale-aware fake data from JSON templates. Locale-aware fake data for tests and fixtures, generated from JSON templates. Use
Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes). it as a Go library or the CLI — no data on disk, no dependencies, and a seed makes output
reproducible.
## Goals ```sh
go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest
1. **Valid by construction** — every value passes the check its real consumer fejkdata sv_SE.person # Sara Eriksson
applies; facts that belong together come from one draw, within a value and ```
across categories.
2. **Text means what it says** — a format renders as written; only `{…}` varies,
random characters included (`{digits(3)}`). One spelling per result; the wrong
one is a load error naming the right one.
3. **Every mistake is a load error**`New` rejects; `Fake` on a loaded generator
fails only for an unknown path.
4. **Zero to a value in one command**`go install`, then `fejkdata sv_SE.person`:
no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position;
the first custom template needs no escape and no option.
5. **Data lives in JSON** — a builtin only for what data can't express.
6. **Reproducible** — seed in, same stream out; no builtin reads a clock.
7. **Zero dependencies** — standard library only.
8. **Docs index the grammar** — every syntax feature is a heading; every example
runs under test and shows its output; a rule is stated once.
## CLI ## CLI
```sh ```sh
go install gitea.larvit.se/larvit/fejkdata/cmd/fejkdata@latest
fejkdata sv_SE.person # Sara Eriksson fejkdata sv_SE.person # Sara Eriksson
fejkdata sv_SE.person.last # Eriksson fejkdata sv_SE.person.last # Eriksson
fejkdata --seed 42 sv_SE.address # the same address every run fejkdata --seed 42 sv_SE.address # the same address every run
@@ -34,25 +19,39 @@ fejkdata -n 3 --separator ', ' sv_SE.word # nät, barn, sol
fejkdata --list # every path the data offers fejkdata --list # every path the data offers
fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data
fejkdata --no-shipped-data -d ./mydata --list # only your data fejkdata --no-shipped-data -d ./mydata --list # only your data
fejkdata 'name: {/sv_SE.person.last}' # name: <a surname> — an inline template
fejkdata '{"format":"name: {x}","x":["bosse","lina"]}' # name: bosse or name: lina
``` ```
A path names a category, or a field inside one: each dot segment descends one A path names a category, or a field inside one: each dot segment descends one
level — folders, then the category (a JSON file), then fields. level — folders, then the category (a JSON file), then fields. An argument that is
a JSON object, array or string, or that carries a `{` token, is instead an
**inline template**: a format string or a JSON value compiled and rendered on the
spot. Its tokens reach the data by reference from the root —
`{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike
available. An inline template sits in no folder, so the folder-relative `{.name}`
and `{..name}` are rejected naming the root spelling. A path never contains a
brace, a bracket or a quote, so the two cannot collide (see
[Decisions](#decisions)).
| Flag | | | Flag | |
|------|--| |------|--|
| `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash | | `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash |
| `--no-shipped-data` | load only the `--data-path` directories | | `--no-shipped-data` | load only the `--data-path` directories |
| `-s`, `--seed N` | reproducible output | | `-s`, `--seed N` | reproducible output |
| `-n`, `--repeat N` | render the path N times, each an independent draw | | `-n`, `--repeat N` | render the value N times (up to 1048576), each an independent draw, streamed |
| `--separator S` | between repeated values (default a newline) | | `--separator S` | between repeated values (default a newline) |
| `--format F` | `text` (default), `json`, `ndjson`, `csv` or `sql` — a record's columns, one record per row (json frames them as an array) |
| `--table T` | the INSERT target for `--format sql` (default: the path's last segment, or `records` for an inline template) |
| `--list` | print every path, then exit | | `--list` | print every path, then exit |
| `--version`, `-h`, `--help` | print, then exit | | `--version`, `-h`, `--help` | print, then exit |
`--name value` and `--name=value` both work, a short flag's value attaches or `--name value` and `--name=value` both work, a short flag's value attaches or
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
[Decisions](#decisions); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing [Decisions](#decisions); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing
dir, unknown path), `2` misuse. From a checkout: `go run ./cmd/fejkdata …`. dir, unknown path), `2` misuse — a bad flag, an argument that names neither a
template nor a path, or an inline template that does not compile. From a checkout:
`go run ./cmd/fejkdata …`.
### Your own data ### Your own data
@@ -76,6 +75,67 @@ fejkdata --seed 1 --data-path ./mydata sql
fejkdata --repeat 100 --data-path ./mydata sql > seed.sql fejkdata --repeat 100 --data-path ./mydata sql > seed.sql
``` ```
That `format` string is the free-form spelling — you hand-write the whole row.
For structured output a record writes the row for you.
### Records
A record is a template seen as columns: its fields are the columns, its `format`
the whole. `--format json|ndjson|csv|sql` writes the records; the library's
`Record` (below) hands back the columns. Every column is a string — typed scalars
are on the release checklist, see [`todo.md`](todo.md). Save
`mydata/users.json`:
```json
{
"format": "{first} {last}",
"first": ["Ada", "Bo"],
"last": ["Lovelace", "Ek"]
}
```
```sh
fejkdata --seed 1 --data-path ./mydata --format json users # [{"first":"Bo","last":"Lovelace"}]
fejkdata --seed 1 --data-path ./mydata --format ndjson users # {"first":"Bo","last":"Lovelace"}
fejkdata --seed 1 --data-path ./mydata --format csv users # first,last → Bo,Lovelace
fejkdata --seed 1 --data-path ./mydata --format sql users # INSERT INTO "users" ("first", "last") VALUES ('Bo', 'Lovelace');
fejkdata --seed 1 --data-path ./mydata --format sql --table people users # INSERT into another table
```
`--repeat` streams that many records — `json` frames them as one array document,
`ndjson` writes one object per line, `csv` a row after a header, `sql` one INSERT
per line. Only a category-level template is a record; a field, choice or folder
errors, and so does a `repeat` on
the template itself, which composes the format into one string rather than
projecting columns — ask for more records with `--repeat`. A `repeat` on a column
is fine.
A column carrying a newline keeps it inside the quoted CSV field or the SQL string
literal, so a row can span physical lines: read the stream with a CSV or SQL
parser rather than splitting it on newlines.
The SQL is ANSI — identifiers in double quotes, a literal quote doubled (`''`),
backslashes passed through — so a hyphenated field like `postal-code` stays a
valid identifier. PostgreSQL and SQLite take it as written; MySQL and MariaDB need
`ANSI_QUOTES` and `NO_BACKSLASH_ESCAPES` set first, or they read `"users"` as a
string and a backslash as an escape.
A record written only to emit columns still needs a `format` — the grammar's one
required key — so `"format": ""` carries the fields with an inert format: it
renders nothing by `Fake`, and is compiled only so the tree's fences still run.
The columns are the point, and their facts stay together: two columns that read a
path into one category — `{/currency.code}` and `{/currency.symbol}` — share one
draw of it, so the record is internally consistent. A bare `{/currency}` names no
field, so it keeps drawing on its own. That one draw is also why two
columns may not read overlapping reference *paths*`{/cat.a}` beside
`{/cat.a.b}` is refused, naming the fields to write instead, as
[One draw, one spelling](#one-draw-one-spelling) refuses that pair inside a single
format. A column may not reference the record it belongs to by any spelling: `{/users.first}`
or a bare `{/users}` inside `users` describes a draw other than the columns beside
it, so put a value two columns share in its own category and reference that. A field hold, transform or
operand ties fields together within one column as always (see
[Correlated fields](#correlated-fields) and [Decisions](#decisions)).
## Library ## Library
```sh ```sh
@@ -89,6 +149,12 @@ if err != nil {
} }
v, err := f.Fake("sv_SE.address") // "Kungsvägen 68\n379 17 Stockholm" v, err := f.Fake("sv_SE.address") // "Kungsvägen 68\n379 17 Stockholm"
paths := f.List() // every path Fake accepts, sorted paths := f.List() // every path Fake accepts, sorted
v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call
t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once
v = t.Fake() // render many times, no re-parse
r, err := f.Record("users") // one record: each field a column
s := r.JSON() // {"first":"Ada","last":"Lovelace"}
r, err = f.FakeRecord(`{"format":"{x}","x":["a","b"]}`) // compile + render inline
``` ```
| Option | | | Option | |
@@ -98,6 +164,11 @@ paths := f.List() // every path Fake accepts, sorted
| `WithDataFS(fsys)` | layer an `fs.FS`, such as your own `embed.FS` | | `WithDataFS(fsys)` | layer an `fs.FS`, such as your own `embed.FS` |
| `WithoutShippedData()` | load only what you give | | `WithoutShippedData()` | load only what you give |
A `*Record` carries its columns via `Columns()`, and serializes them with `JSON()`
(one object), `CSVHeader()`/`CSVLine()`, or `SQLInsert(table)` — the shapes the
CLI's `--format` writes. `Record` and `FakeRecord` take a record; a path or
template that is not one — a bare string, a choice, or a folder — errors.
A `*Generator` is safe for concurrent use; a seeded sequence is reproducible only A `*Generator` is safe for concurrent use; a seeded sequence is reproducible only
when drawn from one goroutine. Changing how a value is composed shifts the seeded when drawn from one goroutine. Changing how a value is composed shifts the seeded
stream for that value and everything drawn after it. stream for that value and everything drawn after it.
@@ -110,8 +181,8 @@ work with no data on disk. A directory is a namespace: each JSON file is a
category named after the file, each subdirectory a dot-path segment, so category named after the file, each subdirectory a dot-path segment, so
`mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one. `mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one.
Sources merge in order; matching folders combine, any other clash is won by the Sources merge in order; matching folders combine, any other clash is won by the
last loaded. Names may not use `.`, `|`, `(`, `{`, `}` or `/`; dot-prefixed entries last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`;
are skipped, so a data directory can also be a checkout. dot-prefixed entries are skipped, so a data directory can also be a checkout.
Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`, Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`,
`person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`, `person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`,
@@ -180,9 +251,9 @@ many times — each an independent draw — joined by `separator` (default `""`)
{ "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] } { "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] }
``` ```
Renders e.g. `bar foo baz`. A `separator` without a `repeat` is rejected, and so Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat`,
is a `repeat` that multiplies to more than 1 048 576 renders along any path of a `separator` of `""` (the default), and a `repeat` that multiplies to more than
nested repeats. 1 048 576 renders along any path of nested repeats.
### Options and fields ### Options and fields
@@ -194,9 +265,11 @@ choice naming its item.
### Functions ### Functions
A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad
count, range, country or expression fails fast, and a length, count or decimal count, range, country or expression fails fast; an integer is written plain
place beyond a sane maximum is rejected, so a fat-fingered `hex(2000000000)` (`5`, not `+5` or `05`); bounds are finite; a sample that could only ever emit one
never tries to allocate gigabytes. Every builtin draws only from the seed — a value (`int(5,5)`, `float(1,1,2)`) is rejected naming the text to write instead;
and a length, count or decimal place beyond a sane maximum is rejected, so a
fat-fingered `hex(2000000000)` never tries to allocate gigabytes. Every builtin draws only from the seed — a
time-based id takes its timestamp from the rng, not the clock — so seeded output time-based id takes its timestamp from the rng, not the clock — so seeded output
stays reproducible. stays reproducible.
@@ -243,8 +316,10 @@ hyphenated field can't be an operand.
``` ```
Renders e.g. `19.99 x 3 = 59.97`. An operand that can never be a number (`"abc"`, Renders e.g. `19.99 x 3 = 59.97`. An operand that can never be a number (`"abc"`,
or a choice of such) is rejected at load; one that sometimes is not yields `NaN`, or a choice of such) is rejected at load, as is a division by a constant zero
and a division by zero `Inf` — both print rather than fail. (`1/0`, or a fixed `"0"` field); an operand that sometimes is not a number yields
`NaN`, and a division by one that is not constant `Inf` — both print rather than
fail.
### Transforms ### Transforms
@@ -337,6 +412,29 @@ then costs about what its output costs: an unweighted pick is O(1) whatever the
list's length, a weighted one O(log n), and long formats, deep nesting and many list's length, a weighted one O(log n), and long formats, deep nesting and many
tokens add cost in proportion to the output. tokens add cost in proportion to the output.
## Goals
1. **Valid by construction** — every value passes the check its real consumer
applies; facts that belong together come from one draw, within a value and
across categories.
2. **Text means what it says** — a format renders as written; only `{…}` varies,
random characters included (`{digits(3)}`). One spelling per result; the wrong
one is a load error naming the right one.
3. **Every mistake is a load error**`New` rejects the data and `NewTemplate`
the inline template; on a loaded generator `Fake` fails only for an unknown
path, and `Template.Fake` cannot fail at all.
4. **Zero to a value in one command**`go install`, then `fejkdata sv_SE.person`:
no checkout, no flag. Flags are GNU-form (`--seed 42`, `-n 3`) in any position;
the first custom template needs no escape and no option.
5. **Data lives in JSON** — a builtin only for what data can't express.
6. **Reproducible** — seed in, same stream out; no builtin reads a clock.
7. **Zero dependencies** — standard library only.
8. **Docs index the grammar** — every syntax feature is a heading; every example
runs under test and shows its output; a rule is stated once.
9. **Fast enough to be free** — a value renders in about a microsecond and `New`
parses and validates the whole set once upfront, so generating fixtures stays
noise against a test's own runtime.
## Decisions ## Decisions
- **Options and fields share one namespace.** `format`, `weight`, `repeat` and - **Options and fields share one namespace.** `format`, `weight`, `repeat` and
@@ -352,6 +450,33 @@ tokens add cost in proportion to the output.
naming the double-dash spelling, and `-s=42` is rejected naming both short naming the double-dash spelling, and `-s=42` is rejected naming both short
spellings: `=` belongs to the long form, and reading `=42` as the value would spellings: `=` belongs to the long form, and reading `=42` as the value would
make `-d=./x` a directory named `=./x`. make `-d=./x` a directory named `=./x`.
- **An argument is a template by its shape, not by a flag.** A JSON object, array
or string, or a string carrying a `{` token, is an inline template; anything else
is a path. A name may not contain a brace, a bracket or a quote, so a path can
never collide with any of those spellings, and the leading `[` or `"` is gated on
valid JSON so a stray copied bracket never swallows an argument — it names
nothing, and says so. No `--template` flag is needed. Reserving the characters
whole — though only a leading one could collide — keeps one simple name rule
instead of a leading-position special case. The JSON string is what makes the
library's own advice reachable: the error for an object holding only a format
names `"…"`, and that spelling has to work where it is printed.
- **An inline template skips the cycle fence, and only that one.** `New` proves the
loaded tree acyclic, an inline node is a finite tree of its own, and nothing in
the tree can reference it, so no render of it reaches itself. Every other fence
runs over both, from one `checkScope`.
- **An inline template that does not compile is misuse (exit 2), including a
reference that resolves to nothing** — the whole argument is the spelling under
test, and `NewTemplate` compiles, links and validates as one step. An unknown
*path* stays a runtime error (exit 1): there the argument is well-formed and only
the data is absent.
- **A padded JSON argument is rejected, not trimmed.** Padding is the one place the
two readings disagree — a format string renders it, JSON drops it — so the
spelling that renders is named rather than silently chosen.
- **`FakeTemplate` and `NewTemplate` both stay.** They reach the same value but not
at the same cost: `NewTemplate` pays the compile and validation once and renders
many times, `FakeTemplate` is the one-shot call, and `--repeat` is exactly the
case that needs the first. The pair is `regexp.MustCompile` and `regexp.Match`,
not two spellings of one result.
- **The shipped data is embedded, not discovered.** A directory a machine happens - **The shipped data is embedded, not discovered.** A directory a machine happens
to have would make `--seed 42` machine-dependent. Data still lives in `data/` to have would make `--seed 42` machine-dependent. Data still lives in `data/`
as JSON; `--data-path` layers over it. as JSON; `--data-path` layers over it.
@@ -379,9 +504,65 @@ tokens add cost in proportion to the output.
when that file lacks a field those references read. Accepted: overriding is the when that file lacks a field those references read. Accepted: overriding is the
point of layering, the error names the reference and the field, and the fix is point of layering, the error names the reference and the field, and the fix is
the consumer's file carrying the fields the shipped tree reads. the consumer's file carrying the fields the shipped tree reads.
- **The repeat cap bounds renders, not bytes.** A repeat, alone or nested, may
ask for at most 1 048 576 renders; how large each render is stays what the data
asked for, so `{hex(1048576)}` repeated to the cap is a terabyte, loaded without
complaint. A byte estimate would need every builtin to declare a width to fence
a shape no data comes near, and the harm lands on the author who wrote it.
- **64-bit targets only.** The gate builds amd64, and the buffer sizing a render
pre-computes (renders × bytes) assumes a 64-bit int; on a 32-bit target it could
overflow and panic.
- **A constant zero divisor is a load error; a divisor that is not constant prints
`Inf`.** `1/0` and a fixed `"0"` field are decidable, so they join the
never-numeric operand as a load error; the fold stops where an operand varies,
so `a/(b*c)` with `b` fixed at `0` and `c` varying loads and prints `Inf` every
draw — catching it needs zero-absorbing algebra for a shape nobody writes.
- **In data, a default written out and a constant spelled as a sample are load
errors.** `weight: 1`, `repeat: 1`, `separator: ""`, `int(5,5)`, `float(1,1,2)`,
`+5` and `05` each spell what a shorter form already spells, so each is rejected
naming that form. The CLI's numbers follow the shell instead: `--seed 007` and
`--repeat +3` are 7 and 3, as every command line reads them.
- **Samples say what they emit, transforms what they do.** `{upper(2)}` is two - **Samples say what they emit, transforms what they do.** `{upper(2)}` is two
letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on
whether the argument looks like a number. whether the argument looks like a number.
- **A record is a template seen as columns, not a second schema format.** A
template's `format` composes its fields into one string; `Record` and
`--format` project the same fields as columns. Two views of one dataset, so a
record author writes the same JSON they already know, and a column is the same
field `Fake` renders by dotted path. The `format` is inert to a record — a
record-only template writes `"format": ""` — but it is compiled and fenced, so
a template that loads renders as whichever shape is asked for.
- **A record shares one reference draw per category.** Two columns that reference
one category — `{/currency.code}` beside `{/currency.symbol}` — read one draw of
it, so a record's facts agree the way a template's [correlated
fields](#correlated-fields) do. The draw is one per record, so it spans a
column's `repeat` and nested templates too (one record is one coherent unit);
a bare reference — `{/currency}`, no field — stays an independent draw every
time, the rule a format string already follows. Only references share: a sibling
field is local to its own column, so a `first` column does not silently bind to
a `first` in the column next to it.
The string view of that same template does not share. `Fake` renders each
sibling field as its own expansion, so a `{/currency.code}` field beside a
`{/currency.symbol}` field is two draws and may render `EUR $`; writing both
references in one `format` holds them together, as
[One draw, one spelling](#one-draw-one-spelling) says. The scope is what makes a
row coherent when the columns *are* the output, and there the caller cannot fall
back on one format string. Widening it to every render would change what `Fake`
has emitted since the start, for a correlation a single format already reaches.
- **A record's column set is fixed before the first draw.** Only a category-level
template is a record: a path descending into a field, or naming a folder or a
choice, errors. A tail may pass through a choice whose variants carry different
fields, so the columns — and with them the CSV header written once ahead of every
row — would vary per draw. A fixed column set is what the CSV and `INSERT`
contracts rest on, so the restriction holds even where a particular choice would
happen to agree.
- **The performance gate asserts allocations, not wall-clock time.** `AllocsPerRun`
is deterministic across machines, so a ±10% ceiling does not flake under CI load,
while time varies with the machine and its neighbours. A rendering slowdown
almost always costs an allocation too (a lost pre-size, a per-item map, an extra
copy). The benchmark suite (see Development) reports time for a human, not as a
pass/fail gate.
## Development ## Development
@@ -422,6 +603,8 @@ fejkdata.go Generator, New, options, the embedded data set, List
node.go the node model and JSON -> node compilation node.go the node model and JSON -> node compilation
path.go the dotted-path walk, and proving a path resolves path.go the dotted-path walk, and proving a path resolves
render.go Fake and the recursive renderer (choices, format strings, expansions) render.go Fake and the recursive renderer (choices, format strings, expansions)
record.go records: Record, the JSON/CSV/SQL serializers, and their entry points
inline.go inline templates: Template, NewTemplate, FakeTemplate, and their compile and link
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
hold.go the hold: one draw per expansion for paths and operands, and its fences hold.go the hold: one draw per expansion for paths and operands, and its fences
reference.go reference sigils, and binding references across the tree reference.go reference sigils, and binding references across the tree
@@ -435,4 +618,4 @@ data/ shipped data (JSON), embedded at build: locale folders + a misc
## License ## License
MIT — see [LICENSE](LICENSE). MIT — see [LICENSE](LICENSE). Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).
+57 -20
View File
@@ -2,6 +2,7 @@ package fejkdata
import ( import (
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"math" "math"
"strconv" "strconv"
@@ -9,12 +10,12 @@ import (
"unicode" "unicode"
) )
// maxLen caps sample output lengths (hex, nanoid, base64) and the renders a repeat // maxLen caps sample output lengths (hex, nanoid, base64, digits, upper, lower)
// multiplies to along any path; maxDecimals caps float/calc decimal places. So a // and maxDecimals float/calc decimal places, so a fat-fingered or overflowing
// fat-fingered or overflowing argument fails at New instead of trying to allocate // argument fails at New instead of trying to allocate gigabytes — or panicking —
// gigabytes — or panicking — at render. // at render.
const ( const (
maxLen = 1 << 20 // 1,048,576 chars/bytes maxLen = 1 << 20
maxDecimals = 1024 maxDecimals = 1024
) )
@@ -132,7 +133,7 @@ func transformArg(fields map[string]node, a []string) error {
_, _, err := refShape(leaf) _, _, err := refShape(leaf)
return err return err
} }
return checkArm(leaf, fields) return checkArm(leaf, fields, false)
} }
func transformOperand(a []string) []string { func transformOperand(a []string) []string {
@@ -176,7 +177,7 @@ var asciiFolds = map[rune]string{
func asciiFold(s string) string { func asciiFold(s string) string {
var b strings.Builder var b strings.Builder
for _, r := range s { for _, r := range s {
if r < unicode.MaxASCII { if r <= unicode.MaxASCII {
b.WriteRune(r) b.WriteRune(r)
} else { } else {
b.WriteString(asciiFolds[r]) b.WriteString(asciiFolds[r])
@@ -221,10 +222,31 @@ func randChars(r rng, n int, alphabet string) string {
return string(b) return string(b)
} }
// plainInt parses an integer arg written the one way: no sign, no leading zero.
func plainInt(s string) (int, error) {
n, err := strconv.Atoi(s)
if errors.Is(err, strconv.ErrRange) {
return 0, fmt.Errorf("%q is past the integer range: %w", s, err)
}
if err != nil {
return 0, fmt.Errorf("%q is not an integer", s)
}
if strconv.Itoa(n) != s {
return 0, fmt.Errorf("%q is not a plain integer; write %d", s, n)
}
return n, nil
}
func posIntArg(_ map[string]node, a []string) error { func posIntArg(_ map[string]node, a []string) error {
n, err := strconv.Atoi(a[0]) n, err := plainInt(a[0])
if err != nil || n < 1 { if errors.Is(err, strconv.ErrRange) {
return fmt.Errorf("count %q must be a positive integer", a[0]) return fmt.Errorf("count %q exceeds the maximum %d", a[0], maxLen)
}
if err != nil {
return fmt.Errorf("count %w", err)
}
if n < 1 {
return fmt.Errorf("count %q must be positive", a[0])
} }
if n > maxLen { if n > maxLen {
return fmt.Errorf("count %d exceeds the maximum %d", n, maxLen) return fmt.Errorf("count %d exceeds the maximum %d", n, maxLen)
@@ -233,14 +255,20 @@ func posIntArg(_ map[string]node, a []string) error {
} }
func intRangeArgs(_ map[string]node, a []string) error { func intRangeArgs(_ map[string]node, a []string) error {
lo, e1 := strconv.Atoi(a[0]) lo, err := plainInt(a[0])
hi, e2 := strconv.Atoi(a[1]) if err != nil {
if e1 != nil || e2 != nil { return fmt.Errorf("int(min,max): min %w", err)
return fmt.Errorf("int(min,max) needs integer args, got %q,%q", a[0], a[1]) }
hi, err := plainInt(a[1])
if err != nil {
return fmt.Errorf("int(min,max): max %w", err)
} }
if lo > hi { if lo > hi {
return fmt.Errorf("int(min,max): min %d > max %d", lo, hi) return fmt.Errorf("int(min,max): min %d > max %d", lo, hi)
} }
if lo == hi {
return fmt.Errorf("int(%d,%d) is the constant %d; write it as text", lo, hi, lo)
}
if uint64(hi)-uint64(lo) >= uint64(math.MaxInt64) { // span hi-lo+1 would overflow int -> IntN panic if uint64(hi)-uint64(lo) >= uint64(math.MaxInt64) { // span hi-lo+1 would overflow int -> IntN panic
return fmt.Errorf("int(min,max): range %d..%d is too wide", lo, hi) return fmt.Errorf("int(min,max): range %d..%d is too wide", lo, hi)
} }
@@ -250,19 +278,28 @@ func intRangeArgs(_ map[string]node, a []string) error {
func floatArgs(_ map[string]node, a []string) error { func floatArgs(_ map[string]node, a []string) error {
lo, e1 := strconv.ParseFloat(a[0], 64) lo, e1 := strconv.ParseFloat(a[0], 64)
hi, e2 := strconv.ParseFloat(a[1], 64) hi, e2 := strconv.ParseFloat(a[1], 64)
dp, e3 := strconv.Atoi(a[2]) if e1 != nil || e2 != nil {
if e1 != nil || e2 != nil || e3 != nil { return fmt.Errorf("float(min,max,dp) needs numeric bounds, got %q,%q", a[0], a[1])
return fmt.Errorf("float(min,max,dp) needs numeric args, got %q,%q,%q", a[0], a[1], a[2]) }
dp, err := plainInt(a[2])
if err != nil {
return fmt.Errorf("float(min,max,dp): decimals %w", err)
}
if math.IsNaN(lo) || math.IsNaN(hi) || math.IsInf(lo, 0) || math.IsInf(hi, 0) {
return fmt.Errorf("float(min,max,dp) needs finite bounds, got %q,%q", a[0], a[1])
} }
if lo > hi { if lo > hi {
return fmt.Errorf("float(min,max,dp): min %v > max %v", lo, hi) return fmt.Errorf("float(min,max,dp): min %v > max %v", lo, hi)
} }
if math.IsInf(hi-lo, 0) { // an overflowing span would render as "+Inf"
return fmt.Errorf("float(min,max,dp): range %v..%v is too wide", lo, hi)
}
if dp < 0 || dp > maxDecimals { if dp < 0 || dp > maxDecimals {
return fmt.Errorf("float(min,max,dp): decimals %d out of range 0..%d", dp, maxDecimals) return fmt.Errorf("float(min,max,dp): decimals %d out of range 0..%d", dp, maxDecimals)
} }
if lo == hi {
return fmt.Errorf("float(%s,%s,%d) is the constant %q; write it as text", a[0], a[1], dp, strconv.FormatFloat(lo, 'f', dp, 64))
}
if math.IsInf(hi-lo, 0) { // an overflowing span would render as "+Inf"
return fmt.Errorf("float(min,max,dp): range %v..%v is too wide", lo, hi)
}
return nil return nil
} }
+70 -2
View File
@@ -78,14 +78,82 @@ func checkCalc(fields map[string]node, args []string) error {
return fmt.Errorf("calc(%q): operand %q is never a number: it renders %q", args[0], name, text) return fmt.Errorf("calc(%q): operand %q is never a number: it renders %q", args[0], name, text)
} }
} }
if divisor, zero := constantZeroDivisor(expr, fields); zero {
return fmt.Errorf("calc(%q) divides by %s, which is always zero", args[0], divisor)
}
if len(args) == 2 { if len(args) == 2 {
if dp, err := strconv.Atoi(args[1]); err != nil || dp < 0 || dp > maxDecimals { dp, err := plainInt(args[1])
return fmt.Errorf("calc decimals %q must be an integer in 0..%d", args[1], maxDecimals) if err != nil {
return fmt.Errorf("calc decimals %w", err)
}
if dp < 0 || dp > maxDecimals {
return fmt.Errorf("calc decimals %d must be in 0..%d", dp, maxDecimals)
} }
} }
return nil return nil
} }
// constantZeroDivisor finds a division whose right side is a constant zero: number
// literals and fixed operands folded, anything that varies left unknown.
func constantZeroDivisor(n calcNode, fields map[string]node) (string, bool) {
switch n := n.(type) {
case calcNeg:
return constantZeroDivisor(n.x, fields)
case calcBin:
if n.op == '/' {
if v, known := constantValue(n.r, fields); known && v == 0 {
return calcText(n.r), true
}
}
if d, zero := constantZeroDivisor(n.l, fields); zero {
return d, true
}
return constantZeroDivisor(n.r, fields)
}
return "", false
}
// constantValue evaluates an expression whose every operand is fixed.
func constantValue(n calcNode, fields map[string]node) (float64, bool) {
switch n := n.(type) {
case calcNum:
return float64(n), true
case calcVar:
t, ok := fields[string(n)].(*template)
if !ok || !t.fixed || t.repeat > 1 {
return 0, false
}
v, err := strconv.ParseFloat(strings.TrimSpace(t.lit), 64)
return v, err == nil
case calcNeg:
v, ok := constantValue(n.x, fields)
return -v, ok
case calcBin:
l, lok := constantValue(n.l, fields)
r, rok := constantValue(n.r, fields)
if !lok || !rok {
return 0, false
}
return calcBin{n.op, calcNum(l), calcNum(r)}.eval(nil), true
}
return 0, false
}
// calcText spells an expression node the way an author would read it.
func calcText(n calcNode) string {
switch n := n.(type) {
case calcNum:
return strconv.FormatFloat(float64(n), 'f', -1, 64)
case calcVar:
return string(n)
case calcNeg:
return "-" + calcText(n.x)
case calcBin:
return "(" + calcText(n.l) + " " + string(n.op) + " " + calcText(n.r) + ")"
}
return "?"
}
// neverNumeric reports a node no render of which is a number: fixed text that does // neverNumeric reports a node no render of which is a number: fixed text that does
// not parse, or a choice of only such items. text is one such render. // not parse, or a choice of only such items. text is one such render.
func neverNumeric(n node) (text string, never bool) { func neverNumeric(n node) (text string, never bool) {
+25
View File
@@ -245,3 +245,28 @@ func TestCalcOverANeverNumericOperandIsRejected(t *testing.T) {
} }
} }
} }
func TestCalcConstantZeroDivisorIsRejected(t *testing.T) {
for src, want := range map[string]string{
`"{calc(1/0)}"`: "divides by 0",
`"{calc(2/(1-1))}"`: "divides by (1 - 1)",
`{"format":"{calc(x/y)}","x":"1","y":"0"}`: "divides by y",
`{"format":"{calc(x/(y*2))}","x":"1","y":" 0 "}`: "divides by (y * 2)",
`{"format":"{calc((a/0)+b)}","a":"1","b":"2"}`: "divides by 0",
`{"format":"{calc(a/-0)}","a":"1"}`: "divides by -0",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want the constant zero divisor rejected naming %q", src, err, want)
}
}
if _, err := compile(parse(t, `{"format":"{calc(a/(b*c))}","a":"1","b":"0","c":["1","2"]}`)); err != nil {
t.Errorf("compile(a/(b*c)) = %v, want a divisor that varies accepted", err)
}
f := engine(1)
for i := 0; i < 50; i++ {
if got := mustRender(t, f, `{"format":"{calc(x/y)}","x":"1","y":["0","1"]}`); got == "+Inf" {
return
}
}
t.Fatal("a sometimes-zero divisor never printed +Inf in 50 draws")
}
+245 -31
View File
@@ -8,44 +8,68 @@
package main package main
import ( import (
"bufio"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"os" "os"
"runtime/debug" "runtime/debug"
"sort"
"strconv" "strconv"
"strings" "strings"
"gitea.larvit.se/larvit/fejkdata" "gitea.larvit.se/larvit/fejkdata"
) )
const usage = `Usage: fejkdata [flags] <path> const usage = `Usage: fejkdata [flags] <path|template>
<path> a category, or a dotted path into one (person, person.last) <path> a category, or a dotted path into one (person, person.last)
<template> a format string or JSON value to render inline, e.g.
'name: {/sv_SE.person.last}' or '{"format":"{x}","x":["bosse","lina"]}'
An argument containing a { token, or a JSON object, array or string, is a
template; any other argument is a path (a path never contains a brace, a bracket
or a quote). Templates reach the data by reference from the root —
{/sv_SE.person.last} — whether the data is shipped or layered with --data-path. An
argument carrying a bracket, a closing brace or a quote but no valid JSON names
neither.
With --format json, ndjson, csv or sql the argument must name a record — a
template whose fields are its columns — and the rows are written as one JSON
array, one JSON object per line, one CSV row (after a header), or one INSERT.
-d, --data-path D a data directory to layer over the shipped data (repeatable; last wins on a clash) -d, --data-path D a data directory to layer over the shipped data (repeatable; last wins on a clash)
--format F output form: text (default), json, ndjson, csv or sql
-h, --help print this help, then exit -h, --help print this help, then exit
--list list the paths the data offers, then exit --list list the paths the data offers, then exit
--no-shipped-data load only the --data-path directories --no-shipped-data load only the --data-path directories
-n, --repeat N render the path N times (default 1) -n, --repeat N render the value N times, 1..1048576 (default 1)
-s, --seed N seed for reproducible output -s, --seed N seed for reproducible output
--separator S string between repeated values (default newline) --separator S string between repeated values (default newline)
--table T the INSERT target for --format sql (default: the path's last segment, or records for an inline template)
--version print the version, then exit --version print the version, then exit
Flags may come before or after <path>; -- ends the flags. A short flag's value Flags may come before or after <path|template>; -- ends the flags. A short flag's
attaches or follows (-n3, -n 3); short flags bundle (-hn 3). value attaches or follows (-n3, -n 3); short flags bundle (-hn 3).
` `
type invocation struct { type invocation struct {
dirs []string dirs []string
format string
formatSet bool
help bool help bool
list bool list bool
noShipped bool noShipped bool
paths []string paths []string
repeat int repeat int
repeatSet bool
seed uint64 seed uint64
seeded bool seeded bool
separator string separator string
separatorSet bool
table string
tableSet bool
version bool version bool
} }
@@ -58,15 +82,16 @@ type flagDef struct {
var flagDefs = []flagDef{ var flagDefs = []flagDef{
{"data-path", "d", true, func(in *invocation, v string) error { in.dirs = append(in.dirs, v); return nil }}, {"data-path", "d", true, func(in *invocation, v string) error { in.dirs = append(in.dirs, v); return nil }},
{"format", "", true, func(in *invocation, v string) error { in.format, in.formatSet = v, true; return nil }},
{"help", "h", false, func(in *invocation, _ string) error { in.help = true; return nil }}, {"help", "h", false, func(in *invocation, _ string) error { in.help = true; return nil }},
{"list", "", false, func(in *invocation, _ string) error { in.list = true; return nil }}, {"list", "", false, func(in *invocation, _ string) error { in.list = true; return nil }},
{"no-shipped-data", "", false, func(in *invocation, _ string) error { in.noShipped = true; return nil }}, {"no-shipped-data", "", false, func(in *invocation, _ string) error { in.noShipped = true; return nil }},
{"repeat", "n", true, func(in *invocation, v string) error { {"repeat", "n", true, func(in *invocation, v string) error {
n, err := strconv.Atoi(v) n, err := strconv.Atoi(v)
if err != nil || n < 1 { if err != nil || n < 1 || n > fejkdata.MaxRepeat {
return fmt.Errorf("--repeat needs a positive integer, got %q", v) return fmt.Errorf("--repeat needs an integer in 1..%d, got %q", fejkdata.MaxRepeat, v)
} }
in.repeat = n in.repeat, in.repeatSet = n, true
return nil return nil
}}, }},
{"seed", "s", true, func(in *invocation, v string) error { {"seed", "s", true, func(in *invocation, v string) error {
@@ -77,7 +102,8 @@ var flagDefs = []flagDef{
in.seed, in.seeded = n, true in.seed, in.seeded = n, true
return nil return nil
}}, }},
{"separator", "", true, func(in *invocation, v string) error { in.separator = v; return nil }}, {"separator", "", true, func(in *invocation, v string) error { in.separator, in.separatorSet = v, true; return nil }},
{"table", "", true, func(in *invocation, v string) error { in.table, in.tableSet = v, true; return nil }},
{"version", "", false, func(in *invocation, _ string) error { in.version = true; return nil }}, {"version", "", false, func(in *invocation, _ string) error { in.version = true; return nil }},
} }
@@ -124,8 +150,8 @@ func splitFlags(arg string) ([]flagArg, error) {
} }
var flags []flagArg var flags []flagArg
letters := arg[1:] letters := arg[1:]
for i := 0; i < len(letters); i++ { for i, r := range letters {
letter := letters[i : i+1] letter := string(r)
def := flagByShort(letter) def := flagByShort(letter)
if def == nil { if def == nil {
return nil, fmt.Errorf("unknown flag -%s", letter) return nil, fmt.Errorf("unknown flag -%s", letter)
@@ -134,7 +160,7 @@ func splitFlags(arg string) ([]flagArg, error) {
flags = append(flags, flagArg{def: def}) flags = append(flags, flagArg{def: def})
continue continue
} }
rest := letters[i+1:] rest := letters[i+len(letter):]
if strings.HasPrefix(rest, "=") { if strings.HasPrefix(rest, "=") {
return nil, fmt.Errorf("-%s takes its value attached (-%s%s) or next (-%s %s); = belongs to --%s=%s", return nil, fmt.Errorf("-%s takes its value attached (-%s%s) or next (-%s %s); = belongs to --%s=%s",
letter, letter, rest[1:], letter, rest[1:], def.long, rest[1:]) letter, letter, rest[1:], letter, rest[1:], def.long, rest[1:])
@@ -145,7 +171,7 @@ func splitFlags(arg string) ([]flagArg, error) {
} }
func parseArgs(argv []string) (invocation, error) { func parseArgs(argv []string) (invocation, error) {
in := invocation{repeat: 1, separator: "\n"} in := invocation{repeat: 1, separator: "\n", format: "text"}
for i := 0; i < len(argv); i++ { for i := 0; i < len(argv); i++ {
arg := argv[i] arg := argv[i]
if arg == "--" { if arg == "--" {
@@ -183,17 +209,81 @@ func parseArgs(argv []string) (invocation, error) {
return in, nil return in, nil
} }
// check rejects a flag combination that cannot run. // recordFormat is one way to write a record out: the line each record renders,
func (in invocation) check() error { // the header that precedes the first one, and the open/close frame plus the
if in.list && len(in.paths) > 0 { // between-record separator a document form needs.
return errors.New("--list takes no path") type recordFormat struct {
header func(*fejkdata.Record) string
line func(r *fejkdata.Record, table string) string
open string
close string
sep string
}
// recordFormats is every --format that writes records. json frames the records
// as one array document; ndjson is the same column, one object per line.
var recordFormats = map[string]recordFormat{
"csv": {header: (*fejkdata.Record).CSVHeader, line: func(r *fejkdata.Record, _ string) string { return r.CSVLine() }, sep: "\n"},
"json": {line: jsonLine, open: "[", close: "]", sep: ",\n"},
"ndjson": {line: jsonLine, sep: "\n"},
"sql": {line: func(r *fejkdata.Record, table string) string { return r.SQLInsert(table) }, sep: "\n"},
}
func jsonLine(r *fejkdata.Record, _ string) string { return r.JSON() }
// writesRecords reports whether the format writes records rather than plain text.
func (in invocation) writesRecords() bool {
return in.format != "text"
}
// formatNames lists the --format values, text included, for the misuse error.
func formatNames() string {
names := []string{"text"}
for name := range recordFormats {
names = append(names, name)
} }
if !in.list && len(in.paths) != 1 { sort.Strings(names)
return fmt.Errorf("expected one path, got %d", len(in.paths)) return strings.Join(names[:len(names)-1], ", ") + " or " + names[len(names)-1]
}
// checkFlags rejects a flag value or combination that cannot run.
func (in invocation) checkFlags() error {
if _, ok := recordFormats[in.format]; !ok && in.format != "text" {
return fmt.Errorf("--format takes %s, got %q", formatNames(), in.format)
}
if in.tableSet && in.table == "" {
return errors.New("--table names the INSERT target, so it cannot be empty")
}
if in.tableSet && in.format != "sql" {
return errors.New("--table names the INSERT target, so it needs --format sql")
}
if in.writesRecords() && in.separatorSet {
return errors.New("--separator joins text values, so it has no effect with --format " + in.format)
} }
return nil return nil
} }
// check rejects a flag combination or an argument that cannot run, and reports
// what the argument names, so its shape is settled before any data is read.
func (in invocation) check() (argKind, error) {
if err := in.checkFlags(); err != nil {
return argPath, err
}
if in.list && len(in.paths) > 0 {
return argPath, errors.New("--list takes no path")
}
if in.list && (in.repeatSet || in.separatorSet || in.formatSet || in.tableSet) {
return argPath, errors.New("--list takes no --repeat, --separator, --format or --table")
}
if in.list {
return argPath, nil
}
if len(in.paths) != 1 {
return argPath, fmt.Errorf("expected one path or template, got %d", len(in.paths))
}
return classify(in.paths[0])
}
func (in invocation) options() []fejkdata.Option { func (in invocation) options() []fejkdata.Option {
var opts []fejkdata.Option var opts []fejkdata.Option
if in.noShipped { if in.noShipped {
@@ -208,17 +298,137 @@ func (in invocation) options() []fejkdata.Option {
return opts return opts
} }
// values renders the path repeat times, joined by the separator. // write streams the argument's renders to w, repeat of them joined by the
func (in invocation) values(f *fejkdata.Generator) (string, error) { // separator and ended by a newline. A value that renders once renders every time,
vals := make([]string, in.repeat) // so a render failure comes before anything is written; a write failure surfaces
for i := range vals { // from Flush, bufio keeping the first one.
v, err := f.Fake(in.paths[0]) func (in invocation) write(f *fejkdata.Generator, kind argKind, w io.Writer) error {
if in.writesRecords() {
return in.writeRecords(f, kind, w)
}
draw, err := in.textDraw(f, kind, in.paths[0])
if err != nil { if err != nil {
return "", err return err
} }
vals[i] = v out := bufio.NewWriter(w)
for i := 0; i < in.repeat; i++ {
v, err := draw()
if err != nil {
return err
} }
return strings.Join(vals, in.separator), nil if i > 0 {
out.WriteString(in.separator)
}
out.WriteString(v)
}
out.WriteString("\n")
return out.Flush()
}
// textDraw builds what one text render yields: the value, from a path or an
// inline template.
func (in invocation) textDraw(f *fejkdata.Generator, kind argKind, arg string) (func() (string, error), error) {
if kind != argTemplate {
return func() (string, error) { return f.Fake(arg) }, nil
}
t, err := f.NewTemplate(arg)
if err != nil {
return nil, templateError{err}
}
return func() (string, error) { return t.Fake(), nil }, nil
}
// recordStream builds the record drawer for the argument, plus the INSERT table
// a sql format names.
func (in invocation) recordStream(f *fejkdata.Generator, kind argKind, arg string) (func() (*fejkdata.Record, error), string, error) {
record := func() (*fejkdata.Record, error) { return f.Record(arg) }
if kind == argTemplate {
t, err := f.NewRecordTemplate(arg)
if err != nil {
return nil, "", templateError{err}
}
record = func() (*fejkdata.Record, error) { return t.Fake(), nil }
}
table := in.table
if table == "" {
table = defaultTable(arg, kind)
}
return record, table, nil
}
// writeRecords streams a record per line in the chosen format, framing a document
// form with its open/close brackets and a header preceding the first record.
func (in invocation) writeRecords(f *fejkdata.Generator, kind argKind, w io.Writer) error {
record, table, err := in.recordStream(f, kind, in.paths[0])
if err != nil {
return err
}
format := recordFormats[in.format]
out := bufio.NewWriter(w)
if format.open != "" {
out.WriteString(format.open)
out.WriteByte('\n')
}
for i := 0; i < in.repeat; i++ {
r, err := record()
if err != nil {
return err
}
if i == 0 && format.header != nil {
out.WriteString(format.header(r))
out.WriteByte('\n')
}
if i > 0 {
out.WriteString(format.sep)
}
out.WriteString(format.line(r, table))
}
if format.close != "" {
out.WriteByte('\n')
out.WriteString(format.close)
}
out.WriteByte('\n')
return out.Flush()
}
// defaultTable names the INSERT target when --table is absent: the path's last
// segment, or "records" for an inline template that sits in no folder.
func defaultTable(arg string, kind argKind) string {
if kind == argTemplate {
return "records"
}
segments := strings.Split(arg, ".")
return segments[len(segments)-1]
}
// templateError marks a render failure that is the argument's own fault — an
// inline template that does not compile. run reports it as misuse (exit 2, with a
// pointer to --help), unlike an unknown path, which is a runtime error (exit 1).
type templateError struct{ error }
func (e templateError) Unwrap() error { return e.error }
type argKind int
const (
argPath argKind = iota
argTemplate
)
// classify reads what a positional argument names by its shape: a { token, or a
// JSON object, array or string, is an inline template; anything else is a path.
func classify(arg string) (argKind, error) {
if strings.ContainsRune(arg, '{') || (isJSONStart(strings.TrimSpace(arg)) && json.Valid([]byte(arg))) {
return argTemplate, nil
}
if i := strings.IndexAny(arg, `[]}"`); i >= 0 {
return argPath, fmt.Errorf("%q holds a %q, which no path may, and it is not valid JSON, so it names no template either", arg, arg[i:i+1])
}
return argPath, nil
}
func isJSONStart(arg string) bool {
return strings.HasPrefix(arg, "[") || strings.HasPrefix(arg, `"`)
} }
func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) }
@@ -237,7 +447,8 @@ func run(args []string, stdout, stderr io.Writer) int {
fmt.Fprintln(stdout, "fejkdata "+buildVersion()) fmt.Fprintln(stdout, "fejkdata "+buildVersion())
return 0 return 0
} }
if err := in.check(); err != nil { kind, err := in.check()
if err != nil {
return misuse(stderr, err) return misuse(stderr, err)
} }
f, err := fejkdata.New(in.options()...) f, err := fejkdata.New(in.options()...)
@@ -254,17 +465,20 @@ func run(args []string, stdout, stderr io.Writer) int {
} }
return 0 return 0
} }
out, err := in.values(f) if err := in.write(f, kind, stdout); err != nil {
if err != nil { var te templateError
if errors.As(err, &te) {
return misuse(stderr, te.error)
}
fmt.Fprintln(stderr, err) fmt.Fprintln(stderr, err)
return 1 return 1
} }
fmt.Fprintln(stdout, out)
return 0 return 0
} }
func misuse(stderr io.Writer, err error) int { func misuse(stderr io.Writer, err error) int {
fmt.Fprintf(stderr, "fejkdata: %v\ntry 'fejkdata --help'\n", err) // A library error already names the program, so the prefix is not doubled.
fmt.Fprintf(stderr, "fejkdata: %s\ntry 'fejkdata --help'\n", strings.TrimPrefix(err.Error(), "fejkdata: "))
return 2 return 2
} }
+244 -1
View File
@@ -2,8 +2,11 @@ package main
import ( import (
"bytes" "bytes"
"encoding/csv"
"encoding/json"
"os" "os"
"path/filepath" "path/filepath"
"regexp"
"strings" "strings"
"testing" "testing"
) )
@@ -92,7 +95,7 @@ func TestRunShortFlagValues(t *testing.T) {
{[]string{"-s=42", "-d", svSE, "address"}, "-s42"}, {[]string{"-s=42", "-d", svSE, "address"}, "-s42"},
{[]string{"-s=42", "-d", svSE, "address"}, "--seed=42"}, {[]string{"-s=42", "-d", svSE, "address"}, "--seed=42"},
{[]string{"-d=" + svSE, "address"}, "--data-path="}, {[]string{"-d=" + svSE, "address"}, "--data-path="},
{[]string{"-nd", "3", "-d", svSE, "word"}, `--repeat needs a positive integer, got "d"`}, {[]string{"-nd", "3", "-d", svSE, "word"}, `--repeat needs an integer in 1..1048576, got "d"`},
{[]string{"--seed=", "-d", svSE, "word"}, `--seed needs an unsigned integer, got ""`}, {[]string{"--seed=", "-d", svSE, "word"}, `--seed needs an unsigned integer, got ""`},
} { } {
code, out, errb := runOut(c.args...) code, out, errb := runOut(c.args...)
@@ -301,6 +304,92 @@ func TestRunShippedDataByDefault(t *testing.T) {
} }
} }
func TestClassify(t *testing.T) {
for arg, want := range map[string]argKind{
"sv_SE.person": argPath,
"person.last": argPath,
"name: {x}": argTemplate, // a { token: a path can never carry a brace
`{"format":"x"}`: argTemplate,
`["a","b"]`: argTemplate, // a JSON array carries no brace
`[1, 2]`: argTemplate,
` ["a","b"]`: argTemplate, // padding is the template's own error, not a shape verdict
`"hello"`: argTemplate, // a JSON string, the spelling a format-only object names
} {
got, err := classify(arg)
if err != nil || got != want {
t.Errorf("classify(%q) = %v, %v; want %v", arg, got, err, want)
}
}
for arg, want := range map[string]string{
"[abc]": `holds a "["`,
"[abc].field": `holds a "["`,
"x[1]": `holds a "["`,
"a]b": `holds a "]"`,
"a}b": `holds a "}"`,
`"abc`: `holds a "\""`,
`"a]b`: `holds a "\""`, // the opener the reader typed, not the bracket behind it
} {
_, err := classify(arg)
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("classify(%q) = %v; want it rejected naming %s", arg, err, want)
}
}
}
func TestUsageReferencesResolve(t *testing.T) {
for _, token := range regexp.MustCompile(`\{/[^}]+\}`).FindAllString(usage, -1) {
code, out, errb := runOut("--seed", "1", token)
if code != 0 || strings.TrimSpace(out) == "" {
t.Errorf("usage advertises %s: run = %d, %q, stderr %q", token, code, out, errb)
}
}
}
func TestRunShapeMisuseBeforeLoad(t *testing.T) {
code, _, errb := runOut("--no-shipped-data", "[abc]")
if code != 2 || !strings.Contains(errb, "[abc]") || strings.Contains(errb, "--data-path") {
t.Fatalf("shape misuse with no data = %d, %q; want the shape error before any load", code, errb)
}
}
func TestRunInlineTemplate(t *testing.T) {
code, out, errb := runOut("--seed", "1", "name: {/sv_SE.person.last}")
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline format string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", `{"format":"name: {x}","x":["bosse","lina"]}`)
if code != 0 || (out != "name: bosse\n" && out != "name: lina\n") {
t.Fatalf("inline JSON template = %d, %q, want one name, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", `"name: {/sv_SE.person.last}"`)
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline JSON string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("--seed", "1", "-n", "2", `{digits(1)}`)
if code != 0 || len(strings.Split(strings.TrimRight(out, "\n"), "\n")) != 2 {
t.Fatalf("inline template with --repeat = %d, %q, stderr %q", code, out, errb)
}
}
func TestRunTemplateMisuse(t *testing.T) {
for arg, want := range map[string]string{
"{bad": "unterminated",
"[red,green]": "names no template either",
`{"format":"x"}`: "is a string",
"{/no.such.path}": "no entry",
"x[1]": "names no template either",
` ["a","b"] `: "may not be padded",
} {
code, out, errb := runOut("--seed", "1", arg)
if code != 2 || out != "" || !strings.Contains(errb, "try 'fejkdata --help'") || !strings.Contains(errb, want) {
t.Errorf("run(%q) = %d, %q, %q; want misuse naming %q and --help", arg, code, out, errb, want)
}
if strings.Contains(errb, "fejkdata: fejkdata:") {
t.Errorf("run(%q) doubled the program prefix: %q", arg, errb)
}
}
}
func TestRunNoShippedData(t *testing.T) { func TestRunNoShippedData(t *testing.T) {
code, list, errb := runOut("--no-shipped-data", "-d", svSE, "--list") code, list, errb := runOut("--no-shipped-data", "-d", svSE, "--list")
if code != 0 { if code != 0 {
@@ -355,3 +444,157 @@ func TestRunUnknownFlagIsNamedByRune(t *testing.T) {
t.Errorf("run(-ä) = %d, %q, want the flag named whole", code, errb) t.Errorf("run(-ä) = %d, %q, want the flag named whole", code, errb)
} }
} }
func TestRunEmptyDataPathIsNamed(t *testing.T) {
code, _, errb := runOut("--data-path=", "sv_SE.word")
if code != 1 || !strings.Contains(errb, "empty") {
t.Errorf("run(--data-path=) = %d, %q, want the empty path named", code, errb)
}
}
func recordDir(t *testing.T) string {
t.Helper()
dir := t.TempDir()
content := `{"format":"{first} {last}","first":["Ada","Bo"],"last":["Lovelace","Ek"]}`
if err := os.WriteFile(filepath.Join(dir, "users.json"), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
return dir
}
func TestRunRecordJSON(t *testing.T) {
code, out, errb := runOut("--seed", "1", "--format", "json", "--repeat", "2", "--data-path", recordDir(t), "users")
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
var arr []map[string]string
if err := json.Unmarshal([]byte(out), &arr); err != nil {
t.Fatalf("json output is not one JSON array: %v\n%q", err, out)
}
if len(arr) != 2 || len(arr[0]) != 2 || arr[0]["first"] == "" || arr[0]["last"] == "" {
t.Fatalf("json output = %q, want an array of two records with first and last columns", out)
}
}
func TestRunRecordNDJSON(t *testing.T) {
code, out, errb := runOut("--seed", "1", "--format", "ndjson", "--repeat", "2", "--data-path", recordDir(t), "users")
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
lines := strings.Split(strings.TrimRight(out, "\n"), "\n")
if len(lines) != 2 {
t.Fatalf("ndjson output = %d lines %q, want one object per line", len(lines), out)
}
for _, line := range lines {
var m map[string]string
if err := json.Unmarshal([]byte(line), &m); err != nil || len(m) != 2 {
t.Fatalf("ndjson line %q is not one object: %v", line, err)
}
}
}
func TestRunRecordCSVRoundTrips(t *testing.T) {
code, out, errb := runOut("--seed", "1", "--format", "csv", "--repeat", "3", "--data-path", recordDir(t), "users")
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
lines := strings.Split(strings.TrimRight(out, "\n"), "\n")
if len(lines) != 4 {
t.Fatalf("csv output = %d lines %q, want a header and 3 rows", len(lines), out)
}
if lines[0] != "first,last" {
t.Fatalf("csv header = %q, want first,last", lines[0])
}
r, err := csv.NewReader(strings.NewReader(out)).ReadAll()
if err != nil {
t.Fatalf("csv output is not valid CSV: %v", err)
}
if len(r) != 4 || len(r[0]) != 2 {
t.Fatalf("csv parsed to %v, want 4 records of 2 fields", r)
}
}
func TestRunRecordSQL(t *testing.T) {
code, out, errb := runOut("--seed", "1", "--format", "sql", "--repeat", "2", "--table", "people", "--data-path", recordDir(t), "users")
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
lines := strings.Split(strings.TrimRight(out, "\n"), "\n")
if len(lines) != 2 || !strings.HasPrefix(lines[0], `INSERT INTO "people" ("first", "last") VALUES (`) {
t.Fatalf("sql output = %q, want two INSERT INTO people statements", out)
}
}
func TestRunRecordDefaultTable(t *testing.T) {
code, out, errb := runOut("--seed", "1", "--format", "sql", "--data-path", recordDir(t), "users")
if code != 0 || !strings.HasPrefix(out, `INSERT INTO "users" (`) {
t.Fatalf("default table = %d, %q, want the path's last segment; stderr %q", code, out, errb)
}
}
func TestRunRecordInlineTemplate(t *testing.T) {
code, out, errb := runOut("--seed", "1", "--format", "json", `{"format":"{x}","x":["a","b"]}`)
if code != 0 {
t.Fatalf("inline record = %d, stderr=%q", code, errb)
}
var arr []map[string]string
if err := json.Unmarshal([]byte(out), &arr); err != nil || len(arr) != 1 || (arr[0]["x"] != "a" && arr[0]["x"] != "b") {
t.Fatalf("inline record json = %q, want one record with a column x (err %v)", out, err)
}
}
func TestRunRecordMisuse(t *testing.T) {
for _, c := range []struct {
args []string
want string
}{
{[]string{"--format", "yaml", "users"}, "--format takes csv, json, ndjson, sql or text"},
{[]string{"--format", "json", "--separator", ",", "users"}, "--separator joins text values"},
{[]string{"--table", "t", "users"}, "--table names the INSERT target"},
{[]string{"--format", "json", "--table", "t", "users"}, "--table names the INSERT target"},
{[]string{"--format", "sql", "--table", "", "users"}, "--table names the INSERT target"},
} {
code, out, errb := runOut(c.args...)
if code != 2 || out != "" || !strings.Contains(errb, c.want) {
t.Errorf("run(%v) = %d, %q, %q; want misuse naming %q", c.args, code, out, errb, c.want)
}
}
}
func TestRunRecordOnABareValueIsRuntimeError(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "word.json"), []byte(`["a","b"]`), 0o644); err != nil {
t.Fatal(err)
}
code, _, errb := runOut("--format", "json", "--data-path", dir, "word")
if code != 1 {
t.Fatalf("record on a choice = %d, want exit 1 (runtime error), stderr %q", code, errb)
}
}
func TestRunRecordRejectsATopLevelRepeat(t *testing.T) {
dir := t.TempDir()
content := `{"format":"{a}-","repeat":3,"separator":"|","a":["x","y"]}`
if err := os.WriteFile(filepath.Join(dir, "rep.json"), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
code, _, errb := runOut("--format", "json", "--data-path", dir, "rep")
if code != 1 || !strings.Contains(errb, "carries repeat 3") {
t.Errorf("record on a repeating path = %d, %q; want exit 1 naming the repeat", code, errb)
}
code, _, errb = runOut("--format", "json", content)
if code != 2 || !strings.Contains(errb, "carries repeat 3") {
t.Errorf("record on a repeating inline template = %d, %q; want exit 2 naming the repeat", code, errb)
}
if code, out, _ := runOut("--seed", "1", "--data-path", dir, "rep"); code != 0 || strings.TrimSpace(out) != "x-|x-|x-" {
t.Errorf("text view = %d, %q; want the repeat still composed", code, out)
}
}
func TestRunNumbersFollowTheShell(t *testing.T) {
_, want, _ := runOut("--seed", "7", "--repeat", "3", "sv_SE.word")
code, got, errb := runOut("--seed", "007", "--repeat", "+3", "sv_SE.word")
if code != 0 || got != want {
t.Errorf("run(--seed 007 --repeat +3) = %d, %q, stderr %q; want the same as --seed 7 --repeat 3 %q", code, got, errb, want)
}
}
+9 -8
View File
@@ -10,11 +10,12 @@ import (
) )
// dataSource is one tree to load: an fs.FS and the directory in it to start from. // dataSource is one tree to load: an fs.FS and the directory in it to start from.
// label prefixes file names in errors; path, when set, is a directory on disk that // label prefixes file names in errors; onDisk marks path as a directory that must
// must exist. // exist.
type dataSource struct { type dataSource struct {
fsys fs.FS fsys fs.FS
label string label string
onDisk bool
path string path string
root string root string
} }
@@ -36,10 +37,13 @@ func (s dataSource) name(p string) string {
func loadData(sources []dataSource) (map[string]node, error) { func loadData(sources []dataSource) (map[string]node, error) {
root := map[string]node{} root := map[string]node{}
for _, src := range sources { for _, src := range sources {
if src.path != "" { if src.onDisk {
if src.path == "" {
return nil, fmt.Errorf("a data path is empty")
}
info, err := os.Stat(src.path) info, err := os.Stat(src.path)
if err != nil { if err != nil {
return nil, fmt.Errorf("%s: no such directory", src.path) return nil, err
} }
if !info.IsDir() { if !info.IsDir() {
return nil, fmt.Errorf("%s is not a directory", src.path) return nil, fmt.Errorf("%s is not a directory", src.path)
@@ -64,10 +68,7 @@ func loadData(sources []dataSource) (map[string]node, error) {
if err := checkNoCycles(root); err != nil { if err := checkNoCycles(root); err != nil {
return nil, err return nil, err
} }
if err := checkRepeatReach(root); err != nil { if err := checkScope(treeScope(root)); err != nil {
return nil, err
}
if err := checkBoundLevelsHeld(root); err != nil {
return nil, err return nil, err
} }
return root, nil return root, nil
+27 -8
View File
@@ -28,16 +28,24 @@ import (
//go:embed data //go:embed data
var shippedFS embed.FS var shippedFS embed.FS
// MaxRepeat caps a repeat, and the renders nested repeats multiply to along any
// path; the CLI's --repeat shares it.
const MaxRepeat = 1 << 20
var _ [^uint(0)>>63 - 1]struct{} // 64-bit only, per the README's Decisions
// ErrNoData is returned by New when no source is loaded at all. // ErrNoData is returned by New when no source is loaded at all.
var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS") var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS")
// Generator generates fake data from a loaded namespace tree. Create one with [New]. // Generator generates fake data from a loaded namespace tree. Create one with [New].
// It is safe for concurrent use; a seeded sequence is reproducible only when drawn // It is safe for concurrent use; a seeded sequence is reproducible only when drawn
// from one goroutine. // from one goroutine. The compiled tree is immutable after [New], and Fake,
// NewTemplate and List read it concurrently without a lock.
type Generator struct { type Generator struct {
mu sync.Mutex mu sync.Mutex
rand *session rand *session
categories map[string]node categories map[string]node
records map[node]recordShape
} }
// session is one generator's mutable render state: the seeded rng plus the {seq()} // session is one generator's mutable render state: the seeded rng plus the {seq()}
@@ -72,7 +80,7 @@ func WithSeed(seed uint64) Option {
// layer several; the last wins a name clash. // layer several; the last wins a name clash.
func WithDataPath(dir string) Option { func WithDataPath(dir string) Option {
return func(c *config) { return func(c *config) {
c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, path: dir}) c.sources = append(c.sources, dataSource{fsys: os.DirFS(dir), label: dir, onDisk: true, path: dir})
} }
} }
@@ -109,7 +117,11 @@ func New(opts ...Option) (*Generator, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err) return nil, fmt.Errorf("fejkdata: %w", err)
} }
return &Generator{rand: newRand(c.seed, c.seeded), categories: cats}, nil rng, err := newRand(c.seed, c.seeded)
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Generator{rand: rng, categories: cats}, nil
} }
// List returns the sorted dotted paths Fake can render: every category, the dotted // List returns the sorted dotted paths Fake can render: every category, the dotted
@@ -200,12 +212,19 @@ func join(prefix, name string) string {
return prefix + "." + name return prefix + "." + name
} }
func newRand(seed uint64, seeded bool) *session { // randomBytes seeds an unseeded generator.
r := rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15)) var randomBytes = crand.Read
if !seeded {
func newRand(seed uint64, seeded bool) (*session, error) {
var r *rand.Rand
if seeded {
r = rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15))
} else {
var b [16]byte var b [16]byte
_, _ = crand.Read(b[:]) if _, err := randomBytes(b[:]); err != nil {
return nil, fmt.Errorf("seeding from crypto/rand: %w", err)
}
r = rand.New(rand.NewPCG(binary.LittleEndian.Uint64(b[:8]), binary.LittleEndian.Uint64(b[8:]))) r = rand.New(rand.NewPCG(binary.LittleEndian.Uint64(b[:8]), binary.LittleEndian.Uint64(b[8:])))
} }
return &session{Rand: r, counters: map[string]uint64{}} return &session{Rand: r, counters: map[string]uint64{}}, nil
} }
+22 -2
View File
@@ -1,8 +1,11 @@
package fejkdata package fejkdata
import ( import (
"errors"
"io/fs"
"strings" "strings"
"testing" "testing"
"testing/fstest"
) )
// newGenerator creates a generator over a single data directory, failing on // newGenerator creates a generator over a single data directory, failing on
@@ -38,8 +41,25 @@ func fake(t *testing.T, f *Generator, path string) string {
func TestNewMissingDirectory(t *testing.T) { func TestNewMissingDirectory(t *testing.T) {
_, err := New(WithoutShippedData(), WithDataPath("data/de_DE")) _, err := New(WithoutShippedData(), WithDataPath("data/de_DE"))
if err == nil || !strings.Contains(err.Error(), "de_DE") { if err == nil || !strings.Contains(err.Error(), "de_DE") || !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("New(missing) error = %v, want it to name the path", err) t.Fatalf("New(missing) error = %v, want the real error, naming the path", err)
}
}
func TestNewRejectsAnEmptyDataPath(t *testing.T) {
_, err := New(WithoutShippedData(), WithDataPath(""))
if err == nil || !strings.Contains(err.Error(), "empty") {
t.Fatalf("New(WithDataPath(\"\")) = %v, want the empty path named", err)
}
}
func TestNewReportsAnEntropyFailure(t *testing.T) {
saved := randomBytes
randomBytes = func([]byte) (int, error) { return 0, errors.New("no entropy") }
defer func() { randomBytes = saved }()
_, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"w.json": {Data: []byte(`"x"`)}}))
if err == nil || !strings.Contains(err.Error(), "no entropy") {
t.Fatalf("New() without entropy = %v, want the failure reported", err)
} }
} }
+59 -30
View File
@@ -10,29 +10,36 @@ import (
// visiting keys in sorted order so which of several broken nodes gets reported does // visiting keys in sorted order so which of several broken nodes gets reported does
// not depend on map iteration. // not depend on map iteration.
func walkNodes(root map[string]node, fn func(path string, n node) error) error { func walkNodes(root map[string]node, fn func(path string, n node) error) error {
seen := map[node]bool{} for _, name := range sortedNames(root) {
var visit func(string, node) error if err := eachNode(root[name], name, fn); err != nil {
visit = func(path string, n node) error {
if n == nil || seen[n] {
return nil
}
seen[n] = true
if err := fn(path, n); err != nil {
return err return err
} }
for _, c := range contained(n) { }
return nil
}
// eachNode visits n and every node contained within it once, passing the dot path
// that reaches each. It never crosses a reference edge — a bound {/path} field is
// skipped — so a single inline node is walked on its own.
func eachNode(n node, path string, fn func(path string, n node) error) error {
seen := map[node]bool{}
var visit func(string, node) error
visit = func(path string, m node) error {
if m == nil || seen[m] {
return nil
}
seen[m] = true
if err := fn(path, m); err != nil {
return err
}
for _, c := range contained(m) {
if err := visit(join(path, c.name), c.node); err != nil { if err := visit(join(path, c.name), c.node); err != nil {
return err return err
} }
} }
return nil return nil
} }
for _, name := range sortedNames(root) { return visit(path, n)
if err := visit(name, root[name]); err != nil {
return err
}
}
return nil
} }
// namedNode is a contained child and the segment reaching it; a choice's items carry // namedNode is a contained child and the segment reaching it; a choice's items carry
@@ -158,34 +165,56 @@ func pathLeaves(n node, tail []string) []node {
return out return out
} }
// checkRepeatReach bounds the renders a repeat multiplies to along any root-to-leaf // nodeScope is the set of nodes one validation pass covers: a whole loaded tree,
// path, so nested repeats cannot build what one repeat may not. It runs after // or a single inline node.
// checkNoCycles, whose guarantee is what lets the walk terminate. type nodeScope func(fn func(path string, n node) error) error
func checkRepeatReach(root map[string]node) error {
reach := map[node]int{} func treeScope(root map[string]node) nodeScope {
var of func(n node) int return func(fn func(path string, n node) error) error { return walkNodes(root, fn) }
of = func(n node) int { }
if r, done := reach[n]; done {
func inlineScope(n node) nodeScope {
return func(fn func(path string, m node) error) error { return eachNode(n, "template", fn) }
}
// checkScope runs the per-node fences over a scope, each over the whole scope
// before the next, so which of several broken nodes is reported does not depend on
// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks
// terminate.
func checkScope(s nodeScope) error {
mem := reachMemo{}
if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil {
return err
}
return s(heldCheck)
}
type reachMemo map[node]int
func (m reachMemo) of(n node) int {
if r, done := m[n]; done {
return r return r
} }
r := 1 r := 1
for _, e := range renderEdges(n) { for _, e := range renderEdges(n) {
if c := of(e.to); c > r { if c := m.of(e.to); c > r {
r = c r = c
} }
} }
if t, ok := n.(*template); ok { if t, ok := n.(*template); ok {
r *= t.repeat r *= t.repeat
} }
reach[n] = r m[n] = r
return r return r
} }
return walkNodes(root, func(path string, n node) error {
if t, ok := n.(*template); ok && t.repeat > 1 && of(n) > maxLen { // repeatCheck bounds the renders a repeat multiplies to along any root-to-leaf
return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, of(n), maxLen) // path, so nested repeats cannot build what one repeat may not.
func repeatCheck(path string, n node, mem reachMemo) error {
if t, ok := n.(*template); ok && t.repeat > 1 && mem.of(n) > MaxRepeat {
return fmt.Errorf("%s: repeat %d multiplies to %d renders along one path, above the maximum %d", path, t.repeat, mem.of(n), MaxRepeat)
} }
return nil return nil
})
} }
// checkNoCycles rejects a reference cycle: a node whose rendering can reach itself // checkNoCycles rejects a reference cycle: a node whose rendering can reach itself
+20 -15
View File
@@ -6,15 +6,12 @@ import (
"strings" "strings"
) )
// checkBoundLevelsHeld rejects every route to a held name except the ones that read // heldCheck rejects every route to a held name except the ones that read its draw.
// its draw. An expansion holds one draw of that name; anything else that renders it // An expansion holds one draw of that name; anything else that renders it draws
// draws again, and the two disagree. checkNoOverlap settles the spellings within one // again, and the two disagree. checkNoOverlap settles the spellings within one
// format (a token, an operand); this settles the rest — a reference, whether it // format (a token, an operand); this settles the rest — a reference, whether it
// sits in that format or in anything the format renders, however deep. // sits in that format or in anything the format renders, however deep.
// func heldCheck(path string, n node) error {
// It runs after checkNoCycles, whose guarantee is what lets the walk terminate.
func checkBoundLevelsHeld(root map[string]node) error {
return walkNodes(root, func(path string, n node) error {
t, ok := n.(*template) t, ok := n.(*template)
if !ok || len(t.held) == 0 { if !ok || len(t.held) == 0 {
return nil return nil
@@ -26,7 +23,6 @@ func checkBoundLevelsHeld(root map[string]node) error {
} }
} }
return nil return nil
})
} }
// heldHeads lists a template's held names, operand heads first, then paths, each // heldHeads lists a template's held names, operand heads first, then paths, each
@@ -271,11 +267,20 @@ type draws struct {
// gives one value, and a shown operand is the operand computed. Every other name is // gives one value, and a shown operand is the operand computed. Every other name is
// drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and // drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and
// linkRefs prove every step, so the walk cannot fail. // linkRefs prove every step, so the walk cannot fail.
func readField(s *session, t *template, held *draws, a arm) string { func readField(s *session, t *template, held, refScope *draws, a arm) string {
if !t.held[a.key] { if !t.held[a.key] {
return render(s, t.fields[a.key]) if len(a.tail) > 0 {
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
} }
if v, read := held.value[a.path]; read { return render(s, t.fields[a.key], refScope)
}
// A reference that reads a path reads the caller's scope, so its draw outlives
// this expansion; a sibling, and a reference read whole, stay local to it.
d := held
if isRef(a.key) && refScope != nil && len(a.tail) > 0 {
d = refScope
}
if v, read := d.value[a.path]; read {
return v return v
} }
var v string var v string
@@ -287,16 +292,16 @@ func readField(s *session, t *template, held *draws, a arm) string {
if consumed := len(a.tail) - len(rest); consumed > 0 { if consumed := len(a.tail) - len(rest); consumed > 0 {
key = a.steps[consumed-1] key = a.steps[consumed-1]
} }
n, drew := held.variant[key] n, drew := d.variant[key]
if !drew { if !drew {
n = drawn(s, c) n = drawn(s, c)
held.variant[key] = n d.variant[key] = n
} }
return []node{n}, nil return []node{n}, nil
}, },
leaf: func(n node) error { v = render(s, n); return nil }, leaf: func(n node) error { v = render(s, n, refScope); return nil },
}) })
held.value[a.path] = v d.value[a.path] = v
return v return v
} }
+10
View File
@@ -711,3 +711,13 @@ func TestRepeatedBareTokenOfAHeldNameIsRejected(t *testing.T) {
} }
} }
} }
func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) {
tm, ok := compiled(t, `{"format":"{w}","w":{"format":"{x}","x":"1"}}`).(*template)
if !ok {
t.Fatal("not a template")
}
mustPanic(t, "unheld arm with a path", func() {
readField(engine(1).rand, tm, nil, nil, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"})
})
}
+88
View File
@@ -0,0 +1,88 @@
package fejkdata
import (
"encoding/json"
"fmt"
"strings"
)
// Template is an inline template compiled, referenced and validated against a
// generator's loaded data once, ready to render many times with [Template.Fake].
// It is safe for concurrent use: Fake serializes on its generator's lock, so a
// seeded sequence is reproducible only when a generator — and its templates — are
// drawn from one goroutine.
type Template struct {
g *Generator
n node
}
// Fake renders the template with one draw.
func (t *Template) Fake() string {
t.g.mu.Lock()
defer t.g.mu.Unlock()
return render(t.g.rand, t.n, nil)
}
// NewTemplate compiles an inline template — a format string or a JSON value — and
// binds its references against the loaded tree, so repeated renders pay the
// compile and validation once. It shares [New]'s guarantees: a bad template errors
// here, and rendering cannot fail.
func (f *Generator) NewTemplate(input string) (*Template, error) {
n, err := compileInput(input)
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
scope := inlineScope(n)
if err := linkNodeRefs(scope, f.categories); err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
if err := checkScope(scope); err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
}
return &Template{g: f, n: n}, nil
}
// FakeTemplate compiles and renders an inline template in one call. It is
// [NewTemplate] then [Template.Fake]; to render the same template many times, hold
// the *Template and call its Fake.
func (f *Generator) FakeTemplate(input string) (string, error) {
t, err := f.NewTemplate(input)
if err != nil {
return "", err
}
return t.Fake(), nil
}
// compileInput compiles an inline template: a JSON value, or a bare format string
// when the input is not JSON.
func compileInput(input string) (node, error) {
var raw any
if err := json.Unmarshal([]byte(input), &raw); err != nil {
return compile(input)
}
if trimmed := strings.TrimSpace(input); trimmed != input {
return nil, fmt.Errorf("a JSON template may not be padded with spaces, which a format string would render; write %s", trimmed)
}
return compile(raw)
}
// linkNodeRefs binds the references in an inline node's templates against the
// loaded tree.
func linkNodeRefs(scope nodeScope, root map[string]node) error {
return scope(func(path string, m node) error {
t, ok := m.(*template)
if !ok {
return nil
}
for _, name := range refTokens(t.format) {
sigil, rest, err := refShape(name)
if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
if sigil != "/" {
return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest)
}
}
return linkTemplateRefs(nil, path, t, root)
})
}
+159
View File
@@ -0,0 +1,159 @@
package fejkdata
import (
"strings"
"testing"
)
func tmpl(t *testing.T, f *Generator, input string) string {
t.Helper()
s, err := f.FakeTemplate(input)
if err != nil {
t.Fatalf("FakeTemplate(%q): %v", input, err)
}
return s
}
func shipped(t *testing.T, opts ...Option) *Generator {
t.Helper()
f, err := New(append([]Option{WithSeed(1)}, opts...)...)
if err != nil {
t.Fatal(err)
}
return f
}
func TestFakeTemplateFormatString(t *testing.T) {
f := shipped(t)
got := tmpl(t, f, "name: {/sv_SE.person.last}")
if !strings.HasPrefix(got, "name: ") || strings.HasSuffix(got, " ") || strings.Contains(got, "{") {
t.Fatalf("FakeTemplate = %q, want a rendered last name after the prefix", got)
}
}
func TestFakeTemplateJSONObject(t *testing.T) {
f := shipped(t)
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[tmpl(t, f, `{"format":"name: {x}","x":["bosse","lina"]}`)] = true
}
if !seen["name: bosse"] || !seen["name: lina"] || len(seen) != 2 {
t.Fatalf("JSON template produced %v, want both names", seen)
}
}
func TestFakeTemplateJSONArray(t *testing.T) {
f := shipped(t)
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[tmpl(t, f, `["foo","bar","baz"]`)] = true
}
if len(seen) != 3 {
t.Fatalf("JSON array choice produced %v, want three items", seen)
}
}
func TestFakeTemplateCorrelatedReferences(t *testing.T) {
dir := writeData(t, map[string]string{
"person": `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"}]`,
})
f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 100; i++ {
got := tmpl(t, f, "{/person.first} {/person.last}")
if got != "Ada Lovelace" && got != "Bo Ek" {
t.Fatalf("correlated references = %q, want one person's first and last", got)
}
}
}
func TestFakeTemplateDeterministic(t *testing.T) {
a, b := shipped(t), shipped(t)
for i := 0; i < 20; i++ {
in := "row: {/misc.uuid} {digits(3)}"
if x, y := tmpl(t, a, in), tmpl(t, b, in); x != y {
t.Fatalf("same seed diverged: %q != %q", x, y)
}
}
}
func TestFieldlessTokenHint(t *testing.T) {
f := shipped(t)
_, err := f.FakeTemplate(`{sv_SE.person.last}`)
if err == nil || !strings.Contains(err.Error(), "write {/sv_SE.person.last}") {
t.Fatalf("FakeTemplate(bare token) = %v, want a hint naming {/sv_SE.person.last}", err)
}
_, err = f.FakeTemplate(`{"format":"{x}","repeat":2}`)
if err == nil || !strings.Contains(err.Error(), `this template has none — write {/x}`) {
t.Errorf("FakeTemplate(fieldless object) = %v, want the hint without calling it a bare string", err)
}
// A hint is only a drop-in where the name is the whole token: {/x} inside a
// transform or an alternation renders a different value, so none is offered.
for _, input := range []string{`{ /sv_SE.person.last }`, "{lowercase(x)}", "{x|y}"} {
_, err := f.FakeTemplate(input)
if err == nil || strings.Contains(err.Error(), "write {") {
t.Errorf("FakeTemplate(%q) = %v, want no hint naming a spelling that means something else", input, err)
}
}
}
func TestFakeTemplateErrors(t *testing.T) {
f := shipped(t)
for _, c := range []struct {
input string
want string
}{
{`{"x":"Q"}`, "missing string \"format\""},
{`"{x}"`, `no field "x"`},
{`"{digits(0)}"`, "must be positive"},
{`name: {/no.such.path}`, "no entry"},
{`name: {..nope}`, "write {/nope}"},
{`{"format":"x"}`, "is a string"},
{`{/misc.country} {/misc.country.alpha2}`, "renders a level"},
{`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"},
} {
_, err := f.FakeTemplate(c.input)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("FakeTemplate(%q) = %v, want an error containing %q", c.input, err, c.want)
}
}
}
func TestFakeTemplateJSONString(t *testing.T) {
f := shipped(t)
got := tmpl(t, f, `"name: {/sv_SE.person.last}"`)
if !strings.HasPrefix(got, "name: ") || strings.Contains(got, "{") {
t.Fatalf("FakeTemplate(JSON string) = %q, want a rendered last name after the prefix", got)
}
}
func TestPaddedJSONIsRejected(t *testing.T) {
f := shipped(t)
in := `{"format":"{x}","x":["a","b"]}`
_, err := f.NewTemplate(" " + in + " ")
if err == nil || !strings.Contains(err.Error(), "write "+in) {
t.Fatalf("NewTemplate(padded JSON) = %v, want an error naming the unpadded spelling", err)
}
}
func TestNewTemplateReusable(t *testing.T) {
f := shipped(t)
reusable, err := f.NewTemplate(`{digits(2)}`)
if err != nil {
t.Fatalf("NewTemplate: %v", err)
}
seen := map[string]bool{}
for i := 0; i < 50; i++ {
seen[reusable.Fake()] = true
}
if len(seen) < 2 {
t.Fatalf("Template.Fake() repeated %v, want varied draws from one compile", seen)
}
}
func TestFakeTemplateRepeatBound(t *testing.T) {
f := shipped(t)
_, err := f.FakeTemplate(`{"format":"{x}","repeat":200,"x":{"format":"{y}","repeat":200,"y":{"format":"z","repeat":200}}}`)
if err == nil || !strings.Contains(err.Error(), "maximum") {
t.Errorf("nested repeat over the cap = %v, want it rejected naming the maximum", err)
}
}
+12
View File
@@ -275,6 +275,18 @@ func TestNewErrors(t *testing.T) {
map[string]string{"a|b": `"1"`}, map[string]string{"a|b": `"1"`},
`category "a|b" contains "|"`, `category "a|b" contains "|"`,
}, },
"field name with a bracket": {
map[string]string{"a": `{"format":"{x}","x":"1","b[c":"2"}`},
`field "b[c" contains "["`,
},
"category name with a bracket": {
map[string]string{"[abc]": `"1"`},
`category "[abc]" contains "["`,
},
"field name with a quote": {
map[string]string{"a": `{"format":"{x}","x":"1","b\"c":"2"}`},
`field "b\"c" contains "\""`,
},
// An empty name is not a path segment, so List never offered it — while a // An empty name is not a path segment, so List never offered it — while a
// bare {}, a trailing dot in Fake("a.") and a {/a.} reference all reached // bare {}, a trailing dot in Fake("a.") and a {/a.} reference all reached
// it. The engine accepted spellings it would never advertise. // it. The engine accepted spellings it would never advertise.
+37 -11
View File
@@ -57,6 +57,15 @@ type template struct {
func (*template) isNode() {} func (*template) isNode() {}
// field is the node a path segment names; a binding is a render edge, not a field.
func (t *template) field(seg string) (node, bool) {
if isRef(seg) {
return nil, false
}
n, ok := t.fields[seg]
return n, ok
}
// compile converts parsed JSON into a node tree, validating structure up front. // compile converts parsed JSON into a node tree, validating structure up front.
// Only a choice's items carry a weight, so one here would be inert whatever its type. // Only a choice's items carry a weight, so one here would be inert whatever its type.
func compile(v any) (node, error) { func compile(v any) (node, error) {
@@ -78,10 +87,24 @@ func compileItem(v any) (node, error) {
case map[string]any: case map[string]any:
return compileTemplate(v) return compileTemplate(v)
default: default:
return nil, fmt.Errorf("unsupported node type %T", v) return nil, fmt.Errorf("a template value must be a string, a list or an object, not %s", jsonKind(v))
} }
} }
// jsonKind names a JSON value a template cannot hold, in the data format's own
// terms rather than the decoding library's.
func jsonKind(v any) string {
switch v.(type) {
case float64:
return "a number"
case bool:
return "a boolean"
case nil:
return "null"
}
return fmt.Sprintf("%T", v)
}
func compileString(s string) (node, error) { func compileString(s string) (node, error) {
if err := checkTokens(s, nil); err != nil { if err := checkTokens(s, nil); err != nil {
return nil, err return nil, err
@@ -229,6 +252,9 @@ func readOptions(m map[string]any) (templateOptions, error) {
if repeat == 1 { if repeat == 1 {
return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1") return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1")
} }
if o.separator == "" {
return o, fmt.Errorf("separator \"\" is the default, so it has no effect; drop it")
}
} }
_, o.weighted = m["weight"] _, o.weighted = m["weight"]
return o, nil return o, nil
@@ -276,8 +302,8 @@ func repeatOf(m map[string]any) (int, error) {
if r == 1 { if r == 1 {
return 0, fmt.Errorf("repeat 1 is the default, so it has no effect; drop it") return 0, fmt.Errorf("repeat 1 is the default, so it has no effect; drop it")
} }
if r > maxLen { // cap so a fat-fingered repeat can't build a multi-GB string if r > MaxRepeat { // caps the renders one repeat asks for; repeatCheck bounds what nested ones multiply to
return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, maxLen) return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, MaxRepeat)
} }
return int(r), nil return int(r), nil
} }
@@ -311,23 +337,23 @@ func weightOf(raw any) (float64, error) {
// reservedInName is what a category, folder or field name may not contain: a dot // reservedInName is what a category, folder or field name may not contain: a dot
// separates the segments of a path, '|' the arms of a token, '(' opens a function // separates the segments of a path, '|' the arms of a token, '(' opens a function
// call, braces delimit the token and '/' starts a reference. A name carrying one is // call, braces delimit the token, '/' starts a reference, and brackets and a quote
// reachable by no format, so it is rejected where it is authored rather than at // open a JSON value. A name carrying one is rejected where it is authored rather
// the token that cannot reach it. // than where it would be unreachable.
const reservedInName = ".|({}/" const reservedInName = ".|({}/[]\""
// reservedList spells reservedInName for an error message, so the two cannot drift. // reservedList spells reservedInName for an error message, so the two cannot drift.
var reservedList = strings.Join(strings.Split(reservedInName, ""), " ") var reservedList = strings.Join(strings.Split(reservedInName, ""), " ")
// checkName rejects a name the dot path and {token} grammars cannot spell. Both a // checkName rejects a name the dot path, {token} and JSON grammars cannot spell.
// category or folder and a field go through it, so there is one answer to what a // Both a category or folder and a field go through it, so there is one answer to
// name may contain. // what a name may contain.
func checkName(name string) error { func checkName(name string) error {
if name == "" { if name == "" {
return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name) return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name)
} }
if i := strings.IndexAny(name, reservedInName); i >= 0 { if i := strings.IndexAny(name, reservedInName); i >= 0 {
return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path and {token} grammars reserve", return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path, {token} and JSON grammars reserve",
name, name[i:i+1], reservedList) name, name[i:i+1], reservedList)
} }
return nil return nil
+11
View File
@@ -35,6 +35,7 @@ func TestInertObjectIsRejected(t *testing.T) {
`[{"format":"a","weight":1},"b"]`: "weight 1", `[{"format":"a","weight":1},"b"]`: "weight 1",
`{"format":"{x}","x":"v","repeat":1}`: "repeat 1", `{"format":"{x}","x":"v","repeat":1}`: "repeat 1",
`{"format":"{x}","x":"v","separator":","}`: "separator", `{"format":"{x}","x":"v","separator":","}`: "separator",
`{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default",
} { } {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want) t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want)
@@ -46,3 +47,13 @@ func TestInertObjectIsRejected(t *testing.T) {
} }
} }
} }
func TestInlineFolderSigilsAreRejected(t *testing.T) {
f := shipped(t)
for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} {
_, err := f.NewTemplate(input)
if err == nil || !strings.Contains(err.Error(), "write {/sv_SE.person.last}") {
t.Errorf("NewTemplate(%q) = %v, want an error naming the root spelling", input, err)
}
}
}
+1 -1
View File
@@ -40,7 +40,7 @@ func walkPath(n node, tail []string, w pathWalk) error {
return err return err
} }
} }
child, ok := n.fields[tail[0]] child, ok := n.field(tail[0])
if !ok { if !ok {
return fmt.Errorf("no field %q", tail[0]) return fmt.Errorf("no field %q", tail[0])
} }
+14
View File
@@ -136,3 +136,17 @@ func TestMissingFieldNamesItself(t *testing.T) {
t.Errorf("Fake(person.typo) = %v, want it to name the missing field", err) t.Errorf("Fake(person.typo) = %v, want it to name the missing field", err)
} }
} }
func TestBindingKeyIsNotAPathSegment(t *testing.T) {
dir := writeData(t, map[string]string{
"color": `["red","blue"]`,
"name": `{"format":"{/color} {w}","w":"x"}`,
})
f := newGenerator(t, dir, WithSeed(1))
if slices.Contains(f.List(), "name./color") {
t.Error("List() advertises a binding key")
}
if _, err := f.Fake("name./color"); err == nil || !strings.Contains(err.Error(), `no field "/color"`) {
t.Errorf("Fake(name./color) = %v, want a no-field error", err)
}
}
+81
View File
@@ -0,0 +1,81 @@
package fejkdata
import (
"fmt"
"strings"
"testing"
"testing/fstest"
)
// One Fake call descends depth levels: each level's "a" is the next template down.
func nestedJSON(depth int) string {
s := `"leaf"`
for i := 0; i < depth; i++ {
s = fmt.Sprintf(`{"format":"{a}","a":%s}`, s)
}
return s
}
// One Fake call expands n sibling tokens.
func wideTokenJSON(n int) string {
var toks, fields strings.Builder
for i := 0; i < n; i++ {
fmt.Fprintf(&toks, "{f%d}", i)
if i > 0 {
fields.WriteByte(',')
}
fmt.Fprintf(&fields, `"f%d":"x"`, i)
}
return fmt.Sprintf(`{"format":"%s",%s}`, toks.String(), fields.String())
}
func TestNoRenderAllocRegression(t *testing.T) {
shapes := []struct {
name string
json string
base float64
}{
{"nested depth 25", nestedJSON(25), 27},
{"nested depth 100", nestedJSON(100), 102},
{"wide 500 tokens", wideTokenJSON(500), 9},
}
for _, s := range shapes {
f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"x.json": {Data: []byte(s.json)}}))
if err != nil {
t.Fatalf("New(%s): %v", s.name, err)
}
allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") })
if allocs > s.base*1.10 {
t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); bump the baseline only as a deliberate change", s.name, allocs, s.base*1.10, s.base)
}
}
}
// A record's fences read the compiled tree, so they belong to New, not to a draw.
func TestNoRecordAllocRegression(t *testing.T) {
for _, s := range []struct{ name, json string }{
{"record 3 columns", `{"format":"","a":"x","b":"y","c":"z"}`},
{"record 50 columns", wideTokenJSON(50)},
} {
f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{"x.json": {Data: []byte(s.json)}}))
if err != nil {
t.Fatalf("New(%s): %v", s.name, err)
}
if _, err := f.Record("x"); err != nil {
t.Fatalf("Record(%s): %v", s.name, err) // else the gate would measure the error path
}
const base = 4.0
if allocs := testing.AllocsPerRun(10000, func() { f.Record("x") }); allocs > base*1.10 {
t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a record fence running per draw is the usual cause", s.name, allocs, base*1.10, base)
}
}
}
func BenchmarkNestedDepth25(b *testing.B) { benchPath(b, tmpData(b, "deep", nestedJSON(25)), "deep") }
func BenchmarkNestedDepth100(b *testing.B) { benchPath(b, tmpData(b, "deep", nestedJSON(100)), "deep") }
func BenchmarkWideTokens100(b *testing.B) {
benchPath(b, tmpData(b, "wide", wideTokenJSON(100)), "wide")
}
func BenchmarkWideTokens500(b *testing.B) {
benchPath(b, tmpData(b, "wide", wideTokenJSON(500)), "wide")
}
+28
View File
@@ -43,6 +43,34 @@ func TestReadmeExamplesLoadAndRender(t *testing.T) {
} }
} }
func TestReadmeRecordExample(t *testing.T) {
src := readme(t)
section := src[strings.Index(src, "### Records"):]
section = section[:strings.Index(section, "## Library")]
block := jsonBlock.FindStringSubmatch(section)
if block == nil {
t.Fatal("README lost the Records example")
}
f, err := New(WithDataPath(writeData(t, map[string]string{"users": block[1]})), WithSeed(1))
if err != nil {
t.Fatal(err)
}
r, err := f.Record("users")
if err != nil {
t.Fatal(err)
}
got := r.Columns()
if len(got) != 2 || got[0].Name != "first" || got[0].Value != "Bo" || got[1].Name != "last" || got[1].Value != "Lovelace" {
t.Fatalf("record columns = %v, want first=Bo, last=Lovelace with seed 1", got)
}
if r.CSVHeader() != "first,last" || r.CSVLine() != "Bo,Lovelace" {
t.Fatalf("csv = %q, %q, want first,last / Bo,Lovelace", r.CSVHeader(), r.CSVLine())
}
if r.SQLInsert("users") != `INSERT INTO "users" ("first", "last") VALUES ('Bo', 'Lovelace');` {
t.Fatalf("sql = %q, want the seeded INSERT", r.SQLInsert("users"))
}
}
func TestReadmeSQLExampleOutput(t *testing.T) { func TestReadmeSQLExampleOutput(t *testing.T) {
src := readme(t) src := readme(t)
src = src[strings.Index(src, "### Your own data"):] src = src[strings.Index(src, "### Your own data"):]
+285
View File
@@ -0,0 +1,285 @@
package fejkdata
import (
"encoding/csv"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
)
// Column is one rendered column of a record.
type Column struct {
Name string
Value string
}
// Record is one record rendered from a template: every direct field is a column,
// listed in name order. Each column is its own expansion, so a sibling field is
// local to it, while a reference that reads a path is drawn once for the whole
// record.
type Record struct {
columns []Column
}
// Columns returns the record's columns in name order.
func (r *Record) Columns() []Column {
return append([]Column(nil), r.columns...)
}
// JSON renders the record as one JSON object, every column a string.
func (r *Record) JSON() string {
m := make(map[string]string, len(r.columns))
for _, c := range r.columns {
m[c.Name] = c.Value
}
b, _ := json.Marshal(m)
return string(b)
}
// CSVHeader renders the column names as one CSV header line.
func (r *Record) CSVHeader() string {
return csvLine(r.names())
}
// CSVLine renders the column values as one CSV row.
func (r *Record) CSVLine() string {
return csvLine(r.values())
}
func (r *Record) names() []string {
out := make([]string, len(r.columns))
for i, c := range r.columns {
out[i] = c.Name
}
return out
}
func (r *Record) values() []string {
out := make([]string, len(r.columns))
for i, c := range r.columns {
out[i] = c.Value
}
return out
}
func csvLine(cols []string) string {
var b strings.Builder
w := csv.NewWriter(&b)
_ = w.Write(cols)
w.Flush()
line := strings.TrimSuffix(b.String(), "\n")
if line == "" {
return `""` // a blank line is a row every CSV reader drops
}
return line
}
// SQLInsert renders the record as one INSERT statement into table: identifiers in
// ANSI double quotes, every value a single-quoted string literal.
func (r *Record) SQLInsert(table string) string {
cols := make([]string, len(r.columns))
vals := make([]string, len(r.columns))
for i, c := range r.columns {
cols[i] = quoteIdent(c.Name)
vals[i] = "'" + strings.ReplaceAll(c.Value, "'", "''") + "'"
}
return fmt.Sprintf("INSERT INTO %s (%s) VALUES (%s);", quoteIdent(table), strings.Join(cols, ", "), strings.Join(vals, ", "))
}
func quoteIdent(s string) string {
return `"` + strings.ReplaceAll(s, `"`, `""`) + `"`
}
// Record renders a path as one record: the template it names, with each direct
// field drawn as a column. Only a category-level template is a record — a path
// that descends into a field, or that names a folder or a choice, is an error.
func (f *Generator) Record(path string) (*Record, error) {
f.mu.Lock()
defer f.mu.Unlock()
_, n, tail, err := resolveCategory(f.categories, strings.Split(path, "."))
if err != nil {
return nil, fmt.Errorf("fejkdata: %s: %w", path, err)
}
if len(tail) > 0 {
return nil, fmt.Errorf("fejkdata: %s descends into %q, a field; only a category-level template is a record", path, tail[0])
}
shape := f.recordShapeOf(n)
if shape.err != nil {
return nil, fmt.Errorf("fejkdata: %s %w", path, shape.err)
}
return renderRecord(f.rand, shape.t, shape.columns), nil
}
// recordShape is what recordOf settled about a node: the template to project, its
// columns, or why it is not a record.
type recordShape struct {
t *template
columns []string
err error
}
// recordShapeOf fences a node once and remembers the answer. Callers hold the
// generator's lock.
func (f *Generator) recordShapeOf(n node) recordShape {
if shape, done := f.records[n]; done {
return shape
}
t, columns, err := recordOf(n)
shape := recordShape{t: t, columns: columns, err: err}
if f.records == nil {
f.records = map[node]recordShape{}
}
f.records[n] = shape
return shape
}
// RecordTemplate is an inline record compiled, referenced and validated once,
// ready to render many times with [RecordTemplate.Fake].
type RecordTemplate struct {
g *Generator
t *template
columns []string
}
// Fake renders the record with one draw.
func (t *RecordTemplate) Fake() *Record {
t.g.mu.Lock()
defer t.g.mu.Unlock()
return renderRecord(t.g.rand, t.t, t.columns)
}
// NewRecordTemplate compiles an inline record — a JSON object with a format and
// fields — and binds its references against the loaded tree.
func (f *Generator) NewRecordTemplate(input string) (*RecordTemplate, error) {
t, err := f.NewTemplate(input)
if err != nil {
return nil, err
}
tm, columns, err := recordOf(t.n)
if err != nil {
return nil, fmt.Errorf("fejkdata: an inline record %w", err)
}
return &RecordTemplate{g: f, t: tm, columns: columns}, nil
}
// FakeRecord compiles and renders an inline record in one call.
func (f *Generator) FakeRecord(input string) (*Record, error) {
t, err := f.NewRecordTemplate(input)
if err != nil {
return nil, err
}
return t.Fake(), nil
}
// recordOf is the fence both record entry points pass. The columns come back with
// the template, fixed for every draw the caller goes on to make.
func recordOf(n node) (*template, []string, error) {
t, ok := n.(*template)
if !ok {
return nil, nil, errors.New("names a choice, not a template; a record is a template whose fields are its columns")
}
if t.repeat != 1 {
return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat)
}
columns := recordColumns(t)
if len(columns) == 0 {
return nil, nil, errors.New("has no fields, so no columns")
}
if err := checkColumnRefs(t, columns); err != nil {
return nil, nil, err
}
return t, columns, nil
}
// checkColumnRefs rejects the reference reads a record's shared draw cannot answer
// for: one column rendering a level another reads a path into, and a column
// reading the record back through its own path.
func checkColumnRefs(t *template, columns []string) error {
reads, err := columnRefs(t, columns)
if err != nil {
return err
}
sort.Slice(reads, func(i, j int) bool {
if reads[i].a.path != reads[j].a.path {
return reads[i].a.path < reads[j].a.path
}
return reads[i].column < reads[j].column
})
for i, level := range reads {
for _, into := range reads[i+1:] {
if strings.HasPrefix(into.a.path, level.a.path+".") {
return fmt.Errorf("column %q renders {%s}, a level column %q reads a path into with {%s}; name the fields you want instead",
level.column, level.a.name, into.column, into.a.name)
}
}
}
return nil
}
// columnRef is one held reference read, and the column whose render reaches it.
type columnRef struct {
column string
a arm
}
// columnRefs lists every reference that reads a path, anywhere a column renders,
// following the same edges expand does. A reference landing back on the record
// itself is reported rather than collected, whether it reads a path or the record
// whole: either way the column describes a draw other than its neighbours'.
func columnRefs(t *template, columns []string) ([]columnRef, error) {
var out []columnRef
var err error
for _, name := range columns {
seen := map[node]bool{}
var walk func(n node)
walk = func(n node) {
if n == nil || seen[n] || err != nil {
return
}
seen[n] = true
if tm, ok := n.(*template); ok {
for _, ref := range refTokens(tm.format) {
a := splitArm(ref, tm.refs)
if tm.fields[a.key] == node(t) {
err = fmt.Errorf("column %q reads {%s}, which points back at this record; a column cannot read another column — move the shared value into its own category and reference that", name, a.name)
return
}
if len(a.tail) > 0 {
out = append(out, columnRef{name, a})
}
}
}
for _, e := range renderEdges(n) {
walk(e.to)
}
}
walk(t.fields[name])
}
return out, err
}
// renderRecord draws each column once, in the name order recordOf fixed, over one
// reference scope shared across them.
func renderRecord(s *session, t *template, columns []string) *Record {
scope := &draws{variant: map[string]node{}, value: map[string]string{}}
r := &Record{columns: make([]Column, len(columns))}
for i, name := range columns {
r.columns[i] = Column{Name: name, Value: render(s, t.fields[name], scope)}
}
return r
}
// recordColumns is the sorted non-reference field names — the columns a record
// projects. A {/path} binding is carried in fields under its root path, so only a
// name that is not a reference is a column.
func recordColumns(t *template) []string {
var names []string
for _, name := range sortedNames(t.fields) {
if !isRef(name) {
names = append(names, name)
}
}
return names
}
+408
View File
@@ -0,0 +1,408 @@
package fejkdata
import (
"encoding/csv"
"encoding/json"
"strings"
"testing"
)
func recordCat(t *testing.T) *Generator {
t.Helper()
dir := writeData(t, map[string]string{
"users": `{
"format": "{first} {last}",
"first": ["Ada", "Bo"],
"last": ["Lovelace", "Ek"]
}`,
})
return newGenerator(t, dir, WithSeed(1))
}
func TestRecordProjectsFieldsAsColumns(t *testing.T) {
f := recordCat(t)
r, err := f.Record("users")
if err != nil {
t.Fatal(err)
}
got := r.Columns()
if len(got) != 2 || got[0].Name != "first" || got[1].Name != "last" {
t.Fatalf("Record.Columns() = %v, want columns first, last in name order", got)
}
if (got[0].Value != "Ada" && got[0].Value != "Bo") || (got[1].Value != "Lovelace" && got[1].Value != "Ek") {
t.Fatalf("columns = %v, want the field values", got)
}
}
func TestRecordIsDeterministic(t *testing.T) {
a, b := recordCat(t), recordCat(t)
for i := 0; i < 20; i++ {
x, _ := a.Record("users")
y, _ := b.Record("users")
if x.JSON() != y.JSON() {
t.Fatalf("same seed diverged: %s != %s", x.JSON(), y.JSON())
}
}
}
func TestRecordSkipsReferenceBindings(t *testing.T) {
dir := writeData(t, map[string]string{
"name": `{"format": "{first} {last}", "first": "Ada", "last": "Lovelace"}`,
"user": `{"format": "they are {/name}", "id": "1"}`,
})
f := newGenerator(t, dir, WithSeed(1))
r, err := f.Record("user")
if err != nil {
t.Fatal(err)
}
got := r.Columns()
if len(got) != 1 || got[0].Name != "id" {
t.Fatalf("Record.Columns() = %v, want only the id column (a {/path} is not a column)", got)
}
}
func TestRecordErrors(t *testing.T) {
dir := writeData(t, map[string]string{
"bare": `"hello"`,
"pick": `["a", "b"]`,
"group/cat": `"x"`,
})
f := newGenerator(t, dir, WithSeed(1))
for _, c := range []struct {
path string
want string
}{
{"bare", "has no fields, so no columns"},
{"pick", "names a choice"},
{"group", "names a folder"},
{"nope", "no entry"},
} {
if _, err := f.Record(c.path); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("Record(%q) = %v, want an error containing %q", c.path, err, c.want)
}
}
}
func TestRecordJSON(t *testing.T) {
f := recordCat(t)
r, err := f.Record("users")
if err != nil {
t.Fatal(err)
}
var m map[string]string
if err := json.Unmarshal([]byte(r.JSON()), &m); err != nil {
t.Fatalf("Record.JSON() is not valid JSON: %v\n%s", err, r.JSON())
}
if len(m) != 2 || (m["first"] != "Ada" && m["first"] != "Bo") {
t.Fatalf("Record.JSON() = %s, want two addressable columns", r.JSON())
}
for _, f := range r.Columns() {
if m[f.Name] != f.Value {
t.Fatalf("JSON column %q = %q, want %q", f.Name, m[f.Name], f.Value)
}
}
}
func TestRecordCSV(t *testing.T) {
dir := writeData(t, map[string]string{
"note": `{"format": "{word}, {word}", "word": ["a", "b,c", "d\"e", "f\n"]}`,
})
f := newGenerator(t, dir, WithSeed(1))
seen := map[string]bool{}
for i := 0; i < 200 && len(seen) < 4; i++ {
r, err := f.Record("note")
if err != nil {
t.Fatal(err)
}
if got := r.CSVHeader(); got != "word" {
t.Fatalf("CSVHeader() = %q, want word", got)
}
rec, err := csv.NewReader(strings.NewReader(r.CSVLine() + "\n")).Read()
if err != nil {
t.Fatalf("CSVLine() is not valid CSV: %v\n%q", err, r.CSVLine())
}
want := r.Columns()[0].Value
if len(rec) != 1 || rec[0] != want {
t.Fatalf("CSVLine() = %v, want the field value %q round-tripped", rec, want)
}
seen[want] = true
}
if len(seen) != 4 {
t.Fatalf("round-tripped %d of the 4 values; the comma, quote and newline shapes must each survive", len(seen))
}
}
func TestRecordCSVEmptyValueStaysARow(t *testing.T) {
dir := writeData(t, map[string]string{"blank": `{"format": "", "note": ""}`})
f := newGenerator(t, dir, WithSeed(1))
r, err := f.Record("blank")
if err != nil {
t.Fatal(err)
}
rows, err := csv.NewReader(strings.NewReader(r.CSVHeader() + "\n" + r.CSVLine() + "\n")).ReadAll()
if err != nil {
t.Fatalf("csv: %v", err)
}
if len(rows) != 2 || len(rows[1]) != 1 || rows[1][0] != "" {
t.Fatalf("one empty column parsed to %v, want a header and one row of one empty field", rows)
}
}
func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) {
cat := `{"format":"","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}`
for _, c := range []struct{ name, row string }{
{"sibling columns", `{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`},
{"through a nested template", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b} {x}","x":"1"}}`},
{"through a column repeat", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`},
{"through a choice variant", `{"format":"","whole":"{/cat.a}","inner":[{"format":"{/cat.a.b} {x}","x":"1"},{"format":"{/cat.a.b}! {x}","x":"2"}]}`},
{"as a builtin operand", `{"format":"","whole":"{uppercase(/cat.a)}","inner":"{/cat.a.b}"}`},
} {
f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "row": c.row}), WithSeed(1))
_, err := f.Record("row")
if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: Record = %v, want the overlap rejected the way one format is", c.name, err)
continue
}
if !strings.Contains(err.Error(), `"whole"`) || !strings.Contains(err.Error(), `"inner"`) {
t.Errorf("%s: error %q names neither column; it must name both", c.name, err)
}
if _, err := f.Fake("row"); err != nil {
t.Errorf("%s: Fake(row) = %v, want the string view untouched", c.name, err)
}
}
}
func TestInlineRecordRejectsOverlappingColumns(t *testing.T) {
dir := writeData(t, map[string]string{
"cat": `{"format":"","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}`,
})
f := newGenerator(t, dir, WithSeed(1))
_, err := f.FakeRecord(`{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`)
if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Fatalf("inline record over an overlapping pair = %v, want the inline entry point to refuse it too", err)
}
}
func TestRecordRejectsAColumnReadingItsOwnRecord(t *testing.T) {
for _, c := range []struct{ name, column string }{
{"a path into itself", `"full":"{/person.first} {/person.last}"`},
{"the record read whole", `"whole":"{/person}"`},
{"the record as an operand", `"up":"{uppercase(/person)}"`},
} {
person := `{"format":"{first} {last}","first":["Ada","Bo"],"last":["Lovelace","Ek"],` + c.column + `}`
f := newGenerator(t, writeData(t, map[string]string{"person": person}), WithSeed(1))
if _, err := f.Record("person"); err == nil || !strings.Contains(err.Error(), "points back at this record") {
t.Errorf("%s: Record = %v, want it refused; the column would contradict the columns beside it", c.name, err)
}
}
}
func TestRecordBareReferenceStaysIndependentAsAnOperand(t *testing.T) {
dir := writeData(t, map[string]string{
"cur": `[{"format":"{code}","code":"aud"},{"format":"{code}","code":"eur"}]`,
"row": `{"format":"","up":"{uppercase(/cur)}","low":"{lowercase(/cur)}"}`,
})
f := newGenerator(t, dir, WithSeed(1))
sawMismatch := false
for i := 0; i < 200 && !sawMismatch; i++ {
r, err := f.Record("row")
if err != nil {
t.Fatal(err)
}
m := map[string]string{}
for _, c := range r.Columns() {
m[c.Name] = c.Value
}
sawMismatch = !strings.EqualFold(m["up"], m["low"])
}
if !sawMismatch {
t.Error("two bare-reference operand columns never disagreed; a bare reference draws on its own, as the plain spelling does")
}
for i := 0; i < 50; i++ {
v, err := f.FakeTemplate("{/cur}|{uppercase(/cur)}")
if err != nil {
t.Fatal(err)
}
if parts := strings.Split(v, "|"); !strings.EqualFold(parts[0], parts[1]) {
t.Fatalf("one format rendered %q; within an expansion a bare reference is still one draw", v)
}
}
}
func TestRecordSQLInsert(t *testing.T) {
dir := writeData(t, map[string]string{
"person": `{"format": "{last}", "last": "O'Brien"}`,
})
f := newGenerator(t, dir, WithSeed(1))
r, err := f.Record("person")
if err != nil {
t.Fatal(err)
}
got := r.SQLInsert("people")
if got != `INSERT INTO "people" ("last") VALUES ('O''Brien');` {
t.Fatalf("SQLInsert() = %q, want quoted identifiers and the single quote doubled", got)
}
}
func TestRecordRejectsFieldDescent(t *testing.T) {
dir := writeData(t, map[string]string{
"cat": `{"format":"{sub}","sub":{"format":"{x}","x":"1"}}`,
"row": `[{"format":"{x}","x":"1"},{"format":"{x}","x":"2"}]`,
})
f := newGenerator(t, dir, WithSeed(1))
for _, path := range []string{"cat.sub", "row.x"} {
if _, err := f.Record(path); err == nil || !strings.Contains(err.Error(), "field") {
t.Errorf("Record(%q) = %v, want a 'descends into a field' error", path, err)
}
}
}
func TestRecordSharesAReferenceAcrossColumns(t *testing.T) {
dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD","symbol":"$"},{"format":"{code}","code":"EUR","symbol":"€"}]`,
"price": `{"format":"","code":"{/currency.code}","symbol":"{/currency.symbol}"}`,
})
f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 100; i++ {
r, err := f.Record("price")
if err != nil {
t.Fatal(err)
}
m := map[string]string{}
for _, c := range r.Columns() {
m[c.Name] = c.Value
}
switch m["code"] {
case "AUD":
if m["symbol"] != "$" {
t.Fatalf("record %q: code AUD but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"])
}
case "EUR":
if m["symbol"] != "€" {
t.Fatalf("record %q: code EUR but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"])
}
default:
t.Fatalf("record %q has unexpected code %q", r.JSON(), m["code"])
}
}
}
func TestRecordSharesAReferenceIntoAColumnRepeat(t *testing.T) {
dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`,
"order": `{"format":"","codes":{"format":"{/currency.code}","repeat":3,"separator":"-"}}`,
})
f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 50; i++ {
r, err := f.Record("order")
if err != nil {
t.Fatal(err)
}
parts := strings.Split(r.Columns()[0].Value, "-")
if len(parts) != 3 || parts[0] != parts[1] || parts[1] != parts[2] {
t.Fatalf("codes column = %q, want one shared draw across its repeat", r.Columns()[0].Value)
}
}
}
func TestRecordBareReferenceStaysIndependent(t *testing.T) {
dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`,
"order": `{"format":"","whole":"{/currency}","code":"{/currency.code}"}`,
})
f := newGenerator(t, dir, WithSeed(1))
sawMismatch := false
for i := 0; i < 100; i++ {
r, err := f.Record("order")
if err != nil {
t.Fatal(err)
}
m := map[string]string{}
for _, c := range r.Columns() {
m[c.Name] = c.Value
}
if m["whole"] != m["code"] {
sawMismatch = true
break
}
}
if !sawMismatch {
t.Fatal("a bare {/currency} column never disagreed with a tailed {/currency.code} column; a bare reference should draw independently")
}
}
func TestRecordSQLQuotesIdentifiers(t *testing.T) {
dir := writeData(t, map[string]string{
"row": `{"format": "", "postal-code": "1", "street-number": "2"}`,
})
f := newGenerator(t, dir, WithSeed(1))
r, err := f.Record("row")
if err != nil {
t.Fatal(err)
}
got := r.SQLInsert("my-table")
if got != `INSERT INTO "my-table" ("postal-code", "street-number") VALUES ('1', '2');` {
t.Fatalf("SQLInsert() = %q, want hyphenated identifiers and table quoted", got)
}
}
func TestFakeRecordAndTemplate(t *testing.T) {
f := recordCat(t)
in := `{"format":"{x} {y}","x":["1","2"],"y":["3","4"]}`
want, err := f.FakeRecord(in)
if err != nil {
t.Fatalf("FakeRecord: %v", err)
}
if len(want.Columns()) != 2 {
t.Fatalf("FakeRecord columns = %v, want two columns", want.Columns())
}
reusable, err := f.NewRecordTemplate(in)
if err != nil {
t.Fatalf("NewRecordTemplate: %v", err)
}
for i := 0; i < 20; i++ {
if got := reusable.Fake(); len(got.Columns()) != 2 {
t.Fatalf("RecordTemplate.Fake() = %v, want two columns", got.Columns())
}
}
}
func TestInlineRecordErrors(t *testing.T) {
f := recordCat(t)
for _, c := range []struct {
input string
want string
}{
{`"hello"`, "has no fields, so no columns"},
{`["a","b"]`, "a record is a template whose fields are its columns"},
{`{"format":"{a}-","repeat":3,"separator":"|","a":["x","y"]}`, "carries repeat 3"},
} {
if _, err := f.FakeRecord(c.input); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("FakeRecord(%q) = %v, want an error naming %q", c.input, err, c.want)
}
}
}
func TestRecordRejectsATopLevelRepeat(t *testing.T) {
dir := writeData(t, map[string]string{
"rep": `{"format":"{a}-","repeat":3,"separator":"|","a":["x","y"]}`,
})
f := newGenerator(t, dir, WithSeed(1))
if _, err := f.Record("rep"); err == nil || !strings.Contains(err.Error(), "carries repeat 3") {
t.Errorf("Record on a repeating template = %v, want an error naming the repeat", err)
}
if v, err := f.Fake("rep"); err != nil || v != "x-|x-|x-" {
t.Errorf("Fake(rep) = %q, %v, want the repeat still composed for the string view", v, err)
}
}
func TestRecordTemplateRejectsATopLevelRepeat(t *testing.T) {
f := recordCat(t)
in := `{"format":"{a}-","repeat":3,"separator":"|","a":["x","y"]}`
if _, err := f.NewRecordTemplate(in); err == nil || !strings.Contains(err.Error(), "carries repeat 3") {
t.Errorf("NewRecordTemplate on a repeating template = %v, want an error naming the repeat", err)
}
if _, err := f.NewTemplate(in); err != nil {
t.Errorf("NewTemplate on the same input = %v, want the string view to still compile", err)
}
}
+14 -7
View File
@@ -72,6 +72,14 @@ func refSegments(name string, folder []string) ([]string, error) {
// error, never a random render-time one. // error, never a random render-time one.
func linkRefs(root map[string]node) error { func linkRefs(root map[string]node) error {
return eachTemplate(root, func(folder []string, path string, t *template) error { return eachTemplate(root, func(folder []string, path string, t *template) error {
return linkTemplateRefs(folder, path, t, root)
})
}
// linkTemplateRefs binds one template's references against root. A template with
// none is left untouched, so an inline format that references nothing costs only
// the refTokens scan.
func linkTemplateRefs(folder []string, path string, t *template, root map[string]node) error {
names := refTokens(t.format) names := refTokens(t.format)
if len(names) == 0 { if len(names) == 0 {
return nil return nil
@@ -85,7 +93,7 @@ func linkRefs(root map[string]node) error {
if err != nil { if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err) return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
} }
head, target, tail, err := resolveRef(root, segments) head, target, tail, err := resolveCategory(root, segments)
if err != nil { if err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err) return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
} }
@@ -100,7 +108,6 @@ func linkRefs(root map[string]node) error {
return fmt.Errorf("%s: %w", path, err) return fmt.Errorf("%s: %w", path, err)
} }
return nil return nil
})
} }
// eachTemplate calls fn once per template, with the folder its category sits in // eachTemplate calls fn once per template, with the folder its category sits in
@@ -139,11 +146,11 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t
return inFolder(nil, root) return inFolder(nil, root)
} }
// resolveRef walks a reference path through the folders to the category it names, // resolveCategory walks a dotted path through the folders to the category it
// returning that head, the node, and the tail left to read into it. A descent of // names, returning that head, the node, and the tail left to read into it. A
// its own rather than a walkPath: it walks groups only and returns where they end, // descent of its own rather than a walkPath: it walks groups only and returns
// not a leaf. // where they end, not a leaf.
func resolveRef(root map[string]node, segments []string) (head []string, target node, tail []string, err error) { func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) {
var n node = &group{children: root} var n node = &group{children: root}
i := 0 i := 0
for ; i < len(segments); i++ { for ; i < len(segments); i++ {
+12 -10
View File
@@ -27,7 +27,7 @@ func (f *Generator) Fake(path string) (string, error) {
if _, ok := n.(*group); ok { if _, ok := n.(*group); ok {
return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path)
} }
return render(f.rand, n), nil return render(f.rand, n, nil), nil
} }
// descend walks named fields to the node a path names. It is the one render-side // descend walks named fields to the node a path names. It is the one render-side
@@ -50,17 +50,18 @@ func descend(s *session, root node, segments []string) (node, error) {
} }
// render evaluates a compiled node to a string. compile validates every node up // render evaluates a compiled node to a string. compile validates every node up
// front, so rendering a compiled tree cannot fail. // front, so rendering a compiled tree cannot fail. refScope carries the draws a
func render(s *session, n node) string { // reference shares beyond its own expansion; nil keeps every reference local.
func render(s *session, n node, refScope *draws) string {
switch n := n.(type) { switch n := n.(type) {
case *choice: case *choice:
return render(s, pick(s, n)) return render(s, pick(s, n), refScope)
case *template: case *template:
if n.repeat == 1 { if n.repeat == 1 {
if n.fixed { if n.fixed {
return n.lit return n.lit
} }
return expand(s, n) return expand(s, n, refScope)
} }
var b strings.Builder var b strings.Builder
b.Grow(n.repeat * (n.grow + len(n.separator))) b.Grow(n.repeat * (n.grow + len(n.separator)))
@@ -68,7 +69,7 @@ func render(s *session, n node) string {
if i > 0 { if i > 0 {
b.WriteString(n.separator) b.WriteString(n.separator)
} }
b.WriteString(expand(s, n)) b.WriteString(expand(s, n, refScope))
} }
return b.String() return b.String()
default: default:
@@ -90,11 +91,12 @@ func pick(r rng, c *choice) node {
// expand renders a template's compiled ops. compile validated every token, so this // expand renders a template's compiled ops. compile validated every token, so this
// cannot fail. // cannot fail.
func expand(s *session, t *template) string { func expand(s *session, t *template, refScope *draws) string {
var b strings.Builder var b strings.Builder
b.Grow(t.grow) b.Grow(t.grow)
// One draw per held name, for this expansion only: a nested template and each // One draw per held name, for this expansion only: a nested template and each
// repeat iteration get their own, since each is its own expansion. // repeat iteration get their own, since each is its own expansion. A reference
// reads the caller's scope instead, whenever one was supplied.
var held *draws var held *draws
if len(t.held) > 0 { if len(t.held) > 0 {
held = &draws{ held = &draws{
@@ -108,7 +110,7 @@ func expand(s *session, t *template) string {
case 'l': case 'l':
b.WriteString(o.lit) b.WriteString(o.lit)
case 'f': case 'f':
b.WriteString(readField(s, t, held, o.arms[s.IntN(len(o.arms))])) b.WriteString(readField(s, t, held, refScope, o.arms[s.IntN(len(o.arms))]))
case 'b': case 'b':
// Read before the call, so the value a calc computes is the value the // Read before the call, so the value a calc computes is the value the
// format showed. calcVars fixed the order op.operands holds. // format showed. calcVars fixed the order op.operands holds.
@@ -116,7 +118,7 @@ func expand(s *session, t *template) string {
if len(o.operands) > 0 { if len(o.operands) > 0 {
operands = make([]string, len(o.operands)) operands = make([]string, len(o.operands))
for j, a := range o.operands { for j, a := range o.operands {
operands[j] = readField(s, t, held, a) operands[j] = readField(s, t, held, refScope, a)
} }
} }
b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far
+10
View File
@@ -108,6 +108,16 @@ func TestFakeIsSafeForConcurrentUse(t *testing.T) {
return return
} }
f.List() f.List()
if _, err := f.Record("sv_SE.person"); err != nil {
t.Error(err)
return
}
tmpl, err := f.NewTemplate("{/sv_SE.person.last}")
if err != nil {
t.Error(err)
return
}
tmpl.Fake()
} }
}() }()
} }
+22 -2
View File
@@ -140,7 +140,7 @@ func checkTokens(format string, fields map[string]node) error {
} }
continue // its target is checked at New (see linkRefs) continue // its target is checked at New (see linkRefs)
} }
if err := checkArm(name, fields); err != nil { if err := checkArm(name, fields, len(names) == 1); err != nil {
return fmt.Errorf("token {%s}: %w", t.body, err) return fmt.Errorf("token {%s}: %w", t.body, err)
} }
} }
@@ -151,7 +151,9 @@ func checkTokens(format string, fields map[string]node) error {
} }
// checkArm validates one sibling name or path against a template's fields. // checkArm validates one sibling name or path against a template's fields.
func checkArm(name string, fields map[string]node) error { // wholeToken says the name is the token's entire body, so {/name} would render
// the same value and can be offered as the reference spelling.
func checkArm(name string, fields map[string]node, wholeToken bool) error {
a := splitArm(name, nil) a := splitArm(name, nil)
if err := checkSegments(a); err != nil { if err := checkSegments(a); err != nil {
return err return err
@@ -164,6 +166,13 @@ func checkArm(name string, fields map[string]node) error {
if isOption(a.key) { if isOption(a.key) {
return fmt.Errorf("%q is an option and can never be a field", a.key) return fmt.Errorf("%q is an option and can never be a field", a.key)
} }
if len(fields) == 0 {
hint := ""
if wholeToken && hintableRef(name) {
hint = fmt.Sprintf(" — write {/%s} to reference the data", name)
}
return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.key, hint)
}
return fmt.Errorf("no field %q", a.key) return fmt.Errorf("no field %q", a.key)
} }
if err := checkPath(head, a.tail, a.key); err != nil { if err := checkPath(head, a.tail, a.key); err != nil {
@@ -172,6 +181,17 @@ func checkArm(name string, fields map[string]node) error {
return nil return nil
} }
// hintableRef reports whether {/name} is a reference the grammar accepts, so the
// hint never names a spelling that fails too.
func hintableRef(name string) bool {
for _, seg := range strings.Split(name, ".") {
if checkName(seg) != nil {
return false
}
}
return true
}
// tokenOperands lists the fields one {token} body reads as operands, empty for a // tokenOperands lists the fields one {token} body reads as operands, empty for a
// field token or a builtin that reads none. // field token or a builtin that reads none.
func tokenOperands(body string) []string { func tokenOperands(body string) []string {
+35 -3
View File
@@ -8,7 +8,13 @@ import (
) )
// engine builds a seeded generator with no loaded categories, for rendering tests. // engine builds a seeded generator with no loaded categories, for rendering tests.
func engine(seed uint64) *Generator { return &Generator{rand: newRand(seed, true)} } func engine(seed uint64) *Generator {
s, err := newRand(seed, true)
if err != nil {
panic(err)
}
return &Generator{rand: s}
}
// parse unmarshals a JSON template fragment into its dynamic form. // parse unmarshals a JSON template fragment into its dynamic form.
func parse(t *testing.T, s string) any { func parse(t *testing.T, s string) any {
@@ -32,7 +38,7 @@ func compiled(t *testing.T, s string) node {
func mustRender(t *testing.T, f *Generator, s string) string { func mustRender(t *testing.T, f *Generator, s string) string {
t.Helper() t.Helper()
return render(f.rand, compiled(t, s)) return render(f.rand, compiled(t, s), nil)
} }
func TestStringIsAFormat(t *testing.T) { func TestStringIsAFormat(t *testing.T) {
@@ -238,6 +244,18 @@ func TestCompileErrors(t *testing.T) {
`"{base64(0)}"`, // count must be positive `"{base64(0)}"`, // count must be positive
`"{float(1,2)}"`, // wrong arity `"{float(1,2)}"`, // wrong arity
`"{float(1,2,-1)}"`, // negative decimals `"{float(1,2,-1)}"`, // negative decimals
`"{float(NaN,NaN,2)}"`, // bounds must be finite
`"{float(Inf,Inf,2)}"`, // same-sign infinities
`"{float(1,NaN,2)}"`, // one NaN bound
`"{float(-Inf,1,2)}"`, // one infinite bound
`"{digits(+5)}"`, // a count is a plain integer
`"{digits(05)}"`, // no leading zero
`"{int(+1,5)}"`, // a bound is a plain integer
`"{int(5,5)}"`, // a constant is written as text
`"{float(1,1,2)}"`, // a constant is written as text
`{"format":"{x}","x":"v","repeat":2,"separator":""}`, // separator "" is the default
`"{calc(1/0)}"`, // a constant zero divisor
`{"format":"{calc(x/y)}","x":"1","y":"0"}`, // a fixed zero divisor
`"{iban(US)}"`, // unsupported country `"{iban(US)}"`, // unsupported country
`"{seq(a,b)}"`, // seq takes at most one name `"{seq(a,b)}"`, // seq takes at most one name
`"{calc()}"`, // calc needs an expression `"{calc()}"`, // calc needs an expression
@@ -297,7 +315,7 @@ func TestGrowIsALowerBound(t *testing.T) {
t.Fatalf("format %q did not compile to a template", format) t.Fatalf("format %q did not compile to a template", format)
} }
for i := 0; i < 50; i++ { for i := 0; i < 50; i++ {
if got := len(expand(f.rand, tmpl)); got < tmpl.grow { if got := len(expand(f.rand, tmpl, nil)); got < tmpl.grow {
t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow)
} }
} }
@@ -347,3 +365,17 @@ func TestAlternationThreeWay(t *testing.T) {
t.Fatalf("3-way alternation produced %v, want A, B and C", seen) t.Fatalf("3-way alternation produced %v, want A, B and C", seen)
} }
} }
func TestArgErrorsNameTheSpelling(t *testing.T) {
for src, want := range map[string]string{
`"{float(1,2,02)}"`: "write 2",
`{"format":"{calc(a,02)}","a":"1"}`: "write 2",
`"{digits(+5)}"`: "write 5",
`"{hex(99999999999999999999)}"`: "exceeds the maximum",
`"{int(007,9)}"`: "write 7",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error saying %q", src, err, want)
}
}
}
+6
View File
@@ -51,3 +51,9 @@ func TestTransformArgs(t *testing.T) {
} }
} }
} }
func TestAsciiKeepsDEL(t *testing.T) {
if got := mustRender(t, engine(1), `{"format":"{ascii(x)}","x":"a\u007fb"}`); got != "a\u007fb" {
t.Errorf("ascii over DEL = %q, want it kept: DEL is ASCII", got)
}
}