Compare commits

...

14 Commits

Author SHA1 Message Date
lilleman b45a1edae0 Cut the comments that restate the README or narrate the arrangement, and name the record fence asymmetry in Decisions
Tests / vet + fmt + tests (pull_request) Successful in 1m6s
Tests / vet + fmt + tests (push) Successful in 7s
2026-09-16 10:43:25 +02:00
lilleman 0f645d7092 Delete the fresh-draw fence the self-reference rule made unreachable, and pay for the draw walks only where data binds a reference
Tests / vet + fmt + tests (pull_request) Successful in 1m6s
2026-09-16 10:29:23 +02:00
lilleman 0443ecdf13 Tests: gate the allocations a reference path costs across a repeat and in a named draw group 2026-09-16 10:29:00 +02:00
lilleman c60ebb3faa Merge the two reference-path walks, and correct the bare reference Decision, a stale comment and the draw map spellings
Tests / vet + fmt + tests (pull_request) Successful in 1m8s
2026-09-16 09:55:58 +02:00
lilleman 9f4331a1f3 Tests: a record column reaching back into a category through another one is refused at load 2026-09-16 09:55:36 +02:00
lilleman a995aba4e9 Refuse a reference into the category it sits in, and run a record's column fences at load
Tests / vet + fmt + tests (pull_request) Successful in 1m8s
2026-09-16 09:42:36 +02:00
lilleman 3e535bcc43 Tests: a reference into the category it sits in is refused at load, and a record's column fences run there too 2026-09-16 09:42:18 +02:00
lilleman 68d2461704 Name the draw option drawGroup, refuse one whose reads all sit behind a nested draw group, and record why the two fences stay apart
Tests / vet + fmt + tests (pull_request) Successful in 1m7s
2026-09-16 08:55:55 +02:00
lilleman 53a87da9b7 Tests: name the draw option drawGroup, and refuse a draw group whose reads all sit behind a nested one 2026-09-16 08:55:39 +02:00
lilleman d3f430bf53 Render a repeat iteration in no group, refuse a group that names nothing, key group names by category, and own reference overlap in one fence
Tests / vet + fmt + tests (pull_request) Successful in 1m6s
2026-09-15 23:31:48 +02:00
lilleman f6b8f4fb50 Tests: a repeat iteration renders in no group, a group that names nothing is refused, a group name is local to its category, and one overlap verdict wherever the reads sit 2026-09-15 23:30:46 +02:00
lilleman 5ed45cc55c Rename the folder node type from group to folder, freeing group for the draw option 2026-09-15 23:25:39 +02:00
lilleman 246f99a319 One reference draw per render in both views: a repeat iteration draws anew, a group option draws apart, and one fence refuses what a render's draws cannot answer for
Tests / vet + fmt + tests (pull_request) Successful in 1m6s
2026-09-15 22:59:30 +02:00
lilleman 4b31482a53 Tests: a reference path is one draw per render in both views, a repeat iteration draws anew, a group draws apart, and a bare reference beside a path is refused 2026-09-15 22:45:13 +02:00
20 changed files with 932 additions and 357 deletions
+103 -48
View File
@@ -124,16 +124,17 @@ string and a backslash as an escape.
A record written only to emit columns still needs a `format` — the grammar's one
required key — so `"format": ""` carries the fields with an inert format: it
renders nothing by `Fake`, and is compiled only so the tree's fences still run.
The columns are the point, and their facts stay together: two columns that read a
path into one category — `{/currency.code}` and `{/currency.symbol}` — share one
draw of it, so the record is internally consistent. A bare `{/currency}` names no
field, so it keeps drawing on its own. That one draw is also why two
columns may not read overlapping reference *paths* — `{/cat.a}` beside
The columns are the point, and their facts stay together: a record is one render, so
columns that read a path into one category — `{/currency.code}` and
`{/currency.symbol}` — read one draw of it ([References](#references)), and a
[draw group](#draw-group) draws a column apart. That one draw is also why the columns of one
record may not overlap — `{/cat.a}`, or a bare `{/cat}`, beside
`{/cat.a.b}` is refused, naming the fields to write instead, as
[One draw, one spelling](#one-draw-one-spelling) refuses that pair inside a single
format. A column may not reference the record it belongs to by any spelling: `{/users.first}`
or a bare `{/users}` inside `users` describes a draw other than the columns beside
it, so put a value two columns share in its own category and reference that. A field hold, transform or
format. Both fences run at load, so a category that loads renders as either shape. A
category never references itself — `{/users.first}` or a bare `{/users}` inside `users`
describes a draw other than the fields beside it — so read a sibling as a path, and put
a value two fields share in its own category and reference that. A field hold, transform or
operand ties fields together within one column as always (see
[Correlated fields](#correlated-fields) and [Decisions](#decisions)).
@@ -341,8 +342,8 @@ different datatypes.
### Options and fields
`format`, `weight`, `repeat`, `separator` and `datatype` are the only options; **any
other key is a field** (see [Decisions](#decisions)). An object that does nothing a
`format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options;
**any other key is a field** (see [Decisions](#decisions)). An object that does nothing a
string can't — only a `format` — is rejected naming the string, as is a one-item
choice naming its item.
@@ -438,14 +439,41 @@ without naming `sv_SE`:
"Hej, {/en_US.person}!"
```
Renders e.g. `Hej, Pat Smith!`. A reference into a category is held like a
[correlated](#correlated-fields) path — `{.person.femalefirst} {.person.last}` name
one person, `{lowercase(.person.femalefirst)}` reads that same draw, and
`{.person.femalefirst}` beside `{/sv_SE.person.last}` in `sv_SE` is one person too — while a bare
`{/misc.uuid} {/misc.uuid}` is two draws. Rejected at `New`: a path that is
unknown, names a folder, has no folder above, or reads a field not every variant
of a choice carries, and a reference that leads back to its own value, directly,
mutually or through a chain.
Renders e.g. `Hej, Pat Smith!`. A reference path into a category is held like a
[correlated](#correlated-fields) path, but for the whole render — one `Fake`, or one
record — rather than one format: `{.person.femalefirst} {.person.last}` name one
person, as do the same two references in sibling fields or a nested template, and
`{lowercase(.person.femalefirst)}` reads that same draw. Each `repeat` iteration is
a render of its own, in no group, so it draws anew, and a [draw group](#draw-group) holds a
draw apart. A bare reference names no field and makes its own picks each time —
`{/misc.uuid} {/misc.uuid}` is two draws — while the reference paths inside what it
renders still read the render's draws. Rejected at `New`: a path that is
unknown, names a folder, has no folder above, reads a field not every variant
of a choice carries, or names the category the reference sits in, and a reference
that leads back to its own value, directly, mutually or through a chain.
### Draw group
A template may carry `drawGroup` to hold its reference draws apart: every reference path
it renders, however deep short of a `repeat` or a nested `drawGroup`, reads the draw of
that group, and the templates of one category naming one group in one render read one
draw. A name is local to its category, so a category another one references never joins
its groups by name; the unnamed group spans them all.
```json
{ "format": "{payer} pays {payee}; signed {signature}",
"payer": { "format": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}", "drawGroup": "payer" },
"payee": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}",
"signature": { "format": "{/sv_SE.person.last}", "drawGroup": "payer" } }
```
Renders e.g. `Sara Eriksson pays Ebba Lind; signed Eriksson`: the signature reads the
payer's draw, while the payee is drawn apart. Rejected at load, each naming nothing: a
`drawGroup` of `""` (the default); one naming the draw group its template already draws
in; one on a template that renders no reference path short of a `repeat` or a nested
`drawGroup`, on a `repeat` itself — each iteration renders in no draw group — or on an
inline template's root, which nothing references. So is a path reading into a level that
carries a `drawGroup`.
### Correlated fields
@@ -478,12 +506,14 @@ The sub-fields stay addressable — `Fake("address.place.locality")` renders, an
### One draw, one spelling
A name any token reads as a path (`{p.first}`) or as an operand (`{calc(net * 2)}`,
`{uppercase(w)}`) is drawn **once per expansion**, and every other route to it —
a bare `{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering
`{/cat.p.last}`, at any depth — is a load error naming the spelling to use. A
name nothing reads that way is drawn each time: `{word} {word}` differs. An
expansion is one render of one format, so each `repeat` iteration and each nested
template draws again.
`{uppercase(w)}`) is drawn **once per expansion**, a reference path (`{/cat.p.first}`)
**once per render** in its [draw group](#draw-group), and every other route to either — a bare
`{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering `{/cat.p.last}`
beside `{p.first}`, a bare `{/cat}` beside `{/cat.p.first}`, at any
depth — is a load error naming the spelling to use. A name nothing reads that way is
drawn each time: `{word} {word}` differs. An expansion is one render of one format, so
each nested template draws its own names again; a render is one `Fake` or one record,
and each `repeat` iteration is an expansion and a render of its own.
```text
token {p} renders a level that {p.first} reads a path into; name the fields you want instead
@@ -492,7 +522,9 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan
### Performance
Each file is parsed, validated and weight-indexed once, in `New`. A `Fake` call
Each file is parsed, validated and weight-indexed once, in `New`. Proving the draw
fences adds one pass over the loaded tree, and walks what a render reads only where
data binds a reference, so a set that binds none pays for the pass alone. A `Fake` call
then costs about what its output costs: an unweighted pick is O(1) whatever the
list's length, a weighted one O(log n), and long formats, deep nesting and many
tokens add cost in proportion to the output.
@@ -524,7 +556,7 @@ tokens add cost in proportion to the output.
## Decisions
- **Options and fields share one namespace.** `format`, `weight`, `repeat`,
`separator` and `datatype` are reserved; every other key is a field. Nesting fields under a
`separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a
key, or prefixing options, would tax every template to guard against a
misspelt option.
- **`{a|b}` stays beside nested choices.** `[[…], […]]` picks the same way, but
@@ -571,10 +603,12 @@ tokens add cost in proportion to the output.
to have would make `--seed 42` machine-dependent. Data still lives in `data/`
as JSON; `--data-path` layers over it.
- **A bare reference draws each time; a reference path is held.** `{/p} {/p}`
is two draws, as `{word} {word}` is, while `{/p.first}` beside a nested template
rendering `{/p.first}` is a load error: a bare token is by contract an
independent draw, a path pins its level, and any route into a pinned level from
another expansion could show another row.
is two draws, as `{word} {word}` is, while every `{/p.first}` in one render reads
one draw, and a bare `{/p}` beside them is a load error: a bare token
is by contract an independent draw, a path pins its level, and a fresh draw of a
pinned level could show another row. A builtin's operand holds what it reads for
its expansion, references included, so `{uppercase(/p)} {/p}` is one draw — the
rule every operand follows.
- **Reference sigils follow the filesystem.** `/` is the root, `.` this file's
folder, `..` the folder above — what those spellings already mean to anyone who
has typed a path. A locale's files reach each other without naming the locale,
@@ -640,24 +674,44 @@ tokens add cost in proportion to the output.
see a caller's types, so the first `FakeStruct` for a type compiles its tags and the
answer, error included, is kept per type: a test's first call is its load, and no
`NewStruct` handle is needed, as the cache already compiles once.
- **A record shares one reference draw per category.** Two columns that reference
one category — `{/currency.code}` beside `{/currency.symbol}` — read one draw of
it, so a record's facts agree the way a template's [correlated
fields](#correlated-fields) do. The draw is one per record, so it spans a
column's `repeat` and nested templates too (one record is one coherent unit);
a bare reference — `{/currency}`, no field — stays an independent draw every
time, the rule a format string already follows. Only references share: a sibling
field is local to its own column, so a `first` column does not silently bind to
a `first` in the column next to it.
The string view of that same template does not share. `Fake` renders each
sibling field as its own expansion, so a `{/currency.code}` field beside a
`{/currency.symbol}` field is two draws and may render `EUR $`; writing both
references in one `format` holds them together, as
[One draw, one spelling](#one-draw-one-spelling) says. The scope is what makes a
row coherent when the columns *are* the output, and there the caller cannot fall
back on one format string. Widening it to every render would change what `Fake`
has emitted since the start, for a correlation a single format already reaches.
- **A render shares one reference draw per category, per group.** Every reference
path into a category in one `Fake`, or one record, reads one draw of it, so a
value's facts agree across its fields, nested templates and columns alike —
`{/currency.code}` in one field and `{/currency.symbol}` in another name one
currency, whichever view renders them. A `repeat` iteration is a render of its
own, since repeating asks for another entity, and a [draw group](#draw-group) names further
entities within one render, so a payer and a payee are two groups over one
`person` rather than two copies of it. Only references share: a sibling field is
local to its own expansion, so a `first` column does not silently bind to a
`first` in the column next to it.
- **A draw group name is local to its category.** A category's groups are its own
entities, so a caller naming a group the same way never joins them by accident,
and renaming a group inside one file changes no render elsewhere. The unnamed
group still spans categories, since facts that belong together across categories
must agree.
- **The expansion hold and the render's draws are two fences.** One proves a sibling
path or an operand is reached only by its readers within an expansion, the other
does the same for reference paths across a render and its draw groups. They pin
different things — an operand pins the value its own render produced and stops at a
reference, a path pins every level it passes through — so one walk would carry both
rules and both scopes anyway, and tell them apart at every step.
- **A record makes its draw maps up front, a `Fake` on its first read.** A record's
columns always read through the render's draws, so making the maps where the set is
declared keeps them on that frame's stack. A `Fake` often reads no reference path at
all, and making them anyway cost about a fifth of the cheapest render, so it makes
them on the first read instead, at two heap allocations for a render that does share
a draw. The allocation gate over a repeat of a reference path and over a named draw
group prices that, and pins the two measures that keep a draw set off the heap.
- **A category never references itself, and a record's fences run at load.** A category
is one unit: a reference back into it — `{/users.first}` inside `users` — describes a
draw other than the fields beside it, so `New` refuses it and the sibling path stays
the one spelling for a field of one's own. A value two fields share goes in its own
category, which both reference. That settled, a record's column fences run at `New`
too, so a category that loads renders as whichever shape is asked for, and a reference
reaching back into a category through another one is refused there as the overlap it
is. Which reads those fences weigh differs on purpose: a record-only template's inert
format renders nothing, so a `drawGroup` on it can never matter and is refused, while
one on a rendering format can matter to a caller that bare-references it.
- **A record's column set is fixed before the first draw.** Only a category-level
template is a record: a path descending into a field, or naming a folder or a
choice, errors. A tail may pass through a choice whose variants carry different
@@ -737,6 +791,7 @@ struct.go structs: FakeStruct, fake tags, and a field's Go type as its col
inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
hold.go the hold: one draw per expansion for paths and operands, and its fences
draw.go one reference draw per render and group: draw sets, the group option, and its fence
reference.go reference sigils, and binding references across the tree
graph.go the render graph: edges, cycles, the repeat bound, tree walks
builtins.go the {name()} function registry and its implementations
+12 -12
View File
@@ -28,10 +28,10 @@ func (s dataSource) name(p string) string {
}
// loadData loads every source into one namespace tree and returns its root
// children. A directory becomes a group; each *.json file in it compiles to a node
// children. A directory becomes a folder; each *.json file in it compiles to a node
// keyed by its base name (address.json -> "address"); each subdirectory becomes a
// nested group, so folders turn into dot-path segments. Sources merge left to right:
// matching groups merge by their children, and any other clash is won by the last
// nested folder, so folders turn into dot-path segments. Sources merge left to right:
// matching folders merge by their children, and any other clash is won by the last
// source loaded. Once merged, linkRefs binds every reference against the
// final tree.
func loadData(sources []dataSource) (map[string]node, error) {
@@ -74,15 +74,15 @@ func loadData(sources []dataSource) (map[string]node, error) {
return root, nil
}
// loadDir compiles one directory into a group. fs.ReadDir yields entries sorted
// loadDir compiles one directory into a folder. fs.ReadDir yields entries sorted
// by name, so the tree is built deterministically. Empty subdirectories (no JSON
// anywhere under them) are skipped rather than added as empty namespaces.
func loadDir(src dataSource, dir string) (*group, error) {
func loadDir(src dataSource, dir string) (*folder, error) {
entries, err := fs.ReadDir(src.fsys, dir)
if err != nil {
return nil, fmt.Errorf("%s: %w", src.name(dir), err)
}
g := &group{children: map[string]node{}}
g := &folder{children: map[string]node{}}
for _, e := range entries {
if strings.HasPrefix(e.Name(), ".") { // hidden: a checkout or an editor's file, never data
continue
@@ -99,8 +99,8 @@ func loadDir(src dataSource, dir string) (*group, error) {
return g, nil
}
// loadFolder adds a subdirectory as a nested group, unless nothing under it is data.
func loadFolder(src dataSource, g *group, full, name string) error {
// loadFolder adds a subdirectory as a nested folder, unless nothing under it is data.
func loadFolder(src dataSource, g *folder, full, name string) error {
child, err := loadDir(src, full)
if err != nil {
return err
@@ -117,7 +117,7 @@ func loadFolder(src dataSource, g *group, full, name string) error {
// loadFile compiles a *.json file into a category named after it; any other file
// is skipped.
func loadFile(src dataSource, g *group, full, file string) error {
func loadFile(src dataSource, g *folder, full, file string) error {
if !strings.HasSuffix(file, ".json") {
return nil
}
@@ -141,13 +141,13 @@ func loadFile(src dataSource, g *group, full, file string) error {
return nil
}
// mergeChildren overlays src onto dst. Two groups under the same key merge
// mergeChildren overlays src onto dst. Two folders under the same key merge
// recursively (so locales/categories from several paths combine); every other
// key is replaced, making the last-loaded directory win on a conflict.
func mergeChildren(dst, src map[string]node) {
for k, v := range src {
if dg, ok := dst[k].(*group); ok {
if sg, ok := v.(*group); ok {
if dg, ok := dst[k].(*folder); ok {
if sg, ok := v.(*folder); ok {
mergeChildren(dg.children, sg.children)
continue
}
+326
View File
@@ -0,0 +1,326 @@
package fejkdata
import (
"fmt"
"sort"
"strings"
)
// drawSet is one render's reference draws: the unnamed draw group's, and each named one's.
type drawSet struct {
unnamed draws
named map[string]*draws
}
// drawScope is where a render reads its reference paths: a draw set, in the draw group of the
// template rendering.
type drawScope struct {
set *drawSet
group string
}
// newDrawSet makes the unnamed group's maps where the set is declared, keeping them on that frame's
// stack for a render that reads through them; a zero drawSet makes them on its first read instead.
func newDrawSet() drawSet {
return drawSet{unnamed: draws{variant: map[string]node{}, value: map[string]draw{}}}
}
// renderOnce renders n as one render, over draws of its own.
func renderOnce(s *session, n node) string {
var set drawSet
return render(s, n, drawScope{set: &set})
}
// in is the scope t renders in: its draw group where it names one, else its caller's.
func (sc drawScope) in(t *template) drawScope {
if t.drawGroupKey != "" {
sc.group = t.drawGroupKey
}
return sc
}
// draws is the set's draws for sc's draw group.
func (sc drawScope) draws() *draws {
if sc.group == "" {
return &sc.set.unnamed
}
d, drew := sc.set.named[sc.group]
if !drew {
if sc.set.named == nil {
sc.set.named = map[string]*draws{}
}
d = &draws{variant: map[string]node{}, value: map[string]draw{}}
sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's draw set, to the heap
}
return d
}
// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each
// iteration renders in no draw group.
func drawGroupOf(m map[string]any, repeat int) (string, error) {
v, ok := m["drawGroup"]
if !ok {
return "", nil
}
name, ok := v.(string)
switch {
case !ok:
return "", fmt.Errorf("drawGroup must be a string, got %T", v)
case name == "":
return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`)
case repeat > 1:
return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name)
}
return name, nil
}
// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name
// is local to its category.
func (t *template) keyDrawGroup(category string) {
if t.drawGroup != "" {
t.drawGroupKey = category + "/" + t.drawGroup
}
}
// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again,
// short of a repeat or another draw group.
func checkNestedDrawGroup(fields map[string]node, group string) error {
if group == "" {
return nil
}
var walk func(path string, n node) error
walk = func(path string, n node) error {
t, isTemplate := n.(*template)
switch {
case isTemplate && t.drawGroup == group:
return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group)
case isTemplate && (t.drawGroup != "" || t.repeat > 1):
return nil
}
for _, c := range contained(n) {
if err := walk(join(path, c.name), c.node); err != nil {
return err
}
}
return nil
}
for _, name := range sortedNames(fields) {
if err := walk(name, fields[name]); err != nil {
return err
}
}
return nil
}
// drawCheck fences each template of a scope as a render of its own, remembering which nodes read a
// reference path.
type drawCheck struct {
memo map[readsMemo]bool
}
type readsMemo struct {
n node
stopAtGroup bool
}
// checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference
// path inside a repeat or a nested draw group, which draw apart from it whatever it names.
func (c *drawCheck) checkDrawGroup(path string, n node) error {
if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) {
return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup)
}
return nil
}
func (c *drawCheck) checkDraws(path string, n node) error {
t, ok := n.(*template)
if !ok || !c.readsPath(t) {
return nil
}
w := newDrawWalk()
for _, e := range renderEdges(t) {
w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}})
}
if err := w.check(); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
// checkRecordDraws fences the columns of a record — a template compiled at the top without a
// repeat — so a load proves the record view of it as well as the string view.
func (c *drawCheck) checkRecordDraws(path string, n node) error {
t, ok := n.(*template)
if !ok || !t.record {
return nil
}
// A record-only template's format renders nothing, so weigh the columns, not the format.
columns := recordColumns(t)
reads := false
for _, name := range columns {
reads = reads || c.readsPath(t.fields[name])
}
if !reads {
return nil
}
if err := checkColumnDraws(t, columns); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
// readsPath reports whether rendering n reads a reference path, short of a repeat, which renders
// over draws of its own.
func (c *drawCheck) readsPath(n node) bool { return c.reads(n, false) }
// splitsDraws reports whether rendering n reads a reference path that n's own draw group answers
// for: one outside a repeat and outside a nested draw group, which hold their own draws.
func (c *drawCheck) splitsDraws(n node) bool { return c.reads(n, true) }
// reads walks what rendering n renders for a reference path, stopping at a repeat — and at a nested
// draw group when stopAtGroup — since each holds draws of its own.
func (c *drawCheck) reads(n node, stopAtGroup bool) bool {
k := readsMemo{n, stopAtGroup}
if r, done := c.memo[k]; done {
return r
}
r := false
for _, e := range renderEdges(n) {
if a, isRef := refRead(n, e.label); (isRef && len(a.tail) > 0) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) {
r = true
break
}
}
if c.memo == nil {
c.memo = map[readsMemo]bool{}
}
c.memo[k] = r
return r
}
func repeats(n node) bool {
t, isTemplate := n.(*template)
return isTemplate && t.repeat > 1
}
func grouped(n node) bool {
t, isTemplate := n.(*template)
return isTemplate && t.drawGroupKey != ""
}
// refRead is the reference an edge of n reads; false when the edge reads none.
func refRead(n node, label string) (arm, bool) {
t, isTemplate := n.(*template)
if !isTemplate {
return arm{}, false
}
a := splitArm(label, t.refs)
return a, isRef(a.key)
}
// checkColumnDraws fences a record's columns as one render.
func checkColumnDraws(t *template, columns []string) error {
w := newDrawWalk()
for _, name := range columns {
w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}})
}
return w.check()
}
// drawWalk gathers the references one render reads, by draw group, for check to compare.
type drawWalk struct {
reads []pathRead
read map[drawKey]bool
seen map[drawVisit]bool
}
// drawAt is where a walk stands: the draw group it draws in, and how the render's root reached it.
type drawAt struct {
group string
route drawRoute
}
// drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label.
type drawRoute struct{ spelling, label string }
// pathRead is one reference a render reads: a path, or a bare reference with no tail.
type pathRead struct {
at drawAt
a arm
}
type drawVisit struct {
n node
group string
}
type drawKey struct {
group string
path string
}
func newDrawWalk() *drawWalk {
return &drawWalk{read: map[drawKey]bool{}, seen: map[drawVisit]bool{}}
}
// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops
// there.
func (w *drawWalk) walk(n node, at drawAt) {
v := drawVisit{n, at.group}
if w.seen[v] {
return
}
w.seen[v] = true
if repeats(n) {
return
}
if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" {
at.group = t.drawGroupKey
}
for _, e := range renderEdges(n) {
w.edge(n, e, at)
}
}
func (w *drawWalk) edge(from node, e renderEdge, at drawAt) {
if a, reads := refRead(from, e.label); reads {
if k := (drawKey{at.group, a.path}); !w.read[k] {
w.read[k] = true
w.reads = append(w.reads, pathRead{at, a})
}
}
w.walk(e.to, at)
}
// check refuses what one draw per reference path cannot answer for: a read of a level beside a path
// another read takes into it. Reads are compared in path order, so which pair is reported does not
// vary.
func (w *drawWalk) check() error {
sort.SliceStable(w.reads, func(i, j int) bool {
if w.reads[i].at.group != w.reads[j].at.group {
return w.reads[i].at.group < w.reads[j].at.group
}
return w.reads[i].a.path < w.reads[j].a.path
})
for i, level := range w.reads {
for _, into := range w.reads[i+1:] {
if into.at.group == level.at.group && strings.HasPrefix(into.a.path, level.a.path+".") {
return overlapError(level.at.route, level.a.name, into)
}
}
}
return nil
}
func overlapError(route drawRoute, ref string, into pathRead) error {
return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name))
}
// spelled names the route, and the reference it reaches a draw by where its root edge is not that
// reference.
func (r drawRoute) spelled(ref string) string {
if ref == "" || ref == r.label {
return r.spelling
}
return fmt.Sprintf("%s with {%s}", r.spelling, ref)
}
+2 -2
View File
@@ -142,10 +142,10 @@ func (f *Generator) List() []string {
}
// paths lists the dot paths addressable from n, relative to it, where "" is n
// itself. A group has no value of its own, so it contributes only its children's.
// itself. A folder has no value of its own, so it contributes only its children's.
func paths(n node) []string {
switch n := n.(type) {
case *group:
case *folder:
var out []string
for _, name := range sortedNames(n.children) {
for _, p := range paths(n.children[name]) {
+21 -3
View File
@@ -51,7 +51,7 @@ type namedNode struct {
func contained(n node) []namedNode {
switch n := n.(type) {
case *group:
case *folder:
return named(n.children)
case *choice:
out := make([]namedNode, len(n.items))
@@ -69,7 +69,7 @@ func contained(n node) []namedNode {
// named skips a bound {/path} key: it is a render edge, not containment, so using
// it as a path segment would report a node under a path that does not reach it. Only
// a template's fields hold bindings — loadDir skips a dot-prefixed entry, so a
// group's children never carry the prefix — so this one skip serves both.
// folder's children never carry the prefix — so this one skip serves both.
func named(m map[string]node) []namedNode {
out := make([]namedNode, 0, len(m))
for _, name := range sortedNames(m) {
@@ -111,7 +111,7 @@ func (e renderEdge) reached() string {
// renderEdges lists the children rendering n recurses into, mirroring expand: a
// choice's items, and a template's field/reference tokens plus its operands. A
// group renders nothing, so it has no edges.
// folder renders nothing, so it has no edges.
func renderEdges(n node) []renderEdge {
switch n := n.(type) {
case *choice:
@@ -196,6 +196,24 @@ func checkRenders(s nodeScope) error {
if err := s(heldCheck); err != nil {
return err
}
fence := &drawCheck{}
refs := false
if err := s(func(path string, n node) error {
if t, ok := n.(*template); ok && len(t.refs) > 0 {
refs = true
}
return fence.checkDrawGroup(path, n)
}); err != nil {
return err
}
if refs {
if err := s(fence.checkDraws); err != nil {
return err
}
if err := s(fence.checkRecordDraws); err != nil {
return err
}
}
return s((&valueProof{}).checkDatatype)
}
+39 -30
View File
@@ -6,11 +6,12 @@ import (
"strings"
)
// heldCheck rejects every route to a held name except the ones that read its draw.
// An expansion holds one draw of that name; anything else that renders it draws
// again, and the two disagree. checkNoOverlap settles the spellings within one
// format (a token, an operand); this settles the rest — a reference, whether it
// sits in that format or in anything the format renders, however deep.
// heldCheck rejects every route to a held sibling name except the ones that read its
// draw. An expansion holds one draw of that name; anything else that renders it draws
// again, and the two disagree. checkNoOverlap settles the spellings within one format
// (a token, an operand); this settles the rest — a reference, whether it sits in that
// format or in anything the format renders, however deep. A reference path is held
// for the whole render instead, which drawCheck fences.
func heldCheck(path string, n node) error {
t, ok := n.(*template)
if !ok || len(t.held) == 0 {
@@ -18,6 +19,9 @@ func heldCheck(path string, n node) error {
}
readers := boundReaders(t.format, t.bound, t.refs)
for _, head := range heldHeads(t) {
if _, isPath := t.bound[head]; isPath && isRef(head) {
continue
}
if err := checkHeadHeld(t, head, readers); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
@@ -175,12 +179,11 @@ func renders(n node, want, seen map[node]bool) bool {
return false
}
// checkNoOverlap rejects a format that both renders a level and reads a path into
// it — {p} beside {p.first}, {p.addr} beside {p.addr.city}, {.p} beside
// {/sv_SE.p.first}. The path reads the level's held draw while rendering the level
// expands it afresh, so their values would disagree. Reads are compared by their
// one spelling, in sorted order, so which pair is reported depends neither on how
// a reference was written nor on where the tokens sit.
// checkNoOverlap rejects a format that both renders a sibling level and reads a path
// into it — {p} beside {p.first}, {p.addr} beside {p.addr.city}. The path reads the
// level's held draw while rendering the level expands it afresh, so their values would
// disagree. Reads are compared in sorted order, so which pair is reported does not
// depend on where the tokens sit.
func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error {
names := boundReaders(format, bound, refs)
// Stable over one format-order scan, so two readers of one name (a token and a
@@ -200,7 +203,7 @@ func checkNoOverlap(format string, bound map[string]string, refs map[string]refB
// spelling, and how to name it.
type reader struct{ name, path, label string }
// boundReaders lists every way a format reaches a bound field, in the order the
// boundReaders lists every way a format reaches a bound sibling field, in the order the
// format writes them. An operand renders its field, so it names a level exactly
// as a token does; one scan finds both, which is what puts them in one order.
func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader {
@@ -212,14 +215,14 @@ func boundReaders(format string, bound map[string]string, refs map[string]refBin
if fn, _, isFunc := funcCall(t.body); isFunc {
for _, operand := range tokenOperands(t.body) {
a := splitArm(operand, refs)
if _, isBound := bound[a.key]; isBound {
if _, isBound := bound[a.key]; isBound && !isRef(a.key) {
names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)})
}
}
return nil
}
for _, a := range splitArms(t.body, refs) {
if _, isBound := bound[a.key]; isBound {
if _, isBound := bound[a.key]; isBound && !isRef(a.key) {
names = append(names, reader{a.name, a.path, "token {" + a.name + "}"})
}
}
@@ -252,9 +255,10 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding)
})
}
// draws is what an expansion has already drawn for its held names: the variant each
// was drawn as, so every path under it reads one row, and the draw each read made, by
// its one spelling, so the same read written twice reads one draw.
// draws is what has already been drawn for held names: the variant each was drawn as,
// so every path under it reads one row, and the draw each read made, by its one
// spelling, so the same read written twice reads one draw. An expansion keeps one for
// its sibling names, and a render one per group for its reference paths.
type draws struct {
variant map[string]node
value map[string]draw
@@ -273,14 +277,14 @@ type draw struct {
// gives one value, and a shown operand is the operand computed. Every other name is
// drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and
// linkRefs prove every step, so the walk cannot fail.
func readField(s *session, t *template, held, refScope *draws, a arm) draw {
func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw {
if !t.held[a.key] {
if len(a.tail) > 0 {
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
}
return draw{text: render(s, t.fields[a.key], refScope)}
return draw{text: render(s, t.fields[a.key], sc)}
}
d := readScope(held, refScope, a)
d := readScope(held, sc, a)
if r, done := d.value[a.path]; done {
return r
}
@@ -296,37 +300,42 @@ func readField(s *session, t *template, held, refScope *draws, a arm) draw {
n, drew := d.variant[key]
if !drew {
n = drawn(s, c)
if d.variant == nil {
d.variant = map[string]node{}
}
d.variant[key] = n
}
return []node{n}, nil
},
leaf: func(n node) error { r = renderLeaf(s, n, refScope); return nil },
leaf: func(n node) error { r = renderLeaf(s, n, sc); return nil },
})
if d.value == nil {
d.value = map[string]draw{}
}
d.value[a.path] = r
return r
}
// readScope is the draws a held read keeps its draw in: for a reference that reads a path,
// refScope, so its draw outlives this expansion; for a sibling, or a reference read whole, held.
func readScope(held, refScope *draws, a arm) *draws {
if isRef(a.key) && refScope != nil && len(a.tail) > 0 {
return refScope
// the render's draws for its group, so its draw spans the render; for a sibling, or a
// reference read whole, held.
func readScope(held *draws, sc drawScope, a arm) *draws {
if isRef(a.key) && len(a.tail) > 0 {
return sc.draws()
}
return held
}
// renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one
// reference alone whose read drew null.
func renderLeaf(s *session, n node, scope *draws) draw {
func renderLeaf(s *session, n node, sc drawScope) draw {
n = drawn(s, n)
if _, isNull := n.(*null); isNull {
return draw{null: true}
}
r := draw{text: render(s, n, scope)}
r := draw{text: render(s, n, sc)}
if t, _ := n.(*template); t != nil && t.readsColumn != nil {
if d := readScope(nil, scope, t.readsColumn.a); d != nil {
r.null = d.value[t.readsColumn.a.path].null
}
r.null = sc.in(t).draws().value[t.readsColumn.a.path].null
}
return r
}
+82 -34
View File
@@ -91,17 +91,28 @@ func TestReferenceNamingABoundLevelIsRejected(t *testing.T) {
// A reference can name a bound level from the data root, which renders it
// afresh beside the path that reads its held draw — the same overlap by
// another spelling.
rejected := map[string]string{
"reference names the head": `{"format":"{p.first}|{/cat.p}","p":{"format":"{first}","first":["Anna","Bo"]}}`,
"reference names the leaf": `{"format":"{p.addr}|{/cat.p.addr}","p":{"format":"x","addr":["A","B","C","D"]}}`,
// A category never references itself, so each reference back into cat sits in a
// second category cat renders.
rejected := map[string]map[string]string{
"reference names the head": {
"cat": `{"format":"{p.first}|{/hop}","p":{"format":"{first}","first":["Anna","Bo"]}}`,
"hop": `"{/cat.p}"`,
},
"reference names the leaf": {
"cat": `{"format":"{p.addr}|{/hop}","p":{"format":"x","addr":["A","B","C","D"]}}`,
"hop": `"{/cat.p.addr}"`,
},
// The reference need not sit in the format that binds: any field it renders
// reaches the level just the same, however deep.
"reference from a sibling field": `{"format":"{p.first}|{inner}","p":[` +
"reference from a sibling field": {
"cat": `{"format":"{p.first}|{inner}","p":[` +
`{"format":"{first}-{last}","first":"A","last":"1"},{"format":"{first}-{last}","first":"B","last":"2"}],` +
`"inner":"{/cat.p}"}`,
`"inner":"{/hop}"}`,
"hop": `"{/cat.p}"`,
},
}
for name, file := range rejected {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file})))
for name, files := range rejected {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, files)))
if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: New = %v, want the reference rejected as an overlap", name, err)
}
@@ -120,7 +131,8 @@ func TestCycleReachedOnlyByAPathTokenIsRejected(t *testing.T) {
// cycle walk has to follow it there. A head whose own format names nothing
// would otherwise hide the cycle until render, where it is fatal.
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"a": `{"format":"{p.x}","p":{"format":"static","x":"{/a}"}}`,
"a": `{"format":"{p.x}","p":{"format":"static","x":"{/b}"}}`,
"b": `"{/a}"`,
})))
if err == nil || !strings.Contains(err.Error(), "reference cycle") {
t.Fatalf("New = %v, want the cycle through {p.x} rejected", err)
@@ -131,8 +143,9 @@ func TestALevelRenderedOnlyByAPathTokenIsHeld(t *testing.T) {
// {p.a} renders q, so it is a route to the level {q.x} holds — even though p's
// own format names nothing.
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"thing": `{"format":"{p.a} {q.x}","p":{"format":"static","a":"{/thing.q}"},` +
"thing": `{"format":"{p.a} {q.x}","p":{"format":"static","a":"{/hop}"},` +
`"q":{"format":"{x}","x":["1","2"]}}`,
"hop": `"{/thing.q}"`,
})))
if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Fatalf("New = %v, want the second route to q rejected", err)
@@ -149,16 +162,20 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
files map[string]string
want string // the route the error names, spelled as the author wrote it
}{
// A category never references itself, so each route back into cat sits in a
// second category cat renders.
"a reference beside the operand": {
map[string]string{
"cat": `{"format":"{/cat.net} x 2 = {calc(net * 2, 2)}","net":["10.00","20.00"]}`,
"cat": `{"format":"{/hop} x 2 = {calc(net * 2, 2)}","net":["10.00","20.00"]}`,
"hop": `"{/cat.net}"`,
},
`{/cat.net} renders "net"`,
`{/hop} renders "net"`,
},
"a reference one level down": {
map[string]string{
"cat": `{"format":"{calc(net * 2, 2)} {q}","net":["10.00","20.00"],` +
`"q":"{/cat.net}"}`,
`"q":"{/hop}"}`,
"hop": `"{/cat.net}"`,
},
`{q} renders "net"`,
},
@@ -167,7 +184,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"a reference to an operand wrapped in a choice": {
map[string]string{
"cat": `{"format":"{calc(n * 2, 2)} {q}","n":{"format":"{v}","v":["1","2"]},` +
`"q":"{/cat.n}"}`,
`"q":"{/hop}"}`,
"hop": `"{/cat.n}"`,
},
`{q} renders "n"`,
},
@@ -176,7 +194,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"an operand reaching another operand": {
map[string]string{
"cat": `{"format":"{calc(a + b, 0)}","a":{"format":"{x}","x":["1","2"]},` +
`"b":"{/cat.a}"}`,
`"b":"{/hop}"}`,
"hop": `"{/cat.a}"`,
},
`calc operand "b" renders "a"`,
},
@@ -185,15 +204,17 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
// rejected, so the reference spelling has to be.
"a reference into the operand": {
map[string]string{
"cat": `{"format":"{calc(net * 2, 2)}|{/cat.net.v}",` +
"cat": `{"format":"{calc(net * 2, 2)}|{/hop}",` +
`"net":{"format":"{v}","v":["10.00","20.00"]}}`,
"hop": `"{/cat.net.v}"`,
},
`{/cat.net.v} renders "net"`,
`{/hop} renders "net"`,
},
"a reference into the operand one level down": {
map[string]string{
"cat": `{"format":"{calc(net * 2, 2)}|{q}","net":{"format":"{v}","v":["10.00","20.00"]},` +
`"q":"{/cat.net.v}"}`,
`"q":"{/hop}"}`,
"hop": `"{/cat.net.v}"`,
},
`{q} renders "net"`,
},
@@ -203,7 +224,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"a violation on the second of two held heads": {
map[string]string{
"cat": `{"format":"{calc(a + b, 0)} {w}","a":{"format":"{x}","x":["1","2"]},` +
`"b":{"format":"{y}","y":["3","4"]},"w":"{/cat.b}"}`,
`"b":{"format":"{y}","y":["3","4"]},"w":"{/hop}"}`,
"hop": `"{/cat.b}"`,
},
`{w} renders "b"`,
},
@@ -230,7 +252,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"a reference to a sibling the operand never renders": {
"cat": `{"format":"{calc(net * 2, 2)} {unit}",` +
`"net":{"format":"{v}","v":["1","2"],"spare":["kg","lb"]},` +
`"unit":"{/cat.net.spare}"}`,
`"unit":"{/hop}"}`,
"hop": `"{/cat.net.spare}"`,
},
// Two operands drawing from one source are two names, so two draws: each is
// held under its own name and shown once, and neither can disagree.
@@ -247,7 +270,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
},
// A fixed string cannot disagree with itself, so it needs no fence.
"a literal operand named twice": {
"cat": `{"format":"{calc(n * 2, 0)} {/cat.n}","n":"5"}`,
"cat": `{"format":"{calc(n * 2, 0)} {/hop}","n":"5"}`,
"hop": `"{/cat.n}"`,
},
// One node reached twice while walking the operand: the walk must not
// revisit it, and the repeat is not a second route to anything.
@@ -267,19 +291,28 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) {
// all of them. Reaching only the first leaves whatever hides in a later
// variant to be found at render — a cycle there is fatal, and a second route
// to a held level disagrees silently.
rejected := map[string]struct{ file, want string }{
rejected := map[string]struct {
files map[string]string
want string
}{
"a cycle in a later variant": {
`{"format":"{p.x}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/cat}"}]}`,
map[string]string{
"cat": `{"format":"{p.x}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/hop}"}]}`,
"hop": `"{/cat}"`,
},
"reference cycle",
},
"a second route in a later variant": {
`{"format":"{p.x} {q.y}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/cat.q}"}],` +
map[string]string{
"cat": `{"format":"{p.x} {q.y}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/hop}"}],` +
`"q":{"format":"{y}","y":["1","2"]}}`,
"hop": `"{/cat.q}"`,
},
"reads a path into",
},
}
for name, c := range rejected {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": c.file})))
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, c.files)))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: New = %v, want it to mention %q", name, err, c.want)
}
@@ -289,15 +322,22 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) {
func TestALevelAPathNeverRendersIsAccepted(t *testing.T) {
// A path token does not expand its head's format, so a reference sitting in
// that format is not a second route to anything: it is never rendered by the
// path at all. Both orders must load.
accepted := map[string]string{
"reference in the head's own format": `{"format":"{p.first} {q.a}",` +
`"p":{"format":"{first} {/thing.q}","first":["A","B"]},"q":{"format":"{a}","a":["1","2"]}}`,
"the mirror shape": `{"format":"{p.first} {q.a}",` +
`"p":{"format":"{first}","first":["A","B"]},"q":{"format":"{a} {/thing.p}","a":["1","2"]}}`,
// path at all. Both orders must load. p and q sit one level down, so they are
// one column of thing rather than two that would read one another.
accepted := map[string]map[string]string{
"reference in the head's own format": {
"thing": `{"format":"{inner}","inner":{"format":"{p.first} {q.a}",` +
`"p":{"format":"{first} {/hop}","first":["A","B"]},"q":{"format":"{a}","a":["1","2"]}}}`,
"hop": `"{/thing.inner.q}"`,
},
"the mirror shape": {
"thing": `{"format":"{inner}","inner":{"format":"{p.first} {q.a}",` +
`"p":{"format":"{first}","first":["A","B"]},"q":{"format":"{a} {/hop}","a":["1","2"]}}}`,
"hop": `"{/thing.inner.p}"`,
},
}
for name, file := range accepted {
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"thing": file}))); err != nil {
for name, files := range accepted {
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files))); err != nil {
t.Errorf("%s: New = %v, want it accepted", name, err)
}
}
@@ -404,6 +444,13 @@ func TestPathIntoARepeatingLevelIsRejected(t *testing.T) {
if err == nil || !strings.Contains(err.Error(), "repeat") {
t.Fatalf("New = %v, want a path into a repeating level rejected", err)
}
_, err = New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"cat": `{"format":"[{p.a}]","p":{"format":"{a}","a":"{/word.w}","drawGroup":"g"}}`,
"word": `{"format":"{w}","w":["x","y"]}`,
})))
if err == nil || !strings.Contains(err.Error(), `the level "p" carries a drawGroup`) {
t.Fatalf("New = %v, want a path into a level carrying a group rejected", err)
}
}
func TestPathIntoAPlainTemplateNamesTheMissingField(t *testing.T) {
@@ -666,7 +713,8 @@ func TestCycleThroughAPathTokenIsRejected(t *testing.T) {
// must be caught at New. Reaching render would be fatal: the recursion never
// terminates, and a stack overflow cannot be recovered.
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"a": `{"format":"{p.x}","p":{"format":"{x}","x":"{/a}"}}`,
"a": `{"format":"{p.x}","p":{"format":"{x}","x":"{/b}"}}`,
"b": `"{/a}"`,
})))
if err == nil || !strings.Contains(err.Error(), "reference cycle") {
t.Fatalf("New = %v, want the cycle through {p.x} rejected", err)
@@ -718,6 +766,6 @@ func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) {
t.Fatal("not a template")
}
mustPanic(t, "unheld arm with a path", func() {
readField(engine(1).rand, tm, nil, nil, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"})
readField(engine(1).rand, tm, nil, drawScope{}, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"})
})
}
+6 -2
View File
@@ -20,7 +20,7 @@ type Template struct {
func (t *Template) Fake() string {
t.g.mu.Lock()
defer t.g.mu.Unlock()
return render(t.g.rand, t.n, nil)
return renderOnce(t.g.rand, t.n)
}
// NewTemplate compiles an inline template — a format string or a JSON value — and
@@ -136,6 +136,9 @@ func inputValue(input string) (any, error) {
// bindInline links an inline node's references against root and runs check over it, naming its
// nodes from label.
func bindInline(n node, label string, root map[string]node, check func(nodeScope) error) error {
if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" {
return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup)
}
scope := inlineScope(n, label)
if err := linkNodeRefs(scope, root); err != nil {
return err
@@ -151,6 +154,7 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error {
if !ok {
return nil
}
t.keyDrawGroup("")
for _, name := range refTokens(t.format) {
sigil, rest, err := refShape(name)
if err != nil {
@@ -160,6 +164,6 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error {
return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest)
}
}
return linkTemplateRefs(nil, path, t, root)
return linkTemplateRefs(nil, path, "", t, root)
})
}
+2 -1
View File
@@ -108,8 +108,9 @@ func TestFakeTemplateErrors(t *testing.T) {
{`name: {/no.such.path}`, "no entry"},
{`name: {..nope}`, "write {/nope}"},
{`{"format":"x"}`, "is a string"},
{`{/misc.country} {/misc.country.alpha2}`, "renders a level"},
{`{/misc.country} {/misc.country.alpha2}`, "renders a level that {/misc.country.alpha2} reads a path into; name the fields you want instead, or draw them apart with a drawGroup"},
{`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"},
{`{"format":"{/misc.country.alpha2}","drawGroup":"g"}`, "nothing can reference"},
} {
_, err := f.FakeTemplate(c.input)
if err == nil || !strings.Contains(err.Error(), c.want) {
+15 -6
View File
@@ -13,12 +13,12 @@ import (
// re-sums weights.
type node interface{ isNode() }
// group is a namespace of named children, built from a directory of JSON files
// folder is a namespace of named children, built from a directory of JSON files
// and subdirectories. It has no value of its own: descend into a named child by
// dot path; rendering one is an error (see Fake).
type group struct{ children map[string]node }
type folder struct{ children map[string]node }
func (*group) isNode() {}
func (*folder) isNode() {}
// choice picks one of its items. cum holds cumulative weights for a weighted
// pick; when nil the choice is uniform and selection is O(1). shared is the set of
@@ -62,6 +62,8 @@ type template struct {
fromString bool // written as a JSON string rather than an object
readsColumn *columnRead // set when the format is one reference alone reading a record's column
record bool // compiled at the top without a repeat, so its fields are record columns
drawGroup string // the draw group it draws in, as written; "" keeps its caller's
drawGroupKey string // its draw group keyed by its category once linked: what a render reads its reference paths under
}
func (*template) isNode() {}
@@ -241,13 +243,16 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
if err != nil {
return nil, err
}
if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString {
if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" {
return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format)
}
if err := checkTokens(o.format, fields); err != nil {
return nil, err
}
t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, record: fieldPos == inColumn}
if err := checkNestedDrawGroup(fields, o.group); err != nil {
return nil, err
}
t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn}
if err := t.compileFormat(); err != nil {
return nil, err
}
@@ -258,6 +263,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
type templateOptions struct {
datatype DataType
format string
group string
repeat int
separator string
weighted bool
@@ -278,6 +284,9 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) {
if o.datatype, err = datatypeOf(m, pos); err != nil {
return o, err
}
if o.group, err = drawGroupOf(m, repeat); err != nil {
return o, err
}
if sv, ok := m["separator"]; ok {
if o.separator, ok = sv.(string); !ok {
return o, fmt.Errorf("separator must be a string, got %T", sv)
@@ -410,7 +419,7 @@ func checkPathNames(path string) error {
// field. These names can never be fields.
func isOption(name string) bool {
switch name {
case "datatype", "format", "repeat", "separator", "weight":
case "datatype", "drawGroup", "format", "repeat", "separator", "weight":
return true
}
return false
+28 -1
View File
@@ -38,18 +38,45 @@ func TestInertObjectIsRejected(t *testing.T) {
`{"format":"{x}","x":"v","separator":","}`: "separator",
`{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default",
`{"format":"","n":{"format":"1","datatype":"string"}}`: `datatype "string" is the default`,
`{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":""}}`: `drawGroup "" is the default`,
`{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":1}}`: "drawGroup must be a string",
} {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want)
}
}
for _, ok := range []string{`[{"format":"a","weight":2},"b"]`, `{"format":"ab","repeat":2}`, `{"format":"{x}","x":"v"}`, `"{digits(3)}"`} {
for _, ok := range []string{`[{"format":"a","weight":2},"b"]`, `{"format":"ab","repeat":2}`, `{"format":"{x}","x":"v"}`, `"{digits(3)}"`, `{"format":"{/cat.x}","drawGroup":"g"}`} {
if _, err := compile(parse(t, ok)); err != nil {
t.Errorf("compile(%s) = %v", ok, err)
}
}
}
func TestAGroupThatSplitsNothingIsRejected(t *testing.T) {
files := map[string]string{"mail": `"{/word.w}@example.com"`, "word": `{"format":"{w}","w":["a","b"]}`}
for src, want := range map[string]string{
`{"format":"{x}","x":{"format":"{y}","y":["a","b"],"drawGroup":"g"}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{/word}","drawGroup":"g"}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{r}","drawGroup":"g","r":{"format":"{/word.w}","repeat":2}}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{y}","drawGroup":"g","y":{"format":"{/word.w}","drawGroup":"h"}}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{y}","y":"{/word.w}","repeat":2,"drawGroup":"g"}}`: `drawGroup "g" on a repeat`,
`{"format":"{x}","x":{"format":"{/word.w} {y}","drawGroup":"g","y":{"format":"{/word.w}!","drawGroup":"g"}}}`: `"y" names drawGroup "g", the draw group this template draws in already`,
`{"format":"{/word.w}","drawGroup":"g"}`: "",
`{"format":"{x}","x":{"format":"{/mail}","drawGroup":"g"}}`: "",
`{"format":"{x}","x":{"format":"{/word.w} {y}","drawGroup":"g","y":{"format":"{/word.w}!","drawGroup":"h"}}}`: "",
`{"format":"{x}","x":{"format":"{/word.w} {r}","drawGroup":"g","r":{"format":"{y}","repeat":2,"y":{"format":"{/word.w}","drawGroup":"g"}}}}`: "",
} {
files["cat"] = src
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, files)))
switch {
case want == "" && err != nil:
t.Errorf("New(%s) = %v, want a group over a reference path accepted, however deep", src, err)
case want != "" && (err == nil || !strings.Contains(err.Error(), want)):
t.Errorf("New(%s) = %v, want an error mentioning %s", src, err, want)
}
}
}
func TestInlineFolderSigilsAreRejected(t *testing.T) {
f := shipped(t)
for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} {
+11 -7
View File
@@ -16,7 +16,7 @@ type pathWalk struct {
leaf func(n node) error
}
// walkPath descends tail from n: a group or template by its next segment, a
// walkPath descends tail from n: a folder or template by its next segment, a
// choice by w.choice, which consumes no segment. A missing segment is an error,
// so no walk reaches past what the data holds. A table-shaped dispatch, one case
// per node kind, kept whole on purpose.
@@ -28,7 +28,7 @@ func walkPath(n node, tail []string, w pathWalk) error {
return nil
}
switch n := n.(type) {
case *group:
case *folder:
child, ok := n.children[tail[0]]
if !ok {
return fmt.Errorf("no entry %q", tail[0])
@@ -89,9 +89,9 @@ func unreachableInChoice(c *choice, want string) error {
// checkPath proves a dotted tail resolves whichever way the draws go — a choice
// must carry the rest of the path in the set every variant shares — and that no
// level a path reads carries a repeat, which one draw of it could not apply. So a
// path that validates here resolves on every render, and a typo is a New-time
// error.
// level a path reads carries a repeat or a drawGroup, which one draw of it could not
// apply. So a path that validates here resolves on every render, and a typo is a
// New-time error.
func checkPath(n node, tail []string, level string) error {
return walkPath(n, tail, pathWalk{
choice: func(c *choice, rest []string) ([]node, error) {
@@ -101,8 +101,12 @@ func checkPath(n node, tail []string, level string) error {
return c.items, nil
},
level: func(t *template, rest []string) error {
if t.repeat > 1 {
return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", join(level, strings.Join(tail[:len(tail)-len(rest)], ".")))
name := join(level, strings.Join(tail[:len(tail)-len(rest)], "."))
switch {
case t.repeat > 1:
return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", name)
case t.drawGroup != "":
return fmt.Errorf("the level %q carries a drawGroup, which a path reading into it cannot apply", name)
}
return nil
},
+23
View File
@@ -51,6 +51,29 @@ func TestNoRenderAllocRegression(t *testing.T) {
}
}
// The repeat shape prices both escape measures: dropping either costs an alloc an iteration.
func TestNoReferenceAllocRegression(t *testing.T) {
word := `{"format":"{w}","w":["alpha","beta","gamma","delta"]}`
for _, s := range []struct {
name, json string
base float64
}{
{"a repeat of a reference path", `{"format":"{r}","r":{"format":"{/word.w}","repeat":20,"separator":", "}}`, 66},
{"a named draw group", `{"format":"{a}","a":{"format":"{/word.w}","drawGroup":"g"}}`, 11},
} {
f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{
"word.json": {Data: []byte(word)},
"x.json": {Data: []byte(s.json)},
}))
if err != nil {
t.Fatalf("New(%s): %v", s.name, err)
}
if allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") }); allocs > s.base*1.10 {
t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a draw set reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base)
}
}
}
// A record's fences read the compiled tree, so they belong to New, not to a draw.
func TestNoRecordAllocRegression(t *testing.T) {
for _, s := range []struct{ name, json string }{
+5 -76
View File
@@ -4,7 +4,6 @@ import (
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"unicode"
"unicode/utf8"
@@ -23,7 +22,7 @@ type Column struct {
// Record is one record rendered from a template: every direct field is a column,
// listed in name order. Each column is its own expansion, so a sibling field is
// local to it, while a reference that reads a path is drawn once for the whole
// record.
// record, per group.
type Record struct {
columns []Column
}
@@ -212,9 +211,6 @@ func recordOf(n node) (*template, []Column, error) {
if !t.record {
return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat)
}
if err := checkColumnRefs(t, names); err != nil {
return nil, nil, err
}
columns := make([]Column, len(names))
for i, name := range names {
datatype, _ := columnDatatype(t.fields[name]) // checkColumns refused items that disagree wherever DataType is read
@@ -223,80 +219,13 @@ func recordOf(n node) (*template, []Column, error) {
return t, columns, nil
}
// checkColumnRefs rejects the reference reads a record's shared draw cannot answer
// for: one column rendering a level another reads a path into, and a column
// reading the record back through its own path.
func checkColumnRefs(t *template, columns []string) error {
reads, err := columnRefs(t, columns)
if err != nil {
return err
}
sort.Slice(reads, func(i, j int) bool {
if reads[i].a.path != reads[j].a.path {
return reads[i].a.path < reads[j].a.path
}
return reads[i].column < reads[j].column
})
for i, level := range reads {
for _, into := range reads[i+1:] {
if strings.HasPrefix(into.a.path, level.a.path+".") {
return fmt.Errorf("column %q renders {%s}, a level column %q reads a path into with {%s}; name the fields you want instead",
level.column, level.a.name, into.column, into.a.name)
}
}
}
return nil
}
// columnRef is one held reference read, and the column whose render reaches it.
type columnRef struct {
column string
a arm
}
// columnRefs lists every reference that reads a path, anywhere a column renders,
// following the same edges expand does. A reference landing back on the record
// itself is reported rather than collected, whether it reads a path or the record
// whole: either way the column describes a draw other than its neighbours'.
func columnRefs(t *template, columns []string) ([]columnRef, error) {
var out []columnRef
var err error
for _, name := range columns {
seen := map[node]bool{}
var walk func(n node)
walk = func(n node) {
if n == nil || seen[n] || err != nil {
return
}
seen[n] = true
if tm, ok := n.(*template); ok {
for _, ref := range refTokens(tm.format) {
a := splitArm(ref, tm.refs)
if tm.fields[a.key] == node(t) {
err = fmt.Errorf("column %q reads {%s}, which points back at this record; a column cannot read another column — move the shared value into its own category and reference that", name, a.name)
return
}
if len(a.tail) > 0 {
out = append(out, columnRef{name, a})
}
}
}
for _, e := range renderEdges(n) {
walk(e.to)
}
}
walk(t.fields[name])
}
return out, err
}
// renderRecord draws each column once, in the name order recordOf fixed, over one
// reference scope shared across them.
// renderRecord draws each column once, in the name order recordOf fixed, as one render.
func renderRecord(s *session, t *template, columns []Column) *Record {
scope := &draws{variant: map[string]node{}, value: map[string]draw{}}
set := newDrawSet()
sc := drawScope{set: &set}.in(t)
r := &Record{columns: append([]Column(nil), columns...)}
for i := range r.columns {
column := renderLeaf(s, t.fields[r.columns[i].Name], scope)
column := renderLeaf(s, t.fields[r.columns[i].Name], sc)
r.columns[i].Value, r.columns[i].Null = column.text, column.null
}
return r
+68 -40
View File
@@ -200,26 +200,33 @@ func TestRecordWritesTypedAndNullColumns(t *testing.T) {
}
func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) {
cat := `{"format":"","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}`
cat := `{"format":"{a}","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}`
mid := `"{/cat.a}"`
for _, c := range []struct{ name, row string }{
{"through a column repeat, which draws anew", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`},
{"in a group of its own", `{"format":"","whole":{"format":"{/cat.a}","drawGroup":"g"},"inner":"{/cat.a.b}"}`},
} {
f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}), WithSeed(1))
if _, err := f.FakeRecord("row"); err != nil {
t.Errorf("%s: FakeRecord = %v, want it accepted", c.name, err)
}
}
for _, c := range []struct{ name, row string }{
{"sibling columns", `{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`},
{"through a nested template", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b} {x}","x":"1"}}`},
{"through a column repeat", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`},
{"through a choice variant", `{"format":"","whole":"{/cat.a}","inner":[{"format":"{/cat.a.b} {x}","x":"1"},{"format":"{/cat.a.b}! {x}","x":"2"}]}`},
{"as a builtin operand", `{"format":"","whole":"{uppercase(/cat.a)}","inner":"{/cat.a.b}"}`},
{"a bare reference beside a path", `{"format":"","whole":"{/cat}","inner":"{/cat.a.b}"}`},
{"a column reaching back through another category", `{"format":"","whole":"{/mid}","inner":"{/cat.a.b}"}`},
} {
f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "row": c.row}), WithSeed(1))
_, err := f.FakeRecord("row")
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row})))
if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: FakeRecord = %v, want the overlap rejected the way one format is", c.name, err)
t.Errorf("%s: New = %v, want the overlap refused at load, the way one format is", c.name, err)
continue
}
if !strings.Contains(err.Error(), `"whole"`) || !strings.Contains(err.Error(), `"inner"`) {
t.Errorf("%s: error %q names neither column; it must name both", c.name, err)
}
if _, err := f.Fake("row"); err != nil {
t.Errorf("%s: Fake(row) = %v, want the string view untouched", c.name, err)
}
}
}
@@ -241,9 +248,9 @@ func TestRecordRejectsAColumnReadingItsOwnRecord(t *testing.T) {
{"the record as an operand", `"up":"{uppercase(/person)}"`},
} {
person := `{"format":"{first} {last}","first":["Ada","Bo"],"last":["Lovelace","Ek"],` + c.column + `}`
f := newGenerator(t, writeData(t, map[string]string{"person": person}), WithSeed(1))
if _, err := f.FakeRecord("person"); err == nil || !strings.Contains(err.Error(), "points back at this record") {
t.Errorf("%s: FakeRecord = %v, want it refused; the column would contradict the columns beside it", c.name, err)
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"person": person})))
if err == nil || !strings.Contains(err.Error(), "names the category it sits in") {
t.Errorf("%s: New = %v, want it refused at load; the column would contradict the columns beside it", c.name, err)
}
}
}
@@ -311,9 +318,10 @@ func TestRecordRejectsFieldDescent(t *testing.T) {
func TestRecordSharesAReferenceAcrossColumns(t *testing.T) {
dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD","symbol":"$"},{"format":"{code}","code":"EUR","symbol":"€"}]`,
"price": `{"format":"","code":"{/currency.code}","symbol":"{/currency.symbol}"}`,
"price": `{"format":"{code} {symbol}","code":"{/currency.code}","symbol":"{/currency.symbol}"}`,
})
f := newGenerator(t, dir, WithSeed(1))
symbols := map[string]string{"AUD": "$", "EUR": "€"}
for i := 0; i < 100; i++ {
r, err := f.FakeRecord("price")
if err != nil {
@@ -323,48 +331,63 @@ func TestRecordSharesAReferenceAcrossColumns(t *testing.T) {
for _, c := range r.Columns() {
m[c.Name] = c.Value
}
switch m["code"] {
case "AUD":
if m["symbol"] != "$" {
t.Fatalf("record %q: code AUD but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"])
if symbol, known := symbols[m["code"]]; !known || m["symbol"] != symbol {
t.Fatalf("record %s, want one currency draw across columns", r.JSON())
}
case "EUR":
if m["symbol"] != "€" {
t.Fatalf("record %q: code EUR but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"])
}
default:
t.Fatalf("record %q has unexpected code %q", r.JSON(), m["code"])
v := fake(t, f, "price")
if code, symbol, _ := strings.Cut(v, " "); symbol == "" || symbols[code] != symbol {
t.Fatalf("Fake(price) = %q, want its fields one currency draw, as the record's columns are", v)
}
}
}
func TestRecordSharesAReferenceIntoAColumnRepeat(t *testing.T) {
func TestRepeatIterationsDrawReferencesAnew(t *testing.T) {
dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`,
"order": `{"format":"","codes":{"format":"{/currency.code}","repeat":3,"separator":"-"}}`,
"party": `{"format":"{host}: {guests}","guests":{"format":"{/person.first} {/person.last}","repeat":3,"separator":", "},"host":"{/person.first} {/person.last}"}`,
"person": drawPeople,
})
f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 50; i++ {
r, err := f.FakeRecord("order")
differed := map[[2]int]bool{}
check := func(view string, names []string) {
t.Helper()
if len(names) != 4 {
t.Fatalf("%s rendered %q, want a host and three guests", view, names)
}
for i, name := range names {
if !onePerson(name) {
t.Fatalf("%s rendered %q, want each name one person", view, names)
}
for j := range names[:i] {
differed[[2]int{j, i}] = differed[[2]int{j, i}] || names[j] != name
}
}
}
for i := 0; i < 100; i++ {
r, err := f.FakeRecord("party")
if err != nil {
t.Fatal(err)
}
parts := strings.Split(r.Columns()[0].Value, "-")
if len(parts) != 3 || parts[0] != parts[1] || parts[1] != parts[2] {
t.Fatalf("codes column = %q, want one shared draw across its repeat", r.Columns()[0].Value)
guests, host := r.Columns()[0].Value, r.Columns()[1].Value
check("FakeRecord", append([]string{host}, strings.Split(guests, ", ")...))
host, guests, _ = strings.Cut(fake(t, f, "party"), ": ")
check("Fake", append([]string{host}, strings.Split(guests, ", ")...))
}
for pair, ok := range differed {
if !ok {
t.Errorf("names %v never differed in 200 renders; the host and each repeat iteration are a draw of their own, so four people", pair)
}
}
}
func TestRecordBareReferenceStaysIndependent(t *testing.T) {
func TestRecordGroupsDrawApart(t *testing.T) {
dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`,
"order": `{"format":"","whole":"{/currency}","code":"{/currency.code}"}`,
"person": drawPeople,
"transfer": `{"format":"{from_first} {from_last} to {to_first} {to_last}","from_first":{"format":"{/person.first}","drawGroup":"from"},"from_last":{"format":"{/person.last}","drawGroup":"from"},"to_first":{"format":"{/person.first}","drawGroup":"to"},"to_last":{"format":"{/person.last}","drawGroup":"to"}}`,
})
f := newGenerator(t, dir, WithSeed(1))
sawMismatch := false
apart := map[string]bool{}
for i := 0; i < 100; i++ {
r, err := f.FakeRecord("order")
r, err := f.FakeRecord("transfer")
if err != nil {
t.Fatal(err)
}
@@ -372,13 +395,18 @@ func TestRecordBareReferenceStaysIndependent(t *testing.T) {
for _, c := range r.Columns() {
m[c.Name] = c.Value
}
if m["whole"] != m["code"] {
sawMismatch = true
break
from, to, _ := strings.Cut(fake(t, f, "transfer"), " to ")
for view, pair := range map[string][2]string{"FakeRecord": {m["from_first"] + " " + m["from_last"], m["to_first"] + " " + m["to_last"]}, "Fake": {from, to}} {
if !onePerson(pair[0]) || !onePerson(pair[1]) {
t.Fatalf("%s drew %q and %q, want each group one person", view, pair[0], pair[1])
}
apart[view] = apart[view] || pair[0] != pair[1]
}
}
if !sawMismatch {
t.Fatal("a bare {/currency} column never disagreed with a tailed {/currency.code} column; a bare reference should draw independently")
for _, view := range []string{"FakeRecord", "Fake"} {
if !apart[view] {
t.Errorf("%s: groups from and to drew one person in 100 renders, want a draw each", view)
}
}
}
+21 -15
View File
@@ -72,14 +72,17 @@ func refSegments(name string, folder []string) ([]string, error) {
// error, never a random render-time one.
func linkRefs(root map[string]node) error {
return eachTemplate(root, func(folder []string, path string, t *template) error {
return linkTemplateRefs(folder, path, t, root)
category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".")
t.keyDrawGroup(category)
return linkTemplateRefs(folder, path, category, t, root)
})
}
// linkTemplateRefs binds one template's references against root. A template with
// none is left untouched, so an inline format that references nothing costs only
// the refTokens scan.
func linkTemplateRefs(folder []string, path string, t *template, root map[string]node) error {
// linkTemplateRefs binds one template's references against root, refusing one that names the
// category it sits in: a category is a unit, and a reference back into it describes a draw other
// than the fields beside it. A template with no reference is left untouched, so an inline format
// that references nothing costs only the refTokens scan.
func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error {
names := refTokens(t.format)
if len(names) == 0 {
return nil
@@ -98,6 +101,9 @@ func linkTemplateRefs(folder []string, path string, t *template, root map[string
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
key := "/" + strings.Join(head, ".")
if category != "" && key == "/"+category {
return fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name)
}
if err := checkPath(target, tail, key); err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
}
@@ -162,17 +168,17 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t
}
return nil
}
var inFolder func(folder []string, children map[string]node) error
inFolder = func(folder []string, children map[string]node) error {
var inFolder func(dir []string, children map[string]node) error
inFolder = func(dir []string, children map[string]node) error {
for _, name := range sortedNames(children) {
path := join(strings.Join(folder, "."), name)
if g, ok := children[name].(*group); ok {
if err := inFolder(append(folder[:len(folder):len(folder)], name), g.children); err != nil {
path := join(strings.Join(dir, "."), name)
if g, ok := children[name].(*folder); ok {
if err := inFolder(append(dir[:len(dir):len(dir)], name), g.children); err != nil {
return err
}
continue
}
if err := inCategory(folder, path, children[name]); err != nil {
if err := inCategory(dir, path, children[name]); err != nil {
return err
}
}
@@ -183,13 +189,13 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t
// resolveCategory walks a dotted path through the folders to the category it
// names, returning that head, the node, and the tail left to read into it. A
// descent of its own rather than a walkPath: it walks groups only and returns
// descent of its own rather than a walkPath: it walks folders only and returns
// where they end, not a leaf.
func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) {
var n node = &group{children: root}
var n node = &folder{children: root}
i := 0
for ; i < len(segments); i++ {
g, ok := n.(*group)
g, ok := n.(*folder)
if !ok {
break
}
@@ -199,7 +205,7 @@ func resolveCategory(root map[string]node, segments []string) (head []string, ta
}
n = child
}
if _, ok := n.(*group); ok {
if _, ok := n.(*folder); ok {
return nil, nil, nil, fmt.Errorf("names a folder, not a value")
}
return segments[:i], n, segments[i:], nil
+125 -31
View File
@@ -83,25 +83,23 @@ func TestReferenceErrors(t *testing.T) {
"card": `"{/who.f}"`,
},
"empty reference path": {"card": `"{/}"`},
// A reference that leads back to its own value never terminates at render,
// so New must reject the cycle up front (direct, mutual, or chained).
"direct cycle": {"a": `"x{/a}"`},
// A reference that leads back to its own value never terminates at render, so
// New must reject the cycle up front (mutual or chained). One into its own
// category is refused before the cycle walk reaches it, as a unit rule.
"a category referencing itself": {"a": `"x{/a}"`},
"mutual cycle": {"a": `"{/b}"`, "b": `"{/a}"`},
"chain cycle": {"a": `"{/b}"`, "b": `"{/c}"`, "c": `"{/a}"`},
// calc renders its operands, so a cycle through one must be caught too.
"calc operand cycle": {"x": `{"format":"{calc(y)}","y":"{/x}"}`},
// calc renders its operands, so a reference through one is caught too.
"a calc operand into its own category": {"x": `{"format":"{calc(y)}","y":"{/x}"}`},
// A field its parent's format never renders is still reachable by dot path,
// so a cycle hiding in one must fail at New rather than at render.
"cycle in an unrendered field": {"cat": `{"format":"hi","x":"{/cat.x}"}`},
"mutual cycle between unrendered fields": {
// so what hides in one must fail at New rather than at render.
"an unrendered field into its own category": {"cat": `{"format":"hi","x":"{/cat.x}"}`},
"two unrendered fields into their own category": {
"cat": `{"format":"hi","x":"{/cat.y}","y":"{/cat.x}"}`,
},
"cycle in an unrendered field of a choice arm": {
"cat": `{"format":"hi","x":"{/cat.x}"}`,
},
// The shipped layout puts categories in folders, so a cycle one level down
// is the common case, not an edge case.
"cycle in a subfolder": {"sv_SE/a": `"x{/sv_SE.a}"`},
// The shipped layout puts categories in folders, so one level down is the
// common case, not an edge case.
"a category in a subfolder referencing itself": {"sv_SE/a": `"x{/sv_SE.a}"`},
"mutual cycle within a subfolder": {"sv_SE/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/sv_SE.a}"`},
"mutual cycle across two folders": {"en_US/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/en_US.a}"`},
// ".." is reserved for bound references, so an authored key using it would
@@ -131,17 +129,26 @@ func TestDotPrefixedDataEntriesAreSkipped(t *testing.T) {
}
}
// TestReferenceFromUnrenderedFieldTerminates guards the cycle check against
// over-rejecting: a field the format never renders may point back at its own
// category, which terminates, and stays renderable by path.
func TestReferenceFromUnrenderedFieldTerminates(t *testing.T) {
dir := writeData(t, map[string]string{"cat": `{"format":"hi","x":"see {/cat}"}`})
f := newGenerator(t, dir, WithSeed(1))
if got := fake(t, f, "cat"); got != "hi" {
t.Fatalf("cat = %q, want hi", got)
// TestReferenceIntoItsOwnCategoryIsRejected pins the unit a category is: a reference
// back into it describes a draw other than the fields beside it, whether the format
// renders that field or not, so it is refused at load rather than left to disagree in
// the record view.
func TestReferenceIntoItsOwnCategoryIsRejected(t *testing.T) {
for name, file := range map[string]string{
"the category whole": `{"format":"hi","x":"see {/cat}"}`,
"a field of its own": `{"format":"hi","x":"{/cat.y}","y":"1"}`,
"a path the format reads": `{"format":"{/cat.y}","y":"1"}`,
} {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file})))
if err == nil || !strings.Contains(err.Error(), "names the category it sits in") {
t.Errorf("%s: New = %v, want the reference into its own category refused", name, err)
}
if got := fake(t, f, "cat.x"); got != "see hi" {
t.Fatalf("cat.x = %q, want \"see hi\"", got)
}
// Reading a sibling as a path is the spelling that stays.
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"cat": `{"format":"{y.v}","y":{"format":"{v}","v":["1","2"]}}`,
}))); err != nil {
t.Errorf("New = %v, want the sibling path accepted", err)
}
}
@@ -189,9 +196,9 @@ func TestNewErrorPathIsCanonical(t *testing.T) {
want string
}{
{
"cycle inside a choice arm",
map[string]string{"cat": `{"format":"hi","x":"{/cat.x}"}`},
"fejkdata: reference cycle: cat.x -> /cat.x",
"cycle through another category",
map[string]string{"cat": `{"format":"hi","x":"{/hop}"}`, "hop": `"{/cat.x}"`},
"fejkdata: reference cycle: cat.x -> /hop -> /cat.x",
},
{
"bad reference reached through another reference",
@@ -255,15 +262,102 @@ func TestBareReferenceDrawsEachTime(t *testing.T) {
}
func TestReferenceOverlapIsRejected(t *testing.T) {
for name, file := range map[string]string{
"head beside a path": `{"format":"{/cat.p} {/cat.p.first}","p":[{"format":"{first}","first":"A"},{"format":"{first}","first":"B"}]}`,
"sibling path beside a reference path": `{"format":"{p.first} {/cat.p.last}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`,
// A category never references itself, so the reads sit in a second category.
cat := `{"format":"x","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`
for name, row := range map[string]string{
"head beside a path": `"{/cat.p} {/cat.p.first}"`,
"sibling fields reading a level and a path into it": `{"format":"{a} {b}","a":"{/cat.p}","b":"{/cat.p.first}"}`,
"a field rendering the level a nested reference reads into": `{"format":"{x} {p}","x":"{/cat.p.first}","p":"{/cat.p}"}`,
"a bare reference beside a path into what it never renders": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`,
} {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file})))
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row})))
if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: New = %v, want the overlap rejected", name, err)
}
}
// A sibling path and a reference into the level it holds are the same overlap,
// and the reference reaches it from a category row renders.
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"row": `{"format":"{p.first} {/hop}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`,
"hop": `"{/row.p.last}"`,
}))); err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("New = %v, want a sibling path beside a reference into it rejected", err)
}
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"cat": cat,
"row": `{"format":"{a} {b}","a":{"format":"{/cat.p}","drawGroup":"g"},"b":"{/cat.p.first}"}`,
}))); err != nil {
t.Errorf("New = %v, want a level and a path into it accepted in groups of their own", err)
}
}
const drawPeople = `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"},{"format":"{first} {last}","first":"Cy","last":"Young"}]`
// onePerson reports whether name is the first name and surname of one drawPeople row.
func onePerson(name string) bool {
first, last, _ := strings.Cut(name, " ")
return last != "" && map[string]string{"Ada": "Lovelace", "Bo": "Ek", "Cy": "Young"}[first] == last
}
func TestAReferencePathIsOneDrawPerRender(t *testing.T) {
dir := writeData(t, map[string]string{
"apart": `{"format":"{a} & {b}","a":{"format":"{/person.first} {/person.last}","drawGroup":"x"},"b":{"format":"{/person.first} {/person.last}","drawGroup":"y"}}`,
"caller": `{"format":"{a} & {b}","a":{"format":"{/person.first} {/person.last}","drawGroup":"x"},"b":"{/pay}"}`,
"contact": `{"format":"{first} {last} <{email}>","email":"{lowercase(/person.first)}.{lowercase(/person.last)}@example.com","first":"{/person.first}","last":"{/person.last}"}`,
"iterations": `{"format":"{/person.first} {r}","drawGroup":"outer","r":{"format":"{a}={b}","repeat":3,"separator":",","a":"{/person.first}","b":{"format":"{/person.first}","drawGroup":"outer"}}}`,
"nested": `{"format":"{/person.first} {inner}","inner":"{/person.last}"}`,
"pair": `{"format":"{a} & {b}","a":"{/person.first} {/person.last}","b":"{/person.first} {/person.last}"}`,
"pay": `{"format":"{p}","p":{"format":"{/person.first} {/person.last}","drawGroup":"x"}}`,
"person": drawPeople,
})
f := newGenerator(t, dir, WithSeed(1))
apart, local, noGroup := false, false, false
for i := 0; i < 100; i++ {
a, b, _ := strings.Cut(fake(t, f, "caller"), " & ")
if !onePerson(a) || !onePerson(b) {
t.Fatalf("caller = %q & %q, want each one person", a, b)
}
local = local || a != b
_, iterations, _ := strings.Cut(fake(t, f, "iterations"), " ")
for _, pair := range strings.Split(iterations, ",") {
a, b, _ := strings.Cut(pair, "=")
noGroup = noGroup || a != b
}
if got := fake(t, f, "nested"); !onePerson(got) {
t.Fatalf("nested = %q, want a nested template's path one draw with its parent's", got)
}
name, email, _ := strings.Cut(fake(t, f, "contact"), " <")
first, last, _ := strings.Cut(name, " ")
if !onePerson(name) || email != strings.ToLower(first)+"."+strings.ToLower(last)+"@example.com>" {
t.Fatalf("contact = %q <%s, want first, last and email one person", name, email)
}
r, err := f.FakeRecord("contact")
if err != nil {
t.Fatal(err)
}
c := r.Columns()
if !onePerson(c[1].Value+" "+c[2].Value) || c[0].Value != strings.ToLower(c[1].Value)+"."+strings.ToLower(c[2].Value)+"@example.com" {
t.Fatalf("contact record %s, want first, last and email one person", r.JSON())
}
a, b, _ = strings.Cut(fake(t, f, "pair"), " & ")
if !onePerson(a) || a != b {
t.Fatalf("pair = %q & %q, want both fields one person", a, b)
}
a, b, _ = strings.Cut(fake(t, f, "apart"), " & ")
if !onePerson(a) || !onePerson(b) {
t.Fatalf("apart = %q & %q, want each group one person", a, b)
}
apart = apart || a != b
}
if !apart {
t.Error("groups x and y drew one person in 100 renders, want a draw each")
}
if !local {
t.Error("group x in caller and group x in the pay it references drew one person in 100 renders; a group name is local to its category")
}
if !noGroup {
t.Error("an iteration's plain read and its group outer always agreed; a repeat iteration renders in no group, so they are a draw each")
}
}
func TestRelativeReferences(t *testing.T) {
+23 -13
View File
@@ -20,14 +20,14 @@ type rng interface {
func (f *Generator) Fake(path string) (string, error) {
f.mu.Lock()
defer f.mu.Unlock()
n, err := descend(f.rand, &group{children: f.categories}, strings.Split(path, "."))
n, err := descend(f.rand, &folder{children: f.categories}, strings.Split(path, "."))
if err != nil {
return "", fmt.Errorf("fejkdata: %s: %w", path, err)
}
if _, ok := n.(*group); ok {
if _, ok := n.(*folder); ok {
return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path)
}
return render(f.rand, n, nil), nil
return renderOnce(f.rand, n), nil
}
// descend walks named fields to the node a path names. It is the one render-side
@@ -50,20 +50,21 @@ func descend(s *session, root node, segments []string) (node, error) {
}
// render evaluates a compiled node to a string. compile validates every node up
// front, so rendering a compiled tree cannot fail. refScope carries the draws a
// reference shares beyond its own expansion; nil keeps every reference local.
func render(s *session, n node, refScope *draws) string {
// front, so rendering a compiled tree cannot fail. sc holds the reference draws the
// render shares; each repeat iteration renders over draws of its own.
func render(s *session, n node, sc drawScope) string {
switch n := n.(type) {
case *choice:
return render(s, pick(s, n), refScope)
return render(s, pick(s, n), sc)
case *null:
return ""
case *template:
sc = sc.in(n)
if n.repeat == 1 {
if n.fixed {
return n.lit
}
return expand(s, n, refScope)
return expand(s, n, sc)
}
var b strings.Builder
b.Grow(n.repeat * (n.grow + len(n.separator)))
@@ -71,7 +72,7 @@ func render(s *session, n node, refScope *draws) string {
if i > 0 {
b.WriteString(n.separator)
}
b.WriteString(expand(s, n, refScope))
b.WriteString(expandAnew(s, n))
}
return b.String()
default:
@@ -79,6 +80,15 @@ func render(s *session, n node, refScope *draws) string {
}
}
// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into
// render's loop, its draw set would move to the heap.
//
//go:noinline
func expandAnew(s *session, t *template) string {
var set drawSet
return expand(s, t, drawScope{set: &set})
}
// pick selects one item. Uniform choices are O(1); weighted choices are an
// O(log n) search over precomputed cumulative weights. compile guarantees a
// non-empty choice and a finite positive total, so the index is always in range.
@@ -93,12 +103,12 @@ func pick(r rng, c *choice) node {
// expand renders a template's compiled ops. compile validated every token, so this
// cannot fail.
func expand(s *session, t *template, refScope *draws) string {
func expand(s *session, t *template, sc drawScope) string {
var b strings.Builder
b.Grow(t.grow)
// One draw per held name, for this expansion only: a nested template and each
// repeat iteration get their own, since each is its own expansion. A reference
// reads the caller's scope instead, whenever one was supplied.
// path reads the render's draws in sc instead.
var held *draws
if len(t.held) > 0 {
held = &draws{
@@ -112,7 +122,7 @@ func expand(s *session, t *template, refScope *draws) string {
case 'l':
b.WriteString(o.lit)
case 'f':
b.WriteString(readField(s, t, held, refScope, o.arms[s.IntN(len(o.arms))]).text)
b.WriteString(readField(s, t, held, sc, o.arms[s.IntN(len(o.arms))]).text)
case 'b':
// Read before the call, so the value a calc computes is the value the
// format showed. calcVars fixed the order op.operands holds.
@@ -120,7 +130,7 @@ func expand(s *session, t *template, refScope *draws) string {
if len(o.operands) > 0 {
operands = make([]string, len(o.operands))
for j, a := range o.operands {
operands[j] = readField(s, t, held, refScope, a).text
operands[j] = readField(s, t, held, sc, a).text
}
}
b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far
+2 -2
View File
@@ -38,7 +38,7 @@ func compiled(t *testing.T, s string) node {
func mustRender(t *testing.T, f *Generator, s string) string {
t.Helper()
return render(f.rand, compiled(t, s), nil)
return renderOnce(f.rand, compiled(t, s))
}
func TestStringIsAFormat(t *testing.T) {
@@ -315,7 +315,7 @@ func TestGrowIsALowerBound(t *testing.T) {
t.Fatalf("format %q did not compile to a template", format)
}
for i := 0; i < 50; i++ {
if got := len(expand(f.rand, tmpl, nil)); got < tmpl.grow {
if got := len(expand(f.rand, tmpl, drawScope{set: &drawSet{}})); got < tmpl.grow {
t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow)
}
}
-16
View File
@@ -2,22 +2,6 @@
## Before v0.1.0
The record API lands first, so the data update can use it.
### Record API
- Draw groups — references into one category share one draw per render (one
record, or one `Fake`) in both views; each `repeat` iteration draws anew, and a
bare reference draws each time. An option naming a draw group splits a render
into several entities. Replaces the Decision "A record shares one reference draw
per category". Each expectation becomes a test:
- `first`, `last` and `email` reading `person` → one person
- `from_first`/`from_last` grouped `from`, `to_first`/`to_last` grouped `to` → two people
- `host`, plus `guests` repeated 3 times → four people
- `code` and `symbol` sibling fields reading `currency`, as `{code} {symbol}` → a matching pair
- `{a} & {b}`, each reading `person` → one person, or two when `a` and `b` name different groups
- two bare `{/sv_SE.word}` → two words
### Data
- Major data update. Shipped categories render as records with their building