larvit/fejkdata
3
0
Fork 0
Code Pull Requests Actions Activity

Draw groups: one reference draw per render, a drawGroup option, and a record's fences at load #16

Merged
lilleman merged 14 commits from draw-groups into main 2026-09-16 10:45:45 +02:00
Conversation 0 Commits 14 Files Changed 20
+932 -357
20 changed files with 932 additions and 357 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+103 -48
View File
@@ -124,16 +124,17 @@ string and a backslash as an escape.
A record written only to emit columns still needs a `format` — the grammar's one A record written only to emit columns still needs a `format` — the grammar's one
required key — so `"format": ""` carries the fields with an inert format: it required key — so `"format": ""` carries the fields with an inert format: it
renders nothing by `Fake`, and is compiled only so the tree's fences still run. renders nothing by `Fake`, and is compiled only so the tree's fences still run.
The columns are the point, and their facts stay together: two columns that read a The columns are the point, and their facts stay together: a record is one render, so
path into one category — `{/currency.code}` and `{/currency.symbol}` — share one columns that read a path into one category — `{/currency.code}` and
draw of it, so the record is internally consistent. A bare `{/currency}` names no `{/currency.symbol}` — read one draw of it ([References](#references)), and a
field, so it keeps drawing on its own. That one draw is also why two [draw group](#draw-group) draws a column apart. That one draw is also why the columns of one
columns may not read overlapping reference *paths* — `{/cat.a}` beside record may not overlap — `{/cat.a}`, or a bare `{/cat}`, beside
`{/cat.a.b}` is refused, naming the fields to write instead, as `{/cat.a.b}` is refused, naming the fields to write instead, as
[One draw, one spelling](#one-draw-one-spelling) refuses that pair inside a single [One draw, one spelling](#one-draw-one-spelling) refuses that pair inside a single
format. A column may not reference the record it belongs to by any spelling: `{/users.first}` format. Both fences run at load, so a category that loads renders as either shape. A
or a bare `{/users}` inside `users` describes a draw other than the columns beside category never references itself — `{/users.first}` or a bare `{/users}` inside `users`
it, so put a value two columns share in its own category and reference that. A field hold, transform or describes a draw other than the fields beside it — so read a sibling as a path, and put
a value two fields share in its own category and reference that. A field hold, transform or
operand ties fields together within one column as always (see operand ties fields together within one column as always (see
[Correlated fields](#correlated-fields) and [Decisions](#decisions)). [Correlated fields](#correlated-fields) and [Decisions](#decisions)).
@@ -341,8 +342,8 @@ different datatypes.
### Options and fields ### Options and fields
`format`, `weight`, `repeat`, `separator` and `datatype` are the only options; **any `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options;
other key is a field** (see [Decisions](#decisions)). An object that does nothing a **any other key is a field** (see [Decisions](#decisions)). An object that does nothing a
string can't — only a `format` — is rejected naming the string, as is a one-item string can't — only a `format` — is rejected naming the string, as is a one-item
choice naming its item. choice naming its item.
@@ -438,14 +439,41 @@ without naming `sv_SE`:
"Hej, {/en_US.person}!" "Hej, {/en_US.person}!"
``` ```
Renders e.g. `Hej, Pat Smith!`. A reference into a category is held like a Renders e.g. `Hej, Pat Smith!`. A reference path into a category is held like a
[correlated](#correlated-fields) path — `{.person.femalefirst} {.person.last}` name [correlated](#correlated-fields) path, but for the whole render — one `Fake`, or one
one person, `{lowercase(.person.femalefirst)}` reads that same draw, and record — rather than one format: `{.person.femalefirst} {.person.last}` name one
`{.person.femalefirst}` beside `{/sv_SE.person.last}` in `sv_SE` is one person too — while a bare person, as do the same two references in sibling fields or a nested template, and
`{/misc.uuid} {/misc.uuid}` is two draws. Rejected at `New`: a path that is `{lowercase(.person.femalefirst)}` reads that same draw. Each `repeat` iteration is
unknown, names a folder, has no folder above, or reads a field not every variant a render of its own, in no group, so it draws anew, and a [draw group](#draw-group) holds a
of a choice carries, and a reference that leads back to its own value, directly, draw apart. A bare reference names no field and makes its own picks each time —
mutually or through a chain. `{/misc.uuid} {/misc.uuid}` is two draws — while the reference paths inside what it
renders still read the render's draws. Rejected at `New`: a path that is
unknown, names a folder, has no folder above, reads a field not every variant
of a choice carries, or names the category the reference sits in, and a reference
that leads back to its own value, directly, mutually or through a chain.
### Draw group
A template may carry `drawGroup` to hold its reference draws apart: every reference path
it renders, however deep short of a `repeat` or a nested `drawGroup`, reads the draw of
that group, and the templates of one category naming one group in one render read one
draw. A name is local to its category, so a category another one references never joins
its groups by name; the unnamed group spans them all.
```json
{ "format": "{payer} pays {payee}; signed {signature}",
"payer": { "format": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}", "drawGroup": "payer" },
"payee": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}",
"signature": { "format": "{/sv_SE.person.last}", "drawGroup": "payer" } }
```
Renders e.g. `Sara Eriksson pays Ebba Lind; signed Eriksson`: the signature reads the
payer's draw, while the payee is drawn apart. Rejected at load, each naming nothing: a
`drawGroup` of `""` (the default); one naming the draw group its template already draws
in; one on a template that renders no reference path short of a `repeat` or a nested
`drawGroup`, on a `repeat` itself — each iteration renders in no draw group — or on an
inline template's root, which nothing references. So is a path reading into a level that
carries a `drawGroup`.
### Correlated fields ### Correlated fields
@@ -478,12 +506,14 @@ The sub-fields stay addressable — `Fake("address.place.locality")` renders, an
### One draw, one spelling ### One draw, one spelling
A name any token reads as a path (`{p.first}`) or as an operand (`{calc(net * 2)}`, A name any token reads as a path (`{p.first}`) or as an operand (`{calc(net * 2)}`,
`{uppercase(w)}`) is drawn **once per expansion**, and every other route to it — `{uppercase(w)}`) is drawn **once per expansion**, a reference path (`{/cat.p.first}`)
a bare `{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering **once per render** in its [draw group](#draw-group), and every other route to either — a bare
`{/cat.p.last}`, at any depth — is a load error naming the spelling to use. A `{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering `{/cat.p.last}`
name nothing reads that way is drawn each time: `{word} {word}` differs. An beside `{p.first}`, a bare `{/cat}` beside `{/cat.p.first}`, at any
expansion is one render of one format, so each `repeat` iteration and each nested depth — is a load error naming the spelling to use. A name nothing reads that way is
template draws again. drawn each time: `{word} {word}` differs. An expansion is one render of one format, so
each nested template draws its own names again; a render is one `Fake` or one record,
and each `repeat` iteration is an expansion and a render of its own.
```text ```text
token {p} renders a level that {p.first} reads a path into; name the fields you want instead token {p} renders a level that {p.first} reads a path into; name the fields you want instead
@@ -492,7 +522,9 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan
### Performance ### Performance
Each file is parsed, validated and weight-indexed once, in `New`. A `Fake` call Each file is parsed, validated and weight-indexed once, in `New`. Proving the draw
fences adds one pass over the loaded tree, and walks what a render reads only where
data binds a reference, so a set that binds none pays for the pass alone. A `Fake` call
then costs about what its output costs: an unweighted pick is O(1) whatever the then costs about what its output costs: an unweighted pick is O(1) whatever the
list's length, a weighted one O(log n), and long formats, deep nesting and many list's length, a weighted one O(log n), and long formats, deep nesting and many
tokens add cost in proportion to the output. tokens add cost in proportion to the output.
@@ -524,7 +556,7 @@ tokens add cost in proportion to the output.
## Decisions ## Decisions
- **Options and fields share one namespace.** `format`, `weight`, `repeat`, - **Options and fields share one namespace.** `format`, `weight`, `repeat`,
`separator` and `datatype` are reserved; every other key is a field. Nesting fields under a `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a
key, or prefixing options, would tax every template to guard against a key, or prefixing options, would tax every template to guard against a
misspelt option. misspelt option.
- **`{a|b}` stays beside nested choices.** `[[…], […]]` picks the same way, but - **`{a|b}` stays beside nested choices.** `[[…], […]]` picks the same way, but
@@ -571,10 +603,12 @@ tokens add cost in proportion to the output.
to have would make `--seed 42` machine-dependent. Data still lives in `data/` to have would make `--seed 42` machine-dependent. Data still lives in `data/`
as JSON; `--data-path` layers over it. as JSON; `--data-path` layers over it.
- **A bare reference draws each time; a reference path is held.** `{/p} {/p}` - **A bare reference draws each time; a reference path is held.** `{/p} {/p}`
is two draws, as `{word} {word}` is, while `{/p.first}` beside a nested template is two draws, as `{word} {word}` is, while every `{/p.first}` in one render reads
rendering `{/p.first}` is a load error: a bare token is by contract an one draw, and a bare `{/p}` beside them is a load error: a bare token
independent draw, a path pins its level, and any route into a pinned level from is by contract an independent draw, a path pins its level, and a fresh draw of a
another expansion could show another row. pinned level could show another row. A builtin's operand holds what it reads for
its expansion, references included, so `{uppercase(/p)} {/p}` is one draw — the
rule every operand follows.
- **Reference sigils follow the filesystem.** `/` is the root, `.` this file's - **Reference sigils follow the filesystem.** `/` is the root, `.` this file's
folder, `..` the folder above — what those spellings already mean to anyone who folder, `..` the folder above — what those spellings already mean to anyone who
has typed a path. A locale's files reach each other without naming the locale, has typed a path. A locale's files reach each other without naming the locale,
@@ -640,24 +674,44 @@ tokens add cost in proportion to the output.
see a caller's types, so the first `FakeStruct` for a type compiles its tags and the see a caller's types, so the first `FakeStruct` for a type compiles its tags and the
answer, error included, is kept per type: a test's first call is its load, and no answer, error included, is kept per type: a test's first call is its load, and no
`NewStruct` handle is needed, as the cache already compiles once. `NewStruct` handle is needed, as the cache already compiles once.
- **A record shares one reference draw per category.** Two columns that reference - **A render shares one reference draw per category, per group.** Every reference
one category — `{/currency.code}` beside `{/currency.symbol}` — read one draw of path into a category in one `Fake`, or one record, reads one draw of it, so a
it, so a record's facts agree the way a template's [correlated value's facts agree across its fields, nested templates and columns alike —
fields](#correlated-fields) do. The draw is one per record, so it spans a `{/currency.code}` in one field and `{/currency.symbol}` in another name one
column's `repeat` and nested templates too (one record is one coherent unit); currency, whichever view renders them. A `repeat` iteration is a render of its
a bare reference — `{/currency}`, no field — stays an independent draw every own, since repeating asks for another entity, and a [draw group](#draw-group) names further
time, the rule a format string already follows. Only references share: a sibling entities within one render, so a payer and a payee are two groups over one
field is local to its own column, so a `first` column does not silently bind to `person` rather than two copies of it. Only references share: a sibling field is
a `first` in the column next to it. local to its own expansion, so a `first` column does not silently bind to a
`first` in the column next to it.
The string view of that same template does not share. `Fake` renders each - **A draw group name is local to its category.** A category's groups are its own
sibling field as its own expansion, so a `{/currency.code}` field beside a entities, so a caller naming a group the same way never joins them by accident,
`{/currency.symbol}` field is two draws and may render `EUR $`; writing both and renaming a group inside one file changes no render elsewhere. The unnamed
references in one `format` holds them together, as group still spans categories, since facts that belong together across categories
[One draw, one spelling](#one-draw-one-spelling) says. The scope is what makes a must agree.
row coherent when the columns *are* the output, and there the caller cannot fall - **The expansion hold and the render's draws are two fences.** One proves a sibling
back on one format string. Widening it to every render would change what `Fake` path or an operand is reached only by its readers within an expansion, the other
has emitted since the start, for a correlation a single format already reaches. does the same for reference paths across a render and its draw groups. They pin
different things — an operand pins the value its own render produced and stops at a
reference, a path pins every level it passes through — so one walk would carry both
rules and both scopes anyway, and tell them apart at every step.
- **A record makes its draw maps up front, a `Fake` on its first read.** A record's
columns always read through the render's draws, so making the maps where the set is
declared keeps them on that frame's stack. A `Fake` often reads no reference path at
all, and making them anyway cost about a fifth of the cheapest render, so it makes
them on the first read instead, at two heap allocations for a render that does share
a draw. The allocation gate over a repeat of a reference path and over a named draw
group prices that, and pins the two measures that keep a draw set off the heap.
- **A category never references itself, and a record's fences run at load.** A category
is one unit: a reference back into it — `{/users.first}` inside `users` — describes a
draw other than the fields beside it, so `New` refuses it and the sibling path stays
the one spelling for a field of one's own. A value two fields share goes in its own
category, which both reference. That settled, a record's column fences run at `New`
too, so a category that loads renders as whichever shape is asked for, and a reference
reaching back into a category through another one is refused there as the overlap it
is. Which reads those fences weigh differs on purpose: a record-only template's inert
format renders nothing, so a `drawGroup` on it can never matter and is refused, while
one on a rendering format can matter to a caller that bare-references it.
- **A record's column set is fixed before the first draw.** Only a category-level - **A record's column set is fixed before the first draw.** Only a category-level
template is a record: a path descending into a field, or naming a folder or a template is a record: a path descending into a field, or naming a folder or a
choice, errors. A tail may pass through a choice whose variants carry different choice, errors. A tail may pass through a choice whose variants carry different
@@ -737,6 +791,7 @@ struct.go structs: FakeStruct, fake tags, and a field's Go type as its col
inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
hold.go the hold: one draw per expansion for paths and operands, and its fences hold.go the hold: one draw per expansion for paths and operands, and its fences
draw.go one reference draw per render and group: draw sets, the group option, and its fence
reference.go reference sigils, and binding references across the tree reference.go reference sigils, and binding references across the tree
graph.go the render graph: edges, cycles, the repeat bound, tree walks graph.go the render graph: edges, cycles, the repeat bound, tree walks
builtins.go the {name()} function registry and its implementations builtins.go the {name()} function registry and its implementations
data.go
+12 -12
View File
@@ -28,10 +28,10 @@ func (s dataSource) name(p string) string {
} }
// loadData loads every source into one namespace tree and returns its root // loadData loads every source into one namespace tree and returns its root
// children. A directory becomes a group; each *.json file in it compiles to a node // children. A directory becomes a folder; each *.json file in it compiles to a node
// keyed by its base name (address.json -> "address"); each subdirectory becomes a // keyed by its base name (address.json -> "address"); each subdirectory becomes a
// nested group, so folders turn into dot-path segments. Sources merge left to right: // nested folder, so folders turn into dot-path segments. Sources merge left to right:
// matching groups merge by their children, and any other clash is won by the last // matching folders merge by their children, and any other clash is won by the last
// source loaded. Once merged, linkRefs binds every reference against the // source loaded. Once merged, linkRefs binds every reference against the
// final tree. // final tree.
func loadData(sources []dataSource) (map[string]node, error) { func loadData(sources []dataSource) (map[string]node, error) {
@@ -74,15 +74,15 @@ func loadData(sources []dataSource) (map[string]node, error) {
return root, nil return root, nil
} }
// loadDir compiles one directory into a group. fs.ReadDir yields entries sorted // loadDir compiles one directory into a folder. fs.ReadDir yields entries sorted
// by name, so the tree is built deterministically. Empty subdirectories (no JSON // by name, so the tree is built deterministically. Empty subdirectories (no JSON
// anywhere under them) are skipped rather than added as empty namespaces. // anywhere under them) are skipped rather than added as empty namespaces.
func loadDir(src dataSource, dir string) (*group, error) { func loadDir(src dataSource, dir string) (*folder, error) {
entries, err := fs.ReadDir(src.fsys, dir) entries, err := fs.ReadDir(src.fsys, dir)
if err != nil { if err != nil {
return nil, fmt.Errorf("%s: %w", src.name(dir), err) return nil, fmt.Errorf("%s: %w", src.name(dir), err)
} }
g := &group{children: map[string]node{}} g := &folder{children: map[string]node{}}
for _, e := range entries { for _, e := range entries {
if strings.HasPrefix(e.Name(), ".") { // hidden: a checkout or an editor's file, never data if strings.HasPrefix(e.Name(), ".") { // hidden: a checkout or an editor's file, never data
continue continue
@@ -99,8 +99,8 @@ func loadDir(src dataSource, dir string) (*group, error) {
return g, nil return g, nil
} }
// loadFolder adds a subdirectory as a nested group, unless nothing under it is data. // loadFolder adds a subdirectory as a nested folder, unless nothing under it is data.
func loadFolder(src dataSource, g *group, full, name string) error { func loadFolder(src dataSource, g *folder, full, name string) error {
child, err := loadDir(src, full) child, err := loadDir(src, full)
if err != nil { if err != nil {
return err return err
@@ -117,7 +117,7 @@ func loadFolder(src dataSource, g *group, full, name string) error {
// loadFile compiles a *.json file into a category named after it; any other file // loadFile compiles a *.json file into a category named after it; any other file
// is skipped. // is skipped.
func loadFile(src dataSource, g *group, full, file string) error { func loadFile(src dataSource, g *folder, full, file string) error {
if !strings.HasSuffix(file, ".json") { if !strings.HasSuffix(file, ".json") {
return nil return nil
} }
@@ -141,13 +141,13 @@ func loadFile(src dataSource, g *group, full, file string) error {
return nil return nil
} }
// mergeChildren overlays src onto dst. Two groups under the same key merge // mergeChildren overlays src onto dst. Two folders under the same key merge
// recursively (so locales/categories from several paths combine); every other // recursively (so locales/categories from several paths combine); every other
// key is replaced, making the last-loaded directory win on a conflict. // key is replaced, making the last-loaded directory win on a conflict.
func mergeChildren(dst, src map[string]node) { func mergeChildren(dst, src map[string]node) {
for k, v := range src { for k, v := range src {
if dg, ok := dst[k].(*group); ok { if dg, ok := dst[k].(*folder); ok {
if sg, ok := v.(*group); ok { if sg, ok := v.(*folder); ok {
mergeChildren(dg.children, sg.children) mergeChildren(dg.children, sg.children)
continue continue
} }
draw.go
+326
View File
@@ -0,0 +1,326 @@
package fejkdata
import (
"fmt"
"sort"
"strings"
)
// drawSet is one render's reference draws: the unnamed draw group's, and each named one's.
type drawSet struct {
unnamed draws
named map[string]*draws
}
// drawScope is where a render reads its reference paths: a draw set, in the draw group of the
// template rendering.
type drawScope struct {
set *drawSet
group string
}
// newDrawSet makes the unnamed group's maps where the set is declared, keeping them on that frame's
// stack for a render that reads through them; a zero drawSet makes them on its first read instead.
func newDrawSet() drawSet {
return drawSet{unnamed: draws{variant: map[string]node{}, value: map[string]draw{}}}
}
// renderOnce renders n as one render, over draws of its own.
func renderOnce(s *session, n node) string {
var set drawSet
return render(s, n, drawScope{set: &set})
}
// in is the scope t renders in: its draw group where it names one, else its caller's.
func (sc drawScope) in(t *template) drawScope {
if t.drawGroupKey != "" {
sc.group = t.drawGroupKey
}
return sc
}
// draws is the set's draws for sc's draw group.
func (sc drawScope) draws() *draws {
if sc.group == "" {
return &sc.set.unnamed
}
d, drew := sc.set.named[sc.group]
if !drew {
if sc.set.named == nil {
sc.set.named = map[string]*draws{}
}
d = &draws{variant: map[string]node{}, value: map[string]draw{}}
sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's draw set, to the heap
}
return d
}
// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each
// iteration renders in no draw group.
func drawGroupOf(m map[string]any, repeat int) (string, error) {
v, ok := m["drawGroup"]
if !ok {
return "", nil
}
name, ok := v.(string)
switch {
case !ok:
return "", fmt.Errorf("drawGroup must be a string, got %T", v)
case name == "":
return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`)
case repeat > 1:
return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name)
}
return name, nil
}
// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name
// is local to its category.
func (t *template) keyDrawGroup(category string) {
if t.drawGroup != "" {
t.drawGroupKey = category + "/" + t.drawGroup
}
}
// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again,
// short of a repeat or another draw group.
func checkNestedDrawGroup(fields map[string]node, group string) error {
if group == "" {
return nil
}
var walk func(path string, n node) error
walk = func(path string, n node) error {
t, isTemplate := n.(*template)
switch {
case isTemplate && t.drawGroup == group:
return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group)
case isTemplate && (t.drawGroup != "" || t.repeat > 1):
return nil
}
for _, c := range contained(n) {
if err := walk(join(path, c.name), c.node); err != nil {
return err
}
}
return nil
}
for _, name := range sortedNames(fields) {
if err := walk(name, fields[name]); err != nil {
return err
}
}
return nil
}
// drawCheck fences each template of a scope as a render of its own, remembering which nodes read a
// reference path.
type drawCheck struct {
memo map[readsMemo]bool
}
type readsMemo struct {
n node
stopAtGroup bool
}
// checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference
// path inside a repeat or a nested draw group, which draw apart from it whatever it names.
func (c *drawCheck) checkDrawGroup(path string, n node) error {
if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) {
return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup)
}
return nil
}
func (c *drawCheck) checkDraws(path string, n node) error {
t, ok := n.(*template)
if !ok || !c.readsPath(t) {
return nil
}
w := newDrawWalk()
for _, e := range renderEdges(t) {
w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}})
}
if err := w.check(); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
// checkRecordDraws fences the columns of a record — a template compiled at the top without a
// repeat — so a load proves the record view of it as well as the string view.
func (c *drawCheck) checkRecordDraws(path string, n node) error {
t, ok := n.(*template)
if !ok || !t.record {
return nil
}
// A record-only template's format renders nothing, so weigh the columns, not the format.
columns := recordColumns(t)
reads := false
for _, name := range columns {
reads = reads || c.readsPath(t.fields[name])
}
if !reads {
return nil
}
if err := checkColumnDraws(t, columns); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
// readsPath reports whether rendering n reads a reference path, short of a repeat, which renders
// over draws of its own.
func (c *drawCheck) readsPath(n node) bool { return c.reads(n, false) }
// splitsDraws reports whether rendering n reads a reference path that n's own draw group answers
// for: one outside a repeat and outside a nested draw group, which hold their own draws.
func (c *drawCheck) splitsDraws(n node) bool { return c.reads(n, true) }
// reads walks what rendering n renders for a reference path, stopping at a repeat — and at a nested
// draw group when stopAtGroup — since each holds draws of its own.
func (c *drawCheck) reads(n node, stopAtGroup bool) bool {
k := readsMemo{n, stopAtGroup}
if r, done := c.memo[k]; done {
return r
}
r := false
for _, e := range renderEdges(n) {
if a, isRef := refRead(n, e.label); (isRef && len(a.tail) > 0) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) {
r = true
break
}
}
if c.memo == nil {
c.memo = map[readsMemo]bool{}
}
c.memo[k] = r
return r
}
func repeats(n node) bool {
t, isTemplate := n.(*template)
return isTemplate && t.repeat > 1
}
func grouped(n node) bool {
t, isTemplate := n.(*template)
return isTemplate && t.drawGroupKey != ""
}
// refRead is the reference an edge of n reads; false when the edge reads none.
func refRead(n node, label string) (arm, bool) {
t, isTemplate := n.(*template)
if !isTemplate {
return arm{}, false
}
a := splitArm(label, t.refs)
return a, isRef(a.key)
}
// checkColumnDraws fences a record's columns as one render.
func checkColumnDraws(t *template, columns []string) error {
w := newDrawWalk()
for _, name := range columns {
w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}})
}
return w.check()
}
// drawWalk gathers the references one render reads, by draw group, for check to compare.
type drawWalk struct {
reads []pathRead
read map[drawKey]bool
seen map[drawVisit]bool
}
// drawAt is where a walk stands: the draw group it draws in, and how the render's root reached it.
type drawAt struct {
group string
route drawRoute
}
// drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label.
type drawRoute struct{ spelling, label string }
// pathRead is one reference a render reads: a path, or a bare reference with no tail.
type pathRead struct {
at drawAt
a arm
}
type drawVisit struct {
n node
group string
}
type drawKey struct {
group string
path string
}
func newDrawWalk() *drawWalk {
return &drawWalk{read: map[drawKey]bool{}, seen: map[drawVisit]bool{}}
}
// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops
// there.
func (w *drawWalk) walk(n node, at drawAt) {
v := drawVisit{n, at.group}
if w.seen[v] {
return
}
w.seen[v] = true
if repeats(n) {
return
}
if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" {
at.group = t.drawGroupKey
}
for _, e := range renderEdges(n) {
w.edge(n, e, at)
}
}
func (w *drawWalk) edge(from node, e renderEdge, at drawAt) {
if a, reads := refRead(from, e.label); reads {
if k := (drawKey{at.group, a.path}); !w.read[k] {
w.read[k] = true
w.reads = append(w.reads, pathRead{at, a})
}
}
w.walk(e.to, at)
}
// check refuses what one draw per reference path cannot answer for: a read of a level beside a path
// another read takes into it. Reads are compared in path order, so which pair is reported does not
// vary.
func (w *drawWalk) check() error {
sort.SliceStable(w.reads, func(i, j int) bool {
if w.reads[i].at.group != w.reads[j].at.group {
return w.reads[i].at.group < w.reads[j].at.group
}
return w.reads[i].a.path < w.reads[j].a.path
})
for i, level := range w.reads {
for _, into := range w.reads[i+1:] {
if into.at.group == level.at.group && strings.HasPrefix(into.a.path, level.a.path+".") {
return overlapError(level.at.route, level.a.name, into)
}
}
}
return nil
}
func overlapError(route drawRoute, ref string, into pathRead) error {
return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name))
}
// spelled names the route, and the reference it reaches a draw by where its root edge is not that
// reference.
func (r drawRoute) spelled(ref string) string {
if ref == "" || ref == r.label {
return r.spelling
}
return fmt.Sprintf("%s with {%s}", r.spelling, ref)
}
fejkdata.go
+2 -2
View File
@@ -142,10 +142,10 @@ func (f *Generator) List() []string {
} }
// paths lists the dot paths addressable from n, relative to it, where "" is n // paths lists the dot paths addressable from n, relative to it, where "" is n
// itself. A group has no value of its own, so it contributes only its children's. // itself. A folder has no value of its own, so it contributes only its children's.
func paths(n node) []string { func paths(n node) []string {
switch n := n.(type) { switch n := n.(type) {
case *group: case *folder:
var out []string var out []string
for _, name := range sortedNames(n.children) { for _, name := range sortedNames(n.children) {
for _, p := range paths(n.children[name]) { for _, p := range paths(n.children[name]) {
graph.go
+21 -3
View File
@@ -51,7 +51,7 @@ type namedNode struct {
func contained(n node) []namedNode { func contained(n node) []namedNode {
switch n := n.(type) { switch n := n.(type) {
case *group: case *folder:
return named(n.children) return named(n.children)
case *choice: case *choice:
out := make([]namedNode, len(n.items)) out := make([]namedNode, len(n.items))
@@ -69,7 +69,7 @@ func contained(n node) []namedNode {
// named skips a bound {/path} key: it is a render edge, not containment, so using // named skips a bound {/path} key: it is a render edge, not containment, so using
// it as a path segment would report a node under a path that does not reach it. Only // it as a path segment would report a node under a path that does not reach it. Only
// a template's fields hold bindings — loadDir skips a dot-prefixed entry, so a // a template's fields hold bindings — loadDir skips a dot-prefixed entry, so a
// group's children never carry the prefix — so this one skip serves both. // folder's children never carry the prefix — so this one skip serves both.
func named(m map[string]node) []namedNode { func named(m map[string]node) []namedNode {
out := make([]namedNode, 0, len(m)) out := make([]namedNode, 0, len(m))
for _, name := range sortedNames(m) { for _, name := range sortedNames(m) {
@@ -111,7 +111,7 @@ func (e renderEdge) reached() string {
// renderEdges lists the children rendering n recurses into, mirroring expand: a // renderEdges lists the children rendering n recurses into, mirroring expand: a
// choice's items, and a template's field/reference tokens plus its operands. A // choice's items, and a template's field/reference tokens plus its operands. A
// group renders nothing, so it has no edges. // folder renders nothing, so it has no edges.
func renderEdges(n node) []renderEdge { func renderEdges(n node) []renderEdge {
switch n := n.(type) { switch n := n.(type) {
case *choice: case *choice:
@@ -196,6 +196,24 @@ func checkRenders(s nodeScope) error {
if err := s(heldCheck); err != nil { if err := s(heldCheck); err != nil {
return err return err
} }
fence := &drawCheck{}
refs := false
if err := s(func(path string, n node) error {
if t, ok := n.(*template); ok && len(t.refs) > 0 {
refs = true
}
return fence.checkDrawGroup(path, n)
}); err != nil {
return err
}
if refs {
if err := s(fence.checkDraws); err != nil {
return err
}
if err := s(fence.checkRecordDraws); err != nil {
return err
}
}
return s((&valueProof{}).checkDatatype) return s((&valueProof{}).checkDatatype)
} }
hold.go
+39 -30
View File
@@ -6,11 +6,12 @@ import (
"strings" "strings"
) )
// heldCheck rejects every route to a held name except the ones that read its draw. // heldCheck rejects every route to a held sibling name except the ones that read its
// An expansion holds one draw of that name; anything else that renders it draws // draw. An expansion holds one draw of that name; anything else that renders it draws
// again, and the two disagree. checkNoOverlap settles the spellings within one // again, and the two disagree. checkNoOverlap settles the spellings within one format
// format (a token, an operand); this settles the rest — a reference, whether it // (a token, an operand); this settles the rest — a reference, whether it sits in that
// sits in that format or in anything the format renders, however deep. // format or in anything the format renders, however deep. A reference path is held
// for the whole render instead, which drawCheck fences.
func heldCheck(path string, n node) error { func heldCheck(path string, n node) error {
t, ok := n.(*template) t, ok := n.(*template)
if !ok || len(t.held) == 0 { if !ok || len(t.held) == 0 {
@@ -18,6 +19,9 @@ func heldCheck(path string, n node) error {
} }
readers := boundReaders(t.format, t.bound, t.refs) readers := boundReaders(t.format, t.bound, t.refs)
for _, head := range heldHeads(t) { for _, head := range heldHeads(t) {
if _, isPath := t.bound[head]; isPath && isRef(head) {
continue
}
if err := checkHeadHeld(t, head, readers); err != nil { if err := checkHeadHeld(t, head, readers); err != nil {
return fmt.Errorf("%s: %w", path, err) return fmt.Errorf("%s: %w", path, err)
} }
@@ -175,12 +179,11 @@ func renders(n node, want, seen map[node]bool) bool {
return false return false
} }
// checkNoOverlap rejects a format that both renders a level and reads a path into // checkNoOverlap rejects a format that both renders a sibling level and reads a path
// it — {p} beside {p.first}, {p.addr} beside {p.addr.city}, {.p} beside // into it — {p} beside {p.first}, {p.addr} beside {p.addr.city}. The path reads the
// {/sv_SE.p.first}. The path reads the level's held draw while rendering the level // level's held draw while rendering the level expands it afresh, so their values would
// expands it afresh, so their values would disagree. Reads are compared by their // disagree. Reads are compared in sorted order, so which pair is reported does not
// one spelling, in sorted order, so which pair is reported depends neither on how // depend on where the tokens sit.
// a reference was written nor on where the tokens sit.
func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error { func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error {
names := boundReaders(format, bound, refs) names := boundReaders(format, bound, refs)
// Stable over one format-order scan, so two readers of one name (a token and a // Stable over one format-order scan, so two readers of one name (a token and a
@@ -200,7 +203,7 @@ func checkNoOverlap(format string, bound map[string]string, refs map[string]refB
// spelling, and how to name it. // spelling, and how to name it.
type reader struct{ name, path, label string } type reader struct{ name, path, label string }
// boundReaders lists every way a format reaches a bound field, in the order the // boundReaders lists every way a format reaches a bound sibling field, in the order the
// format writes them. An operand renders its field, so it names a level exactly // format writes them. An operand renders its field, so it names a level exactly
// as a token does; one scan finds both, which is what puts them in one order. // as a token does; one scan finds both, which is what puts them in one order.
func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader { func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader {
@@ -212,14 +215,14 @@ func boundReaders(format string, bound map[string]string, refs map[string]refBin
if fn, _, isFunc := funcCall(t.body); isFunc { if fn, _, isFunc := funcCall(t.body); isFunc {
for _, operand := range tokenOperands(t.body) { for _, operand := range tokenOperands(t.body) {
a := splitArm(operand, refs) a := splitArm(operand, refs)
if _, isBound := bound[a.key]; isBound { if _, isBound := bound[a.key]; isBound && !isRef(a.key) {
names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)}) names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)})
} }
} }
return nil return nil
} }
for _, a := range splitArms(t.body, refs) { for _, a := range splitArms(t.body, refs) {
if _, isBound := bound[a.key]; isBound { if _, isBound := bound[a.key]; isBound && !isRef(a.key) {
names = append(names, reader{a.name, a.path, "token {" + a.name + "}"}) names = append(names, reader{a.name, a.path, "token {" + a.name + "}"})
} }
} }
@@ -252,9 +255,10 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding)
}) })
} }
// draws is what an expansion has already drawn for its held names: the variant each // draws is what has already been drawn for held names: the variant each was drawn as,
// was drawn as, so every path under it reads one row, and the draw each read made, by // so every path under it reads one row, and the draw each read made, by its one
// its one spelling, so the same read written twice reads one draw. // spelling, so the same read written twice reads one draw. An expansion keeps one for
// its sibling names, and a render one per group for its reference paths.
type draws struct { type draws struct {
variant map[string]node variant map[string]node
value map[string]draw value map[string]draw
@@ -273,14 +277,14 @@ type draw struct {
// gives one value, and a shown operand is the operand computed. Every other name is // gives one value, and a shown operand is the operand computed. Every other name is
// drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and // drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and
// linkRefs prove every step, so the walk cannot fail. // linkRefs prove every step, so the walk cannot fail.
func readField(s *session, t *template, held, refScope *draws, a arm) draw { func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw {
if !t.held[a.key] { if !t.held[a.key] {
if len(a.tail) > 0 { if len(a.tail) > 0 {
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key)) panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
} }
return draw{text: render(s, t.fields[a.key], refScope)} return draw{text: render(s, t.fields[a.key], sc)}
} }
d := readScope(held, refScope, a) d := readScope(held, sc, a)
if r, done := d.value[a.path]; done { if r, done := d.value[a.path]; done {
return r return r
} }
@@ -296,37 +300,42 @@ func readField(s *session, t *template, held, refScope *draws, a arm) draw {
n, drew := d.variant[key] n, drew := d.variant[key]
if !drew { if !drew {
n = drawn(s, c) n = drawn(s, c)
if d.variant == nil {
d.variant = map[string]node{}
}
d.variant[key] = n d.variant[key] = n
} }
return []node{n}, nil return []node{n}, nil
}, },
leaf: func(n node) error { r = renderLeaf(s, n, refScope); return nil }, leaf: func(n node) error { r = renderLeaf(s, n, sc); return nil },
}) })
if d.value == nil {
d.value = map[string]draw{}
}
d.value[a.path] = r d.value[a.path] = r
return r return r
} }
// readScope is the draws a held read keeps its draw in: for a reference that reads a path, // readScope is the draws a held read keeps its draw in: for a reference that reads a path,
// refScope, so its draw outlives this expansion; for a sibling, or a reference read whole, held. // the render's draws for its group, so its draw spans the render; for a sibling, or a
func readScope(held, refScope *draws, a arm) *draws { // reference read whole, held.
if isRef(a.key) && refScope != nil && len(a.tail) > 0 { func readScope(held *draws, sc drawScope, a arm) *draws {
return refScope if isRef(a.key) && len(a.tail) > 0 {
return sc.draws()
} }
return held return held
} }
// renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one
// reference alone whose read drew null. // reference alone whose read drew null.
func renderLeaf(s *session, n node, scope *draws) draw { func renderLeaf(s *session, n node, sc drawScope) draw {
n = drawn(s, n) n = drawn(s, n)
if _, isNull := n.(*null); isNull { if _, isNull := n.(*null); isNull {
return draw{null: true} return draw{null: true}
} }
r := draw{text: render(s, n, scope)} r := draw{text: render(s, n, sc)}
if t, _ := n.(*template); t != nil && t.readsColumn != nil { if t, _ := n.(*template); t != nil && t.readsColumn != nil {
if d := readScope(nil, scope, t.readsColumn.a); d != nil { r.null = sc.in(t).draws().value[t.readsColumn.a.path].null
r.null = d.value[t.readsColumn.a.path].null
}
} }
return r return r
} }
hold_test.go
+84 -36
View File
@@ -91,17 +91,28 @@ func TestReferenceNamingABoundLevelIsRejected(t *testing.T) {
// A reference can name a bound level from the data root, which renders it // A reference can name a bound level from the data root, which renders it
// afresh beside the path that reads its held draw — the same overlap by // afresh beside the path that reads its held draw — the same overlap by
// another spelling. // another spelling.
rejected := map[string]string{ // A category never references itself, so each reference back into cat sits in a
"reference names the head": `{"format":"{p.first}|{/cat.p}","p":{"format":"{first}","first":["Anna","Bo"]}}`, // second category cat renders.
"reference names the leaf": `{"format":"{p.addr}|{/cat.p.addr}","p":{"format":"x","addr":["A","B","C","D"]}}`, rejected := map[string]map[string]string{
"reference names the head": {
"cat": `{"format":"{p.first}|{/hop}","p":{"format":"{first}","first":["Anna","Bo"]}}`,
"hop": `"{/cat.p}"`,
},
"reference names the leaf": {
"cat": `{"format":"{p.addr}|{/hop}","p":{"format":"x","addr":["A","B","C","D"]}}`,
"hop": `"{/cat.p.addr}"`,
},
// The reference need not sit in the format that binds: any field it renders // The reference need not sit in the format that binds: any field it renders
// reaches the level just the same, however deep. // reaches the level just the same, however deep.
"reference from a sibling field": `{"format":"{p.first}|{inner}","p":[` + "reference from a sibling field": {
`{"format":"{first}-{last}","first":"A","last":"1"},{"format":"{first}-{last}","first":"B","last":"2"}],` + "cat": `{"format":"{p.first}|{inner}","p":[` +
`"inner":"{/cat.p}"}`, `{"format":"{first}-{last}","first":"A","last":"1"},{"format":"{first}-{last}","first":"B","last":"2"}],` +
`"inner":"{/hop}"}`,
"hop": `"{/cat.p}"`,
},
} }
for name, file := range rejected { for name, files := range rejected {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file}))) _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files)))
if err == nil || !strings.Contains(err.Error(), "reads a path into") { if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: New = %v, want the reference rejected as an overlap", name, err) t.Errorf("%s: New = %v, want the reference rejected as an overlap", name, err)
} }
@@ -120,7 +131,8 @@ func TestCycleReachedOnlyByAPathTokenIsRejected(t *testing.T) {
// cycle walk has to follow it there. A head whose own format names nothing // cycle walk has to follow it there. A head whose own format names nothing
// would otherwise hide the cycle until render, where it is fatal. // would otherwise hide the cycle until render, where it is fatal.
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"a": `{"format":"{p.x}","p":{"format":"static","x":"{/a}"}}`, "a": `{"format":"{p.x}","p":{"format":"static","x":"{/b}"}}`,
"b": `"{/a}"`,
}))) })))
if err == nil || !strings.Contains(err.Error(), "reference cycle") { if err == nil || !strings.Contains(err.Error(), "reference cycle") {
t.Fatalf("New = %v, want the cycle through {p.x} rejected", err) t.Fatalf("New = %v, want the cycle through {p.x} rejected", err)
@@ -131,8 +143,9 @@ func TestALevelRenderedOnlyByAPathTokenIsHeld(t *testing.T) {
// {p.a} renders q, so it is a route to the level {q.x} holds — even though p's // {p.a} renders q, so it is a route to the level {q.x} holds — even though p's
// own format names nothing. // own format names nothing.
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"thing": `{"format":"{p.a} {q.x}","p":{"format":"static","a":"{/thing.q}"},` + "thing": `{"format":"{p.a} {q.x}","p":{"format":"static","a":"{/hop}"},` +
`"q":{"format":"{x}","x":["1","2"]}}`, `"q":{"format":"{x}","x":["1","2"]}}`,
"hop": `"{/thing.q}"`,
}))) })))
if err == nil || !strings.Contains(err.Error(), "reads a path into") { if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Fatalf("New = %v, want the second route to q rejected", err) t.Fatalf("New = %v, want the second route to q rejected", err)
@@ -149,16 +162,20 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
files map[string]string files map[string]string
want string // the route the error names, spelled as the author wrote it want string // the route the error names, spelled as the author wrote it
}{ }{
// A category never references itself, so each route back into cat sits in a
// second category cat renders.
"a reference beside the operand": { "a reference beside the operand": {
map[string]string{ map[string]string{
"cat": `{"format":"{/cat.net} x 2 = {calc(net * 2, 2)}","net":["10.00","20.00"]}`, "cat": `{"format":"{/hop} x 2 = {calc(net * 2, 2)}","net":["10.00","20.00"]}`,
"hop": `"{/cat.net}"`,
}, },
`{/cat.net} renders "net"`, `{/hop} renders "net"`,
}, },
"a reference one level down": { "a reference one level down": {
map[string]string{ map[string]string{
"cat": `{"format":"{calc(net * 2, 2)} {q}","net":["10.00","20.00"],` + "cat": `{"format":"{calc(net * 2, 2)} {q}","net":["10.00","20.00"],` +
`"q":"{/cat.net}"}`, `"q":"{/hop}"}`,
"hop": `"{/cat.net}"`,
}, },
`{q} renders "net"`, `{q} renders "net"`,
}, },
@@ -167,7 +184,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"a reference to an operand wrapped in a choice": { "a reference to an operand wrapped in a choice": {
map[string]string{ map[string]string{
"cat": `{"format":"{calc(n * 2, 2)} {q}","n":{"format":"{v}","v":["1","2"]},` + "cat": `{"format":"{calc(n * 2, 2)} {q}","n":{"format":"{v}","v":["1","2"]},` +
`"q":"{/cat.n}"}`, `"q":"{/hop}"}`,
"hop": `"{/cat.n}"`,
}, },
`{q} renders "n"`, `{q} renders "n"`,
}, },
@@ -176,7 +194,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"an operand reaching another operand": { "an operand reaching another operand": {
map[string]string{ map[string]string{
"cat": `{"format":"{calc(a + b, 0)}","a":{"format":"{x}","x":["1","2"]},` + "cat": `{"format":"{calc(a + b, 0)}","a":{"format":"{x}","x":["1","2"]},` +
`"b":"{/cat.a}"}`, `"b":"{/hop}"}`,
"hop": `"{/cat.a}"`,
}, },
`calc operand "b" renders "a"`, `calc operand "b" renders "a"`,
}, },
@@ -185,15 +204,17 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
// rejected, so the reference spelling has to be. // rejected, so the reference spelling has to be.
"a reference into the operand": { "a reference into the operand": {
map[string]string{ map[string]string{
"cat": `{"format":"{calc(net * 2, 2)}|{/cat.net.v}",` + "cat": `{"format":"{calc(net * 2, 2)}|{/hop}",` +
`"net":{"format":"{v}","v":["10.00","20.00"]}}`, `"net":{"format":"{v}","v":["10.00","20.00"]}}`,
"hop": `"{/cat.net.v}"`,
}, },
`{/cat.net.v} renders "net"`, `{/hop} renders "net"`,
}, },
"a reference into the operand one level down": { "a reference into the operand one level down": {
map[string]string{ map[string]string{
"cat": `{"format":"{calc(net * 2, 2)}|{q}","net":{"format":"{v}","v":["10.00","20.00"]},` + "cat": `{"format":"{calc(net * 2, 2)}|{q}","net":{"format":"{v}","v":["10.00","20.00"]},` +
`"q":"{/cat.net.v}"}`, `"q":"{/hop}"}`,
"hop": `"{/cat.net.v}"`,
}, },
`{q} renders "net"`, `{q} renders "net"`,
}, },
@@ -203,7 +224,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"a violation on the second of two held heads": { "a violation on the second of two held heads": {
map[string]string{ map[string]string{
"cat": `{"format":"{calc(a + b, 0)} {w}","a":{"format":"{x}","x":["1","2"]},` + "cat": `{"format":"{calc(a + b, 0)} {w}","a":{"format":"{x}","x":["1","2"]},` +
`"b":{"format":"{y}","y":["3","4"]},"w":"{/cat.b}"}`, `"b":{"format":"{y}","y":["3","4"]},"w":"{/hop}"}`,
"hop": `"{/cat.b}"`,
}, },
`{w} renders "b"`, `{w} renders "b"`,
}, },
@@ -230,7 +252,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
"a reference to a sibling the operand never renders": { "a reference to a sibling the operand never renders": {
"cat": `{"format":"{calc(net * 2, 2)} {unit}",` + "cat": `{"format":"{calc(net * 2, 2)} {unit}",` +
`"net":{"format":"{v}","v":["1","2"],"spare":["kg","lb"]},` + `"net":{"format":"{v}","v":["1","2"],"spare":["kg","lb"]},` +
`"unit":"{/cat.net.spare}"}`, `"unit":"{/hop}"}`,
"hop": `"{/cat.net.spare}"`,
}, },
// Two operands drawing from one source are two names, so two draws: each is // Two operands drawing from one source are two names, so two draws: each is
// held under its own name and shown once, and neither can disagree. // held under its own name and shown once, and neither can disagree.
@@ -247,7 +270,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) {
}, },
// A fixed string cannot disagree with itself, so it needs no fence. // A fixed string cannot disagree with itself, so it needs no fence.
"a literal operand named twice": { "a literal operand named twice": {
"cat": `{"format":"{calc(n * 2, 0)} {/cat.n}","n":"5"}`, "cat": `{"format":"{calc(n * 2, 0)} {/hop}","n":"5"}`,
"hop": `"{/cat.n}"`,
}, },
// One node reached twice while walking the operand: the walk must not // One node reached twice while walking the operand: the walk must not
// revisit it, and the repeat is not a second route to anything. // revisit it, and the repeat is not a second route to anything.
@@ -267,19 +291,28 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) {
// all of them. Reaching only the first leaves whatever hides in a later // all of them. Reaching only the first leaves whatever hides in a later
// variant to be found at render — a cycle there is fatal, and a second route // variant to be found at render — a cycle there is fatal, and a second route
// to a held level disagrees silently. // to a held level disagrees silently.
rejected := map[string]struct{ file, want string }{ rejected := map[string]struct {
files map[string]string
want string
}{
"a cycle in a later variant": { "a cycle in a later variant": {
`{"format":"{p.x}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/cat}"}]}`, map[string]string{
"cat": `{"format":"{p.x}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/hop}"}]}`,
"hop": `"{/cat}"`,
},
"reference cycle", "reference cycle",
}, },
"a second route in a later variant": { "a second route in a later variant": {
`{"format":"{p.x} {q.y}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/cat.q}"}],` + map[string]string{
`"q":{"format":"{y}","y":["1","2"]}}`, "cat": `{"format":"{p.x} {q.y}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/hop}"}],` +
`"q":{"format":"{y}","y":["1","2"]}}`,
"hop": `"{/cat.q}"`,
},
"reads a path into", "reads a path into",
}, },
} }
for name, c := range rejected { for name, c := range rejected {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": c.file}))) _, err := New(WithoutShippedData(), WithDataPath(writeData(t, c.files)))
if err == nil || !strings.Contains(err.Error(), c.want) { if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: New = %v, want it to mention %q", name, err, c.want) t.Errorf("%s: New = %v, want it to mention %q", name, err, c.want)
} }
@@ -289,15 +322,22 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) {
func TestALevelAPathNeverRendersIsAccepted(t *testing.T) { func TestALevelAPathNeverRendersIsAccepted(t *testing.T) {
// A path token does not expand its head's format, so a reference sitting in // A path token does not expand its head's format, so a reference sitting in
// that format is not a second route to anything: it is never rendered by the // that format is not a second route to anything: it is never rendered by the
// path at all. Both orders must load. // path at all. Both orders must load. p and q sit one level down, so they are
accepted := map[string]string{ // one column of thing rather than two that would read one another.
"reference in the head's own format": `{"format":"{p.first} {q.a}",` + accepted := map[string]map[string]string{
`"p":{"format":"{first} {/thing.q}","first":["A","B"]},"q":{"format":"{a}","a":["1","2"]}}`, "reference in the head's own format": {
"the mirror shape": `{"format":"{p.first} {q.a}",` + "thing": `{"format":"{inner}","inner":{"format":"{p.first} {q.a}",` +
`"p":{"format":"{first}","first":["A","B"]},"q":{"format":"{a} {/thing.p}","a":["1","2"]}}`, `"p":{"format":"{first} {/hop}","first":["A","B"]},"q":{"format":"{a}","a":["1","2"]}}}`,
"hop": `"{/thing.inner.q}"`,
},
"the mirror shape": {
"thing": `{"format":"{inner}","inner":{"format":"{p.first} {q.a}",` +
`"p":{"format":"{first}","first":["A","B"]},"q":{"format":"{a} {/hop}","a":["1","2"]}}}`,
"hop": `"{/thing.inner.p}"`,
},
} }
for name, file := range accepted { for name, files := range accepted {
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"thing": file}))); err != nil { if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files))); err != nil {
t.Errorf("%s: New = %v, want it accepted", name, err) t.Errorf("%s: New = %v, want it accepted", name, err)
} }
} }
@@ -404,6 +444,13 @@ func TestPathIntoARepeatingLevelIsRejected(t *testing.T) {
if err == nil || !strings.Contains(err.Error(), "repeat") { if err == nil || !strings.Contains(err.Error(), "repeat") {
t.Fatalf("New = %v, want a path into a repeating level rejected", err) t.Fatalf("New = %v, want a path into a repeating level rejected", err)
} }
_, err = New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"cat": `{"format":"[{p.a}]","p":{"format":"{a}","a":"{/word.w}","drawGroup":"g"}}`,
"word": `{"format":"{w}","w":["x","y"]}`,
})))
if err == nil || !strings.Contains(err.Error(), `the level "p" carries a drawGroup`) {
t.Fatalf("New = %v, want a path into a level carrying a group rejected", err)
}
} }
func TestPathIntoAPlainTemplateNamesTheMissingField(t *testing.T) { func TestPathIntoAPlainTemplateNamesTheMissingField(t *testing.T) {
@@ -666,7 +713,8 @@ func TestCycleThroughAPathTokenIsRejected(t *testing.T) {
// must be caught at New. Reaching render would be fatal: the recursion never // must be caught at New. Reaching render would be fatal: the recursion never
// terminates, and a stack overflow cannot be recovered. // terminates, and a stack overflow cannot be recovered.
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"a": `{"format":"{p.x}","p":{"format":"{x}","x":"{/a}"}}`, "a": `{"format":"{p.x}","p":{"format":"{x}","x":"{/b}"}}`,
"b": `"{/a}"`,
}))) })))
if err == nil || !strings.Contains(err.Error(), "reference cycle") { if err == nil || !strings.Contains(err.Error(), "reference cycle") {
t.Fatalf("New = %v, want the cycle through {p.x} rejected", err) t.Fatalf("New = %v, want the cycle through {p.x} rejected", err)
@@ -718,6 +766,6 @@ func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) {
t.Fatal("not a template") t.Fatal("not a template")
} }
mustPanic(t, "unheld arm with a path", func() { mustPanic(t, "unheld arm with a path", func() {
readField(engine(1).rand, tm, nil, nil, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) readField(engine(1).rand, tm, nil, drawScope{}, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"})
}) })
} }
inline.go
+6 -2
View File
@@ -20,7 +20,7 @@ type Template struct {
func (t *Template) Fake() string { func (t *Template) Fake() string {
t.g.mu.Lock() t.g.mu.Lock()
defer t.g.mu.Unlock() defer t.g.mu.Unlock()
return render(t.g.rand, t.n, nil) return renderOnce(t.g.rand, t.n)
} }
// NewTemplate compiles an inline template — a format string or a JSON value — and // NewTemplate compiles an inline template — a format string or a JSON value — and
@@ -136,6 +136,9 @@ func inputValue(input string) (any, error) {
// bindInline links an inline node's references against root and runs check over it, naming its // bindInline links an inline node's references against root and runs check over it, naming its
// nodes from label. // nodes from label.
func bindInline(n node, label string, root map[string]node, check func(nodeScope) error) error { func bindInline(n node, label string, root map[string]node, check func(nodeScope) error) error {
if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" {
return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup)
}
scope := inlineScope(n, label) scope := inlineScope(n, label)
if err := linkNodeRefs(scope, root); err != nil { if err := linkNodeRefs(scope, root); err != nil {
return err return err
@@ -151,6 +154,7 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error {
if !ok { if !ok {
return nil return nil
} }
t.keyDrawGroup("")
for _, name := range refTokens(t.format) { for _, name := range refTokens(t.format) {
sigil, rest, err := refShape(name) sigil, rest, err := refShape(name)
if err != nil { if err != nil {
@@ -160,6 +164,6 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error {
return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest) return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest)
} }
} }
return linkTemplateRefs(nil, path, t, root) return linkTemplateRefs(nil, path, "", t, root)
}) })
} }
inline_test.go
+2 -1
View File
@@ -108,8 +108,9 @@ func TestFakeTemplateErrors(t *testing.T) {
{`name: {/no.such.path}`, "no entry"}, {`name: {/no.such.path}`, "no entry"},
{`name: {..nope}`, "write {/nope}"}, {`name: {..nope}`, "write {/nope}"},
{`{"format":"x"}`, "is a string"}, {`{"format":"x"}`, "is a string"},
{`{/misc.country} {/misc.country.alpha2}`, "renders a level"}, {`{/misc.country} {/misc.country.alpha2}`, "renders a level that {/misc.country.alpha2} reads a path into; name the fields you want instead, or draw them apart with a drawGroup"},
{`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"}, {`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"},
{`{"format":"{/misc.country.alpha2}","drawGroup":"g"}`, "nothing can reference"},
} { } {
_, err := f.FakeTemplate(c.input) _, err := f.FakeTemplate(c.input)
if err == nil || !strings.Contains(err.Error(), c.want) { if err == nil || !strings.Contains(err.Error(), c.want) {
node.go
+19 -10
View File
@@ -13,12 +13,12 @@ import (
// re-sums weights. // re-sums weights.
type node interface{ isNode() } type node interface{ isNode() }
// group is a namespace of named children, built from a directory of JSON files // folder is a namespace of named children, built from a directory of JSON files
// and subdirectories. It has no value of its own: descend into a named child by // and subdirectories. It has no value of its own: descend into a named child by
// dot path; rendering one is an error (see Fake). // dot path; rendering one is an error (see Fake).
type group struct{ children map[string]node } type folder struct{ children map[string]node }
func (*group) isNode() {} func (*folder) isNode() {}
// choice picks one of its items. cum holds cumulative weights for a weighted // choice picks one of its items. cum holds cumulative weights for a weighted
// pick; when nil the choice is uniform and selection is O(1). shared is the set of // pick; when nil the choice is uniform and selection is O(1). shared is the set of
@@ -58,10 +58,12 @@ type template struct {
bound map[string]string bound map[string]string
// held is every name drawn once per expansion: the bound levels above, plus the // held is every name drawn once per expansion: the bound levels above, plus the
// siblings a {calc()} reads. nil when the format holds nothing (see expand). // siblings a {calc()} reads. nil when the format holds nothing (see expand).
held map[string]bool held map[string]bool
fromString bool // written as a JSON string rather than an object fromString bool // written as a JSON string rather than an object
readsColumn *columnRead // set when the format is one reference alone reading a record's column readsColumn *columnRead // set when the format is one reference alone reading a record's column
record bool // compiled at the top without a repeat, so its fields are record columns record bool // compiled at the top without a repeat, so its fields are record columns
drawGroup string // the draw group it draws in, as written; "" keeps its caller's
drawGroupKey string // its draw group keyed by its category once linked: what a render reads its reference paths under
} }
func (*template) isNode() {} func (*template) isNode() {}
@@ -241,13 +243,16 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString { if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" {
return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format) return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format)
} }
if err := checkTokens(o.format, fields); err != nil { if err := checkTokens(o.format, fields); err != nil {
return nil, err return nil, err
} }
t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, record: fieldPos == inColumn} if err := checkNestedDrawGroup(fields, o.group); err != nil {
return nil, err
}
t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn}
if err := t.compileFormat(); err != nil { if err := t.compileFormat(); err != nil {
return nil, err return nil, err
} }
@@ -258,6 +263,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
type templateOptions struct { type templateOptions struct {
datatype DataType datatype DataType
format string format string
group string
repeat int repeat int
separator string separator string
weighted bool weighted bool
@@ -278,6 +284,9 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) {
if o.datatype, err = datatypeOf(m, pos); err != nil { if o.datatype, err = datatypeOf(m, pos); err != nil {
return o, err return o, err
} }
if o.group, err = drawGroupOf(m, repeat); err != nil {
return o, err
}
if sv, ok := m["separator"]; ok { if sv, ok := m["separator"]; ok {
if o.separator, ok = sv.(string); !ok { if o.separator, ok = sv.(string); !ok {
return o, fmt.Errorf("separator must be a string, got %T", sv) return o, fmt.Errorf("separator must be a string, got %T", sv)
@@ -410,7 +419,7 @@ func checkPathNames(path string) error {
// field. These names can never be fields. // field. These names can never be fields.
func isOption(name string) bool { func isOption(name string) bool {
switch name { switch name {
case "datatype", "format", "repeat", "separator", "weight": case "datatype", "drawGroup", "format", "repeat", "separator", "weight":
return true return true
} }
return false return false
one_spelling_test.go
+35 -8
View File
@@ -31,25 +31,52 @@ func TestRepeatedChoiceItemIsRejected(t *testing.T) {
func TestInertObjectIsRejected(t *testing.T) { func TestInertObjectIsRejected(t *testing.T) {
for src, want := range map[string]string{ for src, want := range map[string]string{
`{"format":"Malmö"}`: `write "Malmö"`, `{"format":"Malmö"}`: `write "Malmö"`,
`{"format":"{digits(3)}"}`: `write "{digits(3)}"`, `{"format":"{digits(3)}"}`: `write "{digits(3)}"`,
`[{"format":"a","weight":1},"b"]`: "weight 1", `[{"format":"a","weight":1},"b"]`: "weight 1",
`{"format":"{x}","x":"v","repeat":1}`: "repeat 1", `{"format":"{x}","x":"v","repeat":1}`: "repeat 1",
`{"format":"{x}","x":"v","separator":","}`: "separator", `{"format":"{x}","x":"v","separator":","}`: "separator",
`{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default", `{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default",
`{"format":"","n":{"format":"1","datatype":"string"}}`: `datatype "string" is the default`, `{"format":"","n":{"format":"1","datatype":"string"}}`: `datatype "string" is the default`,
`{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":""}}`: `drawGroup "" is the default`,
`{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":1}}`: "drawGroup must be a string",
} { } {
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want) t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want)
} }
} }
for _, ok := range []string{`[{"format":"a","weight":2},"b"]`, `{"format":"ab","repeat":2}`, `{"format":"{x}","x":"v"}`, `"{digits(3)}"`} { for _, ok := range []string{`[{"format":"a","weight":2},"b"]`, `{"format":"ab","repeat":2}`, `{"format":"{x}","x":"v"}`, `"{digits(3)}"`, `{"format":"{/cat.x}","drawGroup":"g"}`} {
if _, err := compile(parse(t, ok)); err != nil { if _, err := compile(parse(t, ok)); err != nil {
t.Errorf("compile(%s) = %v", ok, err) t.Errorf("compile(%s) = %v", ok, err)
} }
} }
} }
func TestAGroupThatSplitsNothingIsRejected(t *testing.T) {
files := map[string]string{"mail": `"{/word.w}@example.com"`, "word": `{"format":"{w}","w":["a","b"]}`}
for src, want := range map[string]string{
`{"format":"{x}","x":{"format":"{y}","y":["a","b"],"drawGroup":"g"}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{/word}","drawGroup":"g"}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{r}","drawGroup":"g","r":{"format":"{/word.w}","repeat":2}}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{y}","drawGroup":"g","y":{"format":"{/word.w}","drawGroup":"h"}}}`: `drawGroup "g" splits nothing`,
`{"format":"{x}","x":{"format":"{y}","y":"{/word.w}","repeat":2,"drawGroup":"g"}}`: `drawGroup "g" on a repeat`,
`{"format":"{x}","x":{"format":"{/word.w} {y}","drawGroup":"g","y":{"format":"{/word.w}!","drawGroup":"g"}}}`: `"y" names drawGroup "g", the draw group this template draws in already`,
`{"format":"{/word.w}","drawGroup":"g"}`: "",
`{"format":"{x}","x":{"format":"{/mail}","drawGroup":"g"}}`: "",
`{"format":"{x}","x":{"format":"{/word.w} {y}","drawGroup":"g","y":{"format":"{/word.w}!","drawGroup":"h"}}}`: "",
`{"format":"{x}","x":{"format":"{/word.w} {r}","drawGroup":"g","r":{"format":"{y}","repeat":2,"y":{"format":"{/word.w}","drawGroup":"g"}}}}`: "",
} {
files["cat"] = src
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, files)))
switch {
case want == "" && err != nil:
t.Errorf("New(%s) = %v, want a group over a reference path accepted, however deep", src, err)
case want != "" && (err == nil || !strings.Contains(err.Error(), want)):
t.Errorf("New(%s) = %v, want an error mentioning %s", src, err, want)
}
}
}
func TestInlineFolderSigilsAreRejected(t *testing.T) { func TestInlineFolderSigilsAreRejected(t *testing.T) {
f := shipped(t) f := shipped(t)
for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} { for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} {
path.go
+11 -7
View File
@@ -16,7 +16,7 @@ type pathWalk struct {
leaf func(n node) error leaf func(n node) error
} }
// walkPath descends tail from n: a group or template by its next segment, a // walkPath descends tail from n: a folder or template by its next segment, a
// choice by w.choice, which consumes no segment. A missing segment is an error, // choice by w.choice, which consumes no segment. A missing segment is an error,
// so no walk reaches past what the data holds. A table-shaped dispatch, one case // so no walk reaches past what the data holds. A table-shaped dispatch, one case
// per node kind, kept whole on purpose. // per node kind, kept whole on purpose.
@@ -28,7 +28,7 @@ func walkPath(n node, tail []string, w pathWalk) error {
return nil return nil
} }
switch n := n.(type) { switch n := n.(type) {
case *group: case *folder:
child, ok := n.children[tail[0]] child, ok := n.children[tail[0]]
if !ok { if !ok {
return fmt.Errorf("no entry %q", tail[0]) return fmt.Errorf("no entry %q", tail[0])
@@ -89,9 +89,9 @@ func unreachableInChoice(c *choice, want string) error {
// checkPath proves a dotted tail resolves whichever way the draws go — a choice // checkPath proves a dotted tail resolves whichever way the draws go — a choice
// must carry the rest of the path in the set every variant shares — and that no // must carry the rest of the path in the set every variant shares — and that no
// level a path reads carries a repeat, which one draw of it could not apply. So a // level a path reads carries a repeat or a drawGroup, which one draw of it could not
// path that validates here resolves on every render, and a typo is a New-time // apply. So a path that validates here resolves on every render, and a typo is a
// error. // New-time error.
func checkPath(n node, tail []string, level string) error { func checkPath(n node, tail []string, level string) error {
return walkPath(n, tail, pathWalk{ return walkPath(n, tail, pathWalk{
choice: func(c *choice, rest []string) ([]node, error) { choice: func(c *choice, rest []string) ([]node, error) {
@@ -101,8 +101,12 @@ func checkPath(n node, tail []string, level string) error {
return c.items, nil return c.items, nil
}, },
level: func(t *template, rest []string) error { level: func(t *template, rest []string) error {
if t.repeat > 1 { name := join(level, strings.Join(tail[:len(tail)-len(rest)], "."))
return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", join(level, strings.Join(tail[:len(tail)-len(rest)], "."))) switch {
case t.repeat > 1:
return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", name)
case t.drawGroup != "":
return fmt.Errorf("the level %q carries a drawGroup, which a path reading into it cannot apply", name)
} }
return nil return nil
}, },
perf_test.go
+23
View File
@@ -51,6 +51,29 @@ func TestNoRenderAllocRegression(t *testing.T) {
} }
} }
// The repeat shape prices both escape measures: dropping either costs an alloc an iteration.
func TestNoReferenceAllocRegression(t *testing.T) {
word := `{"format":"{w}","w":["alpha","beta","gamma","delta"]}`
for _, s := range []struct {
name, json string
base float64
}{
{"a repeat of a reference path", `{"format":"{r}","r":{"format":"{/word.w}","repeat":20,"separator":", "}}`, 66},
{"a named draw group", `{"format":"{a}","a":{"format":"{/word.w}","drawGroup":"g"}}`, 11},
} {
f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{
"word.json": {Data: []byte(word)},
"x.json": {Data: []byte(s.json)},
}))
if err != nil {
t.Fatalf("New(%s): %v", s.name, err)
}
if allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") }); allocs > s.base*1.10 {
t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a draw set reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base)
}
}
}
// A record's fences read the compiled tree, so they belong to New, not to a draw. // A record's fences read the compiled tree, so they belong to New, not to a draw.
func TestNoRecordAllocRegression(t *testing.T) { func TestNoRecordAllocRegression(t *testing.T) {
for _, s := range []struct{ name, json string }{ for _, s := range []struct{ name, json string }{
record.go
+5 -76
View File
@@ -4,7 +4,6 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"sort"
"strings" "strings"
"unicode" "unicode"
"unicode/utf8" "unicode/utf8"
@@ -23,7 +22,7 @@ type Column struct {
// Record is one record rendered from a template: every direct field is a column, // Record is one record rendered from a template: every direct field is a column,
// listed in name order. Each column is its own expansion, so a sibling field is // listed in name order. Each column is its own expansion, so a sibling field is
// local to it, while a reference that reads a path is drawn once for the whole // local to it, while a reference that reads a path is drawn once for the whole
// record. // record, per group.
type Record struct { type Record struct {
columns []Column columns []Column
} }
@@ -212,9 +211,6 @@ func recordOf(n node) (*template, []Column, error) {
if !t.record { if !t.record {
return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat) return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat)
} }
if err := checkColumnRefs(t, names); err != nil {
return nil, nil, err
}
columns := make([]Column, len(names)) columns := make([]Column, len(names))
for i, name := range names { for i, name := range names {
datatype, _ := columnDatatype(t.fields[name]) // checkColumns refused items that disagree wherever DataType is read datatype, _ := columnDatatype(t.fields[name]) // checkColumns refused items that disagree wherever DataType is read
@@ -223,80 +219,13 @@ func recordOf(n node) (*template, []Column, error) {
return t, columns, nil return t, columns, nil
} }
// checkColumnRefs rejects the reference reads a record's shared draw cannot answer // renderRecord draws each column once, in the name order recordOf fixed, as one render.
// for: one column rendering a level another reads a path into, and a column
// reading the record back through its own path.
func checkColumnRefs(t *template, columns []string) error {
reads, err := columnRefs(t, columns)
if err != nil {
return err
}
sort.Slice(reads, func(i, j int) bool {
if reads[i].a.path != reads[j].a.path {
return reads[i].a.path < reads[j].a.path
}
return reads[i].column < reads[j].column
})
for i, level := range reads {
for _, into := range reads[i+1:] {
if strings.HasPrefix(into.a.path, level.a.path+".") {
return fmt.Errorf("column %q renders {%s}, a level column %q reads a path into with {%s}; name the fields you want instead",
level.column, level.a.name, into.column, into.a.name)
}
}
}
return nil
}
// columnRef is one held reference read, and the column whose render reaches it.
type columnRef struct {
column string
a arm
}
// columnRefs lists every reference that reads a path, anywhere a column renders,
// following the same edges expand does. A reference landing back on the record
// itself is reported rather than collected, whether it reads a path or the record
// whole: either way the column describes a draw other than its neighbours'.
func columnRefs(t *template, columns []string) ([]columnRef, error) {
var out []columnRef
var err error
for _, name := range columns {
seen := map[node]bool{}
var walk func(n node)
walk = func(n node) {
if n == nil || seen[n] || err != nil {
return
}
seen[n] = true
if tm, ok := n.(*template); ok {
for _, ref := range refTokens(tm.format) {
a := splitArm(ref, tm.refs)
if tm.fields[a.key] == node(t) {
err = fmt.Errorf("column %q reads {%s}, which points back at this record; a column cannot read another column — move the shared value into its own category and reference that", name, a.name)
return
}
if len(a.tail) > 0 {
out = append(out, columnRef{name, a})
}
}
}
for _, e := range renderEdges(n) {
walk(e.to)
}
}
walk(t.fields[name])
}
return out, err
}
// renderRecord draws each column once, in the name order recordOf fixed, over one
// reference scope shared across them.
func renderRecord(s *session, t *template, columns []Column) *Record { func renderRecord(s *session, t *template, columns []Column) *Record {
scope := &draws{variant: map[string]node{}, value: map[string]draw{}} set := newDrawSet()
sc := drawScope{set: &set}.in(t)
r := &Record{columns: append([]Column(nil), columns...)} r := &Record{columns: append([]Column(nil), columns...)}
for i := range r.columns { for i := range r.columns {
column := renderLeaf(s, t.fields[r.columns[i].Name], scope) column := renderLeaf(s, t.fields[r.columns[i].Name], sc)
r.columns[i].Value, r.columns[i].Null = column.text, column.null r.columns[i].Value, r.columns[i].Null = column.text, column.null
} }
return r return r
record_test.go
+69 -41
View File
@@ -200,26 +200,33 @@ func TestRecordWritesTypedAndNullColumns(t *testing.T) {
} }
func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) { func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) {
cat := `{"format":"","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}` cat := `{"format":"{a}","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}`
mid := `"{/cat.a}"`
for _, c := range []struct{ name, row string }{
{"through a column repeat, which draws anew", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`},
{"in a group of its own", `{"format":"","whole":{"format":"{/cat.a}","drawGroup":"g"},"inner":"{/cat.a.b}"}`},
} {
f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}), WithSeed(1))
if _, err := f.FakeRecord("row"); err != nil {
t.Errorf("%s: FakeRecord = %v, want it accepted", c.name, err)
}
}
for _, c := range []struct{ name, row string }{ for _, c := range []struct{ name, row string }{
{"sibling columns", `{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`}, {"sibling columns", `{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`},
{"through a nested template", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b} {x}","x":"1"}}`}, {"through a nested template", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b} {x}","x":"1"}}`},
{"through a column repeat", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`},
{"through a choice variant", `{"format":"","whole":"{/cat.a}","inner":[{"format":"{/cat.a.b} {x}","x":"1"},{"format":"{/cat.a.b}! {x}","x":"2"}]}`}, {"through a choice variant", `{"format":"","whole":"{/cat.a}","inner":[{"format":"{/cat.a.b} {x}","x":"1"},{"format":"{/cat.a.b}! {x}","x":"2"}]}`},
{"as a builtin operand", `{"format":"","whole":"{uppercase(/cat.a)}","inner":"{/cat.a.b}"}`}, {"as a builtin operand", `{"format":"","whole":"{uppercase(/cat.a)}","inner":"{/cat.a.b}"}`},
{"a bare reference beside a path", `{"format":"","whole":"{/cat}","inner":"{/cat.a.b}"}`},
{"a column reaching back through another category", `{"format":"","whole":"{/mid}","inner":"{/cat.a.b}"}`},
} { } {
f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "row": c.row}), WithSeed(1)) _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row})))
_, err := f.FakeRecord("row")
if err == nil || !strings.Contains(err.Error(), "reads a path into") { if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: FakeRecord = %v, want the overlap rejected the way one format is", c.name, err) t.Errorf("%s: New = %v, want the overlap refused at load, the way one format is", c.name, err)
continue continue
} }
if !strings.Contains(err.Error(), `"whole"`) || !strings.Contains(err.Error(), `"inner"`) { if !strings.Contains(err.Error(), `"whole"`) || !strings.Contains(err.Error(), `"inner"`) {
t.Errorf("%s: error %q names neither column; it must name both", c.name, err) t.Errorf("%s: error %q names neither column; it must name both", c.name, err)
} }
if _, err := f.Fake("row"); err != nil {
t.Errorf("%s: Fake(row) = %v, want the string view untouched", c.name, err)
}
} }
} }
@@ -241,9 +248,9 @@ func TestRecordRejectsAColumnReadingItsOwnRecord(t *testing.T) {
{"the record as an operand", `"up":"{uppercase(/person)}"`}, {"the record as an operand", `"up":"{uppercase(/person)}"`},
} { } {
person := `{"format":"{first} {last}","first":["Ada","Bo"],"last":["Lovelace","Ek"],` + c.column + `}` person := `{"format":"{first} {last}","first":["Ada","Bo"],"last":["Lovelace","Ek"],` + c.column + `}`
f := newGenerator(t, writeData(t, map[string]string{"person": person}), WithSeed(1)) _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"person": person})))
if _, err := f.FakeRecord("person"); err == nil || !strings.Contains(err.Error(), "points back at this record") { if err == nil || !strings.Contains(err.Error(), "names the category it sits in") {
t.Errorf("%s: FakeRecord = %v, want it refused; the column would contradict the columns beside it", c.name, err) t.Errorf("%s: New = %v, want it refused at load; the column would contradict the columns beside it", c.name, err)
} }
} }
} }
@@ -311,9 +318,10 @@ func TestRecordRejectsFieldDescent(t *testing.T) {
func TestRecordSharesAReferenceAcrossColumns(t *testing.T) { func TestRecordSharesAReferenceAcrossColumns(t *testing.T) {
dir := writeData(t, map[string]string{ dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD","symbol":"$"},{"format":"{code}","code":"EUR","symbol":"€"}]`, "currency": `[{"format":"{code}","code":"AUD","symbol":"$"},{"format":"{code}","code":"EUR","symbol":"€"}]`,
"price": `{"format":"","code":"{/currency.code}","symbol":"{/currency.symbol}"}`, "price": `{"format":"{code} {symbol}","code":"{/currency.code}","symbol":"{/currency.symbol}"}`,
}) })
f := newGenerator(t, dir, WithSeed(1)) f := newGenerator(t, dir, WithSeed(1))
symbols := map[string]string{"AUD": "$", "EUR": "€"}
for i := 0; i < 100; i++ { for i := 0; i < 100; i++ {
r, err := f.FakeRecord("price") r, err := f.FakeRecord("price")
if err != nil { if err != nil {
@@ -323,48 +331,63 @@ func TestRecordSharesAReferenceAcrossColumns(t *testing.T) {
for _, c := range r.Columns() { for _, c := range r.Columns() {
m[c.Name] = c.Value m[c.Name] = c.Value
} }
switch m["code"] { if symbol, known := symbols[m["code"]]; !known || m["symbol"] != symbol {
case "AUD": t.Fatalf("record %s, want one currency draw across columns", r.JSON())
if m["symbol"] != "$" { }
t.Fatalf("record %q: code AUD but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"]) v := fake(t, f, "price")
} if code, symbol, _ := strings.Cut(v, " "); symbol == "" || symbols[code] != symbol {
case "EUR": t.Fatalf("Fake(price) = %q, want its fields one currency draw, as the record's columns are", v)
if m["symbol"] != "€" {
t.Fatalf("record %q: code EUR but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"])
}
default:
t.Fatalf("record %q has unexpected code %q", r.JSON(), m["code"])
} }
} }
} }
func TestRecordSharesAReferenceIntoAColumnRepeat(t *testing.T) { func TestRepeatIterationsDrawReferencesAnew(t *testing.T) {
dir := writeData(t, map[string]string{ dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`, "party": `{"format":"{host}: {guests}","guests":{"format":"{/person.first} {/person.last}","repeat":3,"separator":", "},"host":"{/person.first} {/person.last}"}`,
"order": `{"format":"","codes":{"format":"{/currency.code}","repeat":3,"separator":"-"}}`, "person": drawPeople,
}) })
f := newGenerator(t, dir, WithSeed(1)) f := newGenerator(t, dir, WithSeed(1))
for i := 0; i < 50; i++ { differed := map[[2]int]bool{}
r, err := f.FakeRecord("order") check := func(view string, names []string) {
t.Helper()
if len(names) != 4 {
t.Fatalf("%s rendered %q, want a host and three guests", view, names)
}
for i, name := range names {
if !onePerson(name) {
t.Fatalf("%s rendered %q, want each name one person", view, names)
}
for j := range names[:i] {
differed[[2]int{j, i}] = differed[[2]int{j, i}] || names[j] != name
}
}
}
for i := 0; i < 100; i++ {
r, err := f.FakeRecord("party")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
parts := strings.Split(r.Columns()[0].Value, "-") guests, host := r.Columns()[0].Value, r.Columns()[1].Value
if len(parts) != 3 || parts[0] != parts[1] || parts[1] != parts[2] { check("FakeRecord", append([]string{host}, strings.Split(guests, ", ")...))
t.Fatalf("codes column = %q, want one shared draw across its repeat", r.Columns()[0].Value) host, guests, _ = strings.Cut(fake(t, f, "party"), ": ")
check("Fake", append([]string{host}, strings.Split(guests, ", ")...))
}
for pair, ok := range differed {
if !ok {
t.Errorf("names %v never differed in 200 renders; the host and each repeat iteration are a draw of their own, so four people", pair)
} }
} }
} }
func TestRecordBareReferenceStaysIndependent(t *testing.T) { func TestRecordGroupsDrawApart(t *testing.T) {
dir := writeData(t, map[string]string{ dir := writeData(t, map[string]string{
"currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`, "person": drawPeople,
"order": `{"format":"","whole":"{/currency}","code":"{/currency.code}"}`, "transfer": `{"format":"{from_first} {from_last} to {to_first} {to_last}","from_first":{"format":"{/person.first}","drawGroup":"from"},"from_last":{"format":"{/person.last}","drawGroup":"from"},"to_first":{"format":"{/person.first}","drawGroup":"to"},"to_last":{"format":"{/person.last}","drawGroup":"to"}}`,
}) })
f := newGenerator(t, dir, WithSeed(1)) f := newGenerator(t, dir, WithSeed(1))
sawMismatch := false apart := map[string]bool{}
for i := 0; i < 100; i++ { for i := 0; i < 100; i++ {
r, err := f.FakeRecord("order") r, err := f.FakeRecord("transfer")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -372,13 +395,18 @@ func TestRecordBareReferenceStaysIndependent(t *testing.T) {
for _, c := range r.Columns() { for _, c := range r.Columns() {
m[c.Name] = c.Value m[c.Name] = c.Value
} }
if m["whole"] != m["code"] { from, to, _ := strings.Cut(fake(t, f, "transfer"), " to ")
sawMismatch = true for view, pair := range map[string][2]string{"FakeRecord": {m["from_first"] + " " + m["from_last"], m["to_first"] + " " + m["to_last"]}, "Fake": {from, to}} {
break if !onePerson(pair[0]) || !onePerson(pair[1]) {
t.Fatalf("%s drew %q and %q, want each group one person", view, pair[0], pair[1])
}
apart[view] = apart[view] || pair[0] != pair[1]
} }
} }
if !sawMismatch { for _, view := range []string{"FakeRecord", "Fake"} {
t.Fatal("a bare {/currency} column never disagreed with a tailed {/currency.code} column; a bare reference should draw independently") if !apart[view] {
t.Errorf("%s: groups from and to drew one person in 100 renders, want a draw each", view)
}
} }
} }
reference.go
+21 -15
View File
@@ -72,14 +72,17 @@ func refSegments(name string, folder []string) ([]string, error) {
// error, never a random render-time one. // error, never a random render-time one.
func linkRefs(root map[string]node) error { func linkRefs(root map[string]node) error {
return eachTemplate(root, func(folder []string, path string, t *template) error { return eachTemplate(root, func(folder []string, path string, t *template) error {
return linkTemplateRefs(folder, path, t, root) category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".")
t.keyDrawGroup(category)
return linkTemplateRefs(folder, path, category, t, root)
}) })
} }
// linkTemplateRefs binds one template's references against root. A template with // linkTemplateRefs binds one template's references against root, refusing one that names the
// none is left untouched, so an inline format that references nothing costs only // category it sits in: a category is a unit, and a reference back into it describes a draw other
// the refTokens scan. // than the fields beside it. A template with no reference is left untouched, so an inline format
func linkTemplateRefs(folder []string, path string, t *template, root map[string]node) error { // that references nothing costs only the refTokens scan.
func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error {
names := refTokens(t.format) names := refTokens(t.format)
if len(names) == 0 { if len(names) == 0 {
return nil return nil
@@ -98,6 +101,9 @@ func linkTemplateRefs(folder []string, path string, t *template, root map[string
return fmt.Errorf("%s: reference {%s}: %w", path, name, err) return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
} }
key := "/" + strings.Join(head, ".") key := "/" + strings.Join(head, ".")
if category != "" && key == "/"+category {
return fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name)
}
if err := checkPath(target, tail, key); err != nil { if err := checkPath(target, tail, key); err != nil {
return fmt.Errorf("%s: reference {%s}: %w", path, name, err) return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
} }
@@ -162,17 +168,17 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t
} }
return nil return nil
} }
var inFolder func(folder []string, children map[string]node) error var inFolder func(dir []string, children map[string]node) error
inFolder = func(folder []string, children map[string]node) error { inFolder = func(dir []string, children map[string]node) error {
for _, name := range sortedNames(children) { for _, name := range sortedNames(children) {
path := join(strings.Join(folder, "."), name) path := join(strings.Join(dir, "."), name)
if g, ok := children[name].(*group); ok { if g, ok := children[name].(*folder); ok {
if err := inFolder(append(folder[:len(folder):len(folder)], name), g.children); err != nil { if err := inFolder(append(dir[:len(dir):len(dir)], name), g.children); err != nil {
return err return err
} }
continue continue
} }
if err := inCategory(folder, path, children[name]); err != nil { if err := inCategory(dir, path, children[name]); err != nil {
return err return err
} }
} }
@@ -183,13 +189,13 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t
// resolveCategory walks a dotted path through the folders to the category it // resolveCategory walks a dotted path through the folders to the category it
// names, returning that head, the node, and the tail left to read into it. A // names, returning that head, the node, and the tail left to read into it. A
// descent of its own rather than a walkPath: it walks groups only and returns // descent of its own rather than a walkPath: it walks folders only and returns
// where they end, not a leaf. // where they end, not a leaf.
func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) { func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) {
var n node = &group{children: root} var n node = &folder{children: root}
i := 0 i := 0
for ; i < len(segments); i++ { for ; i < len(segments); i++ {
g, ok := n.(*group) g, ok := n.(*folder)
if !ok { if !ok {
break break
} }
@@ -199,7 +205,7 @@ func resolveCategory(root map[string]node, segments []string) (head []string, ta
} }
n = child n = child
} }
if _, ok := n.(*group); ok { if _, ok := n.(*folder); ok {
return nil, nil, nil, fmt.Errorf("names a folder, not a value") return nil, nil, nil, fmt.Errorf("names a folder, not a value")
} }
return segments[:i], n, segments[i:], nil return segments[:i], n, segments[i:], nil
reference_test.go
+129 -35
View File
@@ -83,27 +83,25 @@ func TestReferenceErrors(t *testing.T) {
"card": `"{/who.f}"`, "card": `"{/who.f}"`,
}, },
"empty reference path": {"card": `"{/}"`}, "empty reference path": {"card": `"{/}"`},
// A reference that leads back to its own value never terminates at render, // A reference that leads back to its own value never terminates at render, so
// so New must reject the cycle up front (direct, mutual, or chained). // New must reject the cycle up front (mutual or chained). One into its own
"direct cycle": {"a": `"x{/a}"`}, // category is refused before the cycle walk reaches it, as a unit rule.
"mutual cycle": {"a": `"{/b}"`, "b": `"{/a}"`}, "a category referencing itself": {"a": `"x{/a}"`},
"chain cycle": {"a": `"{/b}"`, "b": `"{/c}"`, "c": `"{/a}"`}, "mutual cycle": {"a": `"{/b}"`, "b": `"{/a}"`},
// calc renders its operands, so a cycle through one must be caught too. "chain cycle": {"a": `"{/b}"`, "b": `"{/c}"`, "c": `"{/a}"`},
"calc operand cycle": {"x": `{"format":"{calc(y)}","y":"{/x}"}`}, // calc renders its operands, so a reference through one is caught too.
"a calc operand into its own category": {"x": `{"format":"{calc(y)}","y":"{/x}"}`},
// A field its parent's format never renders is still reachable by dot path, // A field its parent's format never renders is still reachable by dot path,
// so a cycle hiding in one must fail at New rather than at render. // so what hides in one must fail at New rather than at render.
"cycle in an unrendered field": {"cat": `{"format":"hi","x":"{/cat.x}"}`}, "an unrendered field into its own category": {"cat": `{"format":"hi","x":"{/cat.x}"}`},
"mutual cycle between unrendered fields": { "two unrendered fields into their own category": {
"cat": `{"format":"hi","x":"{/cat.y}","y":"{/cat.x}"}`, "cat": `{"format":"hi","x":"{/cat.y}","y":"{/cat.x}"}`,
}, },
"cycle in an unrendered field of a choice arm": { // The shipped layout puts categories in folders, so one level down is the
"cat": `{"format":"hi","x":"{/cat.x}"}`, // common case, not an edge case.
}, "a category in a subfolder referencing itself": {"sv_SE/a": `"x{/sv_SE.a}"`},
// The shipped layout puts categories in folders, so a cycle one level down "mutual cycle within a subfolder": {"sv_SE/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/sv_SE.a}"`},
// is the common case, not an edge case. "mutual cycle across two folders": {"en_US/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/en_US.a}"`},
"cycle in a subfolder": {"sv_SE/a": `"x{/sv_SE.a}"`},
"mutual cycle within a subfolder": {"sv_SE/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/sv_SE.a}"`},
"mutual cycle across two folders": {"en_US/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/en_US.a}"`},
// ".." is reserved for bound references, so an authored key using it would // ".." is reserved for bound references, so an authored key using it would
// name a node nothing can reach and nothing would validate. // name a node nothing can reach and nothing would validate.
"field key using the reference prefix": {"cat": `{"format":"hi","..x":"{/nope}"}`}, "field key using the reference prefix": {"cat": `{"format":"hi","..x":"{/nope}"}`},
@@ -131,17 +129,26 @@ func TestDotPrefixedDataEntriesAreSkipped(t *testing.T) {
} }
} }
// TestReferenceFromUnrenderedFieldTerminates guards the cycle check against // TestReferenceIntoItsOwnCategoryIsRejected pins the unit a category is: a reference
// over-rejecting: a field the format never renders may point back at its own // back into it describes a draw other than the fields beside it, whether the format
// category, which terminates, and stays renderable by path. // renders that field or not, so it is refused at load rather than left to disagree in
func TestReferenceFromUnrenderedFieldTerminates(t *testing.T) { // the record view.
dir := writeData(t, map[string]string{"cat": `{"format":"hi","x":"see {/cat}"}`}) func TestReferenceIntoItsOwnCategoryIsRejected(t *testing.T) {
f := newGenerator(t, dir, WithSeed(1)) for name, file := range map[string]string{
if got := fake(t, f, "cat"); got != "hi" { "the category whole": `{"format":"hi","x":"see {/cat}"}`,
t.Fatalf("cat = %q, want hi", got) "a field of its own": `{"format":"hi","x":"{/cat.y}","y":"1"}`,
"a path the format reads": `{"format":"{/cat.y}","y":"1"}`,
} {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file})))
if err == nil || !strings.Contains(err.Error(), "names the category it sits in") {
t.Errorf("%s: New = %v, want the reference into its own category refused", name, err)
}
} }
if got := fake(t, f, "cat.x"); got != "see hi" { // Reading a sibling as a path is the spelling that stays.
t.Fatalf("cat.x = %q, want \"see hi\"", got) if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"cat": `{"format":"{y.v}","y":{"format":"{v}","v":["1","2"]}}`,
}))); err != nil {
t.Errorf("New = %v, want the sibling path accepted", err)
} }
} }
@@ -189,9 +196,9 @@ func TestNewErrorPathIsCanonical(t *testing.T) {
want string want string
}{ }{
{ {
"cycle inside a choice arm", "cycle through another category",
map[string]string{"cat": `{"format":"hi","x":"{/cat.x}"}`}, map[string]string{"cat": `{"format":"hi","x":"{/hop}"}`, "hop": `"{/cat.x}"`},
"fejkdata: reference cycle: cat.x -> /cat.x", "fejkdata: reference cycle: cat.x -> /hop -> /cat.x",
}, },
{ {
"bad reference reached through another reference", "bad reference reached through another reference",
@@ -255,15 +262,102 @@ func TestBareReferenceDrawsEachTime(t *testing.T) {
} }
func TestReferenceOverlapIsRejected(t *testing.T) { func TestReferenceOverlapIsRejected(t *testing.T) {
for name, file := range map[string]string{ // A category never references itself, so the reads sit in a second category.
"head beside a path": `{"format":"{/cat.p} {/cat.p.first}","p":[{"format":"{first}","first":"A"},{"format":"{first}","first":"B"}]}`, cat := `{"format":"x","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`
"sibling path beside a reference path": `{"format":"{p.first} {/cat.p.last}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`, for name, row := range map[string]string{
"head beside a path": `"{/cat.p} {/cat.p.first}"`,
"sibling fields reading a level and a path into it": `{"format":"{a} {b}","a":"{/cat.p}","b":"{/cat.p.first}"}`,
"a field rendering the level a nested reference reads into": `{"format":"{x} {p}","x":"{/cat.p.first}","p":"{/cat.p}"}`,
"a bare reference beside a path into what it never renders": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`,
} { } {
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file}))) _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row})))
if err == nil || !strings.Contains(err.Error(), "reads a path into") { if err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("%s: New = %v, want the overlap rejected", name, err) t.Errorf("%s: New = %v, want the overlap rejected", name, err)
} }
} }
// A sibling path and a reference into the level it holds are the same overlap,
// and the reference reaches it from a category row renders.
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"row": `{"format":"{p.first} {/hop}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`,
"hop": `"{/row.p.last}"`,
}))); err == nil || !strings.Contains(err.Error(), "reads a path into") {
t.Errorf("New = %v, want a sibling path beside a reference into it rejected", err)
}
if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
"cat": cat,
"row": `{"format":"{a} {b}","a":{"format":"{/cat.p}","drawGroup":"g"},"b":"{/cat.p.first}"}`,
}))); err != nil {
t.Errorf("New = %v, want a level and a path into it accepted in groups of their own", err)
}
}
const drawPeople = `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"},{"format":"{first} {last}","first":"Cy","last":"Young"}]`
// onePerson reports whether name is the first name and surname of one drawPeople row.
func onePerson(name string) bool {
first, last, _ := strings.Cut(name, " ")
return last != "" && map[string]string{"Ada": "Lovelace", "Bo": "Ek", "Cy": "Young"}[first] == last
}
func TestAReferencePathIsOneDrawPerRender(t *testing.T) {
dir := writeData(t, map[string]string{
"apart": `{"format":"{a} & {b}","a":{"format":"{/person.first} {/person.last}","drawGroup":"x"},"b":{"format":"{/person.first} {/person.last}","drawGroup":"y"}}`,
"caller": `{"format":"{a} & {b}","a":{"format":"{/person.first} {/person.last}","drawGroup":"x"},"b":"{/pay}"}`,
"contact": `{"format":"{first} {last} <{email}>","email":"{lowercase(/person.first)}.{lowercase(/person.last)}@example.com","first":"{/person.first}","last":"{/person.last}"}`,
"iterations": `{"format":"{/person.first} {r}","drawGroup":"outer","r":{"format":"{a}={b}","repeat":3,"separator":",","a":"{/person.first}","b":{"format":"{/person.first}","drawGroup":"outer"}}}`,
"nested": `{"format":"{/person.first} {inner}","inner":"{/person.last}"}`,
"pair": `{"format":"{a} & {b}","a":"{/person.first} {/person.last}","b":"{/person.first} {/person.last}"}`,
"pay": `{"format":"{p}","p":{"format":"{/person.first} {/person.last}","drawGroup":"x"}}`,
"person": drawPeople,
})
f := newGenerator(t, dir, WithSeed(1))
apart, local, noGroup := false, false, false
for i := 0; i < 100; i++ {
a, b, _ := strings.Cut(fake(t, f, "caller"), " & ")
if !onePerson(a) || !onePerson(b) {
t.Fatalf("caller = %q & %q, want each one person", a, b)
}
local = local || a != b
_, iterations, _ := strings.Cut(fake(t, f, "iterations"), " ")
for _, pair := range strings.Split(iterations, ",") {
a, b, _ := strings.Cut(pair, "=")
noGroup = noGroup || a != b
}
if got := fake(t, f, "nested"); !onePerson(got) {
t.Fatalf("nested = %q, want a nested template's path one draw with its parent's", got)
}
name, email, _ := strings.Cut(fake(t, f, "contact"), " <")
first, last, _ := strings.Cut(name, " ")
if !onePerson(name) || email != strings.ToLower(first)+"."+strings.ToLower(last)+"@example.com>" {
t.Fatalf("contact = %q <%s, want first, last and email one person", name, email)
}
r, err := f.FakeRecord("contact")
if err != nil {
t.Fatal(err)
}
c := r.Columns()
if !onePerson(c[1].Value+" "+c[2].Value) || c[0].Value != strings.ToLower(c[1].Value)+"."+strings.ToLower(c[2].Value)+"@example.com" {
t.Fatalf("contact record %s, want first, last and email one person", r.JSON())
}
a, b, _ = strings.Cut(fake(t, f, "pair"), " & ")
if !onePerson(a) || a != b {
t.Fatalf("pair = %q & %q, want both fields one person", a, b)
}
a, b, _ = strings.Cut(fake(t, f, "apart"), " & ")
if !onePerson(a) || !onePerson(b) {
t.Fatalf("apart = %q & %q, want each group one person", a, b)
}
apart = apart || a != b
}
if !apart {
t.Error("groups x and y drew one person in 100 renders, want a draw each")
}
if !local {
t.Error("group x in caller and group x in the pay it references drew one person in 100 renders; a group name is local to its category")
}
if !noGroup {
t.Error("an iteration's plain read and its group outer always agreed; a repeat iteration renders in no group, so they are a draw each")
}
} }
func TestRelativeReferences(t *testing.T) { func TestRelativeReferences(t *testing.T) {
render.go
+23 -13
View File
@@ -20,14 +20,14 @@ type rng interface {
func (f *Generator) Fake(path string) (string, error) { func (f *Generator) Fake(path string) (string, error) {
f.mu.Lock() f.mu.Lock()
defer f.mu.Unlock() defer f.mu.Unlock()
n, err := descend(f.rand, &group{children: f.categories}, strings.Split(path, ".")) n, err := descend(f.rand, &folder{children: f.categories}, strings.Split(path, "."))
if err != nil { if err != nil {
return "", fmt.Errorf("fejkdata: %s: %w", path, err) return "", fmt.Errorf("fejkdata: %s: %w", path, err)
} }
if _, ok := n.(*group); ok { if _, ok := n.(*folder); ok {
return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path)
} }
return render(f.rand, n, nil), nil return renderOnce(f.rand, n), nil
} }
// descend walks named fields to the node a path names. It is the one render-side // descend walks named fields to the node a path names. It is the one render-side
@@ -50,20 +50,21 @@ func descend(s *session, root node, segments []string) (node, error) {
} }
// render evaluates a compiled node to a string. compile validates every node up // render evaluates a compiled node to a string. compile validates every node up
// front, so rendering a compiled tree cannot fail. refScope carries the draws a // front, so rendering a compiled tree cannot fail. sc holds the reference draws the
// reference shares beyond its own expansion; nil keeps every reference local. // render shares; each repeat iteration renders over draws of its own.
func render(s *session, n node, refScope *draws) string { func render(s *session, n node, sc drawScope) string {
switch n := n.(type) { switch n := n.(type) {
case *choice: case *choice:
return render(s, pick(s, n), refScope) return render(s, pick(s, n), sc)
case *null: case *null:
return "" return ""
case *template: case *template:
sc = sc.in(n)
if n.repeat == 1 { if n.repeat == 1 {
if n.fixed { if n.fixed {
return n.lit return n.lit
} }
return expand(s, n, refScope) return expand(s, n, sc)
} }
var b strings.Builder var b strings.Builder
b.Grow(n.repeat * (n.grow + len(n.separator))) b.Grow(n.repeat * (n.grow + len(n.separator)))
@@ -71,7 +72,7 @@ func render(s *session, n node, refScope *draws) string {
if i > 0 { if i > 0 {
b.WriteString(n.separator) b.WriteString(n.separator)
} }
b.WriteString(expand(s, n, refScope)) b.WriteString(expandAnew(s, n))
} }
return b.String() return b.String()
default: default:
@@ -79,6 +80,15 @@ func render(s *session, n node, refScope *draws) string {
} }
} }
// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into
// render's loop, its draw set would move to the heap.
//
//go:noinline
func expandAnew(s *session, t *template) string {
var set drawSet
return expand(s, t, drawScope{set: &set})
}
// pick selects one item. Uniform choices are O(1); weighted choices are an // pick selects one item. Uniform choices are O(1); weighted choices are an
// O(log n) search over precomputed cumulative weights. compile guarantees a // O(log n) search over precomputed cumulative weights. compile guarantees a
// non-empty choice and a finite positive total, so the index is always in range. // non-empty choice and a finite positive total, so the index is always in range.
@@ -93,12 +103,12 @@ func pick(r rng, c *choice) node {
// expand renders a template's compiled ops. compile validated every token, so this // expand renders a template's compiled ops. compile validated every token, so this
// cannot fail. // cannot fail.
func expand(s *session, t *template, refScope *draws) string { func expand(s *session, t *template, sc drawScope) string {
var b strings.Builder var b strings.Builder
b.Grow(t.grow) b.Grow(t.grow)
// One draw per held name, for this expansion only: a nested template and each // One draw per held name, for this expansion only: a nested template and each
// repeat iteration get their own, since each is its own expansion. A reference // repeat iteration get their own, since each is its own expansion. A reference
// reads the caller's scope instead, whenever one was supplied. // path reads the render's draws in sc instead.
var held *draws var held *draws
if len(t.held) > 0 { if len(t.held) > 0 {
held = &draws{ held = &draws{
@@ -112,7 +122,7 @@ func expand(s *session, t *template, refScope *draws) string {
case 'l': case 'l':
b.WriteString(o.lit) b.WriteString(o.lit)
case 'f': case 'f':
b.WriteString(readField(s, t, held, refScope, o.arms[s.IntN(len(o.arms))]).text) b.WriteString(readField(s, t, held, sc, o.arms[s.IntN(len(o.arms))]).text)
case 'b': case 'b':
// Read before the call, so the value a calc computes is the value the // Read before the call, so the value a calc computes is the value the
// format showed. calcVars fixed the order op.operands holds. // format showed. calcVars fixed the order op.operands holds.
@@ -120,7 +130,7 @@ func expand(s *session, t *template, refScope *draws) string {
if len(o.operands) > 0 { if len(o.operands) > 0 {
operands = make([]string, len(o.operands)) operands = make([]string, len(o.operands))
for j, a := range o.operands { for j, a := range o.operands {
operands[j] = readField(s, t, held, refScope, a).text operands[j] = readField(s, t, held, sc, a).text
} }
} }
b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far
template_test.go
+2 -2
View File
@@ -38,7 +38,7 @@ func compiled(t *testing.T, s string) node {
func mustRender(t *testing.T, f *Generator, s string) string { func mustRender(t *testing.T, f *Generator, s string) string {
t.Helper() t.Helper()
return render(f.rand, compiled(t, s), nil) return renderOnce(f.rand, compiled(t, s))
} }
func TestStringIsAFormat(t *testing.T) { func TestStringIsAFormat(t *testing.T) {
@@ -315,7 +315,7 @@ func TestGrowIsALowerBound(t *testing.T) {
t.Fatalf("format %q did not compile to a template", format) t.Fatalf("format %q did not compile to a template", format)
} }
for i := 0; i < 50; i++ { for i := 0; i < 50; i++ {
if got := len(expand(f.rand, tmpl, nil)); got < tmpl.grow { if got := len(expand(f.rand, tmpl, drawScope{set: &drawSet{}})); got < tmpl.grow {
t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow)
} }
} }
todo.md
-16
View File
@@ -2,22 +2,6 @@
## Before v0.1.0 ## Before v0.1.0
The record API lands first, so the data update can use it.
### Record API
- Draw groups — references into one category share one draw per render (one
record, or one `Fake`) in both views; each `repeat` iteration draws anew, and a
bare reference draws each time. An option naming a draw group splits a render
into several entities. Replaces the Decision "A record shares one reference draw
per category". Each expectation becomes a test:
- `first`, `last` and `email` reading `person` → one person
- `from_first`/`from_last` grouped `from`, `to_first`/`to_last` grouped `to` → two people
- `host`, plus `guests` repeated 3 times → four people
- `code` and `symbol` sibling fields reading `currency`, as `{code} {symbol}` → a matching pair
- `{a} & {b}`, each reading `person` → one person, or two when `a` and `b` name different groups
- two bare `{/sv_SE.word}` → two words
### Data ### Data
- Major data update. Shipped categories render as records with their building - Major data update. Shipped categories render as records with their building