diff --git a/README.md b/README.md index de34fa8..3a11981 100644 --- a/README.md +++ b/README.md @@ -124,16 +124,17 @@ string and a backslash as an escape. A record written only to emit columns still needs a `format` — the grammar's one required key — so `"format": ""` carries the fields with an inert format: it renders nothing by `Fake`, and is compiled only so the tree's fences still run. -The columns are the point, and their facts stay together: two columns that read a -path into one category — `{/currency.code}` and `{/currency.symbol}` — share one -draw of it, so the record is internally consistent. A bare `{/currency}` names no -field, so it keeps drawing on its own. That one draw is also why two -columns may not read overlapping reference *paths* — `{/cat.a}` beside +The columns are the point, and their facts stay together: a record is one render, so +columns that read a path into one category — `{/currency.code}` and +`{/currency.symbol}` — read one draw of it ([References](#references)), and a +[draw group](#draw-group) draws a column apart. That one draw is also why the columns of one +record may not overlap — `{/cat.a}`, or a bare `{/cat}`, beside `{/cat.a.b}` is refused, naming the fields to write instead, as [One draw, one spelling](#one-draw-one-spelling) refuses that pair inside a single -format. A column may not reference the record it belongs to by any spelling: `{/users.first}` -or a bare `{/users}` inside `users` describes a draw other than the columns beside -it, so put a value two columns share in its own category and reference that. A field hold, transform or +format. Both fences run at load, so a category that loads renders as either shape. A +category never references itself — `{/users.first}` or a bare `{/users}` inside `users` +describes a draw other than the fields beside it — so read a sibling as a path, and put +a value two fields share in its own category and reference that. A field hold, transform or operand ties fields together within one column as always (see [Correlated fields](#correlated-fields) and [Decisions](#decisions)). @@ -341,8 +342,8 @@ different datatypes. ### Options and fields -`format`, `weight`, `repeat`, `separator` and `datatype` are the only options; **any -other key is a field** (see [Decisions](#decisions)). An object that does nothing a +`format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options; +**any other key is a field** (see [Decisions](#decisions)). An object that does nothing a string can't — only a `format` — is rejected naming the string, as is a one-item choice naming its item. @@ -438,14 +439,41 @@ without naming `sv_SE`: "Hej, {/en_US.person}!" ``` -Renders e.g. `Hej, Pat Smith!`. A reference into a category is held like a -[correlated](#correlated-fields) path — `{.person.femalefirst} {.person.last}` name -one person, `{lowercase(.person.femalefirst)}` reads that same draw, and -`{.person.femalefirst}` beside `{/sv_SE.person.last}` in `sv_SE` is one person too — while a bare -`{/misc.uuid} {/misc.uuid}` is two draws. Rejected at `New`: a path that is -unknown, names a folder, has no folder above, or reads a field not every variant -of a choice carries, and a reference that leads back to its own value, directly, -mutually or through a chain. +Renders e.g. `Hej, Pat Smith!`. A reference path into a category is held like a +[correlated](#correlated-fields) path, but for the whole render — one `Fake`, or one +record — rather than one format: `{.person.femalefirst} {.person.last}` name one +person, as do the same two references in sibling fields or a nested template, and +`{lowercase(.person.femalefirst)}` reads that same draw. Each `repeat` iteration is +a render of its own, in no group, so it draws anew, and a [draw group](#draw-group) holds a +draw apart. A bare reference names no field and makes its own picks each time — +`{/misc.uuid} {/misc.uuid}` is two draws — while the reference paths inside what it +renders still read the render's draws. Rejected at `New`: a path that is +unknown, names a folder, has no folder above, reads a field not every variant +of a choice carries, or names the category the reference sits in, and a reference +that leads back to its own value, directly, mutually or through a chain. + +### Draw group + +A template may carry `drawGroup` to hold its reference draws apart: every reference path +it renders, however deep short of a `repeat` or a nested `drawGroup`, reads the draw of +that group, and the templates of one category naming one group in one render read one +draw. A name is local to its category, so a category another one references never joins +its groups by name; the unnamed group spans them all. + +```json +{ "format": "{payer} pays {payee}; signed {signature}", + "payer": { "format": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}", "drawGroup": "payer" }, + "payee": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}", + "signature": { "format": "{/sv_SE.person.last}", "drawGroup": "payer" } } +``` + +Renders e.g. `Sara Eriksson pays Ebba Lind; signed Eriksson`: the signature reads the +payer's draw, while the payee is drawn apart. Rejected at load, each naming nothing: a +`drawGroup` of `""` (the default); one naming the draw group its template already draws +in; one on a template that renders no reference path short of a `repeat` or a nested +`drawGroup`, on a `repeat` itself — each iteration renders in no draw group — or on an +inline template's root, which nothing references. So is a path reading into a level that +carries a `drawGroup`. ### Correlated fields @@ -478,12 +506,14 @@ The sub-fields stay addressable — `Fake("address.place.locality")` renders, an ### One draw, one spelling A name any token reads as a path (`{p.first}`) or as an operand (`{calc(net * 2)}`, -`{uppercase(w)}`) is drawn **once per expansion**, and every other route to it — -a bare `{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering -`{/cat.p.last}`, at any depth — is a load error naming the spelling to use. A -name nothing reads that way is drawn each time: `{word} {word}` differs. An -expansion is one render of one format, so each `repeat` iteration and each nested -template draws again. +`{uppercase(w)}`) is drawn **once per expansion**, a reference path (`{/cat.p.first}`) +**once per render** in its [draw group](#draw-group), and every other route to either — a bare +`{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering `{/cat.p.last}` +beside `{p.first}`, a bare `{/cat}` beside `{/cat.p.first}`, at any +depth — is a load error naming the spelling to use. A name nothing reads that way is +drawn each time: `{word} {word}` differs. An expansion is one render of one format, so +each nested template draws its own names again; a render is one `Fake` or one record, +and each `repeat` iteration is an expansion and a render of its own. ```text token {p} renders a level that {p.first} reads a path into; name the fields you want instead @@ -492,7 +522,9 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan ### Performance -Each file is parsed, validated and weight-indexed once, in `New`. A `Fake` call +Each file is parsed, validated and weight-indexed once, in `New`. Proving the draw +fences adds one pass over the loaded tree, and walks what a render reads only where +data binds a reference, so a set that binds none pays for the pass alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one O(log n), and long formats, deep nesting and many tokens add cost in proportion to the output. @@ -524,7 +556,7 @@ tokens add cost in proportion to the output. ## Decisions - **Options and fields share one namespace.** `format`, `weight`, `repeat`, - `separator` and `datatype` are reserved; every other key is a field. Nesting fields under a + `separator`, `datatype` and `drawGroup` are reserved; every other key is a field. Nesting fields under a key, or prefixing options, would tax every template to guard against a misspelt option. - **`{a|b}` stays beside nested choices.** `[[…], […]]` picks the same way, but @@ -571,10 +603,12 @@ tokens add cost in proportion to the output. to have would make `--seed 42` machine-dependent. Data still lives in `data/` as JSON; `--data-path` layers over it. - **A bare reference draws each time; a reference path is held.** `{/p} {/p}` - is two draws, as `{word} {word}` is, while `{/p.first}` beside a nested template - rendering `{/p.first}` is a load error: a bare token is by contract an - independent draw, a path pins its level, and any route into a pinned level from - another expansion could show another row. + is two draws, as `{word} {word}` is, while every `{/p.first}` in one render reads + one draw, and a bare `{/p}` beside them is a load error: a bare token + is by contract an independent draw, a path pins its level, and a fresh draw of a + pinned level could show another row. A builtin's operand holds what it reads for + its expansion, references included, so `{uppercase(/p)} {/p}` is one draw — the + rule every operand follows. - **Reference sigils follow the filesystem.** `/` is the root, `.` this file's folder, `..` the folder above — what those spellings already mean to anyone who has typed a path. A locale's files reach each other without naming the locale, @@ -640,24 +674,44 @@ tokens add cost in proportion to the output. see a caller's types, so the first `FakeStruct` for a type compiles its tags and the answer, error included, is kept per type: a test's first call is its load, and no `NewStruct` handle is needed, as the cache already compiles once. -- **A record shares one reference draw per category.** Two columns that reference - one category — `{/currency.code}` beside `{/currency.symbol}` — read one draw of - it, so a record's facts agree the way a template's [correlated - fields](#correlated-fields) do. The draw is one per record, so it spans a - column's `repeat` and nested templates too (one record is one coherent unit); - a bare reference — `{/currency}`, no field — stays an independent draw every - time, the rule a format string already follows. Only references share: a sibling - field is local to its own column, so a `first` column does not silently bind to - a `first` in the column next to it. - - The string view of that same template does not share. `Fake` renders each - sibling field as its own expansion, so a `{/currency.code}` field beside a - `{/currency.symbol}` field is two draws and may render `EUR $`; writing both - references in one `format` holds them together, as - [One draw, one spelling](#one-draw-one-spelling) says. The scope is what makes a - row coherent when the columns *are* the output, and there the caller cannot fall - back on one format string. Widening it to every render would change what `Fake` - has emitted since the start, for a correlation a single format already reaches. +- **A render shares one reference draw per category, per group.** Every reference + path into a category in one `Fake`, or one record, reads one draw of it, so a + value's facts agree across its fields, nested templates and columns alike — + `{/currency.code}` in one field and `{/currency.symbol}` in another name one + currency, whichever view renders them. A `repeat` iteration is a render of its + own, since repeating asks for another entity, and a [draw group](#draw-group) names further + entities within one render, so a payer and a payee are two groups over one + `person` rather than two copies of it. Only references share: a sibling field is + local to its own expansion, so a `first` column does not silently bind to a + `first` in the column next to it. +- **A draw group name is local to its category.** A category's groups are its own + entities, so a caller naming a group the same way never joins them by accident, + and renaming a group inside one file changes no render elsewhere. The unnamed + group still spans categories, since facts that belong together across categories + must agree. +- **The expansion hold and the render's draws are two fences.** One proves a sibling + path or an operand is reached only by its readers within an expansion, the other + does the same for reference paths across a render and its draw groups. They pin + different things — an operand pins the value its own render produced and stops at a + reference, a path pins every level it passes through — so one walk would carry both + rules and both scopes anyway, and tell them apart at every step. +- **A record makes its draw maps up front, a `Fake` on its first read.** A record's + columns always read through the render's draws, so making the maps where the set is + declared keeps them on that frame's stack. A `Fake` often reads no reference path at + all, and making them anyway cost about a fifth of the cheapest render, so it makes + them on the first read instead, at two heap allocations for a render that does share + a draw. The allocation gate over a repeat of a reference path and over a named draw + group prices that, and pins the two measures that keep a draw set off the heap. +- **A category never references itself, and a record's fences run at load.** A category + is one unit: a reference back into it — `{/users.first}` inside `users` — describes a + draw other than the fields beside it, so `New` refuses it and the sibling path stays + the one spelling for a field of one's own. A value two fields share goes in its own + category, which both reference. That settled, a record's column fences run at `New` + too, so a category that loads renders as whichever shape is asked for, and a reference + reaching back into a category through another one is refused there as the overlap it + is. Which reads those fences weigh differs on purpose: a record-only template's inert + format renders nothing, so a `drawGroup` on it can never matter and is refused, while + one on a rendering format can matter to a caller that bare-references it. - **A record's column set is fixed before the first draw.** Only a category-level template is a record: a path descending into a field, or naming a folder or a choice, errors. A tail may pass through a choice whose variants carry different @@ -737,6 +791,7 @@ struct.go structs: FakeStruct, fake tags, and a field's Go type as its col inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format hold.go the hold: one draw per expansion for paths and operands, and its fences +draw.go one reference draw per render and group: draw sets, the group option, and its fence reference.go reference sigils, and binding references across the tree graph.go the render graph: edges, cycles, the repeat bound, tree walks builtins.go the {name()} function registry and its implementations diff --git a/data.go b/data.go index ce69695..cf5448d 100644 --- a/data.go +++ b/data.go @@ -28,10 +28,10 @@ func (s dataSource) name(p string) string { } // loadData loads every source into one namespace tree and returns its root -// children. A directory becomes a group; each *.json file in it compiles to a node +// children. A directory becomes a folder; each *.json file in it compiles to a node // keyed by its base name (address.json -> "address"); each subdirectory becomes a -// nested group, so folders turn into dot-path segments. Sources merge left to right: -// matching groups merge by their children, and any other clash is won by the last +// nested folder, so folders turn into dot-path segments. Sources merge left to right: +// matching folders merge by their children, and any other clash is won by the last // source loaded. Once merged, linkRefs binds every reference against the // final tree. func loadData(sources []dataSource) (map[string]node, error) { @@ -74,15 +74,15 @@ func loadData(sources []dataSource) (map[string]node, error) { return root, nil } -// loadDir compiles one directory into a group. fs.ReadDir yields entries sorted +// loadDir compiles one directory into a folder. fs.ReadDir yields entries sorted // by name, so the tree is built deterministically. Empty subdirectories (no JSON // anywhere under them) are skipped rather than added as empty namespaces. -func loadDir(src dataSource, dir string) (*group, error) { +func loadDir(src dataSource, dir string) (*folder, error) { entries, err := fs.ReadDir(src.fsys, dir) if err != nil { return nil, fmt.Errorf("%s: %w", src.name(dir), err) } - g := &group{children: map[string]node{}} + g := &folder{children: map[string]node{}} for _, e := range entries { if strings.HasPrefix(e.Name(), ".") { // hidden: a checkout or an editor's file, never data continue @@ -99,8 +99,8 @@ func loadDir(src dataSource, dir string) (*group, error) { return g, nil } -// loadFolder adds a subdirectory as a nested group, unless nothing under it is data. -func loadFolder(src dataSource, g *group, full, name string) error { +// loadFolder adds a subdirectory as a nested folder, unless nothing under it is data. +func loadFolder(src dataSource, g *folder, full, name string) error { child, err := loadDir(src, full) if err != nil { return err @@ -117,7 +117,7 @@ func loadFolder(src dataSource, g *group, full, name string) error { // loadFile compiles a *.json file into a category named after it; any other file // is skipped. -func loadFile(src dataSource, g *group, full, file string) error { +func loadFile(src dataSource, g *folder, full, file string) error { if !strings.HasSuffix(file, ".json") { return nil } @@ -141,13 +141,13 @@ func loadFile(src dataSource, g *group, full, file string) error { return nil } -// mergeChildren overlays src onto dst. Two groups under the same key merge +// mergeChildren overlays src onto dst. Two folders under the same key merge // recursively (so locales/categories from several paths combine); every other // key is replaced, making the last-loaded directory win on a conflict. func mergeChildren(dst, src map[string]node) { for k, v := range src { - if dg, ok := dst[k].(*group); ok { - if sg, ok := v.(*group); ok { + if dg, ok := dst[k].(*folder); ok { + if sg, ok := v.(*folder); ok { mergeChildren(dg.children, sg.children) continue } diff --git a/draw.go b/draw.go new file mode 100644 index 0000000..ba2d061 --- /dev/null +++ b/draw.go @@ -0,0 +1,326 @@ +package fejkdata + +import ( + "fmt" + "sort" + "strings" +) + +// drawSet is one render's reference draws: the unnamed draw group's, and each named one's. +type drawSet struct { + unnamed draws + named map[string]*draws +} + +// drawScope is where a render reads its reference paths: a draw set, in the draw group of the +// template rendering. +type drawScope struct { + set *drawSet + group string +} + +// newDrawSet makes the unnamed group's maps where the set is declared, keeping them on that frame's +// stack for a render that reads through them; a zero drawSet makes them on its first read instead. +func newDrawSet() drawSet { + return drawSet{unnamed: draws{variant: map[string]node{}, value: map[string]draw{}}} +} + +// renderOnce renders n as one render, over draws of its own. +func renderOnce(s *session, n node) string { + var set drawSet + return render(s, n, drawScope{set: &set}) +} + +// in is the scope t renders in: its draw group where it names one, else its caller's. +func (sc drawScope) in(t *template) drawScope { + if t.drawGroupKey != "" { + sc.group = t.drawGroupKey + } + return sc +} + +// draws is the set's draws for sc's draw group. +func (sc drawScope) draws() *draws { + if sc.group == "" { + return &sc.set.unnamed + } + d, drew := sc.set.named[sc.group] + if !drew { + if sc.set.named == nil { + sc.set.named = map[string]*draws{} + } + d = &draws{variant: map[string]node{}, value: map[string]draw{}} + sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's draw set, to the heap + } + return d +} + +// drawGroupOf reads a template's "drawGroup" (default ""), which a repeat cannot carry: each +// iteration renders in no draw group. +func drawGroupOf(m map[string]any, repeat int) (string, error) { + v, ok := m["drawGroup"] + if !ok { + return "", nil + } + name, ok := v.(string) + switch { + case !ok: + return "", fmt.Errorf("drawGroup must be a string, got %T", v) + case name == "": + return "", fmt.Errorf(`drawGroup "" is the default, so it has no effect; drop it`) + case repeat > 1: + return "", fmt.Errorf("drawGroup %q on a repeat names nothing, since each iteration is a render of its own; drop it", name) + } + return name, nil +} + +// keyDrawGroup keys t's draw group by the category t sits in, "" for an inline template, so a name +// is local to its category. +func (t *template) keyDrawGroup(category string) { + if t.drawGroup != "" { + t.drawGroupKey = category + "/" + t.drawGroup + } +} + +// checkNestedDrawGroup refuses a template beneath one drawing in group that names group again, +// short of a repeat or another draw group. +func checkNestedDrawGroup(fields map[string]node, group string) error { + if group == "" { + return nil + } + var walk func(path string, n node) error + walk = func(path string, n node) error { + t, isTemplate := n.(*template) + switch { + case isTemplate && t.drawGroup == group: + return fmt.Errorf("%q names drawGroup %q, the draw group this template draws in already; drop it", path, group) + case isTemplate && (t.drawGroup != "" || t.repeat > 1): + return nil + } + for _, c := range contained(n) { + if err := walk(join(path, c.name), c.node); err != nil { + return err + } + } + return nil + } + for _, name := range sortedNames(fields) { + if err := walk(name, fields[name]); err != nil { + return err + } + } + return nil +} + +// drawCheck fences each template of a scope as a render of its own, remembering which nodes read a +// reference path. +type drawCheck struct { + memo map[readsMemo]bool +} + +type readsMemo struct { + n node + stopAtGroup bool +} + +// checkDrawGroup refuses a draw group that splits nothing: one whose render reads every reference +// path inside a repeat or a nested draw group, which draw apart from it whatever it names. +func (c *drawCheck) checkDrawGroup(path string, n node) error { + if t, ok := n.(*template); ok && t.drawGroup != "" && !c.splitsDraws(t) { + return fmt.Errorf("%s: drawGroup %q splits nothing, since nothing it renders reads a reference path outside a repeat or a nested drawGroup; drop it", path, t.drawGroup) + } + return nil +} + +func (c *drawCheck) checkDraws(path string, n node) error { + t, ok := n.(*template) + if !ok || !c.readsPath(t) { + return nil + } + w := newDrawWalk() + for _, e := range renderEdges(t) { + w.edge(t, e, drawAt{group: t.drawGroupKey, route: drawRoute{e.reached(), e.label}}) + } + if err := w.check(); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + return nil +} + +// checkRecordDraws fences the columns of a record — a template compiled at the top without a +// repeat — so a load proves the record view of it as well as the string view. +func (c *drawCheck) checkRecordDraws(path string, n node) error { + t, ok := n.(*template) + if !ok || !t.record { + return nil + } + // A record-only template's format renders nothing, so weigh the columns, not the format. + columns := recordColumns(t) + reads := false + for _, name := range columns { + reads = reads || c.readsPath(t.fields[name]) + } + if !reads { + return nil + } + if err := checkColumnDraws(t, columns); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + return nil +} + +// readsPath reports whether rendering n reads a reference path, short of a repeat, which renders +// over draws of its own. +func (c *drawCheck) readsPath(n node) bool { return c.reads(n, false) } + +// splitsDraws reports whether rendering n reads a reference path that n's own draw group answers +// for: one outside a repeat and outside a nested draw group, which hold their own draws. +func (c *drawCheck) splitsDraws(n node) bool { return c.reads(n, true) } + +// reads walks what rendering n renders for a reference path, stopping at a repeat — and at a nested +// draw group when stopAtGroup — since each holds draws of its own. +func (c *drawCheck) reads(n node, stopAtGroup bool) bool { + k := readsMemo{n, stopAtGroup} + if r, done := c.memo[k]; done { + return r + } + r := false + for _, e := range renderEdges(n) { + if a, isRef := refRead(n, e.label); (isRef && len(a.tail) > 0) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) { + r = true + break + } + } + if c.memo == nil { + c.memo = map[readsMemo]bool{} + } + c.memo[k] = r + return r +} + +func repeats(n node) bool { + t, isTemplate := n.(*template) + return isTemplate && t.repeat > 1 +} + +func grouped(n node) bool { + t, isTemplate := n.(*template) + return isTemplate && t.drawGroupKey != "" +} + +// refRead is the reference an edge of n reads; false when the edge reads none. +func refRead(n node, label string) (arm, bool) { + t, isTemplate := n.(*template) + if !isTemplate { + return arm{}, false + } + a := splitArm(label, t.refs) + return a, isRef(a.key) +} + +// checkColumnDraws fences a record's columns as one render. +func checkColumnDraws(t *template, columns []string) error { + w := newDrawWalk() + for _, name := range columns { + w.walk(t.fields[name], drawAt{group: t.drawGroupKey, route: drawRoute{spelling: fmt.Sprintf("column %q", name)}}) + } + return w.check() +} + +// drawWalk gathers the references one render reads, by draw group, for check to compare. +type drawWalk struct { + reads []pathRead + read map[drawKey]bool + seen map[drawVisit]bool +} + +// drawAt is where a walk stands: the draw group it draws in, and how the render's root reached it. +type drawAt struct { + group string + route drawRoute +} + +// drawRoute is how a render reaches a draw: as its author spells it, and the root edge's label. +type drawRoute struct{ spelling, label string } + +// pathRead is one reference a render reads: a path, or a bare reference with no tail. +type pathRead struct { + at drawAt + a arm +} + +type drawVisit struct { + n node + group string +} + +type drawKey struct { + group string + path string +} + +func newDrawWalk() *drawWalk { + return &drawWalk{read: map[drawKey]bool{}, seen: map[drawVisit]bool{}} +} + +// walk follows what rendering n renders. A repeat renders over draws of its own, so the walk stops +// there. +func (w *drawWalk) walk(n node, at drawAt) { + v := drawVisit{n, at.group} + if w.seen[v] { + return + } + w.seen[v] = true + if repeats(n) { + return + } + if t, isTemplate := n.(*template); isTemplate && t.drawGroupKey != "" { + at.group = t.drawGroupKey + } + for _, e := range renderEdges(n) { + w.edge(n, e, at) + } +} + +func (w *drawWalk) edge(from node, e renderEdge, at drawAt) { + if a, reads := refRead(from, e.label); reads { + if k := (drawKey{at.group, a.path}); !w.read[k] { + w.read[k] = true + w.reads = append(w.reads, pathRead{at, a}) + } + } + w.walk(e.to, at) +} + +// check refuses what one draw per reference path cannot answer for: a read of a level beside a path +// another read takes into it. Reads are compared in path order, so which pair is reported does not +// vary. +func (w *drawWalk) check() error { + sort.SliceStable(w.reads, func(i, j int) bool { + if w.reads[i].at.group != w.reads[j].at.group { + return w.reads[i].at.group < w.reads[j].at.group + } + return w.reads[i].a.path < w.reads[j].a.path + }) + for i, level := range w.reads { + for _, into := range w.reads[i+1:] { + if into.at.group == level.at.group && strings.HasPrefix(into.a.path, level.a.path+".") { + return overlapError(level.at.route, level.a.name, into) + } + } + } + return nil +} + +func overlapError(route drawRoute, ref string, into pathRead) error { + return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name)) +} + +// spelled names the route, and the reference it reaches a draw by where its root edge is not that +// reference. +func (r drawRoute) spelled(ref string) string { + if ref == "" || ref == r.label { + return r.spelling + } + return fmt.Sprintf("%s with {%s}", r.spelling, ref) +} diff --git a/fejkdata.go b/fejkdata.go index 24bcf7e..b19e785 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -142,10 +142,10 @@ func (f *Generator) List() []string { } // paths lists the dot paths addressable from n, relative to it, where "" is n -// itself. A group has no value of its own, so it contributes only its children's. +// itself. A folder has no value of its own, so it contributes only its children's. func paths(n node) []string { switch n := n.(type) { - case *group: + case *folder: var out []string for _, name := range sortedNames(n.children) { for _, p := range paths(n.children[name]) { diff --git a/graph.go b/graph.go index 962a51c..66a03e6 100644 --- a/graph.go +++ b/graph.go @@ -51,7 +51,7 @@ type namedNode struct { func contained(n node) []namedNode { switch n := n.(type) { - case *group: + case *folder: return named(n.children) case *choice: out := make([]namedNode, len(n.items)) @@ -69,7 +69,7 @@ func contained(n node) []namedNode { // named skips a bound {/path} key: it is a render edge, not containment, so using // it as a path segment would report a node under a path that does not reach it. Only // a template's fields hold bindings — loadDir skips a dot-prefixed entry, so a -// group's children never carry the prefix — so this one skip serves both. +// folder's children never carry the prefix — so this one skip serves both. func named(m map[string]node) []namedNode { out := make([]namedNode, 0, len(m)) for _, name := range sortedNames(m) { @@ -111,7 +111,7 @@ func (e renderEdge) reached() string { // renderEdges lists the children rendering n recurses into, mirroring expand: a // choice's items, and a template's field/reference tokens plus its operands. A -// group renders nothing, so it has no edges. +// folder renders nothing, so it has no edges. func renderEdges(n node) []renderEdge { switch n := n.(type) { case *choice: @@ -196,6 +196,24 @@ func checkRenders(s nodeScope) error { if err := s(heldCheck); err != nil { return err } + fence := &drawCheck{} + refs := false + if err := s(func(path string, n node) error { + if t, ok := n.(*template); ok && len(t.refs) > 0 { + refs = true + } + return fence.checkDrawGroup(path, n) + }); err != nil { + return err + } + if refs { + if err := s(fence.checkDraws); err != nil { + return err + } + if err := s(fence.checkRecordDraws); err != nil { + return err + } + } return s((&valueProof{}).checkDatatype) } diff --git a/hold.go b/hold.go index 16b00a8..591e823 100644 --- a/hold.go +++ b/hold.go @@ -6,11 +6,12 @@ import ( "strings" ) -// heldCheck rejects every route to a held name except the ones that read its draw. -// An expansion holds one draw of that name; anything else that renders it draws -// again, and the two disagree. checkNoOverlap settles the spellings within one -// format (a token, an operand); this settles the rest — a reference, whether it -// sits in that format or in anything the format renders, however deep. +// heldCheck rejects every route to a held sibling name except the ones that read its +// draw. An expansion holds one draw of that name; anything else that renders it draws +// again, and the two disagree. checkNoOverlap settles the spellings within one format +// (a token, an operand); this settles the rest — a reference, whether it sits in that +// format or in anything the format renders, however deep. A reference path is held +// for the whole render instead, which drawCheck fences. func heldCheck(path string, n node) error { t, ok := n.(*template) if !ok || len(t.held) == 0 { @@ -18,6 +19,9 @@ func heldCheck(path string, n node) error { } readers := boundReaders(t.format, t.bound, t.refs) for _, head := range heldHeads(t) { + if _, isPath := t.bound[head]; isPath && isRef(head) { + continue + } if err := checkHeadHeld(t, head, readers); err != nil { return fmt.Errorf("%s: %w", path, err) } @@ -175,12 +179,11 @@ func renders(n node, want, seen map[node]bool) bool { return false } -// checkNoOverlap rejects a format that both renders a level and reads a path into -// it — {p} beside {p.first}, {p.addr} beside {p.addr.city}, {.p} beside -// {/sv_SE.p.first}. The path reads the level's held draw while rendering the level -// expands it afresh, so their values would disagree. Reads are compared by their -// one spelling, in sorted order, so which pair is reported depends neither on how -// a reference was written nor on where the tokens sit. +// checkNoOverlap rejects a format that both renders a sibling level and reads a path +// into it — {p} beside {p.first}, {p.addr} beside {p.addr.city}. The path reads the +// level's held draw while rendering the level expands it afresh, so their values would +// disagree. Reads are compared in sorted order, so which pair is reported does not +// depend on where the tokens sit. func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error { names := boundReaders(format, bound, refs) // Stable over one format-order scan, so two readers of one name (a token and a @@ -200,7 +203,7 @@ func checkNoOverlap(format string, bound map[string]string, refs map[string]refB // spelling, and how to name it. type reader struct{ name, path, label string } -// boundReaders lists every way a format reaches a bound field, in the order the +// boundReaders lists every way a format reaches a bound sibling field, in the order the // format writes them. An operand renders its field, so it names a level exactly // as a token does; one scan finds both, which is what puts them in one order. func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader { @@ -212,14 +215,14 @@ func boundReaders(format string, bound map[string]string, refs map[string]refBin if fn, _, isFunc := funcCall(t.body); isFunc { for _, operand := range tokenOperands(t.body) { a := splitArm(operand, refs) - if _, isBound := bound[a.key]; isBound { + if _, isBound := bound[a.key]; isBound && !isRef(a.key) { names = append(names, reader{a.name, a.path, fmt.Sprintf("%s operand %q", fn, operand)}) } } return nil } for _, a := range splitArms(t.body, refs) { - if _, isBound := bound[a.key]; isBound { + if _, isBound := bound[a.key]; isBound && !isRef(a.key) { names = append(names, reader{a.name, a.path, "token {" + a.name + "}"}) } } @@ -252,9 +255,10 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) }) } -// draws is what an expansion has already drawn for its held names: the variant each -// was drawn as, so every path under it reads one row, and the draw each read made, by -// its one spelling, so the same read written twice reads one draw. +// draws is what has already been drawn for held names: the variant each was drawn as, +// so every path under it reads one row, and the draw each read made, by its one +// spelling, so the same read written twice reads one draw. An expansion keeps one for +// its sibling names, and a render one per group for its reference paths. type draws struct { variant map[string]node value map[string]draw @@ -273,14 +277,14 @@ type draw struct { // gives one value, and a shown operand is the operand computed. Every other name is // drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and // linkRefs prove every step, so the walk cannot fail. -func readField(s *session, t *template, held, refScope *draws, a arm) draw { +func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw { if !t.held[a.key] { if len(a.tail) > 0 { panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key)) } - return draw{text: render(s, t.fields[a.key], refScope)} + return draw{text: render(s, t.fields[a.key], sc)} } - d := readScope(held, refScope, a) + d := readScope(held, sc, a) if r, done := d.value[a.path]; done { return r } @@ -296,37 +300,42 @@ func readField(s *session, t *template, held, refScope *draws, a arm) draw { n, drew := d.variant[key] if !drew { n = drawn(s, c) + if d.variant == nil { + d.variant = map[string]node{} + } d.variant[key] = n } return []node{n}, nil }, - leaf: func(n node) error { r = renderLeaf(s, n, refScope); return nil }, + leaf: func(n node) error { r = renderLeaf(s, n, sc); return nil }, }) + if d.value == nil { + d.value = map[string]draw{} + } d.value[a.path] = r return r } // readScope is the draws a held read keeps its draw in: for a reference that reads a path, -// refScope, so its draw outlives this expansion; for a sibling, or a reference read whole, held. -func readScope(held, refScope *draws, a arm) *draws { - if isRef(a.key) && refScope != nil && len(a.tail) > 0 { - return refScope +// the render's draws for its group, so its draw spans the render; for a sibling, or a +// reference read whole, held. +func readScope(held *draws, sc drawScope, a arm) *draws { + if isRef(a.key) && len(a.tail) > 0 { + return sc.draws() } return held } // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. -func renderLeaf(s *session, n node, scope *draws) draw { +func renderLeaf(s *session, n node, sc drawScope) draw { n = drawn(s, n) if _, isNull := n.(*null); isNull { return draw{null: true} } - r := draw{text: render(s, n, scope)} + r := draw{text: render(s, n, sc)} if t, _ := n.(*template); t != nil && t.readsColumn != nil { - if d := readScope(nil, scope, t.readsColumn.a); d != nil { - r.null = d.value[t.readsColumn.a.path].null - } + r.null = sc.in(t).draws().value[t.readsColumn.a.path].null } return r } diff --git a/hold_test.go b/hold_test.go index ac1ac56..53f6f9f 100644 --- a/hold_test.go +++ b/hold_test.go @@ -91,17 +91,28 @@ func TestReferenceNamingABoundLevelIsRejected(t *testing.T) { // A reference can name a bound level from the data root, which renders it // afresh beside the path that reads its held draw — the same overlap by // another spelling. - rejected := map[string]string{ - "reference names the head": `{"format":"{p.first}|{/cat.p}","p":{"format":"{first}","first":["Anna","Bo"]}}`, - "reference names the leaf": `{"format":"{p.addr}|{/cat.p.addr}","p":{"format":"x","addr":["A","B","C","D"]}}`, + // A category never references itself, so each reference back into cat sits in a + // second category cat renders. + rejected := map[string]map[string]string{ + "reference names the head": { + "cat": `{"format":"{p.first}|{/hop}","p":{"format":"{first}","first":["Anna","Bo"]}}`, + "hop": `"{/cat.p}"`, + }, + "reference names the leaf": { + "cat": `{"format":"{p.addr}|{/hop}","p":{"format":"x","addr":["A","B","C","D"]}}`, + "hop": `"{/cat.p.addr}"`, + }, // The reference need not sit in the format that binds: any field it renders // reaches the level just the same, however deep. - "reference from a sibling field": `{"format":"{p.first}|{inner}","p":[` + - `{"format":"{first}-{last}","first":"A","last":"1"},{"format":"{first}-{last}","first":"B","last":"2"}],` + - `"inner":"{/cat.p}"}`, + "reference from a sibling field": { + "cat": `{"format":"{p.first}|{inner}","p":[` + + `{"format":"{first}-{last}","first":"A","last":"1"},{"format":"{first}-{last}","first":"B","last":"2"}],` + + `"inner":"{/hop}"}`, + "hop": `"{/cat.p}"`, + }, } - for name, file := range rejected { - _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file}))) + for name, files := range rejected { + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files))) if err == nil || !strings.Contains(err.Error(), "reads a path into") { t.Errorf("%s: New = %v, want the reference rejected as an overlap", name, err) } @@ -120,7 +131,8 @@ func TestCycleReachedOnlyByAPathTokenIsRejected(t *testing.T) { // cycle walk has to follow it there. A head whose own format names nothing // would otherwise hide the cycle until render, where it is fatal. _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ - "a": `{"format":"{p.x}","p":{"format":"static","x":"{/a}"}}`, + "a": `{"format":"{p.x}","p":{"format":"static","x":"{/b}"}}`, + "b": `"{/a}"`, }))) if err == nil || !strings.Contains(err.Error(), "reference cycle") { t.Fatalf("New = %v, want the cycle through {p.x} rejected", err) @@ -131,8 +143,9 @@ func TestALevelRenderedOnlyByAPathTokenIsHeld(t *testing.T) { // {p.a} renders q, so it is a route to the level {q.x} holds — even though p's // own format names nothing. _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ - "thing": `{"format":"{p.a} {q.x}","p":{"format":"static","a":"{/thing.q}"},` + + "thing": `{"format":"{p.a} {q.x}","p":{"format":"static","a":"{/hop}"},` + `"q":{"format":"{x}","x":["1","2"]}}`, + "hop": `"{/thing.q}"`, }))) if err == nil || !strings.Contains(err.Error(), "reads a path into") { t.Fatalf("New = %v, want the second route to q rejected", err) @@ -149,16 +162,20 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { files map[string]string want string // the route the error names, spelled as the author wrote it }{ + // A category never references itself, so each route back into cat sits in a + // second category cat renders. "a reference beside the operand": { map[string]string{ - "cat": `{"format":"{/cat.net} x 2 = {calc(net * 2, 2)}","net":["10.00","20.00"]}`, + "cat": `{"format":"{/hop} x 2 = {calc(net * 2, 2)}","net":["10.00","20.00"]}`, + "hop": `"{/cat.net}"`, }, - `{/cat.net} renders "net"`, + `{/hop} renders "net"`, }, "a reference one level down": { map[string]string{ "cat": `{"format":"{calc(net * 2, 2)} {q}","net":["10.00","20.00"],` + - `"q":"{/cat.net}"}`, + `"q":"{/hop}"}`, + "hop": `"{/cat.net}"`, }, `{q} renders "net"`, }, @@ -167,7 +184,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { "a reference to an operand wrapped in a choice": { map[string]string{ "cat": `{"format":"{calc(n * 2, 2)} {q}","n":{"format":"{v}","v":["1","2"]},` + - `"q":"{/cat.n}"}`, + `"q":"{/hop}"}`, + "hop": `"{/cat.n}"`, }, `{q} renders "n"`, }, @@ -176,7 +194,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { "an operand reaching another operand": { map[string]string{ "cat": `{"format":"{calc(a + b, 0)}","a":{"format":"{x}","x":["1","2"]},` + - `"b":"{/cat.a}"}`, + `"b":"{/hop}"}`, + "hop": `"{/cat.a}"`, }, `calc operand "b" renders "a"`, }, @@ -185,15 +204,17 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { // rejected, so the reference spelling has to be. "a reference into the operand": { map[string]string{ - "cat": `{"format":"{calc(net * 2, 2)}|{/cat.net.v}",` + + "cat": `{"format":"{calc(net * 2, 2)}|{/hop}",` + `"net":{"format":"{v}","v":["10.00","20.00"]}}`, + "hop": `"{/cat.net.v}"`, }, - `{/cat.net.v} renders "net"`, + `{/hop} renders "net"`, }, "a reference into the operand one level down": { map[string]string{ "cat": `{"format":"{calc(net * 2, 2)}|{q}","net":{"format":"{v}","v":["10.00","20.00"]},` + - `"q":"{/cat.net.v}"}`, + `"q":"{/hop}"}`, + "hop": `"{/cat.net.v}"`, }, `{q} renders "net"`, }, @@ -203,7 +224,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { "a violation on the second of two held heads": { map[string]string{ "cat": `{"format":"{calc(a + b, 0)} {w}","a":{"format":"{x}","x":["1","2"]},` + - `"b":{"format":"{y}","y":["3","4"]},"w":"{/cat.b}"}`, + `"b":{"format":"{y}","y":["3","4"]},"w":"{/hop}"}`, + "hop": `"{/cat.b}"`, }, `{w} renders "b"`, }, @@ -230,7 +252,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { "a reference to a sibling the operand never renders": { "cat": `{"format":"{calc(net * 2, 2)} {unit}",` + `"net":{"format":"{v}","v":["1","2"],"spare":["kg","lb"]},` + - `"unit":"{/cat.net.spare}"}`, + `"unit":"{/hop}"}`, + "hop": `"{/cat.net.spare}"`, }, // Two operands drawing from one source are two names, so two draws: each is // held under its own name and shown once, and neither can disagree. @@ -247,7 +270,8 @@ func TestACalcOperandIsHeldAgainstEveryRoute(t *testing.T) { }, // A fixed string cannot disagree with itself, so it needs no fence. "a literal operand named twice": { - "cat": `{"format":"{calc(n * 2, 0)} {/cat.n}","n":"5"}`, + "cat": `{"format":"{calc(n * 2, 0)} {/hop}","n":"5"}`, + "hop": `"{/cat.n}"`, }, // One node reached twice while walking the operand: the walk must not // revisit it, and the repeat is not a second route to anything. @@ -267,19 +291,28 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) { // all of them. Reaching only the first leaves whatever hides in a later // variant to be found at render — a cycle there is fatal, and a second route // to a held level disagrees silently. - rejected := map[string]struct{ file, want string }{ + rejected := map[string]struct { + files map[string]string + want string + }{ "a cycle in a later variant": { - `{"format":"{p.x}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/cat}"}]}`, + map[string]string{ + "cat": `{"format":"{p.x}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/hop}"}]}`, + "hop": `"{/cat}"`, + }, "reference cycle", }, "a second route in a later variant": { - `{"format":"{p.x} {q.y}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/cat.q}"}],` + - `"q":{"format":"{y}","y":["1","2"]}}`, + map[string]string{ + "cat": `{"format":"{p.x} {q.y}","p":[{"format":"h","x":"safe"},{"format":"h","x":"{/hop}"}],` + + `"q":{"format":"{y}","y":["1","2"]}}`, + "hop": `"{/cat.q}"`, + }, "reads a path into", }, } for name, c := range rejected { - _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": c.file}))) + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, c.files))) if err == nil || !strings.Contains(err.Error(), c.want) { t.Errorf("%s: New = %v, want it to mention %q", name, err, c.want) } @@ -289,15 +322,22 @@ func TestAPathReachesEveryVariantItMightDraw(t *testing.T) { func TestALevelAPathNeverRendersIsAccepted(t *testing.T) { // A path token does not expand its head's format, so a reference sitting in // that format is not a second route to anything: it is never rendered by the - // path at all. Both orders must load. - accepted := map[string]string{ - "reference in the head's own format": `{"format":"{p.first} {q.a}",` + - `"p":{"format":"{first} {/thing.q}","first":["A","B"]},"q":{"format":"{a}","a":["1","2"]}}`, - "the mirror shape": `{"format":"{p.first} {q.a}",` + - `"p":{"format":"{first}","first":["A","B"]},"q":{"format":"{a} {/thing.p}","a":["1","2"]}}`, + // path at all. Both orders must load. p and q sit one level down, so they are + // one column of thing rather than two that would read one another. + accepted := map[string]map[string]string{ + "reference in the head's own format": { + "thing": `{"format":"{inner}","inner":{"format":"{p.first} {q.a}",` + + `"p":{"format":"{first} {/hop}","first":["A","B"]},"q":{"format":"{a}","a":["1","2"]}}}`, + "hop": `"{/thing.inner.q}"`, + }, + "the mirror shape": { + "thing": `{"format":"{inner}","inner":{"format":"{p.first} {q.a}",` + + `"p":{"format":"{first}","first":["A","B"]},"q":{"format":"{a} {/hop}","a":["1","2"]}}}`, + "hop": `"{/thing.inner.p}"`, + }, } - for name, file := range accepted { - if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"thing": file}))); err != nil { + for name, files := range accepted { + if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files))); err != nil { t.Errorf("%s: New = %v, want it accepted", name, err) } } @@ -404,6 +444,13 @@ func TestPathIntoARepeatingLevelIsRejected(t *testing.T) { if err == nil || !strings.Contains(err.Error(), "repeat") { t.Fatalf("New = %v, want a path into a repeating level rejected", err) } + _, err = New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ + "cat": `{"format":"[{p.a}]","p":{"format":"{a}","a":"{/word.w}","drawGroup":"g"}}`, + "word": `{"format":"{w}","w":["x","y"]}`, + }))) + if err == nil || !strings.Contains(err.Error(), `the level "p" carries a drawGroup`) { + t.Fatalf("New = %v, want a path into a level carrying a group rejected", err) + } } func TestPathIntoAPlainTemplateNamesTheMissingField(t *testing.T) { @@ -666,7 +713,8 @@ func TestCycleThroughAPathTokenIsRejected(t *testing.T) { // must be caught at New. Reaching render would be fatal: the recursion never // terminates, and a stack overflow cannot be recovered. _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ - "a": `{"format":"{p.x}","p":{"format":"{x}","x":"{/a}"}}`, + "a": `{"format":"{p.x}","p":{"format":"{x}","x":"{/b}"}}`, + "b": `"{/a}"`, }))) if err == nil || !strings.Contains(err.Error(), "reference cycle") { t.Fatalf("New = %v, want the cycle through {p.x} rejected", err) @@ -718,6 +766,6 @@ func TestReadFieldPanicsOnAnUnheldPath(t *testing.T) { t.Fatal("not a template") } mustPanic(t, "unheld arm with a path", func() { - readField(engine(1).rand, tm, nil, nil, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) + readField(engine(1).rand, tm, nil, drawScope{}, arm{name: "w.x", key: "w", tail: []string{"x"}, path: "w.x"}) }) } diff --git a/inline.go b/inline.go index 23b592b..f37a007 100644 --- a/inline.go +++ b/inline.go @@ -20,7 +20,7 @@ type Template struct { func (t *Template) Fake() string { t.g.mu.Lock() defer t.g.mu.Unlock() - return render(t.g.rand, t.n, nil) + return renderOnce(t.g.rand, t.n) } // NewTemplate compiles an inline template — a format string or a JSON value — and @@ -136,6 +136,9 @@ func inputValue(input string) (any, error) { // bindInline links an inline node's references against root and runs check over it, naming its // nodes from label. func bindInline(n node, label string, root map[string]node, check func(nodeScope) error) error { + if t, isTemplate := n.(*template); isTemplate && t.drawGroup != "" { + return fmt.Errorf("%s: drawGroup %q names nothing, since nothing can reference an inline template; drop it", label, t.drawGroup) + } scope := inlineScope(n, label) if err := linkNodeRefs(scope, root); err != nil { return err @@ -151,6 +154,7 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { if !ok { return nil } + t.keyDrawGroup("") for _, name := range refTokens(t.format) { sigil, rest, err := refShape(name) if err != nil { @@ -160,6 +164,6 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { return fmt.Errorf("%s: reference {%s}: an inline template has no folder; write {/%s}", path, name, rest) } } - return linkTemplateRefs(nil, path, t, root) + return linkTemplateRefs(nil, path, "", t, root) }) } diff --git a/inline_test.go b/inline_test.go index b3a7f9d..8cdb217 100644 --- a/inline_test.go +++ b/inline_test.go @@ -108,8 +108,9 @@ func TestFakeTemplateErrors(t *testing.T) { {`name: {/no.such.path}`, "no entry"}, {`name: {..nope}`, "write {/nope}"}, {`{"format":"x"}`, "is a string"}, - {`{/misc.country} {/misc.country.alpha2}`, "renders a level"}, + {`{/misc.country} {/misc.country.alpha2}`, "renders a level that {/misc.country.alpha2} reads a path into; name the fields you want instead, or draw them apart with a drawGroup"}, {`{"format":"{/misc.country.alpha2} {x}","x":"{/misc.country}"}`, "reads a path into"}, + {`{"format":"{/misc.country.alpha2}","drawGroup":"g"}`, "nothing can reference"}, } { _, err := f.FakeTemplate(c.input) if err == nil || !strings.Contains(err.Error(), c.want) { diff --git a/node.go b/node.go index 604ea0c..b5c9cb0 100644 --- a/node.go +++ b/node.go @@ -13,12 +13,12 @@ import ( // re-sums weights. type node interface{ isNode() } -// group is a namespace of named children, built from a directory of JSON files +// folder is a namespace of named children, built from a directory of JSON files // and subdirectories. It has no value of its own: descend into a named child by // dot path; rendering one is an error (see Fake). -type group struct{ children map[string]node } +type folder struct{ children map[string]node } -func (*group) isNode() {} +func (*folder) isNode() {} // choice picks one of its items. cum holds cumulative weights for a weighted // pick; when nil the choice is uniform and selection is O(1). shared is the set of @@ -58,10 +58,12 @@ type template struct { bound map[string]string // held is every name drawn once per expansion: the bound levels above, plus the // siblings a {calc()} reads. nil when the format holds nothing (see expand). - held map[string]bool - fromString bool // written as a JSON string rather than an object - readsColumn *columnRead // set when the format is one reference alone reading a record's column - record bool // compiled at the top without a repeat, so its fields are record columns + held map[string]bool + fromString bool // written as a JSON string rather than an object + readsColumn *columnRead // set when the format is one reference alone reading a record's column + record bool // compiled at the top without a repeat, so its fields are record columns + drawGroup string // the draw group it draws in, as written; "" keeps its caller's + drawGroupKey string // its draw group keyed by its category once linked: what a render reads its reference paths under } func (*template) isNode() {} @@ -241,13 +243,16 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if err != nil { return nil, err } - if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString { + if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" { return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format) } if err := checkTokens(o.format, fields); err != nil { return nil, err } - t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, record: fieldPos == inColumn} + if err := checkNestedDrawGroup(fields, o.group); err != nil { + return nil, err + } + t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, record: fieldPos == inColumn} if err := t.compileFormat(); err != nil { return nil, err } @@ -258,6 +263,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) { type templateOptions struct { datatype DataType format string + group string repeat int separator string weighted bool @@ -278,6 +284,9 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) { if o.datatype, err = datatypeOf(m, pos); err != nil { return o, err } + if o.group, err = drawGroupOf(m, repeat); err != nil { + return o, err + } if sv, ok := m["separator"]; ok { if o.separator, ok = sv.(string); !ok { return o, fmt.Errorf("separator must be a string, got %T", sv) @@ -410,7 +419,7 @@ func checkPathNames(path string) error { // field. These names can never be fields. func isOption(name string) bool { switch name { - case "datatype", "format", "repeat", "separator", "weight": + case "datatype", "drawGroup", "format", "repeat", "separator", "weight": return true } return false diff --git a/one_spelling_test.go b/one_spelling_test.go index 8253b1c..6f1f559 100644 --- a/one_spelling_test.go +++ b/one_spelling_test.go @@ -31,25 +31,52 @@ func TestRepeatedChoiceItemIsRejected(t *testing.T) { func TestInertObjectIsRejected(t *testing.T) { for src, want := range map[string]string{ - `{"format":"Malmö"}`: `write "Malmö"`, - `{"format":"{digits(3)}"}`: `write "{digits(3)}"`, - `[{"format":"a","weight":1},"b"]`: "weight 1", - `{"format":"{x}","x":"v","repeat":1}`: "repeat 1", - `{"format":"{x}","x":"v","separator":","}`: "separator", - `{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default", - `{"format":"","n":{"format":"1","datatype":"string"}}`: `datatype "string" is the default`, + `{"format":"Malmö"}`: `write "Malmö"`, + `{"format":"{digits(3)}"}`: `write "{digits(3)}"`, + `[{"format":"a","weight":1},"b"]`: "weight 1", + `{"format":"{x}","x":"v","repeat":1}`: "repeat 1", + `{"format":"{x}","x":"v","separator":","}`: "separator", + `{"format":"{x}","x":"v","repeat":2,"separator":""}`: "default", + `{"format":"","n":{"format":"1","datatype":"string"}}`: `datatype "string" is the default`, + `{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":""}}`: `drawGroup "" is the default`, + `{"format":"{x}","x":{"format":"{y}","y":"v","drawGroup":1}}`: "drawGroup must be a string", } { if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) { t.Errorf("compile(%s) = %v, want an error mentioning %s", src, err, want) } } - for _, ok := range []string{`[{"format":"a","weight":2},"b"]`, `{"format":"ab","repeat":2}`, `{"format":"{x}","x":"v"}`, `"{digits(3)}"`} { + for _, ok := range []string{`[{"format":"a","weight":2},"b"]`, `{"format":"ab","repeat":2}`, `{"format":"{x}","x":"v"}`, `"{digits(3)}"`, `{"format":"{/cat.x}","drawGroup":"g"}`} { if _, err := compile(parse(t, ok)); err != nil { t.Errorf("compile(%s) = %v", ok, err) } } } +func TestAGroupThatSplitsNothingIsRejected(t *testing.T) { + files := map[string]string{"mail": `"{/word.w}@example.com"`, "word": `{"format":"{w}","w":["a","b"]}`} + for src, want := range map[string]string{ + `{"format":"{x}","x":{"format":"{y}","y":["a","b"],"drawGroup":"g"}}`: `drawGroup "g" splits nothing`, + `{"format":"{x}","x":{"format":"{/word}","drawGroup":"g"}}`: `drawGroup "g" splits nothing`, + `{"format":"{x}","x":{"format":"{r}","drawGroup":"g","r":{"format":"{/word.w}","repeat":2}}}`: `drawGroup "g" splits nothing`, + `{"format":"{x}","x":{"format":"{y}","drawGroup":"g","y":{"format":"{/word.w}","drawGroup":"h"}}}`: `drawGroup "g" splits nothing`, + `{"format":"{x}","x":{"format":"{y}","y":"{/word.w}","repeat":2,"drawGroup":"g"}}`: `drawGroup "g" on a repeat`, + `{"format":"{x}","x":{"format":"{/word.w} {y}","drawGroup":"g","y":{"format":"{/word.w}!","drawGroup":"g"}}}`: `"y" names drawGroup "g", the draw group this template draws in already`, + `{"format":"{/word.w}","drawGroup":"g"}`: "", + `{"format":"{x}","x":{"format":"{/mail}","drawGroup":"g"}}`: "", + `{"format":"{x}","x":{"format":"{/word.w} {y}","drawGroup":"g","y":{"format":"{/word.w}!","drawGroup":"h"}}}`: "", + `{"format":"{x}","x":{"format":"{/word.w} {r}","drawGroup":"g","r":{"format":"{y}","repeat":2,"y":{"format":"{/word.w}","drawGroup":"g"}}}}`: "", + } { + files["cat"] = src + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, files))) + switch { + case want == "" && err != nil: + t.Errorf("New(%s) = %v, want a group over a reference path accepted, however deep", src, err) + case want != "" && (err == nil || !strings.Contains(err.Error(), want)): + t.Errorf("New(%s) = %v, want an error mentioning %s", src, err, want) + } + } +} + func TestInlineFolderSigilsAreRejected(t *testing.T) { f := shipped(t) for _, input := range []string{"{.sv_SE.person.last}", "{..sv_SE.person.last}"} { diff --git a/path.go b/path.go index 687d9b9..cb5e444 100644 --- a/path.go +++ b/path.go @@ -16,7 +16,7 @@ type pathWalk struct { leaf func(n node) error } -// walkPath descends tail from n: a group or template by its next segment, a +// walkPath descends tail from n: a folder or template by its next segment, a // choice by w.choice, which consumes no segment. A missing segment is an error, // so no walk reaches past what the data holds. A table-shaped dispatch, one case // per node kind, kept whole on purpose. @@ -28,7 +28,7 @@ func walkPath(n node, tail []string, w pathWalk) error { return nil } switch n := n.(type) { - case *group: + case *folder: child, ok := n.children[tail[0]] if !ok { return fmt.Errorf("no entry %q", tail[0]) @@ -89,9 +89,9 @@ func unreachableInChoice(c *choice, want string) error { // checkPath proves a dotted tail resolves whichever way the draws go — a choice // must carry the rest of the path in the set every variant shares — and that no -// level a path reads carries a repeat, which one draw of it could not apply. So a -// path that validates here resolves on every render, and a typo is a New-time -// error. +// level a path reads carries a repeat or a drawGroup, which one draw of it could not +// apply. So a path that validates here resolves on every render, and a typo is a +// New-time error. func checkPath(n node, tail []string, level string) error { return walkPath(n, tail, pathWalk{ choice: func(c *choice, rest []string) ([]node, error) { @@ -101,8 +101,12 @@ func checkPath(n node, tail []string, level string) error { return c.items, nil }, level: func(t *template, rest []string) error { - if t.repeat > 1 { - return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", join(level, strings.Join(tail[:len(tail)-len(rest)], "."))) + name := join(level, strings.Join(tail[:len(tail)-len(rest)], ".")) + switch { + case t.repeat > 1: + return fmt.Errorf("the level %q carries a repeat, which a path reading one draw of it cannot apply", name) + case t.drawGroup != "": + return fmt.Errorf("the level %q carries a drawGroup, which a path reading into it cannot apply", name) } return nil }, diff --git a/perf_test.go b/perf_test.go index ab92408..75d1e86 100644 --- a/perf_test.go +++ b/perf_test.go @@ -51,6 +51,29 @@ func TestNoRenderAllocRegression(t *testing.T) { } } +// The repeat shape prices both escape measures: dropping either costs an alloc an iteration. +func TestNoReferenceAllocRegression(t *testing.T) { + word := `{"format":"{w}","w":["alpha","beta","gamma","delta"]}` + for _, s := range []struct { + name, json string + base float64 + }{ + {"a repeat of a reference path", `{"format":"{r}","r":{"format":"{/word.w}","repeat":20,"separator":", "}}`, 66}, + {"a named draw group", `{"format":"{a}","a":{"format":"{/word.w}","drawGroup":"g"}}`, 11}, + } { + f, err := New(WithoutShippedData(), WithDataFS(fstest.MapFS{ + "word.json": {Data: []byte(word)}, + "x.json": {Data: []byte(s.json)}, + })) + if err != nil { + t.Fatalf("New(%s): %v", s.name, err) + } + if allocs := testing.AllocsPerRun(10000, func() { f.Fake("x") }); allocs > s.base*1.10 { + t.Errorf("%s: %.1f allocs/op regressed past %.1f (baseline %.1f + 10%%); a draw set reaching the heap is the usual cause", s.name, allocs, s.base*1.10, s.base) + } + } +} + // A record's fences read the compiled tree, so they belong to New, not to a draw. func TestNoRecordAllocRegression(t *testing.T) { for _, s := range []struct{ name, json string }{ diff --git a/record.go b/record.go index b630526..748338c 100644 --- a/record.go +++ b/record.go @@ -4,7 +4,6 @@ import ( "encoding/json" "errors" "fmt" - "sort" "strings" "unicode" "unicode/utf8" @@ -23,7 +22,7 @@ type Column struct { // Record is one record rendered from a template: every direct field is a column, // listed in name order. Each column is its own expansion, so a sibling field is // local to it, while a reference that reads a path is drawn once for the whole -// record. +// record, per group. type Record struct { columns []Column } @@ -212,9 +211,6 @@ func recordOf(n node) (*template, []Column, error) { if !t.record { return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat) } - if err := checkColumnRefs(t, names); err != nil { - return nil, nil, err - } columns := make([]Column, len(names)) for i, name := range names { datatype, _ := columnDatatype(t.fields[name]) // checkColumns refused items that disagree wherever DataType is read @@ -223,80 +219,13 @@ func recordOf(n node) (*template, []Column, error) { return t, columns, nil } -// checkColumnRefs rejects the reference reads a record's shared draw cannot answer -// for: one column rendering a level another reads a path into, and a column -// reading the record back through its own path. -func checkColumnRefs(t *template, columns []string) error { - reads, err := columnRefs(t, columns) - if err != nil { - return err - } - sort.Slice(reads, func(i, j int) bool { - if reads[i].a.path != reads[j].a.path { - return reads[i].a.path < reads[j].a.path - } - return reads[i].column < reads[j].column - }) - for i, level := range reads { - for _, into := range reads[i+1:] { - if strings.HasPrefix(into.a.path, level.a.path+".") { - return fmt.Errorf("column %q renders {%s}, a level column %q reads a path into with {%s}; name the fields you want instead", - level.column, level.a.name, into.column, into.a.name) - } - } - } - return nil -} - -// columnRef is one held reference read, and the column whose render reaches it. -type columnRef struct { - column string - a arm -} - -// columnRefs lists every reference that reads a path, anywhere a column renders, -// following the same edges expand does. A reference landing back on the record -// itself is reported rather than collected, whether it reads a path or the record -// whole: either way the column describes a draw other than its neighbours'. -func columnRefs(t *template, columns []string) ([]columnRef, error) { - var out []columnRef - var err error - for _, name := range columns { - seen := map[node]bool{} - var walk func(n node) - walk = func(n node) { - if n == nil || seen[n] || err != nil { - return - } - seen[n] = true - if tm, ok := n.(*template); ok { - for _, ref := range refTokens(tm.format) { - a := splitArm(ref, tm.refs) - if tm.fields[a.key] == node(t) { - err = fmt.Errorf("column %q reads {%s}, which points back at this record; a column cannot read another column — move the shared value into its own category and reference that", name, a.name) - return - } - if len(a.tail) > 0 { - out = append(out, columnRef{name, a}) - } - } - } - for _, e := range renderEdges(n) { - walk(e.to) - } - } - walk(t.fields[name]) - } - return out, err -} - -// renderRecord draws each column once, in the name order recordOf fixed, over one -// reference scope shared across them. +// renderRecord draws each column once, in the name order recordOf fixed, as one render. func renderRecord(s *session, t *template, columns []Column) *Record { - scope := &draws{variant: map[string]node{}, value: map[string]draw{}} + set := newDrawSet() + sc := drawScope{set: &set}.in(t) r := &Record{columns: append([]Column(nil), columns...)} for i := range r.columns { - column := renderLeaf(s, t.fields[r.columns[i].Name], scope) + column := renderLeaf(s, t.fields[r.columns[i].Name], sc) r.columns[i].Value, r.columns[i].Null = column.text, column.null } return r diff --git a/record_test.go b/record_test.go index 9094172..486ecac 100644 --- a/record_test.go +++ b/record_test.go @@ -200,26 +200,33 @@ func TestRecordWritesTypedAndNullColumns(t *testing.T) { } func TestRecordRejectsOverlappingReferenceColumns(t *testing.T) { - cat := `{"format":"","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}` + cat := `{"format":"{a}","a":[{"format":"A={b}","b":"1"},{"format":"A={b}","b":"2"}]}` + mid := `"{/cat.a}"` + for _, c := range []struct{ name, row string }{ + {"through a column repeat, which draws anew", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`}, + {"in a group of its own", `{"format":"","whole":{"format":"{/cat.a}","drawGroup":"g"},"inner":"{/cat.a.b}"}`}, + } { + f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}), WithSeed(1)) + if _, err := f.FakeRecord("row"); err != nil { + t.Errorf("%s: FakeRecord = %v, want it accepted", c.name, err) + } + } for _, c := range []struct{ name, row string }{ {"sibling columns", `{"format":"","whole":"{/cat.a}","inner":"{/cat.a.b}"}`}, {"through a nested template", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b} {x}","x":"1"}}`}, - {"through a column repeat", `{"format":"","whole":"{/cat.a}","inner":{"format":"{/cat.a.b}","repeat":2,"separator":"-"}}`}, {"through a choice variant", `{"format":"","whole":"{/cat.a}","inner":[{"format":"{/cat.a.b} {x}","x":"1"},{"format":"{/cat.a.b}! {x}","x":"2"}]}`}, {"as a builtin operand", `{"format":"","whole":"{uppercase(/cat.a)}","inner":"{/cat.a.b}"}`}, + {"a bare reference beside a path", `{"format":"","whole":"{/cat}","inner":"{/cat.a.b}"}`}, + {"a column reaching back through another category", `{"format":"","whole":"{/mid}","inner":"{/cat.a.b}"}`}, } { - f := newGenerator(t, writeData(t, map[string]string{"cat": cat, "row": c.row}), WithSeed(1)) - _, err := f.FakeRecord("row") + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "mid": mid, "row": c.row}))) if err == nil || !strings.Contains(err.Error(), "reads a path into") { - t.Errorf("%s: FakeRecord = %v, want the overlap rejected the way one format is", c.name, err) + t.Errorf("%s: New = %v, want the overlap refused at load, the way one format is", c.name, err) continue } if !strings.Contains(err.Error(), `"whole"`) || !strings.Contains(err.Error(), `"inner"`) { t.Errorf("%s: error %q names neither column; it must name both", c.name, err) } - if _, err := f.Fake("row"); err != nil { - t.Errorf("%s: Fake(row) = %v, want the string view untouched", c.name, err) - } } } @@ -241,9 +248,9 @@ func TestRecordRejectsAColumnReadingItsOwnRecord(t *testing.T) { {"the record as an operand", `"up":"{uppercase(/person)}"`}, } { person := `{"format":"{first} {last}","first":["Ada","Bo"],"last":["Lovelace","Ek"],` + c.column + `}` - f := newGenerator(t, writeData(t, map[string]string{"person": person}), WithSeed(1)) - if _, err := f.FakeRecord("person"); err == nil || !strings.Contains(err.Error(), "points back at this record") { - t.Errorf("%s: FakeRecord = %v, want it refused; the column would contradict the columns beside it", c.name, err) + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"person": person}))) + if err == nil || !strings.Contains(err.Error(), "names the category it sits in") { + t.Errorf("%s: New = %v, want it refused at load; the column would contradict the columns beside it", c.name, err) } } } @@ -311,9 +318,10 @@ func TestRecordRejectsFieldDescent(t *testing.T) { func TestRecordSharesAReferenceAcrossColumns(t *testing.T) { dir := writeData(t, map[string]string{ "currency": `[{"format":"{code}","code":"AUD","symbol":"$"},{"format":"{code}","code":"EUR","symbol":"€"}]`, - "price": `{"format":"","code":"{/currency.code}","symbol":"{/currency.symbol}"}`, + "price": `{"format":"{code} {symbol}","code":"{/currency.code}","symbol":"{/currency.symbol}"}`, }) f := newGenerator(t, dir, WithSeed(1)) + symbols := map[string]string{"AUD": "$", "EUR": "€"} for i := 0; i < 100; i++ { r, err := f.FakeRecord("price") if err != nil { @@ -323,48 +331,63 @@ func TestRecordSharesAReferenceAcrossColumns(t *testing.T) { for _, c := range r.Columns() { m[c.Name] = c.Value } - switch m["code"] { - case "AUD": - if m["symbol"] != "$" { - t.Fatalf("record %q: code AUD but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"]) - } - case "EUR": - if m["symbol"] != "€" { - t.Fatalf("record %q: code EUR but symbol %q, want one currency draw across columns", r.JSON(), m["symbol"]) - } - default: - t.Fatalf("record %q has unexpected code %q", r.JSON(), m["code"]) + if symbol, known := symbols[m["code"]]; !known || m["symbol"] != symbol { + t.Fatalf("record %s, want one currency draw across columns", r.JSON()) + } + v := fake(t, f, "price") + if code, symbol, _ := strings.Cut(v, " "); symbol == "" || symbols[code] != symbol { + t.Fatalf("Fake(price) = %q, want its fields one currency draw, as the record's columns are", v) } } } -func TestRecordSharesAReferenceIntoAColumnRepeat(t *testing.T) { +func TestRepeatIterationsDrawReferencesAnew(t *testing.T) { dir := writeData(t, map[string]string{ - "currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`, - "order": `{"format":"","codes":{"format":"{/currency.code}","repeat":3,"separator":"-"}}`, + "party": `{"format":"{host}: {guests}","guests":{"format":"{/person.first} {/person.last}","repeat":3,"separator":", "},"host":"{/person.first} {/person.last}"}`, + "person": drawPeople, }) f := newGenerator(t, dir, WithSeed(1)) - for i := 0; i < 50; i++ { - r, err := f.FakeRecord("order") + differed := map[[2]int]bool{} + check := func(view string, names []string) { + t.Helper() + if len(names) != 4 { + t.Fatalf("%s rendered %q, want a host and three guests", view, names) + } + for i, name := range names { + if !onePerson(name) { + t.Fatalf("%s rendered %q, want each name one person", view, names) + } + for j := range names[:i] { + differed[[2]int{j, i}] = differed[[2]int{j, i}] || names[j] != name + } + } + } + for i := 0; i < 100; i++ { + r, err := f.FakeRecord("party") if err != nil { t.Fatal(err) } - parts := strings.Split(r.Columns()[0].Value, "-") - if len(parts) != 3 || parts[0] != parts[1] || parts[1] != parts[2] { - t.Fatalf("codes column = %q, want one shared draw across its repeat", r.Columns()[0].Value) + guests, host := r.Columns()[0].Value, r.Columns()[1].Value + check("FakeRecord", append([]string{host}, strings.Split(guests, ", ")...)) + host, guests, _ = strings.Cut(fake(t, f, "party"), ": ") + check("Fake", append([]string{host}, strings.Split(guests, ", ")...)) + } + for pair, ok := range differed { + if !ok { + t.Errorf("names %v never differed in 200 renders; the host and each repeat iteration are a draw of their own, so four people", pair) } } } -func TestRecordBareReferenceStaysIndependent(t *testing.T) { +func TestRecordGroupsDrawApart(t *testing.T) { dir := writeData(t, map[string]string{ - "currency": `[{"format":"{code}","code":"AUD"},{"format":"{code}","code":"EUR"}]`, - "order": `{"format":"","whole":"{/currency}","code":"{/currency.code}"}`, + "person": drawPeople, + "transfer": `{"format":"{from_first} {from_last} to {to_first} {to_last}","from_first":{"format":"{/person.first}","drawGroup":"from"},"from_last":{"format":"{/person.last}","drawGroup":"from"},"to_first":{"format":"{/person.first}","drawGroup":"to"},"to_last":{"format":"{/person.last}","drawGroup":"to"}}`, }) f := newGenerator(t, dir, WithSeed(1)) - sawMismatch := false + apart := map[string]bool{} for i := 0; i < 100; i++ { - r, err := f.FakeRecord("order") + r, err := f.FakeRecord("transfer") if err != nil { t.Fatal(err) } @@ -372,13 +395,18 @@ func TestRecordBareReferenceStaysIndependent(t *testing.T) { for _, c := range r.Columns() { m[c.Name] = c.Value } - if m["whole"] != m["code"] { - sawMismatch = true - break + from, to, _ := strings.Cut(fake(t, f, "transfer"), " to ") + for view, pair := range map[string][2]string{"FakeRecord": {m["from_first"] + " " + m["from_last"], m["to_first"] + " " + m["to_last"]}, "Fake": {from, to}} { + if !onePerson(pair[0]) || !onePerson(pair[1]) { + t.Fatalf("%s drew %q and %q, want each group one person", view, pair[0], pair[1]) + } + apart[view] = apart[view] || pair[0] != pair[1] } } - if !sawMismatch { - t.Fatal("a bare {/currency} column never disagreed with a tailed {/currency.code} column; a bare reference should draw independently") + for _, view := range []string{"FakeRecord", "Fake"} { + if !apart[view] { + t.Errorf("%s: groups from and to drew one person in 100 renders, want a draw each", view) + } } } diff --git a/reference.go b/reference.go index 08f209e..2fea914 100644 --- a/reference.go +++ b/reference.go @@ -72,14 +72,17 @@ func refSegments(name string, folder []string) ([]string, error) { // error, never a random render-time one. func linkRefs(root map[string]node) error { return eachTemplate(root, func(folder []string, path string, t *template) error { - return linkTemplateRefs(folder, path, t, root) + category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") + t.keyDrawGroup(category) + return linkTemplateRefs(folder, path, category, t, root) }) } -// linkTemplateRefs binds one template's references against root. A template with -// none is left untouched, so an inline format that references nothing costs only -// the refTokens scan. -func linkTemplateRefs(folder []string, path string, t *template, root map[string]node) error { +// linkTemplateRefs binds one template's references against root, refusing one that names the +// category it sits in: a category is a unit, and a reference back into it describes a draw other +// than the fields beside it. A template with no reference is left untouched, so an inline format +// that references nothing costs only the refTokens scan. +func linkTemplateRefs(folder []string, path, category string, t *template, root map[string]node) error { names := refTokens(t.format) if len(names) == 0 { return nil @@ -98,6 +101,9 @@ func linkTemplateRefs(folder []string, path string, t *template, root map[string return fmt.Errorf("%s: reference {%s}: %w", path, name, err) } key := "/" + strings.Join(head, ".") + if category != "" && key == "/"+category { + return fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", path, name) + } if err := checkPath(target, tail, key); err != nil { return fmt.Errorf("%s: reference {%s}: %w", path, name, err) } @@ -162,17 +168,17 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t } return nil } - var inFolder func(folder []string, children map[string]node) error - inFolder = func(folder []string, children map[string]node) error { + var inFolder func(dir []string, children map[string]node) error + inFolder = func(dir []string, children map[string]node) error { for _, name := range sortedNames(children) { - path := join(strings.Join(folder, "."), name) - if g, ok := children[name].(*group); ok { - if err := inFolder(append(folder[:len(folder):len(folder)], name), g.children); err != nil { + path := join(strings.Join(dir, "."), name) + if g, ok := children[name].(*folder); ok { + if err := inFolder(append(dir[:len(dir):len(dir)], name), g.children); err != nil { return err } continue } - if err := inCategory(folder, path, children[name]); err != nil { + if err := inCategory(dir, path, children[name]); err != nil { return err } } @@ -183,13 +189,13 @@ func eachTemplate(root map[string]node, fn func(folder []string, path string, t // resolveCategory walks a dotted path through the folders to the category it // names, returning that head, the node, and the tail left to read into it. A -// descent of its own rather than a walkPath: it walks groups only and returns +// descent of its own rather than a walkPath: it walks folders only and returns // where they end, not a leaf. func resolveCategory(root map[string]node, segments []string) (head []string, target node, tail []string, err error) { - var n node = &group{children: root} + var n node = &folder{children: root} i := 0 for ; i < len(segments); i++ { - g, ok := n.(*group) + g, ok := n.(*folder) if !ok { break } @@ -199,7 +205,7 @@ func resolveCategory(root map[string]node, segments []string) (head []string, ta } n = child } - if _, ok := n.(*group); ok { + if _, ok := n.(*folder); ok { return nil, nil, nil, fmt.Errorf("names a folder, not a value") } return segments[:i], n, segments[i:], nil diff --git a/reference_test.go b/reference_test.go index 65494d5..8219fd9 100644 --- a/reference_test.go +++ b/reference_test.go @@ -83,27 +83,25 @@ func TestReferenceErrors(t *testing.T) { "card": `"{/who.f}"`, }, "empty reference path": {"card": `"{/}"`}, - // A reference that leads back to its own value never terminates at render, - // so New must reject the cycle up front (direct, mutual, or chained). - "direct cycle": {"a": `"x{/a}"`}, - "mutual cycle": {"a": `"{/b}"`, "b": `"{/a}"`}, - "chain cycle": {"a": `"{/b}"`, "b": `"{/c}"`, "c": `"{/a}"`}, - // calc renders its operands, so a cycle through one must be caught too. - "calc operand cycle": {"x": `{"format":"{calc(y)}","y":"{/x}"}`}, + // A reference that leads back to its own value never terminates at render, so + // New must reject the cycle up front (mutual or chained). One into its own + // category is refused before the cycle walk reaches it, as a unit rule. + "a category referencing itself": {"a": `"x{/a}"`}, + "mutual cycle": {"a": `"{/b}"`, "b": `"{/a}"`}, + "chain cycle": {"a": `"{/b}"`, "b": `"{/c}"`, "c": `"{/a}"`}, + // calc renders its operands, so a reference through one is caught too. + "a calc operand into its own category": {"x": `{"format":"{calc(y)}","y":"{/x}"}`}, // A field its parent's format never renders is still reachable by dot path, - // so a cycle hiding in one must fail at New rather than at render. - "cycle in an unrendered field": {"cat": `{"format":"hi","x":"{/cat.x}"}`}, - "mutual cycle between unrendered fields": { + // so what hides in one must fail at New rather than at render. + "an unrendered field into its own category": {"cat": `{"format":"hi","x":"{/cat.x}"}`}, + "two unrendered fields into their own category": { "cat": `{"format":"hi","x":"{/cat.y}","y":"{/cat.x}"}`, }, - "cycle in an unrendered field of a choice arm": { - "cat": `{"format":"hi","x":"{/cat.x}"}`, - }, - // The shipped layout puts categories in folders, so a cycle one level down - // is the common case, not an edge case. - "cycle in a subfolder": {"sv_SE/a": `"x{/sv_SE.a}"`}, - "mutual cycle within a subfolder": {"sv_SE/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/sv_SE.a}"`}, - "mutual cycle across two folders": {"en_US/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/en_US.a}"`}, + // The shipped layout puts categories in folders, so one level down is the + // common case, not an edge case. + "a category in a subfolder referencing itself": {"sv_SE/a": `"x{/sv_SE.a}"`}, + "mutual cycle within a subfolder": {"sv_SE/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/sv_SE.a}"`}, + "mutual cycle across two folders": {"en_US/a": `"{/sv_SE.b}"`, "sv_SE/b": `"{/en_US.a}"`}, // ".." is reserved for bound references, so an authored key using it would // name a node nothing can reach and nothing would validate. "field key using the reference prefix": {"cat": `{"format":"hi","..x":"{/nope}"}`}, @@ -131,17 +129,26 @@ func TestDotPrefixedDataEntriesAreSkipped(t *testing.T) { } } -// TestReferenceFromUnrenderedFieldTerminates guards the cycle check against -// over-rejecting: a field the format never renders may point back at its own -// category, which terminates, and stays renderable by path. -func TestReferenceFromUnrenderedFieldTerminates(t *testing.T) { - dir := writeData(t, map[string]string{"cat": `{"format":"hi","x":"see {/cat}"}`}) - f := newGenerator(t, dir, WithSeed(1)) - if got := fake(t, f, "cat"); got != "hi" { - t.Fatalf("cat = %q, want hi", got) +// TestReferenceIntoItsOwnCategoryIsRejected pins the unit a category is: a reference +// back into it describes a draw other than the fields beside it, whether the format +// renders that field or not, so it is refused at load rather than left to disagree in +// the record view. +func TestReferenceIntoItsOwnCategoryIsRejected(t *testing.T) { + for name, file := range map[string]string{ + "the category whole": `{"format":"hi","x":"see {/cat}"}`, + "a field of its own": `{"format":"hi","x":"{/cat.y}","y":"1"}`, + "a path the format reads": `{"format":"{/cat.y}","y":"1"}`, + } { + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file}))) + if err == nil || !strings.Contains(err.Error(), "names the category it sits in") { + t.Errorf("%s: New = %v, want the reference into its own category refused", name, err) + } } - if got := fake(t, f, "cat.x"); got != "see hi" { - t.Fatalf("cat.x = %q, want \"see hi\"", got) + // Reading a sibling as a path is the spelling that stays. + if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ + "cat": `{"format":"{y.v}","y":{"format":"{v}","v":["1","2"]}}`, + }))); err != nil { + t.Errorf("New = %v, want the sibling path accepted", err) } } @@ -189,9 +196,9 @@ func TestNewErrorPathIsCanonical(t *testing.T) { want string }{ { - "cycle inside a choice arm", - map[string]string{"cat": `{"format":"hi","x":"{/cat.x}"}`}, - "fejkdata: reference cycle: cat.x -> /cat.x", + "cycle through another category", + map[string]string{"cat": `{"format":"hi","x":"{/hop}"}`, "hop": `"{/cat.x}"`}, + "fejkdata: reference cycle: cat.x -> /hop -> /cat.x", }, { "bad reference reached through another reference", @@ -255,15 +262,102 @@ func TestBareReferenceDrawsEachTime(t *testing.T) { } func TestReferenceOverlapIsRejected(t *testing.T) { - for name, file := range map[string]string{ - "head beside a path": `{"format":"{/cat.p} {/cat.p.first}","p":[{"format":"{first}","first":"A"},{"format":"{first}","first":"B"}]}`, - "sibling path beside a reference path": `{"format":"{p.first} {/cat.p.last}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`, + // A category never references itself, so the reads sit in a second category. + cat := `{"format":"x","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}` + for name, row := range map[string]string{ + "head beside a path": `"{/cat.p} {/cat.p.first}"`, + "sibling fields reading a level and a path into it": `{"format":"{a} {b}","a":"{/cat.p}","b":"{/cat.p.first}"}`, + "a field rendering the level a nested reference reads into": `{"format":"{x} {p}","x":"{/cat.p.first}","p":"{/cat.p}"}`, + "a bare reference beside a path into what it never renders": `{"format":"{a} {b}","a":"{/cat}","b":"{/cat.p.first}"}`, } { - _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": file}))) + _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row}))) if err == nil || !strings.Contains(err.Error(), "reads a path into") { t.Errorf("%s: New = %v, want the overlap rejected", name, err) } } + // A sibling path and a reference into the level it holds are the same overlap, + // and the reference reaches it from a category row renders. + if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ + "row": `{"format":"{p.first} {/hop}","p":[{"format":"{first}","first":"A","last":"1"},{"format":"{first}","first":"B","last":"2"}]}`, + "hop": `"{/row.p.last}"`, + }))); err == nil || !strings.Contains(err.Error(), "reads a path into") { + t.Errorf("New = %v, want a sibling path beside a reference into it rejected", err) + } + if _, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{ + "cat": cat, + "row": `{"format":"{a} {b}","a":{"format":"{/cat.p}","drawGroup":"g"},"b":"{/cat.p.first}"}`, + }))); err != nil { + t.Errorf("New = %v, want a level and a path into it accepted in groups of their own", err) + } +} + +const drawPeople = `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"},{"format":"{first} {last}","first":"Cy","last":"Young"}]` + +// onePerson reports whether name is the first name and surname of one drawPeople row. +func onePerson(name string) bool { + first, last, _ := strings.Cut(name, " ") + return last != "" && map[string]string{"Ada": "Lovelace", "Bo": "Ek", "Cy": "Young"}[first] == last +} + +func TestAReferencePathIsOneDrawPerRender(t *testing.T) { + dir := writeData(t, map[string]string{ + "apart": `{"format":"{a} & {b}","a":{"format":"{/person.first} {/person.last}","drawGroup":"x"},"b":{"format":"{/person.first} {/person.last}","drawGroup":"y"}}`, + "caller": `{"format":"{a} & {b}","a":{"format":"{/person.first} {/person.last}","drawGroup":"x"},"b":"{/pay}"}`, + "contact": `{"format":"{first} {last} <{email}>","email":"{lowercase(/person.first)}.{lowercase(/person.last)}@example.com","first":"{/person.first}","last":"{/person.last}"}`, + "iterations": `{"format":"{/person.first} {r}","drawGroup":"outer","r":{"format":"{a}={b}","repeat":3,"separator":",","a":"{/person.first}","b":{"format":"{/person.first}","drawGroup":"outer"}}}`, + "nested": `{"format":"{/person.first} {inner}","inner":"{/person.last}"}`, + "pair": `{"format":"{a} & {b}","a":"{/person.first} {/person.last}","b":"{/person.first} {/person.last}"}`, + "pay": `{"format":"{p}","p":{"format":"{/person.first} {/person.last}","drawGroup":"x"}}`, + "person": drawPeople, + }) + f := newGenerator(t, dir, WithSeed(1)) + apart, local, noGroup := false, false, false + for i := 0; i < 100; i++ { + a, b, _ := strings.Cut(fake(t, f, "caller"), " & ") + if !onePerson(a) || !onePerson(b) { + t.Fatalf("caller = %q & %q, want each one person", a, b) + } + local = local || a != b + _, iterations, _ := strings.Cut(fake(t, f, "iterations"), " ") + for _, pair := range strings.Split(iterations, ",") { + a, b, _ := strings.Cut(pair, "=") + noGroup = noGroup || a != b + } + if got := fake(t, f, "nested"); !onePerson(got) { + t.Fatalf("nested = %q, want a nested template's path one draw with its parent's", got) + } + name, email, _ := strings.Cut(fake(t, f, "contact"), " <") + first, last, _ := strings.Cut(name, " ") + if !onePerson(name) || email != strings.ToLower(first)+"."+strings.ToLower(last)+"@example.com>" { + t.Fatalf("contact = %q <%s, want first, last and email one person", name, email) + } + r, err := f.FakeRecord("contact") + if err != nil { + t.Fatal(err) + } + c := r.Columns() + if !onePerson(c[1].Value+" "+c[2].Value) || c[0].Value != strings.ToLower(c[1].Value)+"."+strings.ToLower(c[2].Value)+"@example.com" { + t.Fatalf("contact record %s, want first, last and email one person", r.JSON()) + } + a, b, _ = strings.Cut(fake(t, f, "pair"), " & ") + if !onePerson(a) || a != b { + t.Fatalf("pair = %q & %q, want both fields one person", a, b) + } + a, b, _ = strings.Cut(fake(t, f, "apart"), " & ") + if !onePerson(a) || !onePerson(b) { + t.Fatalf("apart = %q & %q, want each group one person", a, b) + } + apart = apart || a != b + } + if !apart { + t.Error("groups x and y drew one person in 100 renders, want a draw each") + } + if !local { + t.Error("group x in caller and group x in the pay it references drew one person in 100 renders; a group name is local to its category") + } + if !noGroup { + t.Error("an iteration's plain read and its group outer always agreed; a repeat iteration renders in no group, so they are a draw each") + } } func TestRelativeReferences(t *testing.T) { diff --git a/render.go b/render.go index ec12628..b29894c 100644 --- a/render.go +++ b/render.go @@ -20,14 +20,14 @@ type rng interface { func (f *Generator) Fake(path string) (string, error) { f.mu.Lock() defer f.mu.Unlock() - n, err := descend(f.rand, &group{children: f.categories}, strings.Split(path, ".")) + n, err := descend(f.rand, &folder{children: f.categories}, strings.Split(path, ".")) if err != nil { return "", fmt.Errorf("fejkdata: %s: %w", path, err) } - if _, ok := n.(*group); ok { + if _, ok := n.(*folder); ok { return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path) } - return render(f.rand, n, nil), nil + return renderOnce(f.rand, n), nil } // descend walks named fields to the node a path names. It is the one render-side @@ -50,20 +50,21 @@ func descend(s *session, root node, segments []string) (node, error) { } // render evaluates a compiled node to a string. compile validates every node up -// front, so rendering a compiled tree cannot fail. refScope carries the draws a -// reference shares beyond its own expansion; nil keeps every reference local. -func render(s *session, n node, refScope *draws) string { +// front, so rendering a compiled tree cannot fail. sc holds the reference draws the +// render shares; each repeat iteration renders over draws of its own. +func render(s *session, n node, sc drawScope) string { switch n := n.(type) { case *choice: - return render(s, pick(s, n), refScope) + return render(s, pick(s, n), sc) case *null: return "" case *template: + sc = sc.in(n) if n.repeat == 1 { if n.fixed { return n.lit } - return expand(s, n, refScope) + return expand(s, n, sc) } var b strings.Builder b.Grow(n.repeat * (n.grow + len(n.separator))) @@ -71,7 +72,7 @@ func render(s *session, n node, refScope *draws) string { if i > 0 { b.WriteString(n.separator) } - b.WriteString(expand(s, n, refScope)) + b.WriteString(expandAnew(s, n)) } return b.String() default: @@ -79,6 +80,15 @@ func render(s *session, n node, refScope *draws) string { } } +// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into +// render's loop, its draw set would move to the heap. +// +//go:noinline +func expandAnew(s *session, t *template) string { + var set drawSet + return expand(s, t, drawScope{set: &set}) +} + // pick selects one item. Uniform choices are O(1); weighted choices are an // O(log n) search over precomputed cumulative weights. compile guarantees a // non-empty choice and a finite positive total, so the index is always in range. @@ -93,12 +103,12 @@ func pick(r rng, c *choice) node { // expand renders a template's compiled ops. compile validated every token, so this // cannot fail. -func expand(s *session, t *template, refScope *draws) string { +func expand(s *session, t *template, sc drawScope) string { var b strings.Builder b.Grow(t.grow) // One draw per held name, for this expansion only: a nested template and each // repeat iteration get their own, since each is its own expansion. A reference - // reads the caller's scope instead, whenever one was supplied. + // path reads the render's draws in sc instead. var held *draws if len(t.held) > 0 { held = &draws{ @@ -112,7 +122,7 @@ func expand(s *session, t *template, refScope *draws) string { case 'l': b.WriteString(o.lit) case 'f': - b.WriteString(readField(s, t, held, refScope, o.arms[s.IntN(len(o.arms))]).text) + b.WriteString(readField(s, t, held, sc, o.arms[s.IntN(len(o.arms))]).text) case 'b': // Read before the call, so the value a calc computes is the value the // format showed. calcVars fixed the order op.operands holds. @@ -120,7 +130,7 @@ func expand(s *session, t *template, refScope *draws) string { if len(o.operands) > 0 { operands = make([]string, len(o.operands)) for j, a := range o.operands { - operands[j] = readField(s, t, held, refScope, a).text + operands[j] = readField(s, t, held, sc, a).text } } b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far diff --git a/template_test.go b/template_test.go index 080972e..12d9ae7 100644 --- a/template_test.go +++ b/template_test.go @@ -38,7 +38,7 @@ func compiled(t *testing.T, s string) node { func mustRender(t *testing.T, f *Generator, s string) string { t.Helper() - return render(f.rand, compiled(t, s), nil) + return renderOnce(f.rand, compiled(t, s)) } func TestStringIsAFormat(t *testing.T) { @@ -315,7 +315,7 @@ func TestGrowIsALowerBound(t *testing.T) { t.Fatalf("format %q did not compile to a template", format) } for i := 0; i < 50; i++ { - if got := len(expand(f.rand, tmpl, nil)); got < tmpl.grow { + if got := len(expand(f.rand, tmpl, drawScope{set: &drawSet{}})); got < tmpl.grow { t.Errorf("format %q: expand emitted %d bytes, below grow %d", format, got, tmpl.grow) } } diff --git a/todo.md b/todo.md index 1b0f4a6..2d65711 100644 --- a/todo.md +++ b/todo.md @@ -2,22 +2,6 @@ ## Before v0.1.0 -The record API lands first, so the data update can use it. - -### Record API - -- Draw groups — references into one category share one draw per render (one - record, or one `Fake`) in both views; each `repeat` iteration draws anew, and a - bare reference draws each time. An option naming a draw group splits a render - into several entities. Replaces the Decision "A record shares one reference draw - per category". Each expectation becomes a test: - - `first`, `last` and `email` reading `person` → one person - - `from_first`/`from_last` grouped `from`, `to_first`/`to_last` grouped `to` → two people - - `host`, plus `guests` repeated 3 times → four people - - `code` and `symbol` sibling fields reading `currency`, as `{code} {symbol}` → a matching pair - - `{a} & {b}`, each reading `person` → one person, or two when `a` and `b` name different groups - - two bare `{/sv_SE.word}` → two words - ### Data - Major data update. Shipped categories render as records with their building