From 23e426016decfa20b63cdc458ee5aab1c3476f5f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 01:51:14 +0200 Subject: [PATCH] Bind a pick to a name with {ref as n}, and read it with {n} and {n.path} --- AGENTS.md | 1 + CHANGELOG.md | 8 ++ README.md | 30 ++++- doc.go | 5 +- docs/decisions.md | 13 ++ draws.go | 16 ++- format.go | 114 ++++++++++++---- inline.go | 10 +- name.go | 331 ++++++++++++++++++++++++++++++++++++++++++++++ node.go | 35 +++-- path.go | 16 +++ record.go | 6 +- reference.go | 13 +- render.go | 39 ++++-- table.go | 23 +++- todo.md | 12 +- 16 files changed, 609 insertions(+), 63 deletions(-) create mode 100644 name.go diff --git a/AGENTS.md b/AGENTS.md index 88a90b3..6babe68 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -118,3 +118,4 @@ In [docs/decisions.md](docs/decisions.md): - The US given names come from a mirror of the SSA file - `List` advertises direct descents only - A path draws through its compiled steps +- A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read diff --git a/CHANGELOG.md b/CHANGELOG.md index 0dae6ef..230d153 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,14 @@ replacement, and each removed path, column or flag. ### Added +- A name keeps one pick: `{/misc.territory as t}` binds one row of `misc.territory` to `t` + and prints nothing, and `{t}`, `{t.capital}` and `{t.calling-code}` read that one row. + A name lives in the category binding it, so every field and record column of the + category reads one pick, and a name a `repeat` binds picks again on every iteration. + Refused at `New`: a name bound twice in one category, or inside a `repeat` and outside + it; a name that is a field or an option; a binding of anything but a reference; a + binding nothing reads; a path through a name that selects a row; and a binding in a + table. A field, column, category or folder name may not contain ` as `. - First release: the CLI, the library and the shipped data set. - Table categories: a category JSON naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and `parent`; a path selects a row by key or name, diff --git a/README.md b/README.md index 00b1082..6ca5188 100644 --- a/README.md +++ b/README.md @@ -359,7 +359,8 @@ Every character is literal except a `{…}` token: | Token | Renders | |-------|---------| -| `{name}` | the sibling field `name` | +| `{name}` | the sibling field `name`, or the pick bound to the name ([Names](#names)) | +| `{ref as name}` | nothing: binds one pick of the reference `ref` to `name` ([Names](#names)) | | `{name.field}` | `field` of one draw of `name` ([Correlated fields](#correlated-fields)) | | `{a\|b}` | one of the named fields, even odds | | `{fn(args)}` | a builtin ([Functions](#functions)) | @@ -574,7 +575,8 @@ or the reference draws it. `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options; **any other key is a field** (see [Decisions](docs/decisions.md#options-and-fields-share-one-namespace)), and `rows` makes a category -a [table](#table), so no template carries a field of that name. An object that does nothing a +a [table](#table), so no template carries a field of that name. A [name](#names) shares the +namespace too. An object that does nothing a string can't — only a `format` — is rejected naming the string, as is a one-item choice naming its item. @@ -696,6 +698,29 @@ unknown, names a folder, has no folder above, reads a field not every variant of a choice carries, or names the category the reference sits in, and a reference that leads back to its own value, directly, mutually or through a chain. +### Names + +`{ref as n}` binds one pick of a reference to the name `n` and prints nothing. `{n}` then +prints that pick, and `{n.path}` reads a path through it, so every read of `n` describes one +row: + +```json +"{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}" +``` + +Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn +once, so `{n}` and `{n.path}` agree, and a field `n` renders twice prints the same text twice. +A name is drawn on its first read, and lives in the category binding it: any field of the +category reads it, and a record's columns read one pick. Each render of the category picks +anew, and so does each bare reference to it. A name a `repeat` binds picks again on every +iteration, and one bound outside the repeat keeps its pick on every line. + +Rejected at load: a name bound twice in one category, or both inside a `repeat` and outside +it; a name that is a field or an option; a binding of anything but a reference; a binding +nothing reads; a path through a name that selects a row, which the reference selects instead; +and a binding in a table's format or cell. A field, column, category or folder name may not +contain ` as `. + ### Draw group A template may carry `drawGroup` to hold its reference draws apart: every reference path @@ -889,6 +914,7 @@ doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, the embedded data set, List node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws +name.go names: {ref as n} bindings, the scopes they live in, and the picks a render keeps under them path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make record.go records: Record, the JSON/CSV/SQL serializers, and their entry points diff --git a/doc.go b/doc.go index 2f0fd96..7841405 100644 --- a/doc.go +++ b/doc.go @@ -27,7 +27,10 @@ package fejkdata // an expansion holds a sibling path's head and an operand's field, its draw // group a reference path's: `formatOps.held` and `heldCheck` an expansion's, // `readReference` a draw group's. -// - memo — what a hold or a draw group keeps its draws in, the variant drawn at +// - named pick — one draw of a reference a binding token, {ref as n}, gives a name, kept for one +// render of the scope it is bound in, a category or a repeat iteration, and drawn on its +// first read: `nameBinding`, `nameScope`, `namedPick`, `pickFrame`. +// - memo — what a hold, a draw group or a named pick keeps its draws in, the variant drawn at // each level and the value each path read: `drawMemo`, `readMemo`. // - draw group — reference draws kept apart inside one render: // `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a diff --git a/docs/decisions.md b/docs/decisions.md index 2eb6010..73658b5 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -888,3 +888,16 @@ since Go tracks a struct's fields as one, and a repeat of a reference path rose to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node, so no benchmark gained an allocation. Valid while Go's escape analysis tracks a struct's fields as one. + +## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read + +2026-10-03, larv-review. Goals 5.3 and 5.6. +`bindNames` scopes a name at compile, and a render keeps one `pickFrame` per scope. One +field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside +`"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category is the smallest unit holding every field and record column. The +scope is lexical: a category referencing another never sees its names, so what a category +renders never depends on who references it. A pick is drawn on its first read, so a record's +columns, rendered in name order, read one pick whichever binds it. A read entering a category +past its root, `Fake("cat.field")` or `{/cat.field}`, keeps the category's frame in the memo of +that read: per render and group, as the read's own draw is. Valid while a name is read only +inside the category binding it. diff --git a/draws.go b/draws.go index a7f5894..f49728b 100644 --- a/draws.go +++ b/draws.go @@ -2,12 +2,14 @@ package fejkdata import "strings" -// drawMemo is what a hold or a draw group has drawn: the variant each level was drawn as, so -// every path under it reads one variant; and the value each read produced, by its path, so the -// same read written twice reads one value. +// drawMemo is what a hold, a draw group or a named pick has drawn: the variant each level was +// drawn as, so every path under it reads one variant; the value each read produced, by its path, +// so the same read written twice reads one value; and the frame of each name scope a read +// entered past its owner. type drawMemo struct { variant map[string]node value map[string]readValue + frames map[*nameScope]*pickFrame } // groupDraws is what one render's draw group has drawn for its reference paths: a memo, and the @@ -27,11 +29,12 @@ type readValue struct { type renderTrace func(group string, row renderedRow, a arm) // renderDraws is one render's reference draws: the unnamed draw group's, and each named one's; -// and a test's trace of its reads. +// a test's trace of its reads; and the frames of the name scopes rendering. type renderDraws struct { unnamed groupDraws named map[string]*groupDraws trace renderTrace + frames *frameStack } // renderScope is where a render reads its reference paths: its reference draws, in the draw group of the @@ -42,6 +45,11 @@ type renderScope struct { draws *renderDraws group string row renderedRow + entry *drawMemo // the memo of the read that entered what renders + // pick is the named pick what renders is part of, at pickKey, its path from the name; nil + // outside one. + pick *namedPick + pickKey string } type renderedRow struct { diff --git a/format.go b/format.go index e962806..185be82 100644 --- a/format.go +++ b/format.go @@ -1,6 +1,7 @@ package fejkdata import ( + "errors" "fmt" "strings" ) @@ -63,7 +64,8 @@ type formatToken struct { body string // the braces' content, as written fn string // builtinCall args []string // builtinCall - names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads + names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads; nameBind: the reference it binds + bound string // nameBind: the name } type tokenKind uint8 @@ -72,8 +74,12 @@ const ( builtinCall tokenKind = iota + 1 nameRead literalRun + nameBind ) +// asWord separates a binding's reference from the name it binds: {/person as p}. +const asWord = " as " + // parseFormat is the one reading of a format's tokens: a '(' outside a selector makes // a token a call. func parseFormat(format string) ([]formatToken, error) { @@ -91,6 +97,10 @@ func parseFormat(format string) ([]formatToken, error) { return nil } flush() + if ref, name, binds := cutOutside(u.body, asWord); binds && indexOutside(u.body, '(') < 0 { + toks = append(toks, formatToken{kind: nameBind, body: u.body, names: []string{ref}, bound: name}) + return nil + } if indexOutside(u.body, '(') < 0 { toks = append(toks, formatToken{kind: nameRead, body: u.body, names: splitOutside(u.body, '|')}) return nil @@ -195,51 +205,103 @@ func checkFunc(tok formatToken, fields map[string]node) error { return nil } -func parseChecked(format string, fields map[string]node) ([]formatToken, error) { +func parseChecked(format string, fields map[string]node) ([]formatToken, []unboundRead, error) { toks, err := parseFormat(format) if err != nil { - return nil, err + return nil, nil, err } - return toks, checkTokens(toks, fields) + unbound, err := checkTokens(toks, fields) + return toks, unbound, err } +// unboundRead is a token reading a head no field holds, which only a name bound around the +// template can answer; err is the refusal where none does. +type unboundRead struct { + head string + err error +} + +// fieldMiss is a head no field of the template holds. +type fieldMiss struct { + head string + err error +} + +func (m *fieldMiss) Error() string { return m.err.Error() } + // checkTokens proves every parsed token names an existing field or a known function, -// so a typo'd or dangling reference is a New-time error. -func checkTokens(toks []formatToken, fields map[string]node) error { +// so a typo'd or dangling reference is a New-time error. A head no field holds comes +// back unbound, for bindNames to look up among the names. +func checkTokens(toks []formatToken, fields map[string]node) ([]unboundRead, error) { + var unbound []unboundRead for _, t := range toks { - if err := checkToken(t, fields); err != nil { - return err + u, err := checkToken(t, fields) + if err != nil { + return nil, err } + unbound = append(unbound, u...) } - return nil + return unbound, nil } -func checkToken(t formatToken, fields map[string]node) error { +func checkToken(t formatToken, fields map[string]node) ([]unboundRead, error) { switch t.kind { case builtinCall: - return checkFunc(t, fields) + return nil, checkFunc(t, fields) + case nameBind: + return nil, checkBind(t) case nameRead: + var unbound []unboundRead for _, name := range t.names { if isRef(name) { if _, _, err := refShape(name); err != nil { - return fmt.Errorf("token {%s}: %w", t.body, err) + return nil, fmt.Errorf("token {%s}: %w", t.body, err) } continue // its target is checked at New (see linkRefs) } - if err := checkArm(name, fields, len(t.names) == 1); err != nil { - return fmt.Errorf("token {%s}: %w", t.body, err) + err := checkArm(name, fields, len(t.names) == 1) + var miss *fieldMiss + if errors.As(err, &miss) { + unbound = append(unbound, unboundRead{miss.head, fmt.Errorf("token {%s}: %w", t.body, err)}) + continue + } + if err != nil { + return nil, fmt.Errorf("token {%s}: %w", t.body, err) } } // Last, so an arm broken on its own terms is reported as that: a repeat is // the consequence of such a mistake, not the mistake itself. - return checkNoRepeatedArm(t.body, t.names) + return unbound, checkNoRepeatedArm(t.body, t.names) + } + return nil, nil +} + +// checkBind proves a {ref as name} token binds a reference to a name no other token +// could read as something else. +func checkBind(t formatToken) error { + ref, name := t.names[0], t.bound + if trimmed := strings.TrimSpace(ref) + asWord + strings.TrimSpace(name); trimmed != t.body { + return fmt.Errorf("token {%s}: write {%s}", t.body, trimmed) + } + if !isRef(ref) { + return fmt.Errorf("token {%s}: %q is no reference; a name binds a pick of a reference, which starts with /, . or ..", t.body, ref) + } + if _, _, err := refShape(ref); err != nil { + return fmt.Errorf("token {%s}: %w", t.body, err) + } + if isOption(name) { + return fmt.Errorf("token {%s}: %q is an option and can never be a name", t.body, name) + } + if err := checkName(name); err != nil { + return fmt.Errorf("token {%s}: name %w", t.body, err) } return nil } // checkArm validates one sibling name or path against a template's fields. // wholeToken says the name is the token's entire body, so {/name} would render -// the same value and can be offered as the reference spelling. +// the same value and can be offered as the reference spelling. A head no field +// holds is a *fieldMiss. func checkArm(name string, fields map[string]node, wholeToken bool) error { a := splitArm(name, nil) if err := checkSegments(a); err != nil { @@ -258,9 +320,9 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error { if wholeToken && hintableRef(name) { hint = fmt.Sprintf(" — write {/%s} to reference the data", name) } - return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint) + return &fieldMiss{a.head, fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint)} } - return fmt.Errorf("no field %q", a.head) + return &fieldMiss{a.head, fmt.Errorf("no field %q", a.head)} } if err := checkPathResolves(field, a.tail, a.head); err != nil { return fmt.Errorf("field %q: %w", a.head, err) @@ -316,6 +378,7 @@ type arm struct { leaves []node // every node the path may land on, one per variant it passes cover node // what holding the path pins: the first choice it passes, else its leaf kind armKind + bind *nameBinding // namedRead: the binding of the name it reads through } // armKind is how expand reads an arm, fixed at compile. @@ -325,6 +388,7 @@ const ( freshRead armKind = iota // drawn afresh at every read heldRead // drawn once per expansion, kept in its hold refPathRead // drawn once per draw group, kept in its memo + namedRead // read through a name, kept in its pick ) func (a arm) isRefPath() bool { @@ -439,10 +503,14 @@ func (c *formatOps) function(tok formatToken, refs map[string]refBinding) { c.ops = append(c.ops, op{formatToken: tok, call: builtins[tok.fn].prep(tok.args), operands: operands}) } -func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { +func (c *formatOps) field(tok formatToken, refs map[string]refBinding, isName func(string) bool) { arms := make([]arm, len(tok.names)) for i, name := range tok.names { arms[i] = splitArm(name, refs) + if isName(arms[i].head) { + arms[i].kind = namedRead + continue + } if len(arms[i].tail) > 0 { c.holdName(arms[i], "token {"+arms[i].spelling+"}") } @@ -450,9 +518,9 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding) { c.ops = append(c.ops, op{formatToken: tok, arms: arms}) } -// compileFormat compiles a parsed format. Call checkTokens first: it is what proves -// every token valid. -func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps { +// compileFormat compiles a parsed format, a binding to nothing, since it prints nothing. Call +// checkTokens first: it is what proves every token valid. +func compileFormat(toks []formatToken, refs map[string]refBinding, isName func(head string) bool) formatOps { var c formatOps for _, tok := range toks { switch tok.kind { @@ -462,7 +530,7 @@ func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps { case builtinCall: c.function(tok, refs) case nameRead: - c.field(tok, refs) + c.field(tok, refs, isName) } } return c diff --git a/inline.go b/inline.go index f03fda9..e7fec5c 100644 --- a/inline.go +++ b/inline.go @@ -163,7 +163,7 @@ func inlineBinding(n node, label string, root map[string]node) binding { // linkNodeRefs binds the references in an inline node's templates against the // loaded tree. func linkNodeRefs(scope nodeScope, root map[string]node) error { - return scope(func(path string, m node) error { + if err := scope(func(path string, m node) error { t, ok := m.(*template) if !ok { return nil @@ -178,5 +178,13 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error { } } return linkTemplate(nil, path, "", t, root) + }); err != nil { + return err + } + return scope(func(path string, m node) error { + if t, ok := m.(*template); ok { + return linkNames(path, t) + } + return nil }) } diff --git a/name.go b/name.go new file mode 100644 index 0000000..b87b7e7 --- /dev/null +++ b/name.go @@ -0,0 +1,331 @@ +package fejkdata + +import ( + "fmt" + "strings" +) + +// nameScope is where a name is bound: a category, or one iteration of a repeat inside it. +// Every token of the category outside a nested repeat reads the category's names, so one +// field may bind a name and another read it. +type nameScope struct { + up *nameScope + owner node // the category's root, or the repeat + bindings map[string]*nameBinding + order []*nameBinding +} + +// nameBinding is one {ref as name} token: the scope it binds name in, its index among +// that scope's bindings, and, once linked, the head node and tail its reference reads. +type nameBinding struct { + name string + ref string + body string + where string // the fields reaching the binding template, as a compile error spells them + scope *nameScope + index int + binder *template + read bool + head node + tail []string +} + +func (sc *nameScope) lookup(name string) *nameBinding { + for ; sc != nil; sc = sc.up { + if b, ok := sc.bindings[name]; ok { + return b + } + } + return nil +} + +// bindNames gives every template of a compiled category or inline template the scope its +// names live in, and refuses a name bound twice along one chain of scopes, a name a field +// spells too, a read no field or name answers, and a binding nothing reads. +func bindNames(root node) error { + var scopes []*nameScope + var gather func(n node, scope *nameScope, where string) error + gather = func(n node, scope *nameScope, where string) error { + if t, isTemplate := n.(*template); isTemplate { + if t.repeat > 1 && n != root { + scope = &nameScope{up: scope, owner: t} + scopes = append(scopes, scope) + } + t.scope = scope + for _, tok := range t.tokens { + if tok.kind != nameBind { + continue + } + if err := scope.bind(tok, t, where); err != nil { + return err + } + } + } + return eachContained(n, where, func(c node, where string) error { return gather(c, scope, where) }) + } + top := &nameScope{owner: root} + scopes = append(scopes, top) + if err := gather(root, top, ""); err != nil { + return err + } + if err := resolveReads(root, ""); err != nil { + return err + } + for _, sc := range scopes { + if err := sc.close(); err != nil { + return err + } + } + return nil +} + +// close refuses a binding nothing reads, and gives the owner of a scope binding names the scope. +func (sc *nameScope) close() error { + for _, b := range sc.order { + if !b.read { + return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name) + } + } + if len(sc.order) == 0 { + return nil + } + switch o := sc.owner.(type) { + case *template: + o.ownScope = sc + case *choice: + o.ownScope = sc + } + return nil +} + +// eachContained calls fn for each node n contains, with where naming the fields reaching it as +// compile errors do. +func eachContained(n node, where string, fn func(c node, where string) error) error { + for _, c := range contained(n) { + next := where + if c.name != "" { + next += fmt.Sprintf("field %q: ", c.name) + } + if err := fn(c.node, next); err != nil { + return err + } + } + return nil +} + +func (sc *nameScope) bind(tok formatToken, binder *template, where string) error { + if b, twice := sc.bindings[tok.bound]; twice { + return fmt.Errorf("%stoken {%s}: name %q is bound twice in one scope, by {%s} too; rename one", where, tok.body, tok.bound, b.body) + } + if b := sc.up.lookup(tok.bound); b != nil { + return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat already, by {%s}; rename one", where, tok.body, tok.bound, b.body) + } + b := &nameBinding{name: tok.bound, ref: tok.names[0], body: tok.body, where: where, scope: sc, index: len(sc.order), binder: binder} + if sc.bindings == nil { + sc.bindings = map[string]*nameBinding{} + } + sc.bindings[b.name] = b + sc.order = append(sc.order, b) + return nil +} + +// resolveReads answers each unbound read under n with a name its template sees, fields before the +// format as compile reports them, and refuses a field spelling a name. +func resolveReads(n node, where string) error { + if err := eachContained(n, where, resolveReads); err != nil { + return err + } + t, isTemplate := n.(*template) + if !isTemplate { + return nil + } + for _, name := range sortedNames(t.fields) { + if b := t.scope.lookup(name); b != nil { + return fmt.Errorf("%stoken {%s}: name %q is a field too; options, fields and names share one namespace, so rename one", b.where, b.body, name) + } + } + for _, u := range t.unbound { + b := t.scope.lookup(u.head) + if b == nil { + return fmt.Errorf("%s%w", where, u.err) + } + b.read = true + } + return nil +} + +// isName reports whether a head t reads is a name: neither a reference nor a field. +func (t *template) isName(head string) bool { + return !isRef(head) && t.fields[head] == nil && t.scope.lookup(head) != nil +} + +// linkNames compiles t's reads of a name as paths from the head its binding's reference names, once +// every template is linked, so the binder has resolved that reference. +func linkNames(path string, t *template) error { + for i := range t.compiled.ops { + for j := range t.compiled.ops[i].arms { + a := &t.compiled.ops[i].arms[j] + if a.kind != namedRead { + continue + } + if err := linkName(t, a); err != nil { + return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), a.spelling, err) + } + } + } + return nil +} + +func linkName(t *template, a *arm) error { + b := t.scope.lookup(a.head) + if b.head == nil { + ref := b.binder.link.refs[b.ref] + b.head, b.tail = b.binder.link.refHeads[ref.head], ref.tail + } + if hasSelector(a.tail) { + return fmt.Errorf("a path through name %q selects no row; select it in the reference %q binds", a.head, a.head) + } + full := append(b.tail[:len(b.tail):len(b.tail)], a.tail...) + if err := checkPathResolves(b.head, full, a.head); err != nil { + return err + } + w := compilePath(b.head, full) + a.bind, a.steps, a.leaves, a.cover = b, w.steps, w.leaves, w.cover + a.levels = make([]string, len(full)+1) + for i := range a.levels { + a.levels[i] = joinSegments(full[:i]) + } + a.path = joinSegments(full) + return nil +} + +// namedPick is one draw of a name: the variant drawn at each level under it, the value each +// path through it read, keyed by the path from the name, and the table rows they pinned. +type namedPick struct { + memo drawMemo + pins pinSet +} + +// pickFrame is one render of a name scope: a pick per binding, each drawn on its first read. +type pickFrame struct { + scope *nameScope + picks []namedPick +} + +func newPickFrame(scope *nameScope) *pickFrame { + return &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))} +} + +// frameStack is the frames of the name scopes rendering, innermost last. A render's draws point +// at it, and a repeat iteration's share it: holding the frames in the draws, or in a renderScope, +// would move every render's draws to the heap. +type frameStack struct { + frames []*pickFrame +} + +// pushFrame opens a frame of scope until popFrame closes it, returning the mark popFrame takes. +func (d *renderDraws) pushFrame(scope *nameScope) int { + if d.frames == nil { + d.frames = &frameStack{} + } + mark := len(d.frames.frames) + d.frames.frames = append(d.frames.frames, newPickFrame(scope)) + return mark +} + +func (d *renderDraws) popFrame(mark int) { d.frames.frames = d.frames.frames[:mark] } + +// pickOf is b's pick in the frame rendering its scope. A read entering the scope past its owner, +// as Fake("cat.field") or {/cat.field} do, keeps the frame in the memo of that read. +func (sc renderScope) pickOf(b *nameBinding) *namedPick { + if stack := sc.draws.frames; stack != nil { + for i := len(stack.frames) - 1; i >= 0; i-- { + if f := stack.frames[i]; f.scope == b.scope { + return &f.picks[b.index] + } + } + } + memo := sc.entry + if memo == nil { + memo = &sc.groupDraws().memo + } + f, ok := memo.frames[b.scope] + if !ok { + f = newPickFrame(b.scope) + if memo.frames == nil { + memo.frames = map[*nameScope]*pickFrame{} + } + memo.frames[b.scope] = f + } + return &f.picks[b.index] +} + +func readName(s *generatorState, sc renderScope, a arm) readValue { + return readPicked(s, sc.pickOf(a.bind), a.bind.head, a.steps, a.levels, a.path, sc) +} + +// drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one. +func (sc renderScope) drawRowOf(s *generatorState, t *table) int { + if sc.pick != nil { + return t.drawIn(s, &sc.pick.pins) + } + return t.drawRow(s) +} + +// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey: once per pick, +// by its path from the name. +func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue { + levels := make([]string, len(a.levels)) + for i, l := range a.levels { + levels[i] = underKey(sc.pickKey, l) + } + return readPicked(s, sc.pick, t.head(a.head), a.steps, levels, underKey(sc.pickKey, a.path), sc) +} + +// underKey is the key of path under the pick key prefix. It never returns prefix itself, which a +// memo would keep: anything a renderScope holds reaching the heap moves every render's draws there. +func underKey(prefix, path string) string { + if prefix == "" { + return path + } + return prefix + "." + path +} + +// readPicked draws the path key names under pick p, once, and renders what it lands on as part of p. +func readPicked(s *generatorState, p *namedPick, head node, steps []pathStep, levels []string, key string, sc renderScope) readValue { + if r, done := p.memo.value[key]; done { + return r + } + leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels) + mark := -1 + if c, isChoice := leaf.(*choice); isChoice { + if c.ownScope != nil { + mark = sc.draws.pushFrame(c.ownScope) + } + leaf = p.memo.variantOf(s, c, key) + } + sc = sc.at(leaf, &p.pins) + sc.pick, sc.pickKey, sc.entry = p, key, &p.memo + r := renderLeaf(s, leaf, sc) + if mark >= 0 { + sc.draws.popFrame(mark) + } + if p.memo.value == nil { + p.memo.value = map[string]readValue{} + } + p.memo.value[key] = r + return r +} + +// variantOf is the variant of c drawn at level, drawn now where none was. +func (m *drawMemo) variantOf(s *generatorState, c *choice, level string) node { + n, drew := m.variant[level] + if !drew { + n = resolveChoice(s, c) + if m.variant == nil { + m.variant = map[string]node{} + } + m.variant[strings.Clone(level)] = n + } + return n +} diff --git a/node.go b/node.go index 841a324..9920cc7 100644 --- a/node.go +++ b/node.go @@ -25,9 +25,10 @@ func (*folder) isNode() {} // relative dot paths every item can address, so carriedByAll and List both read the one // answer to what a path may reach through this choice. type choice struct { - items []node - cum []float64 - shared map[string]bool + items []node + cum []float64 + shared map[string]bool + ownScope *nameScope // set on a category's root binding names, by bindNames } func (*choice) isNode() {} @@ -53,6 +54,11 @@ type template struct { fromString bool // written as a JSON string rather than an object isRecord bool // compiled at the top without a repeat, so its fields are record columns site tableSite + unbound []unboundRead // the heads its tokens read that no field holds + + // Filled by `bindNames`, from the compiled category: + scope *nameScope // where its tokens look a name up + ownScope *nameScope // the scope it renders a frame of: a category's, on its root, or a repeat's, per iteration // Filled by `linkTemplate`, from the assembled tree: link templateLink @@ -105,7 +111,11 @@ func (t *template) head(name string) node { // compile converts parsed JSON — a category or an inline template — into a node tree, // validating structure up front. func compile(v any) (node, error) { - return compileAt(v, atTop) + n, err := compileAt(v, atTop) + if err != nil { + return nil, err + } + return n, bindNames(n) } // compileCategory compiles a data file's value, which may be a table over a rows @@ -220,11 +230,17 @@ func jsonKind(v any) string { } func compileString(s string, site tableSite) (*template, error) { - toks, err := parseChecked(s, nil) + toks, unbound, err := parseChecked(s, nil) if err != nil { return nil, err } - return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site}, nil + if site.table != nil { + if err := checkTableToks(toks, unbound); err != nil { + return nil, err + } + unbound = nil + } + return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site, unbound: unbound}, nil } // fixedText is one render's output when the format holds no token; repeat is the caller's. @@ -329,14 +345,14 @@ func compileTemplate(m map[string]any, pos position) (node, error) { if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" { return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format) } - toks, err := parseChecked(o.format, fields) + toks, unbound, err := parseChecked(o.format, fields) if err != nil { return nil, err } if err := checkNestedDrawGroup(fields, o.group); err != nil { return nil, err } - return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn}, nil + return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn, unbound: unbound}, nil } // templateOptions is what a template object's option keys say. @@ -525,6 +541,9 @@ func checkName(name string) error { if name == "-" { return fmt.Errorf(`%q is reserved: the struct tag fake:"-" leaves a field unfilled, so no tag could read it; rename it`, name) } + if strings.Contains(name, asWord) { + return fmt.Errorf("%q contains %q, which a token reads as binding a name; rename it", name, asWord) + } if i := strings.IndexAny(name, reservedInName); i >= 0 { return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path, {token} and JSON grammars reserve", name, name[i:i+1], reservedList) diff --git a/path.go b/path.go index ed99016..0666214 100644 --- a/path.go +++ b/path.go @@ -87,6 +87,22 @@ func indexOutside(s string, c byte) int { return -1 } +// cutOutside cuts s around the first sep outside a [selector]. +func cutOutside(s, sep string) (before, after string, found bool) { + depth := 0 + for i := 0; i < len(s); i++ { + switch { + case s[i] == '[': + depth++ + case s[i] == ']' && depth > 0: + depth-- + case depth == 0 && strings.HasPrefix(s[i:], sep): + return s[:i], s[i+len(sep):], true + } + } + return s, "", false +} + func splitOutside(s string, c byte) []string { var parts []string for { diff --git a/record.go b/record.go index 3b001e7..85ab697 100644 --- a/record.go +++ b/record.go @@ -22,7 +22,7 @@ type Column struct { // Record is one record rendered from a template: every direct field is a column, // listed in name order. Each column is its own expansion, so a sibling field is // local to it, while a reference that reads a path is drawn once for the whole -// record, per group. +// record, per group, and a name once for the whole record. type Record struct { columns []Column } @@ -263,6 +263,10 @@ func recordOf(n node) (*template, []Column, error) { // over sc's draws; a table's columns read the row pinned there. func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record { sc = sc.in(t) + if t.ownScope != nil { + mark := sc.draws.pushFrame(t.ownScope) + defer sc.draws.popFrame(mark) + } if t.site.isFormat() { sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins) } diff --git a/reference.go b/reference.go index 80080e8..784362c 100644 --- a/reference.go +++ b/reference.go @@ -72,10 +72,13 @@ func refSegments(name string, folder []string) ([]string, error) { // refHeads under its root path and its tail reads like a sibling path, so two // spellings of one target are one draw. func linkRefs(root map[string]node) error { - return eachTemplate(root, func(folder []string, path string, t *template) error { + if err := eachTemplate(root, func(folder []string, path string, t *template) error { category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".") return linkTemplate(folder, path, category, t, root) - }) + }); err != nil { + return err + } + return eachTemplate(root, func(_ []string, path string, t *template) error { return linkNames(path, t) }) } // linkTemplate resolves t's references in category, "" for an inline template, and @@ -85,7 +88,7 @@ func linkTemplate(folder []string, path, category string, t *template, root map[ if err != nil { return err } - t.link, t.compiled = link, compileFormat(t.tokens, link.refs) + t.link, t.compiled = link, compileFormat(t.tokens, link.refs, t.isName) compileArms(t) return nil } @@ -100,7 +103,9 @@ func compileArms(t *template) { compileArm(t, &o.operands[j]) } for j := range o.arms { - compileArm(t, &o.arms[j]) + if o.arms[j].kind != namedRead { + compileArm(t, &o.arms[j]) + } } } } diff --git a/render.go b/render.go index b9d3901..c703bc1 100644 --- a/render.go +++ b/render.go @@ -61,11 +61,15 @@ func renderOnce(s *generatorState, n node) string { func render(s *generatorState, n node, sc renderScope) string { switch n := n.(type) { case *choice: + if n.ownScope != nil { + mark := sc.draws.pushFrame(n.ownScope) + defer sc.draws.popFrame(mark) + } return render(s, pick(s, n), sc) case *nullItem: return "" case *table: - sc.row = renderedRow{n, n.drawRow(s)} + sc.row = renderedRow{n, sc.drawRowOf(s, n)} return expand(s, n.formatTemplate, sc) case *tableRow: return expand(s, n.t.formatTemplate, sc) @@ -78,6 +82,10 @@ func render(s *generatorState, n node, sc renderScope) string { case *template: sc = sc.in(n) if n.repeat == 1 { + if n.ownScope != nil { + mark := sc.draws.pushFrame(n.ownScope) + defer sc.draws.popFrame(mark) + } if lit, fixed := n.fixedText(); fixed { return lit } @@ -89,7 +97,7 @@ func render(s *generatorState, n node, sc renderScope) string { if i > 0 { b.WriteString(n.separator) } - b.WriteString(expandAnew(s, n)) + b.WriteString(expandAnew(s, n, sc.draws.frames)) } return b.String() default: @@ -97,12 +105,15 @@ func render(s *generatorState, n node, sc renderScope) string { } } -// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into -// render's loop, its draws would move to the heap. +// expandAnew expands one repeat iteration of t as a render of its own, in no group, inside the +// name scopes of frames. Inlined into render's loop, its draws would move to the heap. // //go:noinline -func expandAnew(s *generatorState, t *template) string { - var draws renderDraws +func expandAnew(s *generatorState, t *template, frames *frameStack) string { + draws := renderDraws{frames: frames} + if t.ownScope != nil { + defer draws.popFrame(draws.pushFrame(t.ownScope)) + } return expand(s, t, renderScope{draws: &draws}) } @@ -155,7 +166,8 @@ func expand(s *generatorState, t *template, sc renderScope) string { // token addresses by a dotted path that is not a reference, or a field an operand // reads — is drawn once and kept in hold, so {place.postal-code} and {place.locality} // read one row, either read twice gives one value, and a shown operand is the operand -// computed. Every other name is drawn afresh, so {word} {word} still draws twice. +// computed. A field of a template rendering as part of a named pick is kept in the pick. +// Every other field is drawn afresh, so {word} {word} still draws twice. func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue { if a.kind == refPathRead { return readReference(s, t, sc, a) @@ -163,7 +175,13 @@ func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a if sc.draws.trace != nil { traceRead(sc.draws.trace, t, sc, a) } - if a.kind == freshRead { + switch { + case a.kind == namedRead: + return readName(s, sc, a) + case sc.pick != nil && !isRef(a.head): + return readUnder(s, t, sc, a) + case a.kind == freshRead: + sc.pick = nil return readValue{text: render(s, t.head(a.head), sc)} } return readMemo(s, t, hold, nil, sc, a) @@ -187,6 +205,7 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r if pins != nil { sc = sc.at(leaf, pins) } + sc.pick, sc.entry = nil, memo r := renderLeaf(s, leaf, sc) if memo.value == nil { memo.value = map[string]readValue{} @@ -209,6 +228,10 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) { // renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one // reference alone whose read drew null. func renderLeaf(s *generatorState, n node, sc renderScope) readValue { + if c, isChoice := n.(*choice); isChoice && c.ownScope != nil { + mark := sc.draws.pushFrame(c.ownScope) + defer sc.draws.popFrame(mark) + } n = resolveChoice(s, n) switch leaf := n.(type) { case *nullItem: diff --git a/table.go b/table.go index 6a7e9cd..1375324 100644 --- a/table.go +++ b/table.go @@ -323,6 +323,9 @@ func (t *table) compileRowFormat(format string) error { if err != nil { return err } + if err := checkTableToks(toks, nil); err != nil { + return err + } for _, tok := range toks { if tok.kind != nameRead { continue @@ -338,14 +341,32 @@ func (t *table) compileRowFormat(format string) error { for i, name := range t.header { fields[name] = &tableColumn{t, i} } - if err := checkTokens(toks, fields); err != nil { + unbound, err := checkTokens(toks, fields) + if err != nil { return err } + if len(unbound) > 0 { + return unbound[0].err + } t.formatTemplate = &template{format: format, tokens: toks, fields: fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}} t.rowNode = &tableRow{t} return nil } +// checkTableToks refuses a table's format or cell binding a name, or reading a head no column +// holds: a table holds no names. +func checkTableToks(toks []formatToken, unbound []unboundRead) error { + for _, tok := range toks { + if tok.kind == nameBind { + return fmt.Errorf("token {%s}: a table binds no name; bind it in a template reading the table", tok.body) + } + } + if len(unbound) > 0 { + return unbound[0].err + } + return nil +} + // setTablePaths gives every table the path a selector on it is written at, before a // link or a draw can name one. func setTablePaths(root map[string]node) { diff --git a/todo.md b/todo.md index 451cb0a..a3a8ad9 100644 --- a/todo.md +++ b/todo.md @@ -55,7 +55,6 @@ | 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 | | 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 | | 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 | -| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 5.3, 4.1 | 25.8 | | 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 | | 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 | | 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 4 | 23.1 | @@ -186,23 +185,16 @@ Shape: T = table, t = template, c = choice. | `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — | | `direction`, `continent`, `ulid` | c/t | — | — | -### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice. - -- A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country. -- A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read. -- A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line. -- Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace. - ### 3. Rewrite the shipped categories so facts that belong together come from one named pick. - `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`. - `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once. -- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4. +- Needs item 2. Every shipped value must still pass its consumer's check before and after item 4. - Re-pin seeded output in its own commit. ### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly. -- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. +- Needs items 2 and 3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load. - Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap. - Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render. - Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides.