Bind a pick to a name with {ref as n}, and read it with {n} and {n.path}

This commit is contained in:
2026-10-03 01:51:14 +02:00
committed by Lillem4n
parent ba9477f2e5
commit 23e426016d
16 changed files with 609 additions and 63 deletions
+1
View File
@@ -118,3 +118,4 @@ In [docs/decisions.md](docs/decisions.md):
- The US given names come from a mirror of the SSA file
- `List` advertises direct descents only
- A path draws through its compiled steps
- A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read
+8
View File
@@ -8,6 +8,14 @@ replacement, and each removed path, column or flag.
### Added
- A name keeps one pick: `{/misc.territory as t}` binds one row of `misc.territory` to `t`
and prints nothing, and `{t}`, `{t.capital}` and `{t.calling-code}` read that one row.
A name lives in the category binding it, so every field and record column of the
category reads one pick, and a name a `repeat` binds picks again on every iteration.
Refused at `New`: a name bound twice in one category, or inside a `repeat` and outside
it; a name that is a field or an option; a binding of anything but a reference; a
binding nothing reads; a path through a name that selects a row; and a binding in a
table. A field, column, category or folder name may not contain ` as `.
- First release: the CLI, the library and the shipped data set.
- Table categories: a category JSON naming a `rows` TSV beside it, with the options
`key`, `name`, `weight` and `parent`; a path selects a row by key or name,
+28 -2
View File
@@ -359,7 +359,8 @@ Every character is literal except a `{…}` token:
| Token | Renders |
|-------|---------|
| `{name}` | the sibling field `name` |
| `{name}` | the sibling field `name`, or the pick bound to the name ([Names](#names)) |
| `{ref as name}` | nothing: binds one pick of the reference `ref` to `name` ([Names](#names)) |
| `{name.field}` | `field` of one draw of `name` ([Correlated fields](#correlated-fields)) |
| `{a\|b}` | one of the named fields, even odds |
| `{fn(args)}` | a builtin ([Functions](#functions)) |
@@ -574,7 +575,8 @@ or the reference draws it.
`format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options;
**any other key is a field** (see [Decisions](docs/decisions.md#options-and-fields-share-one-namespace)), and `rows` makes a category
a [table](#table), so no template carries a field of that name. An object that does nothing a
a [table](#table), so no template carries a field of that name. A [name](#names) shares the
namespace too. An object that does nothing a
string can't — only a `format` — is rejected naming the string, as is a one-item
choice naming its item.
@@ -696,6 +698,29 @@ unknown, names a folder, has no folder above, reads a field not every variant
of a choice carries, or names the category the reference sits in, and a reference
that leads back to its own value, directly, mutually or through a chain.
### Names
`{ref as n}` binds one pick of a reference to the name `n` and prints nothing. `{n}` then
prints that pick, and `{n.path}` reads a path through it, so every read of `n` describes one
row:
```json
"{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}"
```
Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. Everything under a name is drawn
once, so `{n}` and `{n.path}` agree, and a field `n` renders twice prints the same text twice.
A name is drawn on its first read, and lives in the category binding it: any field of the
category reads it, and a record's columns read one pick. Each render of the category picks
anew, and so does each bare reference to it. A name a `repeat` binds picks again on every
iteration, and one bound outside the repeat keeps its pick on every line.
Rejected at load: a name bound twice in one category, or both inside a `repeat` and outside
it; a name that is a field or an option; a binding of anything but a reference; a binding
nothing reads; a path through a name that selects a row, which the reference selects instead;
and a binding in a table's format or cell. A field, column, category or folder name may not
contain ` as `.
### Draw group
A template may carry `drawGroup` to hold its reference draws apart: every reference path
@@ -889,6 +914,7 @@ doc.go the package doc, and the vocabulary the package is written in
fejkdata.go Generator, New, options, the embedded data set, List
node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup
table.go tables: the rows TSV, its options and links, row selection and draws
name.go names: {ref as n} bindings, the scopes they live in, and the picks a render keeps under them
path.go the dotted-path walks with their selectors, and proving a path resolves
render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make
record.go records: Record, the JSON/CSV/SQL serializers, and their entry points
+4 -1
View File
@@ -27,7 +27,10 @@ package fejkdata
// an expansion holds a sibling path's head and an operand's field, its draw
// group a reference path's: `formatOps.held` and `heldCheck` an expansion's,
// `readReference` a draw group's.
// - memo — what a hold or a draw group keeps its draws in, the variant drawn at
// - named pick — one draw of a reference a binding token, {ref as n}, gives a name, kept for one
// render of the scope it is bound in, a category or a repeat iteration, and drawn on its
// first read: `nameBinding`, `nameScope`, `namedPick`, `pickFrame`.
// - memo — what a hold, a draw group or a named pick keeps its draws in, the variant drawn at
// each level and the value each path read: `drawMemo`, `readMemo`.
// - draw group — reference draws kept apart inside one render:
// `template.drawGroup` as the data spells it, `templateLink.drawGroupKey` as a
+13
View File
@@ -888,3 +888,16 @@ since Go tracks a struct's fields as one, and a repeat of a reference path rose
to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node,
so no benchmark gained an allocation. Valid while Go's escape analysis tracks
a struct's fields as one.
## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read
2026-10-03, larv-review. Goals 5.3 and 5.6.
`bindNames` scopes a name at compile, and a render keeps one `pickFrame` per scope. One
field binds a name where another reads it, `"first": "{/person as p}{p.first}"` beside
`"last": "{p.last}"`, so the scope cannot be the template holding the binding; the category is the smallest unit holding every field and record column. The
scope is lexical: a category referencing another never sees its names, so what a category
renders never depends on who references it. A pick is drawn on its first read, so a record's
columns, rendered in name order, read one pick whichever binds it. A read entering a category
past its root, `Fake("cat.field")` or `{/cat.field}`, keeps the category's frame in the memo of
that read: per render and group, as the read's own draw is. Valid while a name is read only
inside the category binding it.
+12 -4
View File
@@ -2,12 +2,14 @@ package fejkdata
import "strings"
// drawMemo is what a hold or a draw group has drawn: the variant each level was drawn as, so
// every path under it reads one variant; and the value each read produced, by its path, so the
// same read written twice reads one value.
// drawMemo is what a hold, a draw group or a named pick has drawn: the variant each level was
// drawn as, so every path under it reads one variant; the value each read produced, by its path,
// so the same read written twice reads one value; and the frame of each name scope a read
// entered past its owner.
type drawMemo struct {
variant map[string]node
value map[string]readValue
frames map[*nameScope]*pickFrame
}
// groupDraws is what one render's draw group has drawn for its reference paths: a memo, and the
@@ -27,11 +29,12 @@ type readValue struct {
type renderTrace func(group string, row renderedRow, a arm)
// renderDraws is one render's reference draws: the unnamed draw group's, and each named one's;
// and a test's trace of its reads.
// a test's trace of its reads; and the frames of the name scopes rendering.
type renderDraws struct {
unnamed groupDraws
named map[string]*groupDraws
trace renderTrace
frames *frameStack
}
// renderScope is where a render reads its reference paths: its reference draws, in the draw group of the
@@ -42,6 +45,11 @@ type renderScope struct {
draws *renderDraws
group string
row renderedRow
entry *drawMemo // the memo of the read that entered what renders
// pick is the named pick what renders is part of, at pickKey, its path from the name; nil
// outside one.
pick *namedPick
pickKey string
}
type renderedRow struct {
+91 -23
View File
@@ -1,6 +1,7 @@
package fejkdata
import (
"errors"
"fmt"
"strings"
)
@@ -63,7 +64,8 @@ type formatToken struct {
body string // the braces' content, as written
fn string // builtinCall
args []string // builtinCall
names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads
names []string // nameRead: the '|' arms; builtinCall: the operands its builtin reads; nameBind: the reference it binds
bound string // nameBind: the name
}
type tokenKind uint8
@@ -72,8 +74,12 @@ const (
builtinCall tokenKind = iota + 1
nameRead
literalRun
nameBind
)
// asWord separates a binding's reference from the name it binds: {/person as p}.
const asWord = " as "
// parseFormat is the one reading of a format's tokens: a '(' outside a selector makes
// a token a call.
func parseFormat(format string) ([]formatToken, error) {
@@ -91,6 +97,10 @@ func parseFormat(format string) ([]formatToken, error) {
return nil
}
flush()
if ref, name, binds := cutOutside(u.body, asWord); binds && indexOutside(u.body, '(') < 0 {
toks = append(toks, formatToken{kind: nameBind, body: u.body, names: []string{ref}, bound: name})
return nil
}
if indexOutside(u.body, '(') < 0 {
toks = append(toks, formatToken{kind: nameRead, body: u.body, names: splitOutside(u.body, '|')})
return nil
@@ -195,51 +205,103 @@ func checkFunc(tok formatToken, fields map[string]node) error {
return nil
}
func parseChecked(format string, fields map[string]node) ([]formatToken, error) {
func parseChecked(format string, fields map[string]node) ([]formatToken, []unboundRead, error) {
toks, err := parseFormat(format)
if err != nil {
return nil, err
return nil, nil, err
}
return toks, checkTokens(toks, fields)
unbound, err := checkTokens(toks, fields)
return toks, unbound, err
}
// unboundRead is a token reading a head no field holds, which only a name bound around the
// template can answer; err is the refusal where none does.
type unboundRead struct {
head string
err error
}
// fieldMiss is a head no field of the template holds.
type fieldMiss struct {
head string
err error
}
func (m *fieldMiss) Error() string { return m.err.Error() }
// checkTokens proves every parsed token names an existing field or a known function,
// so a typo'd or dangling reference is a New-time error.
func checkTokens(toks []formatToken, fields map[string]node) error {
// so a typo'd or dangling reference is a New-time error. A head no field holds comes
// back unbound, for bindNames to look up among the names.
func checkTokens(toks []formatToken, fields map[string]node) ([]unboundRead, error) {
var unbound []unboundRead
for _, t := range toks {
if err := checkToken(t, fields); err != nil {
return err
u, err := checkToken(t, fields)
if err != nil {
return nil, err
}
unbound = append(unbound, u...)
}
return nil
return unbound, nil
}
func checkToken(t formatToken, fields map[string]node) error {
func checkToken(t formatToken, fields map[string]node) ([]unboundRead, error) {
switch t.kind {
case builtinCall:
return checkFunc(t, fields)
return nil, checkFunc(t, fields)
case nameBind:
return nil, checkBind(t)
case nameRead:
var unbound []unboundRead
for _, name := range t.names {
if isRef(name) {
if _, _, err := refShape(name); err != nil {
return fmt.Errorf("token {%s}: %w", t.body, err)
return nil, fmt.Errorf("token {%s}: %w", t.body, err)
}
continue // its target is checked at New (see linkRefs)
}
if err := checkArm(name, fields, len(t.names) == 1); err != nil {
return fmt.Errorf("token {%s}: %w", t.body, err)
err := checkArm(name, fields, len(t.names) == 1)
var miss *fieldMiss
if errors.As(err, &miss) {
unbound = append(unbound, unboundRead{miss.head, fmt.Errorf("token {%s}: %w", t.body, err)})
continue
}
if err != nil {
return nil, fmt.Errorf("token {%s}: %w", t.body, err)
}
}
// Last, so an arm broken on its own terms is reported as that: a repeat is
// the consequence of such a mistake, not the mistake itself.
return checkNoRepeatedArm(t.body, t.names)
return unbound, checkNoRepeatedArm(t.body, t.names)
}
return nil, nil
}
// checkBind proves a {ref as name} token binds a reference to a name no other token
// could read as something else.
func checkBind(t formatToken) error {
ref, name := t.names[0], t.bound
if trimmed := strings.TrimSpace(ref) + asWord + strings.TrimSpace(name); trimmed != t.body {
return fmt.Errorf("token {%s}: write {%s}", t.body, trimmed)
}
if !isRef(ref) {
return fmt.Errorf("token {%s}: %q is no reference; a name binds a pick of a reference, which starts with /, . or ..", t.body, ref)
}
if _, _, err := refShape(ref); err != nil {
return fmt.Errorf("token {%s}: %w", t.body, err)
}
if isOption(name) {
return fmt.Errorf("token {%s}: %q is an option and can never be a name", t.body, name)
}
if err := checkName(name); err != nil {
return fmt.Errorf("token {%s}: name %w", t.body, err)
}
return nil
}
// checkArm validates one sibling name or path against a template's fields.
// wholeToken says the name is the token's entire body, so {/name} would render
// the same value and can be offered as the reference spelling.
// the same value and can be offered as the reference spelling. A head no field
// holds is a *fieldMiss.
func checkArm(name string, fields map[string]node, wholeToken bool) error {
a := splitArm(name, nil)
if err := checkSegments(a); err != nil {
@@ -258,9 +320,9 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error {
if wholeToken && hintableRef(name) {
hint = fmt.Sprintf(" — write {/%s} to reference the data", name)
}
return fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint)
return &fieldMiss{a.head, fmt.Errorf("no field %q; a token names a sibling field, and this template has none%s", a.head, hint)}
}
return fmt.Errorf("no field %q", a.head)
return &fieldMiss{a.head, fmt.Errorf("no field %q", a.head)}
}
if err := checkPathResolves(field, a.tail, a.head); err != nil {
return fmt.Errorf("field %q: %w", a.head, err)
@@ -316,6 +378,7 @@ type arm struct {
leaves []node // every node the path may land on, one per variant it passes
cover node // what holding the path pins: the first choice it passes, else its leaf
kind armKind
bind *nameBinding // namedRead: the binding of the name it reads through
}
// armKind is how expand reads an arm, fixed at compile.
@@ -325,6 +388,7 @@ const (
freshRead armKind = iota // drawn afresh at every read
heldRead // drawn once per expansion, kept in its hold
refPathRead // drawn once per draw group, kept in its memo
namedRead // read through a name, kept in its pick
)
func (a arm) isRefPath() bool {
@@ -439,10 +503,14 @@ func (c *formatOps) function(tok formatToken, refs map[string]refBinding) {
c.ops = append(c.ops, op{formatToken: tok, call: builtins[tok.fn].prep(tok.args), operands: operands})
}
func (c *formatOps) field(tok formatToken, refs map[string]refBinding) {
func (c *formatOps) field(tok formatToken, refs map[string]refBinding, isName func(string) bool) {
arms := make([]arm, len(tok.names))
for i, name := range tok.names {
arms[i] = splitArm(name, refs)
if isName(arms[i].head) {
arms[i].kind = namedRead
continue
}
if len(arms[i].tail) > 0 {
c.holdName(arms[i], "token {"+arms[i].spelling+"}")
}
@@ -450,9 +518,9 @@ func (c *formatOps) field(tok formatToken, refs map[string]refBinding) {
c.ops = append(c.ops, op{formatToken: tok, arms: arms})
}
// compileFormat compiles a parsed format. Call checkTokens first: it is what proves
// every token valid.
func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps {
// compileFormat compiles a parsed format, a binding to nothing, since it prints nothing. Call
// checkTokens first: it is what proves every token valid.
func compileFormat(toks []formatToken, refs map[string]refBinding, isName func(head string) bool) formatOps {
var c formatOps
for _, tok := range toks {
switch tok.kind {
@@ -462,7 +530,7 @@ func compileFormat(toks []formatToken, refs map[string]refBinding) formatOps {
case builtinCall:
c.function(tok, refs)
case nameRead:
c.field(tok, refs)
c.field(tok, refs, isName)
}
}
return c
+9 -1
View File
@@ -163,7 +163,7 @@ func inlineBinding(n node, label string, root map[string]node) binding {
// linkNodeRefs binds the references in an inline node's templates against the
// loaded tree.
func linkNodeRefs(scope nodeScope, root map[string]node) error {
return scope(func(path string, m node) error {
if err := scope(func(path string, m node) error {
t, ok := m.(*template)
if !ok {
return nil
@@ -178,5 +178,13 @@ func linkNodeRefs(scope nodeScope, root map[string]node) error {
}
}
return linkTemplate(nil, path, "", t, root)
}); err != nil {
return err
}
return scope(func(path string, m node) error {
if t, ok := m.(*template); ok {
return linkNames(path, t)
}
return nil
})
}
+331
View File
@@ -0,0 +1,331 @@
package fejkdata
import (
"fmt"
"strings"
)
// nameScope is where a name is bound: a category, or one iteration of a repeat inside it.
// Every token of the category outside a nested repeat reads the category's names, so one
// field may bind a name and another read it.
type nameScope struct {
up *nameScope
owner node // the category's root, or the repeat
bindings map[string]*nameBinding
order []*nameBinding
}
// nameBinding is one {ref as name} token: the scope it binds name in, its index among
// that scope's bindings, and, once linked, the head node and tail its reference reads.
type nameBinding struct {
name string
ref string
body string
where string // the fields reaching the binding template, as a compile error spells them
scope *nameScope
index int
binder *template
read bool
head node
tail []string
}
func (sc *nameScope) lookup(name string) *nameBinding {
for ; sc != nil; sc = sc.up {
if b, ok := sc.bindings[name]; ok {
return b
}
}
return nil
}
// bindNames gives every template of a compiled category or inline template the scope its
// names live in, and refuses a name bound twice along one chain of scopes, a name a field
// spells too, a read no field or name answers, and a binding nothing reads.
func bindNames(root node) error {
var scopes []*nameScope
var gather func(n node, scope *nameScope, where string) error
gather = func(n node, scope *nameScope, where string) error {
if t, isTemplate := n.(*template); isTemplate {
if t.repeat > 1 && n != root {
scope = &nameScope{up: scope, owner: t}
scopes = append(scopes, scope)
}
t.scope = scope
for _, tok := range t.tokens {
if tok.kind != nameBind {
continue
}
if err := scope.bind(tok, t, where); err != nil {
return err
}
}
}
return eachContained(n, where, func(c node, where string) error { return gather(c, scope, where) })
}
top := &nameScope{owner: root}
scopes = append(scopes, top)
if err := gather(root, top, ""); err != nil {
return err
}
if err := resolveReads(root, ""); err != nil {
return err
}
for _, sc := range scopes {
if err := sc.close(); err != nil {
return err
}
}
return nil
}
// close refuses a binding nothing reads, and gives the owner of a scope binding names the scope.
func (sc *nameScope) close() error {
for _, b := range sc.order {
if !b.read {
return fmt.Errorf("%stoken {%s}: nothing reads name %q; drop the token", b.where, b.body, b.name)
}
}
if len(sc.order) == 0 {
return nil
}
switch o := sc.owner.(type) {
case *template:
o.ownScope = sc
case *choice:
o.ownScope = sc
}
return nil
}
// eachContained calls fn for each node n contains, with where naming the fields reaching it as
// compile errors do.
func eachContained(n node, where string, fn func(c node, where string) error) error {
for _, c := range contained(n) {
next := where
if c.name != "" {
next += fmt.Sprintf("field %q: ", c.name)
}
if err := fn(c.node, next); err != nil {
return err
}
}
return nil
}
func (sc *nameScope) bind(tok formatToken, binder *template, where string) error {
if b, twice := sc.bindings[tok.bound]; twice {
return fmt.Errorf("%stoken {%s}: name %q is bound twice in one scope, by {%s} too; rename one", where, tok.body, tok.bound, b.body)
}
if b := sc.up.lookup(tok.bound); b != nil {
return fmt.Errorf("%stoken {%s}: name %q is bound outside this repeat already, by {%s}; rename one", where, tok.body, tok.bound, b.body)
}
b := &nameBinding{name: tok.bound, ref: tok.names[0], body: tok.body, where: where, scope: sc, index: len(sc.order), binder: binder}
if sc.bindings == nil {
sc.bindings = map[string]*nameBinding{}
}
sc.bindings[b.name] = b
sc.order = append(sc.order, b)
return nil
}
// resolveReads answers each unbound read under n with a name its template sees, fields before the
// format as compile reports them, and refuses a field spelling a name.
func resolveReads(n node, where string) error {
if err := eachContained(n, where, resolveReads); err != nil {
return err
}
t, isTemplate := n.(*template)
if !isTemplate {
return nil
}
for _, name := range sortedNames(t.fields) {
if b := t.scope.lookup(name); b != nil {
return fmt.Errorf("%stoken {%s}: name %q is a field too; options, fields and names share one namespace, so rename one", b.where, b.body, name)
}
}
for _, u := range t.unbound {
b := t.scope.lookup(u.head)
if b == nil {
return fmt.Errorf("%s%w", where, u.err)
}
b.read = true
}
return nil
}
// isName reports whether a head t reads is a name: neither a reference nor a field.
func (t *template) isName(head string) bool {
return !isRef(head) && t.fields[head] == nil && t.scope.lookup(head) != nil
}
// linkNames compiles t's reads of a name as paths from the head its binding's reference names, once
// every template is linked, so the binder has resolved that reference.
func linkNames(path string, t *template) error {
for i := range t.compiled.ops {
for j := range t.compiled.ops[i].arms {
a := &t.compiled.ops[i].arms[j]
if a.kind != namedRead {
continue
}
if err := linkName(t, a); err != nil {
return fmt.Errorf("%s: token {%s}: %w", t.site.label(path), a.spelling, err)
}
}
}
return nil
}
func linkName(t *template, a *arm) error {
b := t.scope.lookup(a.head)
if b.head == nil {
ref := b.binder.link.refs[b.ref]
b.head, b.tail = b.binder.link.refHeads[ref.head], ref.tail
}
if hasSelector(a.tail) {
return fmt.Errorf("a path through name %q selects no row; select it in the reference %q binds", a.head, a.head)
}
full := append(b.tail[:len(b.tail):len(b.tail)], a.tail...)
if err := checkPathResolves(b.head, full, a.head); err != nil {
return err
}
w := compilePath(b.head, full)
a.bind, a.steps, a.leaves, a.cover = b, w.steps, w.leaves, w.cover
a.levels = make([]string, len(full)+1)
for i := range a.levels {
a.levels[i] = joinSegments(full[:i])
}
a.path = joinSegments(full)
return nil
}
// namedPick is one draw of a name: the variant drawn at each level under it, the value each
// path through it read, keyed by the path from the name, and the table rows they pinned.
type namedPick struct {
memo drawMemo
pins pinSet
}
// pickFrame is one render of a name scope: a pick per binding, each drawn on its first read.
type pickFrame struct {
scope *nameScope
picks []namedPick
}
func newPickFrame(scope *nameScope) *pickFrame {
return &pickFrame{scope: scope, picks: make([]namedPick, len(scope.order))}
}
// frameStack is the frames of the name scopes rendering, innermost last. A render's draws point
// at it, and a repeat iteration's share it: holding the frames in the draws, or in a renderScope,
// would move every render's draws to the heap.
type frameStack struct {
frames []*pickFrame
}
// pushFrame opens a frame of scope until popFrame closes it, returning the mark popFrame takes.
func (d *renderDraws) pushFrame(scope *nameScope) int {
if d.frames == nil {
d.frames = &frameStack{}
}
mark := len(d.frames.frames)
d.frames.frames = append(d.frames.frames, newPickFrame(scope))
return mark
}
func (d *renderDraws) popFrame(mark int) { d.frames.frames = d.frames.frames[:mark] }
// pickOf is b's pick in the frame rendering its scope. A read entering the scope past its owner,
// as Fake("cat.field") or {/cat.field} do, keeps the frame in the memo of that read.
func (sc renderScope) pickOf(b *nameBinding) *namedPick {
if stack := sc.draws.frames; stack != nil {
for i := len(stack.frames) - 1; i >= 0; i-- {
if f := stack.frames[i]; f.scope == b.scope {
return &f.picks[b.index]
}
}
}
memo := sc.entry
if memo == nil {
memo = &sc.groupDraws().memo
}
f, ok := memo.frames[b.scope]
if !ok {
f = newPickFrame(b.scope)
if memo.frames == nil {
memo.frames = map[*nameScope]*pickFrame{}
}
memo.frames[b.scope] = f
}
return &f.picks[b.index]
}
func readName(s *generatorState, sc renderScope, a arm) readValue {
return readPicked(s, sc.pickOf(a.bind), a.bind.head, a.steps, a.levels, a.path, sc)
}
// drawRowOf draws the row a render of t reads: inside the pick's rows where t renders as part of one.
func (sc renderScope) drawRowOf(s *generatorState, t *table) int {
if sc.pick != nil {
return t.drawIn(s, &sc.pick.pins)
}
return t.drawRow(s)
}
// readUnder reads a of t, which renders as part of the pick sc.pick at sc.pickKey: once per pick,
// by its path from the name.
func readUnder(s *generatorState, t *template, sc renderScope, a arm) readValue {
levels := make([]string, len(a.levels))
for i, l := range a.levels {
levels[i] = underKey(sc.pickKey, l)
}
return readPicked(s, sc.pick, t.head(a.head), a.steps, levels, underKey(sc.pickKey, a.path), sc)
}
// underKey is the key of path under the pick key prefix. It never returns prefix itself, which a
// memo would keep: anything a renderScope holds reaching the heap moves every render's draws there.
func underKey(prefix, path string) string {
if prefix == "" {
return path
}
return prefix + "." + path
}
// readPicked draws the path key names under pick p, once, and renders what it lands on as part of p.
func readPicked(s *generatorState, p *namedPick, head node, steps []pathStep, levels []string, key string, sc renderScope) readValue {
if r, done := p.memo.value[key]; done {
return r
}
leaf := drawSteps(s, head, steps, &p.pins, &p.memo, levels)
mark := -1
if c, isChoice := leaf.(*choice); isChoice {
if c.ownScope != nil {
mark = sc.draws.pushFrame(c.ownScope)
}
leaf = p.memo.variantOf(s, c, key)
}
sc = sc.at(leaf, &p.pins)
sc.pick, sc.pickKey, sc.entry = p, key, &p.memo
r := renderLeaf(s, leaf, sc)
if mark >= 0 {
sc.draws.popFrame(mark)
}
if p.memo.value == nil {
p.memo.value = map[string]readValue{}
}
p.memo.value[key] = r
return r
}
// variantOf is the variant of c drawn at level, drawn now where none was.
func (m *drawMemo) variantOf(s *generatorState, c *choice, level string) node {
n, drew := m.variant[level]
if !drew {
n = resolveChoice(s, c)
if m.variant == nil {
m.variant = map[string]node{}
}
m.variant[strings.Clone(level)] = n
}
return n
}
+27 -8
View File
@@ -25,9 +25,10 @@ func (*folder) isNode() {}
// relative dot paths every item can address, so carriedByAll and List both read the one
// answer to what a path may reach through this choice.
type choice struct {
items []node
cum []float64
shared map[string]bool
items []node
cum []float64
shared map[string]bool
ownScope *nameScope // set on a category's root binding names, by bindNames
}
func (*choice) isNode() {}
@@ -53,6 +54,11 @@ type template struct {
fromString bool // written as a JSON string rather than an object
isRecord bool // compiled at the top without a repeat, so its fields are record columns
site tableSite
unbound []unboundRead // the heads its tokens read that no field holds
// Filled by `bindNames`, from the compiled category:
scope *nameScope // where its tokens look a name up
ownScope *nameScope // the scope it renders a frame of: a category's, on its root, or a repeat's, per iteration
// Filled by `linkTemplate`, from the assembled tree:
link templateLink
@@ -105,7 +111,11 @@ func (t *template) head(name string) node {
// compile converts parsed JSON — a category or an inline template — into a node tree,
// validating structure up front.
func compile(v any) (node, error) {
return compileAt(v, atTop)
n, err := compileAt(v, atTop)
if err != nil {
return nil, err
}
return n, bindNames(n)
}
// compileCategory compiles a data file's value, which may be a table over a rows
@@ -220,11 +230,17 @@ func jsonKind(v any) string {
}
func compileString(s string, site tableSite) (*template, error) {
toks, err := parseChecked(s, nil)
toks, unbound, err := parseChecked(s, nil)
if err != nil {
return nil, err
}
return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site}, nil
if site.table != nil {
if err := checkTableToks(toks, unbound); err != nil {
return nil, err
}
unbound = nil
}
return &template{format: s, tokens: toks, repeat: 1, fromString: true, site: site, unbound: unbound}, nil
}
// fixedText is one render's output when the format holds no token; repeat is the caller's.
@@ -329,14 +345,14 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString && o.group == "" {
return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format)
}
toks, err := parseChecked(o.format, fields)
toks, unbound, err := parseChecked(o.format, fields)
if err != nil {
return nil, err
}
if err := checkNestedDrawGroup(fields, o.group); err != nil {
return nil, err
}
return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn}, nil
return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, drawGroup: o.group, isRecord: fieldPos == inColumn, unbound: unbound}, nil
}
// templateOptions is what a template object's option keys say.
@@ -525,6 +541,9 @@ func checkName(name string) error {
if name == "-" {
return fmt.Errorf(`%q is reserved: the struct tag fake:"-" leaves a field unfilled, so no tag could read it; rename it`, name)
}
if strings.Contains(name, asWord) {
return fmt.Errorf("%q contains %q, which a token reads as binding a name; rename it", name, asWord)
}
if i := strings.IndexAny(name, reservedInName); i >= 0 {
return fmt.Errorf("%q contains %q; a name may not use %s, which the dot path, {token} and JSON grammars reserve",
name, name[i:i+1], reservedList)
+16
View File
@@ -87,6 +87,22 @@ func indexOutside(s string, c byte) int {
return -1
}
// cutOutside cuts s around the first sep outside a [selector].
func cutOutside(s, sep string) (before, after string, found bool) {
depth := 0
for i := 0; i < len(s); i++ {
switch {
case s[i] == '[':
depth++
case s[i] == ']' && depth > 0:
depth--
case depth == 0 && strings.HasPrefix(s[i:], sep):
return s[:i], s[i+len(sep):], true
}
}
return s, "", false
}
func splitOutside(s string, c byte) []string {
var parts []string
for {
+5 -1
View File
@@ -22,7 +22,7 @@ type Column struct {
// Record is one record rendered from a template: every direct field is a column,
// listed in name order. Each column is its own expansion, so a sibling field is
// local to it, while a reference that reads a path is drawn once for the whole
// record, per group.
// record, per group, and a name once for the whole record.
type Record struct {
columns []Column
}
@@ -263,6 +263,10 @@ func recordOf(n node) (*template, []Column, error) {
// over sc's draws; a table's columns read the row pinned there.
func renderRecord(s *generatorState, t *template, columns []Column, sc renderScope) *Record {
sc = sc.in(t)
if t.ownScope != nil {
mark := sc.draws.pushFrame(t.ownScope)
defer sc.draws.popFrame(mark)
}
if t.site.isFormat() {
sc = sc.at(t.site.table.rowNode, &sc.groupDraws().pins)
}
+9 -4
View File
@@ -72,10 +72,13 @@ func refSegments(name string, folder []string) ([]string, error) {
// refHeads under its root path and its tail reads like a sibling path, so two
// spellings of one target are one draw.
func linkRefs(root map[string]node) error {
return eachTemplate(root, func(folder []string, path string, t *template) error {
if err := eachTemplate(root, func(folder []string, path string, t *template) error {
category := strings.Join(strings.Split(path, ".")[:len(folder)+1], ".")
return linkTemplate(folder, path, category, t, root)
})
}); err != nil {
return err
}
return eachTemplate(root, func(_ []string, path string, t *template) error { return linkNames(path, t) })
}
// linkTemplate resolves t's references in category, "" for an inline template, and
@@ -85,7 +88,7 @@ func linkTemplate(folder []string, path, category string, t *template, root map[
if err != nil {
return err
}
t.link, t.compiled = link, compileFormat(t.tokens, link.refs)
t.link, t.compiled = link, compileFormat(t.tokens, link.refs, t.isName)
compileArms(t)
return nil
}
@@ -100,7 +103,9 @@ func compileArms(t *template) {
compileArm(t, &o.operands[j])
}
for j := range o.arms {
compileArm(t, &o.arms[j])
if o.arms[j].kind != namedRead {
compileArm(t, &o.arms[j])
}
}
}
}
+31 -8
View File
@@ -61,11 +61,15 @@ func renderOnce(s *generatorState, n node) string {
func render(s *generatorState, n node, sc renderScope) string {
switch n := n.(type) {
case *choice:
if n.ownScope != nil {
mark := sc.draws.pushFrame(n.ownScope)
defer sc.draws.popFrame(mark)
}
return render(s, pick(s, n), sc)
case *nullItem:
return ""
case *table:
sc.row = renderedRow{n, n.drawRow(s)}
sc.row = renderedRow{n, sc.drawRowOf(s, n)}
return expand(s, n.formatTemplate, sc)
case *tableRow:
return expand(s, n.t.formatTemplate, sc)
@@ -78,6 +82,10 @@ func render(s *generatorState, n node, sc renderScope) string {
case *template:
sc = sc.in(n)
if n.repeat == 1 {
if n.ownScope != nil {
mark := sc.draws.pushFrame(n.ownScope)
defer sc.draws.popFrame(mark)
}
if lit, fixed := n.fixedText(); fixed {
return lit
}
@@ -89,7 +97,7 @@ func render(s *generatorState, n node, sc renderScope) string {
if i > 0 {
b.WriteString(n.separator)
}
b.WriteString(expandAnew(s, n))
b.WriteString(expandAnew(s, n, sc.draws.frames))
}
return b.String()
default:
@@ -97,12 +105,15 @@ func render(s *generatorState, n node, sc renderScope) string {
}
}
// expandAnew expands one repeat iteration of t as a render of its own, in no group. Inlined into
// render's loop, its draws would move to the heap.
// expandAnew expands one repeat iteration of t as a render of its own, in no group, inside the
// name scopes of frames. Inlined into render's loop, its draws would move to the heap.
//
//go:noinline
func expandAnew(s *generatorState, t *template) string {
var draws renderDraws
func expandAnew(s *generatorState, t *template, frames *frameStack) string {
draws := renderDraws{frames: frames}
if t.ownScope != nil {
defer draws.popFrame(draws.pushFrame(t.ownScope))
}
return expand(s, t, renderScope{draws: &draws})
}
@@ -155,7 +166,8 @@ func expand(s *generatorState, t *template, sc renderScope) string {
// token addresses by a dotted path that is not a reference, or a field an operand
// reads — is drawn once and kept in hold, so {place.postal-code} and {place.locality}
// read one row, either read twice gives one value, and a shown operand is the operand
// computed. Every other name is drawn afresh, so {word} {word} still draws twice.
// computed. A field of a template rendering as part of a named pick is kept in the pick.
// Every other field is drawn afresh, so {word} {word} still draws twice.
func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a arm) readValue {
if a.kind == refPathRead {
return readReference(s, t, sc, a)
@@ -163,7 +175,13 @@ func readField(s *generatorState, t *template, hold *drawMemo, sc renderScope, a
if sc.draws.trace != nil {
traceRead(sc.draws.trace, t, sc, a)
}
if a.kind == freshRead {
switch {
case a.kind == namedRead:
return readName(s, sc, a)
case sc.pick != nil && !isRef(a.head):
return readUnder(s, t, sc, a)
case a.kind == freshRead:
sc.pick = nil
return readValue{text: render(s, t.head(a.head), sc)}
}
return readMemo(s, t, hold, nil, sc, a)
@@ -187,6 +205,7 @@ func readMemo(s *generatorState, t *template, memo *drawMemo, pins *pinSet, sc r
if pins != nil {
sc = sc.at(leaf, pins)
}
sc.pick, sc.entry = nil, memo
r := renderLeaf(s, leaf, sc)
if memo.value == nil {
memo.value = map[string]readValue{}
@@ -209,6 +228,10 @@ func traceRead(trace renderTrace, t *template, sc renderScope, a arm) {
// renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one
// reference alone whose read drew null.
func renderLeaf(s *generatorState, n node, sc renderScope) readValue {
if c, isChoice := n.(*choice); isChoice && c.ownScope != nil {
mark := sc.draws.pushFrame(c.ownScope)
defer sc.draws.popFrame(mark)
}
n = resolveChoice(s, n)
switch leaf := n.(type) {
case *nullItem:
+22 -1
View File
@@ -323,6 +323,9 @@ func (t *table) compileRowFormat(format string) error {
if err != nil {
return err
}
if err := checkTableToks(toks, nil); err != nil {
return err
}
for _, tok := range toks {
if tok.kind != nameRead {
continue
@@ -338,14 +341,32 @@ func (t *table) compileRowFormat(format string) error {
for i, name := range t.header {
fields[name] = &tableColumn{t, i}
}
if err := checkTokens(toks, fields); err != nil {
unbound, err := checkTokens(toks, fields)
if err != nil {
return err
}
if len(unbound) > 0 {
return unbound[0].err
}
t.formatTemplate = &template{format: format, tokens: toks, fields: fields, repeat: 1, isRecord: true, site: tableSite{t, formatRow}}
t.rowNode = &tableRow{t}
return nil
}
// checkTableToks refuses a table's format or cell binding a name, or reading a head no column
// holds: a table holds no names.
func checkTableToks(toks []formatToken, unbound []unboundRead) error {
for _, tok := range toks {
if tok.kind == nameBind {
return fmt.Errorf("token {%s}: a table binds no name; bind it in a template reading the table", tok.body)
}
}
if len(unbound) > 0 {
return unbound[0].err
}
return nil
}
// setTablePaths gives every table the path a selector on it is written at, before a
// link or a draw can name one.
func setTablePaths(root map[string]node) {
+2 -10
View File
@@ -55,7 +55,6 @@
| 61 | 0.1.0 | | **Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.** | 4 | 9 | 9 | 10 | 2.1 | 30.8 |
| 16 | 0.1.0 | | **Add the remaining locale categories: company, phone, finance, vehicle, words.** | 3 | 8 | 8 | 9 | 2.2, 14 | 28.1 |
| 15 | 0.1.0 | | **Add the remaining `misc` tables and templates, one row of its detail's table per chunk.** | 3 | 9 | 8 | 8 | 2, 14, 13 | 25.9 |
| 1 | 0.1.0 | | **Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.** | 4 | 6 | 8 | 9 | 5.3, 4.1 | 25.8 |
| 3 | 0.1.0 | | **Rewrite the shipped categories so facts that belong together come from one named pick.** | 5 | 5 | 8 | 9 | 4.1 | 25.0 |
| 4 | 0.1.0 | decision | **Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.** | 7 | 8 | 8 | 10 | 5.2, 4.1 | 24.0 |
| 25 | 0.1.0 | defect | **Draw `en_US.phone`'s `exch` as a NANP central office code, and assert the rule in `TestShippedUSPhone`.** | 2 | 2 | 6 | 8 | 4 | 23.1 |
@@ -186,23 +185,16 @@ Shape: T = table, t = template, c = choice.
| `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — |
| `direction`, `continent`, `ulid` | c/t | — | — |
### 1. Bind a pick to a name with `{x as n}`, read it with `{n}` and `{n.path}`, and refuse a name bound twice.
- A name is one pick of everything under it, drawn once: `{/country as k} {k.city.name} ({k.city.population})` is one city of `k`'s country.
- A name is visible in the template that binds it and everything that template renders, its sibling fields and a record's columns included, so one field may bind and another read.
- A name bound inside a `repeat` level picks again on every iteration; one bound outside it keeps its pick on every line.
- Refused at load: a name bound twice in one scope, and a name that is also a field or an option of the template, since options and fields share one namespace.
### 3. Rewrite the shipped categories so facts that belong together come from one named pick.
- `sv_SE.person` and `en_US.person` bind the first name and read sex and title through it: `"first": "{.first-name as n}{n.name}"`, `"sex": "{n..sex.name}"`, a title from `{n..sex.title.name}`.
- `geo.SE.address` and `geo.US.address` bind the street and read locality, postal code and region through it; the locale `address` categories bind `/geo.XX.address` once.
- Needs items 1 and 2. Every shipped value must still pass its consumer's check before and after item 4.
- Needs item 2. Every shipped value must still pass its consumer's check before and after item 4.
- Re-pin seeded output in its own commit.
### 4. Draw every `{…}` afresh, keep a pick only by name, and delete `drawGroup` with the fences that held picks implicitly.
- Needs items 1–3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load.
- Needs items 2 and 3. After it, `{/person} ({/person.first})` renders two people, `{w} {uppercase(w)}` renders `b G` where `w` is a choice of letters, and `{/city.name}, {/country[SE].name}` and `{/country[SE].name} / {/country[NO].name}` both load.
- Deletes the token-order rules (`drawsApart`, `readFold`, `branches`, the pairwise replay), the expansion hold fence, `drawGroup`, the own-family fence and the cell-reference fences, and revises the decisions that point here. The remaining draw checks are names, paths through names, cycles and the repeat cap.
- Fixes on the way: `{/sel}|{/sel}` panics out of `Fake` today where two draws land on rows whose cells select different rows of another table; it must render.
- Rewrites the README's References, Draw group, Correlated fields, Linked tables and "One draw, one spelling" sections; the Linked tables section says which pick's weights govern a family, since the bound row decides.