Tests for reference sigils: / the root, . this folder, .. the folder above

README.md

@@ -110,7 +110,7 @@ work with no data on disk. A directory is a namespace: each JSON file is a
 category named after the file, each subdirectory a dot-path segment, so
 `mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one.
 Sources merge in order; matching folders combine, any other clash is won by the
-last loaded. Names may not use `.`, `|`, `(`, `{`, `}` or `/`; dot-prefixed entries
+last loaded. Names may not use `.`, `|`, `(`, `{` or `}`; dot-prefixed entries
 are skipped, so a data directory can also be a checkout.
 
 Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`,
@@ -141,7 +141,7 @@ Every character is literal except a `{…}` token:
 | `{name.field}` | `field` of one draw of `name` ([Correlated fields](#correlated-fields)) |
 | `{a\|b}` | one of the named fields, even odds |
 | `{fn(args)}` | a builtin ([Functions](#functions)) |
-| `{/path}`, `{.path}`, `{..path}` | a node reached from the data root, this file's folder, or the folder above ([References](#references)) |
+| `{..path}` | a node reached from the data root ([References](#references)) |
 | `{{`, `}}` | a literal `{` or `}` |
 
 ```json
@@ -267,25 +267,21 @@ a mix.
 
 ### References
 
-A reference renders a node from elsewhere in the data — the path `Fake` takes,
+`{..path}` renders a node from the **data root** — the path `Fake` takes, across
-across every loaded source — so one category borrows another. The sigil says
+every loaded source — so one category borrows another, even across folders or
-where the path starts, as in a filesystem: `{/en_US.person}` from the data root,
+layered directories:
-`{.username}` from the folder this file sits in, `{..username}` from the folder
-above. `data/sv_SE/email.json` can therefore read its own locale's `username`
-without naming `sv_SE`:
 
 ```json
-"Hej, {/en_US.person}!"
+"Hej, {..en_US.person}!"
 ```
 
 Renders e.g. `Hej, Pat Smith!`. A reference into a category is held like a
-[correlated](#correlated-fields) path — `{.person.first} {.person.last}` name one
+[correlated](#correlated-fields) path — `{..sv_SE.person.first} {..sv_SE.person.last}`
-person, `{lowercase(.person.first)}` reads that same draw, and `{.person.first}`
+name one person, `{lowercase(..sv_SE.person.first)}` reads that same draw — while
-beside `{/sv_SE.person.last}` in `sv_SE` is one person too — while a bare
+a bare `{..misc.uuid} {..misc.uuid}` is two draws. Rejected at `New`: a path that
-`{/misc.uuid} {/misc.uuid}` is two draws. Rejected at `New`: a path that is
+is unknown, names a folder, or reads a field not every variant of a choice
-unknown, names a folder, has no folder above, or reads a field not every variant
+carries, and a reference that leads back to its own value, directly, mutually or
-of a choice carries, and a reference that leads back to its own value, directly,
+through a chain.
-mutually or through a chain.
 
 ### Correlated fields
 
@@ -319,8 +315,8 @@ The sub-fields stay addressable — `Fake("address.place.locality")` renders, an
 
 A name any token reads as a path (`{p.first}`) or as an operand (`{calc(net * 2)}`,
 `{uppercase(w)}`) is drawn **once per expansion**, and every other route to it —
-a bare `{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering
+a bare `{p}`, a second bare `{w}`, `{..cat.net}`, a nested template rendering
-`{/cat.p.last}`, at any depth — is a load error naming the spelling to use. A
+`{..cat.p.last}`, at any depth — is a load error naming the spelling to use. A
 name nothing reads that way is drawn each time: `{word} {word}` differs. An
 expansion is one render of one format, so each `repeat` iteration and each nested
 template draws again.
@@ -355,15 +351,11 @@ tokens add cost in proportion to the output.
 - **The shipped data is embedded, not discovered.** A directory a machine happens
   to have would make `--seed 42` machine-dependent. Data still lives in `data/`
   as JSON; `--data-path` layers over it.
-- **A bare reference draws each time; a reference path is held.** `{/p} {/p}`
+- **A bare reference draws each time; a reference path is held.** `{..p} {..p}`
-  is two draws, as `{word} {word}` is, while `{/p.first}` beside a nested template
+  is two draws, as `{word} {word}` is, while `{..p.first}` beside a nested template
-  rendering `{/p.first}` is a load error: a bare token is by contract an
+  rendering `{..p.first}` is a load error: a bare token is by contract an
   independent draw, a path pins its level, and any route into a pinned level from
   another expansion could show another row.
-- **Reference sigils follow the filesystem.** `/` is the root, `.` this file's
-  folder, `..` the folder above — what those spellings already mean to anyone who
-  has typed a path. A locale's files reach each other without naming the locale,
-  so a folder renames and copies without editing its references.
 - **Samples say what they emit, transforms what they do.** `{upper(2)}` is two
   letters, `{uppercase(x)}` is `x` upper-cased; one name for both would turn on
   whether the argument looks like a number.
@@ -406,7 +398,7 @@ fejkdata.go        Generator, New, options, the embedded data set, List
 node.go            the node model and JSON -> node compilation
 render.go          Fake and the recursive renderer (choices, format strings, paths, held draws)
 template.go        the {token} grammar: scanning, arms, operands, validation
-reference.go       {/path} binding across the tree, the render graph, and the walks over it
+reference.go       {..path} binding across the tree, the render graph, and the walks over it
 builtins.go        the {name()} function registry and its implementations
 calc.go            the {calc()} arithmetic evaluator: parser, eval, validation
 data.go            data loading: fs.FS folders/files -> namespace tree, multi-source merge

bound_test.go

@@ -642,11 +642,12 @@ func TestDeepPathsUnderOneHeadStayIndependentWhereTheyDiverge(t *testing.T) {
 }
 
 func TestEmptyPathSegmentIsRejected(t *testing.T) {
-	// "{a.}" and "{a..b}" are unfinished paths, and the segment naming nothing is
+	// "{a.}", "{.b}" and "{a..b}" are unfinished paths, and the segment naming
-	// reported as that rather than as a missing field. An empty name is rejected
+	// nothing is reported as that rather than as a missing field. An empty name is
-	// where it is authored, so no data can make these resolve.
+	// rejected where it is authored, so no data can make these resolve.
 	rejected := map[string]string{
 		"trailing dot": `{"format":"[{a.}]","a":"x"}`,
+		"leading dot":  `{"format":"[{.b}]","a":{"format":"{b}","b":"V"}}`,
 		"double dot":   `{"format":"[{a..b}]","a":"x"}`,
 	}
 	for name, file := range rejected {

builtins.go

@@ -129,8 +129,10 @@ func transformArg(fields map[string]node, a []string) error {
 		return err
 	}
 	if isRef(leaf) {
-		_, _, err := refShape(leaf)
+		if leaf == refPrefix {
-		return err
+			return fmt.Errorf("reference has no path")
+		}
+		return nil
 	}
 	return checkArm(leaf, fields)
 }

data.go

@@ -31,7 +31,7 @@ func (s dataSource) name(p string) string {
 // keyed by its base name (address.json -> "address"); each subdirectory becomes a
 // nested group, so folders turn into dot-path segments. Sources merge left to right:
 // matching groups merge by their children, and any other clash is won by the last
-// source loaded. Once merged, linkRefs binds every reference against the
+// source loaded. Once merged, linkRefs binds every {..path} reference against the
 // final tree.
 func loadData(sources []dataSource) (map[string]node, error) {
 	root := map[string]node{}

edge_test.go

@@ -146,13 +146,13 @@ func TestNewErrors(t *testing.T) {
 		// A reference arm is the only kind that reaches the repeat check by passing
 		// the per-arm checks rather than falling through them.
 		"repeated reference arm": {
-			map[string]string{"a": `"x"`, "b": `"{/a|/a}"`},
+			map[string]string{"a": `"x"`, "b": `"{/a|..a}"`},
-			`arm "/a" is repeated`,
+			`arm "..a" is repeated`,
 		},
 		// An arm that is broken on its own terms is reported as that, not as a
 		// repeat: the repeat is a consequence of the real mistake.
 		"repeated arm with no path": {
-			map[string]string{"a": `"{/|/}"`},
+			map[string]string{"a": `"{/|..}"`},
 			"reference has no path",
 		},
 		// No field can be named "", so the token is told that rather than sent to

node.go

@@ -41,11 +41,11 @@ type template struct {
 	fields    map[string]node
 	repeat    int
 	separator string
-	ops       []op                  // format compiled once (see compileOps); what expand walks
+	ops       []op              // format compiled once (see compileOps); what expand walks
-	grow      int                   // minimum output size, to size the render buffer
+	grow      int               // minimum output size, to size the render buffer
-	fixed     bool                  // no op varies, so every render is lit
+	fixed     bool              // no op varies, so every render is lit
-	lit       string                // the whole output when fixed
+	lit       string            // the whole output when fixed
-	refs      map[string]refBinding // each reference the format reads -> what it is bound to
+	refs      map[string]string // each {..path} the format reads -> the head it is bound under
 	// bound maps each field the format addresses by dotted path to one path token
 	// reading it, which is the half of an overlap the fences name. nil when the
 	// format takes no path.
@@ -208,6 +208,9 @@ func compileTemplate(m map[string]any) (node, error) {
 		if isOption(k) {
 			continue
 		}
+		if isRef(k) {
+			return nil, fmt.Errorf("field %q starts with %q, which is reserved for {..path} bindings", k, refPrefix)
+		}
 		if err := checkName(k); err != nil {
 			return nil, fmt.Errorf("field %w", err)
 		}
@@ -318,10 +321,10 @@ func weightOf(raw any) (float64, error) {
 
 // reservedInName is what a category, folder or field name may not contain: a dot
 // separates the segments of a path, '|' the arms of a token, '(' opens a function
-// call, braces delimit the token and '/' starts a reference. A name carrying one is
+// call and braces delimit the token. A name carrying one is reachable by no format,
-// reachable by no format, so it is rejected where it is authored rather than at
+// so it is rejected where it is authored rather than at the token that cannot
-// the token that cannot reach it.
+// reach it.
-const reservedInName = ".|({}/"
+const reservedInName = ".|({}"
 
 // reservedList spells reservedInName for an error message, so the two cannot drift.
 var reservedList = strings.Join(strings.Split(reservedInName, ""), " ")

reference.go

@@ -6,70 +6,26 @@ import (
 	"strings"
 )
 
-// A reference names a node by path rather than as a sibling field: {/a.b} from the
+// refPrefix marks a {..path} token: a reference to a node elsewhere in the data
-// data root, {.a} from the folder this file sits in, {..a} from the folder above.
+// root rather than a sibling field. The path is resolved across every loaded
-func isRef(name string) bool { return strings.HasPrefix(name, ".") || strings.HasPrefix(name, "/") }
+// directory (see linkRefs).
+const refPrefix = ".."
 
-// refBinding is what a reference was bound to: the head key its category is held
+func isRef(name string) bool { return strings.HasPrefix(name, refPrefix) }
-// under, and the tail read into it.
-type refBinding struct {
-	key  string
-	tail []string
-}
 
-// refShape splits a reference into its sigil and the dotted path after it.
+// linkRefs resolves every {..path} reference in the assembled tree. The head of the
-func refShape(name string) (sigil, rest string, err error) {
+// path — up to the category it names — is bound into the referring template's
-	switch {
+// fields, and the rest reads into it the way a sibling path does, so a reference
-	case strings.HasPrefix(name, "/"):
+// is held like a sibling. It runs once, after all data is merged, so a reference
-		sigil, rest = "/", name[1:]
+// sees the final (override-resolved) tree. A path that is unknown, names a folder,
-	case strings.HasPrefix(name, ".."):
+// or reads a field not every variant carries fails here, keeping a bad reference a
-		sigil, rest = "..", name[2:]
+// New-time error, never a random render-time one.
-	default:
-		sigil, rest = ".", name[1:]
-	}
-	if rest == "" {
-		return "", "", fmt.Errorf("reference has no path")
-	}
-	if strings.HasPrefix(rest, ".") {
-		return "", "", fmt.Errorf("a reference starts with / (the root), . (this folder) or .. (the folder above)")
-	}
-	for _, seg := range strings.Split(rest, ".") {
-		if seg == "" {
-			return "", "", fmt.Errorf("path has an empty segment")
-		}
-	}
-	return sigil, rest, nil
-}
-
-// refSegments resolves a reference written in folder to a path from the root.
-func refSegments(name string, folder []string) ([]string, error) {
-	sigil, rest, err := refShape(name)
-	if err != nil {
-		return nil, err
-	}
-	var base []string
-	switch sigil {
-	case ".":
-		base = folder
-	case "..":
-		if len(folder) == 0 {
-			return nil, fmt.Errorf("no folder above the root")
-		}
-		base = folder[:len(folder)-1]
-	}
-	return append(append([]string{}, base...), strings.Split(rest, ".")...), nil
-}
-
-// linkRefs resolves every reference in the assembled tree. The head of the path —
-// up to the category it names — is bound into the referring template's fields
-// under its root path, and the rest reads into it the way a sibling path does, so
-// a reference is held like a sibling and two spellings of one target are one
-// draw. It runs once, after all data is merged, so a reference sees the final
-// (override-resolved) tree. A path that is unknown, names a folder, or reads a
-// field not every variant carries fails here, keeping a bad reference a New-time
-// error, never a random render-time one.
 func linkRefs(root map[string]node) error {
-	return eachTemplate(root, func(folder []string, path string, t *template) error {
+	return walkNodes(root, func(path string, n node) error {
+		t, ok := n.(*template)
+		if !ok {
+			return nil
+		}
 		names := refTokens(t.format)
 		if len(names) == 0 {
 			return nil
@@ -77,22 +33,18 @@ func linkRefs(root map[string]node) error {
 		if t.fields == nil {
 			t.fields = map[string]node{}
 		}
-		t.refs = make(map[string]refBinding, len(names))
+		t.refs = make(map[string]string, len(names))
 		for _, name := range names {
-			segments, err := refSegments(name, folder)
+			head, target, tail, err := resolveRef(root, strings.Split(name[len(refPrefix):], "."))
 			if err != nil {
 				return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
 			}
-			head, target, tail, err := resolveRef(root, segments)
+			key := refPrefix + strings.Join(head, ".")
-			if err != nil {
-				return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
-			}
-			key := "/" + strings.Join(head, ".")
 			if err := checkPath(target, tail, key); err != nil {
 				return fmt.Errorf("%s: reference {%s}: %w", path, name, err)
 			}
 			t.fields[key] = target
-			t.refs[name] = refBinding{key, tail}
+			t.refs[name] = key
 		}
 		if err := t.compileFormat(); err != nil {
 			return fmt.Errorf("%s: %w", path, err)
@@ -101,42 +53,6 @@ func linkRefs(root map[string]node) error {
 	})
 }
 
-// eachTemplate calls fn once per template, with the folder its category sits in
-// and the dot path reaching it, folders and names in sorted order.
-func eachTemplate(root map[string]node, fn func(folder []string, path string, t *template) error) error {
-	var inCategory func(folder []string, path string, n node) error
-	inCategory = func(folder []string, path string, n node) error {
-		if t, ok := n.(*template); ok {
-			if err := fn(folder, path, t); err != nil {
-				return err
-			}
-		}
-		for _, c := range contained(n) {
-			if err := inCategory(folder, join(path, c.name), c.node); err != nil {
-				return err
-			}
-		}
-		return nil
-	}
-	var inFolder func(folder []string, children map[string]node) error
-	inFolder = func(folder []string, children map[string]node) error {
-		for _, name := range sortedNames(children) {
-			path := join(strings.Join(folder, "."), name)
-			if g, ok := children[name].(*group); ok {
-				if err := inFolder(append(folder[:len(folder):len(folder)], name), g.children); err != nil {
-					return err
-				}
-				continue
-			}
-			if err := inCategory(folder, path, children[name]); err != nil {
-				return err
-			}
-		}
-		return nil
-	}
-	return inFolder(nil, root)
-}
-
 // checkBoundLevelsHeld rejects every route to a held name except the ones that read
 // its draw. An expansion holds one draw of that name; anything else that renders it
 // draws again, and the two disagree. checkNoOverlap settles the spellings within one
@@ -257,7 +173,7 @@ func cover(n node, into map[node]bool, inChoice bool) {
 // whole, so that draw fixes every value the render produced, and a second route to
 // any of them disagrees with it.
 //
-// The walk stops at a reference edge, which is where the operand's own value ends
+// The walk stops at a {..path} edge, which is where the operand's own value ends
 // and a shared source begins: two names referencing one category are two draws, the
 // same rule {word} {word} follows.
 func operandDraw(n node, into map[node]bool) {
@@ -355,7 +271,7 @@ func contained(n node) []namedNode {
 	}
 }
 
-// named skips a bound {/path} key: it is a render edge, not containment, so using
+// named skips a bound {..path} key: it is a render edge, not containment, so using
 // it as a path segment would report a node under a path that does not reach it. Only
 // a template's fields hold bindings — loadDir skips a dot-prefixed entry, so a
 // group's children never carry the prefix — so this one skip serves both.
@@ -401,7 +317,7 @@ func resolveRef(root map[string]node, segments []string) (head []string, target
 	return segments[:i], n, segments[i:], nil
 }
 
-// refTokens returns the reference names a format reads, as tokens or as operands.
+// refTokens returns the {..path} names a format reads, as tokens or as operands.
 func refTokens(format string) []string {
 	var refs []string
 	seen := map[string]bool{}

reference_test.go

@@ -30,12 +30,12 @@ func TestReferenceIntoAField(t *testing.T) {
 	}
 }
 
-// TestReferenceInAlternation lets a reference stand as one arm of a {a|/b}
+// TestReferenceInAlternation lets a reference stand as one arm of a {a|..b}
 // alternation, so a field and a cross-file value share one slot.
 func TestReferenceInAlternation(t *testing.T) {
 	dir := writeData(t, map[string]string{
 		"far":  `"X"`,
-		"near": `{"format":"{here|/far}","here":"H"}`,
+		"near": `{"format":"{here|..far}","here":"H"}`,
 	})
 	f := newGenerator(t, dir, WithSeed(2))
 	seen := map[string]bool{}
@@ -191,7 +191,7 @@ func TestNewErrorPathIsCanonical(t *testing.T) {
 		{
 			"cycle inside a choice arm",
 			map[string]string{"cat": `{"format":"hi","x":"{/cat.x}"}`},
-			"fejkdata: reference cycle: cat.x -> /cat.x",
+			"fejkdata: reference cycle: cat.x -> ..cat.x",
 		},
 		{
 			"bad reference reached through another reference",

render.go

@@ -162,7 +162,7 @@ type draws struct {
 }
 
 // readField renders one arm of a token. An arm's key is a sibling field or a
-// {/path} reference, which linkRefs bound into fields too. A name the expansion
+// {..path} reference, which linkRefs bound into fields too. A name the expansion
 // holds — a level some token addresses by dotted path, or a sibling a {calc()}
 // reads — is drawn once and kept, so {place.postal-code} and {place.locality} read
 // one row, either read twice gives one value, and a shown operand is the operand

template.go

@@ -135,10 +135,10 @@ func checkTokens(format string, fields map[string]node) error {
 		names := strings.Split(t.body, "|")
 		for _, name := range names {
 			if isRef(name) {
-				if _, _, err := refShape(name); err != nil {
+				if name == refPrefix {
-					return fmt.Errorf("token {%s}: %w", t.body, err)
+					return fmt.Errorf("token {%s}: reference has no path", t.body)
 				}
-				continue // its target is checked at New (see linkRefs)
+				continue // a root reference; its target is checked at New (see linkRefs)
 			}
 			if err := checkArm(name, fields); err != nil {
 				return fmt.Errorf("token {%s}: %w", t.body, err)
@@ -230,7 +230,7 @@ func fieldTokens(format string) []string {
 }
 
 // arm is one alternative of a {a|b} token or one operand, split into the key
-// naming the node in a template's fields (a sibling field, or the head a
+// naming the node in a template's fields (a sibling field, or the "..path" head a
 // reference is bound under) and the tail of a dotted path into it. A non-empty
 // tail is what makes the arm a bound draw: its head is drawn once per expansion
 // (see compileOps).
@@ -241,19 +241,15 @@ type arm struct {
 	steps []string // key per level passed through; the head and leaf hold their own
 }
 
-// splitArm splits one name into key and tail. refs maps a reference to what
+// splitArm splits one name into key and tail. refs maps a reference to the head
-// linkRefs bound it to; before linking, a reference is whole.
+// linkRefs bound it under; before linking, a reference is whole.
-func splitArm(name string, refs map[string]refBinding) arm {
+func splitArm(name string, refs map[string]string) arm {
 	if isRef(name) {
-		b, bound := refs[name]
+		key, bound := refs[name]
-		if !bound || len(b.tail) == 0 {
+		if !bound || key == name {
-			key := name
+			return arm{name: name, key: name}
-			if bound {
-				key = b.key
-			}
-			return arm{name: name, key: key}
 		}
-		return pathArm(name, b.key, b.tail)
+		return pathArm(name, key, strings.Split(name[len(key)+1:], "."))
 	}
 	head, tail, dotted := strings.Cut(name, ".")
 	if !dotted {
@@ -275,7 +271,7 @@ func pathArm(name, key string, segs []string) arm {
 // level's held draw while rendering the level expands it afresh, so their values
 // would disagree. Names are compared in sorted order, so which pair is reported
 // does not depend on where the tokens sit.
-func checkNoOverlap(format string, bound map[string]string, refs map[string]refBinding) error {
+func checkNoOverlap(format string, bound map[string]string, refs map[string]string) error {
 	names := boundReaders(format, bound, refs)
 	// Stable over one format-order scan, so two readers of one name (a token and a
 	// calc operand both naming "p") are reported as the format writes them.
@@ -296,7 +292,7 @@ type reader struct{ name, label string }
 // boundReaders lists every way a format reaches a bound field, in the order the
 // format writes them. An operand renders its field, so it names a level exactly
 // as a token does; one scan finds both, which is what puts them in one order.
-func boundReaders(format string, bound map[string]string, refs map[string]refBinding) []reader {
+func boundReaders(format string, bound map[string]string, refs map[string]string) []reader {
 	var names []reader
 	_ = eachToken(format, func(t ftoken) error {
 		if t.kind != 'b' {
@@ -340,7 +336,7 @@ func checkSegments(a arm) error {
 }
 
 // splitArms splits a token body's '|' alternatives.
-func splitArms(body string, refs map[string]refBinding) []arm {
+func splitArms(body string, refs map[string]string) []arm {
 	parts := strings.Split(body, "|")
 	arms := make([]arm, len(parts))
 	for i, p := range parts {
@@ -400,7 +396,7 @@ func (c *formatOps) hold(a arm, label string) {
 	}
 }
 
-func (c *formatOps) function(body string, refs map[string]refBinding) {
+func (c *formatOps) function(body string, refs map[string]string) {
 	name, args, _ := funcCall(body)
 	var operands []arm
 	for _, operand := range tokenOperands(body) {
@@ -411,7 +407,7 @@ func (c *formatOps) function(body string, refs map[string]refBinding) {
 	c.ops = append(c.ops, op{kind: 'b', call: builtins[name].prep(args), operands: operands})
 }
 
-func (c *formatOps) field(body string, refs map[string]refBinding) {
+func (c *formatOps) field(body string, refs map[string]string) {
 	arms := splitArms(body, refs)
 	for _, a := range arms {
 		if len(a.tail) > 0 {
@@ -423,7 +419,7 @@ func (c *formatOps) field(body string, refs map[string]refBinding) {
 
 // compileOps compiles a format string. Call checkTokens first: it is what proves
 // the scan and every token are valid.
-func compileOps(format string, refs map[string]refBinding) formatOps {
+func compileOps(format string, refs map[string]string) formatOps {
 	var c formatOps
 	var lit strings.Builder
 	flush := func() {
@@ -454,7 +450,7 @@ func compileOps(format string, refs map[string]refBinding) formatOps {
 // checkNoRepeatedRead rejects a bare token repeated on a held name: {w} {w} beside
 // {uppercase(w)} would read one draw twice, where {w} {w} alone draws twice. The
 // error names the single-token spelling.
-func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding) error {
+func checkNoRepeatedRead(format string, c formatOps, refs map[string]string) error {
 	count := map[string]int{}
 	return eachToken(format, func(t ftoken) error {
 		if t.kind != 'b' {