From 13e3de39fe50a963d5fb6f22f823b564c80edf9f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 10 Oct 2026 11:14:19 +0200 Subject: [PATCH] Answer product-owner round 1: the parent error and the README name the way to link under another source's table, a missing indexed file names the manifest, the README warns a copied data/ carries its index, and the docs say what a load failure and a replacement do --- CHANGELOG.md | 7 +++-- README.md | 54 +++++++++++++++++++--------------- docs/decisions.md | 13 ++++---- fejkdata.go | 4 +-- index.go | 24 +++++++++++---- internal/datafiles/manifest.go | 2 +- node.go | 2 +- todo.md | 4 +-- 8 files changed, 64 insertions(+), 46 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7793a5a..a6dde29 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -127,6 +127,7 @@ replacement, and each removed path, column or flag. that gives it one. - A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each category on the first call reaching it, as the shipped set does. Data failing to load, - in `New` or there, gives an error matching `ErrLoad`, and the CLI exits 1. A source without - one loads in `New`, with the shipped categories it reads; the rest of the shipped set, - a reader of a category it replaces too, loads on first reach. + in `New` or on that first call, gives an error matching `ErrLoad`, and the CLI exits 1. + A source without an index loads in `New`, with the shipped categories it reads. Every + other shipped category loads on the first call reaching it, so one your replacement + breaks fails there. diff --git a/README.md b/README.md index d7bb700..80880f6 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,8 @@ template holding one from a quoted heredoc or a file. The rest of stdin's rules `--name value` and `--name=value` both work, a short flag's value attaches or follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see -[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing -dir, data that fails to load, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument +[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (data +that fails to load, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument other than a flag, an unreadable stdin, nothing piped in, or a template that does not compile. From a checkout: `go run ./cmd/fejkdata …`. @@ -173,15 +173,20 @@ entry is never a category or a folder, so a data directory can also be a checkou a category may name a hidden rows file beside it. A source whose root holds `.fejkdata.json` with an `index` loads each category on the -first call reaching it, and its author proves each loads. Any other source loads in -`New`, with the indexed categories it reads. A shipped category reading one that another -source replaced loads on its own first reach, and fails there if the replacement breaks -it. The index names every category of its source, so a category file it leaves out never -loads. The shipped set's [`data/.fejkdata.json`](data/.fejkdata.json) shows the shape: -each category's dot path, its `parent` table, an entry of the same index, and every path -`List` advertises below it, `""` for the category itself. A table whose parent column -differs from its entry's `parent` fails at first reach. -`//go:embed` of a directory leaves the manifest out; name it in the pattern too. +first call reaching it, so `New` does not check them: load each category once in a test, +as CI does for the shipped set. Any other source loads in `New`, with the indexed +categories it reads. An indexed category reading one that another source replaced loads +on its own first reach, and fails there if the replacement breaks it. The index names +every category of its source, so a category file it leaves out never loads. + +The index maps each category's dot path to an entry: `parent`, the table it links to, +which needs its own entry in the same folder; and `paths`, every path `List` advertises +below the category, `""` for the category itself. The shipped set's +[`data/.fejkdata.json`](data/.fejkdata.json) is one, so delete it from a copy of `data/` +you edit, or keep its index current. A table whose parent column differs from its +entry's `parent` fails at first reach, and a table linking under another source's table +needs a source without an index. `//go:embed` of a directory leaves the manifest out, and +the source then loads whole in `New`; name it in the pattern too. Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, @@ -342,13 +347,14 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el `New` refuses data that breaks the grammar, could mean two things or cannot render a valid value, and `NewTemplate` refuses such a template. A category of an indexed source is -checked on the first call reaching it instead. Data that fails to load, in `New` or -there, gives an error matching `ErrLoad`, the same every call. On a loaded generator: +checked on the first call reaching it instead. Data that fails to load, in `New` or on +that first call, gives an error matching `ErrLoad`. On a loaded generator: -- `Fake` fails only for a path that names nothing, could name two things, or reads one - draw of a level carrying a `repeat`, with the same error every call, or with `ErrLoad`. -- `FakeStruct` fails only for a non-struct argument or a type its tags do not describe, - with the same error every call, or with `ErrLoad`. +- `Fake` fails only for a path that names nothing, could name two things or reads one + draw of a level carrying a `repeat`, or for data that fails to load (`ErrLoad`), with + the same error every call. +- `FakeStruct` fails only for a non-struct argument, a type its tags do not describe, or + data that fails to load (`ErrLoad`), with the same error every call. - `Template.Fake` cannot fail. A path, wherever a call takes one, may start with one `/`, as a reference does. @@ -848,12 +854,12 @@ The target is about a microsecond per value (goal 13). A source with an [index](#data), as the shipped set has, loads nothing in `New`: a category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads -it, with every category it reads and its whole table family. Each file is parsed, validated and weight-indexed once, bar a -table's name and parent columns, which are mapped on the first `Fake` that selects by -name or descends through it. A `Fake` call then costs about what its output -costs: an unweighted pick is O(1) whatever the list's length, a weighted one -O(log n), and long formats, deep nesting and many tokens add cost in proportion to -the output. +it, with every category it reads and its whole table family. Each file is parsed, +validated and weight-indexed once, bar a table's name and parent columns, which are +mapped on the first `Fake` that selects by name or descends through it. A `Fake` call then +costs about what its output costs: an unweighted pick is O(1) whatever the list's length, +a weighted one O(log n), and long formats, deep nesting and many tokens add cost in +proportion to the output. ## Versioning @@ -870,7 +876,7 @@ and regenerate seeded fixtures and expected values when you raise that version. |---------|-------|-------| | Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row | | Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category | -| Data format | a fence: a spelling `New` rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin | +| Data format | a fence: a spelling a load rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin | | CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format | | Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option | diff --git a/docs/decisions.md b/docs/decisions.md index 24f9dfd..f04b18a 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -156,8 +156,7 @@ Valid while the shipped data stays under 100 MiB, which a gate test holds (`todo ## A source whose manifest carries an index loads each category on first reach; any other loads in `New`, with what it reads -2026-10-03, a restructure the maintainer approved; widened to any source on 2026-10-09, -and narrowed by the maintainer on 2026-10-10. +2026-10-03, a restructure the maintainer approved; widened to any source on 2026-10-09. Goal 13: `New` pays nothing for a category a run never reaches. A manifest, `.fejkdata.json` at a source's root, may carry an index: each category's parent table and @@ -171,11 +170,11 @@ at first reach. Goal 6: a source without an index loads whole in `New`, with every indexed category it reads, so each mistake in it is `New`'s error. Goal 6.3: an indexed source from anyone else may fail at first reach; the call then returns the load's error, matching -`ErrLoad`, the same every call. So does an indexed category reading one that another -source replaced, where the replacement breaks it. The maintainer chose that on -2026-10-10 over loading in `New` every indexed category depending on a replaced one: -readers of three rounds ruled that rule harder to follow (goal 3), and it needed every -entry to list what its templates read. +`ErrLoad`, the same every call. An indexed category reading one that another source +replaced fails the same way, at its first reach, when the replacement breaks it. The +maintainer chose that on 2026-10-10 over loading in `New` every indexed category +depending on a replaced one: three rounds of the comprehension panel ruled that rule +harder to follow (goal 3), and it needed every entry to list what its templates read. A first reach checks only a table's `parent` against its entry, so a table loads with its parent and a failed load puts back whole. Proving the rest of an index is its author's, diff --git a/fejkdata.go b/fejkdata.go index 3da4e72..ac3257a 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -84,8 +84,8 @@ func (loadError) Is(target error) bool { return target == ErrLoad } // namespace segment. It errors on a missing directory, invalid JSON or invalid data; a // generator with no data at all renders templates that read none. A source whose // .fejkdata.json carries an index, as the shipped set's does, loads each category on the -// first call reaching it. Data failing to load, here or there, gives an error matching -// [ErrLoad]. +// first call reaching it. Data failing to load, in New or on that first call, gives an +// error matching [ErrLoad]. func New(opts ...Option) (*Generator, error) { c := config{shipped: true} for _, opt := range opts { diff --git a/index.go b/index.go index c3803ad..fa1c67e 100644 --- a/index.go +++ b/index.go @@ -1,7 +1,9 @@ package fejkdata import ( + "errors" "fmt" + "io/fs" "slices" "strings" @@ -15,8 +17,8 @@ type indexed struct { src datafiles.Source } -// placeIndex sets every category index names unloaded in root, in place of what root holds -// at its path. +// placeIndex sets every category the index names as unloaded in root, in place of what +// root holds at its path. func placeIndex(root *folder, src datafiles.Source, index map[string]datafiles.IndexEntry) { for _, p := range sortedNames(index) { segs := strings.Split(p, ".") @@ -128,9 +130,9 @@ type loaded struct { was *indexed } -// load loads each unloaded category in queue and what it needs. A table whose parent -// column disagrees with its entry stops it, so a table loads with its parent and no table -// loaded before links to one putBack removes. +// load loads each unloaded category in queue and what it needs. It stops at a table whose +// parent column disagrees with its entry. Otherwise a table could load without its parent, +// or an earlier table could link to one that putBack then removes. func (b *batch) load(root *folder, queue []categoryAt) error { for ; len(queue) > 0; queue = queue[1:] { c := queue[0] @@ -139,12 +141,15 @@ func (b *batch) load(root *folder, queue []categoryAt) error { continue } if err := x.src.Load(c.dir, c.name, compileInto(func([]string) *folder { return c.in })); err != nil { + if errors.Is(err, fs.ErrNotExist) { + err = fmt.Errorf("%s indexes %s: %w", x.src.ManifestPath(), categoryPath(c.dir, c.name), err) + } return err } s := siteIn(c.dir, c.in, c.name) *b = append(*b, loaded{site: s, was: x}) if parent := parentOf(s.n); parent != x.entry.Parent { - return fmt.Errorf("%s: the index entry for %s names parent %q, and its table's parent column is %q", x.src.ManifestPath(), s.path, x.entry.Parent, parent) + return fmt.Errorf("%s: the index entry for %s %s, and its table %s", x.src.ManifestPath(), s.path, namesParent(x.entry.Parent), namesParent(parent)) } queue = append(queue, needs(root, s)...) } @@ -168,6 +173,13 @@ func (b batch) putBack() { } } +func namesParent(parent string) string { + if parent == "" { + return "names no parent" + } + return fmt.Sprintf("names parent %q", parent) +} + func parentOf(n node) string { if t, isTable := n.(*table); isTable { return t.rows.Options().Parent diff --git a/internal/datafiles/manifest.go b/internal/datafiles/manifest.go index 25f1a2a..f39384b 100644 --- a/internal/datafiles/manifest.go +++ b/internal/datafiles/manifest.go @@ -176,7 +176,7 @@ func checkParent(index map[string]IndexEntry, p string) error { sibling = p[:dot+1] + parent } if _, ok := index[sibling]; !ok { - return fmt.Errorf("parent %q names no entry beside it in this index", parent) + return fmt.Errorf("parent %q names no entry in this entry's folder; a table in an indexed source links only to a parent in the same index, so to link under another source's table, drop this index and the source loads in New", parent) } return nil } diff --git a/node.go b/node.go index 5e23f5d..fe19297 100644 --- a/node.go +++ b/node.go @@ -23,7 +23,7 @@ type node interface{ isNode() } // A name stands in children or in unloaded, never both; put and putUnloaded keep it so. type folder struct { children map[string]node - unloaded map[string]*indexed // the categories here an index names and no call has reached yet + unloaded map[string]*indexed } func (*folder) isNode() {} diff --git a/todo.md b/todo.md index 5715054..871c4b5 100644 --- a/todo.md +++ b/todo.md @@ -192,7 +192,7 @@ Carries out the decision "The shipped data is Go packages a library imports by c ### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names. -Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. An entry lists every path below its category, its linked tables' too, so `geo.*.region` repeats its whole family; an entry holding only its own paths, `List` composing a family from the entries naming it as `parent`, would shrink the manifest that decoding makes most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10). Record the ruling in `docs/`. +Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`. ### 16. Add locale categories: company, phone, finance, vehicle, words. @@ -425,7 +425,7 @@ A hyphen is always subtraction in a calc, and the shipped data has hyphenated fi ### 194. Let a data author write and check their own `.fejkdata.json` index with the library or the CLI, and fail a load whose index leaves out a category file. -Goal 6.3 has an author prove their module, and goal 9.1 has fejkdata's data load as anyone's; today only a test of fejkdata's own writes and checks an index (`TestShippedManifestIsCurrent` over `indexEntry`). A category file the index leaves out never loads, silently. The principles audit of item 180's chunk found it. +Goal 6.3 has an author prove their module, and goal 9.1 has fejkdata's data load as anyone's; today only fejkdata's own test, `TestShippedManifestIsCurrent` over `indexEntry`, writes and checks an index. A category file the index leaves out never loads, silently. ### 53. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.