Answer product-owner round 1: the parent error and the README name the way to link under another source's table, a missing indexed file names the manifest, the README warns a copied data/ carries its index, and the docs say what a load failure and a replacement do

This commit is contained in:
2026-10-10 11:14:19 +02:00
committed by Lillem4n
parent 15477e989f
commit 13e3de39fe
8 changed files with 64 additions and 46 deletions
+4 -3
View File
@@ -127,6 +127,7 @@ replacement, and each removed path, column or flag.
that gives it one. that gives it one.
- A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each - A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each
category on the first call reaching it, as the shipped set does. Data failing to load, category on the first call reaching it, as the shipped set does. Data failing to load,
in `New` or there, gives an error matching `ErrLoad`, and the CLI exits 1. A source without in `New` or on that first call, gives an error matching `ErrLoad`, and the CLI exits 1.
one loads in `New`, with the shipped categories it reads; the rest of the shipped set, A source without an index loads in `New`, with the shipped categories it reads. Every
a reader of a category it replaces too, loads on first reach. other shipped category loads on the first call reaching it, so one your replacement
breaks fails there.
+30 -24
View File
@@ -63,8 +63,8 @@ template holding one from a quoted heredoc or a file. The rest of stdin's rules
`--name value` and `--name=value` both work, a short flag's value attaches or `--name value` and `--name=value` both work, a short flag's value attaches or
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing [Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (data
dir, data that fails to load, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument that fails to load, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument
other than a flag, an unreadable stdin, nothing piped in, or a other than a flag, an unreadable stdin, nothing piped in, or a
template that does not compile. From a checkout: template that does not compile. From a checkout:
`go run ./cmd/fejkdata …`. `go run ./cmd/fejkdata …`.
@@ -173,15 +173,20 @@ entry is never a category or a folder, so a data directory can also be a checkou
a category may name a hidden rows file beside it. a category may name a hidden rows file beside it.
A source whose root holds `.fejkdata.json` with an `index` loads each category on the A source whose root holds `.fejkdata.json` with an `index` loads each category on the
first call reaching it, and its author proves each loads. Any other source loads in first call reaching it, so `New` does not check them: load each category once in a test,
`New`, with the indexed categories it reads. A shipped category reading one that another as CI does for the shipped set. Any other source loads in `New`, with the indexed
source replaced loads on its own first reach, and fails there if the replacement breaks categories it reads. An indexed category reading one that another source replaced loads
it. The index names every category of its source, so a category file it leaves out never on its own first reach, and fails there if the replacement breaks it. The index names
loads. The shipped set's [`data/.fejkdata.json`](data/.fejkdata.json) shows the shape: every category of its source, so a category file it leaves out never loads.
each category's dot path, its `parent` table, an entry of the same index, and every path
`List` advertises below it, `""` for the category itself. A table whose parent column The index maps each category's dot path to an entry: `parent`, the table it links to,
differs from its entry's `parent` fails at first reach. which needs its own entry in the same folder; and `paths`, every path `List` advertises
`//go:embed` of a directory leaves the manifest out; name it in the pattern too. below the category, `""` for the category itself. The shipped set's
[`data/.fejkdata.json`](data/.fejkdata.json) is one, so delete it from a copy of `data/`
you edit, or keep its index current. A table whose parent column differs from its
entry's `parent` fails at first reach, and a table linking under another source's table
needs a source without an index. `//go:embed` of a directory leaves the manifest out, and
the source then loads whole in `New`; name it in the pattern too.
Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
@@ -342,13 +347,14 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el
`New` refuses data that breaks the grammar, could mean two things or cannot render a valid `New` refuses data that breaks the grammar, could mean two things or cannot render a valid
value, and `NewTemplate` refuses such a template. A category of an indexed source is value, and `NewTemplate` refuses such a template. A category of an indexed source is
checked on the first call reaching it instead. Data that fails to load, in `New` or checked on the first call reaching it instead. Data that fails to load, in `New` or on
there, gives an error matching `ErrLoad`, the same every call. On a loaded generator: that first call, gives an error matching `ErrLoad`. On a loaded generator:
- `Fake` fails only for a path that names nothing, could name two things, or reads one - `Fake` fails only for a path that names nothing, could name two things or reads one
draw of a level carrying a `repeat`, with the same error every call, or with `ErrLoad`. draw of a level carrying a `repeat`, or for data that fails to load (`ErrLoad`), with
- `FakeStruct` fails only for a non-struct argument or a type its tags do not describe, the same error every call.
with the same error every call, or with `ErrLoad`. - `FakeStruct` fails only for a non-struct argument, a type its tags do not describe, or
data that fails to load (`ErrLoad`), with the same error every call.
- `Template.Fake` cannot fail. - `Template.Fake` cannot fail.
A path, wherever a call takes one, may start with one `/`, as a reference does. A path, wherever a call takes one, may start with one `/`, as a reference does.
@@ -848,12 +854,12 @@ The target is about a microsecond per value (goal 13).
A source with an [index](#data), as the shipped set has, loads nothing in `New`: a A source with an [index](#data), as the shipped set has, loads nothing in `New`: a
category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads
it, with every category it reads and its whole table family. Each file is parsed, validated and weight-indexed once, bar a it, with every category it reads and its whole table family. Each file is parsed,
table's name and parent columns, which are mapped on the first `Fake` that selects by validated and weight-indexed once, bar a table's name and parent columns, which are
name or descends through it. A `Fake` call then costs about what its output mapped on the first `Fake` that selects by name or descends through it. A `Fake` call then
costs: an unweighted pick is O(1) whatever the list's length, a weighted one costs about what its output costs: an unweighted pick is O(1) whatever the list's length,
O(log n), and long formats, deep nesting and many tokens add cost in proportion to a weighted one O(log n), and long formats, deep nesting and many tokens add cost in
the output. proportion to the output.
## Versioning ## Versioning
@@ -870,7 +876,7 @@ and regenerate seeded fixtures and expected values when you raise that version.
|---------|-------|-------| |---------|-------|-------|
| Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row | | Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row |
| Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category | | Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category |
| Data format | a fence: a spelling `New` rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin | | Data format | a fence: a spelling a load rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin |
| CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format | | CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format |
| Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option | | Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option |
+6 -7
View File
@@ -156,8 +156,7 @@ Valid while the shipped data stays under 100 MiB, which a gate test holds (`todo
## A source whose manifest carries an index loads each category on first reach; any other loads in `New`, with what it reads ## A source whose manifest carries an index loads each category on first reach; any other loads in `New`, with what it reads
2026-10-03, a restructure the maintainer approved; widened to any source on 2026-10-09, 2026-10-03, a restructure the maintainer approved; widened to any source on 2026-10-09.
and narrowed by the maintainer on 2026-10-10.
Goal 13: `New` pays nothing for a category a run never reaches. A manifest, Goal 13: `New` pays nothing for a category a run never reaches. A manifest,
`.fejkdata.json` at a source's root, may carry an index: each category's parent table and `.fejkdata.json` at a source's root, may carry an index: each category's parent table and
@@ -171,11 +170,11 @@ at first reach.
Goal 6: a source without an index loads whole in `New`, with every indexed category it Goal 6: a source without an index loads whole in `New`, with every indexed category it
reads, so each mistake in it is `New`'s error. Goal 6.3: an indexed source from anyone reads, so each mistake in it is `New`'s error. Goal 6.3: an indexed source from anyone
else may fail at first reach; the call then returns the load's error, matching else may fail at first reach; the call then returns the load's error, matching
`ErrLoad`, the same every call. So does an indexed category reading one that another `ErrLoad`, the same every call. An indexed category reading one that another source
source replaced, where the replacement breaks it. The maintainer chose that on replaced fails the same way, at its first reach, when the replacement breaks it. The
2026-10-10 over loading in `New` every indexed category depending on a replaced one: maintainer chose that on 2026-10-10 over loading in `New` every indexed category
readers of three rounds ruled that rule harder to follow (goal 3), and it needed every depending on a replaced one: three rounds of the comprehension panel ruled that rule
entry to list what its templates read. harder to follow (goal 3), and it needed every entry to list what its templates read.
A first reach checks only a table's `parent` against its entry, so a table loads with its A first reach checks only a table's `parent` against its entry, so a table loads with its
parent and a failed load puts back whole. Proving the rest of an index is its author's, parent and a failed load puts back whole. Proving the rest of an index is its author's,
+2 -2
View File
@@ -84,8 +84,8 @@ func (loadError) Is(target error) bool { return target == ErrLoad }
// namespace segment. It errors on a missing directory, invalid JSON or invalid data; a // namespace segment. It errors on a missing directory, invalid JSON or invalid data; a
// generator with no data at all renders templates that read none. A source whose // generator with no data at all renders templates that read none. A source whose
// .fejkdata.json carries an index, as the shipped set's does, loads each category on the // .fejkdata.json carries an index, as the shipped set's does, loads each category on the
// first call reaching it. Data failing to load, here or there, gives an error matching // first call reaching it. Data failing to load, in New or on that first call, gives an
// [ErrLoad]. // error matching [ErrLoad].
func New(opts ...Option) (*Generator, error) { func New(opts ...Option) (*Generator, error) {
c := config{shipped: true} c := config{shipped: true}
for _, opt := range opts { for _, opt := range opts {
+18 -6
View File
@@ -1,7 +1,9 @@
package fejkdata package fejkdata
import ( import (
"errors"
"fmt" "fmt"
"io/fs"
"slices" "slices"
"strings" "strings"
@@ -15,8 +17,8 @@ type indexed struct {
src datafiles.Source src datafiles.Source
} }
// placeIndex sets every category index names unloaded in root, in place of what root holds // placeIndex sets every category the index names as unloaded in root, in place of what
// at its path. // root holds at its path.
func placeIndex(root *folder, src datafiles.Source, index map[string]datafiles.IndexEntry) { func placeIndex(root *folder, src datafiles.Source, index map[string]datafiles.IndexEntry) {
for _, p := range sortedNames(index) { for _, p := range sortedNames(index) {
segs := strings.Split(p, ".") segs := strings.Split(p, ".")
@@ -128,9 +130,9 @@ type loaded struct {
was *indexed was *indexed
} }
// load loads each unloaded category in queue and what it needs. A table whose parent // load loads each unloaded category in queue and what it needs. It stops at a table whose
// column disagrees with its entry stops it, so a table loads with its parent and no table // parent column disagrees with its entry. Otherwise a table could load without its parent,
// loaded before links to one putBack removes. // or an earlier table could link to one that putBack then removes.
func (b *batch) load(root *folder, queue []categoryAt) error { func (b *batch) load(root *folder, queue []categoryAt) error {
for ; len(queue) > 0; queue = queue[1:] { for ; len(queue) > 0; queue = queue[1:] {
c := queue[0] c := queue[0]
@@ -139,12 +141,15 @@ func (b *batch) load(root *folder, queue []categoryAt) error {
continue continue
} }
if err := x.src.Load(c.dir, c.name, compileInto(func([]string) *folder { return c.in })); err != nil { if err := x.src.Load(c.dir, c.name, compileInto(func([]string) *folder { return c.in })); err != nil {
if errors.Is(err, fs.ErrNotExist) {
err = fmt.Errorf("%s indexes %s: %w", x.src.ManifestPath(), categoryPath(c.dir, c.name), err)
}
return err return err
} }
s := siteIn(c.dir, c.in, c.name) s := siteIn(c.dir, c.in, c.name)
*b = append(*b, loaded{site: s, was: x}) *b = append(*b, loaded{site: s, was: x})
if parent := parentOf(s.n); parent != x.entry.Parent { if parent := parentOf(s.n); parent != x.entry.Parent {
return fmt.Errorf("%s: the index entry for %s names parent %q, and its table's parent column is %q", x.src.ManifestPath(), s.path, x.entry.Parent, parent) return fmt.Errorf("%s: the index entry for %s %s, and its table %s", x.src.ManifestPath(), s.path, namesParent(x.entry.Parent), namesParent(parent))
} }
queue = append(queue, needs(root, s)...) queue = append(queue, needs(root, s)...)
} }
@@ -168,6 +173,13 @@ func (b batch) putBack() {
} }
} }
func namesParent(parent string) string {
if parent == "" {
return "names no parent"
}
return fmt.Sprintf("names parent %q", parent)
}
func parentOf(n node) string { func parentOf(n node) string {
if t, isTable := n.(*table); isTable { if t, isTable := n.(*table); isTable {
return t.rows.Options().Parent return t.rows.Options().Parent
+1 -1
View File
@@ -176,7 +176,7 @@ func checkParent(index map[string]IndexEntry, p string) error {
sibling = p[:dot+1] + parent sibling = p[:dot+1] + parent
} }
if _, ok := index[sibling]; !ok { if _, ok := index[sibling]; !ok {
return fmt.Errorf("parent %q names no entry beside it in this index", parent) return fmt.Errorf("parent %q names no entry in this entry's folder; a table in an indexed source links only to a parent in the same index, so to link under another source's table, drop this index and the source loads in New", parent)
} }
return nil return nil
} }
+1 -1
View File
@@ -23,7 +23,7 @@ type node interface{ isNode() }
// A name stands in children or in unloaded, never both; put and putUnloaded keep it so. // A name stands in children or in unloaded, never both; put and putUnloaded keep it so.
type folder struct { type folder struct {
children map[string]node children map[string]node
unloaded map[string]*indexed // the categories here an index names and no call has reached yet unloaded map[string]*indexed
} }
func (*folder) isNode() {} func (*folder) isNode() {}
+2 -2
View File
@@ -192,7 +192,7 @@ Carries out the decision "The shipped data is Go packages a library imports by c
### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names. ### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names.
Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. An entry lists every path below its category, its linked tables' too, so `geo.*.region` repeats its whole family; an entry holding only its own paths, `List` composing a family from the entries naming it as `parent`, would shrink the manifest that decoding makes most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10). Record the ruling in `docs/`. Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`.
### 16. Add locale categories: company, phone, finance, vehicle, words. ### 16. Add locale categories: company, phone, finance, vehicle, words.
@@ -425,7 +425,7 @@ A hyphen is always subtraction in a calc, and the shipped data has hyphenated fi
### 194. Let a data author write and check their own `.fejkdata.json` index with the library or the CLI, and fail a load whose index leaves out a category file. ### 194. Let a data author write and check their own `.fejkdata.json` index with the library or the CLI, and fail a load whose index leaves out a category file.
Goal 6.3 has an author prove their module, and goal 9.1 has fejkdata's data load as anyone's; today only a test of fejkdata's own writes and checks an index (`TestShippedManifestIsCurrent` over `indexEntry`). A category file the index leaves out never loads, silently. The principles audit of item 180's chunk found it. Goal 6.3 has an author prove their module, and goal 9.1 has fejkdata's data load as anyone's; today only fejkdata's own test, `TestShippedManifestIsCurrent` over `indexEntry`, writes and checks an index. A category file the index leaves out never loads, silently.
### 53. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean. ### 53. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.