Answer product-owner round 1: the parent error and the README name the way to link under another source's table, a missing indexed file names the manifest, the README warns a copied data/ carries its index, and the docs say what a load failure and a replacement do
This commit is contained in:
+4
-3
@@ -127,6 +127,7 @@ replacement, and each removed path, column or flag.
|
|||||||
that gives it one.
|
that gives it one.
|
||||||
- A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each
|
- A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each
|
||||||
category on the first call reaching it, as the shipped set does. Data failing to load,
|
category on the first call reaching it, as the shipped set does. Data failing to load,
|
||||||
in `New` or there, gives an error matching `ErrLoad`, and the CLI exits 1. A source without
|
in `New` or on that first call, gives an error matching `ErrLoad`, and the CLI exits 1.
|
||||||
one loads in `New`, with the shipped categories it reads; the rest of the shipped set,
|
A source without an index loads in `New`, with the shipped categories it reads. Every
|
||||||
a reader of a category it replaces too, loads on first reach.
|
other shipped category loads on the first call reaching it, so one your replacement
|
||||||
|
breaks fails there.
|
||||||
|
|||||||
@@ -63,8 +63,8 @@ template holding one from a quoted heredoc or a file. The rest of stdin's rules
|
|||||||
|
|
||||||
`--name value` and `--name=value` both work, a short flag's value attaches or
|
`--name value` and `--name=value` both work, a short flag's value attaches or
|
||||||
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
|
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
|
||||||
[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing
|
[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (data
|
||||||
dir, data that fails to load, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument
|
that fails to load, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument
|
||||||
other than a flag, an unreadable stdin, nothing piped in, or a
|
other than a flag, an unreadable stdin, nothing piped in, or a
|
||||||
template that does not compile. From a checkout:
|
template that does not compile. From a checkout:
|
||||||
`go run ./cmd/fejkdata …`.
|
`go run ./cmd/fejkdata …`.
|
||||||
@@ -173,15 +173,20 @@ entry is never a category or a folder, so a data directory can also be a checkou
|
|||||||
a category may name a hidden rows file beside it.
|
a category may name a hidden rows file beside it.
|
||||||
|
|
||||||
A source whose root holds `.fejkdata.json` with an `index` loads each category on the
|
A source whose root holds `.fejkdata.json` with an `index` loads each category on the
|
||||||
first call reaching it, and its author proves each loads. Any other source loads in
|
first call reaching it, so `New` does not check them: load each category once in a test,
|
||||||
`New`, with the indexed categories it reads. A shipped category reading one that another
|
as CI does for the shipped set. Any other source loads in `New`, with the indexed
|
||||||
source replaced loads on its own first reach, and fails there if the replacement breaks
|
categories it reads. An indexed category reading one that another source replaced loads
|
||||||
it. The index names every category of its source, so a category file it leaves out never
|
on its own first reach, and fails there if the replacement breaks it. The index names
|
||||||
loads. The shipped set's [`data/.fejkdata.json`](data/.fejkdata.json) shows the shape:
|
every category of its source, so a category file it leaves out never loads.
|
||||||
each category's dot path, its `parent` table, an entry of the same index, and every path
|
|
||||||
`List` advertises below it, `""` for the category itself. A table whose parent column
|
The index maps each category's dot path to an entry: `parent`, the table it links to,
|
||||||
differs from its entry's `parent` fails at first reach.
|
which needs its own entry in the same folder; and `paths`, every path `List` advertises
|
||||||
`//go:embed` of a directory leaves the manifest out; name it in the pattern too.
|
below the category, `""` for the category itself. The shipped set's
|
||||||
|
[`data/.fejkdata.json`](data/.fejkdata.json) is one, so delete it from a copy of `data/`
|
||||||
|
you edit, or keep its index current. A table whose parent column differs from its
|
||||||
|
entry's `parent` fails at first reach, and a table linking under another source's table
|
||||||
|
needs a source without an index. `//go:embed` of a directory leaves the manifest out, and
|
||||||
|
the source then loads whole in `New`; name it in the pattern too.
|
||||||
|
|
||||||
Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
|
Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
|
||||||
`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
|
`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
|
||||||
@@ -342,13 +347,14 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el
|
|||||||
|
|
||||||
`New` refuses data that breaks the grammar, could mean two things or cannot render a valid
|
`New` refuses data that breaks the grammar, could mean two things or cannot render a valid
|
||||||
value, and `NewTemplate` refuses such a template. A category of an indexed source is
|
value, and `NewTemplate` refuses such a template. A category of an indexed source is
|
||||||
checked on the first call reaching it instead. Data that fails to load, in `New` or
|
checked on the first call reaching it instead. Data that fails to load, in `New` or on
|
||||||
there, gives an error matching `ErrLoad`, the same every call. On a loaded generator:
|
that first call, gives an error matching `ErrLoad`. On a loaded generator:
|
||||||
|
|
||||||
- `Fake` fails only for a path that names nothing, could name two things, or reads one
|
- `Fake` fails only for a path that names nothing, could name two things or reads one
|
||||||
draw of a level carrying a `repeat`, with the same error every call, or with `ErrLoad`.
|
draw of a level carrying a `repeat`, or for data that fails to load (`ErrLoad`), with
|
||||||
- `FakeStruct` fails only for a non-struct argument or a type its tags do not describe,
|
the same error every call.
|
||||||
with the same error every call, or with `ErrLoad`.
|
- `FakeStruct` fails only for a non-struct argument, a type its tags do not describe, or
|
||||||
|
data that fails to load (`ErrLoad`), with the same error every call.
|
||||||
- `Template.Fake` cannot fail.
|
- `Template.Fake` cannot fail.
|
||||||
|
|
||||||
A path, wherever a call takes one, may start with one `/`, as a reference does.
|
A path, wherever a call takes one, may start with one `/`, as a reference does.
|
||||||
@@ -848,12 +854,12 @@ The target is about a microsecond per value (goal 13).
|
|||||||
|
|
||||||
A source with an [index](#data), as the shipped set has, loads nothing in `New`: a
|
A source with an [index](#data), as the shipped set has, loads nothing in `New`: a
|
||||||
category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads
|
category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads
|
||||||
it, with every category it reads and its whole table family. Each file is parsed, validated and weight-indexed once, bar a
|
it, with every category it reads and its whole table family. Each file is parsed,
|
||||||
table's name and parent columns, which are mapped on the first `Fake` that selects by
|
validated and weight-indexed once, bar a table's name and parent columns, which are
|
||||||
name or descends through it. A `Fake` call then costs about what its output
|
mapped on the first `Fake` that selects by name or descends through it. A `Fake` call then
|
||||||
costs: an unweighted pick is O(1) whatever the list's length, a weighted one
|
costs about what its output costs: an unweighted pick is O(1) whatever the list's length,
|
||||||
O(log n), and long formats, deep nesting and many tokens add cost in proportion to
|
a weighted one O(log n), and long formats, deep nesting and many tokens add cost in
|
||||||
the output.
|
proportion to the output.
|
||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
@@ -870,7 +876,7 @@ and regenerate seeded fixtures and expected values when you raise that version.
|
|||||||
|---------|-------|-------|
|
|---------|-------|-------|
|
||||||
| Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row |
|
| Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row |
|
||||||
| Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category |
|
| Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category |
|
||||||
| Data format | a fence: a spelling `New` rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin |
|
| Data format | a fence: a spelling a load rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin |
|
||||||
| CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format |
|
| CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format |
|
||||||
| Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option |
|
| Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option |
|
||||||
|
|
||||||
|
|||||||
+6
-7
@@ -156,8 +156,7 @@ Valid while the shipped data stays under 100 MiB, which a gate test holds (`todo
|
|||||||
|
|
||||||
## A source whose manifest carries an index loads each category on first reach; any other loads in `New`, with what it reads
|
## A source whose manifest carries an index loads each category on first reach; any other loads in `New`, with what it reads
|
||||||
|
|
||||||
2026-10-03, a restructure the maintainer approved; widened to any source on 2026-10-09,
|
2026-10-03, a restructure the maintainer approved; widened to any source on 2026-10-09.
|
||||||
and narrowed by the maintainer on 2026-10-10.
|
|
||||||
|
|
||||||
Goal 13: `New` pays nothing for a category a run never reaches. A manifest,
|
Goal 13: `New` pays nothing for a category a run never reaches. A manifest,
|
||||||
`.fejkdata.json` at a source's root, may carry an index: each category's parent table and
|
`.fejkdata.json` at a source's root, may carry an index: each category's parent table and
|
||||||
@@ -171,11 +170,11 @@ at first reach.
|
|||||||
Goal 6: a source without an index loads whole in `New`, with every indexed category it
|
Goal 6: a source without an index loads whole in `New`, with every indexed category it
|
||||||
reads, so each mistake in it is `New`'s error. Goal 6.3: an indexed source from anyone
|
reads, so each mistake in it is `New`'s error. Goal 6.3: an indexed source from anyone
|
||||||
else may fail at first reach; the call then returns the load's error, matching
|
else may fail at first reach; the call then returns the load's error, matching
|
||||||
`ErrLoad`, the same every call. So does an indexed category reading one that another
|
`ErrLoad`, the same every call. An indexed category reading one that another source
|
||||||
source replaced, where the replacement breaks it. The maintainer chose that on
|
replaced fails the same way, at its first reach, when the replacement breaks it. The
|
||||||
2026-10-10 over loading in `New` every indexed category depending on a replaced one:
|
maintainer chose that on 2026-10-10 over loading in `New` every indexed category
|
||||||
readers of three rounds ruled that rule harder to follow (goal 3), and it needed every
|
depending on a replaced one: three rounds of the comprehension panel ruled that rule
|
||||||
entry to list what its templates read.
|
harder to follow (goal 3), and it needed every entry to list what its templates read.
|
||||||
|
|
||||||
A first reach checks only a table's `parent` against its entry, so a table loads with its
|
A first reach checks only a table's `parent` against its entry, so a table loads with its
|
||||||
parent and a failed load puts back whole. Proving the rest of an index is its author's,
|
parent and a failed load puts back whole. Proving the rest of an index is its author's,
|
||||||
|
|||||||
+2
-2
@@ -84,8 +84,8 @@ func (loadError) Is(target error) bool { return target == ErrLoad }
|
|||||||
// namespace segment. It errors on a missing directory, invalid JSON or invalid data; a
|
// namespace segment. It errors on a missing directory, invalid JSON or invalid data; a
|
||||||
// generator with no data at all renders templates that read none. A source whose
|
// generator with no data at all renders templates that read none. A source whose
|
||||||
// .fejkdata.json carries an index, as the shipped set's does, loads each category on the
|
// .fejkdata.json carries an index, as the shipped set's does, loads each category on the
|
||||||
// first call reaching it. Data failing to load, here or there, gives an error matching
|
// first call reaching it. Data failing to load, in New or on that first call, gives an
|
||||||
// [ErrLoad].
|
// error matching [ErrLoad].
|
||||||
func New(opts ...Option) (*Generator, error) {
|
func New(opts ...Option) (*Generator, error) {
|
||||||
c := config{shipped: true}
|
c := config{shipped: true}
|
||||||
for _, opt := range opts {
|
for _, opt := range opts {
|
||||||
|
|||||||
@@ -1,7 +1,9 @@
|
|||||||
package fejkdata
|
package fejkdata
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io/fs"
|
||||||
"slices"
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
@@ -15,8 +17,8 @@ type indexed struct {
|
|||||||
src datafiles.Source
|
src datafiles.Source
|
||||||
}
|
}
|
||||||
|
|
||||||
// placeIndex sets every category index names unloaded in root, in place of what root holds
|
// placeIndex sets every category the index names as unloaded in root, in place of what
|
||||||
// at its path.
|
// root holds at its path.
|
||||||
func placeIndex(root *folder, src datafiles.Source, index map[string]datafiles.IndexEntry) {
|
func placeIndex(root *folder, src datafiles.Source, index map[string]datafiles.IndexEntry) {
|
||||||
for _, p := range sortedNames(index) {
|
for _, p := range sortedNames(index) {
|
||||||
segs := strings.Split(p, ".")
|
segs := strings.Split(p, ".")
|
||||||
@@ -128,9 +130,9 @@ type loaded struct {
|
|||||||
was *indexed
|
was *indexed
|
||||||
}
|
}
|
||||||
|
|
||||||
// load loads each unloaded category in queue and what it needs. A table whose parent
|
// load loads each unloaded category in queue and what it needs. It stops at a table whose
|
||||||
// column disagrees with its entry stops it, so a table loads with its parent and no table
|
// parent column disagrees with its entry. Otherwise a table could load without its parent,
|
||||||
// loaded before links to one putBack removes.
|
// or an earlier table could link to one that putBack then removes.
|
||||||
func (b *batch) load(root *folder, queue []categoryAt) error {
|
func (b *batch) load(root *folder, queue []categoryAt) error {
|
||||||
for ; len(queue) > 0; queue = queue[1:] {
|
for ; len(queue) > 0; queue = queue[1:] {
|
||||||
c := queue[0]
|
c := queue[0]
|
||||||
@@ -139,12 +141,15 @@ func (b *batch) load(root *folder, queue []categoryAt) error {
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if err := x.src.Load(c.dir, c.name, compileInto(func([]string) *folder { return c.in })); err != nil {
|
if err := x.src.Load(c.dir, c.name, compileInto(func([]string) *folder { return c.in })); err != nil {
|
||||||
|
if errors.Is(err, fs.ErrNotExist) {
|
||||||
|
err = fmt.Errorf("%s indexes %s: %w", x.src.ManifestPath(), categoryPath(c.dir, c.name), err)
|
||||||
|
}
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
s := siteIn(c.dir, c.in, c.name)
|
s := siteIn(c.dir, c.in, c.name)
|
||||||
*b = append(*b, loaded{site: s, was: x})
|
*b = append(*b, loaded{site: s, was: x})
|
||||||
if parent := parentOf(s.n); parent != x.entry.Parent {
|
if parent := parentOf(s.n); parent != x.entry.Parent {
|
||||||
return fmt.Errorf("%s: the index entry for %s names parent %q, and its table's parent column is %q", x.src.ManifestPath(), s.path, x.entry.Parent, parent)
|
return fmt.Errorf("%s: the index entry for %s %s, and its table %s", x.src.ManifestPath(), s.path, namesParent(x.entry.Parent), namesParent(parent))
|
||||||
}
|
}
|
||||||
queue = append(queue, needs(root, s)...)
|
queue = append(queue, needs(root, s)...)
|
||||||
}
|
}
|
||||||
@@ -168,6 +173,13 @@ func (b batch) putBack() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func namesParent(parent string) string {
|
||||||
|
if parent == "" {
|
||||||
|
return "names no parent"
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("names parent %q", parent)
|
||||||
|
}
|
||||||
|
|
||||||
func parentOf(n node) string {
|
func parentOf(n node) string {
|
||||||
if t, isTable := n.(*table); isTable {
|
if t, isTable := n.(*table); isTable {
|
||||||
return t.rows.Options().Parent
|
return t.rows.Options().Parent
|
||||||
|
|||||||
@@ -176,7 +176,7 @@ func checkParent(index map[string]IndexEntry, p string) error {
|
|||||||
sibling = p[:dot+1] + parent
|
sibling = p[:dot+1] + parent
|
||||||
}
|
}
|
||||||
if _, ok := index[sibling]; !ok {
|
if _, ok := index[sibling]; !ok {
|
||||||
return fmt.Errorf("parent %q names no entry beside it in this index", parent)
|
return fmt.Errorf("parent %q names no entry in this entry's folder; a table in an indexed source links only to a parent in the same index, so to link under another source's table, drop this index and the source loads in New", parent)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ type node interface{ isNode() }
|
|||||||
// A name stands in children or in unloaded, never both; put and putUnloaded keep it so.
|
// A name stands in children or in unloaded, never both; put and putUnloaded keep it so.
|
||||||
type folder struct {
|
type folder struct {
|
||||||
children map[string]node
|
children map[string]node
|
||||||
unloaded map[string]*indexed // the categories here an index names and no call has reached yet
|
unloaded map[string]*indexed
|
||||||
}
|
}
|
||||||
|
|
||||||
func (*folder) isNode() {}
|
func (*folder) isNode() {}
|
||||||
|
|||||||
@@ -192,7 +192,7 @@ Carries out the decision "The shipped data is Go packages a library imports by c
|
|||||||
|
|
||||||
### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names.
|
### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names.
|
||||||
|
|
||||||
Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. An entry lists every path below its category, its linked tables' too, so `geo.*.region` repeats its whole family; an entry holding only its own paths, `List` composing a family from the entries naming it as `parent`, would shrink the manifest that decoding makes most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10). Record the ruling in `docs/`.
|
Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`.
|
||||||
|
|
||||||
### 16. Add locale categories: company, phone, finance, vehicle, words.
|
### 16. Add locale categories: company, phone, finance, vehicle, words.
|
||||||
|
|
||||||
@@ -425,7 +425,7 @@ A hyphen is always subtraction in a calc, and the shipped data has hyphenated fi
|
|||||||
|
|
||||||
### 194. Let a data author write and check their own `.fejkdata.json` index with the library or the CLI, and fail a load whose index leaves out a category file.
|
### 194. Let a data author write and check their own `.fejkdata.json` index with the library or the CLI, and fail a load whose index leaves out a category file.
|
||||||
|
|
||||||
Goal 6.3 has an author prove their module, and goal 9.1 has fejkdata's data load as anyone's; today only a test of fejkdata's own writes and checks an index (`TestShippedManifestIsCurrent` over `indexEntry`). A category file the index leaves out never loads, silently. The principles audit of item 180's chunk found it.
|
Goal 6.3 has an author prove their module, and goal 9.1 has fejkdata's data load as anyone's; today only fejkdata's own test, `TestShippedManifestIsCurrent` over `indexEntry`, writes and checks an index. A category file the index leaves out never loads, silently.
|
||||||
|
|
||||||
### 53. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.
|
### 53. Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user