Make great documentation goal 1.2, and plan the README's reorganisation and the nested-template gaps #33

Open
lilleman wants to merge 966 commits from worktree-todo-relative-dates-exact-counts into main
9 changed files with 56 additions and 52 deletions
Showing only changes of commit 53431ab908 - Show all commits
+2 -2
View File
@@ -43,7 +43,7 @@ In [docs/decisions.md](docs/decisions.md):
- A padded JSON argument is rejected, not trimmed - A padded JSON argument is rejected, not trimmed
- `FakeTemplate` and `NewTemplate` both stay - `FakeTemplate` and `NewTemplate` both stay
- The shipped data is embedded, not discovered - The shipped data is embedded, not discovered
- The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` - With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`
- A bare reference draws each time; a reference path is held - A bare reference draws each time; a reference path is held
- Reference sigils follow the filesystem - Reference sigils follow the filesystem
- A change to what exists is a major; a minor only adds - A change to what exists is a major; a minor only adds
@@ -94,7 +94,7 @@ In [docs/decisions.md](docs/decisions.md):
- Tables carrying token cells stay small - Tables carrying token cells stay small
- A path is walked once without drawing before it is walked for real - A path is walked once without drawing before it is walked for real
- A country's postal codes and streets are siblings under its locality - A country's postal codes and streets are siblings under its locality
- A locale's `address` reads its country's `geo` tree, so a locale folder never loads alone - A locale's `address` reads its country's `geo` tree, so a locale folder is no data set on its own
- The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more - The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more
- A locale's `address` restates its country record's format - A locale's `address` restates its country record's format
- A postort's kommun comes from its name, its tätort or its codes, never from distance - A postort's kommun comes from its name, its tätort or its codes, never from distance
+7 -6
View File
@@ -499,7 +499,7 @@ table, `{/misc.currency[SEK].symbol}` on one row and `{/misc.currency[EUR].symbo
on the next, since only one row renders. `New` proves the header, the options and every on the next, since only one row renders. `New` proves the header, the options and every
cell token, and refuses a TSV no category names, a key that is empty or repeats, a cell token, and refuses a TSV no category names, a key that is empty or repeats, a
weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`, weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`,
`"` or `|`, which a selector cannot spell; `New` maps the keys, and a name or parent column `"` or `|`, which a selector cannot spell; the table's load maps the keys, and a name or parent column
is mapped on the first draw that selects by name or descends through the table. A `name` needs a `key`, since a name naming several rows is reported by is mapped on the first draw that selects by name or descends through the table. A `name` needs a `key`, since a name naming several rows is reported by
their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]` their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]`
is settled by the `sex` selected before it and a name repeating inside one parent row is settled by the `sex` selected before it and a name repeating inside one parent row
@@ -813,11 +813,12 @@ token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expan
The target is about a microsecond per value (goal 12). The target is about a microsecond per value (goal 12).
Each file is parsed, validated and weight-indexed once, bar a table's name and parent With only the shipped set, `New` loads nothing: a category loads on the first `Fake`,
columns, which are mapped on the first `Fake` that selects by name or descends through `FakeRecord`, `FakeStruct` or template that reads it, with every category it reads and
it. With the shipped set alone, `New` loads nothing: a category loads on the first call its whole table family. Beside `--data-path`, `WithDataPath` or `WithDataFS`, every
reaching it, with every category it reads and every table linked to it. Beside category loads in `New`. Each file is parsed, validated and weight-indexed once, bar a
`--data-path`, `WithDataPath` or `WithDataFS`, every category loads in `New`. Proving the draw fences adds a pass over the loaded tree per table's name and parent columns, which are mapped on the first `Fake` that selects by
name or descends through it. Proving the draw fences adds a pass over the loaded tree per
fence, and walks what a render reads only where data holds a reference, so a set that fence, and walks what a render reads only where data holds a reference, so a set that
holds none pays for the passes alone. A `Fake` call then costs about what its output holds none pays for the passes alone. A `Fake` call then costs about what its output
costs: an unweighted pick is O(1) whatever the list's length, a weighted one costs: an unweighted pick is O(1) whatever the list's length, a weighted one
+2 -4
View File
@@ -42,7 +42,5 @@ package fejkdata
// `table.parentT`, named by the root that chain ends at: `table.familyRoot`. // `table.parentT`, named by the root that chain ends at: `table.familyRoot`.
// - whole — a read of a table with no selector and no descent, {/city}, landing on // - whole — a read of a table with no selector and no descent, {/city}, landing on
// the `table` itself, so it draws a row apart from every pin: `branches.wholePins`. // the `table` itself, so it draws a row apart from every pin: `branches.wholePins`.
// - fence — a load-time check, so rendering a compiled tree cannot fail: `New` // - fence — a load-time check, run over each category, template or struct as it
// runs them over the sources it loads, the first call reaching a shipped category // loads or compiles, so rendering a compiled tree cannot fail.
// over what that call loads, and `NewTemplate` and `FakeStruct` over what those
// compile.
+22 -17
View File
@@ -80,7 +80,7 @@ tags and any other caller read one.
2026-09-03, Lilleman auf Larv. 2026-09-03, Lilleman auf Larv.
`New` proves the loaded tree acyclic, an inline node is a finite tree of its own, and Each load proves what it loads acyclic, an inline node is a finite tree of its own, and
nothing in the tree can reference it, so no render of it reaches itself. nothing in the tree can reference it, so no render of it reaches itself.
## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing ## An inline template that does not compile is misuse (exit 2), including a reference that resolves to nothing
@@ -116,19 +116,23 @@ still lives in `data/` as JSON; `--data-path` layers over it.
`todo.md` item 77 revises it: a library imports the shipped data as Go modules. `todo.md` item 77 revises it: a library imports the shipped data as Go modules.
## The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New` ## With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`
2026-10-03, a restructure the maintainer approved. Goal 12: a 2026-10-03, a restructure the maintainer approved.
`New` pays nothing for the categories a run never reaches, which matters once the shipped
set grows to hundreds of MB. A category loads with every category it references and its Goal 12: `New` pays nothing for a category a run never reaches, which matters once the
whole table family, and is bound as a whole load binds, so it renders as it would from shipped set grows to hundreds of MB. A category loads with every category it references
`New`. `List` reads `shippedindex.go`, which `go generate` writes from `data/`, and and its whole table family, and binds as a whole load does, so it renders the same as
`TestShippedIndexIsCurrent` fails while it is stale. CI loads the whole shipped set, and after a whole load. `List` reads `shippedindex.go`, which `go generate` writes from
loads each shipped category alone into a fresh `New`, so none fails at first reach. Goal 6: beside a `--data-path` or a `data/`; `TestShippedIndexIsCurrent` fails while the index is stale. CI loads the whole
`WithDataFS`, every category loads in `New`, so each mistake in a source the user wrote is shipped set, and each shipped category alone into a fresh `New`, so none fails at first
`New`'s error, though it parses the whole shipped set, against goal 12, until reach.
`todo.md` item 79 loads only what the user's sources reach. Valid while CI proves the shipped set
whole and each category alone. Goal 6: beside a `--data-path`, `WithDataPath` or `WithDataFS`, every category loads in
`New`, so each mistake in the user's data is `New`'s error. That parses the whole shipped
set, against goal 12, until `todo.md` item 79 ends it.
Valid while CI proves the shipped set whole and each category alone.
## A bare reference draws each time; a reference path is held ## A bare reference draws each time; a reference path is held
@@ -171,7 +175,8 @@ release instead. Before `v1.0.0` a minor carries what a major would.
Any edit to a category shifts its stream and everything drawn after it, so a promise Any edit to a category shifts its stream and everything drawn after it, so a promise
across versions would freeze every shipped list; a fixture is re-pinned on a bump, as across versions would freeze every shipped list; a fixture is re-pinned on a bump, as
this repo's own are. Goal 9 reads wider than this until `todo.md` item 85 rewords it. this repo's own are. This works against goal 9's "always" until `todo.md` item 85
rewords the goal.
## An error is a contract by what it names, not its bytes ## An error is a contract by what it names, not its bytes
@@ -654,7 +659,7 @@ its ZIPs would shape the two trees differently, so both draw inside the pinned l
and an address agrees at that level. A street's own code is the exact pairing to add and an address agrees at that level. A street's own code is the exact pairing to add
when a source carries it. when a source carries it.
## A locale's `address` reads its country's `geo` tree, so a locale folder never loads alone ## A locale's `address` reads its country's `geo` tree, so a locale folder is no data set on its own
2026-09-24, Lilleman auf Larv. 2026-09-24, Lilleman auf Larv.
@@ -911,8 +916,8 @@ mode field, leaked the check's leaves and errors with the draw's pins, arm and m
since Go tracks a struct's fields as one, and a repeat of a reference path rose from 66 since Go tracks a struct's fields as one, and a repeat of a reference path rose from 66
to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node, to 106 allocations. `drawSteps` takes its state as parameters and a step holds no node,
so no benchmark gained an allocation. Valid while Go's escape analysis tracks so no benchmark gained an allocation. Valid while Go's escape analysis tracks
a struct's fields as one. The two loops have drifted, against DRY, until `todo.md` item 80 a struct's fields as one. Two loops over one rule work against DRY, and they already
walks both through one per-step function. disagree on a repeat level; `todo.md` item 80 walks both through one per-step function.
## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read ## A name lives in the category binding it, or in the repeat binding it, and is drawn on its first read
+6 -6
View File
@@ -22,11 +22,11 @@ var _ [^uint(0)>>63 - 1]struct{} // docs/decisions.md#64-bit-targets-only
// ErrNoData is returned by New when no source is loaded at all. // ErrNoData is returned by New when no source is loaded at all.
var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS") var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithDataPath or WithDataFS")
// Generator generates fake data from a loaded namespace tree. Create one with [New]. // Generator generates fake data from a namespace tree. Create one with [New].
// It is safe for concurrent use; a seeded sequence is reproducible only when drawn // It is safe for concurrent use; a seeded sequence is reproducible only when drawn
// from one goroutine. // from one goroutine.
type Generator struct { type Generator struct {
// mu guards rand, records, structs and root, which loads a shipped category on the // mu guards rand, records, structs and root, which gains a shipped category on the
// first call reaching it. // first call reaching it.
mu sync.Mutex mu sync.Mutex
rand *generatorState rand *generatorState
@@ -86,8 +86,8 @@ func WithoutShippedData() Option {
// in order with the last winning a name clash. Each JSON file becomes a category // in order with the last winning a name clash. Each JSON file becomes a category
// named after the file (address.json -> "address") and each subdirectory a // named after the file (address.json -> "address") and each subdirectory a
// namespace segment. It errors on a missing directory, invalid JSON, invalid data, // namespace segment. It errors on a missing directory, invalid JSON, invalid data,
// or no data at all. The shipped set alone loads each category on the first call // or no data at all. With only the shipped data, a category loads on the first call
// reaching it; beside WithDataPath or WithDataFS, every category loads here. // reaching it; with WithDataPath or WithDataFS, every category loads here.
func New(opts ...Option) (*Generator, error) { func New(opts ...Option) (*Generator, error) {
c := config{shipped: true} c := config{shipped: true}
for _, opt := range opts { for _, opt := range opts {
@@ -104,8 +104,8 @@ func New(opts ...Option) (*Generator, error) {
return &Generator{rand: rng, root: root}, nil return &Generator{rand: rng, root: root}, nil
} }
// load is the tree New starts from: beside a source of the caller's, every category loads // load is the tree New starts from.
// here, so each mistake in that source is New's error. // docs/decisions.md#with-only-the-shipped-set-a-category-loads-on-the-first-call-reaching-it-beside-a---data-path-every-category-loads-in-new
func (c config) load() (folder, error) { func (c config) load() (folder, error) {
if c.shipped && len(c.sources) == 0 { if c.shipped && len(c.sources) == 0 {
return unloadedTree(), nil return unloadedTree(), nil
+3 -3
View File
@@ -148,8 +148,7 @@ func inputValue(input string) (any, error) {
return raw, nil return raw, nil
} }
// bindInline loads the shipped categories n reads, then binds n against root; typedByGo is // bindInline loads the shipped categories n reads, then binds n against root.
// binding's.
func bindInline(root *folder, n node, label string, typedByGo bool) error { func bindInline(root *folder, n node, label string, typedByGo bool) error {
scope := inlineScope(n, label) scope := inlineScope(n, label)
if err := refuseFolderRefs(scope); err != nil { if err := refuseFolderRefs(scope); err != nil {
@@ -169,7 +168,8 @@ func bindInline(root *folder, n node, label string, typedByGo bool) error {
}.bind() }.bind()
} }
// refuseFolderRefs refuses a reference an inline node writes from a folder, which it has none of. // refuseFolderRefs refuses each reference in scope not written from the root, {/x}: an
// inline node sits in no folder.
func refuseFolderRefs(scope nodeScope) error { func refuseFolderRefs(scope nodeScope) error {
return scope(func(path string, m node) error { return scope(func(path string, m node) error {
t, ok := m.(*template) t, ok := m.(*template)
+9 -9
View File
@@ -15,8 +15,8 @@ var shippedFS embed.FS
var shippedSource = dataSource{fsys: shippedFS, baseDir: "data"} var shippedSource = dataSource{fsys: shippedFS, baseDir: "data"}
// shippedEntry is a shipped category's entry in shippedindex.go: the table beside it that // shippedEntry is a shipped category's entry in shippedindex.go: its parent table, "" for
// it links to, "" for none, and the paths List advertises below it. After changing it, empty // none, and the paths List advertises below it. After changing it, empty
// shippedIndex's literal by hand so the package builds, then run generate. // shippedIndex's literal by hand so the package builds, then run generate.
type shippedEntry struct { type shippedEntry struct {
parent string parent string
@@ -97,9 +97,10 @@ func unloadedReads(root *folder, dir []string, scope nodeScope) []unloadedCatego
return out return out
} }
// loadShipped loads the shipped categories wanted, with every category they read and every // loadShipped loads the shipped categories wanted, with every category they read and the
// table linked to one, then binds what it loaded, as New binds a whole load. // family of each, then binds what it loaded as a whole load binds.
// TestEveryShippedCategoryLoadsAlone proves each closure, so a failure is the index's. // TestEveryShippedCategoryLoadsAlone reaches each category, so a failure here means a stale
// index.
func loadShipped(root *folder, wanted []unloadedCategory) { func loadShipped(root *folder, wanted []unloadedCategory) {
var sites []categorySite var sites []categorySite
for queue := wanted; len(queue) > 0; queue = queue[1:] { for queue := wanted; len(queue) > 0; queue = queue[1:] {
@@ -124,8 +125,8 @@ func loadShipped(root *folder, wanted []unloadedCategory) {
} }
} }
// load parses and compiles the category, and moves it from its folder's unloaded // load parses and compiles the category, and moves it from its folder's unloaded map to
// categories to its children. // the folder's children.
func (u unloadedCategory) load() (categorySite, error) { func (u unloadedCategory) load() (categorySite, error) {
dir := path.Join(append([]string{shippedSource.baseDir}, u.dir...)...) dir := path.Join(append([]string{shippedSource.baseDir}, u.dir...)...)
entries, err := fs.ReadDir(shippedSource.fsys, dir) entries, err := fs.ReadDir(shippedSource.fsys, dir)
@@ -140,8 +141,7 @@ func (u unloadedCategory) load() (categorySite, error) {
return siteIn(u.dir, u.in, u.name), nil return siteIn(u.dir, u.in, u.name), nil
} }
// linkedTables is the tables beside u that link to it or that it links to: its parent, and // linkedTables is u's parent beside it, and every table beside it whose parent u is.
// every table whose parent it is.
func (u unloadedCategory) linkedTables(e shippedEntry) []unloadedCategory { func (u unloadedCategory) linkedTables(e shippedEntry) []unloadedCategory {
var out []unloadedCategory var out []unloadedCategory
if e.parent != "" { if e.parent != "" {
+1 -1
View File
@@ -39,7 +39,7 @@ func loadedCategories(f *Generator) []string {
return out return out
} }
// REGENERATE=1, which go generate sets, rewrites the index. // REGENERATE=1 rewrites the index.
func TestShippedIndexIsCurrent(t *testing.T) { func TestShippedIndexIsCurrent(t *testing.T) {
got, err := shippedIndexSource(newShippedWhole(t)) got, err := shippedIndexSource(newShippedWhole(t))
if err != nil { if err != nil {
+4 -4
View File
@@ -67,7 +67,7 @@
| 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 | | 12 | 0.1.0 | defect | **Strip the whitespace `data-import/territory.py` copies from its register.** | 1 | 1 | 5 | 7 | 4 | 20.6 |
| 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 | | 5 | 0.1.0 | | **Run the nine-seat comprehension panel after the draw restructure, and file what it names.** | 1 | 3 | 3 | 9 | 3 | 20.4 |
| 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 | | 71 | 0.1.0 | | **Accept a name as a `calc` operand, so `{calc(n * 2)}` computes from the pick `{n}` prints.** | 3 | 4 | 5 | 7 | 5.3 | 17.2 |
| 79 | 0.1.0 | decision | **Beside a `--data-path`, load only the shipped categories the user's sources override or reach, as the shipped set alone does.** | 6 | 6 | 7 | 8 | 12 | 13.7 | | 79 | 0.1.0 | decision | **Beside a `--data-path`, load in `New` only the user's categories, the shipped ones they override and the shipped ones they read; load the rest on first reach.** | 6 | 6 | 7 | 8 | 12 | 13.7 |
| 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 | | 58 | 0.2.0 | | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 |
| 81 | 0.2.0 | question | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 4, 6 | 20.6 | | 81 | 0.2.0 | question | **Refuse at load every `calc` divisor not proven nonzero, in a string column too.** | 4 | 4 | 6 | 8 | 4, 6 | 20.6 |
| 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 | | 36 | 0.2.0 | | **Report every mistake a load finds in one error, as the decision "A load reports every mistake at once" states.** | 5 | 6 | 7 | 8 | 6 | 19.5 |
@@ -136,7 +136,7 @@
### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI. ### 77. Split the shipped data into Go modules per locale and country, which a library imports by choice, and embed every one in the CLI.
The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where since 2026-10-03. The CLI binary carries every module and may grow to hundreds of MB; a library imports only what it uses, as it imports any dependency, and nothing is fetched at run time. Decide the module boundaries (a locale, a country's `geo/` tree, `misc`) and how an imported module registers with `New`. Goal 2 says what ships where.
### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population. ### 78. Ship 10% of every place's streets, at least 10, and 10% of all US places drawn by population, each place weighted by population.
@@ -255,9 +255,9 @@ Needs item 4.
`checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does. `checkCalc` and the value proof read an operand as a sibling field, so `{calc(n * 2)}` is refused with `no field "n"` today. Item 4 stops an operand holding its field for the expansion, after which a name is the one way to show the operand a calc computes from, as the README's Computation example does.
### 79. Beside a `--data-path`, load only the shipped categories the user's sources override or reach, as the shipped set alone does. ### 79. Beside a `--data-path`, load in `New` only the user's categories, the shipped ones they override and the shipped ones they read; load the rest on first reach.
The decision "The shipped set alone loads a category on the first call reaching it; beside a `--data-path` every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, and the shipped ones whose reads reach those, so the index would carry each category's reads. The decision "With only the shipped set, a category loads on the first call reaching it; beside a `--data-path`, every category loads in `New`" parses the whole shipped set on every `--data-path` run, about 45 ms today and seconds once items 61 and 78 land. Goal 6 needs only the user's categories, the shipped ones they override, the shipped ones they read, and the shipped ones whose reads reach an overridden category, so the index would carry each category's reads.
### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. ### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.