Make great documentation goal 1.2, and plan the README's reorganisation and the nested-template gaps #33

Open
lilleman wants to merge 966 commits from worktree-todo-relative-dates-exact-counts into main
10 changed files with 50 additions and 40 deletions
Showing only changes of commit 7f7d260f83 - Show all commits
+1 -1
View File
@@ -88,7 +88,7 @@ In [docs/decisions.md](docs/decisions.md):
- A cell may read a reference, and each row its own
- A path is walked once without drawing before it is walked for real
- A country's postal codes and streets are siblings under its locality
- A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads that module
- A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads its country's geo module
- The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more
- A locale's `address` binds its country's record once and prints it whole
- A postort's kommun comes from its name, its tätort or its codes, never from distance
+3 -2
View File
@@ -130,8 +130,9 @@ replacement, and each removed path, column or flag.
`…/geo/US`. `data.Modules()`, from `github.com/larvit/fejkdata/data`, lists every one,
and the CLI carries them all. `New` loads the modules `WithDataFS(modules...)` names and
the folders `WithDataPath` names, and fails, naming `WithDataFS`, where no such option
is given; `WithDataFS()` loads none. A manifest's `reads` lists the modules its source
reads by default, and a reference naming nothing names them, or says there are none.
is given. A manifest's `reads` lists the modules its source reads by default, and a
reference to a path nothing provides fails, naming those modules or saying there are
none.
- A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each
category on the first call reaching it, as each shipped module does. Data failing to load,
in `New` or on that first call, gives an error matching `ErrLoad`, and the CLI exits 1.
+22 -12
View File
@@ -160,9 +160,9 @@ it to a name and read the name
## Data
The shipped data comes in modules, each a Go package under [`data/`](data) that embeds
its files, so nothing is read from disk: one per locale (`en_US`, `sv_SE`), one per
country's tree of places (`geo/SE`, `geo/US`), and the locale-neutral `misc`. The CLI
The shipped data comes in modules, each a Go package under [`data/`](data): one per
locale (`en_US`, `sv_SE`), one per country's tree of places (`geo/SE`, `geo/US`), and
the locale-neutral `misc`. Each embeds its files, so nothing is read from disk. The CLI
carries every module, and a library loads those it passes to `WithDataFS`
([Library](#library)). A
directory is a namespace: each JSON file is a category named after the file, each
@@ -191,11 +191,12 @@ entry's `parent` fails at first reach, and a table linking under another source'
needs a source without an index. `//go:embed` of a directory leaves the manifest out, and
the source then loads whole in `New`; name it in the pattern too.
A manifest's `reads` lists, by any name its author chooses, every module the source
reads by default, directly or through another; a manifest may carry it without an
index. A shipped module names a Go package: `sv_SE`'s manifest reads
`github.com/larvit/fejkdata/data/geo/SE`. A reference naming nothing fails with the
modules its category's manifest reads, or says it names none.
A manifest's `reads` lists, by any name its author chooses, every module the source's
references read unless another source provides the same paths, directly or through
another module; a manifest may carry it without an index. Nothing loads these modules:
pass them yourself. A shipped module names a Go package: `sv_SE`'s manifest reads
`github.com/larvit/fejkdata/data/geo/SE`. A reference to a path nothing provides fails,
naming those modules, or saying the manifest names none.
Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
@@ -360,9 +361,18 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el
`data.Modules()` is every shipped module, and links all of them into your program. To link
only the ones you read, pass each package's `FS` with the modules its manifest reads:
`WithDataFS(sv_SE.FS, SE.FS)` from `github.com/larvit/fejkdata/data/sv_SE` and
`github.com/larvit/fejkdata/data/geo/SE`, or `en_US.FS` with `US.FS` from
`github.com/larvit/fejkdata/data/geo/US`; `misc` reads none. `New` fails where no `WithDataFS` or
```go
import (
"github.com/larvit/fejkdata"
"github.com/larvit/fejkdata/data/geo/SE"
"github.com/larvit/fejkdata/data/sv_SE"
)
f, err := fejkdata.New(fejkdata.WithDataFS(sv_SE.FS, SE.FS))
```
`en_US` reads `geo/US`; `misc` reads none. `New` fails where no `WithDataFS` or
`WithDataPath` names the data to load.
`New` refuses data that breaks the grammar, could mean two things or cannot render a valid
@@ -1056,7 +1066,7 @@ internal/invariant/ the one phrase every package panics with when an invariant b
internal/jsonvalue/ a decoded JSON value, its numbers kept as written, the one input a template compiles from, and the kind an error names it by
internal/proven/ what a proof knows of a value, and the bounds a calc takes from its operands
internal/rows/ a table's rows: the TSV, the options proved over it, the links between tables, row selection and draws, and the pin set one path or one named pick fixes
data/ the shipped modules, a Go package each that embeds its categories (JSON, and a TSV per table) and its manifest, and Modules, which lists them
data/ the shipped modules, each a Go package embedding its categories (JSON, and a TSV per table) and its manifest; and Modules, which lists them
data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md), and the modules they share
docs/ the decision log, the register research behind the shipped data, the survey of other libraries, panels' scores and rulings on how readable the code is, and what each chunk cost an agent to read
release-tooling/ the release CI publishes from the changelog heading
+4 -3
View File
@@ -8,7 +8,8 @@ import (
// loadSources merges sources in order, the last winning a clash. A source whose manifest
// carries an index leaves its categories unloaded; every other source loads here, with
// the unloaded categories it needs: docs/decisions.md#a-source-whose-manifest-carries-an-index-loads-each-category-on-first-reach-any-other-loads-in-new-with-what-it-reads
// the unloaded categories it needs.
// docs/decisions.md#a-source-whose-manifest-carries-an-index-loads-each-category-on-first-reach-any-other-loads-in-new-with-what-it-reads
func loadSources(sources []datafiles.Source) (folder, error) {
root := folder{children: map[string]node{}}
walked := map[string][]string{}
@@ -24,8 +25,8 @@ func loadSources(sources []datafiles.Source) (folder, error) {
return root, loadReached(&root, sites, nil)
}
// addSource places src's categories in root, and sets in walked, for each category it loads
// here, the modules src reads by default.
// addSource places src's categories in root. For each category it loads now, it records in
// walked the modules src reads by default.
func addSource(root *folder, src datafiles.Source, walked map[string][]string) error {
m, err := src.Manifest()
if err != nil {
+3 -3
View File
@@ -1,9 +1,9 @@
// Package fejkdata generates fake data from recursive JSON templates.
//
// Data lives in JSON, not in Go, and comes in modules: a Go package exporting an fs.FS,
// or a folder. A generator loads the modules you name, such as every one fejkdata ships
// from the package github.com/larvit/fejkdata/data; folders and files become a dot-path
// namespace, then generate values by path:
// or a folder. A generator loads the modules you name, such as data.Modules(), every
// module fejkdata ships, from github.com/larvit/fejkdata/data; folders and files become a
// dot-path namespace, then generate values by path:
//
// f, _ := fejkdata.New(fejkdata.WithDataFS(data.Modules()...), fejkdata.WithSeed(42))
// f.Fake("sv_SE.address") // "Räfsalsvägen 43\n442 42 Kungälv"
+9 -11
View File
@@ -131,11 +131,10 @@ and `regexp.Match`, not two spellings of one result.
## The shipped data is embedded, not discovered
2026-09-02, Lilleman auf Larv.
2026-09-02, Lilleman auf Larv. Serves goal 2: nothing is fetched at run time.
A directory a machine happens to have would make `--seed 42` machine-dependent. Each Go
package under `data/` embeds its own files, and a library imports the packages it reads;
`--data-path` layers over them.
A directory a machine happens to have would make `--seed 42` machine-dependent;
`--data-path` layers over the embedded packages.
## The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0
@@ -143,7 +142,7 @@ package under `data/` embeds its own files, and a library imports the packages i
- One Go package per locale, one per country's `geo/` tree, and one for `misc`, all in the core's Go module. A package registers nothing when imported.
- A bare `New()` loads no data and fails, naming the option to add. `WithoutShippedData` goes.
- A data package's `FS` is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. It is an exported `var FS embed.FS`, the shape `//go:embed` gives; a function returning it would guard only against a program reassigning a dependency of its own. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. Such a function reaches the randomness through an interface the root declares; `internal/drawstate`'s type stays internal.
- A data package's `FS` is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. It is an exported `var FS embed.FS`, the shape `//go:embed` gives; a function returning it would only stop a program reassigning the FS variable of a package it imports. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. Such a function reaches the randomness through an interface the root declares; `internal/drawstate`'s type stays internal.
- A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. A manifest is optional: a `--data-path` folder without one is a module. Nothing loads a default on its own.
- A manifest names no fejkdata version, and `New` checks none: the shipped packages ship in the core's Go module, and a module from anyone else keeps loading across a minor or a patch, as the README's Versioning promises. The maintainer chose so on 2026-10-09.
- In the CLI, a flag of its own says a `--data-path` replaces what it clashes with, so flags still go anywhere on the line. Data authors and hand fixture authors bench it.
@@ -578,15 +577,14 @@ its ZIPs would shape the two trees differently, so both draw inside the pinned l
and an address agrees at that level. A street's own code is the exact pairing to add
when a source carries it.
## A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads that module
## A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads its country's geo module
2026-09-24, Lilleman auf Larv; revised 2026-10-10, when the shipped data became Go packages.
Serves goal 6.2. `sv_SE.FS` loads alone, and `sv_SE.address` then fails at first reach,
naming `github.com/larvit/fejkdata/data/geo/SE`, which `sv_SE`'s manifest reads. `New`
checks no `reads`: a check by module name fails a stand-in providing the same paths (goal
9.2), and a check by path needs every reference, which only loading each indexed category
gives, against goal 13.
`--no-shipped-data -d` takes a locale's folder with its country's, or a set of one's own.
naming `github.com/larvit/fejkdata/data/geo/SE`, which `sv_SE`'s manifest reads.
`New` does not check that each module a manifest reads was given. A check by module name
would refuse a stand-in providing the same paths (goal 9.2). A check by path needs every
reference, and only loading each indexed category finds them, which goal 13 rules out.
## The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more
+2 -2
View File
@@ -62,7 +62,7 @@ func WithDataPath(dir string) Option {
// WithDataFS layers modules, each a data tree held in an [fs.FS], over what is loaded
// before them, in order. A module may be a data package's FS, such as one of
// github.com/larvit/fejkdata/data's Modules, or an embed.FS of your own. WithDataFS()
// with none loads no data.
// loads no data.
func WithDataFS(modules ...fs.FS) Option {
return func(c *config) {
for i, fsys := range modules {
@@ -76,7 +76,7 @@ func WithDataFS(modules ...fs.FS) Option {
}
}
var errNoData = errors.New("fejkdata: New was given no data to load; pass WithDataFS(data.Modules()...), importing github.com/larvit/fejkdata/data, for every module fejkdata ships, WithDataFS or WithDataPath with your own, or WithDataFS() for none")
var errNoData = errors.New("fejkdata: New was given no data to load; pass WithDataFS(data.Modules()...) for every module fejkdata ships (import github.com/larvit/fejkdata/data); WithDataFS or WithDataPath for your own; or WithDataFS() for none")
// ErrLoad marks data that fails to load: in New, or on the first call reaching a category
// of an indexed source.
+2 -2
View File
@@ -18,8 +18,8 @@ const ManifestFile = ".fejkdata.json"
type Manifest struct {
// Index is every category of the source by dot path; nil where the manifest has none.
Index map[string]IndexEntry `json:"index"`
// Reads is every module the source reads by default, directly or through another; the
// error for a reference naming nothing names them, and nothing loads them.
// Reads is every module the source reads by default, directly or through another;
// nothing loads them.
Reads []string `json:"reads"`
}
+2 -2
View File
@@ -91,8 +91,8 @@ func resolveTemplates(ts []templateSite, root map[string]node) error {
return nil
}
// withDefaultModules adds to err, where a reference of a category names nothing, the modules
// the category's source reads by default, or that it names none.
// Where err is a category's reference finding nothing, withDefaultModules adds the modules the
// category's source reads by default, or says it reads none.
func (s templateSite) withDefaultModules(err error) error {
var ne noEntry
if s.category == nil || !errors.As(err, &ne) {
+2 -2
View File
@@ -182,7 +182,7 @@ Pick the ten from a published ranking of languages by total speakers, such as Et
### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names.
Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10, 2864 allocations once each module carried its own). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`.
Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10, 2864 allocations once each module carried its own manifest). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`.
### 16. Add locale categories: company, phone, finance, vehicle, words.
@@ -491,7 +491,7 @@ Reread: `readField`'s note on clearing the pick (`render.go`), `pickKey.under`,
### 195. Parse each builtin argument once, in `checkArgs`, and hand `prep` and `proveNumber` the parsed value.
Applies "Validate data and build DTOs as early as possible" in `~/.claude/principles/technical-principles.md`. Today `checkArgs` (`internal/builtinfunc/builtins.go`) checks an argument, then `prep` and `proveNumber` parse it again through `atoi`/`atof`, which panic, as do `layout.go`, `transform.go` and `calc.go`; two parsers can drift, and a drift panics at compile instead of failing the load.
Applies "Validate data and build DTOs as early as possible" in `~/.claude/principles/technical-principles.md`. Today `checkArgs` (`internal/builtinfunc/builtins.go`) checks an argument, then `prep` and `proveNumber` parse it again through `atoi`/`atof`, which panic on bad input; `layout.go`, `transform.go` and `calc.go` call them too; two parsers can drift, and a drift panics at compile instead of failing the load.
### 112. Check a calc's field operands where its template resolves, beside its name operands, once item 36 reports every mistake at once.