From 7f7d260f83a553acbc88dd93333ff93145bfbb83 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 10 Oct 2026 13:33:26 +0200 Subject: [PATCH] Answer prose round 1: errNoData, the reads paragraph and the changelog entry read in one pass, the Library section shows the import block for one locale, and the decisions state why New checks no reads and which goal the embed serves --- AGENTS.md | 2 +- CHANGELOG.md | 5 +++-- README.md | 34 ++++++++++++++++++++++------------ data.go | 7 ++++--- doc.go | 6 +++--- docs/decisions.md | 20 +++++++++----------- fejkdata.go | 4 ++-- internal/datafiles/manifest.go | 4 ++-- resolve.go | 4 ++-- todo.md | 4 ++-- 10 files changed, 50 insertions(+), 40 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b866da8..dc1af32 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -88,7 +88,7 @@ In [docs/decisions.md](docs/decisions.md): - A cell may read a reference, and each row its own - A path is walked once without drawing before it is walked for real - A country's postal codes and streets are siblings under its locality -- A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads that module +- A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads its country's geo module - The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more - A locale's `address` binds its country's record once and prints it whole - A postort's kommun comes from its name, its tätort or its codes, never from distance diff --git a/CHANGELOG.md b/CHANGELOG.md index 77350f8..ef74664 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -130,8 +130,9 @@ replacement, and each removed path, column or flag. `…/geo/US`. `data.Modules()`, from `github.com/larvit/fejkdata/data`, lists every one, and the CLI carries them all. `New` loads the modules `WithDataFS(modules...)` names and the folders `WithDataPath` names, and fails, naming `WithDataFS`, where no such option - is given; `WithDataFS()` loads none. A manifest's `reads` lists the modules its source - reads by default, and a reference naming nothing names them, or says there are none. + is given. A manifest's `reads` lists the modules its source reads by default, and a + reference to a path nothing provides fails, naming those modules or saying there are + none. - A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each category on the first call reaching it, as each shipped module does. Data failing to load, in `New` or on that first call, gives an error matching `ErrLoad`, and the CLI exits 1. diff --git a/README.md b/README.md index 5c44394..d3594d7 100644 --- a/README.md +++ b/README.md @@ -160,9 +160,9 @@ it to a name and read the name ## Data -The shipped data comes in modules, each a Go package under [`data/`](data) that embeds -its files, so nothing is read from disk: one per locale (`en_US`, `sv_SE`), one per -country's tree of places (`geo/SE`, `geo/US`), and the locale-neutral `misc`. The CLI +The shipped data comes in modules, each a Go package under [`data/`](data): one per +locale (`en_US`, `sv_SE`), one per country's tree of places (`geo/SE`, `geo/US`), and +the locale-neutral `misc`. Each embeds its files, so nothing is read from disk. The CLI carries every module, and a library loads those it passes to `WithDataFS` ([Library](#library)). A directory is a namespace: each JSON file is a category named after the file, each @@ -191,11 +191,12 @@ entry's `parent` fails at first reach, and a table linking under another source' needs a source without an index. `//go:embed` of a directory leaves the manifest out, and the source then loads whole in `New`; name it in the pattern too. -A manifest's `reads` lists, by any name its author chooses, every module the source -reads by default, directly or through another; a manifest may carry it without an -index. A shipped module names a Go package: `sv_SE`'s manifest reads -`github.com/larvit/fejkdata/data/geo/SE`. A reference naming nothing fails with the -modules its category's manifest reads, or says it names none. +A manifest's `reads` lists, by any name its author chooses, every module the source's +references read unless another source provides the same paths, directly or through +another module; a manifest may carry it without an index. Nothing loads these modules: +pass them yourself. A shipped module names a Go package: `sv_SE`'s manifest reads +`github.com/larvit/fejkdata/data/geo/SE`. A reference to a path nothing provides fails, +naming those modules, or saying the manifest names none. Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, @@ -360,9 +361,18 @@ ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, el `data.Modules()` is every shipped module, and links all of them into your program. To link only the ones you read, pass each package's `FS` with the modules its manifest reads: -`WithDataFS(sv_SE.FS, SE.FS)` from `github.com/larvit/fejkdata/data/sv_SE` and -`github.com/larvit/fejkdata/data/geo/SE`, or `en_US.FS` with `US.FS` from -`github.com/larvit/fejkdata/data/geo/US`; `misc` reads none. `New` fails where no `WithDataFS` or + +```go +import ( + "github.com/larvit/fejkdata" + "github.com/larvit/fejkdata/data/geo/SE" + "github.com/larvit/fejkdata/data/sv_SE" +) + +f, err := fejkdata.New(fejkdata.WithDataFS(sv_SE.FS, SE.FS)) +``` + +`en_US` reads `geo/US`; `misc` reads none. `New` fails where no `WithDataFS` or `WithDataPath` names the data to load. `New` refuses data that breaks the grammar, could mean two things or cannot render a valid @@ -1056,7 +1066,7 @@ internal/invariant/ the one phrase every package panics with when an invariant b internal/jsonvalue/ a decoded JSON value, its numbers kept as written, the one input a template compiles from, and the kind an error names it by internal/proven/ what a proof knows of a value, and the bounds a calc takes from its operands internal/rows/ a table's rows: the TSV, the options proved over it, the links between tables, row selection and draws, and the pin set one path or one named pick fixes -data/ the shipped modules, a Go package each that embeds its categories (JSON, and a TSV per table) and its manifest, and Modules, which lists them +data/ the shipped modules, each a Go package embedding its categories (JSON, and a TSV per table) and its manifest; and Modules, which lists them data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md), and the modules they share docs/ the decision log, the register research behind the shipped data, the survey of other libraries, panels' scores and rulings on how readable the code is, and what each chunk cost an agent to read release-tooling/ the release CI publishes from the changelog heading diff --git a/data.go b/data.go index 86ec6bf..e565c23 100644 --- a/data.go +++ b/data.go @@ -8,7 +8,8 @@ import ( // loadSources merges sources in order, the last winning a clash. A source whose manifest // carries an index leaves its categories unloaded; every other source loads here, with -// the unloaded categories it needs: docs/decisions.md#a-source-whose-manifest-carries-an-index-loads-each-category-on-first-reach-any-other-loads-in-new-with-what-it-reads +// the unloaded categories it needs. +// docs/decisions.md#a-source-whose-manifest-carries-an-index-loads-each-category-on-first-reach-any-other-loads-in-new-with-what-it-reads func loadSources(sources []datafiles.Source) (folder, error) { root := folder{children: map[string]node{}} walked := map[string][]string{} @@ -24,8 +25,8 @@ func loadSources(sources []datafiles.Source) (folder, error) { return root, loadReached(&root, sites, nil) } -// addSource places src's categories in root, and sets in walked, for each category it loads -// here, the modules src reads by default. +// addSource places src's categories in root. For each category it loads now, it records in +// walked the modules src reads by default. func addSource(root *folder, src datafiles.Source, walked map[string][]string) error { m, err := src.Manifest() if err != nil { diff --git a/doc.go b/doc.go index a73a496..f964cf3 100644 --- a/doc.go +++ b/doc.go @@ -1,9 +1,9 @@ // Package fejkdata generates fake data from recursive JSON templates. // // Data lives in JSON, not in Go, and comes in modules: a Go package exporting an fs.FS, -// or a folder. A generator loads the modules you name, such as every one fejkdata ships -// from the package github.com/larvit/fejkdata/data; folders and files become a dot-path -// namespace, then generate values by path: +// or a folder. A generator loads the modules you name, such as data.Modules(), every +// module fejkdata ships, from github.com/larvit/fejkdata/data; folders and files become a +// dot-path namespace, then generate values by path: // // f, _ := fejkdata.New(fejkdata.WithDataFS(data.Modules()...), fejkdata.WithSeed(42)) // f.Fake("sv_SE.address") // "Räfsalsvägen 43\n442 42 Kungälv" diff --git a/docs/decisions.md b/docs/decisions.md index c9b5fa6..1213bbc 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -131,11 +131,10 @@ and `regexp.Match`, not two spellings of one result. ## The shipped data is embedded, not discovered -2026-09-02, Lilleman auf Larv. +2026-09-02, Lilleman auf Larv. Serves goal 2: nothing is fetched at run time. -A directory a machine happens to have would make `--seed 42` machine-dependent. Each Go -package under `data/` embeds its own files, and a library imports the packages it reads; -`--data-path` layers over them. +A directory a machine happens to have would make `--seed 42` machine-dependent; +`--data-path` layers over the embedded packages. ## The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0 @@ -143,7 +142,7 @@ package under `data/` embeds its own files, and a library imports the packages i - One Go package per locale, one per country's `geo/` tree, and one for `misc`, all in the core's Go module. A package registers nothing when imported. - A bare `New()` loads no data and fails, naming the option to add. `WithoutShippedData` goes. -- A data package's `FS` is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. It is an exported `var FS embed.FS`, the shape `//go:embed` gives; a function returning it would guard only against a program reassigning a dependency of its own. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. Such a function reaches the randomness through an interface the root declares; `internal/drawstate`'s type stays internal. +- A data package's `FS` is an `fs.FS` passed to `WithDataFS`, as anyone's data is, and `New` takes several. It is an exported `var FS embed.FS`, the shape `//go:embed` gives; a function returning it would only stop a program reassigning the FS variable of a package it imports. A module carrying functions loads through an option of its own, so a data module that starts carrying functions breaks its users. Such a function reaches the randomness through an interface the root declares; `internal/drawstate`'s type stays internal. - A module names, in a manifest, every module it reads by default, directly or through another, so the first error names the whole set to import. A manifest is optional: a `--data-path` folder without one is a module. Nothing loads a default on its own. - A manifest names no fejkdata version, and `New` checks none: the shipped packages ship in the core's Go module, and a module from anyone else keeps loading across a minor or a patch, as the README's Versioning promises. The maintainer chose so on 2026-10-09. - In the CLI, a flag of its own says a `--data-path` replaces what it clashes with, so flags still go anywhere on the line. Data authors and hand fixture authors bench it. @@ -578,15 +577,14 @@ its ZIPs would shape the two trees differently, so both draw inside the pinned l and an address agrees at that level. A street's own code is the exact pairing to add when a source carries it. -## A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads that module +## A locale's `address` reads its country's `geo` tree, and the locale module's manifest reads its country's geo module 2026-09-24, Lilleman auf Larv; revised 2026-10-10, when the shipped data became Go packages. Serves goal 6.2. `sv_SE.FS` loads alone, and `sv_SE.address` then fails at first reach, -naming `github.com/larvit/fejkdata/data/geo/SE`, which `sv_SE`'s manifest reads. `New` -checks no `reads`: a check by module name fails a stand-in providing the same paths (goal -9.2), and a check by path needs every reference, which only loading each indexed category -gives, against goal 13. -`--no-shipped-data -d` takes a locale's folder with its country's, or a set of one's own. +naming `github.com/larvit/fejkdata/data/geo/SE`, which `sv_SE`'s manifest reads. +`New` does not check that each module a manifest reads was given. A check by module name +would refuse a stand-in providing the same paths (goal 9.2). A check by path needs every +reference, and only loading each indexed category finds them, which goal 13 rules out. ## The default embed holds every Swedish postort the import can place and give a street-delivery code and a street, and the US places of 25,000 or more diff --git a/fejkdata.go b/fejkdata.go index 53ce259..4fcb36a 100644 --- a/fejkdata.go +++ b/fejkdata.go @@ -62,7 +62,7 @@ func WithDataPath(dir string) Option { // WithDataFS layers modules, each a data tree held in an [fs.FS], over what is loaded // before them, in order. A module may be a data package's FS, such as one of // github.com/larvit/fejkdata/data's Modules, or an embed.FS of your own. WithDataFS() -// with none loads no data. +// loads no data. func WithDataFS(modules ...fs.FS) Option { return func(c *config) { for i, fsys := range modules { @@ -76,7 +76,7 @@ func WithDataFS(modules ...fs.FS) Option { } } -var errNoData = errors.New("fejkdata: New was given no data to load; pass WithDataFS(data.Modules()...), importing github.com/larvit/fejkdata/data, for every module fejkdata ships, WithDataFS or WithDataPath with your own, or WithDataFS() for none") +var errNoData = errors.New("fejkdata: New was given no data to load; pass WithDataFS(data.Modules()...) for every module fejkdata ships (import github.com/larvit/fejkdata/data); WithDataFS or WithDataPath for your own; or WithDataFS() for none") // ErrLoad marks data that fails to load: in New, or on the first call reaching a category // of an indexed source. diff --git a/internal/datafiles/manifest.go b/internal/datafiles/manifest.go index 77568f8..75df2da 100644 --- a/internal/datafiles/manifest.go +++ b/internal/datafiles/manifest.go @@ -18,8 +18,8 @@ const ManifestFile = ".fejkdata.json" type Manifest struct { // Index is every category of the source by dot path; nil where the manifest has none. Index map[string]IndexEntry `json:"index"` - // Reads is every module the source reads by default, directly or through another; the - // error for a reference naming nothing names them, and nothing loads them. + // Reads is every module the source reads by default, directly or through another; + // nothing loads them. Reads []string `json:"reads"` } diff --git a/resolve.go b/resolve.go index 7b90db4..32144f2 100644 --- a/resolve.go +++ b/resolve.go @@ -91,8 +91,8 @@ func resolveTemplates(ts []templateSite, root map[string]node) error { return nil } -// withDefaultModules adds to err, where a reference of a category names nothing, the modules -// the category's source reads by default, or that it names none. +// Where err is a category's reference finding nothing, withDefaultModules adds the modules the +// category's source reads by default, or says it reads none. func (s templateSite) withDefaultModules(err error) error { var ne noEntry if s.category == nil || !errors.As(err, &ne) { diff --git a/todo.md b/todo.md index 4de0b65..5670120 100644 --- a/todo.md +++ b/todo.md @@ -182,7 +182,7 @@ Pick the ten from a published ranking of languages by total speakers, such as Et ### 185. Seat a panel of the README's audience personas to pull data in through the module API and build a custom CLI, and fix what it names. -Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10, 2864 allocations once each module carried its own). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`. +Needs item 184; 0.1.0 is not cut before it. Carries out the decision "The shipped data is Go packages a library imports by choice, and the CLI carries every one, from 0.1.0". Seats: the Go test author, the bulk fixture author, a custom CLI's builder, and a data author with a `-d` folder and the replace flag, each in a scratch Go module against the checkout or a pre-release tag. Bench the geo packages' names, `data.Modules()`, the error naming the import set, and the `.fejkdata.json` manifest a data author writes by hand. Decoding the manifest is most of `New`'s cost (about 0.55 ms and 2449 allocations on 2026-10-10, 2864 allocations once each module carried its own manifest). An entry lists every path below its category, its linked tables' paths too, so `geo.*.region` repeats its whole family. An entry holding only its own paths, with `List` composing a family from the entries naming it as `parent`, would shrink it. Record the ruling in `docs/`. ### 16. Add locale categories: company, phone, finance, vehicle, words. @@ -491,7 +491,7 @@ Reread: `readField`'s note on clearing the pick (`render.go`), `pickKey.under`, ### 195. Parse each builtin argument once, in `checkArgs`, and hand `prep` and `proveNumber` the parsed value. -Applies "Validate data and build DTOs as early as possible" in `~/.claude/principles/technical-principles.md`. Today `checkArgs` (`internal/builtinfunc/builtins.go`) checks an argument, then `prep` and `proveNumber` parse it again through `atoi`/`atof`, which panic, as do `layout.go`, `transform.go` and `calc.go`; two parsers can drift, and a drift panics at compile instead of failing the load. +Applies "Validate data and build DTOs as early as possible" in `~/.claude/principles/technical-principles.md`. Today `checkArgs` (`internal/builtinfunc/builtins.go`) checks an argument, then `prep` and `proveNumber` parse it again through `atoi`/`atof`, which panic on bad input; `layout.go`, `transform.go` and `calc.go` call them too; two parsers can drift, and a drift panics at compile instead of failing the load. ### 112. Check a calc's field operands where its template resolves, beside its name operands, once item 36 reports every mistake at once.