Make great documentation goal 1.2, and plan the README's reorganisation and the nested-template gaps #33

Open
lilleman wants to merge 966 commits from worktree-todo-relative-dates-exact-counts into main
3 changed files with 66 additions and 54 deletions
Showing only changes of commit 143311270c - Show all commits
+63 -51
View File
@@ -7,62 +7,74 @@ replacement, and each removed path, column or flag.
## [Unreleased] ## [Unreleased]
The first release: a Go library and a CLI that render locale-aware fake data from JSON The first release: a Go library and a CLI that render locale-aware fake data from JSON
templates, and the data they ship with. The [README](README.md) documents each part. templates, and the data they ship with. The
[README](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md) documents each part.
Install the CLI with `go install github.com/larvit/fejkdata/cmd/fejkdata@v0.1.0`, or the
library with `go get github.com/larvit/fejkdata@v0.1.0`. Before 1.0.0 a minor may break,
and any release may change what a seed draws, so pin one exact version
([Versioning](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#versioning)).
### Added ### Added
- The CLI, `fejkdata`, renders exactly the template on its stdin and adds nothing: - The CLI, `fejkdata`, renders exactly the template on its stdin and adds nothing: `echo
`echo '{/sv_SE.person} hihi' | fejkdata` prints `Sara Eriksson hihi`. `--seed` makes a '{/sv_SE.person} hihi' | fejkdata` prints e.g. `Sara Eriksson hihi`. `--seed` makes a
run reproducible, `-n` repeats it, `--list` prints every path, and `--data-path` layers run reproducible for one version and data, `-n` repeats it, `--list` prints every
a directory of your own data over the shipped set, or alone with `--no-shipped-data`. path, and `--data-path` layers a directory of your own data over the shipped set, or
`--format json|ndjson|csv|sql` writes a record's columns, one record per row. It exits alone with `--no-shipped-data`. `--format json|ndjson|csv|sql` writes a record's
`0` on success, `1` on a runtime error such as data that fails to load, and `2` on columns, one record per row. It exits `0` on success, `1` on a runtime error such as
misuse. data that fails to load, and `2` on misuse.
- The library, `github.com/larvit/fejkdata`, for Go 1.22 or later. `New` loads the - The library, `github.com/larvit/fejkdata`, for Go 1.22 or later on a 64-bit target.
modules `WithDataFS` names and the folders `WithDataPath` names, and fails, naming `New` loads the modules `WithDataFS` names and the folders `WithDataPath` names, and
`WithDataFS`, where neither is given; `WithSeed` makes its draws reproducible. `Fake` fails, naming `WithDataFS`, where neither is given; `WithSeed` makes its draws
renders a path, `FakeTemplate` and `NewTemplate` a template, `FakeRecord`, reproducible for one version and data. `Fake` renders a path, `FakeTemplate` and
`FakeRecordTemplate` and `NewRecordTemplate` a record, whose columns `JSON`, `NewTemplate` a template, `FakeRecord`, `FakeRecordTemplate` and `NewRecordTemplate` a
`CSVHeader`, `CSVLine` and `SQLInsert` write as the CLI does, and `FakeStruct` fills a record, whose columns `JSON`, `CSVHeader`, `CSVLine` and `SQLInsert` write as the CLI
struct's `fake:"…"` tagged fields. Data that fails to load gives an error matching does, and `FakeStruct` fills a struct's `fake:"…"` tagged fields. `List` gives every
`ErrLoad`, and a record with no fields one matching `ErrNoColumns`. path `Fake` renders, and `IsTemplate` tells an inline template from a path. Data that
- Package `github.com/larvit/fejkdata/cli` is the `fejkdata` command: `cli.Main(modules...)` fails to load gives an error matching `ErrLoad`, and a record with no fields one
runs it over the modules given, so `cli.Main(append(data.Modules(), mydata)...)` builds a matching `ErrNoColumns`.
CLI carrying your own data. That CLI names itself by the file it runs from, and its - Package `github.com/larvit/fejkdata/cli` is the `fejkdata` command:
`--version` prints its own module's version before fejkdata's. `cli.Main(modules...)` runs it over the modules given, so
- The template language ([Data format](README.md#data-format)). A node is a string, a `cli.Main(append(data.Modules(), mydata)...)` builds a CLI carrying your own data.
choice, a template, a table, a number or a boolean. A format string's `{…}` tokens read That CLI names itself by the file it runs from, and its `--version` prints its own
sibling fields, references from the data root or the file's folder module's version before fejkdata's.
(`{/sv_SE.person.last}`, `{.x}`, `{..x}`), builtins and names. Every `{…}` draws - The template language ([Data
afresh, and a name is the one way to keep a pick: `{/sv_SE.person as p}{p.first} format](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#data-format)). A node
{p.last}` is one person. The builtins draw check digits (`luhn`, `mod11`, `ean`), is a string, a choice, a template, a table, a number or a boolean. A format string's
samples (`digits`, `int`, `float`, `uuid`, `ulid`, `iban`, `date`, `time` and more), a `{…}` tokens read sibling fields, references from the data root, the file's folder or
counter (`seq`), arithmetic (`calc`) and transforms (`lowercase`, `uppercase`, the folder above (`{/sv_SE.person.last}`, `{.x}`, `{..x}`), builtins and names. Every
`ascii`). `New` refuses data that breaks the grammar, could mean two things or cannot `{…}` draws afresh, and a name is the one way to keep a pick: `{/sv_SE.person as
render a valid value. p}{p.first} {p.last}` is one person. The builtins draw check digits (`luhn`, `mod11`,
- Records ([Records](README.md#records)): a template seen as columns. A column is a `ean`), samples (`digits`, `int`, `float`, `uuid`, `ulid`, `iban`, `date`, `time` and
string unless it declares a `datatype`, `integer`, `number` or `boolean`, and a `null` more), a counter (`seq`), arithmetic (`calc`) and transforms (`lowercase`,
item draws it as null. `uppercase`, `ascii`). `New` refuses data that breaks the grammar, could mean two
- Tables ([Table](README.md#table)): a category naming a `rows` TSV beside it, with the things or cannot render a valid value.
options `key`, `name`, `weight` and `parent`. A path selects a row by key or name, - Records ([Records](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#records)):
`misc.territory[SE]`, descends to a linked table, `geo.SE.locality[Lund].street`, and a template seen as columns. A column is a string unless it declares a `datatype`,
steps up to the parent row, `{/misc.timezone[Europe/Oslo]..territory.name}`. `integer`, `number` or `boolean`, and a `null` item draws it as null.
- Data sources ([Data](README.md#data)): a directory is a namespace, and each JSON file - Tables ([Table](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#table)): a
in it a category. A `.fejkdata.json` manifest at a source's root may carry a category naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and
`namespace`, the dot path the source mounts under, and an `index`, with which each `parent`. A path selects a row by key or name, `misc.territory[SE]`, descends to a
category loads on the first call reaching it. linked table, `geo.SE.locality[Lund].street`, and steps up to the parent row,
`{/misc.timezone[Europe/Oslo]..territory.name}`.
- Data sources ([Data](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#data)):
a directory is a namespace, and each JSON file in it a category. A `.fejkdata.json`
manifest at a source's root may carry a `namespace`, the dot path the source mounts
under, and an `index`, with which each category loads on the first call reaching it.
- The shipped data, in Go packages that each export `FS` and `Modules()`: - The shipped data, in Go packages that each export `FS` and `Modules()`:
`github.com/larvit/fejkdata/data/en_US` (package `enus`), `…/sv_SE` (`svse`), `github.com/larvit/fejkdata/data/en_US` (package `enus`), `…/sv_SE` (`svse`),
`…/misc`, `…/geo/SE` (`geose`) and `…/geo/US` (`geous`). `data.Modules()`, from `…/misc`, `…/geo/SE` (`geose`) and `…/geo/US` (`geous`). `data.Modules()`, from
`github.com/larvit/fejkdata/data`, lists every one, and the CLI carries them all. `github.com/larvit/fejkdata/data`, lists every one, and the CLI carries them all.
[`DATA-LICENSES.md`](DATA-LICENSES.md) names each source. [`DATA-LICENSES.md`](https://github.com/larvit/fejkdata/blob/v0.1.0/DATA-LICENSES.md)
names each source.
- `sv_SE` and `en_US`: an `address`, a `person` and more per locale. `sex`, - `sv_SE` and `en_US`: an `address`, a `person` and more per locale. `sex`,
`first-name` and `last-name` are tables weighted by bearers, from SCB, the SSA and `first-name` and `last-name` are tables weighted by bearers, from SCB, the SSA and
the Census Bureau. A person's first name and sex agree, and so does an `en_US` the Census Bureau. A person's first name and sex agree, and so does an `en_US`
person's title; `person.prefix` is null where a person has no title. `sv_SE.personnummer` and person's title; `person.prefix` is null where a person has no title.
`sv_SE.samordningsnummer` draw Skatteverket's test series. `en_US.ssn` and `sv_SE.personnummer` and `sv_SE.samordningsnummer` draw Skatteverket's test series.
`en_US.itin` draw the ranges the SSA and the IRS assign, with the columns `area`, `en_US.ssn` and `en_US.itin` draw the ranges the SSA and the IRS assign, with the
`group` and `serial`, and `en_US.phone` draws an exchange NANPA may assign. columns `area`, `group` and `serial`, and `en_US.phone` draws an exchange NANPA may
assign.
- `misc`: IANA's registries of HTTP methods, protocols, ports and root-zone TLDs, the - `misc`: IANA's registries of HTTP methods, protocols, ports and root-zone TLDs, the
HTTP status codes, ISO 3166-1 territories, ISO 4217 currencies with their CLDR HTTP status codes, ISO 3166-1 territories, ISO 4217 currencies with their CLDR
symbols, ISO 639 languages, the syslog severities, mime-db's media types, tzdb's symbols, ISO 639 languages, the syslog severities, mime-db's media types, tzdb's
@@ -73,8 +85,8 @@ templates, and the data they ship with. The [README](README.md) documents each p
`locality`, `postal-code` and `street`, weighted by population and address counts `locality`, `postal-code` and `street`, weighted by population and address counts
and built from SCB, GeoNames, Trafikverket NVDB, the US Census Bureau and Wikidata, and built from SCB, GeoNames, Trafikverket NVDB, the US Census Bureau and Wikidata,
and an `address` record drawing a street and postal code inside one locality, which and an `address` record drawing a street and postal code inside one locality, which
`sv_SE.address` and `en_US.address` print. Every Swedish postort the import can place `sv_SE.address` and `en_US.address` print. Every Swedish postort the import can
ships with all its streets. Of the US incorporated places that hold most of some place ships with all its streets. Of the US incorporated places that hold most of
postal code's land, every state capital but Honolulu, every place of 100,000 or more some postal code's land, every state capital but Honolulu, every place of 100,000 or
and a population-weighted tenth of the rest ship, each with a quarter of its streets, more and a population-weighted tenth of them ship, each with a quarter of its
at least ten. streets, at least ten.
+2 -2
View File
@@ -331,7 +331,7 @@ Swedish region but `abbr` on a US one.
## Library ## Library
```sh ```sh
go get github.com/larvit/fejkdata # Go 1.22+ go get github.com/larvit/fejkdata # Go 1.22+, 64-bit targets
``` ```
```go ```go
@@ -416,7 +416,7 @@ type Address struct {
a column of one record, its tag a path or an inline template — told apart by a column of one record, its tag a path or an inline template — told apart by
`IsTemplate` — and its Go type the column's `IsTemplate` — and its Go type the column's
[datatype](#datatype): a string, bool, integer or float kind, or a pointer to one, [datatype](#datatype): a string, bool, integer or float kind, or a pointer to one,
which a [`null`](#null) item leaves nil. An integer stays within int64 whatever its which a column that can be null requires and a [`null`](#null) item leaves nil. An integer stays within int64 whatever its
kind, and a value the kind cannot hold, such as `{int(0,300)}` in a `uint8`, is refused kind, and a value the kind cannot hold, such as `{int(0,300)}` in a `uint8`, is refused
naming a kind that holds it. The tags are a record's columns, so they share its naming a kind that holds it. The tags are a record's columns, so they share its
[names](#names): `First` tagged `{/sv_SE.person as p}{p.first}` and `Sex` tagged `{p.sex}` [names](#names): `First` tagged `{/sv_SE.person as p}{p.first}` and `Sex` tagged `{p.sex}`
+1 -1
View File
@@ -217,7 +217,7 @@
### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system. ### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.
Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0, which already makes `New` refuse a call that names no data, so `New` breaks once, before anyone depends on it. Mark `WithSeed` required in the Library section's options table. `New`'s tests then stop swapping `randomBytes`. `fejkdata_test.go` swaps it today, and while that test runs, every `New` in the package fails. Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0, which already makes `New` refuse a call that names no data, so `New` breaks once, before anyone depends on it. Mark `WithSeed` required in the Library section's options table. `New`'s tests then stop swapping `randomBytes`. `fejkdata_test.go` swaps it today, and while that test runs, every `New` in the package fails. Rewrite the changelog's library line to say `New` fails naming `WithSeed` where none is given, and its CLI line to say the CLI seeds from the system without `--seed`.
### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go. ### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.