Make great documentation goal 1.2, and plan the README's reorganisation and the nested-template gaps #33
+63
-51
@@ -7,62 +7,74 @@ replacement, and each removed path, column or flag.
|
|||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
The first release: a Go library and a CLI that render locale-aware fake data from JSON
|
The first release: a Go library and a CLI that render locale-aware fake data from JSON
|
||||||
templates, and the data they ship with. The [README](README.md) documents each part.
|
templates, and the data they ship with. The
|
||||||
|
[README](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md) documents each part.
|
||||||
|
Install the CLI with `go install github.com/larvit/fejkdata/cmd/fejkdata@v0.1.0`, or the
|
||||||
|
library with `go get github.com/larvit/fejkdata@v0.1.0`. Before 1.0.0 a minor may break,
|
||||||
|
and any release may change what a seed draws, so pin one exact version
|
||||||
|
([Versioning](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#versioning)).
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
- The CLI, `fejkdata`, renders exactly the template on its stdin and adds nothing:
|
- The CLI, `fejkdata`, renders exactly the template on its stdin and adds nothing: `echo
|
||||||
`echo '{/sv_SE.person} hihi' | fejkdata` prints `Sara Eriksson hihi`. `--seed` makes a
|
'{/sv_SE.person} hihi' | fejkdata` prints e.g. `Sara Eriksson hihi`. `--seed` makes a
|
||||||
run reproducible, `-n` repeats it, `--list` prints every path, and `--data-path` layers
|
run reproducible for one version and data, `-n` repeats it, `--list` prints every
|
||||||
a directory of your own data over the shipped set, or alone with `--no-shipped-data`.
|
path, and `--data-path` layers a directory of your own data over the shipped set, or
|
||||||
`--format json|ndjson|csv|sql` writes a record's columns, one record per row. It exits
|
alone with `--no-shipped-data`. `--format json|ndjson|csv|sql` writes a record's
|
||||||
`0` on success, `1` on a runtime error such as data that fails to load, and `2` on
|
columns, one record per row. It exits `0` on success, `1` on a runtime error such as
|
||||||
misuse.
|
data that fails to load, and `2` on misuse.
|
||||||
- The library, `github.com/larvit/fejkdata`, for Go 1.22 or later. `New` loads the
|
- The library, `github.com/larvit/fejkdata`, for Go 1.22 or later on a 64-bit target.
|
||||||
modules `WithDataFS` names and the folders `WithDataPath` names, and fails, naming
|
`New` loads the modules `WithDataFS` names and the folders `WithDataPath` names, and
|
||||||
`WithDataFS`, where neither is given; `WithSeed` makes its draws reproducible. `Fake`
|
fails, naming `WithDataFS`, where neither is given; `WithSeed` makes its draws
|
||||||
renders a path, `FakeTemplate` and `NewTemplate` a template, `FakeRecord`,
|
reproducible for one version and data. `Fake` renders a path, `FakeTemplate` and
|
||||||
`FakeRecordTemplate` and `NewRecordTemplate` a record, whose columns `JSON`,
|
`NewTemplate` a template, `FakeRecord`, `FakeRecordTemplate` and `NewRecordTemplate` a
|
||||||
`CSVHeader`, `CSVLine` and `SQLInsert` write as the CLI does, and `FakeStruct` fills a
|
record, whose columns `JSON`, `CSVHeader`, `CSVLine` and `SQLInsert` write as the CLI
|
||||||
struct's `fake:"…"` tagged fields. Data that fails to load gives an error matching
|
does, and `FakeStruct` fills a struct's `fake:"…"` tagged fields. `List` gives every
|
||||||
`ErrLoad`, and a record with no fields one matching `ErrNoColumns`.
|
path `Fake` renders, and `IsTemplate` tells an inline template from a path. Data that
|
||||||
- Package `github.com/larvit/fejkdata/cli` is the `fejkdata` command: `cli.Main(modules...)`
|
fails to load gives an error matching `ErrLoad`, and a record with no fields one
|
||||||
runs it over the modules given, so `cli.Main(append(data.Modules(), mydata)...)` builds a
|
matching `ErrNoColumns`.
|
||||||
CLI carrying your own data. That CLI names itself by the file it runs from, and its
|
- Package `github.com/larvit/fejkdata/cli` is the `fejkdata` command:
|
||||||
`--version` prints its own module's version before fejkdata's.
|
`cli.Main(modules...)` runs it over the modules given, so
|
||||||
- The template language ([Data format](README.md#data-format)). A node is a string, a
|
`cli.Main(append(data.Modules(), mydata)...)` builds a CLI carrying your own data.
|
||||||
choice, a template, a table, a number or a boolean. A format string's `{…}` tokens read
|
That CLI names itself by the file it runs from, and its `--version` prints its own
|
||||||
sibling fields, references from the data root or the file's folder
|
module's version before fejkdata's.
|
||||||
(`{/sv_SE.person.last}`, `{.x}`, `{..x}`), builtins and names. Every `{…}` draws
|
- The template language ([Data
|
||||||
afresh, and a name is the one way to keep a pick: `{/sv_SE.person as p}{p.first}
|
format](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#data-format)). A node
|
||||||
{p.last}` is one person. The builtins draw check digits (`luhn`, `mod11`, `ean`),
|
is a string, a choice, a template, a table, a number or a boolean. A format string's
|
||||||
samples (`digits`, `int`, `float`, `uuid`, `ulid`, `iban`, `date`, `time` and more), a
|
`{…}` tokens read sibling fields, references from the data root, the file's folder or
|
||||||
counter (`seq`), arithmetic (`calc`) and transforms (`lowercase`, `uppercase`,
|
the folder above (`{/sv_SE.person.last}`, `{.x}`, `{..x}`), builtins and names. Every
|
||||||
`ascii`). `New` refuses data that breaks the grammar, could mean two things or cannot
|
`{…}` draws afresh, and a name is the one way to keep a pick: `{/sv_SE.person as
|
||||||
render a valid value.
|
p}{p.first} {p.last}` is one person. The builtins draw check digits (`luhn`, `mod11`,
|
||||||
- Records ([Records](README.md#records)): a template seen as columns. A column is a
|
`ean`), samples (`digits`, `int`, `float`, `uuid`, `ulid`, `iban`, `date`, `time` and
|
||||||
string unless it declares a `datatype`, `integer`, `number` or `boolean`, and a `null`
|
more), a counter (`seq`), arithmetic (`calc`) and transforms (`lowercase`,
|
||||||
item draws it as null.
|
`uppercase`, `ascii`). `New` refuses data that breaks the grammar, could mean two
|
||||||
- Tables ([Table](README.md#table)): a category naming a `rows` TSV beside it, with the
|
things or cannot render a valid value.
|
||||||
options `key`, `name`, `weight` and `parent`. A path selects a row by key or name,
|
- Records ([Records](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#records)):
|
||||||
`misc.territory[SE]`, descends to a linked table, `geo.SE.locality[Lund].street`, and
|
a template seen as columns. A column is a string unless it declares a `datatype`,
|
||||||
steps up to the parent row, `{/misc.timezone[Europe/Oslo]..territory.name}`.
|
`integer`, `number` or `boolean`, and a `null` item draws it as null.
|
||||||
- Data sources ([Data](README.md#data)): a directory is a namespace, and each JSON file
|
- Tables ([Table](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#table)): a
|
||||||
in it a category. A `.fejkdata.json` manifest at a source's root may carry a
|
category naming a `rows` TSV beside it, with the options `key`, `name`, `weight` and
|
||||||
`namespace`, the dot path the source mounts under, and an `index`, with which each
|
`parent`. A path selects a row by key or name, `misc.territory[SE]`, descends to a
|
||||||
category loads on the first call reaching it.
|
linked table, `geo.SE.locality[Lund].street`, and steps up to the parent row,
|
||||||
|
`{/misc.timezone[Europe/Oslo]..territory.name}`.
|
||||||
|
- Data sources ([Data](https://github.com/larvit/fejkdata/blob/v0.1.0/README.md#data)):
|
||||||
|
a directory is a namespace, and each JSON file in it a category. A `.fejkdata.json`
|
||||||
|
manifest at a source's root may carry a `namespace`, the dot path the source mounts
|
||||||
|
under, and an `index`, with which each category loads on the first call reaching it.
|
||||||
- The shipped data, in Go packages that each export `FS` and `Modules()`:
|
- The shipped data, in Go packages that each export `FS` and `Modules()`:
|
||||||
`github.com/larvit/fejkdata/data/en_US` (package `enus`), `…/sv_SE` (`svse`),
|
`github.com/larvit/fejkdata/data/en_US` (package `enus`), `…/sv_SE` (`svse`),
|
||||||
`…/misc`, `…/geo/SE` (`geose`) and `…/geo/US` (`geous`). `data.Modules()`, from
|
`…/misc`, `…/geo/SE` (`geose`) and `…/geo/US` (`geous`). `data.Modules()`, from
|
||||||
`github.com/larvit/fejkdata/data`, lists every one, and the CLI carries them all.
|
`github.com/larvit/fejkdata/data`, lists every one, and the CLI carries them all.
|
||||||
[`DATA-LICENSES.md`](DATA-LICENSES.md) names each source.
|
[`DATA-LICENSES.md`](https://github.com/larvit/fejkdata/blob/v0.1.0/DATA-LICENSES.md)
|
||||||
|
names each source.
|
||||||
- `sv_SE` and `en_US`: an `address`, a `person` and more per locale. `sex`,
|
- `sv_SE` and `en_US`: an `address`, a `person` and more per locale. `sex`,
|
||||||
`first-name` and `last-name` are tables weighted by bearers, from SCB, the SSA and
|
`first-name` and `last-name` are tables weighted by bearers, from SCB, the SSA and
|
||||||
the Census Bureau. A person's first name and sex agree, and so does an `en_US`
|
the Census Bureau. A person's first name and sex agree, and so does an `en_US`
|
||||||
person's title; `person.prefix` is null where a person has no title. `sv_SE.personnummer` and
|
person's title; `person.prefix` is null where a person has no title.
|
||||||
`sv_SE.samordningsnummer` draw Skatteverket's test series. `en_US.ssn` and
|
`sv_SE.personnummer` and `sv_SE.samordningsnummer` draw Skatteverket's test series.
|
||||||
`en_US.itin` draw the ranges the SSA and the IRS assign, with the columns `area`,
|
`en_US.ssn` and `en_US.itin` draw the ranges the SSA and the IRS assign, with the
|
||||||
`group` and `serial`, and `en_US.phone` draws an exchange NANPA may assign.
|
columns `area`, `group` and `serial`, and `en_US.phone` draws an exchange NANPA may
|
||||||
|
assign.
|
||||||
- `misc`: IANA's registries of HTTP methods, protocols, ports and root-zone TLDs, the
|
- `misc`: IANA's registries of HTTP methods, protocols, ports and root-zone TLDs, the
|
||||||
HTTP status codes, ISO 3166-1 territories, ISO 4217 currencies with their CLDR
|
HTTP status codes, ISO 3166-1 territories, ISO 4217 currencies with their CLDR
|
||||||
symbols, ISO 639 languages, the syslog severities, mime-db's media types, tzdb's
|
symbols, ISO 639 languages, the syslog severities, mime-db's media types, tzdb's
|
||||||
@@ -73,8 +85,8 @@ templates, and the data they ship with. The [README](README.md) documents each p
|
|||||||
`locality`, `postal-code` and `street`, weighted by population and address counts
|
`locality`, `postal-code` and `street`, weighted by population and address counts
|
||||||
and built from SCB, GeoNames, Trafikverket NVDB, the US Census Bureau and Wikidata,
|
and built from SCB, GeoNames, Trafikverket NVDB, the US Census Bureau and Wikidata,
|
||||||
and an `address` record drawing a street and postal code inside one locality, which
|
and an `address` record drawing a street and postal code inside one locality, which
|
||||||
`sv_SE.address` and `en_US.address` print. Every Swedish postort the import can place
|
`sv_SE.address` and `en_US.address` print. Every Swedish postort the import can
|
||||||
ships with all its streets. Of the US incorporated places that hold most of some
|
place ships with all its streets. Of the US incorporated places that hold most of
|
||||||
postal code's land, every state capital but Honolulu, every place of 100,000 or more
|
some postal code's land, every state capital but Honolulu, every place of 100,000 or
|
||||||
and a population-weighted tenth of the rest ship, each with a quarter of its streets,
|
more and a population-weighted tenth of them ship, each with a quarter of its
|
||||||
at least ten.
|
streets, at least ten.
|
||||||
|
|||||||
@@ -331,7 +331,7 @@ Swedish region but `abbr` on a US one.
|
|||||||
## Library
|
## Library
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go get github.com/larvit/fejkdata # Go 1.22+
|
go get github.com/larvit/fejkdata # Go 1.22+, 64-bit targets
|
||||||
```
|
```
|
||||||
|
|
||||||
```go
|
```go
|
||||||
@@ -416,7 +416,7 @@ type Address struct {
|
|||||||
a column of one record, its tag a path or an inline template — told apart by
|
a column of one record, its tag a path or an inline template — told apart by
|
||||||
`IsTemplate` — and its Go type the column's
|
`IsTemplate` — and its Go type the column's
|
||||||
[datatype](#datatype): a string, bool, integer or float kind, or a pointer to one,
|
[datatype](#datatype): a string, bool, integer or float kind, or a pointer to one,
|
||||||
which a [`null`](#null) item leaves nil. An integer stays within int64 whatever its
|
which a column that can be null requires and a [`null`](#null) item leaves nil. An integer stays within int64 whatever its
|
||||||
kind, and a value the kind cannot hold, such as `{int(0,300)}` in a `uint8`, is refused
|
kind, and a value the kind cannot hold, such as `{int(0,300)}` in a `uint8`, is refused
|
||||||
naming a kind that holds it. The tags are a record's columns, so they share its
|
naming a kind that holds it. The tags are a record's columns, so they share its
|
||||||
[names](#names): `First` tagged `{/sv_SE.person as p}{p.first}` and `Sex` tagged `{p.sex}`
|
[names](#names): `First` tagged `{/sv_SE.person as p}{p.first}` and `Sex` tagged `{p.sex}`
|
||||||
|
|||||||
@@ -217,7 +217,7 @@
|
|||||||
|
|
||||||
### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.
|
### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.
|
||||||
|
|
||||||
Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0, which already makes `New` refuse a call that names no data, so `New` breaks once, before anyone depends on it. Mark `WithSeed` required in the Library section's options table. `New`'s tests then stop swapping `randomBytes`. `fejkdata_test.go` swaps it today, and while that test runs, every `New` in the package fails.
|
Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0, which already makes `New` refuse a call that names no data, so `New` breaks once, before anyone depends on it. Mark `WithSeed` required in the Library section's options table. `New`'s tests then stop swapping `randomBytes`. `fejkdata_test.go` swaps it today, and while that test runs, every `New` in the package fails. Rewrite the changelog's library line to say `New` fails naming `WithSeed` where none is given, and its CLI line to say the CLI seeds from the system without `--seed`.
|
||||||
|
|
||||||
### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.
|
### 58. Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user