2 changed files with 69 additions and 144 deletions
@@ -6,145 +6,75 @@ replacement, and each removed path, column or flag.
 ## [Unreleased]
 The first release: a Go library and a CLI that render locale-aware fake data from JSON
 templates, and the data they ship with. The [README](README.md) documents each part.
 ### Added
- Step up: `..` after a row steps up to the row its link column names, so
+- The CLI, `fejkdata`, renders exactly the template on its stdin and adds nothing:
-  `{/city[Oslo]..country.name}` renders `Norway`, and `.` after it steps down again,
+  `echo '{/sv_SE.person} hihi' | fejkdata` prints `Sara Eriksson hihi`. `--seed` makes a
-  drawing afresh. `New` refuses a `..` naming anything but the parent table, and a
+  run reproducible, `-n` repeats it, `--list` prints every path, and `--data-path` layers
-  selector after a `..`.
+  a directory of your own data over the shipped set, or alone with `--no-shipped-data`.
- A path, given to `Fake` or referenced in a template, may not read into a
+  `--format json|ndjson|csv|sql` writes a record's columns, one record per row. It exits
-  level carrying a `repeat`, and `List` advertises no such path: where `x` carries
+  `0` on success, `1` on a runtime error such as data that fails to load, and `2` on
-  `"repeat": 3`, `Fake("x.a")` fails naming the level `x`, and `Fake("x")` renders it
+  misuse.
-  whole.
+- The library, `github.com/larvit/fejkdata`, for Go 1.22 or later. `New` loads the
- Every `{…}` draws afresh, and a name is the one way to keep a pick: `{p.a} {p.b}`,
+  modules `WithDataFS` names and the folders `WithDataPath` names, and fails, naming
-  `{w} {uppercase(w)}` and `{/person.first} {/person.last}` are two draws each, which
+  `WithDataFS`, where neither is given; `WithSeed` makes its draws reproducible. `Fake`
-  `{p as q}{q.a} {q.b}` and `{/person as n}{n.first} {n.last}` make one.
+  renders a path, `FakeTemplate` and `NewTemplate` a template, `FakeRecord`,
-  `{/misc.territory as t}` binds one row of `misc.territory` to `t` and prints nothing;
+  `FakeRecordTemplate` and `NewRecordTemplate` a record, whose columns `JSON`,
-  `{t}`, `{t.capital}` and `{lowercase(t.name)}` read that one row. `{x as n}` binds a
+  `CSVHeader`, `CSVLine` and `SQLInsert` write as the CLI does, and `FakeStruct` fills a
-  reference, a sibling field or a path into one; a name lives in the category binding it,
+  struct's `fake:"…"` tagged fields. Data that fails to load gives an error matching
-  and picks again on every iteration of a `repeat`, or every draw of a choice item,
+  `ErrLoad`, and a record with no fields one matching `ErrNoColumns`.
-  binding it. A struct's tags share its
+- Package `github.com/larvit/fejkdata/cli` is the `fejkdata` command: `cli.Main(modules...)`
-  names, `{/sv_SE.person as p}{p.first}` beside `{p.sex}`. A name is a `{calc()}`
+  runs it over the modules given, so `cli.Main(append(data.Modules(), mydata)...)` builds a
-  operand: `{/misc.coordinate.lat as lat}{calc(lat * 60)}`. What `New` refuses is listed
+  CLI carrying your own data. That CLI names itself by the file it runs from, and its
-  under [Names](README.md#names).
+  `--version` prints its own module's version before fejkdata's.
- First release: the CLI, the library and the shipped data set.
+- The template language ([Data format](README.md#data-format)). A node is a string, a
- The CLI renders exactly the template on its stdin, `echo '{/sv_SE.person} hihi' |
+  choice, a template, a table, a number or a boolean. A format string's `{…}` tokens read
-  fejkdata`, and adds nothing under `--format text`: `echo`'s newline ends the render, and `--separator`
+  sibling fields, references from the data root or the file's folder
-  defaults to empty. Whitespace around a JSON object, array or string template is
+  (`{/sv_SE.person.last}`, `{.x}`, `{..x}`), builtins and names. Every `{…}` draws
-  dropped, and a bare number or boolean is a format string, printed as written. A lone reference,
+  afresh, and a name is the one way to keep a pick: `{/sv_SE.person as p}{p.first}
-  `{/users}`, is the record `users` under `--format`. An argument other than a flag, or
+  {p.last}` is one person. The builtins draw check digits (`luhn`, `mod11`, `ean`),
-  nothing piped in, exits 2; an empty or blank stdin is a format string, and renders as
+  samples (`digits`, `int`, `float`, `uuid`, `ulid`, `iban`, `date`, `time` and more), a
-  sent.
+  counter (`seq`), arithmetic (`calc`) and transforms (`lowercase`, `uppercase`,
- A template or data file loads whatever is well-formed and means one thing, even where
+  `ascii`). `New` refuses data that breaks the grammar, could mean two things or cannot
-  part of it does nothing: a default written out; a one-item choice; an object holding only
+  render a valid value.
-  a format; a constant sample such as `{int(5,5)}`; a `weight`, `separator` or `datatype`
+- Records ([Records](README.md#records)): a template seen as columns. A column is a
-  where it does nothing; a name nothing reads; a repeated item, which counts as written; a
+  string unless it declares a `datatype`, `integer`, `number` or `boolean`, and a `null`
-  weight of `0`, which is never drawn; a `repeat` or count of `0`, which renders nothing;
+  item draws it as null.
-  `+5`, `05` and `-0`; and a reference into the category it sits in. A rows file no
+- Tables ([Table](README.md#table)): a category naming a `rows` TSV beside it, with the
-  category names is ignored. `New(WithDataFS())` loads no data, and renders templates that
+  options `key`, `name`, `weight` and `parent`. A path selects a row by key or name,
-  read none. The CLI's `-n` takes `0`; `--list` ignores `--repeat`, `--separator`, `--format`
+  `misc.territory[SE]`, descends to a linked table, `geo.SE.locality[Lund].street`, and
-  and `--table`, and `--table` is ignored without `--format sql`.
+  steps up to the parent row, `{/misc.timezone[Europe/Oslo]..territory.name}`.
- Table categories: a category JSON naming a `rows` TSV beside it, with the options
+- Data sources ([Data](README.md#data)): a directory is a namespace, and each JSON file
-  `key`, `name`, `weight` and `parent`; a path selects a row by key or name,
+  in it a category. A `.fejkdata.json` manifest at a source's root may carry a
-  `misc.territory[SE]`, and descends to a linked table by name, one path drawing each
+  `namespace`, the dot path the source mounts under, and an `index`, with which each
-  row inside the row above it. `rows` is an option, so no template may carry a field
+  category loads on the first call reaching it.
-  of that name. A `name` without a `key` resolves inside the table's `parent`. Refused
+- The shipped data, in Go packages that each export `FS` and `Modules()`:
-  at `New`: a `name` without a `key` or a `parent`, a name repeating inside one parent
+  `github.com/larvit/fejkdata/data/en_US` (package `enus`), `…/sv_SE` (`svse`),
-  row, a name spelling another row's key, and a table named like a column of any table
+  `…/misc`, `…/geo/SE` (`geose`) and `…/geo/US` (`geous`). `data.Modules()`, from
-  above it.
+  `github.com/larvit/fejkdata/data`, lists every one, and the CLI carries them all.
- `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and
+  [`DATA-LICENSES.md`](DATA-LICENSES.md) names each source.
-  `misc.mimetype` are tables. `misc.territory` is every ISO 3166-1 territory that has
+  - `sv_SE` and `en_US`: an `address`, a `person` and more per locale. `sex`,
-  a capital, a currency and a TLD, with the columns `calling-code`, `capital`,
+    `first-name` and `last-name` are tables weighted by bearers, from SCB, the SSA and
-  `country`, `currency`, `flag`, `languages`, `numeric` and `tld` added, `country`
+    the Census Bureau. A person's first name and sex agree, and so does an `en_US`
-  holding the alpha-2 code of the sovereign state it belongs to and its own where it
+    person's title; `person.prefix` is null where a person has no title. `sv_SE.personnummer` and
-  is one; `misc.currency` the
+    `sv_SE.samordningsnummer` draw Skatteverket's test series. `en_US.ssn` and
-  current ISO 4217 currencies with a minor unit, with `decimals` and `numeric`
+    `en_US.itin` draw the ranges the SSA and the IRS assign, with the columns `area`,
-  added, and its symbols from CLDR; `misc.language` every ISO 639-2 entry carrying a
+    `group` and `serial`, and `en_US.phone` draws an exchange NANPA may assign.
-  639-1 code, with a `code3` column holding its 639-2/T code; `misc.httpstatus` each
+  - `misc`: IANA's registries of HTTP methods, protocols, ports and root-zone TLDs, the
-  code the IANA registry lists with a plain reason phrase; and `misc.mimetype` each
+    HTTP status codes, ISO 3166-1 territories, ISO 4217 currencies with their CLDR
-  type mime-db records as IANA-registered and gives a filename extension.
+    symbols, ISO 639 languages, the syslog severities, mime-db's media types, tzdb's
-  `DATA-LICENSES.md` lists each source.
+    zones weighted by the people living in each, user agents and cars, all as tables;
- `misc.timezone`, `misc.car` and `misc.useragent` are tables too. `misc.timezone`
+    and coordinates, credit card numbers, RFC 3339 instants, emoji, MAC addresses,
-  is every zone tzdb `zone.tab` gives a shipped territory, with `offset` holding the
+    ObjectIds and UUIDs.
  zone's standard UTC offset and `territory` linking to `misc.territory`, so
  `misc.territory[SE].timezone` draws `Europe/Stockholm`; zones of a territory
  `misc.territory` does not ship, Antarctica's among them, and the constant `UTC`
  are gone; spell that one as the text `"UTC"`. A `weight` column carries the
  population GeoNames records in each zone, so a draw favours the zones people live
  in. `misc.useragent` is the top-user-agents desktop and mobile lists with `browser`,
  `device` and `os` columns. `misc.car` keeps its makes and models in a `make` and a
  `model` column.
 - `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries as tables.
  `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH, carrying
  `safe` and `idempotent` as `true` or `false`; `misc.protocol` the keyword, its
  `number` and its `name`, selectable by either; `misc.port` renders a TCP port number,
  with the IANA `service` name beside it, keeping the first service the registry
  describes for a port so a number selects one row.
 - `misc.tld` is the IANA root zone as a table: every delegated TLD keyed with its
  leading dot, `.se` and `.xn--p1ai`, the register's `type` beside it, and a `unicode`
  column holding the form the register displays, so `misc.tld[.рф]` selects the row
  `.xn--p1ai` renders. A delegation the register marks "Not assigned" does not ship.
 - `misc.loglevel` is the eight syslog severities: the `keyword` a configuration writes,
  `err` and `info`, keyed by the numerical `code` a PRI encodes and selectable by
  either, with the `severity` RFC 5424 spells beside it.
  - `geo.SE` and `geo.US`: five linked tables per country, `region`, `municipality`,
    `locality`, `postal-code` and `street`, weighted by population and address counts
-  and built from SCB, GeoNames, Trafikverket NVDB, the US Census Bureau and Wikidata, and an
+    and built from SCB, GeoNames, Trafikverket NVDB, the US Census Bureau and Wikidata,
-  `address` record binding one locality and drawing its street and postal code inside
+    and an `address` record drawing a street and postal code inside one locality, which
-  it. `sv_SE.address` and `en_US.address` bind those records once, so
+    `sv_SE.address` and `en_US.address` print. Every Swedish postort the import can place
-  `en_US.address.street` no longer carries `name` and `suffix`, and a locale's `address`
+    ships with all its streets. Of the US incorporated places that hold most of some
-  reads its country's `geo` module. Every Swedish postort the import can place and give
+    postal code's land, every state capital but Honolulu, every place of 100,000 or more
-  a street ships with all its streets. Of the US incorporated places that hold most of
+    and a population-weighted tenth of the rest ship, each with a quarter of its streets,
-  some postal code's land, a tenth ships, drawn by population down to villages of a
+    at least ten.
  hundred people, and so do every state capital but Honolulu and every place of 100,000
  or more. A smaller city may be missing. Each place carries a quarter of its streets, at
  least ten.
 - `{date(from,to,'layout')}` and `{time('layout')}`: a second between two days, or
  within one, in a single-quoted Go layout, drawn in UTC; `from` may equal `to`.
  `sv_SE.date`, `en_US.date`, `sv_SE.time` and `en_US.time` render through them, so
  `date.year`, `date.month`, `date.day`, `time.hour`, `time.minute`, `time.minute.t`,
  `time.sec` and `time.ampm` are no longer paths — a part of a date is now its own
  `{date(…,'2006')}`. `misc.datetime` is an RFC 3339 instant.
 - `sex`, `first-name` and `last-name` tables in `sv_SE` and `en_US`, weighted by
  bearers from SCB, the SSA and the Census Bureau; `first-name` links to `sex`, and a
  name both sexes carry is a row under each. `person` binds one first name and reads
  its sex through it, and its columns are `first`, `last`, `prefix` and `sex`;
  `person.femalefirst` and `person.malefirst` are no longer paths — draw
  `sex[f].first-name` and `sex[m].first-name` instead. `en_US.title` links to `sex` as
  well, so `en_US.person.prefix` draws `Mr` or `Ms` without contradicting the record's
  `sex`; `sv_SE.title` is an unsexed table of the same shape.
 - `sv_SE.personnummer` and `sv_SE.samordningsnummer`, Skatteverket's test series
  from a `sv_SE.birth-number` table under `sex`, in place of `sv_SE.ssn`, whose
  `ssn.mmdd`, `ssn.mmdd.m` and `ssn.mmdd.d` go with it. `en_US.ssn` now draws the
  ranges the SSA assigns and carries the columns `area`, `group` and `serial`, so
  `echo '{/en_US.ssn}' | fejkdata --format csv` writes a header where it used to fail; `en_US.itin` is new
  and carries the same three. `person.prefix` is null where a person has no title,
  where it used to be an empty string, so `--format sql` writes `NULL` and a `string`
  struct field reading it becomes `*string`.
 - `en_US.phone-exchange` draws the three digits after a US area code evenly from the codes
  NANPA may assign: `200` to `999`, except `211`, `311` … `911`, `555`, `950`, `958`,
  `959` and `976`. `en_US.phone` draws its `exchange` from it.
 - `ErrNoColumns` is exported, so a caller can tell the one record fence a path can
  answer from the rest.
 - An error names a spelling that runs: a layout is named single-quoted and free of
  its own quotes, a row of a table with no key is named as the path that selects it,
  `sv_SE.sex[f].first-name[Kim]`, and a category with no columns names the record
  that gives it one.
 - The shipped data comes in Go packages, one per module, each exporting `FS`:
  `github.com/larvit/fejkdata/data/en_US` (package `enus`), `…/sv_SE` (`svse`), `…/misc`,
  `…/geo/SE` (`geose`) and `…/geo/US` (`geous`). `data.Modules()`, from `github.com/larvit/fejkdata/data`, lists every one,
  and the CLI carries them all. Each package's `Modules()` returns its `FS` and every module
  it reads: `svse.Modules()` holds the Swedish places too. `New` loads the modules `WithDataFS(modules...)` names and
  the folders `WithDataPath` names, and fails, naming `WithDataFS`, where no such option
  is given. A manifest's `namespace` mounts its source under a dot path.
 - Package `github.com/larvit/fejkdata/cli` is the `fejkdata` command: `cli.Main(modules...)`
  runs it over the modules given, and returns its exit code, so a `main` package calling
  `cli.Main(append(data.Modules(), mydata)...)` builds a CLI carrying your own data. That
  CLI names itself in its help and errors by the file it runs from, and its `--version`
  prints its own module's version before fejkdata's.
 - A data source whose root holds a `.fejkdata.json` manifest with an `index` loads each
  category on the first call reaching it, as each shipped module does. Data failing to load,
  in `New` or on that first call, gives an error matching `ErrLoad`, and the CLI exits 1.
  A source without an index loads in `New`, with the indexed categories it reads. Every
  other shipped category loads on the first call reaching it, so one your replacement
  breaks fails there.
@@ -65,7 +65,6 @@
 | ID | Release | Exempt | Item | R | S | A | G | Goals | Score |
 |---|---|---|---|---|---|---|---|---|---|
 | 57 | 0.1.0 |  | **Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.** | 1 | 2 | 6 | 6 | 1.1 | 22.5 |
 | 89 | 0.1.0 |  | **Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.** | 3 | 3 | 6 | 7 | 10.1 | 16.0 |
 | 58 | 0.2.0 |  | **Ship prebuilt binaries, a container image, and packages for Homebrew, Scoop and the biggest Linux package managers, so the CLI needs no Go.** | 3 | 7 | 8 | 9 | 1.1, 7 | 29.2 |
 | 243 | 0.2.0 |  | **Add `en_US.ein`, drawing only the prefixes the IRS assigns.** | 2 | 2 | 7 | 8 | 2.1, 4 | 26.7 |
@@ -216,10 +215,6 @@
 ## Details
 ### 57. Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds.
 Nothing has shipped, so "no longer paths", "where it used to fail" and "where it used to be an empty string" describe versions no reader can have installed.
 ### 89. Fail `New` without `WithSeed`, naming the option, and seed the CLI from the system.
 Goal 10.1 has the caller supply the seed, and today `New` without `WithSeed` seeds itself. It ships in 0.1.0, which already makes `New` refuse a call that names no data, so `New` breaks once, before anyone depends on it. Mark `WithSeed` required in the Library section's options table. `New`'s tests then stop swapping `randomBytes`. `fejkdata_test.go` swaps it today, and while that test runs, every `New` in the package fails.