Document date() and time(), the name tables, the personal ids and their sources

CHANGELOG.md

@@ -13,8 +13,9 @@ replacement, and each removed path, column or flag.
   `key`, `name`, `weight` and `parent`; a path selects a row by key or name,
   `misc.country[SE]`, and descends to a linked table by name; linked tables draw
   consistently within one render and draw group. `rows` is an option, so no
-  template may carry a field of that name. Refused at `New`: a `name` without a
+  template may carry a field of that name. A `name` without a `key` resolves
-  `key`, a name spelling another row's key, a table named like a column of any
+  inside the table's `parent`. Refused at `New`: a `name` without a `key` or a
+  `parent`, a name repeating inside one parent row, a name spelling another row's key, a table named like a column of any
   table above it, a table whose format or cell references its own family, and,
   within one render and draw group, a path drawing a table another path selects a
   row of, or two paths pinning different rows of one table.
@@ -32,3 +33,15 @@ replacement, and each removed path, column or flag.
   `address` record over one consistent draw of them. `sv_SE.address` and
   `en_US.address` read those records, so `en_US.address.street` no longer carries
     `name` and `suffix`, and a locale folder loads only beside `geo`.
+- `{date(from,to,'layout')}` and `{time('layout')}`: a second between two days, or
+  within one, in a quoted Go layout. `sv_SE.date`, `en_US.date`, `sv_SE.time` and
+  `en_US.time` render through them, so `date.year`, `date.month`, `date.day` and
+  `time.hour` are no longer paths, and `misc.datetime` is an RFC 3339 instant.
+- `sex`, `first-name` and `last-name` tables in `sv_SE` and `en_US`, weighted by
+  bearers from SCB, the SSA and the Census Bureau; `first-name` links to `sex`, and a
+  name both sexes carry is a row under each. `person` reads them, so its columns are
+  `first`, `last`, `prefix` and `sex`, and `person.femalefirst` and
+  `person.malefirst` are no longer paths.
+- `sv_SE.personnummer` and `sv_SE.samordningsnummer`, Skatteverket's test series
+  over the sex the render drew, in place of `sv_SE.ssn`; `en_US.ssn` in the ranges
+  the SSA assigns, and `en_US.itin`.

DATA-LICENSES.md

@@ -11,6 +11,10 @@ Every shipped dataset, its source, its licence and the attribution it asks for.
 | `geo/SE/street.tsv` | [Trafikverket NVDB](https://www.trafikverket.se/) Gatunamn, through the open API | CC0 1.0 | none required | `data-import/geo-se.py` |
 | `geo/US/region.tsv`, `municipality.tsv`, `locality.tsv` | [Census Bureau](https://www.census.gov/) Gazetteer 2026 and population estimates 2025 | [public domain](https://www.usa.gov/government-works) | none required | `data-import/geo-us.py` |
 | `geo/US/postal-code.tsv`, `street.tsv` | Census Bureau ZCTA to place relationships 2020 and TIGER/Line 2025 address ranges and feature names | public domain | none required | `data-import/geo-us.py` |
+| `sv_SE/first-name.tsv`, `last-name.tsv` | [SCB](https://www.scb.se/) names with at least two bearers, 31 December 2022 | CC0 1.0 | "Källa: SCB" | `data-import/names-se.py` |
+| `en_US/first-name.tsv` | [SSA](https://www.ssa.gov/oact/babynames/) baby names, births 1930 to 2020, through [hackerb9/ssa-baby-names](https://github.com/hackerb9/ssa-baby-names) | public domain | none required | `data-import/names-us.py` |
+| `en_US/last-name.tsv` | Census Bureau surnames occurring 100 or more times, 2010 | public domain | none required | `data-import/names-us.py` |
+| `sv_SE/sex.tsv`, `en_US/sex.tsv` | curated (Skatteverket's test birth numbers are facts) | — | — | — |
 | `misc/country.tsv` | [datasets/country-codes](https://github.com/datasets/country-codes) | [PDDL 1.0](https://opendatacommons.org/licenses/pddl/1-0/) | none required | `data-import/country.py` |
 | `misc/currency.tsv` | [datasets/currency-codes](https://github.com/datasets/currency-codes); symbols from [Unicode CLDR](https://github.com/unicode-org/cldr) `en.xml` and `root.xml` | PDDL 1.0; [Unicode License v3](https://www.unicode.org/license.txt) | CLDR: "Copyright © 1991-2025 Unicode, Inc. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other countries." | `data-import/currency.py` |
 | `misc/httpstatus.tsv` | curated (IANA HTTP status codes are facts) | — | — | — |

README.md

@@ -217,17 +217,27 @@ Sources merge in order; matching folders combine, any other clash is won by the
 last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`, nor be
 `-`, which a struct tag reserves; dot-prefixed entries are skipped, so a data directory can also be a checkout.
 
-Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`,
+Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
-`person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`,
+`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
-`version` and `word`, formatted per locale. `misc` carries `car`, `coordinate`,
+`username`, `version` and `word`, formatted per locale; `sv_SE` adds
-`country` (ISO 3166), `creditcard` (Luhn-valid), `currency` (ISO 4217), `emoji`,
+`personnummer` and `samordningsnummer`, `en_US` adds `ssn` and `itin`. `misc`
-`httpstatus`, `language` (ISO 639), `mac`, `mimetype`, `objectid`, `timezone`
+carries `car`, `coordinate`, `country` (ISO 3166), `creditcard` (Luhn-valid),
-(IANA), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
+`currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpstatus`, `language`
+(ISO 639), `mac`, `mimetype`, `objectid`, `timezone` (IANA), `useragent` and
+`uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
 `misc.country.alpha2`, `misc.httpstatus.code` — which `--list` shows. `country`,
 `currency`, `httpstatus`, `language` and `mimetype` are [tables](#table), so
 `misc.country[SE].capital` and `misc.currency[Euro].symbol` select a row;
 [`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence.
 
+`sex`, `first-name` and `last-name` are tables weighted by bearers, from SCB, the
+SSA and the Census Bureau. `first-name` links to `sex`, so `sv_SE.sex[f].first-name`
+draws a woman's name, and a name both sexes carry is a row under each, so
+`en_US.sex[m].first-name[Taylor]` names the one a `first-name[Taylor]` alone cannot.
+`person` reads one draw of the three, so its `first` and `sex` columns agree, and so
+does a `personnummer` in the same render: its birth number is Skatteverket's test
+series, 238 for a woman and 239 for a man, which no real person is ever given.
+
 A `geo` folder holds one tree per country under its alpha-2 code: five
 [linked tables](#linked-tables) named alike, and an `address` record over one
 consistent draw of them, which the locale's `address` reads.
@@ -407,7 +417,9 @@ cell token, and refuses a TSV no category names, a key that is empty or repeats,
 weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`,
 `"` or `|`, which a selector cannot spell; the rows are indexed on the first draw that
 selects one. A `name` needs a `key`, since a name naming several rows is reported by
-their keys, and a name spelling another row's key is refused, since the key would
+their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]`
+is settled by the `sex` selected before it and a name repeating inside one parent row
+is refused; a name spelling another row's key is refused, since the key would
 select first and the name never. The table's options are its own — `rows`, `key`,
 `name`, `weight` and `parent` — so a column may be named `name`, as one usually is.
 
@@ -505,21 +517,30 @@ stays reproducible.
 | `{ulid()}` | sample | ULID, 26 Crockford base32 chars |
 | `{nanoid(n)}` | sample | URL-safe Nano ID, `n` chars |
 | `{iban(CC)}` | sample | length- and mod-97-valid IBAN for BE, DE, DK, ES, FI, NO or SE |
+| `{date(from,to,'layout')}` | sample | a second between two `YYYY-MM-DD` days, both included, in a quoted Go layout: `'2006-01-02'`, `'January 2, 2006'`, `'060102'`, `'2006-01-02T15:04:05Z'` |
+| `{time('layout')}` | sample | a second within a day: `'15:04'`, `'3:04 PM'` |
 | `{seq()}`, `{seq(name)}` | counter | next integer from 1 in this generator; `name` selects an independent counter |
 | `{calc(expr)}`, `{calc(expr,dp)}` | computation | an arithmetic expression over sibling fields ([Computation](#computation)) |
 | `{lowercase(x)}`, `{uppercase(x)}`, `{ascii(x)}` | transform | a field's value rewritten ([Transforms](#transforms)) |
 
 A derivation reads what is to its left, so place it after its payload; the
 buffer is per expansion, so a nested template keeps fixed parts out of the sum. A
-Swedish personnummer is a Luhn checksum over the nine digits before it:
+Swedish personnummer is a Luhn checksum over the nine digits before it, six of
+them a birthdate:
 
 ```json
-{ "format": "{century}{core}", "century": ["19", "20"],
+{ "format": "{date(1930-01-01,2010-12-31,'060102')}-{birth}{luhn()}", "birth": ["238", "239"] }
-  "core": { "format": "{digits(2)}{mmdd}-{digits(3)}{luhn()}", "mmdd": ["0115", "0704", "1218"] } }
 ```
 
-Renders e.g. `19811218-9876`. `{seq()}` spans `Fake` calls and `repeat`, resets
+Renders e.g. `811218-2389`. A layout is Go's: the reference time `Mon Jan 2
-with a new generator, and is the natural primary key for the SQL example above.
+15:04:05 MST 2006` spelled as the output should look, quoted, since a layout may
+carry the comma that separates arguments, with English names. Every second
+between the two days is reachable, so a layout with a clock draws the time too.
+Rejected at `New`: a bound that is no calendar date, or not before the other; an
+unquoted layout, naming the quoted one; a layout naming no field, which is text;
+and for `time` a layout naming a date field, naming `date`. `{seq()}` spans `Fake`
+calls and `repeat`, resets with a new generator, and is the natural primary key for
+the SQL example above.
 
 ### Computation
 
@@ -573,9 +594,9 @@ without naming `sv_SE`:
 
 Renders e.g. `Hej, Pat Smith!`. A reference path into a category is held like a
 [correlated](#correlated-fields) path, but for the whole render — one `Fake`, or one
-record — rather than one format: `{.person.femalefirst} {.person.last}` name one
+record — rather than one format: `{.person.first} {.person.last}` name one
 person, as do the same two references in sibling fields or a nested template, and
-`{lowercase(.person.femalefirst)}` reads that same draw. Each `repeat` iteration is
+`{lowercase(.person.first)}` reads that same draw. Each `repeat` iteration is
 a render of its own, in no group, so it draws anew, and a [draw group](#draw-group) holds a
 draw apart. A bare reference names no field and makes its own picks each time —
 `{/misc.uuid} {/misc.uuid}` is two draws — while the reference paths inside what it
@@ -594,8 +615,8 @@ its groups by name; the unnamed group spans them all.
 
 ```json
 { "format": "{payer} pays {payee}; signed {signature}",
-  "payer": { "format": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}", "drawGroup": "payer" },
+    "payer": { "format": "{/sv_SE.person.first} {/sv_SE.person.last}", "drawGroup": "payer" },
-  "payee": "{/sv_SE.person.femalefirst} {/sv_SE.person.last}",
+  "payee": "{/sv_SE.person.first} {/sv_SE.person.last}",
   "signature": { "format": "{/sv_SE.person.last}", "drawGroup": "payer" } }
 ```
 
@@ -728,8 +749,7 @@ renamed or retyped line is a major.
   key, or prefixing options, would tax every template to guard against a
   misspelt option.
 - **`{a|b}` stays beside nested choices.** `[[…], […]]` picks the same way, but
-  its arms are anonymous; `{femalefirst|malefirst}` keeps `person.femalefirst`
+  its arms are anonymous; `{female|male}` keeps `person.female` addressable.
-  addressable.
 - **Flags follow getopt_long.** `--name value` and `--name=value` both work; a
   short flag's value attaches or follows (`-s42`, `-s 42`) and short flags bundle
   (`-hn 3`), as every shell user expects. A single-dash long flag is rejected
@@ -1040,6 +1060,30 @@ renamed or retyped line is a major.
   bigger neighbour ship no address; counting the land outside every place too would
   drop a quarter of the places, whose codes straddle unincorporated land, for a
   postal city the USPS mostly names the same way.
+- **A layout is always quoted.** A layout may carry the comma that separates
+  arguments, `'January 2, 2006'`, and one spelling for every layout beats a rule
+  about which ones need the quotes, so the bare spelling is refused naming the
+  quoted one. The layout is Go's reference time because the library renders with
+  it and a Go caller already knows it; its names are English, and a locale's own
+  month and weekday names are data.
+- **No builtin reads the clock, so a date is bounded by days, never by an age.**
+  An `age(min,max)` would make a seeded fixture change with the day it runs on,
+  which is what a seed exists to prevent; a birthdate for someone 20 to 60 is
+  `date(1966-01-01,2006-12-31,…)`, re-pinned as any fixture is.
+- **A name column without a key resolves inside its parent.** A given name both
+  sexes carry is a row under each, so `name` cannot be the key; the parent's row
+  tells the two apart, `sex[f].first-name[Kim]`, the ambiguity error spells each
+  row inside its parent, and a name repeating inside one parent row is refused at
+  load, since nothing could then select it.
+- **The Swedish ids draw Skatteverket's test series.** A Luhn-valid personnummer
+  over a random birth number may be a living person's; 238 and 239 after any date
+  are blocked from assignment, so the shipped `personnummer` and
+  `samordningsnummer` use those, read from the `sex` table's `birth-number`
+  column so the number and the name agree on sex.
+- **The US given names come from a mirror of the SSA file.** ssa.gov refuses a
+  client outside the US, so `names-us.py` reads a GitHub copy that ends at 2020,
+  which a count over the births since 1930 barely feels; `--names` takes the
+  official zip.
 - **`List` advertises direct descents only.** `region.municipality.locality` is
   listed, and `region.locality` resolves too but is not: the set of every descent
   through a chain of five tables is every subsequence of it, and the direct chain is
@@ -1091,15 +1135,19 @@ A shipped table built from a source is rebuilt by its script under
 [`data-import/`](data-import), one command per dataset, fetching the source named in
 [`DATA-LICENSES.md`](DATA-LICENSES.md). Downloads are cached under
 `data-import/cache/`, so delete it to fetch afresh; `geo-us.py` fetches two
-TIGER/Line files per county it ships, a few hundred megabytes, and `geo-se.py` needs
+TIGER/Line files per county it ships, a few hundred megabytes, `geo-se.py` needs
 a Trafikverket API key, free at [data.trafikverket.se](https://data.trafikverket.se/),
-in `TRAFIKVERKET_API_KEY` or a `--key-file`:
+in `TRAFIKVERKET_API_KEY` or a `--key-file`, and the Census host behind `geo-us.py`
+and `names-us.py` rejects a client for a while after a burst, so `--surnames` takes
+a copy of the surname file:
 
 ```sh
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/country.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/currency.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/geo-us.py
 docker compose run --rm --user "$(id -u):$(id -g)" -e TRAFIKVERKET_API_KEY data-import data-import/geo-se.py
+docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-se.py
+docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-us.py
 ```
 
 To release, head `CHANGELOG.md` with the version's section in place of `Unreleased`

todo.md

@@ -42,7 +42,7 @@ value is composed; TSV says which values exist.
   and a `data-import/` directory of Python scripts that rebuild each TSV from its
   source, so a refresh is one command per dataset.
 - Make every shipped category a record with its building blocks as columns
-  (`sv_SE.person` → `femalefirst`, `malefirst`; `misc.uuid` → `variant`).
+  (`sv_SE.person` → `first`, `last`, `sex`, done in step 3; `misc.uuid` → `variant`).
 - Share the handle lists between `email.local` and `username` only if that is a
   clean win — a reference between shipped categories is a major once tagged.
 
@@ -75,9 +75,8 @@ countries; the README maps each to the native term.
 
 #### Builtins the data cannot express
 
-- `{date(from,to,layout)}`: a date in `[from, to]` in a Go layout, and `{age(min,max)}`
+- `{date(from,to,'layout')}` and `{time('layout')}` shipped in step 3; `age()` is
-  as the birthdate spelling; `{time(layout)}`. Personnummer, SSN, birthdate, card
+  rejected, README Decisions. Add a `unix` layout once something needs it.
-  expiry, unix time and ISO datetime all build on it.
 - Derivations: `{isin()}` (Luhn over letters expanded to digits), `{cusip()}`,
   `{aba()}` (3-7-1 weights), `{vin()}` (position 9 over the whole; a sample taking
   the WMI, since the check sits mid-string).
@@ -165,7 +164,9 @@ address, phone, national id, company and date names each.
 1. Table node, key and name selection, parent links, consistent draws, the
    choice-of-rows fence, `DATA-LICENSES.md`, `data-import/` — done.
 2. `geo/SE` and `geo/US`, and `address` in both locales on top of them — done.
-3. Weighted person names and valid ids in both locales; `date()`.
+3. Weighted person names and valid ids in both locales; `date()` — done. Middle
+   names wait for a draw group that shares its family's pins, so a second name
+   is drawn under the same sex.
 4. `misc` conversions and the new `misc` tables.
 5. The remaining locale categories: company, phone, finance, vehicle, words.
 6. Records with building-block columns across the shipped set; shape re-pin.