fejkdata/todo.md

# Plan

Ordered as the releases that ship it.

## v0.1.0

### Comprehension, until the panel scores 7.0

Nothing else ships while the score is under 7.0 — no feature, no category, no data — bar
what `AGENTS.md` excepts, per README goal 2. This section is one
round, replayed: the `comprehension-panel` skill's nine-seat findings run at depth 1,
a PR per item it names, then the run again, until the score passes. The run is the
nine-seat one: the round is meant to buy architectural change, and the four-seat
scoring run names too little to steer one. A fresh run replaces the items below.

Scores, newest last: 5.9 on 2026-09-20 (Navigation 7.0, Locality 5.2, Shape 5.9,
Self-sufficiency 5.7); 5.8 on 2026-09-22 (Navigation 6.7, Locality 5.0, Shape 5.9,
Self-sufficiency 5.6); 5.7 on 2026-09-26 (Navigation 6.7, Locality 5.0, Shape 5.8,
Self-sufficiency 5.6), every seat capped by Locality, and all nine naming
`drawWalk.walk` among the hardest; 5.7 on 2026-09-28 (Navigation 6.7, Locality 5.1,
Shape 5.8, Self-sufficiency 5.7), every seat capped by Locality, all nine naming
`drawWalk.walk` and the four path walks among the hardest; 5.8 on 2026-09-29
(Navigation 6.8, Locality 5.3, Shape 5.8, Self-sufficiency 5.7), seven seats capped
by Locality, all nine naming `drawWalk.walk` hardest and eight the unit they would
least modify; 5.7 on 2026-09-29 again (Navigation 6.6, Locality 5.2, Shape 5.9,
Self-sufficiency 5.8), four seats capped by Locality, six of the seven unit seats
ranking `readSurvey.walk` hardest and all seven the unit they would least modify; 6.1
on 2026-09-29 a third time (Navigation 7.0, Locality 5.6, Shape 6.3, Self-sufficiency
6.1), every seat's lowest dimension Locality, all seven unit seats ranking the
`readSurvey` walk or check hardest and the unit they would least modify.

This round, in order:

- Give `arm` its head as written, set where `splitArm` splits, and build
  `pathProbe.readRow`'s selector spelling from it: six seats found the slice of
  `arm.spelling` by `len(joinSegments(tail))` resting on an invariant nothing states.
- Pick `disagreement`'s fix through a named `itemKind` comparison: six seats could not
  see that the kinds' declaration order is the rule.
- Give `tableSite` one test for a cell, and have `traceRead` report no row outside a
  table: the zero value reads as row 0, a real cell, so each reader tests `table` first.
- Rename `ftoken` to `scanUnit`, beside `formatToken`, and `session` to
  `generatorState`: three seats confused the token types, and both architects read
  `session` as per-render state.
- State at `pinSet.pin` that its caller proved r sits inside the nearest pinned
  ancestor, which `pinRow` and `drawIn` do.
- Delete the prose two or more seats found empty: `loadData`'s comment, which also names
  `linkRefs` where `bind` runs, `table.drawRow`'s, `table.selector`'s, and `render.go`'s
  second and third "compile validated" notes.
- Run the nine-seat panel again and file what it names here.

### Data

Shape: T = table, t = template, c = choice. Read every factual list from a register
by a `data-import/` script, as README goal 12 asks.

- Fetch NVDB through `source.fetch` in `data-import/geo-se.py`: `nvdb_segments` calls
  urllib itself, with no retry and no User-Agent.
- Test that every `geo` tree holds the five table names and the `address` columns the
  README says port across countries: nothing checks it, and a new country breaks it silently.
- Strip the whitespace `data-import/territory.py` copies from its register:
  `misc.territory[CW].capital` renders ` Willemstad`, with a leading space.
- Place xlsx cells by their `r` reference in `data-import/xlsx.py`: a sheet omitting an
  empty cell shifts every later column left.
- Spell `misc.creditcard`'s digit runs `{digits(n)}`, and refuse a repeat over a lone
  `{digits(1)}` at `New`, naming that spelling: both render the same run.
- Add the remaining `misc` tables and templates, one row of the table below per chunk.
- Read `misc.emoji` from the Unicode `emoji-test.txt` register, and `misc.car` from an
  international make and model source; vPIC and Mobility Sweden are national, so they
  build `en_US.car` and `sv_SE.car`.
- Add the remaining locale categories: company, phone, finance, vehicle, words.
- Give every shipped category building-block columns, `misc.uuid` → `variant` among
  them, and one record shape across the national ids; re-pin the shape.
- Give the address records one column set across countries: `region` and
  `municipality` as columns on `geo.SE.address` too.
- Fill the 398 Swedish localities weighted 200 from SCB småorter.
- List `title` in both locales, and `sv_SE`'s `birth-number`, in the README's Data list
  of what each locale carries, which names neither.
- Give `url` and `email` a path that draws only domains nobody can register, keeping
  the wide set as the default, per goal 14: 17 of the 40 distinct ones shipped today
  sit on `.se`, `.nu`, `.io` and `.co`, which anyone may register, and only RFC 2606's
  `example.com`, `.net`, `.org`, `.test`, `.example`, `.invalid` and `.localhost`
  provably reach nothing.
- Audit the rest of the shipped set against goal 14 and give each a never-reaching
  path where it lacks one: `phone` draws live PTS and NANP ranges — PTS's five
  fiction series are the Swedish inert set, sourced in
  [research-sources-se.md](docs/research/research-sources-se.md), and the NANP one
  still wants a source — and `bankgiro`, `plusgiro` and `routing` draw live prefixes.
- Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does
  not hold: ISO 3166 reserves it for Western Sahara and the root zone has never been
  delegated it, so no resolver answers for it and goal 3 promises otherwise.
- Draw `en_US.phone`'s `exch` as a NANP central office code: `{int(100,999)}` renders
  a leading 1 in about an eighth of draws, which libphonenumber rejects, and
  `TestShippedUSPhone` proves the shape rather than the rule. Assert the rule with it.
- Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop
  drawing birthdates that reach 100: the format hard-codes `-`, and the 1930 floor
  makes the oldest draws invalid from 2031.
- Merge `en_US.ip` and `sv_SE.ip`, today byte-identical, into one `misc.ip`, per goal 4's
  one spelling per result.
- Accept a middle name, and draw a shipped `personnummer` inside a *selected* sex;
  both want a draw group sharing its family's pins. Stop the conflict error naming a
  rewrite that returns a different value where the read it conflicts with sits inside
  another category, and report a struct column's draw conflict with the path spelling
  a tag takes.
- Let `geo.US.locality[1714000].address` descend from a selected row into the template
  beside the family: require the path step to reach a sibling category, and seed every
  draw group of the render from the outer selector's pins.
- Read `email.local`'s and `username`'s handles from the shipped name tables, which
  goal 12 asks for and the hand-written list they use today does not meet. Keep the
  handle shape: `username` shortens a surname to `ahl` or `sjo`, and the US one draws
  unisex given names, neither of which falls out of the tables unaided.
- Draw `email.local` from `username`, so the two share one handle list.
- Rename `misc.territory.country` to `sovereign`, per goal 4: it holds a code, and
  `country` collides with the `misc.country` path it replaced.
- Add `misc.browser` as the parent of `misc.useragent`, so
  `misc.browser[Chrome].useragent` resolves, per goal 3.
- Group `data/misc` into folders where a group name makes a path easier to guess, per
  goal 4.

`misc` (locale-neutral)

| Category | Shape | Source | Licence |
|---|---|---|---|
| `ip` v4 documentation ranges, private, CIDR; `ipv6` `2001:db8::/32`; `mac` locally administered | t | RFC 5737, 1918, 9637 | facts |
| `creditcard` per network: IIN, length, CVV, expiry | T+t | network rules, `{luhn()}` | facts |
| `bic`, `isin`, `cusip` | t | structure rules | facts |
| `ean`, `upc`, `gtin`, `isbn`, `issn`, `imei`, `asin`, `sku` | t | check-digit rules, GS1 demo prefix 952 | facts |
| `product` name, adjective, material, department, `garmentsize` | T/c | LLM-written lists | — |
| `airport` IATA, ICAO, territory, municipality; `airline`; `aircraft`; `flight`, `seat`, `pnr` | T+t | OurAirports, Wikidata | public domain, CC0 |
| `element`, `unit`, `planet` | T | PubChem, BIPM, NASA | public domain, CC BY 4.0 |
| `animal` by class, `plant` | T | Wikidata | CC0 |
| `bloodtype` weighted (global) | c | AABB | facts |
| `color` named, hex, rgb, hsl, cmyk | T+t | CSS Color 4, XKCD | W3C, CC0 |
| `programminglanguage`, `os`, `browser`, `database` column/type/engine, `fileext` | T/c | Linguist, curated | MIT |
| `commit` message, branch, `sha` | t | LLM-written, `{hex(40)}` | — |
| `password`, `hash`, `jwt`, `slug`, `hostname`, `avatar`, `placeholderimage` | t | shapes | — |
| `book` title, author; `genre`; `instrument`; `dish`, `ingredient` | T | Gutenberg catalog, MusicBrainz, USDA | CC0 |
| `lorem`, `hacker`, `hipster`, `catchphrase`, `buzzword`, `quote` | T/c | lorem ipsum, LLM-written | — |
| `direction`, `continent`, `ulid` | c/t | — | — |

`sv_SE` ([source research](docs/research/research-sources-se.md))

| Category | Shape | Source | Licence |
|---|---|---|---|
| `person` title, birthdate, age, blood type weighted | t | geblod.nu distribution | facts |
| `organisationsnummer` by form, `vat` | t | Bolagsverket group digits, Luhn, `SE…01` | facts |
| `company` name patterns, legal form weighted | t | Bolagsverket registrations 2025 | CC BY 2.5 SE |
| `sni` industry codes | T | SCB SNI 2025 | "Källa: SCB" |
| `ssyk` occupations | T | SCB SSYK 2012 | "Källa: SCB" |
| `phone` mobile, fixed per area code, E.164 | T+t | PTS numbering plan 2024 | facts |
| `bankgiro`, `plusgiro`, `bankaccount` clearing + Luhn/mod-11 | T+t | Bankinfrastruktur CSV, Bankgirot rules | facts |
| `licenseplate` `ABC 123`, `ABC 12A`, blocked combinations | t | Transportstyrelsen | facts |
| `car` make, model weighted by registrations | T | Mobility Sweden 2025 | cite |
| `word` noun, verb, adjective, adverb; `sentence` | T+t | SALDO | CC BY 4.0 |
| month and weekday names, `holiday` | t+T | CLDR sv, lag 1989:253 | Unicode |
| `territory` localised names keyed by alpha2 | T | CLDR sv territory names | Unicode |

`en_US` ([source research](docs/research/research-sources-world.md), and `part1`–`part4`)

| Category | Shape | Source | Licence |
|---|---|---|---|
| `ein` valid ranges | t | IRS | facts |
| `company` names, suffix, `naics` | t+T | SEC tickers for patterns, NAICS 2022 | public domain |
| `phone` NANP with valid NPA/NXX | t | NANPA rules | facts |
| `routing` ABA with check, `bankaccount` | t | Fed prefix ranges | facts |
| `licenseplate` per state | T | hand-authored patterns | facts |
| `car` make, model | T | NHTSA vPIC | public domain |
| `word`, `sentence`, `paragraph` | T+t | Moby POS or WordNet | public domain / WordNet |
| month and weekday names, `holiday` | t+T | CLDR en | Unicode |

### Builtins the data cannot express

- Add `{isin()}` (Luhn over letters expanded to digits), `{cusip()}`, `{aba()}`
  (3-7-1 weights) and `{vin()}` (position 9 over the whole; a sample taking the WMI,
  since the check sits mid-string).
- Add `{base64url(n)}` for JWT shapes.
- Add a `unix` layout to `date()` once something needs it.

### Library and CLI

- Compare `calcParser.binary`'s operator as a rune: `byte(p.rs[p.pos])` reads U+012B
  `ī` as `+`, so `{calc(a ī b)}` compiles as `a + b`.
- Stop the `columnKinds` suggestion naming a field's own kind or a narrower one: an
  `int64` or `float64` field unproven in range is told to become itself, and a `uint64`
  one to become `int64`.
- Say in the README's Linked tables that the first table a render reads decides whose
  weights govern the family: `geo.SE.address` reads `street` first, so it draws by
  street segments and the region, kommun and postort populations never apply.
- Parse `defaultTable`'s path with the library's grammar: the CLI counts brackets of its
  own, so a change to the selector grammar desyncs the default `--table` silently.
- Step up from a row to its parent as the decision "A link column after a row steps up
  to the parent row, and a path may not end on it" states, refusing a step back down:
  `city[Oslo].country.name` is a load error today, and `city[Oslo].country` renders the
  cell `NO`, as `locality.municipality` does, which `List` stops advertising. Replace the
  README's `city[Oslo].country` example with
  `fejkdata -d ./mydata 'city[Oslo].country.name'`, rendering `Norway`.
- Key every table as the decision "A table selects by one key column, a code, never a
  free-form name" states: `geo.SE.municipality[Lund]` and `misc.territory[Sweden]` select
  a row today.
  - Have a selector that misses name the key of every row whose cell spells it,
    matched without case, else say the table selects by its key:
    `misc.territory[Sweden]` and `misc.territory[se]` name `[SE]`, `misc.protocol[tcp]`
    its key, and `misc.loglevel[Error]` `[err]`.
  - Delete the `name` option and the load checks it carries, a name spelling another
    row's key and a name repeating inside its parent, and have `New` refuse a table
    still carrying `name`, naming `key`.
  - Key `misc.loglevel` on `keyword`, and `geo.SE.locality` on the integer
    `data-import/geo-se.py` assigns as the decision states, linking `street` and
    `postal-code` by it.
  - Drop `key` from `last-name` and `sv_SE.title`, and re-pin `testdata/shipped_shape.txt`.
  - Leave no selection by name in README.md, CHANGELOG.md, docs/ or `--help`: no
    selector spelling a free-form name, `geo.SE.locality[Lund]`, `misc.currency[Euro]`, `first-name[Kim]`
    and the rest, no "key or name", "by name" or "either" wording about selectors, and no
    `name` option in an example.
- Render selected rows as the decision "A selected row is fixed, not drawn, and an
  unnamed read beside it is refused" states: `New` refuses
  `{/misc.territory[SE].capital} / {/misc.territory[FI].capital}` today, and a choice
  whose items select different rows. Have the refusal of an unnamed read name the
  rewrite through each selection: beside `{/geo.US.locality[1714000].name}`,
  `{/geo.US.municipality.name}` names none today, where
  `{/geo.US.locality[1714000].municipality.name}` holds once the step up above lands.
  Scope the README's "A reference path into a category is held" to a path that
  selects no row.
- Refuse a struct tag's `datatype` naming the Go type that sets it before proving its
  values, as the README's Library section promises: `datatype: boolean` over
  `{int(1,2)}` on an `int` field answers `prints an integer, not a boolean`.
- Call `templateError` a compile failure: its doc says render failure, and it wraps
  what `NewTemplate` and `NewRecordTemplate` refuse.
- Report the same error every load for a table with two bad options, and for a
  folder with two unnamed rows files: `readTableOptions` and `loadDir` return on the
  first in Go's map order.
- Report a CRLF rows file holding only its header as having no rows: `parseRows`
  answers "line 2 is empty".
- Reword `prove`'s refusal of a typed column reading a row, `{/region}`: say the
  format is the table's own (`region's format "{name}"`), name the read to write,
  `{/region.<column>}` with its columns, and stop calling a one-column format,
  `{code}`, composed text.
- Keep one copy of the selector characters in `data-import/`: `httpmethod.py` and
  `protocol.py` each copy `inSelector`, and `tsv.py`'s `write` enforces neither.
- Refuse two whole reads of one table family in one render, which goal 5 promises is a
  load error: `{/sel}|{/sel}` panics out of `Fake` where the two draws land on rows whose
  cells select different rows of another table, and
  `{/geo.SE.locality}|{/geo.SE.municipality}` renders a locality outside the
  municipality beside it. Each read draws its own row, so cells of two rows render
  together, where the fence counts them alternatives and compares neither. Have the error
  name the path spelling that holds, `{/geo.SE.locality.name}|{/geo.SE.municipality.name}`,
  as the README's rule on a rejected spelling asks.
- Name the file a link error comes from beside its category path, `sub.x
  (/d/a/sub/x.json): …`: with several `--data-path` layers the author has to work out
  which directory won, while a parse error on the same file names it.
- Name the row a refused read sits in: where both reads come from TSV cells the overlap
  error names the category and the two columns but no row, so the author greps a
  register-sized file for the cell — and "name the fields you want instead" rewrites a
  cell that is not the one to fix.
- Spell a table one way across the family errors: `checkFamilyPair` names
  `drawn.category`, `pinRow` names `t.path` and `mustRow` panics with `t.category`, so one
  table is `territory` and `misc.territory`, and the short spelling names no file where two
  folders hold that name.
- Let a table column carry a `datatype`, so `--format json` writes
  `"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only
  a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text
  everywhere else. Typing a shipped column changes what `json` and `sql` write, so the
  capability is a minor and applying it to `misc.httpmethod` is a major.

### Release

- Read the tag through `git/ref/tags/{tag}` in `publish_release.py`: GitHub answers
  `tags/{tag}` with 404 whatever exists, so the burnt-version guard never fires and a
  release lands on a tag already pointing at another commit.
- Document `NewRecordTemplate`, `ErrNoColumns`, `ErrNoData` and `MaxRepeat` in the
  README's Library section, which names none of them.
- Build on a manual run of `test.yml`: `workflow_dispatch` leaves `github.event.before`
  empty, so the diff compares `HEAD` with itself and skips `docker build`.
- Rewrite `CHANGELOG.md`'s `[Unreleased]` as what v0.1.0 holds: nothing has shipped, so
  "no longer paths", "where it used to fail" and "where it used to be an empty string"
  describe versions no reader can have installed.
- Ship prebuilt binaries so the CLI needs no Go: GoReleaser attaching them to the Gitea
  release the tag workflow publishes, a container image, Homebrew and Scoop. A checkout
  build prints `devel` for `--version`; the binaries carry the stamped tag.

## v0.2.0

- **Promise in `Generator`'s godoc that its renders run one at a time**, so a caller
  wanting parallel throughput makes one generator per goroutine.
- Ship the full registers as packs, each a Go module with its own `embed.FS` and a zip
  for `--data-path`: every US place of 10,000 and more streets per locality, built by
  `--min-population` and `--streets-per-locality`. The default embed stays under ~1 MB
  per country and `New` under ~50 ms.
- Add the locales nb_NO, da_DK, fi_FI, de_DE, en_GB, nl_NL, fr_FR and es_ES, with
  person, address, phone, national id, company and date names each, and the `geo/`
  trees NO, DK, FI, NL, FR, AU, CA, ES, GB and DE their `address` reads by reference;
  AU and CA get a tree with no locale of their own.
- Add `{btc()}` and `{eth()}`, which need sha256 and keccak for Base58Check and EIP-55.
- Pair a street with its exact postnummer, which needs an application to Lantmäteriet;
  today a street goes to the nearest postal code centroid. It rewrites shipped rows, so
  it is a major once 1.0 is cut and a minor before that.

## v0.3.0

Names no items yet, so planning it is a chunk of its own: it takes the additive work
that lands after v0.2.0, and it has to add something, since a minor only adds. Its job
is to be the release with no breaking change that v1.0.0 waits on — v0.2.0 cannot be,
because pairing a street with its exact postnummer rewrites shipped rows.

## v1.0.0

- Cut it once the shipped data is in its record shape and one full minor has shipped
  with no breaking change, per the README's Versioning table: v0.1.0 settles the
  record shape and v0.3.0 is the quiet minor.
- Announce it where a developer choosing a fake-data tool already reads, which is
  goal 1's second half and waits for the tag: complete, stable, installable without
  Go. Write the README's first screen for someone deciding in a minute.

## Not release-bound

- Give every entry in `docs/decisions.md` the goal it serves, its date, who made it and
  the premise it rests on; propose a goal for an entry that serves none.
- Hold every change the charter says owes a `CHANGELOG.md` entry to one in CI: the
  check covers `data` and `testdata/shipped_shape.txt` alone, where `AGENTS.md` adds a
  flag, an exit code, an exported name, a fence, a builtin and the lowest Go.
- Measure the allocation gate over the shipped `geo` trees too: `pinSet.inline [8]` is sized
  for them, and `perf_test.go` checks a synthetic five-deep tree only.
- Name goal 2's four criteria with the axes the panel scores, so a recorded score maps
  back to the clause it came from: the goal spells them out in prose while `todo.md`
  records `Navigation`, `Locality`, `Shape` and `Self-sufficiency`.
- Delete `doc.go`'s "The draw fences are …" sentence: it names a function, a type and
  a helper reached only through `readSurvey.check`, and misses `checkDrawGroup`,
  `checkOwnFamily` and `checkNestedDrawGroup`; the README's Layout indexes the fences by file.
  Say "fences" in the `pin` entry then, which names the draw fences too.
- Cut the README's Layout block to the lines that say what a file name cannot:
  `calc.go`, `doc.go` and `cmd/fejkdata/` restate their names.
- Cut `AGENTS.md` to the rules only it states: the round procedure stands word for word
  in `todo.md` and the `gh` rule opens with the README's GitHub decision, so each is a
  second copy that drifts. Split the comprehension rule while there, which packs the
  gate, its exceptions, the round and the at-or-above-7.0 regime into one line.
- Scope `AGENTS.md`'s `Hard tabs.` to the Go source, or drop it: `gofmt` already gates
  Go at `Dockerfile:16`, and the shipped JSON under `data/` is two-space, so the rule as
  written is one no Go file can break and every data file does.
- Make `gitea.larvit.se/larvit/fejkdata` a pull mirror of GitHub:
  Gitea converts a repository to a mirror only by re-creating it, so until that runs
  the copy there is a second owner of one history and goes stale from this commit.
- Stop `/cache` collecting root-owned files, which today every command documented
  without `--user` leaves for the next `--user` one to trip over, and which the README
  answers with a chown a reader has to repeat. Put `--user` on all of them, or give the
  `x-go` anchor a `user:` — that one needs a mechanism, since Compose interpolates from
  the process environment and neither `UID` nor `GID` is exported there, so
  `user: "${UID}:${GID}"` resolves to `":"`.
- Spell the latest supported Go once: `Dockerfile`'s `ARG GO_VERSION` and
  `compose.yaml`'s `golang:${GO_VERSION:-…}` pin one version twice, and a bump that
  moved only the first left `docker compose run --rm fmt` formatting to a `gofmt` the
  gate rejects. A `renovate.json` custom manager reads the second today; drop it with
  the duplication.
- Publish a homepage with an in-browser generator: the library compiled to WebAssembly,
  so visitors generate as much data as they like in their own browser.