# fejkdata Locale-aware fake data for tests and fixtures, generated from JSON templates. Use it as a Go library or the CLI — no data on disk, no dependencies, and a seed makes output reproducible. ```sh go install github.com/larvit/fejkdata/cmd/fejkdata@latest fejkdata sv_SE.person # Sara Eriksson ``` ## CLI ```sh fejkdata sv_SE.person # Sara Eriksson fejkdata sv_SE.person.last # Eriksson fejkdata --seed 42 sv_SE.address # the same address every run fejkdata -n 3 --separator ', ' sv_SE.word # nät, barn, sol fejkdata --list # every path the data offers fejkdata 'misc.territory[SE].capital' # Stockholm — a table's row, selected by key or name fejkdata 'geo.SE.locality[Lund].street' # Fjelievägen — a linked table, drawn inside the row fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data fejkdata --no-shipped-data -d ./mydata --list # only your data fejkdata 'name: {/sv_SE.person.last}' # name: — an inline template fejkdata "{date(1990-01-01,2010-12-31,'2006-01-02')}" # 2003-11-27 — the argument in "…", the layout in '…' fejkdata '{"format":"name: {x}","x":["bosse","lina"]}' # name: bosse or name: lina ``` A path names a category, or a field inside one: each dot segment descends one level — folders, then the category (a JSON file), then fields — and `[SE]` after a [table](#table) selects its row. An argument that is a JSON object, array or string, or that carries a `{` token, is instead an **inline template**: a format string or a JSON value compiled and rendered on the spot. Its tokens reach the data by reference from the root — `{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike available. A path never contains a brace or a quote, and a bracket only as a selector after a name, so the two spellings cannot collide; which spellings an inline template rejects, and what each names instead, is under [Decisions](docs/decisions.md#an-argument-is-a-template-by-its-shape-not-by-a-flag). | Flag | | |------|--| | `-d`, `--data-path D` | a directory to layer over the shipped data; repeatable, the last wins a name clash | | `--no-shipped-data` | load only the `--data-path` directories | | `-s`, `--seed N` | same seed, version and data: identical output | | `-n`, `--repeat N` | render the value N times (up to 1048576), each an independent draw, streamed | | `--separator S` | between repeated values (default a newline) | | `--format F` | `text` (default), `json`, `ndjson`, `csv` or `sql` — a record's columns, one record per row (json frames them as an array) | | `--table T` | the INSERT target for `--format sql` (default: the path's last segment, or `records` for an inline template) | | `--list` | print every path, then exit | | `--version`, `-h`, `--help` | print, then exit | `--name value` and `--name=value` both work, a short flag's value attaches or follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see [Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing dir, unknown path), `2` misuse — a bad flag, an argument that names neither a template nor a path, or an inline template that does not compile. From a checkout: `go run ./cmd/fejkdata …`. ### Your own data A category is a JSON file in a directory. Save this as `mydata/sql.json`: ```json { "format": "INSERT INTO users VALUES({sql-username});", "sql-username": { "format": "'{username}'", "repeat": 3, "separator": "),(", "username": ["pixelfox", "snork", "turbohund", "blip", "zoom", "wahoo"] } } ``` ```sh fejkdata --seed 1 --data-path ./mydata sql # INSERT INTO users VALUES('zoom'),('wahoo'),('blip'); fejkdata --repeat 100 --data-path ./mydata sql > seed.sql ``` That `format` string is the free-form spelling — you hand-write the whole row. For structured output a record writes the row for you. ### Records A record is a template seen as columns: its fields are the columns, its `format` the whole. `--format json|ndjson|csv|sql` writes the records; the library's `FakeRecord` (below) hands back the columns. A column is a string unless it declares a [datatype](#datatype), and a [`null`](#null) item draws it as null. Save `mydata/users.json`: ```json { "format": "{first} {last}", "first": ["Ada", "Bo"], "last": ["Lovelace", "Ek"] } ``` ```sh fejkdata --seed 1 --data-path ./mydata --format json users # [{"first":"Bo","last":"Lovelace"}] fejkdata --seed 1 --data-path ./mydata --format ndjson users # {"first":"Bo","last":"Lovelace"} fejkdata --seed 1 --data-path ./mydata --format csv users # first,last → Bo,Lovelace fejkdata --seed 1 --data-path ./mydata --format sql users # INSERT INTO "users" ("first", "last") VALUES ('Bo', 'Lovelace'); fejkdata --seed 1 --data-path ./mydata --format sql --table people users # INSERT into another table ``` `--repeat` streams that many records — `json` frames them as one array document, `ndjson` writes one object per line, `csv` a row after a header, `sql` one INSERT per line. Only a category-level template is a record; a field, choice or folder errors, and so does a `repeat` on the template itself, which composes the format into one string rather than projecting columns — ask for more records with `--repeat`. A `repeat` on a column is fine. Columns are written in name order, whatever order the fields appear in. A category whose fields are the parts of one value — `sv_SE.price`, `sv_SE.version`, `misc.uuid` — projects those parts rather than the value, so for one column holding what `Fake` renders, write `{"format":"","price":"{/sv_SE.price}"}` as a fieldless category asks for. A column carrying a newline keeps it inside the quoted CSV field or the SQL string literal, so a row can span physical lines: read the stream with a CSV or SQL parser rather than splitting it on newlines. The SQL is ANSI — identifiers in double quotes, a literal quote doubled (`''`), backslashes passed through — so a hyphenated field like `postal-code` stays a valid identifier. PostgreSQL and SQLite take it as written; MySQL and MariaDB need `ANSI_QUOTES` and `NO_BACKSLASH_ESCAPES` set first, or they read `"users"` as a string and a backslash as an escape. A record written only to emit columns still needs a `format` — the grammar's one required key — so `"format": ""` carries the fields with an inert format: it renders nothing by `Fake`, and is compiled only so the tree's fences still run. The columns are the point, and their facts stay together: a record is one render, so columns that read a path into one category — `{/currency.code}` and `{/currency.symbol}` — read one draw of it ([References](#references)), and a [draw group](#draw-group) draws a column apart. That one draw is also why the columns of one record may not overlap — `{/cat.a}`, or a bare `{/cat}`, beside `{/cat.a.b}` is refused, naming the fields to write instead, as [One draw, one spelling](#one-draw-one-spelling) refuses that pair inside a single format. Both fences run at load, so a category that loads renders as either shape. A category never references itself — `{/users.first}` or a bare `{/users}` inside `users` describes a draw other than the fields beside it — so read a sibling as a path, and put a value two fields share in its own category and reference that. A field hold, transform or operand ties fields together within one column as always (see [Correlated fields](#correlated-fields) and [Decisions](docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load)). ## Data The shipped set under [`data/`](data) — one folder per locale (`en_US`, `sv_SE`) plus a locale-neutral `misc` folder — is embedded, so the CLI and the library work with no data on disk. A directory is a namespace: each JSON file is a category named after the file, each subdirectory a dot-path segment, so `mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one. Sources merge in order; matching folders combine, any other clash is won by the last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"`, `/` or ` as `, which binds a [name](#names), nor be `-`, which a struct tag reserves; dot-prefixed entries are skipped, so a data directory can also be a checkout. Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`, `ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`, `username`, `version` and `word`, formatted per locale; `sv_SE` adds `personnummer` and `samordningsnummer`, `en_US` adds `ssn`, `itin` and `phone-exchange` (the three digits after the area code). `misc` carries `car`, `coordinate`, `creditcard` (Luhn-valid), `currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpmethod`, `httpstatus`, `language` (ISO 639-1, with its 639-2/T code), `loglevel` (syslog severity), `mac`, `mimetype`, `objectid`, `port`, `protocol`, `territory` (ISO 3166-1), `timezone` (IANA), `tld` (IANA root zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.symbol`, `misc.territory.alpha2`, `misc.httpstatus.code` — which `--list` shows. `car`, `currency`, `httpmethod`, `httpstatus`, `language`, `loglevel`, `mimetype`, `port`, `protocol`, `territory`, `timezone`, `tld` and `useragent` are [tables](#table), so `misc.territory[SE].capital` and `misc.currency[Euro].symbol` select a row; `car` and `useragent` carry no key or name, so they are drawn from rather than selected in. Where a script under [`data-import/`](data-import) reads a source, the facts come from that source, and the script drops a row only by a rule it states; elsewhere a small hand-written set ships, and [`todo.md`](todo.md) carries the step that replaces it (goal 14). Copy that states no fact, such as `word` and `sentence`, stays hand-written. Where a category has no path that draws only values reaching nothing real (goal 16), write a fixed value such as `example.com` into the template; `todo.md` plans the path. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's other entries, WebDAV and DeltaV among them, do not ship. Its `safe` and `idempotent` are `true` or `false`, so a Go `bool` reads them, though `--format json` and `sql` write them as text, a table column carrying no datatype. `misc.protocol` carries a `number` and a `name`, the keyword itself where the register spells none out, and selects by either spelling. `misc.port` is the TCP assignments, rendering the number a port field holds with the IANA `service` beside it, and selecting by number alone. A draw spans the whole register, so pin `misc.port[443]` where a fixture needs a port a reader recognises; it stops at 49150, IANA assigning nothing above, so an ephemeral source port is `{int(49152,65535)}`. `misc.tld` is the root zone's delegated TLDs, keyed with the leading dot `misc.territory.tld` already carries, `.se`. Its `type` is the register's — `country-code`, `generic`, `generic-restricted`, `infrastructure` or `sponsored` — and its `unicode` the form the register displays, `.рф` for `.xn--p1ai` and the key itself for an ASCII one, which selects the row too, so `misc.tld[.рф]` renders `.xn--p1ai`. A draw spans the whole zone, `.arpa` and `.zuerich` alike, so pin `misc.tld[.com]` where a fixture needs one a reader recognises. `misc.loglevel` is the eight syslog severities, rendering the keyword a configuration writes, `err` and `info`, keyed by the numerical code a PRI encodes and selectable by either, so `misc.loglevel[3]` and `misc.loglevel[err]` are one row. `misc.loglevel.severity` is the name RFC 5424 spells, `Error` beside `err`; it selects nothing, the code and the keyword do. [`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence. `misc.timezone` is every zone tzdb gives a shipped territory, from one apiece for most of them to dozens for the largest, weighted by the population GeoNames records in each, so `misc.territory[US].timezone` draws `America/New_York` far more often than `America/Nome`. It links to `misc.territory`, so `misc.territory[SE].timezone` is `Europe/Stockholm` and a drawn territory and zone agree. Its `offset` is the zone's *standard* offset, so do not pair it with a drawn `misc.datetime`: in a zone that observes DST it is the wrong one half the year, which is what storing a zone name avoids. A zone tzdb named recently — `Europe/Kyiv`, `America/Ciudad_Juarez` — is rejected outright by a consumer resolving it against older tzdata, so where the consumer validates the zone, pin one rather than draw it. There is no `UTC` row, tzdb giving that name no territory — spell it as the text `"UTC"`. `misc.useragent` carries `browser`, `device` and `os` beside the string, and `misc.car` a `make` and a `model`, until an international source replaces them. ISO 3166-1 codes territories, not sovereign states, so that is what the table is called: Greenland and Åland have codes of their own, and `misc.territory.country` names the state each belongs to — `DK` for Greenland, `FI` for Åland, and its own code for a sovereign one, or for one the register names no state for. `sex`, `first-name` and `last-name` are tables weighted by bearers, from SCB, the SSA and the Census Bureau. `first-name` links to `sex`, so `sv_SE.sex[f].first-name` draws a woman's name, and a name both sexes carry is a row under each, so `en_US.sex[m].first-name[Taylor]` names the one a `first-name[Taylor]` alone cannot. `person` reads one draw of the three, so its `first` and `sex` columns agree, and so does a `personnummer` in the same render: its birth number, `sv_SE.birth-number` under `sex`, is Skatteverket's test series, 238 for a woman and 239 for a man, which no real person is ever given. `en_US.title` links to `sex` too, so a person's prefix never contradicts it. A person of a chosen sex is assembled from the tables — `sex[f].first-name` beside `last-name` — while a shipped `personnummer` agrees with the sex its own render *drew*, not with one a path selects. That test series is also small: a personnummer is one of about 70,000 values, a day in 1930–2025 against the two birth numbers, so a fixture past a few hundred rows repeats one and a `UNIQUE` column needs a category of your own. `sv_SE.date` and `en_US.date` are uniform over 1970-01-01 to 2029-12-31, `misc.datetime` over 2000-01-01 to 2029-12-31. What the two locales do not share: `en_US.address` carries a `region` column the Swedish one has no use for, and `sv_SE.title` has no `parent`, so it is selected as `sv_SE.title[dr]` rather than inside a sex. A `geo` folder holds one tree per country under its alpha-2 code: five [linked tables](#linked-tables) named alike, and an `address` record over one consistent draw of them, which the locale's `address` reads. | Table | `geo.SE` | `geo.US` | Weight | |-------|----------|----------|--------| | `region` | län, by code or name | state, by USPS abbreviation or name; `code` is the FIPS code | population | | `municipality` | kommun, by code or name | county, by FIPS code or name | population | | `locality` | postort, by name | incorporated place of 25,000 people or more with a postal code of its own, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population | | `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges | | `street` | gatunamn, the ten with most road segments per postort | street name, the ten with most address ranges per place | segments; address ranges | `geo.SE.region[Skåne län].municipality` draws a kommun in Skåne, `geo.SE.locality[Lund].street` a street in Lund, and `geo.US.region[IL].locality[Springfield]` settles which Springfield. A region row carries its `timezone`, the state's predominant zone, and a locality its `lat` and `lon`. What ports across countries is the five table names, the `name` column, selection by name, and the `address` record's columns `street`, `street-number`, `postal-code` and `locality`; every other column is the country's own, `code` on a Swedish region but `abbr` on a US one. ## Library ```sh go get github.com/larvit/fejkdata # Go 1.22+ ``` ```go f, err := fejkdata.New(fejkdata.WithSeed(42)) if err != nil { log.Fatal(err) } v, err := f.Fake("sv_SE.address") // "Järvedsvägen 43\n891 77 Järved" paths := f.List() // the paths Fake renders, sorted; direct descents only v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once v = t.Fake() // render many times, no re-parse r, err := f.FakeRecord("users") // one record: each field a column s := r.JSON() // {"first":"Ada","last":"Lovelace"} r, err = f.FakeRecordTemplate(`{"format":"{x}","x":["a","b"]}`) // compile + render inline err = f.FakeStruct(&user) // fill a struct's fake:"…" tagged fields ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, else a path ``` | Option | | |--------|--| | `WithSeed(n)` | same seed, version and data: identical sequence | | `WithDataPath(dir)` | layer a directory; repeat to layer several, the last wins a clash | | `WithDataFS(fsys)` | layer an `fs.FS`, such as your own `embed.FS` | | `WithoutShippedData()` | load only what you give | `New` refuses a mistake in the data, and `NewTemplate` one in the template. On a loaded generator: - `Fake` fails only for a path that names nothing or could name two things, with the same error every call. - `FakeStruct` fails only for a non-struct argument or a type its tags do not describe, with the same error every call. - `Template.Fake` cannot fail. A `*Record` carries its columns via `Columns()` — each a `Column` of `Name`, `DataType`, rendered `Value` and `Null` — and serializes them with `JSON()` (one object), `CSVHeader()`/`CSVLine()`, or `SQLInsert(table)` — the shapes the CLI's `--format` writes. `FakeRecord` and `FakeRecordTemplate` take a record; a path or template that is not one — a bare string, a choice, or a folder — errors. ```go type User struct { ID int64 `fake:"{seq()}"` Last string `fake:"sv_SE.person.last"` Age uint8 `fake:"{int(18,99)}"` Nick *string `fake:"[null, \"{/sv_SE.username}\"]"` Home Address // filled from Address's own tags } ``` `FakeStruct` fills a struct through a pointer: each exported field tagged `fake:"…"` is a column of one record, its tag a path or an inline template — told apart by `IsTemplate`, as the CLI tells an argument — and its Go type the column's [datatype](#datatype): a string, bool, integer or float kind, or a pointer to one, which a [`null`](#null) item leaves nil. An integer stays within int64 whatever its kind, and a value the kind cannot hold, such as `{int(0,300)}` in a `uint8`, is refused naming a kind that holds it. The fields an embedded struct promotes are columns of the same record; a named struct field, or a pointer to one, fills from its own tags as a record of its own, so its references draw apart from its parent's. `fake:"-"` leaves a struct field, embedded or named, or a pointer to one, unfilled. Untagged fields keep their values, and so does a pointer back to a struct already being filled; a type whose fields reach more than 1024 structs is refused, naming `fake:"-"` to cut it. The first call for a type compiles its tags and reports what they get wrong, with the same error on every later call; a `datatype` in a tag names the Go type that already sets it. A `*Generator` is safe for concurrent use; a seeded sequence is reproducible only when drawn from one goroutine. Changing how a value is composed shifts the seeded stream for that value and everything drawn after it. ## Data format Every value is a **node**, nestable without limit: | Node | JSON | Renders | |------|------|---------| | string | `"Malmö"` | its text, with any `{…}` tokens expanded | | choice | `["a", "b", …]` | one item, picked at random | | template | `{"format": "…", …}` | its format, with `{name}` tokens rendering the named fields | | table | `{"format": "…", "rows": "x.tsv", …}` | its format over one row of the TSV beside it ([Table](#table)) | ### Format string Every character is literal except a `{…}` token: | Token | Renders | |-------|---------| | `{name}` | the sibling field `name`, or the pick bound to the name ([Names](#names)) | | `{ref as name}` | nothing: binds one pick of the reference `ref` to `name` ([Names](#names)) | | `{name.field}` | `field` of one draw of `name` ([Correlated fields](#correlated-fields)) | | `{a\|b}` | one of the named fields, even odds | | `{fn(args)}` | a builtin ([Functions](#functions)) | | `{/path}`, `{.path}`, `{..path}` | a node reached from the data root, this file's folder, or the folder above ([References](#references)) | | `{{`, `}}` | a literal `{` or `}` | ```json "100 Main St, Apt {int(1,9)}{upper(1)} — tel {digits(3)}-{digits(4)}" ``` Renders e.g. `100 Main St, Apt 4B — tel 555-0199`. Random characters come from the sample functions, so a phone pattern is `070-{digits(3)} {digits(2)} {digits(2)}`. A lone `}` is a load error naming `}}`; the arms of `{a|b}` must differ, a repeated arm being a second spelling of [weight](#weight). ### Weight An item of a choice may carry a `weight` (default `1`) to skew its odds: ```json [ { "format": "070-{digits(3)} {digits(2)} {digits(2)}", "weight": 10 }, "08-{digits(3)} {digits(2)} {digits(2)}", { "format": "010-{digits(3)} {digits(2)} {digits(2)}", "weight": 2 } ] ``` Renders `070-412 38 91` ten times as often as `08-…`. A string item is weighted by writing it as `{ "format": "AB", "weight": 3 }`. Rejected at load: a weight that is negative, non-numeric, `0` (never drawn — remove the item), `1` (the default) or outside a choice, and a repeated item — `["a", "a", "b"]` is `[{ "format": "a", "weight": 2 }, "b"]`, and the error says so. ### Repeat A template may carry `repeat` (an integer above `1`) to render its format that many times — each an independent draw — joined by `separator` (default `""`): ```json { "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] } ``` Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat`, a `separator` of `""` (the default), and a `repeat` that multiplies to more than 1 048 576 renders along any path of nested repeats. ### Datatype A record column may declare `datatype` — `integer`, `number` or `boolean` — so `json` writes `42` rather than `"42"` and `sql` a bare literal; a column without one is a string: ```json { "format": "", "id": { "format": "{seq()}", "datatype": "integer" }, "paid": { "format": "{p}", "p": ["true", "false"], "datatype": "boolean" }, "total": { "format": "{calc(net * qty, 2)}", "net": ["19.99", "5.00"], "qty": ["3", "7"], "datatype": "number" } } ``` Writes e.g. `{"id":1,"paid":true,"total":59.97}`. A column is a field of the top-level template, or an item of a choice standing in for one; `datatype` anywhere else is a load error. A typed column holds one value, alone in its format: a literal, one `{int()}`, `{float()}`, `{seq()}` or `{calc()}` call, or a read that lands only on such values. `integer` is an int64 written `0|-?[1-9][0-9]*` — `{float()}` prints one at `0` decimals within int64 — `number` a JSON number, `boolean` `true` or `false`. A value its datatype cannot hold is a load error naming it: ```text order.id: datatype integer: {digits(3)} prints text, not an integer order.id: datatype integer: "1{digits(2)}" is not one value; write one literal or one {float()}, {int()}, {seq()} or {calc()}, or read one ``` A column of one reference alone to another record's column — `"score": "{/src.score}"` — is that column: it takes the column's datatype and is null where the column is, and a struct field tagged `src.score` is nil there. A `datatype` of its own types the column's values where they prove it, and one restating the datatype it takes is refused. Any other read renders the column's text, a null as `""`. A typed column's `{calc()}` must be proven to print a number: each operand a number literal, an `{int()}`, `{float()}`, `{seq()}` or `{digits()}` call, a calc, or a read of such values, whose bounds keep every divisor from zero and the result within `1e300`. What the bounds cannot show is refused — `{calc(a / b)}: divides by b, which is not proven nonzero`. The calc fills an `integer` column at `0` decimals, or over whole operands with no `/`, while its bounds stay within int64. ### Null A `null` item draws a record column as null: `json` writes `null`, `sql` `NULL`, and `csv` an empty field, with an empty string written `""` — the convention PostgreSQL's `COPY … CSV` reads. A record of one null column is a blank line, which `COPY` reads as null but most CSV readers skip, so write such a record as `json` or `sql`. `Fake` renders a null as `""`. The other items' weights skew its odds: ```json { "format": "", "deleted_at": null, "middle": [null, { "format": "{n}", "n": ["Ann", "Eva"], "weight": 3 }] } ``` `deleted_at` is null every draw, `middle` a name three draws in four. Rejected at load: `null` anywhere but a column, naming `""`, and a column whose items hold different datatypes. ### Table A table is a category whose rows come from a TSV beside its JSON file: the header names the columns, each line below it is one row, and the format renders the row drawn. Save `mydata/country.tsv` and `mydata/country.json`: ```tsv alpha2 name population DK Denmark 5900000 NO Norway 5500000 SE Sweden 10500000 ``` ```json { "format": "{name} ({alpha2})", "rows": "country.tsv", "key": "alpha2", "name": "name", "weight": "population" } ``` ```sh fejkdata -d ./mydata country # Sweden (SE), about half the time fejkdata -d ./mydata country.alpha2 # NO fejkdata -d ./mydata 'country[SE]' # Sweden (SE) fejkdata -d ./mydata 'country[Norway].alpha2' # NO fejkdata -d ./mydata --format csv country # alpha2,name,population → SE,Sweden,10500000 ``` `rows` names the TSV beside the category file; `key` names the column a path selects a row by, `name` a column it also selects by, `weight` a column of positive numbers that skews the draw, and `parent` the table a column links to ([Linked tables](#linked-tables)). The format's `{tokens}` read the columns, and the columns are the [record](#records)'s columns, so `--format csv` writes the rows and `--list` shows `country.alpha2`. A cell is a string node: `1{digits(2)} {digits(2)}` in a cell draws digits and `{/misc.uuid}` reads a reference, while `{name}` in a cell is refused, since a cell has no sibling. Each cell may select its own row of another table, `{/misc.currency[SEK].symbol}` on one row and `{/misc.currency[EUR].symbol}` on the next, since only one row renders. `New` proves the header, the options and every cell token, and refuses a TSV no category names, a key that is empty or repeats, a weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`, `"` or `|`, which a selector cannot spell; `New` maps the keys, and a name or parent column is mapped on the first draw that selects by name or descends through the table. A `name` needs a `key`, since a name naming several rows is reported by their keys, or a `parent`, inside whose row a name names one row, so `first-name[Kim]` is settled by the `sex` selected before it and a name repeating inside one parent row is refused; a name spelling another row's key is refused, since the key would select first and the name never. The table's options are its own — `rows`, `key`, `name`, `weight` and `parent` — so a column may be named `name`, as one usually is. A choice of templates sharing one format and one set of string fields is a table written by hand, and `New` refuses it in a data file naming the TSV to write; an inline template has no file beside it, so there it stays a choice. ### Row selection `[key]` or `[name]` after a table's name selects one row: `misc.territory[SE]` and `misc.territory[Sweden]` name one row, and `misc.territory[SE].capital` reads its column. A name naming several rows is an error listing their keys, unless a row selected before it settles which ([Linked tables](#linked-tables)). A selector is part of the path, so it works wherever a path does: `Fake`, `FakeRecord`, a `{/misc.territory[SE].capital}` reference and a struct tag. A dot inside the brackets belongs to the key or name, so `city[St. Louis]` selects it. A path starts with a name, and `[` still opens a JSON array at the start of a CLI argument, so `'[SE]'` alone names nothing. ### Linked tables A table's `parent` names a column and, by the same name, the table beside it that the column links to by key. Save `mydata/city.tsv` and `mydata/city.json` beside the `country` table above: ```tsv name country population Copenhagen DK 660000 Göteborg SE 600000 Oslo NO 710000 Stockholm SE 990000 ``` ```json { "format": "{name}", "rows": "city.tsv", "key": "name", "parent": "country", "weight": "population" } ``` ```sh fejkdata -d ./mydata 'country[SE].city' # Stockholm or Göteborg fejkdata -d ./mydata country.city.name # a country drawn, then a city inside it fejkdata -d ./mydata 'city[Oslo].country' # NO — the link column's cell fejkdata -d ./mydata '{/city.name}, {/country.name}' # Oslo, Norway — one consistent draw ``` A path descends from a row to a linked table by name, at any depth, and `--list` advertises each direct step. Within one render and [draw group](#draw-group), linked tables agree: the first table a reference path reads pins its ancestors, and a descendant read after it is drawn inside them, so `{/city.name}` and `{/country.name}` are a city and its country whichever is read first. A selected row pins the render the same way, so every reference path into one family of linked tables in one render and group selects the same rows: one that selects none beside one that does is refused naming the spelling that does, `{/country[SE].city.name}` beside `{/country[SE].name}`, and two selecting different rows are refused naming a `drawGroup` to draw them apart in. A bare `{/city}` beside a path into its family is refused too, since a bare reference draws each time, and so is a path that draws a table another path in the group selects a row of, `{/city.name}` beside `{/country[SE].name}`, which the first token rendered would otherwise decide. `New` also refuses a link cell that is no key of the parent, a parent row no child links to, a chain of parents that closes, a table named like a column of any table above it, and a cell or format of a table that references a table of its own family, through any template, a `repeat` or a `drawGroup` included, since a bare read of the referencing table draws its row without pinning it, so the family would draw apart from that row: read the family from a template beside it, or add the value as a column. A cell's reads are the render's too, weighed against every read that renders with them. A reference renders one row of a table, so reads in cells of different rows of it, or of different rows of their ancestors, never render together, whether `[key]` selects the row or the reference draws it. ### Options and fields `format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options; **any other key is a field** (see [Decisions](docs/decisions.md#options-and-fields-share-one-namespace)), and `rows` makes a category a [table](#table), so no template carries a field of that name. A [name](#names) shares the namespace too. An object that does nothing a string can't — only a `format` — is rejected naming the string, as is a one-item choice naming its item. ### Functions A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad count, range, country or expression fails fast; an integer is written plain (`5`, not `+5` or `05`); bounds are finite; a sample that could only ever emit one value (`int(5,5)`, `float(1,1,2)`) is rejected naming the text to write instead; and a length, count or decimal place beyond a sane maximum is rejected, so a fat-fingered `hex(2000000000)` never tries to allocate gigabytes. Every builtin draws only from the seed — a time-based id takes its timestamp from the rng, not the clock — so seeded output stays reproducible. | Function | Kind | Renders | |----------|------|---------| | `{luhn()}` | derivation | Luhn check digit over the digits emitted so far in this expansion | | `{mod11()}` | derivation | weighted mod-11 check char (weights 2–7 from the right); `X` for 10 | | `{ean()}` | derivation | EAN-13 / UPC-A / ISBN-13 / GTIN check digit | | `{digits(n)}` | sample | `n` digits 0–9 | | `{upper(n)}`, `{lower(n)}` | sample | `n` letters A–Z, a–z | | `{int(min,max)}` | sample | uniform integer in `[min, max]` | | `{float(min,max,dp)}` | sample | number in `[min, max]` with `dp` decimals | | `{hex(n)}` | sample | `n` lowercase hex digits | | `{base64(n)}` | sample | `n` random bytes, base64 | | `{uuid()}` | sample | UUID v7 (`misc.uuid` ships v4 as data) | | `{ulid()}` | sample | ULID, 26 Crockford base32 chars | | `{nanoid(n)}` | sample | URL-safe Nano ID, `n` chars | | `{iban(CC)}` | sample | length- and mod-97-valid IBAN for BE, DE, DK, ES, FI, NO or SE | | `{date(from,to,'layout')}` | sample | a second between two `YYYY-MM-DD` days, both included, in a quoted Go layout: `'2006-01-02'`, `'January 2, 2006'`, `'060102'`, `'2006-01-02T15:04:05Z'` | | `{time('layout')}` | sample | a second within a day: `'15:04'`, `'3:04 PM'` | | `{seq()}`, `{seq(name)}` | counter | next integer from 1 in this generator; `name` selects an independent counter | | `{calc(expr)}`, `{calc(expr,dp)}` | computation | an arithmetic expression over sibling fields ([Computation](#computation)) | | `{lowercase(x)}`, `{uppercase(x)}`, `{ascii(x)}` | transform | a field's value rewritten ([Transforms](#transforms)) | A derivation reads what is to its left, so place it after its payload; the buffer is per expansion, so a nested template keeps fixed parts out of the sum. A Swedish personnummer is a Luhn checksum over the nine digits before it, six of them a birthdate: ```json { "format": "{date(1930-01-01,2010-12-31,'060102')}-{birth}{luhn()}", "birth": ["238", "239"] } ``` Renders e.g. `811218-2389`. A layout is Go's: the reference time `Mon Jan 2 15:04:05 MST 2006` spelled as the output should look, quoted, since a layout may carry the comma that separates arguments, with English names. Every second between the two days is reachable, so a layout with a clock draws the time too, and `from` may equal `to`, which is that one day. The instant is UTC, so a zone in the layout prints `UTC` or `Z`. The quotes delimit a layout outside a selector only, so `[O'Fallon]` in an argument stays a name. Rejected at `New`: a bound that is no calendar date, or not before the other; an unquoted layout, naming the single-quoted one; a layout naming no field, which is text, as is one day in a layout with no clock; for `date` a layout naming no date field, naming `time`; and for `time` a layout naming a date field, naming `date`. `{seq()}` spans `Fake` calls and `repeat`, resets with a new generator, and is the natural primary key for the SQL example above. ### Computation `{calc(expr)}` evaluates `+ - * /`, parentheses and unary minus over number literals and sibling field names, each rendered then read as a number; a second argument rounds to that many decimals. A hyphen is always subtraction, so a hyphenated field can't be an operand. ```json { "format": "{net} x {qty} = {calc(net * qty, 2)}", "net": ["19.99", "5.00"], "qty": ["3", "7"] } ``` Renders e.g. `19.99 x 3 = 59.97`. A result that rounds to zero prints unsigned — `0`, `0.00` — as `{float()}`'s does. An operand that can never be a number (`"abc"`, or a choice of such) is rejected at load, as is a division by a constant zero (`1/0`, or a fixed `"0"` field); an operand that sometimes is not a number yields `NaN`, and a division by one that is not constant `Inf` — both print rather than fail, except in a [typed column](#datatype), which must prove neither happens. ### Transforms `{lowercase(x)}`, `{uppercase(x)}` and `{ascii(x)}` rewrite the value of `x` — a field, a path, a `..path` or a [name](#names) — and nest. `ascii` folds Latin letters (`Åsa Öberg` → `Asa Oberg`) and drops any other non-ASCII rune. `x` is held ([One draw, one spelling](#one-draw-one-spelling)), so an email built from a name matches the name beside it: ```json { "format": "{p.first} {p.last} <{lowercase(ascii(p.first))}.{lowercase(ascii(p.last))}@example.com>", "p": [ { "format": "{first} {last}", "first": "Åsa", "last": "Öberg" }, { "format": "{first} {last}", "first": "Bo", "last": "Ek" } ] } ``` Renders `Åsa Öberg ` or `Bo Ek `, never a mix. ### References A reference renders a node from elsewhere in the data — the path `Fake` takes, across every loaded source — so one category borrows another. The sigil says where the path starts, as in a filesystem: `{/en_US.person}` from the data root, `{.username}` from the folder this file sits in, `{..username}` from the folder above. `data/sv_SE/email.json` can therefore read its own locale's `username` without naming `sv_SE`: ```json "Hej, {/en_US.person}!" ``` Renders e.g. `Hej, Pat Smith!`. A reference path into a category is held like a [correlated](#correlated-fields) path, but for the whole render — one `Fake`, or one record — rather than one format: `{.person.first} {.person.last}` name one person, as do the same two references in sibling fields or a nested template, and `{lowercase(.person.first)}` reads that same draw. Each `repeat` iteration is a render of its own, in no group, so it draws anew, and a [draw group](#draw-group) holds a draw apart. A bare reference names no field and makes its own picks each time — `{/misc.uuid} {/misc.uuid}` is two draws — while the reference paths inside what it renders still read the render's draws. Rejected at `New`: a path that is unknown, names a folder, has no folder above, reads a field not every variant of a choice carries, or names the category the reference sits in, and a reference that leads back to its own value, directly, mutually or through a chain. ### Names `{ref as n}` binds one pick of a reference to the name `n` and prints nothing. `{n}` then prints that pick, and `{n.path}` reads a path through it, so every read of `n` describes one row: ```json "{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}" ``` Renders e.g. `🇳🇴 Norway: capital Oslo, calling code +47`. A name keeps what each `{n.path}` lands on and every level it passes, so `{n}` and `{n.path}` agree, and so does a [transform](#transforms) of `n.path`. Everything else under the name draws as it would anywhere: if `n`'s category has the format `{w}-{w}`, it draws `w` twice, and `{n.w}` beside `{n}` is refused, since it could read either draw. A name is a pick of its own: `{t.name}` and `{/misc.territory.name}` beside it are two draws. A reference path the picked category reads is still held for the render as usual, so two names bound to `sv_SE.person`, whose fields read `{.first-name.name}` and `{.last-name.name}`, print the same first and last name unless their reads sit in two [draw groups](#draw-group). A name is drawn on its first read, and lives in the category binding it: any field of the category may read it, and a record's columns read one pick. Each render of the category picks anew, and so does each bare reference to it. A name a `repeat` binds picks again on every iteration, and one bound outside the repeat keeps its pick on every line. Refused at `New`, each error naming what to write instead: - a name bound twice outside any `repeat`, twice in one `repeat`, or both inside a `repeat` and outside it; - a name that is a field or an option; - a binding of anything but a reference; - a binding in a choice's item, which every other item would leave unbound; - a binding nothing reads, or a binding of a whole category read once whole where it is bound, which the bare reference spells: `{/word as w}{w}` is `{/word}`; - a path through a name that selects a row; - `{n}` beside `{n.w}` where `n`'s category renders `w` twice, as in `{w}-{w}`; - reads of a name in two draw groups, or inside a `repeat` nested where the name is bound, where what it names reads a reference path, which each draw group and iteration draws apart; - a binding in a table's format or cell. ### Draw group A template may carry `drawGroup` to hold its reference draws apart: every reference path it renders, however deep short of a `repeat` or a nested `drawGroup`, reads the draw of that group, and the templates of one category naming one group in one render read one draw. A name is local to its category, so a category another one references never joins its groups by name; the unnamed group spans them all. ```json { "format": "{payer} pays {payee}; signed {signature}", "payer": { "format": "{/sv_SE.person.first} {/sv_SE.person.last}", "drawGroup": "payer" }, "payee": "{/sv_SE.person.first} {/sv_SE.person.last}", "signature": { "format": "{/sv_SE.person.last}", "drawGroup": "payer" } } ``` Renders e.g. `Sara Eriksson pays Ebba Lind; signed Eriksson`: the signature reads the payer's draw, while the payee is drawn apart. Rejected at load, each naming nothing: a `drawGroup` of `""` (the default); one naming the draw group its template already draws in; one on a template that renders no reference path — a bare reference to a [table](#table) counts, since the group answers for the family it draws in — short of a `repeat` or a nested `drawGroup`, on a `repeat` itself — each iteration renders in no draw group — or on an inline template's root, which nothing references. So is a path reading into a level that carries a `drawGroup`. ### Correlated fields `{name.field}` reads a path into a sibling, which holds the sibling to one draw ([One draw, one spelling](#one-draw-one-spelling)), so several tokens read one row — a locality and the postal code that really covers it: ```json { "format": "{street} {int(1,99)}\n{place.postal-code} {place.locality}", "street": ["Kungsgatan", "Storgatan"], "place": [ { "format": "{locality}", "locality": "Stockholm", "postal-code": "1{digits(2)} {digits(2)}", "weight": 975 }, { "format": "{locality}", "locality": "Tranås", "postal-code": "573 {digits(2)}", "weight": 18 } ] } ``` Renders e.g. `Kungsgatan 35` / `176 99 Stockholm`, never a Stockholm code beside Tranås; each row's `weight` says how often it appears. A path is held at every level it passes through: `{p.geo.town.name} {p.geo.town.zip}` share the town. Every variant of a choice on the path must carry the rest of it, so a row missing a field is named at load: ```text token {place.postal-code}: field "place": not every variant of this 2-way choice carries "postal-code"; all carry [locality] ``` The sub-fields stay addressable — `Fake("address.place.locality")` renders, and `List` advertises it. A path may not read into a level carrying a `repeat`. ### One draw, one spelling A name any token reads as a path (`{p.first}`) or as an operand (`{calc(net * 2)}`, `{uppercase(w)}`) is drawn **once per expansion**, a reference path (`{/cat.p.first}`) **once per render** in its [draw group](#draw-group), and every other route to either — a bare `{p}`, a second bare `{w}`, `{/cat.net}`, a nested template rendering `{/cat.p.last}` beside `{p.first}`, a bare `{/cat}` beside `{/cat.p.first}`, at any depth — is a load error naming the spelling to use. A name nothing reads that way is drawn each time: `{word} {word}` differs. An expansion is one render of one format, so each nested template draws its own names again; a render is one `Fake` or one record, and each `repeat` iteration is an expansion and a render of its own. ```text token {p} renders a level that {p.first} reads a path into; name the fields you want instead token {w} is repeated, and uppercase operand "w" holds "w" to one draw per expansion; write {w} once ``` ### Performance The target is about a microsecond per value (goal 13). With only the shipped set, `New` loads nothing: a category loads on the first `Fake`, `FakeRecord`, `FakeStruct` or template that reads it, with every category it reads and its whole table family. Beside `--data-path`, `WithDataPath` or `WithDataFS`, every category loads in `New`. Each file is parsed, validated and weight-indexed once, bar a table's name and parent columns, which are mapped on the first `Fake` that selects by name or descends through it. Proving the draw fences adds a pass over the loaded tree per fence, and walks what a render reads only where data holds a reference, so a set that holds none pays for the passes alone. A `Fake` call then costs about what its output costs: an unweighted pick is O(1) whatever the list's length, a weighted one O(log n), and long formats, deep nesting and many tokens add cost in proportion to the output. ## Versioning Semver tags on `main`, `v0.1.0` first; one version covers the shipped data, the library and the CLI, and [`CHANGELOG.md`](CHANGELOG.md) names what each release changed. A consumer's data, code and scripts keep working across a minor or a patch. A minor only adds, and a major is the only release that changes what exists. Seeded output is the exception: any release, a patch too, may change what a seed draws. Where you compare seeded output, install one exact version (`go install github.com/larvit/fejkdata/cmd/fejkdata@vX.Y.Z`; `go.mod` already pins one), and regenerate seeded fixtures and expected values when you raise that version. | Surface | Major | Minor | |---------|-------|-------| | Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row | | Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category | | Data format | a fence: a spelling `New` rejects that it accepted; a template option, since it reserves a field name | a builtin | | CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format | | Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option | A patch changes no row of this table: performance, docs, or a fix inside a promised behaviour that adds or removes no value and changes no path, format or spelling. An error's wording may improve in a minor; the path, rejected spelling and replacement it names may not. Before `v1.0.0` a minor is the breaking unit: `0.(x+1).0` may carry a major's changes, each named in the changelog, and a `0.x.y` patch may not. `v1.0.0` is cut once the shipped data is in its record shape and one full minor has shipped with no breaking change. From `v2` the module path carries `/vN`, so fences ship batched into as few majors as possible. [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) pins every path, each template category's format, the categories each category reads, each column's datatype and nullability, and each table's key, name, weight and parent columns; a pull request that changes it or `data/` adds its `CHANGELOG.md` entry, which CI checks. A removed, renamed or retyped line is a major. ## Audience App developers writing tests and fixtures, in Go and at a shell: - a **bulk fixture author**, thousands of rows into CSV or SQL - a **Go test author**, filling a struct with `FakeStruct` - a **hand fixture author**, one value at a shell - a **validator-facing author**, who needs a value a real checker accepts - a **data author**, who writes categories under `--data-path` and meets every load error the fences raise - a **custom-CLI builder**, who ships a CLI like `fejkdata` carrying modules of their own and a **contributor**, who reads [`todo.md`](todo.md), [`AGENTS.md`](AGENTS.md) and the Development section below, and who ships a register the six above then draw from. ## Goals Where the project is heading; the sections before Audience document what ships today. 1. **The best fake-data tool there is, and the one developers reach for.** 1. You can install it the way you install your other tools, and people hear about it where developers read. 2. **Enough data ships built in to fill a real test database: the CLI carries all of it, a library imports what it needs, and nothing is fetched at run time.** 1. It covers the ten most-spoken languages and the Nordic countries. 2. Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates. 3. A library gets only the data and the module functions it is handed. 3. **The code is easy to understand.** 1. You can go from a bug to the code behind it without asking anyone. 2. You can change one part without keeping the rest in your head. 3. Every name says what the thing really is. 4. Each file makes sense on its own, without another document open beside it. 4. **Every value is valid.** It passes the same check the real system would apply, such as the checksum in a Swedish personal number. 1. Values that belong together come from the same random pick. You choose which ones by giving that pick a name. 5. **Templates are easy to write and read.** A template is the text you write to describe a value, such as `{first} {last}`. 1. Everything outside `{…}` prints exactly as written. 2. Each `{…}` makes a new random pick. 3. A name keeps one pick. `{/person as p}` picks a person and prints nothing. `{p}` then prints that person, and `{p.last}` their last name. 4. A path works like a folder path: `.` goes down a level, and `..` goes up one. 5. `[SE]` selects one row of a table, and means that row wherever you write it. 6. If two people could read a template differently, loading it fails, and the error shows how to write each meaning. 7. Each result has one way to write it. Any other way fails to load, and the error shows the right one. 8. When a clearer name turns up, it replaces the old one, even if that needs a new major version. 6. **Mistakes are caught when the data loads, and the error says plainly what is wrong.** 1. A template that could render a value its own rules forbid, such as `NaN` or `Inf`, fails to load instead. 2. Data that reads something nobody provided fails to load, and the error names the module that provides it by default, or says that none does. 7. **One command gets you a value.** 1. Flags work like in other command-line tools, and can go anywhere on the line. 2. Your first template of your own needs no escaping and no options. 3. The CLI fills in what you leave out with a sane default: all the shipped data, the system's randomness and today's date. 8. **Data is JSON files and TSV tables. Code is only for what they can't express.** 9. **Flexible and extensible.** 1. Data and functions come in modules, from fejkdata or anyone else, and fejkdata's modules load as anyone else's do. 2. A module names the modules it reads by default, and anything providing the same paths can stand in for one. 3. Loading two things under one name fails, in the library and the CLI alike, unless you say the second replaces the first, and then every read of that name reads the second. 4. The builtins ship with fejkdata itself, so a template means the same in every program. 5. A function loaded from a module can ask for the input it needs, and gets the randomness and the date the generator uses. 6. A CLI carrying your own modules is a few lines of Go, and works just like `fejkdata`. 10. **The same seed, date, data and versions always give the same output.** 1. In the library, the caller supplies everything that can change the output: data, randomness and the date. 11. **It depends on nothing but Go's standard library.** 12. **The docs are a map of the template language.** 1. Every feature has its own heading. 2. Every example runs as a test and shows its output. 3. Each rule is stated in one place. 13. **It is fast enough that you never notice it with the shipped data.** 1. A render's cost grows only with the data it reaches. 14. **Every fact comes from an official or open source,** such as a government register. 1. A table holds every row its source holds, and nothing added by hand. 15. **The most common data comes first,** such as names, addresses, phone numbers and dates. 16. **Values look real by default. You can also ask for values that can never reach anything real,** such as a domain nobody can register. ## Layout ``` doc.go the package doc, and the vocabulary the package is written in fejkdata.go Generator, New, options, List shipped.go the embedded data set, and loading a shipped category on the first call reaching it shippedindex.go the index of shipped categories, generated from data/ node.go the node model and JSON -> node compilation, with its fence on a nested drawGroup table.go tables: the rows TSV, its options and links, row selection and draws name.go names: {ref as n} bindings, the scopes they live in, and the reads they answer namefence.go the load fences over names pick.go a render's named picks: the frames of the name scopes rendering, and the reads kept under a name path.go the dotted-path walks with their selectors, and proving a path resolves render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make record.go records: Record, the JSON/CSV/SQL serializers, and their entry points struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link format.go the {token} grammar: scanning, tokens, operands, validation, compiling a format draws.go the memo a hold, draw group or named pick keeps its draws in; and a render's draws and the scope it reads its reference paths and names through holdfence.go the load fences over the hold: the routes and spellings that read one held name two ways drawfence.go the load fences over reference draws and their refusals: a drawGroup's, a render's and a record's columns judged on what they read, and a table reaching its own family reads.go the load-time fold of what a render reads, and what a path reads of a table family pins.go the pin set: the table rows a render fixes, which the draw fences replay reference.go reference sigils, and resolving references across the tree graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences builtins.go the {name()} function registry and its implementations layout.go date and time layouts: the instants one is proved against, and the two samples checksum.go the check characters a derivation appends, and the IBAN they sit inside transform.go the builtins that rewrite an operand's value, and the ASCII folding calc.go the {calc()} arithmetic evaluator: parser, eval, validation datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype valueproof.go the value proof: what a typed column, calc operand or struct field holds, checked at load data.go the load path: fs.FS folders/files -> merged namespace tree, and the bind pipeline every entry point runs cmd/fejkdata/ the fejkdata CLI data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) docs/ the decision log, the register research behind the shipped data, and the survey of other libraries release-tooling/ the release CI publishes from the changelog heading testdata/ the pinned shipped shape (see Versioning) ``` ## Development Everything runs in Docker — **no local tooling beyond Docker is needed**. Source is bind-mounted; build caches persist in the `gocache` volume. ```sh docker compose run --rm test # go test -race docker compose run --rm cover # tests with coverage docker compose run --rm bench # benchmarks docker compose run --rm build # compile the library docker compose run --rm vet # go vet docker compose run --rm cyclo # cyclomatic complexity over 14 (test files excluded) docker compose run --rm dev # interactive shell ``` Commands that rewrite source keep your file ownership when run with `--user`: ```sh docker compose run --rm --user "$(id -u):$(id -g)" fmt # gofmt -w . docker compose run --rm --user "$(id -u):$(id -g)" generate # go generate: shippedindex.go from data/ docker compose run --rm --user "$(id -u):$(id -g)" tidy # go mod tidy ``` A `--user` command answering `permission denied` on `/cache` has met root-owned files in the `gocache` volume: Docker creates the volume root-owned, and every command here that omits `--user` writes into it as root. Hand it back, and again whenever it recurs: ```sh docker compose run --rm --user root --entrypoint chown test -R "$(id -u):$(id -g)" /cache ``` Every pull request runs `docker build .` against both the latest and the lowest supported Go, and must pass before it can be merged — unless it changes none of the files the build and its tests read, nor the workflow itself, in which case it's skipped. That build is the whole gate but the changelog check, which CI runs against the PR base — vet, complexity, format check and tests — so run it locally before pushing. The lowest Go builds the `portable` stage, all of that bar the format check: ```sh docker build . # latest docker build --build-arg GO_VERSION=1.22.12 --target portable . # lowest supported GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build ``` Goal 3 is scored by a panel of simulated readers, and [`AGENTS.md`](AGENTS.md) says what the score asks of a pull request. Goal 5 is checked against simulated template writers from the Audience. A change to the shipped data runs `generate` above first and commits [`shippedindex.go`](shippedindex.go) with the change, then re-pins [`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) in its own commit: ```sh REPIN=1 docker compose run --rm --user "$(id -u):$(id -g)" test ``` A shipped table built from a source is rebuilt by its script under [`data-import/`](data-import), one command per dataset, fetching the source named in [`DATA-LICENSES.md`](DATA-LICENSES.md). Downloads are cached under `data-import/cache/`, which is ignored by version control and grows past a gigabyte, so pass `--cache DIR` to reuse a copy you already have and delete it to fetch afresh; `geo-us.py` fetches two TIGER/Line files per county it ships, a few hundred megabytes, `geo-se.py` needs a Trafikverket API key, free at [data.trafikverket.se](https://data.trafikverket.se/), in `TRAFIKVERKET_API_KEY` or a `--key-file`, and the Census host behind `geo-us.py` and `names-us.py` rejects a client for a while after a burst, so `--surnames` takes a copy of the surname file: ```sh docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/currency.py docker compose run --rm --user "$(id -u):$(id -g)" -e TRAFIKVERKET_API_KEY data-import data-import/geo-se.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/geo-us.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/httpmethod.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/httpstatus.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/language.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/loglevel.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/mimetype.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-se.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-us.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/port.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/protocol.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/territory.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/timezone.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/tld.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/useragent.py ``` To release, head `CHANGELOG.md` with the version's section in place of `Unreleased` and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publishes the GitHub release with the section as its body. A top heading of `[Unreleased]` publishes nothing. ## License MIT — see [LICENSE](LICENSE). Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).