fejkdata
Locale-aware fake data for tests and fixtures, generated from JSON templates. Use it as a Go library or the CLI — no data on disk, no dependencies, and a seed makes output reproducible.
go install github.com/larvit/fejkdata/cmd/fejkdata@latest
echo '{/sv_SE.person}' | fejkdata # Sara Eriksson
CLI
echo '{/sv_SE.person}' | fejkdata # Sara Eriksson
echo '{/sv_SE.person.last}' | fejkdata # Eriksson
echo '{/sv_SE.person} hihi' | fejkdata # Sara Eriksson hihi
echo '{/sv_SE.address}' | fejkdata --seed 42 # the same address every run
echo '{/sv_SE.word}' | fejkdata -n 3 # nät, barn and sol, a line each
printf '%s' '{/sv_SE.word}' | fejkdata -n 3 --separator ', ' # nät, barn, sol
fejkdata --list # every path the data offers
echo '{/misc.territory[SE].capital}' | fejkdata # Stockholm — a table's row, selected by key or name
echo '{/geo.SE.locality[Lund].street}' | fejkdata # Fjelievägen — a linked table, drawn inside the row
echo '{/sv_SE.word}' | fejkdata --data-path ./mydata # layer a directory over the shipped data
fejkdata --no-shipped-data -d ./mydata --list # only your data
echo '{"format":"name: {x}","x":["bosse","lina"]}' | fejkdata # name: bosse or name: lina
fejkdata < template.txt # a template kept in a file
fejkdata <<'EOF' # born 2003-11-27 — a quoted heredoc passes ', $ and \ as written
born {date(1990-01-01,2010-12-31,'2006-01-02')}
EOF
fejkdata renders the template on its stdin: a format string, or a JSON object, array, string or null. Its {…} tokens reach the data by reference from the root,
{/sv_SE.person.last}, so shipped and --data-path categories are alike available,
and everything outside them prints as written. A reference names a category, or a field
inside one: each dot segment descends one level — folders, then the category (a JSON
file), then fields — and [SE] after a table selects its row. --list prints
these paths; a template reads one as {/path}.
Under --format text, the default, what you send is what renders, and fejkdata adds nothing. A format string keeps every
byte, so echo, a file or a heredoc ends each render with its newline, and printf '%s'
sends none; -n joins renders with --separator, empty by default. A JSON object, array
or string drops the whitespace around it, and null alone renders nothing; 42 or true
alone is a format string, so it prints as written, newline included. An empty or blank stdin is a format string like
any other, and renders as sent. A JSON template that should end
each render in a newline writes \n in its format, from a quoted heredoc or a file. A
lone reference, one {/…} and nothing else, is the record it names under --format, one
newline after it allowed. Some shells' echo reads a backslash as an escape, so pipe a
template holding one from a quoted heredoc or a file. The rest of stdin's rules are under
Decisions.
| Flag | |
|---|---|
-d, --data-path D |
a directory to layer over the shipped data; repeatable, the last wins a name clash |
--no-shipped-data |
load only the --data-path directories |
-s, --seed N |
same seed, version and data: identical output (default: a random seed) |
-n, --repeat N |
render the template N times (0 to 1048576), each an independent draw, streamed; 0 prints nothing, [] under --format json, or the header under --format csv |
--separator S |
between repeated renders (default empty); refused with a record --format, which has its own separators |
--format F |
text (default), json, ndjson, csv or sql — a record's columns, one record per row (json frames them as an array) |
--table T |
the INSERT target for --format sql, ignored under any other (default: a lone reference's last segment, such as users for {/users}, else records) |
--list |
print every path, then exit; --repeat, --separator, --format and --table do nothing here |
--version, -h, --help |
print, then exit |
--name value and --name=value both work, a short flag's value attaches or
follows (-n3, -n 3) and short flags bundle (-hn 3) — see
Decisions; flags go anywhere, -- ends them. Exit codes: 0 success, 1 runtime error (data
that fails to load, a missing -d folder included, a lone reference to nothing under --format), 2 misuse — a bad flag, an argument
other than a flag, an unreadable stdin, nothing piped in, or a
template that does not compile. From a checkout:
go run ./cmd/fejkdata ….
Your own data
A category is a JSON file in a directory. Save this as mydata/sql.json:
{
"format": "INSERT INTO users VALUES({sql-username});",
"sql-username": {
"format": "'{username}'",
"repeat": 3,
"separator": "),(",
"username": ["pixelfox", "snork", "turbohund", "blip", "zoom", "wahoo"]
}
}
echo '{/sql}' | fejkdata --seed 1 --data-path ./mydata
# INSERT INTO users VALUES('zoom'),('wahoo'),('blip');
echo '{/sql}' | fejkdata --repeat 100 --data-path ./mydata > seed.sql
That format string is the free-form spelling — you hand-write the whole row.
For structured output a record writes the row for you.
Records
A record is a template seen as columns: its fields are the columns, its format
the whole. --format json|ndjson|csv|sql writes the records; the library's
FakeRecord (below) hands back the columns. A column is a string unless it declares a
datatype, and a null item draws it as null. Save
mydata/users.json:
{
"format": "{first} {last}",
"first": ["Ada", "Bo"],
"last": ["Lovelace", "Ek"]
}
echo '{/users}' | fejkdata --seed 1 -d ./mydata --format json # [{"first":"Bo","last":"Lovelace"}]
echo '{/users}' | fejkdata --seed 1 -d ./mydata --format ndjson # {"first":"Bo","last":"Lovelace"}
echo '{/users}' | fejkdata --seed 1 -d ./mydata --format csv # first,last → Bo,Lovelace
echo '{/users}' | fejkdata --seed 1 -d ./mydata --format sql # INSERT INTO "users" ("first", "last") VALUES ('Bo', 'Lovelace');
echo '{/users}' | fejkdata --seed 1 -d ./mydata --format sql --table people # INSERT into another table
--repeat streams that many records — json frames them as one array document,
ndjson writes one object per line, csv a row after a header, sql one INSERT
per line. Only a category-level template is a record, or a one-item choice holding one; a
field, a choice of more items or a folder errors, and so does a repeat on
the template itself, which composes the format into one string rather than
projecting columns — ask for more records with --repeat. A repeat on a column
is fine.
Columns are written in name order, whatever order the fields appear in. A category
whose fields are the parts of one value — sv_SE.price, sv_SE.version, misc.uuid —
projects those parts rather than the value, so for one column holding what Fake
renders, write {"format":"","price":"{/sv_SE.price}"} as a fieldless category asks
for.
A column carrying a newline keeps it inside the quoted CSV field or the SQL string literal, so a row can span physical lines: read the stream with a CSV or SQL parser rather than splitting it on newlines.
The SQL is ANSI — identifiers in double quotes, a literal quote doubled (''),
backslashes passed through — so a hyphenated field like postal-code stays a
valid identifier. PostgreSQL and SQLite take it as written; MySQL and MariaDB need
ANSI_QUOTES and NO_BACKSLASH_ESCAPES set first, or they read "users" as a
string and a backslash as an escape.
A record written only to emit columns still needs a format — the grammar's one
required key — so "format": "" carries the fields with an inert format: Fake renders
it as nothing.
A record is one render, so its columns read one pick of each name its category
binds: "code": "{/currency as c}{c.code}" beside "symbol": "{c.symbol}" is one
currency, while {/currency.code} and {/currency.symbol} in two columns may name two. A
column the format binds to a name, as {place as p} binds place, holds that same pick, so it
agrees with the columns reading p; a column bound only in part, {place.town as t}, draws
afresh, so bind it whole and read {p.town}.
A reference into a category's own fields, {/users.first} inside users, is a draw of its
own, apart from the fields beside it; to share one draw, read the sibling as a field, or bind
it to a name and read the name
(Decisions).
Data
The shipped data comes in modules, each a Go package in the folder under data/
its namespace spells, a dot a slash: one per locale (data/en_US, data/sv_SE), one per
country's tree of places (data/geo/SE, data/geo/US), and the locale-neutral
data/misc. A package's Go name is its namespace in lower case without separators:
svse, enus, geose, geous, misc. Each embeds its files, so nothing is read from
disk. The CLI carries every module, and a library loads those it passes to WithDataFS
(Library). A directory is a namespace: each JSON file is a category named
after the file, each subdirectory a dot-path segment, so mydata/sv_SE/person.json is
sv_SE.person and replaces the shipped one.
Sources merge in order; matching folders combine, any other clash is won by the
last loaded. Names may not use ., |, (, {, }, [, ], ", / or as,
which binds a name, nor be -, which a struct tag reserves. A dot-prefixed
entry is never a category or a folder, so a data directory can also be a checkout, though
a category may name a hidden rows file beside it.
A source whose root holds .fejkdata.json with an index loads each category on the
first call reaching it, so New does not check them: load each category once in a test,
as CI does for the shipped set. Any other source loads in New, with the indexed
categories it reads. An indexed category reading one that another source replaced loads
on its own first reach, and fails there if the replacement breaks it. The index names
every category of its source, so a category file it leaves out never loads.
The index maps each category's dot path to an entry: parent, the table it links to,
which needs its own entry in the same folder; and paths, every path List advertises
below the category, "" for the category itself. Each shipped module's manifest, such as
data/sv_SE/.fejkdata.json, carries one, so in a copy you
edit, delete its index and keep its namespace, or keep the index current.
A table whose parent column differs from its entry's parent fails at first reach, and a
table linking under another source's table needs a source without an index. //go:embed
of a directory leaves the manifest out, and the source then loads whole in New and at
the root, with no namespace; name the manifest in the pattern too.
A manifest's namespace is the dot path its source's root sits under: a folder holding
person.json with "namespace": "sv_SE" provides sv_SE.person, its subfolders adding
segments below that. It needs no index. Each shipped module carries one, so
data/sv_SE/person.json is sv_SE.person.
Each locale carries address, color, company, date, email, first-name,
ip, last-name, person, phone, price, sentence, sex, time, url,
username, version and word, formatted per locale; sv_SE adds personnummer
and samordningsnummer, en_US adds ssn, itin and phone-exchange (the three
digits after the area code). en_US.phone draws its exchange from en_US.phone-exchange,
which draws evenly from the codes the North American Numbering Plan Administration (NANPA)
may assign: 200 to 999, except 211, 311 … 911, 555, 950, 958, 959 and
976. misc carries car, coordinate, creditcard
(Luhn-valid), currency (ISO 4217), datetime (RFC 3339), emoji, httpmethod,
httpstatus, language (ISO 639-1, with its 639-2/T code), loglevel (syslog
severity), mac, mimetype, objectid, port, protocol, territory (ISO
3166-1), timezone (IANA), tld (IANA root zone), useragent and uuid (v4). Many
carry sub-fields — misc.currency.symbol, misc.territory.alpha2,
misc.httpstatus.code — which --list shows. car, currency, httpmethod,
httpstatus, language, loglevel, mimetype, port, protocol, territory,
timezone, tld and useragent are tables, so
misc.territory[SE].capital and misc.currency[Euro].symbol select a row; car and
useragent carry no key or name, so they are drawn from rather than selected in.
Where a script under data-import/ reads a source, the facts come from
that source, and the script drops a row only by a rule it states; elsewhere a small hand-written
set ships, and todo.md carries the step that replaces it (goal 14). Copy
that states no fact, such as word and sentence, stays hand-written. Where a category
has no path that draws only values reaching nothing real (goal 16), write a fixed value such
as example.com into the template; todo.md plans the path.
misc.httpmethod, misc.protocol and misc.port are IANA's registries.
misc.httpmethod is the eight methods RFC 9110 defines and PATCH; the register's
other entries, WebDAV and DeltaV among them, do not ship. Its safe and
idempotent are true or false, so a Go bool reads them, though --format json
and sql write them as text, a table column carrying no datatype. misc.protocol
carries a number and a name, the keyword itself where the register spells none out,
and selects by either spelling.
misc.port is the TCP assignments, rendering the number a port field holds with the
IANA service beside it, and selecting by number alone. A draw spans the whole
register, so pin misc.port[443] where a fixture needs a port a reader recognises;
it stops at 49150, IANA assigning nothing above, so an ephemeral source port is
{int(49152,65535)}.
misc.tld is the root zone's delegated TLDs, keyed with the leading dot
misc.territory.tld already carries, .se. Its type is the register's —
country-code, generic, generic-restricted, infrastructure or sponsored — and
its unicode the form the register displays, .рф for .xn--p1ai and the key itself
for an ASCII one, which selects the row too, so misc.tld[.рф] renders .xn--p1ai. A
draw spans the whole zone, .arpa and .zuerich alike, so pin misc.tld[.com] where
a fixture needs one a reader recognises.
misc.loglevel is the eight syslog severities, rendering the keyword a configuration
writes, err and info, keyed by the numerical code a PRI encodes and selectable by
either, so misc.loglevel[3] and misc.loglevel[err] are one row.
misc.loglevel.severity is the name RFC 5424 spells, Error beside err; it
selects nothing, the code and the keyword do.
DATA-LICENSES.md names each table's source and licence.
misc.timezone is every zone tzdb gives a shipped territory, from one apiece for most
of them to dozens for the largest, weighted by the population GeoNames records in each,
so misc.territory[US].timezone draws America/New_York far more often than
America/Nome. It links to misc.territory, so misc.territory[SE].timezone is
Europe/Stockholm and a drawn territory and zone agree. Its offset is the zone's
standard offset, so do not pair it with a drawn misc.datetime: in a zone that
observes DST it is the wrong one half the year, which is what storing a zone name
avoids. A zone tzdb named recently — Europe/Kyiv, America/Ciudad_Juarez — is
rejected outright by a consumer resolving it against older tzdata, so where the
consumer validates the zone, pin one rather than draw it. There is no UTC row, tzdb
giving that name no territory — spell it as the text "UTC". misc.useragent carries
browser, device and os beside the string, and misc.car a make and a model,
until an international source replaces them.
ISO 3166-1 codes territories, not sovereign states, so that is what the table is
called: Greenland and Åland have codes of their own, and misc.territory.country
names the state each belongs to — DK for Greenland, FI for Åland, and its own
code for a sovereign one, or for one the register names no state for.
sex, first-name and last-name are tables weighted by bearers, from SCB, the
SSA and the Census Bureau. first-name links to sex, so sv_SE.sex[f].first-name
draws a woman's name, and a name both sexes carry is a row under each, so
en_US.sex[m].first-name[Taylor] names the one a first-name[Taylor] alone cannot.
person binds one first name and reads its sex through it, {.first-name as n} and
{n..sex.name}, so its first and sex columns agree. en_US.title links to sex
too, so a person's prefix never contradicts it. A personnummer's birth number,
sv_SE.birth-number under sex, is Skatteverket's test series, 238 for a woman and 239
for a man, which no real person is ever given.
A person of a chosen sex is assembled from the tables — sex[f].first-name beside
last-name. person and personnummer draw apart, so where a name and a number must
agree, read both through one bound sex, {/sv_SE.sex[f] as s} for a woman:
{ "format": "{first} {last}, {pnr}",
"first": "{/sv_SE.sex as s}{s.first-name}",
"last": "{/sv_SE.last-name}",
"pnr": "{date(1930-01-01,2025-12-31,'060102')}-{s.birth-number.number}{luhn()}" }
Renders e.g. Anne Grönberg, 011012-2389.
Skatteverket's test series is small: a personnummer is one of about 70,000 values, a day
in 1930–2025 against the two birth numbers, so a fixture past a few hundred rows repeats
one and a UNIQUE column needs a category of your own. sv_SE.date and en_US.date
are uniform over 1970-01-01 to 2029-12-31, misc.datetime over 2000-01-01 to 2029-12-31.
What the two locales do not share: en_US.address carries a region column the Swedish
one has no use for, and sv_SE.title has no parent, so it is selected as
sv_SE.title[dr] rather than inside a sex.
The geo namespace holds one tree per country under its alpha-2 code: five
linked tables named alike, and an address record binding one locality
and drawing its street and postal code inside it. The locale's address prints that
record, and each of its columns describes the address printed. Do not assert that a Swedish
street lies in its postort: the import counts each road segment toward the shipped postort
owning the postal code whose centre point lies nearest, so a street may sit under a
neighbouring one.
| Table | geo.SE |
geo.US |
Weight |
|---|---|---|---|
region |
län, by code or name | state, by USPS abbreviation or name; code is the FIPS code |
population |
municipality |
kommun, by code or name | county, by FIPS code or name | population |
locality |
postort, by name; geo-se.py's localities picks the candidates, and one with no street is dropped |
incorporated place, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population |
postal-code |
postnummer with street delivery, by code | ZCTA, by code | one; address ranges |
street |
gatunamn, every one geo-se.py's street_name accepts |
street name, per place the quarter with most address ranges, at least ten | segments; address ranges |
geo.SE.region[Skåne län].municipality draws a kommun in Skåne,
geo.SE.locality[Lund].street a street in Lund, and
geo.US.region[IL].locality[Springfield] settles which Springfield. The US places come
from the incorporated places that hold most of some postal code's land: a tenth of them,
drawn with odds by population, every state capital but Honolulu, and every place of
100,000 or more. A smaller city may be missing, so look one up in
locality.tsv before selecting it. A region row
carries its timezone, the state's predominant zone, and a locality its lat and
lon. What ports across countries is the five table names, the name column,
selection by name, and the address record's columns street, street-number,
postal-code and locality; every other column is the country's own, code on a
Swedish region but abbr on a US one.
Library
go get github.com/larvit/fejkdata # Go 1.22+, 64-bit targets
import (
"github.com/larvit/fejkdata"
svse "github.com/larvit/fejkdata/data/sv_SE"
)
f, err := fejkdata.New(fejkdata.WithDataFS(svse.Modules()...), fejkdata.WithSeed(42))
if err != nil {
log.Fatal(err)
}
v, err := f.Fake("sv_SE.address") // "Trekungagatan 43\n442 42 Kungälv"
paths := f.List() // the paths Fake renders, sorted; direct descents only
v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call
t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once
v = t.Fake() // render many times, no re-parse
r, err := f.FakeRecord("users") // one record: each field a column
s := r.JSON() // {"first":"Ada","last":"Lovelace"}
r, err = f.FakeRecordTemplate(`{"format":"{x}","x":["a","b"]}`) // compile + render inline
rt, err := f.NewRecordTemplate(`{"format":"","id":{"format":"{seq(users)}","datatype":"integer"}}`) // compile once
r = rt.Fake() // render a row; loop this for bulk output
err = f.FakeStruct(&user) // fill a struct's fake:"…" tagged fields
ok, err := fejkdata.IsTemplate(arg) // an inline template by its shape, else a path
| Option | |
|---|---|
WithSeed(n) |
required; the same seed, version and data give the same sequence. For a new seed each run, set n := rand.Uint64() (math/rand/v2), and log n to replay the run |
WithDataPath(dir) |
layer a directory; repeat to layer several, the last wins a clash |
WithDataFS(modules...) |
layer modules in order, each an fs.FS: a shipped package's Modules()..., or your own embed.FS; WithDataFS() loads none |
A shipped package's Modules() returns its own FS and every module it reads, such as a
locale's country's places; misc.* comes from misc.Modules(), in
github.com/larvit/fejkdata/data/misc. Your program links only the packages it imports.
data.Modules(), from github.com/larvit/fejkdata/data, is every shipped module, and
links all of them. Two locales, with enus from github.com/larvit/fejkdata/data/en_US,
are WithDataFS(append(svse.Modules(), enus.Modules()...)...). To stand in your
own places, pass the locale's FS alone beside a tree providing geo.SE.address, a record
with a format and the columns sv_SE.address reads, street, street-number,
postal-code and locality: WithDataFS(svse.FS, myPlaces). New fails where no
WithDataFS or WithDataPath names the data to load.
New refuses data that breaks the grammar, could mean two things or cannot render a valid
value, and NewTemplate refuses such a template. A category of an indexed source is
checked on the first call reaching it instead. Data that fails to load, in New or on
that first call, gives an error matching ErrLoad. On a loaded generator:
Fakefails only for a path that names nothing, could name two things or reads one draw of a level carrying arepeat, or for data that fails to load (ErrLoad), with the same error every call.FakeStructfails only for a non-struct argument, a type its tags do not describe, or data that fails to load (ErrLoad), with the same error every call.Template.Fakecannot fail.
A path, wherever a call takes one, may start with one /, as a reference does.
A *Record carries its columns via Columns() — each a Column of Name,
DataType, rendered Value and Null — and serializes them with JSON()
(one object), CSVHeader()/CSVLine(), or SQLInsert(table) — the shapes the
CLI's --format writes, each without a trailing newline. FakeRecord and
FakeRecordTemplate take a record; a path or template that is not one — a bare string, a
choice of more than one item, or a folder — errors.
type User struct {
ID int64 `fake:"{seq()}"`
Last string `fake:"sv_SE.person.last"`
Age uint8 `fake:"{int(18,99)}"`
Nick *string `fake:"[null, \"{/sv_SE.username}\"]"`
Home Address // filled from Address's own tags
}
type Address struct {
Street string `fake:"{/sv_SE.address as a}{a.street} {a.street-number}"`
PostalCode string `fake:"{a.postal-code}"` // the same address's code
Locality string `fake:"{a.locality}"`
}
FakeStruct fills a struct through a pointer: each exported field tagged fake:"…" is
a column of one record, its tag a path or an inline template — told apart by
IsTemplate — and its Go type the column's
datatype: a string, bool, integer or float kind, or a pointer to one. A
column that can be null needs a pointer, which a null item leaves nil. An
integer stays within int64 whatever its kind, and a value the kind cannot hold, such as
{int(0,300)} in a uint8, is refused naming a kind that holds it. The tags are a record's columns, so they share its
names: First tagged {/sv_SE.person as p}{p.first} and Sex tagged {p.sex}
describe one person, while two path tags into sv_SE.person are two draws. The fields an
embedded struct promotes are columns of the same record; a named struct field, or a
pointer to one, fills from its own tags as a record of its own, so its names and picks are
its own. fake:"-" leaves a field unfilled.
Untagged fields keep their values, and so does a pointer back to a struct already being filled; a type
whose fields reach more than 1024 structs is refused, naming fake:"-" to cut it. The
first call for a type compiles its tags and reports what they get wrong, with the same
error on every later call; a datatype in a tag other than the one its Go type sets is
refused, naming that type.
A *Generator is safe for concurrent use; a sequence is reproducible only
when drawn from one goroutine. Editing a template shifts what it draws, and
everything drawn after it.
Your own CLI
Package cli is the fejkdata command, and its Main takes the modules the command
loads, so a CLI carrying your own data is a main package of your own:
package main
import (
"embed"
"os"
"github.com/larvit/fejkdata/cli"
"github.com/larvit/fejkdata/data"
)
//go:embed *.json
var mydata embed.FS
func main() { os.Exit(cli.Main(append(data.Modules(), mydata)...)) }
The embed's folders are the paths: //go:embed acme serves acme/customer.json as
{/acme.customer}. Add *.tsv to the embed where your data holds a table. The CLI loads
its data only when it runs. To catch broken data at go test, call
fejkdata.New(fejkdata.WithDataFS(...), fejkdata.WithSeed(1)) over the same modules, and
render each path once: a module with an index checks a category only when a call first
reaches it. The CLI takes fejkdata's flags and exit codes, and names itself by the file
it runs from in its help and errors. --no-shipped-data leaves out the modules passed to
Main, and --version prints your module's version beside the fejkdata version it is built with.
Data format
Every value is a node, nestable without limit:
| Node | JSON | Renders |
|---|---|---|
| string | "Malmö" |
its text, with any {…} tokens expanded |
| choice | ["a", "b", …] |
one item, picked at random |
| template | {"format": "…", …} |
its format, with {name} tokens rendering the named fields |
| table | {"format": "…", "rows": "x.tsv", …} |
its format over one row of the TSV beside it (Table) |
| number | 5, 1.50 |
its text as written |
| boolean | true |
its text as written |
In a record column a bare number or boolean is refused, since it could be the text "5"
or the typed value {"format": "5", "datatype": "integer"}. In a struct tag, the
field's Go type decides.
Format string
Every character is literal except a {…} token:
| Token | Renders |
|---|---|
{name} |
the sibling field name, or the pick bound to the name (Names) |
{x as name} |
nothing: binds one pick of x, a reference, a sibling field or a path into one, to name (Names) |
{name.field} |
field of a fresh draw of the sibling name, or of the pick bound to it (Correlated fields) |
{a|b} |
one of the named fields, even odds |
{fn(args)} |
a builtin (Functions) |
{/path}, {.path}, {..path} |
a node reached from the data root, this file's folder, or the folder above (References) |
{{, }} |
a literal { or } |
Each {…} draws afresh, so {w} {w}, {p.a} {p.b} and {/person.first} {/person.last}
are two draws each. Only a name keeps one pick for every read of it.
"100 Main St, Apt {int(1,9)}{upper(1)} — tel {digits(3)}-{digits(4)}"
Renders e.g. 100 Main St, Apt 4B — tel 555-0199. Random characters come from
the sample functions, so a phone pattern is 070-{digits(3)} {digits(2)} {digits(2)}.
A lone } is a load error naming }}. A repeated arm counts as written, so
{a|a|b} draws a two times in three.
Weight
An item of a choice may carry a weight (default 1) to skew its odds:
[
{ "format": "070-{digits(3)} {digits(2)} {digits(2)}", "weight": 10 },
"08-{digits(3)} {digits(2)} {digits(2)}",
{ "format": "010-{digits(3)} {digits(2)} {digits(2)}", "weight": 2 }
]
Renders 070-412 38 91 ten times as often as 08-…. A string item is weighted by
writing it as { "format": "AB", "weight": 3 }. A repeated item counts as written,
so ["a", "a", "b"] draws a two times in three, and an item of weight 0 is never
drawn. A weight outside a choice does nothing, and so does one above 0 on the item of
a one-item choice. Rejected at load: a weight that is negative, not a number, or too
close to 0 to tell from it, such as 1e-400; and a choice whose every weight is 0, a one-item choice of weight 0 among them.
Repeat
A template may carry repeat (an integer of 0 or more, default 1; 0 renders nothing) to render its format
that many times — each an independent draw — joined by separator (default ""):
{ "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] }
Renders e.g. bar foo baz. A separator without a repeat above 1 does nothing.
Rejected at load: a repeat that multiplies to more than 1 048 576 renders along any
path of nested repeats.
Datatype
A record column may declare datatype — integer, number or boolean — so json
writes 42 rather than "42" and sql a bare literal; a column without one, or with
string, is a string:
{ "format": "",
"id": { "format": "{seq()}", "datatype": "integer" },
"paid": { "format": "{p}", "p": ["true", "false"], "datatype": "boolean" },
"total": { "format": "{calc(net * qty, 2)}", "net": ["19.99", "5.00"], "qty": ["3", "7"], "datatype": "number" } }
Writes e.g. {"id":1,"paid":true,"total":59.97}. A column is a field of the top-level
template, or an item of a choice standing in for one; datatype anywhere else does
nothing. A typed column holds one value, alone in its format: a
literal, one {int()}, {float()}, {seq()} or {calc()} call, or a read that lands only on such
values. integer is an int64 written -?(0|[1-9][0-9]*) — {float()} prints one at
0 decimals within int64 — number a JSON number, boolean true or false. A value its
datatype cannot hold is a load error naming it:
order.id: datatype integer: {digits(3)} prints text, not an integer
order.id: datatype integer: "1{digits(2)}" is not one value; write one literal or one {float()}, {int()}, {seq()} or {calc()}, or read one
A column whose format is one read of a record's column and nothing else —
"score": "{/src.score}", "score": "{s.score}" with s bound to /src, or "a": "{p}"
beside {place as p} in its own record — is that column: it takes the column's datatype and
is null where the column is, and a struct field tagged src.score is nil there. A
datatype of its own types the column's values where they prove it, and "string"
reads them as text. Any other read renders the column's text, a null as "".
A typed column's {calc()} must be proven to print a number: each operand a number
literal, an {int()}, {float()}, {seq()} or {digits()} call, a calc, or a read of
such values, whose bounds keep every divisor from zero and the result within 1e300.
What the bounds cannot show is refused — {calc(a / b)}: divides by b, which is not proven nonzero. The calc fills an integer column at 0 decimals, or over whole
operands with no /, while its bounds stay within int64.
Null
A null item draws a record column as null: json writes null, sql NULL, and
csv an empty field, with an empty string written "" — the convention PostgreSQL's
COPY … CSV reads. A record of one null column is a blank line, which COPY reads as
null but most CSV readers skip, so write such a record as json or sql. Fake
renders a null as "". The other items' weights skew its odds:
{ "format": "", "deleted_at": null, "middle": [null, { "format": "{n}", "n": ["Ann", "Eva"], "weight": 3 }] }
deleted_at is null every draw, middle a name three draws in four. Anywhere but a
column, null renders "". Rejected at load: a column whose items hold different
datatypes.
Table
A table is a category whose rows come from a TSV beside its JSON file: the header
names the columns, each line below it is one row, and the format renders the row
drawn. Save mydata/country.tsv and mydata/country.json:
alpha2 name population
DK Denmark 5900000
NO Norway 5500000
SE Sweden 10500000
{ "format": "{name} ({alpha2})", "rows": "country.tsv", "key": "alpha2", "name": "name", "weight": "population" }
echo '{/country}' | fejkdata -d ./mydata # Sweden (SE), about half the time
echo '{/country.alpha2}' | fejkdata -d ./mydata # NO
echo '{/country[SE]}' | fejkdata -d ./mydata # Sweden (SE)
echo '{/country[Norway].alpha2}' | fejkdata -d ./mydata # NO
echo '{/country}' | fejkdata -d ./mydata --format csv # alpha2,name,population → SE,Sweden,10500000
rows names the TSV beside the category file; key names the column a path
selects a row by, name a column it also selects by, weight a column of numbers
of 0 or more that skews the draw, and parent the table a column links to
(Linked tables). The format's {tokens} read the columns, and the
columns are the record's columns, so --format csv writes the rows and
--list shows country.alpha2. A cell is a string node: 1{digits(2)} {digits(2)}
in a cell draws digits and {/misc.uuid} reads a reference, while {name} in a cell
is refused, since a cell has no sibling. Each cell may select its own row of another
table, {/misc.currency[SEK].symbol} on one row and {/misc.currency[EUR].symbol}
on the next, since only one row renders. New proves the header, the options and every
cell token, and refuses:
- a JSON member other than
format,rows,key,name,weightandparent, which could be meant as a field or as an option a table does not take; - a key cell that is empty or repeats;
- a weight that is negative, not a number, or too close to
0to tell from it; - every weight
0, across the table or inside one parent row; - a key or name holding
[,],{,},"or|, which a selector cannot spell.
New maps the keys, and a name or parent column is mapped on the first draw that selects
by name or descends through the table. A name needs a key, since a name naming several rows is reported by
their keys, or a parent, inside whose row a name names one row, so first-name[Kim]
is settled by the sex selected before it and a name repeating inside one parent row
is refused; a name spelling another row's key is refused, since the key would
select first and the name never. A table's options are its own, so a column may be named
name, as one usually is.
Row selection
[key] or [name] after a table's name selects one row: misc.territory[SE] and
misc.territory[Sweden] name one row, and misc.territory[SE].capital reads its column.
A name naming several rows is an error listing their keys, unless a row selected
before it settles which (Linked tables). A selector is part of the path, so it works
wherever a path does: Fake, FakeRecord, a {/misc.territory[SE].capital} reference
and a struct tag. A dot inside the brackets belongs to the key or name, so
city[St. Louis] selects it.
Linked tables
A table's parent names a column and, by the same name, the table beside it that the
column links to by key. Save mydata/city.tsv and mydata/city.json beside the
country table above:
name country population
Copenhagen DK 660000
Göteborg SE 600000
Oslo NO 710000
Stockholm SE 990000
{ "format": "{name}", "rows": "city.tsv", "key": "name", "parent": "country", "weight": "population" }
echo '{/country[SE].city}' | fejkdata -d ./mydata # Stockholm or Göteborg
echo '{/country.city.name}' | fejkdata -d ./mydata # a country drawn, then a city inside it
echo '{/city[Oslo].country}' | fejkdata -d ./mydata # NO — the link column's cell
echo '{/city as c}{c.name}, {c..country.name}' | fejkdata -d ./mydata # Oslo, Norway — one city and its country
A path descends from a row to a linked table by name, at any depth, and --list
advertises each direct step. One path reads one chain of rows: country.city.name draws a
country by its weights, then a city inside it by theirs. Two paths are two draws, so
{/city.name}, {/country.name} may name a city and another country. The reads of one
name agree instead, and the table it binds decides whose weights govern:
{/city as c} draws a city by city weight and {c..country.name} is its country, while
{/country as k} draws a country by country weight and {k.city.name} a city inside it.
A name keeps every row its reads draw, so {k.city.name} and {k.city.population} describe
one city, and {k.name} its country.
New refuses a link cell that is no key of the parent, a parent row no child links to, a
chain of parents that closes, and a table named like a column of any table above it.
Step up
.. after a row steps up to the row its link column names. Name the parent table right
after the ..:
echo '{/city[Oslo]..country.name}' | fejkdata -d ./mydata # Norway
echo '{/city..country}' | fejkdata -d ./mydata # a city drawn, then its country: Sweden (SE)
echo '{/city[Göteborg]..country.city}' | fejkdata -d ./mydata # Stockholm or Göteborg, drawn afresh
echo '{/city as c}{c}, {c..country.name}' | fejkdata -d ./mydata # Oslo, Norway
It steps up from a selected row, a drawn row or a name bound to a row, one
level per ..: geo.SE.locality..municipality..region.name. .country after a row
still reads the link column's cell, NO.
After a .., . steps down again and draws afresh inside the row stepped up to, so
city[Göteborg]..country.city may draw Göteborg again. Under a name, the paths stepping
down from one .. read one draw: {c..country.city.name} and
{c..country.city.population} describe one city.
A .. that names anything but the parent table is an error naming the parent. A selector
after a .. is an error naming the table to select from, since it could name a row outside
the one stepped up to. --list advertises no path with a ... At the start of a
reference, .. is the folder above (References).
Options and fields
format, weight, repeat, separator and datatype are the only options;
any other key is a field (see Decisions), and rows makes a category
a table, so no template carries a field of that name. A name shares the
namespace too.
Functions
A {name(args)} token calls a builtin. Arguments are checked at New: a bad
count, range, country or expression fails fast; an integer may carry a sign or leading
zeros (+5 and 05 are 5); a count of 0 renders nothing; bounds are finite; and a length, count or decimal place beyond a sane maximum is rejected, so a
fat-fingered hex(2000000000) never tries to allocate gigabytes. Every builtin draws only from the seed — a
time-based id takes its timestamp from the rng, not the clock — so seeded output
stays reproducible.
| Function | Kind | Renders |
|---|---|---|
{luhn()} |
derivation | Luhn check digit over the digits emitted so far in this expansion |
{mod11()} |
derivation | weighted mod-11 check char (weights 2–7 from the right); X for 10 |
{ean()} |
derivation | EAN-13 / UPC-A / ISBN-13 / GTIN check digit |
{digits(n)} |
sample | n digits 0–9 |
{upper(n)}, {lower(n)} |
sample | n letters A–Z, a–z |
{int(min,max)} |
sample | uniform integer in [min, max] |
{float(min,max,dp)} |
sample | number in [min, max] with dp decimals |
{hex(n)} |
sample | n lowercase hex digits |
{base64(n)} |
sample | n random bytes, base64 |
{uuid()} |
sample | UUID v7 (misc.uuid ships v4 as data) |
{ulid()} |
sample | ULID, 26 Crockford base32 chars |
{nanoid(n)} |
sample | URL-safe Nano ID, n chars |
{iban(CC)} |
sample | length- and mod-97-valid IBAN for BE, DE, DK, ES, FI, NO or SE |
{date(from,to,'layout')} |
sample | a second between two YYYY-MM-DD days, both included, in a quoted Go layout: '2006-01-02', 'January 2, 2006', '060102', '2006-01-02T15:04:05Z' |
{time('layout')} |
sample | a second within a day: '15:04', '3:04 PM' |
{seq()}, {seq(name)} |
counter | next integer from 1 in this generator; name selects an independent counter |
{calc(expr)}, {calc(expr,dp)} |
computation | an arithmetic expression over sibling fields and names (Computation) |
{lowercase(x)}, {uppercase(x)}, {ascii(x)} |
transform | a field's value rewritten (Transforms) |
A derivation reads what is to its left, so place it after its payload; the buffer is per expansion, so a nested template keeps fixed parts out of the sum. A Swedish personnummer is a Luhn checksum over the nine digits before it, six of them a birthdate:
{ "format": "{date(1930-01-01,2010-12-31,'060102')}-{birth}{luhn()}", "birth": ["238", "239"] }
Renders e.g. 811218-2389. A layout is Go's: the reference time Mon Jan 2 15:04:05 MST 2006 spelled as the output should look, quoted, since a layout may
carry the comma that separates arguments, with English names. Every second
between the two days is reachable, so a layout with a clock draws the time too, and
from may equal to, which is that one day. The instant is UTC, so a zone in the
layout prints UTC or Z.
The quotes delimit a layout outside a selector only, so [O'Fallon] in an
argument stays a name. A layout naming no field prints itself, date with a clock-only
layout prints the drawn instant's clock, and date over one day with no clock field prints
that day.
Rejected at New: a bound that is no calendar date, or after the other; an unquoted
or double-quoted layout, naming the single-quoted one; and time with a layout naming a
date field, naming date, since time draws no date. {seq()} spans Fake
calls and repeat, resets with a new generator, and is the natural primary key for
the SQL example above; give each record its own counter, {seq(orders)}, where one
generator fills several.
Computation
{calc(expr)} evaluates + - * /, parentheses and unary minus over number
literals, sibling fields and names, each rendered then read as a number;
a second argument rounds to that many decimals. A hyphen is always subtraction, so a
hyphenated field or name can't be an operand.
A calc computes from the values its format shows where both read one name:
{ "format": "{net as n}{qty as q}{n} x {q} = {calc(n * q, 2)}", "net": ["19.99", "5.00"], "qty": ["3", "7"] }
Renders e.g. 19.99 x 3 = 59.97.
A result that rounds to zero prints unsigned — 0, 0.00 — as {float()}'s does. An
operand that can never be a number ("abc", or a choice of such) is rejected at load, as
is a division by a constant zero (1/0, or a fixed "0" field); an operand that sometimes is not a number yields
NaN, and a division by one that is not constant Inf — both print rather than
fail, except in a typed column, which must prove neither happens.
Transforms
{lowercase(x)}, {uppercase(x)} and {ascii(x)} rewrite the value of x — a
field, a path, a ..path or a name — and nest. ascii folds Latin letters (Åsa Öberg
→ Asa Oberg) and drops any other non-ASCII rune. An email
matches the name beside it where both read one name:
{ "format": "{person as p}{p.first} {p.last} <{lowercase(ascii(p.first))}.{lowercase(ascii(p.last))}@example.com>",
"person": [
{ "format": "{first} {last}", "first": "Åsa", "last": "Öberg" },
{ "format": "{first} {last}", "first": "Bo", "last": "Ek" }
] }
Renders Åsa Öberg <asa.oberg@example.com> or Bo Ek <bo.ek@example.com>, never
a mix.
References
A reference renders a node from elsewhere in the data — the path Fake takes,
across every loaded source — so one category borrows another. The sigil says
where the path starts, as in a filesystem: {/en_US.person} from the data root,
{.username} from the folder this file sits in, {..username} from the folder
above. sv_SE.email can therefore read its own locale's username
without naming sv_SE:
"Hej, {/en_US.person}!"
Renders e.g. Hej, Pat Smith!. Each reference draws afresh (Format string),
so {.person.first} {.person.last} may name two people. Bind the person to a name to read one: {.person as p}{p.first} {p.last}. Rejected at New: a path that is
unknown, names a folder, has no folder above, reads a field not every variant
of a choice carries, and a reference that leads back to its own value, directly,
mutually or through a chain.
Names
{x as n} binds one pick of x, a reference, a sibling field or a path into one, to the name
n and prints nothing. {n} then
prints that pick, and {n.path} reads a path through it, so every read of n describes one
row:
"{/misc.territory as t}{t.flag} {t}: capital {t.capital}, calling code +{t.calling-code}"
Renders e.g. 🇳🇴 Norway: capital Oslo, calling code +47. A name keeps what each {n.path}
lands on and every level it passes, so {n} and {n.path} agree, and so does a
transform of n.path. Everything else under the name draws as it would
anywhere: if n's category has the format {w}-{w}, it draws w twice, and {n.w} beside
{n} is refused, since it could read either draw. A sibling field is bound the same way, so
{place as p}{p.postal-code} {p.locality} reads one place (Correlated fields).
A name is a pick of its own: {t.name} and {/misc.territory.name} beside it are two draws,
and so are two names bound to one category.
A name is drawn on its first read, and lives in the category binding it: any field of the
category may read it, and a record's columns read one pick. Each render of the category picks
anew, and so does each bare reference to it. A name a repeat binds picks again on every
iteration, and one bound outside the repeat keeps its pick on every line. A name a choice's
item binds lives in that item, and picks anew each time the item is drawn. A category a
reference enters does not see the name; bind it in the category that reads it.
Refused at New, each error naming what to write instead:
- a name bound twice outside any
repeat, twice in onerepeat, or both inside arepeatand outside it; - a name that is a field or an option;
- a binding of anything but a reference, or a field of the template binding it or a path into one;
- a read of a name inside the field bound to it: read it outside that field;
- reading a name outside the choice item that binds it, since the other items leave it unbound;
- a path through a name that selects a row:
{r.municipality[0180]}could mean municipality 0180 whatever regionrpicked, or 0180 only whererpicked the region holding it; {n}beside{n.w}wheren's category readswtwice, as in{w}-{w}or{w.a} {w.b};- a binding in a table's format or cell.
Correlated fields
{name.field} reads a path into a sibling. Several tokens read one draw of it through a
name: bind the sibling, and every read of
the name reads that one row — a locality and the postal code that really covers it:
{ "format": "{place as p}{street} {int(1,99)}\n{p.postal-code} {p.locality}",
"street": ["Kungsgatan", "Storgatan"],
"place": [
{ "format": "{locality}", "locality": "Stockholm", "postal-code": "1{digits(2)} {digits(2)}", "weight": 975 },
{ "format": "{locality}", "locality": "Tranås", "postal-code": "573 {digits(2)}", "weight": 18 }
] }
Renders e.g. Kungsgatan 35 / 176 99 Stockholm, never a Stockholm code beside
Tranås; each row's weight says how often it appears. {p.geo.town.name} {p.geo.town.zip} share the town too, as a name keeps every level its reads pass. Every variant of a choice
on the path must carry the rest of it, so a row missing a field is named at load:
token {p.postal-code}: not every variant of this 2-way choice carries "postal-code"; all carry [locality]
The sub-fields stay addressable — Fake("address.place.locality") renders, and
List advertises it. A path may not read into a level carrying a repeat, and List
advertises no such path; read that level whole.
Performance
The target is about a microsecond per value (goal 13).
A source with an index, as the shipped set has, loads nothing in New: a
category loads on the first Fake, FakeRecord, FakeStruct or template that reads
it, with every category it reads and its whole table family. Each file is parsed,
validated and weight-indexed once, bar a table's name and parent columns, which are
mapped on the first Fake that selects by name or descends through it. A Fake call then
costs about what its output costs: an unweighted pick is O(1) whatever the list's length,
a weighted one O(log n), and long formats, deep nesting and many tokens add cost in
proportion to the output.
Versioning
Semver tags on main, v0.1.0 first; one version covers the shipped data, the
library and the CLI, and CHANGELOG.md names what each release
changed. A consumer's data, code and scripts keep working across a minor or a patch. A
minor only adds, and a major is the only release that changes what exists. Seeded output
is the exception: any release, a patch too, may change what a seed draws. Where you
compare seeded output, install one exact version
(go install github.com/larvit/fejkdata/cmd/fejkdata@vX.Y.Z; go.mod already pins one),
and regenerate seeded fixtures and expected values when you raise that version.
| Surface | Major | Minor |
|---|---|---|
| Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row |
| Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category |
| Data format | a fence: a spelling a load rejects that it accepted, in a category or a manifest; a template option, since it reserves a field name | a builtin |
| CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a --format writes (header, quoting, statement shape), the --list layout, or what an error names |
a flag, a format |
| Library | change or remove an exported name; raise the lowest supported Go | an exported name, a With… option |
| Data packages | rename or remove a package; move a category to another package; make a package read another it did not | a package |
A patch changes no row of this table: performance, docs, or a fix inside a promised behaviour that adds or removes no value and changes no path, format or spelling.
An error's wording may improve in a minor; the path, rejected spelling and replacement it names may not.
Before v1.0.0 a minor is the breaking unit: 0.(x+1).0 may carry a major's
changes, each named in the changelog, and a 0.x.y patch may not. v1.0.0 is
cut once the shipped data is in its record shape and one full minor has shipped
with no breaking change. From v2 the module path carries /vN, so fences ship
batched into as few majors as possible.
testdata/shipped_shape.txt pins every path, each
template category's format, the categories each category reads, each column's
datatype and nullability, and each table's key, name, weight and parent columns; a pull request that
changes it or data/ adds its CHANGELOG.md entry, which CI checks. A removed,
renamed or retyped line is a major.
Audience
App developers writing tests and fixtures, in Go and at a shell:
- a bulk fixture author, millions of rows into CSV or SQL
- a Go test author, filling a struct with
FakeStruct - a hand fixture author, a few values at a shell
- a validator-facing author, who needs a value a real checker accepts
- a data author, who writes categories under
--data-pathand meets every load error the fences raise - a custom-CLI builder, who ships a CLI like
fejkdatawith data and functions of their own
and a contributor, who reads todo.md, AGENTS.md and
the Development section below, and who ships a register the six above then draw from.
Goals
Where the project is heading; the sections before Audience document what ships today.
- The best fake-data tool there is, and the one developers reach for.
- You can install it the way you install your other tools, and people hear about it where developers read.
- Enough data ships built in to fill a real test database: the CLI carries all of it, a
library imports what it needs, and nothing is fetched at run time.
- Each locale covers the common categories: names, addresses, phone numbers, ids, companies and dates.
- The shipped data comes in Go packages, and a program links only the packages it imports.
- The code is easy to understand.
- You can go from a bug to the code behind it without asking anyone.
- You can change one part without keeping the rest in your head.
- Every name says what the thing really is.
- Each file makes sense on its own, without another document open beside it.
- Every value is valid. It passes the same check the real system would apply,
such as the checksum in a Swedish personal number.
- Values that belong together come from the same random pick. You choose which ones by giving that pick a name.
- Templates are easy to write and read. A template is the text you write to
describe a value, such as
{first} {last}.- Everything outside
{…}prints exactly as written. - Each
{…}makes a new random pick. - A name keeps one pick.
{/person as p}picks a person and prints nothing.{p}then prints that person, and{p.last}their last name. - A path works like a folder path:
.goes down a level, and..goes up one. [SE]selects one row of a table, and means that row wherever you write it.- If two people could read a template differently, loading it fails, and the error shows how to write each meaning.
- Whatever is well-formed and means one thing loads, even where part of it does nothing. fejkdata does not guess at mistakes.
- When a clearer name turns up, it replaces the old one, even if that needs a new major version.
- Everything fejkdata renders is written as a template: on the command line, in a struct tag and in the library.
- Everything outside
- Errors surface when the data loads, never mid-render, and say plainly what is
wrong.
- A template that could render a value its own rules forbid, such as
NaNorInf, fails to load instead. - Data that reads something nobody provided fails to load, and the error names the module that provides it by default, or says that none does.
- A module from fejkdata is proven whole before it ships; a module from anyone else may load on first reach, and its author proves it.
- A template that could render a value its own rules forbid, such as
- One command line gets you a value.
- Flags work like in other command-line tools, and can go anywhere on the line.
- Your first template of your own needs no escaping and no options.
- The CLI fills in what you leave out with a sane default: all the shipped data, the system's randomness and today's date.
- Human readable and editable data formats — source data is saved in JSON or TSV formats when possible.
- Flexible and extensible.
- Data and functions come in modules, a Go package or a data folder, from fejkdata or anyone else, and fejkdata's load as anyone else's do.
- A module names the modules it reads by default, and anything providing the same paths can stand in for one.
- Loading two things under one name fails, in the library and the CLI alike, unless you say the second replaces the first, and then every read of that name reads the second.
- The builtins ship with fejkdata itself, so a template means the same in every program.
- A function loaded from a module can ask for the input it needs, and gets the randomness and the date the generator uses.
- A CLI carrying your own modules is a few lines of Go, and works just like
fejkdata.
- The same seed, date, data and versions always give the same output.
- In the library, the caller supplies everything that can change the output: data, seed and date.
- No unnecessary runtime dependencies — use standard library by default.
- The docs are a map of the template language.
- Every feature has its own heading.
- Every example runs as a test and shows its output.
- Each rule is stated in one place.
- It is fast enough that you never notice it with the shipped data.
- A render's cost grows only with the data it reaches.
- Strongly prioritize data sourced from good online sources
- The most common data comes first, such as names, addresses, phone numbers and dates.
- Values look real by default. You can also ask for values that can never reach anything real, such as a domain nobody can register.
Layout
doc.go the package doc, and the vocabulary the package is written in
fejkdata.go Generator, New, options, List
index.go a source's index: its categories left unloaded, and loading one on the first call reaching it
node.go the node model and JSON -> node compilation
table.go tables: their options, format and cells, the link to the parent beside them, and how a path passes and draws a table
name.go names: {x as n} bindings, the scopes they live in, the reads they answer, and each binding's target
namefence.go the load fences over names
pick.go a render's named picks: each level's pick key, the frame each name scope renders in, the memo a pick keeps, and the reads kept under it
env.go renderEnv: what a render reads names and rows through, the stack of frames it renders in, and the frames a read opens and finds
path.go the dotted-path walks: the paths List advertises, the sub-paths every variant of a choice carries, proving a path reaches a node, and drawing the path
render.go Fake, the recursive renderer (choices, format strings, expansions) and the reads its tokens make
record.go records: Record, the JSON/CSV/SQL serializers, and their entry points
struct.go structs: FakeStruct, fake tags as one record's columns, and a field's Go type as its column's datatype
inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and resolve
format.go a format's tokens checked against their fields, and compiled into ops
arm.go the compiled read of one {a|b} alternative or operand: its head, the path to its leaf, and the op holding it
reference.go one reference's resolution: its sigil, its category, its tail
resolve.go resolving templates against the assembled tree in steps, each over every template before the next
graph.go the node graph: tree walks, containment and render edges, and the cycle and repeat fences
sorted.go sortedNames: a map's keys, sorted
calc.go the {calc()} operand checks: never a number, and a constant zero divisor
datatype.go column datatypes: DataType, a template's declared datatype, a column's datatype, and a format that only reads a column
valueproof.go the value proof: what a typed column, calc operand or struct field holds, checked at load
data.go the load path: the categories internal/datafiles hands over, compiled into one merged tree, and the load pipeline every entry point runs
cli/ the fejkdata command, as the library call Main
cmd/fejkdata/ the fejkdata binary: Main over every shipped module
internal/builtinfunc/ the {name()} functions: their checks and draws, checksums, date and time layouts, transforms and calc arithmetic
internal/datafiles/ the walk of a data tree: its folders, its category files and the rows files beside them, and its manifest
internal/datatype/ the datatype a record column holds; the root's DataType is an alias of it
internal/drawstate/ the seeded randomness and the {seq()} counters a generator draws through
internal/grammar/ how the template language is written: format tokens, paths and selectors, reference sigils, identifiers, calc syntax, and whether a string is a template or a path
internal/invariant/ the one phrase every package panics with when an invariant breaks
internal/jsonvalue/ a decoded JSON value, its numbers kept as written, the one input a template compiles from, and the kind an error names it by
internal/proven/ what a proof knows of a value, and the bounds a calc takes from its operands
internal/rows/ a table's rows: the TSV, the options proved over it, the links between tables, row selection and draws, and the pin set one path or one named pick fixes
data/ the shipped modules, each a Go package embedding its categories (JSON, and a TSV per table) and its manifest; and Modules, which lists them
data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md), and the modules they share
docs/ the decision log, the register research behind the shipped data, the survey of other libraries, panels' scores and rulings on how readable the code is, and what each chunk cost an agent to read
release-tooling/ the release CI publishes from the changelog heading
testdata/ the pinned shipped shape (see Versioning)
Development
Everything runs in Docker — no local tooling beyond Docker is needed.
Source is bind-mounted; build caches persist in the gocache volume.
docker compose run --rm test # go test -race
docker compose run --rm cover # tests with coverage
docker compose run --rm bench # benchmarks
docker compose run --rm build # compile the library
docker compose run --rm vet # go vet
docker compose run --rm cyclo # cyclomatic complexity over 14 (test files excluded)
docker compose run --rm dev # interactive shell
Commands that rewrite source keep your file ownership when run with --user:
docker compose run --rm --user "$(id -u):$(id -g)" fmt # gofmt -w .
docker compose run --rm --user "$(id -u):$(id -g)" generate # go generate: each data/ module's .fejkdata.json
docker compose run --rm --user "$(id -u):$(id -g)" tidy # go mod tidy
A --user command answering permission denied on /cache has met root-owned
files in the gocache volume: Docker creates the volume root-owned, and every
command here that omits --user writes into it as root. Hand it back, and again
whenever it recurs:
docker compose run --rm --user root --entrypoint chown test -R "$(id -u):$(id -g)" /cache
Every pull request runs docker build . against both the latest and the lowest
supported Go, and must pass before it can be merged — unless it changes none of
the files the build and its tests read, nor the workflow itself, in which case
it's skipped. That build is the whole gate except the changelog check, which CI
runs against the PR base: vet, complexity, the format check, the Go tests and the
release tooling's tests. Run it locally before pushing. The lowest Go builds the
portable stage, which skips the format check and the release tooling's tests:
docker build . # latest
docker build --build-arg GO_VERSION=1.22.12 --target portable . # lowest supported
GO_VERSION=1.22.12 docker compose run --rm test # the same tests, without the image build
Goal 3 is scored by a panel of simulated readers, and AGENTS.md says
what the score asks of a pull request. Goal 5 is checked against simulated template
writers from the Audience.
A change to the shipped data runs generate above first and commits each module's
.fejkdata.json with the change, then re-pins
testdata/shipped_shape.txt in its own commit:
REPIN=1 docker compose run --rm --user "$(id -u):$(id -g)" test
A shipped table built from a source is rebuilt by its script under
data-import/, one command per dataset, fetching the source named in
DATA-LICENSES.md. Downloads are cached under
data-import/cache/, which is ignored by version control and grows past a
gigabyte, so pass --cache DIR to reuse a copy you already have and delete it to
fetch afresh; geo-us.py fetches two
TIGER/Line files per county it ships, about a gigabyte, geo-se.py needs
a Trafikverket API key, free at data.trafikverket.se,
in TRAFIKVERKET_API_KEY or a --key-file, and the Census host behind geo-us.py
and names-us.py rejects a client for a while after a burst, so --surnames takes
a copy of the surname file:
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/currency.py
docker compose run --rm --user "$(id -u):$(id -g)" -e TRAFIKVERKET_API_KEY data-import data-import/geo-se.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/geo-us.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/httpmethod.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/httpstatus.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/language.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/loglevel.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/mimetype.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-se.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-us.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/port.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/protocol.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/territory.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/timezone.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/tld.py
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/useragent.py
To release, head CHANGELOG.md with the version's section in place of Unreleased
and merge: once main passes the gate, CI tags that commit vX.Y.Z and publishes
the GitHub release with the section as its body. A top heading of [Unreleased]
publishes nothing.
License
MIT — see LICENSE. Forked from github.com/Timewave-AB/fakes.