Put the shipped data before the library, state what a record projects and what the locales do not share

CHANGELOG.md

@@ -51,7 +51,11 @@ replacement, and each removed path, column or flag.
   from a `sv_SE.birth-number` table under `sex`, in place of `sv_SE.ssn`, whose
   `ssn.mmdd`, `ssn.mmdd.m` and `ssn.mmdd.d` go with it. `en_US.ssn` now draws the
   ranges the SSA assigns and carries the columns `area`, `group` and `serial`, so
-  `--format csv en_US.ssn` writes a header where it used to fail; `en_US.itin` is new.
+  `--format csv en_US.ssn` writes a header where it used to fail; `en_US.itin` is new
+  and carries the same three. `person.prefix` is null where a person has no title,
+  where it used to be an empty string, so `--format sql` writes `NULL`.
+- `ErrNoColumns` is exported, so a caller can tell the one record fence a path can
+  answer from the rest.
 - An error names a spelling that runs: a layout is named single-quoted and free of
   its own quotes, a row of a table with no key is named as the path that selects it,
   `sv_SE.sex[f].first-name[Kim]`, and a category with no columns names the record

README.md

@@ -33,11 +33,9 @@ a JSON object, array or string, or that carries a `{` token, is instead an
 **inline template**: a format string or a JSON value compiled and rendered on the
 spot. Its tokens reach the data by reference from the root —
 `{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike
-available. An inline template sits in no folder, so the folder-relative `{.name}`
+available. A path never contains a brace or a quote, and a bracket only as a selector
-and `{..name}` are rejected naming the root spelling, and one reference alone —
+after a name, so the two spellings cannot collide; which spellings an inline template
-`{/sv_SE.person}` — is the path written as a template, rejected naming the path, as is
+rejects, and what each names instead, is under [Decisions](#decisions).
-a path written `/sv_SE.person`. A path never contains a brace or a quote, and a
-bracket only as a selector after a name, so the two cannot collide (see [Decisions](#decisions)).
 
 | Flag | |
 |------|--|
@@ -115,6 +113,12 @@ the template itself, which composes the format into one string rather than
 projecting columns — ask for more records with `--repeat`. A `repeat` on a column
 is fine.
 
+Columns are written in name order, whatever order the fields appear in. A category
+whose fields are the parts of one value — `sv_SE.price`, `sv_SE.version`, `misc.uuid` —
+projects those parts rather than the value, so for one column holding what `Fake`
+renders, write `{"format":"","price":"{/sv_SE.price}"}` as a fieldless category asks
+for.
+
 A column carrying a newline keeps it inside the quoted CSV field or the SQL string
 literal, so a row can span physical lines: read the stream with a CSV or SQL
 parser rather than splitting it on newlines.
@@ -142,6 +146,72 @@ a value two fields share in its own category and reference that. A field hold, t
 operand ties fields together within one column as always (see
 [Correlated fields](#correlated-fields) and [Decisions](#decisions)).
 
+## Data
+
+The shipped set under [`data/`](data) — one folder per locale (`en_US`, `sv_SE`)
+plus a locale-neutral `misc` folder — is embedded, so the CLI and the library
+work with no data on disk. A directory is a namespace: each JSON file is a
+category named after the file, each subdirectory a dot-path segment, so
+`mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one.
+Sources merge in order; matching folders combine, any other clash is won by the
+last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`, nor be
+`-`, which a struct tag reserves; dot-prefixed entries are skipped, so a data directory can also be a checkout.
+
+Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
+`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
+`username`, `version` and `word`, formatted per locale; `sv_SE` adds
+`personnummer` and `samordningsnummer`, `en_US` adds `ssn` and `itin`. `misc`
+carries `car`, `coordinate`, `country` (ISO 3166), `creditcard` (Luhn-valid),
+`currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpstatus`, `language`
+(ISO 639), `mac`, `mimetype`, `objectid`, `timezone` (IANA), `useragent` and
+`uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
+`misc.country.alpha2`, `misc.httpstatus.code` — which `--list` shows. `country`,
+`currency`, `httpstatus`, `language` and `mimetype` are [tables](#table), so
+`misc.country[SE].capital` and `misc.currency[Euro].symbol` select a row;
+[`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence.
+
+`sex`, `first-name` and `last-name` are tables weighted by bearers, from SCB, the
+SSA and the Census Bureau. `first-name` links to `sex`, so `sv_SE.sex[f].first-name`
+draws a woman's name, and a name both sexes carry is a row under each, so
+`en_US.sex[m].first-name[Taylor]` names the one a `first-name[Taylor]` alone cannot.
+`person` reads one draw of the three, so its `first` and `sex` columns agree, and so
+does a `personnummer` in the same render: its birth number, `sv_SE.birth-number`
+under `sex`, is Skatteverket's test series, 238 for a woman and 239 for a man, which no
+real person is ever given. `en_US.title` links to `sex` too, so a person's prefix
+never contradicts it.
+
+A person of a chosen sex is assembled from the tables — `sex[f].first-name` beside
+`last-name` — while a shipped `personnummer` agrees with the sex its own render
+*drew*, not with one a path selects. That test series is also small: a personnummer
+is one of about 70,000 values, a day in 1930–2025 against the two birth numbers, so a
+fixture past a few hundred rows repeats one and a `UNIQUE` column needs a category of
+your own. `sv_SE.date` and `en_US.date` are uniform over 1970-01-01 to 2029-12-31,
+`misc.datetime` over 2000-01-01 to 2029-12-31. What the two locales do not share:
+`en_US.address` carries a `region` column the Swedish one has no use for, and
+`sv_SE.title` has no `parent`, so it is selected as `sv_SE.title[dr]` rather than
+inside a sex.
+
+A `geo` folder holds one tree per country under its alpha-2 code: five
+[linked tables](#linked-tables) named alike, and an `address` record over one
+consistent draw of them, which the locale's `address` reads.
+
+| Table | `geo.SE` | `geo.US` | Weight |
+|-------|----------|----------|--------|
+| `region` | län, by code or name | state, by USPS abbreviation or name; `code` is the FIPS code | population |
+| `municipality` | kommun, by code or name | county, by FIPS code or name | population |
+| `locality` | postort, by name | incorporated place of 25,000 people or more with a postal code of its own, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population |
+| `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges |
+| `street` | gatunamn, the ten with most road segments per postort | street name, the ten with most address ranges per place | segments; address ranges |
+
+`geo.SE.region[Skåne län].municipality` draws a kommun in Skåne,
+`geo.SE.locality[Lund].street` a street in Lund, and
+`geo.US.region[IL].locality[Springfield]` settles which Springfield. A region row
+carries its `timezone`, the state's predominant zone, and a locality its `lat` and
+`lon`. What ports across countries is the five table names, the `name` column,
+selection by name, and the `address` record's columns `street`, `street-number`,
+`postal-code` and `locality`; every other column is the country's own, `code` on a
+Swedish region but `abbr` on a US one.
+
 ## Library
 
 ```sh
@@ -207,69 +277,6 @@ A `*Generator` is safe for concurrent use; a seeded sequence is reproducible onl
 when drawn from one goroutine. Changing how a value is composed shifts the seeded
 stream for that value and everything drawn after it.
 
-## Data
-
-The shipped set under [`data/`](data) — one folder per locale (`en_US`, `sv_SE`)
-plus a locale-neutral `misc` folder — is embedded, so the CLI and the library
-work with no data on disk. A directory is a namespace: each JSON file is a
-category named after the file, each subdirectory a dot-path segment, so
-`mydata/sv_SE/person.json` is `sv_SE.person` and replaces the shipped one.
-Sources merge in order; matching folders combine, any other clash is won by the
-last loaded. Names may not use `.`, `|`, `(`, `{`, `}`, `[`, `]`, `"` or `/`, nor be
-`-`, which a struct tag reserves; dot-prefixed entries are skipped, so a data directory can also be a checkout.
-
-Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`,
-`ip`, `last-name`, `person`, `phone`, `price`, `sentence`, `sex`, `time`, `url`,
-`username`, `version` and `word`, formatted per locale; `sv_SE` adds
-`personnummer` and `samordningsnummer`, `en_US` adds `ssn` and `itin`. `misc`
-carries `car`, `coordinate`, `country` (ISO 3166), `creditcard` (Luhn-valid),
-`currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpstatus`, `language`
-(ISO 639), `mac`, `mimetype`, `objectid`, `timezone` (IANA), `useragent` and
-`uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
-`misc.country.alpha2`, `misc.httpstatus.code` — which `--list` shows. `country`,
-`currency`, `httpstatus`, `language` and `mimetype` are [tables](#table), so
-`misc.country[SE].capital` and `misc.currency[Euro].symbol` select a row;
-[`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence.
-
-`sex`, `first-name` and `last-name` are tables weighted by bearers, from SCB, the
-SSA and the Census Bureau. `first-name` links to `sex`, so `sv_SE.sex[f].first-name`
-draws a woman's name, and a name both sexes carry is a row under each, so
-`en_US.sex[m].first-name[Taylor]` names the one a `first-name[Taylor]` alone cannot.
-`person` reads one draw of the three, so its `first` and `sex` columns agree, and so
-does a `personnummer` in the same render: its birth number, `sv_SE.birth-number`
-under `sex`, is Skatteverket's test series, 238 for a woman and 239 for a man, which no
-real person is ever given. `en_US.title` links to `sex` too, so a person's prefix
-never contradicts it.
-
-A person of a chosen sex is assembled from the tables — `sex[f].first-name` beside
-`last-name` — while a shipped `personnummer` agrees with the sex its own render
-*drew*, not with one a path selects. That test series is also small: a personnummer
-is one of about 70,000 values, a day in 1930–2025 against the two birth numbers, so a
-fixture past a few hundred rows repeats one and a `UNIQUE` column needs a category of
-your own. `sv_SE.date` and `en_US.date` are uniform over 1970-01-01 to 2029-12-31,
-`misc.datetime` over 2000-01-01 to 2029-12-31.
-
-A `geo` folder holds one tree per country under its alpha-2 code: five
-[linked tables](#linked-tables) named alike, and an `address` record over one
-consistent draw of them, which the locale's `address` reads.
-
-| Table | `geo.SE` | `geo.US` | Weight |
-|-------|----------|----------|--------|
-| `region` | län, by code or name | state, by USPS abbreviation or name; `code` is the FIPS code | population |
-| `municipality` | kommun, by code or name | county, by FIPS code or name | population |
-| `locality` | postort, by name | incorporated place of 25,000 people or more with a postal code of its own, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population |
-| `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges |
-| `street` | gatunamn, the ten with most road segments per postort | street name, the ten with most address ranges per place | segments; address ranges |
-
-`geo.SE.region[Skåne län].municipality` draws a kommun in Skåne,
-`geo.SE.locality[Lund].street` a street in Lund, and
-`geo.US.region[IL].locality[Springfield]` settles which Springfield. A region row
-carries its `timezone`, the state's predominant zone, and a locality its `lat` and
-`lon`. What ports across countries is the five table names, the `name` column,
-selection by name, and the `address` record's columns `street`, `street-number`,
-`postal-code` and `locality`; every other column is the country's own, `code` on a
-Swedish region but `abbr` on a US one.
-
 ## Data format
 
 Every value is a **node**, nestable without limit:

builtins.go

@@ -225,7 +225,7 @@ func floatArgs(_ map[string]node, a []string) error {
 }
 func seqArg(_ map[string]node, a []string) error {
 	if len(a) > 1 {
-		return fmt.Errorf("seq takes at most one name, got %d args", len(a))
+		return fmt.Errorf("seq takes at most one name, got %d", len(a))
 	}
 	if len(a) == 1 && a[0] == "" {
 		return fmt.Errorf("seq name must not be empty")

cmd/fejkdata/main.go

@@ -282,7 +282,7 @@ func (in invocation) check() (argKind, error) {
 		return argPath, nil
 	}
 	if len(in.paths) != 1 {
-		return argPath, fmt.Errorf("expected one path or template, got %d", len(in.paths))
+		return argPath, fmt.Errorf("expected one path or template, got %d%s", len(in.paths), templateSplit(in.paths))
 	}
 	return classify(in.paths[0])
 }
@@ -478,11 +478,25 @@ func run(args []string, stdout, stderr io.Writer) int {
 			return misuse(stderr, te.error)
 		}
 		fmt.Fprintln(stderr, err)
+		if errors.Is(err, fejkdata.ErrNoColumns) {
+			fmt.Fprintln(stderr, "wrap that JSON in single quotes, which keep a shell from expanding its braces")
+		}
 		return 1
 	}
 	return 0
 }
 
+// templateSplit names the cure for a template a shell split on its spaces, which is
+// what several arguments carrying a token mean.
+func templateSplit(paths []string) string {
+	for _, p := range paths {
+		if strings.Contains(p, "{") {
+			return `; a template's spaces split the argument, so wrap the whole argument in "…"`
+		}
+	}
+	return ""
+}
+
 func misuse(stderr io.Writer, err error) int {
 	// A library error already names the program, so the prefix is not doubled.
 	fmt.Fprintf(stderr, "fejkdata: %s\ntry 'fejkdata --help'\n", strings.TrimPrefix(err.Error(), "fejkdata: "))

data.go

@@ -43,7 +43,7 @@ func loadData(sources []dataSource) (map[string]node, error) {
 			}
 			info, err := os.Stat(src.path)
 			if err != nil {
-				return nil, err
+				return nil, fmt.Errorf("data path %s: %w", src.path, err)
 			}
 			if !info.IsDir() {
 				return nil, fmt.Errorf("%s is not a directory", src.path)

data/en_US/itin.json

@@ -1,5 +1,6 @@
 {
-  "format": "9{digits(2)}-{group}-{serial}",
+  "format": "{area}-{group}-{serial}",
+  "area": "9{digits(2)}",
   "group": [
     { "format": "{int(50,65)}", "weight": 16 },
     { "format": "{int(70,88)}", "weight": 19 },

data/en_US/person.json

@@ -2,6 +2,6 @@
   "format": "{prefix}{first} {last}",
   "first": "{.first-name.name}",
   "last": "{.last-name.name}",
-  "prefix": ["", { "format": "{.title.name} ", "weight": 0.1 }],
+  "prefix": [null, { "format": "{.title.name} ", "weight": 0.1 }],
   "sex": "{.sex.name}"
 }

data/sv_SE/person.json

@@ -2,6 +2,6 @@
   "format": "{prefix}{first} {last}",
   "first": "{.first-name.name}",
   "last": "{.last-name.name}",
-  "prefix": ["", { "format": "{.title.name} ", "weight": 0.05 }],
+  "prefix": [null, { "format": "{.title.name} ", "weight": 0.05 }],
   "sex": "{.sex.name}"
 }

record.go

@@ -140,7 +140,7 @@ func (f *Generator) FakeRecord(path string) (*Record, error) {
 		return nil, fmt.Errorf("fejkdata: %s descends into %q, a field; only a category-level template is a record", path, tail[0])
 	}
 	shape := f.recordShapeOf(n)
-	if errors.Is(shape.err, errNoColumns) {
+	if errors.Is(shape.err, ErrNoColumns) {
 		ns := names(segments)
 		return nil, fmt.Errorf(`fejkdata: %s %w; render it as a column of one: {"format":"","%s":"{/%s}"}`, path, shape.err, ns[len(ns)-1], path)
 	}
@@ -231,9 +231,9 @@ func (f *Generator) FakeRecordTemplate(input string) (*Record, error) {
 	return t.Fake(), nil
 }
 
-// errNoColumns is the one record fence a path can answer, so its entry point names
+// ErrNoColumns is the one record fence a path can answer, so its entry point names
 // the record to write instead.
-var errNoColumns = errors.New("has no fields, so no columns")
+var ErrNoColumns = errors.New("has no fields, so no columns")
 
 // recordOf is the fence both record entry points pass. The columns come back with
 // the template, fixed for every draw the caller goes on to make.
@@ -247,7 +247,7 @@ func recordOf(n node) (*template, []Column, error) {
 	}
 	names := recordColumns(t)
 	if len(names) == 0 {
-		return nil, nil, errNoColumns
+		return nil, nil, ErrNoColumns
 	}
 	if !t.record {
 		return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat)

struct.go

@@ -340,7 +340,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro
 			return fmt.Errorf("%s (%s): %s", label, ft, reason)
 		}
 		if !kind.holds(v) {
-			return fmt.Errorf("%s (%s): %q is not proven within %s; make it %s", label, ft, it.format, elem.Kind(), kind.wider)
+			return fmt.Errorf("%s (%s): %q is not proven within %s; narrow it to that range, or make the field %s", label, ft, it.format, elem.Kind(), kind.wider)
 		}
 	}
 	return nil

struct_test.go

@@ -245,16 +245,16 @@ func TestFakeStructErrors(t *testing.T) {
 		}{}, `"Ada" is not an integer`},
 		{&struct {
 			A int8 `fake:"{int(0,300)}"`
-		}{}, `"{int(0,300)}" is not proven within int8; make it int64`},
+		}{}, `"{int(0,300)}" is not proven within int8; narrow it to that range, or make the field int64`},
 		{&struct {
 			A uint `fake:"{int(-1,5)}"`
-		}{}, `"{int(-1,5)}" is not proven within uint; make it int64`},
+		}{}, `"{int(-1,5)}" is not proven within uint; narrow it to that range, or make the field int64`},
 		{&struct {
 			A float32 `fake:"[\"1\",\"1e39\"]"`
-		}{}, `"1e39" is not proven within float32; make it float64`},
+		}{}, `"1e39" is not proven within float32; narrow it to that range, or make the field float64`},
 		{&struct {
 			A int32 `fake:"{seq()}"`
-		}{}, `"{seq()}" is not proven within int32; make it int64`},
+		}{}, `"{seq()}" is not proven within int32; narrow it to that range, or make the field int64`},
 		{&struct {
 			A bool `fake:"{int(0,1)}"`
 		}{}, "prints an integer, not a boolean"},

testdata/shipped_shape.txt

@@ -18,7 +18,8 @@ en_US.first-name.count	string
 en_US.first-name.name	string
 en_US.first-name.sex	string
 en_US.ip
-en_US.itin	format "9{digits(2)}-{group}-{serial}"
+en_US.itin	format "{area}-{group}-{serial}"
+en_US.itin.area	string
 en_US.itin.group	string
 en_US.itin.serial	string
 en_US.last-name	format "{name}"	key name	weight count
@@ -27,7 +28,7 @@ en_US.last-name.name	string
 en_US.person	format "{prefix}{first} {last}"	reads en_US.first-name en_US.last-name en_US.sex en_US.title
 en_US.person.first	string
 en_US.person.last	string
-en_US.person.prefix	string
+en_US.person.prefix	string null
 en_US.person.sex	string
 en_US.phone
 en_US.phone.area
@@ -282,7 +283,7 @@ sv_SE.last-name.name	string
 sv_SE.person	format "{prefix}{first} {last}"	reads sv_SE.first-name sv_SE.last-name sv_SE.sex sv_SE.title
 sv_SE.person.first	string
 sv_SE.person.last	string
-sv_SE.person.prefix	string
+sv_SE.person.prefix	string null
 sv_SE.person.sex	string
 sv_SE.personnummer	format "{date(1930-01-01,2025-12-31,'060102')}-{.birth-number.number}{luhn()}"	reads sv_SE.birth-number
 sv_SE.phone