Make great documentation goal 1.2, and plan the README's reorganisation and the nested-template gaps #33

Open
lilleman wants to merge 966 commits from worktree-todo-relative-dates-exact-counts into main
9 changed files with 188 additions and 182 deletions
Showing only changes of commit e5babf413d - Show all commits
+3 -3
View File
@@ -39,9 +39,9 @@ In [docs/decisions.md](docs/decisions.md):
- Options and fields share one namespace
- `{a|b}` stays beside nested choices
- Flags follow getopt_long
- A struct tag is a template by its shape, not by a flag
- The CLI renders the template on its stdin, keeping a format string's every byte
- A template that does not compile is misuse (exit 2), including a reference that resolves to nothing, save the CLI's lone reference until item 129
- A struct tag is a template by its shape
- The CLI renders exactly the template on its stdin, and adds nothing
- A template that does not compile is misuse (exit 2), including a reference that resolves to nothing, save a lone reference under the CLI's `--format` until item 129
- A padded JSON template is rejected, not trimmed
- `FakeTemplate` and `NewTemplate` both stay
- The shipped data is embedded, not discovered
+5 -5
View File
@@ -27,11 +27,11 @@ replacement, and each removed path, column or flag.
operand: `{/misc.coordinate.lat as lat}{calc(lat * 60)}`. What `New` refuses is listed
under [Names](README.md#names).
- First release: the CLI, the library and the shipped data set.
- The CLI renders the template on its stdin, `echo '{/sv_SE.person} hihi' | fejkdata`,
and takes no operand. A format string keeps every byte, `echo`'s newline included; a
JSON template or one reference alone drops one newline ending stdin, `\n` or `\r\n`.
One reference alone, `{/users}`, is the record `users` under `--format`. An operand,
or nothing piped in, exits 2.
- The CLI renders exactly the template on its stdin, `echo '{/sv_SE.person} hihi' |
fejkdata`, and adds nothing: `echo`'s newline ends the render, and `--separator`
defaults to empty. Whitespace around a JSON template is dropped. A lone reference,
`{/users}`, is the record `users` under `--format`. An argument other than a flag, an
empty stdin, or nothing piped in, exits 2.
- Table categories: a category JSON naming a `rows` TSV beside it, with the options
`key`, `name`, `weight` and `parent`; a path selects a row by key or name,
`misc.territory[SE]`, and descends to a linked table by name, one path drawing each
+28 -26
View File
@@ -12,19 +12,20 @@ echo '{/sv_SE.person}' | fejkdata # Sara Eriksson
## CLI
```sh
echo '{/sv_SE.person}' | fejkdata # Sara Eriksson
echo '{/sv_SE.person.last}' | fejkdata # Eriksson
echo '{/sv_SE.person} hihi' | fejkdata # Sara Eriksson hihi, then echo's newline
echo '{/sv_SE.address}' | fejkdata --seed 42 # the same address every run
echo '{/sv_SE.word}' | fejkdata -n 3 --separator ', ' # nät, barn, sol
fejkdata --list # every path the data offers
echo '{/misc.territory[SE].capital}' | fejkdata # Stockholm — a table's row, selected by key or name
echo '{/geo.SE.locality[Lund].street}' | fejkdata # Fjelievägen — a linked table, drawn inside the row
echo '{/sv_SE.word}' | fejkdata --data-path ./mydata # layer a directory over the shipped data
fejkdata --no-shipped-data -d ./mydata --list # only your data
echo '{"format":"name: {x}","x":["bosse","lina"]}' | fejkdata # name: bosse or name: lina
fejkdata < template.txt # a template kept in a file
fejkdata <<'EOF' # born 2003-11-27 — a quoted heredoc passes ', $ and \ as written
echo '{/sv_SE.person}' | fejkdata # Sara Eriksson
echo '{/sv_SE.person.last}' | fejkdata # Eriksson
echo '{/sv_SE.person} hihi' | fejkdata # Sara Eriksson hihi
echo '{/sv_SE.address}' | fejkdata --seed 42 # the same address every run
echo '{/sv_SE.word}' | fejkdata -n 3 # nät, barn and sol, a line each
printf '%s' '{/sv_SE.word}' | fejkdata -n 3 --separator ', ' # nät, barn, sol
fejkdata --list # every path the data offers
echo '{/misc.territory[SE].capital}' | fejkdata # Stockholm — a table's row, selected by key or name
echo '{/geo.SE.locality[Lund].street}' | fejkdata # Fjelievägen — a linked table, drawn inside the row
echo '{/sv_SE.word}' | fejkdata --data-path ./mydata # layer a directory over the shipped data
fejkdata --no-shipped-data -d ./mydata --list # only your data
echo '{"format":"name: {x}\n","x":["bosse","lina"]}' | fejkdata # name: bosse or name: lina
fejkdata < template.txt # a template kept in a file
fejkdata <<'EOF' # born 2003-11-27 — a quoted heredoc passes ', $ and \ as written
born {date(1990-01-01,2010-12-31,'2006-01-02')}
EOF
```
@@ -32,18 +33,19 @@ EOF
fejkdata renders the template on its stdin: a format string, or a JSON object, array or
string. Its `{…}` tokens reach the data by reference from the root,
`{/sv_SE.person.last}`, so shipped and `--data-path` categories are alike available,
and everything outside them prints as written; stdin that is JSON is read as JSON, so
`42`, `true` or `null` alone is refused. A reference names a category, or a field
and everything outside them prints as written. A reference names a category, or a field
inside one: each dot segment descends one level — folders, then the category (a JSON
file), then fields — and `[SE]` after a [table](#table) selects its row. `--list` prints
these paths; a template reads one as `{/path}`.
A format string keeps every byte of stdin, so `echo`'s newline renders too; `printf '%s'`
leaves it out. A JSON template or one reference alone drops one newline ending stdin,
`\n` or `\r\n`, which neither could print. One reference alone, `{/users}`, is the record
`users` under `--format`. Some shells' `echo` reads a backslash as an escape, so pipe a
What you send is what renders, and fejkdata adds nothing. A format string keeps every
byte, so `echo`, a file or a heredoc ends each render with its newline, and `printf '%s'`
sends none; `-n` joins renders with `--separator`, empty by default. A JSON template is
its JSON, the whitespace around it dropped, so `42`, `true` or `null` alone is refused. A
lone reference, one `{/…}` and nothing else, is the record it names under `--format`, one
newline after it allowed. Some shells' `echo` reads a backslash as an escape, so pipe a
template holding one from a quoted heredoc or a file. The rest of stdin's rules are under
[Decisions](docs/decisions.md#the-cli-renders-the-template-on-its-stdin-keeping-a-format-strings-every-byte).
[Decisions](docs/decisions.md#the-cli-renders-exactly-the-template-on-its-stdin-and-adds-nothing).
| Flag | |
|------|--|
@@ -51,7 +53,7 @@ template holding one from a quoted heredoc or a file. The rest of stdin's rules
| `--no-shipped-data` | load only the `--data-path` directories |
| `-s`, `--seed N` | same seed, version and data: identical output |
| `-n`, `--repeat N` | render the template N times (up to 1048576), each an independent draw, streamed |
| `--separator S` | between repeated values (default a newline) |
| `--separator S` | between repeated renders (default empty) |
| `--format F` | `text` (default), `json`, `ndjson`, `csv` or `sql` — a record's columns, one record per row (json frames them as an array) |
| `--table T` | the INSERT target for `--format sql` (default: a lone reference's last segment, such as `users` for `{/users}`, else `records`) |
| `--list` | print every path, then exit |
@@ -60,7 +62,8 @@ template holding one from a quoted heredoc or a file. The rest of stdin's rules
`--name value` and `--name=value` both work, a short flag's value attaches or
follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
[Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing
dir, a lone reference to nothing), `2` misuse — a bad flag, an operand, nothing on stdin, or a
dir, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument
other than a flag, an empty or unreadable stdin, nothing piped in, or a
template that does not compile. From a checkout:
`go run ./cmd/fejkdata …`.
@@ -113,8 +116,7 @@ echo '{/users}' | fejkdata --seed 1 -d ./mydata --format sql # IN
echo '{/users}' | fejkdata --seed 1 -d ./mydata --format sql --table people # INSERT into another table
```
A record is one reference alone, `{/users}`, or a template whose fields are its
columns. `--repeat` streams that many records — `json` frames them as one array document,
`--repeat` streams that many records — `json` frames them as one array document,
`ndjson` writes one object per line, `csv` a row after a header, `sql` one INSERT
per line. Only a category-level template is a record; a field, choice or folder
errors, and so does a `repeat` on
@@ -550,7 +552,7 @@ A name naming several rows is an error listing their keys, unless a row selected
before it settles which ([Linked tables](#linked-tables)). A selector is part of the path, so it works
wherever a path does: `Fake`, `FakeRecord`, a `{/misc.territory[SE].capital}` reference
and a struct tag. A dot inside the brackets belongs to the key or name, so
`city[St. Louis]` selects it. A path starts with a name, so `[SE]` alone selects nothing.
`city[St. Louis]` selects it.
### Linked tables
@@ -996,7 +998,7 @@ internal/builtinfunc/ the {name()} functions: their checks and draws, checksums,
internal/datafiles/ the walk of a data tree: its folders, its category files and the rows files beside them
internal/datatype/ the datatype a record column holds; the root's DataType is an alias of it
internal/drawstate/ the seeded randomness and the {seq()} counters a generator draws through
internal/grammar/ how the template language is written: format tokens, paths and selectors, reference sigils, identifiers, calc syntax, and whether a struct tag is a template or a path
internal/grammar/ how the template language is written: format tokens, paths and selectors, reference sigils, identifiers, calc syntax, and whether a string is a template or a path
internal/invariant/ the one phrase every package panics with when an invariant breaks
internal/proven/ what a proof knows of a value, and the bounds a calc takes from its operands
internal/rows/ a table's rows: the TSV, the options proved over it, the links between tables, row selection and draws, and the pin set one path or one named pick fixes
+71 -76
View File
@@ -28,38 +28,39 @@ const usage = `Usage: <template> | fejkdata [flags]
fejkdata reads one template from stdin and renders it:
echo 'name: {/sv_SE.person.last}' | fejkdata
echo '{"format":"{x}","x":["bosse","lina"]}' | fejkdata
echo '{"format":"{x}\n","x":["bosse","lina"]}' | fejkdata
fejkdata < template.txt
fejkdata <<'EOF'
born {date(1990-01-01,2010-12-31,'2006-01-02')}
EOF
fejkdata <<'EOF'
born {date(1990-01-01,2010-12-31,'2006-01-02')}
EOF
A template is a format string, or a JSON object, array or string. Its {…} tokens
reach the data by reference from the root, {/sv_SE.person.last}, whether the data
is shipped or layered with --data-path; everything outside them prints as written,
and stdin that is JSON is read as JSON, so 42, true or null alone is refused.
A reference names a category or a dotted path into one ({/sv_SE.person.last}), a
table's row by key or name ({/misc.territory[SE]}), and .. steps up from a row to
the row it links to ({/geo.SE.locality..municipality.name}).
is shipped or layered with --data-path. A reference names a category or a dotted
path into one ({/sv_SE.person.last}), a table's row by key or name
({/misc.territory[SE]}), and .. steps up from a row to the row it links to
({/geo.SE.locality..municipality.name}).
A format string keeps every byte of stdin, so echo's newline renders too. JSON and a
lone reference drop one newline ending stdin, \n or \r\n, which neither could print.
A quoted heredoc, <<'EOF', passes a template holding $, a backslash or a quote as
written.
What you send is what renders, and fejkdata adds nothing: a format string keeps
every byte, so echo's newline ends each render, and printf '%s' sends none. -n
joins renders with --separator, empty by default. A JSON template is its JSON, the
whitespace around it dropped, so 42, true or null alone is refused. A quoted
heredoc, <<'EOF', passes $, a backslash or a quote as written.
With --format json, ndjson, csv or sql the template must name a record — a
template whose fields are its columns, or one reference alone, {/sv_SE.person} —
and the rows are written as one JSON array, one JSON object per line, one CSV row
(after a header), or one INSERT.
With --format json, ndjson, csv or sql the template must be a record — a JSON
template whose fields are its columns, or a lone reference, {/sv_SE.person}, one
newline after it allowed — and the rows are written as one JSON array, one JSON
object per line, one CSV row (after a header), or one INSERT.
-d, --data-path D a data directory to layer over the shipped data (repeatable; last wins on a clash)
--format F output form: text (default), json, ndjson, csv or sql
-h, --help print this help, then exit
--list list the paths the data offers, each one a reference's path, then exit
--list list the paths the data offers, each one a reference from the root, then exit
--no-shipped-data load only the --data-path directories
-n, --repeat N render the template N times, 1..1048576 (default 1)
-s, --seed N same seed, version and data: identical output
--separator S string between repeated values (default newline)
--separator S string between repeated renders (default empty)
--table T the INSERT target for --format sql (default: a lone reference's last segment, else records)
--version print the version, then exit
@@ -184,7 +185,7 @@ func splitFlags(arg string) ([]flagArg, error) {
}
func parseArgs(argv []string) (invocation, error) {
in := invocation{repeat: 1, separator: "\n", format: "text"}
in := invocation{repeat: 1, format: "text"}
for i := 0; i < len(argv); i++ {
arg := argv[i]
if arg == "--" {
@@ -277,16 +278,21 @@ func (in invocation) checkFlags() error {
const pipeHint = "echo '{/sv_SE.person}' | fejkdata"
// operandError refuses an operand, naming the spelling that reads what it most likely
// means: stdin for -, a file for a file's name, else the template piped in.
// operandError refuses an argument that is no flag, naming the spelling that reads what
// it most likely means: stdin for -, a file for a file's name, else a template piped in.
func operandError(arg string) error {
unexpected := "unexpected argument '" + arg + "'"
if arg == "-" {
return errors.New("fejkdata takes no operand, and reads stdin already; drop the -")
return fmt.Errorf("%s; stdin is read already, so drop it", unexpected)
}
if fi, err := os.Stat(arg); err == nil && fi.Mode().IsRegular() {
return fmt.Errorf("fejkdata takes no operand; read the file from stdin: fejkdata < %s", arg)
return fmt.Errorf("%s; read a file from stdin: fejkdata < %s", unexpected, shellQuoted(arg))
}
return fmt.Errorf("fejkdata takes no operand; pipe the template to stdin: %s", pipeHint)
example := pipeHint
if inline, err := fejkdata.IsTemplate(arg); !inline && err == nil {
example = "echo '{/" + arg + "}' | fejkdata"
}
return fmt.Errorf("%s; the template is read from stdin: %s", unexpected, example)
}
// check rejects a flag combination or an operand that cannot run, before any data is
@@ -318,50 +324,32 @@ func (in invocation) options() []fejkdata.Option {
return opts
}
// write streams the input's renders to w, repeat of them joined by the
// separator and ended by a newline. A value that renders once renders every time,
// so a render failure comes before anything is written; a write failure surfaces
// from Flush, bufio keeping the first one.
// write streams the input's renders to w, repeat of them joined by the separator. A
// value that renders once renders every time, so a render failure comes before anything
// is written; a write failure surfaces from Flush, bufio keeping the first one.
func (in invocation) write(f *fejkdata.Generator, src input, w io.Writer) error {
if in.writesRecords() {
return in.writeRecords(f, src, w)
}
draw, err := textDraw(f, src)
t, err := f.NewTemplate(src.template)
if err != nil {
return err
return inputError{err}
}
out := bufio.NewWriter(w)
for i := 0; i < in.repeat; i++ {
v, err := draw()
if err != nil {
return err
}
if i > 0 {
out.WriteString(in.separator)
}
out.WriteString(v)
out.WriteString(t.Fake())
}
out.WriteString("\n")
return out.Flush()
}
// textDraw builds what one text render yields: the value, from a path or a template.
func textDraw(f *fejkdata.Generator, src input) (func() (string, error), error) {
if src.path != "" {
return func() (string, error) { return f.Fake(src.path) }, nil
}
t, err := f.NewTemplate(src.template)
if err != nil {
return nil, inputError{err}
}
return func() (string, error) { return t.Fake(), nil }, nil
}
// recordStream builds the record drawer for the input, plus the INSERT table
// a sql format names.
func (in invocation) recordStream(f *fejkdata.Generator, src input) (func() (*fejkdata.Record, error), string, error) {
record := func() (*fejkdata.Record, error) { return f.FakeRecord(src.path) }
if src.path == "" {
record := func() (*fejkdata.Record, error) { return f.FakeRecord(src.record) }
if src.record == "" {
t, err := f.NewRecordTemplate(src.template)
if err != nil {
return nil, "", inputError{spacedReference(src.template, err)}
@@ -370,7 +358,7 @@ func (in invocation) recordStream(f *fejkdata.Generator, src input) (func() (*fe
}
table := in.table
if table == "" {
table = defaultTable(src.path)
table = defaultTable(src.record)
}
return record, table, nil
}
@@ -411,7 +399,7 @@ func (in invocation) writeRecords(f *fejkdata.Generator, src input, w io.Writer)
}
// defaultTable names the INSERT target when --table is absent: the path's last
// segment, or "records" for a template that reads no path alone.
// segment, or "records" when stdin is not a lone reference.
func defaultTable(path string) string {
if path == "" {
return "records"
@@ -438,38 +426,37 @@ type inputError struct{ error }
func (e inputError) Unwrap() error { return e.error }
// input is what stdin names: the path a lone reference reads, or else the template.
// input is what stdin holds: the template, and the path a lone reference names as a
// record.
type input struct {
path string
template string
record string
}
// parseInput reads stdin's text. A format string keeps every byte; JSON or a lone
// reference sheds one newline ending it, which neither could print.
// parseInput reads stdin. A format string is its every byte; a JSON template sheds the
// whitespace around it; a lone reference names a record less one newline ending it.
func parseInput(raw string) (input, error) {
if raw == "" {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return input{}, fmt.Errorf("stdin holds no template: %s", pipeHint)
}
if text, found := strings.CutSuffix(raw, "\n"); found {
template := raw
if strings.ContainsRune(`{["`, rune(trimmed[0])) && json.Valid([]byte(trimmed)) {
template = trimmed
}
text, found := strings.CutSuffix(template, "\n")
if found {
text = strings.TrimSuffix(text, "\r")
if path, lone := loneReference(text); lone {
return input{path: path}, nil
}
if json.Valid([]byte(text)) {
return input{template: text}, nil
}
}
if path, lone := loneReference(raw); lone {
return input{path: path}, nil
}
return input{template: raw}, nil
record, _ := loneReference(text)
return input{template: template, record: record}, nil
}
// loneReference is the path that text, one reference from the root and nothing else,
// reads: the path IsTemplate advises writing instead. A JSON string holding one is one.
// loneReference returns the path that text reads when it is one reference from the root
// and nothing else, bare or as a JSON string: the path IsTemplate advises writing instead.
func loneReference(text string) (string, bool) {
var unquoted string
if json.Unmarshal([]byte(text), &unquoted) == nil {
if strings.HasPrefix(text, `"`) && strings.HasSuffix(text, `"`) && json.Unmarshal([]byte(text), &unquoted) == nil {
text = unquoted
}
if !strings.HasPrefix(text, "{/") || !strings.HasSuffix(text, "}") {
@@ -485,10 +472,10 @@ func loneReference(text string) (string, bool) {
return path, true
}
// spacedReference adds to err, a refusal of template as a record, that the whitespace
// around a lone reference made it text, where it does.
// spacedReference adds to a record refusal that whitespace around a lone reference made
// the template text.
func spacedReference(template string, err error) error {
if trimmed := strings.TrimSpace(template); trimmed != template {
if trimmed := strings.TrimSpace(template); trimmed != template && errors.Is(err, fejkdata.ErrNoColumns) {
if path, lone := loneReference(trimmed); lone {
return fmt.Errorf("%w; the whitespace around {/%s} makes it text, so remove it to write the record %s", err, path, path)
}
@@ -496,6 +483,14 @@ func spacedReference(template string, err error) error {
return err
}
// shellQuoted is s as a shell reads it back: bare where it holds nothing a shell expands.
func shellQuoted(s string) string {
if s != "" && !strings.ContainsAny(s, " \t'\"$`\\*?[]{}()<>|&;#~!") {
return s
}
return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
}
func main() {
var stdin io.Reader = os.Stdin
if fi, err := os.Stdin.Stat(); err == nil && fi.Mode()&os.ModeCharDevice != 0 {
@@ -512,7 +507,7 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
return misuse(stderr, err)
}
if in.help {
fmt.Fprint(stdout, usage)
io.WriteString(stdout, usage)
return 0
}
if in.version {
+47 -41
View File
@@ -36,11 +36,8 @@ func TestRunOutputsValue(t *testing.T) {
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
if strings.TrimSpace(out) == "" {
t.Fatalf("empty output, stderr=%q", errb)
}
if !strings.HasSuffix(out, "\n") {
t.Errorf("output should end with newline, got %q", out)
if out == "" || strings.Contains(out, "\n") {
t.Fatalf("output = %q, stderr=%q; want one value and nothing added", out, errb)
}
}
@@ -93,7 +90,7 @@ func TestRunShortFlagValues(t *testing.T) {
t.Errorf("run(%v) = %q, want %q", args, got, want)
}
}
_, three, _ := runOut("{/word}", "-s", "1", "-n3", "-d", svSE)
_, three, _ := runOut("{/word}\n", "-s", "1", "-n3", "-d", svSE)
if lines := strings.Split(strings.TrimRight(three, "\n"), "\n"); len(lines) != 3 {
t.Errorf("-n3 gave %d lines: %q", len(lines), three)
}
@@ -177,7 +174,7 @@ func TestRunFlagValues(t *testing.T) {
}
func TestRunRepeat(t *testing.T) {
code, out, errb := runOut("{/word}", "--seed", "1", "--repeat", "3", "--data-path", svSE)
code, out, errb := runOut("{/word}\n", "--seed", "1", "--repeat", "3", "--data-path", svSE)
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
@@ -185,7 +182,7 @@ func TestRunRepeat(t *testing.T) {
if len(lines) != 3 {
t.Errorf("repeat=3 gave %d lines: %q", len(lines), out)
}
_, short, _ := runOut("{/word}", "--seed", "1", "-n", "3", "-d", svSE)
_, short, _ := runOut("{/word}\n", "--seed", "1", "-n", "3", "-d", svSE)
if short != out {
t.Errorf("-n 3 = %q, want the same as --repeat 3 %q", short, out)
}
@@ -196,8 +193,8 @@ func TestRunSeparator(t *testing.T) {
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
if n := strings.Count(out, "\n"); n != 1 {
t.Errorf("want one trailing newline, got %d: %q", n, out)
if n := strings.Count(out, "\n"); n != 0 {
t.Errorf("want no newline, got %d: %q", n, out)
}
if !strings.Contains(out, ",") {
t.Errorf("values should be comma-joined: %q", out)
@@ -205,7 +202,7 @@ func TestRunSeparator(t *testing.T) {
}
func TestRunRepeatAdvancesRNG(t *testing.T) {
code, out, errb := runOut("{/person}", "--seed", "1", "--repeat", "5", "--data-path", svSE)
code, out, errb := runOut("{/person}\n", "--seed", "1", "--repeat", "5", "--data-path", svSE)
if code != 0 {
t.Fatalf("run = %d, stderr=%q", code, errb)
}
@@ -253,12 +250,13 @@ func TestRunTakesNoOperand(t *testing.T) {
}{
{[]string{"sv_SE.person"}, "echo '{/sv_SE.person}' | fejkdata"},
{[]string{"{/sv_SE.person}"}, "echo '{/sv_SE.person}' | fejkdata"},
{[]string{"name:", "{/sv_SE.person.last}"}, "echo '{/sv_SE.person}' | fejkdata"},
{[]string{"-"}, "drop the -"},
{[]string{"name: {/sv_SE.person.last}"}, "echo '{/sv_SE.person}' | fejkdata"},
{[]string{"en_US.address"}, "echo '{/en_US.address}' | fejkdata"},
{[]string{"-"}, "drop it"},
{[]string{file}, "fejkdata < " + file},
} {
code, out, errb := runOut("", c.args...)
if code != 2 || out != "" || !strings.Contains(errb, "takes no operand") || !strings.Contains(errb, c.want) {
if code != 2 || out != "" || !strings.Contains(errb, "unexpected argument") || !strings.Contains(errb, c.want) {
t.Errorf("run(%v) = %d, %q, %q; want misuse naming %q", c.args, code, out, errb, c.want)
}
}
@@ -287,33 +285,43 @@ func TestRunTerminalStdinIsMisuse(t *testing.T) {
}
}
func TestRunNewlineEndingStdin(t *testing.T) {
func TestRunRendersExactlyWhatIsSent(t *testing.T) {
for stdin, want := range map[string]string{
"hihi": "hihi\n",
"hihi\n": "hihi\n\n",
"hihi\r\n": "hihi\r\n\n",
"\n": "\n\n",
"{{x}}\n": "{x}\n\n",
"\"a\"\n": "a\n",
"\"a\"\r\n": "a\n",
"{/misc.territory[SE].alpha2}\n": "SE\n",
"{/misc.territory[SE].alpha2}\r\n": "SE\n",
"hihi": "hihi",
"hihi\n": "hihi\n",
"hihi\r\n": "hihi\r\n",
"hihi\n\n": "hihi\n\n",
"{{x}}\n": "{x}\n",
"\"a\"\n": "a",
" \t\"a\"\r\n\n": "a",
"{/misc.territory[SE].alpha2}": "SE",
"{/misc.territory[SE].alpha2}\n": "SE\n",
} {
code, out, errb := runOut(stdin, "--seed", "1")
if code != 0 || out != want {
t.Errorf("run(%q) = %d, %q, %q; want %q", stdin, code, out, errb, want)
}
}
code, out, errb := runOut("x: {/sv_SE.person.last}\n", "--seed", "1")
if code != 0 || !strings.HasPrefix(out, "x: ") || !strings.HasSuffix(out, "\n\n") || strings.Count(out, "\n") != 2 {
t.Errorf("echo's format string = %d, %q, %q; want echo's newline, then the line's", code, out, errb)
if code, out, errb := runOut("{\"format\":\"{x}\\n\",\"x\":[\"a\",\"b\"]}\n"); code != 0 || (out != "a\n" && out != "b\n") {
t.Errorf("a JSON template's format = %d, %q, %q; want its own newline only", code, out, errb)
}
if code, out, errb := runOut("{/sv_SE.word}\n", "--seed", "1", "-n", "3", "--separator", ","); code != 0 || strings.Count(out, "\n,") != 2 || !strings.HasSuffix(out, "\n") {
t.Errorf("echo's newline and a separator = %d, %q, %q; want each render's newline, then the separator", code, out, errb)
}
}
func TestRunRefusesBlankStdin(t *testing.T) {
for _, stdin := range []string{"", "\n", " \r\n\t"} {
if code, out, errb := runOut(stdin); code != 2 || out != "" || !strings.Contains(errb, "holds no template") {
t.Errorf("run(%q) = %d, %q, %q; want misuse naming the empty stdin", stdin, code, out, errb)
}
}
}
func TestRunTextPrintsAsWritten(t *testing.T) {
for _, stdin := range []string{"sv_SE.person", "[Skåne län]", "x[1]y", `say "hi"`} {
code, out, errb := runOut(stdin)
if code != 0 || out != stdin+"\n" {
if code != 0 || out != stdin {
t.Errorf("run(%q) = %d, %q, %q; want the text as written", stdin, code, out, errb)
}
}
@@ -330,7 +338,7 @@ func TestRunMultipleDataPaths(t *testing.T) {
}
func TestRunUnknownCategoryFails(t *testing.T) {
code, _, errb := runOut("{/nope}", "--data-path", svSE)
code, _, errb := runOut("{/nope}", "--data-path", svSE, "--format", "csv")
if code != 1 {
t.Fatalf("run = %d, want 1", code)
}
@@ -394,7 +402,7 @@ func TestRunShippedDataByDefault(t *testing.T) {
t.Errorf("--list without --data-path = %d, %q", code, list)
}
code, _, errb = runOut("{/person}")
if code != 1 || !strings.Contains(errb, "person") {
if code != 2 || !strings.Contains(errb, "person") {
t.Errorf("a category outside the shipped tree: code %d, stderr %q", code, errb)
}
}
@@ -409,17 +417,17 @@ func TestRunSelectsATableRow(t *testing.T) {
t.Fatal(err)
}
}
if code, out, errb := runOut("{/region[12]}", "--no-shipped-data", "-d", dir); code != 0 || out != "Skåne län\n" {
if code, out, errb := runOut("{/region[12]}", "--no-shipped-data", "-d", dir); code != 0 || out != "Skåne län" {
t.Fatalf("{/region[12]} = %d, %q, stderr=%q", code, out, errb)
}
if code, out, _ := runOut("{/region[Skåne län].code}", "--no-shipped-data", "-d", dir); code != 0 || out != "12\n" {
if code, out, _ := runOut("{/region[Skåne län].code}", "--no-shipped-data", "-d", dir); code != 0 || out != "12" {
t.Fatalf("{/region[Skåne län].code} = %d, %q", code, out)
}
if code, out, _ := runOut("{/region[12]}", "--no-shipped-data", "-d", dir, "--format", "sql"); code != 0 || out != `INSERT INTO "region" ("code", "name") VALUES ('12', 'Skåne län');`+"\n" {
t.Fatalf("--format sql {/region[12]} = %d, %q, want the table named without its selector", code, out)
}
if code, _, errb := runOut("{/region[99]}", "--no-shipped-data", "-d", dir); code != 1 || !strings.Contains(errb, `"99"`) {
t.Fatalf("{/region[99]} = %d, stderr=%q, want a runtime error naming the row", code, errb)
if code, _, errb := runOut("{/region[99]}", "--no-shipped-data", "-d", dir); code != 2 || !strings.Contains(errb, `"99"`) {
t.Fatalf("{/region[99]} = %d, stderr=%q, want misuse naming the row", code, errb)
}
if code, out, _ := runOut("{/misc.territory[SE]}", "--seed", "1", "--format", "csv"); code != 0 || !strings.HasPrefix(out, "alpha2,") || !strings.Contains(out, "\nSE,SWE,") {
t.Fatalf("--format csv {/misc.territory[SE]} = %d, %q", code, out)
@@ -437,7 +445,7 @@ func TestUsageReferencesResolve(t *testing.T) {
func TestRunOperandMisuseBeforeLoad(t *testing.T) {
code, _, errb := runOut("", "--no-shipped-data", "x")
if code != 2 || !strings.Contains(errb, "takes no operand") || strings.Contains(errb, "--data-path") {
if code != 2 || !strings.Contains(errb, "unexpected argument") || strings.Contains(errb, "--data-path") {
t.Fatalf("an operand with no data = %d, %q; want the operand refused before any load", code, errb)
}
}
@@ -448,19 +456,19 @@ func TestRunInlineTemplate(t *testing.T) {
t.Fatalf("inline format string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut(`{"format":"name: {x}","x":["bosse","lina"]}`, "--seed", "1")
if code != 0 || (out != "name: bosse\n" && out != "name: lina\n") {
if code != 0 || (out != "name: bosse" && out != "name: lina") {
t.Fatalf("inline JSON template = %d, %q, want one name, stderr %q", code, out, errb)
}
code, out, errb = runOut(`"name: {/sv_SE.person.last}"`, "--seed", "1")
if code != 0 || !strings.HasPrefix(out, "name: ") || strings.Contains(out, "{") {
t.Fatalf("inline JSON string = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut(`{digits(1)}`, "--seed", "1", "-n", "2")
code, out, errb = runOut("{digits(1)}\n", "--seed", "1", "-n", "2")
if code != 0 || len(strings.Split(strings.TrimRight(out, "\n"), "\n")) != 2 {
t.Fatalf("inline template with --repeat = %d, %q, stderr %q", code, out, errb)
}
code, out, errb = runOut("{/sv_SE.person} hihi", "--seed", "1")
if code != 0 || !strings.HasSuffix(out, " hihi\n") || strings.Contains(out, "{") {
if code != 0 || !strings.HasSuffix(out, " hihi") || strings.Contains(out, "{") {
t.Fatalf("a reference and text = %d, %q, stderr %q", code, out, errb)
}
}
@@ -471,8 +479,6 @@ func TestRunTemplateMisuse(t *testing.T) {
`{"format":"x"}`: "is a string",
"name: {/no.such.path}": "no entry",
"a } b": "}}",
" [\"a\",\"b\"]": "may not be padded",
"[\"a\",\"b\"]\n\n": "may not be padded",
"42": "number",
"{//sv_SE.person}": "write {/sv_SE.person}",
} {
@@ -520,7 +526,7 @@ func TestRunRepeatIsBounded(t *testing.T) {
t.Fatal(err)
}
code, out, errb := runOut("{/x}", "--no-shipped-data", "-d", dir, "--repeat", "1048576", "--separator", "")
if code != 0 || len(out) != 1048576+1 {
if code != 0 || len(out) != 1048576 {
t.Errorf("repeat at the cap = %d, %d bytes, stderr %q; want every render streamed", code, len(out), errb)
}
}
+24 -20
View File
@@ -55,7 +55,7 @@ single-dash long flag is rejected naming the double-dash spelling, and `-s=42` i
rejected naming both short spellings: `=` belongs to the long form, and reading `=42` as
the value would make `-d=./x` a directory named `=./x`.
## A struct tag is a template by its shape, not by a flag
## A struct tag is a template by its shape
2026-09-03, Lilleman auf Larv.
@@ -63,7 +63,7 @@ A JSON object, array or string, or a string carrying a `{` token, is an inline t
anything else is a path. A name may not contain a brace, a bracket or a quote, so a path
can never collide with any of those spellings, and the leading `[` or `"` is gated on
valid JSON so a stray copied bracket never swallows a tag — it names nothing, and says
so. No option is needed. Reserving the characters whole — though only a leading one
so. Reserving the characters whole — though only a leading one
could collide — keeps one simple name rule instead of a leading-position special case.
The JSON string is what makes the library's own advice reachable: the error for an
object holding only a format names `"…"`, and that spelling has to work where it is
@@ -72,32 +72,36 @@ printed. A struct tag of one reference alone, `{/users}`, is refused naming the
`{.name}` or `{..name}` is refused naming `{/name}`, since an inline template sits in no
folder. `IsTemplate` exports the rule, so struct tags and any other caller read one.
## The CLI renders the template on its stdin, keeping a format string's every byte
## The CLI renders exactly the template on its stdin, and adds nothing
2026-10-07, Lilleman auf Larv. Serves goals 7, 7.1 and 7.2.
The template comes from stdin, from `echo`, a file or a heredoc, so one command line gets
a value and a quoted heredoc needs no escape. The CLI takes no operand: one is misuse
naming the spelling that reads what it likely means, `fejkdata < FILE` for a file and
nothing for `-`, and so is a terminal with nothing piped in, which would otherwise wait
for typed input. A format string keeps every byte, `echo`'s newline included, since
everything outside `{…}` prints as written. JSON and one reference alone drop one newline
ending stdin, `\n` or `\r\n`, which neither could print; the rest reaches `NewTemplate`
unchanged, so a padded JSON template is refused there, as goal 5.7 asks. One reference
from the root and nothing else, `{/users}` or `"{/users}"`, reads the path it names: it
renders the same text, and under `--format` it is the record `users`, with `users` the
default `--table`; one to nothing exits 1, as an unknown path does. That reading sits in
the CLI against goal 3.2 until `todo.md` item 129 moves it into the library. `--list`
still prints paths, each read as `{/path}`. Valid while the CLI reads one template per
run.
a value and a quoted heredoc needs no escape. A format string is stdin's every byte, and
fejkdata adds no newline of its own: `echo` ends each render with its newline,
`printf '%s'` sends none, and `-n` joins renders with `--separator`, empty by default. A
bench of the README's personas picked this over dropping a newline, or adding one, on
2026-10-07: one rule a user states in a sentence, at the cost of `printf` or a
`--separator` now and then. A JSON template is its JSON: the maintainer ruled that the
whitespace around it is not part of the template, so goal 5.7 does not refuse it on
stdin, while `NewTemplate` still refuses a padded one. A lone reference, `{/users}` or
`"{/users}"`, names the record `users` under `--format`, one newline after it allowed,
with `users` the default `--table`; one to nothing there exits 1, as an unknown path
does, until `todo.md` item 129 moves the reading into the library, against goal 3.2
meanwhile. Stdin holding only whitespace is misuse, since an unset variable sends just a
newline. The CLI takes no argument but its flags: one is misuse naming the spelling that
reads what it likely means, `fejkdata < FILE` for a file's name, dropping it for `-`,
and the template piped in otherwise; so is a terminal with nothing piped in, which would
otherwise wait for typed input. `--list` still prints paths, each read as `{/path}`.
Valid while the CLI reads one template per run.
## A template that does not compile is misuse (exit 2), including a reference that resolves to nothing, save the CLI's lone reference until item 129
## A template that does not compile is misuse (exit 2), including a reference that resolves to nothing, save a lone reference under the CLI's `--format` until item 129
2026-09-24, Lilleman auf Larv.
The whole template is the spelling under test, and `NewTemplate` compiles, links and
validates as one step. An unknown *path* stays a runtime error (exit 1): there the
template is well-formed and only the data is absent.
validates as one step. Under the CLI's `--format`, a lone reference to nothing exits 1
until item 129: it reads a path, and only the data is absent.
## A padded JSON template is rejected, not trimmed
@@ -880,6 +884,6 @@ a lone `}` stays an error. Valid while a template escapes a brace by doubling it
## Goal 7.2 holds while a first template can be written with no escape; a literal brace still takes `{{` or `}}`
2026-10-05, Lilleman auf Larv. Serves goal 7.2. Goals are aims: it is enough that a first
template can be written with no escape and no options, as `echo -n 'name: {/sv_SE.person.last}' | fejkdata`
template can be written with no escape and no options, as `echo 'name: {/sv_SE.person.last}' | fejkdata`
is. A template that prints a literal brace writes `{{` or `}}`, and the goal allows that. Valid
while `{{` and `}}` are the template syntax's only escape.
+1 -1
View File
@@ -1,5 +1,5 @@
// Package grammar is how the template language is written: format tokens, paths and
// their selectors, reference sigils, identifiers, calc expressions, and whether an argument is a
// their selectors, reference sigils, identifiers, calc expressions, and whether a string is a
// template or a path. It reads strings only.
package grammar
+1 -2
View File
@@ -68,8 +68,7 @@ func rendersInside(nodes []node, t *template) bool {
// template reaches the binder through fields. A single read as a calc operand stands where a calc
// could not read the spelling, a reference or a path. So {/misc.coordinate.lat as lat}{calc(lat * 60)}
// stands, since {calc(/misc.coordinate.lat * 60)} does not compile.
// Where the spelling would be a whole struct tag, that entry point refuses it and names the bare
// path.
// A whole struct tag of that spelling is refused, naming the bare path.
func (b *nameBinding) refuseSingleRead(uses []nameUse) error {
if len(uses) == 0 {
panic(invariant.Broken("name %q has no read at resolve, though its compile found one", b.name))
+8 -8
View File
@@ -79,7 +79,7 @@
| 145 | 0.1.0 | | **Move `doc.go`'s definitions of a level, a pick key and an addressed key, with their worked example, to `pickKey` in `pick.go`, and gather `levelKeys`, `addressedKeys`, `renderEnv.keeps` and `readField`'s kept read beside `readUnder`, so a reader follows a named read in one file.** | 2 | 3 | 1 | 9 | 3.4, 3.2 | 15.4 |
| 127 | 0.1.0 | | **Give a Swedish street only postal codes whose delivery digit means street or rural delivery, dropping codes whose delivery digit is 9, or record why they stay.** | 3 | 3 | 4 | 6 | 4 | 14.4 |
| 110 | 0.1.0 | | **Refuse an empty segment in a path `Fake` and `FakeRecord` take, as a reference does: `sv_SE.person.` reports `path has an empty segment`.** | 1 | 1 | 3 | 6 | 6 | 13.8 |
| 156 | 0.1.0 | question | **Refuse a JSON string whose text stands alone as a format string, `"x {/a}"`, naming the bare text, or keep it as the decision "An argument is a template by its shape, not by a flag" does.** | 2 | 2 | 3 | 6 | 5.7 | 13.1 |
| 156 | 0.1.0 | question | **Refuse a JSON string whose text stands alone as a format string, `"x {/a}"`, naming the bare text, or keep it as the decision "A struct tag is a template by its shape" does.** | 2 | 2 | 3 | 6 | 5.7 | 13.1 |
| 10 | 0.1.0 | | **Test that every node kind reaches each switch over node kinds.** | 1 | 3 | 2 | 6 | 3.2 | 13.1 |
| 148 | 0.1.0 | | **State in `geo.py` the locality row each country's script hands it, and meet the fence that every parent row has a child row in one shared step.** | 2 | 3 | 2 | 6 | 2.1, 3.2 | 12.6 |
| 146 | 0.1.0 | | **Move `sharedPaths` and `subPaths` from `list.go` beside `carriedByAll` in `path.go`, and `join` beside `sortedNames` in `sorted.go`.** | 1 | 2 | 1 | 6 | 3.1, 3.2 | 11.3 |
@@ -120,7 +120,7 @@
| 159 | 0.2.0 | | **Name the quoted spelling where a template that is a bare JSON number, boolean or null is refused: `42` names `"42"`.** | 1 | 2 | 3 | 6 | 6, 5.1 | 14.1 |
| 22 | 0.2.0 | | **Give `url` and `email` a path that draws only domains nobody can register, keeping the wide set as the default.** | 3 | 4 | 7 | 8 | 16 | 14.0 |
| 46 | 0.2.0 | | **Report the same error every load for a table with two bad options, and for a folder with two unnamed rows files.** | 2 | 2 | 3 | 7 | 6, 10 | 14.0 |
| 158 | 0.2.0 | | **Drop a UTF-8 byte order mark that starts stdin, and say in the README how PowerShell 5.1 keeps `Göteborg` from reaching the CLI as `G?teborg`.** | 2 | 2 | 4 | 5 | 7.2, 5.1 | 13.5 |
| 158 | 0.2.0 | | **Drop a UTF-8 byte order mark that starts stdin, and say in the README how to stop PowerShell 5.1 sending `Göteborg` as `G?teborg`.** | 2 | 2 | 4 | 5 | 7.2, 5.1 | 13.5 |
| 79 | 0.2.0 | decision | **Beside a `--data-path`, load in `New` the user's categories, the shipped ones they override or read, and the shipped ones whose reads reach an overridden one; load the rest on first reach.** | 6 | 6 | 7 | 8 | 13 | 12.9 |
| 107 | 0.2.0 | decision | **Move the template engine into `internal/`, leaving the root the public API and its options.** | 6 | 9 | 3 | 8 | 3.2, 3.4 | 12.2 |
| 112 | 0.2.0 | | **Check a calc's field operands where its template resolves, beside its name operands, once item 36 reports every mistake at once.** | 2 | 2 | 1 | 6 | 3.2 | 10.3 |
@@ -253,9 +253,9 @@ Goal 5.9. Today a path is a second spelling of the template that reads it; after
- `List` and `--list` give each entry as the template that renders it, `{/sv_SE.person}`.
- Text that is not a JSON value and holds no `{…}` prints as written (goal 5.1), in a string field's struct tag and `Fake` alike, as stdin's text does in the CLI: `Fake("sv_SE.person")` returns `sv_SE.person`. `CHANGELOG.md` says that a path with no `{…}` prints as its text. A struct tag's `-` still skips its field.
- Where text with no `{…}` is refused, as a record or in a typed struct field, and it spells a path the data offers, the error names `{/that.path}`.
- An unknown reference exits 2, where an unknown path exits 1 today.
- Decide whether a lone reference draws as its path draws today or as a template draws: under one seed, `{/sv_SE.person}` and `{/sv_SE.person} hihi` pick different people.
- This revises the decisions "A struct tag is a template by its shape, not by a flag", "`FakeTemplate` and `NewTemplate` both stay", "`--list` stays a plain list of paths", "The CLI renders the template on its stdin, keeping a format string's every byte", and the exit-1 rule for an unknown path in "A template that does not compile is misuse (exit 2), including a reference that resolves to nothing, save the CLI's lone reference until item 129". The README's examples and exit codes, the godoc examples in `doc.go`, the CLI's usage text and header comment, and `CHANGELOG.md`'s entries move to the template spelling. Delete items 40 and 110, which this closes.
- A lone reference to nothing exits 2 under `--format`, where it exits 1 today.
- Decide whether a lone reference under `--format` draws as its path draws today or as a template draws: under one seed, the two pick different rows.
- This revises the decisions "A struct tag is a template by its shape", "`FakeTemplate` and `NewTemplate` both stay", "`--list` stays a plain list of paths", "The CLI renders exactly the template on its stdin, and adds nothing", and the exit-1 rule for an unknown path in "A template that does not compile is misuse (exit 2), including a reference that resolves to nothing, save a lone reference under the CLI's `--format` until item 129". The README's examples and exit codes, the godoc examples in `doc.go` and `CHANGELOG.md`'s entries move to the template spelling. Delete items 40 and 110, which this closes.
Split it into items for the CLI, struct tags and the library before starting.
@@ -291,9 +291,9 @@ All 13 seats of the 2026-10-07 panel and scoring run on ad59967 again named the
Today `Fake("sv_SE.person.")` reports `no field ""`, while `{/sv_SE.person.}` reports `path has an empty segment`. `grammar.CheckSegments` is the check a reference, a sibling path and a step up `..` run.
### 156. Refuse a JSON string whose text stands alone as a format string, `"x {/a}"`, naming the bare text, or keep it as the decision "An argument is a template by its shape, not by a flag" does.
### 156. Refuse a JSON string whose text stands alone as a format string, `"x {/a}"`, naming the bare text, or keep it as the decision "A struct tag is a template by its shape" does.
Goal 5.7 refuses a wrapper around what could stand alone. That decision keeps the JSON string because, as a CLI argument, bare text read as a path; stdin reads bare text as a format string, so `"x {/a}"` and `x {/a}` render alike there and in `NewTemplate`. A JSON string still earns its place where its escapes carry what bare text cannot. The goals audit of the stdin change found it.
Goal 5.7 refuses a wrapper around what could stand alone. That decision keeps the JSON string so the library's advice `"…"` works where it is printed; stdin reads bare text as a format string, so `"x {/a}"` and `x {/a}` render alike there and in `NewTemplate`. A JSON string still earns its place where its escapes carry what bare text cannot. The goals audit of the stdin change found it.
### 10. Test that every node kind reaches each switch over node kinds.
@@ -450,7 +450,7 @@ On stdin `42` is refused as "a number" and `null` with advice to write `""`, whi
`readTableOptions` in `table.go` and `walkDir` in `internal/datafiles` return on the first in Go's map order.
### 158. Drop a UTF-8 byte order mark that starts stdin, and say in the README how PowerShell 5.1 keeps `Göteborg` from reaching the CLI as `G?teborg`.
### 158. Drop a UTF-8 byte order mark that starts stdin, and say in the README how to stop PowerShell 5.1 sending `Göteborg` as `G?teborg`.
A file a Windows editor saves with a BOM prints an invisible U+FEFF first, and a JSON template with one is read as a format string. PowerShell 5.1 encodes what it pipes to a program by `$OutputEncoding`, ASCII by default. The maintainer chose 0.2.0 on 2026-10-07.