Render blank stdin, read a category's own fields as fresh draws, refuse a bare number or boolean in a record column, and refuse a one-item choice of weight 0

CHANGELOG.md

@@ -30,8 +30,8 @@ replacement, and each removed path, column or flag.
 - The CLI renders exactly the template on its stdin, `echo '{/sv_SE.person} hihi' |
   fejkdata`, and adds nothing under `--format text`: `echo`'s newline ends the render, and `--separator`
   defaults to empty. Whitespace around a JSON template is dropped. A lone reference,
-  `{/users}`, is the record `users` under `--format`. An argument other than a flag, an
+  `{/users}`, is the record `users` under `--format`. An argument other than a flag, or
-  empty or blank stdin, or nothing piped in, exits 2.
+  nothing piped in, exits 2; an empty or blank stdin is a template, and renders as it is.
 - Table categories: a category JSON naming a `rows` TSV beside it, with the options
   `key`, `name`, `weight` and `parent`; a path selects a row by key or name,
   `misc.territory[SE]`, and descends to a linked table by name, one path drawing each

README.md

@@ -42,7 +42,8 @@ Under `--format text`, the default, what you send is what renders, and fejkdata
 byte, so `echo`, a file or a heredoc ends each render with its newline, and `printf '%s'`
 sends none; `-n` joins renders with `--separator`, empty by default. A JSON template is
 any JSON value, and the whitespace around it is dropped; `42` or `true` alone renders its
-text, and `null` alone renders nothing. A JSON template that should end
+text, and `null` alone renders nothing. An empty or blank stdin is a template too, and
+renders as it is. A JSON template that should end
 each render in a newline writes `\n` in its format, from a quoted heredoc or a file. A
 lone reference, one `{/…}` and nothing else, is the record it names under `--format`, one
 newline after it allowed. Some shells' `echo` reads a backslash as an escape, so pipe a
@@ -65,7 +66,7 @@ template holding one from a quoted heredoc or a file. The rest of stdin's rules
 follows (`-n3`, `-n 3`) and short flags bundle (`-hn 3`) — see
 [Decisions](docs/decisions.md#flags-follow-getopt_long); flags go anywhere, `--` ends them. Exit codes: `0` success, `1` runtime error (missing
 dir, a lone reference to nothing under `--format`), `2` misuse — a bad flag, an argument
-other than a flag, an empty, blank or unreadable stdin, nothing piped in, or a
+other than a flag, an unreadable stdin, nothing piped in, or a
 template that does not compile. From a checkout:
 `go run ./cmd/fejkdata …`.
 
@@ -387,6 +388,8 @@ Every value is a **node**, nestable without limit:
 | choice | `["a", "b", …]` | one item, picked at random |
 | template | `{"format": "…", …}` | its format, with `{name}` tokens rendering the named fields |
 | table | `{"format": "…", "rows": "x.tsv", …}` | its format over one row of the TSV beside it ([Table](#table)) |
+| number | `5`, `1.50` | its text as written; refused in a record column, where it could be text, `"5"`, or a typed value, `{"format": "5", "datatype": "integer"}` |
+| boolean | `true` | its text; refused in a record column, where it could be text, `"true"`, or a typed value, `{"format": "true", "datatype": "boolean"}` |
 
 ### Format string
 
@@ -429,9 +432,9 @@ An item of a choice may carry a `weight` (default `1`) to skew its odds:
 Renders `070-412 38 91` ten times as often as `08-…`. A string item is weighted by
 writing it as `{ "format": "AB", "weight": 3 }`. A repeated item counts as written,
 so `["a", "a", "b"]` draws `a` two times in three, and an item of weight `0` is never
-drawn. A weight where nothing is drawn against it, outside a choice or on the item of a
+drawn. A weight outside a choice does nothing, and so does one above `0` on the item of
-one-item choice, does nothing. Rejected at load: a weight that is negative or not a
+a one-item choice. Rejected at load: a weight that is negative or not a number, and a
-number, and a choice whose every weight is `0`.
+choice whose every weight is `0`, a one-item choice of weight `0` among them.
 
 ### Repeat
 
@@ -737,8 +740,9 @@ Renders e.g. `Hej, Pat Smith!`. Each reference draws afresh ([Format string](#fo
 so `{.person.first} {.person.last}` may name two people. Bind the person to a [name](#names) to read one: `{.person as p}{p.first}
 {p.last}`. Rejected at `New`: a path that is
 unknown, names a folder, has no folder above, reads a field not every variant
-of a choice carries, or names the category the reference sits in, and a reference
+of a choice carries, and a reference that leads back to its own value, directly,
-that leads back to its own value, directly, mutually or through a chain.
+mutually or through a chain. A reference into the category it sits in is a fresh draw
+like any other.
 
 ### Names
 

cmd/fejkdata/main.go

@@ -46,8 +46,8 @@ Under --format text, the default, what you send is what renders, and fejkdata ad
 nothing: a format string keeps every byte, so echo's newline ends each render, and
 printf '%s' sends none. -n joins renders with --separator, empty by default. A JSON
 template is any JSON value, and the whitespace around it is dropped; 42 or true alone
-renders its text, and null nothing. A quoted heredoc, <<'EOF',
+renders its text, and null nothing. An empty or blank stdin renders as it is. A
-passes $, a backslash or a quote as written.
+quoted heredoc, <<'EOF', passes $, a backslash or a quote as written.
 
 With --format json, ndjson, csv or sql the template must be a record — a JSON
 template whose fields are its columns, or a lone reference such as {/sv_SE.person}:
@@ -439,8 +439,8 @@ func withoutSelectors(path string) string {
 	return names.String()
 }
 
-// inputError marks a failure that is the input's own fault — no template, one that does
+// inputError marks a failure that is the input's own fault — nothing piped in, a template
-// not compile, or a stdin that cannot be read. run reports it as misuse (exit 2, with a
+// that does not compile, or a stdin that cannot be read. run reports it as misuse (exit 2, with a
 // pointer to --help), unlike a lone reference to nothing under --format, which is a runtime
 // error (exit 1).
 type inputError struct{ error }
@@ -460,11 +460,8 @@ type input struct {
 // parseInput reads stdin as the template, and as a record where it is a lone reference: JSON
 // with the whitespace around it dropped, else a format string with one newline ending it
 // allowed.
-func parseInput(raw string) (input, error) {
+func parseInput(raw string) input {
 	trimmed := strings.Trim(raw, jsonSpace)
-	if trimmed == "" {
-		return input{}, fmt.Errorf("stdin holds no template: %s", pipeHint)
-	}
 	text := raw
 	if json.Valid([]byte(trimmed)) {
 		text = trimmed
@@ -474,7 +471,7 @@ func parseInput(raw string) (input, error) {
 		text = strings.TrimSuffix(text, "\r")
 	}
 	record, _ := loneReference(text)
-	return input{template: raw, record: record}, nil
+	return input{template: raw, record: record}
 }
 
 // loneReference returns the path that text reads when it is one reference from the root
@@ -611,11 +608,7 @@ func readStdin(stdin io.Reader) (input, error) {
 	if err != nil {
 		return input{}, inputError{fmt.Errorf("stdin cannot be read: %w", err)}
 	}
-	src, err := parseInput(string(raw))
+	return parseInput(string(raw)), nil
-	if err != nil {
-		return input{}, inputError{err}
-	}
-	return src, nil
 }
 
 func fail(stderr io.Writer, err error) int {

internal/datafiles/datafiles.go

@@ -57,8 +57,8 @@ func (s Source) dirPath(dir []string) string {
 }
 
 // Walk hands compile every category of the tree, folders and files in name order. A
-// folder holding no category anywhere below it is skipped; a hidden file or folder is
+// folder holding no category anywhere below it is skipped; a hidden file or folder is no
-// never data.
+// category, though a category may name a hidden rows file beside it.
 func (s Source) Walk(compile func(Category) error) error {
 	if s.onDisk {
 		if s.label == "" {

node.go

@@ -5,6 +5,7 @@ import (
 	"fmt"
 	"math"
 	"strconv"
+	"strings"
 
 	"github.com/larvit/fejkdata/internal/datafiles"
 	"github.com/larvit/fejkdata/internal/grammar"
@@ -141,13 +142,34 @@ func compileItem(v any, pos position) (node, error) {
 		}
 		return &nullItem{}, nil
 	case json.Number:
+		if pos == inColumn {
+			return nil, typedOrText(string(v), numberType(v))
+		}
 		return compileString(string(v))
 	case bool:
+		if pos == inColumn {
+			return nil, typedOrText(strconv.FormatBool(v), DataTypeBoolean)
+		}
 		return compileString(strconv.FormatBool(v))
 	}
 	panic(invariant.Broken("compileItem has no case for the JSON value %T", v))
 }
 
+// typedOrText refuses a bare JSON number or boolean in a record column, which could be the
+// text it spells or a value of datatype d, naming both spellings.
+func typedOrText(text string, d DataType) error {
+	return fmt.Errorf(`%s in a record column could be text or a value of datatype %s; write %q for text, or {"format":%q,"datatype":%q}`, text, d, text, text, d)
+}
+
+// numberType is the datatype a JSON number spells: an integer where it has no fraction or
+// exponent, else a number.
+func numberType(n json.Number) DataType {
+	if strings.ContainsAny(string(n), ".eE") {
+		return DataTypeNumber
+	}
+	return DataTypeInteger
+}
+
 // jsonKind names the kind of a JSON value an option cannot take, in the data format's own
 // terms rather than the decoding library's.
 func jsonKind(v any) string {
@@ -194,6 +216,9 @@ func compileChoice(items []any, pos position) (node, error) {
 	case 0:
 		return nil, fmt.Errorf("empty choice")
 	case 1:
+		if w, err := weightOf(items[0]); err == nil && w == 0 {
+			return nil, fmt.Errorf("every weight is 0, so the choice has nothing to draw")
+		}
 		return compileItem(items[0], pos)
 	}
 	c := &choice{items: make([]node, len(items))}

reference.go

@@ -37,9 +37,8 @@ func refSegments(name string, folder []string) ([]string, error) {
 	return append(append([]string{}, base...), segs...), nil
 }
 
-// resolveRefs resolves every reference t reads, refusing one to t's own category:
+// resolveRefs resolves every reference t reads.
-// docs/decisions.md#a-category-never-references-itself-and-a-records-fences-run-at-load
+func (t *template) resolveRefs(folder []string, label string, root map[string]node) (templateRefs, error) {
-func (t *template) resolveRefs(folder []string, label, category string, root map[string]node) (templateRefs, error) {
 	var refs templateRefs
 	names := refTokens(t.tokens)
 	if len(names) == 0 {
@@ -57,9 +56,6 @@ func (t *template) resolveRefs(folder []string, label, category string, root map
 			return refs, fmt.Errorf("%s: reference {%s}: %w", label, name, err)
 		}
 		head := "/" + strings.Join(categorySegs, ".")
-		if category != "" && head == "/"+category {
-			return refs, fmt.Errorf("%s: reference {%s}: names the category it sits in; read a sibling field as a path, or move the shared value into its own category and reference that", label, name)
-		}
 		if err := provePath(target, tail, head); err != nil {
 			return refs, fmt.Errorf("%s: reference {%s}: %w", label, name, err)
 		}

resolve.go

@@ -9,9 +9,9 @@ import (
 // templateSite is a template to resolve, with its folder and category, both empty for an inline
 // template, and eachNode's label for it.
 type templateSite struct {
-	t               *template
+	t      *template
-	folder          []string
+	folder []string
-	category, label string
+	label  string
 }
 
 // resolveCategoryTemplates resolves every template of the categories, once all data is merged, so a
@@ -21,7 +21,7 @@ func resolveCategoryTemplates(sites []categorySite, root map[string]node) error
 	for _, s := range sites {
 		if err := eachNode(s.n, s.path, func(label string, n node) error {
 			if t, isTemplate := n.(*template); isTemplate {
-				ts = append(ts, templateSite{t: t, folder: s.dir, category: s.path, label: label})
+				ts = append(ts, templateSite{t: t, folder: s.dir, label: label})
 			}
 			return nil
 		}); err != nil {
@@ -56,7 +56,7 @@ type resolvedNames struct {
 // targets and keys only once every check has passed.
 func resolveTemplates(ts []templateSite, root map[string]node) error {
 	for _, s := range ts {
-		refs, err := s.t.resolveRefs(s.folder, s.label, s.category, root)
+		refs, err := s.t.resolveRefs(s.folder, s.label, root)
 		if err != nil {
 			return err
 		}