Load each spelling that only restated a shorter one, as that shorter form

This commit is contained in:
2026-10-07 22:32:18 +02:00
committed by Lillem4n
parent 91b65e3ded
commit e63efe327f
19 changed files with 101 additions and 366 deletions
-2
View File
@@ -40,8 +40,6 @@ replacement, and each removed path, column or flag.
at `New`: a `name` without a `key` or a `parent`, a name repeating inside one parent
row, a name spelling another row's key, and a table named like a column of any table
above it.
- `New` refuses a root choice of templates sharing one format and one set of string
fields, naming the rows TSV to write instead.
- `misc.territory`, `misc.currency`, `misc.language`, `misc.httpstatus` and
`misc.mimetype` are tables. `misc.territory` is every ISO 3166-1 territory that has
a capital, a currency and a TLD, with the columns `calling-code`, `capital`,
+26 -33
View File
@@ -41,9 +41,9 @@ these paths; a template reads one as `{/path}`.
Under `--format text`, the default, what you send is what renders, and fejkdata adds nothing. A format string keeps every
byte, so `echo`, a file or a heredoc ends each render with its newline, and `printf '%s'`
sends none; `-n` joins renders with `--separator`, empty by default. A JSON template is
an object, array or string, and the whitespace around it is dropped; `42`, `true` or
`null` alone is refused. A JSON template that should end each render in a newline writes
`\n` in its format, from a quoted heredoc or a file. A
an object, array or string, and the whitespace around it is dropped; `null` alone
renders nothing, and `42` or `true` alone is refused. A JSON template that should end
each render in a newline writes `\n` in its format, from a quoted heredoc or a file. A
lone reference, one `{/…}` and nothing else, is the record it names under `--format`, one
newline after it allowed. Some shells' `echo` reads a backslash as an escape, so pipe a
template holding one from a quoted heredoc or a file. The rest of stdin's rules are under
@@ -336,6 +336,8 @@ generator:
with the same error every call.
- `Template.Fake` cannot fail.
A path, wherever a call takes one, may start with `/`, as a reference does.
A `*Record` carries its columns via `Columns()` — each a `Column` of `Name`,
`DataType`, rendered `Value` and `Null` — and serializes them with `JSON()`
(one object), `CSVHeader()`/`CSVLine()`, or `SQLInsert(table)` — the shapes the
@@ -363,7 +365,7 @@ naming a kind that holds it. The tags are a record's columns, so they share its
describe one person, while two path tags into `sv_SE.person` are two draws. The fields an
embedded struct promotes are columns of the same record; a named struct field, or a
pointer to one, fills from its own tags as a record of its own, so its names and picks are
its own. `fake:"-"` leaves a struct field, embedded or named, or a pointer to one, unfilled.
its own. `fake:"-"` leaves a field unfilled.
Untagged fields keep their values, and so does a pointer back to a struct already being filled; a type
whose fields reach more than 1024 structs is refused, naming `fake:"-"` to cut it. The
first call for a type compiles its tags and reports what they get wrong, with the same
@@ -424,28 +426,28 @@ An item of a choice may carry a `weight` (default `1`) to skew its odds:
Renders `070-412 38 91` ten times as often as `08-…`. A string item is weighted by
writing it as `{ "format": "AB", "weight": 3 }`. Rejected at load: a weight that
is negative, non-numeric, `0` (never drawn — remove the item), `1` (the default)
or outside a choice, and a repeated item — `["a", "a", "b"]` is `[{ "format": "a", "weight": 2 }, "b"]`,
is negative, non-numeric, `0` (never drawn — remove the item) or outside a choice,
and a repeated item — `["a", "a", "b"]` is `[{ "format": "a", "weight": 2 }, "b"]`,
and the error says so.
### Repeat
A template may carry `repeat` (an integer above `1`) to render its format that
many times — each an independent draw — joined by `separator` (default `""`):
A template may carry `repeat` (a positive integer, default `1`) to render its format
that many times — each an independent draw — joined by `separator` (default `""`):
```json
{ "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] }
```
Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat`,
a `separator` of `""` (the default), and a `repeat` that multiplies to more than
1 048 576 renders along any path of nested repeats.
Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat` above
`1`, and a `repeat` that multiplies to more than 1 048 576 renders along any path of
nested repeats.
### Datatype
A record column may declare `datatype` — `integer`, `number` or `boolean` — so `json`
writes `42` rather than `"42"` and `sql` a bare literal; a column without one is a
string:
writes `42` rather than `"42"` and `sql` a bare literal; a column without one, or with
`string`, is a string:
```json
{ "format": "",
@@ -455,10 +457,10 @@ string:
```
Writes e.g. `{"id":1,"paid":true,"total":59.97}`. A column is a field of the top-level
template, or an item of a choice standing in for one; `datatype` anywhere else is a
load error. A typed column holds one value, alone in its format: a literal, one
`{int()}`, `{float()}`, `{seq()}` or `{calc()}` call, or a read that lands only on such
values. `integer` is an int64 written `0|-?[1-9][0-9]*` — `{float()}` prints one at
template, or an item of a choice standing in for one; `datatype` anywhere else, bar
`string`, is a load error. A typed column holds one value, alone in its format: a
literal, one `{int()}`, `{float()}`, `{seq()}` or `{calc()}` call, or a read that lands only on such
values. `integer` is an int64 written `-?(0|[1-9][0-9]*)` — `{float()}` prints one at
`0` decimals within int64 — `number` a JSON number, `boolean` `true` or `false`. A value its
datatype cannot hold is a load error naming it:
@@ -493,9 +495,9 @@ renders a null as `""`. The other items' weights skew its odds:
{ "format": "", "deleted_at": null, "middle": [null, { "format": "{n}", "n": ["Ann", "Eva"], "weight": 3 }] }
```
`deleted_at` is null every draw, `middle` a name three draws in four. Rejected at
load: `null` anywhere but a column, naming `""`, and a column whose items hold
different datatypes.
`deleted_at` is null every draw, `middle` a name three draws in four. Anywhere but a
column, `null` is `""`. Rejected at load: a column whose items hold different
datatypes.
### Table
@@ -542,10 +544,6 @@ is refused; a name spelling another row's key is refused, since the key would
select first and the name never. The table's options are its own — `rows`, `key`,
`name`, `weight` and `parent` — so a column may be named `name`, as one usually is.
A choice of templates sharing one format and one set of string fields is a table
written by hand, and `New` refuses it in a data file naming the TSV to write; an
inline template has no file beside it, so there it stays a choice.
### Row selection
`[key]` or `[name]` after a table's name selects one row: `misc.territory[SE]` and
@@ -625,15 +623,13 @@ reference, `..` is the folder above ([References](#references)).
`format`, `weight`, `repeat`, `separator` and `datatype` are the only options;
**any other key is a field** (see [Decisions](docs/decisions.md#options-and-fields-share-one-namespace)), and `rows` makes a category
a [table](#table), so no template carries a field of that name. A [name](#names) shares the
namespace too. An object that does nothing a
string can't — only a `format` — is rejected naming the string, as is a one-item
choice naming its item.
namespace too.
### Functions
A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad
count, range, country or expression fails fast; an integer is written plain
(`5`, not `+5` or `05`); bounds are finite; a sample that could only ever emit one
count, range, country or expression fails fast; an integer may carry a sign or leading
zeros (`+5` and `05` are `5`); bounds are finite; a sample that could only ever emit one
value (`int(5,5)`, `float(1,1,2)`) is rejected naming the text to write instead;
and a length, count or decimal place beyond a sane maximum is rejected, so a
fat-fingered `hex(2000000000)` never tries to allocate gigabytes. Every builtin draws only from the seed — a
@@ -778,10 +774,7 @@ Refused at `New`, each error naming what to write instead:
one;
- a read of a name inside the field bound to it: read it outside that field;
- a binding in a choice's item, which every other item would leave unbound;
- a binding nothing reads, or one read only once, unless that read sits in a `repeat`
nested where it is bound: write what it binds there instead, `{/word}` for
`{/word as w}{w}` and `{/region.name}` for `{/region as r}{r.name}`; a calc cannot read a
reference or a path, so a calc's one read of a name bound to either is not refused;
- a binding nothing reads;
- a path through a name that selects a row;
- `{n}` beside `{n.w}` where `n`'s category reads `w` twice, as in `{w}-{w}` or
`{w.a} {w.b}`;
+29 -15
View File
@@ -45,9 +45,9 @@ path into one ({/sv_SE.person.last}), a table's row by key or name
Under --format text, the default, what you send is what renders, and fejkdata adds
nothing: a format string keeps every byte, so echo's newline ends each render, and
printf '%s' sends none. -n joins renders with --separator, empty by default. A JSON
template is an object, array or string, and the whitespace around it is dropped; 42,
true or null alone is refused. A quoted heredoc, <<'EOF', passes $, a backslash or a
quote as written.
template is an object, array or string, and the whitespace around it is dropped; null
alone renders nothing, and 42 or true alone is refused. A quoted heredoc, <<'EOF',
passes $, a backslash or a quote as written.
With --format json, ndjson, csv or sql the template must be a record — a JSON
template whose fields are its columns, or a lone reference such as {/sv_SE.person}:
@@ -410,7 +410,13 @@ func defaultTable(path string) string {
if path == "" {
return "records"
}
var names strings.Builder // the path with its [selectors] cut out
segments := strings.Split(withoutSelectors(path), ".")
return segments[len(segments)-1]
}
// withoutSelectors is path with its [selectors] cut out.
func withoutSelectors(path string) string {
var names strings.Builder
for depth, i := 0, 0; i < len(path); i++ {
switch {
case path[i] == '[':
@@ -421,8 +427,7 @@ func defaultTable(path string) string {
names.WriteByte(path[i])
}
}
segments := strings.Split(names.String(), ".")
return segments[len(segments)-1]
return names.String()
}
// inputError marks a failure that is the input's own fault — no template, one that does
@@ -464,25 +469,34 @@ func parseInput(raw string) (input, error) {
}
// loneReference returns the path that text reads when it is one reference from the root
// and nothing else, bare or as a JSON string: the path IsTemplate advises writing instead.
// and nothing else, bare or as a JSON string.
func loneReference(text string) (string, bool) {
var unquoted string
if strings.HasPrefix(text, `"`) && strings.HasSuffix(text, `"`) && json.Unmarshal([]byte(text), &unquoted) == nil {
text = unquoted
}
if !strings.HasPrefix(text, "{/") || !strings.HasSuffix(text, "}") {
return "", false
}
if inline, err := fejkdata.IsTemplate(text); inline || err == nil {
return "", false
}
path := text[2 : len(text)-1]
if inline, err := fejkdata.IsTemplate(path); inline || err != nil {
path, opens := strings.CutPrefix(text, "{/")
path, closes := strings.CutSuffix(path, "}")
if !opens || !closes || !isPath(path) {
return "", false
}
return path, true
}
// isPath reports whether a reference's body after its / is a path alone: no second /, no
// folder sigil, and none of the |, ( or " as " a token reads outside a selector.
func isPath(body string) bool {
if body == "" || strings.ContainsAny(body[:1], "/.") {
return false
}
names := withoutSelectors(body)
if strings.ContainsAny(names, "|(") || strings.Contains(names, " as ") {
return false
}
inline, err := fejkdata.IsTemplate(body)
return !inline && err == nil
}
// spacedReference adds to a record refusal that whitespace around a lone reference made
// the template text.
func spacedReference(template string, err error) error {
+2 -2
View File
@@ -31,7 +31,7 @@ func datatypeOf(m map[string]any, pos position) (DataType, error) {
return 0, fmt.Errorf("datatype must be a string, got %T", v)
}
if name == DataTypeString.String() {
return 0, fmt.Errorf("datatype %q is the default, so it has no effect; drop it", name)
return DataTypeString, nil
}
for d := DataTypeInteger; d < datatype.Count; d++ {
if name != d.String() {
@@ -42,7 +42,7 @@ func datatypeOf(m map[string]any, pos position) (DataType, error) {
}
return d, nil
}
return 0, fmt.Errorf(`datatype takes "integer", "number" or "boolean", got %q`, name)
return 0, fmt.Errorf(`datatype takes "string", "integer", "number" or "boolean", got %q`, name)
}
// checkColumns rejects a record column whose items hold different datatypes, checking a column
+8 -17
View File
@@ -3,7 +3,6 @@ package fejkdata
import (
"encoding/json"
"fmt"
"strings"
"github.com/larvit/fejkdata/internal/grammar"
)
@@ -55,9 +54,8 @@ func (f *Generator) FakeTemplate(input string) (string, error) {
// IsTemplate reports whether arg is an inline template rather than a path, by its shape: an
// arg holding a {, or a valid JSON array or string, is a template, and anything else is a
// path. A path errors where it holds a } or a quote, starts with [, or is no valid path; so
// do a template of one reference alone, which is a path written as a template, and a path
// written with a leading /.
// path. A path errors where it holds a } or a quote, starts with [, or is no valid path. A path
// may start with /, as a reference does, and every call taking a path drops it.
func IsTemplate(arg string) (bool, error) {
inline, err := grammar.IsTemplate(arg)
if err != nil {
@@ -67,24 +65,17 @@ func IsTemplate(arg string) (bool, error) {
}
func compileInput(input string) (node, error) {
v, err := inputValue(input)
if err != nil {
return nil, err
}
return compile(v)
return compile(inputValue(input))
}
// inputValue reads an inline template as the value compile takes: the JSON value it holds, or
// the input itself as a format string when it is not JSON.
func inputValue(input string) (any, error) {
// inputValue reads an inline template as the value compile takes: the JSON value it holds, the
// whitespace around it dropped, or the input itself as a format string when it is not JSON.
func inputValue(input string) any {
var raw any
if err := json.Unmarshal([]byte(input), &raw); err != nil {
return input, nil
return input
}
if trimmed := strings.TrimSpace(input); trimmed != input {
return nil, fmt.Errorf("a JSON template may not be padded with spaces, which a format string would render; write %s", trimmed)
}
return raw, nil
return raw
}
// loadInline loads the shipped categories n reads, then runs n through the pipeline against root.
-5
View File
@@ -28,8 +28,6 @@ type builtin struct {
// operands names the fields the call reads, which a render reads for it; nil
// for a builtin that reads none.
operands func(args []string) []string
// noRefOperands says no operand can be a reference.
noRefOperands bool
// proveNumber bounds the number a call's text reads as, token its body, and says which
// datatypes that text is not; set it where every render reads as a finite number, which
// makes the call a calc operand, and leave it nil otherwise.
@@ -79,9 +77,6 @@ func Operands(name string, args []string) []string {
return b.operands(args)
}
// NoRefOperands reports a builtin no operand of which can be a reference.
func NoRefOperands(name string) bool { return builtins[name].noRefOperands }
// IsTransform reports a builtin that rewrites the text of its one operand.
func IsTransform(name string) bool {
_, isTransform := transforms[name]
-13
View File
@@ -2,11 +2,9 @@ package builtinfunc
import (
"fmt"
"slices"
"strings"
"testing"
"github.com/larvit/fejkdata/internal/grammar"
"github.com/larvit/fejkdata/internal/invariant"
)
@@ -23,17 +21,6 @@ func TestRegistryShapes(t *testing.T) {
}
}
func TestNoRefOperandsMatchesTheOperandGrammar(t *testing.T) {
for name, b := range builtins {
if b.operands == nil {
continue
}
if takesRef := slices.ContainsFunc(b.operands([]string{"/x"}), grammar.IsRef); b.noRefOperands == takesRef {
t.Errorf("%s: noRefOperands = %v, yet a reference /x read as an operand: %v", name, b.noRefOperands, takesRef)
}
}
}
func TestCheckNamesWhatACallGetsWrong(t *testing.T) {
for _, c := range []struct {
name string
+6 -10
View File
@@ -66,7 +66,7 @@ var builtins = withTransforms(map[string]builtin{
}},
"date": {arity: -1, checkArgs: dateArgs, prep: datePrep},
"time": {arity: -1, checkArgs: timeArg, prep: timePrep},
CalcName: {arity: -1, checkArgs: checkCalc, prep: calcPrep, operands: calcOperands, noRefOperands: true},
CalcName: {arity: -1, checkArgs: checkCalc, prep: calcPrep, operands: calcOperands},
// seq is the one stateful builtin: a per-generator counter from 1, advancing on
// each call. An optional name selects an independent counter; no name uses the
// default one. Deterministic by construction, so seeded output stays stable.
@@ -148,8 +148,7 @@ func randChars(r rng, n int, alphabet string) string {
return string(b)
}
// plainInt parses an integer arg written the one way: no sign, no leading zero.
func plainInt(s string) (int, error) {
func intArg(s string) (int, error) {
n, err := strconv.Atoi(s)
if errors.Is(err, strconv.ErrRange) {
return 0, fmt.Errorf("%q is past the integer range: %w", s, err)
@@ -157,13 +156,10 @@ func plainInt(s string) (int, error) {
if err != nil {
return 0, fmt.Errorf("%q is not an integer", s)
}
if strconv.Itoa(n) != s {
return 0, fmt.Errorf("%q is not a plain integer; write %d", s, n)
}
return n, nil
}
func posIntArg(a []string) error {
n, err := plainInt(a[0])
n, err := intArg(a[0])
if errors.Is(err, strconv.ErrRange) {
return fmt.Errorf("count %q exceeds the maximum %d", a[0], maxLen)
}
@@ -179,11 +175,11 @@ func posIntArg(a []string) error {
return nil
}
func intRangeArgs(a []string) error {
lo, err := plainInt(a[0])
lo, err := intArg(a[0])
if err != nil {
return fmt.Errorf("int(min,max): min %w", err)
}
hi, err := plainInt(a[1])
hi, err := intArg(a[1])
if err != nil {
return fmt.Errorf("int(min,max): max %w", err)
}
@@ -204,7 +200,7 @@ func floatArgs(a []string) error {
if e1 != nil || e2 != nil {
return fmt.Errorf("float(min,max,dp) needs numeric bounds, got %q,%q", a[0], a[1])
}
dp, err := plainInt(a[2])
dp, err := intArg(a[2])
if err != nil {
return fmt.Errorf("float(min,max,dp): decimals %w", err)
}
+1 -1
View File
@@ -56,7 +56,7 @@ func checkCalc(args []string) error {
return fmt.Errorf("calc(%q): %w", args[0], err)
}
if len(args) == 2 {
dp, err := plainInt(args[1])
dp, err := intArg(args[1])
if err != nil {
return fmt.Errorf("calc decimals %w", err)
}
+4 -54
View File
@@ -8,9 +8,6 @@ import (
func IsTemplate(arg string) (bool, error) {
if strings.ContainsRune(arg, '{') || (isJSONStart(strings.TrimSpace(arg)) && json.Valid([]byte(arg))) {
if path, lone := loneReference(arg); lone {
return false, pathAdvice(path, fmt.Sprintf("%s is the path %s written as a template", arg, path))
}
return true, nil
}
if i := strings.IndexAny(arg, `}"`); i >= 0 {
@@ -19,60 +16,13 @@ func IsTemplate(arg string) (bool, error) {
if strings.HasPrefix(strings.TrimSpace(arg), "[") {
return false, fmt.Errorf(`%q starts with "[" and is not valid JSON, so it names no template; a path starts with a name`, arg)
}
if _, err := SplitPath(arg); err != nil {
_, err := SplitPath(CallerPath(arg))
return false, err
}
if path := strings.TrimLeft(arg, "/"); path != arg && path != "" {
return false, pathAdvice(path, fmt.Sprintf("path %s starts with /, and every path starts at the root already", arg))
}
return false, nil
}
// CallerPath is a path as a caller passed it, without the slashes it may start with.
func CallerPath(path string) string { return strings.TrimLeft(path, "/") }
func isJSONStart(arg string) bool {
return strings.HasPrefix(arg, "[") || strings.HasPrefix(arg, `"`)
}
// pathAdvice refuses a spelling of path, naming path to write, or why no name can spell it.
func pathAdvice(path, refusal string) error {
if err := CheckPathIdentifiers(path); err != nil {
return err
}
return fmt.Errorf("%s; write %s", refusal, path)
}
// loneReference is the path a template spells when the value it holds — a format string, a
// JSON string, or an object holding only a format — is one reference token and nothing else.
func loneReference(arg string) (string, bool) {
var raw any
if json.Unmarshal([]byte(arg), &raw) != nil {
raw = arg
}
if m, isObject := raw.(map[string]any); isObject && len(m) == 1 {
raw = m["format"]
}
format, isString := raw.(string)
if !isString {
return "", false
}
toks, err := ParseFormat(format)
if err != nil {
return "", false
}
body, lone := loneRef(toks)
if !lone {
return "", false
}
if strings.HasPrefix(body, "/") {
body = "/" + strings.TrimLeft(body, "/")
}
_, path, err := RefShape(body)
return path, err == nil
}
// loneRef is the reference a format of one reference token and nothing else reads.
func loneRef(toks []Token) (string, bool) {
if len(toks) != 1 || toks[0].Kind != PathRead || len(toks[0].Arms) != 1 || !IsRef(toks[0].Arms[0]) {
return "", false
}
return toks[0].Arms[0], true
}
-14
View File
@@ -192,19 +192,5 @@ func Literal(text string) Facts {
if text != "true" && text != "false" {
v.Not[datatype.Boolean] = fmt.Sprintf("%q is not a boolean", text)
}
return signedZero(text, v)
}
// signedZero refuses a zero written with a sign as a typed value, naming it unsigned.
func signedZero(text string, v Facts) Facts {
mantissa, _, _ := strings.Cut(strings.ToLower(text), "e")
if !strings.HasPrefix(text, "-") || v.NotOperand != "" || strings.Trim(mantissa, "-0.") != "" {
return v
}
for _, d := range []datatype.DataType{datatype.Integer, datatype.Number} {
if v.Not[d] == "" {
v.Not[d] = fmt.Sprintf("%q is zero written with a sign; write %q", text, text[1:])
}
}
return v
}
+3 -9
View File
@@ -103,7 +103,7 @@ func (t *Table[O]) parseRows(data string) error {
t.col[name] = i
}
if header == data {
return fmt.Errorf("has no rows below its header; a table is at least two rows")
return fmt.Errorf("has no rows below its header; a table is at least one row")
}
m := len(t.header)
t.cells = make([]string, 0, m*(strings.Count(rest, "\n")+1))
@@ -118,9 +118,6 @@ func (t *Table[O]) parseRows(data string) error {
}
rest = more
}
if t.Len() < 2 {
return fmt.Errorf("has one row, which is a template; write it as one")
}
return nil
}
@@ -141,13 +138,10 @@ func (t *Table[O]) appendRow(row string, line int) error {
return nil
}
// bindOptions resolves each option to its column, refuses a name option naming the key column, and
// proves what the options claim of the cells: a key is unique, a weight a positive number.
// bindOptions resolves each option to its column, and proves what the options claim of the cells: a
// key is unique, a weight a positive number.
func (t *Table[O]) bindOptions() error {
o := t.options
if o.Key != "" && o.Key == o.Name {
return fmt.Errorf("name names the key column %q, which a selector already reads; drop it", o.Name)
}
for _, opt := range []struct {
name, value string
into *int
+2 -31
View File
@@ -6,7 +6,6 @@ import (
"strconv"
"strings"
"github.com/larvit/fejkdata/internal/builtinfunc"
"github.com/larvit/fejkdata/internal/grammar"
)
@@ -37,15 +36,6 @@ type nameBinding struct {
addressed map[pickKey]string
}
// nameUse is one read of a name: the path it reads into the name, "" for the name itself; the
// template reading it; whether a builtin reads it as an operand, and whether no reference can
// stand there; and whether it sits in a repeat nested inside the name's scope.
type nameUse struct {
tail string
in *template
operand, noRef, nested bool
}
// nameTarget is what a binding resolves to in the assembled tree: start, the node its head names, and
// the path it reads into that node.
type nameTarget struct {
@@ -239,12 +229,10 @@ func (t *template) isName(head string) bool {
return !grammar.IsRef(head) && t.fields[head] == nil && t.nameScope.lookup(head) != nil
}
// namedReadAt is one read of a name in a format: the op holding it, the compiled arm, and whether a
// builtin reads it as an operand.
// namedReadAt is one read of a name in a format: the op holding it, and the compiled arm.
type namedReadAt struct {
o *op
a *arm
operand bool
}
// namedReads is every read of a name t's format makes, in format order.
@@ -259,7 +247,7 @@ func namedReads(t *template) []namedReadAt {
}
for j := range o.operands {
if o.operands[j].kind == namedRead {
out = append(out, namedReadAt{o: o, a: &o.operands[j], operand: true})
out = append(out, namedReadAt{o: o, a: &o.operands[j]})
}
}
}
@@ -299,20 +287,3 @@ func addressedKeys(ts []templateSite, targets map[*nameBinding]nameTarget) map[*
}
return keys
}
func nameUses(ts []templateSite) map[*nameBinding][]nameUse {
uses := map[*nameBinding][]nameUse{}
for _, s := range ts {
for _, r := range namedReads(s.t) {
b := r.a.named
uses[b] = append(uses[b], nameUse{
tail: grammar.JoinSegments(r.a.tail),
in: s.t,
operand: r.operand,
noRef: r.operand && builtinfunc.NoRefOperands(r.o.Fn),
nested: s.t.nameScope != b.scope,
})
}
}
return uses
}
+2 -73
View File
@@ -3,24 +3,13 @@ package fejkdata
import (
"fmt"
"slices"
"strings"
"github.com/larvit/fejkdata/internal/grammar"
"github.com/larvit/fejkdata/internal/invariant"
)
// checkNameReads refuses each binding of t that refuseSingleRead refuses, a read of a name inside
// the field bound to it, and a read {n} beside {n.w} where n's pick renders w twice.
// checkNameReads refuses a read of a name inside the field bound to it, and a read {n} beside {n.w}
// where n's pick renders w twice.
func checkNameReads(label string, t *template, names resolvedNames) error {
for _, tok := range t.tokens {
if tok.Kind != grammar.NameBind {
continue
}
b := t.nameScope.bindings[tok.Bound]
if err := b.refuseSingleRead(names.uses[b]); err != nil {
return fmt.Errorf("%s: token {%s}: %w", label, tok.Body, err)
}
}
for _, r := range namedReads(t) {
b, target := r.a.named, names.targets[r.a.named]
if b.bindsField() && rendersInside(compilePath(target.start, target.tail).leaves, t) {
@@ -62,66 +51,6 @@ func rendersInside(nodes []node, t *template) bool {
return false
}
// refuseSingleRead refuses b when its one read sits in b's own scope, not in a repeat nested inside
// it, at a spot where the bound spelling could stand. That spelling draws the same without the
// name: write {/word} for {/word as w}{w}. A bound field can stand in only where the reading
// template reaches the binder through fields. A single read as a calc operand stands where a calc
// could not read the spelling, a reference or a path. So {/misc.coordinate.lat as lat}{calc(lat * 60)}
// stands, since {calc(/misc.coordinate.lat * 60)} does not compile.
// A whole struct tag of that spelling is refused, naming the bare path.
func (b *nameBinding) refuseSingleRead(uses []nameUse) error {
if len(uses) == 0 {
panic(invariant.Broken("name %q has no read at resolve, though its compile found one", b.name))
}
r := uses[0]
if len(uses) > 1 || r.nested {
return nil
}
spelling := b.ref
if b.bindsField() {
down, reaches := fieldPathTo(r.in, b.binder)
if !reaches {
return nil
}
spelling = grammar.JoinSegments(append(down, b.ref))
}
switch {
case strings.HasPrefix(r.tail, ".."):
spelling += r.tail
case r.tail != "":
spelling += "." + r.tail
}
if r.noRef && !calcReads(spelling) {
return nil
}
if !r.operand {
spelling = "{" + spelling + "}"
}
return fmt.Errorf("name %q is read once, so it keeps no pick for another read; write %s where it is read, and drop the token", b.name, spelling)
}
// fieldPathTo is the fields leading from t down to the template target, if t reaches it so.
func fieldPathTo(t, target *template) ([]string, bool) {
if t == target {
return nil, true
}
for _, name := range sortedNames(t.fields) {
if sub, isTemplate := t.fields[name].(*template); isTemplate {
if rest, reaches := fieldPathTo(sub, target); reaches {
return append([]string{name}, rest...), true
}
}
}
return nil, false
}
// calcReads reports whether a calc reads spelling as one operand.
func calcReads(spelling string) bool {
c, err := grammar.ParseCalc(spelling)
v, isVar := c.Expr.(grammar.CalcVar)
return err == nil && isVar && v.Name == spelling
}
// refuseTwiceDrawn refuses read, a read of a name that lands on n, where n renders a field twice and
// another read of the name addresses that field. The pick keeps one draw of the field: where n binds
// a category whose format is {w}-{w}, a {n.w} beside {n} cannot say which draw it reads. key is n's
+7 -75
View File
@@ -4,9 +4,6 @@ import (
"encoding/json"
"fmt"
"math"
"slices"
"sort"
"strings"
"github.com/larvit/fejkdata/internal/datafiles"
"github.com/larvit/fejkdata/internal/grammar"
@@ -99,70 +96,16 @@ func compile(v any) (node, error) {
}
// compileCategory compiles a data file's value, which may be a table over a rows
// file beside it, and refuses a choice that is a table written as templates.
// file beside it.
func compileCategory(c datafiles.Category) (node, error) {
if m, ok := c.JSON.(map[string]any); ok {
if _, isTable := m["rows"]; isTable {
return compileTable(m, c.Folders, c.Name, c.ReadRows)
}
}
if items, ok := c.JSON.([]any); ok {
if err := checkNotRows(items, c.Name); err != nil {
return nil, err
}
}
return compile(c.JSON)
}
// checkNotRows refuses a choice of templates sharing one format and one set of
// string fields: each item is a row, and the rows file is the spelling for that.
// docs/decisions.md#the-choice-of-rows-fence-guards-a-data-files-root-and-requires-string-fields
func checkNotRows(items []any, name string) error {
var format string
var fields []string
weighted := false
for i, raw := range items {
f, keys, w, isRow := rowShape(raw)
if !isRow || i > 0 && (f != format || !slices.Equal(keys, fields)) {
return nil
}
format, fields, weighted = f, keys, weighted || w
}
if len(items) < 2 || len(fields) == 0 {
return nil
}
weight := ""
if weighted {
weight = `, a weight column, and "weight" naming it`
}
return fmt.Errorf("a choice of %d templates with one format and the fields %v is a table; write the rows in %s.tsv with the header %q%s, and the category as {\"format\": %q, \"rows\": \"%s.tsv\"}",
len(items), fields, name, strings.Join(fields, "\t"), weight, format, name)
}
// rowShape reads a choice item as a row: a template whose every field is a string,
// with its format, its sorted field names, and whether it carries a weight.
func rowShape(raw any) (format string, fields []string, weighted, isRow bool) {
m, ok := raw.(map[string]any)
if !ok {
return "", nil, false, false
}
format, _ = m["format"].(string)
for k, v := range m {
if k == "weight" {
weighted = true
continue
}
if _, isString := v.(string); !isString || isOption(k) && k != "format" {
return "", nil, false, false
}
if k != "format" {
fields = append(fields, k)
}
}
sort.Strings(fields)
return format, fields, weighted, true
}
// position is where a JSON value sits, which decides whether it may carry a datatype or
// be null.
type position int
@@ -199,7 +142,7 @@ func compileItem(v any, pos position) (node, error) {
return compileTemplate(v, pos)
case nil:
if pos != inColumn {
return nil, fmt.Errorf(`null is a record column's value; here it only renders "", so write ""`)
return compileString("")
}
return &nullItem{}, nil
default:
@@ -247,7 +190,10 @@ func compileChoice(items []any, pos position) (node, error) {
return nil, fmt.Errorf("empty choice")
}
if len(items) == 1 {
return nil, fmt.Errorf("a one-item choice is its item; write the item")
if _, err := weightOf(items[0]); err != nil {
return nil, err
}
return compileItem(items[0], pos)
}
if err := checkNoRepeatedItem(items); err != nil {
return nil, err
@@ -326,9 +272,6 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
if err != nil {
return nil, err
}
if len(fields) == 0 && o.repeat == 1 && !o.weighted && o.datatype == DataTypeString {
return nil, fmt.Errorf("an object holding only a format is a string; write %q", o.format)
}
toks, unbound, err := parseChecked(o.format, fields)
if err != nil {
return nil, err
@@ -342,7 +285,6 @@ type templateOptions struct {
format string
repeat int
separator string
weighted bool
}
func readOptions(m map[string]any, pos position) (templateOptions, error) {
@@ -367,11 +309,7 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) {
if repeat == 1 {
return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1")
}
if o.separator == "" {
return o, fmt.Errorf("separator \"\" is the default, so it has no effect; drop it")
}
}
_, o.weighted = m["weight"]
return o, nil
}
@@ -394,7 +332,7 @@ func compileFields(m map[string]any, pos position) (map[string]node, error) {
}
// repeatOf reads a template's "repeat" (default 1): how many times its format
// is rendered and concatenated. A present one must be an integer above 1.
// is rendered and concatenated. A present one must be a positive integer.
func repeatOf(m map[string]any) (int, error) {
rv, ok := m["repeat"]
if !ok {
@@ -407,9 +345,6 @@ func repeatOf(m map[string]any) (int, error) {
if math.IsNaN(r) || math.IsInf(r, 0) || r < 1 || r != math.Trunc(r) {
return 0, fmt.Errorf("repeat must be a positive integer, got %v", rv)
}
if r == 1 {
return 0, fmt.Errorf("repeat 1 is the default, so it has no effect; drop it")
}
if r > MaxRepeat { // caps the renders one repeat asks for; repeatCheck bounds what nested ones multiply to
return 0, fmt.Errorf("repeat %v exceeds the maximum %d", rv, MaxRepeat)
}
@@ -437,9 +372,6 @@ func weightOf(raw any) (float64, error) {
if w == 0 {
return 0, fmt.Errorf("weight 0 means the item is never drawn; remove the item instead")
}
if w == 1 {
return 0, fmt.Errorf("weight 1 is the default, so it has no effect; drop it")
}
return w, nil
}
+1
View File
@@ -125,6 +125,7 @@ func literal(c Column, quote func(string) string, nullText string) string {
func (f *Generator) FakeRecord(path string) (*Record, error) {
f.mu.Lock()
defer f.mu.Unlock()
path = grammar.CallerPath(path)
segments, err := grammar.SplitPath(path)
if err != nil {
return nil, fmt.Errorf("fejkdata: %w", err)
+1
View File
@@ -16,6 +16,7 @@ import (
func (f *Generator) Fake(path string) (string, error) {
f.mu.Lock()
defer f.mu.Unlock()
path = grammar.CallerPath(path)
segments, err := grammar.SplitPath(path)
if err != nil {
return "", fmt.Errorf("fejkdata: %w", err)
+2 -3
View File
@@ -45,11 +45,10 @@ func resolveInlineTemplates(nodes nodeSet, root map[string]node) error {
}
// resolvedNames is what resolveTemplates resolved about the names of a set of templates: each binding's
// target, the keys its reads address, and its reads.
// target, and the keys its reads address.
type resolvedNames struct {
targets map[*nameBinding]nameTarget
addressed map[*nameBinding]map[pickKey]string
uses map[*nameBinding][]nameUse
}
// resolveTemplates resolves ts in steps, each over every template before the next starts. The
@@ -71,7 +70,7 @@ func resolveTemplates(ts []templateSite, root map[string]node) error {
}
s.t.compiled = compiled
}
names := resolvedNames{targets: targets, addressed: addressedKeys(ts, targets), uses: nameUses(ts)}
names := resolvedNames{targets: targets, addressed: addressedKeys(ts, targets)}
for _, s := range ts {
s.t.readsColumn = columnReadOf(s.t, targets)
}
+4 -6
View File
@@ -144,9 +144,6 @@ func (c *structFields) gatherField(sf reflect.StructField) error {
elem := structOf(sf.Type)
switch {
case tagged && tag == "-":
if elem == nil {
return fmt.Errorf(`%s.%s: fake:"-" leaves a struct field unfilled, and any other untagged field keeps its value already; drop the tag`, c.label, sf.Name)
}
return nil
case tagged:
return c.addTag(sf, tag)
@@ -229,12 +226,13 @@ func tagValue(sf reflect.StructField, tag string) (any, error) {
case err != nil:
return nil, err
case inline:
return inputValue(tag)
return inputValue(tag), nil
}
if err := grammar.CheckPathIdentifiers(tag); err != nil {
path := grammar.CallerPath(tag)
if err := grammar.CheckPathIdentifiers(path); err != nil {
return nil, err
}
return "{/" + tag + "}", nil
return "{/" + path + "}", nil
}
// checkTaggedType rejects a tagged field no column can fill.