A column of one reference alone takes the datatype and null of the column it reads #15
@@ -310,6 +310,12 @@ order.id: datatype integer: {digits(3)} prints text, not an integer
|
|||||||
order.id: datatype integer: "1{digits(2)}" is not one value; write one literal or one {int()}, {float()}, {seq()} or {calc()}, or read one
|
order.id: datatype integer: "1{digits(2)}" is not one value; write one literal or one {int()}, {float()}, {seq()} or {calc()}, or read one
|
||||||
```
|
```
|
||||||
|
|
||||||
|
A column of one reference alone to another record's column — `"score": "{/src.score}"`
|
||||||
|
— is that column: it takes the column's datatype and is null where the column is, and
|
||||||
|
a struct field tagged `src.score` is nil there. A `datatype` of its own types the
|
||||||
|
column's values where they prove it, and one restating the datatype it takes is refused.
|
||||||
|
Any other read renders the column's text, a null as `""`.
|
||||||
|
|
||||||
A typed column's `{calc()}` must be proven to print a number: each operand a number
|
A typed column's `{calc()}` must be proven to print a number: each operand a number
|
||||||
literal, an `{int()}`, `{float()}`, `{seq()}` or `{digits()}` call, a calc, or a read of
|
literal, an `{int()}`, `{float()}`, `{seq()}` or `{digits()}` call, a calc, or a read of
|
||||||
such values, whose bounds keep every divisor from zero and the result within `1e300`.
|
such values, whose bounds keep every divisor from zero and the result within `1e300`.
|
||||||
@@ -330,7 +336,7 @@ renders a null as `""`. The other items' weights skew its odds:
|
|||||||
```
|
```
|
||||||
|
|
||||||
`deleted_at` is null every draw, `middle` a name three draws in four. Rejected at
|
`deleted_at` is null every draw, `middle` a name three draws in four. Rejected at
|
||||||
load: `null` anywhere but a column, naming `""`, and a column whose items declare
|
load: `null` anywhere but a column, naming `""`, and a column whose items hold
|
||||||
different datatypes.
|
different datatypes.
|
||||||
|
|
||||||
### Options and fields
|
### Options and fields
|
||||||
@@ -543,10 +549,11 @@ tokens add cost in proportion to the output.
|
|||||||
tag of one reference alone, `{/users}`, is refused naming the path `users`: both
|
tag of one reference alone, `{/users}`, is refused naming the path `users`: both
|
||||||
render the same text, and only the path names a record. `IsTemplate` exports the
|
render the same text, and only the path names a record. `IsTemplate` exports the
|
||||||
rule, so the CLI, struct tags and any other caller read one.
|
rule, so the CLI, struct tags and any other caller read one.
|
||||||
- **An inline template skips the cycle fence, and only that one.** `New` proves the
|
- **An inline template skips the cycle fence.** `New` proves the loaded tree
|
||||||
loaded tree acyclic, an inline node is a finite tree of its own, and nothing in
|
acyclic, an inline node is a finite tree of its own, and nothing in the tree can
|
||||||
the tree can reference it, so no render of it reaches itself. Every other fence
|
reference it, so no render of it reaches itself. Every other fence runs over both,
|
||||||
runs over both, from one `checkScope`.
|
from one `checkScope`, except that struct tags leave column agreement to their Go
|
||||||
|
types.
|
||||||
- **An inline template that does not compile is misuse (exit 2), including a
|
- **An inline template that does not compile is misuse (exit 2), including a
|
||||||
reference that resolves to nothing** — the whole argument is the spelling under
|
reference that resolves to nothing** — the whole argument is the spelling under
|
||||||
test, and `NewTemplate` compiles, links and validates as one step. An unknown
|
test, and `NewTemplate` compiles, links and validates as one step. An unknown
|
||||||
@@ -621,7 +628,8 @@ tokens add cost in proportion to the output.
|
|||||||
its columns from a struct instead, because a Go caller has already written that
|
its columns from a struct instead, because a Go caller has already written that
|
||||||
schema: the fields name the columns and their types are the datatypes, so a tag
|
schema: the fields name the columns and their types are the datatypes, so a tag
|
||||||
says only what to draw, and a `datatype` in it would be a second spelling of the
|
says only what to draw, and a `datatype` in it would be a second spelling of the
|
||||||
type.
|
type. The Go type is a struct column's one datatype, so its items need not agree on
|
||||||
|
one among themselves; each must only hold that type.
|
||||||
- **A struct's records follow Go's field access, and compile on first use.** The
|
- **A struct's records follow Go's field access, and compile on first use.** The
|
||||||
fields an embedded struct promotes are the struct's own — `e.First`, as
|
fields an embedded struct promotes are the struct's own — `e.First`, as
|
||||||
`encoding/json` and SQL mappers read them — so they are columns of its record and
|
`encoding/json` and SQL mappers read them — so they are columns of its record and
|
||||||
@@ -663,6 +671,11 @@ tokens add cost in proportion to the output.
|
|||||||
- **A typed column holds one value, not composed text.** Its bounds come from a
|
- **A typed column holds one value, not composed text.** Its bounds come from a
|
||||||
literal or a call's arguments, so a load error names a real value, a range check is
|
literal or a call's arguments, so a load error names a real value, a range check is
|
||||||
one comparison, and `1{digits(2)}` is a second spelling of `{int(100,199)}`.
|
one comparison, and `1{digits(2)}` is a second spelling of `{int(100,199)}`.
|
||||||
|
- **A column of one reference alone is the column it reads.** `{/src.score}` renders
|
||||||
|
exactly what `src.score` draws, so it takes that column's datatype and null rather
|
||||||
|
than restating them, and a `datatype` restating the one it takes is a second
|
||||||
|
spelling. Any other `datatype` still types the values — the one way to type a column
|
||||||
|
someone else wrote.
|
||||||
- **A typed column's calc is refused unless proven.** Operand bounds must keep each
|
- **A typed column's calc is refused unless proven.** Operand bounds must keep each
|
||||||
divisor from zero and the result finite; what they cannot show is refused rather
|
divisor from zero and the result finite; what they cannot show is refused rather
|
||||||
than trusted, since a bare `NaN` breaks the JSON and SQL it lands in.
|
than trusted, since a bare `NaN` breaks the JSON and SQL it lands in.
|
||||||
|
|||||||
+109
-14
@@ -64,19 +64,78 @@ func datatypeOf(m map[string]any, pos position) (DataType, error) {
|
|||||||
return 0, fmt.Errorf(`datatype takes "integer", "number" or "boolean", got %q`, name)
|
return 0, fmt.Errorf(`datatype takes "integer", "number" or "boolean", got %q`, name)
|
||||||
}
|
}
|
||||||
|
|
||||||
// columnDatatype is the datatype a column's items declare. They must agree, since a
|
// checkColumns rejects a record column whose items hold different datatypes, checking a column
|
||||||
|
// after the columns its items read, so a column read is named before its readers.
|
||||||
|
func checkColumns(s nodeScope) error {
|
||||||
|
checked := map[node]bool{}
|
||||||
|
var check func(path, name string, column node) error
|
||||||
|
check = func(path, name string, column node) error {
|
||||||
|
if checked[column] {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
checked[column] = true
|
||||||
|
items, _ := columnItems(column)
|
||||||
|
for _, it := range items {
|
||||||
|
if r := it.readsColumn; r != nil {
|
||||||
|
if err := check(r.a.key[1:], r.a.tail[0], r.column); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := columnDatatype(column); err != nil {
|
||||||
|
return fmt.Errorf("%s: field %q: %w", path, name, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return s(func(path string, n node) error {
|
||||||
|
t, ok := n.(*template)
|
||||||
|
if !ok || !t.record {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, name := range recordColumns(t) {
|
||||||
|
if err := check(path, name, t.fields[name]); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// columnDatatype is the datatype a column's items hold. They must agree, since a
|
||||||
// column holds one; a column only ever null is a string.
|
// column holds one; a column only ever null is a string.
|
||||||
func columnDatatype(n node) (DataType, error) {
|
func columnDatatype(n node) (DataType, error) {
|
||||||
items, _ := columnItems(n)
|
items, _ := columnItems(n)
|
||||||
if len(items) == 0 {
|
if len(items) == 0 {
|
||||||
return DataTypeString, nil
|
return DataTypeString, nil
|
||||||
}
|
}
|
||||||
|
first := itemDatatype(items[0])
|
||||||
for _, t := range items[1:] {
|
for _, t := range items[1:] {
|
||||||
if t.datatype != items[0].datatype {
|
if d := itemDatatype(t); d != first {
|
||||||
return items[0].datatype, disagreement(items[0], t)
|
return first, disagreement(items[0], first, t, d)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return items[0].datatype, nil
|
return first, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// itemDatatype is the datatype a column item declares, else that of the column it is.
|
||||||
|
func itemDatatype(t *template) DataType {
|
||||||
|
if t.datatype != DataTypeString {
|
||||||
|
return t.datatype
|
||||||
|
}
|
||||||
|
return readDatatype(t)
|
||||||
|
}
|
||||||
|
|
||||||
|
// readDatatype is the datatype of the column t is, reading it by one reference alone; a string
|
||||||
|
// when t reads none.
|
||||||
|
func readDatatype(t *template) DataType {
|
||||||
|
if t.readsColumn == nil {
|
||||||
|
return DataTypeString
|
||||||
|
}
|
||||||
|
items, _ := columnItems(t.readsColumn.column)
|
||||||
|
if len(items) == 0 {
|
||||||
|
return DataTypeString
|
||||||
|
}
|
||||||
|
return itemDatatype(items[0]) // checkColumns refuses that column where its items disagree
|
||||||
}
|
}
|
||||||
|
|
||||||
// columnItems is a column's template items, its choices unwrapped, and whether one is null.
|
// columnItems is a column's template items, its choices unwrapped, and whether one is null.
|
||||||
@@ -98,17 +157,53 @@ func columnItems(n node) (items []*template, nullable bool) {
|
|||||||
return items, nullable
|
return items, nullable
|
||||||
}
|
}
|
||||||
|
|
||||||
// disagreement names the fix for two items of one column declaring different datatypes.
|
// disagreement names the fix for two items of one column holding different datatypes.
|
||||||
func disagreement(a, b *template) error {
|
func disagreement(a *template, da DataType, b *template, db DataType) error {
|
||||||
typed, bare := a, b
|
bare, typed, want := b, a, da
|
||||||
if typed.datatype == DataTypeString {
|
if da == DataTypeString {
|
||||||
typed, bare = b, a
|
bare, typed, want = a, b, db
|
||||||
}
|
}
|
||||||
switch {
|
switch {
|
||||||
case bare.datatype != DataTypeString:
|
case da != DataTypeString && db != DataTypeString:
|
||||||
return fmt.Errorf("its items declare %s and %s; a column holds one datatype", a.datatype, b.datatype)
|
return bothTyped(a, da, b, db)
|
||||||
case bare.fields == nil: // a JSON string; an object, which may carry a weight, has a fields map
|
case typed.datatype == DataTypeString && (&valueProof{}).columnItem(bare).not[want] != "":
|
||||||
return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, bare.format, bare.format, typed.datatype)
|
return fmt.Errorf(`item %q is not %s, the datatype item %q takes from the column it reads; to read that column as text, %s`, bare.format, dataTypeNouns[want], typed.format, asText(typed))
|
||||||
|
case bare.fromString: // an object may carry a weight, which this spelling would drop
|
||||||
|
return fmt.Errorf(`item %q declares no datatype, and a column holds one; write it as {"format":%q,"datatype":%q}`, bare.format, bare.format, want)
|
||||||
}
|
}
|
||||||
return fmt.Errorf(`an item declares no datatype beside one declaring %s; a column holds one, so give it "datatype": %q`, typed.datatype, typed.datatype)
|
return fmt.Errorf(`item %q declares no datatype beside one holding %s; a column holds one, so give it "datatype": %q`, bare.format, want, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// bothTyped names the fix for two items holding different datatypes: the one taking its datatype
|
||||||
|
// from the column it reads is typed as the other where its values prove it, else read as text.
|
||||||
|
func bothTyped(a *template, da DataType, b *template, db DataType) error {
|
||||||
|
read, other := a, db
|
||||||
|
if a.datatype != DataTypeString {
|
||||||
|
read, other = b, da
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case read.datatype != DataTypeString:
|
||||||
|
return fmt.Errorf("its items hold %s and %s; a column holds one datatype", da, db)
|
||||||
|
case (&valueProof{}).columnItem(read).not[other] == "":
|
||||||
|
return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so %s", da, db, typedAs(read, other))
|
||||||
|
}
|
||||||
|
return fmt.Errorf("its items hold %s and %s; a column holds one datatype, so to read %q as text, %s", da, db, read.format, asText(read))
|
||||||
|
}
|
||||||
|
|
||||||
|
// typedAs names the spelling giving a column-read item datatype d, keeping the other keys an object
|
||||||
|
// item carries.
|
||||||
|
func typedAs(t *template, d DataType) string {
|
||||||
|
if t.fromString {
|
||||||
|
return fmt.Sprintf(`write %q as {"format":%q,"datatype":%q}`, t.format, t.format, d)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf(`give %q "datatype": %q`, t.format, d)
|
||||||
|
}
|
||||||
|
|
||||||
|
// asText names the spelling that reads the column a column-read item reads as text, keeping the
|
||||||
|
// other keys an object item carries.
|
||||||
|
func asText(t *template) string {
|
||||||
|
if t.fromString {
|
||||||
|
return fmt.Sprintf(`write {"format":"{text}","text":%q}`, t.format)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf(`set its "format" to "{text}" and add "text": %q`, t.format)
|
||||||
}
|
}
|
||||||
|
|||||||
+25
-6
@@ -31,9 +31,6 @@ func TestDatatypeAndNullSitOnlyInAColumn(t *testing.T) {
|
|||||||
`{"format":"{n}","repeat":2,"n":{"format":"1","datatype":"integer"}}`: "datatype only types a record column",
|
`{"format":"{n}","repeat":2,"n":{"format":"1","datatype":"integer"}}`: "datatype only types a record column",
|
||||||
`null`: `so write ""`,
|
`null`: `so write ""`,
|
||||||
`{"format":"{p}","p":{"format":"{x}","x":[null,"a"]}}`: `so write ""`,
|
`{"format":"{p}","p":{"format":"{x}","x":[null,"a"]}}`: `so write ""`,
|
||||||
`{"format":"","c":[{"format":"1","datatype":"integer"},"x"]}`: `write it as {"format":"x","datatype":"integer"}`,
|
|
||||||
`{"format":"","c":[{"format":"1","datatype":"integer"},{"format":"2","weight":3}]}`: `give it "datatype": "integer"`,
|
|
||||||
`{"format":"","c":[{"format":"1","datatype":"integer"},{"format":"true","datatype":"boolean"}]}`: "a column holds one datatype",
|
|
||||||
} {
|
} {
|
||||||
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
|
if _, err := compile(parse(t, src)); err == nil || !strings.Contains(err.Error(), want) {
|
||||||
t.Errorf("compile(%s) = %v, want an error containing %q", src, err, want)
|
t.Errorf("compile(%s) = %v, want an error containing %q", src, err, want)
|
||||||
@@ -44,9 +41,22 @@ func TestDatatypeAndNullSitOnlyInAColumn(t *testing.T) {
|
|||||||
func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) {
|
func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) {
|
||||||
tree := map[string]string{
|
tree := map[string]string{
|
||||||
"cat": `[{"format":"{code}","code":"200"},{"format":"{code}","code":"2x"}]`,
|
"cat": `[{"format":"{code}","code":"200"},{"format":"{code}","code":"2x"}]`,
|
||||||
"src": `{"format":"","score":[null,{"format":"{int(1,9)}","datatype":"integer"}]}`,
|
"src": `{"format":"","code":[null,"200","2x"],"flag":{"format":"{b}","b":["true","false"],"datatype":"boolean"},"score":[null,{"format":"{int(1,9)}","datatype":"integer"}]}`,
|
||||||
}
|
}
|
||||||
for _, c := range []struct{ name, column, want string }{
|
for _, c := range []struct{ name, column, want string }{
|
||||||
|
{"an item beside a typed one", `[{"format":"1","datatype":"integer"},"x"]`, `write it as {"format":"x","datatype":"integer"}`},
|
||||||
|
{"a weighted item beside a typed one", `[{"format":"1","datatype":"integer"},{"format":"2","weight":3}]`, `give it "datatype": "integer"`},
|
||||||
|
{"items of two datatypes", `[{"format":"1","datatype":"integer"},{"format":"true","datatype":"boolean"}]`, "a column holds one datatype"},
|
||||||
|
{"an item beside a typed column it reads", `["{/src.score}","5"]`, `write it as {"format":"5","datatype":"integer"}`},
|
||||||
|
{"a typed item beside a string column it reads", `["{/src.code}",{"format":"1","datatype":"integer"}]`, `write it as {"format":"{/src.code}","datatype":"integer"}`},
|
||||||
|
{"text beside a typed column it reads", `["{/src.score}","n/a"]`, `to read that column as text, write {"format":"{text}","text":"{/src.score}"}`},
|
||||||
|
{"a datatype over a typed column", `{"format":"{/src.score}","datatype":"integer"}`, `{/src.score} takes datatype integer from the column it reads; drop "datatype"`},
|
||||||
|
{"a datatype a typed column's values reject", `{"format":"{/src.score}","datatype":"boolean"}`, "{int(1,9)} prints an integer, not a boolean"},
|
||||||
|
{"two typed columns it reads", `["{/src.score}","{/src.flag}"]`, `so to read "{/src.score}" as text, write {"format":"{text}","text":"{/src.score}"}`},
|
||||||
|
{"a typed column read that holds the datatype declared beside it", `[{"format":"1.5","datatype":"number"},"{/src.score}"]`, `so write "{/src.score}" as {"format":"{/src.score}","datatype":"number"}`},
|
||||||
|
{"text beside a weighted typed column read", `[{"format":"{/src.score}","weight":3},"n/a"]`, `to read that column as text, set its "format" to "{text}" and add "text": "{/src.score}"`},
|
||||||
|
{"a value of the column it reads", `{"format":"{/src.code}","datatype":"integer"}`, `"2x" is not an integer`},
|
||||||
|
{"a null read into text", `{"format":"{x}","x":"{/src.score}","datatype":"integer"}`, "reads a null"},
|
||||||
{"a sample with leading zeros", `{"format":"{digits(3)}","datatype":"integer"}`, "{digits(3)} prints text, not an integer"},
|
{"a sample with leading zeros", `{"format":"{digits(3)}","datatype":"integer"}`, "{digits(3)} prints text, not an integer"},
|
||||||
{"a fraction", `{"format":"{v}","v":["1","1.5"],"datatype":"integer"}`, `"1.5" is not an integer`},
|
{"a fraction", `{"format":"{v}","v":["1","1.5"],"datatype":"integer"}`, `"1.5" is not an integer`},
|
||||||
{"past int64", `{"format":"9223372036854775808","datatype":"integer"}`, "past the int64 range"},
|
{"past int64", `{"format":"9223372036854775808","datatype":"integer"}`, "past the int64 range"},
|
||||||
@@ -55,7 +65,6 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) {
|
|||||||
{"a sign before a sample", `{"format":"-{int(1,9)}","datatype":"integer"}`, "is not one value"},
|
{"a sign before a sample", `{"format":"-{int(1,9)}","datatype":"integer"}`, "is not one value"},
|
||||||
{"a repeat", `{"format":"{int(1,9)}","repeat":2,"separator":",","datatype":"integer"}`, "carries a repeat"},
|
{"a repeat", `{"format":"{int(1,9)}","repeat":2,"separator":",","datatype":"integer"}`, "carries a repeat"},
|
||||||
{"through a reference", `{"format":"{/cat.code}","datatype":"integer"}`, `"2x" is not an integer`},
|
{"through a reference", `{"format":"{/cat.code}","datatype":"integer"}`, `"2x" is not an integer`},
|
||||||
{"a null through a reference", `{"format":"{/src.score}","datatype":"integer"}`, "reads a null"},
|
|
||||||
{"a bare dot", `{"format":".5","datatype":"number"}`, `".5" is not a number`},
|
{"a bare dot", `{"format":".5","datatype":"number"}`, `".5" is not a number`},
|
||||||
{"a plus sign", `{"format":"+1","datatype":"number"}`, `"+1" is not a number`},
|
{"a plus sign", `{"format":"+1","datatype":"number"}`, `"+1" is not a number`},
|
||||||
{"a text sample", `{"format":"{hex(4)}","datatype":"number"}`, "{hex(4)} prints text, not a number"},
|
{"a text sample", `{"format":"{hex(4)}","datatype":"number"}`, "{hex(4)} prints text, not a number"},
|
||||||
@@ -93,6 +102,13 @@ func TestDatatypeRejectsAValueItsTypeRejects(t *testing.T) {
|
|||||||
t.Errorf("%s: NewTemplate = %v, want the inline template refused the same way", c.name, err)
|
t.Errorf("%s: NewTemplate = %v, want the inline template refused the same way", c.name, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
_, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{
|
||||||
|
"a": `{"format":"","c":["{/b.x}",{"format":"1","datatype":"integer"}]}`,
|
||||||
|
"b": `{"format":"","x":["2",{"format":"1","datatype":"integer"}]}`,
|
||||||
|
})))
|
||||||
|
if want := `b: field "x": item "2"`; err == nil || !strings.Contains(err.Error(), want) {
|
||||||
|
t.Errorf("a column reading one whose items disagree: New = %v, want the column read reported first, containing %q", err, want)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var jsonInteger = regexp.MustCompile(`^(0|-?[1-9][0-9]*)$`)
|
var jsonInteger = regexp.MustCompile(`^(0|-?[1-9][0-9]*)$`)
|
||||||
@@ -120,9 +136,12 @@ func TestDatatypeAcceptsAColumnThatAlwaysParses(t *testing.T) {
|
|||||||
`{"format":"{calc(a / (b + 1), 2)}","a":"{int(1,9)}","b":"{digits(2)}","datatype":"number"}`,
|
`{"format":"{calc(a / (b + 1), 2)}","a":"{int(1,9)}","b":"{digits(2)}","datatype":"number"}`,
|
||||||
`{"format":"{calc(a / b, 0)}","a":"{int(1,9)}","b":"{int(1,9)}","datatype":"integer"}`,
|
`{"format":"{calc(a / b, 0)}","a":"{int(1,9)}","b":"{int(1,9)}","datatype":"integer"}`,
|
||||||
`{"format":"{calc(sub * 1.25, 2)}","sub":{"format":"{calc(a * b)}","a":"{int(1,9)}","b":"{float(0,5,2)}"},"datatype":"number"}`,
|
`{"format":"{calc(sub * 1.25, 2)}","sub":{"format":"{calc(a * b)}","a":"{int(1,9)}","b":"{float(0,5,2)}"},"datatype":"number"}`,
|
||||||
|
`{"format":"{/src.code}","datatype":"integer"}`,
|
||||||
|
`{"format":"{/src.n}","datatype":"number"}`,
|
||||||
} {
|
} {
|
||||||
row := `{"format":"","col":` + column + `}`
|
row := `{"format":"","col":` + column + `}`
|
||||||
f, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row})), WithSeed(1))
|
src := `{"format":"","code":["200","404"],"n":{"format":"{int(1,9)}","datatype":"integer"}}`
|
||||||
|
f, err := New(WithoutShippedData(), WithDataPath(writeData(t, map[string]string{"cat": cat, "row": row, "src": src})), WithSeed(1))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Errorf("%s: New = %v, want it loaded", column, err)
|
t.Errorf("%s: New = %v, want it loaded", column, err)
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -177,11 +177,18 @@ func inlineScope(n node, label string) nodeScope {
|
|||||||
return func(fn func(path string, m node) error) error { return eachNode(n, label, fn) }
|
return func(fn func(path string, m node) error) error { return eachNode(n, label, fn) }
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkScope runs the per-node fences over a scope, each over the whole scope
|
func checkScope(s nodeScope) error {
|
||||||
|
if err := checkColumns(s); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return checkRenders(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkRenders runs the per-node fences over a scope, each over the whole scope
|
||||||
// before the next, so which of several broken nodes is reported does not depend on
|
// before the next, so which of several broken nodes is reported does not depend on
|
||||||
// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks
|
// the walk. It runs after checkNoCycles, whose guarantee is what lets the walks
|
||||||
// terminate.
|
// terminate.
|
||||||
func checkScope(s nodeScope) error {
|
func checkRenders(s nodeScope) error {
|
||||||
mem := reachMemo{}
|
mem := reachMemo{}
|
||||||
if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil {
|
if err := s(func(path string, n node) error { return repeatCheck(path, n, mem) }); err != nil {
|
||||||
return err
|
return err
|
||||||
|
|||||||
@@ -253,11 +253,17 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding)
|
|||||||
}
|
}
|
||||||
|
|
||||||
// draws is what an expansion has already drawn for its held names: the variant each
|
// draws is what an expansion has already drawn for its held names: the variant each
|
||||||
// was drawn as, so every path under it reads one row, and the value each read, by
|
// was drawn as, so every path under it reads one row, and the draw each read made, by
|
||||||
// its one spelling, so the same read written twice reads one value.
|
// its one spelling, so the same read written twice reads one draw.
|
||||||
type draws struct {
|
type draws struct {
|
||||||
variant map[string]node
|
variant map[string]node
|
||||||
value map[string]string
|
value map[string]draw
|
||||||
|
}
|
||||||
|
|
||||||
|
// draw is what one read drew: its text, and whether it landed on a null.
|
||||||
|
type draw struct {
|
||||||
|
text string
|
||||||
|
null bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// readField renders one arm of a token. An arm's key is a sibling field or a
|
// readField renders one arm of a token. An arm's key is a sibling field or a
|
||||||
@@ -267,23 +273,18 @@ type draws struct {
|
|||||||
// gives one value, and a shown operand is the operand computed. Every other name is
|
// gives one value, and a shown operand is the operand computed. Every other name is
|
||||||
// drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and
|
// drawn afresh, so {word} {word} still draws twice. checkTokens, checkPath and
|
||||||
// linkRefs prove every step, so the walk cannot fail.
|
// linkRefs prove every step, so the walk cannot fail.
|
||||||
func readField(s *session, t *template, held, refScope *draws, a arm) string {
|
func readField(s *session, t *template, held, refScope *draws, a arm) draw {
|
||||||
if !t.held[a.key] {
|
if !t.held[a.key] {
|
||||||
if len(a.tail) > 0 {
|
if len(a.tail) > 0 {
|
||||||
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
|
panic(fmt.Sprintf("fejkdata: %q reads a path into %q, which the expansion does not hold", a.name, a.key))
|
||||||
}
|
}
|
||||||
return render(s, t.fields[a.key], refScope)
|
return draw{text: render(s, t.fields[a.key], refScope)}
|
||||||
}
|
}
|
||||||
// A reference that reads a path reads the caller's scope, so its draw outlives
|
d := readScope(held, refScope, a)
|
||||||
// this expansion; a sibling, and a reference read whole, stay local to it.
|
if r, done := d.value[a.path]; done {
|
||||||
d := held
|
return r
|
||||||
if isRef(a.key) && refScope != nil && len(a.tail) > 0 {
|
|
||||||
d = refScope
|
|
||||||
}
|
}
|
||||||
if v, read := d.value[a.path]; read {
|
var r draw
|
||||||
return v
|
|
||||||
}
|
|
||||||
var v string
|
|
||||||
_ = walkPath(t.fields[a.key], a.tail, pathWalk{
|
_ = walkPath(t.fields[a.key], a.tail, pathWalk{
|
||||||
// Hold the draw at every level passed through, so two paths sharing a
|
// Hold the draw at every level passed through, so two paths sharing a
|
||||||
// prefix share it.
|
// prefix share it.
|
||||||
@@ -299,10 +300,35 @@ func readField(s *session, t *template, held, refScope *draws, a arm) string {
|
|||||||
}
|
}
|
||||||
return []node{n}, nil
|
return []node{n}, nil
|
||||||
},
|
},
|
||||||
leaf: func(n node) error { v = render(s, n, refScope); return nil },
|
leaf: func(n node) error { r = renderLeaf(s, n, refScope); return nil },
|
||||||
})
|
})
|
||||||
d.value[a.path] = v
|
d.value[a.path] = r
|
||||||
return v
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
// readScope is the draws a held read keeps its draw in: for a reference that reads a path,
|
||||||
|
// refScope, so its draw outlives this expansion; for a sibling, or a reference read whole, held.
|
||||||
|
func readScope(held, refScope *draws, a arm) *draws {
|
||||||
|
if isRef(a.key) && refScope != nil && len(a.tail) > 0 {
|
||||||
|
return refScope
|
||||||
|
}
|
||||||
|
return held
|
||||||
|
}
|
||||||
|
|
||||||
|
// renderLeaf draws and renders what a read lands on: null on a null item, or on a column of one
|
||||||
|
// reference alone whose read drew null.
|
||||||
|
func renderLeaf(s *session, n node, scope *draws) draw {
|
||||||
|
n = drawn(s, n)
|
||||||
|
if _, isNull := n.(*null); isNull {
|
||||||
|
return draw{null: true}
|
||||||
|
}
|
||||||
|
r := draw{text: render(s, n, scope)}
|
||||||
|
if t, _ := n.(*template); t != nil && t.readsColumn != nil {
|
||||||
|
if d := readScope(nil, scope, t.readsColumn.a); d != nil {
|
||||||
|
r.null = d.value[t.readsColumn.a.path].null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return r
|
||||||
}
|
}
|
||||||
|
|
||||||
// drawn resolves a choice to one variant, so a bound head is a concrete node the
|
// drawn resolves a choice to one variant, so a bound head is a concrete node the
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ func (f *Generator) NewTemplate(input string) (*Template, error) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("fejkdata: %w", err)
|
return nil, fmt.Errorf("fejkdata: %w", err)
|
||||||
}
|
}
|
||||||
if err := bindInline(n, "template", f.categories); err != nil {
|
if err := bindInline(n, "template", f.categories, checkScope); err != nil {
|
||||||
return nil, fmt.Errorf("fejkdata: %w", err)
|
return nil, fmt.Errorf("fejkdata: %w", err)
|
||||||
}
|
}
|
||||||
return &Template{g: f, n: n}, nil
|
return &Template{g: f, n: n}, nil
|
||||||
@@ -101,12 +101,8 @@ func loneReference(arg string) (string, bool) {
|
|||||||
raw = m["format"]
|
raw = m["format"]
|
||||||
}
|
}
|
||||||
format, isString := raw.(string)
|
format, isString := raw.(string)
|
||||||
var units []ftoken
|
body, lone := loneRef(format)
|
||||||
if !isString || eachToken(format, func(t ftoken) error { units = append(units, t); return nil }) != nil || len(units) != 1 {
|
if !isString || !lone {
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
body := units[0].body
|
|
||||||
if units[0].kind != 'b' || !isRef(body) || strings.ContainsAny(body, "|(") {
|
|
||||||
return "", false
|
return "", false
|
||||||
}
|
}
|
||||||
if strings.HasPrefix(body, "/") {
|
if strings.HasPrefix(body, "/") {
|
||||||
@@ -137,14 +133,14 @@ func inputValue(input string) (any, error) {
|
|||||||
return raw, nil
|
return raw, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// bindInline links an inline node's references against root and runs the fences over it,
|
// bindInline links an inline node's references against root and runs check over it, naming its
|
||||||
// naming its nodes from label.
|
// nodes from label.
|
||||||
func bindInline(n node, label string, root map[string]node) error {
|
func bindInline(n node, label string, root map[string]node, check func(nodeScope) error) error {
|
||||||
scope := inlineScope(n, label)
|
scope := inlineScope(n, label)
|
||||||
if err := linkNodeRefs(scope, root); err != nil {
|
if err := linkNodeRefs(scope, root); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
return checkScope(scope)
|
return check(scope)
|
||||||
}
|
}
|
||||||
|
|
||||||
// linkNodeRefs binds the references in an inline node's templates against the
|
// linkNodeRefs binds the references in an inline node's templates against the
|
||||||
|
|||||||
@@ -59,6 +59,9 @@ type template struct {
|
|||||||
// held is every name drawn once per expansion: the bound levels above, plus the
|
// held is every name drawn once per expansion: the bound levels above, plus the
|
||||||
// siblings a {calc()} reads. nil when the format holds nothing (see expand).
|
// siblings a {calc()} reads. nil when the format holds nothing (see expand).
|
||||||
held map[string]bool
|
held map[string]bool
|
||||||
|
fromString bool // written as a JSON string rather than an object
|
||||||
|
readsColumn *columnRead // set when the format is one reference alone reading a record's column
|
||||||
|
record bool // compiled at the top without a repeat, so its fields are record columns
|
||||||
}
|
}
|
||||||
|
|
||||||
func (*template) isNode() {}
|
func (*template) isNode() {}
|
||||||
@@ -124,7 +127,7 @@ func compileString(s string) (node, error) {
|
|||||||
if err := checkTokens(s, nil); err != nil {
|
if err := checkTokens(s, nil); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
t := &template{format: s, repeat: 1}
|
t := &template{format: s, repeat: 1, fromString: true}
|
||||||
if err := t.compileFormat(); err != nil {
|
if err := t.compileFormat(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -231,7 +234,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
fieldPos := inFormat
|
fieldPos := inFormat
|
||||||
if pos == atTop && projectsColumns(o.repeat) {
|
if pos == atTop && o.repeat == 1 {
|
||||||
fieldPos = inColumn
|
fieldPos = inColumn
|
||||||
}
|
}
|
||||||
fields, err := compileFields(m, fieldPos)
|
fields, err := compileFields(m, fieldPos)
|
||||||
@@ -244,7 +247,7 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
|
|||||||
if err := checkTokens(o.format, fields); err != nil {
|
if err := checkTokens(o.format, fields); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype}
|
t := &template{format: o.format, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, record: fieldPos == inColumn}
|
||||||
if err := t.compileFormat(); err != nil {
|
if err := t.compileFormat(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -307,9 +310,6 @@ func compileFields(m map[string]any, pos position) (map[string]node, error) {
|
|||||||
return nil, fmt.Errorf("field %w", err)
|
return nil, fmt.Errorf("field %w", err)
|
||||||
}
|
}
|
||||||
n, err := compileAt(m[k], pos)
|
n, err := compileAt(m[k], pos)
|
||||||
if err == nil && pos == inColumn {
|
|
||||||
_, err = columnDatatype(n)
|
|
||||||
}
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("field %q: %w", k, err)
|
return nil, fmt.Errorf("field %q: %w", k, err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -205,28 +205,24 @@ func recordOf(n node) (*template, []Column, error) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
return nil, nil, errors.New("names a choice, not a template; a record is a template whose fields are its columns")
|
return nil, nil, errors.New("names a choice, not a template; a record is a template whose fields are its columns")
|
||||||
}
|
}
|
||||||
if !projectsColumns(t.repeat) {
|
|
||||||
return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat)
|
|
||||||
}
|
|
||||||
names := recordColumns(t)
|
names := recordColumns(t)
|
||||||
if len(names) == 0 {
|
if len(names) == 0 {
|
||||||
return nil, nil, errors.New("has no fields, so no columns")
|
return nil, nil, errors.New("has no fields, so no columns")
|
||||||
}
|
}
|
||||||
|
if !t.record {
|
||||||
|
return nil, nil, fmt.Errorf("carries repeat %d, which composes its format into one string; a record projects columns instead — drop the repeat and render the record again for more rows", t.repeat)
|
||||||
|
}
|
||||||
if err := checkColumnRefs(t, names); err != nil {
|
if err := checkColumnRefs(t, names); err != nil {
|
||||||
return nil, nil, err
|
return nil, nil, err
|
||||||
}
|
}
|
||||||
columns := make([]Column, len(names))
|
columns := make([]Column, len(names))
|
||||||
for i, name := range names {
|
for i, name := range names {
|
||||||
datatype, _ := columnDatatype(t.fields[name]) // compile refused a column whose items disagree
|
datatype, _ := columnDatatype(t.fields[name]) // checkColumns refused items that disagree wherever DataType is read
|
||||||
columns[i] = Column{Name: name, DataType: datatype}
|
columns[i] = Column{Name: name, DataType: datatype}
|
||||||
}
|
}
|
||||||
return t, columns, nil
|
return t, columns, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// projectsColumns reports whether a category or inline template with this repeat is a
|
|
||||||
// record, its fields the columns; a repeat composes the format into one string instead.
|
|
||||||
func projectsColumns(repeat int) bool { return repeat == 1 }
|
|
||||||
|
|
||||||
// checkColumnRefs rejects the reference reads a record's shared draw cannot answer
|
// checkColumnRefs rejects the reference reads a record's shared draw cannot answer
|
||||||
// for: one column rendering a level another reads a path into, and a column
|
// for: one column rendering a level another reads a path into, and a column
|
||||||
// reading the record back through its own path.
|
// reading the record back through its own path.
|
||||||
@@ -297,15 +293,11 @@ func columnRefs(t *template, columns []string) ([]columnRef, error) {
|
|||||||
// renderRecord draws each column once, in the name order recordOf fixed, over one
|
// renderRecord draws each column once, in the name order recordOf fixed, over one
|
||||||
// reference scope shared across them.
|
// reference scope shared across them.
|
||||||
func renderRecord(s *session, t *template, columns []Column) *Record {
|
func renderRecord(s *session, t *template, columns []Column) *Record {
|
||||||
scope := &draws{variant: map[string]node{}, value: map[string]string{}}
|
scope := &draws{variant: map[string]node{}, value: map[string]draw{}}
|
||||||
r := &Record{columns: append([]Column(nil), columns...)}
|
r := &Record{columns: append([]Column(nil), columns...)}
|
||||||
for i := range r.columns {
|
for i := range r.columns {
|
||||||
n := drawn(s, t.fields[r.columns[i].Name])
|
column := renderLeaf(s, t.fields[r.columns[i].Name], scope)
|
||||||
if _, isNull := n.(*null); isNull {
|
r.columns[i].Value, r.columns[i].Null = column.text, column.null
|
||||||
r.columns[i].Null = true
|
|
||||||
} else {
|
|
||||||
r.columns[i].Value = render(s, n, scope)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
return r
|
return r
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -382,6 +382,68 @@ func TestRecordBareReferenceStaysIndependent(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRecordColumnOfOneReferenceIsTheColumnItReads(t *testing.T) {
|
||||||
|
dir := writeData(t, map[string]string{
|
||||||
|
"mid": `{"format":"","score":"{/src.score}"}`,
|
||||||
|
"row": `{"format":"","chain":"{/mid.score}","code":{"format":"{/src.code}","datatype":"integer"},"dot":"{.src.score}","label":"n={/src.score}","mixed":[{"format":"{text}","text":"{/src.score}"},"n/a"],"pick":["{/src.score}",{"format":"7","datatype":"integer"}],"same":"{/src.score}","score":"{/src.score}","text":"{/src.code}"}`,
|
||||||
|
"src": `{"format":"","code":[null,"200","404"],"score":[null,{"format":"{int(1,9)}","datatype":"integer"}]}`,
|
||||||
|
})
|
||||||
|
f := newGenerator(t, dir, WithSeed(1))
|
||||||
|
inline, err := f.NewRecordTemplate(`{"format":"","score":"{/src.score}"}`)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
nulls := map[string]int{}
|
||||||
|
for i := 0; i < 200; i++ {
|
||||||
|
r, err := f.FakeRecord("row")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cols := map[string]Column{}
|
||||||
|
for _, c := range r.Columns() {
|
||||||
|
cols[c.Name] = c
|
||||||
|
}
|
||||||
|
score, code := cols["score"], cols["code"]
|
||||||
|
agree := func(name string, want Column) bool {
|
||||||
|
return cols[name].Null == want.Null && cols[name].Value == want.Value
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case score.DataType != DataTypeInteger || cols["chain"].DataType != DataTypeInteger || code.DataType != DataTypeInteger || cols["pick"].DataType != DataTypeInteger || cols["label"].DataType != DataTypeString || cols["mixed"].DataType != DataTypeString || cols["text"].DataType != DataTypeString:
|
||||||
|
t.Fatalf("%s: want score, chain, code and pick integer columns, label, mixed and text string ones", r.JSON())
|
||||||
|
case score.Null == (len(score.Value) == 1 && score.Value >= "1" && score.Value <= "9"):
|
||||||
|
t.Fatalf("score = %+v, want null or a digit from src.score", score)
|
||||||
|
case code.Null == (code.Value == "200" || code.Value == "404"):
|
||||||
|
t.Fatalf("code = %+v, want null or a code from src.code", code)
|
||||||
|
case !agree("same", score) || !agree("chain", score) || !agree("dot", score) || !agree("text", code):
|
||||||
|
t.Fatalf("%s: want every read of one src column one draw, through mid too", r.JSON())
|
||||||
|
case cols["pick"].Value != "7" && !agree("pick", score), cols["mixed"].Null || cols["mixed"].Value != "n/a" && cols["mixed"].Value != score.Value:
|
||||||
|
t.Fatalf("pick = %+v, mixed = %+v beside score %+v: want pick 7 or the score's draw, mixed n/a or the score's text", cols["pick"], cols["mixed"], score)
|
||||||
|
case cols["label"].Null || cols["label"].Value != "n="+score.Value:
|
||||||
|
t.Fatalf("label = %+v beside score %+v, want the text of the read, a null as \"\"", cols["label"], score)
|
||||||
|
case strings.Contains(r.JSON(), `"score":null`) != score.Null:
|
||||||
|
t.Fatalf("JSON() = %s, want a null score written null", r.JSON())
|
||||||
|
}
|
||||||
|
ir := inline.Fake()
|
||||||
|
ic, sqlValue := ir.Columns()[0], ir.Columns()[0].Value
|
||||||
|
if ic.Null {
|
||||||
|
sqlValue = "NULL"
|
||||||
|
}
|
||||||
|
if ic.DataType != DataTypeInteger || ic.Null == (len(ic.Value) == 1) || ir.SQLInsert("t") != `INSERT INTO "t" ("score") VALUES (`+sqlValue+`);` || ir.CSVLine() != ic.Value {
|
||||||
|
t.Fatalf("inline score = %+v written %s and %q, want an integer column, NULL and an empty field or a bare digit", ic, ir.SQLInsert("t"), ir.CSVLine())
|
||||||
|
}
|
||||||
|
for name, c := range map[string]Column{"code": code, "inline": ic, "score": score} {
|
||||||
|
if c.Null {
|
||||||
|
nulls[name]++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, name := range []string{"code", "inline", "score"} {
|
||||||
|
if n := nulls[name]; n == 0 || n == 200 {
|
||||||
|
t.Errorf("%s drew null %d times in 200 records, want both outcomes", name, n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRecordSQLQuotesIdentifiers(t *testing.T) {
|
func TestRecordSQLQuotesIdentifiers(t *testing.T) {
|
||||||
dir := writeData(t, map[string]string{
|
dir := writeData(t, map[string]string{
|
||||||
"row": `{"format": "", "postal-code": "1", "street-number": "2"}`,
|
"row": `{"format": "", "postal-code": "1", "street-number": "2"}`,
|
||||||
|
|||||||
@@ -107,9 +107,44 @@ func linkTemplateRefs(folder []string, path string, t *template, root map[string
|
|||||||
if err := t.compileFormat(); err != nil {
|
if err := t.compileFormat(); err != nil {
|
||||||
return fmt.Errorf("%s: %w", path, err)
|
return fmt.Errorf("%s: %w", path, err)
|
||||||
}
|
}
|
||||||
|
t.readsColumn = columnReadOf(t)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// columnRead is a record's column read by a format of that one reference alone, which is the
|
||||||
|
// column: it takes the column's datatype and null.
|
||||||
|
type columnRead struct {
|
||||||
|
a arm
|
||||||
|
column node
|
||||||
|
}
|
||||||
|
|
||||||
|
func columnReadOf(t *template) *columnRead {
|
||||||
|
name, lone := loneRef(t.format)
|
||||||
|
if !lone || t.repeat != 1 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
a := splitArm(name, t.refs)
|
||||||
|
target, isTemplate := t.fields[a.key].(*template)
|
||||||
|
if !isTemplate || !target.record || len(a.tail) != 1 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return &columnRead{a: a, column: target.fields[a.tail[0]]}
|
||||||
|
}
|
||||||
|
|
||||||
|
// loneRef is the reference a format of one reference token and nothing else reads.
|
||||||
|
func loneRef(format string) (string, bool) {
|
||||||
|
units, body := 0, ""
|
||||||
|
err := eachToken(format, func(t ftoken) error {
|
||||||
|
units++
|
||||||
|
body = t.body
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil || units != 1 || !isRef(body) || strings.ContainsAny(body, "|(") {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return body, true
|
||||||
|
}
|
||||||
|
|
||||||
// eachTemplate calls fn once per template, with the folder its category sits in
|
// eachTemplate calls fn once per template, with the folder its category sits in
|
||||||
// and the dot path reaching it, folders and names in sorted order.
|
// and the dot path reaching it, folders and names in sorted order.
|
||||||
func eachTemplate(root map[string]node, fn func(folder []string, path string, t *template) error) error {
|
func eachTemplate(root map[string]node, fn func(folder []string, path string, t *template) error) error {
|
||||||
|
|||||||
@@ -103,7 +103,7 @@ func expand(s *session, t *template, refScope *draws) string {
|
|||||||
if len(t.held) > 0 {
|
if len(t.held) > 0 {
|
||||||
held = &draws{
|
held = &draws{
|
||||||
variant: make(map[string]node, len(t.held)),
|
variant: make(map[string]node, len(t.held)),
|
||||||
value: make(map[string]string, len(t.held)),
|
value: make(map[string]draw, len(t.held)),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for i := range t.ops {
|
for i := range t.ops {
|
||||||
@@ -112,7 +112,7 @@ func expand(s *session, t *template, refScope *draws) string {
|
|||||||
case 'l':
|
case 'l':
|
||||||
b.WriteString(o.lit)
|
b.WriteString(o.lit)
|
||||||
case 'f':
|
case 'f':
|
||||||
b.WriteString(readField(s, t, held, refScope, o.arms[s.IntN(len(o.arms))]))
|
b.WriteString(readField(s, t, held, refScope, o.arms[s.IntN(len(o.arms))]).text)
|
||||||
case 'b':
|
case 'b':
|
||||||
// Read before the call, so the value a calc computes is the value the
|
// Read before the call, so the value a calc computes is the value the
|
||||||
// format showed. calcVars fixed the order op.operands holds.
|
// format showed. calcVars fixed the order op.operands holds.
|
||||||
@@ -120,7 +120,7 @@ func expand(s *session, t *template, refScope *draws) string {
|
|||||||
if len(o.operands) > 0 {
|
if len(o.operands) > 0 {
|
||||||
operands = make([]string, len(o.operands))
|
operands = make([]string, len(o.operands))
|
||||||
for j, a := range o.operands {
|
for j, a := range o.operands {
|
||||||
operands[j] = readField(s, t, held, refScope, a)
|
operands[j] = readField(s, t, held, refScope, a).text
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far
|
b.WriteString(o.call(s, b.String(), operands)) // b.String() is the output so far
|
||||||
|
|||||||
@@ -258,7 +258,7 @@ func (s *structShape) compileRecord(root map[string]node, t reflect.Type, label
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("%s: %w", label, err)
|
return fmt.Errorf("%s: %w", label, err)
|
||||||
}
|
}
|
||||||
if err := bindInline(n, label, root); err != nil {
|
if err := bindInline(n, label, root, checkRenders); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
record, columns, err := recordOf(n)
|
record, columns, err := recordOf(n)
|
||||||
@@ -269,9 +269,6 @@ func (s *structShape) compileRecord(root map[string]node, t reflect.Type, label
|
|||||||
s.fields = make([][]int, len(columns))
|
s.fields = make([][]int, len(columns))
|
||||||
for i, c := range columns {
|
for i, c := range columns {
|
||||||
sf, _ := t.FieldByName(c.Name)
|
sf, _ := t.FieldByName(c.Name)
|
||||||
if c.DataType != DataTypeString {
|
|
||||||
return fmt.Errorf("%s.%s: its Go type %s sets the datatype; drop \"datatype\"", label, c.Name, sf.Type)
|
|
||||||
}
|
|
||||||
if err := proof.checkField(label+"."+c.Name, sf.Type, record.fields[c.Name]); err != nil {
|
if err := proof.checkField(label+"."+c.Name, sf.Type, record.fields[c.Name]); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -318,14 +315,19 @@ func (k columnKind) holds(v proven) bool {
|
|||||||
return v.lo >= k.lo && v.hi <= k.hi
|
return v.lo >= k.lo && v.hi <= k.hi
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkField rejects a column some render of which a field of Go type ft cannot hold: a null
|
// checkField rejects a column a field of Go type ft cannot fill: a datatype, which the Go type
|
||||||
// outside a pointer, or a value its kind's datatype or range refuses.
|
// sets, a null outside a pointer, or a value its kind's datatype or range refuses.
|
||||||
func (p *valueProof) checkField(label string, ft reflect.Type, column node) error {
|
func (p *valueProof) checkField(label string, ft reflect.Type, column node) error {
|
||||||
items, nullable := columnItems(column)
|
items, _ := columnItems(column)
|
||||||
|
for _, it := range items {
|
||||||
|
if it.datatype != DataTypeString {
|
||||||
|
return fmt.Errorf("%s: its Go type %s sets the datatype; drop \"datatype\"", label, ft)
|
||||||
|
}
|
||||||
|
}
|
||||||
elem := ft
|
elem := ft
|
||||||
if ft.Kind() == reflect.Pointer {
|
if ft.Kind() == reflect.Pointer {
|
||||||
elem = ft.Elem()
|
elem = ft.Elem()
|
||||||
} else if nullable {
|
} else if p.column(column).null {
|
||||||
return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft)
|
return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft)
|
||||||
}
|
}
|
||||||
kind := columnKinds[elem.Kind()]
|
kind := columnKinds[elem.Kind()]
|
||||||
@@ -333,7 +335,7 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
for _, it := range items {
|
for _, it := range items {
|
||||||
v := p.of(it)
|
v := p.columnItem(it)
|
||||||
if reason := v.not[kind.datatype]; reason != "" {
|
if reason := v.not[kind.datatype]; reason != "" {
|
||||||
return fmt.Errorf("%s (%s): %s", label, ft, reason)
|
return fmt.Errorf("%s (%s): %s", label, ft, reason)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package fejkdata
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"reflect"
|
"reflect"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
@@ -53,10 +54,68 @@ func structData(t *testing.T) *Generator {
|
|||||||
return newGenerator(t, writeData(t, map[string]string{
|
return newGenerator(t, writeData(t, map[string]string{
|
||||||
"person": `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"}]`,
|
"person": `[{"format":"{first} {last}","first":"Ada","last":"Lovelace"},{"format":"{first} {last}","first":"Bo","last":"Ek"}]`,
|
||||||
"place": `[{"format":"{city}","city":"Stockholm","zip":"111 22"},{"format":"{city}","city":"Tranås","zip":"573 31"}]`,
|
"place": `[{"format":"{city}","city":"Stockholm","zip":"111 22"},{"format":"{city}","city":"Tranås","zip":"573 31"}]`,
|
||||||
|
"mid": `{"format":"","score":["{/src.score}",{"format":"5","datatype":"integer"}]}`,
|
||||||
|
"src": `{"format":"","code":[null,"200","404"],"del":null,"score":[null,{"format":"{int(1,9)}","datatype":"integer"}]}`,
|
||||||
"trip": `{"format":"","leg":[{"format":"{to}","to":"Oslo"},{"format":"{to}","to":"Rome"}]}`,
|
"trip": `{"format":"","leg":[{"format":"{to}","to":"Oslo"},{"format":"{to}","to":"Rome"}]}`,
|
||||||
}), WithSeed(1))
|
}), WithSeed(1))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestFakeStructFieldTaggedWithAColumnIsThatColumn(t *testing.T) {
|
||||||
|
f := structData(t)
|
||||||
|
nils := map[string]int{}
|
||||||
|
show := func(p any) string {
|
||||||
|
switch p := p.(type) {
|
||||||
|
case *string:
|
||||||
|
if p != nil {
|
||||||
|
return *p
|
||||||
|
}
|
||||||
|
case *int64:
|
||||||
|
if p != nil {
|
||||||
|
return strconv.FormatInt(*p, 10)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "nil"
|
||||||
|
}
|
||||||
|
for i := 0; i < 200; i++ {
|
||||||
|
var v struct {
|
||||||
|
Code *string `fake:"src.code"`
|
||||||
|
Codes *string `fake:"[\"{/src.score}\",\"{/src.code}\"]"`
|
||||||
|
Del *int64 `fake:"src.del"`
|
||||||
|
Label string `fake:"n={/src.score}"`
|
||||||
|
Mixed *string `fake:"[\"{/src.score}\",\"x\"]"`
|
||||||
|
Pair *int64 `fake:"[\"{/src.score}\",\"5\"]"`
|
||||||
|
Score *int64 `fake:"src.score"`
|
||||||
|
}
|
||||||
|
if err := f.FakeStruct(&v); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
score, code := show(v.Score), show(v.Code)
|
||||||
|
switch {
|
||||||
|
case show(v.Del) != "nil":
|
||||||
|
t.Fatalf("Del = %s, want nil from a column only ever null", show(v.Del))
|
||||||
|
case show(v.Mixed) != "x" && show(v.Mixed) != score, show(v.Pair) != "5" && show(v.Pair) != score, show(v.Codes) != score && show(v.Codes) != code:
|
||||||
|
t.Fatalf("Mixed %s, Pair %s, Codes %s beside score %s and code %s: want each the literal or the draw of the column it reads", show(v.Mixed), show(v.Pair), show(v.Codes), score, code)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case v.Code == nil:
|
||||||
|
nils["code"]++
|
||||||
|
case *v.Code != "200" && *v.Code != "404":
|
||||||
|
t.Fatalf("Code = %q, want nil or a code, never a null's \"\"", *v.Code)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case v.Score == nil && v.Label == "n=":
|
||||||
|
nils["score"]++
|
||||||
|
case v.Score == nil || *v.Score < 1 || *v.Score > 9 || v.Label != "n="+strconv.FormatInt(*v.Score, 10):
|
||||||
|
t.Fatalf("%+v, want Score nil beside Label n=, or one digit in both", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, name := range []string{"code", "score"} {
|
||||||
|
if n := nils[name]; n == 0 || n == 200 {
|
||||||
|
t.Errorf("%s was nil %d times in 200 fills, want both outcomes", name, n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestFakeStructFillsTaggedFields(t *testing.T) {
|
func TestFakeStructFillsTaggedFields(t *testing.T) {
|
||||||
a, b := structData(t), structData(t)
|
a, b := structData(t), structData(t)
|
||||||
people := map[string]string{"Ada": "Lovelace", "Bo": "Ek"}
|
people := map[string]string{"Ada": "Lovelace", "Bo": "Ek"}
|
||||||
@@ -166,6 +225,18 @@ func TestFakeStructErrors(t *testing.T) {
|
|||||||
{&struct {
|
{&struct {
|
||||||
A int `fake:"[null,\"{int(1,9)}\"]"`
|
A int `fake:"[null,\"{int(1,9)}\"]"`
|
||||||
}{}, "can draw null, which int cannot hold; make it *int"},
|
}{}, "can draw null, which int cannot hold; make it *int"},
|
||||||
|
{&struct {
|
||||||
|
A int64 `fake:"src.score"`
|
||||||
|
}{}, "can draw null, which int64 cannot hold; make it *int64"},
|
||||||
|
{&struct {
|
||||||
|
A string `fake:"src.code"`
|
||||||
|
}{}, "can draw null, which string cannot hold; make it *string"},
|
||||||
|
{&struct {
|
||||||
|
A *bool `fake:"src.score"`
|
||||||
|
}{}, ".A (*bool): {int(1,9)} prints an integer, not a boolean"},
|
||||||
|
{&struct {
|
||||||
|
A int64 `fake:"mid.score"`
|
||||||
|
}{}, "can draw null, which int64 cannot hold; make it *int64"},
|
||||||
{&struct {
|
{&struct {
|
||||||
A int `fake:"{digits(3)}"`
|
A int `fake:"{digits(3)}"`
|
||||||
}{}, ".A (int): {digits(3)} prints text, not an integer"},
|
}{}, ".A (int): {digits(3)} prints text, not an integer"},
|
||||||
|
|||||||
@@ -17,11 +17,6 @@ The record API lands first, so the data update can use it.
|
|||||||
- `code` and `symbol` sibling fields reading `currency`, as `{code} {symbol}` → a matching pair
|
- `code` and `symbol` sibling fields reading `currency`, as `{code} {symbol}` → a matching pair
|
||||||
- `{a} & {b}`, each reading `person` → one person, or two when `a` and `b` name different groups
|
- `{a} & {b}`, each reading `person` → one person, or two when `a` and `b` name different groups
|
||||||
- two bare `{/sv_SE.word}` → two words
|
- two bare `{/sv_SE.word}` → two words
|
||||||
- Reference inheritance — settle whether a column that is exactly one reference to
|
|
||||||
another record's column, like `{/src.score}`, takes that column's datatype and
|
|
||||||
null. Today a null there writes `""`, a `*T` struct field reading it gets `""`
|
|
||||||
rather than nil, and a typed column reading it is refused. Settle before draw
|
|
||||||
groups and the data update.
|
|
||||||
|
|
||||||
### Data
|
### Data
|
||||||
|
|
||||||
@@ -29,7 +24,9 @@ The record API lands first, so the data update can use it.
|
|||||||
blocks as columns (`sv_SE.person` → `femalefirst`, `malefirst`; `misc.uuid` →
|
blocks as columns (`sv_SE.person` → `femalefirst`, `malefirst`; `misc.uuid` →
|
||||||
`variant`), and `sv_SE.address` draws its postal code apart from its locality.
|
`variant`), and `sv_SE.address` draws its postal code apart from its locality.
|
||||||
It also settles what a version promises about shipped data: its paths, its
|
It also settles what a version promises about shipped data: its paths, its
|
||||||
record columns, and whether a seed renders the same output across versions.
|
record columns with their datatypes and nulls — a column of one reference alone
|
||||||
|
takes both, so typing a column or adding a null breaks its readers — and whether
|
||||||
|
a seed renders the same output across versions.
|
||||||
- `email.local` and `username` share their handle lists, while their name
|
- `email.local` and `username` share their handle lists, while their name
|
||||||
variants differ on purpose. Share the lists only if that is a clean win.
|
variants differ on purpose. Share the lists only if that is a clean win.
|
||||||
|
|
||||||
|
|||||||
@@ -16,31 +16,60 @@ type proven struct {
|
|||||||
integral bool
|
integral bool
|
||||||
notOperand string // why some render reads as no finite number, the way calc reads it
|
notOperand string // why some render reads as no finite number, the way calc reads it
|
||||||
not [len(dataTypeNames)]string
|
not [len(dataTypeNames)]string
|
||||||
|
null bool // some draw of a column is null
|
||||||
}
|
}
|
||||||
|
|
||||||
// valueProof proves what typed columns and their calc operands hold, each node once per scope.
|
// valueProof proves what typed columns and their calc operands hold, each node once per scope.
|
||||||
type valueProof struct {
|
type valueProof struct {
|
||||||
memo map[node]proven
|
memo map[node]proven
|
||||||
|
columns map[node]proven
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkDatatype rejects a typed column some render of which is not text of its datatype.
|
// checkDatatype rejects a typed column item some render of which is not text of its datatype,
|
||||||
|
// and one restating the datatype of the column it is.
|
||||||
func (p *valueProof) checkDatatype(path string, n node) error {
|
func (p *valueProof) checkDatatype(path string, n node) error {
|
||||||
t, ok := n.(*template)
|
t, ok := n.(*template)
|
||||||
if !ok || t.datatype == DataTypeString {
|
if !ok || t.datatype == DataTypeString {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if err := p.prove(t, t.datatype); err != nil {
|
if d := readDatatype(t); d == t.datatype {
|
||||||
return fmt.Errorf("%s: %w", path, err)
|
return fmt.Errorf(`%s: %s takes datatype %s from the column it reads; drop "datatype"`, path, t.format, d)
|
||||||
|
}
|
||||||
|
if reason := p.columnItem(t).not[t.datatype]; reason != "" {
|
||||||
|
return fmt.Errorf("%s: datatype %s: %s", path, t.datatype, reason)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// prove reports why some render of n is not text of datatype d.
|
// columnItem proves a column item: what it renders, or, when it is a column it reads, that column.
|
||||||
func (p *valueProof) prove(n node, d DataType) error {
|
func (p *valueProof) columnItem(t *template) proven {
|
||||||
if reason := p.of(n).not[d]; reason != "" {
|
if t.readsColumn == nil {
|
||||||
return fmt.Errorf("datatype %s: %s", d, reason)
|
return p.of(t)
|
||||||
}
|
}
|
||||||
return nil
|
return p.column(t.readsColumn.column)
|
||||||
|
}
|
||||||
|
|
||||||
|
// column proves a column over what its items draw, a null item marking it null rather than
|
||||||
|
// rendering "".
|
||||||
|
func (p *valueProof) column(n node) proven {
|
||||||
|
if v, done := p.columns[n]; done {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
if p.columns == nil {
|
||||||
|
p.columns = map[node]proven{}
|
||||||
|
}
|
||||||
|
items, nullable := columnItems(n)
|
||||||
|
var v proven
|
||||||
|
for i, it := range items {
|
||||||
|
if w := p.columnItem(it); i == 0 {
|
||||||
|
v = w
|
||||||
|
} else {
|
||||||
|
v = v.or(w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
v.null = v.null || nullable
|
||||||
|
p.columns[n] = v
|
||||||
|
return v
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *valueProof) of(n node) proven {
|
func (p *valueProof) of(n node) proven {
|
||||||
@@ -66,9 +95,15 @@ func (p *valueProof) of(n node) proven {
|
|||||||
func (p *valueProof) unite(nodes []node) proven {
|
func (p *valueProof) unite(nodes []node) proven {
|
||||||
v := p.of(nodes[0])
|
v := p.of(nodes[0])
|
||||||
for _, n := range nodes[1:] {
|
for _, n := range nodes[1:] {
|
||||||
w := p.of(n)
|
v = v.or(p.of(n))
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// or is what a proof knows of a render that is either v or w.
|
||||||
|
func (v proven) or(w proven) proven {
|
||||||
v.lo, v.hi, v.nonZero = min(v.lo, w.lo), max(v.hi, w.hi), min(v.nonZero, w.nonZero)
|
v.lo, v.hi, v.nonZero = min(v.lo, w.lo), max(v.hi, w.hi), min(v.nonZero, w.nonZero)
|
||||||
v.integral = v.integral && w.integral
|
v.integral, v.null = v.integral && w.integral, v.null || w.null
|
||||||
if v.notOperand == "" {
|
if v.notOperand == "" {
|
||||||
v.notOperand = w.notOperand
|
v.notOperand = w.notOperand
|
||||||
}
|
}
|
||||||
@@ -77,7 +112,6 @@ func (p *valueProof) unite(nodes []node) proven {
|
|||||||
v.not[d] = w.not[d]
|
v.not[d] = w.not[d]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
|
||||||
return v
|
return v
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user