larvit/fejkdata
3
0
Fork 0
Code Pull Requests 1 Actions Activity

Load a default written out in every position, refuse null outside a column again, read a field once per alternation, refuse an empty segment in the CLI's lone reference, and hold a declared datatype as one value

Browse Source
This commit is contained in:
Lilleman auf Larv
2026-10-07 23:20:59 +02:00
committed by Lillem4n
parent 49736bd2b6
commit d8acc9f3f3
8 changed files with 104 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+17 -13
View File
@@ -120,8 +120,8 @@ echo '{/users}' | fejkdata --seed 1 -d ./mydata --format sql --table people # IN
`--repeat` streams that many records — `json` frames them as one array document,
`ndjson` writes one object per line, `csv` a row after a header, `sql` one INSERT
per line. Only a category-level template is a record; a field, choice or folder
errors, and so does a `repeat` on
per line. Only a category-level template is a record, or a one-item choice of one; a
field, a choice of more items or a folder errors, and so does a `repeat` on
the template itself, which composes the format into one string rather than
projecting columns — ask for more records with `--repeat`. A `repeat` on a column
is fine.
@@ -342,7 +342,8 @@ A `*Record` carries its columns via `Columns()` — each a `Column` of `Name`,
`DataType`, rendered `Value` and `Null` — and serializes them with `JSON()`
(one object), `CSVHeader()`/`CSVLine()`, or `SQLInsert(table)` — the shapes the
CLI's `--format` writes. `FakeRecord` and `FakeRecordTemplate` take a record; a
path or template that is not one — a bare string, a choice, or a folder — errors.
path or template that is not one — a bare string, a choice of more than one item, or a
folder — errors.
```go
type User struct {
@@ -369,7 +370,8 @@ its own. `fake:"-"` leaves a field unfilled.
Untagged fields keep their values, and so does a pointer back to a struct already being filled; a type
whose fields reach more than 1024 structs is refused, naming `fake:"-"` to cut it. The
first call for a type compiles its tags and reports what they get wrong, with the same
error on every later call; a `datatype` in a tag names the Go type that already sets it.
error on every later call; a `datatype` in a tag other than the one its Go type sets is
refused, naming that type.
A `*Generator` is safe for concurrent use; a seeded sequence is reproducible only
when drawn from one goroutine. Changing how a value is composed shifts the seeded
@@ -427,8 +429,8 @@ An item of a choice may carry a `weight` (default `1`) to skew its odds:
Renders `070-412 38 91` ten times as often as `08-…`. A string item is weighted by
writing it as `{ "format": "AB", "weight": 3 }`. A repeated item counts as written,
so `["a", "a", "b"]` draws `a` two times in three. Rejected at load: a weight that is
negative, non-numeric or `0` (never drawn — remove the item), and one where it does
nothing: outside a choice, or on the item of a one-item choice.
negative, non-numeric or `0` (never drawn — remove the item), and one other than `1`
where nothing is drawn against it: outside a choice, or on the item of a one-item choice.
### Repeat
@@ -439,8 +441,8 @@ that many times — each an independent draw — joined by `separator` (default
{ "format": "{word}", "repeat": 3, "separator": " ", "word": ["foo", "bar", "baz"] }
```
Renders e.g. `bar foo baz`. Rejected at load: a `separator` without a `repeat` above
`1`, and a `repeat` that multiplies to more than 1 048 576 renders along any path of
Renders e.g. `bar foo baz`. Rejected at load: a `separator` other than `""` without a
`repeat` above `1`, and a `repeat` that multiplies to more than 1 048 576 renders along any path of
nested repeats.
### Datatype
@@ -495,9 +497,9 @@ renders a null as `""`. The other items' weights skew its odds:
{ "format": "", "deleted_at": null, "middle": [null, { "format": "{n}", "n": ["Ann", "Eva"], "weight": 3 }] }
```
`deleted_at` is null every draw, `middle` a name three draws in four. Anywhere but a
column, `null` is `""`. Rejected at load: a column whose items hold different
datatypes.
`deleted_at` is null every draw, `middle` a name three draws in four. Rejected at
load: `null` anywhere but a column, where it only renders `""`, naming `""`, and a
column whose items hold different datatypes.
### Table
@@ -629,8 +631,10 @@ namespace too.
A `{name(args)}` token calls a builtin. Arguments are checked at `New`: a bad
count, range, country or expression fails fast; an integer may carry a sign or leading
zeros (`+5` and `05` are `5`); bounds are finite; a sample that could only ever emit one
value (`int(5,5)`, `float(1,1,2)`) is rejected naming the text to write instead;
zeros (`+5` and `05` are `5`); bounds are finite; a constant sample — `int` or `float`
with min equal to max, `date` with from equal to to and a layout with no clock field,
and a `date` or `time` layout naming no field — is rejected naming the text to write
instead;
and a length, count or decimal place beyond a sane maximum is rejected, so a
fat-fingered `hex(2000000000)` never tries to allocate gigabytes. Every builtin draws only from the seed — a
time-based id takes its timestamp from the rng, not the clock — so seeded output
cmd/fejkdata/main.go
+30 -11
View File
@@ -291,12 +291,15 @@ func operandError(arg string) error {
if fi, err := os.Stat(arg); err == nil && fi.Mode().IsRegular() {
return fmt.Errorf("%s; read a file from stdin: fejkdata < %s", unexpected, shellQuoted(arg))
}
example, ref := pipeHint, "{/"+strings.TrimPrefix(arg, "/")+"}"
if _, lone := loneReference(arg); lone {
ref := "{/" + strings.TrimPrefix(arg, "/") + "}"
_, argLone := loneReference(arg)
inline, err := fejkdata.IsTemplate(arg)
_, refLone := loneReference(ref)
example := pipeHint
switch {
case argLone || inline:
example = piped(arg)
} else if inline, err := fejkdata.IsTemplate(arg); inline {
example = piped(arg)
} else if _, lone := loneReference(ref); lone && err == nil {
case err == nil && refLone:
example = piped(ref)
}
return fmt.Errorf("%s; the template is read from stdin: %s", unexpected, example)
@@ -453,11 +456,8 @@ type input struct {
// allowed.
func parseInput(raw string) (input, error) {
trimmed := strings.Trim(raw, jsonSpace)
switch trimmed {
case "":
if trimmed == "" {
return input{}, fmt.Errorf("stdin holds no template: %s", pipeHint)
case "null":
return input{}, fmt.Errorf("stdin holds only null, as jq prints for a missing key: %s", pipeHint)
}
text := raw
if json.Valid([]byte(trimmed)) {
@@ -487,20 +487,39 @@ func loneReference(text string) (string, bool) {
}
// isPath reports whether a reference's body after its / is a path alone: no second /, no
// folder sigil, and none of the |, ( or " as " a token reads outside a selector.
// folder sigil, no empty segment, and none of the |, ( or " as " a token reads outside a selector.
// It copies the grammar, since the CLI reads only the library's public API.
func isPath(body string) bool {
if body == "" || strings.ContainsAny(body[:1], "/.") {
return false
}
names := withoutSelectors(body)
if strings.ContainsAny(names, "|(") || strings.Contains(names, " as ") {
if strings.ContainsAny(names, "|(") || strings.Contains(names, " as ") || hasEmptySegment(names) {
return false
}
inline, err := fejkdata.IsTemplate(body)
return !inline && err == nil
}
// hasEmptySegment reports whether a dotted path holds a segment with no name: two dots are a
// step up, and only a step up may end the path.
func hasEmptySegment(path string) bool {
segments := strings.Split(path, ".")
stepUp := false
for i, seg := range segments[1:] {
last := i == len(segments)-2
switch {
case seg != "":
stepUp = false
case stepUp && !last, !stepUp && last:
return true
default:
stepUp = !stepUp
}
}
return false
}
// spacedReference adds to a record refusal that whitespace around a lone reference made
// the template text.
func spacedReference(template string, err error) error {
datatype.go
+15 -15
View File
@@ -20,29 +20,26 @@ const (
DataTypeBoolean DataType = datatype.Boolean
)
// datatypeOf reads a template's "datatype" (default DataTypeString).
func datatypeOf(m map[string]any, pos position) (DataType, error) {
// datatypeOf reads a template's "datatype", nil where it carries none.
func datatypeOf(m map[string]any, pos position) (*DataType, error) {
v, ok := m["datatype"]
if !ok {
return DataTypeString, nil
return nil, nil
}
name, ok := v.(string)
if !ok {
return 0, fmt.Errorf("datatype must be a string, got %T", v)
return nil, fmt.Errorf("datatype must be a string, got %T", v)
}
if name == DataTypeString.String() {
return DataTypeString, nil
}
for d := DataTypeInteger; d < datatype.Count; d++ {
for d := DataTypeString; d < datatype.Count; d++ {
if name != d.String() {
continue
}
if pos != inColumn {
return 0, errors.New("datatype only types a record column — a field of the top-level template — so it has no effect here")
if pos != inColumn && d != DataTypeString {
return nil, errors.New(`datatype only types a record column — a field of the top-level template — so it has no effect here; drop "datatype"`)
}
return d, nil
return &d, nil
}
return 0, fmt.Errorf(`datatype takes "string", "integer", "number" or "boolean", got %q`, name)
return nil, fmt.Errorf(`datatype takes "string", "integer", "number" or "boolean", got %q`, name)
}
// checkColumns rejects a record column whose items hold different datatypes, checking a column
@@ -100,10 +97,13 @@ func columnDatatype(n node) DataType {
// itemDatatype is the datatype a column item declares, else that of the column it reads.
func itemDatatype(t *template) DataType {
if !t.declared && t.readsColumn != nil {
switch {
case t.datatype != nil:
return *t.datatype
case t.readsColumn != nil:
return columnDatatype(t.readsColumn.column)
}
return t.datatype
return DataTypeString
}
// columnItems is a column's template items, its choices unwrapped, and whether one is null.
@@ -139,7 +139,7 @@ const (
func kindOf(t *template) itemKind {
switch {
case t.declared:
case t.datatype != nil:
return kindDeclares
case itemDatatype(t) != DataTypeString:
return kindReads
namefence.go
+5 -1
View File
@@ -77,6 +77,7 @@ func refuseTwiceDrawnIn(addressed map[pickKey]string, read string, t *template,
reads := map[string]int{}
var into []arm
for _, o := range t.compiled.ops {
heads := map[string]bool{} // one op draws once, whichever arm it takes
for _, a := range slices.Concat(o.arms, o.operands) {
if grammar.IsRef(a.head) || a.kind == namedRead {
continue
@@ -84,7 +85,10 @@ func refuseTwiceDrawnIn(addressed map[pickKey]string, read string, t *template,
if _, kept := addressed[key.under(a.key())]; kept {
into = append(into, a)
}
reads[a.head]++
heads[a.head] = true
}
for head := range heads {
reads[head]++
}
}
for _, head := range sortedNames(reads) {
node.go
+20 -15
View File
@@ -51,8 +51,7 @@ type template struct {
fields map[string]node
repeat int
separator string
datatype DataType
declared bool // carries a "datatype", which wins over the datatype of a column it reads
datatype *DataType // the "datatype" it carries, which wins over that of a column it reads; nil where none
fromString bool // written as a JSON string rather than an object
isRecord bool // compiled at the top without a repeat, so its fields are record columns
unbound []unboundRead // the heads its tokens read that no field holds
@@ -116,13 +115,15 @@ const (
inColumn // a column, or a choice item standing in for one
)
// compileAt compiles a node that is no choice's item. Only a choice's items carry a
// weight, so one here would be inert whatever its type.
// compileAt compiles a node drawn against no other: no choice's item, or a one-item choice's.
// A weight there does nothing, bar the default 1 written out.
func compileAt(v any, pos position) (node, error) {
if m, ok := v.(map[string]any); ok {
if _, weighted := m["weight"]; weighted {
return nil, fmt.Errorf("weight only skews a choice's items, so it has no effect here; it is an option and can never be a field")
w, err := weightOf(v)
if err != nil {
return nil, err
}
if w != 1 {
return nil, fmt.Errorf(`weight %v skews a draw between items, and nothing here is drawn against this one; drop "weight"`, w)
}
return compileItem(v, pos)
}
@@ -142,7 +143,7 @@ func compileItem(v any, pos position) (node, error) {
return compileTemplate(v, pos)
case nil:
if pos != inColumn {
return compileString("")
return nil, fmt.Errorf(`null is a record column's value, and here it only renders "", so write ""`)
}
return &nullItem{}, nil
default:
@@ -158,6 +159,12 @@ func jsonKind(v any) string {
return "a number"
case bool:
return "a boolean"
case string:
return "a string"
case []any:
return "a list"
case map[string]any:
return "an object"
}
return fmt.Sprintf("%T", v)
}
@@ -242,13 +249,12 @@ func compileTemplate(m map[string]any, pos position) (node, error) {
if err != nil {
return nil, err
}
return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, declared: o.declared, isRecord: fieldPos == inColumn, unbound: unbound}, nil
return &template{format: o.format, tokens: toks, fields: fields, repeat: o.repeat, separator: o.separator, datatype: o.datatype, isRecord: fieldPos == inColumn, unbound: unbound}, nil
}
// templateOptions is what a template object's option keys say.
type templateOptions struct {
datatype DataType
declared bool
datatype *DataType
format string
repeat int
separator string
@@ -266,7 +272,6 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) {
return o, err
}
o.repeat = repeat
_, o.declared = m["datatype"]
if o.datatype, err = datatypeOf(m, pos); err != nil {
return o, err
}
@@ -274,8 +279,8 @@ func readOptions(m map[string]any, pos position) (templateOptions, error) {
if o.separator, ok = sv.(string); !ok {
return o, fmt.Errorf("separator must be a string, got %T", sv)
}
if repeat == 1 {
return o, fmt.Errorf("separator joins repeated renders, so it has no effect without a repeat above 1")
if repeat == 1 && o.separator != "" {
return o, fmt.Errorf(`separator %q joins repeated renders, and this template renders once; drop "separator", or add a repeat above 1`, o.separator)
}
}
return o, nil
@@ -332,7 +337,7 @@ func weightOf(raw any) (float64, error) {
}
w, ok := wv.(float64)
if !ok {
return 0, fmt.Errorf("weight must be a number, got %T", wv)
return 0, fmt.Errorf(`weight is an option and takes a number, not %s; a field cannot be named "weight"`, jsonKind(wv))
}
if w < 0 || math.IsNaN(w) || math.IsInf(w, 0) {
return 0, fmt.Errorf("weight must be finite and positive, got %v", w)
resolve.go
+1 -1
View File
@@ -104,7 +104,7 @@ type columnRead struct {
// or name, and nil where it does not or t declares a string, which reads the column as text.
func columnReadOf(t *template, targets map[*nameBinding]nameTarget) *columnRead {
ops := t.compiled.ops
if t.declared && t.datatype == DataTypeString || t.repeat != 1 || len(ops) != 1 || ops[0].Kind != grammar.PathRead || len(ops[0].arms) != 1 {
if t.datatype != nil && *t.datatype == DataTypeString || t.repeat != 1 || len(ops) != 1 || ops[0].Kind != grammar.PathRead || len(ops[0].arms) != 1 {
return nil
}
a := ops[0].arms[0]
spellings_test.go
+1 -1
View File
@@ -86,7 +86,7 @@ func TestADefaultWrittenOutIsTheDefault(t *testing.T) {
} {
sameRenders(t, long, short)
}
if r := compiled(t, `{"format":"","n":{"format":"1","datatype":"string"}}`).(*template); !r.isRecord || r.fields["n"].(*template).datatype != DataTypeString {
if r := compiled(t, `{"format":"","n":{"format":"1","datatype":"string"}}`).(*template); !r.isRecord || *r.fields["n"].(*template).datatype != DataTypeString {
t.Error("a column of datatype string is no string column of a record")
}
}
valueproof.go
+14 -13
View File
@@ -19,31 +19,32 @@ type valueProof struct {
// checkDatatype rejects a typed column item some render of which is not text of its datatype.
func (p *valueProof) checkDatatype(label string, n node) error {
t, ok := n.(*template)
if !ok || t.datatype == DataTypeString {
if !ok || t.datatype == nil || *t.datatype == DataTypeString {
return nil
}
if reason := p.proveColumnItem(t).Not[t.datatype]; reason != "" {
return fmt.Errorf("%s: datatype %s: %s", label, t.datatype, reason)
if reason := p.proveColumnItem(t).Not[*t.datatype]; reason != "" {
return fmt.Errorf("%s: datatype %s: %s", label, *t.datatype, reason)
}
return nil
}
// checkField rejects a column a field of Go type ft cannot fill: a datatype, which the Go type
// sets, a null outside a pointer, or a value its kind's datatype or range refuses.
// checkField rejects a column a field of Go type ft cannot fill: a datatype other than the one the
// Go type sets, a null outside a pointer, or a value its kind's datatype or range refuses.
func (p *valueProof) checkField(label string, ft reflect.Type, column node) error {
items, _ := columnItems(column)
for _, it := range items {
if it.declared {
return fmt.Errorf("%s: its Go type %s sets the datatype; drop \"datatype\"", label, ft)
}
}
elem := ft
if ft.Kind() == reflect.Pointer {
elem = ft.Elem()
} else if p.proveColumn(column).Nullable {
return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft)
}
kind := columnKinds[elem.Kind()]
items, _ := columnItems(column)
for _, it := range items {
if it.datatype != nil && *it.datatype != kind.datatype {
return fmt.Errorf("%s: its Go type %s sets the datatype %s; drop \"datatype\"", label, ft, kind.datatype)
}
}
if ft.Kind() != reflect.Pointer && p.proveColumn(column).Nullable {
return fmt.Errorf("%s: its tag can draw null, which %s cannot hold; make it *%s", label, ft, ft)
}
if kind.datatype == DataTypeString {
return nil
}