Add the table node: rows TSV, key and name selection, linked tables drawn consistently, and the choice-of-rows fence
This commit is contained in:
@@ -9,3 +9,15 @@ replacement, and each removed path, column or flag.
|
||||
### Added
|
||||
|
||||
- First release: the CLI, the library and the shipped data set.
|
||||
- Table categories: a category JSON naming a `rows` TSV beside it, with the options
|
||||
`key`, `name`, `weight` and `parent`; a path selects a row by key or name,
|
||||
`misc.country[SE]`, and descends to a linked table by name; linked tables draw
|
||||
consistently within one render and draw group. `rows` is an option, so no
|
||||
template may carry a field of that name.
|
||||
- `New` refuses a root choice of templates sharing one format and one set of string
|
||||
fields, naming the rows TSV to write instead.
|
||||
- `misc.country`, `misc.currency`, `misc.language`, `misc.httpstatus` and
|
||||
`misc.mimetype` are tables. `misc.country` is the full ISO 3166 register with the
|
||||
columns `calling-code`, `capital`, `currency`, `flag`, `languages`, `numeric` and
|
||||
`tld` added; `misc.currency` the current ISO 4217 list with `decimals` and
|
||||
`numeric` added, and its symbols from CLDR. `DATA-LICENSES.md` lists each source.
|
||||
|
||||
@@ -17,6 +17,7 @@ fejkdata sv_SE.person.last # Eriksson
|
||||
fejkdata --seed 42 sv_SE.address # the same address every run
|
||||
fejkdata -n 3 --separator ', ' sv_SE.word # nät, barn, sol
|
||||
fejkdata --list # every path the data offers
|
||||
fejkdata 'misc.country[SE].capital' # Stockholm — a table's row, selected by key or name
|
||||
fejkdata --data-path ./mydata sv_SE.word # layer a directory over the shipped data
|
||||
fejkdata --no-shipped-data -d ./mydata --list # only your data
|
||||
fejkdata 'name: {/sv_SE.person.last}' # name: <a surname> — an inline template
|
||||
@@ -24,7 +25,8 @@ fejkdata '{"format":"name: {x}","x":["bosse","lina"]}' # name: bosse or name: l
|
||||
```
|
||||
|
||||
A path names a category, or a field inside one: each dot segment descends one
|
||||
level — folders, then the category (a JSON file), then fields. An argument that is
|
||||
level — folders, then the category (a JSON file), then fields — and `[SE]` after a
|
||||
[table](#table) selects its row. An argument that is
|
||||
a JSON object, array or string, or that carries a `{` token, is instead an
|
||||
**inline template**: a format string or a JSON value compiled and rendered on the
|
||||
spot. Its tokens reach the data by reference from the root —
|
||||
@@ -32,8 +34,8 @@ spot. Its tokens reach the data by reference from the root —
|
||||
available. An inline template sits in no folder, so the folder-relative `{.name}`
|
||||
and `{..name}` are rejected naming the root spelling, and one reference alone —
|
||||
`{/sv_SE.person}` — is the path written as a template, rejected naming the path, as is
|
||||
a path written `/sv_SE.person`. A path never contains a brace, a bracket or a quote,
|
||||
so the two cannot collide (see [Decisions](#decisions)).
|
||||
a path written `/sv_SE.person`. A path never contains a brace or a quote, and a
|
||||
bracket only as a selector after a name, so the two cannot collide (see [Decisions](#decisions)).
|
||||
|
||||
| Flag | |
|
||||
|------|--|
|
||||
@@ -220,7 +222,10 @@ Each locale carries `address`, `color`, `company`, `date`, `email`, `ip`,
|
||||
`country` (ISO 3166), `creditcard` (Luhn-valid), `currency` (ISO 4217), `emoji`,
|
||||
`httpstatus`, `language` (ISO 639), `mac`, `mimetype`, `objectid`, `timezone`
|
||||
(IANA), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
|
||||
`misc.country.alpha2`, `misc.httpstatus.code` — which `--list` shows.
|
||||
`misc.country.alpha2`, `misc.httpstatus.code` — which `--list` shows. `country`,
|
||||
`currency`, `httpstatus`, `language` and `mimetype` are [tables](#table), so
|
||||
`misc.country[SE].capital` and `misc.currency[Euro].symbol` select a row;
|
||||
[`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence.
|
||||
|
||||
## Data format
|
||||
|
||||
@@ -231,6 +236,7 @@ Every value is a **node**, nestable without limit:
|
||||
| string | `"Malmö"` | its text, with any `{…}` tokens expanded |
|
||||
| choice | `["a", "b", …]` | one item, picked at random |
|
||||
| template | `{"format": "…", …}` | its format, with `{name}` tokens rendering the named fields |
|
||||
| table | `{"format": "…", "rows": "x.tsv", …}` | its format over one row of the TSV beside it ([Table](#table)) |
|
||||
|
||||
### Format string
|
||||
|
||||
@@ -340,10 +346,105 @@ renders a null as `""`. The other items' weights skew its odds:
|
||||
load: `null` anywhere but a column, naming `""`, and a column whose items hold
|
||||
different datatypes.
|
||||
|
||||
### Table
|
||||
|
||||
A table is a category whose rows come from a TSV beside its JSON file: the header
|
||||
names the columns, each line below it is one row, and the format renders the row
|
||||
drawn. Save `mydata/country.tsv` and `mydata/country.json`:
|
||||
|
||||
```tsv
|
||||
alpha2 name population
|
||||
DK Denmark 5900000
|
||||
NO Norway 5500000
|
||||
SE Sweden 10500000
|
||||
```
|
||||
|
||||
```json
|
||||
{ "format": "{name} ({alpha2})", "rows": "country.tsv", "key": "alpha2", "name": "name", "weight": "population" }
|
||||
```
|
||||
|
||||
```sh
|
||||
fejkdata -d ./mydata country # Sweden (SE), about half the time
|
||||
fejkdata -d ./mydata country.alpha2 # NO
|
||||
fejkdata -d ./mydata 'country[SE]' # Sweden (SE)
|
||||
fejkdata -d ./mydata 'country[Norway].alpha2' # NO
|
||||
fejkdata -d ./mydata --format csv country # alpha2,name,population → SE,Sweden,10500000
|
||||
```
|
||||
|
||||
`rows` names the TSV beside the category file; `key` names the column a path
|
||||
selects a row by, `name` a column it also selects by, `weight` a column of positive
|
||||
numbers that skews the draw, and `parent` the table a column links to
|
||||
([Linked tables](#linked-tables)). The format's `{tokens}` read the columns, and the
|
||||
columns are the [record](#records)'s columns, so `--format csv` writes the rows and
|
||||
`--list` shows `country.alpha2`. A cell is a string node: `1{digits(2)} {digits(2)}`
|
||||
in a cell draws digits and `{/misc.uuid}` reads a reference, while `{name}` in a cell
|
||||
is refused, since a cell has no sibling. `New` proves the header, the options and every
|
||||
cell token, and refuses a TSV no category names, a key that is empty or repeats, a
|
||||
weight that is not a positive number, and a key or name holding `[`, `]`, `{`, `}`,
|
||||
`"` or `|`, which a selector cannot spell; the rows are indexed on the first draw that
|
||||
selects one. The table's options are its own — `rows`, `key`, `name`, `weight` and
|
||||
`parent` — so a column may be named `name`, as one usually is.
|
||||
|
||||
A choice of templates sharing one format and one set of string fields is a table
|
||||
written by hand, and `New` refuses it in a data file naming the TSV to write; an
|
||||
inline template has no file beside it, so there it stays a choice.
|
||||
|
||||
### Row selection
|
||||
|
||||
`[key]` or `[name]` after a table's name selects one row: `misc.country[SE]` and
|
||||
`misc.country[Sweden]` name one row, and `misc.country[SE].capital` reads its column.
|
||||
A key wins over a name that spells the same, and a name naming several rows is an
|
||||
error listing their keys, unless a row selected before it settles which
|
||||
([Linked tables](#linked-tables)). A selector is part of the path, so it works
|
||||
wherever a path does: `Fake`, `FakeRecord`, a `{/misc.country[SE].capital}` reference
|
||||
and a struct tag. A dot inside the brackets belongs to the key or name, so
|
||||
`city[St. Louis]` selects it. A path starts with a name, and `[` still opens a JSON
|
||||
array at the start of a CLI argument, so `'[SE]'` alone names nothing.
|
||||
|
||||
### Linked tables
|
||||
|
||||
A table's `parent` names a column and, by the same name, the table beside it that the
|
||||
column links to by key. Save `mydata/city.tsv` and `mydata/city.json` beside the
|
||||
`country` table above:
|
||||
|
||||
```tsv
|
||||
name country population
|
||||
Copenhagen DK 660000
|
||||
Göteborg SE 600000
|
||||
Oslo NO 710000
|
||||
Stockholm SE 990000
|
||||
```
|
||||
|
||||
```json
|
||||
{ "format": "{name}", "rows": "city.tsv", "key": "name", "parent": "country", "weight": "population" }
|
||||
```
|
||||
|
||||
```sh
|
||||
fejkdata -d ./mydata 'country[SE].city' # Stockholm or Göteborg
|
||||
fejkdata -d ./mydata country.city.name # a country drawn, then a city inside it
|
||||
fejkdata -d ./mydata 'city[Oslo].country' # NO — the link column's cell
|
||||
fejkdata -d ./mydata '{/city.name}, {/country.name}' # Oslo, Norway — one consistent draw
|
||||
```
|
||||
|
||||
A path descends from a row to a linked table by name, at any depth, and `--list`
|
||||
advertises each direct step. Within one render and [draw group](#draw-group), linked
|
||||
tables agree: the first table a reference path reads pins its ancestors, and a
|
||||
descendant read after it is drawn inside them, so `{/city.name}` and
|
||||
`{/country.name}` are a city and its country whichever is read first. A selected row
|
||||
pins the render the same way, so every reference path into one family of linked
|
||||
tables in one render and group selects the same rows: one that selects none beside
|
||||
one that does is refused naming the spelling that does, `{/country[SE].city.name}`
|
||||
beside `{/country[SE].name}`, and two selecting different rows are refused naming a
|
||||
`drawGroup` to draw them apart in. A bare `{/city}` beside a path into its family is
|
||||
refused too, since a bare reference draws each time. `New` also refuses a link cell
|
||||
that is no key of the parent, a parent row no child links to, a chain of parents that
|
||||
closes, and a child named like one of its parent's columns.
|
||||
|
||||
### Options and fields
|
||||
|
||||
`format`, `weight`, `repeat`, `separator`, `datatype` and `drawGroup` are the only options;
|
||||
**any other key is a field** (see [Decisions](#decisions)). An object that does nothing a
|
||||
**any other key is a field** (see [Decisions](#decisions)), and `rows` makes a category
|
||||
a [table](#table), so no template carries a field of that name. An object that does nothing a
|
||||
string can't — only a `format` — is rejected naming the string, as is a one-item
|
||||
choice naming its item.
|
||||
|
||||
@@ -470,10 +571,11 @@ its groups by name; the unnamed group spans them all.
|
||||
Renders e.g. `Sara Eriksson pays Ebba Lind; signed Eriksson`: the signature reads the
|
||||
payer's draw, while the payee is drawn apart. Rejected at load, each naming nothing: a
|
||||
`drawGroup` of `""` (the default); one naming the draw group its template already draws
|
||||
in; one on a template that renders no reference path short of a `repeat` or a nested
|
||||
`drawGroup`, on a `repeat` itself — each iteration renders in no draw group — or on an
|
||||
inline template's root, which nothing references. So is a path reading into a level that
|
||||
carries a `drawGroup`.
|
||||
in; one on a template that renders no reference path — a bare reference to a
|
||||
[table](#table) counts, since the group answers for the family it draws in — short of a
|
||||
`repeat` or a nested `drawGroup`, on a `repeat` itself — each iteration renders in no
|
||||
draw group — or on an inline template's root, which nothing references. So is a path
|
||||
reading into a level that carries a `drawGroup`.
|
||||
|
||||
### Correlated fields
|
||||
|
||||
@@ -538,7 +640,7 @@ a minor only adds, and a major is the only release that changes what exists.
|
||||
|
||||
| Surface | Major | Minor |
|
||||
|---------|-------|-------|
|
||||
| Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another | a path outside a record's columns, a locale, a value in a list |
|
||||
| Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row |
|
||||
| Records | remove, rename, retype or add a column; let a column be null | a record, as a new category |
|
||||
| Data format | a fence: a spelling `New` rejects that it accepted; a template option, since it reserves a field name | a builtin |
|
||||
| CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format |
|
||||
@@ -559,8 +661,8 @@ with no breaking change. From `v2` the module path carries `/vN`, so fences ship
|
||||
batched into as few majors as possible.
|
||||
|
||||
[`testdata/shipped_shape.txt`](testdata/shipped_shape.txt) pins every path, each
|
||||
template category's format, the categories each category reads, and each column's
|
||||
datatype and nullability; a pull request that
|
||||
template category's format, the categories each category reads, each column's
|
||||
datatype and nullability, and each table's key, name, weight and parent columns; a pull request that
|
||||
changes it or `data/` adds its `CHANGELOG.md` entry, which CI checks. A removed,
|
||||
renamed or retyped line is a major.
|
||||
|
||||
@@ -796,6 +898,52 @@ renamed or retyped line is a major.
|
||||
almost always costs an allocation too (a lost pre-size, a per-item map, an extra
|
||||
copy). The benchmark suite (see Development) reports time for a human, not as a
|
||||
pass/fail gate.
|
||||
- **Rows live in a TSV, the shape in JSON.** `New` allocates once per node, so a
|
||||
register of thirty thousand rows written as JSON objects would cost it a second;
|
||||
a TSV is one allocation whose cells are substrings, and the JSON says only how a
|
||||
row is composed. The TSV sits beside its category file, named by `rows`, so a
|
||||
data directory stays a directory of categories, and one nothing names is a
|
||||
load error rather than a file silently ignored.
|
||||
- **A selector is bracketed, and a dot inside it is literal.** `municipality[0180]`
|
||||
reads as selection to anyone who has indexed an array, and `[St. Louis]` keeps a
|
||||
name whole where a colon or a dot-separated spelling could not; zsh needs the
|
||||
brackets quoted, which the README's examples show. A key wins over a name that
|
||||
spells the same, since a key names one row by contract and a name may not.
|
||||
- **A table read into is pinned; a table rendered whole draws afresh.** A path into a
|
||||
table pins its row for the render and group, as a reference path pins its level,
|
||||
and a bare `{/city}` draws each time, as a bare reference does; so a bare table
|
||||
beside a path into its family is refused like a bare reference beside a path into
|
||||
it. A bare table reference still counts as a read for a `drawGroup`, since the group
|
||||
is what draws it apart from the family's pins.
|
||||
- **Every reference path into one family selects the same rows, per render and
|
||||
group.** A selector pins rows for the render, and a read that draws freely before it
|
||||
could pin a row the selector contradicts, so accepting both would make the result
|
||||
depend on which token rendered first. Requiring one selection per family per group
|
||||
is checkable at load with no data lookup beyond the selectors themselves, and the
|
||||
error names the rewrite. Two selectors naming one row by key and by name compare
|
||||
equal, since the load check resolves them.
|
||||
- **A parent row with no child row is a load error.** A descendant is drawn inside the
|
||||
nearest pinned ancestor, so every ancestor row must lead to a row at every level
|
||||
below it, or a render could find nothing to draw. The import script drops or fills
|
||||
such rows; the alternative, falling back to a free draw, would break the consistency
|
||||
the link exists for without saying so.
|
||||
- **The choice-of-rows fence guards a data file's root, and requires string fields.**
|
||||
A table is a category with a TSV beside its file, so only a root choice has the
|
||||
spelling the fence names; a nested choice of same-shaped templates and an inline
|
||||
one keep loading. Fields must all be strings because a cell is a string node: a
|
||||
choice whose items carry a nested choice, as `misc.car` does, is not one table but
|
||||
two linked ones, which a later conversion writes.
|
||||
- **A table is a record of string columns.** Its columns are the CSV header and the
|
||||
`INSERT` column list, fixed by the TSV header, so a table is a record by
|
||||
construction; every column is a string until a typed column option earns its place.
|
||||
- **The key index is built at load, the rest on first draw.** A link is proved
|
||||
against the parent's keys and a key's uniqueness is a data mistake, so both are
|
||||
load-time; the name index and the per-parent child lists serve only a draw or a
|
||||
selection, so they wait for the first one, keeping `New` linear in the bytes read.
|
||||
- **`List` advertises direct descents only.** `region.municipality.locality` is
|
||||
listed, and `region.locality` resolves too but is not: the set of every descent
|
||||
through a chain of five tables is every subsequence of it, and the direct chain is
|
||||
the one a reader can predict from the tables' parents.
|
||||
|
||||
## Development
|
||||
|
||||
@@ -839,6 +987,15 @@ in its own commit:
|
||||
REPIN=1 docker compose run --rm --user "$(id -u):$(id -g)" test
|
||||
```
|
||||
|
||||
A shipped table built from a source is rebuilt by its script under
|
||||
[`data-import/`](data-import), one command per dataset, fetching the source named in
|
||||
[`DATA-LICENSES.md`](DATA-LICENSES.md):
|
||||
|
||||
```sh
|
||||
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/country.py
|
||||
docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/currency.py
|
||||
```
|
||||
|
||||
To release, head `CHANGELOG.md` with the version's section in place of `Unreleased`
|
||||
and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publishes
|
||||
the Gitea release with the section as its body. A top heading of `[Unreleased]`
|
||||
@@ -849,7 +1006,8 @@ publishes nothing.
|
||||
```
|
||||
fejkdata.go Generator, New, options, the embedded data set, List
|
||||
node.go the node model and JSON -> node compilation
|
||||
path.go the dotted-path walk, and proving a path resolves
|
||||
table.go tables: the rows TSV, its options and links, row selection and draws
|
||||
path.go the dotted-path walk with its selectors, and proving a path resolves
|
||||
render.go Fake and the recursive renderer (choices, format strings, expansions)
|
||||
record.go records: Record, the JSON/CSV/SQL serializers, and their entry points
|
||||
struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype
|
||||
@@ -865,7 +1023,8 @@ datatype.go column datatypes: DataType, where datatype and null may sit, a c
|
||||
value.go the value proof: what a typed column or calc operand holds, checked at load
|
||||
data.go data loading: fs.FS folders/files -> namespace tree, multi-source merge
|
||||
cmd/fejkdata/ the fejkdata CLI
|
||||
data/ shipped data (JSON), embedded at build: locale folders + a misc folder
|
||||
data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders + a misc folder
|
||||
data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md)
|
||||
release-tooling/ the release CI publishes from the changelog heading
|
||||
testdata/ the pinned shipped shape (see Versioning)
|
||||
```
|
||||
|
||||
+19
-7
@@ -23,16 +23,17 @@ import (
|
||||
|
||||
const usage = `Usage: fejkdata [flags] <path|template>
|
||||
|
||||
<path> a category, or a dotted path into one (person, person.last)
|
||||
<path> a category, or a dotted path into one (person, person.last);
|
||||
a table's row by key or name: 'misc.country[SE]', 'misc.country[Sweden].tld'
|
||||
<template> a format string or JSON value to render inline, e.g.
|
||||
'name: {/sv_SE.person.last}' or '{"format":"{x}","x":["bosse","lina"]}'
|
||||
|
||||
An argument containing a { token, or a JSON object, array or string, is a
|
||||
template; any other argument is a path (a path never contains a brace, a bracket
|
||||
or a quote). Templates reach the data by reference from the root —
|
||||
{/sv_SE.person.last} — whether the data is shipped or layered with --data-path. An
|
||||
argument carrying a bracket, a closing brace or a quote but no valid JSON names
|
||||
neither.
|
||||
template; any other argument is a path (a path never contains a brace or a quote,
|
||||
and a bracket only as a [selector] after a table's name). Templates reach the
|
||||
data by reference from the root — {/sv_SE.person.last} — whether the data is
|
||||
shipped or layered with --data-path. An argument carrying a closing brace or a
|
||||
quote but no valid JSON names neither.
|
||||
|
||||
With --format json, ndjson, csv or sql the argument must name a record — a
|
||||
template whose fields are its columns — and the rows are written as one JSON
|
||||
@@ -396,7 +397,18 @@ func defaultTable(arg string, kind argKind) string {
|
||||
if kind == argTemplate {
|
||||
return "records"
|
||||
}
|
||||
segments := strings.Split(arg, ".")
|
||||
var names strings.Builder // the path with its [selectors] cut out
|
||||
for depth, i := 0, 0; i < len(arg); i++ {
|
||||
switch {
|
||||
case arg[i] == '[':
|
||||
depth++
|
||||
case arg[i] == ']':
|
||||
depth--
|
||||
case depth == 0:
|
||||
names.WriteByte(arg[i])
|
||||
}
|
||||
}
|
||||
segments := strings.Split(names.String(), ".")
|
||||
return segments[len(segments)-1]
|
||||
}
|
||||
|
||||
|
||||
@@ -62,6 +62,9 @@ func loadData(sources []dataSource) (map[string]node, error) {
|
||||
if len(root) == 0 {
|
||||
return nil, fmt.Errorf("no .json data found")
|
||||
}
|
||||
if err := linkTables(root); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := linkRefs(root); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -83,22 +86,56 @@ func loadDir(src dataSource, dir string) (*folder, error) {
|
||||
return nil, fmt.Errorf("%s: %w", src.name(dir), err)
|
||||
}
|
||||
g := &folder{children: map[string]node{}}
|
||||
files := &categoryFiles{src: src, dir: dir, tsv: map[string]bool{}}
|
||||
for _, e := range entries {
|
||||
if strings.HasSuffix(e.Name(), ".tsv") && !e.IsDir() {
|
||||
files.tsv[e.Name()] = false
|
||||
}
|
||||
}
|
||||
for _, e := range entries {
|
||||
if strings.HasPrefix(e.Name(), ".") { // hidden: a checkout or an editor's file, never data
|
||||
continue
|
||||
}
|
||||
full := path.Join(dir, e.Name())
|
||||
load := loadFile
|
||||
if e.IsDir() {
|
||||
load = loadFolder
|
||||
if err := loadFolder(src, g, full, e.Name()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := load(src, g, full, e.Name()); err != nil {
|
||||
continue
|
||||
}
|
||||
if err := loadFile(src, g, full, e.Name(), files); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
for name, named := range files.tsv {
|
||||
if !named && !strings.HasPrefix(name, ".") {
|
||||
return nil, fmt.Errorf("%s: no category names it in its rows; a table's rows file sits beside a category file naming it", src.name(path.Join(dir, name)))
|
||||
}
|
||||
}
|
||||
return g, nil
|
||||
}
|
||||
|
||||
// categoryFiles is what a category may name beside itself: the rows files of its
|
||||
// directory, each marked once a category names it.
|
||||
type categoryFiles struct {
|
||||
src dataSource
|
||||
dir string
|
||||
tsv map[string]bool
|
||||
}
|
||||
|
||||
// rows reads the rows file a category names beside it.
|
||||
func (c *categoryFiles) rows(name string) (string, error) {
|
||||
if _, present := c.tsv[name]; !present {
|
||||
return "", fmt.Errorf("rows names %s, which is not beside it in %s", name, c.src.name(c.dir))
|
||||
}
|
||||
c.tsv[name] = true
|
||||
b, err := fs.ReadFile(c.src.fsys, path.Join(c.dir, name))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("%s: %w", c.src.name(path.Join(c.dir, name)), err)
|
||||
}
|
||||
return string(b), nil
|
||||
}
|
||||
|
||||
// loadFolder adds a subdirectory as a nested folder, unless nothing under it is data.
|
||||
func loadFolder(src dataSource, g *folder, full, name string) error {
|
||||
child, err := loadDir(src, full)
|
||||
@@ -117,7 +154,7 @@ func loadFolder(src dataSource, g *folder, full, name string) error {
|
||||
|
||||
// loadFile compiles a *.json file into a category named after it; any other file
|
||||
// is skipped.
|
||||
func loadFile(src dataSource, g *folder, full, file string) error {
|
||||
func loadFile(src dataSource, g *folder, full, file string, files *categoryFiles) error {
|
||||
if !strings.HasSuffix(file, ".json") {
|
||||
return nil
|
||||
}
|
||||
@@ -133,7 +170,7 @@ func loadFile(src dataSource, g *folder, full, file string) error {
|
||||
if err := json.Unmarshal(b, &raw); err != nil {
|
||||
return fmt.Errorf("%s: %w", src.name(full), err)
|
||||
}
|
||||
n, err := compile(raw)
|
||||
n, err := compileCategory(raw, name, files)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s: %w", src.name(full), err)
|
||||
}
|
||||
|
||||
@@ -13,21 +13,127 @@ type drawSet struct {
|
||||
}
|
||||
|
||||
// drawScope is where a render reads its reference paths: a draw set, in the draw group of the
|
||||
// template rendering.
|
||||
// template rendering, and the row a table's format is rendering, which its columns read.
|
||||
type drawScope struct {
|
||||
set *drawSet
|
||||
group string
|
||||
t *table
|
||||
row int
|
||||
}
|
||||
|
||||
// pinned is the row the render pinned for t, if any.
|
||||
func (d *draws) pinned(t *table) (int, bool) {
|
||||
for _, p := range d.pins[:d.npins] {
|
||||
if p.t == t {
|
||||
return p.row, true
|
||||
}
|
||||
}
|
||||
r, ok := d.more[t]
|
||||
return r, ok
|
||||
}
|
||||
|
||||
// mustRow is the row pinned for t, which the walk reaching a column pinned.
|
||||
func (d *draws) mustRow(t *table) int {
|
||||
r, ok := d.pinned(t)
|
||||
if !ok {
|
||||
panic(fmt.Sprintf("fejkdata: a column of %s is rendered with no row pinned", t.category))
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// pin pins row r of t, and the rows of t's ancestors it links to.
|
||||
func (d *draws) pin(t *table, r int) {
|
||||
for {
|
||||
if _, done := d.pinned(t); done {
|
||||
return
|
||||
}
|
||||
if d.npins < len(d.pins) {
|
||||
d.pins[d.npins] = tablePin{t, r}
|
||||
d.npins++
|
||||
} else {
|
||||
if d.more == nil {
|
||||
d.more = map[*table]int{}
|
||||
}
|
||||
d.more[t] = r
|
||||
}
|
||||
if t.parentT == nil {
|
||||
return
|
||||
}
|
||||
t, r = t.parentT, t.parentRow(r)
|
||||
}
|
||||
}
|
||||
|
||||
// rowOf is the render's row of t: the one pinned, else one drawn inside the
|
||||
// nearest pinned ancestor — its parent drawn inside that first where the ancestor
|
||||
// is further up — or over the whole table, and pinned with its ancestors.
|
||||
func (d *draws) rowOf(t *table) int {
|
||||
if r, ok := d.pinned(t); ok {
|
||||
return r
|
||||
}
|
||||
r := -1
|
||||
for a := t.parentT; a != nil && r < 0; a = a.parentT {
|
||||
if _, ok := d.pinned(a); !ok {
|
||||
continue
|
||||
}
|
||||
if t.parentT != a {
|
||||
d.rowOf(t.parentT)
|
||||
}
|
||||
pr, _ := d.pinned(t.parentT)
|
||||
r = t.drawUnder(d.s, pr)
|
||||
}
|
||||
if r < 0 {
|
||||
r = t.draw(d.s)
|
||||
}
|
||||
d.pin(t, r)
|
||||
return r
|
||||
}
|
||||
|
||||
// selectRow pins the row a selector names, refusing one outside the rows pinned
|
||||
// before it.
|
||||
func (d *draws) selectRow(t *table, sel string) error {
|
||||
r, err := t.find(sel, d)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if pr, ok := d.pinned(t); ok && pr != r {
|
||||
return fmt.Errorf("%s is not %s, the row already drawn", t.selectorSpelling(r), t.selectorSpelling(pr))
|
||||
}
|
||||
for a := t.parentT; a != nil; a = a.parentT {
|
||||
if pa, ok := d.pinned(a); ok && !t.under(r, a, pa) {
|
||||
return fmt.Errorf("%s is not inside %s", t.selectorSpelling(r), a.selectorSpelling(pa))
|
||||
}
|
||||
}
|
||||
d.pin(t, r)
|
||||
return nil
|
||||
}
|
||||
|
||||
// inside keeps the rows of t that sit inside every pinned ancestor.
|
||||
func (d *draws) inside(t *table, rows []int) []int {
|
||||
for a := t.parentT; a != nil; a = a.parentT {
|
||||
pa, ok := d.pinned(a)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
var kept []int
|
||||
for _, r := range rows {
|
||||
if t.under(r, a, pa) {
|
||||
kept = append(kept, r)
|
||||
}
|
||||
}
|
||||
rows = kept
|
||||
}
|
||||
return rows
|
||||
}
|
||||
|
||||
// newDrawSet makes the unnamed group's maps where the set is declared, keeping them on that frame's
|
||||
// stack for a render that reads through them; a zero drawSet makes them on its first read instead.
|
||||
func newDrawSet() drawSet {
|
||||
return drawSet{unnamed: draws{variant: map[string]node{}, value: map[string]draw{}}}
|
||||
func newDrawSet(s *session) drawSet {
|
||||
return drawSet{unnamed: draws{variant: map[string]node{}, value: map[string]draw{}, s: s}}
|
||||
}
|
||||
|
||||
// renderOnce renders n as one render, over draws of its own.
|
||||
func renderOnce(s *session, n node) string {
|
||||
var set drawSet
|
||||
set := drawSet{unnamed: draws{s: s}}
|
||||
return render(s, n, drawScope{set: &set})
|
||||
}
|
||||
|
||||
@@ -39,8 +145,9 @@ func (sc drawScope) in(t *template) drawScope {
|
||||
return sc
|
||||
}
|
||||
|
||||
// draws is the set's draws for sc's draw group.
|
||||
func (sc drawScope) draws() *draws {
|
||||
// draws is the set's draws for sc's draw group. The session is passed in rather than
|
||||
// read out of the set: copying it from there would leak the set's maps to the heap.
|
||||
func (sc drawScope) draws(s *session) *draws {
|
||||
if sc.group == "" {
|
||||
return &sc.set.unnamed
|
||||
}
|
||||
@@ -49,7 +156,7 @@ func (sc drawScope) draws() *draws {
|
||||
if sc.set.named == nil {
|
||||
sc.set.named = map[string]*draws{}
|
||||
}
|
||||
d = &draws{variant: map[string]node{}, value: map[string]draw{}}
|
||||
d = &draws{variant: map[string]node{}, value: map[string]draw{}, s: s}
|
||||
sc.set.named[strings.Clone(sc.group)] = d // a key from sc would leak sc, and with it every render's draw set, to the heap
|
||||
}
|
||||
return d
|
||||
@@ -186,7 +293,7 @@ func (c *drawCheck) reads(n node, stopAtGroup bool) bool {
|
||||
}
|
||||
r := false
|
||||
for _, e := range renderEdges(n) {
|
||||
if a, isRef := refRead(n, e.label); (isRef && len(a.tail) > 0) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) {
|
||||
if a, isRef := refRead(n, e.label); (isRef && (len(a.tail) > 0 || readsTable(n, a))) || (!repeats(e.to) && !(stopAtGroup && grouped(e.to)) && c.reads(e.to, stopAtGroup)) {
|
||||
r = true
|
||||
break
|
||||
}
|
||||
@@ -208,6 +315,17 @@ func grouped(n node) bool {
|
||||
return isTemplate && t.drawGroupKey != ""
|
||||
}
|
||||
|
||||
// readsTable reports whether a reference of n names a table, whose draw the render's
|
||||
// group answers for even when read whole.
|
||||
func readsTable(n node, a arm) bool {
|
||||
t, isTemplate := n.(*template)
|
||||
if !isTemplate {
|
||||
return false
|
||||
}
|
||||
_, isTable := t.fields[a.key].(*table)
|
||||
return isTable
|
||||
}
|
||||
|
||||
// refRead is the reference an edge of n reads; false when the edge reads none.
|
||||
func refRead(n node, label string) (arm, bool) {
|
||||
t, isTemplate := n.(*template)
|
||||
@@ -247,6 +365,79 @@ type drawRoute struct{ spelling, label string }
|
||||
type pathRead struct {
|
||||
at drawAt
|
||||
a arm
|
||||
tr *tableRead // set where the reference names a table
|
||||
}
|
||||
|
||||
// tableRead is what a reference reads of a table family: the table named, the rows
|
||||
// its selectors pin along the way, and whether it lands on a row rendered whole.
|
||||
type tableRead struct {
|
||||
head *table
|
||||
sels []tableSel
|
||||
whole bool
|
||||
}
|
||||
|
||||
// tableSel is one selector on the way: the table it selects a row of, and the
|
||||
// selector's spelling up to its closing bracket.
|
||||
type tableSel struct {
|
||||
t *table
|
||||
row int
|
||||
spelling string
|
||||
}
|
||||
|
||||
// tableReadOf reads what a reference path does of a table, replaying its selectors
|
||||
// over a walk of its own — checkPath proved each names a row — so two paths naming
|
||||
// one row by key and by name compare equal.
|
||||
func tableReadOf(head node, a arm, leaf node) *tableRead {
|
||||
t, isTable := head.(*table)
|
||||
if !isTable {
|
||||
return nil
|
||||
}
|
||||
tr := &tableRead{head: t}
|
||||
var pins draws
|
||||
cur, seen := t, a.name
|
||||
for _, seg := range a.tail {
|
||||
switch {
|
||||
case isSelector(seg):
|
||||
end := strings.Index(seen, "]") + 1
|
||||
_ = pins.selectRow(cur, selectorOf(seg))
|
||||
row, _ := pins.pinned(cur)
|
||||
tr.sels = append(tr.sels, tableSel{cur, row, a.name[:len(a.name)-len(seen)+end]})
|
||||
seen = seen[end:]
|
||||
case cur.children[seg] != nil:
|
||||
cur = cur.children[seg]
|
||||
}
|
||||
}
|
||||
c, isColumn := leaf.(*column)
|
||||
tr.whole = isColumn && c.i < 0
|
||||
return tr
|
||||
}
|
||||
|
||||
// family is the table a chain of parents ends at.
|
||||
func (t *table) family() *table {
|
||||
for t.parentT != nil {
|
||||
t = t.parentT
|
||||
}
|
||||
return t
|
||||
}
|
||||
|
||||
// selected is the selector in r that pins t, or the nearest ancestor of t it pins.
|
||||
func (r *tableRead) selected(t *table) (tableSel, bool) {
|
||||
for ; t != nil; t = t.parentT {
|
||||
for _, s := range r.sels {
|
||||
if s.t == t {
|
||||
return s, true
|
||||
}
|
||||
}
|
||||
}
|
||||
return tableSel{}, false
|
||||
}
|
||||
|
||||
func (r *tableRead) selection() string {
|
||||
parts := make([]string, len(r.sels))
|
||||
for i, s := range r.sels {
|
||||
parts[i] = fmt.Sprintf("%s[%d]", s.t.category, s.row)
|
||||
}
|
||||
return strings.Join(parts, " ")
|
||||
}
|
||||
|
||||
type drawVisit struct {
|
||||
@@ -286,15 +477,15 @@ func (w *drawWalk) edge(from node, e renderEdge, at drawAt) {
|
||||
if a, reads := refRead(from, e.label); reads {
|
||||
if k := (drawKey{at.group, a.path}); !w.read[k] {
|
||||
w.read[k] = true
|
||||
w.reads = append(w.reads, pathRead{at, a})
|
||||
w.reads = append(w.reads, pathRead{at, a, tableReadOf(from.(*template).fields[a.key], a, e.to)})
|
||||
}
|
||||
}
|
||||
w.walk(e.to, at)
|
||||
}
|
||||
|
||||
// check refuses what one draw per reference path cannot answer for: a read of a level beside a path
|
||||
// another read takes into it. Reads are compared in path order, so which pair is reported does not
|
||||
// vary.
|
||||
// another read takes into it, and two reads of one table family that select different rows. Reads
|
||||
// are compared in path order, so which pair is reported does not vary.
|
||||
func (w *drawWalk) check() error {
|
||||
sort.SliceStable(w.reads, func(i, j int) bool {
|
||||
if w.reads[i].at.group != w.reads[j].at.group {
|
||||
@@ -304,14 +495,53 @@ func (w *drawWalk) check() error {
|
||||
})
|
||||
for i, level := range w.reads {
|
||||
for _, into := range w.reads[i+1:] {
|
||||
if into.at.group == level.at.group && strings.HasPrefix(into.a.path, level.a.path+".") {
|
||||
if into.at.group != level.at.group {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(into.a.path, level.a.path+".") && !(level.tr != nil && level.tr.whole) {
|
||||
return overlapError(level.at.route, level.a.name, into)
|
||||
}
|
||||
if err := checkFamily(level, into); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkFamily refuses two reads of one table family in one group that cannot read one consistent
|
||||
// draw: a table rendered whole beside a path into the family, and two paths selecting different rows.
|
||||
func checkFamily(a, b pathRead) error {
|
||||
if a.tr == nil || b.tr == nil || a.tr.head.family() != b.tr.head.family() {
|
||||
return nil
|
||||
}
|
||||
for _, pair := range [][2]pathRead{{a, b}, {b, a}} {
|
||||
bare, path := pair[0], pair[1]
|
||||
if len(bare.a.tail) == 0 && len(path.a.tail) > 0 {
|
||||
return overlapError(bare.at.route, bare.a.name, path)
|
||||
}
|
||||
}
|
||||
if len(a.a.tail) == 0 || len(b.a.tail) == 0 || a.tr.selection() == b.tr.selection() {
|
||||
return nil
|
||||
}
|
||||
for _, pair := range [][2]pathRead{{a, b}, {b, a}} {
|
||||
selected, plain := pair[0], pair[1]
|
||||
if len(plain.tr.sels) > 0 {
|
||||
continue
|
||||
}
|
||||
if s, ok := selected.tr.selected(plain.tr.head); ok {
|
||||
tail := plain.a.tail
|
||||
if s.t != plain.tr.head {
|
||||
tail = append([]string{plain.tr.head.category}, tail...)
|
||||
}
|
||||
return fmt.Errorf("%s reads %s without the row %s selects; write {%s.%s}, or draw them apart with a drawGroup",
|
||||
plain.at.route.spelled(plain.a.name), plain.tr.head.category, selected.at.route.spelled(selected.a.name), s.spelling, strings.Join(tail, "."))
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("%s and %s select different rows of one table family; select the same rows in both, or draw them apart with a drawGroup",
|
||||
a.at.route.spelled(a.a.name), b.at.route.spelled(b.a.name))
|
||||
}
|
||||
|
||||
func overlapError(route drawRoute, ref string, into pathRead) error {
|
||||
return fmt.Errorf("%s renders a level that %s reads a path into; name the fields you want instead, or draw them apart with a drawGroup", route.spelled(ref), into.at.route.spelled(into.a.name))
|
||||
}
|
||||
|
||||
+25
-1
@@ -46,6 +46,8 @@ type Generator struct {
|
||||
mu sync.Mutex
|
||||
rand *session
|
||||
categories map[string]node
|
||||
root *folder // the categories as the node a path walks from
|
||||
set drawSet // one Fake's draws, owned here so a walk pinning rows keeps them off the heap
|
||||
records map[node]recordShape
|
||||
structs map[reflect.Type]structResult
|
||||
}
|
||||
@@ -123,7 +125,7 @@ func New(opts ...Option) (*Generator, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("fejkdata: %w", err)
|
||||
}
|
||||
return &Generator{rand: rng, categories: cats}, nil
|
||||
return &Generator{rand: rng, categories: cats, root: &folder{children: cats}}, nil
|
||||
}
|
||||
|
||||
// List returns the sorted dotted paths Fake can render: every category, the dotted
|
||||
@@ -166,6 +168,10 @@ func paths(n node) []string {
|
||||
return out
|
||||
case *null:
|
||||
return []string{""}
|
||||
case *table:
|
||||
return tablePaths(n)
|
||||
case *column:
|
||||
return []string{""}
|
||||
case *choice:
|
||||
out := []string{""}
|
||||
for p := range n.shared {
|
||||
@@ -176,6 +182,24 @@ func paths(n node) []string {
|
||||
return nil
|
||||
}
|
||||
|
||||
// tablePaths is a table's columns, then each table linked to it under its name: the
|
||||
// direct descents, a step at a time.
|
||||
func tablePaths(t *table) []string {
|
||||
out := append([]string{""}, t.columns...)
|
||||
sort.Strings(out[1:])
|
||||
children := make([]string, 0, len(t.children))
|
||||
for name := range t.children {
|
||||
children = append(children, name)
|
||||
}
|
||||
sort.Strings(children)
|
||||
for _, name := range children {
|
||||
for _, p := range paths(t.children[name]) {
|
||||
out = append(out, join(name, p))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// sharedPaths is the sub-paths every item carries — the only ones a path may step
|
||||
// through a choice to reach. It intersects, bailing as soon as the set
|
||||
// is empty, which is immediate for a choice of plain strings.
|
||||
|
||||
@@ -3,7 +3,6 @@ package fejkdata
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// walkNodes calls fn once per contained node, passing the dot path that reaches it,
|
||||
@@ -61,6 +60,19 @@ func contained(n node) []namedNode {
|
||||
return out
|
||||
case *template:
|
||||
return named(n.fields)
|
||||
case *table:
|
||||
return append([]namedNode{{node: n.format}}, named(n.fields)...)
|
||||
case *column:
|
||||
if n.i < 0 {
|
||||
return nil
|
||||
}
|
||||
var out []namedNode
|
||||
for r := 0; r < n.t.rows(); r++ {
|
||||
if cell := n.t.cellNode(r, n.i); cell != nil {
|
||||
out = append(out, namedNode{node: cell})
|
||||
}
|
||||
}
|
||||
return out
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
@@ -142,12 +154,23 @@ func renderEdges(n node) []renderEdge {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for _, name := range strings.Split(t.body, "|") {
|
||||
for _, name := range splitOutside(t.body, '|') {
|
||||
add(name, "")
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return es
|
||||
case *table:
|
||||
return []renderEdge{{to: n.format, label: "format"}}
|
||||
case *column:
|
||||
if n.i < 0 {
|
||||
return []renderEdge{{to: n.t.format, label: "format"}}
|
||||
}
|
||||
var es []renderEdge
|
||||
for _, c := range contained(n) {
|
||||
es = append(es, renderEdge{to: c.node, label: n.t.columns[n.i]})
|
||||
}
|
||||
return es
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
@@ -158,7 +181,7 @@ func renderEdges(n node) []renderEdge {
|
||||
// already proved the tail resolves in every variant.
|
||||
func pathLeaves(n node, tail []string) []node {
|
||||
var out []node
|
||||
_ = walkPath(n, tail, pathWalk{
|
||||
_, _ = walkPath(n, tail, pathWalk{
|
||||
choice: func(c *choice, _ []string) ([]node, error) { return c.items, nil },
|
||||
leaf: func(n node) error { out = append(out, n); return nil },
|
||||
})
|
||||
|
||||
@@ -109,7 +109,7 @@ func operandReader(t *template, head string) string {
|
||||
// coverPath collects what holding one path pins: every choice level the path
|
||||
// passes through, whole, and the leaf it renders.
|
||||
func coverPath(n node, tail []string, into map[node]bool) {
|
||||
_ = walkPath(n, tail, pathWalk{
|
||||
_, _ = walkPath(n, tail, pathWalk{
|
||||
choice: func(c *choice, _ []string) ([]node, error) { cover(c, into, false); return nil, nil },
|
||||
leaf: func(n node) error { cover(n, into, false); return nil },
|
||||
})
|
||||
@@ -262,6 +262,16 @@ func checkNoRepeatedRead(format string, c formatOps, refs map[string]refBinding)
|
||||
type draws struct {
|
||||
variant map[string]node
|
||||
value map[string]draw
|
||||
pins [4]tablePin // the rows pinned, inline so a render pinning a few tables stays off the heap
|
||||
npins int
|
||||
more map[*table]int // the rows pinned past the inline four
|
||||
s *session // what draws a row; nil where a walk only proves selectors
|
||||
}
|
||||
|
||||
// tablePin is one table's pinned row.
|
||||
type tablePin struct {
|
||||
t *table
|
||||
row int
|
||||
}
|
||||
|
||||
// draw is what one read drew: its text, and whether it landed on a null.
|
||||
@@ -284,12 +294,11 @@ func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw {
|
||||
}
|
||||
return draw{text: render(s, t.fields[a.key], sc)}
|
||||
}
|
||||
d := readScope(held, sc, a)
|
||||
d := readScope(s, held, sc, a)
|
||||
if r, done := d.value[a.path]; done {
|
||||
return r
|
||||
}
|
||||
var r draw
|
||||
_ = walkPath(t.fields[a.key], a.tail, pathWalk{
|
||||
leaf, _ := walkPath(t.fields[a.key], a.tail, pathWalk{
|
||||
// Hold the draw at every level passed through, so two paths sharing a
|
||||
// prefix share it.
|
||||
choice: func(c *choice, rest []string) ([]node, error) {
|
||||
@@ -307,8 +316,9 @@ func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw {
|
||||
}
|
||||
return []node{n}, nil
|
||||
},
|
||||
leaf: func(n node) error { r = renderLeaf(s, n, sc); return nil },
|
||||
pins: d,
|
||||
})
|
||||
r := renderLeaf(s, leaf, sc)
|
||||
if d.value == nil {
|
||||
d.value = map[string]draw{}
|
||||
}
|
||||
@@ -319,9 +329,9 @@ func readField(s *session, t *template, held *draws, sc drawScope, a arm) draw {
|
||||
// readScope is the draws a held read keeps its draw in: for a reference that reads a path,
|
||||
// the render's draws for its group, so its draw spans the render; for a sibling, or a
|
||||
// reference read whole, held.
|
||||
func readScope(held *draws, sc drawScope, a arm) *draws {
|
||||
func readScope(s *session, held *draws, sc drawScope, a arm) *draws {
|
||||
if isRef(a.key) && len(a.tail) > 0 {
|
||||
return sc.draws()
|
||||
return sc.draws(s)
|
||||
}
|
||||
return held
|
||||
}
|
||||
@@ -335,7 +345,7 @@ func renderLeaf(s *session, n node, sc drawScope) draw {
|
||||
}
|
||||
r := draw{text: render(s, n, sc)}
|
||||
if t, _ := n.(*template); t != nil && t.readsColumn != nil {
|
||||
r.null = sc.in(t).draws().value[t.readsColumn.a.path].null
|
||||
r.null = sc.in(t).draws(s).value[t.readsColumn.a.path].null
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
@@ -69,9 +69,15 @@ func isTemplate(arg string) (bool, error) {
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
if i := strings.IndexAny(arg, `[]}"`); i >= 0 {
|
||||
if i := strings.IndexAny(arg, `}"`); i >= 0 {
|
||||
return false, fmt.Errorf("%q holds a %q, which no path may, and it is not valid JSON, so it names no template either", arg, arg[i:i+1])
|
||||
}
|
||||
if strings.HasPrefix(strings.TrimSpace(arg), "[") {
|
||||
return false, fmt.Errorf(`%q starts with "[" and is not valid JSON, so it names no template; a path starts with a name`, arg)
|
||||
}
|
||||
if _, err := splitPath(arg); err != nil {
|
||||
return false, err
|
||||
}
|
||||
if path := strings.TrimLeft(arg, "/"); path != arg && path != "" {
|
||||
return false, pathAdvice(path, fmt.Sprintf("path %s starts with /, and every path starts at the root already", arg))
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
@@ -59,9 +60,11 @@ type template struct {
|
||||
// held is every name drawn once per expansion: the bound levels above, plus the
|
||||
// siblings a {calc()} reads. nil when the format holds nothing (see expand).
|
||||
held map[string]bool
|
||||
heldLocal bool // some held name is kept by the expansion itself, so expand makes its draws
|
||||
fromString bool // written as a JSON string rather than an object
|
||||
readsColumn *columnRead // set when the format is one reference alone reading a record's column
|
||||
record bool // compiled at the top without a repeat, so its fields are record columns
|
||||
table *table // the table whose format this is, whose columns are the fields
|
||||
drawGroup string // the draw group it draws in, as written; "" keeps its caller's
|
||||
drawGroupKey string // its draw group keyed by its category once linked: what a render reads its reference paths under
|
||||
}
|
||||
@@ -83,6 +86,70 @@ func compile(v any) (node, error) {
|
||||
return compileAt(v, atTop)
|
||||
}
|
||||
|
||||
// compileCategory compiles a data file's value, which may be a table over a rows
|
||||
// file beside it, and refuses a choice that is a table written as templates.
|
||||
func compileCategory(v any, name string, files *categoryFiles) (node, error) {
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
if _, isTable := m["rows"]; isTable {
|
||||
return compileTable(m, name, files)
|
||||
}
|
||||
}
|
||||
if items, ok := v.([]any); ok {
|
||||
if err := checkNotRows(items, name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return compile(v)
|
||||
}
|
||||
|
||||
// checkNotRows refuses a choice of templates sharing one format and one set of
|
||||
// string fields: each item is a row, and the rows file is the spelling for that.
|
||||
func checkNotRows(items []any, name string) error {
|
||||
var format string
|
||||
var fields []string
|
||||
weighted := false
|
||||
for i, raw := range items {
|
||||
f, keys, w, isRow := rowShape(raw)
|
||||
if !isRow || i > 0 && (f != format || !slices.Equal(keys, fields)) {
|
||||
return nil
|
||||
}
|
||||
format, fields, weighted = f, keys, weighted || w
|
||||
}
|
||||
if len(items) < 2 || len(fields) == 0 {
|
||||
return nil
|
||||
}
|
||||
weight := ""
|
||||
if weighted {
|
||||
weight = `, a weight column, and "weight" naming it`
|
||||
}
|
||||
return fmt.Errorf("a choice of %d templates with one format and the fields %v is a table; write the rows in %s.tsv with the header %q%s, and the category as {\"format\": %q, \"rows\": \"%s.tsv\"}",
|
||||
len(items), fields, name, strings.Join(fields, "\t"), weight, format, name)
|
||||
}
|
||||
|
||||
// rowShape reads a choice item as a row: a template whose every field is a string,
|
||||
// with its format, its sorted field names, and whether it carries a weight.
|
||||
func rowShape(raw any) (format string, fields []string, weighted, isRow bool) {
|
||||
m, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return "", nil, false, false
|
||||
}
|
||||
format, _ = m["format"].(string)
|
||||
for k, v := range m {
|
||||
if k == "weight" {
|
||||
weighted = true
|
||||
continue
|
||||
}
|
||||
if _, isString := v.(string); !isString || isOption(k) && k != "format" {
|
||||
return "", nil, false, false
|
||||
}
|
||||
if k != "format" {
|
||||
fields = append(fields, k)
|
||||
}
|
||||
}
|
||||
sort.Strings(fields)
|
||||
return format, fields, weighted, true
|
||||
}
|
||||
|
||||
// compileAt compiles a node that is no choice's item. Only a choice's items carry a
|
||||
// weight, so one here would be inert whatever its type.
|
||||
func compileAt(v any, pos position) (node, error) {
|
||||
@@ -140,7 +207,7 @@ func compileString(s string) (node, error) {
|
||||
// applies the fences that need the compiled reads.
|
||||
func (t *template) compileFormat() error {
|
||||
c := compileOps(t.format, t.refs)
|
||||
t.ops, t.grow, t.bound, t.held = c.ops, c.grow, c.bound, c.held
|
||||
t.ops, t.grow, t.bound, t.held, t.heldLocal = c.ops, c.grow, c.bound, c.held, c.heldLocal
|
||||
t.fixed = true
|
||||
for _, o := range t.ops {
|
||||
if o.kind != 'l' {
|
||||
@@ -231,6 +298,9 @@ func checkNoRepeatedItem(items []any) error {
|
||||
}
|
||||
|
||||
func compileTemplate(m map[string]any, pos position) (node, error) {
|
||||
if _, isTable := m["rows"]; isTable {
|
||||
return nil, fmt.Errorf("rows names a TSV beside a category's file, so only a category is a table; an inline template has no file beside it")
|
||||
}
|
||||
o, err := readOptions(m, pos)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -407,7 +477,11 @@ func checkName(name string) error {
|
||||
|
||||
// checkPathNames rejects a dotted path with a segment no name may be.
|
||||
func checkPathNames(path string) error {
|
||||
for _, seg := range strings.Split(path, ".") {
|
||||
segs, err := splitPath(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, seg := range names(segs) {
|
||||
if err := checkName(seg); err != nil {
|
||||
return fmt.Errorf("path %w", err)
|
||||
}
|
||||
|
||||
@@ -6,61 +6,244 @@ import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// splitPath splits a dotted path into segments, a selector — [key or name] after a
|
||||
// table's name — becoming a segment of its own, brackets kept. A dot inside a
|
||||
// selector is part of the key or name.
|
||||
func splitPath(path string) ([]string, error) {
|
||||
segs := make([]string, 0, strings.Count(path, ".")+2*strings.Count(path, "[")+1)
|
||||
start, open := 0, -1
|
||||
for i := 0; i < len(path); i++ {
|
||||
switch path[i] {
|
||||
case '[':
|
||||
if err := checkOpen(path, i, start, open); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
segs = append(segs, path[start:i])
|
||||
start, open = i, i
|
||||
case ']':
|
||||
if err := checkClose(path, i, open); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
segs = append(segs, path[start:i+1])
|
||||
start, open = i+2, -1
|
||||
i++
|
||||
case '.':
|
||||
if open < 0 {
|
||||
segs = append(segs, path[start:i])
|
||||
start = i + 1
|
||||
}
|
||||
}
|
||||
}
|
||||
if open >= 0 {
|
||||
return nil, fmt.Errorf(`%q opens a selector with "[" and never closes it with "]"`, path)
|
||||
}
|
||||
if start <= len(path) {
|
||||
segs = append(segs, path[start:])
|
||||
}
|
||||
return segs, nil
|
||||
}
|
||||
|
||||
// checkOpen refuses a "[" at i that opens no selector: one inside a selector, or one
|
||||
// following no name.
|
||||
func checkOpen(path string, i, start, open int) error {
|
||||
switch {
|
||||
case open >= 0:
|
||||
return fmt.Errorf(`%q holds a "[" inside a selector, which no key or name may`, path)
|
||||
case i == 0:
|
||||
return fmt.Errorf(`%q starts with "["; a path starts with a name, and a selector follows a table's name`, path)
|
||||
case i == start:
|
||||
return fmt.Errorf(`%q: a selector follows its table's name; write %s`, path, path[:i-1]+path[i:])
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkClose refuses a "]" at i that closes no selector, closes an empty one, or is
|
||||
// followed by anything but a dot or the end.
|
||||
func checkClose(path string, i, open int) error {
|
||||
switch {
|
||||
case open < 0:
|
||||
return fmt.Errorf(`%q holds a "]" that closes no "["`, path)
|
||||
case i == open+1:
|
||||
return fmt.Errorf("%q holds an empty selector; a selector names a row by key or name", path)
|
||||
case i+1 < len(path) && path[i+1] != '.':
|
||||
return fmt.Errorf(`%q: a "]" ends its selector, so a dot or the end must follow it`, path)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// indexOutside is the first c in s outside a [selector], or -1.
|
||||
func indexOutside(s string, c byte) int {
|
||||
depth := 0
|
||||
for i := 0; i < len(s); i++ {
|
||||
switch {
|
||||
case s[i] == '[':
|
||||
depth++
|
||||
case s[i] == ']' && depth > 0:
|
||||
depth--
|
||||
case s[i] == c && depth == 0:
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// splitOutside splits s on c outside any [selector].
|
||||
func splitOutside(s string, c byte) []string {
|
||||
var parts []string
|
||||
for {
|
||||
i := indexOutside(s, c)
|
||||
if i < 0 {
|
||||
return append(parts, s)
|
||||
}
|
||||
parts, s = append(parts, s[:i]), s[i+1:]
|
||||
}
|
||||
}
|
||||
|
||||
// isSelector reports whether a segment is a [key or name] rather than a name.
|
||||
func isSelector(seg string) bool { return strings.HasPrefix(seg, "[") }
|
||||
|
||||
// selectorOf is the key or name a selector segment holds.
|
||||
func selectorOf(seg string) string { return seg[1 : len(seg)-1] }
|
||||
|
||||
// names is the segments of a path that are names, its selectors left out.
|
||||
func names(segs []string) []string {
|
||||
out := segs[:0:0]
|
||||
for _, s := range segs {
|
||||
if !isSelector(s) {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// pathWalk is what one walk of a dotted path does at each kind of level: choice
|
||||
// returns the variants to continue into (none stops the walk); level runs at each
|
||||
// template a segment descends into; leaf runs where the tail ends. A nil action
|
||||
// is skipped.
|
||||
// returns the variants to continue into (none stops the walk), and where it is nil
|
||||
// the walk draws one with the pins' session, or stops; level runs at each template
|
||||
// a segment descends into; leaf runs where the tail ends; pins is where the walk
|
||||
// pins the table rows it selects and draws, nil skipping tables. A nil action is
|
||||
// skipped. A render's walk sets only pins, so it allocates nothing.
|
||||
type pathWalk struct {
|
||||
choice func(c *choice, rest []string) ([]node, error)
|
||||
level func(t *template, rest []string) error
|
||||
leaf func(n node) error
|
||||
pins *draws
|
||||
}
|
||||
|
||||
// walkPath descends tail from n: a folder or template by its next segment, a
|
||||
// choice by w.choice, which consumes no segment. A missing segment is an error,
|
||||
// so no walk reaches past what the data holds. A table-shaped dispatch, one case
|
||||
// per node kind, kept whole on purpose.
|
||||
func walkPath(n node, tail []string, w pathWalk) error {
|
||||
// walkPath descends tail from n and returns the node it ends at: a folder or
|
||||
// template by its next segment, a choice by w.choice, which consumes no segment, a
|
||||
// table by walkTable. A missing segment is an error, so no walk reaches past what
|
||||
// the data holds. A table-shaped dispatch, one case per node kind, kept whole on
|
||||
// purpose.
|
||||
func walkPath(n node, tail []string, w pathWalk) (node, error) {
|
||||
if len(tail) == 0 {
|
||||
if w.leaf != nil {
|
||||
return w.leaf(n)
|
||||
return n, w.atLeaf(n)
|
||||
}
|
||||
return nil
|
||||
if t, isTable := n.(*table); isTable {
|
||||
return walkTable(t, tail, w, false)
|
||||
}
|
||||
if isSelector(tail[0]) {
|
||||
return nil, fmt.Errorf("%s is not a table, so it has no row to select", selectorOf(tail[0]))
|
||||
}
|
||||
switch n := n.(type) {
|
||||
case *folder:
|
||||
child, ok := n.children[tail[0]]
|
||||
if !ok {
|
||||
return fmt.Errorf("no entry %q", tail[0])
|
||||
return nil, fmt.Errorf("no entry %q", tail[0])
|
||||
}
|
||||
return walkPath(child, tail[1:], w)
|
||||
case *template:
|
||||
if w.level != nil {
|
||||
if err := w.level(n, tail); err != nil {
|
||||
return err
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
child, ok := n.field(tail[0])
|
||||
if !ok {
|
||||
return fmt.Errorf("no field %q", tail[0])
|
||||
return nil, fmt.Errorf("no field %q", tail[0])
|
||||
}
|
||||
return walkPath(child, tail[1:], w)
|
||||
case *choice:
|
||||
return walkChoice(n, tail, w)
|
||||
case *column:
|
||||
return nil, fmt.Errorf("no field %q: %q is a column, and a cell holds no fields", tail[0], n.t.columns[n.i])
|
||||
}
|
||||
return nil, fmt.Errorf("no field %q", tail[0])
|
||||
}
|
||||
|
||||
func (w pathWalk) atLeaf(n node) error {
|
||||
if w.leaf != nil {
|
||||
return w.leaf(n)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// walkChoice continues a walk into the variants w.choice returns, or into one drawn
|
||||
// by the pins' session where the walk names no choice action.
|
||||
func walkChoice(c *choice, tail []string, w pathWalk) (node, error) {
|
||||
if w.choice == nil {
|
||||
return nil
|
||||
if w.pins == nil || w.pins.s == nil {
|
||||
return nil, nil
|
||||
}
|
||||
next, err := w.choice(n, tail)
|
||||
if err := carriedByAll(c, tail); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return walkPath(pick(w.pins.s, c), tail, w)
|
||||
}
|
||||
next, err := w.choice(c, tail)
|
||||
if err != nil {
|
||||
return err
|
||||
return nil, err
|
||||
}
|
||||
var last node
|
||||
for _, item := range next {
|
||||
if err := walkPath(item, tail, w); err != nil {
|
||||
return err
|
||||
if last, err = walkPath(item, tail, w); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return last, nil
|
||||
}
|
||||
|
||||
// walkTable descends tail from a table: a selector first, then a column or a
|
||||
// linked table by name. The walk reads a row wherever a segment follows or the
|
||||
// table was reached from another; a table reached whole, with no selector, is left
|
||||
// to a render's own draw.
|
||||
func walkTable(t *table, tail []string, w pathWalk, descended bool) (node, error) {
|
||||
sel := ""
|
||||
if len(tail) > 0 && isSelector(tail[0]) {
|
||||
sel, tail = selectorOf(tail[0]), tail[1:]
|
||||
if len(tail) > 0 && isSelector(tail[0]) {
|
||||
return nil, fmt.Errorf("%s[%s] is selected twice; one selector names its row", t.category, sel)
|
||||
}
|
||||
}
|
||||
if w.pins != nil {
|
||||
if err := readRow(w.pins, t, sel, descended || len(tail) > 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if len(tail) == 0 {
|
||||
if sel == "" && !descended {
|
||||
return walkPath(t, nil, w)
|
||||
}
|
||||
return walkPath(t.whole, nil, w)
|
||||
}
|
||||
if i, ok := t.col[tail[0]]; ok {
|
||||
return walkPath(t.fields[t.columns[i]], tail[1:], w)
|
||||
}
|
||||
if child := t.descendant(tail[0]); child != nil {
|
||||
return walkTable(child, tail[1:], w, true)
|
||||
}
|
||||
return nil, fmt.Errorf("no column or linked table %q in %s", tail[0], t.category)
|
||||
}
|
||||
|
||||
// readRow pins the row a path reads of t: the one its selector names, or, where the
|
||||
// walk draws, one drawn where the path reads into the table.
|
||||
func readRow(d *draws, t *table, sel string, draw bool) error {
|
||||
if sel != "" {
|
||||
return d.selectRow(t, sel)
|
||||
}
|
||||
if draw && d.s != nil {
|
||||
d.rowOf(t)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("no field %q", tail[0])
|
||||
}
|
||||
|
||||
// carriedByAll is the choice rule a path that must resolve on every call obeys:
|
||||
@@ -88,12 +271,13 @@ func unreachableInChoice(c *choice, want string) error {
|
||||
}
|
||||
|
||||
// checkPath proves a dotted tail resolves whichever way the draws go — a choice
|
||||
// must carry the rest of the path in the set every variant shares — and that no
|
||||
// level a path reads carries a repeat or a drawGroup, which one draw of it could not
|
||||
// apply. So a path that validates here resolves on every render, and a typo is a
|
||||
// New-time error.
|
||||
// must carry the rest of the path in the set every variant shares, a selector must
|
||||
// name one row inside the rows selected before it — and that no level a path reads
|
||||
// carries a repeat or a drawGroup, which one draw of it could not apply. So a path
|
||||
// that validates here resolves on every render, and a typo is a New-time error.
|
||||
func checkPath(n node, tail []string, level string) error {
|
||||
return walkPath(n, tail, pathWalk{
|
||||
var pins draws
|
||||
_, err := walkPath(n, tail, pathWalk{
|
||||
choice: func(c *choice, rest []string) ([]node, error) {
|
||||
if err := carriedByAll(c, rest); err != nil {
|
||||
return nil, err
|
||||
@@ -110,5 +294,7 @@ func checkPath(n node, tail []string, level string) error {
|
||||
}
|
||||
return nil
|
||||
},
|
||||
pins: &pins,
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
+4
-4
@@ -13,14 +13,14 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) {
|
||||
level: func(tm *template, rest []string) error { seen = append(seen, "level:"+rest[0]); return nil },
|
||||
leaf: func(n node) error { seen = append(seen, "leaf"); return nil },
|
||||
}
|
||||
if err := walkPath(n, []string{"a", "b"}, walk); err != nil {
|
||||
if _, err := walkPath(n, []string{"a", "b"}, walk); err != nil {
|
||||
t.Fatalf("walkPath(a.b) = %v", err)
|
||||
}
|
||||
if want := []string{"level:a", "level:b", "leaf"}; !slices.Equal(seen, want) {
|
||||
t.Errorf("walk visited %v, want %v", seen, want)
|
||||
}
|
||||
seen = nil
|
||||
err := walkPath(n, []string{"a", "nope", "deeper"}, walk)
|
||||
_, err := walkPath(n, []string{"a", "nope", "deeper"}, walk)
|
||||
if err == nil || !strings.Contains(err.Error(), `no field "nope"`) {
|
||||
t.Errorf("walkPath(a.nope.deeper) = %v, want the missing segment named", err)
|
||||
}
|
||||
@@ -32,7 +32,7 @@ func TestWalkPathStopsAtAMissingSegment(t *testing.T) {
|
||||
func TestWalkPathChoiceConsumesNoSegment(t *testing.T) {
|
||||
n := compiled(t, `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`)
|
||||
var leaves []node
|
||||
err := walkPath(n, []string{"f"}, pathWalk{
|
||||
_, err := walkPath(n, []string{"f"}, pathWalk{
|
||||
choice: func(c *choice, rest []string) ([]node, error) {
|
||||
if len(rest) != 1 || rest[0] != "f" {
|
||||
t.Errorf("choice saw rest %v, want [f]", rest)
|
||||
@@ -74,7 +74,7 @@ func TestDescendIntoStringErrors(t *testing.T) {
|
||||
// one call and failing on the next: every variant must carry the rest of the path.
|
||||
func TestPathThroughChoice(t *testing.T) {
|
||||
dir := writeData(t, map[string]string{
|
||||
"every": `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2"}]`,
|
||||
"every": `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2","g":"x"}]`,
|
||||
"notall": `[{"format":"{f}","f":"1"},"plain"]`,
|
||||
"some": `[{"format":"{f}","f":"1"},{"format":"{f}","f":"2","extra":"x"}]`,
|
||||
})
|
||||
|
||||
@@ -122,18 +122,48 @@ func literal(c Column, quote func(string) string, nullText string) string {
|
||||
func (f *Generator) FakeRecord(path string) (*Record, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
_, n, tail, err := resolveCategory(f.categories, strings.Split(path, "."))
|
||||
segments, err := splitPath(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("fejkdata: %w", err)
|
||||
}
|
||||
_, n, tail, err := resolveCategory(f.categories, segments)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("fejkdata: %s: %w", path, err)
|
||||
}
|
||||
if len(tail) > 0 {
|
||||
set := newDrawSet(f.rand)
|
||||
sc := drawScope{set: &set}
|
||||
if t, isTable := n.(*table); isTable {
|
||||
if n, err = tableRecord(f.rand, t, tail, sc); err != nil {
|
||||
return nil, fmt.Errorf("fejkdata: %s: %w", path, err)
|
||||
}
|
||||
} else if len(tail) > 0 {
|
||||
return nil, fmt.Errorf("fejkdata: %s descends into %q, a field; only a category-level template is a record", path, tail[0])
|
||||
}
|
||||
shape := f.recordShapeOf(n)
|
||||
if shape.err != nil {
|
||||
return nil, fmt.Errorf("fejkdata: %s %w", path, shape.err)
|
||||
}
|
||||
return renderRecord(f.rand, shape.t, shape.columns), nil
|
||||
return renderRecord(f.rand, shape.t, shape.columns, sc), nil
|
||||
}
|
||||
|
||||
// tableRecord walks a path's tail from a table to the table whose row is the record,
|
||||
// pinning the rows it selects or draws.
|
||||
func tableRecord(s *session, t *table, tail []string, sc drawScope) (node, error) {
|
||||
n, err := descend(s, t, tail, sc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch n := n.(type) {
|
||||
case *table:
|
||||
sc.draws(s).rowOf(n)
|
||||
return n, nil
|
||||
case *column:
|
||||
if n.i < 0 {
|
||||
return n.t, nil
|
||||
}
|
||||
return nil, fmt.Errorf("descends into %q, a column; a record is a table's row", n.t.columns[n.i])
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// recordShape is what recordOf settled about a node: the template to project, its
|
||||
@@ -171,7 +201,8 @@ type RecordTemplate struct {
|
||||
func (t *RecordTemplate) Fake() *Record {
|
||||
t.g.mu.Lock()
|
||||
defer t.g.mu.Unlock()
|
||||
return renderRecord(t.g.rand, t.t, t.columns)
|
||||
set := newDrawSet(t.g.rand)
|
||||
return renderRecord(t.g.rand, t.t, t.columns, drawScope{set: &set})
|
||||
}
|
||||
|
||||
// NewRecordTemplate compiles an inline record — a JSON object with a format and
|
||||
@@ -200,6 +231,9 @@ func (f *Generator) FakeRecordTemplate(input string) (*Record, error) {
|
||||
// recordOf is the fence both record entry points pass. The columns come back with
|
||||
// the template, fixed for every draw the caller goes on to make.
|
||||
func recordOf(n node) (*template, []Column, error) {
|
||||
if tb, isTable := n.(*table); isTable {
|
||||
n = tb.format
|
||||
}
|
||||
t, ok := n.(*template)
|
||||
if !ok {
|
||||
return nil, nil, errors.New("names a choice, not a template; a record is a template whose fields are its columns")
|
||||
@@ -219,10 +253,13 @@ func recordOf(n node) (*template, []Column, error) {
|
||||
return t, columns, nil
|
||||
}
|
||||
|
||||
// renderRecord draws each column once, in the name order recordOf fixed, as one render.
|
||||
func renderRecord(s *session, t *template, columns []Column) *Record {
|
||||
set := newDrawSet()
|
||||
sc := drawScope{set: &set}.in(t)
|
||||
// renderRecord draws each column once, in the name order recordOf fixed, as one render
|
||||
// over sc's draws; a table's columns read the row pinned there.
|
||||
func renderRecord(s *session, t *template, columns []Column, sc drawScope) *Record {
|
||||
sc = sc.in(t)
|
||||
if t.table != nil {
|
||||
sc.t, sc.row = t.table, sc.draws(s).mustRow(t.table)
|
||||
}
|
||||
r := &Record{columns: append([]Column(nil), columns...)}
|
||||
for i := range r.columns {
|
||||
column := renderLeaf(s, t.fields[r.columns[i].Name], sc)
|
||||
|
||||
+10
-2
@@ -35,7 +35,11 @@ func refShape(name string) (sigil, rest string, err error) {
|
||||
if strings.HasPrefix(rest, ".") {
|
||||
return "", "", fmt.Errorf("a reference starts with / (the root), . (this folder) or .. (the folder above)")
|
||||
}
|
||||
for _, seg := range strings.Split(rest, ".") {
|
||||
segs, err := splitPath(rest)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
for _, seg := range segs {
|
||||
if seg == "" {
|
||||
return "", "", fmt.Errorf("path has an empty segment")
|
||||
}
|
||||
@@ -59,7 +63,8 @@ func refSegments(name string, folder []string) ([]string, error) {
|
||||
}
|
||||
base = folder[:len(folder)-1]
|
||||
}
|
||||
return append(append([]string{}, base...), strings.Split(rest, ".")...), nil
|
||||
segs, _ := splitPath(rest) // refShape proved it splits
|
||||
return append(append([]string{}, base...), segs...), nil
|
||||
}
|
||||
|
||||
// linkRefs resolves every reference in the assembled tree. The head of the path —
|
||||
@@ -199,6 +204,9 @@ func resolveCategory(root map[string]node, segments []string) (head []string, ta
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
if isSelector(segments[i]) {
|
||||
return nil, nil, nil, fmt.Errorf("%s is a folder, not a table, so it has no row to select", strings.Join(segments[:i], "."))
|
||||
}
|
||||
child, ok := g.children[segments[i]]
|
||||
if !ok {
|
||||
return nil, nil, nil, fmt.Errorf("no entry %q", segments[i])
|
||||
|
||||
@@ -2,7 +2,6 @@ package fejkdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
@@ -20,33 +19,33 @@ type rng interface {
|
||||
func (f *Generator) Fake(path string) (string, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
n, err := descend(f.rand, &folder{children: f.categories}, strings.Split(path, "."))
|
||||
segments, err := splitPath(path)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("fejkdata: %w", err)
|
||||
}
|
||||
f.set = drawSet{unnamed: draws{s: f.rand}}
|
||||
sc := drawScope{set: &f.set}
|
||||
if f.root == nil {
|
||||
f.root = &folder{children: f.categories}
|
||||
}
|
||||
n, err := descend(f.rand, f.root, segments, sc)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("fejkdata: %s: %w", path, err)
|
||||
}
|
||||
if _, ok := n.(*folder); ok {
|
||||
return "", fmt.Errorf("fejkdata: %s names a folder, not a value", path)
|
||||
}
|
||||
return renderOnce(f.rand, n), nil
|
||||
return render(f.rand, n, sc), nil
|
||||
}
|
||||
|
||||
// descend walks named fields to the node a path names. It is the one render-side
|
||||
// step that can fail, because the path comes from the caller and may name a field
|
||||
// that does not exist. A choice consumes no segment, so the rest of the path must
|
||||
// be one every variant carries before a variant is picked — a path that resolves
|
||||
// at all resolves on every call.
|
||||
func descend(s *session, root node, segments []string) (node, error) {
|
||||
var found node
|
||||
err := walkPath(root, segments, pathWalk{
|
||||
choice: func(c *choice, rest []string) ([]node, error) {
|
||||
if err := carriedByAll(c, rest); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return []node{pick(s, c)}, nil
|
||||
},
|
||||
leaf: func(n node) error { found = n; return nil },
|
||||
})
|
||||
return found, err
|
||||
// descend walks named fields to the node a path names, pinning the table rows it
|
||||
// selects or draws in sc. It is the one render-side step that can fail, because the
|
||||
// path comes from the caller and may name a field or a row that does not exist. A
|
||||
// choice consumes no segment, so the rest of the path must be one every variant
|
||||
// carries before a variant is picked — a path that resolves at all resolves on
|
||||
// every call.
|
||||
func descend(s *session, root node, segments []string, sc drawScope) (node, error) {
|
||||
return walkPath(root, segments, pathWalk{pins: sc.draws(s)})
|
||||
}
|
||||
|
||||
// render evaluates a compiled node to a string. compile validates every node up
|
||||
@@ -58,6 +57,20 @@ func render(s *session, n node, sc drawScope) string {
|
||||
return render(s, pick(s, n), sc)
|
||||
case *null:
|
||||
return ""
|
||||
case *table:
|
||||
sc.t, sc.row = n, n.draw(s)
|
||||
return expand(s, n.format, sc)
|
||||
case *column:
|
||||
if sc.t != n.t {
|
||||
sc.t, sc.row = n.t, sc.draws(s).mustRow(n.t)
|
||||
}
|
||||
if n.i < 0 {
|
||||
return expand(s, n.t.format, sc)
|
||||
}
|
||||
if cell := n.t.cellNode(sc.row, n.i); cell != nil {
|
||||
return render(s, cell, sc)
|
||||
}
|
||||
return n.t.cell(sc.row, n.i)
|
||||
case *template:
|
||||
sc = sc.in(n)
|
||||
if n.repeat == 1 {
|
||||
@@ -85,20 +98,20 @@ func render(s *session, n node, sc drawScope) string {
|
||||
//
|
||||
//go:noinline
|
||||
func expandAnew(s *session, t *template) string {
|
||||
var set drawSet
|
||||
set := drawSet{unnamed: draws{s: s}}
|
||||
return expand(s, t, drawScope{set: &set})
|
||||
}
|
||||
|
||||
// pick selects one item. Uniform choices are O(1); weighted choices are an
|
||||
// O(log n) search over precomputed cumulative weights. compile guarantees a
|
||||
// non-empty choice and a finite positive total, so the index is always in range.
|
||||
func pick(r rng, c *choice) node {
|
||||
// The session is concrete rather than the rng interface, which would make the
|
||||
// walk that draws through it leak its draw set to the heap.
|
||||
func pick(s *session, c *choice) node {
|
||||
if c.cum == nil {
|
||||
return c.items[r.IntN(len(c.items))]
|
||||
return c.items[s.IntN(len(c.items))]
|
||||
}
|
||||
x := r.Float64() * c.cum[len(c.cum)-1]
|
||||
i := sort.Search(len(c.cum), func(i int) bool { return c.cum[i] > x })
|
||||
return c.items[i]
|
||||
return c.items[pickCum(s, c.cum)]
|
||||
}
|
||||
|
||||
// expand renders a template's compiled ops. compile validated every token, so this
|
||||
@@ -110,7 +123,7 @@ func expand(s *session, t *template, sc drawScope) string {
|
||||
// repeat iteration get their own, since each is its own expansion. A reference
|
||||
// path reads the render's draws in sc instead.
|
||||
var held *draws
|
||||
if len(t.held) > 0 {
|
||||
if t.heldLocal {
|
||||
held = &draws{
|
||||
variant: make(map[string]node, len(t.held)),
|
||||
value: make(map[string]draw, len(t.held)),
|
||||
|
||||
@@ -43,6 +43,11 @@ func shippedShape(f *Generator) string {
|
||||
}
|
||||
case *choice:
|
||||
facts[prefix] = reads(n)
|
||||
case *table:
|
||||
facts[prefix] = "\tformat " + strconv.Quote(n.format.format) + tableFacts(n) + reads(n)
|
||||
for _, name := range n.columns {
|
||||
facts[join(prefix, name)] = "\tstring"
|
||||
}
|
||||
case *template:
|
||||
facts[prefix] = "\tformat " + strconv.Quote(n.format) + reads(n)
|
||||
if _, columns, err := recordOf(n); err == nil {
|
||||
@@ -66,6 +71,20 @@ func shippedShape(f *Generator) string {
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// tableFacts names the columns a table's options read.
|
||||
func tableFacts(t *table) string {
|
||||
var b strings.Builder
|
||||
for _, o := range []struct {
|
||||
name string
|
||||
col int
|
||||
}{{"key", t.key}, {"name", t.name}, {"weight", t.weight}, {"parent", t.parent}} {
|
||||
if o.col >= 0 {
|
||||
b.WriteString("\t" + o.name + " " + t.columns[o.col])
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// reads names the categories any template under n references, sorted.
|
||||
func reads(n node) string {
|
||||
set := map[string]bool{}
|
||||
@@ -76,6 +95,11 @@ func reads(n node) string {
|
||||
for _, it := range n.items {
|
||||
collect(it)
|
||||
}
|
||||
case *table:
|
||||
collect(n.format)
|
||||
for _, cell := range n.tokens {
|
||||
collect(cell)
|
||||
}
|
||||
case *template:
|
||||
for _, b := range n.refs {
|
||||
set[strings.TrimPrefix(b.key, "/")] = true
|
||||
|
||||
@@ -349,7 +349,8 @@ func (p *valueProof) checkField(label string, ft reflect.Type, column node) erro
|
||||
// fill draws the record into v's tagged fields, then each nested struct as a record of its own.
|
||||
func (s *structShape) fill(sess *session, v reflect.Value) {
|
||||
if s.record != nil {
|
||||
for i, c := range renderRecord(sess, s.record, s.columns).columns {
|
||||
set := newDrawSet(sess)
|
||||
for i, c := range renderRecord(sess, s.record, s.columns, drawScope{set: &set}).columns {
|
||||
setColumn(fieldAt(v, s.fields[i]), c)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,486 @@
|
||||
package fejkdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"math"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// table is a category whose rows come from a TSV beside it: the header names the
|
||||
// columns, each row is one draw, and the format renders the drawn row. A column is a
|
||||
// cell of the row a render pinned; a cell carrying tokens compiles to a string node.
|
||||
type table struct {
|
||||
category string
|
||||
file string
|
||||
format *template // fields are the column nodes
|
||||
columns []string
|
||||
col map[string]int
|
||||
fields map[string]node // column nodes, the format's fields
|
||||
whole *column // the pinned row rendered by the format
|
||||
cells []string // rows × columns, flat
|
||||
tokens map[int]*template
|
||||
key int // column index, or -1
|
||||
name int
|
||||
weight int
|
||||
parent int
|
||||
cum []float64 // cumulative weights, nil when uniform
|
||||
byKey map[string]int
|
||||
parentT *table
|
||||
children map[string]*table
|
||||
once sync.Once
|
||||
index tableIndex
|
||||
}
|
||||
|
||||
// tableIndex is what selection and linked draws look up, built on the first draw
|
||||
// that needs it.
|
||||
type tableIndex struct {
|
||||
byName map[string][]int
|
||||
children map[string][]int // rows by parent key
|
||||
childCum map[string][]float64 // cumulative weights per parent key, nil when uniform
|
||||
}
|
||||
|
||||
func (*table) isNode() {}
|
||||
|
||||
// column is one column of a table, rendered as the cell of the row the render
|
||||
// pinned; i < 0 is the whole row rendered by the table's format.
|
||||
type column struct {
|
||||
t *table
|
||||
i int
|
||||
}
|
||||
|
||||
func (*column) isNode() {}
|
||||
|
||||
func (t *table) rows() int { return len(t.cells) / len(t.columns) }
|
||||
|
||||
func (t *table) cell(row, col int) string { return t.cells[row*len(t.columns)+col] }
|
||||
|
||||
// cellNode is what a cell renders: its compiled template where it carries tokens,
|
||||
// else nil for its text.
|
||||
func (t *table) cellNode(row, col int) *template { return t.tokens[row*len(t.columns)+col] }
|
||||
|
||||
// tableOptions are the keys a table object takes; every other key is refused.
|
||||
var tableOptions = []string{"format", "key", "name", "parent", "rows", "weight"}
|
||||
|
||||
func isTableOption(name string) bool {
|
||||
for _, o := range tableOptions {
|
||||
if o == name {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// inSelector is what a key or name cell may not contain: the path grammar reserves
|
||||
// brackets and braces, and a quote or a bar would make the selector no path and no
|
||||
// token arm.
|
||||
const inSelector = `[]{}"|`
|
||||
|
||||
// compileTable compiles a category object naming a rows file into a table.
|
||||
func compileTable(m map[string]any, category string, files *categoryFiles) (*table, error) {
|
||||
o, err := readTableOptions(m)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data, err := files.rows(o.rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
t := &table{category: category, file: o.rows, key: -1, name: -1, weight: -1, parent: -1}
|
||||
if err := t.parseRows(data); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", o.rows, err)
|
||||
}
|
||||
if err := t.bindOptions(o); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := t.checkCells(); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", o.rows, err)
|
||||
}
|
||||
if err := t.compileFormat(o.format); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
|
||||
type tableOptionValues struct {
|
||||
format, key, name, parent, rows, weight string
|
||||
}
|
||||
|
||||
func readTableOptions(m map[string]any) (tableOptionValues, error) {
|
||||
var o tableOptionValues
|
||||
for k := range m {
|
||||
if !isTableOption(k) {
|
||||
return o, fmt.Errorf("a table takes %s; %q is none of them", strings.Join(tableOptions, ", "), k)
|
||||
}
|
||||
}
|
||||
for k, into := range map[string]*string{"format": &o.format, "key": &o.key, "name": &o.name, "parent": &o.parent, "rows": &o.rows, "weight": &o.weight} {
|
||||
v, ok := m[k]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
s, isString := v.(string)
|
||||
if !isString {
|
||||
return o, fmt.Errorf("%s must be a string, got %T", k, v)
|
||||
}
|
||||
if k != "format" && s == "" {
|
||||
return o, fmt.Errorf("%s names a column, so it cannot be empty; drop it", k)
|
||||
}
|
||||
*into = s
|
||||
}
|
||||
if _, ok := m["format"]; !ok {
|
||||
return o, fmt.Errorf("template object missing string \"format\"")
|
||||
}
|
||||
if !strings.HasSuffix(o.rows, ".tsv") || strings.Contains(o.rows, "/") {
|
||||
return o, fmt.Errorf("rows names a .tsv file beside the category, not %q", o.rows)
|
||||
}
|
||||
if o.key != "" && o.key == o.name {
|
||||
return o, fmt.Errorf("name names the key column %q, which a selector already reads; drop it", o.name)
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// parseRows reads the TSV: the header line names the columns, each following line
|
||||
// is a row of as many cells. Cells are substrings of data, so the file is held once.
|
||||
func (t *table) parseRows(data string) error {
|
||||
data = strings.TrimSuffix(data, "\n")
|
||||
header, rest, _ := strings.Cut(data, "\n")
|
||||
if data == "" {
|
||||
return fmt.Errorf("has no header line naming its columns")
|
||||
}
|
||||
t.columns = strings.Split(strings.TrimSuffix(header, "\r"), "\t")
|
||||
t.col = make(map[string]int, len(t.columns))
|
||||
t.fields = make(map[string]node, len(t.columns))
|
||||
for i, name := range t.columns {
|
||||
if err := checkName(name); err != nil {
|
||||
return fmt.Errorf("column %w", err)
|
||||
}
|
||||
if _, dup := t.col[name]; dup {
|
||||
return fmt.Errorf("column %q is named twice", name)
|
||||
}
|
||||
t.col[name] = i
|
||||
t.fields[name] = &column{t, i}
|
||||
}
|
||||
if header == data {
|
||||
return fmt.Errorf("has no rows below its header; a table is at least two rows")
|
||||
}
|
||||
m := len(t.columns)
|
||||
t.cells = make([]string, 0, m*(strings.Count(rest, "\n")+1))
|
||||
for line, n := 2, 0; rest != ""; line++ {
|
||||
var row string
|
||||
row, rest, _ = strings.Cut(rest, "\n")
|
||||
row = strings.TrimSuffix(row, "\r")
|
||||
for n = 0; n < m; n++ {
|
||||
cell, more, tab := strings.Cut(row, "\t")
|
||||
if !tab && n < m-1 || tab && n == m-1 {
|
||||
return fmt.Errorf("line %d has %s cells than the %d columns", line, map[bool]string{true: "more", false: "fewer"}[tab], m)
|
||||
}
|
||||
t.cells = append(t.cells, cell)
|
||||
row = more
|
||||
}
|
||||
}
|
||||
if t.rows() == 1 {
|
||||
return fmt.Errorf("has one row, which is a template; write it as one")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// bindOptions resolves each option to its column and proves what it claims of the
|
||||
// cells: a key is unique, a weight a positive number.
|
||||
func (t *table) bindOptions(o tableOptionValues) error {
|
||||
for _, opt := range []struct {
|
||||
name, value string
|
||||
into *int
|
||||
}{{"key", o.key, &t.key}, {"name", o.name, &t.name}, {"parent", o.parent, &t.parent}, {"weight", o.weight, &t.weight}} {
|
||||
if opt.value == "" {
|
||||
continue
|
||||
}
|
||||
i, ok := t.col[opt.value]
|
||||
if !ok {
|
||||
return fmt.Errorf("%s names no column %q of %s; the columns are %v", opt.name, opt.value, o.rows, t.columns)
|
||||
}
|
||||
*opt.into = i
|
||||
}
|
||||
if err := t.indexKeys(); err != nil {
|
||||
return err
|
||||
}
|
||||
return t.sumWeights()
|
||||
}
|
||||
|
||||
// indexKeys proves every key names one row, and keeps the index a link is proved by.
|
||||
func (t *table) indexKeys() error {
|
||||
if t.key < 0 {
|
||||
return nil
|
||||
}
|
||||
t.byKey = make(map[string]int, t.rows())
|
||||
for r := 0; r < t.rows(); r++ {
|
||||
k := t.cell(r, t.key)
|
||||
if k == "" {
|
||||
return fmt.Errorf("%s line %d: the key is empty", t.file, r+2)
|
||||
}
|
||||
if first, dup := t.byKey[k]; dup {
|
||||
return fmt.Errorf("%s line %d: key %q repeats line %d; a key selects one row", t.file, r+2, k, first+2)
|
||||
}
|
||||
t.byKey[k] = r
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sumWeights proves every weight is a positive number and builds the cumulative table.
|
||||
func (t *table) sumWeights() error {
|
||||
if t.weight < 0 {
|
||||
return nil
|
||||
}
|
||||
t.cum = make([]float64, t.rows())
|
||||
total := 0.0
|
||||
for r := range t.cum {
|
||||
w, err := strconv.ParseFloat(t.cell(r, t.weight), 64)
|
||||
if err != nil || math.IsInf(w, 0) || math.IsNaN(w) || w <= 0 {
|
||||
return fmt.Errorf("%s line %d: weight %q is not a positive number", t.file, r+2, t.cell(r, t.weight))
|
||||
}
|
||||
total += w
|
||||
t.cum[r] = total
|
||||
}
|
||||
if math.IsInf(total, 0) {
|
||||
return fmt.Errorf("%s: the weights sum past the largest number", t.file)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkCells compiles every cell carrying a token, and fences what a selector reads.
|
||||
func (t *table) checkCells() error {
|
||||
for i, cell := range t.cells {
|
||||
row, col := i/len(t.columns), i%len(t.columns)
|
||||
if (col == t.key || col == t.name) && strings.ContainsAny(cell, inSelector) {
|
||||
return fmt.Errorf("line %d: %s %q contains %q, which a selector cannot spell", row+2, t.columns[col], cell, cell[strings.IndexAny(cell, inSelector):][:1])
|
||||
}
|
||||
if !strings.ContainsAny(cell, "{}") {
|
||||
continue
|
||||
}
|
||||
n, err := compileString(cell)
|
||||
if err != nil {
|
||||
return fmt.Errorf("line %d, %s: %w", row+2, t.columns[col], err)
|
||||
}
|
||||
if t.tokens == nil {
|
||||
t.tokens = map[int]*template{}
|
||||
}
|
||||
t.tokens[i] = n.(*template)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// compileFormat compiles the format over the columns as its fields.
|
||||
func (t *table) compileFormat(format string) error {
|
||||
for _, name := range fieldTokens(format) {
|
||||
a := splitArm(name, nil)
|
||||
if _, ok := t.col[a.key]; !ok && !isRef(a.key) && a.key != "" {
|
||||
return fmt.Errorf("format names no column %q of %s; the columns are %v", a.key, t.file, t.columns)
|
||||
}
|
||||
}
|
||||
if err := checkTokens(format, t.fields); err != nil {
|
||||
return err
|
||||
}
|
||||
t.format = &template{format: format, fields: t.fields, repeat: 1, record: true, table: t}
|
||||
t.whole = &column{t, -1}
|
||||
return t.format.compileFormat()
|
||||
}
|
||||
|
||||
// linkTables binds every table's parent to the table beside it, and proves the
|
||||
// links: a parent has a key, every link cell is one, every parent row is linked
|
||||
// to, no chain of parents closes, and no child is named like a parent's column.
|
||||
func linkTables(root map[string]node) error {
|
||||
var walk func(dir string, children map[string]node) error
|
||||
walk = func(dir string, children map[string]node) error {
|
||||
for _, name := range sortedNames(children) {
|
||||
path := join(dir, name)
|
||||
switch n := children[name].(type) {
|
||||
case *folder:
|
||||
if err := walk(path, n.children); err != nil {
|
||||
return err
|
||||
}
|
||||
case *table:
|
||||
n.category = name
|
||||
if n.parent < 0 {
|
||||
continue
|
||||
}
|
||||
if err := n.linkParent(path, children); err != nil {
|
||||
return fmt.Errorf("%s: %w", path, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return walk("", root)
|
||||
}
|
||||
|
||||
func (t *table) linkParent(path string, siblings map[string]node) error {
|
||||
name := t.columns[t.parent]
|
||||
p, ok := siblings[name].(*table)
|
||||
switch {
|
||||
case siblings[name] == nil:
|
||||
return fmt.Errorf("parent %q names no table beside it", name)
|
||||
case !ok:
|
||||
return fmt.Errorf("parent %q is not a table; a link column reads a table's key", name)
|
||||
case p.key < 0:
|
||||
return fmt.Errorf("parent %q has no key column to link to", name)
|
||||
}
|
||||
for q, seen := p, map[*table]bool{t: true}; q != nil; q, _ = siblings[q.columns[q.parent]].(*table) {
|
||||
if seen[q] {
|
||||
return fmt.Errorf("parent cycle: %s reaches itself through its parents", q.category)
|
||||
}
|
||||
seen[q] = true
|
||||
if q.parent < 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
if _, clash := p.col[t.category]; clash {
|
||||
return fmt.Errorf("%q is named like a column of its parent %q, so %s.%s could read either; rename one", t.category, name, name, t.category)
|
||||
}
|
||||
linked := make(map[string]bool, p.rows())
|
||||
for r := 0; r < t.rows(); r++ {
|
||||
k := t.cell(r, t.parent)
|
||||
if _, ok := p.byKey[k]; !ok {
|
||||
return fmt.Errorf("%s line %d: %s %q is no key of %s", t.file, r+2, name, k, p.file)
|
||||
}
|
||||
linked[k] = true
|
||||
}
|
||||
for r := 0; r < p.rows(); r++ {
|
||||
if k := p.cell(r, p.key); !linked[k] {
|
||||
return fmt.Errorf("%s links no row to %s %q; every %s row needs one, or drop line %d of %s", t.file, name, k, name, r+2, p.file)
|
||||
}
|
||||
}
|
||||
t.parentT = p
|
||||
if p.children == nil {
|
||||
p.children = map[string]*table{}
|
||||
}
|
||||
p.children[t.category] = t
|
||||
return nil
|
||||
}
|
||||
|
||||
func (t *table) indexed() *tableIndex {
|
||||
t.once.Do(func() {
|
||||
if t.name >= 0 {
|
||||
t.index.byName = make(map[string][]int, t.rows())
|
||||
for r := 0; r < t.rows(); r++ {
|
||||
n := t.cell(r, t.name)
|
||||
t.index.byName[n] = append(t.index.byName[n], r)
|
||||
}
|
||||
}
|
||||
if t.parent >= 0 {
|
||||
t.index.children = map[string][]int{}
|
||||
for r := 0; r < t.rows(); r++ {
|
||||
k := t.cell(r, t.parent)
|
||||
t.index.children[k] = append(t.index.children[k], r)
|
||||
}
|
||||
if t.cum != nil {
|
||||
t.index.childCum = make(map[string][]float64, len(t.index.children))
|
||||
for k, rows := range t.index.children {
|
||||
cum, total := make([]float64, len(rows)), 0.0
|
||||
for i, r := range rows {
|
||||
total += t.cum[r] - t.cumBefore(r)
|
||||
cum[i] = total
|
||||
}
|
||||
t.index.childCum[k] = cum
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
return &t.index
|
||||
}
|
||||
|
||||
func (t *table) cumBefore(r int) float64 {
|
||||
if r == 0 {
|
||||
return 0
|
||||
}
|
||||
return t.cum[r-1]
|
||||
}
|
||||
|
||||
// draw picks a row over the whole table. The session is concrete rather than the rng
|
||||
// interface so that a walk holding the draws allocates nothing.
|
||||
func (t *table) draw(s *session) int {
|
||||
if t.cum == nil {
|
||||
return s.IntN(t.rows())
|
||||
}
|
||||
return pickCum(s, t.cum)
|
||||
}
|
||||
|
||||
func pickCum(s *session, cum []float64) int {
|
||||
x := s.Float64() * cum[len(cum)-1]
|
||||
return sort.Search(len(cum), func(i int) bool { return cum[i] > x })
|
||||
}
|
||||
|
||||
// drawUnder picks a row among those linked to parent row pr.
|
||||
func (t *table) drawUnder(s *session, pr int) int {
|
||||
ix := t.indexed()
|
||||
k := t.parentT.cell(pr, t.parentT.key)
|
||||
rows := ix.children[k]
|
||||
if ix.childCum == nil {
|
||||
return rows[s.IntN(len(rows))]
|
||||
}
|
||||
return rows[pickCum(s, ix.childCum[k])]
|
||||
}
|
||||
|
||||
// descendant is the table named name among those linked to t, at any depth.
|
||||
func (t *table) descendant(name string) *table {
|
||||
if c, ok := t.children[name]; ok {
|
||||
return c
|
||||
}
|
||||
for _, c := range t.children {
|
||||
if d := c.descendant(name); d != nil {
|
||||
return d
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// parentRow is the row of t's parent that row r links to.
|
||||
func (t *table) parentRow(r int) int { return t.parentT.byKey[t.cell(r, t.parent)] }
|
||||
|
||||
// under reports whether row r of t sits inside row pr of ancestor a.
|
||||
func (t *table) under(r int, a *table, pr int) bool {
|
||||
for c, row := t, r; c.parentT != nil; c, row = c.parentT, c.parentRow(row) {
|
||||
if c.parentT == a {
|
||||
return c.parentRow(row) == pr
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// find is the row a selector names: by key first, then by name, where a name
|
||||
// naming several rows resolves inside the ancestors pinned in d.
|
||||
func (t *table) find(sel string, d *draws) (int, error) {
|
||||
if t.key < 0 && t.name < 0 {
|
||||
return 0, fmt.Errorf("%s has no key or name column to select a row by", t.category)
|
||||
}
|
||||
if r, ok := t.byKey[sel]; ok {
|
||||
return r, nil
|
||||
}
|
||||
rows := t.indexed().byName[sel]
|
||||
if len(rows) > 1 {
|
||||
rows = d.inside(t, rows)
|
||||
}
|
||||
switch len(rows) {
|
||||
case 1:
|
||||
return rows[0], nil
|
||||
case 0:
|
||||
return 0, fmt.Errorf("no row of %s has key or name %q", t.category, sel)
|
||||
}
|
||||
keys := make([]string, len(rows))
|
||||
for i, r := range rows {
|
||||
keys[i] = t.cell(r, t.key)
|
||||
}
|
||||
inside := ""
|
||||
if t.parentT != nil {
|
||||
inside = fmt.Sprintf(", or select it inside its %s", t.parentT.category)
|
||||
}
|
||||
return 0, fmt.Errorf("%q names %d rows of %s; select one by key, one of %v%s", sel, len(rows), t.category, keys, inside)
|
||||
}
|
||||
|
||||
// selectorSpelling is how a path writes a selected row, for messages.
|
||||
func (t *table) selectorSpelling(r int) string {
|
||||
if t.key >= 0 {
|
||||
return t.category + "[" + t.cell(r, t.key) + "]"
|
||||
}
|
||||
return t.category + "[" + t.cell(r, t.name) + "]"
|
||||
}
|
||||
+17
-17
@@ -81,19 +81,20 @@ type builtin struct {
|
||||
// for a plain field or alternation body. A '(' without a trailing ')' yields
|
||||
// ok=false; checkFunc reports it as malformed at compile time.
|
||||
func funcCall(body string) (name string, args []string, ok bool) {
|
||||
lp := strings.IndexByte(body, '(')
|
||||
lp := indexOutside(body, '(')
|
||||
if lp < 0 || !strings.HasSuffix(body, ")") {
|
||||
return "", nil, false
|
||||
}
|
||||
return body[:lp], splitArgs(body[lp+1 : len(body)-1]), true
|
||||
}
|
||||
|
||||
// splitArgs parses a function arg list: comma-separated, trimmed; empty -> none.
|
||||
// splitArgs parses a function arg list: comma-separated outside a selector,
|
||||
// trimmed; empty -> none.
|
||||
func splitArgs(s string) []string {
|
||||
if strings.TrimSpace(s) == "" {
|
||||
return nil
|
||||
}
|
||||
args := strings.Split(s, ",")
|
||||
args := splitOutside(s, ',')
|
||||
for i := range args {
|
||||
args[i] = strings.TrimSpace(args[i])
|
||||
}
|
||||
@@ -132,10 +133,10 @@ func checkTokens(format string, fields map[string]node) error {
|
||||
if t.kind != 'b' {
|
||||
return nil
|
||||
}
|
||||
if strings.IndexByte(t.body, '(') >= 0 { // a function token, not a field
|
||||
if indexOutside(t.body, '(') >= 0 { // a function token, not a field
|
||||
return checkFunc(t.body, fields)
|
||||
}
|
||||
names := strings.Split(t.body, "|")
|
||||
names := splitOutside(t.body, '|')
|
||||
for _, name := range names {
|
||||
if isRef(name) {
|
||||
if _, _, err := refShape(name); err != nil {
|
||||
@@ -187,12 +188,7 @@ func checkArm(name string, fields map[string]node, wholeToken bool) error {
|
||||
// hintableRef reports whether {/name} is a reference the grammar accepts, so the
|
||||
// hint never names a spelling that fails too.
|
||||
func hintableRef(name string) bool {
|
||||
for _, seg := range strings.Split(name, ".") {
|
||||
if checkName(seg) != nil {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
return checkPathNames(name) == nil
|
||||
}
|
||||
|
||||
// tokenOperands lists the fields one {token} body reads as operands, empty for a
|
||||
@@ -244,8 +240,8 @@ func checkNoRepeatedArm(body string, names []string) error {
|
||||
func fieldTokens(format string) []string {
|
||||
var names []string
|
||||
_ = eachToken(format, func(t ftoken) error {
|
||||
if t.kind == 'b' && strings.IndexByte(t.body, '(') < 0 {
|
||||
names = append(names, strings.Split(t.body, "|")...)
|
||||
if t.kind == 'b' && indexOutside(t.body, '(') < 0 {
|
||||
names = append(names, splitOutside(t.body, '|')...)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
@@ -279,11 +275,11 @@ func splitArm(name string, refs map[string]refBinding) arm {
|
||||
}
|
||||
return pathArm(name, b.key, b.tail)
|
||||
}
|
||||
head, tail, dotted := strings.Cut(name, ".")
|
||||
if !dotted {
|
||||
segs, err := splitPath(name)
|
||||
if err != nil || len(segs) == 1 {
|
||||
return arm{name: name, key: name, path: name}
|
||||
}
|
||||
return pathArm(name, head, strings.Split(tail, "."))
|
||||
return pathArm(name, segs[0], segs[1:])
|
||||
}
|
||||
|
||||
func pathArm(name, key string, segs []string) arm {
|
||||
@@ -296,7 +292,7 @@ func pathArm(name, key string, segs []string) arm {
|
||||
|
||||
// splitArms splits a token body's '|' alternatives.
|
||||
func splitArms(body string, refs map[string]refBinding) []arm {
|
||||
parts := strings.Split(body, "|")
|
||||
parts := splitOutside(body, '|')
|
||||
arms := make([]arm, len(parts))
|
||||
for i, p := range parts {
|
||||
arms[i] = splitArm(p, refs)
|
||||
@@ -352,6 +348,7 @@ type formatOps struct {
|
||||
bound map[string]string
|
||||
held map[string]bool
|
||||
holder map[string]string
|
||||
heldLocal bool // some held name is kept by the expansion itself rather than the render's draws
|
||||
}
|
||||
|
||||
func (c *formatOps) hold(a arm, label string) {
|
||||
@@ -360,6 +357,9 @@ func (c *formatOps) hold(a arm, label string) {
|
||||
c.holder = map[string]string{}
|
||||
}
|
||||
c.held[a.key] = true
|
||||
if !isRef(a.key) || len(a.tail) == 0 {
|
||||
c.heldLocal = true
|
||||
}
|
||||
if _, named := c.holder[a.key]; !named {
|
||||
c.holder[a.key] = label
|
||||
}
|
||||
|
||||
@@ -85,6 +85,8 @@ func (p *valueProof) of(n node) proven {
|
||||
v = p.unite(n.items)
|
||||
case *template:
|
||||
v = p.template(n)
|
||||
case *column:
|
||||
v = p.cells(n)
|
||||
default:
|
||||
v = unproven(`it reads a null, which renders "" outside its own column`)
|
||||
}
|
||||
@@ -92,6 +94,28 @@ func (p *valueProof) of(n node) proven {
|
||||
return v
|
||||
}
|
||||
|
||||
// cells proves a table column over every cell it may render.
|
||||
func (p *valueProof) cells(c *column) proven {
|
||||
if c.i < 0 {
|
||||
return unproven(fmt.Sprintf("%q renders a row of %s, which is composed text", c.t.format.format, c.t.category))
|
||||
}
|
||||
var v proven
|
||||
for r := 0; r < c.t.rows(); r++ {
|
||||
var w proven
|
||||
if cell := c.t.cellNode(r, c.i); cell != nil {
|
||||
w = p.of(cell)
|
||||
} else {
|
||||
w = literalValue(c.t.cell(r, c.i))
|
||||
}
|
||||
if r == 0 {
|
||||
v = w
|
||||
} else {
|
||||
v = v.or(w)
|
||||
}
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func (p *valueProof) unite(nodes []node) proven {
|
||||
v := p.of(nodes[0])
|
||||
for _, n := range nodes[1:] {
|
||||
|
||||
Reference in New Issue
Block a user