Correct what the readers proved false, and lift the file map out of last place #30
@@ -270,7 +270,7 @@ f, err := fejkdata.New(fejkdata.WithSeed(42))
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatal(err)
|
log.Fatal(err)
|
||||||
}
|
}
|
||||||
v, err := f.Fake("sv_SE.address") // "Kungsvägen 68\n379 17 Stockholm"
|
v, err := f.Fake("sv_SE.address") // "Järvedsvägen 43\n891 77 Järved"
|
||||||
paths := f.List() // every path Fake accepts, sorted
|
paths := f.List() // every path Fake accepts, sorted
|
||||||
v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call
|
v, err = f.FakeTemplate("name: {/sv_SE.person.last}") // compile + render in one call
|
||||||
t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once
|
t, err := f.NewTemplate(`{"format":"name: {x}","x":["bosse","lina"]}`) // compile once
|
||||||
@@ -1280,6 +1280,38 @@ the Development section below, and who ships a register the four above then draw
|
|||||||
through a chain of five tables is every subsequence of it, and the direct chain is
|
through a chain of five tables is every subsequence of it, and the direct chain is
|
||||||
the one a reader can predict from the tables' parents.
|
the one a reader can predict from the tables' parents.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
fejkdata.go Generator, New, options, the embedded data set, List
|
||||||
|
node.go the node model and JSON -> node compilation
|
||||||
|
table.go tables: the rows TSV, its options and links, row selection and draws
|
||||||
|
path.go the dotted-path walk with its selectors, and proving a path resolves
|
||||||
|
render.go Fake and the recursive renderer (choices, format strings, expansions)
|
||||||
|
record.go records: Record, the JSON/CSV/SQL serializers, and their entry points
|
||||||
|
struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype
|
||||||
|
inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link
|
||||||
|
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
|
||||||
|
hold.go the hold: one draw per expansion for paths and operands, and its fences
|
||||||
|
draw.go one reference draw per render and group: draw sets, the group option, and its fence
|
||||||
|
family.go a family of linked tables: the rows a render pins, and the fence over paths into one family
|
||||||
|
reference.go reference sigils, and binding references across the tree
|
||||||
|
graph.go the render graph: edges, cycles, the repeat bound, tree walks
|
||||||
|
builtins.go the {name()} function registry and its implementations
|
||||||
|
layout.go date and time layouts: the instants one is proved against, and the two samples
|
||||||
|
checksum.go the check characters a derivation appends, and the IBAN they sit inside
|
||||||
|
transform.go the builtins that rewrite an operand's value, and the ASCII folding
|
||||||
|
calc.go the {calc()} arithmetic evaluator: parser, eval, validation
|
||||||
|
datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype
|
||||||
|
value.go the value proof: what a typed column or calc operand holds, checked at load
|
||||||
|
data.go data loading: fs.FS folders/files -> namespace tree, multi-source merge
|
||||||
|
cmd/fejkdata/ the fejkdata CLI
|
||||||
|
data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc
|
||||||
|
data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md)
|
||||||
|
release-tooling/ the release CI publishes from the changelog heading
|
||||||
|
testdata/ the pinned shipped shape (see Versioning)
|
||||||
|
```
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
Everything runs in Docker — **no local tooling beyond Docker is needed**.
|
Everything runs in Docker — **no local tooling beyond Docker is needed**.
|
||||||
@@ -1367,38 +1399,6 @@ and merge: once `main` passes the gate, CI tags that commit `vX.Y.Z` and publish
|
|||||||
the Gitea release with the section as its body. A top heading of `[Unreleased]`
|
the Gitea release with the section as its body. A top heading of `[Unreleased]`
|
||||||
publishes nothing.
|
publishes nothing.
|
||||||
|
|
||||||
## Layout
|
|
||||||
|
|
||||||
```
|
|
||||||
fejkdata.go Generator, New, options, the embedded data set, List
|
|
||||||
node.go the node model and JSON -> node compilation
|
|
||||||
table.go tables: the rows TSV, its options and links, row selection and draws
|
|
||||||
path.go the dotted-path walk with its selectors, and proving a path resolves
|
|
||||||
render.go Fake and the recursive renderer (choices, format strings, expansions)
|
|
||||||
record.go records: Record, the JSON/CSV/SQL serializers, and their entry points
|
|
||||||
struct.go structs: FakeStruct, fake tags, and a field's Go type as its column's datatype
|
|
||||||
inline.go inline templates: Template, NewTemplate, FakeTemplate, IsTemplate, and their compile and link
|
|
||||||
template.go the {token} grammar: scanning, tokens, operands, validation, compiling a format
|
|
||||||
hold.go the hold: one draw per expansion for paths and operands, and its fences
|
|
||||||
draw.go one reference draw per render and group: draw sets, the group option, and its fence
|
|
||||||
family.go a family of linked tables: the rows a render pins, and the fence over paths into one family
|
|
||||||
reference.go reference sigils, and binding references across the tree
|
|
||||||
graph.go the render graph: edges, cycles, the repeat bound, tree walks
|
|
||||||
builtins.go the {name()} function registry and its implementations
|
|
||||||
layout.go date and time layouts: the instants one is proved against, and the two samples
|
|
||||||
checksum.go the check characters a derivation appends, and the IBAN they sit inside
|
|
||||||
transform.go the builtins that rewrite an operand's value, and the ASCII folding
|
|
||||||
calc.go the {calc()} arithmetic evaluator: parser, eval, validation
|
|
||||||
datatype.go column datatypes: DataType, where datatype and null may sit, a column's datatype
|
|
||||||
value.go the value proof: what a typed column or calc operand holds, checked at load
|
|
||||||
data.go data loading: fs.FS folders/files -> namespace tree, multi-source merge
|
|
||||||
cmd/fejkdata/ the fejkdata CLI
|
|
||||||
data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc
|
|
||||||
data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md)
|
|
||||||
release-tooling/ the release CI publishes from the changelog heading
|
|
||||||
testdata/ the pinned shipped shape (see Versioning)
|
|
||||||
```
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT — see [LICENSE](LICENSE). Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).
|
MIT — see [LICENSE](LICENSE). Forked from [github.com/Timewave-AB/fakes](https://github.com/Timewave-AB/fakes).
|
||||||
|
|||||||
+6
-6
@@ -5,8 +5,8 @@
|
|||||||
// then generate values by path:
|
// then generate values by path:
|
||||||
//
|
//
|
||||||
// f, _ := fejkdata.New(fejkdata.WithSeed(42))
|
// f, _ := fejkdata.New(fejkdata.WithSeed(42))
|
||||||
// f.Fake("sv_SE.address") // "Storgatan 12\n234 56 Göteborg"
|
// f.Fake("sv_SE.address") // "Järvedsvägen 43\n891 77 Järved"
|
||||||
// f.Fake("sv_SE.address.locality") // "Göteborg"
|
// f.Fake("sv_SE.address.locality") // "Sundbyberg": a second call draws afresh
|
||||||
//
|
//
|
||||||
// Several sources merge in order, the last winning a name clash, so custom data
|
// Several sources merge in order, the last winning a name clash, so custom data
|
||||||
// layers over the built-ins. The JSON template format is documented in the README.
|
// layers over the built-ins. The JSON template format is documented in the README.
|
||||||
@@ -29,8 +29,8 @@ import (
|
|||||||
//go:embed data
|
//go:embed data
|
||||||
var shippedFS embed.FS
|
var shippedFS embed.FS
|
||||||
|
|
||||||
// MaxRepeat caps a repeat, and the renders nested repeats multiply to along any
|
// MaxRepeat caps a repeat, and caps the renders nested repeats multiply to along
|
||||||
// path; the CLI's --repeat shares it.
|
// any one path; the CLI's --repeat shares it.
|
||||||
const MaxRepeat = 1 << 20
|
const MaxRepeat = 1 << 20
|
||||||
|
|
||||||
var _ [^uint(0)>>63 - 1]struct{} // 64-bit only, per the README's Decisions
|
var _ [^uint(0)>>63 - 1]struct{} // 64-bit only, per the README's Decisions
|
||||||
@@ -40,8 +40,8 @@ var ErrNoData = errors.New("no data: WithoutShippedData needs at least one WithD
|
|||||||
|
|
||||||
// Generator generates fake data from a loaded namespace tree. Create one with [New].
|
// Generator generates fake data from a loaded namespace tree. Create one with [New].
|
||||||
// It is safe for concurrent use; a seeded sequence is reproducible only when drawn
|
// It is safe for concurrent use; a seeded sequence is reproducible only when drawn
|
||||||
// from one goroutine. The compiled tree is immutable after [New], and Fake,
|
// from one goroutine. The compiled tree is immutable after [New], so List reads it
|
||||||
// NewTemplate and List read it concurrently without a lock.
|
// without a lock; mu guards the rest, which every other entry point takes.
|
||||||
type Generator struct {
|
type Generator struct {
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
rand *session
|
rand *session
|
||||||
|
|||||||
@@ -9,9 +9,9 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
// node is a compiled template element: a choice or a template. Compiling JSON
|
// node is a compiled element of the namespace tree: a folder, choice, null,
|
||||||
// into these once (see compile) means rendering never re-inspects the raw JSON or
|
// template, table, column or row. Compiling JSON into these once (see compile)
|
||||||
// re-sums weights.
|
// means rendering never re-inspects the raw JSON or re-sums weights.
|
||||||
type node interface{ isNode() }
|
type node interface{ isNode() }
|
||||||
|
|
||||||
// folder is a namespace of named children, built from a directory of JSON files
|
// folder is a namespace of named children, built from a directory of JSON files
|
||||||
|
|||||||
@@ -5,8 +5,8 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
// rng is the randomness the renderer draws from. Passing it in keeps the render
|
// rng is the randomness a builtin sample draws from; *rand.Rand satisfies it. The
|
||||||
// functions a pure core over an explicit effect; *rand.Rand satisfies it.
|
// render path takes the concrete *session instead, for the reason below.
|
||||||
type rng interface {
|
type rng interface {
|
||||||
IntN(n int) int
|
IntN(n int) int
|
||||||
Float64() float64
|
Float64() float64
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ type table struct {
|
|||||||
whole *row // the pinned row rendered by the format
|
whole *row // the pinned row rendered by the format
|
||||||
cells []string // rows × columns, flat
|
cells []string // rows × columns, flat
|
||||||
tokens map[int]*template
|
tokens map[int]*template
|
||||||
key int // column index, or -1
|
key int // key, name, weight and parent are column indexes, or -1
|
||||||
name int
|
name int
|
||||||
weight int
|
weight int
|
||||||
parent int
|
parent int
|
||||||
|
|||||||
@@ -30,6 +30,15 @@ by a `data-import/` script, as README goal 10 asks.
|
|||||||
- Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does
|
- Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does
|
||||||
not hold: ISO 3166 reserves it for Western Sahara and the root zone has never been
|
not hold: ISO 3166 reserves it for Western Sahara and the root zone has never been
|
||||||
delegated it, so no resolver answers for it and goal 1 promises otherwise.
|
delegated it, so no resolver answers for it and goal 1 promises otherwise.
|
||||||
|
- Draw `en_US.phone`'s `exch` as a NANP central office code: `{int(100,999)}` renders
|
||||||
|
a leading 1 in about an eighth of draws, which libphonenumber rejects, and
|
||||||
|
`TestShippedUSPhone` proves the shape rather than the rule. Assert the rule with it.
|
||||||
|
- Give a `sv_SE.personnummer` over 100 the `+` separator Skatteverket spells, or stop
|
||||||
|
drawing birthdates that reach 100: the format hard-codes `-`, and the 1930 floor
|
||||||
|
makes the oldest draws invalid from 2031.
|
||||||
|
- Settle `en_US.ip` and `sv_SE.ip`, today byte-identical: either one `misc.ip` as the
|
||||||
|
decision "`misc` is what every locale shares" asks, or a decision saying why an
|
||||||
|
address that carries no locale stays per-locale.
|
||||||
- Accept a middle name, and draw a shipped `personnummer` inside a *selected* sex;
|
- Accept a middle name, and draw a shipped `personnummer` inside a *selected* sex;
|
||||||
both want a draw group sharing its family's pins. Stop the conflict error naming a
|
both want a draw group sharing its family's pins. Stop the conflict error naming a
|
||||||
rewrite that returns a different value where the read it conflicts with sits inside
|
rewrite that returns a different value where the read it conflicts with sits inside
|
||||||
@@ -113,6 +122,34 @@ by a `data-import/` script, as README goal 10 asks.
|
|||||||
on the leading dot its keys carry rather than on case, and `misc.loglevel[Error]`
|
on the leading dot its keys carry rather than on case, and `misc.loglevel[Error]`
|
||||||
misses although a column carries `Error`, so the near miss is worth naming whatever
|
misses although a column carries `Error`, so the near miss is worth naming whatever
|
||||||
shape it takes.
|
shape it takes.
|
||||||
|
- Split `hold.go`, `draw.go` and `family.go` by the phase each half runs in: every
|
||||||
|
one holds both a load-time fence and render-time machinery, `draws` is declared in
|
||||||
|
`hold.go` with its methods in `family.go`, and a reader cannot tell from a file name
|
||||||
|
whether a unit runs at `New` or at `Fake`. A comprehension panel of nine readers put
|
||||||
|
this cluster first for reading cost, unanimously.
|
||||||
|
- Move `nodeScope`, `checkScope` and `checkRenders` out of `graph.go`, whose README
|
||||||
|
Layout line promises edges, cycles, the repeat bound and walks: the order every load
|
||||||
|
fence runs in is the load path's spine and no file name carries it.
|
||||||
|
- Name the `draw` stem apart: the free `pinned` in `hold.go` is not `(*draws).pinned`
|
||||||
|
in `family.go`, `compileFormat` is a method on both `*template` and `*table`, and
|
||||||
|
`draws`, `drawSet`, `drawScope`, `drawWalk`, `drawAt`, `drawKey`, `drawVisit` and
|
||||||
|
`drawRoute` no longer tell each other apart.
|
||||||
|
- Spell one fresh draw set one way: `newDrawSet` exists, and `render.go` and `draw.go`
|
||||||
|
hand-roll `drawSet{unnamed: draws{s: s}}` at three sites; name the lazy-map variant
|
||||||
|
if the difference is deliberate.
|
||||||
|
- Say on the `template` struct which pass fills which field, and that `compileFormat`
|
||||||
|
runs a second time once `linkTemplateRefs` binds the references: nine of its fields
|
||||||
|
are written by four other files and the order lives only in `loadData`'s body.
|
||||||
|
- State on `rowSet` that it simulates `(*draws).pin` and `rowOf`, and prove the two
|
||||||
|
agree: the load-time walk reimplements the render-time pinning, and nothing catches
|
||||||
|
them drifting apart — the failure is data that loads and then renders a family that
|
||||||
|
disagrees.
|
||||||
|
- Define the vocabulary the fences are written in — fence, hold, pin, draw, draw group,
|
||||||
|
expansion against render, family, whole — where a reader of `hold.go` meets it. Today
|
||||||
|
it exists only across five README sections, and every panel reader paid for it first.
|
||||||
|
- Decide whether `data/misc`'s 33 flat files gain a level before v1.0.0: a folder is a
|
||||||
|
path segment, so `misc/net/tld` is a rename a consumer pays for, and goal 11 promises
|
||||||
|
the directory keeps growing.
|
||||||
- Let a table column carry a `datatype`, so `--format json` writes
|
- Let a table column carry a `datatype`, so `--format json` writes
|
||||||
`"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only
|
`"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only
|
||||||
a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text
|
a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text
|
||||||
|
|||||||
Reference in New Issue
Block a user