diff --git a/README.md b/README.md index cc89e05..deafb52 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,11 @@ lacked the locale coverage and format control we needed. - **Standards first** — formats, names and structures follow international and local standards first and foremost. - **Locale-aware** — names, addresses, postal codes and phone numbers follow - per-locale data and formats. Locales are always full language + territory - tags (`sv_SE`, never `sv`). + per-locale data and formats. The shipped data is organised by full locale tag + (`sv_SE`), but the engine treats folders as plain namespaces — name yours + anything. - **Data lives in JSON** — all source data is recursive JSON on disk, read when - you create a faker and then served from memory. Add or change locales without + you create a faker and then served from memory. Add or change data without touching the library. Behavior belongs in data too: the engine grows a built-in function only for what data can't express (a checksum, a time-based id), never for what character classes and choices already do. @@ -23,25 +24,27 @@ lacked the locale coverage and format control we needed. ## CLI -Install the `fakes` command, then point it at a locale directory and a category -path — it prints one value to stdout. The first path segment names a category (a -JSON file); deeper dotted segments descend into it. +Install the `fakes` command, then point it at one or more data directories and a +path — it prints one value to stdout. Each dot segment descends one level: +folders, then the category (a JSON file), then fields inside it. ```sh go install github.com/Timewave-AB/fakes/cmd/fakes@latest -fakes ./locales/sv_SE person # Sara Eriksson -fakes ./locales/sv_SE person.last # Eriksson (dotted path into a category) -fakes -seed 42 ./locales/sv_SE address +fakes ./data/sv_SE person # Sara Eriksson +fakes ./data/sv_SE person.last # Eriksson (dotted path into a category) +fakes ./data sv_SE.person # point at the tree; the folder is a segment +fakes ./data/sv_SE ./mydata word # layer dirs; the last wins a name clash +fakes -seed 42 ./data/sv_SE address ``` Without installing, run it from a checkout with `go run ./cmd/fakes …`. Exit -codes: `0` success, `1` runtime error (bad locale, unknown path), `2` misuse. +codes: `0` success, `1` runtime error (missing dir, unknown path), `2` misuse. ### Generating a file from a custom template -A category is just a JSON file in the locale directory, so you can drop in your -own and render it — no code change. Save this as `locales/sv_SE/sql.json`: +A category is just a JSON file in a data directory, so you can drop in your +own and render it — no code change. Save this as `data/sv_SE/sql.json`: ```json { @@ -58,10 +61,10 @@ own and render it — no code change. Save this as `locales/sv_SE/sql.json`: `sql-username` renders `'{username}'` `repeat` times and joins the results with the `),(` separator; the outer `V#ALUES(…)` wraps that into one valid row list. (`#A` escapes the literal `A`, which a format string would otherwise read as a -letter token — see [Locale data format](#locale-data-format).) +letter token — see [Data format](#data-format).) ```sh -fakes -seed 1 ./locales/sv_SE sql +fakes -seed 1 ./data/sv_SE sql # INSERT INTO users VALUES('zoom'),('wahoo'),('blip'); ``` @@ -69,7 +72,7 @@ Raise `repeat` for more rows per statement, or loop in the shell to build a whole seed file: ```sh -for _ in $(seq 100); do fakes ./locales/sv_SE sql; done > seed.sql +for _ in $(seq 100); do fakes ./data/sv_SE sql; done > seed.sql ``` ## Library @@ -78,9 +81,9 @@ for _ in $(seq 100); do fakes ./locales/sv_SE sql; done > seed.sql go get github.com/Timewave-AB/fakes # requires Go 1.22+ (for math/rand/v2) ``` -Point `New` at a locale directory, then generate values by category path with -`Fake`. The first path segment names a category (a JSON file); deeper dotted -segments descend into it. +Point `New` at one or more data directories, then generate values by path with +`Fake`. Each dot segment descends one level: folders, then the category (a JSON +file), then fields inside it. ```go package main @@ -93,7 +96,7 @@ import ( ) func main() { - f, err := fakes.New("./locales/sv_SE") + f, err := fakes.New([]string{"./data/sv_SE"}) if err != nil { log.Fatal(err) } @@ -120,8 +123,8 @@ Seed a faker for reproducible output — same seed + locale yields an identical sequence, handy for stable tests: ```go -a, _ := fakes.New("./locales/sv_SE", fakes.WithSeed(42)) -b, _ := fakes.New("./locales/sv_SE", fakes.WithSeed(42)) +a, _ := fakes.New([]string{"./data/sv_SE"}, fakes.WithSeed(42)) +b, _ := fakes.New([]string{"./data/sv_SE"}, fakes.WithSeed(42)) av, _ := a.Fake("person") bv, _ := b.Fake("person") av == bv // true @@ -129,25 +132,34 @@ av == bv // true A `*Fakes` is **not** safe for concurrent use — create one per goroutine. -## Locales +## Data -The library ships a ready-to-use set under [`locales/`](locales) (`en_US`, -`sv_SE`). Point either tool at one of them, a copy, or your own directory — -anywhere on disk. +The library ships a ready-to-use set under [`data/`](data), organised by locale +(`en_US`, `sv_SE`). Point either tool at the whole tree, a single locale, a +copy, or your own directory — anywhere on disk; no naming rules. -Each ships these categories, formatted per locale (e.g. `date` is `MM/DD/YYYY` -in `en_US`, `YYYY-MM-DD` in `sv_SE`; `ssn` is a US SSN vs a Swedish -personnummer): `address`, `color`, `company`, `date`, `email`, `ip`, `person`, -`phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`, `uuid`, -`version`, `word`. +A directory is just a namespace. Each JSON file is a category named after the +file; each subdirectory is a dot-path segment — folders nest exactly like JSON +objects do. So `data/sv_SE/person.json` is `Fake("person")` when you point at +`data/sv_SE`, or `Fake("sv_SE.person")` when you point at `data`. -The directory's name is the locale and must be a full tag (`sv_SE`, never -`sv`). Any casing or separator is accepted and canonicalised (`sv-se`, -`SV_SE` → `sv_SE`); a non-full name returns an error. +Pass several directories and they merge, left to right: matching folders combine +by their children, and any other clash is won by the last directory loaded. That +lets you layer your own data over the built-ins without copying them: -## Locale data format +```go +fakes.New([]string{"./data/sv_SE", "./mydata"}) // mydata overrides on a clash +``` -Each JSON file in a locale directory is a **category** named after the file +Each shipped locale carries these categories, formatted per locale (e.g. `date` +is `MM/DD/YYYY` in `en_US`, `YYYY-MM-DD` in `sv_SE`; `ssn` is a US SSN vs a +Swedish personnummer): `address`, `color`, `company`, `date`, `email`, `ip`, +`person`, `phone`, `price`, `sentence`, `ssn`, `time`, `url`, `username`, +`uuid`, `version`, `word`. + +## Data format + +Each JSON file in a data directory is a **category** named after the file (`address.json` → `address`), rendered by `Fake("address")`. Drop in a new file or folder — no code change, no recompile. @@ -297,13 +309,13 @@ docker compose run --rm test # latest ``` fakes.go Fakes, New, options, seeding template.go Fake, the recursive renderer (choices, format strings, paths) -locale.go locale loading + tag parsing +data.go data loading: folders/files -> namespace tree, multi-path merge cmd/fakes/ the `fakes` CLI (New + Fake over stdout) -locales/ shipped locale data (JSON) +data/ shipped data (JSON), organised by locale ``` -To add a category, drop a JSON file into a locale directory; to add a locale, -add a directory named with its full tag. +To add a category, drop a JSON file into a data directory; to add a locale, add +a subdirectory of JSON files. ## License diff --git a/cmd/fakes/main.go b/cmd/fakes/main.go index e0fc4c3..536738c 100644 --- a/cmd/fakes/main.go +++ b/cmd/fakes/main.go @@ -1,10 +1,12 @@ -// Command fakes prints one fake value from a locale directory. +// Command fakes prints one fake value from one or more data directories. // -// fakes ./locales/sv_SE person # a full person -// fakes ./locales/sv_SE person.last # just the surname (dotted path) -// fakes -seed 42 ./locales/sv_SE address +// fakes ./data/sv_SE person # a full person +// fakes ./data/sv_SE person.last # just the surname (dotted path) +// fakes ./data sv_SE.person # point at the tree, address by folder +// fakes ./data/sv_SE ./mydata person # layer custom data; last dir wins +// fakes -seed 42 ./data/sv_SE address // -// It is a thin CLI over the fakes library: New(dir) then Fake(path). +// It is a thin CLI over the fakes library: New(dirs) then Fake(path). package main import ( @@ -16,11 +18,11 @@ import ( "github.com/Timewave-AB/fakes" ) -const usage = `Usage: fakes [-seed N] +const usage = `Usage: fakes [-seed N] ... - a locale directory, e.g. ./locales/sv_SE - a category, or a dotted path into one (person, person.last) - -seed N seed for reproducible output` + one or more data directories, e.g. ./data/sv_SE (last wins on clash) + a category, or a dotted path into one (person, person.last) + -seed N seed for reproducible output` func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } @@ -34,7 +36,7 @@ func run(args []string, stdout, stderr io.Writer) int { if err := fs.Parse(args); err != nil { return 2 } - if fs.NArg() != 2 { + if fs.NArg() < 2 { fs.Usage() return 2 } @@ -46,12 +48,13 @@ func run(args []string, stdout, stderr io.Writer) int { } }) - f, err := fakes.New(fs.Arg(0), opts...) + dirs, path := fs.Args()[:fs.NArg()-1], fs.Arg(fs.NArg()-1) + f, err := fakes.New(dirs, opts...) if err != nil { fmt.Fprintln(stderr, err) return 1 } - v, err := f.Fake(fs.Arg(1)) + v, err := f.Fake(path) if err != nil { fmt.Fprintln(stderr, err) return 1 diff --git a/cmd/fakes/main_test.go b/cmd/fakes/main_test.go index 2a879fa..a5e5452 100644 --- a/cmd/fakes/main_test.go +++ b/cmd/fakes/main_test.go @@ -6,7 +6,10 @@ import ( "testing" ) -const svSE = "../../locales/sv_SE" +const ( + svSE = "../../data/sv_SE" + enUS = "../../data/en_US" +) // runOut runs the CLI and returns exit code, stdout, stderr. func runOut(args ...string) (int, string, string) { @@ -54,8 +57,9 @@ func TestRunSeedDeterministic(t *testing.T) { } } -func TestRunUsageOnWrongArgCount(t *testing.T) { - for _, args := range [][]string{{}, {svSE}, {svSE, "person", "extra"}} { +func TestRunUsageOnTooFewArgs(t *testing.T) { + // Need at least one data dir plus a path; fewer is misuse. + for _, args := range [][]string{{}, {svSE}} { code, _, errb := runOut(args...) if code != 2 { t.Errorf("run(%v) = %d, want 2", args, code) @@ -66,6 +70,17 @@ func TestRunUsageOnWrongArgCount(t *testing.T) { } } +func TestRunMultipleDirs(t *testing.T) { + // Several data dirs then a final path: all but the last arg are dirs. + code, out, errb := runOut(enUS, svSE, "person") + if code != 0 { + t.Fatalf("run(multi-dir) = %d, stderr=%q", code, errb) + } + if strings.TrimSpace(out) == "" { + t.Fatal("empty output for multi-dir run") + } +} + func TestRunUnknownCategoryFails(t *testing.T) { code, _, errb := runOut(svSE, "nope") if code != 1 { @@ -76,8 +91,8 @@ func TestRunUnknownCategoryFails(t *testing.T) { } } -func TestRunBadLocaleFails(t *testing.T) { - code, _, errb := runOut("../../locales/sv", "person") +func TestRunMissingDirFails(t *testing.T) { + code, _, errb := runOut("../../data/nope", "person") if code != 1 { t.Fatalf("run = %d, want 1", code) } diff --git a/data.go b/data.go new file mode 100644 index 0000000..be1bcaa --- /dev/null +++ b/data.go @@ -0,0 +1,93 @@ +package fakes + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" +) + +// loadData loads every given directory into one namespace tree and returns its +// root children. A directory becomes a group; each *.json file in it compiles to +// a node keyed by its base name (address.json -> "address"); each subdirectory +// becomes a nested group, so folders turn into dot-path segments. Paths are +// merged left to right: matching groups merge by their children, and any other +// clash (a leaf, or a leaf-vs-group) is won by the last directory loaded. +func loadData(paths []string) (map[string]node, error) { + root := map[string]node{} + for _, p := range paths { + g, err := loadDir(p) + if err != nil { + return nil, err + } + mergeChildren(root, g.children) + } + if len(root) == 0 { + return nil, fmt.Errorf("no .json data found in %v", paths) + } + return root, nil +} + +// loadDir compiles one directory into a group. os.ReadDir yields entries sorted +// by name, so the tree is built deterministically. Empty subdirectories (no JSON +// anywhere under them) are skipped rather than added as empty namespaces. +func loadDir(dir string) (*group, error) { + info, err := os.Stat(dir) + if err != nil { + return nil, err + } + if !info.IsDir() { + return nil, fmt.Errorf("%s is not a directory", dir) + } + entries, err := os.ReadDir(dir) + if err != nil { + return nil, err + } + g := &group{children: map[string]node{}} + for _, e := range entries { + full := filepath.Join(dir, e.Name()) + if e.IsDir() { + child, err := loadDir(full) + if err != nil { + return nil, err + } + if len(child.children) > 0 { + g.children[e.Name()] = child + } + continue + } + if !strings.HasSuffix(e.Name(), ".json") { + continue + } + b, err := os.ReadFile(full) + if err != nil { + return nil, err + } + var raw any + if err := json.Unmarshal(b, &raw); err != nil { + return nil, fmt.Errorf("%s: %w", full, err) + } + n, err := compile(raw) + if err != nil { + return nil, fmt.Errorf("%s: %w", full, err) + } + g.children[strings.TrimSuffix(e.Name(), ".json")] = n + } + return g, nil +} + +// mergeChildren overlays src onto dst. Two groups under the same key merge +// recursively (so locales/categories from several paths combine); every other +// key is replaced, making the last-loaded directory win on a conflict. +func mergeChildren(dst, src map[string]node) { + for k, v := range src { + if dg, ok := dst[k].(*group); ok { + if sg, ok := v.(*group); ok { + mergeChildren(dg.children, sg.children) + continue + } + } + dst[k] = v + } +} diff --git a/locales/en_US/address.json b/data/en_US/address.json similarity index 100% rename from locales/en_US/address.json rename to data/en_US/address.json diff --git a/locales/en_US/color.json b/data/en_US/color.json similarity index 100% rename from locales/en_US/color.json rename to data/en_US/color.json diff --git a/locales/en_US/company.json b/data/en_US/company.json similarity index 100% rename from locales/en_US/company.json rename to data/en_US/company.json diff --git a/locales/en_US/date.json b/data/en_US/date.json similarity index 100% rename from locales/en_US/date.json rename to data/en_US/date.json diff --git a/locales/en_US/email.json b/data/en_US/email.json similarity index 100% rename from locales/en_US/email.json rename to data/en_US/email.json diff --git a/locales/en_US/ip.json b/data/en_US/ip.json similarity index 100% rename from locales/en_US/ip.json rename to data/en_US/ip.json diff --git a/locales/en_US/person.json b/data/en_US/person.json similarity index 100% rename from locales/en_US/person.json rename to data/en_US/person.json diff --git a/locales/en_US/phone.json b/data/en_US/phone.json similarity index 100% rename from locales/en_US/phone.json rename to data/en_US/phone.json diff --git a/locales/en_US/price.json b/data/en_US/price.json similarity index 100% rename from locales/en_US/price.json rename to data/en_US/price.json diff --git a/locales/en_US/sentence.json b/data/en_US/sentence.json similarity index 100% rename from locales/en_US/sentence.json rename to data/en_US/sentence.json diff --git a/locales/en_US/ssn.json b/data/en_US/ssn.json similarity index 100% rename from locales/en_US/ssn.json rename to data/en_US/ssn.json diff --git a/locales/en_US/time.json b/data/en_US/time.json similarity index 100% rename from locales/en_US/time.json rename to data/en_US/time.json diff --git a/locales/en_US/url.json b/data/en_US/url.json similarity index 100% rename from locales/en_US/url.json rename to data/en_US/url.json diff --git a/locales/en_US/username.json b/data/en_US/username.json similarity index 100% rename from locales/en_US/username.json rename to data/en_US/username.json diff --git a/locales/en_US/uuid.json b/data/en_US/uuid.json similarity index 100% rename from locales/en_US/uuid.json rename to data/en_US/uuid.json diff --git a/locales/en_US/version.json b/data/en_US/version.json similarity index 100% rename from locales/en_US/version.json rename to data/en_US/version.json diff --git a/locales/en_US/word.json b/data/en_US/word.json similarity index 100% rename from locales/en_US/word.json rename to data/en_US/word.json diff --git a/locales/sv_SE/address.json b/data/sv_SE/address.json similarity index 100% rename from locales/sv_SE/address.json rename to data/sv_SE/address.json diff --git a/locales/sv_SE/color.json b/data/sv_SE/color.json similarity index 100% rename from locales/sv_SE/color.json rename to data/sv_SE/color.json diff --git a/locales/sv_SE/company.json b/data/sv_SE/company.json similarity index 100% rename from locales/sv_SE/company.json rename to data/sv_SE/company.json diff --git a/locales/sv_SE/date.json b/data/sv_SE/date.json similarity index 100% rename from locales/sv_SE/date.json rename to data/sv_SE/date.json diff --git a/locales/sv_SE/email.json b/data/sv_SE/email.json similarity index 100% rename from locales/sv_SE/email.json rename to data/sv_SE/email.json diff --git a/locales/sv_SE/ip.json b/data/sv_SE/ip.json similarity index 100% rename from locales/sv_SE/ip.json rename to data/sv_SE/ip.json diff --git a/locales/sv_SE/person.json b/data/sv_SE/person.json similarity index 100% rename from locales/sv_SE/person.json rename to data/sv_SE/person.json diff --git a/locales/sv_SE/phone.json b/data/sv_SE/phone.json similarity index 100% rename from locales/sv_SE/phone.json rename to data/sv_SE/phone.json diff --git a/locales/sv_SE/price.json b/data/sv_SE/price.json similarity index 100% rename from locales/sv_SE/price.json rename to data/sv_SE/price.json diff --git a/locales/sv_SE/sentence.json b/data/sv_SE/sentence.json similarity index 100% rename from locales/sv_SE/sentence.json rename to data/sv_SE/sentence.json diff --git a/locales/sv_SE/ssn.json b/data/sv_SE/ssn.json similarity index 100% rename from locales/sv_SE/ssn.json rename to data/sv_SE/ssn.json diff --git a/locales/sv_SE/time.json b/data/sv_SE/time.json similarity index 100% rename from locales/sv_SE/time.json rename to data/sv_SE/time.json diff --git a/locales/sv_SE/url.json b/data/sv_SE/url.json similarity index 100% rename from locales/sv_SE/url.json rename to data/sv_SE/url.json diff --git a/locales/sv_SE/username.json b/data/sv_SE/username.json similarity index 100% rename from locales/sv_SE/username.json rename to data/sv_SE/username.json diff --git a/locales/sv_SE/uuid.json b/data/sv_SE/uuid.json similarity index 100% rename from locales/sv_SE/uuid.json rename to data/sv_SE/uuid.json diff --git a/locales/sv_SE/version.json b/data/sv_SE/version.json similarity index 100% rename from locales/sv_SE/version.json rename to data/sv_SE/version.json diff --git a/locales/sv_SE/word.json b/data/sv_SE/word.json similarity index 100% rename from locales/sv_SE/word.json rename to data/sv_SE/word.json diff --git a/data_test.go b/data_test.go index 0ef735d..d392ad3 100644 --- a/data_test.go +++ b/data_test.go @@ -52,8 +52,8 @@ func TestShippedDataCategories(t *testing.T) { regexp.MustCompile(`^\d{1,3}( \d{3})?(,\d{2})? kr$`)}, } - en := newFakes(t, "locales/en_US", WithSeed(1)) - sv := newFakes(t, "locales/sv_SE", WithSeed(1)) + en := newFakes(t, "data/en_US", WithSeed(1)) + sv := newFakes(t, "data/sv_SE", WithSeed(1)) for _, c := range cases { for i := 0; i < 200; i++ { if v := fake(t, en, c.path); !c.en.MatchString(v) { @@ -70,7 +70,7 @@ func TestShippedDataCategories(t *testing.T) { // is a real calendar date (so month-length variants never emit e.g. Apr 31 or // Feb 30) and the trailing digit is a valid Luhn checksum over the other nine. func TestSwedishPersonnummer(t *testing.T) { - sv := newFakes(t, "locales/sv_SE", WithSeed(1)) + sv := newFakes(t, "data/sv_SE", WithSeed(1)) sawLongMonthEnd := false for i := 0; i < 2000; i++ { v := fake(t, sv, "ssn") @@ -93,6 +93,69 @@ func TestSwedishPersonnummer(t *testing.T) { } } +func TestShippedSwedishPhone(t *testing.T) { + f := newFakes(t, "data/sv_SE", WithSeed(11)) + re := regexp.MustCompile(`^0\d{1,2}-\d{3} \d{2} \d{2}$`) + for i := 0; i < 50; i++ { + if n := fake(t, f, "phone"); !re.MatchString(n) { + t.Fatalf("phone %q does not match %s", n, re) + } + } +} + +func TestShippedSwedishAddress(t *testing.T) { + f := newFakes(t, "data/sv_SE", WithSeed(3)) + digit := regexp.MustCompile(`\d`) + for i := 0; i < 30; i++ { + a := fake(t, f, "address") + if !regexp.MustCompile(`\n`).MatchString(a) || !digit.MatchString(a) { + t.Fatalf("address %q is not a multi-line address with a number", a) + } + } + + locality := regexp.MustCompile(`^\p{L}+( \p{L}+)*$`) + for i := 0; i < 30; i++ { + if c := fake(t, f, "address.locality"); !locality.MatchString(c) { + t.Fatalf("locality %q is not a Swedish place name", c) + } + } +} + +func TestShippedPersonHasParts(t *testing.T) { + for _, dir := range []string{"data/sv_SE", "data/en_US"} { + f := newFakes(t, dir, WithSeed(7)) + for i := 0; i < 30; i++ { + if name := fake(t, f, "person"); len(name) < 3 || !regexp.MustCompile(`\S \S`).MatchString(name) { + t.Fatalf("%s person %q lacks first and last name", dir, name) + } + } + } +} + +func TestShippedUSPhone(t *testing.T) { + f := newFakes(t, "data/en_US", WithSeed(11)) + re := regexp.MustCompile(`^(\(\d{3}\) \d{3}-\d{4}|\d{3}-\d{3}-\d{4})$`) + for i := 0; i < 50; i++ { + if n := fake(t, f, "phone"); !re.MatchString(n) { + t.Fatalf("phone %q does not match %s", n, re) + } + } +} + +// TestShippedNamespacedTree loads the whole data/ tree (not a single locale) and +// reaches each locale through its folder segment: data/sv_SE/person -> sv_SE.person. +func TestShippedNamespacedTree(t *testing.T) { + f := newFakes(t, "data", WithSeed(1)) + for _, path := range []string{"sv_SE.person", "en_US.person", "sv_SE.address.locality"} { + if got := fake(t, f, path); got == "" { + t.Fatalf("Fake(%q) returned empty", path) + } + } + if _, err := f.Fake("person"); err == nil { + t.Fatal("Fake(person) = nil error; categories should be namespaced under the locale folder") + } +} + func digitsOnly(s string) string { var b strings.Builder for _, r := range s { diff --git a/edge_test.go b/edge_test.go index 879016e..c99e518 100644 --- a/edge_test.go +++ b/edge_test.go @@ -54,11 +54,11 @@ func TestNewErrors(t *testing.T) { if err := os.WriteFile(file, []byte("{}"), 0o644); err != nil { t.Fatal(err) } - if _, err := New(file); err == nil { + if _, err := New([]string{file}); err == nil { t.Error("New(file) = nil error, want not-a-directory error") } // Invalid JSON in a category file fails. - if _, err := New(writeLocale(t, "xx_XX", map[string]string{"broken": `{ not json`})); err == nil { + if _, err := New([]string{writeData(t, map[string]string{"broken": `{ not json`})}); err == nil { t.Error("New(invalid JSON) = nil error") } } @@ -92,7 +92,7 @@ func TestDescendIntoLiteralErrors(t *testing.T) { // --- category root shapes --- func TestCategoryRootShapes(t *testing.T) { - dir := writeLocale(t, "xx_XX", map[string]string{ + dir := writeData(t, map[string]string{ "obj": `{"format":"00"}`, // object root "lit": `"hello"`, // bare-string root }) @@ -117,7 +117,7 @@ func TestLongStringList(t *testing.T) { if err != nil { t.Fatal(err) } - f := newFakes(t, writeLocale(t, "xx_XX", map[string]string{"name": string(list)}), WithSeed(1)) + f := newFakes(t, writeData(t, map[string]string{"name": string(list)}), WithSeed(1)) valid := map[string]bool{} for _, v := range names { @@ -145,7 +145,7 @@ var swedishName = regexp.MustCompile(`^\p{L}+([ -]\p{L}+)*$`) func TestShippedStreetComposition(t *testing.T) { // street is a choice of composed {first}{last} templates and literal names. - f := newFakes(t, "locales/sv_SE", WithSeed(5)) + f := newFakes(t, "data/sv_SE", WithSeed(5)) for i := 0; i < 300; i++ { if s := fake(t, f, "address.street"); !swedishName.MatchString(s) { t.Fatalf("street %q is not a Swedish street name", s) @@ -155,7 +155,7 @@ func TestShippedStreetComposition(t *testing.T) { func TestShippedLastNameComposition(t *testing.T) { // last is a choice of patronymic {first}sson templates and literal surnames. - f := newFakes(t, "locales/sv_SE", WithSeed(6)) + f := newFakes(t, "data/sv_SE", WithSeed(6)) for i := 0; i < 300; i++ { if s := fake(t, f, "person.last"); !swedishName.MatchString(s) { t.Fatalf("last name %q is not a Swedish surname", s) @@ -165,7 +165,7 @@ func TestShippedLastNameComposition(t *testing.T) { func TestShippedStreetNumberFormats(t *testing.T) { // Reachable via a hyphenated path; covers all five weighted number variants. - f := newFakes(t, "locales/sv_SE", WithSeed(8)) + f := newFakes(t, "data/sv_SE", WithSeed(8)) re := regexp.MustCompile(`^[1-9]\d{0,2}[A-Z]?$`) for i := 0; i < 300; i++ { if n := fake(t, f, "address.street-number"); !re.MatchString(n) { diff --git a/fakes.go b/fakes.go index 3cf9488..a7d878f 100644 --- a/fakes.go +++ b/fakes.go @@ -1,15 +1,17 @@ -// Package fakes generates fake data from recursive, locale-specific JSON -// templates. +// Package fakes generates fake data from recursive JSON templates. // -// Locale data lives in JSON on disk, not in Go. Point [New] at a locale -// directory, then generate values by category path: +// Data lives in JSON on disk, not in Go. Point [New] at one or more data +// directories; folders and files become a dot-path namespace, then generate +// values by path: // -// f, _ := fakes.New("./locales/sv_SE", fakes.WithSeed(42)) +// f, _ := fakes.New([]string{"./data/sv_SE"}, fakes.WithSeed(42)) // f.Fake("address") // "Storgatan 12\n234 56 Göteborg" // f.Fake("address.locality") // "Göteborg" // -// The directory's name is the locale and must be a full tag ("sv_SE", never -// "sv"). The JSON template format is documented in the README. +// A subdirectory is a namespace segment, so pointing at "./data" instead reaches +// a category as "sv_SE.address". Several directories are merged left to right; +// the last one wins on a name clash, so you can layer custom data over the +// built-ins. The JSON template format is documented in the README. // // A [Fakes] is not safe for concurrent use; create one per goroutine. package fakes @@ -19,14 +21,12 @@ import ( "encoding/binary" "fmt" "math/rand/v2" - "path/filepath" ) -// Fakes generates fake data for a single locale. Create one with [New]. +// Fakes generates fake data from a loaded namespace tree. Create one with [New]. type Fakes struct { rand *rand.Rand - locale string - categories map[string]node // category name -> compiled node tree + categories map[string]node // root namespace: name -> compiled node tree } type config struct { @@ -43,29 +43,23 @@ func WithSeed(seed uint64) Option { return func(c *config) { c.seed, c.seeded = seed, true } } -// New builds a faker from a locale directory (e.g. "./locales/sv_SE"). The -// directory's name is the locale tag and must be a full tag like "sv_SE"; each -// JSON file in it becomes a category named after the file (address.json -> -// "address"). It errors on a non-full tag, a missing directory, or invalid JSON. -func New(dir string, opts ...Option) (*Fakes, error) { - name, ok := canonicalLocale(filepath.Base(dir)) - if !ok { - return nil, fmt.Errorf("fakes: %q is not a full locale tag like \"sv_SE\"", filepath.Base(dir)) - } - cats, err := loadLocale(dir) +// New builds a faker from one or more data directories (e.g. "./data/sv_SE"). +// Each JSON file becomes a category named after the file (address.json -> +// "address") and each subdirectory a namespace segment; directories are merged +// in order, the last winning a name clash. It errors on a missing directory, +// invalid JSON, or no data found. +func New(paths []string, opts ...Option) (*Fakes, error) { + cats, err := loadData(paths) if err != nil { - return nil, fmt.Errorf("fakes: %s: %w", name, err) + return nil, fmt.Errorf("fakes: %w", err) } var c config for _, opt := range opts { opt(&c) } - return &Fakes{rand: newRand(c.seed, c.seeded), locale: name, categories: cats}, nil + return &Fakes{rand: newRand(c.seed, c.seeded), categories: cats}, nil } -// Locale returns the faker's canonical locale tag. -func (f *Fakes) Locale() string { return f.locale } - func newRand(seed uint64, seeded bool) *rand.Rand { if !seeded { var b [16]byte diff --git a/fakes_test.go b/fakes_test.go index e005c9d..361d661 100644 --- a/fakes_test.go +++ b/fakes_test.go @@ -5,12 +5,18 @@ import ( "testing" ) -// newFakes creates a faker for a locale directory, failing the test on error. +// newFakes creates a faker over a single data directory, failing on error. Most +// tests load one dir; newFakesN loads several (last-loaded wins on conflicts). func newFakes(t *testing.T, dir string, opts ...Option) *Fakes { t.Helper() - f, err := New(dir, opts...) + return newFakesN(t, []string{dir}, opts...) +} + +func newFakesN(t *testing.T, dirs []string, opts ...Option) *Fakes { + t.Helper() + f, err := New(dirs, opts...) if err != nil { - t.Fatalf("New(%q): %v", dir, err) + t.Fatalf("New(%q): %v", dirs, err) } return f } @@ -25,23 +31,15 @@ func fake(t *testing.T, f *Fakes, path string) string { return s } -func TestNewRejectsNonFullLocale(t *testing.T) { - for _, bad := range []string{"locales/sv", "locales/se", "locales/swedish", "locales/sv_S"} { - if _, err := New(bad); err == nil { - t.Errorf("New(%q) = nil error, want full-locale error", bad) - } - } -} - func TestNewMissingDirectory(t *testing.T) { - _, err := New("locales/de_DE") + _, err := New([]string{"data/de_DE"}) if err == nil || !strings.Contains(err.Error(), "de_DE") { - t.Fatalf("New(missing) error = %v, want it to name the locale", err) + t.Fatalf("New(missing) error = %v, want it to name the path", err) } } func TestWithSeedIsDeterministic(t *testing.T) { - a, b := newFakes(t, "locales/sv_SE", WithSeed(42)), newFakes(t, "locales/sv_SE", WithSeed(42)) + a, b := newFakes(t, "data/sv_SE", WithSeed(42)), newFakes(t, "data/sv_SE", WithSeed(42)) for i := 0; i < 50; i++ { if x, y := fake(t, a, "person"), fake(t, b, "person"); x != y { t.Fatalf("same seed diverged at %d: %q != %q", i, x, y) @@ -50,7 +48,7 @@ func TestWithSeedIsDeterministic(t *testing.T) { } func TestDifferentSeedsDiffer(t *testing.T) { - a, b := newFakes(t, "locales/en_US", WithSeed(1)), newFakes(t, "locales/en_US", WithSeed(2)) + a, b := newFakes(t, "data/en_US", WithSeed(1)), newFakes(t, "data/en_US", WithSeed(2)) for i := 0; i < 50; i++ { if fake(t, a, "person") != fake(t, b, "person") { return diff --git a/loading_test.go b/loading_test.go new file mode 100644 index 0000000..780d9f6 --- /dev/null +++ b/loading_test.go @@ -0,0 +1,117 @@ +package fakes + +import ( + "os" + "path/filepath" + "testing" +) + +// writeData builds a temp data directory from a map of relative path (without +// ".json") -> file content, creating parent folders as needed. The directory's +// name carries no meaning anymore, so callers pick any layout they like — +// including nested folders, which become dot-path segments. +func writeData(t *testing.T, files map[string]string) string { + t.Helper() + dir := t.TempDir() + for name, content := range files { + p := filepath.Join(dir, name+".json") + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(p, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + return dir +} + +func TestNewLoadsAnyDirName(t *testing.T) { + // No locale tag required: a directory named anything loads fine. + dir := writeData(t, map[string]string{"greeting": `["hej", "hallå"]`}) + f := newFakes(t, dir, WithSeed(1)) + if got := fake(t, f, "greeting"); got != "hej" && got != "hallå" { + t.Fatalf("greeting = %q, want hej or hallå", got) + } +} + +func TestNewEmptyDirErrors(t *testing.T) { + if _, err := New([]string{writeData(t, nil)}); err == nil { + t.Fatal("New(empty dir) = nil error") + } +} + +// TestFoldersBecomeDotPaths checks the core of the new model: a subfolder is a +// namespace, so data//person.json is reachable as ".person". +func TestFoldersBecomeDotPaths(t *testing.T) { + dir := writeData(t, map[string]string{ + "sv_SE/greeting": `["hej"]`, + "en_US/greeting": `["hi"]`, + }) + f := newFakes(t, dir, WithSeed(1)) + if got := fake(t, f, "sv_SE.greeting"); got != "hej" { + t.Fatalf("sv_SE.greeting = %q, want hej", got) + } + if got := fake(t, f, "en_US.greeting"); got != "hi" { + t.Fatalf("en_US.greeting = %q, want hi", got) + } +} + +// TestNestedFoldersAndJSON crosses both nesting kinds in one dot path: folders +// a/b/c, then deep JSON fields inside the file at the end of that path. +func TestNestedFoldersAndJSON(t *testing.T) { + dir := writeData(t, map[string]string{ + "a/b/c/thing": `{"format":"{x}","x":{"format":"{y}","y":{"format":"{z}","z":["leaf"]}}}`, + }) + f := newFakes(t, dir, WithSeed(1)) + // Folders a.b.c, file thing, then JSON fields x.y.z — one continuous path. + if got := fake(t, f, "a.b.c.thing.x.y.z"); got != "leaf" { + t.Fatalf("a.b.c.thing.x.y.z = %q, want leaf", got) + } + // Rendering the file resolves the same chain top-down. + if got := fake(t, f, "a.b.c.thing"); got != "leaf" { + t.Fatalf("a.b.c.thing = %q, want leaf", got) + } +} + +func TestRenderingAFolderErrors(t *testing.T) { + dir := writeData(t, map[string]string{"sv_SE/greeting": `["hej"]`}) + f := newFakes(t, dir, WithSeed(1)) + if _, err := f.Fake("sv_SE"); err == nil { + t.Fatal("Fake(folder) = nil error, want a not-a-value error") + } +} + +// TestMultiPathLastWins loads two dirs; on a name clash the later dir wins, and +// non-clashing entries from both are reachable (data combines). +func TestMultiPathLastWins(t *testing.T) { + a := writeData(t, map[string]string{"greeting": `["from-a"]`, "only-a": `["a"]`}) + b := writeData(t, map[string]string{"greeting": `["from-b"]`, "only-b": `["b"]`}) + f := newFakesN(t, []string{a, b}, WithSeed(1)) + if got := fake(t, f, "greeting"); got != "from-b" { + t.Fatalf("greeting = %q, want from-b (last loaded wins)", got) + } + if got := fake(t, f, "only-a"); got != "a" { + t.Fatalf("only-a = %q, want a", got) + } + if got := fake(t, f, "only-b"); got != "b" { + t.Fatalf("only-b = %q, want b", got) + } +} + +// TestMultiPathMergesFolders checks that clashing folders merge by their +// children rather than replacing wholesale: each dir adds a file to sv_SE, and +// a per-file clash inside still resolves last-wins. +func TestMultiPathMergesFolders(t *testing.T) { + a := writeData(t, map[string]string{"sv_SE/person": `["from-a"]`, "sv_SE/shared": `["a"]`}) + b := writeData(t, map[string]string{"sv_SE/company": `["from-b"]`, "sv_SE/shared": `["b"]`}) + f := newFakesN(t, []string{a, b}, WithSeed(1)) + if got := fake(t, f, "sv_SE.person"); got != "from-a" { + t.Fatalf("sv_SE.person = %q, want from-a (folder merged, not replaced)", got) + } + if got := fake(t, f, "sv_SE.company"); got != "from-b" { + t.Fatalf("sv_SE.company = %q, want from-b", got) + } + if got := fake(t, f, "sv_SE.shared"); got != "b" { + t.Fatalf("sv_SE.shared = %q, want b (last loaded wins)", got) + } +} diff --git a/locale.go b/locale.go deleted file mode 100644 index aff22e5..0000000 --- a/locale.go +++ /dev/null @@ -1,66 +0,0 @@ -package fakes - -import ( - "encoding/json" - "fmt" - "os" - "path/filepath" - "regexp" - "sort" - "strings" -) - -// loadLocale parses and compiles every JSON file in dir into a category node -// tree, keyed by the file's base name without extension (address.json -> -// "address"). Compiling here means structural errors surface at New, not at -// Fake time. -func loadLocale(dir string) (map[string]node, error) { - info, err := os.Stat(dir) - if err != nil { - return nil, err - } - if !info.IsDir() { - return nil, fmt.Errorf("%s is not a directory", dir) - } - files, err := filepath.Glob(filepath.Join(dir, "*.json")) - if err != nil { - return nil, err - } - sort.Strings(files) - if len(files) == 0 { - return nil, fmt.Errorf("no .json files in %s", dir) - } - categories := make(map[string]node, len(files)) - for _, file := range files { - b, err := os.ReadFile(file) - if err != nil { - return nil, err - } - var raw any - if err := json.Unmarshal(b, &raw); err != nil { - return nil, fmt.Errorf("%s: %w", filepath.Base(file), err) - } - n, err := compile(raw) - if err != nil { - return nil, fmt.Errorf("%s: %w", filepath.Base(file), err) - } - categories[strings.TrimSuffix(filepath.Base(file), ".json")] = n - } - return categories, nil -} - -var localeTag = regexp.MustCompile(`^[a-z]{2,3}_[A-Z]{2}$`) - -// canonicalLocale normalises a tag to "lang_REGION" form, accepting '-' or '_' -// separators and any casing. It reports whether the result is a full tag. -func canonicalLocale(s string) (string, bool) { - parts := strings.Split(strings.ReplaceAll(strings.TrimSpace(s), "-", "_"), "_") - if len(parts) != 2 { - return "", false - } - tag := strings.ToLower(parts[0]) + "_" + strings.ToUpper(parts[1]) - if !localeTag.MatchString(tag) { - return "", false - } - return tag, true -} diff --git a/locale_test.go b/locale_test.go deleted file mode 100644 index 9c93b9c..0000000 --- a/locale_test.go +++ /dev/null @@ -1,95 +0,0 @@ -package fakes - -import ( - "os" - "path/filepath" - "regexp" - "testing" -) - -// writeLocale creates a locale directory named tag under a temp root, with one -// JSON file per category, and returns its path. -func writeLocale(t *testing.T, tag string, categories map[string]string) string { - t.Helper() - dir := filepath.Join(t.TempDir(), tag) - if err := os.MkdirAll(dir, 0o755); err != nil { - t.Fatal(err) - } - for name, content := range categories { - if err := os.WriteFile(filepath.Join(dir, name+".json"), []byte(content), 0o644); err != nil { - t.Fatal(err) - } - } - return dir -} - -func TestNewLoadsCustomLocale(t *testing.T) { - dir := writeLocale(t, "is_IS", map[string]string{"greeting": `["hej", "hallå"]`}) - f := newFakes(t, dir, WithSeed(1)) - if got := fake(t, f, "greeting"); got != "hej" && got != "hallå" { - t.Fatalf("greeting = %q, want hej or hallå", got) - } -} - -func TestNewCanonicalizesDirName(t *testing.T) { - // A directory named with a lowercase region resolves to the canonical tag. - dir := writeLocale(t, "sv_se", map[string]string{"x": `["a"]`}) - if got := newFakes(t, dir).Locale(); got != "sv_SE" { - t.Fatalf("Locale() = %q, want sv_SE", got) - } -} - -func TestNewEmptyLocaleErrors(t *testing.T) { - if _, err := New(writeLocale(t, "xx_XX", nil)); err == nil { - t.Fatal("New(empty locale) = nil error") - } -} - -func TestShippedSwedishPhone(t *testing.T) { - f := newFakes(t, "locales/sv_SE", WithSeed(11)) - re := regexp.MustCompile(`^0\d{1,2}-\d{3} \d{2} \d{2}$`) - for i := 0; i < 50; i++ { - if n := fake(t, f, "phone"); !re.MatchString(n) { - t.Fatalf("phone %q does not match %s", n, re) - } - } -} - -func TestShippedSwedishAddress(t *testing.T) { - f := newFakes(t, "locales/sv_SE", WithSeed(3)) - digit := regexp.MustCompile(`\d`) - for i := 0; i < 30; i++ { - a := fake(t, f, "address") - if !regexp.MustCompile(`\n`).MatchString(a) || !digit.MatchString(a) { - t.Fatalf("address %q is not a multi-line address with a number", a) - } - } - - locality := regexp.MustCompile(`^\p{L}+( \p{L}+)*$`) - for i := 0; i < 30; i++ { - if c := fake(t, f, "address.locality"); !locality.MatchString(c) { - t.Fatalf("locality %q is not a Swedish place name", c) - } - } -} - -func TestShippedPersonHasParts(t *testing.T) { - for _, locale := range []string{"locales/sv_SE", "locales/en_US"} { - f := newFakes(t, locale, WithSeed(7)) - for i := 0; i < 30; i++ { - if name := fake(t, f, "person"); len(name) < 3 || !regexp.MustCompile(`\S \S`).MatchString(name) { - t.Fatalf("%s person %q lacks first and last name", locale, name) - } - } - } -} - -func TestShippedUSPhone(t *testing.T) { - f := newFakes(t, "locales/en_US", WithSeed(11)) - re := regexp.MustCompile(`^(\(\d{3}\) \d{3}-\d{4}|\d{3}-\d{3}-\d{4})$`) - for i := 0; i < 50; i++ { - if n := fake(t, f, "phone"); !re.MatchString(n) { - t.Fatalf("phone %q does not match %s", n, re) - } - } -} diff --git a/template.go b/template.go index 949edb5..034da1a 100644 --- a/template.go +++ b/template.go @@ -24,6 +24,13 @@ type literal string func (literal) isNode() {} +// group is a namespace of named children, built from a directory of JSON files +// and subdirectories. It has no value of its own: descend into a named child by +// dot path; rendering one is an error (see Fake). +type group struct{ children map[string]node } + +func (*group) isNode() {} + // choice picks one of its items. cum holds cumulative weights for a weighted // pick; when nil the choice is uniform and selection is O(1). type choice struct { @@ -45,20 +52,18 @@ type template struct { func (*template) isNode() {} -// Fake generates a value for a category path. The first segment names a -// category (a JSON file); further dot-separated segments descend into named -// fields, e.g. "address" or "address.street". Choices along the way are -// resolved at random. +// Fake generates a value for a dot path. Each segment descends one level: folder +// names and the category (JSON file) come first, then named fields within it, +// e.g. "sv_SE.address" or "sv_SE.address.street". Choices along the way are +// resolved at random. A path naming a folder (no value of its own) is an error. func (f *Fakes) Fake(path string) (string, error) { - segments := strings.Split(path, ".") - n, ok := f.categories[segments[0]] - if !ok { - return "", fmt.Errorf("fakes: unknown category %q", segments[0]) - } - n, err := descend(f.rand, n, segments[1:]) + n, err := descend(f.rand, &group{children: f.categories}, strings.Split(path, ".")) if err != nil { return "", fmt.Errorf("fakes: %s: %w", path, err) } + if _, ok := n.(*group); ok { + return "", fmt.Errorf("fakes: %s names a folder, not a value", path) + } return render(f.rand, n), nil } @@ -70,6 +75,12 @@ func descend(r rng, n node, segments []string) (node, error) { return n, nil } switch n := n.(type) { + case *group: + child, ok := n.children[segments[0]] + if !ok { + return nil, fmt.Errorf("no entry %q", segments[0]) + } + return descend(r, child, segments[1:]) case *template: child, ok := n.fields[segments[0]] if !ok {