Answer prose round 1 and product-owner round 1: CheckIdentifier's doc counts a bound name, FuncCall and the grammar doc say PathRead and identifiers, the Layout's list.go, name.go and data-import rows read on one pass, the locality row says a postort with no street is dropped, geo.py's docs name what they take, well_cased says what it drops, and item 61 names the next steps to share

This commit is contained in:
2026-10-07 02:21:37 +02:00
committed by Lillem4n
parent 2431edeab0
commit c2147d2282
8 changed files with 24 additions and 22 deletions
+4 -4
View File
@@ -272,7 +272,7 @@ neighbouring one.
|-------|----------|----------|--------| |-------|----------|----------|--------|
| `region` | län, by code or name | state, by USPS abbreviation or name; `code` is the FIPS code | population | | `region` | län, by code or name | state, by USPS abbreviation or name; `code` is the FIPS code | population |
| `municipality` | kommun, by code or name | county, by FIPS code or name | population | | `municipality` | kommun, by code or name | county, by FIPS code or name | population |
| `locality` | postort, by name; [`geo-se.py`](data-import/geo-se.py)'s `localities` picks which ship | incorporated place of 25,000 people or more with a postal code of its own, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population | | `locality` | postort, by name; [`geo-se.py`](data-import/geo-se.py)'s `localities` picks the candidates, and one with no street is dropped | incorporated place of 25,000 people or more with a postal code of its own, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population |
| `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges | | `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges |
| `street` | gatunamn, the ten with most road segments per postort | street name, the ten with most address ranges per place | segments; address ranges | | `street` | gatunamn, the ten with most road segments per postort | street name, the ten with most address ranges per place | segments; address ranges |
@@ -960,12 +960,12 @@ Where the project is heading; the sections before Audience document what ships t
``` ```
doc.go the package doc, and the vocabulary the package is written in doc.go the package doc, and the vocabulary the package is written in
fejkdata.go Generator, New, options, List fejkdata.go Generator, New, options, List
list.go the paths List advertises, every one a node reaches a direct descent at a time list.go the paths List advertises: direct descents only
shipped.go the embedded data set, and loading a shipped category on the first call reaching it shipped.go the embedded data set, and loading a shipped category on the first call reaching it
shippedindex.go the index of shipped categories, generated from data/ shippedindex.go the index of shipped categories, generated from data/
node.go the node model and JSON -> node compilation node.go the node model and JSON -> node compilation
table.go tables: their options, format and cells, the link to the parent beside them, and how a path passes and draws a table table.go tables: their options, format and cells, the link to the parent beside them, and how a path passes and draws a table
name.go names: {x as n} bindings, the scopes they live in, the reads they answer, and what each resolves to at load: its target, its reads and the keys they address name.go names: {x as n} bindings, the scopes they live in, the reads they answer, and each binding's target and the keys its reads address
namefence.go the load fences over names namefence.go the load fences over names
pick.go a render's named picks: the frame of each name scope rendering, the memo a pick keeps, and the reads kept under it pick.go a render's named picks: the frame of each name scope rendering, the memo a pick keeps, and the reads kept under it
env.go renderEnv: what a render reads names and rows through, the stack of frames it renders in, and the frames a read opens and finds env.go renderEnv: what a render reads names and rows through, the stack of frames it renders in, and the frames a read opens and finds
@@ -994,7 +994,7 @@ internal/invariant/ the one phrase every package panics with when an invariant b
internal/proven/ what a proof knows of a value, and the bounds a calc takes from its operands internal/proven/ what a proof knows of a value, and the bounds a calc takes from its operands
internal/rows/ a table's rows: the TSV, the options proved over it, the links between tables, row selection and draws, and the pin set one path or one named pick fixes internal/rows/ a table's rows: the TSV, the options proved over it, the links between tables, row selection and draws, and the pin set one path or one named pick fixes
data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc data/ shipped data (JSON, and a TSV per table), embedded at build: locale folders, geo, misc
data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md) data-import/ the scripts that rebuild each sourced table (see DATA-LICENSES.md), and the modules they share
docs/ the decision log, the register research behind the shipped data, and the survey of other libraries docs/ the decision log, the register research behind the shipped data, and the survey of other libraries
release-tooling/ the release CI publishes from the changelog heading release-tooling/ the release CI publishes from the changelog heading
testdata/ the pinned shipped shape (see Versioning) testdata/ the pinned shipped shape (see Versioning)
+3 -1
View File
@@ -137,6 +137,8 @@ def population_of(name, municipality, tatorter, municipalities, population):
def well_cased(name): def well_cased(name):
"""Whether each word of name is capitalised: GeoNames also spells some postorter, such as STOCKHOLM and örebro, in
capitals or lower case."""
return all(part[:1].isupper() and (len(part) == 1 or not part.isupper()) for part in re.split(r"[ -]", name)) return all(part[:1].isupper() and (len(part) == 1 or not part.isupper()) for part in re.split(r"[ -]", name))
@@ -155,7 +157,7 @@ def localities(codes, tatorter, municipalities, population):
continue continue
lat, lon = point lat, lon = point
out[name] = {"name": name, "municipality": municipality, "population": population_of(name, municipality, tatorter, municipalities, population), "lat": f"{lat:.4f}", "lon": f"{lon:.4f}", "codes": kept} out[name] = {"name": name, "municipality": municipality, "population": population_of(name, municipality, tatorter, municipalities, population), "lat": f"{lat:.4f}", "lon": f"{lon:.4f}", "codes": kept}
print(f"municipality by {dict(how)}; {len(by_locality) - len(out)} postorter dropped", file=sys.stderr) print(f"municipality by {dict(how)}; {len(by_locality) - len(out)} postorter dropped before streets", file=sys.stderr)
return out return out
+5 -4
View File
@@ -10,7 +10,8 @@ CACHE = Path(__file__).resolve().parent / "cache"
def parser(doc, country, **own): def parser(doc, country, **own):
"""The options every country's import takes and the script's own, keyed by name with "_" for "-", in name order.""" """A parser for the options every country's import takes, plus the script's own: each keyword names one option, with
"_" standing for "-"."""
options = {"cache": {"default": str(CACHE)}, "out": {"default": str(DATA / country)}, "streets_per_locality": {"type": int, "default": 10}, **own} options = {"cache": {"default": str(CACHE)}, "out": {"default": str(DATA / country)}, "streets_per_locality": {"type": int, "default": 10}, **own}
p = argparse.ArgumentParser(description=doc.splitlines()[0]) p = argparse.ArgumentParser(description=doc.splitlines()[0])
for name in sorted(options): for name in sorted(options):
@@ -63,8 +64,8 @@ class Nearest:
def top_streets(count, per_locality, column): def top_streets(count, per_locality, column):
"""Per locality, the per_locality street names counted most in count, keyed (locality, name), ties by name; each """count maps (locality, street name) to a count. Per locality, the per_locality names counted most, ties broken by
row holds its count in column.""" name, each row holding its count under column."""
of = collections.defaultdict(list) of = collections.defaultdict(list)
for (locality, name), n in count.items(): for (locality, name), n in count.items():
of[locality].append((n, name)) of[locality].append((n, name))
@@ -72,7 +73,7 @@ def top_streets(count, per_locality, column):
def with_streets(places, named): def with_streets(places, named):
"""The places named holds streets for; each other one is logged as dropped.""" """The places that named holds streets for; logs each other place as dropped."""
for key, place in places.items(): for key, place in places.items():
if key not in named: if key not in named:
label = key if key == place["name"] else f"{key} {place['name']}" label = key if key == place["name"] else f"{key} {place['name']}"
+2 -2
View File
@@ -161,8 +161,8 @@ func disagreement(a *template, da DataType, b *template, db DataType) error {
return message(c) return message(c)
} }
// clash is two items of one column holding different datatypes: fix, the item to fix, beside // clash is two items of one column holding different datatypes: other holds want; held says
// other, which holds want; held says what each holds. // what each holds.
type clash struct { type clash struct {
fix, other *template fix, other *template
want DataType want DataType
+2 -2
View File
@@ -1,5 +1,5 @@
// Package grammar is how the template language is written: format tokens, paths and // Package grammar is how the template language is written: format tokens, paths and
// their selectors, reference sigils, names, calc expressions, and whether an argument is a // their selectors, reference sigils, identifiers, calc expressions, and whether an argument is a
// template or a path. It reads strings only. // template or a path. It reads strings only.
package grammar package grammar
@@ -122,7 +122,7 @@ func ParseFormat(format string) ([]Token, error) {
} }
// FuncCall splits a "{token}" body shaped name(args) into its parts; ok is false // FuncCall splits a "{token}" body shaped name(args) into its parts; ok is false
// for a name-read body. A '(' without a trailing ')' yields ok=false. // for a PathRead body. A '(' without a trailing ')' yields ok=false.
func FuncCall(body string) (name string, args []string, ok bool) { func FuncCall(body string) (name string, args []string, ok bool) {
lp := indexOutside(body, '(') lp := indexOutside(body, '(')
if lp < 0 || !strings.HasSuffix(body, ")") { if lp < 0 || !strings.HasSuffix(body, ")") {
+3 -4
View File
@@ -8,14 +8,13 @@ import (
// reservedInIdentifier is what an identifier may not contain: a dot // reservedInIdentifier is what an identifier may not contain: a dot
// separates the segments of a path, '|' the arms of a token, '(' opens a function // separates the segments of a path, '|' the arms of a token, '(' opens a function
// call, braces delimit the token, '/' starts a reference, and brackets and a quote // call, braces delimit the token, '/' starts a reference, and brackets and a quote
// open a JSON value. An identifier carrying one is rejected where it is authored rather // open a JSON value. An identifier carrying one is rejected where it is authored.
// than where it would be unreachable.
const reservedInIdentifier = ".|({}/[]\"" const reservedInIdentifier = ".|({}/[]\""
var reservedList = strings.Join(strings.Split(reservedInIdentifier, ""), " ") var reservedList = strings.Join(strings.Split(reservedInIdentifier, ""), " ")
// CheckIdentifier rejects an identifier, the name of a folder, category, field or column, that the // An identifier names a folder, category, field, column or bound name. CheckIdentifier rejects one
// dot path, {token} and JSON grammars cannot spell, or a struct tag cannot read. // that the dot path, {token} and JSON grammars cannot spell, or that a struct tag cannot read.
func CheckIdentifier(name string) error { func CheckIdentifier(name string) error {
if name == "" { if name == "" {
return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name) return fmt.Errorf("%q is empty, which is not a path segment, so List never offers it", name)
+4 -4
View File
@@ -65,10 +65,10 @@ const (
stepParent // up to the table's parent, at the row its own row links to stepParent // up to the table's parent, at the row its own row links to
) )
// compiledPath is a path proved to reach a node whichever way the draws go: every variant of a // compiledPath walks a path and proves it reaches a node whichever way the draws go: every
// choice carries the rest of it, and is walked, a selector names a row inside the // variant of a choice carries the rest of it, and is walked, a selector names a row inside the
// rows selected before it, and no level read carries a repeat. It holds the steps // rows selected before it, and no level read carries a repeat. Once run, it holds the steps a
// a draw takes, and every leaf the path may render; level names the head in its errors. // draw takes, and every leaf the path may render; level names the head in its errors.
type compiledPath struct { type compiledPath struct {
pins pinSet pins pinSet
level string level string
+1 -1
View File
@@ -171,7 +171,7 @@ Needs item 77. Ten streets per place crowds 500 Stockholm customers onto ten str
### 61. Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country. ### 61. Add a locale, with its `geo/` tree, for each of the ten most-spoken languages and each Nordic country.
Pick the ten from a published ranking of languages by total speakers, such as Ethnologue's, each in the country where it has the most speakers, and record the ranking in the decision that names them. The Nordic ones are sv_SE (shipped), nb_NO, da_DK, fi_FI and is_IS. Each locale carries goal 2.2's categories, and its `address` reads its country's `geo/` tree by reference. `rows.Pins` holds eight pins inline, sized for a five-deep geo tree; a deeper tree needs more to keep a render off the heap. Pick the ten from a published ranking of languages by total speakers, such as Ethnologue's, each in the country where it has the most speakers, and record the ranking in the decision that names them. The Nordic ones are sv_SE (shipped), nb_NO, da_DK, fi_FI and is_IS. A second country reading GeoNames' postal codes moves `geonames()` from `geo-se.py`, and the nearest-centroid street assignment in its `streets()`, into `geo.py`. Each locale carries goal 2.2's categories, and its `address` reads its country's `geo/` tree by reference. `rows.Pins` holds eight pins inline, sized for a five-deep geo tree; a deeper tree needs more to keep a render off the heap.
### 16. Add the remaining locale categories: company, phone, finance, vehicle, words. ### 16. Add the remaining locale categories: company, phone, finance, vehicle, words.