Answer stability round 1: the docs say the US tenth is drawn from every place, a capital matches by a name only one place in its state carries, and the test comment says what it checks

This commit is contained in:
2026-10-11 19:02:37 +02:00
committed by Lillem4n
parent 64e99927f5
commit d3d9547876
6 changed files with 23 additions and 20 deletions
+1 -1
View File
@@ -90,7 +90,7 @@ In [docs/decisions.md](docs/decisions.md):
- A path is walked once without drawing before it is walked for real - A path is walked once without drawing before it is walked for real
- A country's postal codes and streets are siblings under its locality - A country's postal codes and streets are siblings under its locality
- A locale package imports its country's geo package, and its `Modules()` returns both - A locale package imports its country's geo package, and its `Modules()` returns both
- The default embed holds every Swedish postort the import can place with all its streets, and of the US places that hold most of a postal code's land every state capital, every place of 100,000 and a population-weighted tenth of the rest, each with a quarter of its streets, at least ten - The default embed holds every Swedish postort the import can place with all its streets, and of the US places that hold most of a postal code's land a population-weighted tenth, every state capital and every place of 100,000, each with a quarter of its streets, at least ten
- A locale's `address` binds its country's record once and prints it whole - A locale's `address` binds its country's record once and prints it whole
- A postort's kommun comes from its name, its tätort or its codes, never from distance - A postort's kommun comes from its name, its tätort or its codes, never from distance
- A highway designation is not a street, and a US postal code belongs to the place holding most of its land inside places - A highway designation is not a street, and a US postal code belongs to the place holding most of its land inside places
+4 -4
View File
@@ -95,10 +95,10 @@ replacement, and each removed path, column or flag.
`en_US.address.street` no longer carries `name` and `suffix`, and a locale's `address` `en_US.address.street` no longer carries `name` and `suffix`, and a locale's `address`
reads its country's `geo` module. Every Swedish postort the import can place and give reads its country's `geo` module. Every Swedish postort the import can place and give
a street ships with all its streets. Of the US incorporated places that hold most of a street ships with all its streets. Of the US incorporated places that hold most of
some postal code's land, every state capital but Honolulu, every place of 100,000 or some postal code's land, a tenth drawn by population down to villages of a hundred
more, and a tenth of the rest drawn by population down to villages of a hundred people people ships, and every state capital but Honolulu and every place of 100,000 or more
ship, so a smaller city may be missing; each carries a quarter of its streets, at least beside, so a smaller city may be missing; each carries a quarter of its streets, at
ten. least ten.
- `{date(from,to,'layout')}` and `{time('layout')}`: a second between two days, or - `{date(from,to,'layout')}` and `{time('layout')}`: a second between two days, or
within one, in a single-quoted Go layout, drawn in UTC; `from` may equal `to`. within one, in a single-quoted Go layout, drawn in UTC; `from` may equal `to`.
`sv_SE.date`, `en_US.date`, `sv_SE.time` and `en_US.time` render through them, so `sv_SE.date`, `en_US.date`, `sv_SE.time` and `en_US.time` render through them, so
+4 -4
View File
@@ -313,14 +313,14 @@ neighbouring one.
| `municipality` | kommun, by code or name | county, by FIPS code or name | population | | `municipality` | kommun, by code or name | county, by FIPS code or name | population |
| `locality` | postort, by name; [`geo-se.py`](data-import/geo-se.py)'s `localities` picks the candidates, and one with no street is dropped | incorporated place, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population | | `locality` | postort, by name; [`geo-se.py`](data-import/geo-se.py)'s `localities` picks the candidates, and one with no street is dropped | incorporated place, by GEOID or name; Hawaii has none | tätort population, the kommun's where the postort names it, else 200; place population |
| `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges | | `postal-code` | postnummer with street delivery, by code | ZCTA, by code | one; address ranges |
| `street` | gatunamn, every one | street name, per place the quarter with most address ranges, at least ten | segments; address ranges | | `street` | gatunamn, every one; [`geo-se.py`](data-import/geo-se.py)'s `street_name` drops a name that is not a street's | street name, per place the quarter with most address ranges, at least ten | segments; address ranges |
`geo.SE.region[Skåne län].municipality` draws a kommun in Skåne, `geo.SE.region[Skåne län].municipality` draws a kommun in Skåne,
`geo.SE.locality[Lund].street` a street in Lund, and `geo.SE.locality[Lund].street` a street in Lund, and
`geo.US.region[IL].locality[Springfield]` settles which Springfield. The US places are `geo.US.region[IL].locality[Springfield]` settles which Springfield. The US places are
the incorporated places that hold most of some postal code's land: every state capital, the incorporated places that hold most of some postal code's land: a tenth of them drawn
every place of 100,000 or more, and a tenth of the rest drawn with odds by population, with odds by population, and every state capital but Honolulu and every place of 100,000
so a smaller city may be missing: look one up in or more beside, so a smaller city may be missing: look one up in
[`locality.tsv`](data/geo/US/locality.tsv) before selecting it. A region row [`locality.tsv`](data/geo/US/locality.tsv) before selecting it. A region row
carries its `timezone`, the state's predominant zone, and a locality its `lat` and carries its `timezone`, the state's predominant zone, and a locality its `lat` and
`lon`. What ports across countries is the five table names, the `name` column, `lon`. What ports across countries is the five table names, the `name` column,
+9 -6
View File
@@ -28,18 +28,18 @@ ZCTA_PLACE = "https://www2.census.gov/geo/docs/maps-data/data/rel2020/zcta520/ta
TIGER = "https://www2.census.gov/geo/tiger/TIGER2025/{0}/tl_2025_{1}_{2}.zip" TIGER = "https://www2.census.gov/geo/tiger/TIGER2025/{0}/tl_2025_{1}_{2}.zip"
ESTIMATE = "POPESTIMATE2025" ESTIMATE = "POPESTIMATE2025"
CDP = "57" CDP = "57"
# Each state's capital from Wikidata (CC0), with the capital's GNIS ids, which the Gazetteer carries as ANSICODE. # Each state's capital from Wikidata, with the capital's GNIS ids, which the Gazetteer carries as ANSICODE.
CAPITALS = "https://query.wikidata.org/sparql?" + urllib.parse.urlencode({"query": """ CAPITALS = "https://query.wikidata.org/sparql?" + urllib.parse.urlencode({"query": """
SELECT ?iso ?capitalLabel ?gnis WHERE { SELECT ?iso ?capitalLabel ?gnis WHERE {
?state wdt:P31 wd:Q35657; wdt:P300 ?iso; wdt:P36 ?capital. ?state wdt:P31 wd:Q35657; wdt:P300 ?iso; wdt:P36 ?capital.
OPTIONAL { ?capital wdt:P590 ?gnis. } OPTIONAL { ?capital wdt:P590 ?gnis. }
SERVICE wikibase:label { bd:serviceParam wikibase:language "en". } SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}"""}) }"""})
ALWAYS = 100000 SHIPPED_POPULATION = 100000
HIGHWAY = re.compile(r"\b(I- |Hwy |Highway |Loop |Rte |Route |Rd )\d") HIGHWAY = re.compile(r"\b(I- |Hwy |Highway |Loop |Rte |Route |Rd )\d")
SUFFIX = re.compile(r" (city and borough|city|town|village|borough|municipality|comunidad|zona urbana|metropolitan government|metro government|consolidated government|unified government|urban county|corporation|plantation)( \(balance\))?$") SUFFIX = re.compile(r" (city and borough|city|town|village|borough|municipality|comunidad|zona urbana|metropolitan government|metro government|consolidated government|unified government|urban county|corporation|plantation)( \(balance\))?$")
# Places whose Census name is not the postal city an address carries. # Places whose Census name is not the postal city an address carries.
NAMES = {"1303440": "Athens", "1304204": "Augusta", "1349008": "Macon", "1608830": "Boise", "2148006": "Louisville", "3011397": "Butte", "4732742": "Hartsville", "4752006": "Nashville"} NAMES = {"0665042": "Ventura", "1303440": "Athens", "1304204": "Augusta", "1349008": "Macon", "1608830": "Boise", "2146027": "Lexington", "2148006": "Louisville", "3011397": "Butte", "4732742": "Hartsville", "4744382": "Lynchburg", "4752006": "Nashville"}
TIMEZONES = { TIMEZONES = {
"AK": "America/Anchorage", "AL": "America/Chicago", "AR": "America/Chicago", "AZ": "America/Phoenix", "AK": "America/Anchorage", "AL": "America/Chicago", "AR": "America/Chicago", "AZ": "America/Phoenix",
"CA": "America/Los_Angeles", "CO": "America/Denver", "CT": "America/New_York", "DC": "America/New_York", "CA": "America/Los_Angeles", "CO": "America/Denver", "CT": "America/New_York", "DC": "America/New_York",
@@ -137,9 +137,12 @@ def localities(cache, counties):
def capitals(cache, places, counties): def capitals(cache, places, counties):
"""The codes of the places that are a state's capital, matched by any of its GNIS ids, else by name inside the state.""" """The codes of the places that are a state's capital, matched by any of its GNIS ids, else by a name no other place in the state carries."""
by_gnis = {p["gnis"]: code for code, p in places.items()} by_gnis = {p["gnis"]: code for code, p in places.items()}
by_name = {(counties[p["municipality"]]["region"], p["name"]): code for code, p in places.items()} named = collections.defaultdict(list)
for code, p in places.items():
named[(counties[p["municipality"]]["region"], p["name"])].append(code)
by_name = {key: codes[0] for key, codes in named.items() if len(codes) == 1}
gnis_of = collections.defaultdict(set) gnis_of = collections.defaultdict(set)
for r in csv.DictReader(io.StringIO(text(source.fetch(CAPITALS, cache, "capitals.csv", magic=b"iso,", headers={"Accept": "text/csv"})))): for r in csv.DictReader(io.StringIO(text(source.fetch(CAPITALS, cache, "capitals.csv", magic=b"iso,", headers={"Accept": "text/csv"})))):
gnis_of[(r["iso"].removeprefix("US-"), r["capitalLabel"])].add(int(r["gnis"] or 0)) gnis_of[(r["iso"].removeprefix("US-"), r["capitalLabel"])].add(int(r["gnis"] or 0))
@@ -211,7 +214,7 @@ def main():
owners = {p: places[p] for p in set(owner_of_zcta.values())} owners = {p: places[p] for p in set(owner_of_zcta.values())}
for code in sorted(capital_codes - set(owners)): for code in sorted(capital_codes - set(owners)):
print(f"{code} {places[code]['name']}: capital owns no ZCTA, dropped", file=sys.stderr) print(f"{code} {places[code]['name']}: capital owns no ZCTA, dropped", file=sys.stderr)
places = drawn(owners, a.place_share) | {code: p for code, p in owners.items() if code in capital_codes or p["population"] >= ALWAYS} places = drawn(owners, a.place_share) | {code: p for code, p in owners.items() if code in capital_codes or p["population"] >= SHIPPED_POPULATION}
locality_of_zcta = {zcta: p for zcta, p in owner_of_zcta.items() if p in places} locality_of_zcta = {zcta: p for zcta, p in owner_of_zcta.items() if p in places}
named, addresses = streets(cache, sorted({l["municipality"] for l in places.values()}), locality_of_zcta, a.street_share, a.min_streets) named, addresses = streets(cache, sorted({l["municipality"] for l in places.values()}), locality_of_zcta, a.street_share, a.min_streets)
zctas = collections.defaultdict(list) zctas = collections.defaultdict(list)
+2 -2
View File
@@ -376,8 +376,8 @@ func TestShippedAddressIsOneNamedPick(t *testing.T) {
} }
} }
// TestShippedPlacesSpreadAddresses pins that Göteborg ships every street, Los Angeles a // TestShippedPlacesSpreadAddresses pins that Göteborg ships 1,500 streets or more, Los
// quarter of its streets, and that some US place holds under 2,500 people. // Angeles 2,500 or more, and that some US place holds under 2,500 people.
func TestShippedPlacesSpreadAddresses(t *testing.T) { func TestShippedPlacesSpreadAddresses(t *testing.T) {
for country, city := range map[string]struct { for country, city := range map[string]struct {
key string key string
+3 -3
View File
@@ -603,7 +603,7 @@ providing `geo.SE`'s paths, and without one `sv_SE.address` fails at first reach
`FS` works against goal 6.2. Every package exports `Modules()`, so one spelling loads any of `FS` works against goal 6.2. Every package exports `Modules()`, so one spelling loads any of
them; a data package imports only the packages it reads. them; a data package imports only the packages it reads.
## The default embed holds every Swedish postort the import can place with all its streets, and of the US places that hold most of a postal code's land every state capital, every place of 100,000 and a population-weighted tenth of the rest, each with a quarter of its streets, at least ten ## The default embed holds every Swedish postort the import can place with all its streets, and of the US places that hold most of a postal code's land a population-weighted tenth, every state capital and every place of 100,000, each with a quarter of its streets, at least ten
2026-10-11, Lilleman auf Larv; revised the same day, after a tenth of the streets gave 2026-10-11, Lilleman auf Larv; revised the same day, after a tenth of the streets gave
Göteborg 163 and Stockholm 37. Goals 2 and 14: ten streets per place put 500 Stockholm Göteborg 163 and Stockholm 37. Goals 2 and 14: ten streets per place put 500 Stockholm
@@ -615,8 +615,8 @@ postal code. The draw alone kept 310 of the 343 places of 100,000 or more and le
or name`; every capital and every place of 100,000 now ships beside the draw, and its or name`; every capital and every place of 100,000 now ships beside the draw, and its
smallest place holds 93 people. The capitals are each state's `capital` (P36) in smallest place holds 93 people. The capitals are each state's `capital` (P36) in
[Wikidata](https://www.wikidata.org/), matched to a Census place by its GNIS id and [Wikidata](https://www.wikidata.org/), matched to a Census place by its GNIS id and
else by name in its state. Honolulu, a census-designated place, is the one capital and else by a name no other place in its state carries. No census-designated place ships,
the one place of 100,000 that cannot ship. Every Swedish street is 174k rows; a quarter so Honolulu is the one capital missing. Every Swedish street is 174k rows; a quarter
of the US streets 475k. On 2026-10-11 the embed grew from 5.2 MB to 16 MB, and a CLI of the US streets 475k. On 2026-10-11 the embed grew from 5.2 MB to 16 MB, and a CLI
run printing the first `en_US.address` took 145 ms, `sv_SE.address` 95 ms and run printing the first `en_US.address` took 145 ms, `sv_SE.address` 95 ms and
`misc.territory` 3 ms, within goal 13. `--place-share`, `--street-share` and `misc.territory` 3 ms, within goal 13. `--place-share`, `--street-share` and