Name the everyday media type extensions, refuse an emptied table, and cut the false register claim

CHANGELOG.md

@@ -26,10 +26,11 @@ replacement, and each removed path, column or flag.
   capital, a currency and a TLD, with the columns `calling-code`, `capital`,
   `currency`, `flag`, `languages`, `numeric` and `tld` added; `misc.currency` the
   current ISO 4217 currencies with a minor unit, with `decimals` and `numeric`
-  added, and its symbols from CLDR; `misc.language` every ISO 639-1 language, with
+  added, and its symbols from CLDR; `misc.language` every ISO 639-2 entry carrying a
-  a `code3` column carrying its ISO 639-2 code; `misc.httpstatus` every HTTP status
+  639-1 code, with a `code3` column holding its 639-2/T code; `misc.httpstatus` each
-  code in use in the IANA registry; and `misc.mimetype` every IANA media type that
+  code the IANA registry lists with a plain reason phrase; and `misc.mimetype` each
-  has a filename extension. `DATA-LICENSES.md` lists each source.
+  type mime-db records as IANA-registered and gives a filename extension.
+  `DATA-LICENSES.md` lists each source.
 - `geo.SE` and `geo.US`: five linked tables per country, `region`, `municipality`,
   `locality`, `postal-code` and `street`, weighted by population and address counts
   and built from SCB, GeoNames, Trafikverket NVDB and the US Census Bureau, and an

DATA-LICENSES.md

@@ -19,6 +19,6 @@ Every shipped dataset, its source, its licence and the attribution it asks for.
 | `misc/currency.tsv` | [datasets/currency-codes](https://github.com/datasets/currency-codes); symbols from [Unicode CLDR](https://github.com/unicode-org/cldr) `en.xml` and `root.xml` | PDDL 1.0; [Unicode License v3](https://www.unicode.org/license.txt) | CLDR: "Copyright © 1991-2025 Unicode, Inc. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other countries." | `data-import/currency.py` |
 | `misc/httpstatus.tsv` | [IANA HTTP Status Code Registry](https://www.iana.org/assignments/http-status-codes/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpstatus.py` |
 | `misc/language.tsv` | [datasets/language-codes](https://github.com/datasets/language-codes), the [Library of Congress](https://www.loc.gov/standards/iso639-2/) ISO 639-2 register | [PDDL 1.0](https://opendatacommons.org/licenses/pddl/1-0/) | none required | `data-import/language.py` |
-| `misc/mimetype.tsv` | [mime-db](https://github.com/jshttp/mime-db), the IANA media type registry with filename extensions | [MIT](https://github.com/jshttp/mime-db/blob/master/LICENSE) | none required | `data-import/mimetype.py` |
+| `misc/mimetype.tsv` | [mime-db](https://github.com/jshttp/mime-db), the IANA media type registry with filename extensions | [MIT](https://github.com/jshttp/mime-db/blob/master/LICENSE) | "Copyright (c) 2014 Jonathan Ong, Copyright (c) 2015-2022 Douglas Christopher Wilson" | `data-import/mimetype.py` |
 
 Every other category is hand-written JSON under [`data/`](data), MIT like the code.

README.md

@@ -163,15 +163,12 @@ Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`
 `personnummer` and `samordningsnummer`, `en_US` adds `ssn` and `itin`. `misc`
 carries `car`, `coordinate`, `country` (ISO 3166), `creditcard` (Luhn-valid),
 `currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpstatus`, `language`
-(ISO 639-1 and -2), `mac`, `mimetype`, `objectid`, `timezone` (IANA), `useragent` and
+(ISO 639-1, with its 639-2/T code), `mac`, `mimetype`, `objectid`, `timezone` (IANA),
-`uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
+`useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
 `misc.country.alpha2`, `misc.httpstatus.code` — which `--list` shows. `country`,
 `currency`, `httpstatus`, `language` and `mimetype` are [tables](#table), so
-`misc.country[SE].capital` and `misc.currency[Euro].symbol` select a row. Each
+`misc.country[SE].capital` and `misc.currency[Euro].symbol` select a row;
-holds its whole register: every ISO 3166 country, ISO 4217 currency and ISO 639-1
+[`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence.
-language, every HTTP status code in use, and every IANA media type that has a
-filename extension. [`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's
-source and licence.
 
 `sex`, `first-name` and `last-name` are tables weighted by bearers, from SCB, the
 SSA and the Census Bureau. `first-name` links to `sex`, so `sv_SE.sex[f].first-name`

data-import/mimetype.py

@@ -3,7 +3,7 @@
 
     data-import/mimetype.py [--source URL_OR_FILE] [--cache DIR] [--out FILE]
 
-Only IANA-registered types with a filename extension ship; the first extension is the one listed.
+Only types mime-db records as IANA-registered, with a filename extension, ship.
 """
 import argparse
 import json
@@ -16,12 +16,14 @@ SOURCE = "https://raw.githubusercontent.com/jshttp/mime-db/master/db.json"
 OUT = Path(__file__).resolve().parent.parent / "data" / "misc" / "mimetype.tsv"
 CACHE = Path(__file__).resolve().parent / "cache"
 COLUMNS = ["ext", "type"]
+# mime-db lists extensions in no particular order, so where the everyday one is not first, name it.
+FIXUPS = {"application/mp4": "mp4s", "audio/mpeg": "mp3", "audio/ogg": "ogg", "video/quicktime": "mov"}
 
 
 def rows(db):
     for mimetype, entry in db.items():
         if entry.get("source") == "iana" and entry.get("extensions"):
-            yield {"ext": "." + entry["extensions"][0], "type": mimetype}
+            yield {"ext": "." + FIXUPS.get(mimetype, entry["extensions"][0]), "type": mimetype}
 
 
 def main():

data-import/tsv.py

@@ -5,12 +5,14 @@ from pathlib import Path
 
 
 def write(path, columns, rows):
-    """Write the rows as a TSV; every cell must be non-empty and free of tabs, newlines and braces."""
+    """Write the rows as a TSV; the table needs a row, and every cell must be non-empty and free of tabs, newlines and braces."""
     lines = ["\t".join(columns)]
     for row in rows:
         cells = [str(row[c]) for c in columns]
         if not all(cells) or any(re.search(r"[\t\n{}]", c) for c in cells):
             raise ValueError(f"{path}: a cell is empty or holds a tab, newline or brace: {row}")
         lines.append("\t".join(cells))
+    if len(lines) < 2:
+        raise ValueError(f"{path}: has no rows below its header; a table is at least two rows")
     Path(path).write_text("\n".join(lines) + "\n", encoding="utf-8")
     print(f"{path}: {len(lines) - 1} rows", file=sys.stderr)

data/misc/mimetype.tsv

@@ -62,7 +62,7 @@ ext	type
 .musd	application/mmt-usd+xml
 .mods	application/mods+xml
 .m21	application/mp21
-.mp4	application/mp4
+.mp4s	application/mp4
 .doc	application/msword
 .mxf	application/mxf
 .nq	application/n-quads
@@ -511,8 +511,8 @@ ext	type
 .mka	audio/matroska
 .mxmf	audio/mobile-xmf
 .m4a	audio/mp4
-.mpga	audio/mpeg
+.mp3	audio/mpeg
-.oga	audio/ogg
+.ogg	audio/ogg
 .uva	audio/vnd.dece.audio
 .eol	audio/vnd.digital-winds
 .dra	audio/vnd.dra
@@ -688,7 +688,7 @@ ext	type
 .mp4	video/mp4
 .mpeg	video/mpeg
 .ogv	video/ogg
-.qt	video/quicktime
+.mov	video/quicktime
 .uvh	video/vnd.dece.hd
 .uvm	video/vnd.dece.mobile
 .uvp	video/vnd.dece.pd

todo.md

@@ -22,8 +22,7 @@ value is composed; TSV says which values exist.
 - In a data file, refuse a choice whose items are templates with one identical
   field set, naming the TSV to write; an inline template is exempt, having no
   file. A choice of strings stays a choice.
-- Convert `misc.timezone`, `car` and `useragent` to tables; `country`, `currency`,
+- Convert `misc.timezone`, `car` and `useragent` to tables.
-  `language`, `httpstatus` and `mimetype` are done.
 - Let a path select a row by key or by name, `geo.SE.municipality[0180]` and
   `geo.SE.municipality[Stockholm]`, and descend to a linked table by name,
   `geo.SE.region[Skåne län].street`; a key or a linked table name may not equal
@@ -171,10 +170,8 @@ address, phone, national id, company and date names each.
    conflicts with sits inside another category. Give the national ids one
    record shape in step 6, and report a struct column's draw conflict with the path
    spelling a tag takes rather than the reference spelling.
-4. `misc` conversions and the new `misc` tables. `language`, `httpstatus` and
+4. `misc` conversions and the new `misc` tables. Convert `timezone`, `car` and
-   `mimetype` read their registers now, and `misc.country` keeps a currency code
+   `useragent`, and add the new `misc` tables.
-   rather than a `parent` link to `misc.currency` (README Decisions). Convert
-   `timezone`, `car` and `useragent`, and add the new `misc` tables.
 5. The remaining locale categories: company, phone, finance, vehicle, words.
 6. Records with building-block columns across the shipped set; shape re-pin.