CHANGELOG.md

@@ -53,6 +53,9 @@ replacement, and each removed path, column or flag.
   leading dot, `.se` and `.xn--p1ai`, the register's `type` beside it, and a `unicode`
   column holding the form the register displays, so `misc.tld[.рф]` selects the row
   `.xn--p1ai` renders. A delegation the register marks "Not assigned" does not ship.
+- `misc.loglevel` is the eight syslog severities: the `keyword` a configuration writes,
+  `err` and `info`, keyed by the numerical `code` a PRI encodes and selectable by
+  either, with the `severity` RFC 5424 spells beside it.
 - `geo.SE` and `geo.US`: five linked tables per country, `region`, `municipality`,
   `locality`, `postal-code` and `street`, weighted by population and address counts
   and built from SCB, GeoNames, Trafikverket NVDB and the US Census Bureau, and an

DATA-LICENSES.md

@@ -19,6 +19,7 @@ Every shipped dataset, its source, its licence and the attribution it asks for.
 | `misc/httpmethod.tsv` | [IANA HTTP Method Registry](https://www.iana.org/assignments/http-methods/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpmethod.py` |
 | `misc/httpstatus.tsv` | [IANA HTTP Status Code Registry](https://www.iana.org/assignments/http-status-codes/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/httpstatus.py` |
 | `misc/language.tsv` | [datasets/language-codes](https://github.com/datasets/language-codes), the [Library of Congress](https://www.loc.gov/standards/iso639-2/) ISO 639-2 register | [PDDL 1.0](https://opendatacommons.org/licenses/pddl/1-0/) | none required | `data-import/language.py` |
+| `misc/loglevel.tsv` | [RFC 5424](https://www.rfc-editor.org/rfc/rfc5424.txt) Table 2, the syslog message severities; keywords from the [POSIX](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/syslog.h.html) `<syslog.h>` severity macros | facts | none required | `data-import/loglevel.py` |
 | `misc/mimetype.tsv` | [mime-db](https://github.com/jshttp/mime-db), the IANA media type registry with filename extensions | [MIT](https://github.com/jshttp/mime-db/blob/master/LICENSE) | "Copyright (c) 2014 Jonathan Ong, Copyright (c) 2015-2022 Douglas Christopher Wilson" | `data-import/mimetype.py` |
 | `misc/port.tsv` | [IANA Service Name and Transport Protocol Port Number Registry](https://www.iana.org/assignments/service-names-port-numbers/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/port.py` |
 | `misc/protocol.tsv` | [IANA Protocol Numbers](https://www.iana.org/assignments/protocol-numbers/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/protocol.py` |

README.md

@@ -163,14 +163,14 @@ Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name`
 `personnummer` and `samordningsnummer`, `en_US` adds `ssn` and `itin`. `misc`
 carries `car`, `coordinate`, `creditcard` (Luhn-valid), `currency` (ISO 4217),
 `datetime` (RFC 3339), `emoji`, `httpmethod`, `httpstatus`, `language` (ISO 639-1,
-with its 639-2/T code), `mac`, `mimetype`, `objectid`, `port`, `protocol`,
+with its 639-2/T code), `loglevel` (syslog severity), `mac`, `mimetype`, `objectid`,
-`territory` (ISO 3166-1), `timezone` (IANA), `tld` (IANA root zone), `useragent` and
+`port`, `protocol`, `territory` (ISO 3166-1), `timezone` (IANA), `tld` (IANA root
-`uuid` (v4). Many carry sub-fields — `misc.currency.symbol`, `misc.territory.alpha2`,
+zone), `useragent` and `uuid` (v4). Many carry sub-fields — `misc.currency.symbol`,
-`misc.httpstatus.code` — which `--list` shows. `car`, `currency`, `httpmethod`,
+`misc.territory.alpha2`, `misc.httpstatus.code` — which `--list` shows. `car`,
-`httpstatus`, `language`, `mimetype`, `port`, `protocol`, `territory`, `timezone`,
+`currency`, `httpmethod`, `httpstatus`, `language`, `loglevel`, `mimetype`, `port`,
-`tld` and `useragent` are [tables](#table), so `misc.territory[SE].capital` and
+`protocol`, `territory`, `timezone`, `tld` and `useragent` are [tables](#table), so
-`misc.currency[Euro].symbol` select a row; `car` and `useragent` carry no key or name,
+`misc.territory[SE].capital` and `misc.currency[Euro].symbol` select a row; `car` and
-so they are drawn from rather than selected in.
+`useragent` carry no key or name, so they are drawn from rather than selected in.
 
 `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries.
 `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's
@@ -191,6 +191,11 @@ its `unicode` the form the register displays, `.рф` for `.xn--p1ai` and the ke
 for an ASCII one, which selects the row too, so `misc.tld[.рф]` renders `.xn--p1ai`. A
 draw spans the whole zone, `.arpa` and `.zuerich` alike, so pin `misc.tld[.com]` where
 a fixture needs one a reader recognises.
+`misc.loglevel` is the eight syslog severities, rendering the keyword a configuration
+writes, `err` and `info`, keyed by the numerical code a PRI encodes and selectable by
+either, so `misc.loglevel[3]` and `misc.loglevel[err]` are one row.
+`misc.loglevel.severity` is the name RFC 5424 spells, `Error` beside `err`; it
+selects nothing, the code and the keyword do.
 [`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence.
 
 `misc.timezone` is every zone tzdb gives a shipped territory, from one apiece for most
@@ -757,7 +762,7 @@ a minor only adds, and a major is the only release that changes what exists.
 | Surface | Major | Minor |
 |---------|-------|-------|
 | Shipped data | remove or rename a path; change a category's format; remove a value, or change a weight or a repeat; add a reference from one shipped category into another; change a table's key, name, weight or parent column, or remove a row | a path outside a record's columns, a locale, a value in a list, a row |
-| Records | remove, rename, retype or add a column; let a column be null | a record, as a new category |
+| Records (a table is one) | remove, rename, retype or add a column; let a column be null | a record, as a new category |
 | Data format | a fence: a spelling `New` rejects that it accepted; a template option, since it reserves a field name | a builtin |
 | CLI | remove or rename a flag, or change its default; change what an exit code means; change the framing a `--format` writes (header, quoting, statement shape), the `--list` layout, or what an error names | a flag, a format |
 | Library | change or remove an exported name; raise the lowest supported Go | an exported name, a `With…` option |
@@ -1230,6 +1235,12 @@ the Development section below, and who ships a register the four above then draw
   every value names a row instead. A territory the register records no state for
   stands alone, which is `EH` alone, and naming one for it would be a claim
   fejkdata has no business making.
+- **`misc.loglevel` is a table keyed by the code, rendering POSIX's keyword.** A flat
+  list of names carries neither the code a PRI encodes nor a selector reaching it, and
+  goal 1 draws the two as one fact. The canonical spelling losing to the one its domain
+  writes, above, settles the rest: a configuration writes `info`, so RFC 5424's
+  `Informational` stays the `severity` column, and the code and the keyword hold the
+  two selector slots.
 - **`misc.tld` keys carry the leading dot, where other tables key on a bare code.**
   The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a
   bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a
@@ -1339,6 +1350,7 @@ docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/geo-u
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/httpmethod.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/httpstatus.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/language.py
+docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/loglevel.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/mimetype.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-se.py
 docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/names-us.py

data-import/loglevel.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+"""Rebuild data/misc/loglevel.tsv from RFC 5424 and POSIX <syslog.h>.
+
+    data-import/loglevel.py [--source URL_OR_FILE] [--posix URL_OR_FILE] [--cache DIR] [--out FILE]
+
+Every severity in Table 2 ships, paired to POSIX's keywords by the order each source
+lists them, which holds only because a keyword prefixes the severity RFC 5424 numbers
+alike.
+"""
+import argparse
+import html
+import re
+import sys
+from pathlib import Path
+
+import source
+import tsv
+
+SOURCE = "https://www.rfc-editor.org/rfc/rfc5424.txt"
+POSIX = "https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/syslog.h.html"
+OUT = Path(__file__).resolve().parent.parent / "data" / "misc" / "loglevel.tsv"
+CACHE = Path(__file__).resolve().parent / "cache"
+COLUMNS = ["code", "keyword", "severity"]
+TABLE = re.compile(r"Table 1\.\s+Syslog Message Facilities(.*?)Table 2\.\s+Syslog Message Severities", re.S)
+ROW = re.compile(r"^ +(\d) +([A-Z][a-z]+): ", re.M)
+SECTION = re.compile(r"severity level portion of the.*?macros:(.*?)The following shall be declared as functions", re.S)
+MACRO = re.compile(r"\bLOG_([A-Z]+)\b")
+EXPECTED = 8
+
+
+def severities(text):
+    table = TABLE.search(text)
+    if not table:
+        sys.exit("the RFC holds no text between the facility and the severity caption; its tables have moved")
+    return ROW.findall(table.group(1))
+
+
+def keywords(page):
+    section = SECTION.search(html.unescape(re.sub(r"<[^>]+>", "", page)))
+    if not section:
+        sys.exit("POSIX spells no severity list between its own caption and the function declarations; the page has moved")
+    return [m.lower() for m in MACRO.findall(section.group(1))]
+
+
+def main():
+    p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    p.add_argument("--cache", default=str(CACHE))
+    p.add_argument("--source", default=SOURCE)
+    p.add_argument("--posix", default=POSIX)
+    p.add_argument("--out", default=str(OUT))
+    a = p.parse_args()
+    table = sorted(severities(source.fetch(a.source, a.cache, "rfc5424.txt").decode("utf-8")))
+    codes = [code for code, _ in table]
+    if codes != [str(i) for i in range(EXPECTED)]:
+        sys.exit(f"the severity table numbers {codes}, not 0 through {EXPECTED - 1}")
+    words = keywords(source.fetch(a.posix, a.cache, "posix-syslog.h.html").decode("utf-8"))
+    if len(words) != EXPECTED:
+        sys.exit(f"POSIX lists {len(words)} severity macros, not {EXPECTED}: {words}")
+    rows = []
+    for (code, severity), keyword in zip(table, words):
+        if not severity.lower().startswith(keyword):
+            sys.exit(f"severity {code} is {severity} in the RFC and {keyword} in POSIX, which is no prefix of it")
+        rows.append({"code": code, "keyword": keyword, "severity": severity})
+    tsv.write(a.out, COLUMNS, rows)
+
+
+if __name__ == "__main__":
+    main()

data/misc/loglevel.json

@@ -0,0 +1 @@
+{ "format": "{keyword}", "rows": "loglevel.tsv", "key": "code", "name": "keyword" }

data/misc/loglevel.tsv

@@ -0,0 +1,9 @@
+code	keyword	severity
+0	emerg	Emergency
+1	alert	Alert
+2	crit	Critical
+3	err	Error
+4	warning	Warning
+5	notice	Notice
+6	info	Informational
+7	debug	Debug

data_test.go

@@ -119,6 +119,9 @@ func TestShippedMiscReferenceData(t *testing.T) {
 		"protocol":              regexp.MustCompile(`^\S+( \S+)*$`),
 		"protocol.name":         regexp.MustCompile(`\S`),
 		"protocol.number":       regexp.MustCompile(`^\d{1,3}$`),
+		"loglevel":              regexp.MustCompile(`^(alert|crit|debug|emerg|err|info|notice|warning)$`),
+		"loglevel.code":         regexp.MustCompile(`^[0-7]$`),
+		"loglevel.severity":     regexp.MustCompile(`^(Alert|Critical|Debug|Emergency|Error|Informational|Notice|Warning)$`),
 		"tld":                   regexp.MustCompile(`^\.[a-z0-9]([a-z0-9-]*[a-z0-9])?$`),
 		"tld.type":              regexp.MustCompile(`^(country-code|generic|generic-restricted|infrastructure|sponsored)$`),
 		"tld.unicode":           regexp.MustCompile(`^\.\S+$`),

table_test.go

@@ -725,6 +725,16 @@ func TestShippedTables(t *testing.T) {
 		"tld[.рф]":                              ".xn--p1ai",
 		"tld[.xn--p1ai].unicode":                ".рф",
 		"territory[NP].timezone.offset":         "+05:45",
+		"loglevel[0]":                           "emerg",
+		"loglevel[1]":                           "alert",
+		"loglevel[2]":                           "crit",
+		"loglevel[3]":                           "err",
+		"loglevel[4]":                           "warning",
+		"loglevel[5]":                           "notice",
+		"loglevel[6]":                           "info",
+		"loglevel[7]":                           "debug",
+		"loglevel[err].code":                    "3",
+		"loglevel[6].severity":                  "Informational",
 	} {
 		if got := fake(t, f, path); got != want {
 			t.Errorf("Fake(%q) = %q, want %q", path, got, want)

testdata/shipped_shape.txt

@@ -240,6 +240,10 @@ misc.language	format "{name}"	key code	name name
 misc.language.code	string
 misc.language.code3	string
 misc.language.name	string
+misc.loglevel	format "{keyword}"	key code	name keyword
+misc.loglevel.code	string
+misc.loglevel.keyword	string
+misc.loglevel.severity	string
 misc.mac	format "{hex(2)}:{hex(2)}:{hex(2)}:{hex(2)}:{hex(2)}:{hex(2)}"
 misc.mimetype	format "{type}"	key type	name ext
 misc.mimetype.ext	string

todo.md

@@ -49,7 +49,6 @@ by a `data-import/` script, as README goal 10 asks.
 
 | Category | Shape | Source | Licence |
 |---|---|---|---|
-| `loglevel` | c | RFC 5424 severities | facts |
 | `ip` v4 documentation ranges, private, CIDR; `ipv6` `2001:db8::/32`; `mac` locally administered | t | RFC 5737, 1918, 9637 | facts |
 | `creditcard` per network: IIN, length, CVV, expiry | T+t | network rules, `{luhn()}` | facts |
 | `bic`, `isin`, `cusip` | t | structure rules | facts |
@@ -111,8 +110,9 @@ by a `data-import/` script, as README goal 10 asks.
   `misc.httpmethod[get]` and `misc.territory[se]` all answer "no row … has key or name"
   and stop there, where a case-insensitive match could name the row that exists, and a
   table with no `name` column could say it selects by key alone. `misc.tld[se]` misses
-  on the leading dot its keys carry rather than on case, so the near miss is worth
+  on the leading dot its keys carry rather than on case, and `misc.loglevel[Error]`
-  naming whatever shape it takes.
+  misses although a column carries `Error`, so the near miss is worth naming whatever
+  shape it takes.
 - Let a table column carry a `datatype`, so `--format json` writes
   `"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only
   a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text