Check in the CommonMark spec suite and pin its exception list #58

Open
lilleman wants to merge 6 commits from commonmark-spec-suite into main
9 changed files with 6371 additions and 12 deletions
+12 -5
View File
@@ -17,8 +17,10 @@ When losslessness and readability conflict, losslessness wins.
The other direction is a canonical fixpoint, not byte-identity: human markdown normalizes, the way
back yields the library's canonical spelling, and that spelling round-trips byte-identically —
where there is a way back. CommonMark spells link destinations the flavour has no escape for, so a
parse succeeding does not imply a spellable document; `todo.md` 3k's exception list names those.
where there is a way back. CommonMark spells some things the flavour has no escape for — a link
destination or title holding a backslash or newline, a paragraph opening with a code span whose
backticks read back as a fence — so a parse succeeding does not imply a spellable document;
`corpus/commonmark-spec/exceptions.json` names those.
"Equals" is structural equality over editor-normal ADF — adjacent text nodes with identical marks
merged, JSON number semantics, an empty attrs object, marks array or content array the absent
@@ -57,8 +59,12 @@ Round-trip equality is a property tested over a corpus, not a claim made in pros
why ~20 lines of own code cannot do the job, who maintains it, and what auditing it costs. So the
CommonMark and HTML parsers are written in this repo. A table a standard fixes is data rather than
a dependency: HTML5's 2125 semicolon-terminated character references ship packed in their own
module, so entity decoding is complete without one. `devDependencies`: few, each earning its keep;
they never reach a consumer.
module, so entity decoding is complete without one. The CommonMark spec suite is the same shape of
data and ships vendored at `corpus/commonmark-spec/` rather than as the `commonmark-spec` dev
dependency — that package is CommonJS-only, and Renovate auto-bumping a spec version would silently
point the vendored exception list's example numbers at a renumbered suite. A spec bump is a
deliberate re-pin, exceptions re-derived by hand beside it. `devDependencies`: few, each earning its
keep; they never reach a consumer.
## 6. The package contract
@@ -198,7 +204,8 @@ resolver maps them, under `NodeNext` alone; a `.d.ts` reader that is not `tsc` s
`node-floor.js` round-trips the installed package under a Node pinned to `engines.node`'s floor.
A fourth engine reads the build rather than the source: a headless Firefox loads `dist/index.js`
over HTTP and converts the whole corpus, which is §6's browser half and the only SpiderMonkey
over HTTP and converts the round-trip, normalization and error fixtures — the `commonmark-spec`
sort is the Node suite's to check — which is §6's browser half and the only SpiderMonkey
there is — the gate's other three engines are two V8s and a JavaScriptCore that is not Safari's.
A WebDriver session is what carries a verdict back out, the driver and the page's server sharing
one network namespace so each is the other's `127.0.0.1`; `--headless --screenshot` has no such
+1
View File
@@ -72,6 +72,7 @@ assert.deepEqual(
readdirSync(corpusRoot, { withFileTypes: true })
.filter((entry) => entry.isDirectory())
.map((entry) => entry.name)
.filter((name) => name !== 'commonmark-spec')
.sort(),
['errors', 'normalization', 'round-trip'],
'a corpus kind the browser leg does not convert',
+9 -1
View File
@@ -12,5 +12,13 @@ One directory per contract kind, each landing with its milestone:
pins which error.
- `real-payloads/``<name>.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No
expected markdown.
- `commonmark-spec/` — the CommonMark suite run against `markdownToAdf` by three checks. `spec.json`
is the suite; `refusals.json` pins each refusing example to its error `code`; `exceptions.json`
pins each known divergence by `check`, `example`, `kind` and the exact `divergence`, with a
`reason`. `kind` is `mark-model` (the permanent count divergence from ADF's mark-per-text-node
model), `unspellable` (parses but the flavour has no spelling) or `pending` (a parser gap a later
milestone may close).
JSON is editor-normal (AGENTS.md §2), two-space indent, keys sorted.
JSON is editor-normal (AGENTS.md §2), two-space indent, keys sorted. `spec.json` is the vendored,
upstream machine-readable suite, byte-exact (CommonMark 0.31.2, CC-BY-SA-4.0, © John MacFarlane),
and is not re-serialized by the corpus gate.
+436
View File
@@ -0,0 +1,436 @@
[
{
"check": "fixpoint",
"divergence": "unspellable-link",
"example": 196,
"kind": "unspellable",
"reason": "The link title holds literal newlines no escape spells."
},
{
"check": "fixpoint",
"divergence": "unspellable-link",
"example": 202,
"kind": "unspellable",
"reason": "The link destination holds a backslash the flavour cannot escape."
},
{
"check": "count",
"divergence": "ul 2/1",
"example": 301,
"kind": "pending",
"reason": "A list continuing past a marker change renders as two lists, the parser opens one."
},
{
"check": "count",
"divergence": "ol 2/1",
"example": 302,
"kind": "pending",
"reason": "A list continuing past a marker change renders as two lists, the parser opens one."
},
{
"check": "fixpoint",
"divergence": "unspellable-line-start",
"example": 330,
"kind": "unspellable",
"reason": "A paragraph opens with a code span whose backticks read back as a fence."
},
{
"check": "fixpoint",
"divergence": "unspellable-line-start",
"example": 331,
"kind": "unspellable",
"reason": "A paragraph opens with a code span whose backticks read back as a fence."
},
{
"check": "fixpoint",
"divergence": "unspellable-line-start",
"example": 340,
"kind": "unspellable",
"reason": "A paragraph opens with a code span whose backticks read back as a fence."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 369,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 373,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 389,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 393,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/5",
"example": 394,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/3",
"example": 395,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 399,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 404,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 406,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 407,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 408,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 409,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 410,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 411,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 413,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 414,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 415,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 3/1",
"example": 417,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 2/5 strong 1/3",
"example": 418,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/2",
"example": 422,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/3",
"example": 424,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 425,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 426,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 427,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 1/3",
"example": 428,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/3",
"example": 429,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/2",
"example": 430,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/2",
"example": 431,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/3 strong 2/5",
"example": 432,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "strong 1/2",
"example": 433,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 461,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 2/1",
"example": 463,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 464,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 465,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 3/1",
"example": 466,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "strong 2/1",
"example": 468,
"kind": "mark-model",
"reason": "CommonMark nests same-kind elements; the single mark collapses them to one."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 470,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 478,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 479,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "a 1/0",
"example": 484,
"kind": "pending",
"reason": "An empty link text stays literal text; CommonMark renders an empty link."
},
{
"check": "text",
"divergence": "\"\" against \"[](./target.md)\"",
"example": 484,
"kind": "pending",
"reason": "An empty link text stays literal text; CommonMark renders an empty link."
},
{
"check": "count",
"divergence": "a 1/0",
"example": 487,
"kind": "pending",
"reason": "An empty link text stays literal text; CommonMark renders an empty link."
},
{
"check": "text",
"divergence": "\"\" against \"[]()\"",
"example": 487,
"kind": "pending",
"reason": "An empty link text stays literal text; CommonMark renders an empty link."
},
{
"check": "fixpoint",
"divergence": "unspellable-link",
"example": 502,
"kind": "unspellable",
"reason": "The link destination holds a backslash the flavour cannot escape."
},
{
"check": "count",
"divergence": "a 1/5 em 1/4",
"example": 516,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/3",
"example": 519,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "a 1/5 em 1/4",
"example": 530,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 533,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "a 1/0",
"example": 540,
"kind": "pending",
"reason": "The case-folding shortcut reference is unresolved; CommonMark folds case and links."
},
{
"check": "text",
"divergence": "\"ẞ\" against \"[ẞ]\"",
"example": 540,
"kind": "pending",
"reason": "A case-folding shortcut reference is unresolved; CommonMark folds case and links."
},
{
"check": "count",
"divergence": "a 1/2",
"example": 554,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "a 1/2",
"example": 558,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "a 1/2",
"example": 559,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 638,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
},
{
"check": "count",
"divergence": "em 1/2",
"example": 639,
"kind": "mark-model",
"reason": "The mark spans adjacent text nodes, counted once per node where CommonMark nests one element."
}
]
+346
View File
@@ -0,0 +1,346 @@
[
{
"code": "unmappable-html",
"example": 21
},
{
"code": "unmappable-html",
"example": 31
},
{
"code": "unmappable-html",
"example": 148
},
{
"code": "unmappable-html",
"example": 149
},
{
"code": "unmappable-html",
"example": 150
},
{
"code": "unmappable-html",
"example": 151
},
{
"code": "unmappable-html",
"example": 152
},
{
"code": "unmappable-html",
"example": 153
},
{
"code": "unmappable-html",
"example": 154
},
{
"code": "unmappable-html",
"example": 155
},
{
"code": "unmappable-html",
"example": 156
},
{
"code": "unmappable-html",
"example": 157
},
{
"code": "unmappable-html",
"example": 158
},
{
"code": "unmappable-html",
"example": 159
},
{
"code": "unmappable-html",
"example": 160
},
{
"code": "unmappable-html",
"example": 161
},
{
"code": "unmappable-html",
"example": 162
},
{
"code": "unmappable-html",
"example": 163
},
{
"code": "unmappable-html",
"example": 164
},
{
"code": "unmappable-html",
"example": 165
},
{
"code": "unmappable-html",
"example": 166
},
{
"code": "unmappable-html",
"example": 167
},
{
"code": "unmappable-html",
"example": 168
},
{
"code": "unmappable-html",
"example": 169
},
{
"code": "unmappable-html",
"example": 170
},
{
"code": "unmappable-html",
"example": 171
},
{
"code": "unmappable-html",
"example": 172
},
{
"code": "unmappable-html",
"example": 173
},
{
"code": "unmappable-html",
"example": 174
},
{
"code": "unmappable-html",
"example": 175
},
{
"code": "unmappable-html",
"example": 176
},
{
"code": "unmappable-html",
"example": 177
},
{
"code": "unmappable-html",
"example": 178
},
{
"code": "unmappable-html",
"example": 179
},
{
"code": "unmappable-html",
"example": 180
},
{
"code": "unmappable-html",
"example": 181
},
{
"code": "unmappable-html",
"example": 182
},
{
"code": "unmappable-html",
"example": 183
},
{
"code": "unmappable-html",
"example": 184
},
{
"code": "unmappable-html",
"example": 185
},
{
"code": "unmappable-html",
"example": 186
},
{
"code": "unmappable-html",
"example": 187
},
{
"code": "unmappable-html",
"example": 188
},
{
"code": "unmappable-html",
"example": 189
},
{
"code": "unmappable-html",
"example": 190
},
{
"code": "unmappable-html",
"example": 191
},
{
"code": "unmappable-html",
"example": 201
},
{
"code": "unmappable-html",
"example": 308
},
{
"code": "unmappable-html",
"example": 309
},
{
"code": "unmappable-html",
"example": 344
},
{
"code": "unmappable-html",
"example": 475
},
{
"code": "unmappable-html",
"example": 476
},
{
"code": "unmappable-html",
"example": 477
},
{
"code": "unmappable-html",
"example": 491
},
{
"code": "unmappable-html",
"example": 494
},
{
"code": "unmappable-image",
"example": 517
},
{
"code": "unmappable-html",
"example": 524
},
{
"code": "unmappable-image",
"example": 531
},
{
"code": "unmappable-html",
"example": 536
},
{
"code": "unmappable-image",
"example": 572
},
{
"code": "unmappable-image",
"example": 573
},
{
"code": "unmappable-image",
"example": 576
},
{
"code": "unmappable-image",
"example": 577
},
{
"code": "unmappable-image",
"example": 579
},
{
"code": "unmappable-image",
"example": 584
},
{
"code": "unmappable-image",
"example": 585
},
{
"code": "unmappable-image",
"example": 586
},
{
"code": "unmappable-image",
"example": 587
},
{
"code": "unmappable-image",
"example": 588
},
{
"code": "unmappable-image",
"example": 589
},
{
"code": "unmappable-image",
"example": 591
},
{
"code": "unmappable-html",
"example": 613
},
{
"code": "unmappable-html",
"example": 614
},
{
"code": "unmappable-html",
"example": 615
},
{
"code": "unmappable-html",
"example": 616
},
{
"code": "unmappable-html",
"example": 617
},
{
"code": "unmappable-html",
"example": 623
},
{
"code": "unmappable-html",
"example": 625
},
{
"code": "unmappable-html",
"example": 626
},
{
"code": "unmappable-html",
"example": 627
},
{
"code": "unmappable-html",
"example": 628
},
{
"code": "unmappable-html",
"example": 629
},
{
"code": "unmappable-html",
"example": 630
},
{
"code": "unmappable-html",
"example": 631
},
{
"code": "unmappable-html",
"example": 642
},
{
"code": "unmappable-html",
"example": 643
}
]
File diff suppressed because it is too large Load Diff
+340
View File
@@ -0,0 +1,340 @@
import assert from 'node:assert/strict'
import { createHash } from 'node:crypto'
import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import test from 'node:test'
import { fileURLToPath } from 'node:url'
import type { AdfDocument, AdfNode } from './adf/document.ts'
import { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts'
import { markdownToAdf } from './markdown/parse/markdown-to-adf.ts'
const root = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus', 'commonmark-spec')
const checks = ['count', 'fixpoint', 'text'] as const
type Check = (typeof checks)[number]
type ExceptionKind = 'mark-model' | 'pending' | 'unspellable'
type SpecExample = { example: number; html: string; markdown: string; section: string }
type Exception = { check: Check; divergence: string; example: number; kind: ExceptionKind; reason: string }
type Refusal = { code: string; example: number }
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value)
}
function isCheck(value: unknown): value is Check {
return checks.some((check) => check === value)
}
function isKind(value: unknown): value is ExceptionKind {
return value === 'mark-model' || value === 'pending' || value === 'unspellable'
}
function isSpecExample(value: unknown): value is SpecExample {
if (!isRecord(value)) return false
return typeof value['example'] === 'number' && typeof value['html'] === 'string' && typeof value['markdown'] === 'string' && typeof value['section'] === 'string'
}
function isException(value: unknown): value is Exception {
if (!isRecord(value)) return false
return (
isCheck(value['check']) &&
typeof value['divergence'] === 'string' &&
value['divergence'].length > 0 &&
typeof value['example'] === 'number' &&
isKind(value['kind']) &&
typeof value['reason'] === 'string' &&
value['reason'].length > 0
)
}
function isRefusal(value: unknown): value is Refusal {
if (!isRecord(value)) return false
return typeof value['code'] === 'string' && value['code'].length > 0 && typeof value['example'] === 'number'
}
function readJson<T>(name: string, guard: (value: unknown) => value is T, shape: string): T[] {
const parsed: unknown = JSON.parse(readFileSync(join(root, name), 'utf8'))
assert.ok(Array.isArray(parsed), `${name} is not an array`)
return parsed.map((value, index) => {
assert.ok(guard(value), `${name} holds a ${shape} with the wrong shape at ${index}`)
return value
})
}
const spec = readJson('spec.json', isSpecExample, 'spec example')
const exceptions = readJson('exceptions.json', isException, 'exception')
const refusals = readJson('refusals.json', isRefusal, 'refusal')
const exampleToRefusal = new Map(refusals.map((refusal) => [refusal.example, refusal.code]))
const exceptionIndex = new Map(exceptions.map((entry) => [`${entry.example}:${entry.check}`, entry]))
test('the CommonMark spec suite is 0.31.2, vendored byte-exact', () => {
const digest = createHash('sha256').update(readFileSync(join(root, 'spec.json'))).digest('hex')
assert.equal(digest, 'd431b29d97b6f73e69d547109cf5081578fac931e72afe95639ebe766c1b2a20')
})
test('every exception is unique, names a parsing example, and files a fixpoint only as unspellable', () => {
assert.equal(exceptionIndex.size, exceptions.length, 'one exception repeats an example and check another holds')
for (const entry of exceptions) {
assert.ok(spec.some((candidate) => candidate.example === entry.example), `exception ${entry.example} names no example in the suite`)
assert.equal(exampleToRefusal.get(entry.example), undefined, `exception ${entry.example} is on the refusal list, not an exception`)
if (entry.check === 'fixpoint') assert.equal(entry.kind, 'unspellable', `exception ${entry.example} files a fixpoint divergence as ${entry.kind}; a fixable hole is given the spelling instead`)
}
})
test('the refusal list is unique per example and names real examples', () => {
assert.equal(exampleToRefusal.size, refusals.length, 'one refusal repeats an example another holds')
for (const example of exampleToRefusal.keys()) assert.ok(spec.some((entry) => entry.example === example), `refusal ${example} names no example in the suite`)
})
// A mark is counted once per text node it touches (AGENTS.md §14).
const countKeys = ['a', 'blockquote', 'br', 'code', 'em', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'hr', 'img', 'li', 'ol', 'pre', 'strong', 'ul']
const nodeElement: Record<string, string> = {
blockquote: 'blockquote',
bulletList: 'ul',
codeBlock: 'pre',
hardBreak: 'br',
listItem: 'li',
media: 'img',
mediaInline: 'img',
orderedList: 'ol',
rule: 'hr',
}
const markElement: Record<string, string> = { code: 'code', em: 'em', link: 'a', strong: 'strong' }
const blockTags = new Set(['blockquote', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'hr', 'li', 'ol', 'p', 'pre', 'ul'])
function tagName(tag: string): string {
return tag.slice(1).replace(/^\//, '').split(/[\s/>]/)[0] ?? ''
}
function emptyCounts(): Record<string, number> {
return Object.fromEntries(countKeys.map((key) => [key, 0]))
}
function referenceCounts(html: string): Record<string, number> {
const counts = emptyCounts()
let inPre = false
for (let index = 0; index < html.length; index += 1) {
if (html[index] !== '<') continue
const close = html.indexOf('>', index)
if (close === -1) break
const tag = html.slice(index, close + 1)
if (tag.startsWith('</')) {
if (tagName(tag) === 'pre') inPre = false
index = close
continue
}
const name = tagName(tag)
if (name === 'pre') {
inPre = true
counts['pre'] = (counts['pre'] ?? 0) + 1
} else if (name === 'code' && inPre) {
// A code block's `<code>` is the `<pre>`'s body, already counted.
} else if (countKeys.includes(name)) {
counts[name] = (counts[name] ?? 0) + 1
}
index = close
}
return counts
}
function nodeCounts(document: AdfNode): Record<string, number> {
const counts = emptyCounts()
const pending: AdfNode[] = [document]
while (pending.length > 0) {
const node = pending.pop()
if (node === undefined) continue
if (node.text !== undefined) {
const seen = new Set<string>()
for (const mark of node.marks ?? []) {
const element = markElement[mark.type]
if (element !== undefined) seen.add(element)
}
for (const element of seen) counts[element] = (counts[element] ?? 0) + 1
continue
}
if (node.type === 'heading') {
const level = node.attrs?.['level']
if (typeof level === 'number') counts[`h${level}`] = (counts[`h${level}`] ?? 0) + 1
pending.push(...(node.content ?? []))
continue
}
const element = nodeElement[node.type]
if (element !== undefined) counts[element] = (counts[element] ?? 0) + 1
pending.push(...(node.content ?? []))
}
return counts
}
const namedEntity: Record<string, string> = { amp: '&', gt: '>', lt: '<', ouml: 'ö', quot: '"' }
function decodeHtmlEntity(text: string, index: number): { length: number; text: string } | undefined {
if (text[index] !== '&') return undefined
const end = text.indexOf(';', index)
if (end === -1 || end - index > 8) return undefined
const reference = text.slice(index, end + 1)
const named = namedEntity[reference.slice(1, -1)]
return named === undefined ? undefined : { length: reference.length, text: named }
}
test('the oracle decodes every entity the reference HTML holds', () => {
for (const example of spec) {
for (const [reference] of example.html.matchAll(/&#?[0-9A-Za-z]+;/g)) {
assert.ok(decodeHtmlEntity(reference, 0) !== undefined, `example ${example.example} holds ${reference}, which the oracle would leave literal`)
}
}
})
function referenceText(html: string): string {
const parts: string[] = []
let preDepth = 0
let atBoundary = true
let skipNewline = false
for (let index = 0; index < html.length; index += 1) {
const character = html.charAt(index)
if (character === '<') {
const close = html.indexOf('>', index)
if (close === -1) break
const tag = html.slice(index, close + 1)
const name = tagName(tag)
if (name === 'br') {
parts.push(' ')
atBoundary = false
skipNewline = true
index = close
continue
}
if (name === 'pre') {
if (tag.startsWith('</')) {
preDepth -= 1
trimTrailingNewline(parts)
} else {
preDepth += 1
}
atBoundary = true
} else {
atBoundary = blockTags.has(name)
}
index = close
continue
}
if (character === '\n') {
if (skipNewline) {
skipNewline = false
continue
}
if (preDepth > 0) {
parts.push('\n')
continue
}
if (!atBoundary && !followedByBlock(html, index + 1)) parts.push(' ')
continue
}
const reference = decodeHtmlEntity(html, index)
if (reference !== undefined) {
parts.push(reference.text)
atBoundary = false
index += reference.length - 1
continue
}
parts.push(character)
atBoundary = false
}
return parts.join('')
}
function trimTrailingNewline(parts: string[]): void {
const last = parts[parts.length - 1]
if (last === undefined) return
parts[parts.length - 1] = last.endsWith('\n') ? last.slice(0, -1) : last
}
// A newline beside a block open or close is a boundary rather than a soft break, so it spells no space.
function followedByBlock(html: string, index: number): boolean {
let next = index
while (next < html.length && (html[next] === '\n' || html[next] === ' ' || html[next] === '\t')) next += 1
if (next >= html.length) return true
if (html[next] !== '<') return false
const close = html.indexOf('>', next)
return close !== -1 && blockTags.has(tagName(html.slice(next, close + 1)))
}
function concatenatedText(document: AdfNode): string {
const parts: string[] = []
const pending: { inCode: boolean; node: AdfNode }[] = [{ inCode: false, node: document }]
while (pending.length > 0) {
const frame = pending.pop()
if (frame === undefined) continue
const { inCode, node } = frame
if (node.text !== undefined) {
parts.push(inCode ? node.text : node.text.replace(/\n/g, ' '))
continue
}
if (node.type === 'hardBreak') {
parts.push(' ')
continue
}
const childInCode = inCode || node.type === 'codeBlock'
const content = node.content ?? []
for (let index = content.length - 1; index >= 0; index -= 1) {
const child = content[index]
if (child !== undefined) pending.push({ inCode: childInCode, node: child })
}
}
return parts.join('')
}
function fixpointRefused(example: SpecExample, document: AdfDocument): string | undefined {
const emitted = adfToMarkdown(document)
if (!emitted.ok) return emitted.error.code
const again = markdownToAdf(emitted.value)
assert.ok(again.ok, `example ${example.example} emits markdown it cannot read back`)
assert.deepEqual(again.value, document, `example ${example.example} does not hold its own round-trip`)
return undefined
}
function textMismatch(example: SpecExample, document: AdfDocument): string | undefined {
const expected = referenceText(example.html)
const actual = concatenatedText(document)
return expected === actual ? undefined : `${JSON.stringify(expected)} against ${JSON.stringify(actual)}`
}
function countMismatch(example: SpecExample, document: AdfDocument): string | undefined {
const expected = referenceCounts(example.html)
const actual = nodeCounts(document)
const names = countKeys.filter((key) => expected[key] !== actual[key])
return names.length === 0 ? undefined : names.map((name) => `${name} ${expected[name]}/${actual[name]}`).join(' ')
}
for (const example of spec) {
test(`CommonMark example ${example.example} => ${example.section}`, () => {
const parse = markdownToAdf(example.markdown)
const refused = exampleToRefusal.get(example.example)
if (refused !== undefined) {
assert.ok(!parse.ok, `example ${example.example} was expected to refuse with ${refused} but parsed`)
assert.equal(parse.error.code, refused, `example ${example.example} refused with a different code`)
return
}
if (!parse.ok) assert.fail(`example ${example.example} was expected to parse but refused with ${parse.error.code}`)
const divergences: Record<Check, string | undefined> = {
count: countMismatch(example, parse.value),
fixpoint: fixpointRefused(example, parse.value),
text: textMismatch(example, parse.value),
}
for (const check of checks) {
const entry = exceptionIndex.get(`${example.example}:${check}`)
const divergence = divergences[check]
if (divergence === undefined) {
assert.equal(entry, undefined, `example ${example.example} passes its ${check} check but files an exception`)
} else {
assert.ok(entry !== undefined, `example ${example.example} ${check} check fails: ${divergence}`)
assert.equal(entry.divergence, divergence, `example ${example.example} ${check} diverged differently than filed`)
}
}
})
}
+3 -2
View File
@@ -1,6 +1,6 @@
import assert from 'node:assert/strict'
import { readFileSync, readdirSync } from 'node:fs'
import { basename, dirname, join } from 'node:path'
import { basename, dirname, join, sep } from 'node:path'
import test from 'node:test'
import { fileURLToPath } from 'node:url'
@@ -53,12 +53,13 @@ function pairedNames(root: string, first: string, second: string): string[] {
function corpusJsonPaths(): string[] {
return readdirSync(corpusRoot, { encoding: 'utf8', recursive: true })
.filter((name) => name.endsWith('.json'))
.filter((name) => name !== `commonmark-spec${sep}spec.json`)
.map((name) => join(corpusRoot, name))
.sort()
}
test('every corpus directory is a kind the runner reads', () => {
assert.deepEqual(directoryNames(corpusRoot), ['errors', 'normalization', 'round-trip'])
assert.deepEqual(directoryNames(corpusRoot), ['commonmark-spec', 'errors', 'normalization', 'round-trip'])
})
test('every round-trip directory is a kind the runner reads', () => {
+6 -4
View File
@@ -53,7 +53,7 @@ The numbering is the order the work was planned in, not the order it ships.
- [x] **3i — The inline nodes and the marks.**
- [x] **3j — The carry and the combinations.**
- [ ] **3k — The CommonMark spec suite (`0.2.0`).** Checked in at `corpus/commonmark-spec/`,
pinned to the version it ships — the one `html-blocks.ts` names for its start
pinned to the version it ships — the one `commonmark-grammar.ts` names for its start
conditions — `corpus/README.md` gaining the kind.
**Settled** (the maintainer, 2026-08-27): three checks an example must pass, the reference
HTML each ships read as corpus data — which adds no format and no direction (§1). §2's
@@ -67,9 +67,11 @@ The numbering is the order the work was planned in, not the order it ships.
counts close both. The exception list stays the maintainer's, and one entry is owed
already: 3h continues a list across the marker change CommonMark splits on, so an example
the reference HTML gives two `<ul>` counts one `bulletList`. One outcome is no
exception and must not be filed as one: valid CommonMark parsing to a document
`adfToMarkdown` refuses is a §2 hole, which is what `corpus/unspellable/` held until 3c,
3e and 3h landed their answers and emptied it.
exception and must not be filed as one: a fixable §2 hole — valid CommonMark parsing to a
document `adfToMarkdown` refuses which is what `corpus/unspellable/` held until 3c, 3e
and 3h landed their answers and emptied it. The permanent ones — a link destination or
title no escape spells, a paragraph opening with a code span — are the exceptions, named
by AGENTS.md §2.
- [ ] **4 — Round-trip property tests (`0.2.0`)**, widening 3j's corpus round-trip past the
documents a human wrote — the thing that proves 2 and 3 beyond them. Editor-normal (§2) is
finished here, on 3i's merging — `toEditorNormal(doc)` and the equality the round-trip