Spell every document the emitter refused for a carriage return, a null character, a code-span line start or an empty text node
CI / gate (push) Successful in 57s
CI / publish (push) Has been skipped

This commit is contained in:
2026-10-04 20:29:29 +02:00
parent bfbe4a51fb
commit 03f18ee17b
19 changed files with 160 additions and 190 deletions
+8
View File
@@ -17,6 +17,14 @@
`content` or `marks` is spelled `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is `content` or `marks` is spelled `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is
spelled `-0`, and a `codeBlock` of several text nodes is a fence per node. A `codeBlock` holding spelled `-0`, and a `codeBlock` of several text nodes is a fence per node. A `codeBlock` holding
other than plain text nodes rides the block carry, where it was refused. other than plain text nodes rides the block carry, where it was refused.
- **Breaking:** `unspellable-character` and `unspellable-line-start` leave `ConvertErrorCode`, and
`adfToMarkdown` no longer refuses `unspellable-whitespace`. A carriage return in text is spelled
`
` and a null character `!adf:text{text="\u0000"}`; a code block holding either, a text node
holding no text or holding `content`, a paragraph line opening with a code span whose backticks
read back as a fence, and an `emoji`, `mention` or `status` whose `text` holds a line ending or a
null character ride the carry. See `MIGRATION.md`.
- Fix a link `title` holding a null character: it is written as `!adf:link[text]{attrs}`, where it
read back as U+FFFD.
- **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no - **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no
CommonMark escape spells is written as `!adf:link[text]{attrs}`. CommonMark escape spells is written as `!adf:link[text]{attrs}`.
- **Breaking:** some directive refusals carry `malformed-directive` where they carried - **Breaking:** some directive refusals carry `malformed-directive` where they carried
+8 -2
View File
@@ -62,12 +62,18 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc
### Error codes ### Error codes
`unspellable-link` leaves `ConvertErrorCode`: a `switch` naming it stops compiling, and the link `unspellable-character`, `unspellable-line-start` and `unspellable-link` leave `ConvertErrorCode`:
it named converts. a `switch` naming one stops compiling, and the document it named converts.
| Input | `0.1.0` | `0.2.0` | | Input | `0.1.0` | `0.2.0` |
| --- | --- | --- | | --- | --- | --- |
| a link whose `href` or `title` no CommonMark escape spells, on emit | `unspellable-link` | spells `!adf:link[text]{attrs}` | | a link whose `href` or `title` no CommonMark escape spells, on emit | `unspellable-link` | spells `!adf:link[text]{attrs}` |
| a carriage return in text, on emit | `unspellable-character` | spells `
` |
| a null character in text, on emit | `unspellable-character` | spells `!adf:text{text="\u0000"}` |
| a carriage return or null character in a code block or code span, on emit | `unspellable-character` | rides the carry |
| a paragraph line opening with a code span whose backticks read back as a fence, on emit | `unspellable-line-start` | rides the inline carry |
| an `emoji`, `mention` or `status` whose `text` holds a line ending or a null character, on emit | `unspellable-whitespace`, `unspellable-character` | rides the inline carry |
| a text node holding no text, or holding `content`, on emit | `unsupported-node-shape` | rides the carry |
| a `codeBlock` holding other than plain text nodes, on emit | `unsupported-node-shape` | rides the block carry | | a `codeBlock` holding other than plain text nodes, on emit | `unsupported-node-shape` | rides the block carry |
| a leaf node given a body (`media`, `listBreak`) | `unsupported-node-shape` | `malformed-directive` | | a leaf node given a body (`media`, `listBreak`) | `unsupported-node-shape` | `malformed-directive` |
| a node with a block body written as a leaf (`panel`) | `unsupported-node-shape` | `malformed-directive` | | a node with a block body written as a leaf (`panel`) | `unsupported-node-shape` | `malformed-directive` |
+5 -10
View File
@@ -180,6 +180,7 @@ Parsing — `markdownToAdf` and `plainMarkdownToAdf`, and `htmlToAdf` at `0.2.0`
| `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in `spec/flavour.md`, or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning | | `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in `spec/flavour.md`, or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning |
| `unmappable-html` | the input holds an HTML construct the documented element set does not map, a comment and a processing instruction among them — at this version that is every raw HTML construct in markdown, the element set landing at `0.2.0` | remove the construct, or write what it holds in the lossless flavour | | `unmappable-html` | the input holds an HTML construct the documented element set does not map, a comment and a processing instruction among them — at this version that is every raw HTML construct in markdown, the element set landing at `0.2.0` | remove the construct, or write what it holds in the lossless flavour |
| `unmappable-image` | an image sits inside other content that is not another image's description, or carries a title | give the image a paragraph of its own and drop the title | | `unmappable-image` | an image sits inside other content that is not another image's description, or carries a title | give the image a paragraph of its own and drop the title |
| `unspellable-whitespace` | an `emoji`, `mention` or `status` directive's content slot spells a newline or a carriage return — `
`, `
`, `!adf:text{text="\n"}` | replace it with a space, or write the node as `!adf:carry` — an inline directive never spans lines |
Emitting — `adfToMarkdown`, and `adfToHtml` at `0.2.0`: Emitting — `adfToMarkdown`, and `adfToHtml` at `0.2.0`:
@@ -194,9 +195,6 @@ emit refuses:
| Code | Fires when | What you can do | | Code | Fires when | What you can do |
| --- | --- | --- | | --- | --- | --- |
| `unspellable-character` | text or a code block holds a carriage return or a null character, which CommonMark rewrites wherever it sits | strip or replace the character; no escape carries it through the round-trip |
| `unspellable-line-start` | a paragraph line begins with a code span whose backticks would read back as a code fence | put any text before the code span |
| `unspellable-whitespace` | an `emoji`, `mention` or `status` holds a newline in the text its inline directive spells in the content slot | replace it with a space — an inline directive never spans lines |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive stands out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` anywhere but as the whole document | fix what the message names; `spec/flavour.md` lists every type's attributes and body | | `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive stands out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` anywhere but as the whole document | fix what the message names; `spec/flavour.md` lists every type's attributes and body |
@@ -217,9 +215,7 @@ Serves Goals 1, 3 and 4.
`spec/flavour.md`) — and one gap: a CommonMark image fits only as its own title-less paragraph; `spec/flavour.md`) — and one gap: a CommonMark image fits only as its own title-less paragraph;
mid-text and titled images are error results, save an image inside another's description, which mid-text and titled images are error results, save an image inside another's description, which
flattens into the alt text. Converting back yields the library's canonical spelling, which flattens into the alt text. Converting back yields the library's canonical spelling, which
round-trips byte-identically — where it converts back at all: a parse succeeding is no promise of round-trips byte-identically.
that, so keep the source until the way back succeeds. ``` ` `` ` ``` reads cleanly and then
refuses.
- Four CommonMark spellings parse without an error and build a document the reference - Four CommonMark spellings parse without an error and build a document the reference
implementation renders differently: `[](/url)` and `[]()` stay literal text against CommonMark's implementation renders differently: `[](/url)` and `[]()` stay literal text against CommonMark's
empty link, a list continuing past a marker change stays one list against CommonMark's two, a empty link, a list continuing past a marker change stays one list against CommonMark's two, a
@@ -228,10 +224,9 @@ Serves Goals 1, 3 and 4.
text, which the spec requires and the reference itself breaks, nesting one `<a>` in the other. text, which the spec requires and the reference itself breaks, nesting one `<a>` in the other.
The first three are pinned `pending` in `corpus/commonmark-spec/exceptions.json`; the suite The first three are pinned `pending` in `corpus/commonmark-spec/exceptions.json`; the suite
holds no example of the fourth. holds no example of the fourth.
- Not every document converts back: `adfToMarkdown` is partial on valid ADF — a text node holding - Not every document converts back: `adfToMarkdown` refuses with `unsupported-nesting-depth` or
a carriage return, or a paragraph line beginning with a code span whose backticks read back as a `unsupported-node-shape` as the tables above say. Show the refusal and keep the document
fence. Show the refusal and keep the document read-only; saving markdown you could not produce read-only; saving markdown you could not produce is the loss the round-trip exists to stop.
is the loss the round-trip exists to stop.
- The pipe table narrows GFM's twice: every row opens with a pipe, so GFM's bare form is an error - The pipe table narrows GFM's twice: every row opens with a pipe, so GFM's bare form is an error
result rather than the prose it reads as, and an alignment colon in the delimiter row is an result rather than the prose it reads as, and an alignment colon in the delimiter row is an
error too — ADF holds no column alignment. The trailing pipe is canonical output, optional in error too — ADF holds no column alignment. The trailing pipe is canonical output, optional in
-21
View File
@@ -13,27 +13,6 @@
"kind": "pending", "kind": "pending",
"reason": "A list continuing past a marker change renders as two lists, the parser opens one." "reason": "A list continuing past a marker change renders as two lists, the parser opens one."
}, },
{
"check": "fixpoint",
"divergence": "unspellable-line-start",
"example": 330,
"kind": "unspellable",
"reason": "A paragraph opens with a code span whose backticks read back as a fence."
},
{
"check": "fixpoint",
"divergence": "unspellable-line-start",
"example": 331,
"kind": "unspellable",
"reason": "A paragraph opens with a code span whose backticks read back as a fence."
},
{
"check": "fixpoint",
"divergence": "unspellable-line-start",
"example": 340,
"kind": "unspellable",
"reason": "A paragraph opens with a code span whose backticks read back as a fence."
},
{ {
"check": "count", "check": "count",
"divergence": "em 2/1", "divergence": "em 2/1",
+12 -14
View File
@@ -23,13 +23,11 @@ sanitized and a mention keeps the test user's real account id.
## Markdown in is a canonical fixpoint ## Markdown in is a canonical fixpoint
2026-08-23, the maintainer. Goals 1 and 4. Valid while markdown input may be written by hand. 2026-08-23, the maintainer; the way back made total 2026-10-04. Goals 1 and 4. Valid while
markdown input may be written by hand.
The other direction is a canonical fixpoint, not byte-identity: human markdown normalizes, the way The other direction is a canonical fixpoint, not byte-identity: human markdown normalizes, the way
back yields the library's canonical spelling, and that spelling round-trips byte-identically — back yields the library's canonical spelling, and that spelling round-trips byte-identically.
where there is a way back. CommonMark spells some things the flavour has no escape for — a
paragraph opening with a code span whose backticks read back as a fence — so a parse succeeding
does not imply a spellable document; `corpus/commonmark-spec/exceptions.json` names those.
## Equality is deep ## Equality is deep
@@ -338,8 +336,9 @@ switches on `code` with no `default`.
`unsupported-node-shape` — since a code no input reaches is one no consumer can switch on `unsupported-node-shape` — since a code no input reaches is one no consumer can switch on
(2026-09-20). (2026-09-20).
- A refusal no spelling recovers from is a gap in the flavour rather than a code: give the flavour - A refusal no spelling recovers from is a gap in the flavour rather than a code: give the flavour
the spelling and the code goes (`unspellable-link`, 2026-09-13). A cause the carry answers gets the spelling and the code goes (`unspellable-link`, 2026-09-13; `unspellable-character` and
no code: a mark no spelling writes rides the carry with its node. `unspellable-line-start`, 2026-10-04). A cause the carry answers gets no code: a mark no
spelling writes rides the carry with its node.
## Which code a cause takes ## Which code a cause takes
@@ -348,11 +347,10 @@ handles one cause alike whichever node, attribute or direction raised it.
- A code names the cause; where one cause recurs across node types, across one mark's attributes - A code names the cause; where one cause recurs across node types, across one mark's attributes
or across directions, one code covers them all and `path` and `message` say which — or across directions, one code covers them all and `path` and `message` say which —
`unsupported-nesting-depth` is the 500-level guard whichever direction hits it, `unsupported-nesting-depth` is the 500-level guard whichever direction hits it. Where two codes
`unspellable-character` the text node and the code block alike. Where two codes stay apart, the stay apart, the line between them is what they name: `unspellable-whitespace` is the newline an
line between them is what they name: `unspellable-character` is a character CommonMark rewrites inline directive's content slot spells in input, where the emitter carries the node
wherever text holds it, `unspellable-whitespace` the newline no inline directive's content slot (2026-10-04).
spans, in either direction.
- A claim code names the spelling claimed, never the node that spelling would have built: a - A claim code names the spelling claimed, never the node that spelling would have built: a
malformed `!adf:table` is a `malformed-directive`, and an alignment colon a malformed `!adf:table` is a `malformed-directive`, and an alignment colon a
`malformed-pipe-table` — the flavour's own delimiter row is `-` runs, so the grammar refuses the `malformed-pipe-table` — the flavour's own delimiter row is `-` runs, so the grammar refuses the
@@ -525,8 +523,8 @@ Each example is a named error or markdown that parses and emits to itself byte f
reference HTML's text, tags stripped and entities decoded, equals the parsed document's; and its reference HTML's text, tags stripped and entities decoded, equals the parsed document's; and its
elements count the marks and nodes they map to. The fixpoint alone passes a parser returning the elements count the marks and nodes they map to. The fixpoint alone passes a parser returning the
empty document, the text alone one dropping every emphasis. An exception is the maintainer's to empty document, the text alone one dropping every emphasis. An exception is the maintainer's to
add, and valid CommonMark parsing to a document `adfToMarkdown` refuses where a spelling could exist add, and valid CommonMark parsing to a document `adfToMarkdown` refuses is a bug to fix, never an
is a bug to fix, never an exception. exception.
## The flavour spec is read as a source ## The flavour spec is read as a source
+16 -12
View File
@@ -245,7 +245,8 @@ form.
leaves empty, or in both, is a named error, and so are fences whose info strings differ and an leaves empty, or in both, is a named error, and so are fences whose info strings differ and an
empty fence beside another. Each fence is an ordinary code block, and its info string decodes empty fence beside another. Each fence is an ordinary code block, and its info string decodes
escapes and entity references as any other does. A node holding a child no fence holds — any but escapes and entity references as any other does. A node holding a child no fence holds — any but
a text node carrying no marks, `attrs` or `content` — rides the block carry. a text node holding text free of carriage returns and null characters, and carrying no marks,
`attrs` or `content` — rides the block carry.
- `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is - `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is
the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no
CommonMark spelling. CommonMark spelling.
@@ -449,6 +450,7 @@ the nodes below, `emoji`, `mention` and `status` spell their `text` attribute in
plain text: `[]` is the empty string, absent content is the absent attribute, non-empty content plain text: `[]` is the empty string, absent content is the absent attribute, non-empty content
parsing to anything but one text node carrying neither marks, attributes nor content is a named parsing to anything but one text node carrying neither marks, attributes nor content is a named
error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot. error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot.
A node whose `text` holds a line ending or a null character rides the inline carry.
The rest take no content, `!adf:text` included; content on a node that takes none is a named error. The rest take no content, `!adf:text` included; content on a node that takes none is a named error.
- `date` — Attributes: `localId` (string), `timestamp` (string, epoch milliseconds). - `date` — Attributes: `localId` (string), `timestamp` (string, epoch milliseconds).
@@ -471,14 +473,15 @@ The rest take no content, `!adf:text` included; content on a node that takes non
Shipped !adf:emoji[🎉]{shortName=":tada:"} on !adf:date{timestamp=1756080000000}. Shipped !adf:emoji[🎉]{shortName=":tada:"} on !adf:date{timestamp=1756080000000}.
``` ```
**Whitespace CommonMark cannot hold.** A newline inside a text node, and a space or tab where **Characters CommonMark cannot hold.** A carriage return in text is `&#13;`, which CommonMark reads
CommonMark strips or refuses one — a block's inline content edges, either side of a line break, an back as the character. A newline inside a text node, a null character, which CommonMark reads as
em, strong or strike spelling's inner edges, a pipe cell's edges — is spelled `!adf:text{text="…"}`, U+FFFD, and a space or tab where CommonMark strips or refuses one — a block's inline content edges,
the reserved key carrying the node's text, escaped by the attribute grammar and never literal: pipe either side of a line break, an em, strong or strike spelling's inner edges, a pipe cell's edges —
cells trim and pad. The emitter wraps the whitespace run alone and leaves the rest plain text, which is spelled `!adf:text{text="…"}`, the reserved key carrying the node's text, escaped by the
the spelled run joins on reading. Input reads that spelling alone: the value is one run of spaces attribute grammar and never literal: pipe cells trim and pad. The emitter wraps the run alone and
and tabs, or one run of newlines, and anything else — a mixed run, or text CommonMark carries leaves the rest plain text, which the spelled run joins on reading. Input reads that spelling
plainly — is a named error. alone: the value is one run of spaces and tabs, one run of newlines, or one run of null characters,
and anything else — a mixed run, or text CommonMark carries plainly — is a named error.
``` ```
!adf:text{text=" "}Two leading spaces held, and one text node split!adf:text{text="\n"}over two lines. !adf:text{text=" "}Two leading spaces held, and one text node split!adf:text{text="\n"}over two lines.
@@ -489,8 +492,8 @@ carried, neither holds `attrs` or an empty key, and their marks are identical, a
The reserved leaf `!adf:textBreak{}` parts such a pair, inside every mark spelling the two share; a The reserved leaf `!adf:textBreak{}` parts such a pair, inside every mark spelling the two share; a
code span holds no directive, so it closes and reopens. It builds no node and reads only between code span holds no directive, so it closes and reopens. It builds no node and reads only between
two such nodes: elsewhere, or with `[content]` or `{attrs}`, it is a named error (`docs/decisions.md` two such nodes: elsewhere, or with `[content]` or `{attrs}`, it is a named error (`docs/decisions.md`
§`!adf:textBreak{}` parts text CommonMark would join). A text node holding `attrs` or an empty key §`!adf:textBreak{}` parts text CommonMark would join). A text node holding `attrs`, `content` or an
rides the inline carry. empty key, or holding no text, rides the inline carry.
``` ```
Hello, !adf:textBreak{}world — **Hello, !adf:textBreak{}world** — `a`!adf:textBreak{}`b` Hello, !adf:textBreak{}world — **Hello, !adf:textBreak{}world** — `a`!adf:textBreak{}`b`
@@ -531,7 +534,8 @@ breaks at every node the emitter carries, so no emitted carry sits inside a mark
An inline node whose marks no nesting spells — a mark type not listed here, an attrs key its An inline node whose marks no nesting spells — a mark type not listed here, an attrs key its
spelling does not list, a value that is not the spelling's type, an attribute the spelling needs and spelling does not list, a value that is not the spelling's type, an attribute the spelling needs and
the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside
another mark, `code` over anything but a text node or over text holding a newline, or a spelling another mark, `code` over anything but a text node holding text, over text holding a line ending or
a null character, or opening a paragraph line its backticks would read back as a fence, or a spelling
CommonMark's flanking rules cannot open or close where the run sits (`un**-real**istic`), or one CommonMark's flanking rules cannot open or close where the run sits (`un**-real**istic`), or one
CommonMark's matching pairs elsewhere — the intra-word `*` runs together with a neighbouring `**`, CommonMark's matching pairs elsewhere — the intra-word `*` runs together with a neighbouring `**`,
and the multiple-of-3 rule can leave the merged run's pairing to another delimiter — rides the and the multiple-of-3 rule can leave the merged run's pairing to another delimiter — rides the
+4 -4
View File
@@ -13,7 +13,7 @@ const root = join(dirname(fileURLToPath(import.meta.url)), '..', '..', 'corpus',
const checks = ['count', 'fixpoint', 'text'] as const const checks = ['count', 'fixpoint', 'text'] as const
type Check = (typeof checks)[number] type Check = (typeof checks)[number]
type ExceptionKind = 'mark-model' | 'pending' | 'unspellable' type ExceptionKind = 'mark-model' | 'pending'
type SpecExample = { example: number; html: string; markdown: string; section: string } type SpecExample = { example: number; html: string; markdown: string; section: string }
@@ -30,7 +30,7 @@ function isCheck(value: unknown): value is Check {
} }
function isKind(value: unknown): value is ExceptionKind { function isKind(value: unknown): value is ExceptionKind {
return value === 'mark-model' || value === 'pending' || value === 'unspellable' return value === 'mark-model' || value === 'pending'
} }
function isSpecExample(value: unknown): value is SpecExample { function isSpecExample(value: unknown): value is SpecExample {
@@ -77,12 +77,12 @@ test('the CommonMark spec suite is 0.31.2, vendored byte-exact', () => {
assert.equal(digest, 'd431b29d97b6f73e69d547109cf5081578fac931e72afe95639ebe766c1b2a20') assert.equal(digest, 'd431b29d97b6f73e69d547109cf5081578fac931e72afe95639ebe766c1b2a20')
}) })
test('every exception is unique, names a parsing example, and files a fixpoint only as unspellable', () => { test('every exception is unique, names a parsing example, and files no fixpoint divergence', () => {
assert.equal(exceptionIndex.size, exceptions.length, 'one exception repeats an example and check another holds') assert.equal(exceptionIndex.size, exceptions.length, 'one exception repeats an example and check another holds')
for (const entry of exceptions) { for (const entry of exceptions) {
assert.ok(spec.some((candidate) => candidate.example === entry.example), `exception ${entry.example} names no example in the suite`) assert.ok(spec.some((candidate) => candidate.example === entry.example), `exception ${entry.example} names no example in the suite`)
assert.equal(exampleToRefusal.get(entry.example), undefined, `exception ${entry.example} is on the refusal list, not an exception`) assert.equal(exampleToRefusal.get(entry.example), undefined, `exception ${entry.example} is on the refusal list, not an exception`)
if (entry.check === 'fixpoint') assert.equal(entry.kind, 'unspellable', `exception ${entry.example} files a fixpoint divergence as ${entry.kind}; a fixable hole is given the spelling instead`) assert.notEqual(entry.check, 'fixpoint', `exception ${entry.example} files a fixpoint divergence; the emitter spells every document the parser builds`)
} }
}) })
+2 -2
View File
@@ -418,7 +418,7 @@ function holdsDirectiveShape(document: AdfDocument): boolean {
return false return false
} }
test('generated markdown refuses, or what it parses to refuses to emit, or its spelling reads back and spells itself', { timeout: propertyTimeout }, () => { test('generated markdown refuses, or its spelling reads back and spells itself', { timeout: propertyTimeout }, () => {
const parameters = propertyRuns(gateRuns) const parameters = propertyRuns(gateRuns)
let directiveShaped = 0 let directiveShaped = 0
let fixpoints = 0 let fixpoints = 0
@@ -427,7 +427,7 @@ test('generated markdown refuses, or what it parses to refuses to emit, or its s
const parsed = markdownToAdf(input) const parsed = markdownToAdf(input)
if (!parsed.ok) return if (!parsed.ok) return
const emitted = adfToMarkdown(parsed.value) const emitted = adfToMarkdown(parsed.value)
if (!emitted.ok) return assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message} — spelling ${JSON.stringify(input)}`)
fixpoints += 1 fixpoints += 1
if (holdsDirectiveShape(parsed.value)) directiveShaped += 1 if (holdsDirectiveShape(parsed.value)) directiveShaped += 1
const read = markdownToAdf(emitted.value) const read = markdownToAdf(emitted.value)
+2 -2
View File
@@ -24,7 +24,7 @@ export const propertyTimeout = 600000
const depthIdentifier = fc.createDepthIdentifier() const depthIdentifier = fc.createDepthIdentifier()
const emptyCell: AdfNode = { content: [{ type: 'paragraph' }], type: 'tableCell' } const emptyCell: AdfNode = { content: [{ type: 'paragraph' }], type: 'tableCell' }
const flatCommonMarkShapeWeight = 4 const flatCommonMarkShapeWeight = 4
export const markdownPieces = fc.constantFrom(...'aZ09 \t\n!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~é\xa0🎉日ー한𠀀', '==', '[!NOTE]', '[x]', 'ab:', 'http://', directivePrefix, `${directivePrefix}a[`, `${directivePrefix}a{`) export const markdownPieces = fc.constantFrom(...'aZ09 \t\n\r\0!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~é\xa0🎉日ー한𠀀', '==', '[!NOTE]', '[x]', 'ab:', 'http://', directivePrefix, `${directivePrefix}a[`, `${directivePrefix}a{`)
const nestingCommonMarkShapeWeight = 21 const nestingCommonMarkShapeWeight = 21
const spelledTypes = new Set(['text', ...Object.keys(blockNodes), ...Object.keys(inlineNodes), ...Object.keys(markAttributes)]) const spelledTypes = new Set(['text', ...Object.keys(blockNodes), ...Object.keys(inlineNodes), ...Object.keys(markAttributes)])
@@ -106,7 +106,7 @@ const mark: Arbitrary<AdfMark> = occasionallyEmpty(fc.oneof(
)) ))
const marks = fc.uniqueArray(mark, { maxLength: 3, selector: (held) => held.type }) const marks = fc.uniqueArray(mark, { maxLength: 3, selector: (held) => held.type })
const textNode = occasionallyEmpty(fc.record({ marks, text }).map((held): AdfNode => ({ ...held, type: 'text' }))) const textNode = occasionallyEmpty(fc.record({ marks, text: fc.oneof({ arbitrary: text, weight: 19 }, { arbitrary: fc.constant(''), weight: 1 }) }).map((held): AdfNode => ({ ...held, type: 'text' })))
const backtickRunNode = fc const backtickRunNode = fc
.record({ marks: fc.oneof(fc.constant<AdfMark[]>([]), fc.constant<AdfMark[]>([{ type: 'code' }]), marks), text: fc.string({ maxLength: 6, minLength: 1, unit: fc.constantFrom('`', '``', ' ', 'a') }) }) .record({ marks: fc.oneof(fc.constant<AdfMark[]>([]), fc.constant<AdfMark[]>([{ type: 'code' }]), marks), text: fc.string({ maxLength: 6, minLength: 1, unit: fc.constantFrom('`', '``', ' ', 'a') }) })
+1 -1
View File
@@ -117,5 +117,5 @@ export function escapeUnbalanced(spelling: string, opener: string, closer: strin
} }
function spellTitle(title: string): string | undefined { function spellTitle(title: string): string | undefined {
return /[\n\r\\]/.test(title) || holdsEntityReference(title) ? undefined : ` "${title.replaceAll('"', '\\"')}"` return /[\n\r\0\\]/.test(title) || holdsEntityReference(title) ? undefined : ` "${title.replaceAll('"', '\\"')}"`
} }
+1 -1
View File
@@ -100,7 +100,7 @@ export function readSoleStringAttribute(span: DirectiveSpan, key: string): Read<
return { value: spelled.decoded } return { value: spelled.decoded }
} }
// Both directions answer alike: an inline directive never spans lines, so no content slot holds a line ending. // An inline directive never spans lines, so no content slot holds a line ending.
export function slotLineEndingFault(type: string, text: string): ConvertFault | undefined { export function slotLineEndingFault(type: string, text: string): ConvertFault | undefined {
if (!/[\n\r]/.test(text)) return undefined if (!/[\n\r]/.test(text)) return undefined
return { code: 'unspellable-whitespace', message: `the ${type} content slot holds a newline no inline directive spans` } return { code: 'unspellable-whitespace', message: `the ${type} content slot holds a newline no inline directive spans` }
+35 -24
View File
@@ -35,7 +35,7 @@ test('names the node a refusal came from, and no source the emitter never read',
const unspellable: AdfNode = { text: 'x', type: 'paragraph' } const unspellable: AdfNode = { text: 'x', type: 'paragraph' }
const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }, { content: [unspellable], type: 'listItem' }], type: 'bulletList' } const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }, { content: [unspellable], type: 'listItem' }], type: 'bulletList' }
assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }), list))), ['content', 1, 'content', 1, 'content', 0]) assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }), list))), ['content', 1, 'content', 1, 'content', 0])
assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, { type: 'text' })))), ['content', 0, 'content', 1]) assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, { text: 'x', type: 'status' })))), ['content', 0, 'content', 1])
assert.deepEqual(path(adfToMarkdown({ type: 'doc', version: 2 })), []) assert.deepEqual(path(adfToMarkdown({ type: 'doc', version: 2 })), [])
assert.equal(position(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }), list))), undefined) assert.equal(position(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }), list))), undefined)
}) })
@@ -147,8 +147,17 @@ test('carries a mark the canonical spellings cannot nest', () => {
) )
}) })
test('refuses a line whose start block parsing would claim', () => { function readsBack(doc: AdfDocument): string {
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: '```', type: 'text' })))), 'unspellable-line-start') const spelled = markdown(adfToMarkdown(doc))
assert.deepEqual(markdownToAdf(spelled), { ok: true, value: doc }, `reading ${JSON.stringify(spelled)}`)
return spelled
}
test('carries a code span whose backticks would open its line as a fence', () => {
const fence: AdfNode = { marks: [{ type: 'code' }], text: '```', type: 'text' }
const carried = '!adf:carry{json="{\\"marks\\":[{\\"type\\":\\"code\\"}],\\"text\\":\\"\\u0060\\u0060\\u0060\\",\\"type\\":\\"text\\"}"}'
assert.equal(readsBack(document(paragraph(fence))), `${carried}\n`)
assert.equal(readsBack(document(paragraph({ text: 'a', type: 'text' }, { type: 'hardBreak' }, fence, fence))), `a\\\n${carried}\`\`\`\` \`\`\` \`\`\`\`\n`)
}) })
test('escapes the delimiter row a hard break leaves opening a pipe table with no leading pipe', () => { test('escapes the delimiter row a hard break leaves opening a pipe table with no leading pipe', () => {
@@ -456,28 +465,26 @@ test('spells a list item whose marker completes a thematic break as a directive'
) )
}) })
test('refuses the characters CommonMark rewrites', () => { test('spells the characters CommonMark rewrites', () => {
assert.equal( assert.equal(readsBack(document(paragraph({ text: 'a\r\r\nb', type: 'text' }))), 'a&#13;&#13;!adf:text{text="\\n"}b\n')
markdown(adfToMarkdown(document({ content: [{ text: 'a\rb', type: 'text' }], type: 'codeBlock' }))), assert.equal(readsBack(document(paragraph({ marks: [{ type: 'strong' }], text: '\u0000\u0000a\u0000', type: 'text' }))), '**!adf:text{text="\\u0000\\u0000"}a!adf:text{text="\\u0000"}**\n')
'unspellable-character: a codeBlock holds no carriage return CommonMark keeps: this text holds one', assert.equal(readsBack(document(paragraph({ marks: [{ attrs: { href: 'u' }, type: 'link' }], text: 'a\rb\u0000', type: 'text' }))), '[a&#13;b!adf:text{text="\\u0000"}](u)\n')
) assert.equal(readsBack(document({ attrs: { level: 1 }, content: [{ text: '\r', type: 'text' }], type: 'heading' })), '# &#13;\n')
assert.equal( assert.equal(readsBack(document({ content: [{ text: 'a\rb', type: 'text' }], type: 'codeBlock' })), '```adf:codeBlock\n{\n "content": [\n {\n "text": "a\\rb",\n "type": "text"\n }\n ]\n}\n```\n')
markdown(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), readsBack(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))
'unspellable-character: a text node holds a carriage return CommonMark rewrites', readsBack(document(paragraph({ marks: [{ type: 'code' }], text: 'a\u0000b', type: 'text' })))
)
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\u0000b', type: 'text' })))), 'unspellable-character')
assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))), 'unspellable-character')
assert.equal(markdown(adfToMarkdown(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))), 'unsupported-node-shape: a text node holds text: this one has none')
}) })
test('refuses a text node the spelling would empty out', () => { test('carries a text node holding no text, or holding content', () => {
const nested: AdfNode[] = [{ text: 'lost', type: 'text' }] const nested: AdfNode[] = [{ text: 'lost', type: 'text' }]
const empty = 'unsupported-node-shape: a text node holds text: this one has none' assert.equal(readsBack(document(paragraph({ text: '', type: 'text' }))), '!adf:carry{json="{\\"text\\":\\"\\",\\"type\\":\\"text\\"}"}\n')
const holding = 'unsupported-node-shape: a text node holds no content: this one holds some' assert.equal(readsBack(document(paragraph({ type: 'text' }, { text: 'a', type: 'text' }))), '!adf:carry{json="{\\"type\\":\\"text\\"}"}a\n')
assert.equal(markdown(adfToMarkdown(document(paragraph({ text: '', type: 'text' })))), empty) readsBack(document(paragraph({ marks: [{ type: 'code' }], text: '', type: 'text' })))
assert.equal(markdown(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: '', type: 'text' })))), empty) readsBack(document(paragraph({ marks: [{ type: 'em' }], text: 'a', type: 'text' }, { marks: [{ type: 'em' }], text: '', type: 'text' })))
assert.equal(markdown(adfToMarkdown(document(paragraph({ content: nested, text: 'x', type: 'text' })))), holding) readsBack(document(paragraph({ content: nested, text: 'x', type: 'text' })))
assert.equal(markdown(adfToMarkdown(document(paragraph({ content: nested, marks: [{ type: 'code' }], text: 'x', type: 'text' })))), holding) readsBack(document(paragraph({ content: nested, marks: [{ type: 'code' }], text: 'x', type: 'text' })))
readsBack(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))
readsBack(document({ content: [{ text: 'a', type: 'text' }, { type: 'text' }], type: 'codeBlock' }))
}) })
test('carries a mark run whose edge holds whitespace CommonMark flanking counts', () => { test('carries a mark run whose edge holds whitespace CommonMark flanking counts', () => {
@@ -688,8 +695,12 @@ test('refuses the content and slot an inline directive has no room for', () => {
assert.equal(refused({ text: 'x', type: 'status' }), neither('status', 'text')) assert.equal(refused({ text: 'x', type: 'status' }), neither('status', 'text'))
assert.equal(refused({ content: [{ text: 'x', type: 'text' }], type: 'hardBreak' }), neither('hardBreak', 'content')) assert.equal(refused({ content: [{ text: 'x', type: 'text' }], type: 'hardBreak' }), neither('hardBreak', 'content'))
assert.equal(refused({ text: 'x', type: 'hardBreak' }), neither('hardBreak', 'text')) assert.equal(refused({ text: 'x', type: 'hardBreak' }), neither('hardBreak', 'text'))
assert.equal(refused({ attrs: { text: 'a\nb' }, type: 'status' }), 'unspellable-whitespace: the status content slot holds a newline no inline directive spans') })
assert.equal(refused({ attrs: { text: 'a\u0000b' }, type: 'status' }), 'unspellable-character: a status content slot holds a null character CommonMark replaces')
test('carries an inline node whose content slot holds a line ending or a null character', () => {
assert.equal(readsBack(document(paragraph({ attrs: { text: 'a\nb' }, type: 'status' }))), '!adf:carry{json="{\\"attrs\\":{\\"text\\":\\"a\\\\nb\\"},\\"type\\":\\"status\\"}"}\n')
readsBack(document(paragraph({ attrs: { text: 'a\rb' }, type: 'emoji' })))
readsBack(document(paragraph({ attrs: { id: 'x', text: 'a\u0000b' }, type: 'mention' })))
}) })
test('spells the directive marks around the longest run they cover', () => { test('spells the directive marks around the longest run they cover', () => {
+15 -22
View File
@@ -128,9 +128,9 @@ function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
if (plain !== undefined) return plain if (plain !== undefined) return plain
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing) if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing) if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
if (node.type === 'codeBlock') return tryCodeBlock(node, path) if (node.type === 'codeBlock') return readableText(tryCodeBlock(node))
if (node.type === 'heading') return tryHeading(node, path, writing.flavour) if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
if (node.type === 'mediaSingle') return readableText(tryImage(node, path)) if (node.type === 'mediaSingle') return readableText(tryImage(node))
if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour) if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
if (node.type === 'rule') return readableText(tryRule(node)) if (node.type === 'rule') return readableText(tryRule(node))
if (node.type === 'table') return readableText(tryPipeTable(node, path, writing.flavour)) if (node.type === 'table') return readableText(tryPipeTable(node, path, writing.flavour))
@@ -251,43 +251,36 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri
return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom)) return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom))
} }
function tryCodeBlock(node: AdfNode, path: ConvertErrorPath): Result<EmittedBlock> | undefined { function tryCodeBlock(node: AdfNode): string | undefined {
if (!holdsOnlyAttributes(node, ['language'])) return undefined if (!holdsOnlyAttributes(node, ['language'])) return undefined
const slot = languageSlot(nodeAttrs(node)['language']) const slot = languageSlot(nodeAttrs(node)['language'])
if (slot.kind === 'attribute') return undefined if (slot.kind === 'attribute') return undefined
const texts = fencedTexts(node, path) const [only, ...others] = fencedTexts(node) ?? []
if (!texts.ok) return texts
const [only, ...others] = texts.value ?? []
if (only === undefined || others.length > 0) return undefined if (only === undefined || others.length > 0) return undefined
return success(commonMarkText(fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only))) return fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only)
} }
// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does. // spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does.
function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number): Result<EmittedBlock> { function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
const texts = fencedTexts(node, path) const texts = fencedTexts(node)
if (!texts.ok) return texts if (texts === undefined) return commonMarkLine(carriedBlock(node, path, depth))
if (texts.value === undefined) return commonMarkLine(carriedBlock(node, path, depth)) const slot: LanguageSlot = texts.length === 0 ? { kind: 'attribute' } : languageSlot(nodeAttrs(node)['language'])
const slot: LanguageSlot = texts.value.length === 0 ? { kind: 'attribute' } : languageSlot(nodeAttrs(node)['language'])
const opener = spellBlockDirectiveOpener(node, model, path, slot.kind === 'attribute' ? [] : ['language']) const opener = spellBlockDirectiveOpener(node, model, path, slot.kind === 'attribute' ? [] : ['language'])
if (opener === undefined) return commonMarkLine(carriedBlock(node, path, depth)) if (opener === undefined) return commonMarkLine(carriedBlock(node, path, depth))
if (!opener.ok) return opener if (!opener.ok) return opener
const info = slot.kind === 'fence' ? slot.info : '' const info = slot.kind === 'fence' ? slot.info : ''
return success(directivePair(node, opener.value, texts.value.map((text) => fencedCodeBlock(info, text)).join('\n'))) return success(directivePair(node, opener.value, texts.map((text) => fencedCodeBlock(info, text)).join('\n')))
} }
// The text each fence holds, or `undefined` where a child is no plain text node, which the carry holds instead. // The text each fence holds, or `undefined` where a child is no text a fence keeps, which the carry holds instead.
function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result<string[] | undefined> { function fencedTexts(node: AdfNode): string[] | undefined {
if (node.content === undefined) return success(['']) if (node.content === undefined) return ['']
if (node.content.some((child) => !isUnmarkedBareText(child))) return success(undefined)
const texts: string[] = [] const texts: string[] = []
for (const [index, child] of node.content.entries()) { for (const child of node.content) {
const childPath = [...path, 'content', index] if (!isUnmarkedBareText(child) || typeof child.text !== 'string' || child.text === '' || /\r/.test(child.text) || holdsNullCharacter(child.text)) return undefined
if (typeof child.text !== 'string' || child.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', childPath)
if (/\r/.test(child.text)) return failure('unspellable-character', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath)
if (holdsNullCharacter(child.text)) return failure('unspellable-character', 'a codeBlock holds a null character CommonMark replaces', childPath)
texts.push(child.text) texts.push(child.text)
} }
return success(texts) return texts
} }
function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result<EmittedBlock> | undefined { function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result<EmittedBlock> | undefined {
+2 -3
View File
@@ -1,5 +1,4 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfNode } from '../../adf/document.ts'
import type { ConvertErrorPath } from '../../result.ts'
import { holdsOnlyAttributes, nodeAttrs, nodeContent } from '../../adf/document.ts' import { holdsOnlyAttributes, nodeAttrs, nodeContent } from '../../adf/document.ts'
import { serializeCanonicalJson } from '../../canonical-json.ts' import { serializeCanonicalJson } from '../../canonical-json.ts'
import { tryImageLine } from './inline-line.ts' import { tryImageLine } from './inline-line.ts'
@@ -7,10 +6,10 @@ import { tryImageLine } from './inline-line.ts'
const centeredMediaSingle = '{"layout":"center"}' const centeredMediaSingle = '{"layout":"center"}'
const imageAttributes = ['alt', 'type', 'url'] const imageAttributes = ['alt', 'type', 'url']
export function tryImage(node: AdfNode, path: ConvertErrorPath): string | undefined { export function tryImage(node: AdfNode): string | undefined {
const image = imageShape(node) const image = imageShape(node)
if (image === undefined) return undefined if (image === undefined) return undefined
return tryImageLine(image.alt, image.url, [...path, 'content', 0]) return tryImageLine(image.alt, image.url)
} }
function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined {
+39 -44
View File
@@ -8,7 +8,7 @@ import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark
import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts'
import { identicalMark, isBareText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { identicalMark, isBareText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { highlightDelimiter } from '../plain/conventions.ts' import { highlightDelimiter } from '../plain/conventions.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts' import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts'
@@ -67,13 +67,12 @@ export function tryPipeCell(nodes: readonly AdfNode[], path: ConvertErrorPath, f
return emitted.value.line return emitted.value.line
} }
export function tryImageLine(alt: string | undefined, href: string, path: ConvertErrorPath): string | undefined { export function tryImageLine(alt: string | undefined, href: string): string | undefined {
if (alt !== undefined && (/^[ \t]|[ \t]$|[\n\r]/.test(alt) || holdsNullCharacter(alt))) return undefined if (alt !== undefined && (/^[ \t]|[ \t]$|[\n\r]/.test(alt) || holdsNullCharacter(alt))) return undefined
const destination = spellDestination(href) const destination = spellDestination(href)
if (destination === undefined) return undefined if (destination === undefined) return undefined
const description: InlineSegment[] = alt === undefined ? [] : [{ escaping: 'bracketed', text: alt }] const description: InlineSegment[] = alt === undefined ? [] : [{ escaping: 'bracketed', text: alt }]
const attempt = attemptLine([syntax('!['), ...description, syntax(`](${destination})`)], 'paragraph', path, 'lossless') return attemptLine([syntax('!['), ...description, syntax(`](${destination})`)], 'paragraph', 'lossless').line
return attempt.ok ? attempt.value.line : undefined
} }
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result<EmittedLine> { function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result<EmittedLine> {
@@ -87,12 +86,11 @@ function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: Con
if (!taken.ok) return taken if (!taken.ok) return taken
continue continue
} }
const attempt = attemptLine(emission.value.segments, container, path, flavour) const attempt = attemptLine(emission.value.segments, container, flavour)
if (!attempt.ok) return attempt if (attempt.line !== undefined) {
if (attempt.value.line !== undefined) { return success({ line: attempt.line, openingLinkAsDirective: fallbacks.openingLinkAsDirective, segments: emission.value.segments })
return success({ line: attempt.value.line, openingLinkAsDirective: fallbacks.openingLinkAsDirective, segments: emission.value.segments })
} }
const taken = takeFallback(fallbacks, attempt.value.fallback, path) const taken = takeFallback(fallbacks, attempt.fallback, path)
if (!taken.ok) return taken if (!taken.ok) return taken
} }
} }
@@ -117,12 +115,23 @@ function lineSegments(nodes: readonly AdfNode[], container: LineContainer, path:
return success({ segments: spellEdgeWhitespace(emission.value.segments) }) return success({ segments: spellEdgeWhitespace(emission.value.segments) })
} }
function attemptLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result<LineAttempt> { function attemptLine(segments: readonly InlineSegment[], container: LineContainer, flavour: Flavour): LineAttempt {
const verdict = lineVerdict(segments, container, flavour) const verdict = lineVerdict(segments, container, flavour)
if (verdict.kind === 'opening-link') return success({ fallback: 'opening-link' }) if (verdict.kind === 'opening-link') return { fallback: 'opening-link' }
if (verdict.kind === 'unspellable-run') return success({ fallback: verdict.runs[0] }) if (verdict.kind === 'unspellable-run') return { fallback: verdict.runs[0] }
if (verdict.kind === 'claimed-line') return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(verdict.text)}`, path) if (verdict.kind === 'line') return { line: verdict.text }
return success({ line: verdict.text }) return { fallback: lineOpener(segments, verdict.line) }
}
// A code span's backticks are what block parsing claims, and its segment names its node; a segment naming none leaves takeFallback to refuse the line.
function lineOpener(segments: readonly InlineSegment[], line: number): NodeRange {
let breaks = 0
const opener = segments.find((segment) => {
const opens = breaks === line && segment.text !== ''
breaks += segment.text.split('\n').length - 1
return opens
})
return opener?.nodes ?? { first: 0, last: -1 }
} }
// The fallbacks in the order a line takes them, or the line where it takes none. // The fallbacks in the order a line takes them, or the line where it takes none.
@@ -238,7 +247,7 @@ function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result<
const types = nodeMarks(node).map((mark) => mark.type) const types = nodeMarks(node).map((mark) => mark.type)
if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path) if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path)
const model = inlineNodeModel(node.type) const model = inlineNodeModel(node.type)
if (model === undefined) return emitText(node, context, index, path) if (model === undefined) return success(emitText(node, context, index))
if (node.type === 'hardBreak') return emitHardBreak(node, model, context, index, path) if (node.type === 'hardBreak') return emitHardBreak(node, model, context, index, path)
return emitInlineDirective(node, model, index, path) return emitInlineDirective(node, model, index, path)
} }
@@ -260,38 +269,27 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe
const slot = model.textAttribute === undefined ? undefined : nodeAttrs(node)[model.textAttribute] const slot = model.textAttribute === undefined ? undefined : nodeAttrs(node)[model.textAttribute]
if (slot === undefined) return success({ segments: [syntax(spellInlineLeafDirective(node.type, attributes))] }) if (slot === undefined) return success({ segments: [syntax(spellInlineLeafDirective(node.type, attributes))] })
if (typeof slot !== 'string') return success({ carry: { first: index, last: index } }) if (typeof slot !== 'string') return success({ carry: { first: index, last: index } })
const spans = slotLineEndingFault(node.type, slot) if (slotLineEndingFault(node.type, slot) !== undefined || holdsNullCharacter(slot)) return success({ carry: { first: index, last: index } })
if (spans !== undefined) return faulted(spans, path)
if (holdsNullCharacter(slot)) return failure('unspellable-character', `a ${node.type} content slot holds a null character CommonMark replaces`, path)
const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }] const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }]
return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }) return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] })
} }
function textHoldingContent(node: AdfNode, path: ConvertErrorPath): Result<never> | undefined { // spec/flavour.md, Inline nodes: CommonMark reads a carriage return as a line ending, and a null character as U+FFFD.
return node.type === 'text' && nodeContent(node).length > 0 ? failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) : undefined function emitText(node: AdfNode, context: InlineContext, index: number): Emission {
} if (!isBareText(node) || typeof node.text !== 'string' || node.text === '') return { carry: { first: index, last: index } }
function emitText(node: AdfNode, context: InlineContext, index: number, path: ConvertErrorPath): Result<Emission> {
const holding = textHoldingContent(node, path)
if (holding !== undefined) return holding
if (!isBareText(node)) return success({ carry: { first: index, last: index } })
if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path)
if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path)
if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path)
const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash' const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash'
const parts = node.text.split(/(\n+)/).filter((part) => part !== '') const parts = node.text.split(/(\n+|\0+|\r)/).filter((part) => part !== '')
return success({ segments: parts.map((part) => (part.startsWith('\n') ? textDirectiveSegment(part) : { escaping, text: part })) }) return { segments: parts.map((part) => (part === '\r' ? syntax('&#13;') : /^[\n\0]/.test(part) ? textDirectiveSegment(part) : { escaping, text: part })) }
} }
function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result<Emission> { function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result<Emission> {
const path = nodePath(context, index)
const range: NodeRange = { first: index, last: index + nodes.length - 1 } const range: NodeRange = { first: index, last: index + nodes.length - 1 }
if (mark.type === 'backgroundColor' && context.flavour === 'plain') return emitHighlight(nodes, depth, range, context) if (mark.type === 'backgroundColor' && context.flavour === 'plain') return emitHighlight(nodes, depth, range, context)
const spelling = markSpelling(mark.type) const spelling = markSpelling(mark.type)
if (spelling === undefined) return success({ carry: range }) if (spelling === undefined) return success({ carry: range })
const attributes = spellMarkAttributes(mark, spelling) const attributes = spellMarkAttributes(mark, spelling)
if (attributes === undefined) return success({ carry: range }) if (attributes === undefined) return success({ carry: range })
if (spelling.kind === 'code') return emitCodeSpan(nodes, depth, range, path) if (spelling.kind === 'code') return success(emitCodeSpan(nodes, depth, range))
if (spelling.kind === 'emphasis') return emitEmphasis(nodes, spelling.spelling, depth, range, context) if (spelling.kind === 'emphasis') return emitEmphasis(nodes, spelling.spelling, depth, range, context)
const link = spelling.kind === 'link' ? tryLink(nodes, mark, depth, range, context) : undefined const link = spelling.kind === 'link' ? tryLink(nodes, mark, depth, range, context) : undefined
if (link !== undefined) return link if (link !== undefined) return link
@@ -325,20 +323,17 @@ function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRang
} }
// Each node its own span: CommonMark reads two adjacent text nodes in one span back as one. // Each node its own span: CommonMark reads two adjacent text nodes in one span back as one.
function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result<Emission> { function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange): Emission {
const spans: string[] = [] const segments: InlineSegment[] = []
for (const node of nodes) { for (const [offset, node] of nodes.entries()) {
const holding = textHoldingContent(node, path)
if (holding !== undefined) return holding
if (!isBareText(node) || nodeMarks(node).length !== depth + 1) return success({ carry: range })
const { text } = node const { text } = node
if (typeof text !== 'string' || text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (!isBareText(node) || nodeMarks(node).length !== depth + 1 || typeof text !== 'string' || text === '' || /[\n\r]/.test(text) || holdsNullCharacter(text)) return { carry: range }
if (/[\n\r]/.test(text)) return success({ carry: range }) if (offset > 0) segments.push(syntax(textBreakSpelling))
if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path)
const fence = '`'.repeat(longestBacktickRun(text) + 1) const fence = '`'.repeat(longestBacktickRun(text) + 1)
spans.push(`${fence}${needsPadding(text) ? ` ${text} ` : text}${fence}`) const index = range.first + offset
segments.push({ escaping: 'none', nodes: { first: index, last: index }, text: `${fence}${needsPadding(text) ? ` ${text} ` : text}${fence}` })
} }
return success({ segments: [syntax(spans.join(textBreakSpelling))] }) return { segments }
} }
function needsPadding(text: string): boolean { function needsPadding(text: string): boolean {
+8 -5
View File
@@ -304,10 +304,8 @@ test('names the directive form a node CommonMark spells refuses', () => {
assert.deepEqual(content(markdownToAdf('!adf:bulletList\n!adf:listItem\n---\n!adf:/listItem\n!adf:/bulletList\n')), [bulletList(item({ type: 'rule' }))]) assert.deepEqual(content(markdownToAdf('!adf:bulletList\n!adf:listItem\n---\n!adf:/listItem\n!adf:/bulletList\n')), [bulletList(item({ type: 'rule' }))])
}) })
// The spelling the emitter refuses gives the emitter's own error, never a second name for it. test('refuses the directive form of a blockquote opening with a code span the carry spells', () => {
test('gives back the refusal the CommonMark spelling itself raises', () => { assert.equal(content(markdownToAdf('!adf:blockquote\n` `` `\n!adf:/blockquote\n')), 'unsupported-node-shape: blockquote takes the CommonMark spelling, not the directive form')
const lineStart = '!adf:blockquote\n` `` `\n!adf:/blockquote\n'
assert.equal(content(markdownToAdf(lineStart)), 'unspellable-line-start: block parsing would claim the emitted line "``` `` ```"')
}) })
test('names the directive name no node reads back to', () => { test('names the directive name no node reads back to', () => {
@@ -1010,6 +1008,11 @@ test('names the content slot no lone plain text node reads back from', () => {
assert.equal(content(markdownToAdf('Part !adf:mention{id=b1c2 text=A}.\n')), 'unsupported-node-shape: mention spells its text attribute in the content slot, never in {attrs}') assert.equal(content(markdownToAdf('Part !adf:mention{id=b1c2 text=A}.\n')), 'unsupported-node-shape: mention spells its text attribute in the content slot, never in {attrs}')
}) })
test('reads the characters CommonMark rewrites from their spellings', () => {
assert.deepEqual(content(markdownToAdf('a&#13;b!adf:text{text="\\u0000\\u0000"}\n')), [paragraph('a\rb\u0000\u0000')])
assert.equal(content(markdownToAdf('!adf:text{text="\\u0000a"}\n')), 'unsupported-node-shape: text spells one run of spaces and tabs, one run of newlines, or one run of null characters: this one spells none of them')
})
test('reads the whitespace the reserved text directive carries', () => { test('reads the whitespace the reserved text directive carries', () => {
assert.deepEqual(content(markdownToAdf('!adf:text{text=" "}a\n')), [paragraph(' a')]) assert.deepEqual(content(markdownToAdf('!adf:text{text=" "}a\n')), [paragraph(' a')])
assert.deepEqual(content(markdownToAdf('a!adf:text{text="\\n"}b\n')), [paragraph('a\nb')]) assert.deepEqual(content(markdownToAdf('a!adf:text{text="\\n"}b\n')), [paragraph('a\nb')])
@@ -1018,7 +1021,7 @@ test('reads the whitespace the reserved text directive carries', () => {
}) })
test('names the text directive spelling no whitespace run reads back from', () => { test('names the text directive spelling no whitespace run reads back from', () => {
const named = 'unsupported-node-shape: text spells one run of spaces and tabs, or one run of newlines: this one spells neither' const named = 'unsupported-node-shape: text spells one run of spaces and tabs, one run of newlines, or one run of null characters: this one spells none of them'
assert.equal(content(markdownToAdf('!adf:text{text=hi}\n')), named) assert.equal(content(markdownToAdf('!adf:text{text=hi}\n')), named)
assert.equal(content(markdownToAdf('!adf:text{text=" \\n"}\n')), named) assert.equal(content(markdownToAdf('!adf:text{text=" \\n"}\n')), named)
assert.equal(content(markdownToAdf('!adf:text{text=""}\n')), named) assert.equal(content(markdownToAdf('!adf:text{text=""}\n')), named)
+2 -2
View File
@@ -2,7 +2,7 @@ import type { DirectiveSpan, Read } from './directive-syntax.ts'
import { readSoleStringAttribute, spellAttributes, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' import { readSoleStringAttribute, spellAttributes, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts'
const name = 'text' const name = 'text'
const whitespaceRun = /^(?:[ \t]+|\n+)$/ const spelledRun = /^(?:[ \t]+|\n+|\0+)$/
export const textDirectiveName = name export const textDirectiveName = name
@@ -14,6 +14,6 @@ export function readTextDirective(span: DirectiveSpan): Read<string> | undefined
if (span.name !== name) return undefined if (span.name !== name) return undefined
const spelled = readSoleStringAttribute(span, name) const spelled = readSoleStringAttribute(span, name)
if (spelled.fault !== undefined) return spelled if (spelled.fault !== undefined) return spelled
if (!whitespaceRun.test(spelled.value)) return { fault: unsupportedNodeShape(`${name} spells one run of spaces and tabs, or one run of newlines: this one spells neither`) } if (!spelledRun.test(spelled.value)) return { fault: unsupportedNodeShape(`${name} spells one run of spaces and tabs, one run of newlines, or one run of null characters: this one spells none of them`) }
return spelled return spelled
} }
-2
View File
@@ -5,8 +5,6 @@ export type ConvertErrorCode =
| 'unknown-directive-name' | 'unknown-directive-name'
| 'unmappable-html' | 'unmappable-html'
| 'unmappable-image' | 'unmappable-image'
| 'unspellable-character'
| 'unspellable-line-start'
| 'unspellable-whitespace' | 'unspellable-whitespace'
| 'unsupported-document-version' | 'unsupported-document-version'
| 'unsupported-nesting-depth' | 'unsupported-nesting-depth'
-19
View File
@@ -25,7 +25,6 @@
| ID | Release | Exempt | Item | R | S | A | G | Goals | Score | | ID | Release | Exempt | Item | R | S | A | G | Goals | Score |
|---|---|---|---|---|---|---|---|---|---| |---|---|---|---|---|---|---|---|---|---|
| 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 |
| 55 | 0.2.0 | defect | **Spell every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node whose text is empty or missing, or which holds content.** | 5 | 5 | 7 | 9 | 1 | 25.8 |
| 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`.** | 2 | 3 | 6 | 8 | 1 | 25.2 |
| 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 |
| 64 | 0.2.0 | defect | **Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 | | 64 | 0.2.0 | defect | **Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 |
@@ -63,24 +62,6 @@ Lands after items 6, 59, 60, 61 and 62. The CommonMark spec suite also runs agai
`markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and `markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and
`package.json`'s `description` regain HTML. `package.json`'s `description` regain HTML.
### 55. Spell every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node whose text is empty or missing, or which holds content.
Today `inline-line.ts` refuses a carriage return or NUL in text, and a NUL in an emoji, mention or
status (`unspellable-character`); and a paragraph opening with a code-span run
(`unspellable-line-start`); `fencedTexts` refuses the same characters in a code block;
`inline-line.ts` refuses a newline in an emoji, mention or status (`unspellable-whitespace`);
`inline-line.ts` and `fencedTexts` refuse a text node whose `text` is empty or missing, and
`inline-line.ts` one holding `content`. In text, a carriage return is CommonMark's `&#13;` (Goal 4),
and a NUL is `!adf:text{text="\u0000"}` holding just the NUL, since CommonMark reads `&#0;` as
U+FFFD (a writer panel, 3 of 3, 2026-10-04). The carries spell every other case: the inline carry in
a paragraph, the block carry in a code block.
The maintainer approved on 2026-10-04: §Markdown in is a canonical fixpoint drops its code-span
exception, §Which code a cause takes drops `unspellable-whitespace` as an emitter cause;
`unspellable-character` and `unspellable-line-start` retire. `unspellable-whitespace` stays, since
the reader raises it for a content slot spanning a newline. Found by the README-goals audit,
2026-10-03.
### 45. Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`. ### 45. Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`.
Goal 1 has every call return a result; the boolean guard is the one export that does not, and it Goal 1 has every call return a result; the boolean guard is the one export that does not, and it