5b3: the code list and the flavour's gaps #46

Merged
lilleman merged 4 commits from code-list into main 2026-09-03 19:33:52 +02:00
24 changed files with 506 additions and 82 deletions
+21 -5
View File
@@ -97,13 +97,25 @@ names the escape that unclaims the form claimed: `\:::` for a directive line, `\
row, `\:` for an inline directive.
Adding, removing or renaming a code is breaking, so a milestone meeting a new failure cause
reuses a code where one fits; the list is complete at `0.1.0`. A code names the
cause; where one cause recurs across node types or across directions, one code covers them all and
cause; where one cause recurs across node types, across one mark's attributes or across
directions, one code covers them all and
`path` and `message` say which — `unsupported-nesting-depth` is the 500-level guard whichever
direction hits it. A claim code names the spelling claimed, never the node that spelling would have built:
a malformed `:::table` is a `malformed-directive`. A cause the carry answers gets no code: a mark no
direction hits it, `unspellable-link` the destination and the title alike. Where two codes stay
apart, the line between them is what they name: `unspellable-character` is a character CommonMark
rewrites wherever text holds it, `unspellable-whitespace` the newline no inline directive's
content slot spans, in either direction. A claim code names the spelling claimed, never the node that spelling would have built:
a malformed `:::table` is a `malformed-directive`, and an alignment colon a `malformed-pipe-table`
the flavour's own delimiter row is `-` runs, so the grammar refuses the colon rather than ADF's
missing column model doing it. A refusal no spelling recovers from is a gap in the flavour rather
than a code: give the flavour the spelling and the code goes, which the freeze is the last moment
for. A cause the carry answers gets no code: a mark no
spelling writes rides the carry with its node. A directive whose name reads back to no node is
`unknown-directive-name` rather than a claim code — the spelling is well formed, and telling that
apart from a typo is what a consumer switches on when a later MINOR gives the name meaning. What
apart from a typo is what a consumer switches on when a later MINOR gives the name meaning. A
reserved name is a known name, so never that code, and the two the flavour reserves part on form:
a form the grammar does not have is a claim code — `::adf`, whose carry is the fence — and a
well-formed form in the wrong place is `unsupported-node-shape`, `::listBreak` parting anything
but two adjacent lists of one type. What
the grammar itself refuses stays a claim code, key order among it; a well-formed directive the
node tables refuse — an attribute a node does not hold or spells elsewhere, a value outside its
kind or its canonical spelling, an argument or a body its content model does not take — is
@@ -113,7 +125,11 @@ them after `0.1.0` is MAJOR. `unmappable-html` names the version rather than the
converts no raw HTML, so at `0.3.0` the mapped elements stop erroring and the code stays for what
no ADF node carries. A refusal found before its path is known — the block walk's, a directive
reader's — is a `ConvertFault`, the code and message alone; the node walk attaches the path as it
descends, so a document reports its first error in document order.
descends, so a document reports its first error in document order. `not-an-adf-document` carries
the document's own path throughout: eight of the guard's nine branches read the document's own
shape, and threading a path to the ninth — a malformed node anywhere in the tree — wants the
manual stack §11's no-recursion rule forces, whose empty half no input reaches. The message names
the violation instead.
`position` is the parse side's alone: an emitter reads no source, so an emit error carries `path`
and nothing more. It is `{ line, offset }` at the start of the line the block holding the refusal
+1
View File
@@ -0,0 +1 @@
unsupported-node-shape
+5
View File
@@ -0,0 +1,5 @@
- Bolt M8
::listBreak
Packed.
+1
View File
@@ -0,0 +1 @@
malformed-pipe-table
+3
View File
@@ -0,0 +1,3 @@
Part | Qty
--- | ---
Bolt M8 | 40
@@ -0,0 +1,182 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Bolt",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
},
{
"content": [
{
"content": [
{
"content": [
{
"text": "Nut",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
},
{
"attrs": {
"order": 1
},
"content": [
{
"content": [
{
"content": [
{
"text": "Cut",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
},
{
"attrs": {
"order": 1
},
"content": [
{
"content": [
{
"content": [
{
"text": "Weld",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
},
{
"attrs": {
"panelType": "info"
},
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Left",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
},
{
"content": [
{
"content": [
{
"content": [
{
"text": "Right",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "panel"
},
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Head",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
},
{
"content": [
{
"content": [
{
"content": [
{
"text": "Tail",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,23 @@
- Bolt
::listBreak
- Nut
1. Cut
::listBreak
1. Weld
:::panel info
- Left
::listBreak
- Right
:::
- - Head
::listBreak
- Tail
@@ -0,0 +1,38 @@
{
"content": [
{
"content": [
{
"text": "Part | Qty",
"type": "text"
},
{
"type": "hardBreak"
},
{
"text": "--- | ---",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Part | Qty",
"type": "text"
},
{
"type": "hardBreak"
},
{
"text": ":--- | ---:",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,5 @@
Part | Qty\
\--- | ---
Part | Qty\
\:--- | ---:
+15 -7
View File
@@ -2,7 +2,7 @@
The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` parses. Plain
CommonMark is a subset with three carve-outs: literal text that matches directive syntax below or
opens a pipe table is claimed by the flavour, and a matched `~~` pair spells `strike` (escape the
reads as a pipe table is claimed by the flavour, and a matched `~~` pair spells `strike` (escape the
`:`, `|` or `~` to keep it literal) — and one gap: a CommonMark image fits only as its own
title-less paragraph — mid-text and titled images are named errors. The emitted form is contract
(AGENTS.md §8). Per-node syntaxes build on this grammar in the sections below.
@@ -23,8 +23,11 @@ normalizes to it through the round-trip.
marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or a list
whose first item is empty), whatever block sits above it. Blank lines between items normalize
away, and no list opens beside one of its own kind — the marker change CommonMark starts a
second list on merges instead: ADF records no tightness, and one `- ` spelling leaves two
adjacent lists of a kind no way back.
second list on merges instead: ADF records no tightness, so one `- ` spelling reads two
adjacent lists of a kind back as one. The leaf `::listBreak` parts them, taking the separation
any directive block takes where it sits. It builds no node, and it reads only between two
adjacent lists of one type: elsewhere, or carrying an argument, `{attrs}` or a body, it is a
named error.
- Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`.
- ATX headings (`#``######`); setext input normalizes to ATX.
- Code fences ``` with the node's language as info string, the fence lengthened past any backtick
@@ -51,7 +54,8 @@ normalizes to it through the round-trip.
`*not emphasis*` is `\*not emphasis\*` — no delimiter the emitter did not write reaches the
matching below, which is what lets the emitter decide its own pairings.
- Blocks separated by one blank line at document level, inside a blockquote and between CommonMark
blocks; two directive blocks inside a container take none. No trailing whitespace outside a code
blocks; inside a directive container a pair holding a directive block takes none. No trailing
whitespace outside a code
block's
content, single trailing newline; a document with no blocks is the empty string.
@@ -64,8 +68,9 @@ whether the name is known, and an unknown name is an error result naming it —
emitter escaped stays escaped, and erroring input gaining meaning later is MINOR, never a reparse
(§8). Each name belongs to one position, and a name the other one spells — a mark or an inline
node written as a block directive, a block node written inline — is a different error, naming the
spelling it takes. The name `adf` is reserved for the opaque carry, as both directive name and fence info
string.
spelling it takes. Two reserved names read back to no node: `adf` for the opaque carry, as both
directive name and fence info string, and `listBreak` for the leaf that parts two adjacent lists
(Canonical form).
**Inline**: `:name[content]{attrs}`, on one line — an inline directive never spans lines.
`[content]` is inline markdown; brackets inside balance as in CommonMark link text, `\]` for a
@@ -291,7 +296,10 @@ takes the directive form instead. A pipe table parses back to exactly that shape
Claiming at block level, symmetric with directives: a line opening with an unescaped `|` is
claimed and must parse as part of a pipe table, else it is a named error — escape the pipe
(`\|`) to keep it literal text. A pipe table is a header row, a delimiter row whose cells are
(`\|`) to keep it literal text. The shape a missing leading pipe leaves is claimed too: a line
holding an unescaped `|`, followed inside one paragraph by a line whose cells are all `-` runs,
an alignment colon among them, and match it in count — GFM's table without the outer pipes — is
a named error rather than the prose it reads as. A pipe table is a header row, a delimiter row whose cells are
runs of one or more `-` (canonical `---`), and body rows; rows follow code-fence indentation.
Cells split on unescaped `|` before inline parsing — `\|` stays in the cell text, and the
inline layer's ordinary CommonMark escaping yields the pipe; each cell is the inline content of
+27 -15
View File
@@ -81,13 +81,13 @@ test('spells a code block language no info string holds as an attribute', () =>
test('refuses a link destination CommonMark cannot spell', () => {
const link = (href: string): AdfDocument => document(paragraph({ marks: [{ attrs: { href }, type: 'link' }], text: 't', type: 'text' }))
assert.equal(code(adfToMarkdown(link('https://example.com/a b>c'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('<https://example.com/'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('https://example.com/a\\b'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('https://example.com/?a=1&amp;b=2'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('https://example.com/a\nb'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('https://example.com/a b>c'))), 'unspellable-link')
assert.equal(code(adfToMarkdown(link('<https://example.com/'))), 'unspellable-link')
assert.equal(code(adfToMarkdown(link('https://example.com/a\\b'))), 'unspellable-link')
assert.equal(code(adfToMarkdown(link('https://example.com/?a=1&amp;b=2'))), 'unspellable-link')
assert.equal(code(adfToMarkdown(link('https://example.com/a\nb'))), 'unspellable-link')
const entity = 'https://example.com/?a=1&amp;b=2'
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { href: entity }, type: 'link' }], text: entity, type: 'text' })))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { href: entity }, type: 'link' }], text: entity, type: 'text' })))), 'unspellable-link')
})
test('escapes the parenthesis a link destination leaves unbalanced, and no other', () => {
@@ -103,8 +103,8 @@ test('escapes the quote a link title holds, and refuses the rest', () => {
const titled = (title: string): AdfDocument =>
document(paragraph({ marks: [{ attrs: { href: 'https://example.com/', title }, type: 'link' }], text: 't', type: 'text' }))
assert.equal(markdown(adfToMarkdown(titled('He said "hi"'))), '[t](https://example.com/ "He said \\"hi\\"")\n')
assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link-title')
assert.equal(code(adfToMarkdown(titled('a\\b'))), 'unspellable-link-title')
assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link')
assert.equal(code(adfToMarkdown(titled('a\\b'))), 'unspellable-link')
})
test('carries a link mark the link spelling cannot write', () => {
@@ -134,17 +134,25 @@ test('carries a mark the canonical spellings cannot nest', () => {
)
})
test('refuses whitespace CommonMark cannot hold', () => {
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), 'unspellable-whitespace')
})
test('refuses a line whose start block parsing would claim', () => {
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: '```', type: 'text' })))), 'unspellable-line-start')
})
test('refuses two adjacent lists of the same kind, the marker spelling being what merges', () => {
test('escapes the delimiter row a hard break leaves opening a pipe table with no leading pipe', () => {
const broken = (second: string): string => markdown(adfToMarkdown(document(paragraph({ text: 'a | b', type: 'text' }, { type: 'hardBreak' }, { text: second, type: 'text' }))))
assert.equal(broken('--- | ---'), 'a | b\\\n\\--- | ---\n')
assert.equal(broken(':--- | ---:'), 'a | b\\\n\\:--- | ---:\n')
assert.equal(broken('c | d'), 'a | b\\\nc | d\n')
})
test('parts two adjacent lists of the same kind, the marker spelling being what merges', () => {
const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'bulletList' }
assert.equal(code(adfToMarkdown(document(list, list))), 'unspellable-adjacent-lists')
assert.equal(markdown(adfToMarkdown(document(list, list))), '- x\n\n::listBreak\n\n- x\n')
const ordered: AdfNode = { attrs: { order: 1 }, content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'orderedList' }
assert.equal(markdown(adfToMarkdown(document(ordered, ordered))), '1. x\n\n::listBreak\n\n1. x\n')
assert.equal(markdown(adfToMarkdown(document({ attrs: { panelType: 'info' }, content: [list, list], type: 'panel' }))), ':::panel info\n- x\n::listBreak\n- x\n:::\n')
const nested: AdfNode = { content: [{ content: [list, list], type: 'listItem' }], type: 'bulletList' }
assert.equal(markdown(adfToMarkdown(document(nested))), '- - x\n\n ::listBreak\n\n - x\n')
const carried: AdfNode = { ...list, attrs: { unknown: 'x' } }
assert.ok(markdown(adfToMarkdown(document(carried, carried))).includes('```\n\n```adf\n'))
assert.ok(markdown(adfToMarkdown(document(carried, list))).endsWith('```\n\n- x\n'))
@@ -394,7 +402,11 @@ test('spells a list item whose marker completes a thematic break as a directive'
test('refuses the characters CommonMark rewrites', () => {
assert.equal(
markdown(adfToMarkdown(document({ content: [{ text: 'a\rb', type: 'text' }], type: 'codeBlock' }))),
'unspellable-whitespace: a codeBlock holds no carriage return CommonMark keeps: this text holds one',
'unspellable-character: a codeBlock holds no carriage return CommonMark keeps: this text holds one',
)
assert.equal(
markdown(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))),
'unspellable-character: a text node holds a carriage return CommonMark rewrites',
)
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\u0000b', type: 'text' })))), 'unspellable-character')
assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))), 'unspellable-character')
+11 -14
View File
@@ -9,6 +9,7 @@ import { fencedCodeBlock } from '../backtick-runs.ts'
import { holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts'
import { languageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts'
import { listBreakSpelling } from '../list-break.ts'
import { spellDirectiveHeader } from './block-directive-spelling.ts'
import { tryImage } from './image.ts'
import { tryPipeTable } from './pipe-table.ts'
@@ -17,7 +18,7 @@ type BlockContainer = 'directive' | 'document' | 'list-item'
type BlockSpelling = 'commonmark' | 'directive' | 'list'
type EmittedBody = { fenceColons: number; text: string }
type EmittedBlock = EmittedBody & { spelling: BlockSpelling }
type PlacedBlock = EmittedBlock & { node: AdfNode; path: ConvertErrorPath }
type PlacedBlock = EmittedBlock & { node: AdfNode }
const largestListMarker = 999999999
@@ -34,35 +35,31 @@ function emitBlocks(nodes: readonly AdfNode[], container: BlockContainer, path:
if (depth > largestNesting) return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
const blocks: PlacedBlock[] = []
for (const [index, node] of nodes.entries()) {
const nodePath = [...path, 'content', index]
const block = emitBlock(node, nodePath, depth)
const block = emitBlock(node, [...path, 'content', index], depth)
if (!block.ok) return block
blocks.push({ ...block.value, node, path: nodePath })
blocks.push({ ...block.value, node })
}
let fenceColons = 0
let text = ''
for (const [index, block] of blocks.entries()) {
const previous = blocks[index - 1]
if (previous !== undefined) {
const separation = separationBetween(previous, block, container)
if (!separation.ok) return separation
text += separation.value
}
if (previous !== undefined) text += separationBetween(previous, block, container)
fenceColons = Math.max(fenceColons, block.fenceColons)
text += block.text
}
return success({ fenceColons, text })
}
function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): Result<string> {
function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): string {
const plainPair = previous.spelling !== 'directive' && next.spelling !== 'directive'
if (plainPair && next.spelling === 'list') {
if (previous.spelling === 'list' && previous.node.type === next.node.type) {
return failure('unspellable-adjacent-lists', `two adjacent ${next.node.type} nodes read back as one list`, next.path)
const gap = container === 'directive' ? '\n' : '\n\n'
return `${gap}${listBreakSpelling}${gap}`
}
if (container === 'list-item') return success(interruptsParagraph(next.node) ? '\n' : '\n\n')
if (container === 'list-item') return interruptsParagraph(next.node) ? '\n' : '\n\n'
}
return success(container === 'directive' && !plainPair ? '\n' : '\n\n')
return container === 'directive' && !plainPair ? '\n' : '\n\n'
}
function interruptsParagraph(node: AdfNode): boolean {
@@ -179,7 +176,7 @@ function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result<string> {
) {
return failure('unsupported-node-shape', `a codeBlock holds plain text nodes only: this ${child.type} node is not one`, childPath)
}
if (/\r/.test(child.text)) return failure('unspellable-whitespace', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath)
if (/\r/.test(child.text)) return failure('unspellable-character', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath)
if (holdsNullCharacter(child.text)) return failure('unspellable-character', 'a codeBlock holds a null character CommonMark replaces', childPath)
text += child.text
}
+1 -1
View File
@@ -220,7 +220,7 @@ function emitText(node: AdfNode, context: InlineContext, index: number, path: Co
if (Object.keys(node.attrs ?? {}).length > 0) return success({ carry: { first: index, last: index } })
if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path)
if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path)
if (/\r/.test(node.text)) return failure('unspellable-whitespace', 'a text node holds a carriage return CommonMark rewrites', path)
if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path)
if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path)
const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash'
const parts = node.text.split(/(\n+)/).filter((part) => part !== '')
+5 -1
View File
@@ -1,6 +1,7 @@
import { backtickRun, closingBacktickRun } from '../backtick-runs.ts'
import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../emphasis-matching.ts'
import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark-grammar.ts'
import { isBareDelimiterRow } from '../pipe-table-syntax.ts'
import { opensInlineDirective } from '../directive-syntax.ts'
import { readEntityReference } from '../entity-references.ts'
@@ -196,8 +197,11 @@ function opensConstruct(
return claimsCharacter(scan, linkClose, index, inBrackets, container, escaped)
}
// A hard break is the one spelling that puts a delimiter row under a row of its own, so only a later line claims.
function claimsLineStart(line: ScanLine, index: number, container: LineContainer): boolean {
return container === 'paragraph' && escapesLineClaim(line.text, index - line.start, line.position)
if (container !== 'paragraph') return false
if (index === line.start && line.position === 'later' && isBareDelimiterRow(line.text)) return true
return escapesLineClaim(line.text, index - line.start, line.position)
}
function scanLine(scan: string, start: number): ScanLine {
+7 -7
View File
@@ -91,26 +91,26 @@ export function skipLinkWhitespace(text: string, offset: number): number {
}
export function spellDestination(href: string, path: ConvertErrorPath): Result<string> {
if (holdsControlCharacter(href)) return failure('unspellable-link-destination', 'a link destination holds a control character', path)
if (href.includes('\\')) return failure('unspellable-link-destination', 'no canonical escape spells a backslash in a link destination', path)
if (holdsControlCharacter(href)) return failure('unspellable-link', 'a link destination holds a control character', path)
if (href.includes('\\')) return failure('unspellable-link', 'no canonical escape spells a backslash in a link destination', path)
if (holdsEntityReference(href)) {
return failure('unspellable-link-destination', 'a link destination holds an entity reference that decodes on the way back', path)
return failure('unspellable-link', 'a link destination holds an entity reference that decodes on the way back', path)
}
if (href.includes(' ')) {
if (/[<>]/.test(href)) {
return failure('unspellable-link-destination', 'no canonical escape spells an angle bracket beside a space in a link destination', path)
return failure('unspellable-link', 'no canonical escape spells an angle bracket beside a space in a link destination', path)
}
return success(`<${href}>`)
}
if (href.startsWith('<')) return failure('unspellable-link-destination', 'a bare link destination cannot begin with an angle bracket', path)
if (href.startsWith('<')) return failure('unspellable-link', 'a bare link destination cannot begin with an angle bracket', path)
return success(escapeUnbalanced(href))
}
export function spellTitle(title: string, path: ConvertErrorPath): Result<string> {
if (/[\n\r\\]/.test(title)) {
return failure('unspellable-link-title', 'no canonical escape spells a backslash or newline in a link title', path)
return failure('unspellable-link', 'no canonical escape spells a backslash or newline in a link title', path)
}
if (holdsEntityReference(title)) return failure('unspellable-link-title', 'a link title holds an entity reference that decodes on the way back', path)
if (holdsEntityReference(title)) return failure('unspellable-link', 'a link title holds an entity reference that decodes on the way back', path)
return success(` "${title.replaceAll('"', '\\"')}"`)
}
+3
View File
@@ -0,0 +1,3 @@
export const listBreakName = 'listBreak'
export const listBreakSpelling = `::${listBreakName}`
+16 -3
View File
@@ -16,7 +16,7 @@ import {
setextHeadingLevel,
} from '../commonmark-grammar.ts'
import { directiveLineEscape, malformedDirective, readDirectiveLine } from '../directive-syntax.ts'
import { isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts'
import { barePipeCells, isDelimiterRow, isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts'
import { readLinkDefinitions } from './link-reference-definitions.ts'
export type Block = { position: SourcePosition } & (
@@ -358,7 +358,7 @@ function readLineBlock(walk: Walk, opener: string): boolean {
if (level !== undefined) {
const paragraph = takeParagraph(walk)
if (paragraph !== undefined) {
currentBlocks(walk).push({ kind: 'heading', level, position: paragraph.position, text: paragraph.text })
currentBlocks(walk).push(bareTableFault(paragraph) ?? { kind: 'heading', level, position: paragraph.position, text: paragraph.text })
return true
}
}
@@ -390,7 +390,7 @@ function closeLeaf(walk: Walk): void {
if (leaf === undefined) return
if (leaf.kind === 'paragraph') {
const paragraph = takeParagraph(walk)
if (paragraph !== undefined) currentBlocks(walk).push(paragraph)
if (paragraph !== undefined) currentBlocks(walk).push(bareTableFault(paragraph) ?? paragraph)
return
}
walk.leaf = undefined
@@ -413,6 +413,19 @@ function pipeTableBlock(rows: readonly [string[], ...string[][]], position: Sour
return { kind: 'table', position, rows: [header, ...body] }
}
// spec/flavour.md, Tables: GFM's table without the leading pipes, which no line of it claims.
function bareTableFault(paragraph: Extract<Block, { kind: 'paragraph' }>): Block | undefined {
let header: string[] | undefined
for (const line of paragraph.text.split('\n')) {
const cells = barePipeCells(line)
if (header !== undefined && cells !== undefined && cells.length === header.length && isDelimiterRow(cells)) {
return faultedBlock('a pipe table opens every row with `|`: this one does not; \\| keeps a pipe literal text', paragraph.position)
}
header = cells
}
return undefined
}
function cellCount(count: number): string {
return `${count} cell${count === 1 ? '' : 's'}`
}
+2
View File
@@ -10,6 +10,7 @@ import { carryName } from '../opaque-carry.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { inlineDirective } from '../../adf/inline-directives.ts'
import { inlineMarkSpellingFault } from './directive-marks.ts'
import { listBreakName, listBreakSpelling } from '../list-break.ts'
import { marksAttribute, readMarkValues } from '../block-directive-marks.ts'
import { readVocabulary } from './directive-attributes.ts'
import { slotLineEndingFault } from '../directive-syntax.ts'
@@ -76,6 +77,7 @@ function inlineSpellingFault(name: string): ConvertFault | undefined {
}
function blockSpellingFault(name: string): ConvertFault | undefined {
if (name === listBreakName) return { code: 'unsupported-node-shape', message: `${name} takes the block form, ${listBreakSpelling}, never the inline form` }
const directive = blockDirective(name)
if (directive === undefined) return undefined
const form = directive.contentModel === 'none' ? `::${name}` : `:::${name}`
+45 -1
View File
@@ -190,6 +190,50 @@ test('names the pipe table a claimed line does not spell', () => {
assert.deepEqual(path(markdownToAdf('Part.\n\n| a |\n')), ['content', 1])
})
test('names the pipe table whose rows open with no pipe', () => {
const bare = 'malformed-pipe-table: a pipe table opens every row with `|`: this one does not; \\| keeps a pipe literal text'
assert.equal(content(markdownToAdf('a | b\n--- | ---\n')), bare)
assert.equal(content(markdownToAdf('Intro.\na | b\n--- | ---\n')), bare)
assert.equal(content(markdownToAdf('a | b\n--- | ---\n===\n')), bare)
assert.equal(content(markdownToAdf('a | b\n:--- | ---:\n')), bare)
assert.deepEqual(content(markdownToAdf('a | b\nc | d\n')), [paragraph('a | b c | d')])
assert.deepEqual(content(markdownToAdf('a | b\n--- | --- | ---\n')), [paragraph('a | b --- | --- | ---')])
assert.deepEqual(content(markdownToAdf('a \\| b\n--- | ---\n')), [paragraph('a | b --- | ---')])
assert.deepEqual(content(markdownToAdf('a\n---\n')), [{ attrs: { level: 2 }, content: [text('a')], type: 'heading' }])
assert.deepEqual(path(markdownToAdf('Part.\n\na | b\n--- | ---\n')), ['content', 1])
})
test('gives back the refusal an inline body holds, never the shape check above it', () => {
const bare = 'malformed-pipe-table: a pipe table opens every row with `|`: this one does not; \\| keeps a pipe literal text'
assert.equal(content(markdownToAdf(':::caption\na | b\n--- | ---\n:::\n')), bare)
assert.deepEqual(position(markdownToAdf(':::caption\na | b\n--- | ---\n:::\n')), { line: 2, offset: 11 })
assert.equal(code(markdownToAdf(':::caption\n| a |\n:::\n')), 'malformed-pipe-table')
assert.equal(content(markdownToAdf(':::caption\n- a\n:::\n')), 'unsupported-node-shape: caption takes one paragraph as its body: this body is not one')
})
test('reads the separator that parts two adjacent lists of one kind', () => {
const parted = [bulletList(item(paragraph('a'))), bulletList(item(paragraph('b')))]
assert.deepEqual(content(markdownToAdf('- a\n\n::listBreak\n\n- b\n')), parted)
assert.deepEqual(content(markdownToAdf('- a\n::listBreak\n- b\n')), parted)
assert.deepEqual(content(markdownToAdf('1. a\n\n::listBreak\n\n1. b\n')), [orderedList(1, item(paragraph('a'))), orderedList(1, item(paragraph('b')))])
assert.deepEqual(content(markdownToAdf('> - a\n> ::listBreak\n> - b\n')), [quote(...parted)])
assert.deepEqual(path(markdownToAdf('- a\n\n::listBreak\n\n- b\n\n| x |\n')), ['content', 2])
})
test('refuses the list separator that parts anything else', () => {
const parts = 'unsupported-node-shape: listBreak parts two adjacent lists of one type: this one parts something else'
assert.equal(content(markdownToAdf('::listBreak\n')), parts)
assert.equal(content(markdownToAdf('- a\n\n::listBreak\n')), parts)
assert.equal(content(markdownToAdf('- a\n\n::listBreak\n\n1. b\n')), parts)
assert.equal(content(markdownToAdf('Part.\n\n::listBreak\n\n- b\n')), parts)
const bare = 'unsupported-node-shape: listBreak spells the bare leaf form, ::listBreak: this one spells more'
assert.equal(content(markdownToAdf('- a\n\n::listBreak x\n\n- b\n')), bare)
assert.equal(content(markdownToAdf('- a\n\n::listBreak {id=x}\n\n- b\n')), bare)
assert.equal(content(markdownToAdf(':::listBreak\n- a\n:::\n')), bare)
assert.equal(content(markdownToAdf(':listBreak{}\n')), 'unsupported-node-shape: listBreak takes the block form, ::listBreak, never the inline form')
assert.deepEqual(path(markdownToAdf('Part.\n\n::listBreak\n')), ['content', 1])
})
test('refuses the image a pipe cell holds no ADF node for', () => {
assert.equal(content(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), 'unmappable-image: no ADF node carries an image inside a paragraph')
assert.deepEqual(path(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), ['content', 0, 'content', 1, 'content', 0, 'content', 0])
@@ -243,7 +287,7 @@ test('names the directive form a node CommonMark spells refuses', () => {
// The spelling the emitter refuses gives the emitter's own error, never a second name for it.
test('gives back the refusal the CommonMark spelling itself raises', () => {
const destination = ':::blockquote\n[t](https://example.com/a\\b)\n:::\n'
assert.equal(content(markdownToAdf(destination)), 'unspellable-link-destination: no canonical escape spells a backslash in a link destination')
assert.equal(content(markdownToAdf(destination)), 'unspellable-link: no canonical escape spells a backslash in a link destination')
})
test('names the directive name no node reads back to', () => {
+24 -1
View File
@@ -1,15 +1,18 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import type { Block, DirectiveBlock } from './blocks.ts'
import type { BlockDirectiveNode } from './directive-nodes.ts'
import type { ConvertFault } from '../../result.ts'
import type { LinkDefinitions } from './inline-content.ts'
import { carryName, readCarriedBlock } from '../opaque-carry.ts'
import { commonMarkSpelling } from '../emit/adf-to-markdown.ts'
import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts'
import { languageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts'
import { listBreakName, listBreakSpelling } from '../list-break.ts'
import { parseBlocks } from './blocks.ts'
import { parseInlineContent } from './inline-content.ts'
import { readBlockDirectiveNode } from './directive-nodes.ts'
import { unsupportedNodeShape } from '../directive-syntax.ts'
const documentStart: SourcePosition = { line: 1, offset: 0 }
@@ -24,13 +27,32 @@ function blockNodes(blocks: readonly Block[], definitions: LinkDefinitions, path
if (depth > largestNesting) return failure('unsupported-nesting-depth', `the input nests deeper than the ${largestNesting} levels the parser carries`, path)
const content: AdfNode[] = []
for (const [index, block] of blocks.entries()) {
const node = positioned(blockNode(block, definitions, [...path, 'content', index], depth), block.position)
const nodePath = [...path, 'content', content.length]
if (block.kind === 'directive' && block.name === listBreakName) {
const fault = listBreakFault(block, blocks[index - 1], blocks[index + 1])
if (fault !== undefined) return positioned(faulted(fault, nodePath), block.position)
continue
}
const node = positioned(blockNode(block, definitions, nodePath, depth), block.position)
if (!node.ok) return node
content.push(node.value)
}
return success(content)
}
// spec/flavour.md, Directives: the separator builds no node, so only the pair it parts spells it.
function listBreakFault(block: DirectiveBlock, previous: Block | undefined, next: Block | undefined): ConvertFault | undefined {
if (block.blocks !== undefined || block.argument !== undefined || block.attributes.size > 0) {
return unsupportedNodeShape(`${listBreakName} spells the bare leaf form, ${listBreakSpelling}: this one spells more`)
}
if (previous?.kind !== 'bulletList' && previous?.kind !== 'orderedList') return partsFault()
return previous.kind === next?.kind ? undefined : partsFault()
}
function partsFault(): ConvertFault {
return unsupportedNodeShape(`${listBreakName} parts two adjacent lists of one type: this one parts something else`)
}
function blockNode(block: Block, definitions: LinkDefinitions, path: ConvertErrorPath, depth: number): Result<AdfNode> {
switch (block.kind) {
case 'blockquote':
@@ -112,6 +134,7 @@ function tableNode(rows: readonly string[][], definitions: LinkDefinitions, path
function inlineBodyNode(node: AdfNode, blocks: readonly Block[], definitions: LinkDefinitions, path: ConvertErrorPath): Result<AdfNode> {
if (blocks.length === 0) return failure('unsupported-node-shape', `an empty ${node.type} takes the leaf form, ::, never an empty container`, path)
const only = blocks.length === 1 ? blocks[0] : undefined
if (only?.kind === 'fault') return positioned(faulted(only.fault, path), only.position)
if (only?.kind !== 'paragraph') return failure('unsupported-node-shape', `${node.type} takes one paragraph as its body: this body is not one`, path)
return positioned(contentNode(node, only.text, definitions, path), only.position)
}
+36 -18
View File
@@ -4,6 +4,21 @@ import { backslashEscape, claimsPipeLine, trimSpace } from './commonmark-grammar
const alignmentCell = /^:-+:?$|^-+:$/
const delimiterCell = /^-+$/
// spec/flavour.md, Tables: the cells of a row no leading `|` claimed — GFM's form without the outer pipes.
export function barePipeCells(line: string): string[] | undefined {
const cells = splitPipeCells(line, 0)
return cells.length > 1 ? cells : undefined
}
export function isBareDelimiterRow(line: string): boolean {
const cells = barePipeCells(line)
return cells !== undefined && isDelimiterRow(cells)
}
export function isDelimiterRow(cells: readonly string[]): boolean {
return cells.every((cell) => isPipeDelimiter(cell) || isPipeAlignment(cell))
}
export function isPipeAlignment(cell: string): boolean {
return alignmentCell.test(cell)
}
@@ -18,24 +33,7 @@ export function malformedPipeTable(message: string): ConvertFault {
// spec/flavour.md, Tables: the cells of a claimed row, the closing `|` the spelling writes optional here.
export function pipeCells(line: string): string[] | undefined {
if (!claimsPipeLine(line)) return undefined
const cells: string[] = []
let start = 1
let index = 1
while (index < line.length) {
if (backslashEscape(line, index) !== undefined) {
index += 2
continue
}
if (line.charAt(index) === '|') {
cells.push(trimSpace(line.slice(start, index)))
start = index + 1
}
index += 1
}
cells.push(trimSpace(line.slice(start)))
if (cells.length > 1 && cells.at(-1) === '') cells.pop()
return cells
return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined
}
export function spellPipeDelimiter(columns: number): string {
@@ -45,3 +43,23 @@ export function spellPipeDelimiter(columns: number): string {
export function spellPipeRow(cells: readonly string[]): string {
return `| ${cells.join(' | ')} |`
}
function splitPipeCells(line: string, start: number): string[] {
const cells: string[] = []
let cellStart = start
let index = start
while (index < line.length) {
if (backslashEscape(line, index) !== undefined) {
index += 2
continue
}
if (line.charAt(index) === '|') {
cells.push(trimSpace(line.slice(cellStart, index)))
cellStart = index + 1
}
index += 1
}
cells.push(trimSpace(line.slice(cellStart)))
if (cells.length > 1 && cells.at(-1) === '') cells.pop()
return cells
}
+1 -3
View File
@@ -5,11 +5,9 @@ export type ConvertErrorCode =
| 'unknown-directive-name'
| 'unmappable-html'
| 'unmappable-image'
| 'unspellable-adjacent-lists'
| 'unspellable-character'
| 'unspellable-line-start'
| 'unspellable-link-destination'
| 'unspellable-link-title'
| 'unspellable-link'
| 'unspellable-whitespace'
| 'unsupported-document-version'
| 'unsupported-nesting-depth'
+26
View File
@@ -429,3 +429,29 @@ Under **3 — `markdownToAdf` (`0.1.0`)**:
claimed — `\:::` on the directive line's, `\|` on the pipe table's, `\:` on the inline
directive's, the attribute-pair and unknown-name faults taking whichever form read them.
`a pipe table row holds 1 cells` gained its plural.
- [x] **5b3 — The code list and the flavour's gaps.** A second read of the surface, this one on
the fifteen names §8 freezes at `0.1.0`: two pairs of them are one cause each, and one
names a state the flavour leaves no way out of. `unspellable-character` and the text half
of `unspellable-whitespace` are one refusal — a character CommonMark rewrites, the message
naming it — and merge, `unspellable-whitespace` keeping the code for its other cause, the
content slot no inline directive spans. `unspellable-link-destination` and
`unspellable-link-title` become `unspellable-link`, the message naming the attribute.
`unspellable-adjacent-lists` goes entirely: two adjacent `bulletList` nodes are valid ADF a
site writes, and refusing them leaves the viewer persona a document it cannot render at
all, so the flavour gains the separator that spells the pair apart, both directions,
`spec/flavour.md` and fixtures. Thirteen codes stand — `unspellable-whitespace` keeps its own.
The bare pipe table — `a | b` over `--- | ---`, GFM's shape without the leading pipes — is
the one input that loses structure silently, reading back as a paragraph of prose; it
becomes a `malformed-pipe-table` naming the form a row takes. That code keeps its name for
the alignment colon: the flavour's own delimiter row is `-` runs, so the grammar is what
refuses, and §8 records it rather than answering it again each review. The ninth
`adfDocumentFault` branch names no node and carries the document's own path, the one branch
the other eight outshine; §8 records why the guard stays a boolean.
`::listBreak` is the separator's spelling: the grammar's leaf form, and a second reserved
name beside `adf` — every other name is an ADF node or mark type, and this one builds none.
It reads only between two adjacent lists of one type, and takes the separation any
directive block takes where it sits, so a directive container holds it with no blank line.
A hard break is the one spelling that can put a bare delimiter row under a row of its own,
so the emitter escapes that line's first character rather than refusing the document.
`spec/flavour.md` had two directive blocks inside a container taking no blank line; the
rule both directions keep is that a pair holding one takes none.
+8 -6
View File
@@ -143,20 +143,22 @@ The numbering is the order the work was planned in, not the order it ships.
- [ ] **5b — The consumer's error surface (`0.1.0`).** A product-owner read of the public surface
found the error result legible to the library and opaque to the consumer holding it, and the
README documenting no part of it. The sub-items are that read's answers, and they land before
5 because §8 freezes the code list at `0.1.0` and 5b3's table is what reads the list before
5 because §8 freezes the code list at `0.1.0` and 5b4's table is what reads the list before
the freeze closes it.
- [x] **5b1 — The error's source position.**
- [x] **5b2 — The error messages.**
- [ ] **5b3 — The README's consumer surface.** §8 invites an exhaustive switch on `code` and no
- [x] **5b3 — The code list and the flavour's gaps.**
- [ ] **5b4 — The README's consumer surface.** §8 invites an exhaustive switch on `code` and no
code name appears in the README, so it gains a table — code, when it fires, what the
consumer does — grouped by direction; drafting it is the audit that reads the fifteen names
before 5 freezes them. Four things a reader who has not opened the code cannot know: raw
consumer does — grouped by direction, over the thirteen names 5b3 settled. Four things a
reader who has not opened the code cannot know: raw
HTML is core CommonMark and every construct in input is an error until `0.3.0`, which the
guarantees' "three carve-outs and one gap" denies and which is the bot and LLM personas'
most common failure; `adfToHtml`, `htmlToAdf`, `markdownToHtml` and `htmlToMarkdown` sit
unmarked in the code block people copy from, as do the two HTML guarantee bullets, and take
a `0.3.0` mark or leave the block; `adfToMarkdown` is partial on valid ADF — two adjacent
`bulletList` nodes are an error result — which the viewer persona needs told along with
a `0.3.0` mark or leave the block; `adfToMarkdown` is partial on valid ADF — a text node
holding a carriage return, a link destination no canonical escape spells — which the viewer
persona needs told along with
what to do about it; and GFM past tables and strikethrough is literal text, task lists
taking `:::taskList`. One sentence for the LLM persona: `code` is stable across minors,
`message` is free text. The type-level surface freezes at the same moment and gets the same