diff --git a/AGENTS.md b/AGENTS.md index 8d6e6d2..771f2ca 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -97,10 +97,15 @@ names the escape that unclaims the form claimed: `\:::` for a directive line, `\ row, `\:` for an inline directive. Adding, removing or renaming a code is breaking, so a milestone meeting a new failure cause reuses a code where one fits; the list is complete at `0.1.0`. A code names the -cause; where one cause recurs across node types or across directions, one code covers them all and +cause; where one cause recurs across node types, across one mark's attributes or across +directions, one code covers them all and `path` and `message` say which — `unsupported-nesting-depth` is the 500-level guard whichever -direction hits it. A claim code names the spelling claimed, never the node that spelling would have built: -a malformed `:::table` is a `malformed-directive`. A cause the carry answers gets no code: a mark no +direction hits it, `unspellable-link` the destination and the title alike. A claim code names the spelling claimed, never the node that spelling would have built: +a malformed `:::table` is a `malformed-directive`, and an alignment colon a `malformed-pipe-table` — +the flavour's own delimiter row is `-` runs, so the grammar refuses the colon rather than ADF's +missing column model doing it. A refusal no spelling recovers from is a gap in the flavour rather +than a code: give the flavour the spelling and the code goes, which the freeze is the last moment +for. A cause the carry answers gets no code: a mark no spelling writes rides the carry with its node. A directive whose name reads back to no node is `unknown-directive-name` rather than a claim code — the spelling is well formed, and telling that apart from a typo is what a consumer switches on when a later MINOR gives the name meaning. What @@ -113,7 +118,11 @@ them after `0.1.0` is MAJOR. `unmappable-html` names the version rather than the converts no raw HTML, so at `0.3.0` the mapped elements stop erroring and the code stays for what no ADF node carries. A refusal found before its path is known — the block walk's, a directive reader's — is a `ConvertFault`, the code and message alone; the node walk attaches the path as it -descends, so a document reports its first error in document order. +descends, so a document reports its first error in document order. `not-an-adf-document` carries +the document's own path throughout: eight of the guard's nine branches read the document's own +shape, and threading a path to the ninth — a malformed node anywhere in the tree — wants the +manual stack §11's no-recursion rule forces, whose empty half no input reaches. The message names +the violation instead. `position` is the parse side's alone: an emitter reads no source, so an emit error carries `path` and nothing more. It is `{ line, offset }` at the start of the line the block holding the refusal diff --git a/corpus/errors/list-break-misplaced.error b/corpus/errors/list-break-misplaced.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/list-break-misplaced.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/list-break-misplaced.md b/corpus/errors/list-break-misplaced.md new file mode 100644 index 0000000..b739d5b --- /dev/null +++ b/corpus/errors/list-break-misplaced.md @@ -0,0 +1,5 @@ +- Bolt M8 + +::listBreak + +Packed. diff --git a/corpus/errors/pipe-row-bare.error b/corpus/errors/pipe-row-bare.error new file mode 100644 index 0000000..5bf92ee --- /dev/null +++ b/corpus/errors/pipe-row-bare.error @@ -0,0 +1 @@ +malformed-pipe-table diff --git a/corpus/errors/pipe-row-bare.md b/corpus/errors/pipe-row-bare.md new file mode 100644 index 0000000..a7b72a9 --- /dev/null +++ b/corpus/errors/pipe-row-bare.md @@ -0,0 +1,3 @@ +Part | Qty +--- | --- +Bolt M8 | 40 diff --git a/corpus/round-trip/block-nodes/adjacent-lists.json b/corpus/round-trip/block-nodes/adjacent-lists.json new file mode 100644 index 0000000..3038748 --- /dev/null +++ b/corpus/round-trip/block-nodes/adjacent-lists.json @@ -0,0 +1,134 @@ +{ + "content": [ + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Bolt", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Nut", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "attrs": { + "order": 1 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Cut", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + }, + { + "attrs": { + "order": 1 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Weld", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + }, + { + "attrs": { + "panelType": "info" + }, + "content": [ + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Left", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Right", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + } + ], + "type": "panel" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/adjacent-lists.md b/corpus/round-trip/block-nodes/adjacent-lists.md new file mode 100644 index 0000000..dc50549 --- /dev/null +++ b/corpus/round-trip/block-nodes/adjacent-lists.md @@ -0,0 +1,17 @@ +- Bolt + +::listBreak + +- Nut + +1. Cut + +::listBreak + +1. Weld + +:::panel info +- Left +::listBreak +- Right +::: diff --git a/spec/flavour.md b/spec/flavour.md index d3dccb8..ff0425e 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -2,7 +2,7 @@ The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` parses. Plain CommonMark is a subset with three carve-outs: literal text that matches directive syntax below or -opens a pipe table is claimed by the flavour, and a matched `~~` pair spells `strike` (escape the +shapes a pipe table is claimed by the flavour, and a matched `~~` pair spells `strike` (escape the `:`, `|` or `~` to keep it literal) — and one gap: a CommonMark image fits only as its own title-less paragraph — mid-text and titled images are named errors. The emitted form is contract (AGENTS.md §8). Per-node syntaxes build on this grammar in the sections below. @@ -23,8 +23,10 @@ normalizes to it through the round-trip. marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or a list whose first item is empty), whatever block sits above it. Blank lines between items normalize away, and no list opens beside one of its own kind — the marker change CommonMark starts a - second list on merges instead: ADF records no tightness, and one `- ` spelling leaves two - adjacent lists of a kind no way back. + second list on merges instead: ADF records no tightness, so one `- ` spelling reads two + adjacent lists of a kind back as one. The leaf `::listBreak` parts them, taking the separation + any directive block takes where it sits. It builds no node, and it reads only between two + adjacent lists of one type: elsewhere, or carrying an argument or `{attrs}`, it is a named error. - Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`. - ATX headings (`#` … `######`); setext input normalizes to ATX. - Code fences ``` with the node's language as info string, the fence lengthened past any backtick @@ -51,7 +53,8 @@ normalizes to it through the round-trip. `*not emphasis*` is `\*not emphasis\*` — no delimiter the emitter did not write reaches the matching below, which is what lets the emitter decide its own pairings. - Blocks separated by one blank line at document level, inside a blockquote and between CommonMark - blocks; two directive blocks inside a container take none. No trailing whitespace outside a code + blocks; inside a directive container a pair holding a directive block takes none. No trailing + whitespace outside a code block's content, single trailing newline; a document with no blocks is the empty string. @@ -64,8 +67,9 @@ whether the name is known, and an unknown name is an error result naming it — emitter escaped stays escaped, and erroring input gaining meaning later is MINOR, never a reparse (§8). Each name belongs to one position, and a name the other one spells — a mark or an inline node written as a block directive, a block node written inline — is a different error, naming the -spelling it takes. The name `adf` is reserved for the opaque carry, as both directive name and fence info -string. +spelling it takes. Two reserved names read back to no node: `adf` for the opaque carry, as both +directive name and fence info string, and `listBreak` for the leaf that parts two adjacent lists +(Canonical form). **Inline**: `:name[content]{attrs}`, on one line — an inline directive never spans lines. `[content]` is inline markdown; brackets inside balance as in CommonMark link text, `\]` for a @@ -291,7 +295,10 @@ takes the directive form instead. A pipe table parses back to exactly that shape Claiming at block level, symmetric with directives: a line opening with an unescaped `|` is claimed and must parse as part of a pipe table, else it is a named error — escape the pipe -(`\|`) to keep it literal text. A pipe table is a header row, a delimiter row whose cells are +(`\|`) to keep it literal text. The shape a missing leading pipe leaves is claimed too: a line +holding an unescaped `|`, followed inside one paragraph by a line whose cells are all `-` runs +and match it in count — GFM's table without the outer pipes — is a named error rather than the +prose it reads as. A pipe table is a header row, a delimiter row whose cells are runs of one or more `-` (canonical `---`), and body rows; rows follow code-fence indentation. Cells split on unescaped `|` before inline parsing — `\|` stays in the cell text, and the inline layer's ordinary CommonMark escaping yields the pipe; each cell is the inline content of diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index 9429bba..c50c679 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -81,13 +81,13 @@ test('spells a code block language no info string holds as an attribute', () => test('refuses a link destination CommonMark cannot spell', () => { const link = (href: string): AdfDocument => document(paragraph({ marks: [{ attrs: { href }, type: 'link' }], text: 't', type: 'text' })) - assert.equal(code(adfToMarkdown(link('https://example.com/a b>c'))), 'unspellable-link-destination') - assert.equal(code(adfToMarkdown(link('c'))), 'unspellable-link') + assert.equal(code(adfToMarkdown(link(' { @@ -103,8 +103,8 @@ test('escapes the quote a link title holds, and refuses the rest', () => { const titled = (title: string): AdfDocument => document(paragraph({ marks: [{ attrs: { href: 'https://example.com/', title }, type: 'link' }], text: 't', type: 'text' })) assert.equal(markdown(adfToMarkdown(titled('He said "hi"'))), '[t](https://example.com/ "He said \\"hi\\"")\n') - assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link-title') - assert.equal(code(adfToMarkdown(titled('a\\b'))), 'unspellable-link-title') + assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link') + assert.equal(code(adfToMarkdown(titled('a\\b'))), 'unspellable-link') }) test('carries a link mark the link spelling cannot write', () => { @@ -134,17 +134,25 @@ test('carries a mark the canonical spellings cannot nest', () => { ) }) -test('refuses whitespace CommonMark cannot hold', () => { - assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), 'unspellable-whitespace') -}) - test('refuses a line whose start block parsing would claim', () => { assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: '```', type: 'text' })))), 'unspellable-line-start') }) -test('refuses two adjacent lists of the same kind, the marker spelling being what merges', () => { +test('escapes the delimiter row a hard break leaves opening a pipe table with no leading pipe', () => { + const broken = (second: string): string => markdown(adfToMarkdown(document(paragraph({ text: 'a | b', type: 'text' }, { type: 'hardBreak' }, { text: second, type: 'text' })))) + assert.equal(broken('--- | ---'), 'a | b\\\n\\--- | ---\n') + assert.equal(broken(':--- | ---:'), 'a | b\\\n\\:--- | ---:\n') + assert.equal(broken('c | d'), 'a | b\\\nc | d\n') +}) + +test('parts two adjacent lists of the same kind, the marker spelling being what merges', () => { const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'bulletList' } - assert.equal(code(adfToMarkdown(document(list, list))), 'unspellable-adjacent-lists') + assert.equal(markdown(adfToMarkdown(document(list, list))), '- x\n\n::listBreak\n\n- x\n') + const ordered: AdfNode = { attrs: { order: 1 }, content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'orderedList' } + assert.equal(markdown(adfToMarkdown(document(ordered, ordered))), '1. x\n\n::listBreak\n\n1. x\n') + assert.equal(markdown(adfToMarkdown(document({ attrs: { panelType: 'info' }, content: [list, list], type: 'panel' }))), ':::panel info\n- x\n::listBreak\n- x\n:::\n') + const nested: AdfNode = { content: [{ content: [list, list], type: 'listItem' }], type: 'bulletList' } + assert.equal(markdown(adfToMarkdown(document(nested))), '- - x\n\n ::listBreak\n\n - x\n') const carried: AdfNode = { ...list, attrs: { unknown: 'x' } } assert.ok(markdown(adfToMarkdown(document(carried, carried))).includes('```\n\n```adf\n')) assert.ok(markdown(adfToMarkdown(document(carried, list))).endsWith('```\n\n- x\n')) @@ -394,7 +402,11 @@ test('spells a list item whose marker completes a thematic break as a directive' test('refuses the characters CommonMark rewrites', () => { assert.equal( markdown(adfToMarkdown(document({ content: [{ text: 'a\rb', type: 'text' }], type: 'codeBlock' }))), - 'unspellable-whitespace: a codeBlock holds no carriage return CommonMark keeps: this text holds one', + 'unspellable-character: a codeBlock holds no carriage return CommonMark keeps: this text holds one', + ) + assert.equal( + markdown(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), + 'unspellable-character: a text node holds a carriage return CommonMark rewrites', ) assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\u0000b', type: 'text' })))), 'unspellable-character') assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))), 'unspellable-character') diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index fee61a6..d9af780 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -9,6 +9,7 @@ import { fencedCodeBlock } from '../backtick-runs.ts' import { holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' +import { listBreakSpelling } from '../list-break.ts' import { spellDirectiveHeader } from './block-directive-spelling.ts' import { tryImage } from './image.ts' import { tryPipeTable } from './pipe-table.ts' @@ -17,7 +18,7 @@ type BlockContainer = 'directive' | 'document' | 'list-item' type BlockSpelling = 'commonmark' | 'directive' | 'list' type EmittedBody = { fenceColons: number; text: string } type EmittedBlock = EmittedBody & { spelling: BlockSpelling } -type PlacedBlock = EmittedBlock & { node: AdfNode; path: ConvertErrorPath } +type PlacedBlock = EmittedBlock & { node: AdfNode } const largestListMarker = 999999999 @@ -34,35 +35,31 @@ function emitBlocks(nodes: readonly AdfNode[], container: BlockContainer, path: if (depth > largestNesting) return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) const blocks: PlacedBlock[] = [] for (const [index, node] of nodes.entries()) { - const nodePath = [...path, 'content', index] - const block = emitBlock(node, nodePath, depth) + const block = emitBlock(node, [...path, 'content', index], depth) if (!block.ok) return block - blocks.push({ ...block.value, node, path: nodePath }) + blocks.push({ ...block.value, node }) } let fenceColons = 0 let text = '' for (const [index, block] of blocks.entries()) { const previous = blocks[index - 1] - if (previous !== undefined) { - const separation = separationBetween(previous, block, container) - if (!separation.ok) return separation - text += separation.value - } + if (previous !== undefined) text += separationBetween(previous, block, container) fenceColons = Math.max(fenceColons, block.fenceColons) text += block.text } return success({ fenceColons, text }) } -function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): Result { +function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): string { const plainPair = previous.spelling !== 'directive' && next.spelling !== 'directive' if (plainPair && next.spelling === 'list') { if (previous.spelling === 'list' && previous.node.type === next.node.type) { - return failure('unspellable-adjacent-lists', `two adjacent ${next.node.type} nodes read back as one list`, next.path) + const gap = container === 'directive' ? '\n' : '\n\n' + return `${gap}${listBreakSpelling}${gap}` } - if (container === 'list-item') return success(interruptsParagraph(next.node) ? '\n' : '\n\n') + if (container === 'list-item') return interruptsParagraph(next.node) ? '\n' : '\n\n' } - return success(container === 'directive' && !plainPair ? '\n' : '\n\n') + return container === 'directive' && !plainPair ? '\n' : '\n\n' } function interruptsParagraph(node: AdfNode): boolean { @@ -179,7 +176,7 @@ function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result { ) { return failure('unsupported-node-shape', `a codeBlock holds plain text nodes only: this ${child.type} node is not one`, childPath) } - if (/\r/.test(child.text)) return failure('unspellable-whitespace', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath) + if (/\r/.test(child.text)) return failure('unspellable-character', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath) if (holdsNullCharacter(child.text)) return failure('unspellable-character', 'a codeBlock holds a null character CommonMark replaces', childPath) text += child.text } diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index df0b0d1..6588593 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -220,7 +220,7 @@ function emitText(node: AdfNode, context: InlineContext, index: number, path: Co if (Object.keys(node.attrs ?? {}).length > 0) return success({ carry: { first: index, last: index } }) if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) - if (/\r/.test(node.text)) return failure('unspellable-whitespace', 'a text node holds a carriage return CommonMark rewrites', path) + if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path) if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path) const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash' const parts = node.text.split(/(\n+)/).filter((part) => part !== '') diff --git a/src/markdown/emit/line-escaping.ts b/src/markdown/emit/line-escaping.ts index d677375..4861771 100644 --- a/src/markdown/emit/line-escaping.ts +++ b/src/markdown/emit/line-escaping.ts @@ -1,6 +1,7 @@ import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../emphasis-matching.ts' import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark-grammar.ts' +import { isBareDelimiterRow } from '../pipe-table-syntax.ts' import { opensInlineDirective } from '../directive-syntax.ts' import { readEntityReference } from '../entity-references.ts' @@ -196,8 +197,11 @@ function opensConstruct( return claimsCharacter(scan, linkClose, index, inBrackets, container, escaped) } +// A hard break is the one spelling that puts a delimiter row under a row of its own, so only a later line claims. function claimsLineStart(line: ScanLine, index: number, container: LineContainer): boolean { - return container === 'paragraph' && escapesLineClaim(line.text, index - line.start, line.position) + if (container !== 'paragraph') return false + if (index === line.start && line.position === 'later' && isBareDelimiterRow(line.text)) return true + return escapesLineClaim(line.text, index - line.start, line.position) } function scanLine(scan: string, start: number): ScanLine { diff --git a/src/markdown/link-syntax.ts b/src/markdown/link-syntax.ts index 022adbb..488011e 100644 --- a/src/markdown/link-syntax.ts +++ b/src/markdown/link-syntax.ts @@ -91,26 +91,26 @@ export function skipLinkWhitespace(text: string, offset: number): number { } export function spellDestination(href: string, path: ConvertErrorPath): Result { - if (holdsControlCharacter(href)) return failure('unspellable-link-destination', 'a link destination holds a control character', path) - if (href.includes('\\')) return failure('unspellable-link-destination', 'no canonical escape spells a backslash in a link destination', path) + if (holdsControlCharacter(href)) return failure('unspellable-link', 'a link destination holds a control character', path) + if (href.includes('\\')) return failure('unspellable-link', 'no canonical escape spells a backslash in a link destination', path) if (holdsEntityReference(href)) { - return failure('unspellable-link-destination', 'a link destination holds an entity reference that decodes on the way back', path) + return failure('unspellable-link', 'a link destination holds an entity reference that decodes on the way back', path) } if (href.includes(' ')) { if (/[<>]/.test(href)) { - return failure('unspellable-link-destination', 'no canonical escape spells an angle bracket beside a space in a link destination', path) + return failure('unspellable-link', 'no canonical escape spells an angle bracket beside a space in a link destination', path) } return success(`<${href}>`) } - if (href.startsWith('<')) return failure('unspellable-link-destination', 'a bare link destination cannot begin with an angle bracket', path) + if (href.startsWith('<')) return failure('unspellable-link', 'a bare link destination cannot begin with an angle bracket', path) return success(escapeUnbalanced(href)) } export function spellTitle(title: string, path: ConvertErrorPath): Result { if (/[\n\r\\]/.test(title)) { - return failure('unspellable-link-title', 'no canonical escape spells a backslash or newline in a link title', path) + return failure('unspellable-link', 'no canonical escape spells a backslash or newline in a link title', path) } - if (holdsEntityReference(title)) return failure('unspellable-link-title', 'a link title holds an entity reference that decodes on the way back', path) + if (holdsEntityReference(title)) return failure('unspellable-link', 'a link title holds an entity reference that decodes on the way back', path) return success(` "${title.replaceAll('"', '\\"')}"`) } diff --git a/src/markdown/list-break.ts b/src/markdown/list-break.ts new file mode 100644 index 0000000..d12bc99 --- /dev/null +++ b/src/markdown/list-break.ts @@ -0,0 +1,5 @@ +const name = 'listBreak' + +export const listBreakName = name + +export const listBreakSpelling = `::${name}` diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index 0eb6a49..866ff31 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -16,7 +16,7 @@ import { setextHeadingLevel, } from '../commonmark-grammar.ts' import { directiveLineEscape, malformedDirective, readDirectiveLine } from '../directive-syntax.ts' -import { isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts' +import { barePipeCells, isDelimiterRow, isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts' import { readLinkDefinitions } from './link-reference-definitions.ts' export type Block = { position: SourcePosition } & ( @@ -358,7 +358,7 @@ function readLineBlock(walk: Walk, opener: string): boolean { if (level !== undefined) { const paragraph = takeParagraph(walk) if (paragraph !== undefined) { - currentBlocks(walk).push({ kind: 'heading', level, position: paragraph.position, text: paragraph.text }) + currentBlocks(walk).push(bareTableFault(paragraph) ?? { kind: 'heading', level, position: paragraph.position, text: paragraph.text }) return true } } @@ -390,7 +390,7 @@ function closeLeaf(walk: Walk): void { if (leaf === undefined) return if (leaf.kind === 'paragraph') { const paragraph = takeParagraph(walk) - if (paragraph !== undefined) currentBlocks(walk).push(paragraph) + if (paragraph !== undefined) currentBlocks(walk).push(bareTableFault(paragraph) ?? paragraph) return } walk.leaf = undefined @@ -413,6 +413,19 @@ function pipeTableBlock(rows: readonly [string[], ...string[][]], position: Sour return { kind: 'table', position, rows: [header, ...body] } } +// spec/flavour.md, Tables: GFM's table without the leading pipes, which no line of it claims. +function bareTableFault(paragraph: Extract): Block | undefined { + let header: string[] | undefined + for (const line of paragraph.text.split('\n')) { + const cells = barePipeCells(line) + if (header !== undefined && cells !== undefined && cells.length === header.length && isDelimiterRow(cells)) { + return faultedBlock('a pipe table opens every row with `|`: this one does not; \\| keeps a pipe literal text', paragraph.position) + } + header = cells + } + return undefined +} + function cellCount(count: number): string { return `${count} cell${count === 1 ? '' : 's'}` } diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 3d3dfba..9b81ede 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -10,6 +10,7 @@ import { carryName } from '../opaque-carry.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { inlineDirective } from '../../adf/inline-directives.ts' import { inlineMarkSpellingFault } from './directive-marks.ts' +import { listBreakName, listBreakSpelling } from '../list-break.ts' import { marksAttribute, readMarkValues } from '../block-directive-marks.ts' import { readVocabulary } from './directive-attributes.ts' import { slotLineEndingFault } from '../directive-syntax.ts' @@ -76,6 +77,7 @@ function inlineSpellingFault(name: string): ConvertFault | undefined { } function blockSpellingFault(name: string): ConvertFault | undefined { + if (name === listBreakName) return { code: 'unsupported-node-shape', message: `${name} takes the block form, ${listBreakSpelling}, never the inline form` } const directive = blockDirective(name) if (directive === undefined) return undefined const form = directive.contentModel === 'none' ? `::${name}` : `:::${name}` diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 93675a5..75c5306 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -190,6 +190,42 @@ test('names the pipe table a claimed line does not spell', () => { assert.deepEqual(path(markdownToAdf('Part.\n\n| a |\n')), ['content', 1]) }) +test('names the pipe table whose rows open with no pipe', () => { + const bare = 'malformed-pipe-table: a pipe table opens every row with `|`: this one does not; \\| keeps a pipe literal text' + assert.equal(content(markdownToAdf('a | b\n--- | ---\n')), bare) + assert.equal(content(markdownToAdf('Intro.\na | b\n--- | ---\n')), bare) + assert.equal(content(markdownToAdf('a | b\n--- | ---\n===\n')), bare) + assert.equal(content(markdownToAdf('a | b\n:--- | ---:\n')), bare) + assert.deepEqual(content(markdownToAdf('a | b\nc | d\n')), [paragraph('a | b c | d')]) + assert.deepEqual(content(markdownToAdf('a | b\n--- | --- | ---\n')), [paragraph('a | b --- | --- | ---')]) + assert.deepEqual(content(markdownToAdf('a \\| b\n--- | ---\n')), [paragraph('a | b --- | ---')]) + assert.deepEqual(content(markdownToAdf('a\n---\n')), [{ attrs: { level: 2 }, content: [text('a')], type: 'heading' }]) + assert.deepEqual(path(markdownToAdf('Part.\n\na | b\n--- | ---\n')), ['content', 1]) +}) + +test('reads the separator that parts two adjacent lists of one kind', () => { + const parted = [bulletList(item(paragraph('a'))), bulletList(item(paragraph('b')))] + assert.deepEqual(content(markdownToAdf('- a\n\n::listBreak\n\n- b\n')), parted) + assert.deepEqual(content(markdownToAdf('- a\n::listBreak\n- b\n')), parted) + assert.deepEqual(content(markdownToAdf('1. a\n\n::listBreak\n\n1. b\n')), [orderedList(1, item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('> - a\n> ::listBreak\n> - b\n')), [quote(...parted)]) + assert.deepEqual(path(markdownToAdf('- a\n\n::listBreak\n\n- b\n\n| x |\n')), ['content', 2]) +}) + +test('refuses the list separator that parts anything else', () => { + const parts = 'unsupported-node-shape: listBreak parts two adjacent lists of one type: this one parts something else' + assert.equal(content(markdownToAdf('::listBreak\n')), parts) + assert.equal(content(markdownToAdf('- a\n\n::listBreak\n')), parts) + assert.equal(content(markdownToAdf('- a\n\n::listBreak\n\n1. b\n')), parts) + assert.equal(content(markdownToAdf('Part.\n\n::listBreak\n\n- b\n')), parts) + const bare = 'unsupported-node-shape: listBreak spells the bare leaf form, ::listBreak: this one spells more' + assert.equal(content(markdownToAdf('- a\n\n::listBreak x\n\n- b\n')), bare) + assert.equal(content(markdownToAdf('- a\n\n::listBreak {id=x}\n\n- b\n')), bare) + assert.equal(content(markdownToAdf(':::listBreak\n- a\n:::\n')), bare) + assert.equal(content(markdownToAdf(':listBreak{}\n')), 'unsupported-node-shape: listBreak takes the block form, ::listBreak, never the inline form') + assert.deepEqual(path(markdownToAdf('Part.\n\n::listBreak\n')), ['content', 1]) +}) + test('refuses the image a pipe cell holds no ADF node for', () => { assert.equal(content(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.deepEqual(path(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), ['content', 0, 'content', 1, 'content', 0, 'content', 0]) @@ -243,7 +279,7 @@ test('names the directive form a node CommonMark spells refuses', () => { // The spelling the emitter refuses gives the emitter's own error, never a second name for it. test('gives back the refusal the CommonMark spelling itself raises', () => { const destination = ':::blockquote\n[t](https://example.com/a\\b)\n:::\n' - assert.equal(content(markdownToAdf(destination)), 'unspellable-link-destination: no canonical escape spells a backslash in a link destination') + assert.equal(content(markdownToAdf(destination)), 'unspellable-link: no canonical escape spells a backslash in a link destination') }) test('names the directive name no node reads back to', () => { diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index b57c96e..721a129 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -1,15 +1,18 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { Block, DirectiveBlock } from './blocks.ts' import type { BlockDirectiveNode } from './directive-nodes.ts' +import type { ConvertFault } from '../../result.ts' import type { LinkDefinitions } from './inline-content.ts' import { carryName, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling } from '../emit/adf-to-markdown.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' +import { listBreakName, listBreakSpelling } from '../list-break.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from './directive-nodes.ts' +import { unsupportedNodeShape } from '../directive-syntax.ts' const documentStart: SourcePosition = { line: 1, offset: 0 } @@ -24,13 +27,32 @@ function blockNodes(blocks: readonly Block[], definitions: LinkDefinitions, path if (depth > largestNesting) return failure('unsupported-nesting-depth', `the input nests deeper than the ${largestNesting} levels the parser carries`, path) const content: AdfNode[] = [] for (const [index, block] of blocks.entries()) { - const node = positioned(blockNode(block, definitions, [...path, 'content', index], depth), block.position) + const nodePath = [...path, 'content', content.length] + if (block.kind === 'directive' && block.name === listBreakName) { + const fault = listBreakFault(block, blocks[index - 1], blocks[index + 1]) + if (fault !== undefined) return positioned(faulted(fault, nodePath), block.position) + continue + } + const node = positioned(blockNode(block, definitions, nodePath, depth), block.position) if (!node.ok) return node content.push(node.value) } return success(content) } +// spec/flavour.md, Directives: the separator builds no node, so only the pair it parts spells it. +function listBreakFault(block: DirectiveBlock, previous: Block | undefined, next: Block | undefined): ConvertFault | undefined { + if (block.blocks !== undefined || block.argument !== undefined || block.attributes.size > 0) { + return unsupportedNodeShape(`${listBreakName} spells the bare leaf form, ${listBreakSpelling}: this one spells more`) + } + if (previous?.kind !== 'bulletList' && previous?.kind !== 'orderedList') return partsFault() + return previous.kind === next?.kind ? undefined : partsFault() +} + +function partsFault(): ConvertFault { + return unsupportedNodeShape(`${listBreakName} parts two adjacent lists of one type: this one parts something else`) +} + function blockNode(block: Block, definitions: LinkDefinitions, path: ConvertErrorPath, depth: number): Result { switch (block.kind) { case 'blockquote': diff --git a/src/markdown/pipe-table-syntax.ts b/src/markdown/pipe-table-syntax.ts index f665a75..026e33a 100644 --- a/src/markdown/pipe-table-syntax.ts +++ b/src/markdown/pipe-table-syntax.ts @@ -4,6 +4,21 @@ import { backslashEscape, claimsPipeLine, trimSpace } from './commonmark-grammar const alignmentCell = /^:-+:?$|^-+:$/ const delimiterCell = /^-+$/ +// spec/flavour.md, Tables: the cells of a row the leading `|` no claim read, GFM's form without it. +export function barePipeCells(line: string): string[] | undefined { + const cells = splitPipeCells(line, 0) + return cells.length > 1 ? cells : undefined +} + +export function isBareDelimiterRow(line: string): boolean { + const cells = barePipeCells(line) + return cells !== undefined && isDelimiterRow(cells) +} + +export function isDelimiterRow(cells: readonly string[]): boolean { + return cells.every((cell) => isPipeDelimiter(cell) || isPipeAlignment(cell)) +} + export function isPipeAlignment(cell: string): boolean { return alignmentCell.test(cell) } @@ -18,24 +33,7 @@ export function malformedPipeTable(message: string): ConvertFault { // spec/flavour.md, Tables: the cells of a claimed row, the closing `|` the spelling writes optional here. export function pipeCells(line: string): string[] | undefined { - if (!claimsPipeLine(line)) return undefined - const cells: string[] = [] - let start = 1 - let index = 1 - while (index < line.length) { - if (backslashEscape(line, index) !== undefined) { - index += 2 - continue - } - if (line.charAt(index) === '|') { - cells.push(trimSpace(line.slice(start, index))) - start = index + 1 - } - index += 1 - } - cells.push(trimSpace(line.slice(start))) - if (cells.length > 1 && cells.at(-1) === '') cells.pop() - return cells + return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined } export function spellPipeDelimiter(columns: number): string { @@ -45,3 +43,23 @@ export function spellPipeDelimiter(columns: number): string { export function spellPipeRow(cells: readonly string[]): string { return `| ${cells.join(' | ')} |` } + +function splitPipeCells(line: string, start: number): string[] { + const cells: string[] = [] + let cellStart = start + let index = start + while (index < line.length) { + if (backslashEscape(line, index) !== undefined) { + index += 2 + continue + } + if (line.charAt(index) === '|') { + cells.push(trimSpace(line.slice(cellStart, index))) + cellStart = index + 1 + } + index += 1 + } + cells.push(trimSpace(line.slice(cellStart))) + if (cells.length > 1 && cells.at(-1) === '') cells.pop() + return cells +} diff --git a/src/result.ts b/src/result.ts index 62ba6e8..60d212b 100644 --- a/src/result.ts +++ b/src/result.ts @@ -5,11 +5,9 @@ export type ConvertErrorCode = | 'unknown-directive-name' | 'unmappable-html' | 'unmappable-image' - | 'unspellable-adjacent-lists' | 'unspellable-character' | 'unspellable-line-start' - | 'unspellable-link-destination' - | 'unspellable-link-title' + | 'unspellable-link' | 'unspellable-whitespace' | 'unsupported-document-version' | 'unsupported-nesting-depth' diff --git a/todo.md b/todo.md index 60e23b0..288ac09 100644 --- a/todo.md +++ b/todo.md @@ -143,20 +143,39 @@ The numbering is the order the work was planned in, not the order it ships. - [ ] **5b — The consumer's error surface (`0.1.0`).** A product-owner read of the public surface found the error result legible to the library and opaque to the consumer holding it, and the README documenting no part of it. The sub-items are that read's answers, and they land before - 5 because §8 freezes the code list at `0.1.0` and 5b3's table is what reads the list before + 5 because §8 freezes the code list at `0.1.0` and 5b4's table is what reads the list before the freeze closes it. - [x] **5b1 — The error's source position.** - [x] **5b2 — The error messages.** - - [ ] **5b3 — The README's consumer surface.** §8 invites an exhaustive switch on `code` and no + - [ ] **5b3 — The code list and the flavour's gaps.** A second read of the surface, this one on + the fifteen names §8 freezes at `0.1.0`: two pairs of them are one cause each, and one + names a state the flavour leaves no way out of. `unspellable-character` and the text half + of `unspellable-whitespace` are one refusal — a character CommonMark rewrites, the message + naming it — and merge, `unspellable-whitespace` keeping the code for its other cause, the + content slot no inline directive spans. `unspellable-link-destination` and + `unspellable-link-title` become `unspellable-link`, the message naming the attribute. + `unspellable-adjacent-lists` goes entirely: two adjacent `bulletList` nodes are valid ADF a + site writes, and refusing them leaves the viewer persona a document it cannot render at + all, so the flavour gains the separator that spells the pair apart, both directions, + `spec/flavour.md` and fixtures. Thirteen codes stand — `unspellable-whitespace` keeps its own. + The bare pipe table — `a | b` over `--- | ---`, GFM's shape without the leading pipes — is + the one input that loses structure silently, reading back as a paragraph of prose; it + becomes a `malformed-pipe-table` naming the form a row takes. That code keeps its name for + the alignment colon: the flavour's own delimiter row is `-` runs, so the grammar is what + refuses, and §8 records it rather than answering it again each review. The ninth + `adfDocumentFault` branch names no node and carries the document's own path, the one branch + the other eight outshine; §8 records why the guard stays a boolean. + - [ ] **5b4 — The README's consumer surface.** §8 invites an exhaustive switch on `code` and no code name appears in the README, so it gains a table — code, when it fires, what the - consumer does — grouped by direction; drafting it is the audit that reads the fifteen names - before 5 freezes them. Four things a reader who has not opened the code cannot know: raw + consumer does — grouped by direction, over the thirteen names 5b3 settled. Four things a + reader who has not opened the code cannot know: raw HTML is core CommonMark and every construct in input is an error until `0.3.0`, which the guarantees' "three carve-outs and one gap" denies and which is the bot and LLM personas' most common failure; `adfToHtml`, `htmlToAdf`, `markdownToHtml` and `htmlToMarkdown` sit unmarked in the code block people copy from, as do the two HTML guarantee bullets, and take - a `0.3.0` mark or leave the block; `adfToMarkdown` is partial on valid ADF — two adjacent - `bulletList` nodes are an error result — which the viewer persona needs told along with + a `0.3.0` mark or leave the block; `adfToMarkdown` is partial on valid ADF — a text node + holding a carriage return, a link destination no canonical escape spells — which the viewer + persona needs told along with what to do about it; and GFM past tables and strikethrough is literal text, task lists taking `:::taskList`. One sentence for the LLM persona: `code` is stable across minors, `message` is free text. The type-level surface freezes at the same moment and gets the same