From 2527d1e3d8a18e503e1d0fe3749d95eb5d19dba7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 1 Sep 2026 17:58:29 +0200 Subject: [PATCH] Read the block nodes back, and stop a marker change splitting a list --- corpus/README.md | 3 - corpus/errors/code-block-body.error | 1 + corpus/errors/code-block-body.md | 3 + corpus/errors/code-block-language-slot.error | 1 + corpus/errors/code-block-language-slot.md | 5 ++ corpus/errors/pipe-alignment-colon.error | 1 + corpus/errors/pipe-alignment-colon.md | 3 + corpus/errors/pipe-cell-count.error | 1 + corpus/errors/pipe-cell-count.md | 3 + corpus/normalization/loose-list.md | 2 +- .../container-block-separation.json | 0 .../block-nodes/container-block-separation.md | 6 ++ .../container-block-separation.error | 1 - spec/flavour.md | 6 +- src/corpus.test.ts | 15 +--- src/markdown/code-language.ts | 12 +++ src/markdown/emit/adf-to-markdown.test.ts | 8 +- src/markdown/emit/adf-to-markdown.ts | 23 +---- src/markdown/emit/pipe-table.ts | 5 +- src/markdown/parse/blocks.ts | 44 ++++++++-- src/markdown/parse/markdown-to-adf.test.ts | 84 +++++++++++++++++-- src/markdown/parse/markdown-to-adf.ts | 34 +++++++- src/markdown/pipe-table-syntax.ts | 48 +++++++++++ src/result.ts | 1 - todo-history.md | 23 ++++- todo.md | 32 ++----- 26 files changed, 282 insertions(+), 83 deletions(-) create mode 100644 corpus/errors/code-block-body.error create mode 100644 corpus/errors/code-block-body.md create mode 100644 corpus/errors/code-block-language-slot.error create mode 100644 corpus/errors/code-block-language-slot.md create mode 100644 corpus/errors/pipe-alignment-colon.error create mode 100644 corpus/errors/pipe-alignment-colon.md create mode 100644 corpus/errors/pipe-cell-count.error create mode 100644 corpus/errors/pipe-cell-count.md rename corpus/{unspellable => round-trip/block-nodes}/container-block-separation.json (100%) create mode 100644 corpus/round-trip/block-nodes/container-block-separation.md delete mode 100644 corpus/unspellable/container-block-separation.error create mode 100644 src/markdown/code-language.ts create mode 100644 src/markdown/pipe-table-syntax.ts diff --git a/corpus/README.md b/corpus/README.md index 8aa572c..d84d3aa 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -9,9 +9,6 @@ One directory per contract kind, each landing with its milestone: `markdownToAdf` must build from it. One-way; the markdown is not canonical. - `errors/` — `.md`: markdown input that must not convert. A `.error` beside it pins which error. -- `unspellable/` — `.json`: ADF `adfToMarkdown` must refuse, the `ConvertErrorCode` in the - `.error` beside it. Two populations live here: refusals that stay contract, and documents - a maintainer decision (`todo.md`) moves to `round-trip/`. - `real-payloads/` — `.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No expected markdown. diff --git a/corpus/errors/code-block-body.error b/corpus/errors/code-block-body.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-body.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-body.md b/corpus/errors/code-block-body.md new file mode 100644 index 0000000..35990e4 --- /dev/null +++ b/corpus/errors/code-block-body.md @@ -0,0 +1,3 @@ +:::codeBlock {wrap=true} +fn main() {} +::: diff --git a/corpus/errors/code-block-language-slot.error b/corpus/errors/code-block-language-slot.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-language-slot.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-language-slot.md b/corpus/errors/code-block-language-slot.md new file mode 100644 index 0000000..d08ee3b --- /dev/null +++ b/corpus/errors/code-block-language-slot.md @@ -0,0 +1,5 @@ +:::codeBlock {language=rust wrap=true} +``` +fn main() {} +``` +::: diff --git a/corpus/errors/pipe-alignment-colon.error b/corpus/errors/pipe-alignment-colon.error new file mode 100644 index 0000000..5bf92ee --- /dev/null +++ b/corpus/errors/pipe-alignment-colon.error @@ -0,0 +1 @@ +malformed-pipe-table diff --git a/corpus/errors/pipe-alignment-colon.md b/corpus/errors/pipe-alignment-colon.md new file mode 100644 index 0000000..8231165 --- /dev/null +++ b/corpus/errors/pipe-alignment-colon.md @@ -0,0 +1,3 @@ +| Part | Qty | +| :--- | ---: | +| Bolt M8 | 40 | diff --git a/corpus/errors/pipe-cell-count.error b/corpus/errors/pipe-cell-count.error new file mode 100644 index 0000000..5bf92ee --- /dev/null +++ b/corpus/errors/pipe-cell-count.error @@ -0,0 +1 @@ +malformed-pipe-table diff --git a/corpus/errors/pipe-cell-count.md b/corpus/errors/pipe-cell-count.md new file mode 100644 index 0000000..3b1f464 --- /dev/null +++ b/corpus/errors/pipe-cell-count.md @@ -0,0 +1,3 @@ +| Part | Qty | +| --- | --- | +| Bolt M8 | diff --git a/corpus/normalization/loose-list.md b/corpus/normalization/loose-list.md index 97056bc..ea18ebd 100644 --- a/corpus/normalization/loose-list.md +++ b/corpus/normalization/loose-list.md @@ -1,6 +1,6 @@ * Bolt M8 -* Nut M8 ++ Nut M8 Then: diff --git a/corpus/unspellable/container-block-separation.json b/corpus/round-trip/block-nodes/container-block-separation.json similarity index 100% rename from corpus/unspellable/container-block-separation.json rename to corpus/round-trip/block-nodes/container-block-separation.json diff --git a/corpus/round-trip/block-nodes/container-block-separation.md b/corpus/round-trip/block-nodes/container-block-separation.md new file mode 100644 index 0000000..a4bbc97 --- /dev/null +++ b/corpus/round-trip/block-nodes/container-block-separation.md @@ -0,0 +1,6 @@ +::::expand {title="Full build log"} +The build ran for 11 minutes. +:::panel warning +Three warnings went unread. +::: +:::: diff --git a/corpus/unspellable/container-block-separation.error b/corpus/unspellable/container-block-separation.error deleted file mode 100644 index 7fdd935..0000000 --- a/corpus/unspellable/container-block-separation.error +++ /dev/null @@ -1 +0,0 @@ -unspelled-block-separation diff --git a/spec/flavour.md b/spec/flavour.md index 73f4085..68bc70a 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -22,7 +22,8 @@ normalizes to it through the round-trip. between a nested list and a CommonMark block above it — one wherever the nested list's own marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or a list whose first item is empty), whatever block sits above it. Blank lines between items normalize - away; ADF does not record tightness. + away, and so does the marker change CommonMark starts a second list on: ADF records no + tightness, and one `- ` spelling leaves two adjacent lists of a type no way back. - Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`. - ATX headings (`#` … `######`); setext input normalizes to ATX. - Code fences ``` with the node's language as info string, the fence lengthened past any backtick @@ -190,7 +191,8 @@ form. (string), `uniqueId` (string), `wrap` (boolean). A language no info string carries back — empty, the reserved `adf`, or holding a backtick, a backslash, a control character, edge whitespace or an entity reference — rides the `language` attribute instead and the fence carries no info - string; writing both is a named error. + string; writing it in the slot that rule leaves empty, or in both, is a named error. The body is + an ordinary fence, so its info string decodes escapes and entity references as any other does. - `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no CommonMark spelling. diff --git a/src/corpus.test.ts b/src/corpus.test.ts index 161c551..adb9439 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -14,11 +14,10 @@ const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus') const errorsRoot = join(corpusRoot, 'errors') const normalizationRoot = join(corpusRoot, 'normalization') const roundTripRoot = join(corpusRoot, 'round-trip') -const unspellableRoot = join(corpusRoot, 'unspellable') const emittingDirectories = ['block-nodes', 'combinations', 'commonmark-subset', 'inline-nodes', 'opaque-carry'] // A directory joins once every fixture in it reads back to its document. -const parsingDirectories = ['commonmark-subset'] +const parsingDirectories = ['block-nodes', 'commonmark-subset'] function directoryNames(root: string): string[] { return readdirSync(root, { withFileTypes: true }) @@ -61,7 +60,7 @@ function corpusJsonPaths(): string[] { } test('every corpus directory is a kind the runner reads', () => { - assert.deepEqual(directoryNames(corpusRoot), ['errors', 'normalization', 'round-trip', 'unspellable']) + assert.deepEqual(directoryNames(corpusRoot), ['errors', 'normalization', 'round-trip']) }) test('every round-trip directory emits', () => { @@ -187,16 +186,6 @@ for (const directory of emittingDirectories) { } } -for (const name of pairedNames(unspellableRoot, '.json', '.error')) { - test(`unspellable/${name} is refused with the error it names`, () => { - const parsed: unknown = JSON.parse(readFileSync(join(unspellableRoot, `${name}.json`), 'utf8')) - assert.ok(isAdfDocument(parsed), `${name}.json is not an ADF document`) - const result = adfToMarkdown(parsed) - assert.ok(!result.ok, result.ok ? `emitted ${JSON.stringify(result.value)}` : '') - assert.equal(result.error.code, readFileSync(join(unspellableRoot, `${name}.error`), 'utf8').trimEnd()) - }) -} - for (const name of pairedNames(normalizationRoot, '.md', '.json')) { test(`normalization/${name} parses to the document beside it`, () => { const expected: unknown = JSON.parse(readFileSync(join(normalizationRoot, `${name}.json`), 'utf8')) diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts new file mode 100644 index 0000000..d254371 --- /dev/null +++ b/src/markdown/code-language.ts @@ -0,0 +1,12 @@ +import type { JsonValue } from '../json-value.ts' +import { carryName } from './opaque-carry.ts' +import { holdsControlCharacter } from './commonmark-grammar.ts' +import { holdsEntityReference } from './entity-references.ts' + +// spec/flavour.md, The CommonMark blocks: the info string the language rides, `undefined` where the attribute carries it. +export function fenceInfo(language: JsonValue | undefined): string | undefined { + if (language === undefined) return '' + if (typeof language !== 'string' || language === '' || language === carryName) return undefined + if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return undefined + return language +} diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index aebacc5..4eea9d0 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -472,15 +472,15 @@ test('refuses the content a directive body has no room for', () => { assert.equal(code(adfToMarkdown(document({ text: 'x', type: 'panel' }))), 'unsupported-node-shape') }) -test('separates two directive blocks in a container body by one line, two CommonMark blocks by a blank one', () => { +test('separates blocks in a container body by a blank line only where the fence is not separation already', () => { const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) const panel = (...content: AdfNode[]): AdfDocument => document({ attrs: { panelType: 'info' }, content, type: 'panel' }) assert.equal(markdown(adfToMarkdown(panel(text('a'), text('b')))), ':::panel info\na\n\nb\n:::\n') const caption: AdfNode = { content: [{ text: 'c', type: 'text' }], type: 'caption' } assert.equal(markdown(adfToMarkdown(panel(caption, caption))), '::::panel info\n:::caption\nc\n:::\n:::caption\nc\n:::\n::::\n') - assert.equal(code(adfToMarkdown(panel(text('a'), caption))), 'unspelled-block-separation') - assert.equal(code(adfToMarkdown(panel(caption, text('a')))), 'unspelled-block-separation') - assert.equal(code(adfToMarkdown(panel(paragraph(), text('a')))), 'unspelled-block-separation') + assert.equal(markdown(adfToMarkdown(panel(text('a'), caption))), '::::panel info\na\n:::caption\nc\n:::\n::::\n') + assert.equal(markdown(adfToMarkdown(panel(caption, text('a')))), '::::panel info\n:::caption\nc\n:::\na\n::::\n') + assert.equal(markdown(adfToMarkdown(panel(paragraph(), text('a')))), ':::panel info\n::paragraph\na\n:::\n') }) test('spells the image form for exactly the centered external media shape', () => { diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index b1a5076..896d40d 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,14 +1,13 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockDirective } from '../../adf/block-directives.ts' -import type { JsonValue } from '../../json-value.ts' import { blockDirective } from '../../adf/block-directives.ts' -import { carriedBlock, carryName } from '../opaque-carry.ts' +import { carriedBlock } from '../opaque-carry.ts' import { carriesOnly, isAdfDocument } from '../../adf/document.ts' import { emitInlineLine } from './inline-line.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { fencedCodeBlock } from '../backtick-runs.ts' -import { holdsControlCharacter, holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' -import { holdsEntityReference } from '../entity-references.ts' +import { fenceInfo } from '../code-language.ts' +import { holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' import { largestNesting } from '../../nesting.ts' import { spellDirectiveHeader } from './block-directive-spelling.ts' import { tryImage } from './image.ts' @@ -62,13 +61,7 @@ function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: } if (container === 'list-item') return success(interruptsParagraph(next.node) ? '\n' : '\n\n') } - if (container !== 'directive' || plainPair) return success('\n\n') - if (previous.spelling === 'directive' && next.spelling === 'directive') return success('\n') - return failure( - 'unspelled-block-separation', - `the canonical form leaves the separation between a ${previous.spelling} and a ${next.spelling} block in a container body unspelled`, - next.path, - ) + return success(container === 'directive' && !plainPair ? '\n' : '\n\n') } function interruptsParagraph(node: AdfNode): boolean { @@ -191,14 +184,6 @@ function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result { return success(text) } -// spec/flavour.md, The CommonMark blocks. -function fenceInfo(language: JsonValue | undefined): string | undefined { - if (language === undefined) return '' - if (typeof language !== 'string' || language === '' || language === carryName) return undefined - if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return undefined - return language -} - function emitHeading(node: AdfNode, path: ConvertErrorPath): Result | undefined { if (!carriesOnly(node, ['level'])) return undefined const level = node.attrs?.['level'] diff --git a/src/markdown/emit/pipe-table.ts b/src/markdown/emit/pipe-table.ts index 6439b44..c47ca57 100644 --- a/src/markdown/emit/pipe-table.ts +++ b/src/markdown/emit/pipe-table.ts @@ -1,5 +1,6 @@ import type { AdfNode } from '../../adf/document.ts' import { carriesOnly } from '../../adf/document.ts' +import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts' import { tryPipeCell } from './inline-line.ts' import type { ConvertErrorPath } from '../../result.ts' @@ -15,8 +16,8 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath): string | un if (line === undefined) return undefined cells.push(line) } - lines.push(`| ${cells.join(' | ')} |`) - if (rowIndex === 0) lines.push(`| ${cells.map(() => '---').join(' | ')} |`) + lines.push(spellPipeRow(cells)) + if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length)) } return lines.join('\n') } diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index c07bd21..52cc2f2 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -14,6 +14,7 @@ import { openingHtmlBlock, setextHeadingLevel, } from '../commonmark-grammar.ts' +import { isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts' import { malformedDirective, readDirectiveLine } from '../directive-syntax.ts' import { readLinkDefinitions } from './link-reference-definitions.ts' @@ -28,6 +29,7 @@ export type Block = | { kind: 'heading'; level: number; text: string } | { kind: 'paragraph'; text: string } | { kind: 'rule' } + | { kind: 'table'; rows: string[][] } export type ParsedBlocks = { blocks: Block[]; definitions: Map } @@ -40,15 +42,16 @@ type OpenDirective = { blocks: Block[]; colons: number; index: number; kind: 'di type OpenContainer = | Extract | OpenDirective - | { blocks: Block[]; indentation: number; kind: 'item'; list: ListBlock; marker: string } + | { blocks: Block[]; indentation: number; kind: 'item'; list: ListBlock } type OpenLeaf = | { closer: RegExp | undefined; construct: string; kind: 'html' } | { held: string[]; kind: 'indented-code'; lines: string[] } | { indentation: number; info: string; kind: 'fenced-code'; lines: string[]; marker: string } | { kind: 'paragraph'; lines: string[] } + | { kind: 'pipe-table'; rows: [string[], ...string[][]] } -type ContainerStart = { kind: 'blockquote'; rest: Line } | { fresh: boolean; indentation: number; kind: 'item'; list: ListBlock; marker: string; rest: Line } +type ContainerStart = { kind: 'blockquote'; rest: Line } | { fresh: boolean; indentation: number; kind: 'item'; list: ListBlock; rest: Line } // The line from an absolute column on: a tab a cut splits keeps the stop it is measured against. type Line = { column: number; text: string } @@ -151,13 +154,12 @@ function itemStart(line: Line, opener: Line, paragraphOpen: boolean, enclosing: const spaces = leadingColumns(after) const padding = blank || spaces > indentedCodeColumns ? 1 : spaces const kind = marker.start === undefined ? 'bulletList' : 'orderedList' - const continued = enclosing?.kind === 'item' && enclosing.list.kind === kind && enclosing.marker === marker.delimiter + const continued = enclosing?.kind === 'item' && enclosing.list.kind === kind return { fresh: !continued, indentation: leadingColumns(line) + marker.width + padding, kind: 'item', list: continued ? enclosing.list : openList(marker.start), - marker: marker.delimiter, rest: blank ? after : removeColumns(after, padding), } } @@ -176,7 +178,7 @@ function openContainer(walk: Walk, start: ContainerStart): void { } if (start.fresh) currentBlocks(walk).push(start.list) start.list.items.push(blocks) - walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list: start.list, marker: start.marker }) + walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list: start.list }) } function closeContainers(walk: Walk, depth: number): void { @@ -262,6 +264,14 @@ function readBlockLine(walk: Walk, line: Line): void { if (leaf.closer === undefined ? blankLine.test(line.text) : leaf.closer.test(line.text)) closeLeaf(walk) return } + if (leaf?.kind === 'pipe-table') { + const cells = pipeCells(removeColumns(line, largestOpenerIndentation).text) + if (cells !== undefined) { + leaf.rows.push(cells) + return + } + closeLeaf(walk) + } if (leaf?.kind === 'indented-code') { if (readIndentedCodeLine(leaf, line)) return closeLeaf(walk) @@ -297,9 +307,10 @@ function openLeaf(walk: Walk, line: Line): void { else pushFault(walk, directive.fault) return } - if (claimsPipeLine(opener)) { + const cells = pipeCells(opener) + if (cells !== undefined) { closeLeaf(walk) - pushFault(walk, { code: 'malformed-pipe-table', message: 'the line claims a pipe table and parses as none' }) + walk.leaf = { kind: 'pipe-table', rows: [cells] } return } if (readLineBlock(walk, opener)) return @@ -358,9 +369,28 @@ function closeLeaf(walk: Walk): void { } walk.leaf = undefined if (leaf.kind === 'html') currentBlocks(walk).push({ construct: leaf.construct, kind: 'html' }) + else if (leaf.kind === 'pipe-table') currentBlocks(walk).push(pipeTableBlock(leaf.rows)) else currentBlocks(walk).push({ kind: 'code', language: leaf.kind === 'fenced-code' ? decodeTextEscapes(leaf.info) : '', text: leaf.lines.join('\n') }) } +// spec/flavour.md, Tables: the delimiter row underlines the header and leaves the body its cell count. +function pipeTableBlock(rows: readonly [string[], ...string[][]]): Block { + const [header, delimiter, ...body] = rows + if (delimiter !== undefined && delimiter.some(isPipeAlignment)) { + return faultedBlock('a pipe table carries no column alignment ADF could hold') + } + if (delimiter === undefined || !delimiter.every(isPipeDelimiter)) { + return faultedBlock('a pipe table underlines its header with a row of `-` runs') + } + const ragged = [delimiter, ...body].find((row) => row.length !== header.length) + if (ragged !== undefined) return faultedBlock(`a pipe table row holds ${ragged.length} cells where its header holds ${header.length}`) + return { kind: 'table', rows: [header, ...body] } +} + +function faultedBlock(message: string): Block { + return { fault: malformedPipeTable(message), kind: 'fault' } +} + function takeParagraph(walk: Walk): string | undefined { const leaf = walk.leaf if (leaf?.kind !== 'paragraph') return undefined diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 91c3ad1..5798838 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -67,6 +67,18 @@ function image(url: string, alt?: string): AdfNode { return { attrs: { layout: 'center' }, content: [media], type: 'mediaSingle' } } +function cell(type: string, ...content: AdfNode[]): AdfNode { + return { content: [content.length === 0 ? { type: 'paragraph' } : { content, type: 'paragraph' }], type } +} + +function row(...cells: AdfNode[]): AdfNode { + return { content: cells, type: 'tableRow' } +} + +function table(...rows: AdfNode[]): AdfNode { + return { content: rows, type: 'table' } +} + test('builds an empty document from input holding no block', () => { assert.deepEqual(markdownToAdf(''), { ok: true, value: { type: 'doc', version: 1 } }) assert.deepEqual(content(markdownToAdf('\n \n\t\n')), []) @@ -114,6 +126,67 @@ test('reads a fenced code block, its info string the language', () => { assert.deepEqual(content(markdownToAdf('```\n- x\n> y\n```\n')), [{ content: [text('- x\n> y')], type: 'codeBlock' }]) }) +test('reads the codeBlock directive body as the node content, the info string its language', () => { + const fenced = ':::codeBlock {wrap=true}\n```rust\nfn main() {}\n```\n:::\n' + assert.deepEqual(content(markdownToAdf(fenced)), [ + { attrs: { language: 'rust', wrap: true }, content: [text('fn main() {}')], type: 'codeBlock' }, + ]) + assert.deepEqual(content(markdownToAdf(':::codeBlock {wrap=true}\n```\n```\n:::\n')), [{ attrs: { wrap: true }, type: 'codeBlock' }]) + assert.deepEqual(content(markdownToAdf(':::codeBlock {language=""}\n```\nx\n```\n:::\n')), [ + { attrs: { language: '' }, content: [text('x')], type: 'codeBlock' }, + ]) + // The body is a CommonMark fence, so its info string decodes escapes the way any other fence's does. + assert.deepEqual(content(markdownToAdf(':::codeBlock {wrap=true}\n```\\#c\nx\n```\n:::\n')), [ + { attrs: { language: '#c', wrap: true }, content: [text('x')], type: 'codeBlock' }, + ]) +}) + +test('names the slot a codeBlock spells its language outside of', () => { + const slot = 'unsupported-node-shape: codeBlock spells its language in the fence info string, or in the attribute where no info string carries it back' + assert.equal(content(markdownToAdf(':::codeBlock {language=rust wrap=true}\n```\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf(':::codeBlock {language=rust}\n```sql\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\n```adf\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\n```a\\b\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf('::codeBlock {wrap=true}\n')), 'unsupported-node-shape: codeBlock spells its body in the container form, :::') +}) + +test('reads a pipe table into the header row and the body rows under it', () => { + const pipes = '| Part | Note |\n| --- | --- |\n| Nut \\| washer | `8.8` |\n| Spare | |\n' + assert.deepEqual(content(markdownToAdf(pipes)), [ + table( + row(cell('tableHeader', text('Part')), cell('tableHeader', text('Note'))), + row(cell('tableCell', text('Nut | washer')), cell('tableCell', codeSpan('8.8'))), + row(cell('tableCell', text('Spare')), cell('tableCell')), + ), + ]) + assert.deepEqual(content(markdownToAdf('| Part\n| -\n')), [table(row(cell('tableHeader', text('Part'))))]) + assert.deepEqual(content(markdownToAdf(' | Part |\n | --- |\n')), [table(row(cell('tableHeader', text('Part'))))]) +}) + +test('claims the line a pipe opens and gives the rest back to the block walk', () => { + const header = table(row(cell('tableHeader', text('a')))) + assert.deepEqual(content(markdownToAdf('Part.\n| a |\n| --- |\n')), [paragraph('Part.'), header]) + assert.deepEqual(content(markdownToAdf('| a |\n| --- |\nPart.\n')), [header, paragraph('Part.')]) + assert.deepEqual(content(markdownToAdf('> | a |\n> | --- |\n')), [quote(header)]) + assert.deepEqual(content(markdownToAdf('- | a |\n | --- |\n')), [bulletList(item(header))]) + assert.deepEqual(content(markdownToAdf('| a |\n| --- |\n x\n')), [header, { content: [text('x')], type: 'codeBlock' }]) + assert.deepEqual(content(markdownToAdf('\\| a |\n')), [paragraph('| a |')]) +}) + +test('names the pipe table a claimed line does not spell', () => { + assert.equal(content(markdownToAdf('| a | b |\n')), 'malformed-pipe-table: a pipe table underlines its header with a row of `-` runs') + assert.equal(content(markdownToAdf('| a |\n| x |\n')), 'malformed-pipe-table: a pipe table underlines its header with a row of `-` runs') + assert.equal(content(markdownToAdf('| a | b |\n| :--- | ---: |\n')), 'malformed-pipe-table: a pipe table carries no column alignment ADF could hold') + assert.equal(content(markdownToAdf('| a | b |\n| --- |\n')), 'malformed-pipe-table: a pipe table row holds 1 cells where its header holds 2') + assert.equal(content(markdownToAdf('| a |\n| --- |\n| b | c |\n')), 'malformed-pipe-table: a pipe table row holds 2 cells where its header holds 1') + assert.deepEqual(path(markdownToAdf('Part.\n\n| a |\n')), ['content', 1]) +}) + +test('refuses the image a pipe cell holds no ADF node for', () => { + assert.equal(content(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') + assert.deepEqual(path(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), ['content', 0, 'content', 1, 'content', 0, 'content', 0]) +}) + test('strips the opening fence indentation from the content lines it holds', () => { assert.deepEqual(content(markdownToAdf(' ```\n x\n y\n ```\n')), [{ content: [text(' x\ny')], type: 'codeBlock' }]) assert.deepEqual(content(markdownToAdf(' ```\n\tx\n ```\n')), [{ content: [text(' x')], type: 'codeBlock' }]) @@ -161,8 +234,8 @@ test('names the directive form a node CommonMark spells refuses', () => { // The spelling the emitter refuses gives the emitter's own error, never a second name for it. test('gives back the refusal the CommonMark spelling itself raises', () => { - const nested = '::::::::bulletList\n:::::::listItem\n---\n\n::::::bulletList\n:::::listItem\n---\n\n::::bulletList\n:::listItem\n---\n:::\n::::\n:::::\n::::::\n:::::::\n::::::::\n' - assert.equal(code(markdownToAdf(nested)), 'unspelled-block-separation') + const destination = ':::blockquote\n[t](https://example.com/a\\b)\n:::\n' + assert.equal(content(markdownToAdf(destination)), 'unspellable-link-destination: no canonical escape spells a backslash in a link destination') }) test('names the directive name no node reads back to', () => { @@ -232,7 +305,7 @@ test('names the argument and the body a node takes no reading for', () => { assert.equal(content(markdownToAdf(':::paragraph\n:::\n')), 'unsupported-node-shape: an empty paragraph takes the leaf form, ::') assert.equal(content(markdownToAdf(':::paragraph\nOne.\n\nTwo.\n:::\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body') assert.equal(content(markdownToAdf(':::paragraph\n---\n:::\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body') - assert.equal(content(markdownToAdf(':::codeBlock\n```\nx\n```\n:::\n')), 'unsupported-node-shape: the fenced body of codeBlock is unsupported') + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\nx\n:::\n')), 'unsupported-node-shape: codeBlock takes one fenced code block as its body') assert.equal(content(markdownToAdf(':::paragraph\n![a](/u)\n:::\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.equal(content(markdownToAdf('Part :date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content') assert.equal( @@ -315,14 +388,15 @@ test('reads a bullet list, the marker width setting the continuation', () => { assert.deepEqual(content(markdownToAdf('-\n')), [bulletList(item())]) assert.deepEqual(content(markdownToAdf('- One\n\n Two.\n')), [bulletList(item(paragraph('One'), paragraph('Two.')))]) assert.deepEqual(content(markdownToAdf('- Code.\n')), [bulletList(item({ content: [text('Code.')], type: 'codeBlock' }))]) - assert.deepEqual(content(markdownToAdf('- a\n* b\n')), [bulletList(item(paragraph('a'))), bulletList(item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('- a\n* b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('- a\n\n+ b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('-\n\n Part.\n')), [bulletList(item()), paragraph('Part.')]) }) test('reads an ordered list, its first marker the order attribute', () => { assert.deepEqual(content(markdownToAdf('9. Bolt M8\n10. Nut M8\n')), [orderedList(9, item(paragraph('Bolt M8')), item(paragraph('Nut M8')))]) assert.deepEqual(content(markdownToAdf('1) Loosen the clamp\n')), [orderedList(1, item(paragraph('Loosen the clamp')))]) - assert.deepEqual(content(markdownToAdf('1. a\n1) b\n')), [orderedList(1, item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('1. a\n1) b\n')), [orderedList(1, item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('0. Zero\n')), [orderedList(0, item(paragraph('Zero')))]) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 2aa7782..f49c107 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -5,6 +5,7 @@ import type { LinkDefinitions } from './inline-content.ts' import { carryName } from '../opaque-carry.ts' import { commonMarkSpelling } from '../emit/adf-to-markdown.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' +import { fenceInfo } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' @@ -50,6 +51,8 @@ function blockNode(block: Block, definitions: LinkDefinitions, path: ConvertErro return paragraphNode(block.text, definitions, path) case 'rule': return success({ type: 'rule' }) + case 'table': + return tableNode(block.rows, definitions, path) } } @@ -71,11 +74,40 @@ function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, de return failure('unsupported-node-shape', `${node.type} spells its body in the container form, :::`, path) } if (contentModel === 'none') return failure('unsupported-node-shape', `${node.type} holds no content`, path) - if (contentModel === 'code') return failure('unsupported-node-shape', `the fenced body of ${node.type} is unsupported`, path) + if (contentModel === 'code') return codeDirectiveNode(node, blocks, path) if (contentModel === 'block') return containerNode(node, blocks, definitions, path, depth) return inlineBodyNode(node, blocks, definitions, path) } +// spec/flavour.md, The CommonMark blocks: the language rides the one slot fenceInfo picks for it. +function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: ConvertErrorPath): Result { + const only = blocks.length === 1 ? blocks[0] : undefined + if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one fenced code block as its body`, path) + const attribute = node.attrs?.['language'] + const fromFence = only.language !== '' + const info = fenceInfo(fromFence ? only.language : attribute) + if ((info !== undefined && info !== '') !== fromFence || (fromFence && attribute !== undefined)) { + return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) + } + const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: only.language } } : node + return success(withContent(spelled, only.text === '' ? [] : [{ text: only.text, type: 'text' }])) +} + +function tableNode(rows: readonly string[][], definitions: LinkDefinitions, path: ConvertErrorPath): Result { + const content: AdfNode[] = [] + for (const [rowIndex, cells] of rows.entries()) { + const type = rowIndex === 0 ? 'tableHeader' : 'tableCell' + const row: AdfNode[] = [] + for (const [cellIndex, cell] of cells.entries()) { + const paragraph = contentNode({ type: 'paragraph' }, cell, definitions, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0]) + if (!paragraph.ok) return paragraph + row.push({ content: [paragraph.value], type }) + } + content.push({ content: row, type: 'tableRow' }) + } + return success({ content, type: 'table' }) +} + function inlineBodyNode(node: AdfNode, blocks: readonly Block[], definitions: LinkDefinitions, path: ConvertErrorPath): Result { if (blocks.length === 0) return failure('unsupported-node-shape', `an empty ${node.type} takes the leaf form, ::`, path) const only = blocks.length === 1 ? blocks[0] : undefined diff --git a/src/markdown/pipe-table-syntax.ts b/src/markdown/pipe-table-syntax.ts new file mode 100644 index 0000000..5935d40 --- /dev/null +++ b/src/markdown/pipe-table-syntax.ts @@ -0,0 +1,48 @@ +import type { ConvertFault } from '../result.ts' +import { backslashEscape, claimsPipeLine, trimSpace } from './commonmark-grammar.ts' + +const alignmentCell = /^:-+:?$|^-+:$/ +const delimiterCell = /^-+$/ + +export function isPipeAlignment(cell: string): boolean { + return alignmentCell.test(cell) +} + +export function isPipeDelimiter(cell: string): boolean { + return delimiterCell.test(cell) +} + +export function malformedPipeTable(message: string): ConvertFault { + return { code: 'malformed-pipe-table', message } +} + +// spec/flavour.md, Tables: the cells of a claimed row, the closing `|` the spelling writes optional here. +export function pipeCells(line: string): string[] | undefined { + if (!claimsPipeLine(line)) return undefined + const row = line.replace(/[ \t]+$/, '') + const cells: string[] = [] + let start = 1 + let index = 1 + while (index < row.length) { + if (backslashEscape(row, index) !== undefined) { + index += 2 + continue + } + if (row.charAt(index) === '|') { + cells.push(trimSpace(row.slice(start, index))) + start = index + 1 + } + index += 1 + } + cells.push(trimSpace(row.slice(start))) + if (cells.length > 1 && cells.at(-1) === '') cells.pop() + return cells +} + +export function spellPipeDelimiter(columns: number): string { + return spellPipeRow(Array.from({ length: columns }, () => '---')) +} + +export function spellPipeRow(cells: readonly string[]): string { + return `| ${cells.join(' | ')} |` +} diff --git a/src/result.ts b/src/result.ts index 4ad939c..316e778 100644 --- a/src/result.ts +++ b/src/result.ts @@ -11,7 +11,6 @@ export type ConvertErrorCode = | 'unspellable-link-destination' | 'unspellable-link-title' | 'unspellable-whitespace' - | 'unspelled-block-separation' | 'unsupported-document-version' | 'unsupported-nesting-depth' | 'unsupported-node-shape' diff --git a/todo-history.md b/todo-history.md index c6937d9..ab162a3 100644 --- a/todo-history.md +++ b/todo-history.md @@ -281,7 +281,28 @@ Under **3 — `markdownToAdf` (`0.1.0`)**: `spec/flavour.md`'s closing-fence sentence now says: a run reaching past the innermost leaves the fence it did not close a named error, which §2 prefers to closing more than the author wrote. - - [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first + - [x] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the + `language` attribute a bare fence leaves it; the media family's composition; and both + table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both + directions answer alike and moves to the `markdown/` root with the language attribute. + **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one + `container-block-separation` holds — a CommonMark block and a directive block sit adjacent + in a container body with no blank line between them. That reduces the three cases to one + rule, separation only where its absence would merge the blocks: the `:::` fence is + separation already, and 3c's claim ends the lazy continuation that would otherwise swallow + it. The fixture becomes a round-trip pair, and with `nested-list-separation` and 3e's pair + that empties `corpus/unspellable/`: this chunk settles the directory's own guard in + `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. + The emitter's other refusals survive on causes no fixture in that directory covers, so + 3k's one-list pass is where they get fixtures or the directory goes. + **Settled** (the maintainer, 2026-09-01): losing that cause closed one of the two shapes + input accepted and emit refused, not the last. The other surfaced here — CommonMark opens a + second list on a marker change, so `- a` over `* b` built two adjacent `bulletList` nodes + `adfToMarkdown` refuses — and the parser continues the list instead, the way it already + drops the blank lines between items. With both closed the parse-then-emit fixpoint the + README promises holds for every construct the parser reads, rather than only for what the + emitter wrote. +- [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first directory to. A parsed directive becomes its node: the name to the type and an unknown one to a named error, the arg to the attribute it names, each value to the type its section assigns, the body to `content`, the reserved `marks` key to the marks array. 3a's drift diff --git a/todo.md b/todo.md index 751786a..97e47e7 100644 --- a/todo.md +++ b/todo.md @@ -49,23 +49,7 @@ numbering is the order the work was planned in, not the order it ships. - [x] **3e — Emphasis and links.** - [x] **3f — The directive grammar.** - [x] **3g — The node tables read backwards.** - - [ ] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the - `language` attribute a bare fence leaves it; the media family's composition; and both - table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both - directions answer alike and moves to the `markdown/` root with the language attribute. - **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one - `container-block-separation` holds — a CommonMark block and a directive block sit adjacent - in a container body with no blank line between them. That reduces the three cases to one - rule, separation only where its absence would merge the blocks: the `:::` fence is - separation already, and 3c's claim ends the lazy continuation that would otherwise swallow - it. The fixture becomes a round-trip pair, and with `nested-list-separation` and 3e's pair that - empties `corpus/unspellable/`: this chunk settles the directory's own guard in - `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. - The emitter's other refusals survive on causes no fixture in that directory covers, so - 3k's one-list pass is where they get fixtures or the directory goes. - Losing that cause closes the last shape input accepts and emit refuses — a CommonMark block - beside a directive one inside a list item — so the parse-then-emit fixpoint the README - promises holds from here rather than only for what the emitter wrote. + - [x] **3h — The block nodes.** - [ ] **3i — The inline nodes and the marks.** `inline-nodes/` reads back: the content slot's `text` attribute and the error a slot holding anything but one unmarked text node is; the `:text{text="…"}` whitespace spelling; the four directive marks and their nesting order, @@ -102,10 +86,12 @@ numbering is the order the work was planned in, not the order it ships. 3e collapses a spelling nested inside its own kind and `*(*a*)*` is two `` against one `em`. The fixpoint alone is self-consistency a parser returning the empty document passes, and the text alone one dropping every emphasis; the - counts close both. The exception list stays the maintainer's. One outcome is no + counts close both. The exception list stays the maintainer's, and one entry is owed + already: 3h continues a list across the marker change CommonMark splits on, so an example + the reference HTML gives two `
    ` counts one `bulletList`. One outcome is no exception and must not be filed as one: valid CommonMark parsing to a document - `adfToMarkdown` refuses is a §2 hole, which is what three of `corpus/unspellable/`'s four - hold until 3c, 3e and 3h land their answers. + `adfToMarkdown` refuses is a §2 hole, which is what `corpus/unspellable/` held until 3c, + 3e and 3h landed their answers and emptied it. - [ ] **4 — Round-trip property tests (`0.2.0`)**, widening 3j's corpus round-trip past the documents a human wrote — the thing that proves 2 and 3 beyond them. Editor-normal (§2) is finished here, on 3i's merging — `toEditorNormal(doc)` and the equality the round-trip @@ -133,9 +119,9 @@ numbering is the order the work was planned in, not the order it ships. another node's. `0.1.0` ships with the retry in it, so a deep document is slow rather than wrong until the patch. - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, - the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: every - `corpus/unspellable/` document is a decision or a deferred trigger this file names, so the - directory empties as they land and whatever survives is permanent. The parser's own code + the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: 3h + landed the last decision `corpus/unspellable/` held and the directory went with it, so what + the code list holds from here is permanent. The parser's own code additions are read here as one list before that freeze — nine sessions mint them independently, and one cause wearing two codes is breaking to undo after `0.1.0`. `0.1.0` is the markdown round-trip: both markdown directions, the types, `isAdfDocument`. The build