From 2527d1e3d8a18e503e1d0fe3749d95eb5d19dba7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 1 Sep 2026 17:58:29 +0200 Subject: [PATCH 1/5] Read the block nodes back, and stop a marker change splitting a list --- corpus/README.md | 3 - corpus/errors/code-block-body.error | 1 + corpus/errors/code-block-body.md | 3 + corpus/errors/code-block-language-slot.error | 1 + corpus/errors/code-block-language-slot.md | 5 ++ corpus/errors/pipe-alignment-colon.error | 1 + corpus/errors/pipe-alignment-colon.md | 3 + corpus/errors/pipe-cell-count.error | 1 + corpus/errors/pipe-cell-count.md | 3 + corpus/normalization/loose-list.md | 2 +- .../container-block-separation.json | 0 .../block-nodes/container-block-separation.md | 6 ++ .../container-block-separation.error | 1 - spec/flavour.md | 6 +- src/corpus.test.ts | 15 +--- src/markdown/code-language.ts | 12 +++ src/markdown/emit/adf-to-markdown.test.ts | 8 +- src/markdown/emit/adf-to-markdown.ts | 23 +---- src/markdown/emit/pipe-table.ts | 5 +- src/markdown/parse/blocks.ts | 44 ++++++++-- src/markdown/parse/markdown-to-adf.test.ts | 84 +++++++++++++++++-- src/markdown/parse/markdown-to-adf.ts | 34 +++++++- src/markdown/pipe-table-syntax.ts | 48 +++++++++++ src/result.ts | 1 - todo-history.md | 23 ++++- todo.md | 32 ++----- 26 files changed, 282 insertions(+), 83 deletions(-) create mode 100644 corpus/errors/code-block-body.error create mode 100644 corpus/errors/code-block-body.md create mode 100644 corpus/errors/code-block-language-slot.error create mode 100644 corpus/errors/code-block-language-slot.md create mode 100644 corpus/errors/pipe-alignment-colon.error create mode 100644 corpus/errors/pipe-alignment-colon.md create mode 100644 corpus/errors/pipe-cell-count.error create mode 100644 corpus/errors/pipe-cell-count.md rename corpus/{unspellable => round-trip/block-nodes}/container-block-separation.json (100%) create mode 100644 corpus/round-trip/block-nodes/container-block-separation.md delete mode 100644 corpus/unspellable/container-block-separation.error create mode 100644 src/markdown/code-language.ts create mode 100644 src/markdown/pipe-table-syntax.ts diff --git a/corpus/README.md b/corpus/README.md index 8aa572c..d84d3aa 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -9,9 +9,6 @@ One directory per contract kind, each landing with its milestone: `markdownToAdf` must build from it. One-way; the markdown is not canonical. - `errors/` — `.md`: markdown input that must not convert. A `.error` beside it pins which error. -- `unspellable/` — `.json`: ADF `adfToMarkdown` must refuse, the `ConvertErrorCode` in the - `.error` beside it. Two populations live here: refusals that stay contract, and documents - a maintainer decision (`todo.md`) moves to `round-trip/`. - `real-payloads/` — `.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No expected markdown. diff --git a/corpus/errors/code-block-body.error b/corpus/errors/code-block-body.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-body.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-body.md b/corpus/errors/code-block-body.md new file mode 100644 index 0000000..35990e4 --- /dev/null +++ b/corpus/errors/code-block-body.md @@ -0,0 +1,3 @@ +:::codeBlock {wrap=true} +fn main() {} +::: diff --git a/corpus/errors/code-block-language-slot.error b/corpus/errors/code-block-language-slot.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-language-slot.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-language-slot.md b/corpus/errors/code-block-language-slot.md new file mode 100644 index 0000000..d08ee3b --- /dev/null +++ b/corpus/errors/code-block-language-slot.md @@ -0,0 +1,5 @@ +:::codeBlock {language=rust wrap=true} +``` +fn main() {} +``` +::: diff --git a/corpus/errors/pipe-alignment-colon.error b/corpus/errors/pipe-alignment-colon.error new file mode 100644 index 0000000..5bf92ee --- /dev/null +++ b/corpus/errors/pipe-alignment-colon.error @@ -0,0 +1 @@ +malformed-pipe-table diff --git a/corpus/errors/pipe-alignment-colon.md b/corpus/errors/pipe-alignment-colon.md new file mode 100644 index 0000000..8231165 --- /dev/null +++ b/corpus/errors/pipe-alignment-colon.md @@ -0,0 +1,3 @@ +| Part | Qty | +| :--- | ---: | +| Bolt M8 | 40 | diff --git a/corpus/errors/pipe-cell-count.error b/corpus/errors/pipe-cell-count.error new file mode 100644 index 0000000..5bf92ee --- /dev/null +++ b/corpus/errors/pipe-cell-count.error @@ -0,0 +1 @@ +malformed-pipe-table diff --git a/corpus/errors/pipe-cell-count.md b/corpus/errors/pipe-cell-count.md new file mode 100644 index 0000000..3b1f464 --- /dev/null +++ b/corpus/errors/pipe-cell-count.md @@ -0,0 +1,3 @@ +| Part | Qty | +| --- | --- | +| Bolt M8 | diff --git a/corpus/normalization/loose-list.md b/corpus/normalization/loose-list.md index 97056bc..ea18ebd 100644 --- a/corpus/normalization/loose-list.md +++ b/corpus/normalization/loose-list.md @@ -1,6 +1,6 @@ * Bolt M8 -* Nut M8 ++ Nut M8 Then: diff --git a/corpus/unspellable/container-block-separation.json b/corpus/round-trip/block-nodes/container-block-separation.json similarity index 100% rename from corpus/unspellable/container-block-separation.json rename to corpus/round-trip/block-nodes/container-block-separation.json diff --git a/corpus/round-trip/block-nodes/container-block-separation.md b/corpus/round-trip/block-nodes/container-block-separation.md new file mode 100644 index 0000000..a4bbc97 --- /dev/null +++ b/corpus/round-trip/block-nodes/container-block-separation.md @@ -0,0 +1,6 @@ +::::expand {title="Full build log"} +The build ran for 11 minutes. +:::panel warning +Three warnings went unread. +::: +:::: diff --git a/corpus/unspellable/container-block-separation.error b/corpus/unspellable/container-block-separation.error deleted file mode 100644 index 7fdd935..0000000 --- a/corpus/unspellable/container-block-separation.error +++ /dev/null @@ -1 +0,0 @@ -unspelled-block-separation diff --git a/spec/flavour.md b/spec/flavour.md index 73f4085..68bc70a 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -22,7 +22,8 @@ normalizes to it through the round-trip. between a nested list and a CommonMark block above it — one wherever the nested list's own marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or a list whose first item is empty), whatever block sits above it. Blank lines between items normalize - away; ADF does not record tightness. + away, and so does the marker change CommonMark starts a second list on: ADF records no + tightness, and one `- ` spelling leaves two adjacent lists of a type no way back. - Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`. - ATX headings (`#` … `######`); setext input normalizes to ATX. - Code fences ``` with the node's language as info string, the fence lengthened past any backtick @@ -190,7 +191,8 @@ form. (string), `uniqueId` (string), `wrap` (boolean). A language no info string carries back — empty, the reserved `adf`, or holding a backtick, a backslash, a control character, edge whitespace or an entity reference — rides the `language` attribute instead and the fence carries no info - string; writing both is a named error. + string; writing it in the slot that rule leaves empty, or in both, is a named error. The body is + an ordinary fence, so its info string decodes escapes and entity references as any other does. - `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no CommonMark spelling. diff --git a/src/corpus.test.ts b/src/corpus.test.ts index 161c551..adb9439 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -14,11 +14,10 @@ const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus') const errorsRoot = join(corpusRoot, 'errors') const normalizationRoot = join(corpusRoot, 'normalization') const roundTripRoot = join(corpusRoot, 'round-trip') -const unspellableRoot = join(corpusRoot, 'unspellable') const emittingDirectories = ['block-nodes', 'combinations', 'commonmark-subset', 'inline-nodes', 'opaque-carry'] // A directory joins once every fixture in it reads back to its document. -const parsingDirectories = ['commonmark-subset'] +const parsingDirectories = ['block-nodes', 'commonmark-subset'] function directoryNames(root: string): string[] { return readdirSync(root, { withFileTypes: true }) @@ -61,7 +60,7 @@ function corpusJsonPaths(): string[] { } test('every corpus directory is a kind the runner reads', () => { - assert.deepEqual(directoryNames(corpusRoot), ['errors', 'normalization', 'round-trip', 'unspellable']) + assert.deepEqual(directoryNames(corpusRoot), ['errors', 'normalization', 'round-trip']) }) test('every round-trip directory emits', () => { @@ -187,16 +186,6 @@ for (const directory of emittingDirectories) { } } -for (const name of pairedNames(unspellableRoot, '.json', '.error')) { - test(`unspellable/${name} is refused with the error it names`, () => { - const parsed: unknown = JSON.parse(readFileSync(join(unspellableRoot, `${name}.json`), 'utf8')) - assert.ok(isAdfDocument(parsed), `${name}.json is not an ADF document`) - const result = adfToMarkdown(parsed) - assert.ok(!result.ok, result.ok ? `emitted ${JSON.stringify(result.value)}` : '') - assert.equal(result.error.code, readFileSync(join(unspellableRoot, `${name}.error`), 'utf8').trimEnd()) - }) -} - for (const name of pairedNames(normalizationRoot, '.md', '.json')) { test(`normalization/${name} parses to the document beside it`, () => { const expected: unknown = JSON.parse(readFileSync(join(normalizationRoot, `${name}.json`), 'utf8')) diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts new file mode 100644 index 0000000..d254371 --- /dev/null +++ b/src/markdown/code-language.ts @@ -0,0 +1,12 @@ +import type { JsonValue } from '../json-value.ts' +import { carryName } from './opaque-carry.ts' +import { holdsControlCharacter } from './commonmark-grammar.ts' +import { holdsEntityReference } from './entity-references.ts' + +// spec/flavour.md, The CommonMark blocks: the info string the language rides, `undefined` where the attribute carries it. +export function fenceInfo(language: JsonValue | undefined): string | undefined { + if (language === undefined) return '' + if (typeof language !== 'string' || language === '' || language === carryName) return undefined + if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return undefined + return language +} diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index aebacc5..4eea9d0 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -472,15 +472,15 @@ test('refuses the content a directive body has no room for', () => { assert.equal(code(adfToMarkdown(document({ text: 'x', type: 'panel' }))), 'unsupported-node-shape') }) -test('separates two directive blocks in a container body by one line, two CommonMark blocks by a blank one', () => { +test('separates blocks in a container body by a blank line only where the fence is not separation already', () => { const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) const panel = (...content: AdfNode[]): AdfDocument => document({ attrs: { panelType: 'info' }, content, type: 'panel' }) assert.equal(markdown(adfToMarkdown(panel(text('a'), text('b')))), ':::panel info\na\n\nb\n:::\n') const caption: AdfNode = { content: [{ text: 'c', type: 'text' }], type: 'caption' } assert.equal(markdown(adfToMarkdown(panel(caption, caption))), '::::panel info\n:::caption\nc\n:::\n:::caption\nc\n:::\n::::\n') - assert.equal(code(adfToMarkdown(panel(text('a'), caption))), 'unspelled-block-separation') - assert.equal(code(adfToMarkdown(panel(caption, text('a')))), 'unspelled-block-separation') - assert.equal(code(adfToMarkdown(panel(paragraph(), text('a')))), 'unspelled-block-separation') + assert.equal(markdown(adfToMarkdown(panel(text('a'), caption))), '::::panel info\na\n:::caption\nc\n:::\n::::\n') + assert.equal(markdown(adfToMarkdown(panel(caption, text('a')))), '::::panel info\n:::caption\nc\n:::\na\n::::\n') + assert.equal(markdown(adfToMarkdown(panel(paragraph(), text('a')))), ':::panel info\n::paragraph\na\n:::\n') }) test('spells the image form for exactly the centered external media shape', () => { diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index b1a5076..896d40d 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,14 +1,13 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockDirective } from '../../adf/block-directives.ts' -import type { JsonValue } from '../../json-value.ts' import { blockDirective } from '../../adf/block-directives.ts' -import { carriedBlock, carryName } from '../opaque-carry.ts' +import { carriedBlock } from '../opaque-carry.ts' import { carriesOnly, isAdfDocument } from '../../adf/document.ts' import { emitInlineLine } from './inline-line.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { fencedCodeBlock } from '../backtick-runs.ts' -import { holdsControlCharacter, holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' -import { holdsEntityReference } from '../entity-references.ts' +import { fenceInfo } from '../code-language.ts' +import { holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' import { largestNesting } from '../../nesting.ts' import { spellDirectiveHeader } from './block-directive-spelling.ts' import { tryImage } from './image.ts' @@ -62,13 +61,7 @@ function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: } if (container === 'list-item') return success(interruptsParagraph(next.node) ? '\n' : '\n\n') } - if (container !== 'directive' || plainPair) return success('\n\n') - if (previous.spelling === 'directive' && next.spelling === 'directive') return success('\n') - return failure( - 'unspelled-block-separation', - `the canonical form leaves the separation between a ${previous.spelling} and a ${next.spelling} block in a container body unspelled`, - next.path, - ) + return success(container === 'directive' && !plainPair ? '\n' : '\n\n') } function interruptsParagraph(node: AdfNode): boolean { @@ -191,14 +184,6 @@ function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result { return success(text) } -// spec/flavour.md, The CommonMark blocks. -function fenceInfo(language: JsonValue | undefined): string | undefined { - if (language === undefined) return '' - if (typeof language !== 'string' || language === '' || language === carryName) return undefined - if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return undefined - return language -} - function emitHeading(node: AdfNode, path: ConvertErrorPath): Result | undefined { if (!carriesOnly(node, ['level'])) return undefined const level = node.attrs?.['level'] diff --git a/src/markdown/emit/pipe-table.ts b/src/markdown/emit/pipe-table.ts index 6439b44..c47ca57 100644 --- a/src/markdown/emit/pipe-table.ts +++ b/src/markdown/emit/pipe-table.ts @@ -1,5 +1,6 @@ import type { AdfNode } from '../../adf/document.ts' import { carriesOnly } from '../../adf/document.ts' +import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts' import { tryPipeCell } from './inline-line.ts' import type { ConvertErrorPath } from '../../result.ts' @@ -15,8 +16,8 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath): string | un if (line === undefined) return undefined cells.push(line) } - lines.push(`| ${cells.join(' | ')} |`) - if (rowIndex === 0) lines.push(`| ${cells.map(() => '---').join(' | ')} |`) + lines.push(spellPipeRow(cells)) + if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length)) } return lines.join('\n') } diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index c07bd21..52cc2f2 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -14,6 +14,7 @@ import { openingHtmlBlock, setextHeadingLevel, } from '../commonmark-grammar.ts' +import { isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts' import { malformedDirective, readDirectiveLine } from '../directive-syntax.ts' import { readLinkDefinitions } from './link-reference-definitions.ts' @@ -28,6 +29,7 @@ export type Block = | { kind: 'heading'; level: number; text: string } | { kind: 'paragraph'; text: string } | { kind: 'rule' } + | { kind: 'table'; rows: string[][] } export type ParsedBlocks = { blocks: Block[]; definitions: Map } @@ -40,15 +42,16 @@ type OpenDirective = { blocks: Block[]; colons: number; index: number; kind: 'di type OpenContainer = | Extract | OpenDirective - | { blocks: Block[]; indentation: number; kind: 'item'; list: ListBlock; marker: string } + | { blocks: Block[]; indentation: number; kind: 'item'; list: ListBlock } type OpenLeaf = | { closer: RegExp | undefined; construct: string; kind: 'html' } | { held: string[]; kind: 'indented-code'; lines: string[] } | { indentation: number; info: string; kind: 'fenced-code'; lines: string[]; marker: string } | { kind: 'paragraph'; lines: string[] } + | { kind: 'pipe-table'; rows: [string[], ...string[][]] } -type ContainerStart = { kind: 'blockquote'; rest: Line } | { fresh: boolean; indentation: number; kind: 'item'; list: ListBlock; marker: string; rest: Line } +type ContainerStart = { kind: 'blockquote'; rest: Line } | { fresh: boolean; indentation: number; kind: 'item'; list: ListBlock; rest: Line } // The line from an absolute column on: a tab a cut splits keeps the stop it is measured against. type Line = { column: number; text: string } @@ -151,13 +154,12 @@ function itemStart(line: Line, opener: Line, paragraphOpen: boolean, enclosing: const spaces = leadingColumns(after) const padding = blank || spaces > indentedCodeColumns ? 1 : spaces const kind = marker.start === undefined ? 'bulletList' : 'orderedList' - const continued = enclosing?.kind === 'item' && enclosing.list.kind === kind && enclosing.marker === marker.delimiter + const continued = enclosing?.kind === 'item' && enclosing.list.kind === kind return { fresh: !continued, indentation: leadingColumns(line) + marker.width + padding, kind: 'item', list: continued ? enclosing.list : openList(marker.start), - marker: marker.delimiter, rest: blank ? after : removeColumns(after, padding), } } @@ -176,7 +178,7 @@ function openContainer(walk: Walk, start: ContainerStart): void { } if (start.fresh) currentBlocks(walk).push(start.list) start.list.items.push(blocks) - walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list: start.list, marker: start.marker }) + walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list: start.list }) } function closeContainers(walk: Walk, depth: number): void { @@ -262,6 +264,14 @@ function readBlockLine(walk: Walk, line: Line): void { if (leaf.closer === undefined ? blankLine.test(line.text) : leaf.closer.test(line.text)) closeLeaf(walk) return } + if (leaf?.kind === 'pipe-table') { + const cells = pipeCells(removeColumns(line, largestOpenerIndentation).text) + if (cells !== undefined) { + leaf.rows.push(cells) + return + } + closeLeaf(walk) + } if (leaf?.kind === 'indented-code') { if (readIndentedCodeLine(leaf, line)) return closeLeaf(walk) @@ -297,9 +307,10 @@ function openLeaf(walk: Walk, line: Line): void { else pushFault(walk, directive.fault) return } - if (claimsPipeLine(opener)) { + const cells = pipeCells(opener) + if (cells !== undefined) { closeLeaf(walk) - pushFault(walk, { code: 'malformed-pipe-table', message: 'the line claims a pipe table and parses as none' }) + walk.leaf = { kind: 'pipe-table', rows: [cells] } return } if (readLineBlock(walk, opener)) return @@ -358,9 +369,28 @@ function closeLeaf(walk: Walk): void { } walk.leaf = undefined if (leaf.kind === 'html') currentBlocks(walk).push({ construct: leaf.construct, kind: 'html' }) + else if (leaf.kind === 'pipe-table') currentBlocks(walk).push(pipeTableBlock(leaf.rows)) else currentBlocks(walk).push({ kind: 'code', language: leaf.kind === 'fenced-code' ? decodeTextEscapes(leaf.info) : '', text: leaf.lines.join('\n') }) } +// spec/flavour.md, Tables: the delimiter row underlines the header and leaves the body its cell count. +function pipeTableBlock(rows: readonly [string[], ...string[][]]): Block { + const [header, delimiter, ...body] = rows + if (delimiter !== undefined && delimiter.some(isPipeAlignment)) { + return faultedBlock('a pipe table carries no column alignment ADF could hold') + } + if (delimiter === undefined || !delimiter.every(isPipeDelimiter)) { + return faultedBlock('a pipe table underlines its header with a row of `-` runs') + } + const ragged = [delimiter, ...body].find((row) => row.length !== header.length) + if (ragged !== undefined) return faultedBlock(`a pipe table row holds ${ragged.length} cells where its header holds ${header.length}`) + return { kind: 'table', rows: [header, ...body] } +} + +function faultedBlock(message: string): Block { + return { fault: malformedPipeTable(message), kind: 'fault' } +} + function takeParagraph(walk: Walk): string | undefined { const leaf = walk.leaf if (leaf?.kind !== 'paragraph') return undefined diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 91c3ad1..5798838 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -67,6 +67,18 @@ function image(url: string, alt?: string): AdfNode { return { attrs: { layout: 'center' }, content: [media], type: 'mediaSingle' } } +function cell(type: string, ...content: AdfNode[]): AdfNode { + return { content: [content.length === 0 ? { type: 'paragraph' } : { content, type: 'paragraph' }], type } +} + +function row(...cells: AdfNode[]): AdfNode { + return { content: cells, type: 'tableRow' } +} + +function table(...rows: AdfNode[]): AdfNode { + return { content: rows, type: 'table' } +} + test('builds an empty document from input holding no block', () => { assert.deepEqual(markdownToAdf(''), { ok: true, value: { type: 'doc', version: 1 } }) assert.deepEqual(content(markdownToAdf('\n \n\t\n')), []) @@ -114,6 +126,67 @@ test('reads a fenced code block, its info string the language', () => { assert.deepEqual(content(markdownToAdf('```\n- x\n> y\n```\n')), [{ content: [text('- x\n> y')], type: 'codeBlock' }]) }) +test('reads the codeBlock directive body as the node content, the info string its language', () => { + const fenced = ':::codeBlock {wrap=true}\n```rust\nfn main() {}\n```\n:::\n' + assert.deepEqual(content(markdownToAdf(fenced)), [ + { attrs: { language: 'rust', wrap: true }, content: [text('fn main() {}')], type: 'codeBlock' }, + ]) + assert.deepEqual(content(markdownToAdf(':::codeBlock {wrap=true}\n```\n```\n:::\n')), [{ attrs: { wrap: true }, type: 'codeBlock' }]) + assert.deepEqual(content(markdownToAdf(':::codeBlock {language=""}\n```\nx\n```\n:::\n')), [ + { attrs: { language: '' }, content: [text('x')], type: 'codeBlock' }, + ]) + // The body is a CommonMark fence, so its info string decodes escapes the way any other fence's does. + assert.deepEqual(content(markdownToAdf(':::codeBlock {wrap=true}\n```\\#c\nx\n```\n:::\n')), [ + { attrs: { language: '#c', wrap: true }, content: [text('x')], type: 'codeBlock' }, + ]) +}) + +test('names the slot a codeBlock spells its language outside of', () => { + const slot = 'unsupported-node-shape: codeBlock spells its language in the fence info string, or in the attribute where no info string carries it back' + assert.equal(content(markdownToAdf(':::codeBlock {language=rust wrap=true}\n```\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf(':::codeBlock {language=rust}\n```sql\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\n```adf\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\n```a\\b\nx\n```\n:::\n')), slot) + assert.equal(content(markdownToAdf('::codeBlock {wrap=true}\n')), 'unsupported-node-shape: codeBlock spells its body in the container form, :::') +}) + +test('reads a pipe table into the header row and the body rows under it', () => { + const pipes = '| Part | Note |\n| --- | --- |\n| Nut \\| washer | `8.8` |\n| Spare | |\n' + assert.deepEqual(content(markdownToAdf(pipes)), [ + table( + row(cell('tableHeader', text('Part')), cell('tableHeader', text('Note'))), + row(cell('tableCell', text('Nut | washer')), cell('tableCell', codeSpan('8.8'))), + row(cell('tableCell', text('Spare')), cell('tableCell')), + ), + ]) + assert.deepEqual(content(markdownToAdf('| Part\n| -\n')), [table(row(cell('tableHeader', text('Part'))))]) + assert.deepEqual(content(markdownToAdf(' | Part |\n | --- |\n')), [table(row(cell('tableHeader', text('Part'))))]) +}) + +test('claims the line a pipe opens and gives the rest back to the block walk', () => { + const header = table(row(cell('tableHeader', text('a')))) + assert.deepEqual(content(markdownToAdf('Part.\n| a |\n| --- |\n')), [paragraph('Part.'), header]) + assert.deepEqual(content(markdownToAdf('| a |\n| --- |\nPart.\n')), [header, paragraph('Part.')]) + assert.deepEqual(content(markdownToAdf('> | a |\n> | --- |\n')), [quote(header)]) + assert.deepEqual(content(markdownToAdf('- | a |\n | --- |\n')), [bulletList(item(header))]) + assert.deepEqual(content(markdownToAdf('| a |\n| --- |\n x\n')), [header, { content: [text('x')], type: 'codeBlock' }]) + assert.deepEqual(content(markdownToAdf('\\| a |\n')), [paragraph('| a |')]) +}) + +test('names the pipe table a claimed line does not spell', () => { + assert.equal(content(markdownToAdf('| a | b |\n')), 'malformed-pipe-table: a pipe table underlines its header with a row of `-` runs') + assert.equal(content(markdownToAdf('| a |\n| x |\n')), 'malformed-pipe-table: a pipe table underlines its header with a row of `-` runs') + assert.equal(content(markdownToAdf('| a | b |\n| :--- | ---: |\n')), 'malformed-pipe-table: a pipe table carries no column alignment ADF could hold') + assert.equal(content(markdownToAdf('| a | b |\n| --- |\n')), 'malformed-pipe-table: a pipe table row holds 1 cells where its header holds 2') + assert.equal(content(markdownToAdf('| a |\n| --- |\n| b | c |\n')), 'malformed-pipe-table: a pipe table row holds 2 cells where its header holds 1') + assert.deepEqual(path(markdownToAdf('Part.\n\n| a |\n')), ['content', 1]) +}) + +test('refuses the image a pipe cell holds no ADF node for', () => { + assert.equal(content(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') + assert.deepEqual(path(markdownToAdf('| a |\n| --- |\n| ![x](/u) |\n')), ['content', 0, 'content', 1, 'content', 0, 'content', 0]) +}) + test('strips the opening fence indentation from the content lines it holds', () => { assert.deepEqual(content(markdownToAdf(' ```\n x\n y\n ```\n')), [{ content: [text(' x\ny')], type: 'codeBlock' }]) assert.deepEqual(content(markdownToAdf(' ```\n\tx\n ```\n')), [{ content: [text(' x')], type: 'codeBlock' }]) @@ -161,8 +234,8 @@ test('names the directive form a node CommonMark spells refuses', () => { // The spelling the emitter refuses gives the emitter's own error, never a second name for it. test('gives back the refusal the CommonMark spelling itself raises', () => { - const nested = '::::::::bulletList\n:::::::listItem\n---\n\n::::::bulletList\n:::::listItem\n---\n\n::::bulletList\n:::listItem\n---\n:::\n::::\n:::::\n::::::\n:::::::\n::::::::\n' - assert.equal(code(markdownToAdf(nested)), 'unspelled-block-separation') + const destination = ':::blockquote\n[t](https://example.com/a\\b)\n:::\n' + assert.equal(content(markdownToAdf(destination)), 'unspellable-link-destination: no canonical escape spells a backslash in a link destination') }) test('names the directive name no node reads back to', () => { @@ -232,7 +305,7 @@ test('names the argument and the body a node takes no reading for', () => { assert.equal(content(markdownToAdf(':::paragraph\n:::\n')), 'unsupported-node-shape: an empty paragraph takes the leaf form, ::') assert.equal(content(markdownToAdf(':::paragraph\nOne.\n\nTwo.\n:::\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body') assert.equal(content(markdownToAdf(':::paragraph\n---\n:::\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body') - assert.equal(content(markdownToAdf(':::codeBlock\n```\nx\n```\n:::\n')), 'unsupported-node-shape: the fenced body of codeBlock is unsupported') + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\nx\n:::\n')), 'unsupported-node-shape: codeBlock takes one fenced code block as its body') assert.equal(content(markdownToAdf(':::paragraph\n![a](/u)\n:::\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.equal(content(markdownToAdf('Part :date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content') assert.equal( @@ -315,14 +388,15 @@ test('reads a bullet list, the marker width setting the continuation', () => { assert.deepEqual(content(markdownToAdf('-\n')), [bulletList(item())]) assert.deepEqual(content(markdownToAdf('- One\n\n Two.\n')), [bulletList(item(paragraph('One'), paragraph('Two.')))]) assert.deepEqual(content(markdownToAdf('- Code.\n')), [bulletList(item({ content: [text('Code.')], type: 'codeBlock' }))]) - assert.deepEqual(content(markdownToAdf('- a\n* b\n')), [bulletList(item(paragraph('a'))), bulletList(item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('- a\n* b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('- a\n\n+ b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('-\n\n Part.\n')), [bulletList(item()), paragraph('Part.')]) }) test('reads an ordered list, its first marker the order attribute', () => { assert.deepEqual(content(markdownToAdf('9. Bolt M8\n10. Nut M8\n')), [orderedList(9, item(paragraph('Bolt M8')), item(paragraph('Nut M8')))]) assert.deepEqual(content(markdownToAdf('1) Loosen the clamp\n')), [orderedList(1, item(paragraph('Loosen the clamp')))]) - assert.deepEqual(content(markdownToAdf('1. a\n1) b\n')), [orderedList(1, item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('1. a\n1) b\n')), [orderedList(1, item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('0. Zero\n')), [orderedList(0, item(paragraph('Zero')))]) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 2aa7782..f49c107 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -5,6 +5,7 @@ import type { LinkDefinitions } from './inline-content.ts' import { carryName } from '../opaque-carry.ts' import { commonMarkSpelling } from '../emit/adf-to-markdown.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' +import { fenceInfo } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' @@ -50,6 +51,8 @@ function blockNode(block: Block, definitions: LinkDefinitions, path: ConvertErro return paragraphNode(block.text, definitions, path) case 'rule': return success({ type: 'rule' }) + case 'table': + return tableNode(block.rows, definitions, path) } } @@ -71,11 +74,40 @@ function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, de return failure('unsupported-node-shape', `${node.type} spells its body in the container form, :::`, path) } if (contentModel === 'none') return failure('unsupported-node-shape', `${node.type} holds no content`, path) - if (contentModel === 'code') return failure('unsupported-node-shape', `the fenced body of ${node.type} is unsupported`, path) + if (contentModel === 'code') return codeDirectiveNode(node, blocks, path) if (contentModel === 'block') return containerNode(node, blocks, definitions, path, depth) return inlineBodyNode(node, blocks, definitions, path) } +// spec/flavour.md, The CommonMark blocks: the language rides the one slot fenceInfo picks for it. +function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: ConvertErrorPath): Result { + const only = blocks.length === 1 ? blocks[0] : undefined + if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one fenced code block as its body`, path) + const attribute = node.attrs?.['language'] + const fromFence = only.language !== '' + const info = fenceInfo(fromFence ? only.language : attribute) + if ((info !== undefined && info !== '') !== fromFence || (fromFence && attribute !== undefined)) { + return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) + } + const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: only.language } } : node + return success(withContent(spelled, only.text === '' ? [] : [{ text: only.text, type: 'text' }])) +} + +function tableNode(rows: readonly string[][], definitions: LinkDefinitions, path: ConvertErrorPath): Result { + const content: AdfNode[] = [] + for (const [rowIndex, cells] of rows.entries()) { + const type = rowIndex === 0 ? 'tableHeader' : 'tableCell' + const row: AdfNode[] = [] + for (const [cellIndex, cell] of cells.entries()) { + const paragraph = contentNode({ type: 'paragraph' }, cell, definitions, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0]) + if (!paragraph.ok) return paragraph + row.push({ content: [paragraph.value], type }) + } + content.push({ content: row, type: 'tableRow' }) + } + return success({ content, type: 'table' }) +} + function inlineBodyNode(node: AdfNode, blocks: readonly Block[], definitions: LinkDefinitions, path: ConvertErrorPath): Result { if (blocks.length === 0) return failure('unsupported-node-shape', `an empty ${node.type} takes the leaf form, ::`, path) const only = blocks.length === 1 ? blocks[0] : undefined diff --git a/src/markdown/pipe-table-syntax.ts b/src/markdown/pipe-table-syntax.ts new file mode 100644 index 0000000..5935d40 --- /dev/null +++ b/src/markdown/pipe-table-syntax.ts @@ -0,0 +1,48 @@ +import type { ConvertFault } from '../result.ts' +import { backslashEscape, claimsPipeLine, trimSpace } from './commonmark-grammar.ts' + +const alignmentCell = /^:-+:?$|^-+:$/ +const delimiterCell = /^-+$/ + +export function isPipeAlignment(cell: string): boolean { + return alignmentCell.test(cell) +} + +export function isPipeDelimiter(cell: string): boolean { + return delimiterCell.test(cell) +} + +export function malformedPipeTable(message: string): ConvertFault { + return { code: 'malformed-pipe-table', message } +} + +// spec/flavour.md, Tables: the cells of a claimed row, the closing `|` the spelling writes optional here. +export function pipeCells(line: string): string[] | undefined { + if (!claimsPipeLine(line)) return undefined + const row = line.replace(/[ \t]+$/, '') + const cells: string[] = [] + let start = 1 + let index = 1 + while (index < row.length) { + if (backslashEscape(row, index) !== undefined) { + index += 2 + continue + } + if (row.charAt(index) === '|') { + cells.push(trimSpace(row.slice(start, index))) + start = index + 1 + } + index += 1 + } + cells.push(trimSpace(row.slice(start))) + if (cells.length > 1 && cells.at(-1) === '') cells.pop() + return cells +} + +export function spellPipeDelimiter(columns: number): string { + return spellPipeRow(Array.from({ length: columns }, () => '---')) +} + +export function spellPipeRow(cells: readonly string[]): string { + return `| ${cells.join(' | ')} |` +} diff --git a/src/result.ts b/src/result.ts index 4ad939c..316e778 100644 --- a/src/result.ts +++ b/src/result.ts @@ -11,7 +11,6 @@ export type ConvertErrorCode = | 'unspellable-link-destination' | 'unspellable-link-title' | 'unspellable-whitespace' - | 'unspelled-block-separation' | 'unsupported-document-version' | 'unsupported-nesting-depth' | 'unsupported-node-shape' diff --git a/todo-history.md b/todo-history.md index c6937d9..ab162a3 100644 --- a/todo-history.md +++ b/todo-history.md @@ -281,7 +281,28 @@ Under **3 — `markdownToAdf` (`0.1.0`)**: `spec/flavour.md`'s closing-fence sentence now says: a run reaching past the innermost leaves the fence it did not close a named error, which §2 prefers to closing more than the author wrote. - - [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first + - [x] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the + `language` attribute a bare fence leaves it; the media family's composition; and both + table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both + directions answer alike and moves to the `markdown/` root with the language attribute. + **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one + `container-block-separation` holds — a CommonMark block and a directive block sit adjacent + in a container body with no blank line between them. That reduces the three cases to one + rule, separation only where its absence would merge the blocks: the `:::` fence is + separation already, and 3c's claim ends the lazy continuation that would otherwise swallow + it. The fixture becomes a round-trip pair, and with `nested-list-separation` and 3e's pair + that empties `corpus/unspellable/`: this chunk settles the directory's own guard in + `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. + The emitter's other refusals survive on causes no fixture in that directory covers, so + 3k's one-list pass is where they get fixtures or the directory goes. + **Settled** (the maintainer, 2026-09-01): losing that cause closed one of the two shapes + input accepted and emit refused, not the last. The other surfaced here — CommonMark opens a + second list on a marker change, so `- a` over `* b` built two adjacent `bulletList` nodes + `adfToMarkdown` refuses — and the parser continues the list instead, the way it already + drops the blank lines between items. With both closed the parse-then-emit fixpoint the + README promises holds for every construct the parser reads, rather than only for what the + emitter wrote. +- [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first directory to. A parsed directive becomes its node: the name to the type and an unknown one to a named error, the arg to the attribute it names, each value to the type its section assigns, the body to `content`, the reserved `marks` key to the marks array. 3a's drift diff --git a/todo.md b/todo.md index 751786a..97e47e7 100644 --- a/todo.md +++ b/todo.md @@ -49,23 +49,7 @@ numbering is the order the work was planned in, not the order it ships. - [x] **3e — Emphasis and links.** - [x] **3f — The directive grammar.** - [x] **3g — The node tables read backwards.** - - [ ] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the - `language` attribute a bare fence leaves it; the media family's composition; and both - table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both - directions answer alike and moves to the `markdown/` root with the language attribute. - **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one - `container-block-separation` holds — a CommonMark block and a directive block sit adjacent - in a container body with no blank line between them. That reduces the three cases to one - rule, separation only where its absence would merge the blocks: the `:::` fence is - separation already, and 3c's claim ends the lazy continuation that would otherwise swallow - it. The fixture becomes a round-trip pair, and with `nested-list-separation` and 3e's pair that - empties `corpus/unspellable/`: this chunk settles the directory's own guard in - `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. - The emitter's other refusals survive on causes no fixture in that directory covers, so - 3k's one-list pass is where they get fixtures or the directory goes. - Losing that cause closes the last shape input accepts and emit refuses — a CommonMark block - beside a directive one inside a list item — so the parse-then-emit fixpoint the README - promises holds from here rather than only for what the emitter wrote. + - [x] **3h — The block nodes.** - [ ] **3i — The inline nodes and the marks.** `inline-nodes/` reads back: the content slot's `text` attribute and the error a slot holding anything but one unmarked text node is; the `:text{text="…"}` whitespace spelling; the four directive marks and their nesting order, @@ -102,10 +86,12 @@ numbering is the order the work was planned in, not the order it ships. 3e collapses a spelling nested inside its own kind and `*(*a*)*` is two `` against one `em`. The fixpoint alone is self-consistency a parser returning the empty document passes, and the text alone one dropping every emphasis; the - counts close both. The exception list stays the maintainer's. One outcome is no + counts close both. The exception list stays the maintainer's, and one entry is owed + already: 3h continues a list across the marker change CommonMark splits on, so an example + the reference HTML gives two `
    ` counts one `bulletList`. One outcome is no exception and must not be filed as one: valid CommonMark parsing to a document - `adfToMarkdown` refuses is a §2 hole, which is what three of `corpus/unspellable/`'s four - hold until 3c, 3e and 3h land their answers. + `adfToMarkdown` refuses is a §2 hole, which is what `corpus/unspellable/` held until 3c, + 3e and 3h landed their answers and emptied it. - [ ] **4 — Round-trip property tests (`0.2.0`)**, widening 3j's corpus round-trip past the documents a human wrote — the thing that proves 2 and 3 beyond them. Editor-normal (§2) is finished here, on 3i's merging — `toEditorNormal(doc)` and the equality the round-trip @@ -133,9 +119,9 @@ numbering is the order the work was planned in, not the order it ships. another node's. `0.1.0` ships with the retry in it, so a deep document is slow rather than wrong until the patch. - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, - the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: every - `corpus/unspellable/` document is a decision or a deferred trigger this file names, so the - directory empties as they land and whatever survives is permanent. The parser's own code + the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: 3h + landed the last decision `corpus/unspellable/` held and the directory went with it, so what + the code list holds from here is permanent. The parser's own code additions are read here as one list before that freeze — nine sessions mint them independently, and one cause wearing two codes is breaking to undo after `0.1.0`. `0.1.0` is the markdown round-trip: both markdown directions, the types, `isAdfDocument`. The build -- 2.52.0 From aae1ed4bafa10beba9f8339ca5d95fe6780a9373 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 1 Sep 2026 18:20:53 +0200 Subject: [PATCH 2/5] Answer the architecture pass: the list a blank line split, and the slot the seam names --- corpus/normalization/list-markers.json | 92 ++++++++++++++++++++++ corpus/normalization/list-markers.md | 8 ++ corpus/normalization/loose-list.md | 2 +- src/corpus.test.ts | 7 +- src/markdown/code-language.ts | 14 ++-- src/markdown/commonmark-grammar.ts | 6 +- src/markdown/emit/adf-to-markdown.ts | 15 ++-- src/markdown/parse/blocks.ts | 16 +++- src/markdown/parse/markdown-to-adf.test.ts | 2 + src/markdown/parse/markdown-to-adf.ts | 6 +- todo-history.md | 18 +++-- todo.md | 12 ++- 12 files changed, 163 insertions(+), 35 deletions(-) create mode 100644 corpus/normalization/list-markers.json create mode 100644 corpus/normalization/list-markers.md diff --git a/corpus/normalization/list-markers.json b/corpus/normalization/list-markers.json new file mode 100644 index 0000000..df8ae34 --- /dev/null +++ b/corpus/normalization/list-markers.json @@ -0,0 +1,92 @@ +{ + "content": [ + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Bolt M8", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + }, + { + "content": [ + { + "content": [ + { + "text": "Nut M8", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + }, + { + "type": "listItem" + }, + { + "content": [ + { + "content": [ + { + "text": "Washer M8", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "attrs": { + "order": 1 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Loosen the clamp", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + }, + { + "content": [ + { + "content": [ + { + "text": "Lift the cover", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/normalization/list-markers.md b/corpus/normalization/list-markers.md new file mode 100644 index 0000000..ae8bc61 --- /dev/null +++ b/corpus/normalization/list-markers.md @@ -0,0 +1,8 @@ +- Bolt M8 +* Nut M8 +- + ++ Washer M8 + +1. Loosen the clamp +1) Lift the cover diff --git a/corpus/normalization/loose-list.md b/corpus/normalization/loose-list.md index ea18ebd..97056bc 100644 --- a/corpus/normalization/loose-list.md +++ b/corpus/normalization/loose-list.md @@ -1,6 +1,6 @@ * Bolt M8 -+ Nut M8 +* Nut M8 Then: diff --git a/src/corpus.test.ts b/src/corpus.test.ts index adb9439..df292bc 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -187,12 +187,17 @@ for (const directory of emittingDirectories) { } for (const name of pairedNames(normalizationRoot, '.md', '.json')) { - test(`normalization/${name} parses to the document beside it`, () => { + test(`normalization/${name} parses to the document beside it, which emits and reads back to itself`, () => { const expected: unknown = JSON.parse(readFileSync(join(normalizationRoot, `${name}.json`), 'utf8')) assert.ok(isAdfDocument(expected), `${name}.json is not an ADF document`) const result = markdownToAdf(readFileSync(join(normalizationRoot, `${name}.md`), 'utf8')) assert.ok(result.ok, result.ok ? '' : `${result.error.code}: ${result.error.message}`) assert.deepEqual(result.value, expected) + const emitted = adfToMarkdown(result.value) + assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message}`) + const again = markdownToAdf(emitted.value) + assert.ok(again.ok, again.ok ? '' : `${again.error.code}: ${again.error.message}`) + assert.deepEqual(again.value, expected) }) } diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts index d254371..6d0b4b7 100644 --- a/src/markdown/code-language.ts +++ b/src/markdown/code-language.ts @@ -3,10 +3,12 @@ import { carryName } from './opaque-carry.ts' import { holdsControlCharacter } from './commonmark-grammar.ts' import { holdsEntityReference } from './entity-references.ts' -// spec/flavour.md, The CommonMark blocks: the info string the language rides, `undefined` where the attribute carries it. -export function fenceInfo(language: JsonValue | undefined): string | undefined { - if (language === undefined) return '' - if (typeof language !== 'string' || language === '' || language === carryName) return undefined - if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return undefined - return language +export type LanguageSlot = { info: string; kind: 'fence' } | { kind: 'attribute' } | { kind: 'none' } + +// spec/flavour.md, The CommonMark blocks: the one slot a codeBlock's language rides, both directions. +export function languageSlot(language: JsonValue | undefined): LanguageSlot { + if (language === undefined) return { kind: 'none' } + if (typeof language !== 'string' || language === '' || language === carryName) return { kind: 'attribute' } + if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return { kind: 'attribute' } + return { info: language, kind: 'fence' } } diff --git a/src/markdown/commonmark-grammar.ts b/src/markdown/commonmark-grammar.ts index 570da7c..7a9a828 100644 --- a/src/markdown/commonmark-grammar.ts +++ b/src/markdown/commonmark-grammar.ts @@ -171,13 +171,13 @@ export function isUnicodeWhitespace(character: string): boolean { } // `start` is the list's first number, `undefined` for a bullet. -export function listMarker(line: string): { delimiter: string; start: number | undefined; width: number } | undefined { +export function listMarker(line: string): { start: number | undefined; width: number } | undefined { const ordered = orderedListOpener.exec(line) if (ordered !== null) { const digits = ordered[1] ?? '' - return { delimiter: ordered[2] ?? '', start: Number(digits), width: digits.length + 1 } + return { start: Number(digits), width: digits.length + 1 } } - return bulletListOpener.test(line) ? { delimiter: line.charAt(0), start: undefined, width: 1 } : undefined + return bulletListOpener.test(line) ? { start: undefined, width: 1 } : undefined } export function markerInterruptsParagraph(start: number | undefined, empty: boolean): boolean { diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 896d40d..5b5bfef 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -6,7 +6,7 @@ import { carriesOnly, isAdfDocument } from '../../adf/document.ts' import { emitInlineLine } from './inline-line.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { fencedCodeBlock } from '../backtick-runs.ts' -import { fenceInfo } from '../code-language.ts' +import { languageSlot } from '../code-language.ts' import { holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' import { largestNesting } from '../../nesting.ts' import { spellDirectiveHeader } from './block-directive-spelling.ts' @@ -147,20 +147,21 @@ function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): R function emitCodeBlock(node: AdfNode, path: ConvertErrorPath): Result | undefined { if (!carriesOnly(node, ['language'])) return undefined - const info = fenceInfo(node.attrs?.['language']) - if (info === undefined) return undefined + const slot = languageSlot(node.attrs?.['language']) + if (slot.kind === 'attribute') return undefined const text = codeBlockText(node, path) if (!text.ok) return text - return success(commonMarkText(fencedCodeBlock(info, text.value))) + return success(commonMarkText(fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', text.value))) } function emitCodeDirective(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath): Result { - const info = fenceInfo(node.attrs?.['language']) - const header = spellDirectiveHeader(node, directive, info === undefined ? [] : ['language']) + const slot = languageSlot(node.attrs?.['language']) + const header = spellDirectiveHeader(node, directive, slot.kind === 'attribute' ? [] : ['language']) if (header === undefined) return commonMarkLine(carriedBlock(node, path)) const text = codeBlockText(node, path) if (!text.ok) return text - return success({ fenceColons: 3, spelling: 'directive', text: `:::${header}\n${fencedCodeBlock(info ?? '', text.value)}\n:::` }) + const info = slot.kind === 'fence' ? slot.info : '' + return success({ fenceColons: 3, spelling: 'directive', text: `:::${header}\n${fencedCodeBlock(info, text.value)}\n:::` }) } function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result { diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index 52cc2f2..125c435 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -176,9 +176,19 @@ function openContainer(walk: Walk, start: ContainerStart): void { walk.stack.push(blockquote) return } - if (start.fresh) currentBlocks(walk).push(start.list) - start.list.items.push(blocks) - walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list: start.list }) + const list = openedList(walk, start) + list.items.push(blocks) + walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list }) +} + +// Two lists of a kind never sit adjacent: one `- ` spelling reads them back as one (spec/flavour.md). +function openedList(walk: Walk, start: Extract): ListBlock { + if (!start.fresh) return start.list + const blocks = currentBlocks(walk) + const previous = blocks.at(-1) + if ((previous?.kind === 'bulletList' || previous?.kind === 'orderedList') && previous.kind === start.list.kind) return previous + blocks.push(start.list) + return start.list } function closeContainers(walk: Walk, depth: number): void { diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 5798838..d32f96f 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -390,6 +390,8 @@ test('reads a bullet list, the marker width setting the continuation', () => { assert.deepEqual(content(markdownToAdf('- Code.\n')), [bulletList(item({ content: [text('Code.')], type: 'codeBlock' }))]) assert.deepEqual(content(markdownToAdf('- a\n* b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('- a\n\n+ b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('- a\n-\n\n- c\n')), [bulletList(item(paragraph('a')), item(), item(paragraph('c')))]) + assert.deepEqual(content(markdownToAdf('- a\n1. b\n')), [bulletList(item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('-\n\n Part.\n')), [bulletList(item()), paragraph('Part.')]) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index f49c107..9009f5f 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -5,7 +5,7 @@ import type { LinkDefinitions } from './inline-content.ts' import { carryName } from '../opaque-carry.ts' import { commonMarkSpelling } from '../emit/adf-to-markdown.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { fenceInfo } from '../code-language.ts' +import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' @@ -85,8 +85,8 @@ function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: Conver if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one fenced code block as its body`, path) const attribute = node.attrs?.['language'] const fromFence = only.language !== '' - const info = fenceInfo(fromFence ? only.language : attribute) - if ((info !== undefined && info !== '') !== fromFence || (fromFence && attribute !== undefined)) { + const slot = languageSlot(fromFence ? only.language : attribute) + if ((slot.kind === 'fence') !== fromFence || (fromFence && attribute !== undefined)) { return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) } const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: only.language } } : node diff --git a/todo-history.md b/todo-history.md index ab162a3..5e5de69 100644 --- a/todo-history.md +++ b/todo-history.md @@ -295,13 +295,17 @@ Under **3 — `markdownToAdf` (`0.1.0`)**: `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. The emitter's other refusals survive on causes no fixture in that directory covers, so 3k's one-list pass is where they get fixtures or the directory goes. - **Settled** (the maintainer, 2026-09-01): losing that cause closed one of the two shapes - input accepted and emit refused, not the last. The other surfaced here — CommonMark opens a - second list on a marker change, so `- a` over `* b` built two adjacent `bulletList` nodes - `adfToMarkdown` refuses — and the parser continues the list instead, the way it already - drops the blank lines between items. With both closed the parse-then-emit fixpoint the - README promises holds for every construct the parser reads, rather than only for what the - emitter wrote. + **Settled** (the maintainer, 2026-09-01): losing that cause closed one of the shapes input + accepted and emit refused, not the last. Two adjacent lists of a kind are what + `adfToMarkdown` refuses and one `- ` spelling cannot hold apart, and the walk reached them + two ways — a marker change, which CommonMark opens a second list on, and an empty last item, + whose blank line pops the container the list's identity hung from. The parser opens no list + beside one of its own kind instead, the way it already drops the blank lines between items; + 3k owes the CommonMark suite an exception where the reference HTML holds two `
      `. The + `normalization/` arm emits each document and reads it back from here, so the population that + class lives in is checked rather than read. The README's canonical-fixpoint sentence still + claims more than the parser keeps — 3e's three shapes — which stays milestone 5's to + narrow. - [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first directory to. A parsed directive becomes its node: the name to the type and an unknown one to a named error, the arg to the attribute it names, each value to the type its section diff --git a/todo.md b/todo.md index 97e47e7..61ec51a 100644 --- a/todo.md +++ b/todo.md @@ -61,9 +61,10 @@ numbering is the order the work was planned in, not the order it ships. parsing the slot inside it is a cycle; the four directive marks get `parse/directive-marks.ts` that `inline-content.ts` tries ahead of the node reader, as `mark-spellings.ts` sits apart from `emit/inline-directive-spelling.ts`; and the five markdown-spelled mark names in inline - directive position want a claim code — `:em[x]` is an error forever, so - `unknown-directive-name`'s "a later MINOR may give the name meaning" is the wrong signal, - as it was for `adf`. `corpus/errors/directive-content-slot` goes when the slot opens. + directive position take `unsupported-node-shape` rather than a code of their own — §8 + already answers a well-formed directive the node tables refuse, and the message names the + spelling to use (`*x*`), while `unknown-directive-name`'s "a later MINOR may give the name + meaning" stays the wrong signal, as it was for `adf`. `corpus/errors/directive-content-slot` goes when the slot opens. The marks a spelling wraps answer the same question 3g settled for a block's form: only the nesting the emitter writes parses back. - [ ] **3j — The carry and the combinations.** `opaque-carry/` and `combinations/` read back: @@ -123,7 +124,10 @@ numbering is the order the work was planned in, not the order it ships. landed the last decision `corpus/unspellable/` held and the directory went with it, so what the code list holds from here is permanent. The parser's own code additions are read here as one list before that freeze — nine sessions mint them - independently, and one cause wearing two codes is breaking to undo after `0.1.0`. `0.1.0` + independently, and one cause wearing two codes is breaking to undo after `0.1.0`. That read + gets a test rather than an eye — every `ConvertErrorCode` member named at a production call + site, the way `spec.test.ts` guards the node tables — since `unspelled-block-separation` + outlived its cause until 3h went looking. `0.1.0` is the markdown round-trip: both markdown directions, the types, `isAdfDocument`. The build lands here: `tsconfig.build.json` gains emit of JS and `.d.ts` to `dist/` (its own `allowImportingTsExtensions` forces `noEmit`, so `rewriteRelativeImportExtensions` lands -- 2.52.0 From e68dbc41a38c4dc7592b6e91c518ca126dfffb5c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 1 Sep 2026 19:13:13 +0200 Subject: [PATCH 3/5] Answer the stability pass: the strip that changed nothing, and the words the body earns --- spec/flavour.md | 8 +++++--- src/markdown/code-language.ts | 2 +- src/markdown/commonmark-grammar.ts | 2 +- src/markdown/emit/adf-to-markdown.ts | 2 +- src/markdown/parse/blocks.ts | 1 - src/markdown/parse/markdown-to-adf.test.ts | 6 +++++- src/markdown/parse/markdown-to-adf.ts | 3 +-- src/markdown/pipe-table-syntax.ts | 11 +++++------ 8 files changed, 19 insertions(+), 16 deletions(-) diff --git a/spec/flavour.md b/spec/flavour.md index 68bc70a..878bafe 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -22,8 +22,9 @@ normalizes to it through the round-trip. between a nested list and a CommonMark block above it — one wherever the nested list's own marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or a list whose first item is empty), whatever block sits above it. Blank lines between items normalize - away, and so does the marker change CommonMark starts a second list on: ADF records no - tightness, and one `- ` spelling leaves two adjacent lists of a type no way back. + away, and no list opens beside one of its own kind — the marker change CommonMark starts a + second list on merges instead: ADF records no tightness, and one `- ` spelling leaves two + adjacent lists of a kind no way back. - Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`. - ATX headings (`#` … `######`); setext input normalizes to ATX. - Code fences ``` with the node's language as info string, the fence lengthened past any backtick @@ -192,7 +193,8 @@ form. the reserved `adf`, or holding a backtick, a backslash, a control character, edge whitespace or an entity reference — rides the `language` attribute instead and the fence carries no info string; writing it in the slot that rule leaves empty, or in both, is a named error. The body is - an ordinary fence, so its info string decodes escapes and entity references as any other does. + one ordinary code block, and a fence's info string decodes escapes and entity references as any + other does. - `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no CommonMark spelling. diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts index 6d0b4b7..27f3a07 100644 --- a/src/markdown/code-language.ts +++ b/src/markdown/code-language.ts @@ -5,7 +5,7 @@ import { holdsEntityReference } from './entity-references.ts' export type LanguageSlot = { info: string; kind: 'fence' } | { kind: 'attribute' } | { kind: 'none' } -// spec/flavour.md, The CommonMark blocks: the one slot a codeBlock's language rides, both directions. +// spec/flavour.md, The CommonMark blocks: the one slot a codeBlock's language rides. export function languageSlot(language: JsonValue | undefined): LanguageSlot { if (language === undefined) return { kind: 'none' } if (typeof language !== 'string' || language === '' || language === carryName) return { kind: 'attribute' } diff --git a/src/markdown/commonmark-grammar.ts b/src/markdown/commonmark-grammar.ts index 7a9a828..fd93d8d 100644 --- a/src/markdown/commonmark-grammar.ts +++ b/src/markdown/commonmark-grammar.ts @@ -60,7 +60,7 @@ const firstCharacterOpeners = [atxHeadingOpener, /^>/, bulletListOpener, codeFen const emailNameSource = "[A-Za-z0-9.!#$%&'*+/=?^_`{|}~-]+" const emailLabelSource = '[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?' const emailAutolink = new RegExp(`<${emailNameSource}@${emailLabelSource}(?:\\.${emailLabelSource})*>`, 'y') -const orderedListOpener = /^(\d{1,9})([.)])(?:[ \t]|$)/ +const orderedListOpener = /^(\d{1,9})(?:[.)])(?:[ \t]|$)/ const setextUnderline = /^(=+|-+)[ \t]*$/ const thematicBreak = /^(?:(?:\*[ \t]*){3,}|(?:-[ \t]*){3,}|(?:_[ \t]*){3,})$/ const unicodeWhitespace = /[\t\n\f\r \p{Zs}]/u diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 5b5bfef..7125c34 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -6,8 +6,8 @@ import { carriesOnly, isAdfDocument } from '../../adf/document.ts' import { emitInlineLine } from './inline-line.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { fencedCodeBlock } from '../backtick-runs.ts' -import { languageSlot } from '../code-language.ts' import { holdsNullCharacter, isThematicBreak, markerInterruptsParagraph } from '../commonmark-grammar.ts' +import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { spellDirectiveHeader } from './block-directive-spelling.ts' import { tryImage } from './image.ts' diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index 125c435..0564db0 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -383,7 +383,6 @@ function closeLeaf(walk: Walk): void { else currentBlocks(walk).push({ kind: 'code', language: leaf.kind === 'fenced-code' ? decodeTextEscapes(leaf.info) : '', text: leaf.lines.join('\n') }) } -// spec/flavour.md, Tables: the delimiter row underlines the header and leaves the body its cell count. function pipeTableBlock(rows: readonly [string[], ...string[][]]): Block { const [header, delimiter, ...body] = rows if (delimiter !== undefined && delimiter.some(isPipeAlignment)) { diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index d32f96f..c5f612c 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -135,6 +135,9 @@ test('reads the codeBlock directive body as the node content, the info string it assert.deepEqual(content(markdownToAdf(':::codeBlock {language=""}\n```\nx\n```\n:::\n')), [ { attrs: { language: '' }, content: [text('x')], type: 'codeBlock' }, ]) + assert.deepEqual(content(markdownToAdf(':::codeBlock {wrap=true}\n fn()\n:::\n')), [ + { attrs: { wrap: true }, content: [text('fn()')], type: 'codeBlock' }, + ]) // The body is a CommonMark fence, so its info string decodes escapes the way any other fence's does. assert.deepEqual(content(markdownToAdf(':::codeBlock {wrap=true}\n```\\#c\nx\n```\n:::\n')), [ { attrs: { language: '#c', wrap: true }, content: [text('x')], type: 'codeBlock' }, @@ -305,7 +308,7 @@ test('names the argument and the body a node takes no reading for', () => { assert.equal(content(markdownToAdf(':::paragraph\n:::\n')), 'unsupported-node-shape: an empty paragraph takes the leaf form, ::') assert.equal(content(markdownToAdf(':::paragraph\nOne.\n\nTwo.\n:::\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body') assert.equal(content(markdownToAdf(':::paragraph\n---\n:::\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body') - assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\nx\n:::\n')), 'unsupported-node-shape: codeBlock takes one fenced code block as its body') + assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\nx\n:::\n')), 'unsupported-node-shape: codeBlock takes one code block as its body') assert.equal(content(markdownToAdf(':::paragraph\n![a](/u)\n:::\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.equal(content(markdownToAdf('Part :date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content') assert.equal( @@ -392,6 +395,7 @@ test('reads a bullet list, the marker width setting the continuation', () => { assert.deepEqual(content(markdownToAdf('- a\n\n+ b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('- a\n-\n\n- c\n')), [bulletList(item(paragraph('a')), item(), item(paragraph('c')))]) assert.deepEqual(content(markdownToAdf('- a\n1. b\n')), [bulletList(item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) + assert.deepEqual(content(markdownToAdf('- a\n\n[r]: /u\n\n- b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(markdownToAdf('-\n\n Part.\n')), [bulletList(item()), paragraph('Part.')]) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 9009f5f..52d6928 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -79,10 +79,9 @@ function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, de return inlineBodyNode(node, blocks, definitions, path) } -// spec/flavour.md, The CommonMark blocks: the language rides the one slot fenceInfo picks for it. function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: ConvertErrorPath): Result { const only = blocks.length === 1 ? blocks[0] : undefined - if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one fenced code block as its body`, path) + if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one code block as its body`, path) const attribute = node.attrs?.['language'] const fromFence = only.language !== '' const slot = languageSlot(fromFence ? only.language : attribute) diff --git a/src/markdown/pipe-table-syntax.ts b/src/markdown/pipe-table-syntax.ts index 5935d40..f665a75 100644 --- a/src/markdown/pipe-table-syntax.ts +++ b/src/markdown/pipe-table-syntax.ts @@ -19,22 +19,21 @@ export function malformedPipeTable(message: string): ConvertFault { // spec/flavour.md, Tables: the cells of a claimed row, the closing `|` the spelling writes optional here. export function pipeCells(line: string): string[] | undefined { if (!claimsPipeLine(line)) return undefined - const row = line.replace(/[ \t]+$/, '') const cells: string[] = [] let start = 1 let index = 1 - while (index < row.length) { - if (backslashEscape(row, index) !== undefined) { + while (index < line.length) { + if (backslashEscape(line, index) !== undefined) { index += 2 continue } - if (row.charAt(index) === '|') { - cells.push(trimSpace(row.slice(start, index))) + if (line.charAt(index) === '|') { + cells.push(trimSpace(line.slice(start, index))) start = index + 1 } index += 1 } - cells.push(trimSpace(row.slice(start))) + cells.push(trimSpace(line.slice(start))) if (cells.length > 1 && cells.at(-1) === '') cells.pop() return cells } -- 2.52.0 From 5a41547222f4a2e09e0e15e47feceffeae8f62db Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 1 Sep 2026 20:11:41 +0200 Subject: [PATCH 4/5] Answer the stability nits: the runs a trailing-anchored regex re-walks --- corpus/README.md | 3 +- src/markdown/commonmark-grammar.ts | 16 +++++++-- src/markdown/parse/inline-content.ts | 8 ++--- todo-history.md | 52 ++++++++++++++-------------- 4 files changed, 45 insertions(+), 34 deletions(-) diff --git a/corpus/README.md b/corpus/README.md index d84d3aa..1100d66 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -6,7 +6,8 @@ One directory per contract kind, each landing with its milestone: document, byte for byte, and that `markdownToAdf` must read back to it (AGENTS.md §2). Grouped by what the fixture exercises. - `normalization/` — `.md` + `.json`: markdown input, and the document - `markdownToAdf` must build from it. One-way; the markdown is not canonical. + `markdownToAdf` must build from it, which must in turn emit and read back to itself. The + markdown is not canonical. - `errors/` — `.md`: markdown input that must not convert. A `.error` beside it pins which error. - `real-payloads/` — `.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No diff --git a/src/markdown/commonmark-grammar.ts b/src/markdown/commonmark-grammar.ts index fd93d8d..7c4e074 100644 --- a/src/markdown/commonmark-grammar.ts +++ b/src/markdown/commonmark-grammar.ts @@ -223,6 +223,18 @@ export function setextHeadingLevel(line: string): number | undefined { return underline.startsWith('=') ? 1 : 2 } -export function trimSpace(text: string): string { - return text.replace(/^[ \t]+|[ \t]+$/g, '') +function spaceOrTab(character: string): boolean { + return character === ' ' || character === '\t' +} + +export function trimSpace(text: string): string { + let start = 0 + while (start < text.length && spaceOrTab(text.charAt(start))) start += 1 + return trimTrailingSpace(text.slice(start)) +} + +export function trimTrailingSpace(text: string): string { + let end = text.length + while (end > 0 && spaceOrTab(text.charAt(end - 1))) end -= 1 + return text.slice(0, end) } diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 3a7b70b..a867fe1 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -1,7 +1,7 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts' import type { EmphasisPairing } from '../emphasis-matching.ts' import type { LinkDefinition } from '../link-syntax.ts' -import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink } from '../commonmark-grammar.ts' +import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark-grammar.ts' import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { delimiterFlags, matchEmphasis, runLength } from '../emphasis-matching.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' @@ -28,9 +28,7 @@ type Run = { canClose: boolean; canOpen: boolean; character: string; index: numb type Scan = { definitions: LinkDefinitions; path: ConvertErrorPath; pending: string; pieces: Piece[]; source: string } -const hardBreakSpaces = / {2,}$/ const imageAlone = 'an image fits only as a paragraph of its own' -const trailingSpace = /[ \t]+$/ export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath): Result { const scan: Scan = { definitions, path, pending: '', pieces: [], source } @@ -95,7 +93,7 @@ function readBackslash(scan: Scan, index: number): number { } function readLineEnding(scan: Scan, index: number): number { - const hard = hardBreakSpaces.test(scan.pending) + const hard = scan.pending.endsWith(' ') flush(scan, true) if (hard) pushNode(scan, { type: 'hardBreak' }) else scan.pending = ' ' @@ -154,7 +152,7 @@ function readDirective(scan: Scan, index: number): Result { } function flush(scan: Scan, strip: boolean): void { - const raw = strip ? scan.pending.replace(trailingSpace, '') : scan.pending + const raw = strip ? trimTrailingSpace(scan.pending) : scan.pending scan.pending = '' if (raw !== '') scan.pieces.push({ kind: 'nodes', nodes: [{ text: decodeTextEscapes(raw), type: 'text' }] }) } diff --git a/todo-history.md b/todo-history.md index 5e5de69..60c297d 100644 --- a/todo-history.md +++ b/todo-history.md @@ -281,32 +281,7 @@ Under **3 — `markdownToAdf` (`0.1.0`)**: `spec/flavour.md`'s closing-fence sentence now says: a run reaching past the innermost leaves the fence it did not close a named error, which §2 prefers to closing more than the author wrote. - - [x] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the - `language` attribute a bare fence leaves it; the media family's composition; and both - table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both - directions answer alike and moves to the `markdown/` root with the language attribute. - **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one - `container-block-separation` holds — a CommonMark block and a directive block sit adjacent - in a container body with no blank line between them. That reduces the three cases to one - rule, separation only where its absence would merge the blocks: the `:::` fence is - separation already, and 3c's claim ends the lazy continuation that would otherwise swallow - it. The fixture becomes a round-trip pair, and with `nested-list-separation` and 3e's pair - that empties `corpus/unspellable/`: this chunk settles the directory's own guard in - `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. - The emitter's other refusals survive on causes no fixture in that directory covers, so - 3k's one-list pass is where they get fixtures or the directory goes. - **Settled** (the maintainer, 2026-09-01): losing that cause closed one of the shapes input - accepted and emit refused, not the last. Two adjacent lists of a kind are what - `adfToMarkdown` refuses and one `- ` spelling cannot hold apart, and the walk reached them - two ways — a marker change, which CommonMark opens a second list on, and an empty last item, - whose blank line pops the container the list's identity hung from. The parser opens no list - beside one of its own kind instead, the way it already drops the blank lines between items; - 3k owes the CommonMark suite an exception where the reference HTML holds two `
        `. The - `normalization/` arm emits each document and reads it back from here, so the population that - class lives in is checked rather than read. The README's canonical-fixpoint sentence still - claims more than the parser keeps — 3e's three shapes — which stays milestone 5's to - narrow. -- [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first + - [x] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first directory to. A parsed directive becomes its node: the name to the type and an unknown one to a named error, the arg to the attribute it names, each value to the type its section assigns, the body to `content`, the reserved `marks` key to the marks array. 3a's drift @@ -334,3 +309,28 @@ Under **3 — `markdownToAdf` (`0.1.0`)**: path and returns `Result`, so no second reader took it. The drift guard earned itself on the way in: the spec's `text` attribute was missing from three inline table entries, which the content slot spells and the vocabulary walk already passes over. + - [x] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the + `language` attribute a bare fence leaves it; the media family's composition; and both + table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both + directions answer alike and moves to the `markdown/` root with the language attribute. + **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one + `container-block-separation` holds — a CommonMark block and a directive block sit adjacent + in a container body with no blank line between them. That reduces the three cases to one + rule, separation only where its absence would merge the blocks: the `:::` fence is + separation already, and 3c's claim ends the lazy continuation that would otherwise swallow + it. The fixture becomes a round-trip pair, and with `nested-list-separation` and 3e's pair + that empties `corpus/unspellable/`: this chunk settles the directory's own guard in + `corpus.test.ts` too, and `unspelled-block-separation`, which loses its only cause here. + The emitter's other refusals survive on causes no fixture in that directory covers, so + 3k's one-list pass is where they get fixtures or the directory goes. + **Settled** (the maintainer, 2026-09-01): losing that cause closed one of the shapes input + accepted and emit refused, not the last. Two adjacent lists of a kind are what + `adfToMarkdown` refuses and one `- ` spelling cannot hold apart, and the walk reached them + two ways — a marker change, which CommonMark opens a second list on, and an empty last item, + whose blank line pops the container the list's identity hung from. The parser opens no list + beside one of its own kind instead, the way it already drops the blank lines between items; + 3k owes the CommonMark suite an exception where the reference HTML holds two `
          `. The + `normalization/` arm emits each document and reads it back from here, so the population that + class lives in is checked rather than read. The README's canonical-fixpoint sentence still + claims more than the parser keeps — 3e's three shapes — which stays milestone 5's to + narrow. -- 2.52.0 From 9912fa91a1a5be71c764125ffcaace73da17da82 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 1 Sep 2026 20:21:47 +0200 Subject: [PATCH 5/5] File the scanning rule's remaining sites, and rewrap the moved entry --- todo-history.md | 9 +++++---- todo.md | 12 ++++++++++-- 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/todo-history.md b/todo-history.md index 60c297d..a3d2bf6 100644 --- a/todo-history.md +++ b/todo-history.md @@ -309,10 +309,11 @@ Under **3 — `markdownToAdf` (`0.1.0`)**: path and returns `Result`, so no second reader took it. The drift guard earned itself on the way in: the spec's `text` attribute was missing from three inline table entries, which the content slot spells and the vocabulary walk already passes over. - - [x] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced body and the - `language` attribute a bare fence leaves it; the media family's composition; and both - table forms, the pipe table's cell split and its named errors. `fenceInfo` is a rule both - directions answer alike and moves to the `markdown/` root with the language attribute. + - [x] **3h — The block nodes.** `block-nodes/` reads back: the `codeBlock` directive's fenced + body and the `language` attribute a bare fence leaves it; the media family's composition; + and both table forms, the pipe table's cell split and its named errors. `fenceInfo` is a + rule both directions answer alike and moves to the `markdown/` root with the language + attribute. **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one `container-block-separation` holds — a CommonMark block and a directive block sit adjacent in a container body with no blank line between them. That reduces the three cases to one diff --git a/todo.md b/todo.md index 61ec51a..500c69e 100644 --- a/todo.md +++ b/todo.md @@ -5,8 +5,8 @@ milestone. A done item shrinks to its title here; its full text moves to `todo-h ## Milestones -Shipping order: 3h, 3i, 3j, 5 → `0.1.0`; 4b → `0.1.1`; 4, 3k → `0.2.0`; 6, 7 → `0.3.0`. The -numbering is the order the work was planned in, not the order it ships. +Shipping order: 3h, 3i, 3j, 5 → `0.1.0`; 4b and 4c → `0.1.1`; 4, 3k → `0.2.0`; 6, 7 → `0.3.0`. +The numbering is the order the work was planned in, not the order it ships. - [x] **0 — Scaffold.** - [x] **1a — The directive grammar.** @@ -119,6 +119,14 @@ numbering is the order the work was planned in, not the order it ships. caller may hold one node object at two positions, where the cached depth and path are another node's. `0.1.0` ships with the retry in it, so a deep document is slow rather than wrong until the patch. +- [ ] **4c — The scanning rule's remaining sites (`0.1.1`).** A trailing-anchored regex re-walks + its run from every start position, so an interior whitespace run costs quadratic time rather + than linear — 3h measured 80k spaces inside an ATX heading at 11.3s, and 3ms once the walk + replaced the regex. Three sites the same sweep did not reach: `normalizeLabel` in + `link-syntax.ts`, whose shortcut-reference input is `scan.source.slice(...)` rather than the + 999-capped `readLabel` value, and two in `emit/inline-line.ts`. The fix is the one 3h used — + an index walk, `trimTrailingSpace` where the ends match. §11's scanning rule is the whole + argument; the pipeline persona feeds documents nobody typed. - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: 3h landed the last decision `corpus/unspellable/` held and the directory went with it, so what -- 2.52.0