From 56072d36ef1e0c8eba65789f831242ed5a9437d5 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 25 Aug 2026 15:10:02 +0200 Subject: [PATCH 1/6] Emitter 2b: the block-node directives, the pipe table and the refusal corpus --- README.md | 3 +- ci.sh | 5 +- corpus/README.md | 3 + corpus/unspellable/block-local-id.error | 1 + corpus/unspellable/block-local-id.json | 18 ++ .../code-block-empty-language.error | 1 + .../code-block-empty-language.json | 18 ++ .../container-block-separation.error | 1 + .../container-block-separation.json | 40 +++++ .../link-destination-parenthesis.error | 1 + .../link-destination-parenthesis.json | 23 +++ corpus/unspellable/link-title-quote.error | 1 + corpus/unspellable/link-title-quote.json | 24 +++ corpus/unspellable/media-empty-alt.error | 1 + corpus/unspellable/media-empty-alt.json | 22 +++ corpus/unspellable/nested-list-tight.error | 1 + corpus/unspellable/nested-list-tight.json | 47 ++++++ .../unspellable/ordered-list-start-one.error | 1 + .../unspellable/ordered-list-start-one.json | 28 ++++ package.json | 2 +- src/adf-to-markdown.test.ts | 94 ++++++++++- src/adf-to-markdown.ts | 157 ++++++++++++++---- src/block-directives.ts | 116 +++++++++++++ src/corpus.test.ts | 29 +++- src/directive-attributes.ts | 35 ++++ src/markdown-escaping.ts | 10 +- src/markdown-inline.ts | 19 ++- src/markdown-tables.ts | 53 ++++++ src/result.ts | 2 + todo.md | 15 +- 30 files changed, 715 insertions(+), 56 deletions(-) create mode 100644 corpus/unspellable/block-local-id.error create mode 100644 corpus/unspellable/block-local-id.json create mode 100644 corpus/unspellable/code-block-empty-language.error create mode 100644 corpus/unspellable/code-block-empty-language.json create mode 100644 corpus/unspellable/container-block-separation.error create mode 100644 corpus/unspellable/container-block-separation.json create mode 100644 corpus/unspellable/link-destination-parenthesis.error create mode 100644 corpus/unspellable/link-destination-parenthesis.json create mode 100644 corpus/unspellable/link-title-quote.error create mode 100644 corpus/unspellable/link-title-quote.json create mode 100644 corpus/unspellable/media-empty-alt.error create mode 100644 corpus/unspellable/media-empty-alt.json create mode 100644 corpus/unspellable/nested-list-tight.error create mode 100644 corpus/unspellable/nested-list-tight.json create mode 100644 corpus/unspellable/ordered-list-start-one.error create mode 100644 corpus/unspellable/ordered-list-start-one.json create mode 100644 src/block-directives.ts create mode 100644 src/directive-attributes.ts create mode 100644 src/markdown-tables.ts diff --git a/README.md b/README.md index 062cbb0..30b8317 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@ Lossless conversion between **Atlassian Document Format** (ADF), an extended markdown flavour, and an HTML dialect. -**Status: pre-release — `adfToMarkdown` emits the CommonMark subset, nothing else is built.** +**Status: pre-release — `adfToMarkdown` emits the CommonMark subset and the block nodes, nothing +else is built.** Plan: `todo.md`. Decisions: `AGENTS.md`. The flavour's grammar: [`spec/flavour.md`](spec/flavour.md). diff --git a/ci.sh b/ci.sh index 3d17b21..ceb3dbb 100755 --- a/ci.sh +++ b/ci.sh @@ -8,7 +8,10 @@ in_node() { docker run --rm -u "$(id -u):$(id -g)" -e HOME=/tmp -v "$PWD:/app" - in_node npm ci in_node npm run typecheck -test_output=$(in_node npm test) +if ! test_output=$(in_node npm test 2>&1); then + printf '%s\n' "$test_output" + exit 1 +fi printf '%s\n' "$test_output" if printf '%s' "$test_output" | grep -q 'ℹ tests 0'; then echo 'the gate ran zero tests — failing instead of a vacuous green' diff --git a/corpus/README.md b/corpus/README.md index 2137c61..6df1efe 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -9,6 +9,9 @@ One directory per contract kind: `markdownToAdf` must build from it. One-way; the markdown is not canonical. - `errors/` — `.md`: markdown input that must not convert. A `.error` beside it pins which error. +- `unspellable/` — `.json`: ADF `adfToMarkdown` must refuse, the `ConvertErrorCode` in the + `.error` beside it. A maintainer decision (`todo.md`) moves a document from here to + `round-trip/`. - `real-payloads/` — `.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No expected markdown. diff --git a/corpus/unspellable/block-local-id.error b/corpus/unspellable/block-local-id.error new file mode 100644 index 0000000..b4a73d2 --- /dev/null +++ b/corpus/unspellable/block-local-id.error @@ -0,0 +1 @@ +unspelled-node-attribute diff --git a/corpus/unspellable/block-local-id.json b/corpus/unspellable/block-local-id.json new file mode 100644 index 0000000..681618c --- /dev/null +++ b/corpus/unspellable/block-local-id.json @@ -0,0 +1,18 @@ +{ + "content": [ + { + "attrs": { + "localId": "01a03d5c-9b21-73f4-8e6a-0c47b1d9e2f8" + }, + "content": [ + { + "text": "Every block node in the schema carries one.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/code-block-empty-language.error b/corpus/unspellable/code-block-empty-language.error new file mode 100644 index 0000000..a53e1bc --- /dev/null +++ b/corpus/unspellable/code-block-empty-language.error @@ -0,0 +1 @@ +ambiguous-empty-code-block-language diff --git a/corpus/unspellable/code-block-empty-language.json b/corpus/unspellable/code-block-empty-language.json new file mode 100644 index 0000000..d0b5da7 --- /dev/null +++ b/corpus/unspellable/code-block-empty-language.json @@ -0,0 +1,18 @@ +{ + "content": [ + { + "attrs": { + "language": "" + }, + "content": [ + { + "text": "cargo build --release", + "type": "text" + } + ], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/container-block-separation.error b/corpus/unspellable/container-block-separation.error new file mode 100644 index 0000000..7fdd935 --- /dev/null +++ b/corpus/unspellable/container-block-separation.error @@ -0,0 +1 @@ +unspelled-block-separation diff --git a/corpus/unspellable/container-block-separation.json b/corpus/unspellable/container-block-separation.json new file mode 100644 index 0000000..93d1a55 --- /dev/null +++ b/corpus/unspellable/container-block-separation.json @@ -0,0 +1,40 @@ +{ + "content": [ + { + "attrs": { + "title": "Full build log" + }, + "content": [ + { + "content": [ + { + "text": "The build ran for 11 minutes.", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "attrs": { + "panelType": "warning" + }, + "content": [ + { + "content": [ + { + "text": "Three warnings went unread.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "panel" + } + ], + "type": "expand" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/link-destination-parenthesis.error b/corpus/unspellable/link-destination-parenthesis.error new file mode 100644 index 0000000..8da7bcc --- /dev/null +++ b/corpus/unspellable/link-destination-parenthesis.error @@ -0,0 +1 @@ +unspellable-link-destination diff --git a/corpus/unspellable/link-destination-parenthesis.json b/corpus/unspellable/link-destination-parenthesis.json new file mode 100644 index 0000000..ce1a2f4 --- /dev/null +++ b/corpus/unspellable/link-destination-parenthesis.json @@ -0,0 +1,23 @@ +{ + "content": [ + { + "content": [ + { + "marks": [ + { + "attrs": { + "href": "https://example.com/a)b" + }, + "type": "link" + } + ], + "text": "The torque table", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/link-title-quote.error b/corpus/unspellable/link-title-quote.error new file mode 100644 index 0000000..a1f2950 --- /dev/null +++ b/corpus/unspellable/link-title-quote.error @@ -0,0 +1 @@ +unspellable-link-title diff --git a/corpus/unspellable/link-title-quote.json b/corpus/unspellable/link-title-quote.json new file mode 100644 index 0000000..0db4010 --- /dev/null +++ b/corpus/unspellable/link-title-quote.json @@ -0,0 +1,24 @@ +{ + "content": [ + { + "content": [ + { + "marks": [ + { + "attrs": { + "href": "https://example.com/torque", + "title": "He said \"hi\"" + }, + "type": "link" + } + ], + "text": "The torque table", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/media-empty-alt.error b/corpus/unspellable/media-empty-alt.error new file mode 100644 index 0000000..a7d35ec --- /dev/null +++ b/corpus/unspellable/media-empty-alt.error @@ -0,0 +1 @@ +ambiguous-empty-media-alt diff --git a/corpus/unspellable/media-empty-alt.json b/corpus/unspellable/media-empty-alt.json new file mode 100644 index 0000000..eaa9b0b --- /dev/null +++ b/corpus/unspellable/media-empty-alt.json @@ -0,0 +1,22 @@ +{ + "content": [ + { + "attrs": { + "layout": "center" + }, + "content": [ + { + "attrs": { + "alt": "", + "type": "external", + "url": "https://example.com/moon.png" + }, + "type": "media" + } + ], + "type": "mediaSingle" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/nested-list-tight.error b/corpus/unspellable/nested-list-tight.error new file mode 100644 index 0000000..2dc039a --- /dev/null +++ b/corpus/unspellable/nested-list-tight.error @@ -0,0 +1 @@ +unspellable-line-start diff --git a/corpus/unspellable/nested-list-tight.json b/corpus/unspellable/nested-list-tight.json new file mode 100644 index 0000000..b3752d9 --- /dev/null +++ b/corpus/unspellable/nested-list-tight.json @@ -0,0 +1,47 @@ +{ + "content": [ + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Torque the bolts", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "attrs": { + "order": 2 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Then sign off", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/unspellable/ordered-list-start-one.error b/corpus/unspellable/ordered-list-start-one.error new file mode 100644 index 0000000..e7b3b24 --- /dev/null +++ b/corpus/unspellable/ordered-list-start-one.error @@ -0,0 +1 @@ +ambiguous-ordered-list-start diff --git a/corpus/unspellable/ordered-list-start-one.json b/corpus/unspellable/ordered-list-start-one.json new file mode 100644 index 0000000..93d7709 --- /dev/null +++ b/corpus/unspellable/ordered-list-start-one.json @@ -0,0 +1,28 @@ +{ + "content": [ + { + "attrs": { + "order": 1 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Loosen the clamp", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + } + ], + "type": "doc", + "version": 1 +} diff --git a/package.json b/package.json index 93bc33d..89ec78f 100644 --- a/package.json +++ b/package.json @@ -13,7 +13,7 @@ "node": ">=24" }, "scripts": { - "test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=93 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"", + "test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=94 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"", "typecheck": "tsc --noEmit" }, "devDependencies": { diff --git a/src/adf-to-markdown.test.ts b/src/adf-to-markdown.test.ts index 9126351..28f3f71 100644 --- a/src/adf-to-markdown.test.ts +++ b/src/adf-to-markdown.test.ts @@ -1,7 +1,7 @@ import assert from 'node:assert/strict' import test from 'node:test' -import type { AdfDocument, AdfMark, AdfNode } from './adf-document.ts' +import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './adf-document.ts' import type { Result } from './result.ts' import { adfToMarkdown } from './index.ts' @@ -116,7 +116,8 @@ test('refuses two adjacent lists of the same kind', () => { }) test('refuses a node type the canonical form does not cover', () => { - assert.equal(code(adfToMarkdown(document({ type: 'panel' }))), 'unsupported-node-type') + assert.equal(code(adfToMarkdown(document({ type: 'blockCard' }))), 'unsupported-node-type') + assert.equal(code(adfToMarkdown(document({ type: 'toString' }))), 'unsupported-node-type') assert.equal(code(adfToMarkdown(document(paragraph({ type: 'mention' })))), 'unsupported-node-type') }) @@ -283,3 +284,92 @@ test('refuses a document nested deeper than the emitter carries', () => { test('emits an empty list item without trailing whitespace', () => { assert.equal(markdown(adfToMarkdown(document({ content: [{ type: 'listItem' }], type: 'bulletList' }))), '-\n') }) + +test('spells a block directive as its node type, arg and attributes', () => { + const panel = (attrs: AdfAttributes): AdfDocument => document({ attrs, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' }) + assert.equal(markdown(adfToMarkdown(panel({ panelType: 'warning' }))), ':::panel warning\nx\n:::\n') + assert.equal(markdown(adfToMarkdown(panel({}))), ':::panel\nx\n:::\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'caption' }))), ':::caption\n:::\n') + assert.equal(markdown(adfToMarkdown(document({ attrs: { localId: 'a' }, type: 'syncBlock' }))), '::syncBlock {localId=a}\n') +}) + +test('refuses a directive attribute no section spells', () => { + assert.equal(code(adfToMarkdown(document({ attrs: { rounded: true }, type: 'panel' }))), 'unspelled-node-attribute') + assert.equal(code(adfToMarkdown(document({ attrs: { toString: 'x' }, type: 'panel' }))), 'unspelled-node-attribute') + assert.equal(code(adfToMarkdown(document({ attrs: { localId: 4 }, type: 'panel' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ attrs: { width: '50' }, type: 'layoutColumn' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ attrs: { isNumberColumnEnabled: 'true' }, type: 'table' }))), 'unsupported-node-shape') +}) + +test('refuses an arg slot value no bare token spells', () => { + assert.equal(code(adfToMarkdown(document({ attrs: { panelType: 'extra info' }, type: 'panel' }))), 'unspelled-node-attribute') + assert.equal(code(adfToMarkdown(document({ attrs: { state: 2 }, type: 'taskItem' }))), 'unspelled-node-attribute') +}) + +test('carries a block node mark in the reserved attribute', () => { + const section = (...marks: AdfMark[]): AdfDocument => document({ marks, type: 'layoutSection' }) + assert.equal(markdown(adfToMarkdown(section({ type: 'breakout' }))), ':::layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n:::\n') + assert.equal(markdown(adfToMarkdown(section({ attrs: {}, type: 'breakout' }))), ':::layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n:::\n') +}) + +test('refuses the content a directive body has no room for', () => { + assert.equal(code(adfToMarkdown(document({ content: [paragraph()], type: 'media' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ text: 'x', type: 'panel' }))), 'unsupported-node-shape') +}) + +test('separates two directive blocks in a container body by one line, two CommonMark blocks by a blank one', () => { + const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) + const panel = (...content: AdfNode[]): AdfDocument => document({ attrs: { panelType: 'info' }, content, type: 'panel' }) + assert.equal(markdown(adfToMarkdown(panel(text('a'), text('b')))), ':::panel info\na\n\nb\n:::\n') + assert.equal(markdown(adfToMarkdown(panel({ type: 'caption' }, { type: 'caption' }))), '::::panel info\n:::caption\n:::\n:::caption\n:::\n::::\n') + assert.equal(code(adfToMarkdown(panel(text('a'), { type: 'caption' }))), 'unspelled-block-separation') + assert.equal(code(adfToMarkdown(panel({ type: 'caption' }, text('a')))), 'unspelled-block-separation') + assert.equal(code(adfToMarkdown(panel(paragraph(), text('a')))), 'unspelled-block-separation') +}) + +test('spells the image form for exactly the centered external media shape', () => { + const url = 'https://example.com/moon.png' + const single = (attrs: AdfAttributes, ...content: AdfNode[]): AdfDocument => + document({ attrs: { layout: 'center' }, content: [{ attrs, content, type: 'media' }], type: 'mediaSingle' }) + assert.equal(markdown(adfToMarkdown(single({ alt: 'The moon', type: 'external', url }))), `![The moon](${url})\n`) + assert.equal(markdown(adfToMarkdown(single({ type: 'external', url }))), `![](${url})\n`) + assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`) + assert.equal(code(adfToMarkdown(single({ alt: '', type: 'external', url }))), 'ambiguous-empty-media-alt') + assert.equal(code(adfToMarkdown(single({ alt: 'a\nb', type: 'external', url }))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(single({ alt: 'a\u0000b', type: 'external', url }))), 'unspellable-character') + assert.equal(code(adfToMarkdown(single({ type: 'external', url: 'https://example.com/a b>c' }))), 'unspellable-link-destination') + assert.equal(code(adfToMarkdown(single({ alt: 4, type: 'external', url }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(single({ type: 'external', url: 4 }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(single({ type: 'external', url }, paragraph()))), 'unsupported-node-shape') +}) + +test('spells a mediaSingle the image form does not fit as a directive', () => { + const media: AdfNode = { attrs: { type: 'external', url: 'https://example.com/moon.png' }, type: 'media' } + const single: AdfNode = { attrs: { layout: 'center' }, content: [media], marks: [{ type: 'border' }], type: 'mediaSingle' } + assert.equal( + markdown(adfToMarkdown(document(single))), + ':::mediaSingle {layout=center marks="[{\\"type\\":\\"border\\"}]"}\n::media {type=external url="https://example.com/moon.png"}\n:::\n', + ) +}) + +test('spells a table as a pipe table only where every row and cell is plain', () => { + const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) + const cell = (type: string, ...content: AdfNode[]): AdfNode => ({ content, type }) + const row = (...cells: AdfNode[]): AdfNode => ({ content: cells, type: 'tableRow' }) + const table = (...rows: AdfNode[]): AdfDocument => document({ content: rows, type: 'table' }) + const directive = (result: Result): boolean => markdown(result).startsWith(':') + const header = row(cell('tableHeader', text('Part'))) + assert.equal(markdown(adfToMarkdown(table(header, row(cell('tableCell', text('Bolt M8')))))), '| Part |\n| --- |\n| Bolt M8 |\n') + assert.equal(markdown(adfToMarkdown(table(row(cell('tableHeader', text('a|b')))))), '| a\\|b |\n| --- |\n') + assert.ok(directive(adfToMarkdown(table()))) + assert.ok(directive(adfToMarkdown(table(row())))) + assert.ok(directive(adfToMarkdown(table({ type: 'tableRow' })))) + assert.ok(directive(adfToMarkdown(table(header, row())))) + assert.ok(directive(adfToMarkdown(table(row(cell('tableCell', text('Bolt M8'))))))) + assert.ok(directive(adfToMarkdown(table(cell('tableHeader', text('Part')))))) + assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader')))))) + assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', text('a'), text('b'))))))) + assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { attrs: { localId: 'a' }, type: 'paragraph' }))))), 'unspelled-node-attribute') + assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' })))))) + assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace') +}) diff --git a/src/adf-to-markdown.ts b/src/adf-to-markdown.ts index 6378c79..9554e31 100644 --- a/src/adf-to-markdown.ts +++ b/src/adf-to-markdown.ts @@ -1,46 +1,75 @@ import type { AdfDocument, AdfNode } from './adf-document.ts' +import type { BlockDirective } from './block-directives.ts' import type { JsonValue } from './json-value.ts' -import { emitInlineLine } from './markdown-inline.ts' +import { blockDirective, spellDirectiveHeader } from './block-directives.ts' +import { emitImageLine, emitInlineLine } from './markdown-inline.ts' +import { emitPipeTable } from './markdown-tables.ts' import { failure, success, type ConvertErrorPath, type Result } from './result.ts' import { holdsNullCharacter, isThematicBreak } from './commonmark-grammar.ts' import { isAdfDocument } from './adf-document.ts' import { largestNesting } from './nesting.ts' import { longestBacktickRun } from './backtick-runs.ts' +import { serializeCanonicalJson } from './canonical-json.ts' +type BlockContainer = 'directive' | 'document' | 'list-item' +type EmittedBody = { fenceColons: number; text: string } +type EmittedBlock = EmittedBody & { node: AdfNode; path: ConvertErrorPath } + +const centeredMediaSingle = '{"layout":"center"}' +const imageAttributes = ['alt', 'type', 'url'] const largestListMarker = 999999999 const listTypes = ['bulletList', 'orderedList'] export function adfToMarkdown(document: AdfDocument): Result { if (!isAdfDocument(document)) return failure('not-an-adf-document', 'the value is not an ADF document', []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) - const blocks = emitBlocks(document.content ?? [], false, [], 0) + const blocks = emitBlocks(document.content ?? [], 'document', [], 0) if (!blocks.ok) return blocks - return success(blocks.value === '' ? '' : `${blocks.value}\n`) + return success(blocks.value.text === '' ? '' : `${blocks.value.text}\n`) } -function emitBlocks(nodes: readonly AdfNode[], inListItem: boolean, path: ConvertErrorPath, depth: number): Result { +function emitBlocks(nodes: readonly AdfNode[], container: BlockContainer, path: ConvertErrorPath, depth: number): Result { if (depth > largestNesting) return failure('unsupported-node-shape', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) - let output = '' - let previous: AdfNode | undefined + const blocks: EmittedBlock[] = [] for (const [index, node] of nodes.entries()) { const nodePath = [...path, 'content', index] - if (previous !== undefined) { - if (listTypes.includes(node.type) && previous.type === node.type) { - return failure('unspellable-adjacent-lists', `two adjacent ${node.type} nodes read back as one list`, nodePath) - } - if (inListItem && listTypes.includes(node.type)) { - if (!interruptsParagraph(node)) { - return failure('unspellable-line-start', `a ${node.type} that cannot interrupt the block above it has no tight spelling`, nodePath) - } - output += '\n' - } else output += '\n\n' - } const block = emitBlock(node, nodePath, depth) if (!block.ok) return block - output += block.value - previous = node + blocks.push({ ...block.value, node, path: nodePath }) } - return success(output) + let fenceColons = 0 + let text = '' + for (const [index, block] of blocks.entries()) { + const previous = blocks[index - 1] + if (previous !== undefined) { + const separation = separationBetween(previous, block, container) + if (!separation.ok) return separation + text += separation.value + } + fenceColons = Math.max(fenceColons, block.fenceColons) + text += block.text + } + return success({ fenceColons, text }) +} + +function separationBetween(previous: EmittedBlock, next: EmittedBlock, container: BlockContainer): Result { + if (listTypes.includes(next.node.type) && previous.node.type === next.node.type) { + return failure('unspellable-adjacent-lists', `two adjacent ${next.node.type} nodes read back as one list`, next.path) + } + if (container === 'list-item' && listTypes.includes(next.node.type)) { + if (!interruptsParagraph(next.node)) { + return failure('unspellable-line-start', `a ${next.node.type} that cannot interrupt the block above it has no tight spelling`, next.path) + } + return success('\n') + } + if (container !== 'directive') return success('\n\n') + if (previous.fenceColons === 0 && next.fenceColons === 0) return success('\n\n') + if (previous.fenceColons > 0 && next.fenceColons > 0) return success('\n') + return failure( + 'unspelled-block-separation', + `the canonical form leaves the separation between a ${previous.node.type} and a ${next.node.type} in a container body unspelled`, + next.path, + ) } function interruptsParagraph(node: AdfNode): boolean { @@ -48,26 +77,86 @@ function interruptsParagraph(node: AdfNode): boolean { return ((node.content ?? [])[0]?.content ?? []).length > 0 } -function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result { - if (node.type === 'blockquote') return emitBlockquote(node, path, depth) - if (node.type === 'bulletList' || node.type === 'orderedList') return emitList(node, path, depth) - if (node.type === 'codeBlock') return emitCodeBlock(node, path) - if (node.type === 'heading') return emitHeading(node, path) +function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result { + if (node.type === 'blockquote') return plainBlock(emitBlockquote(node, path, depth)) + if (node.type === 'bulletList' || node.type === 'orderedList') return plainBlock(emitList(node, path, depth)) + if (node.type === 'codeBlock') return plainBlock(emitCodeBlock(node, path)) + if (node.type === 'heading') return plainBlock(emitHeading(node, path)) if (node.type === 'paragraph') return emitParagraph(node, path) - if (node.type === 'rule') return emitRule(node, path) + if (node.type === 'rule') return plainBlock(emitRule(node, path)) + const directive = blockDirective(node.type) + if (directive !== undefined) { + if (node.type === 'mediaSingle') return emitMediaSingle(node, directive, path, depth) + if (node.type === 'table') return emitTable(node, directive, path, depth) + return emitDirectiveBlock(node, directive, path, depth) + } if (node.type === 'hardBreak' || node.type === 'listItem' || node.type === 'text') { return failure('unsupported-node-shape', `a ${node.type} node cannot stand where a block belongs`, path) } return failure('unsupported-node-type', `the canonical form spells no block node of type ${node.type}`, path) } +function plainBlock(text: Result): Result { + if (!text.ok) return text + return success({ fenceColons: 0, text: text.value }) +} + +function emitDirectiveBlock(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { + if (node.text !== undefined) return failure('unsupported-node-shape', `a ${node.type} carries no text`, path) + const header = spellDirectiveHeader(node, directive, path) + if (!header.ok) return header + const content = node.content ?? [] + if (directive.body === 'none') { + if (content.length > 0) return failure('unsupported-node-shape', `a ${node.type} holds no content`, path) + return success({ fenceColons: 2, text: `::${header.value}` }) + } + const body = directive.body === 'inline' ? emitInlineBody(content, path) : emitBlocks(content, 'directive', path, depth + 1) + if (!body.ok) return body + const fenceColons = Math.max(3, body.value.fenceColons + 1) + const fence = ':'.repeat(fenceColons) + const lines = body.value.text === '' ? '' : `${body.value.text}\n` + return success({ fenceColons, text: `${fence}${header.value}\n${lines}${fence}` }) +} + +function emitInlineBody(content: readonly AdfNode[], path: ConvertErrorPath): Result { + if (content.length === 0) return success({ fenceColons: 0, text: '' }) + return plainBlock(emitInlineLine(content, 'paragraph', path)) +} + +function emitMediaSingle(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { + const image = imageShape(node) + if (image === undefined) return emitDirectiveBlock(node, directive, path, depth) + const mediaPath = [...path, 'content', 0] + if (image.alt === '') return failure('ambiguous-empty-media-alt', 'an empty media alt and an absent one share one image spelling', mediaPath) + return plainBlock(emitImageLine(image.alt, image.url, mediaPath)) +} + +function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { + const content = node.content ?? [] + const media = content[0] + if (serializeCanonicalJson(node.attrs ?? {}, 'compact') !== centeredMediaSingle || (node.marks ?? []).length > 0) return undefined + if (media === undefined || content.length !== 1 || media.type !== 'media' || (media.marks ?? []).length > 0 || (media.content ?? []).length > 0) return undefined + const attrs = media.attrs ?? {} + const alt = attrs['alt'] + const url = attrs['url'] + if (Object.keys(attrs).some((key) => !imageAttributes.includes(key)) || attrs['type'] !== 'external') return undefined + if (typeof url !== 'string' || (alt !== undefined && typeof alt !== 'string')) return undefined + return { alt, url } +} + +function emitTable(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { + const pipe = emitPipeTable(node, path) + if (pipe === undefined) return emitDirectiveBlock(node, directive, path, depth) + return plainBlock(pipe) +} + function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result { const validation = validateBlockNode(node, [], path) if (!validation.ok) return validation - const inner = emitBlocks(node.content ?? [], false, path, depth + 1) + const inner = emitBlocks(node.content ?? [], 'document', path, depth + 1) if (!inner.ok) return inner return success( - inner.value + inner.value.text .split('\n') .map((line) => (line === '' ? '>' : `> ${line}`)) .join('\n'), @@ -152,11 +241,11 @@ function emitList(node: AdfNode, path: ConvertErrorPath, depth: number): Result< function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, depth: number): Result { const validation = validateBlockNode(item, [], path) if (!validation.ok) return validation - const inner = emitBlocks(item.content ?? [], true, path, depth + 1) + const inner = emitBlocks(item.content ?? [], 'list-item', path, depth + 1) if (!inner.ok) return inner - if (inner.value === '') return success(marker.trimEnd()) + if (inner.value.text === '') return success(marker.trimEnd()) const indent = ' '.repeat(marker.length) - const lines = inner.value.split('\n').map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`)) + const lines = inner.value.text.split('\n').map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`)) const first = lines[0] ?? '' if (isThematicBreak(first)) { return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(first)}`, path) @@ -164,12 +253,12 @@ function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, dep return success(lines.join('\n')) } -function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result { +function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result { const validation = validateBlockNode(node, [], path) if (!validation.ok) return validation const content = node.content ?? [] - if (content.length === 0) return success('::paragraph') - return emitInlineLine(content, 'paragraph', path) + if (content.length === 0) return success({ fenceColons: 2, text: '::paragraph' }) + return plainBlock(emitInlineLine(content, 'paragraph', path)) } function emitRule(node: AdfNode, path: ConvertErrorPath): Result { diff --git a/src/block-directives.ts b/src/block-directives.ts new file mode 100644 index 0000000..5c88841 --- /dev/null +++ b/src/block-directives.ts @@ -0,0 +1,116 @@ +import type { AdfMark, AdfNode } from './adf-document.ts' +import type { AttributeKind } from './directive-attributes.ts' +import type { JsonValue } from './json-value.ts' +import { failure, success, type ConvertErrorPath, type Result } from './result.ts' +import { isBareToken, spellAttributeValue, spellAttributes, spellJsonAttribute } from './directive-attributes.ts' + +export type BlockDirective = { + argument?: string + attributes: Readonly> + body: 'block' | 'inline' | 'none' +} + +const cellAttributes: Readonly> = { + background: 'string', + colspan: 'number', + colwidth: 'json', + localId: 'string', + rowspan: 'number', + valign: 'string', +} + +const expandAttributes: Readonly> = { localId: 'string', title: 'string' } + +const extensionAttributes: Readonly> = { + extensionKey: 'string', + extensionType: 'string', + layout: 'string', + localId: 'string', + parameters: 'json', + text: 'string', +} + +const itemAttributes: Readonly> = { localId: 'string' } + +const mediaAttributes: Readonly> = { + alt: 'string', + collection: 'string', + height: 'number', + id: 'string', + localId: 'string', + occurrenceKey: 'string', + type: 'string', + url: 'string', + width: 'number', +} + +const syncBlockAttributes: Readonly> = { localId: 'string', resourceId: 'string' } + +const blockDirectives: Readonly> = { + blockTaskItem: { argument: 'state', attributes: itemAttributes, body: 'block' }, + bodiedExtension: { attributes: extensionAttributes, body: 'block' }, + bodiedSyncBlock: { attributes: syncBlockAttributes, body: 'block' }, + caption: { attributes: itemAttributes, body: 'inline' }, + decisionItem: { attributes: { localId: 'string', state: 'string' }, body: 'inline' }, + decisionList: { attributes: itemAttributes, body: 'block' }, + expand: { attributes: expandAttributes, body: 'block' }, + extension: { attributes: extensionAttributes, body: 'none' }, + extensionFrame: { attributes: {}, body: 'block' }, + layoutColumn: { attributes: { localId: 'string', valign: 'string', width: 'number' }, body: 'block' }, + layoutSection: { attributes: itemAttributes, body: 'block' }, + media: { attributes: mediaAttributes, body: 'none' }, + mediaGroup: { attributes: {}, body: 'block' }, + mediaSingle: { attributes: { layout: 'string', localId: 'string', width: 'number', widthType: 'string' }, body: 'block' }, + multiBodiedExtension: { attributes: extensionAttributes, body: 'block' }, + nestedExpand: { attributes: expandAttributes, body: 'block' }, + panel: { + argument: 'panelType', + attributes: { localId: 'string', panelColor: 'string', panelIcon: 'string', panelIconId: 'string', panelIconText: 'string' }, + body: 'block', + }, + syncBlock: { attributes: syncBlockAttributes, body: 'none' }, + table: { attributes: { displayMode: 'string', isNumberColumnEnabled: 'boolean', layout: 'string', localId: 'string', width: 'number' }, body: 'block' }, + tableCell: { attributes: cellAttributes, body: 'block' }, + tableHeader: { attributes: cellAttributes, body: 'block' }, + tableRow: { attributes: itemAttributes, body: 'block' }, + taskItem: { argument: 'state', attributes: itemAttributes, body: 'inline' }, + taskList: { attributes: itemAttributes, body: 'block' }, +} + +export function blockDirective(type: string): BlockDirective | undefined { + return Object.hasOwn(blockDirectives, type) ? blockDirectives[type] : undefined +} + +export function spellDirectiveHeader(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath): Result { + const pairs: [string, string][] = [] + let argument = '' + for (const [key, value] of Object.entries(node.attrs ?? {})) { + if (key === directive.argument) { + if (typeof value !== 'string' || !isBareToken(value)) { + return failure('unspelled-node-attribute', `the ${node.type} attribute ${key} holds no bare token the arg slot spells`, path) + } + argument = ` ${value}` + continue + } + const kind = attributeKind(directive, key) + if (kind === undefined) return failure('unspelled-node-attribute', `the ${node.type} attribute ${key} has no canonical markdown spelling`, path) + const spelled = spellAttributeValue(value, kind) + if (spelled === undefined) return failure('unsupported-node-shape', `the ${node.type} attribute ${key} holds no ${kind}`, path) + pairs.push([key, spelled]) + } + const marks = node.marks ?? [] + if (marks.length > 0) pairs.push(['marks', spellJsonAttribute(markValues(marks))]) + const attributes = spellAttributes(pairs) + return success(`${node.type}${argument}${attributes === '' ? '' : ` ${attributes}`}`) +} + +function attributeKind(directive: BlockDirective, key: string): AttributeKind | undefined { + return Object.hasOwn(directive.attributes, key) ? directive.attributes[key] : undefined +} + +function markValues(marks: readonly AdfMark[]): JsonValue { + return marks.map((mark) => { + const attrs = mark.attrs ?? {} + return Object.keys(attrs).length === 0 ? { type: mark.type } : { attrs, type: mark.type } + }) +} diff --git a/src/corpus.test.ts b/src/corpus.test.ts index 18345e6..79118fd 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -11,9 +11,10 @@ import { serializeCanonicalJson } from './canonical-json.ts' const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus') const roundTripRoot = join(corpusRoot, 'round-trip') +const unspellableRoot = join(corpusRoot, 'unspellable') -const emittingDirectories = ['commonmark-subset'] -const pendingDirectories = ['block-nodes', 'inline-nodes'] +const emittingDirectories = ['block-nodes', 'commonmark-subset'] +const pendingDirectories = ['inline-nodes'] function directoryNames(root: string): string[] { return readdirSync(root, { withFileTypes: true }) @@ -23,7 +24,11 @@ function directoryNames(root: string): string[] { } function fixtureNames(directory: string, extension: string): string[] { - return readdirSync(join(roundTripRoot, directory)) + return names(join(roundTripRoot, directory), extension) +} + +function names(root: string, extension: string): string[] { + return readdirSync(root) .filter((name) => name.endsWith(extension)) .map((name) => name.slice(0, -extension.length)) .sort() @@ -67,6 +72,24 @@ for (const directory of emittingDirectories) { } } +test('unspellable pairs every .json with an .error', () => { + assert.deepEqual(names(unspellableRoot, '.json'), names(unspellableRoot, '.error')) +}) + +test('unspellable holds fixtures', () => { + assert.ok(names(unspellableRoot, '.json').length > 0) +}) + +for (const name of names(unspellableRoot, '.json')) { + test(`unspellable/${name} is refused with the error it names`, () => { + const parsed: unknown = JSON.parse(readFileSync(join(unspellableRoot, `${name}.json`), 'utf8')) + assert.ok(isAdfDocument(parsed), `${name}.json is not an ADF document`) + const result = adfToMarkdown(parsed) + assert.ok(!result.ok, result.ok ? `emitted ${JSON.stringify(result.value)}` : '') + assert.equal(result.error.code, readFileSync(join(unspellableRoot, `${name}.error`), 'utf8').trimEnd()) + }) +} + test('the corpus holds JSON to gate', () => { assert.ok(corpusJsonPaths().length > 0) }) diff --git a/src/directive-attributes.ts b/src/directive-attributes.ts new file mode 100644 index 0000000..76f697b --- /dev/null +++ b/src/directive-attributes.ts @@ -0,0 +1,35 @@ +import type { JsonValue } from './json-value.ts' +import { serializeCanonicalJson } from './canonical-json.ts' + +export type AttributeKind = 'boolean' | 'json' | 'number' | 'string' + +const bareToken = /^[A-Za-z0-9_-]+$/ + +export function isBareToken(text: string): boolean { + return bareToken.test(text) +} + +export function spellAttributes(pairs: readonly (readonly [string, string])[]): string { + if (pairs.length === 0) return '' + const spelled = [...pairs].sort(([left], [right]) => (left < right ? -1 : 1)).map(([key, value]) => `${key}=${value}`) + return `{${spelled.join(' ')}}` +} + +export function spellAttributeValue(value: JsonValue, kind: AttributeKind): string | undefined { + if (kind === 'json') return spellJsonAttribute(value) + if (kind === 'boolean') return typeof value === 'boolean' ? `${value}` : undefined + if (kind === 'number') return typeof value === 'number' ? spell(JSON.stringify(value)) : undefined + return typeof value === 'string' ? spell(value) : undefined +} + +export function spellJsonAttribute(value: JsonValue): string { + return quote(serializeCanonicalJson(value, 'compact')) +} + +function spell(text: string): string { + return isBareToken(text) ? text : quote(text) +} + +function quote(text: string): string { + return JSON.stringify(text) +} diff --git a/src/markdown-escaping.ts b/src/markdown-escaping.ts index 833528d..d2a8d3c 100644 --- a/src/markdown-escaping.ts +++ b/src/markdown-escaping.ts @@ -6,7 +6,7 @@ export type InlineSegment = export type AssembledLine = { line: string; unspellableMark: string | undefined } -export type LineContainer = 'heading' | 'paragraph' +export type LineContainer = 'heading' | 'paragraph' | 'table-cell' type DelimiterRun = { character: string; closeMark: string | undefined; end: number; openMark: string | undefined; start: number } @@ -127,8 +127,9 @@ function isSyntax(kind: InlineSegment['kind'] | undefined): boolean { } function opensConstruct(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet): boolean { - const claimsLine = container === 'heading' ? closesHeading(scan, index) : claimsLineStart(scan, index) - return claimsLine || claimsCharacter(scan, index, inLinkText, escaped) + if (container === 'heading' && closesHeading(scan, index)) return true + if (container === 'paragraph' && claimsLineStart(scan, index)) return true + return claimsCharacter(scan, index, inLinkText, container, escaped) } function claimsLineStart(scan: string, index: number): boolean { @@ -144,10 +145,11 @@ function closesHeading(scan: string, index: number): boolean { return index === 0 || /[ \t]/.test(scan.charAt(index - 1)) } -function claimsCharacter(scan: string, index: number, inLinkText: boolean, escaped: ReadonlySet): boolean { +function claimsCharacter(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet): boolean { const character = scan.charAt(index) const rest = scan.slice(index) if (inLinkText && (character === '[' || character === ']')) return true + if (character === '|') return container === 'table-cell' if (character === '\\') return asciiPunctuation.test(scan.charAt(index + 1)) if (character === '&') return startsEntityReference(rest) if (character === '<') return opensBracketedAutolink(rest) || htmlConstructs.some((construct) => construct.test(rest)) diff --git a/src/markdown-inline.ts b/src/markdown-inline.ts index 66c8c58..54f0332 100644 --- a/src/markdown-inline.ts +++ b/src/markdown-inline.ts @@ -20,7 +20,20 @@ const linkAttributes = ['href', 'title'] export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result { const segments = emitRun(nodes, 0, 0, { atBlockEnd: true, container, inLinkText: false, path }) if (!segments.ok) return segments - const assembled = assembleInlineLine(segments.value, container) + return finishLine(segments.value, container, path) +} + +export function emitImageLine(alt: string | undefined, href: string, path: ConvertErrorPath): Result { + if (alt !== undefined && /[\n\r]/.test(alt)) return failure('unspellable-whitespace', 'a media alt holds a newline no image description spells', path) + if (alt !== undefined && holdsNullCharacter(alt)) return failure('unspellable-character', 'a media alt holds a null character CommonMark replaces', path) + const destination = spellDestination(href, path) + if (!destination.ok) return destination + const description: InlineSegment[] = alt === undefined ? [] : [{ kind: 'link-text', text: alt }] + return finishLine([{ kind: 'syntax', text: '![' }, ...description, { kind: 'syntax', text: `](${destination.value})` }], 'paragraph', path) +} + +function finishLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath): Result { + const assembled = assembleInlineLine(segments, container) if (assembled.unspellableMark !== undefined) { return failure('unspellable-mark', `the ${assembled.unspellableMark} spelling cannot open or close where it sits`, path) } @@ -83,8 +96,8 @@ function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result< const types = (node.marks ?? []).map((mark) => mark.type) if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path) if (node.type === 'hardBreak') { - if (context.container === 'heading' || context.atBlockEnd) return success([{ kind: 'syntax', text: ':hardBreak{}' }]) - return success([{ kind: 'syntax', text: '\\\n' }]) + if (context.container === 'paragraph' && !context.atBlockEnd) return success([{ kind: 'syntax', text: '\\\n' }]) + return success([{ kind: 'syntax', text: ':hardBreak{}' }]) } if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node carries no text', path) if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node carries content', path) diff --git a/src/markdown-tables.ts b/src/markdown-tables.ts new file mode 100644 index 0000000..e4d6dc3 --- /dev/null +++ b/src/markdown-tables.ts @@ -0,0 +1,53 @@ +import type { AdfNode } from './adf-document.ts' +import { emitInlineLine } from './markdown-inline.ts' +import { success, type ConvertErrorPath, type Result } from './result.ts' + +export function emitPipeTable(node: AdfNode, path: ConvertErrorPath): Result | undefined { + const rows = pipeRows(node) + if (rows === undefined) return undefined + const lines: string[] = [] + for (const [rowIndex, row] of rows.entries()) { + const cells: string[] = [] + for (const [cellIndex, paragraph] of row.entries()) { + const content = paragraph.content ?? [] + const line = content.length === 0 ? success('') : emitInlineLine(content, 'table-cell', [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0]) + if (!line.ok) return line + cells.push(line.value) + } + lines.push(`| ${cells.join(' | ')} |`) + if (rowIndex === 0) lines.push(`| ${cells.map(() => '---').join(' | ')} |`) + } + return success(lines.join('\n')) +} + +function pipeRows(node: AdfNode): AdfNode[][] | undefined { + const rows = node.content ?? [] + const columns = (rows[0]?.content ?? []).length + if (!isPlain(node) || columns === 0) return undefined + const grid: AdfNode[][] = [] + for (const [index, row] of rows.entries()) { + const cells = row.content ?? [] + if (row.type !== 'tableRow' || !isPlain(row) || cells.length !== columns) return undefined + const wanted = index === 0 ? 'tableHeader' : 'tableCell' + const paragraphs: AdfNode[] = [] + for (const cell of cells) { + const paragraph = plainParagraph(cell) + if (paragraph === undefined || cell.type !== wanted || !isPlain(cell)) return undefined + paragraphs.push(paragraph) + } + grid.push(paragraphs) + } + return grid +} + +function isPlain(node: AdfNode): boolean { + return Object.keys(node.attrs ?? {}).length === 0 && (node.marks ?? []).length === 0 && node.text === undefined +} + +function plainParagraph(cell: AdfNode): AdfNode | undefined { + const content = cell.content ?? [] + const paragraph = content[0] + if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !isPlain(paragraph)) return undefined + const pipedCode = (paragraph.content ?? []).some((child) => (child.marks ?? []).some((mark) => mark.type === 'code') && (child.text ?? '').includes('|')) + return pipedCode ? undefined : paragraph +} diff --git a/src/result.ts b/src/result.ts index 89969bc..f2edebd 100644 --- a/src/result.ts +++ b/src/result.ts @@ -1,5 +1,6 @@ export type ConvertErrorCode = | 'ambiguous-empty-code-block-language' + | 'ambiguous-empty-media-alt' | 'ambiguous-ordered-list-start' | 'not-an-adf-document' | 'reserved-adf-language' @@ -13,6 +14,7 @@ export type ConvertErrorCode = | 'unspellable-mark' | 'unspellable-whitespace' | 'unspelled-block-marks' + | 'unspelled-block-separation' | 'unspelled-node-attribute' | 'unsupported-document-version' | 'unsupported-heading-level' diff --git a/todo.md b/todo.md index 447e9c8..bf79310 100644 --- a/todo.md +++ b/todo.md @@ -28,12 +28,12 @@ detail is settled at its own milestone. carries a `localId` with no spelling, `codeBlock` also `hideLineNumbers`, `uniqueId` and `wrap`, `blockquote` also marks, and `hardBreak` `text` and `localId` with no section for the carry fallback to reach. Picking one (directive sections for those nodes, or the opaque - carry) is a permanent format decision (§8). Three collision sites are held out of the corpus - meanwhile, each a choice between the absent attribute and the empty value: a `codeBlock` - whose info string is empty and `media` with an empty `alt`, which one "exactly that shape" - rule — as the CommonMark image already uses — could settle together, and an `orderedList` - starting at 1, independent of the totality answer since `order: 9` keeps the markdown form - either way. **Also blocked**: the link rule covers destination spaces only, so two shapes + carry) is a permanent format decision (§8). Three collision sites wait in + `corpus/unspellable/` meanwhile, each a choice between the absent attribute and the empty + value: a `codeBlock` whose info string is empty and `media` with an empty `alt`, which one + "exactly that shape" rule — as the CommonMark image already uses — could settle together, + and an `orderedList` starting at 1, independent of the totality answer since `order: 9` + keeps the markdown form either way. **Also blocked**: the link rule covers destination spaces only, so two shapes break §2 silently — href `https://example.com/a)b` emits `[t](https://example.com/a)b)`, read back as href `…/a` plus literal `b)`; title `He said "hi"` emits `[t](u "He said "hi"")`, which holds no title. Two defensible spellings each — angle @@ -42,7 +42,8 @@ detail is settled at its own milestone. directive block in a container body — an `expand` whose content is `paragraph` "A" then a `panel` (`panelType` `warning`) holding "B" spells `A` and `:::panel warning` either on consecutive lines or with a blank line between. Two defensible spellings, so §8 leaves the - pick here. + pick here; `unspelled-block-separation` refuses the pair meanwhile, an empty paragraph's + `::paragraph` beside a CommonMark block included. - [x] **1d1 — The CommonMark subset**: blockquote, bulletList, codeBlock, heading, orderedList, paragraph, rule, listItem, hardBreak, text, code spans, and the `code`, `em`, `link`, `strike` and `strong` marks — one mark per text node; nesting is 1d3's. From f065783f8fd3423876aebf3502c580b4335439f6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 25 Aug 2026 15:33:24 +0200 Subject: [PATCH 2/6] Carry the fence depth through blockquotes and lists, and split the block spelling from it --- corpus/README.md | 6 +- .../block-nodes/container-crossing.json | 77 ++++++++++ .../block-nodes/container-crossing.md | 10 ++ src/adf-document.ts | 2 + src/adf-to-markdown.ts | 140 +++++++++--------- src/block-directives.ts | 3 +- src/corpus.test.ts | 44 ++++++ src/directive-attributes.ts | 3 +- src/markdown-image.ts | 28 ++++ ...kdown-tables.ts => markdown-pipe-table.ts} | 0 todo.md | 9 +- 11 files changed, 243 insertions(+), 79 deletions(-) create mode 100644 corpus/round-trip/block-nodes/container-crossing.json create mode 100644 corpus/round-trip/block-nodes/container-crossing.md create mode 100644 src/markdown-image.ts rename src/{markdown-tables.ts => markdown-pipe-table.ts} (100%) diff --git a/corpus/README.md b/corpus/README.md index 6df1efe..23ecb71 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -1,6 +1,6 @@ # The corpus -One directory per contract kind: +One directory per contract kind, each landing with its milestone: - `round-trip/` — `.json` + `.md`: the markdown `adfToMarkdown` must emit for that document, byte for byte, and that `markdownToAdf` must read back to it (AGENTS.md §2). Grouped @@ -10,8 +10,8 @@ One directory per contract kind: - `errors/` — `.md`: markdown input that must not convert. A `.error` beside it pins which error. - `unspellable/` — `.json`: ADF `adfToMarkdown` must refuse, the `ConvertErrorCode` in the - `.error` beside it. A maintainer decision (`todo.md`) moves a document from here to - `round-trip/`. + `.error` beside it. Two populations live here: refusals that stay contract, and documents + a maintainer decision (`todo.md`) moves to `round-trip/`. - `real-payloads/` — `.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No expected markdown. diff --git a/corpus/round-trip/block-nodes/container-crossing.json b/corpus/round-trip/block-nodes/container-crossing.json new file mode 100644 index 0000000..95bd7b3 --- /dev/null +++ b/corpus/round-trip/block-nodes/container-crossing.json @@ -0,0 +1,77 @@ +{ + "content": [ + { + "attrs": { + "panelType": "info" + }, + "content": [ + { + "content": [ + { + "content": [ + { + "attrs": { + "panelType": "warning" + }, + "content": [ + { + "content": [ + { + "text": "Check the torque before signing off.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "panel" + } + ], + "type": "listItem" + }, + { + "content": [ + { + "content": [ + { + "text": "Plain item.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "content": [ + { + "attrs": { + "title": "Full build log" + }, + "content": [ + { + "content": [ + { + "text": "The build ran for 11 minutes.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "expand" + } + ], + "type": "blockquote" + } + ], + "type": "panel" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/container-crossing.md b/corpus/round-trip/block-nodes/container-crossing.md new file mode 100644 index 0000000..8af9334 --- /dev/null +++ b/corpus/round-trip/block-nodes/container-crossing.md @@ -0,0 +1,10 @@ +::::panel info +- :::panel warning + Check the torque before signing off. + ::: +- Plain item. + +> :::expand {title="Full build log"} +> The build ran for 11 minutes. +> ::: +:::: diff --git a/src/adf-document.ts b/src/adf-document.ts index 139457f..747906e 100644 --- a/src/adf-document.ts +++ b/src/adf-document.ts @@ -2,6 +2,8 @@ import { isJsonValue, type JsonValue } from './json-value.ts' export type AdfAttributes = { [key: string]: JsonValue } +export type AttributeKind = 'boolean' | 'json' | 'number' | 'string' + export type AdfMark = { attrs?: AdfAttributes type: string diff --git a/src/adf-to-markdown.ts b/src/adf-to-markdown.ts index 9554e31..54c5b1a 100644 --- a/src/adf-to-markdown.ts +++ b/src/adf-to-markdown.ts @@ -2,21 +2,21 @@ import type { AdfDocument, AdfNode } from './adf-document.ts' import type { BlockDirective } from './block-directives.ts' import type { JsonValue } from './json-value.ts' import { blockDirective, spellDirectiveHeader } from './block-directives.ts' -import { emitImageLine, emitInlineLine } from './markdown-inline.ts' -import { emitPipeTable } from './markdown-tables.ts' +import { emitImage } from './markdown-image.ts' +import { emitInlineLine } from './markdown-inline.ts' +import { emitPipeTable } from './markdown-pipe-table.ts' import { failure, success, type ConvertErrorPath, type Result } from './result.ts' import { holdsNullCharacter, isThematicBreak } from './commonmark-grammar.ts' import { isAdfDocument } from './adf-document.ts' import { largestNesting } from './nesting.ts' import { longestBacktickRun } from './backtick-runs.ts' -import { serializeCanonicalJson } from './canonical-json.ts' type BlockContainer = 'directive' | 'document' | 'list-item' +type BlockSpelling = 'commonmark' | 'directive' type EmittedBody = { fenceColons: number; text: string } -type EmittedBlock = EmittedBody & { node: AdfNode; path: ConvertErrorPath } +type EmittedBlock = EmittedBody & { spelling: BlockSpelling } +type PlacedBlock = EmittedBlock & { node: AdfNode; path: ConvertErrorPath } -const centeredMediaSingle = '{"layout":"center"}' -const imageAttributes = ['alt', 'type', 'url'] const largestListMarker = 999999999 const listTypes = ['bulletList', 'orderedList'] @@ -30,7 +30,7 @@ export function adfToMarkdown(document: AdfDocument): Result { function emitBlocks(nodes: readonly AdfNode[], container: BlockContainer, path: ConvertErrorPath, depth: number): Result { if (depth > largestNesting) return failure('unsupported-node-shape', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) - const blocks: EmittedBlock[] = [] + const blocks: PlacedBlock[] = [] for (const [index, node] of nodes.entries()) { const nodePath = [...path, 'content', index] const block = emitBlock(node, nodePath, depth) @@ -52,7 +52,7 @@ function emitBlocks(nodes: readonly AdfNode[], container: BlockContainer, path: return success({ fenceColons, text }) } -function separationBetween(previous: EmittedBlock, next: EmittedBlock, container: BlockContainer): Result { +function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): Result { if (listTypes.includes(next.node.type) && previous.node.type === next.node.type) { return failure('unspellable-adjacent-lists', `two adjacent ${next.node.type} nodes read back as one list`, next.path) } @@ -63,8 +63,8 @@ function separationBetween(previous: EmittedBlock, next: EmittedBlock, container return success('\n') } if (container !== 'directive') return success('\n\n') - if (previous.fenceColons === 0 && next.fenceColons === 0) return success('\n\n') - if (previous.fenceColons > 0 && next.fenceColons > 0) return success('\n') + if (previous.spelling === 'commonmark' && next.spelling === 'commonmark') return success('\n\n') + if (previous.spelling === 'directive' && next.spelling === 'directive') return success('\n') return failure( 'unspelled-block-separation', `the canonical form leaves the separation between a ${previous.node.type} and a ${next.node.type} in a container body unspelled`, @@ -77,13 +77,13 @@ function interruptsParagraph(node: AdfNode): boolean { return ((node.content ?? [])[0]?.content ?? []).length > 0 } -function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result { - if (node.type === 'blockquote') return plainBlock(emitBlockquote(node, path, depth)) - if (node.type === 'bulletList' || node.type === 'orderedList') return plainBlock(emitList(node, path, depth)) - if (node.type === 'codeBlock') return plainBlock(emitCodeBlock(node, path)) - if (node.type === 'heading') return plainBlock(emitHeading(node, path)) +function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result { + if (node.type === 'blockquote') return commonMarkContainer(emitBlockquote(node, path, depth)) + if (node.type === 'bulletList' || node.type === 'orderedList') return commonMarkContainer(emitList(node, path, depth)) + if (node.type === 'codeBlock') return commonMarkLine(emitCodeBlock(node, path)) + if (node.type === 'heading') return commonMarkLine(emitHeading(node, path)) if (node.type === 'paragraph') return emitParagraph(node, path) - if (node.type === 'rule') return plainBlock(emitRule(node, path)) + if (node.type === 'rule') return commonMarkLine(emitRule(node, path)) const directive = blockDirective(node.type) if (directive !== undefined) { if (node.type === 'mediaSingle') return emitMediaSingle(node, directive, path, depth) @@ -96,71 +96,62 @@ function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result return failure('unsupported-node-type', `the canonical form spells no block node of type ${node.type}`, path) } -function plainBlock(text: Result): Result { +function commonMarkLine(text: Result): Result { if (!text.ok) return text - return success({ fenceColons: 0, text: text.value }) + return success({ fenceColons: 0, spelling: 'commonmark', text: text.value }) } -function emitDirectiveBlock(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { +function commonMarkContainer(body: Result): Result { + if (!body.ok) return body + return success({ ...body.value, spelling: 'commonmark' }) +} + +function emitDirectiveBlock(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { if (node.text !== undefined) return failure('unsupported-node-shape', `a ${node.type} carries no text`, path) const header = spellDirectiveHeader(node, directive, path) if (!header.ok) return header const content = node.content ?? [] if (directive.body === 'none') { if (content.length > 0) return failure('unsupported-node-shape', `a ${node.type} holds no content`, path) - return success({ fenceColons: 2, text: `::${header.value}` }) + return success({ fenceColons: 2, spelling: 'directive', text: `::${header.value}` }) } const body = directive.body === 'inline' ? emitInlineBody(content, path) : emitBlocks(content, 'directive', path, depth + 1) if (!body.ok) return body const fenceColons = Math.max(3, body.value.fenceColons + 1) const fence = ':'.repeat(fenceColons) const lines = body.value.text === '' ? '' : `${body.value.text}\n` - return success({ fenceColons, text: `${fence}${header.value}\n${lines}${fence}` }) + return success({ fenceColons, spelling: 'directive', text: `${fence}${header.value}\n${lines}${fence}` }) } function emitInlineBody(content: readonly AdfNode[], path: ConvertErrorPath): Result { if (content.length === 0) return success({ fenceColons: 0, text: '' }) - return plainBlock(emitInlineLine(content, 'paragraph', path)) + const line = emitInlineLine(content, 'paragraph', path) + if (!line.ok) return line + return success({ fenceColons: 0, text: line.value }) } -function emitMediaSingle(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { - const image = imageShape(node) +function emitMediaSingle(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { + const image = emitImage(node, path) if (image === undefined) return emitDirectiveBlock(node, directive, path, depth) - const mediaPath = [...path, 'content', 0] - if (image.alt === '') return failure('ambiguous-empty-media-alt', 'an empty media alt and an absent one share one image spelling', mediaPath) - return plainBlock(emitImageLine(image.alt, image.url, mediaPath)) + return commonMarkLine(image) } -function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { - const content = node.content ?? [] - const media = content[0] - if (serializeCanonicalJson(node.attrs ?? {}, 'compact') !== centeredMediaSingle || (node.marks ?? []).length > 0) return undefined - if (media === undefined || content.length !== 1 || media.type !== 'media' || (media.marks ?? []).length > 0 || (media.content ?? []).length > 0) return undefined - const attrs = media.attrs ?? {} - const alt = attrs['alt'] - const url = attrs['url'] - if (Object.keys(attrs).some((key) => !imageAttributes.includes(key)) || attrs['type'] !== 'external') return undefined - if (typeof url !== 'string' || (alt !== undefined && typeof alt !== 'string')) return undefined - return { alt, url } -} - -function emitTable(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { +function emitTable(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result { const pipe = emitPipeTable(node, path) if (pipe === undefined) return emitDirectiveBlock(node, directive, path, depth) - return plainBlock(pipe) + return commonMarkLine(pipe) } -function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result { +function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result { const validation = validateBlockNode(node, [], path) if (!validation.ok) return validation const inner = emitBlocks(node.content ?? [], 'document', path, depth + 1) if (!inner.ok) return inner - return success( - inner.value.text - .split('\n') - .map((line) => (line === '' ? '>' : `> ${line}`)) - .join('\n'), - ) + const text = inner.value.text + .split('\n') + .map((line) => (line === '' ? '>' : `> ${line}`)) + .join('\n') + return success({ fenceColons: inner.value.fenceColons, text }) } function emitCodeBlock(node: AdfNode, path: ConvertErrorPath): Result { @@ -211,54 +202,63 @@ function emitHeading(node: AdfNode, path: ConvertErrorPath): Result { return success(`${hashes} ${line.value}`) } -function emitList(node: AdfNode, path: ConvertErrorPath, depth: number): Result { +function emitList(node: AdfNode, path: ConvertErrorPath, depth: number): Result { const ordered = node.type === 'orderedList' const validation = validateBlockNode(node, ordered ? ['order'] : [], path) if (!validation.ok) return validation const items = node.content ?? [] - if (items.length === 0) return failure('unsupported-node-shape', `a ${node.type} holds at least one listItem`, path) - const start = ordered ? node.attrs?.['order'] : 0 - if (ordered && (start === undefined || start === 1)) { - return failure('ambiguous-ordered-list-start', 'an orderedList starting at 1 and one with no order share one markdown spelling', path) - } - if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) { - return failure('unsupported-node-shape', `no list marker spells the order ${JSON.stringify(start ?? null)}`, path) - } - if (start + items.length - 1 > largestListMarker) { - return failure('unspellable-list-marker', `no list marker spells the ${items.length} items a list starting at ${start} needs`, path) - } + const start = listStart(node, items.length, path) + if (!start.ok) return start const lines: string[] = [] + let fenceColons = 0 for (const [offset, item] of items.entries()) { const itemPath = [...path, 'content', offset] if (item.type !== 'listItem') return failure('unsupported-node-shape', `a ${node.type} holds listItem nodes only`, itemPath) - const emitted = emitListItem(item, ordered ? `${start + offset}. ` : '- ', itemPath, depth) + const emitted = emitListItem(item, ordered ? `${start.value + offset}. ` : '- ', itemPath, depth) if (!emitted.ok) return emitted - lines.push(emitted.value) + fenceColons = Math.max(fenceColons, emitted.value.fenceColons) + lines.push(emitted.value.text) } - return success(lines.join('\n')) + return success({ fenceColons, text: lines.join('\n') }) } -function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, depth: number): Result { +function listStart(node: AdfNode, items: number, path: ConvertErrorPath): Result { + if (items === 0) return failure('unsupported-node-shape', `a ${node.type} holds at least one listItem`, path) + if (node.type !== 'orderedList') return success(0) + const start = node.attrs?.['order'] + if (start === undefined || start === 1) { + return failure('ambiguous-ordered-list-start', 'an orderedList starting at 1 and one with no order share one markdown spelling', path) + } + if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) { + return failure('unsupported-node-shape', `no list marker spells the order ${JSON.stringify(start)}`, path) + } + if (start + items - 1 > largestListMarker) { + return failure('unspellable-list-marker', `no list marker spells the ${items} items a list starting at ${start} needs`, path) + } + return success(start) +} + +function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, depth: number): Result { const validation = validateBlockNode(item, [], path) if (!validation.ok) return validation const inner = emitBlocks(item.content ?? [], 'list-item', path, depth + 1) if (!inner.ok) return inner - if (inner.value.text === '') return success(marker.trimEnd()) + if (inner.value.text === '') return success({ fenceColons: 0, text: marker.trimEnd() }) const indent = ' '.repeat(marker.length) const lines = inner.value.text.split('\n').map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`)) const first = lines[0] ?? '' if (isThematicBreak(first)) { return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(first)}`, path) } - return success(lines.join('\n')) + return success({ fenceColons: inner.value.fenceColons, text: lines.join('\n') }) } -function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result { +function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result { const validation = validateBlockNode(node, [], path) if (!validation.ok) return validation const content = node.content ?? [] - if (content.length === 0) return success({ fenceColons: 2, text: '::paragraph' }) - return plainBlock(emitInlineLine(content, 'paragraph', path)) + if (content.length === 0) return success({ fenceColons: 2, spelling: 'directive', text: '::paragraph' }) + return commonMarkLine(emitInlineLine(content, 'paragraph', path)) } function emitRule(node: AdfNode, path: ConvertErrorPath): Result { diff --git a/src/block-directives.ts b/src/block-directives.ts index 5c88841..9a34d65 100644 --- a/src/block-directives.ts +++ b/src/block-directives.ts @@ -1,5 +1,4 @@ -import type { AdfMark, AdfNode } from './adf-document.ts' -import type { AttributeKind } from './directive-attributes.ts' +import type { AdfMark, AdfNode, AttributeKind } from './adf-document.ts' import type { JsonValue } from './json-value.ts' import { failure, success, type ConvertErrorPath, type Result } from './result.ts' import { isBareToken, spellAttributeValue, spellAttributes, spellJsonAttribute } from './directive-attributes.ts' diff --git a/src/corpus.test.ts b/src/corpus.test.ts index 79118fd..762408f 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -72,6 +72,50 @@ for (const directory of emittingDirectories) { } } +// The rule spec/flavour.md states as "a container's fence is longer than every directive fence line +// in its body", checked against the emitted bytes: a hand-written fixture cannot be its own witness. +function fenceNestingFault(markdown: string): string | undefined { + const open: number[] = [] + let codeFence: string | undefined + for (const line of markdown.split('\n')) { + const content = line.replace(/^ {0,3}(?:(?:> ?|[-*+] |\d{1,9}[.)] ) {0,3})*/, '') + const backticks = /^(`{3,}|~{3,})/.exec(content)?.[1] + if (codeFence !== undefined) { + if (backticks !== undefined && backticks[0] === codeFence[0] && backticks.length >= codeFence.length) codeFence = undefined + continue + } + if (backticks !== undefined) { + codeFence = backticks + continue + } + const colons = /^(:{2,})(.*)$/.exec(content) + if (colons === null) continue + const run = colons[1]?.length ?? 0 + const enclosing = open[open.length - 1] + if (colons[2] === '') { + open.pop() + continue + } + if (enclosing !== undefined && run >= enclosing) return `${JSON.stringify(line)} sits in a container fenced with ${enclosing} colons` + if (run > 2) open.push(run) + } + return undefined +} + +test('the fence nesting check catches a fence a container cannot hold', () => { + assert.equal(fenceNestingFault(':::panel info\n- :::panel warning\n B\n :::\n:::'), '"- :::panel warning" sits in a container fenced with 3 colons') + assert.equal(fenceNestingFault('::::panel info\n- :::panel warning\n B\n :::\n::::'), undefined) + assert.equal(fenceNestingFault(':::tableCell\n```text\n:::::::panel warning\n:::\n```\n:::'), undefined) +}) + +for (const directory of emittingDirectories) { + for (const name of fixtureNames(directory, '.md')) { + test(`${directory}/${name} fences every container longer than its body`, () => { + assert.equal(fenceNestingFault(readFileSync(join(roundTripRoot, directory, `${name}.md`), 'utf8')), undefined) + }) + } +} + test('unspellable pairs every .json with an .error', () => { assert.deepEqual(names(unspellableRoot, '.json'), names(unspellableRoot, '.error')) }) diff --git a/src/directive-attributes.ts b/src/directive-attributes.ts index 76f697b..6ef832c 100644 --- a/src/directive-attributes.ts +++ b/src/directive-attributes.ts @@ -1,8 +1,7 @@ +import type { AttributeKind } from './adf-document.ts' import type { JsonValue } from './json-value.ts' import { serializeCanonicalJson } from './canonical-json.ts' -export type AttributeKind = 'boolean' | 'json' | 'number' | 'string' - const bareToken = /^[A-Za-z0-9_-]+$/ export function isBareToken(text: string): boolean { diff --git a/src/markdown-image.ts b/src/markdown-image.ts new file mode 100644 index 0000000..5778ad8 --- /dev/null +++ b/src/markdown-image.ts @@ -0,0 +1,28 @@ +import type { AdfNode } from './adf-document.ts' +import { emitImageLine } from './markdown-inline.ts' +import { failure, type ConvertErrorPath, type Result } from './result.ts' +import { serializeCanonicalJson } from './canonical-json.ts' + +const centeredMediaSingle = '{"layout":"center"}' +const imageAttributes = ['alt', 'type', 'url'] + +export function emitImage(node: AdfNode, path: ConvertErrorPath): Result | undefined { + const image = imageShape(node) + if (image === undefined) return undefined + const mediaPath = [...path, 'content', 0] + if (image.alt === '') return failure('ambiguous-empty-media-alt', 'an empty media alt and an absent one share one image spelling', mediaPath) + return emitImageLine(image.alt, image.url, mediaPath) +} + +function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { + const content = node.content ?? [] + const media = content[0] + if (serializeCanonicalJson(node.attrs ?? {}, 'compact') !== centeredMediaSingle || (node.marks ?? []).length > 0) return undefined + if (media === undefined || content.length !== 1 || media.type !== 'media' || (media.marks ?? []).length > 0 || (media.content ?? []).length > 0) return undefined + const attrs = media.attrs ?? {} + const alt = attrs['alt'] + const url = attrs['url'] + if (Object.keys(attrs).some((key) => !imageAttributes.includes(key)) || attrs['type'] !== 'external') return undefined + if (typeof url !== 'string' || (alt !== undefined && typeof alt !== 'string')) return undefined + return { alt, url } +} diff --git a/src/markdown-tables.ts b/src/markdown-pipe-table.ts similarity index 100% rename from src/markdown-tables.ts rename to src/markdown-pipe-table.ts diff --git a/todo.md b/todo.md index bf79310..99d1f6e 100644 --- a/todo.md +++ b/todo.md @@ -100,11 +100,16 @@ detail is settled at its own milestone. assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin. `src/` gets its hierarchy at the same split — `adf/`, `markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is - still mechanical. + still mechanical. `block-directives.ts` is the one file that does not move whole: the node + table is ADF knowledge milestones 6-7 need too and belongs in `adf/`, `spellDirectiveHeader` + in `markdown/`. The table is a second copy of `spec/flavour.md`'s prose with no drift guard, + and a mistyped attribute name degrades into a false refusal no test catches. - [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and 3. Editor-normal (§2) gets its implementation here — `toEditorNormal(doc)` and the equality the round-trip asserts, which over normalized input is the canonical serializer's compact - spelling — rather than staying spelled inline as `?? []` at every reader. + spelling — rather than staying spelled inline as `?? []` at every reader. The reading half is + `nodeContent`/`nodeAttrs`/`nodeMarks` over the ~28 sites spelling it inline today, which also + lifts the branch floor §10 keeps below 100 for exactly those halves. Generators emit editor-normal ADF (§2). Real sanitized ADF from live Atlassian APIs lands here too (§10), in `corpus/real-payloads/`: an ADF→markdown→ADF check with no expected markdown, the payloads supplied by the maintainer. From 9ce5946173743aab723aefccf9095b75275b5eb1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 25 Aug 2026 15:38:16 +0200 Subject: [PATCH 3/6] Fold the per-node ambiguity codes into one, and sharpen the container fence rule --- AGENTS.md | 3 ++- corpus/unspellable/code-block-empty-language.error | 2 +- corpus/unspellable/media-empty-alt.error | 2 +- corpus/unspellable/ordered-list-start-one.error | 2 +- spec/flavour.md | 6 +++--- src/adf-to-markdown.test.ts | 8 ++++---- src/adf-to-markdown.ts | 4 ++-- src/markdown-image.ts | 2 +- src/result.ts | 4 +--- 9 files changed, 16 insertions(+), 17 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f741040..6677d43 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -75,7 +75,8 @@ Pre-1.0, normal 0.x rules. The error surface is a contract too. `ConvertError` is `{ code, message, path }` — the code from a closed list a consumer may switch exhaustively, the message free text, the path the node's position from the document root. Adding, removing or renaming a code is breaking, so a milestone meeting a -new failure cause reuses a code where one fits; the list is complete at `0.1.0`. +new failure cause reuses a code where one fits; the list is complete at `0.1.0`. A code names the +cause, never the node — `path` and `message` carry that. ## 9. Release automation diff --git a/corpus/unspellable/code-block-empty-language.error b/corpus/unspellable/code-block-empty-language.error index a53e1bc..01daa5e 100644 --- a/corpus/unspellable/code-block-empty-language.error +++ b/corpus/unspellable/code-block-empty-language.error @@ -1 +1 @@ -ambiguous-empty-code-block-language +ambiguous-attribute-spelling diff --git a/corpus/unspellable/media-empty-alt.error b/corpus/unspellable/media-empty-alt.error index a7d35ec..01daa5e 100644 --- a/corpus/unspellable/media-empty-alt.error +++ b/corpus/unspellable/media-empty-alt.error @@ -1 +1 @@ -ambiguous-empty-media-alt +ambiguous-attribute-spelling diff --git a/corpus/unspellable/ordered-list-start-one.error b/corpus/unspellable/ordered-list-start-one.error index e7b3b24..01daa5e 100644 --- a/corpus/unspellable/ordered-list-start-one.error +++ b/corpus/unspellable/ordered-list-start-one.error @@ -1 +1 @@ -ambiguous-ordered-list-start +ambiguous-attribute-spelling diff --git a/spec/flavour.md b/spec/flavour.md index 14be46f..7df17af 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -74,9 +74,9 @@ block content The fence is three or more colons. `arg` is one optional bare token whose meaning each node defines (e.g. the panel type). The body is block markdown. The closing fence is a line of at -least the opening's length, and a container's fence is longer than every directive fence line in -its body — counting only lines that parse as directive fences in the body's block structure; a -colon run inside a code fence or opaque carry is content. Canonical form uses minimal lengths. +least the opening's length, and a container's fence is longer than every directive fence line +anywhere in its body, however deeply a list item or blockquote nests it; a colon run inside a code +fence or opaque carry is content. Canonical form uses minimal lengths. Directive fence lines follow code-fence indentation (up to three spaces relative to their container); trailing whitespace on a fence line is tolerated in input, never emitted. diff --git a/src/adf-to-markdown.test.ts b/src/adf-to-markdown.test.ts index 28f3f71..74c4948 100644 --- a/src/adf-to-markdown.test.ts +++ b/src/adf-to-markdown.test.ts @@ -53,13 +53,13 @@ test('refuses marks on a block node', () => { test('refuses an ordered list whose markdown start is ambiguous', () => { const items: AdfNode[] = [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }] - assert.equal(code(adfToMarkdown(document({ content: items, type: 'orderedList' }))), 'ambiguous-ordered-list-start') - assert.equal(code(adfToMarkdown(document({ attrs: { order: 1 }, content: items, type: 'orderedList' }))), 'ambiguous-ordered-list-start') + assert.equal(code(adfToMarkdown(document({ content: items, type: 'orderedList' }))), 'ambiguous-attribute-spelling') + assert.equal(code(adfToMarkdown(document({ attrs: { order: 1 }, content: items, type: 'orderedList' }))), 'ambiguous-attribute-spelling') assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: items, type: 'orderedList' }))), '2. x\n') }) test('refuses the code block info strings the fence cannot hold', () => { - assert.equal(code(adfToMarkdown(document({ attrs: { language: '' }, type: 'codeBlock' }))), 'ambiguous-empty-code-block-language') + assert.equal(code(adfToMarkdown(document({ attrs: { language: '' }, type: 'codeBlock' }))), 'ambiguous-attribute-spelling') assert.equal(code(adfToMarkdown(document({ attrs: { language: 'adf' }, type: 'codeBlock' }))), 'reserved-adf-language') assert.equal(code(adfToMarkdown(document({ attrs: { language: 'a`b' }, type: 'codeBlock' }))), 'unspellable-code-block-language') assert.equal(code(adfToMarkdown(document({ attrs: { language: ' sql' }, type: 'codeBlock' }))), 'unspellable-code-block-language') @@ -334,7 +334,7 @@ test('spells the image form for exactly the centered external media shape', () = assert.equal(markdown(adfToMarkdown(single({ alt: 'The moon', type: 'external', url }))), `![The moon](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ type: 'external', url }))), `![](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`) - assert.equal(code(adfToMarkdown(single({ alt: '', type: 'external', url }))), 'ambiguous-empty-media-alt') + assert.equal(code(adfToMarkdown(single({ alt: '', type: 'external', url }))), 'ambiguous-attribute-spelling') assert.equal(code(adfToMarkdown(single({ alt: 'a\nb', type: 'external', url }))), 'unspellable-whitespace') assert.equal(code(adfToMarkdown(single({ alt: 'a\u0000b', type: 'external', url }))), 'unspellable-character') assert.equal(code(adfToMarkdown(single({ type: 'external', url: 'https://example.com/a b>c' }))), 'unspellable-link-destination') diff --git a/src/adf-to-markdown.ts b/src/adf-to-markdown.ts index 54c5b1a..5e23d8c 100644 --- a/src/adf-to-markdown.ts +++ b/src/adf-to-markdown.ts @@ -178,7 +178,7 @@ function spellCodeFenceInfo(language: JsonValue | undefined, path: ConvertErrorP if (language === undefined) return success('') if (typeof language !== 'string') return failure('unsupported-node-shape', 'a codeBlock language is no string', path) if (language === '') { - return failure('ambiguous-empty-code-block-language', 'an empty codeBlock language and an absent one share one markdown spelling', path) + return failure('ambiguous-attribute-spelling', 'an empty codeBlock language and an absent one share one markdown spelling', path) } if (language === 'adf') return failure('reserved-adf-language', 'the adf info string is reserved for the opaque carry', path) if (/[`\n\r]/.test(language) || language !== language.trim()) { @@ -227,7 +227,7 @@ function listStart(node: AdfNode, items: number, path: ConvertErrorPath): Result if (node.type !== 'orderedList') return success(0) const start = node.attrs?.['order'] if (start === undefined || start === 1) { - return failure('ambiguous-ordered-list-start', 'an orderedList starting at 1 and one with no order share one markdown spelling', path) + return failure('ambiguous-attribute-spelling', 'an orderedList starting at 1 and one with no order share one markdown spelling', path) } if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) { return failure('unsupported-node-shape', `no list marker spells the order ${JSON.stringify(start)}`, path) diff --git a/src/markdown-image.ts b/src/markdown-image.ts index 5778ad8..f7d8317 100644 --- a/src/markdown-image.ts +++ b/src/markdown-image.ts @@ -10,7 +10,7 @@ export function emitImage(node: AdfNode, path: ConvertErrorPath): Result const image = imageShape(node) if (image === undefined) return undefined const mediaPath = [...path, 'content', 0] - if (image.alt === '') return failure('ambiguous-empty-media-alt', 'an empty media alt and an absent one share one image spelling', mediaPath) + if (image.alt === '') return failure('ambiguous-attribute-spelling', 'an empty media alt and an absent one share one image spelling', mediaPath) return emitImageLine(image.alt, image.url, mediaPath) } diff --git a/src/result.ts b/src/result.ts index f2edebd..00a1a9e 100644 --- a/src/result.ts +++ b/src/result.ts @@ -1,7 +1,5 @@ export type ConvertErrorCode = - | 'ambiguous-empty-code-block-language' - | 'ambiguous-empty-media-alt' - | 'ambiguous-ordered-list-start' + | 'ambiguous-attribute-spelling' | 'not-an-adf-document' | 'reserved-adf-language' | 'unspellable-adjacent-lists' From 5be1bf677a48e8cdbe4d8d54f11c805b44266bb6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 25 Aug 2026 15:43:30 +0200 Subject: [PATCH 4/6] Narrow the error-code rule to what it forbids, and name the fence rule instead of quoting it --- AGENTS.md | 3 ++- src/corpus.test.ts | 3 +-- todo.md | 6 +++++- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6677d43..aca7e6a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -76,7 +76,8 @@ The error surface is a contract too. `ConvertError` is `{ code, message, path }` closed list a consumer may switch exhaustively, the message free text, the path the node's position from the document root. Adding, removing or renaming a code is breaking, so a milestone meeting a new failure cause reuses a code where one fits; the list is complete at `0.1.0`. A code names the -cause, never the node — `path` and `message` carry that. +cause; where one cause recurs across node types, one code covers them all and `path` and `message` +say which. ## 9. Release automation diff --git a/src/corpus.test.ts b/src/corpus.test.ts index 762408f..3084dd1 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -72,8 +72,7 @@ for (const directory of emittingDirectories) { } } -// The rule spec/flavour.md states as "a container's fence is longer than every directive fence line -// in its body", checked against the emitted bytes: a hand-written fixture cannot be its own witness. +// spec/flavour.md, Directives: the container fence rule, checked against the emitted bytes. function fenceNestingFault(markdown: string): string | undefined { const open: number[] = [] let codeFence: string | undefined diff --git a/todo.md b/todo.md index 99d1f6e..e0c511d 100644 --- a/todo.md +++ b/todo.md @@ -82,6 +82,8 @@ detail is settled at its own milestone. attribute canonicalization, a pipe cell's whitespace edges and `\u007c` for a `|` inside a quoted attribute value, documents combining nodes rather than isolating one, and a paragraph line inside a container body shaped like a closing fence (`:::`, `::: x`). + Guard `fenceNestingFault`'s bare-run pop here too — a run shorter than the open fence is a + fault, not a close — which today's emitter cannot reach. The gate gains the collision property here: no two corpus documents may emit the same bytes — one spelling for two documents is a round-trip break no parser can undo, and it is provable without one. It also settles the emitter's one known approximation: delimiter @@ -114,7 +116,9 @@ detail is settled at its own milestone. here too (§10), in `corpus/real-payloads/`: an ADF→markdown→ADF check with no expected markdown, the payloads supplied by the maintainer. - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, - the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown + the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: every + `corpus/unspellable/` document is one of 1d's decisions, so the directory empties as they land + and whatever survives is permanent. `0.1.0` is the markdown round-trip: both markdown directions, the types, `isAdfDocument`. The build lands here: a build tsconfig emitting JS and `.d.ts` to `dist/` (the dev config's `allowImportingTsExtensions` forces `noEmit`, so the build config needs `rewriteRelativeImportExtensions`), plus `exports`/`files` in From 0121a5f06edf666bb57cceb124dc4df87d2cf31a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 25 Aug 2026 16:12:56 +0200 Subject: [PATCH 5/6] Refuse the pipe form where a link hides a pipe, and fall back to the directive form for every image CommonMark cannot spell --- .../block-nodes/container-crossing.json | 41 +++++++++++++++++-- .../block-nodes/container-crossing.md | 8 +++- .../block-nodes/image-boundary.json | 31 ++++++++++++++ .../round-trip/block-nodes/image-boundary.md | 8 ++++ corpus/unspellable/media-empty-alt.error | 1 - corpus/unspellable/media-empty-alt.json | 22 ---------- spec/flavour.md | 17 ++++---- src/adf-to-markdown.test.ts | 12 ++++-- src/adf-to-markdown.ts | 2 +- src/corpus.test.ts | 6 ++- src/markdown-image.ts | 11 +++-- src/markdown-inline.ts | 4 +- src/markdown-pipe-table.ts | 17 +++++++- todo.md | 19 +++++---- 14 files changed, 140 insertions(+), 59 deletions(-) delete mode 100644 corpus/unspellable/media-empty-alt.error delete mode 100644 corpus/unspellable/media-empty-alt.json diff --git a/corpus/round-trip/block-nodes/container-crossing.json b/corpus/round-trip/block-nodes/container-crossing.json index 95bd7b3..32c9bb1 100644 --- a/corpus/round-trip/block-nodes/container-crossing.json +++ b/corpus/round-trip/block-nodes/container-crossing.json @@ -34,11 +34,29 @@ { "content": [ { - "text": "Plain item.", - "type": "text" + "content": [ + { + "attrs": { + "title": "Deeper still" + }, + "content": [ + { + "content": [ + { + "text": "Two list levels down.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "expand" + } + ], + "type": "listItem" } ], - "type": "paragraph" + "type": "bulletList" } ], "type": "listItem" @@ -64,6 +82,23 @@ } ], "type": "expand" + }, + { + "attrs": { + "panelType": "note" + }, + "content": [ + { + "content": [ + { + "text": "Superseded by the next run.", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "panel" } ], "type": "blockquote" diff --git a/corpus/round-trip/block-nodes/container-crossing.md b/corpus/round-trip/block-nodes/container-crossing.md index 8af9334..c880706 100644 --- a/corpus/round-trip/block-nodes/container-crossing.md +++ b/corpus/round-trip/block-nodes/container-crossing.md @@ -2,9 +2,15 @@ - :::panel warning Check the torque before signing off. ::: -- Plain item. +- - :::expand {title="Deeper still"} + Two list levels down. + ::: > :::expand {title="Full build log"} > The build ran for 11 minutes. > ::: +> +> :::panel note +> Superseded by the next run. +> ::: :::: diff --git a/corpus/round-trip/block-nodes/image-boundary.json b/corpus/round-trip/block-nodes/image-boundary.json index 418c450..62881e0 100644 --- a/corpus/round-trip/block-nodes/image-boundary.json +++ b/corpus/round-trip/block-nodes/image-boundary.json @@ -82,6 +82,37 @@ } ], "type": "mediaSingle" + }, + { + "attrs": { + "layout": "center" + }, + "content": [ + { + "attrs": { + "alt": "", + "type": "external", + "url": "https://example.com/moon.png" + }, + "type": "media" + } + ], + "type": "mediaSingle" + }, + { + "attrs": { + "layout": "center" + }, + "content": [ + { + "attrs": { + "type": "external", + "url": "https://example.com/plan.png?a=1&b=2" + }, + "type": "media" + } + ], + "type": "mediaSingle" } ], "type": "doc", diff --git a/corpus/round-trip/block-nodes/image-boundary.md b/corpus/round-trip/block-nodes/image-boundary.md index e8f58f9..0672446 100644 --- a/corpus/round-trip/block-nodes/image-boundary.md +++ b/corpus/round-trip/block-nodes/image-boundary.md @@ -16,3 +16,11 @@ Taken from Kiruna. :::mediaSingle {layout=center} ::media {alt="The assembly drawing" marks="[{\"attrs\":{\"href\":\"https://example.com/assembly\"},\"type\":\"link\"}]" type=external url="https://example.com/assembly.png"} ::: + +:::mediaSingle {layout=center} +::media {alt="" type=external url="https://example.com/moon.png"} +::: + +:::mediaSingle {layout=center} +::media {type=external url="https://example.com/plan.png?a=1&b=2"} +::: diff --git a/corpus/unspellable/media-empty-alt.error b/corpus/unspellable/media-empty-alt.error deleted file mode 100644 index 01daa5e..0000000 --- a/corpus/unspellable/media-empty-alt.error +++ /dev/null @@ -1 +0,0 @@ -ambiguous-attribute-spelling diff --git a/corpus/unspellable/media-empty-alt.json b/corpus/unspellable/media-empty-alt.json deleted file mode 100644 index eaa9b0b..0000000 --- a/corpus/unspellable/media-empty-alt.json +++ /dev/null @@ -1,22 +0,0 @@ -{ - "content": [ - { - "attrs": { - "layout": "center" - }, - "content": [ - { - "attrs": { - "alt": "", - "type": "external", - "url": "https://example.com/moon.png" - }, - "type": "media" - } - ], - "type": "mediaSingle" - } - ], - "type": "doc", - "version": 1 -} diff --git a/spec/flavour.md b/spec/flavour.md index 7df17af..f481966 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -42,8 +42,8 @@ normalizes to it through the round-trip. the leading delimiter of a construct that would otherwise open, re-scan from there, and repeat — with the opener literal the closer parses as text, so `*not emphasis*` is `\*not emphasis*`, one backslash. -- Blocks separated by one blank line at document level and between CommonMark blocks; two - directive blocks inside a container take none. No trailing whitespace outside a code block's +- Blocks separated by one blank line at document level, inside a blockquote and between CommonMark + blocks; two directive blocks inside a container take none. No trailing whitespace outside a code block's content, single trailing newline; a document with no blocks is the empty string. ## Directives @@ -90,7 +90,8 @@ container is open, a named error otherwise. **Attributes**: `{key=value key2="two words"}`. `{attrs}` is optional in every form, and `{}` is valid — no attributes. A bare value matches `[A-Za-z0-9_-]+`; any other value is double-quoted with JSON string escaping (`\"` `\\` `\n` `\t` `\uXXXX`, …) — total over -Unicode, and raw newlines never appear inside quotes. All values are strings at the grammar +Unicode, and raw newlines never appear inside quotes. The closing `}` is the first one outside +quotes, since a quoted value holds `}` unescaped. All values are strings at the grammar level; each node's section assigns types. Canonical form orders keys alphabetically, spells values bare wherever allowed, inside quotes escapes only what it must using the shortest escape form, and omits empty `{attrs}` except where the `{` itself claims the directive @@ -200,7 +201,9 @@ The moon, at night. a `mediaSingle` with attrs exactly `{"layout":"center"}` holding an `external` `media` — `url` from the destination, `alt` the description's plain-text content when non-empty. `adfToMarkdown` emits the image form for exactly that shape — those attrs and no others, no marks on either -node, no caption, and a `media` carrying nothing beyond `alt`, `type` and `url`. An image amid +node, no caption, and a `media` carrying nothing beyond `alt`, `type` and `url` — and only where +CommonMark spells the pair: a destination or a description the image form cannot hold, an empty +`alt` included, takes the directive form instead. An image amid other text, or one carrying a title, is a named error: `mediaInline` carries a media `collection` + `id`, never a URL, and no media node carries a title. @@ -210,9 +213,9 @@ One header row plus plain inline cells is a pipe table; anything richer is the d (AGENTS.md §4). Precisely: a table emits as a pipe table exactly when the `table`, every row and every cell carry no attrs and no marks, the first row is all `tableHeader` and the rest all `tableCell`, every row has the header's cell count, and every cell holds exactly one attr-less, -mark-less paragraph — an empty cell holds one empty paragraph — with no `|` in a code span: -backslash escapes are inert there, so pipe form cannot spell that pipe and the table takes the -directive form. A pipe table parses back to exactly that shape. +mark-less paragraph — an empty cell holds one empty paragraph — with no `|` in a code span, +link destination or link title: backslash escapes are inert in everything the inline layer spells +as syntax, so pipe form cannot spell that pipe and the table takes the directive form. A pipe table parses back to exactly that shape. ``` | Part | Qty | diff --git a/src/adf-to-markdown.test.ts b/src/adf-to-markdown.test.ts index 74c4948..fa6c96c 100644 --- a/src/adf-to-markdown.test.ts +++ b/src/adf-to-markdown.test.ts @@ -334,10 +334,10 @@ test('spells the image form for exactly the centered external media shape', () = assert.equal(markdown(adfToMarkdown(single({ alt: 'The moon', type: 'external', url }))), `![The moon](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ type: 'external', url }))), `![](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`) - assert.equal(code(adfToMarkdown(single({ alt: '', type: 'external', url }))), 'ambiguous-attribute-spelling') - assert.equal(code(adfToMarkdown(single({ alt: 'a\nb', type: 'external', url }))), 'unspellable-whitespace') - assert.equal(code(adfToMarkdown(single({ alt: 'a\u0000b', type: 'external', url }))), 'unspellable-character') - assert.equal(code(adfToMarkdown(single({ type: 'external', url: 'https://example.com/a b>c' }))), 'unspellable-link-destination') + const fallback = (attrs: AdfAttributes): boolean => markdown(adfToMarkdown(single(attrs))).startsWith(':::mediaSingle {layout=center}') + assert.ok(fallback({ alt: '', type: 'external', url })) + assert.ok(fallback({ alt: 'a\nb', type: 'external', url }) && fallback({ alt: 'a\u0000b', type: 'external', url })) + assert.ok(fallback({ type: 'external', url: 'https://example.com/a b>c' }) && fallback({ alt: ' moon ', type: 'external', url })) assert.equal(code(adfToMarkdown(single({ alt: 4, type: 'external', url }))), 'unsupported-node-shape') assert.equal(code(adfToMarkdown(single({ type: 'external', url: 4 }))), 'unsupported-node-shape') assert.equal(code(adfToMarkdown(single({ type: 'external', url }, paragraph()))), 'unsupported-node-shape') @@ -372,4 +372,8 @@ test('spells a table as a pipe table only where every row and cell is plain', () assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { attrs: { localId: 'a' }, type: 'paragraph' }))))), 'unspelled-node-attribute') assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' })))))) assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace') + const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' }))) + assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/?x|y' }, type: 'link' })))) + assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/', title: 'a|b' }, type: 'link' })))) + assert.equal(markdown(adfToMarkdown(marked({ attrs: { href: 'https://example.com/x' }, type: 'link' }))), '| [l](https://example.com/x) |\n| --- |\n') }) diff --git a/src/adf-to-markdown.ts b/src/adf-to-markdown.ts index 5e23d8c..50a2b80 100644 --- a/src/adf-to-markdown.ts +++ b/src/adf-to-markdown.ts @@ -67,7 +67,7 @@ function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: if (previous.spelling === 'directive' && next.spelling === 'directive') return success('\n') return failure( 'unspelled-block-separation', - `the canonical form leaves the separation between a ${previous.node.type} and a ${next.node.type} in a container body unspelled`, + `the canonical form leaves the separation between a ${previous.spelling} and a ${next.spelling} block in a container body unspelled`, next.path, ) } diff --git a/src/corpus.test.ts b/src/corpus.test.ts index 3084dd1..c381035 100644 --- a/src/corpus.test.ts +++ b/src/corpus.test.ts @@ -77,7 +77,7 @@ function fenceNestingFault(markdown: string): string | undefined { const open: number[] = [] let codeFence: string | undefined for (const line of markdown.split('\n')) { - const content = line.replace(/^ {0,3}(?:(?:> ?|[-*+] |\d{1,9}[.)] ) {0,3})*/, '') + const content = line.replace(/^[ \t]*(?:(?:> ?|[-*+] |\d{1,9}[.)] )[ \t]*)*/, '') const backticks = /^(`{3,}|~{3,})/.exec(content)?.[1] if (codeFence !== undefined) { if (backticks !== undefined && backticks[0] === codeFence[0] && backticks.length >= codeFence.length) codeFence = undefined @@ -103,7 +103,9 @@ function fenceNestingFault(markdown: string): string | undefined { test('the fence nesting check catches a fence a container cannot hold', () => { assert.equal(fenceNestingFault(':::panel info\n- :::panel warning\n B\n :::\n:::'), '"- :::panel warning" sits in a container fenced with 3 colons') - assert.equal(fenceNestingFault('::::panel info\n- :::panel warning\n B\n :::\n::::'), undefined) + assert.equal(fenceNestingFault(':::panel info\n- - :::panel warning\n B\n :::\n:::'), '"- - :::panel warning" sits in a container fenced with 3 colons') + assert.equal(fenceNestingFault(':::panel info\n10. :::panel warning\n B\n :::\n:::'), '"10. :::panel warning" sits in a container fenced with 3 colons') + assert.equal(fenceNestingFault('::::panel info\n- - :::panel warning\n B\n :::\n::::'), undefined) assert.equal(fenceNestingFault(':::tableCell\n```text\n:::::::panel warning\n:::\n```\n:::'), undefined) }) diff --git a/src/markdown-image.ts b/src/markdown-image.ts index f7d8317..fce8bd9 100644 --- a/src/markdown-image.ts +++ b/src/markdown-image.ts @@ -1,6 +1,6 @@ import type { AdfNode } from './adf-document.ts' +import type { ConvertErrorPath, Result } from './result.ts' import { emitImageLine } from './markdown-inline.ts' -import { failure, type ConvertErrorPath, type Result } from './result.ts' import { serializeCanonicalJson } from './canonical-json.ts' const centeredMediaSingle = '{"layout":"center"}' @@ -9,9 +9,8 @@ const imageAttributes = ['alt', 'type', 'url'] export function emitImage(node: AdfNode, path: ConvertErrorPath): Result | undefined { const image = imageShape(node) if (image === undefined) return undefined - const mediaPath = [...path, 'content', 0] - if (image.alt === '') return failure('ambiguous-attribute-spelling', 'an empty media alt and an absent one share one image spelling', mediaPath) - return emitImageLine(image.alt, image.url, mediaPath) + const line = emitImageLine(image.alt, image.url, [...path, 'content', 0]) + return line.ok ? line : undefined } function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { @@ -22,7 +21,7 @@ function imageShape(node: AdfNode): { alt: string | undefined; url: string } | u const attrs = media.attrs ?? {} const alt = attrs['alt'] const url = attrs['url'] - if (Object.keys(attrs).some((key) => !imageAttributes.includes(key)) || attrs['type'] !== 'external') return undefined - if (typeof url !== 'string' || (alt !== undefined && typeof alt !== 'string')) return undefined + if (Object.keys(attrs).some((key) => !imageAttributes.includes(key)) || attrs['type'] !== 'external' || typeof url !== 'string') return undefined + if (alt !== undefined && (typeof alt !== 'string' || alt === '')) return undefined return { alt, url } } diff --git a/src/markdown-inline.ts b/src/markdown-inline.ts index 54f0332..3f0fcc4 100644 --- a/src/markdown-inline.ts +++ b/src/markdown-inline.ts @@ -24,7 +24,9 @@ export function emitInlineLine(nodes: readonly AdfNode[], container: LineContain } export function emitImageLine(alt: string | undefined, href: string, path: ConvertErrorPath): Result { - if (alt !== undefined && /[\n\r]/.test(alt)) return failure('unspellable-whitespace', 'a media alt holds a newline no image description spells', path) + if (alt !== undefined && /^[ \t]|[ \t]$|[\n\r]/.test(alt)) { + return failure('unspellable-whitespace', 'a media alt holds whitespace no image description spells', path) + } if (alt !== undefined && holdsNullCharacter(alt)) return failure('unspellable-character', 'a media alt holds a null character CommonMark replaces', path) const destination = spellDestination(href, path) if (!destination.ok) return destination diff --git a/src/markdown-pipe-table.ts b/src/markdown-pipe-table.ts index e4d6dc3..d606281 100644 --- a/src/markdown-pipe-table.ts +++ b/src/markdown-pipe-table.ts @@ -1,4 +1,5 @@ import type { AdfNode } from './adf-document.ts' +import type { JsonValue } from './json-value.ts' import { emitInlineLine } from './markdown-inline.ts' import { success, type ConvertErrorPath, type Result } from './result.ts' @@ -48,6 +49,18 @@ function plainParagraph(cell: AdfNode): AdfNode | undefined { const content = cell.content ?? [] const paragraph = content[0] if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !isPlain(paragraph)) return undefined - const pipedCode = (paragraph.content ?? []).some((child) => (child.marks ?? []).some((mark) => mark.type === 'code') && (child.text ?? '').includes('|')) - return pipedCode ? undefined : paragraph + return (paragraph.content ?? []).some(spellsPipeAsSyntax) ? undefined : paragraph +} + +// A pipe the inline layer emits as syntax takes no backslash, so the cell has no pipe spelling. +function spellsPipeAsSyntax(child: AdfNode): boolean { + return (child.marks ?? []).some((mark) => { + if (mark.type === 'code') return (child.text ?? '').includes('|') + if (mark.type !== 'link') return false + return holdsPipe(mark.attrs?.['href']) || holdsPipe(mark.attrs?.['title']) + }) +} + +function holdsPipe(value: JsonValue | undefined): boolean { + return typeof value === 'string' && value.includes('|') } diff --git a/todo.md b/todo.md index e0c511d..b497c82 100644 --- a/todo.md +++ b/todo.md @@ -28,15 +28,14 @@ detail is settled at its own milestone. carries a `localId` with no spelling, `codeBlock` also `hideLineNumbers`, `uniqueId` and `wrap`, `blockquote` also marks, and `hardBreak` `text` and `localId` with no section for the carry fallback to reach. Picking one (directive sections for those nodes, or the opaque - carry) is a permanent format decision (§8). Three collision sites wait in + carry) is a permanent format decision (§8). Two collision sites wait in `corpus/unspellable/` meanwhile, each a choice between the absent attribute and the empty - value: a `codeBlock` whose info string is empty and `media` with an empty `alt`, which one - "exactly that shape" rule — as the CommonMark image already uses — could settle together, - and an `orderedList` starting at 1, independent of the totality answer since `order: 9` - keeps the markdown form either way. **Also blocked**: the link rule covers destination spaces only, so two shapes - break §2 silently — href `https://example.com/a)b` emits `[t](https://example.com/a)b)`, - read back as href `…/a` plus literal `b)`; title `He said "hi"` emits - `[t](u "He said "hi"")`, which holds no title. Two defensible spellings each — angle + value: a `codeBlock` whose info string is empty, and an `orderedList` starting at 1, + independent of the totality answer since `order: 9` keeps the markdown form either way. + Neither has a second spelling to fall back to, which is what settled the third — a `media` + with an empty `alt` takes the directive form (`spec/flavour.md`, the CommonMark image). **Also blocked**: the link rule covers destination spaces only, so two shapes + have no spelling and are refused meanwhile — href `https://example.com/a)b` and title + `He said "hi"`, both in `corpus/unspellable/`. Two defensible spellings each — angle brackets or a backslash escape, and for titles `'…'` or `(…)` besides — so §8 leaves the pick here. **Also blocked**: block separation is unstated for a CommonMark block beside a directive block in a container body — an `expand` whose content is `paragraph` "A" then a @@ -71,7 +70,9 @@ detail is settled at its own milestone. a `.json` beside the `ConvertErrorCode` it must return, the emitter half of `corpus/errors/`. - [ ] **2c — Inline nodes and marks.** `inline-nodes/` green. `InlineSegment.kind` splits into its two axes here — escapability (`attribute` for `:text{text="…"}`, `backslash`, `none`) - and the emphasis role — rather than gaining a third value that means one of each. + and the emphasis role — rather than gaining a third value that means one of each. A lone + surrogate in a text node emits verbatim and becomes U+FFFD on any UTF-8 encode, a §2 break + plain text still holds open — attribute values already escape it. - [ ] **2d — The opaque carry** (§3). Fixtures and emitter together, into `corpus/round-trip/opaque-carry/`: an unknown node in both positions, the reserved `adf` info string, and the `codeBlock` whose language is `adf`. From 536672219d7266803530f76b73081ea2c5ace221 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Tue, 25 Aug 2026 16:19:58 +0200 Subject: [PATCH 6/6] Bind the pipe guard and the attribute vocabulary to the milestones that inherit them --- spec/flavour.md | 6 ++++-- src/adf-to-markdown.test.ts | 6 ++++-- todo.md | 14 ++++++++++---- 3 files changed, 18 insertions(+), 8 deletions(-) diff --git a/spec/flavour.md b/spec/flavour.md index f481966..906addd 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -43,7 +43,8 @@ normalizes to it through the round-trip. repeat — with the opener literal the closer parses as text, so `*not emphasis*` is `\*not emphasis*`, one backslash. - Blocks separated by one blank line at document level, inside a blockquote and between CommonMark - blocks; two directive blocks inside a container take none. No trailing whitespace outside a code block's + blocks; two directive blocks inside a container take none. No trailing whitespace outside a code + block's content, single trailing newline; a document with no blocks is the empty string. ## Directives @@ -215,7 +216,8 @@ and every cell carry no attrs and no marks, the first row is all `tableHeader` a `tableCell`, every row has the header's cell count, and every cell holds exactly one attr-less, mark-less paragraph — an empty cell holds one empty paragraph — with no `|` in a code span, link destination or link title: backslash escapes are inert in everything the inline layer spells -as syntax, so pipe form cannot spell that pipe and the table takes the directive form. A pipe table parses back to exactly that shape. +as syntax, so pipe form cannot spell that pipe and the table takes the directive form. A pipe table +parses back to exactly that shape. ``` | Part | Qty | diff --git a/src/adf-to-markdown.test.ts b/src/adf-to-markdown.test.ts index fa6c96c..ac6aee1 100644 --- a/src/adf-to-markdown.test.ts +++ b/src/adf-to-markdown.test.ts @@ -336,8 +336,10 @@ test('spells the image form for exactly the centered external media shape', () = assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`) const fallback = (attrs: AdfAttributes): boolean => markdown(adfToMarkdown(single(attrs))).startsWith(':::mediaSingle {layout=center}') assert.ok(fallback({ alt: '', type: 'external', url })) - assert.ok(fallback({ alt: 'a\nb', type: 'external', url }) && fallback({ alt: 'a\u0000b', type: 'external', url })) - assert.ok(fallback({ type: 'external', url: 'https://example.com/a b>c' }) && fallback({ alt: ' moon ', type: 'external', url })) + assert.ok(fallback({ alt: 'a\nb', type: 'external', url })) + assert.ok(fallback({ alt: 'a\u0000b', type: 'external', url })) + assert.ok(fallback({ type: 'external', url: 'https://example.com/a b>c' })) + assert.ok(fallback({ alt: ' moon ', type: 'external', url })) assert.equal(code(adfToMarkdown(single({ alt: 4, type: 'external', url }))), 'unsupported-node-shape') assert.equal(code(adfToMarkdown(single({ type: 'external', url: 4 }))), 'unsupported-node-shape') assert.equal(code(adfToMarkdown(single({ type: 'external', url }, paragraph()))), 'unsupported-node-shape') diff --git a/todo.md b/todo.md index b497c82..6562ae8 100644 --- a/todo.md +++ b/todo.md @@ -33,7 +33,8 @@ detail is settled at its own milestone. value: a `codeBlock` whose info string is empty, and an `orderedList` starting at 1, independent of the totality answer since `order: 9` keeps the markdown form either way. Neither has a second spelling to fall back to, which is what settled the third — a `media` - with an empty `alt` takes the directive form (`spec/flavour.md`, the CommonMark image). **Also blocked**: the link rule covers destination spaces only, so two shapes + with an empty `alt` takes the directive form (`spec/flavour.md`, the CommonMark image). + **Also blocked**: the link rule covers destination spaces only, so two shapes have no spelling and are refused meanwhile — href `https://example.com/a)b` and title `He said "hi"`, both in `corpus/unspellable/`. Two defensible spellings each — angle brackets or a backslash escape, and for titles `'…'` or `(…)` besides — so §8 leaves @@ -42,7 +43,9 @@ detail is settled at its own milestone. `panel` (`panelType` `warning`) holding "B" spells `A` and `:::panel warning` either on consecutive lines or with a blank line between. Two defensible spellings, so §8 leaves the pick here; `unspelled-block-separation` refuses the pair meanwhile, an empty paragraph's - `::paragraph` beside a CommonMark block included. + `::paragraph` beside a CommonMark block included — and, since a `mediaSingle`'s spelling now + follows whether CommonMark can spell its URL, two sibling images differing only by an + `&` land in the same refusal. - [x] **1d1 — The CommonMark subset**: blockquote, bulletList, codeBlock, heading, orderedList, paragraph, rule, listItem, hardBreak, text, code spans, and the `code`, `em`, `link`, `strike` and `strong` marks — one mark per text node; nesting is 1d3's. @@ -72,7 +75,9 @@ detail is settled at its own milestone. its two axes here — escapability (`attribute` for `:text{text="…"}`, `backslash`, `none`) and the emphasis role — rather than gaining a third value that means one of each. A lone surrogate in a text node emits verbatim and becomes U+FFFD on any UTF-8 encode, a §2 break - plain text still holds open — attribute values already escape it. + plain text still holds open — attribute values already escape it. The inline directives + arriving here emit their attribute values as syntax, so `spellsPipeAsSyntax` grows with + them alongside the `|` escaping `spec/flavour.md` already mandates in a pipe cell. - [ ] **2d — The opaque carry** (§3). Fixtures and emitter together, into `corpus/round-trip/opaque-carry/`: an unknown node in both positions, the reserved `adf` info string, and the `codeBlock` whose language is `adf`. @@ -105,7 +110,8 @@ detail is settled at its own milestone. `markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is still mechanical. `block-directives.ts` is the one file that does not move whole: the node table is ADF knowledge milestones 6-7 need too and belongs in `adf/`, `spellDirectiveHeader` - in `markdown/`. The table is a second copy of `spec/flavour.md`'s prose with no drift guard, + in `markdown/`. `AttributeKind` stays above both — it is the vocabulary a string-typed + attribute grammar needs, which is why HTML will want it too, not a markdown spelling. The table is a second copy of `spec/flavour.md`'s prose with no drift guard, and a mistyped attribute name degrades into a false refusal no test catches. - [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and 3. Editor-normal (§2) gets its implementation here — `toEditorNormal(doc)` and the equality