diff --git a/AGENTS.md b/AGENTS.md index ab0d639..f741040 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,6 +72,11 @@ The emitted markdown and HTML are contracts. After 1.0: previously-emitted outpu differently, or not at all, is MAJOR; new syntax while old output still round-trips is MINOR. Pre-1.0, normal 0.x rules. +The error surface is a contract too. `ConvertError` is `{ code, message, path }` — the code from a +closed list a consumer may switch exhaustively, the message free text, the path the node's position +from the document root. Adding, removing or renaming a code is breaking, so a milestone meeting a +new failure cause reuses a code where one fits; the list is complete at `0.1.0`. + ## 9. Release automation - `package.json` version on `main` is the source of truth. CI on `main`: tests green and version @@ -87,6 +92,11 @@ Test for the behaviour wanted first, then implement until green. `node --test`, Node, tsc and npm never run on the host — only via the pinned images (§9). Tests are independent, coverage does not decline, containers are torn down after a run. +The floors live in the `test` script, so `npm test` and the gate are one path: 100% of lines and +functions, and a branch floor that only ever moves upward. It sits below 100 because the guards +`noUncheckedIndexedAccess` and ADF's optional keys force — `?? []`, `?? {}`, `?.`, an index +compared against `undefined` — have a half no valid document reaches. + The corpus, all checked in: hand-built fixtures per node and combination; real sanitized ADF from live Atlassian APIs; property-generated ADF trees; the CommonMark spec suite against `markdownToAdf` and `markdownToHtml`. @@ -99,6 +109,9 @@ live Atlassian APIs; property-generated ADF trees; the CommonMark spec suite aga `Result` — `{ ok: true; value } | { ok: false; error: ConvertError }` — nothing throws. `try/catch` only wrapped tightly around a call that genuinely throws, converted to a result on the spot. +- Nothing recurses unbounded: the guards walk iteratively, and blocks, marks and attribute values + are all held to 500 levels, so a deep document is a `Result` rather than the stack overflow that + waits near 2000. - No casts: `as`, `as unknown as`, non-null `!`. A boundary owes a type guard validating the fields it claims (`isAdfDocument`); past it everything is typed. Make invalid states unrepresentable. @@ -126,7 +139,8 @@ One-line commit messages and PR titles; short PR summaries. No AI-attribution ma ## 14. Non-goals No wiki markup (§1), no network or filesystem I/O, no name→id resolution (§3), no ADF schema -validation or exported validator, no shipped CSS (§4), no streaming APIs, no performance budget — +validation or exported validator — a refusal that keeps the round-trip is not schema validation, +so the one a node carrying the same mark type twice earns stays, no shipped CSS (§4), no streaming APIs, no performance budget — conversions are O(n), real documents are kilobytes. A CLI is a later goal (`todo.md`), not a non-goal. diff --git a/README.md b/README.md index a70ce34..062cbb0 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,9 @@ Lossless conversion between **Atlassian Document Format** (ADF), an extended markdown flavour, and an HTML dialect. -**Status: scaffold only, no conversion code yet.** Plan: `todo.md`. Decisions: `AGENTS.md`. The -flavour's grammar: [`spec/flavour.md`](spec/flavour.md). +**Status: pre-release — `adfToMarkdown` emits the CommonMark subset, nothing else is built.** +Plan: `todo.md`. Decisions: `AGENTS.md`. The flavour's grammar: +[`spec/flavour.md`](spec/flavour.md). ## What it is for @@ -31,6 +32,8 @@ isAdfDocument(v: unknown): v is AdfDocument ``` `Result` is `{ ok: true; value: T } | { ok: false; error: ConvertError }` — nothing throws. +`ConvertError` is `{ code, message, path }`: a code from a closed set, and the path of the node it +names, from the document root. ## The guarantees @@ -45,6 +48,7 @@ isAdfDocument(v: unknown): v is AdfDocument spelling, which round-trips byte-identically. - Foreign HTML maps a documented element set; an unmappable element is an error, never a silent drop. Well-formed HTML only — no tag-soup recovery. +- A document nested deeper than 500 levels is an error result, not a stack overflow. - The emitted formats are semver surface (AGENTS.md §8). ## Who it is for diff --git a/package.json b/package.json index b0e722a..93bc33d 100644 --- a/package.json +++ b/package.json @@ -13,7 +13,7 @@ "node": ">=24" }, "scripts": { - "test": "node --test \"src/**/*.test.ts\"", + "test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=93 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"", "typecheck": "tsc --noEmit" }, "devDependencies": { diff --git a/src/adf-document.test.ts b/src/adf-document.test.ts new file mode 100644 index 0000000..1c5eaba --- /dev/null +++ b/src/adf-document.test.ts @@ -0,0 +1,34 @@ +import assert from 'node:assert/strict' +import test from 'node:test' + +import { isAdfDocument } from './adf-document.ts' + +test('accepts an editor-normal document', () => { + assert.equal(isAdfDocument({ content: [{ content: [{ text: 'x', type: 'text' }], type: 'paragraph' }], type: 'doc', version: 1 }), true) + assert.equal(isAdfDocument({ type: 'doc', version: 1 }), true) +}) + +test('rejects anything that is not a doc node', () => { + assert.equal(isAdfDocument(null), false) + assert.equal(isAdfDocument([]), false) + assert.equal(isAdfDocument('doc'), false) + assert.equal(isAdfDocument({ type: 'paragraph', version: 1 }), false) + assert.equal(isAdfDocument({ type: 'doc' }), false) + assert.equal(isAdfDocument({ type: 'doc', version: Number.NaN }), false) + assert.equal(isAdfDocument({ extra: 1, type: 'doc', version: 1 }), false) +}) + +test('rejects a node whose shape ProseMirror JSON cannot hold', () => { + assert.equal(isAdfDocument({ content: [{ type: 1 }], type: 'doc', version: 1 }), false) + assert.equal(isAdfDocument({ content: [{ text: 1, type: 'text' }], type: 'doc', version: 1 }), false) + assert.equal(isAdfDocument({ content: [{ node: 'x', type: 'paragraph' }], type: 'doc', version: 1 }), false) + assert.equal(isAdfDocument({ content: [{ content: {}, type: 'paragraph' }], type: 'doc', version: 1 }), false) + assert.equal(isAdfDocument({ content: [{ marks: [{ type: 1 }], text: 'x', type: 'text' }], type: 'doc', version: 1 }), false) + assert.equal(isAdfDocument({ content: [{ attrs: { a: Number.POSITIVE_INFINITY }, type: 'paragraph' }], type: 'doc', version: 1 }), false) + assert.equal(isAdfDocument({ content: [{ attrs: [], type: 'paragraph' }], type: 'doc', version: 1 }), false) +}) + +test('accepts the JSON values an attribute may hold', () => { + assert.equal(isAdfDocument({ content: [{ attrs: { a: [1, 'x', null, true, { b: 2 }] }, type: 'paragraph' }], type: 'doc', version: 1 }), true) + assert.equal(isAdfDocument({ content: [{ attrs: { a: [() => 1] }, type: 'paragraph' }], type: 'doc', version: 1 }), false) +}) diff --git a/src/adf-document.ts b/src/adf-document.ts new file mode 100644 index 0000000..139457f --- /dev/null +++ b/src/adf-document.ts @@ -0,0 +1,74 @@ +import { isJsonValue, type JsonValue } from './json-value.ts' + +export type AdfAttributes = { [key: string]: JsonValue } + +export type AdfMark = { + attrs?: AdfAttributes + type: string +} + +export type AdfNode = { + attrs?: AdfAttributes + content?: AdfNode[] + marks?: AdfMark[] + text?: string + type: string +} + +export type AdfDocument = { + content?: AdfNode[] + type: 'doc' + version: number +} + +const documentKeys = ['content', 'type', 'version'] +const markKeys = ['attrs', 'type'] +const nodeKeys = ['attrs', 'content', 'marks', 'text', 'type'] + +export function isAdfDocument(value: unknown): value is AdfDocument { + if (!isRecord(value) || !holdsOnly(value, documentKeys)) return false + if (value['type'] !== 'doc') return false + if (typeof value['version'] !== 'number' || !Number.isFinite(value['version'])) return false + return !('content' in value) || isNodeArray(value['content']) +} + +function isAdfMark(value: unknown): value is AdfMark { + if (!isRecord(value) || !holdsOnly(value, markKeys)) return false + if (typeof value['type'] !== 'string') return false + return !('attrs' in value) || isAttributes(value['attrs']) +} + +function isNodeArray(value: unknown): value is AdfNode[] { + if (!Array.isArray(value)) return false + const pending: unknown[] = [...value] + while (pending.length > 0) { + const node = pending.pop() + if (!isRecord(node) || !holdsOnly(node, nodeKeys)) return false + if (typeof node['type'] !== 'string') return false + if ('attrs' in node && !isAttributes(node['attrs'])) return false + if ('marks' in node && !isArrayOf(node['marks'], isAdfMark)) return false + if ('text' in node && typeof node['text'] !== 'string') return false + if ('content' in node) { + const content = node['content'] + if (!Array.isArray(content)) return false + pending.push(...content) + } + } + return true +} + +function isArrayOf(value: unknown, guard: (item: unknown) => item is T): value is T[] { + return Array.isArray(value) && [...value].every(guard) +} + +function isAttributes(value: unknown): value is AdfAttributes { + return isRecord(value) && isJsonValue(value) +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +function holdsOnly(value: Record, keys: readonly string[]): boolean { + return Object.keys(value).every((key) => keys.includes(key)) +} diff --git a/src/adf-to-markdown.test.ts b/src/adf-to-markdown.test.ts new file mode 100644 index 0000000..9126351 --- /dev/null +++ b/src/adf-to-markdown.test.ts @@ -0,0 +1,285 @@ +import assert from 'node:assert/strict' +import test from 'node:test' + +import type { AdfDocument, AdfMark, AdfNode } from './adf-document.ts' +import type { Result } from './result.ts' +import { adfToMarkdown } from './index.ts' + +function document(...content: AdfNode[]): AdfDocument { + return { content, type: 'doc', version: 1 } +} + +function paragraph(...content: AdfNode[]): AdfNode { + return { content, type: 'paragraph' } +} + +function code(result: Result): string { + return result.ok ? `emitted ${JSON.stringify(result.value)}` : result.error.code +} + +function markdown(result: Result): string { + return result.ok ? result.value : `${result.error.code}: ${result.error.message}` +} + +function path(result: Result): readonly (number | string)[] { + return result.ok ? ['emitted'] : result.error.path +} + +test('names the node a refusal came from', () => { + const unspellable: AdfNode = { attrs: { localId: 'a' }, type: 'paragraph' } + const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }, { content: [unspellable], type: 'listItem' }], type: 'bulletList' } + assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }), list))), ['content', 1, 'content', 1, 'content', 0]) + assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, { type: 'mention' })))), ['content', 0, 'content', 1]) + assert.deepEqual(path(adfToMarkdown({ type: 'doc', version: 2 })), []) +}) + +test('refuses a value that is not an ADF document', () => { + assert.equal(code(adfToMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document') +}) + +test('refuses a document version the markdown cannot carry', () => { + assert.equal(code(adfToMarkdown({ type: 'doc', version: 2 })), 'unsupported-document-version') +}) + +test('refuses an attribute the canonical form does not spell', () => { + assert.equal(code(adfToMarkdown(document({ attrs: { localId: 'a' }, type: 'paragraph' }))), 'unspelled-node-attribute') + assert.equal(code(adfToMarkdown(document({ attrs: { wrap: true }, type: 'codeBlock' }))), 'unspelled-node-attribute') + assert.equal(code(adfToMarkdown(document(paragraph({ attrs: { localId: 'a' }, type: 'hardBreak' }, { text: 'x', type: 'text' })))), 'unspelled-node-attribute') +}) + +test('refuses marks on a block node', () => { + assert.equal(code(adfToMarkdown(document({ marks: [{ type: 'border' }], type: 'blockquote' }))), 'unspelled-block-marks') +}) + +test('refuses an ordered list whose markdown start is ambiguous', () => { + const items: AdfNode[] = [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }] + assert.equal(code(adfToMarkdown(document({ content: items, type: 'orderedList' }))), 'ambiguous-ordered-list-start') + assert.equal(code(adfToMarkdown(document({ attrs: { order: 1 }, content: items, type: 'orderedList' }))), 'ambiguous-ordered-list-start') + assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: items, type: 'orderedList' }))), '2. x\n') +}) + +test('refuses the code block info strings the fence cannot hold', () => { + assert.equal(code(adfToMarkdown(document({ attrs: { language: '' }, type: 'codeBlock' }))), 'ambiguous-empty-code-block-language') + assert.equal(code(adfToMarkdown(document({ attrs: { language: 'adf' }, type: 'codeBlock' }))), 'reserved-adf-language') + assert.equal(code(adfToMarkdown(document({ attrs: { language: 'a`b' }, type: 'codeBlock' }))), 'unspellable-code-block-language') + assert.equal(code(adfToMarkdown(document({ attrs: { language: ' sql' }, type: 'codeBlock' }))), 'unspellable-code-block-language') +}) + +test('refuses a link destination CommonMark cannot spell', () => { + const link = (href: string): AdfDocument => document(paragraph({ marks: [{ attrs: { href }, type: 'link' }], text: 't', type: 'text' })) + assert.equal(code(adfToMarkdown(link('https://example.com/a)b'))), 'unspellable-link-destination') + assert.equal(code(adfToMarkdown(link('https://example.com/a b>c'))), 'unspellable-link-destination') + assert.equal(code(adfToMarkdown(link(' { + const titled = (title: string): AdfDocument => + document(paragraph({ marks: [{ attrs: { href: 'https://example.com/', title }, type: 'link' }], text: 't', type: 'text' })) + assert.equal(code(adfToMarkdown(titled('He said "hi"'))), 'unspellable-link-title') + assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link-title') +}) + +test('refuses a link attribute no markdown spelling holds', () => { + assert.equal( + code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { href: 'https://example.com/', id: 'x' }, type: 'link' }], text: 't', type: 'text' })))), + 'unspellable-mark', + ) +}) + +test('refuses a mark the canonical spellings cannot nest', () => { + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'underline' }], text: 'x', type: 'text' })))), 'unspellable-mark') + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }, { type: 'strong' }], text: 'x', type: 'text' })))), 'unspellable-mark') + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { colour: 'red' }, type: 'em' }], text: 'x', type: 'text' })))), 'unspellable-mark') +}) + +test('refuses whitespace CommonMark cannot hold', () => { + assert.equal(code(adfToMarkdown(document(paragraph({ text: ' lead', type: 'text' })))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'trail ', type: 'text' })))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\nb', type: 'text' })))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(document(paragraph({ text: '\fa', type: 'text' })))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'em' }], text: 'x ', type: 'text' }, { text: 'y', type: 'text' })))), 'unspellable-whitespace') +}) + +test('refuses a line whose start block parsing would claim', () => { + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: '```', type: 'text' })))), 'unspellable-line-start') +}) + +test('refuses two adjacent lists of the same kind', () => { + const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'bulletList' } + assert.equal(code(adfToMarkdown(document(list, list))), 'unspellable-adjacent-lists') +}) + +test('refuses a node type the canonical form does not cover', () => { + assert.equal(code(adfToMarkdown(document({ type: 'panel' }))), 'unsupported-node-type') + assert.equal(code(adfToMarkdown(document(paragraph({ type: 'mention' })))), 'unsupported-node-type') +}) + +test('refuses a node whose content model the canonical form cannot emit', () => { + assert.equal(code(adfToMarkdown(document({ type: 'listItem' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ content: [paragraph()], type: 'codeBlock' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ content: [paragraph()], type: 'bulletList' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ type: 'bulletList' }))), 'unsupported-node-shape') + assert.equal(code(adfToMarkdown(document({ attrs: { order: 2 }, content: [], type: 'orderedList' }))), 'unsupported-node-shape') +}) + +test('refuses an ordered list no marker spells', () => { + const item: AdfNode = { content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' } + const list = (order: number, items: number): AdfDocument => + document({ attrs: { order }, content: Array.from({ length: items }, () => item), type: 'orderedList' }) + assert.equal(code(adfToMarkdown(list(1.5, 1))), 'unsupported-node-shape') + assert.equal(markdown(adfToMarkdown(list(999999999, 1))), '999999999. x\n') + assert.equal(code(adfToMarkdown(list(999999999, 2))), 'unspellable-list-marker') +}) + +test('refuses a code span over anything but one text node', () => { + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], type: 'hardBreak' })))), 'unspellable-mark') +}) + +test('refuses a heading level outside the ATX range', () => { + assert.equal(code(adfToMarkdown(document({ attrs: { level: 7 }, content: [{ text: 'x', type: 'text' }], type: 'heading' }))), 'unsupported-heading-level') + assert.equal(code(adfToMarkdown(document({ content: [{ text: 'x', type: 'text' }], type: 'heading' }))), 'unsupported-heading-level') +}) + +test('escapes only text that would otherwise open a construct', () => { + const emitted = (text: string): string => markdown(adfToMarkdown(document(paragraph({ text, type: 'text' })))) + assert.equal(emitted('
'), '\\
\n') + assert.equal(emitted('a < b'), 'a < b\n') + assert.equal(emitted('& & x'), '\\& & x\n') + assert.equal(emitted('| a | b |'), '\\| a | b |\n') + assert.equal(emitted(':mention[@x]{id=1}'), '\\:mention[@x]{id=1}\n') + assert.equal(emitted(':::panel info'), '\\:::panel info\n') + assert.equal(emitted('10:30 tomorrow'), '10:30 tomorrow\n') + assert.equal(emitted('[a](b)'), '\\[a](b)\n') + assert.equal(emitted('**bold**'), '\\*\\*bold**\n') + assert.equal(emitted('a `x` b'), 'a \\`x` b\n') + assert.equal(emitted('~~struck~~'), '\\~~struck~~\n') + assert.equal(emitted('a \\* b'), 'a \\\\* b\n') + assert.equal(emitted('1. not a list'), '1\\. not a list\n') + assert.equal(emitted('*"quoted"*'), '\\*"quoted"*\n') + assert.equal(emitted('x"_y"'), 'x"\\_y"\n') +}) + +test('escapes a heading closing sequence', () => { + const heading = (text: string): string => markdown(adfToMarkdown(document({ attrs: { level: 2 }, content: [{ text, type: 'text' }], type: 'heading' }))) + assert.equal(heading('done #'), '## done \\#\n') + assert.equal(heading('#tag first'), '## #tag first\n') +}) + +test('wraps adjacent nodes carrying one mark once, and a differing mark twice', () => { + const marked = (text: string, ...marks: AdfMark[]): AdfNode => ({ marks, text, type: 'text' }) + const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content)))) + assert.equal(emitted(marked('a', { type: 'strong' }), marked('b', { type: 'strong' }, { type: 'em' })), '**a*b***\n') + assert.equal(emitted(marked('a', { type: 'strong' }), marked('b', { type: 'em' })), '**a**_b_\n') + const link = (href: string): AdfMark => ({ attrs: { href }, type: 'link' }) + assert.equal(emitted(marked('a', link('http://x')), marked('b', link('http://y'))), '[a](http://x)[b](http://y)\n') +}) + +test('refuses a mark spelling that cannot open or close where it sits', () => { + const marked = (text: string, ...marks: AdfMark[]): AdfNode => ({ marks, text, type: 'text' }) + const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content)))) + const strong: AdfMark = { type: 'strong' } + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'un', type: 'text' }, marked('-real', strong), { text: 'istic', type: 'text' })))), 'unspellable-mark') + assert.equal(code(adfToMarkdown(document(paragraph(marked('C++', { type: 'em' }), { text: 'ish', type: 'text' })))), 'unspellable-mark') + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, marked('.a', strong))))), 'unspellable-mark') + assert.equal(emitted({ text: 'un ', type: 'text' }, marked('-real', strong), { text: ' istic', type: 'text' }), 'un **-real** istic\n') + assert.equal(emitted(marked('a.', strong)), '**a.**\n') + assert.equal(emitted({ text: 'x', type: 'text' }, marked('a', strong), { text: 'y', type: 'text' }), 'x**a**y\n') + const em: AdfMark = { type: 'em' } + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, marked('a.', em), marked('b', strong))))), 'unspellable-mark') + assert.equal(emitted({ text: 'x', type: 'text' }, marked('ab', em, strong), { text: 'y', type: 'text' }), 'x***ab***y\n') +}) + +test('refuses a node carrying one mark type twice', () => { + const em: AdfMark = { type: 'em' } + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [em, em], text: 'x', type: 'text' })))), 'unsupported-node-shape') +}) + +test('refuses a nested list the tight spelling would swallow', () => { + const item = (...content: AdfNode[]): AdfNode => ({ content, type: 'listItem' }) + const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) + const outer = (...content: AdfNode[]): AdfDocument => document({ content: [item(...content)], type: 'bulletList' }) + const ordered: AdfNode = { attrs: { order: 2 }, content: [item(text('b'))], type: 'orderedList' } + assert.equal(code(adfToMarkdown(outer(text('a'), ordered))), 'unspellable-line-start') + assert.equal(code(adfToMarkdown(outer(text('a'), { content: [item()], type: 'bulletList' }))), 'unspellable-line-start') + assert.equal(markdown(adfToMarkdown(outer(text('a'), { content: [item(text('b'))], type: 'bulletList' }))), '- a\n - b\n') +}) + +test('refuses marks and attributes nested deeper than the emitter carries', () => { + const marks: AdfMark[] = Array.from({ length: 600 }, (_, index) => ({ type: index % 2 === 0 ? 'em' : 'strong' })) + assert.equal(code(adfToMarkdown(document(paragraph({ marks, text: 'x', type: 'text' })))), 'unsupported-node-shape') + let attrs: AdfMark['attrs'] = { depth: 'x' } + for (let depth = 0; depth < 600; depth += 1) attrs = { depth: attrs } + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs, type: 'em' }], text: 'x', type: 'text' })))), 'not-an-adf-document') +}) + +test('escapes a literal delimiter that would merge with an emitted one', () => { + const marked = (text: string, ...marks: AdfMark[]): AdfNode => ({ marks, text, type: 'text' }) + const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content)))) + assert.equal(emitted(marked('a_', { type: 'em' })), '_a\\__\n') + assert.equal(emitted(marked('_a', { type: 'em' })), '_\\_a_\n') + assert.equal(emitted(marked('a*', { type: 'strong' })), '**a\\***\n') + assert.equal(emitted(marked('~a', { type: 'strike' })), '~~\\~a~~\n') + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, marked('~a', { type: 'strike' }))))), 'unspellable-mark') + assert.equal(emitted({ text: '`', type: 'text' }, marked('x', { type: 'code' })), '\\``x`\n') + assert.equal(emitted(marked('x', { type: 'code' }), { text: '`', type: 'text' }), '`x`\\`\n') + assert.equal(emitted({ text: '!', type: 'text' }, marked('x', { attrs: { href: 'https://example.com/' }, type: 'link' })), '\\![x](https://example.com/)\n') +}) + +test('escapes a hyphen underline a hard break would expose', () => { + const line = (text: string): string => markdown(adfToMarkdown(document(paragraph({ text: 'foo', type: 'text' }, { type: 'hardBreak' }, { text, type: 'text' })))) + assert.equal(line('--'), 'foo\\\n\\--\n') + assert.equal(line('=='), 'foo\\\n\\==\n') +}) + +test('refuses a list item whose marker completes a thematic break', () => { + const item = (...content: AdfNode[]): AdfNode => ({ content, type: 'listItem' }) + assert.equal(code(adfToMarkdown(document({ content: [item({ type: 'rule' })], type: 'bulletList' }))), 'unspellable-line-start') + const nested: AdfNode = { content: [item({ content: [item()], type: 'bulletList' })], type: 'bulletList' } + assert.equal(markdown(adfToMarkdown(document(nested))), '- -\n') + assert.equal(code(adfToMarkdown(document({ content: [item(nested)], type: 'bulletList' }))), 'unspellable-line-start') +}) + +test('refuses the characters CommonMark rewrites', () => { + assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\rb', type: 'text' }], type: 'codeBlock' }))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\u0000b', type: 'text' })))), 'unspellable-character') + assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))), 'unspellable-character') + assert.equal(code(adfToMarkdown(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))), 'unsupported-node-shape') +}) + +test('refuses a text node carrying no text at all', () => { + assert.equal(code(adfToMarkdown(document(paragraph({ text: '', type: 'text' })))), 'unsupported-node-shape') +}) + +test('refuses a mark run whose edge holds whitespace CommonMark flanking counts', () => { + const em = { type: 'em' } + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [em], text: 'a', type: 'text' }, { marks: [em], type: 'hardBreak' }, { text: 'b', type: 'text' })))), 'unspellable-whitespace') + assert.equal(code(adfToMarkdown(document(paragraph({ marks: [em], text: '\u00a0a', type: 'text' })))), 'unspellable-whitespace') +}) + +test('pads a code span whose edges CommonMark would strip', () => { + assert.equal(markdown(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: ' \t ', type: 'text' })))), '` \t `\n') +}) + +test('spells one code span over a run of code-marked nodes', () => { + const code_ = { type: 'code' } + assert.equal( + markdown(adfToMarkdown(document(paragraph({ marks: [code_], text: 'a', type: 'text' }, { marks: [code_], text: 'b', type: 'text' })))), + '`ab`\n', + ) +}) + +test('refuses a document nested deeper than the emitter carries', () => { + let node: AdfNode = paragraph({ text: 'x', type: 'text' }) + for (let depth = 0; depth < 600; depth += 1) node = { content: [node], type: 'blockquote' } + assert.equal(code(adfToMarkdown(document(node))), 'unsupported-node-shape') +}) + +test('emits an empty list item without trailing whitespace', () => { + assert.equal(markdown(adfToMarkdown(document({ content: [{ type: 'listItem' }], type: 'bulletList' }))), '-\n') +}) diff --git a/src/adf-to-markdown.ts b/src/adf-to-markdown.ts new file mode 100644 index 0000000..6378c79 --- /dev/null +++ b/src/adf-to-markdown.ts @@ -0,0 +1,192 @@ +import type { AdfDocument, AdfNode } from './adf-document.ts' +import type { JsonValue } from './json-value.ts' +import { emitInlineLine } from './markdown-inline.ts' +import { failure, success, type ConvertErrorPath, type Result } from './result.ts' +import { holdsNullCharacter, isThematicBreak } from './commonmark-grammar.ts' +import { isAdfDocument } from './adf-document.ts' +import { largestNesting } from './nesting.ts' +import { longestBacktickRun } from './backtick-runs.ts' + +const largestListMarker = 999999999 +const listTypes = ['bulletList', 'orderedList'] + +export function adfToMarkdown(document: AdfDocument): Result { + if (!isAdfDocument(document)) return failure('not-an-adf-document', 'the value is not an ADF document', []) + if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) + const blocks = emitBlocks(document.content ?? [], false, [], 0) + if (!blocks.ok) return blocks + return success(blocks.value === '' ? '' : `${blocks.value}\n`) +} + +function emitBlocks(nodes: readonly AdfNode[], inListItem: boolean, path: ConvertErrorPath, depth: number): Result { + if (depth > largestNesting) return failure('unsupported-node-shape', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) + let output = '' + let previous: AdfNode | undefined + for (const [index, node] of nodes.entries()) { + const nodePath = [...path, 'content', index] + if (previous !== undefined) { + if (listTypes.includes(node.type) && previous.type === node.type) { + return failure('unspellable-adjacent-lists', `two adjacent ${node.type} nodes read back as one list`, nodePath) + } + if (inListItem && listTypes.includes(node.type)) { + if (!interruptsParagraph(node)) { + return failure('unspellable-line-start', `a ${node.type} that cannot interrupt the block above it has no tight spelling`, nodePath) + } + output += '\n' + } else output += '\n\n' + } + const block = emitBlock(node, nodePath, depth) + if (!block.ok) return block + output += block.value + previous = node + } + return success(output) +} + +function interruptsParagraph(node: AdfNode): boolean { + if (node.type === 'orderedList') return false + return ((node.content ?? [])[0]?.content ?? []).length > 0 +} + +function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result { + if (node.type === 'blockquote') return emitBlockquote(node, path, depth) + if (node.type === 'bulletList' || node.type === 'orderedList') return emitList(node, path, depth) + if (node.type === 'codeBlock') return emitCodeBlock(node, path) + if (node.type === 'heading') return emitHeading(node, path) + if (node.type === 'paragraph') return emitParagraph(node, path) + if (node.type === 'rule') return emitRule(node, path) + if (node.type === 'hardBreak' || node.type === 'listItem' || node.type === 'text') { + return failure('unsupported-node-shape', `a ${node.type} node cannot stand where a block belongs`, path) + } + return failure('unsupported-node-type', `the canonical form spells no block node of type ${node.type}`, path) +} + +function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result { + const validation = validateBlockNode(node, [], path) + if (!validation.ok) return validation + const inner = emitBlocks(node.content ?? [], false, path, depth + 1) + if (!inner.ok) return inner + return success( + inner.value + .split('\n') + .map((line) => (line === '' ? '>' : `> ${line}`)) + .join('\n'), + ) +} + +function emitCodeBlock(node: AdfNode, path: ConvertErrorPath): Result { + const validation = validateBlockNode(node, ['language'], path) + if (!validation.ok) return validation + const info = spellCodeFenceInfo(node.attrs?.['language'], path) + if (!info.ok) return info + let text = '' + for (const [index, child] of (node.content ?? []).entries()) { + const childPath = [...path, 'content', index] + if (child.type !== 'text' || typeof child.text !== 'string' || child.text === '' || (child.marks ?? []).length > 0 || Object.keys(child.attrs ?? {}).length > 0) { + return failure('unsupported-node-shape', 'a codeBlock holds plain text nodes only', childPath) + } + if (/\r/.test(child.text)) return failure('unspellable-whitespace', 'a codeBlock holds no carriage return CommonMark keeps', childPath) + if (holdsNullCharacter(child.text)) return failure('unspellable-character', 'a codeBlock holds a null character CommonMark replaces', childPath) + text += child.text + } + const fence = '`'.repeat(Math.max(3, longestBacktickRun(text) + 1)) + const opening = `${fence}${info.value}` + return success(text === '' ? `${opening}\n${fence}` : `${opening}\n${text}\n${fence}`) +} + +function spellCodeFenceInfo(language: JsonValue | undefined, path: ConvertErrorPath): Result { + if (language === undefined) return success('') + if (typeof language !== 'string') return failure('unsupported-node-shape', 'a codeBlock language is no string', path) + if (language === '') { + return failure('ambiguous-empty-code-block-language', 'an empty codeBlock language and an absent one share one markdown spelling', path) + } + if (language === 'adf') return failure('reserved-adf-language', 'the adf info string is reserved for the opaque carry', path) + if (/[`\n\r]/.test(language) || language !== language.trim()) { + return failure('unspellable-code-block-language', 'a fence info string holds no backtick and no edge whitespace', path) + } + return success(language) +} + +function emitHeading(node: AdfNode, path: ConvertErrorPath): Result { + const validation = validateBlockNode(node, ['level'], path) + if (!validation.ok) return validation + const level = node.attrs?.['level'] + if (typeof level !== 'number' || !Number.isInteger(level) || level < 1 || level > 6) { + return failure('unsupported-heading-level', `no ATX heading spells level ${JSON.stringify(level ?? null)}`, path) + } + const hashes = '#'.repeat(level) + const content = node.content ?? [] + if (content.length === 0) return success(hashes) + const line = emitInlineLine(content, 'heading', path) + if (!line.ok) return line + return success(`${hashes} ${line.value}`) +} + +function emitList(node: AdfNode, path: ConvertErrorPath, depth: number): Result { + const ordered = node.type === 'orderedList' + const validation = validateBlockNode(node, ordered ? ['order'] : [], path) + if (!validation.ok) return validation + const items = node.content ?? [] + if (items.length === 0) return failure('unsupported-node-shape', `a ${node.type} holds at least one listItem`, path) + const start = ordered ? node.attrs?.['order'] : 0 + if (ordered && (start === undefined || start === 1)) { + return failure('ambiguous-ordered-list-start', 'an orderedList starting at 1 and one with no order share one markdown spelling', path) + } + if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) { + return failure('unsupported-node-shape', `no list marker spells the order ${JSON.stringify(start ?? null)}`, path) + } + if (start + items.length - 1 > largestListMarker) { + return failure('unspellable-list-marker', `no list marker spells the ${items.length} items a list starting at ${start} needs`, path) + } + const lines: string[] = [] + for (const [offset, item] of items.entries()) { + const itemPath = [...path, 'content', offset] + if (item.type !== 'listItem') return failure('unsupported-node-shape', `a ${node.type} holds listItem nodes only`, itemPath) + const emitted = emitListItem(item, ordered ? `${start + offset}. ` : '- ', itemPath, depth) + if (!emitted.ok) return emitted + lines.push(emitted.value) + } + return success(lines.join('\n')) +} + +function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, depth: number): Result { + const validation = validateBlockNode(item, [], path) + if (!validation.ok) return validation + const inner = emitBlocks(item.content ?? [], true, path, depth + 1) + if (!inner.ok) return inner + if (inner.value === '') return success(marker.trimEnd()) + const indent = ' '.repeat(marker.length) + const lines = inner.value.split('\n').map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`)) + const first = lines[0] ?? '' + if (isThematicBreak(first)) { + return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(first)}`, path) + } + return success(lines.join('\n')) +} + +function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result { + const validation = validateBlockNode(node, [], path) + if (!validation.ok) return validation + const content = node.content ?? [] + if (content.length === 0) return success('::paragraph') + return emitInlineLine(content, 'paragraph', path) +} + +function emitRule(node: AdfNode, path: ConvertErrorPath): Result { + const validation = validateBlockNode(node, [], path) + if (!validation.ok) return validation + if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a rule holds no content', path) + return success('---') +} + +function validateBlockNode(node: AdfNode, spelled: readonly string[], path: ConvertErrorPath): Result { + if ((node.marks ?? []).length > 0) { + return failure('unspelled-block-marks', `the canonical form has no place for the marks a ${node.type} carries`, path) + } + if (node.text !== undefined) return failure('unsupported-node-shape', `a ${node.type} carries no text`, path) + const unspelled = Object.keys(node.attrs ?? {}).find((key) => !spelled.includes(key)) + if (unspelled !== undefined) { + return failure('unspelled-node-attribute', `the ${node.type} attribute ${unspelled} has no canonical markdown spelling`, path) + } + return success(null) +} diff --git a/src/backtick-runs.ts b/src/backtick-runs.ts new file mode 100644 index 0000000..e5580db --- /dev/null +++ b/src/backtick-runs.ts @@ -0,0 +1,9 @@ +export function longestBacktickRun(text: string): number { + let longest = 0 + let current = 0 + for (const character of text) { + current = character === '`' ? current + 1 : 0 + longest = Math.max(longest, current) + } + return longest +} diff --git a/src/canonical-json.test.ts b/src/canonical-json.test.ts new file mode 100644 index 0000000..b6fdaf4 --- /dev/null +++ b/src/canonical-json.test.ts @@ -0,0 +1,41 @@ +import assert from 'node:assert/strict' +import test from 'node:test' + +import { serializeCanonicalJson } from './canonical-json.ts' + +test('sorts object keys recursively', () => { + const value = { b: 1, a: { d: 2, c: 3 } } + assert.equal(serializeCanonicalJson(value, 'compact'), '{"a":{"c":3,"d":2},"b":1}') +}) + +test('spells two-space indentation the way the corpus holds it', () => { + const value = { content: [{ text: 'x', type: 'text' }], type: 'doc', version: 1 } + assert.equal( + serializeCanonicalJson(value, 'two-space'), + [ + '{', + ' "content": [', + ' {', + ' "text": "x",', + ' "type": "text"', + ' }', + ' ],', + ' "type": "doc",', + ' "version": 1', + '}', + ].join('\n'), + ) +}) + +test('keeps empty objects and arrays on one line in both spellings', () => { + assert.equal(serializeCanonicalJson({ a: {}, b: [] }, 'two-space'), '{\n "a": {},\n "b": []\n}') + assert.equal(serializeCanonicalJson({ a: {}, b: [] }, 'compact'), '{"a":{},"b":[]}') +}) + +test('leaves non-ASCII raw', () => { + assert.equal(serializeCanonicalJson({ text: '🎉 räksmörgås' }, 'compact'), '{"text":"🎉 räksmörgås"}') +}) + +test('spells scalars in canonical JSON', () => { + assert.equal(serializeCanonicalJson([null, true, false, 0, -1.5, 'a"b'], 'compact'), '[null,true,false,0,-1.5,"a\\"b"]') +}) diff --git a/src/canonical-json.ts b/src/canonical-json.ts new file mode 100644 index 0000000..2f08e20 --- /dev/null +++ b/src/canonical-json.ts @@ -0,0 +1,29 @@ +import type { JsonValue } from './json-value.ts' + +export type JsonSpelling = 'compact' | 'two-space' + +export function serializeCanonicalJson(value: JsonValue, spelling: JsonSpelling): string { + return serialize(value, spelling === 'compact' ? '' : ' ', 0) +} + +function serialize(value: JsonValue, indent: string, depth: number): string { + if (Array.isArray(value)) { + if (value.length === 0) return '[]' + const items = value.map((item) => serialize(item, indent, depth + 1)) + return `[${join(items, indent, depth)}]` + } + if (value !== null && typeof value === 'object') { + const keys = Object.keys(value).sort() + if (keys.length === 0) return '{}' + const separator = indent === '' ? ':' : ': ' + const entries = keys.map((key) => `${JSON.stringify(key)}${separator}${serialize(value[key] ?? null, indent, depth + 1)}`) + return `{${join(entries, indent, depth)}}` + } + return JSON.stringify(value) +} + +function join(parts: readonly string[], indent: string, depth: number): string { + if (indent === '') return parts.join(',') + const inner = `\n${indent.repeat(depth + 1)}` + return `${inner}${parts.join(`,${inner}`)}\n${indent.repeat(depth)}` +} diff --git a/src/commonmark-grammar.ts b/src/commonmark-grammar.ts new file mode 100644 index 0000000..2fd77a6 --- /dev/null +++ b/src/commonmark-grammar.ts @@ -0,0 +1,63 @@ +export type LinePosition = 'first' | 'later' + +const controlCharacterRange = '\\u0000-\\u001f\\u007f' +const autolinkSource = `[A-Za-z][A-Za-z0-9+.-]{1,31}:[^\\s<>${controlCharacterRange}]*` +const nullCharacterSource = '\\u0000' +const entityReferenceSource = '&(?:[A-Za-z][A-Za-z0-9]{1,31}|#\\d{1,7}|#[Xx][A-Fa-f0-9]{1,6});' + +const anchoredEntityReference = new RegExp(`^(?:${entityReferenceSource})`) +const autolink = new RegExp(`^(?:${autolinkSource})$`) +const bracketedAutolink = new RegExp(`^<(?:${autolinkSource})>`) +const controlCharacter = new RegExp(`[${controlCharacterRange}]`) +const entityReference = new RegExp(entityReferenceSource) +const nullCharacter = new RegExp(nullCharacterSource) +const firstCharacterOpeners = [/^#{1,6}(?:[ \t]|$)/, /^>/, /^[*+-](?:[ \t]|$)/, /^`{3,}/, /^~{3,}/, /^:{2,}/, /^\|/] +const orderedListOpener = /^(\d{1,9})[.)](?:[ \t]|$)/ +const setextUnderline = /^(?:=+|-+)$/ +const thematicBreak = /^(?:(?:\*[ \t]*){3,}|(?:-[ \t]*){3,}|(?:_[ \t]*){3,})$/ +const unicodeWhitespace = /[\t\n\f\r \p{Zs}]/u + +export function claimsLine(line: string, position: LinePosition): boolean { + return escapesLineClaim(line, 0, position) || orderedListOpener.test(line) +} + +export function escapesLineClaim(line: string, offset: number, position: LinePosition): boolean { + if (offset === 0) { + if (firstCharacterOpeners.some((opener) => opener.test(line)) || thematicBreak.test(line)) return true + return position === 'later' && setextUnderline.test(line) + } + const digits = orderedListOpener.exec(line)?.[1] + return digits !== undefined && offset === digits.length +} + +export function holdsControlCharacter(text: string): boolean { + return controlCharacter.test(text) +} + +export function holdsEntityReference(text: string): boolean { + return entityReference.test(text) +} + +export function holdsNullCharacter(text: string): boolean { + return nullCharacter.test(text) +} + +export function isAutolink(text: string): boolean { + return autolink.test(text) +} + +export function isThematicBreak(line: string): boolean { + return thematicBreak.test(line) +} + +export function isUnicodeWhitespace(character: string): boolean { + return unicodeWhitespace.test(character) +} + +export function opensBracketedAutolink(text: string): boolean { + return bracketedAutolink.test(text) +} + +export function startsEntityReference(text: string): boolean { + return anchoredEntityReference.test(text) +} diff --git a/src/corpus.test.ts b/src/corpus.test.ts new file mode 100644 index 0000000..18345e6 --- /dev/null +++ b/src/corpus.test.ts @@ -0,0 +1,81 @@ +import assert from 'node:assert/strict' +import { readFileSync, readdirSync } from 'node:fs' +import { dirname, join } from 'node:path' +import test from 'node:test' +import { fileURLToPath } from 'node:url' + +import { adfToMarkdown } from './adf-to-markdown.ts' +import { isAdfDocument } from './adf-document.ts' +import { isJsonValue } from './json-value.ts' +import { serializeCanonicalJson } from './canonical-json.ts' + +const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus') +const roundTripRoot = join(corpusRoot, 'round-trip') + +const emittingDirectories = ['commonmark-subset'] +const pendingDirectories = ['block-nodes', 'inline-nodes'] + +function directoryNames(root: string): string[] { + return readdirSync(root, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort() +} + +function fixtureNames(directory: string, extension: string): string[] { + return readdirSync(join(roundTripRoot, directory)) + .filter((name) => name.endsWith(extension)) + .map((name) => name.slice(0, -extension.length)) + .sort() +} + +function corpusJsonPaths(): string[] { + return readdirSync(corpusRoot, { encoding: 'utf8', recursive: true }) + .filter((name) => name.endsWith('.json')) + .map((name) => join(corpusRoot, name)) + .sort() +} + +test('every round-trip directory is either emitting or explicitly pending', () => { + assert.deepEqual(directoryNames(roundTripRoot), [...emittingDirectories, ...pendingDirectories].sort()) +}) + +for (const directory of [...emittingDirectories, ...pendingDirectories].sort()) { + test(`${directory} pairs every .json with a .md`, () => { + assert.deepEqual(fixtureNames(directory, '.json'), fixtureNames(directory, '.md')) + }) +} + +for (const directory of emittingDirectories) { + const names = [...new Set([...fixtureNames(directory, '.json'), ...fixtureNames(directory, '.md')])].sort() + + test(`${directory} holds fixtures`, () => { + assert.ok(names.length > 0, `${directory} is expected to emit but holds no fixture pairs`) + }) + + for (const name of names) { + test(`${directory}/${name} emits its markdown byte for byte`, () => { + const parsed: unknown = JSON.parse(readFileSync(join(roundTripRoot, directory, `${name}.json`), 'utf8')) + assert.ok(isAdfDocument(parsed), `${name}.json is not an ADF document`) + const result = adfToMarkdown(parsed) + assert.ok(result.ok, result.ok ? '' : `${result.error.code}: ${result.error.message}`) + const expected = readFileSync(join(roundTripRoot, directory, `${name}.md`)) + const emitted = Buffer.from(result.value, 'utf8') + if (!emitted.equals(expected)) assert.equal(result.value, expected.toString('utf8')) + assert.ok(emitted.equals(expected)) + }) + } +} + +test('the corpus holds JSON to gate', () => { + assert.ok(corpusJsonPaths().length > 0) +}) + +for (const path of corpusJsonPaths()) { + test(`${path.slice(corpusRoot.length + 1)} re-serializes to itself`, () => { + const raw = readFileSync(path, 'utf8') + const parsed: unknown = JSON.parse(raw) + assert.ok(isJsonValue(parsed), `${path} does not hold a JSON value`) + assert.equal(`${serializeCanonicalJson(parsed, 'two-space')}\n`, raw) + }) +} diff --git a/src/index.test.ts b/src/index.test.ts deleted file mode 100644 index fc6fe0c..0000000 --- a/src/index.test.ts +++ /dev/null @@ -1,7 +0,0 @@ -import assert from 'node:assert/strict' -import test from 'node:test' - -test('the test harness runs TypeScript', () => { - const answer: number = 42 - assert.equal(answer, 42) -}) diff --git a/src/index.ts b/src/index.ts index 336ce12..6afbe03 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1 +1,5 @@ -export {} +export type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './adf-document.ts' +export type { ConvertError, ConvertErrorCode, Result } from './result.ts' +export type { JsonValue } from './json-value.ts' +export { adfToMarkdown } from './adf-to-markdown.ts' +export { isAdfDocument } from './adf-document.ts' diff --git a/src/json-value.ts b/src/json-value.ts new file mode 100644 index 0000000..88b548a --- /dev/null +++ b/src/json-value.ts @@ -0,0 +1,23 @@ +import { largestNesting } from './nesting.ts' + +export type JsonValue = JsonValue[] | boolean | null | number | string | { [key: string]: JsonValue } + +export function isJsonValue(value: unknown): value is JsonValue { + const pending: { depth: number; item: unknown }[] = [{ depth: 0, item: value }] + while (pending.length > 0) { + const entry = pending.pop() + if (entry === undefined) continue + const { depth, item } = entry + if (depth > largestNesting) return false + if (item === null || typeof item === 'boolean' || typeof item === 'string') continue + if (typeof item === 'number') { + if (!Number.isFinite(item)) return false + continue + } + // A hole is not a JSON value, and Array.prototype methods skip holes — spreading materialises them. + if (Array.isArray(item)) for (const child of [...item]) pending.push({ depth: depth + 1, item: child }) + else if (typeof item === 'object') for (const child of Object.values(item)) pending.push({ depth: depth + 1, item: child }) + else return false + } + return true +} diff --git a/src/markdown-escaping.ts b/src/markdown-escaping.ts new file mode 100644 index 0000000..833528d --- /dev/null +++ b/src/markdown-escaping.ts @@ -0,0 +1,217 @@ +import { escapesLineClaim, isUnicodeWhitespace, opensBracketedAutolink, startsEntityReference, type LinePosition } from './commonmark-grammar.ts' + +export type InlineSegment = + | { kind: 'emphasis-close' | 'emphasis-open'; mark: string; text: string } + | { kind: 'link-text' | 'literal' | 'syntax'; text: string } + +export type AssembledLine = { line: string; unspellableMark: string | undefined } + +export type LineContainer = 'heading' | 'paragraph' + +type DelimiterRun = { character: string; closeMark: string | undefined; end: number; openMark: string | undefined; start: number } + +const delimiters = ['*', '_', '`', '~'] + +const asciiPunctuation = /[!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~]/ +const htmlConstructs = [/^<[!?]/, /^<\/?[A-Za-z][A-Za-z0-9-]*(?:[\s/>]|$)/, /^<[^\s<>@]+@[^\s<>@]+>/] +const inlineDirective = /^:[a-z][A-Za-z0-9]*[[{]/ +const linkOpener = /\](?=[([:])/ +const unicodePunctuation = /[\p{P}\p{S}]/u + +export function assembleInlineLine(segments: readonly InlineSegment[], container: LineContainer): AssembledLine { + return escape(resolveEmphasis(segments), container) +} + +function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] { + const resolved = segments.map((segment) => ({ ...segment })) + // Offsets index the pre-swap text: every emphasis spelling this swaps between is one character wide. + const scan = resolved.map((segment) => segment.text).join('') + const offsets: number[] = [] + let offset = 0 + for (const segment of resolved) { + offsets.push(offset) + offset += segment.text.length + } + const open: number[] = [] + for (let index = 0; index < resolved.length; index += 1) { + const segment = resolved[index] + if (segment === undefined) continue + if (segment.kind === 'emphasis-open') open.push(index) + if (segment.kind !== 'emphasis-close') continue + const openerIndex = open.pop() + const opener = openerIndex === undefined ? undefined : resolved[openerIndex] + if (openerIndex === undefined || opener === undefined) continue + const openOffset = offsets[openerIndex] ?? 0 + const closeOffset = offsets[index] ?? 0 + if (opener.text !== '_') continue + if (!isWordCharacter(charAt(scan, openOffset - 1)) && !isWordCharacter(charAt(scan, closeOffset + 1))) continue + opener.text = '*' + segment.text = '*' + } + return resolved +} + +function escape(segments: readonly InlineSegment[], container: LineContainer): AssembledLine { + const scan = segments.map((segment) => segment.text).join('') + const kinds: InlineSegment['kind'][] = [] + for (const segment of segments) for (let index = 0; index < segment.text.length; index += 1) kinds.push(segment.kind) + const escaped = new Set() + const placements: number[] = [] + let output = '' + for (let index = 0; index < scan.length; index += 1) { + const kind = kinds[index] + const escapable = kind === 'literal' || kind === 'link-text' + if (escapable && (mergesWithSyntax(scan, kinds, index) || opensConstruct(scan, index, kind === 'link-text', container, escaped))) { + output += '\\' + escaped.add(index) + } + placements.push(output.length) + output += scan.charAt(index) + } + return { line: output, unspellableMark: unspellableMark(segments, output, placements) } +} + +function unspellableMark(segments: readonly InlineSegment[], output: string, placements: readonly number[]): string | undefined { + for (const run of delimiterRuns(segments, placements)) { + const before = charAt(output, run.start - 1) + const after = output.charAt(run.end) + if (run.openMark !== undefined && !isLeftFlanking(before, after)) return run.openMark + if (run.closeMark !== undefined && !isRightFlanking(before, after)) return run.closeMark + } + return undefined +} + +function delimiterRuns(segments: readonly InlineSegment[], placements: readonly number[]): DelimiterRun[] { + const runs: DelimiterRun[] = [] + let cursor = 0 + for (const segment of segments) { + const start = placements[cursor] ?? 0 + cursor += segment.text.length + if (segment.kind !== 'emphasis-close' && segment.kind !== 'emphasis-open') continue + const closes = segment.kind === 'emphasis-close' + const end = start + segment.text.length + const previous = runs[runs.length - 1] + if (previous !== undefined && previous.end === start && previous.character === segment.text.charAt(0)) { + previous.closeMark = previous.closeMark ?? (closes ? segment.mark : undefined) + previous.end = end + previous.openMark = previous.openMark ?? (closes ? undefined : segment.mark) + continue + } + runs.push({ + character: segment.text.charAt(0), + closeMark: closes ? segment.mark : undefined, + end, + openMark: closes ? undefined : segment.mark, + start, + }) + } + return runs +} + +function mergesWithSyntax(scan: string, kinds: readonly (InlineSegment['kind'] | undefined)[], index: number): boolean { + const character = scan.charAt(index) + if (character === '!') return scan.charAt(index + 1) === '[' && isSyntax(kinds[index + 1]) + if (!delimiters.includes(character)) return false + return touchesSyntax(scan, kinds, index, -1) || touchesSyntax(scan, kinds, index, 1) +} + +function touchesSyntax(scan: string, kinds: readonly (InlineSegment['kind'] | undefined)[], index: number, step: number): boolean { + const character = scan.charAt(index) + let cursor = index + step + while (scan.charAt(cursor) === character && !isSyntax(kinds[cursor])) cursor += step + return scan.charAt(cursor) === character && isSyntax(kinds[cursor]) +} + +function isSyntax(kind: InlineSegment['kind'] | undefined): boolean { + return kind === 'emphasis-close' || kind === 'emphasis-open' || kind === 'syntax' +} + +function opensConstruct(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet): boolean { + const claimsLine = container === 'heading' ? closesHeading(scan, index) : claimsLineStart(scan, index) + return claimsLine || claimsCharacter(scan, index, inLinkText, escaped) +} + +function claimsLineStart(scan: string, index: number): boolean { + const start = scan.lastIndexOf('\n', index - 1) + 1 + const end = scan.indexOf('\n', index) + const line = scan.slice(start, end === -1 ? undefined : end) + const position: LinePosition = start === 0 ? 'first' : 'later' + return escapesLineClaim(line, index - start, position) +} + +function closesHeading(scan: string, index: number): boolean { + if (scan.charAt(index) !== '#' || !/^#+$/.test(scan.slice(index))) return false + return index === 0 || /[ \t]/.test(scan.charAt(index - 1)) +} + +function claimsCharacter(scan: string, index: number, inLinkText: boolean, escaped: ReadonlySet): boolean { + const character = scan.charAt(index) + const rest = scan.slice(index) + if (inLinkText && (character === '[' || character === ']')) return true + if (character === '\\') return asciiPunctuation.test(scan.charAt(index + 1)) + if (character === '&') return startsEntityReference(rest) + if (character === '<') return opensBracketedAutolink(rest) || htmlConstructs.some((construct) => construct.test(rest)) + if (character === ':') return inlineDirective.test(rest) + if (character === '[') return linkOpener.test(rest) + if (character === '`') return opensCodeSpan(scan, index, escaped) + if (character === '*' || character === '_' || character === '~') return opensEmphasis(scan, index, escaped) + return false +} + +function opensCodeSpan(scan: string, index: number, escaped: ReadonlySet): boolean { + if (!startsRun(scan, index, escaped)) return false + const length = runLength(scan, index) + return new RegExp('(?): boolean { + if (!startsRun(scan, index, escaped)) return false + const character = scan.charAt(index) + const length = runLength(scan, index) + const before = index === 0 ? '' : scan.charAt(index - 1) + const after = scan.charAt(index + length) + if (character === '~') return length === 2 && isLeftFlanking(before, after) + if (!isLeftFlanking(before, after)) return false + if (character === '*') return true + return !isRightFlanking(before, after) || isPunctuation(before) +} + +function startsRun(scan: string, index: number, escaped: ReadonlySet): boolean { + if (index === 0 || escaped.has(index - 1)) return true + return scan.charAt(index - 1) !== scan.charAt(index) +} + +function runLength(scan: string, index: number): number { + const character = scan.charAt(index) + let length = 0 + while (scan.charAt(index + length) === character) length += 1 + return length +} + +function isLeftFlanking(before: string, after: string): boolean { + if (isWhitespace(after)) return false + if (!isPunctuation(after)) return true + return isWhitespace(before) || isPunctuation(before) +} + +function isRightFlanking(before: string, after: string): boolean { + if (isWhitespace(before)) return false + if (!isPunctuation(before)) return true + return isWhitespace(after) || isPunctuation(after) +} + +function isPunctuation(character: string): boolean { + return character !== '' && unicodePunctuation.test(character) +} + +function isWhitespace(character: string): boolean { + return character === '' || isUnicodeWhitespace(character) +} + +function isWordCharacter(character: string): boolean { + return character !== '' && !isUnicodeWhitespace(character) && !unicodePunctuation.test(character) +} + +function charAt(text: string, index: number): string { + return index < 0 ? '' : text.charAt(index) +} diff --git a/src/markdown-inline.ts b/src/markdown-inline.ts new file mode 100644 index 0000000..66c8c58 --- /dev/null +++ b/src/markdown-inline.ts @@ -0,0 +1,191 @@ +import type { AdfMark, AdfNode } from './adf-document.ts' +import { assembleInlineLine, type InlineSegment, type LineContainer } from './markdown-escaping.ts' +import { largestNesting } from './nesting.ts' +import { claimsLine, holdsControlCharacter, holdsEntityReference, holdsNullCharacter, isAutolink, isUnicodeWhitespace } from './commonmark-grammar.ts' +import { failure, success, type ConvertErrorPath, type Result } from './result.ts' +import { longestBacktickRun } from './backtick-runs.ts' +import { serializeCanonicalJson } from './canonical-json.ts' + +type InlineContext = { + atBlockEnd: boolean + container: LineContainer + inLinkText: boolean + path: ConvertErrorPath +} + +type InlineRun = { index: number; kind: 'marked'; mark: AdfMark; nodes: AdfNode[] } | { index: number; kind: 'plain'; node: AdfNode } + +const linkAttributes = ['href', 'title'] + +export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result { + const segments = emitRun(nodes, 0, 0, { atBlockEnd: true, container, inLinkText: false, path }) + if (!segments.ok) return segments + const assembled = assembleInlineLine(segments.value, container) + if (assembled.unspellableMark !== undefined) { + return failure('unspellable-mark', `the ${assembled.unspellableMark} spelling cannot open or close where it sits`, path) + } + const line = assembled.line + for (const [index, single] of line.split('\n').entries()) { + if (/^[ \t\v\f]|[ \t\v\f]$/.test(single)) { + return failure('unspellable-whitespace', 'a line begins or ends with whitespace CommonMark strips', path) + } + if (container === 'paragraph' && claimsLine(single, index === 0 ? 'first' : 'later')) { + return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(single)}`, path) + } + } + return success(line) +} + +function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, context: InlineContext): Result { + if (depth > largestNesting) { + return failure('unsupported-node-shape', `the marks nest deeper than the ${largestNesting} levels the emitter carries`, context.path) + } + const runs = inlineRuns(nodes, depth, firstIndex) + const segments: InlineSegment[] = [] + for (const [offset, run] of runs.entries()) { + const runContext = { ...context, atBlockEnd: context.atBlockEnd && offset === runs.length - 1 } + const emitted = run.kind === 'plain' ? emitLeaf(run.node, runContext, run.index) : emitMarkedRun(run.nodes, run.mark, depth, run.index, runContext) + if (!emitted.ok) return emitted + segments.push(...emitted.value) + } + return success(segments) +} + +function inlineRuns(nodes: readonly AdfNode[], depth: number, firstIndex: number): InlineRun[] { + const runs: InlineRun[] = [] + for (const [offset, node] of nodes.entries()) { + const index = firstIndex + offset + const mark = (node.marks ?? [])[depth] + if (mark === undefined) { + runs.push({ index, kind: 'plain', node }) + continue + } + const previous = runs[runs.length - 1] + if (previous?.kind === 'marked' && sameMark(previous.mark, mark)) previous.nodes.push(node) + else runs.push({ index, kind: 'marked', mark, nodes: [node] }) + } + return runs +} + +function nodePath(context: InlineContext, index: number): ConvertErrorPath { + return [...context.path, 'content', index] +} + +function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result { + const path = nodePath(context, index) + if (node.type !== 'hardBreak' && node.type !== 'text') { + return failure('unsupported-node-type', `the canonical form spells no inline node of type ${node.type}`, path) + } + const unspelled = Object.keys(node.attrs ?? {})[0] + if (unspelled !== undefined) { + return failure('unspelled-node-attribute', `the ${node.type} attribute ${unspelled} has no canonical markdown spelling`, path) + } + const types = (node.marks ?? []).map((mark) => mark.type) + if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path) + if (node.type === 'hardBreak') { + if (context.container === 'heading' || context.atBlockEnd) return success([{ kind: 'syntax', text: ':hardBreak{}' }]) + return success([{ kind: 'syntax', text: '\\\n' }]) + } + if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node carries no text', path) + if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node carries content', path) + if (/[\n\r]/.test(node.text)) return failure('unspellable-whitespace', 'a text node holds a newline CommonMark cannot spell', path) + if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path) + return success([{ kind: context.inLinkText ? 'link-text' : 'literal', text: node.text }]) +} + +function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result { + if (mark.type === 'code') return emitCodeSpan(nodes, depth, nodePath(context, index)) + if (mark.type === 'link') return emitLink(nodes, mark, depth, index, context) + const path = nodePath(context, index) + const spelling = mark.type === 'em' ? '_' : mark.type === 'strike' ? '~~' : mark.type === 'strong' ? '**' : undefined + if (spelling === undefined) return failure('unspellable-mark', `no markdown spelling holds the ${mark.type} mark`, path) + if (Object.keys(mark.attrs ?? {}).length > 0) return failure('unspellable-mark', `the ${mark.type} spelling holds no attributes`, path) + const inner = emitRun(nodes, depth + 1, index, context) + if (!inner.ok) return inner + const text = inner.value.map((segment) => segment.text).join('') + if (holdsEdgeWhitespace(text)) return failure('unspellable-whitespace', `the ${mark.type} spelling cannot open or close beside whitespace`, path) + return success([{ kind: 'emphasis-open', mark: mark.type, text: spelling }, ...inner.value, { kind: 'emphasis-close', mark: mark.type, text: spelling }]) +} + +function emitCodeSpan(nodes: readonly AdfNode[], depth: number, path: ConvertErrorPath): Result { + let text = '' + for (const node of nodes) { + if (node.type !== 'text' || typeof node.text !== 'string' || node.text === '') return failure('unspellable-mark', 'a code span holds text nodes only', path) + if ((node.marks ?? []).length !== depth + 1) return failure('unspellable-mark', 'a code span cannot sit inside the marks it carries', path) + text += node.text + } + if (/[\n\r]/.test(text)) return failure('unspellable-mark', 'a code span holds no newline', path) + if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path) + const fence = '`'.repeat(longestBacktickRun(text) + 1) + const padded = needsPadding(text) ? ` ${text} ` : text + return success([{ kind: 'syntax', text: `${fence}${padded}${fence}` }]) +} + +function holdsEdgeWhitespace(text: string): boolean { + return isUnicodeWhitespace(text.charAt(0)) || isUnicodeWhitespace(text.charAt(text.length - 1)) +} + +function needsPadding(text: string): boolean { + if (text.startsWith('`') || text.endsWith('`')) return true + return text.startsWith(' ') && text.endsWith(' ') && /[^ ]/.test(text) +} + +function emitLink(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result { + const path = nodePath(context, index) + const unspelled = Object.keys(mark.attrs ?? {}).find((key) => !linkAttributes.includes(key)) + if (unspelled !== undefined) return failure('unspellable-mark', `the link spelling holds no ${unspelled} attribute`, path) + const href = mark.attrs?.['href'] + const title = mark.attrs?.['title'] + if (typeof href !== 'string') return failure('unsupported-node-shape', 'a link mark carries no href', path) + if (title !== undefined && typeof title !== 'string') return failure('unsupported-node-shape', 'a link title is no string', path) + const node = nodes[0] + const bare = nodes.length === 1 && node !== undefined && node.type === 'text' && node.text === href && (node.marks ?? []).length === depth + 1 + if (bare && title === undefined && isAutolink(href) && !holdsEntityReference(href)) return success([{ kind: 'syntax', text: `<${href}>` }]) + const destination = spellDestination(href, path) + if (!destination.ok) return destination + const spelledTitle = title === undefined ? success('') : spellTitle(title, path) + if (!spelledTitle.ok) return spelledTitle + const inner = emitRun(nodes, depth + 1, index, { ...context, inLinkText: true }) + if (!inner.ok) return inner + return success([{ kind: 'syntax', text: '[' }, ...inner.value, { kind: 'syntax', text: `](${destination.value}${spelledTitle.value})` }]) +} + +function spellDestination(href: string, path: ConvertErrorPath): Result { + if (holdsControlCharacter(href)) return failure('unspellable-link-destination', 'a link destination holds a control character', path) + if (href.includes('\\')) return failure('unspellable-link-destination', 'no canonical escape spells a backslash in a link destination', path) + if (holdsEntityReference(href)) { + return failure('unspellable-link-destination', 'a link destination shaped like an entity reference decodes on the way back', path) + } + if (href.includes(' ')) { + if (/[<>]/.test(href)) { + return failure('unspellable-link-destination', 'no canonical escape spells an angle bracket beside a space in a link destination', path) + } + return success(`<${href}>`) + } + if (href.startsWith('<')) return failure('unspellable-link-destination', 'a bare link destination cannot begin with an angle bracket', path) + if (!balanced(href)) return failure('unspellable-link-destination', 'no canonical escape spells an unbalanced parenthesis in a link destination', path) + return success(href) +} + +function spellTitle(title: string, path: ConvertErrorPath): Result { + if (/["\n\r\\]/.test(title)) { + return failure('unspellable-link-title', 'no canonical escape spells a quote, backslash or newline in a link title', path) + } + if (holdsEntityReference(title)) return failure('unspellable-link-title', 'a link title shaped like an entity reference decodes on the way back', path) + return success(` "${title}"`) +} + +function balanced(href: string): boolean { + let depth = 0 + for (const character of href) { + if (character === '(') depth += 1 + if (character === ')') depth -= 1 + if (depth < 0) return false + } + return depth === 0 +} + +function sameMark(candidate: AdfMark, mark: AdfMark): boolean { + if (candidate.type !== mark.type) return false + return serializeCanonicalJson(candidate.attrs ?? {}, 'compact') === serializeCanonicalJson(mark.attrs ?? {}, 'compact') +} diff --git a/src/nesting.ts b/src/nesting.ts new file mode 100644 index 0000000..1019551 --- /dev/null +++ b/src/nesting.ts @@ -0,0 +1 @@ +export const largestNesting = 500 diff --git a/src/result.ts b/src/result.ts new file mode 100644 index 0000000..89969bc --- /dev/null +++ b/src/result.ts @@ -0,0 +1,38 @@ +export type ConvertErrorCode = + | 'ambiguous-empty-code-block-language' + | 'ambiguous-ordered-list-start' + | 'not-an-adf-document' + | 'reserved-adf-language' + | 'unspellable-adjacent-lists' + | 'unspellable-character' + | 'unspellable-code-block-language' + | 'unspellable-line-start' + | 'unspellable-link-destination' + | 'unspellable-link-title' + | 'unspellable-list-marker' + | 'unspellable-mark' + | 'unspellable-whitespace' + | 'unspelled-block-marks' + | 'unspelled-node-attribute' + | 'unsupported-document-version' + | 'unsupported-heading-level' + | 'unsupported-node-shape' + | 'unsupported-node-type' + +export type ConvertErrorPath = readonly (number | string)[] + +export type ConvertError = { + code: ConvertErrorCode + message: string + path: ConvertErrorPath +} + +export type Result = { error: ConvertError; ok: false } | { ok: true; value: T } + +export function failure(code: ConvertErrorCode, message: string, path: ConvertErrorPath): Result { + return { error: { code, message, path }, ok: false } +} + +export function success(value: T): Result { + return { ok: true, value } +} diff --git a/todo.md b/todo.md index 7fbbb26..3b3d4bb 100644 --- a/todo.md +++ b/todo.md @@ -20,7 +20,7 @@ detail is settled at its own milestone. escape-based, never literal, since pipe cells trim and pad. At `mediaInline`, check real payloads for external-URL support — if it exists, revisit the media section's mid-text-image error and its "no slot" ground. -- [ ] **1d — Corpus start** (§10): checked-in fixtures per spec'd node, in `corpus/`, one +- [x] **1d — Corpus start** (§10): checked-in fixtures per spec'd node, in `corpus/`, one directory per contract kind (`corpus/README.md`). **Blocked on the maintainer** (§15), not to be guessed: Canonical form has no totality guard. Per `@atlaskit/adf-schema` 57.1.0 every block node it spells — `blockquote`, @@ -37,8 +37,8 @@ detail is settled at its own milestone. break §2 silently — href `https://example.com/a)b` emits `[t](https://example.com/a)b)`, read back as href `…/a` plus literal `b)`; title `He said "hi"` emits `[t](u "He said "hi"")`, which holds no title. Two defensible spellings each — angle - brackets or a backslash escape, and for titles `'…'` or `(…)` besides — so §8 leaves the - pick here. **Also blocked**: block separation is unstated for a CommonMark block beside a + brackets or a backslash escape, and for titles `'…'` or `(…)` besides — so §8 leaves + the pick here. **Also blocked**: block separation is unstated for a CommonMark block beside a directive block in a container body — an `expand` whose content is `paragraph` "A" then a `panel` (`panelType` `warning`) holding "B" spells `A` and `:::panel warning` either on consecutive lines or with a blank line between. Two defensible spellings, so §8 leaves the @@ -52,30 +52,59 @@ detail is settled at its own milestone. - [x] **1d3 — Inline nodes and marks**: date, emoji, inlineCard, mediaInline, mention, status; border, subsup, textColor, underline; the content slot's `text` attribute and the `:text{text="…"}` whitespace spelling. - - [ ] **1d4 — Opaque carry** (§3): an unknown node in both positions, the reserved `adf` info - string, and the `codeBlock` whose language is `adf`. - - [ ] **1d5 — Carve-outs and combinations**: the three carve-outs and their escapes, mark runs — - the longest-run rule, attributes included — and the runs a carry breaks, a mark spelling - that cannot open where it sits (`un**-real**istic`; the spec owes the carry a trigger), +- [ ] **2 — `adfToMarkdown`.** First real code. Each sub-item turns one corpus directory green; + the two that have no fixtures yet write them in the same chunk, tests first (§10). + - [ ] **2a — The runner and the CommonMark subset.** The corpus runner: walk + `corpus/round-trip/`, assert `adfToMarkdown` emits each `.md` byte for byte. Decide here + where §10's coverage check lives, and gate that every `corpus/**/*.json` re-serializes to + itself under the library's own canonical serializer — one implementation, keys sorted, two + spellings: two-space indent for the corpus files and the block carry's body, compact for + the inline carry. `commonmark-subset/` green. + - [ ] **2b — Block nodes.** `block-nodes/` green. A nested list that cannot interrupt the block + above it is refused meanwhile, not spelled: the maintainer's answer on tight-versus-blank + separation turns that refusal into an emission. Block separation becomes + `separationBetween(previous, next, container)` here — a boolean cannot hold the third case + `spec/flavour.md` states for two directive blocks in a container body, and the maintainer's + answer on a CommonMark block beside a directive block (1d) drops into the same seam. Give + the emitter's refusals a corpus home while the directories grow: `corpus/unspellable/`, + a `.json` beside the `ConvertErrorCode` it must return, the emitter half of `corpus/errors/`. + - [ ] **2c — Inline nodes and marks.** `inline-nodes/` green. `InlineSegment.kind` splits into + its two axes here — escapability (`attribute` for `:text{text="…"}`, `backslash`, `none`) + and the emphasis role — rather than gaining a third value that means one of each. + - [ ] **2d — The opaque carry** (§3). Fixtures and emitter together, into + `corpus/round-trip/opaque-carry/`: an unknown node in both positions, the reserved `adf` + info string, and the `codeBlock` whose language is `adf`. + - [ ] **2e — Carve-outs and combinations.** Fixtures and emitter together, into + `corpus/round-trip/combinations/`: the three carve-outs and their escapes, mark runs — the + longest-run rule, attributes included — and the runs a carry breaks, a mark spelling that + cannot open where it sits (`un**-real**istic`; the spec owes the carry a trigger), attribute canonicalization, a pipe cell's whitespace edges and `\u007c` for a `|` inside a - quoted attribute value, documents combining nodes rather than isolating one, and a paragraph - line inside a container body shaped like a closing fence (`:::`, `::: x`). - - [ ] **1d6 — Input normalization**: one-way markdown→ADF fixtures, not pairs — setext - headings, indented code, loose lists, `*`/`+` bullets, entity references, soft wraps. - - [ ] **1d7 — Error input**: also one-way, a markdown input per named error, asserting only - that conversion fails — malformed directives, the image gap, a claimed pipe-table line - that does not parse, the content slot, raw HTML with no mapping. Which error each returns - is pinned at milestone 3, where they are named. -- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives, and - gate that every `corpus/**/*.json` re-serializes to itself under the library's own canonical - serializer: one implementation, keys sorted, two spellings — two-space indent for the corpus - files and the block carry's body, compact for the inline carry. -- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML - element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input - is an error result. The CommonMark spec suite runs against it from here (§10), and - `corpus/errors/` gains the error each fixture must return (1d7). + quoted attribute value, documents combining nodes rather than isolating one, and a + paragraph line inside a container body shaped like a closing fence (`:::`, `::: x`). + The gate gains the collision property here: no two corpus documents may emit the same + bytes — one spelling for two documents is a round-trip break no parser can undo, and it is + provable without one. It also settles the emitter's one known approximation: delimiter + flanking is exact, but CommonMark's *matching* — the multiple-of-3 rule and the way a run + splits across several openers — is not modelled. No reachable violation has been found by + hand; the property test is what decides it. +- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component; split it + into sub-items before starting (§15). Fixtures land with the code that reads them: + `corpus/normalization/` (setext, indented code, loose lists, `*`/`+` bullets, entity + references, soft wraps — one-way, the markdown not canonical) and `corpus/errors/` (a + markdown input per named error — malformed directives, the image gap, a claimed pipe-table + line that does not parse, the content slot, raw HTML with no mapping — each with the error + it must return). The raw-HTML element mapping is empty until milestone 6, so at `0.1.0` + every raw-HTML construct in input is an error result. The CommonMark spec suite runs + against it from here (§10). The parser owes `~` the same `can_open`/`can_close` the emitter + assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin. + `src/` gets its hierarchy at the same split — `adf/`, + `markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is + still mechanical. - [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and - 3. Generators emit editor-normal ADF (§2). Real sanitized ADF from live Atlassian APIs lands + 3. Editor-normal (§2) gets its implementation here — `toEditorNormal(doc)` and the equality + the round-trip asserts, which over normalized input is the canonical serializer's compact + spelling — rather than staying spelled inline as `?? []` at every reader. + Generators emit editor-normal ADF (§2). Real sanitized ADF from live Atlassian APIs lands here too (§10), in `corpus/real-payloads/`: an ADF→markdown→ADF check with no expected markdown, the payloads supplied by the maintainer. - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret,