Emitter 2c: the inline node directives, the directive marks and the carried whitespace #13

Merged
lilleman merged 4 commits from inline-nodes into main 2026-08-26 09:49:03 +02:00
16 changed files with 442 additions and 169 deletions
+5
View File
@@ -111,6 +111,11 @@ live Atlassian APIs; property-generated ADF trees; the CommonMark spec suite aga
`Result<T>``{ ok: true; value } | { ok: false; error: ConvertError }` — nothing throws.
`try/catch` only wrapped tightly around a call that genuinely throws, converted to a result on
the spot.
- Only the hard break's inline segment holds a raw newline — every other spelling escapes one or
refuses it — which is how the whitespace carry finds a line edge.
- A readable spelling tried ahead of a general one — the image, the pipe table, a pipe cell —
returns `string | undefined`, never a `Result`: any failure is the fallback signal, and the
general form owns the refusal. Refusing there refuses a document the general form spells.
- Nothing recurses unbounded: the guards walk iteratively, and blocks, marks and attribute values
are all held to 500 levels, so a deep document is a `Result` rather than the stack overflow that
waits near 2000.
+2 -2
View File
@@ -3,8 +3,8 @@
Lossless conversion between **Atlassian Document Format** (ADF), an extended markdown flavour, and
an HTML dialect.
**Status: pre-release — `adfToMarkdown` emits the CommonMark subset and the block nodes, nothing
else is built.**
**Status: pre-release — `adfToMarkdown` emits every node the flavour spells but the opaque carry,
nothing else is built.**
Plan: `todo.md`. Decisions: `AGENTS.md`. The flavour's grammar:
[`spec/flavour.md`](spec/flavour.md).
+1 -1
View File
@@ -13,7 +13,7 @@
"node": ">=24"
},
"scripts": {
"test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=94 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"",
"test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=95 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"",
"typecheck": "tsc --noEmit"
},
"devDependencies": {
+10 -7
View File
@@ -63,7 +63,9 @@ literal bracket. Whitespace at either edge of `[content]`, space or tab, is part
survives inline parsing. Each section below says whether content is required. `:` opens a
directive only when the name is followed immediately by `[` or `{`, and `{attrs}` must follow
`]` (or the name) with no gap — anything else (`10:30`, `:smile:`, a stray `{…}` in text) is
literal text.
literal text. An inline directive binds as a unit before bracket matching, the way a code span
does: a `]` or `(` inside its `{attrs}` is the directive's, never the enclosing content's, and a
`(` after its closing `]` opens no link.
**Container block**:
@@ -99,8 +101,9 @@ form, and omits empty `{attrs}` except where the `{` itself claims the directive
(`:hardBreak{}`).
**Escaping**: the emitter backslash-escapes whatever literal text would otherwise parse as
directive syntax — the leading `:` of a would-be directive, `]` inside content; outside code
spans and code blocks, a backslash before `:` in input yields a literal colon.
directive syntax — the leading `:` of a would-be directive, `]` inside content, a `{` right
after a directive's closing `]`, which would otherwise be read as the attributes it has none
of; outside code spans and code blocks, a backslash before `:` in input yields a literal colon.
**Malformed directives are error results**, named: an unclosed container at end of input, a body
fence line of the container's length or longer, a bare colon-run line outside any container or
@@ -214,10 +217,10 @@ One header row plus plain inline cells is a pipe table; anything richer is the d
(AGENTS.md §4). Precisely: a table emits as a pipe table exactly when the `table`, every row
and every cell carry no attrs and no marks, the first row is all `tableHeader` and the rest all
`tableCell`, every row has the header's cell count, and every cell holds exactly one attr-less,
mark-less paragraph — an empty cell holds one empty paragraph — with no `|` in a code span,
link destination or link title: backslash escapes are inert in everything the inline layer spells
as syntax, so pipe form cannot spell that pipe and the table takes the directive form. A pipe table
parses back to exactly that shape.
mark-less paragraph — an empty cell holds one empty paragraph — with no `|` anywhere the
inline layer spells as syntax — a code span, a link destination or title, an inline directive's
attributes: backslash escapes are inert there, so pipe form cannot spell that pipe and the table
takes the directive form. A pipe table parses back to exactly that shape.
```
| Part | Qty |
+2
View File
@@ -4,6 +4,8 @@ export type AdfAttributes = { [key: string]: JsonValue }
export type AttributeKind = 'boolean' | 'json' | 'number' | 'string'
export type AttributeVocabulary = Readonly<Record<string, AttributeKind>>
export type AdfMark = {
attrs?: AdfAttributes
type: string
+84 -8
View File
@@ -29,7 +29,7 @@ test('names the node a refusal came from', () => {
const unspellable: AdfNode = { attrs: { localId: 'a' }, type: 'paragraph' }
const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }, { content: [unspellable], type: 'listItem' }], type: 'bulletList' }
assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }), list))), ['content', 1, 'content', 1, 'content', 0])
assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, { type: 'mention' })))), ['content', 0, 'content', 1])
assert.deepEqual(path(adfToMarkdown(document(paragraph({ text: 'x', type: 'text' }, { type: 'blockCard' })))), ['content', 0, 'content', 1])
assert.deepEqual(path(adfToMarkdown({ type: 'doc', version: 2 })), [])
})
@@ -93,17 +93,15 @@ test('refuses a link attribute no markdown spelling holds', () => {
})
test('refuses a mark the canonical spellings cannot nest', () => {
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'underline' }], text: 'x', type: 'text' })))), 'unspellable-mark')
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'annotation' }], text: 'x', type: 'text' })))), 'unspellable-mark')
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }, { type: 'strong' }], text: 'x', type: 'text' })))), 'unspellable-mark')
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { colour: 'red' }, type: 'em' }], text: 'x', type: 'text' })))), 'unspellable-mark')
})
test('refuses whitespace CommonMark cannot hold', () => {
assert.equal(code(adfToMarkdown(document(paragraph({ text: ' lead', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'trail ', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\nb', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ text: '\fa', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'em' }], text: 'x ', type: 'text' }, { text: 'y', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ type: 'em' }], text: 'x ', type: 'text' }, { text: 'y', type: 'text' })))), 'unspellable-mark')
})
test('refuses a line whose start block parsing would claim', () => {
@@ -118,7 +116,7 @@ test('refuses two adjacent lists of the same kind', () => {
test('refuses a node type the canonical form does not cover', () => {
assert.equal(code(adfToMarkdown(document({ type: 'blockCard' }))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(document({ type: 'toString' }))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(document(paragraph({ type: 'mention' })))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(document(paragraph({ type: 'blockCard' })))), 'unsupported-node-type')
})
test('refuses a node whose content model the canonical form cannot emit', () => {
@@ -230,6 +228,9 @@ test('escapes a literal delimiter that would merge with an emitted one', () => {
assert.equal(emitted({ text: '`', type: 'text' }, marked('x', { type: 'code' })), '\\``x`\n')
assert.equal(emitted(marked('x', { type: 'code' }), { text: '`', type: 'text' }), '`x`\\`\n')
assert.equal(emitted({ text: '!', type: 'text' }, marked('x', { attrs: { href: 'https://example.com/' }, type: 'link' })), '\\![x](https://example.com/)\n')
assert.equal(emitted(marked('x', { type: 'underline' }), { text: '{}', type: 'text' }), ':underline[x]\\{}\n')
assert.equal(emitted({ attrs: { text: '' }, type: 'status' }, { text: '{color=red}', type: 'text' }), ':status[]\\{color=red}\n')
assert.equal(emitted(marked('x', { attrs: { href: 'https://example.com/' }, type: 'link' }), { text: '{}', type: 'text' }), '[x](https://example.com/){}\n')
})
test('escapes a hyphen underline a hard break would expose', () => {
@@ -372,10 +373,85 @@ test('spells a table as a pipe table only where every row and cell is plain', ()
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader'))))))
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', text('a'), text('b')))))))
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { attrs: { localId: 'a' }, type: 'paragraph' }))))), 'unspelled-node-attribute')
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ type: 'blockCard' }], type: 'paragraph' }))))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: '\fa', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace')
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' }))))))
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace')
assert.equal(markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), '| :text{text=" "}a |\n| --- |\n')
const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' })))
assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/?x|y' }, type: 'link' }))))
assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/', title: 'a|b' }, type: 'link' }))))
const fallsBack = (node: AdfNode): boolean => directive(adfToMarkdown(table(row(cell('tableHeader', { content: [node], type: 'paragraph' })))))
assert.ok(fallsBack({ marks: [{ type: 'code' }], text: 'a|b', type: 'text' }))
assert.ok(fallsBack({ attrs: { style: 'a|b' }, type: 'status' }))
assert.equal(markdown(adfToMarkdown(marked({ attrs: { href: 'https://example.com/x' }, type: 'link' }))), '| [l](https://example.com/x) |\n| --- |\n')
})
test('spells an inline node as a directive with its content slot and attributes', () => {
const emitted = (node: AdfNode): string => markdown(adfToMarkdown(document(paragraph(node))))
assert.equal(emitted({ attrs: { timestamp: '1756080000000' }, type: 'date' }), ':date{timestamp=1756080000000}\n')
assert.equal(emitted({ type: 'mention' }), ':mention{}\n')
assert.equal(emitted({ attrs: { text: '' }, type: 'status' }), ':status[]\n')
assert.equal(emitted({ attrs: { color: 'yellow', text: 'In review' }, type: 'status' }), ':status[In review]{color=yellow}\n')
assert.equal(emitted({ attrs: { id: '1f389', text: 'a]b' }, type: 'emoji' }), ':emoji[a\\]b]{id=1f389}\n')
assert.equal(emitted({ attrs: { data: { url: 'https://example.com/' } }, type: 'inlineCard' }), ':inlineCard{data="{\\"url\\":\\"https://example.com/\\"}"}\n')
assert.equal(emitted({ attrs: { height: 24 }, type: 'mediaInline' }), ':mediaInline{height=24}\n')
})
test('refuses an inline node attribute no section spells', () => {
const refused = (node: AdfNode): string => code(adfToMarkdown(document(paragraph(node))))
assert.equal(refused({ attrs: { rounded: true }, type: 'status' }), 'unspelled-node-attribute')
assert.equal(refused({ attrs: { toString: 'x' }, type: 'status' }), 'unspelled-node-attribute')
assert.equal(refused({ attrs: { color: 4 }, type: 'status' }), 'unsupported-node-shape')
assert.equal(refused({ attrs: { width: '2' }, type: 'mediaInline' }), 'unsupported-node-shape')
})
test('refuses the content and slot an inline directive has no room for', () => {
const refused = (node: AdfNode): string => code(adfToMarkdown(document(paragraph(node, { text: 'y', type: 'text' }))))
assert.equal(refused({ content: [{ text: 'x', type: 'text' }], type: 'status' }), 'unsupported-node-shape')
assert.equal(refused({ text: 'x', type: 'status' }), 'unsupported-node-shape')
assert.equal(refused({ content: [{ text: 'x', type: 'text' }], type: 'hardBreak' }), 'unsupported-node-shape')
assert.equal(refused({ text: 'x', type: 'hardBreak' }), 'unsupported-node-shape')
assert.equal(refused({ attrs: { text: 4 }, type: 'status' }), 'unsupported-node-shape')
assert.equal(refused({ attrs: { text: 'a\nb' }, type: 'status' }), 'unspellable-whitespace')
assert.equal(refused({ attrs: { text: 'a\u0000b' }, type: 'status' }), 'unspellable-character')
})
test('spells the directive marks around the longest run they cover', () => {
const marked = (text: string, ...marks: AdfMark[]): AdfNode => ({ marks, text, type: 'text' })
const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content))))
const underline: AdfMark = { type: 'underline' }
assert.equal(emitted(marked('x', underline)), ':underline[x]\n')
assert.equal(emitted(marked('a', underline), marked('b', underline)), ':underline[ab]\n')
assert.equal(emitted(marked('x', { attrs: { type: 'sub' }, type: 'subsup' })), ':subsup[x]{type=sub}\n')
assert.equal(emitted(marked('x', { attrs: { color: '#ae2e24' }, type: 'textColor' })), ':textColor[x]{color="#ae2e24"}\n')
assert.equal(emitted(marked('x', { attrs: { color: '#091e42', size: 2 }, type: 'border' })), ':border[x]{color="#091e42" size=2}\n')
assert.equal(emitted(marked('x', { type: 'em' }, underline)), '_:underline[x]_\n')
assert.equal(emitted(marked('x', underline, { type: 'em' })), ':underline[_x_]\n')
assert.equal(emitted(marked('a', underline), { marks: [underline], type: 'hardBreak' }, marked('b', underline)), ':underline[a:hardBreak{}b]\n')
const link: AdfMark = { attrs: { href: 'https://example.com/' }, type: 'link' }
assert.equal(
emitted(marked('a', underline, link), { marks: [underline, link], type: 'hardBreak' }, marked('b', underline, link)),
':underline[[a:hardBreak{}b](https://example.com/)]\n',
)
assert.equal(
emitted(marked('a', link), { marks: [link], type: 'hardBreak' }, marked('b', link)),
'[a\\\nb](https://example.com/)\n',
)
})
test('refuses a mark directive attribute no spelling holds', () => {
const refused = (mark: AdfMark): string => code(adfToMarkdown(document(paragraph({ marks: [mark], text: 'x', type: 'text' }))))
assert.equal(refused({ attrs: { width: 2 }, type: 'border' }), 'unspellable-mark')
assert.equal(refused({ attrs: { size: '2' }, type: 'border' }), 'unspellable-mark')
})
test('carries whitespace CommonMark strips in the reserved text directive', () => {
const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content))))
assert.equal(emitted({ text: ' lead', type: 'text' }), ':text{text=" "}lead\n')
assert.equal(emitted({ text: 'trail ', type: 'text' }), 'trail:text{text=" "}\n')
assert.equal(emitted({ text: 'a\nb', type: 'text' }), 'a:text{text="\\n"}b\n')
assert.equal(emitted({ text: '\t', type: 'text' }), ':text{text="\\t"}\n')
assert.equal(emitted({ text: 'a ', type: 'text' }, { type: 'hardBreak' }, { text: ' b', type: 'text' }), 'a:text{text=" "}\\\n:text{text=" "}b\n')
assert.equal(emitted({ marks: [{ type: 'em' }], text: ' a ', type: 'text' }), '_:text{text=" "}a:text{text=" "}_\n')
assert.equal(markdown(adfToMarkdown(document({ attrs: { level: 1 }, content: [{ text: 'x ', type: 'text' }], type: 'heading' }))), '# x:text{text=" "}\n')
})
+11 -7
View File
@@ -2,9 +2,9 @@ import type { AdfDocument, AdfNode } from './adf-document.ts'
import type { BlockDirective } from './block-directives.ts'
import type { JsonValue } from './json-value.ts'
import { blockDirective, spellDirectiveHeader } from './block-directives.ts'
import { emitImage } from './markdown-image.ts'
import { tryImage } from './markdown-image.ts'
import { emitInlineLine } from './markdown-inline.ts'
import { emitPipeTable } from './markdown-pipe-table.ts'
import { tryPipeTable } from './markdown-pipe-table.ts'
import { failure, success, type ConvertErrorPath, type Result } from './result.ts'
import { holdsNullCharacter, isThematicBreak } from './commonmark-grammar.ts'
import { isAdfDocument } from './adf-document.ts'
@@ -98,7 +98,11 @@ function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result
function commonMarkLine(text: Result<string>): Result<EmittedBlock> {
if (!text.ok) return text
return success({ fenceColons: 0, spelling: 'commonmark', text: text.value })
return success(commonMarkText(text.value))
}
function commonMarkText(text: string): EmittedBlock {
return { fenceColons: 0, spelling: 'commonmark', text }
}
function commonMarkContainer(body: Result<EmittedBody>): Result<EmittedBlock> {
@@ -131,15 +135,15 @@ function emitInlineBody(content: readonly AdfNode[], path: ConvertErrorPath): Re
}
function emitMediaSingle(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
const image = emitImage(node, path)
const image = tryImage(node, path)
if (image === undefined) return emitDirectiveBlock(node, directive, path, depth)
return commonMarkLine(image)
return success(commonMarkText(image))
}
function emitTable(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
const pipe = emitPipeTable(node, path)
const pipe = tryPipeTable(node, path)
if (pipe === undefined) return emitDirectiveBlock(node, directive, path, depth)
return commonMarkLine(pipe)
return success(commonMarkText(pipe))
}
function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result<EmittedBody> {
+23 -30
View File
@@ -1,15 +1,15 @@
import type { AdfMark, AdfNode, AttributeKind } from './adf-document.ts'
import type { AdfMark, AdfNode, AttributeVocabulary } from './adf-document.ts'
import type { JsonValue } from './json-value.ts'
import { failure, success, type ConvertErrorPath, type Result } from './result.ts'
import { isBareToken, spellAttributeValue, spellAttributes, spellJsonAttribute } from './directive-attributes.ts'
import { attributeFailure, isBareToken, spellAttributes, spellJsonAttribute, vocabularyPairs } from './directive-attributes.ts'
export type BlockDirective = {
argument?: string
attributes: Readonly<Record<string, AttributeKind>>
attributes: AttributeVocabulary
body: 'block' | 'inline' | 'none'
}
const cellAttributes: Readonly<Record<string, AttributeKind>> = {
const cellAttributes: AttributeVocabulary = {
background: 'string',
colspan: 'number',
colwidth: 'json',
@@ -18,9 +18,9 @@ const cellAttributes: Readonly<Record<string, AttributeKind>> = {
valign: 'string',
}
const expandAttributes: Readonly<Record<string, AttributeKind>> = { localId: 'string', title: 'string' }
const expandAttributes: AttributeVocabulary = { localId: 'string', title: 'string' }
const extensionAttributes: Readonly<Record<string, AttributeKind>> = {
const extensionAttributes: AttributeVocabulary = {
extensionKey: 'string',
extensionType: 'string',
layout: 'string',
@@ -29,9 +29,9 @@ const extensionAttributes: Readonly<Record<string, AttributeKind>> = {
text: 'string',
}
const itemAttributes: Readonly<Record<string, AttributeKind>> = { localId: 'string' }
const itemAttributes: AttributeVocabulary = { localId: 'string' }
const mediaAttributes: Readonly<Record<string, AttributeKind>> = {
const mediaAttributes: AttributeVocabulary = {
alt: 'string',
collection: 'string',
height: 'number',
@@ -43,7 +43,7 @@ const mediaAttributes: Readonly<Record<string, AttributeKind>> = {
width: 'number',
}
const syncBlockAttributes: Readonly<Record<string, AttributeKind>> = { localId: 'string', resourceId: 'string' }
const syncBlockAttributes: AttributeVocabulary = { localId: 'string', resourceId: 'string' }
const blockDirectives: Readonly<Record<string, BlockDirective>> = {
blockTaskItem: { argument: 'state', attributes: itemAttributes, body: 'block' },
@@ -81,30 +81,23 @@ export function blockDirective(type: string): BlockDirective | undefined {
}
export function spellDirectiveHeader(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath): Result<string> {
const pairs: [string, string][] = []
let argument = ''
for (const [key, value] of Object.entries(node.attrs ?? {})) {
if (key === directive.argument) {
if (typeof value !== 'string' || !isBareToken(value)) {
return failure('unspelled-node-attribute', `the ${node.type} attribute ${key} holds no bare token the arg slot spells`, path)
}
argument = ` ${value}`
continue
}
const kind = attributeKind(directive, key)
if (kind === undefined) return failure('unspelled-node-attribute', `the ${node.type} attribute ${key} has no canonical markdown spelling`, path)
const spelled = spellAttributeValue(value, kind)
if (spelled === undefined) return failure('unsupported-node-shape', `the ${node.type} attribute ${key} holds no ${kind}`, path)
pairs.push([key, spelled])
}
const argument = spellArgument(node, directive, path)
if (!argument.ok) return argument
const spelled = vocabularyPairs(node.attrs ?? {}, directive.attributes, directive.argument)
if (spelled.fault !== undefined) return attributeFailure(node.type, spelled.fault, path)
const marks = node.marks ?? []
if (marks.length > 0) pairs.push(['marks', spellJsonAttribute(markValues(marks))])
const attributes = spellAttributes(pairs)
return success(`${node.type}${argument}${attributes === '' ? '' : ` ${attributes}`}`)
if (marks.length > 0) spelled.pairs.push(['marks', spellJsonAttribute(markValues(marks))])
const attributes = spellAttributes(spelled.pairs)
return success(`${node.type}${argument.value}${attributes === '' ? '' : ` ${attributes}`}`)
}
function attributeKind(directive: BlockDirective, key: string): AttributeKind | undefined {
return Object.hasOwn(directive.attributes, key) ? directive.attributes[key] : undefined
function spellArgument(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath): Result<string> {
const value = directive.argument === undefined ? undefined : node.attrs?.[directive.argument]
if (value === undefined) return success('')
if (typeof value !== 'string' || !isBareToken(value)) {
return failure('unspelled-node-attribute', `the ${node.type} attribute ${directive.argument} holds no bare token the arg slot spells`, path)
}
return success(` ${value}`)
}
function markValues(marks: readonly AdfMark[]): JsonValue {
+5 -11
View File
@@ -13,8 +13,7 @@ const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus')
const roundTripRoot = join(corpusRoot, 'round-trip')
const unspellableRoot = join(corpusRoot, 'unspellable')
const emittingDirectories = ['block-nodes', 'commonmark-subset']
const pendingDirectories = ['inline-nodes']
const emittingDirectories = ['block-nodes', 'commonmark-subset', 'inline-nodes']
function directoryNames(root: string): string[] {
return readdirSync(root, { withFileTypes: true })
@@ -41,21 +40,16 @@ function corpusJsonPaths(): string[] {
.sort()
}
test('every round-trip directory is either emitting or explicitly pending', () => {
assert.deepEqual(directoryNames(roundTripRoot), [...emittingDirectories, ...pendingDirectories].sort())
test('every round-trip directory emits', () => {
assert.deepEqual(directoryNames(roundTripRoot), [...emittingDirectories].sort())
})
for (const directory of [...emittingDirectories, ...pendingDirectories].sort()) {
test(`${directory} pairs every .json with a .md`, () => {
assert.deepEqual(fixtureNames(directory, '.json'), fixtureNames(directory, '.md'))
})
}
for (const directory of emittingDirectories) {
const names = [...new Set([...fixtureNames(directory, '.json'), ...fixtureNames(directory, '.md')])].sort()
test(`${directory} holds fixtures`, () => {
test(`${directory} pairs every .json with a .md`, () => {
assert.ok(names.length > 0, `${directory} is expected to emit but holds no fixture pairs`)
assert.deepEqual(fixtureNames(directory, '.json'), fixtureNames(directory, '.md'))
})
for (const name of names) {
+27 -4
View File
@@ -1,13 +1,36 @@
import type { AttributeKind } from './adf-document.ts'
import type { AdfAttributes, AttributeKind, AttributeVocabulary } from './adf-document.ts'
import type { JsonValue } from './json-value.ts'
import { failure, type ConvertErrorPath, type Result } from './result.ts'
import { serializeCanonicalJson } from './canonical-json.ts'
export type AttributeFault = { key: string; kind: AttributeKind | undefined }
export type SpelledPairs = { fault: AttributeFault; pairs?: undefined } | { fault?: undefined; pairs: [string, string][] }
const bareToken = /^[A-Za-z0-9_-]+$/
export function isBareToken(text: string): boolean {
return bareToken.test(text)
}
export function attributeFailure<T>(type: string, fault: AttributeFault, path: ConvertErrorPath): Result<T> {
if (fault.kind === undefined) return failure('unspelled-node-attribute', `the ${type} attribute ${fault.key} has no canonical markdown spelling`, path)
return failure('unsupported-node-shape', `the ${type} attribute ${fault.key} holds no ${fault.kind}`, path)
}
export function vocabularyPairs(attrs: AdfAttributes, vocabulary: AttributeVocabulary, slot: string | undefined): SpelledPairs {
const pairs: [string, string][] = []
for (const [key, value] of Object.entries(attrs)) {
if (key === slot) continue
const kind = Object.hasOwn(vocabulary, key) ? vocabulary[key] : undefined
if (kind === undefined) return { fault: { key, kind: undefined } }
const spelled = spellAttributeValue(value, kind)
if (spelled === undefined) return { fault: { key, kind } }
pairs.push([key, spelled])
}
return { pairs }
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => (left < right ? -1 : 1)).map(([key, value]) => `${key}=${value}`)
@@ -17,15 +40,15 @@ export function spellAttributes(pairs: readonly (readonly [string, string])[]):
export function spellAttributeValue(value: JsonValue, kind: AttributeKind): string | undefined {
if (kind === 'json') return spellJsonAttribute(value)
if (kind === 'boolean') return typeof value === 'boolean' ? `${value}` : undefined
if (kind === 'number') return typeof value === 'number' ? spell(JSON.stringify(value)) : undefined
return typeof value === 'string' ? spell(value) : undefined
if (kind === 'number') return typeof value === 'number' ? spellStringAttribute(JSON.stringify(value)) : undefined
return typeof value === 'string' ? spellStringAttribute(value) : undefined
}
export function spellJsonAttribute(value: JsonValue): string {
return quote(serializeCanonicalJson(value, 'compact'))
}
function spell(text: string): string {
export function spellStringAttribute(text: string): string {
return isBareToken(text) ? text : quote(text)
}
+62
View File
@@ -0,0 +1,62 @@
import type { AdfMark, AdfNode, AttributeVocabulary } from './adf-document.ts'
import type { AttributeFault } from './directive-attributes.ts'
import { failure, success, type ConvertErrorPath, type Result } from './result.ts'
import { attributeFailure, spellAttributes, vocabularyPairs } from './directive-attributes.ts'
export type InlineDirective = {
attributes: AttributeVocabulary
slot?: string
}
const inlineDirectives: Readonly<Record<string, InlineDirective>> = {
date: { attributes: { localId: 'string', timestamp: 'string' } },
emoji: { attributes: { id: 'string', localId: 'string', shortName: 'string' }, slot: 'text' },
inlineCard: { attributes: { data: 'json', localId: 'string', url: 'string' } },
mediaInline: {
attributes: {
alt: 'string',
collection: 'string',
data: 'json',
height: 'number',
id: 'string',
localId: 'string',
occurrenceKey: 'string',
type: 'string',
width: 'number',
},
},
mention: { attributes: { accessLevel: 'string', id: 'string', localId: 'string', userType: 'string' }, slot: 'text' },
status: { attributes: { color: 'string', localId: 'string', style: 'string' }, slot: 'text' },
}
const markDirectives: Readonly<Record<string, AttributeVocabulary>> = {
border: { color: 'string', size: 'number' },
subsup: { type: 'string' },
textColor: { color: 'string' },
underline: {},
}
export function inlineDirective(type: string): InlineDirective | undefined {
return Object.hasOwn(inlineDirectives, type) ? inlineDirectives[type] : undefined
}
export function markDirective(type: string): AttributeVocabulary | undefined {
return Object.hasOwn(markDirectives, type) ? markDirectives[type] : undefined
}
export function spellInlineNodeAttributes(node: AdfNode, directive: InlineDirective, path: ConvertErrorPath): Result<string> {
const spelled = vocabularyPairs(node.attrs ?? {}, directive.attributes, directive.slot)
if (spelled.fault !== undefined) return attributeFailure(node.type, spelled.fault, path)
return success(spellAttributes(spelled.pairs))
}
export function spellMarkAttributes(mark: AdfMark, vocabulary: AttributeVocabulary, path: ConvertErrorPath): Result<string> {
const spelled = vocabularyPairs(mark.attrs ?? {}, vocabulary, undefined)
if (spelled.fault !== undefined) return markFailure(mark.type, spelled.fault, path)
return success(spellAttributes(spelled.pairs))
}
function markFailure<T>(type: string, fault: AttributeFault, path: ConvertErrorPath): Result<T> {
if (fault.kind === undefined) return failure('unspellable-mark', `the ${type} spelling holds no ${fault.key} attribute`, path)
return failure('unspellable-mark', `the ${type} attribute ${fault.key} holds no ${fault.kind}`, path)
}
+30 -25
View File
@@ -1,8 +1,12 @@
import { escapesLineClaim, isUnicodeWhitespace, opensBracketedAutolink, startsEntityReference, type LinePosition } from './commonmark-grammar.ts'
export type EmphasisRole = 'close' | 'open'
export type InlineEscaping = 'attribute' | 'backslash' | 'bracketed' | 'none'
export type InlineSegment =
| { kind: 'emphasis-close' | 'emphasis-open'; mark: string; text: string }
| { kind: 'link-text' | 'literal' | 'syntax'; text: string }
| { emphasis: EmphasisRole; escaping: 'none'; mark: string; text: string }
| { emphasis?: undefined; escaping: InlineEscaping; text: string }
export type AssembledLine = { line: string; unspellableMark: string | undefined }
@@ -14,7 +18,7 @@ const delimiters = ['*', '_', '`', '~']
const asciiPunctuation = /[!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~]/
const htmlConstructs = [/^<[!?]/, /^<\/?[A-Za-z][A-Za-z0-9-]*(?:[\s/>]|$)/, /^<[^\s<>@]+@[^\s<>@]+>/]
const inlineDirective = /^:[a-z][A-Za-z0-9]*[[{]/
const inlineDirectiveOpener = /^:[a-z][A-Za-z0-9]*[[{]/
const linkOpener = /\](?=[([:])/
const unicodePunctuation = /[\p{P}\p{S}]/u
@@ -36,8 +40,8 @@ function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] {
for (let index = 0; index < resolved.length; index += 1) {
const segment = resolved[index]
if (segment === undefined) continue
if (segment.kind === 'emphasis-open') open.push(index)
if (segment.kind !== 'emphasis-close') continue
if (segment.emphasis === 'open') open.push(index)
if (segment.emphasis !== 'close') continue
const openerIndex = open.pop()
const opener = openerIndex === undefined ? undefined : resolved[openerIndex]
if (openerIndex === undefined || opener === undefined) continue
@@ -53,15 +57,15 @@ function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] {
function escape(segments: readonly InlineSegment[], container: LineContainer): AssembledLine {
const scan = segments.map((segment) => segment.text).join('')
const kinds: InlineSegment['kind'][] = []
for (const segment of segments) for (let index = 0; index < segment.text.length; index += 1) kinds.push(segment.kind)
const escapings: InlineEscaping[] = []
for (const segment of segments) for (let index = 0; index < segment.text.length; index += 1) escapings.push(segment.escaping)
const escaped = new Set<number>()
const placements: number[] = []
let output = ''
for (let index = 0; index < scan.length; index += 1) {
const kind = kinds[index]
const escapable = kind === 'literal' || kind === 'link-text'
if (escapable && (mergesWithSyntax(scan, kinds, index) || opensConstruct(scan, index, kind === 'link-text', container, escaped))) {
const escaping = escapings[index]
const escapable = escaping === 'backslash' || escaping === 'bracketed'
if (escapable && (mergesWithSyntax(scan, escapings, index) || opensConstruct(scan, index, escaping === 'bracketed', container, escaped))) {
output += '\\'
escaped.add(index)
}
@@ -87,8 +91,8 @@ function delimiterRuns(segments: readonly InlineSegment[], placements: readonly
for (const segment of segments) {
const start = placements[cursor] ?? 0
cursor += segment.text.length
if (segment.kind !== 'emphasis-close' && segment.kind !== 'emphasis-open') continue
const closes = segment.kind === 'emphasis-close'
if (segment.emphasis === undefined) continue
const closes = segment.emphasis === 'close'
const end = start + segment.text.length
const previous = runs[runs.length - 1]
if (previous !== undefined && previous.end === start && previous.character === segment.text.charAt(0)) {
@@ -108,28 +112,29 @@ function delimiterRuns(segments: readonly InlineSegment[], placements: readonly
return runs
}
function mergesWithSyntax(scan: string, kinds: readonly (InlineSegment['kind'] | undefined)[], index: number): boolean {
function mergesWithSyntax(scan: string, escapings: readonly (InlineEscaping | undefined)[], index: number): boolean {
const character = scan.charAt(index)
if (character === '!') return scan.charAt(index + 1) === '[' && isSyntax(kinds[index + 1])
if (character === '!') return scan.charAt(index + 1) === '[' && isSyntax(escapings[index + 1])
if (character === '{') return scan.charAt(index - 1) === ']' && isSyntax(escapings[index - 1])
if (!delimiters.includes(character)) return false
return touchesSyntax(scan, kinds, index, -1) || touchesSyntax(scan, kinds, index, 1)
return touchesSyntax(scan, escapings, index, -1) || touchesSyntax(scan, escapings, index, 1)
}
function touchesSyntax(scan: string, kinds: readonly (InlineSegment['kind'] | undefined)[], index: number, step: number): boolean {
function touchesSyntax(scan: string, escapings: readonly (InlineEscaping | undefined)[], index: number, step: number): boolean {
const character = scan.charAt(index)
let cursor = index + step
while (scan.charAt(cursor) === character && !isSyntax(kinds[cursor])) cursor += step
return scan.charAt(cursor) === character && isSyntax(kinds[cursor])
while (scan.charAt(cursor) === character && !isSyntax(escapings[cursor])) cursor += step
return scan.charAt(cursor) === character && isSyntax(escapings[cursor])
}
function isSyntax(kind: InlineSegment['kind'] | undefined): boolean {
return kind === 'emphasis-close' || kind === 'emphasis-open' || kind === 'syntax'
function isSyntax(escaping: InlineEscaping | undefined): boolean {
return escaping === 'attribute' || escaping === 'none'
}
function opensConstruct(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
function opensConstruct(scan: string, index: number, inBrackets: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
if (container === 'heading' && closesHeading(scan, index)) return true
if (container === 'paragraph' && claimsLineStart(scan, index)) return true
return claimsCharacter(scan, index, inLinkText, container, escaped)
return claimsCharacter(scan, index, inBrackets, container, escaped)
}
function claimsLineStart(scan: string, index: number): boolean {
@@ -145,15 +150,15 @@ function closesHeading(scan: string, index: number): boolean {
return index === 0 || /[ \t]/.test(scan.charAt(index - 1))
}
function claimsCharacter(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
function claimsCharacter(scan: string, index: number, inBrackets: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
const character = scan.charAt(index)
const rest = scan.slice(index)
if (inLinkText && (character === '[' || character === ']')) return true
if (inBrackets && (character === '[' || character === ']')) return true
if (character === '|') return container === 'table-cell'
if (character === '\\') return asciiPunctuation.test(scan.charAt(index + 1))
if (character === '&') return startsEntityReference(rest)
if (character === '<') return opensBracketedAutolink(rest) || htmlConstructs.some((construct) => construct.test(rest))
if (character === ':') return inlineDirective.test(rest)
if (character === ':') return inlineDirectiveOpener.test(rest)
if (character === '[') return linkOpener.test(rest)
if (character === '`') return opensCodeSpan(scan, index, escaped)
if (character === '*' || character === '_' || character === '~') return opensEmphasis(scan, index, escaped)
+4 -5
View File
@@ -1,16 +1,15 @@
import type { AdfNode } from './adf-document.ts'
import type { ConvertErrorPath, Result } from './result.ts'
import { emitImageLine } from './markdown-inline.ts'
import type { ConvertErrorPath } from './result.ts'
import { serializeCanonicalJson } from './canonical-json.ts'
import { tryImageLine } from './markdown-inline.ts'
const centeredMediaSingle = '{"layout":"center"}'
const imageAttributes = ['alt', 'type', 'url']
export function emitImage(node: AdfNode, path: ConvertErrorPath): Result<string> | undefined {
export function tryImage(node: AdfNode, path: ConvertErrorPath): string | undefined {
const image = imageShape(node)
if (image === undefined) return undefined
const line = emitImageLine(image.alt, image.url, [...path, 'content', 0])
return line.ok ? line : undefined
return tryImageLine(image.alt, image.url, [...path, 'content', 0])
}
function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined {
+143 -35
View File
@@ -1,37 +1,53 @@
import type { AdfMark, AdfNode } from './adf-document.ts'
import { assembleInlineLine, type InlineSegment, type LineContainer } from './markdown-escaping.ts'
import type { InlineDirective } from './inline-directives.ts'
import { assembleInlineLine, type InlineEscaping, type InlineSegment, type LineContainer } from './markdown-escaping.ts'
import { inlineDirective, markDirective, spellInlineNodeAttributes, spellMarkAttributes } from './inline-directives.ts'
import { largestNesting } from './nesting.ts'
import { claimsLine, holdsControlCharacter, holdsEntityReference, holdsNullCharacter, isAutolink, isUnicodeWhitespace } from './commonmark-grammar.ts'
import { failure, success, type ConvertErrorPath, type Result } from './result.ts'
import { longestBacktickRun } from './backtick-runs.ts'
import { serializeCanonicalJson } from './canonical-json.ts'
import { spellAttributes, spellStringAttribute } from './directive-attributes.ts'
type InlineContext = {
atBlockEnd: boolean
container: LineContainer
inLinkText: boolean
bracketed: boolean
path: ConvertErrorPath
spansLines: boolean
}
type InlineRun = { index: number; kind: 'marked'; mark: AdfMark; nodes: AdfNode[] } | { index: number; kind: 'plain'; node: AdfNode }
const emphasisSpellings: Readonly<Record<string, string>> = { em: '_', strike: '~~', strong: '**' }
const linkAttributes = ['href', 'title']
export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<string> {
const segments = emitRun(nodes, 0, 0, { atBlockEnd: true, container, inLinkText: false, path })
const segments = lineSegments(nodes, container, path)
if (!segments.ok) return segments
return finishLine(segments.value, container, path)
}
export function emitImageLine(alt: string | undefined, href: string, path: ConvertErrorPath): Result<string> {
if (alt !== undefined && /^[ \t]|[ \t]$|[\n\r]/.test(alt)) {
return failure('unspellable-whitespace', 'a media alt holds whitespace no image description spells', path)
}
if (alt !== undefined && holdsNullCharacter(alt)) return failure('unspellable-character', 'a media alt holds a null character CommonMark replaces', path)
export function tryPipeCell(nodes: readonly AdfNode[], path: ConvertErrorPath): string | undefined {
const segments = lineSegments(nodes, 'table-cell', path)
if (!segments.ok) return undefined
if (segments.value.some((segment) => segment.escaping === 'none' && segment.text.includes('|'))) return undefined
const line = finishLine(segments.value, 'table-cell', path)
return line.ok ? line.value : undefined
}
export function tryImageLine(alt: string | undefined, href: string, path: ConvertErrorPath): string | undefined {
if (alt !== undefined && (/^[ \t]|[ \t]$|[\n\r]/.test(alt) || holdsNullCharacter(alt))) return undefined
const destination = spellDestination(href, path)
if (!destination.ok) return destination
const description: InlineSegment[] = alt === undefined ? [] : [{ kind: 'link-text', text: alt }]
return finishLine([{ kind: 'syntax', text: '![' }, ...description, { kind: 'syntax', text: `](${destination.value})` }], 'paragraph', path)
if (!destination.ok) return undefined
const description: InlineSegment[] = alt === undefined ? [] : [{ escaping: 'bracketed', text: alt }]
const line = finishLine([syntax('!['), ...description, syntax(`](${destination.value})`)], 'paragraph', path)
return line.ok ? line.value : undefined
}
function lineSegments(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<InlineSegment[]> {
const segments = emitRun(nodes, 0, 0, { atBlockEnd: true, bracketed: false, path, spansLines: container === 'paragraph' })
if (!segments.ok) return segments
return success(carryStrippedWhitespace(segments.value))
}
function finishLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath): Result<string> {
@@ -51,6 +67,51 @@ function finishLine(segments: readonly InlineSegment[], container: LineContainer
return success(line)
}
// spec/flavour.md, Inline nodes.
function carryStrippedWhitespace(segments: readonly InlineSegment[]): InlineSegment[] {
const carried: InlineSegment[] = []
for (const [index, segment] of segments.entries()) {
const previous = segments[index - 1]
const next = segments[index + 1]
const leading = previous === undefined || previous.text.includes('\n')
const trailing = next === undefined || next.text.includes('\n')
carried.push(...carryEdges(segment, leading, trailing))
}
return carried
}
function carryEdges(segment: InlineSegment, leading: boolean, trailing: boolean): InlineSegment[] {
if (segment.escaping !== 'backslash' && segment.escaping !== 'bracketed') return [segment]
const head = leading ? (/^[ \t]+/.exec(segment.text)?.[0] ?? '') : ''
const body = segment.text.slice(head.length)
const tail = trailing ? (/[ \t]+$/.exec(body)?.[0] ?? '') : ''
const middle = body.slice(0, body.length - tail.length)
const edges: InlineSegment[] = []
if (head !== '') edges.push(carriedText(head))
if (middle !== '') edges.push({ escaping: segment.escaping, text: middle })
if (tail !== '') edges.push(carriedText(tail))
return edges
}
function carriedText(text: string): InlineSegment {
return { escaping: 'attribute', text: spellLeafDirective('text', spellAttributes([['text', spellStringAttribute(text)]])) }
}
function spellLeafDirective(name: string, attributes: string): string {
return `:${name}${attributes === '' ? '{}' : attributes}`
}
function syntax(text: string): InlineSegment {
return { escaping: 'none', text }
}
function refuseContentAndText(node: AdfNode, path: ConvertErrorPath): Result<null> {
if ((node.content ?? []).length > 0 || node.text !== undefined) {
return failure('unsupported-node-shape', `a ${node.type} node holds neither content nor text`, path)
}
return success(null)
}
function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, context: InlineContext): Result<InlineSegment[]> {
if (depth > largestNesting) {
return failure('unsupported-node-shape', `the marks nest deeper than the ${largestNesting} levels the emitter carries`, context.path)
@@ -88,38 +149,85 @@ function nodePath(context: InlineContext, index: number): ConvertErrorPath {
function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result<InlineSegment[]> {
const path = nodePath(context, index)
if (node.type !== 'hardBreak' && node.type !== 'text') {
return failure('unsupported-node-type', `the canonical form spells no inline node of type ${node.type}`, path)
}
const unspelled = Object.keys(node.attrs ?? {})[0]
if (unspelled !== undefined) {
return failure('unspelled-node-attribute', `the ${node.type} attribute ${unspelled} has no canonical markdown spelling`, path)
}
const types = (node.marks ?? []).map((mark) => mark.type)
if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path)
if (node.type === 'hardBreak') {
if (context.container === 'paragraph' && !context.atBlockEnd) return success([{ kind: 'syntax', text: '\\\n' }])
return success([{ kind: 'syntax', text: ':hardBreak{}' }])
}
if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node carries no text', path)
if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node carries content', path)
if (/[\n\r]/.test(node.text)) return failure('unspellable-whitespace', 'a text node holds a newline CommonMark cannot spell', path)
if (node.type === 'text') return emitText(node, context, path)
if (node.type === 'hardBreak') return emitHardBreak(node, context, path)
const directive = inlineDirective(node.type)
if (directive === undefined) return failure('unsupported-node-type', `the canonical form spells no inline node of type ${node.type}`, path)
return emitInlineDirective(node, directive, path)
}
function emitHardBreak(node: AdfNode, context: InlineContext, path: ConvertErrorPath): Result<InlineSegment[]> {
const unspelled = Object.keys(node.attrs ?? {})[0]
if (unspelled !== undefined) return failure('unspelled-node-attribute', `the hardBreak attribute ${unspelled} has no canonical markdown spelling`, path)
const empty = refuseContentAndText(node, path)
if (!empty.ok) return empty
if (context.spansLines && !context.atBlockEnd) return success([syntax('\\\n')])
return success([syntax(spellLeafDirective('hardBreak', ''))])
}
function emitInlineDirective(node: AdfNode, directive: InlineDirective, path: ConvertErrorPath): Result<InlineSegment[]> {
const empty = refuseContentAndText(node, path)
if (!empty.ok) return empty
const attributes = spellInlineNodeAttributes(node, directive, path)
if (!attributes.ok) return attributes
const slot = directive.slot === undefined ? undefined : node.attrs?.[directive.slot]
if (slot === undefined) return success([syntax(spellLeafDirective(node.type, attributes.value))])
if (typeof slot !== 'string') return failure('unsupported-node-shape', `the ${node.type} attribute ${directive.slot} holds no string`, path)
if (/[\n\r]/.test(slot)) return failure('unspellable-whitespace', `a ${node.type} content slot holds a newline no inline directive spans`, path)
if (holdsNullCharacter(slot)) return failure('unspellable-character', `a ${node.type} content slot holds a null character CommonMark replaces`, path)
const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }]
return success([syntax(`:${node.type}[`), ...content, syntax(`]${attributes.value}`)])
}
function emitText(node: AdfNode, context: InlineContext, path: ConvertErrorPath): Result<InlineSegment[]> {
const unspelled = Object.keys(node.attrs ?? {})[0]
if (unspelled !== undefined) return failure('unspelled-node-attribute', `the text attribute ${unspelled} has no canonical markdown spelling`, path)
if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text', path)
if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node holds no content', path)
if (/\r/.test(node.text)) return failure('unspellable-whitespace', 'a text node holds a carriage return CommonMark rewrites', path)
if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path)
return success([{ kind: context.inLinkText ? 'link-text' : 'literal', text: node.text }])
const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash'
const parts = node.text.split(/(\n+)/).filter((part) => part !== '')
return success(parts.map((part) => (part.startsWith('\n') ? carriedText(part) : { escaping, text: part })))
}
function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result<InlineSegment[]> {
if (mark.type === 'code') return emitCodeSpan(nodes, depth, nodePath(context, index))
if (mark.type === 'link') return emitLink(nodes, mark, depth, index, context)
const path = nodePath(context, index)
const spelling = mark.type === 'em' ? '_' : mark.type === 'strike' ? '~~' : mark.type === 'strong' ? '**' : undefined
if (spelling === undefined) return failure('unspellable-mark', `no markdown spelling holds the ${mark.type} mark`, path)
const spelling = Object.hasOwn(emphasisSpellings, mark.type) ? emphasisSpellings[mark.type] : undefined
if (spelling !== undefined) return emitEmphasis(nodes, mark, spelling, depth, index, context, path)
const vocabulary = markDirective(mark.type)
if (vocabulary === undefined) return failure('unspellable-mark', `no markdown spelling holds the ${mark.type} mark`, path)
const attributes = spellMarkAttributes(mark, vocabulary, path)
if (!attributes.ok) return attributes
const inner = emitRun(nodes, depth + 1, index, { ...context, bracketed: true, spansLines: false })
if (!inner.ok) return inner
return success([syntax(`:${mark.type}[`), ...inner.value, syntax(`]${attributes.value}`)])
}
function emitEmphasis(
nodes: readonly AdfNode[],
mark: AdfMark,
spelling: string,
depth: number,
index: number,
context: InlineContext,
path: ConvertErrorPath,
): Result<InlineSegment[]> {
if (Object.keys(mark.attrs ?? {}).length > 0) return failure('unspellable-mark', `the ${mark.type} spelling holds no attributes`, path)
const inner = emitRun(nodes, depth + 1, index, context)
if (!inner.ok) return inner
const text = inner.value.map((segment) => segment.text).join('')
const carried = carryStrippedWhitespace(inner.value)
const text = carried.map((segment) => segment.text).join('')
if (holdsEdgeWhitespace(text)) return failure('unspellable-whitespace', `the ${mark.type} spelling cannot open or close beside whitespace`, path)
return success([{ kind: 'emphasis-open', mark: mark.type, text: spelling }, ...inner.value, { kind: 'emphasis-close', mark: mark.type, text: spelling }])
return success([
{ emphasis: 'open', escaping: 'none', mark: mark.type, text: spelling },
...carried,
{ emphasis: 'close', escaping: 'none', mark: mark.type, text: spelling },
])
}
function emitCodeSpan(nodes: readonly AdfNode[], depth: number, path: ConvertErrorPath): Result<InlineSegment[]> {
@@ -133,7 +241,7 @@ function emitCodeSpan(nodes: readonly AdfNode[], depth: number, path: ConvertErr
if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path)
const fence = '`'.repeat(longestBacktickRun(text) + 1)
const padded = needsPadding(text) ? ` ${text} ` : text
return success([{ kind: 'syntax', text: `${fence}${padded}${fence}` }])
return success([syntax(`${fence}${padded}${fence}`)])
}
function holdsEdgeWhitespace(text: string): boolean {
@@ -155,14 +263,14 @@ function emitLink(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index
if (title !== undefined && typeof title !== 'string') return failure('unsupported-node-shape', 'a link title is no string', path)
const node = nodes[0]
const bare = nodes.length === 1 && node !== undefined && node.type === 'text' && node.text === href && (node.marks ?? []).length === depth + 1
if (bare && title === undefined && isAutolink(href) && !holdsEntityReference(href)) return success([{ kind: 'syntax', text: `<${href}>` }])
if (bare && title === undefined && isAutolink(href) && !holdsEntityReference(href)) return success([syntax(`<${href}>`)])
const destination = spellDestination(href, path)
if (!destination.ok) return destination
const spelledTitle = title === undefined ? success('') : spellTitle(title, path)
if (!spelledTitle.ok) return spelledTitle
const inner = emitRun(nodes, depth + 1, index, { ...context, inLinkText: true })
const inner = emitRun(nodes, depth + 1, index, { ...context, bracketed: true })
if (!inner.ok) return inner
return success([{ kind: 'syntax', text: '[' }, ...inner.value, { kind: 'syntax', text: `](${destination.value}${spelledTitle.value})` }])
return success([syntax('['), ...inner.value, syntax(`](${destination.value}${spelledTitle.value})`)])
}
function spellDestination(href: string, path: ConvertErrorPath): Result<string> {
+8 -22
View File
@@ -1,9 +1,8 @@
import type { AdfNode } from './adf-document.ts'
import type { JsonValue } from './json-value.ts'
import { emitInlineLine } from './markdown-inline.ts'
import { success, type ConvertErrorPath, type Result } from './result.ts'
import { tryPipeCell } from './markdown-inline.ts'
import type { ConvertErrorPath } from './result.ts'
export function emitPipeTable(node: AdfNode, path: ConvertErrorPath): Result<string> | undefined {
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath): string | undefined {
const rows = pipeRows(node)
if (rows === undefined) return undefined
const lines: string[] = []
@@ -11,14 +10,14 @@ export function emitPipeTable(node: AdfNode, path: ConvertErrorPath): Result<str
const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) {
const content = paragraph.content ?? []
const line = content.length === 0 ? success('') : emitInlineLine(content, 'table-cell', [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0])
if (!line.ok) return line
cells.push(line.value)
const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0])
if (line === undefined) return undefined
cells.push(line)
}
lines.push(`| ${cells.join(' | ')} |`)
if (rowIndex === 0) lines.push(`| ${cells.map(() => '---').join(' | ')} |`)
}
return success(lines.join('\n'))
return lines.join('\n')
}
function pipeRows(node: AdfNode): AdfNode[][] | undefined {
@@ -49,18 +48,5 @@ function plainParagraph(cell: AdfNode): AdfNode | undefined {
const content = cell.content ?? []
const paragraph = content[0]
if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !isPlain(paragraph)) return undefined
return (paragraph.content ?? []).some(spellsPipeAsSyntax) ? undefined : paragraph
}
// A pipe the inline layer emits as syntax takes no backslash, so the cell has no pipe spelling.
function spellsPipeAsSyntax(child: AdfNode): boolean {
return (child.marks ?? []).some((mark) => {
if (mark.type === 'code') return (child.text ?? '').includes('|')
if (mark.type !== 'link') return false
return holdsPipe(mark.attrs?.['href']) || holdsPipe(mark.attrs?.['title'])
})
}
function holdsPipe(value: JsonValue | undefined): boolean {
return typeof value === 'string' && value.includes('|')
return paragraph
}
+25 -12
View File
@@ -71,13 +71,13 @@ detail is settled at its own milestone.
answer on a CommonMark block beside a directive block (1d) drops into the same seam. Give
the emitter's refusals a corpus home while the directories grow: `corpus/unspellable/`,
a `.json` beside the `ConvertErrorCode` it must return, the emitter half of `corpus/errors/`.
- [ ] **2c — Inline nodes and marks.** `inline-nodes/` green. `InlineSegment.kind` splits into
its two axes here — escapability (`attribute` for `:text{text="…"}`, `backslash`, `none`)
and the emphasis role — rather than gaining a third value that means one of each. A lone
surrogate in a text node emits verbatim and becomes U+FFFD on any UTF-8 encode, a §2 break
plain text still holds open — attribute values already escape it. The inline directives
arriving here emit their attribute values as syntax, so `spellsPipeAsSyntax` grows with
them alongside the `|` escaping `spec/flavour.md` already mandates in a pipe cell.
- [ ] **2c — Inline nodes and marks.** `inline-nodes/` green. `InlineSegment` splits into its
two axes — escapability (`attribute` for `:text{text="…"}`, `backslash`, `bracketed`,
`none`) and the emphasis role. A lone surrogate in a text node emits verbatim and becomes
U+FFFD on any UTF-8 encode, a §2 break plain text still holds open — attribute values
already escape it. The pipe form's fallback reads the emitted segments rather than naming
the nodes whose attribute values spell a pipe as syntax, so 2e's `\u007c` narrows it in
one place.
- [ ] **2d — The opaque carry** (§3). Fixtures and emitter together, into
`corpus/round-trip/opaque-carry/`: an unknown node in both positions, the reserved `adf`
info string, and the `codeBlock` whose language is `adf`.
@@ -90,6 +90,13 @@ detail is settled at its own milestone.
paragraph line inside a container body shaped like a closing fence (`:::`, `::: x`).
Guard `fenceNestingFault`'s bare-run pop here too — a run shorter than the open fence is a
fault, not a close — which today's emitter cannot reach.
Two moves land before the attribute spelling changes. One mark vocabulary:
`emphasisSpellings`, `linkAttributes` and the `code`/`link` names join
`inline-directives.ts`, which holds four of the nine marks while the rest are branch
literals in the emitter — and the parser (3) needs every name to make `:em[x]` the named
error `spec/flavour.md` promises. And `escaping: 'attribute'` earns its keep at the
`\u007c` rule or collapses into `none`: nothing the escaper does tells the two apart
today, since a carried segment holds only spaces, tabs and newlines.
The gate gains the collision property here: no two corpus documents may emit the same
bytes — one spelling for two documents is a round-trip break no parser can undo, and it is
provable without one. It also settles the emitter's one known approximation: delimiter
@@ -108,11 +115,17 @@ detail is settled at its own milestone.
assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin.
`src/` gets its hierarchy at the same split — `adf/`,
`markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is
still mechanical. `block-directives.ts` is the one file that does not move whole: the node
table is ADF knowledge milestones 6-7 need too and belongs in `adf/`, `spellDirectiveHeader`
in `markdown/`. `AttributeKind` stays above both — it is the vocabulary a string-typed
attribute grammar needs, which is why HTML will want it too, not a markdown spelling. The table is a second copy of `spec/flavour.md`'s prose with no drift guard,
and a mistyped attribute name degrades into a false refusal no test catches.
still mechanical. Three files do not move whole: `block-directives.ts` and
`inline-directives.ts` each hold a node table milestones 6-7 need in `adf/` beside a
markdown spelling that belongs in `markdown/`, and `directive-attributes.ts` fuses the
format-neutral conformance walk (`vocabularyPairs`) with the markdown value spelling HTML
has no use for. `spellDestination`, `spellTitle` and `balanced` leave `markdown-inline.ts`
here too — CommonMark destination spelling `emitLink` and `tryImageLine` share, and the six
concerns that file carries are one fewer for it. `AttributeKind` and `AttributeVocabulary`
stay above all of it — the vocabulary a string-typed attribute grammar needs, which is why
HTML will want them too, not a markdown spelling. Both node tables are a second copy of
`spec/flavour.md`'s prose with no drift guard, and a mistyped attribute name degrades into a
false refusal no test catches.
- [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and
3. Editor-normal (§2) gets its implementation here — `toEditorNormal(doc)` and the equality
the round-trip asserts, which over normalized input is the canonical serializer's compact