Read the emphasis, the links and the lone image CommonMark spells #34

Merged
lilleman merged 4 commits from tick-3e into main 2026-08-31 22:33:55 +02:00
28 changed files with 741 additions and 184 deletions
Showing only changes of commit 163906dd6a - Show all commits
+1
View File
@@ -0,0 +1 @@
unmappable-image
+1
View File
@@ -0,0 +1 @@
See ![the moon](https://example.com/moon.png) tonight.
@@ -0,0 +1,88 @@
{
"content": [
{
"content": [
{
"marks": [
{
"type": "em"
}
],
"text": "Torque",
"type": "text"
},
{
"text": " the ",
"type": "text"
},
{
"marks": [
{
"type": "strong"
}
],
"text": "M8 bolt",
"type": "text"
},
{
"text": " to 25 Nm, then ",
"type": "text"
},
{
"marks": [
{
"type": "em"
},
{
"type": "strong"
}
],
"text": "log it",
"type": "text"
},
{
"text": ".",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Un",
"type": "text"
},
{
"marks": [
{
"type": "em"
}
],
"text": "bolt",
"type": "text"
},
{
"text": "ing needs a ",
"type": "text"
},
{
"marks": [
{
"type": "strike"
}
],
"text": "10 mm",
"type": "text"
},
{
"text": " 13 mm spanner.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,3 @@
*Torque* the __M8 bolt__ to 25 Nm, then ***log it***.
Un*bolt*ing needs a ~~10 mm~~ 13 mm spanner.
+64
View File
@@ -0,0 +1,64 @@
{
"content": [
{
"content": [
{
"text": "See ",
"type": "text"
},
{
"marks": [
{
"attrs": {
"href": "https://example.com/changelog"
},
"type": "link"
}
],
"text": "the changelog",
"type": "text"
},
{
"text": " and ",
"type": "text"
},
{
"marks": [
{
"attrs": {
"href": "https://example.com/guide",
"title": "Setup guide"
},
"type": "link"
}
],
"text": "the guide",
"type": "text"
},
{
"text": ", or mail ",
"type": "text"
},
{
"marks": [
{
"attrs": {
"href": "mailto:ops@example.com"
},
"type": "link"
}
],
"text": "ops@example.com",
"type": "text"
},
{
"text": ".",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
+4
View File
@@ -0,0 +1,4 @@
See [the changelog][changes] and [the guide], or mail <ops@example.com>.
[changes]: https://example.com/changelog
[the guide]: https://example.com/guide "Setup guide"
@@ -0,0 +1 @@
[The torque table](https://example.com/a\)b)
@@ -0,0 +1 @@
[The torque table](https://example.com/torque "He said \"hi\"")
@@ -1 +0,0 @@
unspellable-link-destination
@@ -1 +0,0 @@
unspellable-link-title
+4 -3
View File
@@ -37,9 +37,10 @@ normalizes to it through the round-trip.
CommonMark admits no spelling — the end of a block, inside an ATX heading — or where the node CommonMark admits no spelling — the end of a block, inside an ATX heading — or where the node
carries an attribute, it is the inline directive. carries an attribute, it is the inline directive.
- An empty paragraph — real payloads carry them — is `::paragraph`. - An empty paragraph — real payloads carry them — is `::paragraph`.
- Links `[text](url)`; `<…>` around a destination containing spaces; title in double quotes. - Links `[text](url)`; `<…>` around a destination containing spaces; title in double quotes. A
`<url>` autolink form only when the text equals the destination and the destination is a valid backslash escapes a parenthesis the destination leaves unbalanced, and a quote inside the title;
CommonMark autolink (absolute URI). a balanced pair stays bare. `<url>` autolink form only when the text equals the destination and
the destination is a valid CommonMark autolink (absolute URI).
- Paragraphs on one line — no soft wrapping; a soft line break in input becomes a single space. - Paragraphs on one line — no soft wrapping; a soft line break in input becomes a single space.
- Entity references in input decode to their characters; output backslash-escapes only where text - Entity references in input decode to their characters; output backslash-escapes only where text
would otherwise parse as syntax, scanning the assembled line rather than each text node: escape would otherwise parse as syntax, scanning the assembled line rather than each text node: escape
+11 -3
View File
@@ -197,13 +197,21 @@ export function openingHtmlBlock(line: string, interrupting: boolean): OpenHtmlB
} }
export function opensBracketedAutolink(text: string, index: number): boolean { export function opensBracketedAutolink(text: string, index: number): boolean {
bracketedAutolink.lastIndex = index return readBracketedAutolink(text, index) !== undefined
return bracketedAutolink.test(text)
} }
export function opensEmailAutolink(text: string, index: number): boolean { export function opensEmailAutolink(text: string, index: number): boolean {
return readEmailAutolink(text, index) !== undefined
}
export function readBracketedAutolink(text: string, index: number): number | undefined {
bracketedAutolink.lastIndex = index
return bracketedAutolink.exec(text)?.[0].length
}
export function readEmailAutolink(text: string, index: number): number | undefined {
emailAutolink.lastIndex = index emailAutolink.lastIndex = index
return emailAutolink.test(text) return emailAutolink.exec(text)?.[0].length
} }
export function setextHeadingLevel(line: string): number | undefined { export function setextHeadingLevel(line: string): number | undefined {
+12 -4
View File
@@ -76,7 +76,6 @@ test('spells a code block language no info string holds as an attribute', () =>
test('refuses a link destination CommonMark cannot spell', () => { test('refuses a link destination CommonMark cannot spell', () => {
const link = (href: string): AdfDocument => document(paragraph({ marks: [{ attrs: { href }, type: 'link' }], text: 't', type: 'text' })) const link = (href: string): AdfDocument => document(paragraph({ marks: [{ attrs: { href }, type: 'link' }], text: 't', type: 'text' }))
assert.equal(code(adfToMarkdown(link('https://example.com/a)b'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('https://example.com/a b>c'))), 'unspellable-link-destination') assert.equal(code(adfToMarkdown(link('https://example.com/a b>c'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('<https://example.com/'))), 'unspellable-link-destination') assert.equal(code(adfToMarkdown(link('<https://example.com/'))), 'unspellable-link-destination')
assert.equal(code(adfToMarkdown(link('https://example.com/a\\b'))), 'unspellable-link-destination') assert.equal(code(adfToMarkdown(link('https://example.com/a\\b'))), 'unspellable-link-destination')
@@ -84,14 +83,23 @@ test('refuses a link destination CommonMark cannot spell', () => {
assert.equal(code(adfToMarkdown(link('https://example.com/a\nb'))), 'unspellable-link-destination') assert.equal(code(adfToMarkdown(link('https://example.com/a\nb'))), 'unspellable-link-destination')
const entity = 'https://example.com/?a=1&amp;b=2' const entity = 'https://example.com/?a=1&amp;b=2'
assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { href: entity }, type: 'link' }], text: entity, type: 'text' })))), 'unspellable-link-destination') assert.equal(code(adfToMarkdown(document(paragraph({ marks: [{ attrs: { href: entity }, type: 'link' }], text: entity, type: 'text' })))), 'unspellable-link-destination')
assert.equal(markdown(adfToMarkdown(link('https://en.example.com/a_(b)'))), '[t](https://en.example.com/a_(b))\n')
}) })
test('refuses a link title CommonMark cannot spell', () => { test('escapes the parenthesis a link destination leaves unbalanced, and no other', () => {
const link = (href: string): string => markdown(adfToMarkdown(document(paragraph({ marks: [{ attrs: { href }, type: 'link' }], text: 't', type: 'text' }))))
assert.equal(link('https://en.example.com/a_(b)'), '[t](https://en.example.com/a_(b))\n')
assert.equal(link('https://example.com/a)b'), '[t](https://example.com/a\\)b)\n')
assert.equal(link('https://example.com/a(b'), '[t](https://example.com/a\\(b)\n')
assert.equal(link('https://example.com/)(') , '[t](https://example.com/\\)\\()\n')
assert.equal(link('https://example.com/a (b'), '[t](<https://example.com/a (b>)\n')
})
test('escapes the quote a link title holds, and refuses the rest', () => {
const titled = (title: string): AdfDocument => const titled = (title: string): AdfDocument =>
document(paragraph({ marks: [{ attrs: { href: 'https://example.com/', title }, type: 'link' }], text: 't', type: 'text' })) document(paragraph({ marks: [{ attrs: { href: 'https://example.com/', title }, type: 'link' }], text: 't', type: 'text' }))
assert.equal(code(adfToMarkdown(titled('He said "hi"'))), 'unspellable-link-title') assert.equal(markdown(adfToMarkdown(titled('He said "hi"'))), '[t](https://example.com/ "He said \\"hi\\"")\n')
assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link-title') assert.equal(code(adfToMarkdown(titled('a\nb'))), 'unspellable-link-title')
assert.equal(code(adfToMarkdown(titled('a\\b'))), 'unspellable-link-title')
}) })
test('carries a link mark the link spelling cannot write', () => { test('carries a link mark the link spelling cannot write', () => {
-38
View File
@@ -1,38 +0,0 @@
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { holdsControlCharacter } from '../commonmark-grammar.ts'
import { holdsEntityReference } from '../entity-references.ts'
export function spellDestination(href: string, path: ConvertErrorPath): Result<string> {
if (holdsControlCharacter(href)) return failure('unspellable-link-destination', 'a link destination holds a control character', path)
if (href.includes('\\')) return failure('unspellable-link-destination', 'no canonical escape spells a backslash in a link destination', path)
if (holdsEntityReference(href)) {
return failure('unspellable-link-destination', 'a link destination holds an entity reference that decodes on the way back', path)
}
if (href.includes(' ')) {
if (/[<>]/.test(href)) {
return failure('unspellable-link-destination', 'no canonical escape spells an angle bracket beside a space in a link destination', path)
}
return success(`<${href}>`)
}
if (href.startsWith('<')) return failure('unspellable-link-destination', 'a bare link destination cannot begin with an angle bracket', path)
if (!balanced(href)) return failure('unspellable-link-destination', 'no canonical escape spells an unbalanced parenthesis in a link destination', path)
return success(href)
}
export function spellTitle(title: string, path: ConvertErrorPath): Result<string> {
if (/["\n\r\\]/.test(title)) {
return failure('unspellable-link-title', 'no canonical escape spells a quote, backslash or newline in a link title', path)
}
if (holdsEntityReference(title)) return failure('unspellable-link-title', 'a link title holds an entity reference that decodes on the way back', path)
return success(` "${title}"`)
}
function balanced(href: string): boolean {
let depth = 0
for (const character of href) {
if (character === '(') depth += 1
if (character === ')') depth -= 1
if (depth < 0) return false
}
return depth === 0
}
+1 -1
View File
@@ -11,7 +11,7 @@ import { longestBacktickRun } from '../backtick-runs.ts'
import { markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { markSpelling, spellMarkAttributes } from '../mark-spellings.ts'
import { serializeCanonicalJson } from '../../canonical-json.ts' import { serializeCanonicalJson } from '../../canonical-json.ts'
import { spellAttributes, spellStringAttribute } from '../directive-attributes.ts' import { spellAttributes, spellStringAttribute } from '../directive-attributes.ts'
import { spellDestination, spellTitle } from './destination-spelling.ts' import { spellDestination, spellTitle } from '../link-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
type EmittedLine = { line: string; segments: InlineSegment[] } type EmittedLine = { line: string; segments: InlineSegment[] }
+1 -8
View File
@@ -1,5 +1,5 @@
import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../backtick-runs.ts'
import { delimiterFlags, isWordCharacter, matchEmphasis } from '../emphasis-matching.ts' import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../emphasis-matching.ts'
import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark-grammar.ts' import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark-grammar.ts'
import { opensInlineDirective } from '../directive-attributes.ts' import { opensInlineDirective } from '../directive-attributes.ts'
import { readEntityReference } from '../entity-references.ts' import { readEntityReference } from '../entity-references.ts'
@@ -260,13 +260,6 @@ function startsRun(scan: string, index: number, escaped: ReadonlySet<number>): b
return scan.charAt(index - 1) !== scan.charAt(index) return scan.charAt(index - 1) !== scan.charAt(index)
} }
function runLength(scan: string, index: number): number {
const character = scan.charAt(index)
let length = 0
while (scan.charAt(index + length) === character) length += 1
return length
}
function charAt(text: string, index: number): string { function charAt(text: string, index: number): string {
return index < 0 ? '' : text.charAt(index) return index < 0 ? '' : text.charAt(index)
} }
+7
View File
@@ -64,6 +64,13 @@ export function matchEmphasis<Run extends DelimiterRun>(runs: readonly Run[]): E
return pairings return pairings
} }
export function runLength(text: string, index: number): number {
const character = text.charAt(index)
let length = 0
while (text.charAt(index + length) === character) length += 1
return length
}
function candidates<Run extends DelimiterRun>(runs: readonly Run[]): Candidate<Run> | undefined { function candidates<Run extends DelimiterRun>(runs: readonly Run[]): Candidate<Run> | undefined {
let first: Candidate<Run> | undefined let first: Candidate<Run> | undefined
let previous: Candidate<Run> | undefined let previous: Candidate<Run> | undefined
+129
View File
@@ -0,0 +1,129 @@
import { backslashEscape, decodeTextEscapes, holdsControlCharacter } from './commonmark-grammar.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../result.ts'
import { holdsEntityReference } from './entity-references.ts'
export type LinkDefinition = { destination: string; title?: string }
export type LinkPart = { length: number; value: string }
const bracketedDestination = /<((?:[^\n<>\\]|\\[^\n])*)>/y
const linkLabel = /\[((?:[^[\]\\]|\\[\s\S]){0,999})\]/y
const titleClosers: Readonly<Record<string, string>> = { '"': '"', "'": "'", '(': ')' }
// The label is matched by its normalized raw text, escapes and all.
export function readLabel(text: string, offset: number): LinkPart | undefined {
linkLabel.lastIndex = offset
const value = linkLabel.exec(text)?.[1]
return value === undefined ? undefined : { length: value.length + 2, value }
}
export function normalizeLabel(raw: string): string {
return raw
.replace(/^[ \t\n]+|[ \t\n]+$/g, '')
.replace(/[ \t\n]+/g, ' ')
.toLowerCase()
}
export function readDestination(text: string, offset: number): LinkPart | undefined {
bracketedDestination.lastIndex = offset
const bracketed = bracketedDestination.exec(text)?.[0]
if (bracketed !== undefined) return { length: bracketed.length, value: decodeTextEscapes(bracketed.slice(1, -1)) }
if (text.charAt(offset) === '<') return undefined
let depth = 0
let index = offset
while (index < text.length) {
const character = text.charAt(index)
if (character === ' ' || holdsControlCharacter(character)) break
if (backslashEscape(text, index) !== undefined) {
index += 2
continue
}
if (character === '(') depth += 1
if (character === ')') {
depth -= 1
if (depth < 0) break
}
index += 1
}
return index <= offset ? undefined : { length: index - offset, value: decodeTextEscapes(text.slice(offset, index)) }
}
// `offset` sits on the `(` a link text closes into.
export function readInlineTarget(text: string, offset: number): { definition: LinkDefinition; length: number } | undefined {
let index = skipLinkWhitespace(text, offset + 1)
let destination = ''
if (text.charAt(index) !== ')') {
const read = readDestination(text, index)
if (read === undefined) return undefined
destination = read.value
index += read.length
}
const afterDestination = index
index = skipLinkWhitespace(text, index)
const title = index > afterDestination ? readTitle(text, index) : undefined
if (title !== undefined) index = skipLinkWhitespace(text, index + title.length)
if (text.charAt(index) !== ')') return undefined
return { definition: title === undefined ? { destination } : { destination, title: title.value }, length: index + 1 - offset }
}
export function readTitle(text: string, offset: number): LinkPart | undefined {
const opener = text.charAt(offset)
const closer = titleClosers[opener]
if (closer === undefined) return undefined
let index = offset + 1
while (index < text.length) {
const character = text.charAt(index)
if (backslashEscape(text, index) !== undefined) {
index += 2
continue
}
if (character === closer) return { length: index + 1 - offset, value: decodeTextEscapes(text.slice(offset + 1, index)) }
if (character === opener) return undefined
index += 1
}
return undefined
}
// The label, the destination and the title each take at most one line ending with them.
export function skipLinkWhitespace(text: string, offset: number): number {
const rest = text.slice(offset)
return offset + rest.length - rest.replace(/^[ \t]*\n?[ \t]*/, '').length
}
export function spellDestination(href: string, path: ConvertErrorPath): Result<string> {
if (holdsControlCharacter(href)) return failure('unspellable-link-destination', 'a link destination holds a control character', path)
if (href.includes('\\')) return failure('unspellable-link-destination', 'no canonical escape spells a backslash in a link destination', path)
if (holdsEntityReference(href)) {
return failure('unspellable-link-destination', 'a link destination holds an entity reference that decodes on the way back', path)
}
if (href.includes(' ')) {
if (/[<>]/.test(href)) {
return failure('unspellable-link-destination', 'no canonical escape spells an angle bracket beside a space in a link destination', path)
}
return success(`<${href}>`)
}
if (href.startsWith('<')) return failure('unspellable-link-destination', 'a bare link destination cannot begin with an angle bracket', path)
return success(escapeUnbalanced(href))
}
export function spellTitle(title: string, path: ConvertErrorPath): Result<string> {
if (/[\n\r\\]/.test(title)) {
return failure('unspellable-link-title', 'no canonical escape spells a backslash or newline in a link title', path)
}
if (holdsEntityReference(title)) return failure('unspellable-link-title', 'a link title holds an entity reference that decodes on the way back', path)
return success(` "${title.replaceAll('"', '\\"')}"`)
}
function escapeUnbalanced(href: string): string {
const open: number[] = []
const unbalanced = new Set<number>()
for (let index = 0; index < href.length; index += 1) {
const character = href.charAt(index)
if (character === '(') open.push(index)
if (character === ')' && open.pop() === undefined) unbalanced.add(index)
}
for (const index of open) unbalanced.add(index)
let spelled = ''
for (let index = 0; index < href.length; index += 1) spelled += (unbalanced.has(index) ? '\\' : '') + href.charAt(index)
return spelled
}
+5 -3
View File
@@ -1,7 +1,7 @@
import assert from 'node:assert/strict' import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { LinkDefinition } from './link-reference-definitions.ts' import type { LinkDefinition } from '../link-syntax.ts'
import { parseBlocks } from './blocks.ts' import { parseBlocks } from './blocks.ts'
function definitions(markdown: string): [string, LinkDefinition][] { function definitions(markdown: string): [string, LinkDefinition][] {
@@ -21,9 +21,10 @@ test('keeps the link reference definitions a paragraph gives up, the first of a
]) ])
assert.deepEqual(definitions('[a\\]b]: /url\n'), [['a\\]b', { destination: '/url' }]]) assert.deepEqual(definitions('[a\\]b]: /url\n'), [['a\\]b', { destination: '/url' }]])
assert.deepEqual(definitions('[a]: /url(x)y\n'), [['a', { destination: '/url(x)y' }]]) assert.deepEqual(definitions('[a]: /url(x)y\n'), [['a', { destination: '/url(x)y' }]])
assert.deepEqual(definitions('[a]: /url\\(x\n'), [['a', { destination: '/url\\(x' }]]) assert.deepEqual(definitions('[a]: /url\\(x\n'), [['a', { destination: '/url(x' }]])
assert.deepEqual(definitions('[a]: /url&amp;x\n'), [['a', { destination: '/url&x' }]])
assert.deepEqual(definitions('[a]: <>\n'), [['a', { destination: '' }]]) assert.deepEqual(definitions('[a]: <>\n'), [['a', { destination: '' }]])
assert.deepEqual(definitions('[a]: /url "He said \\"hi\\""\n'), [['a', { destination: '/url', title: 'He said \\"hi\\"' }]]) assert.deepEqual(definitions('[a]: /url "He said \\"hi\\""\n'), [['a', { destination: '/url', title: 'He said "hi"' }]])
assert.deepEqual(definitions('[a]: /url\\\n[b]: /b\n'), [ assert.deepEqual(definitions('[a]: /url\\\n[b]: /b\n'), [
['a', { destination: '/url\\' }], ['a', { destination: '/url\\' }],
['b', { destination: '/b' }], ['b', { destination: '/b' }],
@@ -47,6 +48,7 @@ test('leaves the paragraph a line no definition spells', () => {
assert.deepEqual(definitions('[a]: /url "One" and more\n'), []) assert.deepEqual(definitions('[a]: /url "One" and more\n'), [])
assert.deepEqual(definitions('[a]:\n'), []) assert.deepEqual(definitions('[a]:\n'), [])
assert.deepEqual(definitions('[a]: /url "unclosed\n'), []) assert.deepEqual(definitions('[a]: /url "unclosed\n'), [])
assert.deepEqual(definitions('[a]: /url (a(b)\n'), [])
assert.deepEqual(kinds('[a]: /url\nPart.\n'), ['paragraph']) assert.deepEqual(kinds('[a]: /url\nPart.\n'), ['paragraph'])
}) })
+1 -1
View File
@@ -1,4 +1,4 @@
import type { LinkDefinition } from './link-reference-definitions.ts' import type { LinkDefinition } from '../link-syntax.ts'
import { import {
atxHeading, atxHeading,
claimsDirectiveLine, claimsDirectiveLine,
+228 -28
View File
@@ -1,69 +1,269 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfMark, AdfNode } from '../../adf/document.ts'
import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct } from '../commonmark-grammar.ts' import type { LinkDefinition } from '../link-syntax.ts'
import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink } from '../commonmark-grammar.ts'
import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../backtick-runs.ts'
import { delimiterFlags, matchEmphasis, runLength } from '../emphasis-matching.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { normalizeLabel, readInlineTarget, readLabel } from '../link-syntax.ts'
import { serializeCanonicalJson } from '../../canonical-json.ts'
type Run = { nodes: AdfNode[]; text: string; undecodedFrom: number } export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] }
export type LinkDefinitions = ReadonlyMap<string, LinkDefinition>
type Bracket = { active: boolean; image: boolean; kind: 'open'; start: number }
type Piece = Bracket | { kind: 'nodes'; nodes: AdfNode[] } | { canClose: boolean; canOpen: boolean; character: string; kind: 'run'; length: number }
type Run = { canClose: boolean; canOpen: boolean; character: string; index: number; length: number }
type Scan = { definitions: LinkDefinitions; image: AdfNode | undefined; path: ConvertErrorPath; pending: string; pieces: Piece[]; source: string }
const emphasisCharacters = '*_~'
const hardBreakSpaces = / {2,}$/ const hardBreakSpaces = / {2,}$/
const imageAlone = 'an image fits only as a paragraph of its own'
const trailingSpace = /[ \t]+$/ const trailingSpace = /[ \t]+$/
export function parseInlineContent(source: string, path: ConvertErrorPath): Result<AdfNode[]> { export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath): Result<InlineContent> {
const run: Run = { nodes: [], text: '', undecodedFrom: 0 } const scan: Scan = { definitions, image: undefined, path, pending: '', pieces: [], source }
let index = 0 let index = 0
while (index < source.length) { while (index < source.length) {
const character = source.charAt(index) const character = source.charAt(index)
if (character === '\\' && source.charAt(index + 1) === '\n') { if (character === '\\' && source.charAt(index + 1) === '\n') {
// CommonMark strips the spaces the two-space break is spelled with, and keeps those before a backslash. // CommonMark strips the spaces the two-space break is spelled with, and keeps those before a backslash.
takeRun(run, source, index, index + 2, false) flush(scan, false)
pushNode(run, { type: 'hardBreak' }) pushNode(scan, { type: 'hardBreak' })
index += 2
continue
}
if (backslashEscape(source, index) !== undefined) {
scan.pending += source.slice(index, index + 2)
index += 2 index += 2
continue continue
} }
if (character === '\n') { if (character === '\n') {
const hard = hardBreakSpaces.test(source.slice(run.undecodedFrom, index)) const hard = hardBreakSpaces.test(scan.pending)
takeRun(run, source, index, index + 1, true) flush(scan, true)
if (hard) pushNode(run, { type: 'hardBreak' }) if (hard) pushNode(scan, { type: 'hardBreak' })
else run.text += ' ' else scan.pending = ' '
index += 1 index += 1
continue continue
} }
if (character === '`') { if (character === '`') {
const span = readCodeSpan(source, index) const span = readCodeSpan(source, index)
if (span === undefined) { if (span === undefined) {
index += backtickRun(source, index) const run = backtickRun(source, index)
scan.pending += source.slice(index, index + run)
index += run
continue continue
} }
takeRun(run, source, index, span.end, false) flush(scan, false)
pushNode(run, { marks: [{ type: 'code' }], text: span.text, type: 'text' }) pushNode(scan, { marks: [{ type: 'code' }], text: span.text, type: 'text' })
index = span.end index = span.end
continue continue
} }
if (character === '<') { if (character === '<') {
const autolink = readAutolink(source, index)
if (autolink !== undefined) {
flush(scan, false)
pushNode(scan, autolink.node)
index += autolink.length
continue
}
const construct = inlineHtmlConstruct(source, index) const construct = inlineHtmlConstruct(source, index)
if (construct !== undefined) return failure('unmappable-html', `no ADF node carries ${construct}`, path) if (construct !== undefined) return failure('unmappable-html', `no ADF node carries ${construct}`, path)
} }
index += backslashEscape(source, index) === undefined ? 1 : 2 if (character === '[' || (character === '!' && source.charAt(index + 1) === '[')) {
const image = character === '!'
const width = image ? 2 : 1
flush(scan, false)
scan.pieces.push({ active: true, image, kind: 'open', start: index + width })
index += width
continue
}
if (character === ']') {
const closed = closeBracket(scan, index)
if (!closed.ok) return closed
index = closed.value
continue
}
if (emphasisCharacters.includes(character)) {
index = readDelimiterRun(scan, index)
continue
}
scan.pending += character
index += 1
} }
takeRun(run, source, source.length, source.length, true) flush(scan, true)
pushText(run) return assemble(scan)
return success(run.nodes)
} }
function takeRun(run: Run, source: string, end: number, resume: number, strip: boolean): void { function flush(scan: Scan, strip: boolean): void {
const raw = source.slice(run.undecodedFrom, end) const raw = strip ? scan.pending.replace(trailingSpace, '') : scan.pending
run.text += decodeTextEscapes(strip ? raw.replace(trailingSpace, '') : raw) scan.pending = ''
run.undecodedFrom = resume if (raw !== '') scan.pieces.push({ kind: 'nodes', nodes: [{ text: decodeTextEscapes(raw), type: 'text' }] })
} }
function pushText(run: Run): void { function pushNode(scan: Scan, node: AdfNode): void {
if (run.text !== '') run.nodes.push({ text: run.text, type: 'text' }) scan.pieces.push({ kind: 'nodes', nodes: [node] })
run.text = ''
} }
function pushNode(run: Run, node: AdfNode): void { function assemble(scan: Scan): Result<InlineContent> {
pushText(run) if (scan.image !== undefined) {
run.nodes.push(node) if (scan.pieces.length > 0) return failure('unmappable-image', imageAlone, scan.path)
return success({ image: scan.image })
}
return success({ nodes: resolveNodes(scan.pieces) })
}
function readDelimiterRun(scan: Scan, index: number): number {
const character = scan.source.charAt(index)
const length = runLength(scan.source, index)
const flags = delimiterFlags(character, scan.source.charAt(index - 1), scan.source.charAt(index + length))
// spec/flavour.md: strike is exactly two tildes.
if ((character === '~' && length !== 2) || (!flags.canOpen && !flags.canClose)) scan.pending += scan.source.slice(index, index + length)
else {
flush(scan, false)
scan.pieces.push({ canClose: flags.canClose, canOpen: flags.canOpen, character, kind: 'run', length })
}
return index + length
}
function readAutolink(source: string, index: number): { length: number; node: AdfNode } | undefined {
const bracketed = readBracketedAutolink(source, index)
if (bracketed !== undefined) return { length: bracketed, node: linkedText(source.slice(index + 1, index + bracketed - 1), '') }
const email = readEmailAutolink(source, index)
if (email === undefined) return undefined
return { length: email, node: linkedText(source.slice(index + 1, index + email - 1), 'mailto:') }
}
function linkedText(text: string, scheme: string): AdfNode {
return { marks: [{ attrs: { href: `${scheme}${text}` }, type: 'link' }], text, type: 'text' }
}
function closeBracket(scan: Scan, index: number): Result<number> {
flush(scan, false)
const open = lastBracket(scan.pieces)
if (open === undefined) return success(literalClose(scan, index))
const target = open.bracket.active ? resolveTarget(scan, open.bracket, index) : undefined
if (target === undefined) {
scan.pieces[open.index] = { kind: 'nodes', nodes: bracketNodes(open.bracket) }
return success(literalClose(scan, index))
}
const inner = scan.pieces.splice(open.index).slice(1)
if (open.bracket.image) {
const built = buildImage(scan, inner, target.definition)
if (!built.ok) return built
} else buildLink(scan, inner, target.definition)
return success(index + 1 + target.length)
}
function literalClose(scan: Scan, index: number): number {
scan.pending += ']'
return index + 1
}
function bracketNodes(bracket: Bracket): AdfNode[] {
return [{ text: bracket.image ? '![' : '[', type: 'text' }]
}
function lastBracket(pieces: readonly Piece[]): { bracket: Bracket; index: number } | undefined {
for (let index = pieces.length - 1; index >= 0; index -= 1) {
const piece = pieces[index]
if (piece?.kind === 'open') return { bracket: piece, index }
}
return undefined
}
function resolveTarget(scan: Scan, bracket: Bracket, index: number): { definition: LinkDefinition; length: number } | undefined {
const after = index + 1
if (scan.source.charAt(after) === '(') {
const inline = readInlineTarget(scan.source, after)
if (inline !== undefined) return { definition: inline.definition, length: inline.length }
}
const label = scan.source.charAt(after) === '[' ? readLabel(scan.source, after) : undefined
const name = label === undefined || label.value === '' ? scan.source.slice(bracket.start, index) : label.value
const definition = scan.definitions.get(normalizeLabel(name))
if (definition === undefined) return undefined
return { definition, length: label?.length ?? 0 }
}
function buildLink(scan: Scan, inner: readonly Piece[], definition: LinkDefinition): void {
const attrs = definition.title === undefined ? { href: definition.destination } : { href: definition.destination, title: definition.title }
const nodes = applyMark(resolveNodes(inner), { attrs, type: 'link' })
// CommonMark: no link nests inside another.
for (const piece of scan.pieces) if (piece.kind === 'open') piece.active = false
scan.pieces.push({ kind: 'nodes', nodes })
}
function buildImage(scan: Scan, inner: readonly Piece[], definition: LinkDefinition): Result<null> {
if (definition.title !== undefined) return failure('unmappable-image', 'no media node carries a link title', scan.path)
if (scan.pieces.length > 0 || scan.image !== undefined) return failure('unmappable-image', imageAlone, scan.path)
const alt = resolveNodes(inner)
.map((node) => node.text ?? '')
.join('')
const attrs = alt === '' ? { type: 'external', url: definition.destination } : { alt, type: 'external', url: definition.destination }
scan.image = { attrs: { layout: 'center' }, content: [{ attrs, type: 'media' }], type: 'mediaSingle' }
return success(null)
}
function resolveNodes(pieces: readonly Piece[]): AdfNode[] {
const nodes = pieces.map((piece) => (piece.kind === 'nodes' ? piece.nodes : piece.kind === 'open' ? bracketNodes(piece) : []))
const runs: Run[] = []
for (const [index, piece] of pieces.entries()) {
if (piece.kind === 'run') runs.push({ canClose: piece.canClose, canOpen: piece.canOpen, character: piece.character, index, length: piece.length })
}
const pairings = matchEmphasis(runs)
const heads = new Map<Run, number>()
const tails = new Map<Run, number>()
for (const pairing of pairings) {
heads.set(pairing.closer, pairing.closerOffset + pairing.used)
tails.set(pairing.opener, pairing.openerOffset)
}
for (const run of runs) {
const head = heads.get(run) ?? 0
const tail = tails.get(run) ?? run.length
if (tail > head) nodes[run.index] = [{ text: run.character.repeat(tail - head), type: 'text' }]
}
for (const pairing of pairings) {
const mark: AdfMark = { type: markType(pairing.opener.character, pairing.used) }
for (let index = pairing.opener.index + 1; index < pairing.closer.index; index += 1) {
nodes[index] = applyMark(nodes[index] ?? [], mark)
}
}
return mergeText(nodes.flat())
}
function markType(character: string, used: number): string {
if (character === '~') return 'strike'
return used === 2 ? 'strong' : 'em'
}
// CommonMark nests a spelling inside its own kind (`*(*a*)*`); the mark it names is idempotent, and
// a node carrying it twice is the shape AGENTS.md §14 has the emitter refuse.
function applyMark(nodes: readonly AdfNode[], mark: AdfMark): AdfNode[] {
return nodes.map((node) => {
const marks = node.marks ?? []
return marks.some((carried) => carried.type === mark.type) ? node : { ...node, marks: [mark, ...marks] }
})
}
// Editor-normal (AGENTS.md §2): adjacent text nodes carrying identical marks are one node.
function mergeText(nodes: readonly AdfNode[]): AdfNode[] {
const merged: AdfNode[] = []
for (const node of nodes) {
const previous = merged[merged.length - 1]
if (previous !== undefined && previous.type === 'text' && node.type === 'text' && markKey(previous) === markKey(node)) {
merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` }
continue
}
merged.push(node)
}
return merged
}
function markKey(node: AdfNode): string {
return (node.marks ?? []).map((mark) => `${mark.type}${serializeCanonicalJson(mark.attrs ?? {}, 'compact')}`).join(' ')
} }
function readCodeSpan(source: string, index: number): { end: number; text: string } | undefined { function readCodeSpan(source: string, index: number): { end: number; text: string } | undefined {
@@ -1,14 +1,9 @@
import { holdsControlCharacter, isAsciiPunctuation } from '../commonmark-grammar.ts' import type { LinkDefinition, LinkPart } from '../link-syntax.ts'
import { normalizeLabel, readDestination, readLabel, readTitle, skipLinkWhitespace } from '../link-syntax.ts'
export type LinkDefinition = { destination: string; title?: string }
type ReadDefinition = { definition: LinkDefinition; label: string; length: number } type ReadDefinition = { definition: LinkDefinition; label: string; length: number }
type ReadValue = { length: number; value: string }
const bracketedDestination = /^<((?:[^\n<>\\]|\\[^\n])*)>/
const label = /^\[((?:[^[\]\\]|\\[\s\S]){1,999})\]:/
const restOfLine = /^[ \t]*(?:\n|$)/ const restOfLine = /^[ \t]*(?:\n|$)/
const titleClosers: Readonly<Record<string, string>> = { '"': '"', "'": "'", '(': ')' }
export function readLinkDefinitions(definitions: Map<string, LinkDefinition>, text: string): string { export function readLinkDefinitions(definitions: Map<string, LinkDefinition>, text: string): string {
let rest = text let rest = text
@@ -22,11 +17,11 @@ export function readLinkDefinitions(definitions: Map<string, LinkDefinition>, te
} }
function readDefinition(text: string): ReadDefinition | undefined { function readDefinition(text: string): ReadDefinition | undefined {
const matched = label.exec(text) const label = readLabel(text, 0)
if (matched === null) return undefined if (label === undefined || text.charAt(label.length) !== ':') return undefined
const name = normalizeLabel(matched[1] ?? '') const name = normalizeLabel(label.value)
if (name === '') return undefined if (name === '') return undefined
const afterLabel = skipSpace(text, matched[0].length) const afterLabel = skipLinkWhitespace(text, label.length + 1)
const destination = readDestination(text, afterLabel) const destination = readDestination(text, afterLabel)
if (destination === undefined) return undefined if (destination === undefined) return undefined
const afterDestination = afterLabel + destination.length const afterDestination = afterLabel + destination.length
@@ -37,15 +32,8 @@ function readDefinition(text: string): ReadDefinition | undefined {
return { definition: { destination: destination.value }, label: name, length: plain } return { definition: { destination: destination.value }, label: name, length: plain }
} }
function normalizeLabel(raw: string): string { function readTitledEnd(text: string, offset: number): LinkPart | undefined {
return raw const afterSpace = skipLinkWhitespace(text, offset)
.replace(/^[ \t\n]+|[ \t\n]+$/g, '')
.replace(/[ \t\n]+/g, ' ')
.toLowerCase()
}
function readTitledEnd(text: string, offset: number): ReadValue | undefined {
const afterSpace = skipSpace(text, offset)
if (afterSpace === offset) return undefined if (afterSpace === offset) return undefined
const title = readTitle(text, afterSpace) const title = readTitle(text, afterSpace)
if (title === undefined) return undefined if (title === undefined) return undefined
@@ -53,58 +41,6 @@ function readTitledEnd(text: string, offset: number): ReadValue | undefined {
return end === undefined ? undefined : { length: end, value: title.value } return end === undefined ? undefined : { length: end, value: title.value }
} }
function readDestination(text: string, offset: number): ReadValue | undefined {
const bracketed = bracketedDestination.exec(text.slice(offset))
if (bracketed !== null) return { length: bracketed[0].length, value: bracketed[0].slice(1, -1) }
if (text.charAt(offset) === '<') return undefined
let depth = 0
let index = offset
while (index < text.length) {
const character = text.charAt(index)
if (character === ' ' || holdsControlCharacter(character)) break
if (escapesNext(text, index)) {
index += 2
continue
}
if (character === '(') depth += 1
if (character === ')') {
depth -= 1
if (depth < 0) break
}
index += 1
}
return index <= offset ? undefined : { length: index - offset, value: text.slice(offset, index) }
}
function readTitle(text: string, offset: number): ReadValue | undefined {
const opener = text.charAt(offset)
const closer = titleClosers[opener]
if (closer === undefined) return undefined
let index = offset + 1
while (index < text.length) {
const character = text.charAt(index)
if (escapesNext(text, index)) {
index += 2
continue
}
if (character === closer) return { length: index + 1 - offset, value: text.slice(offset + 1, index) }
if (character === opener) return undefined
index += 1
}
return undefined
}
// A backslash escapes ASCII punctuation only, so a line ending always ends the destination it follows.
function escapesNext(text: string, index: number): boolean {
return text.charAt(index) === '\\' && isAsciiPunctuation(text.charAt(index + 1))
}
// The label, the destination and the title each take at most one line ending with them.
function skipSpace(text: string, offset: number): number {
const rest = text.slice(offset)
return offset + rest.length - rest.replace(/^[ \t]*\n?[ \t]*/, '').length
}
function endOfLine(text: string, offset: number): number | undefined { function endOfLine(text: string, offset: number): number | undefined {
const rest = restOfLine.exec(text.slice(offset))?.[0] const rest = restOfLine.exec(text.slice(offset))?.[0]
return rest === undefined ? undefined : offset + rest.length return rest === undefined ? undefined : offset + rest.length
+135 -1
View File
@@ -1,10 +1,14 @@
import assert from 'node:assert/strict' import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts'
import type { Result } from '../../result.ts' import type { Result } from '../../result.ts'
import { markdownToAdf } from './markdown-to-adf.ts' import { markdownToAdf } from './markdown-to-adf.ts'
const em: AdfMark = { type: 'em' }
const strike: AdfMark = { type: 'strike' }
const strong: AdfMark = { type: 'strong' }
function code(result: Result<AdfDocument>): string { function code(result: Result<AdfDocument>): string {
return result.ok ? `built ${JSON.stringify(result.value)}` : result.error.code return result.ok ? `built ${JSON.stringify(result.value)}` : result.error.code
} }
@@ -49,6 +53,19 @@ function quote(...content: AdfNode[]): AdfNode {
return content.length === 0 ? { type: 'blockquote' } : { content, type: 'blockquote' } return content.length === 0 ? { type: 'blockquote' } : { content, type: 'blockquote' }
} }
function marked(value: string, ...marks: AdfMark[]): AdfNode {
return { marks, text: value, type: 'text' }
}
function link(href: string, title?: string): AdfMark {
return { attrs: title === undefined ? { href } : { href, title }, type: 'link' }
}
function image(url: string, alt?: string): AdfNode {
const media: AdfNode = { attrs: alt === undefined ? { type: 'external', url } : { alt, type: 'external', url }, type: 'media' }
return { attrs: { layout: 'center' }, content: [media], type: 'mediaSingle' }
}
test('builds an empty document from input holding no block', () => { test('builds an empty document from input holding no block', () => {
assert.deepEqual(markdownToAdf(''), { ok: true, value: { type: 'doc', version: 1 } }) assert.deepEqual(markdownToAdf(''), { ok: true, value: { type: 'doc', version: 1 } })
assert.deepEqual(content(markdownToAdf('\n \n\t\n')), []) assert.deepEqual(content(markdownToAdf('\n \n\t\n')), [])
@@ -323,3 +340,120 @@ test('leaves the angle bracket that opens no HTML construct to the text it sits
assert.deepEqual(content(markdownToAdf('`<span>`\n')), [{ content: [codeSpan('<span>')], type: 'paragraph' }]) assert.deepEqual(content(markdownToAdf('`<span>`\n')), [{ content: [codeSpan('<span>')], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('\\<span>\n')), [paragraph('<span>')]) assert.deepEqual(content(markdownToAdf('\\<span>\n')), [paragraph('<span>')])
}) })
test('reads the emphasis CommonMark matches, the marks nesting outermost first', () => {
assert.deepEqual(content(markdownToAdf('*a*\n')), [{ content: [marked('a', em)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('_a_\n')), [{ content: [marked('a', em)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('**a**\n')), [{ content: [marked('a', strong)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('__a__\n')), [{ content: [marked('a', strong)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('***a***\n')), [{ content: [marked('a', em, strong)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('*a **b** c*\n')), [
{ content: [marked('a ', em), marked('b', em, strong), marked(' c', em)], type: 'paragraph' },
])
assert.deepEqual(content(markdownToAdf('a*b*c\n')), [{ content: [text('a'), marked('b', em), text('c')], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('# *a*\n')), [{ attrs: { level: 1 }, content: [marked('a', em)], type: 'heading' }])
})
test('leaves a delimiter run CommonMark pairs with nothing in the text it sits in', () => {
assert.deepEqual(content(markdownToAdf('a_b_c\n')), [paragraph('a_b_c')])
assert.deepEqual(content(markdownToAdf('*a\n')), [paragraph('*a')])
assert.deepEqual(content(markdownToAdf('a * b\n')), [paragraph('a * b')])
assert.deepEqual(content(markdownToAdf('**a*\n')), [{ content: [text('*'), marked('a', em)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('*a**\n')), [{ content: [marked('a', em), text('*')], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('\\*a\\*\n')), [paragraph('*a*')])
assert.deepEqual(content(markdownToAdf('`*a*`\n')), [{ content: [codeSpan('*a*')], type: 'paragraph' }])
})
test('reads two tildes as strike, a single tilde and a longer run literal', () => {
assert.deepEqual(content(markdownToAdf('~~a~~\n')), [{ content: [marked('a', strike)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('a ~b~ c\n')), [paragraph('a ~b~ c')])
assert.deepEqual(content(markdownToAdf('a ~~~b~~~ c\n')), [paragraph('a ~~~b~~~ c')])
assert.deepEqual(content(markdownToAdf('~~a **b**~~\n')), [{ content: [marked('a ', strike), marked('b', strike, strong)], type: 'paragraph' }])
})
test('reads an inline link, its destination and title', () => {
assert.deepEqual(content(markdownToAdf('[a](/url)\n')), [{ content: [marked('a', link('/url'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a](/url "t")\n')), [{ content: [marked('a', link('/url', 't'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a](\n/url\n"t" )\n')), [{ content: [marked('a', link('/url', 't'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a](<u v>)\n')), [{ content: [marked('a', link('u v'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a]()\n')), [{ content: [marked('a', link(''))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a](/x(y))\n')), [{ content: [marked('a', link('/x(y)'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[**a**](/u)\n')), [{ content: [marked('a', link('/u'), strong)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a `b`](/u)\n')), [
{ content: [marked('a ', link('/u')), { marks: [link('/u'), { type: 'code' }], text: 'b', type: 'text' }], type: 'paragraph' },
])
})
test('decodes the escapes and the references a destination and a title hold', () => {
assert.deepEqual(content(markdownToAdf('[a](/x\\)y)\n')), [{ content: [marked('a', link('/x)y'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a](/u "He said \\"hi\\"")\n')), [{ content: [marked('a', link('/u', 'He said "hi"'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a](/x&amp;y)\n')), [{ content: [marked('a', link('/x&y'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a][r]\n\n[r]: /x\\)y "He said \\"hi\\""\n')), [
{ content: [marked('a', link('/x)y', 'He said "hi"'))], type: 'paragraph' },
])
})
test('leaves the bracket pair no link parses as the text it holds', () => {
assert.deepEqual(content(markdownToAdf('[a\n')), [paragraph('[a')])
assert.deepEqual(content(markdownToAdf('[a] (/u)\n')), [paragraph('[a] (/u)')])
assert.deepEqual(content(markdownToAdf('a ] b\n')), [paragraph('a ] b')])
assert.deepEqual(content(markdownToAdf('[a](/u\n')), [paragraph('[a](/u')])
assert.deepEqual(content(markdownToAdf('[a](/u x)\n')), [paragraph('[a](/u x)')])
assert.deepEqual(content(markdownToAdf('[a](<u\n')), [paragraph('[a](<u')])
assert.deepEqual(content(markdownToAdf('![a\n')), [paragraph('![a')])
assert.deepEqual(content(markdownToAdf('[a [b](/u) c](/v)\n')), [
{ content: [text('[a '), marked('b', link('/u')), text(' c](/v)')], type: 'paragraph' },
])
})
test('reads the reference links a definition resolves, and leaves the rest literal', () => {
assert.deepEqual(content(markdownToAdf('[a][r]\n\n[r]: /url\n')), [{ content: [marked('a', link('/url'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a][]\n\n[a]: /url\n')), [{ content: [marked('a', link('/url'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a]\n\n[a]: /url\n')), [{ content: [marked('a', link('/url'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[Foo\nBar][]\n\n[foo bar]: /url\n')), [{ content: [marked('Foo Bar', link('/url'))], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('[a][z]\n\n[a]: /url\n')), [paragraph('[a][z]')])
assert.deepEqual(content(markdownToAdf('[a]\n')), [paragraph('[a]')])
assert.deepEqual(content(markdownToAdf('[][]\n')), [paragraph('[][]')])
})
test('reads an autolink, the email form as the mailto link it means', () => {
assert.deepEqual(content(markdownToAdf('<https://example.com/>\n')), [
{ content: [marked('https://example.com/', link('https://example.com/'))], type: 'paragraph' },
])
assert.deepEqual(content(markdownToAdf('a <a@b.example.com> c\n')), [
{ content: [text('a '), marked('a@b.example.com', link('mailto:a@b.example.com')), text(' c')], type: 'paragraph' },
])
assert.deepEqual(content(markdownToAdf('<https://example.com/?a=\\*>\n')), [
{ content: [marked('https://example.com/?a=\\*', link('https://example.com/?a=\\*'))], type: 'paragraph' },
])
assert.equal(code(markdownToAdf('<https://example.com/> <span>\n')), 'unmappable-html')
})
test('reads a lone image as the media the flavour spells for it', () => {
assert.deepEqual(content(markdownToAdf('![The moon](https://example.com/moon.png)\n')), [
image('https://example.com/moon.png', 'The moon'),
])
assert.deepEqual(content(markdownToAdf('![](/u)\n')), [image('/u')])
assert.deepEqual(content(markdownToAdf('![*a*](/u)\n')), [image('/u', 'a')])
assert.deepEqual(content(markdownToAdf('- ![a](/u)\n')), [bulletList(item(image('/u', 'a')))])
})
test('refuses the image no ADF node carries where it sits', () => {
assert.equal(content(markdownToAdf('![a](/u "t")\n')), 'unmappable-image: no media node carries a link title')
assert.equal(content(markdownToAdf('See ![a](/u).\n')), 'unmappable-image: an image fits only as a paragraph of its own')
assert.equal(code(markdownToAdf('# ![a](/u)\n')), 'unmappable-image')
assert.equal(code(markdownToAdf('*![a](/u)*\n')), 'unmappable-image')
assert.equal(code(markdownToAdf('[![a](/u)](/v)\n')), 'unmappable-image')
assert.equal(code(markdownToAdf('![a](/u)![b](/v)\n')), 'unmappable-image')
assert.deepEqual(path(markdownToAdf('Part.\n\nSee ![a](/u).\n')), ['content', 1])
assert.deepEqual(content(markdownToAdf('![a]\n')), [paragraph('![a]')])
assert.deepEqual(content(markdownToAdf('a ! b\n')), [paragraph('a ! b')])
})
test('carries the mark a spelling nested inside its own kind names once', () => {
assert.deepEqual(content(markdownToAdf('*(*a*)*\n')), [{ content: [marked('(a)', em)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf(`${'*'.repeat(600)}a${'*'.repeat(600)}\n`)), [{ content: [marked('a', strong)], type: 'paragraph' }])
assert.deepEqual(content(markdownToAdf('*a **b** c*\n')), [
{ content: [marked('a ', em), marked('b', em, strong), marked(' c', em)], type: 'paragraph' },
])
})
+23 -16
View File
@@ -1,60 +1,62 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import type { Block, ClaimedConstruct } from './blocks.ts' import type { Block, ClaimedConstruct } from './blocks.ts'
import type { LinkDefinitions } from './inline-content.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { parseBlocks } from './blocks.ts' import { parseBlocks } from './blocks.ts'
import { parseInlineContent } from './inline-content.ts' import { parseInlineContent } from './inline-content.ts'
export function markdownToAdf(markdown: string): Result<AdfDocument> { export function markdownToAdf(markdown: string): Result<AdfDocument> {
const content = blockNodes(parseBlocks(markdown).blocks, [], 0) const parsed = parseBlocks(markdown)
const content = blockNodes(parsed.blocks, parsed.definitions, [], 0)
if (!content.ok) return content if (!content.ok) return content
return success(content.value.length === 0 ? { type: 'doc', version: 1 } : { content: content.value, type: 'doc', version: 1 }) return success(content.value.length === 0 ? { type: 'doc', version: 1 } : { content: content.value, type: 'doc', version: 1 })
} }
function blockNodes(blocks: readonly Block[], path: ConvertErrorPath, depth: number): Result<AdfNode[]> { function blockNodes(blocks: readonly Block[], definitions: LinkDefinitions, path: ConvertErrorPath, depth: number): Result<AdfNode[]> {
if (depth > largestNesting) return failure('unsupported-nesting-depth', `the input nests deeper than the ${largestNesting} levels the parser carries`, path) if (depth > largestNesting) return failure('unsupported-nesting-depth', `the input nests deeper than the ${largestNesting} levels the parser carries`, path)
const content: AdfNode[] = [] const content: AdfNode[] = []
for (const [index, block] of blocks.entries()) { for (const [index, block] of blocks.entries()) {
const node = blockNode(block, [...path, 'content', index], depth) const node = blockNode(block, definitions, [...path, 'content', index], depth)
if (!node.ok) return node if (!node.ok) return node
content.push(node.value) content.push(node.value)
} }
return success(content) return success(content)
} }
function blockNode(block: Block, path: ConvertErrorPath, depth: number): Result<AdfNode> { function blockNode(block: Block, definitions: LinkDefinitions, path: ConvertErrorPath, depth: number): Result<AdfNode> {
switch (block.kind) { switch (block.kind) {
case 'blockquote': case 'blockquote':
return containerNode({ type: 'blockquote' }, block.blocks, path, depth) return containerNode({ type: 'blockquote' }, block.blocks, definitions, path, depth)
case 'bulletList': case 'bulletList':
return listNode({ type: 'bulletList' }, block.items, path, depth) return listNode({ type: 'bulletList' }, block.items, definitions, path, depth)
case 'claim': case 'claim':
return claimFailure(block.construct, path) return claimFailure(block.construct, path)
case 'code': case 'code':
return success(codeBlockNode(block.language, block.text)) return success(codeBlockNode(block.language, block.text))
case 'heading': case 'heading':
return contentNode({ attrs: { level: block.level }, type: 'heading' }, block.text, path) return contentNode({ attrs: { level: block.level }, type: 'heading' }, block.text, definitions, path)
case 'html': case 'html':
return failure('unmappable-html', `no ADF node carries ${block.construct}`, path) return failure('unmappable-html', `no ADF node carries ${block.construct}`, path)
case 'orderedList': case 'orderedList':
return listNode({ attrs: { order: block.start }, type: 'orderedList' }, block.items, path, depth) return listNode({ attrs: { order: block.start }, type: 'orderedList' }, block.items, definitions, path, depth)
case 'paragraph': case 'paragraph':
return contentNode({ type: 'paragraph' }, block.text, path) return contentNode({ type: 'paragraph' }, block.text, definitions, path)
case 'rule': case 'rule':
return success({ type: 'rule' }) return success({ type: 'rule' })
} }
} }
function containerNode(node: AdfNode, blocks: readonly Block[], path: ConvertErrorPath, depth: number): Result<AdfNode> { function containerNode(node: AdfNode, blocks: readonly Block[], definitions: LinkDefinitions, path: ConvertErrorPath, depth: number): Result<AdfNode> {
const content = blockNodes(blocks, path, depth + 1) const content = blockNodes(blocks, definitions, path, depth + 1)
if (!content.ok) return content if (!content.ok) return content
return success(content.value.length === 0 ? node : { ...node, content: content.value }) return success(content.value.length === 0 ? node : { ...node, content: content.value })
} }
function listNode(node: AdfNode, items: readonly Block[][], path: ConvertErrorPath, depth: number): Result<AdfNode> { function listNode(node: AdfNode, items: readonly Block[][], definitions: LinkDefinitions, path: ConvertErrorPath, depth: number): Result<AdfNode> {
const content: AdfNode[] = [] const content: AdfNode[] = []
for (const [index, blocks] of items.entries()) { for (const [index, blocks] of items.entries()) {
const item = containerNode({ type: 'listItem' }, blocks, [...path, 'content', index], depth) const item = containerNode({ type: 'listItem' }, blocks, definitions, [...path, 'content', index], depth)
if (!item.ok) return item if (!item.ok) return item
content.push(item.value) content.push(item.value)
} }
@@ -75,8 +77,13 @@ function codeBlockNode(language: string, text: string): AdfNode {
return text === '' ? node : { ...node, content: [{ text, type: 'text' }] } return text === '' ? node : { ...node, content: [{ text, type: 'text' }] }
} }
function contentNode(node: AdfNode, text: string, path: ConvertErrorPath): Result<AdfNode> { // spec/flavour.md, The CommonMark image: only a paragraph gives an image the block it needs.
const content = parseInlineContent(text, path) function contentNode(node: AdfNode, text: string, definitions: LinkDefinitions, path: ConvertErrorPath): Result<AdfNode> {
const content = parseInlineContent(text, definitions, path)
if (!content.ok) return content if (!content.ok) return content
return success(content.value.length === 0 ? node : { ...node, content: content.value }) const image = content.value.image
if (image !== undefined) {
return node.type === 'paragraph' ? success(image) : failure('unmappable-image', `no ADF node carries an image inside a ${node.type}`, path)
}
return success(content.value.nodes.length === 0 ? node : { ...node, content: content.value.nodes })
} }
+1
View File
@@ -3,6 +3,7 @@ export type ConvertErrorCode =
| 'malformed-pipe-table' | 'malformed-pipe-table'
| 'not-an-adf-document' | 'not-an-adf-document'
| 'unmappable-html' | 'unmappable-html'
| 'unmappable-image'
| 'unspellable-adjacent-lists' | 'unspellable-adjacent-lists'
| 'unspellable-character' | 'unspellable-character'
| 'unspellable-line-start' | 'unspellable-line-start'
+12 -4
View File
@@ -240,7 +240,7 @@ detail is settled at its own milestone.
autolink parting off as 3e's own predicate: refusing on the superset would refuse autolink parting off as 3e's own predicate: refusing on the superset would refuse
`1 <b 2`, a fourth carve-out §4 and the README do not list. `holdsEntityReference` reads `1 <b 2`, a fourth carve-out §4 and the README do not list. `holdsEntityReference` reads
the table for the same reason, so `&notareference;` is emitted bare. the table for the same reason, so `&notareference;` is emitted bare.
- [ ] **3e — Emphasis and links.** `_`, `*` and `~~` runs through `matchEmphasis` to the `em`, - [x] **3e — Emphasis and links.** `_`, `*` and `~~` runs through `matchEmphasis` to the `em`,
`strong` and `strike` marks; links inline and reference, 3b's definitions resolved here, `strong` and `strike` marks; links inline and reference, 3b's definitions resolved here,
autolinks, and the image gap's named errors — a titled image, and one amid other text. autolinks, and the image gap's named errors — a titled image, and one amid other text.
`spec/flavour.md` does not yet pin `~`'s `can_open`/`can_close`, which is transcription `spec/flavour.md` does not yet pin `~`'s `can_open`/`can_close`, which is transcription
@@ -252,6 +252,14 @@ detail is settled at its own milestone.
where nothing else works, so each construct keeps one spelling and a destination holding where nothing else works, so each construct keeps one spelling and a destination holding
both composes. `link-destination-parenthesis` and `link-title-quote` become round-trip both composes. `link-destination-parenthesis` and `link-title-quote` become round-trip
pairs. pairs.
**Settled** (the maintainer, 2026-08-31): the destination escapes only the parenthesis it
leaves unbalanced, so `/wiki/Foo_(bar)` keeps its bytes, and the backslash stays refused in
both the destination and the title — which leaves `[a](/a\b)` a §2 hole 3k's exception list
answers, as `<http://x?a=1&amp;b=2>` is, autolinks decoding neither escapes nor references.
The image gap mints `unmappable-image`, mirroring `unmappable-html` — a construct in input
no ADF node carries. And the CommonMark image shape lands here rather than at 3h: once
`[]()` reads, a lone `![alt](url)` would otherwise misparse as text plus a link, so 3h
keeps the rest of the media family and loses only that line.
- [ ] **3f — The directive grammar.** The three forms — inline `:name[content]{attrs}`, - [ ] **3f — The directive grammar.** The three forms — inline `:name[content]{attrs}`,
container `:::name arg {attrs}`, leaf `::name arg {attrs}` — the attribute grammar with container `:::name arg {attrs}`, leaf `::name arg {attrs}` — the attribute grammar with
its quoting and escapes, the fence-length and nesting rules, and the malformed list its quoting and escapes, the fence-length and nesting rules, and the malformed list
@@ -269,9 +277,9 @@ detail is settled at its own milestone.
guard is built here if the answer there was yes. guard is built here if the answer there was yes.
- [ ] **3h — The block nodes.** `block-nodes/` reads back: the plain-versus-directive choice 2f - [ ] **3h — The block nodes.** `block-nodes/` reads back: the plain-versus-directive choice 2f
settles, read from the other side; the `codeBlock` directive's fenced body and the settles, read from the other side; the `codeBlock` directive's fenced body and the
`language` attribute a bare fence leaves it; the media family's composition and the `language` attribute a bare fence leaves it; the media family's composition; both table
CommonMark image shape; both table forms, the pipe table's cell split and its named forms, the pipe table's cell split and its named errors; and the one paragraph an inline
errors; and the one paragraph an inline body takes. body takes.
**Settled** (the maintainer, 2026-08-27): 1d's last pick, the one **Settled** (the maintainer, 2026-08-27): 1d's last pick, the one
`container-block-separation` holds — a CommonMark block and a directive block sit adjacent `container-block-separation` holds — a CommonMark block and a directive block sit adjacent
in a container body with no blank line between them. That reduces the three cases to one in a container body with no blank line between them. That reduces the three cases to one