Read the inline nodes and the marks back, and keep the whitespace CommonMark does not strip #41

Merged
lilleman merged 5 commits from inline-nodes-read-back into main 2026-09-02 12:34:20 +02:00
24 changed files with 394 additions and 93 deletions
+1 -1
View File
@@ -1 +1 @@
Part :mention[@Mikael]{id=b1c2}. Part :mention[**@Mikael**]{id=b1c2}.
@@ -0,0 +1 @@
unsupported-node-shape
+1
View File
@@ -0,0 +1 @@
Part :em[emphasised] here.
+1
View File
@@ -0,0 +1 @@
unsupported-node-shape
+1
View File
@@ -0,0 +1 @@
Part :text{text=plain} here.
@@ -98,7 +98,7 @@
{ {
"content": [ "content": [
{ {
"text": "Sold as a pair", "text": "\fSold as a pair\u000b",
"type": "text" "type": "text"
} }
], ],
+2 -2
View File
@@ -1,6 +1,6 @@
| Part | Note | | Part | Note |
| --- | --- | | --- | --- |
| Bolt M8 | Grade `8.8` | | Bolt M8 | Grade `8.8` |
| Nut \| washer | Sold as a pair | | Nut \| washer | Sold as a pair |
| Washer M8 | 100 pcs:hardBreak{}zinc-plated | | Washer M8 | 100 pcs:hardBreak{}zinc-plated |
| Spare | | | :text{text=" "}Spare | |
@@ -80,6 +80,15 @@
} }
], ],
"type": "paragraph" "type": "paragraph"
},
{
"content": [
{
"text": "\u000bEdged with the whitespace CommonMark keeps\f",
"type": "text"
}
],
"type": "paragraph"
} }
], ],
"type": "doc", "type": "doc",
@@ -11,3 +11,5 @@ Space after the break\
:text{text=" "}survives too. :text{text=" "}survives too.
Kept _:text{text=" "}inside:text{text=" "}_, exactly. Kept _:text{text=" "}inside:text{text=" "}_, exactly.
Edged with the whitespace CommonMark keeps
+1 -1
View File
@@ -13,7 +13,7 @@
"node": ">=18" "node": ">=18"
}, },
"scripts": { "scripts": {
"test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=97 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"", "test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=97.8 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"",
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.build.json" "typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.build.json"
}, },
"devDependencies": { "devDependencies": {
+10 -5
View File
@@ -62,7 +62,9 @@ node and mark names the sections below spell as directives. Recognition is synta
name-set-independent: anything matching the forms below parses as a directive regardless of name-set-independent: anything matching the forms below parses as a directive regardless of
whether the name is known, and an unknown name is an error result naming it — so output an old whether the name is known, and an unknown name is an error result naming it — so output an old
emitter escaped stays escaped, and erroring input gaining meaning later is MINOR, never a reparse emitter escaped stays escaped, and erroring input gaining meaning later is MINOR, never a reparse
(§8). The name `adf` is reserved for the opaque carry, as both directive name and fence info (§8). Each name belongs to one position, and a name the other one spells — a mark or an inline
node written as a block directive, a block node written inline — is a different error, naming the
spelling it takes. The name `adf` is reserved for the opaque carry, as both directive name and fence info
string. string.
**Inline**: `:name[content]{attrs}`, on one line — an inline directive never spans lines. **Inline**: `:name[content]{attrs}`, on one line — an inline directive never spans lines.
@@ -260,9 +262,10 @@ The moon, at night.
**The CommonMark image.** A paragraph whose entire inline content is one image `![alt](url)` is **The CommonMark image.** A paragraph whose entire inline content is one image `![alt](url)` is
a `mediaSingle` with attrs exactly `{"layout":"center"}` holding an `external` `media` — `url` a `mediaSingle` with attrs exactly `{"layout":"center"}` holding an `external` `media` — `url`
from the destination, `alt` the description's plain-text content when non-empty — a link or image from the destination, `alt` the description's plain-text content when non-empty — a link or image
inside it contributing its own text, and a break of either kind a space. `adfToMarkdown` inside it contributing its own text, a node spelling its text in the content slot contributing
emits the image form for exactly that shape — those attrs and no others, no marks on either that text, and a break of either kind a space. `adfToMarkdown` emits the image form for exactly
node, no caption, and a `media` carrying nothing beyond `alt`, `type` and `url` — and only where that shape — those attrs and no others, no marks on either node, no caption, and a `media`
carrying nothing beyond `alt`, `type` and `url` — and only where
CommonMark spells the pair: a destination or a description the image form cannot hold, an empty CommonMark spells the pair: a destination or a description the image form cannot hold, an empty
`alt` included, takes the directive form instead. An image amid `alt` included, takes the directive form instead. An image amid
other text, or one carrying a title, is a named error: `mediaInline` carries a media other text, or one carrying a title, is a named error: `mediaInline` carries a media
@@ -421,7 +424,9 @@ an em, strong or strike spelling's inner edges, a pipe cell's edges — is spell
`:text{text="…"}`, the reserved key carrying the node's text, escaped by the attribute grammar `:text{text="…"}`, the reserved key carrying the node's text, escaped by the attribute grammar
and never literal: pipe cells trim and pad. The emitter wraps the whitespace run alone and leaves and never literal: pipe cells trim and pad. The emitter wraps the whitespace run alone and leaves
the rest plain text; `markdownToAdf` merges adjacent text nodes carrying identical marks the rest plain text; `markdownToAdf` merges adjacent text nodes carrying identical marks
(AGENTS.md §2). (AGENTS.md §2). Input reads that spelling alone: the value is one run of spaces and tabs, or one
run of newlines, and anything else — a mixed run, or text CommonMark carries plainly — is a named
error.
``` ```
:text{text=" "}Two leading spaces held, and one text node split:text{text="\n"}over two lines. :text{text=" "}Two leading spaces held, and one text node split:text{text="\n"}over two lines.
+1 -1
View File
@@ -17,7 +17,7 @@ const roundTripRoot = join(corpusRoot, 'round-trip')
const emittingDirectories = ['block-nodes', 'combinations', 'commonmark-subset', 'inline-nodes', 'opaque-carry'] const emittingDirectories = ['block-nodes', 'combinations', 'commonmark-subset', 'inline-nodes', 'opaque-carry']
// A directory joins once every fixture in it reads back to its document. // A directory joins once every fixture in it reads back to its document.
const parsingDirectories = ['block-nodes', 'commonmark-subset'] const parsingDirectories = ['block-nodes', 'commonmark-subset', 'inline-nodes']
function directoryNames(root: string): string[] { function directoryNames(root: string): string[] {
return readdirSync(root, { withFileTypes: true }) return readdirSync(root, { withFileTypes: true })
+10
View File
@@ -77,6 +77,12 @@ export function readInlineDirective(text: string, index: number): Read<Directive
return readNestedDirective(text, index, 1) return readNestedDirective(text, index, 1)
} }
// Both directions answer alike: an inline directive never spans lines, so no content slot holds a line ending.
export function slotLineEndingFault(type: string, text: string): ConvertFault | undefined {
if (!/[\n\r]/.test(text)) return undefined
return { code: 'unspellable-whitespace', message: `the ${type} content slot holds a newline no inline directive spans` }
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string { export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return '' if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`) const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`)
@@ -87,6 +93,10 @@ export function spellJsonAttribute(value: JsonValue): string {
return quote(serializeCanonicalJson(value, 'compact')) return quote(serializeCanonicalJson(value, 'compact'))
} }
export function spellLeafDirective(name: string, attributes: string): string {
return `:${name}${attributes === '' ? '{}' : attributes}`
}
export function spellStringAttribute(text: string): string { export function spellStringAttribute(text: string): string {
return isBareToken(text) ? text : quote(text) return isBareToken(text) ? text : quote(text)
} }
+3 -2
View File
@@ -130,7 +130,6 @@ test('carries a mark the canonical spellings cannot nest', () => {
}) })
test('refuses whitespace CommonMark cannot hold', () => { test('refuses whitespace CommonMark cannot hold', () => {
assert.equal(code(adfToMarkdown(document(paragraph({ text: '\fa', type: 'text' })))), 'unspellable-whitespace')
assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), 'unspellable-whitespace') assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\rb', type: 'text' })))), 'unspellable-whitespace')
}) })
@@ -535,7 +534,7 @@ test('spells a table as a pipe table only where every row and cell is plain', ()
markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ attrs: { url: 'a|b' }, type: 'blockCard' }], type: 'paragraph' }))))), markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ attrs: { url: 'a|b' }, type: 'blockCard' }], type: 'paragraph' }))))),
'| :adf{json="{\\"attrs\\":{\\"url\\":\\"a\\u007cb\\"},\\"type\\":\\"blockCard\\"}"} |\n| --- |\n', '| :adf{json="{\\"attrs\\":{\\"url\\":\\"a\\u007cb\\"},\\"type\\":\\"blockCard\\"}"} |\n| --- |\n',
) )
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: '\fa', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace') assert.equal(markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: '\fa', type: 'text' }], type: 'paragraph' }))))), '| \fa |\n| --- |\n')
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' })))))) assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' }))))))
assert.equal(markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), '| :text{text=" "}a |\n| --- |\n') assert.equal(markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), '| :text{text=" "}a |\n| --- |\n')
const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' }))) const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' })))
@@ -621,4 +620,6 @@ test('carries whitespace CommonMark strips in the reserved text directive', () =
assert.equal(emitted({ text: 'a ', type: 'text' }, { type: 'hardBreak' }, { text: ' b', type: 'text' }), 'a:text{text=" "}\\\n:text{text=" "}b\n') assert.equal(emitted({ text: 'a ', type: 'text' }, { type: 'hardBreak' }, { text: ' b', type: 'text' }), 'a:text{text=" "}\\\n:text{text=" "}b\n')
assert.equal(emitted({ marks: [{ type: 'em' }], text: ' a ', type: 'text' }), '_:text{text=" "}a:text{text=" "}_\n') assert.equal(emitted({ marks: [{ type: 'em' }], text: ' a ', type: 'text' }), '_:text{text=" "}a:text{text=" "}_\n')
assert.equal(markdown(adfToMarkdown(document({ attrs: { level: 1 }, content: [{ text: 'x ', type: 'text' }], type: 'heading' }))), '# x:text{text=" "}\n') assert.equal(markdown(adfToMarkdown(document({ attrs: { level: 1 }, content: [{ text: 'x ', type: 'text' }], type: 'heading' }))), '# x:text{text=" "}\n')
// CommonMark strips spaces and tabs alone, so the whitespace beside them is plain text.
assert.equal(emitted({ text: '\va\f', type: 'text' }), '\va\f\n')
}) })
+6 -11
View File
@@ -3,16 +3,17 @@ import type { InlineDirective } from '../../adf/inline-directives.ts'
import { assembleInlineLine, type InlineEscaping, type InlineSegment, type LineContainer, type NodeRange } from './line-escaping.ts' import { assembleInlineLine, type InlineEscaping, type InlineSegment, type LineContainer, type NodeRange } from './line-escaping.ts'
import { carriedInline } from '../opaque-carry.ts' import { carriedInline } from '../opaque-carry.ts'
import { claimsLine, holdsNullCharacter, isAutolink } from '../commonmark-grammar.ts' import { claimsLine, holdsNullCharacter, isAutolink } from '../commonmark-grammar.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { holdsEntityReference } from '../entity-references.ts' import { holdsEntityReference } from '../entity-references.ts'
import { inlineDirective } from '../../adf/inline-directives.ts' import { inlineDirective } from '../../adf/inline-directives.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../backtick-runs.ts' import { longestBacktickRun } from '../backtick-runs.ts'
import { markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { markSpelling, spellMarkAttributes } from '../mark-spellings.ts'
import { sameMark } from '../../adf/editor-normal.ts' import { sameMark } from '../../adf/editor-normal.ts'
import { spellAttributes, spellStringAttribute } from '../directive-syntax.ts' import { slotLineEndingFault, spellLeafDirective } from '../directive-syntax.ts'
import { spellDestination, spellTitle } from '../link-syntax.ts' import { spellDestination, spellTitle } from '../link-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
import { spellTextDirective } from '../text-directive.ts'
type EmittedLine = { line: string; segments: InlineSegment[] } type EmittedLine = { line: string; segments: InlineSegment[] }
@@ -84,9 +85,6 @@ function attemptLine(segments: readonly InlineSegment[], container: LineContaine
const assembled = assembleInlineLine(segments, container) const assembled = assembleInlineLine(segments, container)
if (assembled.unspellableRun !== undefined) return success({ carry: assembled.unspellableRun }) if (assembled.unspellableRun !== undefined) return success({ carry: assembled.unspellableRun })
for (const [index, single] of assembled.line.split('\n').entries()) { for (const [index, single] of assembled.line.split('\n').entries()) {
if (/^[ \t\v\f]|[ \t\v\f]$/.test(single)) {
return failure('unspellable-whitespace', 'a line begins or ends with whitespace CommonMark strips', path)
}
if (container === 'paragraph' && claimsLine(single, index === 0 ? 'first' : 'later')) { if (container === 'paragraph' && claimsLine(single, index === 0 ? 'first' : 'later')) {
return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(single)}`, path) return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(single)}`, path)
} }
@@ -121,11 +119,7 @@ function carryEdges(segment: InlineSegment, leading: boolean, trailing: boolean)
} }
function carriedText(text: string): InlineSegment { function carriedText(text: string): InlineSegment {
return syntax(spellLeafDirective('text', spellAttributes([['text', spellStringAttribute(text)]]))) return syntax(spellTextDirective(text))
}
function spellLeafDirective(name: string, attributes: string): string {
return `:${name}${attributes === '' ? '{}' : attributes}`
} }
function syntax(text: string): InlineSegment { function syntax(text: string): InlineSegment {
@@ -213,7 +207,8 @@ function emitInlineDirective(node: AdfNode, directive: InlineDirective, index: n
const slot = directive.textAttribute === undefined ? undefined : node.attrs?.[directive.textAttribute] const slot = directive.textAttribute === undefined ? undefined : node.attrs?.[directive.textAttribute]
if (slot === undefined) return success({ segments: [syntax(spellLeafDirective(node.type, attributes))] }) if (slot === undefined) return success({ segments: [syntax(spellLeafDirective(node.type, attributes))] })
if (typeof slot !== 'string') return success({ carry: { first: index, last: index } }) if (typeof slot !== 'string') return success({ carry: { first: index, last: index } })
if (/[\n\r]/.test(slot)) return failure('unspellable-whitespace', `a ${node.type} content slot holds a newline no inline directive spans`, path) const spans = slotLineEndingFault(node.type, slot)
if (spans !== undefined) return faulted(spans, path)
if (holdsNullCharacter(slot)) return failure('unspellable-character', `a ${node.type} content slot holds a null character CommonMark replaces`, path) if (holdsNullCharacter(slot)) return failure('unspellable-character', `a ${node.type} content slot holds a null character CommonMark replaces`, path)
const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }] const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }]
return success({ segments: [syntax(`:${node.type}[`), ...content, syntax(`]${attributes}`)] }) return success({ segments: [syntax(`:${node.type}[`), ...content, syntax(`]${attributes}`)] })
@@ -0,0 +1,31 @@
import type { AdfAttributes } from '../../adf/document.ts'
import type { AttributeVocabulary } from '../../adf/attribute-vocabulary.ts'
import type { DirectiveAttributes } from '../directive-syntax.ts'
import { attributeValue, spellAttributeValue } from '../directive-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
export type Elsewhere = { key: string; slot: 'argument' | 'content' }
export function readVocabulary(
type: string,
attributes: DirectiveAttributes,
vocabulary: AttributeVocabulary,
elsewhere: Elsewhere | undefined,
path: ConvertErrorPath,
): Result<AdfAttributes> {
const attrs: AdfAttributes = {}
for (const [key, spelled] of attributes) {
if (key === elsewhere?.key) {
const place = elsewhere.slot === 'argument' ? 'as the directive argument' : 'in the content slot'
return failure('unsupported-node-shape', `${type} spells its ${key} attribute ${place}`, path)
}
const kind = Object.hasOwn(vocabulary, key) ? vocabulary[key] : undefined
if (kind === undefined) return failure('unsupported-node-shape', `${type} holds no ${key} attribute`, path)
const read = attributeValue(spelled.decoded, kind)
if (read === undefined) return failure('unsupported-node-shape', `the ${key} attribute of ${type} is no ${kind}`, path)
const spelling = spellAttributeValue(read)
if (spelling !== spelled.spelling) return failure('unsupported-node-shape', `${type} spells its ${key} attribute as ${key}=${spelling}`, path)
attrs[key] = read.value
}
return success(attrs)
}
+36
View File
@@ -0,0 +1,36 @@
import type { AdfMark } from '../../adf/document.ts'
import type { ConvertFault } from '../../result.ts'
import type { DirectiveAttributes } from '../directive-syntax.ts'
import type { MarkSpelling } from '../mark-spellings.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { markSpelling } from '../mark-spellings.ts'
import { readVocabulary } from './directive-attributes.ts'
export function readDirectiveMark(name: string, attributes: DirectiveAttributes, path: ConvertErrorPath): Result<AdfMark> | undefined {
const spelling = markSpelling(name)
if (spelling === undefined) return undefined
const markdown = markdownForm(spelling)
if (markdown !== undefined) return failure('unsupported-node-shape', `${name} is spelled ${markdown}, never as a directive`, path)
const attrs = readVocabulary(name, attributes, spelling.attributes, undefined, path)
if (!attrs.ok) return attrs
return success(Object.keys(attrs.value).length === 0 ? { type: name } : { attrs: attrs.value, type: name })
}
export function inlineMarkSpellingFault(name: string): ConvertFault | undefined {
const spelling = markSpelling(name)
if (spelling === undefined) return undefined
return { code: 'unsupported-node-shape', message: `${name} is spelled ${markdownForm(spelling) ?? `:${name}[…]`}, never as a block directive` }
}
function markdownForm(spelling: MarkSpelling): string | undefined {
switch (spelling.kind) {
case 'code':
return '`x`'
case 'directive':
return undefined
case 'emphasis':
return `${spelling.spelling}x${spelling.spelling}`
case 'link':
return '[x](url)'
}
}
+45 -34
View File
@@ -1,19 +1,22 @@
import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts'
import type { AttributeVocabulary } from '../../adf/attribute-vocabulary.ts'
import type { BlockDirective } from '../../adf/block-directives.ts' import type { BlockDirective } from '../../adf/block-directives.ts'
import type { DirectiveAttributes, DirectiveSpan, DirectiveValue } from '../directive-syntax.ts' import type { ConvertFault } from '../../result.ts'
import type { DirectiveAttributes, DirectiveValue } from '../directive-syntax.ts'
import type { Elsewhere } from './directive-attributes.ts'
import { attributeValue, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts' import { attributeValue, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts'
import { blockArgument } from '../block-directive-arguments.ts' import { blockArgument } from '../block-directive-arguments.ts'
import { blockDirective } from '../../adf/block-directives.ts' import { blockDirective } from '../../adf/block-directives.ts'
import { carryName } from '../opaque-carry.ts' import { carryName } from '../opaque-carry.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { inlineDirective } from '../../adf/inline-directives.ts' import { inlineDirective } from '../../adf/inline-directives.ts'
import { inlineMarkSpellingFault } from './directive-marks.ts'
import { marksAttribute, readMarkValues } from '../block-directive-marks.ts' import { marksAttribute, readMarkValues } from '../block-directive-marks.ts'
import { readVocabulary } from './directive-attributes.ts'
import { slotLineEndingFault } from '../directive-syntax.ts'
import { textDirectiveName } from '../text-directive.ts'
export type BlockDirectiveNode = { contentModel: BlockDirective['contentModel']; node: AdfNode } export type BlockDirectiveNode = { contentModel: BlockDirective['contentModel']; node: AdfNode }
type Elsewhere = { key: string; slot: 'argument' | 'content' }
export function readBlockDirectiveNode( export function readBlockDirectiveNode(
name: string, name: string,
argument: string | undefined, argument: string | undefined,
@@ -24,7 +27,7 @@ export function readBlockDirectiveNode(
return failure('malformed-directive', `the name ${carryName} is reserved for the opaque carry, whose block form is the fence`, path) return failure('malformed-directive', `the name ${carryName} is reserved for the opaque carry, whose block form is the fence`, path)
} }
const directive = blockDirective(name) const directive = blockDirective(name)
if (directive === undefined) return faulted(unknownDirectiveFault(name), path) if (directive === undefined) return faulted(inlineSpellingFault(name) ?? unknownDirectiveFault(name), path)
const argumentKey = blockArgument(name) const argumentKey = blockArgument(name)
const rest = new Map(attributes) const rest = new Map(attributes)
rest.delete(marksAttribute) rest.delete(marksAttribute)
@@ -41,42 +44,50 @@ export function readBlockDirectiveNode(
return success({ contentModel: directive.contentModel, node: namedNode(name, attrs.value, marks.value) }) return success({ contentModel: directive.contentModel, node: namedNode(name, attrs.value, marks.value) })
} }
export function readInlineDirectiveNode(span: DirectiveSpan, path: ConvertErrorPath): Result<AdfNode> { export function readInlineDirectiveNode(
const directive = inlineDirective(span.name) name: string,
if (directive === undefined) return faulted(unknownDirectiveFault(span.name), path) attributes: DirectiveAttributes,
content: readonly AdfNode[] | undefined,
path: ConvertErrorPath,
): Result<AdfNode> {
const directive = inlineDirective(name)
if (directive === undefined) return faulted(blockSpellingFault(name) ?? unknownDirectiveFault(name), path)
const slot = directive.textAttribute const slot = directive.textAttribute
if (span.content !== undefined) { if (slot === undefined && content !== undefined) return failure('unsupported-node-shape', `${name} takes no content`, path)
const message = slot === undefined ? `${span.name} takes no content` : `the content slot ${span.name} spells its ${slot} attribute in is unsupported`
return failure('unsupported-node-shape', message, path)
}
const elsewhere: Elsewhere | undefined = slot === undefined ? undefined : { key: slot, slot: 'content' } const elsewhere: Elsewhere | undefined = slot === undefined ? undefined : { key: slot, slot: 'content' }
const attrs = readVocabulary(span.name, span.attributes, directive.attributes, elsewhere, path) const attrs = readVocabulary(name, attributes, directive.attributes, elsewhere, path)
if (!attrs.ok) return attrs if (!attrs.ok) return attrs
return success(namedNode(span.name, attrs.value, undefined)) if (slot !== undefined && content !== undefined) {
const text = slotText(content)
if (text === undefined) return failure('unsupported-node-shape', `the ${name} content slot holds one unmarked text node`, path)
const spans = slotLineEndingFault(name, text)
if (spans !== undefined) return faulted(spans, path)
attrs.value[slot] = text
}
return success(namedNode(name, attrs.value, undefined))
} }
function readVocabulary( // A name the other position spells names that spelling, never the code a later MINOR may fill (AGENTS.md §8).
type: string, function inlineSpellingFault(name: string): ConvertFault | undefined {
attributes: DirectiveAttributes, const mark = inlineMarkSpellingFault(name)
vocabulary: AttributeVocabulary, if (mark !== undefined) return mark
elsewhere: Elsewhere | undefined, if (inlineDirective(name) === undefined && name !== textDirectiveName) return undefined
path: ConvertErrorPath, return { code: 'unsupported-node-shape', message: `${name} takes the inline form, :${name}{…}` }
): Result<AdfAttributes> {
const attrs: AdfAttributes = {}
for (const [key, spelled] of attributes) {
if (key === elsewhere?.key) {
const place = elsewhere.slot === 'argument' ? 'as the directive argument' : 'in the content slot'
return failure('unsupported-node-shape', `${type} spells its ${key} attribute ${place}`, path)
} }
const kind = Object.hasOwn(vocabulary, key) ? vocabulary[key] : undefined
if (kind === undefined) return failure('unsupported-node-shape', `${type} holds no ${key} attribute`, path) function blockSpellingFault(name: string): ConvertFault | undefined {
const read = attributeValue(spelled.decoded, kind) const directive = blockDirective(name)
if (read === undefined) return failure('unsupported-node-shape', `the ${key} attribute of ${type} is no ${kind}`, path) if (directive === undefined) return undefined
const spelling = spellAttributeValue(read) const form = directive.contentModel === 'none' ? `::${name}` : `:::${name}`
if (spelling !== spelled.spelling) return failure('unsupported-node-shape', `${type} spells its ${key} attribute as ${key}=${spelling}`, path) return { code: 'unsupported-node-shape', message: `${name} takes the block form, ${form}` }
attrs[key] = read.value
} }
return success(attrs)
// spec/flavour.md, Inline nodes: the slot is plain text, its adjacent nodes already merged.
function slotText(content: readonly AdfNode[]): string | undefined {
if (content.length === 0) return ''
const only = content.length === 1 ? content[0] : undefined
if (only?.type !== 'text' || (only.marks ?? []).length > 0 || typeof only.text !== 'string') return undefined
return only.text
} }
function readMarks(type: string, spelled: DirectiveValue, path: ConvertErrorPath): Result<AdfMark[]> { function readMarks(type: string, spelled: DirectiveValue, path: ConvertErrorPath): Result<AdfMark[]> {
+48 -7
View File
@@ -1,14 +1,18 @@
import type { AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfMark, AdfNode } from '../../adf/document.ts'
import type { DirectiveSpan } from '../directive-syntax.ts'
import type { EmphasisPairing } from '../emphasis-matching.ts' import type { EmphasisPairing } from '../emphasis-matching.ts'
import type { LinkDefinition } from '../link-syntax.ts' import type { LinkDefinition } from '../link-syntax.ts'
import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark-grammar.ts' import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark-grammar.ts'
import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../backtick-runs.ts'
import { delimiterFlags, matchEmphasis, runLength } from '../emphasis-matching.ts' import { delimiterFlags, matchEmphasis, runLength } from '../emphasis-matching.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { inlineDirective } from '../../adf/inline-directives.ts'
import { mergeAdjacentText } from '../../adf/editor-normal.ts' import { mergeAdjacentText } from '../../adf/editor-normal.ts'
import { normalizeLabel, readInlineTarget, readLabel } from '../link-syntax.ts' import { normalizeLabel, readInlineTarget, readLabel } from '../link-syntax.ts'
import { readDirectiveMark } from './directive-marks.ts'
import { readInlineDirective } from '../directive-syntax.ts' import { readInlineDirective } from '../directive-syntax.ts'
import { readInlineDirectiveNode } from './directive-nodes.ts' import { readInlineDirectiveNode } from './directive-nodes.ts'
import { readTextDirective } from '../text-directive.ts'
export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] } export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] }
@@ -31,6 +35,10 @@ type Scan = { definitions: LinkDefinitions; path: ConvertErrorPath; pending: str
const imageAlone = 'an image fits only as a paragraph of its own' const imageAlone = 'an image fits only as a paragraph of its own'
export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath): Result<InlineContent> { export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath): Result<InlineContent> {
return parseInline(source, definitions, path, true)
}
function parseInline(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, strip: boolean): Result<InlineContent> {
const scan: Scan = { definitions, path, pending: '', pieces: [], source } const scan: Scan = { definitions, path, pending: '', pieces: [], source }
let index = 0 let index = 0
while (index < source.length) { while (index < source.length) {
@@ -76,7 +84,7 @@ export function parseInlineContent(source: string, definitions: LinkDefinitions,
index += 1 index += 1
} }
} }
flush(scan, true) flush(scan, strip)
return assemble(scan) return assemble(scan)
} }
@@ -144,13 +152,40 @@ function readDirective(scan: Scan, index: number): Result<number> {
return success(index + 1) return success(index + 1)
} }
if (directive.fault !== undefined) return faulted(directive.fault, scan.path) if (directive.fault !== undefined) return faulted(directive.fault, scan.path)
const node = readInlineDirectiveNode(directive.value, scan.path) const nodes = directiveNodes(scan, directive.value)
if (!node.ok) return node if (!nodes.ok) return nodes
flush(scan, false) flush(scan, false)
pushNode(scan, node.value) scan.pieces.push({ kind: 'nodes', nodes: nodes.value })
return success(index + directive.value.length) return success(index + directive.value.length)
} }
function directiveNodes(scan: Scan, span: DirectiveSpan): Result<AdfNode[]> {
const text = readTextDirective(span)
if (text?.fault !== undefined) return faulted(text.fault, scan.path)
if (text !== undefined) return success([{ text: text.value, type: 'text' }])
const slot = slotNodes(scan, span.content)
if (!slot.ok) return slot
const mark = readDirectiveMark(span.name, span.attributes, scan.path)
if (mark !== undefined) {
if (!mark.ok) return mark
if (slot.value === undefined || slot.value.length === 0) {
return failure('unsupported-node-shape', `the ${span.name} mark wraps the [content] it marks`, scan.path)
}
return success(applyMark(slot.value, mark.value))
}
const node = readInlineDirectiveNode(span.name, span.attributes, slot.value, scan.path)
if (!node.ok) return node
return success([node.value])
}
function slotNodes(scan: Scan, content: string | undefined): Result<AdfNode[] | undefined> {
if (content === undefined) return success(undefined)
const parsed = parseInline(content, scan.definitions, scan.path, false)
if (!parsed.ok) return parsed
if (parsed.value.image !== undefined) return failure('unmappable-image', imageAlone, scan.path)
return success(parsed.value.nodes)
}
function flush(scan: Scan, strip: boolean): void { function flush(scan: Scan, strip: boolean): void {
const raw = strip ? trimTrailingSpace(scan.pending) : scan.pending const raw = strip ? trimTrailingSpace(scan.pending) : scan.pending
scan.pending = '' scan.pending = ''
@@ -271,9 +306,15 @@ function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition:
} }
function imageAlt(inner: readonly Piece[]): string { function imageAlt(inner: readonly Piece[]): string {
return resolveNodes(inner) return resolveNodes(inner).map(altText).join('')
.map((node) => (node.type === 'hardBreak' ? ' ' : (node.text ?? ''))) }
.join('')
// spec/flavour.md, The CommonMark image: the description's plain text, the content slot included.
function altText(node: AdfNode): string {
if (node.type === 'hardBreak') return ' '
const slot = inlineDirective(node.type)?.textAttribute
const spelled = slot === undefined ? undefined : node.attrs?.[slot]
return typeof spelled === 'string' ? spelled : (node.text ?? '')
} }
function resolveNodes(pieces: readonly Piece[]): AdfNode[] { function resolveNodes(pieces: readonly Piece[]): AdfNode[] {
+96 -5
View File
@@ -1,7 +1,7 @@
import assert from 'node:assert/strict' import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts'
import type { Result } from '../../result.ts' import type { Result } from '../../result.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { markdownToAdf } from './markdown-to-adf.ts' import { markdownToAdf } from './markdown-to-adf.ts'
@@ -9,6 +9,7 @@ import { markdownToAdf } from './markdown-to-adf.ts'
const em: AdfMark = { type: 'em' } const em: AdfMark = { type: 'em' }
const strike: AdfMark = { type: 'strike' } const strike: AdfMark = { type: 'strike' }
const strong: AdfMark = { type: 'strong' } const strong: AdfMark = { type: 'strong' }
const underline: AdfMark = { type: 'underline' }
function code(result: Result<AdfDocument>): string { function code(result: Result<AdfDocument>): string {
return result.ok ? `built ${JSON.stringify(result.value)}` : result.error.code return result.ok ? `built ${JSON.stringify(result.value)}` : result.error.code
@@ -250,6 +251,17 @@ test('names the directive name no node reads back to', () => {
assert.deepEqual(path(markdownToAdf('Part.\n:::x\n')), ['content', 1]) assert.deepEqual(path(markdownToAdf('Part.\n:::x\n')), ['content', 1])
}) })
test('names the position a directive name the other one spells belongs to', () => {
assert.equal(content(markdownToAdf(':::em\na\n:::\n')), 'unsupported-node-shape: em is spelled _x_, never as a block directive')
assert.equal(content(markdownToAdf('::underline\n')), 'unsupported-node-shape: underline is spelled :underline[…], never as a block directive')
assert.equal(content(markdownToAdf('::text {text=" "}\n')), 'unsupported-node-shape: text takes the inline form, :text{…}')
assert.equal(content(markdownToAdf('::date {timestamp=1}\n')), 'unsupported-node-shape: date takes the inline form, :date{…}')
assert.equal(content(markdownToAdf(':paragraph[a]\n')), 'unsupported-node-shape: paragraph takes the block form, :::paragraph')
assert.equal(content(markdownToAdf(':rule[a]\n')), 'unsupported-node-shape: rule takes the block form, ::rule')
assert.equal(code(markdownToAdf(':::widget\na\n:::\n')), 'unknown-directive-name')
assert.equal(code(markdownToAdf(':widget[a]\n')), 'unknown-directive-name')
})
test('names the reserved carry name a block directive spells', () => { test('names the reserved carry name a block directive spells', () => {
const reserved = 'malformed-directive: the name adf is reserved for the opaque carry, whose block form is the fence' const reserved = 'malformed-directive: the name adf is reserved for the opaque carry, whose block form is the fence'
assert.equal(content(markdownToAdf('::adf\n')), reserved) assert.equal(content(markdownToAdf('::adf\n')), reserved)
@@ -311,10 +323,6 @@ test('names the argument and the body a node takes no reading for', () => {
assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\nx\n:::\n')), 'unsupported-node-shape: codeBlock takes one code block as its body') assert.equal(content(markdownToAdf(':::codeBlock {wrap=true}\nx\n:::\n')), 'unsupported-node-shape: codeBlock takes one code block as its body')
assert.equal(content(markdownToAdf(':::paragraph\n![a](/u)\n:::\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.equal(content(markdownToAdf(':::paragraph\n![a](/u)\n:::\n')), 'unmappable-image: no ADF node carries an image inside a paragraph')
assert.equal(content(markdownToAdf('Part :date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content') assert.equal(content(markdownToAdf('Part :date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content')
assert.equal(
content(markdownToAdf('Part :emoji[x]{shortName=":x:"}.\n')),
'unsupported-node-shape: the content slot emoji spells its text attribute in is unsupported',
)
}) })
test('leaves the colon that opens no directive the text it is', () => { test('leaves the colon that opens no directive the text it is', () => {
@@ -460,6 +468,9 @@ test('names the block the claim inside a container opens', () => {
test('refuses input nested deeper than the parser carries', () => { test('refuses input nested deeper than the parser carries', () => {
assert.equal(code(markdownToAdf('> '.repeat(501))), 'unsupported-nesting-depth') assert.equal(code(markdownToAdf('> '.repeat(501))), 'unsupported-nesting-depth')
assert.ok(markdownToAdf('> '.repeat(500)).ok) assert.ok(markdownToAdf('> '.repeat(500)).ok)
const marks = (levels: number): string => `${':underline['.repeat(levels)}a${']'.repeat(levels)}\n`
assert.equal(code(markdownToAdf(marks(largestNesting + 1))), 'unsupported-nesting-depth')
assert.deepEqual(content(markdownToAdf(marks(largestNesting))), [{ content: [marked('a', underline)], type: 'paragraph' }])
}) })
test('decodes the backslash escapes CommonMark spells, and keeps the rest literal', () => { test('decodes the backslash escapes CommonMark spells, and keeps the rest literal', () => {
@@ -647,6 +658,8 @@ test('flattens the description of a lone image to the plain text alt holds', ()
assert.deepEqual(content(markdownToAdf('![a\nb](/u)\n')), [image('/u', 'a b')]) assert.deepEqual(content(markdownToAdf('![a\nb](/u)\n')), [image('/u', 'a b')])
assert.deepEqual(content(markdownToAdf('![a \nb](/u)\n')), [image('/u', 'a b')]) assert.deepEqual(content(markdownToAdf('![a \nb](/u)\n')), [image('/u', 'a b')])
assert.deepEqual(content(markdownToAdf('![a `b`](/u)\n')), [image('/u', 'a b')]) assert.deepEqual(content(markdownToAdf('![a `b`](/u)\n')), [image('/u', 'a b')])
assert.deepEqual(content(markdownToAdf('![a :mention[@A]{id=b1c2} b](/u)\n')), [image('/u', 'a @A b')])
assert.deepEqual(content(markdownToAdf('![:mention[@A]{id=b1c2}](/u)\n')), [image('/u', '@A')])
}) })
test('leaves the brackets of an empty link text the text they are', () => { test('leaves the brackets of an empty link text the text they are', () => {
@@ -677,3 +690,81 @@ test('carries the mark a spelling nested inside its own kind names once', () =>
{ content: [marked('a ', em), marked('b', em, strong), marked(' c', em)], type: 'paragraph' }, { content: [marked('a ', em), marked('b', em, strong), marked(' c', em)], type: 'paragraph' },
]) ])
}) })
test('reads the content slot as the text attribute the node spells there', () => {
const status = (attrs: AdfAttributes): AdfNode[] => [{ content: [{ attrs, type: 'status' }], type: 'paragraph' }]
assert.deepEqual(content(markdownToAdf(':status[In review]{color=yellow}\n')), status({ color: 'yellow', text: 'In review' }))
assert.deepEqual(content(markdownToAdf(':status{color=neutral}\n')), status({ color: 'neutral' }))
assert.deepEqual(content(markdownToAdf(':status[]{color=neutral}\n')), status({ color: 'neutral', text: '' }))
assert.deepEqual(content(markdownToAdf(':status[ In review ]{color=yellow}\n')), status({ color: 'yellow', text: ' In review ' }))
assert.deepEqual(content(markdownToAdf(':status[In:text{text=" "}review]{color=yellow}\n')), status({ color: 'yellow', text: 'In review' }))
assert.deepEqual(content(markdownToAdf(':status[a\\]b]{color=yellow}\n')), status({ color: 'yellow', text: 'a]b' }))
assert.deepEqual(content(markdownToAdf('**:mention[@A]{id=b1c2}**\n')), [
{ content: [{ attrs: { id: 'b1c2', text: '@A' }, marks: [strong], type: 'mention' }], type: 'paragraph' },
])
})
test('names the content slot no one unmarked text node reads back from', () => {
assert.equal(content(markdownToAdf(':status[**A**]{color=yellow}\n')), 'unsupported-node-shape: the status content slot holds one unmarked text node')
assert.equal(code(markdownToAdf(':status[a`b`]{color=yellow}\n')), 'unsupported-node-shape')
assert.equal(code(markdownToAdf(':status[:date{timestamp=1}]{color=yellow}\n')), 'unsupported-node-shape')
assert.equal(content(markdownToAdf(':status[![a](/u)]{color=yellow}\n')), 'unmappable-image: an image fits only as a paragraph of its own')
assert.equal(code(markdownToAdf(':status[<div>]{color=yellow}\n')), 'unmappable-html')
assert.equal(code(markdownToAdf(':date[<div>]{timestamp=1}\n')), 'unmappable-html')
assert.equal(code(markdownToAdf(':widget[<div>]\n')), 'unmappable-html')
const spans = 'unspellable-whitespace: the status content slot holds a newline no inline directive spans'
assert.equal(content(markdownToAdf(':status[:text{text="\\n"}]{color=yellow}\n')), spans)
assert.equal(content(markdownToAdf(':status[a&#10;b]{color=yellow}\n')), spans)
assert.equal(content(markdownToAdf(':status[a&#13;b]{color=yellow}\n')), spans)
assert.equal(content(markdownToAdf('Part :mention{id=b1c2 text=A}.\n')), 'unsupported-node-shape: mention spells its text attribute in the content slot')
})
test('reads the whitespace the reserved text directive carries', () => {
assert.deepEqual(content(markdownToAdf(':text{text=" "}a\n')), [paragraph(' a')])
assert.deepEqual(content(markdownToAdf('a:text{text="\\n"}b\n')), [paragraph('a\nb')])
assert.deepEqual(content(markdownToAdf('a:text{text="\\t"}\n')), [paragraph('a\t')])
assert.deepEqual(content(markdownToAdf('_:text{text=" "}a_\n')), [{ content: [marked(' a', em)], type: 'paragraph' }])
})
test('names the text directive spelling no whitespace run reads back from', () => {
const named = 'unsupported-node-shape: text spells one run of spaces and tabs, or one run of newlines'
assert.equal(content(markdownToAdf(':text{text=hi}\n')), named)
assert.equal(content(markdownToAdf(':text{text=" \\n"}\n')), named)
assert.equal(content(markdownToAdf(':text{text=""}\n')), named)
assert.equal(content(markdownToAdf(':text{}\n')), 'unsupported-node-shape: text holds one text attribute alone')
assert.equal(content(markdownToAdf(':text{localId=a text=" "}\n')), 'unsupported-node-shape: text holds one text attribute alone')
assert.equal(content(markdownToAdf(':text[a]{text=" "}\n')), 'unsupported-node-shape: text takes no content')
assert.equal(content(markdownToAdf(':text{text="\\u0020"}\n')), 'unsupported-node-shape: text spells its text attribute as text=" "')
})
test('reads the directive marks, the nesting outermost first', () => {
const wrapped = (...marks: AdfMark[]): AdfNode[] => [{ content: [marked('a', ...marks)], type: 'paragraph' }]
assert.deepEqual(content(markdownToAdf(':underline[a]\n')), wrapped(underline))
assert.deepEqual(content(markdownToAdf('_:underline[a]_\n')), wrapped(em, underline))
assert.deepEqual(content(markdownToAdf(':underline[_a_]\n')), wrapped(underline, em))
assert.deepEqual(content(markdownToAdf(':underline[:underline[a]]\n')), wrapped(underline))
assert.deepEqual(content(markdownToAdf(':textColor[a]{color="#ae2e24"}\n')), wrapped({ attrs: { color: '#ae2e24' }, type: 'textColor' }))
assert.deepEqual(content(markdownToAdf(':subsup[a]{type=sub}\n')), wrapped({ attrs: { type: 'sub' }, type: 'subsup' }))
assert.deepEqual(content(markdownToAdf(':border[a]{color="#091e42" size=2}\n')), wrapped({ attrs: { color: '#091e42', size: 2 }, type: 'border' }))
assert.deepEqual(content(markdownToAdf(':underline[a:date{timestamp=1}]\n')), [
{ content: [marked('a', underline), { attrs: { timestamp: '1' }, marks: [underline], type: 'date' }], type: 'paragraph' },
])
assert.deepEqual(content(markdownToAdf('_:underline[a:date{timestamp=1}]_\n')), [
{ content: [marked('a', em, underline), { attrs: { timestamp: '1' }, marks: [em, underline], type: 'date' }], type: 'paragraph' },
])
assert.equal(content(markdownToAdf(':border[a]{color="#091e42" size=x}\n')), 'unsupported-node-shape: the size attribute of border is no number')
})
test('names the mark markdown spells, never a directive', () => {
assert.equal(content(markdownToAdf(':em[a]\n')), 'unsupported-node-shape: em is spelled _x_, never as a directive')
assert.equal(content(markdownToAdf(':strong[a]\n')), 'unsupported-node-shape: strong is spelled **x**, never as a directive')
assert.equal(content(markdownToAdf(':strike[a]\n')), 'unsupported-node-shape: strike is spelled ~~x~~, never as a directive')
assert.equal(content(markdownToAdf(':code[a]\n')), 'unsupported-node-shape: code is spelled `x`, never as a directive')
assert.equal(content(markdownToAdf(':link[a]{href="/u"}\n')), 'unsupported-node-shape: link is spelled [x](url), never as a directive')
})
test('names the directive mark left without the content it wraps', () => {
const named = 'unsupported-node-shape: the underline mark wraps the [content] it marks'
assert.equal(content(markdownToAdf(':underline[]\n')), named)
assert.equal(content(markdownToAdf(':underline{}\n')), named)
})
+27
View File
@@ -0,0 +1,27 @@
import type { ConvertFault } from '../result.ts'
import type { DirectiveSpan, Read } from './directive-syntax.ts'
import { spellAttributes, spellLeafDirective, spellStringAttribute } from './directive-syntax.ts'
const name = 'text'
const whitespaceRun = /^(?:[ \t]+|\n+)$/
export const textDirectiveName = name
export function spellTextDirective(text: string): string {
return spellLeafDirective(name, spellAttributes([[name, spellStringAttribute(text)]]))
}
export function readTextDirective(span: DirectiveSpan): Read<string> | undefined {
if (span.name !== name) return undefined
if (span.content !== undefined) return { fault: unsupported(`${name} takes no content`) }
const spelled = span.attributes.get(name)
if (spelled === undefined || span.attributes.size !== 1) return { fault: unsupported(`${name} holds one ${name} attribute alone`) }
const spelling = spellStringAttribute(spelled.decoded)
if (spelling !== spelled.spelling) return { fault: unsupported(`${name} spells its ${name} attribute as ${name}=${spelling}`) }
if (!whitespaceRun.test(spelled.decoded)) return { fault: unsupported(`${name} spells one run of spaces and tabs, or one run of newlines`) }
return { value: spelled.decoded }
}
function unsupported(message: string): ConvertFault {
return { code: 'unsupported-node-shape', message }
}
+7
View File
@@ -8,6 +8,7 @@ import type { AttributeKind, AttributeVocabulary } from './adf/attribute-vocabul
import { blockDirectives } from './adf/block-directives.ts' import { blockDirectives } from './adf/block-directives.ts'
import { inlineDirectives } from './adf/inline-directives.ts' import { inlineDirectives } from './adf/inline-directives.ts'
import { markAttributes } from './adf/mark-attributes.ts' import { markAttributes } from './adf/mark-attributes.ts'
import { textDirectiveName } from './markdown/text-directive.ts'
type Declared = { attributes: AttributeVocabulary } type Declared = { attributes: AttributeVocabulary }
@@ -96,6 +97,12 @@ test('the inline node table holds the attributes spec/flavour.md gives each node
assert.deepEqual(declarations('Inline nodes'), vocabularies(inlineDirectives)) assert.deepEqual(declarations('Inline nodes'), vocabularies(inlineDirectives))
}) })
// A name in two tables would make the position a directive is read in ambiguous.
test('no name is spelled in more than one position', () => {
const names = [...Object.keys(blockDirectives), ...Object.keys(inlineDirectives), ...Object.keys(markAttributes), textDirectiveName]
assert.equal(new Set(names).size, names.length)
})
test('the mark table holds the attributes spec/flavour.md gives each mark', () => { test('the mark table holds the attributes spec/flavour.md gives each mark', () => {
assert.deepEqual(declarations('Marks'), vocabularies(Object.fromEntries(Object.entries(markAttributes).map(([type, attributes]) => [type, { attributes }])))) assert.deepEqual(declarations('Marks'), vocabularies(Object.fromEntries(Object.entries(markAttributes).map(([type, attributes]) => [type, { attributes }]))))
}) })
+40
View File
@@ -335,3 +335,43 @@ Under **3 — `markdownToAdf` (`0.1.0`)**:
class lives in is checked rather than read. The README's canonical-fixpoint sentence still class lives in is checked rather than read. The README's canonical-fixpoint sentence still
claims more than the parser keeps — 3e's three shapes — which stays milestone 5's to claims more than the parser keeps — 3e's three shapes — which stays milestone 5's to
narrow. narrow.
- [x] **3i — The inline nodes and the marks.** `inline-nodes/` reads back: the content slot's
`text` attribute and the error a slot holding anything but one unmarked text node is; the
`:text{text="…"}` whitespace spelling; the four directive marks and their nesting order,
outermost first; and `:em[x]` as the error `spec/flavour.md` promises. Editor-normal's
merging half lands here, `text-whitespace` being the first fixture that forces it, and 4's
`toEditorNormal` is built on it.
3g's shape leaves three: `readInlineDirectiveNode` takes the name, the attributes and the
slot's parsed text rather than the span, since `inline-content.ts` already imports it and
parsing the slot inside it is a cycle; the four directive marks get `parse/directive-marks.ts`
that `inline-content.ts` tries ahead of the node reader, as `mark-spellings.ts` sits apart
from `emit/inline-directive-spelling.ts`; and the five markdown-spelled mark names in inline
directive position take `unsupported-node-shape` rather than a code of their own — §8
already answers a well-formed directive the node tables refuse, and the message names the
spelling to use (`*x*`), while `unknown-directive-name`'s "a later MINOR may give the name
meaning" stays the wrong signal, as it was for `adf`. `corpus/errors/directive-content-slot` goes when the slot opens.
The marks a spelling wraps answer the same question 3g settled for a block's form: only the
nesting the emitter writes parses back.
**Settled** (the maintainer, 2026-09-01): `:text` reads back what the emitter writes and
nothing else — one run of spaces and tabs, or one run of newlines. A mixed run, and text
CommonMark carries plainly, are named errors, as 3g refuses the directive form of a node
CommonMark spells. The reader takes the slot's parsed nodes rather than its text, so the
rule refusing anything but one unmarked text node sits beside the node tables that own the
slot. Only `text`, read ahead of the slot, names a refusal before the slot's own: a
doubly-broken span reports what its content holds, `:date[<div>]{timestamp=1}` being
`unmappable-html` rather than `date takes no content`, which the maintainer pinned with an
assertion rather than reordering the readers. `directive-content-slot` stays with the
fixtures, its cause now a marked slot rather than a slot at all. The slot's own whitespace
answers the rule the spelling does: `:text{text="\n"}` and `&#10;` alike reach a slot the
emitter refuses a line ending in, so one function answers both directions.
**Settled** (the maintainer, 2026-09-01): a name the other position spells names that
spelling rather than reading as unknown — `:::em` and `::date` take
`unsupported-node-shape` naming the inline form, `:paragraph[a]` the block one — leaving
`unknown-directive-name` for a name no table holds, which is the meaning §8 gives it. The
two readers lean on the tables being disjoint, so that is a test beside the spec drift
guard now.
The same read found the hole the other way: `attemptLine` refused a line edged with a
vertical tab or a form feed, where CommonMark strips spaces and tabs alone, so valid
CommonMark parsed to a document `adfToMarkdown` then refused. The edges that check covered
are carried before the line is assembled, so narrowing it to spaces and tabs left it no
cause and it goes with them.
+11 -20
View File
@@ -50,28 +50,15 @@ The numbering is the order the work was planned in, not the order it ships.
- [x] **3f — The directive grammar.** - [x] **3f — The directive grammar.**
- [x] **3g — The node tables read backwards.** - [x] **3g — The node tables read backwards.**
- [x] **3h — The block nodes.** - [x] **3h — The block nodes.**
- [ ] **3i — The inline nodes and the marks.** `inline-nodes/` reads back: the content slot's - [x] **3i — The inline nodes and the marks.**
`text` attribute and the error a slot holding anything but one unmarked text node is; the
`:text{text="…"}` whitespace spelling; the four directive marks and their nesting order,
outermost first; and `:em[x]` as the error `spec/flavour.md` promises. Editor-normal's
merging half lands here, `text-whitespace` being the first fixture that forces it, and 4's
`toEditorNormal` is built on it.
3g's shape leaves three: `readInlineDirectiveNode` takes the name, the attributes and the
slot's parsed text rather than the span, since `inline-content.ts` already imports it and
parsing the slot inside it is a cycle; the four directive marks get `parse/directive-marks.ts`
that `inline-content.ts` tries ahead of the node reader, as `mark-spellings.ts` sits apart
from `emit/inline-directive-spelling.ts`; and the five markdown-spelled mark names in inline
directive position take `unsupported-node-shape` rather than a code of their own — §8
already answers a well-formed directive the node tables refuse, and the message names the
spelling to use (`*x*`), while `unknown-directive-name`'s "a later MINOR may give the name
meaning" stays the wrong signal, as it was for `adf`. `corpus/errors/directive-content-slot` goes when the slot opens.
The marks a spelling wraps answer the same question 3g settled for a block's form: only the
nesting the emitter writes parses back.
- [ ] **3j — The carry and the combinations.** `opaque-carry/` and `combinations/` read back: - [ ] **3j — The carry and the combinations.** `opaque-carry/` and `combinations/` read back:
the `adf` fence and `:adf{json="…"}` restoring a deep-equal node, invalid JSON in either a the `adf` fence and `:adf{json="…"}` restoring a deep-equal node, invalid JSON in either a
named error, a carry inside a mark spelling another, and the three carve-outs' escapes named error, a carry inside a mark spelling another, and the three carve-outs' escapes
reading as the literal text they hold. 3g refuses the `adf` fence rather than reading a reading as the literal text they hold. 3g refuses the `adf` fence rather than reading a
`codeBlock` from it; the refusal goes when the carry reads it. `codeBlock` from it; the refusal goes when the carry reads it. 3i left the slot parse
contextless, so the refusal a carry inside a mark spelling earns needs a channel — a reader
context in place of `parseInline`'s `strip` flag, or a return arm from the slot — and
`directiveNodes` takes its fourth reader beside it.
`index.ts` gains `markdownToAdf` here, and the README's status line with it: this is the `index.ts` gains `markdownToAdf` here, and the README's status line with it: this is the
last parser chunk, so `parsingDirectories` becomes `emittingDirectories` and the whole last parser chunk, so `parsingDirectories` becomes `emittingDirectories` and the whole
corpus round-trips both ways — `0.1.0`'s proof, which 4 widens rather than replaces. corpus round-trips both ways — `0.1.0`'s proof, which 4 widens rather than replaces.
@@ -125,8 +112,12 @@ The numbering is the order the work was planned in, not the order it ships.
replaced the regex. Three sites the same sweep did not reach: `normalizeLabel` in replaced the regex. Three sites the same sweep did not reach: `normalizeLabel` in
`link-syntax.ts`, whose shortcut-reference input is `scan.source.slice(...)` rather than the `link-syntax.ts`, whose shortcut-reference input is `scan.source.slice(...)` rather than the
999-capped `readLabel` value, and two in `emit/inline-line.ts`. The fix is the one 3h used — 999-capped `readLabel` value, and two in `emit/inline-line.ts`. The fix is the one 3h used —
an index walk, `trimTrailingSpace` where the ends match. §11's scanning rule is the whole an index walk, `trimTrailingSpace` where the ends match. A fourth of another shape joins
argument; the pipeline persona feeds documents nobody typed. them: `readNestedDirective` restarts its depth counter per level, so each parse level
re-scans the region below it and nested inline directives cost O(depth × content) — 3f's
cost, which 3i's slot parse doubles rather than changes in class, bounded by the 500-level
guard. §11's scanning rule is the whole argument; the pipeline persona feeds documents
nobody typed.
- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret,
the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: 3h the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: 3h
landed the last decision `corpus/unspellable/` held and the directory went with it, so what landed the last decision `corpus/unspellable/` held and the directory went with it, so what