Read the three directive forms, their attributes and the fences that nest them #35

Merged
lilleman merged 3 commits from tick-3f into main 2026-09-01 10:27:32 +02:00
26 changed files with 616 additions and 92 deletions
+8 -2
View File
@@ -87,7 +87,12 @@ cause; where one cause recurs across node types or across directions, one code c
`path` and `message` say which — `unsupported-nesting-depth` is the 500-level guard whichever `path` and `message` say which — `unsupported-nesting-depth` is the 500-level guard whichever
direction hits it. A claim code names the spelling claimed, never the node that spelling would have built: direction hits it. A claim code names the spelling claimed, never the node that spelling would have built:
a malformed `:::table` is a `malformed-directive`. A cause the carry answers gets no code: a mark no a malformed `:::table` is a `malformed-directive`. A cause the carry answers gets no code: a mark no
spelling writes rides the carry with its node. spelling writes rides the carry with its node. A directive whose name reads back to no node is
`unknown-directive-name` rather than a claim code — the spelling is well formed, and telling that
apart from a typo is what a consumer switches on when a later MINOR gives the name meaning. A
refusal found before its position is known — the block walk's, a directive reader's — is a
`ConvertFault`, the code and message without the path; the node walk attaches the path as it
descends, so a document reports its first error in document order.
## 9. Release automation ## 9. Release automation
@@ -120,7 +125,8 @@ live Atlassian APIs; property-generated ADF trees; the CommonMark spec suite aga
- Failures are values: everything returns - Failures are values: everything returns
`Result<T>``{ ok: true; value } | { ok: false; error: ConvertError }` — nothing throws. `Result<T>``{ ok: true; value } | { ok: false; error: ConvertError }` — nothing throws.
`try/catch` only wrapped tightly around a call that genuinely throws, converted to a result on `try/catch` only wrapped tightly around a call that genuinely throws, converted to a result on
the spot. the spot. A reader with no path to name returns `Read<T>` instead, the same two arms over a
`ConvertFault`, and `faulted` attaches the path where the walk knows it.
- Only the hard break's inline segment holds a raw newline — every other spelling escapes one or - Only the hard break's inline segment holds a raw newline — every other spelling escapes one or
refuses it — which is how the whitespace carry finds a line edge. refuses it — which is how the whitespace carry finds a line edge.
- Emphasis is spelled against CommonMark's matching, never flanking alone: a delimiter run in text - Emphasis is spelled against CommonMark's matching, never flanking alone: a delimiter run in text
@@ -0,0 +1 @@
malformed-directive
@@ -0,0 +1,2 @@
:::panel info
Part.
@@ -0,0 +1 @@
malformed-directive
@@ -0,0 +1 @@
Part :mention[@A and more.
@@ -0,0 +1 @@
unknown-directive-name
+3
View File
@@ -0,0 +1,3 @@
:::widget info
Part.
:::
@@ -0,0 +1 @@
unsupported-nesting-depth
+1
View File
@@ -0,0 +1 @@
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - :a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[:a[x]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
+7 -3
View File
@@ -83,15 +83,19 @@ block content
The fence is three or more colons. `arg` is one optional bare token whose meaning each node The fence is three or more colons. `arg` is one optional bare token whose meaning each node
defines (e.g. the panel type). The body is block markdown. The closing fence is a line of at defines (e.g. the panel type). The body is block markdown. The closing fence is a line of at
least the opening's length, and a container's fence is longer than every directive fence line least the opening's length and closes the innermost open container however long its run, and a
anywhere in its body, however deeply a list item or blockquote nests it; a colon run inside a code container's fence is longer than every directive fence line anywhere in its body, however deeply a list item or blockquote nests it; a colon run inside a code
fence or opaque carry is content. Canonical form uses minimal lengths. fence or opaque carry is content. Canonical form uses minimal lengths.
Directive fence lines follow code-fence indentation (up to three spaces relative to their Directive fence lines follow code-fence indentation (up to three spaces relative to their
container); trailing whitespace on a fence line is tolerated in input, never emitted. container).
**Leaf block**: `::name arg {attrs}` — a block-position node with no body, `arg` reading as **Leaf block**: `::name arg {attrs}` — a block-position node with no body, `arg` reading as
above. above.
Canonical spacing is the only spacing input reads: one space parts the name, `arg` and `{attrs}`,
and one parts each attribute pair, with no padding inside the braces. Trailing whitespace on a
directive block line is tolerated in input, never emitted.
**Claiming at block level**, symmetric with inline: a line whose leading run of two or more **Claiming at block level**, symmetric with inline: a line whose leading run of two or more
colons is followed immediately by a name character is claimed and must parse fully as a container colons is followed immediately by a name character is claimed and must parse fully as a container
opening or a leaf, else it is a named error. A bare colon-run line is a closing fence while a opening or a leaf, else it is a named error. A bare colon-run line is a closing fence while a
-47
View File
@@ -1,47 +0,0 @@
import type { JsonValue } from '../json-value.ts'
import type { VocabularyPair } from '../adf/attribute-vocabulary.ts'
import { serializeCanonicalJson } from '../canonical-json.ts'
const bareToken = /^[A-Za-z0-9_-]+$/
const inlineDirectiveOpener = /:[a-z][A-Za-z0-9]*[[{]/y
// spec/flavour.md, Attributes.
const quotedEscapes = /[&<`|]/g
export function isBareToken(text: string): boolean {
return bareToken.test(text)
}
export function opensInlineDirective(text: string, index: number): boolean {
inlineDirectiveOpener.lastIndex = index
return inlineDirectiveOpener.test(text)
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => (left < right ? -1 : 1)).map(([key, value]) => `${key}=${value}`)
return `{${spelled.join(' ')}}`
}
export function spellVocabulary(pairs: readonly VocabularyPair[]): [string, string][] {
return pairs.map((pair): [string, string] => [pair.key, spellAttributeValue(pair)])
}
export function spellJsonAttribute(value: JsonValue): string {
return quote(serializeCanonicalJson(value, 'compact'))
}
export function spellStringAttribute(text: string): string {
return isBareToken(text) ? text : quote(text)
}
function spellAttributeValue(pair: VocabularyPair): string {
if (pair.kind === 'boolean') return `${pair.value}`
if (pair.kind === 'json') return spellJsonAttribute(pair.value)
if (pair.kind === 'number') return spellStringAttribute(JSON.stringify(pair.value))
return spellStringAttribute(pair.value)
}
function quote(text: string): string {
return JSON.stringify(text).replace(quotedEscapes, (character) => `\\u${character.charCodeAt(0).toString(16).padStart(4, '0')}`)
}
+130
View File
@@ -0,0 +1,130 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import type { DirectiveLine } from './directive-syntax.ts'
import { largestNesting } from '../nesting.ts'
import { readDirectiveLine, readInlineDirective } from './directive-syntax.ts'
function attributes(...pairs: [string, string][]): ReadonlyMap<string, string> {
return new Map(pairs)
}
function header(colons: number, name: string, argument?: string, ...pairs: [string, string][]): { value: DirectiveLine } {
return { value: { argument, attributes: attributes(...pairs), colons, kind: 'header', name } }
}
function fault(line: string): string {
const read = readDirectiveLine(line)
return read?.fault === undefined ? `read ${JSON.stringify(read)}` : read.fault.message
}
function inline(text: string): unknown {
const read = readInlineDirective(text, 0)
if (read === undefined) return 'unclaimed'
if (read.fault !== undefined) return read.fault.message
return { attributes: read.value.attributes, content: read.value.content, length: read.value.length, name: read.value.name }
}
function spans(text: string, name: string, content: string | undefined, ...pairs: [string, string][]): void {
assert.deepEqual(inline(text), { attributes: attributes(...pairs), content, length: text.length, name })
}
test('claims a colon-run line only where a name or nothing follows the colons', () => {
assert.equal(readDirectiveLine('Part.'), undefined)
assert.equal(readDirectiveLine(':: two'), undefined)
assert.equal(readDirectiveLine(':panel'), undefined)
assert.equal(readDirectiveLine(' ::rule'), undefined)
})
test('reads a bare colon run as the fence that closes a container', () => {
assert.deepEqual(readDirectiveLine(':::'), { value: { colons: 3, kind: 'closing' } })
assert.deepEqual(readDirectiveLine('::'), { value: { colons: 2, kind: 'closing' } })
assert.deepEqual(readDirectiveLine(':::::: \t'), { value: { colons: 6, kind: 'closing' } })
})
test('reads the leaf and container forms, their argument and their attributes', () => {
assert.deepEqual(readDirectiveLine('::rule'), header(2, 'rule'))
assert.deepEqual(readDirectiveLine('::rule '), header(2, 'rule'))
assert.deepEqual(readDirectiveLine('::taskItem TODO'), header(2, 'taskItem', 'TODO'))
assert.deepEqual(readDirectiveLine('::hardBreak {}'), header(2, 'hardBreak'))
assert.deepEqual(readDirectiveLine('::media {id=a-1 type=file}'), header(2, 'media', undefined, ['id', 'a-1'], ['type', 'file']))
assert.deepEqual(readDirectiveLine('::panel info {panelColor="#ff0000"} '), header(2, 'panel', 'info', ['panelColor', '#ff0000']))
assert.deepEqual(readDirectiveLine(':::panel info'), header(3, 'panel', 'info'))
})
test('decodes a quoted attribute value, the escapes {attrs} reserves included', () => {
assert.deepEqual(readDirectiveLine('::extension {text="two words"}'), header(2, 'extension', undefined, ['text', 'two words']))
assert.deepEqual(readDirectiveLine('::extension {text="a\\u0060b\\u0026c\\u003cd\\u007ce"}'), header(2, 'extension', undefined, ['text', 'a`b&c<d|e']))
assert.deepEqual(readDirectiveLine('::extension {text="a\\"b\\\\c\\nd"}'), header(2, 'extension', undefined, ['text', 'a"b\\c\nd']))
assert.deepEqual(readDirectiveLine('::extension {text="}{"}'), header(2, 'extension', undefined, ['text', '}{']))
})
test('names the directive line no spelling reads', () => {
assert.equal(fault('::Panel'), 'a directive name reads [a-z][A-Za-z0-9]*')
assert.equal(fault('::1panel'), 'a directive name reads [a-z][A-Za-z0-9]*')
assert.equal(fault('::panel info'), 'a directive line reads a name, one bare argument and {attrs}, one space apart')
assert.equal(fault('::panel info extra'), 'a directive line reads a name, one bare argument and {attrs}, one space apart')
assert.equal(fault('::panel{}'), 'a directive line reads a name, one bare argument and {attrs}, one space apart')
assert.equal(fault('::panel info{}'), 'a directive line reads a name, one bare argument and {attrs}, one space apart')
assert.equal(fault('::panel {} x'), 'a directive line reads a name, one bare argument and {attrs}, one space apart')
})
test('names the attributes no spelling reads', () => {
assert.equal(fault('::panel {a=1'), 'the {attrs} closing brace is missing')
assert.equal(fault('::panel {a="x}'), 'the {attrs} quoted value is unclosed')
assert.equal(fault('::panel {a="\\uzzzz"}'), 'the {attrs} quoted value is not a JSON string')
assert.equal(fault('::panel {a}'), 'an attribute reads key=value, the value bare or double-quoted')
assert.equal(fault('::panel {a=}'), 'an attribute reads key=value, the value bare or double-quoted')
assert.equal(fault('::panel {=1}'), 'an attribute reads key=value, the value bare or double-quoted')
assert.equal(fault('::panel {a=1 b=2}'), 'an attribute reads key=value, the value bare or double-quoted')
assert.equal(fault('::panel { a=1}'), 'an attribute reads key=value, the value bare or double-quoted')
assert.equal(fault('::panel {a=1 }'), 'an attribute reads key=value, the value bare or double-quoted')
assert.equal(fault('::panel {a=1 a=2}'), 'the attribute key a is spelled twice')
})
test('breaks the directive on the raw characters a quoted value spells as escapes', () => {
assert.equal(fault('::panel {a="x`y"}'), 'a raw ` inside {attrs} breaks the directive: spell it \\u0060')
assert.equal(fault('::panel {a="x&y"}'), 'a raw & inside {attrs} breaks the directive: spell it \\u0026')
assert.equal(fault('::panel {a="x<y"}'), 'a raw < inside {attrs} breaks the directive: spell it \\u003c')
assert.equal(fault('::panel {a="x|y"}'), 'a raw | inside {attrs} breaks the directive: spell it \\u007c')
})
test('reads an inline directive only where a bracket or a brace follows the name', () => {
assert.equal(inline('Part.'), 'unclaimed')
assert.equal(inline(':10:30'), 'unclaimed')
assert.equal(inline(':smile:'), 'unclaimed')
assert.equal(inline(':Mention[@A]'), 'unclaimed')
assert.equal(inline(':mention @A'), 'unclaimed')
spans(':mention[@A]', 'mention', '@A')
spans(':date{timestamp=1756080000000}', 'date', undefined, ['timestamp', '1756080000000'])
spans(':emoji[]{shortName=":tada:"}', 'emoji', '', ['shortName', ':tada:'])
spans(':underline[ a ]', 'underline', ' a ')
})
test('binds an inline directive as a unit, its content balancing brackets like link text', () => {
spans(':underline[a [b] c]', 'underline', 'a [b] c')
spans(':underline[a \\] b]', 'underline', 'a \\] b')
spans(':underline[a `]` b]', 'underline', 'a `]` b')
spans(':underline[a `b c]', 'underline', 'a `b c')
spans(':underline[:status[x]{color=red}]', 'underline', ':status[x]{color=red}')
spans(':status[x]{color=red style="bold "}', 'status', 'x', ['color', 'red'], ['style', 'bold '])
assert.deepEqual(inline(':underline[a]{}(b)'), { attributes: attributes(), content: 'a', length: 15, name: 'underline' })
assert.deepEqual(inline(':underline[a] {}'), { attributes: attributes(), content: 'a', length: 13, name: 'underline' })
assert.deepEqual(inline(':text{text=" "} and more'), { attributes: attributes(['text', ' ']), content: undefined, length: 15, name: 'text' })
})
test('names the inline directive left unclosed at the end of its line', () => {
assert.equal(inline(':mention[@A'), 'an inline directive [content] is unclosed')
assert.equal(inline(':mention[@A\nB]'), 'an inline directive [content] is unclosed')
assert.equal(inline(':mention[a `b\nc` d]'), 'an inline directive [content] is unclosed')
assert.equal(inline(':underline[:status[x'), 'an inline directive [content] is unclosed')
assert.equal(inline(':mention[@A]{id=1'), 'the {attrs} closing brace is missing')
assert.equal(inline(':mention{id=1'), 'the {attrs} closing brace is missing')
assert.equal(inline(':text{text="a\nb"}'), 'the {attrs} quoted value is not a JSON string')
})
test('refuses inline directives nested deeper than the parser carries', () => {
const nest = (depth: number): string => `${':underline['.repeat(depth)}x${']'.repeat(depth)}`
spans(nest(largestNesting), 'underline', nest(largestNesting - 1))
assert.equal(inline(nest(largestNesting + 1)), `the input nests inline directives deeper than the ${largestNesting} levels the parser carries`)
})
+249
View File
@@ -0,0 +1,249 @@
import type { ConvertFault } from '../result.ts'
import type { JsonValue } from '../json-value.ts'
import type { VocabularyPair } from '../adf/attribute-vocabulary.ts'
import { backslashEscape, claimsDirectiveLine } from './commonmark-grammar.ts'
import { backtickRun, closingBacktickRun } from './backtick-runs.ts'
import { largestNesting } from '../nesting.ts'
import { runLength } from './emphasis-matching.ts'
import { serializeCanonicalJson } from '../canonical-json.ts'
export type DirectiveAttributes = ReadonlyMap<string, string>
export type DirectiveLine =
| { argument: string | undefined; attributes: DirectiveAttributes; colons: number; kind: 'header'; name: string }
| { colons: number; kind: 'closing' }
export type DirectiveSpan = { attributes: DirectiveAttributes; content: string | undefined; length: number; name: string }
export type Read<T> = { fault: ConvertFault; value?: undefined } | { fault?: undefined; value: T }
type Attributes = { attributes: DirectiveAttributes; length: number }
type AttributePair = { end: number; key: string; value: string }
const bareTokenSource = '[A-Za-z0-9_-]+'
const bareRun = new RegExp(bareTokenSource, 'y')
const bareToken = new RegExp(`^${bareTokenSource}$`)
const directiveName = /[a-z][A-Za-z0-9]*/y
const inlineDirectiveOpener = /:[a-z][A-Za-z0-9]*[[{]/y
const lineEnd = /^[ \t]*$/
// spec/flavour.md, Attributes.
const reservedSource = '[&<`|]'
const quotedEscapes = new RegExp(reservedSource, 'g')
const rawReserved = new RegExp(reservedSource)
const noAttributes: DirectiveAttributes = new Map()
const nameFault = 'a directive name reads [a-z][A-Za-z0-9]*'
const pairFault = 'an attribute reads key=value, the value bare or double-quoted'
const shapeFault = 'a directive line reads a name, one bare argument and {attrs}, one space apart'
export function isBareToken(text: string): boolean {
return bareToken.test(text)
}
export function malformedDirective(message: string): ConvertFault {
return { code: 'malformed-directive', message }
}
export function opensInlineDirective(text: string, index: number): boolean {
return inlineDirectiveName(text, index) !== undefined
}
export function readDirectiveLine(line: string): Read<DirectiveLine> | undefined {
if (!claimsDirectiveLine(line)) return undefined
const colons = runLength(line, 0)
const rest = line.slice(colons)
if (lineEnd.test(rest)) return { value: { colons, kind: 'closing' } }
const header = readDirectiveHeader(rest)
if (header.fault !== undefined) return { fault: header.fault }
return { value: { argument: header.value.argument, attributes: header.value.attributes, colons, kind: 'header', name: header.value.name } }
}
export function readInlineDirective(text: string, index: number): Read<DirectiveSpan> | undefined {
return readNestedDirective(text, index, 1)
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => (left < right ? -1 : 1)).map(([key, value]) => `${key}=${value}`)
return `{${spelled.join(' ')}}`
}
export function spellJsonAttribute(value: JsonValue): string {
return quote(serializeCanonicalJson(value, 'compact'))
}
export function spellStringAttribute(text: string): string {
return isBareToken(text) ? text : quote(text)
}
export function spellVocabulary(pairs: readonly VocabularyPair[]): [string, string][] {
return pairs.map((pair): [string, string] => [pair.key, spellAttributeValue(pair)])
}
export function unknownDirectiveFault(name: string): ConvertFault {
return { code: 'unknown-directive-name', message: `the directive name ${name} reads back to no node` }
}
function spellAttributeValue(pair: VocabularyPair): string {
if (pair.kind === 'boolean') return `${pair.value}`
if (pair.kind === 'json') return spellJsonAttribute(pair.value)
if (pair.kind === 'number') return spellStringAttribute(JSON.stringify(pair.value))
return spellStringAttribute(pair.value)
}
function quote(text: string): string {
return JSON.stringify(text).replace(quotedEscapes, (character) => `\\u${escapeDigits(character)}`)
}
function escapeDigits(character: string): string {
return character.charCodeAt(0).toString(16).padStart(4, '0')
}
function inlineDirectiveName(text: string, index: number): string | undefined {
inlineDirectiveOpener.lastIndex = index
const opened = inlineDirectiveOpener.exec(text)?.[0]
return opened === undefined ? undefined : opened.slice(1, -1)
}
function readDirectiveHeader(rest: string): Read<{ argument: string | undefined; attributes: DirectiveAttributes; name: string }> {
directiveName.lastIndex = 0
const name = directiveName.exec(rest)?.[0]
if (name === undefined) return { fault: malformedDirective(nameFault) }
let cursor = name.length
let argument: string | undefined
let attributes = noAttributes
if (rest.charAt(cursor) === ' ' && rest.charAt(cursor + 1) !== '{' && !lineEnd.test(rest.slice(cursor))) {
bareRun.lastIndex = cursor + 1
argument = bareRun.exec(rest)?.[0]
if (argument === undefined) return { fault: malformedDirective(shapeFault) }
cursor += 1 + argument.length
}
if (rest.charAt(cursor) === ' ' && rest.charAt(cursor + 1) === '{') {
const read = readAttributes(rest, cursor + 1)
if (read.fault !== undefined) return { fault: read.fault }
attributes = read.value.attributes
cursor += 1 + read.value.length
}
if (!lineEnd.test(rest.slice(cursor))) return { fault: malformedDirective(shapeFault) }
return { value: { argument, attributes, name } }
}
function readNestedDirective(text: string, index: number, depth: number): Read<DirectiveSpan> | undefined {
const name = inlineDirectiveName(text, index)
if (name === undefined) return undefined
if (depth > largestNesting) {
return { fault: { code: 'unsupported-nesting-depth', message: `the input nests inline directives deeper than the ${largestNesting} levels the parser carries` } }
}
let cursor = index + 1 + name.length
let content: string | undefined
if (text.charAt(cursor) === '[') {
const end = readDirectiveContent(text, cursor + 1, depth)
if (end.fault !== undefined) return { fault: end.fault }
content = text.slice(cursor + 1, end.value)
cursor = end.value + 1
}
let attributes = noAttributes
if (text.charAt(cursor) === '{') {
const read = readAttributes(text, cursor)
if (read.fault !== undefined) return { fault: read.fault }
attributes = read.value.attributes
cursor += read.value.length
}
return { value: { attributes, content, length: cursor - index, name } }
}
// A code span, an escape and a nested directive each bind before the content's own closing bracket.
function readDirectiveContent(text: string, start: number, depth: number): Read<number> {
let brackets = 0
let cursor = start
while (cursor < text.length && text.charAt(cursor) !== '\n') {
const character = text.charAt(cursor)
if (character === '\\') {
cursor += backslashEscape(text, cursor) === undefined ? 1 : 2
continue
}
if (character === '`') {
const span = readCodeSpanEnd(text, cursor)
if (span === undefined) break
cursor = span
continue
}
const nested = character === ':' ? readNestedDirective(text, cursor, depth + 1) : undefined
if (nested !== undefined) {
if (nested.fault !== undefined) return { fault: nested.fault }
cursor += nested.value.length
continue
}
if (character === ']' && brackets === 0) return { value: cursor }
if (character === '[') brackets += 1
if (character === ']') brackets -= 1
cursor += 1
}
return { fault: malformedDirective('an inline directive [content] is unclosed') }
}
// `undefined` where the span crosses the line ending an inline directive may not cross.
function readCodeSpanEnd(text: string, index: number): number | undefined {
const opener = backtickRun(text, index)
const closer = closingBacktickRun(text, index + opener, opener)
if (closer === undefined) return index + opener
return text.slice(index, closer + opener).includes('\n') ? undefined : closer + opener
}
function readAttributes(text: string, index: number): Read<Attributes> {
const attributes = new Map<string, string>()
let cursor = index + 1
while (cursor < text.length && text.charAt(cursor) !== '}') {
if (attributes.size > 0) {
if (text.charAt(cursor) !== ' ') return { fault: malformedDirective(pairFault) }
cursor += 1
}
const pair = readAttributePair(text, cursor)
if (pair.fault !== undefined) return { fault: pair.fault }
if (attributes.has(pair.value.key)) return { fault: malformedDirective(`the attribute key ${pair.value.key} is spelled twice`) }
attributes.set(pair.value.key, pair.value.value)
cursor = pair.value.end
}
if (text.charAt(cursor) !== '}') return { fault: malformedDirective('the {attrs} closing brace is missing') }
return { value: { attributes, length: cursor + 1 - index } }
}
function readAttributePair(text: string, index: number): Read<AttributePair> {
bareRun.lastIndex = index
const key = bareRun.exec(text)?.[0]
if (key === undefined || text.charAt(index + key.length) !== '=') return { fault: malformedDirective(pairFault) }
const start = index + key.length + 1
if (text.charAt(start) === '"') {
const quoted = readQuotedValue(text, start)
if (quoted.fault !== undefined) return { fault: quoted.fault }
return { value: { end: quoted.value.end, key, value: quoted.value.value } }
}
bareRun.lastIndex = start
const value = bareRun.exec(text)?.[0]
if (value === undefined) return { fault: malformedDirective(pairFault) }
return { value: { end: start + value.length, key, value } }
}
function readQuotedValue(text: string, index: number): Read<{ end: number; value: string }> {
let cursor = index + 1
while (cursor < text.length && text.charAt(cursor) !== '"') cursor += text.charAt(cursor) === '\\' ? 2 : 1
if (text.charAt(cursor) !== '"') return { fault: malformedDirective('the {attrs} quoted value is unclosed') }
const raw = text.slice(index, cursor + 1)
const character = rawReserved.exec(raw)?.[0]
if (character !== undefined) {
return { fault: malformedDirective(`a raw ${character} inside {attrs} breaks the directive: spell it \\u${escapeDigits(character)}`) }
}
const value = parseJsonString(raw)
if (value === undefined) return { fault: malformedDirective('the {attrs} quoted value is not a JSON string') }
return { value: { end: cursor + 1, value } }
}
function parseJsonString(raw: string): string | undefined {
try {
const value: unknown = JSON.parse(raw)
return typeof value === 'string' ? value : undefined
} catch {
return undefined
}
}
@@ -2,7 +2,7 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts'
import type { BlockDirective } from '../../adf/block-directives.ts' import type { BlockDirective } from '../../adf/block-directives.ts'
import type { JsonValue } from '../../json-value.ts' import type { JsonValue } from '../../json-value.ts'
import { blockArgument } from '../block-directive-arguments.ts' import { blockArgument } from '../block-directive-arguments.ts'
import { isBareToken, spellAttributes, spellJsonAttribute, spellVocabulary } from '../directive-attributes.ts' import { isBareToken, spellAttributes, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
export function spellDirectiveHeader(node: AdfNode, directive: BlockDirective, spelledByBody: readonly string[] = []): string | undefined { export function spellDirectiveHeader(node: AdfNode, directive: BlockDirective, spelledByBody: readonly string[] = []): string | undefined {
@@ -1,6 +1,6 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfNode } from '../../adf/document.ts'
import type { InlineDirective } from '../../adf/inline-directives.ts' import type { InlineDirective } from '../../adf/inline-directives.ts'
import { spellAttributes, spellVocabulary } from '../directive-attributes.ts' import { spellAttributes, spellVocabulary } from '../directive-syntax.ts'
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
export function spellInlineNodeAttributes(node: AdfNode, directive: InlineDirective): string | undefined { export function spellInlineNodeAttributes(node: AdfNode, directive: InlineDirective): string | undefined {
+1 -1
View File
@@ -10,7 +10,7 @@ import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../backtick-runs.ts' import { longestBacktickRun } from '../backtick-runs.ts'
import { markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { markSpelling, spellMarkAttributes } from '../mark-spellings.ts'
import { sameMark } from '../../adf/editor-normal.ts' import { sameMark } from '../../adf/editor-normal.ts'
import { spellAttributes, spellStringAttribute } from '../directive-attributes.ts' import { spellAttributes, spellStringAttribute } from '../directive-syntax.ts'
import { spellDestination, spellTitle } from '../link-syntax.ts' import { spellDestination, spellTitle } from '../link-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
+1 -1
View File
@@ -1,7 +1,7 @@
import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../backtick-runs.ts'
import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../emphasis-matching.ts' import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../emphasis-matching.ts'
import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark-grammar.ts' import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark-grammar.ts'
import { opensInlineDirective } from '../directive-attributes.ts' import { opensInlineDirective } from '../directive-syntax.ts'
import { readEntityReference } from '../entity-references.ts' import { readEntityReference } from '../entity-references.ts'
export type EmphasisRole = 'close' | 'open' export type EmphasisRole = 'close' | 'open'
+1 -1
View File
@@ -2,7 +2,7 @@ import type { AdfMark } from '../adf/document.ts'
import type { AttributeVocabulary } from '../adf/attribute-vocabulary.ts' import type { AttributeVocabulary } from '../adf/attribute-vocabulary.ts'
import type { MarkType } from '../adf/mark-attributes.ts' import type { MarkType } from '../adf/mark-attributes.ts'
import { isMarkType, markAttributes } from '../adf/mark-attributes.ts' import { isMarkType, markAttributes } from '../adf/mark-attributes.ts'
import { spellAttributes, spellVocabulary } from './directive-attributes.ts' import { spellAttributes, spellVocabulary } from './directive-syntax.ts'
import { vocabularyPairs } from '../adf/attribute-vocabulary.ts' import { vocabularyPairs } from '../adf/attribute-vocabulary.ts'
type Spelling = { kind: 'code' | 'directive' | 'link'; spelling?: undefined } | { kind: 'emphasis'; spelling: string } type Spelling = { kind: 'code' | 'directive' | 'link'; spelling?: undefined } | { kind: 'emphasis'; spelling: string }
+1 -1
View File
@@ -5,7 +5,7 @@ import { isJsonValue } from '../json-value.ts'
import { fencedCodeBlock } from './backtick-runs.ts' import { fencedCodeBlock } from './backtick-runs.ts'
import { largestNesting } from '../nesting.ts' import { largestNesting } from '../nesting.ts'
import { serializeCanonicalJson } from '../canonical-json.ts' import { serializeCanonicalJson } from '../canonical-json.ts'
import { spellAttributes, spellStringAttribute } from './directive-attributes.ts' import { spellAttributes, spellStringAttribute } from './directive-syntax.ts'
export const carryName = 'adf' export const carryName = 'adf'
+45 -2
View File
@@ -1,6 +1,7 @@
import assert from 'node:assert/strict' import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { Block } from './blocks.ts'
import type { LinkDefinition } from '../link-syntax.ts' import type { LinkDefinition } from '../link-syntax.ts'
import { parseBlocks } from './blocks.ts' import { parseBlocks } from './blocks.ts'
@@ -12,6 +13,20 @@ function kinds(markdown: string): string[] {
return parseBlocks(markdown).blocks.map((block) => block.kind) return parseBlocks(markdown).blocks.map((block) => block.kind)
} }
function faults(markdown: string): string[] {
const messages: string[] = []
const walk = (blocks: readonly Block[]): void => {
for (const block of blocks) {
if (block.kind === 'fault') messages.push(block.fault.message)
if (block.kind === 'blockquote') walk(block.blocks)
if (block.kind === 'directive' && block.blocks !== undefined) walk(block.blocks)
if (block.kind === 'bulletList' || block.kind === 'orderedList') for (const item of block.items) walk(item)
}
}
walk(parseBlocks(markdown).blocks)
return messages
}
test('keeps the link reference definitions a paragraph gives up, the first of a label winning', () => { test('keeps the link reference definitions a paragraph gives up, the first of a label winning', () => {
assert.deepEqual(definitions('[a]: /url\n'), [['a', { destination: '/url' }]]) assert.deepEqual(definitions('[a]: /url\n'), [['a', { destination: '/url' }]])
assert.deepEqual(definitions('[Foo Bar]:\n<the url>\n"Title"\n'), [['foo bar', { destination: 'the url', title: 'Title' }]]) assert.deepEqual(definitions('[Foo Bar]:\n<the url>\n"Title"\n'), [['foo bar', { destination: 'the url', title: 'Title' }]])
@@ -61,6 +76,34 @@ test('swallows an HTML block to the end condition its start sets', () => {
}) })
test('carries a claimed line as the block it opens, the refusal the node layer builds', () => { test('carries a claimed line as the block it opens, the refusal the node layer builds', () => {
assert.deepEqual(kinds(':::\nPart.\n'), ['claim', 'paragraph']) assert.deepEqual(kinds(':::\nPart.\n'), ['fault', 'paragraph'])
assert.deepEqual(kinds('Part.\n| x |\n'), ['paragraph', 'claim']) assert.deepEqual(kinds('Part.\n| x |\n'), ['paragraph', 'fault'])
})
test('holds a directive container open until the fence that closes it', () => {
assert.deepEqual(kinds(':::panel info\nPart.\n:::\nMore.\n'), ['directive', 'paragraph'])
assert.deepEqual(kinds('::rule\nPart.\n'), ['directive', 'paragraph'])
assert.deepEqual(faults(':::panel info\n\nPart.\n\n:::\n'), [])
assert.deepEqual(faults(':::panel info\n> Part.\n> :::\n'), [])
assert.deepEqual(faults('::::panel info\n- :::expand\n Part.\n :::\n::::\n'), [])
assert.deepEqual(faults(':::panel info\n```\n:::\n```\n:::\n'), [])
assert.deepEqual(parseBlocks(':::panel info {panelColor="#ff0000"}\nPart.\n:::\n').blocks, [
{
argument: 'info',
attributes: new Map([['panelColor', '#ff0000']]),
blocks: [{ kind: 'paragraph', text: 'Part.' }],
kind: 'directive',
name: 'panel',
},
])
assert.deepEqual(parseBlocks('::rule\n').blocks, [{ argument: undefined, attributes: new Map(), blocks: undefined, kind: 'directive', name: 'rule' }])
})
test('names the directive fence a container does not sit longer than', () => {
assert.deepEqual(faults(':::panel info\n:::expand\nPart.\n:::\n'), ["a directive fence line is at least as long as the container's 3 colons"])
assert.deepEqual(faults('::::panel info\n:::\n::::\n'), ['a closing fence is shorter than the 4 colons it would close'])
assert.deepEqual(faults('::::panel info\n:::expand\nPart.\n:::::\n::::\n'), [])
assert.deepEqual(faults(':::panel info\nPart.\n'), ['a container fenced with 3 colons is unclosed'])
assert.deepEqual(faults('- :::panel info\n\nPart.\n'), ['a container fenced with 3 colons is unclosed'])
assert.deepEqual(faults('Part.\n\n:::\n'), ['a closing fence closes no open container'])
}) })
+85 -14
View File
@@ -1,3 +1,5 @@
import type { ConvertFault } from '../../result.ts'
import type { DirectiveAttributes, DirectiveLine } from '../directive-syntax.ts'
import type { LinkDefinition } from '../link-syntax.ts' import type { LinkDefinition } from '../link-syntax.ts'
import { import {
atxHeading, atxHeading,
@@ -12,14 +14,14 @@ import {
openingHtmlBlock, openingHtmlBlock,
setextHeadingLevel, setextHeadingLevel,
} from '../commonmark-grammar.ts' } from '../commonmark-grammar.ts'
import { malformedDirective, readDirectiveLine } from '../directive-syntax.ts'
import { readLinkDefinitions } from './link-reference-definitions.ts' import { readLinkDefinitions } from './link-reference-definitions.ts'
export type ClaimedConstruct = 'directive' | 'pipe-table'
export type Block = export type Block =
| { argument: string | undefined; attributes: DirectiveAttributes; blocks: Block[] | undefined; kind: 'directive'; name: string }
| { blocks: Block[]; kind: 'blockquote' } | { blocks: Block[]; kind: 'blockquote' }
| { construct: ClaimedConstruct; kind: 'claim' }
| { construct: string; kind: 'html' } | { construct: string; kind: 'html' }
| { fault: ConvertFault; kind: 'fault' }
| { items: Block[][]; kind: 'bulletList' } | { items: Block[][]; kind: 'bulletList' }
| { items: Block[][]; kind: 'orderedList'; start: number } | { items: Block[][]; kind: 'orderedList'; start: number }
| { kind: 'code'; language: string; text: string } | { kind: 'code'; language: string; text: string }
@@ -29,9 +31,16 @@ export type Block =
export type ParsedBlocks = { blocks: Block[]; definitions: Map<string, LinkDefinition> } export type ParsedBlocks = { blocks: Block[]; definitions: Map<string, LinkDefinition> }
type DirectiveBlock = Extract<Block, { kind: 'directive' }>
type ListBlock = Extract<Block, { items: Block[][] }> type ListBlock = Extract<Block, { items: Block[][] }>
type OpenContainer = Extract<Block, { kind: 'blockquote' }> | { blocks: Block[]; indentation: number; kind: 'item'; list: ListBlock; marker: string } type OpenDirective = { blocks: Block[]; colons: number; index: number; kind: 'directive'; parent: Block[] }
type OpenContainer =
| Extract<Block, { kind: 'blockquote' }>
| OpenDirective
| { blocks: Block[]; indentation: number; kind: 'item'; list: ListBlock; marker: string }
type OpenLeaf = type OpenLeaf =
| { closer: RegExp | undefined; construct: string; kind: 'html' } | { closer: RegExp | undefined; construct: string; kind: 'html' }
@@ -49,12 +58,13 @@ type Walk = ParsedBlocks & { leaf: OpenLeaf | undefined; stack: OpenContainer[]
const blankLine = /^[ \t]*$/ const blankLine = /^[ \t]*$/
const indentedCodeColumns = 4 const indentedCodeColumns = 4
const largestOpenerIndentation = 3 const largestOpenerIndentation = 3
const leafColons = 2
const tabStop = 4 const tabStop = 4
export function parseBlocks(markdown: string): ParsedBlocks { export function parseBlocks(markdown: string): ParsedBlocks {
const walk: Walk = { blocks: [], definitions: new Map(), leaf: undefined, stack: [] } const walk: Walk = { blocks: [], definitions: new Map(), leaf: undefined, stack: [] }
for (const text of normalizeInput(markdown).split('\n')) readLine(walk, { column: 0, text }) for (const text of normalizeInput(markdown).split('\n')) readLine(walk, { column: 0, text })
closeLeaf(walk) closeContainers(walk, 0)
return { blocks: walk.blocks, definitions: walk.definitions } return { blocks: walk.blocks, definitions: walk.definitions }
} }
@@ -95,6 +105,8 @@ function matchContainers(walk: Walk, line: Line): { depth: number; rest: Line }
function continuesContainer(walk: Walk, container: OpenContainer, line: Line): Line | undefined { function continuesContainer(walk: Walk, container: OpenContainer, line: Line): Line | undefined {
if (container.kind === 'blockquote') return blockquoteRest(removeColumns(line, largestOpenerIndentation)) if (container.kind === 'blockquote') return blockquoteRest(removeColumns(line, largestOpenerIndentation))
// A directive container has no continuation marker: only its own fence closes it.
if (container.kind === 'directive') return line
// A list item begins with at most one blank line: an empty one gives the second up. // A list item begins with at most one blank line: an empty one gives the second up.
if (blankLine.test(line.text)) { if (blankLine.test(line.text)) {
return container.blocks.length === 0 && walk.leaf === undefined ? undefined : { column: line.column, text: '' } return container.blocks.length === 0 && walk.leaf === undefined ? undefined : { column: line.column, text: '' }
@@ -169,15 +181,73 @@ function openContainer(walk: Walk, start: ContainerStart): void {
function closeContainers(walk: Walk, depth: number): void { function closeContainers(walk: Walk, depth: number): void {
closeLeaf(walk) closeLeaf(walk)
for (const container of walk.stack.slice(depth)) {
if (container.kind !== 'directive') continue
container.parent[container.index] = { fault: malformedDirective(`a container fenced with ${container.colons} colons is unclosed`), kind: 'fault' }
}
dropContainers(walk, depth)
}
function dropContainers(walk: Walk, depth: number): void {
walk.stack.length = depth walk.stack.length = depth
} }
function openDirective(walk: Walk, directive: Extract<DirectiveLine, { kind: 'header' }>): void {
const block: DirectiveBlock = {
argument: directive.argument,
attributes: directive.attributes,
blocks: directive.colons > leafColons ? [] : undefined,
kind: 'directive',
name: directive.name,
}
const parent = currentBlocks(walk)
parent.push(block)
if (block.blocks !== undefined) walk.stack.push({ blocks: block.blocks, colons: directive.colons, index: parent.length - 1, kind: 'directive', parent })
}
function applyDirectiveLine(walk: Walk, directive: DirectiveLine): void {
const enclosing = innermostDirective(walk)
if (directive.kind === 'closing') {
closeDirective(walk, directive.colons, enclosing)
return
}
if (enclosing !== undefined && directive.colons >= enclosing.container.colons) {
pushFault(walk, malformedDirective(`a directive fence line is at least as long as the container's ${enclosing.container.colons} colons`))
return
}
openDirective(walk, directive)
}
function closeDirective(walk: Walk, colons: number, enclosing: { container: OpenDirective; depth: number } | undefined): void {
if (enclosing === undefined) {
pushFault(walk, malformedDirective('a closing fence closes no open container'))
return
}
if (colons < enclosing.container.colons) {
pushFault(walk, malformedDirective(`a closing fence is shorter than the ${enclosing.container.colons} colons it would close`))
return
}
dropContainers(walk, enclosing.depth)
}
function innermostDirective(walk: Walk): { container: OpenDirective; depth: number } | undefined {
for (let depth = walk.stack.length - 1; depth >= 0; depth -= 1) {
const container = walk.stack[depth]
if (container?.kind === 'directive') return { container, depth }
}
return undefined
}
function pushFault(walk: Walk, fault: ConvertFault): void {
currentBlocks(walk).push({ fault, kind: 'fault' })
}
// A claimed line ends the lazy continuation CommonMark would fold it into (spec/flavour.md). // A claimed line ends the lazy continuation CommonMark would fold it into (spec/flavour.md).
function continuesLazily(walk: Walk, line: Line): boolean { function continuesLazily(walk: Walk, line: Line): boolean {
if (walk.leaf?.kind !== 'paragraph' || blankLine.test(line.text)) return false if (walk.leaf?.kind !== 'paragraph' || blankLine.test(line.text)) return false
if (leadingColumns(line) >= indentedCodeColumns) return true if (leadingColumns(line) >= indentedCodeColumns) return true
const opener = removeColumns(line, largestOpenerIndentation).text const opener = removeColumns(line, largestOpenerIndentation).text
if (claimedConstruct(opener) !== undefined || isThematicBreak(opener)) return false if (claimsDirectiveLine(opener) || claimsPipeLine(opener) || isThematicBreak(opener)) return false
return atxHeading(opener) === undefined && openingCodeFence(opener) === undefined && openingHtmlBlock(opener, true) === undefined return atxHeading(opener) === undefined && openingCodeFence(opener) === undefined && openingHtmlBlock(opener, true) === undefined
} }
@@ -220,10 +290,16 @@ function readIndentedCodeLine(leaf: Extract<OpenLeaf, { kind: 'indented-code' }>
function openLeaf(walk: Walk, line: Line): void { function openLeaf(walk: Walk, line: Line): void {
const opener = removeColumns(line, largestOpenerIndentation).text const opener = removeColumns(line, largestOpenerIndentation).text
const claimed = claimedConstruct(opener) const directive = readDirectiveLine(opener)
if (claimed !== undefined) { if (directive !== undefined) {
closeLeaf(walk) closeLeaf(walk)
currentBlocks(walk).push({ construct: claimed, kind: 'claim' }) if (directive.fault === undefined) applyDirectiveLine(walk, directive.value)
else pushFault(walk, directive.fault)
return
}
if (claimsPipeLine(opener)) {
closeLeaf(walk)
pushFault(walk, { code: 'malformed-pipe-table', message: 'the line claims a pipe table and parses as none' })
return return
} }
if (readLineBlock(walk, opener)) return if (readLineBlock(walk, opener)) return
@@ -297,11 +373,6 @@ function currentBlocks(walk: Walk): Block[] {
return walk.stack.at(-1)?.blocks ?? walk.blocks return walk.stack.at(-1)?.blocks ?? walk.blocks
} }
function claimedConstruct(opener: string): ClaimedConstruct | undefined {
if (claimsDirectiveLine(opener)) return 'directive'
return claimsPipeLine(opener) ? 'pipe-table' : undefined
}
function normalizeInput(markdown: string): string { function normalizeInput(markdown: string): string {
return markdown return markdown
.replace(/\r\n?/g, '\n') .replace(/\r\n?/g, '\n')
+17 -1
View File
@@ -4,9 +4,10 @@ import type { LinkDefinition } from '../link-syntax.ts'
import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink } from '../commonmark-grammar.ts' import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink } from '../commonmark-grammar.ts'
import { backtickRun, closingBacktickRun } from '../backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../backtick-runs.ts'
import { delimiterFlags, matchEmphasis, runLength } from '../emphasis-matching.ts' import { delimiterFlags, matchEmphasis, runLength } from '../emphasis-matching.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { mergeAdjacentText } from '../../adf/editor-normal.ts' import { mergeAdjacentText } from '../../adf/editor-normal.ts'
import { normalizeLabel, readInlineTarget, readLabel } from '../link-syntax.ts' import { normalizeLabel, readInlineTarget, readLabel } from '../link-syntax.ts'
import { readInlineDirective, unknownDirectiveFault } from '../directive-syntax.ts'
export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] } export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] }
@@ -50,6 +51,12 @@ export function parseInlineContent(source: string, definitions: LinkDefinitions,
index = angle.value index = angle.value
break break
} }
case ':': {
const directive = readDirective(scan, index)
if (!directive.ok) return directive
index = directive.value
break
}
case '!': case '!':
case '[': case '[':
index = openBracket(scan, index) index = openBracket(scan, index)
@@ -131,6 +138,15 @@ function openBracket(scan: Scan, index: number): number {
return index + width return index + width
} }
function readDirective(scan: Scan, index: number): Result<number> {
const directive = readInlineDirective(scan.source, index)
if (directive === undefined) {
scan.pending += ':'
return success(index + 1)
}
return faulted(directive.fault ?? unknownDirectiveFault(directive.value.name), scan.path)
}
function flush(scan: Scan, strip: boolean): void { function flush(scan: Scan, strip: boolean): void {
const raw = strip ? scan.pending.replace(trailingSpace, '') : scan.pending const raw = strip ? scan.pending.replace(trailingSpace, '') : scan.pending
scan.pending = '' scan.pending = ''
+24 -3
View File
@@ -128,13 +128,34 @@ test('reads an indented code block where no paragraph is open', () => {
test('claims a block-level colon run with no directive to parse it', () => { test('claims a block-level colon run with no directive to parse it', () => {
assert.equal(code(markdownToAdf(':::\n')), 'malformed-directive') assert.equal(code(markdownToAdf(':::\n')), 'malformed-directive')
assert.equal(code(markdownToAdf('::panel\n')), 'malformed-directive') assert.equal(code(markdownToAdf('::Panel\n')), 'malformed-directive')
assert.equal(code(markdownToAdf(' :::panel info\nx\n:::\n')), 'malformed-directive') assert.equal(code(markdownToAdf('::panel {a=1 a=2}\n')), 'malformed-directive')
assert.deepEqual(path(markdownToAdf('Part.\n:::x\n')), ['content', 1])
assert.deepEqual(content(markdownToAdf(':10:30\n')), [paragraph(':10:30')]) assert.deepEqual(content(markdownToAdf(':10:30\n')), [paragraph(':10:30')])
assert.deepEqual(content(markdownToAdf(':: two\n')), [paragraph(':: two')]) assert.deepEqual(content(markdownToAdf(':: two\n')), [paragraph(':: two')])
}) })
test('reads the three directive forms, and names the node none of them reads back to', () => {
assert.equal(code(markdownToAdf('::rule\n')), 'unknown-directive-name')
assert.equal(code(markdownToAdf(' :::panel info\nx\n:::\n')), 'unknown-directive-name')
assert.equal(code(markdownToAdf('Part :mention[@A]{id=b1c2}.\n')), 'unknown-directive-name')
assert.equal(content(markdownToAdf('::rule\n')), 'unknown-directive-name: the directive name rule reads back to no node')
assert.deepEqual(path(markdownToAdf('Part.\n\n::rule\n')), ['content', 1])
assert.equal(content(markdownToAdf('Part.\n:::x\n')), 'malformed-directive: a container fenced with 3 colons is unclosed')
assert.deepEqual(path(markdownToAdf('Part.\n:::x\n')), ['content', 1])
})
test('leaves the colon that opens no directive the text it is', () => {
assert.deepEqual(content(markdownToAdf('At 10:30 :smile: today.\n')), [paragraph('At 10:30 :smile: today.')])
assert.deepEqual(content(markdownToAdf('\\:mention[@A]\n')), [paragraph(':mention[@A]')])
assert.deepEqual(content(markdownToAdf('`:mention[@A]`\n')), [{ content: [codeSpan(':mention[@A]')], type: 'paragraph' }])
})
test('names the inline directive left unclosed at the end of its line', () => {
assert.equal(code(markdownToAdf('Part :mention[@A\n')), 'malformed-directive')
assert.equal(code(markdownToAdf('Part :mention[@A]{id=\n')), 'malformed-directive')
assert.deepEqual(path(markdownToAdf('> Part :mention[@A\n')), ['content', 0, 'content', 0])
})
test('claims a block-level pipe with no table to parse it', () => { test('claims a block-level pipe with no table to parse it', () => {
assert.equal(code(markdownToAdf('| Part | Qty |\n')), 'malformed-pipe-table') assert.equal(code(markdownToAdf('| Part | Qty |\n')), 'malformed-pipe-table')
assert.deepEqual(content(markdownToAdf('\\| Part\n')), [paragraph('| Part')]) assert.deepEqual(content(markdownToAdf('\\| Part\n')), [paragraph('| Part')])
+7 -13
View File
@@ -1,10 +1,11 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import type { Block, ClaimedConstruct } from './blocks.ts' import type { Block } from './blocks.ts'
import type { LinkDefinitions } from './inline-content.ts' import type { LinkDefinitions } from './inline-content.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { parseBlocks } from './blocks.ts' import { parseBlocks } from './blocks.ts'
import { parseInlineContent } from './inline-content.ts' import { parseInlineContent } from './inline-content.ts'
import { unknownDirectiveFault } from '../directive-syntax.ts'
export function markdownToAdf(markdown: string): Result<AdfDocument> { export function markdownToAdf(markdown: string): Result<AdfDocument> {
const parsed = parseBlocks(markdown) const parsed = parseBlocks(markdown)
@@ -30,10 +31,12 @@ function blockNode(block: Block, definitions: LinkDefinitions, path: ConvertErro
return containerNode({ type: 'blockquote' }, block.blocks, definitions, path, depth) return containerNode({ type: 'blockquote' }, block.blocks, definitions, path, depth)
case 'bulletList': case 'bulletList':
return listNode({ type: 'bulletList' }, block.items, definitions, path, depth) return listNode({ type: 'bulletList' }, block.items, definitions, path, depth)
case 'claim':
return claimFailure(block.construct, path)
case 'code': case 'code':
return success(codeBlockNode(block.language, block.text)) return success(codeBlockNode(block.language, block.text))
case 'directive':
return faulted(unknownDirectiveFault(block.name), path)
case 'fault':
return faulted(block.fault, path)
case 'heading': case 'heading':
return contentNode({ attrs: { level: block.level }, type: 'heading' }, block.text, definitions, path) return contentNode({ attrs: { level: block.level }, type: 'heading' }, block.text, definitions, path)
case 'html': case 'html':
@@ -63,15 +66,6 @@ function listNode(node: AdfNode, items: readonly Block[][], definitions: LinkDef
return success({ ...node, content }) return success({ ...node, content })
} }
function claimFailure(construct: ClaimedConstruct, path: ConvertErrorPath): Result<AdfNode> {
switch (construct) {
case 'directive':
return failure('malformed-directive', 'the line claims a directive and parses as none', path)
case 'pipe-table':
return failure('malformed-pipe-table', 'the line claims a pipe table and parses as none', path)
}
}
function codeBlockNode(language: string, text: string): AdfNode { function codeBlockNode(language: string, text: string): AdfNode {
const node: AdfNode = language === '' ? { type: 'codeBlock' } : { attrs: { language }, type: 'codeBlock' } const node: AdfNode = language === '' ? { type: 'codeBlock' } : { attrs: { language }, type: 'codeBlock' }
return text === '' ? node : { ...node, content: [{ text, type: 'text' }] } return text === '' ? node : { ...node, content: [{ text, type: 'text' }] }
+7
View File
@@ -2,6 +2,7 @@ export type ConvertErrorCode =
| 'malformed-directive' | 'malformed-directive'
| 'malformed-pipe-table' | 'malformed-pipe-table'
| 'not-an-adf-document' | 'not-an-adf-document'
| 'unknown-directive-name'
| 'unmappable-html' | 'unmappable-html'
| 'unmappable-image' | 'unmappable-image'
| 'unspellable-adjacent-lists' | 'unspellable-adjacent-lists'
@@ -23,12 +24,18 @@ export type ConvertError = {
path: ConvertErrorPath path: ConvertErrorPath
} }
export type ConvertFault = Omit<ConvertError, 'path'>
export type Result<T> = { error: ConvertError; ok: false } | { ok: true; value: T } export type Result<T> = { error: ConvertError; ok: false } | { ok: true; value: T }
export function failure<T>(code: ConvertErrorCode, message: string, path: ConvertErrorPath): Result<T> { export function failure<T>(code: ConvertErrorCode, message: string, path: ConvertErrorPath): Result<T> {
return { error: { code, message, path }, ok: false } return { error: { code, message, path }, ok: false }
} }
export function faulted<T>(fault: ConvertFault, path: ConvertErrorPath): Result<T> {
return failure(fault.code, fault.message, path)
}
export function success<T>(value: T): Result<T> { export function success<T>(value: T): Result<T> {
return { ok: true, value } return { ok: true, value }
} }
+20 -1
View File
@@ -268,7 +268,7 @@ detail is settled at its own milestone.
break of either kind inside one reads as a space. And a destination or title whose entity break of either kind inside one reads as a space. And a destination or title whose entity
reference decodes to a control character — `[a](/x&#10;y)` — joins 3k's exception list reference decodes to a control character — `[a](/x&#10;y)` — joins 3k's exception list
beside the two above: the reader takes cmark's reading, the emitter has no spelling for it. beside the two above: the reader takes cmark's reading, the emitter has no spelling for it.
- [ ] **3f — The directive grammar.** The three forms — inline `:name[content]{attrs}`, - [x] **3f — The directive grammar.** The three forms — inline `:name[content]{attrs}`,
container `:::name arg {attrs}`, leaf `::name arg {attrs}` — the attribute grammar with container `:::name arg {attrs}`, leaf `::name arg {attrs}` — the attribute grammar with
its quoting and escapes, the fence-length and nesting rules, and the malformed list its quoting and escapes, the fence-length and nesting rules, and the malformed list
`spec/flavour.md` spells, each a named error. `corpus.test.ts`'s `fenceNestingFault` stays a `spec/flavour.md` spells, each a named error. `corpus.test.ts`'s `fenceNestingFault` stays a
@@ -278,11 +278,30 @@ detail is settled at its own milestone.
`|` inside `{attrs}` breaks the directive and is a named error, the author writing the `|` inside `{attrs}` breaks the directive and is a named error, the author writing the
`\u0060` the emitter writes. One precedence covers both directions, and CommonMark's own `\u0060` the emitter writes. One precedence covers both directions, and CommonMark's own
ordering stays untouched. ordering stays untouched.
**Settled** (the maintainer, 2026-09-01): a directive whose name reads back to no node
takes its own code, `unknown-directive-name` — a well-formed spelling the vocabulary does
not hold is not a malformed one, and §8's "erroring input gaining meaning later is MINOR"
is what a consumer switches the two apart for. And input reads canonical spacing only: one
space parting the name, the argument, `{attrs}` and each attribute pair, no padding inside
the braces, trailing whitespace on a directive block line tolerated — §8 makes loosening a
MINOR, so strict is the reversible direction. `directive-attributes.ts` becomes
`directive-syntax.ts` with the readers in it: the whole directive grammar, both
directions, beside the escaping regexes and the spellings it must not drift from. And the
500-level guards compose here for the first time — a recursive reader stacked on the block
walk — so `nesting-depth-composed` pins both axes now rather than waiting for 3i's third.
A closing fence closes the innermost open container however long its run, which
`spec/flavour.md`'s closing-fence sentence now says: a run reaching past the innermost
leaves the fence it did not close a named error, which §2 prefers to closing more than the
author wrote.
- [ ] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first - [ ] **3g — The node tables read backwards.** `commonmark-subset/` reads back, the first
directory to. A parsed directive becomes its node: the name to the type and an unknown one directory to. A parsed directive becomes its node: the name to the type and an unknown one
to a named error, the arg to the attribute it names, each value to the type its section to a named error, the arg to the attribute it names, each value to the type its section
assigns, the body to `content`, the reserved `marks` key to the marks array. 3a's drift assigns, the body to `content`, the reserved `marks` key to the marks array. 3a's drift
guard is built here if the answer there was yes. guard is built here if the answer there was yes.
3f leaves two here: `Read<T>` moves to `src/result.ts` once a second reader takes it, and
the reserved `adf` name in block position needs an error of its own — 3f reports it as
`unknown-directive-name`, which §8 makes the signal that a later MINOR may give the name
meaning, and `adf` never will.
- [ ] **3h — The block nodes.** `block-nodes/` reads back: the plain-versus-directive choice 2f - [ ] **3h — The block nodes.** `block-nodes/` reads back: the plain-versus-directive choice 2f
settles, read from the other side; the `codeBlock` directive's fenced body and the settles, read from the other side; the `codeBlock` directive's fenced body and the
`language` attribute a bare fence leaves it; the media family's composition; both table `language` attribute a bare fence leaves it; the media family's composition; both table