35b - the writer spells the plain flavour #146

Merged
lilleman merged 14 commits from 35b into main 2026-09-29 18:40:30 +02:00
16 changed files with 418 additions and 169 deletions
+6 -1
View File
@@ -147,12 +147,17 @@ read replaces mentions, attachments and macros with text.
`_(image not included)_`, `_(jira-issues-table not included)_`, `_(synced block not included)_`, `_(image not included)_`, `_(jira-issues-table not included)_`, `_(synced block not included)_`,
`_(link card not included)_`, `_(extension not included)_`. `_(link card not included)_`, `_(extension not included)_`.
- `backgroundColor`, `code`, `em`, `link`, `strike` and `strong` stay; every other mark drops, - `backgroundColor`, `code`, `em`, `link`, `strike` and `strong` stay; every other mark drops,
keeping its text, and so does a mark CommonMark cannot spell where it stands. keeping its text, and so does a mark the flavour cannot spell where it stands.
- A newline in text is a hard break and in an expand's title a space, edge whitespace outside a - A newline in text is a hard break and in an expand's title a space, edge whitespace outside a
link or code span is trimmed, carriage returns and null characters are removed, and an empty link or code span is trimmed, carriage returns and null characters are removed, and an empty
paragraph drops. paragraph drops.
- An ordered list numbered past `999999999`, or adjacent ordered lists whose numbering does not - An ordered list numbered past `999999999`, or adjacent ordered lists whose numbering does not
continue, is one bullet list keeping its numbers as text. continue, is one bullet list keeping its numbers as text.
- A task list beside a bullet or decision list, or holding a block other than a task, joins one
bullet list keeping its states as text: `- \[x] Done`.
- Text that would read as a marker takes a backslash: `==` wherever it could open or close a
highlight, `[!…]` opening a quote, and `[x]` or `[ ]` opening any list item, since GitHub reads
that marker per item.
- A node read back carries no `localId`, which Atlassian's schema requires on `taskList`, - A node read back carries no `localId`, which Atlassian's schema requires on `taskList`,
`taskItem` and `blockTaskItem`: mint one where the receiving site requires it. `taskItem` and `blockTaskItem`: mint one where the receiving site requires it.
+1 -3
View File
@@ -67,9 +67,7 @@ normalizes to it through the round-trip.
matching below, which is what lets the emitter decide its own pairings. matching below, which is what lets the emitter decide its own pairings.
- Blocks separated by one blank line at document level, inside a blockquote and between CommonMark - Blocks separated by one blank line at document level, inside a blockquote and between CommonMark
blocks; inside a directive container a pair holding a directive block takes none. No trailing blocks; inside a directive container a pair holding a directive block takes none. No trailing
whitespace outside a code whitespace outside a code block's content, single trailing newline; a document with no blocks is the empty string.
block's
content, single trailing newline; a document with no blocks is the empty string.
## Directives ## Directives
+19 -2
View File
@@ -2,9 +2,10 @@ import assert from 'node:assert/strict'
import fc from 'fast-check' import fc from 'fast-check'
import test from 'node:test' import test from 'node:test'
import type { AdfNode } from '../adf/document.ts'
import { adfDocument, propertyRuns, propertyTimeout } from './property-harness.ts' import { adfDocument, propertyRuns, propertyTimeout } from './property-harness.ts'
import { adfToMarkdown } from '../markdown/emit/adf-to-markdown.ts' import { adfToMarkdown } from '../markdown/emit/adf-to-markdown.ts'
import { adfToPlainMarkdown } from '../markdown/emit/plain-reduction.ts' import { adfToPlainMarkdown, reduceToPlain } from '../markdown/emit/plain-reduction.ts'
import { directivePrefix } from '../markdown/directive-syntax.ts' import { directivePrefix } from '../markdown/directive-syntax.ts'
import { markdownToAdf, plainMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' import { markdownToAdf, plainMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts'
import { toEditorNormal } from '../adf/editor-normal.ts' import { toEditorNormal } from '../adf/editor-normal.ts'
@@ -19,6 +20,20 @@ function readsNoDirective(markdown: string): boolean {
return read.ok && renamed.ok && JSON.stringify(read.value).replaceAll(directivePrefix, renamedPrefix) === JSON.stringify(renamed.value) return read.ok && renamed.ok && JSON.stringify(read.value).replaceAll(directivePrefix, renamedPrefix) === JSON.stringify(renamed.value)
} }
// Each block's text, an expand's title and an image's alt and url, in document order: what the plain pair keeps.
function shownText(nodes: readonly AdfNode[]): string[] {
const shown: string[] = []
for (const node of nodes) {
const attrs = node.attrs ?? {}
if ((node.type === 'expand' || node.type === 'nestedExpand') && typeof attrs['title'] === 'string') shown.push(attrs['title'])
if (node.type === 'media') shown.push(`${JSON.stringify(attrs['alt'] ?? '')} ${JSON.stringify(attrs['url'])}`)
const content = node.content ?? []
if (content.some((child) => child.type === 'text' || child.type === 'hardBreak')) shown.push(content.map((child) => child.text ?? '\n').join(''))
else for (const text of shownText(content)) shown.push(text)
}
return shown.filter((text) => text !== '')
}
test('a generated document refuses to emit, or its markdown reads back to it', { timeout: propertyTimeout }, () => { test('a generated document refuses to emit, or its markdown reads back to it', { timeout: propertyTimeout }, () => {
fc.assert( fc.assert(
fc.property(adfDocument, (document) => { fc.property(adfDocument, (document) => {
@@ -32,7 +47,7 @@ test('a generated document refuses to emit, or its markdown reads back to it', {
) )
}) })
test('a generated document writes plain markdown refusing only what the guard refuses, and that markdown reads back to itself', { timeout: propertyTimeout }, () => { test('a generated document writes plain markdown refusing only what the guard refuses, and that markdown reads back to its text and to itself', { timeout: propertyTimeout }, () => {
fc.assert( fc.assert(
fc.property(adfDocument, (document) => { fc.property(adfDocument, (document) => {
const written = adfToPlainMarkdown(document) const written = adfToPlainMarkdown(document)
@@ -40,6 +55,8 @@ test('a generated document writes plain markdown refusing only what the guard re
assert.ok(readsNoDirective(written.value), `a directive in ${JSON.stringify(written.value)}`) assert.ok(readsNoDirective(written.value), `a directive in ${JSON.stringify(written.value)}`)
const read = plainMarkdownToAdf(written.value) const read = plainMarkdownToAdf(written.value)
assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(written.value)}`) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(written.value)}`)
const reduced = reduceToPlain(document)
assert.deepEqual(shownText(read.value.content ?? []), reduced.ok ? shownText(reduced.value.content ?? []) : reduced, `reading ${JSON.stringify(written.value)}`)
assert.deepEqual(adfToPlainMarkdown(read.value), written, `reading ${JSON.stringify(written.value)}`) assert.deepEqual(adfToPlainMarkdown(read.value), written, `reading ${JSON.stringify(written.value)}`)
}), }),
propertyRuns(gateRuns), propertyRuns(gateRuns),
+6 -1
View File
@@ -23,7 +23,7 @@ export const propertyTimeout = 600000
const depthIdentifier = fc.createDepthIdentifier() const depthIdentifier = fc.createDepthIdentifier()
const emptyCell: AdfNode = { content: [{ type: 'paragraph' }], type: 'tableCell' } const emptyCell: AdfNode = { content: [{ type: 'paragraph' }], type: 'tableCell' }
const flatCommonMarkShapeWeight = 4 const flatCommonMarkShapeWeight = 4
export const markdownPieces = fc.constantFrom(...'aZ09 \t\n!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~é\xa0🎉', 'ab:', 'http://', directivePrefix, `${directivePrefix}a[`, `${directivePrefix}a{`) export const markdownPieces = fc.constantFrom(...'aZ09 \t\n!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~é\xa0🎉', '==', '[!NOTE]', '[x]', 'ab:', 'http://', directivePrefix, `${directivePrefix}a[`, `${directivePrefix}a{`)
const nestingCommonMarkShapeWeight = 21 const nestingCommonMarkShapeWeight = 21
const spelledTypes = new Set(['text', ...Object.keys(blockNodes), ...Object.keys(inlineNodes), ...Object.keys(markAttributes)]) const spelledTypes = new Set(['text', ...Object.keys(blockNodes), ...Object.keys(inlineNodes), ...Object.keys(markAttributes)])
@@ -80,6 +80,7 @@ function pipeTable({ body, header }: { body: AdfNode[][]; header: AdfNode[] }):
const mark: Arbitrary<AdfMark> = fc.oneof( const mark: Arbitrary<AdfMark> = fc.oneof(
{ arbitrary: fc.oneof(...Object.entries(markAttributes).map(([type, vocabulary]) => attributes(vocabulary).map((attrs) => ({ attrs, type })))), weight: 9 }, { arbitrary: fc.oneof(...Object.entries(markAttributes).map(([type, vocabulary]) => attributes(vocabulary).map((attrs) => ({ attrs, type })))), weight: 9 },
{ arbitrary: attributes({ color: 'string' }).map((attrs) => ({ attrs, type: 'backgroundColor' })), weight: 2 },
{ arbitrary: fc.record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), type: unknownType }), weight: 1 }, { arbitrary: fc.record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), type: unknownType }), weight: 1 },
) )
const marks = fc.uniqueArray(mark, { maxLength: 3, selector: (held) => held.type }) const marks = fc.uniqueArray(mark, { maxLength: 3, selector: (held) => held.type })
@@ -135,8 +136,12 @@ const positions = fc.letrec<Positions>((tie) => {
paragraph, paragraph,
fc.record({ body: fc.array(fc.array(cell('tableCell'), { maxLength: 3 }), { maxLength: 2 }), header: fc.array(cell('tableHeader'), { maxLength: 3, minLength: 1 }) }).map(pipeTable), fc.record({ body: fc.array(fc.array(cell('tableCell'), { maxLength: 3 }), { maxLength: 2 }), header: fc.array(cell('tableHeader'), { maxLength: 3, minLength: 1 }) }).map(pipeTable),
] ]
const task = (type: string, content: Arbitrary<AdfNode[]>) =>
fc.record({ content, state: fc.constantFrom('DONE', 'TODO') }).map(({ content: held, state }): AdfNode => ({ attrs: { state }, content: held, type }))
const taskItem = fc.oneof({ arbitrary: task('taskItem', inlineContent), weight: 3 }, { arbitrary: task('blockTaskItem', blockContent), weight: 1 })
const nestingCommonMarkShapes = [ const nestingCommonMarkShapes = [
blockContent.map((content): AdfNode => ({ content, type: 'blockquote' })), blockContent.map((content): AdfNode => ({ content, type: 'blockquote' })),
fc.array(fc.oneof({ arbitrary: taskItem, weight: 3 }, { arbitrary: tie('block'), weight: 1 }), { depthIdentifier, maxLength: 3, minLength: 1 }).map((content): AdfNode => ({ content, type: 'taskList' })),
listItems.map((content): AdfNode => ({ content, type: 'bulletList' })), listItems.map((content): AdfNode => ({ content, type: 'bulletList' })),
fc fc
.record({ content: listItems, order: fc.oneof({ arbitrary: fc.integer({ max: 3, min: 0 }), weight: 4 }, { arbitrary: fc.integer({ max: 999999999, min: 0 }), weight: 1 }) }) .record({ content: listItems, order: fc.oneof({ arbitrary: fc.integer({ max: 3, min: 0 }), weight: 4 }, { arbitrary: fc.integer({ max: 999999999, min: 0 }), weight: 1 }) })
+109 -33
View File
@@ -1,6 +1,8 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts'
import type { Flavour } from '../plain-conventions.ts'
import { adfDocumentFault, carriesOnly, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { adfDocumentFault, carriesOnly, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain-conventions.ts'
import { blockDirectiveForm, listBreakSpelling } from '../block-directive.ts' import { blockDirectiveForm, listBreakSpelling } from '../block-directive.ts'
import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts'
import { carriedBlock } from '../opaque-carry.ts' import { carriedBlock } from '../opaque-carry.ts'
@@ -24,28 +26,33 @@ type PlacedBlock = Omit<EmittedBlock, 'headroom'> & { node: AdfNode }
export type SpellingMemo = Map<AdfNode, KeptSpelling> export type SpellingMemo = Map<AdfNode, KeptSpelling>
type Walk = { blocks: readonly PlacedBlock[]; headroom: number } type Walk = { blocks: readonly PlacedBlock[]; headroom: number }
type WalkedItem = { node: AdfNode; walk: Walk } type WalkedItem = { node: AdfNode; walk: Walk }
export type Writing = { flavour: Flavour; memo: SpellingMemo | undefined }
export const largestListMarker = 999999999 export const largestListMarker = 999999999
// Bare because tryList admits no item carrying attributes, marks or text. // Bare because tryList admits no item carrying attributes, marks or text.
const listItemOpener = spellDirectiveOpener('listItem', undefined, '') const listItemOpener = spellDirectiveOpener('listItem', undefined, '')
export function adfToMarkdown(document: AdfDocument): Result<string> { export function adfToMarkdown(document: AdfDocument): Result<string> {
return writeMarkdown(document, 'lossless')
}
export function writeMarkdown(document: AdfDocument, flavour: Flavour): Result<string> {
const fault = adfDocumentFault(document) const fault = adfDocumentFault(document)
if (fault !== undefined) return faulted(fault, []) if (fault !== undefined) return faulted(fault, [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
const walk = walkBlocks(nodeContent(document), [], 0, undefined) const walk = walkBlocks(nodeContent(document), [], 0, { flavour, memo: undefined })
if (!walk.ok) return walk if (!walk.ok) return walk
const text = joinBlocks(walk.value.blocks, 'document') const text = joinBlocks(walk.value.blocks, 'document')
return success(text === '' ? '' : `${text}\n`) return success(text === '' ? '' : `${text}\n`)
} }
// headroom: the least slack any depth guard below the walk has. // headroom: the least slack any depth guard below the walk has.
function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, memo: SpellingMemo | undefined): Result<Walk> { function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<Walk> {
let headroom = largestNesting - depth let headroom = largestNesting - depth
if (headroom < 0) return tooDeep(path) if (headroom < 0) return tooDeep(path)
const blocks: PlacedBlock[] = [] const blocks: PlacedBlock[] = []
for (const [index, node] of nodes.entries()) { for (const [index, node] of nodes.entries()) {
const block = emitBlock(node, [...path, 'content', index], depth, memo) const block = emitBlock(node, [...path, 'content', index], depth, writing)
if (!block.ok) return block if (!block.ok) return block
headroom = Math.min(headroom, block.value.headroom) headroom = Math.min(headroom, block.value.headroom)
blocks.push({ ...block.value, node }) blocks.push({ ...block.value, node })
@@ -81,51 +88,120 @@ function separationBetween(previous: PlacedBlock, next: PlacedBlock, container:
function interruptsParagraph(node: AdfNode): boolean { function interruptsParagraph(node: AdfNode): boolean {
const items = nodeContent(node) const items = nodeContent(node)
const empty = items[0] === undefined || nodeContent(items[0]).length === 0 const empty = node.type !== 'taskList' && (items[0] === undefined || nodeContent(items[0]).length === 0)
if (node.type !== 'orderedList') return markerInterruptsParagraph(undefined, empty) if (node.type !== 'orderedList') return markerInterruptsParagraph(undefined, empty)
return markerInterruptsParagraph(listStart(node, items.length) ?? 0, empty) return markerInterruptsParagraph(listStart(node, items.length) ?? 0, empty)
} }
function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, memo: SpellingMemo | undefined): Result<EmittedBlock> { function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const model = blockNodeModel(node.type) const model = blockNodeModel(node.type)
if (model === undefined) return commonMarkLine(carriedBlock(node, path, depth)) if (model === undefined) return commonMarkLine(carriedBlock(node, path, depth))
const readable = readableBlock(node, path, depth, memo) const readable = readableBlock(node, path, depth, writing)
if (readable !== undefined) return readable if (readable !== undefined) return readable
return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, memo)) return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing))
} }
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, memo: SpellingMemo): Result<null> | undefined { export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<null> | undefined {
const readable = readableBlock(node, path, depth, memo) const readable = readableBlock(node, path, depth, writing)
if (readable === undefined) return undefined if (readable === undefined) return undefined
if (!readable.ok) return readable if (!readable.ok) return readable
return readable.value.spelling === 'directive' ? undefined : success(null) return readable.value.spelling === 'directive' ? undefined : success(null)
} }
function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, memo: SpellingMemo | undefined): Result<EmittedBlock> | undefined { function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
const { memo } = writing
const kept = memo?.get(node) const kept = memo?.get(node)
if (kept !== undefined) { if (kept !== undefined) {
if (kept.block === undefined) return undefined if (kept.block === undefined) return undefined
// A read below the fill would skip the depth guards the walk it replaces runs (docs/decisions.md §The spelling memo). // A read below the fill would skip the depth guards the walk it replaces runs (docs/decisions.md §The spelling memo).
if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth }) if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth })
} }
const spelled = spellReadableBlock(node, path, depth, memo) const spelled = spellReadableBlock(node, path, depth, writing)
if (spelled === undefined) memo?.set(node, { block: undefined, depth }) if (spelled === undefined) memo?.set(node, { block: undefined, depth })
else if (spelled.ok) memo?.set(node, { block: spelled.value, depth }) else if (spelled.ok) memo?.set(node, { block: spelled.value, depth })
return spelled return spelled
} }
function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, memo: SpellingMemo | undefined): Result<EmittedBlock> | undefined { function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, memo) const plain = writing.flavour === 'plain' ? spellPlainBlock(node, path, depth, writing) : undefined
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, memo) if (plain !== undefined) return plain
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
if (node.type === 'codeBlock') return tryCodeBlock(node, path) if (node.type === 'codeBlock') return tryCodeBlock(node, path)
if (node.type === 'heading') return tryHeading(node, path) if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
if (node.type === 'mediaSingle') return readableText(tryImage(node, path)) if (node.type === 'mediaSingle') return readableText(tryImage(node, path))
if (node.type === 'paragraph') return tryParagraph(node, path) if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
if (node.type === 'rule') return readableText(tryRule(node)) if (node.type === 'rule') return readableText(tryRule(node))
if (node.type === 'table') return readableText(tryPipeTable(node, path)) if (node.type === 'table') return readableText(tryPipeTable(node, path, writing.flavour))
return undefined return undefined
} }
// The plain flavour's nodes, in the shapes the plain reduction leaves them.
function spellPlainBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
if (node.type === 'panel') return quotedUnder(alertMarker(nodeAttrs(node)['panelType']), node, path, depth, writing)
if (node.type === 'taskList') return tryTaskList(node, path, depth, writing)
if (node.type !== 'expand' && node.type !== 'nestedExpand') return undefined
const title = nodeAttrs(node)['title']
if (typeof title !== 'string') return quotedUnder(foldedAlertMarker, node, path, depth, writing)
// The reader takes a title as lossless inline text.
const line = emitInlineLine([{ text: title, type: 'text' }], 'paragraph', path, 'lossless')
return line.ok ? quotedUnder(`${foldedAlertMarker} ${line.value}`, node, path, depth, writing) : line
}
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
if (!inner.ok) return inner
const body = joinBlocks(inner.value.blocks, 'document')
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
}
function quoted(text: string): string {
return text
.split('\n')
.map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n')
}
function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const items: PlacedBlock[][] = []
let headroom = largestNesting - depth - 1
// A child other than a task nests in the task before it.
for (const [index, child] of nodeContent(node).entries()) {
const task = child.type === 'taskItem' || child.type === 'blockTaskItem'
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, writing) : placedBlock(child, [...path, 'content', index], depth + 1, writing)
if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom)
const previous = items.at(-1)
if (task || previous === undefined) items.push([...walk.value.blocks])
else for (const block of walk.value.blocks) previous.push(block)
}
const lines = items.map((blocks) => tryListItemLines(joinBlocks(blocks, 'list-item'), '- '))
// The plain reduction leaves no task a list item cannot hold: a directive here would break the flavour.
return lines.includes(undefined) ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success({ headroom, spelling: 'list', text: lines.join('\n') })
}
function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<Walk> {
const block = emitBlock(node, path, depth, writing)
return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom }) : block
}
// The marker leads the first paragraph, or stands as one where the blocks open with another.
function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<Walk> {
const marker = taskMarker(nodeAttrs(task)['state'])
const markerBlock: PlacedBlock = { node: { type: 'paragraph' }, spelling: 'commonmark', text: marker }
if (task.type === 'taskItem') {
const content = nodeContent(task)
const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour)
if (!line.ok) return line
return success({ blocks: [{ ...markerBlock, text: line.value === '' ? marker : `${marker} ${line.value}` }], headroom: Number.POSITIVE_INFINITY })
}
const walk = walkBlocks(nodeContent(task), path, depth, writing)
if (!walk.ok) return walk
const [first, ...rest] = walk.value.blocks
const blocks = first?.node.type === 'paragraph' ? [{ ...first, text: `${marker} ${first.text}` }, ...rest] : [markerBlock, ...walk.value.blocks]
return success({ blocks, headroom: walk.value.headroom })
}
function readableText(text: string | undefined): Result<EmittedBlock> | undefined { function readableText(text: string | undefined): Result<EmittedBlock> | undefined {
return text === undefined ? undefined : success(commonMarkText(text)) return text === undefined ? undefined : success(commonMarkText(text))
} }
@@ -156,7 +232,7 @@ function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertE
function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, path: ConvertErrorPath, walkBody: () => Result<Walk>): Result<EmittedBlock> { function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, path: ConvertErrorPath, walkBody: () => Result<Walk>): Result<EmittedBlock> {
if (blockDirectiveForm(node.type) === 'leaf') return success({ headroom: Number.POSITIVE_INFINITY, spelling: 'directive', text: opener }) if (blockDirectiveForm(node.type) === 'leaf') return success({ headroom: Number.POSITIVE_INFINITY, spelling: 'directive', text: opener })
if (model.contentModel === 'inline') { if (model.contentModel === 'inline') {
const line = emitInlineLine(nodeContent(node), 'paragraph', path) const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless')
if (!line.ok) return line if (!line.ok) return line
return success(directivePair(node, opener, line.value)) return success(directivePair(node, opener, line.value))
} }
@@ -165,15 +241,13 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom)) return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom))
} }
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, memo: SpellingMemo | undefined): Result<EmittedBlock> | undefined { function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
if (!carriesOnly(node, [])) return undefined if (!carriesOnly(node, [])) return undefined
const inner = walkBlocks(nodeContent(node), path, depth + 1, memo) const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
if (!inner.ok) return inner if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document') const text = joinBlocks(inner.value.blocks, 'document')
.split('\n') const alert = writing.flavour === 'plain' && leadingMarker(text, readAlertMarker) !== undefined
.map((line) => (line === '' ? '>' : `> ${line}`)) return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom))
.join('\n')
return success(commonMarkText(text, inner.value.headroom))
} }
function tryCodeBlock(node: AdfNode, path: ConvertErrorPath): Result<EmittedBlock> | undefined { function tryCodeBlock(node: AdfNode, path: ConvertErrorPath): Result<EmittedBlock> | undefined {
@@ -216,19 +290,19 @@ function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result<string> {
return success(text) return success(text)
} }
function tryHeading(node: AdfNode, path: ConvertErrorPath): Result<EmittedBlock> | undefined { function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result<EmittedBlock> | undefined {
if (!carriesOnly(node, ['level'])) return undefined if (!carriesOnly(node, ['level'])) return undefined
const level = nodeAttrs(node)['level'] const level = nodeAttrs(node)['level']
if (typeof level !== 'number' || !Number.isInteger(level) || level < 1 || level > 6) return undefined if (typeof level !== 'number' || !Number.isInteger(level) || level < 1 || level > 6) return undefined
const hashes = '#'.repeat(level) const hashes = '#'.repeat(level)
const content = nodeContent(node) const content = nodeContent(node)
if (content.length === 0) return success(commonMarkText(hashes)) if (content.length === 0) return success(commonMarkText(hashes))
const line = emitInlineLine(content, 'heading', path) const line = emitInlineLine(content, 'heading', path, flavour)
if (!line.ok) return line if (!line.ok) return line
return success(commonMarkText(`${hashes} ${line.value}`)) return success(commonMarkText(`${hashes} ${line.value}`))
} }
function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, memo: SpellingMemo | undefined): Result<EmittedBlock> | undefined { function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
const ordered = node.type === 'orderedList' const ordered = node.type === 'orderedList'
if (!carriesOnly(node, ordered ? ['order'] : [])) return undefined if (!carriesOnly(node, ordered ? ['order'] : [])) return undefined
const items = nodeContent(node) const items = nodeContent(node)
@@ -238,14 +312,17 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, memo: Spe
const walked: WalkedItem[] = [] const walked: WalkedItem[] = []
let headroom = Number.POSITIVE_INFINITY let headroom = Number.POSITIVE_INFINITY
for (const [offset, item] of items.entries()) { for (const [offset, item] of items.entries()) {
const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, memo) const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, writing)
if (!walk.ok) return walk if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom) headroom = Math.min(headroom, walk.value.headroom)
walked.push({ node: item, walk: walk.value }) walked.push({ node: item, walk: walk.value })
} }
const lines: string[] = [] const lines: string[] = []
for (const [offset, item] of walked.entries()) { for (const [offset, item] of walked.entries()) {
const line = tryListItemLines(item.walk.blocks, ordered ? `${start + offset}. ` : '- ') const inner = joinBlocks(item.walk.blocks, 'list-item')
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
const escaped = writing.flavour === 'plain' && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner
const line = tryListItemLines(escaped, ordered ? `${start + offset}. ` : '- ')
if (line === undefined) { if (line === undefined) {
// The directive form spends a level the walk did not count. // The directive form spends a level the walk did not count.
if (headroom < 1) return tooDeep(path) if (headroom < 1) return tooDeep(path)
@@ -267,8 +344,7 @@ function listStart(node: AdfNode, items: number): number | undefined {
return start + items - 1 > largestListMarker ? undefined : start return start + items - 1 > largestListMarker ? undefined : start
} }
function tryListItemLines(blocks: readonly PlacedBlock[], marker: string): string | undefined { function tryListItemLines(inner: string, marker: string): string | undefined {
const inner = joinBlocks(blocks, 'list-item')
if (inner === '') return marker.trimEnd() if (inner === '') return marker.trimEnd()
const body = inner.split('\n') const body = inner.split('\n')
if (body.some((line) => line !== '' && isBlankLine(line))) return undefined if (body.some((line) => line !== '' && isBlankLine(line))) return undefined
@@ -278,10 +354,10 @@ function tryListItemLines(blocks: readonly PlacedBlock[], marker: string): strin
return lines.join('\n') return lines.join('\n')
} }
function tryParagraph(node: AdfNode, path: ConvertErrorPath): Result<EmittedBlock> | undefined { function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result<EmittedBlock> | undefined {
const content = nodeContent(node) const content = nodeContent(node)
if (content.length === 0 || !carriesOnly(node, [])) return undefined if (content.length === 0 || !carriesOnly(node, [])) return undefined
const line = emitInlineLine(content, 'paragraph', path) const line = emitInlineLine(content, 'paragraph', path, flavour)
if (!line.ok) return line if (!line.ok) return line
return success(commonMarkText(line.value)) return success(commonMarkText(line.value))
} }
+33 -19
View File
@@ -1,4 +1,5 @@
import type { AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfMark, AdfNode } from '../../adf/document.ts'
import type { Flavour } from '../plain-conventions.ts'
import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import type { InlineNodeModel } from '../../adf/inline-nodes.ts'
import type { LineContainer } from '../line-container.ts' import type { LineContainer } from '../line-container.ts'
import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange } from './line-escaping.ts' import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange } from './line-escaping.ts'
@@ -7,6 +8,7 @@ import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark
import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts'
import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { highlightDelimiter } from '../plain-conventions.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../commonmark/backtick-runs.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts'
@@ -24,6 +26,7 @@ type InlineContext = {
atBlockEnd: boolean atBlockEnd: boolean
bracketed: boolean bracketed: boolean
carried: ReadonlySet<number> carried: ReadonlySet<number>
flavour: Flavour
openingLinkAsDirective: boolean openingLinkAsDirective: boolean
path: ConvertErrorPath path: ConvertErrorPath
spansLines: boolean spansLines: boolean
@@ -33,32 +36,32 @@ type InlineRun = { index: number; kind: 'marked'; mark: AdfMark; nodes: AdfNode[
type LineAttempt = { fallback: NodeRange | 'opening-link'; line?: undefined } | { fallback?: undefined; line: string } type LineAttempt = { fallback: NodeRange | 'opening-link'; line?: undefined } | { fallback?: undefined; line: string }
type LineFallbacks = { carried: Set<number>; openingLinkAsDirective: boolean } type LineFallbacks = { carried: Set<number>; flavour: Flavour; openingLinkAsDirective: boolean }
export type PlainLineFallback = { kind: 'claimed-line'; line: number; text: string } | { kind: 'opening-link' } | { kind: 'unspellable-run'; run: MarkRun } export type PlainLineFallback = { kind: 'claimed-line'; line: number; text: string } | { kind: 'opening-link' } | { kind: 'unspellable-run'; runs: [MarkRun, ...MarkRun[]] }
export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<string> { export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result<string> {
const emitted = emitLine(nodes, container, path) const emitted = emitLine(nodes, container, path, flavour)
if (!emitted.ok) return emitted if (!emitted.ok) return emitted
return success(emitted.value.line) return success(emitted.value.line)
} }
export function openingLinkTakesDirective(nodes: readonly AdfNode[], path: ConvertErrorPath): Result<boolean> { export function openingLinkTakesDirective(nodes: readonly AdfNode[], path: ConvertErrorPath): Result<boolean> {
const emitted = emitLine(nodes, 'paragraph', path) const emitted = emitLine(nodes, 'paragraph', path, 'lossless')
if (!emitted.ok) return emitted if (!emitted.ok) return emitted
return success(emitted.value.openingLinkAsDirective) return success(emitted.value.openingLinkAsDirective)
} }
export function plainLineFallback(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<PlainLineFallback | undefined> { export function plainLineFallback(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<PlainLineFallback | undefined> {
const emission = lineSegments(nodes, container, path, { carried: new Set(), openingLinkAsDirective: false }) const emission = lineSegments(nodes, container, path, { carried: new Set(), flavour: 'plain', openingLinkAsDirective: false })
if (!emission.ok) return emission if (!emission.ok) return emission
if (emission.value.carry !== undefined) return failure('unsupported-node-shape', 'an inline node on a plain line has no spelling but the carry', path) if (emission.value.carry !== undefined) return failure('unsupported-node-shape', 'an inline node on a plain line has no spelling but the carry', path)
const verdict = lineVerdict(emission.value.segments, container) const verdict = lineVerdict(emission.value.segments, container, 'plain')
return success(verdict.kind === 'line' ? undefined : verdict) return success(verdict.kind === 'line' ? undefined : verdict)
} }
export function tryPipeCell(nodes: readonly AdfNode[], path: ConvertErrorPath): string | undefined { export function tryPipeCell(nodes: readonly AdfNode[], path: ConvertErrorPath, flavour: Flavour): string | undefined {
const emitted = emitLine(nodes, 'table-cell', path) const emitted = emitLine(nodes, 'table-cell', path, flavour)
if (!emitted.ok) return undefined if (!emitted.ok) return undefined
if (emitted.value.segments.some((segment) => isSyntax(segment.escaping) && segment.text.includes('|'))) return undefined if (emitted.value.segments.some((segment) => isSyntax(segment.escaping) && segment.text.includes('|'))) return undefined
return emitted.value.line return emitted.value.line
@@ -69,12 +72,12 @@ export function tryImageLine(alt: string | undefined, href: string, path: Conver
const destination = spellDestination(href) const destination = spellDestination(href)
if (destination === undefined) return undefined if (destination === undefined) return undefined
const description: InlineSegment[] = alt === undefined ? [] : [{ escaping: 'bracketed', text: alt }] const description: InlineSegment[] = alt === undefined ? [] : [{ escaping: 'bracketed', text: alt }]
const attempt = attemptLine([syntax('!['), ...description, syntax(`](${destination})`)], 'paragraph', path) const attempt = attemptLine([syntax('!['), ...description, syntax(`](${destination})`)], 'paragraph', path, 'lossless')
return attempt.ok ? attempt.value.line : undefined return attempt.ok ? attempt.value.line : undefined
} }
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<EmittedLine> { function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result<EmittedLine> {
const fallbacks: LineFallbacks = { carried: new Set(), openingLinkAsDirective: false } const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false }
// Terminates because takeFallback refuses a pass that took no new fallback. // Terminates because takeFallback refuses a pass that took no new fallback.
for (;;) { for (;;) {
const emission = lineSegments(nodes, container, path, fallbacks) const emission = lineSegments(nodes, container, path, fallbacks)
@@ -84,7 +87,7 @@ function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: Con
if (!taken.ok) return taken if (!taken.ok) return taken
continue continue
} }
const attempt = attemptLine(emission.value.segments, container, path) const attempt = attemptLine(emission.value.segments, container, path, flavour)
if (!attempt.ok) return attempt if (!attempt.ok) return attempt
if (attempt.value.line !== undefined) { if (attempt.value.line !== undefined) {
return success({ line: attempt.value.line, openingLinkAsDirective: fallbacks.openingLinkAsDirective, segments: emission.value.segments }) return success({ line: attempt.value.line, openingLinkAsDirective: fallbacks.openingLinkAsDirective, segments: emission.value.segments })
@@ -114,19 +117,20 @@ function lineSegments(nodes: readonly AdfNode[], container: LineContainer, path:
return success({ segments: carryStrippedWhitespace(emission.value.segments) }) return success({ segments: carryStrippedWhitespace(emission.value.segments) })
} }
function attemptLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath): Result<LineAttempt> { function attemptLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result<LineAttempt> {
const verdict = lineVerdict(segments, container) const verdict = lineVerdict(segments, container, flavour)
if (verdict.kind === 'opening-link') return success({ fallback: 'opening-link' }) if (verdict.kind === 'opening-link') return success({ fallback: 'opening-link' })
if (verdict.kind === 'unspellable-run') return success({ fallback: verdict.run }) if (verdict.kind === 'unspellable-run') return success({ fallback: verdict.runs[0] })
if (verdict.kind === 'claimed-line') return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(verdict.text)}`, path) if (verdict.kind === 'claimed-line') return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(verdict.text)}`, path)
return success({ line: verdict.text }) return success({ line: verdict.text })
} }
// The fallbacks in the order a line takes them, or the line where it takes none. // The fallbacks in the order a line takes them, or the line where it takes none.
function lineVerdict(segments: readonly InlineSegment[], container: LineContainer): PlainLineFallback | { kind: 'line'; text: string } { function lineVerdict(segments: readonly InlineSegment[], container: LineContainer, flavour: Flavour): PlainLineFallback | { kind: 'line'; text: string } {
const assembled = assembleInlineLine(segments, container) const assembled = assembleInlineLine(segments, container, flavour)
if (assembled.openingLinkAsDirective) return { kind: 'opening-link' } if (assembled.openingLinkAsDirective) return { kind: 'opening-link' }
if (assembled.unspellableRun !== undefined) return { kind: 'unspellable-run', run: assembled.unspellableRun } const [run, ...others] = assembled.unspellableRuns
if (run !== undefined) return { kind: 'unspellable-run', runs: [run, ...others] }
const lines = assembled.line.split('\n') const lines = assembled.line.split('\n')
const claimed = container === 'paragraph' ? lines.findIndex((single, index) => claimsLine(single, index === 0 ? 'first' : 'later')) : -1 const claimed = container === 'paragraph' ? lines.findIndex((single, index) => claimsLine(single, index === 0 ? 'first' : 'later')) : -1
return claimed === -1 ? { kind: 'line', text: assembled.line } : { kind: 'claimed-line', line: claimed, text: lines[claimed] ?? '' } return claimed === -1 ? { kind: 'line', text: assembled.line } : { kind: 'claimed-line', line: claimed, text: lines[claimed] ?? '' }
@@ -270,6 +274,7 @@ function emitText(node: AdfNode, context: InlineContext, index: number, path: Co
function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result<Emission> { function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result<Emission> {
const path = nodePath(context, index) const path = nodePath(context, index)
const range: NodeRange = { first: index, last: index + nodes.length - 1 } const range: NodeRange = { first: index, last: index + nodes.length - 1 }
if (mark.type === 'backgroundColor' && context.flavour === 'plain') return emitHighlight(nodes, depth, range, context)
const spelling = markSpelling(mark.type) const spelling = markSpelling(mark.type)
if (spelling === undefined) return success({ carry: range }) if (spelling === undefined) return success({ carry: range })
const attributes = spellMarkAttributes(mark, spelling.attributes) const attributes = spellMarkAttributes(mark, spelling.attributes)
@@ -298,6 +303,15 @@ function emitEmphasis(nodes: readonly AdfNode[], spelling: string, depth: number
}) })
} }
function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRange, context: InlineContext): Result<Emission> {
const inner = emitRun(nodes, depth + 1, range.first, context)
if (!inner.ok || inner.value.carry !== undefined) return inner
const run = { ...range, depth }
return success({
segments: [{ escaping: 'none', highlight: 'open', nodes: run, text: highlightDelimiter }, ...inner.value.segments, { escaping: 'none', highlight: 'close', nodes: run, text: highlightDelimiter }],
})
}
function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result<Emission> { function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result<Emission> {
let text = '' let text = ''
for (const node of nodes) { for (const node of nodes) {
+47 -16
View File
@@ -1,13 +1,15 @@
import type { Flavour } from '../plain-conventions.ts'
import type { LineContainer } from '../line-container.ts' import type { LineContainer } from '../line-container.ts'
import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark/grammar.ts' import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark/grammar.ts'
import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts'
import { claimsDirectivePrefix } from '../directive-syntax.ts' import { claimsDirectivePrefix } from '../directive-syntax.ts'
import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts'
import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts'
import { isBareDelimiterRow } from '../pipe-table-syntax.ts' import { isBareDelimiterRow } from '../pipe-table-syntax.ts'
import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts' import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts'
import { readEntityReference } from '../commonmark/entity-references.ts' import { readEntityReference } from '../commonmark/entity-references.ts'
export type EmphasisRole = 'close' | 'open' export type DelimiterRole = 'close' | 'open'
export type InlineEscaping = 'backslash' | 'bracketed' | 'bracketed-link-target' | 'none' export type InlineEscaping = 'backslash' | 'bracketed' | 'bracketed-link-target' | 'none'
@@ -16,11 +18,12 @@ export type NodeRange = { first: number; last: number }
export type MarkRun = NodeRange & { depth: number } export type MarkRun = NodeRange & { depth: number }
export type InlineSegment = export type InlineSegment =
| { emphasis: EmphasisRole; escaping: 'none'; nodes: MarkRun; text: string } | { emphasis: DelimiterRole; escaping: 'none'; highlight?: undefined; nodes: MarkRun; text: string }
| { emphasis?: undefined; escaping: 'none'; nodes: NodeRange; text: string } | { emphasis?: undefined; escaping: 'none'; highlight: DelimiterRole; nodes: MarkRun; text: string }
| { emphasis?: undefined; escaping: InlineEscaping; nodes?: undefined; text: string } | { emphasis?: undefined; escaping: 'none'; highlight?: undefined; nodes: NodeRange; text: string }
| { emphasis?: undefined; escaping: InlineEscaping; highlight?: undefined; nodes?: undefined; text: string }
export type AssembledLine = { line: string; openingLinkAsDirective?: true; unspellableRun: MarkRun | undefined } export type AssembledLine = { line: string; openingLinkAsDirective?: true; unspellableRuns: MarkRun[] }
type ScanLine = { position: LinePosition; start: number; text: string } type ScanLine = { position: LinePosition; start: number; text: string }
@@ -32,8 +35,8 @@ const delimiters = ['*', '_', '`', '~']
const followsLinkText = /[([]/ const followsLinkText = /[([]/
export function assembleInlineLine(segments: readonly InlineSegment[], container: LineContainer): AssembledLine { export function assembleInlineLine(segments: readonly InlineSegment[], container: LineContainer, flavour: Flavour): AssembledLine {
return escape(resolveEmphasis(segments), container) return escape(resolveEmphasis(segments), container, flavour === 'plain')
} }
function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] { function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] {
@@ -65,11 +68,11 @@ function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] {
return resolved return resolved
} }
function escape(segments: readonly InlineSegment[], container: LineContainer): AssembledLine { function escape(segments: readonly InlineSegment[], container: LineContainer, highlights: boolean): AssembledLine {
const scan = segments.map((segment) => segment.text).join('') const scan = segments.map((segment) => segment.text).join('')
const escapings: InlineEscaping[] = [] const escapings: InlineEscaping[] = []
for (const segment of segments) for (let index = 0; index < segment.text.length; index += 1) escapings.push(segment.escaping) for (const segment of segments) for (let index = 0; index < segment.text.length; index += 1) escapings.push(segment.escaping)
const escaped = escapeClosedRuns(scan, escapings, escapeClaims(scan, escapings, container)) const escaped = escapeClosedRuns(scan, escapings, escapeClaims(scan, escapings, container, highlights))
const placements: number[] = [] const placements: number[] = []
let output = '' let output = ''
for (let index = 0; index < scan.length; index += 1) { for (let index = 0; index < scan.length; index += 1) {
@@ -78,34 +81,45 @@ function escape(segments: readonly InlineSegment[], container: LineContainer): A
output += scan.charAt(index) output += scan.charAt(index)
} }
if (container === 'paragraph' && opensLinkDefinition(output)) { if (container === 'paragraph' && opensLinkDefinition(output)) {
if (segments[0]?.nodes !== undefined) return { line: output, openingLinkAsDirective: true, unspellableRun: undefined } if (segments[0]?.nodes !== undefined) return { line: output, openingLinkAsDirective: true, unspellableRuns: [] }
return { line: `\\${output}`, unspellableRun: unspellableRun(segments, output, placements) } return { line: `\\${output}`, unspellableRuns: unspellableRuns(segments, output, placements) }
} }
return { line: output, unspellableRun: unspellableRun(segments, output, placements) } return { line: output, unspellableRuns: unspellableRuns(segments, output, placements) }
} }
function escapeClaims(scan: string, escapings: readonly InlineEscaping[], container: LineContainer): ReadonlySet<number> { function escapeClaims(scan: string, escapings: readonly InlineEscaping[], container: LineContainer, highlights: boolean): ReadonlySet<number> {
const escaped = new Set<number>() const escaped = new Set<number>()
const linkClose = lastLinkClose(scan, escapings) const linkClose = lastLinkClose(scan, escapings)
let line = scanLine(scan, 0) let line = scanLine(scan, 0)
let afterEscape = false let afterEscape = false
// Whether the `=` before opens a `==` the reader takes whole, so this one starts nothing.
let pairsEquals = false
for (let index = 0; index < scan.length; index += 1) { for (let index = 0; index < scan.length; index += 1) {
if (index > line.start + line.text.length) line = scanLine(scan, line.start + line.text.length + 1) if (index > line.start + line.text.length) line = scanLine(scan, line.start + line.text.length + 1)
const escaping = escapings[index] const escaping = escapings[index]
const escapable = escaping === 'backslash' || escaping === 'bracketed' const escapable = escaping === 'backslash' || escaping === 'bracketed'
const opensEquals: boolean = highlights && !pairsEquals && scan.startsWith(highlightDelimiter, index)
const claimed: boolean = const claimed: boolean =
(escapable && (escapable &&
(claimsLineStart(line, index, container) || ((opensEquals && claimsHighlight(scan, index)) ||
claimsLineStart(line, index, container) ||
mergesWithSyntax(scan, escapings, index) || mergesWithSyntax(scan, escapings, index) ||
opensConstruct(scan, linkClose, index, escaping === 'bracketed', container, afterEscape))) || opensConstruct(scan, linkClose, index, escaping === 'bracketed', container, afterEscape))) ||
(escaping === 'bracketed-link-target' && (escaping === 'bracketed-link-target' &&
((scan.charAt(index) === '`' && opensCodeSpan(scan, index, afterEscape)) || claimsDirectivePrefix(scan, index))) ((scan.charAt(index) === '`' && opensCodeSpan(scan, index, afterEscape)) || claimsDirectivePrefix(scan, index)))
if (claimed) escaped.add(index) if (claimed) escaped.add(index)
afterEscape = claimed afterEscape = claimed
pairsEquals = opensEquals && !claimed
} }
return escaped return escaped
} }
// Like an emphasis run, a `==` in text escapes where the reader can open or close with it.
function claimsHighlight(scan: string, index: number): boolean {
const flanking = highlightFlanking(scan, index)
return flanking.opens || flanking.closes
}
// CommonMark reads no escape inside a code span, so a backtick string an escape forms or splits off still closes one an earlier bare run opens. // CommonMark reads no escape inside a code span, so a backtick string an escape forms or splits off still closes one an earlier bare run opens.
function escapeClosedRuns(scan: string, escapings: readonly InlineEscaping[], claimed: ReadonlySet<number>): ReadonlySet<number> { function escapeClosedRuns(scan: string, escapings: readonly InlineEscaping[], claimed: ReadonlySet<number>): ReadonlySet<number> {
const escaped = new Set(claimed) const escaped = new Set(claimed)
@@ -134,10 +148,27 @@ function escapeClosedRuns(scan: string, escapings: readonly InlineEscaping[], cl
return escaped return escaped
} }
function unspellableRun(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun | undefined { // One emphasis run, the innermost, or every highlight run the line cannot spell.
function unspellableRuns(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun[] {
const { nodes, runs } = emittedRuns(segments, placements, output) const { nodes, runs } = emittedRuns(segments, placements, output)
const pair = misflanked(runs) ?? unpaired(runs) const pair = misflanked(runs) ?? unpaired(runs)
return pair === undefined ? undefined : nodes[pair] const run = pair === undefined ? undefined : nodes[pair]
return run === undefined ? unreadHighlights(segments, output, placements) : [run]
}
// No `==` in text can open or close, and highlights never nest, so a pair reads back where each delimiter flanks.
function unreadHighlights(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun[] {
const unread: MarkRun[] = []
let cursor = 0
for (const segment of segments) {
const start = placements[cursor] ?? 0
cursor += segment.text.length
if (segment.highlight === undefined) continue
const flanking = highlightFlanking(output, start)
const flanks = segment.highlight === 'open' ? flanking.opens : flanking.closes
if (!flanks && unread.at(-1) !== segment.nodes) unread.push(segment.nodes)
}
return unread
} }
function misflanked(runs: readonly EmittedRun[]): number | undefined { function misflanked(runs: readonly EmittedRun[]): number | undefined {
+3 -2
View File
@@ -1,10 +1,11 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfNode } from '../../adf/document.ts'
import type { ConvertErrorPath } from '../../result.ts' import type { ConvertErrorPath } from '../../result.ts'
import type { Flavour } from '../plain-conventions.ts'
import { carriesOnly, nodeContent } from '../../adf/document.ts' import { carriesOnly, nodeContent } from '../../adf/document.ts'
import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts' import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts'
import { tryPipeCell } from './inline-line.ts' import { tryPipeCell } from './inline-line.ts'
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath): string | undefined { export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): string | undefined {
const rows = pipeRows(node) const rows = pipeRows(node)
if (rows === undefined) return undefined if (rows === undefined) return undefined
const lines: string[] = [] const lines: string[] = []
@@ -12,7 +13,7 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath): string | un
const cells: string[] = [] const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) { for (const [cellIndex, paragraph] of row.entries()) {
const content = nodeContent(paragraph) const content = nodeContent(paragraph)
const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0]) const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0], flavour)
if (line === undefined) return undefined if (line === undefined) return undefined
cells.push(line) cells.push(line)
} }
+20 -11
View File
@@ -1,8 +1,8 @@
import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts'
import type { LineContainer } from '../line-container.ts' import type { LineContainer } from '../line-container.ts'
import type { MarkRun } from './line-escaping.ts'
import { blockNodeModel } from '../../adf/block-nodes.ts' import { blockNodeModel } from '../../adf/block-nodes.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { highlightDelimiter } from '../plain-conventions.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { mergeAdjacentText, sameMark } from '../../adf/editor-normal.ts' import { mergeAdjacentText, sameMark } from '../../adf/editor-normal.ts'
import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
@@ -10,6 +10,7 @@ import { plainLineFallback, type PlainLineFallback } from './inline-line.ts'
import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
const highlight = 'backgroundColor' const highlight = 'backgroundColor'
const highlightMark: AdfMark = { type: highlight }
const edgeStrippingMarks: readonly string[] = [highlight, 'em', 'strike', 'strong'] const edgeStrippingMarks: readonly string[] = [highlight, 'em', 'strike', 'strong']
const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link'] const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link']
@@ -130,7 +131,7 @@ function textLeaf(text: string, marks: readonly AdfMark[]): AdfNode {
return marks.length === 0 ? { text, type: 'text' } : { marks: [...marks], text, type: 'text' } return marks.length === 0 ? { text, type: 'text' } : { marks: [...marks], text, type: 'text' }
} }
// A highlight goes outermost so its run is one run at depth 0, and code innermost, the only place its spelling holds. // A highlight goes first, where `highlighted` looks for it, and code last, the only place its spelling holds.
function plainMarks(marks: readonly AdfMark[], container: LineContainer, text: string): AdfMark[] { function plainMarks(marks: readonly AdfMark[], container: LineContainer, text: string): AdfMark[] {
const kept: AdfMark[] = [] const kept: AdfMark[] = []
for (const mark of marks) { for (const mark of marks) {
@@ -140,7 +141,9 @@ function plainMarks(marks: readonly AdfMark[], container: LineContainer, text: s
if (plain !== undefined) kept.push(plain) if (plain !== undefined) kept.push(plain)
} }
const rank = (mark: AdfMark): number => (mark.type === highlight ? 0 : mark.type === 'code' ? 2 : 1) const rank = (mark: AdfMark): number => (mark.type === highlight ? 0 : mark.type === 'code' ? 2 : 1)
return kept.sort((first, second) => rank(first) - rank(second)) // The reader highlights no code, as Atlassian's schema allows none.
const code = kept.some((mark) => mark.type === 'code')
return kept.filter((mark) => !code || mark.type !== highlight).sort((first, second) => rank(first) - rank(second))
} }
function plainLink(mark: AdfMark, container: LineContainer): AdfMark | undefined { function plainLink(mark: AdfMark, container: LineContainer): AdfMark | undefined {
@@ -163,7 +166,7 @@ export function writableHref(href: string): string {
return written return written
} }
// The delimiters carry the marks the whole run shares, so they open and close inside them. // The marks a whole highlight run shares go outside the highlight, so its delimiters open and close inside them.
function highlighted(leaves: readonly AdfNode[]): AdfNode[] { function highlighted(leaves: readonly AdfNode[]): AdfNode[] {
const spelled: AdfNode[] = [] const spelled: AdfNode[] = []
let run: AdfNode[] = [] let run: AdfNode[] = []
@@ -172,11 +175,11 @@ function highlighted(leaves: readonly AdfNode[]): AdfNode[] {
const marks = nodeMarks(leaf) const marks = nodeMarks(leaf)
if (marks[0]?.type === highlight) { if (marks[0]?.type === highlight) {
const held = marks.slice(1) const held = marks.slice(1)
shared = run.length === 0 ? held.filter((mark) => mark.type !== 'code') : shared.filter((mark) => held.some((other) => sameMark(other, mark))) shared = run.length === 0 ? held : shared.filter((mark) => held.some((other) => sameMark(other, mark)))
run.push(withMarks(leaf, held)) run.push(leaf)
continue continue
} }
if (run.length > 0) for (const held of [textLeaf(highlightDelimiter, shared), ...run, textLeaf(highlightDelimiter, shared)]) spelled.push(held) for (const held of run) spelled.push(withMarks(held, [...shared, highlightMark, ...nodeMarks(held).slice(1).filter((mark) => !shared.some((other) => sameMark(other, mark)))]))
run = [] run = []
spelled.push(leaf) spelled.push(leaf)
} }
@@ -263,7 +266,7 @@ function spellableLine(leaves: AdfNode[], container: LineContainer, path: Conver
} }
function withoutFallback(leaves: readonly AdfNode[], fallback: PlainLineFallback): AdfNode[] | undefined { function withoutFallback(leaves: readonly AdfNode[], fallback: PlainLineFallback): AdfNode[] | undefined {
if (fallback.kind === 'unspellable-run') return withoutMark(leaves, fallback.run.first, fallback.run.last, fallback.run.depth) if (fallback.kind === 'unspellable-run') return withoutMarks(leaves, fallback.runs)
const first = fallback.kind === 'opening-link' ? 0 : lineStart(leaves, fallback.line) const first = fallback.kind === 'opening-link' ? 0 : lineStart(leaves, fallback.line)
const mark = nodeMarks(leaves[first] ?? {})[0] const mark = nodeMarks(leaves[first] ?? {})[0]
if (mark === undefined || mark.type !== (fallback.kind === 'opening-link' ? 'link' : 'code')) return undefined if (mark === undefined || mark.type !== (fallback.kind === 'opening-link' ? 'link' : 'code')) return undefined
@@ -272,7 +275,7 @@ function withoutFallback(leaves: readonly AdfNode[], fallback: PlainLineFallback
// A code span is what binds the `]` a link definition reads, and dropping it keeps the link target. // A code span is what binds the `]` a link definition reads, and dropping it keeps the link target.
const spans = leaves.slice(first, last + 1).some((leaf) => nodeMarks(leaf).length > 1 && nodeMarks(leaf).at(-1)?.type === 'code') const spans = leaves.slice(first, last + 1).some((leaf) => nodeMarks(leaf).length > 1 && nodeMarks(leaf).at(-1)?.type === 'code')
if (mark.type === 'link' && spans) return leaves.map((leaf, index) => (index < first || index > last ? leaf : withMarks(leaf, nodeMarks(leaf).filter((held) => held.type !== 'code')))) if (mark.type === 'link' && spans) return leaves.map((leaf, index) => (index < first || index > last ? leaf : withMarks(leaf, nodeMarks(leaf).filter((held) => held.type !== 'code'))))
return withoutMark(leaves, first, last, 0) return withoutMarks(leaves, [{ depth: 0, first, last }])
} }
function lineStart(leaves: readonly AdfNode[], line: number): number { function lineStart(leaves: readonly AdfNode[], line: number): number {
@@ -281,6 +284,12 @@ function lineStart(leaves: readonly AdfNode[], line: number): number {
return index return index
} }
function withoutMark(leaves: readonly AdfNode[], first: number, last: number, depth: number): AdfNode[] { // The runs cover disjoint leaves, so one pass drops them all.
return leaves.map((leaf, index) => (index < first || index > last ? leaf : withMarks(leaf, nodeMarks(leaf).filter((_, held) => held !== depth)))) function withoutMarks(leaves: readonly AdfNode[], runs: readonly MarkRun[]): AdfNode[] {
const depths = new Map<number, number>()
for (const run of runs) for (let index = run.first; index <= run.last; index += 1) depths.set(index, run.depth)
return leaves.map((leaf, index) => {
const depth = depths.get(index)
return depth === undefined ? leaf : withMarks(leaf, nodeMarks(leaf).filter((_, held) => held !== depth))
})
} }
+32 -13
View File
@@ -2,9 +2,8 @@ import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts'
import { adfToMarkdown } from './adf-to-markdown.ts' import { adfToPlainMarkdown, reduceToPlain } from './plain-reduction.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { reduceToPlain } from './plain-reduction.ts'
const code: AdfMark = { type: 'code' } const code: AdfMark = { type: 'code' }
const em: AdfMark = { type: 'em' } const em: AdfMark = { type: 'em' }
@@ -19,11 +18,8 @@ function plain(...content: AdfNode[]): string {
} }
function plainDocument(input: AdfDocument): string { function plainDocument(input: AdfDocument): string {
const reduced = reduceToPlain(input) const markdown = adfToPlainMarkdown(input)
if (!reduced.ok) return `${reduced.error.code} at /${reduced.error.path.join('/')}` return markdown.ok ? markdown.value : `${markdown.error.code} at /${markdown.error.path.join('/')}`
const markdown = adfToMarkdown(reduced.value)
if (!markdown.ok) return `emit ${markdown.error.code}: ${markdown.error.message}`
return markdown.value
} }
function text(value: string, ...marks: AdfMark[]): AdfNode { function text(value: string, ...marks: AdfMark[]): AdfNode {
@@ -119,17 +115,25 @@ test('spells an expand and a nested expand as a folded callout titled by the mar
) )
assert.equal(plain(node('expand', {}, said('Line.'))), '> [!NOTE]-\n>\n> Line.\n') assert.equal(plain(node('expand', {}, said('Line.'))), '> [!NOTE]-\n>\n> Line.\n')
assert.equal(plain(node('expand', { title: ' *Two*\nlines ' })), '> [!NOTE]- \\*Two\\* lines\n') assert.equal(plain(node('expand', { title: ' *Two*\nlines ' })), '> [!NOTE]- \\*Two\\* lines\n')
assert.equal(plain(node('expand', { title: '\ta \t b\t ' })), '> [!NOTE]- a \t b\n')
assert.equal(plain(node('expand', { title: '**x** [y](z) ==w==' }, said('b'))), '> [!NOTE]- \\*\\*x\\*\\* \\[y](z) ==w==\n>\n> b\n')
}) })
test('spells a task list as a bullet list whose items lead with their state', () => { test('spells a task list as a bullet list whose items lead with their state', () => {
const task = (state: string, value: string): AdfNode => node('taskItem', { localId: '01a0d99b-1f58-7b95-829b-6f9860371d54', state }, text(value)) const task = (state: string, value: string): AdfNode => node('taskItem', { localId: '01a0d99b-1f58-7b95-829b-6f9860371d54', state }, text(value))
const nested = node('taskList', {}, task('TODO', 'Review')) const nested = node('taskList', {}, task('TODO', 'Review'))
assert.equal(plain(node('taskList', {}, task('DONE', 'Write the spec'), nested, task('TODO', 'Ship it'))), '- [x] Write the spec\n - [ ] Review\n- [ ] Ship it\n') assert.equal(plain(node('taskList', {}, task('DONE', 'Write the spec'), nested, task('TODO', 'Ship it'))), '- [x] Write the spec\n - [ ] Review\n- [ ] Ship it\n')
assert.equal(plain(node('taskList', {}, nested, task('DONE', ''), said('Stray'))), '- - [ ] Review\n- [x]\n- Stray\n') assert.equal(plain(node('taskList', {}, nested, task('DONE', ''), said('Stray'))), '- - [ ] Review\n- \\[x]\n- Stray\n')
assert.equal(plain(node('taskList', {}, node('taskList', {}), task('DONE', 'a'), said('b'), node('taskList', {}, task('TODO', 'c')))), '- \\[x] a\n- b\n - [ ] c\n')
assert.equal(plain(node('taskList', {}, paragraph(), task('DONE', 'a'))), '- [x] a\n')
assert.equal(plain(bulletList(item(said('x'))), node('taskList', {}, task('DONE', 'a'), node('taskList', {}, task('TODO', 'c')))), '- x\n- \\[x] a\n - [ ] c\n')
assert.equal(plain(node('taskList', {}), task('TODO', 'Loose')), 'Loose\n') assert.equal(plain(node('taskList', {}), task('TODO', 'Loose')), 'Loose\n')
const blockTask = node('blockTaskItem', { state: 'DONE' }, said('First.'), said('Second.')) const blockTask = node('blockTaskItem', { state: 'DONE' }, said('First.'), said('Second.'))
const codeTask = node('blockTaskItem', { state: 'TODO' }, { content: [text('x')], type: 'codeBlock' }) const codeTask = node('blockTaskItem', { state: 'TODO' }, { content: [text('x')], type: 'codeBlock' })
assert.equal(plain(node('taskList', {}, blockTask, codeTask)), '- [x] First.\n\n Second.\n- [ ]\n\n ```\n x\n ```\n') assert.equal(plain(node('taskList', {}, blockTask, codeTask)), '- [x] First.\n\n Second.\n- [ ]\n\n ```\n x\n ```\n')
const listTask = node('blockTaskItem', { state: 'DONE' }, bulletList(item(said('a'))))
assert.equal(plain(node('taskList', {}, task('TODO', ''), listTask, node('taskList', {}, task('TODO', 'b')))), '- [ ]\n- [x]\n - a\n - \\[ ] b\n')
assert.equal(plain(node('taskList', {}, task('DONE', 'a'), node('taskList', {}, task('TODO', '')))), '- [x] a\n - [ ]\n')
}) })
test('spells a decision list as a plain bullet list', () => { test('spells a decision list as a plain bullet list', () => {
@@ -141,7 +145,19 @@ test('spells a highlight as a == pair around the run, whatever its colour', () =
assert.equal(plain(paragraph(text('a '), text('hi', highlight('#fff')), text(' there', highlight('#000')), text(' b'))), 'a ==hi there== b\n') assert.equal(plain(paragraph(text('a '), text('hi', highlight('#fff')), text(' there', highlight('#000')), text(' b'))), 'a ==hi there== b\n')
assert.equal(plain(paragraph(text('hi ', strong, highlight('#fff')), text('b'))), '**==hi==** b\n') assert.equal(plain(paragraph(text('hi ', strong, highlight('#fff')), text('b'))), '**==hi==** b\n')
assert.equal(plain(paragraph(text('a', strong, highlight('#fff')), text('b', highlight('#fff'), em))), '==**a**_b_==\n') assert.equal(plain(paragraph(text('a', strong, highlight('#fff')), text('b', highlight('#fff'), em))), '==**a**_b_==\n')
assert.equal(plain(paragraph(text('a', highlight('#fff'), code))), '==`a`==\n') assert.equal(plain(paragraph(text('a', highlight('#fff'), code), text('b', highlight('#fff')))), '`a`==b==\n')
assert.equal(plain(paragraph(text('=', highlight('#fff')), text(' '), text('a==b', highlight('#fff')))), '==\\=== ==a==b==\n')
assert.equal(plain(paragraph(text('x'), text('y', highlight('#fff')), text(' z'))), 'xy z\n')
})
test('escapes text a renderer would take as a flavour marker, and only there', () => {
assert.equal(plain(said('==x== a == b a==b ===')), '\\==x\\== a == b a==b \\=\\==\n')
assert.equal(plain({ content: [said('[!NOTE] x'), said('[!TIP]')], type: 'blockquote' }), '> \\[!NOTE] x\n>\n> [!TIP]\n')
assert.equal(plain({ content: [said('[!NOTE]x')], type: 'blockquote' }, said('[!NOTE]')), '> [!NOTE]x\n\n[!NOTE]\n')
assert.equal(plain(bulletList(item(said('[x] a')), item(said('[ ]')))), '- \\[x] a\n- \\[ ]\n')
assert.equal(plain(bulletList(item(said('[x] a')), item(said('b'))), node('orderedList', { order: 1 }, item(said('[x] c')))), '- \\[x] a\n- b\n\n1. \\[x] c\n')
const task = (state: string, value: string): AdfNode => node('taskItem', { state }, text(value))
assert.equal(plain(node('taskList', {}, task('DONE', '[x] a'), task('TODO', '==b=='))), '- [x] [x] a\n- [ ] \\==b\\==\n')
}) })
test('unwraps the containers plain markdown has no spelling for to their body blocks in order', () => { test('unwraps the containers plain markdown has no spelling for to their body blocks in order', () => {
@@ -168,7 +184,7 @@ test('keeps the CommonMark blocks in their spelling and drops their attributes a
assert.equal(plain(node('orderedList', { order: 1e10 }, item(said('Alpha')), item(code), item())), '- 10000000000. Alpha\n- 10000000001.\n\n ```\n x\n ```\n- 10000000002.\n') assert.equal(plain(node('orderedList', { order: 1e10 }, item(said('Alpha')), item(code), item())), '- 10000000000. Alpha\n- 10000000001.\n\n ```\n x\n ```\n- 10000000002.\n')
const long = node('orderedList', { order: 1e10 }, item(said('y'))) const long = node('orderedList', { order: 1e10 }, item(said('y')))
assert.equal(plain(bulletList(item(said('x'))), long, bulletList(item(said('z')))), '- x\n- 10000000000. y\n- z\n') assert.equal(plain(bulletList(item(said('x'))), long, bulletList(item(said('z')))), '- x\n- 10000000000. y\n- z\n')
assert.equal(plain(node('taskList', {}, node('taskItem', { state: 'DONE' }, text('t'))), long), '- [x] t\n- 10000000000. y\n') assert.equal(plain(node('taskList', {}, node('taskItem', { state: 'DONE' }, text('t'))), long), '- \\[x] t\n- 10000000000. y\n')
assert.equal(plain(bulletList(item(said('a'), bulletList(item(said('x'))), long))), '- a\n - x\n - 10000000000. y\n') assert.equal(plain(bulletList(item(said('a'), bulletList(item(said('x'))), long))), '- a\n - x\n - 10000000000. y\n')
const givesWay = node('orderedList', { order: 1e10 }, item(node('rule', {})), item(bulletList(item(bulletList(item()))))) const givesWay = node('orderedList', { order: 1e10 }, item(node('rule', {})), item(bulletList(item(bulletList(item())))))
assert.equal(plain(givesWay, bulletList(item(said('z')))), '- 10000000000.\n- 10000000001.\n - -\n- z\n') assert.equal(plain(givesWay, bulletList(item(said('z')))), '- 10000000000.\n- 10000000001.\n - -\n- z\n')
@@ -280,6 +296,8 @@ test('drops the mark of a run CommonMark flanking or matching cannot spell', ()
assert.equal(plain(paragraph(text('un'), text('-real', strong), text('istic'))), 'un-realistic\n') assert.equal(plain(paragraph(text('un'), text('-real', strong), text('istic'))), 'un-realistic\n')
assert.equal(plain(paragraph(text('a', em), text('b', strong), text('c', em))), '_a_**b**_c_\n') assert.equal(plain(paragraph(text('a', em), text('b', strong), text('c', em))), '_a_**b**_c_\n')
assert.equal(plain(paragraph(text('x'), text('*', em), text('y'))), 'x\\*y\n') assert.equal(plain(paragraph(text('x'), text('*', em), text('y'))), 'x\\*y\n')
const highlight: AdfMark = { type: 'backgroundColor' }
assert.equal(plain(paragraph(text('a'), text('b', highlight), text(' c'), text('d', highlight), text(' '), text('e', highlight))), 'ab cd ==e==\n')
}) })
test('breaks a line at a newline and trims whitespace at every edge CommonMark strips', () => { test('breaks a line at a newline and trims whitespace at every edge CommonMark strips', () => {
@@ -311,7 +329,8 @@ test('drops an empty paragraph and merges adjacent lists of one type', () => {
assert.equal(plain(node('layoutSection', {}, column(bulletList(item(said('a')))), column(bulletList(item(said('b')))))), '- a\n- b\n') assert.equal(plain(node('layoutSection', {}, column(bulletList(item(said('a')))), column(bulletList(item(said('b')))))), '- a\n- b\n')
}) })
test('returns a document the lossless emitter spells without a directive', () => { test('keeps the nodes the plain flavour spells and degrades only what it cannot', () => {
const reduced = reduceToPlain(document(node('panel', { panelType: 'info' }, said('x')))) const tasks = node('taskList', {}, node('taskItem', { localId: '01a0d99b-1f58-7b95-829b-6f9860371d54', state: 'DONE' }, text('t')))
assert.deepEqual(reduced.ok ? reduced.value : undefined, document({ content: [said('[!NOTE]'), said('x')], type: 'blockquote' })) const reduced = reduceToPlain(document(node('panel', { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType: 'info' }, said('x')), tasks))
assert.deepEqual(reduced.ok ? reduced.value : undefined, document(node('panel', { panelType: 'info' }, said('x')), { content: [node('taskItem', { state: 'DONE' }, text('t'))], type: 'taskList' }))
}) })
+96 -33
View File
@@ -1,13 +1,13 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import { adfDocumentFault, nodeAttrs, nodeContent } from '../../adf/document.ts' import { adfDocumentFault, nodeAttrs, nodeContent } from '../../adf/document.ts'
import { adfToMarkdown, commonMarkSpelling, largestListMarker, type SpellingMemo } from './adf-to-markdown.ts' import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from './adf-to-markdown.ts'
import { alertMarker, foldedAlertMarker, taskMarker } from '../plain-conventions.ts'
import { blockNodeModel } from '../../adf/block-nodes.ts' import { blockNodeModel } from '../../adf/block-nodes.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { inlineLeaves, isBlockNodeType, oneLine, reduceInline, writableHref } from './plain-inline.ts' import { inlineLeaves, isBlockNodeType, oneLine, reduceInline, writableHref } from './plain-inline.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { languageSlot } from '../code-language.ts' import { languageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { taskMarker } from '../plain-conventions.ts'
// depth: the level the node reduced stands at, counted as the emitter counts it. // depth: the level the node reduced stands at, counted as the emitter counts it.
type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath } type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath }
@@ -20,7 +20,7 @@ type Placed = { index: number; loose: AdfNode[] } | { index: number; loose?: und
const blockReducers: Readonly<Record<string, BlockReducer>> = { const blockReducers: Readonly<Record<string, BlockReducer>> = {
blockCard: paragraphOfNode, blockCard: paragraphOfNode,
blockquote: (node, reduction) => quoted(success([]), node, reduction), blockquote: (node, reduction) => contained({ type: 'blockquote' }, node, reduction),
bulletList: reduceList, bulletList: reduceList,
caption: (node, reduction) => paragraphOf(nodeContent(node), reduction), caption: (node, reduction) => paragraphOf(nodeContent(node), reduction),
codeBlock: reduceCodeBlock, codeBlock: reduceCodeBlock,
@@ -33,7 +33,7 @@ const blockReducers: Readonly<Record<string, BlockReducer>> = {
mediaSingle: (node, reduction) => concatenated(nodeContent(node).map((child, index) => reduceStanding(child, childReduction(reduction, index)))), mediaSingle: (node, reduction) => concatenated(nodeContent(node).map((child, index) => reduceStanding(child, childReduction(reduction, index)))),
nestedExpand: reduceExpand, nestedExpand: reduceExpand,
orderedList: reduceList, orderedList: reduceList,
panel: (node, reduction) => quoted(success([paragraph([text(alertMarker(nodeAttrs(node)['panelType']))])]), node, reduction), panel: reducePanel,
paragraph: (node, reduction) => paragraphOf(nodeContent(node), reduction), paragraph: (node, reduction) => paragraphOf(nodeContent(node), reduction),
rule: () => success([{ type: 'rule' }]), rule: () => success([{ type: 'rule' }]),
syncBlock: paragraphOfNode, syncBlock: paragraphOfNode,
@@ -43,7 +43,7 @@ const blockReducers: Readonly<Record<string, BlockReducer>> = {
export function adfToPlainMarkdown(document: AdfDocument): Result<string> { export function adfToPlainMarkdown(document: AdfDocument): Result<string> {
const reduced = reduceToPlain(document) const reduced = reduceToPlain(document)
return reduced.ok ? adfToMarkdown(reduced.value) : reduced return reduced.ok ? writeMarkdown(reduced.value, 'plain') : reduced
} }
export function reduceToPlain(document: AdfDocument): Result<AdfDocument> { export function reduceToPlain(document: AdfDocument): Result<AdfDocument> {
@@ -112,7 +112,7 @@ function plainSequence(blocks: readonly AdfNode[], reduction: Reduction): Result
const listed = sequence[index] const listed = sequence[index]
if (listed === undefined || (listed.type !== 'bulletList' && listed.type !== 'orderedList')) continue if (listed === undefined || (listed.type !== 'bulletList' && listed.type !== 'orderedList')) continue
const block = numberedPastMarkers(listed) const block = numberedPastMarkers(listed)
const spelled = block === listed && commonMarkSpelling(block, reduction.path, reduction.depth, reduction.memo)?.ok === true const spelled = block === listed && commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'plain', memo: reduction.memo })?.ok === true
if (spelled) continue if (spelled) continue
sequence = spliced(sequence, index, block === listed ? nodeContent(block).flatMap(nodeContent) : [block]) sequence = spliced(sequence, index, block === listed ? nodeContent(block).flatMap(nodeContent) : [block])
index = Math.max(0, index - 1) - 1 index = Math.max(0, index - 1) - 1
@@ -138,11 +138,12 @@ function numberedAsText(list: AdfNode): AdfNode {
return { content: nodeContent(list).map((item, offset) => itemOf(marked(nodeContent(item), `${order + offset}.`))), type: 'bulletList' } return { content: nodeContent(list).map((item, offset) => itemOf(marked(nodeContent(item), `${order + offset}.`))), type: 'bulletList' }
} }
// Adjacent lists of one marker read back as one list.
function mergedLists(blocks: readonly AdfNode[]): AdfNode[] { function mergedLists(blocks: readonly AdfNode[]): AdfNode[] {
const merged: AdfNode[] = [] const merged: AdfNode[] = []
for (const block of blocks) { for (const block of blocks) {
let next = block let next = block
for (let previous = merged.at(-1); previous !== undefined && previous.type === next.type && isList(next); previous = merged.at(-1)) { for (let previous = merged.at(-1); previous !== undefined && listMarker(next) !== undefined && listMarker(previous) === listMarker(next); previous = merged.at(-1)) {
merged.pop() merged.pop()
next = joinedLists(previous, next) next = joinedLists(previous, next)
} }
@@ -151,17 +152,36 @@ function mergedLists(blocks: readonly AdfNode[]): AdfNode[] {
return merged return merged
} }
function isList(block: AdfNode): boolean { function listMarker(block: AdfNode): string | undefined {
return block.type === 'bulletList' || block.type === 'orderedList' if (block.type === 'orderedList') return '.'
return block.type === 'bulletList' || block.type === 'taskList' ? '-' : undefined
} }
// Two numbered lists whose numbering breaks between them keep their numbers as text in one bullet list. // Two numbered lists whose numbering breaks between them keep their numbers as text in one bullet list, and a task list joining a bullet list its markers.
function joinedLists(first: AdfNode, second: AdfNode): AdfNode { function joinedLists(first: AdfNode, second: AdfNode): AdfNode {
const breaks = first.type === 'orderedList' && nodeAttrs(second)['order'] !== Number(nodeAttrs(first)['order']) + nodeContent(first).length const breaks = first.type === 'orderedList' && nodeAttrs(second)['order'] !== Number(nodeAttrs(first)['order']) + nodeContent(first).length
const [head, tail] = breaks ? [numberedAsText(first), numberedAsText(second)] : [first, second] const [head, tail] = breaks ? [numberedAsText(first), numberedAsText(second)] : first.type === second.type ? [first, second] : [tasksAsText(first), tasksAsText(second)]
return { ...head, content: [...nodeContent(head), ...nodeContent(tail)] } return { ...head, content: [...nodeContent(head), ...nodeContent(tail)] }
} }
// A task keeps its marker as text; a list item stands as one, and anything else nests in the item before it.
function tasksAsText(list: AdfNode): AdfNode {
if (list.type !== 'taskList') return list
const items: AdfNode[] = []
for (const child of nodeContent(list)) {
const previous = isTask(child) || child.type === 'listItem' ? undefined : items.pop()
items.push(itemOf(previous === undefined ? taskAsText(child) : mergedLists([...nodeContent(previous), child])))
}
return { content: items, type: 'bulletList' }
}
function taskAsText(child: AdfNode): readonly AdfNode[] {
const marker = taskMarker(nodeAttrs(child)['state'])
if (child.type === 'taskItem') return [paragraph(nodeContent(child).length === 0 ? [text(marker)] : [text(`${marker} `), ...nodeContent(child)])]
if (child.type === 'blockTaskItem') return marked(nodeContent(child), marker)
return child.type === 'listItem' ? nodeContent(child) : [child]
}
function paragraph(content: readonly AdfNode[]): AdfNode { function paragraph(content: readonly AdfNode[]): AdfNode {
return { content: [...content], type: 'paragraph' } return { content: [...content], type: 'paragraph' }
} }
@@ -183,15 +203,27 @@ function paragraphOfNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]>
return paragraphOf([node], reduction) return paragraphOf([node], reduction)
} }
function quoted(head: Result<AdfNode[]>, node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function contained(shell: AdfNode, node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const content = concatenated([head, reduceBlocks(nodeContent(node), { ...reduction, depth: reduction.depth + 1 })]) const content = reduceBlocks(nodeContent(node), { ...reduction, depth: reduction.depth + 1 })
return content.ok ? success([{ content: content.value, type: 'blockquote' }]) : content return content.ok ? success([{ ...shell, content: content.value }]) : content
}
function reducePanel(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const panelType = nodeAttrs(node)['panelType']
return contained(typeof panelType === 'string' ? { attrs: { panelType }, type: 'panel' } : { type: 'panel' }, node, reduction)
} }
function reduceExpand(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceExpand(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const title = nodeAttrs(node)['title'] const held = nodeAttrs(node)['title']
const marker = typeof title === 'string' ? `${foldedAlertMarker} ${title.replace(/^[ \t\n\r]+/, '').replaceAll('\n', ' ')}` : foldedAlertMarker const title = typeof held === 'string' ? withoutTrailingBlanks(oneLine(held).replace(/^[ \t]+/, '')) : ''
return quoted(paragraphOf([text(marker)], { ...reduction, depth: reduction.depth + 1 }), node, reduction) return contained(title === '' ? { type: node.type } : { attrs: { title }, type: node.type }, node, reduction)
}
// A backward scan: an unanchored-end regex retries from every blank in a long run.
function withoutTrailingBlanks(text: string): string {
let end = text.length
while (end > 0 && (text.charAt(end - 1) === ' ' || text.charAt(end - 1) === '\t')) end -= 1
return text.slice(0, end)
} }
function reduceHeading(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceHeading(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
@@ -230,8 +262,11 @@ function listItem(blocks: Result<AdfNode[]>): Result<AdfNode[]> {
// A list item's first line reads as no rule and holds no line of spaces alone: the rule and the spaces give way. // A list item's first line reads as no rule and holds no line of spaces alone: the rule and the spaces give way.
function itemOf(blocks: readonly AdfNode[]): AdfNode { function itemOf(blocks: readonly AdfNode[]): AdfNode {
const rules = blocks.findIndex((block) => block.type !== 'rule') const rules = blocks.findIndex((block) => block.type !== 'rule')
const content = blocks.slice(rules === -1 ? blocks.length : rules).map((block) => (block.type === 'codeBlock' ? { ...block, content: blankedLines(nodeContent(block)) } : block)) return { content: blankedCode(blocks.slice(rules === -1 ? blocks.length : rules)), type: 'listItem' }
return { content, type: 'listItem' } }
function blankedCode(blocks: readonly AdfNode[]): AdfNode[] {
return blocks.map((block) => (block.type === 'codeBlock' ? { ...block, content: blankedLines(nodeContent(block)) } : block))
} }
function blankedLines(code: readonly AdfNode[]): AdfNode[] { function blankedLines(code: readonly AdfNode[]): AdfNode[] {
@@ -239,26 +274,54 @@ function blankedLines(code: readonly AdfNode[]): AdfNode[] {
return blanked === '' ? [] : [text(blanked)] return blanked === '' ? [] : [text(blanked)]
} }
// A task list opening with a task and holding tasks and task lists alone keeps its spelling, a list nesting in the task before it; any other keeps its markers as text. A child reducing to nothing counts for neither.
function reduceTaskList(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceTaskList(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const items: AdfNode[] = [] const kept: { blocks: AdfNode[]; child: AdfNode }[] = []
for (const [index, child] of nodeContent(node).entries()) { for (const [index, child] of nodeContent(node).entries()) {
const blocks = taskBlocks(child, childReduction(reduction, index)) const at = childReduction(reduction, index)
if (!blocks.ok) return blocks const reduced = isTask(child) ? reduceTask(child, at) : reduceStanding(child, at)
const previous = child.type === 'taskList' ? items.pop() : undefined if (!reduced.ok) return reduced
items.push(itemOf(previous === undefined ? blocks.value : mergedLists([...nodeContent(previous), ...blocks.value]))) if (reduced.value.length > 0) kept.push({ blocks: reduced.value, child })
} }
return success(listOf(items, 'bulletList')) const regular = isTask(kept[0]?.child) && kept.every(({ child }) => isTask(child) || child.type === 'taskList')
const tasks: AdfNode[] = []
let nested: AdfNode[] = []
for (const { blocks, child } of kept) {
if (!regular) {
const standsAlone = !isTask(child) && child.type !== 'taskList'
for (const block of standsAlone ? [{ content: blocks, type: 'listItem' }] : blocks) tasks.push(block)
continue
}
if (isTask(child)) {
nestIn(tasks, nested)
nested = []
}
for (const block of blocks) (isTask(child) ? tasks : nested).push(block)
}
nestIn(tasks, nested)
if (regular) return success([{ content: tasks, type: 'taskList' }])
return success(listOf(nodeContent(tasksAsText({ content: tasks, type: 'taskList' })), 'bulletList'))
} }
function taskBlocks(child: AdfNode, at: Reduction): Result<AdfNode[]> { // The writer nests a list in the task before it, so one closing a block task item's blocks merges with it.
const marker = taskMarker(nodeAttrs(child)['state']) function nestIn(tasks: AdfNode[], nested: readonly AdfNode[]): void {
if (child.type === 'taskItem') { const previous = tasks.at(-1)
const content = reduceInline(nodeContent(child), 'paragraph', at.path, at.depth) if (previous?.type === 'blockTaskItem') tasks[tasks.length - 1] = { ...previous, content: mergedLists([...nodeContent(previous), ...nested]) }
return content.ok ? success([paragraph(content.value.length === 0 ? [text(marker)] : [text(`${marker} `), ...content.value])]) : content else for (const block of mergedLists(nested)) tasks.push(block)
}
function isTask(node: AdfNode | undefined): boolean {
return node?.type === 'taskItem' || node?.type === 'blockTaskItem'
}
function reduceTask(task: AdfNode, at: Reduction): Result<AdfNode[]> {
const attrs = { state: nodeAttrs(task)['state'] === 'DONE' ? 'DONE' : 'TODO' }
if (task.type === 'taskItem') {
const content = reduceInline(nodeContent(task), 'paragraph', at.path, at.depth)
return content.ok ? success([{ attrs, content: content.value, type: 'taskItem' }]) : content
} }
if (child.type !== 'blockTaskItem') return reduceStanding(child, at) const blocks = reduceBlocks(nodeContent(task), at)
const blocks = reduceBlocks(nodeContent(child), at) return blocks.ok ? success([{ attrs, content: blankedCode(blocks.value), type: 'blockTaskItem' }]) : blocks
return blocks.ok ? success(marked(blocks.value, marker)) : blocks
} }
// The marker leads the first paragraph, or stands as one where the blocks open with another. // The marker leads the first paragraph, or stands as one where the blocks open with another.
+4 -12
View File
@@ -7,9 +7,9 @@ import type { LinkDefinition } from '../commonmark/link-syntax.ts'
import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark/grammar.ts' import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark/grammar.ts'
import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts' import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts'
import { commonMarkLink, linkHref } from '../mark-spellings.ts' import { commonMarkLink, linkHref } from '../mark-spellings.ts'
import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' import { delimiterFlags, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts'
import { highlightDelimiter } from '../plain-conventions.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { mergeAdjacentText, sameMarks } from '../../adf/editor-normal.ts' import { mergeAdjacentText, sameMarks } from '../../adf/editor-normal.ts'
import { noSpans, readInlineDirective } from '../directive-syntax.ts' import { noSpans, readInlineDirective } from '../directive-syntax.ts'
@@ -301,22 +301,14 @@ function readDelimiterRun(scan: Scan, index: number): number {
return index + length return index + length
} }
// A delimiter is bounded outside by the code point beyond it, and flanks by the character inside it.
function readEquals(scan: Scan, index: number): number { function readEquals(scan: Scan, index: number): number {
if (!scan.highlights || !scan.source.startsWith(highlightDelimiter, index)) { if (!scan.highlights || !scan.source.startsWith(highlightDelimiter, index)) {
scan.pending += '=' scan.pending += '='
return index + 1 return index + 1
} }
const end = index + highlightDelimiter.length
const before = Array.from(scan.source.slice(Math.max(0, index - 2), index)).at(-1) ?? ''
const after = Array.from(scan.source.slice(end, end + 2))[0] ?? ''
flush(scan, false) flush(scan, false)
scan.pieces.push({ closes: flanks(before) && !isWordCharacter(after), kind: 'highlight', opens: flanks(after) && !isWordCharacter(before) }) scan.pieces.push({ ...highlightFlanking(scan.source, index), kind: 'highlight' })
return end return index + highlightDelimiter.length
}
function flanks(character: string): boolean {
return character !== '' && !/\s/.test(character)
} }
function readAutolink(source: string, index: number): { length: number; node: AdfNode } | undefined { function readAutolink(source: string, index: number): { length: number; node: AdfNode } | undefined {
+4 -6
View File
@@ -10,11 +10,11 @@ import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.t
import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts'
import { languageSlot } from '../code-language.ts' import { languageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain-conventions.ts'
import { listBreakName, listBreakSpelling } from '../block-directive.ts' import { listBreakName, listBreakSpelling } from '../block-directive.ts'
import { nodeAttrs, nodeContent } from '../../adf/document.ts' import { nodeAttrs, nodeContent } from '../../adf/document.ts'
import { parseBlocks } from './blocks.ts' import { parseBlocks } from './blocks.ts'
import { parseInlineContent } from './inline-content.ts' import { parseInlineContent } from './inline-content.ts'
import { readAlertMarker, readTaskMarker } from '../plain-conventions.ts'
import { readBlockDirectiveNode } from './directive-nodes.ts' import { readBlockDirectiveNode } from './directive-nodes.ts'
import { unsupportedNodeShape } from '../directive-syntax.ts' import { unsupportedNodeShape } from '../directive-syntax.ts'
@@ -102,10 +102,8 @@ function readBlock(block: Block, reading: Reading, path: ConvertErrorPath, depth
function markerLed<T extends { length: number }>(block: Block | undefined, read: (text: string) => T | undefined): { marker: T; position: SourcePosition; text: string } | undefined { function markerLed<T extends { length: number }>(block: Block | undefined, read: (text: string) => T | undefined): { marker: T; position: SourcePosition; text: string } | undefined {
if (block?.kind !== 'paragraph') return undefined if (block?.kind !== 'paragraph') return undefined
const marker = read(block.text) const marker = leadingMarker(block.text, read)
if (marker === undefined) return undefined return marker === undefined ? undefined : { marker, position: block.position, text: block.text.slice(marker.length) }
const text = block.text.slice(marker.length)
return text === '' || /^(?:[ \t\n]|\\\n)/.test(text) ? { marker, position: block.position, text } : undefined
} }
function markerLine(text: string): { line: string; rest: string } { function markerLine(text: string): { line: string; rest: string } {
@@ -179,7 +177,7 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr
const inExpand = reading.inExpand || read.value.node.type === 'expand' || read.value.node.type === 'nestedExpand' const inExpand = reading.inExpand || read.value.node.type === 'expand' || read.value.node.type === 'nestedExpand'
const built = directiveBody(read.value, block.blocks, { ...reading, inExpand }, path, depth) const built = directiveBody(read.value, block.blocks, { ...reading, inExpand }, path, depth)
if (!built.ok) return built if (!built.ok) return built
const readable = commonMarkSpelling(built.value, path, depth, reading.memo) const readable = commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', memo: reading.memo })
if (readable === undefined) return built if (readable === undefined) return built
if (!readable.ok) return readable if (!readable.ok) return readable
return failure('unsupported-node-shape', `${built.value.type} takes the CommonMark spelling, not the directive form`, path) return failure('unsupported-node-shape', `${built.value.type} takes the CommonMark spelling, not the directive form`, path)
@@ -3,9 +3,9 @@ import test from 'node:test'
import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts'
import { adfToMarkdown } from '../emit/adf-to-markdown.ts' import { adfToMarkdown } from '../emit/adf-to-markdown.ts'
import { adfToPlainMarkdown } from '../emit/plain-reduction.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { markdownToAdf, plainMarkdownToAdf } from './markdown-to-adf.ts' import { markdownToAdf, plainMarkdownToAdf } from './markdown-to-adf.ts'
import { reduceToPlain } from '../emit/plain-reduction.ts'
import { toEditorNormal } from '../../adf/editor-normal.ts' import { toEditorNormal } from '../../adf/editor-normal.ts'
const code: AdfMark = { type: 'code' } const code: AdfMark = { type: 'code' }
@@ -26,10 +26,8 @@ function document(...content: AdfNode[]): AdfDocument {
return { content, type: 'doc', version: 1 } return { content, type: 'doc', version: 1 }
} }
// ADF the reduction wrote, spelled and read back.
function roundTripped(...content: AdfNode[]): readonly AdfNode[] | string { function roundTripped(...content: AdfNode[]): readonly AdfNode[] | string {
const reduced = reduceToPlain(document(...content)) const markdown = adfToPlainMarkdown(document(...content))
const markdown = reduced.ok ? adfToMarkdown(reduced.value) : reduced
return markdown.ok ? read(markdown.value) : markdown.error.code return markdown.ok ? read(markdown.value) : markdown.error.code
} }
@@ -190,6 +188,16 @@ test('reads what the reduction wrote back to the node it reduced, less the attri
assert.deepEqual(roundTripped(tasks), [plainTasks]) assert.deepEqual(roundTripped(tasks), [plainTasks])
const colour: AdfMark = { attrs: { color: '#c6edfb' }, type: 'backgroundColor' } const colour: AdfMark = { attrs: { color: '#c6edfb' }, type: 'backgroundColor' }
assert.deepEqual(roundTripped(paragraph(text('a '), text('hi', colour, strong), text(' b'))), [paragraph(text('a '), text('hi', highlight, strong), text(' b'))]) assert.deepEqual(roundTripped(paragraph(text('a '), text('hi', colour, strong), text(' b'))), [paragraph(text('a '), text('hi', highlight, strong), text(' b'))])
assert.deepEqual(roundTripped(paragraph(text('=', colour), text(' '), text('a==b', colour))), [paragraph(text('=', highlight), text(' '), text('a==b', highlight))])
assert.deepEqual(roundTripped(paragraph(text('x'), text('y', colour))), [said('xy')])
assert.deepEqual(roundTripped(node('expand', { title: '**x** [y](z)' }, said('b'))), [node('expand', { title: '**x** [y](z)' }, said('b'))])
})
test('reads text the writer kept from reading as a marker back as text', () => {
const quote = bare('blockquote', said('[!NOTE] x'))
const list = bare('bulletList', bare('listItem', said('[x] a')), bare('listItem', said('[ ] b')))
assert.deepEqual(roundTripped(said('==x== a==b'), quote, list), [said('==x== a==b'), quote, list])
assert.deepEqual(roundTripped(bare('taskList', task('DONE', text('[x] ==a==')))), [bare('taskList', task('DONE', text('[x] ==a==')))])
}) })
test('keeps what markdownToAdf reads that no row reads, and refuses only what it refuses', () => { test('keeps what markdownToAdf reads that no row reads, and refuses only what it refuses', () => {
+22
View File
@@ -1,3 +1,5 @@
import { isWordCharacter } from './commonmark/emphasis-matching.ts'
export type Flavour = 'lossless' | 'plain' export type Flavour = 'lossless' | 'plain'
type AlertMarker = { folded: boolean; length: number; panelType: string } type AlertMarker = { folded: boolean; length: number; panelType: string }
@@ -45,6 +47,26 @@ export function readAlertMarker(text: string): AlertMarker | undefined {
return { folded: marker[2] !== '', length: marker[0].length, panelType: panelType ?? 'info' } return { folded: marker[2] !== '', length: marker[0].length, panelType: panelType ?? 'info' }
} }
// A marker leads text that ends at it or goes on past whitespace or a hard break.
export function leadingMarker<T extends { length: number }>(text: string, read: (text: string) => T | undefined): T | undefined {
const marker = read(text)
if (marker === undefined) return undefined
const rest = text.slice(marker.length, marker.length + 2)
return rest === '' || /^(?:[ \t\n]|\\\n)/.test(rest) ? marker : undefined
}
// A delimiter is bounded outside by the code point beyond it, and flanks by the character inside it.
export function highlightFlanking(source: string, index: number): { closes: boolean; opens: boolean } {
const end = index + highlightDelimiter.length
const before = Array.from(source.slice(Math.max(0, index - 2), index)).at(-1) ?? ''
const after = Array.from(source.slice(end, end + 2))[0] ?? ''
return { closes: flanks(before) && !isWordCharacter(after), opens: flanks(after) && !isWordCharacter(before) }
}
function flanks(character: string): boolean {
return character !== '' && !/\s/.test(character)
}
export function taskMarker(state: unknown): string { export function taskMarker(state: unknown): string {
return state === 'DONE' ? '[x]' : '[ ]' return state === 'DONE' ? '[x]' : '[ ]'
} }
+4 -13
View File
@@ -2,19 +2,6 @@
## 0.2.0 ## 0.2.0
- **35 — Read and write plain markdown as a flavour of the markdown grammar.** Per Goal 2 and
`docs/decisions.md` §Plain markdown is a flavour of the grammar, README §Plain markdown's rows are
read by `markdownToAdf`'s parser and written by `adfToMarkdown`'s writer. The
exports, their refusals and those rows stay as they are.
- **35b — Spell the plain flavour in the writer.** Panels, expands, task lists and highlights are
written by the writer, which escapes text that would read back as one, so
`plainMarkdownToAdf(adfToPlainMarkdown(doc))` keeps a literal `==x==`, a quote opening `[!NOTE]`
and a list whose items all open `[x] ` as text. A highlighted `=` (today `=====`) and `a==b`
(today `==a==b==`, highlighting `a` alone) come back highlighted whole, or lose the highlight
where no spelling holds them; the plain pair's byte-for-byte property misses both, since the
wrong document re-spells to the same bytes. An expand titled `**x** [y](z)` comes back an
expand, where today its escaped marker reads back as a quote. The reduction keeps only
degrading what the flavour cannot spell.
- **10f — Give task nodes read from plain markdown position ids.** Per `docs/decisions.md` §Plain - **10f — Give task nodes read from plain markdown position ids.** Per `docs/decisions.md` §Plain
task ids come from position, README §Plain markdown's `localId` bullet saying so. The id spelling task ids come from position, README §Plain markdown's `localId` bullet saying so. The id spelling
is part of the chunk. is part of the chunk.
@@ -47,6 +34,10 @@
paragraph of 2000 × `un` plus `**-r**`: each run its flanking cannot spell re-emits the whole line paragraph of 2000 × `un` plus `**-r**`: each run its flanking cannot spell re-emits the whole line
before riding the carry, quadratic in the runs (Goal 9), and the plain reduction's before riding the carry, quadratic in the runs (Goal 9), and the plain reduction's
`spellableLine` drops one mark per re-emit the same way. Make both linear. `spellableLine` drops one mark per re-emit the same way. Make both linear.
- **42 — Trim a text leaf's trailing blanks in linear time.** `plain-inline.ts`'s `leafEdges` finds
the trail with an unanchored `/[ \t]*$/`, quadratic in a run of blanks inside one leaf: a
paragraph of `a`, 80 000 spaces, `b` takes 6.5 s in `adfToPlainMarkdown` (Goal 9). Scan backward,
as the expand title's trim does.
- **34 — Read emphasis flanking by the whole character beside an astral symbol.** Check whether - **34 — Read emphasis flanking by the whole character beside an astral symbol.** Check whether
`line-escaping.ts`'s `charAt` and the parser's flanking read one UTF-16 unit beside an astral `line-escaping.ts`'s `charAt` and the parser's flanking read one UTF-16 unit beside an astral
symbol — a lone surrogate is neither punctuation nor symbol, where CommonMark reads `😀` as symbol — a lone surrogate is neither punctuation nor symbol, where CommonMark reads `😀` as