3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-20 07:05:44 +01:00

Answer architecture round 2: every builder joins its pieces through one bounded join, the readers take the engine guard, every refusal stands at the document's path, and a portable list's refusal names its own place

This commit is contained in:
2026-10-12 17:42:24 +02:00
parent 43c8d63860
commit 88c00d381d
25 changed files with 379 additions and 249 deletions
+4 -2
View File
@@ -85,9 +85,11 @@
where all three hung. where all three hung.
- Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string: - Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on `adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
every engine, before building it, where they threw or ran out of memory. every engine, where they threw or ran out of memory.
- Fix a directive whose CommonMark spelling would be that long: `losslessMarkdownToAdf` refuses it - Fix a directive whose CommonMark spelling would be that long: `losslessMarkdownToAdf` refuses it
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory. as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory. The
markdown readers return `unsupported-node-shape` where they threw past the longest string or
array the JavaScript engine builds.
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as - Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw. `unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from - Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
+3 -3
View File
@@ -212,9 +212,9 @@ Each call's name says which markdown it reads or writes:
| --- | --- | --- | | --- | --- | --- |
| `adfToLosslessMarkdown` | writes the lossless flavour: CommonMark plus GFM's pipe tables and `~~`, with `!adf:` directives and `adf:` fences for everything else | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` | | `adfToLosslessMarkdown` | writes the lossless flavour: CommonMark plus GFM's pipe tables and `~~`, with `!adf:` directives and `adf:` fences for everything else | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` |
| `adfToPortableMarkdown` | writes markdown GitHub, GitLab and Obsidian render; drops what they cannot show, and writes no directive | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` | | `adfToPortableMarkdown` | writes markdown GitHub, GitLab and Obsidian render; drops what they cannot show, and writes no directive | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` |
| `commonMarkToAdf` | reads strict CommonMark as its spec says: a directive, a pipe table, `~~`, `==` and an `adf:` fence read as text and code | `not-a-string`, `unmappable-html`, `unsupported-nesting-depth` | | `commonMarkToAdf` | reads strict CommonMark as its spec says: a directive, a pipe table, `~~`, `==` and an `adf:` fence read as text and code | `not-a-string`, `unmappable-html`, `unsupported-nesting-depth`, `unsupported-node-shape` |
| `losslessMarkdownToAdf` | reads the lossless flavour, markdown `adfToLosslessMarkdown` wrote, edited or not | every code in [The errors](#the-errors) except `not-an-adf-document` and `unsupported-document-version` | | `losslessMarkdownToAdf` | reads the lossless flavour, markdown `adfToLosslessMarkdown` wrote, edited or not | every code in [The errors](#the-errors) except `not-an-adf-document` and `unsupported-document-version` |
| `portableMarkdownToAdf` | reads CommonMark, GFM's pipe tables and `~~`, alerts and task markers, Obsidian's callouts and `==`; a directive and an `adf:` fence read as text and code | `malformed-pipe-table`, `not-a-string`, `unmappable-html`, `unsupported-nesting-depth` | | `portableMarkdownToAdf` | reads CommonMark, GFM's pipe tables and `~~`, alerts and task markers, Obsidian's callouts and `==`; a directive and an `adf:` fence read as text and code | `malformed-pipe-table`, `not-a-string`, `unmappable-html`, `unsupported-nesting-depth`, `unsupported-node-shape` |
`isAdfDocument` is the type guard for the boundary you receive JSON at. It checks the shape the `isAdfDocument` is the type guard for the boundary you receive JSON at. It checks the shape the
converters require: `type: 'doc'`, a finite `version` (the converters take only 1), nodes holding converters require: `type: 'doc'`, a finite `version` (the converters take only 1), nodes holding
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do | | Code | Fires when | What you can do |
| --- | --- | --- | | --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters, keep the ADF and pass the document over, or show it read-only | | `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only |
## The guarantees ## The guarantees
+17 -14
View File
@@ -462,11 +462,11 @@ handles one cause alike whichever node, attribute or direction raised it.
- A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document` - A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document`
emitting — no document holds one, so no round-trip crosses them (2026-09-23). emitting — no document holds one, so no round-trip crosses them (2026-09-23).
- A document whose markdown would be longer than V8's longest string is `unsupported-node-shape` - A document whose markdown would be longer than V8's longest string is `unsupported-node-shape`
at the block that passes it, in either writer, and so is a directive `losslessMarkdownToAdf` at the document's path, in either writer, and so is a directive `losslessMarkdownToAdf` reads
reads whose CommonMark spelling would be: the document does not convert, and no spelling in the whose CommonMark spelling would be: the document does not convert, and no spelling in the input
input causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON,
JSON, since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a
a length linear in its JSON (2026-10-11). length linear in its JSON (2026-10-11).
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at - The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
`not-an-adf-document` names the emitters' input (2026-10-05). `not-an-adf-document` names the emitters' input (2026-10-05).
@@ -665,15 +665,18 @@ or pins it.
is the shortest of the engines the library claims. is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestOutput`) before building it, so the same document converts alike everywhere and no (`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
amplification — an indent on every line of a deep subtree, the carry's two-space JSON, text one converts alike everywhere. Each builder that joins pieces — blocks, quoted and list lines, directive
string shared by many nodes — builds a string no engine was asked for. Each walk of blocks projects pairs and openers, fences, pipe rows, the carry's JSON — joins them through `joinedWithin`, which
their length with the indent their containers will prefix every line with, and a container joining stops once they pass the bound, so none builds a string no engine was asked for. A walk of blocks
several walks — a list's items, a table's rows — sums their projections before it joins them, so the also forecasts each block's length with the indent its containers will prefix every line with, so a
bound trips at the block that passes it, before any container copies it. The projection never deep subtree refuses before any container copies it.
undercounts: a list giving way to the directive form counts its items' marker indent, so markdown a
few characters a line under the bound may be refused. The engines' own throws past their longest A line is held to its text's length alone; what its spelling adds — escapes, link targets,
string or array stay refused as `unsupported-node-shape` by `withinEngineLimits`, behind this bound. attributes — passes V8's longest string only by throwing, and `withinEngineLimits` refuses that
throw, and SpiderMonkey's, as the same refusal: both mean a string past the bound. `todo.md` item
108 bounds the line itself. The engines' throws past their own longest array, and JavaScriptCore's
`Out of memory`, which also reports a heap running short, stay refused with the engine's message.
## Nothing recurses unbounded ## Nothing recurses unbounded
+1 -1
View File
@@ -86,7 +86,7 @@ export function textLength(node: AdfNode): number {
return node.text?.length ?? 0 return node.text?.length ?? 0
} }
// A line spells every character of its nodes' text at least once, so this much text cannot fit. // The length of the text the nodes hold, the nodes below them included.
export function heldTextLength(nodes: readonly AdfNode[]): number { export function heldTextLength(nodes: readonly AdfNode[]): number {
let length = 0 let length = 0
const pending = [...nodes] const pending = [...nodes]
+10 -5
View File
@@ -16,16 +16,21 @@ export function closingBacktickRun(text: string, from: number, opener: number):
} }
export function fencedCodeBlock(info: string, body: string): string { export function fencedCodeBlock(info: string, body: string): string {
return fencePieces(info, body).join('')
}
export function fencePieces(info: string, body: string): string[] {
const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1)) const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1))
return body === '' ? `${fence}${info}\n${fence}` : `${fence}${info}\n${body}\n${fence}` return body === '' ? [fence, info, '\n', fence] : [fence, info, '\n', body, '\n', fence]
} }
export function longestBacktickRun(text: string): number { export function longestBacktickRun(text: string): number {
let longest = 0 let longest = 0
let current = 0 for (let start = text.indexOf('`'); start !== -1; ) {
for (const character of text) { let end = start + 1
current = character === '`' ? current + 1 : 0 while (text.charAt(end) === '`') end += 1
longest = Math.max(longest, current) longest = Math.max(longest, end - start)
start = text.indexOf('`', end)
} }
return longest return longest
} }
+24 -4
View File
@@ -112,9 +112,18 @@ export function slotFault(type: string, text: string): ConvertFault | undefined
} }
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string { export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return '' return [...attributePieces(pairs)].join('')
const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`) }
return `{${spelled.join(' ')}}`
export function* attributePieces(pairs: readonly (readonly [string, string])[]): Generator<string> {
if (pairs.length === 0) return
yield '{'
for (const [index, [key, value]] of [...pairs].sort(([left], [right]) => keyOrder(left, right)).entries()) {
if (index > 0) yield ' '
yield `${key}=`
yield value
}
yield '}'
} }
export function spellDirectiveCloser(name: string): string { export function spellDirectiveCloser(name: string): string {
@@ -122,7 +131,18 @@ export function spellDirectiveCloser(name: string): string {
} }
export function spellDirectiveOpener(name: string, argument: string | undefined, attributes: string): string { export function spellDirectiveOpener(name: string, argument: string | undefined, attributes: string): string {
return `${directivePrefix}${name}${argument === undefined ? '' : ` ${argument}`}${attributes === '' ? '' : ` ${attributes}`}` return [...directiveOpenerPieces(name, argument, attributes === '' ? [] : [attributes])].join('')
}
export function* directiveOpenerPieces(name: string, argument: string | undefined, attributes: Iterable<string>): Generator<string> {
yield `${directivePrefix}${name}`
if (argument !== undefined) yield ` ${argument}`
let first = true
for (const piece of attributes) {
if (first) yield ' '
first = false
yield piece
}
} }
export function spellJsonAttribute(value: JsonValue): string { export function spellJsonAttribute(value: JsonValue): string {
+13 -6
View File
@@ -44,7 +44,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN') assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
}) })
const tooLong = 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds' const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds'
// About 556 million characters of carry JSON: past V8's longest string, short of JavaScriptCore's and SpiderMonkey's. // About 556 million characters of carry JSON: past V8's longest string, short of JavaScriptCore's and SpiderMonkey's.
test('refuses a carry whose JSON would run past V8\'s longest string, on every engine', () => { test('refuses a carry whose JSON would run past V8\'s longest string, on every engine', () => {
@@ -52,15 +52,13 @@ test('refuses a carry whose JSON would run past V8\'s longest string, on every e
for (let level = 0; level < 495; level += 1) value = [value] for (let level = 0; level < 495; level += 1) value = [value]
const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' })) const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' }))
assert.equal(markdown(result), tooLong) assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
}) })
test('refuses lines their containers would prefix past V8\'s longest string, at the block holding them', () => { test('refuses lines their containers would prefix past V8\'s longest string, before any container copies them', () => {
let quote: AdfNode = { attrs: {}, content: [{ text: 'a\n'.repeat(1_100_000), type: 'text' }], type: 'codeBlock' } let quote: AdfNode = { attrs: {}, content: [{ text: 'a\n'.repeat(1_100_000), type: 'text' }], type: 'codeBlock' }
for (let level = 0; level < 245; level += 1) quote = { content: [quote], type: 'blockquote' } for (let level = 0; level < 245; level += 1) quote = { content: [quote], type: 'blockquote' }
const result = adfToLosslessMarkdown(document(quote)) const result = adfToLosslessMarkdown(document(quote))
assert.equal(markdown(result), tooLong) assert.equal(markdown(result), tooLong)
assert.equal(path(result).length, 2 * 246)
}) })
test('refuses a directive whose opener and body together would pass V8\'s longest string', () => { test('refuses a directive whose opener and body together would pass V8\'s longest string', () => {
@@ -73,7 +71,6 @@ test('refuses a list whose items together would pass V8\'s longest string, befor
const item: AdfNode = { content: [{ attrs: {}, content: [{ text: 'a'.repeat(2 ** 28), type: 'text' }], type: 'codeBlock' }], type: 'listItem' } const item: AdfNode = { content: [{ attrs: {}, content: [{ text: 'a'.repeat(2 ** 28), type: 'text' }], type: 'codeBlock' }], type: 'listItem' }
const result = adfToLosslessMarkdown(document({ content: [item, item], type: 'bulletList' })) const result = adfToLosslessMarkdown(document({ content: [item, item], type: 'bulletList' }))
assert.equal(markdown(result), tooLong) assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0, 'content', 1])
}) })
test('refuses a pipe table whose rows their containers would prefix past V8\'s longest string', () => { test('refuses a pipe table whose rows their containers would prefix past V8\'s longest string', () => {
@@ -84,11 +81,21 @@ test('refuses a pipe table whose rows their containers would prefix past V8\'s l
assert.equal(markdown(adfToLosslessMarkdown(document(quote))), tooLong) assert.equal(markdown(adfToLosslessMarkdown(document(quote))), tooLong)
}) })
test('refuses a code block whose fences together would pass V8\'s longest string', () => {
const long: AdfNode = { text: 'a'.repeat(2 ** 28), type: 'text' }
assert.equal(markdown(adfToLosslessMarkdown(document({ attrs: {}, content: [long, long, long], type: 'codeBlock' }))), tooLong)
})
test('refuses a directive opener whose attributes together would pass V8\'s longest string', () => {
const long = 'a'.repeat(2 ** 28)
const panel: AdfNode = { attrs: { panelColor: long, panelIcon: long, panelIconText: long, panelType: 'custom' }, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' }
assert.equal(markdown(adfToLosslessMarkdown(document(panel))), tooLong)
})
test('refuses text past V8\'s longest string before spelling it', () => { test('refuses text past V8\'s longest string before spelling it', () => {
const long = 'a'.repeat(2 ** 28) const long = 'a'.repeat(2 ** 28)
const result = adfToLosslessMarkdown(document(paragraph({ text: long, type: 'text' }, { text: long, type: 'text' }, { text: long, type: 'text' }))) const result = adfToLosslessMarkdown(document(paragraph({ text: long, type: 'text' }, { text: long, type: 'text' }, { text: long, type: 'text' })))
assert.equal(markdown(result), tooLong) assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
}) })
test('refuses a document version the markdown cannot carry', () => { test('refuses a document version the markdown cannot carry', () => {
+119 -100
View File
@@ -7,12 +7,12 @@ import { blockDirectiveForm, documentSpelling } from '../block-directive.ts'
import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts' import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts'
import { carriedBlock } from '../opaque-carry.ts' import { carriedBlock } from '../opaque-carry.ts'
import { emitInlineLine } from './inline-line.ts' import { emitInlineLine } from './inline-line.ts'
import { failure, faulted, success, withinEngineLimits, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { fencedCodeBlock } from '../commonmark/backtick-runs.ts' import { fencePieces } from '../commonmark/backtick-runs.ts'
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts' import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong, withinEngineLimits } from '../length-bound.ts'
import { languageSlot, type LanguageSlot } from '../code-language.ts' import { languageSlot, type LanguageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts' import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts' import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
import { tryImage } from './image.ts' import { tryImage } from './image.ts'
@@ -24,8 +24,7 @@ type Delimiters = readonly [string, string]
type EmittedBlock = SpelledBlock & { headroom: number } type EmittedBlock = SpelledBlock & { headroom: number }
type KeptSpelling = { block: EmittedBlock | undefined; depth: number } type KeptSpelling = { block: EmittedBlock | undefined; depth: number }
type PlacedBlock = SpelledBlock & { node: AdfNode } type PlacedBlock = SpelledBlock & { node: AdfNode }
// projected: the blocks' length once joined and their containers prefix their lines; it never undercounts. type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number }
type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number; projected: number }
// CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`. // CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`.
type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string } type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string }
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node. // Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
@@ -35,12 +34,12 @@ type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
export type Writing = { flavour: WrittenFlavour; indent: number; memo: SpellingMemo | undefined } export type Writing = { flavour: WrittenFlavour; indent: number; memo: SpellingMemo | undefined }
const bulletDelimiters: Delimiters = ['-', '*'] const bulletDelimiters: Delimiters = ['-', '*']
const bulletWidth = 2 const bulletWidth = '- '.length
export const largestListMarker = 999999999 export const largestListMarker = 999999999
// Bare because tryList admits no item carrying attributes, marks or text. // Bare because tryList admits no item carrying attributes, marks or text.
const listItemOpener = spellDirectiveOpener('listItem', undefined, '') const listItemOpener = spellDirectiveOpener('listItem', undefined, '')
const listItemCloser = spellDirectiveCloser('listItem')
const orderedDelimiters: Delimiters = ['.', ')'] const orderedDelimiters: Delimiters = ['.', ')']
const quoteWidth = '> '.length
export function adfToLosslessMarkdown(document: AdfDocument): Result<string> { export function adfToLosslessMarkdown(document: AdfDocument): Result<string> {
return withinEngineLimits(() => writeMarkdown(document, 'lossless')) return withinEngineLimits(() => writeMarkdown(document, 'lossless'))
@@ -53,29 +52,25 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
if (document.content === undefined) return success(`${documentSpelling}\n`) if (document.content === undefined) return success(`${documentSpelling}\n`)
const walk = walkBlocks(document.content, [], 0, { flavour, indent: 0, memo: undefined }) const walk = walkBlocks(document.content, [], 0, { flavour, indent: 0, memo: undefined })
if (!walk.ok) return walk if (!walk.ok) return walk
const text = joinBlocks(walk.value.blocks, 'document') const text = joinedWithin(walk.value.blocks.length === 0 ? [] : [...blockPieces(walk.value.blocks, 'document'), '\n'])
return success(text === '' ? '' : `${text}\n`) return boundedThen(text, success)
} }
function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> { function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
let headroom = largestNesting - depth let headroom = largestNesting - depth
if (headroom < 0) return tooDeep(path) if (headroom < 0) return tooDeep(path)
const blocks: PlacedBlock[] = [] const blocks: PlacedBlock[] = []
let projected = 0 // A forecast, so a deep subtree refuses where its containers would prefix its lines past the bound, before any of them copies those lines.
let forecast = 0
for (const [index, node] of nodes.entries()) { for (const [index, node] of nodes.entries()) {
const block = emitBlock(node, [...path, 'content', index], depth, writing) const block = emitBlock(node, [...path, 'content', index], depth, writing)
if (!block.ok) return block if (!block.ok) return block
projected += (index === 0 ? 0 : separatorLength(writing)) + prefixedLength(block.value.text, writing.indent) + 1 forecast += prefixedLength(block.value.text, writing.indent)
if (projected > longestOutput) return tooLong([...path, 'content', index]) if (forecast > longestMarkdown) return tooLong()
headroom = Math.min(headroom, block.value.headroom) headroom = Math.min(headroom, block.value.headroom)
blocks.push({ ...block.value, node }) blocks.push({ ...block.value, node })
} }
return success({ blocks, headroom, projected }) return success({ blocks, headroom })
}
// A block counts the newline ending it; a blank line between two is prefixed as their lines are.
function separatorLength(writing: Writing): number {
return 1 + writing.indent
} }
function prefixedLength(text: string, indent: number): number { function prefixedLength(text: string, indent: number): number {
@@ -92,16 +87,18 @@ function tooDeep(path: ConvertErrorPath): Result<never> {
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
} }
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string { function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string | undefined {
let text = '' return joinedWithin(blockPieces(blocks, container))
}
function* blockPieces(blocks: readonly PlacedBlock[], container: BlockContainer): Generator<string> {
let alternated = false let alternated = false
for (const [index, block] of blocks.entries()) { for (const [index, block] of blocks.entries()) {
const previous = blocks[index - 1] const previous = blocks[index - 1]
if (previous !== undefined) text += separationBetween(previous, block, container) if (previous !== undefined) yield separationBetween(previous, block, container)
alternated = !alternated && previous !== undefined && sharesMarker(previous, block) alternated = !alternated && previous !== undefined && sharesMarker(previous, block)
text += alternated && block.spelling === 'list' ? block.alternateText : block.text yield alternated && block.spelling === 'list' ? block.alternateText : block.text
} }
return text
} }
// A task list is spelled as a bullet list. // A task list is spelled as a bullet list.
@@ -156,13 +153,13 @@ function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
if (portable !== undefined) return portable if (portable !== undefined) return portable
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing) if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing) if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
if (node.type === 'codeBlock') return readableText(tryCodeBlock(node, writing.flavour)) if (node.type === 'codeBlock') return tryCodeBlock(node, writing.flavour)
if (node.type === 'heading') return tryHeading(node, path, writing.flavour) if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
if (node.type === 'mediaSingle') return readableText(tryImage(node)) if (node.type === 'mediaSingle') return readableText(tryImage(node))
if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour) if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
if (node.type === 'rule') return readableText(tryRule(node)) if (node.type === 'rule') return readableText(tryRule(node))
if (node.type === 'table') { if (node.type === 'table') {
const table = tryPipeTable(node, path, writing) const table = tryPipeTable(node, path, writing.flavour)
return table === undefined || !table.ok ? table : success(commonMarkText(table.value)) return table === undefined || !table.ok ? table : success(commonMarkText(table.value))
} }
return undefined return undefined
@@ -180,27 +177,31 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
} }
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> { function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const quotedWriting = indented(writing, quoteWidth) const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
const inner = walkBlocks(nodeContent(node), path, depth + 1, quotedWriting)
if (!inner.ok) return inner if (!inner.ok) return inner
if (prefixedLength(head, quotedWriting.indent) + 1 + separatorLength(quotedWriting) + inner.value.projected > longestOutput) return tooLong(path) const text = joinedWithin(inner.value.blocks.length === 0 ? [head] : [head, '\n\n', ...blockPieces(inner.value.blocks, 'document')])
const body = joinBlocks(inner.value.blocks, 'document') return quotedBlock(text, inner.value.headroom)
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
} }
const quoteWidth = 2 function quotedBlock(text: string | undefined, headroom: number): Result<EmittedBlock> {
return boundedThen(text, (whole) => boundedThen(joinedWithin(quotedPieces(whole)), (quotedText) => success(commonMarkText(quotedText, headroom))))
}
function quoted(text: string): string {
return text function* quotedPieces(text: string): Generator<string> {
.split('\n') for (let start = 0; ; ) {
.map((line) => (line === '' ? '>' : `> ${line}`)) const end = text.indexOf('\n', start)
.join('\n') const line = text.slice(start, end === -1 ? text.length : end)
yield line === '' ? '>' : '> '
yield line
if (end === -1) return
yield '\n'
start = end + 1
}
} }
function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> { function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const items: PlacedBlock[][] = [] const items: PlacedBlock[][] = []
// Each walk alone stayed within the bound, so the join is checked before it builds.
let projected = 0
let headroom = largestNesting - depth - 1 let headroom = largestNesting - depth - 1
// A child other than a task nests in the task before it. // A child other than a task nests in the task before it.
for (const [index, child] of nodeContent(node).entries()) { for (const [index, child] of nodeContent(node).entries()) {
@@ -208,26 +209,31 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
const itemWriting = indented(writing, bulletWidth) const itemWriting = indented(writing, bulletWidth)
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, itemWriting) : placedBlock(child, [...path, 'content', index], depth + 1, itemWriting) const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, itemWriting) : placedBlock(child, [...path, 'content', index], depth + 1, itemWriting)
if (!walk.ok) return walk if (!walk.ok) return walk
projected += (index === 0 ? 0 : separatorLength(writing)) + walk.value.projected
if (projected > longestOutput) return tooLong([...path, 'content', index])
headroom = Math.min(headroom, walk.value.headroom) headroom = Math.min(headroom, walk.value.headroom)
const previous = items.at(-1) const previous = items.at(-1)
if (task || previous === undefined) items.push([...walk.value.blocks]) if (task || previous === undefined) items.push([...walk.value.blocks])
else for (const block of walk.value.blocks) previous.push(block) else for (const block of walk.value.blocks) previous.push(block)
} }
const list = spelledList(
items.map((blocks) => joinBlocks(blocks, 'list-item')),
() => '',
bulletDelimiters,
headroom,
)
// The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour. // The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour.
return list === undefined ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success(list) return boundedThen(joinedItems(items.map((blocks) => blockPieces(blocks, 'list-item'))), (inners) => spelledList(inners, () => '', bulletDelimiters, headroom) ?? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path))
}
// Each item's text, or `undefined` once all of them together pass the bound; each alone stayed within the forecast.
function joinedItems(items: readonly Iterable<string>[]): string[] | undefined {
const inners: string[] = []
let length = 0
for (const pieces of items) {
const inner = [...pieces].join('')
length += inner.length
if (length > longestMarkdown) return undefined
inners.push(inner)
}
return inners
} }
function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> { function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
const block = emitBlock(node, path, depth, writing) const block = emitBlock(node, path, depth, writing)
return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom, projected: prefixedLength(block.value.text, writing.indent) + 1 }) : block return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom }) : block
} }
// The marker leads the first paragraph, or stands as one where the blocks open with another. // The marker leads the first paragraph, or stands as one where the blocks open with another.
@@ -238,15 +244,15 @@ function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writin
const content = nodeContent(task) const content = nodeContent(task)
const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour) const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour)
if (!line.ok) return line if (!line.ok) return line
const text = line.value === '' ? marker : `${marker} ${line.value}` const text = joinedWithin(line.value === '' ? [marker] : [marker, ' ', line.value])
return success({ blocks: [{ ...markerBlock, text }], headroom: Number.POSITIVE_INFINITY, projected: prefixedLength(text, writing.indent) + 1 }) return boundedThen(text, (spelled) => success({ blocks: [{ ...markerBlock, text: spelled }], headroom: Number.POSITIVE_INFINITY }))
} }
const walk = walkBlocks(nodeContent(task), path, depth, writing) const walk = walkBlocks(nodeContent(task), path, depth, writing)
if (!walk.ok) return walk if (!walk.ok) return walk
const [first, ...rest] = walk.value.blocks const [first, ...rest] = walk.value.blocks
const blocks = first?.node.type === 'paragraph' ? [{ ...first, text: `${marker} ${first.text}` }, ...rest] : [markerBlock, ...walk.value.blocks] if (first?.node.type !== 'paragraph') return success({ blocks: [markerBlock, ...walk.value.blocks], headroom: walk.value.headroom })
// The marker either leads the first line or stands as a block of its own. const text = joinedWithin([marker, ' ', first.text])
return success({ blocks, headroom: walk.value.headroom, projected: walk.value.projected + marker.length + separatorLength(writing) + writing.indent + 1 }) return boundedThen(text, (spelled) => success({ blocks: [{ ...first, text: spelled }, ...rest], headroom: walk.value.headroom }))
} }
function readableText(text: string | undefined): Result<EmittedBlock> | undefined { function readableText(text: string | undefined): Result<EmittedBlock> | undefined {
@@ -262,8 +268,10 @@ function commonMarkText(text: string, headroom: number = Number.POSITIVE_INFINIT
return { headroom, spelling: 'commonmark', text } return { headroom, spelling: 'commonmark', text }
} }
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): EmittedBlock { function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): Result<EmittedBlock> {
return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` } const closer = spellDirectiveCloser(node.type)
const text = joinedWithin(body === '' ? [opener, '\n', closer] : [opener, '\n', body, '\n', closer])
return boundedThen(text, (spelled) => success({ headroom, spelling: 'directive', text: spelled }))
} }
function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> { function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> {
@@ -281,12 +289,12 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
if (model.contentModel === 'inline') { if (model.contentModel === 'inline') {
const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless') const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless')
if (!line.ok) return line if (!line.ok) return line
return success(directivePair(node, opener, line.value)) return directivePair(node, opener, line.value)
} }
const walk = walkBody() const walk = walkBody()
if (!walk.ok) return walk if (!walk.ok) return walk
if (opener.length + walk.value.projected > longestOutput) return tooLong(path) const body = joinBlocks(walk.value.blocks, 'directive')
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom)) return boundedThen(body, (spelled) => directivePair(node, opener, spelled, walk.value.headroom))
} }
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
@@ -294,17 +302,18 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth)) const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document') const text = joinBlocks(inner.value.blocks, 'document')
const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined const alert = text !== undefined && flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom)) return quotedBlock(alert ? joinedWithin(['\\', text]) : text, inner.value.headroom)
} }
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): string | undefined { function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
if (!holdsOnlyAttributes(node, ['language'])) return undefined if (!holdsOnlyAttributes(node, ['language'])) return undefined
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour]) const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour])
if (slot.kind === 'attribute') return undefined if (slot.kind === 'attribute') return undefined
const [only, ...others] = fencedTexts(node) ?? [] const [only, ...others] = fencedTexts(node) ?? []
if (only === undefined || others.length > 0) return undefined if (only === undefined || others.length > 0) return undefined
return fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only) const text = joinedWithin(fencePieces(slot.kind === 'fence' ? slot.info : '', only))
return boundedThen(text, (spelled) => success(commonMarkText(spelled)))
} }
// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does. // spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does.
@@ -316,7 +325,8 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr
if (opener === undefined) return carriedFence(carriedBlock(node, path, depth)) if (opener === undefined) return carriedFence(carriedBlock(node, path, depth))
if (!opener.ok) return opener if (!opener.ok) return opener
const info = slot.kind === 'fence' ? slot.info : '' const info = slot.kind === 'fence' ? slot.info : ''
return success(directivePair(node, opener.value, texts.map((text) => fencedCodeBlock(info, text)).join('\n'))) const body = joinedWithin(separatedBy('\n', texts.map((text) => fencePieces(info, text))))
return boundedThen(body, (spelled) => directivePair(node, opener.value, spelled))
} }
// The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node. // The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node.
@@ -350,8 +360,6 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
if (start === undefined || items.length === 0) return undefined if (start === undefined || items.length === 0) return undefined
if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined
const walked: WalkedItem[] = [] const walked: WalkedItem[] = []
// Each walk alone stayed within the bound, so the join is checked before it builds; the directive form adds an opener and a closer line to each item.
let projected = 0
let headroom = Number.POSITIVE_INFINITY let headroom = Number.POSITIVE_INFINITY
const itemWriting = indented(writing, ordered ? `${start + items.length - 1}. `.length : bulletWidth) const itemWriting = indented(writing, ordered ? `${start + items.length - 1}. `.length : bulletWidth)
for (const [offset, item] of items.entries()) { for (const [offset, item] of items.entries()) {
@@ -359,43 +367,45 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
if (!walk.ok) return walk if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom) headroom = Math.min(headroom, walk.value.headroom)
walked.push({ node: item, walk: walk.value }) walked.push({ node: item, walk: walk.value })
projected += (offset === 0 ? 0 : separatorLength(writing)) + walk.value.projected + directiveItemLength(writing)
if (projected > longestOutput) return tooLong([...path, 'content', offset])
} }
const inners = walked.map((item) => { return boundedThen(joinedItems(walked.map((item) => blockPieces(item.walk.blocks, 'list-item'))), (joined) => {
const inner = joinBlocks(item.walk.blocks, 'list-item')
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold. // GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
return flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner const inners = joined.map((inner) => (flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner))
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
if (list !== undefined) return list
// The directive form spends a level the walk did not count.
if (headroom < 1) return tooDeep(path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => directiveItems(walked, headroom - 1))
}) })
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
if (list !== undefined) return success(list)
// The directive form spends a level the walk did not count.
if (headroom < 1) return tooDeep(path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1, projected }))
} }
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): EmittedBlock | undefined { function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): Result<EmittedBlock> | undefined {
const text = listLines(inners, number, usual) const pieces = listLines(inners, number, usual)
const alternateText = listLines(inners, number, alternate) const alternatePieces = listLines(inners, number, alternate)
return text === undefined || alternateText === undefined ? undefined : { alternateText, headroom, spelling: 'list', text } if (pieces === undefined || alternatePieces === undefined) return undefined
// The alternate delimiter is as wide as the usual one, so its text is as long.
return boundedThen(joinedWithin(pieces), (text) => success({ alternateText: [...alternatePieces].join(''), headroom, spelling: 'list', text }))
} }
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): string | undefined { // `undefined` where an item holds a line no list item spells.
const lines: string[] = [] function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): Iterable<string> | undefined {
const items: Iterable<string>[] = []
for (const [offset, inner] of inners.entries()) { for (const [offset, inner] of inners.entries()) {
const line = tryListItemLines(inner, `${number(offset)}${delimiter} `) const pieces = listItemPieces(inner, `${number(offset)}${delimiter} `)
if (line === undefined) return undefined if (pieces === undefined) return undefined
lines.push(line) items.push(pieces)
} }
return lines.join('\n') return separatedBy('\n', items)
} }
function directiveItemLength(writing: Writing): number { function directiveItems(items: readonly WalkedItem[], headroom: number): Result<PlacedBlocks> {
return listItemOpener.length + listItemCloser.length + 2 * (1 + writing.indent) const blocks: PlacedBlock[] = []
} for (const item of items) {
const pair = boundedThen(joinBlocks(item.walk.blocks, 'directive'), (body) => directivePair(item.node, listItemOpener, body))
function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] { if (!pair.ok) return pair
return items.map((item) => ({ ...directivePair(item.node, listItemOpener, joinBlocks(item.walk.blocks, 'directive')), node: item.node })) blocks.push({ ...pair.value, node: item.node })
}
return success({ blocks, headroom })
} }
function listStart(node: AdfNode, items: number): number | undefined { function listStart(node: AdfNode, items: number): number | undefined {
@@ -405,16 +415,25 @@ function listStart(node: AdfNode, items: number): number | undefined {
return start + items - 1 > largestListMarker ? undefined : start return start + items - 1 > largestListMarker ? undefined : start
} }
function tryListItemLines(inner: string, marker: string): string | undefined { function listItemPieces(inner: string, marker: string): Iterable<string> | undefined {
if (inner === '') return marker.trimEnd() if (inner === '') return [marker.trimEnd()]
const body = inner.split('\n') const [first = '', ...rest] = inner.split('\n')
if (body.some((line) => line !== '' && isBlankLine(line))) return undefined if (rest.some((line) => line !== '' && isBlankLine(line)) || (first !== '' && isBlankLine(first))) return undefined
const indent = ' '.repeat(marker.length)
const lines = body.map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
const first = lines[0] ?? ''
if (!isThematicBreak(first)) return lines.join('\n')
// An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text. // An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text.
return inner.startsWith('*') ? [`${marker}\\${first.slice(marker.length)}`, ...lines.slice(1)].join('\n') : undefined const rule = isThematicBreak(`${marker}${first}`)
if (rule && !inner.startsWith('*')) return undefined
return itemLinePieces(rule ? `${marker}\\` : marker, first, rest, ' '.repeat(marker.length))
}
function* itemLinePieces(opening: string, first: string, rest: readonly string[], indent: string): Generator<string> {
yield opening
yield first
for (const line of rest) {
yield '\n'
if (line === '') continue
yield indent
yield line
}
} }
function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined { function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
@@ -3,7 +3,8 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts'
import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts'
import { blockArgument, markValues, marksAttribute } from '../block-directive.ts' import { blockArgument, markValues, marksAttribute } from '../block-directive.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts' import { joinedWithin, tooLong } from '../length-bound.ts'
import { attributePieces, directiveOpenerPieces, isBareToken, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { overNested } from '../../json-value.ts' import { overNested } from '../../json-value.ts'
import { spellEmptyKeys } from '../empty-keys.ts' import { spellEmptyKeys } from '../empty-keys.ts'
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
@@ -22,7 +23,8 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel,
if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path) if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path)
spelledPairs.push([marksAttribute, spellJsonAttribute(values)]) spelledPairs.push([marksAttribute, spellJsonAttribute(values)])
} }
return success(spellDirectiveOpener(node.type, slot.argument, spellAttributes(spelledPairs))) const opener = joinedWithin(directiveOpenerPieces(node.type, slot.argument, attributePieces(spelledPairs)))
return opener === undefined ? tooLong() : success(opener)
} }
// `undefined` where the argument slot holds a value no bare token spells. // `undefined` where the argument slot holds a value no bare token spells.
+3 -3
View File
@@ -9,13 +9,13 @@ import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../
import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts' import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts'
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
import { heldTextLength, identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { heldTextLength, identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts' import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../commonmark/backtick-runs.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts'
import { longestOutput, tooLong } from '../../output-length.ts' import { longestMarkdown, tooLong } from '../length-bound.ts'
import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
import { spellTextDirective } from '../text-directive.ts' import { spellTextDirective } from '../text-directive.ts'
@@ -81,7 +81,7 @@ export function tryImageLine(alt: string | undefined, url: string, link?: LinkDe
} }
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> { function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> {
if (heldTextLength(nodes) > longestOutput) return tooLong(path) if (heldTextLength(nodes) > longestMarkdown) return tooLong()
const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false } const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false }
// Terminates because takeFallback refuses a pass that took no new fallback. // Terminates because takeFallback refuses a pass that took no new fallback.
for (;;) { for (;;) {
+18 -13
View File
@@ -1,33 +1,38 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfNode } from '../../adf/document.ts'
import type { WrittenFlavour } from '../portable/conventions.ts' import type { WrittenFlavour } from '../portable/conventions.ts'
import { delimiterCells, pipeRowPieces } from '../pipe-table-syntax.ts'
import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts' import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts'
import { longestOutput, tooLong } from '../../output-length.ts' import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong } from '../length-bound.ts'
import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts'
import { success, type ConvertErrorPath, type Result } from '../../result.ts' import { success, type ConvertErrorPath, type Result } from '../../result.ts'
import { tryPipeCell } from './inline-line.ts' import { tryPipeCell } from './inline-line.ts'
// indent: the characters the containers around the table prefix each of its lines with. export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<string> | undefined {
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, { flavour, indent }: { flavour: WrittenFlavour; indent: number }): Result<string> | undefined {
const rows = pipeRows(node) const rows = pipeRows(node)
if (rows === undefined) return undefined if (rows === undefined) return undefined
const lines: string[] = [] const spelled: string[][] = []
// A row spells ` | ` beside each cell, and a newline and the indent after it. // The cells alone must fit before the rows around them are built.
let projected = 0 let length = 0
for (const [rowIndex, row] of rows.entries()) { for (const [rowIndex, row] of rows.entries()) {
const cells: string[] = [] const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) { for (const [cellIndex, paragraph] of row.entries()) {
const content = nodeContent(paragraph) const content = nodeContent(paragraph)
const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0], flavour) const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0], flavour)
if (line === undefined) return undefined if (line === undefined) return undefined
length += line.length
if (length > longestMarkdown) return tooLong()
cells.push(line) cells.push(line)
projected += line.length + 3
} }
projected += 2 + indent + (rowIndex === 0 ? spellPipeDelimiter(cells.length).length + 1 + indent : 0) spelled.push(cells)
if (projected > longestOutput) return tooLong([...path, 'content', rowIndex]) }
lines.push(spellPipeRow(cells)) const text = joinedWithin(separatedBy('\n', tableRows(spelled)))
if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length)) return boundedThen(text, success)
}
function* tableRows(rows: readonly string[][]): Generator<Iterable<string>> {
for (const [index, cells] of rows.entries()) {
yield pipeRowPieces(cells)
if (index === 0) yield pipeRowPieces(delimiterCells(cells.length))
} }
return success(lines.join('\n'))
} }
function pipeRows(node: AdfNode): AdfNode[][] | undefined { function pipeRows(node: AdfNode): AdfNode[][] | undefined {
+49
View File
@@ -0,0 +1,49 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import type { Result } from '../result.ts'
import { boundedThen, joinedWithin, separatedBy, tooLong, withinEngineLimits } from './length-bound.ts'
import { success } from '../result.ts'
function thrown(name: string, message: string): () => Result<string> {
return () => {
const error = new Error(message)
error.name = name
throw error
}
}
test('joins pieces up to the bound, and builds nothing past it', () => {
assert.equal(joinedWithin(separatedBy(', ', [['a', 'b'], ['c']])), 'ab, c')
const half = 'a'.repeat(2 ** 28)
assert.equal(joinedWithin([half, half]), undefined)
})
test('refuses where a bounded builder stopped, and goes on with what it built', () => {
assert.deepEqual(boundedThen(undefined, success), tooLong())
assert.deepEqual(boundedThen('x', (text) => success(`${text}!`)), success('x!'))
})
test('refuses a string past the bound as too long on every engine, and names any other engine limit', () => {
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Invalid string length')), tooLong())
assert.deepEqual(withinEngineLimits(thrown('InternalError', 'allocation size overflow')), tooLong())
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Out of memory')), {
error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (Out of memory)', path: [] },
ok: false,
})
assert.equal(withinEngineLimits(thrown('RangeError', 'Invalid array length')).ok, false)
assert.deepEqual(
withinEngineLimits(() => success('x')),
{ ok: true, value: 'x' },
)
})
test('lets a stack overflow or any other throw through', () => {
assert.throws(() => withinEngineLimits(thrown('RangeError', 'Maximum call stack size exceeded')))
assert.throws(() => withinEngineLimits(thrown('TypeError', 'a bug')))
assert.throws(() =>
withinEngineLimits(() => {
throw 'a string'
}),
)
})
+52
View File
@@ -0,0 +1,52 @@
import { failure, type Result } from '../result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestMarkdown = 2 ** 29 - 24
// Measured on Firefox 155, Node 24 and Bun 1.4. The first two say a string passed longestMarkdown, so they refuse as tooLong; the others name the engine's own limit.
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
export function tooLong<T>(): Result<T> {
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest string V8 holds`, [])
}
// The pieces joined, or `undefined` once their length passes longestMarkdown, so nothing past it is built.
export function joinedWithin(pieces: Iterable<string>): string | undefined {
const kept: string[] = []
let length = 0
for (const piece of pieces) {
length += piece.length
if (length > longestMarkdown) return undefined
kept.push(piece)
}
return kept.join('')
}
// The refusal where a bounded builder stopped, else what `next` makes of what it built.
export function boundedThen<T, U>(built: T | undefined, next: (built: T) => Result<U>): Result<U> {
return built === undefined ? tooLong() : next(built)
}
export function* separatedBy(separator: string, parts: Iterable<Iterable<string>>): Generator<string> {
let first = true
for (const part of parts) {
if (!first) yield separator
first = false
yield* part
}
}
// A builder this module does not bound can still pass an engine's longest string or array, so this wraps a whole conversion.
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try {
return convert()
} catch (error) {
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
if (!(error instanceof Error)) throw error
const thrown = `${error.name}: ${error.message}`
if (pastLongestString.includes(thrown)) return tooLong()
if (!pastEngineLimit.includes(thrown)) throw error
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
}
}
+6 -5
View File
@@ -4,12 +4,12 @@ import type { JsonSpelling } from '../canonical-json.ts'
import type { JsonValue } from '../json-value.ts' import type { JsonValue } from '../json-value.ts'
import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../result.ts'
import { fencedCodeBlock } from './commonmark/backtick-runs.ts' import { fencePieces } from './commonmark/backtick-runs.ts'
import { infoStringCarries } from './commonmark/grammar.ts' import { infoStringCarries } from './commonmark/grammar.ts'
import { isAdfNode } from '../adf/document.ts' import { isAdfNode } from '../adf/document.ts'
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
import { boundedThen, joinedWithin, longestMarkdown } from './length-bound.ts'
import { largestNesting } from '../nesting.ts' import { largestNesting } from '../nesting.ts'
import { longestOutput, tooLong } from '../output-length.ts'
import { parseJsonText } from '../json-text.ts' import { parseJsonText } from '../json-text.ts'
import { serializeCanonicalJsonWithin, spellsCanonicalJson } from '../canonical-json.ts' import { serializeCanonicalJsonWithin, spellsCanonicalJson } from '../canonical-json.ts'
@@ -25,7 +25,8 @@ export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: numbe
const named = infoStringCarries(type) const named = infoStringCarries(type)
const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth) const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth)
if (!json.ok) return json if (!json.ok) return json
return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) }) const text = joinedWithin(fencePieces(`${carryFencePrefix}${named ? type : ''}`, json.value.json))
return boundedThen(text, (fence) => success({ headroom: json.value.headroom, text: fence }))
} }
export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> { export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> {
@@ -59,8 +60,8 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) { if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
} }
const json = serializeCanonicalJsonWithin(spelled, spelling, longestOutput) const json = serializeCanonicalJsonWithin(spelled, spelling, longestMarkdown)
return json === undefined ? tooLong(path) : success({ headroom, json }) return boundedThen(json, (spelled) => success({ headroom, json: spelled }))
} }
// `type` is the one the fence's info string names, `undefined` for the inline carry. // `type` is the one the fence's info string names, `undefined` for the inline carry.
+1 -1
View File
@@ -458,7 +458,7 @@ test('refuses compact JSON in the carry fence without building its two-space spe
test('refuses a directive whose CommonMark spelling would pass V8\'s longest string', () => { test('refuses a directive whose CommonMark spelling would pass V8\'s longest string', () => {
const markdown = `!adf:blockquote\n${'- '.repeat(490)}a\\\n${'b\\\n'.repeat(560_000)}c\n!adf:/blockquote\n` const markdown = `!adf:blockquote\n${'- '.repeat(490)}a\\\n${'b\\\n'.repeat(560_000)}c\n!adf:/blockquote\n`
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds') assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds')
}) })
test('names the node JSON an opaque carry restores alone', () => { test('names the node JSON an opaque carry restores alone', () => {
+8
View File
@@ -20,6 +20,7 @@ import { readBlockDirectiveNode } from './directive-nodes.ts'
import { spellStringAttribute, unsupportedNodeShape } from '../directive-syntax.ts' import { spellStringAttribute, unsupportedNodeShape } from '../directive-syntax.ts'
import { spellsEmpty } from '../empty-keys.ts' import { spellsEmpty } from '../empty-keys.ts'
import { splitPlace, splitTaskList } from '../portable/task-list.ts' import { splitPlace, splitTaskList } from '../portable/task-list.ts'
import { withinEngineLimits } from '../length-bound.ts'
type Paragraph = Extract<Block, { kind: 'paragraph' }> type Paragraph = Extract<Block, { kind: 'paragraph' }>
@@ -44,6 +45,13 @@ export function portableMarkdownToAdf(markdown: string): Result<AdfDocument, Par
} }
function readDocument(markdown: string, claims: Claims): Result<AdfDocument, ParseError> { function readDocument(markdown: string, claims: Claims): Result<AdfDocument, ParseError> {
return positioned(
withinEngineLimits(() => readMarkdown(markdown, claims)),
{ line: 1, offset: 0 },
)
}
function readMarkdown(markdown: string, claims: Claims): Result<AdfDocument, ParseError> {
if (typeof markdown !== 'string') return positioned(failure('not-a-string', `markdown is a string: found ${typeName(markdown)}`, []), { line: 1, offset: 0 }) if (typeof markdown !== 'string') return positioned(failure('not-a-string', `markdown is a string: found ${typeName(markdown)}`, []), { line: 1, offset: 0 })
const parsed = parseBlocks(markdown, claims) const parsed = parseBlocks(markdown, claims)
const [only] = parsed.blocks const [only] = parsed.blocks
+9 -4
View File
@@ -36,12 +36,17 @@ export function pipeCells(line: string): string[] | undefined {
return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined
} }
export function spellPipeDelimiter(columns: number): string { export function delimiterCells(columns: number): string[] {
return spellPipeRow(Array.from({ length: columns }, () => '---')) return Array.from({ length: columns }, () => '---')
} }
export function spellPipeRow(cells: readonly string[]): string { export function* pipeRowPieces(cells: readonly string[]): Generator<string> {
return `| ${cells.join(' | ')} |` yield '|'
for (const cell of cells) {
yield ' '
yield cell
yield ' |'
}
} }
function splitPipeCells(line: string, start: number): string[] { function splitPipeCells(line: string, start: number): string[] {
@@ -73,13 +73,13 @@ test('refuses what the document guard refuses, and nothing else', () => {
test('refuses a paragraph whose text would pass V8\'s longest string, joining none of it', () => { test('refuses a paragraph whose text would pass V8\'s longest string, joining none of it', () => {
const long = 'a'.repeat(2 ** 28) const long = 'a'.repeat(2 ** 28)
const merged = paragraph(...Array.from({ length: 9 }, () => text(long))) const merged = paragraph(...Array.from({ length: 9 }, () => text(long)))
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /content/0') assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /')
}) })
test('refuses an expand whose title and body together would pass V8\'s longest string', () => { test('refuses an expand whose title and body together would pass V8\'s longest string', () => {
let quote: AdfNode = node('expand', { title: 'a'.repeat(1_000_000) }, { attrs: {}, content: [text('a\n'.repeat(1_108_000))], type: 'codeBlock' }) let quote: AdfNode = node('expand', { title: 'a'.repeat(1_000_000) }, { attrs: {}, content: [text('a\n'.repeat(1_108_000))], type: 'codeBlock' })
for (let level = 0; level < 240; level += 1) quote = { content: [quote], type: 'blockquote' } for (let level = 0; level < 240; level += 1) quote = { content: [quote], type: 'blockquote' }
assert.equal(portableDocument(document(quote)), `unsupported-node-shape at /${Array.from({ length: 241 }, () => 'content/0').join('/')}`) assert.equal(portableDocument(document(quote)), 'unsupported-node-shape at /')
}) })
test('refuses nesting past 500 levels wherever the reduction walks', () => { test('refuses nesting past 500 levels wherever the reduction walks', () => {
@@ -106,6 +106,12 @@ test('refuses nesting past 500 levels wherever the reduction walks', () => {
for (const bottom of bottoms) assert.equal(lowest(bottom), 'unsupported-nesting-depth', bottom.type) for (const bottom of bottoms) assert.equal(lowest(bottom), 'unsupported-nesting-depth', bottom.type)
}) })
test('names a list nested too deep for its spelling at its place in the caller\'s document', () => {
let deep: AdfNode = bulletList(item(said('x')))
for (let level = 0; level < largestNesting; level += 1) deep = bulletList(item(deep))
assert.match(portableDocument(document(said('a'), deep)), /^unsupported-nesting-depth at \/content\/1\/content\/0\/content\/0\//)
})
test('spells a panel as an alert in the GitHub word for its colour', () => { test('spells a panel as an alert in the GitHub word for its colour', () => {
const panel = (panelType: string | undefined): string => const panel = (panelType: string | undefined): string =>
portable(node('panel', panelType === undefined ? {} : { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType }, said('Check it.'))) portable(node('panel', panelType === undefined ? {} : { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType }, said('Check it.')))
@@ -3,7 +3,7 @@ import { adfDocumentFault, nodeAttrs, nodeContent, nodeMarks } from '../../adf/d
import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts' import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts'
import { centeredImage, externalMedia } from '../external-image.ts' import { centeredImage, externalMedia } from '../external-image.ts'
import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts'
import { failure, faulted, success, withinEngineLimits, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { flavourClaims, taskMarker } from './conventions.ts' import { flavourClaims, taskMarker } from './conventions.ts'
import { inlineLeaves, isBlockNodeType, oneLine, portableLink, reduceInline, writableHref } from './inline-reduction.ts' import { inlineLeaves, isBlockNodeType, oneLine, portableLink, reduceInline, writableHref } from './inline-reduction.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
@@ -12,6 +12,7 @@ import { largestNesting } from '../../nesting.ts'
import { linkDefinition } from '../mark-spellings.ts' import { linkDefinition } from '../mark-spellings.ts'
import { splitTaskList } from './task-list.ts' import { splitTaskList } from './task-list.ts'
import { toEditorNormal } from './editor-normal.ts' import { toEditorNormal } from './editor-normal.ts'
import { withinEngineLimits } from '../length-bound.ts'
// depth: the level the node reduced stands at, counted as the emitter counts it. // depth: the level the node reduced stands at, counted as the emitter counts it.
type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath } type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath }
@@ -22,6 +23,9 @@ type PlacedCell = { colspan: number; paragraph: AdfNode; rowspan: number }
type Placed = { index: number; loose: AdfNode[] } | { index: number; loose?: undefined; node: AdfNode } type Placed = { index: number; loose: AdfNode[] } | { index: number; loose?: undefined; node: AdfNode }
// path: the caller's node the block was reduced from.
type Sourced = { block: AdfNode; path: ConvertErrorPath }
const blockReducers: Readonly<Record<string, BlockReducer>> = { const blockReducers: Readonly<Record<string, BlockReducer>> = {
blockCard: paragraphOfNode, blockCard: paragraphOfNode,
blockquote: (node, reduction) => contained({ type: 'blockquote' }, node, reduction), blockquote: (node, reduction) => contained({ type: 'blockquote' }, node, reduction),
@@ -74,13 +78,14 @@ function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result<A
else if (previous?.loose !== undefined) previous.loose.push(node) else if (previous?.loose !== undefined) previous.loose.push(node)
else placed.push({ index, loose: [node] }) else placed.push({ index, loose: [node] })
} }
const blocks = concatenated( const sourced: Sourced[] = []
placed.map((entry) => { for (const entry of placed) {
const at = { ...reduction, path: [...reduction.path, 'content', entry.index] } const at = { ...reduction, path: [...reduction.path, 'content', entry.index] }
return entry.loose === undefined ? reduceNode(entry.node, at) : paragraphOf(entry.loose, reduction) const blocks = entry.loose === undefined ? reduceNode(entry.node, at) : paragraphOf(entry.loose, reduction)
}), if (!blocks.ok) return blocks
) for (const block of blocks.value) sourced.push({ block, path: at.path })
return blocks.ok ? portableSequence(blocks.value, reduction) : blocks }
return portableSequence(sourced, reduction)
} }
function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
@@ -91,7 +96,7 @@ function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
function reduceStanding(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceStanding(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const blocks = standsInline(node) ? paragraphOf([node], reduction) : reduceNode(node, reduction) const blocks = standsInline(node) ? paragraphOf([node], reduction) : reduceNode(node, reduction)
return blocks.ok ? portableSequence(blocks.value, reduction) : blocks return blocks.ok ? portableSequence(blocks.value.map((block) => ({ block, path: reduction.path })), reduction) : blocks
} }
function standsInline(node: AdfNode): boolean { function standsInline(node: AdfNode): boolean {
@@ -117,21 +122,22 @@ function concatenated(results: readonly Result<AdfNode[]>[]): Result<AdfNode[]>
return success(blocks) return success(blocks)
} }
// A list still taking the directive form gives way to its items' blocks; one whose spelling is refused, too deep or too long, refuses the reduction (the maintainer, 2026-10-12). // A list still taking the directive form gives way to its items' blocks; one whose spelling is refused, too deep or too long, refuses the reduction.
function portableSequence(blocks: readonly AdfNode[], reduction: Reduction): Result<AdfNode[]> { function portableSequence(blocks: readonly Sourced[], reduction: Reduction): Result<AdfNode[]> {
const pending = blocks.filter((block) => block.type !== 'paragraph' || nodeContent(block).length > 0).reverse() const pending = blocks.filter(({ block }) => block.type !== 'paragraph' || nodeContent(block).length > 0).reverse()
const sequence: AdfNode[] = [] const sequence: AdfNode[] = []
for (let next = pending.pop(); next !== undefined; next = pending.pop()) { for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
const block = numberedPastMarkers(next) const block = numberedPastMarkers(next.block)
const listed = block.type === 'bulletList' || block.type === 'orderedList' const listed = block.type === 'bulletList' || block.type === 'orderedList'
const spelled = listed && block === next ? commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo }) : undefined const spelled = listed && block === next.block ? commonMarkSpelling(block, next.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo }) : undefined
if (spelled !== undefined && !spelled.ok) return spelled if (spelled !== undefined && !spelled.ok) return spelled
if (!listed || spelled !== undefined) { if (!listed || spelled !== undefined) {
sequence.push(block) sequence.push(block)
continue continue
} }
// Back on `pending`: a replacement may take the directive form itself. // Back on `pending`: a replacement may take the directive form itself.
const replacement = block === next ? nodeContent(block).flatMap(nodeContent) : [block] const { path } = next
const replacement = block === next.block ? nodeContent(block).flatMap((item, index) => nodeContent(item).map((node, offset) => ({ block: node, path: [...path, 'content', index, 'content', offset] }))) : [{ block, path }]
for (const node of replacement.reverse()) pending.push(node) for (const node of replacement.reverse()) pending.push(node)
} }
return success(sequence) return success(sequence)
+3 -4
View File
@@ -2,7 +2,7 @@ import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/doc
import type { JsonValue } from '../../json-value.ts' import type { JsonValue } from '../../json-value.ts'
import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts' import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
import { joinsWhenRead } from '../adjacent-text.ts' import { joinsWhenRead } from '../adjacent-text.ts'
import { longestOutput } from '../../output-length.ts' import { longestMarkdown } from '../length-bound.ts'
type JsonContainer = JsonValue[] | { [key: string]: JsonValue } type JsonContainer = JsonValue[] | { [key: string]: JsonValue }
@@ -11,8 +11,6 @@ type NodeHolder = { content?: AdfNode[] }
// The editor joins two text nodes where the reader would join their editor-normal forms. // The editor joins two text nodes where the reader would join their editor-normal forms.
export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean {
if (previous.type !== 'text' || node.type !== 'text') return false if (previous.type !== 'text' || node.type !== 'text') return false
// Text this long is refused as markdown either way; joining it first would build the string.
if (textLength(previous) + textLength(node) > longestOutput) return false
return joinsWhenRead(normalNode(previous), normalNode(node)) return joinsWhenRead(normalNode(previous), normalNode(node))
} }
@@ -24,7 +22,8 @@ export function toEditorNormal(document: AdfDocument): AdfDocument {
const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version } const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version }
const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }] const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }]
for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) { for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) {
const content = mergeAdjacentText(nodeContent(entry.source), joinsWhenEditorNormal) // Text past the bound is refused as markdown either way; joining it first would build the string.
const content = mergeAdjacentText(nodeContent(entry.source), (previous, node) => joinsWhenEditorNormal(previous, node) && textLength(previous) + textLength(node) <= longestMarkdown)
if (content.length === 0) continue if (content.length === 0) continue
entry.holder.content = content.map((source) => { entry.holder.content = content.map((source) => {
const holder = normalNode(source) const holder = normalNode(source)
+3 -3
View File
@@ -3,10 +3,10 @@ import type { LineContainer } from '../line-container.ts'
import type { MarkRun } from '../emit/line-escaping.ts' import type { MarkRun } from '../emit/line-escaping.ts'
import { blockNodeModel } from '../../adf/block-nodes.ts' import { blockNodeModel } from '../../adf/block-nodes.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { heldTextLength, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts' import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts' import { longestMarkdown, tooLong } from '../length-bound.ts'
import { heldTextLength, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts' import { portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts'
import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
@@ -18,7 +18,7 @@ const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link']
export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> { export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> {
const leaves = inlineLeaves(nodes, container, path, depth) const leaves = inlineLeaves(nodes, container, path, depth)
if (!leaves.ok) return leaves if (!leaves.ok) return leaves
if (heldTextLength(leaves.value) > longestOutput) return tooLong(path) if (heldTextLength(leaves.value) > longestMarkdown) return tooLong()
return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path) return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path)
} }
-8
View File
@@ -1,8 +0,0 @@
import { failure, type ConvertErrorPath, type Result } from './result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestOutput = 2 ** 29 - 24
export function tooLong<T>(path: ConvertErrorPath): Result<T> {
return failure('unsupported-node-shape', `the markdown would be longer than ${longestOutput} characters, the longest string V8 holds`, path)
}
-39
View File
@@ -4,12 +4,6 @@ import { fileURLToPath } from 'node:url'
import { readFileSync, readdirSync } from 'node:fs' import { readFileSync, readdirSync } from 'node:fs'
import test from 'node:test' import test from 'node:test'
import { success, withinEngineLimits, type Result } from './result.ts'
function code(result: Result<string>): string {
return result.ok ? `converted ${result.value}` : result.error.code
}
const sourceRoot = dirname(fileURLToPath(import.meta.url)) const sourceRoot = dirname(fileURLToPath(import.meta.url))
const union = /export type ConvertErrorCode =\n((?:\s+\| '[a-z-]+'\n)+)/ const union = /export type ConvertErrorCode =\n((?:\s+\| '[a-z-]+'\n)+)/
const declared = /'([a-z-]+)'/g const declared = /'([a-z-]+)'/g
@@ -35,36 +29,3 @@ function calledCodes(): string[] {
test('every ConvertErrorCode is the code of a production call site, and every call site names a declared one', () => { test('every ConvertErrorCode is the code of a production call site, and every call site names a declared one', () => {
assert.deepEqual(calledCodes(), declaredCodes()) assert.deepEqual(calledCodes(), declaredCodes())
}) })
test('refuses where the engine runs out of string or array, and lets a stack overflow or any other throw through', () => {
const overflow = new Error('allocation size overflow')
overflow.name = 'InternalError'
assert.deepEqual(
withinEngineLimits(() => {
throw overflow
}),
{ error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (allocation size overflow)', path: [] }, ok: false },
)
assert.equal(
code(
withinEngineLimits(() => {
throw new RangeError('Invalid array length')
}),
),
'unsupported-node-shape',
)
assert.throws(() =>
withinEngineLimits(() => {
throw new RangeError('Maximum call stack size exceeded')
}),
)
assert.throws(() =>
withinEngineLimits(() => {
throw new TypeError('a bug')
}),
)
assert.deepEqual(
withinEngineLimits(() => success('x')),
{ ok: true, value: 'x' },
)
})
-14
View File
@@ -41,17 +41,3 @@ export function positioned<T>(result: Result<T>, position: SourcePosition): Resu
export function success<T>(value: T): { ok: true; value: T } { export function success<T>(value: T): { ok: true; value: T } {
return { ok: true, value } return { ok: true, value }
} }
// Past the longest string or array, measured on Firefox 155, Node 24 and Bun 1.4: SpiderMonkey throws the first, V8 the next two, JavaScriptCore the last.
const engineLimitThrows = ['InternalError: allocation size overflow', 'RangeError: Invalid array length', 'RangeError: Invalid string length', 'RangeError: Out of memory']
// Any string concatenation can throw one, so this wraps a whole conversion.
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try {
return convert()
} catch (error) {
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
if (!(error instanceof Error) || !engineLimitThrows.includes(`${error.name}: ${error.message}`)) throw error
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
}
}
+3 -1
View File
@@ -81,7 +81,9 @@ The line escaper keeps an array element per character twice (`escapings`, `place
50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80 50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a
directive's CommonMark ask. Found while shipping item 106, 2026-10-11. directive's CommonMark ask. Bound the line as the blocks are (`joinedWithin`): its segments and its
escaped output, which today pass V8's longest string only by throwing. Found while shipping item
106, 2026-10-11.
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code. ### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.