3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-12 11:48:29 +01:00

Answer architecture round 2: every builder joins its pieces through one bounded join, the readers take the engine guard, every refusal stands at the document's path, and a portable list's refusal names its own place

This commit is contained in:
2026-10-12 17:42:24 +02:00
committed by Lillem4n
parent 2b5d22cbc6
commit 3414c8690a
25 changed files with 379 additions and 249 deletions
+4 -2
View File
@@ -85,9 +85,11 @@
where all three hung.
- Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
every engine, before building it, where they threw or ran out of memory.
every engine, where they threw or ran out of memory.
- Fix a directive whose CommonMark spelling would be that long: `losslessMarkdownToAdf` refuses it
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory.
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory. The
markdown readers return `unsupported-node-shape` where they threw past the longest string or
array the JavaScript engine builds.
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
+3 -3
View File
@@ -212,9 +212,9 @@ Each call's name says which markdown it reads or writes:
| --- | --- | --- |
| `adfToLosslessMarkdown` | writes the lossless flavour: CommonMark plus GFM's pipe tables and `~~`, with `!adf:` directives and `adf:` fences for everything else | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` |
| `adfToPortableMarkdown` | writes markdown GitHub, GitLab and Obsidian render; drops what they cannot show, and writes no directive | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` |
| `commonMarkToAdf` | reads strict CommonMark as its spec says: a directive, a pipe table, `~~`, `==` and an `adf:` fence read as text and code | `not-a-string`, `unmappable-html`, `unsupported-nesting-depth` |
| `commonMarkToAdf` | reads strict CommonMark as its spec says: a directive, a pipe table, `~~`, `==` and an `adf:` fence read as text and code | `not-a-string`, `unmappable-html`, `unsupported-nesting-depth`, `unsupported-node-shape` |
| `losslessMarkdownToAdf` | reads the lossless flavour, markdown `adfToLosslessMarkdown` wrote, edited or not | every code in [The errors](#the-errors) except `not-an-adf-document` and `unsupported-document-version` |
| `portableMarkdownToAdf` | reads CommonMark, GFM's pipe tables and `~~`, alerts and task markers, Obsidian's callouts and `==`; a directive and an `adf:` fence read as text and code | `malformed-pipe-table`, `not-a-string`, `unmappable-html`, `unsupported-nesting-depth` |
| `portableMarkdownToAdf` | reads CommonMark, GFM's pipe tables and `~~`, alerts and task markers, Obsidian's callouts and `==`; a directive and an `adf:` fence read as text and code | `malformed-pipe-table`, `not-a-string`, `unmappable-html`, `unsupported-nesting-depth`, `unsupported-node-shape` |
`isAdfDocument` is the type guard for the boundary you receive JSON at. It checks the shape the
converters require: `type: 'doc'`, a finite `version` (the converters take only 1), nodes holding
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do |
| --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters, keep the ADF and pass the document over, or show it read-only |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only |
## The guarantees
+17 -14
View File
@@ -462,11 +462,11 @@ handles one cause alike whichever node, attribute or direction raised it.
- A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document`
emitting — no document holds one, so no round-trip crosses them (2026-09-23).
- A document whose markdown would be longer than V8's longest string is `unsupported-node-shape`
at the block that passes it, in either writer, and so is a directive `losslessMarkdownToAdf`
reads whose CommonMark spelling would be: the document does not convert, and no spelling in the
input causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested
JSON, since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in
a length linear in its JSON (2026-10-11).
at the document's path, in either writer, and so is a directive `losslessMarkdownToAdf` reads
whose CommonMark spelling would be: the document does not convert, and no spelling in the input
causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON,
since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a
length linear in its JSON (2026-10-11).
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
`not-an-adf-document` names the emitters' input (2026-10-05).
@@ -665,15 +665,18 @@ or pins it.
is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestOutput`) before building it, so the same document converts alike everywhere and no
amplification — an indent on every line of a deep subtree, the carry's two-space JSON, text one
string shared by many nodes — builds a string no engine was asked for. Each walk of blocks projects
their length with the indent their containers will prefix every line with, and a container joining
several walks — a list's items, a table's rows — sums their projections before it joins them, so the
bound trips at the block that passes it, before any container copies it. The projection never
undercounts: a list giving way to the directive form counts its items' marker indent, so markdown a
few characters a line under the bound may be refused. The engines' own throws past their longest
string or array stay refused as `unsupported-node-shape` by `withinEngineLimits`, behind this bound.
(`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
converts alike everywhere. Each builder that joins pieces — blocks, quoted and list lines, directive
pairs and openers, fences, pipe rows, the carry's JSON — joins them through `joinedWithin`, which
stops once they pass the bound, so none builds a string no engine was asked for. A walk of blocks
also forecasts each block's length with the indent its containers will prefix every line with, so a
deep subtree refuses before any container copies it.
A line is held to its text's length alone; what its spelling adds — escapes, link targets,
attributes — passes V8's longest string only by throwing, and `withinEngineLimits` refuses that
throw, and SpiderMonkey's, as the same refusal: both mean a string past the bound. `todo.md` item
108 bounds the line itself. The engines' throws past their own longest array, and JavaScriptCore's
`Out of memory`, which also reports a heap running short, stay refused with the engine's message.
## Nothing recurses unbounded
+1 -1
View File
@@ -86,7 +86,7 @@ export function textLength(node: AdfNode): number {
return node.text?.length ?? 0
}
// A line spells every character of its nodes' text at least once, so this much text cannot fit.
// The length of the text the nodes hold, the nodes below them included.
export function heldTextLength(nodes: readonly AdfNode[]): number {
let length = 0
const pending = [...nodes]
+10 -5
View File
@@ -16,16 +16,21 @@ export function closingBacktickRun(text: string, from: number, opener: number):
}
export function fencedCodeBlock(info: string, body: string): string {
return fencePieces(info, body).join('')
}
export function fencePieces(info: string, body: string): string[] {
const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1))
return body === '' ? `${fence}${info}\n${fence}` : `${fence}${info}\n${body}\n${fence}`
return body === '' ? [fence, info, '\n', fence] : [fence, info, '\n', body, '\n', fence]
}
export function longestBacktickRun(text: string): number {
let longest = 0
let current = 0
for (const character of text) {
current = character === '`' ? current + 1 : 0
longest = Math.max(longest, current)
for (let start = text.indexOf('`'); start !== -1; ) {
let end = start + 1
while (text.charAt(end) === '`') end += 1
longest = Math.max(longest, end - start)
start = text.indexOf('`', end)
}
return longest
}
+24 -4
View File
@@ -112,9 +112,18 @@ export function slotFault(type: string, text: string): ConvertFault | undefined
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`)
return `{${spelled.join(' ')}}`
return [...attributePieces(pairs)].join('')
}
export function* attributePieces(pairs: readonly (readonly [string, string])[]): Generator<string> {
if (pairs.length === 0) return
yield '{'
for (const [index, [key, value]] of [...pairs].sort(([left], [right]) => keyOrder(left, right)).entries()) {
if (index > 0) yield ' '
yield `${key}=`
yield value
}
yield '}'
}
export function spellDirectiveCloser(name: string): string {
@@ -122,7 +131,18 @@ export function spellDirectiveCloser(name: string): string {
}
export function spellDirectiveOpener(name: string, argument: string | undefined, attributes: string): string {
return `${directivePrefix}${name}${argument === undefined ? '' : ` ${argument}`}${attributes === '' ? '' : ` ${attributes}`}`
return [...directiveOpenerPieces(name, argument, attributes === '' ? [] : [attributes])].join('')
}
export function* directiveOpenerPieces(name: string, argument: string | undefined, attributes: Iterable<string>): Generator<string> {
yield `${directivePrefix}${name}`
if (argument !== undefined) yield ` ${argument}`
let first = true
for (const piece of attributes) {
if (first) yield ' '
first = false
yield piece
}
}
export function spellJsonAttribute(value: JsonValue): string {
+13 -6
View File
@@ -44,7 +44,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
})
const tooLong = 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds'
const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds'
// About 556 million characters of carry JSON: past V8's longest string, short of JavaScriptCore's and SpiderMonkey's.
test('refuses a carry whose JSON would run past V8\'s longest string, on every engine', () => {
@@ -52,15 +52,13 @@ test('refuses a carry whose JSON would run past V8\'s longest string, on every e
for (let level = 0; level < 495; level += 1) value = [value]
const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' }))
assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
})
test('refuses lines their containers would prefix past V8\'s longest string, at the block holding them', () => {
test('refuses lines their containers would prefix past V8\'s longest string, before any container copies them', () => {
let quote: AdfNode = { attrs: {}, content: [{ text: 'a\n'.repeat(1_100_000), type: 'text' }], type: 'codeBlock' }
for (let level = 0; level < 245; level += 1) quote = { content: [quote], type: 'blockquote' }
const result = adfToLosslessMarkdown(document(quote))
assert.equal(markdown(result), tooLong)
assert.equal(path(result).length, 2 * 246)
})
test('refuses a directive whose opener and body together would pass V8\'s longest string', () => {
@@ -73,7 +71,6 @@ test('refuses a list whose items together would pass V8\'s longest string, befor
const item: AdfNode = { content: [{ attrs: {}, content: [{ text: 'a'.repeat(2 ** 28), type: 'text' }], type: 'codeBlock' }], type: 'listItem' }
const result = adfToLosslessMarkdown(document({ content: [item, item], type: 'bulletList' }))
assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0, 'content', 1])
})
test('refuses a pipe table whose rows their containers would prefix past V8\'s longest string', () => {
@@ -84,11 +81,21 @@ test('refuses a pipe table whose rows their containers would prefix past V8\'s l
assert.equal(markdown(adfToLosslessMarkdown(document(quote))), tooLong)
})
test('refuses a code block whose fences together would pass V8\'s longest string', () => {
const long: AdfNode = { text: 'a'.repeat(2 ** 28), type: 'text' }
assert.equal(markdown(adfToLosslessMarkdown(document({ attrs: {}, content: [long, long, long], type: 'codeBlock' }))), tooLong)
})
test('refuses a directive opener whose attributes together would pass V8\'s longest string', () => {
const long = 'a'.repeat(2 ** 28)
const panel: AdfNode = { attrs: { panelColor: long, panelIcon: long, panelIconText: long, panelType: 'custom' }, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' }
assert.equal(markdown(adfToLosslessMarkdown(document(panel))), tooLong)
})
test('refuses text past V8\'s longest string before spelling it', () => {
const long = 'a'.repeat(2 ** 28)
const result = adfToLosslessMarkdown(document(paragraph({ text: long, type: 'text' }, { text: long, type: 'text' }, { text: long, type: 'text' })))
assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
})
test('refuses a document version the markdown cannot carry', () => {
+119 -100
View File
@@ -7,12 +7,12 @@ import { blockDirectiveForm, documentSpelling } from '../block-directive.ts'
import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts'
import { carriedBlock } from '../opaque-carry.ts'
import { emitInlineLine } from './inline-line.ts'
import { failure, faulted, success, withinEngineLimits, type ConvertErrorPath, type Result } from '../../result.ts'
import { fencedCodeBlock } from '../commonmark/backtick-runs.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { fencePieces } from '../commonmark/backtick-runs.ts'
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong, withinEngineLimits } from '../length-bound.ts'
import { languageSlot, type LanguageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
import { tryImage } from './image.ts'
@@ -24,8 +24,7 @@ type Delimiters = readonly [string, string]
type EmittedBlock = SpelledBlock & { headroom: number }
type KeptSpelling = { block: EmittedBlock | undefined; depth: number }
type PlacedBlock = SpelledBlock & { node: AdfNode }
// projected: the blocks' length once joined and their containers prefix their lines; it never undercounts.
type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number; projected: number }
type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number }
// CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`.
type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string }
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
@@ -35,12 +34,12 @@ type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
export type Writing = { flavour: WrittenFlavour; indent: number; memo: SpellingMemo | undefined }
const bulletDelimiters: Delimiters = ['-', '*']
const bulletWidth = 2
const bulletWidth = '- '.length
export const largestListMarker = 999999999
// Bare because tryList admits no item carrying attributes, marks or text.
const listItemOpener = spellDirectiveOpener('listItem', undefined, '')
const listItemCloser = spellDirectiveCloser('listItem')
const orderedDelimiters: Delimiters = ['.', ')']
const quoteWidth = '> '.length
export function adfToLosslessMarkdown(document: AdfDocument): Result<string> {
return withinEngineLimits(() => writeMarkdown(document, 'lossless'))
@@ -53,29 +52,25 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
if (document.content === undefined) return success(`${documentSpelling}\n`)
const walk = walkBlocks(document.content, [], 0, { flavour, indent: 0, memo: undefined })
if (!walk.ok) return walk
const text = joinBlocks(walk.value.blocks, 'document')
return success(text === '' ? '' : `${text}\n`)
const text = joinedWithin(walk.value.blocks.length === 0 ? [] : [...blockPieces(walk.value.blocks, 'document'), '\n'])
return boundedThen(text, success)
}
function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
let headroom = largestNesting - depth
if (headroom < 0) return tooDeep(path)
const blocks: PlacedBlock[] = []
let projected = 0
// A forecast, so a deep subtree refuses where its containers would prefix its lines past the bound, before any of them copies those lines.
let forecast = 0
for (const [index, node] of nodes.entries()) {
const block = emitBlock(node, [...path, 'content', index], depth, writing)
if (!block.ok) return block
projected += (index === 0 ? 0 : separatorLength(writing)) + prefixedLength(block.value.text, writing.indent) + 1
if (projected > longestOutput) return tooLong([...path, 'content', index])
forecast += prefixedLength(block.value.text, writing.indent)
if (forecast > longestMarkdown) return tooLong()
headroom = Math.min(headroom, block.value.headroom)
blocks.push({ ...block.value, node })
}
return success({ blocks, headroom, projected })
}
// A block counts the newline ending it; a blank line between two is prefixed as their lines are.
function separatorLength(writing: Writing): number {
return 1 + writing.indent
return success({ blocks, headroom })
}
function prefixedLength(text: string, indent: number): number {
@@ -92,16 +87,18 @@ function tooDeep(path: ConvertErrorPath): Result<never> {
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
}
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string {
let text = ''
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string | undefined {
return joinedWithin(blockPieces(blocks, container))
}
function* blockPieces(blocks: readonly PlacedBlock[], container: BlockContainer): Generator<string> {
let alternated = false
for (const [index, block] of blocks.entries()) {
const previous = blocks[index - 1]
if (previous !== undefined) text += separationBetween(previous, block, container)
if (previous !== undefined) yield separationBetween(previous, block, container)
alternated = !alternated && previous !== undefined && sharesMarker(previous, block)
text += alternated && block.spelling === 'list' ? block.alternateText : block.text
yield alternated && block.spelling === 'list' ? block.alternateText : block.text
}
return text
}
// A task list is spelled as a bullet list.
@@ -156,13 +153,13 @@ function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
if (portable !== undefined) return portable
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
if (node.type === 'codeBlock') return readableText(tryCodeBlock(node, writing.flavour))
if (node.type === 'codeBlock') return tryCodeBlock(node, writing.flavour)
if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
if (node.type === 'mediaSingle') return readableText(tryImage(node))
if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
if (node.type === 'rule') return readableText(tryRule(node))
if (node.type === 'table') {
const table = tryPipeTable(node, path, writing)
const table = tryPipeTable(node, path, writing.flavour)
return table === undefined || !table.ok ? table : success(commonMarkText(table.value))
}
return undefined
@@ -180,27 +177,31 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
}
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const quotedWriting = indented(writing, quoteWidth)
const inner = walkBlocks(nodeContent(node), path, depth + 1, quotedWriting)
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner
if (prefixedLength(head, quotedWriting.indent) + 1 + separatorLength(quotedWriting) + inner.value.projected > longestOutput) return tooLong(path)
const body = joinBlocks(inner.value.blocks, 'document')
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
const text = joinedWithin(inner.value.blocks.length === 0 ? [head] : [head, '\n\n', ...blockPieces(inner.value.blocks, 'document')])
return quotedBlock(text, inner.value.headroom)
}
const quoteWidth = 2
function quotedBlock(text: string | undefined, headroom: number): Result<EmittedBlock> {
return boundedThen(text, (whole) => boundedThen(joinedWithin(quotedPieces(whole)), (quotedText) => success(commonMarkText(quotedText, headroom))))
}
function quoted(text: string): string {
return text
.split('\n')
.map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n')
function* quotedPieces(text: string): Generator<string> {
for (let start = 0; ; ) {
const end = text.indexOf('\n', start)
const line = text.slice(start, end === -1 ? text.length : end)
yield line === '' ? '>' : '> '
yield line
if (end === -1) return
yield '\n'
start = end + 1
}
}
function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const items: PlacedBlock[][] = []
// Each walk alone stayed within the bound, so the join is checked before it builds.
let projected = 0
let headroom = largestNesting - depth - 1
// A child other than a task nests in the task before it.
for (const [index, child] of nodeContent(node).entries()) {
@@ -208,26 +209,31 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
const itemWriting = indented(writing, bulletWidth)
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, itemWriting) : placedBlock(child, [...path, 'content', index], depth + 1, itemWriting)
if (!walk.ok) return walk
projected += (index === 0 ? 0 : separatorLength(writing)) + walk.value.projected
if (projected > longestOutput) return tooLong([...path, 'content', index])
headroom = Math.min(headroom, walk.value.headroom)
const previous = items.at(-1)
if (task || previous === undefined) items.push([...walk.value.blocks])
else for (const block of walk.value.blocks) previous.push(block)
}
const list = spelledList(
items.map((blocks) => joinBlocks(blocks, 'list-item')),
() => '',
bulletDelimiters,
headroom,
)
// The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour.
return list === undefined ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success(list)
return boundedThen(joinedItems(items.map((blocks) => blockPieces(blocks, 'list-item'))), (inners) => spelledList(inners, () => '', bulletDelimiters, headroom) ?? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path))
}
// Each item's text, or `undefined` once all of them together pass the bound; each alone stayed within the forecast.
function joinedItems(items: readonly Iterable<string>[]): string[] | undefined {
const inners: string[] = []
let length = 0
for (const pieces of items) {
const inner = [...pieces].join('')
length += inner.length
if (length > longestMarkdown) return undefined
inners.push(inner)
}
return inners
}
function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
const block = emitBlock(node, path, depth, writing)
return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom, projected: prefixedLength(block.value.text, writing.indent) + 1 }) : block
return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom }) : block
}
// The marker leads the first paragraph, or stands as one where the blocks open with another.
@@ -238,15 +244,15 @@ function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writin
const content = nodeContent(task)
const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour)
if (!line.ok) return line
const text = line.value === '' ? marker : `${marker} ${line.value}`
return success({ blocks: [{ ...markerBlock, text }], headroom: Number.POSITIVE_INFINITY, projected: prefixedLength(text, writing.indent) + 1 })
const text = joinedWithin(line.value === '' ? [marker] : [marker, ' ', line.value])
return boundedThen(text, (spelled) => success({ blocks: [{ ...markerBlock, text: spelled }], headroom: Number.POSITIVE_INFINITY }))
}
const walk = walkBlocks(nodeContent(task), path, depth, writing)
if (!walk.ok) return walk
const [first, ...rest] = walk.value.blocks
const blocks = first?.node.type === 'paragraph' ? [{ ...first, text: `${marker} ${first.text}` }, ...rest] : [markerBlock, ...walk.value.blocks]
// The marker either leads the first line or stands as a block of its own.
return success({ blocks, headroom: walk.value.headroom, projected: walk.value.projected + marker.length + separatorLength(writing) + writing.indent + 1 })
if (first?.node.type !== 'paragraph') return success({ blocks: [markerBlock, ...walk.value.blocks], headroom: walk.value.headroom })
const text = joinedWithin([marker, ' ', first.text])
return boundedThen(text, (spelled) => success({ blocks: [{ ...first, text: spelled }, ...rest], headroom: walk.value.headroom }))
}
function readableText(text: string | undefined): Result<EmittedBlock> | undefined {
@@ -262,8 +268,10 @@ function commonMarkText(text: string, headroom: number = Number.POSITIVE_INFINIT
return { headroom, spelling: 'commonmark', text }
}
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): EmittedBlock {
return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` }
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): Result<EmittedBlock> {
const closer = spellDirectiveCloser(node.type)
const text = joinedWithin(body === '' ? [opener, '\n', closer] : [opener, '\n', body, '\n', closer])
return boundedThen(text, (spelled) => success({ headroom, spelling: 'directive', text: spelled }))
}
function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> {
@@ -281,12 +289,12 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
if (model.contentModel === 'inline') {
const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless')
if (!line.ok) return line
return success(directivePair(node, opener, line.value))
return directivePair(node, opener, line.value)
}
const walk = walkBody()
if (!walk.ok) return walk
if (opener.length + walk.value.projected > longestOutput) return tooLong(path)
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom))
const body = joinBlocks(walk.value.blocks, 'directive')
return boundedThen(body, (spelled) => directivePair(node, opener, spelled, walk.value.headroom))
}
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
@@ -294,17 +302,18 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document')
const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom))
const alert = text !== undefined && flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
return quotedBlock(alert ? joinedWithin(['\\', text]) : text, inner.value.headroom)
}
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): string | undefined {
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
if (!holdsOnlyAttributes(node, ['language'])) return undefined
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour])
if (slot.kind === 'attribute') return undefined
const [only, ...others] = fencedTexts(node) ?? []
if (only === undefined || others.length > 0) return undefined
return fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only)
const text = joinedWithin(fencePieces(slot.kind === 'fence' ? slot.info : '', only))
return boundedThen(text, (spelled) => success(commonMarkText(spelled)))
}
// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does.
@@ -316,7 +325,8 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr
if (opener === undefined) return carriedFence(carriedBlock(node, path, depth))
if (!opener.ok) return opener
const info = slot.kind === 'fence' ? slot.info : ''
return success(directivePair(node, opener.value, texts.map((text) => fencedCodeBlock(info, text)).join('\n')))
const body = joinedWithin(separatedBy('\n', texts.map((text) => fencePieces(info, text))))
return boundedThen(body, (spelled) => directivePair(node, opener.value, spelled))
}
// The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node.
@@ -350,8 +360,6 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
if (start === undefined || items.length === 0) return undefined
if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined
const walked: WalkedItem[] = []
// Each walk alone stayed within the bound, so the join is checked before it builds; the directive form adds an opener and a closer line to each item.
let projected = 0
let headroom = Number.POSITIVE_INFINITY
const itemWriting = indented(writing, ordered ? `${start + items.length - 1}. `.length : bulletWidth)
for (const [offset, item] of items.entries()) {
@@ -359,43 +367,45 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom)
walked.push({ node: item, walk: walk.value })
projected += (offset === 0 ? 0 : separatorLength(writing)) + walk.value.projected + directiveItemLength(writing)
if (projected > longestOutput) return tooLong([...path, 'content', offset])
}
const inners = walked.map((item) => {
const inner = joinBlocks(item.walk.blocks, 'list-item')
return boundedThen(joinedItems(walked.map((item) => blockPieces(item.walk.blocks, 'list-item'))), (joined) => {
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
return flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner
const inners = joined.map((inner) => (flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner))
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
if (list !== undefined) return list
// The directive form spends a level the walk did not count.
if (headroom < 1) return tooDeep(path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => directiveItems(walked, headroom - 1))
})
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
if (list !== undefined) return success(list)
// The directive form spends a level the walk did not count.
if (headroom < 1) return tooDeep(path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1, projected }))
}
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): EmittedBlock | undefined {
const text = listLines(inners, number, usual)
const alternateText = listLines(inners, number, alternate)
return text === undefined || alternateText === undefined ? undefined : { alternateText, headroom, spelling: 'list', text }
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): Result<EmittedBlock> | undefined {
const pieces = listLines(inners, number, usual)
const alternatePieces = listLines(inners, number, alternate)
if (pieces === undefined || alternatePieces === undefined) return undefined
// The alternate delimiter is as wide as the usual one, so its text is as long.
return boundedThen(joinedWithin(pieces), (text) => success({ alternateText: [...alternatePieces].join(''), headroom, spelling: 'list', text }))
}
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): string | undefined {
const lines: string[] = []
// `undefined` where an item holds a line no list item spells.
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): Iterable<string> | undefined {
const items: Iterable<string>[] = []
for (const [offset, inner] of inners.entries()) {
const line = tryListItemLines(inner, `${number(offset)}${delimiter} `)
if (line === undefined) return undefined
lines.push(line)
const pieces = listItemPieces(inner, `${number(offset)}${delimiter} `)
if (pieces === undefined) return undefined
items.push(pieces)
}
return lines.join('\n')
return separatedBy('\n', items)
}
function directiveItemLength(writing: Writing): number {
return listItemOpener.length + listItemCloser.length + 2 * (1 + writing.indent)
}
function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] {
return items.map((item) => ({ ...directivePair(item.node, listItemOpener, joinBlocks(item.walk.blocks, 'directive')), node: item.node }))
function directiveItems(items: readonly WalkedItem[], headroom: number): Result<PlacedBlocks> {
const blocks: PlacedBlock[] = []
for (const item of items) {
const pair = boundedThen(joinBlocks(item.walk.blocks, 'directive'), (body) => directivePair(item.node, listItemOpener, body))
if (!pair.ok) return pair
blocks.push({ ...pair.value, node: item.node })
}
return success({ blocks, headroom })
}
function listStart(node: AdfNode, items: number): number | undefined {
@@ -405,16 +415,25 @@ function listStart(node: AdfNode, items: number): number | undefined {
return start + items - 1 > largestListMarker ? undefined : start
}
function tryListItemLines(inner: string, marker: string): string | undefined {
if (inner === '') return marker.trimEnd()
const body = inner.split('\n')
if (body.some((line) => line !== '' && isBlankLine(line))) return undefined
const indent = ' '.repeat(marker.length)
const lines = body.map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
const first = lines[0] ?? ''
if (!isThematicBreak(first)) return lines.join('\n')
function listItemPieces(inner: string, marker: string): Iterable<string> | undefined {
if (inner === '') return [marker.trimEnd()]
const [first = '', ...rest] = inner.split('\n')
if (rest.some((line) => line !== '' && isBlankLine(line)) || (first !== '' && isBlankLine(first))) return undefined
// An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text.
return inner.startsWith('*') ? [`${marker}\\${first.slice(marker.length)}`, ...lines.slice(1)].join('\n') : undefined
const rule = isThematicBreak(`${marker}${first}`)
if (rule && !inner.startsWith('*')) return undefined
return itemLinePieces(rule ? `${marker}\\` : marker, first, rest, ' '.repeat(marker.length))
}
function* itemLinePieces(opening: string, first: string, rest: readonly string[], indent: string): Generator<string> {
yield opening
yield first
for (const line of rest) {
yield '\n'
if (line === '') continue
yield indent
yield line
}
}
function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
@@ -3,7 +3,8 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts'
import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts'
import { blockArgument, markValues, marksAttribute } from '../block-directive.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { joinedWithin, tooLong } from '../length-bound.ts'
import { attributePieces, directiveOpenerPieces, isBareToken, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { overNested } from '../../json-value.ts'
import { spellEmptyKeys } from '../empty-keys.ts'
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
@@ -22,7 +23,8 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel,
if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path)
spelledPairs.push([marksAttribute, spellJsonAttribute(values)])
}
return success(spellDirectiveOpener(node.type, slot.argument, spellAttributes(spelledPairs)))
const opener = joinedWithin(directiveOpenerPieces(node.type, slot.argument, attributePieces(spelledPairs)))
return opener === undefined ? tooLong() : success(opener)
}
// `undefined` where the argument slot holds a value no bare token spells.
+3 -3
View File
@@ -9,13 +9,13 @@ import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../
import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts'
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
import { heldTextLength, identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts'
import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../commonmark/backtick-runs.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { longestMarkdown, tooLong } from '../length-bound.ts'
import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
import { spellTextDirective } from '../text-directive.ts'
@@ -81,7 +81,7 @@ export function tryImageLine(alt: string | undefined, url: string, link?: LinkDe
}
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> {
if (heldTextLength(nodes) > longestOutput) return tooLong(path)
if (heldTextLength(nodes) > longestMarkdown) return tooLong()
const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false }
// Terminates because takeFallback refuses a pass that took no new fallback.
for (;;) {
+18 -13
View File
@@ -1,33 +1,38 @@
import type { AdfNode } from '../../adf/document.ts'
import type { WrittenFlavour } from '../portable/conventions.ts'
import { delimiterCells, pipeRowPieces } from '../pipe-table-syntax.ts'
import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts'
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong } from '../length-bound.ts'
import { success, type ConvertErrorPath, type Result } from '../../result.ts'
import { tryPipeCell } from './inline-line.ts'
// indent: the characters the containers around the table prefix each of its lines with.
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, { flavour, indent }: { flavour: WrittenFlavour; indent: number }): Result<string> | undefined {
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<string> | undefined {
const rows = pipeRows(node)
if (rows === undefined) return undefined
const lines: string[] = []
// A row spells ` | ` beside each cell, and a newline and the indent after it.
let projected = 0
const spelled: string[][] = []
// The cells alone must fit before the rows around them are built.
let length = 0
for (const [rowIndex, row] of rows.entries()) {
const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) {
const content = nodeContent(paragraph)
const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0], flavour)
if (line === undefined) return undefined
length += line.length
if (length > longestMarkdown) return tooLong()
cells.push(line)
projected += line.length + 3
}
projected += 2 + indent + (rowIndex === 0 ? spellPipeDelimiter(cells.length).length + 1 + indent : 0)
if (projected > longestOutput) return tooLong([...path, 'content', rowIndex])
lines.push(spellPipeRow(cells))
if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length))
spelled.push(cells)
}
const text = joinedWithin(separatedBy('\n', tableRows(spelled)))
return boundedThen(text, success)
}
function* tableRows(rows: readonly string[][]): Generator<Iterable<string>> {
for (const [index, cells] of rows.entries()) {
yield pipeRowPieces(cells)
if (index === 0) yield pipeRowPieces(delimiterCells(cells.length))
}
return success(lines.join('\n'))
}
function pipeRows(node: AdfNode): AdfNode[][] | undefined {
+49
View File
@@ -0,0 +1,49 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import type { Result } from '../result.ts'
import { boundedThen, joinedWithin, separatedBy, tooLong, withinEngineLimits } from './length-bound.ts'
import { success } from '../result.ts'
function thrown(name: string, message: string): () => Result<string> {
return () => {
const error = new Error(message)
error.name = name
throw error
}
}
test('joins pieces up to the bound, and builds nothing past it', () => {
assert.equal(joinedWithin(separatedBy(', ', [['a', 'b'], ['c']])), 'ab, c')
const half = 'a'.repeat(2 ** 28)
assert.equal(joinedWithin([half, half]), undefined)
})
test('refuses where a bounded builder stopped, and goes on with what it built', () => {
assert.deepEqual(boundedThen(undefined, success), tooLong())
assert.deepEqual(boundedThen('x', (text) => success(`${text}!`)), success('x!'))
})
test('refuses a string past the bound as too long on every engine, and names any other engine limit', () => {
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Invalid string length')), tooLong())
assert.deepEqual(withinEngineLimits(thrown('InternalError', 'allocation size overflow')), tooLong())
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Out of memory')), {
error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (Out of memory)', path: [] },
ok: false,
})
assert.equal(withinEngineLimits(thrown('RangeError', 'Invalid array length')).ok, false)
assert.deepEqual(
withinEngineLimits(() => success('x')),
{ ok: true, value: 'x' },
)
})
test('lets a stack overflow or any other throw through', () => {
assert.throws(() => withinEngineLimits(thrown('RangeError', 'Maximum call stack size exceeded')))
assert.throws(() => withinEngineLimits(thrown('TypeError', 'a bug')))
assert.throws(() =>
withinEngineLimits(() => {
throw 'a string'
}),
)
})
+52
View File
@@ -0,0 +1,52 @@
import { failure, type Result } from '../result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestMarkdown = 2 ** 29 - 24
// Measured on Firefox 155, Node 24 and Bun 1.4. The first two say a string passed longestMarkdown, so they refuse as tooLong; the others name the engine's own limit.
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
export function tooLong<T>(): Result<T> {
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest string V8 holds`, [])
}
// The pieces joined, or `undefined` once their length passes longestMarkdown, so nothing past it is built.
export function joinedWithin(pieces: Iterable<string>): string | undefined {
const kept: string[] = []
let length = 0
for (const piece of pieces) {
length += piece.length
if (length > longestMarkdown) return undefined
kept.push(piece)
}
return kept.join('')
}
// The refusal where a bounded builder stopped, else what `next` makes of what it built.
export function boundedThen<T, U>(built: T | undefined, next: (built: T) => Result<U>): Result<U> {
return built === undefined ? tooLong() : next(built)
}
export function* separatedBy(separator: string, parts: Iterable<Iterable<string>>): Generator<string> {
let first = true
for (const part of parts) {
if (!first) yield separator
first = false
yield* part
}
}
// A builder this module does not bound can still pass an engine's longest string or array, so this wraps a whole conversion.
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try {
return convert()
} catch (error) {
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
if (!(error instanceof Error)) throw error
const thrown = `${error.name}: ${error.message}`
if (pastLongestString.includes(thrown)) return tooLong()
if (!pastEngineLimit.includes(thrown)) throw error
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
}
}
+6 -5
View File
@@ -4,12 +4,12 @@ import type { JsonSpelling } from '../canonical-json.ts'
import type { JsonValue } from '../json-value.ts'
import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../result.ts'
import { fencedCodeBlock } from './commonmark/backtick-runs.ts'
import { fencePieces } from './commonmark/backtick-runs.ts'
import { infoStringCarries } from './commonmark/grammar.ts'
import { isAdfNode } from '../adf/document.ts'
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
import { boundedThen, joinedWithin, longestMarkdown } from './length-bound.ts'
import { largestNesting } from '../nesting.ts'
import { longestOutput, tooLong } from '../output-length.ts'
import { parseJsonText } from '../json-text.ts'
import { serializeCanonicalJsonWithin, spellsCanonicalJson } from '../canonical-json.ts'
@@ -25,7 +25,8 @@ export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: numbe
const named = infoStringCarries(type)
const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth)
if (!json.ok) return json
return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) })
const text = joinedWithin(fencePieces(`${carryFencePrefix}${named ? type : ''}`, json.value.json))
return boundedThen(text, (fence) => success({ headroom: json.value.headroom, text: fence }))
}
export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> {
@@ -59,8 +60,8 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
}
const json = serializeCanonicalJsonWithin(spelled, spelling, longestOutput)
return json === undefined ? tooLong(path) : success({ headroom, json })
const json = serializeCanonicalJsonWithin(spelled, spelling, longestMarkdown)
return boundedThen(json, (spelled) => success({ headroom, json: spelled }))
}
// `type` is the one the fence's info string names, `undefined` for the inline carry.
+1 -1
View File
@@ -458,7 +458,7 @@ test('refuses compact JSON in the carry fence without building its two-space spe
test('refuses a directive whose CommonMark spelling would pass V8\'s longest string', () => {
const markdown = `!adf:blockquote\n${'- '.repeat(490)}a\\\n${'b\\\n'.repeat(560_000)}c\n!adf:/blockquote\n`
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds')
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds')
})
test('names the node JSON an opaque carry restores alone', () => {
+8
View File
@@ -20,6 +20,7 @@ import { readBlockDirectiveNode } from './directive-nodes.ts'
import { spellStringAttribute, unsupportedNodeShape } from '../directive-syntax.ts'
import { spellsEmpty } from '../empty-keys.ts'
import { splitPlace, splitTaskList } from '../portable/task-list.ts'
import { withinEngineLimits } from '../length-bound.ts'
type Paragraph = Extract<Block, { kind: 'paragraph' }>
@@ -44,6 +45,13 @@ export function portableMarkdownToAdf(markdown: string): Result<AdfDocument, Par
}
function readDocument(markdown: string, claims: Claims): Result<AdfDocument, ParseError> {
return positioned(
withinEngineLimits(() => readMarkdown(markdown, claims)),
{ line: 1, offset: 0 },
)
}
function readMarkdown(markdown: string, claims: Claims): Result<AdfDocument, ParseError> {
if (typeof markdown !== 'string') return positioned(failure('not-a-string', `markdown is a string: found ${typeName(markdown)}`, []), { line: 1, offset: 0 })
const parsed = parseBlocks(markdown, claims)
const [only] = parsed.blocks
+9 -4
View File
@@ -36,12 +36,17 @@ export function pipeCells(line: string): string[] | undefined {
return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined
}
export function spellPipeDelimiter(columns: number): string {
return spellPipeRow(Array.from({ length: columns }, () => '---'))
export function delimiterCells(columns: number): string[] {
return Array.from({ length: columns }, () => '---')
}
export function spellPipeRow(cells: readonly string[]): string {
return `| ${cells.join(' | ')} |`
export function* pipeRowPieces(cells: readonly string[]): Generator<string> {
yield '|'
for (const cell of cells) {
yield ' '
yield cell
yield ' |'
}
}
function splitPipeCells(line: string, start: number): string[] {
@@ -73,13 +73,13 @@ test('refuses what the document guard refuses, and nothing else', () => {
test('refuses a paragraph whose text would pass V8\'s longest string, joining none of it', () => {
const long = 'a'.repeat(2 ** 28)
const merged = paragraph(...Array.from({ length: 9 }, () => text(long)))
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /content/0')
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /')
})
test('refuses an expand whose title and body together would pass V8\'s longest string', () => {
let quote: AdfNode = node('expand', { title: 'a'.repeat(1_000_000) }, { attrs: {}, content: [text('a\n'.repeat(1_108_000))], type: 'codeBlock' })
for (let level = 0; level < 240; level += 1) quote = { content: [quote], type: 'blockquote' }
assert.equal(portableDocument(document(quote)), `unsupported-node-shape at /${Array.from({ length: 241 }, () => 'content/0').join('/')}`)
assert.equal(portableDocument(document(quote)), 'unsupported-node-shape at /')
})
test('refuses nesting past 500 levels wherever the reduction walks', () => {
@@ -106,6 +106,12 @@ test('refuses nesting past 500 levels wherever the reduction walks', () => {
for (const bottom of bottoms) assert.equal(lowest(bottom), 'unsupported-nesting-depth', bottom.type)
})
test('names a list nested too deep for its spelling at its place in the caller\'s document', () => {
let deep: AdfNode = bulletList(item(said('x')))
for (let level = 0; level < largestNesting; level += 1) deep = bulletList(item(deep))
assert.match(portableDocument(document(said('a'), deep)), /^unsupported-nesting-depth at \/content\/1\/content\/0\/content\/0\//)
})
test('spells a panel as an alert in the GitHub word for its colour', () => {
const panel = (panelType: string | undefined): string =>
portable(node('panel', panelType === undefined ? {} : { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType }, said('Check it.')))
@@ -3,7 +3,7 @@ import { adfDocumentFault, nodeAttrs, nodeContent, nodeMarks } from '../../adf/d
import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts'
import { centeredImage, externalMedia } from '../external-image.ts'
import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts'
import { failure, faulted, success, withinEngineLimits, type ConvertErrorPath, type Result } from '../../result.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { flavourClaims, taskMarker } from './conventions.ts'
import { inlineLeaves, isBlockNodeType, oneLine, portableLink, reduceInline, writableHref } from './inline-reduction.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts'
@@ -12,6 +12,7 @@ import { largestNesting } from '../../nesting.ts'
import { linkDefinition } from '../mark-spellings.ts'
import { splitTaskList } from './task-list.ts'
import { toEditorNormal } from './editor-normal.ts'
import { withinEngineLimits } from '../length-bound.ts'
// depth: the level the node reduced stands at, counted as the emitter counts it.
type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath }
@@ -22,6 +23,9 @@ type PlacedCell = { colspan: number; paragraph: AdfNode; rowspan: number }
type Placed = { index: number; loose: AdfNode[] } | { index: number; loose?: undefined; node: AdfNode }
// path: the caller's node the block was reduced from.
type Sourced = { block: AdfNode; path: ConvertErrorPath }
const blockReducers: Readonly<Record<string, BlockReducer>> = {
blockCard: paragraphOfNode,
blockquote: (node, reduction) => contained({ type: 'blockquote' }, node, reduction),
@@ -74,13 +78,14 @@ function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result<A
else if (previous?.loose !== undefined) previous.loose.push(node)
else placed.push({ index, loose: [node] })
}
const blocks = concatenated(
placed.map((entry) => {
const at = { ...reduction, path: [...reduction.path, 'content', entry.index] }
return entry.loose === undefined ? reduceNode(entry.node, at) : paragraphOf(entry.loose, reduction)
}),
)
return blocks.ok ? portableSequence(blocks.value, reduction) : blocks
const sourced: Sourced[] = []
for (const entry of placed) {
const at = { ...reduction, path: [...reduction.path, 'content', entry.index] }
const blocks = entry.loose === undefined ? reduceNode(entry.node, at) : paragraphOf(entry.loose, reduction)
if (!blocks.ok) return blocks
for (const block of blocks.value) sourced.push({ block, path: at.path })
}
return portableSequence(sourced, reduction)
}
function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
@@ -91,7 +96,7 @@ function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
function reduceStanding(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const blocks = standsInline(node) ? paragraphOf([node], reduction) : reduceNode(node, reduction)
return blocks.ok ? portableSequence(blocks.value, reduction) : blocks
return blocks.ok ? portableSequence(blocks.value.map((block) => ({ block, path: reduction.path })), reduction) : blocks
}
function standsInline(node: AdfNode): boolean {
@@ -117,21 +122,22 @@ function concatenated(results: readonly Result<AdfNode[]>[]): Result<AdfNode[]>
return success(blocks)
}
// A list still taking the directive form gives way to its items' blocks; one whose spelling is refused, too deep or too long, refuses the reduction (the maintainer, 2026-10-12).
function portableSequence(blocks: readonly AdfNode[], reduction: Reduction): Result<AdfNode[]> {
const pending = blocks.filter((block) => block.type !== 'paragraph' || nodeContent(block).length > 0).reverse()
// A list still taking the directive form gives way to its items' blocks; one whose spelling is refused, too deep or too long, refuses the reduction.
function portableSequence(blocks: readonly Sourced[], reduction: Reduction): Result<AdfNode[]> {
const pending = blocks.filter(({ block }) => block.type !== 'paragraph' || nodeContent(block).length > 0).reverse()
const sequence: AdfNode[] = []
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
const block = numberedPastMarkers(next)
const block = numberedPastMarkers(next.block)
const listed = block.type === 'bulletList' || block.type === 'orderedList'
const spelled = listed && block === next ? commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo }) : undefined
const spelled = listed && block === next.block ? commonMarkSpelling(block, next.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo }) : undefined
if (spelled !== undefined && !spelled.ok) return spelled
if (!listed || spelled !== undefined) {
sequence.push(block)
continue
}
// Back on `pending`: a replacement may take the directive form itself.
const replacement = block === next ? nodeContent(block).flatMap(nodeContent) : [block]
const { path } = next
const replacement = block === next.block ? nodeContent(block).flatMap((item, index) => nodeContent(item).map((node, offset) => ({ block: node, path: [...path, 'content', index, 'content', offset] }))) : [{ block, path }]
for (const node of replacement.reverse()) pending.push(node)
}
return success(sequence)
+3 -4
View File
@@ -2,7 +2,7 @@ import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/doc
import type { JsonValue } from '../../json-value.ts'
import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
import { joinsWhenRead } from '../adjacent-text.ts'
import { longestOutput } from '../../output-length.ts'
import { longestMarkdown } from '../length-bound.ts'
type JsonContainer = JsonValue[] | { [key: string]: JsonValue }
@@ -11,8 +11,6 @@ type NodeHolder = { content?: AdfNode[] }
// The editor joins two text nodes where the reader would join their editor-normal forms.
export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean {
if (previous.type !== 'text' || node.type !== 'text') return false
// Text this long is refused as markdown either way; joining it first would build the string.
if (textLength(previous) + textLength(node) > longestOutput) return false
return joinsWhenRead(normalNode(previous), normalNode(node))
}
@@ -24,7 +22,8 @@ export function toEditorNormal(document: AdfDocument): AdfDocument {
const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version }
const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }]
for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) {
const content = mergeAdjacentText(nodeContent(entry.source), joinsWhenEditorNormal)
// Text past the bound is refused as markdown either way; joining it first would build the string.
const content = mergeAdjacentText(nodeContent(entry.source), (previous, node) => joinsWhenEditorNormal(previous, node) && textLength(previous) + textLength(node) <= longestMarkdown)
if (content.length === 0) continue
entry.holder.content = content.map((source) => {
const holder = normalNode(source)
+3 -3
View File
@@ -3,10 +3,10 @@ import type { LineContainer } from '../line-container.ts'
import type { MarkRun } from '../emit/line-escaping.ts'
import { blockNodeModel } from '../../adf/block-nodes.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { heldTextLength, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts'
import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { heldTextLength, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { longestMarkdown, tooLong } from '../length-bound.ts'
import { portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts'
import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
@@ -18,7 +18,7 @@ const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link']
export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> {
const leaves = inlineLeaves(nodes, container, path, depth)
if (!leaves.ok) return leaves
if (heldTextLength(leaves.value) > longestOutput) return tooLong(path)
if (heldTextLength(leaves.value) > longestMarkdown) return tooLong()
return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path)
}
-8
View File
@@ -1,8 +0,0 @@
import { failure, type ConvertErrorPath, type Result } from './result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestOutput = 2 ** 29 - 24
export function tooLong<T>(path: ConvertErrorPath): Result<T> {
return failure('unsupported-node-shape', `the markdown would be longer than ${longestOutput} characters, the longest string V8 holds`, path)
}
-39
View File
@@ -4,12 +4,6 @@ import { fileURLToPath } from 'node:url'
import { readFileSync, readdirSync } from 'node:fs'
import test from 'node:test'
import { success, withinEngineLimits, type Result } from './result.ts'
function code(result: Result<string>): string {
return result.ok ? `converted ${result.value}` : result.error.code
}
const sourceRoot = dirname(fileURLToPath(import.meta.url))
const union = /export type ConvertErrorCode =\n((?:\s+\| '[a-z-]+'\n)+)/
const declared = /'([a-z-]+)'/g
@@ -35,36 +29,3 @@ function calledCodes(): string[] {
test('every ConvertErrorCode is the code of a production call site, and every call site names a declared one', () => {
assert.deepEqual(calledCodes(), declaredCodes())
})
test('refuses where the engine runs out of string or array, and lets a stack overflow or any other throw through', () => {
const overflow = new Error('allocation size overflow')
overflow.name = 'InternalError'
assert.deepEqual(
withinEngineLimits(() => {
throw overflow
}),
{ error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (allocation size overflow)', path: [] }, ok: false },
)
assert.equal(
code(
withinEngineLimits(() => {
throw new RangeError('Invalid array length')
}),
),
'unsupported-node-shape',
)
assert.throws(() =>
withinEngineLimits(() => {
throw new RangeError('Maximum call stack size exceeded')
}),
)
assert.throws(() =>
withinEngineLimits(() => {
throw new TypeError('a bug')
}),
)
assert.deepEqual(
withinEngineLimits(() => success('x')),
{ ok: true, value: 'x' },
)
})
-14
View File
@@ -41,17 +41,3 @@ export function positioned<T>(result: Result<T>, position: SourcePosition): Resu
export function success<T>(value: T): { ok: true; value: T } {
return { ok: true, value }
}
// Past the longest string or array, measured on Firefox 155, Node 24 and Bun 1.4: SpiderMonkey throws the first, V8 the next two, JavaScriptCore the last.
const engineLimitThrows = ['InternalError: allocation size overflow', 'RangeError: Invalid array length', 'RangeError: Invalid string length', 'RangeError: Out of memory']
// Any string concatenation can throw one, so this wraps a whole conversion.
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try {
return convert()
} catch (error) {
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
if (!(error instanceof Error) || !engineLimitThrows.includes(`${error.name}: ${error.message}`)) throw error
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
}
}
+3 -1
View File
@@ -81,7 +81,9 @@ The line escaper keeps an array element per character twice (`escapings`, `place
50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a
directive's CommonMark ask. Found while shipping item 106, 2026-10-11.
directive's CommonMark ask. Bound the line as the blocks are (`joinedWithin`): its segments and its
escaped output, which today pass V8's longest string only by throwing. Found while shipping item
106, 2026-10-11.
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.