3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-12 19:58:28 +01:00

Refuse markdown past V8's longest string on every engine, before building it

This commit is contained in:
2026-10-11 23:26:20 +02:00
parent 7a27f6b81c
commit 25a17e1ed3
19 changed files with 203 additions and 90 deletions
+1
View File
@@ -59,6 +59,7 @@ In `docs/decisions.md`:
- The CommonMark suite checks three ways
- The flavour spec is read as a source
- The node tables answer to Atlassian's schema
- Markdown past V8's longest string is refused on every engine
- Nothing recurses unbounded
- JSON text is read by `parseJsonText`
- No value `JSON.parse` or `structuredClone` builds loops a call
+5 -4
View File
@@ -83,10 +83,11 @@
- Fix a cyclic document, in which an object holds itself: `adfToLosslessMarkdown` and
`adfToPortableMarkdown` refuse it as `not-an-adf-document` and `isAdfDocument` returns `false`,
where all three hung.
- Fix a document whose markdown would be longer than the longest string or array the JavaScript
engine builds, on V8 about 537 million characters, or about 134 million in one text node:
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape`, where
they threw.
- Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
every engine, before building it, where they threw or ran out of memory.
- Fix a directive whose CommonMark spelling would be that long: `losslessMarkdownToAdf` refuses it
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory.
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
+5 -4
View File
@@ -409,9 +409,10 @@ typed on `ConvertError` takes both.
A call reports the first refusal in document order and stops, so a document with several surfaces
them one per call. Every refusal is deterministic on one engine — there is no I/O anywhere — so a
retry returns the identical error: fix the input, or set the document aside. Only the engine-limit
refusal under `unsupported-node-shape` differs between engines; on JavaScriptCore (Bun, Safari),
retry it once, since that engine reports a heap running short the same way.
retry returns the identical error: fix the input, or set the document aside. Only the refusal naming
the longest string or array this JavaScript engine builds differs between engines; on
JavaScriptCore (Bun, Safari), retry it once, since that engine reports a heap running short the
same way.
### Parsing
@@ -444,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do |
| --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than the longest string or array the JavaScript engine builds — on V8, about 537 million characters of markdown, or about 134 million in one text node | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past the engine's longest string or array, keep the ADF and pass the document over, or show it read-only |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters, keep the ADF and pass the document over, or show it read-only |
## The guarantees
+28 -12
View File
@@ -461,11 +461,12 @@ handles one cause alike whichever node, attribute or direction raised it.
knows which direction it called and parting them after `0.1.0` is MAJOR (2026-09-23).
- A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document`
emitting — no document holds one, so no round-trip crosses them (2026-09-23).
- A document whose markdown would be longer than the longest string or array the JavaScript engine
builds is `unsupported-node-shape` at the document's path, in either writer: the document does
not convert, and no spelling in the input causes the bound (§The code list). A carried node
passes it on about 1 MB of deeply nested JSON, since the block carry indents two spaces a level;
`todo.md` item 107 respells the carry in a length linear in its JSON (2026-10-11).
- A document whose markdown would be longer than V8's longest string is `unsupported-node-shape`
at the block that passes it, in either writer, and so is a directive `losslessMarkdownToAdf`
reads whose CommonMark spelling would be: the document does not convert, and no spelling in the
input causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested
JSON, since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in
a length linear in its JSON (2026-10-11).
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
`not-an-adf-document` names the emitters' input (2026-10-05).
@@ -658,6 +659,19 @@ any value round-trips. What the schema holds and the tables do not spell is pinn
attribute as a gap, a type as carried — so a re-pin adding either goes red until someone spells it
or pins it.
## Markdown past V8's longest string is refused on every engine
2026-10-11, the maintainer. Goals 1 and 6. Valid while V8's longest string, 2^29 − 24 characters,
is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestOutput`) before building it, so the same document converts alike everywhere and no
amplification — an indent on every line of a deep subtree, the carry's two-space JSON, text one
string shared by many nodes — builds a string no engine was asked for. Each block counts its lines
once with the indent its containers will prefix them with, so the bound trips at the block that
passes it, before any container copies it. The engines' own throws past their longest string or
array stay refused as `unsupported-node-shape` by `withinEngineLimits`, behind this bound.
## Nothing recurses unbounded
2026-08-25, the directive form's count 2026-09-18, the maintainer. Goal 1. Valid while an engine's
@@ -722,13 +736,15 @@ two cannot disagree — which is what makes the kept value a memo rather than a
2026-09-19, the maintainer. Goal 9. Valid while the `commonMarkSpelling` ask spells a node once
per level above it otherwise.
The parse and the portable reduction keep each node's readable spelling in a memo, so the
`commonMarkSpelling` ask stops spelling a node once per level above it. `text`, `alternateText` and
`spelling` carry no depth and `headroom` is affine in it, so a read at or above the depth that
filled the entry rebases; a read below re-spells, because a hit skips the depth guards the walk it
replaces runs and an ordered list past the marker cap gives way, spending two emitter levels where
the parser spent one. Only what succeeded is kept, so no path minted at another position is ever
read.
The parse and the portable reduction keep the readable spelling of each node they put the
`commonMarkSpelling` ask to in a memo, so the ask stops spelling a node once per level above it. A
walk that reads an entry drops it, and a walk fills none for the nodes below the asked one: the walk
above spells them into its own text, and kept, every level of a deep subtree would hold its whole
text (2026-10-11). `text`, `alternateText` and `spelling` carry no depth and `headroom` is affine in
it, so a read at or above the depth that filled the entry rebases; a read below re-spells, because a
hit skips the depth guards the walk it replaces runs and an ordered list past the marker cap gives
way, spending two emitter levels where the parser spent one. Only what succeeded is kept, so no path
minted at another position is ever read.
## Cost fixes are measured, never timed
+4
View File
@@ -82,6 +82,10 @@ export function identicalMarks(left: readonly AdfMark[], right: readonly AdfMark
return marksKey(left) === marksKey(right)
}
export function textLength(node: AdfNode): number {
return node.text?.length ?? 0
}
export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] {
const merged: AdfNode[] = []
for (const node of nodes) {
+6 -1
View File
@@ -2,7 +2,7 @@ import assert from 'node:assert/strict'
import test from 'node:test'
import type { JsonValue } from './json-value.ts'
import { serializeCanonicalJson, spellsCanonicalJson } from './canonical-json.ts'
import { serializeCanonicalJson, serializeCanonicalJsonWithin, spellsCanonicalJson } from './canonical-json.ts'
test('sorts object keys recursively', () => {
const value = { b: 1, a: { d: 2, c: 3 } }
@@ -62,3 +62,8 @@ test('holds a text to the canonical spelling, whole', () => {
assert.ok(!spellsCanonicalJson(value, 'two-space', '{"a":[1,2]}'))
assert.ok(!spellsCanonicalJson(1e21, 'compact', '1e21'))
})
test('spells canonical JSON up to a length, and nothing past it', () => {
assert.equal(serializeCanonicalJsonWithin({ a: [1, 2] }, 'compact', 11), '{"a":[1,2]}')
assert.equal(serializeCanonicalJsonWithin({ a: [1, 2] }, 'compact', 10), undefined)
})
+12
View File
@@ -15,6 +15,18 @@ export function serializeCanonicalJson(value: JsonValue, spelling: JsonSpelling)
return text.join('')
}
// `undefined` where the spelling runs past `longest` characters, where it stops building.
export function serializeCanonicalJsonWithin(value: JsonValue, spelling: JsonSpelling, longest: number): string | undefined {
const text: string[] = []
let length = 0
const whole = writeCanonicalJson(value, spelling, (piece) => {
length += piece.length
text.push(piece)
return length <= longest
})
return whole ? text.join('') : undefined
}
// Stops at the first piece that does not match `text`: re-spelled with two-space indentation, a short compact text can pass the engine's longest string.
export function spellsCanonicalJson(value: JsonValue, spelling: JsonSpelling, text: string): boolean {
let offset = 0
+28 -5
View File
@@ -44,13 +44,36 @@ test('refuses a value that is not an ADF document, naming the check it failed',
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
})
// About 2.2 billion characters of carry JSON, past V8's, JavaScriptCore's and SpiderMonkey's longest string.
test('refuses a document whose markdown runs past the engine\'s longest string', () => {
let value: JsonValue = new Array<JsonValue>(2_200_000).fill(0)
const tooLong = 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds'
// About 556 million characters of carry JSON: past V8's longest string, short of JavaScriptCore's and SpiderMonkey's.
test('refuses a carry whose JSON would run past V8\'s longest string, on every engine', () => {
let value: JsonValue = new Array<JsonValue>(560_000).fill(0)
for (let level = 0; level < 495; level += 1) value = [value]
const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' }))
assert.match(markdown(result), /^unsupported-node-shape: the output would be longer than the longest string or array this JavaScript engine builds \(/)
assert.deepEqual(path(result), [])
assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
})
test('refuses lines their containers would prefix past V8\'s longest string, at the block holding them', () => {
let quote: AdfNode = { attrs: {}, content: [{ text: 'a\n'.repeat(1_100_000), type: 'text' }], type: 'codeBlock' }
for (let level = 0; level < 245; level += 1) quote = { content: [quote], type: 'blockquote' }
const result = adfToLosslessMarkdown(document(quote))
assert.equal(markdown(result), tooLong)
assert.equal(path(result).length, 2 * 246)
})
test('refuses a directive whose opener and body together would pass V8\'s longest string', () => {
const long = 'a'.repeat(2 ** 28)
const expand: AdfNode = { attrs: { title: long }, content: [{ content: [{ text: long, type: 'text' }], type: 'codeBlock' }], type: 'expand' }
assert.equal(markdown(adfToLosslessMarkdown(document(expand))), tooLong)
})
test('refuses text past V8\'s longest string before spelling it', () => {
const long = 'a'.repeat(2 ** 28)
const result = adfToLosslessMarkdown(document(paragraph({ text: long, type: 'text' }, { text: long, type: 'text' }, { text: long, type: 'text' })))
assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
})
test('refuses a document version the markdown cannot carry', () => {
+37 -14
View File
@@ -12,6 +12,7 @@ import { fencedCodeBlock } from '../commonmark/backtick-runs.ts'
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
import { languageSlot, type LanguageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
import { tryImage } from './image.ts'
@@ -29,9 +30,11 @@ type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { a
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
export type SpellingMemo = Map<AdfNode, KeptSpelling>
type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
export type Writing = { flavour: WrittenFlavour; memo: SpellingMemo | undefined }
// indent: the characters the containers around a block prefix each of its lines with.
export type Writing = { flavour: WrittenFlavour; indent: number; memo: SpellingMemo | undefined }
const bulletDelimiters: Delimiters = ['-', '*']
const bulletWidth = 2
export const largestListMarker = 999999999
// Bare because tryList admits no item carrying attributes, marks or text.
const listItemOpener = spellDirectiveOpener('listItem', undefined, '')
@@ -46,7 +49,7 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
if (fault !== undefined) return faulted(fault, [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
if (document.content === undefined) return success(`${documentSpelling}\n`)
const walk = walkBlocks(document.content, [], 0, { flavour, memo: undefined })
const walk = walkBlocks(document.content, [], 0, { flavour, indent: 0, memo: undefined })
if (!walk.ok) return walk
const text = joinBlocks(walk.value.blocks, 'document')
return success(text === '' ? '' : `${text}\n`)
@@ -56,15 +59,29 @@ function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: nu
let headroom = largestNesting - depth
if (headroom < 0) return tooDeep(path)
const blocks: PlacedBlock[] = []
// Each block's length once its containers prefix its lines, and the two characters parting it from the next.
let projected = 0
for (const [index, node] of nodes.entries()) {
const block = emitBlock(node, [...path, 'content', index], depth, writing)
if (!block.ok) return block
projected += prefixedLength(block.value.text, writing.indent) + 2
if (projected > longestOutput) return tooLong([...path, 'content', index])
headroom = Math.min(headroom, block.value.headroom)
blocks.push({ ...block.value, node })
}
return success({ blocks, headroom })
}
function prefixedLength(text: string, indent: number): number {
let lines = 1
for (let at = text.indexOf('\n'); at !== -1; at = text.indexOf('\n', at + 1)) lines += 1
return text.length + lines * indent
}
function indented(writing: Writing, width: number): Writing {
return { ...writing, indent: writing.indent + width }
}
function tooDeep(path: ConvertErrorPath): Result<never> {
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
}
@@ -107,25 +124,25 @@ function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing
return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing))
}
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<null> | undefined {
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing & { memo: SpellingMemo }): Result<null> | undefined {
const readable = readableBlock(node, path, depth, writing)
if (readable === undefined) writing.memo.set(node, { block: undefined, depth })
else if (readable.ok) writing.memo.set(node, { block: readable.value, depth })
if (readable === undefined) return undefined
if (!readable.ok) return readable
return readable.value.spelling === 'directive' ? undefined : success(null)
}
function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
const { memo } = writing
const kept = memo?.get(node)
const kept = writing.memo?.get(node)
// The walk reading an entry spells the node into its own text, so the entry is dropped: kept, every level of a deep subtree would hold its whole text.
writing.memo?.delete(node)
if (kept !== undefined) {
if (kept.block === undefined) return undefined
// An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards.
if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth })
}
const spelled = spellReadableBlock(node, path, depth, writing)
if (spelled === undefined) memo?.set(node, { block: undefined, depth })
else if (spelled.ok) memo?.set(node, { block: spelled.value, depth })
return spelled
return spellReadableBlock(node, path, depth, writing)
}
function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
@@ -154,12 +171,14 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
}
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner
const body = joinBlocks(inner.value.blocks, 'document')
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
}
const quoteWidth = 2
function quoted(text: string): string {
return text
.split('\n')
@@ -173,7 +192,8 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
// A child other than a task nests in the task before it.
for (const [index, child] of nodeContent(node).entries()) {
const task = isTaskItem(child)
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, writing) : placedBlock(child, [...path, 'content', index], depth + 1, writing)
const itemWriting = indented(writing, bulletWidth)
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, itemWriting) : placedBlock(child, [...path, 'content', index], depth + 1, itemWriting)
if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom)
const previous = items.at(-1)
@@ -248,12 +268,14 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
}
const walk = walkBody()
if (!walk.ok) return walk
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom))
const body = joinBlocks(walk.value.blocks, 'directive')
if (opener.length + body.length > longestOutput) return tooLong(path)
return success(directivePair(node, opener, body, walk.value.headroom))
}
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
if (!holdsOnlyAttributes(node, [])) return undefined
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document')
const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
@@ -313,8 +335,9 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined
const walked: WalkedItem[] = []
let headroom = Number.POSITIVE_INFINITY
const itemWriting = indented(writing, ordered ? `${start + items.length - 1}. `.length : bulletWidth)
for (const [offset, item] of items.entries()) {
const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, writing)
const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, itemWriting)
if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom)
walked.push({ node: item, walk: walk.value })
+33 -26
View File
@@ -10,11 +10,12 @@ import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonma
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts'
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
import { identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts'
import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../commonmark/backtick-runs.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
import { spellTextDirective } from '../text-directive.ts'
@@ -80,6 +81,7 @@ export function tryImageLine(alt: string | undefined, url: string, link?: LinkDe
}
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> {
if (heldTextLength(nodes) > longestOutput) return tooLong(path)
const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false }
// Terminates because takeFallback refuses a pass that took no new fallback.
for (;;) {
@@ -99,6 +101,17 @@ function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: Con
}
}
// The line spells every character of its text at least once, so this much text cannot fit.
export function heldTextLength(nodes: readonly AdfNode[]): number {
let length = 0
const pending = [...nodes]
for (let node = pending.pop(); node !== undefined; node = pending.pop()) {
length += textLength(node)
for (const child of nodeContent(node)) pending.push(child)
}
return length
}
function takeFallback(fallbacks: LineFallbacks, fallback: NodeRange | 'opening-link', path: ConvertErrorPath): Result<null> {
if (fallback === 'opening-link') {
if (fallbacks.openingLinkAsDirective) return failure('unsupported-node-shape', 'an opening link spelled as a directive still reads as a link definition, so the line has no spelling left', path)
@@ -172,13 +185,9 @@ function bracketedLinkTarget(target: string): InlineSegment {
return { escaping: 'bracketed-link-target', text: escapeUnbalanced(target, '[', ']') }
}
function refuseContentAndText(node: AdfNode, path: ConvertErrorPath): Result<null> {
const holdsContent = nodeContent(node).length > 0
if (holdsContent || node.text !== undefined) {
const held = holdsContent ? 'content' : 'text'
return failure('unsupported-node-shape', `a ${node.type} node holds neither content nor text: this one holds ${held}`, path)
}
return success(null)
function heldContentOrText(node: AdfNode): 'content' | 'text' | undefined {
if (nodeContent(node).length > 0) return 'content'
return node.text === undefined ? undefined : 'text'
}
function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, context: InlineContext): Result<Emission> {
@@ -235,40 +244,38 @@ function carries(node: AdfNode, carried: ReadonlySet<number>, index: number): bo
return typeof slot === 'string' && slotFault(node.type, slot) !== undefined && nodeContent(node).length === 0 && node.text === undefined
}
// Builds the node's path only to refuse it: a path copies every level above, and a line may hold a million leaves.
function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result<Emission> {
const path = nodePath(context, index)
if (carries(node, context.carried, index)) {
const carried = carriedInline(node, path)
const carried = carriedInline(node, nodePath(context, index))
if (!carried.ok) return carried
return success({ segments: [syntax(carried.value)] })
}
const types = nodeMarks(node).map((mark) => mark.type)
if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path)
if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, nodePath(context, index))
const model = inlineNodeModel(node.type)
if (model === undefined) return success(emitText(node, context, index))
if (node.type === 'hardBreak') return emitHardBreak(node, model, context, index, path)
return emitInlineDirective(node, model, index, path)
const held = heldContentOrText(node)
if (held !== undefined) return failure('unsupported-node-shape', `a ${node.type} node holds neither content nor text: this one holds ${held}`, nodePath(context, index))
if (node.type === 'hardBreak') return success(emitHardBreak(node, model, context, index))
return success(emitInlineDirective(node, model, index))
}
function emitHardBreak(node: AdfNode, model: InlineNodeModel, context: InlineContext, index: number, path: ConvertErrorPath): Result<Emission> {
const empty = refuseContentAndText(node, path)
if (!empty.ok) return empty
function emitHardBreak(node: AdfNode, model: InlineNodeModel, context: InlineContext, index: number): Emission {
const attributes = spellInlineNodeAttributes(node, model)
if (attributes === undefined) return success({ carry: { first: index, last: index } })
if (attributes === '' && context.spansLines && !context.atBlockEnd) return success({ segments: [syntax('\\\n')] })
return success({ segments: [syntax(spellInlineLeafDirective('hardBreak', attributes))] })
if (attributes === undefined) return { carry: { first: index, last: index } }
if (attributes === '' && context.spansLines && !context.atBlockEnd) return { segments: [syntax('\\\n')] }
return { segments: [syntax(spellInlineLeafDirective('hardBreak', attributes))] }
}
function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: number, path: ConvertErrorPath): Result<Emission> {
const empty = refuseContentAndText(node, path)
if (!empty.ok) return empty
function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: number): Emission {
const attributes = spellInlineNodeAttributes(node, model)
if (attributes === undefined) return success({ carry: { first: index, last: index } })
if (attributes === undefined) return { carry: { first: index, last: index } }
const slot = model.textAttribute === undefined ? undefined : nodeAttrs(node)[model.textAttribute]
if (slot === undefined) return success({ segments: [syntax(spellInlineLeafDirective(node.type, attributes))] })
if (typeof slot !== 'string' || slotFault(node.type, slot) !== undefined) return success({ carry: { first: index, last: index } })
if (slot === undefined) return { segments: [syntax(spellInlineLeafDirective(node.type, attributes))] }
if (typeof slot !== 'string' || slotFault(node.type, slot) !== undefined) return { carry: { first: index, last: index } }
const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }]
return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] })
return { segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }
}
// CommonMark reads a raw carriage return as a line ending and a null character as U+FFFD, so neither is written raw.
+4 -2
View File
@@ -9,8 +9,9 @@ import { infoStringCarries } from './commonmark/grammar.ts'
import { isAdfNode } from '../adf/document.ts'
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
import { largestNesting } from '../nesting.ts'
import { longestOutput, tooLong } from '../output-length.ts'
import { parseJsonText } from '../json-text.ts'
import { serializeCanonicalJson, spellsCanonicalJson } from '../canonical-json.ts'
import { serializeCanonicalJsonWithin, spellsCanonicalJson } from '../canonical-json.ts'
export const carryFencePrefix = 'adf:'
@@ -58,7 +59,8 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
}
return success({ headroom, json: serializeCanonicalJson(spelled, spelling) })
const json = serializeCanonicalJsonWithin(spelled, spelling, longestOutput)
return json === undefined ? tooLong(path) : success({ headroom, json })
}
// `type` is the one the fence's info string names, `undefined` for the inline carry.
+6 -1
View File
@@ -449,13 +449,18 @@ test('names the canonical spelling a carried JSON reads alone', () => {
})
test('refuses compact JSON in the carry fence without building its two-space spelling', () => {
const json = `{"attrs":{"value":${'['.repeat(495)}${new Array(2_200_000).fill(0).join(',')}${']'.repeat(495)}}}`
const json = `{"attrs":{"value":${'['.repeat(495)}${new Array(560_000).fill(0).join(',')}${']'.repeat(495)}}}`
assert.equal(
content(losslessMarkdownToAdf(`\`\`\`adf:unknownBlock\n${json}\n\`\`\`\n`)),
"unsupported-node-shape: the opaque carry spells its node's JSON canonically: two-space indent, keys sorted; !adf:codeBlock {language=\"adf:unknownBlock\"} around a bare fence keeps it a code block",
)
})
test('refuses a directive whose CommonMark spelling would pass V8\'s longest string', () => {
const markdown = `!adf:blockquote\n${'- '.repeat(490)}a\\\n${'b\\\n'.repeat(560_000)}c\n!adf:/blockquote\n`
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds')
})
test('names the node JSON an opaque carry restores alone', () => {
const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; "
assert.equal(content(losslessMarkdownToAdf('```adf:x\n[]\n```\n')), `${node}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block`)
+1 -1
View File
@@ -254,7 +254,7 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr
if (!read.ok) return read
const built = directiveBody(read.value, block.blocks, reading, path, depth)
if (!built.ok) return built
const readable = commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', memo: reading.memo })
const readable = commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', indent: 0, memo: reading.memo })
if (readable === undefined) return built
if (!readable.ok) return readable
return failure('unsupported-node-shape', `${built.value.type} takes the CommonMark spelling, not the directive form`, path)
@@ -70,11 +70,10 @@ test('refuses what the document guard refuses, and nothing else', () => {
assert.match(portableDocument(document(paragraph(deepInline))), /^unsupported-nesting-depth at /)
})
// The merge throws at its concatenation, before any text is scanned, so no engine holds more than the one string.
test('refuses a paragraph whose merged text passes the engine\'s longest string', () => {
test('refuses a paragraph whose text would pass V8\'s longest string, joining none of it', () => {
const long = 'a'.repeat(2 ** 28)
const merged = paragraph(...Array.from({ length: 9 }, () => text(long)))
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /')
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /content/0')
})
test('refuses nesting past 500 levels wherever the reduction walks', () => {
@@ -124,7 +124,7 @@ function portableSequence(blocks: readonly AdfNode[], reduction: Reduction): Res
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
const block = numberedPastMarkers(next)
const listed = block.type === 'bulletList' || block.type === 'orderedList'
if (!listed || (block === next && commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', memo: reduction.memo })?.ok === true)) {
if (!listed || (block === next && commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo })?.ok === true)) {
sequence.push(block)
continue
}
+4 -1
View File
@@ -1,7 +1,8 @@
import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts'
import type { JsonValue } from '../../json-value.ts'
import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
import { joinsWhenRead } from '../adjacent-text.ts'
import { longestOutput } from '../../output-length.ts'
type JsonContainer = JsonValue[] | { [key: string]: JsonValue }
@@ -10,6 +11,8 @@ type NodeHolder = { content?: AdfNode[] }
// The editor joins two text nodes where the reader would join their editor-normal forms.
export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean {
if (previous.type !== 'text' || node.type !== 'text') return false
// Text this long is refused as markdown either way; joining it first would build the string.
if (textLength(previous) + textLength(node) > longestOutput) return false
return joinsWhenRead(normalNode(previous), normalNode(node))
}
+3 -1
View File
@@ -5,8 +5,9 @@ import { blockNodeModel } from '../../adf/block-nodes.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts'
import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts'
import { heldTextLength, portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts'
import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
const highlight = 'backgroundColor'
@@ -17,6 +18,7 @@ const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link']
export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> {
const leaves = inlineLeaves(nodes, container, path, depth)
if (!leaves.ok) return leaves
if (heldTextLength(leaves.value) > longestOutput) return tooLong(path)
return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path)
}
+8
View File
@@ -0,0 +1,8 @@
import { failure, type ConvertErrorPath, type Result } from './result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestOutput = 2 ** 29 - 24
export function tooLong<T>(path: ConvertErrorPath): Result<T> {
return failure('unsupported-node-shape', `the markdown would be longer than ${longestOutput} characters, the longest string V8 holds`, path)
}
+15 -14
View File
@@ -6,7 +6,7 @@
`Bar = 9`
`Next ID = 108`
`Next ID = 110`
| Goal | W |
|---|---|
@@ -25,7 +25,7 @@
| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |
|---|---|---|---|---|---|---|---|---|---|
| 106 | 0.2.0 | defect | **Refuse a document or markdown whose output would pass a stated length before building it, so no call exhausts the heap.** | 4 | 6 | 5 | 8 | 1, 9 | 20.5 |
| 108 | 0.2.0 | defect | **Spell a line in memory linear in its length, so a paragraph shorter than V8's longest string converts within Node's default heap.** | 4 | 6 | 5 | 8 | 1, 9 | 20.5 |
| 103 | 0.2.0 | | **Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.** | 2 | 2 | 5 | 6 | 1 | 19.5 |
| 84 | 0.2.0 | | **Read GFM's pipe table without leading pipes, and with alignment colons, in `portableMarkdownToAdf`, dropping the alignment.** | 3 | 4 | 7 | 7 | 4, 7 | 19.4 |
| 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 6 | 11.5 |
@@ -42,6 +42,7 @@
| 33 | 0.2.1 | defect | **Emit a line in time linear in its mark runs, in `adfToLosslessMarkdown` and `adfToPortableMarkdown`.** | 4 | 5 | 6 | 9 | 9 | 8.7 |
| 98 | 0.2.1 | defect | **Read blank lines inside deeply nested list items in time linear in the markdown.** | 3 | 3 | 5 | 9 | 9 | 8.2 |
| 102 | 0.2.1 | decision | **Place a block a portable task list refuses at its path in the returned document without replaying `splitTaskList` on stand-in nodes.** | 3 | 4 | 2 | 4 | 1 | 8.0 |
| 109 | 0.2.1 | defect | **Spell nested lists and quotes in time linear in the markdown.** | 3 | 5 | 4 | 7 | 9 | 5.3 |
| 7 | 0.3.0 | decision | **Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.** | 6 | 9 | 9 | 9 | 2, 4 | 25.8 |
| 45 | 0.3.0 | | **Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`.** | 2 | 3 | 6 | 8 | 1 | 25.2 |
| 6 | 0.3.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 4 | 24.7 |
@@ -73,19 +74,13 @@
## Details
### 106. Refuse a document or markdown whose output would pass a stated length before building it, so no call exhausts the heap.
### 108. Spell a line in memory linear in its length, so a paragraph shorter than V8's longest string converts within Node's default heap.
The writers return `unsupported-node-shape` where the engine throws past its longest string, but
some documents exhaust the heap first, which no catch survives. `losslessMarkdownToAdf` on 2.1 MB of
markdown, a `!adf:blockquote` holding a list nested 490 levels whose paragraph runs on in 700,000
lazy lines `b\`, crashed Node 24 out of heap: `directiveNode` re-spells the subtree, indenting each
line to its depth. `adfToLosslessMarkdown` on nine text nodes sharing one 2^28-character string got
Bun OOM-killed under 8 GB: `escape` in `line-escaping.ts` pushes one array element per character.
Where the heap holds out, the readers' re-spelling can still throw a `RangeError`. JavaScriptCore's
`Out of memory`, which the writers refuse, may also mean a failed allocation that a retry survives.
The candidate length to refuse past is V8's 2^29 − 24 on every engine, so the same document converts
alike everywhere; no entry states it yet (AGENTS.md §Stated numbers). Found by the principles audit
of item 83, 2026-10-11.
The line escaper keeps an array element per character twice (`escapings`, `placements` in `escape`,
`line-escaping.ts`) and a `Set` entry per escaped character, so `adfToLosslessMarkdown` holds about
50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
exceeded`, which `withinEngineLimits` lets through. Found while shipping item 106, 2026-10-11.
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.
@@ -211,6 +206,12 @@ the fix.
`taskOfParagraphs` keeps the original child objects. Both paired-ruling seats on 2026-10-11 named it
the reason the project read worse.
### 109. Spell nested lists and quotes in time linear in the markdown.
Each level of a nested list or quote re-indents its whole text, so `losslessMarkdownToAdf` on 3 MB
of lazy lines in a list nested 490 levels inside `!adf:blockquote` spends about 5 seconds
re-spelling it, the writers the same on its document. Found while shipping item 106, 2026-10-11.
### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.
Lands after items 6, 59, 60 and 62. Which flavours get a composed call is this item's to settle;