3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-20 15:15:44 +01:00

Refuse markdown past V8's longest string on every engine, before building it

This commit is contained in:
2026-10-11 23:26:20 +02:00
parent 7a27f6b81c
commit 25a17e1ed3
19 changed files with 203 additions and 90 deletions
+1
View File
@@ -59,6 +59,7 @@ In `docs/decisions.md`:
- The CommonMark suite checks three ways - The CommonMark suite checks three ways
- The flavour spec is read as a source - The flavour spec is read as a source
- The node tables answer to Atlassian's schema - The node tables answer to Atlassian's schema
- Markdown past V8's longest string is refused on every engine
- Nothing recurses unbounded - Nothing recurses unbounded
- JSON text is read by `parseJsonText` - JSON text is read by `parseJsonText`
- No value `JSON.parse` or `structuredClone` builds loops a call - No value `JSON.parse` or `structuredClone` builds loops a call
+5 -4
View File
@@ -83,10 +83,11 @@
- Fix a cyclic document, in which an object holds itself: `adfToLosslessMarkdown` and - Fix a cyclic document, in which an object holds itself: `adfToLosslessMarkdown` and
`adfToPortableMarkdown` refuse it as `not-an-adf-document` and `isAdfDocument` returns `false`, `adfToPortableMarkdown` refuse it as `not-an-adf-document` and `isAdfDocument` returns `false`,
where all three hung. where all three hung.
- Fix a document whose markdown would be longer than the longest string or array the JavaScript - Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
engine builds, on V8 about 537 million characters, or about 134 million in one text node: `adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape`, where every engine, before building it, where they threw or ran out of memory.
they threw. - Fix a directive whose CommonMark spelling would be that long: `losslessMarkdownToAdf` refuses it
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory.
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as - Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw. `unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from - Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
+5 -4
View File
@@ -409,9 +409,10 @@ typed on `ConvertError` takes both.
A call reports the first refusal in document order and stops, so a document with several surfaces A call reports the first refusal in document order and stops, so a document with several surfaces
them one per call. Every refusal is deterministic on one engine — there is no I/O anywhere — so a them one per call. Every refusal is deterministic on one engine — there is no I/O anywhere — so a
retry returns the identical error: fix the input, or set the document aside. Only the engine-limit retry returns the identical error: fix the input, or set the document aside. Only the refusal naming
refusal under `unsupported-node-shape` differs between engines; on JavaScriptCore (Bun, Safari), the longest string or array this JavaScript engine builds differs between engines; on
retry it once, since that engine reports a heap running short the same way. JavaScriptCore (Bun, Safari), retry it once, since that engine reports a heap running short the
same way.
### Parsing ### Parsing
@@ -444,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do | | Code | Fires when | What you can do |
| --- | --- | --- | | --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than the longest string or array the JavaScript engine builds — on V8, about 537 million characters of markdown, or about 134 million in one text node | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past the engine's longest string or array, keep the ADF and pass the document over, or show it read-only | | `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters, keep the ADF and pass the document over, or show it read-only |
## The guarantees ## The guarantees
+28 -12
View File
@@ -461,11 +461,12 @@ handles one cause alike whichever node, attribute or direction raised it.
knows which direction it called and parting them after `0.1.0` is MAJOR (2026-09-23). knows which direction it called and parting them after `0.1.0` is MAJOR (2026-09-23).
- A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document` - A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document`
emitting — no document holds one, so no round-trip crosses them (2026-09-23). emitting — no document holds one, so no round-trip crosses them (2026-09-23).
- A document whose markdown would be longer than the longest string or array the JavaScript engine - A document whose markdown would be longer than V8's longest string is `unsupported-node-shape`
builds is `unsupported-node-shape` at the document's path, in either writer: the document does at the block that passes it, in either writer, and so is a directive `losslessMarkdownToAdf`
not convert, and no spelling in the input causes the bound (§The code list). A carried node reads whose CommonMark spelling would be: the document does not convert, and no spelling in the
passes it on about 1 MB of deeply nested JSON, since the block carry indents two spaces a level; input causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested
`todo.md` item 107 respells the carry in a length linear in its JSON (2026-10-11). JSON, since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in
a length linear in its JSON (2026-10-11).
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at - The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
`not-an-adf-document` names the emitters' input (2026-10-05). `not-an-adf-document` names the emitters' input (2026-10-05).
@@ -658,6 +659,19 @@ any value round-trips. What the schema holds and the tables do not spell is pinn
attribute as a gap, a type as carried — so a re-pin adding either goes red until someone spells it attribute as a gap, a type as carried — so a re-pin adding either goes red until someone spells it
or pins it. or pins it.
## Markdown past V8's longest string is refused on every engine
2026-10-11, the maintainer. Goals 1 and 6. Valid while V8's longest string, 2^29 − 24 characters,
is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestOutput`) before building it, so the same document converts alike everywhere and no
amplification — an indent on every line of a deep subtree, the carry's two-space JSON, text one
string shared by many nodes — builds a string no engine was asked for. Each block counts its lines
once with the indent its containers will prefix them with, so the bound trips at the block that
passes it, before any container copies it. The engines' own throws past their longest string or
array stay refused as `unsupported-node-shape` by `withinEngineLimits`, behind this bound.
## Nothing recurses unbounded ## Nothing recurses unbounded
2026-08-25, the directive form's count 2026-09-18, the maintainer. Goal 1. Valid while an engine's 2026-08-25, the directive form's count 2026-09-18, the maintainer. Goal 1. Valid while an engine's
@@ -722,13 +736,15 @@ two cannot disagree — which is what makes the kept value a memo rather than a
2026-09-19, the maintainer. Goal 9. Valid while the `commonMarkSpelling` ask spells a node once 2026-09-19, the maintainer. Goal 9. Valid while the `commonMarkSpelling` ask spells a node once
per level above it otherwise. per level above it otherwise.
The parse and the portable reduction keep each node's readable spelling in a memo, so the The parse and the portable reduction keep the readable spelling of each node they put the
`commonMarkSpelling` ask stops spelling a node once per level above it. `text`, `alternateText` and `commonMarkSpelling` ask to in a memo, so the ask stops spelling a node once per level above it. A
`spelling` carry no depth and `headroom` is affine in it, so a read at or above the depth that walk that reads an entry drops it, and a walk fills none for the nodes below the asked one: the walk
filled the entry rebases; a read below re-spells, because a hit skips the depth guards the walk it above spells them into its own text, and kept, every level of a deep subtree would hold its whole
replaces runs and an ordered list past the marker cap gives way, spending two emitter levels where text (2026-10-11). `text`, `alternateText` and `spelling` carry no depth and `headroom` is affine in
the parser spent one. Only what succeeded is kept, so no path minted at another position is ever it, so a read at or above the depth that filled the entry rebases; a read below re-spells, because a
read. hit skips the depth guards the walk it replaces runs and an ordered list past the marker cap gives
way, spending two emitter levels where the parser spent one. Only what succeeded is kept, so no path
minted at another position is ever read.
## Cost fixes are measured, never timed ## Cost fixes are measured, never timed
+4
View File
@@ -82,6 +82,10 @@ export function identicalMarks(left: readonly AdfMark[], right: readonly AdfMark
return marksKey(left) === marksKey(right) return marksKey(left) === marksKey(right)
} }
export function textLength(node: AdfNode): number {
return node.text?.length ?? 0
}
export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] { export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] {
const merged: AdfNode[] = [] const merged: AdfNode[] = []
for (const node of nodes) { for (const node of nodes) {
+6 -1
View File
@@ -2,7 +2,7 @@ import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { JsonValue } from './json-value.ts' import type { JsonValue } from './json-value.ts'
import { serializeCanonicalJson, spellsCanonicalJson } from './canonical-json.ts' import { serializeCanonicalJson, serializeCanonicalJsonWithin, spellsCanonicalJson } from './canonical-json.ts'
test('sorts object keys recursively', () => { test('sorts object keys recursively', () => {
const value = { b: 1, a: { d: 2, c: 3 } } const value = { b: 1, a: { d: 2, c: 3 } }
@@ -62,3 +62,8 @@ test('holds a text to the canonical spelling, whole', () => {
assert.ok(!spellsCanonicalJson(value, 'two-space', '{"a":[1,2]}')) assert.ok(!spellsCanonicalJson(value, 'two-space', '{"a":[1,2]}'))
assert.ok(!spellsCanonicalJson(1e21, 'compact', '1e21')) assert.ok(!spellsCanonicalJson(1e21, 'compact', '1e21'))
}) })
test('spells canonical JSON up to a length, and nothing past it', () => {
assert.equal(serializeCanonicalJsonWithin({ a: [1, 2] }, 'compact', 11), '{"a":[1,2]}')
assert.equal(serializeCanonicalJsonWithin({ a: [1, 2] }, 'compact', 10), undefined)
})
+12
View File
@@ -15,6 +15,18 @@ export function serializeCanonicalJson(value: JsonValue, spelling: JsonSpelling)
return text.join('') return text.join('')
} }
// `undefined` where the spelling runs past `longest` characters, where it stops building.
export function serializeCanonicalJsonWithin(value: JsonValue, spelling: JsonSpelling, longest: number): string | undefined {
const text: string[] = []
let length = 0
const whole = writeCanonicalJson(value, spelling, (piece) => {
length += piece.length
text.push(piece)
return length <= longest
})
return whole ? text.join('') : undefined
}
// Stops at the first piece that does not match `text`: re-spelled with two-space indentation, a short compact text can pass the engine's longest string. // Stops at the first piece that does not match `text`: re-spelled with two-space indentation, a short compact text can pass the engine's longest string.
export function spellsCanonicalJson(value: JsonValue, spelling: JsonSpelling, text: string): boolean { export function spellsCanonicalJson(value: JsonValue, spelling: JsonSpelling, text: string): boolean {
let offset = 0 let offset = 0
+28 -5
View File
@@ -44,13 +44,36 @@ test('refuses a value that is not an ADF document, naming the check it failed',
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN') assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
}) })
// About 2.2 billion characters of carry JSON, past V8's, JavaScriptCore's and SpiderMonkey's longest string. const tooLong = 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds'
test('refuses a document whose markdown runs past the engine\'s longest string', () => {
let value: JsonValue = new Array<JsonValue>(2_200_000).fill(0) // About 556 million characters of carry JSON: past V8's longest string, short of JavaScriptCore's and SpiderMonkey's.
test('refuses a carry whose JSON would run past V8\'s longest string, on every engine', () => {
let value: JsonValue = new Array<JsonValue>(560_000).fill(0)
for (let level = 0; level < 495; level += 1) value = [value] for (let level = 0; level < 495; level += 1) value = [value]
const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' })) const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' }))
assert.match(markdown(result), /^unsupported-node-shape: the output would be longer than the longest string or array this JavaScript engine builds \(/) assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), []) assert.deepEqual(path(result), ['content', 0])
})
test('refuses lines their containers would prefix past V8\'s longest string, at the block holding them', () => {
let quote: AdfNode = { attrs: {}, content: [{ text: 'a\n'.repeat(1_100_000), type: 'text' }], type: 'codeBlock' }
for (let level = 0; level < 245; level += 1) quote = { content: [quote], type: 'blockquote' }
const result = adfToLosslessMarkdown(document(quote))
assert.equal(markdown(result), tooLong)
assert.equal(path(result).length, 2 * 246)
})
test('refuses a directive whose opener and body together would pass V8\'s longest string', () => {
const long = 'a'.repeat(2 ** 28)
const expand: AdfNode = { attrs: { title: long }, content: [{ content: [{ text: long, type: 'text' }], type: 'codeBlock' }], type: 'expand' }
assert.equal(markdown(adfToLosslessMarkdown(document(expand))), tooLong)
})
test('refuses text past V8\'s longest string before spelling it', () => {
const long = 'a'.repeat(2 ** 28)
const result = adfToLosslessMarkdown(document(paragraph({ text: long, type: 'text' }, { text: long, type: 'text' }, { text: long, type: 'text' })))
assert.equal(markdown(result), tooLong)
assert.deepEqual(path(result), ['content', 0])
}) })
test('refuses a document version the markdown cannot carry', () => { test('refuses a document version the markdown cannot carry', () => {
+37 -14
View File
@@ -12,6 +12,7 @@ import { fencedCodeBlock } from '../commonmark/backtick-runs.ts'
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts' import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
import { languageSlot, type LanguageSlot } from '../code-language.ts' import { languageSlot, type LanguageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts' import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts' import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
import { tryImage } from './image.ts' import { tryImage } from './image.ts'
@@ -29,9 +30,11 @@ type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { a
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node. // Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
export type SpellingMemo = Map<AdfNode, KeptSpelling> export type SpellingMemo = Map<AdfNode, KeptSpelling>
type WalkedItem = { node: AdfNode; walk: PlacedBlocks } type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
export type Writing = { flavour: WrittenFlavour; memo: SpellingMemo | undefined } // indent: the characters the containers around a block prefix each of its lines with.
export type Writing = { flavour: WrittenFlavour; indent: number; memo: SpellingMemo | undefined }
const bulletDelimiters: Delimiters = ['-', '*'] const bulletDelimiters: Delimiters = ['-', '*']
const bulletWidth = 2
export const largestListMarker = 999999999 export const largestListMarker = 999999999
// Bare because tryList admits no item carrying attributes, marks or text. // Bare because tryList admits no item carrying attributes, marks or text.
const listItemOpener = spellDirectiveOpener('listItem', undefined, '') const listItemOpener = spellDirectiveOpener('listItem', undefined, '')
@@ -46,7 +49,7 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
if (fault !== undefined) return faulted(fault, []) if (fault !== undefined) return faulted(fault, [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
if (document.content === undefined) return success(`${documentSpelling}\n`) if (document.content === undefined) return success(`${documentSpelling}\n`)
const walk = walkBlocks(document.content, [], 0, { flavour, memo: undefined }) const walk = walkBlocks(document.content, [], 0, { flavour, indent: 0, memo: undefined })
if (!walk.ok) return walk if (!walk.ok) return walk
const text = joinBlocks(walk.value.blocks, 'document') const text = joinBlocks(walk.value.blocks, 'document')
return success(text === '' ? '' : `${text}\n`) return success(text === '' ? '' : `${text}\n`)
@@ -56,15 +59,29 @@ function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: nu
let headroom = largestNesting - depth let headroom = largestNesting - depth
if (headroom < 0) return tooDeep(path) if (headroom < 0) return tooDeep(path)
const blocks: PlacedBlock[] = [] const blocks: PlacedBlock[] = []
// Each block's length once its containers prefix its lines, and the two characters parting it from the next.
let projected = 0
for (const [index, node] of nodes.entries()) { for (const [index, node] of nodes.entries()) {
const block = emitBlock(node, [...path, 'content', index], depth, writing) const block = emitBlock(node, [...path, 'content', index], depth, writing)
if (!block.ok) return block if (!block.ok) return block
projected += prefixedLength(block.value.text, writing.indent) + 2
if (projected > longestOutput) return tooLong([...path, 'content', index])
headroom = Math.min(headroom, block.value.headroom) headroom = Math.min(headroom, block.value.headroom)
blocks.push({ ...block.value, node }) blocks.push({ ...block.value, node })
} }
return success({ blocks, headroom }) return success({ blocks, headroom })
} }
function prefixedLength(text: string, indent: number): number {
let lines = 1
for (let at = text.indexOf('\n'); at !== -1; at = text.indexOf('\n', at + 1)) lines += 1
return text.length + lines * indent
}
function indented(writing: Writing, width: number): Writing {
return { ...writing, indent: writing.indent + width }
}
function tooDeep(path: ConvertErrorPath): Result<never> { function tooDeep(path: ConvertErrorPath): Result<never> {
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
} }
@@ -107,25 +124,25 @@ function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing
return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing)) return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing))
} }
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<null> | undefined { export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing & { memo: SpellingMemo }): Result<null> | undefined {
const readable = readableBlock(node, path, depth, writing) const readable = readableBlock(node, path, depth, writing)
if (readable === undefined) writing.memo.set(node, { block: undefined, depth })
else if (readable.ok) writing.memo.set(node, { block: readable.value, depth })
if (readable === undefined) return undefined if (readable === undefined) return undefined
if (!readable.ok) return readable if (!readable.ok) return readable
return readable.value.spelling === 'directive' ? undefined : success(null) return readable.value.spelling === 'directive' ? undefined : success(null)
} }
function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
const { memo } = writing const kept = writing.memo?.get(node)
const kept = memo?.get(node) // The walk reading an entry spells the node into its own text, so the entry is dropped: kept, every level of a deep subtree would hold its whole text.
writing.memo?.delete(node)
if (kept !== undefined) { if (kept !== undefined) {
if (kept.block === undefined) return undefined if (kept.block === undefined) return undefined
// An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards. // An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards.
if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth }) if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth })
} }
const spelled = spellReadableBlock(node, path, depth, writing) return spellReadableBlock(node, path, depth, writing)
if (spelled === undefined) memo?.set(node, { block: undefined, depth })
else if (spelled.ok) memo?.set(node, { block: spelled.value, depth })
return spelled
} }
function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
@@ -154,12 +171,14 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
} }
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> { function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing) const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner if (!inner.ok) return inner
const body = joinBlocks(inner.value.blocks, 'document') const body = joinBlocks(inner.value.blocks, 'document')
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom)) return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
} }
const quoteWidth = 2
function quoted(text: string): string { function quoted(text: string): string {
return text return text
.split('\n') .split('\n')
@@ -173,7 +192,8 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
// A child other than a task nests in the task before it. // A child other than a task nests in the task before it.
for (const [index, child] of nodeContent(node).entries()) { for (const [index, child] of nodeContent(node).entries()) {
const task = isTaskItem(child) const task = isTaskItem(child)
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, writing) : placedBlock(child, [...path, 'content', index], depth + 1, writing) const itemWriting = indented(writing, bulletWidth)
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, itemWriting) : placedBlock(child, [...path, 'content', index], depth + 1, itemWriting)
if (!walk.ok) return walk if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom) headroom = Math.min(headroom, walk.value.headroom)
const previous = items.at(-1) const previous = items.at(-1)
@@ -248,12 +268,14 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
} }
const walk = walkBody() const walk = walkBody()
if (!walk.ok) return walk if (!walk.ok) return walk
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom)) const body = joinBlocks(walk.value.blocks, 'directive')
if (opener.length + body.length > longestOutput) return tooLong(path)
return success(directivePair(node, opener, body, walk.value.headroom))
} }
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
if (!holdsOnlyAttributes(node, [])) return undefined if (!holdsOnlyAttributes(node, [])) return undefined
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing) const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
if (!inner.ok) return inner if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document') const text = joinBlocks(inner.value.blocks, 'document')
const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
@@ -313,8 +335,9 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined
const walked: WalkedItem[] = [] const walked: WalkedItem[] = []
let headroom = Number.POSITIVE_INFINITY let headroom = Number.POSITIVE_INFINITY
const itemWriting = indented(writing, ordered ? `${start + items.length - 1}. `.length : bulletWidth)
for (const [offset, item] of items.entries()) { for (const [offset, item] of items.entries()) {
const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, writing) const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, itemWriting)
if (!walk.ok) return walk if (!walk.ok) return walk
headroom = Math.min(headroom, walk.value.headroom) headroom = Math.min(headroom, walk.value.headroom)
walked.push({ node: item, walk: walk.value }) walked.push({ node: item, walk: walk.value })
+33 -26
View File
@@ -10,11 +10,12 @@ import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonma
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts' import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts'
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
import { identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts'
import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts' import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestBacktickRun } from '../commonmark/backtick-runs.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts'
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
import { spellTextDirective } from '../text-directive.ts' import { spellTextDirective } from '../text-directive.ts'
@@ -80,6 +81,7 @@ export function tryImageLine(alt: string | undefined, url: string, link?: LinkDe
} }
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> { function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> {
if (heldTextLength(nodes) > longestOutput) return tooLong(path)
const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false } const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false }
// Terminates because takeFallback refuses a pass that took no new fallback. // Terminates because takeFallback refuses a pass that took no new fallback.
for (;;) { for (;;) {
@@ -99,6 +101,17 @@ function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: Con
} }
} }
// The line spells every character of its text at least once, so this much text cannot fit.
export function heldTextLength(nodes: readonly AdfNode[]): number {
let length = 0
const pending = [...nodes]
for (let node = pending.pop(); node !== undefined; node = pending.pop()) {
length += textLength(node)
for (const child of nodeContent(node)) pending.push(child)
}
return length
}
function takeFallback(fallbacks: LineFallbacks, fallback: NodeRange | 'opening-link', path: ConvertErrorPath): Result<null> { function takeFallback(fallbacks: LineFallbacks, fallback: NodeRange | 'opening-link', path: ConvertErrorPath): Result<null> {
if (fallback === 'opening-link') { if (fallback === 'opening-link') {
if (fallbacks.openingLinkAsDirective) return failure('unsupported-node-shape', 'an opening link spelled as a directive still reads as a link definition, so the line has no spelling left', path) if (fallbacks.openingLinkAsDirective) return failure('unsupported-node-shape', 'an opening link spelled as a directive still reads as a link definition, so the line has no spelling left', path)
@@ -172,13 +185,9 @@ function bracketedLinkTarget(target: string): InlineSegment {
return { escaping: 'bracketed-link-target', text: escapeUnbalanced(target, '[', ']') } return { escaping: 'bracketed-link-target', text: escapeUnbalanced(target, '[', ']') }
} }
function refuseContentAndText(node: AdfNode, path: ConvertErrorPath): Result<null> { function heldContentOrText(node: AdfNode): 'content' | 'text' | undefined {
const holdsContent = nodeContent(node).length > 0 if (nodeContent(node).length > 0) return 'content'
if (holdsContent || node.text !== undefined) { return node.text === undefined ? undefined : 'text'
const held = holdsContent ? 'content' : 'text'
return failure('unsupported-node-shape', `a ${node.type} node holds neither content nor text: this one holds ${held}`, path)
}
return success(null)
} }
function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, context: InlineContext): Result<Emission> { function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, context: InlineContext): Result<Emission> {
@@ -235,40 +244,38 @@ function carries(node: AdfNode, carried: ReadonlySet<number>, index: number): bo
return typeof slot === 'string' && slotFault(node.type, slot) !== undefined && nodeContent(node).length === 0 && node.text === undefined return typeof slot === 'string' && slotFault(node.type, slot) !== undefined && nodeContent(node).length === 0 && node.text === undefined
} }
// Builds the node's path only to refuse it: a path copies every level above, and a line may hold a million leaves.
function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result<Emission> { function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result<Emission> {
const path = nodePath(context, index)
if (carries(node, context.carried, index)) { if (carries(node, context.carried, index)) {
const carried = carriedInline(node, path) const carried = carriedInline(node, nodePath(context, index))
if (!carried.ok) return carried if (!carried.ok) return carried
return success({ segments: [syntax(carried.value)] }) return success({ segments: [syntax(carried.value)] })
} }
const types = nodeMarks(node).map((mark) => mark.type) const types = nodeMarks(node).map((mark) => mark.type)
if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path) if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, nodePath(context, index))
const model = inlineNodeModel(node.type) const model = inlineNodeModel(node.type)
if (model === undefined) return success(emitText(node, context, index)) if (model === undefined) return success(emitText(node, context, index))
if (node.type === 'hardBreak') return emitHardBreak(node, model, context, index, path) const held = heldContentOrText(node)
return emitInlineDirective(node, model, index, path) if (held !== undefined) return failure('unsupported-node-shape', `a ${node.type} node holds neither content nor text: this one holds ${held}`, nodePath(context, index))
if (node.type === 'hardBreak') return success(emitHardBreak(node, model, context, index))
return success(emitInlineDirective(node, model, index))
} }
function emitHardBreak(node: AdfNode, model: InlineNodeModel, context: InlineContext, index: number, path: ConvertErrorPath): Result<Emission> { function emitHardBreak(node: AdfNode, model: InlineNodeModel, context: InlineContext, index: number): Emission {
const empty = refuseContentAndText(node, path)
if (!empty.ok) return empty
const attributes = spellInlineNodeAttributes(node, model) const attributes = spellInlineNodeAttributes(node, model)
if (attributes === undefined) return success({ carry: { first: index, last: index } }) if (attributes === undefined) return { carry: { first: index, last: index } }
if (attributes === '' && context.spansLines && !context.atBlockEnd) return success({ segments: [syntax('\\\n')] }) if (attributes === '' && context.spansLines && !context.atBlockEnd) return { segments: [syntax('\\\n')] }
return success({ segments: [syntax(spellInlineLeafDirective('hardBreak', attributes))] }) return { segments: [syntax(spellInlineLeafDirective('hardBreak', attributes))] }
} }
function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: number, path: ConvertErrorPath): Result<Emission> { function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: number): Emission {
const empty = refuseContentAndText(node, path)
if (!empty.ok) return empty
const attributes = spellInlineNodeAttributes(node, model) const attributes = spellInlineNodeAttributes(node, model)
if (attributes === undefined) return success({ carry: { first: index, last: index } }) if (attributes === undefined) return { carry: { first: index, last: index } }
const slot = model.textAttribute === undefined ? undefined : nodeAttrs(node)[model.textAttribute] const slot = model.textAttribute === undefined ? undefined : nodeAttrs(node)[model.textAttribute]
if (slot === undefined) return success({ segments: [syntax(spellInlineLeafDirective(node.type, attributes))] }) if (slot === undefined) return { segments: [syntax(spellInlineLeafDirective(node.type, attributes))] }
if (typeof slot !== 'string' || slotFault(node.type, slot) !== undefined) return success({ carry: { first: index, last: index } }) if (typeof slot !== 'string' || slotFault(node.type, slot) !== undefined) return { carry: { first: index, last: index } }
const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }] const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }]
return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }) return { segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }
} }
// CommonMark reads a raw carriage return as a line ending and a null character as U+FFFD, so neither is written raw. // CommonMark reads a raw carriage return as a line ending and a null character as U+FFFD, so neither is written raw.
+4 -2
View File
@@ -9,8 +9,9 @@ import { infoStringCarries } from './commonmark/grammar.ts'
import { isAdfNode } from '../adf/document.ts' import { isAdfNode } from '../adf/document.ts'
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
import { largestNesting } from '../nesting.ts' import { largestNesting } from '../nesting.ts'
import { longestOutput, tooLong } from '../output-length.ts'
import { parseJsonText } from '../json-text.ts' import { parseJsonText } from '../json-text.ts'
import { serializeCanonicalJson, spellsCanonicalJson } from '../canonical-json.ts' import { serializeCanonicalJsonWithin, spellsCanonicalJson } from '../canonical-json.ts'
export const carryFencePrefix = 'adf:' export const carryFencePrefix = 'adf:'
@@ -58,7 +59,8 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) { if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
} }
return success({ headroom, json: serializeCanonicalJson(spelled, spelling) }) const json = serializeCanonicalJsonWithin(spelled, spelling, longestOutput)
return json === undefined ? tooLong(path) : success({ headroom, json })
} }
// `type` is the one the fence's info string names, `undefined` for the inline carry. // `type` is the one the fence's info string names, `undefined` for the inline carry.
+6 -1
View File
@@ -449,13 +449,18 @@ test('names the canonical spelling a carried JSON reads alone', () => {
}) })
test('refuses compact JSON in the carry fence without building its two-space spelling', () => { test('refuses compact JSON in the carry fence without building its two-space spelling', () => {
const json = `{"attrs":{"value":${'['.repeat(495)}${new Array(2_200_000).fill(0).join(',')}${']'.repeat(495)}}}` const json = `{"attrs":{"value":${'['.repeat(495)}${new Array(560_000).fill(0).join(',')}${']'.repeat(495)}}}`
assert.equal( assert.equal(
content(losslessMarkdownToAdf(`\`\`\`adf:unknownBlock\n${json}\n\`\`\`\n`)), content(losslessMarkdownToAdf(`\`\`\`adf:unknownBlock\n${json}\n\`\`\`\n`)),
"unsupported-node-shape: the opaque carry spells its node's JSON canonically: two-space indent, keys sorted; !adf:codeBlock {language=\"adf:unknownBlock\"} around a bare fence keeps it a code block", "unsupported-node-shape: the opaque carry spells its node's JSON canonically: two-space indent, keys sorted; !adf:codeBlock {language=\"adf:unknownBlock\"} around a bare fence keeps it a code block",
) )
}) })
test('refuses a directive whose CommonMark spelling would pass V8\'s longest string', () => {
const markdown = `!adf:blockquote\n${'- '.repeat(490)}a\\\n${'b\\\n'.repeat(560_000)}c\n!adf:/blockquote\n`
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds')
})
test('names the node JSON an opaque carry restores alone', () => { test('names the node JSON an opaque carry restores alone', () => {
const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; " const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; "
assert.equal(content(losslessMarkdownToAdf('```adf:x\n[]\n```\n')), `${node}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block`) assert.equal(content(losslessMarkdownToAdf('```adf:x\n[]\n```\n')), `${node}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block`)
+1 -1
View File
@@ -254,7 +254,7 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr
if (!read.ok) return read if (!read.ok) return read
const built = directiveBody(read.value, block.blocks, reading, path, depth) const built = directiveBody(read.value, block.blocks, reading, path, depth)
if (!built.ok) return built if (!built.ok) return built
const readable = commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', memo: reading.memo }) const readable = commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', indent: 0, memo: reading.memo })
if (readable === undefined) return built if (readable === undefined) return built
if (!readable.ok) return readable if (!readable.ok) return readable
return failure('unsupported-node-shape', `${built.value.type} takes the CommonMark spelling, not the directive form`, path) return failure('unsupported-node-shape', `${built.value.type} takes the CommonMark spelling, not the directive form`, path)
@@ -70,11 +70,10 @@ test('refuses what the document guard refuses, and nothing else', () => {
assert.match(portableDocument(document(paragraph(deepInline))), /^unsupported-nesting-depth at /) assert.match(portableDocument(document(paragraph(deepInline))), /^unsupported-nesting-depth at /)
}) })
// The merge throws at its concatenation, before any text is scanned, so no engine holds more than the one string. test('refuses a paragraph whose text would pass V8\'s longest string, joining none of it', () => {
test('refuses a paragraph whose merged text passes the engine\'s longest string', () => {
const long = 'a'.repeat(2 ** 28) const long = 'a'.repeat(2 ** 28)
const merged = paragraph(...Array.from({ length: 9 }, () => text(long))) const merged = paragraph(...Array.from({ length: 9 }, () => text(long)))
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /') assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /content/0')
}) })
test('refuses nesting past 500 levels wherever the reduction walks', () => { test('refuses nesting past 500 levels wherever the reduction walks', () => {
@@ -124,7 +124,7 @@ function portableSequence(blocks: readonly AdfNode[], reduction: Reduction): Res
for (let next = pending.pop(); next !== undefined; next = pending.pop()) { for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
const block = numberedPastMarkers(next) const block = numberedPastMarkers(next)
const listed = block.type === 'bulletList' || block.type === 'orderedList' const listed = block.type === 'bulletList' || block.type === 'orderedList'
if (!listed || (block === next && commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', memo: reduction.memo })?.ok === true)) { if (!listed || (block === next && commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo })?.ok === true)) {
sequence.push(block) sequence.push(block)
continue continue
} }
+4 -1
View File
@@ -1,7 +1,8 @@
import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts'
import type { JsonValue } from '../../json-value.ts' import type { JsonValue } from '../../json-value.ts'
import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
import { joinsWhenRead } from '../adjacent-text.ts' import { joinsWhenRead } from '../adjacent-text.ts'
import { longestOutput } from '../../output-length.ts'
type JsonContainer = JsonValue[] | { [key: string]: JsonValue } type JsonContainer = JsonValue[] | { [key: string]: JsonValue }
@@ -10,6 +11,8 @@ type NodeHolder = { content?: AdfNode[] }
// The editor joins two text nodes where the reader would join their editor-normal forms. // The editor joins two text nodes where the reader would join their editor-normal forms.
export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean {
if (previous.type !== 'text' || node.type !== 'text') return false if (previous.type !== 'text' || node.type !== 'text') return false
// Text this long is refused as markdown either way; joining it first would build the string.
if (textLength(previous) + textLength(node) > longestOutput) return false
return joinsWhenRead(normalNode(previous), normalNode(node)) return joinsWhenRead(normalNode(previous), normalNode(node))
} }
+3 -1
View File
@@ -5,8 +5,9 @@ import { blockNodeModel } from '../../adf/block-nodes.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts' import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestOutput, tooLong } from '../../output-length.ts'
import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts' import { heldTextLength, portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts'
import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
const highlight = 'backgroundColor' const highlight = 'backgroundColor'
@@ -17,6 +18,7 @@ const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link']
export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> { export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> {
const leaves = inlineLeaves(nodes, container, path, depth) const leaves = inlineLeaves(nodes, container, path, depth)
if (!leaves.ok) return leaves if (!leaves.ok) return leaves
if (heldTextLength(leaves.value) > longestOutput) return tooLong(path)
return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path) return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path)
} }
+8
View File
@@ -0,0 +1,8 @@
import { failure, type ConvertErrorPath, type Result } from './result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestOutput = 2 ** 29 - 24
export function tooLong<T>(path: ConvertErrorPath): Result<T> {
return failure('unsupported-node-shape', `the markdown would be longer than ${longestOutput} characters, the longest string V8 holds`, path)
}
+15 -14
View File
@@ -6,7 +6,7 @@
`Bar = 9` `Bar = 9`
`Next ID = 108` `Next ID = 110`
| Goal | W | | Goal | W |
|---|---| |---|---|
@@ -25,7 +25,7 @@
| ID | Release | Exempt | Item | R | S | A | G | Goals | Score | | ID | Release | Exempt | Item | R | S | A | G | Goals | Score |
|---|---|---|---|---|---|---|---|---|---| |---|---|---|---|---|---|---|---|---|---|
| 106 | 0.2.0 | defect | **Refuse a document or markdown whose output would pass a stated length before building it, so no call exhausts the heap.** | 4 | 6 | 5 | 8 | 1, 9 | 20.5 | | 108 | 0.2.0 | defect | **Spell a line in memory linear in its length, so a paragraph shorter than V8's longest string converts within Node's default heap.** | 4 | 6 | 5 | 8 | 1, 9 | 20.5 |
| 103 | 0.2.0 | | **Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.** | 2 | 2 | 5 | 6 | 1 | 19.5 | | 103 | 0.2.0 | | **Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.** | 2 | 2 | 5 | 6 | 1 | 19.5 |
| 84 | 0.2.0 | | **Read GFM's pipe table without leading pipes, and with alignment colons, in `portableMarkdownToAdf`, dropping the alignment.** | 3 | 4 | 7 | 7 | 4, 7 | 19.4 | | 84 | 0.2.0 | | **Read GFM's pipe table without leading pipes, and with alignment colons, in `portableMarkdownToAdf`, dropping the alignment.** | 3 | 4 | 7 | 7 | 4, 7 | 19.4 |
| 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 6 | 11.5 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 6 | 11.5 |
@@ -42,6 +42,7 @@
| 33 | 0.2.1 | defect | **Emit a line in time linear in its mark runs, in `adfToLosslessMarkdown` and `adfToPortableMarkdown`.** | 4 | 5 | 6 | 9 | 9 | 8.7 | | 33 | 0.2.1 | defect | **Emit a line in time linear in its mark runs, in `adfToLosslessMarkdown` and `adfToPortableMarkdown`.** | 4 | 5 | 6 | 9 | 9 | 8.7 |
| 98 | 0.2.1 | defect | **Read blank lines inside deeply nested list items in time linear in the markdown.** | 3 | 3 | 5 | 9 | 9 | 8.2 | | 98 | 0.2.1 | defect | **Read blank lines inside deeply nested list items in time linear in the markdown.** | 3 | 3 | 5 | 9 | 9 | 8.2 |
| 102 | 0.2.1 | decision | **Place a block a portable task list refuses at its path in the returned document without replaying `splitTaskList` on stand-in nodes.** | 3 | 4 | 2 | 4 | 1 | 8.0 | | 102 | 0.2.1 | decision | **Place a block a portable task list refuses at its path in the returned document without replaying `splitTaskList` on stand-in nodes.** | 3 | 4 | 2 | 4 | 1 | 8.0 |
| 109 | 0.2.1 | defect | **Spell nested lists and quotes in time linear in the markdown.** | 3 | 5 | 4 | 7 | 9 | 5.3 |
| 7 | 0.3.0 | decision | **Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.** | 6 | 9 | 9 | 9 | 2, 4 | 25.8 | | 7 | 0.3.0 | decision | **Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.** | 6 | 9 | 9 | 9 | 2, 4 | 25.8 |
| 45 | 0.3.0 | | **Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 45 | 0.3.0 | | **Replace `isAdfDocument` with a reader returning `Result<AdfDocument>`.** | 2 | 3 | 6 | 8 | 1 | 25.2 |
| 6 | 0.3.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 4 | 24.7 | | 6 | 0.3.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 4 | 24.7 |
@@ -73,19 +74,13 @@
## Details ## Details
### 106. Refuse a document or markdown whose output would pass a stated length before building it, so no call exhausts the heap. ### 108. Spell a line in memory linear in its length, so a paragraph shorter than V8's longest string converts within Node's default heap.
The writers return `unsupported-node-shape` where the engine throws past its longest string, but The line escaper keeps an array element per character twice (`escapings`, `placements` in `escape`,
some documents exhaust the heap first, which no catch survives. `losslessMarkdownToAdf` on 2.1 MB of `line-escaping.ts`) and a `Set` entry per escaped character, so `adfToLosslessMarkdown` holds about
markdown, a `!adf:blockquote` holding a list nested 490 levels whose paragraph runs on in 700,000 50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
lazy lines `b\`, crashed Node 24 out of heap: `directiveNode` re-spells the subtree, indenting each million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
line to its depth. `adfToLosslessMarkdown` on nine text nodes sharing one 2^28-character string got exceeded`, which `withinEngineLimits` lets through. Found while shipping item 106, 2026-10-11.
Bun OOM-killed under 8 GB: `escape` in `line-escaping.ts` pushes one array element per character.
Where the heap holds out, the readers' re-spelling can still throw a `RangeError`. JavaScriptCore's
`Out of memory`, which the writers refuse, may also mean a failed allocation that a retry survives.
The candidate length to refuse past is V8's 2^29 − 24 on every engine, so the same document converts
alike everywhere; no entry states it yet (AGENTS.md §Stated numbers). Found by the principles audit
of item 83, 2026-10-11.
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code. ### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.
@@ -211,6 +206,12 @@ the fix.
`taskOfParagraphs` keeps the original child objects. Both paired-ruling seats on 2026-10-11 named it `taskOfParagraphs` keeps the original child objects. Both paired-ruling seats on 2026-10-11 named it
the reason the project read worse. the reason the project read worse.
### 109. Spell nested lists and quotes in time linear in the markdown.
Each level of a nested list or quote re-indents its whole text, so `losslessMarkdownToAdf` on 3 MB
of lazy lines in a list nested 490 levels inside `!adf:blockquote` spends about 5 seconds
re-spelling it, the writers the same on its document. Found while shipping item 106, 2026-10-11.
### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour. ### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.
Lands after items 6, 59, 60 and 62. Which flavours get a composed call is this item's to settle; Lands after items 6, 59, 60 and 62. Which flavours get a composed call is this item's to settle;