3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-12 11:48:29 +01:00

Answer architecture round 5: the builders keep main's spelling, each writer's result and the CommonMark ask's spelling are held to the bound once built, and the engines' own overflows refuse alike

This commit is contained in:
2026-10-12 21:36:11 +02:00
committed by Lillem4n
parent a0bb09b80d
commit 6f518dab93
16 changed files with 165 additions and 228 deletions
+1 -1
View File
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do |
| --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is, edge whitespace included. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only |
## The guarantees
+25
View File
@@ -602,3 +602,28 @@ Inherited architect, decided by:
- Navigation, Shape: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long symptom leads straight to the module named for it
- Shape: `src/markdown/directive-syntax.ts:133` `spellDirectiveOpener`: each spelling is a string function and a pieces generator
- Shape: `src/markdown/opaque-carry.ts:64` `carriedJson`: the lambda parameter `spelled` shadows the function's own
## 2026-10-12T16:40:52Z, PR #36 at acd22aa, against 7a27f6b
Ruling: worse
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | worse | worse | same | worse |
| Inherited architect | better | worse | worse | same | worse |
Mid A, decided by:
- Locality, Shape: `src/markdown/emit/adf-to-markdown.ts:374` `spelledList`: a builder's `undefined` now means either no list spelling or past the length bound, and every caller of `joinBlocks` must remember which it holds
- Shape: `src/markdown/emit/adf-to-markdown.ts:383` `listLines`: split, map and join became generators yielding pieces into `separatedBy` and `joinedWithin`, so a reader rebuilds the string in their head
- Locality: `src/markdown/emit/adf-to-markdown.ts:125` `readableBlock`: the memo is filled in `commonMarkSpelling` and deleted on read, a one-shot cache whose hits depend on which walk came first
- Navigation: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long refusal leads to a module named for it
Inherited architect, decided by:
- Locality, Shape: `src/markdown/emit/adf-to-markdown.ts:374` `spelledList`: `undefined` means two things a few lines apart, told apart only by the side of a nested `boundedThen` callback
- Locality: `src/markdown/emit/adf-to-markdown.ts:125` `readableBlock`: reading the memo is a mutation, and a second read spells again
- Shape: `src/markdown/pipe-table-syntax.ts:43` `pipeRowPieces`: one-line templates became generators whose output cannot be read off one line
- Locality, Shape: `src/markdown/emit/block-directive-spelling.ts:27` `spellBlockDirectiveOpener`: the opener's layout is known here and in `spellDirectiveOpener`
- Shape: `src/markdown/portable/adf-to-portable-markdown.ts:231` `reduceCodeBlock`: an ADF-to-ADF reducer carries a markdown length bound through a callback
- Navigation: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long refusal leads straight to one small, well-named module
+13 -11
View File
@@ -666,18 +666,20 @@ is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
converts alike everywhere. Each builder joins its pieces — blocks, quoted and list lines, directive
pairs and openers, fences, pipe rows, the carry's JSON — through `joinedWithin`, which stops once
they pass the bound, so none builds a string no engine was asked for. A writer first refuses a
document whose text alone passes the bound, since every character of it is spelled at least once. A
deep list or quote re-indents its whole text at each level until the level that passes the bound
refuses (`todo.md` item 109).
converts alike everywhere. A writer first refuses a document whose text alone passes the bound: the
lossless writer spells every character of it, and the portable reduction refuses on the caller's
text, edge whitespace and all, since merging that text first could build past the bound. The
builders join as they always did, and the writer's result and the CommonMark ask's spelling are held
to the bound once built. V8 stops at the bound itself and SpiderMonkey at its own longest string,
and `withinEngineLimits` refuses both throws as the same `tooLong`, so every engine answers alike.
The carry's two-space JSON, which indents every value its depth over, stops at the bound unbuilt
(`joinedWithin`). A deep list or quote re-indents its whole text at each level before the bound
refuses it (`todo.md` item 109).
A line is held to its text's length alone; what its spelling adds — escapes, link targets,
attributes — passes V8's longest string only by throwing, and `withinEngineLimits` refuses that
throw, and SpiderMonkey's, as the same refusal: both mean a string past the bound. `todo.md` item
108 bounds the line itself. The engines' throws past their own longest array, and JavaScriptCore's
`Out of memory`, which also reports a heap running short, stay refused with the engine's message.
JavaScriptCore builds up to its own longest string, 2^31 − 1, before the check. A throw past that,
past any engine's longest array, or JavaScriptCore's `Out of memory`, which also reports a heap
running short, is refused with the engine's message. `todo.md` item 108 bounds the line's own
memory.
## Nothing recurses unbounded
+3 -3
View File
@@ -25,7 +25,7 @@ import {
spellStringAttribute,
spellVocabulary,
} from '../markdown/directive-syntax.ts'
import { fencePieces } from '../markdown/commonmark/backtick-runs.ts'
import { fencedCodeBlock } from '../markdown/commonmark/backtick-runs.ts'
import { inlineNodes } from '../adf/inline-nodes.ts'
import { markAttributes } from '../adf/mark-attributes.ts'
import { markSpelling } from '../markdown/mark-spellings.ts'
@@ -174,8 +174,8 @@ const carriedNode = fc.oneof(
const spellCarry = (json: string) => spellInlineLeafDirective(carryName, `{json=${spellStringAttribute(json)}}`)
const inlineCarry = carriedNode.map((node) => spellCarry(serializeCanonicalJson(node, 'compact')))
const hostileInlineCarry = fc.oneof(carriedNode, jsonValue).map((node) => spellCarry(JSON.stringify(node, null, 1)))
const blockCarry = carriedNode.map((node) => fencePieces(carryName, serializeCanonicalJson(node, 'two-space')).join(''))
const hostileBlockCarry = fc.oneof(carriedNode, jsonValue).map((node) => fencePieces(carryName, JSON.stringify(node)).join(''))
const blockCarry = carriedNode.map((node) => fencedCodeBlock(carryName, serializeCanonicalJson(node, 'two-space')))
const hostileBlockCarry = fc.oneof(carriedNode, jsonValue).map((node) => fencedCodeBlock(carryName, JSON.stringify(node)))
function prefixLines(body: string, first: string, rest: (index: number) => string): string {
return body
+2 -2
View File
@@ -15,9 +15,9 @@ export function closingBacktickRun(text: string, from: number, opener: number):
return undefined
}
export function fencePieces(info: string, body: string): string[] {
export function fencedCodeBlock(info: string, body: string): string {
const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1))
return body === '' ? [fence, info, '\n', fence] : [fence, info, '\n', body, '\n', fence]
return body === '' ? `${fence}${info}\n${fence}` : `${fence}${info}\n${body}\n${fence}`
}
export function longestBacktickRun(text: string): number {
+3 -12
View File
@@ -112,18 +112,9 @@ export function slotFault(type: string, text: string): ConvertFault | undefined
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
return [...attributePieces(pairs)].join('')
}
export function* attributePieces(pairs: readonly (readonly [string, string])[]): Generator<string> {
if (pairs.length === 0) return
yield '{'
for (const [index, [key, value]] of [...pairs].sort(([left], [right]) => keyOrder(left, right)).entries()) {
if (index > 0) yield ' '
yield `${key}=`
yield value
}
yield '}'
if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`)
return `{${spelled.join(' ')}}`
}
export function spellDirectiveCloser(name: string): string {
+1 -1
View File
@@ -46,7 +46,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds'
// A code block of 300 lines, two of them some 1000 characters short of V8's longest string, so only what their spelling adds passes it.
// A code block of 300 lines; two of these together fall some 1000 characters short of V8's longest string, so only what their spelling adds passes it.
function nearlyHalfTheBound(): AdfNode {
return { attrs: {}, content: [{ text: `${'a'.repeat(894_782)}\n`.repeat(300), type: 'text' }], type: 'codeBlock' }
}
+72 -124
View File
@@ -5,14 +5,14 @@ import { adfDocumentFault, heldTextLength, holdsOnlyAttributes, isUnmarkedSpella
import { alertMarker, flavourClaims, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../portable/conventions.ts'
import { blockDirectiveForm, documentSpelling } from '../block-directive.ts'
import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts'
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong, withinEngineLimits } from '../length-bound.ts'
import { carriedBlock } from '../opaque-carry.ts'
import { emitInlineLine } from './inline-line.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { fencePieces } from '../commonmark/backtick-runs.ts'
import { fencedCodeBlock } from '../commonmark/backtick-runs.ts'
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
import { languageSlot, type LanguageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts'
import { longestMarkdown, tooLong, withinEngineLimits, withinLongest } from '../length-bound.ts'
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
import { tryImage } from './image.ts'
@@ -28,9 +28,9 @@ type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number }
// CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`.
type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string }
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
// Holds the spelling of each node the CommonMark ask was put to, until the walk above reads it.
export type SpellingMemo = Map<AdfNode, KeptSpelling>
// inner: the item's blocks joined, for the list form; walk: the blocks themselves, for the directive form.
type WalkedItem = { inner: string; node: AdfNode; walk: PlacedBlocks }
type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
export type Writing = { flavour: WrittenFlavour; memo: SpellingMemo | undefined }
const bulletDelimiters: Delimiters = ['-', '*']
@@ -52,8 +52,8 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
if (heldTextLength(document.content) > longestMarkdown) return tooLong()
const walk = walkBlocks(document.content, [], 0, { flavour, memo: undefined })
if (!walk.ok) return walk
const text = joinedWithin(walk.value.blocks.length === 0 ? [] : [...blockPieces(walk.value.blocks, 'document'), '\n'])
return boundedThen(text, success)
const text = joinBlocks(walk.value.blocks, 'document')
return withinLongest(text === '' ? '' : `${text}\n`)
}
function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
@@ -73,18 +73,16 @@ function tooDeep(path: ConvertErrorPath): Result<never> {
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
}
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string | undefined {
return joinedWithin(blockPieces(blocks, container))
}
function* blockPieces(blocks: readonly PlacedBlock[], container: BlockContainer): Generator<string> {
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string {
let text = ''
let alternated = false
for (const [index, block] of blocks.entries()) {
const previous = blocks[index - 1]
if (previous !== undefined) yield separationBetween(previous, block, container)
if (previous !== undefined) text += separationBetween(previous, block, container)
alternated = !alternated && previous !== undefined && sharesMarker(previous, block)
yield alternated && block.spelling === 'list' ? block.alternateText : block.text
text += alternated && block.spelling === 'list' ? block.alternateText : block.text
}
return text
}
// A task list is spelled as a bullet list.
@@ -119,6 +117,8 @@ export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth:
else if (readable.ok) writing.memo.set(node, { block: readable.value, depth })
if (readable === undefined) return undefined
if (!readable.ok) return readable
// V8 throws past the bound while building the spelling; an engine holding longer strings refuses here alike.
if (readable.value.text.length > longestMarkdown) return tooLong()
return readable.value.spelling === 'directive' ? undefined : success(null)
}
@@ -139,15 +139,12 @@ function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
if (portable !== undefined) return portable
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
if (node.type === 'codeBlock') return tryCodeBlock(node, writing.flavour)
if (node.type === 'codeBlock') return readableText(tryCodeBlock(node, writing.flavour))
if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
if (node.type === 'mediaSingle') return readableText(tryImage(node))
if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
if (node.type === 'rule') return readableText(tryRule(node))
if (node.type === 'table') {
const table = tryPipeTable(node, path, writing.flavour)
return table === undefined || !table.ok ? table : success(commonMarkText(table.value))
}
if (node.type === 'table') return readableText(tryPipeTable(node, path, writing.flavour))
return undefined
}
@@ -165,25 +162,15 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
if (!inner.ok) return inner
const text = joinedWithin(inner.value.blocks.length === 0 ? [head] : [head, '\n\n', ...blockPieces(inner.value.blocks, 'document')])
return quotedBlock(text, inner.value.headroom)
const body = joinBlocks(inner.value.blocks, 'document')
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
}
function quotedBlock(text: string | undefined, headroom: number): Result<EmittedBlock> {
const quoted = text === undefined ? undefined : joinedWithin(quotedPieces(text))
return boundedThen(quoted, (spelled) => success(commonMarkText(spelled, headroom)))
}
function* quotedPieces(text: string): Generator<string> {
for (let start = 0; ; ) {
const end = text.indexOf('\n', start)
const line = text.slice(start, end === -1 ? text.length : end)
yield line === '' ? '>' : '> '
yield line
if (end === -1) return
yield '\n'
start = end + 1
}
function quoted(text: string): string {
return text
.split('\n')
.map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n')
}
function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
@@ -199,21 +186,14 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
if (task || previous === undefined) items.push([...walk.value.blocks])
else for (const block of walk.value.blocks) previous.push(block)
}
const inners = joinedItems(items.map((blocks) => blockPieces(blocks, 'list-item')))
const list = spelledList(
items.map((blocks) => joinBlocks(blocks, 'list-item')),
() => '',
bulletDelimiters,
headroom,
)
// The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour.
const unspellable = failure<EmittedBlock>('unsupported-node-shape', 'a task holds blocks no list item spells', path)
return boundedThen(inners, (spelled) => spelledList(spelled, () => '', bulletDelimiters, headroom) ?? unspellable)
}
// Each item's text, or `undefined` where one would pass the bound.
function joinedItems(items: readonly Iterable<string>[]): string[] | undefined {
const inners: string[] = []
for (const pieces of items) {
const inner = joinedWithin(pieces)
if (inner === undefined) return undefined
inners.push(inner)
}
return inners
return list === undefined ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success(list)
}
function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
@@ -229,15 +209,13 @@ function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writin
const content = nodeContent(task)
const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour)
if (!line.ok) return line
const text = joinedWithin(line.value === '' ? [marker] : [marker, ' ', line.value])
return boundedThen(text, (spelled) => success({ blocks: [{ ...markerBlock, text: spelled }], headroom: Number.POSITIVE_INFINITY }))
return success({ blocks: [{ ...markerBlock, text: line.value === '' ? marker : `${marker} ${line.value}` }], headroom: Number.POSITIVE_INFINITY })
}
const walk = walkBlocks(nodeContent(task), path, depth, writing)
if (!walk.ok) return walk
const [first, ...rest] = walk.value.blocks
if (first?.node.type !== 'paragraph') return success({ blocks: [markerBlock, ...walk.value.blocks], headroom: walk.value.headroom })
const text = joinedWithin([marker, ' ', first.text])
return boundedThen(text, (spelled) => success({ blocks: [{ ...first, text: spelled }, ...rest], headroom: walk.value.headroom }))
const blocks = first?.node.type === 'paragraph' ? [{ ...first, text: `${marker} ${first.text}` }, ...rest] : [markerBlock, ...walk.value.blocks]
return success({ blocks, headroom: walk.value.headroom })
}
function readableText(text: string | undefined): Result<EmittedBlock> | undefined {
@@ -253,10 +231,8 @@ function commonMarkText(text: string, headroom: number = Number.POSITIVE_INFINIT
return { headroom, spelling: 'commonmark', text }
}
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): Result<EmittedBlock> {
const closer = spellDirectiveCloser(node.type)
const text = joinedWithin(body === '' ? [opener, '\n', closer] : [opener, '\n', body, '\n', closer])
return boundedThen(text, (spelled) => success({ headroom, spelling: 'directive', text: spelled }))
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): EmittedBlock {
return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` }
}
function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> {
@@ -274,12 +250,11 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
if (model.contentModel === 'inline') {
const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless')
if (!line.ok) return line
return directivePair(node, opener, line.value)
return success(directivePair(node, opener, line.value))
}
const walk = walkBody()
if (!walk.ok) return walk
const body = joinBlocks(walk.value.blocks, 'directive')
return boundedThen(body, (spelled) => directivePair(node, opener, spelled, walk.value.headroom))
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom))
}
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
@@ -287,18 +262,17 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document')
const alert = text !== undefined && flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
return quotedBlock(alert ? joinedWithin(['\\', text]) : text, inner.value.headroom)
const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom))
}
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): string | undefined {
if (!holdsOnlyAttributes(node, ['language'])) return undefined
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour])
if (slot.kind === 'attribute') return undefined
const [only, ...others] = fencedTexts(node) ?? []
if (only === undefined || others.length > 0) return undefined
const text = joinedWithin(fencePieces(slot.kind === 'fence' ? slot.info : '', only))
return boundedThen(text, (spelled) => success(commonMarkText(spelled)))
return fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only)
}
// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does.
@@ -310,8 +284,7 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr
if (opener === undefined) return carriedFence(carriedBlock(node, path, depth))
if (!opener.ok) return opener
const info = slot.kind === 'fence' ? slot.info : ''
const body = joinedWithin(separatedBy('\n', texts.map((text) => fencePieces(info, text))))
return boundedThen(body, (spelled) => directivePair(node, opener.value, spelled))
return success(directivePair(node, opener.value, texts.map((text) => fencedCodeBlock(info, text)).join('\n')))
}
// The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node.
@@ -349,55 +322,39 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
for (const [offset, item] of items.entries()) {
const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, writing)
if (!walk.ok) return walk
const inner = joinBlocks(walk.value.blocks, 'list-item')
if (inner === undefined) return tooLong()
headroom = Math.min(headroom, walk.value.headroom)
walked.push({ inner, node: item, walk: walk.value })
walked.push({ node: item, walk: walk.value })
}
const number = ordered ? (offset: number): string => `${start + offset}` : (): string => ''
return spelledOrDirectiveList(node, walked, number, { depth, headroom, path, writing })
}
// The list spelling where every item holds lines a list item spells, else the directive form.
function spelledOrDirectiveList(node: AdfNode, walked: readonly WalkedItem[], number: (offset: number) => string, at: { depth: number; headroom: number; path: ConvertErrorPath; writing: Writing }): Result<EmittedBlock> {
const ordered = node.type === 'orderedList'
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
const escapesMarker = (inner: string): boolean => flavourClaims[at.writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined
const inners = walked.map(({ inner }) => (escapesMarker(inner) ? `\\${inner}` : inner))
const list = spelledList(inners, number, ordered ? orderedDelimiters : bulletDelimiters, at.headroom)
if (list !== undefined) return list
const inners = walked.map((item) => {
const inner = joinBlocks(item.walk.blocks, 'list-item')
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
return flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner
})
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
if (list !== undefined) return success(list)
// The directive form spends a level the walk did not count.
if (at.headroom < 1) return tooDeep(at.path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, at.path, at.depth, () => directiveItems(walked, at.headroom - 1))
if (headroom < 1) return tooDeep(path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1 }))
}
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): Result<EmittedBlock> | undefined {
const pieces = listLines(inners, number, usual)
const alternatePieces = listLines(inners, number, alternate)
if (pieces === undefined || alternatePieces === undefined) return undefined
const withAlternate = (text: string): Result<EmittedBlock> => boundedThen(joinedWithin(alternatePieces), (alternateText) => success({ alternateText, headroom, spelling: 'list', text }))
return boundedThen(joinedWithin(pieces), withAlternate)
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): EmittedBlock | undefined {
const text = listLines(inners, number, usual)
const alternateText = listLines(inners, number, alternate)
return text === undefined || alternateText === undefined ? undefined : { alternateText, headroom, spelling: 'list', text }
}
// `undefined` where an item holds a line no list item spells.
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): Iterable<string> | undefined {
const items: Iterable<string>[] = []
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): string | undefined {
const lines: string[] = []
for (const [offset, inner] of inners.entries()) {
const pieces = listItemPieces(inner, `${number(offset)}${delimiter} `)
if (pieces === undefined) return undefined
items.push(pieces)
const line = tryListItemLines(inner, `${number(offset)}${delimiter} `)
if (line === undefined) return undefined
lines.push(line)
}
return separatedBy('\n', items)
return lines.join('\n')
}
function directiveItems(items: readonly WalkedItem[], headroom: number): Result<PlacedBlocks> {
const blocks: PlacedBlock[] = []
for (const item of items) {
const pair = boundedThen(joinBlocks(item.walk.blocks, 'directive'), (body) => directivePair(item.node, listItemOpener, body))
if (!pair.ok) return pair
blocks.push({ ...pair.value, node: item.node })
}
return success({ blocks, headroom })
function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] {
return items.map((item) => ({ ...directivePair(item.node, listItemOpener, joinBlocks(item.walk.blocks, 'directive')), node: item.node }))
}
function listStart(node: AdfNode, items: number): number | undefined {
@@ -407,25 +364,16 @@ function listStart(node: AdfNode, items: number): number | undefined {
return start + items - 1 > largestListMarker ? undefined : start
}
function listItemPieces(inner: string, marker: string): Iterable<string> | undefined {
if (inner === '') return [marker.trimEnd()]
const [first = '', ...rest] = inner.split('\n')
if (rest.some((line) => line !== '' && isBlankLine(line)) || (first !== '' && isBlankLine(first))) return undefined
function tryListItemLines(inner: string, marker: string): string | undefined {
if (inner === '') return marker.trimEnd()
const body = inner.split('\n')
if (body.some((line) => line !== '' && isBlankLine(line))) return undefined
const indent = ' '.repeat(marker.length)
const lines = body.map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
const first = lines[0] ?? ''
if (!isThematicBreak(first)) return lines.join('\n')
// An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text.
const rule = isThematicBreak(`${marker}${first}`)
if (rule && !inner.startsWith('*')) return undefined
return itemLinePieces(rule ? `${marker}\\` : marker, first, rest, ' '.repeat(marker.length))
}
function* itemLinePieces(opening: string, first: string, rest: readonly string[], indent: string): Generator<string> {
yield opening
yield first
for (const line of rest) {
yield '\n'
if (line === '') continue
yield indent
yield line
}
return inner.startsWith('*') ? [`${marker}\\${first.slice(marker.length)}`, ...lines.slice(1)].join('\n') : undefined
}
function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
@@ -1,10 +1,9 @@
import type { AdfNode } from '../../adf/document.ts'
import type { BlockNodeModel } from '../../adf/block-nodes.ts'
import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts'
import { attributePieces, isBareToken, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { blockArgument, markValues, marksAttribute } from '../block-directive.ts'
import { boundedThen, joinedWithin } from '../length-bound.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { overNested } from '../../json-value.ts'
import { spellEmptyKeys } from '../empty-keys.ts'
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
@@ -23,9 +22,7 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel,
if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path)
spelledPairs.push([marksAttribute, spellJsonAttribute(values)])
}
const head = spellDirectiveOpener(node.type, slot.argument, '')
const opener = joinedWithin(spelledPairs.length === 0 ? [head] : [head, ' ', ...attributePieces(spelledPairs)])
return boundedThen(opener, success)
return success(spellDirectiveOpener(node.type, slot.argument, spellAttributes(spelledPairs)))
}
// `undefined` where the argument slot holds a value no bare token spells.
+7 -15
View File
@@ -1,15 +1,14 @@
import type { AdfNode } from '../../adf/document.ts'
import type { ConvertErrorPath } from '../../result.ts'
import type { WrittenFlavour } from '../portable/conventions.ts'
import { boundedThen, joinedWithin, separatedBy } from '../length-bound.ts'
import { delimiterCells, pipeRowPieces } from '../pipe-table-syntax.ts'
import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts'
import { success, type ConvertErrorPath, type Result } from '../../result.ts'
import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts'
import { tryPipeCell } from './inline-line.ts'
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<string> | undefined {
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): string | undefined {
const rows = pipeRows(node)
if (rows === undefined) return undefined
const spelled: string[][] = []
const lines: string[] = []
for (const [rowIndex, row] of rows.entries()) {
const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) {
@@ -18,17 +17,10 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: Wri
if (line === undefined) return undefined
cells.push(line)
}
spelled.push(cells)
}
const text = joinedWithin(separatedBy('\n', tableRows(spelled)))
return boundedThen(text, success)
}
function* tableRows(rows: readonly string[][]): Generator<Iterable<string>> {
for (const [index, cells] of rows.entries()) {
yield pipeRowPieces(cells)
if (index === 0) yield pipeRowPieces(delimiterCells(cells.length))
lines.push(spellPipeRow(cells))
if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length))
}
return lines.join('\n')
}
function pipeRows(node: AdfNode): AdfNode[][] | undefined {
+5 -6
View File
@@ -2,7 +2,7 @@ import assert from 'node:assert/strict'
import test from 'node:test'
import type { Result } from '../result.ts'
import { boundedThen, joinedWithin, separatedBy, tooLong, withinEngineLimits } from './length-bound.ts'
import { joinedWithin, tooLong, withinEngineLimits, withinLongest } from './length-bound.ts'
import { success } from '../result.ts'
function thrown(name: string, message: string): () => Result<string> {
@@ -14,14 +14,13 @@ function thrown(name: string, message: string): () => Result<string> {
}
test('joins pieces up to the bound, and builds nothing past it', () => {
assert.equal(joinedWithin(separatedBy(', ', [['a', 'b'], ['c']])), 'ab, c')
assert.deepEqual(joinedWithin(['a', 'b']), success('ab'))
const half = 'a'.repeat(2 ** 28)
assert.equal(joinedWithin([half, half]), undefined)
assert.deepEqual(joinedWithin([half, half]), tooLong())
})
test('refuses where a bounded builder stopped, and goes on with what it built', () => {
assert.deepEqual(boundedThen(undefined, success), tooLong())
assert.deepEqual(boundedThen('x', (text) => success(`${text}!`)), success('x!'))
test('passes a text within the bound', () => {
assert.deepEqual(withinLongest('x'), success('x'))
})
test('refuses a string past the bound as too long on every engine, and names any other engine limit', () => {
+11 -21
View File
@@ -1,9 +1,9 @@
import { failure, type Result } from '../result.ts'
import { failure, success, type Result } from '../result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestMarkdown = 2 ** 29 - 24
// Measured on Firefox 155, Node 24 and Bun 1.4. The first two say a string passed longestMarkdown, so they refuse as tooLong; the others name the engine's own limit.
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the other two name the engine's own limit.
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
@@ -11,33 +11,23 @@ export function tooLong<T>(): Result<T> {
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest string V8 holds`, [])
}
// The pieces joined, or `undefined` once their length passes longestMarkdown, so nothing past it is built.
export function joinedWithin(pieces: Iterable<string>): string | undefined {
export function withinLongest(text: string): Result<string> {
return text.length > longestMarkdown ? tooLong() : success(text)
}
// Joins the pieces as they come, so a spelling that grows far past its input stops at the bound unbuilt.
export function joinedWithin(pieces: Iterable<string>): Result<string> {
const kept: string[] = []
let length = 0
for (const piece of pieces) {
length += piece.length
if (length > longestMarkdown) return undefined
if (length > longestMarkdown) return tooLong()
kept.push(piece)
}
return kept.join('')
return success(kept.join(''))
}
// The refusal where a bounded builder stopped, else what `next` makes of what it built.
export function boundedThen<T, U>(built: T | undefined, next: (built: T) => Result<U>): Result<U> {
return built === undefined ? tooLong() : next(built)
}
export function* separatedBy(separator: string, parts: Iterable<Iterable<string>>): Generator<string> {
let first = true
for (const part of parts) {
if (!first) yield separator
first = false
yield* part
}
}
// A builder this module does not bound can still pass an engine's longest string or array, so this wraps a whole conversion.
// Any concatenation can pass an engine's longest string or array, so this wraps a whole conversion.
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try {
return convert()
+5 -5
View File
@@ -2,11 +2,11 @@ import type { AdfNode } from '../adf/document.ts'
import type { DirectiveSpan, Read } from './directive-syntax.ts'
import type { JsonSpelling } from '../canonical-json.ts'
import type { JsonValue } from '../json-value.ts'
import { boundedThen, joinedWithin } from './length-bound.ts'
import { joinedWithin } from './length-bound.ts'
import { canonicalJsonPieces, spellsCanonicalJson } from '../canonical-json.ts'
import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../result.ts'
import { fencePieces } from './commonmark/backtick-runs.ts'
import { fencedCodeBlock } from './commonmark/backtick-runs.ts'
import { infoStringCarries } from './commonmark/grammar.ts'
import { isAdfNode } from '../adf/document.ts'
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
@@ -25,8 +25,7 @@ export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: numbe
const named = infoStringCarries(type)
const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth)
if (!json.ok) return json
const text = joinedWithin(fencePieces(`${carryFencePrefix}${named ? type : ''}`, json.value.json))
return boundedThen(text, (fence) => success({ headroom: json.value.headroom, text: fence }))
return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) })
}
export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> {
@@ -60,8 +59,9 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
}
// Two-space JSON indents every value its depth over, so it stops at the bound unbuilt.
const json = joinedWithin(canonicalJsonPieces(spelled, spelling))
return boundedThen(json, (text) => success({ headroom, json: text }))
return json.ok ? success({ headroom, json: json.value }) : json
}
// `type` is the one the fence's info string names, `undefined` for the inline carry.
+4 -9
View File
@@ -36,17 +36,12 @@ export function pipeCells(line: string): string[] | undefined {
return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined
}
export function delimiterCells(columns: number): string[] {
return Array.from({ length: columns }, () => '---')
export function spellPipeDelimiter(columns: number): string {
return spellPipeRow(Array.from({ length: columns }, () => '---'))
}
export function* pipeRowPieces(cells: readonly string[]): Generator<string> {
yield '|'
for (const cell of cells) {
yield ' '
yield cell
yield ' |'
}
export function spellPipeRow(cells: readonly string[]): string {
return `| ${cells.join(' | ')} |`
}
function splitPipeCells(line: string, start: number): string[] {
@@ -1,7 +1,7 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import { adfDocumentFault, heldTextLength, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts'
import { boundedThen, joinedWithin, longestMarkdown, tooLong, withinEngineLimits } from '../length-bound.ts'
import { longestMarkdown, tooLong, withinEngineLimits } from '../length-bound.ts'
import { centeredImage, externalMedia } from '../external-image.ts'
import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
@@ -60,7 +60,7 @@ export function reduceToPortable(document: AdfDocument): Result<AdfDocument> {
const fault = adfDocumentFault(document)
if (fault !== undefined) return faulted(fault, [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
// The reduction drops form, never text, so this much text cannot fit, and none of it is joined.
// Refused on the caller's text, edge whitespace and all: merging it first would build past the bound.
if (heldTextLength(nodeContent(document)) > longestMarkdown) return tooLong()
// Two documents the editor holds equal write the same portable markdown (docs/decisions.md §Equality is deep).
const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] })
@@ -138,14 +138,14 @@ function portableSequence(blocks: readonly Sourced[], reduction: Reduction): Res
continue
}
// Back on `pending`: a replacement may take the directive form itself.
const replacement = block === next.block ? itemBlocks(block, next.path) : [{ block, path: next.path }]
const replacement = block === next.block ? listItemBlocks(block, next.path) : [{ block, path: next.path }]
for (const node of replacement.reverse()) pending.push(node)
}
return success(sequence)
}
// Offsets are the reduced item's, which are the caller's while the item's blocks reduce one to one.
function itemBlocks(list: AdfNode, path: ConvertErrorPath): Sourced[] {
function listItemBlocks(list: AdfNode, path: ConvertErrorPath): Sourced[] {
const blocks: Sourced[] = []
for (const [index, item] of nodeContent(list).entries()) {
for (const [offset, block] of nodeContent(item).entries()) blocks.push({ block, path: [...path, 'content', index, 'content', offset] })
@@ -231,11 +231,10 @@ function reduceHeading(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
function reduceCodeBlock(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const leaves = inlineLeaves(nodeContent(node), 'paragraph', reduction.path, reduction.depth)
if (!leaves.ok) return leaves
const code = leaves.value.map((leaf) => leaf.text ?? '\n').join('')
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims.portable)
return boundedThen(joinedWithin(leaves.value.map((leaf) => leaf.text ?? '\n')), (code) => {
const block: AdfNode = { content: code === '' ? [] : [text(code)], type: 'codeBlock' }
return success([slot.kind === 'fence' ? { ...block, attrs: { language: slot.info } } : block])
})
const block: AdfNode = { content: code === '' ? [] : [text(code)], type: 'codeBlock' }
return success([slot.kind === 'fence' ? { ...block, attrs: { language: slot.info } } : block])
}
function reduceList(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
+4 -5
View File
@@ -81,9 +81,7 @@ The line escaper keeps an array element per character twice (`escapings`, `place
50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a
directive's CommonMark ask. Bound the line as the blocks are (`joinedWithin`): its segments and its
escaped output, which today pass V8's longest string only by throwing. Found while shipping item
106, 2026-10-11.
directive's CommonMark ask. Found while shipping item 106, 2026-10-11.
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.
@@ -215,8 +213,9 @@ Each level of a nested list or quote re-indents its whole text, so the work is t
text. `losslessMarkdownToAdf` on 3 MB of lazy lines in a list nested 490 levels inside
`!adf:blockquote` spends about 5 seconds re-spelling it. The same 2 MB split over two items of a
list nested 480 levels took 11 minutes to refuse on Node 24. A deep subtree past the length bound
refuses only at the level that passes it, after every level below re-indented its text. The writers
do the same on such a document. Found while shipping item 106, 2026-10-11.
refuses only once every level below has re-indented its text, so no test reaches
`losslessMarkdownToAdf`'s length refusal in seconds: add one with this fix. The writers do the same
on such a document. Found while shipping item 106, 2026-10-11.
### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.