3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-13 04:08:29 +01:00

Answer architecture round 5: the builders keep main's spelling, each writer's result and the CommonMark ask's spelling are held to the bound once built, and the engines' own overflows refuse alike

This commit is contained in:
2026-10-12 21:36:11 +02:00
parent acd22aa49a
commit 1f4551a72a
16 changed files with 165 additions and 228 deletions
+1 -1
View File
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do | | Code | Fires when | What you can do |
| --- | --- | --- | | --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only | | `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is, edge whitespace included. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only |
## The guarantees ## The guarantees
+25
View File
@@ -602,3 +602,28 @@ Inherited architect, decided by:
- Navigation, Shape: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long symptom leads straight to the module named for it - Navigation, Shape: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long symptom leads straight to the module named for it
- Shape: `src/markdown/directive-syntax.ts:133` `spellDirectiveOpener`: each spelling is a string function and a pieces generator - Shape: `src/markdown/directive-syntax.ts:133` `spellDirectiveOpener`: each spelling is a string function and a pieces generator
- Shape: `src/markdown/opaque-carry.ts:64` `carriedJson`: the lambda parameter `spelled` shadows the function's own - Shape: `src/markdown/opaque-carry.ts:64` `carriedJson`: the lambda parameter `spelled` shadows the function's own
## 2026-10-12T16:40:52Z, PR #36 at acd22aa, against 7a27f6b
Ruling: worse
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | worse | worse | same | worse |
| Inherited architect | better | worse | worse | same | worse |
Mid A, decided by:
- Locality, Shape: `src/markdown/emit/adf-to-markdown.ts:374` `spelledList`: a builder's `undefined` now means either no list spelling or past the length bound, and every caller of `joinBlocks` must remember which it holds
- Shape: `src/markdown/emit/adf-to-markdown.ts:383` `listLines`: split, map and join became generators yielding pieces into `separatedBy` and `joinedWithin`, so a reader rebuilds the string in their head
- Locality: `src/markdown/emit/adf-to-markdown.ts:125` `readableBlock`: the memo is filled in `commonMarkSpelling` and deleted on read, a one-shot cache whose hits depend on which walk came first
- Navigation: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long refusal leads to a module named for it
Inherited architect, decided by:
- Locality, Shape: `src/markdown/emit/adf-to-markdown.ts:374` `spelledList`: `undefined` means two things a few lines apart, told apart only by the side of a nested `boundedThen` callback
- Locality: `src/markdown/emit/adf-to-markdown.ts:125` `readableBlock`: reading the memo is a mutation, and a second read spells again
- Shape: `src/markdown/pipe-table-syntax.ts:43` `pipeRowPieces`: one-line templates became generators whose output cannot be read off one line
- Locality, Shape: `src/markdown/emit/block-directive-spelling.ts:27` `spellBlockDirectiveOpener`: the opener's layout is known here and in `spellDirectiveOpener`
- Shape: `src/markdown/portable/adf-to-portable-markdown.ts:231` `reduceCodeBlock`: an ADF-to-ADF reducer carries a markdown length bound through a callback
- Navigation: `src/markdown/length-bound.ts:41` `withinEngineLimits`: a too-long refusal leads straight to one small, well-named module
+13 -11
View File
@@ -666,18 +666,20 @@ is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document (`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
converts alike everywhere. Each builder joins its pieces — blocks, quoted and list lines, directive converts alike everywhere. A writer first refuses a document whose text alone passes the bound: the
pairs and openers, fences, pipe rows, the carry's JSON — through `joinedWithin`, which stops once lossless writer spells every character of it, and the portable reduction refuses on the caller's
they pass the bound, so none builds a string no engine was asked for. A writer first refuses a text, edge whitespace and all, since merging that text first could build past the bound. The
document whose text alone passes the bound, since every character of it is spelled at least once. A builders join as they always did, and the writer's result and the CommonMark ask's spelling are held
deep list or quote re-indents its whole text at each level until the level that passes the bound to the bound once built. V8 stops at the bound itself and SpiderMonkey at its own longest string,
refuses (`todo.md` item 109). and `withinEngineLimits` refuses both throws as the same `tooLong`, so every engine answers alike.
The carry's two-space JSON, which indents every value its depth over, stops at the bound unbuilt
(`joinedWithin`). A deep list or quote re-indents its whole text at each level before the bound
refuses it (`todo.md` item 109).
A line is held to its text's length alone; what its spelling adds — escapes, link targets, JavaScriptCore builds up to its own longest string, 2^31 − 1, before the check. A throw past that,
attributes — passes V8's longest string only by throwing, and `withinEngineLimits` refuses that past any engine's longest array, or JavaScriptCore's `Out of memory`, which also reports a heap
throw, and SpiderMonkey's, as the same refusal: both mean a string past the bound. `todo.md` item running short, is refused with the engine's message. `todo.md` item 108 bounds the line's own
108 bounds the line itself. The engines' throws past their own longest array, and JavaScriptCore's memory.
`Out of memory`, which also reports a heap running short, stay refused with the engine's message.
## Nothing recurses unbounded ## Nothing recurses unbounded
+3 -3
View File
@@ -25,7 +25,7 @@ import {
spellStringAttribute, spellStringAttribute,
spellVocabulary, spellVocabulary,
} from '../markdown/directive-syntax.ts' } from '../markdown/directive-syntax.ts'
import { fencePieces } from '../markdown/commonmark/backtick-runs.ts' import { fencedCodeBlock } from '../markdown/commonmark/backtick-runs.ts'
import { inlineNodes } from '../adf/inline-nodes.ts' import { inlineNodes } from '../adf/inline-nodes.ts'
import { markAttributes } from '../adf/mark-attributes.ts' import { markAttributes } from '../adf/mark-attributes.ts'
import { markSpelling } from '../markdown/mark-spellings.ts' import { markSpelling } from '../markdown/mark-spellings.ts'
@@ -174,8 +174,8 @@ const carriedNode = fc.oneof(
const spellCarry = (json: string) => spellInlineLeafDirective(carryName, `{json=${spellStringAttribute(json)}}`) const spellCarry = (json: string) => spellInlineLeafDirective(carryName, `{json=${spellStringAttribute(json)}}`)
const inlineCarry = carriedNode.map((node) => spellCarry(serializeCanonicalJson(node, 'compact'))) const inlineCarry = carriedNode.map((node) => spellCarry(serializeCanonicalJson(node, 'compact')))
const hostileInlineCarry = fc.oneof(carriedNode, jsonValue).map((node) => spellCarry(JSON.stringify(node, null, 1))) const hostileInlineCarry = fc.oneof(carriedNode, jsonValue).map((node) => spellCarry(JSON.stringify(node, null, 1)))
const blockCarry = carriedNode.map((node) => fencePieces(carryName, serializeCanonicalJson(node, 'two-space')).join('')) const blockCarry = carriedNode.map((node) => fencedCodeBlock(carryName, serializeCanonicalJson(node, 'two-space')))
const hostileBlockCarry = fc.oneof(carriedNode, jsonValue).map((node) => fencePieces(carryName, JSON.stringify(node)).join('')) const hostileBlockCarry = fc.oneof(carriedNode, jsonValue).map((node) => fencedCodeBlock(carryName, JSON.stringify(node)))
function prefixLines(body: string, first: string, rest: (index: number) => string): string { function prefixLines(body: string, first: string, rest: (index: number) => string): string {
return body return body
+2 -2
View File
@@ -15,9 +15,9 @@ export function closingBacktickRun(text: string, from: number, opener: number):
return undefined return undefined
} }
export function fencePieces(info: string, body: string): string[] { export function fencedCodeBlock(info: string, body: string): string {
const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1)) const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1))
return body === '' ? [fence, info, '\n', fence] : [fence, info, '\n', body, '\n', fence] return body === '' ? `${fence}${info}\n${fence}` : `${fence}${info}\n${body}\n${fence}`
} }
export function longestBacktickRun(text: string): number { export function longestBacktickRun(text: string): number {
+3 -12
View File
@@ -112,18 +112,9 @@ export function slotFault(type: string, text: string): ConvertFault | undefined
} }
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string { export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
return [...attributePieces(pairs)].join('') if (pairs.length === 0) return ''
} const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`)
return `{${spelled.join(' ')}}`
export function* attributePieces(pairs: readonly (readonly [string, string])[]): Generator<string> {
if (pairs.length === 0) return
yield '{'
for (const [index, [key, value]] of [...pairs].sort(([left], [right]) => keyOrder(left, right)).entries()) {
if (index > 0) yield ' '
yield `${key}=`
yield value
}
yield '}'
} }
export function spellDirectiveCloser(name: string): string { export function spellDirectiveCloser(name: string): string {
+1 -1
View File
@@ -46,7 +46,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds' const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds'
// A code block of 300 lines, two of them some 1000 characters short of V8's longest string, so only what their spelling adds passes it. // A code block of 300 lines; two of these together fall some 1000 characters short of V8's longest string, so only what their spelling adds passes it.
function nearlyHalfTheBound(): AdfNode { function nearlyHalfTheBound(): AdfNode {
return { attrs: {}, content: [{ text: `${'a'.repeat(894_782)}\n`.repeat(300), type: 'text' }], type: 'codeBlock' } return { attrs: {}, content: [{ text: `${'a'.repeat(894_782)}\n`.repeat(300), type: 'text' }], type: 'codeBlock' }
} }
+72 -124
View File
@@ -5,14 +5,14 @@ import { adfDocumentFault, heldTextLength, holdsOnlyAttributes, isUnmarkedSpella
import { alertMarker, flavourClaims, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../portable/conventions.ts' import { alertMarker, flavourClaims, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../portable/conventions.ts'
import { blockDirectiveForm, documentSpelling } from '../block-directive.ts' import { blockDirectiveForm, documentSpelling } from '../block-directive.ts'
import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts' import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts'
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong, withinEngineLimits } from '../length-bound.ts'
import { carriedBlock } from '../opaque-carry.ts' import { carriedBlock } from '../opaque-carry.ts'
import { emitInlineLine } from './inline-line.ts' import { emitInlineLine } from './inline-line.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { fencePieces } from '../commonmark/backtick-runs.ts' import { fencedCodeBlock } from '../commonmark/backtick-runs.ts'
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts' import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
import { languageSlot, type LanguageSlot } from '../code-language.ts' import { languageSlot, type LanguageSlot } from '../code-language.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { longestMarkdown, tooLong, withinEngineLimits, withinLongest } from '../length-bound.ts'
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts' import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts' import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
import { tryImage } from './image.ts' import { tryImage } from './image.ts'
@@ -28,9 +28,9 @@ type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number }
// CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`. // CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`.
type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string } type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string }
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node. // Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
// Holds the spelling of each node the CommonMark ask was put to, until the walk above reads it.
export type SpellingMemo = Map<AdfNode, KeptSpelling> export type SpellingMemo = Map<AdfNode, KeptSpelling>
// inner: the item's blocks joined, for the list form; walk: the blocks themselves, for the directive form. type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
type WalkedItem = { inner: string; node: AdfNode; walk: PlacedBlocks }
export type Writing = { flavour: WrittenFlavour; memo: SpellingMemo | undefined } export type Writing = { flavour: WrittenFlavour; memo: SpellingMemo | undefined }
const bulletDelimiters: Delimiters = ['-', '*'] const bulletDelimiters: Delimiters = ['-', '*']
@@ -52,8 +52,8 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
if (heldTextLength(document.content) > longestMarkdown) return tooLong() if (heldTextLength(document.content) > longestMarkdown) return tooLong()
const walk = walkBlocks(document.content, [], 0, { flavour, memo: undefined }) const walk = walkBlocks(document.content, [], 0, { flavour, memo: undefined })
if (!walk.ok) return walk if (!walk.ok) return walk
const text = joinedWithin(walk.value.blocks.length === 0 ? [] : [...blockPieces(walk.value.blocks, 'document'), '\n']) const text = joinBlocks(walk.value.blocks, 'document')
return boundedThen(text, success) return withinLongest(text === '' ? '' : `${text}\n`)
} }
function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> { function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
@@ -73,18 +73,16 @@ function tooDeep(path: ConvertErrorPath): Result<never> {
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path) return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
} }
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string | undefined { function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string {
return joinedWithin(blockPieces(blocks, container)) let text = ''
}
function* blockPieces(blocks: readonly PlacedBlock[], container: BlockContainer): Generator<string> {
let alternated = false let alternated = false
for (const [index, block] of blocks.entries()) { for (const [index, block] of blocks.entries()) {
const previous = blocks[index - 1] const previous = blocks[index - 1]
if (previous !== undefined) yield separationBetween(previous, block, container) if (previous !== undefined) text += separationBetween(previous, block, container)
alternated = !alternated && previous !== undefined && sharesMarker(previous, block) alternated = !alternated && previous !== undefined && sharesMarker(previous, block)
yield alternated && block.spelling === 'list' ? block.alternateText : block.text text += alternated && block.spelling === 'list' ? block.alternateText : block.text
} }
return text
} }
// A task list is spelled as a bullet list. // A task list is spelled as a bullet list.
@@ -119,6 +117,8 @@ export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth:
else if (readable.ok) writing.memo.set(node, { block: readable.value, depth }) else if (readable.ok) writing.memo.set(node, { block: readable.value, depth })
if (readable === undefined) return undefined if (readable === undefined) return undefined
if (!readable.ok) return readable if (!readable.ok) return readable
// V8 throws past the bound while building the spelling; an engine holding longer strings refuses here alike.
if (readable.value.text.length > longestMarkdown) return tooLong()
return readable.value.spelling === 'directive' ? undefined : success(null) return readable.value.spelling === 'directive' ? undefined : success(null)
} }
@@ -139,15 +139,12 @@ function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
if (portable !== undefined) return portable if (portable !== undefined) return portable
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing) if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing) if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
if (node.type === 'codeBlock') return tryCodeBlock(node, writing.flavour) if (node.type === 'codeBlock') return readableText(tryCodeBlock(node, writing.flavour))
if (node.type === 'heading') return tryHeading(node, path, writing.flavour) if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
if (node.type === 'mediaSingle') return readableText(tryImage(node)) if (node.type === 'mediaSingle') return readableText(tryImage(node))
if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour) if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
if (node.type === 'rule') return readableText(tryRule(node)) if (node.type === 'rule') return readableText(tryRule(node))
if (node.type === 'table') { if (node.type === 'table') return readableText(tryPipeTable(node, path, writing.flavour))
const table = tryPipeTable(node, path, writing.flavour)
return table === undefined || !table.ok ? table : success(commonMarkText(table.value))
}
return undefined return undefined
} }
@@ -165,25 +162,15 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> { function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing) const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
if (!inner.ok) return inner if (!inner.ok) return inner
const text = joinedWithin(inner.value.blocks.length === 0 ? [head] : [head, '\n\n', ...blockPieces(inner.value.blocks, 'document')]) const body = joinBlocks(inner.value.blocks, 'document')
return quotedBlock(text, inner.value.headroom) return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
} }
function quotedBlock(text: string | undefined, headroom: number): Result<EmittedBlock> { function quoted(text: string): string {
const quoted = text === undefined ? undefined : joinedWithin(quotedPieces(text)) return text
return boundedThen(quoted, (spelled) => success(commonMarkText(spelled, headroom))) .split('\n')
} .map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n')
function* quotedPieces(text: string): Generator<string> {
for (let start = 0; ; ) {
const end = text.indexOf('\n', start)
const line = text.slice(start, end === -1 ? text.length : end)
yield line === '' ? '>' : '> '
yield line
if (end === -1) return
yield '\n'
start = end + 1
}
} }
function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> { function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
@@ -199,21 +186,14 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
if (task || previous === undefined) items.push([...walk.value.blocks]) if (task || previous === undefined) items.push([...walk.value.blocks])
else for (const block of walk.value.blocks) previous.push(block) else for (const block of walk.value.blocks) previous.push(block)
} }
const inners = joinedItems(items.map((blocks) => blockPieces(blocks, 'list-item'))) const list = spelledList(
items.map((blocks) => joinBlocks(blocks, 'list-item')),
() => '',
bulletDelimiters,
headroom,
)
// The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour. // The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour.
const unspellable = failure<EmittedBlock>('unsupported-node-shape', 'a task holds blocks no list item spells', path) return list === undefined ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success(list)
return boundedThen(inners, (spelled) => spelledList(spelled, () => '', bulletDelimiters, headroom) ?? unspellable)
}
// Each item's text, or `undefined` where one would pass the bound.
function joinedItems(items: readonly Iterable<string>[]): string[] | undefined {
const inners: string[] = []
for (const pieces of items) {
const inner = joinedWithin(pieces)
if (inner === undefined) return undefined
inners.push(inner)
}
return inners
} }
function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> { function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
@@ -229,15 +209,13 @@ function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writin
const content = nodeContent(task) const content = nodeContent(task)
const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour) const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour)
if (!line.ok) return line if (!line.ok) return line
const text = joinedWithin(line.value === '' ? [marker] : [marker, ' ', line.value]) return success({ blocks: [{ ...markerBlock, text: line.value === '' ? marker : `${marker} ${line.value}` }], headroom: Number.POSITIVE_INFINITY })
return boundedThen(text, (spelled) => success({ blocks: [{ ...markerBlock, text: spelled }], headroom: Number.POSITIVE_INFINITY }))
} }
const walk = walkBlocks(nodeContent(task), path, depth, writing) const walk = walkBlocks(nodeContent(task), path, depth, writing)
if (!walk.ok) return walk if (!walk.ok) return walk
const [first, ...rest] = walk.value.blocks const [first, ...rest] = walk.value.blocks
if (first?.node.type !== 'paragraph') return success({ blocks: [markerBlock, ...walk.value.blocks], headroom: walk.value.headroom }) const blocks = first?.node.type === 'paragraph' ? [{ ...first, text: `${marker} ${first.text}` }, ...rest] : [markerBlock, ...walk.value.blocks]
const text = joinedWithin([marker, ' ', first.text]) return success({ blocks, headroom: walk.value.headroom })
return boundedThen(text, (spelled) => success({ blocks: [{ ...first, text: spelled }, ...rest], headroom: walk.value.headroom }))
} }
function readableText(text: string | undefined): Result<EmittedBlock> | undefined { function readableText(text: string | undefined): Result<EmittedBlock> | undefined {
@@ -253,10 +231,8 @@ function commonMarkText(text: string, headroom: number = Number.POSITIVE_INFINIT
return { headroom, spelling: 'commonmark', text } return { headroom, spelling: 'commonmark', text }
} }
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): Result<EmittedBlock> { function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): EmittedBlock {
const closer = spellDirectiveCloser(node.type) return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` }
const text = joinedWithin(body === '' ? [opener, '\n', closer] : [opener, '\n', body, '\n', closer])
return boundedThen(text, (spelled) => success({ headroom, spelling: 'directive', text: spelled }))
} }
function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> { function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> {
@@ -274,12 +250,11 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
if (model.contentModel === 'inline') { if (model.contentModel === 'inline') {
const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless') const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless')
if (!line.ok) return line if (!line.ok) return line
return directivePair(node, opener, line.value) return success(directivePair(node, opener, line.value))
} }
const walk = walkBody() const walk = walkBody()
if (!walk.ok) return walk if (!walk.ok) return walk
const body = joinBlocks(walk.value.blocks, 'directive') return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom))
return boundedThen(body, (spelled) => directivePair(node, opener, spelled, walk.value.headroom))
} }
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
@@ -287,18 +262,17 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri
const inner = walkBlocks(nodeContent(node), path, depth + 1, writing) const inner = walkBlocks(nodeContent(node), path, depth + 1, writing)
if (!inner.ok) return inner if (!inner.ok) return inner
const text = joinBlocks(inner.value.blocks, 'document') const text = joinBlocks(inner.value.blocks, 'document')
const alert = text !== undefined && flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
return quotedBlock(alert ? joinedWithin(['\\', text]) : text, inner.value.headroom) return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom))
} }
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): Result<EmittedBlock> | undefined { function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): string | undefined {
if (!holdsOnlyAttributes(node, ['language'])) return undefined if (!holdsOnlyAttributes(node, ['language'])) return undefined
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour]) const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour])
if (slot.kind === 'attribute') return undefined if (slot.kind === 'attribute') return undefined
const [only, ...others] = fencedTexts(node) ?? [] const [only, ...others] = fencedTexts(node) ?? []
if (only === undefined || others.length > 0) return undefined if (only === undefined || others.length > 0) return undefined
const text = joinedWithin(fencePieces(slot.kind === 'fence' ? slot.info : '', only)) return fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only)
return boundedThen(text, (spelled) => success(commonMarkText(spelled)))
} }
// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does. // spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does.
@@ -310,8 +284,7 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr
if (opener === undefined) return carriedFence(carriedBlock(node, path, depth)) if (opener === undefined) return carriedFence(carriedBlock(node, path, depth))
if (!opener.ok) return opener if (!opener.ok) return opener
const info = slot.kind === 'fence' ? slot.info : '' const info = slot.kind === 'fence' ? slot.info : ''
const body = joinedWithin(separatedBy('\n', texts.map((text) => fencePieces(info, text)))) return success(directivePair(node, opener.value, texts.map((text) => fencedCodeBlock(info, text)).join('\n')))
return boundedThen(body, (spelled) => directivePair(node, opener.value, spelled))
} }
// The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node. // The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node.
@@ -349,55 +322,39 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
for (const [offset, item] of items.entries()) { for (const [offset, item] of items.entries()) {
const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, writing) const walk = walkBlocks(nodeContent(item), [...path, 'content', offset], depth + 1, writing)
if (!walk.ok) return walk if (!walk.ok) return walk
const inner = joinBlocks(walk.value.blocks, 'list-item')
if (inner === undefined) return tooLong()
headroom = Math.min(headroom, walk.value.headroom) headroom = Math.min(headroom, walk.value.headroom)
walked.push({ inner, node: item, walk: walk.value }) walked.push({ node: item, walk: walk.value })
} }
const number = ordered ? (offset: number): string => `${start + offset}` : (): string => '' const inners = walked.map((item) => {
return spelledOrDirectiveList(node, walked, number, { depth, headroom, path, writing }) const inner = joinBlocks(item.walk.blocks, 'list-item')
} // GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
return flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner
// The list spelling where every item holds lines a list item spells, else the directive form. })
function spelledOrDirectiveList(node: AdfNode, walked: readonly WalkedItem[], number: (offset: number) => string, at: { depth: number; headroom: number; path: ConvertErrorPath; writing: Writing }): Result<EmittedBlock> { const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
const ordered = node.type === 'orderedList' if (list !== undefined) return success(list)
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
const escapesMarker = (inner: string): boolean => flavourClaims[at.writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined
const inners = walked.map(({ inner }) => (escapesMarker(inner) ? `\\${inner}` : inner))
const list = spelledList(inners, number, ordered ? orderedDelimiters : bulletDelimiters, at.headroom)
if (list !== undefined) return list
// The directive form spends a level the walk did not count. // The directive form spends a level the walk did not count.
if (at.headroom < 1) return tooDeep(at.path) if (headroom < 1) return tooDeep(path)
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, at.path, at.depth, () => directiveItems(walked, at.headroom - 1)) return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1 }))
} }
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): Result<EmittedBlock> | undefined { function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): EmittedBlock | undefined {
const pieces = listLines(inners, number, usual) const text = listLines(inners, number, usual)
const alternatePieces = listLines(inners, number, alternate) const alternateText = listLines(inners, number, alternate)
if (pieces === undefined || alternatePieces === undefined) return undefined return text === undefined || alternateText === undefined ? undefined : { alternateText, headroom, spelling: 'list', text }
const withAlternate = (text: string): Result<EmittedBlock> => boundedThen(joinedWithin(alternatePieces), (alternateText) => success({ alternateText, headroom, spelling: 'list', text }))
return boundedThen(joinedWithin(pieces), withAlternate)
} }
// `undefined` where an item holds a line no list item spells. function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): string | undefined {
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): Iterable<string> | undefined { const lines: string[] = []
const items: Iterable<string>[] = []
for (const [offset, inner] of inners.entries()) { for (const [offset, inner] of inners.entries()) {
const pieces = listItemPieces(inner, `${number(offset)}${delimiter} `) const line = tryListItemLines(inner, `${number(offset)}${delimiter} `)
if (pieces === undefined) return undefined if (line === undefined) return undefined
items.push(pieces) lines.push(line)
} }
return separatedBy('\n', items) return lines.join('\n')
} }
function directiveItems(items: readonly WalkedItem[], headroom: number): Result<PlacedBlocks> { function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] {
const blocks: PlacedBlock[] = [] return items.map((item) => ({ ...directivePair(item.node, listItemOpener, joinBlocks(item.walk.blocks, 'directive')), node: item.node }))
for (const item of items) {
const pair = boundedThen(joinBlocks(item.walk.blocks, 'directive'), (body) => directivePair(item.node, listItemOpener, body))
if (!pair.ok) return pair
blocks.push({ ...pair.value, node: item.node })
}
return success({ blocks, headroom })
} }
function listStart(node: AdfNode, items: number): number | undefined { function listStart(node: AdfNode, items: number): number | undefined {
@@ -407,25 +364,16 @@ function listStart(node: AdfNode, items: number): number | undefined {
return start + items - 1 > largestListMarker ? undefined : start return start + items - 1 > largestListMarker ? undefined : start
} }
function listItemPieces(inner: string, marker: string): Iterable<string> | undefined { function tryListItemLines(inner: string, marker: string): string | undefined {
if (inner === '') return [marker.trimEnd()] if (inner === '') return marker.trimEnd()
const [first = '', ...rest] = inner.split('\n') const body = inner.split('\n')
if (rest.some((line) => line !== '' && isBlankLine(line)) || (first !== '' && isBlankLine(first))) return undefined if (body.some((line) => line !== '' && isBlankLine(line))) return undefined
const indent = ' '.repeat(marker.length)
const lines = body.map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
const first = lines[0] ?? ''
if (!isThematicBreak(first)) return lines.join('\n')
// An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text. // An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text.
const rule = isThematicBreak(`${marker}${first}`) return inner.startsWith('*') ? [`${marker}\\${first.slice(marker.length)}`, ...lines.slice(1)].join('\n') : undefined
if (rule && !inner.startsWith('*')) return undefined
return itemLinePieces(rule ? `${marker}\\` : marker, first, rest, ' '.repeat(marker.length))
}
function* itemLinePieces(opening: string, first: string, rest: readonly string[], indent: string): Generator<string> {
yield opening
yield first
for (const line of rest) {
yield '\n'
if (line === '') continue
yield indent
yield line
}
} }
function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined { function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
@@ -1,10 +1,9 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfNode } from '../../adf/document.ts'
import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts'
import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts'
import { attributePieces, isBareToken, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { blockArgument, markValues, marksAttribute } from '../block-directive.ts' import { blockArgument, markValues, marksAttribute } from '../block-directive.ts'
import { boundedThen, joinedWithin } from '../length-bound.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
import { overNested } from '../../json-value.ts' import { overNested } from '../../json-value.ts'
import { spellEmptyKeys } from '../empty-keys.ts' import { spellEmptyKeys } from '../empty-keys.ts'
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
@@ -23,9 +22,7 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel,
if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path) if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path)
spelledPairs.push([marksAttribute, spellJsonAttribute(values)]) spelledPairs.push([marksAttribute, spellJsonAttribute(values)])
} }
const head = spellDirectiveOpener(node.type, slot.argument, '') return success(spellDirectiveOpener(node.type, slot.argument, spellAttributes(spelledPairs)))
const opener = joinedWithin(spelledPairs.length === 0 ? [head] : [head, ' ', ...attributePieces(spelledPairs)])
return boundedThen(opener, success)
} }
// `undefined` where the argument slot holds a value no bare token spells. // `undefined` where the argument slot holds a value no bare token spells.
+7 -15
View File
@@ -1,15 +1,14 @@
import type { AdfNode } from '../../adf/document.ts' import type { AdfNode } from '../../adf/document.ts'
import type { ConvertErrorPath } from '../../result.ts'
import type { WrittenFlavour } from '../portable/conventions.ts' import type { WrittenFlavour } from '../portable/conventions.ts'
import { boundedThen, joinedWithin, separatedBy } from '../length-bound.ts'
import { delimiterCells, pipeRowPieces } from '../pipe-table-syntax.ts'
import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts' import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts'
import { success, type ConvertErrorPath, type Result } from '../../result.ts' import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts'
import { tryPipeCell } from './inline-line.ts' import { tryPipeCell } from './inline-line.ts'
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<string> | undefined { export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): string | undefined {
const rows = pipeRows(node) const rows = pipeRows(node)
if (rows === undefined) return undefined if (rows === undefined) return undefined
const spelled: string[][] = [] const lines: string[] = []
for (const [rowIndex, row] of rows.entries()) { for (const [rowIndex, row] of rows.entries()) {
const cells: string[] = [] const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) { for (const [cellIndex, paragraph] of row.entries()) {
@@ -18,17 +17,10 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: Wri
if (line === undefined) return undefined if (line === undefined) return undefined
cells.push(line) cells.push(line)
} }
spelled.push(cells) lines.push(spellPipeRow(cells))
} if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length))
const text = joinedWithin(separatedBy('\n', tableRows(spelled)))
return boundedThen(text, success)
}
function* tableRows(rows: readonly string[][]): Generator<Iterable<string>> {
for (const [index, cells] of rows.entries()) {
yield pipeRowPieces(cells)
if (index === 0) yield pipeRowPieces(delimiterCells(cells.length))
} }
return lines.join('\n')
} }
function pipeRows(node: AdfNode): AdfNode[][] | undefined { function pipeRows(node: AdfNode): AdfNode[][] | undefined {
+5 -6
View File
@@ -2,7 +2,7 @@ import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import type { Result } from '../result.ts' import type { Result } from '../result.ts'
import { boundedThen, joinedWithin, separatedBy, tooLong, withinEngineLimits } from './length-bound.ts' import { joinedWithin, tooLong, withinEngineLimits, withinLongest } from './length-bound.ts'
import { success } from '../result.ts' import { success } from '../result.ts'
function thrown(name: string, message: string): () => Result<string> { function thrown(name: string, message: string): () => Result<string> {
@@ -14,14 +14,13 @@ function thrown(name: string, message: string): () => Result<string> {
} }
test('joins pieces up to the bound, and builds nothing past it', () => { test('joins pieces up to the bound, and builds nothing past it', () => {
assert.equal(joinedWithin(separatedBy(', ', [['a', 'b'], ['c']])), 'ab, c') assert.deepEqual(joinedWithin(['a', 'b']), success('ab'))
const half = 'a'.repeat(2 ** 28) const half = 'a'.repeat(2 ** 28)
assert.equal(joinedWithin([half, half]), undefined) assert.deepEqual(joinedWithin([half, half]), tooLong())
}) })
test('refuses where a bounded builder stopped, and goes on with what it built', () => { test('passes a text within the bound', () => {
assert.deepEqual(boundedThen(undefined, success), tooLong()) assert.deepEqual(withinLongest('x'), success('x'))
assert.deepEqual(boundedThen('x', (text) => success(`${text}!`)), success('x!'))
}) })
test('refuses a string past the bound as too long on every engine, and names any other engine limit', () => { test('refuses a string past the bound as too long on every engine, and names any other engine limit', () => {
+11 -21
View File
@@ -1,9 +1,9 @@
import { failure, type Result } from '../result.ts' import { failure, success, type Result } from '../result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine). // V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestMarkdown = 2 ** 29 - 24 export const longestMarkdown = 2 ** 29 - 24
// Measured on Firefox 155, Node 24 and Bun 1.4. The first two say a string passed longestMarkdown, so they refuse as tooLong; the others name the engine's own limit. // Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the other two name the engine's own limit.
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length'] const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory'] const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
@@ -11,33 +11,23 @@ export function tooLong<T>(): Result<T> {
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest string V8 holds`, []) return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest string V8 holds`, [])
} }
// The pieces joined, or `undefined` once their length passes longestMarkdown, so nothing past it is built. export function withinLongest(text: string): Result<string> {
export function joinedWithin(pieces: Iterable<string>): string | undefined { return text.length > longestMarkdown ? tooLong() : success(text)
}
// Joins the pieces as they come, so a spelling that grows far past its input stops at the bound unbuilt.
export function joinedWithin(pieces: Iterable<string>): Result<string> {
const kept: string[] = [] const kept: string[] = []
let length = 0 let length = 0
for (const piece of pieces) { for (const piece of pieces) {
length += piece.length length += piece.length
if (length > longestMarkdown) return undefined if (length > longestMarkdown) return tooLong()
kept.push(piece) kept.push(piece)
} }
return kept.join('') return success(kept.join(''))
} }
// The refusal where a bounded builder stopped, else what `next` makes of what it built. // Any concatenation can pass an engine's longest string or array, so this wraps a whole conversion.
export function boundedThen<T, U>(built: T | undefined, next: (built: T) => Result<U>): Result<U> {
return built === undefined ? tooLong() : next(built)
}
export function* separatedBy(separator: string, parts: Iterable<Iterable<string>>): Generator<string> {
let first = true
for (const part of parts) {
if (!first) yield separator
first = false
yield* part
}
}
// A builder this module does not bound can still pass an engine's longest string or array, so this wraps a whole conversion.
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> { export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try { try {
return convert() return convert()
+5 -5
View File
@@ -2,11 +2,11 @@ import type { AdfNode } from '../adf/document.ts'
import type { DirectiveSpan, Read } from './directive-syntax.ts' import type { DirectiveSpan, Read } from './directive-syntax.ts'
import type { JsonSpelling } from '../canonical-json.ts' import type { JsonSpelling } from '../canonical-json.ts'
import type { JsonValue } from '../json-value.ts' import type { JsonValue } from '../json-value.ts'
import { boundedThen, joinedWithin } from './length-bound.ts' import { joinedWithin } from './length-bound.ts'
import { canonicalJsonPieces, spellsCanonicalJson } from '../canonical-json.ts' import { canonicalJsonPieces, spellsCanonicalJson } from '../canonical-json.ts'
import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts'
import { failure, success, type ConvertErrorPath, type Result } from '../result.ts' import { failure, success, type ConvertErrorPath, type Result } from '../result.ts'
import { fencePieces } from './commonmark/backtick-runs.ts' import { fencedCodeBlock } from './commonmark/backtick-runs.ts'
import { infoStringCarries } from './commonmark/grammar.ts' import { infoStringCarries } from './commonmark/grammar.ts'
import { isAdfNode } from '../adf/document.ts' import { isAdfNode } from '../adf/document.ts'
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
@@ -25,8 +25,7 @@ export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: numbe
const named = infoStringCarries(type) const named = infoStringCarries(type)
const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth) const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth)
if (!json.ok) return json if (!json.ok) return json
const text = joinedWithin(fencePieces(`${carryFencePrefix}${named ? type : ''}`, json.value.json)) return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) })
return boundedThen(text, (fence) => success({ headroom: json.value.headroom, text: fence }))
} }
export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> { export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> {
@@ -60,8 +59,9 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) { if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
} }
// Two-space JSON indents every value its depth over, so it stops at the bound unbuilt.
const json = joinedWithin(canonicalJsonPieces(spelled, spelling)) const json = joinedWithin(canonicalJsonPieces(spelled, spelling))
return boundedThen(json, (text) => success({ headroom, json: text })) return json.ok ? success({ headroom, json: json.value }) : json
} }
// `type` is the one the fence's info string names, `undefined` for the inline carry. // `type` is the one the fence's info string names, `undefined` for the inline carry.
+4 -9
View File
@@ -36,17 +36,12 @@ export function pipeCells(line: string): string[] | undefined {
return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined
} }
export function delimiterCells(columns: number): string[] { export function spellPipeDelimiter(columns: number): string {
return Array.from({ length: columns }, () => '---') return spellPipeRow(Array.from({ length: columns }, () => '---'))
} }
export function* pipeRowPieces(cells: readonly string[]): Generator<string> { export function spellPipeRow(cells: readonly string[]): string {
yield '|' return `| ${cells.join(' | ')} |`
for (const cell of cells) {
yield ' '
yield cell
yield ' |'
}
} }
function splitPipeCells(line: string, start: number): string[] { function splitPipeCells(line: string, start: number): string[] {
@@ -1,7 +1,7 @@
import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { AdfDocument, AdfNode } from '../../adf/document.ts'
import { adfDocumentFault, heldTextLength, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { adfDocumentFault, heldTextLength, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts' import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts'
import { boundedThen, joinedWithin, longestMarkdown, tooLong, withinEngineLimits } from '../length-bound.ts' import { longestMarkdown, tooLong, withinEngineLimits } from '../length-bound.ts'
import { centeredImage, externalMedia } from '../external-image.ts' import { centeredImage, externalMedia } from '../external-image.ts'
import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts'
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
@@ -60,7 +60,7 @@ export function reduceToPortable(document: AdfDocument): Result<AdfDocument> {
const fault = adfDocumentFault(document) const fault = adfDocumentFault(document)
if (fault !== undefined) return faulted(fault, []) if (fault !== undefined) return faulted(fault, [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
// The reduction drops form, never text, so this much text cannot fit, and none of it is joined. // Refused on the caller's text, edge whitespace and all: merging it first would build past the bound.
if (heldTextLength(nodeContent(document)) > longestMarkdown) return tooLong() if (heldTextLength(nodeContent(document)) > longestMarkdown) return tooLong()
// Two documents the editor holds equal write the same portable markdown (docs/decisions.md §Equality is deep). // Two documents the editor holds equal write the same portable markdown (docs/decisions.md §Equality is deep).
const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] })
@@ -138,14 +138,14 @@ function portableSequence(blocks: readonly Sourced[], reduction: Reduction): Res
continue continue
} }
// Back on `pending`: a replacement may take the directive form itself. // Back on `pending`: a replacement may take the directive form itself.
const replacement = block === next.block ? itemBlocks(block, next.path) : [{ block, path: next.path }] const replacement = block === next.block ? listItemBlocks(block, next.path) : [{ block, path: next.path }]
for (const node of replacement.reverse()) pending.push(node) for (const node of replacement.reverse()) pending.push(node)
} }
return success(sequence) return success(sequence)
} }
// Offsets are the reduced item's, which are the caller's while the item's blocks reduce one to one. // Offsets are the reduced item's, which are the caller's while the item's blocks reduce one to one.
function itemBlocks(list: AdfNode, path: ConvertErrorPath): Sourced[] { function listItemBlocks(list: AdfNode, path: ConvertErrorPath): Sourced[] {
const blocks: Sourced[] = [] const blocks: Sourced[] = []
for (const [index, item] of nodeContent(list).entries()) { for (const [index, item] of nodeContent(list).entries()) {
for (const [offset, block] of nodeContent(item).entries()) blocks.push({ block, path: [...path, 'content', index, 'content', offset] }) for (const [offset, block] of nodeContent(item).entries()) blocks.push({ block, path: [...path, 'content', index, 'content', offset] })
@@ -231,11 +231,10 @@ function reduceHeading(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
function reduceCodeBlock(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceCodeBlock(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
const leaves = inlineLeaves(nodeContent(node), 'paragraph', reduction.path, reduction.depth) const leaves = inlineLeaves(nodeContent(node), 'paragraph', reduction.path, reduction.depth)
if (!leaves.ok) return leaves if (!leaves.ok) return leaves
const code = leaves.value.map((leaf) => leaf.text ?? '\n').join('')
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims.portable) const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims.portable)
return boundedThen(joinedWithin(leaves.value.map((leaf) => leaf.text ?? '\n')), (code) => { const block: AdfNode = { content: code === '' ? [] : [text(code)], type: 'codeBlock' }
const block: AdfNode = { content: code === '' ? [] : [text(code)], type: 'codeBlock' } return success([slot.kind === 'fence' ? { ...block, attrs: { language: slot.info } } : block])
return success([slot.kind === 'fence' ? { ...block, attrs: { language: slot.info } } : block])
})
} }
function reduceList(node: AdfNode, reduction: Reduction): Result<AdfNode[]> { function reduceList(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
+4 -5
View File
@@ -81,9 +81,7 @@ The line escaper keeps an array element per character twice (`escapings`, `place
50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80 50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a
directive's CommonMark ask. Bound the line as the blocks are (`joinedWithin`): its segments and its directive's CommonMark ask. Found while shipping item 106, 2026-10-11.
escaped output, which today pass V8's longest string only by throwing. Found while shipping item
106, 2026-10-11.
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code. ### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.
@@ -215,8 +213,9 @@ Each level of a nested list or quote re-indents its whole text, so the work is t
text. `losslessMarkdownToAdf` on 3 MB of lazy lines in a list nested 490 levels inside text. `losslessMarkdownToAdf` on 3 MB of lazy lines in a list nested 490 levels inside
`!adf:blockquote` spends about 5 seconds re-spelling it. The same 2 MB split over two items of a `!adf:blockquote` spends about 5 seconds re-spelling it. The same 2 MB split over two items of a
list nested 480 levels took 11 minutes to refuse on Node 24. A deep subtree past the length bound list nested 480 levels took 11 minutes to refuse on Node 24. A deep subtree past the length bound
refuses only at the level that passes it, after every level below re-indented its text. The writers refuses only once every level below has re-indented its text, so no test reaches
do the same on such a document. Found while shipping item 106, 2026-10-11. `losslessMarkdownToAdf`'s length refusal in seconds: add one with this fix. The writers do the same
on such a document. Found while shipping item 106, 2026-10-11.
### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour. ### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.